lxd-common 0.9.8 → 0.9.9

data/README.md

@@ -1,34 +1,223 @@
-# LXD::Common [![Build Status](https://travis-ci.org/NexusSW/lxd-common.svg?branch=master)](https://travis-ci.org/NexusSW/lxd-common) [![Dependency Status](https://gemnasium.com/badges/github.com/NexusSW/lxd-common.svg)](https://gemnasium.com/github.com/NexusSW/lxd-common)
+# lxd-common [![Build Status](https://travis-ci.org/NexusSW/lxd-common.svg?branch=master)](https://travis-ci.org/NexusSW/lxd-common) [![Dependency Status](https://gemnasium.com/badges/github.com/NexusSW/lxd-common.svg)](https://gemnasium.com/github.com/NexusSW/lxd-common)
 
 [![Maintainability](https://api.codeclimate.com/v1/badges/28fae322a45ffa75b771/maintainability)](https://codeclimate.com/github/NexusSW/lxd-common/maintainability)
 [![Test Coverage](https://api.codeclimate.com/v1/badges/28fae322a45ffa75b771/test_coverage)](https://codeclimate.com/github/NexusSW/lxd-common/test_coverage)
 
 ## Installation
 
+> **NOTE:** Versions < 0.10 are considered pre-release while Version 0.9.8 is considered the first stable pre-release.  Make sure when filing issues that you are on at least 0.9.8 and upgrade to 0.10+ as soon as it is possible and available (imminent).
+
 Add this line to your application's Gemfile:
 
 ```ruby
-gem 'lxd-common'
+gem 'lxd-common', '~> 0.9', '>= 0.9.8'
 ```
 
-And then execute:
+Next execute:
 
-    > bundle
+    > bundle install
 
-Or install it yourself as:
+Or install it yourself with:
 
     > gem install lxd-common
 
+## Background
+
+This gem intends to completely obfuscate the complexities of communicating with an LXD host.  LXD exposes two methods of interaction: a legacy LXC compatible CLI interface, and a new REST API.  Both behave a bit differently, but remain equally valid for today, and will remain available for the forseeable future.  This gem exposes an interface that will remain consistent, no matter the underlying communication mechanism.
+
+Do you want to:
+
+* Control a local LXD host via the CLI?
+* Control a local/remote LXD host via their REST API?
+* Control a LXD host and not care where it's located or want to deal with the API variances?
+* Oh!  and do you want to control a LXD host that's buried under 2-3+ layers of nesting?  _(some scenarios still under dev at LXD & upstream)_
+
+You're covered.  You provide the credentials and we'll provide the agnostic API.
+
 ## Usage
 
-TODO:
+This gem is split up into 2 functional areas:  Driver and Transport.  Constructing a driver object does require some environment specific information.  But once you have the driver object constructed, all else is generic.
+
+Drivers allow you to communicate with the LXD host directly and to perform all command and control operations such as creating a container, as well as starting/stopping/deleting and for setting and querying container configuration.
+
+Transports allow you to make use of a container.  They can execute commands inside of a container, and transfer files in and out of a container.  Transports can be obtained by executing `transport = driver.transport_for 'containername'` on any driver.
+
+### Drivers
+
+There are 2 different drivers at your disposal:
+
+* NexusSW::LXD::Driver::Rest
+* NexusSW::LXD::Driver::CLI
+
+And once they're constructed, they both respond to the same API calls in the same way, and so you no longer need to deal with API variances.  The next sections tell you how to construct these drivers, but the final 'Driver methods' section, and everything afterwards, is completely generic.
+
+#### REST Driver
+
+ex: `driver = NexusSW::LXD::Driver::Rest.new 'https://someserver:8443', verify_ssl: false`
+
+The first parameter of course being the REST endpoint of the LXD host.  SSL verfication is enabled by default and can be disabled with the options shortcut of `verify_ssl: false`.  The other option available is `ssl:` and has the following subkeys:
+
+key | default | description
+---|:---:|---
+verify | true | overrides `verify_ssl`.  verify_ssl is just a shortcut to this option for syntactic sugar when no other ssl options are needed
+client_cert | ~/.config/lxc/client.crt | client certificate file used to authorize your connection to the LXD host
+client_key | ~/.config/lxc/client.key | key file associated with the above certificate
+
+ex2: `driver = NexusSW::LXD::Driver::Rest.new 'https://someserver:8443', ssl: { verify: false, client_cert: '/someother.crt', client_key: '/someother.key' }`
+
+#### CLI Driver
+
+The CLI driver is a different beast.  All it knows is 'how' to construct `lxc` commands, just as you would type them into your shell if you were managing LXD manually.  It doesn't know 'where' to execute those commands by default, and so you have to give it a transport.  You'll see why in the next section.
+
+ex: `driver = NexusSW::LXD::Driver::CLI.new(NexusSW::LXD::Transport::Local.new)`
+
+There are no options at present.  You only need to pass in a transport telling the CLI driver 'where' to run its commands.  Above I've demonstrated using the Local Transport which executes the `lxc` commands on the local machine.  Using the CLI driver with the Local transport is usually synonymous to using the Rest driver pointed at localhost (barring any environmental modifications).
+
+##### CLI Driver _magic_: nested containers
+
+And so I briefly mentioned that you can call `transport_for` on any driver to gain a transport instance.  And I alluded to the CLI driver working with any transport.  By extension this would mean that you can use the CLI driver to execute `lxc` commands almost anywhere.  The CLI driver only cares that the transport it is given implements the NexusSW::LXD::Transport interface.
+
+So what if we did this?
+
+```ruby
+1> outerdriver = NexusSW::LXD::Driver::Rest.new 'https://someserver:8443'
+2> resttransport = outerdriver.transport_for 'somecontainer'
+
+3> middledriver = NexusSW::LXD::Driver::CLI.new resttransport
+4> middletransport = middledriver.transport_for 'nestedcontainer'
+
+5> innerdriver = NexusSW::LXD::Driver::CLI.new middletransport
+6> innertransport = innerdriver.transport_for 'some-very-nested-container'
+
+7> contents = innertransport.read_file '/tmp/something_interesting'
+8> puts innertransport.execute('dmesg').error!.stdout
+```
+
+That would totally work!!!  The final innertransport object is a transport for 'some-very-nested-container' that is itself running within a container named 'nestedcontainer', that is also itself running inside of 'somecontainer' that is remotely located on 'someserver'.
+
+On line 3 you can see the CLI driver accepting a transport produced by the REST API.  And on line 5 you see the CLI driver consuming a transport produced by a different instance of itself.  It all works given the caveat that you've set up nested LXD hosts on both 'somecontainer' and 'nestedcontainer' (an exercise for the reader).
+
+#### Driver methods
+
+To add one more point of emphasis:  drivers talk to an LXD host while transports talk to individual containers.
+
+**NOTE:** Due to the behavior of some of the underlying long-running tasks inherent in LXD (It can take a minute to create, start, or stop a container), these driver methods are implemented with a more convergent philosophy rather than being classically imperitive.  This means that, for example, a call to delete_container will NOT fail if that container does not exist (one would otherwise expect a 404, e.g.).  Similarly, start_container will not fail if the container is already started.  The driver just ensures that the container state is what you just asked for, and if it is, then great!  And if it is not, then it will send the necessary commands.  If this is not your desired behavior, then ample other status check methods are available for you to handle these errors in your desired way.
+
+Here are some of the things that you can do while talking to an LXD host:  (driver methods)
+
+method name | parameters | options | notes
+---|---|---|---
+create_container | container_name | _see next section_ | _see next section_
+update_container | container_name | _see next section_ | _see next section_
+start_container | container_name
+stop_container | container_name | force: false<br />timeout: 0<br />retry_interval:&nbsp;_unset_<br />retry_count:&nbsp;_unset_ | the default behavior is no force, no timeout, and no retries.  This is analogous to sending a SIGPWR to the init system within the container and waiting for a graceful shutdown.  Specifying `force: true` instead sends a SIGKILL to the container
+delete_container | container_name | | This will force stop the container, if it is currently running, prior to deleting.  If you need a graceful shutdown, send the stop_container command on your own.
+container_status | container_name | | returns a simple container status string such as 'running' or 'stopped'.  There are many other intermediate states, but these 2 are most likely the ones to be interested in.
+container | container_name | | returns the current container configuration
+container_state | container_name | | returns various runtime info regarding the running state of the container (e.g. IP Address, Memory utilization, etc...)
+wait_for | container_name,<br />what,<br />timeout = 60 | | 'what' currently supports `:ip`, and `:cloud_init`.  After container start, calling this method will wait for an IP to be assigned, or for cloud-init to complete, respectively
+transport_for | container_name | | returns a transport used to communicate directly with a container.  Note that the container itself does not need to be directly routable.  If you have access to the host, then this transport will just plain work.
+
+##### Driver.create_container and Driver.update_container
+
+_(Create only)_ Source image options:  (one of alias, fingerprint, properties MUST be specified)
+
+option | default | notes
+---|---|---
+:server | none | server:port from which to obtain a base image<br />e.g. `https://images.linuxcontainers.org`,<br />`https://cloud-images.ubuntu.com/releases`,<br />`https://cloud-images.ubuntu.com/daily`
+:protocol | 'lxd' | Use 'lxd' or 'simplestreams' to communicate with the source server.  Use 'simplestreams' for the above public servers
+:alias | none | server dependant.  e.g. `ubuntu:16.04` or `ubuntu/xenial`.  Refer to the image catalog of each server for valid values.
+:fingerprint | none | partial matches accepted.  Each individual image gets a fingerprint.
+:properties | none | A Hash with image search parameters.  Refer to LXD documentation for valid key value pairs.  (Use of :properties is discouraged due to its non-deterministic nature)
+:autostart | true | Start the container immediately upon creation.  This is *_not_* the same as config["boot.autostart"]
+
+The above is how you locate a base image for your new container.  The next options allow you to specify last minute configuration details.
+
+option | default | notes
+---|---|---
+:profiles | ['default'] | An ordered array of profiles to apply to your new container.  If you specify ANY profiles, and still want the 'default' profile applied, then you must explicitly include 'default' in your array
+:config | none | A Hash of configuration options to apply to this specific container.  Overrides configs specified within profiles.  Refer to LXD documentation for valid key value pairs.  https://github.com/lxc/lxd/blob/master/doc/containers.md
+:devices | | Refer to the above link for supported device types and options
+
+In the case of `update_container`, we will merely apply any options that are sent.  If you wish to unset a configuration key, call update_container with an opposing value for that key.
+
+###### Examples
+
+minimum:
+
+```ruby
+driver.create_container 'some-container', alias: 'ubuntu:18.04', \
+    server: 'https://cloud-images.ubuntu.com/daily', protocol: 'simplestreams'
+```
+
+more options:
+
+```ruby
+driver.create_container 'dev-cattle-01', profiles: ['default', 'cattle'], \
+    config: { 'security.nesting': true, 'security.privileged': true }, \
+    server: 'https://images.linuxcontainers.org', protocol: 'simplestreams', \
+    alias: 'ubuntu/bionic'
+```
+
+```ruby
+driver.update_container 'dev-cattle-01', \
+    config: { 'security.nesting': false, 'security.privileged': false, \
+              'security.idmap.isolated': true }, \
+```
+
+### Transports
+
+And having navigated all of the above, you can now get your transport instance with<br />`transport = driver.transport_for 'some-container'`.  Here's what you can do with it:
+
+#### Transport methods
+
+method name | parameters | options | notes
+---|---|---|---
+user | _user |  _options = {})
+read_file | _path)
+write_file | _path, _content |  _options = {})
+download_file | _path, _local_path)
+download_folder | _path, _local_path |  _options = {})
+upload_file | _local_path, _path |  _options = {})
+upload_folder | _local_path, _path |  _options = {})
+execute | command |  _see next section_ | _see next section_
+
+##### Transport.execute
+
+This one merits a section of its own.
+
+## Contributing: Development and Testing
+
+Bug reports and pull requests are welcome on GitHub at <https://github.com/NexusSW/lxd-common>.  DCO signoffs are required on your commits.
+
+After checking out this repo, and installing development dependencies via `bundle install`, you can run some quick smoke tests that exercise most code paths via `rake mock`.  This just exercises the gem without talking to an actual LXD host.
+
+The full integration test suite `rake spec` requires:
+
+* a local LXD host (> 2.0) with the REST API enabled on port 8443
+* your user account to be in the lxd group
+* a client certificate and key located in ~/.config/lxc/ named client.crt and client.key
+* and that certificate to be trusted by the LXD host via `lxc config trust add ~/.config/lxc/client.crt`
+* _recommended_:  a decent internet connection.  Most of the test time is spent in downloading images and spinning up containers.  So the quicker your filesystem and internet, the quicker the tests will run.  As a benchmark, Travis runs the integration tests in (presently) ~6-7 minutes, which includes installation and setup time.  If feasible, set your LXD up to use BTRFS to speed up the nesting tests.
+
+Refer to [spec/provision_recipe.rb](https://github.com/NexusSW/lxd-common/blob/master/spec/provision_recipe.rb) (Chef) if you need hints on how to accomplish this setup.
+
+### TDD Cycle _(suggested)_
+
+**NOTE:** In contrast to normal expectations, these tests are not isolated and are stateful in between test cases.  Pay attention to tests that expect a container to exist, or for a file to exist within a container.  (Apologies: integration test times would be exponentially higher if I didn't make this concession)
 
-## Development
+1. run `rake mock` to verify your dependencies.  Expect it to pass.  Fix whatever is causing failures.
+2. **Write your failing test case(s)**
+3. run `rake mock`.  If there are compilation errors or exceptions generated by the spec/support/mock_transport.rb that cause pre-existing tests to fail, fix them so that past tests pass, and that your new tests compile (if feasible), but fail.
+4. **Create your new functionality**
+5. run `rake mock` again.  Code the mock_transport (and/or potentially the mock_hk) to return the results that you expect the LXD host to return.
+6. Repeat the above until `rake mock` passes
+7. run `rake spec` to see if your changes work for real.  Or if you can't set your workstation up for integration tests as described above, submit a PR and let Travis test it.
 
-After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+`rake mock` and `rake spec` must both pass before Travis will pass.
 
-To install this gem onto your local machine, run `bundle exec rake install`.
+#### Development Goals
 
-## Contributing
+When developing your new functionality, keep in mind that this gem intends to obfuscate the differences in the behavior between LXD's CLI and REST interfaces.  The test suite is designed to expose such differences, and it may become necessary for you to create completely seperate implementations in order to make them behave identically.
 
-Bug reports and pull requests are welcome on GitHub at <https://github.com/NexusSW/lxd-common>.
+Whether to expose the behavior of the CLI, or that of the REST interface, or something in between, will be up for debate on a case by case basis.  But they do need to have the same behavior.  And that should be in line with the behavior of other pre-existing functions, should they fall within the same category or otherwise interact.

data/Rakefile

@@ -1,10 +1,10 @@
 
-require 'bundler/gem_tasks'
-require 'rspec/core/rake_task'
+require "bundler/gem_tasks"
+require "rspec/core/rake_task"
 
 RSpec::Core::RakeTask.new(:spec)
 RSpec::Core::RakeTask.new(:mock) do |task|
-  task.pattern = 'spec/**{,/*/**}/*_mock.rb'
+  task.pattern = "spec/**{,/*/**}/*_mock.rb"
 end
 
 task default: :spec

data/Vagrantfile

@@ -1,11 +1,11 @@
 # -*- mode: ruby -*-
 # vi: set ft=ruby :
 
-Vagrant.configure('2') do |config|
-  config.vm.box = 'ubuntu/xenial64'
+Vagrant.configure("2") do |config|
+  config.vm.box = "ubuntu/xenial64"
   # config.vm.box = 'ubuntu/trusty64'
 
-  config.vm.provision 'chef_apply' do |chef|
-    chef.recipe = File.read 'spec/provision_recipe.rb'
+  config.vm.provision "chef_apply" do |chef|
+    chef.recipe = File.read "spec/provision_recipe.rb"
   end
 end

data/bin/console

@@ -1,10 +1,10 @@
 #!/usr/bin/env ruby
 
-require 'bundler/setup'
-require 'nexussw/lxd/driver/cli'
-require 'nexussw/lxd/driver/rest'
-require 'nexussw/lxd/transport/cli'
-require 'nexussw/lxd/transport/rest'
+require "bundler/setup"
+require "nexussw/lxd/driver/cli"
+require "nexussw/lxd/driver/rest"
+require "nexussw/lxd/transport/cli"
+require "nexussw/lxd/transport/rest"
 
 # You can add fixtures and/or initialization code here to make experimenting
 # with your gem easier. You can also use a different console, if you like.
@@ -13,5 +13,5 @@ require 'nexussw/lxd/transport/rest'
 # require 'pry'
 # Pry.start
 
-require 'irb'
+require "irb"
 IRB.start

data/lib/nexussw/lxd.rb

@@ -1,4 +1,4 @@
-require 'timeout'
+require "timeout"
 
 module NexusSW
   module LXD
@@ -32,9 +32,14 @@ module NexusSW
     def self.symbolize_keys(hash)
       {}.tap do |retval|
         hash.each do |k, v|
-          v.map! do |a|
-            a.is_a?(Hash) ? symbolize_keys(a) : a
-          end if v.is_a?(Array)
+          if %w{config expanded_config}.include? k
+            retval[k.to_sym] = v
+            next
+          elsif v.is_a?(Array)
+            v.map! do |a|
+              a.is_a?(Hash) ? symbolize_keys(a) : a
+            end
+          end
           retval[k.to_sym] = v.is_a?(Hash) ? symbolize_keys(v) : v
         end
       end

data/lib/nexussw/lxd/driver.rb

@@ -1,24 +1,24 @@
-require 'nexussw/lxd'
+require "nexussw/lxd"
 
 module NexusSW
   module LXD
     class Driver
       STATUS_CODES = {
-        100	=> 'created',
-        101	=> 'started',
-        102	=> 'stopped',
-        103	=> 'running',
-        104	=> 'cancelling',
-        105	=> 'pending',
-        106	=> 'starting',
-        107	=> 'stopping',
-        108	=> 'aborting',
-        109	=> 'freezing',
-        110	=> 'frozen',
-        111	=> 'thawed',
-        200	=> 'success',
-        400	=> 'failure',
-        401	=> 'cancelled',
+        100  => "created",
+        101  => "started",
+        102  => "stopped",
+        103  => "running",
+        104  => "cancelling",
+        105  => "pending",
+        106  => "starting",
+        107  => "stopping",
+        108  => "aborting",
+        109  => "freezing",
+        110  => "frozen",
+        111  => "thawed",
+        200  => "success",
+        400  => "failure",
+        401  => "cancelled",
       }.freeze
 
       def create_container(_container_name, _container_options)
@@ -37,6 +37,10 @@ module NexusSW
         raise "#{self.class}#delete_container not implemented"
       end
 
+      def update_container(_container_name, _container_options)
+        raise "#{self.class}#update_container not implemented"
+      end
+
       def container_status(_container_id)
         raise "#{self.class}#container_status not implemented"
       end
@@ -49,13 +53,25 @@ module NexusSW
         raise "#{self.class}#container_state not implemented"
       end
 
-      def wait_for(_what)
+      def wait_for(_container_name, _what, _timeout = 60)
         raise "#{self.class}#wait_for not implemented"
       end
 
       def transport_for(_container_name)
         raise "#{self.class}#transport_for not implemented"
       end
+
+      def self.convert_bools(oldhash)
+        {}.tap do |retval|
+          oldhash.each do |k, v|
+            retval[k] = case v
+                        when "true" then true
+                        when "false" then false
+                        else v.is_a?(Hash) ? convert_bools(v) : v
+                        end
+          end
+        end
+      end
     end
   end
 end

data/lib/nexussw/lxd/driver/cli.rb

@@ -1,5 +1,5 @@
-require 'nexussw/lxd/driver'
-require 'nexussw/lxd/driver/mixins/cli'
+require "nexussw/lxd/driver"
+require "nexussw/lxd/driver/mixins/cli"
 
 module NexusSW
   module LXD

data/lib/nexussw/lxd/driver/mixins/cli.rb

@@ -1,8 +1,8 @@
-require 'nexussw/lxd/driver/mixins/helpers/wait'
-require 'nexussw/lxd/transport/cli'
-require 'tempfile'
-require 'yaml'
-require 'json'
+require "nexussw/lxd/driver/mixins/helpers/wait"
+require "nexussw/lxd/transport/cli"
+require "tempfile"
+require "yaml"
+require "json"
 
 module NexusSW
   module LXD
@@ -17,12 +17,13 @@ module NexusSW
           attr_reader :inner_transport, :driver_options
 
           def transport_for(container_name)
-            Transport::CLI.new inner_transport, container_name, info: YAML.load(inner_transport.execute('lxc info').error!.stdout)
+            Transport::CLI.new inner_transport, container_name, info: YAML.load(inner_transport.execute("lxc info").error!.stdout)
           end
 
           def create_container(container_name, container_options = {})
+            autostart = (container_options.delete(:autostart) != false)
             if container_exists? container_name
-              start_container container_name # Start for Parity with the below logic (`lxc launch` auto starts)
+              start_container(container_name) if autostart
               return container_name
             end
             cline = "lxc launch #{image_alias(container_options)} #{container_name}"
@@ -30,36 +31,101 @@ module NexusSW
             profiles.each { |p| cline += " -p #{p}" }
             configs = container_options[:config] || {}
             configs.each { |k, v| cline += " -c #{k}=#{v}" }
+            if !autostart || container_options[:devices] # append to the cline to avoid potential lag between create & stop
+              cline += " && lxc stop -f #{container_name}"
+              cline = ["sh", "-c", cline] # There's no guarantee that inner_transport is running a shell for the && operator
+            end
             inner_transport.execute(cline).error!
-            wait_for_status container_name, 'running'
+            if container_options[:devices]
+              update_container(container_name, devices: container_options[:devices])
+              start_container(container_name) if autostart
+            else
+              wait_for_status container_name, "running" if autostart
+            end
             container_name
           end
 
+          def update_container(container_name, container_options)
+            raise NexusSW::LXD::RestAPI::Error::NotFound, "Container (#{container_name}) does not exist" unless container_exists? container_name
+            configs = container_options[:config]
+            devices = container_options[:devices]
+            profiles = container_options[:profiles]
+            existing = container(container_name)
+
+            if configs
+              configs.each do |k, v|
+                if v.nil?
+                  next unless existing[:config][k]
+                  inner_transport.execute("lxc config unset #{container_name} #{k}").error!
+                else
+                  next if existing[:config][k] == v
+                  inner_transport.execute("lxc config set #{container_name} #{k} #{v}").error!
+                end
+              end
+            end
+
+            if devices
+              devices.each do |name, device|
+                cmd = "add"
+                if device.nil?
+                  next unless existing[:devices].include? name
+                  inner_transport.execute("lxc config device remove #{container_name} #{name}").error!
+                  next
+                elsif existing[:devices].include?(name)
+                  cmd = "set"
+                  if existing[:devices][name][:type] != device[:type]
+                    inner_transport.execute("lxc config device remove #{container_name} #{name}").error!
+                    cmd = "add"
+                  end
+                end
+                if cmd == "add"
+                  cline = "lxc config device add #{container_name} #{name} #{device[:type]}"
+                  device.each do |k, v|
+                    cline << " #{k}=#{v}"
+                  end
+                  inner_transport.execute(cline).error!
+                else
+                  device.each do |k, v|
+                    next if k == :type
+                    next if v == existing[:devices][name][k]
+                    inner_transport.execute("lxc config device set #{container_name} #{name} #{k} #{v}").error!
+                  end
+                end
+              end
+            end
+
+            if profiles
+              inner_transport.execute("lxc profile assign #{container_name} #{profiles.join(",")}").error! unless profiles == existing[:profiles]
+            end
+
+            container container_name
+          end
+
           def start_container(container_id)
-            return if container_status(container_id) == 'running'
+            return if container_status(container_id) == "running"
             inner_transport.execute("lxc start #{container_id}").error!
-            wait_for_status container_id, 'running'
+            wait_for_status container_id, "running"
           end
 
           def stop_container(container_id, options = {})
             options ||= {} # default behavior: no timeout or retries.  These functions are up to the consumer's context and not really 'sane' defaults
-            return if container_status(container_id) == 'stopped'
+            return if container_status(container_id) == "stopped"
             return inner_transport.execute("lxc stop #{container_id} --force", capture: false).error! if options[:force]
             LXD.with_timeout_and_retries(options) do
-              return if container_status(container_id) == 'stopped'
+              return if container_status(container_id) == "stopped"
               timeout = " --timeout=#{options[:retry_interval]}" if options[:retry_interval]
               retval = inner_transport.execute("lxc stop #{container_id}#{timeout || ''}", capture: false)
               begin
                 retval.error!
               rescue => e
-                return if container_status(container_id) == 'stopped'
+                return if container_status(container_id) == "stopped"
                 # can't distinguish between timeout, or other error.
                 # but if the status call is not popping a 404, and we're not stopped, then a retry is worth it
                 raise Timeout::Retry.new(e) if timeout # rubocop:disable Style/RaiseArgs
                 raise
               end
             end
-            wait_for_status container_id, 'stopped'
+            wait_for_status container_id, "stopped"
           end
 
           def delete_container(container_id)
@@ -71,20 +137,6 @@ module NexusSW
             STATUS_CODES[container(container_id)[:status_code].to_i]
           end
 
-          def convert_keys(oldhash)
-            return oldhash unless oldhash.is_a?(Hash) || oldhash.is_a?(Array)
-            retval = {}
-            if oldhash.is_a? Array
-              retval = []
-              oldhash.each { |v| retval << convert_keys(v) }
-            else
-              oldhash.each do |k, v|
-                retval[k.to_sym] = convert_keys(v)
-              end
-            end
-            retval
-          end
-
           # YAML is not supported until somewhere in the feature branch
           #   the YAML return has :state and :container at the root level
           # the JSON return has no :container (:container is root)
@@ -94,7 +146,7 @@ module NexusSW
             res = inner_transport.execute("lxc list #{container_id} --format=json")
             res.error!
             JSON.parse(res.stdout).each do |c|
-              return convert_keys(c['state']) if c['name'] == container_id
+              return LXD.symbolize_keys(c["state"]) if c["name"] == container_id
             end
             nil
           end
@@ -103,14 +155,14 @@ module NexusSW
             res = inner_transport.execute("lxc list #{container_id} --format=json")
             res.error!
             JSON.parse(res.stdout).each do |c|
-              return convert_keys(c.reject { |k, _| k == 'state' }) if c['name'] == container_id
+              return Driver.convert_bools(LXD.symbolize_keys(c.reject { |k, _| k == "state" })) if c["name"] == container_id
             end
             nil
           end
 
           def container_exists?(container_id)
             return true if container_status(container_id)
-            return false
+            false
           rescue
             false
           end
@@ -130,32 +182,32 @@ module NexusSW
 
           private
 
-          def remote_for!(url, protocol = 'lxd')
-            raise 'Protocol is required' unless protocol # protect me from accidentally slipping in a nil
+          def remote_for!(url, protocol = "lxd")
+            raise "Protocol is required" unless protocol # protect me from accidentally slipping in a nil
             # normalize the url and 'require' protocol to protect against a scenario:
             #   1) user only specifies https://someimageserver.org without specifying the protocol
             #   2) the rest of this function would blindly add that without saying the protocol
             #   3) 'lxc remote add' would add that remote, but defaults to the lxd protocol and appends ':8443' to the saved url
             #   4) the next time this function is called we would not match that same entry due to the ':8443'
             #   5) ultimately resulting in us adding a new remote EVERY time this function is called
-            port = url.split(':', 3)[2]
-            url += ':8443' unless port || protocol != 'lxd'
+            port = url.split(":", 3)[2]
+            url += ":8443" unless port || protocol != "lxd"
             remotes = begin
-                        YAML.load(inner_transport.read_file('~/.config/lxc/config.yml')) || {}
+                        YAML.load(inner_transport.read_file("~/.config/lxc/config.yml")) || {}
                       rescue
                         {}
                       end
             # make sure these default entries are available to us even if config.yml isn't created yet
             # and i've seen instances where these defaults don't live in the config.yml
-            remotes = { 'remotes' => {
-              'images' => { 'addr' => 'https://images.linuxcontainers.org' },
-              'ubuntu' => { 'addr' => 'https://cloud-images.ubuntu.com/releases' },
-              'ubuntu-daily' => { 'addr' => 'https://cloud-images.ubuntu.com/daily' },
+            remotes = { "remotes" => {
+              "images" => { "addr" => "https://images.linuxcontainers.org" },
+              "ubuntu" => { "addr" => "https://cloud-images.ubuntu.com/releases" },
+              "ubuntu-daily" => { "addr" => "https://cloud-images.ubuntu.com/daily" },
             } }.merge remotes
             max = 0
-            remotes['remotes'].each do |remote, data|
-              return remote.to_s if data['addr'] == url
-              num = remote.to_s.split('-', 2)[1] if remote.to_s.start_with? 'images-'
+            remotes["remotes"].each do |remote, data|
+              return remote.to_s if data["addr"] == url
+              num = remote.to_s.split("-", 2)[1] if remote.to_s.start_with? "images-"
               max = num.to_i if num && num.to_i > max
             end
             remote = "images-#{max + 1}"
@@ -163,22 +215,22 @@ module NexusSW
             remote
           end
 
-          def image(properties, remote = '')
+          def image(properties, remote = "")
             return nil unless properties && properties.any?
             cline = "lxc image list #{remote} --format=json"
             properties.each { |k, v| cline += " #{k}=#{v}" }
             res = inner_transport.execute cline
             res.error!
             res = JSON.parse(res.stdout)
-            return res[0]['fingerprint'] if res.any?
+            return res[0]["fingerprint"] if res.any?
           end
 
           def image_alias(container_options)
-            remote = container_options[:server] ? remote_for!(container_options[:server], container_options[:protocol] || 'lxd') + ':' : ''
+            remote = container_options[:server] ? remote_for!(container_options[:server], container_options[:protocol] || "lxd") + ":" : ""
             name = container_options[:alias]
             name ||= container_options[:fingerprint]
             name ||= image(container_options[:properties], remote)
-            raise 'No image parameters.  One of alias, fingerprint, or properties must be specified (The CLI interface does not support empty containers)' unless name
+            raise "No image parameters.  One of alias, fingerprint, or properties must be specified (The CLI interface does not support empty containers)" unless name
             "#{remote}#{name}"
           end
         end