onceover 3.8.0 → 3.9.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 760bc0e986513a337cfe07a8808d5aca8cae1675
-  data.tar.gz: 7ed6e0c9d5278aef45ddc95f8e136793ff5a1236
+  metadata.gz: a37e753c95878c5de3f89a71d73e97b835e3ee54
+  data.tar.gz: 5a71d30c65ef86e645110719ef752ad6306c99bd
 SHA512:
-  metadata.gz: 88ae6311f20abb1a997cf68e547de5a7d0b62bac25ac07152f035fcb39061e7f31a2fbee1b998f0f1f972429a1765c89af1508a1fa890854b2b49c53b9fc2548
-  data.tar.gz: 3064ffc747c54d87125feefafb5cb42b031278ad9dd94738c5f0ea750039e97904dc2d7c3df2dee357f338270106badbccf499d5b5faf25c75e1c67a9c892539
+  metadata.gz: 61c0d522ccd4864f4bb224a6a60560f8c3852eb2037472d727b73b768132665bf137277248e59bac088ddedcb3eac421af30598e8830db619df9473ee3e69c5d
+  data.tar.gz: 56b657a2e74bb3344cb88785024afd4e977d74a04dbd1690c33e497157a2fe6c993b4b720e8c01c1347116e8e8d7e20856157bc9c9c9481241428c09bd4af697

data/.travis.yml

@@ -9,12 +9,6 @@ script: bundle exec rake full_tests
 bundler_args: --path vendor/bundle
 
 rvm:
-  - 2.1.6
-  - 2.1.7
-  - 2.1.8
-  - 2.1.9
-  - 2.2.7
-  - 2.3.4
-  - 2.4.0
-  - 2.4.1
-  - 2.4.2
+  - 2.3.8
+  - 2.4.4
+  - 2.5.3

data/README.md

@@ -2,7 +2,7 @@
 
 *The gateway drug to automated infrastructure testing with Puppet*
 
-Onceover is a tool to automatically run basic tests on an entire Puppet controlrepo. It includes automatic parsing the `Puppetfile`, `environment.conf` and others in order to run both basic compilation tests and also full acceptance tests!
+Onceover is a tool to automatically run basic tests on an entire Puppet controlrepo. It includes automatic parsing of the `Puppetfile`, `environment.conf` and others in order to run stop silly mistakes ever reaching your Puppet Master!
 
 ## Table of Contents
 
@@ -12,11 +12,9 @@ Onceover is a tool to automatically run basic tests on an entire Puppet controlr
   - [Config files](#config-files)
     - [onceover.yaml](#onceoveryaml)
     - [factsets](#factsets)
-    - [nodesets](#nodesets)
     - [Hiera Data](#hiera-data)
   - [Spec testing](#spec-testing)
     - [Adding your own spec tests](#adding-your-own-spec-tests)
-  - [Acceptance testing](#acceptance-testing)
   - [Using Workarounds](#using-workarounds)
   - [Extra tooling](#extra-tooling)
     - [Plugins](#plugins)
@@ -66,7 +64,7 @@ Run your spec tests!
 
 ## Overview
 
-This gem provides a toolset for testing Puppet Controlrepos (Repos used with r10k). The main purpose of this project is to provide a set of tools to help smooth out the process of setting up and running both spec and acceptance tests for a controlrepo. Due to the fact that controlrepos are fairly standardised in nature it seemed ridiculous that you would need to set up the same testing framework that we would normally use within a module for a controlrepo. This is because at this level we are normally just running very basic tests that cover a lot of code. It would also mean that we would need to essentially duplicated our `Puppetfile` into a `.fixtures.yml` file, along with a few other things.
+This gem provides a toolset for testing Puppet Controlrepos (Repos used with r10k). The main purpose of this project is to provide a set of tools to help smooth out the process of setting up and running rspec-puppet tests for a controlrepo. Due to the fact that controlrepos are fairly standardised in nature it seemed ridiculous that you would need to set up the same testing framework that we would normally use within a module for a controlrepo. This is because at this level we are normally just running very basic tests that cover a lot of code. It would also mean that we would need to essentially duplicated our `Puppetfile` into a `.fixtures.yml` file, along with a few other things.
 
 This toolset requires some config before it can be used so definitely read that section before getting started.
 
@@ -82,17 +80,15 @@ This project uses one main config file to determine what classes we should be te
 
 If we are doing spec testing we need sets of facts to compile the puppet code against, these are stored in [factsets](#factsets). (A few are provided out of the box for you)
 
-If we are doing acceptance testing then we need information about how to spin up VMs to do the testing on, these are configured in [nodesets](#nodesets). (Once again these are auto-generated with `onceover init`)
-
 ### onceover.yaml
 
 `spec/onceover.yaml` _(override with environment variable: `ONCEOVER_YAML`)_
 
-Hopefully this config file will be fairly self explanatory once you see it, but basically this is the place where we define what classes we want to test and the [factsets](#factsets)/[nodesets](#nodesets) that we want to test them against. The config file must contain the following sections:
+Hopefully this config file will be fairly self explanatory once you see it, but basically this is the place where we define what classes we want to test and the [factsets](#factsets) that we want to test them against. The config file must contain the following sections:
 
 **classes:** A list (array) of classes that we want to test, usually this would be your roles, possibly profiles if you want. (If you don't know what roles and profiles are please [READ THIS](http://garylarizza.com/blog/2014/02/17/puppet-workflow-part-2/)). To make life easier you can also specify one or many **regular expressions** in this section. A good one to start with would be `/^role::/`. Regular expressions are just strings that start and end with a forward slash.
 
-**nodes:** The nodes that we want to test against. The nodes that we list here map directly to either a [factset](#factsets) or a [nodeset](#nodesets) depending on weather we are running spec or acceptance tests respectively.
+**nodes:** The nodes that we want to test against. The nodes that we list here map directly to a [factset](#factsets).
 
 **node_groups:** The `node_groups` section is just for saving us some typing. Here we can set up groups of nodes which we can then refer to in our test matrix. We can create groups by simply specifying an array of servers to be in the group, or we can use the subtractive *include/exclude* syntax. The names used for the actual `class_groups` and `node_groups` must be unique.
 
@@ -103,7 +99,7 @@ Hopefully this config file will be fairly self explanatory once you see it, but
 ```yaml
   - {nodes_to_test}: # The name of a node or node group
       classes: '{classes_to_test}' # the name of a class or
-      tests: '{all_tests|acceptance|spec}' # One of the three
+      tests: '{all_tests|acceptance|spec}' # acceptance deprecated/broken, set to spec
       {valid_option}: {value} # Check the doco for available options
 ```
 
@@ -190,10 +186,10 @@ test_matrix:
       classes: 'non_windows_roles'
   - ubuntu_servers:
       classes: 'all_classes'
-      tests: 'all_tests'
+      tests: 'spec'
   - centos_severs:
       classes: 'roles::frontend_webserver'
-      tests: 'acceptance'
+      tests: 'spec'
       runs_before_idempotency: 2
       tags:
         - 'frontend'
@@ -224,14 +220,6 @@ It's important to note that in order to reference a group using the *include/exc
 
 #### Optional test parameters
 
-**check_idempotency** *Default: true*
-
-Weather or not to check that puppet will be idempotent (Acceptance testing only)
-
-**runs_before_idempotency** *Default: 1*
-
-The number of runs to try before checking that it is idempotent. Required for some things that require restarts of the server or restarts of puppet. (Acceptance testing only)
-
 **tags** *Default: nil*
 
 One or many tags that tests in this group should be tagged with. This allows you to run only certain tests using the `--tags` command line parameter. **NOTE:** Custom spec tests will always be run as they are not subject to tags
@@ -258,7 +246,7 @@ Would map to a node named `server2008r2` in `onceover.yaml`
 
 #### Trusted Facts
 
-You can add trusted facts to the nodesets by creating a new section called trusted:
+You can add trusted facts to the factsets by creating a new section called trusted:
 
 ```
 {
@@ -276,69 +264,6 @@ You can add trusted facts to the nodesets by creating a new section called trust
 
 Notice that the `extensions` part is implied. The first fact in that example translates to `$trusted['extensions']['pp_role']` in Puppet code.
 
-### nodesets
-
-`spec/acceptance/nodesets/onceover-nodes.yml`
-
-Nodesets are used when running acceptance tests. They instruct the onceover gem how to spin up virtual machines to run the code on. Actually, that's a lie... What's really happening with nodesets is that we are using [Beaker](https://github.com/puppetlabs/beaker) to spin up the machines and then a combination of Beaker and RSpec to test them. But you don't need to worry about that too much. Due to the fact that we are using beaker to do the heavy lifting here the nodeset files follow the same format they would for normal Beaker tests, which at the time of writing supports the following hypervisors:
-
-  - [VMWare Fusion](https://github.com/puppetlabs/beaker/blob/master/docs/VMWare-Fusion-Support.md)
-  - [Amazon EC2](https://github.com/puppetlabs/beaker/blob/master/docs/EC2-Support.md)
-  - [vSphere](https://github.com/puppetlabs/beaker/blob/master/docs/vSphere-Support.md)
-  - [Vagrant](https://github.com/puppetlabs/beaker/blob/master/docs/Vagrant-Support.md)
-  - [Google Compute Engine](https://github.com/puppetlabs/beaker/blob/master/docs/Google-Compute-Engine-Support.md)
-  - [Docker](https://github.com/puppetlabs/beaker/blob/master/docs/Docker-Support.md)
-  - [Openstack](https://github.com/puppetlabs/beaker/blob/master/docs/Openstack-Support.md)
-  - [Solaris](https://github.com/puppetlabs/beaker/blob/master/docs/Solaris-Support.md)
-
-Before we configure a hypervisor to spin up a node however, we have to make sure that it can clone from a machine which is ready. The onceover gem **requires it's VMs to have puppet pre-installed.** It doesn't matter what version of puppet, as long as it is on the PATH and the `type` setting is configured correctly i.e.
-
-```yaml
-type: AIO # For machines that have the all-in-one agent installed (>=4.0 or >=2015.2)
-# OR
-type: pe # For puppet enterprise agents <2015.2
-# OR
-type: foss # For open source puppet <4.0
-```
-
-Here is an example of a nodeset file that you can use yourselves. It uses freely available Vagrant boxes from Puppet and Virtualbox as the Vagrant provider. (`onceover init` will generate most of this for you)
-
-```yaml
-HOSTS:
-  centos6a:
-    roles:
-      - agent
-    type: aio
-    platform: el-6-64
-    box: puppetlabs/centos-6.6-64-puppet
-    box_url: https://atlas.hashicorp.com/puppetlabs/boxes/centos-6.6-64-puppet
-    hypervisor: vagrant_virtualbox
-  centos7b:
-    roles:
-      - agent
-    type: aio
-    platform: el-7-64
-    box: puppetlabs/centos-7.0-64-puppet
-    box_url: https://atlas.hashicorp.com/puppetlabs/boxes/centos-7.0-64-puppet
-    hypervisor: vagrant_virtualbox
-  ubuntu1204:
-    roles:
-      - agent
-    type: aio
-    platform: ubuntu-12.04-32
-    box: puppetlabs/ubuntu-12.04-32-puppet
-    box_url: https://atlas.hashicorp.com/puppetlabs/boxes/ubuntu-12.04-32-puppet
-    hypervisor: vagrant_virtualbox
-  debian78:
-    roles:
-      - agent
-    type: aio
-    platform: debian-7.8-64
-    box: puppetlabs/debian-7.8-64-puppet
-    box_url: https://atlas.hashicorp.com/puppetlabs/boxes/debian-7.8-64-puppet
-    hypervisor: vagrant_virtualbox
-```
-
 ### Hiera Data
 
 If you have hiera data inside your controlrepo (or somewhere else) Onceover can be configured to use it. It is however worth noting the the `hiera.yaml` file that you currently use may not be applicable for testing right away. For example; if you are using `hiera-eyaml` I recommend creating a `hiera.yaml` purely for testing that simply uses the `yaml` backend, meaning that you don't need to provide the private keys to the testing machines.
@@ -405,26 +330,6 @@ If you want to see Puppet's output, you can set the `SHOW_PUPPET_OUTPUT` environ
 
 `SHOW_PUPPET_OUTPUT=true onceover run spec`
 
-## Acceptance testing
-
-Acceptance testing works in much the same way as spec testing except that it requires a nodeset file along with `onceover.yaml`
-
-To run the tests:
-
-`onceover run acceptance`
-
-This will do the following things:
-
-  1. Create a temporary directory under `.onceover`
-  2. Clone all repos in the Puppetfile into the temporary directory
-  3. Generate tests that use RSpec and Beaker
-  4. Run the tests, each test consists of:
-    - Spin up the VM
-    - Copy over the code
-    - Run puppet and catch any errors
-    - Run puppet again to catch anything that might not be idempotent
-    - Destroy the VM
-
 ## Using workarounds
 
 There may be situations where you cannot test everything that is in your puppet code, some common reasons for this include:
@@ -480,29 +385,9 @@ or
   }
 ```
 
-However this is going to pose an issue when we get to acceptance testing. Due to the fact that acceptance tests actually run the code, not just tries to compile a catalog, it will not be able to find the 'pe-pupetserver' service and will fail. One way to get around this is to use some of the optional parameters to the service resource e.g.
-
-```puppet
-# We are not going to actually have this service anywhere on our servers but
-# our code needs to refresh it. This is to trick puppet into doing nothing
-service { 'pe-puppetserver':
-  ensure     => 'running',
-  enable     => false,
-  hasrestart => false, # Force Puppet to use start and stop to restart
-  start      => 'echo "Start"', # This will always exit 0
-  stop       => 'echo "Stop"', # This will also always exit 0
-  hasstatus  => false, # Force puppet to use our command for status
-  status     => 'echo "Status"', # This will always exit 0 and therefore Puppet will think the service is running
-  provider   => 'base',
-}
-```
-
-Here we are specifying custom commands to run for starting, stopping and checking the status of a service. We know what the exit codes of these commands are going to be so we know what puppet will think the service is doing because we have [read the documentation](https://docs.puppetlabs.com/references/latest/type.html#service-attributes). If there are things other than services you need to check then I would recommend checking the documentation to see if you can mock things like we have here. Alternatively you might need to create specific VM images that are pre-prepared.
 
 [Resource collectors](https://docs.puppetlabs.com/puppet/latest/reference/lang_resources_advanced.html#amending-attributes-with-a-collector) are likely to come in handy here too. They allow you to override values of resources that match given criteria. This way we can override things for the sake of testing without having to change the code.
 
-**NOTE:** If you need to run some pre_conditions during acceptance tests but not spec tests or vice versa you can check the status of the `$controlrepo_accpetance` variable. It will be `true` when run as an acceptance test and `undef` otherwise. If you want to limit pre_conditions to only certain nodes just use conditional logic based on facts like you normally would.
-
 **NOTE:** If you want to access the class or factset that onceover is running against just use the `$onceover_class` and `$onceover_node` variables respectively.
 
 ## Extra Tooling

data/factsets/windows-10-64.json

@@ -0,0 +1,103 @@
+{
+  "name": "vagrant-hb9g3rd.lan.asio",
+  "values": {
+    "puppetversion": "6.0.2",
+    "puppet_inventory_metadata": {
+      "packages": {
+        "collection_enabled": false,
+        "last_collection_time": "0.0s"
+      }
+    },
+    "package_provider": "windows",
+    "pe_concat_basedir": "C:/ProgramData/PuppetLabs/puppet/cache/pe_concat",
+    "is_pe": false,
+    "platform_symlink_writable": false,
+    "puppet_files_dir_present": false,
+    "puppet_vardir": "C:/ProgramData/PuppetLabs/puppet/cache",
+    "puppet_environmentpath": "C:/ProgramData/PuppetLabs/code/environments",
+    "puppet_server": "pe-puppet.localdomain",
+    "service_provider": "windows",
+    "staging_http_get": "curl",
+    "common_appdata": "C:\\ProgramData",
+    "architecture": "x64",
+    "kernel": "windows",
+    "virtual": "vmware",
+    "is_virtual": true,
+    "hardwaremodel": "x64",
+    "operatingsystem": "windows",
+    "os": {
+      "name": "windows",
+      "family": "windows",
+      "release": {
+        "major": "10",
+        "full": "10"
+      }
+    },
+    "facterversion": "2.5.1",
+    "fqdn": "VAGRANT-HB9G3RD",
+    "hostname": "VAGRANT-HB9G3RD",
+    "id": "vagrant-hb9g3rd\\vagrant",
+    "interfaces": "Ethernet0_2",
+    "ipaddress_ethernet0_2": "192.168.43.174",
+    "ipaddress6_ethernet0_2": "fd72:b3ab:b8dd:0:f563:fac2:928c:6aa",
+    "macaddress_ethernet0_2": "00:0C:29:A3:53:54",
+    "netmask_ethernet0_2": "255.255.255.0",
+    "mtu_ethernet0_2": 0,
+    "ipaddress": "192.168.43.174",
+    "ipaddress6": "fd72:b3ab:b8dd:0:f563:fac2:928c:6aa",
+    "kernelmajversion": "10.0",
+    "kernelrelease": "10.0.17134",
+    "kernelversion": "10.0.17134",
+    "macaddress": "00:0C:29:A3:53:54",
+    "manufacturer": "Phoenix Technologies LTD",
+    "serialnumber": "VMware-56 4d 36 2e 2e a4 81 f9-0d 05 ae d0 e5 a3 53 54",
+    "productname": "VMware Virtual Platform",
+    "memorysize": "4.00 GB",
+    "memoryfree": "2.30 GB",
+    "memorysize_mb": "4095.49",
+    "memoryfree_mb": "2360.00",
+    "netmask": "255.255.255.0",
+    "network_ethernet0_2": "192.168.43.0",
+    "operatingsystemmajrelease": "10",
+    "operatingsystemrelease": "10",
+    "osfamily": "windows",
+    "path": "C:\\tools\\ruby24\\bin;C:\\Windows\\system32;C:\\Windows;C:\\Windows\\System32\\Wbem;C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\;C:\\Windows\\System32\\OpenSSH\\;C:\\ProgramData\\chocolatey\\bin;C:\\Program Files (x86)\\vim\\vim80;C:\\Program Files\\Git\\cmd;C:\\Program Files\\Puppet Labs\\Puppet\\bin;C:\\Users\\vagrant\\AppData\\Local\\Microsoft\\WindowsApps;",
+    "physicalprocessorcount": 4,
+    "processors": {
+      "models": [
+        "Intel(R) Core(TM) i7-6820HQ CPU @ 2.70GHz",
+        "Intel(R) Core(TM) i7-6820HQ CPU @ 2.70GHz",
+        "Intel(R) Core(TM) i7-6820HQ CPU @ 2.70GHz",
+        "Intel(R) Core(TM) i7-6820HQ CPU @ 2.70GHz"
+      ],
+      "count": 4,
+      "physicalcount": 4
+    },
+    "processor0": "Intel(R) Core(TM) i7-6820HQ CPU @ 2.70GHz",
+    "processor1": "Intel(R) Core(TM) i7-6820HQ CPU @ 2.70GHz",
+    "processor2": "Intel(R) Core(TM) i7-6820HQ CPU @ 2.70GHz",
+    "processor3": "Intel(R) Core(TM) i7-6820HQ CPU @ 2.70GHz",
+    "processorcount": 4,
+    "ps": "tasklist.exe",
+    "rubyplatform": "x64-mingw32",
+    "rubysitedir": "C:/tools/ruby24/lib/ruby/site_ruby/2.4.0",
+    "rubyversion": "2.4.3",
+    "system32": "C:\\Windows\\system32",
+    "system_uptime": {
+      "seconds": 1113359,
+      "hours": 309,
+      "days": 12,
+      "uptime": "12 days"
+    },
+    "timezone": "GMT Standard Time",
+    "uptime": "12 days",
+    "uptime_days": 12,
+    "uptime_hours": 309,
+    "uptime_seconds": 1113359,
+    "clientcert": "vagrant-hb9g3rd.lan.asio",
+    "clientversion": "6.0.2",
+    "clientnoop": false
+  },
+  "timestamp": "2018-12-10T21:43:07.468533000+00:00",
+  "expiration": "2018-12-10T22:13:07.468533000+00:00"
+}

data/features/run.feature

@@ -12,6 +12,11 @@ Feature: Run rspec and acceptance test suites
     When I run onceover command "run spec"
     Then I should not see any errors
 
+  Scenario: Using regexes to define tests
+    Given initialized control repo "caching"
+    When I run onceover command "run spec"
+    Then I should see message pattern "apache::params"
+
   Scenario: Run spec tests with misspelled module in Puppetfile
     Given initialized control repo "basic"
     And in Puppetfile is misspelled module's name

data/features/step_definitions/common.rb

@@ -51,9 +51,13 @@ Then(/^I should see error with message pattern "([^"]*)"$/) do |err_msg_regexp|
   expect(@cmd.output.match err_msg_regexp).to_not be nil
 end
 
-Then(/^I should see message pattern "([^"]*)"$/) do |err_msg_regexp|
-  expect(@cmd.success?).to be true
-  puts @cmd.output unless @cmd.output =~ Regexp.new(err_msg_regexp)
-  expect(@cmd.output).to match(err_msg_regexp)
-  puts @cmd.output.match(err_msg_regexp).to_s
+Then(/^I should see message pattern "([^"]*)"$/) do |msg_regexp|
+  output_surround = 30
+  match = Regexp.new(msg_regexp).match(@cmd.output)
+  expect(@cmd.output).to match(msg_regexp)
+  if match
+    puts match.pre_match[-output_surround..-1] + match.to_s + match.post_match[0..output_surround]
+  else
+    puts @cmd.output
+  end
 end

data/lib/onceover/cli/run.rb

@@ -4,6 +4,7 @@ require 'onceover/cli'
 require 'onceover/runner'
 require 'onceover/testconfig'
 require 'onceover/logger'
+require 'onceover/deploy'
 
 class Onceover
   class CLI
@@ -43,6 +44,7 @@ This includes deploying using r10k and running all custom tests.
 
             run do |opts, args, cmd|
               repo = Onceover::Controlrepo.new(opts)
+              Onceover::Deploy.new.deploy_local(repo, opts)
               runner = Onceover::Runner.new(repo,Onceover::TestConfig.new(repo.onceover_yaml, opts), :spec)
               runner.prepare!
               runner.run_spec!

data/lib/onceover/controlrepo.rb

@@ -156,11 +156,17 @@ class Onceover
     end
 
     def classes
+      logger.debug('scanning for classes specified in onceover.yaml')
+
       # Get all of the possible places for puppet code and look for classes
       code_dirs = self.config['modulepath']
       # Remove interpolated references
       code_dirs.delete_if { |dir| dir[0] == '$'}
 
+      # Include all r10k-downloaded modules to support vendored and/or separate
+      # role and profile classes
+      code_dirs << "#{@tempdir}/#{@environmentpath}/production/modules"
+
       # Make sure that the paths are relative to the controlrepo root
       code_dirs.map! do |dir|
         File.expand_path(dir, @root)

data/lib/onceover/deploy.rb

@@ -0,0 +1,152 @@
+# handle local deployments (run r10k in .onceover dir)
+class Onceover
+  class Deploy
+    def deploy_local(repo = Onceover::Controlrepo.new, opts = {})
+      require 'onceover/controlrepo'
+      require 'pathname'
+
+      logger.debug 'Deploying locally (R10K)...'
+
+      skip_r10k = opts[:skip_r10k] || false
+      force     = opts[:force] || false
+
+      if repo.tempdir == nil
+        repo.tempdir = Dir.mktmpdir('r10k')
+      else
+        logger.debug "Creating #{repo.tempdir}"
+        FileUtils.mkdir_p(repo.tempdir)
+      end
+
+      # We need to do the copy to a tempdir then move the tempdir to the
+      # destination, just in case we get a recursive copy
+      # TODO: Improve this to save I/O
+
+      # We might need to exclude some files
+      #
+      # if we are using bundler to install gems below the controlrepo
+      # we don't want two copies so exclude those
+      #
+      # If there are more situations like this we can add them to this array as
+      # full paths
+      excluded_dirs = []
+      excluded_dirs << Pathname.new("#{repo.root}/.onceover")
+      excluded_dirs << Pathname.new(ENV['GEM_HOME']) if ENV['GEM_HOME']
+
+      #
+      # A Local modules directory likely means that the user installed r10k folders into their local control repo
+      # This conflicts with the step where onceover installs r10k after copying the control repo to the temporary
+      # .onceover directory.  The following skips copying the modules folder, to not later cause an error.
+      #
+      if File.directory?("#{repo.root}/modules")
+        logger.warn "Found modules directory in your controlrepo, skipping the copy of this directory.  If you installed modules locally using r10k, this warning is normal, if you have created modules in a local modules directory, onceover does not support testing these files, please rename this directory to conform with Puppet best practices, as this folder will conflict with Puppet's native installation of modules."
+      end
+      excluded_dirs << Pathname.new("#{repo.root}/modules")
+
+      controlrepo_files = get_children_recursive(Pathname.new(repo.root))
+
+      # Exclude the files that should be skipped
+      controlrepo_files.delete_if do |path|
+        parents = [path]
+        path.ascend do |parent|
+          parents << parent
+        end
+        parents.any? { |x| excluded_dirs.include?(x) }
+      end
+
+      folders_to_copy = controlrepo_files.select { |x| x.directory? }
+      files_to_copy   = controlrepo_files.select { |x| x.file? }
+
+      logger.debug "Creating temp dir as a staging directory for copying the controlrepo to #{repo.tempdir}"
+      temp_controlrepo = Dir.mktmpdir('controlrepo')
+
+      logger.debug "Creating directories under #{temp_controlrepo}"
+      FileUtils.mkdir_p(folders_to_copy.map { |folder| "#{temp_controlrepo}/#{(folder.relative_path_from(Pathname(repo.root))).to_s}"})
+
+      logger.debug "Copying files to #{temp_controlrepo}"
+      files_to_copy.each do |file|
+        FileUtils.cp(file,"#{temp_controlrepo}/#{(file.relative_path_from(Pathname(repo.root))).to_s}")
+      end
+
+      logger.debug "Writing manifest of copied controlrepo files"
+      require 'json'
+      # Create a manifest of all files that were in the original repo
+      manifest = controlrepo_files.map do |file|
+        # Make sure the paths are relative so they remain relevant when used later
+        file.relative_path_from(Pathname(repo.root)).to_s
+      end
+      # Write all but the first as this is the root and we don't care about that
+      File.write("#{temp_controlrepo}/.onceover_manifest.json",manifest[1..-1].to_json)
+
+      # When using puppetfile vs deploy with r10k, we want to respect the :control_branch
+      # located in the Puppetfile. To accomplish that, we use git and find the current
+      # branch name, then replace strings within the staged puppetfile, prior to copying.
+
+      logger.debug "Checking current working branch"
+      git_branch = `git rev-parse --abbrev-ref HEAD`.chomp
+
+      logger.debug "found #{git_branch} as current working branch"
+      puppetfile_contents = File.read("#{temp_controlrepo}/Puppetfile")
+
+      logger.debug "replacing :control_branch mentions in the Puppetfile with #{git_branch}"
+      new_puppetfile_contents = puppetfile_contents.gsub(/:control_branch/, "'#{git_branch}'")
+      File.write("#{temp_controlrepo}/Puppetfile", new_puppetfile_contents)
+
+      # Remove all files written by the laste onceover run, but not the ones
+      # added by r10k, because that's what we are trying to cache but we don't
+      # know what they are
+      old_manifest_path = "#{repo.tempdir}/#{repo.environmentpath}/production/.onceover_manifest.json"
+      if File.exist? old_manifest_path
+        logger.debug "Found manifest from previous run, parsing..."
+        old_manifest = JSON.parse(File.read(old_manifest_path))
+        logger.debug "Removing #{old_manifest.count} files"
+        old_manifest.reverse.each do |file|
+          FileUtils.rm_f(File.join("#{repo.tempdir}/#{repo.environmentpath}/production/",file))
+        end
+      end
+      FileUtils.mkdir_p("#{repo.tempdir}/#{repo.environmentpath}")
+
+      logger.debug "Copying #{temp_controlrepo} to #{repo.tempdir}/#{repo.environmentpath}/production"
+      FileUtils.cp_r("#{temp_controlrepo}/.", "#{repo.tempdir}/#{repo.environmentpath}/production")
+      FileUtils.rm_rf(temp_controlrepo)
+
+      # Pull the trigger! If it's not already been pulled
+      if repo.tempdir and not skip_r10k
+        if File.directory?(repo.tempdir)
+          # TODO: Change this to call out to r10k directly to do this
+          # Probably something like:
+          # R10K::Settings.global_settings.evaluate(with_overrides)
+          # R10K::Action::Deploy::Environment
+          prod_dir = "#{repo.tempdir}/#{repo.environmentpath}/production"
+          Dir.chdir(prod_dir) do
+            install_cmd = []
+            install_cmd << "r10k puppetfile install --verbose --color --puppetfile #{repo.puppetfile}"
+            install_cmd << "--force" if force
+            install_cmd = install_cmd.join(' ')
+            logger.debug "Running #{install_cmd} from #{prod_dir}"
+            system(install_cmd)
+            raise 'r10k could not install all required modules' unless $?.success?
+          end
+        else
+          raise "#{repo.tempdir} is not a directory"
+        end
+      end
+
+      # Return repo.tempdir for use
+      repo.tempdir
+    end
+
+    private
+
+    def get_children_recursive(pathname)
+      results = []
+      results << pathname
+      pathname.each_child do |child|
+        results << child
+        if child.directory?
+          results << get_children_recursive(child)
+        end
+      end
+      results.flatten
+    end
+  end
+end

data/lib/onceover/runner.rb

@@ -11,9 +11,6 @@ class Onceover
     end
 
     def prepare!
-      # Deploy the control repo
-      @config.deploy_local(@repo, {:skip_r10k => @config.skip_r10k})
-
       # Remove the entire spec directory to make sure we have
       # all the latest tests
       FileUtils.rm_rf("#{@repo.tempdir}/spec")

data/lib/onceover/testconfig.rb

@@ -167,137 +167,6 @@ class Onceover
       puppetcode.join("\n")
     end
 
-    def deploy_local(repo = Onceover::Controlrepo.new, opts = {})
-      require 'onceover/controlrepo'
-      require 'pathname'
-
-      skip_r10k = opts[:skip_r10k] || false
-
-      if repo.tempdir == nil
-        repo.tempdir = Dir.mktmpdir('r10k')
-      else
-        logger.debug "Creating #{repo.tempdir}"
-        FileUtils.mkdir_p(repo.tempdir)
-      end
-
-      # We need to do the copy to a tempdir then move the tempdir to the
-      # destination, just in case we get a recursive copy
-      # TODO: Improve this to save I/O
-
-      # We might need to exclude some files
-      #
-      # if we are using bundler to install gems below the controlrepo
-      # we don't want two copies so exclude those
-      #
-      # If there are more situations like this we can add them to this array as
-      # full paths
-      excluded_dirs = []
-      excluded_dirs << Pathname.new("#{repo.root}/.onceover")
-      excluded_dirs << Pathname.new(ENV['GEM_HOME']) if ENV['GEM_HOME']
-
-      #
-      # A Local modules directory likely means that the user installed r10k folders into their local control repo
-      # This conflicts with the step where onceover installs r10k after copying the control repo to the temporary
-      # .onceover directory.  The following skips copying the modules folder, to not later cause an error.
-      #
-      if File.directory?("#{repo.root}/modules")
-        logger.warn "Found modules directory in your controlrepo, skipping the copy of this directory.  If you installed modules locally using r10k, this warning is normal, if you have created modules in a local modules directory, onceover does not support testing these files, please rename this directory to conform with Puppet best practices, as this folder will conflict with Puppet's native installation of modules."
-      end
-      excluded_dirs << Pathname.new("#{repo.root}/modules")
-
-      controlrepo_files = get_children_recursive(Pathname.new(repo.root))
-
-      # Exclude the files that should be skipped
-      controlrepo_files.delete_if do |path|
-        parents = [path]
-        path.ascend do |parent|
-          parents << parent
-        end
-        parents.any? { |x| excluded_dirs.include?(x) }
-      end
-
-      folders_to_copy = controlrepo_files.select { |x| x.directory? }
-      files_to_copy   = controlrepo_files.select { |x| x.file? }
-
-      logger.debug "Creating temp dir as a staging directory for copying the controlrepo to #{repo.tempdir}"
-      temp_controlrepo = Dir.mktmpdir('controlrepo')
-
-      logger.debug "Creating directories under #{temp_controlrepo}"
-      FileUtils.mkdir_p(folders_to_copy.map { |folder| "#{temp_controlrepo}/#{(folder.relative_path_from(Pathname(repo.root))).to_s}"})
-
-      logger.debug "Copying files to #{temp_controlrepo}"
-      files_to_copy.each do |file|
-        FileUtils.cp(file,"#{temp_controlrepo}/#{(file.relative_path_from(Pathname(repo.root))).to_s}")
-      end
-
-      logger.debug "Writing manifest of copied controlrepo files"
-      require 'json'
-      # Create a manifest of all files that were in the original repo
-      manifest = controlrepo_files.map do |file|
-        # Make sure the paths are relative so they remain relevant when used later
-        file.relative_path_from(Pathname(repo.root)).to_s
-      end
-      # Write all but the first as this is the root and we don't care about that
-      File.write("#{temp_controlrepo}/.onceover_manifest.json",manifest[1..-1].to_json)
-
-      # When using puppetfile vs deploy with r10k, we want to respect the :control_branch
-      # located in the Puppetfile. To accomplish that, we use git and find the current
-      # branch name, then replace strings within the staged puppetfile, prior to copying.
-
-      logger.debug "Checking current working branch"
-      git_branch = `git rev-parse --abbrev-ref HEAD`.chomp
-
-      logger.debug "found #{git_branch} as current working branch"
-      puppetfile_contents = File.read("#{temp_controlrepo}/Puppetfile")
-
-      logger.debug "replacing :control_branch mentions in the Puppetfile with #{git_branch}"
-      new_puppetfile_contents = puppetfile_contents.gsub(/:control_branch/, "'#{git_branch}'")
-      File.write("#{temp_controlrepo}/Puppetfile", new_puppetfile_contents)
-
-      # Remove all files written by the laste onceover run, but not the ones
-      # added by r10k, because that's what we are trying to cache but we don't
-      # know what they are
-      old_manifest_path = "#{repo.tempdir}/#{repo.environmentpath}/production/.onceover_manifest.json"
-      if File.exist? old_manifest_path
-        logger.debug "Found manifest from previous run, parsing..."
-        old_manifest = JSON.parse(File.read(old_manifest_path))
-        logger.debug "Removing #{old_manifest.count} files"
-        old_manifest.reverse.each do |file|
-          FileUtils.rm_f(File.join("#{repo.tempdir}/#{repo.environmentpath}/production/",file))
-        end
-      end
-      FileUtils.mkdir_p("#{repo.tempdir}/#{repo.environmentpath}")
-
-      logger.debug "Copying #{temp_controlrepo} to #{repo.tempdir}/#{repo.environmentpath}/production"
-      FileUtils.cp_r("#{temp_controlrepo}/.", "#{repo.tempdir}/#{repo.environmentpath}/production")
-      FileUtils.rm_rf(temp_controlrepo)
-
-      # Pull the trigger! If it's not already been pulled
-      if repo.tempdir and not skip_r10k
-        if File.directory?(repo.tempdir)
-          # TODO: Change this to call out to r10k directly to do this
-          # Probably something like:
-          # R10K::Settings.global_settings.evaluate(with_overrides)
-          # R10K::Action::Deploy::Environment
-          prod_dir = "#{repo.tempdir}/#{repo.environmentpath}/production"
-          Dir.chdir(prod_dir) do
-            install_cmd = []
-            install_cmd << "r10k puppetfile install --verbose --color --puppetfile #{repo.puppetfile}"
-            install_cmd << "--force" if @force
-            install_cmd = install_cmd.join(' ')
-            logger.debug "Running #{install_cmd} from #{prod_dir}"
-            system(install_cmd)
-            raise 'r10k could not install all required modules' unless $?.success?
-          end
-        else
-          raise "#{repo.tempdir} is not a directory"
-        end
-      end
-
-      # Return repo.tempdir for use
-      repo.tempdir
-    end
-
     def write_spec_test(location, test)
       # Use an ERB template to write a spec test
       File.write("#{location}/#{test.to_s}_spec.rb",
@@ -394,19 +263,5 @@ class Onceover
       end
       tests
     end
-
-    private
-
-    def get_children_recursive(pathname)
-      results = []
-      results << pathname
-      pathname.each_child do |child|
-        results << child
-        if child.directory?
-          results << get_children_recursive(child)
-        end
-      end
-      results.flatten
-    end
   end
 end

data/onceover.gemspec

@@ -4,7 +4,7 @@ $LOAD_PATH.unshift File.expand_path("../lib", __FILE__)
 
 Gem::Specification.new do |s|
   s.name        = "onceover"
-  s.version     = "3.8.0"
+  s.version     = "3.9.0"
   s.authors     = ["Dylan Ratcliffe"]
   s.email       = ["dylan.ratcliffe@puppet.com"]
   s.homepage    = "https://github.com/dylanratcliffe/onceover"

data/spec/fixtures/controlrepos/caching/spec/onceover.yaml

@@ -3,9 +3,10 @@ classes:
   - role::database_server
   - role::webserver
   - role::example
+  - /apache::params/
 
 # Nodes to tests classes on, this refers to a 'factset' or 'nodeset'
-# depending on weather you are running 'spec' or 'acceptance' tests
+# depending on whether you are running 'spec' or 'acceptance' tests
 nodes:
   - AIX-7.1-powerpc
   - SLES-12.1-64
@@ -50,4 +51,4 @@ test_matrix:
       tests: 'spec'
   - non_windows_nodes: 
       classes: 'all_classes'
-      tests: 'acceptance'
+      tests: 'acceptance'

data/spec/fixtures/controlrepos/function_mocking/spec/onceover.yaml

@@ -3,7 +3,7 @@ classes:
   - role::test_functions
 
 # Nodes to tests classes on, this refers to a 'factset' or 'nodeset'
-# depending on weather you are running 'spec' or 'acceptance' tests
+# depending on whether you are running 'spec' or 'acceptance' tests
 nodes:
   - CentOS-7.0-64
 

data/templates/controlrepo.yaml.erb

@@ -5,7 +5,7 @@ classes:
 <% end -%>
 
 # Nodes to tests classes on, this refers to a 'factset' or 'nodeset'
-# depending on weather you are running 'spec' or 'acceptance' tests
+# depending on whether you are running 'spec' or 'acceptance' tests
 nodes:
 <% repo.facts_files.each do |file| -%>
   - <%= File.basename(file,'.json') %>

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: onceover
 version: !ruby/object:Gem::Version
-  version: 3.8.0
+  version: 3.9.0
 platform: ruby
 authors:
 - Dylan Ratcliffe
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2018-09-05 00:00:00.000000000 Z
+date: 2018-12-24 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: rake
@@ -341,6 +341,7 @@ files:
 - factsets/Windows_Server-2012r2-64.json
 - factsets/solaris-10_u9-sparc-64.json
 - factsets/solaris-11.2-sparc-64.json
+- factsets/windows-10-64.json
 - features/cache.feature
 - features/help.feature
 - features/init.feature
@@ -364,6 +365,7 @@ files:
 - lib/onceover/cli/show.rb
 - lib/onceover/cli/update.rb
 - lib/onceover/controlrepo.rb
+- lib/onceover/deploy.rb
 - lib/onceover/group.rb
 - lib/onceover/logger.rb
 - lib/onceover/node.rb