controlrepo 1.1.0 → 2.0.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: 1a2459c4880e5865531609c8b94a3d7a1c492133
4
- data.tar.gz: 32094fc438825fb19b1d40cfde19b56837e24566
3
+ metadata.gz: d27924d8a371e6fcc60c20a749013cb8e9eb1240
4
+ data.tar.gz: 9425a314523f65fe3e5321958f5019b0e7452dab
5
5
  SHA512:
6
- metadata.gz: ccc385b3ce617f8f28146fe78cf10ef4c39a21e317ad572cd358bcf1668a818375e33c1c94451cca9bfdc0351b994819183970b187b95f4d4e8a8550db6d0c77
7
- data.tar.gz: 8fc06a94e542a89c3cd5a19907fdedb47c249fc7faf00e4ac967974ddc8b36c0dbeb4419a567bfc82970ad58bdc34493838cbf8f89f8def9a702fd1d9fd4616c
6
+ metadata.gz: c428c645689a91b7e8bc939e6d69d993ddee30880173249b518fb853f94fc24def1b1994596204aca98d95e7139b3118a9322cef1d6798f2d7bd86c897c81e2e
7
+ data.tar.gz: 4ccca700d874ad4c42364c75270811d6b04b2050feb5096337a8b8444661acbf467be5ad929ea6fe52b2daea3b764a63b0140a06dab4137d5538f25d0e207f89
data/Gemfile ADDED
@@ -0,0 +1,4 @@
1
+ source 'https://rubygems.org'
2
+
3
+
4
+ gem 'rubygems-tasks'
data/Gemfile.lock ADDED
@@ -0,0 +1,13 @@
1
+ GEM
2
+ remote: https://rubygems.org/
3
+ specs:
4
+ rubygems-tasks (0.2.4)
5
+
6
+ PLATFORMS
7
+ ruby
8
+
9
+ DEPENDENCIES
10
+ rubygems-tasks
11
+
12
+ BUNDLED WITH
13
+ 1.10.5
data/README.md CHANGED
@@ -1,22 +1,65 @@
1
1
  # Controlrepo Toolset
2
2
 
3
+ ## Table of Contents
4
+
5
+ - [Overview](#overview)
6
+ - [Installation](#installation)
7
+ - [Config files](#config-files)
8
+ - [controlrepo.yaml](#controlrepoyaml)
9
+ - [factsets](#factsets)
10
+ - [nodesets](#nodesets)
11
+ - [Hiera Data](#hiera-data)
12
+ - [R10k.yaml](#r10kyaml)
13
+ - [Spec testing](#spec-testing)
14
+ - [Acceptance testing](#acceptance-testing)
15
+ - [Using Workarounds](#using-workarounds)
16
+ - [Extra tooling](#extra-tooling)
17
+ - [Accessing fact sets in a traditional RSpec test](#accessing-fact-sets-in-a-traditional-rspec-test)
18
+ - [Accessing Roles in a traditional RSpec test](#accessing-roles-in-a-traditional-rspec-test)
19
+ - [Filtering](#filtering)
20
+ - [Using hiera data (In manual tests)](#using-hiera-data-in-manual-tests)
21
+ - [Extra Configuration](#extra-configuration)
22
+ - [Rake tasks](#rake-tasks)
23
+ - [generate_fixtures](#generate_fixtures)
24
+ - [generate_nodesets](#generate_nodesets)
25
+ - [hiera_setup](#hiera_setup)
26
+
27
+
3
28
  ## Overview
4
29
 
5
- This gem gives you a bunch of tools to use for testing and generally managing Puppet controlrepos. The main purpose of this project is to provide a set of tools to help smooth out the process of setting up and running both spec and acceptance tests for a controlrepo. Due to the fact that controlrepos are fairly standardise in nature it seemed ridiculous that you would need to set up the same testing framework that we would normally use within a module for a controlrepo. This is because at this level we are normally just running very basic tests that test a lot of code. It would also mean that we would need to essentially duplicated our `Puppetfile` into a `.fixtures.yml` file, along with a few other things.
30
+ This gem provides a toolset for testing Puppet Controlrepos (Repos used with r10k). The main purpose of this project is to provide a set of tools to help smooth out the process of setting up and running both spec and acceptance tests for a controlrepo. Due to the fact that controlrepos are fairly standardised in nature it seemed ridiculous that you would need to set up the same testing framework that we would normally use within a module for a controlrepo. This is because at this level we are normally just running very basic tests that test a lot of code. It would also mean that we would need to essentially duplicated our `Puppetfile` into a `.fixtures.yml` file, along with a few other things.
31
+
32
+ This toolset requires some config before it can be used so definitely read that section before getting started.
33
+
34
+ ## Installation
35
+
36
+ `gem install controlrepo`
37
+
38
+ This gem can just be installed using `gem install` however I would recommend using [Bundler](http://bundler.io/) to manage your gems.
6
39
 
7
- This toolset has two distinct ways of being used, easy mode and hard mode.
40
+ ## Config Files
8
41
 
9
- ## Easy Mode
42
+ This project uses one main config file to determine what classes we should be testing and how, this is [controlrepo.yaml](#controlrepo.yaml). The `controlrepo.yaml` config file provides information about what classes to test when, however it needs more information than that:
10
43
 
11
- The object of *easy mode* is to allow people to run simple `it { should compile }` acceptance tests without needing to set up any of the extra stuff required by the rspec-puppet testing framework.
44
+ If we are doing spec testing we need sets of facts to compile the puppet code against, these are stored in [factsets](#factsets).
12
45
 
13
- `rake tasks go here once they are done`
46
+ If we are doing acceptance testing then we need information about how to spin up VMs to do the testing on, these are configured in [nodesets](#nodesets).
14
47
 
15
- A stretch goal is to also include acceptance testing, allowing people to spin up boxes for each role they have and test them before merging code into development environments or production. At the moment we can't do this, hold tight.
48
+ ### controlrepo.yaml
16
49
 
17
- ### Easy mode config
50
+ `spec/controlrepo.yaml`
18
51
 
19
- The whole idea of easy mode is that we should just be able to write down which classes we want to test on which machines and this tool should be able to do the rest. This all has to be set up somewhere, this is **spec/controlrepo.yaml** which looks something like this:
52
+ Hopefully this config file will be fairly self explanatory once you see it, but basically this is the place where we define what classes we want to test and the [factsets](#factsets)/[nodesets](#nodesets) that we want to test them against. The config file must contain the following sections:
53
+
54
+ **classes:** A list (array) of classes that we want to test, usually this would be your roles, possibly profiles if you want. (F you don't know what roles and profiles are please [READ THIS](http://garylarizza.com/blog/2014/02/17/puppet-workflow-part-2/))
55
+
56
+ **nodes:** The nodes that we want to test against. The nodes that we list here map directly to either a [factset](#factsets) or a [nodeset](#nodesets) depending on weather we are running spec or acceptance tests respectively.
57
+
58
+ **groups:** The groups section is just for saving us some typing. Here we can set up groups of classes *or* nodes (not a combination of the two) which we can then refer to in our test matrix. There are also two default groups automatically created which you can use, **all_classes** and **all_nodes**. If you can't guess what they are go have a coffee and come back.
59
+
60
+ **test_matrix:** This where the action happens! This is the section where we set up which classes are going to be tested against which nodes. It should be a hash with the node/s as the key and class/es as the values. This is where we would use groups if we had them (you can see us using the `windows_roles` and `centos_servers` groups in the below example). We can also use groups to subtractively build up lists of classes by using `include` and `exclude` as per the example below.
61
+
62
+ Don't be afraid if your test matrix produces duplicate tests, the controlrepo gem de-duplicates the test matrix before flattening it and generating the tests, as a result there should never be any duplicate tests. This also means that tests might not come out in the order that you expect D:
20
63
 
21
64
  ```yaml
22
65
  classes:
@@ -48,100 +91,57 @@ test_matrix:
48
91
  exclude: windows_roles
49
92
  ```
50
93
 
51
- It consists of the following sections:
52
-
53
- #### Classes:
94
+ ### factsets
54
95
 
55
- This is where we list all of the classes that we want to test, normally this will just be a list of roles. Note that these classes must *actually exist* for reasons that should be obvious.
96
+ `spec/factsets/*.yaml`
56
97
 
57
- #### Nodes:
98
+ Factsets are used by the controlrepo gem to generate spec tests, which compile a given class against a certain set of facts. To create these factsets all we need to do is log onto a real machine that has puppet installed and run:
58
99
 
59
- Each node in the list refers one of two things depending on weather we are running **spec** or **acceptance** tests. If we are running **spec** tests each node refers to the name of a [fact set](#fact-sets) because this will be the set of facts that the `it { should compile }` test will be run against. If we are are running **acceptance** tests then each node will refer to a *nodeset* file which we can generate (or at least try to) using the `generate_nodesets` rake task. For acceptance testing the nodeset file will tell us how to spin up the VMs for each machine.
100
+ `puppet facts`
60
101
 
61
- #### Groups:
102
+ Which will give raw json output of every fact which puppet knows about. Usually I would recommend running this on each of the types of machines that you run in your infrastructure so that you have a good coverage. To make life easier you might want to direct it into a file instead of copying it from the command line:
62
103
 
63
- Groups are used to save us a bit of time and code (if you can call yaml that). Unsurprisingly a group is a way to bundle either classes or nodes into a group that we can refer to but it's name instead of repeating ourselves a whole bunch. There are 2 **default groups:**
104
+ `puppet facts > fact_set_name.json`
64
105
 
65
- - all_nodes
66
- - all_classes
106
+ Once we have our factset all we need to do is copy it into `spec/factsets/` inside out controlrepo and commit it to version control. Factsets are named based on their filename, not the ane of the server they came from (Although you can, if you want). i.e the following factset file:
67
107
 
68
- You can guess what they are for I hope.
108
+ `spec/factsets/server2008r2.yaml`
69
109
 
70
- *Note that groups CANNOT contain a mix of classes and nodes, only one or the other.*
110
+ Would map to a node named `server2008r2` in `controlrepo.yaml`
71
111
 
72
- #### Test Matrix:
112
+ ### nodesets
73
113
 
74
- This is the section of th config file that makes the magic happen. In the test matrix we choose on which nodes we will tests which classes. You can use groups anywhere here as you can see in the example above. We also have the option of using *include* and *exclude* which will be useful if you have a lot of groups.
75
-
76
- For example if we want to test all our non-windows roles on all of our linux boxes we can do something like this:
77
-
78
- ```yaml
79
- test_matrix:
80
- linux_nodes:
81
- include: 'all_classes'
82
- exclude: 'windows_roles'
83
- ```
84
-
85
- This is assuming that you have all of your linux nodes in the `linux_nodes` group and all of your Windows roles in the `windows_roles` group.
86
-
87
- When setting up your tests matrix don't worry too much about using groups that will cause duplicate combinations of `node -> class` pairs. The rake tasks run deduplication before running any of the tests to make sure that we are not wasting time. This happens at runtime and does not affect the file or anything.
88
-
89
- ## Hiera Data
90
-
91
- If you have hiera data inside your controlrepo (or somewhere else) the Controlrepo gem can be configured to use it. Just dump your `hiera.yaml` file from the puppet master into the `spec/` directory and you are good to go. **NOTE:** This assumes that the path to your hiera data (datadir) is relative to the root of the controlrepo, if not it will fall over
92
-
93
- ## R10k.yaml
94
-
95
- For the Controlrepo gem to be able to clone the controlrepo (itself) from git (into a temp dir) it needs an `r10k.yaml` file under the `spec/` directory. Don't worry about any of the paths here, we dynamically generate and override them. I realise that this is kind of redundant and will be looking into changing it in the future.
114
+ `spec/acceptance/nodesets/controlrepo-nodes.yml`
96
115
 
97
- TODO: Look into this ^
116
+ Nodesets are used when running acceptance tests. They instruct the controlrepo gem how to spin up virtual machines to run the code on. Actually, that's a lie... What's really happening with nodesets is that we are using [Beaker](https://github.com/puppetlabs/beaker) to spin up the machines and then a combination of Beaker and RSpec to test them. But you don't need to worry about that too much. Due to the fact that we are using beaker to do the heavy lifting here the nodeset files follow the same format they would for normal Beaker tests, which at the time of writing supports the following hypervisors:
98
117
 
99
- ## pre_conditions
118
+ - [VMWare Fusion](https://github.com/puppetlabs/beaker/blob/master/docs/VMWare-Fusion-Support.md)
119
+ - [Amazon EC2](https://github.com/puppetlabs/beaker/blob/master/docs/EC2-Support.md)
120
+ - [vSphere](https://github.com/puppetlabs/beaker/blob/master/docs/vSphere-Support.md)
121
+ - [Vagrant](https://github.com/puppetlabs/beaker/blob/master/docs/Vagrant-Support.md)
122
+ - [Google Compute Engine](https://github.com/puppetlabs/beaker/blob/master/docs/Google-Compute-Engine-Support.md)
123
+ - [Docker](https://github.com/puppetlabs/beaker/blob/master/docs/Docker-Support.md)
124
+ - [Openstack](https://github.com/puppetlabs/beaker/blob/master/docs/Openstack-Support.md)
125
+ - [Solaris](https://github.com/puppetlabs/beaker/blob/master/docs/Solaris-Support.md)
100
126
 
101
- If your spec tests are failing because if dependencies on the (closed source) PE modules, don't stress, there is a way around this! Let's start with an example: Somewhere in my puppet code I am managing something that needs to restart the Puppet Server i.e.
127
+ Before we configure a hypervisor to spin up a node however, we have to make sure that it can clone from a machine which is ready. The controlrepo gem **requires it's VMs to have puppet pre-installed.** It doesn't matter what version of puppet, as long as it is on the PATH and the `type` setting is configured correctly i.e.
102
128
 
103
- ```puppet
104
- file { '/etc/puppetlabs/puppet/puppet.conf':
105
- ensure => file,
106
- content => '#nothing',
107
- notify => Service['pe-puppetserver'], # This will fail without the PE module!
108
- }
109
- ```
110
-
111
- If we try to compile a catalog against this code it will fail because we are not including the PE class that manages `Service['pe-puppetserver']`, therefore it is not in the catalog and we cannot establist dependencies upon it.
112
-
113
- If we run into a situations like this we can use the `spec/pre_conditions` folder to get around them e.g. Putting this in the `spec/pre_conditions` folder will solve our issues:
114
-
115
- ```puppet
116
- service { 'pe-puppetserver':
117
- ensure => 'running',
118
- }
129
+ ```yaml
130
+ type: AIO # For machines that have the all-in-one agent installed (>=4.0 or >=2015.2)
131
+ # OR
132
+ type: pe # For puppet enterprise agents <2015.2
133
+ # OR
134
+ type: foss # For open source puppet <4.0
119
135
  ```
120
136
 
121
- This is because anything in this folder will get add to the catalog along with the class (role) we are testing.
122
-
123
- NOTE: [resource collector ovverides](https://docs.puppetlabs.com/puppet/latest/reference/lang_resources_advanced.html#amending-attributes-with-a-collector) could be useful in this situation.
124
-
125
- ## Lets go!
126
-
127
- Now to **run the spec tests** just do a:
128
-
129
- `bundler exec rake controlrepo_spec`
130
-
131
- ## Acceptance testing
132
-
133
- Now that we have a lot of stuff set up, we can also run acceptance testing! This will do much the same thing as the spec testing, except on an actual box. (It will run `include role::your_role` on the server and check for errors)
134
-
135
- This does however take a little more preparation. The main thing we need is that we need to know which nodes to spin up in order to do the testing. We use Beaker to actually interact with the hypervisor of choice and therefore we use their [nodeset file syntax](https://github.com/puppetlabs/beaker/blob/master/docs/Example-Vagrant-Hosts-Files.md). The only thing we need to do is **name the nodes the same aswe do for spec tests** and **have all the nodes in the same file**.
136
-
137
- Here is an example:
137
+ Here is an example of a nodeset file that you can use yourselves. It uses freely available Vagrant boxes from puppetlabs and Virtualbox as the Vagrant provider.
138
138
 
139
139
  ```yaml
140
- # spec/acceptance/nodesets/controlrepo.yaml
141
140
  HOSTS:
142
141
  centos6a:
143
142
  roles:
144
143
  - agent
144
+ type: aio
145
145
  platform: el-6-64
146
146
  box: puppetlabs/centos-6.6-64-puppet
147
147
  box_url: https://atlas.hashicorp.com/puppetlabs/boxes/centos-6.6-64-puppet
@@ -149,119 +149,147 @@ HOSTS:
149
149
  centos7b:
150
150
  roles:
151
151
  - agent
152
+ type: aio
152
153
  platform: el-7-64
153
154
  box: puppetlabs/centos-7.0-64-puppet
154
155
  box_url: https://atlas.hashicorp.com/puppetlabs/boxes/centos-7.0-64-puppet
155
156
  hypervisor: vagrant_virtualbox
156
- ubuntu1404a:
157
+ ubuntu1204:
158
+ roles:
159
+ - agent
160
+ type: aio
161
+ platform: ubuntu-12.04-32
162
+ box: puppetlabs/ubuntu-12.04-32-puppet
163
+ box_url: https://atlas.hashicorp.com/puppetlabs/boxes/ubuntu-12.04-32-puppet
164
+ hypervisor: vagrant_virtualbox
165
+ debian78:
157
166
  roles:
158
167
  - agent
159
- platform: ubuntu-14.04-64
160
- box: puppetlabs/ubuntu-14.04-64-puppet
161
- box_url: https://atlas.hashicorp.com/puppetlabs/boxes/ubuntu-14.04-64-puppet
168
+ type: aio
169
+ platform: debian-7.8-64
170
+ box: puppetlabs/debian-7.8-64-puppet
171
+ box_url: https://atlas.hashicorp.com/puppetlabs/boxes/debian-7.8-64-puppet
162
172
  hypervisor: vagrant_virtualbox
163
173
  ```
164
174
 
165
- Now when we run:
175
+ ### Hiera Data
166
176
 
167
- `bundle exec rake controlrepo_acceptance`
177
+ If you have hiera data inside your controlrepo (or somewhere else) the Controlrepo gem can be configured to use it. Just dump your `hiera.yaml` file from the puppet master into the `spec/` directory and you are good to go. **NOTE:** This assumes that the path to your hiera data (datadir) is relative to the root of the controlrepo, if not it will fall over.
168
178
 
169
- It will use the same test matrix we have already defined in `controlrepo.yaml` and spin up each node and do the testing.
179
+ ### R10k.yaml
170
180
 
171
- NOTE: The same test deduplication will be applied here as it is in spec tests, also; macines will be classified with **one** role, run, destroyed, re-created and then classified with the next role.
181
+ For the Controlrepo gem to be able to clone the controlrepo (itself) from git (into a temp dir) it needs an `r10k.yaml` file under the `spec/` directory. Don't worry about any of the paths here, we dynamically generate and override them. I realise that this is kind of redundant and will be looking into changing it in the future.
172
182
 
183
+ ## Spec testing
173
184
 
174
- ## Hard mode
185
+ Once you have your `controlrepo.yaml` and factsets set up you are ready to go with spec testing.
175
186
 
176
- The point of *hard mode* is to give people who are familiar with RSpec testing with puppet a set of useful tools that they can mix into their tests to save some hassle. We also want to help in getting your tests set up by automatically generating `.fixtures.yml` and nodesets.
187
+ To run the tests:
177
188
 
178
- ## Fact sets
189
+ `bundle exec rake controlrepo_spec`
179
190
 
180
- This gem introduces the concept of fact sets. Instead of manually specifying facts in each rspec test we can just dump the actual facts from the actual machines in our environment into a folder then test against them. To do this we first need to dump the facts into a json file:
191
+ This will do the following things:
181
192
 
182
- `puppet facts > server01.json`
193
+ 1. Create a temporary directory
194
+ 2. Clone all repos in the Puppetfile into the temporary directory
195
+ 3. Generate tests that use rspec-puppet
196
+ 4. Install required gems into temp dir using Bundler
197
+ 5. Run the tests
183
198
 
184
- Then grab this file and put it in `spec/facts` inside your control repo. It doesn't matter what you name these files, as long as they are in that directory and they end with `json` we will be able to find them. Fact sets are also heavily used by **easy mode**.
199
+ ## Acceptance testing
185
200
 
186
- **That's it!**
201
+ Acceptance testing works in much the same way as spec testing except that it requires a nodeset file along with `controlrepo.yaml`
187
202
 
188
- Now we can access all of these fact sets using `Controlrepo.facts`. Normally it would be implemented something like this:
203
+ To run the tests:
189
204
 
190
- ```ruby
191
- Controlrepo.facts.each do |facts|
192
- context "on #{facts['fqdn']}" do
193
- let(:facts) { facts }
194
- it { should compile }
195
- end
196
- end
197
- ```
205
+ `bundle exec rake controlrepo_acceptance`
198
206
 
207
+ This will do the following things:
199
208
 
200
- ## Rake tasks
209
+ 1. Create a temporary directory
210
+ 2. Clone all repos in the Puppetfile into the temporary directory
211
+ 3. Generate tests that use RSpec and Beaker
212
+ 4. Install required gems into temp dir using Bundler
213
+ 5. Run the tests, each test consists of:
214
+ - Spin up the VM
215
+ - Copy over the code
216
+ - Run puppet and catch any errors
217
+ - Run puppet again to catch anything that might not be idempotent
218
+ - Destroy the VM
201
219
 
202
- I have included a couple of little rake tasks to help get you started with testing your control repos. Set them up by adding this to your `Rakefile`
220
+ ## Using workarounds
203
221
 
204
- ```ruby
205
- require 'controlrepo/rake_tasks'
206
- ```
222
+ There may be situations where you cannot test everything that is in your puppet code, some common reasons for this include:
223
+
224
+ - Code is destined for a Puppet Master but the VM image is not a Puppet Master which means we can't restart certain services etc.
225
+ - A file is being pulled from somewhere that is only accessible in production
226
+ - Something is trying to connect to something else that does not exist
207
227
 
208
- The tasks are as follows:
228
+ Fear not! There is a solution for this, it's also a good way to practice writing *nasty* puppet code. For this exact purpose I have added the ability for the controlrepo gem to include extra bits of code in the tests to fix things. All you need to do is put a file containing puppet code here:
209
229
 
210
- ### generate_fixtures
230
+ `spec/pre_conditions/*.pp`
211
231
 
212
- `bundle exec rake generate_fixtures`
232
+ What this will do is it will take any puppet code from any files it finds in that directory and have it executed alongside the code that you are actually testing. For example if we are testing some code that notifies the `pe-puppetserver` service, but are not managing that service in our code because it is managed by the PE module that ships with Puppet Enterprise:
213
233
 
214
- This task will go though your Puppetfile, grab all of the modules in there and convert them into a `.fixtures.yml` file. It will also take the `environment.conf` file into account, check to see if you have any relative pathed directories and also include them into the `.fixtures.yml` as symlinks. e.g. If your files look like this:
234
+ ```puppet
235
+ # Somewhere in our code
236
+ file { '/etc/puppetlabs/puppet/puppet.conf':
237
+ ensure => file,
238
+ content => '#nothing',
239
+ notify => Service['pe-puppetserver'], # This will fail without the PE module!
240
+ }
241
+ ```
215
242
 
216
- **Puppetfile**
217
- ```ruby
218
- forge "http://forgeapi.puppetlabs.com"
243
+ We can add the service to the pre_conditions to make sure that out catalogs can compile e.g.
219
244
 
220
- # Modules from the Puppet Forge
221
- mod "puppetlabs/stdlib", "4.6.0"
222
- mod "puppetlabs/apache", "1.5.0"
245
+ ```puppet
246
+ # spec/pre_conditions/services.pp
247
+ service { 'pe-puppetserver':
248
+ ensure => 'running',
249
+ }
223
250
  ```
224
251
 
225
- **environment.conf**
226
- ```ini
227
- modulepath = site:modules:$basemodulepath
228
- environment_timeout = 0
229
- ```
252
+ However this is going to pos an issue when we get to acceptance testing. Due to the fact that acceptance tests actually run the code, not just try to compile a catalog, it will not be able to find the 'pe-pupetserver' service and will fail. One way to get around this is to use some of the optional parameters to the service resource e.g.
230
253
 
231
- Then the `.fixtures.yml` file that this rake task will create will look like this:
232
-
233
- ```yaml
234
- ---
235
- fixtures:
236
- symlinks:
237
- profiles: site/profiles
238
- roles: site/roles
239
- forge_modules:
240
- stdlib:
241
- repo: puppetlabs/stdlib
242
- ref: 4.6.0
243
- apache:
244
- repo: puppetlabs/apache
245
- ref: 1.5.0
254
+ ```puppet
255
+ # We are not going to actually have this service anywhere on our servers but
256
+ # our code needs to refresh it. This is to trck puppet into doing nothing
257
+ service { 'pe-puppetserver':
258
+ ensure => 'running',
259
+ enable => false,
260
+ hasrestart => false, # Force Puppet to use start and stop to restart
261
+ start => 'echo "Start"', # This will always exit 0
262
+ stop => 'echo "Stop"', # This will also always exit 0
263
+ hasstatus => false, # Force puppet to use our command for status
264
+ status => 'echo "Status"', # This will always exit 0 and therefore Puppet will think the service is running
265
+ provider => 'base',
266
+ }
246
267
  ```
247
268
 
248
- Notice that the symlinks are not the ones that we provided in `environment.conf`? This is because the rake task will go into each of directories, find the modules and create a symlink for each of them (This is what rspec expects).
269
+ Here we are specifying custom commands to run for starting, stopping and checking the status of a service. We know what the exit codes of these commands are going to be so we know what puppet will think the service is doing because we have [read the documentation](https://docs.puppetlabs.com/references/latest/type.html#service-attributes). If there are things other than services you need to check then I would recommend checking the documentation to see if you can mock things like we have here.
249
270
 
250
- ### generate_nodesets
271
+ [Resource collectors](https://docs.puppetlabs.com/puppet/latest/reference/lang_resources_advanced.html#amending-attributes-with-a-collector) are likely to come in handy here too. They allow you to override values of resources that match given criteria. This way we can override things for the sake of testing without having to change the code.
251
272
 
252
- `bundle exec rake generate_nodesets`
273
+ **NOTE:** If you need to run some pre_conditions during acceptance tests but not spec tests or vice versa you can check the status of the `$controlrepo_accpetance` variable. It will be `true` when run as an acceptance test and `undef` otherwise. If you want to limit pre_conditions to only certain nodes just use conditional logic based on facts like you normally would.
253
274
 
254
- This task will generate nodeset file required by beaker, based on the fact sets that exist in the repository. If you have any fact sets for which puppetlabs has a compatible vagrant box (i.e. centos, debian, ubuntu) it will detect the version specifics and set up the nodeset file, complete with box URL. If it doesn't know how to deal with a fact set it will output a boilerplate nodeset file that will need to be altered before it can be used.
275
+ ## Extra Tooling
255
276
 
256
- ### hiera_setup
277
+ I have provided some extra tools to use if you would prefer to write your own tests which I will go over here.
257
278
 
258
- `bundle exec rake hiera_setup`
279
+ ### Accessing fact sets in a traditional RSpec test
259
280
 
260
- Automatically modifies your hiera.yaml to point at the hieradata relative to it's position.
281
+ We can access all of our fact sets using `Controlrepo.facts`. Normally it would be implemented something like this:
261
282
 
262
- This rake task will look for a hiera.yaml file (Using the same method we use for [this](#using-hiera-data)). It will then look for a hieradata directory in the root for your control repo (needs to match [this](http://rubular.com/?regex=%2Fhiera%28%3F%3A.%2Adata%29%3F%2Fi)). It will then modify the datadirs of any backends it finds in `hiera.yaml` to point at these directories.
283
+ ```ruby
284
+ Controlrepo.facts.each do |facts|
285
+ context "on #{facts['fqdn']}" do
286
+ let(:facts) { facts }
287
+ it { should compile }
288
+ end
289
+ end
290
+ ```
263
291
 
264
- ## Running the tests (Hard mode)
292
+ ### Accessing Roles in a traditional RSpec test
265
293
 
266
294
  This gem allows us to easily and dynamically test all of the roles and profiles in our environment against fact sets from all of the nodes to which they will be applied. All we need to do is create a spec test that calls out to the `Controlrepo` ruby class:
267
295
 
@@ -305,7 +333,7 @@ It is not limited to just doing simple "It should compile" tests. You can put an
305
333
 
306
334
  Also since the `profiles`, `roles` and `facts` methods simply return arrays, you can iterate over them however you would like i.e. you could write a different set of tests for each profile and then just use the `facts` method to run those tests on every fact set.
307
335
 
308
- ## Filtering (Hard mode)
336
+ ### Filtering
309
337
 
310
338
  You can also filter your fact sets based on the value of any fact, including structured facts. (It will drill down into nested hashes and match those too, it's not just a dumb equality match)
311
339
 
@@ -327,7 +355,7 @@ describe 'profile::windows_appserver' do
327
355
  end
328
356
  ```
329
357
 
330
- ## Using hiera data (Hard Mode)
358
+ ### Using hiera data (In manual tests)
331
359
 
332
360
  You can also point these tests at your hiera data, you do this as you [normally would](https://github.com/rodjek/rspec-puppet#enabling-hiera-lookups) with rspec tests. However we do provide one helper to make this marginally easier. `Controlrepo.hiera_config` will look for hiera.yaml in the root of your control repo and also the spec directory, you will however need to set up the file itself e.g.
333
361
 
@@ -339,8 +367,7 @@ RSpec.configure do |c|
339
367
  end
340
368
  ```
341
369
 
342
-
343
- ## Extra Configuration
370
+ ### Extra Configuration
344
371
 
345
372
  You can modify the regexes that the gem uses to filter classes that it finds into roles and profiles. Just set up a Controlrepo object and pass regexes to the below settings.
346
373
 
@@ -352,3 +379,67 @@ repo.profile_regex = /.*/ # Tells the class how to find profiles, will be applie
352
379
 
353
380
  Note that you will need to call the `roles` and `profiles` methods on the object you just instantiated, not the main class e.g. `repo.roles` not `Controlrepo.roles`
354
381
 
382
+ ### Rake tasks
383
+
384
+ I have included a couple of little rake tasks to help get you started with testing your control repos. Set them up by adding this to your `Rakefile`
385
+
386
+ ```ruby
387
+ require 'controlrepo/rake_tasks'
388
+ ```
389
+
390
+ The tasks are as follows:
391
+
392
+ #### generate_fixtures
393
+
394
+ `bundle exec rake generate_fixtures`
395
+
396
+ This task will go though your Puppetfile, grab all of the modules in there and convert them into a `.fixtures.yml` file. It will also take the `environment.conf` file into account, check to see if you have any relative pathed directories and also include them into the `.fixtures.yml` as symlinks. e.g. If your files look like this:
397
+
398
+ **Puppetfile**
399
+ ```ruby
400
+ forge "http://forgeapi.puppetlabs.com"
401
+
402
+ # Modules from the Puppet Forge
403
+ mod "puppetlabs/stdlib", "4.6.0"
404
+ mod "puppetlabs/apache", "1.5.0"
405
+ ```
406
+
407
+ **environment.conf**
408
+ ```ini
409
+ modulepath = site:modules:$basemodulepath
410
+ environment_timeout = 0
411
+ ```
412
+
413
+ Then the `.fixtures.yml` file that this rake task will create will look like this:
414
+
415
+ ```yaml
416
+ ---
417
+ fixtures:
418
+ symlinks:
419
+ profiles: site/profiles
420
+ roles: site/roles
421
+ forge_modules:
422
+ stdlib:
423
+ repo: puppetlabs/stdlib
424
+ ref: 4.6.0
425
+ apache:
426
+ repo: puppetlabs/apache
427
+ ref: 1.5.0
428
+ ```
429
+
430
+ Notice that the symlinks are not the ones that we provided in `environment.conf`? This is because the rake task will go into each of directories, find the modules and create a symlink for each of them (This is what rspec expects).
431
+
432
+ #### generate_nodesets
433
+
434
+ `bundle exec rake generate_nodesets`
435
+
436
+ This task will generate nodeset file required by beaker, based on the fact sets that exist in the repository. If you have any fact sets for which puppetlabs has a compatible vagrant box (i.e. centos, debian, ubuntu) it will detect the version specifics and set up the nodeset file, complete with box URL. If it doesn't know how to deal with a fact set it will output a boilerplate nodeset file that will need to be altered before it can be used.
437
+
438
+ #### hiera_setup
439
+
440
+ `bundle exec rake hiera_setup`
441
+
442
+ Automatically modifies your hiera.yaml to point at the hieradata relative to it's position.
443
+
444
+ This rake task will look for a hiera.yaml file (Using the same method we use for [this](#using-hiera-data)). It will then look for a hieradata directory in the root for your control repo (needs to match [this](http://rubular.com/?regex=%2Fhiera%28%3F%3A.%2Adata%29%3F%2Fi)). It will then modify the datadirs of any backends it finds in `hiera.yaml` to point at these directories.
445
+
data/Rakefile ADDED
@@ -0,0 +1,2 @@
1
+ require 'rubygems/tasks'
2
+ Gem::Tasks.new
data/controlrepo.gemspec CHANGED
@@ -3,7 +3,7 @@ $LOAD_PATH.unshift File.expand_path("../lib", __FILE__)
3
3
 
4
4
  Gem::Specification.new do |s|
5
5
  s.name = "controlrepo"
6
- s.version = "1.1.0"
6
+ s.version = "2.0.0"
7
7
  s.authors = ["Dylan Ratcliffe"]
8
8
  s.email = ["dylan.ratcliffe@puppetlabs.com"]
9
9
  s.homepage = ""
data/lib/controlrepo.rb CHANGED
@@ -17,6 +17,7 @@ class Controlrepo
17
17
  attr_accessor :tempdir
18
18
  attr_accessor :spec_dir
19
19
  attr_accessor :temp_modulepath
20
+ attr_accessor :nodeset_file
20
21
 
21
22
  # Create methods on self so that we can access these basic things without
22
23
  # having to actually instantiate the class, I'm debating how much stuff
@@ -82,9 +83,10 @@ class Controlrepo
82
83
  @root = search_path
83
84
  @puppetfile = File.expand_path('./Puppetfile',@root)
84
85
  @environment_conf = File.expand_path('./environment.conf',@root)
85
- @facts_dir = File.expand_path('./spec/facts',@root)
86
+ @facts_dir = File.expand_path('./spec/factsets',@root)
86
87
  @spec_dir = File.expand_path('./spec',@root)
87
88
  @facts_files = Dir["#{@facts_dir}/*.json"]
89
+ @nodeset_file = File.expand_path('./spec/acceptance/nodesets/controlrepo-nodes.yml',@root)
88
90
  @role_regex = /role[s]?:{2}/
89
91
  @profile_regex = /profile[s]?:{2}/
90
92
  @temp_environmentpath = nil
@@ -11,29 +11,27 @@ class Controlrepo
11
11
 
12
12
  # You need to pass in an array of strings for members, not objects, it will find the objects
13
13
  # by itself, and yes it will reference them, not just create additional ones, woo!
14
+
14
15
  def initialize(name = nil, members = [])
15
16
  @name = name
16
17
  @members = []
17
18
 
18
- if members.any?
19
+ if Controlrepo::Group.valid_members?(members)
20
+ # If it's already a valid list just chuck it in there
21
+ @members = members
22
+ elsif members.is_a?(Hash)
23
+ # if it's a hash then do subtractive stiff
24
+ @members = Controlrepo::Group.subtractive_to_list(members)
25
+ else
26
+ # Turn it into a full list
19
27
  member_objects = []
20
- members.each do |member|
21
- # Try to find the type for each member
22
- if Controlrepo::Class.find(member)
23
- member_objects << Controlrepo::Class.find(member)
24
- elsif Controlrepo::Node.find(member)
25
- member_objects << Controlrepo::Node.find(member)
26
- else
27
- raise "#{member} was not found in the list of nodes or classes!"
28
- end
29
- end
28
+
29
+ # This should also handle lists that include groups
30
+ members.each { |member| member_objects << Controlrepo::TestConfig.find_list(member) }
31
+ member_objects.flatten!
30
32
 
31
33
  # Check that they are all the same type
32
- if member_objects.all? { |item| item.is_a?(Controlrepo::Class) }
33
- type = Controlrepo::Class
34
- elsif member_objects.all? { |item| item.is_a?(Controlrepo::Node) }
35
- type = Controlrepo::Node
36
- else
34
+ unless Controlrepo::Group.valid_members?(member_objects)
37
35
  raise 'Groups must contain either all nodes or all classes. Either there was a mix, or something was spelled wrong'
38
36
  end
39
37
 
@@ -57,5 +55,31 @@ class Controlrepo
57
55
  def self.all
58
56
  @@all
59
57
  end
58
+
59
+ def self.valid_members?(members)
60
+ # Check that they are all the same type
61
+ # Also catch any errors to assume it's invalid
62
+ begin
63
+ if members.all? { |item| item.is_a?(Controlrepo::Class) }
64
+ return true
65
+ elsif members.all? { |item| item.is_a?(Controlrepo::Node) }
66
+ return true
67
+ else
68
+ return false
69
+ end
70
+ rescue
71
+ return false
72
+ end
73
+ end
74
+
75
+ def self.subtractive_to_list(subtractive_hash)
76
+ # Take a hash that looks like this:
77
+ # { 'include' => 'somegroup'
78
+ # 'exclude' => 'other'}
79
+ # and return a list of classes/nodes
80
+ include_list = Controlrepo::TestConfig.find_list(subtractive_hash['include'])
81
+ exclude_list = Controlrepo::TestConfig.find_list(subtractive_hash['exclude'])
82
+ include_list - exclude_list
83
+ end
60
84
  end
61
85
  end
@@ -79,6 +79,10 @@ task :controlrepo_autotest_prep do
79
79
  @repo = Controlrepo.new
80
80
  @config = Controlrepo::TestConfig.new("#{@repo.spec_dir}/controlrepo.yaml")
81
81
 
82
+ # Verify that all the files exist for the tests we have set up
83
+ @config.spec_tests.each { |test| @config.verify_spec_test(@repo,test) }
84
+ @config.acceptance_tests.each { |test| @config.verify_acceptance_test(@repo,test) }
85
+
82
86
  # Deploy r10k to a temp dir
83
87
  @config.r10k_deploy_local(@repo)
84
88
 
@@ -102,11 +106,11 @@ task :controlrepo_autotest_prep do
102
106
  @config.write_gemfile(@repo.tempdir)
103
107
 
104
108
  # Deduplicate and write the tests (Spec and Acceptance)
105
- Controlrepo::Test.deduplicate(@config.tests).each do |test|
109
+ Controlrepo::Test.deduplicate(@config.spec_tests).each do |test|
106
110
  @config.write_spec_test("#{@repo.tempdir}/spec/classes",test)
107
111
  end
108
112
 
109
- @config.write_acceptance_tests("#{@repo.tempdir}/spec/acceptance",Controlrepo::Test.deduplicate(@config.tests))
113
+ @config.write_acceptance_tests("#{@repo.tempdir}/spec/acceptance",Controlrepo::Test.deduplicate(@config.acceptance_tests))
110
114
 
111
115
  # Parse the current hiera config, modify, and write it to the temp dir
112
116
  hiera_config = @repo.hiera_config
@@ -9,8 +9,10 @@ class Controlrepo
9
9
 
10
10
  attr_accessor :classes
11
11
  attr_accessor :nodes
12
- attr_accessor :groups
13
- attr_accessor :tests
12
+ attr_accessor :node_groups
13
+ attr_accessor :class_groups
14
+ attr_accessor :spec_tests
15
+ attr_accessor :acceptance_tests
14
16
  attr_accessor :environment
15
17
 
16
18
  def initialize(file, environment = 'production')
@@ -23,20 +25,65 @@ class Controlrepo
23
25
  @environment = environment
24
26
  @classes = []
25
27
  @nodes = []
26
- @groups = []
27
- @tests = []
28
+ @node_groups = []
29
+ @class_groups = []
30
+ @spec_tests = []
31
+ @acceptance_tests = []
32
+
33
+ # Add the 'all_classes' and 'all_nodes' default groups
34
+ @node_groups << Controlrepo::Group.new('all_nodes',@nodes)
35
+ @class_groups << Controlrepo::Group.new('all_classes',@classes)
28
36
 
29
37
  config['classes'].each { |clarse| @classes << Controlrepo::Class.new(clarse) }
30
38
  config['nodes'].each { |node| @nodes << Controlrepo::Node.new(node) }
31
- config['groups'].each { |name, members| @groups << Controlrepo::Group.new(name, members) }
39
+ config['node_groups'].each { |name, members| @node_groups << Controlrepo::Group.new(name, members) }
40
+ config['class_groups'].each { |name, members| @class_groups << Controlrepo::Group.new(name, members) }
41
+
42
+ config['test_matrix'].each do |machines, settings|
43
+ if settings['tests'] == 'spec'
44
+ @spec_tests << Controlrepo::Test.new(machines,settings['classes'],settings['options'])
45
+ elsif settings['tests'] == 'acceptance'
46
+ @acceptance_tests << Controlrepo::Test.new(machines,settings['classes'],settings['options'])
47
+ elsif settings['tests'] == 'all_tests'
48
+ test = Controlrepo::Test.new(machines,settings['classes'],settings['options'])
49
+ @spec_tests << test
50
+ @acceptance_tests << test
51
+ end
52
+ # TODO: Work out some way to set per-test options like idempotency
53
+ #@spec_tests << Controlrepo::Test.new(machines,roles)
54
+ #@acceptance_tests
55
+ end
56
+ end
32
57
 
33
- # Add the 'all_classes' and 'all_nodes' default groups
34
- @groups << Controlrepo::Group.new('all_nodes',@nodes)
35
- @groups << Controlrepo::Group.new('all_classes',@classes)
58
+ def self.find_list(thing)
59
+ # Takes a string and finds an object or list of objects to match, will
60
+ # take nodes, classes or groups
61
+ if Controlrepo::Group.find(thing)
62
+ return Controlrepo::Group.find(thing).members
63
+ elsif Controlrepo::Class.find(thing)
64
+ return [Controlrepo::Class.find(thing)]
65
+ elsif Controlrepo::Node.find(thing)
66
+ return [Controlrepo::Node.find(thing)]
67
+ else
68
+ raise "Could not find #{thing} in list of classes, nodes or groups"
69
+ end
70
+ end
36
71
 
37
- config['test_matrix'].each do |machines, roles|
38
- # TODO: Work out some way to set per-test options like idempotency
39
- @tests << Controlrepo::Test.new(machines,roles)
72
+ def verify_spec_test(controlrepo,test)
73
+ test.nodes.each do |node|
74
+ unless controlrepo.facts_files.any? { |file| file =~ /\/#{node.name}\.json/ }
75
+ raise "Could not find factset for node: #{node.name}"
76
+ end
77
+ end
78
+ end
79
+
80
+ def verify_acceptance_test(controlrepo,test)
81
+ require 'yaml'
82
+ nodeset = YAML.load_file(controlrepo.nodeset_file)
83
+ test.nodes.each do |node|
84
+ unless nodeset['HOSTS'].has_key?(node.name)
85
+ raise "Could not find nodeset for node: #{node.name}"
86
+ end
40
87
  end
41
88
  end
42
89
 
@@ -15,7 +15,7 @@ describe "Acceptance Testing" do
15
15
  <% test.classes.each do |cls| -%>
16
16
  describe "<%= cls.name %> on <%= node.name %>" do
17
17
  after :all do
18
- logger.line_prefix = " "
18
+ logger.line_prefix = " "
19
19
  $nwm.cleanup
20
20
  end
21
21
 
@@ -75,7 +75,14 @@ describe "Acceptance Testing" do
75
75
  describe "running puppet" do
76
76
  it "should run with no errors" do
77
77
  expect {
78
- apply_manifest_on($hosts,"include <%= cls.name %>",{:catch_failures => true})
78
+ manifest = <<CODE
79
+ $controlrepo_accpetance = true
80
+
81
+ <%= pre_condition %>
82
+
83
+ include <%= cls.name %>
84
+ CODE
85
+ apply_manifest_on($hosts,manifest,{:catch_failures => true})
79
86
  }.not_to raise_exception
80
87
  end
81
88
  end
@@ -83,7 +90,14 @@ describe "Acceptance Testing" do
83
90
  describe "checking for idempotency" do
84
91
  it "should run with no changes" do
85
92
  expect {
86
- apply_manifest_on($hosts,"include <%= cls.name %>",{:catch_changes => true})
93
+ manifest = <<CODE
94
+ $controlrepo_accpetance = true
95
+
96
+ <%= pre_condition %>
97
+
98
+ include <%= cls.name %>
99
+ CODE
100
+ apply_manifest_on($hosts,manifest,{:catch_changes => true})
87
101
  }.not_to raise_exception
88
102
  end
89
103
  end
@@ -1,3 +1,11 @@
1
+ # This has all been hacked out of the beaker-rspec gem. The reason I have
2
+ # hacked it out instead of just using it the way it was intended is that
3
+ # it is very stuck in its ways around how it spins up all the servers
4
+ # first, then goes ahead and runs the tests. Because I don't want to do this
5
+ # I have had to replicate MOST of the functionality, EXCEPT the stuff I don't
6
+ # want. This is annoying but as far as I can tell this is the only way to do
7
+ # it
8
+
1
9
  require 'beaker'
2
10
  require 'beaker-rspec/beaker_shim' # This overloads Rspec's methods and provides the interface between beaker and RSpec
3
11
  require "beaker-rspec/helpers/serverspec"
@@ -27,7 +35,7 @@ RSpec.configure do |c|
27
35
 
28
36
  #default option values
29
37
  defaults = {
30
- :nodeset => 'controlrepo',
38
+ :nodeset => 'controlrepo-nodes',
31
39
  }
32
40
  #read env vars
33
41
  env_vars = {
@@ -68,4 +76,7 @@ RSpec.configure do |c|
68
76
  end
69
77
  end
70
78
 
79
+ # Set the number of lines it will print
80
+ options[:trace_limit] = 1000
81
+
71
82
  OPTIONS = options
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: controlrepo
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.1.0
4
+ version: 2.0.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Dylan Ratcliffe
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2015-10-28 00:00:00.000000000 Z
11
+ date: 2015-11-11 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: rake
@@ -130,7 +130,10 @@ extensions: []
130
130
  extra_rdoc_files: []
131
131
  files:
132
132
  - ".gitignore"
133
+ - Gemfile
134
+ - Gemfile.lock
133
135
  - README.md
136
+ - Rakefile
134
137
  - controlrepo.gemspec
135
138
  - lib/controlrepo.rb
136
139
  - lib/controlrepo/beaker.rb
@@ -168,9 +171,8 @@ required_rubygems_version: !ruby/object:Gem::Requirement
168
171
  version: '0'
169
172
  requirements: []
170
173
  rubyforge_project:
171
- rubygems_version: 2.4.7
174
+ rubygems_version: 2.4.6
172
175
  signing_key:
173
176
  specification_version: 4
174
177
  summary: ''
175
178
  test_files: []
176
- has_rdoc: