pdqtest 1.4.1 → 1.9.9beta2
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.travis.yml +9 -3
- data/doc/acceptance_tests.md +100 -22
- data/doc/caching.md +5 -1
- data/doc/development.md +16 -5
- data/doc/emoji.md +20 -9
- data/doc/enabling_testing.md +15 -3
- data/doc/examples.md +25 -6
- data/doc/hiera.md +30 -11
- data/doc/installation.md +59 -8
- data/doc/pdk.md +334 -0
- data/doc/puppet_facts.md +45 -3
- data/doc/puppet_module_dependencies.md +34 -16
- data/doc/running_tests.md +35 -15
- data/doc/test_generation.md +11 -19
- data/doc/tips_and_tricks.md +17 -8
- data/doc/troubleshooting.md +19 -8
- data/doc/upgrading.md +125 -6
- data/doc/windows.md +51 -0
- data/docker_images/centos/Dockerfile +4 -3
- data/docker_images/centos/Makefile +1 -1
- data/docker_images/ubuntu/Dockerfile +3 -3
- data/docker_images/windows/Dockerfile +26 -0
- data/docker_images/windows/make.ps1 +2 -0
- data/exe/pdqtest +73 -28
- data/lib/pdqtest/core.rb +1 -1
- data/lib/pdqtest/docker.rb +143 -51
- data/lib/pdqtest/emoji.rb +33 -7
- data/lib/pdqtest/fastcheck.rb +19 -0
- data/lib/pdqtest/instance.rb +16 -15
- data/lib/pdqtest/logger.rb +35 -0
- data/lib/pdqtest/pdk.rb +98 -0
- data/lib/pdqtest/pdqtest1x.rb +115 -0
- data/lib/pdqtest/puppet.rb +277 -134
- data/lib/pdqtest/skeleton.rb +119 -39
- data/lib/pdqtest/upgrade.rb +42 -42
- data/lib/pdqtest/util.rb +82 -2
- data/lib/pdqtest/version.rb +1 -2
- data/pdqtest.gemspec +8 -13
- data/res/{skeleton → acceptance}/init.bats +0 -0
- data/res/{skeleton → acceptance}/init__before.bats +0 -0
- data/res/{skeleton → acceptance}/init__setup.sh +0 -0
- data/res/skeleton/.puppet-lint.rc +5 -0
- data/res/skeleton/{dot_travis.yml → .travis.yml} +0 -0
- data/res/skeleton/Makefile +20 -5
- data/res/skeleton/make.ps1 +66 -0
- data/res/skeleton/spec/fixtures/hiera.yaml +13 -0
- data/res/skeleton/spec/fixtures/hieradata/test.yaml +11 -0
- data/res/templates/examples_init.pp.erb +1 -1
- metadata +47 -115
- data/lib/pdqtest/lint.rb +0 -31
- data/lib/pdqtest/rspec.rb +0 -69
- data/lib/pdqtest/syntax.rb +0 -20
- data/res/skeleton/Gemfile +0 -7
- data/res/skeleton/Rakefile +0 -3
- data/res/skeleton/dot_gitignore +0 -9
- data/res/skeleton/dot_rspec +0 -2
- data/res/skeleton/hiera.yaml +0 -7
- data/res/skeleton/spec_helper.rb +0 -4
- data/res/skeleton/test.yaml +0 -3
data/doc/pdk.md
ADDED
@@ -0,0 +1,334 @@
|
|
1
|
+
# PDK Integration
|
2
|
+
|
3
|
+
First off, what is PDK? PDK is Puppet's development kit. The bits we are
|
4
|
+
interested in right now are:
|
5
|
+
* Validation `pdk validate...`
|
6
|
+
* Testing `pdk test...`
|
7
|
+
* Packaging `pdk build...`
|
8
|
+
|
9
|
+
There's a lot more to PDK then just running simple tests though, PDK includes
|
10
|
+
its own version of ruby and various other tools.
|
11
|
+
|
12
|
+
On Linux PDK behaves (or at least appears to behave...) like a thin wrapper
|
13
|
+
around ruby, in the same vein as RBENV/RVM. On Windows, things are more involved
|
14
|
+
since it needs to install specific system libraries and reference them at
|
15
|
+
runtime somehow. This is why running `bundle install` on PDK project in windows
|
16
|
+
can cause strange and interesting blow-ups (don't do that!).
|
17
|
+
|
18
|
+
## What's the point of PDQTest if we have PDK?
|
19
|
+
PDQTest provides an easy way to run acceptance tests using light-weight
|
20
|
+
Docker containers which we abuse to the point of treating them _almost_ like
|
21
|
+
VMs in order to extract maximum speed. It offers a different approach to
|
22
|
+
[Beaker](https://github.com/puppetlabs/beaker):
|
23
|
+
* Minimal to no project configuration required, just `metadata.json` and a few
|
24
|
+
other files we setup for you
|
25
|
+
* Simple acceptance tests written in BASH or PowerShell
|
26
|
+
* Provide _good enough_ (but not perfect) testing:
|
27
|
+
* Generic Ubuntu, Centos, Windows test environments
|
28
|
+
* Capable of mocking large/slow/networked systems by replacing or installing
|
29
|
+
binaries inside the container at the OS level, eg:
|
30
|
+
* Fake installation of Database server's by mocking the installer your
|
31
|
+
puppet `exec` would call with a simple python script
|
32
|
+
* Network tools such as `realmd` with a simple mock `realm` command
|
33
|
+
written in Python
|
34
|
+
* Complex OS commands that need to report different state between puppet
|
35
|
+
runs can be mocked with Python and a small SQLite database or simple
|
36
|
+
text files (eg the `sysctl` command)
|
37
|
+
* Fake different Unix-like Operating Systems like AIX by subverting the
|
38
|
+
commands that your puppet code would executes. This is particularly
|
39
|
+
useful in this case since there are no AIX desktop virtualisation
|
40
|
+
solutions right now and there probably never will be
|
41
|
+
* See [examples](examples.md) for real-world modules using these
|
42
|
+
techniques
|
43
|
+
|
44
|
+
Of course, this will only ever give you an approximation of a real-world system
|
45
|
+
but in many cases this is all that's needed. The complexity of setting up a
|
46
|
+
100% accurate test environment is a daunting task and one that may never provide
|
47
|
+
ROI on the effort spent configuring and maintaining it.
|
48
|
+
|
49
|
+
If perfect accuracy is indeed required, then PDQTest is not what you are looking
|
50
|
+
for and you should look towards solutions like Beaker or
|
51
|
+
[Kitchen CI](https://kitchen.ci/).
|
52
|
+
|
53
|
+
## How does PDQTest work with PDK? Do I have to choose?
|
54
|
+
|
55
|
+
No you don't have to choose! PDQTest integrates with PDK by making the minimal
|
56
|
+
amount of invasive changes:
|
57
|
+
|
58
|
+
### New projects
|
59
|
+
New projects should be created using PDK: `pdk new module`.
|
60
|
+
|
61
|
+
This will give you all the PDK versioned files you would ever need. When you run
|
62
|
+
`pdqtest init` on a new project created with PDK, we only copy in our own
|
63
|
+
integration files and do not not touch the PDK generated files at all (the ones
|
64
|
+
marked 🛠 below),
|
65
|
+
|
66
|
+
### Existing PDQTest projects
|
67
|
+
To make upgrading to PDQTest 2.0 + PDK easy for our existing users, we automate
|
68
|
+
the workflow normally carried out by `pdk convert`:
|
69
|
+
|
70
|
+
We do not include the entire set of files that would have been generated by
|
71
|
+
`pdk convert` or `pdk new module`. If you want the full set, you should run
|
72
|
+
one of these commands yourself before `pdqtest init` and they will be left alone
|
73
|
+
unless PDK starts writing new files that we are already using.
|
74
|
+
|
75
|
+
**metadata.json**
|
76
|
+
|
77
|
+
* Enable PDK by adding the fields:
|
78
|
+
* `pdk-version`
|
79
|
+
* `template-url`
|
80
|
+
* `template-ref`
|
81
|
+
The values for these fields are obtained automatically from PDK itself and are
|
82
|
+
consistent with the system installed PDK.
|
83
|
+
|
84
|
+
**Gemfile.project**
|
85
|
+
|
86
|
+
* Add a reference to the `pdqtest` gem
|
87
|
+
|
88
|
+
**Miscellaneous skeletons**
|
89
|
+
|
90
|
+
When `pdqtest init` is run, we install a small set of skeleton files in the root
|
91
|
+
of your project. This is a one-off operation to get you started. After your
|
92
|
+
module has been marked as PDK compatible, we no longer attempt to update these
|
93
|
+
files and you must only manage them using PDK from then-on.
|
94
|
+
|
95
|
+
Skeletons are generated on the fly using PDK itself, so they are always
|
96
|
+
up-to-date according to the PDK you have installed on your system.
|
97
|
+
|
98
|
+
### Existing non-PDQTest projects
|
99
|
+
If you have an existing Puppet project that does not use PDQTest or PDK (eg
|
100
|
+
created by hand or with the old `puppet module generate` command), then the
|
101
|
+
recommended way to enable PDQTest is to first enable PDK by running
|
102
|
+
`pdk convert` and following the wizard.
|
103
|
+
|
104
|
+
You may then enable `PDQTest` by running `pdqtest init`.
|
105
|
+
|
106
|
+
Depending on the state of your project, you may be able to bypass the
|
107
|
+
`pdk convert` process by just running `pdqtest init` but this isn't recommended
|
108
|
+
or supported.
|
109
|
+
|
110
|
+
|
111
|
+
### PDQTest directory structure
|
112
|
+
These are the files installed, files marked 🛠 are generated by PDK:
|
113
|
+
|
114
|
+
```
|
115
|
+
├── bitbucket-pipelines.yml
|
116
|
+
├── Gemfile 🛠
|
117
|
+
├── Gemfile.local
|
118
|
+
├── Gemfile.project
|
119
|
+
├── .gitignore 🛠
|
120
|
+
├── Makefile
|
121
|
+
├── make.ps1
|
122
|
+
├── .pdkignore 🛠
|
123
|
+
├── .puppet-lint.rc
|
124
|
+
├── Rakefile 🛠
|
125
|
+
├── spec
|
126
|
+
│ ├── default_facts.yml 🛠
|
127
|
+
│ ├── fixtures
|
128
|
+
│ │ ├── hieradata
|
129
|
+
│ │ │ └── test.yaml
|
130
|
+
│ │ └── hiera.yaml
|
131
|
+
│ └── spec_helper.rb 🛠
|
132
|
+
└── .travis.yml
|
133
|
+
```
|
134
|
+
|
135
|
+
The remaining files are specific to PDQTest - notably:
|
136
|
+
* `.travis.yml` - Complete test suite for Linux modules
|
137
|
+
* `bitbucket-pipelines.yaml` - RSpec testing only
|
138
|
+
* `.puppet-lint.rc` - Make lint errors test failures, ignore double quotes, etc
|
139
|
+
* `Makefile` - Essential launch script for Linux
|
140
|
+
* `make.ps1` - Essential launch script for Windows
|
141
|
+
* `spec/fixtures/hiera.yaml` - Mock system-wide `hiera.yaml` file
|
142
|
+
* `spec/fixtures/hieradata/test.yaml` - Mock system-wide hieradata
|
143
|
+
* `Gemfile.project` - Used to enable the PDQTest gem (and any others you need)
|
144
|
+
* `Gemfile.local` - Transient file created at runtime by `Makefile` or
|
145
|
+
`make.ps1` to enable the PDQTest gem at runtime. This is a vital integration
|
146
|
+
point, see details below
|
147
|
+
* `.gitignore` - PDK has a massive list of files to ignore for new projects so
|
148
|
+
these are imported
|
149
|
+
|
150
|
+
|
151
|
+
## Why are the launch scripts essential/How does the PDQTest gem load itself?
|
152
|
+
PDK provides its own `Gemfile` to select the gems available at runtime. There
|
153
|
+
are hooks in this file to load additional configuration from two locations:
|
154
|
+
* `~/.gemfile`
|
155
|
+
* `YOUR_PROJECT/Gemfile.local`
|
156
|
+
|
157
|
+
We support side-by-side installation of all versions of PDQTest so that users
|
158
|
+
are not forced to upgrade when they are not ready to, so we need a per-project
|
159
|
+
way to load our gem into the bundle. `Gemfile.local` is perfect for this but we
|
160
|
+
are not supposed to store permanent data in there according to PDK's generated
|
161
|
+
`.gitignore`.
|
162
|
+
|
163
|
+
The simplest way to workaround this issue is for us to create a symlink from
|
164
|
+
`Gemfile.project` to `Gemfile.local` at runtime and we _must_ use our launch
|
165
|
+
scripts to do this since we are unable to use any Ruby code to do this trick
|
166
|
+
since we are not yet in the bundle.
|
167
|
+
|
168
|
+
For this reason, you **must** launch PDQTest using the provided `Makefile` or
|
169
|
+
`make.ps1` scripts, at least initially.
|
170
|
+
|
171
|
+
For further background see:
|
172
|
+
* [PDK-1177](https://tickets.puppetlabs.com/browse/PDK-1177) and
|
173
|
+
* [#50](https://github.com/declarativesystems/pdqtest/issues/50)
|
174
|
+
|
175
|
+
## When developing PDQTest, your version number **must** match a released version
|
176
|
+
_This should only affects PDQTest developers - for posterity..._
|
177
|
+
|
178
|
+
Due to some strange hackery between `pdk bundle` and `bundle exec` you have one
|
179
|
+
more gotcha when it comes to using developing PDQTest and thats that the version
|
180
|
+
number of the gem you build *MUST* match a released version of the gem on
|
181
|
+
rubygems.org otherwise `bundle exec` will barf with errors like this:
|
182
|
+
|
183
|
+
```shell
|
184
|
+
$ bundle exec pdqtest
|
185
|
+
Could not find hiera-3.4.5 in any of the sources
|
186
|
+
Run `bundle install` to install missing gems.
|
187
|
+
```
|
188
|
+
|
189
|
+
There seems no other way to fix this other than building the development gem
|
190
|
+
with a version matching an existing release. The gem then needs to be installed
|
191
|
+
using `gem install ...` and it will then magically start working. Using other
|
192
|
+
gem tricks like setting `:path` will appear to work but fail when `bundle exec`
|
193
|
+
is called. The other approach of trying to run everything with
|
194
|
+
`pdk bundle exec pdqtest` fails with various errors depending what platform your
|
195
|
+
on, presumably because `pdqtest` runs `pdk` command as part of its tests.
|
196
|
+
|
197
|
+
The final final caveat of this approach is that the PDQTest gem dependencies are
|
198
|
+
pinned at whatever the _real_ release uses, so if any were changed or added they
|
199
|
+
need to be temporarily added to `Gemfile.project`
|
200
|
+
|
201
|
+
The above works on Linux... For windows there is also the situation where:
|
202
|
+
* PDK vendors its own gems that magically appear
|
203
|
+
* PDK can only use "published" gems(specs?) it downloads - or appears to
|
204
|
+
* when you go to `bundle install` without using PDK you blow up the lock
|
205
|
+
file
|
206
|
+
* When you `bundle exec pdqtest` you get the above error because _something_
|
207
|
+
uses the old one `gemspec`
|
208
|
+
* The answer seems to be:
|
209
|
+
1. use `gem 'pdqtest', :path=>'c:/vagrant/pdqtest'` in `Gemfile.local`
|
210
|
+
2. `pdk bundle install`
|
211
|
+
3. `bundle update pdqtest` - yes even though your not supposed to...
|
212
|
+
4. `bundle exec pdqtest` - now it should work... why is beyond me
|
213
|
+
* If you still have errors:
|
214
|
+
1. `bundle install`
|
215
|
+
2. `rm Gemfile.lock`
|
216
|
+
3. `pdk bundle install` - working?
|
217
|
+
|
218
|
+
Somtimes you just have to upload the whole gem file as a pre-release object:
|
219
|
+
* Version set to x.x.xWHATEVER
|
220
|
+
* `gem push pdqtest-x.x.xWHATEVER.gem`
|
221
|
+
* `gem install pdqtest --pre`
|
222
|
+
|
223
|
+
|
224
|
+
If your just want to use PDQTest, you will hopefully never have to worry about
|
225
|
+
this... On the other hand, if this all sounds confusing... That's because it is!
|
226
|
+
|
227
|
+
## What happens when I upgrade PDQTest?
|
228
|
+
When a new version of PDQTest is released, upgrade by running:
|
229
|
+
|
230
|
+
```shell
|
231
|
+
gem install pdqtest
|
232
|
+
cd /your/project
|
233
|
+
pdqtest upgrade
|
234
|
+
pdk bundle install
|
235
|
+
```
|
236
|
+
|
237
|
+
This updates:
|
238
|
+
* Project gems in `Gemfile.project` (currently `pdqtest` and `puppet-strings`)
|
239
|
+
* Our CI and CLI integrations:
|
240
|
+
* `Makefile`
|
241
|
+
* `make.ps1`
|
242
|
+
* `.travis.yml`
|
243
|
+
* `bitbucket-pipelines.yml`
|
244
|
+
|
245
|
+
The final `pdk bundle install` command upgrades PDK's bundle to the include our
|
246
|
+
update and is critical to making the whole process work.
|
247
|
+
|
248
|
+
## What happens when I upgrade PDK?
|
249
|
+
PDK operates independently from PDQTest and maintains its own files. Your free
|
250
|
+
to run `pdk update` to upgrade the your files to the latest PDK templated ones
|
251
|
+
at any time.
|
252
|
+
|
253
|
+
To protect PDQTest files from alteration (notably `.travis.yml` and
|
254
|
+
`bitbucket-pipelines.yml`) we merge them with any existing `.sync.yml` to
|
255
|
+
prevent churn.
|
256
|
+
|
257
|
+
If you have further customisations to PDK controlled files your options are:
|
258
|
+
* Use git to revert any change by PDK
|
259
|
+
* Use `.sync.yml` to influence file (re)generation
|
260
|
+
[blog](https://puppet.com/blog/guide-converting-module-pdk)
|
261
|
+
[reference](https://github.com/puppetlabs/pdk-templates)
|
262
|
+
[example](https://github.com/puppetlabs/puppetlabs-apt/blob/master/.sync.yml)
|
263
|
+
|
264
|
+
## How do I use PDK with PDQTest installed? What can and can't I do?
|
265
|
+
You can run any `pdk` command as described in the PDK documentation. PDQTest
|
266
|
+
does not stop you running anything. If you find this not to be the case please
|
267
|
+
open a [ticket](https://github.com/declarativesystems/pdqtest/issues)
|
268
|
+
|
269
|
+
## How do PDQTest lifecycle tests relate to PDK?
|
270
|
+
|
271
|
+
| PDQTest | PDK |
|
272
|
+
| --- | --- |
|
273
|
+
| `syntax` | `pdk validate metadata,puppet` |
|
274
|
+
| `rspec` | `pdk test unit` |
|
275
|
+
| `build` | `pdk build --force` |
|
276
|
+
|
277
|
+
* Previous `lint` subcommand was removed in PDQTest 2.0 as its now covered by
|
278
|
+
`syntax`
|
279
|
+
|
280
|
+
## What actually happens when I run PDQTest?
|
281
|
+
|
282
|
+
`bundle exec pdqtest all` is the default target executed by `make` and
|
283
|
+
`.\make.ps1`. There are a few other targets that let you skip parts of the build
|
284
|
+
or run individual parts. The complete run looks like this:
|
285
|
+
|
286
|
+
1. `pdk validate 'metadata,puppet'`
|
287
|
+
2. Install modules listed in the `metadata.json` file using R10K against a
|
288
|
+
temporary `Puppetfile` at `Puppetfile.pdqtest`
|
289
|
+
3. Generate a `.fixtures.yml` file based on `metadata.json`
|
290
|
+
Currently impacted by
|
291
|
+
[#47](https://github.com/declarativesystems/pdqtest/issues/47)
|
292
|
+
4. `pdk test unit`
|
293
|
+
5. Run all acceptance tests
|
294
|
+
6. `puppet strings generate --format=markdown` to generate `REFERENCE.md`
|
295
|
+
7. `pdk build --force` to generate your forge package
|
296
|
+
|
297
|
+
At each stage of the process, we output emoji's to keep you informed of
|
298
|
+
progress. Any failure prevents running the next phase of testing and lint errors
|
299
|
+
are considered failures.
|
300
|
+
|
301
|
+
### Technical details (code)
|
302
|
+
If anyone knows a better way to do this, I'd love to hear about it:
|
303
|
+
[PDQTest lifecycle](https://github.com/declarativesystems/pdqtest/blob/master/exe/pdqtest)
|
304
|
+
[PDK Wrapper](https://github.com/declarativesystems/pdqtest/blob/master/lib/pdqtest/pdk.rb)
|
305
|
+
|
306
|
+
## PDQTest is pretty slow!
|
307
|
+
This is a side effect of having to shell out to run PDK via system calls.
|
308
|
+
There's really no other way to do this while maintaining full PDK compatiblity
|
309
|
+
(if you know different, please enlighten me).
|
310
|
+
|
311
|
+
That said you might just want to run syntax and lint tests and acceptance tests
|
312
|
+
as quick as possible, in which case run:
|
313
|
+
|
314
|
+
**Linux**
|
315
|
+
```
|
316
|
+
make fast
|
317
|
+
```
|
318
|
+
|
319
|
+
**Windows**
|
320
|
+
```json
|
321
|
+
.\make.ps1 fast
|
322
|
+
```
|
323
|
+
|
324
|
+
This runs the syntax and lint tests using the original `puppet-syntax` and
|
325
|
+
`puppet-lint` libraries. We can't guarantee PDK identical behaviour or
|
326
|
+
compatibility when used this way but hey... its faster.
|
327
|
+
|
328
|
+
## How do I automatically fix up my lint errors?
|
329
|
+
|
330
|
+
With PDK!:
|
331
|
+
|
332
|
+
```shell
|
333
|
+
pdk validate -a
|
334
|
+
```
|
data/doc/puppet_facts.md
CHANGED
@@ -1,10 +1,52 @@
|
|
1
1
|
# Puppet facts
|
2
|
-
PDQTest
|
2
|
+
PDQTest uses PDK's `spec/default_facts.yml` to add facts to acceptance tests as
|
3
|
+
external facts.
|
4
|
+
|
5
|
+
During test setup, PDQTest will installing the file at one of:
|
6
|
+
* `/etc/puppetlabs/facter/facts.d/default_facts.yaml`
|
7
|
+
* `C:\ProgramData\PuppetLabs\facter\facts.d`
|
8
|
+
|
9
|
+
PDK uses the file as part of the `pdk test unit` command. This way you have a
|
10
|
+
consistent set of facts for both RSpec and acceptance testing, in the file that
|
11
|
+
PDK expects them to be in.
|
12
|
+
|
13
|
+
## 🐉 Differences from PDK 🐉
|
14
|
+
The above behaviour is not how PDK intends this file to be used as some other
|
15
|
+
mechanism is supposed to be used for acceptance tests (see PDK manual).
|
16
|
+
|
17
|
+
It's pretty rare to actually need `spec/default_facts.yml` unless your doing
|
18
|
+
unit testing since you can insert custom facts as part of your test setup.
|
19
|
+
|
20
|
+
Be aware that PDK sets a few facts to specific values that we will pick up and
|
21
|
+
use in your acceptance tests:
|
22
|
+
|
23
|
+
```yaml
|
24
|
+
concat_basedir: "/tmp"
|
25
|
+
ipaddress: "172.16.254.254"
|
26
|
+
is_pe: false
|
27
|
+
macaddress: "AA:AA:AA:AA:AA:AA"
|
28
|
+
```
|
29
|
+
|
30
|
+
You would need to use a forked version of the PDK templates to fix these
|
31
|
+
properly, but your using the structured facts anyway so it makes no difference,
|
32
|
+
right...?
|
33
|
+
|
34
|
+
## Older versions
|
35
|
+
PDQTest < 2.0 includes built-in
|
36
|
+
[factsets](https://github.com/declarativesystems/puppet_factset) covering many
|
37
|
+
common operating systems, however, sometimes additional facts are needed
|
38
|
+
throughout a module to mock an external fact or custom fact from a different
|
39
|
+
module that is expected to be some known value. In this case, creating the
|
40
|
+
directory:
|
3
41
|
|
4
42
|
```
|
5
43
|
spec/merge_facts
|
6
44
|
```
|
7
45
|
|
8
|
-
Inside the module being tested will result in all JSON files contained being
|
46
|
+
Inside the module being tested will result in all JSON files contained being
|
47
|
+
merged into all OS specific factsets used for testing as generated by
|
48
|
+
`pdqtest generate_rspec`.
|
9
49
|
|
10
|
-
To put it simply, drop `.json` file(s) in this directory to add them to every
|
50
|
+
To put it simply, drop `.json` file(s) in this directory to add them to every
|
51
|
+
factset. These facts will be made automatically avaialable to facter when
|
52
|
+
running acceptance testing
|
@@ -1,29 +1,47 @@
|
|
1
1
|
# Puppet module dependencies
|
2
|
-
To test your puppet code, PDQTest needs to configured to obtain any modules the
|
2
|
+
To test your puppet code, PDQTest needs to configured to obtain any modules the
|
3
|
+
code being tested depends on.
|
3
4
|
|
4
5
|
## Specifying dependencies
|
5
6
|
|
6
7
|
### Public modules (PuppetForge)
|
7
|
-
Dependencies on public forge modules must be specified in your module's
|
8
|
+
Dependencies on public forge modules must be specified in your module's
|
9
|
+
`metadata.json` file. When tests are run, we will use this to generate a
|
10
|
+
temporary `Puppetfile` at `Puppetfile.pdqtest` which we will then install into
|
11
|
+
`spec/fixtures/modules` using R10K.
|
12
|
+
|
13
|
+
When RSpec tests are run using PDQtest a `.fixtures.yml` file will be generated
|
14
|
+
for you based on the module metadata (with PDK standalone you must update this
|
15
|
+
file manually). We will then execute `pdk test unit` on your behalf which will
|
16
|
+
install any additional modules from git that can't be specified in
|
17
|
+
`metadata.json`.
|
18
|
+
|
19
|
+
This means you get a complete set of modules in `/spec/fixtures/modules` by the
|
20
|
+
time you come to run acceptance tests.
|
8
21
|
|
9
22
|
### Private modules (from git)
|
10
|
-
If you need to download modules from git, then you must populate the `fixtures`
|
23
|
+
If you need to download modules from git, then you must populate the `fixtures`
|
24
|
+
section of `.fixtures.yml`, eg:
|
11
25
|
|
12
26
|
```
|
13
|
-
|
14
|
-
|
15
|
-
|
16
|
-
|
27
|
+
fixtures:
|
28
|
+
repositories:
|
29
|
+
camera_shy:
|
30
|
+
repo: "git://git.megacorp.com/puppet/camera_shy"
|
31
|
+
ref: "2.6.0"
|
17
32
|
```
|
18
33
|
|
19
|
-
|
20
|
-
|
21
|
-
|
34
|
+
It is an error to define the same module in both `metadata.json` and
|
35
|
+
`.fixtures.yml` and the results of doing this are undefined:
|
36
|
+
* Always use `metadata.json` for public forge modules and we will update
|
37
|
+
`.fixtures.yml` based on it
|
38
|
+
* Modules from git should _only_ be defined in `.fixtures.yml`
|
22
39
|
|
23
40
|
## Notes:
|
24
|
-
* The
|
25
|
-
|
26
|
-
*
|
27
|
-
|
28
|
-
|
29
|
-
`.r10k.yaml` any more. You should remove any `.r10k.yaml` files from your
|
41
|
+
* The file format of `.fixtures.yml` and `fixtures.yml` for specifing git
|
42
|
+
repositories is identical
|
43
|
+
* Previous versions of pdqtest configured the r10k cache via `.r10k.yaml` which
|
44
|
+
caused [#44](https://github.com/declarativesystems/pdqtest/issues/44). To fix
|
45
|
+
this, we now use the default r10k cache dir at `~/.r10k/cache` and don't write
|
46
|
+
`.r10k.yaml` any more. You should remove any `.r10k.yaml` files from your
|
47
|
+
project unless you need it for something specific.
|
data/doc/running_tests.md
CHANGED
@@ -1,38 +1,58 @@
|
|
1
1
|
# Running tests
|
2
|
+
|
3
|
+
## Important
|
4
|
+
You **must** use the provided launch scripts `Makefile` or `make.ps1` to run
|
5
|
+
PDQTest, at least for the first time. See [PDK integration](pdk.md) for more
|
6
|
+
information.
|
7
|
+
|
8
|
+
## Quickstart
|
2
9
|
If you just want to run all tests:
|
3
10
|
|
11
|
+
**Linux**
|
4
12
|
```shell
|
5
13
|
make
|
6
14
|
```
|
7
15
|
|
8
|
-
|
9
|
-
|
10
|
-
## All tests (excludes documentation)
|
11
|
-
|
16
|
+
**Windows**
|
12
17
|
```shell
|
13
|
-
|
18
|
+
.\make.ps1
|
14
19
|
```
|
15
20
|
|
16
|
-
|
21
|
+
Alternatively, you can choose to run different groups of tests by supplying a
|
22
|
+
target from this table:
|
17
23
|
|
18
|
-
|
19
|
-
|
20
|
-
|
24
|
+
| Target | Description | PDK compatible? |
|
25
|
+
| --- | --- | --- |
|
26
|
+
| all | lint, syntax, rspec, acceptance, strings, build | yes |
|
27
|
+
| fast | lint, syntax, acceptance, strings, build | no |
|
28
|
+
| shell | run acceptance tests and print command to get a shell in the container | yes |
|
29
|
+
| shellnopuppet | open a shell in the test container | yes |
|
30
|
+
| logical | syntax, lint | yes |
|
31
|
+
|
32
|
+
* PDK compatible means we run the `pdk` command for this part of the lifecycle
|
21
33
|
|
22
|
-
### Lint
|
23
34
|
|
35
|
+
**Example**
|
36
|
+
|
37
|
+
**Linux**
|
24
38
|
```shell
|
25
|
-
|
39
|
+
make fast
|
26
40
|
```
|
27
41
|
|
28
|
-
|
42
|
+
**Windows**
|
43
|
+
```shell
|
44
|
+
.\make.ps1 fast
|
45
|
+
```
|
29
46
|
|
47
|
+
## Only run acceptance tests
|
30
48
|
```shell
|
31
|
-
bundle exec pdqtest
|
49
|
+
bundle exec pdqtest acceptance
|
32
50
|
```
|
33
51
|
|
34
|
-
### Acceptance
|
35
52
|
|
53
|
+
## See also
|
54
|
+
|
55
|
+
PDQTest help:
|
36
56
|
```shell
|
37
|
-
bundle exec pdqtest
|
57
|
+
bundle exec pdqtest --help
|
38
58
|
```
|
data/doc/test_generation.md
CHANGED
@@ -1,25 +1,12 @@
|
|
1
1
|
# Test Generation
|
2
|
-
Creating a bunch of files manually is an error prone and tedious operation so
|
3
|
-
|
4
|
-
|
5
|
-
## RSpec tests
|
6
|
-
The skeleton tests created by `pdqtest init` only cover the `init.pp` file which is useful but your likely going to need to support more classes as your modules grow. PDQTest can generate basic RSpec testcases for each new puppet **class** that exists in the manifests directory for you:
|
7
|
-
|
8
|
-
```shell
|
9
|
-
bundle exec pdqtest generate_rspec
|
10
|
-
```
|
11
|
-
|
12
|
-
* For every `.pp` file containing a puppet class under `/manifests`, RSpec will be generated to:
|
13
|
-
* Check the catalogue compiles
|
14
|
-
* Check the catalogue contains an instance of the class
|
15
|
-
|
16
|
-
This gives developers an easy place to start writing additional RSpec tests for specific behaviour
|
17
|
-
|
18
|
-
Its safe to run this command whenever you add a new class, it won't overwrite any existing RSpec testcases
|
2
|
+
Creating a bunch of files manually is an error prone and tedious operation so
|
3
|
+
PDQTest can generate files and boilerplate code for you so that your up and
|
4
|
+
running in the quickest time possible.
|
19
5
|
|
20
6
|
## Acceptance tests
|
21
7
|
|
22
|
-
Generate boilerplate files for each example found (including those without a
|
8
|
+
Generate boilerplate files for each example found (including those without a
|
9
|
+
magic marker):
|
23
10
|
|
24
11
|
```shell
|
25
12
|
pdqtest generate_acceptance
|
@@ -31,4 +18,9 @@ Generate boilerplate files for one specific example:
|
|
31
18
|
pdqtest generate_acceptance examples/mynewthing.pp
|
32
19
|
```
|
33
20
|
|
34
|
-
|
21
|
+
This will also create examples/mynewthing.pp if you haven't created it yet.
|
22
|
+
|
23
|
+
|
24
|
+
## RSpec tests
|
25
|
+
PDQTest < 2.0 includes rspec test generation. This functionality is replaced by
|
26
|
+
PDK in later versions.
|
data/doc/tips_and_tricks.md
CHANGED
@@ -1,13 +1,22 @@
|
|
1
1
|
# Tips and tricks
|
2
|
-
* You can put any puppet code you like (including an empty file...) in each of
|
3
|
-
|
4
|
-
*
|
5
|
-
|
6
|
-
|
2
|
+
* You can put any puppet code you like (including an empty file...) in each of
|
3
|
+
the files under `/examples` and it will executed with `puppet apply`
|
4
|
+
* If you need to test multiple different things (eg different parameters for a
|
5
|
+
new type and provider), you need to create a different (acceptance) testcase
|
6
|
+
for each distinct thing to test
|
7
|
+
* PDQTest will only execute the BATS tests and setup scripts that are present,
|
8
|
+
you can delete some or all of these files if some steps aren't required.
|
9
|
+
* If no files are present under `spec/acceptance` for a given example, then
|
10
|
+
PDQTest will just check that puppet runs idempotently for your example
|
11
|
+
* To disable tests temporarily for a specific example, remove the magic marker
|
12
|
+
`#@PDQTest` or `#@PDQTestWin` from the example
|
7
13
|
* Nested examples (subdirectories) are not supported at this time
|
8
|
-
* Since the all of the `*__setup.sh` scripts are run in the container as root
|
14
|
+
* Since the all of the `*__setup.sh` scripts are run in the container as root
|
15
|
+
before executing tests, they can be used to mock _almost anything_ in the
|
16
|
+
test system:
|
9
17
|
* Replace system binaries to fake network operations
|
10
|
-
* Add system binaries to simulate other operating systems such as AIX,
|
18
|
+
* Add system binaries to simulate other operating systems such as AIX,
|
19
|
+
Solaris, etc
|
11
20
|
* Create/copy files, directories, etc.
|
12
21
|
* Install OS packages
|
13
|
-
* Install python scripts to mock database servers using SQLite...
|
22
|
+
* Install python scripts to mock database servers using SQLite... 😉
|
data/doc/troubleshooting.md
CHANGED
@@ -1,9 +1,20 @@
|
|
1
1
|
# Troubleshooting
|
2
|
-
* If you can't find the `pdqtest` command and your using `rbenv` be sure to
|
3
|
-
|
4
|
-
*
|
5
|
-
|
6
|
-
|
7
|
-
|
8
|
-
*
|
9
|
-
|
2
|
+
* If you can't find the `pdqtest` command and your using `rbenv` be sure to
|
3
|
+
run `rbenv rehash` after installing the gem to create the necessary symlinks
|
4
|
+
* If your `pdqtest` command changes version randomly depending which directory
|
5
|
+
your in and you are using `rvm` its probably because `rvm` overrides `cd` and
|
6
|
+
does strange things. You can probably turn this off. Alternatively, use
|
7
|
+
`rbenv`
|
8
|
+
* Don't forget to run `pdqtest setup` before your first `pdqtest` run to
|
9
|
+
download/update the Docker image
|
10
|
+
* If you need to access private git repositories, make sure to use
|
11
|
+
`.fixtures.yml` not `fixtures.yml` (changed in 2.0.0)
|
12
|
+
* If you need a private key to access private repositories, set this up for your
|
13
|
+
regular git command/ssh and `pdqtest` will reuse the settings
|
14
|
+
* Be sure to annotate the examples you wish to acceptance test with the magic
|
15
|
+
marker comment `#@PDQTest` or `#@PDQTestWin`
|
16
|
+
* Be sure to run `make` or `bundle exec pdqtest all` to download dependencies
|
17
|
+
when running acceptance tests. Previous versions (re)downloaded modules as
|
18
|
+
required from inside docker but this step has been replaced with a simple
|
19
|
+
symlink to reduce the amount of downloading so the modules must already be
|
20
|
+
present.
|