pdqtest 1.4.1 → 1.9.9beta2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.travis.yml +9 -3
- data/doc/acceptance_tests.md +100 -22
- data/doc/caching.md +5 -1
- data/doc/development.md +16 -5
- data/doc/emoji.md +20 -9
- data/doc/enabling_testing.md +15 -3
- data/doc/examples.md +25 -6
- data/doc/hiera.md +30 -11
- data/doc/installation.md +59 -8
- data/doc/pdk.md +334 -0
- data/doc/puppet_facts.md +45 -3
- data/doc/puppet_module_dependencies.md +34 -16
- data/doc/running_tests.md +35 -15
- data/doc/test_generation.md +11 -19
- data/doc/tips_and_tricks.md +17 -8
- data/doc/troubleshooting.md +19 -8
- data/doc/upgrading.md +125 -6
- data/doc/windows.md +51 -0
- data/docker_images/centos/Dockerfile +4 -3
- data/docker_images/centos/Makefile +1 -1
- data/docker_images/ubuntu/Dockerfile +3 -3
- data/docker_images/windows/Dockerfile +26 -0
- data/docker_images/windows/make.ps1 +2 -0
- data/exe/pdqtest +73 -28
- data/lib/pdqtest/core.rb +1 -1
- data/lib/pdqtest/docker.rb +143 -51
- data/lib/pdqtest/emoji.rb +33 -7
- data/lib/pdqtest/fastcheck.rb +19 -0
- data/lib/pdqtest/instance.rb +16 -15
- data/lib/pdqtest/logger.rb +35 -0
- data/lib/pdqtest/pdk.rb +98 -0
- data/lib/pdqtest/pdqtest1x.rb +115 -0
- data/lib/pdqtest/puppet.rb +277 -134
- data/lib/pdqtest/skeleton.rb +119 -39
- data/lib/pdqtest/upgrade.rb +42 -42
- data/lib/pdqtest/util.rb +82 -2
- data/lib/pdqtest/version.rb +1 -2
- data/pdqtest.gemspec +8 -13
- data/res/{skeleton → acceptance}/init.bats +0 -0
- data/res/{skeleton → acceptance}/init__before.bats +0 -0
- data/res/{skeleton → acceptance}/init__setup.sh +0 -0
- data/res/skeleton/.puppet-lint.rc +5 -0
- data/res/skeleton/{dot_travis.yml → .travis.yml} +0 -0
- data/res/skeleton/Makefile +20 -5
- data/res/skeleton/make.ps1 +66 -0
- data/res/skeleton/spec/fixtures/hiera.yaml +13 -0
- data/res/skeleton/spec/fixtures/hieradata/test.yaml +11 -0
- data/res/templates/examples_init.pp.erb +1 -1
- metadata +47 -115
- data/lib/pdqtest/lint.rb +0 -31
- data/lib/pdqtest/rspec.rb +0 -69
- data/lib/pdqtest/syntax.rb +0 -20
- data/res/skeleton/Gemfile +0 -7
- data/res/skeleton/Rakefile +0 -3
- data/res/skeleton/dot_gitignore +0 -9
- data/res/skeleton/dot_rspec +0 -2
- data/res/skeleton/hiera.yaml +0 -7
- data/res/skeleton/spec_helper.rb +0 -4
- data/res/skeleton/test.yaml +0 -3
data/doc/pdk.md
ADDED
@@ -0,0 +1,334 @@
|
|
1
|
+
# PDK Integration
|
2
|
+
|
3
|
+
First off, what is PDK? PDK is Puppet's development kit. The bits we are
|
4
|
+
interested in right now are:
|
5
|
+
* Validation `pdk validate...`
|
6
|
+
* Testing `pdk test...`
|
7
|
+
* Packaging `pdk build...`
|
8
|
+
|
9
|
+
There's a lot more to PDK then just running simple tests though, PDK includes
|
10
|
+
its own version of ruby and various other tools.
|
11
|
+
|
12
|
+
On Linux PDK behaves (or at least appears to behave...) like a thin wrapper
|
13
|
+
around ruby, in the same vein as RBENV/RVM. On Windows, things are more involved
|
14
|
+
since it needs to install specific system libraries and reference them at
|
15
|
+
runtime somehow. This is why running `bundle install` on PDK project in windows
|
16
|
+
can cause strange and interesting blow-ups (don't do that!).
|
17
|
+
|
18
|
+
## What's the point of PDQTest if we have PDK?
|
19
|
+
PDQTest provides an easy way to run acceptance tests using light-weight
|
20
|
+
Docker containers which we abuse to the point of treating them _almost_ like
|
21
|
+
VMs in order to extract maximum speed. It offers a different approach to
|
22
|
+
[Beaker](https://github.com/puppetlabs/beaker):
|
23
|
+
* Minimal to no project configuration required, just `metadata.json` and a few
|
24
|
+
other files we setup for you
|
25
|
+
* Simple acceptance tests written in BASH or PowerShell
|
26
|
+
* Provide _good enough_ (but not perfect) testing:
|
27
|
+
* Generic Ubuntu, Centos, Windows test environments
|
28
|
+
* Capable of mocking large/slow/networked systems by replacing or installing
|
29
|
+
binaries inside the container at the OS level, eg:
|
30
|
+
* Fake installation of Database server's by mocking the installer your
|
31
|
+
puppet `exec` would call with a simple python script
|
32
|
+
* Network tools such as `realmd` with a simple mock `realm` command
|
33
|
+
written in Python
|
34
|
+
* Complex OS commands that need to report different state between puppet
|
35
|
+
runs can be mocked with Python and a small SQLite database or simple
|
36
|
+
text files (eg the `sysctl` command)
|
37
|
+
* Fake different Unix-like Operating Systems like AIX by subverting the
|
38
|
+
commands that your puppet code would executes. This is particularly
|
39
|
+
useful in this case since there are no AIX desktop virtualisation
|
40
|
+
solutions right now and there probably never will be
|
41
|
+
* See [examples](examples.md) for real-world modules using these
|
42
|
+
techniques
|
43
|
+
|
44
|
+
Of course, this will only ever give you an approximation of a real-world system
|
45
|
+
but in many cases this is all that's needed. The complexity of setting up a
|
46
|
+
100% accurate test environment is a daunting task and one that may never provide
|
47
|
+
ROI on the effort spent configuring and maintaining it.
|
48
|
+
|
49
|
+
If perfect accuracy is indeed required, then PDQTest is not what you are looking
|
50
|
+
for and you should look towards solutions like Beaker or
|
51
|
+
[Kitchen CI](https://kitchen.ci/).
|
52
|
+
|
53
|
+
## How does PDQTest work with PDK? Do I have to choose?
|
54
|
+
|
55
|
+
No you don't have to choose! PDQTest integrates with PDK by making the minimal
|
56
|
+
amount of invasive changes:
|
57
|
+
|
58
|
+
### New projects
|
59
|
+
New projects should be created using PDK: `pdk new module`.
|
60
|
+
|
61
|
+
This will give you all the PDK versioned files you would ever need. When you run
|
62
|
+
`pdqtest init` on a new project created with PDK, we only copy in our own
|
63
|
+
integration files and do not not touch the PDK generated files at all (the ones
|
64
|
+
marked 🛠 below),
|
65
|
+
|
66
|
+
### Existing PDQTest projects
|
67
|
+
To make upgrading to PDQTest 2.0 + PDK easy for our existing users, we automate
|
68
|
+
the workflow normally carried out by `pdk convert`:
|
69
|
+
|
70
|
+
We do not include the entire set of files that would have been generated by
|
71
|
+
`pdk convert` or `pdk new module`. If you want the full set, you should run
|
72
|
+
one of these commands yourself before `pdqtest init` and they will be left alone
|
73
|
+
unless PDK starts writing new files that we are already using.
|
74
|
+
|
75
|
+
**metadata.json**
|
76
|
+
|
77
|
+
* Enable PDK by adding the fields:
|
78
|
+
* `pdk-version`
|
79
|
+
* `template-url`
|
80
|
+
* `template-ref`
|
81
|
+
The values for these fields are obtained automatically from PDK itself and are
|
82
|
+
consistent with the system installed PDK.
|
83
|
+
|
84
|
+
**Gemfile.project**
|
85
|
+
|
86
|
+
* Add a reference to the `pdqtest` gem
|
87
|
+
|
88
|
+
**Miscellaneous skeletons**
|
89
|
+
|
90
|
+
When `pdqtest init` is run, we install a small set of skeleton files in the root
|
91
|
+
of your project. This is a one-off operation to get you started. After your
|
92
|
+
module has been marked as PDK compatible, we no longer attempt to update these
|
93
|
+
files and you must only manage them using PDK from then-on.
|
94
|
+
|
95
|
+
Skeletons are generated on the fly using PDK itself, so they are always
|
96
|
+
up-to-date according to the PDK you have installed on your system.
|
97
|
+
|
98
|
+
### Existing non-PDQTest projects
|
99
|
+
If you have an existing Puppet project that does not use PDQTest or PDK (eg
|
100
|
+
created by hand or with the old `puppet module generate` command), then the
|
101
|
+
recommended way to enable PDQTest is to first enable PDK by running
|
102
|
+
`pdk convert` and following the wizard.
|
103
|
+
|
104
|
+
You may then enable `PDQTest` by running `pdqtest init`.
|
105
|
+
|
106
|
+
Depending on the state of your project, you may be able to bypass the
|
107
|
+
`pdk convert` process by just running `pdqtest init` but this isn't recommended
|
108
|
+
or supported.
|
109
|
+
|
110
|
+
|
111
|
+
### PDQTest directory structure
|
112
|
+
These are the files installed, files marked 🛠 are generated by PDK:
|
113
|
+
|
114
|
+
```
|
115
|
+
├── bitbucket-pipelines.yml
|
116
|
+
├── Gemfile 🛠
|
117
|
+
├── Gemfile.local
|
118
|
+
├── Gemfile.project
|
119
|
+
├── .gitignore 🛠
|
120
|
+
├── Makefile
|
121
|
+
├── make.ps1
|
122
|
+
├── .pdkignore 🛠
|
123
|
+
├── .puppet-lint.rc
|
124
|
+
├── Rakefile 🛠
|
125
|
+
├── spec
|
126
|
+
│ ├── default_facts.yml 🛠
|
127
|
+
│ ├── fixtures
|
128
|
+
│ │ ├── hieradata
|
129
|
+
│ │ │ └── test.yaml
|
130
|
+
│ │ └── hiera.yaml
|
131
|
+
│ └── spec_helper.rb 🛠
|
132
|
+
└── .travis.yml
|
133
|
+
```
|
134
|
+
|
135
|
+
The remaining files are specific to PDQTest - notably:
|
136
|
+
* `.travis.yml` - Complete test suite for Linux modules
|
137
|
+
* `bitbucket-pipelines.yaml` - RSpec testing only
|
138
|
+
* `.puppet-lint.rc` - Make lint errors test failures, ignore double quotes, etc
|
139
|
+
* `Makefile` - Essential launch script for Linux
|
140
|
+
* `make.ps1` - Essential launch script for Windows
|
141
|
+
* `spec/fixtures/hiera.yaml` - Mock system-wide `hiera.yaml` file
|
142
|
+
* `spec/fixtures/hieradata/test.yaml` - Mock system-wide hieradata
|
143
|
+
* `Gemfile.project` - Used to enable the PDQTest gem (and any others you need)
|
144
|
+
* `Gemfile.local` - Transient file created at runtime by `Makefile` or
|
145
|
+
`make.ps1` to enable the PDQTest gem at runtime. This is a vital integration
|
146
|
+
point, see details below
|
147
|
+
* `.gitignore` - PDK has a massive list of files to ignore for new projects so
|
148
|
+
these are imported
|
149
|
+
|
150
|
+
|
151
|
+
## Why are the launch scripts essential/How does the PDQTest gem load itself?
|
152
|
+
PDK provides its own `Gemfile` to select the gems available at runtime. There
|
153
|
+
are hooks in this file to load additional configuration from two locations:
|
154
|
+
* `~/.gemfile`
|
155
|
+
* `YOUR_PROJECT/Gemfile.local`
|
156
|
+
|
157
|
+
We support side-by-side installation of all versions of PDQTest so that users
|
158
|
+
are not forced to upgrade when they are not ready to, so we need a per-project
|
159
|
+
way to load our gem into the bundle. `Gemfile.local` is perfect for this but we
|
160
|
+
are not supposed to store permanent data in there according to PDK's generated
|
161
|
+
`.gitignore`.
|
162
|
+
|
163
|
+
The simplest way to workaround this issue is for us to create a symlink from
|
164
|
+
`Gemfile.project` to `Gemfile.local` at runtime and we _must_ use our launch
|
165
|
+
scripts to do this since we are unable to use any Ruby code to do this trick
|
166
|
+
since we are not yet in the bundle.
|
167
|
+
|
168
|
+
For this reason, you **must** launch PDQTest using the provided `Makefile` or
|
169
|
+
`make.ps1` scripts, at least initially.
|
170
|
+
|
171
|
+
For further background see:
|
172
|
+
* [PDK-1177](https://tickets.puppetlabs.com/browse/PDK-1177) and
|
173
|
+
* [#50](https://github.com/declarativesystems/pdqtest/issues/50)
|
174
|
+
|
175
|
+
## When developing PDQTest, your version number **must** match a released version
|
176
|
+
_This should only affects PDQTest developers - for posterity..._
|
177
|
+
|
178
|
+
Due to some strange hackery between `pdk bundle` and `bundle exec` you have one
|
179
|
+
more gotcha when it comes to using developing PDQTest and thats that the version
|
180
|
+
number of the gem you build *MUST* match a released version of the gem on
|
181
|
+
rubygems.org otherwise `bundle exec` will barf with errors like this:
|
182
|
+
|
183
|
+
```shell
|
184
|
+
$ bundle exec pdqtest
|
185
|
+
Could not find hiera-3.4.5 in any of the sources
|
186
|
+
Run `bundle install` to install missing gems.
|
187
|
+
```
|
188
|
+
|
189
|
+
There seems no other way to fix this other than building the development gem
|
190
|
+
with a version matching an existing release. The gem then needs to be installed
|
191
|
+
using `gem install ...` and it will then magically start working. Using other
|
192
|
+
gem tricks like setting `:path` will appear to work but fail when `bundle exec`
|
193
|
+
is called. The other approach of trying to run everything with
|
194
|
+
`pdk bundle exec pdqtest` fails with various errors depending what platform your
|
195
|
+
on, presumably because `pdqtest` runs `pdk` command as part of its tests.
|
196
|
+
|
197
|
+
The final final caveat of this approach is that the PDQTest gem dependencies are
|
198
|
+
pinned at whatever the _real_ release uses, so if any were changed or added they
|
199
|
+
need to be temporarily added to `Gemfile.project`
|
200
|
+
|
201
|
+
The above works on Linux... For windows there is also the situation where:
|
202
|
+
* PDK vendors its own gems that magically appear
|
203
|
+
* PDK can only use "published" gems(specs?) it downloads - or appears to
|
204
|
+
* when you go to `bundle install` without using PDK you blow up the lock
|
205
|
+
file
|
206
|
+
* When you `bundle exec pdqtest` you get the above error because _something_
|
207
|
+
uses the old one `gemspec`
|
208
|
+
* The answer seems to be:
|
209
|
+
1. use `gem 'pdqtest', :path=>'c:/vagrant/pdqtest'` in `Gemfile.local`
|
210
|
+
2. `pdk bundle install`
|
211
|
+
3. `bundle update pdqtest` - yes even though your not supposed to...
|
212
|
+
4. `bundle exec pdqtest` - now it should work... why is beyond me
|
213
|
+
* If you still have errors:
|
214
|
+
1. `bundle install`
|
215
|
+
2. `rm Gemfile.lock`
|
216
|
+
3. `pdk bundle install` - working?
|
217
|
+
|
218
|
+
Somtimes you just have to upload the whole gem file as a pre-release object:
|
219
|
+
* Version set to x.x.xWHATEVER
|
220
|
+
* `gem push pdqtest-x.x.xWHATEVER.gem`
|
221
|
+
* `gem install pdqtest --pre`
|
222
|
+
|
223
|
+
|
224
|
+
If your just want to use PDQTest, you will hopefully never have to worry about
|
225
|
+
this... On the other hand, if this all sounds confusing... That's because it is!
|
226
|
+
|
227
|
+
## What happens when I upgrade PDQTest?
|
228
|
+
When a new version of PDQTest is released, upgrade by running:
|
229
|
+
|
230
|
+
```shell
|
231
|
+
gem install pdqtest
|
232
|
+
cd /your/project
|
233
|
+
pdqtest upgrade
|
234
|
+
pdk bundle install
|
235
|
+
```
|
236
|
+
|
237
|
+
This updates:
|
238
|
+
* Project gems in `Gemfile.project` (currently `pdqtest` and `puppet-strings`)
|
239
|
+
* Our CI and CLI integrations:
|
240
|
+
* `Makefile`
|
241
|
+
* `make.ps1`
|
242
|
+
* `.travis.yml`
|
243
|
+
* `bitbucket-pipelines.yml`
|
244
|
+
|
245
|
+
The final `pdk bundle install` command upgrades PDK's bundle to the include our
|
246
|
+
update and is critical to making the whole process work.
|
247
|
+
|
248
|
+
## What happens when I upgrade PDK?
|
249
|
+
PDK operates independently from PDQTest and maintains its own files. Your free
|
250
|
+
to run `pdk update` to upgrade the your files to the latest PDK templated ones
|
251
|
+
at any time.
|
252
|
+
|
253
|
+
To protect PDQTest files from alteration (notably `.travis.yml` and
|
254
|
+
`bitbucket-pipelines.yml`) we merge them with any existing `.sync.yml` to
|
255
|
+
prevent churn.
|
256
|
+
|
257
|
+
If you have further customisations to PDK controlled files your options are:
|
258
|
+
* Use git to revert any change by PDK
|
259
|
+
* Use `.sync.yml` to influence file (re)generation
|
260
|
+
[blog](https://puppet.com/blog/guide-converting-module-pdk)
|
261
|
+
[reference](https://github.com/puppetlabs/pdk-templates)
|
262
|
+
[example](https://github.com/puppetlabs/puppetlabs-apt/blob/master/.sync.yml)
|
263
|
+
|
264
|
+
## How do I use PDK with PDQTest installed? What can and can't I do?
|
265
|
+
You can run any `pdk` command as described in the PDK documentation. PDQTest
|
266
|
+
does not stop you running anything. If you find this not to be the case please
|
267
|
+
open a [ticket](https://github.com/declarativesystems/pdqtest/issues)
|
268
|
+
|
269
|
+
## How do PDQTest lifecycle tests relate to PDK?
|
270
|
+
|
271
|
+
| PDQTest | PDK |
|
272
|
+
| --- | --- |
|
273
|
+
| `syntax` | `pdk validate metadata,puppet` |
|
274
|
+
| `rspec` | `pdk test unit` |
|
275
|
+
| `build` | `pdk build --force` |
|
276
|
+
|
277
|
+
* Previous `lint` subcommand was removed in PDQTest 2.0 as its now covered by
|
278
|
+
`syntax`
|
279
|
+
|
280
|
+
## What actually happens when I run PDQTest?
|
281
|
+
|
282
|
+
`bundle exec pdqtest all` is the default target executed by `make` and
|
283
|
+
`.\make.ps1`. There are a few other targets that let you skip parts of the build
|
284
|
+
or run individual parts. The complete run looks like this:
|
285
|
+
|
286
|
+
1. `pdk validate 'metadata,puppet'`
|
287
|
+
2. Install modules listed in the `metadata.json` file using R10K against a
|
288
|
+
temporary `Puppetfile` at `Puppetfile.pdqtest`
|
289
|
+
3. Generate a `.fixtures.yml` file based on `metadata.json`
|
290
|
+
Currently impacted by
|
291
|
+
[#47](https://github.com/declarativesystems/pdqtest/issues/47)
|
292
|
+
4. `pdk test unit`
|
293
|
+
5. Run all acceptance tests
|
294
|
+
6. `puppet strings generate --format=markdown` to generate `REFERENCE.md`
|
295
|
+
7. `pdk build --force` to generate your forge package
|
296
|
+
|
297
|
+
At each stage of the process, we output emoji's to keep you informed of
|
298
|
+
progress. Any failure prevents running the next phase of testing and lint errors
|
299
|
+
are considered failures.
|
300
|
+
|
301
|
+
### Technical details (code)
|
302
|
+
If anyone knows a better way to do this, I'd love to hear about it:
|
303
|
+
[PDQTest lifecycle](https://github.com/declarativesystems/pdqtest/blob/master/exe/pdqtest)
|
304
|
+
[PDK Wrapper](https://github.com/declarativesystems/pdqtest/blob/master/lib/pdqtest/pdk.rb)
|
305
|
+
|
306
|
+
## PDQTest is pretty slow!
|
307
|
+
This is a side effect of having to shell out to run PDK via system calls.
|
308
|
+
There's really no other way to do this while maintaining full PDK compatiblity
|
309
|
+
(if you know different, please enlighten me).
|
310
|
+
|
311
|
+
That said you might just want to run syntax and lint tests and acceptance tests
|
312
|
+
as quick as possible, in which case run:
|
313
|
+
|
314
|
+
**Linux**
|
315
|
+
```
|
316
|
+
make fast
|
317
|
+
```
|
318
|
+
|
319
|
+
**Windows**
|
320
|
+
```json
|
321
|
+
.\make.ps1 fast
|
322
|
+
```
|
323
|
+
|
324
|
+
This runs the syntax and lint tests using the original `puppet-syntax` and
|
325
|
+
`puppet-lint` libraries. We can't guarantee PDK identical behaviour or
|
326
|
+
compatibility when used this way but hey... its faster.
|
327
|
+
|
328
|
+
## How do I automatically fix up my lint errors?
|
329
|
+
|
330
|
+
With PDK!:
|
331
|
+
|
332
|
+
```shell
|
333
|
+
pdk validate -a
|
334
|
+
```
|
data/doc/puppet_facts.md
CHANGED
@@ -1,10 +1,52 @@
|
|
1
1
|
# Puppet facts
|
2
|
-
PDQTest
|
2
|
+
PDQTest uses PDK's `spec/default_facts.yml` to add facts to acceptance tests as
|
3
|
+
external facts.
|
4
|
+
|
5
|
+
During test setup, PDQTest will installing the file at one of:
|
6
|
+
* `/etc/puppetlabs/facter/facts.d/default_facts.yaml`
|
7
|
+
* `C:\ProgramData\PuppetLabs\facter\facts.d`
|
8
|
+
|
9
|
+
PDK uses the file as part of the `pdk test unit` command. This way you have a
|
10
|
+
consistent set of facts for both RSpec and acceptance testing, in the file that
|
11
|
+
PDK expects them to be in.
|
12
|
+
|
13
|
+
## 🐉 Differences from PDK 🐉
|
14
|
+
The above behaviour is not how PDK intends this file to be used as some other
|
15
|
+
mechanism is supposed to be used for acceptance tests (see PDK manual).
|
16
|
+
|
17
|
+
It's pretty rare to actually need `spec/default_facts.yml` unless your doing
|
18
|
+
unit testing since you can insert custom facts as part of your test setup.
|
19
|
+
|
20
|
+
Be aware that PDK sets a few facts to specific values that we will pick up and
|
21
|
+
use in your acceptance tests:
|
22
|
+
|
23
|
+
```yaml
|
24
|
+
concat_basedir: "/tmp"
|
25
|
+
ipaddress: "172.16.254.254"
|
26
|
+
is_pe: false
|
27
|
+
macaddress: "AA:AA:AA:AA:AA:AA"
|
28
|
+
```
|
29
|
+
|
30
|
+
You would need to use a forked version of the PDK templates to fix these
|
31
|
+
properly, but your using the structured facts anyway so it makes no difference,
|
32
|
+
right...?
|
33
|
+
|
34
|
+
## Older versions
|
35
|
+
PDQTest < 2.0 includes built-in
|
36
|
+
[factsets](https://github.com/declarativesystems/puppet_factset) covering many
|
37
|
+
common operating systems, however, sometimes additional facts are needed
|
38
|
+
throughout a module to mock an external fact or custom fact from a different
|
39
|
+
module that is expected to be some known value. In this case, creating the
|
40
|
+
directory:
|
3
41
|
|
4
42
|
```
|
5
43
|
spec/merge_facts
|
6
44
|
```
|
7
45
|
|
8
|
-
Inside the module being tested will result in all JSON files contained being
|
46
|
+
Inside the module being tested will result in all JSON files contained being
|
47
|
+
merged into all OS specific factsets used for testing as generated by
|
48
|
+
`pdqtest generate_rspec`.
|
9
49
|
|
10
|
-
To put it simply, drop `.json` file(s) in this directory to add them to every
|
50
|
+
To put it simply, drop `.json` file(s) in this directory to add them to every
|
51
|
+
factset. These facts will be made automatically avaialable to facter when
|
52
|
+
running acceptance testing
|
@@ -1,29 +1,47 @@
|
|
1
1
|
# Puppet module dependencies
|
2
|
-
To test your puppet code, PDQTest needs to configured to obtain any modules the
|
2
|
+
To test your puppet code, PDQTest needs to configured to obtain any modules the
|
3
|
+
code being tested depends on.
|
3
4
|
|
4
5
|
## Specifying dependencies
|
5
6
|
|
6
7
|
### Public modules (PuppetForge)
|
7
|
-
Dependencies on public forge modules must be specified in your module's
|
8
|
+
Dependencies on public forge modules must be specified in your module's
|
9
|
+
`metadata.json` file. When tests are run, we will use this to generate a
|
10
|
+
temporary `Puppetfile` at `Puppetfile.pdqtest` which we will then install into
|
11
|
+
`spec/fixtures/modules` using R10K.
|
12
|
+
|
13
|
+
When RSpec tests are run using PDQtest a `.fixtures.yml` file will be generated
|
14
|
+
for you based on the module metadata (with PDK standalone you must update this
|
15
|
+
file manually). We will then execute `pdk test unit` on your behalf which will
|
16
|
+
install any additional modules from git that can't be specified in
|
17
|
+
`metadata.json`.
|
18
|
+
|
19
|
+
This means you get a complete set of modules in `/spec/fixtures/modules` by the
|
20
|
+
time you come to run acceptance tests.
|
8
21
|
|
9
22
|
### Private modules (from git)
|
10
|
-
If you need to download modules from git, then you must populate the `fixtures`
|
23
|
+
If you need to download modules from git, then you must populate the `fixtures`
|
24
|
+
section of `.fixtures.yml`, eg:
|
11
25
|
|
12
26
|
```
|
13
|
-
|
14
|
-
|
15
|
-
|
16
|
-
|
27
|
+
fixtures:
|
28
|
+
repositories:
|
29
|
+
camera_shy:
|
30
|
+
repo: "git://git.megacorp.com/puppet/camera_shy"
|
31
|
+
ref: "2.6.0"
|
17
32
|
```
|
18
33
|
|
19
|
-
|
20
|
-
|
21
|
-
|
34
|
+
It is an error to define the same module in both `metadata.json` and
|
35
|
+
`.fixtures.yml` and the results of doing this are undefined:
|
36
|
+
* Always use `metadata.json` for public forge modules and we will update
|
37
|
+
`.fixtures.yml` based on it
|
38
|
+
* Modules from git should _only_ be defined in `.fixtures.yml`
|
22
39
|
|
23
40
|
## Notes:
|
24
|
-
* The
|
25
|
-
|
26
|
-
*
|
27
|
-
|
28
|
-
|
29
|
-
`.r10k.yaml` any more. You should remove any `.r10k.yaml` files from your
|
41
|
+
* The file format of `.fixtures.yml` and `fixtures.yml` for specifing git
|
42
|
+
repositories is identical
|
43
|
+
* Previous versions of pdqtest configured the r10k cache via `.r10k.yaml` which
|
44
|
+
caused [#44](https://github.com/declarativesystems/pdqtest/issues/44). To fix
|
45
|
+
this, we now use the default r10k cache dir at `~/.r10k/cache` and don't write
|
46
|
+
`.r10k.yaml` any more. You should remove any `.r10k.yaml` files from your
|
47
|
+
project unless you need it for something specific.
|
data/doc/running_tests.md
CHANGED
@@ -1,38 +1,58 @@
|
|
1
1
|
# Running tests
|
2
|
+
|
3
|
+
## Important
|
4
|
+
You **must** use the provided launch scripts `Makefile` or `make.ps1` to run
|
5
|
+
PDQTest, at least for the first time. See [PDK integration](pdk.md) for more
|
6
|
+
information.
|
7
|
+
|
8
|
+
## Quickstart
|
2
9
|
If you just want to run all tests:
|
3
10
|
|
11
|
+
**Linux**
|
4
12
|
```shell
|
5
13
|
make
|
6
14
|
```
|
7
15
|
|
8
|
-
|
9
|
-
|
10
|
-
## All tests (excludes documentation)
|
11
|
-
|
16
|
+
**Windows**
|
12
17
|
```shell
|
13
|
-
|
18
|
+
.\make.ps1
|
14
19
|
```
|
15
20
|
|
16
|
-
|
21
|
+
Alternatively, you can choose to run different groups of tests by supplying a
|
22
|
+
target from this table:
|
17
23
|
|
18
|
-
|
19
|
-
|
20
|
-
|
24
|
+
| Target | Description | PDK compatible? |
|
25
|
+
| --- | --- | --- |
|
26
|
+
| all | lint, syntax, rspec, acceptance, strings, build | yes |
|
27
|
+
| fast | lint, syntax, acceptance, strings, build | no |
|
28
|
+
| shell | run acceptance tests and print command to get a shell in the container | yes |
|
29
|
+
| shellnopuppet | open a shell in the test container | yes |
|
30
|
+
| logical | syntax, lint | yes |
|
31
|
+
|
32
|
+
* PDK compatible means we run the `pdk` command for this part of the lifecycle
|
21
33
|
|
22
|
-
### Lint
|
23
34
|
|
35
|
+
**Example**
|
36
|
+
|
37
|
+
**Linux**
|
24
38
|
```shell
|
25
|
-
|
39
|
+
make fast
|
26
40
|
```
|
27
41
|
|
28
|
-
|
42
|
+
**Windows**
|
43
|
+
```shell
|
44
|
+
.\make.ps1 fast
|
45
|
+
```
|
29
46
|
|
47
|
+
## Only run acceptance tests
|
30
48
|
```shell
|
31
|
-
bundle exec pdqtest
|
49
|
+
bundle exec pdqtest acceptance
|
32
50
|
```
|
33
51
|
|
34
|
-
### Acceptance
|
35
52
|
|
53
|
+
## See also
|
54
|
+
|
55
|
+
PDQTest help:
|
36
56
|
```shell
|
37
|
-
bundle exec pdqtest
|
57
|
+
bundle exec pdqtest --help
|
38
58
|
```
|
data/doc/test_generation.md
CHANGED
@@ -1,25 +1,12 @@
|
|
1
1
|
# Test Generation
|
2
|
-
Creating a bunch of files manually is an error prone and tedious operation so
|
3
|
-
|
4
|
-
|
5
|
-
## RSpec tests
|
6
|
-
The skeleton tests created by `pdqtest init` only cover the `init.pp` file which is useful but your likely going to need to support more classes as your modules grow. PDQTest can generate basic RSpec testcases for each new puppet **class** that exists in the manifests directory for you:
|
7
|
-
|
8
|
-
```shell
|
9
|
-
bundle exec pdqtest generate_rspec
|
10
|
-
```
|
11
|
-
|
12
|
-
* For every `.pp` file containing a puppet class under `/manifests`, RSpec will be generated to:
|
13
|
-
* Check the catalogue compiles
|
14
|
-
* Check the catalogue contains an instance of the class
|
15
|
-
|
16
|
-
This gives developers an easy place to start writing additional RSpec tests for specific behaviour
|
17
|
-
|
18
|
-
Its safe to run this command whenever you add a new class, it won't overwrite any existing RSpec testcases
|
2
|
+
Creating a bunch of files manually is an error prone and tedious operation so
|
3
|
+
PDQTest can generate files and boilerplate code for you so that your up and
|
4
|
+
running in the quickest time possible.
|
19
5
|
|
20
6
|
## Acceptance tests
|
21
7
|
|
22
|
-
Generate boilerplate files for each example found (including those without a
|
8
|
+
Generate boilerplate files for each example found (including those without a
|
9
|
+
magic marker):
|
23
10
|
|
24
11
|
```shell
|
25
12
|
pdqtest generate_acceptance
|
@@ -31,4 +18,9 @@ Generate boilerplate files for one specific example:
|
|
31
18
|
pdqtest generate_acceptance examples/mynewthing.pp
|
32
19
|
```
|
33
20
|
|
34
|
-
|
21
|
+
This will also create examples/mynewthing.pp if you haven't created it yet.
|
22
|
+
|
23
|
+
|
24
|
+
## RSpec tests
|
25
|
+
PDQTest < 2.0 includes rspec test generation. This functionality is replaced by
|
26
|
+
PDK in later versions.
|
data/doc/tips_and_tricks.md
CHANGED
@@ -1,13 +1,22 @@
|
|
1
1
|
# Tips and tricks
|
2
|
-
* You can put any puppet code you like (including an empty file...) in each of
|
3
|
-
|
4
|
-
*
|
5
|
-
|
6
|
-
|
2
|
+
* You can put any puppet code you like (including an empty file...) in each of
|
3
|
+
the files under `/examples` and it will executed with `puppet apply`
|
4
|
+
* If you need to test multiple different things (eg different parameters for a
|
5
|
+
new type and provider), you need to create a different (acceptance) testcase
|
6
|
+
for each distinct thing to test
|
7
|
+
* PDQTest will only execute the BATS tests and setup scripts that are present,
|
8
|
+
you can delete some or all of these files if some steps aren't required.
|
9
|
+
* If no files are present under `spec/acceptance` for a given example, then
|
10
|
+
PDQTest will just check that puppet runs idempotently for your example
|
11
|
+
* To disable tests temporarily for a specific example, remove the magic marker
|
12
|
+
`#@PDQTest` or `#@PDQTestWin` from the example
|
7
13
|
* Nested examples (subdirectories) are not supported at this time
|
8
|
-
* Since the all of the `*__setup.sh` scripts are run in the container as root
|
14
|
+
* Since the all of the `*__setup.sh` scripts are run in the container as root
|
15
|
+
before executing tests, they can be used to mock _almost anything_ in the
|
16
|
+
test system:
|
9
17
|
* Replace system binaries to fake network operations
|
10
|
-
* Add system binaries to simulate other operating systems such as AIX,
|
18
|
+
* Add system binaries to simulate other operating systems such as AIX,
|
19
|
+
Solaris, etc
|
11
20
|
* Create/copy files, directories, etc.
|
12
21
|
* Install OS packages
|
13
|
-
* Install python scripts to mock database servers using SQLite...
|
22
|
+
* Install python scripts to mock database servers using SQLite... 😉
|
data/doc/troubleshooting.md
CHANGED
@@ -1,9 +1,20 @@
|
|
1
1
|
# Troubleshooting
|
2
|
-
* If you can't find the `pdqtest` command and your using `rbenv` be sure to
|
3
|
-
|
4
|
-
*
|
5
|
-
|
6
|
-
|
7
|
-
|
8
|
-
*
|
9
|
-
|
2
|
+
* If you can't find the `pdqtest` command and your using `rbenv` be sure to
|
3
|
+
run `rbenv rehash` after installing the gem to create the necessary symlinks
|
4
|
+
* If your `pdqtest` command changes version randomly depending which directory
|
5
|
+
your in and you are using `rvm` its probably because `rvm` overrides `cd` and
|
6
|
+
does strange things. You can probably turn this off. Alternatively, use
|
7
|
+
`rbenv`
|
8
|
+
* Don't forget to run `pdqtest setup` before your first `pdqtest` run to
|
9
|
+
download/update the Docker image
|
10
|
+
* If you need to access private git repositories, make sure to use
|
11
|
+
`.fixtures.yml` not `fixtures.yml` (changed in 2.0.0)
|
12
|
+
* If you need a private key to access private repositories, set this up for your
|
13
|
+
regular git command/ssh and `pdqtest` will reuse the settings
|
14
|
+
* Be sure to annotate the examples you wish to acceptance test with the magic
|
15
|
+
marker comment `#@PDQTest` or `#@PDQTestWin`
|
16
|
+
* Be sure to run `make` or `bundle exec pdqtest all` to download dependencies
|
17
|
+
when running acceptance tests. Previous versions (re)downloaded modules as
|
18
|
+
required from inside docker but this step has been replaced with a simple
|
19
|
+
symlink to reduce the amount of downloading so the modules must already be
|
20
|
+
present.
|