pdqtest 1.4.1 → 1.9.9beta2

Sign up to get free protection for your applications and to get access to all the features.
Files changed (60) hide show
  1. checksums.yaml +4 -4
  2. data/.travis.yml +9 -3
  3. data/doc/acceptance_tests.md +100 -22
  4. data/doc/caching.md +5 -1
  5. data/doc/development.md +16 -5
  6. data/doc/emoji.md +20 -9
  7. data/doc/enabling_testing.md +15 -3
  8. data/doc/examples.md +25 -6
  9. data/doc/hiera.md +30 -11
  10. data/doc/installation.md +59 -8
  11. data/doc/pdk.md +334 -0
  12. data/doc/puppet_facts.md +45 -3
  13. data/doc/puppet_module_dependencies.md +34 -16
  14. data/doc/running_tests.md +35 -15
  15. data/doc/test_generation.md +11 -19
  16. data/doc/tips_and_tricks.md +17 -8
  17. data/doc/troubleshooting.md +19 -8
  18. data/doc/upgrading.md +125 -6
  19. data/doc/windows.md +51 -0
  20. data/docker_images/centos/Dockerfile +4 -3
  21. data/docker_images/centos/Makefile +1 -1
  22. data/docker_images/ubuntu/Dockerfile +3 -3
  23. data/docker_images/windows/Dockerfile +26 -0
  24. data/docker_images/windows/make.ps1 +2 -0
  25. data/exe/pdqtest +73 -28
  26. data/lib/pdqtest/core.rb +1 -1
  27. data/lib/pdqtest/docker.rb +143 -51
  28. data/lib/pdqtest/emoji.rb +33 -7
  29. data/lib/pdqtest/fastcheck.rb +19 -0
  30. data/lib/pdqtest/instance.rb +16 -15
  31. data/lib/pdqtest/logger.rb +35 -0
  32. data/lib/pdqtest/pdk.rb +98 -0
  33. data/lib/pdqtest/pdqtest1x.rb +115 -0
  34. data/lib/pdqtest/puppet.rb +277 -134
  35. data/lib/pdqtest/skeleton.rb +119 -39
  36. data/lib/pdqtest/upgrade.rb +42 -42
  37. data/lib/pdqtest/util.rb +82 -2
  38. data/lib/pdqtest/version.rb +1 -2
  39. data/pdqtest.gemspec +8 -13
  40. data/res/{skeleton → acceptance}/init.bats +0 -0
  41. data/res/{skeleton → acceptance}/init__before.bats +0 -0
  42. data/res/{skeleton → acceptance}/init__setup.sh +0 -0
  43. data/res/skeleton/.puppet-lint.rc +5 -0
  44. data/res/skeleton/{dot_travis.yml → .travis.yml} +0 -0
  45. data/res/skeleton/Makefile +20 -5
  46. data/res/skeleton/make.ps1 +66 -0
  47. data/res/skeleton/spec/fixtures/hiera.yaml +13 -0
  48. data/res/skeleton/spec/fixtures/hieradata/test.yaml +11 -0
  49. data/res/templates/examples_init.pp.erb +1 -1
  50. metadata +47 -115
  51. data/lib/pdqtest/lint.rb +0 -31
  52. data/lib/pdqtest/rspec.rb +0 -69
  53. data/lib/pdqtest/syntax.rb +0 -20
  54. data/res/skeleton/Gemfile +0 -7
  55. data/res/skeleton/Rakefile +0 -3
  56. data/res/skeleton/dot_gitignore +0 -9
  57. data/res/skeleton/dot_rspec +0 -2
  58. data/res/skeleton/hiera.yaml +0 -7
  59. data/res/skeleton/spec_helper.rb +0 -4
  60. data/res/skeleton/test.yaml +0 -3
data/doc/pdk.md ADDED
@@ -0,0 +1,334 @@
1
+ # PDK Integration
2
+
3
+ First off, what is PDK? PDK is Puppet's development kit. The bits we are
4
+ interested in right now are:
5
+ * Validation `pdk validate...`
6
+ * Testing `pdk test...`
7
+ * Packaging `pdk build...`
8
+
9
+ There's a lot more to PDK then just running simple tests though, PDK includes
10
+ its own version of ruby and various other tools.
11
+
12
+ On Linux PDK behaves (or at least appears to behave...) like a thin wrapper
13
+ around ruby, in the same vein as RBENV/RVM. On Windows, things are more involved
14
+ since it needs to install specific system libraries and reference them at
15
+ runtime somehow. This is why running `bundle install` on PDK project in windows
16
+ can cause strange and interesting blow-ups (don't do that!).
17
+
18
+ ## What's the point of PDQTest if we have PDK?
19
+ PDQTest provides an easy way to run acceptance tests using light-weight
20
+ Docker containers which we abuse to the point of treating them _almost_ like
21
+ VMs in order to extract maximum speed. It offers a different approach to
22
+ [Beaker](https://github.com/puppetlabs/beaker):
23
+ * Minimal to no project configuration required, just `metadata.json` and a few
24
+ other files we setup for you
25
+ * Simple acceptance tests written in BASH or PowerShell
26
+ * Provide _good enough_ (but not perfect) testing:
27
+ * Generic Ubuntu, Centos, Windows test environments
28
+ * Capable of mocking large/slow/networked systems by replacing or installing
29
+ binaries inside the container at the OS level, eg:
30
+ * Fake installation of Database server's by mocking the installer your
31
+ puppet `exec` would call with a simple python script
32
+ * Network tools such as `realmd` with a simple mock `realm` command
33
+ written in Python
34
+ * Complex OS commands that need to report different state between puppet
35
+ runs can be mocked with Python and a small SQLite database or simple
36
+ text files (eg the `sysctl` command)
37
+ * Fake different Unix-like Operating Systems like AIX by subverting the
38
+ commands that your puppet code would executes. This is particularly
39
+ useful in this case since there are no AIX desktop virtualisation
40
+ solutions right now and there probably never will be
41
+ * See [examples](examples.md) for real-world modules using these
42
+ techniques
43
+
44
+ Of course, this will only ever give you an approximation of a real-world system
45
+ but in many cases this is all that's needed. The complexity of setting up a
46
+ 100% accurate test environment is a daunting task and one that may never provide
47
+ ROI on the effort spent configuring and maintaining it.
48
+
49
+ If perfect accuracy is indeed required, then PDQTest is not what you are looking
50
+ for and you should look towards solutions like Beaker or
51
+ [Kitchen CI](https://kitchen.ci/).
52
+
53
+ ## How does PDQTest work with PDK? Do I have to choose?
54
+
55
+ No you don't have to choose! PDQTest integrates with PDK by making the minimal
56
+ amount of invasive changes:
57
+
58
+ ### New projects
59
+ New projects should be created using PDK: `pdk new module`.
60
+
61
+ This will give you all the PDK versioned files you would ever need. When you run
62
+ `pdqtest init` on a new project created with PDK, we only copy in our own
63
+ integration files and do not not touch the PDK generated files at all (the ones
64
+ marked 🛠 below),
65
+
66
+ ### Existing PDQTest projects
67
+ To make upgrading to PDQTest 2.0 + PDK easy for our existing users, we automate
68
+ the workflow normally carried out by `pdk convert`:
69
+
70
+ We do not include the entire set of files that would have been generated by
71
+ `pdk convert` or `pdk new module`. If you want the full set, you should run
72
+ one of these commands yourself before `pdqtest init` and they will be left alone
73
+ unless PDK starts writing new files that we are already using.
74
+
75
+ **metadata.json**
76
+
77
+ * Enable PDK by adding the fields:
78
+ * `pdk-version`
79
+ * `template-url`
80
+ * `template-ref`
81
+ The values for these fields are obtained automatically from PDK itself and are
82
+ consistent with the system installed PDK.
83
+
84
+ **Gemfile.project**
85
+
86
+ * Add a reference to the `pdqtest` gem
87
+
88
+ **Miscellaneous skeletons**
89
+
90
+ When `pdqtest init` is run, we install a small set of skeleton files in the root
91
+ of your project. This is a one-off operation to get you started. After your
92
+ module has been marked as PDK compatible, we no longer attempt to update these
93
+ files and you must only manage them using PDK from then-on.
94
+
95
+ Skeletons are generated on the fly using PDK itself, so they are always
96
+ up-to-date according to the PDK you have installed on your system.
97
+
98
+ ### Existing non-PDQTest projects
99
+ If you have an existing Puppet project that does not use PDQTest or PDK (eg
100
+ created by hand or with the old `puppet module generate` command), then the
101
+ recommended way to enable PDQTest is to first enable PDK by running
102
+ `pdk convert` and following the wizard.
103
+
104
+ You may then enable `PDQTest` by running `pdqtest init`.
105
+
106
+ Depending on the state of your project, you may be able to bypass the
107
+ `pdk convert` process by just running `pdqtest init` but this isn't recommended
108
+ or supported.
109
+
110
+
111
+ ### PDQTest directory structure
112
+ These are the files installed, files marked 🛠 are generated by PDK:
113
+
114
+ ```
115
+ ├── bitbucket-pipelines.yml
116
+ ├── Gemfile 🛠
117
+ ├── Gemfile.local
118
+ ├── Gemfile.project
119
+ ├── .gitignore 🛠
120
+ ├── Makefile
121
+ ├── make.ps1
122
+ ├── .pdkignore 🛠
123
+ ├── .puppet-lint.rc
124
+ ├── Rakefile 🛠
125
+ ├── spec
126
+ │   ├── default_facts.yml 🛠
127
+ │   ├── fixtures
128
+ │   │   ├── hieradata
129
+ │   │   │   └── test.yaml
130
+ │   │   └── hiera.yaml
131
+ │   └── spec_helper.rb 🛠
132
+ └── .travis.yml
133
+ ```
134
+
135
+ The remaining files are specific to PDQTest - notably:
136
+ * `.travis.yml` - Complete test suite for Linux modules
137
+ * `bitbucket-pipelines.yaml` - RSpec testing only
138
+ * `.puppet-lint.rc` - Make lint errors test failures, ignore double quotes, etc
139
+ * `Makefile` - Essential launch script for Linux
140
+ * `make.ps1` - Essential launch script for Windows
141
+ * `spec/fixtures/hiera.yaml` - Mock system-wide `hiera.yaml` file
142
+ * `spec/fixtures/hieradata/test.yaml` - Mock system-wide hieradata
143
+ * `Gemfile.project` - Used to enable the PDQTest gem (and any others you need)
144
+ * `Gemfile.local` - Transient file created at runtime by `Makefile` or
145
+ `make.ps1` to enable the PDQTest gem at runtime. This is a vital integration
146
+ point, see details below
147
+ * `.gitignore` - PDK has a massive list of files to ignore for new projects so
148
+ these are imported
149
+
150
+
151
+ ## Why are the launch scripts essential/How does the PDQTest gem load itself?
152
+ PDK provides its own `Gemfile` to select the gems available at runtime. There
153
+ are hooks in this file to load additional configuration from two locations:
154
+ * `~/.gemfile`
155
+ * `YOUR_PROJECT/Gemfile.local`
156
+
157
+ We support side-by-side installation of all versions of PDQTest so that users
158
+ are not forced to upgrade when they are not ready to, so we need a per-project
159
+ way to load our gem into the bundle. `Gemfile.local` is perfect for this but we
160
+ are not supposed to store permanent data in there according to PDK's generated
161
+ `.gitignore`.
162
+
163
+ The simplest way to workaround this issue is for us to create a symlink from
164
+ `Gemfile.project` to `Gemfile.local` at runtime and we _must_ use our launch
165
+ scripts to do this since we are unable to use any Ruby code to do this trick
166
+ since we are not yet in the bundle.
167
+
168
+ For this reason, you **must** launch PDQTest using the provided `Makefile` or
169
+ `make.ps1` scripts, at least initially.
170
+
171
+ For further background see:
172
+ * [PDK-1177](https://tickets.puppetlabs.com/browse/PDK-1177) and
173
+ * [#50](https://github.com/declarativesystems/pdqtest/issues/50)
174
+
175
+ ## When developing PDQTest, your version number **must** match a released version
176
+ _This should only affects PDQTest developers - for posterity..._
177
+
178
+ Due to some strange hackery between `pdk bundle` and `bundle exec` you have one
179
+ more gotcha when it comes to using developing PDQTest and thats that the version
180
+ number of the gem you build *MUST* match a released version of the gem on
181
+ rubygems.org otherwise `bundle exec` will barf with errors like this:
182
+
183
+ ```shell
184
+ $ bundle exec pdqtest
185
+ Could not find hiera-3.4.5 in any of the sources
186
+ Run `bundle install` to install missing gems.
187
+ ```
188
+
189
+ There seems no other way to fix this other than building the development gem
190
+ with a version matching an existing release. The gem then needs to be installed
191
+ using `gem install ...` and it will then magically start working. Using other
192
+ gem tricks like setting `:path` will appear to work but fail when `bundle exec`
193
+ is called. The other approach of trying to run everything with
194
+ `pdk bundle exec pdqtest` fails with various errors depending what platform your
195
+ on, presumably because `pdqtest` runs `pdk` command as part of its tests.
196
+
197
+ The final final caveat of this approach is that the PDQTest gem dependencies are
198
+ pinned at whatever the _real_ release uses, so if any were changed or added they
199
+ need to be temporarily added to `Gemfile.project`
200
+
201
+ The above works on Linux... For windows there is also the situation where:
202
+ * PDK vendors its own gems that magically appear
203
+ * PDK can only use "published" gems(specs?) it downloads - or appears to
204
+ * when you go to `bundle install` without using PDK you blow up the lock
205
+ file
206
+ * When you `bundle exec pdqtest` you get the above error because _something_
207
+ uses the old one `gemspec`
208
+ * The answer seems to be:
209
+ 1. use `gem 'pdqtest', :path=>'c:/vagrant/pdqtest'` in `Gemfile.local`
210
+ 2. `pdk bundle install`
211
+ 3. `bundle update pdqtest` - yes even though your not supposed to...
212
+ 4. `bundle exec pdqtest` - now it should work... why is beyond me
213
+ * If you still have errors:
214
+ 1. `bundle install`
215
+ 2. `rm Gemfile.lock`
216
+ 3. `pdk bundle install` - working?
217
+
218
+ Somtimes you just have to upload the whole gem file as a pre-release object:
219
+ * Version set to x.x.xWHATEVER
220
+ * `gem push pdqtest-x.x.xWHATEVER.gem`
221
+ * `gem install pdqtest --pre`
222
+
223
+
224
+ If your just want to use PDQTest, you will hopefully never have to worry about
225
+ this... On the other hand, if this all sounds confusing... That's because it is!
226
+
227
+ ## What happens when I upgrade PDQTest?
228
+ When a new version of PDQTest is released, upgrade by running:
229
+
230
+ ```shell
231
+ gem install pdqtest
232
+ cd /your/project
233
+ pdqtest upgrade
234
+ pdk bundle install
235
+ ```
236
+
237
+ This updates:
238
+ * Project gems in `Gemfile.project` (currently `pdqtest` and `puppet-strings`)
239
+ * Our CI and CLI integrations:
240
+ * `Makefile`
241
+ * `make.ps1`
242
+ * `.travis.yml`
243
+ * `bitbucket-pipelines.yml`
244
+
245
+ The final `pdk bundle install` command upgrades PDK's bundle to the include our
246
+ update and is critical to making the whole process work.
247
+
248
+ ## What happens when I upgrade PDK?
249
+ PDK operates independently from PDQTest and maintains its own files. Your free
250
+ to run `pdk update` to upgrade the your files to the latest PDK templated ones
251
+ at any time.
252
+
253
+ To protect PDQTest files from alteration (notably `.travis.yml` and
254
+ `bitbucket-pipelines.yml`) we merge them with any existing `.sync.yml` to
255
+ prevent churn.
256
+
257
+ If you have further customisations to PDK controlled files your options are:
258
+ * Use git to revert any change by PDK
259
+ * Use `.sync.yml` to influence file (re)generation
260
+ [blog](https://puppet.com/blog/guide-converting-module-pdk)
261
+ [reference](https://github.com/puppetlabs/pdk-templates)
262
+ [example](https://github.com/puppetlabs/puppetlabs-apt/blob/master/.sync.yml)
263
+
264
+ ## How do I use PDK with PDQTest installed? What can and can't I do?
265
+ You can run any `pdk` command as described in the PDK documentation. PDQTest
266
+ does not stop you running anything. If you find this not to be the case please
267
+ open a [ticket](https://github.com/declarativesystems/pdqtest/issues)
268
+
269
+ ## How do PDQTest lifecycle tests relate to PDK?
270
+
271
+ | PDQTest | PDK |
272
+ | --- | --- |
273
+ | `syntax` | `pdk validate metadata,puppet` |
274
+ | `rspec` | `pdk test unit` |
275
+ | `build` | `pdk build --force` |
276
+
277
+ * Previous `lint` subcommand was removed in PDQTest 2.0 as its now covered by
278
+ `syntax`
279
+
280
+ ## What actually happens when I run PDQTest?
281
+
282
+ `bundle exec pdqtest all` is the default target executed by `make` and
283
+ `.\make.ps1`. There are a few other targets that let you skip parts of the build
284
+ or run individual parts. The complete run looks like this:
285
+
286
+ 1. `pdk validate 'metadata,puppet'`
287
+ 2. Install modules listed in the `metadata.json` file using R10K against a
288
+ temporary `Puppetfile` at `Puppetfile.pdqtest`
289
+ 3. Generate a `.fixtures.yml` file based on `metadata.json`
290
+ Currently impacted by
291
+ [#47](https://github.com/declarativesystems/pdqtest/issues/47)
292
+ 4. `pdk test unit`
293
+ 5. Run all acceptance tests
294
+ 6. `puppet strings generate --format=markdown` to generate `REFERENCE.md`
295
+ 7. `pdk build --force` to generate your forge package
296
+
297
+ At each stage of the process, we output emoji's to keep you informed of
298
+ progress. Any failure prevents running the next phase of testing and lint errors
299
+ are considered failures.
300
+
301
+ ### Technical details (code)
302
+ If anyone knows a better way to do this, I'd love to hear about it:
303
+ [PDQTest lifecycle](https://github.com/declarativesystems/pdqtest/blob/master/exe/pdqtest)
304
+ [PDK Wrapper](https://github.com/declarativesystems/pdqtest/blob/master/lib/pdqtest/pdk.rb)
305
+
306
+ ## PDQTest is pretty slow!
307
+ This is a side effect of having to shell out to run PDK via system calls.
308
+ There's really no other way to do this while maintaining full PDK compatiblity
309
+ (if you know different, please enlighten me).
310
+
311
+ That said you might just want to run syntax and lint tests and acceptance tests
312
+ as quick as possible, in which case run:
313
+
314
+ **Linux**
315
+ ```
316
+ make fast
317
+ ```
318
+
319
+ **Windows**
320
+ ```json
321
+ .\make.ps1 fast
322
+ ```
323
+
324
+ This runs the syntax and lint tests using the original `puppet-syntax` and
325
+ `puppet-lint` libraries. We can't guarantee PDK identical behaviour or
326
+ compatibility when used this way but hey... its faster.
327
+
328
+ ## How do I automatically fix up my lint errors?
329
+
330
+ With PDK!:
331
+
332
+ ```shell
333
+ pdk validate -a
334
+ ```
data/doc/puppet_facts.md CHANGED
@@ -1,10 +1,52 @@
1
1
  # Puppet facts
2
- PDQTest includes built-in [factsets](https://github.com/declarativesystems/puppet_factset) covering many common operating systems, however, sometimes additional facts are needed throughout a module to mock an external fact or custom fact from a different module that is expected to be some known value. In this case, creating the directory:
2
+ PDQTest uses PDK's `spec/default_facts.yml` to add facts to acceptance tests as
3
+ external facts.
4
+
5
+ During test setup, PDQTest will installing the file at one of:
6
+ * `/etc/puppetlabs/facter/facts.d/default_facts.yaml`
7
+ * `C:\ProgramData\PuppetLabs\facter\facts.d`
8
+
9
+ PDK uses the file as part of the `pdk test unit` command. This way you have a
10
+ consistent set of facts for both RSpec and acceptance testing, in the file that
11
+ PDK expects them to be in.
12
+
13
+ ## 🐉 Differences from PDK 🐉
14
+ The above behaviour is not how PDK intends this file to be used as some other
15
+ mechanism is supposed to be used for acceptance tests (see PDK manual).
16
+
17
+ It's pretty rare to actually need `spec/default_facts.yml` unless your doing
18
+ unit testing since you can insert custom facts as part of your test setup.
19
+
20
+ Be aware that PDK sets a few facts to specific values that we will pick up and
21
+ use in your acceptance tests:
22
+
23
+ ```yaml
24
+ concat_basedir: "/tmp"
25
+ ipaddress: "172.16.254.254"
26
+ is_pe: false
27
+ macaddress: "AA:AA:AA:AA:AA:AA"
28
+ ```
29
+
30
+ You would need to use a forked version of the PDK templates to fix these
31
+ properly, but your using the structured facts anyway so it makes no difference,
32
+ right...?
33
+
34
+ ## Older versions
35
+ PDQTest < 2.0 includes built-in
36
+ [factsets](https://github.com/declarativesystems/puppet_factset) covering many
37
+ common operating systems, however, sometimes additional facts are needed
38
+ throughout a module to mock an external fact or custom fact from a different
39
+ module that is expected to be some known value. In this case, creating the
40
+ directory:
3
41
 
4
42
  ```
5
43
  spec/merge_facts
6
44
  ```
7
45
 
8
- Inside the module being tested will result in all JSON files contained being merged into all OS specific factsets used for testing as generated by `pdqtest generate_rspec`.
46
+ Inside the module being tested will result in all JSON files contained being
47
+ merged into all OS specific factsets used for testing as generated by
48
+ `pdqtest generate_rspec`.
9
49
 
10
- To put it simply, drop `.json` file(s) in this directory to add them to every factset. These facts will be made automatically avaialable to facter when running acceptance testing
50
+ To put it simply, drop `.json` file(s) in this directory to add them to every
51
+ factset. These facts will be made automatically avaialable to facter when
52
+ running acceptance testing
@@ -1,29 +1,47 @@
1
1
  # Puppet module dependencies
2
- To test your puppet code, PDQTest needs to configured to obtain any modules the code being tested depends on.
2
+ To test your puppet code, PDQTest needs to configured to obtain any modules the
3
+ code being tested depends on.
3
4
 
4
5
  ## Specifying dependencies
5
6
 
6
7
  ### Public modules (PuppetForge)
7
- Dependencies on public forge modules must be specified in your module's `metadata.json` file.
8
+ Dependencies on public forge modules must be specified in your module's
9
+ `metadata.json` file. When tests are run, we will use this to generate a
10
+ temporary `Puppetfile` at `Puppetfile.pdqtest` which we will then install into
11
+ `spec/fixtures/modules` using R10K.
12
+
13
+ When RSpec tests are run using PDQtest a `.fixtures.yml` file will be generated
14
+ for you based on the module metadata (with PDK standalone you must update this
15
+ file manually). We will then execute `pdk test unit` on your behalf which will
16
+ install any additional modules from git that can't be specified in
17
+ `metadata.json`.
18
+
19
+ This means you get a complete set of modules in `/spec/fixtures/modules` by the
20
+ time you come to run acceptance tests.
8
21
 
9
22
  ### Private modules (from git)
10
- If you need to download modules from git, then you must populate the `fixtures` section of `fixtures.yml`, eg:
23
+ If you need to download modules from git, then you must populate the `fixtures`
24
+ section of `.fixtures.yml`, eg:
11
25
 
12
26
  ```
13
- repositories:
14
- corporatestuff:
15
- repo: 'https://nonpublicgit.megacorp.com/corporatestuff.git'
16
- ref: 'mybranch'
27
+ fixtures:
28
+ repositories:
29
+ camera_shy:
30
+ repo: "git://git.megacorp.com/puppet/camera_shy"
31
+ ref: "2.6.0"
17
32
  ```
18
33
 
19
-
20
- ## .fixtures.yml
21
- There is no need to maintain a `.fixtures.yml` file and the presence of this file when using `pdqtest` is an error (note the leading period)
34
+ It is an error to define the same module in both `metadata.json` and
35
+ `.fixtures.yml` and the results of doing this are undefined:
36
+ * Always use `metadata.json` for public forge modules and we will update
37
+ `.fixtures.yml` based on it
38
+ * Modules from git should _only_ be defined in `.fixtures.yml`
22
39
 
23
40
  ## Notes:
24
- * The filename is for private modules is `fixtures.yml` NOT `.fixtures.yml`. The leading dot had to be removed to avoid `puppetlabs_spec_helper` also detecting the file and trying to use it.
25
- * The file format of `.fixtures.yml` and `fixtures.yml` for specifing git repositories is identical
26
- * Only the repositories section of the file will be processed as we do not use `puppetlabs_spec_helper` to do this for us.
27
- * We convert the dependencies from `metadata.json` to a temporary puppetfile store at `.Puppetfile.pdqtest` which is then installed using r10k
28
- * Previous versions of pdqtest configured the r10k cache via `.r10k.yaml` which caused #44. To fix this, we now use the default r10k cache dir at ~/.r10k/cache and don't write
29
- `.r10k.yaml` any more. You should remove any `.r10k.yaml` files from your project unless you need it for something specific.
41
+ * The file format of `.fixtures.yml` and `fixtures.yml` for specifing git
42
+ repositories is identical
43
+ * Previous versions of pdqtest configured the r10k cache via `.r10k.yaml` which
44
+ caused [#44](https://github.com/declarativesystems/pdqtest/issues/44). To fix
45
+ this, we now use the default r10k cache dir at `~/.r10k/cache` and don't write
46
+ `.r10k.yaml` any more. You should remove any `.r10k.yaml` files from your
47
+ project unless you need it for something specific.
data/doc/running_tests.md CHANGED
@@ -1,38 +1,58 @@
1
1
  # Running tests
2
+
3
+ ## Important
4
+ You **must** use the provided launch scripts `Makefile` or `make.ps1` to run
5
+ PDQTest, at least for the first time. See [PDK integration](pdk.md) for more
6
+ information.
7
+
8
+ ## Quickstart
2
9
  If you just want to run all tests:
3
10
 
11
+ **Linux**
4
12
  ```shell
5
13
  make
6
14
  ```
7
15
 
8
- Alternatively, you can choose to run individul test phases directly:
9
-
10
- ## All tests (excludes documentation)
11
-
16
+ **Windows**
12
17
  ```shell
13
- bundle exec pdqtest all
18
+ .\make.ps1
14
19
  ```
15
20
 
16
- ### Syntax
21
+ Alternatively, you can choose to run different groups of tests by supplying a
22
+ target from this table:
17
23
 
18
- ```shell
19
- bundle exec pdqtest syntax
20
- ```
24
+ | Target | Description | PDK compatible? |
25
+ | --- | --- | --- |
26
+ | all | lint, syntax, rspec, acceptance, strings, build | yes |
27
+ | fast | lint, syntax, acceptance, strings, build | no |
28
+ | shell | run acceptance tests and print command to get a shell in the container | yes |
29
+ | shellnopuppet | open a shell in the test container | yes |
30
+ | logical | syntax, lint | yes |
31
+
32
+ * PDK compatible means we run the `pdk` command for this part of the lifecycle
21
33
 
22
- ### Lint
23
34
 
35
+ **Example**
36
+
37
+ **Linux**
24
38
  ```shell
25
- bundle exec pdqtest lint
39
+ make fast
26
40
  ```
27
41
 
28
- ### RSpec
42
+ **Windows**
43
+ ```shell
44
+ .\make.ps1 fast
45
+ ```
29
46
 
47
+ ## Only run acceptance tests
30
48
  ```shell
31
- bundle exec pdqtest rspec
49
+ bundle exec pdqtest acceptance
32
50
  ```
33
51
 
34
- ### Acceptance
35
52
 
53
+ ## See also
54
+
55
+ PDQTest help:
36
56
  ```shell
37
- bundle exec pdqtest acceptance
57
+ bundle exec pdqtest --help
38
58
  ```
@@ -1,25 +1,12 @@
1
1
  # Test Generation
2
- Creating a bunch of files manually is an error prone and tedious operation so PDQTest can generate files and boilerplate code for you so that your up and running in the quickest time possible.
3
-
4
-
5
- ## RSpec tests
6
- The skeleton tests created by `pdqtest init` only cover the `init.pp` file which is useful but your likely going to need to support more classes as your modules grow. PDQTest can generate basic RSpec testcases for each new puppet **class** that exists in the manifests directory for you:
7
-
8
- ```shell
9
- bundle exec pdqtest generate_rspec
10
- ```
11
-
12
- * For every `.pp` file containing a puppet class under `/manifests`, RSpec will be generated to:
13
- * Check the catalogue compiles
14
- * Check the catalogue contains an instance of the class
15
-
16
- This gives developers an easy place to start writing additional RSpec tests for specific behaviour
17
-
18
- Its safe to run this command whenever you add a new class, it won't overwrite any existing RSpec testcases
2
+ Creating a bunch of files manually is an error prone and tedious operation so
3
+ PDQTest can generate files and boilerplate code for you so that your up and
4
+ running in the quickest time possible.
19
5
 
20
6
  ## Acceptance tests
21
7
 
22
- Generate boilerplate files for each example found (including those without a magic marker):
8
+ Generate boilerplate files for each example found (including those without a
9
+ magic marker):
23
10
 
24
11
  ```shell
25
12
  pdqtest generate_acceptance
@@ -31,4 +18,9 @@ Generate boilerplate files for one specific example:
31
18
  pdqtest generate_acceptance examples/mynewthing.pp
32
19
  ```
33
20
 
34
- Note: This will also create examples/mynewthing.pp if you haven't created it yet
21
+ This will also create examples/mynewthing.pp if you haven't created it yet.
22
+
23
+
24
+ ## RSpec tests
25
+ PDQTest < 2.0 includes rspec test generation. This functionality is replaced by
26
+ PDK in later versions.
@@ -1,13 +1,22 @@
1
1
  # Tips and tricks
2
- * You can put any puppet code you like (including an empty file...) in each of the files under `/examples` and it will executed with `puppet apply`
3
- * If you need to test multiple different things (eg different parameters for a new type and provider), you need to create a different (acceptance) testcase for each distinct thing to test
4
- * PDQTest will only execute the BATS tests and setup scripts that are present, you can delete some or all of these files if some steps aren't required.
5
- * If no files are present under `spec/acceptance` for a given example, then PDQTest will just check that puppet runs idempotently for your example
6
- * To disable tests temporarily for a specific example, remove the magic marker `#@PDQTest` from the desired example
2
+ * You can put any puppet code you like (including an empty file...) in each of
3
+ the files under `/examples` and it will executed with `puppet apply`
4
+ * If you need to test multiple different things (eg different parameters for a
5
+ new type and provider), you need to create a different (acceptance) testcase
6
+ for each distinct thing to test
7
+ * PDQTest will only execute the BATS tests and setup scripts that are present,
8
+ you can delete some or all of these files if some steps aren't required.
9
+ * If no files are present under `spec/acceptance` for a given example, then
10
+ PDQTest will just check that puppet runs idempotently for your example
11
+ * To disable tests temporarily for a specific example, remove the magic marker
12
+ `#@PDQTest` or `#@PDQTestWin` from the example
7
13
  * Nested examples (subdirectories) are not supported at this time
8
- * Since the all of the `*__setup.sh` scripts are run in the container as root before executing tests, they can be used to mock _almost anything_ in the test system:
14
+ * Since the all of the `*__setup.sh` scripts are run in the container as root
15
+ before executing tests, they can be used to mock _almost anything_ in the
16
+ test system:
9
17
  * Replace system binaries to fake network operations
10
- * Add system binaries to simulate other operating systems such as AIX, Solaris, etc
18
+ * Add system binaries to simulate other operating systems such as AIX,
19
+ Solaris, etc
11
20
  * Create/copy files, directories, etc.
12
21
  * Install OS packages
13
- * Install python scripts to mock database servers using SQLite... :wink:
22
+ * Install python scripts to mock database servers using SQLite... 😉
@@ -1,9 +1,20 @@
1
1
  # Troubleshooting
2
- * If you can't find the `pdqtest` command and your using `rbenv` be sure to run `rbenv rehash` after installing the gem to create the necessary symlinks
3
- * If your `pdqtest` command changes version randomly depending which directory your in and you are using `rvm` its probably because `rvm` overrides `cd` and does strange things. You can probably turn this off. Alternatively, use `rbenv`
4
- * Don't forget to run `pdqtest setup` before your first `pdqtest` run to download/update the Docker image
5
- * If you need to access private git repositories, make sure to use `fixtures.yml` not `.fixtures.yml`
6
- * If you need a private key to access private repositories, set this up for your regular git command/ssh and `pdqtest` will reuse the settings
7
- * Be sure to annotate the examples you wish to acceptance test with the magic marker comment `#@PDQTest`
8
- * Sometimes you might get an error: `Could not resolve the dependencies.` when executing tests. This message is from librarian puppet and usually indicates a conflict between the `metadata.json` files somewhere in the set of modules you are attempting to use or the presence of a `Puppetfile` in a directory above the module your testing. Running the command `librarian-puppet install --path spec/fixtures/ --destructive --verbose` should give you enough information to resolve the error
9
- * Be sure to run `make` or `bundle exec pdqtest all` to download dependencies when running acceptance tests. Previous versions (re)downloaded modules as required from inside docker but this step has been replaced with a simple symlink to reduce the amount of downloading so the modules must already be present.
2
+ * If you can't find the `pdqtest` command and your using `rbenv` be sure to
3
+ run `rbenv rehash` after installing the gem to create the necessary symlinks
4
+ * If your `pdqtest` command changes version randomly depending which directory
5
+ your in and you are using `rvm` its probably because `rvm` overrides `cd` and
6
+ does strange things. You can probably turn this off. Alternatively, use
7
+ `rbenv`
8
+ * Don't forget to run `pdqtest setup` before your first `pdqtest` run to
9
+ download/update the Docker image
10
+ * If you need to access private git repositories, make sure to use
11
+ `.fixtures.yml` not `fixtures.yml` (changed in 2.0.0)
12
+ * If you need a private key to access private repositories, set this up for your
13
+ regular git command/ssh and `pdqtest` will reuse the settings
14
+ * Be sure to annotate the examples you wish to acceptance test with the magic
15
+ marker comment `#@PDQTest` or `#@PDQTestWin`
16
+ * Be sure to run `make` or `bundle exec pdqtest all` to download dependencies
17
+ when running acceptance tests. Previous versions (re)downloaded modules as
18
+ required from inside docker but this step has been replaced with a simple
19
+ symlink to reduce the amount of downloading so the modules must already be
20
+ present.