pdqtest 1.9.9beta7 → 1.9.9beta8

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: dd42c9920bb14bf63ba6dc6d021cb5bc2cb336aca20b76dcb41f8c19a317e545
-  data.tar.gz: 295dec7313f15b1e59348ea7e1106b8a03523ee5efe5bacb61b7a1df0e8e48bd
+  metadata.gz: 318a1841d2bcfda00c332734315ddf1278163cd50f3ce406463e400dad95bbb2
+  data.tar.gz: 1502c06caeea59ac008f102a14d4691e2ca0c9c4387dfc83fe0236d897e0fb6f
 SHA512:
-  metadata.gz: a886738ec27f9f20017f4a42d224b7a2b1edc62e3e55f5e8e472850abe61219e57766793c130d872389e1250b2000cf7fc5f703e0ca99569250ab71efabb8516
-  data.tar.gz: 62b42b4dc521d8f2d36937ef910bceb3fe193d6ddb866bbf11fe29ffc5f0c5d090123f33344cc73a77c1f59299260152377394d96a24681c0e5f86787554b9db
+  metadata.gz: e33f2c4d47f585819125cd74c0dbad0e38798c800df5aad2c1524c83d0687d86edfec2999cdb21db8fd6ad6e810f3b4fb56d13b30c84c064416c5a9091dc42b7
+  data.tar.gz: 9d8ba622a7c9b62e0b5946a444f2e3b1d1b34d76e82b857afb28439cb8a2c823a6c426ea41a51defddd3fdc650c5c9ef2409b2553375462b96c74eb5228d1349

data/README.md

@@ -2,32 +2,53 @@
 
 # PDQTest
 
-PDQTest - Puppet Docker Quick-test - is the quickest and easiest way to test your puppet modules. PDQTest features tests for:
-* Linting
-* Syntax
-* RSpec
-* Acceptance ([BATS-core](https://github.com/bats-core/bats-core))
+PDQTest - Puppet Docker Quick-test - is the quickest and easiest way to test 
+your puppet modules for Linux and Windows.
 
-And can generate code to retrofit testing to a new or existing module, along with skeleton RSpec and acceptance tests to get you started.
+PDQTest features:
+* Static analysis testing (PDK):
+    * Metadata
+    * Syntax
+    * Linting
+    * RSpec
+* Acceptance testing for Linux and Windows:
+    * [BATS-core](https://github.com/bats-core/bats-core)
+    * [PATS](https://github.com/declarativesystems/pats) (Experimental)
 
-PDQTest runs linting, syntax and RSpec tests within the machine it is running from and then loads a docker container to perform acceptance testing, sharing the puppet module and cached dependencies from your host.
+PDQTest acceptance tests can be retrofitted to and PDK enabled module by running
+`pdqtest init`.
+
+PDQTest runs static analysis tests within the machine it is running from and 
+then loads a docker container to perform acceptance testing, sharing the puppet
+module and cached dependencies from your host.
+
+This way you can destructively test your module without risking the host.
 
 ![demo](doc/demo.gif)
 _Adding PDQTest to a project and running acceptance tests in Docker_
 
+## PDQTest 2.0 Release notes
+PDQTest 2.0 new features:
+* [PDK Support](doc/pdk.md)
+* [Windows support](doc/windows.md)
+* [PDQTest 1.x -> 2.x Upgrade guide](doc/upgrade_1_2.md)
+
 ## PDQTest Manual
 1. [Installation](doc/installation.md)
 1. [Enabling testing](doc/enabling_testing.md)
 1. [Running tests](doc/running_tests.md)
-1. [Test generation](doc/test_generation.md)
 1. [Acceptance tests](doc/acceptance_tests.md)
+1. [Windows](doc/windows.md)
+1. [PDK Integration](doc/pdk.md)
 1. [Puppet module dependencies](doc/puppet_module_dependencies.md)
+1. [Test generation](doc/test_generation.md)
 1. [Puppet facts](doc/puppet_facts.md)
 1. [Hiera](doc/hiera.md)
 1. [Caching](doc/caching.md)
-1. [Upgrading](doc/upgrading.md)
 1. [Emoji](doc/emoji.md)
-1. [Examples](doc/examples.md)
+1. [Upgrading](doc/upgrading.md)
 1. [Tips and tricks](doc/tips_and_tricks.md)
 1. [Troubleshooting](doc/troubleshooting.md)
+1. [Examples](doc/examples.md)
 1. [Development](doc/development.md)
+1. [PDQTest 1.x -> 2.x Upgrade guide](doc/upgrade_1_2.md)

data/doc/acceptance_tests.md

@@ -1,14 +1,39 @@
 # Acceptance tests
-* Acceptance tests run within a Docker container managed by PDQTest
-* The Docker container breaks all the rules of Docker and runs a full version of Systemd to allow complete testing of Puppet code in the most portable way (basically treat docker as a high speed VM)... This is deliberate - PDQTest exists to get results, not be a perfect Docker app.
-* There are three supported docker images:
-    * Centos 7
-    * Ubuntu 16.04
-    * Windows (microsoft/windowsservercore)
-* Centos is used by default
-* Ubuntu will be used if your `metadata.json` declares compatibility with Ubuntu
-* Windows will be used if your `metadata.json` declares compatibility with 
-  Windows
+Acceptance tests run Puppet in apply mode inside a Docker container managed by
+PDQTest.
+
+The Docker container breaks all the rules of Docker and runs a full version of
+Systemd to allow complete testing of Puppet code in the most portable way 
+to basically treat docker as a high speed VM... This is deliberate - PDQTest 
+exists to get results, not be a perfect Docker app.
+
+There are three supported docker images:
+* Centos 7
+* Ubuntu 16.04
+* Windows (microsoft/windowsservercore)
+
+Centos is used by default, all other images are activated by the 
+`operatingsystem_support` element of `metadata.json`:
+
+**Windows**
+```json
+  "operatingsystem_support": [
+    {
+      "operatingsystem": "windows"
+    }
+  ],
+```
+
+**Ubuntu**
+```json
+  "operatingsystem_support": [
+    {
+      "operatingsystem": "Ubuntu"
+    },
+  ],
+```
+
+Notes:
 * Centos is great for mocking systems like AIX... if you replace OS binaries as
   required in the `__setup.sh` scripts
 * If you have a specific container you want to use, pass `--image-name` on the
@@ -19,25 +44,25 @@
   [Debugging failed builds](#debugging-failed-builds)
 * See the [docker_images](../docker_images) folder for examples
 
-### Test workflow
+## Test workflow
 1. Scan for all example files under `/examples`.  Files must end in `.pp` and 
    contain the magic marker `#@PDQTest` or `#@PDQTestWin` to be processed
 2. If example files are present, start the docker container
 3. For each example file:
   * Check for a file at `/spec/acceptance/EXAMPLE_NAME__setup.(sh|ps1)`, If it
     exists, run it inside the container.  This is a useful place to install 
-    preqrequisites, edit files, install mock services, clean up after other
-    tests, etc.
+    preqrequisites, edit files, install mock services, replace OS binaries, 
+    clean up after other tests, etc.
   * Check for a file at `/spec/acceptance/EXAMPLE_NAME__before.(bats|pats)`, If
     it exists run BATS or PATS against it.  This is normally used to check the
     system state is clean or that any test prerequisites have been setup 
-    correctly
+    correctly (most of the time you can skip this)
   * Run `puppet apply` on the example file twice (to check idempotency)
   * Check for a file at `/spec/acceptance/EXAMPLE_NAME.(bats|pats)`, If it 
-    exists run BATS against it.  This is normally used to check the state of the
-    system after running puppet.  You can do things like check services are
-    running, check files edited correctly, or basically anything that you can
-    write in bash!
+    exists run BATS/PATS against it.  This is normally used to check the state 
+    of the system after running puppet.  You can do things like check services 
+    are running, check files edited correctly, or basically anything that you 
+    can write in bash (or PowerShell)!
 4. Destroy the container unless we were asked to keep it with `--keep-container`
 
 Note: `EXAMPLE_NAME` is the example filename minus the directory and `.pp`, eg
@@ -55,7 +80,7 @@ the `EXAMPLE_NAME` for `examples/foo.pp` would just be `foo`.
 ### BATS tests
 If you've never seen a [BATS](https://github.com/bats-core/bats-core) test 
 before and have previously been writing server spec code, you'll probably kick
-youself when you see how simple they are.
+yourself when you see how simple they are.
 
 BATS lets you run tests anywhere you can run BASH.
 
@@ -113,14 +138,15 @@ At present you would need to run `pdqtest` twice - once in a Windows environment
 and again in Linux.
 
 This is because Docker doesn't let us run Windows and Linux on the same platform
-at the same time (Because it's not magic...).
+at the same time (Because it's not magic... Actually newer Docker versions do 
+support this on Windows by running Docker in muliple VMs...).
 
 This probably isn't as big an issue as it looks since most projects are going to
 cater to one specific OS family.
 
 #### Worked Example
-Lets say you have two examples, `foo.pp` and `init.pp`.  In this case, the full
-range of files to create for acceptance testing would look like this:
+Lets say you have two examples, `foo.pp` and `init.pp`. The files for acceptance
+testing would look like this:
 
 ```
 mycoolmodule
@@ -139,33 +165,43 @@ mycoolmodule
     ├── ...
 ```
 
-### Debugging failed builds
-PDQTest makes it easy to debug failed builds:
+When we run acceptance tests PDQTest will create a new container and then run:
+1. `sh /testcase/spec/acceptance/foo_setup.sh`
+2. `bats /testcase/spec/acceptance/foo__before.bats`
+3. `puppet apply /testcase/examples/foo.pp`
+4. `puppet apply /testcase/examples/foo.pp`
+5. `bats /testcase/spec/acceptance/foo.bats`
+6. `sh /testcase/spec/acceptance/init_setup.sh`
+7. `bats /testcase/spec/acceptance/init__before.bats`
+8. `puppet apply /testcase/examples/init.pp`
+9. `puppet apply /testcase/examples/init.pp`
+10. `bats /testcase/spec/acceptance/init.bats`
+
+Errors at any stage of the process will cause the test suite to abort once the
+error is returned.
 
-```shell
-# Centos
-pdqtest shell
-
-# Ubuntu
-pdqtest --image-name declarativesystems/pdqtest-ubuntu:2018-08-29-0 shell
+### Debugging failed builds
+PDQTest provides two targets to debug failed builds:
+1. `make shell`/`.\make.ps1 shell` - Load a new docker container, run all tests,
+   then print a message showing how to get a shell
+2. `make shellnopuppet`/`.\make.ps1 shellnopuppet` - Load a new docker container
+   and transport into it, without running Puppet
+   
+You may also want to run PDQTest directly to use a different image (Ubuntu) 
+se `--image-name` to use a different image (Ubuntu)
+
+```json
+cd .pdqtest
+bundle exec pdqtest --image-name declarativesystems/pdqtest-ubuntu:2018-08-29-0 shell
 ```
 
-* Opens a shell inside the default docker container that would be used to run 
-  tests
+Inside the container:
 * Your code is available at `/testcase`
-* Use `--image-name` to use a different image (Ubuntu)
+* Any puppet modules you have installed at `/spec/fixtures/modules` will be
+  available (note that `pdk build` removes these)
+* Hiera and custom facts will be installed if configured
 
-
-```shell
-pdqtest --keep-container all
-```
-* Runs all tests, report pass/fail status
-* Keeps the container Running
-* After testing, the container used to run tests will be left running and a 
-  message will be printed showing how to enter the container used for testing.
-  Your code is avaiable at `/testcase`
-* User is responsible for cleaning up the containers created in this mode
-* Shortcut: `make shell` or `.\make.ps1 shell`
+User is responsible for cleaning up the containers created in this mode
 
 ### Docker privileged mode
 Sometimes you need to run in Docker privileged mode to enable tests to work - 

data/doc/caching.md

@@ -3,8 +3,10 @@ PDQTest will attempt to cache:
 * Puppet modules (via r10k) in `~/.r10k/cache`
 * The yum cache in `~/.pdqtest/cache/yum`
 
-Note that since the yum cache is writen to via a docker volume from the 
-container your running tests in, the files in this directory will be root owned.
+Note that since the yum cache is written to via a docker volume from the 
+container your running tests in, the files in this directory will be `root` 
+owned.
+
 If you need to clear your cache for whatever reason, delete the 
-`~/.pdqtest/cache` directory (you will likely need `sudo`) and PDQtest will 
-recreate it next time it runs.
+`~/.pdqtest/cache` directory (you will need `sudo`) and PDQtest will recreate it
+next time it runs.

data/doc/development.md

@@ -3,14 +3,17 @@
 PRs welcome :)  Please ensure suitable tests cover any new functionality and 
 that all tests are passing before and after your development work.
 
-## Support
-Interested in commercial support of PDQTest?  Please email 
-sales@declarativesystems.com to discuss your needs.
-
 ## Debugging
-* run with `--verbosity debug`
+* run with `--verbosity debug` _and_ `--debug` (transitional while we sort out
+  logging in escort)
 * Debug docker API calls: `export EXCON_DEBUG=debug` or `set EXCON_DEBUG=debug` 
   then run `pdqtest` as normal
+  
+**Example**
+```shell
+cd .pdqtest
+bundle exec pdqtest --debug --verbosity debug all
+```
 
 ## Contributing
 Bug reports and pull requests are welcome on GitHub at 
@@ -24,3 +27,13 @@ https://github.com/declarativesystems/pdqtest.
 * Run specific test case `bundle exec rspec ./spec/SPEC/FILE/TO/RUN.rb:99` 
   (where 99 is the line number of the test)
 * If your using RubyMine, you can just right-click -> debug
+
+## Why not use `pdk bundle exec` instead of having your own `Gemfile`?
+I tried this originally and couldn't get it working. There's seems to be some
+kind of memory error from trying to run `pdk` inside `pdk`. In the end, separate
+`Gemfile`s turned out to be simpler and more robust. For full details see
+[PDK Integration](pdk.md).
+
+## Support
+Interested in commercial support of PDQTest?  Please email 
+sales@declarativesystems.com to discuss your needs.

data/doc/emoji.md

@@ -15,14 +15,14 @@ On windows, we fallback to output emoticons instead of Emoji
 
 ### Progress
 
-| Symbol | Meaning | Platform                                 |
-| ---    | ---                             | ---              |
-| Progress: Test passed                    | `😬` | `:-)`      |
-| Progress: Test failed                    | `💣` | `●~*`      |
-| Overall Status: All tests passed         | `😎` | `=D`       |
-| Overall Status: One or more tests failed | `💩` | `><(((*>`  |
-| Slow operation                           | `🐌` | `(-_-)zzz` |
-| Platform issue/limitation                | - | `(-_-)`       |
+| Description                              | Linux | Windows   |
+| ---                                      | ---   | ---        |
+| Progress: Test passed                    | `😬`  | `:-)`      |
+| Progress: Test failed                    | `💣`  | `●~*`      |
+| Overall Status: All tests passed         | `😎`  | `=D`       |
+| Overall Status: One or more tests failed | `💩`  | `><(((*>`  |
+| Slow operation                           | `🐌`  | `(-_-)zzz` |
+| Platform issue/limitation                | -     | `(-_-)`    |
 
 
 ## Disabling emoji

data/doc/enabling_testing.md

@@ -1,23 +1,47 @@
-# Enabling testing
+# Enabling testing 
+PDQTest is enabled and configured on a per-project basis.
+
+### New projects
+New projects should be created using PDK: `pdk new module`. 
+
+### Enabling
 To add PDQTest to a new or existing puppet module, run the commands:
 
+**Linux**
 ```shell
 pdqtest init
-pdk bundle install
+make pdqtestbundle
+make setup
+```
+
+**Windows**
+```shell
+pdqtest init
+.\make.ps1 pdqtestbundle
+.\make.ps1 setup
 ```
 
 From the top level directory of your puppet module. This will install PDQTest 
-into the `Gemfile.project` (for PDK's bundler), and generate an example set of
-acceptance tests.  
+into the `.pdqtest/Gemfile` and generate an example set of acceptance tests. 
+
+The `make setup` step downloads Docker images for your test platform:
 
-You **must** run `pdk bundle install` to install PDQTest into PDK's bundle and
-you **must not* run `bundle install` (ever!).
+**Linux**
+* Centos
+* Ubuntu
 
-For more info see (PDK-1172)[https://tickets.puppetlabs.com/browse/PDK-1172]
+**Windows**
+* Windows (Windows-Servercore) 
+
+PDQTest now integrates with and requires PDK, so ideally your module was 
+previously generated with `pdk new module`. If this was not the case, PDQTest
+will install the minimal integrations required to also enable PDK on the module.
 
 PDK must already be present on the system for this to work.
 
 Note:  Your puppet module *must* have a valid `metatadata.json` file.  Create
-one before running `pdqtest init` if you don't already have one.  If your 
-creating a puppet module from scratch, try `pdk new module` to create a
+one by running `pdk convert` before running `pdqtest init` if you don't already
+have one.  
+
+If your creating a puppet module from scratch, try `pdk new module` to create a
 complete module skeleton that includes a valid version of this file.

data/doc/hiera.md

@@ -1,15 +1,17 @@
 # Hiera
 Under normal circumstances, only a `profile` module would be consuming Hiera 
-data directly using the `hiera()` function, and its suggested by many that these
-too fall back to automatic data binding parameters rather then use the `hiera()`
-function directly.  In this case it makes more sense to use the excellent 
+data directly using the `lookup()` function, and its suggested by Puppet that 
+most of these fall back to automatic data binding parameters rather then using
+the function directly.  
+
+In this case it makes more sense to use the excellent 
 [Onceover](https://github.com/dylanratcliffe/onceover) tool to perform 
 end-to-end testing of your Hiera data, puppet control repository and 
 roles/profile modules.
 
 That said, there may be occasions where you need to mock hiera data:
 * You have implemented Roles and Profiles as a namespace inside a team module 
-  (for multi-tennanting)
+  (for multi-tenanting)
 * You have a module that performs `lookup()` lookups directly
 * You intend to drive your module by adding data to hiera and including a class
   to make something happen
@@ -18,8 +20,8 @@ That said, there may be occasions where you need to mock hiera data:
   able to test it
 
 ## Alternative method
-In some cases where it may be easiest to directly inject strings in lieu of 
-setting up a fake hiera:
+In most cases it may be easiest to directly inject strings in lieu of setting up
+a fake hiera:
 
 ### RSpec Tests
 ```ruby
@@ -42,7 +44,7 @@ class { "foo":
 Note that you would need to duplicate your hard-coded mock data between the 
 rspec tests and any acceptance tests.
 
-In the more complex cases it may instead be desireable to mock the hiera data in
+In the more complex cases it may instead be desirable to mock the hiera data in
 order to enable Puppet's regular lookup systems to work.
 
 ## Mocking Hiera (RSpec tests)

data/doc/installation.md

@@ -31,6 +31,18 @@ PDQTest and Docker.
    `docker` group.  You will need to log out and back in again after doing this
 8. Install the PDQTest docker images by typing `pdqtest setup`
 
+## Mac instructions
+**UNTESTED! Do you own a mac? Please let me know what works**
+
+Instructions should be as Linux, but from memory, I had to run:
+
+```shell
+eval "$(docker-machine env default)"
+```
+
+to configure the shell last time I owned a mac. If things have moved on since
+then, let me know what else is needed (possibly including a patch) 
+
 ## Windows Instructions
 
 1. You will need Windows 10 (Windows 10 Enterprise, Professional, or Education)
@@ -61,6 +73,6 @@ PDQTest and Docker.
 5. Install PDK: `choco install pdk`
 6. Install bundler `gem install bundler`
 
-**Be sure to read the [Windows notes](windows.md) for guidance on how to run tests
-on Windows**
+**Be sure to read the [Windows notes](windows.md) for guidance on how to run 
+tests on Windows**