pdqtest 1.9.9beta7 → 1.9.9beta8

data/doc/upgrade_1_2.md

@@ -1,13 +1,177 @@
+# PDQTest 1x -> 2x upgrade notes
+PDQTest is now compatible with (and requires) PDK! By and large we let PDK do 
+its own thing and just wrap the output with emojis and run our own acceptance
+tests.
+
+See [PDK Integration](pdk.md) for details of how this works. 
+
+Since this is a major upgrade, you must re-init your project. Make sure all your
+code is checked in to git, 
+[install PDK](https://puppet.com/docs/pdk/1.x/pdk_install.html) then run:
+
+```shell
+gem install pdqtest
+cd /my/project
+pdqtest init
+make pdqtestbundle
+make setup
+```
+
+## Custom facts
+If you were previous using custom facts in the `spec/merge_facts` directory, 
+these need to be converted to yaml and moved to `spec/default_facts.yml`. This
+will give you compatibility between PDK unit tests and PDQTest acceptance tests.
+
+## RSpec tests (inc hiera data)
+PDQTest doesn't deal with RSPec tests any more, we let PDK do all the work by
+shelling out to call `pdk test unit`:
+
+* If you were using PDQTest for your hiera data during the RSpec lifecycle, 
+  update your tests to do what PDK tells you to do
+* Existing `spec/fixtures/hiera.yaml` and `spec/fixtures/test.yaml` files will
+  continue to work during acceptance tests and the `hiera.yaml` file has been
+  upgraded for hiera 5 compatibility. You will have to reconfigure your 
+  hierarchy if you were using more files then just `test.yaml`
+
+PDQTest 1x generated RSpec tests will fail due to missing dependency on 
+[puppet_factset](https://rubygems.org/gems/puppet_factset) gem which is not used
+for new PDK generated classes.
+
+To fix you tests, either:
+
+**Enable `puppet_factset` gem**
+
+You can enable this gem in PDK by adding it to `Gemfile.project` and running:
+
+*Linux*
+
+```shell
+make Gemfile.local
+```
+
+*Windows*
+
+```shell
+.\make.ps1 Gemfile.local
+```
+
+**Rewrite tests to new format**
+
+PDK writes RSpec test using its own template which creates output like this:
+
+```ruby
+require 'spec_helper'
+
+describe 'CHANGETHISTOYOURCLASSNAME' do
+  on_supported_os.each do |os, os_facts|
+    context "on #{os}" do
+      let(:facts) { os_facts }
+
+      it { is_expected.to compile }
+    end
+  end
+end
+```
+
+**Delete your RSpec tests**
+
+If your in a hurry and have good acceptance tests, another quick fix would be to
+just delete the RSpec tests and rely purely on acceptance testing. 
+
+You will miss fast-failure due to invalid puppet code if you do this, and will
+lose any specialised testing you had previously.
+
+
+## `.fixtures.yml` and `fixtures.yml`
+`.fixtures.yml` is back in PDQTest 2.0 (it's presence was an error in PDQTest 
+1.x) and `fixtures.yaml` is no longer used.
+
+If you would like to use test fixtures from git, add them to `.fixtures.yml` and
+you can continue to use them as before. The rest of the file is generated for
+you automatically by PDQTest based on `metadata.json` to support integration 
+with PDK.
+
+[More info on dependencies](puppet_module_dependencies.md)
+
+## Old PDQtest 1x integration points
+* `/spec/spec_helper.rb`
+* `/Gemfile`
+* `/Rakefile`
+* `/.rspec`
+We will do our best to detect any previous PDQTest generated versions of these
+files. If known versions are found, we will upgrade you to the PDK version. If
+an unknown file is found we will stop and ask you to move it out of the way
+first. PDK will be responsible for these files now-on.
+
+## How to resolve `unknown/modified file` errors during upgrade
+1. Move your existing file out of the way
+2. Run `pdqtest init`
+3. See per-file upgrade notes below
+
+**`/Gemfile`**
+
+Add any custom gems that used to be in `Gemfile` to `Gemfile.project` and they
+will be evaluated during `pdk bundle install` and available during 
+`pdk test unit`.
+
+To make gems available to PDQTest itself, add them to `.pdqtest/Gemfile`.
+
+**Other files**
+
+If the upgrade cannot detect where a file that needs replacing came from we ask
+the user to move/delete it. Most other files can safely be updated without issue
+but we ask you to confirm this.
+
+## PDK Managed files
+PDK regenerates files it manages you run `pdk update`. To customise these files
+and have your changes persist between updates, you need to add them to 
+`.sync.yml` which allows you to override _some_ settings in the 
+[PDK default templates](https://github.com/puppetlabs/pdk-templates/).
+
+If the templates don't support the customisation you want, the advice from 
+Puppet is to fork and customise that repository, then configure PDK to use it.
+
+Consult the PDK documentation for instructions on how to do this. 
+
+## Miscellaneous files/directories
+### `/.gitignore`
+We no longer install or manage `.gitignore` for you and we don't upgrade it
+either. PDK ships a default version and we use that if the initial file is 
+missing.
+
+### `/cut` mountpoint
+Previous versions of PDQTest mounted code at `/cut` (Code Under Test), the new
+mountpoint is the more obvious `/testcase` and `/cut` no longer works.
+
+## R10K cache issues
+PDQTest < 2.0 configured the r10k cache via `.r10k.yaml` which caused 
+[#44](https://github.com/declarativesystems/pdqtest/issues/44). To fix this, we
+now use the default r10k cache dir at `~/.r10k/cache` and don't write 
+`.r10k.yaml` any more. You should remove any `.r10k.yaml` files from your 
+project unless you need it for something specific.
+
+## PDK validation errors when puppet code valid
+PDK scans the `/pkg` directory and will choke if old files are present there:
+[PDK-1183](https://tickets.puppetlabs.com/browse/PDK-1183)
+
+To remove them:
+
+*Linux*
+
+```shell
+make clean
+```
+
+*Windows*
+```shell
+.\make.ps1 shell
+```
+
+## What does a real update look like?
+```shell
 geoff@computer:~/tmp/ssh$ git checkout -b pdk
 Switched to a new branch 'pdk'
-geoff@computer:~/tmp/ssh$ pdqtest init
-Puppet metadata not found at metadata.json - not a valid puppet module
-geoff@computer:~/tmp/ssh$ ls
-total 72K
-4.0K bitbucket-pipelines.yml  4.0K Gemfile       4.0K Makefile         4.0K Rakefile
-4.0K CHANGELOG                4.0K Gemfile.lock  4.0K manifests        4.0K README.md
-4.0K data                     4.0K hiera.yaml    4.0K metadata.json    4.0K REFERENCE.md
-4.0K examples                  12K LICENSE       4.0K Puppetfile.lock  4.0K spec
+
 geoff@computer:~/tmp/ssh$ pdqtest init
 Doing one-time upgrade to PDK - Generating fresh set of files...
 pdk (INFO): Creating new module: x
@@ -22,6 +186,7 @@ Detected PDQTest 1.x file at Rakefile (will upgrade to PDK)
 Detected PDQTest 1.x file at .gitignore (will upgrade to PDK)
 Updated .sync.yml with {".travis.yml"=>{:unmanaged=>true}}
 enabling PDK in metadata.json
+
 geoff@computer:~/tmp/ssh$ git status
 On branch pdk
 Changes not staged for commit:
@@ -53,47 +218,7 @@ Untracked files:
 	spec/fixtures/
 
 no changes added to commit (use "git add" and/or "git commit -a")
-geoff@computer:~/tmp/ssh$ cat Makefile 
-all:
-	cd .pdqtest && pwd && bundle exec pdqtest all
-	$(MAKE) docs
-
-fast:
-	cd .pdqtest && pwd && bundle exec pdqtest fast
-
-shell:
-	cd .pdqtest && pwd && bundle exec pdqtest --keep-container acceptance
-
-setup:
-	cd .pdqtest && pwd && bundle exec pdqtest setup
-
-shellnopuppet:
-	cd .pdqtest && pwd && bundle exec pdqtest shell
-
-logical:
-	cd .pdqtest && pwd && bundle exec pdqtest syntax
-	cd .pdqtest && pwd && bundle exec pdqtest rspec
-	$(MAKE) docs
-
-#nastyhack:
-#	# fix for - https://tickets.puppetlabs.com/browse/PDK-1192
-#	find vendor -iname '*.pp' -exec rm {} \;
 
-pdqtestbundle:
-	# Install all gems into _normal world_ bundle so we can use all of em
-	cd .pdqtest && pwd && bundle install
-
-docs:
-	cd .pdqtest && pwd && bundle exec "cd ..&& puppet strings"
-
-
-Gemfile.local:
-	echo "[🐌] Creating symlink and running pdk bundle..."
-	ln -s Gemfile.project Gemfile.local
-	$(MAKE) pdkbundle
-
-pdkbundle:
-	pdk bundle install
 geoff@computer:~/tmp/ssh$ make pdqtestbundle
 # Install all gems into _normal world_ bundle so we can use all of em
 cd .pdqtest && pwd && bundle install
@@ -139,6 +264,7 @@ Using puppet-strings 2.1.0
 Using puppet-syntax 2.4.1
 Bundle complete! 5 Gemfile dependencies, 38 gems now installed.
 Use `bundle info [gemname]` to see where a bundled gem is installed.
+
 geoff@computer:~/tmp/ssh$ make
 cd .pdqtest && pwd && bundle exec pdqtest all
 /home/geoff/tmp/ssh/.pdqtest
@@ -155,3 +281,10 @@ Overall: 💩
 ABORTED - there are test failures! :(
 Makefile:2: recipe for target 'all' failed
 make: *** [all] Error 1
+```
+
+See the error? That came from PDK validating our metadata... Looks like this
+module had invalid metadata all along. At this point tests can be re-run any 
+time just by running `make`.
+
+When all tests are passing again the update to PDQTest 2.0/PDK is complete.

data/doc/upgrading.md

@@ -6,13 +6,23 @@ gem update pdqtest
 ```
 
 Each project you want to use the newer version of PDQTest on should then have 
-it's `Gemfile.project` updated to reference the latest version.  Don't worry, 
+it's `.pdqtest/Gemfile` updated to reference the latest version.  Don't worry, 
 this is easy:
 
+**Linux**
+
 ```shell
-cd /my/cool/project/to/upgrade
+cd /your/project
 pdqtest upgrade
-pdk bundle install
+make pdqtestbundle
+```
+
+**Windows**
+
+```shell
+cd /your/project
+pdqtest upgrade
+.\make.ps1 pdqtestbundle
 ```
 
 Note that since we're using bundler, you only have to upgrade the puppet modules
@@ -23,125 +33,25 @@ via `make` just fine. You are not forced to update all your modules in one go.
 Updated docker images are periodically released and are required to run newer
 PDQTest versions. When you get a message about missing docker containers run:
 
-```shell
-pdqtest setup
-```
-
-To obtain the latest version.
-
-# PDQTest 1x -> 2x
-PDQTest is now compatible with (and requires) PDK! By and large we let PDK do 
-its own thing and just wrap the output with emojis and run our own acceptance
-tests.
-
-See [PDK Integration](pdk.md) for details of how this works. 
-
-Since this is a major upgrade, you must re-init your project. Make sure all your
-code is checked in to git, 
-[install PDK](https://puppet.com/docs/pdk/1.x/pdk_install.html) then run:
+**Linux**
 
 ```shell
-gem install pdqtest
-cd /my/project
-pdqtest init
-pdk bundle install
-bundle exec pdqtest setup
+make setup
 ```
 
-## Custom facts
-If you were previous using custom facts in the `spec/merge_facts` directory, 
-these need to be converted to yaml and moved to `spec/default_facts.yml`. This
-will give you compatibility between PDK unit tests and PDQTest acceptance tests.
-
-## RSpec tests (inc hiera data)
-PDQTest doesn't deal with RSPec tests any more, we let PDK do all the work by
-shelling out to call `pdk test unit`:
-
-* If you were using PDQTest for your hiera data during the RSpec lifecycle, 
-  update your tests to do what PDK tells you to do
-* Existing `spec/fixtures/hiera.yaml` and `spec/fixtures/test.yaml` files will
-  continue to work during acceptance tests and the `hiera.yaml` file has been
-  upgraded for hiera 5 compatibility. You will have to reconfigure your 
-  hierarchy if you were using more files then just `test.yaml`
-
-Old PDQTest generated RSpec tests will fail due to missing dependency on 
-[puppet_factset](https://rubygems.org/gems/puppet_factset) gem which is now no
-longer required (by anything 😁).
-
-You can enable this gem in PDK by adding it to `Gemfile.project` and running
-`pdk bundle install` or you can just rewrite your tests using the new PDK format
-which looks like this:
+**Windows**
 
-```ruby
-require 'spec_helper'
-
-describe 'CHANGETHISTOYOURCLASSNAME' do
-  on_supported_os.each do |os, os_facts|
-    context "on #{os}" do
-      let(:facts) { os_facts }
-
-      it { is_expected.to compile }
-    end
-  end
-end
+```shell
+.\make.ps1 setup
 ```
 
-If your in a hurry and have good acceptance tests, another quick fix would be to
-just delete the RSpec tests and rely purely on real-world testing. You will miss 
-fast-failure due to invalid puppet code if you do this though, since we have to
-launch a container and run puppet to see if your code compiles.
-
-You can use PDK to generate these tests, see the PDK manual for details of how.
-
-## `/.fixtures.yml` and `/fixtures.yml`
-`/.fixtures.yml` is back in PDQTest 2.0 (it's presence was an error in PDQTest 
-1.x) and `/fixtures.yaml` is no longer used.
-
-If you would like to use test fixtures from git, add them to `.fixtures.yml` and
-you can continue to use them as before. The rest of the file is generated for
-you automatically based on `metadata.json` to support integration with PDK.
-
-[More info on dependencies](puppet_module_dependencies.md)
-
-## Old PDQtest 1x integration points
-* `/spec/spec_helper.rb`
-* `/Gemfile`
-* `/Rakefile`
-* `/.rspec`
-We will do our best to detect any previous PDQTest generated versions of these
-files. If known versions are found, we will upgrade you to the PDK version. If
-an unknown file is found we will stop and ask you to move it out of the way
-first. PDK will be responsible for these files now-on.
-
-### How to resolve `unknown/modified file` errors during upgrade
-1. Move your existing file out of the way
-2. Run `pdqtest init`
-3. See per-file upgrade notes below
-
-#### `/Gemfile`
-Add any custom gems/`Gemfile` magic to `gemfile.project` and it will be evaluated
-during `pdk bundle install`
-
-### PDK Managed files
-PDK regenerates files it manages you run `pdk update`. To customise these files
-and have your changes persist between updates, you need to add them to 
-`.sync.yml` which allows you to override _some_ settings in the 
-[PDK default templates](https://github.com/puppetlabs/pdk-templates/).
-
-If the templates don't support the customisation you want, the advice from 
-Puppet is to fork and customise that repository, then configure PDK to use it.
-
-Consult the PDK documentation for instructions on how to do this. 
-
-## `/Gemfile.project`
-New file used for configuring which PDQTest gem to use (replaces customisation
-of `Gemfile`)
-
-## `/.gitignore`
-We no longer install or manage `.gitignore` for you and we don't upgrade it
-either. PDK ships a default version and we use that if the initial file is 
-missing.
+To obtain the latest version.
 
-## `/cut` mountpoint
-Previous versions of PDQTest mounted code at `/cut` (Code Under Test), the new
-mountpoint is the more obvious `/testcase` and `/cut` no longer works.
+## What happens during `pdqtest upgrade`?
+We update:
+* Project gems in `.pdqtest/Gemfile`
+* Our CI and CLI integrations:
+    * `Makefile`
+    * `make.ps1`
+    * `.travis.yml`
+    * `bitbucket-pipelines.yml`

data/doc/windows.md

@@ -1,5 +1,6 @@
 # Windows
-PDQTest now supports windows! There are a few gotchas though:
+PDQTest now supports Windows! There are a few gotchas though and this support is
+still experimental:
 
 * The docker volume doesn't play nice with puppet/ruby: Whole app breaks if 
   `/lib` directory present, so we have to _copy_ the files into the container
@@ -30,22 +31,18 @@ know
 
 Q: what's going on with `.bat` files?
 
-A:They seem to work but you need to pass the full filename to docker run, eg:
+A:They seem to work but you need to pass the full filename to docker run (yes, 
+`puppet` is actually `puppet.bat` on an installed system), eg:
 
 ```shell
-docker exec XXX foo.bat
+docker exec XXX puppet.bat apply c:\testcase\examples\init.pp
 ```
 
 not
 
 ```shell
-docker exec XXX foo
+docker exec XXX puppet
 ```
 _where XXX is the ID of your container. Use `docker ps` to get a listing.
 
-Q: My module refuses to be found or work! (class not found...)
-
-A: Beats me.. This came up during testing but I haven't investigated further. 
-Make sure you have a `depenedencies` section in `metadata.json` even if its 
-empty or puppet seems to break (unverified)
 

data/lib/pdqtest/docker.rb

@@ -10,7 +10,7 @@ module PDQTest
     IMAGES = {
      :DEFAULT => 'declarativesystems/pdqtest-centos:2018-09-15-0',
      :UBUNTU  => 'declarativesystems/pdqtest-ubuntu:2018-09-15-0',
-     :WINDOWS => 'declarativesystems/pdqtest-windows:2018-09-29-0',
+     :WINDOWS => 'declarativesystems/pdqtest-windows:2018-09-30-0',
     }
 
     # volume paths are different for windows containers

data/lib/pdqtest/skeleton.rb

@@ -39,9 +39,13 @@ module PDQTest
         ".travis.yml" => {
             "unmanaged" => true,
         },
+        "bitbucket-pipelines.yml" => {
+            "unmanaged" => true,
+        },
         ".gitignore" => {
             "paths" => [
                 ".Puppetfile.pdqtest",
+                "refresh.ps1",
             ],
         },
         ".gitattributes" => {

data/lib/pdqtest/version.rb

@@ -1,3 +1,3 @@
 module PDQTest
-  VERSION = "1.9.9beta7"
+  VERSION = "1.9.9beta8"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: pdqtest
 version: !ruby/object:Gem::Version
-  version: 1.9.9beta7
+  version: 1.9.9beta8
 platform: ruby
 authors:
 - Geoff Williams
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2018-09-30 00:00:00.000000000 Z
+date: 2018-10-01 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: bundler