pdqtest 1.4.1 → 1.9.9beta2

data/doc/upgrading.md

@@ -5,18 +5,23 @@ To upgrade the current version of PDQTest on your system:
 gem update pdqtest
 ```
 
-Each project you want to use the newer version of PDQTest on should then have it's `Gemfile` updated to reference the latest version.  Don't worry, this is easy:
+Each project you want to use the newer version of PDQTest on should then have 
+it's `Gemfile.project` updated to reference the latest version.  Don't worry, 
+this is easy:
 
 ```shell
 cd /my/cool/project/to/upgrade
 pdqtest upgrade
-bundle install
+pdk bundle install
 ```
 
-Note that since we're using bundler, you only have to upgrade the modules you want to upgrade.  Existing modules can continue to run any previous version via `make` just fine.
+Note that since we're using bundler, you only have to upgrade the puppet modules
+you want to upgrade.  Existing modules can continue to run any previous version
+via `make` just fine. You are not forced to update all your modules in one go.
 
 ## Docker image
-The docker image will be updated as and when required.  Run:
+Updated docker images are periodically released and are required to run newer
+PDQTest versions. When you get a message about missing docker containers run:
 
 ```shell
 pdqtest setup
@@ -24,5 +29,119 @@ pdqtest setup
 
 To obtain the latest version.
 
-## `/cut`
-Previous versions of PDQTest mounted code at `/cut` (Code Under Test), the new mountpoint is the more obvious `/testcase`
+# PDQTest 1x -> 2x
+PDQTest is now compatible with (and requires) PDK! By and large we let PDK do 
+its own thing and just wrap the output with emojis and run our own acceptance
+tests.
+
+See [PDK Integration](pdk.md) for details of how this works. 
+
+Since this is a major upgrade, you must re-init your project. Make sure all your
+code is checked in to git, 
+[install PDK](https://puppet.com/docs/pdk/1.x/pdk_install.html) then run:
+
+```shell
+gem install pdqtest
+cd /my/project
+pdqtest init
+pdk bundle install
+bundle exec pdqtest setup
+```
+
+## Custom facts
+If you were previous using custom facts in the `spec/merge_facts` directory, 
+these need to be converted to yaml and moved to `spec/default_facts.yml`. This
+will give you compatibility between PDK unit tests and PDQTest acceptance tests.
+
+## RSpec tests (inc hiera data)
+PDQTest doesn't deal with RSPec tests any more, we let PDK do all the work by
+shelling out to call `pdk test unit`:
+
+* If you were using PDQTest for your hiera data during the RSpec lifecycle, 
+  update your tests to do what PDK tells you to do
+* Existing `spec/fixtures/hiera.yaml` and `spec/fixtures/test.yaml` files will
+  continue to work during acceptance tests and the `hiera.yaml` file has been
+  upgraded for hiera 5 compatibility. You will have to reconfigure your 
+  hierarchy if you were using more files then just `test.yaml`
+
+Old PDQTest generated RSpec tests will fail due to missing dependency on 
+[puppet_factset](https://rubygems.org/gems/puppet_factset) gem which is now no
+longer required (by anything 😁).
+
+You can enable this gem in PDK by adding it to `Gemfile.project` and running
+`pdk bundle install` or you can just rewrite your tests using the new PDK format
+which looks like this:
+
+```ruby
+require 'spec_helper'
+
+describe 'CHANGETHISTOYOURCLASSNAME' do
+  on_supported_os.each do |os, os_facts|
+    context "on #{os}" do
+      let(:facts) { os_facts }
+
+      it { is_expected.to compile }
+    end
+  end
+end
+```
+
+If your in a hurry and have good acceptance tests, another quick fix would be to
+just delete the RSpec tests and rely purely on real-world testing. You will miss 
+fast-failure due to invalid puppet code if you do this though, since we have to
+launch a container and run puppet to see if your code compiles.
+
+You can use PDK to generate these tests, see the PDK manual for details of how.
+
+## `/.fixtures.yml` and `/fixtures.yml`
+`/.fixtures.yml` is back in PDQTest 2.0 (it's presence was an error in PDQTest 
+1.x) and `/fixtures.yaml` is no longer used.
+
+If you would like to use test fixtures from git, add them to `.fixtures.yml` and
+you can continue to use them as before. The rest of the file is generated for
+you automatically based on `metadata.json` to support integration with PDK.
+
+[More info on dependencies](puppet_module_dependencies.md)
+
+## Old PDQtest 1x integration points
+* `/spec/spec_helper.rb`
+* `/Gemfile`
+* `/Rakefile`
+* `/.rspec`
+We will do our best to detect any previous PDQTest generated versions of these
+files. If known versions are found, we will upgrade you to the PDK version. If
+an unknown file is found we will stop and ask you to move it out of the way
+first. PDK will be responsible for these files now-on.
+
+### How to resolve `unknown/modified file` errors during upgrade
+1. Move your existing file out of the way
+2. Run `pdqtest init`
+3. See per-file upgrade notes below
+
+#### `/Gemfile`
+Add any custom gems/`Gemfile` magic to `gemfile.project` and it will be evaluated
+during `pdk bundle install`
+
+### PDK Managed files
+PDK regenerates files it manages you run `pdk update`. To customise these files
+and have your changes persist between updates, you need to add them to 
+`.sync.yml` which allows you to override _some_ settings in the 
+[PDK default templates](https://github.com/puppetlabs/pdk-templates/).
+
+If the templates don't support the customisation you want, the advice from 
+Puppet is to fork and customise that repository, then configure PDK to use it.
+
+Consult the PDK documentation for instructions on how to do this. 
+
+## `/Gemfile.project`
+New file used for configuring which PDQTest gem to use (replaces customisation
+of `Gemfile`)
+
+## `/.gitignore`
+We no longer install or manage `.gitignore` for you and we don't upgrade it
+either. PDK ships a default version and we use that if the initial file is 
+missing.
+
+## `/cut` mountpoint
+Previous versions of PDQTest mounted code at `/cut` (Code Under Test), the new
+mountpoint is the more obvious `/testcase` and `/cut` no longer works.

data/doc/windows.md

@@ -0,0 +1,51 @@
+# Windows
+PDQTest now supports windows! There are a few gotchas though:
+
+* The docker volume doesn't play nice with puppet/ruby: Whole app breaks if 
+  `/lib` directory present, so we have to _copy_ the files into the container
+  this is of course very slow. You must run `refresh.ps1` inside the container
+  to refresh the files if your doing TDD on your main workstation.
+* You must run PDQTest from `powershell.exe` and NOT `PowerShell ISE` or you
+  encounter [PDK-1168](https://tickets.puppetlabs.com/browse/PDK-1168)
+* Keep an eye on how many windows containers are running `docker ps` - things
+  seem to choke up if you have more then a handful
+* You **must** develop your modules on your **guest** filesystem (eg not at 
+  `c:\vagrant` if your running windows 10 in a VM on linux):
+  [PDK-1169](https://tickets.puppetlabs.com/browse/PDK-1169)
+
+## FAQ
+Q: How do I just get a shell inside my container? 
+
+A:
+```shell
+.\make shellnopuppet
+```
+
+Q: How do I see GUI apps that are running in my container?
+
+A: Beats me. Maybe there's a way to do something with RDP? The idea of windows
+containers seems to be to run CLI applications/services and interact with them
+through powershell or over the network. If anyone has a better answer let me
+know 
+
+Q: what's going on with `.bat` files?
+
+A:They seem to work but you need to pass the full filename to docker run, eg:
+
+```shell
+docker exec XXX foo.bat
+```
+
+not
+
+```shell
+docker exec XXX foo
+```
+_where XXX is the ID of your container. Use `docker ps` to get a listing.
+
+Q: My module refuses to be found or work! (class not found...)
+
+A: Beats me.. This came up during testing but I haven't investigated further. 
+Make sure you have a `depenedencies` section in `metadata.json` even if its 
+empty or puppet seems to break (unverified)
+

data/docker_images/centos/Dockerfile

@@ -24,6 +24,7 @@ RUN yum install -y cronie \
     gpm-libs \
     policycoreutils \
     policycoreutils-restorecond && yum clean all
+RUN echo "metadata_expire=never" >> /etc/yum.conf
 
 # fix locale
 RUN echo 'LANG=en_US.UTF-8' > /etc/locale.conf
@@ -48,12 +49,12 @@ RUN git clone https://github.com/bats-core/bats-core /usr/local/bats
 # r10k
 RUN /opt/puppetlabs/puppet/bin/gem install r10k
 
-# git refresh (for processing .fixtures.yml git modules)
-RUN /opt/puppetlabs/puppet/bin/gem install git_refresh
-
 # testcase (our module)
 RUN mkdir /testcase
 VOLUME /testcase
 
+# hiera
+RUN mkdir -p /spec/fixtures/
+
 # alternate shudown for systemd
 STOPSIGNAL SIGRTMIN+3

data/docker_images/centos/Makefile

@@ -1,3 +1,3 @@
 all:
 	# requires experimental mode - see https://github.com/docker/docker/tree/master/experimental#use-docker-experimental
-	bash -c 'docker build --squash -t declarativesystems/pdqtest-1x-centos:$$(date -I)-0 .'
+	bash -c 'docker build --squash -t declarativesystems/pdqtest-centos:$$(date -I)-0 .'

data/docker_images/ubuntu/Dockerfile

@@ -53,12 +53,12 @@ RUN git clone https://github.com/bats-core/bats-core /usr/local/bats
 # librarian
 RUN /opt/puppetlabs/puppet/bin/gem install r10k
 
-# git refresh (for processing .fixtures.yml git modules)
-RUN /opt/puppetlabs/puppet/bin/gem install git_refresh
-
 # testcase (our module)
 RUN mkdir /testcase
 VOLUME /testcase
 
+# hiera
+RUN mkdir -p /spec/fixtures/
+
 # alternate shudown for systemd
 STOPSIGNAL SIGRTMIN+3

data/docker_images/windows/Dockerfile

@@ -0,0 +1,26 @@
+FROM microsoft/windowsservercore
+
+RUN powershell -command "& { (New-Object Net.WebClient).DownloadFile('https://pm.puppet.com/puppet-agent/2018.1.4/5.5.6/repos/windows/puppet-agent-5.5.6-x64.msi', 'c:\puppet_agent.msi') }"
+RUN msiexec /qn /norestart /i c:\puppet_agent.msi
+RUN mkdir C:\testcase
+RUN mkdir C:\spec
+RUN mkdir C:\spec\fixtures
+RUN mkdir C:\spec\fixtures\hieradata
+
+
+VOLUME C:\\testcase
+
+# the path from "command prompt with puppet"
+ENV PATH="C:\Program Files\Puppet Labs\Puppet\puppet\bin;C:\Program Files\Puppet Labs\Puppet\facter\bin;C:\Program Files\Puppet Labs\Puppet\hiera\bin;C:\Program Files\Puppet Labs\Puppet\mcollective\bin;C:\Program Files\Puppet Labs\Puppet\bin;C:\Program Files\Puppet Labs\Puppet\sys\ruby\bin;C:\Program Files\Puppet Labs\Puppet\sys\tools\bin;C:\Program Files\Docker\Docker\Resources\bin;C:\tools\ruby25\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\ProgramData\chocolatey\bin;C:\Program Files\Git\cmd;C:\Program Files\Puppet Labs\Puppet\bin;c:\windows:\c:\windows\system32;C:\Program Files (x86)\vim\vim80"
+
+# install chocolatey
+RUN @"%SystemRoot%\System32\WindowsPowerShell\v1.0\powershell.exe" -NoProfile -InputFormat None -ExecutionPolicy Bypass -Command "iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))" && SET "PATH=%PATH%;%ALLUSERSPROFILE%\chocolatey\bin"
+
+RUN choco install vim git /y
+RUN git clone https://github.com/declarativesystems/pats c:\pats
+RUN copy c:\pats\pats.ps1 c:\windows
+
+
+# we dont have systemd/sysvinit on windows so just use notepad to keep the
+# container open (nasty hack)
+ENTRYPOINT notepad

data/docker_images/windows/make.ps1

@@ -0,0 +1,2 @@
+docker build -t declarativesystems/pdqtest-windows:$([datetime]::now.tostring("yyyy-MM-dd"))-0 .
+

data/exe/pdqtest

@@ -11,18 +11,21 @@ end
 require 'pdqtest'
 require 'pdqtest/emoji'
 require 'pdqtest/instance'
-require 'pdqtest/rspec'
 require 'pdqtest/util'
 require 'pdqtest/skeleton'
-require 'pdqtest/lint'
-require 'pdqtest/syntax'
 require 'pdqtest/core'
 require 'pdqtest/upgrade'
+require 'pdqtest/pdk'
 require 'escort'
+require 'pdqtest/logger'
+require 'pdqtest/fastcheck'
 
 # display help if nothing specified
 ARGV.push('-h') if ARGV.empty?
 
+# Setup our own hacky logger
+
+
 Escort::App.create do |app|
   app.version PDQTest::VERSION
   app.summary "pdqtest"
@@ -64,12 +67,21 @@ Escort::App.create do |app|
       :default  => false,
     )
 
+    opts.opt(:debug,
+      'Extra debug messages (also needs --verbosity debug for stack traces)',
+      :long     => '--debug',
+      :type     => :boolean,
+      :default  => false,
+    )
   end
 
   app.command :all do |command|
     command.summary "All"
     command.description "Run all tests"
     command.action do |options, arguments|
+      PDQTest::Logger.logger
+      $logger.level = :debug if options[:global][:options][:debug]
+
       PDQTest::Emoji.disable(options[:global][:options][:disable_emoji])
       PDQTest::Puppet.skip_second_run(options[:global][:options][:skip_idempotency])
 
@@ -78,20 +90,48 @@ Escort::App.create do |app|
       PDQTest::Instance.set_docker_image(options[:global][:options][:image_name])
 
       PDQTest::Core.run([
-        lambda {PDQTest::Syntax.puppet},
-        lambda {PDQTest::Lint.puppet},
-        lambda {PDQTest::Rspec.run},
+        lambda {PDQTest::Pdk.run("validate 'metadata,puppet'")},
+        lambda {PDQTest::Puppet.install_modules},
+        lambda {PDQTest::Pdk.run("test unit")},
         lambda {PDQTest::Instance.run},
+        lambda {system("bundle exec puppet strings generate --format=markdown")},
+        lambda {PDQTest::Pdk.run("build --force")},
+      ])
+    end
+  end
+
+  app.command :fast do |command|
+    command.summary "Fast"
+    command.description "Run basic lint/syntax and acceptance tests (skips rspec)"
+    command.action do |options, arguments|
+      PDQTest::Logger.logger
+      $logger.level = :debug if options[:global][:options][:debug]
+
+      PDQTest::Emoji.disable(options[:global][:options][:disable_emoji])
+      PDQTest::Puppet.skip_second_run(options[:global][:options][:skip_idempotency])
+
+      PDQTest::Instance.set_privileged(options[:global][:options][:privileged])
+      PDQTest::Instance.set_keep_container(options[:global][:options][:keep_container])
+      PDQTest::Instance.set_docker_image(options[:global][:options][:image_name])
+
+      PDQTest::Core.run([
+        lambda {PDQTest::Fastcheck.run},
+        lambda {PDQTest::Puppet.install_modules},
+        lambda {PDQTest::Instance.run},
+        lambda {system("bundle exec puppet strings generate --format=markdown")},
       ])
     end
   end
 
   app.command :rspec do |command|
     command.summary "RSpec"
-    command.description "Run the RSpec tests"
+    command.description "Run the RSpec tests (`pdk test unit` with emojis)"
     command.action do |options, arguments|
+      PDQTest::Logger.logger
+      $logger.level = :debug if options[:global][:options][:debug]
+
       PDQTest::Emoji.disable(options[:global][:options][:disable_emoji])
-      PDQTest::Core.run(lambda {PDQTest::Rspec.run})
+      PDQTest::Core.run(lambda {PDQTest::Pdk.run("test unit")})
     end
   end
 
@@ -108,6 +148,9 @@ Escort::App.create do |app|
       )
     end
     command.action do |options, arguments|
+      PDQTest::Logger.logger
+      $logger.level = :debug if options[:global][:options][:debug]
+
       PDQTest::Emoji.disable(options[:global][:options][:disable_emoji])
       PDQTest::Puppet.skip_second_run(options[:global][:options][:skip_idempotency])
       PDQTest::Instance.set_privileged(options[:global][:options][:privileged])
@@ -122,17 +165,11 @@ Escort::App.create do |app|
     command.summary "Initialise testing"
     command.description "Install skeleton testing configuration into this module"
     command.action do |options, arguments|
-      PDQTest::Emoji.disable(options[:global][:options][:disable_emoji])
-      PDQTest::Skeleton.init
-    end
-  end
+      PDQTest::Logger.logger
+      $logger.level = :debug if options[:global][:options][:debug]
 
-  app.command :generate_rspec do |command|
-    command.summary "Generate RSpec"
-    command.description "For each class in the module, generate basic RSpec tests"
-    command.action do |options, arguments|
       PDQTest::Emoji.disable(options[:global][:options][:disable_emoji])
-      PDQTest::Rspec.gen_specs
+      PDQTest::Skeleton.init
     end
   end
 
@@ -148,6 +185,9 @@ Escort::App.create do |app|
       )
     end
     command.action do |options, arguments|
+      PDQTest::Logger.logger
+      $logger.level = :debug if options[:global][:options][:debug]
+
       PDQTest::Emoji.disable(options[:global][:options][:disable_emoji])
       example = options[:global][:commands][:generate_acceptance][:options][:example]
 
@@ -160,6 +200,9 @@ Escort::App.create do |app|
     command.summary "Shell"
     command.description "Open a shell inside a docker container identical to the test environment before anything has run"
     command.action do |options, arguments|
+      PDQTest::Logger.logger
+      $logger.level = :debug if options[:global][:options][:debug]
+
       PDQTest::Instance.set_docker_image(options[:global][:options][:image_name])
       PDQTest::Instance.shell
     end
@@ -167,20 +210,13 @@ Escort::App.create do |app|
 
   app.command :syntax do |command|
     command.summary "Syntax"
-    command.description "Check for syntax errors"
+    command.description "Check for syntax errors (`pdk validate metadata,puppet` with emojis)"
     command.action do |options, arguments|
-      PDQTest::Emoji.disable(options[:global][:options][:disable_emoji])
-      PDQTest::Core.run(lambda {PDQTest::Syntax.puppet})
-    end
-  end
-
-  app.command :lint do |command|
-    command.summary "Lint"
-    command.description "Check for lint errors"
+      PDQTest::Logger.logger
+      $logger.level = :debug if options[:global][:options][:debug]
 
-    command.action do |options, arguments|
       PDQTest::Emoji.disable(options[:global][:options][:disable_emoji])
-      PDQTest::Core.run(lambda {PDQTest::Lint.puppet})
+      PDQTest::Core.run(lambda {PDQTest::Pdk.run("validate 'metadata,puppet'")})
     end
   end
 
@@ -189,6 +225,9 @@ Escort::App.create do |app|
     command.description "Install the docker container required for testing"
 
     command.action do |options, arguments|
+      PDQTest::Logger.logger
+      $logger.level = :debug if options[:global][:options][:debug]
+
       PDQTest::Core.run(lambda {
         PDQTest::Docker::IMAGES.each { |key, image|
           system("docker pull #{image}")
@@ -201,6 +240,9 @@ Escort::App.create do |app|
     command.summary "Info"
     command.description "Print info about this module"
     command.action do |options, arguments|
+      PDQTest::Logger.logger
+      $logger.level = :debug if options[:global][:options][:debug]
+
       PDQTest::Puppet.info
     end
   end
@@ -210,6 +252,9 @@ Escort::App.create do |app|
     command.summary "Upgrade"
     command.description "Upgrade the current module to use this version of PDQTest"
     command.action do |options, arguments|
+      PDQTest::Logger.logger
+      $logger.level = :debug if options[:global][:options][:debug]
+
       PDQTest::Upgrade.upgrade
     end
   end

data/lib/pdqtest/core.rb

@@ -8,7 +8,7 @@ module PDQTest
       functions = Array(functions)
       functions.each { |f|
         if ! f.call
-          Escort::Logger.error.error "Error encountered running #{f.to_s}"
+          $logger.error "Error encountered running #{f.to_s}"
 
           # epic fail, exit program
           PDQTest::Emoji.final_status(false)