puppetlabs_spec_helper 1.1.1 → 1.2.1

data/{README.markdown → README.md}

@@ -23,12 +23,12 @@ Usage
 =====
 
 When developing or testing modules, simply clone this repository and install the
-gem it contains.
+gem it contains. The recommended way to do this is using [bundler](http://bundler.io/#getting-started).
 
-    $ git clone git://github.com/puppetlabs/puppetlabs_spec_helper.git
-    $ cd puppetlabs_spec_helper
-    $ rake package:gem
-    $ gem install pkg/puppetlabs_spec_helper-*.gem
+Example Gemfile:
+
+    source 'https://rubygems.org'
+    gem 'puppetlabs_spec_helper'
 
 Add this to your project's spec\_helper.rb:
 
@@ -61,6 +61,21 @@ Host github.com
 
 Note: parallel downloads is only available for repositories and not forge modules.
 
+### Parallel tests
+It is also possible to use the `parallel_tests` Gem via the `:parallel_spec` Rake task to run rspec commands in parallel on groups of spec files.
+
+Use of parallelization at this level can result in large performance benefits when the Rspec examples tend to cause a number of large, CPU-intensive catalog compilations to occur.  An example of where this might be the case is in a complex module with a lot of tests or a control repo with many hosts.
+
+Be aware however that in other circumstances this parallelization can result in the tests actually taking longer to run.  The best thing to do is time `rspec spec` and `rspec parallel_spec` and use the parallelization only when there is a clear benefit.
+
+To enable this feature, add the `parallel_tests` Gem to your project's Gemfile:
+
+    gem 'parallel_tests'
+
+And then to run spec tests in parallel:
+
+    $ rake parallel_spec
+
 Issues
 ======
 
@@ -93,6 +108,17 @@ branch of this project from github, and install it as a rubygem.
 There should be few or no cases where you would want to have any other
 branch of this project besides master/HEAD.
 
+Running on non-current ruby versions
+------------------------------------
+
+Since gem and bundler, ruby's package management tools, do not take the target ruby version into account when downloading packages, the puppetlabs_spec_helper gem can only depend on gems that are available for all supported ruby versions. If you can/want to use features from other packages, install those additional packages manually, or have a look at the Gemfile, which provides code to specify those dependencies in a more "friendly" way. This currently affects the following gems:
+
+* puppet
+* rubocop
+* rubocop-rspec
+* json_pure
+* rack
+
 Initializing Puppet for Testing
 ===============================
 
@@ -119,8 +145,8 @@ variables for the spec run. These are:
 * ``FUTURE_PARSER`` - set to "yes" to enable the [future parser](http://docs.puppetlabs.com/puppet/latest/reference/experiments_future.html),
   the equivalent of setting [parser=future](http://docs.puppetlabs.com/references/latest/configuration.html#parser)
   in puppet.conf.
-* ``STRICT_VARIABLES`` - set to "yes" to enable strict variable checking,
-  the equivalent of setting [strict_variables](http://docs.puppetlabs.com/references/latest/configuration.html#strictvariables)=true
+* ``STRICT_VARIABLES`` - set to "no" to disable strict variable checking when using Puppet >= 4.0,
+  the equivalent of not setting [strict_variables](http://docs.puppetlabs.com/references/latest/configuration.html#strictvariables)
   in puppet.conf.
 * ``ORDERING`` - set to the desired ordering method ("title-hash", "manifest", or "random")
   to set the order of unrelated resources when applying a catalog. Leave unset for the default
@@ -140,6 +166,18 @@ and manifest ordering, you would:
 
     FUTURE_PARSER=yes STRICT_VARIABLES=yes ORDERING=manifest rake spec
 
+When executing tests in a matrix CI environment, tests can be split up to run
+a share of specs per CI node in parallel.  Set the ``CI_NODE_TOTAL`` environment
+variable to the total number of nodes, and the ``CI_NODE_INDEX`` to a number
+between 1 and the ``CI_NODE_TOTAL``.
+
+If using Travis CI, add new lines to the "env" section of .travis.yml per node,
+remembering to duplicate any existing environment variables:
+
+    env:
+      - FUTURE_PARSER=yes CI_NODE_TOTAL=2 CI_NODE_INDEX=1
+      - FUTURE_PARSER=yes CI_NODE_TOTAL=2 CI_NODE_INDEX=2
+
 Using Utility Classes
 =====================
 If you'd like to use the Utility classes (PuppetlabsSpec::Files,
@@ -158,24 +196,29 @@ Using Fixtures
 `puppetlabs_spec_helper` has the ability to populate the
 `spec/fixtures/modules` directory with dependent modules when `rake spec` or
 `rake spec_prep` is run. To do so, all required modules should be listed in a
-file named `.fixtures.yml` in the root of the project.
+file named `.fixtures.yml` in the root of the project. You can specify a alternate location for that file in the `FIXTURES_YML` environment variable.
 
 When specifying the repo source of the fixture you have a few options as to which revision of the codebase you wish to use.
 
  * repo - the url to the repo
  * scm - options include git or hg. This is an optional step as the helper code will figure out which scm is used.
+
    ```yaml
    scm: git
    scm: hg
    ```
+
  * target - the directory name to clone the repo into ie. `target: mymodule`  defaults to the repo name  (Optional)
+ * subdir - directory to be removed from the cloned repo. Its contents will be moved to the root directory (Optional)
  * ref - used to specify the tag name like version hash of commit (Optional)
+
    ```yaml
    ref: 1.0.0
    ref: 880fca52c
    ```
  * branch - used to specify the branch name you want to use ie. `branch: development`
  * flags - additional flags passed to the module installer (both puppet and scm)
+
    ```yaml
    flags: --verbose
    ```
@@ -187,86 +230,164 @@ Fixtures Examples
 Basic fixtures that will symlink `spec/fixtures/modules/my_modules` to the
 project root:
 
-    fixtures:
-      symlinks:
-        my_module: "#{source_dir}"
+```yaml
+fixtures:
+  symlinks:
+    my_module: "#{source_dir}"
+```
 
+This is the same as specifying no symlinks fixtures at all.
 
 Add `firewall` and `stdlib` as required module fixtures:
 
-    fixtures:
-      repositories:
-        firewall: "git://github.com/puppetlabs/puppetlabs-firewall"
-        stdlib: "git://github.com/puppetlabs/puppetlabs-stdlib"
-      symlinks:
-        my_module: "#{source_dir}"
+```yaml
+fixtures:
+  repositories:
+    firewall: "git://github.com/puppetlabs/puppetlabs-firewall"
+    stdlib: "git://github.com/puppetlabs/puppetlabs-stdlib"
+```
 
 Specify that the git tag `2.4.2` of `stdlib' should be checked out:
 
-    fixtures:
-      repositories:
-        firewall: "git://github.com/puppetlabs/puppetlabs-firewall"
-        stdlib:
-          repo: "git://github.com/puppetlabs/puppetlabs-stdlib"
-          ref: "2.6.0"
-      symlinks:
-        my_module: "#{source_dir}"
+```yaml
+fixtures:
+  repositories:
+    firewall: "git://github.com/puppetlabs/puppetlabs-firewall"
+    stdlib:
+      repo: "git://github.com/puppetlabs/puppetlabs-stdlib"
+      ref: "2.6.0"
+```
+
+Move manifests and siblings to root directory when they are inside a `code` directory:
+
+```yaml
+fixtures:
+  repositories:
+    stdlib:
+      repo: "git://github.com/puppetlabs/puppetlabs-extradirectory"
+      subdir: "code"
+```
 
 Install modules from Puppet Forge:
 
-    fixtures:
-        forge_modules:
-            firewall: "puppetlabs/firewall"
-            stdlib:
-                repo: "puppetlabs/stdlib"
-                ref: "2.6.0"
+```yaml
+fixtures:
+  forge_modules:
+    firewall: "puppetlabs/firewall"
+    stdlib:
+      repo: "puppetlabs/stdlib"
+      ref: "2.6.0"
+```
 
 Pass additional flags to module installation:
 
-    fixtures:
-        forge_modules:
-            stdlib:
-                repo: "puppetlabs/stdlib"
-                ref: "2.6.0"
-                flags: "--module_repository https://my_repo.com"
-        repositories:
-            firewall:
-                repo: "git://github.com/puppetlabs/puppetlabs-firewall"
-                ref: "2.6.0"
-                flags: "--verbose"
+```yaml
+fixtures:
+  forge_modules:
+    stdlib:
+      repo: "puppetlabs/stdlib"
+      ref: "2.6.0"
+      flags: "--module_repository https://my_repo.com"
+    repositories:
+      firewall:
+        repo: "git://github.com/puppetlabs/puppetlabs-firewall"
+        ref: "2.6.0"
+        flags: "--verbose"
+```
 
 Testing Parser Functions
 ========================
 
-This library provides a consistent way to create a Puppet::Parser::Scope object
-suitable for use in a testing harness with the intent of testing the expected
-behavior of parser functions distributed in modules.
+This whole section is superseded by improved support of accessing the scope in
+rspec-puppet.
+
+Generating code coverage reports
+================================
+
+This section describes how to add code coverage reports for Ruby files (types, providers, ...).
+See the documentation of [RSpec-Puppet](https://github.com/rodjek/rspec-puppet)
+for Puppet manifest coverage reports.
+
+Starting with Ruby 1.9, the *de facto* standard for Ruby code coverage is
+[SimpleCov](https://github.com/colszowka/simplecov).
+You can add it to your module like this:
+
+```Ruby
+# First line of spec/spec_helper.rb
+require 'simplecov'
+
+SimpleCov.start do
+  add_filter '/spec/'
+  # Exclude bundled Gems in `/.vendor/`
+  add_filter '/.vendor/'
+end
+
+require 'puppetlabs_spec_helper/module_spec_helper'
+# Further content
+```
+
+The reports will then be generated every time you invoke RSpec, e.g. via `rake spec`,
+and are written to `/coverage/`, which you should add to `.gitignore`.
+
+Remember to add `gem 'simplecov', require: false` to your `Gemfile`.
+
+Using Code Climate
+------------------
 
-Previously, modules would do something like this:
+You can also use [Code Climate](https://codeclimate.com/) together with SimpleCov:
 
-    describe "split()" do
-      let(:scope) { Puppet::Parser::Scope.new }
-      it "should split 'one;two' on ';' into [ 'one', 'two' ]" do
-        scope.function_split(['one;two', ';']).should == [ 'one', 'two' ]
-      end
-    end
+```Ruby
+# First line of spec/spec_helper.rb
+require 'codeclimate-test-reporter'
 
-This will not work beyond Puppet 2.7 as we have changed the behavior of the
-scope initializer in Puppet 3.0.  Modules should instead initialize scope
-instances in a manner decoupled from the internal behavior of Puppet:
+SimpleCov.formatters = [
+  SimpleCov::Formatter::HTMLFormatter,
+  CodeClimate::TestReporter::Formatter
+]
 
-    require 'puppetlabs_spec_helper/puppetlabs_spec/puppet_internals'
-    describe "split()" do
-      let(:scope) { PuppetlabsSpec::PuppetInternals.scope }
-      it "should split 'one;two' on ';' into [ 'one', 'two' ]" do
-        scope.function_split(['one;two', ';']).should == [ 'one', 'two' ]
-      end
-    end
+SimpleCov.start do
+  add_filter '/spec/'
+  # Exclude bundled Gems in `/.vendor/`
+  add_filter '/.vendor/'
+end
+
+require 'puppetlabs_spec_helper/module_spec_helper'
+# Further content
+```
+
+Remember to add `gem 'codeclimate-test-reporter', require: false` to your `Gemfile`.
+
+Using Coveralls
+---------------
+
+You can also use [Coveralls](https://coveralls.io/) together with SimpleCov:
+
+```Ruby
+# First line of spec/spec_helper.rb
+require 'simplecov'
+require 'coveralls'
+
+SimpleCov.formatters = [
+  SimpleCov::Formatter::HTMLFormatter,
+  Coveralls::SimpleCov::Formatter
+]
+
+SimpleCov.start do
+  add_filter '/spec/'
+  # Exclude bundled Gems in `/.vendor/`
+  add_filter '/.vendor/'
+end
+
+require 'puppetlabs_spec_helper/module_spec_helper'
+# Further content
+```
+
+Remember to add `gem 'coveralls', require: false` to your `Gemfile`.
 
 Some Notes for Windows Users
-==============
+============================
 
-A windows users may need to do one of two things to execute 'rake spec'.
+A windows users may need to do one of two things to execute `rake spec`.
 
 Although things may appear to work, the init.pp may not transfer to the fixtures folder as needed
 or may transfer as an empty file.

data/Rakefile

@@ -1,50 +1,15 @@
 # encoding: utf-8
+require "bundler/gem_tasks"
+require "rspec/core/rake_task"
 
-require 'rubygems'
-require 'bundler'
-begin
-  Bundler.setup(:default, :development)
-rescue Bundler::BundlerError => e
-  $stderr.puts e.message
-  $stderr.puts "Run `bundle install` to install missing gems"
-  exit e.status_code
-end
-require 'rake'
-require_relative 'lib/puppetlabs_spec_helper/version'
-require 'jeweler'
-Jeweler::Tasks.new do |gem|
-  # gem is a Gem::Specification... see http://docs.rubygems.org/read/chapter/20 for more options
-  gem.name = "puppetlabs_spec_helper"
-  gem.version = "#{PuppetlabsSpecHelper::Version::STRING}"
-  gem.homepage = "http://github.com/puppetlabs/puppetlabs_spec_helper"
-  gem.license = "Apache-2.0"
-  gem.summary = %Q{Standard tasks and configuration for module spec tests}
-  gem.description = %Q{Contains rake tasks and a standard spec_helper for running spec tests on puppet modules}
-  gem.email = ["modules-dept@puppetlabs.com"]
-  gem.authors = ["Puppet Labs"]
-  # dependencies defined in Gemfile
-end
-Jeweler::RubygemsDotOrgTasks.new
+task :default => :spec
 
-require 'rspec/core'
-require 'rspec/core/rake_task'
 RSpec::Core::RakeTask.new(:spec) do |spec|
   spec.pattern = FileList['spec/**/*_spec.rb'].exclude('spec/fixtures/**/*_spec.rb')
 end
 
-namespace :git do
-  desc "Create a new tag that uses #{PuppetlabsSpecHelper::Version::STRING} as the tag"
-  task :tag do
-    `git tag -m '#{PuppetlabsSpecHelper::Version::STRING}'`
-  end
-  desc "Tag and push to master"
-  task :pl_release do
-    Rake::Task["git:tag"].invoke
-    `git push origin master --tags`
-  end
-end
-
-task :default => :spec
+require 'rubocop/rake_task'
+RuboCop::RakeTask.new
 
 require 'yard'
 YARD::Rake::YardocTask.new

data/lib/puppetlabs_spec_helper/module_spec_helper.rb

@@ -8,7 +8,7 @@ end
 
 def verify_contents(subject, title, expected_lines)
   content = subject.resource('file', title).send(:parameters)[:content]
-  (content.split("\n") & expected_lines).should == expected_lines
+  expect(content.split("\n") & expected_lines).to match_array expected_lines.uniq
 end
 
 spec_path = File.expand_path(File.join(Dir.pwd, 'spec'))
@@ -24,12 +24,6 @@ RSpec.configure do |c|
   c.module_path = module_path
   c.manifest_dir = File.join(fixture_path, 'manifests')
   c.parser = 'future' if ENV['FUTURE_PARSER'] == 'yes'
-  ## This depends on rspec-puppet #209 being released
-  #c.strict_variables = true if ENV['STRICT_VARIABLES'] == 'yes'
-  ## These depend on rspec-puppet #212 being released
-  #c.stringify_facts = false if ENV['STRINGIFY_FACTS'] == 'no'
-  #c.trusted_node_data = true if ENV['TRUSTED_NODE_DATA'] == 'yes'
-  #c.ordering = ENV['ORDERING'] if ENV['ORDERING']
 
   c.before :each do
     Puppet.features.stubs(:root?).returns(true)
@@ -38,7 +32,7 @@ RSpec.configure do |c|
       Puppet.settings[:stringify_facts] = false if ENV['STRINGIFY_FACTS'] == 'no'
       Puppet.settings[:trusted_node_data] = true if ENV['TRUSTED_NODE_DATA'] == 'yes'
     end
-    Puppet.settings[:strict_variables] = true if ENV['STRICT_VARIABLES'] == 'yes'
+    Puppet.settings[:strict_variables] = (ENV['STRICT_VARIABLES'] == 'yes' or (Puppet.version.to_f >= 4.0 and ENV['STRICT_VARIABLES'] != 'no'))
     Puppet.settings[:ordering] = ENV['ORDERING'] if ENV['ORDERING']
   end
 end

data/lib/puppetlabs_spec_helper/puppetlabs_spec/puppet_internals.rb

@@ -8,6 +8,8 @@ module PuppetlabsSpec
     # instance suitable for placing in a test harness with the intent of
     # testing parser functions from modules.
     def scope(parts = {})
+      RSpec.deprecate('scope', :replacement => 'rspec-puppet 2.2.0 provides a scope property')
+
       if Puppet.version =~ /^2\.[67]/
         # loadall should only be necessary prior to 3.x
         # Please note, loadall needs to happen first when creating a scope, otherwise