pdk 1.7.1 → 1.8.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: '08eefaca77036460baaf59107c23f00582b61244'
-  data.tar.gz: '078a7e34d073f5e9571642188d4983d046284fa1'
+  metadata.gz: 45b3709175917e4c02f2195a9333c931393af8d4
+  data.tar.gz: 18e7cb21971f1e6f56906bba22bc46959724616d
 SHA512:
-  metadata.gz: 89d63ff36f12e738d768c1fc592fd6bc8835ff0dd4306f9d5ea10a6cf285d532e3bff21c1075420c7114f557e5fb713874615e13a1749a653c225a945aea1bac
-  data.tar.gz: 58ebcc413564a0866efc92af272b0d7fe26e5f961a9b7625be38c75969a4f2307b8e9339b564916aa9cd3beda8849b93016921c6ba491192e9d0932cb583ef29
+  metadata.gz: fb148b8e30ac0af4974bbda5e1b379edfba820f80f7d13e803ec06d317e3feece0bd81ce59e36f9077c6a52d33c445ef41be394391fef0481581d31e13b5d718
+  data.tar.gz: 8a449fe30e4af21e5316d14f9f9df235c866c702eb6d9ec59b8747e342587ed60b4b7a91ab9756610f4790f75ab2ddec4ad0e3aaafceff98180bc107c0e78e9f

data/CHANGELOG.md

@@ -4,6 +4,42 @@ All changes to this repo will be documented in this file.
 See the [release notes](https://puppet.com/docs/pdk/latest/release_notes.html) for a high-level summary.
 
 
+## [v1.8.0](https://github.com/puppetlabs/pdk/tree/v1.8.0) (2018-11-27)
+[Full Changelog](https://github.com/puppetlabs/pdk/compare/v1.7.1...v1.8.0)
+
+**Implemented enhancements:**
+
+- \(PDK-1090\) Add task name validator for existing tasks [\#598](https://github.com/puppetlabs/pdk/pull/598) ([rodjek](https://github.com/rodjek))
+
+**Fixed bugs:**
+
+- \(PDK-1180\) Cleanly handle a null pdk-version in metadata.json [\#599](https://github.com/puppetlabs/pdk/pull/599) ([rodjek](https://github.com/rodjek))
+- \(PDK-1104\) Don't always override custom template url with default [\#597](https://github.com/puppetlabs/pdk/pull/597) ([rodjek](https://github.com/rodjek))
+- \(PDK-654\) Allow rubocop to determine its own targets by default [\#594](https://github.com/puppetlabs/pdk/pull/594) ([rodjek](https://github.com/rodjek))
+- \(PDK-1187\) Don't override bundler path on gem installs [\#592](https://github.com/puppetlabs/pdk/pull/592) ([rodjek](https://github.com/rodjek))
+- \(PDK-547\) Ensure all PDK created files use LF line endings [\#590](https://github.com/puppetlabs/pdk/pull/590) ([rodjek](https://github.com/rodjek))
+- \(PDK-1172\) Call PDK::Util::Bundler.ensure\_bundle! after module creation [\#589](https://github.com/puppetlabs/pdk/pull/589) ([rodjek](https://github.com/rodjek))
+- \(PDK-1192\) Add module\_root/vendor/ to default ignored paths [\#588](https://github.com/puppetlabs/pdk/pull/588) ([rodjek](https://github.com/rodjek))
+- \(PDK-1194\) Exclude plans/\*\*/\*.pp from PDK::Validate::PuppetSyntax [\#586](https://github.com/puppetlabs/pdk/pull/586) ([rodjek](https://github.com/rodjek))
+- \(PDK-972\) Don't register a pending change when deleting non-existent files [\#585](https://github.com/puppetlabs/pdk/pull/585) ([rodjek](https://github.com/rodjek))
+- \(PDK-1093\) Replace null values and empty data structures in metadata when converting [\#584](https://github.com/puppetlabs/pdk/pull/584) ([rodjek](https://github.com/rodjek))
+- \(PDK-400\) Output the rspec run wall time in test unit summary [\#583](https://github.com/puppetlabs/pdk/pull/583) ([rodjek](https://github.com/rodjek))
+- \(PDK-1200\) Fix bundle env handling with puppet-dev [\#579](https://github.com/puppetlabs/pdk/pull/579) ([bmjen](https://github.com/bmjen))
+- \(PDK-925\) Exclude files that wouldn't be packaged from being validated [\#578](https://github.com/puppetlabs/pdk/pull/578) ([rodjek](https://github.com/rodjek))
+
+**Closed issues:**
+
+- Pdk validate should not assume that all puppet: URL require 'modules/'. [\#591](https://github.com/puppetlabs/pdk/issues/591)
+- Create configuration to Overide default parameters [\#542](https://github.com/puppetlabs/pdk/issues/542)
+
+**Merged pull requests:**
+
+- \(maint\) Update package tests to add task name validation [\#600](https://github.com/puppetlabs/pdk/pull/600) ([bmjen](https://github.com/bmjen))
+- \(maint\) Fix package tests [\#596](https://github.com/puppetlabs/pdk/pull/596) ([rodjek](https://github.com/rodjek))
+- \(maint\) Add --skip-bundle-install to `pdk new module` [\#595](https://github.com/puppetlabs/pdk/pull/595) ([rodjek](https://github.com/rodjek))
+- \(PDK-1208\) Raise lower bound of 'puppet' requirement for new modules [\#581](https://github.com/puppetlabs/pdk/pull/581) ([scotje](https://github.com/scotje))
+- \(maint\) Bump version for dev [\#577](https://github.com/puppetlabs/pdk/pull/577) ([bmjen](https://github.com/bmjen))
+
 ## [v1.7.1](https://github.com/puppetlabs/pdk/tree/v1.7.1) (2018-10-05)
 [Full Changelog](https://github.com/puppetlabs/pdk/compare/v1.7.0...v1.7.1)
 
@@ -20,6 +56,7 @@ See the [release notes](https://puppet.com/docs/pdk/latest/release_notes.html) f
 
 **Merged pull requests:**
 
+- Release Prep for 1.7.1 [\#576](https://github.com/puppetlabs/pdk/pull/576) ([bmjen](https://github.com/bmjen))
 - \(maint\) Update PDK metadata defaults to include Puppet 6 [\#572](https://github.com/puppetlabs/pdk/pull/572) ([bmjen](https://github.com/bmjen))
 - \(maint\) Update package tests for ruby 2.5.1 as the new default [\#571](https://github.com/puppetlabs/pdk/pull/571) ([bmjen](https://github.com/bmjen))
 - Bump version to 1.8.0.pre [\#564](https://github.com/puppetlabs/pdk/pull/564) ([bmjen](https://github.com/bmjen))

data/lib/pdk/answer_file.rb

@@ -1,4 +1,5 @@
 require 'json'
+require 'pdk/util/filesystem'
 
 module PDK
   # Singleton accessor to the current answer file being used by the PDK.
@@ -21,6 +22,8 @@ module PDK
     attr_reader :answers
     attr_reader :answer_file_path
 
+    include PDK::Util::Filesystem
+
     # Initialises the AnswerFile object, which stores the responses to certain
     # interactive questions.
     #
@@ -107,9 +110,7 @@ module PDK
     def save_to_disk
       FileUtils.mkdir_p(File.dirname(answer_file_path))
 
-      File.open(answer_file_path, 'w') do |answer_file|
-        answer_file.puts JSON.pretty_generate(answers)
-      end
+      write_file(answer_file_path, JSON.pretty_generate(answers))
     rescue SystemCallError, IOError => e
       raise PDK::CLI::FatalError, _("Unable to write '%{file}': %{msg}") % {
         file: answer_file_path,

data/lib/pdk/cli.rb

@@ -39,7 +39,9 @@ module PDK::CLI
   end
 
   def self.template_url_option(dsl)
-    dsl.option nil, 'template-url', _('Specifies the URL to the template to use when creating new modules or classes.'), argument: :required, default: PDK::Util.default_template_url
+    desc = _('Specifies the URL to the template to use when creating new modules or classes. (default: %{default_url})') % { default_url: PDK::Util.default_template_url }
+
+    dsl.option nil, 'template-url', desc, argument: :required
   end
 
   def self.skip_interview_option(dsl)

data/lib/pdk/cli/new/module.rb

@@ -10,6 +10,7 @@ module PDK::CLI
 
     option nil, 'license', _('Specifies the license this module is written under. ' \
       "This should be a identifier from https://spdx.org/licenses/. Common values are 'Apache-2.0', 'MIT', or 'proprietary'."), argument: :required
+    option nil, 'skip-bundle-install', _('Do not automatically run `bundle install` after creating the module.'), hidden: true
 
     run do |opts, args, _cmd|
       require 'pdk/generate/module'

data/lib/pdk/cli/test/unit.rb

@@ -70,6 +70,9 @@ module PDK::CLI
         puppet_env = PDK::CLI::Util.puppet_from_opts_or_env(opts)
         PDK::Util::PuppetVersion.fetch_puppet_dev if opts.key?(:'puppet-dev')
         PDK::Util::RubyVersion.use(puppet_env[:ruby_version])
+
+        opts.merge!(puppet_env[:gemset])
+
         PDK::Util::Bundler.ensure_bundle!(puppet_env[:gemset])
 
         exit_code = PDK::Test::Unit.invoke(report, opts)

data/lib/pdk/cli/validate.rb

@@ -6,7 +6,7 @@ module PDK::CLI
     usage _('validate [validators] [options] [targets]')
     summary _('Run static analysis tests.')
     description _(
-      "Run metadata, Puppet, or Ruby validation.\n\n" \
+      "Run metadata, Puppet, Ruby, or Tasks validation.\n\n" \
       '[validators] is an optional comma-separated list of validators to use. ' \
       'If not specified, all validators are used. ' \
       "Note that when using PowerShell, the list of validators must be enclosed in single quotes.\n\n" \
@@ -91,6 +91,9 @@ module PDK::CLI
       puppet_env = PDK::CLI::Util.puppet_from_opts_or_env(opts)
       PDK::Util::PuppetVersion.fetch_puppet_dev if opts.key?(:'puppet-dev')
       PDK::Util::RubyVersion.use(puppet_env[:ruby_version])
+
+      options.merge!(puppet_env[:gemset])
+
       PDK::Util::Bundler.ensure_bundle!(puppet_env[:gemset])
 
       exit_code = 0

data/lib/pdk/generate/module.rb

@@ -17,6 +17,8 @@ require 'pdk/util/version'
 module PDK
   module Generate
     class Module
+      extend PDK::Util::Filesystem
+
       def self.validate_options(opts)
         unless PDK::CLI::Util::OptionValidator.valid_module_name?(opts[:module_name])
           error_msg = _(
@@ -40,7 +42,7 @@ module PDK
 
         begin
           test_file = File.join(parent_dir, '.pdk-test-writable')
-          File.open(test_file, 'w') { |f| f.write('This file was created by the Puppet Development Kit to test if this folder was writable, you can safely remove this file.') }
+          write_file(test_file, 'This file was created by the Puppet Development Kit to test if this folder was writable, you can safely remove this file.')
           FileUtils.rm_f(test_file)
         rescue Errno::EACCES
           raise PDK::CLI::FatalError, _("You do not have permission to write to '%{parent_dir}'") % {
@@ -59,7 +61,7 @@ module PDK
             templates.render do |file_path, file_content|
               file = Pathname.new(temp_target_dir) + file_path
               file.dirname.mkpath
-              file.write(file_content)
+              write_file(file, file_content)
             end
 
             # Add information about the template used to generate the module to the
@@ -84,6 +86,8 @@ module PDK
 
         begin
           if FileUtils.mv(temp_target_dir, target_dir)
+            Dir.chdir(target_dir) { PDK::Util::Bundler.ensure_bundle! } unless opts[:'skip-bundle-install']
+
             PDK.logger.info(_('Module \'%{name}\' generated at path \'%{path}\', from template \'%{template_url}\'.') % { name: opts[:module_name], path: target_dir, template_url: template_url })
             PDK.logger.info(_('In your module directory, add classes with the \'pdk new class\' command.'))
           end

data/lib/pdk/generate/puppet_object.rb

@@ -5,6 +5,7 @@ require 'pdk/logger'
 require 'pdk/module/metadata'
 require 'pdk/module/templatedir'
 require 'pdk/template_file'
+require 'pdk/util/filesystem'
 
 module PDK
   module Generate
@@ -180,7 +181,7 @@ module PDK
           }
         end
 
-        File.open(dest_path, 'w') { |f| f.write file_content }
+        PDK::Util::Filesystem.write_file(dest_path, file_content)
       rescue SystemCallError => e
         raise PDK::CLI::FatalError, _("Unable to write to file '%{path}': %{message}") % {
           path:    dest_path,

data/lib/pdk/module.rb

@@ -0,0 +1,21 @@
+require 'pathspec'
+
+module PDK
+  module Module
+    DEFAULT_IGNORED = [
+      '/pkg/',
+      '.*',
+      '~*',
+      '/coverage',
+      '/checksums.json',
+      '/REVISION',
+      '/spec/fixtures/modules/',
+      '/vendor/',
+    ].freeze
+
+    def default_ignored_pathspec
+      @default_ignored_pathspec ||= PathSpec.new(DEFAULT_IGNORED)
+    end
+    module_function :default_ignored_pathspec
+  end
+end

data/lib/pdk/module/build.rb

@@ -3,21 +3,12 @@ require 'minitar'
 require 'zlib'
 require 'pathspec'
 require 'find'
+require 'pdk/module'
 require 'pdk/tests/unit'
 
 module PDK
   module Module
     class Build
-      DEFAULT_IGNORED = [
-        '/pkg/',
-        '.*',
-        '~*',
-        '/coverage',
-        '/checksums.json',
-        '/REVISION',
-        '/spec/fixtures/modules/',
-      ].freeze
-
       def self.invoke(options = {})
         new(options).build
       end
@@ -213,7 +204,7 @@ module PDK
               ignored = ignored.add("\/#{File.basename(target_dir)}\/")
             end
 
-            DEFAULT_IGNORED.each { |r| ignored.add(r) }
+            PDK::Module::DEFAULT_IGNORED.each { |r| ignored.add(r) }
 
             ignored
           end

data/lib/pdk/module/convert.rb

@@ -108,19 +108,22 @@ module PDK
 
       def update_metadata(metadata_path, template_metadata)
         if File.file?(metadata_path)
-          if File.readable?(metadata_path)
-            begin
-              metadata = PDK::Module::Metadata.from_file(metadata_path)
-              new_values = PDK::Module::Metadata::DEFAULTS.reject { |key, _| metadata.data.key?(key) }
-              metadata.update!(new_values)
-            rescue ArgumentError
-              metadata = PDK::Generate::Module.prepare_metadata(options) unless options[:noop] # rubocop:disable Metrics/BlockNesting
-            end
-          else
+          unless File.readable?(metadata_path)
             raise PDK::CLI::ExitWithError, _('Unable to update module metadata; %{path} exists but it is not readable.') % {
               path: metadata_path,
             }
           end
+
+          begin
+            metadata = PDK::Module::Metadata.from_file(metadata_path)
+            new_values = PDK::Module::Metadata::DEFAULTS.select do |key, _|
+              !metadata.data.key?(key) || metadata.data[key].nil? ||
+                (key == 'requirements' && metadata.data[key].empty?)
+            end
+            metadata.update!(new_values)
+          rescue ArgumentError
+            metadata = PDK::Generate::Module.prepare_metadata(options) unless options[:noop]
+          end
         elsif File.exist?(metadata_path)
           raise PDK::CLI::ExitWithError, _('Unable to update module metadata; %{path} exists but it is not a file.') % {
             path: metadata_path,

data/lib/pdk/module/metadata.rb

@@ -1,10 +1,13 @@
 require 'json'
+require 'pdk/util/filesystem'
 
 module PDK
   module Module
     class Metadata
       attr_accessor :data
 
+      include PDK::Util::Filesystem
+
       OPERATING_SYSTEMS = {
         'RedHat based Linux' => [
           {
@@ -77,7 +80,7 @@ module PDK
           OPERATING_SYSTEMS[os_name]
         }.flatten,
         'requirements' => [
-          { 'name' => 'puppet', 'version_requirement' => '>= 4.7.0 < 7.0.0' },
+          { 'name' => 'puppet', 'version_requirement' => '>= 4.10.0 < 7.0.0' },
         ],
       }.freeze
 
@@ -120,9 +123,7 @@ module PDK
       end
 
       def write!(path)
-        File.open(path, 'w') do |file|
-          file.puts to_json
-        end
+        write_file(path, to_json)
       end
 
       def forge_ready?

data/lib/pdk/module/update_manager.rb

@@ -3,6 +3,7 @@ require 'diff/lcs/hunk'
 require 'English'
 require 'fileutils'
 require 'set'
+require 'pdk/util/filesystem'
 
 module PDK
   module Module
@@ -48,7 +49,7 @@ module PDK
 
         {
           added:    @added_files,
-          removed:  @removed_files,
+          removed:  @removed_files.select { |f| File.exist?(f) },
           modified: @diff_cache.reject { |_, value| value.nil? },
         }
       end
@@ -148,7 +149,7 @@ module PDK
       def write_file(path, content)
         FileUtils.mkdir_p(File.dirname(path))
         PDK.logger.debug(_("writing '%{path}'") % { path: path })
-        File.open(path, 'w') { |f| f.puts content }
+        PDK::Util::Filesystem.write_file(path, content)
       rescue Errno::EACCES
         raise PDK::CLI::ExitWithError, _("You do not have permission to write to '%{path}'") % { path: path }
       end

data/lib/pdk/tests/unit.rb

@@ -69,8 +69,11 @@ module PDK
         setup
 
         tests = options.fetch(:tests)
+
+        environment = { 'CI_SPEC_OPTIONS' => '--format j' }
+        environment['PUPPET_GEM_VERSION'] = options[:puppet] if options[:puppet]
         spinner_msg = options.key?(:parallel) ? _('Running unit tests in parallel.') : _('Running unit tests.')
-        result = rake(cmd(tests, options), spinner_msg, 'CI_SPEC_OPTIONS' => '--format j')
+        result = rake(cmd(tests, options), spinner_msg, environment)
 
         json_result = if options.key?(:parallel)
                         PDK::Util.find_all_json_in(result[:stdout])
@@ -85,16 +88,16 @@ module PDK
 
         raise PDK::CLI::FatalError, _('Unit test output did not contain a valid JSON result: %{output}') % { output: result[:stdout] } if json_result.nil? || json_result.empty?
 
-        json_result = merge_json_results(json_result, result[:duration]) if options.key?(:parallel)
+        json_result = merge_json_results(json_result) if options.key?(:parallel)
 
-        parse_output(report, json_result)
+        parse_output(report, json_result, result[:duration])
 
         result[:exit_code]
       ensure
         tear_down if options[:'clean-fixtures']
       end
 
-      def self.parse_output(report, json_data)
+      def self.parse_output(report, json_data, duration)
         # Output messages to stderr.
         json_data['messages'] && json_data['messages'].each { |msg| $stderr.puts msg }
 
@@ -137,13 +140,13 @@ module PDK
         # TODO: standardize summary output
         $stderr.puts '  ' << _('Evaluated %{total} tests in %{duration} seconds: %{failures} failures, %{pending} pending.') % {
           total: json_data['summary']['example_count'],
-          duration: json_data['summary']['duration'],
+          duration: duration,
           failures: json_data['summary']['failure_count'],
           pending: json_data['summary']['pending_count'],
         }
       end
 
-      def self.merge_json_results(json_data, duration)
+      def self.merge_json_results(json_data)
         merged_json_result = {}
 
         # Merge messages
@@ -164,7 +167,6 @@ module PDK
 
         # Merge summaries
         summary_hash = {
-          'duration'      => duration,
           'example_count' => 0,
           'failure_count' => 0,
           'pending_count' => 0,

data/lib/pdk/util.rb

@@ -4,6 +4,7 @@ require 'tempfile'
 require 'pdk/util/version'
 require 'pdk/util/windows'
 require 'pdk/util/vendored_file'
+require 'pdk/util/filesystem'
 
 module PDK
   module Util
@@ -256,10 +257,10 @@ module PDK
     def module_pdk_version
       metadata = module_metadata
 
-      if !metadata.nil? && metadata.include?('pdk-version')
-        metadata['pdk-version'].split.first
-      else
+      if metadata.nil? || metadata.fetch('pdk-version', nil).nil?
         nil
+      else
+        metadata['pdk-version'].split.first
       end
     rescue ArgumentError => e
       PDK.logger.error(e)

data/lib/pdk/util/bundler.rb

@@ -46,8 +46,9 @@ module PDK
         bundle.update_lock!(with: gem_overrides, local: all_deps_available)
 
         # If there are missing dependencies after updating the lockfile, let `bundle install`
-        # go out and get them.
-        unless bundle.installed?
+        # go out and get them. If the specified puppet gem version points to a remote location
+        # or local filepath, then run bundle install as well.
+        if !bundle.installed? || (gem_overrides[:puppet] && gem_overrides[:puppet].start_with?('file://', 'git://', 'https://'))
           bundle.install!(gem_overrides)
         end
 
@@ -96,7 +97,6 @@ module PDK
           PDK.logger.debug(_('Checking for missing Gemfile dependencies.'))
 
           argv = ['check', "--gemfile=#{gemfile}", '--dry-run']
-          argv << "--path=#{bundle_cachedir}" unless PDK::Util.package_install?
 
           cmd = bundle_command(*argv).tap do |c|
             c.update_environment(gemfile_env(gem_overrides)) unless gem_overrides.empty?
@@ -191,7 +191,6 @@ module PDK
         def install!(gem_overrides = {})
           argv = ['install', "--gemfile=#{gemfile}"]
           argv << '-j4' unless Gem.win_platform? && Gem::Version.new(PDK::Util::RubyVersion.active_ruby_version) < Gem::Version.new('2.3.5')
-          argv << "--path=#{bundle_cachedir}" unless PDK::Util.package_install?
 
           cmd = bundle_command(*argv).tap do |c|
             c.add_spinner(_('Installing missing Gemfile dependencies.'))
@@ -248,10 +247,6 @@ module PDK
             c.context = :module
           end
         end
-
-        def bundle_cachedir
-          @bundle_cachedir ||= PDK::Util.package_install? ? PDK::Util.package_cachedir : File.join(PDK::Util.cachedir)
-        end
       end
     end
   end