pdk 1.13.0 → 1.14.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
-SHA1:
-  metadata.gz: 9ccc8605ca4505c8e57540dafa991de341002352
-  data.tar.gz: b6f8526bbc058be672452fc714b2647d38dbd1a8
+SHA256:
+  metadata.gz: 2b164195fde06a45412e475f2afc5916a45208da08c90841dd30b9b01808ca6f
+  data.tar.gz: e449e823329c01f0542da0c7a864e91019a9dad86054083c62f7f2bdf67488e6
 SHA512:
-  metadata.gz: 9479fdc94261aa410a8ca51c6752386bc24379ffc840e1910c930a830d47b5d544cf8b548a5a06bf44b31239c5d19b60f16c7f0eb224d5b3e1d1afda8d30d056
-  data.tar.gz: 20e678cb7755ad127c93e0429a944a7407399e4b42a24566ae0aad02b05ba87b13f849929d3e54dcf80d30a8d0148c92ecc07e11be6582b662ccc458fa2ad7b3
+  metadata.gz: 76b3ccf9f3b8d4ab51784f9690d44b0faf165af9a7b6df9ebee00d4a2c612050bf29a69b413163177d330ad2bf354a34437c9b870506c8e32a67d26e0d24e220
+  data.tar.gz: 78816fe4269f30053ac201c8eb8bb86be89cc0dc2440e4fa3c9c13312417621a843a8751b4b009ab33b699b835471f012ec18b5e7ba1723566301c1d8ada054e

data/CHANGELOG.md

@@ -4,6 +4,31 @@ All changes to this repo will be documented in this file.
 See the [release notes](https://puppet.com/docs/pdk/latest/release_notes.html) for a high-level summary.
 
 
+## [v1.14.0](https://github.com/puppetlabs/pdk/tree/v1.14.0) (2019-10-09)
+[Full Changelog](https://github.com/puppetlabs/pdk/compare/v1.13.0...v1.14.0)
+
+**Implemented enhancements:**
+
+- Add new "pdk console" command [\#758](https://github.com/puppetlabs/pdk/pull/758) ([logicminds](https://github.com/logicminds))
+- \(PDK-1495\) Update pdk new test UX [\#749](https://github.com/puppetlabs/pdk/pull/749) ([rodjek](https://github.com/rodjek))
+- \(PDK-680\) Make `pdk test unit` interactive by default [\#748](https://github.com/puppetlabs/pdk/pull/748) ([rodjek](https://github.com/rodjek))
+- \(PDK-1047\) Add --add-tests to pdk convert [\#746](https://github.com/puppetlabs/pdk/pull/746) ([rodjek](https://github.com/rodjek))
+
+**Merged pull requests:**
+
+- \(fixup\) Fix Bundle CLI lazy load [\#767](https://github.com/puppetlabs/pdk/pull/767) ([glennsarti](https://github.com/glennsarti))
+- \(fixup\) Missing require from lazy load PR [\#766](https://github.com/puppetlabs/pdk/pull/766) ([rodjek](https://github.com/rodjek))
+- \(maint\) Loosen tests for puppet-dev [\#765](https://github.com/puppetlabs/pdk/pull/765) ([glennsarti](https://github.com/glennsarti))
+- \(PDK-680\) Update package tests for interactive `pdk test unit` [\#756](https://github.com/puppetlabs/pdk/pull/756) ([rodjek](https://github.com/rodjek))
+- \(maint\) - Add new utility method for fixtures dir [\#755](https://github.com/puppetlabs/pdk/pull/755) ([logicminds](https://github.com/logicminds))
+- \(maint\) Lazy load dependencies [\#754](https://github.com/puppetlabs/pdk/pull/754) ([rodjek](https://github.com/rodjek))
+- \(maint\) - Allow no output when reading puppet\_from\_opts [\#753](https://github.com/puppetlabs/pdk/pull/753) ([logicminds](https://github.com/logicminds))
+- \(maint\) Refactors method to use block assignment [\#752](https://github.com/puppetlabs/pdk/pull/752) ([logicminds](https://github.com/logicminds))
+- \(maint\) Add CODEOWNERS file [\#751](https://github.com/puppetlabs/pdk/pull/751) ([glennsarti](https://github.com/glennsarti))
+- \(PDK-1367\) Deprecation warning for Puppet \< 5.0.0 [\#747](https://github.com/puppetlabs/pdk/pull/747) ([rodjek](https://github.com/rodjek))
+- \(PDK-1112\) Create json schema to validate pdk config file [\#742](https://github.com/puppetlabs/pdk/pull/742) ([glennsarti](https://github.com/glennsarti))
+- \(MAINT\) Bump version to 1.14.0.pre [\#741](https://github.com/puppetlabs/pdk/pull/741) ([scotje](https://github.com/scotje))
+
 ## [v1.13.0](https://github.com/puppetlabs/pdk/tree/v1.13.0) (2019-08-29)
 [Full Changelog](https://github.com/puppetlabs/pdk/compare/v1.12.0...v1.13.0)
 

data/README.md

@@ -22,6 +22,7 @@ puppet-syntax | Checks for correct syntax in Puppet manifests, templates, and Hi
 puppetlabs_spec_helper | Provides classes, methods, and Rake tasks to help with spec testing Puppet code.
 rspec-puppet | Tests the behavior of Puppet when it compiles your manifests into a catalog of Puppet resources.
 rspec-puppet-facts | Adds support for running rspec-puppet tests against the facts for your supported operating systems.
+puppet-debugger | Provides a REPL based debugger console.
 
 
 ## Installation
@@ -105,6 +106,46 @@ This command runs all available unit tests.
 
 ## Experimental features
 
+### `pdk console` command
+The pdk console command executes a session of the puppet debugger when inside a module and allows for exploration of puppet code.  See the official [puppet debugger site](https://www.puppet-debugger.com) for more info and the official docs [site here.](https://docs.puppet-debugger.com)
+
+To use, execute `pdk console` from inside your module directory.  You can also supply the `--puppet-version` or `--pe-version` or `--puppet-dev` to swap out the puppet version when using the console.
+
+Example (from within a module):
+
+* `pdk console --puppet-version=5`
+* `pdk console --pe-version=2018.1`
+
+The `pdk console` command will also pass through any puppet debugger arguments you wish to use.
+
+Example:
+
+* `pdk console  --no-facterdb`
+* `pdk console --play https://gist.github.com/logicminds/4f6bcfd723c92aad1f01f6a800319fa4`
+* `pdk console -e "md5('sdfasdfasdf')" --run-once --quiet`
+
+Use `pdk console -h` for a further explanation of pass through arguments.
+
+If you receive the following error you do not have the puppet-debugger gem installed.
+
+```
+pdk console -h
+Error: Unknown Puppet subcommand 'debugger'
+See 'puppet help' for help on available puppet subcommands
+```
+
+To fix this you will need to add the following entry to your .sync.yml file and run pdk update:
+
+```
+Gemfile:
+  required:
+    ":development":
+      - gem: puppet-debugger
+        version: "~> 0.14"
+```  
+
+**NOTE**: The puppet-debugger gem has been added to the [puppet-module-* gems](https://github.com/puppetlabs/puppet-module-gems/pull/117), so once you get the gem update you no longer need the .sync.yml entry.
+
 ### `pdk bundle` command
 
 This command executes arbitrary commands in a bundler context within the module you're currently working on. Arguments to this command are passed straight through to bundler. This command is experimental  and can lead to errors that can't be resolved by PDK itself.

data/lib/pdk.rb

@@ -5,21 +5,8 @@ require 'pdk/generate'
 require 'pdk/i18n'
 require 'pdk/logger'
 require 'pdk/report'
-require 'pdk/template_file'
 require 'pdk/validate'
 require 'pdk/version'
 
 module PDK
-  def self.analytics
-    @analytics ||= PDK::Analytics.build_client(
-      logger:        PDK.logger,
-      disabled:      ENV['PDK_DISABLE_ANALYTICS'] || PDK.config.user['analytics']['disabled'],
-      user_id:       PDK.config.user['analytics']['user-id'],
-      app_id:        "UA-139917834-#{PDK::Util.development_mode? ? '2' : '1'}",
-      client:        :google_analytics,
-      app_name:      'pdk',
-      app_version:   PDK::VERSION,
-      app_installer: PDK::Util.package_install? ? 'package' : 'gem',
-    )
-  end
 end

data/lib/pdk/analytics.rb

@@ -1,9 +1,25 @@
-require 'securerandom'
-require 'pdk/analytics/util'
 require 'pdk/analytics/client/google_analytics'
 require 'pdk/analytics/client/noop'
 
 module PDK
+  def self.analytics
+    require 'pdk/config'
+    require 'pdk/logger'
+    require 'pdk/util'
+    require 'pdk/version'
+
+    @analytics ||= PDK::Analytics.build_client(
+      logger:        PDK.logger,
+      disabled:      ENV['PDK_DISABLE_ANALYTICS'] || PDK.config.user['analytics']['disabled'],
+      user_id:       PDK.config.user['analytics']['user-id'],
+      app_id:        "UA-139917834-#{PDK::Util.development_mode? ? '2' : '1'}",
+      client:        :google_analytics,
+      app_name:      'pdk',
+      app_version:   PDK::VERSION,
+      app_installer: PDK::Util.package_install? ? 'package' : 'gem',
+    )
+  end
+
   module Analytics
     CLIENTS = {
       noop:             Client::Noop,

data/lib/pdk/analytics/client/google_analytics.rb

@@ -25,6 +25,7 @@ module PDK
           require 'concurrent/future'
           require 'httpclient'
           require 'locale'
+          require 'pdk/analytics/util'
 
           @http = HTTPClient.new
           @user_id = opts[:user_id]
@@ -78,6 +79,8 @@ module PDK
           # Handle analytics submission in the background to avoid blocking the
           # app or polluting the log with errors
           Concurrent::Future.execute(executor: @executor) do
+            require 'json'
+
             logger.debug "Submitting analytics: #{JSON.pretty_generate(params)}"
             @http.post(TRACKING_URL, params)
             logger.debug 'Completed analytics submission'

data/lib/pdk/answer_file.rb

@@ -1,4 +1,3 @@
-require 'json'
 require 'pdk/util/filesystem'
 
 module PDK
@@ -80,6 +79,8 @@ module PDK
     #
     # @return [Hash{String => Object}] The existing questions and answers.
     def read_from_disk
+      require 'json'
+
       return {} if !File.file?(answer_file_path) || File.zero?(answer_file_path)
 
       unless File.readable?(answer_file_path)
@@ -108,6 +109,8 @@ module PDK
     #
     # @raise [PDK::CLI::FatalError] if the answer file can not be written to.
     def save_to_disk
+      require 'json'
+
       FileUtils.mkdir_p(File.dirname(answer_file_path))
 
       write_file(answer_file_path, JSON.pretty_generate(answers))

data/lib/pdk/cli.rb

@@ -1,17 +1,18 @@
 require 'cri'
 
+require 'pdk/analytics'
 require 'pdk/cli/errors'
 require 'pdk/cli/util'
 require 'pdk/cli/util/command_redirector'
 require 'pdk/cli/util/option_normalizer'
 require 'pdk/cli/util/option_validator'
-require 'pdk/cli/exec_group'
-require 'pdk/generate/module'
+require 'pdk/config'
 require 'pdk/i18n'
 require 'pdk/logger'
 require 'pdk/report'
 require 'pdk/util/version'
 require 'pdk/util/puppet_version'
+require 'pdk/util/filesystem'
 
 class Cri::Command::CriExitException
   def initialize(is_error:)
@@ -76,6 +77,8 @@ module PDK::CLI
   end
 
   def self.template_url_option(dsl)
+    require 'pdk/util/template_uri'
+
     desc = _('Specifies the URL to the template to use when creating new modules or classes. (default: %{default_url})') % { default_url: PDK::Util::TemplateURI.default_template_uri }
 
     dsl.option nil, 'template-url', desc, argument: :required
@@ -137,6 +140,7 @@ module PDK::CLI
     end
 
     option nil, 'answer-file', _('Path to an answer file.'), argument: :required, hidden: true do |value|
+      require 'pdk/answer_file'
       PDK.answer_file = value
     end
   end
@@ -150,6 +154,7 @@ module PDK::CLI
   require 'pdk/cli/update'
   require 'pdk/cli/validate'
   require 'pdk/cli/module'
+  require 'pdk/cli/console'
 
   @base_cmd.add_command Cri::Command.new_basic_help
 end

data/lib/pdk/cli/build.rb

@@ -1,5 +1,3 @@
-require 'pdk/cli/util'
-
 module PDK::CLI
   @build_cmd = @base_cmd.define_command do
     name 'build'
@@ -14,6 +12,8 @@ module PDK::CLI
 
     run do |opts, _args, _cmd|
       require 'pdk/module/build'
+      require 'pdk/module/metadata'
+      require 'pdk/cli/util'
 
       # Make sure build is being run in a valid module directory with a metadata.json
       PDK::CLI::Util.ensure_in_module!(

data/lib/pdk/cli/bundle.rb

@@ -11,6 +11,9 @@ EOF
     skip_option_parsing
 
     run do |_opts, args, _cmd|
+      require 'pdk/cli/exec/interactive_command'
+      require 'pdk/util/bundler'
+
       PDK::CLI::Util.ensure_in_module!(
         message: _('`pdk bundle` can only be run from inside a valid module directory.'),
       )
@@ -19,12 +22,15 @@ EOF
 
       PDK::CLI::Util.analytics_screen_view('bundle')
 
-      # Ensure that the correct Ruby is activated before running commend.
+      # Ensure that the correct Ruby is activated before running command.
       puppet_env = PDK::CLI::Util.puppet_from_opts_or_env({})
       PDK::Util::RubyVersion.use(puppet_env[:ruby_version])
 
       gemfile_env = PDK::Util::Bundler::BundleHelper.gemfile_env(puppet_env[:gemset])
 
+      require 'pdk/cli/exec'
+      require 'pdk/cli/exec/interactive_command'
+
       command = PDK::CLI::Exec::InteractiveCommand.new(PDK::CLI::Exec.bundle_bin, *args).tap do |c|
         c.context = :pwd
         c.update_environment(gemfile_env)

data/lib/pdk/cli/console.rb

@@ -0,0 +1,148 @@
+module PDK::CLI
+  @console_cmd = @base_cmd.define_command do
+    name 'console'
+    usage _('console [console_options]')
+    summary _('(Experimental) Start a session of the puppet debugger console.')
+    default_subcommand 'help'
+    description _(<<-EOF
+The pdk console runs a interactive session of the puppet debugger tool to test out snippets of code, run
+language evaluations, datatype prototyping and much more.  A virtual playground for your puppet code!
+For usage details see the puppet debugger docs at https://docs.puppet-debugger.com.
+
+EOF
+                 )
+    PDK::CLI.puppet_version_options(self)
+    PDK::CLI.puppet_dev_option(self)
+    # we have to skip option parsing because it is expected the user
+    # will be passing additional args that are passed to the debugger
+    skip_option_parsing
+
+    # TODO: using -h or --help skips the pdk help and passes to puppet debugger help
+    run do |_opts, args, _cmd|
+      require 'pdk/cli/util'
+      require 'pdk/util'
+
+      PDK::CLI::Util.ensure_in_module!(
+        message: _('Console can only be run from inside a valid module directory'),
+        log_level: :fatal,
+      )
+
+      PDK::CLI::Util.module_version_check
+
+      processed_options, processed_args = process_opts(args)
+
+      PDK::CLI::Util.validate_puppet_version_opts(processed_options)
+
+      PDK::CLI::Util.analytics_screen_view('console', args)
+
+      # TODO: figure out if we need to remove default configs set by puppet
+      # so it is scoped for the module only
+      # "--environmentpath"...
+      flags = if PDK::Util.in_module_root?
+                ["--basemodulepath=#{base_module_path}",
+                 "--modulepath=#{base_module_path}"]
+              else
+                []
+              end
+      debugger_args = ['debugger'] + processed_args + flags
+      result = run_in_module(processed_options, debugger_args)
+
+      exit result[:exit_code]
+    end
+
+    # Logs a fatal message about the gem missing and how to add it
+    def inform_user_for_missing_gem(gem_name = 'puppet-debugger', version = '~> 0.14')
+      PDK.logger.fatal(<<-EOF
+Your Gemfile is missing the #{gem_name} gem.  You can add the missing gem
+by updating your #{File.join(PDK::Util.module_root, '.sync.yml')} file with the following
+and running pdk update.
+
+Gemfile:
+  required:
+    ":development":
+      - gem: #{gem_name}
+        version: \"#{version}\"
+
+EOF
+                      )
+    end
+
+    # @return [Boolean] - true if the gem was found in the lockfile
+    # @param [String] - name of ruby gem to check in bundle lockfile
+    def gem_in_bundle_lockfile?(gem_name)
+      require 'bundler'
+      require 'pdk/util/bundler'
+
+      lock_file_path = PDK::Util::Bundler::BundleHelper.new.gemfile_lock
+      PDK.logger.debug("Checking lockfile #{lock_file_path} for #{gem_name}")
+      lock_file = ::Bundler::LockfileParser.new(::Bundler.read_file(lock_file_path))
+      !lock_file.specs.find { |spec| spec.name.eql?(gem_name) }.nil?
+    rescue ::Bundler::GemfileNotFound => e
+      PDK.logger.debug _(e.message)
+      false
+    end
+
+    def check_fixtures_dir
+      existing_path = base_module_path.split(':').find do |path|
+        Dir.exist?(path) && Dir.entries(path).length > 2
+      end
+      PDK.logger.warn _('Module fixtures not found, please run pdk bundle exec rake spec_prep.') unless existing_path
+    end
+
+    # @return [Array] - array of split options [{:"puppet-version"=>"6.9.0"}, ['--loglevel=debug']]
+    # options are for the pdk and debugger pass through
+    def process_opts(opts)
+      args = opts.map do |e|
+        if e =~ %r{\A-{2}puppet|pe\-version|dev}
+          value = e.split('=')
+          (value.count < 2) ? value + [''] : value
+        end
+      end
+      args = args.compact.to_h
+      # symbolize keys
+      args = args.inject({}) do |memo, (k, v)| # rubocop:disable Style/EachWithObject
+        memo[k.sub('--', '').to_sym] = v
+        memo
+      end
+      # pass through all other args that are bound for puppet debugger
+      processed_args = opts.map { |e| e unless e =~ %r{\A-{2}puppet|pe\-version|dev} }.compact
+      [args, processed_args]
+    end
+
+    # @param opts [Hash] - the options passed into the CRI command
+    # @param bundle_args [Array] array of bundle exec args and puppet debugger args
+    # @return [Hash] - a command result hash
+    def run_in_module(opts, bundle_args)
+      require 'pdk/cli/exec'
+      require 'pdk/cli/exec/interactive_command'
+      require 'pdk/util/ruby_version'
+      require 'pdk/util/bundler'
+
+      check_fixtures_dir
+      output = opts[:debug].nil?
+      puppet_env = PDK::CLI::Util.puppet_from_opts_or_env(opts, output)
+      gemfile_env = PDK::Util::Bundler::BundleHelper.gemfile_env(puppet_env[:gemset])
+      PDK::Util::RubyVersion.use(puppet_env[:ruby_version])
+      PDK::Util::RubyVersion.instance(puppet_env[:ruby_version])
+      PDK::Util::Bundler.ensure_bundle!(puppet_env[:gemset])
+      unless gem_in_bundle_lockfile?('puppet-debugger')
+        inform_user_for_missing_gem
+        return { exit_code: 1 }
+      end
+
+      debugger_args = %w[exec puppet] + bundle_args
+      command = PDK::CLI::Exec::InteractiveCommand.new(PDK::CLI::Exec.bundle_bin, *debugger_args).tap do |c|
+        c.context = :pwd
+        c.update_environment(gemfile_env)
+      end
+      command.execute!
+    end
+
+    # @return [String] - the basemodulepath of the fixtures and modules from the current module
+    # also includes ./modules in case librarian puppet is used
+    def base_module_path
+      base_module_path = File.join(PDK::Util.module_fixtures_dir, 'modules')
+      "#{base_module_path}:#{File.join(PDK::Util.module_root, 'modules')}"
+    end
+  end
+end

data/lib/pdk/cli/convert.rb

@@ -1,5 +1,3 @@
-require 'pdk/cli/util'
-
 module PDK::CLI
   @convert_cmd = @base_cmd.define_command do
     name 'convert'
@@ -12,9 +10,11 @@ module PDK::CLI
     PDK::CLI.full_interview_option(self)
     flag nil, :noop, _('Do not convert the module, just output what would be done.')
     flag nil, :force, _('Convert the module automatically, with no prompts.')
+    flag nil, :'add-tests', _('Add any missing tests while converting the module.')
 
     run do |opts, _args, _cmd|
       require 'pdk/module/convert'
+      require 'pdk/cli/util'
 
       PDK::CLI::Util.ensure_in_module!(
         check_module_layout: true,

data/lib/pdk/cli/exec.rb

@@ -1,37 +1,33 @@
-require 'bundler'
-require 'childprocess'
-require 'English'
-require 'tempfile'
-require 'tty-which'
-
-require 'pdk/util'
-require 'pdk/util/git'
-require 'pdk/util/ruby_version'
-require 'pdk/cli/util/spinner'
-
 module PDK
   module CLI
     module Exec
-      require 'pdk/cli/exec/command'
-      require 'pdk/cli/exec/interactive_command'
-
       def self.execute(*cmd)
+        require 'pdk/cli/exec/command'
+
         Command.new(*cmd).execute!
       end
 
       def self.execute_with_env(env, *cmd)
+        require 'pdk/cli/exec/command'
+
         Command.new(*cmd).tap { |c| c.environment = env }.execute!
       end
 
       def self.execute_interactive(*cmd)
+        require 'pdk/cli/exec/interactive_command'
+
         InteractiveCommand.new(*cmd).execute!
       end
 
       def self.execute_interactive_with_env(env, *cmd)
+        require 'pdk/cli/exec/interactive_command'
+
         InteractiveCommand.new(*cmd).tap { |c| c.environment = env }.execute!
       end
 
       def self.ensure_bin_present!(bin_path, bin_name)
+        require 'tty-which'
+
         message = _('Unable to find `%{name}`. Check that it is installed and try again.') % {
           name: bin_name,
         }
@@ -46,6 +42,8 @@ module PDK
       end
 
       def self.bundle_bin
+        require 'pdk/util/ruby_version'
+
         bundle_bin = Gem.win_platform? ? 'bundle.bat' : 'bundle'
         vendored_bin_path = File.join('private', 'ruby', PDK::Util::RubyVersion.active_ruby_version, 'bin', bundle_bin)
 
@@ -53,6 +51,8 @@ module PDK
       end
 
       def self.try_vendored_bin(vendored_bin_path, fallback)
+        require 'pdk/util'
+
         unless PDK::Util.package_install?
           PDK.logger.debug(_("PDK package installation not found. Trying '%{fallback}' from the system PATH instead.") % {
             fallback: fallback,