pdk 0.2.0 → 0.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: f22acd513e00856fbadcd4c5090b2787df655426
-  data.tar.gz: 6636287e1acef4c9ba87f52b4b853c922e4ce92b
+  metadata.gz: 5501e7effa501cd2ba488735cc9abb8257ffef6a
+  data.tar.gz: e4f7a88496280db89b0ee46a42a7b8feacc24594
 SHA512:
-  metadata.gz: 2ce7e0faffaac9a2484c23dc1e88f9b240e0d768d63c8c793439f9a560a95cfa31a8745b510d4c84b83e410639bae7eb004f9d91732fd7740b4650581f71969a
-  data.tar.gz: d7a9e5bae06899f4972add88c8bc49053b257ce7aa10fb38537ea5101ea0e9ceae56e62e212bff6e07e372c558b6f38e9a9dc8ae290a6048d8fb7f72a64ddd0d
+  metadata.gz: 736c543069aa1b1d3eb79c8de901e91cf2491328fc3104f7bd660c37cafd88712583d471cbd2f4afd37b9c49890bfcb60c5b2857121f3cabb81841662fa53557
+  data.tar.gz: 5b5b03c6ac83fca4d730cba0db0277441d9bb062ed5f13ebecf24836d237db11856e6811f276ec2dcfafe28fd5c86a009958ef9e894021d14ba8a1e4d0a15af8

data/CHANGELOG.md

@@ -3,6 +3,25 @@
 All notable changes to this project will be documented in this file.
 
 
+## [v0.3.0](https://github.com/puppetlabs/pdk/tree/v0.3.0) (2017-06-29)
+[Full Changelog](https://github.com/puppetlabs/pdk/compare/v0.2.0...v0.3.0)
+
+**Implemented enhancements:**
+
+- \(MAINT\) Add support for stacktrace to Report::Event class [\#112](https://github.com/puppetlabs/pdk/pull/112) ([scotje](https://github.com/scotje))
+- \(MAINT\) Various CLI::Exec improvements and updates [\#111](https://github.com/puppetlabs/pdk/pull/111) ([scotje](https://github.com/scotje))
+- \(SDK-148\) Add "test unit --list" [\#107](https://github.com/puppetlabs/pdk/pull/107) ([james-stocks](https://github.com/james-stocks))
+- \(SDK-137\) Add puppet syntax validation [\#105](https://github.com/puppetlabs/pdk/pull/105) ([bmjen](https://github.com/bmjen))
+- \(SDK-285\) Add --auto-correct flag to validators that support it [\#104](https://github.com/puppetlabs/pdk/pull/104) ([rodjek](https://github.com/rodjek))
+- \(SDK-284\) Add guidance for users during new module interview [\#103](https://github.com/puppetlabs/pdk/pull/103) ([rodjek](https://github.com/rodjek))
+- \(SDK-147\) Add 'test unit' runner and basic output formatting [\#98](https://github.com/puppetlabs/pdk/pull/98) ([scotje](https://github.com/scotje))
+
+**Fixed bugs:**
+
+- \(SDK-297\) Fixes writing reports to a file [\#119](https://github.com/puppetlabs/pdk/pull/119) ([bmjen](https://github.com/bmjen))
+- \(SDK-290\) Make sure that all usernames are processed when creating a new module [\#108](https://github.com/puppetlabs/pdk/pull/108) ([austb](https://github.com/austb))
+- \(SDK-277\) Exit cleanly if pdk commands are run outside of a module [\#100](https://github.com/puppetlabs/pdk/pull/100) ([rodjek](https://github.com/rodjek))
+
 ## [v0.2.0](https://github.com/puppetlabs/pdk/tree/v0.2.0) (2017-06-21)
 [Full Changelog](https://github.com/puppetlabs/pdk/compare/v0.1.0...v0.2.0)
 

data/README.md

@@ -52,12 +52,48 @@ pdk validate
 This displays results in the console:
 
 ```
-Running validations on `new_module`:
-* ruby syntax: OK!
-* puppet syntax: OK!
+pdk (INFO): Running all available validators...
+[✔] Checking for missing Gemfile dependencies
+[✔] Checking metadata.json
+[✔] Checking Ruby code style
 [...]
 ```
 
+Specific validators can be run by providing the validator name (or a comma
+separated list of names) as an argument to `pdk validate`.
+
+```
+$ pdk validate metadata
+[✔] Checking for missing Gemfile dependencies
+[✔] Checking metadata.json
+```
+
+By default each validator will automatically determine which files in the
+module that it should validate, however validations can be run on specific
+files or directories by providing them as arguments to `pdk validate`
+
+```
+$ pdk validate lib/
+[✔] Checking for missing Gemfile dependencies
+[✔] Checking Ruby code style
+```
+
+Some validators support automatic correction of detected problems (for example,
+both rubocop and puppet-lint can automatically correct many common code style
+problems). To enable this functionality, run `pdk validate` with the
+`--auto-correct` option.
+
+```
+$ pdk validate --auto-correct
+pdk (INFO): Running all available validators...
+[✔] Checking for missing Gemfile dependencies
+[✔] Checking metadata.json
+[✔] Checking Puppet manifest style
+[✔] Checking Puppet manifest syntax
+[✔] Checking Ruby code style
+manifests/init.pp:1:10: corrected: double quoted string containing no variables
+```
+
 ### Run unit tests
 
 The default template sets up [rspec](http://rspec.info/) for Ruby-level unit testing, and [rspec-puppet](https://github.com/rodjek/rspec-puppet/) for catalog-level unit testing.
@@ -262,3 +298,13 @@ bundle binstubs pdk --path ~/bin
 ```
 
 Bug reports and pull requests are welcome on GitHub at https://github.com/puppetlabs/pdk.
+
+### Release Process
+
+1. Bump the version in `lib/pdk/version.rb`.
+1. In a clean checkout of master, run `bundle exec rake changelog`.
+1. Edit PR titles and tags, until `bundle exec rake changelog` output makes sense.
+1. Commit and PR the changes.
+1. When the PR is merged, get a clean checkout of the merged commit, and run `bundle exec rake release[upstream]` (where "upstream" is your local name of the puppetlabs remote)
+1. Profit!
+1. Update `lib/pdk/version.rb` with `x.y.z-pre`, commit, and PR to prepare for next release.

data/lib/pdk/cli.rb

@@ -1,6 +1,7 @@
 require 'cri'
 
 require 'pdk/cli/errors'
+require 'pdk/cli/util'
 require 'pdk/cli/util/option_normalizer'
 require 'pdk/cli/util/option_validator'
 require 'pdk/generators/module'

data/lib/pdk/cli/exec.rb

@@ -55,6 +55,7 @@ module PDK
         attr_reader :argv
         attr_reader :context
         attr_accessor :timeout
+        attr_accessor :environment
 
         def initialize(*argv)
           @argv = argv
@@ -70,6 +71,9 @@ module PDK
 
           # Default to running things in the system context.
           @context = :system
+
+          # Extra environment vars to add to base set.
+          @environment = {}
         end
 
         def context=(new_context)
@@ -84,19 +88,32 @@ module PDK
           @success_message = opts.delete(:success)
           @failure_message = opts.delete(:failure)
 
-          @spinner = TTY::Spinner.new("[:spinner] #{message}", opts)
+          @spinner = Gem.win_platform? ? WindowsSpinner.new(message, opts) : TTY::Spinner.new("[:spinner] #{message}", opts)
         end
 
         def execute!
           # Start spinning if configured.
           @spinner.auto_spin if @spinner
 
+          # Add custom env vars.
+          @environment.each do |k, v|
+            @process.environment[k] = v
+          end
+
           if context == :module
             # TODO: we should probably more carefully manage PATH and maybe other things too
             @process.environment['GEM_HOME'] = File.join(PDK::Util.cachedir, 'bundler', 'ruby', RbConfig::CONFIG['ruby_version'])
             @process.environment['GEM_PATH'] = pdk_gem_path
 
-            Dir.chdir(PDK::Util.module_root) do
+            mod_root = PDK::Util.module_root
+
+            unless mod_root
+              @spinner.error
+
+              raise PDK::CLI::FatalError, _('Current working directory is not part of a module. (No metadata.json was found.)')
+            end
+
+            Dir.chdir(mod_root) do
               ::Bundler.with_clean_env do
                 run_process!
               end
@@ -132,15 +149,12 @@ module PDK
         protected
 
         def run_process!
+          command_string = argv.join(' ')
+          PDK.logger.debug(_("Executing '%{command}'") % { command: command_string })
           begin
             @process.start
           rescue ChildProcess::LaunchError => e
-            msg = if @process.respond_to?(:argv)
-                    _("Failed to execute '%{command}': %{message}") % { command: @process.argv.join(' '), message: e.message }
-                  else
-                    _('Failed to execute process: %{message}') % { message: e.message }
-                  end
-            raise PDK::CLI::FatalError, msg
+            raise PDK::CLI::FatalError, _("Failed to execute '%{command}': %{message}") % { command: command_string, message: e.message }
           end
 
           if timeout
@@ -160,7 +174,6 @@ module PDK
         end
 
         def find_pdk_gem_path
-          # /opt/puppetlabs/sdk/private/ruby/2.1.9/lib/ruby/gems/2.1.0
           package_gem_path = File.join(PDK::CLI::Exec.pdk_basedir, 'private', 'ruby', RUBY_VERSION, 'lib', 'ruby', 'gems', RbConfig::CONFIG['ruby_version'])
 
           if File.directory?(package_gem_path)
@@ -171,6 +184,30 @@ module PDK
           end
         end
       end
+
+      # This is a placeholder until we integrate ansicon into Windows packaging
+      # or come up with some other progress indicator for Windows.
+      class WindowsSpinner
+        def initialize(message, _opts = {})
+          @message = message
+        end
+
+        def auto_spin
+          $stderr.print @message << '...'
+        end
+
+        def success(message = '')
+          message ||= 'done.'
+
+          $stderr.print message << "\n"
+        end
+
+        def error(message = '')
+          message ||= 'FAILED'
+
+          $stderr.print message << "\n"
+        end
+      end
     end
   end
 end

data/lib/pdk/cli/new/class.rb

@@ -1,4 +1,3 @@
-
 module PDK::CLI
   @new_class_cmd = @new_cmd.define_command do
     name 'class'
@@ -10,6 +9,8 @@ module PDK::CLI
     run do |opts, args, _cmd|
       require 'pdk/generators/puppet_class'
 
+      PDK::CLI::Util.ensure_in_module!
+
       class_name = args[0]
       module_dir = Dir.pwd
 

data/lib/pdk/cli/test/unit.rb

@@ -1,3 +1,5 @@
+require 'pdk/cli/util/option_validator'
+require 'pdk/report'
 
 module PDK::CLI
   @test_unit_cmd = @test_cmd.define_command do
@@ -7,32 +9,49 @@ module PDK::CLI
 
     flag nil, :list, _('list all available unit tests and their descriptions')
 
-    option nil, :tests, _('a comma-separated list of tests to run'), argument: :required do |values|
-      OptionValidator.list(values)
+    option nil, :tests, _('a comma-separated list of tests to run'), argument: :required, default: '' do |values|
+      PDK::CLI::Util::OptionValidator.comma_separated_list?(values)
     end
 
-    option nil, :runner_options, _('options to pass through to the actual test-runner'), argument: :required
+    # TODO
+    # option nil, :runner_options, _("options to pass through to the actual test-runner"), argument: :required
 
     run do |opts, _args, _cmd|
       require 'pdk/tests/unit'
 
+      PDK::CLI::Util.ensure_in_module!
+
       report = nil
 
       if opts[:list]
-        puts _('List of all available unit tests: (TODO)')
-      end
-
-      if opts[:tests]
-        tests = opts.fetch(:tests)
+        examples = PDK::Test::Unit.list
+        if examples.empty?
+          puts _('No examples found.')
+        else
+          puts _('Examples:')
+          examples.each do |example|
+            puts _("%{id}\t%{description}" % { id: example[:id], description: example[:full_description] })
+          end
+        end
+      else
+        report = PDK::Report.new
+        report_formats = if opts[:format]
+                           PDK::CLI::Util::OptionNormalizer.report_formats(opts[:format])
+                         else
+                           [{
+                             method: PDK::Report.default_format,
+                             target: PDK::Report.default_target,
+                           }]
+                         end
+
+        exit_code = PDK::Test::Unit.invoke(report, opts)
+
+        report_formats.each do |format|
+          report.send(format[:method], format[:target])
+        end
+
+        exit exit_code
       end
-
-      # Note: Reporting may be delegated to the validation tool itself.
-      if opts[:'report-file']
-        format = opts.fetch(:'report-format', PDK::Report.default_format)
-        report = Report.new(opts.fetch(:'report-file'), format)
-      end
-
-      PDK::Test::Unit.invoke(tests, report)
     end
   end
 end

data/lib/pdk/cli/util.rb

@@ -0,0 +1,15 @@
+module PDK
+  module CLI
+    module Util
+      # Ensures the calling code is being run from inside a module directory.
+      #
+      # @raise [PDK::CLI::FatalError] if the current directory or parents do
+      #   not contain a `metadata.json` file.
+      def ensure_in_module!
+        message = _('This command must be run from inside a module (no metadata.json found)')
+        raise PDK::CLI::FatalError, message if PDK::Util.module_root.nil?
+      end
+      module_function :ensure_in_module!
+    end
+  end
+end

data/lib/pdk/cli/util/interview.rb

@@ -0,0 +1,52 @@
+require 'tty-prompt'
+
+module PDK
+  module CLI
+    module Util
+      class Interview < TTY::Prompt::AnswersCollector
+        def pastel
+          @pastel ||= Pastel.new
+        end
+
+        def add_questions(questions)
+          questions.each do |question|
+            add_question(question)
+          end
+        end
+
+        def add_question(options = {})
+          (@questions ||= {})[options[:name]] = options
+        end
+
+        def num_questions
+          @questions.count
+        end
+
+        def run
+          i = 1
+          num_questions = @questions.count
+          @questions.each do |question_name, question|
+            @name = question_name
+            puts pastel.bold(_('[Q %{current_number}/%{questions_total}]') % { current_number: i, questions_total: num_questions })
+            puts pastel.bold(question[:question])
+            puts question[:help]
+            ask(_('-->')) do |q|
+              q.required(question.fetch(:required, false))
+
+              if question.key?(:validate_pattern)
+                q.validate(question[:validate_pattern], question[:validate_message])
+              end
+
+              q.default(question[:default]) if question.key?(:default)
+            end
+            i += 1
+            puts ''
+          end
+          @answers
+        rescue TTY::Prompt::Reader::InputInterrupt
+          nil
+        end
+      end
+    end
+  end
+end

data/lib/pdk/cli/util/option_normalizer.rb

@@ -42,7 +42,7 @@ module PDK
               target = $stderr
             end
 
-            { method: "to_#{format}".to_sym, target: target }
+            { method: "write_#{format}".to_sym, target: target }
           end
         end
 

data/lib/pdk/cli/validate.rb

@@ -3,11 +3,18 @@ require 'pdk/util/bundler'
 module PDK::CLI
   @validate_cmd = @base_cmd.define_command do
     name 'validate'
-    usage _('validate [options]')
+    usage _('validate [validators] [options] [targets]')
     summary _('Run static analysis tests.')
-    description _('Run metadata, puppet, or ruby validation.')
+    description _(
+      "Run metadata, puppet, or ruby validation.\n\n" \
+      '[validators] is an optional comma separated list of validators to use. ' \
+      "If not specified, all validators will be used.\n\n" \
+      '[targets] is an optional space separated list of files or directories to be validated. ' \
+      'If not specified, the validators will be run against all applicable files in the module.',
+    )
 
     flag nil, :list, _('list all available validators')
+    flag :a, 'auto-correct', _('automatically correct problems (where possible)')
 
     run do |opts, args, _cmd|
       validator_names = PDK::Validate.validators.map { |v| v.name }
@@ -19,6 +26,8 @@ module PDK::CLI
         exit 0
       end
 
+      PDK::CLI::Util.ensure_in_module!
+
       if args[0]
         # This may be a single validator, a list of validators, or a target.
         if Util::OptionValidator.comma_separated_list?(args[0])
@@ -60,6 +69,7 @@ module PDK::CLI
                        end
 
       options = targets.empty? ? {} : { targets: targets }
+      options[:auto_correct] = true if opts.key?(:'auto-correct')
 
       exit_code = 0
 

data/lib/pdk/generators/module.rb

@@ -1,13 +1,14 @@
 require 'etc'
 require 'pathname'
 require 'fileutils'
+require 'tty-prompt'
 
 require 'pdk'
 require 'pdk/logger'
 require 'pdk/module/metadata'
 require 'pdk/module/templatedir'
 require 'pdk/cli/exec'
-require 'pdk/cli/input'
+require 'pdk/cli/util/interview'
 require 'pdk/util'
 require 'pdk/util/version'
 
@@ -17,26 +18,13 @@ module PDK
       DEFAULT_TEMPLATE = 'https://github.com/puppetlabs/pdk-module-template'.freeze
 
       def self.invoke(opts = {})
-        defaults = {
-          'name'         => "#{Etc.getlogin}-#{opts[:name]}",
-          'version'      => '0.1.0',
-          'dependencies' => [
-            { 'name' => 'puppetlabs-stdlib', 'version_requirement' => '>= 4.13.1 < 5.0.0' },
-          ],
-        }
-
-        defaults['license'] = opts[:license] if opts.key? :license
         target_dir = File.expand_path(opts[:target_dir])
 
         if File.exist?(target_dir)
           raise PDK::CLI::FatalError, _("The destination directory '%{dir}' already exists") % { dir: target_dir }
         end
 
-        metadata = PDK::Module::Metadata.new(defaults)
-
-        module_interview(metadata, opts) unless opts[:'skip-interview'] # @todo Build way to get info by answers file
-
-        metadata.update!('pdk-version' => PDK::Util::Version.version_string)
+        metadata = prepare_metadata(opts)
 
         temp_target_dir = PDK::Util.make_tmpdir_name('pdk-module-target')
 
@@ -63,6 +51,31 @@ module PDK
         FileUtils.mv(temp_target_dir, target_dir)
       end
 
+      def self.prepare_metadata(opts)
+        username = Etc.getlogin.gsub(%r{[^0-9a-z]}i, '')
+        username = 'username' if username == ''
+        if Etc.getlogin != username
+          PDK.logger.warn(_('Your username is not a valid Forge username, proceeding with the username %{username}' % { username: username }))
+        end
+
+        defaults = {
+          'name'         => "#{username}-#{opts[:name]}",
+          'version'      => '0.1.0',
+          'dependencies' => [
+            { 'name' => 'puppetlabs-stdlib', 'version_requirement' => '>= 4.13.1 < 5.0.0' },
+          ],
+        }
+        defaults['license'] = opts[:license] if opts.key? :license
+
+        metadata = PDK::Module::Metadata.new(defaults)
+
+        module_interview(metadata, opts) unless opts[:'skip-interview'] # @todo Build way to get info by answers file
+
+        metadata.update!('pdk-version' => PDK::Util::Version.version_string)
+
+        metadata
+      end
+
       def self.prepare_module_directory(target_dir)
         [
           File.join(target_dir, 'manifests'),
@@ -77,63 +90,98 @@ module PDK
       end
 
       def self.module_interview(metadata, opts = {})
-        puts _(
-          'We need to create a metadata.json file for this module. Please answer the ' \
-          'following questions; if the question is not applicable to this module, feel free ' \
-          'to leave it blank.',
-        )
-
-        begin
-          puts ''
-          forge_user = PDK::CLI::Input.get(_('What is your Puppet Forge username?'), metadata.data['author'])
-          metadata.update!('name' => "#{forge_user}-#{opts[:name]}")
-        rescue StandardError => e
-          PDK.logger.error(_("We're sorry, we could not parse your module name: %{message}") % { message: e.message })
-          retry
-        end
+        questions = [
+          {
+            name:             'name',
+            question:         _('What is your Puppet Forge username?'),
+            help:             _('This will be used when uploading your module to the Forge. You can opt out of this at any time.'),
+            required:         true,
+            validate_pattern: %r{\A[a-z0-9]+\Z}i,
+            validate_message: _('Forge usernames can only contain lowercase letters and numbers'),
+            default:          metadata.data['author'],
+          },
+          {
+            name:             'version',
+            question:         _('What version is this module?'),
+            help:             _('Puppet uses Semantic Versioning (semver.org) to version modules.'),
+            required:         true,
+            validate_pattern: %r{\A[0-9]+\.[0-9]+\.[0-9]+},
+            validate_message: _('Semantic Version numbers must be in the form MAJOR.MINOR.PATCH'),
+            default:          metadata.data['version'],
+          },
+          {
+            name:     'author',
+            question: _('Who wrote this module?'),
+            help:     _('The person who gets credit for creating the module. '),
+            required: true,
+            default:  metadata.data['author'],
+          },
+          {
+            name:     'license',
+            question: _('What license does this module code fall under?'),
+            help:     _('This should be an identifier from https://spdk.org/licenses/. Common values are "Apache-2.0", "MIT", or "proprietary".'),
+            required: true,
+            default:  metadata.data['license'],
+          },
+          {
+            name:     'summary',
+            question: _('How would you describe this module in a single sentence?'),
+            help:     _('To help other Puppet users understand what the module does.'),
+            required: true,
+            default:  metadata.data['summary'],
+          },
+          {
+            name:     'source',
+            question: _("Where is this modules's source code repository?"),
+            help:     _('Usually a GitHub URL'),
+            required: true,
+            default:  metadata.data['source'],
+          },
+          {
+            name:     'project_page',
+            question: _('Where can others go to learn more about this module?'),
+            help:     _('A web site that offers full information about your module.'),
+            default:  metadata.data['project_page'],
+          },
+          {
+            name:     'issues_url',
+            question: _('Where can others go to file issues about this module?'),
+            help:     _('A web site with a public bug tracker for your module.'),
+            default:  metadata.data['issues_url'],
+          },
+        ]
+
+        prompt = TTY::Prompt.new
+
+        interview = PDK::CLI::Util::Interview.new(prompt)
+
+        questions.reject! { |q| q[:name] == 'license' } if opts.key?(:license)
+
+        interview.add_questions(questions)
 
-        begin
-          puts "\n" + _('Puppet uses Semantic Versioning (semver.org) to version modules.')
-          module_version = PDK::CLI::Input.get(_('What version is this module?'), metadata.data['version'])
-          metadata.update!('version' => module_version)
-        rescue StandardError => e
-          PDK.logger.error(_("We're sorry, we could not parse that as a Semantic Version: %{message}") % { message: e.message })
-          retry
-        end
+        puts _(
+          "\nWe need to create a metadata.json file for this module, so we're going to ask you %{count} quick questions.\n" \
+          "If the question is not applicable to this module, just leave the answer blank.\n\n",
+        ) % { count: interview.num_questions }
 
-        puts ''
-        module_author = PDK::CLI::Input.get(_('Who wrote this module?'), metadata.data['author'])
-        metadata.update!('author' => module_author)
+        answers = interview.run
 
-        unless opts.key?(:license)
-          puts ''
-          module_license = PDK::CLI::Input.get(_('What license does this module code fall under?'), metadata.data['license'])
-          metadata.update!('license' => module_license)
+        if answers.nil?
+          puts _('Interview cancelled, aborting...')
+          exit 0
         end
 
-        puts ''
-        module_summary = PDK::CLI::Input.get(_('How would you describe this module in a single sentence?'), metadata.data['summary'])
-        metadata.update!('summary' => module_summary)
+        answers['name'] = "#{answers['name']}-#{opts[:name]}"
+        metadata.update!(answers)
 
-        puts ''
-        module_source = PDK::CLI::Input.get(_("Where is this module's source code repository?"), metadata.data['source'])
-        metadata.update!('source' => module_source)
-
-        puts ''
-        module_page = PDK::CLI::Input.get(_('Where can others go to learn more about this module?'), metadata.data['project_page'])
-        metadata.update!('project_page' => module_page)
-
-        puts ''
-        module_issues = PDK::CLI::Input.get(_('Where can others go to file issues about this module?'), metadata.data['issues_url'])
-        metadata.update!('issues_url' => module_issues)
-
-        puts
+        puts '-' * 40
+        puts _('SUMMARY')
         puts '-' * 40
         puts metadata.to_json
         puts '-' * 40
         puts
 
-        unless PDK::CLI::Input.get(_('About to generate this module; continue?'), 'Y') =~ %r{^y(es)?$}i # rubocop:disable Style/GuardClause
+        unless prompt.yes?(_('About to generate this module; continue?')) # rubocop:disable Style/GuardClause
           puts _('Aborting...')
           exit 0
         end