pmdtester 1.0.0.pre.beta2 → 1.1.1

data/README.rdoc

@@ -1,9 +1,12 @@
 = pmdtester
 
+home  :: https://pmd.github.io
 code  :: https://github.com/pmd/pmd-regression-tester
 bugs  :: https://github.com/pmd/pmd-regression-tester/issues
 
-build-status :: {<img src="https://travis-ci.com/pmd/pmd-regression-tester.svg?branch=master" alt="Build Status" />}[https://travis-ci.com/pmd/pmd-regression-tester]
+build-status :: {<img src="https://github.com/pmd/pmd-regression-tester/workflows/build/badge.svg?branch=master" alt="Build Status" />}[https://github.com/pmd/pmd-regression-tester/actions?query=workflow%3Abuild]
+
+gem-version :: {<img src="https://badge.fury.io/rb/pmdtester.svg" alt="Gem Version" />}[https://rubygems.org/gems/pmdtester]
 
 == DESCRIPTION:
 
@@ -16,18 +19,138 @@ on a list of standard projects(Spring Framework, Hibernate, Solr, etc.)
 
 == SYNOPSIS:
 
-* FIX (code sample of usage)
+=== Options:
+    -r, --local-git-repo     path to the local PMD repository
+    -b, --base-branch        name of the base branch in local PMD repository
+    -p, --patch-branch       name of the patch branch in local PMD repository
+    -bc, --base-config       path to the base PMD configuration file default:PMDTESTER_INSTALLED_DIR/config/all-java.xml
+    -pc, --patch-config      path to the patch PMD configuration file default:PMDTESTER_INSTALLED_DIR/config/all-java.xml
+    -c, --config             path to the base and patch PMD configuration file
+    -l, --list-of-project    path to the file which contains the list of standard projects default:PMDTESTER_INSTALLED_DIR/config/project-list.xml
+    -m, --mode               the mode of the tool: 'local', 'online' or 'single'
+          single: Set this option to 'single' if your patch branch contains changes
+            for any option that can't work on master/base branch
+          online: Set this option to 'online' if you want to download
+            the PMD report of master/base branch rather than generating it locally
+          local: Default option is 'local', PMD reports for the base and patch branches are generated locally.
+
+    -t, --threads            Sets the number of threads used by PMD. Set threads to 0 to disable multi-threading processing. default:1
+    -f, --html-flag          whether to not generate the html diff report in single mode
+    -a, --auto-gen-config    whether to generate configurations automatically based on branch differences,this option only works in online and local mode
+    --keep-reports           whether to keep old reports and skip running PMD again if possible
+    -d, --debug              whether change log level to DEBUG to see more information
+    --error-recovery         enable error recovery mode when executing PMD. Might help to analyze errors.
+    --baseline-download-url  download url prefix from where to download the baseline in online mode default:https://sourceforge.net/projects/pmd/files/pmd-regression-tester/
+    -v, --version
+    -h, --help
+
+=== Quick start
+
+==== Run local mode
+
+  pmdtester -b master -p YOUR_DEVELOPMENT_BRANCH -r PATH_TO_LOCAL_PMD_REPO -a
+
+==== Run single mode
+
+  pmdtester -p YOUR_DEVELOPMENT_BRANCH -pc CONFIG_ONLY_CONTAINS_NEW_PMD_JAVA_RULE -m single
+
+==== Run online mode
+
+  pmdtester -b master -p YOUR_DEVELOPMENT_BRANCH -r PATH_TO_LOCAL_PMD_REPO -m online -a
+
+=== Output
+The tool creates the following folders:
+  target
+  ├── repositories         <- the analyzed projects are cloned here
+  │   ├── PROJECT_NAME_1
+  │   ├── ......
+  │   └── PROJECT_NAME_n
+  ├── reports
+  │   ├── BASE_BRANCH_NAME        <- the base baseline is placed here
+  │   ├── PATCH_BRANCH_NAME       <- the patch baseline is placed here
+  │   └── diff
+  │       ├── index.html          <- the summary report of diff reports
+  │       ├── base_config.xml     <- pmd config from the base branch
+  │       ├── patch_config.xml    <- pmd config from the patch branch
+  │       ├── css  <- css resources are placed here
+  │       ├── js   <- js resources
+  │       ├── PROJECT_NAME_1
+  │       │   ├── project_data.js <- contains the violations as js/json
+  │       │   └── index.html      <- the diff report of PROJECT_1
+  │       ├── .......
+  │       └── PROJECT_NAME_n
+  │           ├── project_data.js <- contains the violations as js/json
+  │           └── index.xml       <- the diff report of PROJECT_N
+  ├── pmd-bin-<version>-<branch_name>-<sha1>  <- cached pmd builds that are reused
+  └── pmd-bin-....
+
+==== The baseline format
+  branch_name
+  ├── branch_info.json
+  ├── config.xml
+  ├── STANDARD_PROJECT_NAME_1
+  │   ├── report_info.json
+  │   └── pmd_report.xml
+  ├── ......................
+  │   ├── report_info.json
+  │   └── pmd_report.xml
+  └── STANDARD_PROJECT_NAME_n
+      ├── report_info.info
+      └── pmd_report.xml
 
 == REQUIREMENTS:
 
-* FIX (list of requirements)
+* Ruby 2.7 or higher
 
-== INSTALL:
+=== Runtime dependency
 
-* FIX (sudo gem install, anything else)
+  nokogiri >= 1.11.0.rc4
+  slop ~> 4.6
+  differ ~> 0.1
+  rufus-scheduler ~> 3.5
+  logger-colors ~> 1.0
+  liquid >= 4.0
 
-== DEVELOPERS:
+=== Development dependency
 
-You can list all tasks by running `rake -T`.
+  hoe ~> 3.22
+  hoe-bundler ~> 1.5
+  hoe-git ~> 1.6
+  minitest ~> 5.10
+  mocha ~> 1.5
+  rdoc < 7, >= 4.0
+  rubocop ~> 0.81
+  test-unit ~> 3.2
 
+== INSTALL:
+
+  gem install pmdtester --pre
 
+== DEVELOPERS:
+  git clone https://github.com/pmd/pmd-regression-tester.git
+  cd pmd-regression-tester
+  gem install bundler
+  bundle config set path "vendor/cache"
+  bundle install # once
+  bundle exec rake verify # run this command before commit your changes
+  bundle exec pmdtester ... # run this to directly execute pmdtester from source
+  
+  Run a single test class, e.g.:
+  bundle exec ruby -I test test/test_diff_report_builder.rb
+  
+  Run a single test, e.g.:
+  bundle exec ruby -I test test/test_diff_report_builder.rb -n test_diff_report_builder
+
+=== Releasing
+
+* Update +History.md+ (version and date)
+* Update +lib/pmdtester.rb+ (version)
+* Run "bundle exec rake verify" and add new +pmdtester.gemspec+ (new version)
+* Commit ("Prepare release x.y.z").
+* Tag this commit ("git tag vx.y.z").
+* Update History.md and lib/pmdtester.rb for the next development version,
+  run again "bundle exec rake verify"
+* Commit ("Prepare next development version x.y.z-SNAPSHOT").
+* Push to master.
+* Push the tag. Github Actions will build and publish the new gem
+* Update the release notes on https://github.com/pmd/pmd-regression-tester/releases

data/Rakefile

@@ -3,40 +3,43 @@
 # -*- ruby -*-
 require 'rake/testtask'
 require 'rubocop/rake_task'
-require_relative './lib/pmdtester/parsers/options.rb'
+require_relative 'lib/pmdtester'
 
 gem 'hoe'
 require 'hoe'
+
 Hoe.plugin :bundler
-Hoe.plugin :gemspec
 Hoe.plugin :git
 
 hoe = Hoe.spec 'pmdtester' do
-  self.version = PmdTester::Options::VERSION
+  self.version = PmdTester::VERSION
 
-  developer 'Andreas Dangel', 'andreas.dangel@adangel.org'
+  developer 'Andreas Dangel', 'andreas.dangel@pmd-code.org'
   developer 'Binguo Bao', 'djydewang@gmail.com'
+  developer 'Clément Fournier', 'clement.fournier76@gmail.com'
 
-  self.clean_globs = %w[target/reports/**/* target/test/**/*]
-  self.extra_deps += [['nokogiri', '~> 1.8.2'], ['slop', '~> 4.6.2']]
-  self.extra_dev_deps  += [
-    ['hoe-bundler',   '~> 1.2'],
-    ['hoe-gemspec',   '~> 1.0'],
+  self.clean_globs = %w[target/reports/**/* target/test/**/* target/dynamic-config.xml Gemfile.lock]
+  self.extra_deps += [['nokogiri', '>= 1.11.0.rc4'], ['slop', '~> 4.6'], ['differ', '~> 0.1'],
+                      ['rufus-scheduler', '~> 3.5'], ['logger-colors', '~> 1.0'],
+                      ['liquid', '>= 4.0']]
+  self.extra_dev_deps += [
+    ['hoe-bundler',   '~> 1.5'],
     ['hoe-git',       '~> 1.6'],
-    ['minitest',      '~> 5.10.1'],
-    ['mocha',         '~> 1.5.0'],
-    ['rubocop',       '~> 0.56.0'],
-    ['test-unit',     '~> 3.2.3']
+    ['minitest',      '~> 5.10'],
+    ['mocha',         '~> 1.5'],
+    # use the same version of rubocop as codacy
+    ['rubocop',       '~> 0.81'],
+    ['test-unit',     '~> 3.2'],
+    ['rdoc',          ['>= 4.0', '< 7']]
   ]
+  spec_extras[:required_ruby_version] = '>= 2.7'
 
   license 'BSD-2-Clause'
 end
 
 # Refers to
 # http://rubocop.readthedocs.io/en/latest/integration_with_other_tools/#rake-integration
-RuboCop::RakeTask.new(:rubocop) do |task|
-  task.patterns = %w[lib/**/*.rb test/**/*.rb]
-end
+RuboCop::RakeTask.new(:rubocop)
 
 # Run integration test cases
 Rake::TestTask.new('integration-test') do |task|
@@ -48,7 +51,18 @@ end
 
 desc 'generate the pmdtester.gemspec file'
 task 'hoe:spec' do
-  File.open("#{hoe.name}.gemspec", "w") { |f| f.write hoe.spec.to_ruby}
+  attention_message = '# DO NOT EDIT THIS FILE. Instead, edit Rakefile, and run `rake hoe:spec`.'
+  File.open("#{hoe.name}.gemspec", 'w') do |f|
+    f.puts attention_message
+    f.puts
+    f.write hoe.spec.to_ruby
+    f.puts
+    f.puts attention_message
+  end
 end
 
+desc 'verify code quality before committing changes'
+task 'verify' => ['clean', 'test', 'rubocop', 'git:manifest', 'hoe:spec', 'check_manifest'] do
+  Rake.application.invoke_task('bundler:gemfile[,true]')
+end
 # vim: syntax=ruby

data/bin/pmdtester

@@ -1,7 +1,7 @@
 #!/usr/bin/env ruby
 # frozen_string_literal: true
 
-require_relative '../lib/pmdtester/runner'
+require_relative '../lib/pmdtester'
 
 runner = PmdTester::Runner.new(ARGV)
 runner.run

data/config/all-java.xml

@@ -3,7 +3,7 @@
 <ruleset name="All Java Rules" 
     xmlns="http://pmd.sourceforge.net/ruleset/2.0.0"
     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
-    xsi:schemaLocation="http://pmd.sourceforge.net/ruleset/2.0.0 http://pmd.sourceforge.net/ruleset_2_0_0.xsd">
+    xsi:schemaLocation="http://pmd.sourceforge.net/ruleset/2.0.0 https://pmd.sourceforge.io/ruleset_2_0_0.xsd">
     <description>Every Java Rule in PMD</description>
 
     <rule ref="category/java/bestpractices.xml" />

data/config/design.xml

@@ -3,7 +3,7 @@
 <ruleset name="Design"
     xmlns="http://pmd.sourceforge.net/ruleset/2.0.0"
     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
-    xsi:schemaLocation="http://pmd.sourceforge.net/ruleset/2.0.0 http://pmd.sourceforge.net/ruleset_2_0_0.xsd">
+    xsi:schemaLocation="http://pmd.sourceforge.net/ruleset/2.0.0 https://pmd.sourceforge.io/ruleset_2_0_0.xsd">
 
     <description>
 The Design ruleset contains rules that flag suboptimal code implementations. Alternate approaches

data/config/projectlist_1_0_0.xsd

@@ -1,4 +1,5 @@
 <?xml version="1.0" ?>
+<!-- version 1.0.0 -->
 <xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
   <xs:element name="projectlist">
     <xs:complexType>
@@ -25,4 +26,4 @@
       <xs:element name="exclude-pattern" type="xs:string" minOccurs="0" maxOccurs="unbounded"/>
     </xs:sequence>
   </xs:complexType>
-</xs:schema>
+</xs:schema>

data/config/projectlist_1_1_0.xsd

@@ -0,0 +1,31 @@
+<?xml version="1.0" ?>
+<!-- version 1.1.0 -->
+<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
+  <xs:element name="projectlist">
+    <xs:complexType>
+      <xs:sequence>
+        <xs:element name="description" type="xs:string" minOccurs="1" maxOccurs="1"/>
+        <xs:element name="project" type="project" minOccurs="1" maxOccurs="unbounded"/>
+      </xs:sequence>
+    </xs:complexType>
+  </xs:element>
+  <xs:complexType name="project">
+    <xs:sequence>
+      <xs:element name="name" type="xs:string" minOccurs="1" maxOccurs="1"/>
+      <xs:element name="type" minOccurs="1" maxOccurs="1">
+        <xs:simpleType>
+          <xs:restriction base="xs:string">
+            <xs:enumeration value="git"/>
+            <xs:enumeration value="hg"/>
+          </xs:restriction>
+        </xs:simpleType>
+      </xs:element>
+      <xs:element name="connection" type="xs:string" minOccurs="1" maxOccurs="1"/>
+      <xs:element name="webview-url" type="xs:string" minOccurs="0" maxOccurs="1"/>
+      <xs:element name="tag" type="xs:string" minOccurs="0" maxOccurs="1"/>
+      <xs:element name="exclude-pattern" type="xs:string" minOccurs="0" maxOccurs="unbounded"/>
+      <xs:element name="build-command" type="xs:string" minOccurs="0" maxOccurs="1"/>
+      <xs:element name="auxclasspath-command" type="xs:string" minOccurs="0" maxOccurs="1"/>
+    </xs:sequence>
+  </xs:complexType>
+</xs:schema>

data/lib/pmdtester.rb

@@ -0,0 +1,48 @@
+# frozen_string_literal: true
+
+require 'logger'
+require 'logger/colors'
+
+require_relative 'pmdtester/cmd'
+require_relative 'pmdtester/collection_by_file'
+require_relative 'pmdtester/pmd_branch_detail'
+require_relative 'pmdtester/pmd_configerror'
+require_relative 'pmdtester/pmd_error'
+require_relative 'pmdtester/pmd_report_detail'
+require_relative 'pmdtester/pmd_tester_utils'
+require_relative 'pmdtester/pmd_violation'
+require_relative 'pmdtester/project'
+require_relative 'pmdtester/report_diff'
+require_relative 'pmdtester/resource_locator'
+require_relative 'pmdtester/runner'
+
+require_relative 'pmdtester/builders/simple_progress_logger'
+require_relative 'pmdtester/builders/project_builder'
+require_relative 'pmdtester/builders/project_hasher'
+require_relative 'pmdtester/builders/pmd_report_builder'
+require_relative 'pmdtester/builders/liquid_renderer'
+require_relative 'pmdtester/builders/rule_set_builder'
+require_relative 'pmdtester/builders/summary_report_builder'
+
+require_relative 'pmdtester/parsers/options'
+require_relative 'pmdtester/parsers/pmd_report_document'
+require_relative 'pmdtester/parsers/projects_parser'
+
+# PmdTester is a regression testing tool ensure that new problems
+# and unexpected behaviors will not be introduced to PMD project
+# after fixing an issue and new rules can work as expected.
+module PmdTester
+  VERSION = '1.1.1'
+  BASE = 'base'
+  PATCH = 'patch'
+  PR_NUM_ENV_VAR = 'PMD_CI_PULL_REQUEST_NUMBER' # see PmdBranchDetail
+
+  def logger
+    PmdTester.logger
+  end
+
+  # Global, memoized, lazy initialized instance of a logger
+  def self.logger
+    @logger ||= Logger.new(STDOUT)
+  end
+end

data/lib/pmdtester/builders/liquid_renderer.rb

@@ -0,0 +1,73 @@
+# frozen_string_literal: true
+
+require 'liquid'
+require 'json'
+
+module PmdTester
+  # A module to include in classes that use a Liquid template
+  # to generate content.
+  module LiquidRenderer
+    include PmdTester
+
+    def render_liquid(template_path, env)
+      to_render = File.read(ResourceLocator.resource(template_path))
+      includes = Liquid::LocalFileSystem.new(ResourceLocator.resource('_includes'), '%s.html')
+      Liquid::Template.file_system = includes
+      template = Liquid::Template.parse(to_render, error_mode: :strict)
+      template.render!(env, { strict_variables: true })
+    end
+
+    def render_and_write(template_path, target_file, env)
+      write_file(target_file, render_liquid(template_path, env))
+    end
+
+    def write_file(target_file, contents)
+      dir = File.dirname(target_file)
+      FileUtils.mkdir_p(dir) unless File.directory?(dir)
+
+      index = File.new(target_file, 'w')
+      index&.puts contents # may be nil when stubbing
+      logger&.info "Written #{target_file}"
+    ensure
+      index&.close
+    end
+
+    def copy_resource(dir, to_root)
+      src = ResourceLocator.resource(dir)
+      dest = "#{to_root}/#{dir}"
+      FileUtils.copy_entry(src, dest)
+    end
+  end
+
+  # Renders the index of a project diff report.
+  class LiquidProjectRenderer
+    include PmdTester
+    include ProjectHasher
+    include LiquidRenderer
+
+    def write_project_index(project, root)
+      liquid_env = {
+        'diff' => report_diff_to_h(project.report_diff),
+        'error_diffs' => errors_to_h(project),
+        'configerror_diffs' => configerrors_to_h(project),
+        'project_name' => project.name
+      }
+
+      # Renders index.html using liquid
+      write_file("#{root}/index.html", render_liquid('project_diff_report.html', liquid_env))
+      # generate array of violations in json
+      write_file("#{root}/project_data.js", dump_violations_json(project))
+    end
+
+    def dump_violations_json(project)
+      h = {
+        'source_link_base' => project.webview_url,
+        'source_link_template' => link_template(project),
+        **violations_to_hash(project)
+      }
+
+      project_data = JSON.fast_generate(h)
+      "let project = #{project_data}"
+    end
+  end
+end

data/lib/pmdtester/builders/pmd_report_builder.rb

@@ -1,82 +1,90 @@
 # frozen_string_literal: true
 
 require 'fileutils'
-require_relative '../cmd'
-require_relative '../project'
-require_relative '../pmd_branch_detail'
-require_relative '../pmd_report_detail'
+require 'rufus-scheduler'
 
 module PmdTester
   # Building pmd xml reports according to a list of standard
   # projects and branch of pmd source code
   class PmdReportBuilder
     include PmdTester
-    def initialize(branch_config, projects, local_git_repo, pmd_branch_name)
-      @branch_config = branch_config
+
+    def initialize(projects, options, branch_config, branch_name)
       @projects = projects
-      @local_git_repo = local_git_repo
-      @pmd_branch_name = pmd_branch_name
+      @local_git_repo = options.local_git_repo
+      @threads = options.threads
+      @error_recovery = options.error_recovery
+      @branch_config = branch_config
+      @pmd_branch_name = branch_name
       @pwd = Dir.getwd
 
-      @pmd_branch_details = PmdBranchDetail.new(pmd_branch_name)
+      @pmd_branch_details = PmdBranchDetail.new(@pmd_branch_name)
+      @project_builder = ProjectBuilder.new(@projects)
     end
 
-    def execute_reset_cmd(type, tag)
-      case type
-      when 'git'
-        reset_cmd = "git reset --hard #{tag}"
-      when 'hg'
-        reset_cmd = "hg up #{tag}"
-      end
+    def get_pmd_binary_file
+      logger.info "#{@pmd_branch_name}: Start packaging PMD"
+      Dir.chdir(@local_git_repo) do
+        build_branch_sha = Cmd.execute("git rev-parse #{@pmd_branch_name}^{commit}")
 
-      Cmd.execute(reset_cmd)
-    end
+        checkout_build_branch # needs a clean working tree, otherwise fails
 
-    def get_projects
-      logger.info 'Cloning projects started'
+        raise "Wrong branch #{get_last_commit_sha}" unless build_branch_sha == get_last_commit_sha
 
-      @projects.each do |project|
-        logger.info "Start cloning #{project.name} repository"
-        path = project.local_source_path
-        clone_cmd = "#{project.type} clone #{project.connection} #{path}"
-        if File.exist?(path)
-          logger.warn "Skipping clone, project path #{path} already exists"
+        distro_path = saved_distro_path(build_branch_sha)
+        logger.debug "#{@pmd_branch_name}: PMD Version is #{@pmd_version} " \
+                     "(sha=#{build_branch_sha})"
+        logger.debug "#{@pmd_branch_name}: distro_path=#{distro_path}"
+        if File.directory?(distro_path)
+          logger.info "#{@pmd_branch_name}: Skipping packaging - saved version exists " \
+                      " in #{distro_path}"
         else
-          Cmd.execute(clone_cmd)
+          build_pmd(into_dir: distro_path)
         end
 
-        Dir.chdir(path) do
-          execute_reset_cmd(project.type, project.tag)
-        end
-        logger.info "Cloning #{project.name} completed"
+        # we're still on the build branch
+        @pmd_branch_details.branch_last_sha = build_branch_sha
+        @pmd_branch_details.branch_last_message = get_last_commit_message
       end
+      logger.info "#{@pmd_branch_name}: Packaging PMD completed"
     end
 
-    def get_pmd_binary_file
-      logger.info 'Start packaging PMD'
-      Dir.chdir(@local_git_repo) do
-        checkout_cmd = "git checkout #{@pmd_branch_name}"
-        Cmd.execute(checkout_cmd)
-
-        @pmd_branch_details.branch_last_sha = get_last_commit_sha
-        @pmd_branch_details.branch_last_message = get_last_commit_message
-
-        package_cmd = './mvnw clean package -Dpmd.skip=true -Dmaven.test.skip=true' \
-                      ' -Dmaven.checkstyle.skip=true -Dmaven.javadoc.skip=true'
+    # builds pmd on currently checked out branch
+    def build_pmd(into_dir:)
+      # in CI there might have been a build performed already. In that case
+      # we reuse the build result, otherwise we build PMD freshly
+      pmd_dist_target = "pmd-dist/target/pmd-bin-#{@pmd_version}.zip"
+      binary_exists = File.exist?(pmd_dist_target)
+      logger.debug "#{@pmd_branch_name}: Does the file #{pmd_dist_target} exist? #{binary_exists} (cwd: #{Dir.getwd})"
+      if binary_exists
+        # that's a warning, because we don't know, whether this build really
+        # belongs to the current branch or whether it's from a previous branch.
+        # In CI, that's not a problem, because the workspace is always fresh.
+        logger.warn "#{@pmd_branch_name}: Reusing already existing #{pmd_dist_target}"
+      else
+        logger.info "#{@pmd_branch_name}: Building PMD #{@pmd_version}..."
+        package_cmd = './mvnw clean package' \
+                      ' -Dmaven.test.skip=true' \
+                      ' -Dmaven.javadoc.skip=true' \
+                      ' -Dmaven.source.skip=true' \
+                      ' -Dcheckstyle.skip=true' \
+                      ' -T1C'
         Cmd.execute(package_cmd)
+      end
 
-        version_cmd = "./mvnw -q -Dexec.executable=\"echo\" -Dexec.args='${project.version}' " \
-                      '--non-recursive org.codehaus.mojo:exec-maven-plugin:1.5.0:exec'
-        @pmd_version = Cmd.execute(version_cmd)
+      logger.info "#{@pmd_branch_name}: Extracting the zip"
+      Cmd.execute("unzip -qo #{pmd_dist_target} -d pmd-dist/target/exploded")
+      Cmd.execute("mv pmd-dist/target/exploded/pmd-bin-#{@pmd_version} #{into_dir}")
+    end
 
-        unzip_cmd = "unzip -qo pmd-dist/target/pmd-bin-#{@pmd_version}.zip -d #{@pwd}/target"
-        Cmd.execute(unzip_cmd)
-      end
-      logger.info 'Packaging PMD completed'
+    def determine_pmd_version
+      version_cmd = "./mvnw -q -Dexec.executable=\"echo\" -Dexec.args='${project.version}' " \
+                    '--non-recursive org.codehaus.mojo:exec-maven-plugin:1.5.0:exec'
+      Cmd.execute(version_cmd)
     end
 
     def get_last_commit_sha
-      get_last_commit_sha_cmd = 'git rev-parse HEAD'
+      get_last_commit_sha_cmd = 'git rev-parse HEAD^{commit}'
       Cmd.execute(get_last_commit_sha_cmd)
     end
 
@@ -85,31 +93,55 @@ module PmdTester
       Cmd.execute(get_last_commit_message_cmd)
     end
 
-    def generate_pmd_report(src_root_dir, report_file)
-      run_path = "target/pmd-bin-#{@pmd_version}/bin/run.sh"
-      pmd_cmd = "#{run_path} pmd -d #{src_root_dir} -f xml -R #{@branch_config} " \
-                "-r #{report_file} -failOnViolation false"
+    def generate_pmd_report(project)
+      error_recovery_options = @error_recovery ? 'PMD_JAVA_OPTS="-Dpmd.error_recovery -ea" ' : ''
+      run_path = "#{saved_distro_path(@pmd_branch_details.branch_last_sha)}/bin/run.sh"
+      pmd_cmd = "#{error_recovery_options}" \
+                "#{run_path} pmd -d #{project.local_source_path} -f xml " \
+                "-R #{project.get_config_path(@pmd_branch_name)} " \
+                "-r #{project.get_pmd_report_path(@pmd_branch_name)} " \
+                "-failOnViolation false -t #{@threads} " \
+                "#{project.auxclasspath}"
       start_time = Time.now
-      Cmd.execute(pmd_cmd)
+      if File.exist?(project.get_pmd_report_path(@pmd_branch_name))
+        logger.warn "#{@pmd_branch_name}: Skipping PMD run - report " \
+                    "#{project.get_pmd_report_path(@pmd_branch_name)} already exists"
+      else
+        Cmd.execute(pmd_cmd)
+      end
       end_time = Time.now
       [end_time - start_time, end_time]
     end
 
+    def generate_config_for(project)
+      logger.debug "Generating ruleset with excludes from #{@branch_config}"
+      doc = Nokogiri::XML(File.read(@branch_config))
+      ruleset = doc.at_css('ruleset')
+      ruleset.add_child("\n")
+      project.exclude_pattern.each do |exclude_pattern|
+        ruleset.add_child("    <exclude-pattern>#{exclude_pattern}</exclude-pattern>\n")
+      end
+
+      File.open(project.get_config_path(@pmd_branch_name), 'w') do |x|
+        x << doc.to_s
+      end
+
+      logger.debug "Created file #{project.get_config_path(@pmd_branch_name)}"
+    end
+
     def generate_pmd_reports
       logger.info "Generating PMD report started -- branch #{@pmd_branch_name}"
-      get_pmd_binary_file
 
       sum_time = 0
       @projects.each do |project|
-        logger.info "Generating #{project.name}'s PMD report'"
-        execution_time, end_time =
-          generate_pmd_report(project.local_source_path,
-                              project.get_pmd_report_path(@pmd_branch_name))
+        progress_logger = SimpleProgressLogger.new("generating #{project.name}'s PMD report")
+        progress_logger.start
+        generate_config_for(project)
+        execution_time, end_time = generate_pmd_report(project)
+        progress_logger.stop
         sum_time += execution_time
 
-        report_details = PmdReportDetail.new
-        report_details.execution_time = execution_time
-        report_details.timestamp = end_time
+        report_details = PmdReportDetail.new(execution_time: execution_time, timestamp: end_time)
         report_details.save(project.get_report_info_path(@pmd_branch_name))
         logger.info "#{project.name}'s PMD report was generated successfully"
       end
@@ -120,9 +152,46 @@ module PmdTester
       @pmd_branch_details
     end
 
+    # returns the branch details
     def build
-      get_projects
+      @project_builder.clone_projects
+      @project_builder.build_projects
+      get_pmd_binary_file
       generate_pmd_reports
     end
+
+    private
+
+    def checkout_build_branch
+      logger.info "#{@pmd_branch_name}: Checking out the branch"
+      # note that this would fail if the tree is dirty
+      Cmd.execute("git checkout #{@pmd_branch_name}")
+
+      # determine the version
+      @pmd_version = determine_pmd_version
+
+      return unless wd_has_dirty_git_changes
+
+      # working dir is dirty....
+      # we don't allow this because we need the SHA to address the zip file
+      logger.error "#{@pmd_branch_name}: Won\'t build without a clean working tree, " \
+                   'commit your changes'
+    end
+
+    def work_dir
+      "#{@pwd}/target"
+    end
+
+    # path to the unzipped distribution
+    # e.g. <cwd>/pmd-bin-<version>-<branch>-<sha>
+    def saved_distro_path(build_sha)
+      "#{work_dir}/pmd-bin-#{@pmd_version}" \
+      "-#{PmdBranchDetail.branch_filename(@pmd_branch_name)}" \
+      "-#{build_sha}"
+    end
+
+    def wd_has_dirty_git_changes
+      !Cmd.execute('git status --porcelain').empty?
+    end
   end
 end