pmdtester 1.0.0.pre.beta3 → 1.1.2

data/README.rdoc

@@ -1,9 +1,12 @@
 = pmdtester
 
+home  :: https://pmd.github.io
 code  :: https://github.com/pmd/pmd-regression-tester
 bugs  :: https://github.com/pmd/pmd-regression-tester/issues
 
-build-status :: {<img src="https://travis-ci.com/pmd/pmd-regression-tester.svg?branch=master" alt="Build Status" />}[https://travis-ci.com/pmd/pmd-regression-tester]
+build-status :: {<img src="https://github.com/pmd/pmd-regression-tester/workflows/build/badge.svg?branch=master" alt="Build Status" />}[https://github.com/pmd/pmd-regression-tester/actions?query=workflow%3Abuild]
+
+gem-version :: {<img src="https://badge.fury.io/rb/pmdtester.svg" alt="Gem Version" />}[https://rubygems.org/gems/pmdtester]
 
 == DESCRIPTION:
 
@@ -15,82 +18,110 @@ The diff report can be generated according to the base and patch branch of PMD
 on a list of standard projects(Spring Framework, Hibernate, Solr, etc.)
 
 == SYNOPSIS:
-===Options:
-    -r, --local-git-repo   path to the local PMD repository
-    -b, --base-branch      name of the base branch in local PMD repository
-    -p, --patch-branch     name of the patch branch in local PMD repository
-    -bc, --base-config     path to the base PMD configuration file default:PMDTESTER_INSTALLED_DIR/config/all-java.xml
-    -pc, --patch-config    path to the patch PMD configuration file default:PMDTESTER_INSTALLED_DIR/config/all-java.xml
-    -c, --config           path to the base and patch PMD configuration file
-    -l, --list-of-project  path to the file which contains the list of standard projects default:PMDTESTER_INSTALLED_DIR/config/project-list.xml
-    -m, --mode             the mode of the tool: 'local', 'online' or 'single'
+
+=== Options:
+    -r, --local-git-repo     path to the local PMD repository
+    -b, --base-branch        name of the base branch in local PMD repository
+    -p, --patch-branch       name of the patch branch in local PMD repository
+    -bc, --base-config       path to the base PMD configuration file default:PMDTESTER_INSTALLED_DIR/config/all-java.xml
+    -pc, --patch-config      path to the patch PMD configuration file default:PMDTESTER_INSTALLED_DIR/config/all-java.xml
+    -c, --config             path to the base and patch PMD configuration file
+    -l, --list-of-project    path to the file which contains the list of standard projects default:PMDTESTER_INSTALLED_DIR/config/project-list.xml
+    -m, --mode               the mode of the tool: 'local', 'online' or 'single'
           single: Set this option to 'single' if your patch branch contains changes
             for any option that can't work on master/base branch
           online: Set this option to 'online' if you want to download
             the PMD report of master/base branch rather than generating it locally
           local: Default option is 'local', PMD reports for the base and patch branches are generated locally.
 
-    -f, --html-flag        whether to not generate the html diff report in single mode
-    -a, --auto-gen-config  whether to generate configurations automatically based on branch differences,this option only works in online and local mode
-    -d, --debug            whether change log level to DEBUG to see more information
-    -v, --version          
-    -h, --help 
+    -t, --threads            Sets the number of threads used by PMD. Set threads to 0 to disable multi-threading processing. default:1
+    -f, --html-flag          whether to not generate the html diff report in single mode
+    -a, --auto-gen-config    whether to generate configurations automatically based on branch differences,this option only works in online and local mode
+    --keep-reports           whether to keep old reports and skip running PMD again if possible
+    -d, --debug              whether change log level to DEBUG to see more information
+    --error-recovery         enable error recovery mode when executing PMD. Might help to analyze errors.
+    --baseline-download-url  download url prefix from where to download the baseline in online mode default:https://sourceforge.net/projects/pmd/files/pmd-regression-tester/
+    -v, --version
+    -h, --help
+
 === Quick start
+
 ==== Run local mode
-  pmdtester -b master -p YOUR_DEVELOPMENT_BRANCH
+
+  pmdtester -b master -p YOUR_DEVELOPMENT_BRANCH -r PATH_TO_LOCAL_PMD_REPO -a
+
 ==== Run single mode
+
   pmdtester -p YOUR_DEVELOPMENT_BRANCH -pc CONFIG_ONLY_CONTAINS_NEW_PMD_JAVA_RULE -m single
+
 ==== Run online mode
-  pmdtester -b master -p YOUR_DEVELOPMENT_BRANCH -m online
+
+  pmdtester -b master -p YOUR_DEVELOPMENT_BRANCH -r PATH_TO_LOCAL_PMD_REPO -m online -a
+
 === Output
 The tool creates the following folders:
-  target  
-  ├── repositories         <- the analyzed projects are cloned here  
-  │   ├── PROJECT_NAME_1  
-  │   ├── ......  
-  │   └── PROJECT_NAME_n  
-  └── reports  
-      ├── BASE_BRANCH_NAME      <- the base baseline is placed here  
-      ├── PATCH_BRANCH_NAME      <- the patch baseline is placed here  
-      └── diff  
-          ├── index.xml <- the summary report of diff reports  
-          ├── base_config.xml  <- the resources of the summary report
-          ├── patch_config.xml  <- the resources fo the summary report
-          ├── css  <- css reources are placed here  
-          ├── PROJECT_NAME_1  
-              └── index.xml   <- the diff report of PROJECT_1  
-          ├── .......  
-          └── PROJECT_NAME_n  
-              └── index.xml   <- the diff report of PROJECT_N  
-====The baseline format
-  branch_name  
-  ├── branch_info.json  
-  ├── config.xml  
-  ├── STANDARD_PROJECT_NAME_1  
-  │   ├── report_info.json  
-  │   └── pmd_report.xml    
-  ├── ......................  
-  │   ├── report_info.json  
-  │   └── pmd_report.xml    
-  └── STANDARD_PROJECT_NAME_n  
+  target
+  ├── repositories         <- the analyzed projects are cloned here
+  │   ├── PROJECT_NAME_1
+  │   ├── ......
+  │   └── PROJECT_NAME_n
+  ├── reports
+  │   ├── BASE_BRANCH_NAME        <- the base baseline is placed here
+  │   ├── PATCH_BRANCH_NAME       <- the patch baseline is placed here
+  │   └── diff
+  │       ├── index.html          <- the summary report of diff reports
+  │       ├── base_config.xml     <- pmd config from the base branch
+  │       ├── patch_config.xml    <- pmd config from the patch branch
+  │       ├── css  <- css resources are placed here
+  │       ├── js   <- js resources
+  │       ├── PROJECT_NAME_1
+  │       │   ├── project_data.js <- contains the violations as js/json
+  │       │   └── index.html      <- the diff report of PROJECT_1
+  │       ├── .......
+  │       └── PROJECT_NAME_n
+  │           ├── project_data.js <- contains the violations as js/json
+  │           └── index.xml       <- the diff report of PROJECT_N
+  ├── pmd-bin-<version>-<branch_name>-<sha1>  <- cached pmd builds that are reused
+  └── pmd-bin-....
+
+==== The baseline format
+  branch_name
+  ├── branch_info.json
+  ├── config.xml
+  ├── STANDARD_PROJECT_NAME_1
+  │   ├── report_info.json
+  │   └── pmd_report.xml
+  ├── ......................
+  │   ├── report_info.json
+  │   └── pmd_report.xml
+  └── STANDARD_PROJECT_NAME_n
       ├── report_info.info
-      └── pmd_report.xml  
+      └── pmd_report.xml
 
 == REQUIREMENTS:
-* Ruby 2.4.1 or higher
+
+* Ruby 2.7 or higher
+
 === Runtime dependency
-  nokogiri ~> 1.8.2
-  slop ~> 4.6.2
+
+  nokogiri >= 1.11.0.rc4
+  slop ~> 4.6
+  differ ~> 0.1
+  rufus-scheduler ~> 3.5
+  logger-colors ~> 1.0
+  liquid >= 4.0
+
 === Development dependency
-  hoe ~> 3.17
-  hoe-bundler ~> 1.2
+
+  hoe ~> 3.22
+  hoe-bundler ~> 1.5
   hoe-git ~> 1.6
-  minitest ~> 5.10.1
-  mocha ~> 1.5.0
-  rdoc < 6, >= 4.0
-  rubocop ~> 0.56.0
-  test-unit ~> 3.2.3
-  
+  minitest ~> 5.10
+  mocha ~> 1.5
+  rdoc < 7, >= 4.0
+  rubocop ~> 0.81
+  test-unit ~> 3.2
+
 == INSTALL:
 
   gem install pmdtester --pre
@@ -99,8 +130,27 @@ The tool creates the following folders:
   git clone https://github.com/pmd/pmd-regression-tester.git
   cd pmd-regression-tester
   gem install bundler
-  bundle install
-  rake verify # run this command before commit your changes
+  bundle config set path "vendor/cache"
+  bundle install # once
+  bundle exec rake verify # run this command before commit your changes
+  bundle exec pmdtester ... # run this to directly execute pmdtester from source
+  
+  Run a single test class, e.g.:
+  bundle exec ruby -I test test/test_diff_report_builder.rb
   
+  Run a single test, e.g.:
+  bundle exec ruby -I test test/test_diff_report_builder.rb -n test_diff_report_builder
 
+=== Releasing
 
+* Update +History.md+ (version and date)
+* Update +lib/pmdtester.rb+ (version)
+* Run "bundle exec rake verify" and add new +pmdtester.gemspec+ (new version)
+* Commit ("Prepare release x.y.z").
+* Tag this commit ("git tag vx.y.z").
+* Update History.md and lib/pmdtester.rb for the next development version,
+  run again "bundle exec rake verify"
+* Commit ("Prepare next development version x.y.z-SNAPSHOT").
+* Push to master.
+* Push the tag. Github Actions will build and publish the new gem
+* Update the release notes on https://github.com/pmd/pmd-regression-tester/releases

data/Rakefile

@@ -14,29 +14,32 @@ Hoe.plugin :git
 hoe = Hoe.spec 'pmdtester' do
   self.version = PmdTester::VERSION
 
-  developer 'Andreas Dangel', 'andreas.dangel@adangel.org'
+  developer 'Andreas Dangel', 'andreas.dangel@pmd-code.org'
   developer 'Binguo Bao', 'djydewang@gmail.com'
+  developer 'Clément Fournier', 'clement.fournier76@gmail.com'
 
-  self.clean_globs = %w[target/reports/**/* target/test/**/* Gemfile.lock]
-  self.extra_deps += [['nokogiri', '~> 1.8.2'], ['slop', '~> 4.6.2']]
-  self.extra_dev_deps  += [
-    ['hoe-bundler',   '~> 1.2'],
+  self.clean_globs = %w[target/reports/**/* target/test/**/* target/dynamic-config.xml Gemfile.lock]
+  self.extra_deps += [['nokogiri', '>= 1.11.0.rc4'], ['slop', '~> 4.6'], ['differ', '~> 0.1'],
+                      ['rufus-scheduler', '~> 3.5'], ['logger-colors', '~> 1.0'],
+                      ['liquid', '>= 4.0']]
+  self.extra_dev_deps += [
+    ['hoe-bundler',   '~> 1.5'],
     ['hoe-git',       '~> 1.6'],
-    ['minitest',      '~> 5.10.1'],
-    ['mocha',         '~> 1.5.0'],
-    ['rubocop',       '~> 0.56.0'],
-    ['test-unit',     '~> 3.2.3']
+    ['minitest',      '~> 5.10'],
+    ['mocha',         '~> 1.5'],
+    # use the same version of rubocop as codacy
+    ['rubocop',       '~> 0.81'],
+    ['test-unit',     '~> 3.2'],
+    ['rdoc',          ['>= 4.0', '< 7']]
   ]
-  self.spec_extras[:required_rubygems_version] = '>= 2.4.1'
+  spec_extras[:required_ruby_version] = '>= 2.7'
 
   license 'BSD-2-Clause'
 end
 
 # Refers to
 # http://rubocop.readthedocs.io/en/latest/integration_with_other_tools/#rake-integration
-RuboCop::RakeTask.new(:rubocop) do |task|
-  task.patterns = %w[lib/**/*.rb test/**/*.rb]
-end
+RuboCop::RakeTask.new(:rubocop)
 
 # Run integration test cases
 Rake::TestTask.new('integration-test') do |task|
@@ -48,9 +51,18 @@ end
 
 desc 'generate the pmdtester.gemspec file'
 task 'hoe:spec' do
-  File.open("#{hoe.name}.gemspec", "w") { |f| f.write hoe.spec.to_ruby}
+  attention_message = '# DO NOT EDIT THIS FILE. Instead, edit Rakefile, and run `rake hoe:spec`.'
+  File.open("#{hoe.name}.gemspec", 'w') do |f|
+    f.puts attention_message
+    f.puts
+    f.write hoe.spec.to_ruby
+    f.puts
+    f.puts attention_message
+  end
 end
 
 desc 'verify code quality before committing changes'
-task 'verify' => ['clean', 'test', 'rubocop', 'bundler:gemfile', 'git:manifest']
+task 'verify' => ['clean', 'test', 'rubocop', 'git:manifest', 'hoe:spec', 'check_manifest'] do
+  Rake.application.invoke_task('bundler:gemfile[,true]')
+end
 # vim: syntax=ruby

data/config/all-java.xml

@@ -3,7 +3,7 @@
 <ruleset name="All Java Rules" 
     xmlns="http://pmd.sourceforge.net/ruleset/2.0.0"
     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
-    xsi:schemaLocation="http://pmd.sourceforge.net/ruleset/2.0.0 http://pmd.sourceforge.net/ruleset_2_0_0.xsd">
+    xsi:schemaLocation="http://pmd.sourceforge.net/ruleset/2.0.0 https://pmd.sourceforge.io/ruleset_2_0_0.xsd">
     <description>Every Java Rule in PMD</description>
 
     <rule ref="category/java/bestpractices.xml" />

data/config/design.xml

@@ -3,7 +3,7 @@
 <ruleset name="Design"
     xmlns="http://pmd.sourceforge.net/ruleset/2.0.0"
     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
-    xsi:schemaLocation="http://pmd.sourceforge.net/ruleset/2.0.0 http://pmd.sourceforge.net/ruleset_2_0_0.xsd">
+    xsi:schemaLocation="http://pmd.sourceforge.net/ruleset/2.0.0 https://pmd.sourceforge.io/ruleset_2_0_0.xsd">
 
     <description>
 The Design ruleset contains rules that flag suboptimal code implementations. Alternate approaches

data/config/projectlist_1_0_0.xsd

@@ -1,4 +1,5 @@
 <?xml version="1.0" ?>
+<!-- version 1.0.0 -->
 <xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
   <xs:element name="projectlist">
     <xs:complexType>
@@ -25,4 +26,4 @@
       <xs:element name="exclude-pattern" type="xs:string" minOccurs="0" maxOccurs="unbounded"/>
     </xs:sequence>
   </xs:complexType>
-</xs:schema>
+</xs:schema>

data/config/projectlist_1_1_0.xsd

@@ -0,0 +1,31 @@
+<?xml version="1.0" ?>
+<!-- version 1.1.0 -->
+<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
+  <xs:element name="projectlist">
+    <xs:complexType>
+      <xs:sequence>
+        <xs:element name="description" type="xs:string" minOccurs="1" maxOccurs="1"/>
+        <xs:element name="project" type="project" minOccurs="1" maxOccurs="unbounded"/>
+      </xs:sequence>
+    </xs:complexType>
+  </xs:element>
+  <xs:complexType name="project">
+    <xs:sequence>
+      <xs:element name="name" type="xs:string" minOccurs="1" maxOccurs="1"/>
+      <xs:element name="type" minOccurs="1" maxOccurs="1">
+        <xs:simpleType>
+          <xs:restriction base="xs:string">
+            <xs:enumeration value="git"/>
+            <xs:enumeration value="hg"/>
+          </xs:restriction>
+        </xs:simpleType>
+      </xs:element>
+      <xs:element name="connection" type="xs:string" minOccurs="1" maxOccurs="1"/>
+      <xs:element name="webview-url" type="xs:string" minOccurs="0" maxOccurs="1"/>
+      <xs:element name="tag" type="xs:string" minOccurs="0" maxOccurs="1"/>
+      <xs:element name="exclude-pattern" type="xs:string" minOccurs="0" maxOccurs="unbounded"/>
+      <xs:element name="build-command" type="xs:string" minOccurs="0" maxOccurs="1"/>
+      <xs:element name="auxclasspath-command" type="xs:string" minOccurs="0" maxOccurs="1"/>
+    </xs:sequence>
+  </xs:complexType>
+</xs:schema>

data/lib/pmdtester.rb

@@ -1,21 +1,26 @@
 # frozen_string_literal: true
 
 require 'logger'
+require 'logger/colors'
 
 require_relative 'pmdtester/cmd'
+require_relative 'pmdtester/collection_by_file'
 require_relative 'pmdtester/pmd_branch_detail'
+require_relative 'pmdtester/pmd_configerror'
 require_relative 'pmdtester/pmd_error'
 require_relative 'pmdtester/pmd_report_detail'
+require_relative 'pmdtester/pmd_tester_utils'
 require_relative 'pmdtester/pmd_violation'
 require_relative 'pmdtester/project'
 require_relative 'pmdtester/report_diff'
 require_relative 'pmdtester/resource_locator'
 require_relative 'pmdtester/runner'
 
-require_relative 'pmdtester/builders/html_report_builder'
-require_relative 'pmdtester/builders/diff_builder'
-require_relative 'pmdtester/builders/diff_report_builder'
+require_relative 'pmdtester/builders/simple_progress_logger'
+require_relative 'pmdtester/builders/project_builder'
+require_relative 'pmdtester/builders/project_hasher'
 require_relative 'pmdtester/builders/pmd_report_builder'
+require_relative 'pmdtester/builders/liquid_renderer'
 require_relative 'pmdtester/builders/rule_set_builder'
 require_relative 'pmdtester/builders/summary_report_builder'
 
@@ -27,7 +32,10 @@ require_relative 'pmdtester/parsers/projects_parser'
 # and unexpected behaviors will not be introduced to PMD project
 # after fixing an issue and new rules can work as expected.
 module PmdTester
-  VERSION = '1.0.0-beta3'
+  VERSION = '1.1.2'
+  BASE = 'base'
+  PATCH = 'patch'
+  PR_NUM_ENV_VAR = 'PMD_CI_PULL_REQUEST_NUMBER' # see PmdBranchDetail
 
   def logger
     PmdTester.logger

data/lib/pmdtester/builders/liquid_renderer.rb

@@ -0,0 +1,73 @@
+# frozen_string_literal: true
+
+require 'liquid'
+require 'json'
+
+module PmdTester
+  # A module to include in classes that use a Liquid template
+  # to generate content.
+  module LiquidRenderer
+    include PmdTester
+
+    def render_liquid(template_path, env)
+      to_render = File.read(ResourceLocator.resource(template_path))
+      includes = Liquid::LocalFileSystem.new(ResourceLocator.resource('_includes'), '%s.html')
+      Liquid::Template.file_system = includes
+      template = Liquid::Template.parse(to_render, error_mode: :strict)
+      template.render!(env, { strict_variables: true })
+    end
+
+    def render_and_write(template_path, target_file, env)
+      write_file(target_file, render_liquid(template_path, env))
+    end
+
+    def write_file(target_file, contents)
+      dir = File.dirname(target_file)
+      FileUtils.mkdir_p(dir) unless File.directory?(dir)
+
+      index = File.new(target_file, 'w')
+      index&.puts contents # may be nil when stubbing
+      logger&.info "Written #{target_file}"
+    ensure
+      index&.close
+    end
+
+    def copy_resource(dir, to_root)
+      src = ResourceLocator.resource(dir)
+      dest = "#{to_root}/#{dir}"
+      FileUtils.copy_entry(src, dest)
+    end
+  end
+
+  # Renders the index of a project diff report.
+  class LiquidProjectRenderer
+    include PmdTester
+    include ProjectHasher
+    include LiquidRenderer
+
+    def write_project_index(project, root)
+      liquid_env = {
+        'diff' => report_diff_to_h(project.report_diff),
+        'error_diffs' => errors_to_h(project),
+        'configerror_diffs' => configerrors_to_h(project),
+        'project_name' => project.name
+      }
+
+      # Renders index.html using liquid
+      write_file("#{root}/index.html", render_liquid('project_diff_report.html', liquid_env))
+      # generate array of violations in json
+      write_file("#{root}/project_data.js", dump_violations_json(project))
+    end
+
+    def dump_violations_json(project)
+      h = {
+        'source_link_base' => project.webview_url,
+        'source_link_template' => link_template(project),
+        **violations_to_hash(project)
+      }
+
+      project_data = JSON.fast_generate(h)
+      "let project = #{project_data}"
+    end
+  end
+end