test-pmd-rule 0.1.0 → 1.0.1

package/README.md

@@ -1,6 +1,6 @@
 # test-pmd-rule
 
-A standalone PMD Rule Tester for Apex rules, with XPath analysis and coverage validation.
+A high-performance PMD Rule Tester for Apex rules with directory support, parallel execution, XPath analysis, and LCOV coverage reporting.
 
 [![Node Version](https://img.shields.io/badge/node-%3E%3D25.0.0-brightgreen)](https://nodejs.org/)
 [![TypeScript](https://img.shields.io/badge/TypeScript-5.x-blue)](https://www.typescriptlang.org/)
@@ -10,57 +10,257 @@ A standalone PMD Rule Tester for Apex rules, with XPath analysis and coverage va
 
 This tool validates PMD Apex rules by testing them against examples embedded in the rule XML files. It ensures rules work correctly, provide adequate test coverage, and don't contain hardcoded values that should be parameterized.
 
+**Key Capabilities:**
+
+- **Mass Testing**: Test entire directories of rule files with recursive discovery
+- **High Performance**: CPU-core-based parallel execution for blazing-fast testing
+- **Coverage Analysis**: Generate LCOV coverage reports for XPath expressions
+- **Comprehensive Validation**: Full XPath analysis with line number tracking
+
+This project was born out of necessity as hood tooling for both humans and AI Agents was essential for furthering the [sca-extra](https://github.com/starch-uk/sca-extra) project.
+
+### Output
+
+The tool provides detailed output including:
+
+- **Test Details**: Individual test results for each example (violation/valid tests)
+- **Test Summary**: Overall statistics (examples tested, passed, total violations)
+- **XPath Coverage**: Detailed coverage analysis showing:
+    - Node types coverage with line numbers for missing items
+    - Conditionals coverage (only missing items shown)
+    - Attributes coverage with line numbers for missing items
+    - Operators coverage
+- **Final Status**: Overall pass/fail status
+
+Example output (single file):
+
+```
+🧪 Testing rule: rulesets/code-style/MyRule.xml
+
+📋 Test Details:
+   - Example 1 Test: Violation ✅
+   - Example 1 Test: Valid ✅
+   - Example 2 Test: Violation ✅
+
+📊 Test Summary:
+  Examples tested: 2
+  Examples passed: 2
+  Total violations: 3
+  Rule triggers violations: ✅ Yes
+
+🔍 XPath Coverage:
+  Status: ⚠️ Incomplete
+  Coverage items: 2
+    1. ⚠️ Node types: 4/6 covered
+         Missing:
+          - Line 43: VariableDeclaration
+          - Line 50: VariableExpression
+    2. ⚠️ Attributes: 2/3 covered
+         Missing:
+          - Line 52: BeginLine
+
+✅ All tests passed!
+```
+
+Example output (directory with parallel execution):
+
+```
+🚀 Processing 15 rule file(s) with 8 parallel workers
+   Each file will test examples with up to 16 parallel workers
+
+🧪 Testing rule: rulesets/code-style/MyRule.xml
+
+📋 Test Details:
+   - Example 1 Test: Violation ✅
+   - Example 1 Test: Valid ✅
+   - Example 2 Test: Violation ✅
+
+📊 Test Summary:
+  Examples tested: 2
+  Examples passed: 2
+  Total violations: 3
+  Rule triggers violations: ✅ Yes
+
+🔍 XPath Coverage:
+  Status: ✅ Complete
+
+✅ All tests passed!
+
+[... more files tested in parallel ...]
+
+🎯 OVERALL RESULTS
+============================================================
+Total files processed: 15
+Successful: 15
+Failed: 0
+```
+
 ## Features
 
+- **Directory Support**: Test entire directories recursively - finds and tests all `**/*.xml` files
+- **Parallel Execution**: CPU-core-based thread pools for blazing-fast testing of multiple rules and examples
 - **Rule Validation**: Tests PMD rules against their documented examples with detailed test results
 - **XPath Analysis**: Analyzes XPath expressions for node types, operators, attributes, and conditionals
 - **Coverage Checking**: Validates that XPath expressions are properly tested with line number references
+- **LCOV Coverage Reports**: Generate `coverage/lcov.info` files tracking XPath line coverage
 - **Hardcoded Value Detection**: Identifies values that should be parameterized
 - **Line Number Tracking**: Shows exact line numbers in XML files for missing coverage items
 - **TypeScript**: Written in TypeScript for better maintainability
 - **100% Test Coverage**: All code is thoroughly tested (lines, functions, branches, statements)
 - **Modular Architecture**: Clean separation of concerns with low complexity functions
 
+## How does it work?
+
+The tool extracts examples from `<example>` tags in your PMD rule XML files and validates them against the rule's XPath expression. Examples must be configured to indicate which code should trigger violations and which should be valid.
+
+### Example Configuration
+
+Examples can be marked using two different formats:
+
+#### Format 1: Section Headers
+
+Use `// Violation:` and `// Valid:` comments to mark sections of code:
+
+```xml
+<example>
+// Violation: Public method should trigger rule
+public class TestClass {
+    public void testMethod() {
+        // method body
+    }
+}
+
+// Valid: Private method should not trigger rule
+public class ValidClass {
+    private void testMethod() {
+        // method body
+    }
+}
+</example>
+```
+
+#### Format 2: Inline Markers
+
+Use `// ❌` for violations and `// ✅` for valid code:
+
+```xml
+<example>
+public class TestClass {
+    public void violationMethod() { // ❌ Void method should trigger rule
+        // method body
+    }
+    
+    private Integer validMethod() { // ✅ Integer method should not trigger rule
+        // method body
+    }
+}
+</example>
+```
+
+### How Examples Are Processed
+
+1. **Extraction**: The tool reads all `<example>` tags from your rule XML file
+2. **Parsing**: Each example is parsed to identify violation and valid code sections
+3. **Test File Creation**: Temporary Apex test files are generated with the example code
+4. **PMD Execution**: PMD is run against the test files to check if violations occur as expected
+5. **Validation**: The tool verifies that:
+    - Violation examples actually trigger the rule
+    - Valid examples do not trigger the rule
+    - XPath expressions are properly covered by the examples
+
+### Multiple Examples
+
+You can include multiple `<example>` tags in a single rule XML file. Each example is tested independently:
+
+```xml
+<rule name="MyRule" ...>
+  <example>
+  // First example with violations and valid code
+  ...
+  </example>
+
+  <example>
+  // Second example with different scenarios
+  ...
+  </example>
+</rule>
+```
+
+## Requirements
+
+- **Node.js**: ≥25.0.0
+- **PMD CLI**: Available in PATH (see [PMD Installation](https://pmd.github.io/pmd/pmd_userdocs_installation.html))
+- **Package Manager**: pnpm ≥10.0.0 (recommended)
+
 ## Installation
 
 ```bash
-# Clone the repository
-git clone https://github.com/your-org/test-pmd-rule.git
-cd test-pmd-rule
-
-# Install dependencies
-pnpm install
+# Install from npm
+npm install -g test-pmd-rule
 
-# Build the project
-pnpm build
+# Or use npx to run without installing globally
+npx test-pmd-rule path/to/rule.xml
 ```
 
 ## Usage
 
 ```bash
 # Test a single rule
-node dist/test-pmd-rule.js path/to/rule.xml
+test-pmd-rule path/to/rule.xml
 
-# Or after building
-pnpm build
-node dist/test-pmd-rule.js rulesets/code-style/AvoidMagicNumbers.xml
+# Test all XML files in a directory (recursive)
+test-pmd-rule ../sca-extra/rulesets
+
+# Generate LCOV coverage reports
+test-pmd-rule rulesets/code-style/AvoidMagicNumbers.xml --coverage
+
+# Test directory with coverage reports
+test-pmd-rule ../sca-extra/rulesets --coverage
+
+# Or use npx without installing globally
+npx test-pmd-rule path/to/rule.xml
 ```
 
+**Arguments:**
+
+- `<rule.xml|directory>`: Path to XML rule file or directory containing XML files (recursive)
+- `--coverage`: Generate LCOV coverage report in `coverage/lcov.info`
+
 The tool will:
 
-1. Extract examples from the PMD rule XML file
-2. Parse violation and valid markers from example code
-3. Create temporary Apex test files
-4. Run PMD against the test files
-5. Validate that violations occur for violation examples and don't occur for valid examples
-6. Analyze XPath coverage and show missing items with line numbers
-7. Report comprehensive test results
+1. **Directory Discovery**: If given a directory, recursively find all `**/*.xml` files
+2. **Parallel Processing**: Test multiple files concurrently using CPU-core-based thread pools
+3. **Extract Examples**: Parse examples from `<example>` tags in PMD rule XML files
+4. **Parse Markers**: Identify violation (`// ❌` or `// Violation:`) and valid (`// ✅` or `// Valid:`) code sections
+5. **Test File Creation**: Generate temporary Apex test files with example code
+6. **Parallel PMD Execution**: Run PMD against test files with concurrent workers
+7. **Validation**: Verify violations occur for violation examples and don't occur for valid examples
+8. **XPath Coverage Analysis**: Analyze XPath expressions and show coverage with line numbers
+9. **Coverage Reports**: Generate LCOV format reports when `--coverage` flag is used
+10. **Comprehensive Results**: Report detailed test results with parallel processing stats
 
-## Requirements
+## Coverage Reporting
 
-- **Node.js**: ≥25.0.0
-- **PMD CLI**: Available in PATH (see [PMD Installation](https://pmd.github.io/pmd/pmd_userdocs_installation.html))
-- **Package Manager**: pnpm ≥10.0.0 (recommended)
+When using the `--coverage` flag, the tool generates LCOV format coverage reports in `coverage/lcov.info`. This tracks which lines of your XPath expressions are covered by your test examples.
+
+### Coverage Data
+
+- **XPath Lines**: Tracks coverage of XPath expression lines in the XML rule files
+- **Component Lines**: Tracks coverage of XPath components (node types, attributes, etc.)
+- **LCOV Format**: Compatible with coverage tools like VS Code Coverage Gutters, GitHub Actions, etc.
+
+### Example LCOV Output
+
+```
+SF:rulesets/code-style/MyRule.xml
+DA:75,1
+DA:76,1
+DA:79,0
+DA:82,1
+end_of_record
+```
+
+This shows that lines 75, 76, 79, and 82 in `MyRule.xml` were executed, with line 79 having 0 coverage (uncovered XPath code).
 
 ## Development
 
@@ -110,6 +310,9 @@ pnpm format:check
 
 # Pre-commit checks (format, lint, test coverage)
 pnpm pre-commit
+
+# Test with coverage reports
+pnpm test:coverage
 ```
 
 ### Project Structure
@@ -118,6 +321,9 @@ pnpm pre-commit
 src/
 ├── cli/                    # Command-line interface
 │   └── main.ts             # CLI entry point
+├── coverage/               # Coverage reporting
+│   ├── generateLcov.ts    # LCOV report generation
+│   └── trackCoverage.ts   # Coverage data collection
 ├── pmd/                    # PMD execution utilities
 │   ├── runPMD.ts           # PMD CLI execution
 │   └── parseViolations.ts  # XML violation parsing
@@ -125,6 +331,8 @@ src/
 │   ├── parseExample.ts     # Example code parsing
 │   ├── extractMarkers.ts  # Violation/valid marker extraction
 │   └── createTestFile.ts  # Test file generation
+├── utils/                  # Utility functions
+│   └── concurrency.ts      # Parallel execution utilities
 ├── xpath/                  # XPath analysis and validation
 │   ├── extractXPath.ts    # XPath extraction from XML
 │   ├── analyzeXPath.ts    # XPath analysis orchestration
@@ -151,49 +359,6 @@ tests/                      # Unit and integration tests
 docs/                       # Documentation (symlinked from agent-docs)
 ```
 
-## Output Format
-
-The tool provides detailed output including:
-
-- **Test Details**: Individual test results for each example (violation/valid tests)
-- **Test Summary**: Overall statistics (examples tested, passed, total violations)
-- **XPath Coverage**: Detailed coverage analysis showing:
-    - Node types coverage with line numbers for missing items
-    - Conditionals coverage (only missing items shown)
-    - Attributes coverage with line numbers for missing items
-    - Operators coverage
-- **Final Status**: Overall pass/fail status
-
-Example output:
-
-```
-🧪 Testing rule: rulesets/code-style/MyRule.xml
-
-📋 Test Details:
-   - Example 1 Test: Violation ✅
-   - Example 1 Test: Valid ✅
-   - Example 2 Test: Violation ✅
-
-📊 Test Summary:
-  Examples tested: 2
-  Examples passed: 2
-  Total violations: 3
-  Rule triggers violations: ✅ Yes
-
-🔍 XPath Coverage:
-  Status: ⚠️ Incomplete
-  Coverage items: 2
-    1. ⚠️ Node types: 4/6 covered
-         Missing:
-          - Line 43: VariableDeclaration
-          - Line 50: VariableExpression
-    2. ⚠️ Attributes: 2/3 covered
-         Missing:
-          - Line 52: BeginLine
-
-✅ All tests passed!
-```
-
 ## Documentation
 
 Comprehensive documentation is available in the [`docs/`](docs/) directory (symlinked from [agent-docs](https://github.com/starch-uk/agent-docs/)):

package/dist/test-pmd-rule.js

@@ -976,6 +976,43 @@ function createTestFile({
   } else {
     codeToInclude = [...parsed.violations, ...parsed.valids];
   }
+  const hasClassDefinition = codeToInclude.some(
+    (line) => line.trim().startsWith("public class ") || line.trim().startsWith("class ")
+  );
+  if (hasClassDefinition) {
+    const extractedCode = [];
+    let insideMethod = false;
+    let braceDepth = 0;
+    for (const line of codeToInclude) {
+      const trimmed = line.trim();
+      if (trimmed.startsWith("public class ") || trimmed.startsWith("class ") || trimmed.startsWith("private class ")) {
+        continue;
+      }
+      const isMethodSignature = (trimmed.startsWith("public ") || trimmed.startsWith("private ") || trimmed.startsWith("protected ")) && trimmed.includes("() {") && !insideMethod;
+      const INITIAL_BRACE_DEPTH = 1;
+      const ZERO_BRACE_DEPTH = 0;
+      if (isMethodSignature) {
+        insideMethod = true;
+        braceDepth = INITIAL_BRACE_DEPTH;
+        continue;
+      }
+      if (insideMethod) {
+        const openBraces = (line.match(/{/g) ?? []).length;
+        const closeBraces = (line.match(/}/g) ?? []).length;
+        braceDepth += openBraces - closeBraces;
+        if (braceDepth > ZERO_BRACE_DEPTH) {
+          extractedCode.push(line);
+        }
+        if (braceDepth <= ZERO_BRACE_DEPTH) {
+          insideMethod = false;
+          braceDepth = ZERO_BRACE_DEPTH;
+        }
+      } else if (!trimmed.startsWith("}") && trimmed.length > EMPTY_LENGTH && !trimmed.startsWith("public ") && !trimmed.startsWith("private ")) {
+        extractedCode.push(line);
+      }
+    }
+    codeToInclude = extractedCode.length > EMPTY_LENGTH ? extractedCode : codeToInclude;
+  }
   if (codeToInclude.length > EMPTY_LENGTH) {
     classContent += `    public void testMethod${String(exampleIndex)}() {
 `;
@@ -2083,7 +2120,8 @@ async function main() {
     console.error("\u274C Invalid path argument");
     process.exit(EXIT_CODE_ERROR);
   }
-  const hasCoverageFlag = args.length > SECOND_ARG_INDEX && args[SECOND_ARG_INDEX] === "--coverage";
+  const COVERAGE_FLAG = "--coverage";
+  const hasCoverageFlag = args.length > SECOND_ARG_INDEX && args[SECOND_ARG_INDEX] === COVERAGE_FLAG;
   if (!existsSync2(pathArg)) {
     console.error(`\u274C Path not found: ${pathArg}`);
     process.exit(EXIT_CODE_ERROR);
@@ -2158,9 +2196,13 @@ async function main() {
     ).map(
       (tracker) => tracker.getCoverageData()
     );
+    const COVERAGE_REPORT_PATH = "coverage/lcov.info";
     try {
-      generateLcovReport(coverageData, "coverage/lcov.info");
-      console.log("\n\u{1F4CA} Coverage report generated: coverage/lcov.info");
+      generateLcovReport(coverageData, COVERAGE_REPORT_PATH);
+      console.log(
+        `
+\u{1F4CA} Coverage report generated: ${COVERAGE_REPORT_PATH}`
+      );
     } catch (error) {
       const errorMessage = error instanceof Error ? error.message : String(error);
       console.error(