@freshworks/shiftleft-tools 1.1.8

package/templates/skills/enhance-test-pipeline/SKILL-java.md

@@ -0,0 +1,483 @@
+# Enhance Test Pipeline — Java Spring Boot
+
+Detect gaps and fill them in a Java/Spring Boot repo that ALREADY HAS some test pipeline pieces but not all. The key difference from `/setup-test-pipeline` is being careful NOT to overwrite existing configs that are working correctly.
+
+## When to Use
+
+Invoke when:
+- "enhance the test pipeline", "improve the test pipeline"
+- "we have unit tests but no mutation tests"
+- "add mutation tests to existing setup"
+- "our Jenkinsfile is missing the quality report stage"
+- "update the scripts to the latest version"
+- The repo already has postman/ and/or unit tests but is missing mutation, API coverage, or quality report
+
+---
+
+## Reference Implementations
+
+Before writing any code, check these canonical repos. They are the ground truth — use them instead of guessing.
+
+**Java canonical repos:**
+- `mp-installation` — PIT mutation, JaCoCo, multi-product Postman, API coverage matrix, quality report, Jenkins ShiftLeft
+- `dp-apps` — same stack, original reference
+
+**Script source (the `shiftleft-tools` package — pulled at runtime):**
+```bash
+# Stage the latest library scripts from the installed package into postman/scripts/.
+# Gitignored and refreshed on every run — never copy or commit them by hand.
+shiftleft stage-scripts
+```
+
+The library scripts live only in the `shiftleft-tools` package. `shiftleft stage-scripts`
+(and `shiftleft test`, which stages then runs) pulls the current versions in — do
+not copy from a templates directory or write them from scratch.
+
+---
+
+## Phase 1: Inspect — Full Audit
+
+### 1.1 Unit tests — health check
+
+```bash
+mvn test -q 2>&1 | tail -10
+```
+
+Are they passing? How many? Any flaky failures?
+
+### 1.2 JaCoCo — version check
+
+```bash
+grep -A2 "jacoco-maven-plugin" pom.xml | grep version
+```
+
+If JaCoCo version < 0.8.8 and JDK >= 17: this will cause instrumentation errors. Must upgrade to 0.8.11+.
+
+### 1.3 PIT mutation testing — configuration audit
+
+```bash
+grep -A50 "pitest-maven" pom.xml | head -60
+```
+
+Check:
+- Is it inside a `<profile id="mutation-tests">`? (It should be — if it's in `<build><plugins>` it runs on every `mvn install`, which is wrong)
+- Does `targetClasses` list actual packages (not `com.yourcompany.*`)? Read the actual src/main/java structure and compare
+- Is `<timestampedReports>false</timestampedReports>` set? (Required for CI report linking)
+- Are XML + HTML output formats both enabled? (XML required for quality report)
+- Is `pitest-junit5-plugin` present? (Required for JUnit 5)
+
+```bash
+find src/main/java -type d | sort
+```
+
+Compare actual packages to what's configured in `targetClasses`. If they don't match, the config needs updating.
+
+### 1.4 run-all.sh — version check
+
+```bash
+ls postman/scripts/run-all.sh
+grep "skip-report\|skip-mutation\|skip-postman\|skip-unit\|no-delay\|--env" postman/scripts/run-all.sh 2>/dev/null | head -10
+```
+
+Does it have the full `--skip-*` flag set? Old versions may only have `--skip-mutation`. The current version needs:
+- `--env {local|staging}` — target environment
+- `--skip-unit`, `--skip-mutation`, `--skip-postman`, `--skip-coverage`, `--skip-report`
+- `--no-delay` — for CI
+
+### 1.5 Missing scripts audit
+
+Check which scripts exist and which are missing:
+
+```bash
+ls postman/scripts/
+ls postman/scripts/lib/ 2>/dev/null
+ls postman/scripts/runners/ 2>/dev/null
+```
+
+Required scripts that may be missing:
+
+**Top-level scripts:**
+- `stage-report-artifacts.sh` — copies JaCoCo/PIT HTML to `postman/reports/jacoco/` and `postman/reports/pit/` for Jenkins-safe links in quality report
+- `generate-quality-report.sh` — standalone Jenkins quality report generator
+- `runners/run-tests-local.sh` — runs Postman against local + WireMock
+- `runners/run-tests-staging.sh` — runs Postman against staging (ShiftLeft)
+
+**Python library modules (all required):**
+- `lib/report_generator.py` — CLI entry point (dispatches to report modules, ~100 lines)
+- `lib/report_utils.py` — shared CSS, helper functions, UI component builders
+- `lib/report_consolidated.py` — consolidated Newman test results report
+- `lib/report_migration.py` — migration comparison report
+- `lib/report_mutation.py` — PIT mutation testing report
+- `lib/report_unit.py` — Surefire + JaCoCo + mutation parsers and unit report
+- `lib/report_combined.py` — combined quality report (unit + API tabs) — THE main report
+- `lib/api_coverage.py` — API coverage matrix generator from source + Postman
+
+### 1.6 Jenkinsfile — stage audit
+
+```bash
+grep -E "stage\(|runMutationTests|quality.report|QualityReport|ShiftLeft|stage-report" Jenkinsfile 2>/dev/null
+```
+
+Check for:
+- Mutation Tests stage: does it call `run-all.sh --skip-unit --skip-postman --skip-coverage --skip-report`?
+- ShiftLeft stage: does it generate API coverage and quality report after Postman tests?
+- Quality report publishing: does it `publishHTML` with `includes: '**/*'`? (without `includes`, CSS won't load)
+- Artifacts: does it `archiveArtifacts postman/reports/**`?
+
+---
+
+## Phase 2: Report Gaps
+
+Present a precise audit:
+
+```
+Enhancement Audit Results
+=========================
+Component                Issue / Action Needed
+------------------------------------------------------------------
+Unit Tests               [OK / N failures need fixing before mutation]
+JaCoCo                   [OK / version X.X.X needs upgrade to 0.8.11]
+PIT Config               [OK / targetClasses has placeholders — needs real packages
+                           / missing XML output / missing profile / timestampedReports not set]
+run-all.sh               [OK / STALE — missing --skip-* flags / MISSING]
+stage-report-artifacts.sh [OK / MISSING — required for quality report]
+generate-quality-report.sh [OK / MISSING]
+runners/run-tests-local.sh [OK / MISSING]
+runners/run-tests-staging.sh [OK / MISSING]
+lib/report_generator.py  [OK / MISSING / STALE — entry point only, ~120 lines]
+lib/report_utils.py      [OK / MISSING — shared CSS + helpers]
+lib/report_consolidated.py [OK / MISSING]
+lib/report_migration.py  [OK / MISSING]
+lib/report_mutation.py   [OK / MISSING]
+lib/report_unit.py       [OK / MISSING]
+lib/report_combined.py   [OK / MISSING]
+lib/api_coverage.py      [OK / MISSING / STALE]
+Jenkinsfile              [OK / missing mutation stage / missing quality report / wrong run-all.sh args]
+```
+
+---
+
+## Phase 3: Confirm
+
+Show the user exactly what will change:
+
+```
+I will make the following targeted changes:
+  1. Upgrade JaCoCo from X.X.X to 0.8.11 in pom.xml
+  2. Fix PIT targetClasses — replace placeholders with actual packages
+     Found: service, mapper, util (not validator — doesn't exist in this repo)
+  3. Add <timestampedReports>false</timestampedReports> to PIT config
+  4. Update run-all.sh to current version (adds --skip-*, --env, --no-delay flags)
+  5. Add missing stage-report-artifacts.sh
+  6. Add missing generate-quality-report.sh
+  7. Add missing runners/run-tests-local.sh and runners/run-tests-staging.sh
+  8. Copy all lib/report_*.py modules and lib/api_coverage.py from shiftleft-tools templates
+  9. Update Jenkinsfile ShiftLeft stage to add quality report generation
+
+I will NOT change:
+  - Existing unit tests (passing, no issues)
+  - Existing Postman collections
+  - Existing WireMock mocks
+
+Proceed? [Y/n]
+```
+
+Wait for confirmation before any changes.
+
+---
+
+## Phase 4: Execute — Targeted Changes Only
+
+### 4.1 JaCoCo version upgrade (if needed)
+
+Only if current version < 0.8.8 or JDK >= 17:
+
+```xml
+<!-- Update version only, preserve all other config -->
+<version>0.8.11</version>
+```
+
+### 4.2 Fix PIT configuration (if needed)
+
+For each issue found:
+
+**Issue: not in profile** — wrap existing config in `<profile id="mutation-tests">`:
+```xml
+<profiles>
+    <profile>
+        <id>mutation-tests</id>
+        <activation><activeByDefault>false</activeByDefault></activation>
+        <build><plugins>
+            <!-- move existing pitest-maven config here -->
+        </plugins></build>
+    </profile>
+</profiles>
+```
+
+**Issue: placeholder targetClasses** — read actual packages first, then update:
+```bash
+find src/main/java -type d | grep -v test | sort
+```
+Update `targetClasses` and `targetTests` with the actual package paths found.
+
+**Issue: missing timestampedReports** — add:
+```xml
+<timestampedReports>false</timestampedReports>
+```
+
+**Issue: missing XML output** — add:
+```xml
+<outputFormats>
+    <outputFormat>HTML</outputFormat>
+    <outputFormat>XML</outputFormat>
+</outputFormats>
+```
+
+**Issue: missing pitest-junit5-plugin** — add to PIT plugin dependencies.
+
+### 4.3 Refresh scripts from the shiftleft-tools package
+
+Run `shiftleft stage-scripts` — it re-stages every library script from the
+installed package, so stale or missing ones are replaced automatically. You do
+not pick individual files or copy by hand; staging always brings the current
+versions (the committed `run-all.sh` shim and repo-owned files are left alone).
+
+**Data flow between scripts:**
+```
+mvn test            → target/surefire-reports/*.xml       → report_unit.py
+mvn jacoco:report   → target/site/jacoco/jacoco.xml       → report_unit.py
+mvn pitest          → target/pit-reports/mutations.xml    → report_mutation.py, report_unit.py
+Newman run          → postman/reports/newman/*.json       → report_consolidated.py
+coverage matrix     → postman/reports/api-coverage.json  → api_coverage.py
+stage-artifacts     → postman/reports/jacoco/, pit/       → quality report HTML links
+
+report_generator.py (CLI entry) dispatches to:
+  ├─ report_unit.py      → postman/reports/unit-test-report-*.html
+  ├─ report_consolidated.py → postman/reports/consolidated-report-*.html
+  ├─ report_mutation.py  → postman/reports/mutation-report-*.html
+  └─ report_combined.py  → postman/reports/quality-report-*.html  (THE main report)
+```
+
+**Quality report HTML structure** — `report_combined.py` generates a single-page HTML with:
+
+```
+┌──────────────────────────────────────────────────────────────┐
+│  [Health Badge: 🟢 Healthy | 🟡 Good | 🔴 Needs Attention]   │
+│  Health: all green = Healthy, any warning = Good, any red = Needs Attention │
+├─────────────┬─────────────┬─────────────┬──────────┬─────────┤
+│ Unit Tests  │ Line Cov    │ Mutation    │ API      │ API Cov │
+│ N/M passed  │ XX.X%       │ Score XX.X% │ N passed │ XX.X%   │
+│ [green/red] │ [green/warn │ [green/warn │ [grn/red]│[grn/wrn]│
+│             │ /red]       │ /red]       │          │ /red]   │
+└─────────────┴─────────────┴─────────────┴──────────┴─────────┘
+
+┌─────────────────────────────────────────────────────────────┐
+│ [Unit Tests tab] | [API Tests tab]   ← CSS tab switcher     │
+├─────────────────────────────────────────────────────────────┤
+│ Unit Tests tab contains sub-tabs:                           │
+│   [Test Results] [Coverage] [Mutations] [Survived Mutations]│
+│                                                             │
+│ API Tests tab contains sub-tabs:                           │
+│   [Summary] [by Collection] [API Coverage Matrix]          │
+└─────────────────────────────────────────────────────────────┘
+```
+
+Card thresholds:
+- Unit Tests: green if 0 failures, red if any failures
+- Line Coverage: green ≥80%, yellow ≥60%, red <60%
+- Mutation Score: green ≥80%, yellow ≥60%, red <60%
+- API Tests: green if 0 failed, red if any failed
+- API Coverage: green ≥80%, yellow ≥60%, red <60%
+
+**Survived Mutations tab** shows up to 15 survived mutations (class, method, line, description). Empty state: "No survived mutations. All mutations were killed by tests."
+
+**If the repo has a Maven submodule** (e.g., `installation/`), adjust `stage-report-artifacts.sh`:
+```bash
+# Default template uses target/ — for submodules, change to:
+JACOCO_SRC="$PROJECT_ROOT/installation/target/site/jacoco"
+PIT_DIR="$PROJECT_ROOT/installation/target/pit-reports"
+```
+
+Scripts to update if stale:
+- `run-all.sh` — if missing `--skip-*` flags
+- `lib/report_generator.py` — if it's a monolith (>500 lines); replace with split modules
+
+Scripts to add if missing:
+- `stage-report-artifacts.sh`
+- `generate-quality-report.sh`
+- `runners/run-tests-local.sh` (if only `run-local.sh` exists)
+- `runners/run-tests-staging.sh` (if only `run-staging.sh` exists)
+
+Make all `.sh` files executable:
+```bash
+chmod +x postman/scripts/*.sh postman/scripts/runners/*.sh
+```
+
+### 4.4 Update Jenkinsfile (targeted edits only)
+
+Only modify the specific missing pieces. Do NOT rewrite the Jenkinsfile.
+
+**Missing mutation stage** — add after Unit Tests stage:
+```groovy
+stage('Mutation Tests') {
+    when {
+        expression {
+            params.deployTo == 'dev' && !params.deployOnly
+        }
+    }
+    steps {
+        script {
+            runMutationTests()
+        }
+    }
+}
+```
+
+And add the function:
+```groovy
+def runMutationTests() {
+    echo "Running PIT Mutation Tests..."
+    try {
+        sh """#!/bin/bash --login
+            cd postman/scripts
+            ./run-all.sh --skip-unit --skip-postman --skip-coverage --skip-report --no-delay
+        """
+    } catch (Exception e) {
+        echo "Mutation tests failed: ${e.getMessage()}"
+        throw e
+    }
+}
+```
+
+**Missing quality report in ShiftLeft** — add after Postman test block:
+```groovy
+// API coverage matrix
+sh """cd postman/scripts && ./run-all.sh --skip-unit --skip-mutation --skip-postman --skip-report --no-delay"""
+
+// Quality report
+sh """cd postman/scripts
+./stage-report-artifacts.sh "\${WORKSPACE}" "\${WORKSPACE}/postman/reports" || true
+./run-all.sh --skip-unit --skip-mutation --skip-postman --skip-coverage --no-delay"""
+
+publishHTML([
+    allowMissing: true,
+    alwaysLinkToLastBuild: true,
+    keepAll: true,
+    reportDir: 'postman/reports',
+    reportFiles: 'quality-report-*.html',
+    includes: '**/*',       // REQUIRED — without this, CSS/subdirs won't load
+    reportName: 'QualityReport',
+    reportTitles: 'Combined Quality Report'
+])
+archiveArtifacts artifacts: 'postman/reports/**', allowEmptyArchive: true
+```
+
+---
+
+## PIT XML Format Spec
+
+When writing or debugging `report_mutation.py` or `report_unit.py`, the PIT XML format is:
+
+```xml
+<mutations>
+  <mutation detected="true" status="KILLED" numberOfTestsRun="3">
+    <sourceFile>MyClass.java</sourceFile>
+    <mutatedClass>com.example.service.MyClass</mutatedClass>
+    <mutatedMethod>myMethod</mutatedMethod>
+    <methodDescription>(Ljava/lang/String;)Z</methodDescription>
+    <lineNumber>42</lineNumber>
+    <mutator>org.pitest.mutationtest.engine.gregor.mutators.NegateConditionalsMutator</mutator>
+    <indexes><index>5</index></indexes>
+    <blocks><block>2</block></blocks>
+    <killingTest>com.example.service.MyClassTest.[engine:junit-jupiter]/...</killingTest>
+    <description>negated conditional</description>
+  </mutation>
+</mutations>
+```
+
+**Status attribute values** (read from `mutation.get('status')`, NOT from `detected` attribute):
+- `KILLED` — test caught the mutation (good)
+- `SURVIVED` — no test caught it (bad — report these in Survived Mutations tab)
+- `NO_COVERAGE` — no test covers this line at all
+- `TIMED_OUT` — mutation caused an infinite loop (treated as killed)
+- `RUN_ERROR` — mutation caused a runtime error (treated as killed)
+
+**Common bug:** Do NOT use `mutation.get('detected')` — use `mutation.get('status')`. The `detected` attribute (`"true"/"false"`) loses the distinction between NO_COVERAGE and SURVIVED.
+
+---
+
+## Error Handling Rules
+
+When any input file is missing, degrade gracefully — never crash the report:
+
+| Missing file | Behavior |
+|---|---|
+| `surefire-reports/*.xml` | Show "No unit test data" card, continue |
+| `jacoco.xml` | Show "Coverage data unavailable", skip coverage card |
+| `mutations.xml` | Show "Mutation data unavailable", skip mutation card |
+| `newman/*.json` | Show "No API test results", skip API tabs |
+| `api-coverage.json` | Show "API coverage data unavailable", skip coverage card |
+
+Pattern from `report_unit.py`:
+```python
+try:
+    tree = ET.parse(mutations_xml)
+    # ... parse
+except Exception as e:
+    print(f"Warning: Could not parse mutations.xml: {e}")
+    # return empty/default results dict, not None
+```
+
+Always return a results dict with zero values rather than `None` — callers check keys, not None.
+
+---
+
+## Phase 5: Verify
+
+Run the minimum verification needed for what was changed:
+
+If PIT config was updated:
+```bash
+mvn org.pitest:pitest-maven:mutationCoverage -Pmutation-tests -q
+# For submodule: mvn -pl <module> org.pitest:pitest-maven:mutationCoverage -Pmutation-tests -q
+```
+
+Confirm `target/pit-reports/mutations.xml` exists and has `status` attributes on `<mutation>` elements.
+
+If scripts were updated:
+```bash
+cd postman/scripts
+./run-all.sh --skip-postman --no-delay
+```
+
+If quality report was added:
+```bash
+# Run with existing data to verify quality report generates
+./run-all.sh --skip-unit --skip-mutation --skip-postman --skip-coverage --no-delay
+```
+
+Report what changed and the current state of each component.
+
+---
+
+## Verification Checklist
+
+- [ ] All targeted changes made (none more, none less)
+- [ ] JaCoCo version is 0.8.8+ (if applicable)
+- [ ] PIT targetClasses matches actual packages (verified with `find src/main/java -type d`)
+- [ ] PIT is inside a `mutation-tests` profile
+- [ ] `<timestampedReports>false</timestampedReports>` is set
+- [ ] XML output format is enabled
+- [ ] `target/pit-reports/mutations.xml` has `status` attributes (not just `detected`)
+- [ ] `run-all.sh` has full `--skip-*` flag set
+- [ ] `stage-report-artifacts.sh` exists and uses correct `target/` path for this repo
+- [ ] `generate-quality-report.sh` exists
+- [ ] Jenkinsfile has Mutation Tests stage
+- [ ] Jenkinsfile ShiftLeft stage generates and publishes quality report
+- [ ] Jenkinsfile `publishHTML` has `includes: '**/*'`
+- [ ] `./run-all.sh --skip-postman --no-delay` completes successfully
+- [ ] Quality report HTML is generated with 5 metric cards
+- [ ] Quality report has Unit Tests tab + API Tests tab
+- [ ] Unit Tests tab has Survived Mutations sub-tab
+- [ ] JaCoCo and PIT links in quality report open correctly (not 404)
+- [ ] All `.sh` files are executable