@freshworks/shiftleft-tools 1.1.8

package/templates/skills/setup-test-pipeline/SKILL-java.md

@@ -0,0 +1,454 @@
+# Setup Test Pipeline — Java Spring Boot
+
+Set up a complete test pipeline from scratch (or near-scratch) for a Java/Spring Boot repo. This skill orchestrates the full pipeline: unit tests, JaCoCo coverage, PIT mutation tests, Postman API tests, API coverage matrix, combined quality report, and Jenkins integration.
+
+Use this skill when a repo has NOTHING (or only bare unit tests). For repos that already have partial pipeline setup, use `/enhance-test-pipeline` instead.
+
+## When to Use
+
+Invoke when:
+- "set up the full test pipeline"
+- "add complete test infrastructure to this Java repo"
+- "set up unit tests, mutation tests, and postman tests"
+- "I want everything: unit tests, coverage, mutation, API tests, quality report"
+
+---
+
+## Reference Implementations
+
+Before writing any code, check these canonical repos. They are the ground truth — use them instead of guessing.
+
+**Java canonical repos:**
+- `mp-installation` — PIT mutation, JaCoCo, multi-product Postman, API coverage matrix, quality report, Jenkins ShiftLeft
+- `dp-apps` — same stack, original reference
+
+**Script source (the `shiftleft-tools` package — pulled at runtime):**
+```bash
+# Stage the latest library scripts from the installed package into postman/scripts/.
+# These are gitignored and refreshed on every run — never copy or commit them by hand.
+shiftleft stage-scripts
+```
+
+Do not copy scripts from a templates directory or write them from scratch. The
+library scripts (`runners/`, `report-generators/`, `lib/`, `auth/`, `infra/`,
+`database/`) live only in the `shiftleft-tools` package; `shiftleft stage-scripts`
+(and `shiftleft test`, which stages then runs) pulls the current versions in.
+
+---
+
+## Phase 1: Inspect — Audit What Exists
+
+Run each check and record the result (present/missing/partial).
+
+### 1.1 Unit tests
+
+```bash
+ls src/test/java/
+mvn test -q 2>&1 | tail -5
+```
+
+- Present and passing: unit tests exist and pass
+- Present but failing: unit tests exist but have failures (must fix before mutation tests)
+- Missing: no test directory or no test files
+
+### 1.2 JaCoCo coverage
+
+```bash
+grep -r "jacoco" pom.xml
+```
+
+- Present: JaCoCo already configured
+- Missing: needs to be added
+
+### 1.3 Postman infrastructure
+
+```bash
+ls postman/ 2>/dev/null
+ls postman/collections/ 2>/dev/null
+ls postman/scripts/ 2>/dev/null
+ls postman/scripts/run-all.sh 2>/dev/null
+```
+
+- Present: full postman/ directory with collections and scripts
+- Partial: directory exists but scripts or collections missing
+- Missing: no postman/ directory
+
+### 1.4 PIT mutation testing
+
+```bash
+grep -r "pitest-maven" pom.xml
+```
+
+- Present: PIT configured (check if it's in a profile — it should be)
+- Missing: needs to be added
+
+### 1.5 API coverage matrix
+
+```bash
+ls postman/scripts/report-generators/java-api-coverage-matrix.sh 2>/dev/null
+```
+
+### 1.6 Quality report
+
+```bash
+ls postman/scripts/run-all.sh 2>/dev/null
+grep "skip-report\|skip-unit\|skip-mutation" postman/scripts/run-all.sh 2>/dev/null | head -5
+```
+
+- Present and modern: run-all.sh exists and has --skip-* flags
+- Present but stale: run-all.sh exists but is an old version without flags
+- Missing
+
+### 1.7 Jenkinsfile
+
+```bash
+ls Jenkinsfile
+grep -E "Mutation|ShiftLeft|Quality|quality-report" Jenkinsfile 2>/dev/null
+```
+
+- Complete: has Unit Tests, Mutation Tests, ShiftLeft stages with quality report generation
+- Partial: has some stages but missing mutation or quality report
+- Missing: no Jenkinsfile
+
+---
+
+## Phase 2: Report Gaps
+
+Present a clear status table to the user:
+
+```
+Pipeline Audit Results
+======================
+Component               Status
+-----------------------------------------------
+Unit Tests              [PRESENT / MISSING / FAILING]
+JaCoCo Coverage         [PRESENT / MISSING]
+Postman Infrastructure  [PRESENT / PARTIAL / MISSING]
+PIT Mutation Tests      [PRESENT / MISSING]
+API Coverage Matrix     [PRESENT / MISSING]
+run-all.sh (modern)     [PRESENT / STALE / MISSING]
+Jenkinsfile             [COMPLETE / PARTIAL / MISSING]
+```
+
+---
+
+## Phase 3: Confirm
+
+Tell the user exactly what will be set up:
+
+```
+I will set up the following components in order:
+  1. [if missing] Postman infrastructure (collections, scripts, config)
+  2. [if missing] JaCoCo coverage plugin in pom.xml
+  3. [if missing] PIT mutation testing (profile-gated, discover packages first)
+  4. Stage library scripts from the shiftleft-tools package (`shiftleft stage-scripts`):
+       postman/scripts/run-all.sh
+       postman/scripts/report-generators/mutation-report.sh
+       postman/scripts/report-generators/java-api-coverage-matrix.sh
+       postman/scripts/report-generators/generate-consolidated-report.sh
+       postman/scripts/report-generators/stage-report-artifacts.sh
+       postman/scripts/lib/report_generator.py
+       postman/scripts/lib/report_utils.py
+       postman/scripts/lib/report_consolidated.py
+       postman/scripts/lib/report_migration.py
+       postman/scripts/lib/report_mutation.py
+       postman/scripts/lib/report_unit.py
+       postman/scripts/lib/report_combined.py
+       postman/scripts/lib/api_coverage.py
+  5. [if partial] Update Jenkinsfile with mutation + ShiftLeft + quality report stages
+
+This will take approximately 5-10 minutes.
+Proceed? [Y/n]
+```
+
+Wait for confirmation before making any changes.
+
+---
+
+## Phase 4: Execute in Order
+
+### Step A: Postman infrastructure (if missing)
+
+If no `postman/` directory exists, invoke `/setup-api-tests` skill for the full Postman setup. This skill handles:
+- Directory structure (postman/collections/, scripts/, environments/, config/)
+- Initial test collection scaffolding
+- Local environment configuration
+- WireMock mock setup (if needed)
+
+### Step B: JaCoCo coverage (if missing)
+
+Add to the main `<build><plugins>` section of pom.xml:
+
+```xml
+<plugin>
+    <groupId>org.jacoco</groupId>
+    <artifactId>jacoco-maven-plugin</artifactId>
+    <version>0.8.11</version>
+    <executions>
+        <execution>
+            <goals>
+                <goal>prepare-agent</goal>
+            </goals>
+        </execution>
+        <execution>
+            <id>report</id>
+            <phase>test</phase>
+            <goals>
+                <goal>report</goal>
+            </goals>
+        </execution>
+    </executions>
+</plugin>
+```
+
+Verify: `mvn test jacoco:report -q` and confirm `target/site/jacoco/index.html` exists.
+
+### Step C: PIT mutation testing (if missing)
+
+Follow the `setup-mutation-tests` SKILL-java.md skill in full:
+1. Inspect source packages (never use placeholders)
+2. Confirm targetClasses with user
+3. Add `<profile id="mutation-tests">` to pom.xml
+
+Key requirements:
+- PIT version 1.15.3 with pitest-junit5-plugin 1.2.1
+- STRONGER mutators
+- `<timestampedReports>false</timestampedReports>` — required for CI
+- XML + HTML output — XML required for quality report script
+- Mutation threshold: 60%, coverage threshold: 70%
+
+### Step D: Stage scripts from the shiftleft-tools package
+
+**Do not copy or write these by hand.** Run `shiftleft stage-scripts` to pull the
+current library scripts from the installed package into `postman/scripts/` (they
+are gitignored and refreshed on every `shiftleft test` run). If the project has a
+Maven submodule (e.g., `installation/`), adjust the `target/` paths in
+`stage-report-artifacts.sh` to point to `<module>/target/` instead of `target/`.
+
+After staging, `postman/scripts/` contains (only `run-all.sh` is committed — a
+thin shim; everything else is the gitignored, staged cache):
+
+```
+postman/scripts/
+├── run-all.sh                         # master pipeline runner
+├── mutation-report.sh                 # runs PIT, generates HTML
+├── java-api-coverage-matrix.sh        # endpoint coverage matrix
+├── generate-consolidated-report.sh    # aggregates Newman JSON
+├── stage-report-artifacts.sh          # copies JaCoCo/PIT HTML for Jenkins
+├── generate-quality-report.sh         # standalone Jenkins quality report
+├── runners/
+│   ├── run-tests-local.sh             # Postman against local + WireMock
+│   └── run-tests-staging.sh           # Postman against staging (ShiftLeft)
+└── lib/
+    ├── report_generator.py            # CLI entry point
+    ├── report_utils.py                # shared CSS, helpers
+    ├── report_consolidated.py         # consolidated Newman results
+    ├── report_migration.py            # migration comparison
+    ├── report_mutation.py             # PIT mutation report
+    ├── report_unit.py                 # Surefire + JaCoCo parsers
+    ├── report_combined.py             # combined quality report (tabs)
+    └── api_coverage.py                # API coverage matrix generator
+```
+
+Make all `.sh` files executable:
+```bash
+chmod +x postman/scripts/*.sh postman/scripts/runners/*.sh
+```
+
+**Data flow between scripts:**
+```
+mvn test            → target/surefire-reports/*.xml       → report_unit.py
+mvn jacoco:report   → target/site/jacoco/jacoco.xml       → report_unit.py
+mvn pitest          → target/pit-reports/mutations.xml    → report_mutation.py, report_unit.py
+Newman run          → postman/reports/newman/*.json       → report_consolidated.py
+coverage matrix     → postman/reports/api-coverage.json  → api_coverage.py
+stage-artifacts     → postman/reports/jacoco/, pit/       → quality report HTML links
+
+report_generator.py (CLI entry) dispatches to:
+  ├─ report_unit.py      → postman/reports/unit-test-report-*.html
+  ├─ report_consolidated.py → postman/reports/consolidated-report-*.html
+  ├─ report_mutation.py  → postman/reports/mutation-report-*.html
+  └─ report_combined.py  → postman/reports/quality-report-*.html  (THE main report)
+```
+
+**Quality report HTML structure** — `report_combined.py` generates a single-page HTML with:
+
+```
+┌──────────────────────────────────────────────────────────────┐
+│  [Health Badge: 🟢 Healthy | 🟡 Good | 🔴 Needs Attention]   │
+│  Health: all green = Healthy, any warning = Good, any red = Needs Attention │
+├─────────────┬─────────────┬─────────────┬──────────┬─────────┤
+│ Unit Tests  │ Line Cov    │ Mutation    │ API      │ API Cov │
+│ N/M passed  │ XX.X%       │ Score XX.X% │ N passed │ XX.X%   │
+│ [green/red] │ [green/warn │ [green/warn │ [grn/red]│[grn/wrn]│
+│             │ /red]       │ /red]       │          │ /red]   │
+└─────────────┴─────────────┴─────────────┴──────────┴─────────┘
+
+┌─────────────────────────────────────────────────────────────┐
+│ [Unit Tests tab] | [API Tests tab]   ← CSS tab switcher     │
+├─────────────────────────────────────────────────────────────┤
+│ Unit Tests tab contains sub-tabs:                           │
+│   [Test Results] [Coverage] [Mutations] [Survived Mutations]│
+│                                                             │
+│ API Tests tab contains sub-tabs:                            │
+│   [Summary] [by Collection] [API Coverage Matrix]          │
+└─────────────────────────────────────────────────────────────┘
+```
+
+Card thresholds:
+- Unit Tests: green if 0 failures, red if any failures
+- Line Coverage: green ≥80%, yellow ≥60%, red <60%
+- Mutation Score: green ≥80%, yellow ≥60%, red <60%
+- API Tests: green if 0 failed, red if any failed
+- API Coverage: green ≥80%, yellow ≥60%, red <60%
+
+**Survived Mutations tab** — shows up to 15 survived mutations:
+- Class name, method, line number, mutation description
+- Empty state: "No survived mutations. All mutations were killed by tests."
+
+### Step E: Update Jenkinsfile (if missing stages)
+
+If the Jenkinsfile exists but is missing mutation or quality report stages, update it to match the `templates/jenkins/Jenkinsfile-java.groovy` pattern:
+
+Required stages:
+1. `Unit tests` — `mvn clean install`
+2. `Mutation Tests` — calls `runMutationTests()` which runs `./run-all.sh --skip-unit --skip-postman --skip-coverage --skip-report --no-delay`
+3. `Shift Left Integration Test - Isolation` — runs Postman against staging, generates API coverage matrix, generates quality report
+
+Key quality report pattern in Jenkins:
+```groovy
+// In ShiftLeft stage, after Postman tests:
+sh "./run-all.sh --skip-unit --skip-mutation --skip-postman --skip-coverage --no-delay"
+publishHTML([
+    reportDir: 'postman/reports',
+    reportFiles: 'quality-report-*.html',
+    includes: '**/*',       // REQUIRED — without this, CSS/subdirs won't load
+    reportName: 'QualityReport'
+])
+archiveArtifacts artifacts: 'postman/reports/**', allowEmptyArchive: true
+```
+
+Replace placeholders in the template:
+- `SERVICE_NAME` → your actual service name (e.g. `mp-installation`)
+- `NAMESPACE` → your Kubernetes namespace
+- `ISO_HEADER` → isoforge routing header for this service
+- `ISO_HEADER_VALUE` → the header value (regex or account ID)
+
+---
+
+## PIT XML Format Spec
+
+When writing or debugging `report_mutation.py` or `report_unit.py`, the PIT XML format is:
+
+```xml
+<mutations>
+  <mutation detected="true" status="KILLED" numberOfTestsRun="3">
+    <sourceFile>MyClass.java</sourceFile>
+    <mutatedClass>com.example.service.MyClass</mutatedClass>
+    <mutatedMethod>myMethod</mutatedMethod>
+    <methodDescription>(Ljava/lang/String;)Z</methodDescription>
+    <lineNumber>42</lineNumber>
+    <mutator>org.pitest.mutationtest.engine.gregor.mutators.NegateConditionalsMutator</mutator>
+    <indexes><index>5</index></indexes>
+    <blocks><block>2</block></blocks>
+    <killingTest>com.example.service.MyClassTest.[engine:junit-jupiter]/...</killingTest>
+    <description>negated conditional</description>
+  </mutation>
+</mutations>
+```
+
+**Status attribute values** (read from `mutation.get('status')`, NOT from `detected` attribute):
+- `KILLED` — test caught the mutation (good)
+- `SURVIVED` — no test caught it (bad — report these in Survived Mutations tab)
+- `NO_COVERAGE` — no test covers this line at all
+- `TIMED_OUT` — mutation caused an infinite loop (treated as killed)
+- `RUN_ERROR` — mutation caused a runtime error (treated as killed)
+
+**Common bug:** Do NOT use `mutation.get('detected')` — use `mutation.get('status')`. The `detected` attribute (`"true"/"false"`) loses the distinction between NO_COVERAGE and SURVIVED.
+
+---
+
+## Error Handling Rules
+
+When any input file is missing, degrade gracefully — never crash the report:
+
+| Missing file | Behavior |
+|---|---|
+| `surefire-reports/*.xml` | Show "No unit test data" card, continue |
+| `jacoco.xml` | Show "Coverage data unavailable", skip coverage card |
+| `mutations.xml` | Show "Mutation data unavailable", skip mutation card |
+| `newman/*.json` | Show "No API test results", skip API tabs |
+| `api-coverage.json` | Show "API coverage data unavailable", skip coverage card |
+
+Pattern used in `report_unit.py`:
+```python
+try:
+    tree = ET.parse(mutations_xml)
+    # ... parse
+except Exception as e:
+    print(f"Warning: Could not parse mutations.xml: {e}")
+    # return empty/default results dict, not None
+```
+
+Always return a results dict with zero values rather than `None` — callers check keys, not None.
+
+---
+
+## Phase 5: Verify
+
+Run a two-phase verification:
+
+### Verify Phase A: Unit + Mutation + Coverage (no Postman needed)
+
+```bash
+cd postman/scripts
+./run-all.sh --skip-postman --no-delay
+```
+
+Expected: unit tests pass, PIT mutation tests run, quality report generates.
+
+### Verify Phase B: Postman + API Coverage + Quality Report (skip mutation)
+
+```bash
+cd postman/scripts
+./run-all.sh --skip-mutation --no-delay
+```
+
+Expected: Postman tests run against local environment, API coverage matrix generates, quality report generates.
+
+### Report to user
+
+```
+Pipeline setup complete!
+
+Verification results:
+  Unit Tests:     [PASS/FAIL] — N tests, M% line coverage
+  Mutation Tests: [PASS/FAIL] — X% mutation score
+  Postman Tests:  [PASS/FAIL] — N collections, M assertions
+  API Coverage:   [PASS/FAIL] — X% endpoints covered
+  Quality Report: [GENERATED] — postman/reports/quality-report-*.html
+
+Next steps:
+  - Local run-all: cd postman/scripts && ./run-all.sh
+  - Staging run: cd postman/scripts && ./run-all.sh --env staging
+  - Jenkins: push a PR and comment "shiftleft" to trigger integration tests
+```
+
+---
+
+## Verification Checklist
+
+- [ ] `mvn test` passes
+- [ ] `mvn test jacoco:report` generates `target/site/jacoco/index.html` (or `<module>/target/site/jacoco/index.html` if submodule)
+- [ ] `mvn org.pitest:pitest-maven:mutationCoverage -Pmutation-tests` runs without error
+- [ ] `target/pit-reports/mutations.xml` exists with `<mutation>` elements that have a `status` attribute
+- [ ] `./run-all.sh --skip-postman --no-delay` completes successfully
+- [ ] `./run-all.sh --skip-mutation --no-delay` completes successfully
+- [ ] `postman/reports/quality-report-*.html` is generated
+- [ ] Quality report has 5 metric cards (unit tests, line coverage, mutation score, API assertions, API coverage)
+- [ ] Quality report has Unit Tests tab and API Tests tab
+- [ ] Unit Tests tab has Survived Mutations sub-tab
+- [ ] JaCoCo and PIT report links in quality report are not 404 (check `stage-report-artifacts.sh` ran)
+- [ ] All `.sh` files are executable (`chmod +x`)
+- [ ] Jenkinsfile `publishHTML` has `includes: '**/*'` — otherwise CSS won't load
+- [ ] Jenkinsfile has Mutation Tests stage and Quality Report generation