@freshworks/shiftleft-tools 1.1.8

package/templates/skills/enhance-test-pipeline/SKILL-node.md

@@ -0,0 +1,431 @@
+# Enhance Test Pipeline — Node.js / Express
+
+Detect gaps and fill them in a Node.js/Express repo that ALREADY HAS some test pipeline pieces but not all. The key difference from `/setup-test-pipeline` is being careful NOT to overwrite existing configs that are working correctly.
+
+## When to Use
+
+Invoke when:
+- "enhance the test pipeline", "improve the test pipeline"
+- "we have unit tests but no mutation tests"
+- "add Stryker to existing setup"
+- "our Jenkinsfile is missing the quality report stage"
+- "update the scripts to the latest version"
+- The repo already has postman/ and/or unit tests but is missing mutation, API coverage, or quality report
+
+---
+
+## Phase 1: Inspect — Full Audit
+
+### 1.1 Unit tests — health check
+
+```bash
+cat package.json | grep '"test"'
+yarn test 2>&1 | tail -10
+```
+
+Are they passing? How many? Any flaky failures?
+
+### 1.2 Coverage configuration — check
+
+```bash
+cat package.json | grep -E '"nyc"|"c8"|"istanbul"|"coverage"'
+ls .nycrc .nycrc.json .istanbul.yml 2>/dev/null
+```
+
+Is coverage configured? Does `yarn test:coverage` (or equivalent) work?
+
+### 1.3 Stryker — configuration audit
+
+```bash
+cat package.json | grep stryker
+ls stryker.config.js stryker.config.ts 2>/dev/null
+cat stryker.config.js 2>/dev/null
+```
+
+Check:
+- Is `mutate` array **narrow** (no `src/**`, no blanket `src/helpers/**`)? Count files — must be **under 80**
+- Are `controllers/`, `routes/`, `models/`, `config/` **absent** from mutate?
+- Are external-service wrappers excluded from `mutate`? (aws.js, freshid.js, chargebee.js, etc.)
+- Is `coverageAnalysis: 'perTest'` set? (Fastest mode)
+- Is `incremental: true` set? (Reuse previous results)
+- Are both `mutation-tests` (since mode) and `mutation-tests:full` scripts in package.json?
+- Is `thresholds.break` set to 0? (Should never hard-fail CI on score alone)
+
+```bash
+ls src/
+find src -maxdepth 1 -type d | sort
+```
+
+Compare actual src structure to what's in `mutate` array.
+
+### 1.4 run-all.sh — version check
+
+```bash
+ls postman/scripts/run-all.sh
+grep "skip-report\|skip-mutation\|skip-postman\|skip-unit\|no-delay\|--env" postman/scripts/run-all.sh 2>/dev/null | head -10
+```
+
+Does it have the full `--skip-*` flag set? Does it have `--env {local|staging}` support?
+
+### 1.5 Missing scripts audit
+
+```bash
+ls postman/scripts/
+ls postman/scripts/lib/ 2>/dev/null
+```
+
+Required scripts that may be missing:
+- `run-mutation-tests.sh` — Stryker two-mode runner (since vs full)
+- `lib/cleanup-stryker.sh` — kills Stryker processes, cleans temp files
+- `report-generators/node-api-coverage-matrix.sh` — generates endpoint coverage matrix
+- `report-generators/stage-report-artifacts.sh` — required for Jenkins-safe artifact links
+- `lib/report_generator.py` — CLI entry point (dispatches to report modules)
+- `lib/report_utils.py` — shared CSS, helpers, UI component builders
+- `lib/report_consolidated.py` — consolidated Newman test results report
+- `lib/report_migration.py` — migration comparison report
+- `lib/report_mutation.py` — Stryker/PIT mutation testing report
+- `lib/report_unit.py` — unit test parsers and report
+- `lib/report_combined.py` — combined quality report (unit + API tabs)
+- `lib/api_coverage_node.py` — Express API coverage generator
+
+### 1.6 Node security guardrails — audit
+
+```bash
+cat postman/.npmrc 2>/dev/null || echo "MISSING: postman/.npmrc"
+ls postman/package-lock.json 2>/dev/null || echo "MISSING: package-lock.json"
+grep "npm install\|npm ci\|ignore-scripts" Jenkinsfile 2>/dev/null
+grep "GH_TOKEN\|GITHUB_RUNWAYCI_TOKEN\|AWS_ACCESS_KEY_ID\|AWS_SECRET_ACCESS_KEY" Jenkinsfile 2>/dev/null
+```
+
+Required `.npmrc` in `postman/` directory:
+```ini
+ignore-scripts=true
+audit-level=high
+audit-signatures=true
+strict-ssl=true
+registry=https://registry.npmjs.org/
+```
+
+Check:
+- Does `postman/.npmrc` exist with all five settings?
+- Does `postman/package-lock.json` exist? (CI must use `npm ci`, never `npm install`)
+- Does the Jenkinsfile use `npm ci --ignore-scripts` (not `npm install`)?
+- Does the ShiftLeft stage include `npm audit --audit-level=critical`, `npm audit signatures`, and save `audit-report.json`?
+- Are `GH_TOKEN` / `GITHUB_RUNWAYCI_TOKEN` inside `withCredentials` blocks only — never in `environment {}` block?
+- Are AWS credentials supplied via IRSA (pod IAM role) — never static `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` in Jenkins env?
+
+### 1.7 Jenkinsfile — stage audit
+
+```bash
+grep -E "stage\(|mutation|stryker|quality.report|QualityReport|ShiftLeft|api.coverage" Jenkinsfile 2>/dev/null
+```
+
+Check for:
+- Mutation Tests stage: does it run `yarn mutation-tests:full` (full mode for CI)?
+- Does it publish the Stryker HTML report via `publishHTML`?
+- ShiftLeft stage: does it run `node-api-coverage-matrix.sh`?
+- Quality report: does it call `run-all.sh --skip-unit --skip-mutation --skip-postman --skip-coverage`?
+- Does it `publishHTML` the quality report?
+- Does it `archiveArtifacts postman/reports/**`?
+
+---
+
+## Phase 2: Report Gaps
+
+Present a precise audit:
+
+```
+Enhancement Audit Results
+=========================
+Component                Issue / Action Needed
+------------------------------------------------------------------
+Unit Tests               [OK / N failures need fixing before mutation]
+nyc Coverage             [OK / MISSING]
+Stryker Config           [OK / mutate scope too broad (includes controllers)
+                           / missing since-mode script / break:0 not set
+                           / external-service wrappers not excluded]
+run-all.sh               [OK / STALE — missing --skip-* flags / MISSING]
+run-mutation-tests.sh    [OK / MISSING — needed for two-mode approach]
+lib/cleanup-stryker.sh   [OK / MISSING]
+node-api-coverage-matrix.sh [OK / MISSING]
+stage-report-artifacts.sh [OK / MISSING]
+lib/report_generator.py  [OK / MISSING / STALE — entry point only, ~120 lines]
+lib/report_utils.py      [OK / MISSING — shared CSS + helpers]
+lib/report_consolidated.py [OK / MISSING]
+lib/report_migration.py  [OK / MISSING]
+lib/report_mutation.py   [OK / MISSING]
+lib/report_unit.py       [OK / MISSING]
+lib/report_combined.py   [OK / MISSING]
+lib/api_coverage_node.py [OK / MISSING / STALE]
+postman/.npmrc           [OK / MISSING / incomplete — audit-signatures missing etc.]
+postman/package-lock.json [OK / MISSING — must exist for npm ci]
+Jenkinsfile              [OK / full mutation mode not set / stryker report not published
+                           / missing api coverage stage / missing quality report
+                           / npm audit incomplete / GH_TOKEN in environment{} (wrong) / static AWS keys (wrong)]
+```
+
+---
+
+## Phase 3: Confirm
+
+Show the user exactly what will change:
+
+```
+I will make the following targeted changes:
+  1. Update Stryker mutate scope — remove controllers/, add missing helpers filtering
+     Will add: src/services/** (not currently included)
+     Will exclude: src/helpers/aws.js, src/helpers/freshid.js (wrapper files)
+  2. Add mutation-tests:full script to package.json for Jenkins CI
+  3. Set break:0 in stryker thresholds
+  4. Add missing run-mutation-tests.sh (two-mode runner)
+  5. Add missing lib/cleanup-stryker.sh
+  6. Update run-all.sh to current version
+  7. Add missing stage-report-artifacts.sh
+  8. Update Jenkinsfile: add yarn mutation-tests:full, add quality report generation
+
+I will NOT change:
+  - Existing unit tests
+  - Existing Postman collections
+  - Passing parts of stryker.config.js (reporters, concurrency, mochaOpts)
+
+Proceed? [Y/n]
+```
+
+Wait for confirmation before any changes.
+
+---
+
+## Phase 4: Execute — Targeted Changes Only
+
+### 4.1 Fix Stryker configuration (if needed)
+
+**Issue: mutate scope too broad** — Stryker will run for hours. Read `src/`, count files, then narrow:
+
+```bash
+# Count current scope (adjust to match stryker.config.js mutate globs)
+find src/actions src/serializers src/middlewares -name '*.js' 2>/dev/null | wc -l
+```
+
+If over 80 files, remove pass-through folders and exclude helper wrappers before anything else.
+
+Read actual src structure, then update:
+```bash
+find src -maxdepth 2 -type d | sort
+# For each helpers file, check if it only wraps external services:
+cat src/helpers/aws.js  # → exclude
+cat src/helpers/utils.js  # → include
+```
+
+Update only the `mutate` array — preserve everything else.
+
+**Issue: missing since-mode script** — add to package.json scripts:
+```json
+"mutation-tests": "bash postman/scripts/runners/run-mutation-tests.sh",
+"mutation-tests:full": "MUTATION_MODE=full bash postman/scripts/runners/run-mutation-tests.sh"
+```
+
+**Issue: break:0 not set**:
+```javascript
+thresholds: { high: 80, low: 60, break: 0 }
+```
+
+**Issue: coverageAnalysis not perTest**:
+```javascript
+coverageAnalysis: 'perTest'
+```
+
+**Issue: incremental not enabled**:
+```javascript
+incremental: true,
+incrementalFile: 'coverage/mutation/stryker-incremental.json',
+```
+
+### 4.2 Fix Node security guardrails (if needed)
+
+**Missing or incomplete `postman/.npmrc`** — create/update with all five settings:
+```ini
+ignore-scripts=true
+audit-level=high
+audit-signatures=true
+strict-ssl=true
+registry=https://registry.npmjs.org/
+```
+
+**Missing `postman/package-lock.json`** — if absent, run once locally:
+```bash
+cd postman && npm install
+```
+Then commit `package-lock.json`. After this, all CI runs must use `npm ci --ignore-scripts` — never `npm install`.
+
+**Jenkinsfile npm audit incomplete** — the ShiftLeft `npm ci` block must include all three audit lines:
+```groovy
+npm ci --ignore-scripts
+npm audit --audit-level=critical 2>&1 || echo "WARNING: audit found issues"
+npm audit signatures 2>&1 || echo "WARNING: signature verification failed"
+npm audit --json > ../audit-report.json 2>&1 || true
+```
+And archive the report:
+```groovy
+archiveArtifacts artifacts: 'audit-report.json', allowEmptyArchive: true
+```
+
+**`GH_TOKEN` in `environment {}` block** — move to `withCredentials` wrapping only the `sh` block that needs it:
+```groovy
+// WRONG — exposes token to all stages:
+environment { GH_TOKEN = credentials('github-token') }
+
+// CORRECT — scoped to specific sh block only:
+withCredentials([string(credentialsId: 'github-token', variable: 'GH_TOKEN')]) {
+    sh '...'
+}
+```
+
+**Static AWS keys in Jenkins env** — remove `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` from `environment {}`. Use IRSA (pod IAM role annotation on the pod spec). The pod supplies credentials automatically via the metadata service — no env vars needed.
+
+### 4.3 Refresh scripts from the shiftleft-tools package
+
+Run `shiftleft stage-scripts` — it pulls every library script from the installed
+package into `postman/scripts/` (gitignored, made executable, refreshed on every
+`shiftleft test` run), so missing and stale scripts are both handled in one step:
+
+- `runners/run-mutation-tests.sh`, `lib/cleanup-stryker.sh`
+- `report-generators/node-api-coverage-matrix.sh`, `report-generators/stage-report-artifacts.sh`
+- `lib/report_generator.py` and the split report modules
+  (`report_utils.py`, `report_consolidated.py`, `report_migration.py`,
+  `report_mutation.py`, `report_unit.py`, `report_combined.py`), `lib/api_coverage_node.py`
+
+The committed `run-all.sh` shim and repo-owned files (`setup-mocks.js`,
+collections, config) are never touched by staging. Don't copy or chmod these by hand.
+
+### 4.3 Update Jenkinsfile (targeted edits only)
+
+Only modify the specific missing pieces.
+
+**Mutation Tests stage not using full mode**:
+```groovy
+// BEFORE (wrong for CI):
+sh "yarn mutation-tests"
+
+// AFTER (correct for CI):
+utilObj.runCmd('yarn install --ignore-engines && yarn mutation-tests:full', env.NODE_VERSION)
+```
+
+**Stryker report not published** — add to Mutation Tests stage `post.always`:
+```groovy
+publishHTML([
+    allowMissing: true,
+    alwaysLinkToLastBuild: true,
+    keepAll: true,
+    reportDir: 'coverage/mutation',
+    reportFiles: 'index.html',
+    reportName: 'Stryker Mutation Report',
+    reportTitles: 'Mutation testing (full scan)'
+])
+```
+
+**Missing API coverage in ShiftLeft stage**:
+```groovy
+sh """
+    set -e
+    chmod +x postman/scripts/report-generators/node-api-coverage-matrix.sh
+    ./postman/scripts/report-generators/node-api-coverage-matrix.sh ./src/controllers ./postman
+"""
+publishHTML([
+    allowMissing: true,
+    alwaysLinkToLastBuild: true,
+    keepAll: true,
+    reportDir: 'postman/reports',
+    reportFiles: 'api-coverage-matrix-latest.html',
+    reportName: 'API Coverage Matrix',
+    reportTitles: 'Postman vs Express route coverage'
+])
+```
+
+**Missing quality report generation**:
+```groovy
+def generateQualityReport() {
+    echo "Generating Combined Quality Report..."
+    try {
+        utilObj.runCmd("""
+            set -euo pipefail
+            cd postman/scripts
+            ./stage-report-artifacts.sh "\${WORKSPACE}" "\${WORKSPACE}/postman/reports" || true
+            ./run-all.sh --skip-unit --skip-mutation --skip-postman --skip-coverage --no-delay
+        """, NODE_VERSION)
+    } catch (Exception e) {
+        echo "Quality report generation failed: ${e.getMessage()}"
+        currentBuild.result = 'UNSTABLE'
+    }
+
+    if (fileExists('postman/reports')) {
+        def qualityReport = sh(
+            returnStdout: true,
+            script: "ls -t postman/reports/quality-report-*.html 2>/dev/null | head -1 | xargs -r basename"
+        ).trim()
+        if (qualityReport) {
+            publishHTML([
+                allowMissing: true, alwaysLinkToLastBuild: true, keepAll: true,
+                reportDir: 'postman/reports', reportFiles: qualityReport,
+                includes: '**/*', reportName: 'QualityReport', reportTitles: 'Combined Quality Report'
+            ])
+        }
+        archiveArtifacts artifacts: 'postman/reports/**', allowEmptyArchive: true
+    }
+}
+```
+
+Call `generateQualityReport()` at the end of the ShiftLeft stage steps.
+
+---
+
+## Phase 5: Verify
+
+Run minimum verification for what was changed:
+
+If Stryker config was updated:
+```bash
+yarn mutation-tests       # since mode — fast check
+yarn mutation-tests:full  # full mode — complete check
+```
+
+If scripts were updated:
+```bash
+cd postman/scripts
+./run-all.sh --skip-postman --no-delay
+```
+
+If quality report was added (with existing test data):
+```bash
+cd postman/scripts
+./run-all.sh --skip-unit --skip-mutation --skip-postman --skip-coverage --no-delay
+```
+
+Report what changed and confirm quality-report-*.html was generated.
+
+---
+
+## Verification Checklist
+
+- [ ] Only targeted changes made (nothing extra overwritten)
+- [ ] Stryker mutate scope includes actual business logic folders
+- [ ] External-service wrappers excluded from mutate
+- [ ] `break: 0` is set in thresholds
+- [ ] `coverageAnalysis: 'perTest'` is set
+- [ ] Both `mutation-tests` and `mutation-tests:full` in package.json
+- [ ] `run-mutation-tests.sh` exists and is executable
+- [ ] `lib/cleanup-stryker.sh` exists and is executable
+- [ ] `node-api-coverage-matrix.sh` exists (pointing to correct routes dir)
+- [ ] `stage-report-artifacts.sh` exists
+- [ ] `run-all.sh` has full `--skip-*` flag set
+- [ ] Jenkinsfile runs `yarn mutation-tests:full` in CI (NOT `since` mode)
+- [ ] Jenkinsfile publishes Stryker HTML report
+- [ ] Jenkinsfile runs API coverage matrix and publishes it
+- [ ] Jenkinsfile generates and publishes quality report
+- [ ] `./run-all.sh --skip-postman --no-delay` completes successfully
+- [ ] `postman/.npmrc` exists with all five settings (ignore-scripts, audit-level, audit-signatures, strict-ssl, registry)
+- [ ] `postman/package-lock.json` exists
+- [ ] Jenkinsfile uses `npm ci --ignore-scripts` (never `npm install`)
+- [ ] Jenkinsfile ShiftLeft has: npm audit critical + npm audit signatures + audit-report.json saved
+- [ ] `audit-report.json` archived as build artifact
+- [ ] `GH_TOKEN` / `GITHUB_RUNWAYCI_TOKEN` only inside `withCredentials` blocks (not in `environment {}`)
+- [ ] No static `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` in Jenkins env (IRSA only)

package/templates/skills/enhance-test-pipeline/SKILL.md

@@ -0,0 +1,9 @@
+# Enhance Test Pipeline Skill
+
+## Detect Project Type First
+
+Before doing anything, detect the stack:
+
+1. `pom.xml` in project root → **Java Spring Boot** → read `SKILL-java.md` in this folder and follow it completely
+2. `package.json` (no `pom.xml`) → **Node.js** → read `SKILL-node.md` in this folder and follow it completely
+3. If unsure → ask the user: "Is this a Java or Node.js project?"

package/templates/skills/review-test-suite/SKILL-java.md

@@ -0,0 +1,137 @@
+Validate the integration test suite setup for the current repository by reading and analysing the actual content of relevant files. Do not judge by file names or folder locations — developers may name and organise things differently.
+
+## How to run this validation
+
+1. Find and read the CI pipeline file (Jenkinsfile, or equivalent).
+2. Find and read the build config file (pom.xml, build.gradle, package.json, etc.).
+3. Find the test/scripts folder — search broadly (postman/, integration-test/, e2e/, test/scripts/, etc.).
+4. Find GitHub Actions workflows under `.github/workflows/`.
+5. Read the actual content of collection files, scripts, and config to answer each check.
+
+---
+
+## Validations (20 checks across 6 categories)
+
+### A. Unit Testing & Coverage (3 checks)
+
+**A1. Unit tests run in CI**
+Look for: a CI stage that compiles and executes unit tests. Check whether test results (pass/fail counts, surefire reports) are captured.
+
+**A2. Line/branch coverage measured and published**
+Look for: a coverage tool configured in the build file (JaCoCo, Istanbul, coverage.py, etc.) with both a measurement execution and a report execution. Check that the HTML report is published to a CI tab or artifact — not just generated silently.
+
+**A3. Mutation testing runs in CI and report is published**
+Look for: a mutation testing tool configured (PIT/pitest, Stryker, mutmut, etc.) that runs as part of CI. Check:
+- It targets the right source packages (service, mapper, util layers) and excludes boilerplate (DTOs, entities, configs, generated mapper implementations, Application class)
+- The HTML and XML reports are published to a CI tab or artifact
+- It does NOT re-run in a phase where tests have already been skipped
+
+---
+
+### B. Integration / API Tests (4 checks)
+
+**B1. API test suite is split into small focused collections with a defined lifecycle**
+Look for: multiple smaller collections (not one large monolithic collection), each covering a specific concern (e.g., CRUD happy path, negative/validation tests, auth/401 tests, bulk operations, edge cases). Check that they follow a lifecycle:
+- A bootstrap/setup collection that creates prerequisite test data and always runs first
+- Focused mid-collections grouped by concern (not dumped into one file)
+- A cleanup/teardown collection that removes test data and always runs last
+Check that collections are run in a defined order (e.g., numbered filenames, explicit sequence in the runner script), so the suite is deterministic.
+
+**B2. Environment variable chaining between collections**
+Look for: test runner logic that exports variables created in one collection (e.g., `installation_id`, `account_id`) and passes them as input to subsequent collections. Without this, later collections cannot reference data created by earlier ones.
+
+**B3. Isolation environment with automated setup and teardown**
+Look for: a mechanism that routes test traffic to an isolated service instance (isoforge, docker-compose, testcontainers, feature flags, etc.) so tests do not mutate shared/staging data. Check:
+- Setup of the isolation stack is automated in CI before tests run
+- Teardown is automated after tests complete (even on failure)
+- The isolation routing identifier (header, flag, etc.) is passed to the test runner
+
+**B4. Integration test results published — per-collection and consolidated**
+Look for: each collection producing its own individual HTML report (so failures can be pinpointed to a specific collection), PLUS a consolidated summary report that aggregates pass/fail counts across all collections into a single view. Both should be published as CI artifacts or HTML tabs. A single merged report with no per-collection breakdown is insufficient.
+
+---
+
+### C. API Coverage & Assertion Quality (3 checks)
+
+**C1. API coverage matrix exists**
+Look for: a script or tool that statically compares controller endpoint definitions (Java @GetMapping/@PostMapping/etc., or OpenAPI spec) against what the test collections actually exercise. It should report untested endpoints.
+
+**C2. Coverage matrix checks assertion quality**
+Look for: the coverage matrix also detecting low-quality assertions such as:
+- Endpoints with tests but no 2xx success path tested
+- Weak status code assertions (e.g., `oneOf([200, 201])` instead of a specific code)
+- 5xx assertions in tests (usually a sign of testing the wrong thing)
+- Endpoints with no response body validation
+
+**C3. Skipped tests are documented**
+Look for: any environment-conditional test skips (`pm.execution.skipRequest()`, `pytest.skip()`, etc.) include a documented reason and the environment they are skipped in (e.g., `[SKIP] Reason: requires WireMock | Env: staging`). Undocumented skips should be flagged.
+
+---
+
+### D. Credential & Secret Security (3 checks)
+
+**D1. Secrets fetched from a secure store at runtime**
+Look for: the test runner fetching credentials (JWT signing keys, API keys, etc.) from a secure store (AWS Secrets Manager, HashiCorp Vault, environment variables injected by CI, etc.) rather than hardcoding them in scripts or committing them to the repo.
+
+**D2. Secrets cleared from the shell environment BEFORE the Node/npm layer runs**
+Look for: the test runner following this specific order: (1) fetch secret from secure store, (2) use secret in the shell layer to generate a token (e.g., JWT), (3) unset/clear the secret (`unset JWT_SECRET` or equivalent) BEFORE invoking npm, Newman, or any Node process. The Node layer should receive only the derived token — never the raw signing key or credential. This is critical because Node packages (including transitive dependencies) can read shell environment variables, making an uncleared secret accessible to potentially malicious npm packages.
+
+**D3. npm/package installs are protected against script execution**
+Look for: any mechanism that prevents npm package lifecycle scripts from running during install — either `--ignore-scripts` on the install command, or `ignore-scripts=true` in an `.npmrc` file. This prevents lifecycle scripts in npm packages (including transitive dependencies) from executing arbitrary code during install — a defense against supply chain attacks.
+
+---
+
+### E. Security & Static Analysis (3 checks)
+
+**E1. Dependency vulnerability scan in CI**
+Look for: a step that scans dependencies for known vulnerabilities (`npm audit --audit-level=critical`, OWASP dependency-check, Snyk, etc.). Check that it runs on the test/postman dependencies as well as the main application dependencies.
+
+**E2. Static analysis / SAST in CI**
+Look for: a static analysis step (SonarQube, CodeQL, SpotBugs, etc.) that runs on the main application source code.
+
+**E3. GitHub Actions repo authorization check**
+Look for: a GitHub Actions workflow that fires on every push and verifies the repository belongs to an authorised organisation. This prevents the pipeline from running if the repo is forked to an unauthorised account.
+
+---
+
+### F. Developer Experience & Feedback Loop (4 checks)
+
+**F1. Single command runs the full local suite**
+Look for: a script or Maven/Gradle profile that a developer can run locally with one command to execute unit tests + mutation tests + API tests + report generation. Check that it supports skip flags (e.g., `--skip-unit`, `--skip-postman`) so developers can re-run only the phases they need.
+
+**F2. Combined / unified quality report**
+Look for: a single HTML report that presents unit test results, code coverage, mutation score, API test results, and the API coverage matrix in one place — not just separate disconnected Jenkins tabs. This is the key artifact a developer or reviewer should look at after a build.
+
+**F3. PR / commit status updated with integration test result**
+Look for: the CI pipeline posting a GitHub commit status check (pass/fail) back to the PR specifically for the integration test run, separate from the overall build status. This gives immediate feedback on the PR without navigating into Jenkins.
+
+**F4. Test collection format validated on every PR**
+Look for: a GitHub Actions workflow that fires when test collection files are changed and validates their JSON format / structure (not just that the file is valid JSON, but that it follows the expected collection schema). This catches malformed collections before they break the CI run.
+
+---
+
+## Output format
+
+For each check (A1 through F4), output:
+
+**PASS** — fully implemented
+**PARTIAL** — something is there but incomplete (explain what exists and what is missing)
+**FAIL** — not present
+**N/A** — not applicable for this repo's tech stack (explain why)
+
+One-line note for every check. For every FAIL or PARTIAL, add a **Recommendation** explaining what behaviour to implement (not what to name files).
+
+At the end output:
+- Score: `X / Y applicable checks passed`
+- Top 5 gaps to fix, ranked by impact on test confidence and developer feedback speed
+
+---
+
+## After validation
+
+If significant gaps are found, run the setup-api-tests skill to scaffold the missing infrastructure:
+```
+/setup-api-tests
+```
+
+The setup skill uses `shiftleft init-postman` to generate all scripts, collection templates, and CI configuration — it is the recommended way to implement the patterns this validation checks for.

package/templates/skills/review-test-suite/SKILL-node.md

@@ -0,0 +1,78 @@
+# Review Test Suite — Node.js / Express
+
+Validate test pipeline maturity. Read actual file contents — do not judge by filenames alone.
+
+Reference: `freshapps_api_node`
+
+## How to run
+
+1. Read `Jenkinsfile`, root `package.json`, `stryker.config.js`
+2. Read `postman/config/`, `postman/scripts/run-all.sh`, collections
+3. Answer each check below with PRESENT / MISSING / PARTIAL and evidence
+
+---
+
+## Validations (20 checks)
+
+### A. Unit Testing & Coverage
+
+**A1. Unit tests in CI** — Mocha/Jest stage runs `yarn test` or `yarn unit-tests`
+
+**A2. nyc coverage published** — `coverage/unit-tests/` HTML report archived or published in Jenkins
+
+**A3. Stryker in CI** — `mutation-tests:full` in Jenkins; HTML report published; `mutate` targets actions/helpers/serializers/middlewares, excludes external wrappers
+
+### B. API / Integration Tests
+
+**B1. Split collections with lifecycle** — `collections/crud/` with bootstrap (01), domain (02-N), cleanup (99)
+
+**B2. Env var chaining** — Newman `--export-environment` passes IDs between collections
+
+**B3. Isolation** — ShiftLeft/isoforge or local Docker MySQL; automated setup/teardown
+
+**B4. Per-collection + consolidated reports** — individual HTML per collection + `consolidated-*.html`
+
+### C. API Coverage
+
+**C1. Coverage matrix** — `report-generators/node-api-coverage-matrix.sh` scans `src/controllers/`
+
+**C2. Assertion quality** — matrix flags missing 2xx paths, weak assertions, 5xx-as-success
+
+**C3. Documented skips** — `[SKIP] Reason: ... | Env: ...` format
+
+### D. Security
+
+**D1. Secrets from secure store** — AWS Secrets Manager for staging JWT
+
+**D2. Secrets unset before Newman** — `unset` after JWT generation, before `npm`/`newman`
+
+**D3. npm hardening** — `postman/.npmrc` with `ignore-scripts=true`; CI uses `npm ci --ignore-scripts`
+
+### E. Mutation Testing
+
+**E1. Two modes** — `mutation-tests` (since/git diff) for local; `mutation-tests:full` for CI
+
+**E2. perTest coverage** — `coverageAnalysis: 'perTest'` in stryker.config.js
+
+**E3. Thresholds** — `thresholds.break: 0` (report score, don't hard-fail CI on threshold alone)
+
+### F. Quality Report
+
+**F1. run-all.sh** — `--skip-*`, `--env`, `--no-delay` flags
+
+**F2. Combined report** — `report_generator.py combined` with unit + mutation + Postman + API coverage tabs
+
+**F3. stage-report-artifacts.sh** — copies coverage HTML for Jenkins-relative links
+
+---
+
+## Output format
+
+```
+Category | Check | Status | Evidence
+---------|-------|--------|----------
+A        | A1    | PRESENT| Jenkinsfile stage 'Unit Tests'
+...
+```
+
+Score: X/20. List top 3 gaps with recommended skill (`/setup-mutation-tests`, `/setup-api-tests`, etc.).

package/templates/skills/review-test-suite/SKILL.md

@@ -0,0 +1,9 @@
+# Review Test Suite Skill
+
+## Detect Project Type First
+
+Before doing anything, detect the stack:
+
+1. `pom.xml` in project root → **Java Spring Boot** → read `SKILL-java.md` in this folder and follow it completely
+2. `package.json` (no `pom.xml`) → **Node.js** → read `SKILL-node.md` in this folder and follow it completely
+3. If unsure → ask the user: "Is this a Java or Node.js project?"