@freshworks/shiftleft-tools 1.1.8

package/templates/skills/setup-test-pipeline/SKILL-node.md

@@ -0,0 +1,318 @@
+# Setup Test Pipeline — Node.js / Express
+
+Set up a complete test pipeline from scratch (or near-scratch) for a Node.js/Express repo. This skill orchestrates the full pipeline: unit tests, nyc/Istanbul coverage, Stryker mutation tests, Postman API tests, API coverage matrix, combined quality report, and Jenkins integration.
+
+Use this skill when a repo has NOTHING (or only bare unit tests). For repos that already have partial pipeline setup, use `/enhance-test-pipeline` instead.
+
+## When to Use
+
+Invoke when:
+- "set up the full test pipeline"
+- "add complete test infrastructure to this Node.js repo"
+- "set up unit tests, mutation tests, and postman tests"
+- "I want everything: unit tests, coverage, mutation, API tests, quality report"
+
+---
+
+## Phase 1: Inspect — Audit What Exists
+
+Run each check and record the result (present/missing/partial).
+
+### 1.1 Unit tests
+
+```bash
+cat package.json | grep -E '"test"|"mocha"|"jest"'
+ls test/ test/unit/ 2>/dev/null | head -20
+yarn test 2>&1 | tail -5
+```
+
+- Present and passing: unit tests exist and pass
+- Present but failing: tests exist but have failures (must fix before mutation tests)
+- Missing: no test directory or test script
+
+### 1.2 nyc / Istanbul coverage
+
+```bash
+cat package.json | grep -E '"nyc"|"istanbul"|"c8"'
+grep -r "nyc\|istanbul\|c8" package.json
+```
+
+- Present: coverage already configured
+- Missing: needs to be added
+
+### 1.3 Postman infrastructure
+
+```bash
+ls postman/ 2>/dev/null
+ls postman/collections/ 2>/dev/null
+ls postman/scripts/ 2>/dev/null
+ls postman/scripts/run-all.sh 2>/dev/null
+```
+
+- Present: full postman/ directory with collections and scripts
+- Partial: directory exists but scripts or collections missing
+- Missing: no postman/ directory
+
+### 1.4 Stryker mutation testing
+
+```bash
+cat package.json | grep stryker
+ls stryker.config.js 2>/dev/null
+```
+
+- Present: Stryker configured (check mutate scope — should target business logic only)
+- Missing: needs to be added
+
+### 1.5 API coverage matrix
+
+```bash
+ls postman/scripts/report-generators/node-api-coverage-matrix.sh 2>/dev/null
+```
+
+### 1.6 Quality report
+
+```bash
+ls postman/scripts/run-all.sh 2>/dev/null
+grep "skip-report\|skip-unit\|skip-mutation" postman/scripts/run-all.sh 2>/dev/null | head -5
+```
+
+- Present and modern: run-all.sh exists and has --skip-* flags
+- Present but stale: run-all.sh exists but is an old version without flags
+- Missing
+
+### 1.7 Jenkinsfile
+
+```bash
+ls Jenkinsfile
+grep -E "Mutation|ShiftLeft|Quality|quality-report|stryker" Jenkinsfile 2>/dev/null
+```
+
+- Complete: has Unit Tests, Mutation Tests, ShiftLeft stages with quality report generation
+- Partial: has some stages but missing mutation or quality report
+- Missing: no Jenkinsfile
+
+---
+
+## Phase 2: Report Gaps
+
+Present a clear status table to the user:
+
+```
+Pipeline Audit Results
+======================
+Component               Status
+-----------------------------------------------
+Unit Tests              [PRESENT / MISSING / FAILING]
+nyc Coverage            [PRESENT / MISSING]
+Postman Infrastructure  [PRESENT / PARTIAL / MISSING]
+Stryker Mutation Tests  [PRESENT / MISSING]
+API Coverage Matrix     [PRESENT / MISSING]
+run-all.sh (modern)     [PRESENT / STALE / MISSING]
+Jenkinsfile             [COMPLETE / PARTIAL / MISSING]
+```
+
+---
+
+## Phase 3: Confirm
+
+Tell the user exactly what will be set up:
+
+```
+I will set up the following components in order:
+  1. [if missing] Postman infrastructure (collections, scripts, config)
+  2. [if missing] nyc coverage in package.json
+  3. [if missing] Stryker mutation testing (discover actual src structure first)
+  4. Stage library scripts from the shiftleft-tools package (`shiftleft stage-scripts`):
+     run-all.sh, runners/run-mutation-tests.sh,
+     report-generators/node-api-coverage-matrix.sh, generate-consolidated-report.sh,
+     report-generators/stage-report-artifacts.sh,
+     lib/report_generator.py (entry point), lib/report_utils.py, lib/report_consolidated.py,
+     lib/report_migration.py, lib/report_mutation.py, lib/report_unit.py, lib/report_combined.py,
+     lib/api_coverage_node.py
+  5. [if partial] Update Jenkinsfile with mutation + ShiftLeft + quality report stages
+
+This will take approximately 5-10 minutes.
+Proceed? [Y/n]
+```
+
+Wait for confirmation before making any changes.
+
+---
+
+## Phase 4: Execute in Order
+
+### Step A: Postman infrastructure (if missing)
+
+If no `postman/` directory exists, invoke `/setup-api-tests` skill for the full Postman setup.
+
+### Step B: nyc coverage (if missing)
+
+Add nyc configuration to `package.json`:
+
+```json
+{
+  "scripts": {
+    "test:coverage": "nyc mocha test/unit/**/*.test.js"
+  },
+  "nyc": {
+    "reporter": ["lcov", "text", "html"],
+    "report-dir": "coverage/unit",
+    "exclude": [
+      "test/**",
+      "postman/**",
+      "coverage/**",
+      "node_modules/**"
+    ],
+    "include": [
+      "src/**/*.js"
+    ]
+  }
+}
+```
+
+Install if missing:
+```bash
+yarn add --dev nyc
+```
+
+### Step C: Stryker mutation testing (if missing)
+
+Follow the `setup-mutation-tests` SKILL-node.md skill in full:
+1. Discover source folders — **only** actions, services, serializers, middlewares (+ selective helpers)
+2. **Exclude** controllers, routes, models, config, migrations
+3. Inspect helpers file-by-file — exclude external SDK wrappers
+4. **Count files** — mutate scope must stay **under 80 files** (never `src/**`)
+5. Confirm narrow scope with user before writing config
+6. Install `@stryker-mutator/core` and appropriate runner
+7. Create `stryker.config.js` with the audited scope only
+8. Add `mutation-tests` (since mode) and `mutation-tests:full` scripts to package.json
+
+Key requirements:
+- **Never** `mutate: ['src/**/*.js']` — Stryker will run for hours
+- `coverageAnalysis: 'perTest'` — fastest
+- `incremental: true` — reuse previous results
+- `concurrency: 4`
+- `thresholds: { high: 80, low: 60, break: 0 }` — break:0 means CI never hard-fails on score
+- `htmlReporter: { fileName: 'coverage/mutation/index.html' }`
+- `run-mutation-tests.sh` for the two-mode approach (since vs full)
+
+### Step D: Stage scripts from the shiftleft-tools package
+
+Run `shiftleft stage-scripts` to pull the library scripts from the installed
+package into `postman/scripts/` — do not copy or write them by hand. They are
+gitignored and refreshed on every `shiftleft test` run. The staged set includes:
+
+**Core pipeline scripts (required):**
+- `run-all.sh` — master pipeline runner with `--skip-*` and `--env` flags
+- `runners/run-mutation-tests.sh` — Stryker two-mode runner (since vs full)
+- `report-generators/node-api-coverage-matrix.sh` — Express route coverage matrix
+- `report-generators/generate-consolidated-report.sh` — aggregates Newman JSON results
+- `report-generators/stage-report-artifacts.sh` — copies nyc/Stryker HTML for Jenkins linking
+
+**Support scripts:**
+- `run-tests.sh` (or `run-all.sh`) — runs Postman tests locally and against staging
+- `get-issuer-secret.sh` / `auth/get-issuer-secret.sh` — fetches JWT secret from AWS
+
+**Lib folder (required):**
+- `lib/report_generator.py` — CLI entry point, dispatches to report modules
+- `lib/report_utils.py` — shared CSS, helpers, UI component builders
+- `lib/report_consolidated.py` — consolidated Newman test results report
+- `lib/report_migration.py` — migration comparison report
+- `lib/report_mutation.py` — PIT/Stryker mutation testing report
+- `lib/report_unit.py` — unit test parsers and report
+- `lib/report_combined.py` — combined quality report (unit + API tabs)
+- `lib/api_coverage_node.py` — generates API coverage matrix from Express routes + Postman
+- `lib/cleanup-stryker.sh` — kills Stryker processes and cleans temp files
+
+Make all `.sh` files executable:
+```bash
+chmod +x postman/scripts/*.sh postman/scripts/lib/*.sh
+```
+
+### Step E: Update Jenkinsfile (if missing stages)
+
+If the Jenkinsfile is missing mutation or quality report stages, update it to match the `templates/jenkins/Jenkinsfile-node.groovy` pattern:
+
+Required stages:
+1. `Unit Tests` — `yarn test`
+2. `Mutation Tests` — `yarn mutation-tests:full` (full mode for CI), publish Stryker HTML report
+3. `Staging Postman, API Coverage & Quality Report` — ShiftLeft isolation, Postman, API coverage matrix, quality report
+
+Key quality report pattern:
+```groovy
+// API coverage (after ShiftLeft Postman):
+sh "./postman/scripts/report-generators/node-api-coverage-matrix.sh ./src/controllers ./postman"
+publishHTML([reportDir: 'postman/reports', reportFiles: 'api-coverage-matrix-latest.html', reportName: 'API Coverage Matrix'])
+
+// Quality report:
+sh "./postman/scripts/run-all.sh --skip-unit --skip-mutation --skip-postman --skip-coverage --no-delay"
+publishHTML([reportDir: 'postman/reports', reportFiles: 'quality-report-*.html', reportName: 'QualityReport'])
+archiveArtifacts artifacts: 'postman/reports/**', allowEmptyArchive: true
+```
+
+Replace placeholders:
+- `SERVICE_NAME` → your actual service name (e.g. `freshapps_api_node`)
+- `NAMESPACE` → your Kubernetes namespace
+- `ISO_HEADER` → isoforge routing header
+- `ISO_HEADER_VALUE` → the account ID or regex value
+
+---
+
+## Phase 5: Verify
+
+### Verify Phase A: Unit + Mutation + Coverage (no Postman needed)
+
+```bash
+yarn test               # Unit tests
+yarn mutation-tests      # Stryker since-mode (only changed files)
+```
+
+### Verify Phase B: Postman + API Coverage + Quality Report
+
+```bash
+cd postman/scripts
+./run-all.sh --skip-mutation --no-delay
+```
+
+Expected: Postman tests run locally, API coverage matrix generates, quality report generates.
+
+### Verify Phase C: Full pipeline
+
+```bash
+cd postman/scripts
+./run-all.sh --no-delay
+```
+
+### Report to user
+
+```
+Pipeline setup complete!
+
+Verification results:
+  Unit Tests:     [PASS/FAIL] — N tests, M% line coverage
+  Mutation Tests: [PASS/FAIL] — X% mutation score (since mode)
+  Postman Tests:  [PASS/FAIL] — N collections, M assertions
+  API Coverage:   [PASS/FAIL] — X% endpoints covered
+  Quality Report: [GENERATED] — postman/reports/quality-report-*.html
+
+Next steps:
+  - Local run-all: cd postman/scripts && ./run-all.sh
+  - Fast mutation check: yarn mutation-tests (only changed files)
+  - Full mutation check: yarn mutation-tests:full (all scoped files)
+  - Staging run: cd postman/scripts && ./run-all.sh --env staging
+  - Jenkins: push a PR and comment "shiftleft" to trigger integration tests
+```
+
+---
+
+## Verification Checklist
+
+- [ ] `yarn test` passes
+- [ ] `yarn test:coverage` generates coverage report
+- [ ] `yarn mutation-tests` works in since mode
+- [ ] `yarn mutation-tests:full` runs full Stryker scan
+- [ ] `./run-all.sh --skip-mutation --no-delay` completes successfully
+- [ ] `postman/reports/quality-report-*.html` is generated
+- [ ] All `.sh` files are executable
+- [ ] Jenkinsfile has Mutation Tests stage (full mode) and Quality Report generation
+- [ ] `node-api-coverage-matrix.sh` is configured with correct routes dir

package/templates/skills/setup-test-pipeline/SKILL.md

@@ -0,0 +1,9 @@
+# Setup Test Pipeline Skill
+
+## Detect Project Type First
+
+Before doing anything, detect the stack:
+
+1. `pom.xml` in project root → **Java Spring Boot** → read `SKILL-java.md` in this folder and follow it completely
+2. `package.json` (no `pom.xml`) → **Node.js** → read `SKILL-node.md` in this folder and follow it completely
+3. If unsure → ask the user: "Is this a Java or Node.js project?"

package/templates/skills/write-api-tests/SKILL-java.md

@@ -0,0 +1,115 @@
+# Write API Tests Skill — Java Spring Boot
+
+Write and manage Postman/Newman integration tests for a Java Spring Boot API.
+
+## When to Use
+
+Invoke this skill when the user mentions:
+- "write api tests", "add api test", "add postman test"
+- "add test for endpoint", "write tests for [endpoint]", "test coverage"
+- "API test", "integration test", "newman"
+
+**Note:** If the project doesn't have a `postman/` folder yet, use the **setup-api-tests** skill first to create the infrastructure.
+
+## Arguments
+
+Arguments are passed after the skill invocation. Parse them before deciding what to write.
+
+| Argument form | Behaviour |
+|---|---|
+| `/write-api-tests GET /apps` | Write tests for that specific endpoint only |
+| `/write-api-tests POST /apps PUT /apps/{id}` | Write tests for each listed endpoint |
+| `/write-api-tests` (no argument) | Ask the user which endpoint(s) to target, OR run `java-api-coverage-matrix.sh` to find untested endpoints and offer to write tests for those |
+
+### Argument Parsing Rules
+
+1. Check if the user passed an HTTP method + path after the skill name (e.g. `GET /apps`, `POST /apps/{id}`)
+2. **If a specific endpoint is given** — scope all work to that endpoint only. Do not touch other collections or endpoints.
+3. **If multiple endpoints are given** — handle each one in sequence, same collection if they belong together, separate if not.
+4. **If no argument** — do one of the following (ask the user which they prefer):
+   - **Option A**: Specify an endpoint now (e.g. `GET /apps`)
+   - **Option B**: Run `./postman/scripts/report-generators/java-api-coverage-matrix.sh` to find untested endpoints, then let the user pick
+
+### When Writing for a Specific Endpoint
+
+Before writing any test:
+1. Read the controller file to confirm the exact path, method, request params, and response DTO
+2. Read the response DTO to get correct field names
+3. Check the existing collections to find the right file to add tests to
+4. Confirm the JSON naming strategy (snake_case vs camelCase) from `application.properties`
+5. Write tests, then run `./runners/run-tests-local.sh` and `./java-api-coverage-matrix.sh` to verify
+
+## Dual Environment Support
+
+### Local Environment
+- **Database**: H2 in-memory (no MySQL needed)
+- **External Services**: WireMock mocks
+- **Authentication**: Hardcoded test JWT token
+- **Command**: `cd postman/scripts && ./runners/run-tests-local.sh`
+
+### Staging Environment
+- **Database**: Real MySQL
+- **External Services**: Real services (FreshID, other Freshworks services)
+- **Authentication**: JWT from AWS Secrets Manager
+- **Command**: `cd postman/scripts && ./runners/run-tests-staging.sh`
+
+---
+
+> **Assertion standards, skip patterns, query/include coverage, and the submission checklist are in `../_shared/postman-standards.md`. Read that file for all Postman assertion rules before writing any tests.**
+
+---
+
+## Running Tests
+
+```bash
+# Local environment
+cd postman/scripts && ./runners/run-tests-local.sh
+
+# Staging environment
+cd postman/scripts && ./runners/run-tests-staging.sh
+
+# API coverage report (must show 100% 2xx coverage)
+./postman/scripts/report-generators/java-api-coverage-matrix.sh
+```
+
+## Adding a New Test
+
+1. **Choose the right collection** based on endpoint type
+2. **Add request** with descriptive name: `"Verb - Description"`
+3. **Add status code assertion** (exactly one per request)
+4. **Add value assertions** for response fields
+5. **Add skip logic** if environment-specific
+6. **Run locally**: `cd postman/scripts && ./runners/run-tests-local.sh`
+7. **Check coverage**: `./postman/scripts/report-generators/java-api-coverage-matrix.sh`
+
+## Adding a New Collection
+
+1. Create `postman/collections/XX-feature-name.json` (follow existing collection structure)
+2. Add collection to the arrays in `run-tests-local.sh` and `run-tests-staging.sh`
+3. Format: `cd postman && npm run format`
+4. Test: `cd postman/scripts && ./runners/run-tests-local.sh`
+
+## Setting Up Postman in a New Project
+
+If the project doesn't have Postman tests yet, use the **setup-api-tests** skill which provides:
+- Complete folder structure
+- package.json with secure dependencies
+- Environment files (local/staging)
+- Config files
+- Script templates
+- Maven integration
+- Spring integration profile
+
+## Coverage Report Validation
+
+The `java-api-coverage-matrix.sh` script validates all tests. It will flag:
+
+| Issue | What It Detects | Fix |
+|-------|-----------------|-----|
+| **No 2xx test** | Endpoint has no success path test | Add 200/201 test |
+| **Weak assertions (oneOf)** | `oneOf([200, 201])` patterns | Split into separate tests |
+| **5xx assertions** | Tests asserting 500 status | Remove - bugs aren't expected behavior |
+| **Missing query params** | `@RequestParam` not tested | Add tests for each param |
+| **Missing include values** | Expected include values not tested | Add tests for each value |
+| **Undocumented skips** | `skipRequest()` without `[SKIP] Reason:` | Add skip reason |
+| **Low body assertions** | No response field validation | Add `pm.expect(json.field)` |

package/templates/skills/write-api-tests/SKILL-node.md

@@ -0,0 +1,83 @@
+# Write API Tests Skill — Node.js / Express
+
+Write Postman/Newman tests for Express API endpoints.
+
+Reference: `freshapps_api_node` (`postman/collections/crud/`)
+
+## Arguments
+
+| Form | Behaviour |
+|------|-----------|
+| `/write-api-tests GET /api/v2/apps` | Tests for that endpoint only |
+| `/write-api-tests` (no args) | Run coverage matrix, offer untested endpoints |
+
+---
+
+## Before Writing
+
+1. **Read the controller** — find exact path, method, middleware:
+   ```bash
+   grep -r "router\.\(get\|post\|patch\|put\|delete\)" src/controllers/ -l
+   cat src/controllers/v2/<resource>.js
+   ```
+
+2. **Read the serializer/response shape** — Node APIs often use serializers, not DTOs:
+   ```bash
+   ls src/serializers/
+   ```
+
+3. **Check JSON naming** — default Express is camelCase unless a serializer transforms keys
+
+4. **Check auth** — which `authorize('resource.action')` permission is required?
+
+5. **Find the right collection** — match domain to `postman/collections/crud/NN-*.json`
+
+---
+
+## Express route patterns
+
+```javascript
+appsRouter.get('/', [authorize('apps.read'), ...], handler);
+appsRouter.get(`/:app_id(${numberRegex})`, [...], handler);
+```
+
+Full path = mount prefix from `controllers/index.js` + route path.
+Example: `/api/v2` + `/apps` → `GET /api/v2/apps`
+
+---
+
+## Writing tests
+
+Follow `../_shared/postman-standards.md`:
+- One status code assertion per request
+- Specific value assertions (not just `pm.expect(json).to.be.an('object')`)
+- Document skips: `[SKIP] Reason: ... | Env: staging`
+
+### Collection item naming
+`"Verb - Description"` e.g. `"GET - List apps returns 200"`
+
+### Environment variables
+Use `{{base_url}}`, `{{auth_token}}` from Postman environment files.
+Multi-role: `auth_token_developer`, etc. (set by JWT scripts from `config/local.json` users map)
+
+---
+
+## Run and verify
+
+```bash
+cd postman/scripts
+./runners/run-tests.sh local
+./report-generators/node-api-coverage-matrix.sh ../../src/controllers ..
+```
+
+Target: **100% 2xx coverage** on all controller endpoints.
+
+---
+
+## Adding a new collection
+
+1. Create `postman/collections/crud/NN-feature-name.json` (numbered for order)
+2. Run `cd postman && npm run format`
+3. Verify with `./runners/run-tests.sh local`
+
+Bootstrap (`01-*`) must run first; cleanup (`99-*`) must run last.

package/templates/skills/write-api-tests/SKILL.md

@@ -0,0 +1,9 @@
+# Write API Tests Skill
+
+## Detect Project Type First
+
+Before doing anything, detect the stack:
+
+1. `pom.xml` in project root → **Java Spring Boot** → read `SKILL-java.md` in this folder and follow it completely
+2. `package.json` (no `pom.xml`) → **Node.js** → read `SKILL-node.md` in this folder and follow it completely
+3. If unsure → ask the user: "Is this a Java or Node.js project?"

package/templates/stryker.config.js

@@ -0,0 +1,50 @@
+'use strict';
+
+/** @type {import('@stryker-mutator/api/core').PartialStrykerOptions} */
+module.exports = {
+  packageManager: 'yarn',
+  testRunner: 'mocha',
+  reporters: ['html', 'json', 'clear-text', 'progress'],
+  htmlReporter: {
+    fileName: 'coverage/mutation/index.html'
+  },
+  jsonReporter: {
+    fileName: 'coverage/mutation/mutation-report.json'
+  },
+  mutate: [
+    'src/actions/**/*.js',
+    'src/helpers/**/*.js',
+    'src/serializers/**/*.js',
+    'src/middlewares/**/*.js',
+    '!src/helpers/validator/schema/**/*.js',
+    '!src/helpers/aws.js',
+    '!src/helpers/chargebee.js',
+    '!src/helpers/freshid.js',
+    '!src/helpers/launchdarkly.js',
+    '!src/helpers/emailNotifications.js',
+    '!src/helpers/analytics.js',
+    '!src/helpers/observability.js',
+    '!src/helpers/mcp-gateway.js',
+    '!src/helpers/mcp-gateway-util.js',
+    '!src/helpers/mp-listings.js',
+    '!src/helpers/mp-apps.js',
+    '!src/helpers/constants.js',
+    '!src/helpers/config.js'
+  ],
+  coverageAnalysis: 'perTest',
+  concurrency: 4,
+  incremental: true,
+  incrementalFile: 'coverage/mutation/stryker-incremental.json',
+  ignorePatterns: ['postman/**', 'coverage/**'],
+  mochaOpts: {
+    files: 'test/unit/**/*.test.js',
+    require: ['test/unit/setup.js'],
+    timeout: 10000
+  },
+  thresholds: {
+    high: 80,
+    low: 60,
+    break: 0
+  },
+  tempDirName: 'coverage/mutation/.stryker-tmp'
+};