@freshworks/shiftleft-tools 1.1.8

package/templates/skills/setup-api-tests/SKILL-node.md

@@ -0,0 +1,141 @@
+# Setup API Tests Skill — Node.js / Express
+
+Set up Postman/Newman integration test infrastructure for a Node.js/Express service.
+
+Reference implementation: `freshapps_api_node`
+
+## When to Use
+
+- "setup postman", "add newman", "setup integration tests"
+- Project has `package.json` but no `postman/` directory (or incomplete setup)
+
+If `postman/` is missing, run `shiftleft init-postman --stack node` first, then customize with this skill.
+
+---
+
+## Phase 1: Explore (REQUIRED)
+
+Never assume conventions. Read actual source files first.
+
+```bash
+# Route structure
+ls src/controllers/ src/routes/ 2>/dev/null
+head -50 src/controllers/index.js 2>/dev/null || head -50 src/routes/index.js
+
+# App entry (for mock wiring)
+ls src/index.js src/app.js 2>/dev/null
+
+# Existing tests
+cat package.json | grep -E '"test"|"unit-tests"|"start"'
+ls test/ postman/ 2>/dev/null
+```
+
+Record:
+- **ROUTES_DIR** — usually `src/controllers` (Express router files)
+- **APP_PORT** — from config or `local.env` (default 4013 in templates)
+- **AUTH** — JWT middleware location (`src/middlewares/auth.js` or similar)
+- **EXTERNAL_DEPS** — helpers that call HTTP (aws, freshid, chargebee, etc.) → need nock mocks
+
+---
+
+## Phase 2: Scaffold (if missing)
+
+```bash
+shiftleft init-postman --stack node --name <service-name> --with-staging
+cd postman && npm ci --ignore-scripts
+```
+
+Verify structure:
+```
+postman/
+├── config/local.json          # setup.database, setup.app, auth.jwt.users
+├── collections/crud/          # numbered collections
+├── scripts/runners/run-tests.sh
+├── scripts/setup-mocks.js
+├── mocks/                     # nock stubs per external service
+└── local.test.env
+```
+
+---
+
+## Phase 3: Configure local.json
+
+Edit `postman/config/local.json`:
+
+| Field | What to set |
+|-------|-------------|
+| `api_url` | `http://localhost:<port>` matching your app |
+| `setup.app.port` | Same port |
+| `setup.app.start_command` | `yarn start` or `node src/index.js` |
+| `setup.database.enabled` | `true` if service needs MySQL locally (Podman) |
+| `auth.jwt.users` | One entry per role your API tests need |
+
+---
+
+## Phase 4: Wire nock mocks
+
+1. List external HTTP dependencies:
+   ```bash
+   grep -r "https://" src/helpers/ src/clients/ 2>/dev/null | head -20
+   ```
+
+2. Create `postman/mocks/<service>.js` with nock stubs (see `mocks/external.js` template)
+
+3. Register in `postman/scripts/setup-mocks.js`
+
+4. Load mocks from app entry when `NODE_ENV=integration-tests`:
+   ```javascript
+   if (process.env.NODE_ENV === 'integration-tests') {
+     require('../postman/scripts/setup-mocks').setupMocks();
+   }
+   ```
+
+**Do NOT use WireMock for Node** — use in-process nock (reference: `freshapps_api_node`).
+
+---
+
+## Phase 5: Collections
+
+Follow lifecycle pattern from `freshapps_api_node`:
+
+| Order | Collection | Purpose |
+|-------|------------|---------|
+| `01-*` | Bootstrap | Health check, set shared env vars |
+| `02-NN` | Domain CRUD | One file per resource area |
+| `99-*` | Cleanup | Remove test data |
+
+Add collections to `postman/collections/crud/`. Newman runs all `*.json` in filename order via `run-newman.sh`.
+
+---
+
+## Phase 6: Root package.json scripts
+
+Add to project root `package.json`:
+
+```json
+"mutation-tests": "bash postman/scripts/runners/run-mutation-tests.sh",
+"mutation-tests:full": "MUTATION_MODE=full bash postman/scripts/runners/run-mutation-tests.sh",
+"test:all": "cd postman/scripts && ./run-all.sh"
+```
+
+---
+
+## Phase 7: Verify
+
+```bash
+cd postman/scripts && ./runners/run-tests.sh local
+./report-generators/node-api-coverage-matrix.sh ../../src/controllers ..
+./run-all.sh --skip-mutation
+```
+
+Fix failures before declaring setup complete.
+
+---
+
+## Staging
+
+1. Update `postman/config/staging.json` with real `api_url` and AWS secret name
+2. Run: `./runners/run-tests.sh staging`
+3. JWT: `scripts/auth/generate-jwt.sh` + `get-issuer-secret.sh` (secrets unset before Newman)
+
+See `../_shared/postman-standards.md` for assertion rules.

package/templates/skills/setup-api-tests/SKILL.md

@@ -0,0 +1,9 @@
+# Setup API Tests Skill
+
+## Detect Project Type First
+
+Before doing anything, detect the stack:
+
+1. `pom.xml` in project root → **Java Spring Boot** → read `SKILL-java.md` in this folder and follow it completely
+2. `package.json` (no `pom.xml`) → **Node.js** → read `SKILL-node.md` in this folder and follow it completely
+3. If unsure → ask the user: "Is this a Java or Node.js project?"

package/templates/skills/setup-mutation-tests/SKILL-java.md

@@ -0,0 +1,303 @@
+# Setup Mutation Tests — Java Spring Boot
+
+Add PIT mutation testing to a Java Spring Boot project using an inspect-first approach. Never copy-paste placeholders — always discover the actual package structure before writing any config.
+
+## When to Use
+
+Invoke when the user says:
+- "setup mutation tests", "add mutation testing", "add PIT", "setup pitest"
+- "measure test quality", "check unit test quality"
+- "my unit tests pass but I want to know if they're strong"
+
+---
+
+## What is Mutation Testing?
+
+PIT makes small automated changes (mutations) to your source code — flipping `>` to `>=`, removing return values, negating conditions — then runs your JUnit unit tests against each mutation.
+
+- **Mutant killed** — your tests caught the change (good)
+- **Mutant survived** — your tests didn't catch it (weak assertion)
+
+Target score: 60%+ mutation, 70%+ coverage. Mutation tests run on every Jenkins build but NOT on every `mvn install` (profile-gated).
+
+---
+
+## Phase 1: Inspect
+
+### 1.1 Check if PIT already exists
+
+```bash
+grep -r "pitest-maven" pom.xml
+```
+
+- Found → check the existing configuration, skip to Phase 4 (Verify)
+- Not found → continue
+
+### 1.2 Discover the actual package structure
+
+```bash
+find src/main/java -type d | sort
+```
+
+Map what you find to these categories:
+
+| Category | Typical folder names | Include in PIT? |
+|---|---|---|
+| Business logic | `service`, `services`, `usecase`, `domain` | YES |
+| Data mapping | `mapper`, `mappers`, `converter`, `converters`, `transformer` | YES |
+| Utilities | `util`, `utils`, `helper`, `helpers` | YES |
+| Validation | `validator`, `validators`, `validation` | YES |
+| Filters/interceptors | `filter`, `filters`, `interceptor` | YES |
+| Controllers | `controller`, `controllers`, `api` | NO — integration, not unit |
+| Config | `config`, `configuration` | NO — no branching logic |
+| DTOs | `dto`, `model`, `request`, `response` | NO — data classes |
+| Entities | `entity`, `entities`, `domain/entity` | NO — JPA data classes |
+| Generated | any `MapperImpl` files | NO — MapStruct generated |
+| Main class | `Application.java` | NO |
+
+### 1.3 Find the base package
+
+```bash
+find src/main/java -name "*.java" | head -5 | xargs grep -l "^package" | head -1 | xargs grep "^package"
+```
+
+Example result: `package com.freshworks.myservice.installation;` → base package is `com.freshworks.myservice.installation`
+
+### 1.4 Check for existing test failures
+
+```bash
+mvn test -q 2>&1 | grep -E "FAILED|ERROR|Tests run" | tail -20
+```
+
+Note any failing test classes — these must be excluded from PIT's test scan so PIT doesn't fail on pre-existing issues.
+
+### 1.5 Check if this is a multi-module Maven project
+
+```bash
+ls */pom.xml 2>/dev/null | head -5
+grep "<modules>" pom.xml
+```
+
+Multi-module: the PIT plugin goes in the ROOT pom.xml, and you run `mvn -pl <module-name> org.pitest:pitest-maven:mutationCoverage -Pmutation-tests`.
+
+---
+
+## Phase 2: Reason
+
+Based on what you found, determine:
+
+**targetClasses** — list each business logic package explicitly:
+- `BASE_PACKAGE.service.*` (if `service/` exists)
+- `BASE_PACKAGE.mapper.*` (if `mapper/` exists)
+- etc.
+
+**targetTests** — mirror the source packages:
+- `BASE_PACKAGE.service.*Test` for service tests
+- `BASE_PACKAGE.mapper.*Test` for mapper tests
+- etc.
+
+**excludedClasses** — always exclude:
+- `*DTO`, `*Dto`, `*Request`, `*Response` — data transfer objects
+- `*Entity` — JPA entities
+- `*Config`, `*Configuration` — Spring config
+- `*Exception` — exception classes
+- `*Application` — main class
+- `*MapperImpl` — MapStruct generated implementations
+- `*Constants` — constant-only classes
+
+**excludedTestClasses** — any test classes that are currently failing (from 1.4)
+
+**Profile approach** — NEVER put pitest in the main build. Wrap in `<profile id="mutation-tests">` with `<activeByDefault>false</activeByDefault>`. This means `mvn install` never runs mutation tests — only `mvn ... -Pmutation-tests`.
+
+---
+
+## Phase 3: Confirm
+
+Show the user a summary before writing anything:
+
+```
+I found the following packages in src/main/java/:
+  BASE_PACKAGE.service      (3 classes)
+  BASE_PACKAGE.mapper       (2 classes)
+  BASE_PACKAGE.util         (4 classes)
+
+I will configure PIT to:
+  Target classes: BASE_PACKAGE.service.*, BASE_PACKAGE.mapper.*, BASE_PACKAGE.util.*
+  Target tests:   BASE_PACKAGE.service.*Test, BASE_PACKAGE.mapper.*Test, BASE_PACKAGE.util.*Test
+  Exclude:        *DTO, *Entity, *Config, *MapperImpl, ...
+  Thresholds:     mutation 60%, coverage 70%
+  Profile:        mutation-tests (not run on mvn install)
+
+Proceed? [Y/n]
+```
+
+Wait for confirmation before writing to pom.xml.
+
+---
+
+## Phase 4: Execute
+
+Add this profile to pom.xml (inside `<project>`, after `</build>`):
+
+```xml
+<profiles>
+    <profile>
+        <id>mutation-tests</id>
+        <activation>
+            <activeByDefault>false</activeByDefault>
+        </activation>
+        <build>
+            <plugins>
+                <plugin>
+                    <groupId>org.pitest</groupId>
+                    <artifactId>pitest-maven</artifactId>
+                    <version>1.15.3</version>
+                    <dependencies>
+                        <dependency>
+                            <groupId>org.pitest</groupId>
+                            <artifactId>pitest-junit5-plugin</artifactId>
+                            <version>1.2.1</version>
+                        </dependency>
+                    </dependencies>
+                    <configuration>
+                        <targetClasses>
+                            <!-- REPLACE with actual packages found in Phase 1 -->
+                            <param>BASE_PACKAGE.service.*</param>
+                            <param>BASE_PACKAGE.mapper.*</param>
+                            <param>BASE_PACKAGE.util.*</param>
+                        </targetClasses>
+                        <targetTests>
+                            <!-- REPLACE with actual test packages found in Phase 1 -->
+                            <param>BASE_PACKAGE.service.*Test</param>
+                            <param>BASE_PACKAGE.mapper.*Test</param>
+                            <param>BASE_PACKAGE.util.*Test</param>
+                        </targetTests>
+                        <!-- Add any currently-failing test classes here to prevent PIT failure -->
+                        <!-- <excludedTestClasses><param>BASE_PACKAGE.SomeFailingTest</param></excludedTestClasses> -->
+                        <excludedClasses>
+                            <param>*DTO</param>
+                            <param>*Dto</param>
+                            <param>*Entity</param>
+                            <param>*Config</param>
+                            <param>*Configuration</param>
+                            <param>*Exception</param>
+                            <param>*Application</param>
+                            <param>*MapperImpl</param>
+                            <param>*Constants</param>
+                            <param>*Request</param>
+                            <param>*Response</param>
+                        </excludedClasses>
+                        <excludedMethods>
+                            <param>hashCode</param>
+                            <param>equals</param>
+                            <param>toString</param>
+                        </excludedMethods>
+                        <mutators>
+                            <mutator>STRONGER</mutator>
+                        </mutators>
+                        <outputFormats>
+                            <outputFormat>HTML</outputFormat>
+                            <outputFormat>XML</outputFormat>  <!-- XML required by quality report script -->
+                        </outputFormats>
+                        <timestampedReports>false</timestampedReports>  <!-- Required for CI report linking -->
+                        <threads>4</threads>
+                        <timeoutConstant>8000</timeoutConstant>
+                        <mutationThreshold>60</mutationThreshold>
+                        <coverageThreshold>70</coverageThreshold>
+                    </configuration>
+                </plugin>
+            </plugins>
+        </build>
+    </profile>
+</profiles>
+```
+
+**For multi-module projects**: add the profile to the ROOT `pom.xml`, not the module pom. Use `-pl <module-name>` when running.
+
+Also ensure JaCoCo is present for coverage reporting. Add to the main `<build><plugins>` if missing:
+
+```xml
+<plugin>
+    <groupId>org.jacoco</groupId>
+    <artifactId>jacoco-maven-plugin</artifactId>
+    <version>0.8.11</version>
+    <executions>
+        <execution>
+            <goals>
+                <goal>prepare-agent</goal>
+            </goals>
+        </execution>
+        <execution>
+            <id>report</id>
+            <phase>test</phase>
+            <goals>
+                <goal>report</goal>
+            </goals>
+        </execution>
+    </executions>
+</plugin>
+```
+
+Also ensure Surefire is configured to exclude integration tests:
+
+```xml
+<plugin>
+    <groupId>org.apache.maven.plugins</groupId>
+    <artifactId>maven-surefire-plugin</artifactId>
+    <configuration>
+        <excludes>
+            <exclude>**/*IT.java</exclude>
+            <exclude>**/*IntegrationTest.java</exclude>
+        </excludes>
+    </configuration>
+</plugin>
+```
+
+---
+
+## Phase 5: Verify
+
+Run mutation tests to confirm the setup works:
+
+```bash
+# Single-module project:
+mvn test-compile org.pitest:pitest-maven:mutationCoverage -Pmutation-tests
+
+# Multi-module project (replace 'installation' with your module name):
+mvn -pl installation org.pitest:pitest-maven:mutationCoverage -Pmutation-tests
+```
+
+Or use the script if postman/scripts/ already exists:
+
+```bash
+./postman/scripts/report-generators/mutation-report.sh
+```
+
+Report is generated at `target/pit-reports/index.html`.
+
+### Report results to the user
+
+After the run completes:
+1. Report the mutation score — e.g. "72% of mutants killed (target: 60%+)"
+2. Identify surviving mutants — list which classes have weak coverage
+3. Suggest fixes for the top surviving mutants:
+
+| Surviving Mutant Type | Likely Cause | Fix |
+|---|---|---|
+| Negated condition | Assertion doesn't test boundary | Add test for the false branch |
+| Removed method call | `verify()` missing or no assertion on side effect | Add `ArgumentCaptor` or assert on saved entity |
+| Changed return value | Return value not asserted | Assert on the actual return value |
+| Removed null check | No test for null input | Add test with null argument |
+
+---
+
+## Verification Checklist
+
+- [ ] PIT plugin added inside a `<profile id="mutation-tests">` (NOT in main build)
+- [ ] `targetClasses` matches actual project packages (not placeholder)
+- [ ] `targetTests` matches actual test class names
+- [ ] `<timestampedReports>false</timestampedReports>` is set (required for CI)
+- [ ] XML output format is enabled (required for quality report)
+- [ ] Mutation run completes without errors
+- [ ] Score reported to user
+- [ ] Surviving mutants identified and fixes suggested