@freshworks/shiftleft-tools 1.1.15 → 1.1.17

package/templates/postman-node/scripts/lib/api_coverage_node.py

@@ -385,7 +385,7 @@ def extract_tests_from_item(item, collection_name, depth=0):
 
                                 # Format 1: [SKIP] Reason: ... | Env: ...
                                 skip_match = re.search(
-                                    r"\[SKIP\].*?Reason:\s*([^|,;]+)(?:[|,;]\s*Env:\s*(\S+))?",
+                                    r"\[SKIP\].*?Reason:\s*([^|,;]+)(?:[|,;]\s*Env:\s*(\w+))?",
                                     prereq_text, re.IGNORECASE
                                 )
                                 if skip_match:

package/templates/skills/run-test-suite/SKILL-java.md

@@ -60,12 +60,14 @@ Runs all Postman collections against local environment:
 - Uses WireMock for external service mocks
 - No real JWT needed (hardcoded test token)
 
-Options:
-- `./run-tests-local.sh --v3` — run only V3 API tests
-- `./run-tests-local.sh --v2` — run only V2 API tests
+Options (generic baseline):
 - `./run-tests-local.sh --coverage` — run tests with JaCoCo coverage
 - `./run-tests-local.sh --skip-services` — skip starting services (if already running)
 
+> Repos that added versioned suites via `/setup-api-tests` (and protected the runner) may expose
+> extra flags such as `--v3` / `--v2`. Run `./run-tests-local.sh --help` to see what THIS repo's
+> runner supports.
+
 ### Postman Tests - Staging
 
 ```bash
@@ -79,9 +81,8 @@ Runs all Postman collections against staging environment:
 - Uses real external services
 - Requires AWS credentials for JWT generation (via IRSA in Jenkins, AWS_PROFILE locally)
 
-Options:
-- `./run-tests-staging.sh --v3` — run only V3 tests
-- `./run-tests-staging.sh --multi-product` — test all products
+The generic staging runner takes no suite flags; run `--help` to see what THIS repo's runner
+supports (repos may add flags after customizing + protecting the runner).
 
 ### Full Test Suite
 

package/templates/skills/setup-api-tests/SKILL-java.md

@@ -246,10 +246,17 @@ shiftleft init-postman
 
 This creates:
 - `postman/` folder with all scripts
-- `runners/run-tests-local.sh`, `run-all.sh`, `report-generators/mutation-report.sh`
+- `runners/run-tests-local.sh`, `runners/run-tests-staging.sh`, `run-all.sh`, `report-generators/mutation-report.sh`
 - `report-generators/java-api-coverage-matrix.sh`, `lib/api_coverage.py`
 - `start-mocks.sh`, `stop-mocks.sh`
 
+**IMPORTANT — the runners ship GENERIC.** `runners/run-tests-local.sh` and
+`runners/run-tests-staging.sh` contain a `# >>> repo-specific (set by /setup-api-tests) <<<`
+block with safe defaults. You MUST customize that block (2.3 / 2.5) **and** then run
+`shiftleft protect` (2.6). `shiftleft test` re-stages the library scripts and
+**force-overwrites the runners on every run** — without `protect`, your customizations are
+silently lost on the next test run.
+
 ---
 
 ## 2.2 Create application-postman.properties
@@ -310,38 +317,39 @@ MUST be included to prevent inheriting MySQL credentials from main config.
 
 ---
 
-## 2.3 Customize run-tests-local.sh Based on H2_SCOPE
-
-Edit `postman/scripts/runners/run-tests-local.sh`:
-
-### If H2_SCOPE = test (most common):
+## 2.3 Apply repo-specific runner customization (local)
 
-The app cannot be started with `java -jar`. Use one of:
+Edit the `# >>> repo-specific (set by /setup-api-tests) <<<` block at the top of
+`postman/scripts/runners/run-tests-local.sh` using the values recorded in Phase 1:
 
-**Option A: Use mvn spring-boot:run**
 ```bash
-mvn spring-boot:run -Dspring-boot.run.profiles=postman &
+APP_PORT="${APP_PORT:-8080}"                  # Phase 1.4
+CONTEXT_PATH="${CONTEXT_PATH:-}"              # e.g. "/api" — Phase 1.4
+SPRING_PROFILE="${SPRING_PROFILE:-postman}"   # matches application-postman.properties (2.2)
+MAVEN_MODULE="${MAVEN_MODULE:-}"              # submodule name, or empty for single-module
+JAVA_VERSION="${JAVA_VERSION:-}"              # e.g. "11" or "17"; empty = use current java
+COVERAGE_PACKAGE="${COVERAGE_PACKAGE:-com.freshworks.*}"  # YOUR base package, e.g. com.freshworks.myservice.*
+COVERAGE_MODULE_DIR="${COVERAGE_MODULE_DIR:-$PROJECT_ROOT}"  # module dir if MAVEN_MODULE set
 ```
 
-**Option B: Use java -cp with test classpath**
-```bash
-# Build classpath including test dependencies
-TEST_CLASSPATH=$(mvn dependency:build-classpath -DincludeScope=test -Dmdep.outputFile=/dev/stdout -q)
-CLASSES_DIR="$PROJECT_ROOT/target/classes:$PROJECT_ROOT/target/test-classes"
-
-java -cp "$CLASSES_DIR:$TEST_CLASSPATH" \
-    com.example.YourApplication \
-    --spring.profiles.active=postman \
-    --server.port=$APP_PORT &
-```
+Also set `MUTATION_PACKAGE_PREFIX` (in the environment or run-all) to your base package if you
+want shortened names in the mutation report (optional).
 
-### If H2_SCOPE = runtime:
+### App start command (based on H2_SCOPE)
 
-Standard approach works:
-```bash
-mvn clean package -DskipTests -q
-java -jar target/*.jar --spring.profiles.active=postman &
-```
+The runner's default app-start block uses `mvn $module_args spring-boot:run`. Adjust the
+`# >>> repo-specific … app start command <<<` block in `start_spring_boot()`:
+
+- **H2_SCOPE = runtime** (default block works): `mvn [-pl $MAVEN_MODULE] spring-boot:run` is fine.
+- **H2_SCOPE = test** (app needs test classpath): replace the `nohup mvn … spring-boot:run` line
+  with a `java -cp` launch:
+  ```bash
+  TEST_CLASSPATH=$(mvn dependency:build-classpath -DincludeScope=test -Dmdep.outputFile=/dev/stdout -q)
+  CLASSES_DIR="$PROJECT_ROOT/target/classes:$PROJECT_ROOT/target/test-classes"
+  nohup java -cp "$CLASSES_DIR:$TEST_CLASSPATH" com.example.YourApplication \
+      --spring.profiles.active="$SPRING_PROFILE" --server.port="$APP_PORT" \
+      > "$APP_LOG_FILE" 2>&1 &
+  ```
 
 ---
 
@@ -383,6 +391,42 @@ done
 
 ---
 
+## 2.5 Customize the staging runner
+
+Only if the service has a staging environment. Edit `postman/config/staging.json` (created by
+init) with the real `aws.*`, `base_url`, and `jwt.*` values, then fill the two
+`# >>> repo-specific … <<<` blocks in `postman/scripts/runners/run-tests-staging.sh`:
+
+- **Extra JWT payload fields** — export any additional `.jwt.*` fields your `generate-jwt.sh`
+  needs beyond the generic defaults (e.g. `ORGANISATION_ID`, `FRESHID_ACCOUNT_ID`).
+- **Extra newman env vars** — append app-specific `--env-var` pairs to `NEWMAN_ENV_VARS`
+  (e.g. `extension_id`, `version_id` read from `.test_data` in the config).
+
+The generic baseline already injects `auth_token`, `base_url`, `product_id`, `account_id`,
+`user_role`.
+
+---
+
+## 2.6 MANDATORY: Protect the customized runners
+
+`shiftleft test` force-overwrites runners on every run. Freeze your customizations:
+
+```bash
+shiftleft protect runners/run-tests-local.sh runners/run-tests-staging.sh
+```
+
+This records them under `protected[]` in `.shiftleft.json` so staging skips them. Verify with
+`shiftleft protect --list`. Skip this and every edit from 2.3–2.5 is lost on the next
+`shiftleft test`.
+
+> **Versioned suites / multi-product (optional add-on):** the generic runner runs collections
+> from `postman/collections/` (flat `*.json`, or one suite per subdirectory). If the service
+> needs versioned suites (e.g. v3/v2/legacy) or multi-product matrices, extend `run_tests()` /
+> `NEWMAN_ENV_VARS` in the protected runner — the `mp-installation` app is the reference
+> implementation for that pattern.
+
+---
+
 # Phase 3: Create Postman Collections
 
 ## 3.1 MANDATORY: Read Before Writing