@postman/postman-plugin 0.2.2 → 0.4.0

package/skills/performance-testing/references/jmeter-import/mapping-table.md

@@ -0,0 +1,97 @@
+# JMeter → Postman mapping table
+
+Fidelity: **exact** (behaviour preserved) · **approx** (closest supported equivalent, reported) · **manual** (needs the model/user) · **unsupported** (dropped, reported).
+Owner: **script** (deterministic Python) · **model** (judgment call).
+
+## Structure
+
+| JMeter element | XML tag | Postman | Fidelity | Owner |
+|---|---|---|---|---|
+| Test Plan | `TestPlan` | Collection (`info` + `item`) | exact | script |
+| Thread Group | `ThreadGroup` | load config (VUs/duration/profile) + folder when >1 | approx | script |
+| setUp/tearDown Thread Group | `SetupThreadGroup`/`PostThreadGroup` | folder; suggest `--setup-collection`/`--teardown-collection` | approx | script+model |
+| Concurrency Thread Group | `...ConcurrencyThreadGroup` | VUs=`TargetLevel`, duration=`RampUp`+`Hold`, `ramp-up` | approx | script |
+| Stepping Thread Group | `kg.apc...SteppingThreadGroup` | VUs=`num_threads`, duration=`flighttime`, `ramp-up` | approx | script |
+| Ultimate Thread Group | `kg.apc...UltimateThreadGroup` | VUs=max row concurrency, duration=sum of row times | approx | script |
+| Arrivals Thread Group | `...ArrivalsThreadGroup` | treated like concurrency | approx | script |
+| Transaction Controller | `TransactionController` | folder | exact | script |
+| Simple/Generic Controller | `GenericController`/`SimpleController` | folder | exact | script |
+| Loop Controller | `LoopController` | flattened; loop count → duration | approx | script |
+| If Controller | `IfController` | each enclosed request gets a pre-request skip guard (`pm.execution.skipRequest`); `${var}` → `pm.variables.get(...)` | approx | script |
+| While Controller | `WhileController` | flattened, requests run unconditionally; restore looping with `pm.execution.setNextRequest` | approx | model |
+| Recording Controller | `RecordingController` | transparent (children recursed) | exact | script |
+| Module / Include Controller | `ModuleController`/`IncludeController` | flattened children if inline; external refs reported | manual | model |
+
+## Requests
+
+| JMeter | XML | Postman | Fidelity | Owner |
+|---|---|---|---|---|
+| HTTP Request | `HTTPSamplerProxy` | request item | exact | script |
+| method / path / domain / port / protocol | `HTTPSampler.*` | `request.method` + `url` (blank protocol → `http`) | exact | script |
+| Query/body params | `Arguments`/`HTTPArgument` | query (GET) / urlencoded (POST/PUT/PATCH) / formdata (multipart) | exact | script |
+| Raw body | `postBodyRaw=true` | `body.mode=raw` | exact | script |
+| Header Manager | `HeaderManager` | `request.header` (merged by scope, later wins) | exact | script |
+| HTTP Request Defaults | `ConfigTestElement`/`HttpDefaultsGui` | fallback for blank domain/port/protocol/path | exact | script |
+| Auth Manager Basic/Digest | `AuthManager` | `request.auth` basic/digest (first entry; multiples reported) | exact | script |
+| Auth Manager Kerberos/NTLM | `AuthManager` mechanism | — | unsupported | script |
+| Cookie / Cache / DNS Manager | `CookieManager`/... | auto per-VU cookies; presets not transferred | unsupported | script |
+| Non-HTTP samplers | `JDBC/FTP/LDAP/Java/TCP/Debug/Dummy...Sampler` | — | unsupported | script (reported) |
+
+## Extractors (post-response → test script `pm.collectionVariables.set`)
+
+| JMeter | XML | Fidelity | Owner |
+|---|---|---|---|
+| Regular Expression Extractor | `RegexExtractor` | exact (via `new RegExp`, group from `$n$` template) | script |
+| Boundary Extractor | `BoundaryExtractor` | exact (left/right → non-greedy regex) | script |
+| JSON Extractor (native) | `JSONPostProcessor` | exact for dot/index paths; complex → manual | script→model |
+| JSON Path Extractor (plugin) | `...JSONPathExtractor` | exact for simple paths | script→model |
+| XPath / XPath2 Extractor | `XPathExtractor`/`XPath2Extractor` | manual (no sandbox XPath) | model |
+| CSS/HTML Extractor | `HtmlExtractor` | manual (no sandbox CSS) | model |
+
+## Assertions (→ `pm.test`)
+
+| JMeter | XML | Maps to | Fidelity | Owner |
+|---|---|---|---|---|
+| Response Assertion | `ResponseAssertion` | status / contains / equals / matches, honouring `test_type` bitmask + NOT | exact | script |
+| — test_field response_code | | `pm.response.code` | exact | script |
+| — test_field response_data | | `pm.response.text()` | exact | script |
+| — test_field headers/message | | `pm.response.headers`/`.status` | exact | script |
+| Duration Assertion | `DurationAssertion` | `pm.test` responseTime + `--pass-if less_than(p95, ms)` hint | exact | script |
+| Size Assertion | `SizeAssertion` | `pm.test` on `responseSize` | approx | script |
+| JSON Assertion | `JSONPathAssertion` | `pm.test` on simple path | approx | script→model |
+
+`ResponseAssertion.test_type` bits: `MATCH=1, CONTAINS=2, NOT=4, EQUALS=8, SUBSTRING=16, OR=32`.
+
+## Variables & data
+
+| JMeter | XML | Postman | Fidelity | Owner |
+|---|---|---|---|---|
+| User Defined Variables | `Arguments` under TestPlan/scope | collection `variable[]` | exact | script |
+| `${var}` references | inline | `{{var}}` | exact | script |
+| `${__UUID}` / `${__time}` | function | `{{$guid}}` / `{{$timestamp}}` | exact | script |
+| `${__Random(...)}` and other `__fn` | function | `{{$randomInt}}` or flagged reference | approx/manual | script→model |
+| CSV Data Set Config | `CSVDataSet` | `--data-file` + column list | approx (one file/run; naive CSV) | script |
+| Counter / Random Variable | `CounterConfig`/`RandomVariableConfig` | pre-request script or `{{$randomInt}}` | manual | model |
+
+## Scripting
+
+| JMeter | XML | Postman | Fidelity | Owner |
+|---|---|---|---|---|
+| JSR223 PreProcessor | `JSR223PreProcessor` | prerequest script (JS, IIFE-wrapped) | model-assisted | script+model |
+| JSR223 PostProcessor | `JSR223PostProcessor` | test script | model-assisted | script+model |
+| BeanShell Pre/PostProcessor | `BeanShell*Processor` | pre/test script | model-assisted | script+model |
+| JSR223 / BeanShell Sampler | `JSR223Sampler`/`BeanShellSampler` | script-only stub (no HTTP) | manual | script+model |
+
+## Timing / pacing — approximated, always reported
+
+Postman has no native think time. A fixed delay is recovered where one exists;
+random/throughput/sync shaping is not reproduced.
+
+| JMeter | XML | Mapping |
+|---|---|---|
+| Constant Timer (per-request) | `ConstantTimer` under a sampler | pre-request pause of its delay |
+| Constant / Uniform / Gaussian / Poisson Timer (group scope) | `ConstantTimer`/`UniformRandomTimer`/... | `--delay-request <ms>` (fixed part only) |
+| Constant Throughput Timer | `ConstantThroughputTimer` | not reproduced; reported |
+| Throughput Shaping Timer | plugin | not reproduced; reported |
+| Synchronizing Timer | `SyncTimer` | not reproduced; reported |
+| Test Action (pause/stop) | `TestAction` | dropped; reported |

package/skills/performance-testing/references/jmeter-import/postman-perf-capabilities.md

@@ -0,0 +1,56 @@
+# What Postman performance testing supports (the import target)
+
+Grounded in the `postman-cli`, `performance-test-api`, and `postman-performance-test` source — not the marketing docs. This is the ceiling the importer maps onto. When JMeter asks for something not here, it is reported, never faked.
+
+## `postman performance run` CLI flags that matter for import
+
+| Concern | Flag | Notes |
+|---------|------|-------|
+| Virtual users | `--vu-count <n>` | default 20; cloud runs enforce a **minimum of 10** |
+| Duration | `-d, --duration <minutes>` | **minutes**, not seconds; default 1; cloud ceiling ~120 min (plan-gated) |
+| Load profile | `-p, --load-profile <p>` | one of `fixed`, `ramp-up`, `spike`, `peak`; default `ramp-up` |
+| Data file | `--data-file <path>` | JSON array or CSV; **local runs only**; distribution is forced to `random` |
+| Dataset | `--dataset-id` + `--dataset-view-id` + `--dataset-distribution` | cloud path; distribution `round-robin`/`fixed`/`random` |
+| Pass/fail | `--pass-if "<fn>(<metric>, <value>)"` | **single, not repeatable** |
+| Environment | `-e, --environment <id>` | |
+| Setup/teardown | `--setup-collection` / `--teardown-collection` | run **once** before/after — not concurrent load |
+
+There is **no** `--load-segments`, no `--stages`, no `--iterations`, no think-time flag. A latent YAML "stages" schema exists in the CLI repo but is not wired to the command — do not target it.
+
+## Load profiles
+
+Four named shapes only, each derived from `(vu-count, duration)`:
+
+- `fixed` — constant VUs.
+- `ramp-up` — ramps from 25% to full over the first quarter.
+- `spike` — low baseline, one spike to full mid-run, back down.
+- `peak` — baseline, ramp up, hold, ramp down.
+
+JMeter custom ramp curves, stepping schedules, and arrival-rate shapes **cannot be reproduced**. Pick the closest of the four and report the approximation. Rule used by the importer: any ramp/stepping/concurrency/ultimate/arrivals group → `ramp-up`; a flat group with zero ramp → `fixed`.
+
+## `--pass-if` metrics and operators
+
+- Functions (operators): `less_than`, `greater_than`, `less_than_eq`, `greater_than_eq`.
+- Metrics (CLI): `avg`, `p90`, `p95`, `p99` (ms); `error_rate`, `failure_rate` (%); `rps`.
+- **Server-side (cloud) accepts a narrower set**: `responseTime.average/p90/p95/p99`, `throughput`, `errorRate`. `failure_rate` is local-only — if you emit it, note it won't gate a cloud run.
+- Only one condition per run. A JMeter Duration Assertion is the only clean source (→ `less_than(p95, <ms>)`); otherwise suggest `less_than(error_rate, 5)` and tell the user it's a default.
+
+## Scripts under load
+
+- Pre-request / test scripts run per VU per iteration via `postman-runtime`.
+- `postman.setNextRequest(...)` **is supported** — flow control within one VU's iteration works.
+- Vault access **in scripts** is disabled; per-request timeouts are forced to infinite; each VU gets its own cookie jar.
+- Known engine caveat: mutating `pm.request` inside a script can bleed across VUs. `pm.iterationData` / `pm.variables` are correctly isolated. Avoid importing JMeter logic into `pm.request` mutation.
+
+## Data
+
+- One `--data-file` (JSON array of objects, or CSV) per run. Multiple JMeter CSV Data Sets must be merged or a primary chosen.
+- Rows map to VUs/iterations; rows < VUs repeat. The CLI CSV parser is a naive comma split — **no quoted-field/escape support**, so JMeter CSVs with embedded commas/quotes need cleaning.
+
+## Hard limits to encode in every import
+
+1. One collection, repeated across VUs. **No concurrent thread groups.**
+2. Time-based, not iteration-based. **Loop counts → duration.**
+3. One `--pass-if`.
+4. **No native think time** — timers are approximated (`--delay-request` for group scope, a pre-request pause for per-request), not reproduced exactly.
+5. **HTTP(S) only.**

package/skills/performance-testing/references/jmeter-import/script-translation.md

@@ -0,0 +1,98 @@
+# Translating JSR223 / BeanShell scripts to Postman JavaScript
+
+`scripts/groovy_to_js.py` does a best-effort mechanical pass and the importer wraps every result in an IIFE so it is always syntactically valid. It is **never trusted** — each imported script is flagged as a manual task. This file is how you (the model) finish the job.
+
+## Execution-context map
+
+JMeter runs these inside a sampler's processor. Postman runs them as a request's `prerequest` or `test` event. Equivalents:
+
+| JMeter (Groovy/BeanShell) | Postman JS | Context |
+|---|---|---|
+| `vars.get("x")` / `vars["x"]` | `pm.variables.get("x")` | both |
+| `vars.put("x", v)` | `pm.variables.set("x", v)` | both |
+| `props.get/put(...)` | `pm.variables.get/set(...)` (approx) | both |
+| `log.info/warn/error(...)` | `console.log/warn/error(...)` | both |
+| `prev.getResponseDataAsString()` | `pm.response.text()` | post only |
+| `prev.getResponseCode()` | `String(pm.response.code)` | post only |
+| `prev.getResponseMessage()` | `pm.response.status` | post only |
+| `prev.getTime()` | `pm.response.responseTime` | post only |
+| `ctx.getPreviousResult()` | `pm.response` | post only |
+| `sampler.getUrl()` | `pm.request.url.toString()` | pre only |
+| `sampler.addArgument(k,v)` | `pm.request.url.query.add({key:k,value:v})` or body edit | pre only |
+| `System.currentTimeMillis()` | `Date.now()` | both |
+| `UUID.randomUUID().toString()` | `require('uuid').v4()` | both |
+| `Integer.parseInt(x)` | `parseInt(x)` | both |
+
+## What the mechanical pass cannot do — fix these by hand
+
+- **BeanShell implicit result vars** (`ResponseCode`, `ResponseMessage`, `IsSuccess`, `bsh.args`, `Parameters`). No pm.* equivalent in a processor. For a sampler that sets `IsSuccess=false`, express it as a `pm.test(...)` that fails. For `return X`, delete the return (the IIFE makes it legal, but it's a no-op).
+- **Groovy closures / collection methods** (`.each {}`, `.collect {}`, `->`). Rewrite as `for...of` / `.map()` / arrow functions.
+- **`import` / `new ClassName()` / Java types** (`StringBuilder`, `SimpleDateFormat`, `JsonSlurper`). Replace with JS: string concat, `Date`, `JSON.parse`. `JsonSlurper().parseText(prev...)` → `pm.response.json()`.
+- **`try/catch`, multi-statement logic** — valid JS already; just verify semantics.
+- **GString interpolation** `"${x} and ${y}"` → template literal `` `${x} and ${y}` `` **only if** `x`/`y` are JS locals; if they are JMeter vars use `pm.variables.get`.
+
+## Procedure
+
+1. Read the `// --- imported ... (REVIEW) ---` block in the request's event script.
+2. Compare against the original `<script>` text in the `.jmx` (the IR keeps the element's name).
+3. Rewrite line by line using the table. Preserve variable names so `{{var}}` references elsewhere still resolve.
+4. Remove the `// REVIEW` marker once correct.
+5. Re-run the node syntax gate from `SKILL.md` step 3.
+
+## Example
+
+JMeter (Groovy post-processor):
+```groovy
+import groovy.json.JsonSlurper
+def json = new JsonSlurper().parseText(prev.getResponseDataAsString())
+def token = json.data.token
+vars.put("authToken", token)
+log.info("got " + token)
+```
+Postman (test script):
+```javascript
+const json = pm.response.json();
+const token = json.data.token;
+pm.collectionVariables.set("authToken", token);
+console.log("got " + token);
+```
+
+## Canonical JS templates (match the script byte-for-byte in fallback mode)
+
+When importing by hand, emit exactly these so hand output equals the script's. `<default>` empty-string if JMeter has none. Source is `pm.response.text()` normally, or `pm.response.headers.toString()` when the extractor's `useHeaders` is true.
+
+**Regex extractor** (`$n$` template → group `n`):
+```javascript
+try { var _m = String(pm.response.text()).match(new RegExp("<REGEX>")); pm.collectionVariables.set("<ref>", _m ? _m[<n>] : "<default>"); } catch (e) { pm.collectionVariables.set("<ref>", "<default>"); }
+```
+
+**Boundary extractor** (left/right → non-greedy capture):
+```javascript
+try { var _b = String(pm.response.text()).match(new RegExp("<lboundary>([\\s\\S]*?)<rboundary>")); pm.collectionVariables.set("<ref>", _b ? _b[1] : "<default>"); } catch (e) { pm.collectionVariables.set("<ref>", "<default>"); }
+```
+
+**JSON extractor** (`JSONPostProcessor` / plugin). Only **simple** paths (`$.a.b`, `$.a[0].c`) import; `$`/`$.` → `pm.response.json()`:
+```javascript
+try { pm.collectionVariables.set("<ref>", pm.response.json().a.b[0]); } catch (e) { pm.collectionVariables.set("<ref>", "<default>"); }
+```
+Wildcards / filters / recursive descent (`*`, `..`, `?`, `@`) → do NOT import; emit `// [manual] JSONPath "<expr>" too complex` and add a manual task.
+
+**Implicit HTTP success** (first test on every request unless a `response_code`/`response_message` assertion or `assume_success` exists):
+```javascript
+pm.test('JMeter default success (HTTP 200-399)', function () {
+    pm.expect(pm.response.code).to.be.within(200, 399);
+});
+```
+
+**Response Assertion** — decode `test_type` bits (`MATCH=1, CONTAINS=2, NOT=4, EQUALS=8, SUBSTRING=16, OR=32`). Field getter: `response_code`→`pm.response.code`, `response_message`→`pm.response.status`, `response_headers`→`pm.response.headers.toString()`, else `pm.response.text()`. For each test string `s`:
+- **EQUALS (8)** on `response_code`: `pm.expect(pm.response.code).to.eql(<s>)` (number). Other fields: `.to.eql("<s>")`.
+- **SUBSTRING (16)** → literal: `pm.expect(<getter>).to.include("<s>")`.
+- **CONTAINS (2) or MATCH (1)** → regex: `pm.expect(String(<getter>)).to.match(new RegExp("<s>"))`.
+- **NOT (4)** → insert `.not` before `.to`.
+Wrap each in `pm.test("<name>", function () { <expr>; });`.
+
+**Duration Assertion** → a test plus a perf `--pass-if` hint:
+```javascript
+pm.test("Response time < <ms>ms", function () { pm.expect(pm.response.responseTime).to.be.below(<ms>); });
+```
+and set the run's `--pass-if "less_than(p95, <ms>)"`.

package/skills/performance-testing/references/jmeter-import/scripts/groovy_to_js.py

@@ -0,0 +1,83 @@
+"""Best-effort, deterministic Groovy / BeanShell -> JavaScript translation.
+
+This handles the common mechanical rewrites so the model only has to review and
+fix the genuinely ambiguous parts. Every script that passes through here is ALSO
+flagged as a manual task, because no regex pass can guarantee semantic fidelity
+for arbitrary JSR223/BeanShell code.
+
+Keep the rules ordered from most specific to least specific. The model may extend
+this table; see references/script-translation.md for the patterns and rationale.
+"""
+import re
+
+# (pattern, replacement) applied in order. Patterns are plain regex on source text.
+RULES = [
+    # --- JMeter bound variables -> pm equivalents ---
+    # vars["x"] / vars['x'] -> pm.variables.get("x")  (bracket form, must precede .get/.put rules)
+    (r'\bvars\[\s*([\'"][^\'"]*[\'"])\s*\]', r'pm.variables.get(\1)'),
+    (r'\bprops\[\s*([\'"][^\'"]*[\'"])\s*\]', r'pm.variables.get(\1)'),
+    (r'\bvars\.get\(', 'pm.variables.get('),
+    (r'\bvars\.put\(', 'pm.variables.set('),
+    (r'\bvars\.getObject\(', 'pm.variables.get('),
+    (r'\bvars\.putObject\(', 'pm.variables.set('),
+    (r'\bprops\.get\(', 'pm.variables.get('),
+    (r'\bprops\.put\(', 'pm.variables.set('),
+
+    # --- logging ---
+    (r'\blog\.info\(', 'console.log('),
+    (r'\blog\.warn\(', 'console.warn('),
+    (r'\blog\.error\(', 'console.error('),
+    (r'\blog\.debug\(', 'console.log('),
+
+    # --- previous-sample result accessors (post-processor context) ---
+    (r'\bprev\.getResponseDataAsString\(\)', 'pm.response.text()'),
+    (r'\bprev\.getResponseCode\(\)', 'String(pm.response.code)'),
+    (r'\bprev\.getResponseMessage\(\)', 'pm.response.status'),
+    (r'\bprev\.getResponseHeaders\(\)', 'pm.response.headers.toString()'),
+    (r'\bprev\.getTime\(\)', 'pm.response.responseTime'),
+    (r'\bctx\.getPreviousResult\(\)', 'pm.response'),
+
+    # --- sampler mutation (pre-processor context) ---
+    (r'\bsampler\.getUrl\(\)', 'pm.request.url.toString()'),
+    (r'\bsampler\.addArgument\(', '// [manual] sampler.addArgument -> set a query/body param; '),
+
+    # --- JDK calls that have JS equivalents ---
+    (r'\bSystem\.currentTimeMillis\(\)', 'Date.now()'),
+    (r'\bUUID\.randomUUID\(\)\.toString\(\)', 'require("uuid").v4()'),
+    (r'\bInteger\.parseInt\(', 'parseInt('),
+    (r'\bLong\.parseLong\(', 'parseInt('),
+    (r'\bDouble\.parseDouble\(', 'parseFloat('),
+    (r'\bString\.valueOf\(', 'String('),
+    (r'\bMath\.', 'Math.'),  # identity, keeps it explicit
+
+    # --- declarations: Groovy def / typed decls -> let ---
+    (r'\bdef\s+', 'let '),
+    (r'\b(?:String|int|long|double|float|boolean|Object|var)\s+(\w+\s*=)', r'let \1'),
+
+    # --- Groovy GString interpolation "${x}" stays valid-ish; leave for model ---
+]
+
+# Tokens that, if present after translation, almost certainly need human review.
+REVIEW_MARKERS = [
+    'import ', 'new ', 'class ', '.each', '.collect', '->', 'bsh.', 'ctx.',
+    'OUT.', 'SampleResult', 'Thread.sleep', 'try {', 'catch', '@', '::',
+]
+
+
+def translate(source: str, lang: str = 'groovy') -> dict:
+    """Return {'js': <translated>, 'needs_review': bool, 'reasons': [...]}."""
+    if source is None:
+        return {'js': '', 'needs_review': False, 'reasons': []}
+    js = source
+    for pat, repl in RULES:
+        js = re.sub(pat, repl, js)
+
+    # BeanShell implicit result vars have no clean pm.* mapping.
+    reasons = []
+    if lang and 'beanshell' in lang.lower():
+        reasons.append('BeanShell script: implicit vars (ResponseCode, IsSuccess, bsh.args) have no pm.* equivalent')
+    for marker in REVIEW_MARKERS:
+        if marker in js:
+            reasons.append(f'contains "{marker.strip()}" which needs manual translation')
+    needs_review = True  # always true: no mechanical pass is trustworthy for scripts
+    return {'js': js, 'needs_review': needs_review, 'reasons': sorted(set(reasons))}