@postman/postman-plugin 0.4.0 → 0.4.1-rc.0

package/skills/performance-testing/references/jmeter-import/import-report.md

@@ -1,7 +1,7 @@
 # Import Report — <TEST PLAN NAME>
 
-> Generated by `jmx_to_postman.py`; sections below are auto-filled. After resolving
-> manual tasks, update the "Needs manual work" list and the summary counts.
+> Fill in every section from the import. Once the manual tasks are resolved, update the
+> "Needs manual work" list and the summary counts to match the collection you hand off.
 
 ## Summary
 

package/skills/performance-testing/references/jmeter-import/mapping-table.md

@@ -1,97 +1,95 @@
 # JMeter → Postman mapping table
 
-Fidelity: **exact** (behaviour preserved) · **approx** (closest supported equivalent, reported) · **manual** (needs the model/user) · **unsupported** (dropped, reported).
-Owner: **script** (deterministic Python) · **model** (judgment call).
+Fidelity: **exact** (behaviour preserved) · **approx** (closest supported equivalent, reported) · **manual** (needs a judgment call, often with the user) · **unsupported** (dropped, reported).
 
 ## Structure
 
-| JMeter element | XML tag | Postman | Fidelity | Owner |
-|---|---|---|---|---|
-| Test Plan | `TestPlan` | Collection (`info` + `item`) | exact | script |
-| Thread Group | `ThreadGroup` | load config (VUs/duration/profile) + folder when >1 | approx | script |
-| setUp/tearDown Thread Group | `SetupThreadGroup`/`PostThreadGroup` | folder; suggest `--setup-collection`/`--teardown-collection` | approx | script+model |
-| Concurrency Thread Group | `...ConcurrencyThreadGroup` | VUs=`TargetLevel`, duration=`RampUp`+`Hold`, `ramp-up` | approx | script |
-| Stepping Thread Group | `kg.apc...SteppingThreadGroup` | VUs=`num_threads`, duration=`flighttime`, `ramp-up` | approx | script |
-| Ultimate Thread Group | `kg.apc...UltimateThreadGroup` | VUs=max row concurrency, duration=sum of row times | approx | script |
-| Arrivals Thread Group | `...ArrivalsThreadGroup` | treated like concurrency | approx | script |
-| Transaction Controller | `TransactionController` | folder | exact | script |
-| Simple/Generic Controller | `GenericController`/`SimpleController` | folder | exact | script |
-| Loop Controller | `LoopController` | flattened; loop count → duration | approx | script |
-| If Controller | `IfController` | each enclosed request gets a pre-request skip guard (`pm.execution.skipRequest`); `${var}` → `pm.variables.get(...)` | approx | script |
-| While Controller | `WhileController` | flattened, requests run unconditionally; restore looping with `pm.execution.setNextRequest` | approx | model |
-| Recording Controller | `RecordingController` | transparent (children recursed) | exact | script |
-| Module / Include Controller | `ModuleController`/`IncludeController` | flattened children if inline; external refs reported | manual | model |
+| JMeter element | XML tag | Postman | Fidelity |
+|---|---|---|---|
+| Test Plan | `TestPlan` | Collection (`info` + `item`) | exact |
+| Thread Group | `ThreadGroup` | load config (VUs/duration/profile) + folder when >1 | approx |
+| setUp/tearDown Thread Group | `SetupThreadGroup`/`PostThreadGroup` | folder; suggest `--setup-collection`/`--teardown-collection` | approx |
+| Concurrency Thread Group | `...ConcurrencyThreadGroup` | VUs=`TargetLevel`, duration=`RampUp`+`Hold`, `ramp-up` | approx |
+| Stepping Thread Group | `kg.apc...SteppingThreadGroup` | VUs=`num_threads`, duration=`flighttime`, `ramp-up` | approx |
+| Ultimate Thread Group | `kg.apc...UltimateThreadGroup` | VUs=max row concurrency, duration=sum of row times | approx |
+| Arrivals Thread Group | `...ArrivalsThreadGroup` | treated like concurrency | approx |
+| Transaction Controller | `TransactionController` | folder | exact |
+| Simple/Generic Controller | `GenericController`/`SimpleController` | folder | exact |
+| Loop Controller | `LoopController` | flattened; loop count → duration | approx |
+| If Controller | `IfController` | each enclosed request gets a pre-request skip guard (`pm.execution.skipRequest`); `${var}` → `pm.variables.get(...)` | approx |
+| While Controller | `WhileController` | flattened, requests run unconditionally; restore looping with `pm.execution.setNextRequest` | approx |
+| Recording Controller | `RecordingController` | transparent (children recursed) | exact |
+| Module / Include Controller | `ModuleController`/`IncludeController` | flattened children if inline; external refs reported | manual |
 
 ## Requests
 
-| JMeter | XML | Postman | Fidelity | Owner |
-|---|---|---|---|---|
-| HTTP Request | `HTTPSamplerProxy` | request item | exact | script |
-| method / path / domain / port / protocol | `HTTPSampler.*` | `request.method` + `url` (blank protocol → `http`) | exact | script |
-| Query/body params | `Arguments`/`HTTPArgument` | query (GET) / urlencoded (POST/PUT/PATCH) / formdata (multipart) | exact | script |
-| Raw body | `postBodyRaw=true` | `body.mode=raw` | exact | script |
-| Header Manager | `HeaderManager` | `request.header` (merged by scope, later wins) | exact | script |
-| HTTP Request Defaults | `ConfigTestElement`/`HttpDefaultsGui` | fallback for blank domain/port/protocol/path | exact | script |
-| Auth Manager Basic/Digest | `AuthManager` | `request.auth` basic/digest (first entry; multiples reported) | exact | script |
-| Auth Manager Kerberos/NTLM | `AuthManager` mechanism | — | unsupported | script |
-| Cookie / Cache / DNS Manager | `CookieManager`/... | auto per-VU cookies; presets not transferred | unsupported | script |
-| Non-HTTP samplers | `JDBC/FTP/LDAP/Java/TCP/Debug/Dummy...Sampler` | — | unsupported | script (reported) |
+| JMeter | XML | Postman | Fidelity |
+|---|---|---|---|
+| HTTP Request | `HTTPSamplerProxy` | request item | exact |
+| method / path / domain / port / protocol | `HTTPSampler.*` | `request.method` + `url` (blank protocol → `http`) | exact |
+| Query/body params | `Arguments`/`HTTPArgument` | query (GET) / urlencoded (POST/PUT/PATCH) / formdata (multipart) | exact |
+| Raw body | `postBodyRaw=true` | `body.mode=raw` | exact |
+| Header Manager | `HeaderManager` | `request.header` (merged by scope, later wins) | exact |
+| HTTP Request Defaults | `ConfigTestElement`/`HttpDefaultsGui` | fallback for blank domain/port/protocol/path | exact |
+| Auth Manager Basic/Digest | `AuthManager` | `request.auth` basic/digest (first entry; multiples reported) | exact |
+| Auth Manager Kerberos/NTLM | `AuthManager` mechanism | — | unsupported |
+| Cookie / Cache / DNS Manager | `CookieManager`/... | auto per-VU cookies; presets not transferred | unsupported |
+| Non-HTTP samplers | `JDBC/FTP/LDAP/Java/TCP/Debug/Dummy...Sampler` | — | unsupported |
 
 ## Extractors (post-response → test script `pm.collectionVariables.set`)
 
-| JMeter | XML | Fidelity | Owner |
-|---|---|---|---|
-| Regular Expression Extractor | `RegexExtractor` | exact (via `new RegExp`, group from `$n$` template) | script |
-| Boundary Extractor | `BoundaryExtractor` | exact (left/right → non-greedy regex) | script |
-| JSON Extractor (native) | `JSONPostProcessor` | exact for dot/index paths; complex → manual | script→model |
-| JSON Path Extractor (plugin) | `...JSONPathExtractor` | exact for simple paths | script→model |
-| XPath / XPath2 Extractor | `XPathExtractor`/`XPath2Extractor` | manual (no sandbox XPath) | model |
-| CSS/HTML Extractor | `HtmlExtractor` | manual (no sandbox CSS) | model |
+| JMeter | XML | Fidelity |
+|---|---|---|
+| Regular Expression Extractor | `RegexExtractor` | exact (via `new RegExp`, group from `$n$` template) |
+| Boundary Extractor | `BoundaryExtractor` | exact (left/right → non-greedy regex) |
+| JSON Extractor (native) | `JSONPostProcessor` | exact for dot/index paths; complex → manual |
+| JSON Path Extractor (plugin) | `...JSONPathExtractor` | exact for simple paths |
+| XPath / XPath2 Extractor | `XPathExtractor`/`XPath2Extractor` | manual (no sandbox XPath) |
+| CSS/HTML Extractor | `HtmlExtractor` | manual (no sandbox CSS) |
 
 ## Assertions (→ `pm.test`)
 
-| JMeter | XML | Maps to | Fidelity | Owner |
-|---|---|---|---|---|
-| Response Assertion | `ResponseAssertion` | status / contains / equals / matches, honouring `test_type` bitmask + NOT | exact | script |
-| — test_field response_code | | `pm.response.code` | exact | script |
-| — test_field response_data | | `pm.response.text()` | exact | script |
-| — test_field headers/message | | `pm.response.headers`/`.status` | exact | script |
-| Duration Assertion | `DurationAssertion` | `pm.test` responseTime + `--pass-if less_than(p95, ms)` hint | exact | script |
-| Size Assertion | `SizeAssertion` | `pm.test` on `responseSize` | approx | script |
-| JSON Assertion | `JSONPathAssertion` | `pm.test` on simple path | approx | script→model |
+| JMeter | XML | Maps to | Fidelity |
+|---|---|---|---|
+| Response Assertion | `ResponseAssertion` | status / contains / equals / matches, honouring `test_type` bitmask + NOT | exact |
+| — test_field response_code | | `pm.response.code` | exact |
+| — test_field response_data | | `pm.response.text()` | exact |
+| — test_field headers/message | | `pm.response.headers`/`.status` | exact |
+| Duration Assertion | `DurationAssertion` | `pm.test` responseTime + `--pass-if less_than(p95, ms)` hint | exact |
+| Size Assertion | `SizeAssertion` | `pm.test` on `responseSize` | approx |
+| JSON Assertion | `JSONPathAssertion` | `pm.test` on simple path | approx |
 
 `ResponseAssertion.test_type` bits: `MATCH=1, CONTAINS=2, NOT=4, EQUALS=8, SUBSTRING=16, OR=32`.
 
 ## Variables & data
 
-| JMeter | XML | Postman | Fidelity | Owner |
-|---|---|---|---|---|
-| User Defined Variables | `Arguments` under TestPlan/scope | collection `variable[]` | exact | script |
-| `${var}` references | inline | `{{var}}` | exact | script |
-| `${__UUID}` / `${__time}` | function | `{{$guid}}` / `{{$timestamp}}` | exact | script |
-| `${__Random(...)}` and other `__fn` | function | `{{$randomInt}}` or flagged reference | approx/manual | script→model |
-| CSV Data Set Config | `CSVDataSet` | `--data-file` + column list | approx (one file/run; naive CSV) | script |
-| Counter / Random Variable | `CounterConfig`/`RandomVariableConfig` | pre-request script or `{{$randomInt}}` | manual | model |
+| JMeter | XML | Postman | Fidelity |
+|---|---|---|---|
+| User Defined Variables | `Arguments` under TestPlan/scope | collection `variable[]` | exact |
+| `${var}` references | inline | `{{var}}` | exact |
+| `${__UUID}` / `${__time}` | function | `{{$guid}}` / `{{$timestamp}}` | exact |
+| `${__Random(...)}` and other `__fn` | function | `{{$randomInt}}` or flagged reference | approx/manual |
+| CSV Data Set Config | `CSVDataSet` | `--data-file` + column list | approx (one file/run; naive CSV) |
+| Counter / Random Variable | `CounterConfig`/`RandomVariableConfig` | pre-request script or `{{$randomInt}}` | manual |
 
 ## Scripting
 
-| JMeter | XML | Postman | Fidelity | Owner |
-|---|---|---|---|---|
-| JSR223 PreProcessor | `JSR223PreProcessor` | prerequest script (JS, IIFE-wrapped) | model-assisted | script+model |
-| JSR223 PostProcessor | `JSR223PostProcessor` | test script | model-assisted | script+model |
-| BeanShell Pre/PostProcessor | `BeanShell*Processor` | pre/test script | model-assisted | script+model |
-| JSR223 / BeanShell Sampler | `JSR223Sampler`/`BeanShellSampler` | script-only stub (no HTTP) | manual | script+model |
-
-## Timing / pacing — approximated, always reported
-
-Postman has no native think time. A fixed delay is recovered where one exists;
-random/throughput/sync shaping is not reproduced.
-
-| JMeter | XML | Mapping |
-|---|---|---|
-| Constant Timer (per-request) | `ConstantTimer` under a sampler | pre-request pause of its delay |
-| Constant / Uniform / Gaussian / Poisson Timer (group scope) | `ConstantTimer`/`UniformRandomTimer`/... | `--delay-request <ms>` (fixed part only) |
-| Constant Throughput Timer | `ConstantThroughputTimer` | not reproduced; reported |
-| Throughput Shaping Timer | plugin | not reproduced; reported |
-| Synchronizing Timer | `SyncTimer` | not reproduced; reported |
-| Test Action (pause/stop) | `TestAction` | dropped; reported |
+| JMeter | XML | Postman | Fidelity |
+|---|---|---|---|
+| JSR223 PreProcessor | `JSR223PreProcessor` | prerequest script (JS, IIFE-wrapped) | manual |
+| JSR223 PostProcessor | `JSR223PostProcessor` | test script | manual |
+| BeanShell Pre/PostProcessor | `BeanShell*Processor` | pre/test script | manual |
+| JSR223 / BeanShell Sampler | `JSR223Sampler`/`BeanShellSampler` | script-only stub (no HTTP) | manual |
+
+## Timing / pacing — not supported, always reported
+
+Postman performance runs have no think time and no flag for one. Every timer is
+dropped and listed in the report; none is reproduced.
+
+| JMeter | XML |
+|---|---|
+| Constant / Uniform / Gaussian / Poisson Timer | `ConstantTimer`/`UniformRandomTimer`/... |
+| Constant Throughput Timer | `ConstantThroughputTimer` |
+| Throughput Shaping Timer | plugin |
+| Synchronizing Timer | `SyncTimer` |
+| Test Action (pause/stop) | `TestAction` |

package/skills/performance-testing/references/jmeter-import/postman-perf-capabilities.md

@@ -1,6 +1,6 @@
 # What Postman performance testing supports (the import target)
 
-Grounded in the `postman-cli`, `performance-test-api`, and `postman-performance-test` source — not the marketing docs. This is the ceiling the importer maps onto. When JMeter asks for something not here, it is reported, never faked.
+Grounded in the `postman-cli`, `performance-test-api`, and `postman-performance-test` source — not the marketing docs. This is the ceiling an import maps onto. When JMeter asks for something not here, it is reported, never faked.
 
 ## `postman performance run` CLI flags that matter for import
 
@@ -26,7 +26,7 @@ Four named shapes only, each derived from `(vu-count, duration)`:
 - `spike` — low baseline, one spike to full mid-run, back down.
 - `peak` — baseline, ramp up, hold, ramp down.
 
-JMeter custom ramp curves, stepping schedules, and arrival-rate shapes **cannot be reproduced**. Pick the closest of the four and report the approximation. Rule used by the importer: any ramp/stepping/concurrency/ultimate/arrivals group → `ramp-up`; a flat group with zero ramp → `fixed`.
+JMeter custom ramp curves, stepping schedules, and arrival-rate shapes **cannot be reproduced**. Pick the closest of the four and report the approximation. Rule: any ramp/stepping/concurrency/ultimate/arrivals group → `ramp-up`; a flat group with zero ramp → `fixed`.
 
 ## `--pass-if` metrics and operators
 
@@ -52,5 +52,5 @@ JMeter custom ramp curves, stepping schedules, and arrival-rate shapes **cannot
 1. One collection, repeated across VUs. **No concurrent thread groups.**
 2. Time-based, not iteration-based. **Loop counts → duration.**
 3. One `--pass-if`.
-4. **No native think time** — timers are approximated (`--delay-request` for group scope, a pre-request pause for per-request), not reproduced exactly.
+4. **No think time** — timers are not reproduced; each one is reported.
 5. **HTTP(S) only.**

package/skills/performance-testing/references/jmeter-import/script-translation.md

@@ -1,6 +1,6 @@
 # Translating JSR223 / BeanShell scripts to Postman JavaScript
 
-`scripts/groovy_to_js.py` does a best-effort mechanical pass and the importer wraps every result in an IIFE so it is always syntactically valid. It is **never trusted** — each imported script is flagged as a manual task. This file is how you (the model) finish the job.
+Every JSR223/BeanShell script is translated by hand and treated as a manual task until it's checked. Wrap each translation in an IIFE (`(function () { ... })();`) so a leftover `return` stays legal.
 
 ## Execution-context map
 
@@ -23,7 +23,7 @@ JMeter runs these inside a sampler's processor. Postman runs them as a request's
 | `UUID.randomUUID().toString()` | `require('uuid').v4()` | both |
 | `Integer.parseInt(x)` | `parseInt(x)` | both |
 
-## What the mechanical pass cannot do — fix these by hand
+## What doesn't map line by line
 
 - **BeanShell implicit result vars** (`ResponseCode`, `ResponseMessage`, `IsSuccess`, `bsh.args`, `Parameters`). No pm.* equivalent in a processor. For a sampler that sets `IsSuccess=false`, express it as a `pm.test(...)` that fails. For `return X`, delete the return (the IIFE makes it legal, but it's a no-op).
 - **Groovy closures / collection methods** (`.each {}`, `.collect {}`, `->`). Rewrite as `for...of` / `.map()` / arrow functions.
@@ -33,11 +33,10 @@ JMeter runs these inside a sampler's processor. Postman runs them as a request's
 
 ## Procedure
 
-1. Read the `// --- imported ... (REVIEW) ---` block in the request's event script.
-2. Compare against the original `<script>` text in the `.jmx` (the IR keeps the element's name).
-3. Rewrite line by line using the table. Preserve variable names so `{{var}}` references elsewhere still resolve.
-4. Remove the `// REVIEW` marker once correct.
-5. Re-run the node syntax gate from `SKILL.md` step 3.
+1. Read the original `<script>` text in the `.jmx` (note the element's name, so the report can point at it).
+2. Rewrite line by line using the table. Preserve variable names so `{{var}}` references elsewhere still resolve.
+3. Mark anything you're unsure of with a `// REVIEW` comment, and remove it once the logic is confirmed.
+4. Run the syntax check from `README.md` step 4.
 
 ## Example
 
@@ -57,9 +56,9 @@ pm.collectionVariables.set("authToken", token);
 console.log("got " + token);
 ```
 
-## Canonical JS templates (match the script byte-for-byte in fallback mode)
+## Canonical JS templates
 
-When importing by hand, emit exactly these so hand output equals the script's. `<default>` empty-string if JMeter has none. Source is `pm.response.text()` normally, or `pm.response.headers.toString()` when the extractor's `useHeaders` is true.
+Emit exactly these so every import produces the same code. `<default>` empty-string if JMeter has none. Source is `pm.response.text()` normally, or `pm.response.headers.toString()` when the extractor's `useHeaders` is true.
 
 **Regex extractor** (`$n$` template → group `n`):
 ```javascript

package/skills/performance-testing/references/jmeter-import/scripts/groovy_to_js.py

@@ -1,83 +0,0 @@
-"""Best-effort, deterministic Groovy / BeanShell -> JavaScript translation.
-
-This handles the common mechanical rewrites so the model only has to review and
-fix the genuinely ambiguous parts. Every script that passes through here is ALSO
-flagged as a manual task, because no regex pass can guarantee semantic fidelity
-for arbitrary JSR223/BeanShell code.
-
-Keep the rules ordered from most specific to least specific. The model may extend
-this table; see references/script-translation.md for the patterns and rationale.
-"""
-import re
-
-# (pattern, replacement) applied in order. Patterns are plain regex on source text.
-RULES = [
-    # --- JMeter bound variables -> pm equivalents ---
-    # vars["x"] / vars['x'] -> pm.variables.get("x")  (bracket form, must precede .get/.put rules)
-    (r'\bvars\[\s*([\'"][^\'"]*[\'"])\s*\]', r'pm.variables.get(\1)'),
-    (r'\bprops\[\s*([\'"][^\'"]*[\'"])\s*\]', r'pm.variables.get(\1)'),
-    (r'\bvars\.get\(', 'pm.variables.get('),
-    (r'\bvars\.put\(', 'pm.variables.set('),
-    (r'\bvars\.getObject\(', 'pm.variables.get('),
-    (r'\bvars\.putObject\(', 'pm.variables.set('),
-    (r'\bprops\.get\(', 'pm.variables.get('),
-    (r'\bprops\.put\(', 'pm.variables.set('),
-
-    # --- logging ---
-    (r'\blog\.info\(', 'console.log('),
-    (r'\blog\.warn\(', 'console.warn('),
-    (r'\blog\.error\(', 'console.error('),
-    (r'\blog\.debug\(', 'console.log('),
-
-    # --- previous-sample result accessors (post-processor context) ---
-    (r'\bprev\.getResponseDataAsString\(\)', 'pm.response.text()'),
-    (r'\bprev\.getResponseCode\(\)', 'String(pm.response.code)'),
-    (r'\bprev\.getResponseMessage\(\)', 'pm.response.status'),
-    (r'\bprev\.getResponseHeaders\(\)', 'pm.response.headers.toString()'),
-    (r'\bprev\.getTime\(\)', 'pm.response.responseTime'),
-    (r'\bctx\.getPreviousResult\(\)', 'pm.response'),
-
-    # --- sampler mutation (pre-processor context) ---
-    (r'\bsampler\.getUrl\(\)', 'pm.request.url.toString()'),
-    (r'\bsampler\.addArgument\(', '// [manual] sampler.addArgument -> set a query/body param; '),
-
-    # --- JDK calls that have JS equivalents ---
-    (r'\bSystem\.currentTimeMillis\(\)', 'Date.now()'),
-    (r'\bUUID\.randomUUID\(\)\.toString\(\)', 'require("uuid").v4()'),
-    (r'\bInteger\.parseInt\(', 'parseInt('),
-    (r'\bLong\.parseLong\(', 'parseInt('),
-    (r'\bDouble\.parseDouble\(', 'parseFloat('),
-    (r'\bString\.valueOf\(', 'String('),
-    (r'\bMath\.', 'Math.'),  # identity, keeps it explicit
-
-    # --- declarations: Groovy def / typed decls -> let ---
-    (r'\bdef\s+', 'let '),
-    (r'\b(?:String|int|long|double|float|boolean|Object|var)\s+(\w+\s*=)', r'let \1'),
-
-    # --- Groovy GString interpolation "${x}" stays valid-ish; leave for model ---
-]
-
-# Tokens that, if present after translation, almost certainly need human review.
-REVIEW_MARKERS = [
-    'import ', 'new ', 'class ', '.each', '.collect', '->', 'bsh.', 'ctx.',
-    'OUT.', 'SampleResult', 'Thread.sleep', 'try {', 'catch', '@', '::',
-]
-
-
-def translate(source: str, lang: str = 'groovy') -> dict:
-    """Return {'js': <translated>, 'needs_review': bool, 'reasons': [...]}."""
-    if source is None:
-        return {'js': '', 'needs_review': False, 'reasons': []}
-    js = source
-    for pat, repl in RULES:
-        js = re.sub(pat, repl, js)
-
-    # BeanShell implicit result vars have no clean pm.* mapping.
-    reasons = []
-    if lang and 'beanshell' in lang.lower():
-        reasons.append('BeanShell script: implicit vars (ResponseCode, IsSuccess, bsh.args) have no pm.* equivalent')
-    for marker in REVIEW_MARKERS:
-        if marker in js:
-            reasons.append(f'contains "{marker.strip()}" which needs manual translation')
-    needs_review = True  # always true: no mechanical pass is trustworthy for scripts
-    return {'js': js, 'needs_review': needs_review, 'reasons': sorted(set(reasons))}