@postman/postman-plugin 0.2.2 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +58 -10
- package/dist/hosts/antigravity.js +47 -0
- package/dist/hosts/copilot.js +60 -0
- package/dist/hosts/index.js +3 -1
- package/dist/hosts/opencode-config.js +147 -0
- package/dist/hosts/opencode.js +315 -34
- package/dist/hosts/shared.js +10 -3
- package/dist/source.js +19 -5
- package/dist/system.js +20 -1
- package/hooks/session-start-context.md +1 -1
- package/mcp.pi.json +2 -2
- package/package.json +1 -1
- package/skills/ai-readiness/SKILL.md +40 -44
- package/skills/api-discovery/SKILL.md +64 -129
- package/skills/api-discovery/reference/orbit.md +17 -5
- package/skills/api-documentation/SKILL.md +38 -25
- package/skills/api-engineer/SKILL.md +78 -23
- package/skills/api-mocking/SKILL.md +117 -135
- package/skills/api-monitoring/SKILL.md +52 -129
- package/skills/api-simulation/SKILL.md +138 -0
- package/skills/api-simulation/reference/sim_yaml_schema.md +50 -0
- package/skills/api-testing/SKILL.md +48 -95
- package/skills/bootstrap/SKILL.md +123 -210
- package/skills/bootstrap/reference/cli_installation.md +4 -3
- package/skills/ci-integration/SKILL.md +53 -112
- package/skills/collection-schema-v3/SKILL.md +43 -204
- package/skills/collection-schema-v3/reference/http.md +150 -0
- package/skills/datasets/SKILL.md +89 -314
- package/skills/datasets/reference/spreadsheets.md +25 -0
- package/skills/flows/SKILL.md +57 -206
- package/skills/flows/reference/flow_cli_flags.md +3 -2
- package/skills/performance-testing/SKILL.md +46 -62
- package/skills/performance-testing/references/jmeter-import/README.md +113 -0
- package/skills/performance-testing/references/jmeter-import/import-report.md +52 -0
- package/skills/performance-testing/references/jmeter-import/mapping-table.md +97 -0
- package/skills/performance-testing/references/jmeter-import/postman-perf-capabilities.md +56 -0
- package/skills/performance-testing/references/jmeter-import/script-translation.md +98 -0
- package/skills/performance-testing/references/jmeter-import/scripts/groovy_to_js.py +83 -0
- package/skills/performance-testing/references/jmeter-import/scripts/jmx_to_postman.py +1433 -0
- package/skills/postman-mcp-server/SKILL.md +53 -59
- package/skills/postman-mcp-server/references/docs.md +0 -1
- package/skills/postman-mcp-server/references/learn.md +10 -9
- package/skills/postman-mcp-server/references/mock.md +0 -1
- package/skills/postman-mcp-server/references/search.md +0 -1
- package/skills/postman-mcp-server/references/security.md +0 -1
- package/skills/postman-mcp-server/references/setup.md +0 -1
- package/skills/postman-mcp-server/references/sync.md +0 -1
- package/skills/postman-mcp-server/references/test.md +0 -1
- package/skills/variables/SKILL.md +63 -0
- package/skills/webhook-testing/SKILL.md +103 -0
- package/skills/webhook-testing/reference/forwarding.md +10 -0
- package/skills/webhook-testing/reference/outbound_webhooks.md +25 -0
- package/skills/webhook-testing/reference/webhook_scripts.md +163 -0
package/skills/flows/SKILL.md
CHANGED
|
@@ -1,212 +1,63 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: flows
|
|
3
|
-
description:
|
|
3
|
+
description: Use when running, deploying, or debugging an existing Postman Flow. Distinguishes local graph execution from a deployed webhook, reuses named input scenarios, and follows a run to the failing block rather than treating an HTTP response as whole-flow success. Use for requests such as "why did this flow fail".
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# Postman Flows
|
|
7
7
|
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
`
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
`
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
`--path` is a suffix appended to a generated base URL, not a full URL.
|
|
65
|
-
|
|
66
|
-
### Inputs: `-i` versus a scenario
|
|
67
|
-
|
|
68
|
-
A scenario is a named input set stored **in the flow definition**, generated
|
|
69
|
-
when someone adds an input to the Start block. Because it travels with the
|
|
70
|
-
flow, `-s "Staging"` is reproducible across invocations and across people,
|
|
71
|
-
where `-i key=value` is per-invocation. Start-block inputs can be declared
|
|
72
|
-
secret, which is what `--show-secrets` unmasks in dry-run output.
|
|
73
|
-
|
|
74
|
-
Precedence: `-s` supplies payload, headers, and query; `--headers` and
|
|
75
|
-
`--query` override it; `-i`/`-f` override its values.
|
|
76
|
-
|
|
77
|
-
### Identifiers
|
|
78
|
-
|
|
79
|
-
`flows list` is the only way to turn a flow name into an id —
|
|
80
|
-
`.postman/resources.yaml` maps collections to cloud ids but has no flows
|
|
81
|
-
section, so there is nothing local to read. Both `list` and `list-runs`
|
|
82
|
-
**require** `-w/--workspace`; take that id from `workspace.id` in
|
|
83
|
-
`.postman/resources.yaml`, which `bootstrap` records.
|
|
84
|
-
|
|
85
|
-
Run IDs have no single documented shape (`session-abc123` and `main/1a123ab1`
|
|
86
|
-
both appear in Postman's own material). Use whatever `trigger` or `list-runs`
|
|
87
|
-
printed, verbatim, and apply the same rule to flow ids.
|
|
88
|
-
|
|
89
|
-
### Plan and permission gating
|
|
90
|
-
|
|
91
|
-
`flows run` is documented as Enterprise-only, and the cloud subcommands need
|
|
92
|
-
`postman login`. `Access denied. Please check your permissions for the
|
|
93
|
-
specified resource.` on *every* workspace is a credential-scope or plan
|
|
94
|
-
signal, not a wrong-workspace signal — and never means the workspace has no
|
|
95
|
-
flows.
|
|
96
|
-
|
|
97
|
-
## Deploying: propose, confirm, then verify
|
|
98
|
-
|
|
99
|
-
Deploy is the one multi-phase workflow here, because it is mutating and
|
|
100
|
-
because its result is only half-useful without the follow-up check.
|
|
101
|
-
|
|
102
|
-
1. **Resolve the id.** `flows list --workspace <id> --filter "Checkout"`. On
|
|
103
|
-
multiple matches, show name + id + last-updated and let the user pick.
|
|
104
|
-
2. **Propose the path.** Derive it from the flow name — "Checkout" →
|
|
105
|
-
`/checkout` — so the user is confirming a concrete value rather than
|
|
106
|
-
answering an open question. Raise `--auth` here if the trigger will be
|
|
107
|
-
reachable by anyone who learns the URL.
|
|
108
|
-
3. **Confirm, then run** `flows deploy <flowId> --path /checkout`.
|
|
109
|
-
4. **Report the Trigger URL and whether the trigger is enabled.** A deploy can
|
|
110
|
-
land with the trigger off, which looks identical to a broken deploy at call
|
|
111
|
-
time. If it is off, offer `flows update <flowId> --trigger on`.
|
|
112
|
-
|
|
113
|
-
When the deploy existed only so the flow could be run, trigger it in the same
|
|
114
|
-
turn and report the Run ID — deploy-then-trigger is one job.
|
|
115
|
-
|
|
116
|
-
## Running and triggering
|
|
117
|
-
|
|
118
|
-
Show the command before running it, and map the request onto flags: inputs to
|
|
119
|
-
`-i`, a payload file to `-f`, query to `-q`, headers to `--headers`, a named
|
|
120
|
-
scenario to `-s`.
|
|
121
|
-
|
|
122
|
-
```bash
|
|
123
|
-
postman flows run postman/flows/checkout.json -i amount=4200
|
|
124
|
-
postman flows trigger <flowId> -i amount=4200
|
|
125
|
-
```
|
|
126
|
-
|
|
127
|
-
`run` is documented as Enterprise-only, so check the plan before building a
|
|
128
|
-
workflow on the local path. Where the flow is already in Cloud View, `trigger`
|
|
129
|
-
covers the gap; a Local View flow has no such fallback, since it cannot be
|
|
130
|
-
deployed.
|
|
131
|
-
|
|
132
|
-
`-n/--dry-run` on `trigger` prints the resolved URL and payload without
|
|
133
|
-
sending — worth reaching for when a flow writes to real systems, since a
|
|
134
|
-
trigger is not a read-only probe. For CI, `--output json` and `--reporters
|
|
135
|
-
html` persist results, and `--workspace` is required if the flow contains
|
|
136
|
-
connector blocks (it fails at the block, not at startup).
|
|
137
|
-
|
|
138
|
-
Report the Run ID on every trigger, including successes; it is the only handle
|
|
139
|
-
on the run afterwards.
|
|
140
|
-
|
|
141
|
-
Two failures are recoverable rather than terminal, and both recover through a
|
|
142
|
-
confirmed mutation. A 404 hinting `To deploy it, run: postman flows deploy`
|
|
143
|
-
means the flow exists but was never deployed — offer the deploy above, then
|
|
144
|
-
re-trigger. A disabled-trigger error means it is deployed but not accepting
|
|
145
|
-
calls — offer `flows update <flowId> --trigger on`, then trigger.
|
|
146
|
-
|
|
147
|
-
## Debugging a run
|
|
148
|
-
|
|
149
|
-
A trigger's response body is the Response block's output. A flow can answer
|
|
150
|
-
200 with a failed block upstream, and a 500 says nothing about which block
|
|
151
|
-
produced it — so read the run, not the response.
|
|
152
|
-
|
|
153
|
-
```bash
|
|
154
|
-
postman flows list-runs --workspace <id> --flow <flowId> --range 3d
|
|
155
|
-
postman flows get-run --run-id <runId> --logs
|
|
156
|
-
```
|
|
157
|
-
|
|
158
|
-
`list-runs` recovers a Run ID nobody wrote down; its `--range` defaults to
|
|
159
|
-
`1h`, so widen it before concluding a run is missing. Start `get-run` without
|
|
160
|
-
`--logs` and add them when the summary does not explain the failure;
|
|
161
|
-
`--filter` narrows to a block-id prefix.
|
|
162
|
-
|
|
163
|
-
Report the failing block, the reason, and the run status:
|
|
164
|
-
|
|
165
|
-
```
|
|
166
|
-
Run session-abc123 — failed
|
|
167
|
-
Failing block: "HTTP Request (Get Orders)"
|
|
168
|
-
Reason: downstream returned 504 after 10s timeout
|
|
169
|
-
Status: error
|
|
170
|
-
```
|
|
171
|
-
|
|
172
|
-
## Critical Rules
|
|
173
|
-
|
|
174
|
-
1. **Resolve ids, never infer them.** A name is not an id, and no id format is
|
|
175
|
-
documented well enough to validate against. Ambiguous name → present
|
|
176
|
-
candidates and ask.
|
|
177
|
-
2. **`deploy` and `update` need explicit confirmation.** They change what the
|
|
178
|
-
flow does for every caller: a deploy exposes a trigger path, `--trigger
|
|
179
|
-
on|off` starts or stops accepting calls, and `--auth off` removes
|
|
180
|
-
authentication from a live trigger.
|
|
181
|
-
3. **Report the failing block, not the log.** `--logs` output is input to your
|
|
182
|
-
analysis; the user needs the block, the reason, and the status.
|
|
183
|
-
4. **Surface CLI errors verbatim** and read them literally. "Flow file not
|
|
184
|
-
found", a required-option error, and "Access denied" have three different
|
|
185
|
-
fixes, and only the last is about permissions.
|
|
186
|
-
5. **A missing or unauthenticated CLI is `bootstrap`'s job** — route there
|
|
187
|
-
rather than improvising an install or a second login.
|
|
188
|
-
|
|
189
|
-
## Anti-patterns
|
|
190
|
-
|
|
191
|
-
1. **Don't substitute `run` for `trigger` when a flow isn't deployed.** A
|
|
192
|
-
green local run says nothing about the deployed path a caller hits, and a
|
|
193
|
-
Local View flow cannot be deployed at all.
|
|
194
|
-
2. **Don't hunt for a different workspace id when access is denied across
|
|
195
|
-
every workspace you try.** A blanket denial points at the credential's
|
|
196
|
-
scope or the plan, not at the id.
|
|
197
|
-
3. **Don't pass `-x/--suppress-exit-code` in CI.** It makes a failed flow
|
|
198
|
-
report success to the pipeline, which removes the only thing gating it.
|
|
199
|
-
4. **Don't put reusable inputs on the command line.** A payload that matters
|
|
200
|
-
more than once belongs in a Start-block scenario, where it travels with the
|
|
201
|
-
flow.
|
|
202
|
-
|
|
203
|
-
## Reference
|
|
204
|
-
|
|
205
|
-
- [Flows CLI flags](reference/flow_cli_flags.md) — full flag tables per
|
|
206
|
-
subcommand, the BETA dataset-iteration flags, and the short-flag collisions
|
|
207
|
-
between subcommands. Read before composing a command with flags not shown
|
|
208
|
-
above.
|
|
209
|
-
- `bootstrap` skill — CLI install, login, and the workspace id these commands
|
|
210
|
-
require.
|
|
211
|
-
- `api-discovery` skill — `postman search flows` finds a flow by text across
|
|
212
|
-
Postman, a different dataset from `flows list`'s workspace enumeration.
|
|
8
|
+
A Flow saves an API workflow as a graph of blocks. Named inputs make runs
|
|
9
|
+
repeatable, deployment makes the workflow callable by other systems, and
|
|
10
|
+
block-level run evidence shows where it failed. These benefits come from
|
|
11
|
+
different execution paths; choosing the wrong path can produce a passing
|
|
12
|
+
result that says nothing about the caller's actual workflow.
|
|
13
|
+
|
|
14
|
+
1. Choose the artifact the user means. `postman flows run <path>` executes a
|
|
15
|
+
local JSON file; `postman flows trigger <flow-id>` calls the deployed
|
|
16
|
+
webhook, including trigger auth and configuration. Local View and Cloud
|
|
17
|
+
View artifacts are isolated: local files cannot be assumed deployable,
|
|
18
|
+
and `workspace pull` does not download them. The desktop app's Git-connected
|
|
19
|
+
Local View produces those files. Check current local-run entitlement before
|
|
20
|
+
building CI around it; a deployed flow is not a substitute for a different
|
|
21
|
+
local artifact.
|
|
22
|
+
2. Resolve hosted IDs through `flows list --workspace <id>`; there is no
|
|
23
|
+
local flow mapping in `.postman/resources.yaml`. Reuse its `workspace.id`
|
|
24
|
+
when applicable or use **api-discovery** for cross-workspace search.
|
|
25
|
+
Resolve ambiguous names before mutating a flow. Use returned flow/run IDs
|
|
26
|
+
verbatim; their spelling is not a reliable validation rule.
|
|
27
|
+
3. Reuse a Start-block scenario when the same payload should travel with the
|
|
28
|
+
flow. `-s` supplies payload, headers, and query; explicit headers/query and
|
|
29
|
+
`-i`/`-f` override those values. One-off inputs belong on the invocation.
|
|
30
|
+
For deployed flows with side effects, `trigger --dry-run` lets you inspect
|
|
31
|
+
the resolved call first; keep secrets masked.
|
|
32
|
+
4. Deploy when the workflow needs a callable URL. The hosted flow needs a
|
|
33
|
+
Start block configured for an API request trigger and a Response block;
|
|
34
|
+
changing CLI flags cannot repair missing canvas configuration. Resolve
|
|
35
|
+
the path suffix and intended authentication before deploying. Deploy/auth
|
|
36
|
+
changes affect other callers; a scoped user request authorizes that change,
|
|
37
|
+
while unrequested public exposure or auth removal needs consent.
|
|
38
|
+
5. Inspect the deployed URL and whether its trigger is enabled. A deployed
|
|
39
|
+
but disabled flow is not ready to receive calls; `flows update <id>
|
|
40
|
+
--trigger on` enables it when intended. If deployment is part of a requested
|
|
41
|
+
run, continue through triggering and capture the Run ID. A request only to
|
|
42
|
+
deploy does not authorize executing unrelated downstream writes.
|
|
43
|
+
6. Diagnose from run evidence. A trigger response is the Response block's
|
|
44
|
+
output: HTTP 200 can coexist with another failed block. Use
|
|
45
|
+
`flows get-run --run-id <id>` and add `--logs` or a block filter when the
|
|
46
|
+
summary does not explain it. `flows list-runs --workspace <id> --flow <id>`
|
|
47
|
+
recovers missing run IDs; widen its time range before concluding absence.
|
|
48
|
+
7. In CI, preserve the local run's failure exit code and persist its result
|
|
49
|
+
and reports. Connector blocks need `--workspace` even for a local file;
|
|
50
|
+
otherwise failure occurs inside the graph. An access denial across several
|
|
51
|
+
workspaces points toward credentials or entitlement, not an empty flow list.
|
|
52
|
+
Use **bootstrap** for unresolved CLI/authentication setup.
|
|
53
|
+
|
|
54
|
+
Read [command options](reference/flow_cli_flags.md) for inputs, reporters,
|
|
55
|
+
dataset iteration, and flag collisions; check installed subcommand `-h` before
|
|
56
|
+
carrying an option between commands.
|
|
57
|
+
|
|
58
|
+
## Verification
|
|
59
|
+
|
|
60
|
+
Report which artifact and path executed, its inputs/scenario without secrets,
|
|
61
|
+
and the actual result. For a trigger, retain the Run ID; for a failure, name
|
|
62
|
+
the failing block and cause. For a deployment, report URL, auth, and enabled
|
|
63
|
+
state separately from any execution result.
|
|
@@ -57,8 +57,9 @@ takes `on|off`.
|
|
|
57
57
|
with `Invalid command parameters: trigger: At least one option is required:
|
|
58
58
|
--trigger or --auth`.
|
|
59
59
|
|
|
60
|
-
Both
|
|
61
|
-
|
|
60
|
+
Both change the live trigger for other callers. Apply an already authorized
|
|
61
|
+
change within its scope; obtain consent before an unrequested access change.
|
|
62
|
+
`--auth off` removes authentication, so it is not a remedy for a failed login.
|
|
62
63
|
|
|
63
64
|
## `flows run <path>`
|
|
64
65
|
|
|
@@ -1,71 +1,55 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: performance-testing
|
|
3
|
-
description:
|
|
3
|
+
description: Use when load-testing or benchmarking an API with concurrent virtual users and latency or error thresholds. Reuses a collection to probe a chosen failure mode and distinguishes target limits from load-generator limits. Use for requests such as "load test this API".
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# Performance Testing
|
|
7
7
|
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
1. **Running this against a real, non-mock backend generates real load with
|
|
50
|
-
real consequences — confirm the target, VU count, and duration with the
|
|
51
|
-
user before running,** the same way any action with effects outside this
|
|
52
|
-
session gets confirmed. Default to a low `--vu-count` and short
|
|
53
|
-
`--duration` for a first run against anything live, or point it at a mock
|
|
54
|
-
(`--use-mock`) when the goal is testing the collection, not the backend.
|
|
55
|
-
2. **A `--pass-if` gate doesn't stop the load early.** The full VU count and
|
|
56
|
-
duration run regardless of whether the threshold will ultimately pass —
|
|
57
|
-
plan for that cost, don't assume a failing gate means less traffic was
|
|
58
|
-
sent.
|
|
59
|
-
3. **Cloud runners come from Postman's IP ranges.** Before assuming a
|
|
60
|
-
`postman-cloud` run will reach a target, check whether it's IP-allowlisted
|
|
61
|
-
— use `postman-cloud-static-ip` if so, rather than discovering the
|
|
62
|
-
mismatch as a wall of connection failures.
|
|
8
|
+
A collection supplies realistic request sequences; `postman performance run`
|
|
9
|
+
adds concurrency and a load profile. This makes performance claims repeatable,
|
|
10
|
+
but only for the workload, data, and network path actually exercised.
|
|
11
|
+
Collections run directly: converting with `postman-to-k6` is not required.
|
|
12
|
+
|
|
13
|
+
1. Choose the failure mode before the profile: `fixed` tests sustained
|
|
14
|
+
concurrency, `ramp-up` explores degradation as load grows, `spike` tests
|
|
15
|
+
sudden surges, and `peak` tests staying near maximum load. Reuse the
|
|
16
|
+
collection's auth and assertions, and check that its operations and data
|
|
17
|
+
represent the workload rather than repeating one unusually cheap request.
|
|
18
|
+
2. Establish the target, VU count, duration, and effects of write requests
|
|
19
|
+
before sending load. Existing user authorization for that scope is enough;
|
|
20
|
+
resolve missing limits before stressing a shared or production service.
|
|
21
|
+
Thresholds are evaluated after the run, so a failing gate will not stop
|
|
22
|
+
traffic early. A small initial run can expose wiring failures before a
|
|
23
|
+
larger authorized test.
|
|
24
|
+
3. Choose the runner by reachability and capacity. `local` uses this machine
|
|
25
|
+
or CI runner; its CPU or network can become the bottleneck. `postman-cloud`
|
|
26
|
+
moves generation to Postman's infrastructure, while
|
|
27
|
+
`postman-cloud-static-ip` supports targets with IP allowlists. Verify
|
|
28
|
+
reachability before interpreting connection failures as service capacity.
|
|
29
|
+
4. Set thresholds that represent the user's objective, such as
|
|
30
|
+
`--pass-if "less_than(p95, 500)"`. Supported metrics include `avg`, `p90`,
|
|
31
|
+
`p95`, `p99`, `error_rate`, and `rps`; latency alone can look good when
|
|
32
|
+
requests fail quickly. Use `postman performance run -h` for current syntax,
|
|
33
|
+
units, and runner-specific options.
|
|
34
|
+
5. Isolate test data where concurrent writes would collide. **datasets**
|
|
35
|
+
supports controlled row distribution across VUs; setup/teardown collections
|
|
36
|
+
on hosted runners execute once around the test, not once per iteration.
|
|
37
|
+
`--use-mock` can baseline collection/script overhead, but its numbers
|
|
38
|
+
cannot establish the real backend's capacity.
|
|
39
|
+
|
|
40
|
+
## Importing from JMeter
|
|
41
|
+
|
|
42
|
+
Have a JMeter `.jmx` instead of a collection? Don't rebuild it by hand — run the
|
|
43
|
+
bundled importer (`references/jmeter-import/scripts/jmx_to_postman.py <plan>.jmx
|
|
44
|
+
-o <out>`; see `references/jmeter-import/README.md`), then treat its output as the
|
|
45
|
+
collection above. Read the generated report: it marks what mapped exactly, what
|
|
46
|
+
was approximated, and which JMeter features have no equivalent here — think-time
|
|
47
|
+
timers, concurrent thread groups, and non-HTTP (JDBC/FTP/LDAP/Java) samplers are
|
|
48
|
+
reported, never faked.
|
|
63
49
|
|
|
64
50
|
## Verification
|
|
65
51
|
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
determines whether the numbers reflect the target's real-world reachability
|
|
71
|
-
or only local-network conditions.
|
|
52
|
+
Report the workload, runner, achieved traffic, latency and error metrics,
|
|
53
|
+
and threshold verdict. Identify generator limits or target errors that prevent
|
|
54
|
+
attributing the result to API capacity. See **api-testing** for functional
|
|
55
|
+
assertions and **ci-integration** when the measured threshold becomes a gate.
|
|
@@ -0,0 +1,113 @@
|
|
|
1
|
+
|
|
2
|
+
# Importing a JMeter `.jmx` into Postman
|
|
3
|
+
|
|
4
|
+
> Reference for the `performance-testing` skill. Run the importer, review the report, then the result is an ordinary performance run.
|
|
5
|
+
|
|
6
|
+
Turn a JMeter test plan into three artifacts:
|
|
7
|
+
|
|
8
|
+
1. **Postman Collection v2.1** — requests, flow, headers, auth, variables, extractors, assertions.
|
|
9
|
+
2. **Performance config** — VU count, duration, load profile, data file, `--pass-if`, and the exact `postman performance run` command.
|
|
10
|
+
3. **Import report** — what imported exactly, what was approximated, what needs manual work, what is unsupported.
|
|
11
|
+
|
|
12
|
+
The parsing and building are **deterministic Python** (`scripts/`). You (the model) only do the judgment calls the scripts flag: translating Groovy/BeanShell scripts to JavaScript and resolving extractors Python can't (XPath/CSS/complex JSONPath).
|
|
13
|
+
|
|
14
|
+
## When NOT to guess
|
|
15
|
+
|
|
16
|
+
Postman performance runs do not support everything JMeter does. The scripts already encode the gaps (see `postman-perf-capabilities.md`). Never invent a Postman feature to cover a JMeter one. If something can't import, it goes in the report's "Needs manual work" or "Not supported" section — that is a correct outcome, not a failure.
|
|
17
|
+
|
|
18
|
+
## Correctness rules the importer reproduces (do not regress)
|
|
19
|
+
|
|
20
|
+
These are JMeter semantics that are easy to get subtly wrong. They're covered by `tests/score.py`:
|
|
21
|
+
|
|
22
|
+
- **Implicit HTTP success** — JMeter fails a sample on HTTP ≥ 400 even with no assertion. Each request gets a default `pm.test` asserting the code is 200–399, **unless** the plan already checks the status code or sets *assume_success* (Ignore Status).
|
|
23
|
+
- **Port 0 / -1** → "use protocol default" → omitted from the URL.
|
|
24
|
+
- **Full-URL path** — if `HTTPSampler.path` is itself an absolute URL, it's used verbatim; domain/port are ignored.
|
|
25
|
+
- **HEAD / TRACE arguments** — JMeter does not send separate parameters as a query for these; they're dropped (reported).
|
|
26
|
+
- **Duplicate headers** — in-scope header managers merge; the nearer/last value per name wins and the collapse is reported.
|
|
27
|
+
- **Sampler-scoped config** — a Header Manager / Auth Manager / HTTP Defaults inside a request's own subtree applies to that request.
|
|
28
|
+
- **Security** — the parser refuses DTD/ENTITY (XXE) and caps input at 25 MiB.
|
|
29
|
+
|
|
30
|
+
## Workflow
|
|
31
|
+
|
|
32
|
+
### 1. Run the importer
|
|
33
|
+
|
|
34
|
+
```bash
|
|
35
|
+
python3 scripts/jmx_to_postman.py INPUT.jmx -o OUTDIR --json-report
|
|
36
|
+
# CI gate: add --fail-on-blockers to exit non-zero when any item needs manual work
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
Produces in `OUTDIR`:
|
|
40
|
+
- `<name>.postman_collection.json` — the collection (import-ready)
|
|
41
|
+
- `<name>.perf.md` / `<name>.perf.json` — performance config + CLI command
|
|
42
|
+
- `<name>.report.md` — the import report
|
|
43
|
+
- `<name>.ir.json` — machine-readable intermediate model + `manual_tasks` + `warnings` (from `--json-report`)
|
|
44
|
+
|
|
45
|
+
Python 3.8+ only, standard library — no pip installs.
|
|
46
|
+
|
|
47
|
+
### 2. Resolve the manual tasks
|
|
48
|
+
|
|
49
|
+
Open `<name>.ir.json` and read `manual_tasks`. Each has `request`, `kind`, `detail`. Handle by kind:
|
|
50
|
+
|
|
51
|
+
- **`script`** — a JSR223/BeanShell pre/post-processor was auto-translated to JS and wrapped in an IIFE. Open that request's `event` script in the collection, read the `// REVIEW` block, and correct the translation. Use `script-translation.md` for the mapping. Common fixes: `prev`/`ctx`/`sampler` accessors, `.each{}` loops, `import`/`new` statements.
|
|
52
|
+
- **`script_sampler`** — a standalone JSR223/BeanShell *sampler* (no HTTP). The stub is a GET to an empty URL carrying the translated script. Decide with the user: delete it, fold its logic into an adjacent request's pre/post script, or replace with a real request.
|
|
53
|
+
- **`xpath` / `css`** — the Postman sandbox has no built-in XPath/CSS selector engine. Rewrite the extractor against the JSON body if the response is JSON, or use `xml2js`/`cheerio` via `require(...)` in the script (confirm availability), or drop it with a note.
|
|
54
|
+
- **`jsonpath` too complex** — the path used wildcards/filters/recursion. Rewrite it as explicit property access in the request's test script.
|
|
55
|
+
|
|
56
|
+
Edit the collection JSON in place. Keep the `// REVIEW` comments until the logic is confirmed.
|
|
57
|
+
|
|
58
|
+
### 3. Validate the output
|
|
59
|
+
|
|
60
|
+
Confirm the collection is valid v2.1 JSON and — the part that matters — that **every generated `event` script parses as JavaScript** (`node --check` each `exec`, or equivalent). A syntax error in a generated script is invisible in the file and only surfaces when the perf run executes that request, mid-load. Fix any bad script before handing off.
|
|
61
|
+
|
|
62
|
+
### 4. Finalise the report
|
|
63
|
+
|
|
64
|
+
`<name>.report.md` already has the four sections. Add anything you changed while resolving manual tasks so the report matches the collection you're handing off.
|
|
65
|
+
|
|
66
|
+
### 5. Hand off the perf config
|
|
67
|
+
|
|
68
|
+
`<name>.perf.md`'s command has a `<COLLECTION_UID>` placeholder — the collection must be uploaded to a workspace first to get a real UID. The `--pass-if` value is a suggestion unless the JMeter plan had a Duration Assertion, so surface the VU/duration/profile/pass-if choices for the user to adjust.
|
|
69
|
+
|
|
70
|
+
### 6. (optional) Publish to a Postman workspace
|
|
71
|
+
|
|
72
|
+
Only on explicit user consent — and **never make the user paste a workspace ID**. Pick the destination like this:
|
|
73
|
+
|
|
74
|
+
- **If the current folder is already a connected workspace** (the usual headless case), use it — `postman workspace push` targets the workspace bound to this directory, no ID. Just tell the user which one.
|
|
75
|
+
- **If the user named a workspace** (by name), resolve it with `postman workspace list --json` and match.
|
|
76
|
+
- **Otherwise present a choice, don't demand an ID.** Run `postman workspace list --json`, show the **names**, and ask the user to pick one *or* create a new one:
|
|
77
|
+
- **pick existing** → connect it once with `postman workspace pull <id>` (ID taken from the list), then push.
|
|
78
|
+
- **create new** → the CLI has no `create`; make it via MCP `createWorkspace` (or the Postman app), then `pull` and push.
|
|
79
|
+
- **Get the imported collection into the bound workspace before pushing.** The importer emits a standalone v2.1 `*.postman_collection.json`; a pulled workspace is a git-native v3 tree. After `pull`, bring the file in with `postman collection migrate <file>.postman_collection.json` into the workspace's collections directory (check `postman collection migrate -h`), then `postman workspace push`. Pushing a folder that has no collection in its tree uploads nothing.
|
|
80
|
+
- **Push through the CLI, not MCP.** MCP `createCollection` drops each request's URL (requests land blank); use MCP only if the `postman` binary genuinely can't be installed, and repair URLs afterward.
|
|
81
|
+
- Interrogate `postman <cmd> -h` for exact flags — the help is authoritative, this file is not.
|
|
82
|
+
|
|
83
|
+
Smoke-test with `postman collection run` — the **same runtime that executes the perf test**, not a generic client — so the request is proven under the variable/auth resolution it will actually run under. Then wire the returned UID into the perf command.
|
|
84
|
+
|
|
85
|
+
## Fallback: when you cannot run Python
|
|
86
|
+
|
|
87
|
+
If this environment has no code execution, do **not** refuse — import by hand, following the exact same rules the script encodes. Output must be identical in shape (collection v2.1 + perf config + report).
|
|
88
|
+
|
|
89
|
+
1. **Read first:** `mapping-table.md` (what maps to what), `postman-perf-capabilities.md` (the hard limits), `script-translation.md` (Groovy/BeanShell → JS). These ARE the algorithm.
|
|
90
|
+
2. **Parse the JMX by the one structural rule:** every element is followed by a sibling `<hashTree>` holding its children. Walk those pairs. A config element (Header Manager, Auth, HTTP Defaults, extractor, assertion) applies to every sampler in its subtree; the same element inside a sampler's own hashTree applies only to that sampler.
|
|
91
|
+
3. **Apply the non-negotiable rules** (same as the script). Use the **exact JS templates in `script-translation.md`** for extractors and assertions so a hand import matches the script output byte-for-byte:
|
|
92
|
+
- `${var}` → `{{var}}`; `${__UUID}`→`{{$guid}}`, `${__time}`→`{{$timestamp}}`, `${__Random(..)}`→`{{$randomInt}}`, `${__RandomString(..)}`→`{{$randomAlphaNumeric}}`.
|
|
93
|
+
- Every request gets a default `pm.test` asserting HTTP 200–399 — **unless** that request has a Response Assertion whose `test_field` is `response_code` **or** `response_message`, **or** any assertion with `assume_success=true`. An assertion on the **body** (`response_data`) does NOT suppress it.
|
|
94
|
+
- Port 0/-1 → omit. Full-URL path (`path` starts with `http://`/`https://`) → use verbatim, ignore domain/port. HEAD/TRACE args → drop + note. Keep inherited headers as-is even on a GET (JMeter still sends them); don't prune.
|
|
95
|
+
- Extractors → `pm.collectionVariables.set` in the test script; assertions → `pm.test`. (Templates in the reference.)
|
|
96
|
+
- Thread group → VUs = `num_threads`; **duration (minutes) = round(`ThreadGroup.duration` seconds ÷ 60), minimum 1**; profile = `ramp-up` if `ramp_time`>0 or a plugin/concurrency group, else `fixed`. Loop counts → duration (there is no iteration count). One `--pass-if` only; a Duration Assertion → `less_than(p95, <ms>)`, else default `less_than(error_rate, 5)`.
|
|
97
|
+
- Timers / concurrent thread groups / non-HTTP samplers → **not supported**; list them in the report, never fake them.
|
|
98
|
+
4. **Produce the same files** a run would: the collection JSON, the perf `.md`, and the report with the four sections (imported / approximated / needs manual work / not supported).
|
|
99
|
+
5. **State clearly** in your answer that this was a by-hand import (no Python), so it's best-effort and not the deterministic, tested path.
|
|
100
|
+
|
|
101
|
+
This fallback is lower-reliability than running the script — flag anything you were unsure about.
|
|
102
|
+
|
|
103
|
+
## Mapping
|
|
104
|
+
|
|
105
|
+
The full per-element mapping (fidelity + whether the script or the model owns it) is `mapping-table.md` — it's the algorithm for the fallback and the regression contract, so it's kept there, not restated here.
|
|
106
|
+
|
|
107
|
+
## Scope limits (hard facts, verified against the CLI/engine code)
|
|
108
|
+
|
|
109
|
+
- Performance runs execute **one collection repeated across VUs**. Multiple thread groups cannot run concurrently in a single run.
|
|
110
|
+
- Runs are **time-based**, not iteration-based. JMeter loop counts become a duration.
|
|
111
|
+
- `--pass-if` is a **single, non-repeatable** condition. Extra assertions stay as `pm.test` in scripts.
|
|
112
|
+
- **No native think time.** JMeter timers are approximated — group-scope ones as `--delay-request`, per-request ones as a pre-request pause — not reproduced exactly. Always reported.
|
|
113
|
+
- Only **HTTP(S)** samplers import. JDBC/FTP/LDAP/Java/TCP/Debug/Dummy samplers are reported as skipped.
|
|
@@ -0,0 +1,52 @@
|
|
|
1
|
+
# Import Report — <TEST PLAN NAME>
|
|
2
|
+
|
|
3
|
+
> Generated by `jmx_to_postman.py`; sections below are auto-filled. After resolving
|
|
4
|
+
> manual tasks, update the "Needs manual work" list and the summary counts.
|
|
5
|
+
|
|
6
|
+
## Summary
|
|
7
|
+
|
|
8
|
+
- Thread groups: <N>
|
|
9
|
+
- HTTP requests imported: <N>
|
|
10
|
+
- Folders: <N>
|
|
11
|
+
- Script-only samplers (stubs): <N>
|
|
12
|
+
- Collection variables: <N>
|
|
13
|
+
- Data files: <N>
|
|
14
|
+
- Items needing manual work: <N>
|
|
15
|
+
|
|
16
|
+
## Imported exactly
|
|
17
|
+
|
|
18
|
+
- HTTP requests (method, URL, headers, query/body params)
|
|
19
|
+
- Response assertions → `pm.test` (status / body contains / equals / matches)
|
|
20
|
+
- Regex & Boundary extractors → `pm.collectionVariables.set`
|
|
21
|
+
- Simple JSONPath extractors (dot/index paths) → property access
|
|
22
|
+
- Basic/Digest auth, User Defined Variables → collection variables
|
|
23
|
+
- `${var}` → `{{var}}` interpolation
|
|
24
|
+
|
|
25
|
+
## Approximated
|
|
26
|
+
|
|
27
|
+
- **Thread group load** → VU count + duration + one of 4 load profiles.
|
|
28
|
+
- **Loop counts / iterations** → duration in minutes.
|
|
29
|
+
- <element-specific warnings…>
|
|
30
|
+
|
|
31
|
+
## Needs manual work
|
|
32
|
+
|
|
33
|
+
- **[kind]** `request`: <what and why>
|
|
34
|
+
|
|
35
|
+
## Not supported by Postman performance runs (dropped)
|
|
36
|
+
|
|
37
|
+
- **Think time / timers** — no equivalent.
|
|
38
|
+
- **Multiple concurrent thread groups** — a run is one collection repeated across VUs.
|
|
39
|
+
- **Flow control** (If/While) — requests flattened; restore with `pm.execution.setNextRequest`/`skipRequest`.
|
|
40
|
+
- **Cookie/Cache/DNS managers** — cookies handled automatically per VU.
|
|
41
|
+
- **Non-HTTP samplers** (JDBC/FTP/LDAP/Java/TCP/Debug) — HTTP only.
|
|
42
|
+
|
|
43
|
+
## Performance run
|
|
44
|
+
|
|
45
|
+
```bash
|
|
46
|
+
postman performance run <COLLECTION_UID> \
|
|
47
|
+
--vu-count <N> --duration <MIN> --load-profile <fixed|ramp-up|spike|peak> \
|
|
48
|
+
[--data-file <file>] [--pass-if "<fn>(<metric>, <value>)"]
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
Upload the collection to a workspace first to get its UID. `--pass-if` is a suggested
|
|
52
|
+
default unless the JMeter plan had a Duration Assertion.
|