@postman/postman-plugin 0.5.1 → 0.6.0-rc.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +2 -2
- package/hooks/session-start-context.md +1 -1
- package/mcp.pi.json +2 -2
- package/package.json +1 -1
- package/skills/api-discovery/SKILL.md +7 -2
- package/skills/api-documentation/SKILL.md +2 -3
- package/skills/api-engineer/SKILL.md +13 -15
- package/skills/api-mocking/SKILL.md +4 -2
- package/skills/api-monitoring/SKILL.md +35 -25
- package/skills/api-simulation/SKILL.md +3 -1
- package/skills/api-testing/SKILL.md +3 -3
- package/skills/application-testing/SKILL.md +2 -2
- package/skills/ci-integration/SKILL.md +7 -5
- package/skills/cli-install/SKILL.md +29 -0
- package/skills/collection-schema-v3/SKILL.md +21 -11
- package/skills/collection-schema-v3/reference/environment.md +2 -0
- package/skills/collection-schema-v3/reference/globals.md +40 -0
- package/skills/collection-schema-v3/reference/http.md +149 -11
- package/skills/collection-schema-v3/reference/other_protocols.md +232 -86
- package/skills/datasets/SKILL.md +6 -5
- package/skills/datasets/reference/schema.md +51 -0
- package/skills/flows/SKILL.md +0 -1
- package/skills/performance-testing/SKILL.md +14 -1
- package/skills/performance-testing/references/gatling-import/README.md +141 -0
- package/skills/performance-testing/references/gatling-import/feeders.md +31 -0
- package/skills/performance-testing/references/gatling-import/injection.md +88 -0
- package/skills/performance-testing/references/gatling-import/mapping.md +91 -0
- package/skills/performance-testing/references/gatling-import/unsupported.md +31 -0
- package/skills/postman-mcp-server/SKILL.md +1 -1
- package/skills/variables/SKILL.md +2 -2
- package/skills/webhook-testing/SKILL.md +1 -1
- package/skills/workspaces/SKILL.md +53 -0
- package/skills/bootstrap/SKILL.md +0 -140
- package/skills/bootstrap/reference/cli_installation.md +0 -59
|
@@ -1,86 +1,232 @@
|
|
|
1
|
-
#
|
|
2
|
-
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
-
|
|
11
|
-
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
1
|
+
# Other protocol schemas
|
|
2
|
+
|
|
3
|
+
Consult the section for the request kind being changed. Non-HTTP protocols have
|
|
4
|
+
different body, script, and settings shapes; structural validity does not establish
|
|
5
|
+
that the configured service or provider is available.
|
|
6
|
+
|
|
7
|
+
Shared [auth](http.md#auth), [body](http.md#request-body), and
|
|
8
|
+
[script](http.md#scripts) shapes are documented with HTTP where they apply.
|
|
9
|
+
|
|
10
|
+
- [Request kinds](#request-kinds): GraphQL, gRPC, WebSocket, Socket.IO, MQTT, MCP, LLM
|
|
11
|
+
- [Messages](#messages) and [examples](#examples)
|
|
12
|
+
|
|
13
|
+
## Request kinds
|
|
14
|
+
|
|
15
|
+
`$kind` is required on these request kinds, and it is the *only* required field except on
|
|
16
|
+
`mcp-request`, which also requires `transport`. So for every other kind a half-written
|
|
17
|
+
request can pass structural checks without being executable. Only `*.request.yaml`
|
|
18
|
+
files are discovered as requests; the right fields in a differently named file
|
|
19
|
+
do not make it runnable. Each kind has its own settings; HTTP settings do not
|
|
20
|
+
apply to every protocol. Common metadata fields `id`, `name`, `description` and `order`
|
|
21
|
+
are available on all of them and omitted below.
|
|
22
|
+
|
|
23
|
+
### `graphql-request`
|
|
24
|
+
|
|
25
|
+
`url`, `query`, `variables` (a **string** of JSON), `headers`, `auth`, `schema`,
|
|
26
|
+
`scripts`, `examples`, `settings`.
|
|
27
|
+
|
|
28
|
+
There is **no `queryParams`** — put query-string parameters in the `url`.
|
|
29
|
+
|
|
30
|
+
`settings`: `disabledSystemHeaders`.
|
|
31
|
+
|
|
32
|
+
`schema` is one shape chosen by `source`:
|
|
33
|
+
|
|
34
|
+
```yaml
|
|
35
|
+
schema: {source: none}
|
|
36
|
+
```
|
|
37
|
+
```yaml
|
|
38
|
+
schema: {source: auto}
|
|
39
|
+
```
|
|
40
|
+
```yaml
|
|
41
|
+
schema:
|
|
42
|
+
source: file
|
|
43
|
+
location: ./schema.graphql
|
|
44
|
+
```
|
|
45
|
+
```yaml
|
|
46
|
+
schema:
|
|
47
|
+
source: api
|
|
48
|
+
apiId: <id>
|
|
49
|
+
versionId: <id> # releaseId optional
|
|
50
|
+
```
|
|
51
|
+
```yaml
|
|
52
|
+
schema:
|
|
53
|
+
source: specification
|
|
54
|
+
specificationId: <id>
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
### `grpc-request`
|
|
58
|
+
|
|
59
|
+
`url`, `methodPath` (e.g. `helloworld.Greeter/SayHello`), `methodDescriptor` (an inline
|
|
60
|
+
proto definition), `message` (`{content}`), `metadata` (per-call headers as
|
|
61
|
+
`{key, value}` array entries), `auth`, `schema`, `scripts`, `examples`, `settings`.
|
|
62
|
+
|
|
63
|
+
`settings`: `secureConnection`, `strictSSL`, `serverNameOverride`,
|
|
64
|
+
`maxResponseMessageSize`, `includeDefaultFields`, `connectionTimeout`.
|
|
65
|
+
|
|
66
|
+
### `websocket-request`
|
|
67
|
+
|
|
68
|
+
`url`, `headers`, `queryParams`, `messages`, `examples`, `settings`.
|
|
69
|
+
|
|
70
|
+
`messages` and `examples` are **strings** — relative paths to directories, not arrays.
|
|
71
|
+
An array fails with `Invalid input: expected string, received array`.
|
|
72
|
+
|
|
73
|
+
`settings`: `handshakeTimeout`, `retryCount`, `maxPayload`, `retryDelay`, `strictSSL`.
|
|
74
|
+
|
|
75
|
+
### `socket.io-request`
|
|
76
|
+
|
|
77
|
+
`url`, `headers`, `queryParams`, `events`, `messages`, `examples`, `settings`.
|
|
78
|
+
|
|
79
|
+
```yaml
|
|
80
|
+
events:
|
|
81
|
+
- name: chat:message
|
|
82
|
+
subscribeOnConnect: true
|
|
83
|
+
description: Inbound chat traffic
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
`settings`: `version` (the **strings** `'2'`, `'3'`, `'4'`), `path`,
|
|
87
|
+
`handshakeTimeout`, `retryCount`, `retryDelay`, `strictSSL`. Bare `version: 4` is
|
|
88
|
+
rejected — `Invalid option: expected one of "2"|"3"|"4" (path: /settings/version)`.
|
|
89
|
+
|
|
90
|
+
### `mqtt-request`
|
|
91
|
+
|
|
92
|
+
`url`, `clientId`, `version`, `topics`, `lastWill`, `properties`, `auth`, `messages`,
|
|
93
|
+
`examples`, `settings`.
|
|
94
|
+
|
|
95
|
+
`version` is a **number**, either `4` or `5`, and it sits at the top level rather than
|
|
96
|
+
in `settings`. Do not quote it — that is the opposite of `socket.io-request`, whose
|
|
97
|
+
`settings.version` is the string `'2'`, `'3'` or `'4'`.
|
|
98
|
+
|
|
99
|
+
```yaml
|
|
100
|
+
topics:
|
|
101
|
+
- name: sensors/+/temp
|
|
102
|
+
qos: 1 # a number: 0, 1 or 2
|
|
103
|
+
subscribe: true
|
|
104
|
+
lastWill:
|
|
105
|
+
topic: status/client
|
|
106
|
+
payload: offline
|
|
107
|
+
type: text
|
|
108
|
+
qos: 0
|
|
109
|
+
retain: true
|
|
110
|
+
```
|
|
111
|
+
|
|
112
|
+
`properties`: `sessionExpiryInterval`, `receiveMaximum`, `maximumPacketSize`,
|
|
113
|
+
`requestResponseInformation`, `userProperties`.
|
|
114
|
+
|
|
115
|
+
`settings`: `cleanSession`, `keepAlive`, `autoReconnect`, `connectionTimeout`,
|
|
116
|
+
`strictSSL`.
|
|
117
|
+
|
|
118
|
+
### `mcp-request`
|
|
119
|
+
|
|
120
|
+
Requires `$kind` **and** `transport`, which selects the legal keys. Common to both:
|
|
121
|
+
`message`, `auth`.
|
|
122
|
+
|
|
123
|
+
| `transport` | Its keys | `settings` |
|
|
124
|
+
|---|---|---|
|
|
125
|
+
| `stdio` | `command`, `env` (`{key, value}` array entries) | `version`, `requestTimeout` |
|
|
126
|
+
| `sse` | `url`, `headers` | `version`, `requestTimeout`, `strictSSL`, `sessionTimeout` |
|
|
127
|
+
|
|
128
|
+
There are no other transports. `message` is a JSON-RPC envelope as a **string**;
|
|
129
|
+
`{{variables}}` interpolate inside it.
|
|
130
|
+
|
|
131
|
+
```yaml
|
|
132
|
+
$kind: mcp-request
|
|
133
|
+
transport: stdio # command + env; no url
|
|
134
|
+
command: uvx some-mcp-server
|
|
135
|
+
env:
|
|
136
|
+
- key: API_KEY
|
|
137
|
+
value: '{{KEY}}'
|
|
138
|
+
message: |-
|
|
139
|
+
{"method": "tools/call", "params": {"name": "do_thing", "arguments": {}}}
|
|
140
|
+
order: 1000
|
|
141
|
+
```
|
|
142
|
+
|
|
143
|
+
```yaml
|
|
144
|
+
$kind: mcp-request
|
|
145
|
+
transport: sse # url + headers; no command
|
|
146
|
+
url: https://mcp.example.com/sse
|
|
147
|
+
headers:
|
|
148
|
+
- key: Authorization
|
|
149
|
+
value: 'Bearer {{KEY}}'
|
|
150
|
+
message: |-
|
|
151
|
+
{"method": "tools/list"}
|
|
152
|
+
```
|
|
153
|
+
|
|
154
|
+
### `llm-request`
|
|
155
|
+
|
|
156
|
+
`url`, `config`, `userPrompts`, `systemPrompts`, `mcpConfig`, `enabledTools`,
|
|
157
|
+
`clientTools`, `auth`, `settings`.
|
|
158
|
+
|
|
159
|
+
`config` takes `model` and `provider`, both free-form strings. Schema validation
|
|
160
|
+
does not establish that the configured provider or model is available at run time.
|
|
161
|
+
|
|
162
|
+
Prompt entries are `{id, value, timestamp, active, contentType}`. The content
|
|
163
|
+
format is expressed by `contentType`; `type` is not a prompt-entry field.
|
|
164
|
+
`timestamp` is numeric, and `active` is boolean.
|
|
165
|
+
|
|
166
|
+
`mcpConfig` is a **string containing a JSON array**, not nested YAML and not
|
|
167
|
+
`mcp.json`'s `mcpServers` object. `clientTools`, for tools the model may call with no
|
|
168
|
+
MCP server behind them, is `[{id, tools: [{id}]}]`.
|
|
169
|
+
|
|
170
|
+
For example, a prompt and a stdio MCP tool configuration use different scalar
|
|
171
|
+
shapes within the same request:
|
|
172
|
+
|
|
173
|
+
```yaml
|
|
174
|
+
userPrompts:
|
|
175
|
+
- id: '1'
|
|
176
|
+
value: Summarise the last 10 orders
|
|
177
|
+
timestamp: 1758412800000
|
|
178
|
+
active: true
|
|
179
|
+
contentType: text
|
|
180
|
+
mcpConfig: |-
|
|
181
|
+
[
|
|
182
|
+
{
|
|
183
|
+
"name": "orders",
|
|
184
|
+
"config": {"command": "uvx", "args": ["orders-mcp-server"]},
|
|
185
|
+
"enabled": true,
|
|
186
|
+
"disabledTools": []
|
|
187
|
+
}
|
|
188
|
+
]
|
|
189
|
+
```
|
|
190
|
+
|
|
191
|
+
`settings`: `temperature`, `maxToken`, `maxSteps`, `topP`, `topK`, `seed`,
|
|
192
|
+
`presencePenalty`, `frequencyPenalty`, `repetitionPenalty`, `contextWindow`,
|
|
193
|
+
`streamResponse`, `streamTools`, `responseFormatJSON`. If a tool chain stops early,
|
|
194
|
+
distinguish a step limit from a tool or provider failure before changing `maxSteps`;
|
|
195
|
+
the limit also bounds execution cost and duration.
|
|
196
|
+
|
|
197
|
+
## Messages
|
|
198
|
+
|
|
199
|
+
A `messages` directory holds `*.message.yaml` files. Each kind has different required
|
|
200
|
+
fields; a valid WebSocket message is not automatically a valid MQTT message.
|
|
201
|
+
|
|
202
|
+
```yaml
|
|
203
|
+
$kind: websocket-message # requires content + contentType
|
|
204
|
+
content: '{"op":"subscribe"}'
|
|
205
|
+
contentType: json # text | json | xml | html | hex | base64
|
|
206
|
+
```
|
|
207
|
+
|
|
208
|
+
```yaml
|
|
209
|
+
$kind: socket.io-message # requires eventName + args
|
|
210
|
+
eventName: chat:message
|
|
211
|
+
acknowledgement: true
|
|
212
|
+
args:
|
|
213
|
+
- content: '{"text":"hi"}'
|
|
214
|
+
contentType: json
|
|
215
|
+
```
|
|
216
|
+
|
|
217
|
+
```yaml
|
|
218
|
+
$kind: mqtt-message # requires content + contentType + topic
|
|
219
|
+
topic: sensors/1/temp
|
|
220
|
+
content: '21.5'
|
|
221
|
+
contentType: text # text | json | hex | base64
|
|
222
|
+
qos: 1
|
|
223
|
+
retain: false
|
|
224
|
+
```
|
|
225
|
+
|
|
226
|
+
## Examples
|
|
227
|
+
|
|
228
|
+
Examples live in `.resources/<request-name>.resources/examples/*.example.yaml`, with
|
|
229
|
+
`$kind` matching the request's protocol — `graphql-example`,
|
|
230
|
+
`grpc-example`, `websocket-example`, `socket.io-example`, `mqtt-example`. An
|
|
231
|
+
`*-example` kind is rejected in a `*.request.yaml`, and a `*-request` kind is rejected
|
|
232
|
+
in a `*.example.yaml`.
|
package/skills/datasets/SKILL.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: datasets
|
|
3
|
-
description: Use when a collection needs queried or joined test inputs, a reusable named view, spreadsheet worksheet data, live database rows, or script access through pm.datasets(). Chooses between a simple iteration file and a dataset while preserving query semantics, portability, and credential boundaries. Use for requests such as "run this collection over database rows".
|
|
3
|
+
description: Use when creating or repairing dataset manifests, or when a collection needs queried or joined test inputs, a reusable named view, spreadsheet worksheet data, live database rows, or script access through pm.datasets(). Chooses between a simple iteration file and a dataset while preserving query semantics, portability, and credential boundaries. Use for requests such as "run this collection over database rows".
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# Datasets
|
|
@@ -43,15 +43,16 @@ stay with `collection run -d`; a dataset should earn its extra configuration.
|
|
|
43
43
|
|
|
44
44
|
Create a local manifest with `postman dataset create <path.dataset.yaml>
|
|
45
45
|
--name <name>`, then attach sources with `dataset source add`. Source commands
|
|
46
|
-
supply IDs and validation that hand-written entries can miss.
|
|
47
|
-
|
|
46
|
+
supply IDs and validation that hand-written entries can miss. When inspecting
|
|
47
|
+
or repairing manifest fields, consult [Dataset YAML](reference/schema.md).
|
|
48
|
+
Existing-dataset commands accept a local path or workspace ID; `list` needs a local path for
|
|
48
49
|
local discovery, while workspace create/list use `--workspace`.
|
|
49
50
|
|
|
50
51
|
File sources are copied into `data_dir` unless `--ref-only` is used. That option
|
|
51
52
|
records absolute paths and makes the dataset machine-specific. For a workspace
|
|
52
53
|
dataset, adding `--file` alone registers a local filesystem source; `--upload`
|
|
53
|
-
is what uploads bytes.
|
|
54
|
-
|
|
54
|
+
is what uploads bytes. Datasets are pushed along with the workspace; there is
|
|
55
|
+
no separate `dataset push`.
|
|
55
56
|
|
|
56
57
|
For spreadsheet sources, use the worksheet-aware guidance in
|
|
57
58
|
[Spreadsheet sources](reference/spreadsheets.md). For ordinary source names,
|
|
@@ -0,0 +1,51 @@
|
|
|
1
|
+
# Dataset YAML
|
|
2
|
+
|
|
3
|
+
Consult this reference when inspecting or repairing manifest fields. Source and
|
|
4
|
+
view commands can supply generated IDs when creating new entries.
|
|
5
|
+
|
|
6
|
+
```
|
|
7
|
+
postman/datasets/users/
|
|
8
|
+
├── users.dataset.yaml
|
|
9
|
+
└── .resources/
|
|
10
|
+
└── rows.json
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
```yaml
|
|
14
|
+
type: dataset # not $kind
|
|
15
|
+
name: "Users"
|
|
16
|
+
id: 3f2b6c1e-9d4a-4f81-bc37-5e0a71d9c402 # uuid4
|
|
17
|
+
data_dir: .resources
|
|
18
|
+
sources:
|
|
19
|
+
- from: embedded_datasource
|
|
20
|
+
datasource:
|
|
21
|
+
name: Records
|
|
22
|
+
id: 8a1d0e77-5c62-4b19-9f03-2d6e4a8b7c15
|
|
23
|
+
format: json # this example uses a JSON file
|
|
24
|
+
source_type: static # static for files, dynamic for databases
|
|
25
|
+
location:
|
|
26
|
+
type: local_filesystem
|
|
27
|
+
value: rows.json # relative to data_dir
|
|
28
|
+
views:
|
|
29
|
+
- from: embedded_view
|
|
30
|
+
view:
|
|
31
|
+
name: Records_default_view
|
|
32
|
+
id: c94e5b20-71af-4d36-8e52-0b7c6d1934fa
|
|
33
|
+
query: SELECT id, name FROM Records ORDER BY id
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
`sources` and `views` are optional — `type`, `name`, `id` and `data_dir` alone is a
|
|
37
|
+
valid dataset, and is what `postman dataset create <path> --name X` writes.
|
|
38
|
+
|
|
39
|
+
For example, a MySQL source sets `format: mysql`, `source_type: dynamic`,
|
|
40
|
+
`ingest: false`, a `dsKey` and a `slug`, with `location.type: mysql`, `location.value`
|
|
41
|
+
a DSN and `location.table` the table name.
|
|
42
|
+
|
|
43
|
+
A view's `query` is SQL over the source `name`, not over the file name.
|
|
44
|
+
|
|
45
|
+
There is no `dataset lint`. `postman dataset get <path>` checks the structure offline.
|
|
46
|
+
`postman dataset query <path> -q "<sql>"` runs SQL against the source, which also proves
|
|
47
|
+
the data parses and the query resolves — note the `-q` flag is required.
|
|
48
|
+
|
|
49
|
+
Connector support and credential handling depend on the installed CLI. Use the
|
|
50
|
+
**datasets** skill's connector and credential guidance when changing a source;
|
|
51
|
+
the MySQL example above is not a complete list of supported connectors.
|
package/skills/flows/SKILL.md
CHANGED
|
@@ -49,7 +49,6 @@ result that says nothing about the caller's actual workflow.
|
|
|
49
49
|
and reports. Connector blocks need `--workspace` even for a local file;
|
|
50
50
|
otherwise failure occurs inside the graph. An access denial across several
|
|
51
51
|
workspaces points toward credentials or entitlement, not an empty flow list.
|
|
52
|
-
Use **bootstrap** for unresolved CLI/authentication setup.
|
|
53
52
|
|
|
54
53
|
Read [command options](reference/flow_cli_flags.md) for inputs, reporters,
|
|
55
54
|
dataset iteration, and flag collisions; check installed subcommand `-h` before
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: performance-testing
|
|
3
|
-
description: Use when load-testing or benchmarking an API with concurrent virtual users and latency or error thresholds. Reuses a collection to probe a chosen failure mode and distinguishes target limits from load-generator limits. Use for requests such as "load test this API".
|
|
3
|
+
description: Use when load-testing or benchmarking an API with concurrent virtual users and latency or error thresholds, or when importing or migrating a JMeter `.jmx` plan or a Gatling simulation (Java/Kotlin/Scala/JS/TS) into a Postman collection plus a `postman performance run` configuration. Reuses a collection to probe a chosen failure mode and distinguishes target limits from load-generator limits. Use for requests such as "load test this API", "import my JMeter plan", or "convert my Gatling project to Postman".
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# Performance Testing
|
|
@@ -47,6 +47,19 @@ and which JMeter features have no equivalent here — think-time timers,
|
|
|
47
47
|
concurrent thread groups, and non-HTTP (JDBC/FTP/LDAP/Java) samplers are
|
|
48
48
|
reported, never faked.
|
|
49
49
|
|
|
50
|
+
## Importing from Gatling
|
|
51
|
+
|
|
52
|
+
Have a Gatling project (a Java/Kotlin/Scala/JS/TS `Simulation`) instead of a
|
|
53
|
+
collection? Convert it by following `references/gatling-import/README.md`, then
|
|
54
|
+
treat the result as the collection above. There is no importer to run — you read
|
|
55
|
+
the source and build the collection, environment, `run.sh` and gap report, using
|
|
56
|
+
the files next to that README as the algorithm (`mapping.md`, `injection.md`,
|
|
57
|
+
`feeders.md`, `unsupported.md`). The gap report marks what mapped exactly, what
|
|
58
|
+
was approximated, and what has no equivalent — WebSocket/SSE and other non-HTTP
|
|
59
|
+
steps, arrival-rate (open-model) injection, think-time pauses, throttling and
|
|
60
|
+
retries are reported, never faked. Open-model injection is converted to VUs via
|
|
61
|
+
Little's law with the arithmetic shown, not guessed.
|
|
62
|
+
|
|
50
63
|
## Verification
|
|
51
64
|
|
|
52
65
|
Report the workload, runner, achieved traffic, latency and error metrics,
|
|
@@ -0,0 +1,141 @@
|
|
|
1
|
+
# Importing a Gatling project into Postman
|
|
2
|
+
|
|
3
|
+
> Reference for the `performance-testing` skill. Convert the simulation, review the gap report, then the result is an ordinary performance run. Verifying it runs one iteration of real requests against the target, so confirm the target first — the same way any action with effects outside this session gets confirmed.
|
|
4
|
+
|
|
5
|
+
## Overview
|
|
6
|
+
A Gatling simulation is code: a scenario of `http()` requests, checks, feeders,
|
|
7
|
+
pauses and flow control, plus a `setUp` that injects load and asserts on
|
|
8
|
+
results. Postman splits the same thing in two:
|
|
9
|
+
- the **collection** (requests, test scripts, variables, data file), and
|
|
10
|
+
- the **run config** (`postman performance run` flags: VUs, duration, load profile, pass criteria).
|
|
11
|
+
|
|
12
|
+
You do the whole translation by reading the source and applying the files next to
|
|
13
|
+
this one — there is no importer to run. Two disciplines keep you honest, both
|
|
14
|
+
described below: an **inventory** (list every construct first, so nothing is
|
|
15
|
+
dropped silently) and a **self-verify** pass (prove every construct landed
|
|
16
|
+
somewhere or is explained, and that the collection actually runs).
|
|
17
|
+
|
|
18
|
+
A conversion that runs but silently drops a check, or quietly changes the load
|
|
19
|
+
model, is worse than no conversion, because the user will trust the numbers.
|
|
20
|
+
**The gap report is a first-class output, not an apology.**
|
|
21
|
+
|
|
22
|
+
## Workflow
|
|
23
|
+
|
|
24
|
+
1. **Discover.** Find the simulation(s): classes that `extends Simulation`,
|
|
25
|
+
Kotlin `: Simulation()`, or JS/TS `simulation((setUp) => …)`. Note the language
|
|
26
|
+
and build tool. If there is more than one simulation, ask which to convert.
|
|
27
|
+
Convert one simulation per output directory.
|
|
28
|
+
2. **Inventory.** Read every source file the simulation touches and list **every
|
|
29
|
+
construct a conversion must account for** — see *Inventory: what to list*. This
|
|
30
|
+
is your checklist; each item gets a stable id. Pay attention to requests built
|
|
31
|
+
in loops or helper methods and to dynamic names — a shallow scan misses them.
|
|
32
|
+
3. **Translate** each item using the files next to this one:
|
|
33
|
+
- `mapping.md`: requests, EL, checks, protocol settings, groups, flow control, custom code
|
|
34
|
+
- `feeders.md`: feeders → data file + distribution
|
|
35
|
+
- `injection.md`: injection profiles, throttle, assertions → run flags
|
|
36
|
+
- `unsupported.md`: what becomes a gap, and how to word it
|
|
37
|
+
4. **Write outputs** to the output directory (formats below).
|
|
38
|
+
5. **Verify** against the checklist in *Verify: self-review* — including, after you
|
|
39
|
+
confirm the target, one real smoke iteration via the existing
|
|
40
|
+
`postman collection run` command. Fix everything that fails and re-check, up to
|
|
41
|
+
3 rounds.
|
|
42
|
+
6. **Report** to the user: counts (mapped / approximated / gap), the verify
|
|
43
|
+
result, and the top 3 things they must do by hand. Point them to MIGRATION.md.
|
|
44
|
+
|
|
45
|
+
## Inventory: what to list
|
|
46
|
+
List every construct below that appears in the simulation, each with a **stable
|
|
47
|
+
id** (used verbatim in the MIGRATION.md coverage table). Kinds and id prefixes:
|
|
48
|
+
|
|
49
|
+
- `req:<name>` — each `http("name").get/post(...)` request.
|
|
50
|
+
- `check:<req>#<n>` — each check on a request (`status`, `jsonPath`, `jmesPath`, `regex`, `header`, `bodyString`, `substring`, `responseTimeInMillis`, …). A request with **no** `status` check still has Gatling's implicit 2xx-or-304 check — list it as `implicit-status:<req>`.
|
|
51
|
+
- `scenario:<name>`, `group:<name>` — scenarios and groups.
|
|
52
|
+
- `feeder:<file>` and `feed:<n>` — feeder sources and `feed(...)` calls (one logical data source emits both; see `feeders.md`).
|
|
53
|
+
- `timing:<n>` — `pause`, `pace`, `rendezVous`.
|
|
54
|
+
- `flow:<construct>#<n>` — `repeat`, `during`, `forever`, `doIf`, `tryMax`, `exitHereIfFailed`, `doSwitch`, …
|
|
55
|
+
- `inj#<n>` — each injection step (open or closed model).
|
|
56
|
+
- `throttle#<n>`, `max-duration` — throttle / `maxDuration`.
|
|
57
|
+
- `assert#<n>` — each `setUp(...).assertions(...)` entry.
|
|
58
|
+
- `proto:<setting>` — each `http.baseUrl/header/acceptHeader/shareConnections/enableHttp2/...`.
|
|
59
|
+
- `non_http:<proto>:<name>` — `ws`, `sse`, `grpc`, `mqtt`, … steps (and their checks).
|
|
60
|
+
- `code:<n>` — custom-code `exec(session -> …)` lambdas.
|
|
61
|
+
- `secret:<n>` — any literal that looks like a credential (API key, token, `Authorization` value). **Record only that it exists — never copy the literal anywhere.**
|
|
62
|
+
|
|
63
|
+
A request built in a loop or helper, or named dynamically, still gets an id; find it by reading, not pattern-matching. If you are unsure a construct exists, list it and resolve it to a gap rather than omit it.
|
|
64
|
+
|
|
65
|
+
## Output formats
|
|
66
|
+
### `collection.json`: Postman collection v2.1
|
|
67
|
+
- `info.schema` = `https://schema.getpostman.com/json/collection/v2.1.0/collection.json`
|
|
68
|
+
- `info.description` is a plain string whose first line is exactly `imported-from: gatling` followed by two lines `source: <relative source path>` and `class: <simulation class/name>`. Example: `"imported-from: gatling\nsource: src/test/java/poc/BasicSimulation.java\nclass: BasicSimulation"`.
|
|
69
|
+
- One collection per simulation. One folder per Gatling `group`; with **no** groups, requests sit at the collection root (a bare request name, no folder prefix — that is valid). Requests keep their Gatling names exactly (they drive per-request metrics), and **request leaf names must be unique** (a collection cannot hold two requests with the same path — see `repeat` in mapping.md).
|
|
70
|
+
- Base URL → collection variable `baseUrl`. Requests use `{{baseUrl}}/path`.
|
|
71
|
+
- Each request's test script ends with the translated checks. Mark translated custom code with `// REVIEW: translated from Gatling custom code (<inventory id>)`.
|
|
72
|
+
|
|
73
|
+
### `environment.json`: Postman environment
|
|
74
|
+
`{"name": "<simulation> (from Gatling)", "values": [{"key": "...", "value": "...", "enabled": true}]}`.
|
|
75
|
+
Secrets become keys with an **empty value** and `"type": "secret"`. Never copy a
|
|
76
|
+
secret literal from the source (see Critical rules). This file holds **secrets and
|
|
77
|
+
externalised config only** — `baseUrl` and correlation values are collection variables, not
|
|
78
|
+
environment keys. **Always write the file**, even when there is nothing to put in it: a
|
|
79
|
+
simulation with no secrets and no externalised config gets `"values": []`. (An empty
|
|
80
|
+
environment is correct and expected, not a defect.)
|
|
81
|
+
|
|
82
|
+
### `data/<name>.csv`
|
|
83
|
+
Copy feeder files that the simulation uses. Convert JSON array feeders to CSV if they are flat. If the simulation has **no feeder**, don't create a `data/` directory at all (and omit `--data-file` from run.sh).
|
|
84
|
+
|
|
85
|
+
### `run.sh`
|
|
86
|
+
```bash
|
|
87
|
+
#!/usr/bin/env bash
|
|
88
|
+
# Converted from <Simulation>. Replace <collection-uid> after pushing the collection to a workspace.
|
|
89
|
+
postman performance run "<collection-uid>" \
|
|
90
|
+
--environment "<environment-uid>" \
|
|
91
|
+
--vu-count 20 --duration 5 --load-profile ramp-up \
|
|
92
|
+
--data-file data/users.csv \
|
|
93
|
+
--pass-if "less_than(error_rate, 1)"
|
|
94
|
+
```
|
|
95
|
+
Only flags that exist in `postman performance run --help` (check the user's installed
|
|
96
|
+
version; don't assume a flag from a newer CLI). Duration is in whole minutes, rounded up.
|
|
97
|
+
- `--environment` — include it whenever `environment.json` has any values (the secrets/config requests depend on). Requests using `{{secret}}`/config resolve to nothing without it. Tell the user to import and **populate** `environment.json`, then replace `<environment-uid>` with its UID. Omit the flag only when the environment is empty (`values: []`).
|
|
98
|
+
- `--data-file` only if the simulation has a feeder; **omit the line entirely** when there is none (don't invent a data file).
|
|
99
|
+
- `--pass-if`: **at most one**, chosen by the strict priority in `injection.md` (error-rate, then latency percentile, then mean/rps) — this overrides source order. If **no** source assertion matches any category, **omit `--pass-if` entirely**. Every non-chosen assertion goes to the gap report.
|
|
100
|
+
|
|
101
|
+
### `MIGRATION.md`: gap report (also your accounting record)
|
|
102
|
+
Sections, in order:
|
|
103
|
+
1. **Summary**: source, counts (mapped / approximated / gap), verify result. The counts must equal the coverage-table tally; derive them from the table, don't hand-write them.
|
|
104
|
+
2. **Load model**: the original injection profile, the chosen flags, and the arithmetic (see `injection.md`). Required even when the mapping is exact.
|
|
105
|
+
3. **Coverage**: a table with one row per inventory id — this is the accounting that replaces a manifest. Columns: `id` · `kind` · `status` (`mapped` / `approximated` / `gap`) · `landed` (where it went, or why it was dropped). **Every inventory id appears here**; a missing row is a silently-dropped construct. Use these idioms in `landed`:
|
|
106
|
+
- request → `collection:<folder>/<request>` (root request → just the name)
|
|
107
|
+
- check / saveAs → `collection:<req> › test` (a `saveAs` value is script-set, so it is a test-script line, not a declared collection variable)
|
|
108
|
+
- protocol setting on every request → name each request it lands on
|
|
109
|
+
- scenario → `the collection itself`; group → `folder <name>`
|
|
110
|
+
- injection → the `run:` flags it set
|
|
111
|
+
- secret → `environment:<key> (placeholder, value not copied)`
|
|
112
|
+
4. **Approximations**: one bullet per approximated id: `` `<id>` ``, what changed, why, what to check.
|
|
113
|
+
5. **Gaps**: one bullet per gap id: `` `<id>` ``, what was dropped, why, and the manual workaround if there is one.
|
|
114
|
+
6. **Secrets**: which environment keys the user must fill in.
|
|
115
|
+
7. **Next steps**: push the collection, fill secrets, run a short test, compare with a Gatling baseline.
|
|
116
|
+
|
|
117
|
+
Every `approximated` or `gap` id **must appear verbatim** in both the coverage table and its Approximations/Gaps bullet.
|
|
118
|
+
|
|
119
|
+
## Verify: self-review
|
|
120
|
+
Before claiming done, check each of these against your own output and fix every miss:
|
|
121
|
+
|
|
122
|
+
1. **Accounting** — every inventory id has a row in the MIGRATION.md coverage table; every `approximated`/`gap` id also has its bullet. Nothing dropped silently.
|
|
123
|
+
2. **Collection** — valid v2.1, `info.description` stamped `imported-from: gatling`, every request has a method and a URL, and no two requests share the same folder/name path.
|
|
124
|
+
3. **Variables** — every `{{var}}` used anywhere in the collection is defined somewhere: a collection variable, an environment key, a data-file column, or set by an earlier request's script. An undefined `{{var}}` is a bug.
|
|
125
|
+
4. **Secrets** — no secret literal appears in any output file (collection, environment, run.sh, MIGRATION.md). Re-read the source's credential-looking values and confirm none were copied.
|
|
126
|
+
5. **run.sh** — run `postman performance run --help` and confirm every flag you used exists in this CLI version, values are valid (duration a whole number ≥ 1, load profile one of the listed names), and there is **at most one** `--pass-if`.
|
|
127
|
+
6. **Smoke** — the one executable check. Tell the user it sends **one iteration** of the converted requests to the target host(s), name the host(s), and wait for a yes for any host that is not local. Then run the existing CLI:
|
|
128
|
+
|
|
129
|
+
```
|
|
130
|
+
postman collection run collection.json -n 1 -e environment.json
|
|
131
|
+
```
|
|
132
|
+
|
|
133
|
+
(add `-d data/<file>` when there is a feeder). Confirm every request executed with no transport error. A `pm.test` failure that comes from the conversion needs an explanation in MIGRATION.md; one that comes from the target (e.g. a 4xx the simulation also expected) just gets noted. If the user declines the traffic, or there is no CLI login / network, skip only this step and say in your report that no request was executed — the other five checks still hold.
|
|
134
|
+
|
|
135
|
+
## Critical rules
|
|
136
|
+
1. **Never copy a secret literal into any output.** That covers the collection, environment, run.sh and MIGRATION.md. Items of kind `secret` become an environment key with an empty value, and MIGRATION.md tells the user to fill it in. Don't put secrets in descriptions or metadata either.
|
|
137
|
+
2. **Never drop an item silently.** Every inventory id gets a coverage-table row. If you aren't sure, choose `approximated` or `gap` and explain. Don't choose `mapped`.
|
|
138
|
+
3. **Never change the load model silently.** Open-model (arrival-rate) injection has no exact equivalent. Always show the conversion arithmetic in MIGRATION.md → Load model.
|
|
139
|
+
4. **Confirm before sending traffic.** The smoke sends one iteration to the target. Get a yes for any non-local host first. Never run `postman performance run` yourself; the user runs `run.sh`.
|
|
140
|
+
5. **Keep request names identical** to the Gatling `http("…")` names, so results can be compared with old Gatling reports.
|
|
141
|
+
6. **Don't claim done until the self-verify checklist passes.** If you can't get it to pass in 3 rounds, stop and report what is still failing.
|
|
@@ -0,0 +1,31 @@
|
|
|
1
|
+
# Feeders → data file + distribution
|
|
2
|
+
|
|
3
|
+
## Sources (`feeder:*`)
|
|
4
|
+
| Gatling | Postman | Status |
|
|
5
|
+
|---|---|---|
|
|
6
|
+
| `csv("f.csv")`, `tsv`, `ssv`, `separatedValues` | copy to `data/f.csv` (convert TSV/SSV to CSV) → `--data-file data/f.csv` | mapped |
|
|
7
|
+
| `jsonFile("f.json")` (a flat array of objects) | `data/f.csv`, or keep JSON (`--data-file` accepts JSON) | mapped |
|
|
8
|
+
| `jsonUrl(url)` | download once (with the user's OK) or leave as a gap | gap unless downloaded |
|
|
9
|
+
| `jdbcFeeder`, `redisFeeder`, `sitemap` | export to CSV by hand | gap. Give the export query in MIGRATION.md |
|
|
10
|
+
| `listFeeder` / `arrayFeeder` (inline data) | write the rows to `data/<name>.csv` | mapped |
|
|
11
|
+
|
|
12
|
+
The file must sit in `$OUT/data/` so that `run.sh` and the smoke run can find it. Set `manifest.run.data_file` to the file the smoke run should use.
|
|
13
|
+
|
|
14
|
+
## Strategy
|
|
15
|
+
The Gatling default (no strategy) is **`queue`**. Each record is used once, and the run **crashes** when the records run out.
|
|
16
|
+
| Gatling | `postman performance run --data-file` behaviour | Status |
|
|
17
|
+
|---|---|---|
|
|
18
|
+
| `.circular()` | with `--data-file`, local rows are distributed per the CLI's documented default (confirm in the installed `--help`; it has been random in recent CLIs). Each VU iteration picks a row | approximated. Say the distribution follows the CLI default, not Gatling's round-robin. A Postman Dataset (`--dataset-distribution round-robin`) is the exact match |
|
|
19
|
+
| `.random()` | random | mapped |
|
|
20
|
+
| `.shuffle()` | random (with replacement) | approximated |
|
|
21
|
+
| `.queue()` / default | random; rows can repeat and the run never runs out | approximated. **Always** call out that uniqueness is lost: if the source relies on each row being used once (unique users, one-time tokens), the test will behave differently |
|
|
22
|
+
| `.shard()` | none | gap |
|
|
23
|
+
| `.batch()`, `.eager()`, `.transform(...)`, `.unzip()` | `eager` and `batch` are no-ops. `transform` → a pre-request script | approximated |
|
|
24
|
+
|
|
25
|
+
## `feed(...)` calls (`feed:*`)
|
|
26
|
+
`feed(f)` at the start of the scenario → each iteration gets a row, and the columns become variables `{{column}}`. Mapped, with `target: "data:data/<file>"`.
|
|
27
|
+
`feed(f, n)` (several records at once) → gap.
|
|
28
|
+
|
|
29
|
+
**One data source, two inventory ids.** The inventory emits a `feeder:*` id (the source + its distribution strategy) **and** a `feed:*` id (the binding that hands a row to each iteration), both describing one logical data file. Account for them split by concern: the `feed:*` id is **mapped** (`target: data:data/<file>` — row → `{{column}}` variables), and the `feeder:*` id carries the **strategy** status (e.g. `.circular()` → **approximated**, "distribution follows the CLI default, not round-robin"; default `queue` → **approximated**, "uniqueness lost"). Both may point at the same `data:` target; that is expected, not a duplicate.
|
|
30
|
+
|
|
31
|
+
**Multiple feeders.** A scenario with more than one `feed(...)` (e.g. `feed(users).feed(products)`) has **no exact mapping**: `postman performance run` takes a single `--data-file` and the smoke run a single `-d`, so only one set of columns can bind. Do **not** emit two `--data-file` bindings and mark both mapped — one set would silently go unbound. Instead either: (a) **combine** the sources into one CSV/dataset by joining their columns, and document the changed row pairing/distribution in MIGRATION.md (**approximated**); or (b) map the primary feeder and record the others as **gaps**, naming which `{{columns}}` won't resolve. State the choice in the coverage table.
|