@postman/postman-plugin 0.3.0 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +32 -8
- package/dist/hosts/antigravity.js +47 -0
- package/dist/hosts/copilot.js +60 -0
- package/dist/hosts/index.js +3 -1
- package/mcp.pi.json +2 -2
- package/package.json +1 -1
- package/skills/ai-readiness/SKILL.md +40 -44
- package/skills/api-discovery/SKILL.md +64 -145
- package/skills/api-discovery/reference/orbit.md +17 -5
- package/skills/api-documentation/SKILL.md +41 -46
- package/skills/api-engineer/SKILL.md +78 -23
- package/skills/api-mocking/SKILL.md +117 -163
- package/skills/api-monitoring/SKILL.md +52 -129
- package/skills/api-simulation/SKILL.md +130 -178
- package/skills/api-simulation/reference/sim_yaml_schema.md +41 -103
- package/skills/api-testing/SKILL.md +48 -95
- package/skills/bootstrap/SKILL.md +123 -224
- package/skills/ci-integration/SKILL.md +53 -113
- package/skills/collection-schema-v3/SKILL.md +43 -204
- package/skills/collection-schema-v3/reference/http.md +150 -0
- package/skills/datasets/SKILL.md +89 -315
- package/skills/datasets/reference/spreadsheets.md +25 -0
- package/skills/flows/SKILL.md +57 -206
- package/skills/flows/reference/flow_cli_flags.md +3 -2
- package/skills/performance-testing/SKILL.md +46 -62
- package/skills/performance-testing/references/jmeter-import/README.md +113 -0
- package/skills/performance-testing/references/jmeter-import/import-report.md +52 -0
- package/skills/performance-testing/references/jmeter-import/mapping-table.md +97 -0
- package/skills/performance-testing/references/jmeter-import/postman-perf-capabilities.md +56 -0
- package/skills/performance-testing/references/jmeter-import/script-translation.md +98 -0
- package/skills/performance-testing/references/jmeter-import/scripts/groovy_to_js.py +83 -0
- package/skills/performance-testing/references/jmeter-import/scripts/jmx_to_postman.py +1433 -0
- package/skills/postman-mcp-server/SKILL.md +53 -59
- package/skills/postman-mcp-server/references/docs.md +0 -1
- package/skills/postman-mcp-server/references/learn.md +10 -9
- package/skills/postman-mcp-server/references/mock.md +0 -1
- package/skills/postman-mcp-server/references/search.md +0 -1
- package/skills/postman-mcp-server/references/security.md +0 -1
- package/skills/postman-mcp-server/references/setup.md +0 -1
- package/skills/postman-mcp-server/references/sync.md +0 -1
- package/skills/postman-mcp-server/references/test.md +0 -1
- package/skills/variables/SKILL.md +44 -87
- package/skills/webhook-testing/SKILL.md +103 -0
- package/skills/webhook-testing/reference/forwarding.md +10 -0
- package/skills/webhook-testing/reference/outbound_webhooks.md +25 -0
- package/skills/webhook-testing/reference/webhook_scripts.md +163 -0
|
@@ -1,138 +1,61 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: api-monitoring
|
|
3
|
-
description:
|
|
3
|
+
description: Use when setting up recurring API checks, running or diagnosing a Postman Monitor, changing its schedule, or reaching private APIs through a self-hosted runner. Reuses collection assertions to detect live regressions and traces failures to individual run evidence. Use for requests such as "check this API every minute".
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# API Monitoring
|
|
7
7
|
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
`--
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
ignored, so the CLI rejects it locally rather than returning a page that
|
|
57
|
-
looks right but isn't — use `--cursor` from the previous page's response.
|
|
58
|
-
`--columns` picks which fields show, `-f`/`--filter` matches on name within
|
|
59
|
-
the page already returned (not a server-side search), and `--sort name|active`
|
|
60
|
-
orders it. `monitor get <monitorId>` shows one monitor's full configuration.
|
|
61
|
-
|
|
62
|
-
## Triggering a run
|
|
63
|
-
|
|
64
|
-
`monitor run <monitorId>` runs an existing Monitor synchronously and prints
|
|
65
|
-
the result — useful in CI to get a pass/fail right after a deploy rather
|
|
66
|
-
than waiting for the next scheduled tick. `-t/--timeout` (default 15
|
|
67
|
-
minutes) caps how long the CLI waits for completion; a timeout hit here is
|
|
68
|
-
the CLI giving up on waiting, not the Monitor itself failing. `--async`
|
|
69
|
-
submits the run and returns immediately with its job id and Postman URL
|
|
70
|
-
instead of waiting at all — reach for this over a long `-t` when the caller
|
|
71
|
-
doesn't need the verdict inline. `--json` prints only the verdict as JSON,
|
|
72
|
-
for scripting. `-x/--suppress-exit-code` overrides the default
|
|
73
|
-
fail-on-failed-run exit code, for a caller that wants to see failures
|
|
74
|
-
without breaking a pipeline step.
|
|
75
|
-
|
|
76
|
-
## Diagnosing a run
|
|
77
|
-
|
|
78
|
-
`monitor jobs list <monitorId>` lists a monitor's recent jobs; `monitor jobs
|
|
79
|
-
get <jobId>` reports one job's terminal state and its per-region run
|
|
80
|
-
outcomes — a monitor with runners in multiple regions runs once per region
|
|
81
|
-
per job. `monitor runs get <runId>` goes one level deeper: which test
|
|
82
|
-
assertions ran during one attempt, which failed, and why. Reach for these
|
|
83
|
-
instead of re-running blind after a `-t` timeout, or whenever the task is
|
|
84
|
-
explaining *why* a monitor failed rather than just that it did.
|
|
85
|
-
|
|
86
|
-
## Private (self-hosted) runners
|
|
87
|
-
|
|
88
|
-
A private runner — the CLI's `runner regions` calls the same thing a
|
|
89
|
-
"private-runner" value — is an agent you run inside your own network so
|
|
90
|
-
Monitor traffic originates there instead of from Postman's cloud IPs. Reach
|
|
91
|
-
for one only when the monitored API sits behind a VPN, firewall, or on-prem
|
|
92
|
-
network that Postman's cloud can't reach directly; a public API should just
|
|
93
|
-
use a Postman region (`--runner us-east`, etc.) since that needs no
|
|
94
|
-
infrastructure of your own to run or maintain.
|
|
95
|
-
|
|
96
|
-
`runner start --id <id> --key <key>` (from the Postman app) registers a
|
|
97
|
-
runner that executes monitor runs from your own infrastructure instead of
|
|
98
|
-
Postman's cloud. Extra
|
|
99
|
-
flags cover the runner's own networking: `--region eu` for EU residency,
|
|
100
|
-
`--proxy`/`--egress-proxy`/`--egress-proxy-authz-url` for outbound routing,
|
|
101
|
-
`--ssl-extra-ca-certs` for a private CA, and `--metrics`/`--metrics-port`
|
|
102
|
-
for a health-check endpoint. Runner activity is published to the connected
|
|
103
|
-
workspace by default; `--no-report-events` keeps it local. `runner list`
|
|
104
|
-
shows the team's registered self-hosted runners —
|
|
105
|
-
feed an id from here into `monitor create/update --runner` or `monitor list
|
|
106
|
-
--runner`. `runner regions` lists the Postman-region and private-runner
|
|
107
|
-
values valid for `--runner` on `monitor create`/`update`, including a
|
|
108
|
-
static IP where one is configured — check here before guessing a region
|
|
109
|
-
string.
|
|
110
|
-
|
|
111
|
-
## Critical Rules
|
|
112
|
-
|
|
113
|
-
1. **`update` can't move a monitor to a different collection or environment,
|
|
114
|
-
and doesn't pause/resume it.** Delete and recreate for the former; use
|
|
115
|
-
`pause`/`resume` for the latter.
|
|
116
|
-
2. **A `monitor run -t` timeout is a wait cap, not a monitor failure.**
|
|
117
|
-
Don't report "the monitor failed" from a timeout without checking
|
|
118
|
-
`monitor jobs get`/`monitor runs get` (or the Postman app) for what the
|
|
119
|
-
run actually did after the CLI gave up waiting — or avoid the wait
|
|
120
|
-
entirely with `--async`.
|
|
121
|
-
3. **`--runner` on `monitor create`/`update` takes a region name or a
|
|
122
|
-
self-hosted runner id — check `runner regions`/`runner list` before
|
|
123
|
-
guessing a string,** and don't suggest `runner start` unless the target
|
|
124
|
-
API genuinely isn't reachable from Postman's cloud.
|
|
125
|
-
4. **`monitor list --offset` doesn't work — use `--cursor`,** and
|
|
126
|
-
`--runner` there can't be combined with the other filters.
|
|
127
|
-
5. **`monitor delete` is permanent.** Confirm intent before passing `-y` to
|
|
128
|
-
skip its prompt.
|
|
8
|
+
A monitor turns a saved collection into recurring evidence about a live API.
|
|
9
|
+
It can catch behavior that changes after deployment or between developer
|
|
10
|
+
sessions, but only if its assertions, environment, and execution location
|
|
11
|
+
exercise the guarantees users depend on.
|
|
12
|
+
|
|
13
|
+
1. Reuse the collection's meaningful assertions before scheduling it. A
|
|
14
|
+
request returning 200 is a weak health signal if the response can violate
|
|
15
|
+
the contract. Check auth, target environment, and write side effects; use
|
|
16
|
+
**api-testing** to establish what the collection actually verifies.
|
|
17
|
+
2. Choose cadence, timezone, execution location, and alert recipients from
|
|
18
|
+
the operational need. `postman monitor create -c <collection-id>` uses
|
|
19
|
+
the linked workspace unless one is supplied. Inspect command-specific
|
|
20
|
+
`-h` and available service options for supported schedules and limits.
|
|
21
|
+
Creation starts an immediate run by default; `--no-run-now` avoids that
|
|
22
|
+
when only scheduling was requested. A schedule expressed in the host's
|
|
23
|
+
default timezone can fire at the wrong operational time.
|
|
24
|
+
3. Choose a Postman region for a reachable public API. Use a self-hosted
|
|
25
|
+
runner when the target is reachable only inside the user's network;
|
|
26
|
+
running one adds infrastructure to maintain. `postman runner regions`
|
|
27
|
+
provides valid hosted locations and `runner list` provides registered
|
|
28
|
+
private runner IDs. `monitor create/update --runner` accepts either;
|
|
29
|
+
multiple locations help distinguish API failures from regional reachability.
|
|
30
|
+
4. Preserve monitor identity when updating schedule or run settings.
|
|
31
|
+
`monitor update` cannot change its collection or environment and does not
|
|
32
|
+
pause it; use `pause`/`resume` for that. A different collection/environment
|
|
33
|
+
requires a replacement, so plan the handoff before deleting the old
|
|
34
|
+
monitor and its continuity. Permanent deletion needs explicit intent.
|
|
35
|
+
5. Use `postman monitor run <id>` for an immediate verdict, such as after a
|
|
36
|
+
deploy. `--async` returns a job handle when the caller need not wait.
|
|
37
|
+
The CLI timeout caps waiting, not execution; after a timeout, inspect
|
|
38
|
+
the submitted job instead of sending duplicate traffic or reporting failure.
|
|
39
|
+
6. Diagnose at the level of the evidence. `monitor jobs list <monitor-id>`
|
|
40
|
+
finds jobs, `monitor jobs get <job-id>` shows terminal state and regional
|
|
41
|
+
outcomes, and `monitor runs get <run-id>` shows assertions within an
|
|
42
|
+
attempt. A failing region, a connection error, and a failed assertion
|
|
43
|
+
imply different fixes. Preserve normal exit-code failures for CI gates.
|
|
44
|
+
|
|
45
|
+
For discovery, `monitor list` supports workspace and collection filters;
|
|
46
|
+
use its returned cursor for pagination rather than offset. Its runner filter
|
|
47
|
+
is for a self-hosted ID and cannot be combined with other filters; a page-local
|
|
48
|
+
name filter does not establish absence across all pages.
|
|
49
|
+
|
|
50
|
+
For a private runner, use `runner start` with the registered ID and key and
|
|
51
|
+
verify that it connects before assigning runs. Inspect its help for proxy,
|
|
52
|
+
private CA, residency, and health-metric options only when the network requires
|
|
53
|
+
them. Runner activity is reported by default; `--no-report-events` disables
|
|
54
|
+
that reporting when the task requires it. Use **bootstrap** for unresolved
|
|
55
|
+
authentication rather than placing runner keys in repository files.
|
|
129
56
|
|
|
130
57
|
## Verification
|
|
131
58
|
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
not yet a verdict — resolve it with `monitor jobs get` before reporting a
|
|
136
|
-
result). If a self-hosted runner was started, confirm it registered (the
|
|
137
|
-
Postman app, or `runner list`, shows it as connected) before assuming
|
|
138
|
-
monitor runs will route through it.
|
|
59
|
+
Report the actual monitor configuration or terminal job/run outcome, including
|
|
60
|
+
where it executed and which checks failed. A created monitor, accepted job,
|
|
61
|
+
or connected runner alone does not prove the API passed its health checks.
|
|
@@ -1,186 +1,138 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: api-simulation
|
|
3
|
-
description:
|
|
3
|
+
description: Runs your code for real against stand-ins for the HTTP services it calls, served together as one environment on one local port or one deployed URL, whether that code is a service you're changing, a frontend, a test suite or a demo. Use when you add or change code that calls other services and need to run it, test it, prove a fix or reproduce an upstream failure (timeouts, 5xx, 409, 429) end to end, locally or in CI, without the real services because they're unreachable, need credentials, cost money, have side effects or don't exist yet. Also answers what happens to a request, or to the data it writes, when a service it calls goes down, is slow or returns errors ("what happens to a booking if the email service is down?"), by running each failure instead of reading the code. Applies even when nobody says mock or simulation. To give a frontend or another team a stand-in for an API, use api-mocking. Covers `postman simulation`.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# API Simulation
|
|
7
7
|
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
8
|
+
A simulation serves several mocks as one environment, each reached by its
|
|
9
|
+
routeKey: as a path prefix by default, or as an `x-mock-slug` header when the
|
|
10
|
+
caller's paths can't change.
|
|
11
|
+
|
|
12
|
+
Run your code against it rather than stubbing its HTTP client in tests or
|
|
13
|
+
calling the real services. A stub skips the client, config, timeouts and
|
|
14
|
+
error handling that will run in production; the real services need
|
|
15
|
+
credentials and leave side effects. A simulation runs that code over real
|
|
16
|
+
HTTP, the same way in CI, and its log shows what each dependency was asked.
|
|
17
|
+
|
|
18
|
+
Do only what the task needs. If a `.sim.yaml` in `postman/simulations/`
|
|
19
|
+
already covers it, run that one. One request through the changed path proves
|
|
20
|
+
a change; build failure variants only when asked how the code fails or when
|
|
21
|
+
it must survive a failure.
|
|
22
|
+
|
|
23
|
+
`postman simulation` needs CLI 1.70.0 or later (`postman --version`; an older
|
|
24
|
+
CLI answers `postman simulation -h` with its general help).
|
|
25
|
+
|
|
26
|
+
## Building it
|
|
27
|
+
|
|
28
|
+
- **Members are the code's HTTP dependencies.** Databases, queues and caches
|
|
29
|
+
aren't; run them locally. **api-discovery** can find calls made through
|
|
30
|
+
shared code; ask about the repo by its `git remote` name, because a close
|
|
31
|
+
name matches another repo.
|
|
32
|
+
- **Each member needs a mock: see `api-mocking`,** which looks for the owning
|
|
33
|
+
team's published mock before building one. For a question, one built from
|
|
34
|
+
the contract will do; one that invents responses can send the caller down a
|
|
35
|
+
path the real service never would. In a mock you build, answer `501` for calls this path
|
|
36
|
+
doesn't make: a `404` reads to the caller like a real "not found".
|
|
37
|
+
- **Prefix each mock's `pm.state` keys with its name.** As of CLI 1.73.0,
|
|
38
|
+
members of a local run read each other's keys, so two mocks storing
|
|
39
|
+
`orders` overwrite each other.
|
|
40
|
+
|
|
41
|
+
## Running it
|
|
42
|
+
|
|
43
|
+
The run is a server that never exits, so a foreground command never returns.
|
|
44
|
+
Start it in the background with an explicit port, its log in a file:
|
|
45
|
+
|
|
46
|
+
```bash
|
|
47
|
+
postman simulation run postman/simulations/checkout-dev.sim.yaml --port 4900 --output ndjson > sim.ndjson 2> sim.err &
|
|
25
48
|
```
|
|
26
49
|
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
- `404 Unknown simulation member`: the path has no valid routeKey. The body
|
|
117
|
-
lists the valid ones. It isn't logged.
|
|
118
|
-
- `501`: an upstream call you haven't modelled yet. Add it to your own
|
|
119
|
-
mock. For a pulled one, fork it into `postman/mocks/` first.
|
|
120
|
-
- `502 Proxy error`: the member's handler crashed. See `sim.ndjson`.
|
|
121
|
-
|
|
122
|
-
6. **Optionally, fail a dependency.** Worth doing when the change has to
|
|
123
|
-
survive an upstream that errors or is slow, but not required. Copy the
|
|
124
|
-
baseline `.sim.yaml`, give that member a condition, run the service
|
|
125
|
-
against it, and report what it did.
|
|
126
|
-
|
|
127
|
-
```yaml
|
|
128
|
-
- routeKey: workspaces
|
|
129
|
-
path: ../.dependencies/mocks/workspace-service/config.yaml
|
|
130
|
-
scenarios:
|
|
131
|
-
- overrides:
|
|
132
|
-
conditions:
|
|
133
|
-
error: { status_code: 503 }
|
|
134
|
-
```
|
|
135
|
-
|
|
136
|
-
The conditions are `latency`, `error`, `rate_limit` and `chaos`; the schema
|
|
137
|
-
has all four. Each applies to every route on that member, and the first to
|
|
138
|
-
fail a request ends it, so fail a dependency one way at a time. An injected
|
|
139
|
-
error, `429` or chaos failure never reaches the mock and isn't logged, so
|
|
140
|
-
judge it by what the service returned.
|
|
141
|
-
|
|
142
|
-
7. **Ship.** Commit your mocks, the `.sim.yaml` files and
|
|
143
|
-
`.postman/resources.yaml`, but not `sim.ndjson`: it records request
|
|
144
|
-
headers and bodies, tokens included. After a clone,
|
|
145
|
-
`postman dependency install` restores the pulled mocks. Running the
|
|
146
|
-
simulation in CI is optional: start it as in step 4 before the test step,
|
|
147
|
-
and stop it after.
|
|
148
|
-
|
|
149
|
-
## Critical Rules
|
|
150
|
-
|
|
151
|
-
1. **The service under test is never a member.** If nothing real runs, use
|
|
152
|
-
`api-mocking`.
|
|
153
|
-
2. **Unique routeKeys and an explicit `--port`.** A member's own `port:` is
|
|
154
|
-
ignored. Without `--port`, a busy 3000 silently moves to a random port.
|
|
155
|
-
3. **Members share one `pm.state` store.** Two stateful mocks that use the
|
|
156
|
-
same key read and overwrite each other's records, so give each its own keys.
|
|
157
|
-
4. **Only `overrides.conditions` inject faults.** A bare
|
|
158
|
-
`scenarios: - path: …/default.js` entry, as the Postman app writes, adds
|
|
159
|
-
none. `overrides.bypass` is ignored.
|
|
160
|
-
5. **Never write or generate a mock for a dependency before searching its
|
|
161
|
-
owner's workspace.** A mock you write encodes your assumptions; the
|
|
162
|
-
owner's encodes their contract and their state.
|
|
163
|
-
6. **Pull an owner's mock even when it's deployed.** Only members take faults
|
|
164
|
-
and log requests, and a private mock server needs an `x-api-key` your
|
|
165
|
-
service won't send. Never edit `postman/.dependencies/`:
|
|
166
|
-
`dependency update` overwrites it. To change a pulled mock, copy it into
|
|
167
|
-
`postman/mocks/` and call it a fork.
|
|
168
|
-
7. **Confirm before a public deploy.** `simulation deploy` without
|
|
169
|
-
`--private`, or with `-y`, serves every member's mock to anyone with the
|
|
170
|
-
URL. Ask the user first; a local `run` needs no deploy at all.
|
|
171
|
-
|
|
172
|
-
## Verification
|
|
173
|
-
|
|
174
|
-
Starting is not the same as being used. Before you report, read the whole
|
|
175
|
-
log: every member on the path should have `request` events with the statuses
|
|
176
|
-
you expect. A member with none isn't wired or isn't on this path.
|
|
177
|
-
|
|
178
|
-
Report:
|
|
179
|
-
|
|
180
|
-
- **How the simulation performed:** for each member, the requests it served
|
|
181
|
-
and their statuses, and anything unexpected, such as a `501`, a `502` or a
|
|
182
|
-
member that was never called.
|
|
183
|
-
- **Where each mock came from:** for each dependency, the owner workspace you
|
|
184
|
-
searched, what you found there, and whether its mock was pulled, forked or
|
|
185
|
-
generated here.
|
|
186
|
-
- **What ran for real.**
|
|
50
|
+
Without `--port`, a busy 3000 silently becomes a random port and every base
|
|
51
|
+
URL points at nothing. It's up when `sim.ndjson` has a `listening` event; a
|
|
52
|
+
start-up error is in `sim.err`. It listens on 127.0.0.1 only, so a caller in
|
|
53
|
+
a container or VM can't reach it. Stop it with
|
|
54
|
+
`pkill -INT -f "simulation run <file>"`; `kill $!` leaves it running,
|
|
55
|
+
because the `postman` launcher ignores the signal. Nothing reloads: restart
|
|
56
|
+
it after a mock edit.
|
|
57
|
+
|
|
58
|
+
Point the caller at it without changing what the code does once deployed.
|
|
59
|
+
Under path routing, swap only the host and keep the path:
|
|
60
|
+
`https://catalog.internal.example.com/v2` becomes
|
|
61
|
+
`http://127.0.0.1:4900/catalog/v2`; a dropped `/v2` is the usual cause of a
|
|
62
|
+
`404`. A hard-coded base URL should read an environment variable with
|
|
63
|
+
today's value as the default. If the code mustn't change at all,
|
|
64
|
+
`simulation run --proxy` reaches the members through `HTTPS_PROXY` instead;
|
|
65
|
+
it applies no conditions, and passes any other host through to the real
|
|
66
|
+
service.
|
|
67
|
+
|
|
68
|
+
## Reading the log
|
|
69
|
+
|
|
70
|
+
A passing test command doesn't show the dependency path ran: read
|
|
71
|
+
`sim.ndjson`. A member with no `request` events isn't wired or isn't on this
|
|
72
|
+
path. Some answers come from the simulation itself and mean wiring, not code:
|
|
73
|
+
|
|
74
|
+
- `404 Unknown simulation member`: a base URL lost its routeKey, or a client
|
|
75
|
+
sent the wrong `x-mock-slug`.
|
|
76
|
+
- `501`, or a generated mock's `404 Endpoint not defined`: a call the mock
|
|
77
|
+
doesn't model yet.
|
|
78
|
+
- `502 Proxy error`: the member's process died, usually an async throw in a
|
|
79
|
+
mock without `interceptRequests: true`. It stays down and logs nothing
|
|
80
|
+
until you restart; run that mock alone with `postman mock run` to see why.
|
|
81
|
+
|
|
82
|
+
## When a dependency goes down
|
|
83
|
+
|
|
84
|
+
Run it rather than reason from the code: each way of going down takes a
|
|
85
|
+
different path through the caller, and a stub skips the client code that
|
|
86
|
+
picks the path. Fail the one dependency in each shape that matters, a run
|
|
87
|
+
each, with the others answering normally:
|
|
88
|
+
|
|
89
|
+
- **Unreachable:** point its base URL at a port nothing listens on.
|
|
90
|
+
- **Hanging:** give its member a `latency` condition longer than the
|
|
91
|
+
client's timeout. If the client sets none, that is the finding: Node's
|
|
92
|
+
`fetch` waits about five minutes, so say so rather than wait it out.
|
|
93
|
+
- **An error with a JSON body:** the `error` condition.
|
|
94
|
+
- **An error with an HTML body,** as proxies send: a copy of its mock whose
|
|
95
|
+
default handler answers `500` with HTML. No condition can do this.
|
|
96
|
+
|
|
97
|
+
Only `overrides.conditions` on a member change it. Members always serve
|
|
98
|
+
their default handler and ignore `x-mock-scenario`, and a `scenarios` entry
|
|
99
|
+
with just a `path`, as the Postman app writes, does nothing. Fields and the
|
|
100
|
+
other conditions are in [reference/sim_yaml_schema.md](reference/sim_yaml_schema.md). An injected
|
|
101
|
+
failure never reaches the mock and isn't logged, so judge it by what the
|
|
102
|
+
caller did, and check the other members' logs for side effects, such as
|
|
103
|
+
stock reserved and never released.
|
|
104
|
+
|
|
105
|
+
## Keeping it
|
|
106
|
+
|
|
107
|
+
Commit your own mocks, the `.sim.yaml` and what `postman dependency add`
|
|
108
|
+
changed, so the next session or CI reruns it in one command. Don't commit
|
|
109
|
+
`postman/.dependencies/`: a copy goes stale when its owner updates theirs,
|
|
110
|
+
and `postman dependency install` restores it. Don't commit `sim.ndjson`: the
|
|
111
|
+
CLI redacts common auth headers and fields, but other headers and bodies are
|
|
112
|
+
logged as sent. In CI, `postman dependency install` needs `POSTMAN_API_KEY`,
|
|
113
|
+
and `postman collection run <collection> --simulation <file> --port <n>`
|
|
114
|
+
starts and stops the simulation around a collection run.
|
|
115
|
+
|
|
116
|
+
`postman simulation push` uploads every member to the workspace, so ask
|
|
117
|
+
first. It fails on a pulled member, which keeps its owner's `id:`; copy it to
|
|
118
|
+
`postman/mocks/` and delete that line.
|
|
119
|
+
|
|
120
|
+
## Rules
|
|
121
|
+
|
|
122
|
+
1. **What you're testing is never a member.** It must run for real, or the
|
|
123
|
+
test only proves the mock.
|
|
124
|
+
2. **Pull the owner's mock even if it's deployed.** Only members take
|
|
125
|
+
conditions and appear in the log, and a private mock server wants a key
|
|
126
|
+
your code won't send.
|
|
127
|
+
3. **`simulation deploy` is public by default,** unlike `mock deploy`. Ask
|
|
128
|
+
before deploying without `--private`; a local run needs no deploy.
|
|
129
|
+
|
|
130
|
+
## Before you say it works
|
|
131
|
+
|
|
132
|
+
Claim nothing until what you're testing actually ran: a service printed
|
|
133
|
+
that it's listening and answered your request, or the frontend or test
|
|
134
|
+
suite finished, and its calls are in the log. If its start command exited
|
|
135
|
+
or hung, the run proved nothing; say so and why. Say what a pass proves,
|
|
136
|
+
your code against these stand-ins rather than the real services, what still
|
|
137
|
+
needs the real service before release, and where each mock came from,
|
|
138
|
+
including when you skipped the owner's.
|