@postman/postman-plugin 0.4.1-rc.0 → 0.5.0-rc.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -73,7 +73,7 @@ codex plugin add postman@postman
73
73
 
74
74
  These commands track `main`; review the session-start hook in Codex after a
75
75
  manual install. The npx installer also approves the hook after each install or
76
- update, and moves an existing `codex-release` registration to `main`.
76
+ update.
77
77
 
78
78
  ### Cursor
79
79
 
@@ -232,6 +232,12 @@ exercise failure paths.
232
232
 
233
233
  ## How Postman builds context
234
234
 
235
+ `postman init` is idempotent and safe to run in any project. Run it before API
236
+ engineering work to create the relevant local directories and make Postman
237
+ skills available to guide the task, even if the project is already initialized.
238
+ See [bootstrap](https://github.com/postmanlabs/postman-plugin/blob/main/skills/bootstrap/SKILL.md#start-with-postman-init) for setup
239
+ options and workspace binding.
240
+
235
241
  Work done in a Postman folder produces artifacts: collection runs, lint
236
242
  results, request history, mock deployments. When that folder is connected to a
237
243
  Postman workspace after `postman init`, those artifacts are published to it as
package/mcp.pi.json CHANGED
@@ -6,8 +6,8 @@
6
6
  "description": "Postman workspaces, collections, specs, environments, mocks and monitors",
7
7
  "headers": {
8
8
  "X-Source": "postman-pi-plugin",
9
- "X-Plugin-Version": "0.4.1-rc.0",
10
- "User-Agent": "postman-pi-plugin/0.4.1-rc.0"
9
+ "X-Plugin-Version": "0.5.0-rc.0",
10
+ "User-Agent": "postman-pi-plugin/0.5.0-rc.0"
11
11
  }
12
12
  }
13
13
  }
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@postman/postman-plugin",
3
- "version": "0.4.1-rc.0",
3
+ "version": "0.5.0-rc.0",
4
4
  "description": "Postman's API engineering skills for coding agents: a Pi package, and an npx installer that sets up Claude Code, Codex, Cursor, Kimi Code, Factory Droid, OpenCode and Pi.",
5
5
  "type": "module",
6
6
  "license": "Apache-2.0",
@@ -30,8 +30,7 @@ guesswork each time work resumes.
30
30
  even when the provider is already chosen: those details help you assess fit and
31
31
  implement reliably without relying on remembered API behavior or loading a
32
32
  full vendor specification.
33
- Existing workspace resources can supply maintained contracts and mocks; the
34
- Context Graph can identify consumers and dependencies outside the checkout,
33
+ Context Graph can identify consumers and dependencies outside the current repo,
35
34
  helping you focus investigation before traversing more repositories.
36
35
  5. When you run or test code that calls other services, or are asked how it
37
36
  behaves when one of them fails, use **api-simulation**, which runs that code
@@ -44,20 +43,25 @@ guesswork each time work resumes.
44
43
  be diffed, reviewed, and reused in Git and CI. Use **bootstrap** when setup or
45
44
  workspace bindings are unresolved: reuse an existing local layout, pull the
46
45
  current service's existing workspace, or initialize a new project when neither
47
- exists. `postman init` scaffolds files using an agent-scoped session without a
48
- normal Postman login.
49
- 7. Another service's collection, environment, or mock can be added as a dependency
46
+ exists.
47
+ 7. `postman init` gives API engineering work a ready state. It scaffolds local structure: `postman/`
48
+ directories for API artifacts. It installs skills in `postman/skills` folder with guidance on designing, implementing, and validating APIs.
49
+ The command is idempotent and safe to rerun in an initialized project.
50
+ It creates an agent-scoped session that lets you record work without signing
51
+ in. You can claim that work later if you create an account.
52
+ 8. Another service's collection, environment, or mock can be added as a dependency
50
53
  without replacing this repository's owning workspace binding. Use
51
54
  **api-discovery** to bring in the relevant resource and preserve that separation.
52
- 8. Publishing validated local changes to the bound workspace makes them available
53
- for future work and teammates with access. Because `postman workspace push`
54
- changes shared state, obtain user consent before pushing; an existing request
55
- to publish those changes supplies that consent. For publication tied to reviewed
56
- PR merges, see **ci-integration**.
57
- 9. When requests need credentials, Postman secret variables let them resolve the
55
+ 9. Publishing local changes to the bound workspace makes changes available
56
+ for other agent sessions and teammates. Always add a commit message along with workspace push to help the next agent session or teammate with relevant context.
57
+ 10. A Postman workspace is a place to capture context. Use this information as prior
58
+ knowledge of work done before. For example,
59
+ `postman workspace context --type workspacePush --limit 5 --brief` lists the
60
+ last five workspace push messages and log.
61
+ 11. When requests need credentials, Postman secret variables let them resolve the
58
62
  credentials without exposing their values to the agent or committed files.
59
63
  See **variables**.
60
- 10. When implementing or debugging webhooks, use **webhook-testing** to
64
+ 12. When implementing or debugging webhooks, use **webhook-testing** to
61
65
  capture what the sender actually delivers.
62
66
  Can also be used when the need is to forward incoming events
63
67
  to a local handler and replay captured requests after fixes;
@@ -16,9 +16,12 @@ credentials and leave side effects. A simulation runs that code over real
16
16
  HTTP, the same way in CI, and its log shows what each dependency was asked.
17
17
 
18
18
  Do only what the task needs. If a `.sim.yaml` in `postman/simulations/`
19
- already covers it, run that one. One request through the changed path proves
20
- a change; build failure variants only when asked how the code fails or when
21
- it must survive a failure.
19
+ already covers it, run that one, and read the `<name>.context.md` beside it
20
+ first: it holds the commands, the preconditions and the failures earlier runs
21
+ hit, so reading it costs less than rebuilding any of that from the `.sim.yaml`
22
+ and the handlers. One request through the changed path proves a change; build
23
+ failure variants only when asked how the code fails or when it must survive a
24
+ failure.
22
25
 
23
26
  `postman simulation` needs CLI 1.70.0 or later (`postman --version`; an older
24
27
  CLI answers `postman simulation -h` with its general help).
@@ -34,6 +37,11 @@ CLI answers `postman simulation -h` with its general help).
34
37
  the contract will do; one that invents responses can send the caller down a
35
38
  path the real service never would. In a mock you build, answer `501` for calls this path
36
39
  doesn't make: a `404` reads to the caller like a real "not found".
40
+ - **Bring in each mock you don't own as a dependency first, then build the
41
+ simulation over them.** `postman dependency add mock <name-or-id>` declares
42
+ it in `.postman/resources.yaml` and materialises it under
43
+ `postman/.dependencies/`, and the member's `path:` points there; `-w
44
+ <workspace>` resolves a name from the workspace that owns it.
37
45
  - **Prefix each mock's `pm.state` keys with its name.** As of CLI 1.73.0,
38
46
  members of a local run read each other's keys, so two mocks storing
39
47
  `orders` overwrite each other.
@@ -41,18 +49,17 @@ CLI answers `postman simulation -h` with its general help).
41
49
  ## Running it
42
50
 
43
51
  The run is a server that never exits, so a foreground command never returns.
44
- Start it in the background with an explicit port, its log in a file:
52
+ Start it in the background, its log in a file:
45
53
 
46
54
  ```bash
47
55
  postman simulation run postman/simulations/checkout-dev.sim.yaml --port 4900 --output ndjson > sim.ndjson 2> sim.err &
48
56
  ```
49
57
 
50
- Without `--port`, a busy 3000 silently becomes a random port and every base
51
- URL points at nothing. It's up when `sim.ndjson` has a `listening` event; a
52
- start-up error is in `sim.err`. It listens on 127.0.0.1 only, so a caller in
53
- a container or VM can't reach it. Stop it with
54
- `pkill -INT -f "simulation run <file>"`; `kill $!` leaves it running,
55
- because the `postman` launcher ignores the signal. Nothing reloads: restart
58
+ `--port` is optional. It's up when `sim.ndjson` has a `listening` event; a
59
+ start-up error is in `sim.err`. That event carries the bound port and each
60
+ member's address, so take the base URLs from it rather than from the port you
61
+ asked for. Stop it with
62
+ `pkill -INT -f "simulation run .*<file>.sim.yaml"`. Nothing reloads: restart
56
63
  it after a mock edit.
57
64
 
58
65
  Point the caller at it without changing what the code does once deployed.
@@ -60,10 +67,30 @@ Under path routing, swap only the host and keep the path:
60
67
  `https://catalog.internal.example.com/v2` becomes
61
68
  `http://127.0.0.1:4900/catalog/v2`; a dropped `/v2` is the usual cause of a
62
69
  `404`. A hard-coded base URL should read an environment variable with
63
- today's value as the default. If the code mustn't change at all,
64
- `simulation run --proxy` reaches the members through `HTTPS_PROXY` instead;
65
- it applies no conditions, and passes any other host through to the real
66
- service.
70
+ today's value as the default. If the code mustn't change at all, run it
71
+ behind the proxy instead, below.
72
+
73
+ ## Through a proxy, when no base URL can change
74
+
75
+ `simulation run <file.sim.yaml> --proxy` puts the members behind a local
76
+ forward proxy that answers for the dependencies' own hostnames, so the code
77
+ keeps the base URLs it ships with. Any other host passes through to the real
78
+ service. Reach for it when hostnames are hard-coded, built from config the
79
+ change shouldn't touch, or owned by an SDK — and because nothing it answers
80
+ for has to resolve, an internal hostname with no DNS record works.
81
+
82
+ Read `postman simulation run --help` before driving it. The flags, what the
83
+ caller's own process needs in order to reach the proxy, and what to do about a
84
+ client that ignores it are all there and change with the CLI, so take them
85
+ from the command rather than from here.
86
+
87
+ Two things decide whether a proxied run proves anything. A member with no
88
+ `origin:` in the `.sim.yaml` is never matched, so its traffic reaches the real
89
+ service and a run can come up having intercepted nothing — `simulation create`
90
+ copies that field from the mock's manifest, so editing the manifest alone
91
+ won't fix it ([reference/sim_yaml_schema.md](reference/sim_yaml_schema.md)).
92
+ And conditions don't apply here: member `scenarios` are ignored, so failures
93
+ need the one-port run.
67
94
 
68
95
  ## Reading the log
69
96
 
@@ -102,10 +129,27 @@ failure never reaches the mock and isn't logged, so judge it by what the
102
129
  caller did, and check the other members' logs for side effects, such as
103
130
  stock reserved and never released.
104
131
 
132
+ ## Its context document
133
+
134
+ Every simulation is created with a `<name>.context.md` beside it, holding what
135
+ running this one has taught: the commands that start it and prove it's up, the
136
+ preconditions and what breaks when one fails, what it takes for granted about
137
+ the dependencies themselves, the data seeded to test with, the failures earlier
138
+ runs already hit, and what a passing run still doesn't prove. The `.sim.yaml` says what the simulation is, not how it behaves, so
139
+ this is the file that saves a session the search. A new one is mostly
140
+ `@todo`s, which are the questions a first real run answers.
141
+
142
+ Every line passes one test — the repo doesn't already state it — so it stays
143
+ about a screenful, and a claim carries the date and the run that established
144
+ it, because the log it came from isn't committed. Its sections and conventions
145
+ are in [reference/context_file.md](reference/context_file.md); read that
146
+ before writing it back.
147
+
105
148
  ## Keeping it
106
149
 
107
- Commit your own mocks, the `.sim.yaml` and what `postman dependency add`
108
- changed, so the next session or CI reruns it in one command. Don't commit
150
+ Commit your own mocks, the `.sim.yaml`, its context document and what
151
+ `postman dependency add` changed, so the next session or CI reruns it in one
152
+ command. Don't commit
109
153
  `postman/.dependencies/`: a copy goes stale when its owner updates theirs,
110
154
  and `postman dependency install` restores it. Don't commit `sim.ndjson`: the
111
155
  CLI redacts common auth headers and fields, but other headers and bodies are
@@ -136,3 +180,9 @@ or hung, the run proved nothing; say so and why. Say what a pass proves,
136
180
  your code against these stand-ins rather than the real services, what still
137
181
  needs the real service before release, and where each mock came from,
138
182
  including when you skipped the owner's.
183
+
184
+ Then write the run back into `<name>.context.md`, so the next session starts
185
+ where this one ended: the `@todo`s it answered, whatever it found wrong, a
186
+ precondition you had to discover, what broke and why, and which members served
187
+ or stayed silent — the last is a line every run adds, so a member silent across
188
+ several runs reads as a pattern rather than a one-off.
@@ -0,0 +1,159 @@
1
+ # `<name>.context.md`
2
+
3
+ The handoff document beside a simulation: `postman/simulations/checkout.sim.yaml`
4
+ gets `checkout.context.md`, and a foldered simulation keeps both in its folder.
5
+ `simulation create` and the Postman app write it from what they already know,
6
+ `simulation update` renames it with the simulation, and `simulation delete`
7
+ removes it. An existing one is never overwritten, because it holds what earlier
8
+ sessions learned.
9
+
10
+ It is prose for whoever works here next, human or agent. Nothing parses it, so
11
+ there is no schema to satisfy — but the generated headings and the
12
+ `<!-- begin:observations -->` markers are what a reader and the next write
13
+ expect to find, so keep them.
14
+
15
+ Keep it to the generated sections plus `## What didn't work`, in Markdown,
16
+ under about a screenful, so the line that matters is the one a reader lands on
17
+ rather than one they have to search for.
18
+
19
+ ## What the generated file contains
20
+
21
+ A fresh file states the simulation's name, the date, its members and the hosts
22
+ they stand in for, and the run command for this simulation's own path.
23
+ Everything needing someone who has actually run it is left as an `@todo` HTML
24
+ comment, so a half-filled file reads as half-filled instead of as fact.
25
+
26
+ | Section | Holds |
27
+ | --- | --- |
28
+ | Header | Generation date and what the mocks came from. |
29
+ | `## Purpose` | What the simulation stands in for and what work it unblocks. Generated as the member list and a pointer to the handlers; needs hindsight, so it is the last thing to rewrite. |
30
+ | `## Execution` | The commands, in order: start the simulation, start the service pointed at it, prove it is up, count what got called, stop. The generated block leaves the service's own command, the proof request, the port and the log path as `@todo`. |
31
+ | `## Assumptions` | Every precondition someone had to discover, what the simulation takes for granted about the dependencies' own behaviour, the seeded data worth reusing, and what the simulation cannot reproduce. Each is a dated claim, the consequence of it not holding, and how it was established. |
32
+ | `## Observations` | One dated line per run, newest first, between the `begin`/`end` markers: which members took traffic and which took none. A summary, not a request log, and a failure goes in `## What didn't work` instead. |
33
+ | `## What didn't work` | Failures this simulation has already hit, a dated line each: what broke, the signature you'd recognize it by, where it surfaced, and the cause. Not generated — add the heading at the end, after `## Observations`, the first time a run gives you one. |
34
+
35
+ `## Observations` and `## What didn't work` both accrete, so a run is visible
36
+ in the file after the next one; `## Purpose`, `## Execution` and
37
+ `## Assumptions` are rewritten in place, since they describe the simulation as
38
+ it stands rather than a run. Each accreting section is bounded: `## What didn't
39
+ work` keeps the ten most recent, a line staying until a run shows it fixed, and
40
+ `## Observations` the five most recent runs. Past that the oldest go.
41
+
42
+ A run that keeps its line is how a pattern becomes visible — a member silent
43
+ for three runs running is worth more than the same fact seen once.
44
+
45
+ Don't restate what the generated file already says: its `## Assumptions` ships
46
+ with the ones that hold for every simulation.
47
+
48
+ ## What earns a line
49
+
50
+ Write what running it taught you, and only that. Each line passes one test:
51
+ the repo doesn't already state it. That leaves five kinds of thing, and they
52
+ are the file's whole value:
53
+
54
+ - **A precondition you had to discover** — the setting, the service, the
55
+ cleanup a run needs before it works at all. In `## Assumptions`.
56
+ - **A failure and its cause**, with where it surfaced, so the next session
57
+ recognizes it instead of debugging it again. In `## What didn't work`.
58
+ - **What the run takes for granted about a dependency itself** — that an
59
+ operation is idempotent, that a timeout means nothing committed, what order
60
+ events arrive in. A mock answers these whether or not anyone checked, so its
61
+ answer is a belief your code now relies on. In `## Assumptions`.
62
+ - **What the simulation can't reproduce, and what nobody has settled**, so a
63
+ passing run is read for what it actually proves. In `## Assumptions`.
64
+ - **What the last run served**: which members took traffic, which stayed
65
+ silent, and why. In `## Observations`.
66
+
67
+ For anything the repo does state — routes and request shapes, valid actions,
68
+ auth mechanics, each mock's hosts, config flags — name the file that holds it,
69
+ the way the generated `## Purpose` points at the handlers, and let the reader
70
+ go there. One source stays true; a copy here drifts, and this file gets
71
+ committed, so a named file is also how a credential stays out of it.
72
+
73
+ Write statements that stay true between runs. Counts, timings and coverage
74
+ belong to the run that produced them, so leave them in the log and keep the
75
+ pattern here: "every member served except `workspace`" rather than a tally.
76
+
77
+ ## How a line carries its evidence
78
+
79
+ The log a claim came from isn't committed, so the line is all the next session
80
+ gets. Date every claim in `## Assumptions` and `## What didn't work`, and say
81
+ where it came from when a run didn't show it: `(observed 2026-10-22)` for what
82
+ a run established, `(reasoned 2026-10-22, src/payments/client.ts)` for what was
83
+ read rather than run.
84
+
85
+ A failure also carries the signature the next session will recognize it by —
86
+ the error text, the status, the member that sent it — since that is what
87
+ someone matches a new failure against. Quote the error itself, never a
88
+ response body. For the rest, name the committed file that holds it: the
89
+ handler that threw, the mock example captured from the real service, the test
90
+ that reproduces it. Not the run's log, which is wherever that run redirected
91
+ output and which the next run overwrites.
92
+
93
+ A good `## Assumptions` is a claim, its consequence and its date; a good
94
+ `## Observations` is a few lines; a good `## What didn't work` is one line per
95
+ failure:
96
+
97
+ ```markdown
98
+ ## Assumptions
99
+
100
+ - **`payments` treats a retried charge as idempotent on `Idempotency-Key`.**
101
+ If it doesn't, the retry path charges twice. Nobody has settled it and the
102
+ mock answers as though it does. (reasoned 2026-10-22, the mock's handler)
103
+ - **Redis holds no `EVENT_BATCH` key when a run starts.** A leftover one
104
+ crashes the aggregation flush before any request is sent. (observed
105
+ 2026-10-22)
106
+
107
+ <!-- begin:observations -->
108
+ 2026-10-22: every member served except `workspace`, which the service never
109
+ calls in development, and `god`, which models no endpoints.
110
+ 2026-10-18: same, and `payments` served only `POST /charges`.
111
+ <!-- end:observations -->
112
+
113
+ ## What didn't work
114
+
115
+ - 2026-10-22: the aggregation flush crashed on start —
116
+ `TypeError: batch.map is not a function` in `src/events/flush.ts`, from a
117
+ stale Redis `EVENT_BATCH` key an earlier run left behind. Not a simulation
118
+ error, so the log shows nothing.
119
+ - 2026-09-30: `payments` answered `501` for `POST /refunds` — the mock models
120
+ no refund path, so a run that reaches it proves nothing about refunds.
121
+ ```
122
+
123
+ ## Working with it
124
+
125
+ Read it before starting. An observed claim holds until your run contradicts
126
+ it; a reasoned one is a hypothesis, so check it, cheaply, when your run depends
127
+ on it. Check the assumptions and anything recorded about what a run doesn't
128
+ prove before concluding that something works. A claim this run contradicts is
129
+ a finding to report, not to quietly route around.
130
+
131
+ What it already answers, it answers: `## Execution` holds commands that ran,
132
+ and `## Assumptions` holds preconditions someone already established, so both
133
+ are usable as they stand rather than as a starting point to rebuild from. The
134
+ code you are about to change still needs reading; the setup around it is what
135
+ this file is for. A session that reads the document and then repeats the
136
+ search it was meant to replace has paid for it twice.
137
+
138
+ Then leave it truer than you found it:
139
+
140
+ - Replace the `@todo`s your run can answer, and delete each one you replace.
141
+ - Correct what you found wrong. Say what it is now, not what it used to be.
142
+ - Add the precondition you had to discover, or the thing the simulation turned
143
+ out not to reproduce, as another dated claim-and-consequence pair.
144
+ - Add what broke and why to `## What didn't work`, and delete the lines there
145
+ your run showed fixed.
146
+ - Add this run's summary to the top of the `## Observations` block, read from
147
+ the simulation's log, and drop the oldest line past five.
148
+
149
+ A run that only confirms the file still adds its `## Observations` line.
150
+
151
+ ## Rules
152
+
153
+ - Leave another session's assumptions and limits standing unless your run
154
+ contradicted one: not exercised is not the same as not true.
155
+ - Keep this file for how the simulation behaves, and let the `.sim.yaml`
156
+ speak for what it is.
157
+ - Rewrite `## Purpose` when the simulation's scope genuinely changed.
158
+ - Prefer deleting a line to adding one. Everything here is something the next
159
+ session must either trust or re-check.
@@ -8,8 +8,11 @@ Read this when writing or debugging a `.sim.yaml` by hand.
8
8
  simulation: checkout-dev # required: the name
9
9
  routing: path # path (default) or header
10
10
  mocks: # required: one entry per member
11
- - routeKey: catalog # lowercase letters, digits, - and _
12
- path: ../mocks/catalog/config.yaml # a mock's config.yaml, or a .js handler
11
+ - name: catalog # optional display name
12
+ routeKey: catalog # lowercase letters, digits, - and _
13
+ origin: # optional: hostnames, for --proxy
14
+ - catalog.internal.example.com
15
+ path: postman/mocks/catalog # a mock's folder, config.yaml or handler
13
16
  scenarios: # optional: conditions, below
14
17
  - overrides:
15
18
  conditions:
@@ -17,7 +20,17 @@ mocks: # required: one entry per member
17
20
  ```
18
21
 
19
22
  Run the `.sim.yaml` rather than a simulation id: a run by id drops every
20
- member's conditions (CLI 1.70.0–1.73.0).
23
+ member's conditions (CLI 1.70.0–1.73.0), and `--proxy` takes no id at all,
24
+ since the file is what declares the hostnames.
25
+
26
+ `simulation create` writes each `path:` relative to the repository root, the
27
+ directory holding `postman/`, and puts the simulation in a folder of its own
28
+ (`postman/simulations/checkout-dev/checkout-dev.sim.yaml`) so its
29
+ `<name>.context.md` sits beside it; the app records that document in a
30
+ `context:` ref. Reading is broader than writing — a path relative to
31
+ `postman/simulations/`, to the file, absolute or `~/`, naming the folder or
32
+ the manifest inside it, all resolve — and an unresolvable one names every
33
+ directory it looked in.
21
34
 
22
35
  ## Conditions
23
36
 
@@ -38,6 +51,23 @@ member's conditions (CLI 1.70.0–1.73.0).
38
51
  - A `latency` request is served and logged, with a `duration` that leaves
39
52
  out the delay.
40
53
 
54
+ ## `origin`
55
+
56
+ The hostnames a member answers for under `--proxy`, and the only thing that
57
+ decides which member a proxied request reaches.
58
+
59
+ `simulation create` copies `origin` from the mock's manifest
60
+ (`origin: api.stripe.com`, a string or a list).
61
+
62
+ A scheme and port are stripped (`https://API.Stripe.com:443` →
63
+ `api.stripe.com`). An unmatchable value — a path left on a copied URL, a
64
+ wildcard — is warned about and skipped, leaving the member working; two
65
+ members claiming one hostname fails the run. A member whose mock declares
66
+ none has no `origin` key, since `origin: []` would reach the actual service instead of the mock.
67
+
68
+ Under `--proxy` the members run on throwaway ports, so a manifest's `port` is
69
+ ignored.
70
+
41
71
  ## The log (`--output ndjson`)
42
72
 
43
73
  One JSON event per line on stdout; notices go to stderr, so keep them apart.
@@ -0,0 +1,144 @@
1
+ ---
2
+ name: application-testing
3
+ description: Sets up, runs, debugs, and interprets results from `postman application test` — the Postman layer that runs your Playwright tests, captures their network traffic, matches it against a collection contract, runs pm.test assertions, and publishes results to Application Inventory. Covers the Postman side of the integration only — not Playwright test authoring or Playwright failures unrelated to capture. Use when the user asks to "set up application testing," "test my application with Postman," "connect my Playwright tests to Postman," "capture network traffic from Playwright," "Playwright tests are running but nothing shows up in Postman," "why isn't application testing capturing anything," "exclude certain requests from capture," "generate a collection from my app's traffic," "set up application testing in CI," "what API coverage do my e2e tests have," "which APIs are my Playwright tests not hitting."
4
+ ---
5
+
6
+ # Application Test
7
+
8
+ ## Overview
9
+
10
+ **Requires a Playwright project in the repository.** Without Playwright e2e tests there is nothing to capture — Playwright is the only supported runner.
11
+
12
+ `postman application test` does not send its own requests. It:
13
+ 1. Spawns your Playwright test command (e.g. `npx playwright test`)
14
+ 2. The capture plugin intercepts network traffic and writes it as artifacts
15
+ 3. The CLI reads those artifacts, matches each request against your collections, and runs any `pm.test` assertions saved in the matched items
16
+ 4. Publishes results to Application Inventory in your Postman workspace
17
+
18
+ The question it answers: **did my app's actual HTTP calls conform to the collection contract?** Not "does this endpoint respond correctly" — for that, use `postman collection run`.
19
+
20
+ **Application Inventory** is where results are published — a Postman dashboard that records each run's Playwright test results alongside captured/matched/assertion counts, giving you a combined view of test health and API contract conformance across runs.
21
+
22
+ ---
23
+
24
+ ## Prerequisites
25
+
26
+ **All `postman app` commands must be run from the repository root** — that is where `postman.config.cjs`, `.postman/resources.yaml`, and `postman/collections/` are expected to live.
27
+
28
+ Assess each prerequisite against the current state of the repository before taking any action. Most will already be satisfied in an established project; only act on what is actually missing.
29
+
30
+ ### Playwright project
31
+
32
+ This skill applies only when the repository has Playwright e2e tests. Verify that a `playwright.config.*` file exists and `@playwright/test` is listed as a project dependency. If neither is present, stop — there is nothing to capture and this skill does not cover Playwright setup.
33
+
34
+ ### Authentication
35
+
36
+ Publishing results to Postman requires credentials. Authenticate only when workspace publishing is needed:
37
+
38
+ - If `POSTMAN_API_KEY` is in the environment, run `postman login --with-api-key "$POSTMAN_API_KEY"`. This is the expected path in CI and agentic contexts.
39
+ - If the key is not available, surface the gap to the user — do not initiate `postman login` in an agentic context, as it requires a human to complete a browser flow.
40
+
41
+ If this run does not need to publish results (local debugging, dry run), authentication can be skipped by passing `--report-events false`.
42
+
43
+ ### Workspace link
44
+
45
+ Publishing to Application Inventory requires a workspace binding. Check `.postman/resources.yaml` for a `workspace.id`. Reason about whether publishing is actually needed for this task before treating a missing workspace as a blocker:
46
+
47
+ - If publishing is needed and the workspace is missing, load the `bootstrap` skill to set it up. In a non-interactive context this must be done locally first, then `.postman/resources.yaml` committed.
48
+ - If publishing is not needed, pass `--report-events false` and proceed.
49
+
50
+ ### Capture plugin
51
+
52
+ Network traffic capture requires the `postman-playwright` package to be installed and the Playwright config wrapped with `withPostman()`. Before attempting any setup, check whether it is already configured — `postman app setup-capture --dry-run` shows the current state without touching files.
53
+
54
+ Decide what action (if any) is needed based on what the dry run reports:
55
+
56
+ - Already configured → no action needed.
57
+ - Not configured → run `postman app setup-capture --yes`. If automatic patching fails (non-standard config export patterns), fall back to manual installation: install `postman-playwright` as a dev dependency and wrap the config export with `withPostman()`.
58
+
59
+ **Legacy pattern:** If the project uses `attachNetworkCapture()` in spec files instead of `withPostman()`, that is still supported — do not re-run setup. The CLI detects it automatically. Only migrate to `withPostman()` if the user asks to upgrade.
60
+
61
+ ### Server reachability
62
+
63
+ `postman app test` spawns Playwright, which drives the live application — servers must be running before the command starts.
64
+
65
+ Check `playwright.config.*` for a `webServer` entry. If present unconditionally, Playwright manages server startup automatically. If absent or conditional (e.g. gated on an env var), verify the servers are reachable before running. If they are not, check `webServer.command`, `package.json` scripts, and any README for the startup command, and surface it to the user.
66
+
67
+ ---
68
+
69
+ ## Running
70
+
71
+ Before constructing any command, run `postman app test --help` to confirm the available flags for the installed CLI version. Do not guess flag names.
72
+
73
+ **No `postman.config.cjs` yet** — the first run generates and saves `postman.config.cjs`, writing these choices as the defaults for all future runs. Confirm both with the user before running:
74
+
75
+ - *Test command*: check `package.json` scripts for a project-specific test command (e.g. `npm run test:e2e`). Present what you find and ask the user to confirm — the CLI defaults to `npx playwright test` when a Playwright config is found, but do not assume that is what they want.
76
+ - *Collection*: check `postman/collections/` and list what is available. Ask the user which collection to use. If none exist, ask whether to generate one from captured traffic.
77
+
78
+ Once confirmed, run `postman app test` with the appropriate flags (e.g. `--target-collection <path>` or `--generate-collection`) — the CLI generates and saves `postman.config.cjs` automatically. Do not write the config file manually.
79
+
80
+ Running `postman app test` without a collection in a non-interactive context exits with code 2 unless a collection source is provided explicitly.
81
+
82
+ **`postman.config.cjs` exists** — the config is the source of truth. Run `postman app test` without extra flags. Individual config values can be overridden per-run without editing the file; consult `postman app test --help` for the override flags.
83
+
84
+ **Capture-only mode** — when the goal is to stage and review traffic before committing a collection manually, there is a mode that skips collection matching and writes captured traffic as a collection to disk. Use this only as a review step, not the normal run path.
85
+
86
+ ---
87
+
88
+ ## Config File (`postman.config.cjs`)
89
+
90
+ Lives at the **repository root**. The generated file includes comments explaining every field — read it for the full reference. It stays current with the CLI version installed in the repo.
91
+
92
+ To edit config for common needs:
93
+ - **Change the test command** — update `command`
94
+ - **Add or change a collection** — update `targets.default.collections`
95
+ - **Exclude noisy requests** — add `filters` (urlPatterns, methods, headers)
96
+ - **Rewrite URLs before matching** — add `transformers` (config entries that normalize staging/proxy prefixes before the CLI matches captured URLs against the collection)
97
+ - **Multiple named environments** — add entries under `targets`
98
+
99
+ A **target** is a named set of collections + optional environment. The `default` target is used when no `--target` flag is passed. Individual values can be overridden per-run without editing config — consult `postman app test --help` for the override flags.
100
+
101
+ ---
102
+
103
+ ## Debugging
104
+
105
+ **If a flag in this skill is unrecognized** (e.g. `--yes`, `--generate-collection`, `--dry-run`) or the behaviour described here doesn't match what you see, check the CLI version first:
106
+ ```bash
107
+ postman --version
108
+ ```
109
+ This skill targets the latest CLI release. If the version is outdated, load the `bootstrap` skill for installation and update steps, then re-run. The `--help` output for each command shows the flags actually available in the installed version.
110
+
111
+ Read the CLI error output first — it names the specific failure and what to fix.
112
+
113
+ ### Requests show "not matched" in the summary table
114
+
115
+ Run with `--verbose --no-report-events` to see the unmatched URLs without publishing a diagnostic run to Application Inventory. Then:
116
+
117
+ - Wrong collection selected — check `targets.default.collections` in config or override with the target-collection flag.
118
+ - URLs have a staging prefix or proxy path — add a `transformers` entry in `postman.config.cjs` to rewrite them before matching.
119
+
120
+ ---
121
+
122
+ > **Note — Exit code:** The CLI exits with the **test runner's exit code** (Playwright's), not the Postman assertion pass/fail count. A run where Postman assertions fail but all Playwright `expect()` calls pass will exit 0. If you need CI to fail on Postman assertion failures, check the summary table output or the published results.
123
+
124
+ ---
125
+
126
+ ## Critical Rules
127
+
128
+ 1. **`postman application test` captures traffic; it does not drive requests.** The test runner (Playwright) drives the app; this command observes and asserts. If no test runner is running, nothing is captured.
129
+ 2. **`not matched` is not a test failure — it is a coverage gap.** Requests the CLI cannot find in a collection run no assertions and do not affect the exit code. A high not-matched count means the collection is incomplete for the traffic the app generates.
130
+ 3. **Transformers run before matching, not before capture.** The raw URL is stored; the transformer rewrites it for the matching step only. Use this to normalise staging/prod URL differences without losing the original for debugging.
131
+ 4. **Results are published per run.** Each invocation creates a new entry in Application Inventory. Pass `--report-events false` for dry runs or local debugging you do not want recorded.
132
+
133
+ ## Anti-patterns
134
+
135
+ - **Capture-only mode is for reviewing traffic before committing to a collection** — use it when you want to inspect what the app generates before deciding on a collection structure. It is not a substitute for `--generate-collection`, which generates, tests, and saves to config in one step when no collection is configured.
136
+ - **Do not run `postman app init`** — it is deprecated. `postman app test` runs the same setup on first use.
137
+ - **Do not manually write `postman.config.cjs`** — the CLI generates it automatically on first run when you pass `--target-collection` or `--generate-collection`. Writing it by hand bypasses the CLI's generation logic.
138
+ - **Do not combine capture-only mode with a target collection** — capture-only skips collection matching entirely; passing a collection alongside it has no effect and signals a misunderstanding of the mode.
139
+
140
+ ## Verification
141
+
142
+ State captured / matched / not-matched / assertion counts from the summary table. Note whether results were published to Postman (`View results:` URL present) or kept local. If debugging a "not captured" issue, confirm which capture format was detected (Playwright JSON report or legacy NDJSON) and which directory the CLI scanned.
143
+
144
+ Postman docs: https://learning.postman.com/docs/postman-cli/postman-cli-application/#postman-app-test
@@ -11,6 +11,17 @@ in a PR. A workspace binding connects that work to maintained shared context.
11
11
  Setup is useful when it resolves a missing capability or binding; a local API
12
12
  check does not need a new workspace merely to proceed.
13
13
 
14
+ ## Start with postman init
15
+
16
+ `postman init` is idempotent and safe to run in any project, including one
17
+ already initialized. You can run it before API engineering work to scaffold the
18
+ relevant `postman/` directories and make Postman skills available in
19
+ `postman/skills` to guide the task. It establishes an agent-scoped session for
20
+ local work without signing in; accessing existing workspace resources still
21
+ requires authentication.
22
+ See "Interpret initialization without losing partial success" below for flags
23
+ and outcomes; creating or connecting a workspace depends on the task.
24
+
14
25
  ## Resolve only the missing setup
15
26
 
16
27
  Inspect `postman --version`, `.postman/resources.yaml`, and the relevant