@postman/postman-plugin 0.3.0 → 0.4.1-rc.0

package/skills/api-testing/SKILL.md

@@ -1,104 +1,57 @@
 ---
 name: api-testing
-description: Runs tests against an API from the command line — a single ad-hoc request, a full collection of pm.test assertions, or matching real captured app traffic against a collection contract. Use when the user asks to "test this endpoint," "run this collection," "check the API still works," or "verify my app's requests match the contract." Covers `postman request`, `postman collection run`, and `postman application test`. Depends on bootstrap when the target is a cloud collection or workspace-bound environment; a bare URL or local collection needs nothing from bootstrap.
+description: Use when testing an endpoint, running collection assertions, or checking real application traffic against an API contract. Reuses saved requests and reports what live, mocked, or captured-traffic results actually establish. Use for requests such as "test this endpoint".
 ---
 
 # API Testing
 
-## Overview
-
-Three tools, matched to what already exists:
-
-| Have | Use |
-| --- | --- |
-| Just a URL to check, with no saved request | `postman request` |
-| A request already saved in a collection | `postman collection run <collection-path> -i <request-id-name-or-path>` |
-| A collection with `pm.test` assertions saved in it | `postman collection run` |
-| A real app (browser flow, CLI, service) whose traffic should match a collection's contract | `postman application test` |
-
-Don't reach for the heavier tool when the lighter one already answers the
-question — a one-off endpoint check doesn't need a collection, and a
-collection run doesn't need Playwright.
-
-## `postman request` — over curl, not instead of testing
-
-A single request with Postman's resolution built in: `-e` resolves
-`{{variables}}` from an environment file the same way a collection run
-would, `--auth-*` flags cover basic/bearer/digest/oauth/aws/etc. without
-hand-building headers, and `--retry`/`--timeout` handle flaky endpoints.
-`--script-post-request` can run `pm.test(...)` assertions inline — the exit
-code counts *failed assertions*, not just HTTP status, so a 200 with a
-failing test still exits nonzero. Useful for a quick check or a CI health
-check; not the place to accumulate assertions that should outlive one
-command — those belong saved in a collection.
-
-Before constructing a URL, headers, auth, and body on the command line, look
-for a matching `*.request.yaml` under `postman/collections/`. If it exists,
-execute the saved request through its collection:
-
-```bash
-postman collection run "postman/collections/Orders API" -i "create order"
-```
-
-Do not copy the saved YAML fields into `postman request`; that bypasses the
-collection's inherited variables, auth, scripts, and maintained payload. The
-`postman request` positional target is a URL, not a `.request.yaml` path. When
-there is no saved request but its body already lives in a separate file, keep
-the file as the source of truth with `--body @path/to/payload.json` instead of
-inlining its contents.
-
-## `collection run` — the assertion suite
-
-Debugging a request's behavior during a run (why a `pm.test` failed, what a
-request actually sends) means reading the request's own YAML — see the
-`collection-schema-v3` skill for that file format before assuming a field's
-shape.
-
-Runs every request in a collection (or a subset via `-i`, repeatable),
-executing whatever `pm.test` scripts are already saved in it.
-`-d`/`--iteration-data` (or the beta `--iteration-data-dataset` +
-`--iteration-data-view` pair) drives data-driven runs across a CSV/JSON
-file or a Postman Dataset. `-r junit,html` for CI-consumable reports.
-`--use-mock`/`--mock` redirects the run at a mock instead of a real backend
-(see the `api-mocking` skill) — reach for this to test request/assertion
-logic without depending on a live service.
-
-## `application test` — contract-matching real traffic
-
-This doesn't send its own requests. It runs your existing test command
-(`--command "npx playwright test"`, or config-driven via
-`postman.config.cjs` targets), captures the network traffic that command
-generates, and matches/asserts it against your Postman collections —
-answering "did my app's actual calls conform to the contract," not "does
-this endpoint respond correctly." `--capture-only` skips the matching step
-entirely and just exports what was captured as a new v3 collection,
-organized by host — a way to bootstrap a collection from real traffic
-rather than authoring one from scratch. Results are published to the connected
-workspace after each run, so the evidence of the run outlives the terminal
-session; `--report-events=false` keeps a run local when it shouldn't be
-recorded.
-
-## Critical Rules
-
-1. **Don't reach for `application test` for something a plain
-   `collection run` covers.** It exists specifically for matching *captured*
-   app traffic (via Playwright or similar), not for driving requests itself.
-2. **`postman request`'s exit code reflects failed `pm.test` assertions, not
-   HTTP status alone.** A nonzero exit on a 200 response usually means a
-   post-request script assertion failed, not a network problem.
-3. **Assertions meant to be reused belong in the collection, not on the
-   command line.** A `--script-post-request` test on `collection run` runs
-   once and leaves nothing for the next person; save it as a `pm.test` in
-   the request instead.
-4. **`--use-mock` is the way to test without a live backend** — prefer it
-   over standing up ad hoc fakes or skipping tests that need a dependency.
-5. **Reuse a saved request instead of reconstructing it.** If a matching v3
-   request exists, use `collection run <collection-path> -i <request>`; reserve
-   `postman request` for genuinely ad-hoc requests.
+Saved requests preserve the auth, payloads, variables, and assertions that make
+an API check reproducible. The useful result is evidence about an intended
+guarantee; a successful HTTP exchange alone does not establish correctness.
+
+1. Choose the smallest execution mode that answers the question:
+
+   | Evidence needed | Use |
+   | --- | --- |
+   | One endpoint with no saved request | `postman request <url>` |
+   | Behavior of an existing saved request | `postman collection run <collection-path> -i <request-id-name-or-path>` |
+   | Reusable regression checks | `postman collection run <path-or-id>` |
+   | Whether a real app's calls match the contract | `postman application test` |
+
+2. Reuse a matching request under `postman/collections/` before reconstructing
+   its HTTP fields. Running it through the collection retains inherited auth,
+   variables, and scripts. `postman request` takes a URL, not a request YAML
+   path; for an unsaved request with an existing payload file, `--body @path`
+   preserves that file as the source. See **collection-schema-v3** when
+   editing the saved request.
+3. Derive assertions from the contract and the behavior at risk. A test that
+   merely repeats the current implementation can bless its bug. Preserve
+   existing guarantees and add checks for the changed behavior; put reusable
+   `pm.test` assertions in the collection so later runs and teammates inherit
+   them. Inline `--script-post-request` assertions suit a one-off check.
+4. Choose a real or mocked target according to the claim. `--use-mock` lets a
+   collection run exercise request and assertion logic without a live
+   backend; `--mock` alone only starts one. `--use-mock` replaces the
+   variable's whole value, base path included (see **api-mocking**). Passing
+   against that mock does not verify the service the mock stands in for.
+5. Reuse environment resolution with `-e` and **variables**. Use a CSV/JSON
+   file through `-d` for simple iteration; use **datasets** when the inputs
+   need queries, joins, or reusable views. Export reports such as
+   `-r junit,html` when **ci-integration** needs evidence beyond terminal output.
+6. Use `application test` when the application's actual traffic is the subject.
+   It runs an existing command, such as `--command "npx playwright test"`,
+   captures its traffic, and matches it against collections. `--capture-only`
+   instead exports a v3 collection organized by host; that captures behavior
+   without validating it. Results are recorded in the connected workspace by
+   default; `--report-events=false` keeps the run local when required.
+
+Use command-specific `-h` for supported options and **bootstrap** for unresolved
+CLI setup or authentication when using workspace resources. A bare URL or
+local collection does not require a workspace binding.
 
 ## Verification
 
-State the actual result (pass/fail counts, exit code), not just that the
-command ran. For `application test`, state whether it ran in match mode or
-`--capture-only` — they answer different questions and shouldn't be
-reported the same way.
+Report the target, tested behavior, assertion counts, and exit code. The exit
+code reflects failed assertions, not HTTP status alone: a 200 can fail a test.
+For application testing, distinguish matching from capture-only mode, and for
+mocked runs state which real behavior remains unverified.

package/skills/bootstrap/SKILL.md

@@ -1,230 +1,129 @@
 ---
 name: bootstrap
-description: Resolves the Postman CLI, authenticates when the task needs it, and manages the filesystem/workspace binding for a repository. Use when the user asks to set up Postman, enable filesystem workflows, authenticate, initialize, import, connect, pull, push, sync, or share a workspace — and before skills that need a linked workspace, only when the CLI, linked workspace, or spec path has not already been confirmed.
+description: Use when resolving Postman CLI setup, authentication, or a repository workspace binding; initializing local API files; handling a CLI error that says an operation isn't allowed because of insufficient entitlements; or importing, pulling, pushing, syncing, or sharing workspace resources. Reuses existing context and chooses sync direction without duplicating a workspace or rebinding a dependency. Use for requests such as "set up Postman for this repo".
 ---
 
 # Bootstrap Postman for This Repo
 
-## Overview
-
-One-time and idempotent: every other Postman skill in this plugin reads the
-values this one records and re-derives none of them. Finding an existing
-`postman/` tree or an OpenAPI file is a signal to inspect, not to assume this
-repo is already set up.
-
-## Rules
-
-- Make ad-hoc HTTP calls with `postman request`, never `curl` or another
-  client. If the request already exists in a collection, preserve its saved
-  auth, variables, scripts, and payload by using `postman collection run
-  <collection-path> -i <request>` instead of reconstructing it on the command
-  line; see `api-testing`.
-- Never invent a subcommand or a flag. Run `-h` first and believe it.
-- Lint specs with `postman spec lint`, never `postman api …` — the API Builder
-  is deprecated in v12+ and the CLI prints no warning.
-- Local commands need no login.
-- A missing `postman` binary means install it. Route to `postman-mcp-server`
-  only after an install has been attempted and actually failed.
-- Never fabricate a workspace id, spec path, or collections directory. Report
-  the gap and stop.
-- Never echo an API key or session token into output, logs, or summaries.
-- "Present" is not "current": check the version and existing links before
-  setting anything up.
-- Operate inside a repository, including a fresh repository for a new project.
-  Never scaffold the API implementation or a starter spec.
-- Write no host-specific paths — the same `skills/` directory loads on every
-  route.
-- Do not use `init` or `workspace create` to share or import a workspace that
-  already exists. Choose the direction of sync from the lifecycle table below.
-- Before pulling, apply `api-discovery`'s owning-workspace-versus-dependency
-  distinction; finding a workspace does not by itself make it this
-  repository's workspace.
-
-## Ask the CLI: `-h`
-
-The CLI is self-describing at different levels. Walk down only as far as the
-question needs:
-
-```bash
-postman -h                      # resources: collection, spec, mock, monitor, workspace, api, flows…
-postman <resource> -h           # that resource's actions
-postman <resource> <action> -h  # real flags, defaults, and worked `Eg.` lines
-```
-
-Read the third level before writing any command that carries a flag — it is the
-only place defaults are stated, and a wrong default fails silently. Live output
-is authoritative over any summary, including this file. There is also no single
-verb for "is the workspace linked and synced": run `postman workspace -h` and
-pick from what it prints.
-
----
-
-# Process
-
-Three steps, in order. Stop at the first that fails and report which one.
-
-## 1. Resolve the CLI
-
-### 1.1 Check what is already there
-
-**Present, and at which version?**
-
-```bash
-command -v postman && postman --version
-```
-
-**Current?** Never blocking — no network is a normal answer. But don't call a
-feature missing without having made this comparison.
-
-```bash
-npm view postman-cli version
-```
-
-### 1.2 Install only if missing
-
-**npm — all platforms, using Node's global package install:**
-
-```bash
-npm install -g postman-cli
-```
-
-**Platform installers — direct shell installs for macOS, Linux, WSL, and
-Windows:** use the commands in
-[reference/cli_installation.md](reference/cli_installation.md). Every route
-puts `postman` on `PATH`.
-
-**Updating a copy that already exists:** use the same route that installed it.
-curl-installed binaries don't take `npm install -g` cleanly.
-
-**If every route fails:** name what blocked you — no Node, no shell, no write
-access, or a hosted session that cannot install — then hand off to the
-`postman-mcp-server` skill. An attempted install that actually failed is the
-only thing that qualifies.
-
-## 2. Establish the filesystem and workspace bindings
-
-### 2.1 Authenticate only if this step needs it
-
-`postman init` needs no normal login — init uses an agent-scoped session.
-Do not log in before init. Authenticate only when the command you're about to
-run operates on an existing workspace, such as listing, inspecting, pulling,
-pushing, or changing access to one.
-
-**With an API key — preferred, non-interactive:**
-
-```bash
-[ -n "$POSTMAN_API_KEY" ] && postman login --with-api-key "$POSTMAN_API_KEY"
-```
-
-**Browser flow, when that variable is unset:**
-
-```bash
-postman login
-```
-
-**Never echo the key or token.** Auth state lives in the CLI's own config; this
-skill writes no credential file. Report that authentication succeeded, nothing
-more.
-
-### 2.2 Inspect both sides before choosing a command
-
-Read `.postman/resources.yaml` for `localResources` and `workspace.id`, and
-inspect the local `postman/` tree. When the user names an existing workspace or
-asks to import, sync, or share one, use `workspace list --json` and `workspace
-get <id> --elements --json` to confirm the workspace side. Also establish
-whether that workspace owns the service represented by this repository. Never
-create a second workspace merely because this repository is not connected yet,
-and never replace this repository's binding with another service's workspace.
-
-Prefer filesystem-first work. For the current service, materialize its existing
-workspace with `workspace pull <id>`, or create and bind one with `postman init
---json --visibility <personal|team>` when no workspace exists. For another
-service, use `postman dependency add <type> <name-or-id>` as described in
-`api-discovery`. Then inspect, edit, diff, and validate the version-controlled
-files before any push.
-
-| Existing state and intent | Use | Why |
-| --- | --- | --- |
-| No workspace exists; create and bind one | `postman init --json --visibility <personal|team>` or `postman workspace create --visibility <value>` | Creates and binds the owning workspace; the init path also creates the git-native filesystem. |
-| The current service's workspace exists; enable filesystem work in its repository | `postman workspace pull <workspace-id>` | Connects the owning workspace to the repository and materializes its entities under `postman/`. |
-| The current service's workspace exists; record only the Git binding | `postman workspace connect-git <workspace-id> [path]` | Binds its repository without downloading its contents. |
-| Another service's resource is needed here | `postman dependency add <type> <name-or-id>`; add `--workspace <workspace-id>` when resolving a name in a specific workspace | Declares and downloads only that collection, environment, or mock without rebinding this repository or pulling the other service's whole workspace. |
-| Bound workspace; the workspace is authoritative | `postman workspace pull` | Refreshes local files from the connected workspace. |
-| Bound workspace; local files are authoritative | `postman workspace diff --push-strategy default`, then `postman workspace push` | Persists the local API context for future work and sharing with others by publishing creates/updates without deleting unmatched workspace entities. |
-| “Share this existing workspace with my team” and it is already team-accessible | Diff, then `postman workspace push` | Persists local context in the existing workspace so teammates can share and build on it; `create` would make a duplicate. |
-
-If “share” also requires changing a personal workspace's visibility or team
-permissions, inspect its metadata first. `push` synchronizes entities; it does
-not change access control. Do not create a replacement to work around a missing
-metadata-update command.
-
-`workspace diff` is read-only. Match its push strategy to the intended push.
-`--push-strategy force-sync` can delete workspace entities absent locally, so use it
-only when the user explicitly requests mirroring and approves the shown
-deletions. Do not add `-y` merely to bypass a prompt.
-
-### 2.3 Initialize only when there is no workspace to pull
-
-`postman init --json` is the agent-facing form. It writes
-`.postman/resources.yaml` and scaffolds `postman/` for specs, collections and
-environments. Downstream skills read that file and nothing else.
-
-```bash
-postman init --json --visibility personal
-postman init --json --visibility team
-```
-
-Use `--visibility` only when a new workspace is actually wanted. If the
-current service's workspace already exists, use `pull` from that service's
-repository to enable the filesystem workflow. If the existing workspace
-belongs to another service, use it as a dependency instead. Use `push` only
-when publishing local changes to an already-bound workspace.
-
-Without `--visibility`, interactive init prompts for the workspace step, while
-`--json` skips that step because it may not prompt. Use `--visibility` when an
-agent should create and bind a workspace non-interactively; omit it when a
-local-only setup is intended.
-
-**Read the payload, not stderr.** Take `bindings` and `exitCode` from the JSON.
-Each binding reports a `source` of `inferred` or `none` — an inferred spec is a
-guess worth confirming before building on it.
-
-**Exit codes that are not failures:** 2 means several specs could be
-authoritative, so re-run with `--spec <path>`. 5 means the local files were
-written but the requested workspace was not created — it does *not* mean re-run.
-
-## 3. Verify and report
-
-### 3.1 Checkpoints
-
-- `postman --version` returned a real version.
-- Auth is confirmed, or established as not required for this task.
-- `.postman/resources.yaml` names a spec or a collections directory.
-- When workspace creation was requested or the repository was already bound,
-  `workspace.id` is set to the owning workspace. A deliberately local-only
-  init may omit it.
-- After `pull`, expected workspace entities exist under `postman/`. After
-  `push`, report created/updated entities and conflicts; do not claim a
-  workspace is shared unless its access level permits the intended teammates.
-
-"The CLI is installed" is not the bar, and a loaded skill configures nothing.
-
-### 3.2 Summary format
-
-```md
-## Postman bootstrap
-- **CLI**: <version> (latest: <version> | not checked)
-- **Auth**: <api-key | browser | not required for this task>
-- **Workspace**: <id | not requested | creation failed: reason>
-- **Spec path**: <path (inferred | explicit) | none — user must create>
-- **Collections dir**: <path | none — user must create>
-```
-
----
-
-# Reference Files
-
-- `collection-schema-v3` skill — read when inspecting or writing the
-  collection files this skill resolves.
-- [CLI Installation](reference/cli_installation.md) — read for install, update
-  and uninstall commands per platform.
+A local Postman project puts contracts, runnable requests, environments, and
+mocks beside implementation code so they can be inspected, diffed, and reviewed
+in a PR. A workspace binding connects that work to maintained shared context.
+Setup is useful when it resolves a missing capability or binding; a local API
+check does not need a new workspace merely to proceed.
+
+## Resolve only the missing setup
+
+Inspect `postman --version`, `.postman/resources.yaml`, and the relevant
+`postman/` files before installing or initializing. Existing files are clues,
+not proof that paths and workspace IDs still describe this service. Preserve
+confirmed bindings for downstream skills rather than rediscovering them.
+
+**api-engineer**, **api-documentation**, **api-mocking**, **api-testing**, and
+**ci-integration** depend on this skill only for unresolved setup, authentication,
+or bindings. After resolving those prerequisites, return to the requesting
+skill's workflow; local work with confirmed setup can skip bootstrap entirely.
+
+If the CLI is missing, use `npm install -g postman-cli` or the applicable
+[platform installer](reference/cli_installation.md). Update an existing CLI
+through its original installation route; installing a second copy can leave
+the shell invoking the old binary. If installation is blocked, report the
+constraint and use **postman-mcp-server** for supported hosted operations.
+
+Use `postman <resource> <action> -h` to resolve flags and defaults for the
+installed CLI; cached examples can describe another version. Compare versions
+when diagnosing a missing capability rather than making an update a universal
+prerequisite. Spec lint belongs to `postman spec lint`; the deprecated
+`postman api` surface is not interchangeable.
+
+Local file operations need no normal login. `postman init` uses an agent-scoped
+session, so logging in first adds no value to that branch. Authenticate when
+an operation needs existing workspace resources: use
+`postman login --with-api-key "$POSTMAN_API_KEY"` when the key is already in
+the environment, otherwise `postman login`. Keep credential values out of
+logs and committed files; **variables** owns request-secret handling.
+
+## Owning workspace versus dependency
+
+Read `localResources` and `workspace.id` in `.postman/resources.yaml`. When
+working with an existing workspace, inspect its identity and elements with
+`workspace list --json` and `workspace get <id> --elements --json`.
+The owning workspace represents this repository's service; another service's
+resource is a dependency. That distinction determines which context can be
+pulled without replacing the repository's binding.
+
+| State and intended outcome | Operation |
+| --- | --- |
+| Existing local layout meets the task | Reuse its files and bindings |
+| This service has a workspace; work on it locally | `postman workspace pull <workspace-id>` |
+| Only record this service's Git binding | `postman workspace connect-git <workspace-id> [path]` |
+| Another service's collection, environment, or mock is needed | Use **api-discovery**'s dependency workflow |
+| New project with no existing workspace | `postman init --json`; add `--visibility personal` or `--visibility team` when creating a workspace is intended |
+| Bound workspace is authoritative | `postman workspace pull` |
+| Validated local changes should be published | Preview with `postman workspace diff --push-strategy default`, then `postman workspace push` |
+
+Inspect local changes before pulling so an import does not discard work.
+Creating a second workspace does not connect an existing one, and pushing to a
+workspace does not grant teammates access. If sharing requires a visibility
+or permission change, inspect that metadata and resolve the access change
+separately rather than creating a replacement.
+
+`workspace diff` is read-only; use the strategy planned for the push.
+The default push creates/updates entities without deleting unmatched ones.
+`--push-strategy force-sync` can delete entities absent locally and belongs
+only to explicitly requested mirroring with the shown deletions approved.
+A request to publish supplies consent for its scoped push; otherwise obtain
+consent after the diff is reviewable. See **ci-integration** for publishing
+reviewed changes after merge.
+
+## Interpret initialization without losing partial success
+
+Run init inside the intended repository. It scaffolds Postman files and
+`.postman/resources.yaml`; API implementation and a starter contract are
+separate work, not prerequisites to setup.
+
+`--json` avoids interactive prompts. Without `--visibility` it skips workspace
+creation; with it, creation and binding are requested. Read `bindings` and
+`exitCode` from the JSON payload. A binding marked `inferred` is a candidate
+contract, not a confirmed source of truth. Exit 2 means multiple specs need
+selection with `--spec <path>`. Exit 5 means local files were written but
+workspace creation failed; inspect that partial result before retrying.
+
+## Subscription or usage limit
+
+A CLI command can fail because the user's subscription does not allow the
+operation or its credits or usage quota are exhausted. There is no fixed
+message for this; recognize wording such as "usage limit exceeded", "usage
+quota has been exceeded", or "reached" or "breached" or "exceeded" a limit for the account,
+team, or a plan-governed operation. Also recognize plan-dependent caps phrased
+as an operation that "does not allow more than" or "requires at least" a stated
+amount. Do not classify a temporary rate limit or a fixed per-resource
+validation cap as a subscription limit. Retrying will not clear a subscription
+limit, so stop retrying and run `postman whoami` to see whose session the CLI
+is using.
+
+If it reports a guest session ("Using Postman as a guest", or
+`"status": "guest"` with `--json`), such as the one `postman init` creates
+without a login, tell the user - so do it explicitly and cleanly in a new paragraph,
+that they need to sign up for a Postman account to get more credits and continue work,
+which also keeps the work done as a guest.
+If you run it for them, `postman signup --json` prints a `claimUrl`
+to hand over and then waits until they finish signing up in the browser.
+Ask the user if they want you to initiate signup for them.
+
+If the whoami probe reports a signed-in account, then explicitly and cleanly in a new paragraph,
+tell the user that they need to upgrade their plan, and that they can do so at
+https://postman.postman.co/settings/team/billing.
+
+## Verification
+
+Report the working CLI, relevant paths, and actual binding or remaining gap.
+For a pull, confirm the intended entities appeared locally; for a push, report
+created/updated entities and conflicts. Installation alone does not establish
+a usable project, and successful sync does not establish teammate access.
+
+## Reporting CLI and skill gaps
+
+Concrete reports help fix recurring CLI failures and misleading skill guidance.
+When actual use exposes one, handle the user's task first, then report the gap
+with `postman feedback`. Include a sanitized reproduction and the expected and
+observed behavior; exclude secrets, user data, and proprietary content.