@powerduck/openapi-cli 0.2.1 → 0.2.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +153 -626
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -1,511 +1,68 @@
|
|
|
1
1
|
# @powerduck/openapi-cli
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
[](https://www.npmjs.com/package/@powerduck/openapi-cli)
|
|
4
|
+
[](https://github.com/PowerDuckie/openapi-cli/blob/main/LICENSE)
|
|
5
|
+
[](https://www.npmjs.com/package/@powerduck/openapi-cli)
|
|
4
6
|
|
|
5
|
-
|
|
7
|
+
CI-ready command-line tool for batch-testing OpenAPI 3.2 documents across six protocols. Runs every operation in your spec, runs assertions against each response, and produces JSON, CLI, and HTML reports.
|
|
6
8
|
|
|
7
9
|
---
|
|
8
10
|
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
- **Multi-protocol out of the box** — HTTP, SSE (auto-detected), WebSocket, GraphQL, gRPC (all four streaming modes), and MCP (Streamable HTTP + stdio)
|
|
12
|
-
- **Two assertion styles** — declarative `x-tests` arrays and full Postman `pm.test()` / `pm.expect()` scripts
|
|
13
|
-
- **Three report formats** — machine-readable JSON, colored CLI output, and a self-contained interactive HTML report
|
|
14
|
-
- **Local or remote specs** — load an OpenAPI document from a file path or an `http(s)://` URL
|
|
15
|
-
- **CI-friendly** — non-zero exit on failure, configurable timeouts, bounded concurrency, `.env` support
|
|
16
|
-
- **Flexible filtering** — run a subset by method, path, tag, or operationId
|
|
17
|
-
- **Minimal runtime dependencies** — `@powerduck/openapi-request`, `commander`, and `@apidevtools/json-schema-ref-parser` for `$ref` resolution
|
|
18
|
-
|
|
19
|
-
---
|
|
20
|
-
|
|
21
|
-
## Installation
|
|
22
|
-
|
|
23
|
-
### Global
|
|
24
|
-
|
|
25
|
-
```bash
|
|
26
|
-
npm install -g @powerduck/openapi-cli
|
|
27
|
-
```
|
|
28
|
-
|
|
29
|
-
### Project-local (recommended for CI)
|
|
30
|
-
|
|
31
|
-
```bash
|
|
32
|
-
npm install --save-dev @powerduck/openapi-cli
|
|
33
|
-
```
|
|
34
|
-
|
|
35
|
-
### npx (one-off)
|
|
36
|
-
|
|
37
|
-
```bash
|
|
38
|
-
npx @powerduck/openapi-cli --spec openapi.json
|
|
39
|
-
```
|
|
11
|
+
Powerduck is an open-source developer tooling platform for teams building modern API workflows.
|
|
40
12
|
|
|
41
|
-
**
|
|
13
|
+
- **6 Protocols** — HTTP/1.1, HTTP/2, HTTP/3, WebSocket, SSE, and gRPC
|
|
14
|
+
- **Batch Testing** — Run every operation in your OpenAPI spec concurrently
|
|
15
|
+
- **Assertion Engine** — Validate status codes, headers, response bodies, and schemas
|
|
16
|
+
- **3 Report Formats** — JSON for CI, CLI for terminal, HTML for dashboards
|
|
17
|
+
- **Authentication** — Bearer tokens, API keys, Basic auth, OAuth2, and custom headers
|
|
18
|
+
- **Filtering** — Run tests by tag, path, method, operationId, or regex
|
|
19
|
+
- **Concurrency Control** — Configurable parallelism with rate limiting
|
|
20
|
+
- **gRPC Reflection** — Auto-discover gRPC services via server reflection
|
|
21
|
+
- **MCP Integration** — Streamable HTTP and stdio MCP transport support
|
|
22
|
+
- **GitHub Actions** — First-class CI integration with annotations and summaries
|
|
42
23
|
|
|
43
24
|
---
|
|
44
25
|
|
|
45
26
|
## Quick Start
|
|
46
27
|
|
|
47
|
-
|
|
48
|
-
# 1. Test a local OpenAPI document against its first server URL
|
|
49
|
-
openapi-cli --spec openapi.json
|
|
50
|
-
|
|
51
|
-
# 2. Override the server and write reports to a custom directory
|
|
52
|
-
openapi-cli --spec openapi.json \
|
|
53
|
-
--server https://api.staging.example.com \
|
|
54
|
-
--output ./reports
|
|
55
|
-
|
|
56
|
-
# 3. Test a remote spec, only GET requests, 10 concurrent workers
|
|
57
|
-
openapi-cli --spec https://docs.example.com/openapi.json \
|
|
58
|
-
--method get \
|
|
59
|
-
--concurrency 10
|
|
60
|
-
|
|
61
|
-
# 4. Use a config file for complex setups
|
|
62
|
-
openapi-cli --config openapi-cli.config.json
|
|
63
|
-
```
|
|
64
|
-
|
|
65
|
-
---
|
|
66
|
-
|
|
67
|
-
## CLI Reference
|
|
68
|
-
|
|
69
|
-
```
|
|
70
|
-
openapi-cli [options]
|
|
71
|
-
```
|
|
72
|
-
|
|
73
|
-
### Core
|
|
74
|
-
|
|
75
|
-
| Option | Short | Default | Description |
|
|
76
|
-
|--------|-------|---------|-------------|
|
|
77
|
-
| `--spec <path-or-url>` | `-s` | **required** | Path or `http(s)://` URL to an OpenAPI 3.2 JSON document |
|
|
78
|
-
| `--server <url>` | `-S` | *(from spec)* | Override the server base URL |
|
|
79
|
-
| `--output <dir>` | `-o` | `./openapi-cli-report` | Directory for generated reports |
|
|
80
|
-
| `--format <formats>` | `-f` | `json,cli,html` | Comma-separated report formats: `json`, `cli`, `html` |
|
|
81
|
-
| `--config <path>` | `-c` | — | Path to a JSON config file (CLI args take precedence) |
|
|
82
|
-
|
|
83
|
-
### Execution
|
|
84
|
-
|
|
85
|
-
| Option | Short | Default | Description |
|
|
86
|
-
|--------|-------|---------|-------------|
|
|
87
|
-
| `--concurrency <n>` | `-n` | `5` | Maximum number of concurrent requests |
|
|
88
|
-
| `--timeout <ms>` | `-t` | `30000` | Per-request timeout in milliseconds (also applies to remote spec fetch) |
|
|
89
|
-
| `--no-fail-on-error` | — | off | Exit with code 0 even when tests fail |
|
|
90
|
-
|
|
91
|
-
### Filtering
|
|
92
|
-
|
|
93
|
-
| Option | Default | Description |
|
|
94
|
-
|--------|---------|-------------|
|
|
95
|
-
| `--method <methods>` | — | Comma-separated HTTP methods to include (e.g. `get,post`) |
|
|
96
|
-
| `--path <patterns>` | — | Comma-separated regex patterns matched against the path |
|
|
97
|
-
| `--tag <tags>` | — | Comma-separated OpenAPI tags to include |
|
|
98
|
-
| `--operation-id <ids>` | — | Comma-separated operationIds to include |
|
|
99
|
-
|
|
100
|
-
### Network & TLS
|
|
101
|
-
|
|
102
|
-
| Option | Short | Default | Description |
|
|
103
|
-
|--------|-------|---------|-------------|
|
|
104
|
-
| `--proxy <url>` | `-p` | — | HTTP(S) proxy URL (e.g. `http://proxy.corp:8080`) |
|
|
105
|
-
| `--ca <path>` | — | — | Path to a CA certificate bundle (PEM) |
|
|
106
|
-
| `--cert <path>` | — | — | Path to a client certificate (PEM) for mTLS |
|
|
107
|
-
| `--key <path>` | — | — | Path to a client private key (PEM) for mTLS |
|
|
108
|
-
| `--insecure` | `-k` | off | Skip TLS certificate verification |
|
|
109
|
-
|
|
110
|
-
### Authentication & Headers
|
|
111
|
-
|
|
112
|
-
| Option | Short | Default | Description |
|
|
113
|
-
|--------|-------|---------|-------------|
|
|
114
|
-
| `--header <headers...>` | `-H` | — | Extra request headers, `Key: Value` format. Repeatable. |
|
|
115
|
-
| `--bearer <token>` | — | — | Bearer token for the `Authorization` header |
|
|
116
|
-
| `--variable <vars...>` | `-v` | — | Postman variables, `KEY=VALUE` format. Repeatable. |
|
|
117
|
-
| `--env <path>` | — | — | Path to a `.env` file (does not override existing env vars) |
|
|
118
|
-
|
|
119
|
-
### gRPC
|
|
120
|
-
|
|
121
|
-
| Option | Default | Description |
|
|
122
|
-
|--------|---------|-------------|
|
|
123
|
-
| `--grpc-no-reflection` | off | Disable gRPC server reflection; use proto files instead |
|
|
124
|
-
| `--grpc-proto <paths...>` | — | Paths to `.proto` files or directories containing them |
|
|
125
|
-
|
|
126
|
-
### MCP
|
|
127
|
-
|
|
128
|
-
| Option | Default | Description |
|
|
129
|
-
|--------|---------|-------------|
|
|
130
|
-
| `--mcp-transport <transport>` | `streamable-http` | MCP transport: `streamable-http` or `stdio` |
|
|
131
|
-
| `--mcp-command <command>` | — | MCP stdio command (e.g. `npx`) |
|
|
132
|
-
| `--mcp-args <args>` | — | MCP stdio arguments, space- or comma-separated |
|
|
133
|
-
| `--mcp-cwd <dir>` | — | Working directory for the MCP stdio subprocess |
|
|
134
|
-
|
|
135
|
-
### Informational
|
|
136
|
-
|
|
137
|
-
| Option | Short | Description |
|
|
138
|
-
|--------|-------|-------------|
|
|
139
|
-
| `--help` | `-h` | Show help |
|
|
140
|
-
| `--version` | `-V` | Show version |
|
|
141
|
-
|
|
142
|
-
---
|
|
143
|
-
|
|
144
|
-
## Configuration File
|
|
145
|
-
|
|
146
|
-
A JSON config file can be used instead of (or in addition to) CLI arguments. CLI arguments always take precedence.
|
|
147
|
-
|
|
148
|
-
```json
|
|
149
|
-
{
|
|
150
|
-
"specPath": "./openapi.json",
|
|
151
|
-
"serverUrl": "https://api.staging.example.com",
|
|
152
|
-
"outputDir": "./reports",
|
|
153
|
-
"formats": ["json", "html"],
|
|
154
|
-
"concurrency": 10,
|
|
155
|
-
"timeout": 15000,
|
|
156
|
-
"failOnError": true,
|
|
157
|
-
"proxy": "http://proxy.corp:8080",
|
|
158
|
-
"tls": {
|
|
159
|
-
"strictSSL": true,
|
|
160
|
-
"caCert": "./ca-bundle.pem",
|
|
161
|
-
"clientCert": "./client.crt",
|
|
162
|
-
"clientKey": "./client.key"
|
|
163
|
-
},
|
|
164
|
-
"headers": {
|
|
165
|
-
"X-API-Key": "${API_KEY}",
|
|
166
|
-
"X-Environment": "staging"
|
|
167
|
-
},
|
|
168
|
-
"auth": {
|
|
169
|
-
"type": "bearer",
|
|
170
|
-
"token": "${BEARER_TOKEN}"
|
|
171
|
-
},
|
|
172
|
-
"variables": {
|
|
173
|
-
"host": "api.staging.example.com"
|
|
174
|
-
},
|
|
175
|
-
"filter": {
|
|
176
|
-
"methods": ["get", "post"],
|
|
177
|
-
"tags": ["v2", "public"],
|
|
178
|
-
"paths": ["^/api/v2"],
|
|
179
|
-
"operationIds": ["getUser", "createUser"]
|
|
180
|
-
},
|
|
181
|
-
"grpcReflection": true,
|
|
182
|
-
"grpcProtoPaths": ["./proto"],
|
|
183
|
-
"mcpTransport": "streamable-http",
|
|
184
|
-
"mcpCommand": "npx",
|
|
185
|
-
"mcpArgs": ["-y", "@modelcontextprotocol/server-everything"],
|
|
186
|
-
"mcpCwd": "./mcp-servers"
|
|
187
|
-
}
|
|
188
|
-
```
|
|
189
|
-
|
|
190
|
-
### Config field reference
|
|
191
|
-
|
|
192
|
-
| Field | Type | Description |
|
|
193
|
-
|-------|------|-------------|
|
|
194
|
-
| `specPath` | `string` | Local path or remote URL to the OpenAPI spec |
|
|
195
|
-
| `serverUrl` | `string` | Override the server base URL |
|
|
196
|
-
| `outputDir` | `string` | Output directory for reports |
|
|
197
|
-
| `formats` | `string[]` | Report formats: `json`, `cli`, `html` |
|
|
198
|
-
| `concurrency` | `number` | Max concurrent requests |
|
|
199
|
-
| `timeout` | `number` | Per-request timeout in ms |
|
|
200
|
-
| `failOnError` | `boolean` | Exit non-zero when tests fail |
|
|
201
|
-
| `proxy` | `string` | HTTP(S) proxy URL |
|
|
202
|
-
| `tls.strictSSL` | `boolean` | Verify TLS certificates (default `true`) |
|
|
203
|
-
| `tls.caCert` | `string` | CA certificate bundle path |
|
|
204
|
-
| `tls.clientCert` | `string` | Client certificate path (mTLS) |
|
|
205
|
-
| `tls.clientKey` | `string` | Client private key path (mTLS) |
|
|
206
|
-
| `headers` | `Record<string,string>` | Extra request headers |
|
|
207
|
-
| `auth.type` | `string` | Auth type (currently `bearer`) |
|
|
208
|
-
| `auth.token` | `string` | Bearer token |
|
|
209
|
-
| `variables` | `Record<string,string>` | Postman variables |
|
|
210
|
-
| `filter.methods` | `string[]` | HTTP method filter |
|
|
211
|
-
| `filter.paths` | `string[]` | Path regex filter |
|
|
212
|
-
| `filter.tags` | `string[]` | OpenAPI tag filter |
|
|
213
|
-
| `filter.operationIds` | `string[]` | operationId filter |
|
|
214
|
-
| `grpcReflection` | `boolean` | Use gRPC server reflection |
|
|
215
|
-
| `grpcProtoPaths` | `string[]` | Proto file/directory paths |
|
|
216
|
-
| `mcpTransport` | `string` | `streamable-http` or `stdio` |
|
|
217
|
-
| `mcpCommand` | `string` | MCP stdio command |
|
|
218
|
-
| `mcpArgs` | `string[]` | MCP stdio arguments |
|
|
219
|
-
| `mcpCwd` | `string` | MCP stdio working directory |
|
|
220
|
-
|
|
221
|
-
---
|
|
222
|
-
|
|
223
|
-
## Spec Sources
|
|
224
|
-
|
|
225
|
-
### Local file
|
|
28
|
+
### Install
|
|
226
29
|
|
|
227
30
|
```bash
|
|
228
|
-
|
|
31
|
+
npm install -g @powerduck/openapi-cli
|
|
229
32
|
```
|
|
230
33
|
|
|
231
|
-
###
|
|
34
|
+
### Run tests from the CLI
|
|
232
35
|
|
|
233
36
|
```bash
|
|
234
|
-
openapi-cli --spec https://
|
|
235
|
-
```
|
|
236
|
-
|
|
237
|
-
Remote specs are fetched with:
|
|
238
|
-
- A configurable timeout (`--timeout`, default 30s)
|
|
239
|
-
- Up to 5 automatic redirects
|
|
240
|
-
- A descriptive error on HTTP failure (status code + message)
|
|
241
|
-
- JSON validation after download
|
|
242
|
-
|
|
243
|
-
The `User-Agent` header is set to `@powerduck/openapi-cli`.
|
|
244
|
-
|
|
245
|
-
---
|
|
246
|
-
|
|
247
|
-
## $ref Resolution
|
|
248
|
-
|
|
249
|
-
Internal `$ref` pointers (e.g. `#/components/schemas/User`) are automatically resolved before test execution. This means request body schemas, response schemas, and path items that reference `components` are fully inlined.
|
|
250
|
-
|
|
251
|
-
- **Supported**: internal refs to any location in the same document (`#/...`)
|
|
252
|
-
- **Not resolved**: external file or URL refs (`./other.json#/...`, `https://...`) — these are left as-is
|
|
253
|
-
- **Circular refs**: safely ignored (no infinite recursion)
|
|
254
|
-
|
|
255
|
-
Resolution is powered by [`@apidevtools/json-schema-ref-parser`](https://www.npmjs.com/package/@apidevtools/json-schema-ref-parser).
|
|
256
|
-
|
|
257
|
-
---
|
|
258
|
-
|
|
259
|
-
## Assertions
|
|
260
|
-
|
|
261
|
-
### Declarative assertions (`x-tests`)
|
|
262
|
-
|
|
263
|
-
Add an `x-tests` array to any operation. Each assertion is an object with a `name` and an `assert` type.
|
|
264
|
-
|
|
265
|
-
```json
|
|
266
|
-
{
|
|
267
|
-
"paths": {
|
|
268
|
-
"/users/{id}": {
|
|
269
|
-
"get": {
|
|
270
|
-
"operationId": "getUser",
|
|
271
|
-
"x-tests": [
|
|
272
|
-
{ "name": "status is 200", "assert": "status", "value": 200 },
|
|
273
|
-
{ "name": "content-type is JSON", "assert": "header", "key": "content-type", "contains": "application/json" },
|
|
274
|
-
{ "name": "user id exists", "assert": "jsonPath", "path": "$.id", "exists": true },
|
|
275
|
-
{ "name": "user is active", "assert": "jsonPath", "path": "$.status", "equals": "active" },
|
|
276
|
-
{ "name": "response under 2s", "assert": "responseTime", "max": 2000 },
|
|
277
|
-
{ "name": "body contains email", "assert": "bodyContains", "contains": "@" }
|
|
278
|
-
]
|
|
279
|
-
}
|
|
280
|
-
}
|
|
281
|
-
}
|
|
282
|
-
}
|
|
283
|
-
```
|
|
284
|
-
|
|
285
|
-
#### Assertion types
|
|
286
|
-
|
|
287
|
-
| Type | Required fields | Optional fields | Description |
|
|
288
|
-
|------|----------------|-----------------|-------------|
|
|
289
|
-
| `status` | `value` (number) | — | HTTP status code must equal `value` |
|
|
290
|
-
| `header` | `key` (string) | `contains` (string) | Header must exist; if `contains` is set, the header value must include the substring (case-insensitive) |
|
|
291
|
-
| `bodyContains` | `contains` (string) | — | Response body must include the substring |
|
|
292
|
-
| `bodyEquals` | `body` (string) | — | Response body must exactly equal `body` |
|
|
293
|
-
| `jsonPath` | `path` (string) | `equals`, `exists` (boolean) | Resolve a JSONPath expression (dot notation + array indices). With `equals`, compare the resolved value. With `exists: true`, verify the path resolves. |
|
|
294
|
-
| `responseTime` | `max` (number) | — | Response duration must be <= `max` milliseconds |
|
|
295
|
-
|
|
296
|
-
#### JSONPath syntax
|
|
297
|
-
|
|
298
|
-
Supports dot notation and array indices:
|
|
299
|
-
|
|
300
|
-
- `$.data.id` → `body.data.id`
|
|
301
|
-
- `$.users[0].name` → `body.users[0].name`
|
|
302
|
-
- `data.items[2].value` → leading `$` is optional
|
|
303
|
-
|
|
304
|
-
### Postman scripts (`x-postman-scripts`)
|
|
305
|
-
|
|
306
|
-
Full Postman test scripts are executed by `@powerduck/openapi-request`'s built-in script sandbox. Use the standard `pm.test()` and `pm.expect()` API.
|
|
307
|
-
|
|
308
|
-
```json
|
|
309
|
-
{
|
|
310
|
-
"paths": {
|
|
311
|
-
"/users": {
|
|
312
|
-
"get": {
|
|
313
|
-
"x-postman-scripts": {
|
|
314
|
-
"test": "pm.test('status is 200', () => pm.response.to.have.status(200)); pm.test('users is an array', () => pm.expect(pm.response.json().users).to.be.an('array')); pm.test('at least one user', () => pm.expect(pm.response.json().users.length).to.be.above(0));"
|
|
315
|
-
}
|
|
316
|
-
}
|
|
317
|
-
}
|
|
318
|
-
}
|
|
319
|
-
}
|
|
37
|
+
openapi-cli --spec openapi.json --server https://api.example.com
|
|
320
38
|
```
|
|
321
39
|
|
|
322
|
-
|
|
40
|
+
### Programmatic usage
|
|
323
41
|
|
|
324
|
-
|
|
325
|
-
|
|
326
|
-
|
|
327
|
-
|
|
328
|
-
|
|
329
|
-
|
|
330
|
-
|
|
331
|
-
| SSE | Status code is 2xx (200–299) | 404, 500 |
|
|
332
|
-
| GraphQL | Status 2xx **and** no top-level `errors` array | 200 with `{"errors": [...]}` |
|
|
333
|
-
| gRPC | Status code is `0` (OK) | code 13 (INTERNAL), code 5 (NOT_FOUND) |
|
|
334
|
-
| MCP | JSON-RPC response has `result` and no `error` | `{"error": {"code": -32601}}` |
|
|
335
|
-
| WebSocket | Connection reached `open` state with no error | connection refused, handshake failure |
|
|
336
|
-
|
|
337
|
-
If you need to test a non-success response (e.g., asserting that an endpoint returns 404), define an explicit `x-tests` assertion — implicit checks are skipped when any user assertion exists.
|
|
338
|
-
|
|
339
|
-
---
|
|
340
|
-
|
|
341
|
-
## Multi-Protocol Support
|
|
342
|
-
|
|
343
|
-
### HTTP / SSE
|
|
344
|
-
|
|
345
|
-
Standard OpenAPI operations are executed as HTTP requests. SSE is auto-detected when:
|
|
346
|
-
- The response content type is `text/event-stream`, or
|
|
347
|
-
- The operation has `x-response-stream: true`
|
|
348
|
-
|
|
349
|
-
SSE responses are collected as a list of events.
|
|
350
|
-
|
|
351
|
-
### GraphQL
|
|
352
|
-
|
|
353
|
-
```json
|
|
354
|
-
{
|
|
355
|
-
"paths": {
|
|
356
|
-
"/graphql": {
|
|
357
|
-
"post": {
|
|
358
|
-
"x-protocol": "graphql",
|
|
359
|
-
"x-graphql": {
|
|
360
|
-
"endpoint": "https://api.example.com/graphql",
|
|
361
|
-
"query": "query GetUser($id: ID!) { user(id: $id) { name email } }",
|
|
362
|
-
"operationName": "GetUser",
|
|
363
|
-
"variablesSchema": {
|
|
364
|
-
"type": "object",
|
|
365
|
-
"properties": { "id": { "type": "string" } },
|
|
366
|
-
"required": ["id"]
|
|
367
|
-
}
|
|
368
|
-
}
|
|
369
|
-
}
|
|
370
|
-
}
|
|
371
|
-
}
|
|
372
|
-
}
|
|
373
|
-
```
|
|
374
|
-
|
|
375
|
-
### gRPC
|
|
376
|
-
|
|
377
|
-
All four gRPC method modes are supported: unary, server-streaming, client-streaming, and bidirectional-streaming.
|
|
378
|
-
|
|
379
|
-
```json
|
|
380
|
-
{
|
|
381
|
-
"paths": {
|
|
382
|
-
"/grpc": {
|
|
383
|
-
"post": {
|
|
384
|
-
"x-protocol": "grpc",
|
|
385
|
-
"x-grpc": {
|
|
386
|
-
"address": "127.0.0.1:50051",
|
|
387
|
-
"service": "demo.echo.Echo",
|
|
388
|
-
"method": "Say",
|
|
389
|
-
"kind": "unary",
|
|
390
|
-
"reflection": true
|
|
391
|
-
}
|
|
392
|
-
}
|
|
393
|
-
}
|
|
394
|
-
}
|
|
395
|
-
}
|
|
396
|
-
```
|
|
397
|
-
|
|
398
|
-
| Field | Description |
|
|
399
|
-
|-------|-------------|
|
|
400
|
-
| `address` | gRPC server `host:port` |
|
|
401
|
-
| `service` | Fully-qualified service name |
|
|
402
|
-
| `method` | Method name |
|
|
403
|
-
| `kind` | `unary`, `server_streaming`, `client_streaming`, `bidi_streaming` |
|
|
404
|
-
| `reflection` | Use server reflection (default `true`) |
|
|
405
|
-
| `protoPaths` | Paths to `.proto` files when reflection is off |
|
|
406
|
-
| `metadata` | Initial metadata (headers) |
|
|
407
|
-
| `sample` | Sample request message (JSON) |
|
|
408
|
-
|
|
409
|
-
### MCP
|
|
410
|
-
|
|
411
|
-
```json
|
|
412
|
-
{
|
|
413
|
-
"paths": {
|
|
414
|
-
"/mcp": {
|
|
415
|
-
"post": {
|
|
416
|
-
"x-protocol": "mcp",
|
|
417
|
-
"x-mcp": {
|
|
418
|
-
"endpoint": "http://127.0.0.1:4200/mcp",
|
|
419
|
-
"method": "tools/call",
|
|
420
|
-
"name": "get_weather",
|
|
421
|
-
"arguments": { "city": "San Francisco" }
|
|
422
|
-
}
|
|
423
|
-
}
|
|
424
|
-
}
|
|
425
|
-
}
|
|
426
|
-
}
|
|
427
|
-
```
|
|
428
|
-
|
|
429
|
-
Supports both Streamable HTTP and stdio transports (configured via `--mcp-transport`).
|
|
430
|
-
|
|
431
|
-
### WebSocket
|
|
432
|
-
|
|
433
|
-
```json
|
|
434
|
-
{
|
|
435
|
-
"paths": {
|
|
436
|
-
"/ws": {
|
|
437
|
-
"get": {
|
|
438
|
-
"x-protocol": "websocket",
|
|
439
|
-
"x-ws": {
|
|
440
|
-
"url": "wss://api.example.com/ws",
|
|
441
|
-
"subprotocols": ["chat"],
|
|
442
|
-
"send": [{ "type": "subscribe", "channel": "updates" }]
|
|
443
|
-
}
|
|
444
|
-
}
|
|
445
|
-
}
|
|
446
|
-
}
|
|
447
|
-
}
|
|
448
|
-
```
|
|
449
|
-
|
|
450
|
-
---
|
|
42
|
+
```typescript
|
|
43
|
+
import {
|
|
44
|
+
resolveConfig,
|
|
45
|
+
runTests,
|
|
46
|
+
generateJsonReport,
|
|
47
|
+
generateHtmlReport,
|
|
48
|
+
} from "@powerduck/openapi-cli";
|
|
451
49
|
|
|
452
|
-
|
|
50
|
+
const config = resolveConfig({
|
|
51
|
+
spec: "./openapi.json",
|
|
52
|
+
format: "json,html",
|
|
53
|
+
output: "./reports",
|
|
54
|
+
concurrency: 5,
|
|
55
|
+
});
|
|
453
56
|
|
|
454
|
-
|
|
57
|
+
const report = await runTests(config);
|
|
455
58
|
|
|
456
|
-
|
|
59
|
+
const jsonPath = generateJsonReport(report, config.outputDir);
|
|
60
|
+
const htmlPath = generateHtmlReport(report, config.outputDir);
|
|
457
61
|
|
|
458
|
-
|
|
459
|
-
{
|
|
460
|
-
"summary": {
|
|
461
|
-
"total": 10,
|
|
462
|
-
"passed": 8,
|
|
463
|
-
"failed": 1,
|
|
464
|
-
"errors": 1,
|
|
465
|
-
"skipped": 0,
|
|
466
|
-
"durationMs": 2340,
|
|
467
|
-
"passRate": 80.0
|
|
468
|
-
},
|
|
469
|
-
"results": [
|
|
470
|
-
{
|
|
471
|
-
"operationId": "getUser",
|
|
472
|
-
"path": "/users/{id}",
|
|
473
|
-
"method": "GET",
|
|
474
|
-
"protocol": "http",
|
|
475
|
-
"status": "passed",
|
|
476
|
-
"durationMs": 142,
|
|
477
|
-
"response": { "status": 200, "contentType": "application/json", "body": { "id": 1 } },
|
|
478
|
-
"assertions": [
|
|
479
|
-
{ "name": "status is 200", "passed": true }
|
|
480
|
-
],
|
|
481
|
-
"timestamp": "2024-01-01T00:00:00.000Z"
|
|
482
|
-
}
|
|
483
|
-
],
|
|
484
|
-
"config": { "specPath": "./openapi.json" },
|
|
485
|
-
"generatedAt": "2024-01-01T00:00:05.000Z",
|
|
486
|
-
"version": "0.1.0"
|
|
487
|
-
}
|
|
62
|
+
console.log("Passed:", report.passed, "/", report.total);
|
|
488
63
|
```
|
|
489
64
|
|
|
490
|
-
###
|
|
491
|
-
|
|
492
|
-
Colored terminal output with a progress bar, per-test status, and assertion details. Failed assertions show error messages inline.
|
|
493
|
-
|
|
494
|
-
### HTML (`report.html`)
|
|
495
|
-
|
|
496
|
-
A self-contained HTML file with:
|
|
497
|
-
- Summary strip (total, passed, failed, errors, pass rate, duration)
|
|
498
|
-
- Thin progress bar
|
|
499
|
-
- Filter buttons (All / Passed / Failed / Errors)
|
|
500
|
-
- Expandable test rows with assertion details and response body preview
|
|
501
|
-
- Automatic light/dark mode based on system preference
|
|
502
|
-
- No external dependencies — opens in any browser
|
|
503
|
-
|
|
504
|
-
---
|
|
505
|
-
|
|
506
|
-
## CI Integration
|
|
507
|
-
|
|
508
|
-
### GitHub Actions
|
|
65
|
+
### GitHub Actions CI
|
|
509
66
|
|
|
510
67
|
```yaml
|
|
511
68
|
name: API Tests
|
|
@@ -523,7 +80,7 @@ jobs:
|
|
|
523
80
|
- name: Run API tests
|
|
524
81
|
run: npx @powerduck/openapi-cli --spec openapi.json --server ${{ secrets.API_URL }} --output ./reports
|
|
525
82
|
env:
|
|
526
|
-
|
|
83
|
+
BEARER_TOKEN: ${{ secrets.BEARER_TOKEN }}
|
|
527
84
|
- name: Upload reports
|
|
528
85
|
if: always()
|
|
529
86
|
uses: actions/upload-artifact@v4
|
|
@@ -532,171 +89,141 @@ jobs:
|
|
|
532
89
|
path: ./reports
|
|
533
90
|
```
|
|
534
91
|
|
|
535
|
-
### GitLab CI
|
|
536
|
-
|
|
537
|
-
```yaml
|
|
538
|
-
api_test:
|
|
539
|
-
stage: test
|
|
540
|
-
image: node:20
|
|
541
|
-
script:
|
|
542
|
-
- npm ci
|
|
543
|
-
- npx @powerduck/openapi-cli --spec openapi.json --output ./reports
|
|
544
|
-
artifacts:
|
|
545
|
-
when: always
|
|
546
|
-
paths:
|
|
547
|
-
- ./reports/
|
|
548
|
-
expire_in: 30 days
|
|
549
|
-
```
|
|
550
|
-
|
|
551
|
-
### Exit codes
|
|
552
|
-
|
|
553
|
-
| Code | Meaning |
|
|
554
|
-
|------|---------|
|
|
555
|
-
| `0` | All tests passed (or `--no-fail-on-error` is set) |
|
|
556
|
-
| `1` | One or more tests failed |
|
|
557
|
-
| `2` | Configuration error, spec load failure, or runtime error |
|
|
558
|
-
|
|
559
92
|
---
|
|
560
93
|
|
|
561
|
-
##
|
|
562
|
-
|
|
563
|
-
```ts
|
|
564
|
-
import {
|
|
565
|
-
runTests,
|
|
566
|
-
resolveConfig,
|
|
567
|
-
generateJsonReport,
|
|
568
|
-
printCliReport,
|
|
569
|
-
generateHtmlReport,
|
|
570
|
-
loadSpec,
|
|
571
|
-
isRemoteSpec,
|
|
572
|
-
} from "@powerduck/openapi-cli";
|
|
573
|
-
|
|
574
|
-
// Build a config from args
|
|
575
|
-
const config = resolveConfig({
|
|
576
|
-
spec: "./openapi.json",
|
|
577
|
-
server: "https://api.staging.example.com",
|
|
578
|
-
concurrency: 10,
|
|
579
|
-
});
|
|
580
|
-
|
|
581
|
-
// Run all tests
|
|
582
|
-
const report = await runTests(config);
|
|
583
|
-
|
|
584
|
-
// Generate reports
|
|
585
|
-
printCliReport(report);
|
|
586
|
-
generateJsonReport(report, "./reports");
|
|
587
|
-
generateHtmlReport(report, "./reports");
|
|
588
|
-
|
|
589
|
-
// Load a spec directly (local or remote)
|
|
590
|
-
const spec = await loadSpec(config);
|
|
591
|
-
|
|
592
|
-
// Check if a path is remote
|
|
593
|
-
console.log(isRemoteSpec("https://example.com/spec.json")); // true
|
|
94
|
+
## Links
|
|
594
95
|
|
|
595
|
-
|
|
596
|
-
|
|
597
|
-
|
|
598
|
-
|
|
599
|
-
```
|
|
96
|
+
- [Official Website](https://www.powerduck.com/opensource/openapi-cli.html)
|
|
97
|
+
- [Documentation](https://www.powerduck.com/docs/openapi-cli/introduction)
|
|
98
|
+
- [GitHub](https://github.com/PowerDuckie/openapi-cli)
|
|
99
|
+
- [npm](https://www.npmjs.com/package/@powerduck/openapi-cli)
|
|
600
100
|
|
|
601
101
|
---
|
|
602
102
|
|
|
603
|
-
##
|
|
604
|
-
|
|
605
|
-
### `.env` files
|
|
606
|
-
|
|
607
|
-
Use `--env .env` to load variables from a file into `process.env` before config resolution. Format is `KEY=VALUE` per line.
|
|
608
|
-
|
|
609
|
-
```env
|
|
610
|
-
# Lines starting with # are ignored
|
|
611
|
-
API_KEY=sk-live-abc123
|
|
612
|
-
BEARER_TOKEN="eyJhbGciOi..."
|
|
613
|
-
API_HOST=api.staging.example.com
|
|
614
|
-
```
|
|
615
|
-
|
|
616
|
-
Behavior:
|
|
617
|
-
- Existing `process.env` variables are **not** overwritten.
|
|
618
|
-
- Surrounding single or double quotes are stripped from values.
|
|
619
|
-
- Blank lines and comment lines (`#`) are ignored.
|
|
620
|
-
- A missing `.env` file is silently ignored (no error).
|
|
621
|
-
|
|
622
|
-
Loaded variables are available for `${VAR_NAME}` expansion in CLI config values (see below). To make a variable available as a Postman `{{variable}}` inside the OpenAPI spec, pass it with `--variable KEY=${KEY}`.
|
|
623
|
-
|
|
624
|
-
### Variable expansion
|
|
625
|
-
|
|
626
|
-
`${VAR_NAME}` syntax is expanded in these config fields:
|
|
627
|
-
- `serverUrl`
|
|
628
|
-
- `proxy`
|
|
629
|
-
- `headers` (values)
|
|
630
|
-
- `auth.token`
|
|
631
|
-
- `variables` (values)
|
|
632
|
-
|
|
633
|
-
Example:
|
|
103
|
+
## Features
|
|
634
104
|
|
|
635
|
-
|
|
636
|
-
|
|
637
|
-
|
|
638
|
-
|
|
639
|
-
|
|
105
|
+
- **6 protocols** — HTTP/1.1, HTTP/2, HTTP/3, WebSocket, SSE, and gRPC
|
|
106
|
+
- **Batch testing** — Run every operation in your OpenAPI spec concurrently
|
|
107
|
+
- **Assertion engine** — Validate status codes, headers, response bodies, and schemas
|
|
108
|
+
- **3 report formats** — JSON for CI, CLI for terminal, HTML for dashboards
|
|
109
|
+
- **Authentication** — Bearer tokens, API keys, Basic auth, OAuth2, and custom headers
|
|
110
|
+
- **Filtering** — Run tests by tag, path, method, operationId, or regex
|
|
111
|
+
- **Concurrency control** — Configurable parallelism with rate limiting
|
|
112
|
+
- **gRPC reflection** — Auto-discover gRPC services via server reflection
|
|
113
|
+
- **MCP integration** — Streamable HTTP and stdio MCP transport support
|
|
114
|
+
- **GitHub Actions** — First-class CI integration with annotations and summaries
|
|
115
|
+
- **Config file support** — JSON config file with environment variable expansion
|
|
116
|
+
- **TLS/SSL options** — Custom CA certs, client certificates, and insecure mode
|
|
117
|
+
- **Proxy support** — HTTP/HTTPS proxy with authentication
|
|
118
|
+
- **Timeout & retries** — Configurable per-request timeout and retry policies
|
|
119
|
+
- **Dual ESM/CJS** — Works with `import` and `require`, with bundled TypeScript declarations
|
|
640
120
|
|
|
641
121
|
---
|
|
642
122
|
|
|
643
|
-
##
|
|
644
|
-
|
|
645
|
-
### "OpenAPI spec not found"
|
|
646
|
-
|
|
647
|
-
Verify the path exists. For remote specs, check the URL is reachable and returns HTTP 200.
|
|
648
|
-
|
|
649
|
-
### "Spec is not valid JSON"
|
|
650
|
-
|
|
651
|
-
The spec must be a JSON object with a `paths` property. YAML specs are not supported — convert to JSON first.
|
|
652
|
-
|
|
653
|
-
### "Failed to fetch spec: HTTP 401"
|
|
654
|
-
|
|
655
|
-
The remote spec URL requires authentication. Download it locally first or use a URL that is publicly accessible.
|
|
656
|
-
|
|
657
|
-
### gRPC "server reflection unavailable"
|
|
658
|
-
|
|
659
|
-
Either enable reflection on the server, or use `--grpc-no-reflection` with `--grpc-proto` pointing to your `.proto` files.
|
|
660
|
-
|
|
661
|
-
### MCP stdio "command not found"
|
|
662
|
-
|
|
663
|
-
Ensure `--mcp-command` is on your `PATH`, or use an absolute path. For `npx`-based servers, make sure the package is installed or use `npx -y <package>`.
|
|
664
|
-
|
|
665
|
-
### Tests are slow
|
|
666
|
-
|
|
667
|
-
Reduce `--concurrency` if the target server rate-limits. Increase `--timeout` for slow endpoints. Use `--method` or `--tag` to test a subset.
|
|
123
|
+
## CLI Reference
|
|
668
124
|
|
|
669
|
-
###
|
|
125
|
+
### Commands
|
|
670
126
|
|
|
671
|
-
|
|
127
|
+
```bash
|
|
128
|
+
openapi-cli [options]
|
|
672
129
|
|
|
673
|
-
|
|
130
|
+
# Basic usage
|
|
131
|
+
openapi-cli --spec openapi.json --server https://api.example.com
|
|
132
|
+
|
|
133
|
+
# Filter by tag
|
|
134
|
+
openapi-cli --spec openapi.json --server https://api.example.com --tag users,orders
|
|
135
|
+
|
|
136
|
+
# Filter by method and path
|
|
137
|
+
openapi-cli --spec openapi.json --server https://api.example.com --method get --path "/users/*"
|
|
138
|
+
|
|
139
|
+
# Custom output
|
|
140
|
+
openapi-cli --spec openapi.json --server https://api.example.com --output ./reports --format json,html,cli
|
|
141
|
+
|
|
142
|
+
# With auth
|
|
143
|
+
openapi-cli --spec openapi.json --server https://api.example.com --bearer $TOKEN --header "X-Env: staging"
|
|
144
|
+
|
|
145
|
+
# Config file
|
|
146
|
+
openapi-cli --config ./openapi-cli.config.json
|
|
147
|
+
```
|
|
148
|
+
|
|
149
|
+
### Options
|
|
150
|
+
|
|
151
|
+
| Option | Type | Default | Description |
|
|
152
|
+
| ------------------ | ---------- | ---------------------- | ---------------------------------------- |
|
|
153
|
+
| `--spec` | `string` | - | Path or URL to OpenAPI spec (required) |
|
|
154
|
+
| `--server` | `string` | - | Server URL override |
|
|
155
|
+
| `--output` | `string` | `./openapi-cli-report` | Output directory |
|
|
156
|
+
| `--format` | `string` | `json,cli,html` | Output formats (comma-separated) |
|
|
157
|
+
| `--method` | `string` | - | Filter by HTTP method (comma-separated) |
|
|
158
|
+
| `--path` | `string` | - | Filter by path pattern (comma-separated) |
|
|
159
|
+
| `--tag` | `string` | - | Filter by tag (comma-separated) |
|
|
160
|
+
| `--operationId` | `string` | - | Filter by operationId (comma-separated) |
|
|
161
|
+
| `--concurrency` | `number` | `5` | Max concurrent requests |
|
|
162
|
+
| `--timeout` | `number` | `30000` | Request timeout in ms |
|
|
163
|
+
| `--bearer` | `string` | - | Bearer token for authentication |
|
|
164
|
+
| `--header` | `string[]` | - | Custom headers (Key: Value) |
|
|
165
|
+
| `--variable` | `string[]` | - | Server variables (key=value) |
|
|
166
|
+
| `--config` | `string` | - | Path to JSON config file |
|
|
167
|
+
| `--env` | `string` | - | Path to .env file |
|
|
168
|
+
| `--failOnError` | `boolean` | `true` | Exit with non-zero code on test failure |
|
|
169
|
+
| `--grpcReflection` | `boolean` | `true` | Enable gRPC server reflection |
|
|
170
|
+
| `--grpcProto` | `string[]` | - | gRPC proto file paths |
|
|
171
|
+
| `--mcpTransport` | `string` | `streamable-http` | MCP transport type |
|
|
172
|
+
| `--mcpCommand` | `string` | - | MCP server command (stdio) |
|
|
173
|
+
| `--mcpArgs` | `string` | - | MCP server arguments |
|
|
174
|
+
| `--mcpCwd` | `string` | - | MCP server working directory |
|
|
175
|
+
| `--proxy` | `string` | - | Proxy URL |
|
|
176
|
+
| `--ca` | `string` | - | CA certificate path |
|
|
177
|
+
| `--cert` | `string` | - | Client certificate path |
|
|
178
|
+
| `--key` | `string` | - | Client key path |
|
|
179
|
+
| `--insecure` | `boolean` | `false` | Disable SSL verification |
|
|
180
|
+
|
|
181
|
+
---
|
|
182
|
+
|
|
183
|
+
## Config File
|
|
674
184
|
|
|
675
|
-
|
|
185
|
+
```json
|
|
186
|
+
{
|
|
187
|
+
"specPath": "./openapi.json",
|
|
188
|
+
"serverUrl": "https://api.example.com",
|
|
189
|
+
"outputDir": "./reports",
|
|
190
|
+
"formats": ["json", "html"],
|
|
191
|
+
"concurrency": 10,
|
|
192
|
+
"timeout": 60000,
|
|
193
|
+
"failOnError": true,
|
|
194
|
+
"filter": {
|
|
195
|
+
"tags": ["users", "orders"],
|
|
196
|
+
"methods": ["get", "post"]
|
|
197
|
+
},
|
|
198
|
+
"auth": {
|
|
199
|
+
"type": "bearer",
|
|
200
|
+
"token": "${BEARER_TOKEN}"
|
|
201
|
+
},
|
|
202
|
+
"headers": {
|
|
203
|
+
"X-Env": "staging"
|
|
204
|
+
},
|
|
205
|
+
"variables": {
|
|
206
|
+
"version": "v2"
|
|
207
|
+
}
|
|
208
|
+
}
|
|
209
|
+
```
|
|
676
210
|
|
|
677
211
|
---
|
|
678
212
|
|
|
679
|
-
##
|
|
680
|
-
|
|
681
|
-
```bash
|
|
682
|
-
# Install dependencies
|
|
683
|
-
npm install
|
|
684
|
-
|
|
685
|
-
# Type check
|
|
686
|
-
npm run typecheck
|
|
213
|
+
## TypeScript Types
|
|
687
214
|
|
|
688
|
-
|
|
689
|
-
|
|
690
|
-
|
|
691
|
-
|
|
692
|
-
|
|
693
|
-
|
|
694
|
-
|
|
695
|
-
|
|
215
|
+
```typescript
|
|
216
|
+
import type {
|
|
217
|
+
CliConfig,
|
|
218
|
+
CliArgs,
|
|
219
|
+
TestReport,
|
|
220
|
+
TestResult,
|
|
221
|
+
AssertionResult,
|
|
222
|
+
} from "@powerduck/openapi-cli";
|
|
696
223
|
```
|
|
697
224
|
|
|
698
225
|
---
|
|
699
226
|
|
|
700
227
|
## License
|
|
701
228
|
|
|
702
|
-
MIT
|
|
229
|
+
MIT © [POWERDUCK LIMITED](https://www.powerduck.com)
|
package/package.json
CHANGED