@seldonqa/cli 0.7.0 → 0.9.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +21 -1
- package/package.json +6 -6
package/README.md
CHANGED
|
@@ -92,6 +92,26 @@ seldon run --url "$PREVIEW_URL" --fail-on fail
|
|
|
92
92
|
|
|
93
93
|
`--fail-on fail` returns a non-zero exit code when any case fails; `--fail-on any` also gates on blocked or timed-out cases. Non-interactive shells block until the run finishes; `--no-wait` submits and returns immediately (re-read later with `seldon report`). With `--json` the report is stable, machine-readable output you can post to a pull request or hand a coding agent for an automated fix loop.
|
|
94
94
|
|
|
95
|
+
## Local targets — automatic tunnels
|
|
96
|
+
|
|
97
|
+
Point `--url` at a local address — `localhost`, `127.0.0.1`, `0.0.0.0`, `::1`, or a bare `:PORT` — and Seldon exposes it to the QA agent through a temporary, authenticated **Cloudflare tunnel** for the duration of the run. No deploy, no manual tunnelling:
|
|
98
|
+
|
|
99
|
+
```bash
|
|
100
|
+
seldon run --url http://localhost:3000
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
You'll see a one-line warning that the local address will be exposed and a `[Y/n]` prompt (defaults to yes). The public hostname is protected by Cloudflare Access, so **only this run's agent can reach it**, and the tunnel is **torn down automatically** when the run finishes or you press Ctrl-C. In CI or any non-interactive shell the prompt is skipped automatically; pass **`--auto-approve`** to skip it explicitly:
|
|
104
|
+
|
|
105
|
+
```bash
|
|
106
|
+
seldon run --url localhost:3000 --auto-approve # e.g. in a CI job
|
|
107
|
+
```
|
|
108
|
+
|
|
109
|
+
Remote `--url`s are used as-is (no tunnel).
|
|
110
|
+
|
|
111
|
+
**Tunnels are short-lived.** Each tunnel has a hard maximum lifetime (**20 minutes** by default) and is reclaimed by the server once it's exceeded, regardless of the run's state — so the run must finish within that window. When a tunnel comes up the CLI prints the exact cap for your account (`… active for up to 20 min; the run must finish within that window`). Runs that legitimately need longer should target a deployed/public `--url` instead of a local one.
|
|
112
|
+
|
|
113
|
+
If your account has too many tunnels open at once you'll get `Tunnel limit reached — wait a minute and re-run`; if the same local target is already being provisioned you'll get a short retry hint. Neither starts a run — just re-run once things settle.
|
|
114
|
+
|
|
95
115
|
## Commands
|
|
96
116
|
|
|
97
117
|
| Command | What it does |
|
|
@@ -100,7 +120,7 @@ seldon run --url "$PREVIEW_URL" --fail-on fail
|
|
|
100
120
|
| `seldon cases add` | Append a test case (`--name` `--step …` `--criteria`) |
|
|
101
121
|
| `seldon cases` | List the cases in the current `seldon.yml` |
|
|
102
122
|
| `seldon validate` | Validate the file locally |
|
|
103
|
-
| `seldon run [--url <url>]` | Ingest `seldon.yml` and run it; `--no-wait` to detach, `--fail-on` to gate, `--label <tag>` to profile |
|
|
123
|
+
| `seldon run [--url <url>]` | Ingest `seldon.yml` and run it; a local `--url` is auto-exposed via a Cloudflare tunnel (`--auto-approve` to skip the prompt); `--no-wait` to detach, `--fail-on` to gate, `--label <tag>` to profile |
|
|
104
124
|
| `seldon report [--md \| --json] [--full]` | Show the report for the last run (or `--id <run>`) |
|
|
105
125
|
| `seldon logs --follow` | Stream the agent's live reasoning |
|
|
106
126
|
| `seldon runs` | List QA runs with color-coded status |
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@seldonqa/cli",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.9.0",
|
|
4
4
|
"description": "Seldon QA - generate a QA test plan from any codebase and run AI agents against your live app, from your terminal or CI.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"keywords": [
|
|
@@ -32,11 +32,11 @@
|
|
|
32
32
|
"commander": "^12.1.0"
|
|
33
33
|
},
|
|
34
34
|
"optionalDependencies": {
|
|
35
|
-
"@seldonqa/cli-
|
|
36
|
-
"@seldonqa/cli-darwin-
|
|
37
|
-
"@seldonqa/cli-
|
|
38
|
-
"@seldonqa/cli-linux-x64": "0.
|
|
39
|
-
"@seldonqa/cli-
|
|
35
|
+
"@seldonqa/cli-darwin-arm64": "0.9.0",
|
|
36
|
+
"@seldonqa/cli-darwin-x64": "0.9.0",
|
|
37
|
+
"@seldonqa/cli-linux-arm64": "0.9.0",
|
|
38
|
+
"@seldonqa/cli-linux-x64": "0.9.0",
|
|
39
|
+
"@seldonqa/cli-win32-x64": "0.9.0"
|
|
40
40
|
},
|
|
41
41
|
"devDependencies": {
|
|
42
42
|
"@types/node": "^20.14.0",
|