@seldonqa/cli 0.5.0 → 0.5.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +93 -0
- package/package.json +14 -2
package/README.md
ADDED
|
@@ -0,0 +1,93 @@
|
|
|
1
|
+
# Seldon QA
|
|
2
|
+
|
|
3
|
+
**Generate a QA test plan from any codebase, then run AI agents that use your product like a real user — from your terminal or CI.**
|
|
4
|
+
|
|
5
|
+
`seldon` turns a repository into a **baseline** of prioritized, user-facing test cases, runs those cases against a live URL with autonomous agents, and returns a clean, actionable report for you or your coding agent to act on.
|
|
6
|
+
|
|
7
|
+
```bash
|
|
8
|
+
npm install -g @seldonqa/cli
|
|
9
|
+
```
|
|
10
|
+
|
|
11
|
+
## Why
|
|
12
|
+
|
|
13
|
+
AI is shipping more code than any team can test by hand. Seldon closes the gap: point it at your repo and it proposes the tests that matter; point it at a URL and agents actually exercise them like users — click, type, navigate, verify — then tell you exactly what broke and where.
|
|
14
|
+
|
|
15
|
+
## Get started
|
|
16
|
+
|
|
17
|
+
```bash
|
|
18
|
+
npm install -g @seldonqa/cli
|
|
19
|
+
seldon login seldon_your_api_key # or: seldon config set-key <key>
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
From inside a project (one baseline per project):
|
|
23
|
+
|
|
24
|
+
```bash
|
|
25
|
+
seldon init # create seldon.yml, preview what will be analyzed
|
|
26
|
+
seldon generate # build a QA baseline from this codebase
|
|
27
|
+
seldon cases # review / scope test cases
|
|
28
|
+
seldon run --url https://staging.example.com
|
|
29
|
+
seldon logs --follow # watch the agents test, live
|
|
30
|
+
seldon report --md # clean, actionable results
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
## Highlights
|
|
34
|
+
|
|
35
|
+
- **Baselines from your code.** `seldon generate` reads the project and produces a prioritized plan of real user journeys — no scripts to write.
|
|
36
|
+
- **Agents that behave like users.** `seldon run` drives your live app and judges each case pass / fail / blocked.
|
|
37
|
+
- **Watch it think.** `seldon logs --follow` streams the agent's reasoning, framed by test case and step.
|
|
38
|
+
- **Reports built for action.** `--md` for pull requests, `--json` for coding agents, plain text for the terminal.
|
|
39
|
+
- **Re-evaluate as code changes.** `seldon update` refreshes the plan when the code moves; pinned cases are always preserved.
|
|
40
|
+
- **CI-native gate.** `seldon run --fail-on fail` exits non-zero so a red QA run can block a merge.
|
|
41
|
+
|
|
42
|
+
## Configuration — `seldon.yml`
|
|
43
|
+
|
|
44
|
+
Configuration lives in a single committed file at your project root (config-as-code). With no file, every command still works with sensible defaults.
|
|
45
|
+
|
|
46
|
+
```yaml
|
|
47
|
+
baseline:
|
|
48
|
+
run:
|
|
49
|
+
url: https://staging.example.com # where runs target by default
|
|
50
|
+
testCases:
|
|
51
|
+
only: [] # run ONLY these case ids (wins over skip)
|
|
52
|
+
skip: [] # never run these
|
|
53
|
+
fixed: [] # always keep these when the plan is updated (accepts: [ALL])
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
## In CI
|
|
57
|
+
|
|
58
|
+
```bash
|
|
59
|
+
seldon run --url "$PREVIEW_URL" --fail-on fail
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
`--fail-on fail` returns a non-zero exit code when any case fails; use `--fail-on any` to also gate on blocked or timed-out cases. With `--json` the report is stable, machine-readable output you can post to a pull request or hand to a coding agent for an automated fix loop.
|
|
63
|
+
|
|
64
|
+
## Commands
|
|
65
|
+
|
|
66
|
+
| Command | What it does |
|
|
67
|
+
| --- | --- |
|
|
68
|
+
| `seldon init` | Create `seldon.yml` and preview what will be analyzed |
|
|
69
|
+
| `seldon generate` | Build a QA baseline (test plan) from the current codebase |
|
|
70
|
+
| `seldon cases` | Review and scope test cases — only / skip / fixed |
|
|
71
|
+
| `seldon run --url <url>` | Run the baseline against a live URL |
|
|
72
|
+
| `seldon logs --follow` | Stream the agent's live reasoning (thinking only) |
|
|
73
|
+
| `seldon report [--md \| --json]` | Show the latest report |
|
|
74
|
+
| `seldon runs` | List QA runs with color-coded status |
|
|
75
|
+
| `seldon cancel --id <run>` | Stop a run mid-flight |
|
|
76
|
+
| `seldon update` | Re-evaluate the baseline after code changes |
|
|
77
|
+
| `seldon review` | PR-style: evaluate your changes and run the affected cases |
|
|
78
|
+
| `seldon list` · `versions` · `promote` | Manage baselines and versions |
|
|
79
|
+
|
|
80
|
+
Run `seldon --help` or `seldon <command> --help` for the full reference.
|
|
81
|
+
|
|
82
|
+
## Coding agents (MCP)
|
|
83
|
+
|
|
84
|
+
Seldon is also available as an MCP server, so coding agents can generate baselines, run them, and read reports directly — enabling a closed test-and-fix loop.
|
|
85
|
+
|
|
86
|
+
## Requirements
|
|
87
|
+
|
|
88
|
+
- Node.js 20 or newer
|
|
89
|
+
- A Seldon API key
|
|
90
|
+
|
|
91
|
+
---
|
|
92
|
+
|
|
93
|
+
Run `seldon --help` to get started.
|
package/package.json
CHANGED
|
@@ -1,8 +1,20 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@seldonqa/cli",
|
|
3
|
-
"version": "0.5.
|
|
4
|
-
"description": "Seldon - generate QA
|
|
3
|
+
"version": "0.5.1",
|
|
4
|
+
"description": "Seldon QA - generate a QA test plan from any codebase and run AI agents against your live app, from your terminal or CI.",
|
|
5
5
|
"type": "module",
|
|
6
|
+
"keywords": [
|
|
7
|
+
"qa",
|
|
8
|
+
"testing",
|
|
9
|
+
"test-automation",
|
|
10
|
+
"e2e",
|
|
11
|
+
"end-to-end-testing",
|
|
12
|
+
"ai",
|
|
13
|
+
"agents",
|
|
14
|
+
"ai-testing",
|
|
15
|
+
"ci",
|
|
16
|
+
"seldon"
|
|
17
|
+
],
|
|
6
18
|
"bin": {
|
|
7
19
|
"seldon": "./dist/bin.js"
|
|
8
20
|
},
|