@test-lab-ai/cli 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/AGENTS.md ADDED
@@ -0,0 +1,115 @@
1
+ # Importing test plans into test-lab with an AI agent
2
+
3
+ This guide is for an AI coding agent (Claude Code, Codex, Cursor, etc.) asked to
4
+ move a user's existing tests into test-lab.ai. You do the format translation; the
5
+ `testlab` CLI handles auth and upload.
6
+
7
+ ## Install (zero-dependency)
8
+
9
+ ```bash
10
+ npx @test-lab-ai/cli --help
11
+ # or a global `testlab` command: npm i -g @test-lab-ai/cli
12
+ ```
13
+
14
+ ## The workflow
15
+
16
+ 1. **Read** whatever the user already has: Playwright/Cypress specs, Cucumber
17
+ `.feature` files, a TestRail/Zephyr export, a spreadsheet, or a prose doc.
18
+ 2. **Convert** each test into a plan object (schema below). test-lab tests are
19
+ natural language, not code:
20
+ - Put the explicit, fully-qualified URL in the `prompt` (imported plans have
21
+ no project, so there is no base URL to inherit).
22
+ - Replace any secret (passwords, tokens, test-account logins) with a
23
+ `{{credentials.NAME}}` placeholder, and collect the real values into the
24
+ top-level `credentials` array.
25
+ - Write clear pass/fail expectations into the prompt ("Confirm the dashboard
26
+ loads", "Expect an order-confirmation page with an order number").
27
+ - If the user's repo has a test-lab plan skill/format, prefer it.
28
+ 3. **Write** the plans to a JSON file (or a directory of `*.json`).
29
+ 4. **Run** `testlab import <path>`. Use `--dry-run` first to show the user the
30
+ creation order without writing anything.
31
+
32
+ Authentication: the user runs `testlab login` once (browser), or you set
33
+ `TESTLAB_API_KEY` in the environment for a fully headless run.
34
+
35
+ ## Plan object schema
36
+
37
+ | Field | Type | Required | Notes |
38
+ |-------|------|----------|-------|
39
+ | `name` | string | yes | Max 200 chars |
40
+ | `prompt` | string | yes | The test in natural language, with explicit URL(s) and `{{credentials.X}}`. Max 32 KB |
41
+ | `ref` | string | no | A handle unique within this import, used only to wire pre-steps (see below) |
42
+ | `testType` | `"quickTest"` \| `"deepTest"` | no | Quick is a fast smoke; deep is more thorough |
43
+ | `agentType` | string | no | `functional` (default), `accessibility`, `uiux`, `exploratory`, `performance`, `security` |
44
+ | `devices` | string[] | no | e.g. `["Desktop Chrome"]` (default), `["iPhone 15 Pro"]` |
45
+ | `labels` | (string \| number)[] | no | Names auto-create on the account; ids reuse existing labels. Max 25 |
46
+ | `preSteps` | object[] | no | Pipeline dependencies. Max 25. See below |
47
+ | `failOnPreStepFailure` | boolean | no | Default `true` |
48
+ | `cookies` | `{name,value,domain}[]` | no | Injected at run time |
49
+ | `headers` | `{name,value}[]` | no | Injected at run time |
50
+
51
+ ## File shapes
52
+
53
+ Any of these is valid input to `testlab import`:
54
+
55
+ ```jsonc
56
+ // 1. a single plan
57
+ { "name": "...", "prompt": "..." }
58
+
59
+ // 2. an array of plans
60
+ [ { "name": "...", "prompt": "..." }, { "name": "...", "prompt": "..." } ]
61
+
62
+ // 3. plans + the credentials they reference (recommended)
63
+ {
64
+ "credentials": [ { "key": "email", "value": "qa@example.com" } ],
65
+ "plans": [ { "name": "...", "prompt": "Sign in with {{credentials.email}} ..." } ]
66
+ }
67
+ ```
68
+
69
+ A directory imports every `*.json` file inside it (sorted by filename).
70
+
71
+ ## Pre-steps and ordering
72
+
73
+ A pre-step makes one plan run after another, sharing browser state (a login that
74
+ runs before a checkout). Reference the dependency in one of three ways:
75
+
76
+ ```jsonc
77
+ { "ref": "login" } // another plan IN THIS IMPORT, by its ref (preferred)
78
+ { "name": "Existing Login" } // a plan that already exists in the account, by name
79
+ { "testPlanId": 42 } // an existing plan by id
80
+ ```
81
+
82
+ You do NOT need to order the plans array yourself: the CLI topologically sorts by
83
+ `ref` dependencies and creates them in the correct order, then wires each
84
+ pre-step to the concrete id it just created. Cycles, self-references, and a `ref`
85
+ that matches no plan are rejected before anything is written.
86
+
87
+ ## Credentials
88
+
89
+ Put real secret values in the top-level `credentials` array; reference them in
90
+ prompts (and cookie/header values) as `{{credentials.KEY}}`. Keys start with a
91
+ letter and use letters/numbers/underscores only. The CLI upserts credentials
92
+ before creating plans, and values are stored encrypted (never echoed back).
93
+
94
+ ## End-to-end example
95
+
96
+ ```bash
97
+ # 1. (user, once) authenticate
98
+ testlab login # or: export TESTLAB_API_KEY=tl_xxxxx
99
+
100
+ # 2. you write ./tests/*.json from the user's existing suite (see examples/plans.json)
101
+
102
+ # 3. preview, then import
103
+ testlab import ./tests --dry-run
104
+ testlab import ./tests
105
+ ```
106
+
107
+ ## Notes
108
+
109
+ - Plans are created fresh every run (no de-duplication). Run `testlab plans list`
110
+ first if you need to avoid creating duplicates of plans already in the account.
111
+ - Imported plans have no project and no geolocation proxy; set those in the
112
+ dashboard afterward if the user needs them.
113
+ - Prefer calling the CLI over hand-rolling HTTP. If you must call the API
114
+ directly, it is documented at https://test-lab.ai/docs/api/test-plans and uses
115
+ the same `Authorization: Bearer tl_…` key.
package/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Test-Lab
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
package/README.md ADDED
@@ -0,0 +1,78 @@
1
+ # @test-lab-ai/cli (`testlab`)
2
+
3
+ Import existing test plans into [test-lab.ai](https://test-lab.ai) from the command line, your CI, or an AI agent. Zero dependencies, Node 18+.
4
+
5
+ ## Install
6
+
7
+ Published to npm. The CLI has **zero dependencies**, so an install pulls nothing
8
+ else and runs no install scripts.
9
+
10
+ One-off, no install:
11
+
12
+ ```bash
13
+ npx @test-lab-ai/cli login
14
+ ```
15
+
16
+ Or install the `testlab` command globally:
17
+
18
+ ```bash
19
+ npm i -g @test-lab-ai/cli
20
+ testlab login
21
+ ```
22
+
23
+ Requires Node 18+.
24
+
25
+ ## Authenticate
26
+
27
+ ```bash
28
+ testlab login
29
+ ```
30
+
31
+ Opens your browser to approve access, then stores a key in `~/.test-lab/config.json` (mode 600).
32
+
33
+ Headless / CI / agents (no browser): pass a key directly. Create one under **Settings → API Keys** (`/admin/settings/api-keys`).
34
+
35
+ ```bash
36
+ export TESTLAB_API_KEY=tl_xxxxx
37
+ # or
38
+ testlab login --key tl_xxxxx
39
+ ```
40
+
41
+ The key resolves to a single account, so the key you use IS the import target. To import into an organization account, mint the key with that org selected.
42
+
43
+ ## Commands
44
+
45
+ ```
46
+ testlab whoami Show the authenticated account
47
+ testlab plans list List your test plans
48
+ testlab plans create -f plan.json Create one plan from JSON
49
+ testlab plans create --name N --prompt P
50
+ testlab credentials set KEY --value V Upsert a credential ({{credentials.KEY}})
51
+ testlab import <path> [--dry-run] Import a plan file or a directory of *.json
52
+ ```
53
+
54
+ Global options: `--key`, `--api-url` (or env `TESTLAB_API_KEY` / `TESTLAB_API_URL`). The API base defaults to `https://www.test-lab.ai`.
55
+
56
+ ## Import
57
+
58
+ A file can be a single plan object, an array of plans, or `{ "credentials": [...], "plans": [...] }`. Pointing at a directory imports every `*.json` inside it. See [`examples/plans.json`](./examples/plans.json) and [`AGENTS.md`](./AGENTS.md) for the full schema.
59
+
60
+ ```bash
61
+ testlab import ./tests --dry-run # validate + print creation order, write nothing
62
+ testlab import ./tests # upsert credentials, then create plans
63
+ ```
64
+
65
+ The CLI topologically sorts plans by their pre-step dependencies and creates them one request each, wiring intra-import dependencies via the `ref` handle. Re-running is safe for credentials and labels (both upsert), but plans are always created fresh (no de-duplication), so import a given set once.
66
+
67
+ ## Local development
68
+
69
+ Point at a local dev server (`pnpm dev:web`):
70
+
71
+ ```bash
72
+ testlab login --api-url http://localhost:3000
73
+ testlab import ./tests --api-url http://localhost:3000
74
+ ```
75
+
76
+ ## How it relates to the API
77
+
78
+ Every command is a thin wrapper over the public [Import Test Plans API](https://test-lab.ai/docs/api/test-plans). An agent can call that API directly instead of shelling out to the CLI; the CLI just handles auth storage and import ordering for you.
@@ -0,0 +1,239 @@
1
+ #!/usr/bin/env node
2
+ /**
3
+ * testlab — import existing test plans into test-lab.ai (driveable by a human,
4
+ * a CI job, or an AI agent).
5
+ *
6
+ * Commands:
7
+ * testlab login [--key tl_…] Authenticate (browser flow, or paste/flag a key)
8
+ * testlab whoami Show the authenticated account
9
+ * testlab plans list List the account's test plans
10
+ * testlab plans create -f plan.json Create one plan from a JSON file
11
+ * testlab plans create --name N --prompt P
12
+ * testlab credentials set KEY --value V Upsert an account credential ({{credentials.KEY}})
13
+ * testlab import <path> [--dry-run] Import a plan file or a directory of *.json
14
+ *
15
+ * Auth: --key → $TESTLAB_API_KEY → ~/.test-lab/config.json
16
+ * API: --api-url → $TESTLAB_API_URL → https://www.test-lab.ai
17
+ */
18
+ import { parseArgs } from "node:util"
19
+ import fs from "node:fs"
20
+ import readline from "node:readline"
21
+ import { resolveAuth, loadConfig, saveConfig } from "../lib/config.mjs"
22
+ import { apiFetch } from "../lib/api.mjs"
23
+ import { loadImportFile, runImport } from "../lib/import.mjs"
24
+ import { browserLogin } from "../lib/login.mjs"
25
+
26
+ const log = (...a) => console.log(...a)
27
+ function errExit(msg) {
28
+ console.error(`error: ${msg}`)
29
+ process.exit(1)
30
+ }
31
+
32
+ const HELP = `testlab — import test plans into test-lab.ai
33
+
34
+ Usage:
35
+ testlab login [--key tl_…] Authenticate (browser, or paste/flag a key)
36
+ testlab whoami Show the authenticated account
37
+ testlab plans list List your test plans
38
+ testlab plans create -f plan.json Create one plan from JSON
39
+ testlab plans create --name N --prompt P
40
+ testlab credentials set KEY --value V Upsert a credential ({{credentials.KEY}})
41
+ testlab import <path> [--dry-run] Import a plan file or directory of *.json
42
+
43
+ Options:
44
+ --key <tl_…> API key (else $TESTLAB_API_KEY or ~/.test-lab/config.json)
45
+ --api-url <url> API base (else $TESTLAB_API_URL or https://www.test-lab.ai)
46
+ --stdin Read the value (key/credential) from stdin
47
+ --force (login) re-authenticate even if a stored key still works
48
+ --dry-run (import) validate + print order without writing
49
+
50
+ Get a key at <api-url>/admin/settings/api-keys`
51
+
52
+ function parse() {
53
+ return parseArgs({
54
+ allowPositionals: true,
55
+ options: {
56
+ key: { type: "string" },
57
+ "api-url": { type: "string" },
58
+ value: { type: "string" },
59
+ file: { type: "string", short: "f" },
60
+ name: { type: "string" },
61
+ prompt: { type: "string" },
62
+ "dry-run": { type: "boolean" },
63
+ stdin: { type: "boolean" },
64
+ force: { type: "boolean" },
65
+ help: { type: "boolean", short: "h" },
66
+ },
67
+ })
68
+ }
69
+
70
+ function readStdin() {
71
+ try {
72
+ return fs.readFileSync(0, "utf8").trim()
73
+ } catch {
74
+ return ""
75
+ }
76
+ }
77
+
78
+ function promptLine(question) {
79
+ const rl = readline.createInterface({ input: process.stdin, output: process.stdout })
80
+ return new Promise((resolve) => rl.question(question, (a) => {
81
+ rl.close()
82
+ resolve(a.trim())
83
+ }))
84
+ }
85
+
86
+ function requireAuth(flags) {
87
+ const { apiKey, apiUrl } = resolveAuth(flags)
88
+ if (!apiKey) errExit("not authenticated. Run `testlab login` (or set TESTLAB_API_KEY).")
89
+ return { apiKey, apiUrl }
90
+ }
91
+
92
+ async function verifyKey(apiUrl, apiKey) {
93
+ const r = await apiFetch(apiUrl, apiKey, "GET", "/api/v1/test-plans?limit=1")
94
+ return r
95
+ }
96
+
97
+ async function cmdLogin(flags) {
98
+ const { apiUrl, apiKey: existingKey } = resolveAuth(flags)
99
+
100
+ // Re-running `testlab login` with a still-valid stored key would otherwise
101
+ // mint a brand-new server-side key every time (they accrete). Short-circuit
102
+ // unless the user passed a key explicitly or forced re-auth.
103
+ if (existingKey && !flags.key && !flags.stdin && !flags.force) {
104
+ const check = await verifyKey(apiUrl, existingKey)
105
+ if (check.ok) {
106
+ log(`✓ Already authenticated to ${apiUrl}. Use --force to re-authenticate.`)
107
+ return
108
+ }
109
+ }
110
+
111
+ let apiKey = flags.key || (flags.stdin ? readStdin() : null)
112
+ let who = null
113
+
114
+ if (!apiKey) {
115
+ try {
116
+ who = await browserLogin(apiUrl)
117
+ apiKey = who.apiKey
118
+ } catch (e) {
119
+ log(`Browser login unavailable (${e.message}).`)
120
+ log(`Create a key at ${apiUrl}/admin/settings/api-keys, then paste it below.`)
121
+ apiKey = await promptLine("API key: ")
122
+ }
123
+ }
124
+ if (!apiKey) errExit("no API key provided")
125
+
126
+ const check = await verifyKey(apiUrl, apiKey)
127
+ if (!check.ok) errExit(`key rejected (${check.status}): ${check.json?.error || "invalid key"}`)
128
+
129
+ const cfg = loadConfig()
130
+ cfg.apiKey = apiKey
131
+ cfg.apiUrl = apiUrl
132
+ const path = saveConfig(cfg)
133
+ log(`✓ Authenticated${who?.email ? ` as ${who.email}` : ""}. Credentials saved to ${path}`)
134
+ }
135
+
136
+ async function cmdWhoami(flags) {
137
+ const { apiKey, apiUrl } = requireAuth(flags)
138
+ const r = await verifyKey(apiUrl, apiKey)
139
+ if (!r.ok) errExit(`not authenticated (${r.status}): ${r.json?.error || ""}`)
140
+ log(`✓ Authenticated to ${apiUrl}`)
141
+ log(` Test plans in account: ${r.json?.total ?? "?"}`)
142
+ }
143
+
144
+ async function cmdPlansList(flags) {
145
+ const { apiKey, apiUrl } = requireAuth(flags)
146
+ const r = await apiFetch(apiUrl, apiKey, "GET", "/api/v1/test-plans?limit=100")
147
+ if (!r.ok) errExit(`${r.status}: ${r.json?.error || ""}`)
148
+ const plans = r.json?.testPlans || []
149
+ if (plans.length === 0) {
150
+ log("No test plans yet.")
151
+ return
152
+ }
153
+ for (const p of plans) {
154
+ const labels = (p.labels || []).map((l) => l.name).join(", ")
155
+ log(` #${p.id} ${p.name}${labels ? ` [${labels}]` : ""}`)
156
+ }
157
+ log(`\n${r.json.total} total`)
158
+ }
159
+
160
+ async function cmdPlansCreate(flags) {
161
+ const { apiKey, apiUrl } = requireAuth(flags)
162
+ let plan
163
+ if (flags.file) {
164
+ try {
165
+ plan = JSON.parse(fs.readFileSync(flags.file, "utf8"))
166
+ } catch (e) {
167
+ errExit(`could not read ${flags.file}: ${e.message}`)
168
+ }
169
+ } else if (flags.name && flags.prompt) {
170
+ plan = { name: flags.name, prompt: flags.prompt }
171
+ } else {
172
+ errExit("provide -f <plan.json>, or both --name and --prompt")
173
+ }
174
+ const r = await apiFetch(apiUrl, apiKey, "POST", "/api/v1/test-plans", plan)
175
+ if (!r.ok) errExit(`${r.status}: ${r.json?.error || ""}`)
176
+ log(`✓ Created plan #${r.json.testPlan.id}: ${r.json.testPlan.name}`)
177
+ }
178
+
179
+ async function cmdCredentialsSet(flags, args) {
180
+ const { apiKey, apiUrl } = requireAuth(flags)
181
+ const key = args[2]
182
+ if (!key) errExit("usage: testlab credentials set <key> --value <v>")
183
+ const value = flags.value ?? (flags.stdin ? readStdin() : null)
184
+ if (!value) errExit("provide --value <v> or --stdin")
185
+ const r = await apiFetch(apiUrl, apiKey, "POST", "/api/v1/credentials", { key, value })
186
+ if (!r.ok) errExit(`${r.status}: ${r.json?.error || ""}`)
187
+ log(`✓ Set credential ${key}`)
188
+ }
189
+
190
+ async function cmdImport(flags, args) {
191
+ const { apiKey, apiUrl } = requireAuth(flags)
192
+ const target = args[1]
193
+ if (!target) errExit("usage: testlab import <path> [--dry-run]")
194
+ let loaded
195
+ try {
196
+ loaded = loadImportFile(target)
197
+ } catch (e) {
198
+ errExit(e.message)
199
+ }
200
+ log(`Importing ${loaded.plans.length} plan(s)${loaded.credentials.length ? ` + ${loaded.credentials.length} credential(s)` : ""} from ${target}`)
201
+ const res = await runImport({ apiUrl, apiKey, ...loaded, dryRun: flags["dry-run"], log })
202
+ if (!res.ok && !res.dryRun) process.exit(1)
203
+ }
204
+
205
+ async function main() {
206
+ let parsed
207
+ try {
208
+ parsed = parse()
209
+ } catch (e) {
210
+ errExit(e.message)
211
+ }
212
+ const flags = parsed.values
213
+ const args = parsed.positionals
214
+
215
+ if (flags.help || args.length === 0 || args[0] === "help") {
216
+ log(HELP)
217
+ return
218
+ }
219
+
220
+ switch (args[0]) {
221
+ case "login":
222
+ return cmdLogin(flags)
223
+ case "whoami":
224
+ return cmdWhoami(flags)
225
+ case "plans":
226
+ if (args[1] === "list") return cmdPlansList(flags)
227
+ if (args[1] === "create") return cmdPlansCreate(flags)
228
+ return errExit("usage: testlab plans <list|create>")
229
+ case "credentials":
230
+ if (args[1] === "set") return cmdCredentialsSet(flags, args)
231
+ return errExit("usage: testlab credentials set <key> --value <v>")
232
+ case "import":
233
+ return cmdImport(flags, args)
234
+ default:
235
+ return errExit(`unknown command: ${args[0]}. Run \`testlab help\`.`)
236
+ }
237
+ }
238
+
239
+ main().catch((e) => errExit(e.message))
@@ -0,0 +1,24 @@
1
+ {
2
+ "credentials": [
3
+ { "key": "email", "value": "qa@example.com" },
4
+ { "key": "password", "value": "replace-me" }
5
+ ],
6
+ "plans": [
7
+ {
8
+ "ref": "login",
9
+ "name": "Login",
10
+ "prompt": "Go to https://app.example.com/login and sign in with {{credentials.email}} / {{credentials.password}}. Confirm the dashboard at https://app.example.com/dashboard loads.",
11
+ "testType": "quickTest",
12
+ "agentType": "functional",
13
+ "labels": ["smoke", "auth"]
14
+ },
15
+ {
16
+ "ref": "checkout",
17
+ "name": "Checkout after login",
18
+ "prompt": "Starting on the dashboard, add the first product to the cart and complete checkout. Confirm an order-confirmation page appears with an order number.",
19
+ "testType": "deepTest",
20
+ "labels": ["regression"],
21
+ "preSteps": [{ "ref": "login" }]
22
+ }
23
+ ]
24
+ }
package/lib/api.mjs ADDED
@@ -0,0 +1,31 @@
1
+ /**
2
+ * Tiny fetch wrapper for the test-lab public API. Uses global fetch (Node 18+).
3
+ * Returns { ok, status, json } — json is the parsed body (or { raw } on non-JSON).
4
+ */
5
+ export async function apiFetch(apiUrl, apiKey, method, pathname, body) {
6
+ const headers = {}
7
+ if (apiKey) headers["Authorization"] = `Bearer ${apiKey}`
8
+ if (body !== undefined) headers["Content-Type"] = "application/json"
9
+
10
+ let res
11
+ try {
12
+ res = await fetch(`${apiUrl}${pathname}`, {
13
+ method,
14
+ headers,
15
+ body: body !== undefined ? JSON.stringify(body) : undefined,
16
+ })
17
+ } catch (e) {
18
+ return { ok: false, status: 0, json: { error: `network error: ${e.message}` } }
19
+ }
20
+
21
+ const text = await res.text()
22
+ let json = null
23
+ if (text) {
24
+ try {
25
+ json = JSON.parse(text)
26
+ } catch {
27
+ json = { raw: text }
28
+ }
29
+ }
30
+ return { ok: res.ok, status: res.status, json }
31
+ }
package/lib/config.mjs ADDED
@@ -0,0 +1,44 @@
1
+ /**
2
+ * Config + auth resolution for the testlab CLI.
3
+ *
4
+ * Resolution order (first hit wins):
5
+ * API key: --key flag → $TESTLAB_API_KEY → ~/.test-lab/config.json
6
+ * API base: --api-url → $TESTLAB_API_URL → config → https://www.test-lab.ai
7
+ */
8
+ import fs from "node:fs"
9
+ import os from "node:os"
10
+ import path from "node:path"
11
+
12
+ export const DEFAULT_API_URL = "https://www.test-lab.ai"
13
+ const CONFIG_DIR = path.join(os.homedir(), ".test-lab")
14
+ export const CONFIG_PATH = path.join(CONFIG_DIR, "config.json")
15
+
16
+ export function loadConfig() {
17
+ try {
18
+ return JSON.parse(fs.readFileSync(CONFIG_PATH, "utf8"))
19
+ } catch {
20
+ return {}
21
+ }
22
+ }
23
+
24
+ export function saveConfig(cfg) {
25
+ fs.mkdirSync(CONFIG_DIR, { recursive: true })
26
+ fs.writeFileSync(CONFIG_PATH, JSON.stringify(cfg, null, 2) + "\n", { mode: 0o600 })
27
+ // writeFileSync's mode only applies on create; force it in case it existed.
28
+ try {
29
+ fs.chmodSync(CONFIG_PATH, 0o600)
30
+ } catch {
31
+ /* best effort */
32
+ }
33
+ return CONFIG_PATH
34
+ }
35
+
36
+ export function resolveAuth(flags) {
37
+ const cfg = loadConfig()
38
+ const apiKey = flags.key || process.env.TESTLAB_API_KEY || cfg.apiKey || null
39
+ const apiUrl = (flags["api-url"] || process.env.TESTLAB_API_URL || cfg.apiUrl || DEFAULT_API_URL).replace(
40
+ /\/+$/,
41
+ ""
42
+ )
43
+ return { apiKey, apiUrl }
44
+ }
package/lib/import.mjs ADDED
@@ -0,0 +1,145 @@
1
+ /**
2
+ * Import orchestration: load plan files, topo-sort by pre-step deps, upsert
3
+ * credentials, then create plans one request each (wiring intra-batch pre-steps
4
+ * to the concrete ids returned along the way).
5
+ */
6
+ import fs from "node:fs"
7
+ import path from "node:path"
8
+ import { topoSortPlans } from "./toposort.mjs"
9
+ import { apiFetch } from "./api.mjs"
10
+
11
+ /**
12
+ * Load a plan file or a directory of *.json files into { credentials, plans }.
13
+ * Each file may be: a single plan object, an array of plans, or an object
14
+ * shaped { credentials?: [...], plans: [...] }.
15
+ */
16
+ export function loadImportFile(target) {
17
+ const stat = fs.statSync(target)
18
+ const files = []
19
+ if (stat.isDirectory()) {
20
+ for (const f of fs.readdirSync(target).sort()) {
21
+ if (f.endsWith(".json")) files.push(path.join(target, f))
22
+ }
23
+ if (files.length === 0) throw new Error(`No .json files found in ${target}`)
24
+ } else {
25
+ files.push(target)
26
+ }
27
+
28
+ const credentials = []
29
+ const plans = []
30
+ for (const file of files) {
31
+ let data
32
+ try {
33
+ data = JSON.parse(fs.readFileSync(file, "utf8"))
34
+ } catch (e) {
35
+ throw new Error(`${file}: invalid JSON (${e.message})`)
36
+ }
37
+ if (Array.isArray(data)) {
38
+ plans.push(...data)
39
+ } else if (data && Array.isArray(data.plans)) {
40
+ plans.push(...data.plans)
41
+ if (Array.isArray(data.credentials)) credentials.push(...data.credentials)
42
+ } else if (data && typeof data.name === "string" && typeof data.prompt === "string") {
43
+ plans.push(data)
44
+ } else {
45
+ throw new Error(`${file}: expected a plan object, an array, or { plans, credentials }`)
46
+ }
47
+ }
48
+ return { credentials, plans }
49
+ }
50
+
51
+ /** Resolve a plan's preSteps for the API: intra-batch refs → concrete ids. */
52
+ export function normalizePreSteps(preSteps, refToId) {
53
+ if (!Array.isArray(preSteps) || preSteps.length === 0) return undefined
54
+ return preSteps.map((ps) => {
55
+ if (ps.ref != null && refToId.has(ps.ref)) {
56
+ return { testPlanId: refToId.get(ps.ref), inputValues: ps.inputValues }
57
+ }
58
+ if (ps.testPlanId != null) return { testPlanId: ps.testPlanId, inputValues: ps.inputValues }
59
+ if (ps.name) return { name: ps.name, inputValues: ps.inputValues }
60
+ // Unreachable: topoSortPlans rejects a ref with no in-batch match and no
61
+ // name/testPlanId fallback before we get here. Throw loudly rather than
62
+ // silently shipping a bogus pre-step to the server.
63
+ throw new Error(`Cannot resolve pre-step: ${JSON.stringify(ps)}`)
64
+ })
65
+ }
66
+
67
+ export async function runImport({ apiUrl, apiKey, credentials = [], plans = [], dryRun = false, log = console.log }) {
68
+ if (plans.length === 0) {
69
+ log("Nothing to import (0 plans).")
70
+ return { ok: true, created: 0, failed: 0 }
71
+ }
72
+
73
+ // Basic per-plan shape check before any network calls.
74
+ for (let i = 0; i < plans.length; i++) {
75
+ const p = plans[i]
76
+ if (!p || typeof p.name !== "string" || typeof p.prompt !== "string") {
77
+ const msg = `Plan #${i + 1} is missing a string name and/or prompt`
78
+ log(`✗ ${msg}`)
79
+ return { ok: false, error: msg }
80
+ }
81
+ }
82
+
83
+ const sorted = topoSortPlans(plans)
84
+ if (!sorted.ok) {
85
+ log(`✗ ${sorted.error}`)
86
+ return { ok: false, error: sorted.error }
87
+ }
88
+
89
+ if (dryRun) {
90
+ log(`Dry run — ${plans.length} plan(s), ${credentials.length} credential(s). Creation order:`)
91
+ sorted.order.forEach((idx, n) => {
92
+ const p = plans[idx]
93
+ log(` ${n + 1}. ${p.name}${p.ref ? ` (ref: ${p.ref})` : ""}`)
94
+ })
95
+ return { ok: true, dryRun: true }
96
+ }
97
+
98
+ // Credentials first so plans referencing {{credentials.X}} have them.
99
+ let credOk = 0
100
+ for (const c of credentials) {
101
+ if (!c || typeof c.key !== "string") {
102
+ log(` ✗ credential: missing key`)
103
+ continue
104
+ }
105
+ const r = await apiFetch(apiUrl, apiKey, "POST", "/api/v1/credentials", { key: c.key, value: c.value })
106
+ if (r.ok) {
107
+ credOk++
108
+ log(` ✓ credential ${c.key}`)
109
+ } else {
110
+ log(` ✗ credential ${c.key}: ${r.json?.error || r.status}`)
111
+ }
112
+ }
113
+
114
+ const refToId = new Map()
115
+ let created = 0
116
+ let failed = 0
117
+ for (const idx of sorted.order) {
118
+ const plan = plans[idx]
119
+ const payload = {
120
+ name: plan.name,
121
+ prompt: plan.prompt,
122
+ testType: plan.testType,
123
+ agentType: plan.agentType,
124
+ devices: plan.devices,
125
+ labels: plan.labels,
126
+ cookies: plan.cookies,
127
+ headers: plan.headers,
128
+ failOnPreStepFailure: plan.failOnPreStepFailure,
129
+ preSteps: normalizePreSteps(plan.preSteps, refToId),
130
+ }
131
+ const r = await apiFetch(apiUrl, apiKey, "POST", "/api/v1/test-plans", payload)
132
+ if (r.ok) {
133
+ created++
134
+ const id = r.json?.testPlan?.id
135
+ if (plan.ref != null && id != null) refToId.set(plan.ref, id)
136
+ log(` ✓ ${plan.name}${id != null ? ` (#${id})` : ""}`)
137
+ } else {
138
+ failed++
139
+ log(` ✗ ${plan.name}: ${r.json?.error || r.status}`)
140
+ }
141
+ }
142
+
143
+ log(`\n${created} plan(s) created, ${failed} failed${credentials.length ? `, ${credOk}/${credentials.length} credential(s)` : ""}`)
144
+ return { ok: failed === 0, created, failed }
145
+ }
package/lib/login.mjs ADDED
@@ -0,0 +1,93 @@
1
+ /**
2
+ * Browser device-login. Reuses test-lab's existing one-time-code → API-key
3
+ * handshake (the same pattern the Chrome extension uses):
4
+ *
5
+ * 1. CLI starts a localhost callback server + opens the browser to
6
+ * <api>/cli/authorize?state=…&port=…
7
+ * 2. The signed-in user approves; the page redirects to
8
+ * http://127.0.0.1:<port>/callback?code=…&state=…
9
+ * 3. CLI verifies `state`, then exchanges the one-time code for a full-scope
10
+ * API key at POST /api/v1/cli/token (direct HTTPS — the secret never rides
11
+ * in the browser redirect).
12
+ *
13
+ * If the server doesn't support the flow (older deployment) the caller falls
14
+ * back to manual key paste.
15
+ */
16
+ import http from "node:http"
17
+ import crypto from "node:crypto"
18
+ import { spawn } from "node:child_process"
19
+ import { apiFetch } from "./api.mjs"
20
+
21
+ function openBrowser(url) {
22
+ let cmd
23
+ let args
24
+ if (process.platform === "darwin") {
25
+ cmd = "open"
26
+ args = [url]
27
+ } else if (process.platform === "win32") {
28
+ cmd = "cmd"
29
+ args = ["/c", "start", "", url]
30
+ } else {
31
+ cmd = "xdg-open"
32
+ args = [url]
33
+ }
34
+ try {
35
+ spawn(cmd, args, { stdio: "ignore", detached: true }).unref()
36
+ } catch {
37
+ /* user can open the URL manually */
38
+ }
39
+ }
40
+
41
+ export function browserLogin(apiUrl, { timeoutMs = 180000 } = {}) {
42
+ const state = crypto.randomBytes(16).toString("hex")
43
+
44
+ return new Promise((resolve, reject) => {
45
+ const server = http.createServer(async (req, res) => {
46
+ const url = new URL(req.url, "http://127.0.0.1")
47
+ if (url.pathname !== "/callback") {
48
+ res.writeHead(404)
49
+ res.end()
50
+ return
51
+ }
52
+ const finish = (status, html) => {
53
+ res.writeHead(status, { "Content-Type": "text/html" })
54
+ res.end(`<!doctype html><meta charset="utf-8"><body style="font-family:system-ui;padding:3rem">${html}</body>`)
55
+ }
56
+ const code = url.searchParams.get("code")
57
+ const gotState = url.searchParams.get("state")
58
+ if (!code || gotState !== state) {
59
+ finish(400, "<h2>Login failed</h2><p>Invalid state. Return to your terminal and try again.</p>")
60
+ server.close()
61
+ clearTimeout(timer)
62
+ reject(new Error("state mismatch"))
63
+ return
64
+ }
65
+ const r = await apiFetch(apiUrl, null, "POST", "/api/v1/cli/token", { code })
66
+ if (r.ok && r.json?.apiKey) {
67
+ finish(200, "<h2>Authorized ✓</h2><p>You can close this tab and return to your terminal.</p>")
68
+ server.close()
69
+ clearTimeout(timer)
70
+ resolve({ apiKey: r.json.apiKey, accountId: r.json.accountId, email: r.json.email })
71
+ } else {
72
+ finish(r.status === 404 ? 404 : 400, `<h2>Login failed</h2><p>${r.json?.error || "Token exchange failed."}</p>`)
73
+ server.close()
74
+ clearTimeout(timer)
75
+ reject(new Error(r.json?.error || `token exchange failed (${r.status})`))
76
+ }
77
+ })
78
+
79
+ server.on("error", (e) => reject(e))
80
+
81
+ const timer = setTimeout(() => {
82
+ server.close()
83
+ reject(new Error("login timed out after 3 minutes"))
84
+ }, timeoutMs)
85
+
86
+ server.listen(0, "127.0.0.1", () => {
87
+ const { port } = server.address()
88
+ const authUrl = `${apiUrl}/cli/authorize?state=${state}&port=${port}`
89
+ console.log(`Opening your browser to authorize the CLI:\n ${authUrl}\n`)
90
+ openBrowser(authUrl)
91
+ })
92
+ })
93
+ }
@@ -0,0 +1,77 @@
1
+ /**
2
+ * Topologically sort import plans by intra-batch pre-step `ref` dependencies.
3
+ *
4
+ * Edge: plan A depends on plan B when A has a preStep `{ ref: <B.ref> }`.
5
+ * Pre-steps that reference EXISTING plans by `name`/`testPlanId` are NOT
6
+ * intra-batch edges (the server resolves those) and are ignored here.
7
+ *
8
+ * Returns:
9
+ * { ok: true, order: number[] } // indices into `plans`, deps first
10
+ * { ok: false, error: string, refs?: [] } // duplicate/unknown ref, self-ref, cycle
11
+ *
12
+ * Pure + dependency-free so it's unit-testable under plain `node`.
13
+ */
14
+ export function topoSortPlans(plans) {
15
+ const n = plans.length
16
+
17
+ // Map every declared ref to its plan index (reject duplicates).
18
+ const refToIndex = new Map()
19
+ for (let i = 0; i < n; i++) {
20
+ const ref = plans[i] && plans[i].ref
21
+ if (ref == null) continue
22
+ if (typeof ref !== "string" || !ref.trim()) {
23
+ return { ok: false, error: `Plan #${i + 1} has an invalid ref (must be a non-empty string)` }
24
+ }
25
+ if (refToIndex.has(ref)) {
26
+ return { ok: false, error: `Duplicate plan ref "${ref}"`, refs: [ref] }
27
+ }
28
+ refToIndex.set(ref, i)
29
+ }
30
+
31
+ const adjacency = Array.from({ length: n }, () => [])
32
+ const indegree = new Array(n).fill(0)
33
+
34
+ for (let i = 0; i < n; i++) {
35
+ const preSteps = Array.isArray(plans[i].preSteps) ? plans[i].preSteps : []
36
+ for (const ps of preSteps) {
37
+ if (!ps || ps.ref == null) continue // not an intra-batch edge
38
+ const ref = ps.ref
39
+ if (ref === plans[i].ref) {
40
+ return { ok: false, error: `Plan "${ref}" lists itself as a pre-step`, refs: [ref] }
41
+ }
42
+ if (!refToIndex.has(ref)) {
43
+ // A ref pointing outside the batch is only OK if a name/id fallback is
44
+ // present (then it targets an already-existing plan, server-resolved).
45
+ if (ps.name || ps.testPlanId) continue
46
+ return {
47
+ ok: false,
48
+ error: `Pre-step ref "${ref}" matches no plan in this import (and has no name/testPlanId fallback)`,
49
+ refs: [ref],
50
+ }
51
+ }
52
+ const dep = refToIndex.get(ref)
53
+ adjacency[dep].push(i)
54
+ indegree[i]++
55
+ }
56
+ }
57
+
58
+ // Kahn's algorithm (iterative — no recursion depth concerns).
59
+ const queue = []
60
+ for (let i = 0; i < n; i++) if (indegree[i] === 0) queue.push(i)
61
+ const order = []
62
+ while (queue.length > 0) {
63
+ const node = queue.shift()
64
+ order.push(node)
65
+ for (const next of adjacency[node]) {
66
+ if (--indegree[next] === 0) queue.push(next)
67
+ }
68
+ }
69
+
70
+ if (order.length < n) {
71
+ const cycle = []
72
+ for (let i = 0; i < n; i++) if (indegree[i] > 0) cycle.push(plans[i].ref ?? `#${i + 1}`)
73
+ return { ok: false, error: `Pre-step dependency cycle among: ${cycle.join(", ")}`, refs: cycle }
74
+ }
75
+
76
+ return { ok: true, order }
77
+ }
package/package.json ADDED
@@ -0,0 +1,36 @@
1
+ {
2
+ "name": "@test-lab-ai/cli",
3
+ "version": "0.1.0",
4
+ "description": "Import existing test plans into test-lab.ai from the command line (or an AI agent).",
5
+ "type": "module",
6
+ "bin": {
7
+ "testlab": "bin/testlab.mjs"
8
+ },
9
+ "engines": {
10
+ "node": ">=18"
11
+ },
12
+ "files": [
13
+ "bin",
14
+ "lib",
15
+ "examples",
16
+ "README.md",
17
+ "AGENTS.md"
18
+ ],
19
+ "scripts": {
20
+ "test": "node test/toposort.test.mjs",
21
+ "prepublishOnly": "npm test"
22
+ },
23
+ "keywords": [
24
+ "test-lab",
25
+ "test-lab.ai",
26
+ "testing",
27
+ "qa",
28
+ "e2e",
29
+ "cli"
30
+ ],
31
+ "homepage": "https://test-lab.ai",
32
+ "license": "MIT",
33
+ "publishConfig": {
34
+ "access": "public"
35
+ }
36
+ }