@sifurai/cli 0.0.0-stage → 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,3 +1,134 @@
1
- # Temporary Holding Version
1
+ # Sifur CLI
2
2
 
3
- This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
3
+ Run a Sifur ticket's acceptance cases against your own test server, from your own machine, and
4
+ send the results back to Sifur.
5
+
6
+ Use it when your app only runs on your team's servers, behind a firewall or VPN that Sifur can't
7
+ reach. Sifur never deploys to your server, never holds VPN or server credentials, and never connects
8
+ into your network: you deploy the ticket's branch, this tool runs the cases on your machine, and only
9
+ the results and screenshots go to Sifur.
10
+
11
+ ## Requirements
12
+
13
+ - Node.js 20 or newer
14
+ - Google Chrome or Microsoft Edge (the tool drives the browser you already have; it downloads none)
15
+ - A machine that can reach your test server (usually: on your company VPN)
16
+
17
+ ## Install
18
+
19
+ Nothing to install up front:
20
+
21
+ ```sh
22
+ npx @sifurai/cli login
23
+ ```
24
+
25
+ Or install it globally to get a `sifur` command:
26
+
27
+ ```sh
28
+ npm install -g @sifurai/cli
29
+ sifur login
30
+ ```
31
+
32
+ ## Sign in
33
+
34
+ ```sh
35
+ sifur login
36
+ ```
37
+
38
+ This prints a code and opens Sifur in your browser. Check that the code matches, approve it, and
39
+ the CLI is signed in. The sign-in is listed with your other sessions in Sifur, where you can revoke it.
40
+ It follows your organization's single sign-on and MFA rules, like the console.
41
+
42
+ The token is saved with owner-only permissions (0600) in:
43
+
44
+ - macOS and Linux: `~/.config/sifur/credentials.json` (or `$XDG_CONFIG_HOME/sifur`)
45
+ - Windows: `%APPDATA%\sifur\credentials.json`
46
+
47
+ To sign out (and revoke the session):
48
+
49
+ ```sh
50
+ sifur logout
51
+ ```
52
+
53
+ ## Run a ticket's acceptance cases
54
+
55
+ When a ticket is in **Acceptance testing**, its Overview shows the branch and the commit of each
56
+ repository to deploy. Deploy them to your test server, then run:
57
+
58
+ ```sh
59
+ sifur accept AZA-12 --url https://test.example.internal
60
+ ```
61
+
62
+ `AZA-12` is the ticket's key; its uuid works too. The CLI:
63
+
64
+ 1. Finds the ticket in your organizations and downloads its cases.
65
+ 2. Runs each UI case in a headless browser against `--url` (paths in the cases are relative to
66
+ it), taking a screenshot after every step. Add `--headed` to watch.
67
+ 3. For a step that needs a person's eye, shows you the screenshot's path and asks whether it looks
68
+ right.
69
+ 4. Runs automated cases in your checkout when you pass `--workdir <repository>=<dir>` (repeat it
70
+ for each repository); otherwise asks you for the result.
71
+ 5. For a case with no script, shows its steps and expected result and asks you for the result:
72
+ passed, failed or blocked, with a note.
73
+ 6. Uploads the screenshots and results, prints a summary, and links to the ticket.
74
+
75
+ ```sh
76
+ sifur accept AZA-12 --url https://test.example.internal \
77
+ --workdir api=~/src/azalea-api --workdir web=~/src/azalea-web
78
+ ```
79
+
80
+ Every case passed: the ticket moves on to approval. Any failed or blocked: it goes back to be fixed.
81
+
82
+ Screenshots and results are kept in `./.sifur/AZA-12/` (add `.sifur/` to your `.gitignore`). If the
83
+ upload fails, send them again with `sifur accept AZA-12 --upload`.
84
+
85
+ ## VPN: split tunnel or full tunnel
86
+
87
+ With a **split-tunnel** VPN (only company traffic goes through it), one command does everything:
88
+ your machine reaches the test server and Sifur at the same time.
89
+
90
+ With a **full-tunnel** VPN that blocks the internet while it's up, work in three steps:
91
+
92
+ ```sh
93
+ # VPN down
94
+ sifur accept AZA-12 --download
95
+
96
+ # VPN up: runs the cases, talks only to your test server
97
+ sifur accept AZA-12 --run --url https://test.example.internal
98
+
99
+ # VPN down
100
+ sifur accept AZA-12 --upload
101
+ ```
102
+
103
+ `--download` saves the cases in `./.sifur/AZA-12/`, `--run` runs them from there and saves the
104
+ results next to them, and `--upload` sends them. Run all three from the same directory.
105
+
106
+ ## Secrets
107
+
108
+ A case can use a placeholder such as `{{ADMIN_PASSWORD}}` for a value it types, like a test
109
+ account's password. The CLI takes it from the environment variable `SIFUR_SECRET_ADMIN_PASSWORD`
110
+ or, when that isn't set, asks you for it without echoing. Secrets stay on your machine: they are
111
+ never written to `.sifur/`, never sent to Sifur, and are replaced with `[secret]` in any error
112
+ message.
113
+
114
+ ```sh
115
+ SIFUR_SECRET_ADMIN_PASSWORD='…' sifur accept AZA-12 --url https://test.example.internal
116
+ ```
117
+
118
+ ## Options
119
+
120
+ | Option | |
121
+ |---|---|
122
+ | `--url <url>` | Your test server, where the ticket's branch is deployed |
123
+ | `--headed` | Show the browser while the cases run |
124
+ | `--workdir <repository>=<dir>` | Run that repository's automated cases in `<dir>`; repeatable |
125
+ | `--download` / `--run` / `--upload` | The three steps for a full-tunnel VPN |
126
+ | `--api <url>` | Sifur's API, for `login` (default `https://api.sifur.ai`, or `SIFUR_API_URL`) |
127
+
128
+ ## Exit codes
129
+
130
+ | Code | Meaning |
131
+ |---|---|
132
+ | 0 | Every case passed (or `--download` finished) |
133
+ | 1 | At least one case failed or was blocked |
134
+ | 2 | An error: not signed in, the ticket isn't in acceptance testing, Sifur unreachable, no Chrome or Edge, … |
package/dist/accept.js ADDED
@@ -0,0 +1,304 @@
1
+ import { existsSync, statSync } from 'node:fs';
2
+ import { readFile } from 'node:fs/promises';
3
+ import { basename, extname, join, resolve } from 'node:path';
4
+ import { ApiError, CliError } from './api.js';
5
+ import { runAutomated } from './automated.js';
6
+ import { launchBrowser } from './browser.js';
7
+ import { askStatus, confirm } from './io.js';
8
+ import { resolveTicket } from './resolve.js';
9
+ import { buildResultsPayload, combine, outcome } from './results.js';
10
+ import { runUiCase } from './runner.js';
11
+ import { collectSecrets, redact, secretNames } from './secrets.js';
12
+ import { findTicketDir, loadCases, loadResults, saveCases, saveResults } from './store.js';
13
+ export const MAX_SCREENSHOT_BYTES = 5 * 1024 * 1024;
14
+ export async function download(api, ref) {
15
+ const { ticketUuid, organization } = await resolveTicket(api, ref);
16
+ try {
17
+ const acceptance = await api.request(`tickets/${ticketUuid}/acceptance`, { organization: organization.uuid });
18
+ return { api: api.base, organizationUuid: organization.uuid, acceptance };
19
+ }
20
+ catch (error) {
21
+ if (error instanceof ApiError && error.status === 409) {
22
+ throw new CliError(`${ref} isn't waiting for acceptance testing: ${error.message}`);
23
+ }
24
+ throw error;
25
+ }
26
+ }
27
+ export function describeDeploy(acceptance, io) {
28
+ const { ticket, heads } = acceptance;
29
+ io.out(`${ticket.key}: ${ticket.title}`);
30
+ if (ticket.branch)
31
+ io.out(`Branch: ${ticket.branch}`);
32
+ const repositories = Object.entries(heads ?? {});
33
+ if (repositories.length > 0) {
34
+ io.out('The test server should be running:');
35
+ for (const [repository, sha] of repositories)
36
+ io.out(` ${repository} ${sha}`);
37
+ }
38
+ io.out(`${acceptance.cases.length} case${acceptance.cases.length === 1 ? '' : 's'}`);
39
+ io.out();
40
+ }
41
+ function printCase(io, testCase) {
42
+ for (const [index, step] of (testCase.steps ?? []).entries())
43
+ io.out(` ${index + 1}. ${step}`);
44
+ if (testCase.expected)
45
+ io.out(` Expected: ${testCase.expected}`);
46
+ }
47
+ async function askNote(io, prompt) {
48
+ return (await io.ask(prompt)).trim();
49
+ }
50
+ async function askEngineer(io, testCase, why) {
51
+ io.out(` ${why}. Check it by hand:`);
52
+ printCase(io, testCase);
53
+ if (testCase.automatedTestRef)
54
+ io.out(` Test: ${testCase.automatedTestRef}`);
55
+ const status = await askStatus(io, ' Result');
56
+ const note = await askNote(io, status === 'passed' ? ' Note (optional): ' : ' What happened? ');
57
+ return { status, note };
58
+ }
59
+ function judgeWith(io) {
60
+ return async (step, screenshot, testCase) => {
61
+ io.out(` Judge: ${step.caption ?? 'Does the page look right?'}`);
62
+ if (testCase.expected)
63
+ io.out(` Expected: ${testCase.expected}`);
64
+ io.out(screenshot ? ` Screenshot: ${screenshot}` : ' (No screenshot could be taken.)');
65
+ if (await confirm(io, ' Does it look right?'))
66
+ return { ok: true };
67
+ const note = await askNote(io, " What's wrong? ");
68
+ return { ok: false, note: note || undefined };
69
+ };
70
+ }
71
+ function workdirFor(testCase, workdirs) {
72
+ if (testCase.repository && workdirs[testCase.repository])
73
+ return workdirs[testCase.repository];
74
+ const all = Object.values(workdirs);
75
+ return !testCase.repository && all.length === 1 ? all[0] : null;
76
+ }
77
+ function summarize(status, steps, notes) {
78
+ if (notes.length > 0)
79
+ return notes.join('\n');
80
+ if (status === 'passed' && steps.length > 0)
81
+ return `All ${steps.length} step${steps.length === 1 ? '' : 's'} passed.`;
82
+ return status === 'passed' ? 'Passed.' : `The case ${status}.`;
83
+ }
84
+ /** Runs every case on this machine against the engineer's test server. Nothing here talks to Sifur. */
85
+ export async function runCases(acceptance, options, deps) {
86
+ const { io } = deps;
87
+ const workdirs = options.workdirs ?? {};
88
+ for (const [repository, dir] of Object.entries(workdirs)) {
89
+ if (!existsSync(dir) || !statSync(dir).isDirectory())
90
+ throw new CliError(`--workdir ${repository}=${dir}: no such directory.`);
91
+ }
92
+ const scripted = acceptance.cases.filter((c) => c.check !== 'automated' && (c.uiScript?.length ?? 0) > 0);
93
+ const secrets = await collectSecrets(secretNames(scripted), io, deps.env);
94
+ let browser = null;
95
+ const results = [];
96
+ try {
97
+ for (const testCase of acceptance.cases) {
98
+ io.out(`${testCase.id} ${testCase.title}`);
99
+ const statuses = [];
100
+ const notes = [];
101
+ let steps = [];
102
+ let output;
103
+ if (testCase.check === 'ui' || testCase.check === 'both') {
104
+ if (scripted.includes(testCase)) {
105
+ browser ??= await (deps.launch ?? launchBrowser)({ headed: options.headed });
106
+ const ui = await runUiCase(browser, testCase, {
107
+ baseUrl: options.baseUrl,
108
+ outDir: options.outDir,
109
+ secrets,
110
+ judge: judgeWith(io),
111
+ timeoutMs: deps.timeoutMs,
112
+ });
113
+ for (const step of ui.steps)
114
+ io.out(` ${step.ok ? 'ok ' : 'FAIL'} ${step.n}. ${step.caption}${step.error ? `: ${step.error}` : ''}`);
115
+ statuses.push(ui.status);
116
+ notes.push(...ui.notes);
117
+ steps = ui.steps;
118
+ }
119
+ else {
120
+ const answer = await askEngineer(io, testCase, 'No UI script');
121
+ statuses.push(answer.status);
122
+ if (answer.note)
123
+ notes.push(answer.note);
124
+ }
125
+ }
126
+ if (testCase.check === 'automated' || testCase.check === 'both') {
127
+ const cwd = workdirFor(testCase, workdirs);
128
+ let run;
129
+ if (testCase.automatedTestRef && cwd) {
130
+ io.out(` $ ${testCase.automatedTestRef} (in ${cwd})`);
131
+ run = await runAutomated(testCase.automatedTestRef, resolve(cwd), { echo: deps.echo });
132
+ run = { ...run, note: redact(run.note, secrets), output: redact(run.output ?? '', secrets) };
133
+ }
134
+ else {
135
+ const why = !testCase.automatedTestRef
136
+ ? 'No test command'
137
+ : `No --workdir ${testCase.repository ?? '<repository>'}=<dir> to run \`${testCase.automatedTestRef}\` in`;
138
+ const answer = await askEngineer(io, testCase, why);
139
+ run = { status: answer.status, note: answer.note };
140
+ }
141
+ statuses.push(run.status);
142
+ if (run.note)
143
+ notes.push(run.note);
144
+ if (run.output !== undefined)
145
+ output = run.output;
146
+ }
147
+ const status = combine(statuses);
148
+ io.out(` → ${status}`);
149
+ io.out();
150
+ results.push({
151
+ caseUuid: testCase.uuid,
152
+ id: testCase.id,
153
+ status,
154
+ notes: summarize(status, steps, notes),
155
+ steps,
156
+ ...(output !== undefined ? { output } : {}),
157
+ });
158
+ }
159
+ }
160
+ finally {
161
+ await browser?.close().catch(() => { });
162
+ }
163
+ return { baseUrl: options.baseUrl, ranAt: new Date().toISOString(), results };
164
+ }
165
+ async function uploadScreenshot(api, saved, path) {
166
+ const bytes = await readFile(path);
167
+ const type = extname(path).toLowerCase() === '.png' ? 'image/png' : 'image/jpeg';
168
+ const form = new FormData();
169
+ form.append('file', new Blob([bytes], { type }), basename(path));
170
+ const uploaded = await api.request(`tickets/${saved.acceptance.ticket.uuid}/acceptance/screenshots`, {
171
+ method: 'POST',
172
+ form,
173
+ organization: saved.organizationUuid,
174
+ });
175
+ return uploaded.uuid;
176
+ }
177
+ export async function upload(api, saved, results, io) {
178
+ const screenshots = new Map();
179
+ const paths = [...new Set(results.results.flatMap((r) => r.steps.map((s) => s.screenshot)).filter((p) => Boolean(p)))];
180
+ if (paths.length > 0)
181
+ io.out(`Uploading ${paths.length} screenshot${paths.length === 1 ? '' : 's'}…`);
182
+ for (const path of paths) {
183
+ if (!existsSync(path)) {
184
+ io.out(` Skipped ${path}: the file is gone.`);
185
+ continue;
186
+ }
187
+ if (statSync(path).size > MAX_SCREENSHOT_BYTES) {
188
+ io.out(` Skipped ${path}: larger than 5 MB.`);
189
+ continue;
190
+ }
191
+ screenshots.set(path, await uploadScreenshot(api, saved, path));
192
+ }
193
+ const payload = buildResultsPayload(saved.acceptance.ticket.stateVersion, results.baseUrl, results.results, screenshots);
194
+ try {
195
+ return await api.request(`tickets/${saved.acceptance.ticket.uuid}/acceptance/results`, {
196
+ method: 'POST',
197
+ body: payload,
198
+ organization: saved.organizationUuid,
199
+ });
200
+ }
201
+ catch (error) {
202
+ if (error instanceof ApiError && error.status === 409) {
203
+ throw new CliError(`Sifur refused the results: ${error.message} The ticket has moved on since the cases were downloaded; run accept again.`);
204
+ }
205
+ throw error;
206
+ }
207
+ }
208
+ export function printSummary(io, results) {
209
+ io.out('Summary');
210
+ for (const result of results)
211
+ io.out(` ${result.status.padEnd(7)} ${result.id}`);
212
+ const overall = outcome(results);
213
+ const counts = ['passed', 'failed', 'blocked']
214
+ .map((status) => [status, results.filter((r) => r.status === status).length])
215
+ .filter(([, count]) => count > 0)
216
+ .map(([status, count]) => `${count} ${status}`);
217
+ io.out(`${overall === 'passed' ? 'Passed' : 'Not passed'}: ${counts.join(', ') || 'no cases'}`);
218
+ return overall;
219
+ }
220
+ export function ticketLink(consoleOrigin, ticketUuid) {
221
+ return consoleOrigin ? `${consoleOrigin.replace(/\/+$/, '')}/tickets/${ticketUuid}` : null;
222
+ }
223
+ /** The whole `accept` command; returns the exit code (0 all passed, 1 any not passed). Errors throw, for exit 2. */
224
+ export async function accept(api, args, deps) {
225
+ const { io } = deps;
226
+ if (args.mode === 'download') {
227
+ if (!api)
228
+ throw new CliError('Run `sifur login` first.');
229
+ const saved = await download(api, args.ref);
230
+ const dir = await saveCases(args.cwd, saved);
231
+ describeDeploy(saved.acceptance, io);
232
+ io.out(`Saved to ${dir}. Run \`sifur accept ${saved.acceptance.ticket.key} --run --url <test url>\` next.`);
233
+ return 0;
234
+ }
235
+ if (args.mode === 'upload') {
236
+ if (!api)
237
+ throw new CliError('Run `sifur login` first.');
238
+ const dir = await findTicketDir(args.cwd, args.ref);
239
+ if (!dir)
240
+ throw new CliError(`Nothing downloaded for ${args.ref} in ${join(args.cwd, '.sifur')}.`);
241
+ const saved = await loadCases(dir);
242
+ if (saved.api !== api.base)
243
+ throw new CliError(`${args.ref} was downloaded from ${saved.api}, but you're signed in to ${api.base}.`);
244
+ const results = await loadResults(dir);
245
+ return finish(api, saved, results, args, io);
246
+ }
247
+ if (!args.url)
248
+ throw new CliError('--url <test url> is required: the test server the branch is deployed to.');
249
+ const baseUrl = parseBaseUrl(args.url);
250
+ let saved;
251
+ let dir;
252
+ if (args.mode === 'run') {
253
+ const found = await findTicketDir(args.cwd, args.ref);
254
+ if (!found)
255
+ throw new CliError(`Nothing downloaded for ${args.ref}. Run with --download first.`);
256
+ dir = found;
257
+ saved = await loadCases(dir);
258
+ }
259
+ else {
260
+ if (!api)
261
+ throw new CliError('Run `sifur login` first.');
262
+ saved = await download(api, args.ref);
263
+ dir = await saveCases(args.cwd, saved);
264
+ }
265
+ describeDeploy(saved.acceptance, io);
266
+ const results = await runCases(saved.acceptance, { baseUrl, outDir: join(dir, 'screenshots'), headed: args.headed, workdirs: args.workdirs }, deps);
267
+ const resultsPath = await saveResults(dir, results);
268
+ if (args.mode === 'run') {
269
+ const overall = printSummary(io, results.results);
270
+ io.out(`Saved to ${resultsPath}. Run \`sifur accept ${saved.acceptance.ticket.key} --upload\` to send them.`);
271
+ return overall === 'passed' ? 0 : 1;
272
+ }
273
+ try {
274
+ return await finish(api, saved, results, args, io);
275
+ }
276
+ catch (error) {
277
+ io.out(`The results are saved in ${resultsPath}; send them later with \`sifur accept ${saved.acceptance.ticket.key} --upload\`.`);
278
+ throw error;
279
+ }
280
+ }
281
+ async function finish(api, saved, results, args, io) {
282
+ const response = await upload(api, saved, results, io);
283
+ io.out();
284
+ const overall = printSummary(io, results.results);
285
+ if (response?.run)
286
+ io.out(`Recorded as run ${response.run.runNumber}: ${response.run.result}.`);
287
+ if (response?.state)
288
+ io.out(`The ticket is now in ${response.state.replaceAll('_', ' ')}.`);
289
+ const link = ticketLink(args.consoleOrigin, saved.acceptance.ticket.uuid);
290
+ if (link)
291
+ io.out(link);
292
+ return overall === 'passed' ? 0 : 1;
293
+ }
294
+ export function parseBaseUrl(url) {
295
+ try {
296
+ const parsed = new URL(url);
297
+ if (!['http:', 'https:'].includes(parsed.protocol))
298
+ throw new Error();
299
+ return parsed.toString();
300
+ }
301
+ catch {
302
+ throw new CliError(`--url ${url} isn't an http(s) URL.`);
303
+ }
304
+ }
package/dist/api.js ADDED
@@ -0,0 +1,78 @@
1
+ export const DEFAULT_API = 'https://api.sifur.ai';
2
+ export class ApiError extends Error {
3
+ status;
4
+ code;
5
+ retryAfterSeconds;
6
+ constructor(message, status, code = null, retryAfterSeconds = null) {
7
+ super(message);
8
+ this.status = status;
9
+ this.code = code;
10
+ this.retryAfterSeconds = retryAfterSeconds;
11
+ this.name = 'ApiError';
12
+ }
13
+ }
14
+ /** Something the engineer has to act on, printed without a stack trace. */
15
+ export class CliError extends Error {
16
+ constructor(message) {
17
+ super(message);
18
+ this.name = 'CliError';
19
+ }
20
+ }
21
+ export function normalizeApi(api) {
22
+ return api.replace(/\/+$/, '');
23
+ }
24
+ export class Api {
25
+ token;
26
+ userAgent;
27
+ base;
28
+ constructor(base, token = null, userAgent = 'sifur-cli') {
29
+ this.token = token;
30
+ this.userAgent = userAgent;
31
+ this.base = normalizeApi(base);
32
+ }
33
+ withToken(token) {
34
+ return new Api(this.base, token, this.userAgent);
35
+ }
36
+ async request(path, options = {}) {
37
+ const headers = { Accept: 'application/json', 'User-Agent': this.userAgent };
38
+ if (this.token)
39
+ headers.Authorization = `Bearer ${this.token}`;
40
+ if (options.organization)
41
+ headers['X-Organization'] = options.organization;
42
+ let body;
43
+ if (options.form) {
44
+ body = options.form;
45
+ }
46
+ else if (options.body !== undefined) {
47
+ headers['Content-Type'] = 'application/json';
48
+ body = JSON.stringify(options.body);
49
+ }
50
+ let response;
51
+ try {
52
+ response = await fetch(`${this.base}/api/v1/${path.replace(/^\//, '')}`, {
53
+ method: options.method ?? (body === undefined ? 'GET' : 'POST'),
54
+ headers,
55
+ body,
56
+ });
57
+ }
58
+ catch (error) {
59
+ const cause = error.cause?.message;
60
+ throw new CliError(`Couldn't reach Sifur at ${this.base}${cause ? ` (${cause})` : ''}.`);
61
+ }
62
+ const text = await response.text();
63
+ let json = null;
64
+ try {
65
+ json = text ? JSON.parse(text) : null;
66
+ }
67
+ catch {
68
+ json = null;
69
+ }
70
+ if (!response.ok) {
71
+ const message = json?.message?.messages?.[0] ?? `Sifur answered ${response.status} ${response.statusText}`.trim();
72
+ const code = json?.code ?? json?.data?.code ?? null;
73
+ const retryAfter = Number(response.headers.get('retry-after'));
74
+ throw new ApiError(message, response.status, code, Number.isFinite(retryAfter) && retryAfter > 0 ? retryAfter : null);
75
+ }
76
+ return (json?.data ?? null);
77
+ }
78
+ }
@@ -0,0 +1,28 @@
1
+ import { spawn } from 'node:child_process';
2
+ const TAIL_LINES = 5;
3
+ export const MAX_OUTPUT_CHARS = 200_000;
4
+ /** The case's test command in the engineer's checkout; its output streams to the terminal, its tail goes in the note. */
5
+ export function runAutomated(command, cwd, options = {}) {
6
+ const echo = options.echo ?? ((chunk) => process.stdout.write(chunk));
7
+ return new Promise((resolve) => {
8
+ let output = '';
9
+ const child = spawn(command, { cwd, shell: true, stdio: ['ignore', 'pipe', 'pipe'] });
10
+ const collect = (chunk) => {
11
+ const text = chunk.toString();
12
+ echo(text);
13
+ output = (output + text).slice(-MAX_OUTPUT_CHARS);
14
+ };
15
+ child.stdout.on('data', collect);
16
+ child.stderr.on('data', collect);
17
+ child.on('error', (error) => resolve({ status: 'blocked', note: `\`${command}\` couldn't start in ${cwd}: ${error.message}` }));
18
+ child.on('close', (code, signal) => {
19
+ const tail = output.trimEnd().split('\n').slice(-TAIL_LINES).join('\n');
20
+ const ended = signal ? `was stopped by ${signal}` : `exited with ${code}`;
21
+ resolve({
22
+ status: code === 0 ? 'passed' : 'failed',
23
+ note: `\`${command}\` ${ended}.${tail ? `\n${tail}` : ''}`,
24
+ output,
25
+ });
26
+ });
27
+ });
28
+ }
package/dist/bin.js ADDED
@@ -0,0 +1,16 @@
1
+ #!/usr/bin/env node
2
+ import { describeError, main } from './cli.js';
3
+ import { terminalIo } from './io.js';
4
+ const io = terminalIo();
5
+ let code;
6
+ try {
7
+ code = await main(process.argv.slice(2), io);
8
+ }
9
+ catch (error) {
10
+ process.stderr.write(`${describeError(error)}\n`);
11
+ code = 2;
12
+ }
13
+ finally {
14
+ io.close();
15
+ }
16
+ process.exit(code);
@@ -0,0 +1,16 @@
1
+ import { chromium } from 'playwright-core';
2
+ import { CliError } from './api.js';
3
+ const CHANNELS = ['chrome', 'msedge'];
4
+ /** The engineer's own Chrome, else Edge: playwright-core downloads no browser of its own. */
5
+ export async function launchBrowser(options = {}) {
6
+ const failures = [];
7
+ for (const channel of CHANNELS) {
8
+ try {
9
+ return await chromium.launch({ channel, headless: !options.headed });
10
+ }
11
+ catch (error) {
12
+ failures.push(`${channel}: ${String(error.message ?? error).split('\n')[0]}`);
13
+ }
14
+ }
15
+ throw new CliError(`Couldn't start Google Chrome or Microsoft Edge. Install one of them (https://www.google.com/chrome/) and run this again.\n ${failures.join('\n ')}`);
16
+ }
package/dist/cli.js ADDED
@@ -0,0 +1,123 @@
1
+ import { readFileSync } from 'node:fs';
2
+ import { parseArgs } from 'node:util';
3
+ import { accept } from './accept.js';
4
+ import { Api, ApiError, CliError, DEFAULT_API, normalizeApi } from './api.js';
5
+ import { loadCredentials } from './config.js';
6
+ import { login, logout } from './login.js';
7
+ const USAGE = `Usage:
8
+ sifur login [--api <url>]
9
+ sifur logout
10
+ sifur accept <KEY|uuid> --url <test url> [--headed] [--workdir <repo>=<dir> …]
11
+ sifur accept <KEY|uuid> --download
12
+ sifur accept <KEY|uuid> --run --url <test url> [--headed] [--workdir <repo>=<dir> …]
13
+ sifur accept <KEY|uuid> --upload
14
+
15
+ Options:
16
+ --api <url> Sifur's API (default ${DEFAULT_API}, or SIFUR_API_URL)
17
+ --url <url> Your test server, where the ticket's branch is deployed
18
+ --headed Show the browser while the cases run
19
+ --workdir <repo>=<dir> Run that repository's automated cases in <dir>; repeatable
20
+ --download Save the cases to ./.sifur/<KEY>/ (before a full-tunnel VPN is up)
21
+ --run Run the saved cases and save the results (no connection to Sifur)
22
+ --upload Send the saved results (after the VPN is down)
23
+
24
+ Secrets: {{NAME}} in a case comes from SIFUR_SECRET_NAME, or is asked for. It never leaves this machine.
25
+ Exit codes: 0 every case passed, 1 any case failed or was blocked, 2 an error.`;
26
+ function version() {
27
+ try {
28
+ const pkg = JSON.parse(readFileSync(new URL('../package.json', import.meta.url), 'utf8'));
29
+ return pkg.version;
30
+ }
31
+ catch {
32
+ return 'unknown';
33
+ }
34
+ }
35
+ export function parseWorkdirs(values) {
36
+ const workdirs = {};
37
+ for (const value of values) {
38
+ const at = value.indexOf('=');
39
+ if (at <= 0 || at === value.length - 1)
40
+ throw new CliError(`--workdir ${value}: expected <repository>=<dir>.`);
41
+ workdirs[value.slice(0, at)] = value.slice(at + 1);
42
+ }
43
+ return workdirs;
44
+ }
45
+ export async function main(argv, io, env = process.env, cwd = process.cwd()) {
46
+ const { values, positionals } = parseArgs({
47
+ args: argv,
48
+ allowPositionals: true,
49
+ options: {
50
+ api: { type: 'string' },
51
+ url: { type: 'string' },
52
+ headed: { type: 'boolean', default: false },
53
+ workdir: { type: 'string', multiple: true, default: [] },
54
+ download: { type: 'boolean', default: false },
55
+ run: { type: 'boolean', default: false },
56
+ upload: { type: 'boolean', default: false },
57
+ help: { type: 'boolean', short: 'h', default: false },
58
+ version: { type: 'boolean', short: 'v', default: false },
59
+ },
60
+ });
61
+ const [command, ...rest] = positionals;
62
+ if (values.version) {
63
+ io.out(version());
64
+ return 0;
65
+ }
66
+ if (values.help || !command || command === 'help') {
67
+ io.out(USAGE);
68
+ return command || values.help ? 0 : 2;
69
+ }
70
+ const explicitApi = values.api ?? env.SIFUR_API_URL;
71
+ const userAgent = `sifur-cli/${version()} node/${process.versions.node}`;
72
+ switch (command) {
73
+ case 'login': {
74
+ const api = new Api(explicitApi ?? DEFAULT_API, null, userAgent);
75
+ await login(api, { io, env });
76
+ return 0;
77
+ }
78
+ case 'logout':
79
+ await logout(io, env);
80
+ return 0;
81
+ case 'accept': {
82
+ const ref = rest[0];
83
+ if (!ref || rest.length > 1)
84
+ throw new CliError('Usage: sifur accept <KEY|uuid> --url <test url>');
85
+ const modes = ['download', 'run', 'upload'].filter((mode) => values[mode]);
86
+ if (modes.length > 1)
87
+ throw new CliError('Use one of --download, --run and --upload at a time.');
88
+ const mode = modes[0] ?? 'full';
89
+ const credentials = await loadCredentials(env);
90
+ let api = null;
91
+ if (mode !== 'run') {
92
+ if (!credentials)
93
+ throw new CliError('Not signed in. Run `sifur login` first.');
94
+ if (explicitApi && normalizeApi(explicitApi) !== normalizeApi(credentials.api)) {
95
+ throw new CliError(`You're signed in to ${credentials.api}, not ${explicitApi}. Run \`sifur login --api ${explicitApi}\`.`);
96
+ }
97
+ api = new Api(credentials.api, credentials.token, userAgent);
98
+ }
99
+ return accept(api, {
100
+ ref,
101
+ mode,
102
+ url: values.url,
103
+ headed: values.headed,
104
+ workdirs: parseWorkdirs(values.workdir ?? []),
105
+ cwd,
106
+ consoleOrigin: env.SIFUR_CONSOLE_URL ?? credentials?.console ?? null,
107
+ }, { io, env });
108
+ }
109
+ default:
110
+ throw new CliError(`Unknown command "${command}".\n\n${USAGE}`);
111
+ }
112
+ }
113
+ export function describeError(error) {
114
+ if (error instanceof ApiError && error.status === 401)
115
+ return 'Your sign-in has expired or was revoked. Run `sifur login` again.';
116
+ if (error instanceof ApiError)
117
+ return `Sifur answered ${error.status}: ${error.message}`;
118
+ if (error instanceof CliError)
119
+ return error.message;
120
+ if (error instanceof TypeError && error.code?.startsWith('ERR_PARSE_ARGS'))
121
+ return `${error.message}\n\n${USAGE}`;
122
+ return error instanceof Error ? (error.stack ?? error.message) : String(error);
123
+ }
package/dist/config.js ADDED
@@ -0,0 +1,33 @@
1
+ import { chmod, mkdir, readFile, rm, writeFile } from 'node:fs/promises';
2
+ import { homedir } from 'node:os';
3
+ import { join } from 'node:path';
4
+ export function configDir(env = process.env, platform = process.platform) {
5
+ if (env.SIFUR_CONFIG_DIR)
6
+ return env.SIFUR_CONFIG_DIR;
7
+ if (platform === 'win32')
8
+ return join(env.APPDATA ?? join(homedir(), 'AppData', 'Roaming'), 'sifur');
9
+ return join(env.XDG_CONFIG_HOME || join(homedir(), '.config'), 'sifur');
10
+ }
11
+ export function credentialsPath(env = process.env) {
12
+ return join(configDir(env), 'credentials.json');
13
+ }
14
+ export async function loadCredentials(env = process.env) {
15
+ try {
16
+ const parsed = JSON.parse(await readFile(credentialsPath(env), 'utf8'));
17
+ return parsed.api && parsed.token ? parsed : null;
18
+ }
19
+ catch {
20
+ return null;
21
+ }
22
+ }
23
+ export async function saveCredentials(credentials, env = process.env) {
24
+ const path = credentialsPath(env);
25
+ await mkdir(configDir(env), { recursive: true, mode: 0o700 });
26
+ await writeFile(path, `${JSON.stringify(credentials, null, 2)}\n`, { mode: 0o600 });
27
+ // writeFile's mode only applies when it creates the file; an older one keeps its own.
28
+ await chmod(path, 0o600);
29
+ return path;
30
+ }
31
+ export async function deleteCredentials(env = process.env) {
32
+ await rm(credentialsPath(env), { force: true });
33
+ }
package/dist/io.js ADDED
@@ -0,0 +1,96 @@
1
+ import { createInterface } from 'node:readline';
2
+ import { Writable } from 'node:stream';
3
+ import { CliError } from './api.js';
4
+ /**
5
+ * The terminal: one readline over stdin for every question, muted while a secret is typed. Lines are
6
+ * queued as they arrive, because piped input arrives all at once, before most questions are asked.
7
+ */
8
+ export function terminalIo() {
9
+ let muted = false;
10
+ let rl = null;
11
+ let ended = false;
12
+ const lines = [];
13
+ const waiting = [];
14
+ const output = new Writable({
15
+ write(chunk, encoding, callback) {
16
+ if (!muted)
17
+ process.stdout.write(chunk, encoding);
18
+ callback();
19
+ },
20
+ });
21
+ const inputEnded = () => new CliError('Input ended before every question was answered.');
22
+ function readline() {
23
+ if (!rl) {
24
+ rl = createInterface({ input: process.stdin, output, terminal: Boolean(process.stdin.isTTY) });
25
+ rl.on('line', (line) => {
26
+ const next = waiting.shift();
27
+ if (next)
28
+ next.resolve(line);
29
+ else
30
+ lines.push(line);
31
+ });
32
+ // In terminal mode readline swallows Ctrl-C; without this it would pause instead of quitting.
33
+ rl.on('SIGINT', () => {
34
+ rl?.close();
35
+ process.stdout.write('\n');
36
+ process.exit(130);
37
+ });
38
+ rl.on('close', () => {
39
+ ended = true;
40
+ for (const next of waiting.splice(0))
41
+ next.reject(inputEnded());
42
+ });
43
+ }
44
+ return rl;
45
+ }
46
+ function nextLine() {
47
+ readline();
48
+ const queued = lines.shift();
49
+ if (queued !== undefined)
50
+ return Promise.resolve(queued);
51
+ if (ended)
52
+ return Promise.reject(inputEnded());
53
+ return new Promise((resolve, reject) => waiting.push({ resolve, reject }));
54
+ }
55
+ function question(prompt) {
56
+ const reader = readline();
57
+ reader.setPrompt(prompt);
58
+ reader.prompt();
59
+ return nextLine();
60
+ }
61
+ return {
62
+ out: (line = '') => void process.stdout.write(`${line}\n`),
63
+ ask: (prompt) => question(prompt),
64
+ async askHidden(prompt) {
65
+ const reader = readline();
66
+ process.stdout.write(prompt);
67
+ reader.setPrompt('');
68
+ muted = true;
69
+ try {
70
+ return await nextLine();
71
+ }
72
+ finally {
73
+ muted = false;
74
+ process.stdout.write('\n');
75
+ }
76
+ },
77
+ close: () => rl?.close(),
78
+ };
79
+ }
80
+ export async function confirm(io, prompt) {
81
+ for (;;) {
82
+ const answer = (await io.ask(`${prompt} [y/n] `)).trim().toLowerCase();
83
+ if (['y', 'yes'].includes(answer))
84
+ return true;
85
+ if (['n', 'no'].includes(answer))
86
+ return false;
87
+ }
88
+ }
89
+ export async function askStatus(io, prompt) {
90
+ const choices = { p: 'passed', passed: 'passed', f: 'failed', failed: 'failed', b: 'blocked', blocked: 'blocked' };
91
+ for (;;) {
92
+ const answer = choices[(await io.ask(`${prompt} [p]assed / [f]ailed / [b]locked: `)).trim().toLowerCase()];
93
+ if (answer)
94
+ return answer;
95
+ }
96
+ }
package/dist/login.js ADDED
@@ -0,0 +1,112 @@
1
+ import { spawn } from 'node:child_process';
2
+ import { hostname, userInfo } from 'node:os';
3
+ import { Api, ApiError, CliError } from './api.js';
4
+ import { deleteCredentials, loadCredentials, saveCredentials } from './config.js';
5
+ const wait = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
6
+ export function openInBrowser(url) {
7
+ const [command, args] = process.platform === 'darwin'
8
+ ? ['open', [url]]
9
+ : process.platform === 'win32'
10
+ ? ['cmd', ['/c', 'start', '""', url]]
11
+ : ['xdg-open', [url]];
12
+ try {
13
+ const child = spawn(command, args, { stdio: 'ignore', detached: true });
14
+ child.on('error', () => { });
15
+ child.unref();
16
+ }
17
+ catch {
18
+ // The URL is printed too; a machine without a browser just doesn't open one.
19
+ }
20
+ }
21
+ export function defaultClientName() {
22
+ let user = '';
23
+ try {
24
+ user = userInfo().username;
25
+ }
26
+ catch {
27
+ user = '';
28
+ }
29
+ return user ? `${user}@${hostname()}` : hostname();
30
+ }
31
+ export function withCode(verificationUrl, userCode) {
32
+ try {
33
+ const url = new URL(verificationUrl);
34
+ if (!url.searchParams.has('code'))
35
+ url.searchParams.set('code', userCode);
36
+ return url.toString();
37
+ }
38
+ catch {
39
+ return verificationUrl;
40
+ }
41
+ }
42
+ export async function login(api, deps) {
43
+ const { io } = deps;
44
+ const sleep = deps.sleep ?? wait;
45
+ const now = deps.now ?? Date.now;
46
+ const start = await api.request('auth/device', {
47
+ method: 'POST',
48
+ body: { clientName: (deps.clientName ?? defaultClientName()).slice(0, 100) },
49
+ });
50
+ const url = withCode(start.verificationUrl, start.userCode);
51
+ io.out(`Your code: ${start.userCode}`);
52
+ io.out(`Approve it at ${url}`);
53
+ io.out('Waiting for approval…');
54
+ (deps.openBrowser ?? openInBrowser)(url);
55
+ let interval = Math.max(1, start.interval ?? 5) * 1000;
56
+ const deadline = now() + (start.expiresIn ?? 600) * 1000;
57
+ for (;;) {
58
+ await sleep(interval);
59
+ try {
60
+ const granted = await api.request('auth/device/token', {
61
+ method: 'POST',
62
+ body: { deviceCode: start.deviceCode },
63
+ });
64
+ let consoleOrigin = null;
65
+ try {
66
+ consoleOrigin = new URL(start.verificationUrl).origin;
67
+ }
68
+ catch {
69
+ consoleOrigin = null;
70
+ }
71
+ const credentials = { api: api.base, token: granted.token, console: consoleOrigin };
72
+ const path = await saveCredentials(credentials, deps.env);
73
+ const who = granted.user?.email ?? granted.user?.name;
74
+ io.out(`Signed in${who ? ` as ${who}` : ''}. Saved to ${path}.`);
75
+ return credentials;
76
+ }
77
+ catch (error) {
78
+ if (!(error instanceof ApiError))
79
+ throw error;
80
+ if (error.code === 'authorization_pending' || error.status === 428) {
81
+ if (now() >= deadline)
82
+ throw new CliError('The code expired before it was approved. Run `sifur login` again.');
83
+ continue;
84
+ }
85
+ if (error.status === 429 || error.code === 'slow_down') {
86
+ interval = Math.max(interval + 5000, (error.retryAfterSeconds ?? 0) * 1000);
87
+ continue;
88
+ }
89
+ if (error.code === 'access_denied' || error.status === 403)
90
+ throw new CliError('The sign-in was denied.');
91
+ if (error.code === 'expired_token' || error.status === 410) {
92
+ throw new CliError('The code expired before it was approved. Run `sifur login` again.');
93
+ }
94
+ throw error;
95
+ }
96
+ }
97
+ }
98
+ export async function logout(io, env = process.env) {
99
+ const credentials = await loadCredentials(env);
100
+ if (!credentials) {
101
+ io.out('Not signed in.');
102
+ return;
103
+ }
104
+ try {
105
+ await new Api(credentials.api, credentials.token).request('account/logout', { method: 'POST' });
106
+ }
107
+ catch {
108
+ // The session may already be revoked or Sifur unreachable; the local token goes either way.
109
+ }
110
+ await deleteCredentials(env);
111
+ io.out('Signed out.');
112
+ }
@@ -0,0 +1,35 @@
1
+ import { ApiError, CliError } from './api.js';
2
+ const UUID = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i;
3
+ const KEY = /^([A-Za-z][A-Za-z0-9]*)-(\d+)$/;
4
+ export function parseTicketRef(ref) {
5
+ if (UUID.test(ref))
6
+ return { uuid: ref.toLowerCase() };
7
+ const match = KEY.exec(ref);
8
+ if (!match)
9
+ throw new CliError(`"${ref}" is neither a ticket key (like AZA-12) nor a ticket uuid.`);
10
+ return { projectKey: match[1].toUpperCase(), number: Number(match[2]) };
11
+ }
12
+ /** Asks each of the person's organizations in turn; the first that has the ticket scopes every later call. */
13
+ export async function resolveTicket(api, ref) {
14
+ const parsed = parseTicketRef(ref);
15
+ const organizations = (await api.request('account/organizations')) ?? [];
16
+ if (organizations.length === 0)
17
+ throw new CliError('Your account is not in any organization.');
18
+ const path = 'uuid' in parsed
19
+ ? `tickets/${parsed.uuid}`
20
+ : `ticket-links/${encodeURIComponent(parsed.projectKey)}/${parsed.number}`;
21
+ for (const organization of organizations) {
22
+ try {
23
+ const ticket = await api.request(path, { organization: organization.uuid });
24
+ if (!ticket?.uuid)
25
+ throw new CliError(`Sifur found ${ref} but didn't say which ticket it is.`);
26
+ return { ticketUuid: ticket.uuid, organization };
27
+ }
28
+ catch (error) {
29
+ if (error instanceof ApiError && [403, 404].includes(error.status))
30
+ continue;
31
+ throw error;
32
+ }
33
+ }
34
+ throw new CliError(`No ticket ${ref} in your organizations (${organizations.map((o) => o.name).join(', ')}).`);
35
+ }
@@ -0,0 +1,32 @@
1
+ /** `screenshots` maps a local screenshot path to the uuid its upload returned. */
2
+ export function buildResultsPayload(stateVersion, baseUrl, results, screenshots) {
3
+ return {
4
+ stateVersion,
5
+ via: 'cli',
6
+ baseUrl,
7
+ results: results.map((result) => ({
8
+ caseUuid: result.caseUuid,
9
+ status: result.status,
10
+ notes: result.notes,
11
+ steps: result.steps.map((step) => ({
12
+ n: step.n,
13
+ caption: step.caption,
14
+ ok: step.ok,
15
+ error: step.error,
16
+ screenshotUuid: step.screenshot ? (screenshots.get(step.screenshot) ?? null) : null,
17
+ })),
18
+ ...(result.output !== undefined ? { output: result.output } : {}),
19
+ })),
20
+ };
21
+ }
22
+ /** Failed outranks blocked: a case that both failed a check and couldn't finish another still failed. */
23
+ export function combine(statuses) {
24
+ if (statuses.includes('failed'))
25
+ return 'failed';
26
+ if (statuses.includes('blocked'))
27
+ return 'blocked';
28
+ return 'passed';
29
+ }
30
+ export function outcome(results) {
31
+ return combine(results.map((result) => result.status));
32
+ }
package/dist/runner.js ADDED
@@ -0,0 +1,130 @@
1
+ import { mkdir } from 'node:fs/promises';
2
+ import { join } from 'node:path';
3
+ import { redact, withSecrets } from './secrets.js';
4
+ const ASSERTIONS = new Set(['expectText', 'expectVisible', 'expectUrl']);
5
+ const TIMEOUT = 15_000;
6
+ // An assertion after an action waits for the page to settle, as Playwright's own do: a
7
+ // response that lands half a second later still counts.
8
+ export async function eventually(check, timeoutMs = 10_000) {
9
+ const deadline = Date.now() + timeoutMs;
10
+ for (;;) {
11
+ try {
12
+ return await check();
13
+ }
14
+ catch (error) {
15
+ if (Date.now() >= deadline)
16
+ throw error;
17
+ await new Promise((resolve) => setTimeout(resolve, 250));
18
+ }
19
+ }
20
+ }
21
+ function first(page, selector) {
22
+ if (!selector)
23
+ throw new Error('the step has no selector');
24
+ return page.locator(selector).first();
25
+ }
26
+ async function perform(page, step, baseUrl, secrets, settleMs) {
27
+ const url = step.url ? new URL(step.url, baseUrl).toString() : null;
28
+ switch (step.action) {
29
+ case 'goto':
30
+ if (!url)
31
+ throw new Error('the step has no url');
32
+ await page.goto(url, { waitUntil: 'networkidle' });
33
+ break;
34
+ case 'click':
35
+ await first(page, step.selector).click();
36
+ break;
37
+ case 'fill':
38
+ await first(page, step.selector).fill(withSecrets(String(step.value ?? ''), secrets));
39
+ break;
40
+ case 'press':
41
+ await first(page, step.selector).press(String(step.value));
42
+ break;
43
+ case 'select':
44
+ await first(page, step.selector).selectOption(String(step.value));
45
+ break;
46
+ case 'wait':
47
+ await page.waitForTimeout(Math.min(Number(step.value ?? 500), 10_000));
48
+ break;
49
+ case 'expectText': {
50
+ const target = step.selector ? page.locator(step.selector).first() : page.locator('body');
51
+ await target.waitFor();
52
+ await eventually(async () => {
53
+ const text = await target.innerText();
54
+ if (!text.includes(String(step.text)))
55
+ throw new Error(`expected "${step.text}" in ${step.selector ?? 'the page'}`);
56
+ }, settleMs);
57
+ break;
58
+ }
59
+ case 'expectVisible':
60
+ await first(page, step.selector).waitFor({ state: 'visible' });
61
+ break;
62
+ case 'expectUrl':
63
+ await eventually(async () => {
64
+ if (!page.url().includes(String(step.text))) {
65
+ throw new Error(`expected the URL to contain "${step.text}", got ${page.url()}`);
66
+ }
67
+ }, settleMs);
68
+ break;
69
+ case 'judge':
70
+ break;
71
+ default:
72
+ throw new Error(`unknown action ${step.action}`);
73
+ }
74
+ }
75
+ /** One case's UI script, as the sandbox's runner plays it: a screenshot after every step, stopping at the first failure. */
76
+ export async function runUiCase(browser, testCase, options) {
77
+ const secrets = options.secrets ?? {};
78
+ const dir = join(options.outDir, testCase.id);
79
+ await mkdir(dir, { recursive: true });
80
+ // Light, always: an app that follows the system theme is judged in one, the same every run.
81
+ const context = await browser.newContext({ viewport: { width: 1440, height: 900 }, colorScheme: 'light' });
82
+ const page = await context.newPage();
83
+ page.setDefaultTimeout(options.timeoutMs ?? TIMEOUT);
84
+ const steps = [];
85
+ const notes = [];
86
+ let status = 'passed';
87
+ try {
88
+ for (const [index, step] of (testCase.uiScript ?? []).entries()) {
89
+ const n = index + 1;
90
+ const caption = step.caption ?? step.action;
91
+ let ok = true;
92
+ let error = null;
93
+ try {
94
+ await perform(page, step, options.baseUrl, secrets, Math.min(10_000, options.timeoutMs ?? 10_000));
95
+ }
96
+ catch (caught) {
97
+ ok = false;
98
+ error = redact(String(caught?.message ?? caught).split('\n')[0] ?? '', secrets);
99
+ }
100
+ let screenshot = join(dir, `${String(n).padStart(2, '0')}.png`);
101
+ await page.screenshot({ path: screenshot, fullPage: true }).catch(() => {
102
+ screenshot = null;
103
+ });
104
+ let judged = false;
105
+ if (ok && step.action === 'judge') {
106
+ const answer = await options.judge(step, screenshot, testCase);
107
+ judged = true;
108
+ if (answer.note)
109
+ notes.push(`Step ${n} (${caption}): ${answer.note}`);
110
+ if (!answer.ok) {
111
+ ok = false;
112
+ error = answer.note ? `The engineer judged it wrong: ${answer.note}` : 'The engineer judged it wrong.';
113
+ }
114
+ }
115
+ steps.push({ n, action: step.action, caption, ok, error, screenshot });
116
+ if (!ok) {
117
+ // A failed assertion (or a judged screenshot that's wrong) is a failed case; anything
118
+ // failing before it means the case couldn't be run as written, so it's blocked.
119
+ status = ASSERTIONS.has(step.action) || judged ? 'failed' : 'blocked';
120
+ if (!judged)
121
+ notes.push(`Step ${n} (${caption}): ${error}`);
122
+ break;
123
+ }
124
+ }
125
+ }
126
+ finally {
127
+ await context.close().catch(() => { });
128
+ }
129
+ return { status, steps, notes };
130
+ }
@@ -0,0 +1,32 @@
1
+ const PLACEHOLDER = /\{\{([A-Z][A-Z0-9_]*)\}\}/g;
2
+ export function secretNames(cases) {
3
+ const names = new Set();
4
+ for (const testCase of cases) {
5
+ for (const step of testCase.uiScript ?? []) {
6
+ if (step.action !== 'fill')
7
+ continue;
8
+ for (const match of String(step.value ?? '').matchAll(PLACEHOLDER))
9
+ names.add(match[1]);
10
+ }
11
+ }
12
+ return [...names].sort();
13
+ }
14
+ /** From `SIFUR_SECRET_<NAME>`, or typed without echo. Kept in memory only: never written or sent to Sifur. */
15
+ export async function collectSecrets(names, io, env = process.env) {
16
+ const secrets = {};
17
+ for (const name of names) {
18
+ const fromEnv = env[`SIFUR_SECRET_${name}`];
19
+ secrets[name] = fromEnv !== undefined ? fromEnv : await io.askHidden(`${name} (or set SIFUR_SECRET_${name}): `);
20
+ }
21
+ return secrets;
22
+ }
23
+ export function withSecrets(value, secrets) {
24
+ return value.replace(PLACEHOLDER, (match, name) => secrets[name] ?? match);
25
+ }
26
+ export function redact(text, secrets) {
27
+ let redacted = text;
28
+ for (const value of Object.values(secrets))
29
+ if (value)
30
+ redacted = redacted.replaceAll(value, '[secret]');
31
+ return redacted;
32
+ }
package/dist/store.js ADDED
@@ -0,0 +1,70 @@
1
+ import { mkdir, readdir, readFile, writeFile } from 'node:fs/promises';
2
+ import { isAbsolute, join, relative, resolve } from 'node:path';
3
+ import { CliError } from './api.js';
4
+ export const STORE = '.sifur';
5
+ export function ticketDir(root, key) {
6
+ return join(root, STORE, key.replace(/[^A-Za-z0-9_-]/g, '_'));
7
+ }
8
+ export async function saveCases(root, saved) {
9
+ const dir = ticketDir(root, saved.acceptance.ticket.key);
10
+ await mkdir(dir, { recursive: true });
11
+ await writeFile(join(dir, 'cases.json'), `${JSON.stringify(saved, null, 2)}\n`);
12
+ return dir;
13
+ }
14
+ function mapScreenshots(results, map) {
15
+ return {
16
+ ...results,
17
+ results: results.results.map((result) => ({
18
+ ...result,
19
+ steps: result.steps.map((step) => ({ ...step, screenshot: step.screenshot ? map(step.screenshot) : null })),
20
+ })),
21
+ };
22
+ }
23
+ /** Screenshot paths are kept relative to the ticket's folder, so the folder can move between --run and --upload. */
24
+ export async function saveResults(dir, results) {
25
+ const path = join(dir, 'results.json');
26
+ const portable = mapScreenshots(results, (screenshot) => (isAbsolute(screenshot) ? relative(dir, screenshot) : screenshot));
27
+ await writeFile(path, `${JSON.stringify(portable, null, 2)}\n`);
28
+ return path;
29
+ }
30
+ async function readJson(path) {
31
+ try {
32
+ return JSON.parse(await readFile(path, 'utf8'));
33
+ }
34
+ catch (error) {
35
+ if (error.code === 'ENOENT')
36
+ return null;
37
+ throw new CliError(`${path} isn't readable JSON: ${error.message}`);
38
+ }
39
+ }
40
+ /** A key names its folder; a uuid is looked for among the downloaded tickets. */
41
+ export async function findTicketDir(root, ref) {
42
+ const direct = ticketDir(root, ref.toUpperCase());
43
+ if (await readJson(join(direct, 'cases.json')))
44
+ return direct;
45
+ let entries;
46
+ try {
47
+ entries = await readdir(join(root, STORE));
48
+ }
49
+ catch {
50
+ return null;
51
+ }
52
+ for (const entry of entries) {
53
+ const saved = await readJson(join(root, STORE, entry, 'cases.json'));
54
+ if (saved?.acceptance.ticket.uuid === ref.toLowerCase())
55
+ return join(root, STORE, entry);
56
+ }
57
+ return null;
58
+ }
59
+ export async function loadCases(dir) {
60
+ const saved = await readJson(join(dir, 'cases.json'));
61
+ if (!saved)
62
+ throw new CliError(`No downloaded cases in ${dir}. Run with --download first.`);
63
+ return saved;
64
+ }
65
+ export async function loadResults(dir) {
66
+ const saved = await readJson(join(dir, 'results.json'));
67
+ if (!saved)
68
+ throw new CliError(`No results in ${dir}. Run with --run first.`);
69
+ return mapScreenshots(saved, (screenshot) => resolve(dir, screenshot));
70
+ }
package/dist/types.js ADDED
@@ -0,0 +1 @@
1
+ export {};
package/package.json CHANGED
@@ -1,6 +1,40 @@
1
1
  {
2
2
  "name": "@sifurai/cli",
3
- "version": "0.0.0-stage",
4
- "stub": true,
5
- "description": "Temporary package placeholder for staged publishing"
6
- }
3
+ "version": "0.1.0",
4
+ "description": "Run a Sifur ticket's acceptance cases against your own test server and upload the results.",
5
+ "license": "UNLICENSED",
6
+ "homepage": "https://sifur.ai",
7
+ "keywords": [
8
+ "sifur",
9
+ "acceptance-testing",
10
+ "cli"
11
+ ],
12
+ "type": "module",
13
+ "bin": {
14
+ "sifur": "dist/bin.js"
15
+ },
16
+ "files": [
17
+ "dist",
18
+ "README.md"
19
+ ],
20
+ "engines": {
21
+ "node": ">=20"
22
+ },
23
+ "scripts": {
24
+ "build": "tsc -p tsconfig.build.json",
25
+ "typecheck": "tsc --noEmit",
26
+ "test": "vitest run",
27
+ "prepack": "npm run build"
28
+ },
29
+ "dependencies": {
30
+ "playwright-core": "^1.64.1"
31
+ },
32
+ "devDependencies": {
33
+ "@types/node": "^20.19.43",
34
+ "typescript": "^5.9.3",
35
+ "vitest": "^3.2.7"
36
+ },
37
+ "publishConfig": {
38
+ "access": "public"
39
+ }
40
+ }