@datadisco/qa 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +99 -0
- package/dist/cli.js +458 -0
- package/package.json +57 -0
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 DataDisco, Inc.
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
ADDED
|
@@ -0,0 +1,99 @@
|
|
|
1
|
+
# @datadisco/qa
|
|
2
|
+
|
|
3
|
+
CI companion for [DataDisco QA](https://datadisco.com) — PR-triggered persona
|
|
4
|
+
simulation ("confirmation testing for agentic teams").
|
|
5
|
+
|
|
6
|
+
> **Status: pre-release.** The commands below are wired to the DataDisco QA
|
|
7
|
+
> API but the package is not published to npm yet. Pin to the git repo, or
|
|
8
|
+
> wait for the first tagged release.
|
|
9
|
+
|
|
10
|
+
## Why this exists
|
|
11
|
+
|
|
12
|
+
The DataDisco QA GitHub App needs **zero** customer-side install for the
|
|
13
|
+
common path: it resolves each PR's preview deploy from GitHub
|
|
14
|
+
`deployment_status` events (or a configured URL pattern) and posts a
|
|
15
|
+
"DataDisco QA — persona simulation" check run. This CLI covers the cases the
|
|
16
|
+
App can't see:
|
|
17
|
+
|
|
18
|
+
| Command | For teams that… |
|
|
19
|
+
| ----------------------------------------- | ------------------------------------------------------------------------------------------------------------ |
|
|
20
|
+
| `datadisco-qa report-preview --url <url>` | deploy previews without emitting GitHub Deployment events — report the URL from CI so the pending run starts |
|
|
21
|
+
| `datadisco-qa wait --pr <n>` | want a pipeline step to block on the QA verdict instead of relying on branch protection |
|
|
22
|
+
| `datadisco-qa tunnel --port <port>` | want to run personas against a build served on their own machine |
|
|
23
|
+
|
|
24
|
+
## Authentication
|
|
25
|
+
|
|
26
|
+
Every command needs a **workspace API token** (create one under
|
|
27
|
+
_Workspace settings → Integrations → API tokens_). Pass it with
|
|
28
|
+
`--api-token` or the `DATADISCO_API_TOKEN` env var. The API base URL defaults
|
|
29
|
+
to the hosted app and is overridable with `--api-url` / `DATADISCO_API_URL`.
|
|
30
|
+
|
|
31
|
+
```sh
|
|
32
|
+
export DATADISCO_API_TOKEN=ddqa_xxx
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
## Commands
|
|
36
|
+
|
|
37
|
+
### `report-preview`
|
|
38
|
+
|
|
39
|
+
Report the deployed preview URL for a PR head SHA so its pending QA run leaves
|
|
40
|
+
`PENDING_PREVIEW` and starts. In GitHub Actions, `--repo`, `--pr`, and `--sha`
|
|
41
|
+
are auto-detected from the `pull_request` event.
|
|
42
|
+
|
|
43
|
+
```sh
|
|
44
|
+
datadisco-qa report-preview --url https://pr-482.preview.acme.dev
|
|
45
|
+
# explicit outside Actions:
|
|
46
|
+
datadisco-qa report-preview \
|
|
47
|
+
--url https://pr-482.preview.acme.dev \
|
|
48
|
+
--repo acme/site --pr 482 --sha "$GIT_SHA"
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
### `wait`
|
|
52
|
+
|
|
53
|
+
Block until the PR's QA run reaches a verdict, printing each phase transition.
|
|
54
|
+
Polls every 15s; `--timeout` (minutes, default 30) caps the wait.
|
|
55
|
+
|
|
56
|
+
```sh
|
|
57
|
+
datadisco-qa wait --pr 482
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
**Exit codes** (so pipelines can branch on the outcome):
|
|
61
|
+
|
|
62
|
+
| Code | Meaning |
|
|
63
|
+
| ---- | ---------------------------------------------- |
|
|
64
|
+
| `0` | Passed, or passed with warnings (non-blocking) |
|
|
65
|
+
| `1` | Blocked — one or more blocker findings |
|
|
66
|
+
| `3` | Run failed before reaching a verdict |
|
|
67
|
+
| `4` | No preview deploy arrived; run timed out |
|
|
68
|
+
| `5` | `wait` gave up after `--timeout` |
|
|
69
|
+
|
|
70
|
+
Superseded / cancelled runs exit `0` — a newer commit's run governs the gate.
|
|
71
|
+
|
|
72
|
+
### `tunnel`
|
|
73
|
+
|
|
74
|
+
Expose a locally served build over an HTTPS tunnel and start a QA run against
|
|
75
|
+
it — no preview deploy required. `--repo` and `--sha` default to the local git
|
|
76
|
+
origin remote and `HEAD`; `--pr` is required. Bring your own tunnel with
|
|
77
|
+
`--url` instead of `--port`. Blocks on the verdict unless `--no-wait` is given.
|
|
78
|
+
|
|
79
|
+
```sh
|
|
80
|
+
datadisco-qa tunnel --port 5173 --pr 482
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
## Development
|
|
84
|
+
|
|
85
|
+
```sh
|
|
86
|
+
npm install
|
|
87
|
+
npm test
|
|
88
|
+
npm run build # tsup → dist/cli.js
|
|
89
|
+
node dist/cli.js --help
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
Same toolchain as [datadisco-mcp](https://github.com/Data-Disco-Inc/datadisco-mcp):
|
|
93
|
+
tsup, vitest, oxlint/oxfmt, cac.
|
|
94
|
+
|
|
95
|
+
## Publishing (later)
|
|
96
|
+
|
|
97
|
+
1. Bump the version and tag.
|
|
98
|
+
2. `npm publish` (publishConfig is already `access: public`).
|
|
99
|
+
3. Ship the companion GitHub Action (`Data-Disco-Inc/qa-action`) that wraps this CLI.
|
package/dist/cli.js
ADDED
|
@@ -0,0 +1,458 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
|
|
3
|
+
// src/cli.ts
|
|
4
|
+
import { cac } from "cac";
|
|
5
|
+
import pc2 from "picocolors";
|
|
6
|
+
|
|
7
|
+
// src/config.ts
|
|
8
|
+
var DEFAULT_API_URL = "https://www.datadisco.com";
|
|
9
|
+
var API_TOKEN_ENV_KEY = "DATADISCO_API_TOKEN";
|
|
10
|
+
var ConfigError = class extends Error {
|
|
11
|
+
};
|
|
12
|
+
function resolveConfig(input = {}) {
|
|
13
|
+
const env = input.env ?? process.env;
|
|
14
|
+
const apiUrl = input.apiUrl || env.DATADISCO_API_URL || DEFAULT_API_URL;
|
|
15
|
+
const apiToken = input.apiToken || env[API_TOKEN_ENV_KEY];
|
|
16
|
+
if (!apiToken) {
|
|
17
|
+
throw new ConfigError(`Missing API token: pass --api-token or set ${API_TOKEN_ENV_KEY}.`);
|
|
18
|
+
}
|
|
19
|
+
if (!apiUrl.startsWith("https://")) {
|
|
20
|
+
throw new ConfigError(`API url must be https, got "${apiUrl}".`);
|
|
21
|
+
}
|
|
22
|
+
return { apiUrl, apiToken };
|
|
23
|
+
}
|
|
24
|
+
function assertValidPreviewUrl(url) {
|
|
25
|
+
let parsed;
|
|
26
|
+
try {
|
|
27
|
+
parsed = new URL(url);
|
|
28
|
+
} catch {
|
|
29
|
+
throw new ConfigError(`Preview url is not a valid URL: "${url}".`);
|
|
30
|
+
}
|
|
31
|
+
if (parsed.protocol !== "https:") {
|
|
32
|
+
throw new ConfigError(`Preview url must be https, got "${url}".`);
|
|
33
|
+
}
|
|
34
|
+
}
|
|
35
|
+
|
|
36
|
+
// src/context.ts
|
|
37
|
+
import { readFileSync } from "fs";
|
|
38
|
+
var REPO_PATTERN = /^[^/\s]+\/[^/\s]+$/;
|
|
39
|
+
var PR_REF_PATTERN = /^refs\/pull\/(\d+)\//;
|
|
40
|
+
function resolvePrContext(input = {}) {
|
|
41
|
+
const env = input.env ?? process.env;
|
|
42
|
+
const payload = loadEventPayload(input, env);
|
|
43
|
+
return {
|
|
44
|
+
repo: resolveRepo(input, env),
|
|
45
|
+
prNumber: resolvePrNumber(input, env, payload)
|
|
46
|
+
};
|
|
47
|
+
}
|
|
48
|
+
function resolveRunContext(input = {}) {
|
|
49
|
+
const env = input.env ?? process.env;
|
|
50
|
+
const payload = loadEventPayload(input, env);
|
|
51
|
+
return {
|
|
52
|
+
repo: resolveRepo(input, env),
|
|
53
|
+
prNumber: resolvePrNumber(input, env, payload),
|
|
54
|
+
headSha: resolveHeadSha(input, env, payload)
|
|
55
|
+
};
|
|
56
|
+
}
|
|
57
|
+
function resolveRepo(input, env = {}) {
|
|
58
|
+
const repo = input.repo ?? env.GITHUB_REPOSITORY;
|
|
59
|
+
if (!repo) {
|
|
60
|
+
throw new ConfigError(
|
|
61
|
+
"Repository not set: pass --repo owner/name (auto-detected from $GITHUB_REPOSITORY in GitHub Actions)."
|
|
62
|
+
);
|
|
63
|
+
}
|
|
64
|
+
if (!REPO_PATTERN.test(repo)) {
|
|
65
|
+
throw new ConfigError(`--repo must look like "owner/name", got "${repo}".`);
|
|
66
|
+
}
|
|
67
|
+
return repo;
|
|
68
|
+
}
|
|
69
|
+
function resolvePrNumber(input, env = {}, payload) {
|
|
70
|
+
const candidate = input.pr ?? parsePrFromRef(env.GITHUB_REF) ?? payload.prNumber;
|
|
71
|
+
const prNumber = Number(candidate);
|
|
72
|
+
if (candidate === void 0 || !Number.isInteger(prNumber) || prNumber <= 0) {
|
|
73
|
+
throw new ConfigError(
|
|
74
|
+
"Pull request number not set: pass --pr <number> (auto-detected from the pull_request event in GitHub Actions)."
|
|
75
|
+
);
|
|
76
|
+
}
|
|
77
|
+
return prNumber;
|
|
78
|
+
}
|
|
79
|
+
function resolveHeadSha(input, env = {}, payload) {
|
|
80
|
+
const headSha = input.sha ?? payload.headSha ?? env.GITHUB_SHA;
|
|
81
|
+
if (!headSha) {
|
|
82
|
+
throw new ConfigError(
|
|
83
|
+
"Head SHA not set: pass --sha <sha> (the commit the preview was built from)."
|
|
84
|
+
);
|
|
85
|
+
}
|
|
86
|
+
return headSha;
|
|
87
|
+
}
|
|
88
|
+
function parsePrFromRef(ref) {
|
|
89
|
+
const match = ref?.match(PR_REF_PATTERN);
|
|
90
|
+
return match ? match[1] : void 0;
|
|
91
|
+
}
|
|
92
|
+
function loadEventPayload(input, env = {}) {
|
|
93
|
+
const read = input.readEventPayload ?? (() => readEventFile(env.GITHUB_EVENT_PATH));
|
|
94
|
+
const raw = safeRead(read);
|
|
95
|
+
if (!isRecord(raw)) return {};
|
|
96
|
+
const pullRequest = isRecord(raw.pull_request) ? raw.pull_request : void 0;
|
|
97
|
+
const head = isRecord(pullRequest?.head) ? pullRequest.head : void 0;
|
|
98
|
+
return {
|
|
99
|
+
prNumber: numberOrUndefined(pullRequest?.number ?? raw.number),
|
|
100
|
+
headSha: stringOrUndefined(head?.sha)
|
|
101
|
+
};
|
|
102
|
+
}
|
|
103
|
+
function readEventFile(path) {
|
|
104
|
+
if (!path) return void 0;
|
|
105
|
+
return JSON.parse(readFileSync(path, "utf8"));
|
|
106
|
+
}
|
|
107
|
+
function safeRead(read) {
|
|
108
|
+
try {
|
|
109
|
+
return read();
|
|
110
|
+
} catch {
|
|
111
|
+
return void 0;
|
|
112
|
+
}
|
|
113
|
+
}
|
|
114
|
+
function isRecord(value) {
|
|
115
|
+
return typeof value === "object" && value !== null;
|
|
116
|
+
}
|
|
117
|
+
function numberOrUndefined(value) {
|
|
118
|
+
return typeof value === "number" ? value : void 0;
|
|
119
|
+
}
|
|
120
|
+
function stringOrUndefined(value) {
|
|
121
|
+
return typeof value === "string" ? value : void 0;
|
|
122
|
+
}
|
|
123
|
+
|
|
124
|
+
// src/api.ts
|
|
125
|
+
var REPORT_PREVIEW_PATH = "/api/qa/report-preview";
|
|
126
|
+
var RUN_STATUS_PATH = "/api/qa/run-status";
|
|
127
|
+
function createApiClient(config, fetchImpl = fetch) {
|
|
128
|
+
return {
|
|
129
|
+
reportPreview: (params) => reportPreview(config, params, fetchImpl),
|
|
130
|
+
getRunStatus: (params) => getRunStatus(config, params, fetchImpl)
|
|
131
|
+
};
|
|
132
|
+
}
|
|
133
|
+
async function reportPreview(config, params, fetchImpl) {
|
|
134
|
+
const response = await fetchImpl(joinUrl(config.apiUrl, REPORT_PREVIEW_PATH), {
|
|
135
|
+
method: "POST",
|
|
136
|
+
headers: {
|
|
137
|
+
authorization: `Bearer ${config.apiToken}`,
|
|
138
|
+
"content-type": "application/json"
|
|
139
|
+
},
|
|
140
|
+
body: JSON.stringify({
|
|
141
|
+
repo: params.repo,
|
|
142
|
+
prNumber: params.prNumber,
|
|
143
|
+
headSha: params.headSha,
|
|
144
|
+
url: params.url
|
|
145
|
+
})
|
|
146
|
+
});
|
|
147
|
+
const payload = await readJson(response);
|
|
148
|
+
if (response.ok && isString(payload.runId)) {
|
|
149
|
+
return { ok: true, runId: payload.runId };
|
|
150
|
+
}
|
|
151
|
+
return { ok: false, status: response.status, error: errorMessage(payload) };
|
|
152
|
+
}
|
|
153
|
+
async function getRunStatus(config, params, fetchImpl) {
|
|
154
|
+
const url = new URL(joinUrl(config.apiUrl, RUN_STATUS_PATH));
|
|
155
|
+
url.searchParams.set("repo", params.repo);
|
|
156
|
+
url.searchParams.set("prNumber", String(params.prNumber));
|
|
157
|
+
const response = await fetchImpl(url.toString(), {
|
|
158
|
+
headers: { authorization: `Bearer ${config.apiToken}` }
|
|
159
|
+
});
|
|
160
|
+
const payload = await readJson(response);
|
|
161
|
+
if (response.ok && isRunStatus(payload.run)) {
|
|
162
|
+
return { ok: true, run: payload.run };
|
|
163
|
+
}
|
|
164
|
+
return { ok: false, status: response.status, error: errorMessage(payload) };
|
|
165
|
+
}
|
|
166
|
+
function joinUrl(base, path) {
|
|
167
|
+
return `${base.replace(/\/+$/, "")}${path}`;
|
|
168
|
+
}
|
|
169
|
+
async function readJson(response) {
|
|
170
|
+
try {
|
|
171
|
+
const parsed = await response.json();
|
|
172
|
+
return isRecord2(parsed) ? parsed : {};
|
|
173
|
+
} catch {
|
|
174
|
+
return {};
|
|
175
|
+
}
|
|
176
|
+
}
|
|
177
|
+
function errorMessage(payload) {
|
|
178
|
+
return isString(payload.error) ? payload.error : "Request failed.";
|
|
179
|
+
}
|
|
180
|
+
function isRecord2(value) {
|
|
181
|
+
return typeof value === "object" && value !== null;
|
|
182
|
+
}
|
|
183
|
+
function isString(value) {
|
|
184
|
+
return typeof value === "string";
|
|
185
|
+
}
|
|
186
|
+
function isRunStatus(value) {
|
|
187
|
+
return isRecord2(value) && isString(value.id) && isString(value.status) && typeof value.blockerCount === "number";
|
|
188
|
+
}
|
|
189
|
+
|
|
190
|
+
// src/logger.ts
|
|
191
|
+
import pc from "picocolors";
|
|
192
|
+
var consoleLogger = {
|
|
193
|
+
info: (message) => console.error(message),
|
|
194
|
+
success: (message) => console.error(pc.green(message)),
|
|
195
|
+
warn: (message) => console.error(pc.yellow(message)),
|
|
196
|
+
error: (message) => console.error(pc.red(message))
|
|
197
|
+
};
|
|
198
|
+
var LEVEL_LOGGERS = {
|
|
199
|
+
success: (logger, message) => logger.success(message),
|
|
200
|
+
warn: (logger, message) => logger.warn(message),
|
|
201
|
+
error: (logger, message) => logger.error(message)
|
|
202
|
+
};
|
|
203
|
+
function logAtLevel(logger, level, message) {
|
|
204
|
+
LEVEL_LOGGERS[level](logger, message);
|
|
205
|
+
}
|
|
206
|
+
|
|
207
|
+
// src/commands/deps.ts
|
|
208
|
+
var defaultDeps = {
|
|
209
|
+
createApi: (config) => createApiClient(config),
|
|
210
|
+
logger: consoleLogger,
|
|
211
|
+
sleep: (ms) => new Promise((resolve) => setTimeout(resolve, ms)),
|
|
212
|
+
now: () => Date.now()
|
|
213
|
+
};
|
|
214
|
+
|
|
215
|
+
// src/commands/report-preview.ts
|
|
216
|
+
async function runReportPreview(options, deps = defaultDeps) {
|
|
217
|
+
if (!options.url) {
|
|
218
|
+
deps.logger.error("--url is required (the deployed preview URL).");
|
|
219
|
+
return 1;
|
|
220
|
+
}
|
|
221
|
+
assertValidPreviewUrl(options.url);
|
|
222
|
+
const config = resolveConfig(options);
|
|
223
|
+
const context = resolveRunContext(options);
|
|
224
|
+
const api = deps.createApi(config);
|
|
225
|
+
const outcome = await api.reportPreview({ ...context, url: options.url });
|
|
226
|
+
if (!outcome.ok) {
|
|
227
|
+
deps.logger.error(`Could not report preview URL: ${outcome.error}`);
|
|
228
|
+
return 1;
|
|
229
|
+
}
|
|
230
|
+
deps.logger.success(
|
|
231
|
+
`Preview reported for ${context.repo}#${context.prNumber} \u2014 QA run ${outcome.runId} queued.`
|
|
232
|
+
);
|
|
233
|
+
return 0;
|
|
234
|
+
}
|
|
235
|
+
|
|
236
|
+
// src/git.ts
|
|
237
|
+
import { execFileSync } from "child_process";
|
|
238
|
+
var defaultRunner = (args) => execFileSync("git", args, { encoding: "utf8" }).trim();
|
|
239
|
+
function detectRepoFullName(run2 = defaultRunner) {
|
|
240
|
+
const remote = tryGit(run2, ["remote", "get-url", "origin"]);
|
|
241
|
+
return remote ? parseRepoFromRemote(remote) : void 0;
|
|
242
|
+
}
|
|
243
|
+
function detectHeadSha(run2 = defaultRunner) {
|
|
244
|
+
return tryGit(run2, ["rev-parse", "HEAD"]);
|
|
245
|
+
}
|
|
246
|
+
function parseRepoFromRemote(remote) {
|
|
247
|
+
const match = remote.match(/github\.com[:/]([^/\s]+\/[^/\s]+?)(?:\.git)?$/);
|
|
248
|
+
return match ? match[1] : void 0;
|
|
249
|
+
}
|
|
250
|
+
function tryGit(run2, args) {
|
|
251
|
+
try {
|
|
252
|
+
return run2(args) || void 0;
|
|
253
|
+
} catch {
|
|
254
|
+
return void 0;
|
|
255
|
+
}
|
|
256
|
+
}
|
|
257
|
+
|
|
258
|
+
// src/tunnel-provider.ts
|
|
259
|
+
import localtunnel from "localtunnel";
|
|
260
|
+
async function openTunnel(port) {
|
|
261
|
+
const tunnel = await localtunnel({ port });
|
|
262
|
+
return { url: tunnel.url, close: () => tunnel.close() };
|
|
263
|
+
}
|
|
264
|
+
|
|
265
|
+
// src/verdict.ts
|
|
266
|
+
var EXIT_CODE = {
|
|
267
|
+
PASSED: 0,
|
|
268
|
+
BLOCKED: 1,
|
|
269
|
+
RUN_FAILED: 3,
|
|
270
|
+
TIMED_OUT: 4,
|
|
271
|
+
WAIT_TIMEOUT: 5
|
|
272
|
+
};
|
|
273
|
+
var TERMINAL_STATUSES = /* @__PURE__ */ new Set(["COMPLETED", "FAILED", "TIMED_OUT", "SUPERSEDED", "CANCELLED"]);
|
|
274
|
+
function isTerminalStatus(status) {
|
|
275
|
+
return TERMINAL_STATUSES.has(status);
|
|
276
|
+
}
|
|
277
|
+
function resolveVerdictOutcome(run2) {
|
|
278
|
+
switch (run2.status) {
|
|
279
|
+
case "COMPLETED":
|
|
280
|
+
return completedOutcome(run2);
|
|
281
|
+
case "FAILED":
|
|
282
|
+
return {
|
|
283
|
+
exitCode: EXIT_CODE.RUN_FAILED,
|
|
284
|
+
level: "error",
|
|
285
|
+
headline: run2.statusMessage ?? "QA run failed before reaching a verdict."
|
|
286
|
+
};
|
|
287
|
+
case "TIMED_OUT":
|
|
288
|
+
return {
|
|
289
|
+
exitCode: EXIT_CODE.TIMED_OUT,
|
|
290
|
+
level: "error",
|
|
291
|
+
headline: run2.statusMessage ?? "No preview deploy arrived before the run timed out."
|
|
292
|
+
};
|
|
293
|
+
case "SUPERSEDED":
|
|
294
|
+
return {
|
|
295
|
+
exitCode: EXIT_CODE.PASSED,
|
|
296
|
+
level: "warn",
|
|
297
|
+
headline: "A newer commit superseded this run \u2014 its verdict governs."
|
|
298
|
+
};
|
|
299
|
+
case "CANCELLED":
|
|
300
|
+
return {
|
|
301
|
+
exitCode: EXIT_CODE.PASSED,
|
|
302
|
+
level: "warn",
|
|
303
|
+
headline: run2.statusMessage ?? "QA run was cancelled."
|
|
304
|
+
};
|
|
305
|
+
default:
|
|
306
|
+
return {
|
|
307
|
+
exitCode: EXIT_CODE.RUN_FAILED,
|
|
308
|
+
level: "error",
|
|
309
|
+
headline: `Unexpected terminal status "${run2.status}".`
|
|
310
|
+
};
|
|
311
|
+
}
|
|
312
|
+
}
|
|
313
|
+
function completedOutcome(run2) {
|
|
314
|
+
const counts = `${run2.blockerCount} blocker(s), ${run2.warningCount} warning(s), ${run2.passedCount} passed`;
|
|
315
|
+
if (run2.verdict === "BLOCKED") {
|
|
316
|
+
return {
|
|
317
|
+
exitCode: EXIT_CODE.BLOCKED,
|
|
318
|
+
level: "error",
|
|
319
|
+
headline: `QA blocked the merge \u2014 ${counts}.`
|
|
320
|
+
};
|
|
321
|
+
}
|
|
322
|
+
if (run2.verdict === "PASSED_WITH_WARNINGS") {
|
|
323
|
+
return {
|
|
324
|
+
exitCode: EXIT_CODE.PASSED,
|
|
325
|
+
level: "warn",
|
|
326
|
+
headline: `QA passed with warnings \u2014 ${counts}.`
|
|
327
|
+
};
|
|
328
|
+
}
|
|
329
|
+
return {
|
|
330
|
+
exitCode: EXIT_CODE.PASSED,
|
|
331
|
+
level: "success",
|
|
332
|
+
headline: `QA passed \u2014 ${counts}.`
|
|
333
|
+
};
|
|
334
|
+
}
|
|
335
|
+
|
|
336
|
+
// src/commands/wait-loop.ts
|
|
337
|
+
var DEFAULT_TIMEOUT_MINUTES = 30;
|
|
338
|
+
var POLL_INTERVAL_MS = 15e3;
|
|
339
|
+
var MINUTE_MS = 6e4;
|
|
340
|
+
async function waitForVerdict(api, context, deps, timeoutMs) {
|
|
341
|
+
const deadline = deps.now() + timeoutMs;
|
|
342
|
+
deps.logger.info(`Waiting for QA verdict on ${context.repo}#${context.prNumber}\u2026`);
|
|
343
|
+
let lastStatus;
|
|
344
|
+
while (true) {
|
|
345
|
+
const outcome = await api.getRunStatus(context);
|
|
346
|
+
if (outcome.ok) {
|
|
347
|
+
if (outcome.run.status !== lastStatus) {
|
|
348
|
+
deps.logger.info(` ${statusLine(outcome.run)}`);
|
|
349
|
+
lastStatus = outcome.run.status;
|
|
350
|
+
}
|
|
351
|
+
if (isTerminalStatus(outcome.run.status)) {
|
|
352
|
+
const verdict = resolveVerdictOutcome(outcome.run);
|
|
353
|
+
logAtLevel(deps.logger, verdict.level, verdict.headline);
|
|
354
|
+
return verdict.exitCode;
|
|
355
|
+
}
|
|
356
|
+
} else if (outcome.status !== 404) {
|
|
357
|
+
deps.logger.error(`Could not read QA run status: ${outcome.error}`);
|
|
358
|
+
return 1;
|
|
359
|
+
}
|
|
360
|
+
if (deps.now() >= deadline) {
|
|
361
|
+
deps.logger.error(
|
|
362
|
+
`Timed out after ${Math.round(timeoutMs / MINUTE_MS)} min waiting for a verdict.`
|
|
363
|
+
);
|
|
364
|
+
return EXIT_CODE.WAIT_TIMEOUT;
|
|
365
|
+
}
|
|
366
|
+
await deps.sleep(POLL_INTERVAL_MS);
|
|
367
|
+
}
|
|
368
|
+
}
|
|
369
|
+
function resolveTimeoutMs(timeout) {
|
|
370
|
+
if (timeout === void 0) return DEFAULT_TIMEOUT_MINUTES * MINUTE_MS;
|
|
371
|
+
const minutes = Number(timeout);
|
|
372
|
+
if (!Number.isFinite(minutes) || minutes <= 0) {
|
|
373
|
+
throw new ConfigError(`--timeout must be a positive number of minutes, got "${timeout}".`);
|
|
374
|
+
}
|
|
375
|
+
return minutes * MINUTE_MS;
|
|
376
|
+
}
|
|
377
|
+
function statusLine(run2) {
|
|
378
|
+
return run2.statusMessage ? `${run2.status} \u2014 ${run2.statusMessage}` : run2.status;
|
|
379
|
+
}
|
|
380
|
+
|
|
381
|
+
// src/commands/tunnel.ts
|
|
382
|
+
var defaultTunnelDeps = {
|
|
383
|
+
...defaultDeps,
|
|
384
|
+
openTunnel,
|
|
385
|
+
detectRepo: () => detectRepoFullName(),
|
|
386
|
+
detectHeadSha: () => detectHeadSha()
|
|
387
|
+
};
|
|
388
|
+
async function runTunnel(options, deps = defaultTunnelDeps) {
|
|
389
|
+
const config = resolveConfig(options);
|
|
390
|
+
const context = resolveRunContext({
|
|
391
|
+
repo: options.repo ?? deps.detectRepo(),
|
|
392
|
+
pr: options.pr,
|
|
393
|
+
sha: options.sha ?? deps.detectHeadSha(),
|
|
394
|
+
env: {}
|
|
395
|
+
});
|
|
396
|
+
const api = deps.createApi(config);
|
|
397
|
+
const tunnel = await resolvePublicTunnel(options, deps);
|
|
398
|
+
try {
|
|
399
|
+
assertValidPreviewUrl(tunnel.url);
|
|
400
|
+
deps.logger.info(`Exposing local build at ${tunnel.url}`);
|
|
401
|
+
const reported = await api.reportPreview({ ...context, url: tunnel.url });
|
|
402
|
+
if (!reported.ok) {
|
|
403
|
+
deps.logger.error(`Could not start QA run: ${reported.error}`);
|
|
404
|
+
return 1;
|
|
405
|
+
}
|
|
406
|
+
deps.logger.success(`QA run ${reported.runId} queued for ${context.repo}#${context.prNumber}.`);
|
|
407
|
+
if (options.wait === false) return 0;
|
|
408
|
+
return await waitForVerdict(api, context, deps, resolveTimeoutMs(options.timeout));
|
|
409
|
+
} finally {
|
|
410
|
+
tunnel.close();
|
|
411
|
+
}
|
|
412
|
+
}
|
|
413
|
+
function resolvePublicTunnel(options, deps) {
|
|
414
|
+
if (options.url) {
|
|
415
|
+
return Promise.resolve({
|
|
416
|
+
url: options.url,
|
|
417
|
+
close: () => {
|
|
418
|
+
}
|
|
419
|
+
});
|
|
420
|
+
}
|
|
421
|
+
const port = Number(options.port);
|
|
422
|
+
if (!Number.isInteger(port) || port <= 0) {
|
|
423
|
+
throw new ConfigError(
|
|
424
|
+
"--port <number> is required to open a tunnel (or pass --url for an externally managed tunnel)."
|
|
425
|
+
);
|
|
426
|
+
}
|
|
427
|
+
return deps.openTunnel(port);
|
|
428
|
+
}
|
|
429
|
+
|
|
430
|
+
// src/commands/wait.ts
|
|
431
|
+
async function runWait(options, deps = defaultDeps) {
|
|
432
|
+
const config = resolveConfig(options);
|
|
433
|
+
const context = resolvePrContext(options);
|
|
434
|
+
const api = deps.createApi(config);
|
|
435
|
+
return waitForVerdict(api, context, deps, resolveTimeoutMs(options.timeout));
|
|
436
|
+
}
|
|
437
|
+
|
|
438
|
+
// src/cli.ts
|
|
439
|
+
var cli = cac("datadisco-qa");
|
|
440
|
+
cli.command("report-preview", "Report a PR's preview-deploy URL so its pending QA run can start").option("--url <url>", "The deployed preview URL for the PR head SHA").option("--repo <owner/name>", "Repository the PR belongs to").option("--pr <number>", "Pull request number").option("--sha <sha>", "Head SHA the preview was built from").option("--api-url <url>", "DataDisco API base URL").option("--api-token <token>", "Workspace API token (or DATADISCO_API_TOKEN)").action((options) => run(() => runReportPreview(options)));
|
|
441
|
+
cli.command("wait", "Block until the PR's QA run reaches a verdict").option("--repo <owner/name>", "Repository the PR belongs to").option("--pr <number>", "Pull request number").option("--timeout <minutes>", "Give up after this many minutes (default 30)").option("--api-url <url>", "DataDisco API base URL").option("--api-token <token>", "Workspace API token (or DATADISCO_API_TOKEN)").action((options) => run(() => runWait(options)));
|
|
442
|
+
cli.command("tunnel", "Expose a local build and start a QA run against it").option("--port <port>", "Local port serving the build").option("--url <url>", "Use an externally managed tunnel URL instead of --port").option("--repo <owner/name>", "Repository (defaults to the git origin remote)").option("--pr <number>", "Pull request number").option("--sha <sha>", "Head SHA (defaults to the local git HEAD)").option("--no-wait", "Queue the run and exit without waiting for a verdict").option("--timeout <minutes>", "Give up waiting after this many minutes").option("--api-url <url>", "DataDisco API base URL").option("--api-token <token>", "Workspace API token (or DATADISCO_API_TOKEN)").action((options) => run(() => runTunnel(options)));
|
|
443
|
+
async function run(command) {
|
|
444
|
+
try {
|
|
445
|
+
process.exitCode = await command();
|
|
446
|
+
} catch (error) {
|
|
447
|
+
if (error instanceof ConfigError) {
|
|
448
|
+
console.error(pc2.red(error.message));
|
|
449
|
+
process.exitCode = 1;
|
|
450
|
+
return;
|
|
451
|
+
}
|
|
452
|
+
console.error(pc2.red(error instanceof Error ? error.message : String(error)));
|
|
453
|
+
process.exitCode = 1;
|
|
454
|
+
}
|
|
455
|
+
}
|
|
456
|
+
cli.help();
|
|
457
|
+
cli.version("0.0.1");
|
|
458
|
+
cli.parse();
|
package/package.json
ADDED
|
@@ -0,0 +1,57 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "@datadisco/qa",
|
|
3
|
+
"version": "0.1.0",
|
|
4
|
+
"description": "CI companion for DataDisco QA — report preview URLs, wait on persona-simulation gates, and tunnel local builds into QA runs.",
|
|
5
|
+
"keywords": [
|
|
6
|
+
"ci",
|
|
7
|
+
"datadisco",
|
|
8
|
+
"persona",
|
|
9
|
+
"preview",
|
|
10
|
+
"qa",
|
|
11
|
+
"testing"
|
|
12
|
+
],
|
|
13
|
+
"homepage": "https://github.com/Data-Disco-Inc/datadisco-qa#readme",
|
|
14
|
+
"bugs": "https://github.com/Data-Disco-Inc/datadisco-qa/issues",
|
|
15
|
+
"license": "MIT",
|
|
16
|
+
"repository": {
|
|
17
|
+
"type": "git",
|
|
18
|
+
"url": "git+https://github.com/Data-Disco-Inc/datadisco-qa.git"
|
|
19
|
+
},
|
|
20
|
+
"bin": {
|
|
21
|
+
"datadisco-qa": "dist/cli.js"
|
|
22
|
+
},
|
|
23
|
+
"files": [
|
|
24
|
+
"dist"
|
|
25
|
+
],
|
|
26
|
+
"type": "module",
|
|
27
|
+
"publishConfig": {
|
|
28
|
+
"access": "public"
|
|
29
|
+
},
|
|
30
|
+
"scripts": {
|
|
31
|
+
"build": "tsup",
|
|
32
|
+
"dev": "tsup --watch",
|
|
33
|
+
"typecheck": "tsc --noEmit",
|
|
34
|
+
"lint": "oxlint",
|
|
35
|
+
"format": "oxfmt",
|
|
36
|
+
"format:check": "oxfmt --check",
|
|
37
|
+
"test": "vitest run",
|
|
38
|
+
"test:watch": "vitest",
|
|
39
|
+
"prepare": "npm run build"
|
|
40
|
+
},
|
|
41
|
+
"dependencies": {
|
|
42
|
+
"cac": "^7.0.0",
|
|
43
|
+
"localtunnel": "^2.0.2",
|
|
44
|
+
"picocolors": "^1.1.1"
|
|
45
|
+
},
|
|
46
|
+
"devDependencies": {
|
|
47
|
+
"@types/node": "^22.0.0",
|
|
48
|
+
"oxfmt": "^0.55.0",
|
|
49
|
+
"oxlint": "^1.70.0",
|
|
50
|
+
"tsup": "^8.5.1",
|
|
51
|
+
"typescript": "^6.0.3",
|
|
52
|
+
"vitest": "^4.1.9"
|
|
53
|
+
},
|
|
54
|
+
"engines": {
|
|
55
|
+
"node": ">=20.19.0"
|
|
56
|
+
}
|
|
57
|
+
}
|