@plune-ai/cli 0.22.0 → 0.24.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +16 -8
- package/dist/cli.cjs +307 -33
- package/dist/index.cjs +9 -2
- package/dist/index.js +9 -2
- package/dist/reporter-core.cjs +278 -22
- package/dist/reporter-core.js +274 -18
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -119,7 +119,7 @@ the last resort, used only when `PLUNE_TOKEN` is set nowhere else. Nothing write
|
|
|
119
119
|
| `plune init` | Scaffold `plune.yaml`, a sample dataset, and `.env.example`. Flags: `--yes` (non-interactive), `--force`. |
|
|
120
120
|
| `plune login` | Save a [Plune platform](https://plune.ai/platform) API token so `sync` and `ingest` can reach it. The token is **checked against the API before it is saved**, so a wrong one fails here rather than two commands later. Get one at `https://beta.plune.ai` → Account → API tokens. Flags: `--token <token>` (omit to paste it or pipe it via stdin), `--skip-verify` (save without checking, for offline setup). In CI, skip this step: every platform command reads `PLUNE_TOKEN` from the environment first, and the saved login only when it is not set. |
|
|
121
121
|
| `plune logout` | Remove the saved token. |
|
|
122
|
-
| `plune sync` | Upload the latest local run to the platform. Flags: `--file <path>` to send a specific run JSON, `--fail-on-failures` to read the run's summary back once after the upload and exit with [the gate](#a-release-gate---fail-on-failures)'s code. |
|
|
122
|
+
| `plune sync` | Upload the latest local run to the platform. Flags: `--file <path>` to send a specific run JSON, `--fail-on-failures` to read the run's summary back once after the upload and exit with [the gate](#a-release-gate---fail-on-failures)'s code. Each upload carries an `Idempotency-Key` (SHA-256 of the body), so sending the same run again returns the same run rather than creating a second one. |
|
|
123
123
|
| `plune run import <file>` | Turn a **JUnit XML** or **Playwright JSON** report, or an **Allure `allure-results` folder**, into a run in Plune. Needs no provider key — nothing is generated. Flags: `--format junit\|playwright-json\|allure` (detected from the file when omitted; a folder is Allure), `--key <externalKey>` to land several reports in one run (with `PLUNE_SHARED_RUN=1` — see below), `--create` to offer unmatched tests to the review queue, `--fail-on-missing-attachments` to exit 1 when a file a JUnit report names with `[[ATTACHMENT\|path]]`, or an Allure result names as a `source`, is not there (the results are sent either way), `--upload-concurrency <n>` for files uploaded at once ([1 to 32, default 4](#many-screenshots-upload-concurrency-and-retries)), `--fail-on-failures` to exit with [the gate](#a-release-gate---fail-on-failures)'s code on the summary the platform froze when this import finished the run. |
|
|
124
124
|
| `plune run start` | Open a platform run — or join the one already carrying `--key` — and print its id. Flags: `--key <externalKey>` (generated when absent), `--json` for one machine-readable line. |
|
|
125
125
|
| `plune run finish <id>` | Close a platform run. This is what the reporter tells you to do for a run it had to leave open. Flags: `--terminate` to record it as cut short, `--reason <text>`, `--fail-on-failures` to exit with [the gate](#a-release-gate---fail-on-failures)'s code on the summary the platform froze. |
|
|
@@ -324,8 +324,13 @@ without closing.
|
|
|
324
324
|
|
|
325
325
|
### Many screenshots: upload concurrency and retries
|
|
326
326
|
|
|
327
|
-
|
|
328
|
-
|
|
327
|
+
Files go to Plune up to 100 to a request: `uploads-batch` announces them (with the SHA-256 of each, read as a stream,
|
|
328
|
+
so a file the platform already has is answered `exists` and is not sent again), the bytes go straight to your bucket
|
|
329
|
+
(or, for a project without one, 100 files or 6 MiB at a time through `direct-batch`), and `complete-batch` confirms
|
|
330
|
+
them. Every file has its own verdict: one that is refused is reported and the rest of its request goes through. A
|
|
331
|
+
report of 2760 screenshots is about 56 requests to Plune instead of 5500. The bytes themselves still go one request each,
|
|
332
|
+
so the PUTs to a bucket are bound by the round trip, not by bandwidth: four at a time by default, raise it on a runner
|
|
333
|
+
with a slow link:
|
|
329
334
|
|
|
330
335
|
```bash
|
|
331
336
|
PLUNE_UPLOAD_CONCURRENCY=16 plune run import ./playwright.json # 1 to 32, default 4
|
|
@@ -336,11 +341,14 @@ When the platform answers 429 with a `Retry-After` (seconds or an HTTP date, nev
|
|
|
336
341
|
the request is tried again after that time plus up to half as much, and every other request of the import
|
|
337
342
|
waits through the same pause. `PLUNE_UPLOAD_RETRIES` (0 to 20, default 6) says how many times; a 429 that
|
|
338
343
|
names no time is the quota and is not retried. A platform that is down (a 5xx or no answer) is still tried
|
|
339
|
-
twice (three times for a bucket). A
|
|
340
|
-
|
|
341
|
-
|
|
342
|
-
|
|
343
|
-
the same
|
|
344
|
+
twice (three times for a bucket). A 429 on a whole request waits the same way; a file that the bucket did not
|
|
345
|
+
answer for at `complete` is confirmed again, on its own. A file that stays refused is reported without it, never fails
|
|
346
|
+
the import, and is counted in the line printed before the summary:
|
|
347
|
+
`plune: files — 2758 uploaded, 0 skipped, 0 already there (skipped), 14 retried, 0 dropped, in 41.2 s (16 at once, 56 request(s) to Plune)`.
|
|
348
|
+
`already there` are the files Plune had from an earlier import of the same report (same result, name, type and bytes):
|
|
349
|
+
none of them is sent again. Against a platform older than the bulk routes (it answers 404 to `uploads-batch`) the CLI
|
|
350
|
+
sends one request per file, as before; it decides by that answer, not by a version. A value outside the range is
|
|
351
|
+
ignored with a warning, and the default is used. The Playwright reporter reads the same variables.
|
|
344
352
|
|
|
345
353
|
### When a test is deleted
|
|
346
354
|
|
package/dist/cli.cjs
CHANGED
|
@@ -2789,6 +2789,34 @@ var init_failure_detail = __esm({
|
|
|
2789
2789
|
}
|
|
2790
2790
|
});
|
|
2791
2791
|
|
|
2792
|
+
// src/reporter-core/files-batch.ts
|
|
2793
|
+
async function sha256File(path17) {
|
|
2794
|
+
const hash = (0, import_node_crypto2.createHash)("sha256");
|
|
2795
|
+
for await (const part of (0, import_node_fs4.createReadStream)(path17)) hash.update(part);
|
|
2796
|
+
return hash.digest("hex");
|
|
2797
|
+
}
|
|
2798
|
+
var import_node_crypto2, import_node_fs4, FILES_BATCH_MAX, DIRECT_BATCH_BYTES, BATCH_PIPELINES, sha256Bytes, BUCKET_CODES;
|
|
2799
|
+
var init_files_batch = __esm({
|
|
2800
|
+
"src/reporter-core/files-batch.ts"() {
|
|
2801
|
+
"use strict";
|
|
2802
|
+
init_cjs_shims();
|
|
2803
|
+
import_node_crypto2 = require("crypto");
|
|
2804
|
+
import_node_fs4 = require("fs");
|
|
2805
|
+
FILES_BATCH_MAX = 100;
|
|
2806
|
+
DIRECT_BATCH_BYTES = 6 * 1024 * 1024;
|
|
2807
|
+
BATCH_PIPELINES = 2;
|
|
2808
|
+
sha256Bytes = (bytes) => (0, import_node_crypto2.createHash)("sha256").update(bytes).digest("hex");
|
|
2809
|
+
BUCKET_CODES = /* @__PURE__ */ new Set([
|
|
2810
|
+
"bucket_unreachable",
|
|
2811
|
+
"not_connected",
|
|
2812
|
+
"platform_not_configured",
|
|
2813
|
+
"upload_expired",
|
|
2814
|
+
"file_mismatch",
|
|
2815
|
+
"file_missing"
|
|
2816
|
+
]);
|
|
2817
|
+
}
|
|
2818
|
+
});
|
|
2819
|
+
|
|
2792
2820
|
// src/reporter-core/session.ts
|
|
2793
2821
|
function nameOf(file) {
|
|
2794
2822
|
const own = file.name.split(/[\\/]/).pop()?.trim() || (file.path === void 0 ? "" : (0, import_node_path3.basename)(file.path));
|
|
@@ -2840,7 +2868,7 @@ function tally(counts, noun) {
|
|
|
2840
2868
|
}
|
|
2841
2869
|
function boundKey(key) {
|
|
2842
2870
|
if (key.value.length <= KEY_MAX) return key;
|
|
2843
|
-
const digest = (0,
|
|
2871
|
+
const digest = (0, import_node_crypto3.createHash)("sha256").update(key.value).digest("hex").slice(0, 16);
|
|
2844
2872
|
let head = key.value.slice(0, KEY_MAX - digest.length - 1);
|
|
2845
2873
|
if (/[\uD800-\uDBFF]$/.test(head)) head = head.slice(0, -1);
|
|
2846
2874
|
return { ...key, value: `${head}#${digest}` };
|
|
@@ -2892,7 +2920,7 @@ async function startRun(passed, declared2) {
|
|
|
2892
2920
|
`));
|
|
2893
2921
|
const fallbackPath = cfg.fallbackPath ?? DEFAULT_FALLBACK_PATH;
|
|
2894
2922
|
const externalKey = cfg.externalKey ?? null;
|
|
2895
|
-
const sessionMarker = (0,
|
|
2923
|
+
const sessionMarker = (0, import_node_crypto3.randomUUID)();
|
|
2896
2924
|
const batchSize = Math.min(cfg.batchSize ?? BATCH_DEFAULT, BATCH_MAX);
|
|
2897
2925
|
const offerDiscovered = cfg.offerDiscovered ?? false;
|
|
2898
2926
|
const token = cfg.token ?? loadToken() ?? "";
|
|
@@ -2900,6 +2928,9 @@ async function startRun(passed, declared2) {
|
|
|
2900
2928
|
const uploadRetries = Math.min(Math.max(Math.floor(cfg.uploadRetries ?? UPLOAD_RETRIES.default), UPLOAD_RETRIES.min), UPLOAD_RETRIES.max);
|
|
2901
2929
|
let retried = 0;
|
|
2902
2930
|
let uploadMs = 0;
|
|
2931
|
+
let requests = 0;
|
|
2932
|
+
let existed = 0;
|
|
2933
|
+
let batchMissing = false;
|
|
2903
2934
|
const client = createClient({
|
|
2904
2935
|
apiUrl: resolveApiUrl(cfg.apiUrl),
|
|
2905
2936
|
token,
|
|
@@ -3083,6 +3114,7 @@ async function startRun(passed, declared2) {
|
|
|
3083
3114
|
else log(`plune: ${KINDS[kind].label} "${name}" not uploaded \u2014 ${SKIPPED[why]}.`);
|
|
3084
3115
|
}
|
|
3085
3116
|
function ask(job, size) {
|
|
3117
|
+
requests += 1;
|
|
3086
3118
|
return client.post(`/v1/results/${encodeURIComponent(job.id)}/files/uploads`, {
|
|
3087
3119
|
files: [{ name: job.name, contentType: job.type, size }]
|
|
3088
3120
|
}, uploadRetries);
|
|
@@ -3117,13 +3149,14 @@ async function startRun(passed, declared2) {
|
|
|
3117
3149
|
if (job.body !== void 0) bytes = job.body;
|
|
3118
3150
|
else {
|
|
3119
3151
|
try {
|
|
3120
|
-
bytes = await
|
|
3152
|
+
bytes = await import_node_fs5.promises.readFile(job.path);
|
|
3121
3153
|
} catch {
|
|
3122
3154
|
return skip(kind, job.name, "missing");
|
|
3123
3155
|
}
|
|
3124
3156
|
}
|
|
3125
3157
|
const refusal = kind === "text" ? textRefusal(bytes) : void 0;
|
|
3126
3158
|
if (refusal !== void 0) return skip(kind, job.name, refusal);
|
|
3159
|
+
requests += 1;
|
|
3127
3160
|
const out = await client.upload(
|
|
3128
3161
|
`/v1/results/${encodeURIComponent(job.id)}/files?name=${encodeURIComponent(job.name)}`,
|
|
3129
3162
|
bytes,
|
|
@@ -3146,6 +3179,7 @@ async function startRun(passed, declared2) {
|
|
|
3146
3179
|
if (fresh === void 0) return failedToBucket(job, "bad_answer");
|
|
3147
3180
|
link = fresh;
|
|
3148
3181
|
}
|
|
3182
|
+
requests += 1;
|
|
3149
3183
|
const done2 = await client.post(`/v1/files/${encodeURIComponent(link.id)}/complete`, {}, uploadRetries);
|
|
3150
3184
|
if (!done2.ok) return failedToBucket(job, codeOf2(done2));
|
|
3151
3185
|
countsOf(job.kind).uploaded += 1;
|
|
@@ -3170,7 +3204,7 @@ async function startRun(passed, declared2) {
|
|
|
3170
3204
|
else if (job.path === void 0) return void skip(kind, job.name, "missing");
|
|
3171
3205
|
else {
|
|
3172
3206
|
try {
|
|
3173
|
-
size = (await
|
|
3207
|
+
size = (await import_node_fs5.promises.stat(job.path)).size;
|
|
3174
3208
|
} catch {
|
|
3175
3209
|
return void skip(kind, job.name, "missing");
|
|
3176
3210
|
}
|
|
@@ -3206,6 +3240,245 @@ async function startRun(passed, declared2) {
|
|
|
3206
3240
|
}
|
|
3207
3241
|
return () => uploadToBucket(job, size, link);
|
|
3208
3242
|
}
|
|
3243
|
+
async function uploadEach(jobs) {
|
|
3244
|
+
const byResult = /* @__PURE__ */ new Map();
|
|
3245
|
+
for (const job of jobs) byResult.set(job.id, [...byResult.get(job.id) ?? [], job]);
|
|
3246
|
+
await Promise.all(
|
|
3247
|
+
[...byResult.values()].map(async (group) => {
|
|
3248
|
+
const work = [];
|
|
3249
|
+
for (const job of group) {
|
|
3250
|
+
const act = await plan(job);
|
|
3251
|
+
if (act !== void 0) work.push(gated(act));
|
|
3252
|
+
}
|
|
3253
|
+
await Promise.all(work);
|
|
3254
|
+
})
|
|
3255
|
+
);
|
|
3256
|
+
}
|
|
3257
|
+
async function measure(job) {
|
|
3258
|
+
let size;
|
|
3259
|
+
if (job.body !== void 0) size = job.body.length;
|
|
3260
|
+
else if (job.path === void 0) return void skip(job.kind, job.name, "missing");
|
|
3261
|
+
else {
|
|
3262
|
+
try {
|
|
3263
|
+
size = (await import_node_fs5.promises.stat(job.path)).size;
|
|
3264
|
+
} catch {
|
|
3265
|
+
return void skip(job.kind, job.name, "missing");
|
|
3266
|
+
}
|
|
3267
|
+
}
|
|
3268
|
+
if (size === 0) return void skip(job.kind, job.name, "empty");
|
|
3269
|
+
return size;
|
|
3270
|
+
}
|
|
3271
|
+
const routeMissing = (out) => !out.ok && out.status === 404 && out.detail === "" && out.code === void 0;
|
|
3272
|
+
function takeRoom(job) {
|
|
3273
|
+
if (job.kind === "file") {
|
|
3274
|
+
skip(job.kind, job.name, "noStorage");
|
|
3275
|
+
return false;
|
|
3276
|
+
}
|
|
3277
|
+
const roomKey = `${job.id}/${job.kind}`;
|
|
3278
|
+
const left = rooms.get(roomKey) ?? KINDS[job.kind].room;
|
|
3279
|
+
if (left === 0) {
|
|
3280
|
+
skip(job.kind, job.name, KINDS[job.kind].limit);
|
|
3281
|
+
return false;
|
|
3282
|
+
}
|
|
3283
|
+
rooms.set(roomKey, left - 1);
|
|
3284
|
+
return true;
|
|
3285
|
+
}
|
|
3286
|
+
function refusedItem(job, size, item) {
|
|
3287
|
+
const code = item.code ?? "refused";
|
|
3288
|
+
if (code === "duplicate") existed += 1;
|
|
3289
|
+
else if (code === "no_storage") skip(job.kind, job.name, refusedHere(job, size) ?? "noStorage");
|
|
3290
|
+
else if (code === "too_many_files") skip(job.kind, job.name, KINDS[job.kind].limit);
|
|
3291
|
+
else if (BUCKET_CODES.has(code)) failedToBucket(job, code);
|
|
3292
|
+
else failed(job, code);
|
|
3293
|
+
}
|
|
3294
|
+
const itemPause = (round) => new Promise((r) => setTimeout(r, Math.min(250 * 2 ** round, 4e3)));
|
|
3295
|
+
async function batchOf(jobs) {
|
|
3296
|
+
if (batchMissing) return uploadEach(jobs);
|
|
3297
|
+
const sized = [];
|
|
3298
|
+
for (const job of jobs) {
|
|
3299
|
+
const size = await measure(job);
|
|
3300
|
+
if (size === void 0) continue;
|
|
3301
|
+
try {
|
|
3302
|
+
sized.push({ job, size, sha256: job.body !== void 0 ? sha256Bytes(job.body) : await sha256File(job.path) });
|
|
3303
|
+
} catch {
|
|
3304
|
+
skip(job.kind, job.name, "missing");
|
|
3305
|
+
}
|
|
3306
|
+
}
|
|
3307
|
+
if (sized.length === 0) return;
|
|
3308
|
+
const announced = await gated(() => {
|
|
3309
|
+
requests += 1;
|
|
3310
|
+
return client.post(
|
|
3311
|
+
"/v1/files/uploads-batch",
|
|
3312
|
+
{ items: sized.map((s) => ({ resultId: s.job.id, name: s.job.name, contentType: s.job.type, size: s.size, sha256: s.sha256 })) },
|
|
3313
|
+
uploadRetries
|
|
3314
|
+
);
|
|
3315
|
+
});
|
|
3316
|
+
if (routeMissing(announced)) {
|
|
3317
|
+
batchMissing = true;
|
|
3318
|
+
return uploadEach(sized.map((s) => s.job));
|
|
3319
|
+
}
|
|
3320
|
+
const verdicts = announced.ok ? announced.body?.items : void 0;
|
|
3321
|
+
if (!announced.ok || !Array.isArray(verdicts) || verdicts.length !== sized.length) {
|
|
3322
|
+
for (const s of sized) {
|
|
3323
|
+
if (announced.ok) failed(s.job, "bad_answer");
|
|
3324
|
+
else if (announced.code !== void 0) failedToBucket(s.job, announced.code);
|
|
3325
|
+
else failed(s.job, announced.detail || announced.kind);
|
|
3326
|
+
}
|
|
3327
|
+
return;
|
|
3328
|
+
}
|
|
3329
|
+
const puts = [];
|
|
3330
|
+
const directs = [];
|
|
3331
|
+
sized.forEach((s, i) => {
|
|
3332
|
+
const v = verdicts[i] ?? {};
|
|
3333
|
+
if (v.status === "exists") existed += 1;
|
|
3334
|
+
else if (v.status === "direct") {
|
|
3335
|
+
if (takeRoom(s.job)) directs.push(s);
|
|
3336
|
+
} else if (v.status === "upload") {
|
|
3337
|
+
const link = linkOf(v);
|
|
3338
|
+
if (link === void 0) return failed(s.job, "bad_answer");
|
|
3339
|
+
if (!bucketSeen) {
|
|
3340
|
+
bucketSeen = true;
|
|
3341
|
+
log(BUCKET_FOUND);
|
|
3342
|
+
}
|
|
3343
|
+
puts.push({ s, link });
|
|
3344
|
+
} else if (v.status === "refused") refusedItem(s.job, s.size, v);
|
|
3345
|
+
else failed(s.job, "bad_answer");
|
|
3346
|
+
});
|
|
3347
|
+
const toComplete = [];
|
|
3348
|
+
await Promise.all(
|
|
3349
|
+
puts.map(
|
|
3350
|
+
({ s, link: first }) => gated(async () => {
|
|
3351
|
+
const source = s.job.body !== void 0 ? { body: s.job.body } : { path: s.job.path, size: s.size };
|
|
3352
|
+
let link = first;
|
|
3353
|
+
for (let renewed = false; ; ) {
|
|
3354
|
+
const put = await client.put(link.url, link.headers, source);
|
|
3355
|
+
if (put.ok) break;
|
|
3356
|
+
if (put.status !== 403 || renewed) return failedToBucket(s.job, put.code);
|
|
3357
|
+
renewed = true;
|
|
3358
|
+
requests += 1;
|
|
3359
|
+
const again = await client.post(
|
|
3360
|
+
"/v1/files/uploads-batch",
|
|
3361
|
+
{ items: [{ resultId: s.job.id, name: s.job.name, contentType: s.job.type, size: s.size, sha256: s.sha256 }] },
|
|
3362
|
+
uploadRetries
|
|
3363
|
+
);
|
|
3364
|
+
const fresh = again.ok ? linkOf(again.body?.items?.[0]) : void 0;
|
|
3365
|
+
if (fresh === void 0) return failedToBucket(s.job, again.ok ? "bad_answer" : codeOf2(again));
|
|
3366
|
+
link = fresh;
|
|
3367
|
+
}
|
|
3368
|
+
toComplete.push({ job: s.job, id: link.id });
|
|
3369
|
+
})
|
|
3370
|
+
)
|
|
3371
|
+
);
|
|
3372
|
+
await complete(toComplete);
|
|
3373
|
+
await sendDirect(directs);
|
|
3374
|
+
}
|
|
3375
|
+
async function complete(items) {
|
|
3376
|
+
for (let round = 0; items.length > 0; round += 1) {
|
|
3377
|
+
const out = await gated(() => {
|
|
3378
|
+
requests += 1;
|
|
3379
|
+
return client.post("/v1/files/complete-batch", { ids: items.map((i) => i.id) }, uploadRetries);
|
|
3380
|
+
});
|
|
3381
|
+
if (routeMissing(out)) {
|
|
3382
|
+
await Promise.all(
|
|
3383
|
+
items.map(
|
|
3384
|
+
(i) => gated(async () => {
|
|
3385
|
+
requests += 1;
|
|
3386
|
+
const one = await client.post(`/v1/files/${encodeURIComponent(i.id)}/complete`, {}, uploadRetries);
|
|
3387
|
+
if (one.ok) countsOf(i.job.kind).uploaded += 1;
|
|
3388
|
+
else failedToBucket(i.job, codeOf2(one));
|
|
3389
|
+
})
|
|
3390
|
+
)
|
|
3391
|
+
);
|
|
3392
|
+
return;
|
|
3393
|
+
}
|
|
3394
|
+
const verdicts = out.ok ? out.body?.items : void 0;
|
|
3395
|
+
if (!out.ok || !Array.isArray(verdicts) || verdicts.length !== items.length) {
|
|
3396
|
+
for (const i of items) failedToBucket(i.job, out.ok ? "bad_answer" : codeOf2(out));
|
|
3397
|
+
return;
|
|
3398
|
+
}
|
|
3399
|
+
const again = [];
|
|
3400
|
+
items.forEach((i, at) => {
|
|
3401
|
+
const v = verdicts[at] ?? {};
|
|
3402
|
+
if (v.status === "ready") countsOf(i.job.kind).uploaded += 1;
|
|
3403
|
+
else if (v.retryable === true && round < Math.min(uploadRetries, 3)) again.push(i);
|
|
3404
|
+
else refusedItem(i.job, 0, v.status === "refused" ? v : { code: "bad_answer" });
|
|
3405
|
+
});
|
|
3406
|
+
items = again;
|
|
3407
|
+
if (items.length > 0) {
|
|
3408
|
+
retried += items.length;
|
|
3409
|
+
await itemPause(round);
|
|
3410
|
+
}
|
|
3411
|
+
}
|
|
3412
|
+
}
|
|
3413
|
+
async function sendDirect(items) {
|
|
3414
|
+
const groups = [];
|
|
3415
|
+
let bytes = 0;
|
|
3416
|
+
for (const item of items) {
|
|
3417
|
+
const group = groups[groups.length - 1];
|
|
3418
|
+
if (group === void 0 || group.length >= FILES_BATCH_MAX || bytes + item.size > DIRECT_BATCH_BYTES) {
|
|
3419
|
+
groups.push([item]);
|
|
3420
|
+
bytes = item.size;
|
|
3421
|
+
} else {
|
|
3422
|
+
group.push(item);
|
|
3423
|
+
bytes += item.size;
|
|
3424
|
+
}
|
|
3425
|
+
}
|
|
3426
|
+
for (const group of groups) {
|
|
3427
|
+
const body = [];
|
|
3428
|
+
const sent = [];
|
|
3429
|
+
for (const { job, size } of group) {
|
|
3430
|
+
const { kind } = job;
|
|
3431
|
+
if (size > KINDS[kind].bytes) {
|
|
3432
|
+
skip(kind, job.name, KINDS[kind].size);
|
|
3433
|
+
continue;
|
|
3434
|
+
}
|
|
3435
|
+
let data;
|
|
3436
|
+
try {
|
|
3437
|
+
data = job.body ?? await import_node_fs5.promises.readFile(job.path);
|
|
3438
|
+
} catch {
|
|
3439
|
+
skip(kind, job.name, "missing");
|
|
3440
|
+
continue;
|
|
3441
|
+
}
|
|
3442
|
+
const refusal = kind === "text" ? textRefusal(data) : void 0;
|
|
3443
|
+
if (refusal !== void 0) {
|
|
3444
|
+
skip(kind, job.name, refusal);
|
|
3445
|
+
continue;
|
|
3446
|
+
}
|
|
3447
|
+
body.push({ resultId: job.id, name: job.name, contentType: job.type, data: data.toString("base64") });
|
|
3448
|
+
sent.push({ job, size });
|
|
3449
|
+
}
|
|
3450
|
+
if (body.length === 0) continue;
|
|
3451
|
+
const out = await gated(() => {
|
|
3452
|
+
requests += 1;
|
|
3453
|
+
return client.post("/v1/files/direct-batch", { items: body }, uploadRetries);
|
|
3454
|
+
});
|
|
3455
|
+
if (routeMissing(out)) {
|
|
3456
|
+
batchMissing = true;
|
|
3457
|
+
for (const { job, size } of sent) await gated(() => uploadDirect(job, size));
|
|
3458
|
+
continue;
|
|
3459
|
+
}
|
|
3460
|
+
const verdicts = out.ok ? out.body?.items : void 0;
|
|
3461
|
+
if (!out.ok || !Array.isArray(verdicts) || verdicts.length !== sent.length) {
|
|
3462
|
+
for (const { job } of sent) failed(job, out.ok ? "bad_answer" : out.detail || out.kind);
|
|
3463
|
+
continue;
|
|
3464
|
+
}
|
|
3465
|
+
sent.forEach(({ job }, i) => {
|
|
3466
|
+
const v = verdicts[i] ?? {};
|
|
3467
|
+
if (v.status === "stored") countsOf(job.kind).uploaded += 1;
|
|
3468
|
+
else if (v.status === "exists") existed += 1;
|
|
3469
|
+
else refusedItem(job, 0, v.status === "refused" ? v : { code: "bad_answer" });
|
|
3470
|
+
});
|
|
3471
|
+
}
|
|
3472
|
+
}
|
|
3473
|
+
async function uploadBatched(jobs) {
|
|
3474
|
+
const batches = chunk(jobs, FILES_BATCH_MAX);
|
|
3475
|
+
let next = 0;
|
|
3476
|
+
await Promise.all(
|
|
3477
|
+
Array.from({ length: Math.min(BATCH_PIPELINES, batches.length) }, async () => {
|
|
3478
|
+
for (let at = next++; at < batches.length; at = next++) await batchOf(batches[at]);
|
|
3479
|
+
})
|
|
3480
|
+
);
|
|
3481
|
+
}
|
|
3209
3482
|
async function upload(items, filesByKey) {
|
|
3210
3483
|
if (filesByKey.size === 0 || !Array.isArray(items)) return;
|
|
3211
3484
|
const jobs = [];
|
|
@@ -3231,18 +3504,8 @@ async function startRun(passed, declared2) {
|
|
|
3231
3504
|
}
|
|
3232
3505
|
}
|
|
3233
3506
|
const started = Date.now();
|
|
3234
|
-
|
|
3235
|
-
|
|
3236
|
-
await Promise.all(
|
|
3237
|
-
[...byResult.values()].map(async (group) => {
|
|
3238
|
-
const work = [];
|
|
3239
|
-
for (const job of group) {
|
|
3240
|
-
const act = await plan(job);
|
|
3241
|
-
if (act !== void 0) work.push(gated(act));
|
|
3242
|
-
}
|
|
3243
|
-
await Promise.all(work);
|
|
3244
|
-
})
|
|
3245
|
-
);
|
|
3507
|
+
if (batchMissing) await uploadEach(jobs);
|
|
3508
|
+
else await uploadBatched(jobs);
|
|
3246
3509
|
uploadMs += Date.now() - started;
|
|
3247
3510
|
}
|
|
3248
3511
|
async function flush() {
|
|
@@ -3338,9 +3601,9 @@ async function startRun(passed, declared2) {
|
|
|
3338
3601
|
if (shortened.size > 0) parts.push(`${shortened.size} test name(s) shortened to fit a key`);
|
|
3339
3602
|
const files = [stats.screenshots, stats.textFiles, stats.otherFiles];
|
|
3340
3603
|
const total = (what) => files.reduce((n, counts) => n + counts[what], 0);
|
|
3341
|
-
if (total("uploaded") + total("skipped") + total("failed") > 0) {
|
|
3604
|
+
if (total("uploaded") + total("skipped") + total("failed") + existed > 0) {
|
|
3342
3605
|
log(
|
|
3343
|
-
`plune: files \u2014 ${total("uploaded")} uploaded, ${total("skipped")} skipped, ${retried} retried, ${total("failed")} dropped, in ${(uploadMs / 1e3).toFixed(1)} s (${uploadsAtOnce} at once)`
|
|
3606
|
+
`plune: files \u2014 ${total("uploaded")} uploaded, ${total("skipped")} skipped, ${existed} already there (skipped), ${retried} retried, ${total("failed")} dropped, in ${(uploadMs / 1e3).toFixed(1)} s (${uploadsAtOnce} at once, ${requests} request(s) to Plune)`
|
|
3344
3607
|
);
|
|
3345
3608
|
}
|
|
3346
3609
|
log(`plune: ${parts.join(" \xB7 ")}`);
|
|
@@ -3397,13 +3660,13 @@ async function startRun(passed, declared2) {
|
|
|
3397
3660
|
}
|
|
3398
3661
|
};
|
|
3399
3662
|
}
|
|
3400
|
-
var
|
|
3663
|
+
var import_node_crypto3, import_node_fs5, import_node_path3, import_node_util2, RESOLVE_CHUNK, BATCH_MAX, DISCOVER_MAX, BATCH_DEFAULT, EXPECTED_MAX, KEY_MAX, TITLE_MAX, COUNTED, SCREENSHOT_TYPES, SCREENSHOT_BYTES, SCREENSHOTS_PER_RESULT, TEXT_BYTES, TEXTS_PER_RESULT, UTF8, TYPE_BY_EXTENSION, METADATA_TYPE2, NAME_MAX, SKIPPED, BUCKET_FOUND, KINDS, codeOf2, BATCH_BYTES, ENVELOPE_BYTES;
|
|
3401
3664
|
var init_session = __esm({
|
|
3402
3665
|
"src/reporter-core/session.ts"() {
|
|
3403
3666
|
"use strict";
|
|
3404
3667
|
init_cjs_shims();
|
|
3405
|
-
|
|
3406
|
-
|
|
3668
|
+
import_node_crypto3 = require("crypto");
|
|
3669
|
+
import_node_fs5 = require("fs");
|
|
3407
3670
|
import_node_path3 = require("path");
|
|
3408
3671
|
import_node_util2 = require("util");
|
|
3409
3672
|
init_api_url();
|
|
@@ -3411,6 +3674,7 @@ var init_session = __esm({
|
|
|
3411
3674
|
init_client();
|
|
3412
3675
|
init_env2();
|
|
3413
3676
|
init_failure_detail();
|
|
3677
|
+
init_files_batch();
|
|
3414
3678
|
init_fallback();
|
|
3415
3679
|
RESOLVE_CHUNK = 500;
|
|
3416
3680
|
BATCH_MAX = 500;
|
|
@@ -3822,12 +4086,18 @@ async function handleSync(deps = {}) {
|
|
|
3822
4086
|
const run = readRun(cwd, deps.file);
|
|
3823
4087
|
const doFetch = deps.fetchImpl ?? fetch;
|
|
3824
4088
|
const endpoint = `${apiUrl}/v1/runs`;
|
|
4089
|
+
const payload = JSON.stringify(run);
|
|
4090
|
+
const idempotencyKey = (0, import_node_crypto4.createHash)("sha256").update(payload).digest("hex");
|
|
3825
4091
|
let res;
|
|
3826
4092
|
try {
|
|
3827
4093
|
res = await doFetch(endpoint, {
|
|
3828
4094
|
method: "POST",
|
|
3829
|
-
headers: {
|
|
3830
|
-
|
|
4095
|
+
headers: {
|
|
4096
|
+
"content-type": "application/json",
|
|
4097
|
+
authorization: `Bearer ${token}`,
|
|
4098
|
+
"idempotency-key": idempotencyKey
|
|
4099
|
+
},
|
|
4100
|
+
body: payload
|
|
3831
4101
|
});
|
|
3832
4102
|
} catch {
|
|
3833
4103
|
throw new SyncNetworkError(apiUrl);
|
|
@@ -3863,11 +4133,12 @@ function reportSyncFailure(err, write) {
|
|
|
3863
4133
|
}
|
|
3864
4134
|
return null;
|
|
3865
4135
|
}
|
|
3866
|
-
var fs8, path9, NotLoggedInError, TokenRejectedError, SyncFileError, SyncNetworkError, SyncHttpError;
|
|
4136
|
+
var import_node_crypto4, fs8, path9, NotLoggedInError, TokenRejectedError, SyncFileError, SyncNetworkError, SyncHttpError;
|
|
3867
4137
|
var init_sync = __esm({
|
|
3868
4138
|
"src/cli/commands/sync.ts"() {
|
|
3869
4139
|
"use strict";
|
|
3870
4140
|
init_cjs_shims();
|
|
4141
|
+
import_node_crypto4 = require("crypto");
|
|
3871
4142
|
fs8 = __toESM(require("fs"), 1);
|
|
3872
4143
|
path9 = __toESM(require("path"), 1);
|
|
3873
4144
|
init_run_result_schema();
|
|
@@ -4878,7 +5149,7 @@ function readPlaywrightJson(source, file) {
|
|
|
4878
5149
|
const { config, suites } = parsed;
|
|
4879
5150
|
const rootDir = typeof config?.rootDir === "string" ? config.rootDir : void 0;
|
|
4880
5151
|
const buildHref = config?.metadata?.ci?.buildHref;
|
|
4881
|
-
const repoRoot = rootDir !== void 0 && (0,
|
|
5152
|
+
const repoRoot = rootDir !== void 0 && (0, import_node_fs6.existsSync)(rootDir) ? repoRootOf(rootDir) : void 0;
|
|
4882
5153
|
const ctx = {
|
|
4883
5154
|
...rootDir !== void 0 ? { rootDir } : {},
|
|
4884
5155
|
...repoRoot !== void 0 ? { repoRoot } : {},
|
|
@@ -4897,12 +5168,12 @@ function readPlaywrightJson(source, file) {
|
|
|
4897
5168
|
function looksLikePlaywrightJson(source) {
|
|
4898
5169
|
return /^\s*\{/.test(source) && source.includes('"suites"');
|
|
4899
5170
|
}
|
|
4900
|
-
var
|
|
5171
|
+
var import_node_fs6, SOURCE3, TOKEN3, NOT_FAILED, JsonReportError;
|
|
4901
5172
|
var init_playwright_json = __esm({
|
|
4902
5173
|
"src/importers/playwright-json.ts"() {
|
|
4903
5174
|
"use strict";
|
|
4904
5175
|
init_cjs_shims();
|
|
4905
|
-
|
|
5176
|
+
import_node_fs6 = require("fs");
|
|
4906
5177
|
init_failure_detail();
|
|
4907
5178
|
init_result_key();
|
|
4908
5179
|
SOURCE3 = "playwright";
|
|
@@ -5099,7 +5370,10 @@ function describe(result, apiUrl) {
|
|
|
5099
5370
|
);
|
|
5100
5371
|
}
|
|
5101
5372
|
if (result.offered > 0) {
|
|
5102
|
-
|
|
5373
|
+
const source = result.format === "playwright-json" ? "playwright" : result.format;
|
|
5374
|
+
lines.push(
|
|
5375
|
+
`${result.offered} unknown test(s) offered to the review queue. They wait there for a decision and do not count on the run until approved. To skip review for this source, add \`${source}\` under Settings \u2192 Trusted sources.`
|
|
5376
|
+
);
|
|
5103
5377
|
} else if (result.unresolved > 0 && result.created === 0 && !result.offering) {
|
|
5104
5378
|
lines.push("Nothing was offered to the review queue \u2014 pass --create to propose the unmatched tests.");
|
|
5105
5379
|
} else if (result.unresolved > 0 && result.created === 0 && result.unoffered === 0) {
|
|
@@ -5930,11 +6204,11 @@ module.exports = __toCommonJS(cli_exports);
|
|
|
5930
6204
|
init_cjs_shims();
|
|
5931
6205
|
init_api_url();
|
|
5932
6206
|
var import_commander = require("commander");
|
|
5933
|
-
var
|
|
6207
|
+
var import_node_fs7 = require("fs");
|
|
5934
6208
|
var import_node_path4 = require("path");
|
|
5935
6209
|
var import_node_url = require("url");
|
|
5936
6210
|
var __dirname = (0, import_node_path4.dirname)((0, import_node_url.fileURLToPath)(importMetaUrl));
|
|
5937
|
-
var pkg = JSON.parse((0,
|
|
6211
|
+
var pkg = JSON.parse((0, import_node_fs7.readFileSync)((0, import_node_path4.join)(__dirname, "..", "package.json"), "utf-8"));
|
|
5938
6212
|
function toFormat(s) {
|
|
5939
6213
|
return s === "json" || s === "markdown" ? s : "console";
|
|
5940
6214
|
}
|
|
@@ -5943,7 +6217,7 @@ function shouldColor(outputPath) {
|
|
|
5943
6217
|
}
|
|
5944
6218
|
function writeReport(text2, outputPath) {
|
|
5945
6219
|
if (outputPath !== void 0) {
|
|
5946
|
-
(0,
|
|
6220
|
+
(0, import_node_fs7.writeFileSync)(outputPath, text2);
|
|
5947
6221
|
} else {
|
|
5948
6222
|
process.stdout.write(text2 + "\n");
|
|
5949
6223
|
}
|
|
@@ -6170,7 +6444,7 @@ function createProgram() {
|
|
|
6170
6444
|
false
|
|
6171
6445
|
).option(
|
|
6172
6446
|
"--upload-concurrency <n>",
|
|
6173
|
-
"
|
|
6447
|
+
"File uploads in flight at once, 1 to 32 (default 4; PLUNE_UPLOAD_CONCURRENCY sets it too). Files are announced and confirmed up to 100 to a request; a 429 is retried PLUNE_UPLOAD_RETRIES times (default 6)",
|
|
6174
6448
|
(v) => {
|
|
6175
6449
|
if (!/^\d+$/.test(v) || Number(v) < 1 || Number(v) > 32) throw new import_commander.InvalidArgumentError("a whole number from 1 to 32.");
|
|
6176
6450
|
return Number(v);
|
|
@@ -6442,7 +6716,7 @@ function isMainModule() {
|
|
|
6442
6716
|
const entry = process.argv[1];
|
|
6443
6717
|
if (entry === void 0) return false;
|
|
6444
6718
|
try {
|
|
6445
|
-
return (0,
|
|
6719
|
+
return (0, import_node_fs7.realpathSync)(entry) === (0, import_node_fs7.realpathSync)((0, import_node_url.fileURLToPath)(importMetaUrl));
|
|
6446
6720
|
} catch {
|
|
6447
6721
|
return false;
|
|
6448
6722
|
}
|
package/dist/index.cjs
CHANGED
|
@@ -1610,6 +1610,7 @@ async function handleRun(options, depsFactory) {
|
|
|
1610
1610
|
}
|
|
1611
1611
|
|
|
1612
1612
|
// src/cli/commands/sync.ts
|
|
1613
|
+
var import_node_crypto2 = require("crypto");
|
|
1613
1614
|
var fs6 = __toESM(require("fs"), 1);
|
|
1614
1615
|
var path5 = __toESM(require("path"), 1);
|
|
1615
1616
|
|
|
@@ -1794,12 +1795,18 @@ async function handleSync(deps = {}) {
|
|
|
1794
1795
|
const run = readRun(cwd, deps.file);
|
|
1795
1796
|
const doFetch = deps.fetchImpl ?? fetch;
|
|
1796
1797
|
const endpoint = `${apiUrl}/v1/runs`;
|
|
1798
|
+
const payload = JSON.stringify(run);
|
|
1799
|
+
const idempotencyKey = (0, import_node_crypto2.createHash)("sha256").update(payload).digest("hex");
|
|
1797
1800
|
let res;
|
|
1798
1801
|
try {
|
|
1799
1802
|
res = await doFetch(endpoint, {
|
|
1800
1803
|
method: "POST",
|
|
1801
|
-
headers: {
|
|
1802
|
-
|
|
1804
|
+
headers: {
|
|
1805
|
+
"content-type": "application/json",
|
|
1806
|
+
authorization: `Bearer ${token}`,
|
|
1807
|
+
"idempotency-key": idempotencyKey
|
|
1808
|
+
},
|
|
1809
|
+
body: payload
|
|
1803
1810
|
});
|
|
1804
1811
|
} catch {
|
|
1805
1812
|
throw new SyncNetworkError(apiUrl);
|
package/dist/index.js
CHANGED
|
@@ -1554,6 +1554,7 @@ async function handleRun(options, depsFactory) {
|
|
|
1554
1554
|
}
|
|
1555
1555
|
|
|
1556
1556
|
// src/cli/commands/sync.ts
|
|
1557
|
+
import { createHash as createHash2 } from "crypto";
|
|
1557
1558
|
import * as fs6 from "fs";
|
|
1558
1559
|
import * as path5 from "path";
|
|
1559
1560
|
|
|
@@ -1738,12 +1739,18 @@ async function handleSync(deps = {}) {
|
|
|
1738
1739
|
const run = readRun(cwd, deps.file);
|
|
1739
1740
|
const doFetch = deps.fetchImpl ?? fetch;
|
|
1740
1741
|
const endpoint = `${apiUrl}/v1/runs`;
|
|
1742
|
+
const payload = JSON.stringify(run);
|
|
1743
|
+
const idempotencyKey = createHash2("sha256").update(payload).digest("hex");
|
|
1741
1744
|
let res;
|
|
1742
1745
|
try {
|
|
1743
1746
|
res = await doFetch(endpoint, {
|
|
1744
1747
|
method: "POST",
|
|
1745
|
-
headers: {
|
|
1746
|
-
|
|
1748
|
+
headers: {
|
|
1749
|
+
"content-type": "application/json",
|
|
1750
|
+
authorization: `Bearer ${token}`,
|
|
1751
|
+
"idempotency-key": idempotencyKey
|
|
1752
|
+
},
|
|
1753
|
+
body: payload
|
|
1747
1754
|
});
|
|
1748
1755
|
} catch {
|
|
1749
1756
|
throw new SyncNetworkError(apiUrl);
|