squally-reporter 0.3.0 → 0.4.0

package/README.md

@@ -1,121 +1,211 @@
-# squally-reporter
-
-Playwright reporter that sends test results to your Squally dashboard.
-
-Learn more at [squally-app.vercel.app](https://squally-app.vercel.app).
-
-## Requirements
-
-- Node.js >= 22.19.0 (required by the `undici` dependency)
-- `@playwright/test` >= 1.30.0
-
-## Install
-
-```bash
-npm install squally-reporter
-```
-
-## Config
-
-Set two env vars (e.g. in CI secrets or `.env`):
-
-```bash
-SQUALLY_API_URL=https://squally-app.vercel.app
-SQUALLY_API_KEY=your-project-api-key
-```
-
-Find your API key on your project's Settings page in the Squally dashboard.
-
-Then add the reporter to `playwright.config.ts`:
-
-```ts
-import { defineConfig } from "@playwright/test";
-
-export default defineConfig({
-  reporter: [
-    ["list"],
-    ["squally-reporter"],
-  ],
-});
-```
-
-Alternatively, pass config explicitly instead of using env vars:
-
-```ts
-reporter: [
-  ["squally-reporter", { apiUrl: "https://squally-app.vercel.app", apiKey: "..." }],
-],
-```
-
-That's it - run your tests as usual (`npx playwright test`) and results will
-be sent to Squally after the run finishes.
-
-## Screenshot/trace upload
-
-If your `playwright.config.ts` captures screenshots or traces, the reporter
-uploads them to Squally so they show up next to the failing test in the
-dashboard - no separate setup beyond the two env vars above.
-
-**Only from attempts worth looking at:** screenshots/traces are uploaded
-only for attempts that **failed**, or that **needed a retry** (even if that
-retry then passed) - never for an attempt that passed on its first try,
-regardless of how `screenshot`/`trace` are configured. Nobody looks at a
-screenshot of a test that just passed, and uploading one anyway would only
-eat into your organization's storage for no benefit.
-
-**You need to actually enable capture in `playwright.config.ts`** - without
-it, Playwright doesn't produce any attachments, so there's nothing for the
-reporter to find or upload (no error either way, it's a silent no-op):
-
-```ts
-export default defineConfig({
-  use: {
-    screenshot: "only-on-failure",
-    trace: "retain-on-failure",
-  },
-  // ...
-});
-```
-
-**Limits:**
-- 50 MB per individual file. Larger files are skipped (logged as a
-  `console.warn`, the test result itself is still reported as usual).
-- A total storage cap per organization, depending on your Squally plan.
-  Once reached, further uploads are skipped the same way until you either
-  free up space or upgrade.
-
-Upload is best-effort and fully non-blocking, same principle as the rest of
-this reporter - a failed or skipped upload never fails your test run or
-changes its exit code, it only logs a `console.warn` explaining what
-happened.
-
-## Tested revision (optional)
-
-If your test-automation code lives in the same repo as the application it
-tests, Squally can already tell which commit was tested from the CI
-provider's own commit env vars. If they're **separate repos** (e.g. a
-dedicated E2E-test repo pointed at a deployed app), set one of the following
-so Squally knows which application state was actually tested:
-
-```bash
-# Same repo as the application under test - Squally will trust this CI
-# job's own commit (GITHUB_SHA / CI_COMMIT_SHA) as the tested revision.
-SAME_REPO_AS_APP=true
-
-# Separate repos - pass the tested revision explicitly (a commit SHA, tag,
-# or branch name/ref).
-TESTED_REVISION=v1.2.0
-```
-
-Neither variable is required - without them, test results are still sent
-and stored exactly as before.
-
-## Support
-
-Found a bug or have a question? Please open an issue on
-[GitHub Issues](https://github.com/squallydev-admin/squally-reporter/issues) -
-that's the primary way to report problems or ask questions.
-
-## License
-
-MIT - see [LICENSE](./LICENSE).
+# squally-reporter
+
+Playwright reporter that sends test results to your Squally dashboard.
+
+Learn more at [squally-app.vercel.app](https://squally-app.vercel.app).
+
+## Requirements
+
+- Node.js >= 22.19.0 (required by the `undici` dependency)
+- `@playwright/test` >= 1.30.0
+
+## Install
+
+```bash
+npm install -D squally-reporter
+```
+
+## Config
+
+Set two env vars (e.g. in CI secrets or `.env`):
+
+```bash
+SQUALLY_API_URL=https://squally-app.vercel.app
+SQUALLY_API_KEY=your-project-api-key
+```
+
+Find your API key on your project's Settings page in the Squally dashboard.
+
+Then add the reporter to `playwright.config.ts`:
+
+```ts
+import { defineConfig } from "@playwright/test";
+
+export default defineConfig({
+  reporter: [
+    ["list"],
+    ["squally-reporter"],
+  ],
+});
+```
+
+Alternatively, pass config explicitly instead of using env vars:
+
+```ts
+reporter: [
+  ["squally-reporter", { apiUrl: "https://squally-app.vercel.app", apiKey: "..." }],
+],
+```
+
+That's it - run your tests as usual (`npx playwright test`) and results will
+be sent to Squally after the run finishes.
+
+## Screenshot/trace upload
+
+If your `playwright.config.ts` captures screenshots or traces, the reporter
+uploads them to Squally so they show up next to the failing test in the
+dashboard - no separate setup beyond the two env vars above.
+
+**Only from attempts worth looking at:** screenshots/traces are uploaded
+only for attempts that **failed**, or that **needed a retry** (even if that
+retry then passed) - never for an attempt that passed on its first try,
+regardless of how `screenshot`/`trace` are configured. Nobody looks at a
+screenshot of a test that just passed, and uploading one anyway would only
+eat into your organization's storage for no benefit.
+
+**You need to actually enable capture in `playwright.config.ts`** - without
+it, Playwright doesn't produce any attachments, so there's nothing for the
+reporter to find or upload (no error either way, it's a silent no-op):
+
+```ts
+export default defineConfig({
+  use: {
+    screenshot: "only-on-failure",
+    trace: "retain-on-failure",
+  },
+  // ...
+});
+```
+
+**Limits:**
+- 50 MB per individual file. Larger files are skipped (logged as a
+  `console.warn`, the test result itself is still reported as usual).
+- A total storage cap per organization, depending on your Squally plan.
+  Once reached, further uploads are skipped the same way until you either
+  free up space or upgrade.
+- Two minutes per upload, and 30 seconds for the calls that surround it. An
+  unreachable or very slow server means a skipped upload and a `console.warn`,
+  not a CI job waiting indefinitely.
+
+Upload is best-effort and fully non-blocking, same principle as the rest of
+this reporter - a failed or skipped upload never fails your test run or
+changes its exit code, it only logs a `console.warn` explaining what
+happened.
+
+## Limits per run
+
+Squally accepts at most **5,000 test results in a single run**. Beyond that the
+whole report is rejected, so the reporter checks before sending and skips the
+upload with a warning rather than letting the server discard it after the fact.
+Nothing else about your test run is affected - this only means that one run
+does not appear in the dashboard.
+
+**Retries count separately.** Each attempt is its own result, so a suite of
+2,000 tests having a bad day reaches the ceiling sooner than the test count
+suggests.
+
+If you run into this, the practical way around it today is to let separate
+parts of the suite report separately - for example as separate Playwright
+projects or separate CI jobs, each producing its own run. Splitting one report
+across several requests is not supported yet.
+
+There is also a hard **~4.5 MB limit on the request itself**, coming from the
+hosting platform rather than from Squally. Very large error messages, stack
+traces or console output can hit it before the 5,000 mark; the reporter already
+truncates each field to what the server accepts, which keeps normal runs well
+clear of it.
+
+## Sharding
+
+**Playwright's `--shard` is not merged automatically.** Each shard is a
+separate process with its own reporter instance, so each one reports its own
+run: a job split four ways shows up in Squally as four runs from the same
+commit, each holding only its own slice of the suite. The reporter says so in
+the log when it sees `--shard`.
+
+What that does and does not affect:
+
+- **Per-test flakiness stays correct.** A test lives in exactly one shard, so
+  it still contributes one data point per CI run, and its history, flip rate
+  and retry rate are unaffected.
+- **Run-level numbers are off.** The run count, "runs per day", and each run's
+  own pass/fail (it only covers its slice) all reflect shards rather than CI
+  runs.
+
+### Reporting sharded runs as one run
+
+Use Playwright's own blob reports: have each shard write a blob, then merge
+them in one final job and let this reporter run on the merged result. That
+produces exactly one run with every test in it.
+
+```yaml
+test:
+  strategy:
+    matrix:
+      shard: [1, 2, 3, 4]
+  steps:
+    - uses: actions/checkout@v4
+    - run: npx playwright test --shard=${{ matrix.shard }}/4 --reporter=blob
+    - uses: actions/upload-artifact@v4
+      with:
+        name: blob-${{ matrix.shard }}
+        path: blob-report
+
+merge:
+  needs: test
+  steps:
+    # Required - see the note below.
+    - uses: actions/checkout@v4
+    - uses: actions/download-artifact@v4
+      with:
+        pattern: blob-*
+        merge-multiple: true
+        path: all-blobs
+    - run: npx playwright merge-reports --reporter=squally-reporter ./all-blobs
+      env:
+        SQUALLY_API_URL: ${{ secrets.SQUALLY_API_URL }}
+        SQUALLY_API_KEY: ${{ secrets.SQUALLY_API_KEY }}
+```
+
+**The merge job needs the repository checked out.** Branch, commit, commit
+message and author are read on whichever machine the reporter runs on - during
+a merge that is the merge job, not the shards. Without a checkout (and without
+the CI provider's own commit variables) they all come out empty, and a run
+without a branch is never counted as a run on your stable branch, which is what
+Squally's broken/healthy verdicts are based on. `actions/checkout` in the merge
+job is enough.
+
+The run's start time is taken from the earliest test, not from when the
+reporter started, so a merge that happens minutes after the tests still dates
+the run correctly.
+
+## Tested revision (optional)
+
+If your test-automation code lives in the same repo as the application it
+tests, Squally can already tell which commit was tested from the CI
+provider's own commit env vars. If they're **separate repos** (e.g. a
+dedicated E2E-test repo pointed at a deployed app), set one of the following
+so Squally knows which application state was actually tested:
+
+```bash
+# Same repo as the application under test - Squally will trust this CI
+# job's own commit (GITHUB_SHA / CI_COMMIT_SHA) as the tested revision.
+SAME_REPO_AS_APP=true
+
+# Separate repos - pass the tested revision explicitly (a commit SHA, tag,
+# or branch name/ref).
+TESTED_REVISION=v1.2.0
+```
+
+Neither variable is required - without them, test results are still sent
+and stored exactly as before.
+
+## Support
+
+Found a bug or have a question? Please open an issue on
+[GitHub Issues](https://github.com/squallydev-admin/squally-reporter/issues) -
+that's the primary way to report problems or ask questions.
+
+## License
+
+MIT - see [LICENSE](./LICENSE).

package/dist/index.d.ts

@@ -10,10 +10,12 @@ export default class SquallyReporter implements Reporter {
     private startedAt;
     private results;
     private artifacts;
+    private earliestTestStart;
     constructor(options?: SquallyReporterOptions);
-    onBegin(_config: FullConfig, _suite: Suite): void;
+    onBegin(config: FullConfig, _suite: Suite): void;
     onTestEnd(test: TestCase, result: TestResult): void;
     onEnd(result: FullResult): Promise<void>;
+    private runStartedAt;
     private uploadArtifacts;
     private presignAndUpload;
 }

package/dist/index.js

@@ -8,6 +8,109 @@ const node_fs_1 = require("node:fs");
 const promises_1 = require("node:fs/promises");
 const node_path_1 = __importDefault(require("node:path"));
 const undici_1 = require("undici");
+// Mirrors squally-app's ingest contract (app/api/ingest/route.ts,
+// FIELD_LIMITS / checkStringField). Kept as a copy on purpose - the reporter
+// ships independently of the app and cannot import from it - which means this
+// table has to be updated when that one changes. It is deliberately not
+// "roughly right": the server rejects the ENTIRE payload on the FIRST field
+// that is too long, so one over-long value used to cost the whole run, every
+// test result in it, with a bare "400 Bad Request" as the only clue.
+//
+// ciProvider is absent because the server does not length-check it, and this
+// reporter only ever sends one of four fixed strings for it.
+const FIELD_LIMITS = {
+    branch: 250,
+    commitSha: 100,
+    commitMessage: 500,
+    authorName: 100,
+    testedRevision: 200,
+    eventType: 100,
+    testName: 500,
+    filePath: 500,
+    errorMessage: 5000,
+    errorStack: 20000,
+    consoleOutput: 20000,
+};
+// ASCII only, like everything else in this file, and short: it has to fit
+// INSIDE the limit it is protecting, and authorName only has 100 characters
+// to give away.
+const TRUNCATION_MARKER = "... [truncated]";
+// Squally rejects a payload with more than this many entries outright - the
+// whole run, not the surplus. Mirrored here so the reporter can say so up
+// front instead of the customer finding out from a 400 after the suite has
+// already finished.
+//
+// Deliberately NOT enforced by trimming: unlike an over-long error message,
+// the surplus here is test results, and dropping them would turn a loud
+// failure into a run that quietly looks like those tests never existed. The
+// reporter reports what happened or says why it could not - it does not
+// invent a smaller truth.
+//
+// Retries count as separate entries, so a suite of 2 000 tests reaches this
+// faster than the number suggests.
+const MAX_TEST_RESULTS = 5000;
+// Visible rather than silent: a value that was cut says so, so nobody debugs
+// a "wrong" commit message that is merely shortened. For testName/filePath
+// this also alters Squally's test identity - unavoidable, since the
+// alternative is losing the run outright, and two names sharing their first
+// 500 characters would collide either way.
+function truncate(value, max) {
+    if (value.length <= max)
+        return value;
+    if (max <= TRUNCATION_MARKER.length)
+        return value.slice(0, max);
+    return value.slice(0, max - TRUNCATION_MARKER.length) + TRUNCATION_MARKER;
+}
+function truncateOrNull(value, max) {
+    return value === null ? null : truncate(value, max);
+}
+// How much of a server error body to put in the log. Enough for the server's
+// own one-line explanation, bounded so that an unexpected HTML error page
+// from a proxy cannot dump kilobytes into the customer's CI output.
+// undici leaves headersTimeout and bodyTimeout at 300 000 ms each when they
+// are not set, and nothing here used to set them. A hung server therefore cost
+// 5 minutes for the run POST, 5 for presign, 5 for confirm and 5 per upload
+// round - with a hundred artifacts at five in parallel, well over an hour of a
+// CI job sitting there with no output explaining what it is waiting for. That
+// is worse for a customer than an error: an error at least says something.
+//
+// Two budgets, because the two kinds of request are not comparable. The JSON
+// calls exchange kilobytes and have no business taking longer than half a
+// minute. An upload can be a 50 MB trace over a CI runner's modest uplink,
+// where two minutes is tight-ish but still far short of "hung".
+const API_REQUEST_TIMEOUT_MS = 30000;
+const UPLOAD_REQUEST_TIMEOUT_MS = 120000;
+const ERROR_BODY_LOG_LIMIT = 500;
+// Turns a failed response into something a customer can act on. Today the
+// caller only sees "400 Bad Request"; the reason ("commitMessage exceeds max
+// length of 500 characters") sits in the body and was being thrown away.
+//
+// Nothing from the REQUEST is logged, only the server's own response - the
+// Authorization header must never reach the CI log.
+// A timed-out request throws like any other network failure, and is handled
+// like one - warn, carry on, never touch the exit code. It only gets its own
+// wording because "fetch failed" would leave the customer guessing whether the
+// server said no or simply never answered.
+function describeRequestError(err, timeoutMs) {
+    const name = err instanceof Error ? err.name : "";
+    if (name === "TimeoutError" || name === "AbortError") {
+        return `no response within ${timeoutMs} ms - giving up`;
+    }
+    return String(err);
+}
+async function describeResponseError(response) {
+    let detail = "";
+    try {
+        const body = (await response.text()).trim();
+        if (body)
+            detail = ` - ${truncate(body, ERROR_BODY_LOG_LIMIT)}`;
+    }
+    catch {
+        // Reading the body is best-effort: a body that cannot be read must not
+        // turn a logged warning into a thrown error.
+    }
+    return `${response.status} ${response.statusText}${detail}`;
+}
 // Maps Playwright's more granular statuses down to the three
 // statuses Squally stores (passed | failed | skipped).
 // "timedOut" and "interrupted" both count as failures for reporting purposes.
@@ -194,6 +297,27 @@ function formatConsoleOutput(result) {
 // fast enough that even a few hundred artifacts finish in well under 15
 // minutes, bounded enough not to overwhelm a typical CI runner.
 const UPLOAD_CONCURRENCY = 5;
+// Playwright does not let the process end on its own: the runner calls
+// gracefullyCloseAll().then(() => process.exit(code)) (playwright-core's
+// gracefullyProcessExitDoNotHang). process.exit() tears the event loop down
+// immediately, and on Windows that aborts the whole process with
+//   Assertion failed: !(handle->flags & UV_HANDLE_CLOSING), src/win/async.c
+// if any of our own I/O is still winding down at that moment - the exit code
+// becomes 127 instead of 0, i.e. a green test run reports as failed.
+//
+// Measured, so the number is not a guess: with the run POST alone the process
+// exits cleanly; from the second request on (presign) it aborts reproducibly.
+// A 0 ms or 10 ms wait does not help, 100 ms survives two requests, and 250 ms
+// survives the full path (ingest -> presign -> upload -> confirm) over
+// repeated runs.
+//
+// This is a mitigation, not a cure: it widens the window rather than removing
+// the race, because nothing in Node's public API reports "all my handles are
+// really gone". Do not shrink it without re-running the Windows job in CI -
+// that job exists specifically to catch this (see .github/workflows/ci.yml).
+// The same crash was fixed once before for the single-request path and came
+// back when 0.3.0 added the artifact requests.
+const EXIT_DRAIN_MS = 250;
 // No new dependency for this - a fixed-size worker pool that keeps pulling
 // the next item until the list is exhausted is ~10 lines, not worth pulling
 // in e.g. p-limit for.
@@ -260,6 +384,8 @@ class SquallyReporter {
         this.startedAt = new Date();
         this.results = [];
         this.artifacts = [];
+        // Earliest TestResult.startTime seen, see the startedAt note in onEnd().
+        this.earliestTestStart = null;
         const apiUrl = options.apiUrl ?? process.env.SQUALLY_API_URL;
         const apiKey = options.apiKey ?? process.env.SQUALLY_API_KEY;
         if (!apiUrl || !apiKey) {
@@ -276,27 +402,54 @@ class SquallyReporter {
         this.apiKey = apiKey;
         this.enabled = true;
     }
-    onBegin(_config, _suite) {
+    onBegin(config, _suite) {
         if (!this.enabled)
             return;
         this.startedAt = new Date();
+        // Each shard is its own process with its own reporter instance, so each
+        // one reports a run of its own. Measured: two shards over two spec files
+        // produce two runs holding two results each; a shard that receives no
+        // tests produces none at all.
+        //
+        // Worth saying out loud what this does and does not break: per-test
+        // history is unaffected, because a test lives in exactly one shard and so
+        // still contributes one data point per CI run. What gets distorted is the
+        // run level - the number of runs, each run's pass/fail (it only covers
+        // its own slice), and anything counted per run in the dashboard.
+        if (config.shard) {
+            console.warn(`[squally-reporter] Sharding detected (--shard=${config.shard.current}/${config.shard.total}). ` +
+                `Each shard reports separately, so this CI run will show up in Squally as up to ` +
+                `${config.shard.total} runs, each holding only its own slice of the suite. ` +
+                "Per-test flakiness stays correct; run counts and per-run pass/fail do not. " +
+                "For a single run, write a blob report per shard (--reporter=blob) and merge " +
+                "afterwards: npx playwright merge-reports --reporter=squally-reporter ./blob-report");
+        }
     }
     onTestEnd(test, result) {
         if (!this.enabled)
             return;
         const error = result.errors[0];
-        const testName = test.titlePath().slice(1).join(" > ");
-        const filePath = toPosixPath(node_path_1.default.relative(process.cwd(), test.location.file));
+        // Truncated HERE, once, not when the payload is built: testName/filePath
+        // are Squally's test identity, and /api/artifacts/presign resolves each
+        // artifact back to a TestResult through exactly these two values plus
+        // retryCount. Cutting them in two places, or cutting them only for the
+        // ingest payload, would make every artifact of a long-named test
+        // unmatchable server-side.
+        const testName = truncate(test.titlePath().slice(1).join(" > "), FIELD_LIMITS.testName);
+        const filePath = truncate(toPosixPath(node_path_1.default.relative(process.cwd(), test.location.file)), FIELD_LIMITS.filePath);
         const status = mapStatus(result.status);
+        if (this.earliestTestStart === null || result.startTime < this.earliestTestStart) {
+            this.earliestTestStart = result.startTime;
+        }
         this.results.push({
             testName,
             filePath,
             status,
             durationMs: result.duration,
             retryCount: result.retry,
-            errorMessage: error?.message ? stripAnsi(error.message) : null,
-            errorStack: error?.stack ? stripAnsi(error.stack) : null,
-            consoleOutput: formatConsoleOutput(result),
+            errorMessage: truncateOrNull(error?.message ? stripAnsi(error.message) : null, FIELD_LIMITS.errorMessage),
+            errorStack: truncateOrNull(error?.stack ? stripAnsi(error.stack) : null, FIELD_LIMITS.errorStack),
+            consoleOutput: truncateOrNull(formatConsoleOutput(result), FIELD_LIMITS.consoleOutput),
         });
         if (!attemptQualifiesForArtifacts(status, result.retry))
             return;
@@ -342,32 +495,58 @@ class SquallyReporter {
         // sending anything.
         if (this.results.length === 0)
             return;
+        // Checked before the request, not after: the server would answer 400 and
+        // discard the entire run, and the customer would be left correlating a
+        // status code with a suite that finished minutes ago.
+        if (this.results.length > MAX_TEST_RESULTS) {
+            console.warn(`[squally-reporter] This run has ${this.results.length} test results, and Squally ` +
+                `accepts at most ${MAX_TEST_RESULTS} per run - nothing was sent. Note that retries ` +
+                "count separately. Split the suite across several Playwright projects or " +
+                "configurations so each reports its own run, or get in touch if you need a " +
+                "single run this size.");
+            return;
+        }
         const resolvedRevision = resolveTestedRevision();
+        // commitMessage is the one that bites in practice: `git log -1 %B` is the
+        // full body, and a squash merge carrying a PR description is routinely
+        // past 500 characters. Before this, that single value discarded every test
+        // result in the run.
         const payload = {
-            branch: detectBranch(),
-            commitSha: detectCommitSha(),
-            commitMessage: detectCommitMessage(),
-            authorName: detectAuthorName(),
+            branch: truncateOrNull(detectBranch(), FIELD_LIMITS.branch),
+            commitSha: truncateOrNull(detectCommitSha(), FIELD_LIMITS.commitSha),
+            commitMessage: truncateOrNull(detectCommitMessage(), FIELD_LIMITS.commitMessage),
+            authorName: truncateOrNull(detectAuthorName(), FIELD_LIMITS.authorName),
             ciProvider: detectCiProvider(),
             status: result.status === "passed" ? "passed" : "failed",
-            startedAt: this.startedAt.toISOString(),
+            startedAt: this.runStartedAt().toISOString(),
             finishedAt: new Date().toISOString(),
             testResults: this.results,
-            testedRevision: resolvedRevision?.revision ?? null,
+            testedRevision: truncateOrNull(resolvedRevision?.revision ?? null, FIELD_LIMITS.testedRevision),
             revisionSource: resolvedRevision?.source ?? null,
-            eventType: detectEventType(),
+            eventType: truncateOrNull(detectEventType(), FIELD_LIMITS.eventType),
         };
-        // Node's global fetch() pools connections through a shared, long-lived
-        // keep-alive dispatcher by default. In a short-lived CLI process like
-        // this reporter, that pooled socket can still be mid-teardown when
-        // Playwright exits the process, which crashes with a libuv assertion on
-        // Windows ("UV_HANDLE_CLOSING", src/win/async.c). Using our own Agent
-        // for this one request and closing it explicitly guarantees no open
-        // handle is left behind by the time onEnd() resolves. Importing fetch
-        // from undici too (not Node's global one) avoids a possible instanceof
-        // mismatch between Node's internal bundled undici and this installed
-        // copy when passing our Agent as the dispatcher.
-        const dispatcher = new undici_1.Agent({ keepAliveTimeout: 1, keepAliveMaxTimeout: 1 });
+        // Our own Agent rather than the global one: Node's global fetch() pools
+        // connections through a shared, long-lived keep-alive dispatcher, and a
+        // pooled socket outliving this run is exactly what we do not want in a
+        // short-lived CLI process. Importing fetch from undici too (not Node's
+        // global one) avoids a possible instanceof mismatch between Node's
+        // internal bundled undici and this installed copy when passing our Agent
+        // as the dispatcher.
+        //
+        // NOTE: this does NOT on its own prevent the Windows exit crash - that
+        // claim used to stand here and was wrong. Closing the dispatcher resolves
+        // before the sockets are actually gone; see EXIT_DRAIN_MS above. Measured
+        // with global fetch as well: dropping undici entirely does not help
+        // either, so the client is not what makes the difference.
+        // The per-request AbortSignals below are the real bound; these are a
+        // backstop so that nothing can outlive the longest budget even if a signal
+        // never fires.
+        const dispatcher = new undici_1.Agent({
+            keepAliveTimeout: 1,
+            keepAliveMaxTimeout: 1,
+            headersTimeout: UPLOAD_REQUEST_TIMEOUT_MS,
+            bodyTimeout: UPLOAD_REQUEST_TIMEOUT_MS,
+        });
         try {
             let runId = null;
             try {
@@ -379,11 +558,12 @@ class SquallyReporter {
                     },
                     body: JSON.stringify(payload),
                     dispatcher,
+                    signal: AbortSignal.timeout(API_REQUEST_TIMEOUT_MS),
                 });
                 if (!response.ok) {
                     // Never throw here - a failing reporter upload should not fail the
                     // whole CI run just because the dashboard was unreachable.
-                    console.error(`[squally-reporter] Ingest failed: ${response.status} ${response.statusText}`);
+                    console.error(`[squally-reporter] Ingest failed: ${await describeResponseError(response)}`);
                 }
                 else {
                     const body = (await response.json());
@@ -391,7 +571,7 @@ class SquallyReporter {
                 }
             }
             catch (err) {
-                console.error("[squally-reporter] Failed to reach Squally API:", err);
+                console.error("[squally-reporter] Failed to reach Squally API:", describeRequestError(err, API_REQUEST_TIMEOUT_MS));
             }
             // Artifact upload is a separate, best-effort step layered on top of
             // ingest, never a reason to fail it: it needs a runId (presign looks
@@ -406,8 +586,26 @@ class SquallyReporter {
         }
         finally {
             await dispatcher.close();
+            // See EXIT_DRAIN_MS - closing the dispatcher resolves before the
+            // underlying sockets are fully gone, and Playwright exits the process
+            // the moment onEnd() resolves.
+            await new Promise((resolve) => setTimeout(resolve, EXIT_DRAIN_MS));
         }
     }
+    // onBegin's timestamp is normally the right answer: it fires before the
+    // first test and therefore includes global setup, webServer startup and so
+    // on. It is wrong in exactly one case - `npx playwright merge-reports`,
+    // where onBegin fires on the merging machine, minutes or hours after the
+    // tests actually ran, and would date the whole run to the merge.
+    //
+    // The earlier of the two is right in both situations: during a normal run
+    // onBegin precedes every test, during a merge it follows all of them. No
+    // detection of which mode we are in is needed.
+    runStartedAt() {
+        if (this.earliestTestStart === null)
+            return this.startedAt;
+        return this.earliestTestStart < this.startedAt ? this.earliestTestStart : this.startedAt;
+    }
     // Top-level safety net - presignAndUpload already handles presign/
     // upload/confirm failures individually (see Aufgabe 5's requirement that
     // none of this ever affects the test suite's exit code), but this catch
@@ -441,16 +639,18 @@ class SquallyReporter {
                 },
                 body: JSON.stringify({ runId, artifacts: presignArtifacts }),
                 dispatcher,
+                signal: AbortSignal.timeout(API_REQUEST_TIMEOUT_MS),
             });
             if (!response.ok) {
-                console.warn(`[squally-reporter] Artifact presign failed: ${response.status} ${response.statusText} - skipping artifact upload for this run`);
+                console.warn(`[squally-reporter] Artifact presign failed: ${await describeResponseError(response)}` +
+                    " - skipping artifact upload for this run");
                 return;
             }
             const body = (await response.json());
             presignResults = body.results;
         }
         catch (err) {
-            console.warn("[squally-reporter] Artifact presign request failed:", err);
+            console.warn("[squally-reporter] Artifact presign request failed:", describeRequestError(err, API_REQUEST_TIMEOUT_MS));
             return;
         }
         // Maps a presign result back to the local file it came from -
@@ -483,13 +683,15 @@ class SquallyReporter {
                     method: "PUT",
                     body: fileBody,
                     dispatcher,
+                    signal: AbortSignal.timeout(UPLOAD_REQUEST_TIMEOUT_MS),
                 });
                 if (!response.ok) {
-                    console.warn(`[squally-reporter] Upload failed for ${artifact.type} of '${artifact.testName}': ${response.status} ${response.statusText}`);
+                    console.warn(`[squally-reporter] Upload failed for ${artifact.type} of '${artifact.testName}': ` +
+                        (await describeResponseError(response)));
                 }
             }
             catch (err) {
-                console.warn(`[squally-reporter] Upload failed for ${artifact.type} of '${artifact.testName}':`, err);
+                console.warn(`[squally-reporter] Upload failed for ${artifact.type} of '${artifact.testName}':`, describeRequestError(err, UPLOAD_REQUEST_TIMEOUT_MS));
             }
         });
         // Confirm every artifact that was actually presigned (not the ones
@@ -515,9 +717,10 @@ class SquallyReporter {
                 },
                 body: JSON.stringify({ runId, artifacts: confirmArtifacts }),
                 dispatcher,
+                signal: AbortSignal.timeout(API_REQUEST_TIMEOUT_MS),
             });
             if (!response.ok) {
-                console.warn(`[squally-reporter] Artifact confirm failed: ${response.status} ${response.statusText}`);
+                console.warn(`[squally-reporter] Artifact confirm failed: ${await describeResponseError(response)}`);
                 return;
             }
             const body = (await response.json());
@@ -528,7 +731,7 @@ class SquallyReporter {
             }
         }
         catch (err) {
-            console.warn("[squally-reporter] Artifact confirm request failed:", err);
+            console.warn("[squally-reporter] Artifact confirm request failed:", describeRequestError(err, API_REQUEST_TIMEOUT_MS));
         }
     }
 }

package/package.json

@@ -1,38 +1,40 @@
-{
-  "name": "squally-reporter",
-  "version": "0.3.0",
-  "description": "Playwright reporter that sends test results to your Squally dashboard.",
-  "author": "Squally <squally.dev@gmail.com>",
-  "keywords": [
-    "playwright",
-    "playwright-reporter",
-    "flaky-tests",
-    "test-results",
-    "ci"
-  ],
-  "main": "dist/index.js",
-  "types": "dist/index.d.ts",
-  "files": [
-    "dist"
-  ],
-  "engines": {
-    "node": ">=22.19.0"
-  },
-  "scripts": {
-    "build": "tsc",
-    "prepublishOnly": "npm run build",
-    "prepare": "npm run build"
-  },
-  "peerDependencies": {
-    "@playwright/test": ">=1.30.0"
-  },
-  "devDependencies": {
-    "@playwright/test": "^1.48.0",
-    "@types/node": "^20.14.0",
-    "typescript": "^5.6.0"
-  },
-  "license": "MIT",
-  "dependencies": {
-    "undici": "^8.10.0"
-  }
-}
+{
+  "name": "squally-reporter",
+  "version": "0.4.0",
+  "description": "Playwright reporter that sends test results to your Squally dashboard.",
+  "author": "Squally <squally.dev@gmail.com>",
+  "keywords": [
+    "playwright",
+    "playwright-reporter",
+    "flaky-tests",
+    "test-results",
+    "ci"
+  ],
+  "main": "dist/index.js",
+  "types": "dist/index.d.ts",
+  "files": [
+    "dist"
+  ],
+  "engines": {
+    "node": ">=22.19.0"
+  },
+  "scripts": {
+    "build": "tsc",
+    "prepublishOnly": "npm run build",
+    "prepare": "npm run build",
+    "test:exit-code": "node test/exit-code/run.mjs",
+    "test": "npm run build && npm run test:exit-code"
+  },
+  "peerDependencies": {
+    "@playwright/test": ">=1.30.0"
+  },
+  "devDependencies": {
+    "@playwright/test": "^1.48.0",
+    "@types/node": "^20.14.0",
+    "typescript": "^5.6.0"
+  },
+  "license": "MIT",
+  "dependencies": {
+    "undici": "^8.10.0"
+  }
+}