@ultimat3/testing 23.0.0 → 24.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CLAUDE.md +9 -9
- package/README.md +50 -5
- package/package.json +15 -15
- package/src/budget-store-conformance.ts +81 -0
- package/src/cdp-browser.ts +5 -10
- package/src/cdp-connection.ts +23 -6
- package/src/cdp-e2e-session.ts +10 -1
- package/src/cdp-errors.ts +113 -8
- package/src/cdp-launch-attempt.ts +192 -0
- package/src/cdp-launch.ts +49 -122
- package/src/cdp-offline-script.ts +33 -3
- package/src/cdp-wire-watch.ts +113 -0
- package/src/determinism.ts +33 -4
- package/src/e2e-app.ts +3 -0
- package/src/e2e-run.ts +2 -2
- package/src/e2e-spawn.ts +1 -8
- package/src/errors.ts +14 -1
- package/src/factories.ts +4 -6
- package/src/file-boundary.ts +12 -7
- package/src/fixture-island.ts +18 -45
- package/src/fixture-jobs.ts +9 -5
- package/src/free-port.ts +46 -0
- package/src/harness.ts +2 -1
- package/src/index.ts +12 -0
- package/src/island-dom.ts +8 -4
- package/src/island-html-entities.ts +27 -0
- package/src/island-scratch.ts +41 -0
- package/src/job-driver-checks.ts +177 -0
- package/src/job-driver-conformance.ts +21 -0
- package/src/live-node.ts +2 -0
- package/src/preload.ts +2 -8
- package/src/registry-leak-guard.ts +3 -1
- package/src/registry-snapshot.ts +22 -1
- package/src/sealed-network.ts +5 -1
- package/src/template-db.ts +6 -3
package/CLAUDE.md
CHANGED
|
@@ -72,19 +72,20 @@ is its own entry point and not part of the barrel.
|
|
|
72
72
|
| Teardown restores, never uninstalls | `describeApp`/`testApp` capture the seal and the determinism snapshot before booting and put those back |
|
|
73
73
|
| Teardown is a `finally` | an `app.close()` that rejects still reaches `db.drop()` and still restores the process state; the first failure is what the caller sees. |
|
|
74
74
|
| A boot that rejects is its own teardown | `acquireWorkerDatabase`, `seed` or `boot` throwing returns no `BootedHarness`, so no caller can ever reach `close()` |
|
|
75
|
-
| A found template is not a migrated one | `template-db.ts` tolerates "already exists" for the `CREATE DATABASE` alone. `config.migrate` runs unconditionally
|
|
75
|
+
| A found template is not a migrated one | `template-db.ts` tolerates "already exists" for the `CREATE DATABASE` alone. `config.migrate` runs unconditionally; drop + clone hold the same lock |
|
|
76
76
|
| Fixture teardown | a fixture that installs process-global state (the ambient job or mail driver) implements `Symbol.dispose` / `Symbol.asyncDispose` and restores what was there |
|
|
77
77
|
| Building one by hand | `createRunJobs()` outside a fixture body is not disposed for you — dispose it in `afterEach`, or the next file inherits your queue and your event bus |
|
|
78
78
|
| Factory strategy | an association is built with the strategy that asked for it: `build()` never reaches a database, `create()` writes the parent first. Never a third strategy |
|
|
79
79
|
| One write seam | `usePersister` is the only place `create()` writes. A factory that took a repo argument would put the seam at every call site |
|
|
80
80
|
| Factory seeds | derived from the table name unless given, so two entities never draw the same uuid stream. `reset()` cascades into associated parents — a half-reset row is worse than none |
|
|
81
81
|
| Shared examples | `behavesLike` calls `describe`, so it goes at declaration scope; bun rejects a `describe` inside a test body |
|
|
82
|
+
| A port's contract is shared examples | `jobDriverConformance`, `budgetStoreConformance` — `behavesLike(…, () => yours)` |
|
|
82
83
|
| An island needs a BUILDER, not an import | `buildIslands` is `@ultimat3/cli`'s and both packages are tier 5; the one declared edge is `cli → testing`, so the reverse is a `bun run boundaries` failure. |
|
|
83
84
|
| `mountIsland` AWAITS `mount` | `IslandEntry['mount']` returns `unknown`, not `void`, and the call is awaited |
|
|
84
85
|
| Dispose STOPS the island, then restores the globals | when `mount` resolved to a function, `[Symbol.dispose]` calls it BEFORE `restore()` |
|
|
85
86
|
| The micro-DOM is the fixture's, once **for islands** | `island-dom.ts`. `packages/ui/src/fake-dom.ts` is a second one for keyboard code, and `ui -> testing` is upward, so it is not a copy to collapse. `bun test` has no DOM and no DOM library may be added |
|
|
86
87
|
| `style` and `classList` RECORD | `FakeStyle` is one declaration map behind all four spellings compiled Solid uses (static attribute, `setProperty`, `cssText`, `removeAttribute`), so a test can assert the component set `--form-gap` |
|
|
87
|
-
| `classList` is the class attribute | not a list of its own, so `classList.toggle` —
|
|
88
|
+
| `classList` is the class attribute | not a list of its own, so `classList.toggle` — which the compiler emits INLINE for `classList={{ … }}`, no runtime helper between — and `className` can never answer one element two ways. |
|
|
88
89
|
| A `document` listener is the documentElement's | no bubbling: `document.addEventListener` registers on `documentElement`, and `fire(mounted.documentElement, 'keydown', …)` drives it. One handler per type, last wins |
|
|
89
90
|
| `querySelector` skips `this` | descendants only, as the DOM's does. Matching the element it is called on made a host `<div>` answer `find('div')` with the container the test built rather than the markup the island rendered |
|
|
90
91
|
| The selector grammar is SMALL and REFUSES | `island-selector.ts`: compounds of tag, `#id`, `.class`, `[attr]`, `[attr="value"]`, joined by space or `>`. Anything else is `X_TEST_ISLAND_SELECTOR_UNSUPPORTED` with the offset, never an empty answer |
|
|
@@ -102,12 +103,10 @@ is its own entry point and not part of the barrel.
|
|
|
102
103
|
| Loose in, strict out | `findIslandStates` resolves `Settings`, `settings`, `settings.island.tsx` and the full path to one manifest, and refuses a name nothing answers to by listing EVERY valid one |
|
|
103
104
|
| The disk check is not in `defineIslandStates` | a declaration evaluates wherever it is imported from, so a rule that reads the filesystem at import time fails on the cwd rather than on the path. |
|
|
104
105
|
| The island EXTENSION is not restated here | `.island.tsx` is `@ultimat3/render`'s `ISLAND_EXTENSION` and `render` is not a dependency of this package. |
|
|
105
|
-
|
|
|
106
|
-
| The scratch directory is lazy and removed | made on the first mount, deleted on `exit`; never at module scope — this module is on the `.` barrel |
|
|
106
|
+
| A mount imports from its own temp DIRECTORY | `island-scratch.ts`: per mount (identical chunks stay two modules); removed on dispose, at the file boundary and in the run's `afterAll` — `exit` never fires under `bun test`. Never a `data:` URL (coverage panics past ~4 kB) |
|
|
107
107
|
| Attaching a node MOVES it | `appendChild`, `insertBefore`, `replaceChild` detach it from its old parent first; `removeChild` clears `parentNode` |
|
|
108
108
|
| Globals install all-or-nothing | `installGlobals` saves DESCRIPTORS, not values — a saved value cannot tell "no such global" from "a global holding `undefined`", and the teardown deleted both |
|
|
109
|
-
| Which command shards | `bun test` is one process on one database
|
|
110
|
-
|
|
109
|
+
| Which command shards | `bun test` is one process on one database — what a scaffolded app's `test` script runs |
|
|
111
110
|
|
|
112
111
|
## The frozen instant and the seed are screened
|
|
113
112
|
|
|
@@ -145,11 +144,12 @@ lives beside the `PageLike` it implements. `cli` imports it over the declared `c
|
|
|
145
144
|
| `e2e-errors.ts` | one constructor per refusal |
|
|
146
145
|
| `e2e-dom-fixture.ts` | a document small enough to hold in a test and real enough to RUN the expressions above |
|
|
147
146
|
| `cdp-browser.ts` | the two doors: `openE2eBrowserIfAvailable()` (undefined when there is no browser) and `openE2eBrowser()` (refuses by name), and the close that undoes both halves |
|
|
148
|
-
| `cdp-launch.ts` | which Chrome,
|
|
149
|
-
| `cdp-
|
|
147
|
+
| `cdp-launch.ts` | which Chrome, its flags, the launch deadline (`LAUNCH_TIMEOUT_MS`, 60 s — a cold start is not a call) and the ONE relaunch (`LAUNCH_ATTEMPTS`) |
|
|
148
|
+
| `cdp-launch-attempt.ts` | one start: its own process group, the first answer over the pipe; on none, a full reap (group killed, profile removed) before the next. ONE `close()`, a promise, always awaited — no sync close |
|
|
149
|
+
| `cdp-connection.ts` | CDP over a transport — the launched Chrome's pipe (`cdp-pipe.ts`), a remote one's `WebSocket` (`cdpConnect`): reply correlation by `id`, event waiters, the per-call deadline; why one went unanswered is `cdp-wire-watch.ts` |
|
|
150
150
|
| `cdp-e2e-session.ts` | the BROWSER half, `E2eSession`: every target auto-attached at browser level and PAUSED until its Network domain is on (a SharedWorker opens its socket at start-up) |
|
|
151
151
|
| `cdp-e2e-page.ts` | one TAB, `E2eTab`: `E2eBrowserPage`'s five methods plus `reload`, `waitFor`, `indexedDbNames`, `close`. `offline()` forwards to the session — the switch is browser-wide |
|
|
152
|
-
| `e2e-app.ts` | `startE2eApp({ root, mode, seed })`: reset + seed + spawn on a THROWAWAY `ULTIMATE_STATE_DIR` and free ports, `/readyz`-gated, spawned through the app's own `@ultimat3/cli` bin (`xBin`); `stop()` removes the directory |
|
|
152
|
+
| `e2e-app.ts` | `startE2eApp({ root, mode, seed })`: reset + seed + spawn on a THROWAWAY `ULTIMATE_STATE_DIR` and free ports a browser opens (`free-port.ts`), `/readyz`-gated, spawned through the app's own `@ultimat3/cli` bin (`xBin`); `stop()` removes the directory |
|
|
153
153
|
| `failure-context.ts` | a FAILED test's message ends with the e2e app's last 40 log lines (≤ 4 kB): `e2e-run.ts` installs `app.log()`, `fixtureTest` and `e2eTest` append it. Never on a pass |
|
|
154
154
|
| `e2e-preload.ts` + `e2e-browser-handle.ts` | the `e2e` step's preload (`@ultimat3/testing/e2e-preload`): with `ULTIMATE_E2E_ROOT` set it spawns the app, opens ONE browser, installs its first tab, and publishes `e2eBrowser()`, `e2eApp()`, `e2eBaseUrl()` |
|
|
155
155
|
| `packages/cli/src/verify-e2e.ts` | `withE2eApp` (cli's): in an app on a machine with Chrome, the `e2e` step runs the suite with that preload; otherwise exactly as before |
|
package/README.md
CHANGED
|
@@ -217,6 +217,14 @@ It registers `page` and nothing else. `budget`, `signIn` and `deploy` keep refus
|
|
|
217
217
|
browser's own network state and a second build id are not things a page port can answer for, and a
|
|
218
218
|
fixture that silently no-opped would make the assertion after it read as proof.
|
|
219
219
|
|
|
220
|
+
**A browser call that is never answered says why.** `X_CDP_TIMEOUT` from the connection carries
|
|
221
|
+
`meta.reading` — `target-gone` (session detached, target destroyed or crashed), `lost-in-transport`
|
|
222
|
+
(`meta.framesDropped` frames arrived unparseable after the call) or `no-answer` — beside
|
|
223
|
+
`framesArrived`, `lastFrames` (the last 8), `transportOpen` and `navigations`; `cause` is the
|
|
224
|
+
one-line reading. `E2eSession.offline(false)` rejects with `X_CDP_CALL_FAILED` when a page that is
|
|
225
|
+
still there refuses to take `navigator.onLine` back: the restore is attempted regardless, and the
|
|
226
|
+
page stays in the set.
|
|
227
|
+
|
|
220
228
|
Each helper prefixes the test name with its type (`job · onboards an org`), which is what
|
|
221
229
|
`bun test --test-name-pattern "job · "` selects — the six lines of `x verify` come from the tests
|
|
222
230
|
themselves, not from a directory convention.
|
|
@@ -271,14 +279,36 @@ failure line reads `publishPost > behaves like an authenticated action > denies
|
|
|
271
279
|
— which subject, and which shared rule. `behavesLike` calls `describe`, so it goes at declaration
|
|
272
280
|
scope, never inside a test body.
|
|
273
281
|
|
|
282
|
+
Two ship, for the ports an app may implement itself — a `JobDriver` and `@ultimat3/ai`'s
|
|
283
|
+
`BudgetStore`:
|
|
284
|
+
|
|
285
|
+
```ts
|
|
286
|
+
import { MemoryBudgetStore } from '@ultimat3/ai';
|
|
287
|
+
import { createMemoryDriver } from '@ultimat3/jobs';
|
|
288
|
+
import { behavesLike, budgetStoreConformance, describe, jobDriverConformance } from '@ultimat3/testing';
|
|
289
|
+
|
|
290
|
+
// The shipped ports pass; put your own driver or store where these are.
|
|
291
|
+
describe('my queue', () => behavesLike(jobDriverConformance, () => createMemoryDriver()));
|
|
292
|
+
describe('my budget store', () => behavesLike(budgetStoreConformance, () => new MemoryBudgetStore()));
|
|
293
|
+
```
|
|
294
|
+
|
|
295
|
+
| Suite | Holds a store to |
|
|
296
|
+
|---|---|
|
|
297
|
+
| `jobDriverConformance` | claim once · ack/nack fenced on the claim · a live key dedupes · `queues: []` refused · dead-letter never re-claimed · a lapsed lease re-claimed, the loser unable to renew · a lease lapsed on the final attempt buried, reported through `onExhausted` once, `failed` under `dropExhausted` |
|
|
298
|
+
| `budgetStoreConformance` | 24 concurrent `take`s on one key never overspend · a refused take spends nothing · negative `add` · `reset(key)` scoped |
|
|
299
|
+
|
|
300
|
+
The subject is called once per test. Each check claims from a queue — or counts under a key — of
|
|
301
|
+
its own, so one shared store is enough.
|
|
302
|
+
|
|
274
303
|
## Parallel databases
|
|
275
304
|
|
|
276
305
|
```ts
|
|
277
306
|
const db = await acquireWorkerDatabase({ adminUrl, migrate });
|
|
278
307
|
```
|
|
279
308
|
|
|
280
|
-
The first worker creates the template under a Postgres advisory lock and migrates it
|
|
281
|
-
worker then clones it copy-on-write
|
|
309
|
+
The first worker creates the template under a Postgres advisory lock and migrates it; every
|
|
310
|
+
worker then clones it copy-on-write, under the same lock — a clone waits only 5 s for a session on
|
|
311
|
+
the template to leave, and the next worker's migration holds one. With no Postgres configured it falls back to PGlite, so
|
|
282
312
|
`bun test` works on a laptop with nothing installed.
|
|
283
313
|
|
|
284
314
|
**The gate shards; a bare `bun test` does not.** `As of 2026-08`:
|
|
@@ -286,9 +316,9 @@ worker then clones it copy-on-write. With no Postgres configured it falls back t
|
|
|
286
316
|
| Command | Processes | Worker ids | Databases |
|
|
287
317
|
|---|---|---|---|
|
|
288
318
|
| `bun test` (what a scaffolded app's `test` script still runs) | 1 | `0` | one |
|
|
289
|
-
| `x verify` (`unit`, `contract`, `job`, `eval`; `live` and `e2e` stay serial) | `clamp(round(cpus * 1.5), 2, 8)` | `0..N-1
|
|
319
|
+
| `x verify` (`unit`, `contract`, `job`, `eval`; `live` and `e2e` stay serial) | `clamp(round(cpus * 1.5), 2, 8)` | `1..N`, from Bun's own `BUN_TEST_WORKER_ID` — `0..N-1` from `ULTIMATE_TEST_WORKER` in its coverage run | N |
|
|
290
320
|
| `bun test --parallel[=N]` | N (default: CPU count) | `1..N`, from Bun's own `BUN_TEST_WORKER_ID` | N |
|
|
291
|
-
| `x test --workers N` | N | `
|
|
321
|
+
| `x test --workers N` | N | `1..N`, from Bun's own `BUN_TEST_WORKER_ID` | N |
|
|
292
322
|
|
|
293
323
|
`ULTIMATE_TEST_WORKER` is read first so a runner-assigned shard always beats the index Bun assigns
|
|
294
324
|
its own `--parallel` worker — measured on Bun 1.3.14, `--parallel` populates `BUN_TEST_WORKER_ID`
|
|
@@ -678,6 +708,20 @@ This package still owns the vocabulary, the expansion and the refusals, and owns
|
|
|
678
708
|
capture, the harness page and the crop are `@ultimat3/cli`'s, because `cli → testing` is the
|
|
679
709
|
declared edge and the reverse is a `bun run boundaries` failure.
|
|
680
710
|
|
|
711
|
+
## Launching Chrome
|
|
712
|
+
|
|
713
|
+
`launchChrome({ executable, timeoutMs, launchTimeoutMs? })` — what the `e2e` step and `x shot` both
|
|
714
|
+
start a browser with. Driven over `--remote-debugging-pipe`; the first answer is readiness.
|
|
715
|
+
|
|
716
|
+
| Export | What it is |
|
|
717
|
+
|---|---|
|
|
718
|
+
| `LAUNCH_TIMEOUT_MS` | `60_000` — the first answer's deadline per start. `launchTimeoutMs` defaults to the larger of this and `timeoutMs`; every later call has `timeoutMs` |
|
|
719
|
+
| `LAUNCH_ATTEMPTS` | `2` — an unanswered start is reaped and started once more on a fresh profile. Never more |
|
|
720
|
+
| `CdpLaunchAttempt` | one unanswered start, in `X_CDP_LAUNCH_FAILED`'s `meta.attempts`: `why` (`deadline` \| `closed`), `waitedMs`, `exitCode` (`null` when killed), `stderr` (its last lines) |
|
|
721
|
+
| `LaunchedBrowser.close()` · `E2eBrowser.close()` | THE close, a promise, always awaited: the process, its whole process group, its profile directory. There is no synchronous close |
|
|
722
|
+
|
|
723
|
+
`As of 2026-10-02`.
|
|
724
|
+
|
|
681
725
|
## The one assertion that waits
|
|
682
726
|
|
|
683
727
|
```ts
|
|
@@ -707,7 +751,7 @@ fails on a page that simply has not painted yet.
|
|
|
707
751
|
## Errors
|
|
708
752
|
|
|
709
753
|
`X_TEST_NETWORK_SEALED` `X_TEST_DB_UNAVAILABLE` `X_TEST_NONDETERMINISTIC` `X_TEST_FIXTURE_UNKNOWN`
|
|
710
|
-
`X_TEST_FACTORY_TRAIT_UNKNOWN` `X_TEST_FACTORY_NOT_PERSISTED` `X_TEST_REGISTRY_LEAK`
|
|
754
|
+
`X_TEST_APP_NOT_BOOTED` `X_TEST_FACTORY_TRAIT_UNKNOWN` `X_TEST_FACTORY_NOT_PERSISTED` `X_TEST_REGISTRY_LEAK`
|
|
711
755
|
`X_TEST_ISLAND_NOT_BUILT` `X_TEST_ISLAND_NO_MOUNT` `X_TEST_ISLAND_SELECTOR_UNSUPPORTED`
|
|
712
756
|
`X_TEST_ISLAND_STATES_EMPTY` `X_TEST_ISLAND_STATE_UNKNOWN`
|
|
713
757
|
`X_TEST_ISLAND_STATES_NOT_PURE` `X_TEST_ISLAND_STATES_MISSING_FILE` `X_TEST_ISLAND_STATES_UNKNOWN`
|
|
@@ -722,6 +766,7 @@ a job boundary the class is gone and the `code` is what survives — match on th
|
|
|
722
766
|
|
|
723
767
|
| Class | Code | Declared in |
|
|
724
768
|
|---|---|---|
|
|
769
|
+
| `AppNotBootedError` | `X_TEST_APP_NOT_BOOTED` | `src/errors.ts` |
|
|
725
770
|
| `FixtureUnavailableError` | `X_TEST_FIXTURE_UNAVAILABLE` | `src/errors.ts` |
|
|
726
771
|
| `NetworkOfflineError` | `X_TEST_NETWORK_OFFLINE` | `src/errors.ts` |
|
|
727
772
|
| `NetworkSealedError` | `X_TEST_NETWORK_SEALED` | `src/errors.ts` |
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@ultimat3/testing",
|
|
3
|
-
"version": "
|
|
3
|
+
"version": "24.0.0",
|
|
4
4
|
"description": "Test harness: cloned template DBs per worker, frozen clock, sealed network, 6 test types",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"type": "module",
|
|
@@ -36,19 +36,19 @@
|
|
|
36
36
|
"test": "bun test"
|
|
37
37
|
},
|
|
38
38
|
"dependencies": {
|
|
39
|
-
"@ultimat3/action": "
|
|
40
|
-
"@ultimat3/cache": "
|
|
41
|
-
"@ultimat3/core": "
|
|
42
|
-
"@ultimat3/db": "
|
|
43
|
-
"@ultimat3/entity": "
|
|
44
|
-
"@ultimat3/http": "
|
|
45
|
-
"@ultimat3/i18n": "
|
|
46
|
-
"@ultimat3/jobs": "
|
|
47
|
-
"@ultimat3/mail": "
|
|
48
|
-
"@ultimat3/policy": "
|
|
49
|
-
"@ultimat3/query": "
|
|
50
|
-
"@ultimat3/realtime": "
|
|
51
|
-
"@ultimat3/render": "
|
|
52
|
-
"@ultimat3/time": "
|
|
39
|
+
"@ultimat3/action": "24.0.0",
|
|
40
|
+
"@ultimat3/cache": "24.0.0",
|
|
41
|
+
"@ultimat3/core": "24.0.0",
|
|
42
|
+
"@ultimat3/db": "24.0.0",
|
|
43
|
+
"@ultimat3/entity": "24.0.0",
|
|
44
|
+
"@ultimat3/http": "24.0.0",
|
|
45
|
+
"@ultimat3/i18n": "24.0.0",
|
|
46
|
+
"@ultimat3/jobs": "24.0.0",
|
|
47
|
+
"@ultimat3/mail": "24.0.0",
|
|
48
|
+
"@ultimat3/policy": "24.0.0",
|
|
49
|
+
"@ultimat3/query": "24.0.0",
|
|
50
|
+
"@ultimat3/realtime": "24.0.0",
|
|
51
|
+
"@ultimat3/render": "24.0.0",
|
|
52
|
+
"@ultimat3/time": "24.0.0"
|
|
53
53
|
}
|
|
54
54
|
}
|
|
@@ -0,0 +1,81 @@
|
|
|
1
|
+
// The `BudgetStore` contract of `@ultimat3/ai` as shared examples, for a store of your own (Redis,
|
|
2
|
+
// SQL): `behavesLike(budgetStoreConformance, () => store)`. The race is the point — `take` is the
|
|
3
|
+
// one atomic step, and a store that awaits between its read and its write overspends under it.
|
|
4
|
+
// Structural, so this package takes no dependency on `ai`: a `BudgetStore` is assignable.
|
|
5
|
+
|
|
6
|
+
import { expect, test } from 'bun:test';
|
|
7
|
+
import type { SharedExamples } from './shared-examples';
|
|
8
|
+
import { sharedExamples } from './shared-examples';
|
|
9
|
+
|
|
10
|
+
type Maybe<T> = T | Promise<T>;
|
|
11
|
+
|
|
12
|
+
/** The four members of `@ultimat3/ai`'s `BudgetStore`, structurally. */
|
|
13
|
+
export interface BudgetStoreLike {
|
|
14
|
+
spent(key: string): Maybe<number>;
|
|
15
|
+
add(key: string, tokens: number): Maybe<void>;
|
|
16
|
+
take(key: string, tokens: number, limit: number): Maybe<{ readonly taken: boolean }>;
|
|
17
|
+
reset(key?: string): Maybe<void>;
|
|
18
|
+
}
|
|
19
|
+
|
|
20
|
+
export interface BudgetStoreCheck {
|
|
21
|
+
readonly name: string;
|
|
22
|
+
run(store: BudgetStoreLike): Promise<void>;
|
|
23
|
+
}
|
|
24
|
+
|
|
25
|
+
/** Concurrent takers, and the ceiling only some of them fit under. */
|
|
26
|
+
const BUDGET_RACE_TAKERS = 24;
|
|
27
|
+
const RACE_LIMIT = 5;
|
|
28
|
+
|
|
29
|
+
let minted = 0;
|
|
30
|
+
const keyOf = (): string => {
|
|
31
|
+
minted += 1;
|
|
32
|
+
return `conformance:${process.pid.toString(36)}:${minted.toString(36)}`;
|
|
33
|
+
};
|
|
34
|
+
|
|
35
|
+
export const BUDGET_STORE_CHECKS: readonly BudgetStoreCheck[] = [
|
|
36
|
+
{
|
|
37
|
+
name: `${BUDGET_RACE_TAKERS} concurrent takes against one key never overspend the limit`,
|
|
38
|
+
async run(store) {
|
|
39
|
+
const key = keyOf();
|
|
40
|
+
const takes = Array.from({ length: BUDGET_RACE_TAKERS }, async () =>
|
|
41
|
+
store.take(key, 1, RACE_LIMIT),
|
|
42
|
+
);
|
|
43
|
+
const taken = (await Promise.all(takes)).filter((each) => each.taken).length;
|
|
44
|
+
expect([taken, await store.spent(key)]).toEqual([RACE_LIMIT, RACE_LIMIT]);
|
|
45
|
+
},
|
|
46
|
+
},
|
|
47
|
+
{
|
|
48
|
+
name: 'a refused take spends nothing, and a take that fits exactly is taken',
|
|
49
|
+
async run(store) {
|
|
50
|
+
const key = keyOf();
|
|
51
|
+
expect((await store.take(key, 3, 4)).taken).toBe(true);
|
|
52
|
+
expect((await store.take(key, 2, 4)).taken).toBe(false);
|
|
53
|
+
expect(await store.spent(key)).toBe(3);
|
|
54
|
+
expect((await store.take(key, 1, 4)).taken).toBe(true);
|
|
55
|
+
expect(await store.spent(key)).toBe(4);
|
|
56
|
+
},
|
|
57
|
+
},
|
|
58
|
+
{
|
|
59
|
+
name: 'add takes a negative credit, and reset(key) clears that key alone',
|
|
60
|
+
async run(store) {
|
|
61
|
+
const [key, other] = [keyOf(), keyOf()];
|
|
62
|
+
await store.add(key, 5);
|
|
63
|
+
await store.add(key, -2);
|
|
64
|
+
await store.add(other, 1);
|
|
65
|
+
expect(await store.spent(key)).toBe(3);
|
|
66
|
+
await store.reset(key);
|
|
67
|
+
expect([await store.spent(key), await store.spent(other)]).toEqual([0, 1]);
|
|
68
|
+
},
|
|
69
|
+
},
|
|
70
|
+
];
|
|
71
|
+
|
|
72
|
+
export const budgetStoreConformance: SharedExamples<Maybe<BudgetStoreLike>> = sharedExamples(
|
|
73
|
+
'a budget store',
|
|
74
|
+
(subject) => {
|
|
75
|
+
for (const check of BUDGET_STORE_CHECKS) {
|
|
76
|
+
test(check.name, async () => {
|
|
77
|
+
await check.run(await subject());
|
|
78
|
+
});
|
|
79
|
+
}
|
|
80
|
+
},
|
|
81
|
+
);
|
package/src/cdp-browser.ts
CHANGED
|
@@ -27,13 +27,12 @@ export interface E2eBrowser {
|
|
|
27
27
|
* drives, on the same launch as `page` — one harness, never a second one beside the driver.
|
|
28
28
|
*/
|
|
29
29
|
readonly session: E2eSession;
|
|
30
|
-
/** Idempotent, and it closes both halves: the CDP socket, then the process and its profile. */
|
|
31
|
-
close(): void;
|
|
32
30
|
/**
|
|
33
|
-
*
|
|
34
|
-
*
|
|
31
|
+
* THE close, idempotent, awaited: the CDP connection, then the process, its process group and its
|
|
32
|
+
* profile (bounded — `CLOSE_GRACE_MS` per step). Unawaited, a process that exits next leaves a
|
|
33
|
+
* Chrome child and a profile directory behind.
|
|
35
34
|
*/
|
|
36
|
-
|
|
35
|
+
close(): Promise<void>;
|
|
37
36
|
}
|
|
38
37
|
|
|
39
38
|
export interface OpenE2eBrowserOptions {
|
|
@@ -60,10 +59,6 @@ const compose = (launched: LaunchedBrowser, session: E2eSession, page: E2eTab):
|
|
|
60
59
|
// the process out from under an open connection makes every in-flight call report "the browser
|
|
61
60
|
// closed the CDP connection", which is true and useless.
|
|
62
61
|
close: () => launched.close(),
|
|
63
|
-
closed: async () => {
|
|
64
|
-
if (launched.closed === undefined) launched.close();
|
|
65
|
-
else await launched.closed();
|
|
66
|
-
},
|
|
67
62
|
});
|
|
68
63
|
|
|
69
64
|
/**
|
|
@@ -107,7 +102,7 @@ async function openLaunched(
|
|
|
107
102
|
});
|
|
108
103
|
return compose(launched, session, await session.newTab());
|
|
109
104
|
} catch (error) {
|
|
110
|
-
launched.close();
|
|
105
|
+
await launched.close();
|
|
111
106
|
throw error;
|
|
112
107
|
}
|
|
113
108
|
}
|
package/src/cdp-connection.ts
CHANGED
|
@@ -13,6 +13,7 @@
|
|
|
13
13
|
|
|
14
14
|
import { assert } from '@ultimat3/core';
|
|
15
15
|
import { CdpCallFailedError, CdpTimeoutError } from './cdp-errors';
|
|
16
|
+
import { wireWatch } from './cdp-wire-watch';
|
|
16
17
|
|
|
17
18
|
/** One CDP result. `unknown` because every payload here is somebody else's JSON. */
|
|
18
19
|
export interface CdpResult {
|
|
@@ -20,8 +21,17 @@ export interface CdpResult {
|
|
|
20
21
|
}
|
|
21
22
|
|
|
22
23
|
export interface CdpConnection {
|
|
23
|
-
/**
|
|
24
|
-
|
|
24
|
+
/**
|
|
25
|
+
* Send one command. `sessionId` targets an attached page rather than the browser itself.
|
|
26
|
+
* `deadlineMs` replaces the connection's per-call deadline for THIS call — the launcher's first
|
|
27
|
+
* call is a cold start, not a call, and is the only caller that passes one.
|
|
28
|
+
*/
|
|
29
|
+
send(
|
|
30
|
+
method: string,
|
|
31
|
+
params?: Record<string, unknown>,
|
|
32
|
+
sessionId?: string,
|
|
33
|
+
deadlineMs?: number,
|
|
34
|
+
): Promise<CdpResult>;
|
|
25
35
|
/**
|
|
26
36
|
* Wait for the next occurrence of one CDP **event**, or for the deadline. Answers `true` when the
|
|
27
37
|
* event arrived and `false` when it did not — it never throws, because every caller has a better
|
|
@@ -145,6 +155,8 @@ export function cdpConnectOver(transport: CdpTransport, timeoutMs: number): CdpC
|
|
|
145
155
|
const listeners = new Map<string, Set<CdpEventListener>>();
|
|
146
156
|
let nextId = 0;
|
|
147
157
|
let closed = false;
|
|
158
|
+
// What a call that is never answered reports instead of a bare "did not answer".
|
|
159
|
+
const watch = wireWatch();
|
|
148
160
|
|
|
149
161
|
// Every in-flight call is settled on close. Without this a suite whose browser died waits out
|
|
150
162
|
// one full deadline per call and reports a timeout, where the true fault is a dead browser.
|
|
@@ -167,9 +179,12 @@ export function cdpConnectOver(transport: CdpTransport, timeoutMs: number): CdpC
|
|
|
167
179
|
frame = JSON.parse(raw) as Record<string, unknown>;
|
|
168
180
|
} catch {
|
|
169
181
|
// An unparseable frame is the browser's, not ours, and there is no call to fail with it:
|
|
170
|
-
// correlation is by `id`, and a frame we cannot read has none.
|
|
182
|
+
// correlation is by `id`, and a frame we cannot read has none. COUNTED, though: the call
|
|
183
|
+
// whose reply it was waits out its deadline, and its timeout says frames were dropped.
|
|
184
|
+
watch.drop();
|
|
171
185
|
return;
|
|
172
186
|
}
|
|
187
|
+
watch.frame(frame);
|
|
173
188
|
const id = frame['id'];
|
|
174
189
|
if (typeof id !== 'number') {
|
|
175
190
|
const method = frame['method'];
|
|
@@ -206,7 +221,7 @@ export function cdpConnectOver(transport: CdpTransport, timeoutMs: number): CdpC
|
|
|
206
221
|
transport.listen({ message, closed: abandon });
|
|
207
222
|
|
|
208
223
|
return {
|
|
209
|
-
send(method, params = {}, sessionId): Promise<CdpResult> {
|
|
224
|
+
send(method, params = {}, sessionId, deadlineMs = timeoutMs): Promise<CdpResult> {
|
|
210
225
|
if (closed) {
|
|
211
226
|
return Promise.reject(
|
|
212
227
|
new CdpCallFailedError({ method, detail: 'the CDP connection is already closed' }),
|
|
@@ -215,10 +230,12 @@ export function cdpConnectOver(transport: CdpTransport, timeoutMs: number): CdpC
|
|
|
215
230
|
nextId += 1;
|
|
216
231
|
const id = nextId;
|
|
217
232
|
return new Promise<CdpResult>((resolve, reject) => {
|
|
233
|
+
const sentAt = watch.mark(sessionId);
|
|
218
234
|
const timer = setTimeout(() => {
|
|
219
235
|
pending.delete(id);
|
|
220
|
-
|
|
221
|
-
|
|
236
|
+
const observed = watch.observe(sentAt, sessionId, !closed);
|
|
237
|
+
reject(new CdpTimeoutError({ method, timeoutMs: deadlineMs, observed }));
|
|
238
|
+
}, deadlineMs);
|
|
222
239
|
pending.set(id, { resolve, reject, timer });
|
|
223
240
|
transport.send(
|
|
224
241
|
JSON.stringify({ id, method, params, ...(sessionId === undefined ? {} : { sessionId }) }),
|
package/src/cdp-e2e-session.ts
CHANGED
|
@@ -155,10 +155,19 @@ export async function cdpE2eSession(options: CdpE2eSessionOptions): Promise<E2eS
|
|
|
155
155
|
}
|
|
156
156
|
// And `navigator.onLine` from a new document's first script (`cdp-offline-script.ts`) — the
|
|
157
157
|
// network condition alone reaches a reloaded page only after its scripts have run.
|
|
158
|
+
// A page that is still THERE and refuses is kept and reported once every page has been asked:
|
|
159
|
+
// dropping it meant every later `offline()` skipped that tab in silence. Only a session the
|
|
160
|
+
// loop above just found gone is dropped — a closed tab refuses every call.
|
|
161
|
+
const refused: unknown[] = [];
|
|
158
162
|
for (const session of [...pageSessions]) {
|
|
163
|
+
if (!sessions.has(session)) {
|
|
164
|
+
pageSessions.delete(session);
|
|
165
|
+
continue;
|
|
166
|
+
}
|
|
159
167
|
const toggled = enabled ? onLineScripts.add(session) : onLineScripts.remove(session);
|
|
160
|
-
await toggled.catch(() =>
|
|
168
|
+
await toggled.catch((error: unknown) => refused.push(error));
|
|
161
169
|
}
|
|
170
|
+
if (refused.length > 0) throw refused[0];
|
|
162
171
|
};
|
|
163
172
|
|
|
164
173
|
const attached = async (targetId: string): Promise<string> => {
|
package/src/cdp-errors.ts
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
// out of a BROWSER or off a spawned process's stderr, so every one is rendered rather than
|
|
3
3
|
// interpolated — the rule `e2e-errors.ts` already states.
|
|
4
4
|
|
|
5
|
-
import { renderCauseValue, UltimateError } from '@ultimat3/core';
|
|
5
|
+
import { isFixShellSafe, renderCauseValue, UltimateError } from '@ultimat3/core';
|
|
6
6
|
// Bare: the titles these constructors' codes render with are registered there.
|
|
7
7
|
import './e2e-error-codes';
|
|
8
8
|
|
|
@@ -20,13 +20,50 @@ export class CdpBrowserMissingError extends UltimateError {
|
|
|
20
20
|
}
|
|
21
21
|
}
|
|
22
22
|
|
|
23
|
-
/**
|
|
23
|
+
/** One start of the browser that ended without an answer — what `cdp-launch-attempt.ts` reports. */
|
|
24
|
+
export interface CdpLaunchAttempt {
|
|
25
|
+
/** `deadline`: alive and silent for `waitedMs`. `closed`: its pipe ended before any answer. */
|
|
26
|
+
readonly why: 'deadline' | 'closed';
|
|
27
|
+
readonly waitedMs: number;
|
|
28
|
+
/** The code it exited with on its own, or `null` when the launcher had to kill it. */
|
|
29
|
+
readonly exitCode: number | null;
|
|
30
|
+
/** The last lines it wrote to stderr; empty when it wrote none. */
|
|
31
|
+
readonly stderr: string;
|
|
32
|
+
}
|
|
33
|
+
|
|
34
|
+
const HAND_FLAGS =
|
|
35
|
+
'--headless=new --no-sandbox --disable-dev-shm-usage --disable-gpu --remote-debugging-port=0 about:blank';
|
|
36
|
+
|
|
37
|
+
const attemptText = (attempt: CdpLaunchAttempt, index: number): string => {
|
|
38
|
+
const ended =
|
|
39
|
+
attempt.why === 'deadline'
|
|
40
|
+
? `no answer inside ${String(attempt.waitedMs)}ms, so it was killed`
|
|
41
|
+
: attempt.exitCode === null
|
|
42
|
+
? `closed its DevTools pipe after ${String(attempt.waitedMs)}ms without answering, and was killed`
|
|
43
|
+
: `exited with code ${String(attempt.exitCode)} after ${String(attempt.waitedMs)}ms without answering`;
|
|
44
|
+
const said = attempt.stderr === '' ? 'it printed nothing' : renderCauseValue(attempt.stderr);
|
|
45
|
+
return `launch ${String(index + 1)}: ${ended}; stderr: ${said}`;
|
|
46
|
+
};
|
|
47
|
+
|
|
48
|
+
/**
|
|
49
|
+
* The binary ran and never answered a DevTools call — a crash, a bad flag, a sandbox refusal, or a
|
|
50
|
+
* machine too loaded to start it inside the launch deadline. Every attempt is in the cause with its
|
|
51
|
+
* own stderr, because Chrome's start-up noise reads the same in a healthy browser and a dead one:
|
|
52
|
+
* what tells them apart is whether it EXITED or was still starting when the deadline passed.
|
|
53
|
+
*/
|
|
24
54
|
export class CdpLaunchFailedError extends UltimateError {
|
|
25
|
-
constructor(input: {
|
|
55
|
+
constructor(input: {
|
|
56
|
+
readonly executable: string;
|
|
57
|
+
readonly attempts: readonly CdpLaunchAttempt[];
|
|
58
|
+
}) {
|
|
59
|
+
// Run by hand, a healthy Chrome prints `DevTools listening on ws://…` and a broken one prints
|
|
60
|
+
// why. A path that is not shell-inert is the operator's own CHROME_PATH — every candidate is.
|
|
61
|
+
const binary = isFixShellSafe(input.executable) ? input.executable : '"$CHROME_PATH"';
|
|
26
62
|
super({
|
|
27
63
|
code: 'X_CDP_LAUNCH_FAILED',
|
|
28
|
-
cause: `${renderCauseValue(input.executable)} did not announce a DevTools endpoint: ${
|
|
29
|
-
fix:
|
|
64
|
+
cause: `${renderCauseValue(input.executable)} did not announce a DevTools endpoint: it answered no DevTools call in ${String(input.attempts.length)} ${input.attempts.length === 1 ? 'launch' : 'launches, each on its own fresh profile'} — ${input.attempts.map(attemptText).join(' · ')}`,
|
|
65
|
+
fix: `${binary} ${HAND_FLAGS}`,
|
|
66
|
+
meta: { executable: input.executable, attempts: input.attempts },
|
|
30
67
|
});
|
|
31
68
|
}
|
|
32
69
|
}
|
|
@@ -42,17 +79,85 @@ export class CdpCallFailedError extends UltimateError {
|
|
|
42
79
|
}
|
|
43
80
|
}
|
|
44
81
|
|
|
82
|
+
/** How a call's target went away: its session detached, its target was destroyed, or it crashed. */
|
|
83
|
+
export type CdpTargetGone = 'detached' | 'destroyed' | 'crashed';
|
|
84
|
+
|
|
85
|
+
/** What the wire did between a call being sent and its deadline (`cdp-wire-watch.ts`). */
|
|
86
|
+
export interface CdpTimeoutObservation {
|
|
87
|
+
/** The session the call was sent on; `null` for a browser-level call. */
|
|
88
|
+
readonly sessionId: string | null;
|
|
89
|
+
/** Frames that arrived and parsed after the call was sent — replies to other calls, and events. */
|
|
90
|
+
readonly framesArrived: number;
|
|
91
|
+
/** The last of those, oldest first, bounded: `reply <id>`, or `<method> @<session>`. */
|
|
92
|
+
readonly lastFrames: readonly string[];
|
|
93
|
+
/** Frames that arrived after the call and could not be parsed. One of them may be its reply. */
|
|
94
|
+
readonly framesDropped: number;
|
|
95
|
+
readonly transportOpen: boolean;
|
|
96
|
+
/** `detached`, `destroyed` or `crashed` when the call's target went away; `null` while it is there. */
|
|
97
|
+
readonly targetGone: CdpTargetGone | null;
|
|
98
|
+
/** Main-frame navigations on the call's session since it was sent. */
|
|
99
|
+
readonly navigations: number;
|
|
100
|
+
}
|
|
101
|
+
|
|
102
|
+
/** The one-word verdict a reader — or a script over `--json` — branches on. */
|
|
103
|
+
export type CdpTimeoutReading = 'target-gone' | 'lost-in-transport' | 'no-answer';
|
|
104
|
+
|
|
105
|
+
export const readTimeout = (seen: CdpTimeoutObservation): CdpTimeoutReading =>
|
|
106
|
+
seen.targetGone !== null
|
|
107
|
+
? 'target-gone'
|
|
108
|
+
: seen.framesDropped > 0
|
|
109
|
+
? 'lost-in-transport'
|
|
110
|
+
: 'no-answer';
|
|
111
|
+
|
|
112
|
+
const READING: Record<CdpTimeoutReading, (seen: CdpTimeoutObservation) => string> = {
|
|
113
|
+
'target-gone': (seen) =>
|
|
114
|
+
`its target is gone (${String(seen.targetGone)}), so nothing was left to answer`,
|
|
115
|
+
'lost-in-transport': (seen) =>
|
|
116
|
+
`${String(seen.framesDropped)} frame(s) arrived unparseable after it was sent and were dropped, so its reply may have been lost in transport (${String(seen.framesArrived)} other frame(s) arrived intact)`,
|
|
117
|
+
'no-answer': (seen) => {
|
|
118
|
+
if (seen.framesArrived === 0) {
|
|
119
|
+
return `nothing at all arrived after the call was sent, on a transport that is ${seen.transportOpen ? 'still open' : 'closed'} — the whole browser went silent`;
|
|
120
|
+
}
|
|
121
|
+
const moved =
|
|
122
|
+
seen.navigations === 0
|
|
123
|
+
? ''
|
|
124
|
+
: `; the page navigated ${String(seen.navigations)} time(s) since the call`;
|
|
125
|
+
return `the target did not answer: ${String(seen.framesArrived)} other frame(s) arrived intact after the call, the last ${renderCauseValue(seen.lastFrames)}${moved}`;
|
|
126
|
+
},
|
|
127
|
+
};
|
|
128
|
+
|
|
45
129
|
/**
|
|
46
130
|
* A call that never answered. Its own code rather than `X_TIMEOUT`, because the actionable half is
|
|
47
131
|
* WHICH call: a hung `Page.navigate` is an app that never finishes responding, and a hung
|
|
48
132
|
* `Runtime.evaluate` is an expression that never settles.
|
|
133
|
+
*
|
|
134
|
+
* `observed` is what the connection saw meanwhile, and it is what tells three failures apart that
|
|
135
|
+
* all read "did not answer": the target went away, the reply was dropped on the wire, or the target
|
|
136
|
+
* was there and silent. A deadline the connection did not keep (`waitFor`, the handshake) has none.
|
|
49
137
|
*/
|
|
50
138
|
export class CdpTimeoutError extends UltimateError {
|
|
51
|
-
constructor(input: {
|
|
139
|
+
constructor(input: {
|
|
140
|
+
readonly method: string;
|
|
141
|
+
readonly timeoutMs: number;
|
|
142
|
+
readonly observed?: CdpTimeoutObservation | undefined;
|
|
143
|
+
}) {
|
|
144
|
+
const { observed } = input;
|
|
145
|
+
const base = `${renderCauseValue(input.method)} did not answer inside ${String(input.timeoutMs)}ms`;
|
|
146
|
+
const reading = observed === undefined ? undefined : readTimeout(observed);
|
|
52
147
|
super({
|
|
53
148
|
code: 'X_CDP_TIMEOUT',
|
|
54
|
-
cause:
|
|
55
|
-
|
|
149
|
+
cause:
|
|
150
|
+
observed === undefined || reading === undefined
|
|
151
|
+
? base
|
|
152
|
+
: `${base} — ${READING[reading](observed)}`,
|
|
153
|
+
fix:
|
|
154
|
+
reading === undefined || reading === 'no-answer'
|
|
155
|
+
? 'raise timeoutMs on installE2eDriver({ timeoutMs }), or find the request the page is still waiting on — a navigation that never settles is an app that never finishes its response'
|
|
156
|
+
: // `--json`: the rerun's `meta.reading` and `meta.framesDropped` are only printed there.
|
|
157
|
+
'x test e2e --json',
|
|
158
|
+
...(observed === undefined
|
|
159
|
+
? {}
|
|
160
|
+
: { meta: { method: input.method, timeoutMs: input.timeoutMs, reading, ...observed } }),
|
|
56
161
|
});
|
|
57
162
|
}
|
|
58
163
|
}
|