e2e 0.17.0 → 0.18.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +10 -0
- package/dist/.build.json +1 -1
- package/dist/agent/ai-sdk.js +5 -4
- package/dist/agent/error.js +3 -3
- package/dist/agent/invocation.js +3 -4
- package/dist/agent/model/tool-choice.d.ts +9 -0
- package/dist/agent/model/tool-choice.js +14 -0
- package/dist/agent/observation-feed.js +3 -4
- package/dist/agent/phases.d.ts +7 -1
- package/dist/agent/phases.js +12 -3
- package/dist/agent/prompts.js +3 -0
- package/dist/agent/replay.js +18 -15
- package/dist/agent/step-cache.d.ts +2 -1
- package/dist/agent/step-cache.js +9 -5
- package/dist/agent/tool-loop.js +92 -5
- package/dist/cache/trace.js +6 -2
- package/dist/cli/bin.js +2 -2
- package/dist/cli/index.js +35 -24
- package/dist/cli/init/mcp-config.d.ts +2 -1
- package/dist/cli/init/mcp-config.js +8 -3
- package/dist/cli/init/sibling-versions.json +1 -1
- package/dist/cli/mcp.d.ts +3 -2
- package/dist/cli/mcp.js +52 -16
- package/dist/cli/run-output.d.ts +17 -0
- package/dist/cli/run-output.js +45 -0
- package/dist/collect/collect.js +8 -4
- package/dist/collect/registry.js +27 -16
- package/dist/config/compiled-files.d.ts +36 -0
- package/dist/config/compiled-files.js +50 -0
- package/dist/config/diagnose.js +21 -19
- package/dist/config/esm-hooks.d.ts +39 -44
- package/dist/config/esm-hooks.js +238 -63
- package/dist/config/load.d.ts +1 -7
- package/dist/config/load.js +23 -66
- package/dist/config/register.d.ts +8 -0
- package/dist/config/register.js +9 -0
- package/dist/config/specifiers.d.ts +19 -0
- package/dist/config/specifiers.js +33 -0
- package/dist/config/tsconfig.d.ts +56 -0
- package/dist/config/tsconfig.js +115 -0
- package/dist/config/typescript.d.ts +19 -0
- package/dist/config/typescript.js +253 -0
- package/dist/engine/contract.d.ts +1 -0
- package/dist/engine/index.d.ts +2 -0
- package/dist/engine/session.js +2 -2
- package/dist/expect/async.js +2 -1
- package/dist/expect/poll.js +2 -2
- package/dist/internal/errors.d.ts +10 -3
- package/dist/internal/errors.js +23 -8
- package/dist/internal/node-version.d.ts +16 -7
- package/dist/internal/node-version.js +50 -15
- package/dist/internal/package-manager.js +1 -1
- package/dist/internal/realm-slot.d.ts +5 -4
- package/dist/internal/redact.d.ts +2 -0
- package/dist/internal/redact.js +4 -0
- package/dist/internal/time.d.ts +21 -5
- package/dist/internal/time.js +35 -3
- package/dist/locator/engine.js +47 -11
- package/dist/locator/screen.js +31 -5
- package/dist/mcp/output.d.ts +59 -0
- package/dist/mcp/output.js +129 -0
- package/dist/mcp/server.d.ts +4 -0
- package/dist/mcp/server.js +9 -12
- package/dist/mcp/session.d.ts +8 -1
- package/dist/mcp/session.js +55 -14
- package/dist/mcp/tools.js +7 -5
- package/dist/report/list.d.ts +11 -0
- package/dist/report/list.js +23 -6
- package/dist/run/artifacts.d.ts +4 -4
- package/dist/run/artifacts.js +3 -2
- package/dist/run/execute.d.ts +25 -5
- package/dist/run/execute.js +44 -20
- package/dist/run/failure-evidence.js +6 -2
- package/dist/run/in-process.js +51 -10
- package/dist/run/process-output.d.ts +73 -0
- package/dist/run/process-output.js +130 -0
- package/dist/run/provision.d.ts +2 -0
- package/dist/run/provision.js +1 -0
- package/dist/run/records.d.ts +4 -4
- package/dist/run/retry.d.ts +7 -0
- package/dist/run/retry.js +27 -11
- package/dist/run/runner.d.ts +8 -0
- package/dist/run/runner.js +32 -9
- package/dist/run/scheduler.js +255 -19
- package/dist/run/secrecy.d.ts +22 -19
- package/dist/run/secrecy.js +22 -18
- package/dist/run/serial.d.ts +73 -2
- package/dist/run/serial.js +239 -58
- package/dist/run/standalone.js +1 -1
- package/dist/run/trace-redaction.d.ts +5 -6
- package/dist/run/trace-redaction.js +5 -6
- package/dist/run/worker/entry.js +35 -14
- package/dist/run/worker/handle.js +10 -5
- package/dist/run/worker/protocol.d.ts +83 -3
- package/dist/run/worker/session.js +28 -0
- package/dist/telemetry/environment.d.ts +2 -1
- package/dist/telemetry/environment.js +12 -2
- package/dist/telemetry/project.d.ts +5 -6
- package/dist/telemetry/project.js +6 -11
- package/dist/telemetry/store.d.ts +5 -5
- package/dist/telemetry/store.js +5 -7
- package/dist/telemetry/telemetry.d.ts +2 -3
- package/dist/telemetry/telemetry.js +2 -4
- package/dist/types.d.ts +6 -3
- package/docs/agent-steps.mdx +1 -0
- package/docs/agents.mdx +1 -0
- package/docs/api-testing.mdx +2 -1
- package/docs/authentication.mdx +3 -2
- package/docs/browser.mdx +7 -6
- package/docs/bug-bash.mdx +2 -1
- package/docs/cache.mdx +2 -1
- package/docs/ci.mdx +3 -2
- package/docs/coding-agents.mdx +2 -1
- package/docs/core-concepts.mdx +208 -0
- package/docs/debugging.mdx +19 -9
- package/docs/decision-models.mdx +142 -0
- package/docs/example-projects.mdx +32 -0
- package/docs/executors.mdx +3 -0
- package/docs/explore.mdx +1 -0
- package/docs/github.mdx +2 -1
- package/docs/index.mdx +9 -5
- package/docs/integrations/eas.mdx +1 -0
- package/docs/integrations/index.mdx +1 -0
- package/docs/integrations/kernel.mdx +2 -1
- package/docs/migrate/cypress.mdx +26 -3
- package/docs/migrate/detox.mdx +24 -1
- package/docs/migrate/maestro.mdx +28 -4
- package/docs/migrate/playwright.mdx +27 -4
- package/docs/migrate/selenium.mdx +27 -3
- package/docs/mobile-ci.mdx +6 -5
- package/docs/mobile.mdx +79 -8
- package/docs/models.mdx +7 -2
- package/docs/quickstart.mdx +26 -15
- package/docs/reference/agent.mdx +13 -3
- package/docs/reference/app.mdx +1 -0
- package/docs/reference/cli.mdx +11 -5
- package/docs/reference/config.mdx +42 -6
- package/docs/reference/engine.mdx +22 -10
- package/docs/reference/environment.mdx +2 -1
- package/docs/reference/errors.mdx +11 -8
- package/docs/reference/expect.mdx +2 -1
- package/docs/reference/mcp.mdx +21 -8
- package/docs/reference/mobile.mdx +1 -0
- package/docs/reference/reporters.mdx +1 -0
- package/docs/reference/screen.mdx +19 -9
- package/docs/reference/test.mdx +21 -1
- package/docs/reference/web.mdx +8 -2
- package/docs/security.mdx +33 -27
- package/docs/subscriptions.mdx +1 -0
- package/docs/telemetry.mdx +8 -5
- package/docs/tools.mdx +1 -0
- package/docs/{starting-your-app.mdx → web.mdx} +157 -11
- package/docs/writing-an-engine.mdx +2 -1
- package/package.json +11 -7
- package/skills/e2e/SKILL.md +3 -1
- package/skills/e2e/references/agent.md +11 -0
- package/skills/e2e/references/debugging.md +1 -1
- package/skills/e2e/references/mcp.md +8 -3
- package/skills/e2e/references/running.md +2 -1
- package/skills/e2e/references/setup.md +9 -1
- package/skills/e2e/references/writing-tests.md +1 -1
- package/docs/assertions.mdx +0 -94
- package/docs/goals.mdx +0 -176
- package/docs/locators.mdx +0 -64
- package/docs/physical-devices.mdx +0 -76
- package/docs/viewports.mdx +0 -146
|
@@ -12,7 +12,8 @@ import type { CliOverrides, PortAssignments, ResolvedTarget } from '../../config
|
|
|
12
12
|
import type { AiTraceSnapshot } from '../../internal/ai-trace.ts';
|
|
13
13
|
import type { DebugSnapshot } from '../../internal/debug.ts';
|
|
14
14
|
import type { SerializedError } from '../../internal/errors.ts';
|
|
15
|
-
import type { ResultRecord, RunError, SerialGroupRecord, WireResultRecord } from '../records.ts';
|
|
15
|
+
import type { AttemptRecord, ResultRecord, RunError, SerialGroupRecord, SerialMemberRecord, WireResultRecord } from '../records.ts';
|
|
16
|
+
import type { SerialAttemptRun, SerialAttemptStart } from '../serial.ts';
|
|
16
17
|
import type { StepProgress } from '../steps.ts';
|
|
17
18
|
/** One runnable pair on the wire; the worker resolves the test function. */
|
|
18
19
|
export interface WirePair {
|
|
@@ -107,7 +108,15 @@ export interface ShutdownMessage {
|
|
|
107
108
|
export interface TerminateMessage {
|
|
108
109
|
readonly type: 'terminate';
|
|
109
110
|
}
|
|
110
|
-
|
|
111
|
+
/**
|
|
112
|
+
* Asks the worker to answer with `pong` from its event loop. The scheduler
|
|
113
|
+
* sends it only once an attempt is past its test timeout, to tell a worker
|
|
114
|
+
* still tearing the attempt down from one whose event loop is blocked.
|
|
115
|
+
*/
|
|
116
|
+
export interface PingMessage {
|
|
117
|
+
readonly type: 'ping';
|
|
118
|
+
}
|
|
119
|
+
export type MainToWorker = RunUnitMessage | InterruptMessage | ShutdownMessage | TerminateMessage | PingMessage;
|
|
111
120
|
export interface ReadyMessage {
|
|
112
121
|
readonly type: 'ready';
|
|
113
122
|
}
|
|
@@ -166,6 +175,35 @@ export interface NoticeMessage {
|
|
|
166
175
|
readonly type: 'notice';
|
|
167
176
|
readonly message: string;
|
|
168
177
|
}
|
|
178
|
+
/**
|
|
179
|
+
* An attempt's test timeout started: its `beforeEach` hooks and body run
|
|
180
|
+
* now. The worker enforces the timeout with its own timers, which a body
|
|
181
|
+
* that blocks the event loop never lets fire, so the scheduler keeps the
|
|
182
|
+
* deadline too; see `SchedulerWorker.watch`.
|
|
183
|
+
*/
|
|
184
|
+
export interface AttemptDeadlineMessage {
|
|
185
|
+
readonly type: 'attempt-deadline';
|
|
186
|
+
readonly testId: string;
|
|
187
|
+
readonly agent: string;
|
|
188
|
+
readonly repeat: number;
|
|
189
|
+
/** The attempt's index among the pair's attempts. */
|
|
190
|
+
readonly attempt: number;
|
|
191
|
+
/** The attempt's record id and start, which a watchdog kill reports it with. */
|
|
192
|
+
readonly attemptId: string;
|
|
193
|
+
readonly startedAt: string;
|
|
194
|
+
/** The test's resolved `timeout`. */
|
|
195
|
+
readonly timeoutMs: number;
|
|
196
|
+
/** How long past the timeout the worker may go without answering a `ping`: the cleanup budget. */
|
|
197
|
+
readonly graceMs: number;
|
|
198
|
+
}
|
|
199
|
+
/** The attempt the last `attempt-deadline` announced has ended, verdict and cleanup included. */
|
|
200
|
+
export interface AttemptEndMessage {
|
|
201
|
+
readonly type: 'attempt-end';
|
|
202
|
+
}
|
|
203
|
+
/** The answer to `ping`. */
|
|
204
|
+
export interface PongMessage {
|
|
205
|
+
readonly type: 'pong';
|
|
206
|
+
}
|
|
169
207
|
export interface ResultMessage {
|
|
170
208
|
readonly type: 'result';
|
|
171
209
|
readonly result: WireResultRecord;
|
|
@@ -174,6 +212,48 @@ export interface SerialGroupMessage {
|
|
|
174
212
|
readonly type: 'serial-group';
|
|
175
213
|
readonly group: SerialGroupRecord;
|
|
176
214
|
}
|
|
215
|
+
/**
|
|
216
|
+
* An attempt of an ordinary or setup pair begins, its realm and `beforeAll`
|
|
217
|
+
* hooks included. With `attempt`, it tells a crash during an attempt from
|
|
218
|
+
* one between attempts (an `afterAll` after the last one): only the first
|
|
219
|
+
* is charged a new attempt, with this index.
|
|
220
|
+
*/
|
|
221
|
+
export interface AttemptStartMessage {
|
|
222
|
+
readonly type: 'attempt-start';
|
|
223
|
+
readonly testId: string;
|
|
224
|
+
readonly agent: string;
|
|
225
|
+
readonly repeat: number;
|
|
226
|
+
readonly index: number;
|
|
227
|
+
}
|
|
228
|
+
/**
|
|
229
|
+
* One finished attempt of an ordinary or setup pair. A pair's result waits
|
|
230
|
+
* for its last attempt, so each attempt also goes out as it ends: a worker
|
|
231
|
+
* that dies during a retry must not take the attempts before it along. The
|
|
232
|
+
* result still carries every attempt; the scheduler keeps these only until
|
|
233
|
+
* it arrives.
|
|
234
|
+
*/
|
|
235
|
+
export interface AttemptMessage {
|
|
236
|
+
readonly type: 'attempt';
|
|
237
|
+
readonly testId: string;
|
|
238
|
+
readonly agent: string;
|
|
239
|
+
readonly repeat: number;
|
|
240
|
+
readonly attempt: AttemptRecord;
|
|
241
|
+
}
|
|
242
|
+
/** One member that finished in the serial group attempt running now; see `AttemptMessage`. */
|
|
243
|
+
export interface SerialMemberMessage {
|
|
244
|
+
readonly type: 'serial-member';
|
|
245
|
+
/** The group's report id (`SerialGroupRecord.id`). */
|
|
246
|
+
readonly groupId: string;
|
|
247
|
+
readonly attempt: SerialAttemptStart;
|
|
248
|
+
readonly member: SerialMemberRecord;
|
|
249
|
+
}
|
|
250
|
+
/** One finished serial group attempt; see `AttemptMessage`. */
|
|
251
|
+
export interface SerialAttemptMessage {
|
|
252
|
+
readonly type: 'serial-attempt';
|
|
253
|
+
/** The group's report id (`SerialGroupRecord.id`). */
|
|
254
|
+
readonly groupId: string;
|
|
255
|
+
readonly run: SerialAttemptRun;
|
|
256
|
+
}
|
|
177
257
|
export interface UnitDoneMessage {
|
|
178
258
|
readonly type: 'unit-done';
|
|
179
259
|
readonly unitId: string;
|
|
@@ -208,6 +288,6 @@ export interface RunAbortMessage {
|
|
|
208
288
|
readonly type: 'run-abort';
|
|
209
289
|
readonly error: SerializedError;
|
|
210
290
|
}
|
|
211
|
-
export type WorkerToMain = ReadyMessage | PairStartMessage | ProgressMessage | OutputMessage | NoticeMessage | ResultMessage | SerialGroupMessage | UnitDoneMessage | ShutdownDoneMessage | FatalMessage | RunAbortMessage;
|
|
291
|
+
export type WorkerToMain = ReadyMessage | PairStartMessage | ProgressMessage | AttemptDeadlineMessage | AttemptEndMessage | PongMessage | OutputMessage | NoticeMessage | ResultMessage | SerialGroupMessage | AttemptStartMessage | AttemptMessage | SerialMemberMessage | SerialAttemptMessage | UnitDoneMessage | ShutdownDoneMessage | FatalMessage | RunAbortMessage;
|
|
212
292
|
/** Reattaches the scheduler's resolved target to a wire result. */
|
|
213
293
|
export declare function decodeResult(wire: WireResultRecord, target: ResolvedTarget): ResultRecord;
|
|
@@ -4,6 +4,7 @@
|
|
|
4
4
|
* in-process) drive this class, so unit execution, interrupt handling, and
|
|
5
5
|
* run-error forwarding exist exactly once.
|
|
6
6
|
*/
|
|
7
|
+
import inspector from 'node:inspector';
|
|
7
8
|
import { DebugTrace } from '../../internal/debug.js';
|
|
8
9
|
import { TargetExecutor } from '../execute.js';
|
|
9
10
|
import { encodeResult } from '../records.js';
|
|
@@ -66,6 +67,10 @@ export class TargetWorker {
|
|
|
66
67
|
this.countFailure();
|
|
67
68
|
},
|
|
68
69
|
onSerialGroup: (group) => this.host.emit({ type: 'serial-group', group }),
|
|
70
|
+
onAttemptStart: (pair, index) => this.host.emit({ type: 'attempt-start', testId: pair.test.id, agent: pair.agent, repeat: pair.repeat, index }),
|
|
71
|
+
onAttempt: (pair, attempt) => this.host.emit({ type: 'attempt', testId: pair.test.id, agent: pair.agent, repeat: pair.repeat, attempt }),
|
|
72
|
+
onSerialMember: (groupId, attempt, member) => this.host.emit({ type: 'serial-member', groupId, attempt, member }),
|
|
73
|
+
onSerialAttempt: (groupId, run) => this.host.emit({ type: 'serial-attempt', groupId, run }),
|
|
69
74
|
onPairStart: (pair) => {
|
|
70
75
|
this.inFlight = { testId: pair.test.id, agent: pair.agent, repeat: pair.repeat };
|
|
71
76
|
this.host.emit({
|
|
@@ -78,6 +83,23 @@ export class TargetWorker {
|
|
|
78
83
|
serialId: pair.test.serialId,
|
|
79
84
|
});
|
|
80
85
|
},
|
|
86
|
+
onAttemptDeadline: (pair, attempt) => {
|
|
87
|
+
// A worker paused at a breakpoint looks blocked; with a debugger attached nothing watches its attempts.
|
|
88
|
+
if (inspector.url() !== undefined)
|
|
89
|
+
return;
|
|
90
|
+
this.host.emit({
|
|
91
|
+
type: 'attempt-deadline',
|
|
92
|
+
testId: pair.test.id,
|
|
93
|
+
agent: pair.agent,
|
|
94
|
+
repeat: pair.repeat,
|
|
95
|
+
attempt: attempt.index,
|
|
96
|
+
attemptId: attempt.id,
|
|
97
|
+
startedAt: attempt.startedAt,
|
|
98
|
+
timeoutMs: pair.options.timeout,
|
|
99
|
+
graceMs: deps.config.cleanupTimeout,
|
|
100
|
+
});
|
|
101
|
+
},
|
|
102
|
+
onAttemptEnd: () => this.host.emit({ type: 'attempt-end' }),
|
|
81
103
|
onProgress: (pair, progress) => this.host.emit({ type: 'progress', testId: pair.test.id, agent: pair.agent, repeat: pair.repeat, progress }),
|
|
82
104
|
onNotice: (message) => this.host.emit({ type: 'notice', message }),
|
|
83
105
|
onRunAbort: (runError) => {
|
|
@@ -113,6 +135,12 @@ export class TargetWorker {
|
|
|
113
135
|
case 'terminate':
|
|
114
136
|
this.terminate();
|
|
115
137
|
return;
|
|
138
|
+
case 'ping':
|
|
139
|
+
// Answered now, not queued behind the running unit: the answer is the proof the event loop turns.
|
|
140
|
+
this.host.emit({ type: 'pong' });
|
|
141
|
+
return;
|
|
142
|
+
default:
|
|
143
|
+
message;
|
|
116
144
|
}
|
|
117
145
|
}
|
|
118
146
|
/**
|
|
@@ -42,7 +42,8 @@ export declare function fleetName(env: NodeJS.ProcessEnv): string | null;
|
|
|
42
42
|
* The id the environment states for itself, standing in for a machine that
|
|
43
43
|
* has no preferences file: a fleet by its name, else the CI vendor when `CI`
|
|
44
44
|
* is set. Undefined on a person's machine, where the random per-machine id
|
|
45
|
-
* is the unit
|
|
45
|
+
* is the unit: a vendor marker without `CI` is a shell on a runner, and `CI`
|
|
46
|
+
* with a coding agent but no vendor is the agent's shell on a laptop.
|
|
46
47
|
*/
|
|
47
48
|
export declare function statedIdentity(env: NodeJS.ProcessEnv): string | undefined;
|
|
48
49
|
export declare function collectEnvironment(options: {
|
|
@@ -30,6 +30,10 @@ const CI_VENDORS = [
|
|
|
30
30
|
['SEMAPHORE', 'semaphore'],
|
|
31
31
|
['APPVEYOR', 'appveyor'],
|
|
32
32
|
['WOODPECKER', 'woodpecker'],
|
|
33
|
+
['EAS_BUILD', 'eas'],
|
|
34
|
+
['BITRISE_IO', 'bitrise'],
|
|
35
|
+
['CM_BUILD_ID', 'codemagic'],
|
|
36
|
+
['CI_XCODE_CLOUD', 'xcode-cloud'],
|
|
33
37
|
];
|
|
34
38
|
/**
|
|
35
39
|
* Environment variables coding agents set in the shells they spawn. The
|
|
@@ -99,13 +103,19 @@ export function fleetName(env) {
|
|
|
99
103
|
* The id the environment states for itself, standing in for a machine that
|
|
100
104
|
* has no preferences file: a fleet by its name, else the CI vendor when `CI`
|
|
101
105
|
* is set. Undefined on a person's machine, where the random per-machine id
|
|
102
|
-
* is the unit
|
|
106
|
+
* is the unit: a vendor marker without `CI` is a shell on a runner, and `CI`
|
|
107
|
+
* with a coding agent but no vendor is the agent's shell on a laptop.
|
|
103
108
|
*/
|
|
104
109
|
export function statedIdentity(env) {
|
|
105
110
|
const fleet = fleetName(env);
|
|
106
111
|
if (fleet !== null)
|
|
107
112
|
return `fleet:${fleet}`;
|
|
108
|
-
|
|
113
|
+
if (!isCiMode(env))
|
|
114
|
+
return undefined;
|
|
115
|
+
const vendor = firstMarker(env, CI_VENDORS);
|
|
116
|
+
if (vendor === undefined && firstMarker(env, CODING_AGENTS) !== undefined)
|
|
117
|
+
return undefined;
|
|
118
|
+
return `ci:${vendor ?? 'unknown'}`;
|
|
109
119
|
}
|
|
110
120
|
/** The runtime executing this process: Bun and Deno each announce themselves in `process.versions`. */
|
|
111
121
|
function runtime(versions) {
|
|
@@ -8,10 +8,9 @@
|
|
|
8
8
|
* commit, which every clone shares and no one can produce without the
|
|
9
9
|
* repository itself; a commit hash carries no name, path, or remote. A
|
|
10
10
|
* shallow clone has no root to offer: its boundary commits pose as roots and
|
|
11
|
-
* move with every fetch, so it
|
|
12
|
-
*
|
|
13
|
-
*
|
|
14
|
-
* In CI there is no salt, so a runner without a full history has no id: its
|
|
15
|
-
* working directory is not a project.
|
|
11
|
+
* move with every fetch, so it has no id, like a directory outside git: a
|
|
12
|
+
* path is the same project on one machine only, so it cannot be told apart
|
|
13
|
+
* from a scratch directory nobody returns to.
|
|
16
14
|
*/
|
|
17
|
-
|
|
15
|
+
/** The hashed root commit of the repository at `projectRoot`; undefined outside git and in a shallow clone. */
|
|
16
|
+
export declare function anonymousProjectId(projectRoot: string): Promise<string | undefined>;
|
|
@@ -8,11 +8,9 @@
|
|
|
8
8
|
* commit, which every clone shares and no one can produce without the
|
|
9
9
|
* repository itself; a commit hash carries no name, path, or remote. A
|
|
10
10
|
* shallow clone has no root to offer: its boundary commits pose as roots and
|
|
11
|
-
* move with every fetch, so it
|
|
12
|
-
*
|
|
13
|
-
*
|
|
14
|
-
* In CI there is no salt, so a runner without a full history has no id: its
|
|
15
|
-
* working directory is not a project.
|
|
11
|
+
* move with every fetch, so it has no id, like a directory outside git: a
|
|
12
|
+
* path is the same project on one machine only, so it cannot be told apart
|
|
13
|
+
* from a scratch directory nobody returns to.
|
|
16
14
|
*/
|
|
17
15
|
import { COMMIT_HASH, git } from '../internal/git.js';
|
|
18
16
|
import { sha256Hex } from '../internal/ids.js';
|
|
@@ -31,11 +29,8 @@ async function rootCommit(cwd) {
|
|
|
31
29
|
.filter((line) => COMMIT_HASH.test(line))
|
|
32
30
|
.toSorted()[0];
|
|
33
31
|
}
|
|
34
|
-
|
|
32
|
+
/** The hashed root commit of the repository at `projectRoot`; undefined outside git and in a shallow clone. */
|
|
33
|
+
export async function anonymousProjectId(projectRoot) {
|
|
35
34
|
const commit = await rootCommit(projectRoot);
|
|
36
|
-
|
|
37
|
-
return sha256Hex(`git\n${commit}`);
|
|
38
|
-
if (salt === undefined)
|
|
39
|
-
return undefined;
|
|
40
|
-
return sha256Hex(`path\n${salt}\n${projectRoot}`);
|
|
35
|
+
return commit === undefined ? undefined : sha256Hex(`git\n${commit}`);
|
|
41
36
|
}
|
|
@@ -2,9 +2,11 @@
|
|
|
2
2
|
* The user-level telemetry preferences: `~/.config/e2e/telemetry.json`
|
|
3
3
|
* (`$XDG_CONFIG_HOME/e2e` when set, `%APPDATA%\e2e` on Windows). The file
|
|
4
4
|
* holds the opt-out, the random id every event from this machine is
|
|
5
|
-
* attributed to,
|
|
6
|
-
*
|
|
7
|
-
*
|
|
5
|
+
* attributed to, and which notice was shown when. Nothing in it names the
|
|
6
|
+
* person or the machine: the id is random bytes, generated here and never
|
|
7
|
+
* derived. A complete file also holds a random salt that nothing is hashed
|
|
8
|
+
* with anymore: older versions hashed paths with it and replace the id of a
|
|
9
|
+
* file without one, and a machine often has several versions installed.
|
|
8
10
|
*
|
|
9
11
|
* The store fails closed. `open` completes the file in one write when the id
|
|
10
12
|
* or the salt is missing, and when that write cannot happen there is no
|
|
@@ -38,8 +40,6 @@ export declare class TelemetryStore {
|
|
|
38
40
|
get enabled(): boolean;
|
|
39
41
|
/** The random per-machine id every event from this machine is attributed to. */
|
|
40
42
|
get anonymousId(): string;
|
|
41
|
-
/** The local salt that keeps a hashed project path unrecoverable; never leaves the machine. */
|
|
42
|
-
get pathSalt(): string;
|
|
43
43
|
/** Whole days since the ids were generated; undefined for a file from before the time was recorded. */
|
|
44
44
|
ageDays(now?: number): number | undefined;
|
|
45
45
|
/** Saves the choice; false when the file could not be written and the choice stayed in memory. */
|
package/dist/telemetry/store.js
CHANGED
|
@@ -2,9 +2,11 @@
|
|
|
2
2
|
* The user-level telemetry preferences: `~/.config/e2e/telemetry.json`
|
|
3
3
|
* (`$XDG_CONFIG_HOME/e2e` when set, `%APPDATA%\e2e` on Windows). The file
|
|
4
4
|
* holds the opt-out, the random id every event from this machine is
|
|
5
|
-
* attributed to,
|
|
6
|
-
*
|
|
7
|
-
*
|
|
5
|
+
* attributed to, and which notice was shown when. Nothing in it names the
|
|
6
|
+
* person or the machine: the id is random bytes, generated here and never
|
|
7
|
+
* derived. A complete file also holds a random salt that nothing is hashed
|
|
8
|
+
* with anymore: older versions hashed paths with it and replace the id of a
|
|
9
|
+
* file without one, and a machine often has several versions installed.
|
|
8
10
|
*
|
|
9
11
|
* The store fails closed. `open` completes the file in one write when the id
|
|
10
12
|
* or the salt is missing, and when that write cannot happen there is no
|
|
@@ -113,10 +115,6 @@ export class TelemetryStore {
|
|
|
113
115
|
get anonymousId() {
|
|
114
116
|
return this.preferences.anonymousId;
|
|
115
117
|
}
|
|
116
|
-
/** The local salt that keeps a hashed project path unrecoverable; never leaves the machine. */
|
|
117
|
-
get pathSalt() {
|
|
118
|
-
return this.preferences.salt;
|
|
119
|
-
}
|
|
120
118
|
/** Whole days since the ids were generated; undefined for a file from before the time was recorded. */
|
|
121
119
|
ageDays(now = Date.now()) {
|
|
122
120
|
const created = this.preferences.createdAt;
|
|
@@ -43,7 +43,7 @@ export interface TelemetryOptions {
|
|
|
43
43
|
readonly fetch?: typeof fetch;
|
|
44
44
|
/** Where the notice and the debug output go; stderr when absent. */
|
|
45
45
|
readonly write?: (text: string) => void;
|
|
46
|
-
/** Resolves the anonymous project id;
|
|
46
|
+
/** Resolves the anonymous project id; the hashed root commit when absent. */
|
|
47
47
|
readonly projectId?: typeof anonymousProjectId;
|
|
48
48
|
}
|
|
49
49
|
export declare class Telemetry {
|
|
@@ -114,8 +114,7 @@ export declare class Telemetry {
|
|
|
114
114
|
record(event: TelemetryEvent): void;
|
|
115
115
|
/**
|
|
116
116
|
* Git is asked for the project id now, while the command runs, so the
|
|
117
|
-
* flush at the end waits on the network alone.
|
|
118
|
-
* store, so a path outside git has nothing to salt it and yields no id.
|
|
117
|
+
* flush at the end waits on the network alone.
|
|
119
118
|
*/
|
|
120
119
|
private lookUpProject;
|
|
121
120
|
/** The session event, first in the batch; absent for an invocation that never reached a command. */
|
|
@@ -187,12 +187,10 @@ export class Telemetry {
|
|
|
187
187
|
}
|
|
188
188
|
/**
|
|
189
189
|
* Git is asked for the project id now, while the command runs, so the
|
|
190
|
-
* flush at the end waits on the network alone.
|
|
191
|
-
* store, so a path outside git has nothing to salt it and yields no id.
|
|
190
|
+
* flush at the end waits on the network alone.
|
|
192
191
|
*/
|
|
193
192
|
lookUpProject() {
|
|
194
|
-
|
|
195
|
-
this.project ??= this.projectId(this.cwd, store?.pathSalt);
|
|
193
|
+
this.project ??= this.projectId(this.cwd);
|
|
196
194
|
}
|
|
197
195
|
/** The session event, first in the batch; absent for an invocation that never reached a command. */
|
|
198
196
|
sessionEvent() {
|
package/dist/types.d.ts
CHANGED
|
@@ -265,6 +265,7 @@ export interface SwipeOptions {
|
|
|
265
265
|
momentum?: Momentum;
|
|
266
266
|
from?: never;
|
|
267
267
|
to?: never;
|
|
268
|
+
duration?: never;
|
|
268
269
|
}
|
|
269
270
|
/** A swipe along a path between two viewport points: a touch swipe on a device, a pointer drag on a document platform. */
|
|
270
271
|
export interface SwipePathOptions {
|
|
@@ -272,6 +273,8 @@ export interface SwipePathOptions {
|
|
|
272
273
|
from: Point;
|
|
273
274
|
/** Where it lifts. */
|
|
274
275
|
to: Point;
|
|
276
|
+
/** Gesture duration in milliseconds; omitted = the engine's default. */
|
|
277
|
+
duration?: number;
|
|
275
278
|
direction?: never;
|
|
276
279
|
momentum?: never;
|
|
277
280
|
}
|
|
@@ -1028,9 +1031,9 @@ export interface StoredArtifact {
|
|
|
1028
1031
|
readonly path: string;
|
|
1029
1032
|
/**
|
|
1030
1033
|
* How much of the file the runner masked, as the report records it. A
|
|
1031
|
-
* `download` is `incomplete` unless the runner
|
|
1032
|
-
* secret
|
|
1033
|
-
* this rather than the kind.
|
|
1034
|
+
* `download` is `incomplete` unless the runner ran it as text through the
|
|
1035
|
+
* session's secret values; a store that exports only what the runner
|
|
1036
|
+
* vouches for reads this rather than the kind.
|
|
1034
1037
|
*/
|
|
1035
1038
|
readonly redaction: 'complete' | 'not-required' | 'incomplete';
|
|
1036
1039
|
readonly runId: string;
|
package/docs/agent-steps.mdx
CHANGED
package/docs/agents.mdx
CHANGED
package/docs/api-testing.mdx
CHANGED
package/docs/authentication.mdx
CHANGED
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
title: Signing in
|
|
3
3
|
description: Sign in once, reuse the session across tests, and pass secrets without exposing them to the model.
|
|
4
|
+
icon: "key"
|
|
4
5
|
---
|
|
5
6
|
|
|
6
7
|
Sign in once in a setup test and save a named session. Other tests restore
|
|
@@ -164,7 +165,7 @@ runner keeps it away from test code and the model altogether.
|
|
|
164
165
|
|
|
165
166
|
A `Secret` has no `.value`. Three sinks accept one: the two below in a
|
|
166
167
|
test, and an engine option that declares it, as
|
|
167
|
-
[`basicAuth`](/
|
|
168
|
+
[`basicAuth`](/web#protected-previews) does:
|
|
168
169
|
|
|
169
170
|
<Tabs>
|
|
170
171
|
<Tab title="Deterministic">
|
|
@@ -279,7 +280,7 @@ account, a username with the secrets that sign it in, while a secret is one
|
|
|
279
280
|
value, so `credentials.admin` and `secrets.admin` can both exist.
|
|
280
281
|
|
|
281
282
|
In `e2e.config.ts`, `secrets.get()` hands an engine option a secret it needs
|
|
282
|
-
before any test runs, such as a [basic-auth password](/
|
|
283
|
+
before any test runs, such as a [basic-auth password](/web#protected-previews).
|
|
283
284
|
The config load checks the name, and the value is redacted once the engine
|
|
284
285
|
resolves it, together with the base64 `Authorization` credential it becomes.
|
|
285
286
|
The protection is text only: screenshots and the model's pixels are kept, so
|
package/docs/browser.mdx
CHANGED
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
title: Working with the browser
|
|
3
3
|
description: Mock an API, handle a dialog, download a file, reach into an iframe, and pick a browser.
|
|
4
|
+
icon: "window-maximize"
|
|
4
5
|
---
|
|
5
6
|
|
|
6
7
|
`@e2e-dev/web` runs tests in Chromium, Firefox, and WebKit. It is built on
|
|
@@ -144,7 +145,7 @@ await browser.setCookies([{ url: 'http://localhost:3000', name: 'tz', value: 'UT
|
|
|
144
145
|
A cookie URL that does not parse, or uses a scheme other than `http:` or
|
|
145
146
|
`https:` (`about:blank` included), is `POLICY_DENIED`; a domain cookie is checked as the origin
|
|
146
147
|
it would be sent to. If any entry is refused, no cookies are set. To resize
|
|
147
|
-
the page, see [
|
|
148
|
+
the page, see [Viewports](/web#viewports).
|
|
148
149
|
|
|
149
150
|
## Pick a browser
|
|
150
151
|
|
|
@@ -183,7 +184,7 @@ bunx @e2e-dev/web install webkit --with-deps
|
|
|
183
184
|
|
|
184
185
|
`connect` attaches to a remote Chromium over CDP instead of launching
|
|
185
186
|
one; a [browser provider](#hosted-browsers) does the same and also leases
|
|
186
|
-
and releases the browser. [
|
|
187
|
+
and releases the browser. [Viewports](/web#viewports) covers sizes.
|
|
187
188
|
|
|
188
189
|
## Hosted browsers
|
|
189
190
|
|
|
@@ -197,7 +198,7 @@ when the run no longer uses it. The engine knows no vendor: for
|
|
|
197
198
|
provider, and for any other service a provider is a few dozen lines in your
|
|
198
199
|
project. Add `viewport: null` when
|
|
199
200
|
the service shows the window in a live view: the page then
|
|
200
|
-
[fills the window](/
|
|
201
|
+
[fills the window](/web#fill-the-window) instead of a fixed 1280 by
|
|
201
202
|
720 area of it.
|
|
202
203
|
|
|
203
204
|
```ts
|
|
@@ -285,7 +286,7 @@ same browser.
|
|
|
285
286
|
A provider against a typical session API has three requests: create a
|
|
286
287
|
session, poll it until it exposes a DevTools endpoint, stop it.
|
|
287
288
|
|
|
288
|
-
```ts title="browser-provider.ts"
|
|
289
|
+
```ts title="browser-provider.ts" expandable
|
|
289
290
|
import type { BrowserProvider } from '@e2e-dev/web';
|
|
290
291
|
|
|
291
292
|
/**
|
|
@@ -470,7 +471,7 @@ root declared in HTML (`<template shadowrootmode="closed">`) is out of reach.
|
|
|
470
471
|
<Card title="Web engine reference" icon="masks-theater" href="/reference/web">
|
|
471
472
|
Every `browser` method, its timeout, and what it throws.
|
|
472
473
|
</Card>
|
|
473
|
-
<Card title="
|
|
474
|
-
Let the runner start the dev server
|
|
474
|
+
<Card title="Web" icon="play" href="/web">
|
|
475
|
+
Let the runner start the dev server, reach protected previews, and test viewports.
|
|
475
476
|
</Card>
|
|
476
477
|
</CardGroup>
|
package/docs/bug-bash.mdx
CHANGED
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
title: Bug bashes
|
|
3
3
|
description: Run many explorations at once, then prove each bug with a failing test before anyone reads about it.
|
|
4
|
+
icon: "bug"
|
|
4
5
|
---
|
|
5
6
|
|
|
6
7
|
A bug bash runs several `e2e explore` sessions at once, each on one area
|
|
@@ -50,7 +51,7 @@ itself: `npx e2e guide bug-bash`.
|
|
|
50
51
|
`reuseExisting: true` on the target's `command`, or declare the URL with
|
|
51
52
|
port `0` so each run starts its own copy. `reuseExisting` does not apply
|
|
52
53
|
when `CI` is set; use port `0` there. See
|
|
53
|
-
[Starting your app](/starting-your-app).
|
|
54
|
+
[Starting your app](/web#starting-your-app).
|
|
54
55
|
- When the target's `app.command` brings up a stack of its own, such as a
|
|
55
56
|
database from `docker compose` that it removes on exit, start the stack once
|
|
56
57
|
and explore with a config that points at the running app and declares no
|
package/docs/cache.mdx
CHANGED
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
---
|
|
2
|
-
title: Caching
|
|
2
|
+
title: Caching
|
|
3
3
|
description: Replay verified agent actions without model calls, and let the agent take over when the app changes.
|
|
4
|
+
icon: "box-archive"
|
|
4
5
|
---
|
|
5
6
|
|
|
6
7
|
The replay cache records the actions in an `agent.act()` step after a later
|
package/docs/ci.mdx
CHANGED
|
@@ -1,13 +1,14 @@
|
|
|
1
1
|
---
|
|
2
2
|
title: Continuous integration
|
|
3
3
|
description: Run tests on pull requests, upload reports, and configure model credentials in CI.
|
|
4
|
+
icon: "rotate"
|
|
4
5
|
---
|
|
5
6
|
|
|
6
7
|
Run deterministic and agent tests in the same CI job. This workflow assumes
|
|
7
|
-
your config includes a [command](/starting-your-app) to start the app and
|
|
8
|
+
your config includes a [command](/web#starting-your-app) to start the app and
|
|
8
9
|
uses AI Gateway for model calls:
|
|
9
10
|
|
|
10
|
-
```yaml title=".github/workflows/e2e.yml"
|
|
11
|
+
```yaml title=".github/workflows/e2e.yml" expandable
|
|
11
12
|
name: e2e
|
|
12
13
|
|
|
13
14
|
on:
|
package/docs/coding-agents.mdx
CHANGED
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
title: Coding agents
|
|
3
3
|
description: Give a coding agent the skill, a live app session, and reports it can use to write and debug tests.
|
|
4
|
+
icon: "robot"
|
|
4
5
|
---
|
|
5
6
|
|
|
6
7
|
`e2e init` installs a skill for writing and running tests, plus MCP
|
|
@@ -168,7 +169,7 @@ model call in the flow.
|
|
|
168
169
|
Do not edit recordings by hand. A failed read-write run removes implicated
|
|
169
170
|
entries so a later verified run can replace them. A failure where no model
|
|
170
171
|
answered implicates nothing and removes none. See
|
|
171
|
-
[Caching
|
|
172
|
+
[Caching](/cache).
|
|
172
173
|
|
|
173
174
|
<CardGroup cols={2}>
|
|
174
175
|
<Card title="CLI reference" icon="terminal" href="/reference/cli">
|