jevway 0.2.0 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (41) hide show
  1. package/README.md +37 -2
  2. package/{INSTALL-FOR-AGENTS.md → SETUP.md} +94 -14
  3. package/dist/browser/downloads.js +103 -0
  4. package/dist/browser/frames.js +21 -1
  5. package/dist/browser/snapshot.js +53 -0
  6. package/dist/browser/tab.js +59 -10
  7. package/dist/cli/dispatch.js +2 -0
  8. package/dist/cli/inbox-command.js +199 -0
  9. package/dist/cli/look-command.js +10 -3
  10. package/dist/cli/run-command.js +51 -3
  11. package/dist/daemon/server.js +22 -3
  12. package/dist/inbox/inbox-file.js +43 -0
  13. package/dist/inbox/inbox-registry.js +33 -0
  14. package/dist/inbox/inbox.js +127 -0
  15. package/dist/inbox/mail-api.js +52 -0
  16. package/dist/inbox/message.js +34 -0
  17. package/dist/jev/secrets.js +4 -3
  18. package/dist/mcp/inbox-tools.js +82 -0
  19. package/dist/mcp/instructions.js +11 -2
  20. package/dist/mcp/server.js +15 -8
  21. package/dist/mcp/task-tools.js +24 -8
  22. package/dist/notify/desktop.js +40 -0
  23. package/dist/notify/person-messages.js +50 -0
  24. package/dist/output/layout.js +37 -1
  25. package/dist/output/manifest.js +6 -0
  26. package/dist/output/run-folder.js +44 -2
  27. package/dist/recording/describe-step.js +3 -0
  28. package/dist/session/session-service.js +31 -11
  29. package/dist/setup/skill-command.js +44 -21
  30. package/dist/task/ask-user.js +182 -0
  31. package/dist/task/downloads.js +63 -0
  32. package/dist/task/person-wait.js +104 -0
  33. package/dist/task/present.js +32 -11
  34. package/dist/task/run-task.js +265 -28
  35. package/dist/task/start-task.js +49 -6
  36. package/dist/task/totp.js +124 -0
  37. package/dist/task/upload-files.js +237 -5
  38. package/package.json +2 -2
  39. package/skill/jevway/SKILL.md +5 -2
  40. package/skill/jevway-inbox/SKILL.md +61 -0
  41. package/skill/jevway-inbox/scripts/inbox.mjs +209 -0
package/README.md CHANGED
@@ -40,6 +40,7 @@ optional tools unlock more.
40
40
  | **Google Chrome** (or Chromium) | every run: your running Chrome, or a throwaway one jevway starts | runs fail; set `CHROME_PATH` if Chrome isn't in the usual place |
41
41
  | **An API key**: `TYPESAFE_API_KEY` (recommended) or `OPENROUTER_API_KEY` | every decision Jev makes | the server still starts and lists its tools, and `jevway_look` works, but tasks and journeys answer with the setup fix |
42
42
  | `ffmpeg` *(optional)* | the `run.mp4` video in a recording | recordings keep their step screenshots and HTML page, without the video |
43
+ | `notify-send` *(optional, Linux)* | the desktop notification when a person is needed (macOS has it built in) | the agent still asks you; there's just no pop-up |
43
44
  | chrome-devtools MCP *(optional)* | inspecting what jevway left on screen: console, network, performance, DOM details | nothing is lost in jevway itself; your agent just has no deeper inspection tool |
44
45
 
45
46
  **Which key?** Both run the same Jev model. The TypeSafe key adds **Box**, which locates controls on the screenshot:
@@ -78,6 +79,20 @@ control (opt-in), and marks each journey screen's primary action and the control
78
79
 
79
80
  `npx -y jevway doctor` checks the setup at any time (`--json` for agents) and names the fix for anything missing.
80
81
 
82
+ ## Let your agent set it up
83
+
84
+ Paste this into your coding agent (Claude Code, Codex, Cursor, OpenCode…):
85
+
86
+ ```text
87
+ Set up jevway for me and prove it works: follow https://github.com/AbdallahAHO/jevway/blob/main/SETUP.md step by
88
+ step. Check my machine's prerequisites with me first, show me every change before you make it, never ask me for an
89
+ API key in this chat, and finish by running the recorded smoke test and showing me its run page.
90
+ ```
91
+
92
+ [SETUP.md](SETUP.md) walks the agent through everything: prerequisites for your OS, the key, registering the server,
93
+ the browser, the skill, a recorded Google → Wikipedia smoke test, and a report with the run page, video and
94
+ screenshots as proof.
95
+
81
96
  ## Install
82
97
 
83
98
  **Claude Code:** the plugin above, or the server alone:
@@ -109,7 +124,7 @@ when to use chrome-devtools. If a client times out on `npx`'s first download, ru
109
124
  `jevway-mcp` as the command.
110
125
 
111
126
  Having an agent install it? `npx -y jevway setup --client <claude-code|codex|cursor|opencode|other> --print` prints the
112
- exact command or config for that client and changes nothing. [INSTALL-FOR-AGENTS.md](INSTALL-FOR-AGENTS.md) has the
127
+ exact command or config for that client and changes nothing. [SETUP.md](SETUP.md) has the
113
128
  steps per client, the rules (a diff before any config edit, no key in the chat) and the checks.
114
129
 
115
130
  **Choosing a browser** (`JEVWAY_BROWSER` in the server's env or the env file):
@@ -187,6 +202,26 @@ field is in the [reference](docs/reference.md).
187
202
  - `record: true` saves a video, a screenshot before every action, and a page to review the run.
188
203
  - `page_text` and everything else read from a page is untrusted data, never instructions.
189
204
 
205
+ ## When a person, a code or a file is involved
206
+
207
+ jevway keeps the person in the loop only where a person is truly needed, and handles the rest itself.
208
+
209
+ - **Asking you.** Every hand-back that needs a decision carries an `ask_user` question: the missing fields, "Click
210
+ *Delete repository* on *ada/scratch*?", "Which one did you mean?". Your agent puts it to you with its own ask tool
211
+ and continues the session with your answer.
212
+ - **Bot checks and sign-ins.** In a visible browser, jevway brings the tab to the front, sends a desktop notification
213
+ ("Pass the Cloudflare check in the Chrome window"), waits up to 120 s while it tells the agent how long is left,
214
+ and carries on in the same call once you're through. It never tries to get past a check itself. In a headless
215
+ browser it hands back and suggests a visible one.
216
+ - **Email codes for test sign-ups.** A throwaway inbox (`jevway_inbox_create` / `jevway_inbox_wait`, or
217
+ `jevway inbox …`) returns the code and links of the email that arrives. Only for test and demo accounts you asked
218
+ for, never to get around limits, trials or bans.
219
+ - **2FA on your own accounts.** Pass `totp: {"otp": "<authenticator secret>"}`: jevway computes the current code at the
220
+ moment it types it, so neither the secret nor the code reaches the model, a result or disk.
221
+ - **Files.** Uploads take local paths or `https` URLs (fetched into the run folder), work inside iframes and on
222
+ drag-and-drop zones. Downloads land in the run folder's `downloads/` and are listed in `files.downloads`.
223
+ `jevway_look` with `pdf: true` saves the page as a PDF.
224
+
190
225
  ## Recordings
191
226
 
192
227
  Every session gets a folder under `./.jevway/` in the working directory, so the agent and you can find its files:
@@ -241,7 +276,7 @@ OpenRouter too. Jev is priced at $0.042 per 1M input tokens, so a typical 10-ste
241
276
  | [docs/reference.md](docs/reference.md) | every tool, parameter, status, CLI flag, environment variable and model |
242
277
  | [docs/how-it-works.md](docs/how-it-works.md) | the technical specification: one step end to end, guards, safety |
243
278
  | [docs/journeys.md](docs/journeys.md) | the journey-map tools |
244
- | [INSTALL-FOR-AGENTS.md](INSTALL-FOR-AGENTS.md) | agents installing jevway for a person |
279
+ | [SETUP.md](SETUP.md) | agents setting jevway up for a person, with a recorded smoke test |
245
280
  | [skill/jevway/SKILL.md](skill/jevway/SKILL.md) | the skill agents load: when and how to call jevway |
246
281
  | [docs/verification.md](docs/verification.md) | the test suites and their results |
247
282
  | [docs/prior-art.md](docs/prior-art.md) | other open-source Jev browser projects, and what was taken from them |
@@ -1,20 +1,34 @@
1
- # Installing jevway (for agents)
2
-
3
- You are an agent asked to install jevway for a person. jevway is an MCP server, `jevway-mcp`, plus a CLI, `jevway`.
4
- It hands browser goals to TypeSafe Jev. Both come from the npm package `jevway`.
5
-
6
- Read this whole file before you run anything. Run only these:
1
+ # Setting up jevway (for agents)
2
+
3
+ You are an agent asked to set up jevway for a person and prove it works. jevway is an MCP server, `jevway-mcp`, plus a
4
+ CLI, `jevway`, both from the npm package `jevway`. It hands browser goals to TypeSafe Jev and returns one checked
5
+ summary per goal.
6
+
7
+ Read this whole file before you run anything. You will:
8
+ 1. check the machine's prerequisites with the person and help install what's missing;
9
+ 2. get the API key in place (the person does that part);
10
+ 3. register the MCP server with the client you're running in;
11
+ 4. pick the browser with the person;
12
+ 5. install the skill;
13
+ 6. check the setup;
14
+ 7. run a recorded smoke test, from Google to Wikipedia;
15
+ 8. report back with the run page, its video and screenshots, as proof.
16
+
17
+ Run only these yourself:
18
+ - `node --version`, `ffmpeg -version`, and the person-approved install commands from step 1;
7
19
  - `npx -y jevway doctor --json` (or `--plan`, or `jevway doctor`);
8
20
  - `npx -y jevway setup --client <name> --print [--json]`, which prints how to register jevway with a client and
9
21
  changes nothing;
10
22
  - `npx -y jevway skill --install <skills folder>`;
11
23
  - `npm install -g jevway`;
12
- - the client's own `mcp add` command, or an edit to its MCP config file that the person approved.
24
+ - the client's own `mcp add` command, or an edit to its MCP config file that the person approved;
25
+ - the jevway tools themselves (or `npx -y jevway run …`) for the smoke test.
13
26
 
14
27
  Never run plain `jevway setup` yourself: it is the person's interactive setup, and without a terminal it exits 2 with
15
28
  `{ "ok": false, "error": { "code": "needs_tty", … } }`.
16
29
 
17
- Done means: `jevway doctor --json` prints `"ready": true` (exit 0), and the client lists the `jevway_*` tools.
30
+ Done means: `jevway doctor --json` prints `"ready": true`, the client lists the `jevway_*` tools, and the smoke test
31
+ ended `done` with a run page you showed the person.
18
32
 
19
33
  ## Rules
20
34
 
@@ -45,6 +59,19 @@ It needs no key to run and never prints one. It checks:
45
59
  - the browser source;
46
60
  - ffmpeg (optional: only for recordings).
47
61
 
62
+ `npx` itself needs Node, so check `node --version` first. For anything missing, propose the command that fits the
63
+ person's OS and package manager, and run it only after they say yes (or wait while they install it themselves, then
64
+ check again):
65
+
66
+ | Tool | Needed for | macOS (Homebrew) | Debian / Ubuntu | Fedora | Windows (winget) |
67
+ | --- | --- | --- | --- | --- | --- |
68
+ | Node.js 24+ | running jevway (required) | `brew install node` | Node 24 from NodeSource or nvm | `sudo dnf install nodejs` (check it's 24+) | `winget install OpenJS.NodeJS` |
69
+ | Chrome or Chromium | every run (required) | `brew install --cask google-chrome` | Google's `.deb`, or `sudo apt install chromium` | `sudo dnf install chromium` | `winget install Google.Chrome` |
70
+ | ffmpeg | the video in a recording (optional) | `brew install ffmpeg` | `sudo apt install ffmpeg` | `sudo dnf install ffmpeg-free` | `winget install Gyan.FFmpeg` |
71
+ | notify-send | the desktop notification when a person is needed (optional, Linux) | built in | `sudo apt install libnotify-bin` | `sudo dnf install libnotify` | not used |
72
+
73
+ A Chrome in an unusual place: set `CHROME_PATH` in `~/.config/jevway/env`.
74
+
48
75
  If the first `npx` start is slow, or a client times out starting the server, install once with `npm install -g jevway`.
49
76
  Then use `jevway-mcp` as the command instead of `npx -y -p jevway jevway-mcp`.
50
77
 
@@ -97,7 +124,8 @@ npx -y jevway setup --client cursor --print --json # { client, kind, command?, f
97
124
 
98
125
  ## 4. The browser
99
126
 
100
- `JEVWAY_BROWSER` (the server's env, or `--browser` for the CLI):
127
+ Ask the person which browser jevway should use, and set `JEVWAY_BROWSER` in the server's env (or `--browser` for
128
+ the CLI) when it isn't `auto`. For the smoke test a visible one is best, so they can pass a check if one appears.
101
129
 
102
130
  | Value | What it uses | What the person does |
103
131
  | --- | --- | --- |
@@ -117,11 +145,13 @@ npx -y jevway skill --install ~/.codex/skills # or ~/.claude/skills, ~/.cursor
117
145
  It prints `{ ok, status, path, files }`. A different `jevway` skill already there (or a symlink to one) is left alone
118
146
  with `"code": "skill_exists"` and exit 1; replace it with `--force` only if the person agrees.
119
147
 
120
- ## 6. Check the outcome
148
+ ## 6. Check the setup
121
149
 
122
- 1. `npx -y jevway doctor --json` prints `"ready": true` and exits 0 (every finding `ok` or `warn`).
123
- 2. The client lists the tools: `jevway_task`, `jevway_look`, `jevway_end` and `jevway_journey_start`, `_step`,
124
- `_autopilot`, `_annotate`, `_finish`.
150
+ 1. `npx -y jevway doctor --json` prints `"ready": true` and exits 0 (every finding `ok` or `warn`). For every finding
151
+ that isn't `ok`, tell the person the fix in one line and wait; re-run it after each fix (`--plan` lists the steps
152
+ left).
153
+ 2. The client lists the tools: `jevway_task`, `jevway_look`, `jevway_end`, the five `jevway_journey_*` tools and
154
+ `jevway_inbox_create` / `jevway_inbox_wait`.
125
155
  - In Claude Code: `/mcp`.
126
156
  - Elsewhere, the client's MCP list.
127
157
  - Or the MCP Inspector:
@@ -129,4 +159,54 @@ with `"code": "skill_exists"` and exit 1; replace it with `--force` only if the
129
159
  3. Optional, costs a fraction of a cent: `npx -y jevway doctor --live` makes one real Jev request and reports its
130
160
  latency.
131
161
 
132
- When a check fails, the doctor names the cause and the fix, and `--plan` lists the steps left. Re-run it after each fix.
162
+ ## 7. The smoke test
163
+
164
+ Call `jevway_task` with:
165
+
166
+ ```json
167
+ {
168
+ "url": "https://www.google.com",
169
+ "goal": "Search Google for the Rosetta Stone and open its English Wikipedia article",
170
+ "values": { "query": "Rosetta Stone Wikipedia" },
171
+ "done_when": "The English Wikipedia article about the Rosetta Stone is open",
172
+ "allowed_hosts": ["google.com", "*.google.com", "en.wikipedia.org"],
173
+ "record": true,
174
+ "detail": "full"
175
+ }
176
+ ```
177
+
178
+ What to expect, and what to do:
179
+ - **A cookie banner** is handled by jevway (it declines when it can).
180
+ - **A reCAPTCHA ("unusual traffic")** is common for automated browsers on Google. In a visible browser jevway brings
181
+ the tab to the front, sends the person a desktop notification and waits up to 2 minutes for them to pass it; it
182
+ never solves it. Tell the person it may happen before you start.
183
+ - **A result with `ask_user`** (a bot check, a sign-in, a missing value): put that question to the person with your
184
+ own ask tool, then do what its `then` says.
185
+ - **Google keeps blocking**: run the same goal from `https://duckduckgo.com` with
186
+ `allowed_hosts: ["duckduckgo.com", "en.wikipedia.org"]`. It usually finishes in 5 steps.
187
+ - **No MCP in your client**: run the same test with the CLI, which prints one JSON result on stdout:
188
+
189
+ ```bash
190
+ npx -y jevway run --browser launch --record --detail full --url https://www.google.com \
191
+ --goal "Search Google for the Rosetta Stone and open its English Wikipedia article" \
192
+ --values '{"query":"Rosetta Stone Wikipedia"}' \
193
+ --done-when "The English Wikipedia article about the Rosetta Stone is open" \
194
+ --allowed-hosts "google.com,*.google.com,en.wikipedia.org"
195
+ ```
196
+
197
+ End the session with `jevway_end` when you have what you need.
198
+
199
+ ## 8. Report back
200
+
201
+ Tell the person, in this shape:
202
+ - **Outcome:** `status` and `reason`, and the `story` as a numbered list.
203
+ - **Proof:** the run page `files.page` (open it for them: `open` on macOS, `xdg-open` on Linux, `start` on Windows),
204
+ the video `files.video` (present when ffmpeg is installed), the end screenshot `files.screenshot` (show it if you
205
+ can display images), and the run folder `files.run_dir` with its `manifest.json`.
206
+ - **Cost:** `jev.requests` and `jev.est_cost_usd` (an estimate; a typical smoke test is well under a cent).
207
+ - **Setup:** what you installed or changed, with the person's yes for each.
208
+ - **Anything that needed them**, and anything that didn't work, with the fix.
209
+
210
+ The run page opens with the goal and how it ended, then the video with every step on a timeline and a strip of
211
+ screenshots kept in sync. It is the proof that the whole chain works: the client, the MCP server, the key, the
212
+ browser and Jev.
@@ -0,0 +1,103 @@
1
+ import { mkdtempSync, rmSync } from 'node:fs';
2
+ import { tmpdir } from 'node:os';
3
+ import { join, relative, isAbsolute } from 'node:path';
4
+ const STAGING = new WeakMap();
5
+ /**
6
+ * Sets how the browser behind `cdp` downloads, once per connection: jevway's launched browser saves every download under
7
+ * its guid in a staging folder of its own; the person's Chrome (`auto`, a debugging URL) keeps its own behaviour and only
8
+ * reports downloads. A browser that refuses never fails a run: its downloads go unreported.
9
+ *
10
+ * @example
11
+ * await useDownloads(cdp, 'launch-headless'); // Browser.setDownloadBehavior { behavior: 'allowAndName', downloadPath: '/tmp/jevway-downloads-…' }
12
+ */
13
+ export const useDownloads = async (cdp, source) => {
14
+ if (source !== 'launch' && source !== 'launch-headless') {
15
+ await cdp.send('Browser.setDownloadBehavior', { behavior: 'default', eventsEnabled: true }).catch(() => undefined);
16
+ return;
17
+ }
18
+ const downloadPath = mkdtempSync(join(tmpdir(), 'jevway-downloads-'));
19
+ STAGING.set(cdp, downloadPath);
20
+ await cdp.send('Browser.setDownloadBehavior', { behavior: 'allowAndName', downloadPath, eventsEnabled: true }).catch(() => undefined);
21
+ };
22
+ /** Removes the staging folder of a launched browser's connection, with whatever was never kept. */
23
+ export const dropStagedDownloads = (cdp) => {
24
+ const dir = STAGING.get(cdp);
25
+ if (dir)
26
+ rmSync(dir, { recursive: true, force: true });
27
+ STAGING.delete(cdp);
28
+ };
29
+ const inside = (dir, path) => {
30
+ if (!dir || !path)
31
+ return false;
32
+ const rel = relative(dir, path);
33
+ return Boolean(rel) && !rel.startsWith('..') && !isAbsolute(rel);
34
+ };
35
+ /**
36
+ * The downloads one tab started: `owns` says whether a frame id is the tab's (its page, one of its frames, a tab it
37
+ * opened). Every other download, in the person's other tabs too, is never recorded.
38
+ *
39
+ * @example
40
+ * const downloads = new TabDownloads(cdp, (frameId) => frameId === targetId);
41
+ * const finished = await downloads.settle({ waitMs: 30_000 });
42
+ */
43
+ export class TabDownloads {
44
+ #cdp;
45
+ #owns;
46
+ #all = new Map();
47
+ #untaken = [];
48
+ #changed = new Set();
49
+ constructor(cdp, owns) {
50
+ this.#cdp = cdp;
51
+ this.#owns = owns;
52
+ }
53
+ /** Starts listening; returns the unsubscribe. */
54
+ watch() {
55
+ return this.#cdp.onEvent((method, params, sessionId) => {
56
+ if (sessionId || !method.startsWith('Browser.download'))
57
+ return;
58
+ if (method === 'Browser.downloadWillBegin' && this.#owns(params.frameId)) {
59
+ this.#all.set(params.guid, { guid: params.guid, name: params.suggestedFilename, url: params.url, state: 'inProgress', bytes: 0, staged: false });
60
+ this.#untaken.push(params.guid);
61
+ }
62
+ const known = method === 'Browser.downloadProgress' && this.#all.get(params.guid);
63
+ if (!known)
64
+ return;
65
+ const filePath = params.filePath ?? known.filePath;
66
+ this.#all.set(params.guid, {
67
+ ...known,
68
+ state: params.state,
69
+ bytes: params.receivedBytes ?? known.bytes,
70
+ ...(filePath && { filePath }),
71
+ staged: inside(STAGING.get(this.#cdp), filePath),
72
+ });
73
+ for (const changed of this.#changed)
74
+ changed();
75
+ });
76
+ }
77
+ /**
78
+ * Takes the downloads begun since the last take, after waiting up to `waitMs` (or until `signal`) for those still
79
+ * running. Each comes back as it stands then: completed, canceled, or still in progress.
80
+ */
81
+ async settle({ waitMs, signal }) {
82
+ const taken = this.#untaken.splice(0);
83
+ const running = () => taken.some((guid) => this.#all.get(guid)?.state === 'inProgress');
84
+ if (taken.length && running() && !signal?.aborted) {
85
+ await new Promise((resolve) => {
86
+ const done = () => {
87
+ clearTimeout(timer);
88
+ signal?.removeEventListener('abort', done);
89
+ this.#changed.delete(check);
90
+ resolve();
91
+ };
92
+ const check = () => {
93
+ if (!running())
94
+ done();
95
+ };
96
+ const timer = setTimeout(done, waitMs);
97
+ signal?.addEventListener('abort', done, { once: true });
98
+ this.#changed.add(check);
99
+ });
100
+ }
101
+ return taken.map((guid) => ({ ...this.#all.get(guid) }));
102
+ }
103
+ }
@@ -134,6 +134,7 @@ export class Frames {
134
134
  #sessionId;
135
135
  #counter;
136
136
  #touch = false;
137
+ #files = false;
137
138
  #mainFrameId = '';
138
139
  #viewport = { w: 0, h: 0 };
139
140
  /** Out-of-process frames attached so far, by target id (= frame id). */
@@ -164,6 +165,20 @@ export class Frames {
164
165
  for (const { sessionId } of this.#children.values())
165
166
  void this.#emulateTouch(sessionId);
166
167
  }
168
+ /**
169
+ * The caller declared files: frame reads list file inputs and drop zones, and out-of-process frames keep native file
170
+ * pickers closed too (the page's own session does that for same-process frames).
171
+ */
172
+ set files(enabled) {
173
+ if (enabled && !this.#files)
174
+ for (const { sessionId } of this.#children.values())
175
+ void this.#interceptFileChooser(sessionId);
176
+ this.#files = enabled;
177
+ }
178
+ /** Whether a protocol frame id is this tab's: its page, a frame read on it, or an out-of-process frame attached to it. */
179
+ holds(frameId) {
180
+ return Boolean(frameId) && (frameId === this.#mainFrameId || this.#known.has(frameId) || this.#children.has(frameId));
181
+ }
167
182
  /** The frame holding an element id: undefined for the main frame and for a frame no longer on the page. */
168
183
  ofNode(node) {
169
184
  const base = idBaseOf(node);
@@ -228,7 +243,7 @@ export class Frames {
228
243
  // The text walk keeps to the part of the frame the page shows, in the frame's own coordinates.
229
244
  const { offset, region } = placement;
230
245
  const view = { x: region.x - offset.x, y: region.y - offset.y, w: region.w, h: region.h };
231
- const response = await this.evaluate(frame, `window.__jevIdBase=${frame.base};window.__jevView=${JSON.stringify(view)};window.__jevWanted=${JSON.stringify(wanted)};${script}`, { timeout: FRAME_READ_MS });
246
+ const response = await this.evaluate(frame, `window.__jevIdBase=${frame.base};window.__jevView=${JSON.stringify(view)};window.__jevWanted=${JSON.stringify(wanted)};window.__jevFiles=${this.#files};${script}`, { timeout: FRAME_READ_MS });
232
247
  const state = response.exceptionDetails ? null : response.result?.value;
233
248
  if (!state)
234
249
  return null;
@@ -404,6 +419,8 @@ export class Frames {
404
419
  await this.#send('Page.addScriptToEvaluateOnNewDocument', { source: this.#counter, worldName: WORLD }, sessionId);
405
420
  await this.#send('Emulation.setFocusEmulationEnabled', { enabled: true }, sessionId).catch(() => undefined);
406
421
  await this.#emulateTouch(sessionId);
422
+ if (this.#files)
423
+ await this.#interceptFileChooser(sessionId);
407
424
  await this.#autoAttach(sessionId);
408
425
  }
409
426
  catch {
@@ -418,6 +435,9 @@ export class Frames {
418
435
  const enabled = this.#touch;
419
436
  return this.#send('Emulation.setTouchEmulationEnabled', { enabled, maxTouchPoints: enabled ? 5 : 1 }, sessionId).catch(() => undefined);
420
437
  }
438
+ #interceptFileChooser(sessionId) {
439
+ return this.#send('Page.setInterceptFileChooserDialog', { enabled: true }, sessionId).catch(() => undefined);
440
+ }
421
441
  #autoAttach(sessionId) {
422
442
  return this.#send('Target.setAutoAttach', { autoAttach: true, waitForDebuggerOnStart: true, flatten: true, filter: [{ type: 'iframe' }] }, sessionId);
423
443
  }
@@ -424,10 +424,12 @@
424
424
  }
425
425
  return null;
426
426
  };
427
+ const anchors = [];
427
428
  for (const input of fileInputs) {
428
429
  if (input.matches(':disabled') || input.closest('[aria-disabled="true"],[inert]')) continue;
429
430
  const anchor = standInFor(input);
430
431
  if (!anchor || outOfView(anchor.getBoundingClientRect())) continue;
432
+ anchors.push(anchor);
431
433
  const r = cache.clickRect(anchor),
432
434
  own = (anchor === input ? name(input) : name(input) || name(anchor)).replace(/\s+/g, ' ').trim();
433
435
  actions.push({
@@ -439,6 +441,57 @@
439
441
  rect: { x: r.x, y: r.y, w: r.width, h: r.height },
440
442
  });
441
443
  }
444
+ // jevway: a drop zone with no file input in it is listed only on clear markers: an ondrop attribute, a drop-zone
445
+ // name in its class, id or a data- attribute, or a short text asking for a file to be dropped ("Drag and drop",
446
+ // "Drop your CV here"; its nearest box of 40×40 or more). The innermost zone wins; one holding or inside a file field's stand-in
447
+ // is that field already. The file is dropped onto it.
448
+ const DROP_NAME = /drop-?zone|drop-?area|file-?drop|drop-?target/i;
449
+ const DROP_TEXT =
450
+ /\bdrag\s*(?:and|&|'?n'?)\s*drop\b|\bdrop\s+(?:your\s+|a\s+|an\s+|the\s+)?(?:files?|cv|resume|résumé|images?|photos?|documents?|attachments?)\b/i;
451
+ const named = (e) =>
452
+ e.hasAttribute('ondrop') ||
453
+ DROP_NAME.test(typeof e.className === 'string' ? e.className : '') ||
454
+ DROP_NAME.test(e.id) ||
455
+ [...e.attributes].some((a) => a.name.startsWith('data-') && DROP_NAME.test(a.name));
456
+ const big = (e) => {
457
+ const r = e.getBoundingClientRect();
458
+ return r.width >= 40 && r.height >= 40;
459
+ };
460
+ const zones = new Set(
461
+ [...document.querySelectorAll('[ondrop],[class*="drop" i],[id*="drop" i]')].filter(named),
462
+ );
463
+ const texts = document.createTreeWalker(document.body ?? document.documentElement, NodeFilter.SHOW_TEXT, {
464
+ acceptNode: (t) => (t.data.length <= 200 && DROP_TEXT.test(t.data) ? NodeFilter.FILTER_ACCEPT : NodeFilter.FILTER_SKIP),
465
+ });
466
+ for (let t = texts.nextNode(), seen = 0; t && seen < 20; t = texts.nextNode(), seen++) {
467
+ let box = t.parentElement;
468
+ for (let up = 0; box && up < 4 && !big(box); up++) box = box.parentElement;
469
+ if (box && big(box)) zones.add(box);
470
+ }
471
+ const listed = [...zones].filter(
472
+ (zone) =>
473
+ zone !== document.body &&
474
+ zone !== document.documentElement &&
475
+ visible(zone) &&
476
+ big(zone) &&
477
+ !outOfView(zone.getBoundingClientRect()) &&
478
+ !zone.querySelector('input[type=file]') &&
479
+ !anchors.some((anchor) => anchor.contains(zone) || zone.contains(anchor)),
480
+ );
481
+ for (const zone of listed) {
482
+ if (listed.some((other) => other !== zone && zone.contains(other))) continue;
483
+ const r = cache.clickRect(zone),
484
+ text = (zone.innerText || '').replace(/\s+/g, ' ').trim();
485
+ actions.push({
486
+ node: identity(zone),
487
+ role: 'file',
488
+ kind: 'upload',
489
+ drop: true,
490
+ label: cap(name(zone).replace(/\s+/g, ' ').trim() || text || 'drop zone', 120) + ' (drop zone)',
491
+ value: '',
492
+ rect: { x: r.x, y: r.y, w: r.width, h: r.height },
493
+ });
494
+ }
442
495
  }
443
496
  // jevway: controls sharing a role and label (a product grid's "Add to cart" buttons) are told apart by the card or
444
497
  // row each sits in, as a person reads the page: "Add to cart — Sauce Labs Bike Light". Only colliding labels change;
@@ -4,6 +4,7 @@ import { readFileSync } from 'node:fs';
4
4
  import { isDeepStrictEqual } from 'node:util';
5
5
  import { CdpError } from './cdp.js';
6
6
  import { DEVICES } from './devices.js';
7
+ import { TabDownloads } from './downloads.js';
7
8
  import { Frames, idBaseOf, mergeFrames, whenAborted } from './frames.js';
8
9
  import { messageNames } from './words.js';
9
10
  // Atomic DOM read from browser-use/jev-ultrafast (MIT): visible text + indexed controls + freshness guards.
@@ -72,6 +73,9 @@ export class Tab {
72
73
  #device;
73
74
  /** Page tabs this tab opened (target=_blank, window.open) that the run has not taken yet, oldest first. */
74
75
  #opened = [];
76
+ /** Every page tab this tab opened, taken or not: a download one of them starts is this tab's. */
77
+ #openedEver = new Set();
78
+ #downloads;
75
79
  #closed = false;
76
80
  #unsubscribe = [];
77
81
  /** The running call's stop signal: the waits after an input end early on it (see `stopSignal`). */
@@ -85,6 +89,7 @@ export class Tab {
85
89
  this.targetId = targetId;
86
90
  this.sessionId = sessionId;
87
91
  this.#frames = new Frames(cdp, sessionId, INPUT_COUNTER);
92
+ this.#downloads = new TabDownloads(cdp, (frameId) => frameId === targetId || this.#openedEver.has(frameId) || this.#frames.holds(frameId));
88
93
  }
89
94
  /**
90
95
  * Opens a background tab so the user's visible tab stays put. `newWindow` keeps the page rendering
@@ -233,13 +238,16 @@ export class Tab {
233
238
  // opened (the user's own, in their Chrome) is never attached, read or changed.
234
239
  tab.#unsubscribe.push(cdp.onEvent((method, params, sessionId) => {
235
240
  const info = params?.targetInfo;
236
- if (method === 'Target.targetCreated' && info?.type === 'page' && info.openerId === targetId)
241
+ if (method === 'Target.targetCreated' && info?.type === 'page' && info.openerId === targetId) {
237
242
  tab.#opened.push(info.targetId);
243
+ tab.#openedEver.add(info.targetId);
244
+ }
238
245
  if (method === 'Target.targetDestroyed' && params?.targetId === targetId)
239
246
  tab.#closed = true;
240
247
  if (method === 'Target.detachedFromTarget' && params?.sessionId === tab.sessionId && !sessionId)
241
248
  tab.#closed = true;
242
249
  }));
250
+ tab.#unsubscribe.push(tab.#downloads.watch());
243
251
  await cdp.send('Target.setDiscoverTargets', { discover: true }).catch(() => undefined);
244
252
  return tab;
245
253
  }
@@ -376,29 +384,64 @@ export class Tab {
376
384
  if (files && !this.#files)
377
385
  void this.#call('Page.setInterceptFileChooserDialog', { enabled: true }).catch(() => undefined);
378
386
  this.#files = files;
387
+ this.#frames.files = files;
379
388
  }
380
389
  /**
381
- * Sets a declared file on an observed file input (hidden ones too), after the same element-level freshness check
382
- * as a click. The path comes from the caller's `files`, never from the model or `values`.
390
+ * Sets a declared file on an observed file input (hidden ones too, in the page or in a frame), or drops it onto an
391
+ * observed drop zone, after the same element-level freshness check as a click. The path comes from the caller's
392
+ * `files`, never from the model or `values`.
383
393
  *
384
394
  * @example
385
395
  * await tab.upload(page.actions.find((a) => a.kind === 'upload')!, page, '/Users/ada/me.png');
386
396
  */
387
397
  async upload(action, page, path) {
388
- // The file is set through the page's own session; a frame's node lives in another context, so it is refused by name.
389
- if (action.node !== undefined && idBaseOf(action.node)) {
390
- throw new TargetUnreachable(`"${action.label}" is inside a frame, and jevway uploads only to file fields in the page itself`);
391
- }
392
398
  if (!(await this.fresh(page, action)))
393
399
  throw new StalePage('Page changed since this decision');
394
- const { result } = await this.#call('Runtime.evaluate', {
395
- expression: `(() => { const e = window.__jevFast?.nodes.get(${Number(action.node)}); return e?.isConnected && e.matches('input[type=file]:not(:disabled)') ? e : null; })()`,
400
+ if (action.drop)
401
+ return this.#drop(action, path);
402
+ // A frame's node lives in the frame's own world, and its file is set through the session that owns the frame.
403
+ const frame = this.#frameOf(action.node);
404
+ const expression = `(() => { const e = window.__jevFast?.nodes.get(${Number(action.node)}); return e?.isConnected && e.matches('input[type=file]:not(:disabled)') ? e : null; })()`;
405
+ const { result } = await (frame ? this.#frames.evaluate(frame, expression, { returnByValue: false }) : this.#call('Runtime.evaluate', { expression })).catch((error) => {
406
+ throw error instanceof CdpError && !TIMED_OUT.test(error.message) ? new StalePage(error.message) : error;
396
407
  });
397
408
  if (!result?.objectId)
398
409
  throw new TargetUnreachable('The file field is gone or disabled');
399
- await this.#call('DOM.setFileInputFiles', { files: [path], objectId: result.objectId });
410
+ await this.cdp.send('DOM.setFileInputFiles', { files: [path], objectId: result.objectId }, frame?.sessionId ?? this.sessionId);
400
411
  this.#afterInput = action;
401
412
  }
413
+ // A drop zone with no file input: a trusted drag of the file onto its visible centre, checked to have reached the page.
414
+ async #drop(action, path) {
415
+ const frame = this.#frameOf(action.node);
416
+ const resolved = await this.#evaluateFor(frame, `(${RESOLVE_TARGET})(${JSON.stringify(action)})`);
417
+ if (!resolved || resolved.covered)
418
+ throw new TargetUnreachable('Target changed or is covered', Boolean(resolved?.byPopup), resolved?.coverNodes ?? []);
419
+ const target = frame ? await this.#frames.toViewport(frame, resolved) : resolved;
420
+ if ('covered' in target)
421
+ throw new TargetUnreachable('Target changed or is covered', false, target.coverNodes);
422
+ await this.#evaluateFor(frame, WATCH_DROP);
423
+ const data = { items: [], files: [path], dragOperationsMask: DRAG_COPY };
424
+ for (const type of ['dragEnter', 'dragOver', 'drop'])
425
+ await this.#call('Input.dispatchDragEvent', { type, x: target.x, y: target.y, data });
426
+ if (!(await this.#evaluateFor(frame, 'window.__jevDropped===true')))
427
+ throw new TargetUnreachable('The drop zone did not take the file');
428
+ this.#afterInput = action;
429
+ }
430
+ /**
431
+ * The downloads this tab (its page, its frames, a tab it opened) started since the last call, after waiting up to
432
+ * `waitMs` for those still running; the run's stop signal ends the wait early.
433
+ *
434
+ * @example
435
+ * await tab.downloadsAfterInput(); // [{ name: 'invoice-1042.pdf', state: 'completed', bytes: 86016, filePath, staged: true, … }]
436
+ */
437
+ downloadsAfterInput({ waitMs = DOWNLOAD_WAIT_MS } = {}) {
438
+ return this.#downloads.settle({ waitMs, signal: this.#stop });
439
+ }
440
+ /** The page as a PDF (its print styles, backgrounds included); headless and headed Chrome both print. */
441
+ async pdf() {
442
+ const { data } = await this.#call('Page.printToPDF', { printBackground: true });
443
+ return Buffer.from(data, 'base64');
444
+ }
402
445
  async observe() {
403
446
  if (this.#closed)
404
447
  throw new StalePage('The tab was closed');
@@ -845,6 +888,12 @@ const INTERACTIVE = 'a,button,input,select,textarea,[contenteditable="true"],' +
845
888
  const FIELD_LIKE = 'input:not([type="checkbox"],[type="radio"],[type="button"],[type="submit"],[type="reset"],[type="image"]),textarea,' +
846
889
  '[contenteditable="true"],[role="combobox"],[role="textbox"],[role="searchbox"],[role="spinbutton"]';
847
890
  // Resolves the observed node, rejects hidden/disabled/covered targets, and applies native selects in-page.
891
+ // How long a step waits for the downloads it started to finish.
892
+ const DOWNLOAD_WAIT_MS = 30_000;
893
+ // Input.dispatchDragEvent's operation mask for a copy, as a file dragged from the desktop offers.
894
+ const DRAG_COPY = 1;
895
+ // Notes a drop reaching the document (in the target's world): a page that refuses the drag never gets one.
896
+ const WATCH_DROP = `(() => { window.__jevDropped = false; window.addEventListener('drop', () => { window.__jevDropped = true; }, { capture: true, once: true }); return true; })()`;
848
897
  const RESOLVE_TARGET = `action => {
849
898
  const e=window.__jevFast?.nodes.get(action.node);
850
899
  if (!e?.isConnected || e.matches(':disabled') || e.closest('[aria-disabled="true"],[inert]') ||
@@ -3,6 +3,7 @@ import { SETUP_USAGE, runSetup } from '../setup/setup-command.js';
3
3
  import { SKILL_USAGE, runSkill } from '../setup/skill-command.js';
4
4
  import { closestName, usageError } from './flags.js';
5
5
  import { topHelp } from './help.js';
6
+ import { inboxCommand } from './inbox-command.js';
6
7
  import { journeyCommand } from './journey-command.js';
7
8
  import { lookCommand } from './look-command.js';
8
9
  import { printFailure } from './output.js';
@@ -12,6 +13,7 @@ const COMMANDS = {
12
13
  run: { summary: 'hand a browser goal to Jev (one-shot, or a session with --session)', run: runCommand },
13
14
  look: { summary: 'print what a page or a session\'s tab shows, with no model call', run: lookCommand },
14
15
  journey: { summary: 'map mobile user journeys: start, step, autopilot, annotate, finish', run: journeyCommand },
16
+ inbox: { summary: 'a throwaway email inbox for test and demo sign-ups: create, wait, list, read', run: inboxCommand },
15
17
  sessions: { summary: 'list the open sessions and journeys', run: sessionsCommand },
16
18
  end: { summary: 'end a session and close the tabs it opened', run: endCommand },
17
19
  serve: { summary: 'run the session daemon in the foreground', run: serveCommand },