jevway 0.2.0 → 0.4.0

package/README.md

@@ -40,6 +40,7 @@ optional tools unlock more.
 | **Google Chrome** (or Chromium) | every run: your running Chrome, or a throwaway one jevway starts | runs fail; set `CHROME_PATH` if Chrome isn't in the usual place |
 | **An API key**: `TYPESAFE_API_KEY` (recommended) or `OPENROUTER_API_KEY` | every decision Jev makes | the server still starts and lists its tools, and `jevway_look` works, but tasks and journeys answer with the setup fix |
 | `ffmpeg` *(optional)* | the `run.mp4` video in a recording | recordings keep their step screenshots and HTML page, without the video |
+| `notify-send` *(optional, Linux)* | the desktop notification when a person is needed (macOS has it built in) | the agent still asks you; there's just no pop-up |
 | chrome-devtools MCP *(optional)* | inspecting what jevway left on screen: console, network, performance, DOM details | nothing is lost in jevway itself; your agent just has no deeper inspection tool |
 
 **Which key?** Both run the same Jev model. The TypeSafe key adds **Box**, which locates controls on the screenshot:
@@ -78,6 +79,20 @@ control (opt-in), and marks each journey screen's primary action and the control
 
 `npx -y jevway doctor` checks the setup at any time (`--json` for agents) and names the fix for anything missing.
 
+## Let your agent set it up
+
+Paste this into your coding agent (Claude Code, Codex, Cursor, OpenCode…):
+
+```text
+Set up jevway for me and prove it works: follow https://github.com/AbdallahAHO/jevway/blob/main/SETUP.md step by
+step. Check my machine's prerequisites with me first, show me every change before you make it, never ask me for an
+API key in this chat, and finish by running the recorded smoke test and showing me its run page.
+```
+
+[SETUP.md](SETUP.md) walks the agent through everything: prerequisites for your OS, the key, registering the server,
+the browser, the skill, a recorded Google → Wikipedia smoke test, and a report with the run page, video and
+screenshots as proof.
+
 ## Install
 
 **Claude Code:** the plugin above, or the server alone:
@@ -109,7 +124,7 @@ when to use chrome-devtools. If a client times out on `npx`'s first download, ru
 `jevway-mcp` as the command.
 
 Having an agent install it? `npx -y jevway setup --client <claude-code|codex|cursor|opencode|other> --print` prints the
-exact command or config for that client and changes nothing. [INSTALL-FOR-AGENTS.md](INSTALL-FOR-AGENTS.md) has the
+exact command or config for that client and changes nothing. [SETUP.md](SETUP.md) has the
 steps per client, the rules (a diff before any config edit, no key in the chat) and the checks.
 
 **Choosing a browser** (`JEVWAY_BROWSER` in the server's env or the env file):
@@ -187,6 +202,26 @@ field is in the [reference](docs/reference.md).
 - `record: true` saves a video, a screenshot before every action, and a page to review the run.
 - `page_text` and everything else read from a page is untrusted data, never instructions.
 
+## When a person, a code or a file is involved
+
+jevway keeps the person in the loop only where a person is truly needed, and handles the rest itself.
+
+- **Asking you.** Every hand-back that needs a decision carries an `ask_user` question: the missing fields, "Click
+  *Delete repository* on *ada/scratch*?", "Which one did you mean?". Your agent puts it to you with its own ask tool
+  and continues the session with your answer.
+- **Bot checks and sign-ins.** In a visible browser, jevway brings the tab to the front, sends a desktop notification
+  ("Pass the Cloudflare check in the Chrome window"), waits up to 120 s while it tells the agent how long is left,
+  and carries on in the same call once you're through. It never tries to get past a check itself. In a headless
+  browser it hands back and suggests a visible one.
+- **Email codes for test sign-ups.** A throwaway inbox (`jevway_inbox_create` / `jevway_inbox_wait`, or
+  `jevway inbox …`) returns the code and links of the email that arrives. Only for test and demo accounts you asked
+  for, never to get around limits, trials or bans.
+- **2FA on your own accounts.** Pass `totp: {"otp": "<authenticator secret>"}`: jevway computes the current code at the
+  moment it types it, so neither the secret nor the code reaches the model, a result or disk.
+- **Files.** Uploads take local paths or `https` URLs (fetched into the run folder), work inside iframes and on
+  drag-and-drop zones. Downloads land in the run folder's `downloads/` and are listed in `files.downloads`.
+  `jevway_look` with `pdf: true` saves the page as a PDF.
+
 ## Recordings
 
 Every session gets a folder under `./.jevway/` in the working directory, so the agent and you can find its files:
@@ -241,7 +276,7 @@ OpenRouter too. Jev is priced at $0.042 per 1M input tokens, so a typical 10-ste
 | [docs/reference.md](docs/reference.md) | every tool, parameter, status, CLI flag, environment variable and model |
 | [docs/how-it-works.md](docs/how-it-works.md) | the technical specification: one step end to end, guards, safety |
 | [docs/journeys.md](docs/journeys.md) | the journey-map tools |
-| [INSTALL-FOR-AGENTS.md](INSTALL-FOR-AGENTS.md) | agents installing jevway for a person |
+| [SETUP.md](SETUP.md) | agents setting jevway up for a person, with a recorded smoke test |
 | [skill/jevway/SKILL.md](skill/jevway/SKILL.md) | the skill agents load: when and how to call jevway |
 | [docs/verification.md](docs/verification.md) | the test suites and their results |
 | [docs/prior-art.md](docs/prior-art.md) | other open-source Jev browser projects, and what was taken from them |

package/{INSTALL-FOR-AGENTS.md → SETUP.md}

@@ -1,20 +1,34 @@
-# Installing jevway (for agents)
-
-You are an agent asked to install jevway for a person. jevway is an MCP server, `jevway-mcp`, plus a CLI, `jevway`.
-It hands browser goals to TypeSafe Jev. Both come from the npm package `jevway`.
-
-Read this whole file before you run anything. Run only these:
+# Setting up jevway (for agents)
+
+You are an agent asked to set up jevway for a person and prove it works. jevway is an MCP server, `jevway-mcp`, plus a
+CLI, `jevway`, both from the npm package `jevway`. It hands browser goals to TypeSafe Jev and returns one checked
+summary per goal.
+
+Read this whole file before you run anything. You will:
+1. check the machine's prerequisites with the person and help install what's missing;
+2. get the API key in place (the person does that part);
+3. register the MCP server with the client you're running in;
+4. pick the browser with the person;
+5. install the skill;
+6. check the setup;
+7. run a recorded smoke test, from Google to Wikipedia;
+8. report back with the run page, its video and screenshots, as proof.
+
+Run only these yourself:
+- `node --version`, `ffmpeg -version`, and the person-approved install commands from step 1;
 - `npx -y jevway doctor --json` (or `--plan`, or `jevway doctor`);
 - `npx -y jevway setup --client <name> --print [--json]`, which prints how to register jevway with a client and
   changes nothing;
 - `npx -y jevway skill --install <skills folder>`;
 - `npm install -g jevway`;
-- the client's own `mcp add` command, or an edit to its MCP config file that the person approved.
+- the client's own `mcp add` command, or an edit to its MCP config file that the person approved;
+- the jevway tools themselves (or `npx -y jevway run …`) for the smoke test.
 
 Never run plain `jevway setup` yourself: it is the person's interactive setup, and without a terminal it exits 2 with
 `{ "ok": false, "error": { "code": "needs_tty", … } }`.
 
-Done means: `jevway doctor --json` prints `"ready": true` (exit 0), and the client lists the `jevway_*` tools.
+Done means: `jevway doctor --json` prints `"ready": true`, the client lists the `jevway_*` tools, and the smoke test
+ended `done` with a run page you showed the person.
 
 ## Rules
 
@@ -45,6 +59,19 @@ It needs no key to run and never prints one. It checks:
 - the browser source;
 - ffmpeg (optional: only for recordings).
 
+`npx` itself needs Node, so check `node --version` first. For anything missing, propose the command that fits the
+person's OS and package manager, and run it only after they say yes (or wait while they install it themselves, then
+check again):
+
+| Tool | Needed for | macOS (Homebrew) | Debian / Ubuntu | Fedora | Windows (winget) |
+| --- | --- | --- | --- | --- | --- |
+| Node.js 24+ | running jevway (required) | `brew install node` | Node 24 from NodeSource or nvm | `sudo dnf install nodejs` (check it's 24+) | `winget install OpenJS.NodeJS` |
+| Chrome or Chromium | every run (required) | `brew install --cask google-chrome` | Google's `.deb`, or `sudo apt install chromium` | `sudo dnf install chromium` | `winget install Google.Chrome` |
+| ffmpeg | the video in a recording (optional) | `brew install ffmpeg` | `sudo apt install ffmpeg` | `sudo dnf install ffmpeg-free` | `winget install Gyan.FFmpeg` |
+| notify-send | the desktop notification when a person is needed (optional, Linux) | built in | `sudo apt install libnotify-bin` | `sudo dnf install libnotify` | not used |
+
+A Chrome in an unusual place: set `CHROME_PATH` in `~/.config/jevway/env`.
+
 If the first `npx` start is slow, or a client times out starting the server, install once with `npm install -g jevway`.
 Then use `jevway-mcp` as the command instead of `npx -y -p jevway jevway-mcp`.
 
@@ -97,7 +124,8 @@ npx -y jevway setup --client cursor --print --json # { client, kind, command?, f
 
 ## 4. The browser
 
-`JEVWAY_BROWSER` (the server's env, or `--browser` for the CLI):
+Ask the person which browser jevway should use, and set `JEVWAY_BROWSER` in the server's env (or `--browser` for
+the CLI) when it isn't `auto`. For the smoke test a visible one is best, so they can pass a check if one appears.
 
 | Value | What it uses | What the person does |
 | --- | --- | --- |
@@ -117,11 +145,13 @@ npx -y jevway skill --install ~/.codex/skills   # or ~/.claude/skills, ~/.cursor
 It prints `{ ok, status, path, files }`. A different `jevway` skill already there (or a symlink to one) is left alone
 with `"code": "skill_exists"` and exit 1; replace it with `--force` only if the person agrees.
 
-## 6. Check the outcome
+## 6. Check the setup
 
-1. `npx -y jevway doctor --json` prints `"ready": true` and exits 0 (every finding `ok` or `warn`).
-2. The client lists the tools: `jevway_task`, `jevway_look`, `jevway_end` and `jevway_journey_start`, `_step`,
-   `_autopilot`, `_annotate`, `_finish`.
+1. `npx -y jevway doctor --json` prints `"ready": true` and exits 0 (every finding `ok` or `warn`). For every finding
+   that isn't `ok`, tell the person the fix in one line and wait; re-run it after each fix (`--plan` lists the steps
+   left).
+2. The client lists the tools: `jevway_task`, `jevway_look`, `jevway_end`, the five `jevway_journey_*` tools and
+   `jevway_inbox_create` / `jevway_inbox_wait`.
    - In Claude Code: `/mcp`.
    - Elsewhere, the client's MCP list.
    - Or the MCP Inspector:
@@ -129,4 +159,54 @@ with `"code": "skill_exists"` and exit 1; replace it with `--force` only if the
 3. Optional, costs a fraction of a cent: `npx -y jevway doctor --live` makes one real Jev request and reports its
    latency.
 
-When a check fails, the doctor names the cause and the fix, and `--plan` lists the steps left. Re-run it after each fix.
+## 7. The smoke test
+
+Call `jevway_task` with:
+
+```json
+{
+  "url": "https://www.google.com",
+  "goal": "Search Google for the Rosetta Stone and open its English Wikipedia article",
+  "values": { "query": "Rosetta Stone Wikipedia" },
+  "done_when": "The English Wikipedia article about the Rosetta Stone is open",
+  "allowed_hosts": ["google.com", "*.google.com", "en.wikipedia.org"],
+  "record": true,
+  "detail": "full"
+}
+```
+
+What to expect, and what to do:
+- **A cookie banner** is handled by jevway (it declines when it can).
+- **A reCAPTCHA ("unusual traffic")** is common for automated browsers on Google. In a visible browser jevway brings
+  the tab to the front, sends the person a desktop notification and waits up to 2 minutes for them to pass it; it
+  never solves it. Tell the person it may happen before you start.
+- **A result with `ask_user`** (a bot check, a sign-in, a missing value): put that question to the person with your
+  own ask tool, then do what its `then` says.
+- **Google keeps blocking**: run the same goal from `https://duckduckgo.com` with
+  `allowed_hosts: ["duckduckgo.com", "en.wikipedia.org"]`. It usually finishes in 5 steps.
+- **No MCP in your client**: run the same test with the CLI, which prints one JSON result on stdout:
+
+  ```bash
+  npx -y jevway run --browser launch --record --detail full --url https://www.google.com \
+    --goal "Search Google for the Rosetta Stone and open its English Wikipedia article" \
+    --values '{"query":"Rosetta Stone Wikipedia"}' \
+    --done-when "The English Wikipedia article about the Rosetta Stone is open" \
+    --allowed-hosts "google.com,*.google.com,en.wikipedia.org"
+  ```
+
+End the session with `jevway_end` when you have what you need.
+
+## 8. Report back
+
+Tell the person, in this shape:
+- **Outcome:** `status` and `reason`, and the `story` as a numbered list.
+- **Proof:** the run page `files.page` (open it for them: `open` on macOS, `xdg-open` on Linux, `start` on Windows),
+  the video `files.video` (present when ffmpeg is installed), the end screenshot `files.screenshot` (show it if you
+  can display images), and the run folder `files.run_dir` with its `manifest.json`.
+- **Cost:** `jev.requests` and `jev.est_cost_usd` (an estimate; a typical smoke test is well under a cent).
+- **Setup:** what you installed or changed, with the person's yes for each.
+- **Anything that needed them**, and anything that didn't work, with the fix.
+
+The run page opens with the goal and how it ended, then the video with every step on a timeline and a strip of
+screenshots kept in sync. It is the proof that the whole chain works: the client, the MCP server, the key, the
+browser and Jev.

package/dist/browser/code-boxes.js

@@ -0,0 +1,121 @@
+const MIN_BOXES = 4;
+const MAX_BOXES = 8;
+// One character fits in a box this wide at any common font size; a field for a word is wider.
+const ONE_CHARACTER_PX = 64;
+const CODE_MODES = ['numeric', 'decimal', 'text', 'tel'];
+const CODE_TYPES = ['tel', 'number'];
+// Words a box's own label shares with its neighbours that name no field ("Digit 1 of 6").
+const GENERIC_WORDS = /^(?:digit|character|char|box|number|input|field|code|otp|pin|textbox|spinbutton)s?$/i;
+const FALLBACK_NAME = 'Verification code';
+const holdsOneCharacter = (action) => {
+    const { box, rect } = action;
+    if (action.kind !== 'fill' || !box || !rect || action.hover)
+        return false;
+    if (box.max === 1)
+        return true;
+    return box.max < 0 && rect.w <= ONE_CHARACTER_PX && (CODE_MODES.includes(box.mode) || CODE_TYPES.includes(box.type) || box.otp);
+};
+const centreY = (box) => box.rect.y + box.rect.h / 2;
+// One form, one container (every box in the same parent, or each wrapped once in the same grandparent), one row, read
+// left to right, all about the same width.
+const joins = (run, next) => {
+    const [first] = run;
+    const last = run.at(-1);
+    const widths = [...run, next].map((box) => box.rect.w);
+    return (next.box.form === first.box.form &&
+        [0, 1].some((level) => run.every((box) => box.box.within[level] === next.box.within[level])) &&
+        Math.abs(centreY(next) - centreY(first)) <= first.rect.h / 2 &&
+        next.rect.x > last.rect.x &&
+        Math.max(...widths) <= Math.min(...widths) * 1.5);
+};
+// Runs of one-character boxes that follow one another among the listed fields; another field between them ends a run.
+const runsOf = (actions) => {
+    const runs = [];
+    let run = [];
+    for (const action of actions) {
+        if (action.kind !== 'fill')
+            continue;
+        if (holdsOneCharacter(action) && run.length && joins(run, action)) {
+            run.push(action);
+            continue;
+        }
+        if (run.length)
+            runs.push(run);
+        run = holdsOneCharacter(action) ? [action] : [];
+    }
+    if (run.length)
+        runs.push(run);
+    return runs.filter((boxes) => boxes.length >= MIN_BOXES && boxes.length <= MAX_BOXES);
+};
+// "Security code character 3" → "Security code character": a box's label without its place in the group.
+const stemOf = (label) => label
+    .replace(/\d+/g, ' ')
+    .replace(/\bof\b/gi, ' ')
+    .replace(/^[^\p{L}]+|[^\p{L}]+$/gu, '')
+    .replace(/\s+/g, ' ');
+const nameOf = (run) => {
+    if (run[0].box.name)
+        return run[0].box.name;
+    const stems = new Set(run.map((box) => stemOf(box.label)));
+    const [stem] = stems;
+    return stems.size === 1 && stem && !GENERIC_WORDS.test(stem) ? stem : FALLBACK_NAME;
+};
+const unionOf = (run) => {
+    const left = Math.min(...run.map((box) => box.rect.x));
+    const top = Math.min(...run.map((box) => box.rect.y));
+    const right = Math.max(...run.map((box) => box.rect.x + box.rect.w));
+    const bottom = Math.max(...run.map((box) => box.rect.y + box.rect.h));
+    return { x: left, y: top, w: right - left, h: bottom - top };
+};
+const withoutFacts = ({ box: _box, ...action }) => action;
+/**
+ * A code split into one-character boxes (4–8 `maxlength=1` boxes, or narrow numeric/one-time-code ones with no
+ * `maxlength`, following one another in one form and container, left to right on one row) becomes one `fill` target on
+ * its first box: `boxes` lists every box, `value` joins their values, `rect` covers them all, and the label is the
+ * group's name with the count. The other boxes and their `Open …` clicks are not listed. The reader's facts are removed
+ * from every action, and element ids are renumbered when anything was grouped.
+ *
+ * @example
+ * groupCodeBoxes(snapshot.actions) // [{ kind: 'fill', node: 11, label: 'Verification code (6 boxes)', boxes: [11, …, 16], … }, …]
+ */
+export const groupCodeBoxes = (actions) => {
+    const runs = runsOf(actions);
+    if (!runs.length)
+        return actions.some((action) => action.box) ? actions.map(withoutFacts) : actions;
+    const groupOf = new Map();
+    for (const run of runs) {
+        const group = {
+            first: run[0].node,
+            boxes: run.map((box) => box.node),
+            label: `${nameOf(run)} (${run.length} boxes)`,
+            value: run.map((box) => box.value ?? '').join(''),
+            rect: unionOf(run),
+        };
+        for (const node of group.boxes)
+            groupOf.set(node, group);
+    }
+    let next = 0;
+    return actions.flatMap((action) => {
+        const group = action.node === undefined ? undefined : groupOf.get(action.node);
+        if (group && action.node !== group.first)
+            return [];
+        const { label, value, boxes, rect } = group ?? {};
+        const listed = group ? { ...withoutFacts(action), label: action.kind === 'click' ? `Open ${label}` : label, value, boxes, rect } : withoutFacts(action);
+        return [action.node === undefined ? listed : { ...listed, id: `e${++next}` }];
+    });
+};
+/**
+ * The characters a code puts into `count` boxes, one each: the text as given, or without its spaces and hyphens when
+ * only that fits ("739 214" into 6 boxes). Undefined when neither fits.
+ *
+ * @example
+ * codeCharacters('739-214', 6) // ['7', '3', '9', '2', '1', '4']
+ */
+export const codeCharacters = (text, count) => {
+    for (const form of [text, text.replace(/[\s-]/g, '')]) {
+        const characters = [...form];
+        if (characters.length === count)
+            return characters;
+    }
+    return undefined;
+};

package/dist/browser/downloads.js

@@ -0,0 +1,103 @@
+import { mkdtempSync, rmSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { join, relative, isAbsolute } from 'node:path';
+const STAGING = new WeakMap();
+/**
+ * Sets how the browser behind `cdp` downloads, once per connection: jevway's launched browser saves every download under
+ * its guid in a staging folder of its own; the person's Chrome (`auto`, a debugging URL) keeps its own behaviour and only
+ * reports downloads. A browser that refuses never fails a run: its downloads go unreported.
+ *
+ * @example
+ * await useDownloads(cdp, 'launch-headless'); // Browser.setDownloadBehavior { behavior: 'allowAndName', downloadPath: '/tmp/jevway-downloads-…' }
+ */
+export const useDownloads = async (cdp, source) => {
+    if (source !== 'launch' && source !== 'launch-headless') {
+        await cdp.send('Browser.setDownloadBehavior', { behavior: 'default', eventsEnabled: true }).catch(() => undefined);
+        return;
+    }
+    const downloadPath = mkdtempSync(join(tmpdir(), 'jevway-downloads-'));
+    STAGING.set(cdp, downloadPath);
+    await cdp.send('Browser.setDownloadBehavior', { behavior: 'allowAndName', downloadPath, eventsEnabled: true }).catch(() => undefined);
+};
+/** Removes the staging folder of a launched browser's connection, with whatever was never kept. */
+export const dropStagedDownloads = (cdp) => {
+    const dir = STAGING.get(cdp);
+    if (dir)
+        rmSync(dir, { recursive: true, force: true });
+    STAGING.delete(cdp);
+};
+const inside = (dir, path) => {
+    if (!dir || !path)
+        return false;
+    const rel = relative(dir, path);
+    return Boolean(rel) && !rel.startsWith('..') && !isAbsolute(rel);
+};
+/**
+ * The downloads one tab started: `owns` says whether a frame id is the tab's (its page, one of its frames, a tab it
+ * opened). Every other download, in the person's other tabs too, is never recorded.
+ *
+ * @example
+ * const downloads = new TabDownloads(cdp, (frameId) => frameId === targetId);
+ * const finished = await downloads.settle({ waitMs: 30_000 });
+ */
+export class TabDownloads {
+    #cdp;
+    #owns;
+    #all = new Map();
+    #untaken = [];
+    #changed = new Set();
+    constructor(cdp, owns) {
+        this.#cdp = cdp;
+        this.#owns = owns;
+    }
+    /** Starts listening; returns the unsubscribe. */
+    watch() {
+        return this.#cdp.onEvent((method, params, sessionId) => {
+            if (sessionId || !method.startsWith('Browser.download'))
+                return;
+            if (method === 'Browser.downloadWillBegin' && this.#owns(params.frameId)) {
+                this.#all.set(params.guid, { guid: params.guid, name: params.suggestedFilename, url: params.url, state: 'inProgress', bytes: 0, staged: false });
+                this.#untaken.push(params.guid);
+            }
+            const known = method === 'Browser.downloadProgress' && this.#all.get(params.guid);
+            if (!known)
+                return;
+            const filePath = params.filePath ?? known.filePath;
+            this.#all.set(params.guid, {
+                ...known,
+                state: params.state,
+                bytes: params.receivedBytes ?? known.bytes,
+                ...(filePath && { filePath }),
+                staged: inside(STAGING.get(this.#cdp), filePath),
+            });
+            for (const changed of this.#changed)
+                changed();
+        });
+    }
+    /**
+     * Takes the downloads begun since the last take, after waiting up to `waitMs` (or until `signal`) for those still
+     * running. Each comes back as it stands then: completed, canceled, or still in progress.
+     */
+    async settle({ waitMs, signal }) {
+        const taken = this.#untaken.splice(0);
+        const running = () => taken.some((guid) => this.#all.get(guid)?.state === 'inProgress');
+        if (taken.length && running() && !signal?.aborted) {
+            await new Promise((resolve) => {
+                const done = () => {
+                    clearTimeout(timer);
+                    signal?.removeEventListener('abort', done);
+                    this.#changed.delete(check);
+                    resolve();
+                };
+                const check = () => {
+                    if (!running())
+                        done();
+                };
+                const timer = setTimeout(done, waitMs);
+                signal?.addEventListener('abort', done, { once: true });
+                this.#changed.add(check);
+            });
+        }
+        return taken.map((guid) => ({ ...this.#all.get(guid) }));
+    }
+}

package/dist/browser/frames.js

@@ -1,5 +1,6 @@
 import { centerOf, contains } from '../jev/box.js';
 import { CdpError } from './cdp.js';
+import { groupCodeBoxes } from './code-boxes.js';
 // jevway: every frame is read with the same reader (snapshot.js), each in an isolated world of its own, so ids, guards and
 // the input counter live where the frame's elements live. The main frame keeps the main world, exactly as before frames.
 /** Each frame owns a range of element ids this wide; the main frame is range 0, so its ids are what they always were. */
@@ -134,6 +135,7 @@ export class Frames {
     #sessionId;
     #counter;
     #touch = false;
+    #files = false;
     #mainFrameId = '';
     #viewport = { w: 0, h: 0 };
     /** Out-of-process frames attached so far, by target id (= frame id). */
@@ -164,6 +166,20 @@ export class Frames {
         for (const { sessionId } of this.#children.values())
             void this.#emulateTouch(sessionId);
     }
+    /**
+     * The caller declared files: frame reads list file inputs and drop zones, and out-of-process frames keep native file
+     * pickers closed too (the page's own session does that for same-process frames).
+     */
+    set files(enabled) {
+        if (enabled && !this.#files)
+            for (const { sessionId } of this.#children.values())
+                void this.#interceptFileChooser(sessionId);
+        this.#files = enabled;
+    }
+    /** Whether a protocol frame id is this tab's: its page, a frame read on it, or an out-of-process frame attached to it. */
+    holds(frameId) {
+        return Boolean(frameId) && (frameId === this.#mainFrameId || this.#known.has(frameId) || this.#children.has(frameId));
+    }
     /** The frame holding an element id: undefined for the main frame and for a frame no longer on the page. */
     ofNode(node) {
         const base = idBaseOf(node);
@@ -228,11 +244,11 @@ export class Frames {
         // The text walk keeps to the part of the frame the page shows, in the frame's own coordinates.
         const { offset, region } = placement;
         const view = { x: region.x - offset.x, y: region.y - offset.y, w: region.w, h: region.h };
-        const response = await this.evaluate(frame, `window.__jevIdBase=${frame.base};window.__jevView=${JSON.stringify(view)};window.__jevWanted=${JSON.stringify(wanted)};${script}`, { timeout: FRAME_READ_MS });
+        const response = await this.evaluate(frame, `window.__jevIdBase=${frame.base};window.__jevView=${JSON.stringify(view)};window.__jevWanted=${JSON.stringify(wanted)};window.__jevFiles=${this.#files};${script}`, { timeout: FRAME_READ_MS });
         const state = response.exceptionDetails ? null : response.result?.value;
         if (!state)
             return null;
-        const placed = placeFrameActions(state.actions, { w: state.w, h: state.h }, placement, frame.id);
+        const placed = placeFrameActions(groupCodeBoxes(state.actions), { w: state.w, h: state.h }, placement, frame.id);
         // On-screen controls must be on top in every document around the frame: a banner or sheet over the frame hides them.
         const shown = placed.filter((action) => contains(placement.region, centerOf(action.rect)));
         const onTop = await this.#hits(levels, shown.map((action) => centerOf(action.rect)));
@@ -404,6 +420,8 @@ export class Frames {
             await this.#send('Page.addScriptToEvaluateOnNewDocument', { source: this.#counter, worldName: WORLD }, sessionId);
             await this.#send('Emulation.setFocusEmulationEnabled', { enabled: true }, sessionId).catch(() => undefined);
             await this.#emulateTouch(sessionId);
+            if (this.#files)
+                await this.#interceptFileChooser(sessionId);
             await this.#autoAttach(sessionId);
         }
         catch {
@@ -418,6 +436,9 @@ export class Frames {
         const enabled = this.#touch;
         return this.#send('Emulation.setTouchEmulationEnabled', { enabled, maxTouchPoints: enabled ? 5 : 1 }, sessionId).catch(() => undefined);
     }
+    #interceptFileChooser(sessionId) {
+        return this.#send('Page.setInterceptFileChooserDialog', { enabled: true }, sessionId).catch(() => undefined);
+    }
     #autoAttach(sessionId) {
         return this.#send('Target.setAutoAttach', { autoAttach: true, waitForDebuggerOnStart: true, flatten: true, filter: [{ type: 'iframe' }] }, sessionId);
     }

package/dist/browser/snapshot.js

@@ -332,6 +332,34 @@
       if (['auto', 'scroll'].includes(getComputedStyle(p).overflowX) && p.scrollWidth > p.clientWidth) return row;
     return null;
   };
+  // jevway: a short text input may be one box of a code split into one-character boxes. The tab groups them from these
+  // facts (code-boxes.ts); only the group's name is read here, where the page is: an aria name or legend around the
+  // boxes, else a short line right before them. Container and form keys are unique within this read.
+  const containerKeys = new Map();
+  const keyOf = (n) => (n ? containerKeys.get(n) ?? containerKeys.set(n, containerKeys.size + 1).get(n) : 0);
+  const codeGroupName = (e) => {
+    for (let p = e.parentElement, up = 0; p && up < 3 && p !== document.body && p.tagName !== 'FORM'; p = p.parentElement, up++) {
+      const legend = p.tagName === 'FIELDSET' ? p.querySelector('legend') : null;
+      const own = p.getAttribute('aria-label') || (p.hasAttribute('aria-labelledby') ? name(p) : '') || legend?.innerText || '';
+      if (own.trim()) return cap(own.replace(/\s+/g, ' ').trim(), 80);
+    }
+    for (let p = e, up = 0; p && up < 3 && p !== document.body; p = p.parentElement, up++) {
+      const before = p.previousElementSibling;
+      if (!before || before.matches(interactive) || before.querySelector(interactive) || !visible(before)) continue;
+      const line = before.innerText.replace(/\s+/g, ' ').trim();
+      if (line) return line.length <= 80 ? line : '';
+    }
+    return '';
+  };
+  const codeBoxFacts = (e) => {
+    if (e.tagName !== 'INPUT') return null;
+    const max = e.maxLength;
+    if (!(max >= 1 && max <= 4) && !(max < 0 && e.getBoundingClientRect().width <= 64)) return null;
+    const parent = e.parentElement;
+    const within = [keyOf(parent), keyOf(parent?.parentElement)];
+    const facts = { max, mode: e.inputMode || '', otp: e.autocomplete === 'one-time-code', type: e.type, form: keyOf(e.form), within };
+    return { box: { ...facts, name: codeGroupName(e) } };
+  };
   for (const e of candidates) {
     const shown = eligible(e);
     const hover = !shown && revealedByHover(e);
@@ -402,7 +430,7 @@
           (rname === 'combobox' && (['INPUT', 'TEXTAREA'].includes(e.tagName) || e.isContentEditable)));
       const value =
         'value' in e ? String(e.value) : e.isContentEditable || rname === 'combobox' ? e.innerText.trim() : '';
-      actions.push({ ...base, kind: editable ? 'fill' : 'click', value });
+      actions.push({ ...base, kind: editable ? 'fill' : 'click', value, ...(editable && !hover && codeBoxFacts(e)) });
       if (editable) actions.push({ ...base, kind: 'click', value, label: 'Open ' + base.label });
     }
   }
@@ -424,10 +452,12 @@
       }
       return null;
     };
+    const anchors = [];
     for (const input of fileInputs) {
       if (input.matches(':disabled') || input.closest('[aria-disabled="true"],[inert]')) continue;
       const anchor = standInFor(input);
       if (!anchor || outOfView(anchor.getBoundingClientRect())) continue;
+      anchors.push(anchor);
       const r = cache.clickRect(anchor),
         own = (anchor === input ? name(input) : name(input) || name(anchor)).replace(/\s+/g, ' ').trim();
       actions.push({
@@ -439,6 +469,57 @@
         rect: { x: r.x, y: r.y, w: r.width, h: r.height },
       });
     }
+    // jevway: a drop zone with no file input in it is listed only on clear markers: an ondrop attribute, a drop-zone
+    // name in its class, id or a data- attribute, or a short text asking for a file to be dropped ("Drag and drop",
+    // "Drop your CV here"; its nearest box of 40×40 or more). The innermost zone wins; one holding or inside a file field's stand-in
+    // is that field already. The file is dropped onto it.
+    const DROP_NAME = /drop-?zone|drop-?area|file-?drop|drop-?target/i;
+    const DROP_TEXT =
+      /\bdrag\s*(?:and|&|'?n'?)\s*drop\b|\bdrop\s+(?:your\s+|a\s+|an\s+|the\s+)?(?:files?|cv|resume|résumé|images?|photos?|documents?|attachments?)\b/i;
+    const named = (e) =>
+      e.hasAttribute('ondrop') ||
+      DROP_NAME.test(typeof e.className === 'string' ? e.className : '') ||
+      DROP_NAME.test(e.id) ||
+      [...e.attributes].some((a) => a.name.startsWith('data-') && DROP_NAME.test(a.name));
+    const big = (e) => {
+      const r = e.getBoundingClientRect();
+      return r.width >= 40 && r.height >= 40;
+    };
+    const zones = new Set(
+      [...document.querySelectorAll('[ondrop],[class*="drop" i],[id*="drop" i]')].filter(named),
+    );
+    const texts = document.createTreeWalker(document.body ?? document.documentElement, NodeFilter.SHOW_TEXT, {
+      acceptNode: (t) => (t.data.length <= 200 && DROP_TEXT.test(t.data) ? NodeFilter.FILTER_ACCEPT : NodeFilter.FILTER_SKIP),
+    });
+    for (let t = texts.nextNode(), seen = 0; t && seen < 20; t = texts.nextNode(), seen++) {
+      let box = t.parentElement;
+      for (let up = 0; box && up < 4 && !big(box); up++) box = box.parentElement;
+      if (box && big(box)) zones.add(box);
+    }
+    const listed = [...zones].filter(
+      (zone) =>
+        zone !== document.body &&
+        zone !== document.documentElement &&
+        visible(zone) &&
+        big(zone) &&
+        !outOfView(zone.getBoundingClientRect()) &&
+        !zone.querySelector('input[type=file]') &&
+        !anchors.some((anchor) => anchor.contains(zone) || zone.contains(anchor)),
+    );
+    for (const zone of listed) {
+      if (listed.some((other) => other !== zone && zone.contains(other))) continue;
+      const r = cache.clickRect(zone),
+        text = (zone.innerText || '').replace(/\s+/g, ' ').trim();
+      actions.push({
+        node: identity(zone),
+        role: 'file',
+        kind: 'upload',
+        drop: true,
+        label: cap(name(zone).replace(/\s+/g, ' ').trim() || text || 'drop zone', 120) + ' (drop zone)',
+        value: '',
+        rect: { x: r.x, y: r.y, w: r.width, h: r.height },
+      });
+    }
   }
   // jevway: controls sharing a role and label (a product grid's "Add to cart" buttons) are told apart by the card or
   // row each sits in, as a person reads the page: "Add to cart — Sauce Labs Bike Light". Only colliding labels change;