@revyl/verify 0.4.1 → 0.6.0

package/docs/api.md

@@ -84,16 +84,21 @@ Web only, hosted or normal. Sends one deterministic `open_url` action and checks
 
 ## Exact actions
 
-**Since 0.4.0. Early access.** In TypeScript (`@revyl/verify` 0.4.0), Python (`revyl-verify` 0.2.0) and Go (v0.3.0); the [Python](https://github.com/RevylAI/verify/blob/v0.4.1/python/README.md#exact-actions) and [Go](https://github.com/RevylAI/verify/blob/v0.4.1/go/README.md#exact-actions) READMEs show them with each language's names. Web, iOS and Android sessions take device actions that need no agent.
+**Early access.** Since TypeScript (`@revyl/verify`) 0.4.0, Python (`revyl-verify`) 0.2.0 and Go v0.3.0. `doubleTap`, `longPress`, `drag`, `typeFocused`, `goHome`, and Web keys and modifiers in `pressKey`, are new in TypeScript 0.5.0, Python 0.3.0 and Go v0.4.0. On a Mac computer's screen, `tap` (by point or by description), `screenSize`, `typeFocused` and `pressKey` are new in TypeScript 0.6.0, Python 0.4.0 and Go v0.5.0. The [Python](https://github.com/RevylAI/verify/blob/v0.6.0/python/README.md#exact-actions) and [Go](https://github.com/RevylAI/verify/blob/v0.6.0/go/README.md#exact-actions) READMEs show them with each language's names. Web, iOS and Android sessions, and a Mac computer's screen, take device actions that need no agent.
 
 | Call | Web | iOS | Android | Mac computer |
 | --- | --- | --- | --- | --- |
-| `tap`, `type`, `swipe`, `pressKey`, `locate`, `screenSize` | Yes | Yes | Yes | No, use `exec()` and files |
+| `tap({ x, y })`, `tap({ target })`, `screenSize` | Yes | Yes | Yes | Yes |
+| `locate` | Yes | Yes | Yes | No, use `tap({ target })`, which returns the point it clicked, or read the point off `screenshot()` |
+| `doubleTap`, `longPress`, `type`, `swipe`, `drag` | Yes | Yes | Yes | No |
+| `pressKey(key, { modifiers })` | Yes: `'enter'`, `'backspace'`, `'tab'`, `'escape'`, arrows and more, with modifiers | `'enter'` and `'backspace'` only | `'enter'` and `'backspace'` only | Yes: the Web keys and modifiers, where `'meta'` is Command |
+| `typeFocused(text, { clear })` | Yes, types into the focused field | No, use `type(location, text)` | No, use `type(location, text)` | Yes, with `{ clear: false }` |
 | `back()` | Yes, browser history | No, tap the on-screen back control | Yes, the system back button | No |
+| `goHome()` | No, use `openUrl` or `navigate` | Yes, the home screen | Yes, the home screen | No |
 | `openUrl(url)` | Yes, loads an http(s) URL in the tab | Yes, a URL or deep link | Yes, a URL or deep link | No |
 | `launchApp(bundleId)` | No, use `openUrl` or `navigate` | Yes | Yes | No |
 | `navigate(url)` | Yes | No, use `openUrl` | No, use `openUrl` | No |
-| `instruction`, `validation`, your own model | Yes | Yes | Yes | No, use `exec()` |
+| `instruction`, `validation`, your own model | Yes | Yes | Yes | No, use `exec()` and the screen calls above |
 | `screenshot()` | Yes | Yes | Yes | Yes |
 | `exec`, files, processes | No, open a Mac computer | No, open a Mac computer | No, open a Mac computer | Yes |
 
@@ -103,23 +108,28 @@ A location is `{ x, y }` in screenshot pixels (see [Coordinates](#coordinates))
 
 | Call | What it does |
 | --- | --- |
-| `tap(at)` | Taps and resolves with the point tapped |
+| `tap(at)` | Taps (clicks on Web and Mac computers) and resolves with the point tapped. On a Mac computer a described tap takes several seconds, can click a look-alike, and can click something else when the target isn't on the screen, so check the point or a screenshot afterwards; see [computers.md](computers.md#screen) |
+| `doubleTap(at)` | Double-taps (double-clicks on Web) and resolves with the point |
+| `longPress(at, { duration })` | Presses and holds for `duration` milliseconds, 1 to 5000 (default 1500), and resolves with the point |
 | `type(at, text, { clear })` | Taps a text field and types into it, clearing it first unless `clear` is `false` |
+| `typeFocused(text, { clear })` | Web and Mac computers: types into the field that already has focus, without clicking. Focus the field first, for example with `tap` or `pressKey('tab')`. With no field focused nothing is typed and no error is thrown, so check the result. On Web it replaces the field's text unless `clear` is `false`. On a Mac computer pass `{ clear: false }`: it types where the cursor is, and leaving `clear` out throws `INVALID_ARGUMENT`. To replace text there, select it first with `pressKey('a', { modifiers: ['meta'] })`. On iOS and Android it throws `NOT_AVAILABLE_ON_PLATFORM`; use `type(location, text)` |
 | `swipe(direction, { from })` | Swipes from a location or the screen's center; `'up'` moves the finger up, which scrolls content down |
-| `pressKey('enter' \| 'backspace')` | Presses a key on the focused field |
+| `drag(from, to)` | Presses at one location, moves to the other and releases, for example to reorder a list, move a slider or select text |
+| `pressKey(key, { modifiers })` | Presses a key on the focused element. Every platform takes `'enter'` and `'backspace'`. Web and Mac computers also take `'tab'`, `'escape'`, `'delete'`, `'space'`, `'arrowup'`, `'arrowdown'`, `'arrowleft'`, `'arrowright'`, `'home'`, `'end'`, `'pageup'` and `'pagedown'`, and `modifiers`: up to four of `'ctrl'`, `'shift'`, `'alt'` and `'meta'`, held while the key is pressed. With modifiers a lowercase letter or digit works too. The shortcut key differs: select all with `pressKey('a', { modifiers: ['meta'] })` on a Mac computer, where `'meta'` is Command and `'alt'` is Option, and with `pressKey('a', { modifiers: ['ctrl'] })` on Web, where `'meta'` is not the shortcut key. Other keys and any modifiers throw `NOT_AVAILABLE_ON_PLATFORM` on iOS and Android |
 | `back()` | Browser history on Web, the back button on Android; iOS throws `NOT_AVAILABLE_ON_PLATFORM` |
+| `goHome()` | Goes to the home screen on iOS and Android; Web throws `NOT_AVAILABLE_ON_PLATFORM` |
 | `openUrl(url)` | Opens a URL or deep link on a device, or loads a URL in a browser |
 | `launchApp(bundleId)` | Launches an installed app on a device |
 | `locate(target)` | Resolves with Revyl's best match on the current screen, without acting |
 | `screenSize()` | Resolves with the screenshot's `{ width, height }` in pixels |
 
-A failed action throws `ACTION_FAILED`, including when Revyl cannot find a target or nothing at a point can take the action. One whose outcome Revyl can't confirm throws `ACTION_UNCERTAIN` or `TIMEOUT`, so check the session before acting again. Actions are never repeated. If Verify can't reach Revyl, the action fails with `API_ERROR` and isn't resent either, so check the session before trying it again.
+A failed action throws `ACTION_FAILED`, including when Revyl cannot find a target or nothing at a point can take the action. One whose outcome Revyl can't confirm throws `ACTION_UNCERTAIN` or `TIMEOUT`, so check the session before acting again. Actions are never repeated. If Verify can't reach Revyl, the action fails with `API_ERROR` and isn't resent either, so check the session before trying it again. A Mac computer's screen has a few errors of its own, listed in [computers.md](computers.md#screen).
 
-Exact actions return when Revyl has accepted them, not when the screen has finished changing. After `swipe`, `openUrl` or `launchApp`, let the screen settle or check it before you `locate` or tap coordinates. `locate` returns Revyl's best match on the current screen and can return a point for an element that is off screen, so bring the element on screen first. A point outside `screenSize()` is off screen, and an action at it is refused.
+Exact actions return when Revyl has accepted them, not when the screen has finished changing. After `swipe`, `drag`, `goHome`, `openUrl` or `launchApp`, let the screen settle or check it before you `locate` or tap coordinates. `locate` returns Revyl's best match on the current screen and can return a point for an element that is off screen, so bring the element on screen first. A point outside `screenSize()` is off screen, and an action at it is refused.
 
 ### Coordinates
 
-Prefer a description, such as `tap({ target: 'Sign in button' })`: Revyl finds the element on the current screen, and you never handle coordinates. Otherwise a point is `{ x, y }` in screenshot pixels on every platform: whole numbers from the top-left corner of the image `screenshot()` returns. So a position you read off a screenshot goes to `tap()`, `type()` and `swipe()` unchanged. `screenSize()` returns the screenshot's width and height, and `locate()` and the points `tap()` and `type()` return use the same pixels. A point outside the screenshot is refused with `INVALID_ARGUMENT` before any action is sent, and the message names the valid range. On iOS, Verify converts the pixels to the screen's points for you. On Web they equal CSS pixels at the default device scale factor of 1, and you still pass screenshot pixels at any other. This holds from TypeScript 0.4.1, Python 0.2.1 and Go v0.3.1; earlier versions took iOS points.
+Prefer a description, such as `tap({ target: 'Sign in button' })`: Revyl finds the element on the current screen, and you never handle coordinates. Otherwise a point is `{ x, y }` in screenshot pixels on every platform: whole numbers from the top-left corner of the image `screenshot()` returns. So a position you read off a screenshot goes to `tap()`, `type()`, `swipe()` and every other exact action unchanged. `screenSize()` returns the screenshot's width and height, and `locate()` and the points `tap()`, `doubleTap()`, `longPress()` and `type()` return use the same pixels. A point outside the screenshot is refused with `INVALID_ARGUMENT` before any action is sent, and the message names the valid range. On iOS, Verify converts the pixels to the screen's points for you. On Web they equal CSS pixels at the default device scale factor of 1, and you still pass screenshot pixels at any other. This holds from TypeScript 0.4.1, Python 0.2.1 and Go v0.3.1; earlier versions took iOS points. On a Mac computer a point is in screenshot pixels too, and `tap({ target })` returns the point it clicked in the same pixels. Verify refuses a negative or fractional point before sending it, and Revyl refuses one past the screen's edge without clicking; both throw `INVALID_ARGUMENT` naming the valid range.
 
 ```ts
 await session.screenshot({ path: '.verify/screen.png' });  // open it and find the element
@@ -172,7 +182,7 @@ When inspecting `revyl device report --session-id <id>`, read validation verdict
 
 ## Your own model
 
-**Since 0.4.0. Early access.** In TypeScript (`@revyl/verify` 0.4.0), Python (`revyl-verify` 0.2.0) and Go (v0.3.0); the [Python](https://github.com/RevylAI/verify/blob/v0.4.1/python/README.md#your-own-model) and [Go](https://github.com/RevylAI/verify/blob/v0.4.1/go/README.md#your-own-model) READMEs show them with each language's names. Runs instructions and validations on a model you choose, in your process, instead of Revyl's agent. Revyl still provides the browser or device. Leave `model` out and Revyl's agent runs the step, as before.
+**Early access.** Since TypeScript (`@revyl/verify`) 0.4.0, Python (`revyl-verify`) 0.2.0 and Go v0.3.0; the [Python](https://github.com/RevylAI/verify/blob/v0.6.0/python/README.md#your-own-model) and [Go](https://github.com/RevylAI/verify/blob/v0.6.0/go/README.md#your-own-model) READMEs show them with each language's names. Runs instructions and validations on a model you choose, in your process, instead of Revyl's agent. Revyl still provides the browser or device. Leave `model` out and Revyl's agent runs the step, as before.
 
 ```ts
 const session = await open('ios', { model: 'anthropic/claude-sonnet-5-5' });  // every step on your model
@@ -208,7 +218,7 @@ const session = await open('ios', { agent });
 ```
 
 - `model`: a `ModelAdapter`. `anthropic({ model, apiKey, baseURL, maxTokens, maxRetries })` uses `ANTHROPIC_API_KEY` and `ANTHROPIC_BASE_URL` for what you leave out. `openaiCompatible({ model, baseURL, apiKey, headers, provider, maxTokens, maxRetries })` uses `OPENAI_BASE_URL` and `OPENAI_API_KEY` only when `baseURL` is left out, so a key never reaches another provider; it then sends OpenAI's `max_completion_tokens`. To wrap another SDK or runtime, implement `{ name, complete(request, signal) }`: it gets the system prompt, text, screenshot and tools, and returns the chosen tool call and token usage.
-- `grounding`: `'revyl'` (default) as above. `'model'` has the model give screenshot coordinates instead, so Revyl receives only taps, typing and swipes, never element descriptions; it needs a model that points accurately.
+- `grounding`: `'revyl'` (default) as above. `'model'` has the model give screenshot coordinates instead, so Revyl receives only device actions such as taps, typing and swipes, never element descriptions; it needs a model that points accurately.
 - `maxTurns` (default 20, up to 100), `modelTimeout` per model call (default 60000), `maxImageSide` for screenshots sent to the model (default 1280).
 
 ## session.screenshot(options?)
@@ -223,7 +233,7 @@ For owning computers, cleanup must be confirmed; a terminal state alone does not
 
 ## Computer commands, files, and processes
 
-Mac sessions provide `exec(argv: string[], options?)`, `readFile(path, options?)` returning `Buffer`, `writeFile(path, string | Uint8Array, options?)`, `stat(path, options?)` returning `{ kind, size }`, `processes(options?)` returning `{ id, state, exitCode? }[]`, and `terminate(id, options?)`. They throw `NOT_AVAILABLE_ON_PLATFORM` on web/iOS/Android. See [computers.md](computers.md) for complete signatures, limits, timeouts, and examples. Both support `screenshot()`; instruction/validation do not run on computers.
+Mac sessions provide `exec(argv: string[], options?)`, `readFile(path, options?)` returning `Buffer`, `writeFile(path, string | Uint8Array, options?)`, `stat(path, options?)` returning `{ kind, size }`, `processes(options?)` returning `{ id, state, exitCode? }[]`, and `terminate(id, options?)`. They throw `NOT_AVAILABLE_ON_PLATFORM` on web/iOS/Android. See [computers.md](computers.md) for complete signatures, limits, timeouts, and examples. Both support `screenshot()`, and computers take `tap({ x, y })`, `screenSize()`, `typeFocused(text, { clear: false })` and `pressKey` on the screen (see [computers.md](computers.md#screen)); instruction/validation do not run on computers.
 
 ### `session.recording(): Promise<ComputerRecording | null>`
 
@@ -257,7 +267,7 @@ While a session is open, Verify attempts best-effort cleanup with a 15-second bu
 - on an uncaught error, such as a failed validation that nothing catches: it prints the error, attempts cleanup, then exits with 1;
 - when the script finishes without calling `close()`: it attempts cleanup and prints a reminder.
 
-Verify skips installing a signal or error handler when that event already has listeners, so test runners and apps that manage their own shutdown keep control of it. Always close in `try/finally` or an `after` hook; automatic cleanup cannot guarantee success. A process that is killed outright, or calls `process.exit()` with sessions open, leaves browser/mobile sessions running until their idle timeout (10 minutes by default). Computers have a server-bounded one-hour lifetime instead; no idle timeout is configurable.
+Verify skips installing a signal or error handler when that event already has listeners, so test runners and apps that manage their own shutdown keep control of it. Always close in `try/finally` or an `after` hook; automatic cleanup cannot guarantee success. A process that is killed outright, or calls `process.exit()` with sessions open, leaves browser/mobile sessions running until their idle timeout (10 minutes by default). Computers have a hard lifetime from create instead (two hours today; `expiresAt` is authoritative); no idle timeout is configurable.
 
 ## defineConfig(config)
 

package/docs/ci.md

@@ -71,4 +71,4 @@ A thrown `VerifyError` fails the test with its code, Revyl's reason, and the ses
 
 ## Cost and parallelism
 
-Each `open()` holds one Revyl session until `close()`. Sessions run in parallel when you open several at once, so close each one as soon as you are done with it. A browser/mobile session that loses its process stops itself after `idleTimeoutSeconds` (10 minutes by default). Computers instead have a server-bounded one-hour lifetime and no configurable idle timeout; close in `finally` and do not treat a timed-out CI job as proof of physical release.
+Each `open()` holds one Revyl session until `close()`. Sessions run in parallel when you open several at once, so close each one as soon as you are done with it. A browser/mobile session that loses its process stops itself after `idleTimeoutSeconds` (10 minutes by default). Computers instead have a hard lifetime from create (two hours today; `expiresAt` is authoritative) and no configurable idle timeout; close in `finally` and do not treat a timed-out CI job as proof of physical release.

package/docs/computers.md

@@ -1,6 +1,6 @@
 # Mac computers (early access)
 
-Verify opens a Revyl cloud macOS computer for commands, files, processes, and screenshots. It does not open an assigned physical computer. Mac computers are **early access**. Ask [support@revyl.ai](mailto:support@revyl.ai) about access.
+Verify opens a Revyl cloud macOS computer for commands, files, processes, screenshots, and clicks and keys on its screen. It does not open an assigned physical computer. Mac computers are **early access**. Ask [support@revyl.ai](mailto:support@revyl.ai) about access.
 
 ```bash
 export REVYL_API_KEY=...
@@ -47,6 +47,75 @@ Paths are relative to the computer's workspace: no leading slash, empty componen
 
 Operation `timeout` is milliseconds, defaults to 30,000, and must be an integer from 1 to 300,000. It is also bounded by the run's remaining lifetime. A computer has a hard lifetime **from create** (two hours today; `expiresAt` is authoritative), not from readiness or last activity; `idleTimeoutSeconds` and the agent `stepTimeout` do not configure computer lifetime or commands. `expiresAt` is a `Date` when the API returns `expires_at`, and otherwise undefined (including on devices and older APIs). Verify refuses new operations after that deadline and bounds operation/screenshot observation by it; expiry does not prevent explicit close or recording retrieval. Revyl allows at most 256 operations per run.
 
+## Screen
+
+**Early access**, since TypeScript 0.6.0, Python 0.4.0 and Go v0.5.0. A script can click, type and press keys on the computer's screen, at points it reads off `screenshot()` or at elements it describes:
+
+```ts
+import { open } from '@revyl/verify';
+
+const computer = await open({ platform: 'mac', startTimeout: 600_000 });
+try {
+  await computer.writeFile('notes.txt', 'Notes\n');
+  await computer.exec(['open', '-a', 'TextEdit', 'notes.txt']);   // exec runs in the workspace
+  await new Promise((resolve) => setTimeout(resolve, 3_000));      // let the window open
+  await computer.screenshot({ path: '.verify/mac.png' });           // open it and find the text
+  await computer.tap({ x: 400, y: 300 });                           // the point as you read it
+  await computer.typeFocused('First line\nSecond line', { clear: false });
+  await computer.pressKey('a', { modifiers: ['meta'] });            // Command+A selects all
+  await computer.typeFocused('Replacement', { clear: false });      // typing replaces the selection
+  await computer.screenshot({ path: '.verify/mac-after.png' });
+} finally {
+  await computer.close();
+}
+```
+
+| Call | What it does |
+| --- | --- |
+| `screenshot({ path? })` | A PNG of the whole screen. Its pixels are the coordinates the other calls take |
+| `screenSize()` | The screenshot's `{ width, height }`: x is 0 to width - 1, y is 0 to height - 1 |
+| `tap({ x, y })` | Clicks the point with the left button and resolves with it. Whole numbers from the top-left corner of the screenshot |
+| `tap({ target })` | Revyl finds the described element on a fresh screenshot, clicks it, and resolves with the point it clicked, in screenshot pixels. The description is 1 to 500 characters. Takes several seconds |
+| `typeFocused(text, { clear: false })` | Types the text where the cursor is, in the app that has focus. `{ clear: false }` is required, because Verify doesn't clear anything first: to replace text, select it with `pressKey('a', { modifiers: ['meta'] })` and type over it. Up to 16,384 characters, including line breaks (`\n`) and tabs (`\t`); other control characters are refused before anything is typed. Long text goes in pieces of up to 500 characters, in order |
+| `pressKey(key, { modifiers })` | Presses a key: `'enter'`, `'backspace'`, `'tab'`, `'escape'`, `'delete'`, `'space'`, `'arrowup'`, `'arrowdown'`, `'arrowleft'`, `'arrowright'`, `'home'`, `'end'`, `'pageup'` or `'pagedown'`, with up to four `modifiers`: `'meta'` (Command), `'ctrl'`, `'alt'` (Option) and `'shift'`. With modifiers a lowercase letter or digit works too, so Command+A is `pressKey('a', { modifiers: ['meta'] })`. On Web the shortcut key is `'ctrl'` instead |
+
+`locate()` isn't available on computers, because Revyl can't find a point there without clicking it; `tap({ target })` returns the point it clicked instead. `doubleTap`, `longPress`, `type(location, text)`, `swipe`, `drag`, `back`, `goHome`, `openUrl` and `launchApp` aren't available on computers either, and a description works only in `tap`. They throw `NOT_AVAILABLE_ON_PLATFORM` before anything is sent; open apps and files with `exec()`, such as `exec(['open', '-a', 'TextEdit', 'notes.txt'])`. For TextEdit, open a file rather than the bare app: opened without a file, it may show a file picker instead of a document.
+
+### Described clicks
+
+```ts
+const point = await computer.tap({ target: 'the Help menu in the menu bar' });   // several seconds
+await computer.screenshot({ path: '.verify/help.png' });                       // check what it clicked
+```
+
+A described click takes several seconds, because Revyl takes a screenshot and finds the element first. It can pick a look-alike, and when the target isn't on the screen it can still click something else instead of failing. So check the point it returns or take a screenshot afterwards, and for anything that's hard to undo, read the point off `screenshot()` and use `tap({ x, y })`.
+
+### Behaviour and errors
+
+Each call returns when Revyl has accepted the action, not when the app has finished reacting, so take a screenshot before you rely on what changed. Verify never repeats a screen action, with one exception: when Revyl answers `screen_starting`, nothing happened, so Verify waits as long as Revyl asks and tries again, for up to 30 seconds. Window positions and the desktop size aren't fixed, so for a script that runs unattended, run it once up to the screenshot, read the points off that image, and then write them into the script.
+
+A script that controls the screen keeps control for 120 seconds after its last action. During that time a teammate can't take control of the screen in the Revyl viewer. The other way round, while someone controls the screen in the viewer, the script's actions throw `SESSION_BUSY`.
+
+| What happened | Error | What to do |
+| --- | --- | --- |
+| A point past the screen's edge (`screen_point_outside`), or a negative or fractional one | `INVALID_ARGUMENT`, and nothing was clicked | The message names the valid range |
+| `typeFocused` without `{ clear: false }`, with control characters other than `\n` and `\t`, or with more than 16,384 characters | `INVALID_ARGUMENT` before anything is sent | The message says how to fix the call |
+| The computer can't type some of the characters (`screen_text_unsupported`) | `INVALID_ARGUMENT`, and nothing was typed | Type those characters another way, such as writing a file with `writeFile` |
+| Someone controls the screen in the viewer (`screen_control_held`) | `SESSION_BUSY`, and nothing happened | Try again after they release it; control frees itself after 2 minutes without input |
+| Control of the screen changed hands before the action was sent (`screen_control_lost`) | `SESSION_BUSY`, and nothing happened | Try again; if someone controls the screen in the viewer, wait until they release it |
+| The computer refused a click at a point (`invalid_request` on `tap({ x, y })`), as when the point is no longer on its screen after its display size changed | `INVALID_ARGUMENT`, and nothing was clicked | Take a new screenshot and pick the point from it |
+| The computer refused typing or a key (`invalid_request`) | `INVALID_ARGUMENT`; part of it may have run | Take a `screenshot()` before you try again |
+| The computer can't press that key (`screen_key_unsupported`) | `INVALID_ARGUMENT`, and nothing was pressed | Use a key `pressKey` lists, or `typeFocused` for a printable character |
+| The screen is still starting (`screen_starting`) for more than 30 seconds | `SESSION_BUSY`, and nothing happened | Try again after the number of seconds the message names |
+| Revyl didn't find the described target (`screen_target_not_found`) | `ACTION_FAILED`, and nothing was clicked | Take a screenshot and pass a point to `tap({ x, y })`, or describe the element differently |
+| Revyl couldn't look the target up just then (`screen_target_unavailable`) | `API_ERROR`, and nothing was clicked | It is safe to try again |
+| The backend doesn't have described clicks yet (`invalid_request` on a described tap) | `NOT_AVAILABLE_ON_PLATFORM`, and nothing was clicked | Pass a point to `tap({ x, y })` |
+| Revyl can't confirm the action (`screen_input_uncertain`, `screen_unavailable`, `screen_input_unavailable` or `owner_unavailable`, any other HTTP 5xx, or no answer at all) | `ACTION_UNCERTAIN` | It may have happened. Take a `screenshot()` before you do it again |
+| The API key may not control the screen (HTTP 403) | `AUTH_REJECTED` | The key's owner needs the Member role or above. CI runner keys can't control the screen yet |
+| Revyl couldn't check the key (HTTP 503 `authentication_unavailable`) | `API_ERROR`, and nothing happened | It is safe to try again |
+
+When long text fails part way, the error says how many characters were certainly typed and whether the piece that failed may have been typed, counting characters as Unicode code points, so an emoji is one. Every other refusal names its reason and what to do next, such as starting a new computer after `run_closed`.
+
 ## Recording metadata
 
 After terminal state **and** confirmed release, `await computer.recording()` returns `{ url, durationSeconds?, startedAt? }` or `null` if the recording is unavailable or still processing. `durationSeconds` is a finite nonnegative number; `startedAt` is a `Date`. Missing or nullable optional fields are omitted. Calling before terminal state and confirmed release throws `SESSION_BUSY`; Verify does not wait for the recording or download it. The method works after `close()` and checks the report before requesting the recording.
@@ -59,7 +128,7 @@ Mac computers are early access. A `503 runtime_unavailable` or typed `computer_n
 
 The default `startTimeout` remains 600000 ms; override it explicitly if needed. An unclaimed start can transition to `failed` with `failure_reason: 'no_host'` after 120 seconds from create. Verify stops readiness polling and throws `SESSION_START_FAILED` with that context, then attempts bounded cleanup. This is not a 120-second timeout for every start: claimed guests may take longer. `device_limit` also yields `SESSION_START_FAILED`, with queued-work/concurrency context. A start timeout or unconfirmed close does not prove resources were released. Inspect the session and contact support rather than blindly repeating a start or stop.
 
-`instruction()` and `validation()` throw `NOT_AVAILABLE_ON_PLATFORM` on computers without sending agent requests. Screenshots are supported, but screenshot readiness is separate from command readiness. Agent control of the screen is not part of this release. On web, iOS, and Android, shell/file/process methods throw `NOT_AVAILABLE_ON_PLATFORM`.
+`instruction()` and `validation()` throw `NOT_AVAILABLE_ON_PLATFORM` on computers without sending agent requests. Screenshots are supported, but screenshot readiness is separate from command readiness. For the screen, use the calls in [Screen](#screen). On web, iOS, and Android, shell/file/process methods throw `NOT_AVAILABLE_ON_PLATFORM`.
 
 For the first 30 seconds after a computer becomes active, `screenshot()` retries exact structured `screen_unavailable` responses (HTTP 409 or 503) and `screen_starting` (HTTP 409). Backoff starts at 500 ms and doubles up to 2 seconds; the fixed window does not restart on each call. Cancellation, close, expiry, and the observation deadline still bound the wait. Exhausting that window yields `SESSION_BUSY` with a desktop-not-ready message, rather than requiring a caller sleep. Authentication failures, unrelated errors, malformed responses, and transport failures are not replayed. Commands and other mutations are never retried by this screenshot behavior.
 

package/docs/errors.md

@@ -37,23 +37,23 @@ try {
 
 | Code | What happened | What to do |
 | --- | --- | --- |
-| `INVALID_ARGUMENT` | A call got a bad value: an unknown target name, an empty step, two steps at once on one session, a point outside the screenshot | The message names the argument |
+| `INVALID_ARGUMENT` | A call got a bad value: an unknown target name, an empty step, two steps at once on one session, a point outside the screenshot, `typeFocused` on a computer without `{ clear: false }` | The message names the argument |
 | `CONFIG_NOT_FOUND` | `open('name')` found no `verify.config.ts` at or above the current folder | Run `npx @revyl/verify init`, pass `options.config`, or pass a target object |
 | `CONFIG_INVALID` | The config failed validation or did not load | The message names the exact option and the rule |
 | `AUTH_MISSING` | No `REVYL_API_KEY` and no saved login | Run `npx @revyl/verify login` in a terminal, or set `REVYL_API_KEY` |
-| `AUTH_REJECTED` | Revyl rejected the credentials | Create a new API key, or log in again |
+| `AUTH_REJECTED` | Revyl rejected the credentials, or refused a computer screen action with HTTP 403 | Create a new API key, or log in again. For a screen action, the key's owner needs the Member role or above, and CI runner keys can't control the screen yet |
 | `EDGE_BLOCKED` | An HTTP 403 has a `text/html` content type and nonempty `cf-ray`, without a valid JSON body | Check network access or contact support; Verify does not retry the request |
 | `API_ERROR` | Revyl returned an unexpected response | Try again; after an exact action, check the session first, since the action may have run. If it persists, report it with the session link |
 | `SESSION_START_FAILED` | The browser, device, or computer did not start | Computer `no_host` reports the 120-second unclaimed-start context; `device_limit` reports queued-work/concurrency context. Inspect the session and cleanup before starting again. |
 | `SESSION_ENDED` | The session was closed, stopped, or expired, or Revyl answered 404 for the session itself | Close explicitly; computer expiry does not prove release |
-| `SESSION_BUSY` | A step conflict, computer operation limit, or recording requested before terminal state and confirmed release | Inspect the state before another call; a terminal computer alone does not prove release |
+| `SESSION_BUSY` | A step conflict, computer operation limit, someone controlling a computer's screen in the Revyl viewer, or recording requested before terminal state and confirmed release | Inspect the state before another call; a terminal computer alone does not prove release |
 | `COMPUTER_LIMIT` | Computer admission was refused due to queued work (including iOS) or the concurrency limit | Wait for capacity; Verify does not retry or stop a definitely refused start |
-| `NOT_AVAILABLE_ON_PLATFORM` | The call is unavailable: computer agent steps, device shell/file/recording methods, computer access or runtime unavailable, or a backend without web steps | See [web.md](web.md) and [computers.md](computers.md#availability-and-errors) |
+| `NOT_AVAILABLE_ON_PLATFORM` | The call is unavailable: computer agent steps, device shell/file/recording methods, an exact action the platform lacks (such as `back()` on iOS, `goHome()` or `launchApp()` on Web, `typeFocused()` or keys other than Enter and Backspace on iOS and Android), computer access or runtime unavailable, or a backend without web steps | See [web.md](web.md) and [computers.md](computers.md#availability-and-errors) |
 | `COMMAND_FAILED` | A computer command exited nonzero | Read its reason; use `exec(argv, { check: false })` to inspect the exit code and output instead |
 | `NOT_FOUND` | A computer file or resource was not found | Check the workspace-relative path |
 | `ACCESS_DENIED` | The computer refused access | Check permissions; do not assume host/root access |
 | `ACTION_FAILED` | Revyl refused or failed a request on the session | Read the reason in the message |
-| `ACTION_UNCERTAIN` | The connection failed mid-request, so the step may or may not have run | Verify does not repeat it; check the session link before deciding what to do |
+| `ACTION_UNCERTAIN` | The connection failed mid-request, or Revyl couldn't confirm a computer screen action, so the step may or may not have run | Verify does not repeat it; check the session link before deciding what to do |
 | `INSTRUCTION_FAILED` | Revyl's agent could not do what the instruction asked | Make the instruction more specific, or split it into smaller steps |
 | `VALIDATION_FAILED` | The validation did not hold | The message has Revyl's reason; watch the recording at the session link |
 | `STEP_FAILED` | Revyl could not run the step to an outcome: it errored, it was cancelled on Revyl, a validation came back without a verdict, or Revyl accepted the step but still answered "Live step not found" about 10 seconds later (`httpStatus` 404). This says nothing about your app, and `soft` does not cover it. Verify leaves the session open. | Watch the session at the link. After "Live step not found", the step may still be running, so wait for it to finish (another step meanwhile gets `SESSION_BUSY`), then run the step again |
@@ -65,9 +65,32 @@ try {
 
 Computer operations use `TIMEOUT` for bounded runtime with confirmed termination, and `ACTION_UNCERTAIN` when termination, execution, or cleanup is unconfirmed. An observed successful operation can return its result while logging pending/failed evidence. Timeout never proves rollback, and failed evidence does not prove the command never ran. Never automatically retry a mutation with a fresh request id; inspect the computer first. See [computers.md](computers.md).
 
+### Computer screen actions
+
+Clicks, typing and keys on a Mac computer's screen map Revyl's answers to these codes, read from the answer's `detail`, never from its HTTP status alone. Verify never repeats a screen action, except after `screen_starting`, which it tries again for up to 30 seconds. See [computers.md](computers.md#screen).
+
+| Revyl's answer | `code` | What it means |
+| --- | --- | --- |
+| 409 `screen_control_held` | `SESSION_BUSY` | Nothing happened: someone controls the screen in the viewer. Try again after they release it; control frees itself after 2 minutes without input |
+| 409 `screen_control_lost` | `SESSION_BUSY` | Nothing happened: control of the screen changed hands before the action was sent. Try again; if someone controls the screen in the viewer, wait until they release it |
+| 409 `screen_starting` for more than 30 seconds | `SESSION_BUSY` | Nothing happened: the screen is still starting. Try again after the number of seconds the message names |
+| 422 `screen_target_not_found` | `ACTION_FAILED` | Nothing was clicked: Revyl didn't find the described target. A target that isn't there can also be clicked as a look-alike, so check the point `tap({ target })` returns |
+| 503 `screen_target_unavailable` | `API_ERROR` | Nothing was clicked: Revyl couldn't look the target up just then. It is safe to try again |
+| 422 `invalid_request` on a described tap | `NOT_AVAILABLE_ON_PLATFORM` | Nothing was clicked: this backend doesn't have described clicks yet. Pass a point |
+| 422 `screen_point_outside` | `INVALID_ARGUMENT` | Nothing was clicked: the point is past the screen's edge. The message names the valid range |
+| 422 `invalid_request` on a tap at a point | `INVALID_ARGUMENT` | Nothing was clicked: the computer refused the point, as when it is no longer on the screen after the display size changed. Take a new `screenshot()` and pick the point from it |
+| 422 `invalid_request` on typing or a key | `INVALID_ARGUMENT` | Part of the action may have run. Take a `screenshot()` before you try again |
+| 422 `screen_key_unsupported` | `INVALID_ARGUMENT` | Nothing was pressed: the computer can't press that key. Use a key `pressKey` lists, or `typeFocused` for a printable character |
+| `screen_text_unsupported` | `INVALID_ARGUMENT` | Nothing was typed: the computer can't type some of the characters |
+| `screen_input_uncertain`, `screen_unavailable`, `screen_input_unavailable` or `owner_unavailable` on an action (409 or 503), any other 5xx, or no answer at all | `ACTION_UNCERTAIN` | The action may have happened. Take a `screenshot()` before you do it again |
+| 403 | `AUTH_REJECTED` | The key's owner needs the Member role or above; CI runner keys can't control the screen yet |
+| 503 `authentication_unavailable` | `API_ERROR` | Nothing happened: Revyl couldn't check the key. It is safe to try again |
+
+Any other refusal of a computer request names its reason and what to do next: `run_closed` (`SESSION_ENDED`) says to start a new computer, `inaccessible` (`NOT_FOUND`) says to check the id and the key's organization, a 401 or 403 (`AUTH_REJECTED`) says to check the key and its owner's role, and an answer Verify can't classify (`API_ERROR`) says to try again.
+
 ## Troubleshooting
 
-- **`NOT_AVAILABLE_ON_PLATFORM`:** the requested method is not available for that target. Computer targets do not run instruction or validation steps; Web and mobile targets do not expose computer commands or files.
+- **`NOT_AVAILABLE_ON_PLATFORM`:** the requested method is not available for that target. Computer targets do not run instruction or validation steps or `locate()`; Web and mobile targets do not expose computer commands or files.
 - **`COMPUTER_LIMIT`:** computer admission was refused because of queued work or a concurrency limit. Verify does not retry creation or stop a definitely refused start.
 - **Blank first screen:** mobile startup uses a bounded, best-effort blank-frame observation, not a rendered-readiness guarantee. Begin with an instruction that reaches the screen your workflow needs. See [mobile.md](mobile.md#steps).
 - **A validation report looks successful but the assertion failed:** inspect each step's `effective_status` and `validation_result`, and compare `validations_passed` with `total_validations` rather than transport counters. See [reading the verdict](end-to-end.md#reading-the-verdict).

package/docs/llms.txt

@@ -1,6 +1,6 @@
 # Verify
 
-> Verify (@revyl/verify) is an open-source TypeScript library for Revyl. It opens cloud browsers and iOS/Android devices for agent instruction/validation steps, or Mac computers for commands, files, processes, screenshots, and optional private recording metadata after release. Call it from a plain script, coding agent, CI, or any test runner. It has no test runner of its own.
+> Verify (@revyl/verify) is an open-source TypeScript library for Revyl. It opens cloud browsers and iOS/Android devices for agent instruction/validation steps, or Mac computers for commands, files, processes, screenshots, clicks and keys on the screen, and optional private recording metadata after release. Call it from a plain script, coding agent, CI, or any test runner. It has no test runner of its own.
 
 ## Docs
 
@@ -11,7 +11,7 @@ Mac computers are early access; ask support@revyl.ai about access. See computers
 - [Configuration](configuration.md): verify.config.ts targets, environment variables, the init and whoami commands
 - [Web](web.md): Revyl web sessions, URLs and builds, sign-in, steps, and exact actions
 - [Mobile](mobile.md): iOS and Android devices, installing apps, and exact actions
-- [Computers](computers.md#availability-and-errors): Mac early access, expiry, capacity errors, recording metadata, and confirmed release
+- [Computers](computers.md#availability-and-errors): Mac early access, commands and files, the screen (clicks, typing and keys), expiry, capacity errors, recording metadata, and confirmed release
 - [Verify working end to end](end-to-end.md): live verification scope, observed outcomes, and limits of the evidence
 - [CI and test runners](ci.md): GitHub Actions, node:test, Vitest, Jest, Playwright Test
 - [Use your own model](own-model.md): run instructions and validations on Claude, GPT, Gemini, a local model or any OpenAI-compatible endpoint; providers, limits, privacy and troubleshooting
@@ -20,4 +20,6 @@ Mac computers are early access; ask support@revyl.ai about access. See computers
 
 ## Exact actions and coordinates
 
-Prefer a description, such as `tap({ target: 'Sign in button' })`: Revyl finds the element on the current screen, and you never handle coordinates. Otherwise a point is `{ x, y }` in screenshot pixels on every platform: whole numbers from the top-left corner of the image `screenshot()` returns. So a position you read off a screenshot goes to `tap()`, `type()` and `swipe()` unchanged. `screenSize()` returns the screenshot's width and height, and `locate()` and the points `tap()` and `type()` return use the same pixels. A point outside the screenshot is refused with `INVALID_ARGUMENT` before any action is sent, and the message names the valid range. On iOS, Verify converts the pixels to the screen's points for you. On Web they equal CSS pixels at the default device scale factor of 1, and you still pass screenshot pixels at any other. This holds from TypeScript 0.4.1, Python 0.2.1 and Go v0.3.1; earlier versions took iOS points. See [For coding agents](agents.md#exact-actions-and-coordinates) for a runnable example.
+Prefer a description, such as `tap({ target: 'Sign in button' })`: Revyl finds the element on the current screen, and you never handle coordinates. Otherwise a point is `{ x, y }` in screenshot pixels on every platform: whole numbers from the top-left corner of the image `screenshot()` returns. So a position you read off a screenshot goes to `tap()`, `type()`, `swipe()` and every other exact action unchanged. `screenSize()` returns the screenshot's width and height, and `locate()` and the points `tap()`, `doubleTap()`, `longPress()` and `type()` return use the same pixels. A point outside the screenshot is refused with `INVALID_ARGUMENT` before any action is sent, and the message names the valid range. On iOS, Verify converts the pixels to the screen's points for you. On Web they equal CSS pixels at the default device scale factor of 1, and you still pass screenshot pixels at any other. This holds from TypeScript 0.4.1, Python 0.2.1 and Go v0.3.1; earlier versions took iOS points. On a Mac computer a point is in screenshot pixels too, and `tap({ target })` returns the point it clicked in the same pixels. Verify refuses a negative or fractional point before sending it, and Revyl refuses one past the screen's edge without clicking; both throw `INVALID_ARGUMENT` naming the valid range. See [For coding agents](agents.md#exact-actions-and-coordinates) for a runnable example.
+
+Exact actions are `tap`, `doubleTap`, `longPress`, `type`, `swipe`, `drag`, `pressKey`, `locate` and `screenSize` on Web, iOS and Android. Web adds `typeFocused(text)`, which types into the field that has focus, and `pressKey` keys such as `'escape'`, `'tab'` and arrows with `modifiers`. iOS and Android add `goHome()`; their `pressKey` takes only `'enter'` and `'backspace'`, and you type with `type(location, text)`. A Mac computer's screen takes `tap({ x, y })`, `tap({ target })` (which returns the point it clicked; check it, because grounding can click a look-alike or something else when the target isn't there), `screenSize()`, `typeFocused(text, { clear: false })` and `pressKey` with the Web keys, where `'meta'` is Command: select all with `pressKey('a', { modifiers: ['meta'] })` there and `pressKey('a', { modifiers: ['ctrl'] })` on Web. See [computers.md](computers.md#screen). A call the platform lacks throws `NOT_AVAILABLE_ON_PLATFORM` before anything is sent, naming what to use instead. See [api.md](api.md#exact-actions) for every call.

package/docs/mobile.md

@@ -50,7 +50,7 @@ For Android Settings, instruct the agent to scroll or navigate until the OS vers
 
 ## Exact actions
 
-**Since 0.4.0. Early access.** In TypeScript (`@revyl/verify` 0.4.0), Python (`revyl-verify` 0.2.0) and Go (v0.3.0); the [Python](https://github.com/RevylAI/verify/blob/v0.4.1/python/README.md#exact-actions) and [Go](https://github.com/RevylAI/verify/blob/v0.4.1/go/README.md#exact-actions) READMEs show them with each language's names. When you know exactly what to press, skip the agent. Exact actions run on the device as given, with no model, and mix freely with `instruction()` and `validation()` in one script:
+**Early access.** Since TypeScript (`@revyl/verify`) 0.4.0, Python (`revyl-verify`) 0.2.0 and Go v0.3.0. `doubleTap`, `longPress`, `drag`, `typeFocused`, `goHome`, and Web keys and modifiers in `pressKey`, are new in TypeScript 0.5.0, Python 0.3.0 and Go v0.4.0. The [Python](https://github.com/RevylAI/verify/blob/v0.6.0/python/README.md#exact-actions) and [Go](https://github.com/RevylAI/verify/blob/v0.6.0/go/README.md#exact-actions) READMEs show them with each language's names. When you know exactly what to press, skip the agent. Exact actions run on the device as given, with no model, and mix freely with `instruction()` and `validation()` in one script:
 
 ```ts
 import { open } from '@revyl/verify';
@@ -75,23 +75,50 @@ try {
 }
 ```
 
-A location is `{ target: 'a description' }`, which Revyl's grounding model finds on the current screen, or `{ x, y }` in screenshot pixels from the top-left corner; `screenSize()` gives the bounds. `type()` taps the field and clears it first unless you pass `{ clear: false }`. `pressKey()` takes `'enter'` or `'backspace'`.
+A location is `{ target: 'a description' }`, which Revyl's grounding model finds on the current screen, or `{ x, y }` in screenshot pixels from the top-left corner; `screenSize()` gives the bounds. `type()` taps the field and clears it first unless you pass `{ clear: false }`. `pressKey()` takes `'enter'` or `'backspace'`; for other keys, tap the on-screen control.
 
-On Android, `back()` presses the system back button. On both platforms, `launchApp(bundleId)` brings an installed app to the front and `openUrl('myapp://orders/42')` opens a deep link. A failed action throws `ACTION_FAILED`. One whose outcome Revyl can't confirm throws `ACTION_UNCERTAIN` or `TIMEOUT`, so check the session before acting again. Actions are never repeated. If Verify can't reach Revyl, the action fails with `API_ERROR` and isn't resent either, so check the session before trying it again. [api.md](api.md#exact-actions) lists every call.
+More actions, new in 0.5.0, on the same Settings app:
+
+```ts
+import { open } from '@revyl/verify';
+
+const session = await open({ platform: 'ios', app: { bundleId: 'com.apple.Preferences' } });
+try {
+  await session.goHome();                                        // the home screen
+  await session.longPress({ target: 'Settings app icon' });      // press and hold, 1.5 s by default
+  await session.validation('A menu for the Settings app is shown');
+  await session.goHome();                                        // closes the menu
+  await session.launchApp('com.apple.Preferences');
+  await session.drag({ target: 'General row' }, { target: 'Search field' });  // the list follows the finger
+  await session.doubleTap({ target: 'Settings title' });
+  await session.longPress({ x: 660, y: 1500 }, { duration: 800 });  // a point in screenshot pixels, held for 800 ms
+} finally {
+  await session.close();
+}
+```
+
+`goHome()` goes to the home screen on iOS and Android. `longPress()` holds for 1500 ms unless you pass `duration`, from 1 to 5000. `drag(from, to)` presses at one location, moves to the other and releases. On iOS and Android, type into a field with `type(location, text)`, which taps it first; `typeFocused()` is Web only and throws `NOT_AVAILABLE_ON_PLATFORM` here.
+
+On Android, `back()` presses the system back button. On both platforms, `goHome()` goes to the home screen, `launchApp(bundleId)` brings an installed app to the front and `openUrl('myapp://orders/42')` opens a deep link. A failed action throws `ACTION_FAILED`. One whose outcome Revyl can't confirm throws `ACTION_UNCERTAIN` or `TIMEOUT`, so check the session before acting again. Actions are never repeated. If Verify can't reach Revyl, the action fails with `API_ERROR` and isn't resent either, so check the session before trying it again. [api.md](api.md#exact-actions) lists every call.
 
 | Call | Web | iOS | Android | Mac computer |
 | --- | --- | --- | --- | --- |
-| `tap`, `type`, `swipe`, `pressKey`, `locate`, `screenSize` | Yes | Yes | Yes | No, use `exec()` and files |
+| `tap({ x, y })`, `tap({ target })`, `screenSize` | Yes | Yes | Yes | Yes |
+| `locate` | Yes | Yes | Yes | No, use `tap({ target })`, which returns the point it clicked, or read the point off `screenshot()` |
+| `doubleTap`, `longPress`, `type`, `swipe`, `drag` | Yes | Yes | Yes | No |
+| `pressKey(key, { modifiers })` | Yes: `'enter'`, `'backspace'`, `'tab'`, `'escape'`, arrows and more, with modifiers | `'enter'` and `'backspace'` only | `'enter'` and `'backspace'` only | Yes: the Web keys and modifiers, where `'meta'` is Command |
+| `typeFocused(text, { clear })` | Yes, types into the focused field | No, use `type(location, text)` | No, use `type(location, text)` | Yes, with `{ clear: false }` |
 | `back()` | Yes, browser history | No, tap the on-screen back control | Yes, the system back button | No |
+| `goHome()` | No, use `openUrl` or `navigate` | Yes, the home screen | Yes, the home screen | No |
 | `openUrl(url)` | Yes, loads an http(s) URL in the tab | Yes, a URL or deep link | Yes, a URL or deep link | No |
 | `launchApp(bundleId)` | No, use `openUrl` or `navigate` | Yes | Yes | No |
 | `navigate(url)` | Yes | No, use `openUrl` | No, use `openUrl` | No |
-| `instruction`, `validation`, your own model | Yes | Yes | Yes | No, use `exec()` |
+| `instruction`, `validation`, your own model | Yes | Yes | Yes | No, use `exec()` and the screen calls above |
 | `screenshot()` | Yes | Yes | Yes | Yes |
 | `exec`, files, processes | No, open a Mac computer | No, open a Mac computer | No, open a Mac computer | Yes |
 
-**Coordinates.** Prefer a description, such as `tap({ target: 'Sign in button' })`: Revyl finds the element on the current screen, and you never handle coordinates. Otherwise a point is `{ x, y }` in screenshot pixels on every platform: whole numbers from the top-left corner of the image `screenshot()` returns. So a position you read off a screenshot goes to `tap()`, `type()` and `swipe()` unchanged. `screenSize()` returns the screenshot's width and height, and `locate()` and the points `tap()` and `type()` return use the same pixels. A point outside the screenshot is refused with `INVALID_ARGUMENT` before any action is sent, and the message names the valid range. On iOS, Verify converts the pixels to the screen's points for you. On Web they equal CSS pixels at the default device scale factor of 1, and you still pass screenshot pixels at any other. This holds from TypeScript 0.4.1, Python 0.2.1 and Go v0.3.1; earlier versions took iOS points.
+**Coordinates.** Prefer a description, such as `tap({ target: 'Sign in button' })`: Revyl finds the element on the current screen, and you never handle coordinates. Otherwise a point is `{ x, y }` in screenshot pixels on every platform: whole numbers from the top-left corner of the image `screenshot()` returns. So a position you read off a screenshot goes to `tap()`, `type()`, `swipe()` and every other exact action unchanged. `screenSize()` returns the screenshot's width and height, and `locate()` and the points `tap()`, `doubleTap()`, `longPress()` and `type()` return use the same pixels. A point outside the screenshot is refused with `INVALID_ARGUMENT` before any action is sent, and the message names the valid range. On iOS, Verify converts the pixels to the screen's points for you. On Web they equal CSS pixels at the default device scale factor of 1, and you still pass screenshot pixels at any other. This holds from TypeScript 0.4.1, Python 0.2.1 and Go v0.3.1; earlier versions took iOS points. On a Mac computer a point is in screenshot pixels too, and `tap({ target })` returns the point it clicked in the same pixels. Verify refuses a negative or fractional point before sending it, and Revyl refuses one past the screen's edge without clicking; both throw `INVALID_ARGUMENT` naming the valid range.
 
 A call a platform doesn't support throws `NOT_AVAILABLE_ON_PLATFORM`, and its message names the alternative.
 
-When you pass a [`model`](own-model.md), your model uses `tap`, `type`, `swipe`, `pressKey`, `openUrl` and, where the platform has it, `back` inside `instruction()`.
+When you pass a [`model`](own-model.md), your model uses `tap`, `doubleTap`, `longPress`, `type`, `swipe`, `drag`, `pressKey`, `openUrl` and, where the platform has them, `typeFocused`, `back` and `goHome` inside `instruction()`.

package/docs/own-model.md

@@ -1,10 +1,10 @@
 # Use your own model
 
-**Since 0.4.0. Early access.** In TypeScript (`@revyl/verify` 0.4.0), Python (`revyl-verify` 0.2.0) and Go (v0.3.0); the [Python](https://github.com/RevylAI/verify/blob/v0.4.1/python/README.md#your-own-model) and [Go](https://github.com/RevylAI/verify/blob/v0.4.1/go/README.md#your-own-model) READMEs show them with each language's names. Verify normally runs `instruction()` and `validation()` on Revyl's agent. Pass a `model` and the agent loop runs in your process instead, on a model you choose, with your own key. Revyl still provides the cloud browser or device and, by default, finds the elements your model names on screen.
+**Early access.** Since TypeScript (`@revyl/verify`) 0.4.0, Python (`revyl-verify`) 0.2.0 and Go v0.3.0; the [Python](https://github.com/RevylAI/verify/blob/v0.6.0/python/README.md#your-own-model) and [Go](https://github.com/RevylAI/verify/blob/v0.6.0/go/README.md#your-own-model) READMEs show them with each language's names. Verify normally runs `instruction()` and `validation()` on Revyl's agent. Pass a `model` and the agent loop runs in your process instead, on a model you choose, with your own key. Revyl still provides the cloud browser or device and, by default, finds the elements your model names on screen.
 
 Use it when you want a specific model, your own prompts and key, or faster steps on a cheap model. Leave `model` out and nothing changes.
 
-Your model uses the same [exact actions](api.md#exact-actions) you can call yourself: `tap`, `type`, `swipe`, `pressKey`, `openUrl` and, where the platform has it, `back`. So a script can mix your own actions, steps on your model and steps on Revyl's agent.
+Your model uses the same [exact actions](api.md#exact-actions) you can call yourself: `tap`, `doubleTap`, `longPress`, `type`, `swipe`, `drag`, `pressKey`, `openUrl` and, where the platform has them, `typeFocused` and Web keys with modifiers (Web), `back` (Web and Android) and `goHome` (iOS and Android). So a script can mix your own actions, steps on your model and steps on Revyl's agent.
 
 ## The one-line form
 
@@ -52,7 +52,7 @@ log.Println(step.Model.ID, len(step.Model.Turns), step.Model.InputTokens, step.M
 _, err = s.Validation(ctx, "The screen shows the iOS version")
 ```
 
-The [Python](https://github.com/RevylAI/verify/blob/v0.4.1/python/README.md#your-own-model) and [Go](https://github.com/RevylAI/verify/blob/v0.4.1/go/README.md#your-own-model) READMEs have the full reference for each.
+The [Python](https://github.com/RevylAI/verify/blob/v0.6.0/python/README.md#your-own-model) and [Go](https://github.com/RevylAI/verify/blob/v0.6.0/go/README.md#your-own-model) READMEs have the full reference for each.
 
 Set `model` on `open()` to run every step on it, or on one step to override it:
 
@@ -169,7 +169,7 @@ const session = await open('ios', { agent }); // or session.instruction(text, {
 
 **Your model sees** the system prompt, the step text, a short log of the actions it took so far and whether they worked, and the current screenshot, downscaled so its longest side is at most 1280 pixels (`maxImageSide`).
 
-**Revyl sees** what it always sees for a session it hosts: the device and its screen. On top of that it receives each device action (taps, typed text, swipes, key presses, URLs), the element descriptions your model writes, such as "General row" (not with `grounding: 'model'`), and the model's name, such as `anthropic/claude-sonnet-5-5`, for the session report.
+**Revyl sees** what it always sees for a session it hosts: the device and its screen. On top of that it receives each device action (taps, long presses, drags, typed text, swipes, key presses, URLs), the element descriptions your model writes, such as "General row" (not with `grounding: 'model'`), and the model's name, such as `anthropic/claude-sonnet-5-5`, for the session report.
 
 **Revyl does not receive** the step text, the prompts, your model's output or reasoning, or your key. Keys go only to the provider's endpoint, are never logged, and are replaced with `[redacted]` in error messages, as are custom header values. A custom endpoint's address is never sent to Revyl.
 

package/docs/web.md

@@ -53,7 +53,7 @@ Revyl runs one step at a time on a session. A step sent while another is still r
 
 ## Exact actions
 
-**Since 0.4.0. Early access.** In TypeScript (`@revyl/verify` 0.4.0), Python (`revyl-verify` 0.2.0) and Go (v0.3.0); the [Python](https://github.com/RevylAI/verify/blob/v0.4.1/python/README.md#exact-actions) and [Go](https://github.com/RevylAI/verify/blob/v0.4.1/go/README.md#exact-actions) READMEs show them with each language's names. When you know exactly what to click, skip the agent. Exact actions run in the browser as given, with no model, and mix freely with `instruction()` and `validation()`:
+**Early access.** Since TypeScript (`@revyl/verify`) 0.4.0, Python (`revyl-verify`) 0.2.0 and Go v0.3.0. `doubleTap`, `longPress`, `drag`, `typeFocused`, `goHome`, and Web keys and modifiers in `pressKey`, are new in TypeScript 0.5.0, Python 0.3.0 and Go v0.4.0. The [Python](https://github.com/RevylAI/verify/blob/v0.6.0/python/README.md#exact-actions) and [Go](https://github.com/RevylAI/verify/blob/v0.6.0/go/README.md#exact-actions) READMEs show them with each language's names. When you know exactly what to click, skip the agent. Exact actions run in the browser as given, with no model, and mix freely with `instruction()` and `validation()`:
 
 ```ts
 import { open } from '@revyl/verify';
@@ -76,21 +76,53 @@ try {
 }
 ```
 
-A location is `{ target: 'a description' }`, which Revyl's grounding model finds on the current page, or `{ x, y }` in screenshot pixels from the top-left corner; `screenSize()` gives the bounds and `locate(target)` gives an element's point without acting. `type()` clicks the field and clears it first unless you pass `{ clear: false }`. `pressKey()` takes `'enter'` or `'backspace'`, and `openUrl(url)` loads a page like `navigate(url)`. A failed action throws `ACTION_FAILED`. One whose outcome Revyl can't confirm throws `ACTION_UNCERTAIN` or `TIMEOUT`, so check the session before acting again. Actions are never repeated. If Verify can't reach Revyl, the action fails with `API_ERROR` and isn't resent either, so check the session before trying it again. [api.md](api.md#exact-actions) lists every call.
+A location is `{ target: 'a description' }`, which Revyl's grounding model finds on the current page, or `{ x, y }` in screenshot pixels from the top-left corner; `screenSize()` gives the bounds and `locate(target)` gives an element's point without acting. `type()` clicks the field and clears it first unless you pass `{ clear: false }`. `openUrl(url)` loads a page like `navigate(url)`.
+
+More actions, new in 0.5.0, on the same store:
+
+```ts
+import { open } from '@revyl/verify';
+
+const session = await open('web');                               // https://www.saucedemo.com in examples/web-store
+try {
+  await session.tap({ target: 'Username field' });
+  await session.typeFocused('standard_user');                    // types into the field that has focus, replacing its text
+  await session.pressKey('tab');                                 // focus moves to the password field
+  await session.typeFocused('secret_sauce');
+  await session.pressKey('enter');
+  await session.validation('A list of products is shown');
+  await session.tap({ target: 'Open Menu button' });
+  await session.pressKey('escape');                              // closes the menu
+  await session.validation('The side menu is closed');
+  await session.doubleTap({ target: 'Products heading' });       // a double-click
+  await session.pressKey('a', { modifiers: ['ctrl'] });          // a chord: a lowercase letter or digit with modifiers
+  await session.longPress({ target: 'Products heading' }, { duration: 800 });  // press and hold for 800 ms
+  await session.drag({ target: 'Sauce Labs Backpack description' }, { target: 'Sauce Labs Bike Light description' });  // selects the text between
+} finally {
+  await session.close();
+}
+```
+
+`typeFocused(text)` types where the focus already is, without clicking, and replaces the field's text unless you pass `{ clear: false }`. Focus the field first, with `tap()` or `pressKey('tab')`: with no field focused, nothing is typed and no error is thrown. `pressKey()` takes `'enter'`, `'backspace'`, `'tab'`, `'escape'`, `'delete'`, `'space'`, `'arrowup'`, `'arrowdown'`, `'arrowleft'`, `'arrowright'`, `'home'`, `'end'`, `'pageup'` and `'pagedown'`, with up to four `modifiers` held (`'ctrl'`, `'shift'`, `'alt'`, `'meta'`); with modifiers a lowercase letter or digit works too. The shortcut key on Web is `'ctrl'`, so select all with `pressKey('a', { modifiers: ['ctrl'] })`: `'meta'` is not the shortcut key here, though it is Command on a [Mac computer](computers.md#screen). `longPress()` holds for 1500 ms unless you pass `duration`, from 1 to 5000. `drag(from, to)` presses at one location, moves to the other and releases. A failed action throws `ACTION_FAILED`. One whose outcome Revyl can't confirm throws `ACTION_UNCERTAIN` or `TIMEOUT`, so check the session before acting again. Actions are never repeated. If Verify can't reach Revyl, the action fails with `API_ERROR` and isn't resent either, so check the session before trying it again. [api.md](api.md#exact-actions) lists every call.
 
 | Call | Web | iOS | Android | Mac computer |
 | --- | --- | --- | --- | --- |
-| `tap`, `type`, `swipe`, `pressKey`, `locate`, `screenSize` | Yes | Yes | Yes | No, use `exec()` and files |
+| `tap({ x, y })`, `tap({ target })`, `screenSize` | Yes | Yes | Yes | Yes |
+| `locate` | Yes | Yes | Yes | No, use `tap({ target })`, which returns the point it clicked, or read the point off `screenshot()` |
+| `doubleTap`, `longPress`, `type`, `swipe`, `drag` | Yes | Yes | Yes | No |
+| `pressKey(key, { modifiers })` | Yes: `'enter'`, `'backspace'`, `'tab'`, `'escape'`, arrows and more, with modifiers | `'enter'` and `'backspace'` only | `'enter'` and `'backspace'` only | Yes: the Web keys and modifiers, where `'meta'` is Command |
+| `typeFocused(text, { clear })` | Yes, types into the focused field | No, use `type(location, text)` | No, use `type(location, text)` | Yes, with `{ clear: false }` |
 | `back()` | Yes, browser history | No, tap the on-screen back control | Yes, the system back button | No |
+| `goHome()` | No, use `openUrl` or `navigate` | Yes, the home screen | Yes, the home screen | No |
 | `openUrl(url)` | Yes, loads an http(s) URL in the tab | Yes, a URL or deep link | Yes, a URL or deep link | No |
 | `launchApp(bundleId)` | No, use `openUrl` or `navigate` | Yes | Yes | No |
 | `navigate(url)` | Yes | No, use `openUrl` | No, use `openUrl` | No |
-| `instruction`, `validation`, your own model | Yes | Yes | Yes | No, use `exec()` |
+| `instruction`, `validation`, your own model | Yes | Yes | Yes | No, use `exec()` and the screen calls above |
 | `screenshot()` | Yes | Yes | Yes | Yes |
 | `exec`, files, processes | No, open a Mac computer | No, open a Mac computer | No, open a Mac computer | Yes |
 
-**Coordinates.** Prefer a description, such as `tap({ target: 'Sign in button' })`: Revyl finds the element on the current screen, and you never handle coordinates. Otherwise a point is `{ x, y }` in screenshot pixels on every platform: whole numbers from the top-left corner of the image `screenshot()` returns. So a position you read off a screenshot goes to `tap()`, `type()` and `swipe()` unchanged. `screenSize()` returns the screenshot's width and height, and `locate()` and the points `tap()` and `type()` return use the same pixels. A point outside the screenshot is refused with `INVALID_ARGUMENT` before any action is sent, and the message names the valid range. On iOS, Verify converts the pixels to the screen's points for you. On Web they equal CSS pixels at the default device scale factor of 1, and you still pass screenshot pixels at any other. This holds from TypeScript 0.4.1, Python 0.2.1 and Go v0.3.1; earlier versions took iOS points.
+**Coordinates.** Prefer a description, such as `tap({ target: 'Sign in button' })`: Revyl finds the element on the current screen, and you never handle coordinates. Otherwise a point is `{ x, y }` in screenshot pixels on every platform: whole numbers from the top-left corner of the image `screenshot()` returns. So a position you read off a screenshot goes to `tap()`, `type()`, `swipe()` and every other exact action unchanged. `screenSize()` returns the screenshot's width and height, and `locate()` and the points `tap()`, `doubleTap()`, `longPress()` and `type()` return use the same pixels. A point outside the screenshot is refused with `INVALID_ARGUMENT` before any action is sent, and the message names the valid range. On iOS, Verify converts the pixels to the screen's points for you. On Web they equal CSS pixels at the default device scale factor of 1, and you still pass screenshot pixels at any other. This holds from TypeScript 0.4.1, Python 0.2.1 and Go v0.3.1; earlier versions took iOS points. On a Mac computer a point is in screenshot pixels too, and `tap({ target })` returns the point it clicked in the same pixels. Verify refuses a negative or fractional point before sending it, and Revyl refuses one past the screen's edge without clicking; both throw `INVALID_ARGUMENT` naming the valid range.
 
 A call a platform doesn't support throws `NOT_AVAILABLE_ON_PLATFORM`, and its message names the alternative.
 
-When you pass a [`model`](own-model.md), your model uses `tap`, `type`, `swipe`, `pressKey`, `openUrl` and, where the platform has it, `back` inside `instruction()`.
+When you pass a [`model`](own-model.md), your model uses `tap`, `doubleTap`, `longPress`, `type`, `swipe`, `drag`, `pressKey`, `openUrl` and, where the platform has them, `typeFocused`, `back` and `goHome` inside `instruction()`.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@revyl/verify",
-  "version": "0.4.1",
+  "version": "0.6.0",
   "description": "Open Revyl cloud browsers and mobile devices for agent steps, or Mac computers for commands and files, from TypeScript.",
   "keywords": [
     "e2e",