@revyl/verify 0.4.0 → 0.4.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +22 -0
- package/README.md +15 -13
- package/dist/agent/loop.d.ts +1 -1
- package/dist/session.d.ts +12 -6
- package/dist/session.js +74 -10
- package/docs/agents.md +29 -0
- package/docs/api.md +16 -6
- package/docs/configuration.md +1 -1
- package/docs/errors.md +1 -1
- package/docs/llms.txt +5 -1
- package/docs/mobile.md +6 -4
- package/docs/own-model.md +2 -2
- package/docs/web.md +6 -4
- package/package.json +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,27 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.4.1 (unreleased)
|
|
4
|
+
|
|
5
|
+
- **Breaking for iOS exact actions (early access):** coordinates are now screenshot pixels on every platform. On iOS, `tap`, `type`, `swipe`'s `from`, `locate`, `screenSize` and the points `tap` and `type` return used the screen's points (440 × 956 on an iPhone whose screenshots are 1320 × 2868); they now use the screenshot's pixels, and Verify converts them to points for you. A script that passed iOS points must pass screenshot pixels instead: multiply by the screenshot's width over 0.4.0's `screenSize()` width (3 on that iPhone). Android and Web coordinates are unchanged, because their screenshots already match the screen.
|
|
6
|
+
- A point outside the screenshot is refused with `INVALID_ARGUMENT` before any action is sent, and the message names the valid range. The first call in a session that takes or returns a point reads the screen size once (on iOS, plus one screenshot), and later calls reuse it. Before refusing a point, Verify reads the size again, so a rotated device or a resized Web viewport doesn't get a valid point refused. On Android and Web, if the size can't be read, the action is sent as in 0.4.0; on iOS the call fails, because the scale is needed. Calls by description send the same requests as 0.4.0 on Android and Web.
|
|
7
|
+
- `locate()` keeps an off-screen match outside `screenSize()` instead of moving it to the edge, so you can tell it is off screen, and an action at it is refused.
|
|
8
|
+
- `instruction()` and `validation()` on your own model are unchanged: the model still sees screenshots, and Verify still maps its answers to the device.
|
|
9
|
+
- Docs: one coordinate rule in the same words for TypeScript, Python and Go, in the README, `api.md`, the web and mobile guides, `agents.md` and `llms.txt`. Web coordinates are screenshot pixels, which are CSS pixels at the default device scale factor of 1.
|
|
10
|
+
|
|
11
|
+
## Go 0.3.1 (unreleased)
|
|
12
|
+
|
|
13
|
+
Published as the `go/v0.3.1` tag.
|
|
14
|
+
|
|
15
|
+
- **Breaking for iOS exact actions (early access):** coordinates are now screenshot pixels on every platform, as in TypeScript 0.4.1. On iOS, `Tap`, `Type`, `Swipe`'s `From`, `Locate`, `ScreenSize` and the points `Tap` and `Type` return used the screen's points; they now use the screenshot's pixels, and Verify converts them to points for you. Android and Web coordinates are unchanged.
|
|
16
|
+
- A point outside the screenshot returns `ErrInvalidArgument` before any action is sent, naming the valid range, after Verify reads the size again in case the screen rotated. The screen size is read once per session. On Android and Web a failed size read skips the check and sends the action as before; on iOS it returns the error.
|
|
17
|
+
- README: the same coordinate rule as TypeScript and Python.
|
|
18
|
+
|
|
19
|
+
## Python 0.2.1 (unreleased)
|
|
20
|
+
|
|
21
|
+
- **Breaking for iOS exact actions (early access):** coordinates are now screenshot pixels on every platform, as in TypeScript 0.4.1. On iOS, `tap`, `type`, `swipe`'s `from_`, `locate`, `screen_size` and the points `tap` and `type` return used the screen's points; they now use the screenshot's pixels, and Verify converts them to points for you. Android and Web coordinates are unchanged. `AsyncSession` behaves the same.
|
|
22
|
+
- A point outside the screenshot raises `INVALID_ARGUMENT` before any action is sent, naming the valid range, after Verify reads the size again in case the screen rotated. The screen size is read once per session. On Android and Web a failed size read skips the check and sends the action as before; on iOS it raises.
|
|
23
|
+
- README: the same coordinate rule as TypeScript and Go.
|
|
24
|
+
|
|
3
25
|
## 0.4.0 (2026-11-11)
|
|
4
26
|
|
|
5
27
|
- Early access: run `instruction()` and `validation()` on your own model. `open('ios', { model: 'anthropic/claude-sonnet-5-5' })` runs every step on Claude with the key from `ANTHROPIC_API_KEY`; a step's `{ model }` overrides it. Built-in providers: Anthropic, OpenAI, Google, OpenRouter, Groq, Cerebras, xAI and Ollama, each reading its usual key variable; `anthropic/` and `openai/` also honour `ANTHROPIC_BASE_URL` and `OPENAI_BASE_URL` for proxies and gateways. `{ id, baseURL, apiKey, headers }` reaches any OpenAI-compatible endpoint, with the key taken only from `apiKey`. Your model names elements and Revyl's grounding model finds them. `StepResult.model` reports turns and tokens. Revyl's agent remains the default.
|
package/README.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Verify
|
|
2
2
|
|
|
3
|
-
|
|
4
4
|
|
|
5
5
|
Verify runs end-to-end checks in a cloud browser, iOS simulator, or Android emulator on Revyl. You write what the agent should do and what must be true afterward, in plain sentences, and call it from a script, a coding agent, CI, or the test runner you already use. When you know exactly what to press, you can tap, type and swipe yourself instead. Verify also opens Mac computers for commands, files, and screenshots.
|
|
6
6
|
|
|
@@ -41,9 +41,9 @@ Or set it up yourself:
|
|
|
41
41
|
npx @revyl/verify init
|
|
42
42
|
```
|
|
43
43
|
|
|
44
|
-
`init` adds Verify to your project, asks what to open, writes `verify.config.ts` and an example script, signs you in through the browser on the first interactive run, and prints the command that runs the example. With no app id, the iOS and Android examples open the built-in Settings app, so they run on a new account before you upload anything. To skip the config, run `npm install @revyl/verify` and save the example above as `verify/check.mts`, then run `node verify/check.mts`. [Getting started](https://github.com/RevylAI/verify/blob/v0.4.
|
|
44
|
+
`init` adds Verify to your project, asks what to open, writes `verify.config.ts` and an example script, signs you in through the browser on the first interactive run, and prints the command that runs the example. With no app id, the iOS and Android examples open the built-in Settings app, so they run on a new account before you upload anything. To skip the config, run `npm install @revyl/verify` and save the example above as `verify/check.mts`, then run `node verify/check.mts`. [Getting started](https://github.com/RevylAI/verify/blob/v0.4.1/packages/verify/docs/getting-started.md) covers sign-in, your own app, and [CI](https://github.com/RevylAI/verify/blob/v0.4.1/packages/verify/docs/ci.md).
|
|
45
45
|
|
|
46
|
-
For finished projects, see [`examples/`](https://github.com/RevylAI/verify/tree/v0.4.
|
|
46
|
+
For finished projects, see [`examples/`](https://github.com/RevylAI/verify/tree/v0.4.1/examples/): a [web store](https://github.com/RevylAI/verify/tree/v0.4.1/examples/web-store/), [mobile Settings](https://github.com/RevylAI/verify/tree/v0.4.1/examples/mobile-settings/), a [Mac workspace](https://github.com/RevylAI/verify/tree/v0.4.1/examples/mac-workspace/), a small [agent platform](https://github.com/RevylAI/verify/tree/v0.4.1/examples/agent-platform/) that runs web and iOS side by side, and [your own model](https://github.com/RevylAI/verify/tree/v0.4.1/examples/byo-model/) (with [Python](https://github.com/RevylAI/verify/blob/v0.4.1/python/examples/byo_model.py) and [Go](https://github.com/RevylAI/verify/blob/v0.4.1/go/examples/byomodel/main.go) versions).
|
|
47
47
|
|
|
48
48
|
## Exact actions
|
|
49
49
|
|
|
@@ -53,12 +53,12 @@ Tap, type, swipe and press keys yourself, by a description Revyl locates on scre
|
|
|
53
53
|
await session.type({ target: 'Email field' }, 'demo@example.com'); // Revyl finds the element you describe
|
|
54
54
|
await session.pressKey('enter');
|
|
55
55
|
await session.swipe('up'); // scrolls the content down
|
|
56
|
-
await session.tap({ x: 196, y: 640 }); // or
|
|
56
|
+
await session.tap({ x: 196, y: 640 }); // or a point in screenshot pixels
|
|
57
57
|
await session.instruction('Accept the terms if they are shown'); // the agent where it's fuzzy
|
|
58
58
|
await session.validation('The welcome screen is shown');
|
|
59
59
|
```
|
|
60
60
|
|
|
61
|
-
`back`, `openUrl`, `launchApp`, `locate` and `screenSize` round them out; [which call works where](#which-call-works-where) shows the platform limits. Early access. In TypeScript (`@revyl/verify` 0.4.0), Python (`revyl-verify` 0.2.0) and Go (v0.3.0); the [Python](https://github.com/RevylAI/verify/blob/v0.4.
|
|
61
|
+
`back`, `openUrl`, `launchApp`, `locate` and `screenSize` round them out; [which call works where](#which-call-works-where) shows the platform limits. Early access. In TypeScript (`@revyl/verify` 0.4.0), Python (`revyl-verify` 0.2.0) and Go (v0.3.0); the [Python](https://github.com/RevylAI/verify/blob/v0.4.1/python/README.md#exact-actions) and [Go](https://github.com/RevylAI/verify/blob/v0.4.1/go/README.md#exact-actions) READMEs show them with each language's names. The [web](https://github.com/RevylAI/verify/blob/v0.4.1/packages/verify/docs/web.md#exact-actions) and [mobile](https://github.com/RevylAI/verify/blob/v0.4.1/packages/verify/docs/mobile.md#exact-actions) guides have runnable scripts, and [api.md](https://github.com/RevylAI/verify/blob/v0.4.1/packages/verify/docs/api.md#exact-actions) lists every call.
|
|
62
62
|
|
|
63
63
|
## Your own model
|
|
64
64
|
|
|
@@ -72,7 +72,7 @@ await session.instruction('Open Privacy', { model: 'openai/gpt-5.5' });
|
|
|
72
72
|
await open('ios', { model: { id: 'qwen-3.8-27b', baseURL: 'https://api.cerebras.ai/v1', apiKey: process.env.CEREBRAS_API_KEY } });
|
|
73
73
|
```
|
|
74
74
|
|
|
75
|
-
In TypeScript (`@revyl/verify` 0.4.0), Python (`revyl-verify` 0.2.0) and Go (v0.3.0); the [Python](https://github.com/RevylAI/verify/blob/v0.4.
|
|
75
|
+
In TypeScript (`@revyl/verify` 0.4.0), Python (`revyl-verify` 0.2.0) and Go (v0.3.0); the [Python](https://github.com/RevylAI/verify/blob/v0.4.1/python/README.md#your-own-model) and [Go](https://github.com/RevylAI/verify/blob/v0.4.1/go/README.md#your-own-model) READMEs show them with each language's names. Built in: `anthropic`, `openai`, `google`, `openrouter`, `groq`, `cerebras`, `xai` and `ollama`; anything OpenAI-compatible works through `baseURL`. Early access; [Use your own model](https://github.com/RevylAI/verify/blob/v0.4.1/packages/verify/docs/own-model.md) covers the providers, picking a model, what Revyl still sees, limits and troubleshooting.
|
|
76
76
|
|
|
77
77
|
## Packages
|
|
78
78
|
|
|
@@ -80,8 +80,8 @@ In TypeScript (`@revyl/verify` 0.4.0), Python (`revyl-verify` 0.2.0) and Go (v0.
|
|
|
80
80
|
| --- | --- | --- |
|
|
81
81
|
| [`@revyl/verify`](https://www.npmjs.com/package/@revyl/verify) | `npm install @revyl/verify` | TypeScript SDK and the `verify` CLI (`init`, `login`, `logout`, `whoami`). Published on npm, Node.js 22.18 or later. |
|
|
82
82
|
| [`revyl-verify`](https://www.npmjs.com/package/revyl-verify) (npm) | `npm install revyl-verify` | Alias of `@revyl/verify` at the same version. |
|
|
83
|
-
| [Go module](https://github.com/RevylAI/verify/tree/go/v0.3.
|
|
84
|
-
| [`revyl-verify`](https://pypi.org/project/revyl-verify/) (PyPI) | `pip install revyl-verify` or `uv add revyl-verify` | Python SDK, published on PyPI. Python 3.10 or later, sync and asyncio APIs, no CLI; see the [Python guide](https://github.com/RevylAI/verify/blob/v0.4.
|
|
83
|
+
| [Go module](https://github.com/RevylAI/verify/tree/go/v0.3.1/go) | `go get github.com/RevylAI/verify/go@v0.3.1` | Published as the `go/v0.3.1` tag, Go 1.26.9 or later. Calls return errors instead of throwing. |
|
|
84
|
+
| [`revyl-verify`](https://pypi.org/project/revyl-verify/) (PyPI) | `pip install revyl-verify` or `uv add revyl-verify` | Python SDK, published on PyPI. Python 3.10 or later, sync and asyncio APIs, no CLI; see the [Python guide](https://github.com/RevylAI/verify/blob/v0.4.1/python/README.md). |
|
|
85
85
|
|
|
86
86
|
## Platforms
|
|
87
87
|
|
|
@@ -105,18 +105,20 @@ In TypeScript (`@revyl/verify` 0.4.0), Python (`revyl-verify` 0.2.0) and Go (v0.
|
|
|
105
105
|
| `screenshot()` | Yes | Yes | Yes | Yes |
|
|
106
106
|
| `exec`, files, processes | No, open a Mac computer | No, open a Mac computer | No, open a Mac computer | Yes |
|
|
107
107
|
|
|
108
|
-
Coordinates
|
|
108
|
+
**Coordinates.** Prefer a description, such as `tap({ target: 'Sign in button' })`: Revyl finds the element on the current screen, and you never handle coordinates. Otherwise a point is `{ x, y }` in screenshot pixels on every platform: whole numbers from the top-left corner of the image `screenshot()` returns. So a position you read off a screenshot goes to `tap()`, `type()` and `swipe()` unchanged. `screenSize()` returns the screenshot's width and height, and `locate()` and the points `tap()` and `type()` return use the same pixels. A point outside the screenshot is refused with `INVALID_ARGUMENT` before any action is sent, and the message names the valid range. On iOS, Verify converts the pixels to the screen's points for you. On Web they equal CSS pixels at the default device scale factor of 1, and you still pass screenshot pixels at any other. This holds from TypeScript 0.4.1, Python 0.2.1 and Go v0.3.1; earlier versions took iOS points.
|
|
109
109
|
|
|
110
|
-
|
|
110
|
+
A call a platform doesn't support throws `NOT_AVAILABLE_ON_PLATFORM`, and its message names the alternative.
|
|
111
|
+
|
|
112
|
+
Web is available to every organization. Mac computers are early access and have a server-bounded lifetime; see [computers](https://github.com/RevylAI/verify/blob/v0.4.1/packages/verify/docs/computers.md).
|
|
111
113
|
|
|
112
114
|
## Documentation
|
|
113
115
|
|
|
114
|
-
The [documentation](https://github.com/RevylAI/verify/tree/v0.4.
|
|
116
|
+
The [documentation](https://github.com/RevylAI/verify/tree/v0.4.1/packages/verify/docs/) covers the [API](https://github.com/RevylAI/verify/blob/v0.4.1/packages/verify/docs/api.md), [configuration](https://github.com/RevylAI/verify/blob/v0.4.1/packages/verify/docs/configuration.md), [web](https://github.com/RevylAI/verify/blob/v0.4.1/packages/verify/docs/web.md), [mobile](https://github.com/RevylAI/verify/blob/v0.4.1/packages/verify/docs/mobile.md), [computers](https://github.com/RevylAI/verify/blob/v0.4.1/packages/verify/docs/computers.md), [CI](https://github.com/RevylAI/verify/blob/v0.4.1/packages/verify/docs/ci.md), [your own model](https://github.com/RevylAI/verify/blob/v0.4.1/packages/verify/docs/own-model.md), [errors and troubleshooting](https://github.com/RevylAI/verify/blob/v0.4.1/packages/verify/docs/errors.md), [coding agents](https://github.com/RevylAI/verify/blob/v0.4.1/packages/verify/docs/agents.md), and [live end-to-end results](https://github.com/RevylAI/verify/blob/v0.4.1/packages/verify/docs/end-to-end.md). The npm package ships these pages in `node_modules/@revyl/verify/docs/`, with an [llms.txt](https://github.com/RevylAI/verify/blob/v0.4.1/packages/verify/docs/llms.txt) index, so a coding agent can read the docs for the version you installed. The Revyl docs site has an overview at [docs.revyl.com/verify](https://docs.revyl.com/verify).
|
|
115
117
|
|
|
116
118
|
## Status
|
|
117
119
|
|
|
118
|
-
Verify is at 0.x, so APIs can still change between minor releases; the [changelog](https://github.com/RevylAI/verify/blob/v0.4.
|
|
120
|
+
Verify is at 0.x, so APIs can still change between minor releases; the [changelog](https://github.com/RevylAI/verify/blob/v0.4.1/CHANGELOG.md) records each release. Mac computers are early access. See [Contributing](https://github.com/RevylAI/verify/blob/v0.4.1/CONTRIBUTING.md) and [Releasing](https://github.com/RevylAI/verify/blob/v0.4.1/RELEASING.md) to work on Verify.
|
|
119
121
|
|
|
120
122
|
## License
|
|
121
123
|
|
|
122
|
-
[Apache 2.0](https://github.com/RevylAI/verify/blob/v0.4.
|
|
124
|
+
[Apache 2.0](https://github.com/RevylAI/verify/blob/v0.4.1/LICENSE).
|
package/dist/agent/loop.d.ts
CHANGED
package/dist/session.d.ts
CHANGED
|
@@ -103,7 +103,7 @@ export interface OpenedSession extends Session {
|
|
|
103
103
|
report(): Promise<Record<string, unknown>>;
|
|
104
104
|
navigate(url: string): Promise<void>;
|
|
105
105
|
open(target: HostedTargetConfig): Promise<OpenedSession>;
|
|
106
|
-
/** Taps a point in
|
|
106
|
+
/** Taps a point in screenshot pixels, or an element Revyl locates from a description. Returns the point tapped, in screenshot pixels. Browsers and devices only. */
|
|
107
107
|
tap(at: Location): Promise<Point>;
|
|
108
108
|
/** Taps a text field and types into it, clearing it first unless clear is false. Returns the point tapped. Browsers and devices only. */
|
|
109
109
|
type(at: Location, text: string, options?: {
|
|
@@ -121,9 +121,9 @@ export interface OpenedSession extends Session {
|
|
|
121
121
|
openUrl(url: string): Promise<void>;
|
|
122
122
|
/** Launches an installed app by bundle ID or package name. Devices only. */
|
|
123
123
|
launchApp(bundleId: string): Promise<void>;
|
|
124
|
-
/** Asks Revyl's grounding model where an element is on the current screen, in
|
|
124
|
+
/** Asks Revyl's grounding model where an element is on the current screen, in screenshot pixels. Browsers and devices only. */
|
|
125
125
|
locate(target: string): Promise<Point>;
|
|
126
|
-
/** The
|
|
126
|
+
/** The screenshot's size in pixels, the space every coordinate uses. Browsers and devices only. */
|
|
127
127
|
screenSize(): Promise<{
|
|
128
128
|
width: number;
|
|
129
129
|
height: number;
|
|
@@ -160,6 +160,8 @@ export declare class DeviceSession implements Session {
|
|
|
160
160
|
private parent?;
|
|
161
161
|
private readySignal?;
|
|
162
162
|
private size?;
|
|
163
|
+
private sizeSource?;
|
|
164
|
+
private screenshotSize?;
|
|
163
165
|
private attribution?;
|
|
164
166
|
get expiresAt(): Date | undefined;
|
|
165
167
|
private constructor();
|
|
@@ -195,7 +197,7 @@ export declare class DeviceSession implements Session {
|
|
|
195
197
|
private runStep;
|
|
196
198
|
private runLocal;
|
|
197
199
|
private agentControls;
|
|
198
|
-
/** Taps a point in
|
|
200
|
+
/** Taps a point in screenshot pixels, or an element Revyl locates from a description. Returns the point tapped, in screenshot pixels. */
|
|
199
201
|
tap(at: Location): Promise<Point>;
|
|
200
202
|
/** Taps a text field and types into it, clearing it first unless clear is false. Returns the point tapped. */
|
|
201
203
|
type(at: Location, text: string, options?: {
|
|
@@ -213,13 +215,17 @@ export declare class DeviceSession implements Session {
|
|
|
213
215
|
openUrl(url: string): Promise<void>;
|
|
214
216
|
/** Launches an installed app by bundle ID or package name. Devices only. */
|
|
215
217
|
launchApp(bundleId: string): Promise<void>;
|
|
216
|
-
/** Asks Revyl's grounding model where an element is on the current screen, in
|
|
218
|
+
/** Asks Revyl's grounding model where an element is on the current screen, in screenshot pixels. */
|
|
217
219
|
locate(target: string): Promise<Point>;
|
|
218
|
-
/** The
|
|
220
|
+
/** The screenshot's size in pixels, the space every coordinate uses. */
|
|
219
221
|
screenSize(): Promise<{
|
|
220
222
|
width: number;
|
|
221
223
|
height: number;
|
|
222
224
|
}>;
|
|
225
|
+
/** The screenshot's size and the device size Revyl acts in. They differ on iOS, where Revyl acts in points and a screenshot has more pixels. */
|
|
226
|
+
private readScreenshotSize;
|
|
227
|
+
private toDevice;
|
|
228
|
+
private toScreenshot;
|
|
223
229
|
private readScreenSize;
|
|
224
230
|
private capture;
|
|
225
231
|
private perform;
|
package/dist/session.js
CHANGED
|
@@ -41,6 +41,8 @@ export class DeviceSession {
|
|
|
41
41
|
parent;
|
|
42
42
|
readySignal;
|
|
43
43
|
size;
|
|
44
|
+
sizeSource;
|
|
45
|
+
screenshotSize;
|
|
44
46
|
attribution;
|
|
45
47
|
get expiresAt() {
|
|
46
48
|
return this.parent?.expiresAt;
|
|
@@ -423,21 +425,27 @@ export class DeviceSession {
|
|
|
423
425
|
perform: (action, screenToken, signal) => this.perform(action, screenToken, signal),
|
|
424
426
|
};
|
|
425
427
|
}
|
|
426
|
-
/** Taps a point in
|
|
428
|
+
/** Taps a point in screenshot pixels, or an element Revyl locates from a description. Returns the point tapped, in screenshot pixels. */
|
|
427
429
|
async tap(at) {
|
|
428
|
-
|
|
430
|
+
const location = await this.toDevice(checkLocation(at));
|
|
431
|
+
if (this.platform === 'ios')
|
|
432
|
+
await this.readScreenshotSize();
|
|
433
|
+
return this.toScreenshot((await this.perform({ kind: 'tap', at: location })));
|
|
429
434
|
}
|
|
430
435
|
/** Taps a text field and types into it, clearing it first unless clear is false. Returns the point tapped. */
|
|
431
436
|
async type(at, text, options = {}) {
|
|
432
437
|
if (typeof text !== 'string' || text.length > 16_384)
|
|
433
438
|
throw new VerifyError('INVALID_ARGUMENT', 'type() takes text of at most 16384 characters.');
|
|
434
|
-
|
|
439
|
+
const location = await this.toDevice(checkLocation(at));
|
|
440
|
+
if (this.platform === 'ios')
|
|
441
|
+
await this.readScreenshotSize();
|
|
442
|
+
return this.toScreenshot((await this.perform({ kind: 'type', at: location, text, clear: options.clear !== false })));
|
|
435
443
|
}
|
|
436
444
|
/** Swipes from a point or the screen's center. 'up' moves the finger up, which scrolls content down. */
|
|
437
445
|
async swipe(direction, options = {}) {
|
|
438
446
|
if (!['up', 'down', 'left', 'right'].includes(direction))
|
|
439
447
|
throw new VerifyError('INVALID_ARGUMENT', "swipe() takes 'up', 'down', 'left' or 'right'.");
|
|
440
|
-
await this.perform({ kind: 'swipe', direction, ...(options.from ? { from: checkLocation(options.from) } : {}) });
|
|
448
|
+
await this.perform({ kind: 'swipe', direction, ...(options.from ? { from: await this.toDevice(checkLocation(options.from)) } : {}) });
|
|
441
449
|
}
|
|
442
450
|
/** Presses Enter or Backspace on the focused field. */
|
|
443
451
|
async pressKey(key) {
|
|
@@ -465,14 +473,69 @@ export class DeviceSession {
|
|
|
465
473
|
throw new VerifyError('INVALID_ARGUMENT', 'launchApp() takes a bundle ID or package name.');
|
|
466
474
|
await this.act('launch', { bundle_id: bundleId });
|
|
467
475
|
}
|
|
468
|
-
/** Asks Revyl's grounding model where an element is on the current screen, in
|
|
476
|
+
/** Asks Revyl's grounding model where an element is on the current screen, in screenshot pixels. */
|
|
469
477
|
async locate(target) {
|
|
470
478
|
const { x, y } = await this.resolve(checkLocation({ target }));
|
|
471
|
-
return { x, y };
|
|
479
|
+
return this.toScreenshot({ x, y });
|
|
480
|
+
}
|
|
481
|
+
/** The screenshot's size in pixels, the space every coordinate uses. */
|
|
482
|
+
async screenSize() {
|
|
483
|
+
const { width, height } = await this.readScreenshotSize();
|
|
484
|
+
return { width, height };
|
|
485
|
+
}
|
|
486
|
+
/** The screenshot's size and the device size Revyl acts in. They differ on iOS, where Revyl acts in points and a screenshot has more pixels. */
|
|
487
|
+
async readScreenshotSize(fresh = false) {
|
|
488
|
+
if (fresh) {
|
|
489
|
+
const cached = [this.size, this.sizeSource, this.screenshotSize];
|
|
490
|
+
[this.size, this.sizeSource, this.screenshotSize] = [undefined, undefined, undefined];
|
|
491
|
+
try {
|
|
492
|
+
return await this.readScreenshotSize();
|
|
493
|
+
}
|
|
494
|
+
catch (error) {
|
|
495
|
+
[this.size, this.sizeSource, this.screenshotSize] = cached;
|
|
496
|
+
throw error;
|
|
497
|
+
}
|
|
498
|
+
}
|
|
499
|
+
if (this.screenshotSize)
|
|
500
|
+
return this.screenshotSize;
|
|
501
|
+
const device = await this.readScreenSize();
|
|
502
|
+
let size = device;
|
|
503
|
+
if (this.platform === 'ios') {
|
|
504
|
+
if (this.sizeSource !== 'health')
|
|
505
|
+
throw new VerifyError('ACTION_FAILED', `Revyl did not report this iOS screen's size in points, so Verify cannot map screenshot pixels to it and sent no action. See ${this.viewerUrl}`);
|
|
506
|
+
size = pngSize((await this.capture()).png);
|
|
507
|
+
}
|
|
508
|
+
this.screenshotSize = { width: size.width, height: size.height, device };
|
|
509
|
+
return this.screenshotSize;
|
|
510
|
+
}
|
|
511
|
+
async toDevice(at) {
|
|
512
|
+
if ('target' in at)
|
|
513
|
+
return at;
|
|
514
|
+
const read = async (fresh) => {
|
|
515
|
+
try {
|
|
516
|
+
return await this.readScreenshotSize(fresh);
|
|
517
|
+
}
|
|
518
|
+
catch (error) {
|
|
519
|
+
if (this.platform === 'ios' || !(error instanceof VerifyError))
|
|
520
|
+
throw error;
|
|
521
|
+
return undefined;
|
|
522
|
+
}
|
|
523
|
+
};
|
|
524
|
+
let size = await read(false);
|
|
525
|
+
if (size && (at.x >= size.width || at.y >= size.height))
|
|
526
|
+
size = await read(true);
|
|
527
|
+
if (!size)
|
|
528
|
+
return at;
|
|
529
|
+
if (at.x >= size.width || at.y >= size.height) {
|
|
530
|
+
throw new VerifyError('INVALID_ARGUMENT', `(${at.x}, ${at.y}) is outside the screenshot: x must be 0 to ${size.width - 1} and y 0 to ${size.height - 1}, in screenshot pixels.`);
|
|
531
|
+
}
|
|
532
|
+
return { x: Math.min(size.device.width - 1, Math.round((at.x * size.device.width) / size.width)), y: Math.min(size.device.height - 1, Math.round((at.y * size.device.height) / size.height)) };
|
|
472
533
|
}
|
|
473
|
-
|
|
474
|
-
|
|
475
|
-
|
|
534
|
+
async toScreenshot(point) {
|
|
535
|
+
if (this.platform !== 'ios')
|
|
536
|
+
return point;
|
|
537
|
+
const size = await this.readScreenshotSize();
|
|
538
|
+
return { x: Math.round((point.x * size.width) / size.device.width), y: Math.round((point.y * size.height) / size.device.height) };
|
|
476
539
|
}
|
|
477
540
|
async readScreenSize(signal) {
|
|
478
541
|
if (this.size)
|
|
@@ -481,6 +544,7 @@ export class DeviceSession {
|
|
|
481
544
|
const health = parseJson(await this.worker('GET', 'health', { retry: true, ...(signal ? { signal } : {}) }), 'health');
|
|
482
545
|
if (Number.isInteger(health.screen_width) && Number.isInteger(health.screen_height) && health.screen_width > 0 && health.screen_height > 0) {
|
|
483
546
|
this.size = { width: health.screen_width, height: health.screen_height };
|
|
547
|
+
this.sizeSource = 'health';
|
|
484
548
|
return this.size;
|
|
485
549
|
}
|
|
486
550
|
}
|
|
@@ -765,7 +829,7 @@ function checkLocation(at) {
|
|
|
765
829
|
return { target: at.target.trim() };
|
|
766
830
|
}
|
|
767
831
|
if (!at || typeof at !== 'object' || !Number.isInteger(at.x) || !Number.isInteger(at.y) || at.x < 0 || at.y < 0) {
|
|
768
|
-
throw new VerifyError('INVALID_ARGUMENT', 'A location is { x, y } in non-negative whole
|
|
832
|
+
throw new VerifyError('INVALID_ARGUMENT', 'A location is { x, y } in non-negative whole screenshot pixels, or { target: "description" }.');
|
|
769
833
|
}
|
|
770
834
|
return { x: at.x, y: at.y };
|
|
771
835
|
}
|
package/docs/agents.md
CHANGED
|
@@ -22,6 +22,35 @@ This page is for AI coding agents (Claude Code, Cursor, Codex, and others) that
|
|
|
22
22
|
4. Run it with `node path/to/script.ts`. Share the session link from the first `[verify]` line with the user, so they can watch.
|
|
23
23
|
5. Read the outcome from the exit code and stderr. Each step prints `✓` or `✗` with its text; a failure prints `CODE: message`, ending with the session link. Look at any screenshot you saved before changing the script or the app.
|
|
24
24
|
|
|
25
|
+
## Exact actions and coordinates
|
|
26
|
+
|
|
27
|
+
Use `instruction()` when the step needs judgement, and exact actions (`tap`, `type`, `swipe`, `pressKey`, `locate`, `screenSize`) when you know exactly what to press. They work on Web, iOS and Android, not on computers.
|
|
28
|
+
|
|
29
|
+
Prefer a description, such as `tap({ target: 'Sign in button' })`: Revyl finds the element on the current screen, and you never handle coordinates. Otherwise a point is `{ x, y }` in screenshot pixels on every platform: whole numbers from the top-left corner of the image `screenshot()` returns. So a position you read off a screenshot goes to `tap()`, `type()` and `swipe()` unchanged. `screenSize()` returns the screenshot's width and height, and `locate()` and the points `tap()` and `type()` return use the same pixels. A point outside the screenshot is refused with `INVALID_ARGUMENT` before any action is sent, and the message names the valid range. On iOS, Verify converts the pixels to the screen's points for you. On Web they equal CSS pixels at the default device scale factor of 1, and you still pass screenshot pixels at any other. This holds from TypeScript 0.4.1, Python 0.2.1 and Go v0.3.1; earlier versions took iOS points.
|
|
30
|
+
|
|
31
|
+
`swipe('up')` moves the finger up, so content scrolls down, on every platform. Actions return once Revyl accepts them, so after a swipe, `openUrl` or `launchApp`, wait for the screen to settle before you `locate` or tap by coordinates.
|
|
32
|
+
|
|
33
|
+
```ts
|
|
34
|
+
import { open } from '@revyl/verify';
|
|
35
|
+
|
|
36
|
+
const session = await open('ios');
|
|
37
|
+
try {
|
|
38
|
+
await session.tap({ target: 'General' }); // first choice: describe the element
|
|
39
|
+
const about = await session.locate('About'); // a point in screenshot pixels
|
|
40
|
+
await session.tap(about); // so it goes straight to tap()
|
|
41
|
+
|
|
42
|
+
await session.screenshot({ path: '.verify/screen.png' }); // open it and find what to press
|
|
43
|
+
await session.tap({ x: 660, y: 1500 }); // a position read off the screenshot, unchanged
|
|
44
|
+
await session.swipe('up', { from: { x: 660, y: 2000 } }); // swipe from a screenshot position too
|
|
45
|
+
const { width, height } = await session.screenSize(); // the screenshot's size
|
|
46
|
+
console.log(`x is 0 to ${width - 1}, y is 0 to ${height - 1}`);
|
|
47
|
+
} finally {
|
|
48
|
+
await session.close();
|
|
49
|
+
}
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
The [Python](https://github.com/RevylAI/verify/blob/v0.4.1/python/README.md#exact-actions) and [Go](https://github.com/RevylAI/verify/blob/v0.4.1/go/README.md#exact-actions) READMEs state the same rule with each language's names. [api.md](api.md#exact-actions) lists every call and its errors.
|
|
53
|
+
|
|
25
54
|
## Rules
|
|
26
55
|
|
|
27
56
|
Mac computers are early access. Ask support@revyl.ai about access before starting a computer; do not repeat an uncertain start/stop to probe availability.
|
package/docs/api.md
CHANGED
|
@@ -84,7 +84,7 @@ Web only, hosted or normal. Sends one deterministic `open_url` action and checks
|
|
|
84
84
|
|
|
85
85
|
## Exact actions
|
|
86
86
|
|
|
87
|
-
**Since 0.4.0. Early access.** In TypeScript (`@revyl/verify` 0.4.0), Python (`revyl-verify` 0.2.0) and Go (v0.3.0); the [Python](https://github.com/RevylAI/verify/blob/v0.4.
|
|
87
|
+
**Since 0.4.0. Early access.** In TypeScript (`@revyl/verify` 0.4.0), Python (`revyl-verify` 0.2.0) and Go (v0.3.0); the [Python](https://github.com/RevylAI/verify/blob/v0.4.1/python/README.md#exact-actions) and [Go](https://github.com/RevylAI/verify/blob/v0.4.1/go/README.md#exact-actions) READMEs show them with each language's names. Web, iOS and Android sessions take device actions that need no agent.
|
|
88
88
|
|
|
89
89
|
| Call | Web | iOS | Android | Mac computer |
|
|
90
90
|
| --- | --- | --- | --- | --- |
|
|
@@ -97,9 +97,9 @@ Web only, hosted or normal. Sends one deterministic `open_url` action and checks
|
|
|
97
97
|
| `screenshot()` | Yes | Yes | Yes | Yes |
|
|
98
98
|
| `exec`, files, processes | No, open a Mac computer | No, open a Mac computer | No, open a Mac computer | Yes |
|
|
99
99
|
|
|
100
|
-
|
|
100
|
+
A call a platform doesn't support throws `NOT_AVAILABLE_ON_PLATFORM`, and its message names the alternative.
|
|
101
101
|
|
|
102
|
-
A location is `{ x, y }` in
|
|
102
|
+
A location is `{ x, y }` in screenshot pixels (see [Coordinates](#coordinates)) or `{ target: 'Login button' }`, which Revyl's grounding model finds on the current screen.
|
|
103
103
|
|
|
104
104
|
| Call | What it does |
|
|
105
105
|
| --- | --- |
|
|
@@ -111,11 +111,21 @@ A location is `{ x, y }` in device units (iOS points, Android and Web pixels, fr
|
|
|
111
111
|
| `openUrl(url)` | Opens a URL or deep link on a device, or loads a URL in a browser |
|
|
112
112
|
| `launchApp(bundleId)` | Launches an installed app on a device |
|
|
113
113
|
| `locate(target)` | Resolves with Revyl's best match on the current screen, without acting |
|
|
114
|
-
| `screenSize()` | Resolves with `{ width, height }` in
|
|
114
|
+
| `screenSize()` | Resolves with the screenshot's `{ width, height }` in pixels |
|
|
115
115
|
|
|
116
116
|
A failed action throws `ACTION_FAILED`, including when Revyl cannot find a target or nothing at a point can take the action. One whose outcome Revyl can't confirm throws `ACTION_UNCERTAIN` or `TIMEOUT`, so check the session before acting again. Actions are never repeated. If Verify can't reach Revyl, the action fails with `API_ERROR` and isn't resent either, so check the session before trying it again.
|
|
117
117
|
|
|
118
|
-
Exact actions return when Revyl has accepted them, not when the screen has finished changing. After `swipe`, `openUrl` or `launchApp`, let the screen settle or check it before you `locate` or tap coordinates. `locate` returns Revyl's best match on the current screen and can return a point for an element that is off screen, so bring the element on screen first
|
|
118
|
+
Exact actions return when Revyl has accepted them, not when the screen has finished changing. After `swipe`, `openUrl` or `launchApp`, let the screen settle or check it before you `locate` or tap coordinates. `locate` returns Revyl's best match on the current screen and can return a point for an element that is off screen, so bring the element on screen first. A point outside `screenSize()` is off screen, and an action at it is refused.
|
|
119
|
+
|
|
120
|
+
### Coordinates
|
|
121
|
+
|
|
122
|
+
Prefer a description, such as `tap({ target: 'Sign in button' })`: Revyl finds the element on the current screen, and you never handle coordinates. Otherwise a point is `{ x, y }` in screenshot pixels on every platform: whole numbers from the top-left corner of the image `screenshot()` returns. So a position you read off a screenshot goes to `tap()`, `type()` and `swipe()` unchanged. `screenSize()` returns the screenshot's width and height, and `locate()` and the points `tap()` and `type()` return use the same pixels. A point outside the screenshot is refused with `INVALID_ARGUMENT` before any action is sent, and the message names the valid range. On iOS, Verify converts the pixels to the screen's points for you. On Web they equal CSS pixels at the default device scale factor of 1, and you still pass screenshot pixels at any other. This holds from TypeScript 0.4.1, Python 0.2.1 and Go v0.3.1; earlier versions took iOS points.
|
|
123
|
+
|
|
124
|
+
```ts
|
|
125
|
+
await session.screenshot({ path: '.verify/screen.png' }); // open it and find the element
|
|
126
|
+
await session.tap({ x: 660, y: 1500 }); // the position as you read it, on every platform
|
|
127
|
+
const { width, height } = await session.screenSize(); // the screenshot's size: x is 0 to width - 1
|
|
128
|
+
```
|
|
119
129
|
|
|
120
130
|
## session.report()
|
|
121
131
|
|
|
@@ -162,7 +172,7 @@ When inspecting `revyl device report --session-id <id>`, read validation verdict
|
|
|
162
172
|
|
|
163
173
|
## Your own model
|
|
164
174
|
|
|
165
|
-
**Since 0.4.0. Early access.** In TypeScript (`@revyl/verify` 0.4.0), Python (`revyl-verify` 0.2.0) and Go (v0.3.0); the [Python](https://github.com/RevylAI/verify/blob/v0.4.
|
|
175
|
+
**Since 0.4.0. Early access.** In TypeScript (`@revyl/verify` 0.4.0), Python (`revyl-verify` 0.2.0) and Go (v0.3.0); the [Python](https://github.com/RevylAI/verify/blob/v0.4.1/python/README.md#your-own-model) and [Go](https://github.com/RevylAI/verify/blob/v0.4.1/go/README.md#your-own-model) READMEs show them with each language's names. Runs instructions and validations on a model you choose, in your process, instead of Revyl's agent. Revyl still provides the browser or device. Leave `model` out and Revyl's agent runs the step, as before.
|
|
166
176
|
|
|
167
177
|
```ts
|
|
168
178
|
const session = await open('ios', { model: 'anthropic/claude-sonnet-5-5' }); // every step on your model
|
package/docs/configuration.md
CHANGED
|
@@ -36,7 +36,7 @@ Mac targets are early access. Ask support@revyl.ai about access. See [computer a
|
|
|
36
36
|
| `app.url` | A public HTTPS URL to open, registered as a build each time a session opens. |
|
|
37
37
|
| `app.buildId` | An exact Revyl web build to open instead of a URL. |
|
|
38
38
|
| `app.branch` | Branch to register the URL on, or to take the newest build from. |
|
|
39
|
-
| `viewport` | `{ width, height, deviceScaleFactor? }` in CSS pixels |
|
|
39
|
+
| `viewport` | `{ width, height, deviceScaleFactor? }` in CSS pixels. Screenshots, and so exact-action coordinates, are `deviceScaleFactor` times the CSS size |
|
|
40
40
|
| `signIn` | `{ path, secret, persona? }`: a test-only sign-in route. See [web.md](web.md). |
|
|
41
41
|
| `startTimeout` | Milliseconds to wait for the browser. Default 600000. |
|
|
42
42
|
| `idleTimeoutSeconds` | Revyl stops the session after this much inactivity. Default 600. |
|
package/docs/errors.md
CHANGED
|
@@ -37,7 +37,7 @@ try {
|
|
|
37
37
|
|
|
38
38
|
| Code | What happened | What to do |
|
|
39
39
|
| --- | --- | --- |
|
|
40
|
-
| `INVALID_ARGUMENT` | A call got a bad value: an unknown target name, an empty step, two steps at once on one session | The message names the argument |
|
|
40
|
+
| `INVALID_ARGUMENT` | A call got a bad value: an unknown target name, an empty step, two steps at once on one session, a point outside the screenshot | The message names the argument |
|
|
41
41
|
| `CONFIG_NOT_FOUND` | `open('name')` found no `verify.config.ts` at or above the current folder | Run `npx @revyl/verify init`, pass `options.config`, or pass a target object |
|
|
42
42
|
| `CONFIG_INVALID` | The config failed validation or did not load | The message names the exact option and the rule |
|
|
43
43
|
| `AUTH_MISSING` | No `REVYL_API_KEY` and no saved login | Run `npx @revyl/verify login` in a terminal, or set `REVYL_API_KEY` |
|
package/docs/llms.txt
CHANGED
|
@@ -16,4 +16,8 @@ Mac computers are early access; ask support@revyl.ai about access. See computers
|
|
|
16
16
|
- [CI and test runners](ci.md): GitHub Actions, node:test, Vitest, Jest, Playwright Test
|
|
17
17
|
- [Use your own model](own-model.md): run instructions and validations on Claude, GPT, Gemini, a local model or any OpenAI-compatible endpoint; providers, limits, privacy and troubleshooting
|
|
18
18
|
- [Errors](errors.md): every error code and what to do about it
|
|
19
|
-
- [For coding agents](agents.md): the workflow and rules for agents using Verify
|
|
19
|
+
- [For coding agents](agents.md): the workflow and rules for agents using Verify, and the coordinate rule for exact actions
|
|
20
|
+
|
|
21
|
+
## Exact actions and coordinates
|
|
22
|
+
|
|
23
|
+
Prefer a description, such as `tap({ target: 'Sign in button' })`: Revyl finds the element on the current screen, and you never handle coordinates. Otherwise a point is `{ x, y }` in screenshot pixels on every platform: whole numbers from the top-left corner of the image `screenshot()` returns. So a position you read off a screenshot goes to `tap()`, `type()` and `swipe()` unchanged. `screenSize()` returns the screenshot's width and height, and `locate()` and the points `tap()` and `type()` return use the same pixels. A point outside the screenshot is refused with `INVALID_ARGUMENT` before any action is sent, and the message names the valid range. On iOS, Verify converts the pixels to the screen's points for you. On Web they equal CSS pixels at the default device scale factor of 1, and you still pass screenshot pixels at any other. This holds from TypeScript 0.4.1, Python 0.2.1 and Go v0.3.1; earlier versions took iOS points. See [For coding agents](agents.md#exact-actions-and-coordinates) for a runnable example.
|
package/docs/mobile.md
CHANGED
|
@@ -50,7 +50,7 @@ For Android Settings, instruct the agent to scroll or navigate until the OS vers
|
|
|
50
50
|
|
|
51
51
|
## Exact actions
|
|
52
52
|
|
|
53
|
-
**Since 0.4.0. Early access.** In TypeScript (`@revyl/verify` 0.4.0), Python (`revyl-verify` 0.2.0) and Go (v0.3.0); the [Python](https://github.com/RevylAI/verify/blob/v0.4.
|
|
53
|
+
**Since 0.4.0. Early access.** In TypeScript (`@revyl/verify` 0.4.0), Python (`revyl-verify` 0.2.0) and Go (v0.3.0); the [Python](https://github.com/RevylAI/verify/blob/v0.4.1/python/README.md#exact-actions) and [Go](https://github.com/RevylAI/verify/blob/v0.4.1/go/README.md#exact-actions) READMEs show them with each language's names. When you know exactly what to press, skip the agent. Exact actions run on the device as given, with no model, and mix freely with `instruction()` and `validation()` in one script:
|
|
54
54
|
|
|
55
55
|
```ts
|
|
56
56
|
import { open } from '@revyl/verify';
|
|
@@ -64,7 +64,7 @@ try {
|
|
|
64
64
|
await session.tap({ target: 'Back button' });
|
|
65
65
|
|
|
66
66
|
await session.swipe('down'); // the finger moves down, so the list scrolls to the top
|
|
67
|
-
const search = await session.locate('Search field'); //
|
|
67
|
+
const search = await session.locate('Search field'); // a point in screenshot pixels
|
|
68
68
|
await session.tap(search); // a tap by coordinates
|
|
69
69
|
await session.type(search, 'Wallpaper');
|
|
70
70
|
await session.pressKey('enter');
|
|
@@ -75,7 +75,7 @@ try {
|
|
|
75
75
|
}
|
|
76
76
|
```
|
|
77
77
|
|
|
78
|
-
A location is `{ target: 'a description' }`, which Revyl's grounding model finds on the current screen, or `{ x, y }` from the top-left corner
|
|
78
|
+
A location is `{ target: 'a description' }`, which Revyl's grounding model finds on the current screen, or `{ x, y }` in screenshot pixels from the top-left corner; `screenSize()` gives the bounds. `type()` taps the field and clears it first unless you pass `{ clear: false }`. `pressKey()` takes `'enter'` or `'backspace'`.
|
|
79
79
|
|
|
80
80
|
On Android, `back()` presses the system back button. On both platforms, `launchApp(bundleId)` brings an installed app to the front and `openUrl('myapp://orders/42')` opens a deep link. A failed action throws `ACTION_FAILED`. One whose outcome Revyl can't confirm throws `ACTION_UNCERTAIN` or `TIMEOUT`, so check the session before acting again. Actions are never repeated. If Verify can't reach Revyl, the action fails with `API_ERROR` and isn't resent either, so check the session before trying it again. [api.md](api.md#exact-actions) lists every call.
|
|
81
81
|
|
|
@@ -90,6 +90,8 @@ On Android, `back()` presses the system back button. On both platforms, `launchA
|
|
|
90
90
|
| `screenshot()` | Yes | Yes | Yes | Yes |
|
|
91
91
|
| `exec`, files, processes | No, open a Mac computer | No, open a Mac computer | No, open a Mac computer | Yes |
|
|
92
92
|
|
|
93
|
-
Coordinates
|
|
93
|
+
**Coordinates.** Prefer a description, such as `tap({ target: 'Sign in button' })`: Revyl finds the element on the current screen, and you never handle coordinates. Otherwise a point is `{ x, y }` in screenshot pixels on every platform: whole numbers from the top-left corner of the image `screenshot()` returns. So a position you read off a screenshot goes to `tap()`, `type()` and `swipe()` unchanged. `screenSize()` returns the screenshot's width and height, and `locate()` and the points `tap()` and `type()` return use the same pixels. A point outside the screenshot is refused with `INVALID_ARGUMENT` before any action is sent, and the message names the valid range. On iOS, Verify converts the pixels to the screen's points for you. On Web they equal CSS pixels at the default device scale factor of 1, and you still pass screenshot pixels at any other. This holds from TypeScript 0.4.1, Python 0.2.1 and Go v0.3.1; earlier versions took iOS points.
|
|
94
|
+
|
|
95
|
+
A call a platform doesn't support throws `NOT_AVAILABLE_ON_PLATFORM`, and its message names the alternative.
|
|
94
96
|
|
|
95
97
|
When you pass a [`model`](own-model.md), your model uses `tap`, `type`, `swipe`, `pressKey`, `openUrl` and, where the platform has it, `back` inside `instruction()`.
|
package/docs/own-model.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Use your own model
|
|
2
2
|
|
|
3
|
-
**Since 0.4.0. Early access.** In TypeScript (`@revyl/verify` 0.4.0), Python (`revyl-verify` 0.2.0) and Go (v0.3.0); the [Python](https://github.com/RevylAI/verify/blob/v0.4.
|
|
3
|
+
**Since 0.4.0. Early access.** In TypeScript (`@revyl/verify` 0.4.0), Python (`revyl-verify` 0.2.0) and Go (v0.3.0); the [Python](https://github.com/RevylAI/verify/blob/v0.4.1/python/README.md#your-own-model) and [Go](https://github.com/RevylAI/verify/blob/v0.4.1/go/README.md#your-own-model) READMEs show them with each language's names. Verify normally runs `instruction()` and `validation()` on Revyl's agent. Pass a `model` and the agent loop runs in your process instead, on a model you choose, with your own key. Revyl still provides the cloud browser or device and, by default, finds the elements your model names on screen.
|
|
4
4
|
|
|
5
5
|
Use it when you want a specific model, your own prompts and key, or faster steps on a cheap model. Leave `model` out and nothing changes.
|
|
6
6
|
|
|
@@ -52,7 +52,7 @@ log.Println(step.Model.ID, len(step.Model.Turns), step.Model.InputTokens, step.M
|
|
|
52
52
|
_, err = s.Validation(ctx, "The screen shows the iOS version")
|
|
53
53
|
```
|
|
54
54
|
|
|
55
|
-
The [Python](https://github.com/RevylAI/verify/blob/v0.4.
|
|
55
|
+
The [Python](https://github.com/RevylAI/verify/blob/v0.4.1/python/README.md#your-own-model) and [Go](https://github.com/RevylAI/verify/blob/v0.4.1/go/README.md#your-own-model) READMEs have the full reference for each.
|
|
56
56
|
|
|
57
57
|
Set `model` on `open()` to run every step on it, or on one step to override it:
|
|
58
58
|
|
package/docs/web.md
CHANGED
|
@@ -53,7 +53,7 @@ Revyl runs one step at a time on a session. A step sent while another is still r
|
|
|
53
53
|
|
|
54
54
|
## Exact actions
|
|
55
55
|
|
|
56
|
-
**Since 0.4.0. Early access.** In TypeScript (`@revyl/verify` 0.4.0), Python (`revyl-verify` 0.2.0) and Go (v0.3.0); the [Python](https://github.com/RevylAI/verify/blob/v0.4.
|
|
56
|
+
**Since 0.4.0. Early access.** In TypeScript (`@revyl/verify` 0.4.0), Python (`revyl-verify` 0.2.0) and Go (v0.3.0); the [Python](https://github.com/RevylAI/verify/blob/v0.4.1/python/README.md#exact-actions) and [Go](https://github.com/RevylAI/verify/blob/v0.4.1/go/README.md#exact-actions) READMEs show them with each language's names. When you know exactly what to click, skip the agent. Exact actions run in the browser as given, with no model, and mix freely with `instruction()` and `validation()`:
|
|
57
57
|
|
|
58
58
|
```ts
|
|
59
59
|
import { open } from '@revyl/verify';
|
|
@@ -66,7 +66,7 @@ try {
|
|
|
66
66
|
await session.validation('A list of products is shown');
|
|
67
67
|
await session.swipe('up'); // the finger moves up, so the page scrolls down
|
|
68
68
|
await session.instruction('Add the Sauce Labs Onesie to the cart'); // hand the fuzzy part to the agent
|
|
69
|
-
const cart = await session.locate('Shopping cart icon'); //
|
|
69
|
+
const cart = await session.locate('Shopping cart icon'); // a point in screenshot pixels
|
|
70
70
|
await session.tap(cart); // a tap by coordinates
|
|
71
71
|
await session.validation('The cart lists the Sauce Labs Onesie');
|
|
72
72
|
await session.back(); // browser history
|
|
@@ -76,7 +76,7 @@ try {
|
|
|
76
76
|
}
|
|
77
77
|
```
|
|
78
78
|
|
|
79
|
-
A location is `{ target: 'a description' }`, which Revyl's grounding model finds on the current page, or `{ x, y }` from the top-left corner
|
|
79
|
+
A location is `{ target: 'a description' }`, which Revyl's grounding model finds on the current page, or `{ x, y }` in screenshot pixels from the top-left corner; `screenSize()` gives the bounds and `locate(target)` gives an element's point without acting. `type()` clicks the field and clears it first unless you pass `{ clear: false }`. `pressKey()` takes `'enter'` or `'backspace'`, and `openUrl(url)` loads a page like `navigate(url)`. A failed action throws `ACTION_FAILED`. One whose outcome Revyl can't confirm throws `ACTION_UNCERTAIN` or `TIMEOUT`, so check the session before acting again. Actions are never repeated. If Verify can't reach Revyl, the action fails with `API_ERROR` and isn't resent either, so check the session before trying it again. [api.md](api.md#exact-actions) lists every call.
|
|
80
80
|
|
|
81
81
|
| Call | Web | iOS | Android | Mac computer |
|
|
82
82
|
| --- | --- | --- | --- | --- |
|
|
@@ -89,6 +89,8 @@ A location is `{ target: 'a description' }`, which Revyl's grounding model finds
|
|
|
89
89
|
| `screenshot()` | Yes | Yes | Yes | Yes |
|
|
90
90
|
| `exec`, files, processes | No, open a Mac computer | No, open a Mac computer | No, open a Mac computer | Yes |
|
|
91
91
|
|
|
92
|
-
Coordinates
|
|
92
|
+
**Coordinates.** Prefer a description, such as `tap({ target: 'Sign in button' })`: Revyl finds the element on the current screen, and you never handle coordinates. Otherwise a point is `{ x, y }` in screenshot pixels on every platform: whole numbers from the top-left corner of the image `screenshot()` returns. So a position you read off a screenshot goes to `tap()`, `type()` and `swipe()` unchanged. `screenSize()` returns the screenshot's width and height, and `locate()` and the points `tap()` and `type()` return use the same pixels. A point outside the screenshot is refused with `INVALID_ARGUMENT` before any action is sent, and the message names the valid range. On iOS, Verify converts the pixels to the screen's points for you. On Web they equal CSS pixels at the default device scale factor of 1, and you still pass screenshot pixels at any other. This holds from TypeScript 0.4.1, Python 0.2.1 and Go v0.3.1; earlier versions took iOS points.
|
|
93
|
+
|
|
94
|
+
A call a platform doesn't support throws `NOT_AVAILABLE_ON_PLATFORM`, and its message names the alternative.
|
|
93
95
|
|
|
94
96
|
When you pass a [`model`](own-model.md), your model uses `tap`, `type`, `swipe`, `pressKey`, `openUrl` and, where the platform has it, `back` inside `instruction()`.
|