@revyl/verify 0.3.2 → 0.4.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +72 -0
- package/README.md +60 -23
- package/dist/agent/image.d.ts +7 -0
- package/dist/agent/image.js +29 -0
- package/dist/agent/loop.d.ts +97 -0
- package/dist/agent/loop.js +215 -0
- package/dist/agent/model.d.ts +91 -0
- package/dist/agent/model.js +255 -0
- package/dist/agent/providers.d.ts +23 -0
- package/dist/agent/providers.js +58 -0
- package/dist/api/client.d.ts +1 -0
- package/dist/api/client.js +2 -2
- package/dist/auth.js +2 -2
- package/dist/cli/init.d.ts +1 -0
- package/dist/cli/init.js +34 -9
- package/dist/cli/login.js +7 -7
- package/dist/cli/main.js +84 -29
- package/dist/computer-session.d.ts +9 -0
- package/dist/computer-session.js +27 -0
- package/dist/errors.d.ts +2 -2
- package/dist/index.d.ts +5 -0
- package/dist/index.js +2 -0
- package/dist/open.d.ts +6 -0
- package/dist/open.js +6 -1
- package/dist/session.d.ts +77 -2
- package/dist/session.js +294 -12
- package/docs/README.md +4 -3
- package/docs/agents.md +34 -5
- package/docs/api.md +99 -13
- package/docs/computers.md +9 -11
- package/docs/configuration.md +12 -8
- package/docs/errors.md +8 -6
- package/docs/getting-started.md +22 -17
- package/docs/llms.txt +11 -6
- package/docs/mobile.md +48 -0
- package/docs/own-model.md +216 -0
- package/docs/web.md +58 -1
- package/package.json +2 -3
package/docs/api.md
CHANGED
|
@@ -6,22 +6,21 @@ import { attach, open, defineConfig, VerifyError, isVerifyError } from '@revyl/v
|
|
|
6
6
|
|
|
7
7
|
## open(target, options?)
|
|
8
8
|
|
|
9
|
-
|
|
9
|
+
Starts a Revyl cloud browser, device, or Mac computer. Mobile first-screen readiness is [best-effort](mobile.md#steps), not a foreground/rendered-readiness guarantee. Computer starts retain a 10-minute default timeout; an unclaimed start may fail after 120 seconds with no-host context.
|
|
10
10
|
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
Both computer operating systems are early access. Ask support@revyl.ai about access.
|
|
11
|
+
Mac computers are early access. Ask support@revyl.ai about access.
|
|
14
12
|
|
|
15
13
|
```ts
|
|
16
14
|
const web = await open('web'); // a target named in verify.config.ts
|
|
17
15
|
const ios = await open({ platform: 'ios', app: { appId: '…' } }); // or a target object, no config needed
|
|
18
16
|
const mac = await open({ platform: 'mac' }); // computer, no app needed
|
|
19
|
-
const linux = await open({ platform: 'linux' }); // same computer surface
|
|
20
17
|
```
|
|
21
18
|
|
|
22
19
|
- `target`: a target name from `verify.config.ts`, or a target object with the same shape as a config entry (see [configuration.md](configuration.md)). Verify reads the nearest `verify.config.ts` at or above the current folder.
|
|
23
20
|
- `options.config`: the path of a config file to read instead.
|
|
24
21
|
- `options.log`: where progress lines go. By default Verify writes the session link and each step's outcome to stderr, prefixed with `[verify]`. Pass a function to receive the lines yourself, or `false` to turn them off.
|
|
22
|
+
- `options.model`: runs every instruction and validation on your own model instead of Revyl's agent; see [Your own model](#your-own-model). Browsers and devices only; computers throw `NOT_AVAILABLE_ON_PLATFORM`.
|
|
23
|
+
- `options.agent`: the same with a custom agent; see [Custom agents](#custom-agents). Pass `model` or `agent`, not both.
|
|
25
24
|
|
|
26
25
|
`open` needs credentials: `REVYL_API_KEY` or a Revyl CLI login. It throws `AUTH_MISSING` without them, `CONFIG_NOT_FOUND` or `CONFIG_INVALID` for config problems, and `SESSION_START_FAILED` or `TIMEOUT` when the browser or device does not start.
|
|
27
26
|
|
|
@@ -53,7 +52,7 @@ The attachment, hosted-child and navigation APIs below are included in the TypeS
|
|
|
53
52
|
|
|
54
53
|
`open()` and `computer.open()` return the extended `OpenedSession` interface. The original `Session` interface keeps its existing required members, so applications implementing or mocking it do not need to add the new methods. Use `OpenedSession` when explicitly annotating a value that calls `report`, `navigate` or `open`.
|
|
55
54
|
|
|
56
|
-
Attaches to an active Mac
|
|
55
|
+
Attaches to an active Mac computer in your organization without creating a new one. `options.log` controls progress. The returned computer owns cleanup by default: `close()` stops it, and automatic cleanup also applies. Attaching requires a valid expiry and does not extend the lifetime. Invalid, inaccessible, inactive or expired computers fail without sending a stop request.
|
|
57
56
|
|
|
58
57
|
When attaching to a computer you do not own, such as a CI runner, pass `attach(computerId, { stopOnClose: false })`. The attachment is not registered for automatic cleanup. Its `close()` still cancels pending child starts and gives hosted children the same five-second cleanup phase, but never stops the computer; its owner must release it. Child cleanup failures remain observable. This option is explicit and is never inferred from the environment.
|
|
59
58
|
|
|
@@ -61,7 +60,7 @@ When attaching to a computer you do not own, such as a CI runner, pass `attach(c
|
|
|
61
60
|
|
|
62
61
|
Starts a child app session inside this computer, returning the normal session API. It inherits the parent's authenticated client, logging and step timeout; callers cannot route a child to another backend. Both `target.app.appId` and `target.app.buildId` are required. Web accepts an existing external HTTPS build, plus optional viewport/sign-in settings; native builds use the computer's fixed device profile and accept no bundle, artifact or device override. Named targets and initial URL overrides are not supported here.
|
|
63
62
|
|
|
64
|
-
Mac supports Web and iOS children.
|
|
63
|
+
Mac supports Web and iOS children. The server remains authoritative for access, capability and capacity. Children require and share the parent's known deadline. Closing a child leaves the parent running; closing the parent cancels pending starts and gives child cleanup at most five seconds total before requesting confirmed computer release. The parent stop also stops any remaining children, so a slow child cannot prevent that request. A failed child start never closes the parent.
|
|
65
64
|
|
|
66
65
|
```ts
|
|
67
66
|
const computer = await attach(existingComputerId, { log: false });
|
|
@@ -83,6 +82,51 @@ This requires an already-running service on the computer. The initial Web build
|
|
|
83
82
|
|
|
84
83
|
Web only, hosted or normal. Sends one deterministic `open_url` action and checks the returned navigation outcome. Accepts an HTTP(S) URL without credentials, whitespace, control characters or backslashes. Loopback addresses are accepted on every Web session: a hosted browser can reach its parent computer's loopback address; a normal cloud browser's localhost is its own runtime, not your machine. A cold development server may need a warm-up request before navigation succeeds; after an uncertain timeout, re-navigating is the caller's choice because the original request may still commit. The runtime is responsible for enforcing its navigation network policy. Navigation is not replayed by Verify and does not run an agent step. It resolves with no value, or throws a typed error. Computer and native sessions reject it locally.
|
|
85
84
|
|
|
85
|
+
## Exact actions
|
|
86
|
+
|
|
87
|
+
**Since 0.4.0. Early access.** In TypeScript (`@revyl/verify` 0.4.0), Python (`revyl-verify` 0.2.0) and Go (v0.3.0); the [Python](https://github.com/RevylAI/verify/blob/v0.4.1/python/README.md#exact-actions) and [Go](https://github.com/RevylAI/verify/blob/v0.4.1/go/README.md#exact-actions) READMEs show them with each language's names. Web, iOS and Android sessions take device actions that need no agent.
|
|
88
|
+
|
|
89
|
+
| Call | Web | iOS | Android | Mac computer |
|
|
90
|
+
| --- | --- | --- | --- | --- |
|
|
91
|
+
| `tap`, `type`, `swipe`, `pressKey`, `locate`, `screenSize` | Yes | Yes | Yes | No, use `exec()` and files |
|
|
92
|
+
| `back()` | Yes, browser history | No, tap the on-screen back control | Yes, the system back button | No |
|
|
93
|
+
| `openUrl(url)` | Yes, loads an http(s) URL in the tab | Yes, a URL or deep link | Yes, a URL or deep link | No |
|
|
94
|
+
| `launchApp(bundleId)` | No, use `openUrl` or `navigate` | Yes | Yes | No |
|
|
95
|
+
| `navigate(url)` | Yes | No, use `openUrl` | No, use `openUrl` | No |
|
|
96
|
+
| `instruction`, `validation`, your own model | Yes | Yes | Yes | No, use `exec()` |
|
|
97
|
+
| `screenshot()` | Yes | Yes | Yes | Yes |
|
|
98
|
+
| `exec`, files, processes | No, open a Mac computer | No, open a Mac computer | No, open a Mac computer | Yes |
|
|
99
|
+
|
|
100
|
+
A call a platform doesn't support throws `NOT_AVAILABLE_ON_PLATFORM`, and its message names the alternative.
|
|
101
|
+
|
|
102
|
+
A location is `{ x, y }` in screenshot pixels (see [Coordinates](#coordinates)) or `{ target: 'Login button' }`, which Revyl's grounding model finds on the current screen.
|
|
103
|
+
|
|
104
|
+
| Call | What it does |
|
|
105
|
+
| --- | --- |
|
|
106
|
+
| `tap(at)` | Taps and resolves with the point tapped |
|
|
107
|
+
| `type(at, text, { clear })` | Taps a text field and types into it, clearing it first unless `clear` is `false` |
|
|
108
|
+
| `swipe(direction, { from })` | Swipes from a location or the screen's center; `'up'` moves the finger up, which scrolls content down |
|
|
109
|
+
| `pressKey('enter' \| 'backspace')` | Presses a key on the focused field |
|
|
110
|
+
| `back()` | Browser history on Web, the back button on Android; iOS throws `NOT_AVAILABLE_ON_PLATFORM` |
|
|
111
|
+
| `openUrl(url)` | Opens a URL or deep link on a device, or loads a URL in a browser |
|
|
112
|
+
| `launchApp(bundleId)` | Launches an installed app on a device |
|
|
113
|
+
| `locate(target)` | Resolves with Revyl's best match on the current screen, without acting |
|
|
114
|
+
| `screenSize()` | Resolves with the screenshot's `{ width, height }` in pixels |
|
|
115
|
+
|
|
116
|
+
A failed action throws `ACTION_FAILED`, including when Revyl cannot find a target or nothing at a point can take the action. One whose outcome Revyl can't confirm throws `ACTION_UNCERTAIN` or `TIMEOUT`, so check the session before acting again. Actions are never repeated. If Verify can't reach Revyl, the action fails with `API_ERROR` and isn't resent either, so check the session before trying it again.
|
|
117
|
+
|
|
118
|
+
Exact actions return when Revyl has accepted them, not when the screen has finished changing. After `swipe`, `openUrl` or `launchApp`, let the screen settle or check it before you `locate` or tap coordinates. `locate` returns Revyl's best match on the current screen and can return a point for an element that is off screen, so bring the element on screen first. A point outside `screenSize()` is off screen, and an action at it is refused.
|
|
119
|
+
|
|
120
|
+
### Coordinates
|
|
121
|
+
|
|
122
|
+
Prefer a description, such as `tap({ target: 'Sign in button' })`: Revyl finds the element on the current screen, and you never handle coordinates. Otherwise a point is `{ x, y }` in screenshot pixels on every platform: whole numbers from the top-left corner of the image `screenshot()` returns. So a position you read off a screenshot goes to `tap()`, `type()` and `swipe()` unchanged. `screenSize()` returns the screenshot's width and height, and `locate()` and the points `tap()` and `type()` return use the same pixels. A point outside the screenshot is refused with `INVALID_ARGUMENT` before any action is sent, and the message names the valid range. On iOS, Verify converts the pixels to the screen's points for you. On Web they equal CSS pixels at the default device scale factor of 1, and you still pass screenshot pixels at any other. This holds from TypeScript 0.4.1, Python 0.2.1 and Go v0.3.1; earlier versions took iOS points.
|
|
123
|
+
|
|
124
|
+
```ts
|
|
125
|
+
await session.screenshot({ path: '.verify/screen.png' }); // open it and find the element
|
|
126
|
+
await session.tap({ x: 660, y: 1500 }); // the position as you read it, on every platform
|
|
127
|
+
const { width, height } = await session.screenSize(); // the screenshot's size: x is 0 to width - 1
|
|
128
|
+
```
|
|
129
|
+
|
|
86
130
|
## session.report()
|
|
87
131
|
|
|
88
132
|
Returns authenticated JSON metadata: `/execution/device-sessions/{id}` for normal and hosted app sessions, or the computer report for computers. It works after close, but fails locally with `NOT_FOUND` when no session ID is known. It does not download recordings or turn a device stop acknowledgment into proof of physical release. Treat returned metadata as private.
|
|
@@ -116,8 +160,9 @@ interface StepResult {
|
|
|
116
160
|
type: 'instruction' | 'validation';
|
|
117
161
|
text: string; // the step as sent, trimmed
|
|
118
162
|
passed: boolean;
|
|
119
|
-
reason?: string; // Revyl's explanation,
|
|
163
|
+
reason?: string; // Revyl's explanation, or your model's on a local agent
|
|
120
164
|
durationMs: number;
|
|
165
|
+
model?: ModelRun; // set when the step ran on your model
|
|
121
166
|
}
|
|
122
167
|
```
|
|
123
168
|
|
|
@@ -125,6 +170,47 @@ interface StepResult {
|
|
|
125
170
|
|
|
126
171
|
When inspecting `revyl device report --session-id <id>`, read validation verdicts from the step's `effective_status` and `validation_result`, with `validations_passed` / `total_validations` for the validation summary. `passed_steps` / `failed_steps` and transport `status` are not enough: a successfully executed validation can have a false verdict. The SDK's `StepResult.passed` reflects that verdict, not just transport success.
|
|
127
172
|
|
|
173
|
+
## Your own model
|
|
174
|
+
|
|
175
|
+
**Since 0.4.0. Early access.** In TypeScript (`@revyl/verify` 0.4.0), Python (`revyl-verify` 0.2.0) and Go (v0.3.0); the [Python](https://github.com/RevylAI/verify/blob/v0.4.1/python/README.md#your-own-model) and [Go](https://github.com/RevylAI/verify/blob/v0.4.1/go/README.md#your-own-model) READMEs show them with each language's names. Runs instructions and validations on a model you choose, in your process, instead of Revyl's agent. Revyl still provides the browser or device. Leave `model` out and Revyl's agent runs the step, as before.
|
|
176
|
+
|
|
177
|
+
```ts
|
|
178
|
+
const session = await open('ios', { model: 'anthropic/claude-sonnet-5-5' }); // every step on your model
|
|
179
|
+
await session.instruction('Open General, then About');
|
|
180
|
+
await session.validation('The screen shows the iOS version');
|
|
181
|
+
await session.instruction('Open Privacy', { model: 'openai/gpt-5.5' }); // or one step
|
|
182
|
+
|
|
183
|
+
// Your own OpenAI-compatible endpoint: vLLM, LiteLLM, a gateway
|
|
184
|
+
const custom = await open('ios', {
|
|
185
|
+
model: { id: 'qwen-3.8-27b', baseURL: 'https://api.cerebras.ai/v1', apiKey: process.env.CEREBRAS_API_KEY },
|
|
186
|
+
});
|
|
187
|
+
```
|
|
188
|
+
|
|
189
|
+
`options.model` is a `'provider/model'` string, with the key read from the provider's usual environment variable (and `ANTHROPIC_BASE_URL` or `OPENAI_BASE_URL` for a proxy), or `{ id, baseURL, apiKey?, headers?, provider?, grounding? }` for any OpenAI-compatible endpoint. [Use your own model](own-model.md) has the provider table, which providers are verified live, how to pick a model, what Revyl still sees, the limits and troubleshooting. A missing key throws `MODEL_KEY_MISSING` before any session starts; an unknown prefix throws `INVALID_ARGUMENT`. The model must accept images and tool calls.
|
|
190
|
+
|
|
191
|
+
The object form's key comes only from `apiKey`, never from the environment.
|
|
192
|
+
|
|
193
|
+
`StepResult.model` reports `{ id, grounding, turns, inputTokens, outputTokens }`, with each turn's action, model and action time, and token counts. Provider failures throw `MODEL_ERROR` with the key redacted; model calls are retried up to twice on rate limits and server errors. An instruction stops after 20 turns, and the step's `timeout` bounds the whole step.
|
|
194
|
+
|
|
195
|
+
## Custom agents
|
|
196
|
+
|
|
197
|
+
For more control, build the agent yourself with `localAgent()` and pass it as `agent` on `open()` or a step:
|
|
198
|
+
|
|
199
|
+
```ts
|
|
200
|
+
import { localAgent, anthropic, openaiCompatible } from '@revyl/verify';
|
|
201
|
+
|
|
202
|
+
const agent = localAgent({
|
|
203
|
+
model: openaiCompatible({ baseURL: 'http://localhost:11434/v1', model: 'qwen3-vl:8b' }),
|
|
204
|
+
grounding: 'model',
|
|
205
|
+
maxTurns: 10,
|
|
206
|
+
});
|
|
207
|
+
const session = await open('ios', { agent });
|
|
208
|
+
```
|
|
209
|
+
|
|
210
|
+
- `model`: a `ModelAdapter`. `anthropic({ model, apiKey, baseURL, maxTokens, maxRetries })` uses `ANTHROPIC_API_KEY` and `ANTHROPIC_BASE_URL` for what you leave out. `openaiCompatible({ model, baseURL, apiKey, headers, provider, maxTokens, maxRetries })` uses `OPENAI_BASE_URL` and `OPENAI_API_KEY` only when `baseURL` is left out, so a key never reaches another provider; it then sends OpenAI's `max_completion_tokens`. To wrap another SDK or runtime, implement `{ name, complete(request, signal) }`: it gets the system prompt, text, screenshot and tools, and returns the chosen tool call and token usage.
|
|
211
|
+
- `grounding`: `'revyl'` (default) as above. `'model'` has the model give screenshot coordinates instead, so Revyl receives only taps, typing and swipes, never element descriptions; it needs a model that points accurately.
|
|
212
|
+
- `maxTurns` (default 20, up to 100), `modelTimeout` per model call (default 60000), `maxImageSide` for screenshots sent to the model (default 1280).
|
|
213
|
+
|
|
128
214
|
## session.screenshot(options?)
|
|
129
215
|
|
|
130
216
|
Resolves with a PNG of the screen as a `Uint8Array`. With `options.path`, also writes it there, creating folders as needed.
|
|
@@ -137,7 +223,7 @@ For owning computers, cleanup must be confirmed; a terminal state alone does not
|
|
|
137
223
|
|
|
138
224
|
## Computer commands, files, and processes
|
|
139
225
|
|
|
140
|
-
Mac
|
|
226
|
+
Mac sessions provide `exec(argv: string[], options?)`, `readFile(path, options?)` returning `Buffer`, `writeFile(path, string | Uint8Array, options?)`, `stat(path, options?)` returning `{ kind, size }`, `processes(options?)` returning `{ id, state, exitCode? }[]`, and `terminate(id, options?)`. They throw `NOT_AVAILABLE_ON_PLATFORM` on web/iOS/Android. See [computers.md](computers.md) for complete signatures, limits, timeouts, and examples. Both support `screenshot()`; instruction/validation do not run on computers.
|
|
141
227
|
|
|
142
228
|
### `session.recording(): Promise<ComputerRecording | null>`
|
|
143
229
|
|
|
@@ -151,17 +237,17 @@ Computers only. First reads the report and requires terminal state plus confirme
|
|
|
151
237
|
| --- | --- |
|
|
152
238
|
| `viewerUrl` | The session in Revyl: a live view while it runs, its recording/report after. Use `revyl device report --session-id <id>` for web/mobile, or `revyl computer report <id>` for computers. |
|
|
153
239
|
| `id` | Revyl's session id |
|
|
154
|
-
| `platform` | `'web'`, `'ios'`, `'android'`,
|
|
155
|
-
| `expiresAt` | **Since 0.2.0.** Validated API expiry as a `Date`, or undefined for devices and older APIs. Computers have a hard
|
|
240
|
+
| `platform` | `'web'`, `'ios'`, `'android'`, or `'mac'` |
|
|
241
|
+
| `expiresAt` | **Since 0.2.0.** Validated API expiry as a `Date`, or undefined for devices and older APIs. Computers have a hard lifetime from create (two hours today); this value is the authoritative deadline. Operations/screenshot observation are bounded by expiry; close and recording remain possible. |
|
|
156
242
|
| `target` | The target name, or the platform for a target object |
|
|
157
243
|
| `closed` | Whether `close()` was called |
|
|
158
244
|
|
|
159
245
|
## Rules worth knowing
|
|
160
246
|
|
|
161
247
|
- **One step at a time.** Await each step before starting the next on the same session; a second concurrent step throws `INVALID_ARGUMENT`. Separate sessions can run in parallel.
|
|
162
|
-
- **No hidden retries.** Verify never repeats a step or a device action. A step whose outcome is unknown because the connection failed mid-request throws `ACTION_UNCERTAIN`.
|
|
248
|
+
- **No hidden retries.** Verify never repeats a step or a device action. A local agent retries only its own model calls. A step whose outcome is unknown because the connection failed mid-request throws `ACTION_UNCERTAIN`.
|
|
163
249
|
- **No memory between steps.** Each step is independent: Revyl does not carry values from one step to the next. Put everything a step needs into its text.
|
|
164
|
-
- **No replay or cache.** Every step runs
|
|
250
|
+
- **No replay or cache.** Every step runs its agent again, every time.
|
|
165
251
|
|
|
166
252
|
## Cleanup
|
|
167
253
|
|
package/docs/computers.md
CHANGED
|
@@ -1,16 +1,16 @@
|
|
|
1
|
-
# Mac
|
|
1
|
+
# Mac computers (early access)
|
|
2
2
|
|
|
3
|
-
Verify opens a Revyl cloud macOS
|
|
3
|
+
Verify opens a Revyl cloud macOS computer for commands, files, processes, and screenshots. It does not open an assigned physical computer. Mac computers are **early access**. Ask [support@revyl.ai](mailto:support@revyl.ai) about access.
|
|
4
4
|
|
|
5
5
|
```bash
|
|
6
6
|
export REVYL_API_KEY=...
|
|
7
7
|
npx @revyl/verify init --platform mac
|
|
8
|
-
node verify/example.
|
|
8
|
+
node verify/example.mts
|
|
9
9
|
```
|
|
10
10
|
|
|
11
11
|
`init` installs the package and checks credentials. Running the script starts a paid computer. No app id, build, device selector, image selector, or local Mac is needed. The generated script checks commands and a file round trip without agent steps.
|
|
12
12
|
|
|
13
|
-
|
|
13
|
+
Installed tools, paths, resources, and desktop dimensions are not fixed SDK guarantees.
|
|
14
14
|
|
|
15
15
|
## Commands and files
|
|
16
16
|
|
|
@@ -45,7 +45,7 @@ try {
|
|
|
45
45
|
|
|
46
46
|
Paths are relative to the computer's workspace: no leading slash, empty components, `.` or `..`, backslashes, colons, or control characters. A path is at most 1,024 UTF-8 bytes. To make a directory, run a command such as `['mkdir', '-p', 'output']` before writing into it. There is no recursive sync or snapshot API.
|
|
47
47
|
|
|
48
|
-
Operation `timeout` is milliseconds, defaults to 30,000, and must be an integer from 1 to 300,000. It is also bounded by the run's remaining lifetime. A computer has a hard
|
|
48
|
+
Operation `timeout` is milliseconds, defaults to 30,000, and must be an integer from 1 to 300,000. It is also bounded by the run's remaining lifetime. A computer has a hard lifetime **from create** (two hours today; `expiresAt` is authoritative), not from readiness or last activity; `idleTimeoutSeconds` and the agent `stepTimeout` do not configure computer lifetime or commands. `expiresAt` is a `Date` when the API returns `expires_at`, and otherwise undefined (including on devices and older APIs). Verify refuses new operations after that deadline and bounds operation/screenshot observation by it; expiry does not prevent explicit close or recording retrieval. Revyl allows at most 256 operations per run.
|
|
49
49
|
|
|
50
50
|
## Recording metadata
|
|
51
51
|
|
|
@@ -55,25 +55,23 @@ The HTTPS URL is signed and short-lived. Keep it private, including its query st
|
|
|
55
55
|
|
|
56
56
|
## Availability and errors
|
|
57
57
|
|
|
58
|
-
Mac
|
|
58
|
+
Mac computers are early access. A `503 runtime_unavailable` or typed `computer_not_enabled` becomes `NOT_AVAILABLE_ON_PLATFORM` with this availability guidance. A `429 computer_limit` becomes `COMPUTER_LIMIT`: queued work (including iOS) or the concurrency limit can cause it, and the response does not identify which. Verify does not retry the create or send a stop after these definite admission refusals. `operation_limit` remains `SESSION_BUSY`. Availability is determined by the server.
|
|
59
59
|
|
|
60
60
|
The default `startTimeout` remains 600000 ms; override it explicitly if needed. An unclaimed start can transition to `failed` with `failure_reason: 'no_host'` after 120 seconds from create. Verify stops readiness polling and throws `SESSION_START_FAILED` with that context, then attempts bounded cleanup. This is not a 120-second timeout for every start: claimed guests may take longer. `device_limit` also yields `SESSION_START_FAILED`, with queued-work/concurrency context. A start timeout or unconfirmed close does not prove resources were released. Inspect the session and contact support rather than blindly repeating a start or stop.
|
|
61
61
|
|
|
62
|
-
`instruction()` and `validation()` throw `NOT_AVAILABLE_ON_PLATFORM` on
|
|
62
|
+
`instruction()` and `validation()` throw `NOT_AVAILABLE_ON_PLATFORM` on computers without sending agent requests. Screenshots are supported, but screenshot readiness is separate from command readiness. Agent control of the screen is not part of this release. On web, iOS, and Android, shell/file/process methods throw `NOT_AVAILABLE_ON_PLATFORM`.
|
|
63
63
|
|
|
64
64
|
For the first 30 seconds after a computer becomes active, `screenshot()` retries exact structured `screen_unavailable` responses (HTTP 409 or 503) and `screen_starting` (HTTP 409). Backoff starts at 500 ms and doubles up to 2 seconds; the fixed window does not restart on each call. Cancellation, close, expiry, and the observation deadline still bound the wait. Exhausting that window yields `SESSION_BUSY` with a desktop-not-ready message, rather than requiring a caller sleep. Authentication failures, unrelated errors, malformed responses, and transport failures are not replayed. Commands and other mutations are never retried by this screenshot behavior.
|
|
65
65
|
|
|
66
|
-
Linux named and inline targets use the same computer surface as Mac. `init --platform linux` is intentionally unavailable; use an inline target or write its config entry.
|
|
67
|
-
|
|
68
66
|
Missing files and access denials throw `NOT_FOUND` and `ACCESS_DENIED`. An operation timeout with confirmed termination throws `TIMEOUT`; unconfirmed termination or execution throws `ACTION_UNCERTAIN`. Neither implies rollback. Execution outcome, evidence recording, and physical release are independent: an observed successful command can return its result while logging pending/failed report evidence. Do not repeat a mutation merely because its response was lost. Check the computer in Revyl before deciding what to do next. See [errors.md](errors.md).
|
|
69
67
|
|
|
70
68
|
## Lifecycle and transport
|
|
71
69
|
|
|
72
|
-
Use `attach(computerId)` to take cleanup ownership of an existing active computer without creating one. `computer.open(target)` starts Web
|
|
70
|
+
Use `attach(computerId)` to take cleanup ownership of an existing active computer without creating one. `computer.open(target)` starts Web and iOS children on a Mac. A child requires an existing app/build pair and inherits the parent's client and expiry. Parent close attempts child cleanup for at most five seconds before requesting computer release; the server also stops remaining children. See [the hosted API](api.md#computeropentarget) for arguments, localhost navigation, and cleanup semantics. These are app sessions, not natural-language control of the computer desktop itself.
|
|
73
71
|
|
|
74
72
|
For a computer owned by another caller, such as a CI runner, use `attach(computerId, { stopOnClose: false })`. It is not registered for automatic cleanup. Explicit close still cancels pending child starts and closes hosted children within the five-second cleanup phase, but never sends a stop for the computer. The computer's owner is responsible for releasing it; a child-cleanup failure does not authorize stopping the parent.
|
|
75
73
|
|
|
76
|
-
Verify mints a session UUID, creates `/api/v1/computer-runs` with `os: 'macos'` for `platform: 'mac'
|
|
74
|
+
Verify mints a session UUID, creates `/api/v1/computer-runs` with `os: 'macos'` for `platform: 'mac'`, then polls `GET /computer-runs/{id}/report` until `state: 'active'`, which means it is ready. `starting` alone is not readiness. The public Mac target spelling is `mac`, not `macos`.
|
|
77
75
|
|
|
78
76
|
Each command is submitted once to `/computer-runs/{id}/operations` with its own UUID `request_id` and `Prefer: respond-async`. The operation id is UUIDv5 with the computer UUID as namespace and the request UUID string as name. A `202` response has JSON `null`; Verify reads the operation by id until it settles instead of submitting it again.
|
|
79
77
|
|
package/docs/configuration.md
CHANGED
|
@@ -20,7 +20,7 @@ Unknown keys are rejected, so a typo fails loudly instead of being ignored.
|
|
|
20
20
|
|
|
21
21
|
## Top-level options
|
|
22
22
|
|
|
23
|
-
Mac
|
|
23
|
+
Mac targets are early access. Ask support@revyl.ai about access. See [computer availability](computers.md#availability-and-errors).
|
|
24
24
|
|
|
25
25
|
| Option | Default | Meaning |
|
|
26
26
|
| --- | --- | --- |
|
|
@@ -36,7 +36,7 @@ Mac and Linux targets are early access. Ask support@revyl.ai about access. See [
|
|
|
36
36
|
| `app.url` | A public HTTPS URL to open, registered as a build each time a session opens. |
|
|
37
37
|
| `app.buildId` | An exact Revyl web build to open instead of a URL. |
|
|
38
38
|
| `app.branch` | Branch to register the URL on, or to take the newest build from. |
|
|
39
|
-
| `viewport` | `{ width, height, deviceScaleFactor? }` in CSS pixels |
|
|
39
|
+
| `viewport` | `{ width, height, deviceScaleFactor? }` in CSS pixels. Screenshots, and so exact-action coordinates, are `deviceScaleFactor` times the CSS size |
|
|
40
40
|
| `signIn` | `{ path, secret, persona? }`: a test-only sign-in route. See [web.md](web.md). |
|
|
41
41
|
| `startTimeout` | Milliseconds to wait for the browser. Default 600000. |
|
|
42
42
|
| `idleTimeoutSeconds` | Revyl stops the session after this much inactivity. Default 600. |
|
|
@@ -61,10 +61,9 @@ Give exactly one of `appId`, `buildId`, `artifactUrl`, or `bundleId`.
|
|
|
61
61
|
| Option | Meaning |
|
|
62
62
|
| --- | --- |
|
|
63
63
|
| `platform: 'mac'` | A cloud macOS computer, without `app` or `device` configuration |
|
|
64
|
-
| `platform: 'linux'` | A cloud Linux computer with the same SDK methods as Mac, without `app` or `device` configuration |
|
|
65
64
|
| `startTimeout` | Milliseconds to wait for the computer to become active; default 600000, range 10000..1800000 |
|
|
66
65
|
|
|
67
|
-
Computers do not accept `idleTimeoutSeconds`, image, provider, cwd, or lifetime selectors. The server bounds a run to
|
|
66
|
+
Computers do not accept `idleTimeoutSeconds`, image, provider, cwd, or lifetime selectors. The server bounds a run to its lifetime from create (two hours today); per-operation timeout defaults to 30000 ms and accepts 1..300000 ms. See [computers.md](computers.md).
|
|
68
67
|
|
|
69
68
|
## Environment variables
|
|
70
69
|
|
|
@@ -79,15 +78,20 @@ Hosted children use `computer.open(target)` rather than named config targets. Un
|
|
|
79
78
|
Use the explicitly scoped package command below. Installing `@revyl/verify` provides the `verify` and `revyl-verify` bins. The separate `revyl-verify` npm alias also provides `npx revyl-verify` with the same behavior; bare `npx verify` can fetch an unrelated package. Scripts run with `node`, not through the CLI.
|
|
80
79
|
|
|
81
80
|
```text
|
|
82
|
-
npx @revyl/verify init [--platform
|
|
81
|
+
npx @revyl/verify init [--platform ios|android|web|mac] [--app-id <id>] [--url <https-url>] [--force] [--yes] [--no-install]
|
|
82
|
+
npx @revyl/verify login
|
|
83
|
+
npx @revyl/verify logout
|
|
83
84
|
npx @revyl/verify whoami
|
|
84
85
|
npx @revyl/verify --help
|
|
85
86
|
npx @revyl/verify --version
|
|
86
87
|
```
|
|
87
88
|
|
|
88
|
-
- `init` writes `verify.config.ts` and `verify/example.
|
|
89
|
-
-
|
|
90
|
-
- `
|
|
89
|
+
- `init` writes `verify.config.ts` and `verify/example.mts`, and adds `.verify/` to `.gitignore`. Without `--platform` it asks its questions interactively, defaulting to iOS. It never overwrites files unless you pass `--force`. `init --help` lists its options.
|
|
90
|
+
- iOS and Android without `--app-id` target the device's built-in Settings app by bundle id (`com.apple.Preferences`, `com.android.settings`), so the example runs before you upload a build. Web requires `--app-id`; without it, `init` writes no files and explains how to create the Web app ([web.md](web.md#create-the-web-app)).
|
|
91
|
+
- `npx @revyl/verify init --platform mac` needs no app id and generates a command/file script. `--app-id` is rejected for Mac, and `--url` applies only to web.
|
|
92
|
+
- `init` installs Verify from the project's lockfile when needed and signs in through Revyl's browser approval flow when no credentials are available. Use `--yes` or `--no-install` to control installation. Non-interactive runs do not prompt or open a browser and require `REVYL_API_KEY`.
|
|
93
|
+
- `login` signs in through Revyl's browser approval flow and saves the login to `~/.revyl/credentials.json`, which the Revyl CLI shares. It needs a terminal; in CI, set `REVYL_API_KEY`.
|
|
94
|
+
- `logout` deletes `~/.revyl/credentials.json`. The API key it held keeps working until you delete it in Revyl under Settings → API keys, and a `REVYL_API_KEY` in the environment still applies.
|
|
91
95
|
- `whoami` prints the Revyl account and organization your credentials belong to.
|
|
92
96
|
|
|
93
97
|
Exit codes: `0` success, `1` a Revyl or file error, `2` a usage or authentication problem.
|
package/docs/errors.md
CHANGED
|
@@ -9,7 +9,7 @@
|
|
|
9
9
|
| Web HTTP 409 capacity/availability | `API_ERROR` | `web_runtime_unavailable` |
|
|
10
10
|
| Computer HTTP 429 admission | `COMPUTER_LIMIT` | `computer_limit` |
|
|
11
11
|
| Computer HTTP 503 runtime availability | `NOT_AVAILABLE_ON_PLATFORM` | `runtime_unavailable` |
|
|
12
|
-
| Hosted child capacity | `COMPUTER_LIMIT` | `computer_at_capacity` |
|
|
12
|
+
| Hosted child capacity | `COMPUTER_LIMIT` | `computer_at_capacity` (all children) or `computer_platform_at_capacity` (that platform) |
|
|
13
13
|
| Hosted runtime/capability | `NOT_AVAILABLE_ON_PLATFORM` | `computer_runtime_unavailable` / `computer_cannot_host_platform` / `web_runtime_not_configured` |
|
|
14
14
|
| Hosted parent no longer running | `SESSION_ENDED` | `computer_not_running` |
|
|
15
15
|
| Hosted parent inaccessible | `NOT_FOUND` | `computer_not_found` |
|
|
@@ -37,15 +37,15 @@ try {
|
|
|
37
37
|
|
|
38
38
|
| Code | What happened | What to do |
|
|
39
39
|
| --- | --- | --- |
|
|
40
|
-
| `INVALID_ARGUMENT` | A call got a bad value: an unknown target name, an empty step, two steps at once on one session | The message names the argument |
|
|
40
|
+
| `INVALID_ARGUMENT` | A call got a bad value: an unknown target name, an empty step, two steps at once on one session, a point outside the screenshot | The message names the argument |
|
|
41
41
|
| `CONFIG_NOT_FOUND` | `open('name')` found no `verify.config.ts` at or above the current folder | Run `npx @revyl/verify init`, pass `options.config`, or pass a target object |
|
|
42
42
|
| `CONFIG_INVALID` | The config failed validation or did not load | The message names the exact option and the rule |
|
|
43
|
-
| `AUTH_MISSING` | No `REVYL_API_KEY` and no
|
|
43
|
+
| `AUTH_MISSING` | No `REVYL_API_KEY` and no saved login | Run `npx @revyl/verify login` in a terminal, or set `REVYL_API_KEY` |
|
|
44
44
|
| `AUTH_REJECTED` | Revyl rejected the credentials | Create a new API key, or log in again |
|
|
45
45
|
| `EDGE_BLOCKED` | An HTTP 403 has a `text/html` content type and nonempty `cf-ray`, without a valid JSON body | Check network access or contact support; Verify does not retry the request |
|
|
46
|
-
| `API_ERROR` | Revyl returned an unexpected response | Try again;
|
|
46
|
+
| `API_ERROR` | Revyl returned an unexpected response | Try again; after an exact action, check the session first, since the action may have run. If it persists, report it with the session link |
|
|
47
47
|
| `SESSION_START_FAILED` | The browser, device, or computer did not start | Computer `no_host` reports the 120-second unclaimed-start context; `device_limit` reports queued-work/concurrency context. Inspect the session and cleanup before starting again. |
|
|
48
|
-
| `SESSION_ENDED` | The session was closed, stopped, or expired | Close explicitly; computer expiry does not prove release |
|
|
48
|
+
| `SESSION_ENDED` | The session was closed, stopped, or expired, or Revyl answered 404 for the session itself | Close explicitly; computer expiry does not prove release |
|
|
49
49
|
| `SESSION_BUSY` | A step conflict, computer operation limit, or recording requested before terminal state and confirmed release | Inspect the state before another call; a terminal computer alone does not prove release |
|
|
50
50
|
| `COMPUTER_LIMIT` | Computer admission was refused due to queued work (including iOS) or the concurrency limit | Wait for capacity; Verify does not retry or stop a definitely refused start |
|
|
51
51
|
| `NOT_AVAILABLE_ON_PLATFORM` | The call is unavailable: computer agent steps, device shell/file/recording methods, computer access or runtime unavailable, or a backend without web steps | See [web.md](web.md) and [computers.md](computers.md#availability-and-errors) |
|
|
@@ -56,7 +56,9 @@ try {
|
|
|
56
56
|
| `ACTION_UNCERTAIN` | The connection failed mid-request, so the step may or may not have run | Verify does not repeat it; check the session link before deciding what to do |
|
|
57
57
|
| `INSTRUCTION_FAILED` | Revyl's agent could not do what the instruction asked | Make the instruction more specific, or split it into smaller steps |
|
|
58
58
|
| `VALIDATION_FAILED` | The validation did not hold | The message has Revyl's reason; watch the recording at the session link |
|
|
59
|
-
| `STEP_FAILED` | Revyl could not run the step to an outcome: it errored, it was cancelled on Revyl,
|
|
59
|
+
| `STEP_FAILED` | Revyl could not run the step to an outcome: it errored, it was cancelled on Revyl, a validation came back without a verdict, or Revyl accepted the step but still answered "Live step not found" about 10 seconds later (`httpStatus` 404). This says nothing about your app, and `soft` does not cover it. Verify leaves the session open. | Watch the session at the link. After "Live step not found", the step may still be running, so wait for it to finish (another step meanwhile gets `SESSION_BUSY`), then run the step again |
|
|
60
|
+
| `MODEL_ERROR` | Your model's provider refused the request or could not be reached after retries, the model kept answering without calling a tool, or a reply was cut off at the output token cap | The message has the provider's reason, with the key redacted, and the next step: check the key, the model name, image and tool support, or wait out a rate limit. See [troubleshooting your own model](own-model.md#troubleshooting) |
|
|
61
|
+
| `MODEL_KEY_MISSING` | A `'provider/model'` string's key is not in the environment | Set the variable the message names, or pass `{ id, baseURL, apiKey }` |
|
|
60
62
|
| `TIMEOUT` | A step, a session start, or a request ran out of time | Inspect the session before retrying. The default 10-minute start timeout is separate from the 120-second unclaimed-start failure. See [computer availability](computers.md#availability-and-errors). |
|
|
61
63
|
|
|
62
64
|
`EDGE_BLOCKED` does not require Cloudflare branding in the HTML body. A `cf-ray` header alone is not proof of a block: any syntactically valid JSON body retains its existing error mapping, even if the content type is incorrectly `text/html`. HTTP 401 is unchanged. Edge-block errors include `httpStatus` but never copy the response body or ray ID into the error.
|
package/docs/getting-started.md
CHANGED
|
@@ -24,7 +24,7 @@ try {
|
|
|
24
24
|
|
|
25
25
|
Use `appId` when you want to target your own uploaded app.
|
|
26
26
|
|
|
27
|
-
Run it with `node verify/check.mts`.
|
|
27
|
+
Run it with `node verify/check.mts`. For a Web target, create a Web app in Revyl first (see [web.md](web.md#create-the-web-app)), then use `open({ platform: 'web', app: { appId, url } })`; for a computer, use `open({ platform: 'mac' })`.
|
|
28
28
|
|
|
29
29
|
On the first interactive run, Verify opens the Revyl browser approval flow once and saves the login in `~/.revyl/credentials.json` for the CLI and SDK to share. It prints a live session link, then ordered step results:
|
|
30
30
|
|
|
@@ -42,24 +42,25 @@ The URL is live during the run and keeps the report and available recording afte
|
|
|
42
42
|
|
|
43
43
|

|
|
44
44
|
|
|
45
|
-
In CI, Verify never prompts or opens a browser; set `REVYL_API_KEY` as a secret. Python
|
|
45
|
+
In CI, Verify never prompts or opens a browser; set `REVYL_API_KEY` as a secret. The Python SDK signs in the same way: its first interactive `open()`, `attach()` or `current_computer()` call without credentials opens the browser approval flow, unless stdin or stderr is not a terminal or `CI` is set. The Go SDK never opens a browser; it uses `REVYL_API_KEY` or a login saved by TypeScript, Python, `npx @revyl/verify login` or the Revyl CLI.
|
|
46
46
|
|
|
47
47
|
## Scaffold a config (optional)
|
|
48
48
|
|
|
49
|
-
Use `init` when you want named targets in `verify.config.ts` and a generated example script. It is optional; installing the package is enough to use inline targets. It detects your package manager from the lockfile, asks before adding `@revyl/verify` as a dev dependency, asks what you want to open, adds `.verify/` to `.gitignore`, and prints the command that runs the example:
|
|
49
|
+
Use `init` when you want named targets in `verify.config.ts` and a generated example script. It is optional; installing the package is enough to use inline targets. It detects your package manager from the lockfile, asks before adding `@revyl/verify` as a dev dependency, asks what you want to open (iOS by default), adds `.verify/` to `.gitignore`, signs you in through the browser when no credentials are saved, and prints the command that runs the example:
|
|
50
50
|
|
|
51
51
|
- `verify.config.ts`: the targets `open()` can start, by name
|
|
52
|
-
- `verify/example.
|
|
52
|
+
- `verify/example.mts`: a script that opens the target, runs steps and takes a screenshot (web/mobile) or checks commands and files (Mac), and closes
|
|
53
53
|
|
|
54
54
|
In scripts and CI, pass the answers as flags. Non-interactive runs never prompt or open a browser; provide `REVYL_API_KEY` as a secret:
|
|
55
55
|
|
|
56
56
|
```bash
|
|
57
|
-
npx @revyl/verify init --platform
|
|
58
|
-
npx @revyl/verify init --platform
|
|
57
|
+
npx @revyl/verify init --platform ios
|
|
58
|
+
npx @revyl/verify init --platform android --app-id <revyl-app-id>
|
|
59
|
+
npx @revyl/verify init --platform web --app-id <revyl-web-app-id> --url https://example.com
|
|
59
60
|
npx @revyl/verify init --platform mac
|
|
60
61
|
```
|
|
61
62
|
|
|
62
|
-
Use `--yes` to accept installation without a prompt, or `--no-install` when Verify is already installed.
|
|
63
|
+
Use `--yes` to accept installation without a prompt, or `--no-install` when Verify is already installed. Without an app id, an iOS or Android target opens the device's built-in Settings app (`com.apple.Preferences` or `com.android.settings`) by bundle id, so the generated example runs on a brand-new account before you upload anything. A Web target needs a Web app id; without one, `init` writes no files and tells you how to create the app. A Mac target needs no app id. The generated config is ordinary TypeScript, so edit it when the example needs your own app, a pinned build, a public Web URL, or a different target.
|
|
63
64
|
|
|
64
65
|
## Manual setup
|
|
65
66
|
|
|
@@ -67,38 +68,41 @@ Use manual setup when you need to control the package manager or keep the genera
|
|
|
67
68
|
|
|
68
69
|
1. A usable login in `~/.revyl/credentials.json` marked `local_auth_override` takes precedence, even when `REVYL_API_KEY` is set.
|
|
69
70
|
2. Otherwise, `REVYL_API_KEY`, an API key you create in Revyl's settings.
|
|
70
|
-
3. Otherwise, a usable login
|
|
71
|
+
3. Otherwise, a usable login in `~/.revyl/credentials.json`, saved by Verify's browser sign-in or the Revyl CLI. Expired access tokens fall back to a stored API key when present.
|
|
71
72
|
|
|
72
73
|
```bash
|
|
73
|
-
|
|
74
|
+
npx @revyl/verify login # signs in through your browser; or: export REVYL_API_KEY=...
|
|
74
75
|
npx @revyl/verify whoami # checks the credentials without starting a session
|
|
76
|
+
npx @revyl/verify logout # removes the saved login from this computer
|
|
75
77
|
```
|
|
76
78
|
|
|
79
|
+
`logout` deletes `~/.revyl/credentials.json`, which the Revyl CLI shares. The API key that login held keeps working until you delete it in Revyl under Settings → API keys.
|
|
80
|
+
|
|
77
81
|
In CI, keep the key in a secret manager rather than in source, command arguments, or logs.
|
|
78
82
|
|
|
79
83
|
## Point the config at your app
|
|
80
84
|
|
|
81
|
-
|
|
85
|
+
The generated iOS and Android targets open Settings. To test your own app, replace `bundleId` in `verify.config.ts` with `appId: '<your Revyl app id>'`, copied from the app's page in Revyl, or with the `bundleId` of an app already on the device. Mac targets need no app id.
|
|
82
86
|
|
|
83
|
-
- **Web:** set `app.url` to a public HTTPS URL. Revyl's cloud browser cannot reach `localhost`.
|
|
87
|
+
- **Web:** create a Web app in Revyl (see [web.md](web.md#create-the-web-app)) and set `app.url` to a public HTTPS URL. Revyl's cloud browser cannot reach `localhost`.
|
|
84
88
|
- **iOS and Android:** upload a build to the app in Revyl, for example with `revyl build upload`; each session installs the newest one. See [mobile.md](mobile.md).
|
|
85
|
-
- **Mac
|
|
89
|
+
- **Mac:** computers are early access; ask support@revyl.ai about access. Agent instruction/validation are unavailable. See [computers.md](computers.md#availability-and-errors).
|
|
86
90
|
|
|
87
91
|
## Run the script
|
|
88
92
|
|
|
89
93
|
```bash
|
|
90
|
-
node verify/example.
|
|
94
|
+
node verify/example.mts
|
|
91
95
|
```
|
|
92
96
|
|
|
93
|
-
|
|
97
|
+
The `.mts` extension makes Node run the script as an ES module whatever your `package.json` says about `"type"`. In a Yarn Plug'n'Play project, the default since Yarn 2, run it with `yarn node verify/example.mts`.
|
|
94
98
|
|
|
95
99
|
Illustrative output (durations vary):
|
|
96
100
|
|
|
97
101
|
```text
|
|
98
102
|
[verify] ios: starting ios session https://app.revyl.ai/sessions/…
|
|
99
103
|
[verify] ios: ready
|
|
100
|
-
[verify] ios: ✓ instruction "
|
|
101
|
-
[verify] ios: ✓ validation "The
|
|
104
|
+
[verify] ios: ✓ instruction "Open General, then About" (12.0s)
|
|
105
|
+
[verify] ios: ✓ validation "The screen shows the operating system version" (9.8s)
|
|
102
106
|
[verify] ios: closed
|
|
103
107
|
```
|
|
104
108
|
|
|
@@ -115,10 +119,11 @@ const session = await open('ios'); // a target
|
|
|
115
119
|
try {
|
|
116
120
|
await session.instruction('Add the first product to the cart'); // Revyl's agent does it
|
|
117
121
|
await session.validation('The cart badge shows 1'); // throws if it does not hold
|
|
122
|
+
await session.tap({ target: 'Cart icon' }); // or act yourself, no agent
|
|
118
123
|
await session.screenshot({ path: '.verify/cart.png' });
|
|
119
124
|
} finally {
|
|
120
125
|
await session.close();
|
|
121
126
|
}
|
|
122
127
|
```
|
|
123
128
|
|
|
124
|
-
A failed instruction or validation throws a `VerifyError`. An uncaught error in a plain script exits non-zero. Automatic cleanup is best effort, bounded to 15 seconds, and skips signal/error handlers that already have listeners. Always close in `try/finally`; a successful web/mobile close acknowledges the stop request, not confirmed physical release. Computer close requires confirmed terminal release. See [api.md](api.md) for cleanup details and every option, and [ci.md](ci.md) for calling Verify from a test runner.
|
|
129
|
+
Exact actions such as `tap`, `type`, `swipe` and `pressKey` (TypeScript 0.4.0, Python 0.2.0 and Go v0.3.0) act on the screen directly, by a description Revyl locates or by coordinates; see the [web](web.md#exact-actions) and [mobile](mobile.md#exact-actions) guides. A failed instruction or validation throws a `VerifyError`. An uncaught error in a plain script exits non-zero. Automatic cleanup is best effort, bounded to 15 seconds, and skips signal/error handlers that already have listeners. Always close in `try/finally`; a successful web/mobile close acknowledges the stop request, not confirmed physical release. Computer close requires confirmed terminal release. See [api.md](api.md) for cleanup details and every option, and [ci.md](ci.md) for calling Verify from a test runner.
|
package/docs/llms.txt
CHANGED
|
@@ -1,18 +1,23 @@
|
|
|
1
1
|
# Verify
|
|
2
2
|
|
|
3
|
-
> Verify (@revyl/verify) is an open-source TypeScript library for Revyl. It opens cloud browsers and iOS/Android devices for agent instruction/validation steps, or Mac
|
|
3
|
+
> Verify (@revyl/verify) is an open-source TypeScript library for Revyl. It opens cloud browsers and iOS/Android devices for agent instruction/validation steps, or Mac computers for commands, files, processes, screenshots, and optional private recording metadata after release. Call it from a plain script, coding agent, CI, or any test runner. It has no test runner of its own.
|
|
4
4
|
|
|
5
5
|
## Docs
|
|
6
6
|
|
|
7
|
-
|
|
7
|
+
Mac computers are early access; ask support@revyl.ai about access. See computers.md#availability-and-errors.
|
|
8
8
|
|
|
9
9
|
- [Getting started](getting-started.md): install, authenticate, init, run a script
|
|
10
10
|
- [API](api.md): open(), attach(), hosted children, navigate(), report(), agent steps, screenshot(), exec(), file/process methods, close()
|
|
11
11
|
- [Configuration](configuration.md): verify.config.ts targets, environment variables, the init and whoami commands
|
|
12
|
-
- [Web](web.md): Revyl web sessions, URLs and builds, sign-in, and
|
|
13
|
-
- [Mobile](mobile.md): iOS and Android devices and
|
|
14
|
-
- [Computers](computers.md#availability-and-errors): Mac
|
|
12
|
+
- [Web](web.md): Revyl web sessions, URLs and builds, sign-in, steps, and exact actions
|
|
13
|
+
- [Mobile](mobile.md): iOS and Android devices, installing apps, and exact actions
|
|
14
|
+
- [Computers](computers.md#availability-and-errors): Mac early access, expiry, capacity errors, recording metadata, and confirmed release
|
|
15
15
|
- [Verify working end to end](end-to-end.md): live verification scope, observed outcomes, and limits of the evidence
|
|
16
16
|
- [CI and test runners](ci.md): GitHub Actions, node:test, Vitest, Jest, Playwright Test
|
|
17
|
+
- [Use your own model](own-model.md): run instructions and validations on Claude, GPT, Gemini, a local model or any OpenAI-compatible endpoint; providers, limits, privacy and troubleshooting
|
|
17
18
|
- [Errors](errors.md): every error code and what to do about it
|
|
18
|
-
- [For coding agents](agents.md): the workflow and rules for agents using Verify
|
|
19
|
+
- [For coding agents](agents.md): the workflow and rules for agents using Verify, and the coordinate rule for exact actions
|
|
20
|
+
|
|
21
|
+
## Exact actions and coordinates
|
|
22
|
+
|
|
23
|
+
Prefer a description, such as `tap({ target: 'Sign in button' })`: Revyl finds the element on the current screen, and you never handle coordinates. Otherwise a point is `{ x, y }` in screenshot pixels on every platform: whole numbers from the top-left corner of the image `screenshot()` returns. So a position you read off a screenshot goes to `tap()`, `type()` and `swipe()` unchanged. `screenSize()` returns the screenshot's width and height, and `locate()` and the points `tap()` and `type()` return use the same pixels. A point outside the screenshot is refused with `INVALID_ARGUMENT` before any action is sent, and the message names the valid range. On iOS, Verify converts the pixels to the screen's points for you. On Web they equal CSS pixels at the default device scale factor of 1, and you still pass screenshot pixels at any other. This holds from TypeScript 0.4.1, Python 0.2.1 and Go v0.3.1; earlier versions took iOS points. See [For coding agents](agents.md#exact-actions-and-coordinates) for a runnable example.
|
package/docs/mobile.md
CHANGED
|
@@ -47,3 +47,51 @@ After connection and any explicit bundle launch, `open()` makes a best-effort fi
|
|
|
47
47
|
The generated mobile example starts with an instruction to wait for the first screen before validating it. For your own scripts, make the first instruction describe the screen or action you actually need; a screenshot with content can still show a loading screen or another app.
|
|
48
48
|
|
|
49
49
|
For Android Settings, instruct the agent to scroll or navigate until the OS version is visible before validating it. Opening About alone can leave the version offscreen.
|
|
50
|
+
|
|
51
|
+
## Exact actions
|
|
52
|
+
|
|
53
|
+
**Since 0.4.0. Early access.** In TypeScript (`@revyl/verify` 0.4.0), Python (`revyl-verify` 0.2.0) and Go (v0.3.0); the [Python](https://github.com/RevylAI/verify/blob/v0.4.1/python/README.md#exact-actions) and [Go](https://github.com/RevylAI/verify/blob/v0.4.1/go/README.md#exact-actions) READMEs show them with each language's names. When you know exactly what to press, skip the agent. Exact actions run on the device as given, with no model, and mix freely with `instruction()` and `validation()` in one script:
|
|
54
|
+
|
|
55
|
+
```ts
|
|
56
|
+
import { open } from '@revyl/verify';
|
|
57
|
+
|
|
58
|
+
const session = await open({ platform: 'ios', app: { bundleId: 'com.apple.Preferences' } });
|
|
59
|
+
try {
|
|
60
|
+
await session.tap({ target: 'General row' }); // Revyl finds the element you describe
|
|
61
|
+
await session.tap({ target: 'About row' });
|
|
62
|
+
await session.validation('The screen shows the operating system version');
|
|
63
|
+
await session.tap({ target: 'Back button' }); // iOS has no back(), so tap the button
|
|
64
|
+
await session.tap({ target: 'Back button' });
|
|
65
|
+
|
|
66
|
+
await session.swipe('down'); // the finger moves down, so the list scrolls to the top
|
|
67
|
+
const search = await session.locate('Search field'); // a point in screenshot pixels
|
|
68
|
+
await session.tap(search); // a tap by coordinates
|
|
69
|
+
await session.type(search, 'Wallpaper');
|
|
70
|
+
await session.pressKey('enter');
|
|
71
|
+
await session.instruction('Open the Wallpaper result'); // hand the fuzzy part to the agent
|
|
72
|
+
await session.validation('The Wallpaper settings are shown');
|
|
73
|
+
} finally {
|
|
74
|
+
await session.close();
|
|
75
|
+
}
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
A location is `{ target: 'a description' }`, which Revyl's grounding model finds on the current screen, or `{ x, y }` in screenshot pixels from the top-left corner; `screenSize()` gives the bounds. `type()` taps the field and clears it first unless you pass `{ clear: false }`. `pressKey()` takes `'enter'` or `'backspace'`.
|
|
79
|
+
|
|
80
|
+
On Android, `back()` presses the system back button. On both platforms, `launchApp(bundleId)` brings an installed app to the front and `openUrl('myapp://orders/42')` opens a deep link. A failed action throws `ACTION_FAILED`. One whose outcome Revyl can't confirm throws `ACTION_UNCERTAIN` or `TIMEOUT`, so check the session before acting again. Actions are never repeated. If Verify can't reach Revyl, the action fails with `API_ERROR` and isn't resent either, so check the session before trying it again. [api.md](api.md#exact-actions) lists every call.
|
|
81
|
+
|
|
82
|
+
| Call | Web | iOS | Android | Mac computer |
|
|
83
|
+
| --- | --- | --- | --- | --- |
|
|
84
|
+
| `tap`, `type`, `swipe`, `pressKey`, `locate`, `screenSize` | Yes | Yes | Yes | No, use `exec()` and files |
|
|
85
|
+
| `back()` | Yes, browser history | No, tap the on-screen back control | Yes, the system back button | No |
|
|
86
|
+
| `openUrl(url)` | Yes, loads an http(s) URL in the tab | Yes, a URL or deep link | Yes, a URL or deep link | No |
|
|
87
|
+
| `launchApp(bundleId)` | No, use `openUrl` or `navigate` | Yes | Yes | No |
|
|
88
|
+
| `navigate(url)` | Yes | No, use `openUrl` | No, use `openUrl` | No |
|
|
89
|
+
| `instruction`, `validation`, your own model | Yes | Yes | Yes | No, use `exec()` |
|
|
90
|
+
| `screenshot()` | Yes | Yes | Yes | Yes |
|
|
91
|
+
| `exec`, files, processes | No, open a Mac computer | No, open a Mac computer | No, open a Mac computer | Yes |
|
|
92
|
+
|
|
93
|
+
**Coordinates.** Prefer a description, such as `tap({ target: 'Sign in button' })`: Revyl finds the element on the current screen, and you never handle coordinates. Otherwise a point is `{ x, y }` in screenshot pixels on every platform: whole numbers from the top-left corner of the image `screenshot()` returns. So a position you read off a screenshot goes to `tap()`, `type()` and `swipe()` unchanged. `screenSize()` returns the screenshot's width and height, and `locate()` and the points `tap()` and `type()` return use the same pixels. A point outside the screenshot is refused with `INVALID_ARGUMENT` before any action is sent, and the message names the valid range. On iOS, Verify converts the pixels to the screen's points for you. On Web they equal CSS pixels at the default device scale factor of 1, and you still pass screenshot pixels at any other. This holds from TypeScript 0.4.1, Python 0.2.1 and Go v0.3.1; earlier versions took iOS points.
|
|
94
|
+
|
|
95
|
+
A call a platform doesn't support throws `NOT_AVAILABLE_ON_PLATFORM`, and its message names the alternative.
|
|
96
|
+
|
|
97
|
+
When you pass a [`model`](own-model.md), your model uses `tap`, `type`, `swipe`, `pressKey`, `openUrl` and, where the platform has it, `back` inside `instruction()`.
|