@revyl/verify 0.1.0 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,15 +1,60 @@
1
1
  # Changelog
2
2
 
3
+ ## 0.3.0 (2026-10-31)
4
+
5
+ - Sign in automatically on the first interactive `open()`, `attach()`, or `currentComputer()` call using the Revyl CLI browser-approval flow; CI remains key-only and browser-free.
6
+ - Make `npx @revyl/verify init` install Verify, scaffold a project, sign in through the Revyl CLI browser approval flow when needed, and print the example command. Non-interactive runs use `REVYL_API_KEY` without prompting or opening a browser.
7
+ - Add `currentComputer()`, Python `current_computer()` / `async_current_computer()`, and Go `CurrentComputer` to attach non-owning to `REVYL_COMPUTER_ID`, with explicit ownership opt-in and a typed error when no current computer is configured.
8
+ - Add explicit non-owning computer attachment with `stopOnClose: false`, Python `stop_on_close=False`, and Go `OpenOptions.StopOnClose`. Closing these attachments still cleans up hosted children but leaves the computer running for its owner to release; the default remains owning.
9
+
10
+ - Removed environment overrides.
11
+
12
+ - **Breaking wire change:** encode computer-operation arguments as `argv_b64` and file paths as `path_b64` using padded URL-safe base64 of UTF-8, without changing public APIs, argument boundaries, or existing stdin/data fields. Verify 0.3.0 requires the encoded-only backend, which is already live on revyl.ai; no plaintext fallback or automatic replay is provided.
13
+
14
+ - Classify HTTP 403 responses with a `text/html` content type and nonempty `cf-ray` as `EDGE_BLOCKED` across TypeScript, Go and Python, unless the body is syntactically valid JSON. Preserve origin authentication errors, omit response content from edge-block messages, and do not retry blocked requests.
15
+
16
+ - Map hosted Web navigation `request_timeout` and `protocol_error` responses to `ACTION_UNCERTAIN` with safe reasons and HTTP status. Timeout messages explain that navigation may still commit and the session remains open; Verify never replays it.
17
+
18
+ - Treat ordinary Web `409 web_runtime_unavailable` admission refusals as definite pre-dispatch failures: preserve `API_ERROR` plus safe reason/status details and do not send a stop request. Allocated or otherwise uncertain Web failures retain cleanup.
19
+
20
+ - Bound parent computer cleanup's child phase to five seconds so slow child stops cannot prevent requesting parent release. Recognize `web_runtime_not_configured` as a definite hosted refusal, reject reports without a known session ID locally, preserve safe TypeScript startup diagnostics, and keep the original TypeScript `Session` interface compatible through the extended `OpenedSession` interface.
21
+
22
+ - Add public computer attachment, computer-hosted child sessions and deterministic Web navigation in TypeScript, Go and Python. Child sessions reuse the parent's authenticated client and expire with it; parent close attempts bounded child cleanup before requesting computer release. Initial hosted Web builds remain HTTPS; localhost is reached by navigation after start, subject to runtime availability.
23
+ - Preserve typed server refusal reasons and HTTP status alongside existing error codes, including Web `web_runtime_unavailable`, computer `computer_limit` and `runtime_unavailable`, without replaying starts or navigation.
24
+
25
+ - Preserve Python startup session identity, viewer links and separate cleanup outcomes on typed errors; add optional lifecycle logging to sync/async opening. Clean up identifiable malformed or uncertain starts, interrupt cancelled async startup, validate device stop acknowledgments, and preserve original failures when cleanup also fails.
26
+
27
+ - Prepare Python SDK 0.1.0 (`revyl-verify`, import `revyl_verify`) for Python ≥3.10, with typed sync and thread-adapted async sessions, seconds-based timeouts, web/mobile steps and early-access computer operations. Include fake-server coverage, wheel/sdist checks and an isolated installed-wheel smoke; Python has not been published or verified live. Its independent `py/v*` release workflow validates without publishing on dispatch, skips exact published versions, and requires administrator-enabled, protected-environment PyPI trusted publishing for new tag releases.
28
+
29
+ - Add the source-free `revyl-verify` npm alias with matching API, TypeScript types and CLI, pinned to the exact `@revyl/verify` version. Validate and pack both packages; optional CI publication checks each version independently and publishes the scoped package before the alias.
30
+ - Add a standard-library-only Go SDK for web, iOS, Android, and early-access Mac and Linux computers. Computer commands, binary files, process handles, screenshots, recording metadata, and confirmed-release cleanup require computer access through support@revyl.ai. Capacity refusals have a distinct `COMPUTER_LIMIT` code; no start or operation is automatically replayed.
31
+
32
+ ## 0.2.0
33
+
34
+ Prepared for release; see npm for publication status.
35
+
36
+ - Enable Linux named and inline computer targets with the shared Mac command, file, process, screenshot, and confirmed-release cleanup surface. Both operating systems remain early access; contact support@revyl.ai for access.
37
+ - Add `COMPUTER_LIMIT` for queued work (including iOS) or concurrency admission refusals, without retrying or stopping a definitely refused start. Preserve `SESSION_BUSY` for operation limits.
38
+ - Explain the 120-second unclaimed-start failure without shortening the default 10-minute start timeout.
39
+ - Expose optional `expiresAt` and bound operation/screenshot observation by the hard 60-minute lifetime from create, while keeping explicit close possible.
40
+ - Add computer `recording()` metadata retrieval after terminal state and confirmed release, including after close. Missing recordings return null; signed URLs remain private, with no automatic waiting or download.
41
+ - Keep Linux CLI init intentionally unavailable; use inline targets or config entries. Computer instruction/validation remain locally unavailable.
42
+ - Observe mobile first-screen content best-effort for up to 10 seconds after connection/launch, without pretending the runtime provides rendered-readiness guarantees. Generated mobile examples begin with an instruction to wait for the first screen before validation.
43
+ - Retry structured computer screen-not-ready responses with bounded backoff during the fixed startup window; do not replay mutations or unrelated failures.
44
+ - Skip CI publication when the exact version is already on npm, require explicit administrator opt-in for new CI publications, fail closed on registry lookup errors, and document manual Mac publication, optional trusted publishing, and independent Go module tags.
45
+
46
+ Regression validation uses fake API responses. The [end-to-end record](https://github.com/RevylAI/verify/blob/v0.2.0/packages/verify/docs/end-to-end.md) distinguishes baseline live runs from startup fixes still awaiting live re-verification.
47
+
3
48
  ## 0.1.0
4
49
 
5
- Initial release preparation; this entry does not mean the package has been published.
50
+ Published manually from a maintainer's Mac without provenance.
6
51
 
7
52
  - Open Revyl cloud web browsers, iOS simulators, and Android emulators from TypeScript.
8
53
  - Run natural-language instruction and validation steps, with typed failures and optional soft validation results.
9
54
  - Save screenshots, share session links, and close sessions explicitly or during best-effort process cleanup.
10
55
  - Configure named targets and initialize a script with the `verify` or `revyl-verify` command.
11
56
  - Use environment credentials or an existing Revyl CLI login, with documentation included in the package.
12
- - Include **early-access Mac computers, not yet available in Revyl production**, for commands, binary files, process handles, screenshots, and confirmed-release cleanup. Mac works only on Revyl staging for enabled organizations through the optional `REVYL_BACKEND_URL` override and matching credentials; ask support@revyl.ai about access.
57
+ - Include **early-access Mac computers** for commands, binary files, process handles, screenshots, and confirmed-release cleanup. Ask support@revyl.ai about access.
13
58
  - Reserve Linux target types and document the future contract; opening Linux is not available in this release.
14
59
 
15
- Requires Node.js 22.18 or later and a Revyl account. There is no bundled test runner or step replay. Mac operations remain covered by source-aligned fake-API tests; a production start never reached active, so it did not prove any operation or confirmed cleanup.
60
+ Requires Node.js 22.18 or later and a Revyl account. There is no bundled test runner or step replay. Mac operations remain covered by source-aligned fake-API tests; the attempted live start never reached active, so it did not prove any operation or confirmed cleanup.
package/README.md CHANGED
@@ -1,101 +1,168 @@
1
1
  # Verify
2
2
 
3
- **Open cloud browsers and mobile devices for Revyl's agent, or a Mac computer for commands and files, from TypeScript.**
3
+ Verify runs end-to-end checks in a real browser, iOS simulator, or Android emulator on Revyl. You describe what the agent should do and what must be true afterward, instead of maintaining selectors and coordinates that break when a screen changes. Use it from a script, a coding agent, or the test runner you already have. Mac and Linux computers are also available for commands, files, and screenshots, not natural-language steps.
4
+
5
+ Every run gives you a link to watch the target. The session keeps its report and available recording, and the SDK keeps instruction and validation results in order. A failed check leaves evidence you can inspect.
6
+
7
+ ## Install and run
8
+
9
+ ```bash
10
+ npm install @revyl/verify
11
+ ```
12
+
13
+ Save this as `verify/check.ts`:
4
14
 
5
15
  ```ts
6
16
  import { open } from '@revyl/verify';
7
17
 
8
- const session = await open('ios');
9
- await session.instruction('Add the first product to the cart and open the cart');
10
- await session.validation('The cart lists one item with a price');
11
- await session.close();
18
+ const session = await open({ platform: 'ios', app: { appId: 'YOUR_REVYL_APP_ID' } });
19
+ try {
20
+ await session.instruction('Open General, then About');
21
+ await session.validation('The screen shows the operating system version');
22
+ await session.screenshot({ path: '.verify/about.png' });
23
+ } finally {
24
+ await session.close();
25
+ }
12
26
  ```
13
27
 
28
+ Run the script:
29
+
14
30
  ```bash
15
- npx @revyl/verify init # writes verify.config.ts and verify/example.ts
16
- node verify/example.ts # opens the session, prints a link to watch it live, runs the steps
31
+ node verify/check.ts
32
+ ```
33
+
34
+ On the first interactive run, Verify opens the Revyl browser approval flow once and saves the login in `~/.revyl/credentials.json` for the CLI and SDK to share.
35
+
36
+ Verify prints a live session URL and progress like this:
37
+
38
+ ```text
39
+ [verify] ios: starting ios session https://app.revyl.ai/sessions/…
40
+ [verify] ios: ready
41
+ [verify] ios: ✓ instruction "Open General, then About" (12.0s)
42
+ [verify] ios: ✓ validation "The screen shows the operating system version" (9.8s)
43
+ [verify] ios: closed
17
44
  ```
18
45
 
19
- Verify is a small TypeScript library on top of [Revyl](https://revyl.com). Open a browser or mobile device for **instruction** and **validation** steps, or a Mac computer for commands, files, processes, and screenshots. It has no test framework of its own, so you call it from wherever you already write code:
46
+ The URL is live during the run and keeps the report and available recording after it closes. A failed instruction or validation throws a typed error and includes the same URL. In CI, Verify never opens a browser: provide `REVYL_API_KEY` as a secret.
20
47
 
21
- - **A plain script.** A failed validation throws, the script exits non-zero, and Verify closes the session on the way out.
22
- - **A coding agent.** An agent can check its own change on a real device in five lines, then read a one-line outcome per step and a link to the recording.
23
- - **CI.** Run the script in a job. The browsers and devices run in Revyl's cloud, so the runner needs nothing but Node and an API key.
24
- - **Your test runner.** Call it inside node:test, Vitest, Jest, or Playwright Test, next to the tests you already have.
48
+ For Web, use `open({ platform: 'web', app: { appId, url } })`; for a computer, use `open({ platform: 'mac' })`.
25
49
 
26
- ## Quick start
50
+ ## Scaffold a config (optional)
27
51
 
28
- You need Node.js 22.18 or later and a Revyl account.
52
+ Use `init` when you want named targets in `verify.config.ts` and a generated `verify/example.ts` script:
29
53
 
30
54
  ```bash
31
55
  npx @revyl/verify init
32
- npm install --save-dev @revyl/verify
33
- export REVYL_API_KEY=... # or sign in with the Revyl CLI: revyl auth login
34
- node verify/example.ts
35
56
  ```
36
57
 
37
- Full walkthrough: [getting started](https://github.com/RevylAI/verify/blob/main/packages/verify/docs/getting-started.md).
58
+ `init` detects your package manager, asks before adding `@revyl/verify` as a dev dependency, creates `verify.config.ts` and `verify/example.ts`, and prints the command that runs the example. Use `--yes` to accept installation without a prompt, or `--no-install` when Verify is already installed. It is optional; installing the package is enough to use inline targets.
38
59
 
39
- ## The API
60
+ The generated project starts with a named target. Use `--platform web`, `--platform ios`, `--platform android`, or `--platform mac` when you want to skip the platform question. Web and mobile targets need a Revyl app id; a Mac target does not. The generated config is ordinary TypeScript, so edit it when the example needs a pinned build, a public Web URL, or a different target.
40
61
 
41
- | Call | What it does |
42
- | --- | --- |
43
- | `open(target)` | Starts a Revyl cloud browser, device, or Mac computer, from a target name in `verify.config.ts` or a target object, and waits until it is ready |
44
- | `session.exec(argv, options?)` | Runs structured argv on a Mac computer; returns `{ exitCode, stdout, stderr }` |
45
- | `session.readFile(path)`, `writeFile(path, data)`, `stat(path)` | Reads a `Buffer`, writes a string or bytes, or returns `{ kind, size }` on a computer |
46
- | `session.processes()`, `terminate(id)` | Lists run-owned process handles or terminates one on a computer |
47
- | `session.instruction(text)` | Revyl's agent does what the text says. Throws `INSTRUCTION_FAILED` if it cannot. |
48
- | `session.validation(text)` | Revyl's agent checks that the text holds on screen. Throws `VALIDATION_FAILED` if it does not, or returns the result with `{ soft: true }`. |
49
- | `session.screenshot({ path })` | A PNG of the screen |
50
- | `session.viewerUrl` | The session in Revyl: live while it runs, its recording after |
51
- | `session.close()` | Stops the session. Verify also closes open sessions on Ctrl+C, on an uncaught error, and when a script ends without closing. |
62
+ See the [TypeScript getting-started guide](https://github.com/RevylAI/verify/blob/v0.3.0/packages/verify/docs/getting-started.md), the [Python SDK guide](https://github.com/RevylAI/verify/blob/v0.3.0/python/README.md), and the [Go SDK guide](https://github.com/RevylAI/verify/tree/go/v0.2.0/go).
52
63
 
53
- Revyl failures are `VerifyError`s with a stable code, a safe reason, and the session link when one exists.
64
+ ## CI
54
65
 
55
- ## Platforms
66
+ For a normal GitHub Actions job, install from the lockfile with `npm ci` and pass `REVYL_API_KEY` from repository secrets to the script that runs Verify:
56
67
 
57
- | | Web | iOS | Android | Mac | Linux |
58
- | --- | --- | --- | --- | --- | --- |
59
- | Runs on | Revyl cloud browser | Revyl iOS simulator | Revyl Android emulator | Revyl macOS computer | Contract only; open unavailable |
60
- | Opens | A public URL, or a web build in Revyl | A build in Revyl, a build URL, or an installed app | A build in Revyl, a build URL, or an installed app | A workspace, no app id | Nothing in this release |
61
- | `instruction`, `validation` | Yes | Yes | Yes | Unavailable | Unavailable |
62
- | Commands, files, processes | Unavailable | Unavailable | Unavailable | Early access on staging only for enabled organizations; not yet available in Revyl production | Contract only |
63
- | `screenshot`, `viewerUrl` | Yes | Yes | Yes | Yes | Unconfirmed |
68
+ ```yaml
69
+ - uses: actions/checkout@v5
70
+ - uses: actions/setup-node@v5
71
+ with:
72
+ node-version: 24
73
+ cache: npm
74
+ - run: npm ci
75
+ - run: node verify/example.ts
76
+ env:
77
+ REVYL_API_KEY: ${{ secrets.REVYL_API_KEY }}
78
+ ```
64
79
 
65
- Steps on web go through the same Revyl API as on mobile. Production is the default; optional environment overrides are documented in [web.md](https://github.com/RevylAI/verify/blob/main/packages/verify/docs/web.md). A backend without web-step support returns `NOT_AVAILABLE_ON_PLATFORM`.
80
+ On a `revyl-macos` job, `currentComputer()` attaches to the runner's existing computer without taking ownership. The runner releases the computer; closing the Verify attachment still cleans up hosted children. See the [CI guide](https://github.com/RevylAI/verify/blob/v0.3.0/packages/verify/docs/ci.md) and [computer API](https://github.com/RevylAI/verify/blob/v0.3.0/packages/verify/docs/api.md#inside-a-revyl-macos-github-actions-job).
66
81
 
67
- Each step runs Revyl's agent on the live session every time. Verify does not cache or replay steps, and Revyl does not carry values from one step to the next.
82
+ ## Reference authentication
68
83
 
69
- For a Mac workspace, run `npx @revyl/verify init --platform mac`; it generates a command and file-round-trip script with no app id or agent steps. Mac computers are early access and not yet available in Revyl production. They work only on Revyl staging for enabled organizations using the optional `REVYL_BACKEND_URL` override and matching credentials. Linux targets are typed but `open()` rejects them with `NOT_AVAILABLE_ON_PLATFORM` before credentials or network access. See [computers](https://github.com/RevylAI/verify/blob/main/packages/verify/docs/computers.md#availability-and-errors) and the [Linux contract](https://github.com/RevylAI/verify/blob/main/packages/verify/docs/linux-computers-contract.md).
84
+ Use manual setup when you need to control the package manager or keep the generated files out of the project:
70
85
 
71
- ## Docs
86
+ ```bash
87
+ revyl auth login
88
+ npx @revyl/verify whoami
89
+ ```
72
90
 
73
- - [Getting started](https://github.com/RevylAI/verify/blob/main/packages/verify/docs/getting-started.md) and [API](https://github.com/RevylAI/verify/blob/main/packages/verify/docs/api.md)
74
- - [Configuration and CLI](https://github.com/RevylAI/verify/blob/main/packages/verify/docs/configuration.md)
75
- - [Web](https://github.com/RevylAI/verify/blob/main/packages/verify/docs/web.md) and [iOS and Android](https://github.com/RevylAI/verify/blob/main/packages/verify/docs/mobile.md)
76
- - [Mac computers](https://github.com/RevylAI/verify/blob/main/packages/verify/docs/computers.md) and the [deferred Linux contract](https://github.com/RevylAI/verify/blob/main/packages/verify/docs/linux-computers-contract.md)
77
- - [CI and test runners](https://github.com/RevylAI/verify/blob/main/packages/verify/docs/ci.md) and [error codes](https://github.com/RevylAI/verify/blob/main/packages/verify/docs/errors.md)
78
- - [For coding agents](https://github.com/RevylAI/verify/blob/main/packages/verify/docs/agents.md)
91
+ The SDK resolves credentials in this order: a usable stored CLI login marked `local_auth_override`, `REVYL_API_KEY`, then another usable stored CLI login. In CI, use a secret manager rather than putting a key in source, command arguments, or logs. `whoami` checks the credentials without starting a session.
79
92
 
80
- Examples: [web store](https://github.com/RevylAI/verify/tree/main/examples/web-store), [mobile settings](https://github.com/RevylAI/verify/tree/main/examples/mobile-settings), and [Mac workspace](https://github.com/RevylAI/verify/tree/main/examples/mac-workspace).
93
+ ## TypeScript
81
94
 
82
- ## How it works
95
+ ```ts
96
+ import { open } from '@revyl/verify';
83
97
 
84
- Verify is a client for [Revyl's public API](https://docs.revyl.com). `open()` starts a device session (registering a web URL as a build first when needed) and waits until it is ready. Each instruction or validation runs as a live step on the session: Verify submits it, polls its status until Revyl reports a result, and cancels it on Revyl if it runs past its timeout. `close()` stops the session. Requests identify themselves with `X-Revyl-Client: verify` and a `revyl-verify/<version>` user agent.
98
+ const session = await open('ios');
99
+ try {
100
+ await session.instruction('Open General, then About');
101
+ await session.validation('The screen shows the operating system version');
102
+ } finally {
103
+ await session.close();
104
+ }
105
+ ```
85
106
 
86
- For computers, `open()` creates a computer run and waits for its report to become active. Commands use request UUIDs and async operation polling without resubmitting mutations; `close()` stops the run. See [the computer transport](https://github.com/RevylAI/verify/blob/main/packages/verify/docs/computers.md#lifecycle-and-transport).
107
+ For Web, use a public HTTPS URL in the target config. For iOS and Android, point the target at an app or build in Revyl. The [API guide](https://github.com/RevylAI/verify/blob/v0.3.0/packages/verify/docs/api.md) covers target options, step results, screenshots, reports, and cleanup.
87
108
 
88
- ## Develop
109
+ ## Go
89
110
 
90
- ```bash
91
- npm install
92
- npm run build
93
- npm test # unit tests against a mocked Revyl API; no account needed
94
- npm run typecheck
111
+ Inside a function returning `error`, with a bounded `ctx`:
112
+
113
+ ```go
114
+ session, err := verify.Open(ctx, verify.Target{Platform: verify.IOS, BundleID: "com.apple.Preferences"})
115
+ if err != nil {
116
+ return err
117
+ }
118
+ defer func() { _ = session.Close(ctx) }()
119
+ if _, err := session.Instruction(ctx, "Open General, then About"); err != nil {
120
+ return err
121
+ }
122
+ _, err = session.Validation(ctx, "The screen shows the operating system version")
123
+ return err
124
+ ```
125
+
126
+ Go calls return errors instead of throwing. The [Go reference](https://github.com/RevylAI/verify/tree/go/v0.2.0/go) covers context deadlines, screenshots, computer operations, and hosted sessions.
127
+
128
+ ## Python
129
+
130
+ ```python
131
+ from revyl_verify import Target, open
132
+
133
+ with open(Target(platform="ios", bundle_id="com.apple.Preferences")) as session:
134
+ session.instruction("Open General, then About")
135
+ session.validation("The screen shows the operating system version")
95
136
  ```
96
137
 
97
- The library lives in [`packages/verify`](https://github.com/RevylAI/verify/tree/main/packages/verify) and the examples in [`examples/`](https://github.com/RevylAI/verify/tree/main/examples). Read [Contributing](https://github.com/RevylAI/verify/blob/main/CONTRIBUTING.md), [Security](https://github.com/RevylAI/verify/blob/main/SECURITY.md), and [Releasing](https://github.com/RevylAI/verify/blob/main/RELEASING.md) before changing or publishing the package.
138
+ Python supports synchronous and asyncio APIs, but it has no bundled CLI. The [Python reference](https://github.com/RevylAI/verify/blob/v0.3.0/python/README.md) covers both APIs and computer operations.
139
+
140
+ ## Platforms
141
+
142
+ | Platform | Instructions / validations | Commands / files / processes | PNG screenshot |
143
+ | --- | :---: | :---: | :---: |
144
+ | Web | Yes | — | Yes |
145
+ | iOS simulator | Yes | — | Yes |
146
+ | Android emulator | Yes | — | Yes |
147
+ | Mac computer | — | Yes | Yes |
148
+ | Linux computer | — | Yes | Yes |
149
+
150
+ Computer targets are early access and use commands, files, processes, and screenshots rather than natural-language agent steps. They have a server-bounded lifetime. See [computer setup](https://github.com/RevylAI/verify/blob/v0.3.0/packages/verify/docs/computers.md).
151
+
152
+ ## Documentation
153
+
154
+ - [Documentation index](https://github.com/RevylAI/verify/tree/v0.3.0/packages/verify/docs/) · [API](https://github.com/RevylAI/verify/blob/v0.3.0/packages/verify/docs/api.md) · [CI](https://github.com/RevylAI/verify/blob/v0.3.0/packages/verify/docs/ci.md) · [Coding agents](https://github.com/RevylAI/verify/blob/v0.3.0/packages/verify/docs/agents.md)
155
+ - [End-to-end results and limitations](https://github.com/RevylAI/verify/blob/v0.3.0/packages/verify/docs/end-to-end.md)
156
+ - [Examples](https://github.com/RevylAI/verify/tree/v0.3.0/examples/): [Web store](https://github.com/RevylAI/verify/tree/v0.3.0/examples/web-store/) · [Mobile Settings](https://github.com/RevylAI/verify/tree/v0.3.0/examples/mobile-settings/) · [Mac workspace](https://github.com/RevylAI/verify/tree/v0.3.0/examples/mac-workspace/)
157
+ - [Contributing](https://github.com/RevylAI/verify/blob/v0.3.0/CONTRIBUTING.md) · [Security](https://github.com/RevylAI/verify/blob/v0.3.0/SECURITY.md) · [Releasing](https://github.com/RevylAI/verify/blob/v0.3.0/RELEASING.md)
158
+
159
+ ## Troubleshooting
160
+
161
+ - **`NOT_AVAILABLE_ON_PLATFORM`:** the requested method is not available for that target. Computer targets do not run instruction or validation steps; Web and mobile targets do not expose computer commands or files.
162
+ - **`COMPUTER_LIMIT`:** computer admission was refused because of queued work or a concurrency limit. Verify does not retry creation or stop a definitely refused start.
163
+ - **Blank first screen:** mobile startup uses a bounded, best-effort blank-frame observation, not a rendered-readiness guarantee. Begin with an instruction that reaches the screen your workflow needs.
164
+ - **A validation report looks successful but the assertion failed:** inspect each step's `effective_status` and `validation_result`, and compare `validations_passed` with `total_validations` rather than transport counters.
98
165
 
99
166
  ## License
100
167
 
101
- [Apache 2.0](https://github.com/RevylAI/verify/blob/main/LICENSE)
168
+ [Apache 2.0](https://github.com/RevylAI/verify/blob/v0.3.0/LICENSE).
@@ -1,4 +1,4 @@
1
- import { VerifyError } from '../errors.js';
1
+ import { VerifyError, type RefusalReason } from '../errors.js';
2
2
  import type { AppResponse, BuildResponse, DeviceSessionDetail, DeviceSessionStopResponse, StartDeviceRequest, StartDeviceResponse, WhoAmI, WorkerConnection } from './types.js';
3
3
  export declare const DEFAULT_BACKEND_URL = "https://backend.revyl.ai";
4
4
  export declare const DEFAULT_APP_URL = "https://app.revyl.ai";
@@ -34,7 +34,7 @@ export declare class RevylClient {
34
34
  startDevice(request: StartDeviceRequest, signal?: AbortSignal): Promise<StartDeviceResponse>;
35
35
  getDeviceSession(sessionId: string, signal?: AbortSignal): Promise<DeviceSessionDetail>;
36
36
  getWorkerConnection(workflowRunId: string, signal?: AbortSignal): Promise<WorkerConnection>;
37
- stopDeviceSession(sessionId: string): Promise<DeviceSessionStopResponse>;
37
+ stopDeviceSession(sessionId: string, retry?: boolean): Promise<DeviceSessionStopResponse>;
38
38
  cancelDevice(workflowRunId: string): Promise<unknown>;
39
39
  getApp(appId: string, signal?: AbortSignal): Promise<AppResponse>;
40
40
  registerWebUrlBuild(appId: string, request: {
@@ -55,9 +55,11 @@ export declare class RevylClient {
55
55
  }): Promise<RawResponse>;
56
56
  private sendOnce;
57
57
  }
58
- export declare function appUrl(): string;
59
- export declare function sessionViewerUrl(sessionId: string, kind?: 'computer'): string;
58
+ export declare function appUrl(origin?: string): string;
59
+ export declare function sessionViewerUrl(sessionId: string, kind?: 'computer', origin?: string): string;
60
60
  export declare function parseJson<T>(response: RawResponse, what: string): T;
61
61
  export declare function responseDetail(response: RawResponse): string | undefined;
62
62
  export declare function responseErrorCode(response: RawResponse): string | undefined;
63
63
  export declare function apiError(method: string, path: string, response: RawResponse): VerifyError;
64
+ export declare function edgeBlockedError(response: RawResponse): VerifyError | undefined;
65
+ export declare function responseRefusalReason(response: RawResponse): RefusalReason | undefined;
@@ -15,7 +15,7 @@ export class RevylClient {
15
15
  requestTimeoutMs;
16
16
  constructor(options) {
17
17
  this.token = options.token;
18
- this.backendUrl = validateBackendUrl(options.backendUrl ?? process.env.REVYL_BACKEND_URL ?? DEFAULT_BACKEND_URL);
18
+ this.backendUrl = validateBackendUrl(options.backendUrl ?? DEFAULT_BACKEND_URL);
19
19
  this.fetchImpl = options.fetch ?? fetch;
20
20
  this.requestTimeoutMs = options.requestTimeoutMs ?? DEFAULT_REQUEST_TIMEOUT_MS;
21
21
  }
@@ -55,8 +55,8 @@ export class RevylClient {
55
55
  getWorkerConnection(workflowRunId, signal) {
56
56
  return this.json('GET', `/api/v1/execution/streaming/worker-connection/${encodeURIComponent(workflowRunId)}`, { signal });
57
57
  }
58
- stopDeviceSession(sessionId) {
59
- return this.json('POST', `/api/v1/execution/device/sessions/${encodeURIComponent(sessionId)}/stop`, { retry: true });
58
+ stopDeviceSession(sessionId, retry = true) {
59
+ return this.json('POST', `/api/v1/execution/device/sessions/${encodeURIComponent(sessionId)}/stop`, { retry });
60
60
  }
61
61
  cancelDevice(workflowRunId) {
62
62
  return this.json('POST', `/api/v1/execution/device/status/cancel/${encodeURIComponent(workflowRunId)}`, { retry: true });
@@ -127,11 +127,12 @@ export class RevylClient {
127
127
  }
128
128
  }
129
129
  }
130
- export function appUrl() {
131
- return (process.env.REVYL_APP_URL ?? DEFAULT_APP_URL).replace(/\/+$/, '');
130
+ export function appUrl(origin = DEFAULT_APP_URL) {
131
+ validateBackendUrl(origin, 'viewer');
132
+ return origin.replace(/\/+$/, '');
132
133
  }
133
- export function sessionViewerUrl(sessionId, kind) {
134
- return `${appUrl()}/sessions/${encodeURIComponent(sessionId)}${kind ? `?kind=${kind}` : ''}`;
134
+ export function sessionViewerUrl(sessionId, kind, origin) {
135
+ return `${appUrl(origin)}/sessions/${encodeURIComponent(sessionId)}${kind ? `?kind=${kind}` : ''}`;
135
136
  }
136
137
  export function parseJson(response, what) {
137
138
  try {
@@ -175,27 +176,76 @@ export function responseErrorCode(response) {
175
176
  }
176
177
  }
177
178
  export function apiError(method, path, response) {
178
- const detail = responseDetail(response);
179
+ const blocked = edgeBlockedError(response);
180
+ if (blocked)
181
+ return blocked;
182
+ const detail = path === '/api/v1/execution/start_device' ? responseKnownDetail(response, START_ERROR_DETAILS) : responseDetail(response);
179
183
  const suffix = detail ? `: ${detail}` : '';
180
184
  if (response.status === 401 || response.status === 403) {
181
- return new VerifyError('AUTH_REJECTED', `Revyl rejected the request ${method} ${path} (HTTP ${response.status})${suffix}. Check REVYL_API_KEY or run 'revyl auth login'.`);
185
+ return new VerifyError('AUTH_REJECTED', `Revyl rejected the request ${method} ${path} (HTTP ${response.status})${suffix}. Check REVYL_API_KEY or run 'revyl auth login'.`, { reason: responseRefusalReason(response), httpStatus: response.status });
182
186
  }
183
- return new VerifyError('API_ERROR', `Revyl request ${method} ${path} failed with HTTP ${response.status}${suffix}`);
187
+ return new VerifyError('API_ERROR', `Revyl request ${method} ${path} failed with HTTP ${response.status}${suffix}`, { reason: responseRefusalReason(response), httpStatus: response.status });
184
188
  }
185
- function validateBackendUrl(raw) {
189
+ export function edgeBlockedError(response) {
190
+ if (response.status !== 403 || response.headers.get('content-type')?.split(';')[0]?.trim().toLowerCase() !== 'text/html' || !response.headers.get('cf-ray')?.trim())
191
+ return undefined;
192
+ try {
193
+ JSON.parse(new TextDecoder().decode(response.body));
194
+ return undefined;
195
+ }
196
+ catch { }
197
+ return new VerifyError('EDGE_BLOCKED', 'Revyl request was blocked at the network edge (HTTP 403). Check network access or contact support. Verify will not retry this request.', { httpStatus: response.status });
198
+ }
199
+ const REFUSAL_REASONS = new Set([
200
+ 'web_runtime_unavailable', 'web_runtime_not_configured', 'computer_limit', 'runtime_unavailable', 'computer_runtime_unavailable',
201
+ 'computer_at_capacity', 'computer_not_running', 'computer_cannot_host_platform', 'computer_not_found',
202
+ 'web_build_required', 'web_build_not_supported', 'ios_build_required', 'ios_build_not_supported',
203
+ 'android_build_required', 'android_build_not_supported', 'web_session_conflict', 'ios_session_conflict',
204
+ 'android_session_conflict', 'not_available_on_platform', 'request_timeout', 'protocol_error',
205
+ ]);
206
+ const START_ERROR_DETAILS = new Set([
207
+ ...REFUSAL_REASONS,
208
+ 'web_start_actor_invalid', 'web_app_required', 'web_app_not_found', 'web_idle_timeout_invalid',
209
+ 'web_preview_overrides_unsupported', 'web_env_override_invalid', 'web_env_override_secret',
210
+ 'web_sign_in_config_invalid', 'web_sign_in_override_not_secret', 'web_env_overrides_not_supported',
211
+ 'web_build_selection_invalid', 'web_build_not_found', 'web_build_invalid', 'web_billing_not_configured',
212
+ 'web_concurrency_unavailable', 'web_worker_unavailable', 'web_session_fence_conflict',
213
+ 'web_callback_conflict', 'web_start_in_progress', 'web_start_unconfirmed', 'web_session_inputs_unavailable',
214
+ 'sign_in_secret_not_found', 'sign_in_secret_unavailable', 'proof_sign_in_not_configured',
215
+ ]);
216
+ export function responseRefusalReason(response) {
217
+ return responseKnownDetail(response, REFUSAL_REASONS);
218
+ }
219
+ function responseKnownDetail(response, codes) {
220
+ try {
221
+ const parsed = JSON.parse(new TextDecoder().decode(response.body));
222
+ if (!parsed || typeof parsed !== 'object' || Array.isArray(parsed))
223
+ return undefined;
224
+ const record = parsed;
225
+ const values = [record.detail];
226
+ if (record.detail && typeof record.detail === 'object')
227
+ values.push(record.detail.error);
228
+ values.push(record.error);
229
+ return values.find((value) => typeof value === 'string' && codes.has(value));
230
+ }
231
+ catch {
232
+ return undefined;
233
+ }
234
+ }
235
+ function validateBackendUrl(raw, label = 'backend') {
186
236
  let url;
187
237
  try {
188
238
  url = new URL(raw);
189
239
  }
190
240
  catch {
191
- throw new VerifyError('CONFIG_INVALID', 'REVYL_BACKEND_URL is not a valid URL.');
241
+ throw new VerifyError('CONFIG_INVALID', `The ${label} URL is not a valid URL.`);
192
242
  }
193
243
  const loopback = ['localhost', '127.0.0.1', '[::1]'].includes(url.hostname);
194
244
  if (url.protocol !== 'https:' && !(loopback && url.protocol === 'http:')) {
195
- throw new VerifyError('CONFIG_INVALID', 'REVYL_BACKEND_URL must use HTTPS (plain HTTP is allowed only on loopback).');
245
+ throw new VerifyError('CONFIG_INVALID', `The ${label} URL must use HTTPS (plain HTTP is allowed only on loopback).`);
196
246
  }
197
247
  if (url.username || url.password || url.search || url.hash) {
198
- throw new VerifyError('CONFIG_INVALID', 'REVYL_BACKEND_URL must be an origin without credentials, query, or fragment.');
248
+ throw new VerifyError('CONFIG_INVALID', `The ${label} URL must be an origin without credentials, query, or fragment.`);
199
249
  }
200
250
  return url.origin;
201
251
  }
@@ -15,6 +15,7 @@ export interface StartDeviceRequest {
15
15
  platform: MobilePlatform | 'web';
16
16
  is_simulation?: boolean;
17
17
  session_id?: string;
18
+ computer_session_id?: string;
18
19
  app_id?: string;
19
20
  build_id?: string;
20
21
  app_url?: string;
@@ -130,6 +131,12 @@ export interface ComputerReport {
130
131
  state: ComputerState;
131
132
  release: 'pending' | 'confirmed' | 'unknown';
132
133
  failure_reason?: string | null;
134
+ expires_at?: string | null;
135
+ }
136
+ export interface ComputerRecordingResponse {
137
+ url: string;
138
+ duration_seconds?: number | null;
139
+ started_at?: string | null;
133
140
  }
134
141
  export interface ComputerProcessHandle {
135
142
  computer_session_id: string;
package/dist/cli/init.js CHANGED
@@ -8,7 +8,7 @@ const CONFIG_FILE = 'verify.config.ts';
8
8
  const EXAMPLE_SCRIPT = join('verify', 'example.ts');
9
9
  export function validateInitAnswers(answers) {
10
10
  if (!['web', 'ios', 'android', 'mac'].includes(answers.platform)) {
11
- throw new VerifyError('INVALID_ARGUMENT', "--platform must be 'web', 'ios', 'android', or 'mac'. Linux computers are not available yet.");
11
+ throw new VerifyError('INVALID_ARGUMENT', "--platform must be 'web', 'ios', 'android', or 'mac'. Linux init is unavailable; use open({ platform: 'linux' }) or a Linux config target.");
12
12
  }
13
13
  if (answers.platform === 'mac' && answers.appId !== undefined) {
14
14
  throw new VerifyError('INVALID_ARGUMENT', '--app-id does not apply to Mac computers.');
@@ -101,8 +101,7 @@ try {
101
101
  const steps = platform === 'web'
102
102
  ? `await session.instruction('Follow the link on the page');
103
103
  await session.validation('A page about domains reserved for documentation is shown');`
104
- : `// Describe what a person would do and see, for example:
105
- // await session.instruction('Sign in as the demo user and open Settings');
104
+ : `await session.instruction('Wait for the app to finish opening and show its first screen');
106
105
  await session.validation('The app has opened and shows its first screen without an error');`;
107
106
  return `import { open } from '${PACKAGE_NAME}';
108
107
 
@@ -0,0 +1,2 @@
1
+ export declare function packageHasVerify(cwd: string): Promise<boolean>;
2
+ export declare function installVerify(cwd: string): Promise<void>;
@@ -0,0 +1,64 @@
1
+ import { existsSync } from 'node:fs';
2
+ import { readFile, writeFile } from 'node:fs/promises';
3
+ import { spawn } from 'node:child_process';
4
+ import { dirname, join } from 'node:path';
5
+ import { VerifyError } from '../errors.js';
6
+ import { PACKAGE_NAME } from '../version.js';
7
+ export async function packageHasVerify(cwd) {
8
+ let raw;
9
+ try {
10
+ raw = await readFile(join(cwd, 'package.json'), 'utf8');
11
+ }
12
+ catch (error) {
13
+ if (error.code === 'ENOENT')
14
+ return false;
15
+ throw new VerifyError('CONFIG_INVALID', 'Could not read package.json.', { cause: error });
16
+ }
17
+ let manifest;
18
+ try {
19
+ manifest = JSON.parse(raw);
20
+ }
21
+ catch {
22
+ throw new VerifyError('CONFIG_INVALID', 'package.json is not valid JSON.');
23
+ }
24
+ if (!manifest || typeof manifest !== 'object' || Array.isArray(manifest))
25
+ throw new VerifyError('CONFIG_INVALID', 'package.json must contain an object.');
26
+ return ['dependencies', 'devDependencies', 'optionalDependencies', 'peerDependencies'].some((field) => {
27
+ const dependencies = manifest[field];
28
+ return dependencies && typeof dependencies === 'object' && !Array.isArray(dependencies) && Object.hasOwn(dependencies, PACKAGE_NAME);
29
+ });
30
+ }
31
+ export async function installVerify(cwd) {
32
+ const manager = detectPackageManager(cwd);
33
+ if (!existsSync(join(cwd, 'package.json')))
34
+ await writeFile(join(cwd, 'package.json'), '{\n "private": true,\n "type": "module"\n}\n', { flag: 'wx' });
35
+ const args = manager === 'npm'
36
+ ? ['install', '--save-dev', '--ignore-scripts']
37
+ : manager === 'yarn'
38
+ ? ['add', '--dev', '--mode=skip-builds']
39
+ : ['add', '--dev', '--ignore-scripts'];
40
+ args.push(PACKAGE_NAME);
41
+ process.stdout.write(`Installing ${PACKAGE_NAME} with ${manager}...\n`);
42
+ await new Promise((resolve, reject) => {
43
+ const env = Object.fromEntries(['PATH', 'HOME', 'USERPROFILE', 'TMPDIR', 'TEMP', 'TMP'].flatMap((key) => {
44
+ const value = process.env[key];
45
+ return value === undefined ? [] : [[key, value]];
46
+ }));
47
+ const child = spawn(manager, args, { cwd, env, stdio: 'inherit', shell: process.platform === 'win32', timeout: 180_000, killSignal: 'SIGKILL' });
48
+ const fail = () => reject(new VerifyError('CONFIG_INVALID', `Could not install ${PACKAGE_NAME} with ${manager}. Install it manually, then retry init --no-install.`));
49
+ child.once('error', fail);
50
+ child.once('exit', (code) => code === 0 ? resolve() : fail());
51
+ });
52
+ }
53
+ function detectPackageManager(cwd) {
54
+ const lockfiles = [['pnpm-lock.yaml', 'pnpm'], ['yarn.lock', 'yarn'], ['bun.lock', 'bun'], ['bun.lockb', 'bun'], ['package-lock.json', 'npm'], ['npm-shrinkwrap.json', 'npm']];
55
+ for (let directory = cwd;; directory = dirname(directory)) {
56
+ const managers = [...new Set(lockfiles.filter(([file]) => existsSync(join(directory, file))).map(([, manager]) => manager))];
57
+ if (managers.length > 1)
58
+ throw new VerifyError('CONFIG_INVALID', 'Found lockfiles for multiple package managers. Keep the project lockfile and retry init.');
59
+ if (managers[0])
60
+ return managers[0];
61
+ if (dirname(directory) === directory)
62
+ return 'npm';
63
+ }
64
+ }
@@ -0,0 +1,6 @@
1
+ import { type Credentials } from '../auth.js';
2
+ export declare function signIn(options?: {
3
+ force?: boolean;
4
+ backendUrl?: string;
5
+ appUrl?: string;
6
+ }): Promise<Credentials>;