@revyl/verify 0.1.0 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +21 -1
- package/README.md +103 -66
- package/dist/api/types.d.ts +6 -0
- package/dist/cli/init.js +2 -3
- package/dist/cli/main.js +5 -5
- package/dist/computer-session.d.ts +8 -2
- package/dist/computer-session.js +117 -21
- package/dist/config.d.ts +1 -1
- package/dist/errors.d.ts +1 -1
- package/dist/index.d.ts +1 -1
- package/dist/open.d.ts +1 -1
- package/dist/open.js +2 -5
- package/dist/screen-readiness.d.ts +1 -0
- package/dist/screen-readiness.js +27 -0
- package/dist/session.d.ts +9 -0
- package/dist/session.js +32 -0
- package/docs/README.md +3 -2
- package/docs/agents.md +14 -9
- package/docs/api.md +22 -10
- package/docs/ci.md +1 -1
- package/docs/computers.md +22 -12
- package/docs/configuration.md +11 -10
- package/docs/end-to-end.md +105 -0
- package/docs/errors.md +6 -5
- package/docs/getting-started.md +21 -12
- package/docs/linux-computers-contract.md +34 -88
- package/docs/llms.txt +5 -4
- package/docs/mobile.md +6 -0
- package/docs/web.md +1 -7
- package/package.json +7 -4
package/docs/errors.md
CHANGED
|
@@ -21,10 +21,11 @@ try {
|
|
|
21
21
|
| `AUTH_MISSING` | No `REVYL_API_KEY` and no Revyl CLI login | Set `REVYL_API_KEY` or run `revyl auth login` |
|
|
22
22
|
| `AUTH_REJECTED` | Revyl rejected the credentials | Create a new API key, or log in again |
|
|
23
23
|
| `API_ERROR` | Revyl returned an unexpected response | Try again; if it persists, report it with the session link |
|
|
24
|
-
| `SESSION_START_FAILED` | The browser or
|
|
25
|
-
| `SESSION_ENDED` | The session was closed,
|
|
26
|
-
| `SESSION_BUSY` |
|
|
27
|
-
| `
|
|
24
|
+
| `SESSION_START_FAILED` | The browser, device, or computer did not start | Computer `no_host` reports the 120-second unclaimed-start context; `device_limit` reports queued-work/concurrency context. Inspect the session and cleanup before starting again. |
|
|
25
|
+
| `SESSION_ENDED` | The session was closed, stopped, or expired | Close explicitly; computer expiry does not prove release |
|
|
26
|
+
| `SESSION_BUSY` | A step conflict, computer operation limit, or recording requested before terminal state and confirmed release | Inspect the state before another call; a terminal computer alone does not prove release |
|
|
27
|
+
| `COMPUTER_LIMIT` | Computer admission was refused due to queued work (including iOS) or the concurrency limit | Wait for capacity; Verify does not retry or stop a definitely refused start |
|
|
28
|
+
| `NOT_AVAILABLE_ON_PLATFORM` | The call is unavailable: computer agent steps, device shell/file/recording methods, computer OS/organization not enabled or runtime unavailable, or a backend without web steps | See [web.md](web.md) and [computers.md](computers.md#availability-and-errors) |
|
|
28
29
|
| `COMMAND_FAILED` | A computer command exited nonzero | Read its reason; use `exec(argv, { check: false })` to inspect the exit code and output instead |
|
|
29
30
|
| `NOT_FOUND` | A computer file or resource was not found | Check the workspace-relative path |
|
|
30
31
|
| `ACCESS_DENIED` | The computer refused access | Check permissions; do not assume host/root access |
|
|
@@ -33,6 +34,6 @@ try {
|
|
|
33
34
|
| `INSTRUCTION_FAILED` | Revyl's agent could not do what the instruction asked | Make the instruction more specific, or split it into smaller steps |
|
|
34
35
|
| `VALIDATION_FAILED` | The validation did not hold | The message has Revyl's reason; watch the recording at the session link |
|
|
35
36
|
| `STEP_FAILED` | Revyl could not run the step to an outcome: it errored, it was cancelled on Revyl, or a validation came back without a verdict. This says nothing about your app, and `soft` does not cover it. | Watch the session at the link, then run the step again |
|
|
36
|
-
| `TIMEOUT` | A step, a session start, or a request ran out of time |
|
|
37
|
+
| `TIMEOUT` | A step, a session start, or a request ran out of time | Inspect the session before retrying; computer starts require staging enablement. The default 10-minute start timeout is separate from the 120-second unclaimed-start failure. See [computer availability](computers.md#availability-and-errors). |
|
|
37
38
|
|
|
38
39
|
Computer operations use `TIMEOUT` for bounded runtime with confirmed termination, and `ACTION_UNCERTAIN` when termination, execution, or cleanup is unconfirmed. An observed successful operation can return its result while logging pending/failed evidence. Timeout never proves rollback, and failed evidence does not prove the command never ran. Never automatically retry a mutation with a fresh request id; inspect the computer first. See [computers.md](computers.md).
|
package/docs/getting-started.md
CHANGED
|
@@ -29,23 +29,24 @@ npm install --save-dev @revyl/verify
|
|
|
29
29
|
|
|
30
30
|
## 3. Authenticate
|
|
31
31
|
|
|
32
|
-
Verify uses the same
|
|
32
|
+
Verify uses the same credential precedence as the Revyl CLI:
|
|
33
33
|
|
|
34
|
-
1. `
|
|
35
|
-
2.
|
|
34
|
+
1. A usable login in `~/.revyl/credentials.json` marked `local_auth_override` takes precedence, even when `REVYL_API_KEY` is set.
|
|
35
|
+
2. Otherwise, `REVYL_API_KEY`, an API key you create in Revyl's settings.
|
|
36
|
+
3. Otherwise, a usable login stored by the Revyl CLI (`revyl auth login`). Expired access tokens fall back to a stored API key when present.
|
|
36
37
|
|
|
37
38
|
```bash
|
|
38
39
|
export REVYL_API_KEY=... # or: revyl auth login
|
|
39
|
-
npx verify whoami
|
|
40
|
+
npx @revyl/verify whoami # checks the credentials without starting a session
|
|
40
41
|
```
|
|
41
42
|
|
|
42
43
|
## 4. Point the config at your app
|
|
43
44
|
|
|
44
|
-
Web and mobile targets need an app. Open the app in Revyl and copy its id into `app.appId`, or set `REVYL_APP_ID`, which the generated config reads. Mac targets need no app id.
|
|
45
|
+
Web and mobile targets need an app. Open the app in Revyl and copy its id into `app.appId`, or set `REVYL_APP_ID`, which the generated config reads. Mac and Linux targets need no app id. Linux init is intentionally unavailable; use `open({ platform: 'linux' })` or add a Linux config target.
|
|
45
46
|
|
|
46
47
|
- **Web:** set `app.url` to a public HTTPS URL, such as a staging or preview deployment. Revyl's cloud browser cannot reach `localhost`. See [web.md](web.md).
|
|
47
48
|
- **iOS and Android:** upload a build to the app in Revyl, for example with `revyl build upload`; each session installs the newest one. See [mobile.md](mobile.md).
|
|
48
|
-
- **Mac:** early access is not yet available in Revyl production.
|
|
49
|
+
- **Mac and Linux:** early access is not yet available in Revyl production. Both work only on Revyl staging for organizations enabled for the requested OS through the optional `REVYL_BACKEND_URL` override and matching staging credentials. Agent instruction/validation are unavailable. See [computers.md](computers.md#availability-and-errors).
|
|
49
50
|
|
|
50
51
|
## 5. Run the script
|
|
51
52
|
|
|
@@ -55,14 +56,19 @@ node verify/example.ts
|
|
|
55
56
|
|
|
56
57
|
If Node warns that the module type is not specified, add `"type": "module"` to your `package.json` or rename the script to `example.mts`.
|
|
57
58
|
|
|
59
|
+
Illustrative output (durations vary):
|
|
60
|
+
|
|
58
61
|
```text
|
|
59
62
|
[verify] ios: starting ios session https://app.revyl.ai/sessions/…
|
|
60
63
|
[verify] ios: ready
|
|
64
|
+
[verify] ios: ✓ instruction "Wait for the app to finish opening and show its first screen" (12.0s)
|
|
61
65
|
[verify] ios: ✓ validation "The app has opened and shows its first screen without an error" (9.8s)
|
|
62
66
|
[verify] ios: closed
|
|
63
67
|
```
|
|
64
68
|
|
|
65
|
-
Open the link to watch the session live. After the session ends, the link keeps its recording
|
|
69
|
+
Open the link to watch the session live. After the session ends, the link keeps its report and any available recording. The Revyl CLI prints the stored report with `revyl device report --session-id <id>` for web/mobile, or `revyl computer report <id>` for computers. Computer recordings can be absent or still processing after release; `recording()` retrieves optional private metadata without waiting or downloading.
|
|
70
|
+
|
|
71
|
+
For validation verdicts in device reports, inspect each step's `effective_status` and `validation_result`, and compare `validations_passed` with `total_validations`. `passed_steps`, `failed_steps`, and transport `status: success` can describe successful execution of a validation whose verdict is false. See the [live example](end-to-end.md#reading-the-verdict).
|
|
66
72
|
|
|
67
73
|
## The whole API
|
|
68
74
|
|
|
@@ -70,10 +76,13 @@ Open the link to watch the session live. After the session ends, the link keeps
|
|
|
70
76
|
import { open } from '@revyl/verify';
|
|
71
77
|
|
|
72
78
|
const session = await open('ios'); // a target from verify.config.ts
|
|
73
|
-
|
|
74
|
-
await session.
|
|
75
|
-
await session.
|
|
76
|
-
await session.
|
|
79
|
+
try {
|
|
80
|
+
await session.instruction('Add the first product to the cart'); // Revyl's agent does it
|
|
81
|
+
await session.validation('The cart badge shows 1'); // throws if it does not hold
|
|
82
|
+
await session.screenshot({ path: '.verify/cart.png' });
|
|
83
|
+
} finally {
|
|
84
|
+
await session.close();
|
|
85
|
+
}
|
|
77
86
|
```
|
|
78
87
|
|
|
79
|
-
A failed instruction or validation throws a `VerifyError`.
|
|
88
|
+
A failed instruction or validation throws a `VerifyError`. An uncaught error in a plain script exits non-zero. Automatic cleanup is best effort, bounded to 15 seconds, and skips signal/error handlers that already have listeners. Always close in `try/finally`; a successful web/mobile close acknowledges the stop request, not confirmed physical release. Computer close requires confirmed terminal release. See [api.md](api.md) for cleanup details and every option, and [ci.md](ci.md) for calling Verify from a test runner.
|
|
@@ -1,107 +1,53 @@
|
|
|
1
|
-
# Linux computers:
|
|
1
|
+
# Linux computers: implemented shared contract
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
Verify 0.2.0 enables Linux named and inline targets with the same computer API as Mac. Both operating systems are **early access on staging for organizations enabled for the requested OS, not yet available in production**. Server admission decides availability; SDK support is not a guarantee that capacity is available. No live Linux run was performed for this SDK change.
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
Use `open({ platform: 'linux', startTimeout: 600_000 })`, or add `linux: { platform: 'linux' }` under `targets` in `verify.config.ts`. There is no app, image, provider, cwd, TTL, or device selector. `init --platform linux` remains intentionally unavailable; it does not block inline or named Linux SDK targets. See [computers.md](computers.md) for staging setup and usage.
|
|
6
6
|
|
|
7
|
-
|
|
7
|
+
## Public surface
|
|
8
8
|
|
|
9
|
-
|
|
9
|
+
- `exec(argv, { timeout?, stdin?, check? }?)` returns `{ exitCode, stdout, stderr }`. There is no implicit shell. Nonzero exit throws `COMMAND_FAILED` unless `check: false`.
|
|
10
|
+
- `readFile`, `writeFile`, and `stat` use workspace-relative paths and preserve binary bytes.
|
|
11
|
+
- `processes` and `terminate` use run-owned handles, never OS PIDs.
|
|
12
|
+
- `screenshot({ path? }?)` returns PNG bytes with dimensions from the response; screenshot readiness is independent of command readiness. The SDK retries only structured startup screen-not-ready responses within its fixed 30-second window after active readiness; [computer docs](computers.md) describe the bounds and typed exhaustion error.
|
|
13
|
+
- `close()` succeeds only on terminal state and confirmed physical release.
|
|
14
|
+
- `recording()` returns private recording metadata or null after terminal state and confirmed release, and works after close. It does not wait or download.
|
|
15
|
+
- `expiresAt` exposes a validated API timestamp as a `Date`, or is undefined on older APIs. The hard lifetime is 60 minutes from create, not readiness or activity. Expired operations are rejected, pending observations are bounded by expiry, and close remains possible.
|
|
10
16
|
|
|
11
|
-
|
|
17
|
+
`instruction` and `validation` are rejected locally on both computer operating systems. There is no computer agent-step endpoint. The assigned-physical-computer `shell-sessions` API is not part of this contract. Installed tools, guest paths, resources, architecture, and screen dimensions are not SDK guarantees; use portable commands or inspect what the selected guest provides.
|
|
12
18
|
|
|
13
|
-
|
|
14
|
-
- `readFile(path, options?)` returns `Buffer`; `writeFile(path, string | Uint8Array, options?)` returns `void`.
|
|
15
|
-
- `stat(path, options?)` returns `{ kind: 'file' | 'directory', size: number }` in bytes.
|
|
16
|
-
- `processes(options?)` returns `{ id, state: 'running' | 'exited', exitCode?: number }[]`; `terminate(id, options?)` uses the run-owned handle id, never a PID.
|
|
17
|
-
- `close()` stops the computer and preserves the distinction between terminal state and confirmed release.
|
|
19
|
+
## Requests and readiness
|
|
18
20
|
|
|
19
|
-
|
|
21
|
+
All routes use `/api/v1/computer-runs`, the selected `REVYL_BACKEND_URL`, bearer authentication, `X-Revyl-Client: verify`, and Verify's versioned user agent. Ownership is enforced by the server; possession of a UUID grants no access. `REVYL_APP_URL` controls UI links.
|
|
20
22
|
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
All routes below use the selected `REVYL_BACKEND_URL` plus `/api/v1`, the existing bearer credentials, `X-Revyl-Client: verify`, and the versioned Verify user agent. Organization ownership comes from authentication, not UUID possession. Reads require same-org access; create, operations, and stop require an operator role and computer admission. Preserve the existing `REVYL_APP_URL` override for session links. No provider-specific credentials belong in Verify.
|
|
24
|
-
|
|
25
|
-
| Request | Response and client behavior |
|
|
26
|
-
| --- | --- |
|
|
27
|
-
| `POST /computer-runs` with `{ computer_session_id: C1, os: 'linux' }` | `ComputerReport`. Mint canonical nonzero UUID C1 once on the client. Do not remint/replay after an uncertain create. |
|
|
28
|
-
| `GET /computer-runs/C1/report` | `ComputerReport` for readiness. Poll only while `starting`; resolve open on `active` (owner route ready). `closing`, `stopped`, `failed`, or `owner_lost` are not ready; fail and attempt cleanup within bounds. |
|
|
29
|
-
| `POST /computer-runs/C1/operations` with `{ computer_session_id: C1, request_id: R1, command }`, `Prefer: respond-async` | `ComputerCommandResponse`, or HTTP `202` with exact JSON `null` while still running after the async wait. Mint R1 once per operation; C1 in path and body must agree. |
|
|
30
|
-
| `GET /computer-runs/C1/operations/O1` | Original retained `ComputerCommandResponse`, or `202`/`null` while running. O1 is UUIDv5(namespace=C1, name=canonical R1 string). Poll reads, never resubmit to poll. |
|
|
31
|
-
| `POST /computer-runs/C1/stop` | `ComputerReport` with separate execution end, evidence status, and release certainty. A stopped/failed/lost-owner state does not imply confirmed release. |
|
|
32
|
-
|
|
33
|
-
A report contains `kind: 'computer'`, `version: 1`, UUIDs `report_id`, `computer_session_id`, `org_id`, `os`, timestamps `created_at`, `expires_at`, `as_of`, lifecycle `state`, optional `failure_reason`, execution timestamps, `release: 'pending' | 'confirmed' | 'unknown'`, `released_at`, `evidence_completeness: 'complete' | 'partial'`, `evidence_write_status: 'recorded' | 'pending' | 'failed'`, and operation evidence `steps`. Verify must validate identities and the fields it consumes, without treating recording success as execution success.
|
|
34
|
-
|
|
35
|
-
Identical requests using the same R1 return the retained result; changed input conflicts. The original owner retains raw results; there is no cross-restart command recovery guarantee. Reads must be bounded by an overall deadline. Cancellation/timeout does not authorize an automatic mutation retry.
|
|
36
|
-
|
|
37
|
-
## Command payloads and results
|
|
38
|
-
|
|
39
|
-
Every command has integer `timeout_ms` from 1 to 300,000 (default 30,000), bounded by remaining run lifetime. Current run lifetime is 3,600 seconds and operation count is at most 256.
|
|
40
|
-
|
|
41
|
-
| `command` fields beyond timeout | Successful `result` |
|
|
23
|
+
| Request | Contract |
|
|
42
24
|
| --- | --- |
|
|
43
|
-
| `{
|
|
44
|
-
| `
|
|
45
|
-
| `{
|
|
46
|
-
| `
|
|
47
|
-
| `
|
|
48
|
-
| `
|
|
25
|
+
| `POST /computer-runs` | `{ computer_session_id: C1, os: 'linux' }`; Mac sends `os: 'macos'`. Mint C1 once, never replay an uncertain create. |
|
|
26
|
+
| `GET /computer-runs/C1/report` | Validate C1 and lifecycle state. Poll while starting, resolve on active, fail on other states. `failed/no_host` means an unclaimed start reached the 120-second window, not that all starts must finish within it. The default client start timeout stays 10 minutes. |
|
|
27
|
+
| `POST /computer-runs/C1/operations` | `{ computer_session_id: C1, request_id: R1, command }`, with `Prefer: respond-async`. Response is a command result or `202` with exact JSON null. |
|
|
28
|
+
| `GET /computer-runs/C1/operations/O1` | Read the retained result, or `202`/null while running. O1 is UUIDv5(namespace=C1, name=R1 string). Poll reads without replaying mutations. |
|
|
29
|
+
| `POST /computer-runs/C1/stop` | Report state and release separately. Close requires `stopped`, `failed`, or `owner_lost` plus `release: 'confirmed'`. |
|
|
30
|
+
| `GET /computer-runs/C1/recording` | After a separate report confirms terminal state and release: `{ url, duration_seconds?, started_at? }`, or `404 { error: 'recording_unavailable' }`. Optional fields may be null. No URL is carried by the normal report. |
|
|
49
31
|
|
|
50
|
-
|
|
32
|
+
Recording URLs must be HTTPS and are signed, short-lived, and private, including the query string. Duration must be finite and nonnegative, and timestamps must include a timezone. Malformed responses produce a generic error that does not echo the URL. The SDK never fetches that URL. A pre-release recording request throws `SESSION_BUSY`; missing recording returns null without automatic waiting.
|
|
51
33
|
|
|
52
|
-
|
|
34
|
+
## Commands and uncertainty
|
|
53
35
|
|
|
54
|
-
|
|
36
|
+
Commands retain the shared limits: 30-second default / 5-minute maximum timeout, 256 operations per run, 64 argv entries, 16,384 bytes of argv plus stdin, first 1 MiB combined stdout/stderr, 1 MiB reads, 64 KiB writes, and 64 listed handles. File paths must be workspace-relative without traversal. See [computers.md](computers.md) for complete validation rules.
|
|
55
37
|
|
|
56
|
-
|
|
57
|
-
- `failed`: `error_code: 'not_found' | 'access_denied' | 'exec_nonzero'`; only `exec_nonzero` carries an exec result with an observed nonzero code.
|
|
58
|
-
- `timed_out`: `error_code: 'timed_out'`, `termination: 'confirmed' | 'unknown'`, optional `observed_exit_code`.
|
|
59
|
-
- `unknown`: error code `lost_ack`, `executor_error`, `invalid_executor_result`, `owner_lost`, `stopped`, or `start_evidence_failed`, plus termination certainty.
|
|
60
|
-
|
|
61
|
-
## Errors and uncertainty
|
|
62
|
-
|
|
63
|
-
HTTP errors use `{ detail: code }`, not an invented Linux-specific envelope. Current route/admission codes include:
|
|
64
|
-
|
|
65
|
-
| HTTP | Codes |
|
|
66
|
-
| --- | --- |
|
|
67
|
-
| 401 | `unauthenticated` |
|
|
68
|
-
| 402 | Compute-credit/billing admission refusal (`API_ERROR`; report only the sanitized reason, never raw response contents) |
|
|
69
|
-
| 403 | `forbidden`, `computer_not_enabled` |
|
|
70
|
-
| 404 | `inaccessible`, `invalid_handle`, `operation_not_found` |
|
|
71
|
-
| 409 | `request_conflict`, `run_conflict`, `owner_unavailable`, `run_closed`, `evidence_conflict` |
|
|
72
|
-
| 422 | `invalid_request` |
|
|
73
|
-
| 429 | `computer_limit`, `operation_limit` |
|
|
74
|
-
| 503 | `runtime_unavailable`, `start_failed`, `evidence_unavailable`, `authentication_unavailable` |
|
|
75
|
-
|
|
76
|
-
Do not collapse these into a missing-provider fallback or retry create/commands with a fresh id. Public Verify classifications follow Mac:
|
|
38
|
+
Operation responses identify C1 and O1 and separate `execution` from `evidence: 'recorded' | 'pending' | 'failed'`. Successful execution can return a result even when report evidence is pending or failed. An observed nonzero exec result is distinct from transport failure. Unknown outcome, unconfirmed timeout termination, and lost mutation responses produce `ACTION_UNCERTAIN` and never authorize replay. Terminal lifecycle, physical release, and recording availability are independent.
|
|
77
39
|
|
|
78
40
|
| Condition | Verify code |
|
|
79
41
|
| --- | --- |
|
|
80
|
-
|
|
|
81
|
-
| `
|
|
82
|
-
|
|
|
83
|
-
| `
|
|
84
|
-
| `run_closed
|
|
42
|
+
| `computer_not_enabled`, or `503 runtime_unavailable` | `NOT_AVAILABLE_ON_PLATFORM` |
|
|
43
|
+
| `429 computer_limit` | `COMPUTER_LIMIT`: queued work (including iOS) or concurrency limit; no automatic start retry or stop after definite admission refusal |
|
|
44
|
+
| `operation_limit`, other conflict refusals | `SESSION_BUSY` |
|
|
45
|
+
| `failed/no_host`, `failed/device_limit`, `start_failed` | `SESSION_START_FAILED`; safe no-host or queued-work/concurrency context when known |
|
|
46
|
+
| `run_closed`, or local expiry | `SESSION_ENDED` |
|
|
85
47
|
| `inaccessible`, `invalid_handle`, operation `not_found` | `NOT_FOUND` |
|
|
86
48
|
| Operation `access_denied` | `ACCESS_DENIED` |
|
|
87
|
-
| Operation `exec_nonzero` | `COMMAND_FAILED`
|
|
88
|
-
|
|
|
89
|
-
|
|
|
90
|
-
| Operation timed out with confirmed termination | `TIMEOUT` |
|
|
91
|
-
| POST 5xx or redirect not covered above | `ACTION_UNCERTAIN` |
|
|
92
|
-
| Operation polling transport failure or an unclassified polling 5xx | `ACTION_UNCERTAIN` |
|
|
93
|
-
| Unclassified response, including HTTP 402 | `API_ERROR` |
|
|
94
|
-
|
|
95
|
-
A timed-out operation with unknown termination is `ACTION_UNCERTAIN`, not `TIMEOUT`. An observed successful operation can return its result while logging pending/failed evidence. Failed evidence is not proof that execution failed or never ran. Keep raw command/file contents and credentials out of diagnostics.
|
|
96
|
-
|
|
97
|
-
## Linux assumptions requiring confirmation
|
|
98
|
-
|
|
99
|
-
Before enabling Linux, the Computer runtime owner must confirm:
|
|
49
|
+
| Operation `exec_nonzero` | `COMMAND_FAILED` unless `check: false` |
|
|
50
|
+
| Unknown execution, operation unavailable, uncertain mutation or release | `ACTION_UNCERTAIN` |
|
|
51
|
+
| Confirmed operation timeout | `TIMEOUT` |
|
|
100
52
|
|
|
101
|
-
|
|
102
|
-
2. `active` means the Linux owner route accepts commands and files, with the same report identities, async operation polling, deduplication, and retained-result semantics. Verify this against a provider, not only a fake.
|
|
103
|
-
3. The workspace root, initial working directory, user privileges, permissions, filesystem/symlink policy, persistence/isolation, and installed executable/PATH baseline. Do not assume macOS tools, Darwin output, package managers, writable absolute paths, or host access.
|
|
104
|
-
4. Structured argv launches without an implicit shell; explicit `/bin/sh` availability, process-tree termination, negative exit/signal behavior, output truncation, and binary IO match the shared contract.
|
|
105
|
-
5. Linux process handles remain run-owned, not PIDs; listing limits and termination certainty have the same meaning. Confirm behavior on owner loss and run expiry.
|
|
106
|
-
6. Whether a graphical Linux desktop and screenshots exist. Until confirmed, do not promise screenshots, screen control, instruction, or validation.
|
|
107
|
-
7. Approved live acceptance coverage: create/readiness, argv/stdin and nonzero exit, binary file round trip/stat, missing/denied file, process list/terminate, 202 polling, timeout/unknown outcome, and confirmed close. This work performs no live runs.
|
|
53
|
+
Other HTTP errors retain the shared authentication, input, or API classifications described in [errors.md](errors.md). Do not infer a capacity cause from `computer_limit` alone or expose arbitrary failure text, command data, credentials, or recording URLs in diagnostics.
|
package/docs/llms.txt
CHANGED
|
@@ -1,18 +1,19 @@
|
|
|
1
1
|
# Verify
|
|
2
2
|
|
|
3
|
-
> Verify (@revyl/verify) is an open-source TypeScript library for Revyl. It opens cloud browsers and iOS/Android devices for agent instruction/validation steps, or Mac computers for commands, files, processes,
|
|
3
|
+
> Verify (@revyl/verify) is an open-source TypeScript library for Revyl. It opens cloud browsers and iOS/Android devices for agent instruction/validation steps, or Mac and Linux computers for commands, files, processes, screenshots, and optional private recording metadata after release. Call it from a plain script, coding agent, CI, or any test runner. It has no test runner of its own.
|
|
4
4
|
|
|
5
5
|
## Docs
|
|
6
6
|
|
|
7
|
-
|
|
7
|
+
Both computer operating systems are early access and not yet available in Revyl production. They work only on Revyl staging for organizations enabled for the requested OS using the optional REVYL_BACKEND_URL override and matching staging credentials. Linux init remains intentionally unavailable; use an inline target or config entry. See computers.md#availability-and-errors.
|
|
8
8
|
|
|
9
9
|
- [Getting started](getting-started.md): install, authenticate, init, run a script
|
|
10
10
|
- [API](api.md): open(), agent steps, screenshot(), exec(), file/process methods, close()
|
|
11
11
|
- [Configuration](configuration.md): verify.config.ts targets, environment variables, the init and whoami commands
|
|
12
12
|
- [Web](web.md): Revyl web sessions, URLs and builds, sign-in, steps, and optional staging access
|
|
13
13
|
- [Mobile](mobile.md): iOS and Android devices and installing apps
|
|
14
|
-
- [Computers](computers.md#availability-and-errors): staging-only Mac early access
|
|
15
|
-
- [
|
|
14
|
+
- [Computers](computers.md#availability-and-errors): staging-only Mac/Linux early access, expiry, capacity errors, recording metadata, and confirmed release
|
|
15
|
+
- [Verify working end to end](end-to-end.md): live verification scope, observed outcomes, and limits of the evidence
|
|
16
|
+
- [Linux contract](linux-computers-contract.md): implemented shared requests/responses, readiness, errors, and uncertainty
|
|
16
17
|
- [CI and test runners](ci.md): GitHub Actions, node:test, Vitest, Jest, Playwright Test
|
|
17
18
|
- [Errors](errors.md): every error code and what to do about it
|
|
18
19
|
- [For coding agents](agents.md): the workflow and rules for agents using Verify
|
package/docs/mobile.md
CHANGED
|
@@ -41,3 +41,9 @@ for (const name of ['iphone-se', 'iphone-16']) {
|
|
|
41
41
|
## Steps
|
|
42
42
|
|
|
43
43
|
`instruction` and `validation` run as live steps on the device. Verify polls each step until Revyl reports it done, and cancels it on Revyl if it runs past its timeout.
|
|
44
|
+
|
|
45
|
+
After connection and any explicit bundle launch, `open()` makes a best-effort first-screen observation for up to 10 seconds, bounded by `startTimeout`. It looks for different RGB pixels in the central 80% of a PNG, excluding status and navigation bars. This is a blank-frame heuristic, not proof that the requested app is foreground or finished rendering: the runtime exposes no rendered-readiness signal. A timeout, unavailable screenshot, or undecodable image does not fail `open()`; it emits a `debug:` progress note through the configured logger and continues. `log: false` suppresses that note.
|
|
46
|
+
|
|
47
|
+
The generated mobile example starts with an instruction to wait for the first screen before validating it. For your own scripts, make the first instruction describe the screen or action you actually need; a screenshot with content can still show a loading screen or another app.
|
|
48
|
+
|
|
49
|
+
For Android Settings, instruct the agent to scroll or navigate until the OS version is visible before validating it. Opening About alone can leave the version offscreen.
|
package/docs/web.md
CHANGED
|
@@ -30,13 +30,7 @@ signIn: { path: '/__test/sign-in', secret: 'E2E_SIGN_IN_TOKEN', persona: 'admin'
|
|
|
30
30
|
|
|
31
31
|
`instruction` and `validation` use the same Revyl endpoints on web as on iOS and Android, and each step shows up in the session's report.
|
|
32
32
|
|
|
33
|
-
Production is the default. **Optional staging access** is for users with
|
|
34
|
-
|
|
35
|
-
```bash
|
|
36
|
-
export REVYL_BACKEND_URL=https://backend-staging.revyl.ai
|
|
37
|
-
export REVYL_APP_URL=https://staging.revyl.ai
|
|
38
|
-
export REVYL_API_KEY=... # a key from your Revyl staging organization
|
|
39
|
-
```
|
|
33
|
+
Production is the default. **Optional staging access** is for users with an approved early-access organization and matching credentials; it is not required for normal setup. Get your endpoints from Revyl, set `REVYL_BACKEND_URL` and the matching `REVYL_APP_URL`, and authenticate for that environment. Never mix production and early-access credentials.
|
|
40
34
|
|
|
41
35
|
A Revyl backend that does not run steps on web yet answers with `step_output_not_available`, which Verify reports as `NOT_AVAILABLE_ON_PLATFORM`. The same script works there with no change once it does. `open`, `screenshot`, `viewerUrl`, and `close` work on web everywhere.
|
|
42
36
|
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@revyl/verify",
|
|
3
|
-
"version": "0.
|
|
4
|
-
"description": "Open Revyl cloud browsers and mobile devices for agent steps, or Mac computers for commands and files, from TypeScript.",
|
|
3
|
+
"version": "0.2.0",
|
|
4
|
+
"description": "Open Revyl cloud browsers and mobile devices for agent steps, or Mac and Linux computers for commands and files, from TypeScript.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"e2e",
|
|
7
7
|
"end-to-end",
|
|
@@ -10,6 +10,7 @@
|
|
|
10
10
|
"android",
|
|
11
11
|
"web",
|
|
12
12
|
"macos",
|
|
13
|
+
"linux",
|
|
13
14
|
"ai",
|
|
14
15
|
"agent",
|
|
15
16
|
"revyl"
|
|
@@ -54,13 +55,15 @@
|
|
|
54
55
|
"build": "rm -rf dist && tsc -p tsconfig.build.json && chmod +x dist/cli/main.js",
|
|
55
56
|
"typecheck": "tsc -p tsconfig.json --noEmit",
|
|
56
57
|
"test": "vitest run",
|
|
57
|
-
"prepack": "npm run build &&
|
|
58
|
+
"prepack": "npm run build && node ../../scripts/prepare-package.mjs"
|
|
58
59
|
},
|
|
59
60
|
"dependencies": {
|
|
60
|
-
"jiti": "^2.6.0"
|
|
61
|
+
"jiti": "^2.6.0",
|
|
62
|
+
"pngjs": "^7.0.0"
|
|
61
63
|
},
|
|
62
64
|
"devDependencies": {
|
|
63
65
|
"@types/node": "^22.10.0",
|
|
66
|
+
"@types/pngjs": "^6.0.5",
|
|
64
67
|
"typescript": "^5.9.0",
|
|
65
68
|
"vitest": "^5.0.5"
|
|
66
69
|
},
|