clearcote 0.33.0 → 0.35.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -5,7 +5,8 @@ open, reproducible, anti-fingerprint Chromium build. `launch()` returns a standa
5
5
  `Browser`, so migrating is a one-line import change.
6
6
 
7
7
  The verified Clearcote binary is **auto-downloaded and SHA-256 checked** on first use, then cached —
8
- you don't manage zips or paths.
8
+ you don't manage zips or paths. Or run the same code on a hosted Clearcote browser with
9
+ `launch({ cloud: true })`: see [Local or cloud](#local-or-cloud).
9
10
 
10
11
  > **Platform:** Clearcote ships **Windows x64** and **Linux x64** binaries; `launch()` runs on both
11
12
  > and the SDK auto-downloads the right one for your OS. On Linux the persona is Linux-native (Linux
@@ -134,6 +135,7 @@ npx clearcote login [key] # validates the key, saves it to ~/.clearcote/lic
134
135
  npx clearcote logout
135
136
  npx clearcote clear-cache
136
137
  npx clearcote serve [--port 9222] ...
138
+ npx clearcote cloud run|sessions|stop|events|recording|profile sync|webhooks ... # the hosted API, see Local or cloud
137
139
  ```
138
140
 
139
141
  `info` never downloads: it reports the SDK, the cached builds, which engine features the binary supports, a launch test (skipped with `--quick`), licence seats, fonts and missing system libraries.
@@ -305,6 +307,109 @@ don't set `acceptLanguage` explicitly. See the
305
307
  [collector README](../../tools/fingerprint-collect/README.md) for the full schema and what each
306
308
  field drives.
307
309
 
310
+ ## Local or cloud
311
+
312
+ The same `launch()` runs the browser on this machine or on Clearcote's servers. One flag picks which,
313
+ and both resolve to the same Playwright `Browser`:
314
+
315
+ ```ts
316
+ import { launch } from "clearcote";
317
+
318
+ const browser = await launch({ cloud: true, country: "us", identity: "acct-1", humanize: true });
319
+ const page = await browser.newPage();
320
+ await page.goto("https://example.com");
321
+ await browser.close(); // disconnects and ends the hosted session
322
+ ```
323
+
324
+ Leave `cloud` unset and set `CLEARCOTE_CLOUD=1` to move existing code to the cloud without editing
325
+ it. The API key comes from `apiKey` or `CLEARCOTE_API_KEY`, and `CLEARCOTE_API_URL` points the SDK at
326
+ another server (https only: plain `http://` is accepted for `127.0.0.1`, `::1` and `localhost` alone,
327
+ so the key never crosses a network unencrypted).
328
+
329
+ What changes in the cloud:
330
+
331
+ - **Options** keep the API's names: `fingerprint`, `identity`, `platform`, `brand`, `timezone`,
332
+ `locale` (or `acceptLanguage`), `geoip`, `headless`, `lightStealth`, `proxy` (`"managed"`, a URL, or
333
+ `{ server, username, password }`), `country`/`state`/`city`, `proxySession`, `timeoutSec`,
334
+ `idleTimeoutSec`, `maxGb`, `version`, `profile`, `url`, `adblock`, `keepAlive`, `record`, `note`,
335
+ `worker`. `humanize` and `showCursor` run in the SDK, exactly as for a local browser.
336
+ - **Local-only options are refused, by name.** `executablePath`, `args`, `userDataDir`, `extensions`,
337
+ `ignoreDefaultArgs`, the finer persona switches (`gpuVendor`, `webrtcIp`, ...) and the licence
338
+ options throw `<name> is not available for cloud browsers` before anything starts.
339
+ - **Profiles are cloud profiles.** `profile: "acct-1"` loads the cookies of a named cloud profile, and
340
+ `launchPersistentContext({ cloud: true, profile: "acct-1" })` resolves to a context that also saves
341
+ them back when it closes. Fill one from a browser you are logged in to with `profiles.sync` (below).
342
+ - **The session.** `cloudSessionOf(browser)` (also `browser.cloudSession`) holds its `id`, `worker` and
343
+ `expiresAt`. `close()` disconnects and ends the session; a `keepAlive: true` session is left running
344
+ (stop it with `cloud.browsers.stop(id)`).
345
+
346
+ ### The Cloud client
347
+
348
+ Everything else the hosted API does is on `Cloud`:
349
+
350
+ ```ts
351
+ import { Cloud } from "clearcote";
352
+
353
+ const cloud = new Cloud(); // CLEARCOTE_API_KEY
354
+
355
+ // An agent run: a task in, JSON out
356
+ const run = await cloud.runs.create("Log in with {{password}} and read the current plan's price", {
357
+ url: "https://example.com/login",
358
+ schema: { type: "object", properties: { price: { type: "string" } } },
359
+ secrets: { password: { value: "s3cret", domains: ["example.com"] } }, // the model never sees it
360
+ });
361
+ console.log(run.status, run.result.output, run.costEur.total);
362
+
363
+ // Copy a logged-in state into a cloud profile: only the cookies a browser would use on the domains you
364
+ // name (theirs, their subdomains', and parent-domain ones such as .example.com for www.example.com)
365
+ await cloud.profiles.sync("acct-1", { loginUrl: "https://example.com/login", domains: ["example.com"] });
366
+
367
+ // Recordings, the event timeline, hand-off to a person
368
+ const s = await cloud.browsers.create({ record: true });
369
+ await cloud.browsers.events(s.id); // { events: [...], next }
370
+ await cloud.browsers.downloadRecording(s.id, "session.mp4");
371
+ ```
372
+
373
+ - `browsers`: `create(options)`, `get(id)`, `list({ status, note, limit, before })`, `stop(id)`,
374
+ `live(id, { control })`, `share(id, { control, minutes, recording })`,
375
+ `handoff(id, { reason, timeoutSec })`, `handoffDone(id)`, `waitHandoff(id, { timeout, poll })`,
376
+ `events(id, { after, limit })`, `recordingUrl(id)`, `downloadRecording(id, path)`.
377
+ `waitHandoff` on a session with no hand-off rejects with `CloudError` (code `NO_HANDOFF`).
378
+ - `runs`: `create(task, { url, schema, secrets, wait = true, timeout, poll = 1.5, onUpdate, ...options })`
379
+ (the browser options above plus `maxSteps`, `handoff`, `handoffTimeoutSec`), `get(id)`,
380
+ `list({ limit, before })`, `cancel(id)`, `wait(id, { timeout, poll, onUpdate })`. A run that pauses
381
+ for a person (`waiting_for_human`) is not finished: `onUpdate` hears about it, and without one the
382
+ live link is printed to stderr. `timeout` and `poll` are in seconds, as in the Python SDK.
383
+ - `profiles`: `list()`, `get(name)`, `delete(name)`, `importCookies(name, cookies, { mode })`,
384
+ `sync(name, { fromProfile | fromCdp | fromFile | loginUrl, domains | allDomains, replace })`. `sync`
385
+ refuses to run without `domains` or `allDomains: true`.
386
+ - `webhooks`: `create(url, { events, description })` (the answer carries the `secret`, shown once),
387
+ `list()`, `delete(id)`, `test(id)`
388
+
389
+ Every method resolves to the parsed JSON of its endpoint. An API refusal rejects with `CloudError`
390
+ (`status`, `code` such as `"PROFILE_IN_USE"`, and the server's own `message`); a wait that runs out
391
+ rejects with `CloudTimeoutError`, whose `last` holds the last view (the run carries on on the server).
392
+
393
+ Check a webhook delivery with the raw request body and the `Clearcote-Signature` header:
394
+
395
+ ```ts
396
+ import { verifyWebhook } from "clearcote";
397
+
398
+ const event = verifyWebhook(rawBody, req.headers["clearcote-signature"], "whsec_..."); // throws if forged or stale
399
+ ```
400
+
401
+ The same from the command line (`--json` for machine-readable output):
402
+
403
+ ```bash
404
+ npx clearcote cloud run "Read the price of the Pro plan" --url https://example.com/pricing --schema price.json
405
+ npx clearcote cloud run "Open my account page" --country us --profile acct-1 --persist-profile --max-steps 20
406
+ npx clearcote cloud sessions
407
+ npx clearcote cloud events bs_123
408
+ npx clearcote cloud recording bs_123 -o session.mp4
409
+ npx clearcote cloud profile sync acct-1 --login https://example.com/login --domain example.com
410
+ npx clearcote cloud webhooks add https://hooks.example.com/clearcote --event run.finished
411
+ ```
412
+
308
413
  ## Fingerprint options
309
414
 
310
415
  All optional. Anything not listed here is passed straight through to Playwright
@@ -404,8 +509,9 @@ Profiles are JSON at `~/.clearcote/profiles/<name>.json` (set `CLEARCOTE_PROFILE
404
509
 
405
510
  ## API
406
511
 
407
- - `launch(options?)` → `Promise<Browser>` — launch and get a Playwright `Browser`. Pass `profile` (a name, path, or `Profile`) to launch a saved persona.
408
- - `launchPersistentContext(userDataDir, options?)` → `Promise<BrowserContext>`.
512
+ - `launch(options?)` → `Promise<Browser>` — launch and get a Playwright `Browser`. Pass `profile` (a name, path, or `Profile`) to launch a saved persona; `cloud: true` for a hosted browser ([Local or cloud](#local-or-cloud)).
513
+ - `launchPersistentContext(userDataDir, options?)` → `Promise<BrowserContext>`; `launchPersistentContext({ cloud: true, profile: "name" })` for a cloud profile.
514
+ - `new Cloud({ apiKey, baseUrl })` — the hosted API (`browsers`, `runs`, `profiles`, `webhooks`); `CloudError`, `CloudTimeoutError`, `verifyWebhook(rawBody, signatureHeader, secret, toleranceSec = 300)`, `cloudSessionOf(browser)`.
409
515
  - `serve(options?)` → `Promise<Server>` — a standing, stealthy CDP endpoint (`.cdpUrl` / `.wsUrl()` / `.close()`) any Playwright/Puppeteer/CDP client attaches to via `connectOverCDP`.
410
516
  - `executablePath(options?)` → `Promise<string>` — resolve (download/verify if needed) the chrome.exe path, e.g. for raw `chromium.launch({ executablePath })`.
411
517
  - `download(options?)` → `Promise<string>` — pre-fetch + verify the binary without launching.
@@ -1,5 +1,8 @@
1
1
  import { type SessionSeats } from "./index.js";
2
+ import { type Json } from "./cloud.js";
2
3
  export declare const USAGE: string;
4
+ /** `clearcote cloud --help`. Kept byte-identical to CLOUD_USAGE in the Python SDK's _commands.py. */
5
+ export declare const CLOUD_USAGE = "clearcote cloud -- the hosted Clearcote API from the command line.\n\nUSAGE\n clearcote cloud run <task> [--url <url>] [--schema <file.json>] [--secret <name>=<value>]...\n [--secret-domain <name>=<host>]... [--handoff] [--record] [--json]\n [--country <cc>] [--state <s>] [--city <c>] [--proxy managed|<url>]\n [--profile <name> [--persist-profile]] [--fingerprint <seed>]\n [--timeout-sec <n>] [--max-steps <n>] [--note <text>]\n clearcote cloud sessions [--json]\n clearcote cloud stop <id> [--json]\n clearcote cloud events <id> [--json]\n clearcote cloud recording <id> [-o <file.mp4>] [--json]\n clearcote cloud profile sync <name> (--from-profile <dir> | --from-cdp <url> | --from-file <state.json>\n | --login <url>) (--domain <domain>... | --all-domains) [--replace] [--json]\n clearcote cloud webhooks add <url> [--event <type>]... [--json]\n clearcote cloud webhooks list [--json]\n clearcote cloud webhooks rm <id> [--json]\n clearcote cloud webhooks test <id> [--json]\n\n run waits for the result and exits 0 only when the run succeeded. profile sync uploads only the\n cookies of the --domain names you list (subdomains included); --all-domains uploads every cookie.\n\nENVIRONMENT\n CLEARCOTE_API_KEY (required), CLEARCOTE_API_URL (default https://www.clearcotelabs.com)";
3
6
  /** What `info` reports. Also the `--json` shape. */
4
7
  export interface InfoReport {
5
8
  sdk: {
@@ -53,6 +56,16 @@ export declare function buildInfo(flags: {
53
56
  proxy?: string;
54
57
  }): Promise<InfoReport>;
55
58
  export declare function main(argv?: string[]): Promise<void>;
59
+ /** The human summary `clearcote cloud run` prints. */
60
+ export declare function formatRun(run: Json): string[];
61
+ /** --secret name=value and --secret-domain name=host as the API's `secrets` object: a plain string,
62
+ * or { value, domains } once a domain is given for that name. */
63
+ export declare function cloudSecrets(secretArgs?: string[], domainArgs?: string[]): Record<string, unknown>;
64
+ /** The browser and run options of `clearcote cloud run` (unset ones left out). --profile NAME loads a
65
+ * cloud profile; with --persist-profile the run also saves it back. */
66
+ export declare function cloudRunOptions(v: Record<string, Json>): Record<string, unknown>;
67
+ /** `clearcote cloud …`: resolves to the exit code (0 ok, 1 failed, 2 usage). Never exits itself. */
68
+ export declare function cloudMain(argv: string[]): Promise<number>;
56
69
  /** `--proxy scheme://user:pass@host:port` as a launch proxy option with the credentials split out. */
57
70
  export declare function proxyOption(url: string): {
58
71
  server: string;
@@ -1 +1 @@
1
- {"version":3,"file":"cli-commands.d.ts","sourceRoot":"","sources":["../src/cli-commands.ts"],"names":[],"mappings":"AAiBA,OAAO,EAcL,KAAK,YAAY,EAClB,MAAM,YAAY,CAAC;AAcpB,eAAO,MAAM,KAAK,QAqBiF,CAAC;AAWpG,oDAAoD;AACpD,MAAM,WAAW,UAAU;IACzB,GAAG,EAAE;QAAE,OAAO,EAAE,MAAM,CAAC;QAAC,IAAI,EAAE,MAAM,CAAC;QAAC,QAAQ,EAAE,MAAM,CAAA;KAAE,CAAC;IACzD,OAAO,EAAE;QAAE,MAAM,EAAE,MAAM,CAAC;QAAC,GAAG,CAAC,EAAE,MAAM,CAAC;QAAC,KAAK,CAAC,EAAE,YAAY,CAAA;KAAE,CAAC;IAChE,MAAM,EAAE;QACN,MAAM,EAAE,kBAAkB,GAAG,OAAO,GAAG,MAAM,CAAC;QAC9C,IAAI,CAAC,EAAE,MAAM,CAAC;QACd,GAAG,CAAC,EAAE,MAAM,CAAC;QACb,MAAM,EAAE,KAAK,CAAC;YAAE,GAAG,EAAE,MAAM,CAAC;YAAC,IAAI,EAAE,MAAM,CAAA;SAAE,CAAC,CAAC;QAC7C,UAAU,EAAE,MAAM,CAAC;QACnB,cAAc,EAAE,MAAM,CAAC;KACxB,CAAC;IACF,cAAc,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;IACzC,MAAM,CAAC,EAAE;QAAE,MAAM,EAAE,OAAO,CAAC;QAAC,EAAE,CAAC,EAAE,OAAO,CAAC;QAAC,OAAO,CAAC,EAAE,MAAM,CAAC;QAAC,KAAK,CAAC,EAAE,MAAM,CAAC;QAAC,WAAW,CAAC,EAAE,MAAM,EAAE,CAAC;QAAC,MAAM,CAAC,EAAE,MAAM,CAAA;KAAE,CAAC;IACtH,KAAK,CAAC,EAAE;QAAE,OAAO,EAAE,OAAO,CAAC;QAAC,IAAI,EAAE,MAAM,CAAA;KAAE,CAAC;IAC3C,KAAK,EAAE;QAAE,cAAc,EAAE,OAAO,CAAC;QAAC,IAAI,EAAE,MAAM,CAAC;QAAC,KAAK,CAAC,EAAE;YAAE,MAAM,CAAC,EAAE,MAAM,CAAC;YAAC,OAAO,CAAC,EAAE,MAAM,CAAC;YAAC,QAAQ,CAAC,EAAE,MAAM,CAAC;YAAC,cAAc,CAAC,EAAE,MAAM,CAAC;YAAC,KAAK,CAAC,EAAE,MAAM,CAAA;SAAE,CAAA;KAAE,CAAC;CAC7J;AAYD,wBAAsB,SAAS,CAAC,KAAK,EAAE;IAAE,KAAK,CAAC,EAAE,OAAO,CAAC;IAAC,KAAK,CAAC,EAAE,MAAM,CAAA;CAAE,GAAG,OAAO,CAAC,UAAU,CAAC,CAkE/F;AAyDD,wBAAsB,IAAI,CAAC,IAAI,WAAwB,GAAG,OAAO,CAAC,IAAI,CAAC,CA0HtE;AAED,sGAAsG;AACtG,wBAAgB,WAAW,CAAC,GAAG,EAAE,MAAM,GAAG;IAAE,MAAM,EAAE,MAAM,CAAC;IAAC,QAAQ,CAAC,EAAE,MAAM,CAAC;IAAC,QAAQ,CAAC,EAAE,MAAM,CAAA;CAAE,CAKjG"}
1
+ {"version":3,"file":"cli-commands.d.ts","sourceRoot":"","sources":["../src/cli-commands.ts"],"names":[],"mappings":"AAkBA,OAAO,EAcL,KAAK,YAAY,EAClB,MAAM,YAAY,CAAC;AAKpB,OAAO,EAAsC,KAAK,IAAI,EAAmB,MAAM,YAAY,CAAC;AAU5F,eAAO,MAAM,KAAK,QA0BsC,CAAC;AAEzD,qGAAqG;AACrG,eAAO,MAAM,WAAW,i4CAuBkE,CAAC;AAW3F,oDAAoD;AACpD,MAAM,WAAW,UAAU;IACzB,GAAG,EAAE;QAAE,OAAO,EAAE,MAAM,CAAC;QAAC,IAAI,EAAE,MAAM,CAAC;QAAC,QAAQ,EAAE,MAAM,CAAA;KAAE,CAAC;IACzD,OAAO,EAAE;QAAE,MAAM,EAAE,MAAM,CAAC;QAAC,GAAG,CAAC,EAAE,MAAM,CAAC;QAAC,KAAK,CAAC,EAAE,YAAY,CAAA;KAAE,CAAC;IAChE,MAAM,EAAE;QACN,MAAM,EAAE,kBAAkB,GAAG,OAAO,GAAG,MAAM,CAAC;QAC9C,IAAI,CAAC,EAAE,MAAM,CAAC;QACd,GAAG,CAAC,EAAE,MAAM,CAAC;QACb,MAAM,EAAE,KAAK,CAAC;YAAE,GAAG,EAAE,MAAM,CAAC;YAAC,IAAI,EAAE,MAAM,CAAA;SAAE,CAAC,CAAC;QAC7C,UAAU,EAAE,MAAM,CAAC;QACnB,cAAc,EAAE,MAAM,CAAC;KACxB,CAAC;IACF,cAAc,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;IACzC,MAAM,CAAC,EAAE;QAAE,MAAM,EAAE,OAAO,CAAC;QAAC,EAAE,CAAC,EAAE,OAAO,CAAC;QAAC,OAAO,CAAC,EAAE,MAAM,CAAC;QAAC,KAAK,CAAC,EAAE,MAAM,CAAC;QAAC,WAAW,CAAC,EAAE,MAAM,EAAE,CAAC;QAAC,MAAM,CAAC,EAAE,MAAM,CAAA;KAAE,CAAC;IACtH,KAAK,CAAC,EAAE;QAAE,OAAO,EAAE,OAAO,CAAC;QAAC,IAAI,EAAE,MAAM,CAAA;KAAE,CAAC;IAC3C,KAAK,EAAE;QAAE,cAAc,EAAE,OAAO,CAAC;QAAC,IAAI,EAAE,MAAM,CAAC;QAAC,KAAK,CAAC,EAAE;YAAE,MAAM,CAAC,EAAE,MAAM,CAAC;YAAC,OAAO,CAAC,EAAE,MAAM,CAAC;YAAC,QAAQ,CAAC,EAAE,MAAM,CAAC;YAAC,cAAc,CAAC,EAAE,MAAM,CAAC;YAAC,KAAK,CAAC,EAAE,MAAM,CAAA;SAAE,CAAA;KAAE,CAAC;CAC7J;AAYD,wBAAsB,SAAS,CAAC,KAAK,EAAE;IAAE,KAAK,CAAC,EAAE,OAAO,CAAC;IAAC,KAAK,CAAC,EAAE,MAAM,CAAA;CAAE,GAAG,OAAO,CAAC,UAAU,CAAC,CAmE/F;AAyDD,wBAAsB,IAAI,CAAC,IAAI,WAAwB,GAAG,OAAO,CAAC,IAAI,CAAC,CAgItE;AAuBD,sDAAsD;AACtD,wBAAgB,SAAS,CAAC,GAAG,EAAE,IAAI,GAAG,MAAM,EAAE,CAW7C;AAoBD;iEACiE;AACjE,wBAAgB,YAAY,CAAC,UAAU,CAAC,EAAE,MAAM,EAAE,EAAE,UAAU,CAAC,EAAE,MAAM,EAAE,GAAG,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAQlG;AAYD;uEACuE;AACvE,wBAAgB,eAAe,CAAC,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,IAAI,CAAC,GAAG,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAUhF;AAsBD,oGAAoG;AACpG,wBAAsB,SAAS,CAAC,IAAI,EAAE,MAAM,EAAE,GAAG,OAAO,CAAC,MAAM,CAAC,CAY/D;AAwKD,sGAAsG;AACtG,wBAAgB,WAAW,CAAC,GAAG,EAAE,MAAM,GAAG;IAAE,MAAM,EAAE,MAAM,CAAC;IAAC,QAAQ,CAAC,EAAE,MAAM,CAAC;IAAC,QAAQ,CAAC,EAAE,MAAM,CAAA;CAAE,CAKjG"}
@@ -7,6 +7,7 @@
7
7
  // clearcote login [key] save a licence key (validated first)
8
8
  // clearcote logout remove the saved key
9
9
  // clearcote serve [--port 9222] [--idle-timeout 300] … multi-identity CDP endpoint
10
+ // clearcote cloud run|sessions|stop|events|recording|profile sync|webhooks … the hosted API
10
11
  //
11
12
  // `info` never downloads: it reports what is already cached and what a launch would resolve to.
12
13
  import { parseArgs } from "node:util";
@@ -19,6 +20,7 @@ import { defaultCacheRoot, listCachedBuilds } from "./download.js";
19
20
  import { geoCacheRoot } from "./geoip.js";
20
21
  import { GATED_ENGINE_SWITCHES } from "./launchopts.js";
21
22
  import { toProxySpec } from "./net.js";
23
+ import { Cloud, CloudError, announceHandoff } from "./cloud.js";
22
24
  const SDK_VERSION = (() => {
23
25
  try {
24
26
  return JSON.parse(readFileSync(new URL("../package.json", import.meta.url), "utf8")).version;
@@ -40,6 +42,10 @@ USAGE
40
42
  [--max-browsers 16] [--allow-origin <origin>]... [--allow-host <name>]... [--headed]
41
43
  [--fingerprint <seed>] [--platform <os>] [--proxy <url>] [--timezone <tz>]
42
44
  [--accept-language <l>] [--geoip]
45
+ clearcote cloud run <task> [--url <url>] [--schema <file.json>] [--secret <name>=<value>]... [--json]
46
+ clearcote cloud sessions | stop <id> | events <id> | recording <id> [-o <file.mp4>]
47
+ clearcote cloud profile sync <name> (--from-profile <dir> | --from-cdp <url> | --from-file <file> | --login <url>)
48
+ clearcote cloud webhooks add <url> | list | rm <id> | test <id> (all flags: clearcote cloud --help)
43
49
 
44
50
  INFO FLAGS
45
51
  --quick skip everything that needs the network or a launch (seat count, launch test)
@@ -48,7 +54,33 @@ INFO FLAGS
48
54
 
49
55
  ENVIRONMENT
50
56
  CLEARCOTE_LICENSE_KEY, CLEARCOTE_RELEASE_CHANNEL, CLEARCOTE_GEOIP_TIMEOUT_SECONDS,
51
- CLEARCOTE_LICENSE_THROUGH_PROXY, CLEARCOTE_BINARY, CLEARCOTE_CACHE, CLEARCOTE_SERVE_IDLE_TIMEOUT`;
57
+ CLEARCOTE_LICENSE_THROUGH_PROXY, CLEARCOTE_BINARY, CLEARCOTE_CACHE, CLEARCOTE_SERVE_IDLE_TIMEOUT,
58
+ CLEARCOTE_API_KEY, CLEARCOTE_API_URL, CLEARCOTE_CLOUD`;
59
+ /** `clearcote cloud --help`. Kept byte-identical to CLOUD_USAGE in the Python SDK's _commands.py. */
60
+ export const CLOUD_USAGE = `clearcote cloud -- the hosted Clearcote API from the command line.
61
+
62
+ USAGE
63
+ clearcote cloud run <task> [--url <url>] [--schema <file.json>] [--secret <name>=<value>]...
64
+ [--secret-domain <name>=<host>]... [--handoff] [--record] [--json]
65
+ [--country <cc>] [--state <s>] [--city <c>] [--proxy managed|<url>]
66
+ [--profile <name> [--persist-profile]] [--fingerprint <seed>]
67
+ [--timeout-sec <n>] [--max-steps <n>] [--note <text>]
68
+ clearcote cloud sessions [--json]
69
+ clearcote cloud stop <id> [--json]
70
+ clearcote cloud events <id> [--json]
71
+ clearcote cloud recording <id> [-o <file.mp4>] [--json]
72
+ clearcote cloud profile sync <name> (--from-profile <dir> | --from-cdp <url> | --from-file <state.json>
73
+ | --login <url>) (--domain <domain>... | --all-domains) [--replace] [--json]
74
+ clearcote cloud webhooks add <url> [--event <type>]... [--json]
75
+ clearcote cloud webhooks list [--json]
76
+ clearcote cloud webhooks rm <id> [--json]
77
+ clearcote cloud webhooks test <id> [--json]
78
+
79
+ run waits for the result and exits 0 only when the run succeeded. profile sync uploads only the
80
+ cookies of the --domain names you list (subdomains included); --all-domains uploads every cookie.
81
+
82
+ ENVIRONMENT
83
+ CLEARCOTE_API_KEY (required), CLEARCOTE_API_URL (default https://www.clearcotelabs.com)`;
52
84
  function out(line = "") {
53
85
  process.stdout.write(line + "\n");
54
86
  }
@@ -111,7 +143,8 @@ export async function buildInfo(flags) {
111
143
  }
112
144
  else {
113
145
  try {
114
- const b = await launch({ executablePath: pick.path, headless: true, quiet: true, ephemeralProfile: false });
146
+ // cloud: false — this tests the LOCAL install, whatever CLEARCOTE_CLOUD says.
147
+ const b = await launch({ executablePath: pick.path, headless: true, quiet: true, ephemeralProfile: false, cloud: false });
115
148
  const version = b.version();
116
149
  await b.close();
117
150
  report.launch = { tested: true, ok: true, version };
@@ -197,6 +230,12 @@ export async function main(argv = process.argv.slice(2)) {
197
230
  out(SDK_VERSION);
198
231
  return;
199
232
  }
233
+ if (cmd === "cloud") {
234
+ const code = await cloudMain(rest);
235
+ if (code)
236
+ process.exitCode = code;
237
+ return;
238
+ }
200
239
  if (cmd === "info" || cmd === "doctor") {
201
240
  const { values } = parseArgs({ args: rest, options: { quick: { type: "boolean" }, "no-launch": { type: "boolean" }, json: { type: "boolean" }, proxy: { type: "string" } } });
202
241
  const report = await buildInfo({ quick: !!(values.quick || values["no-launch"]), proxy: values.proxy });
@@ -317,6 +356,308 @@ export async function main(argv = process.argv.slice(2)) {
317
356
  }
318
357
  fail(`unknown command '${cmd}'. Run \`clearcote --help\`.`, 2);
319
358
  }
359
+ // ── clearcote cloud ────────────────────────────────────────────────────────────────────────────────
360
+ // Output is part of the contract: the Python SDK's tests/test_parity_cloud_cli.py runs this CLI and
361
+ // the Python one against the same fake API and compares stdout byte for byte. Change a line here,
362
+ // change it in _commands.py too.
363
+ /** A usage error inside `clearcote cloud`: message already on stderr, exit with `code`. */
364
+ class CloudCliExit extends Error {
365
+ code;
366
+ constructor(code) {
367
+ super(`exit ${code}`);
368
+ this.code = code;
369
+ }
370
+ }
371
+ function cloudFail(msg, code = 1) {
372
+ process.stderr.write(`clearcote: ${msg}\n`);
373
+ throw new CloudCliExit(code);
374
+ }
375
+ const dump = (obj) => out(JSON.stringify(obj, null, 2));
376
+ const str = (v) => (v === null || v === undefined ? "-" : String(v));
377
+ const eur = (v, places) => (typeof v === "number" ? `€${v.toFixed(places)}` : "-");
378
+ /** The human summary `clearcote cloud run` prints. */
379
+ export function formatRun(run) {
380
+ const lines = [`run ${str(run?.id)} ${str(run?.status)}`];
381
+ const r = run?.result ?? {};
382
+ if (r.status)
383
+ lines.push(`result ${r.status}${r.detail ? `: ${r.detail}` : ""}`);
384
+ if (r.url)
385
+ lines.push(`url ${r.url}`);
386
+ if (r.title)
387
+ lines.push(`title ${r.title}`);
388
+ if (r.outputError)
389
+ lines.push(`output error: ${r.outputError}`);
390
+ else if (r.output !== undefined && r.output !== null)
391
+ lines.push(`output ${JSON.stringify(r.output, null, 2)}`);
392
+ const cost = run?.costEur;
393
+ if (cost && typeof cost === "object" && cost.total !== undefined && cost.total !== null)
394
+ lines.push(`cost ${eur(cost.total, 4)}`);
395
+ return lines;
396
+ }
397
+ function sessionLine(s) {
398
+ const line = `${str(s?.id)} ${str(s?.status)} ${str(s?.createdAt)} ${eur(s?.costEur, 4)}`;
399
+ return line + (s?.note ? ` ${s.note}` : "");
400
+ }
401
+ // A --secret argument is never echoed back, not even a malformed one: it may be the bare secret.
402
+ const SECRET_SHAPE = "--secret wants <name>=<value> (the argument given is not shown: it may hold the secret)";
403
+ function parsePairs(values, flag) {
404
+ return (values ?? []).map((raw) => {
405
+ const i = raw.indexOf("=");
406
+ const name = i < 0 ? "" : raw.slice(0, i).trim();
407
+ const value = i < 0 ? "" : raw.slice(i + 1);
408
+ if (!name || !value)
409
+ cloudFail(flag === "--secret" ? SECRET_SHAPE : `${flag} wants <name>=<value>, got '${raw}'`, 2);
410
+ return [name, value];
411
+ });
412
+ }
413
+ /** --secret name=value and --secret-domain name=host as the API's `secrets` object: a plain string,
414
+ * or { value, domains } once a domain is given for that name. */
415
+ export function cloudSecrets(secretArgs, domainArgs) {
416
+ const secrets = new Map(parsePairs(secretArgs, "--secret"));
417
+ const domains = new Map();
418
+ for (const [name, host] of parsePairs(domainArgs, "--secret-domain")) {
419
+ if (!secrets.has(name))
420
+ cloudFail(`--secret-domain ${name}: there is no --secret ${name}=...`, 2);
421
+ domains.set(name, [...(domains.get(name) ?? []), host.trim().toLowerCase()]);
422
+ }
423
+ return Object.fromEntries([...secrets].map(([n, v]) => [n, domains.has(n) ? { value: v, domains: domains.get(n) } : v]));
424
+ }
425
+ // `clearcote cloud run` browser/run options that take a value, mapped to the Runs.create option of the
426
+ // same meaning (the Python launch kwargs): --timeout-sec -> timeoutSec, --max-steps -> maxSteps, ...
427
+ const RUN_OPTION_FLAGS = ["country", "state", "city", "proxy", "profile", "fingerprint", "timeout-sec", "max-steps", "note"];
428
+ function whole(value, flag) {
429
+ if (value === undefined)
430
+ return undefined;
431
+ if (!/^[0-9]+$/.test(value))
432
+ cloudFail(`${flag} wants a whole number, got '${value}'`, 2);
433
+ return Number(value);
434
+ }
435
+ /** The browser and run options of `clearcote cloud run` (unset ones left out). --profile NAME loads a
436
+ * cloud profile; with --persist-profile the run also saves it back. */
437
+ export function cloudRunOptions(v) {
438
+ if (v["persist-profile"] && !v.profile)
439
+ cloudFail("--persist-profile needs --profile <name>", 2);
440
+ const opts = {
441
+ country: v.country, state: v.state, city: v.city, proxy: v.proxy,
442
+ profile: v.profile ? (v["persist-profile"] ? { name: v.profile, persist: true } : v.profile) : undefined,
443
+ fingerprint: v.fingerprint, timeoutSec: whole(v["timeout-sec"], "--timeout-sec"),
444
+ maxSteps: whole(v["max-steps"], "--max-steps"), note: v.note,
445
+ handoff: v.handoff ? true : undefined, record: v.record ? true : undefined,
446
+ };
447
+ return Object.fromEntries(Object.entries(opts).filter(([, x]) => x !== undefined));
448
+ }
449
+ function runProgress(view) {
450
+ if (view?.status === "waiting_for_human")
451
+ announceHandoff(view);
452
+ else
453
+ process.stderr.write(`[clearcote] run ${view?.id}: ${view?.status}\n`);
454
+ }
455
+ function parse(args, options, positionals, what) {
456
+ let r;
457
+ try {
458
+ r = parseArgs({ args, options: { ...options, json: { type: "boolean" } }, allowPositionals: true, strict: true });
459
+ }
460
+ catch (e) {
461
+ cloudFail(e.message, 2);
462
+ }
463
+ if (positionals >= 0 ? r.positionals.length !== positionals : r.positionals.length < 1) {
464
+ cloudFail(`\`clearcote cloud ${what}\`: wrong arguments. Run \`clearcote cloud --help\`.`, 2);
465
+ }
466
+ return { values: r.values, pos: r.positionals };
467
+ }
468
+ /** `clearcote cloud …`: resolves to the exit code (0 ok, 1 failed, 2 usage). Never exits itself. */
469
+ export async function cloudMain(argv) {
470
+ try {
471
+ return await cloudCommand(argv);
472
+ }
473
+ catch (e) {
474
+ if (e instanceof CloudCliExit)
475
+ return e.code;
476
+ if (e instanceof CloudError) {
477
+ process.stderr.write(`clearcote: ${e.message}${e.status ? ` (HTTP ${e.status}${e.code ? `, ${e.code}` : ""})` : ""}\n`);
478
+ return 1;
479
+ }
480
+ process.stderr.write(`clearcote: ${e?.message ?? String(e)}\n`);
481
+ return 1;
482
+ }
483
+ }
484
+ async function cloudCommand(argv) {
485
+ const [cmd, ...rest] = argv;
486
+ if (!cmd || cmd === "-h" || cmd === "--help" || cmd === "help") {
487
+ out(CLOUD_USAGE);
488
+ return 0;
489
+ }
490
+ if (!["run", "sessions", "stop", "events", "recording", "profile", "webhooks"].includes(cmd)) {
491
+ cloudFail(`unknown cloud command '${cmd}'. Run \`clearcote cloud --help\`.`, 2);
492
+ }
493
+ if ((cmd === "profile" || cmd === "webhooks") && !rest[0]) {
494
+ cloudFail(`\`clearcote cloud ${cmd}\` needs a subcommand. Run \`clearcote cloud --help\`.`, 2);
495
+ }
496
+ if (cmd === "run") {
497
+ const { values: v, pos } = parse(rest, {
498
+ url: { type: "string" }, schema: { type: "string" }, secret: { type: "string", multiple: true },
499
+ "secret-domain": { type: "string", multiple: true }, handoff: { type: "boolean" }, record: { type: "boolean" },
500
+ ...Object.fromEntries(RUN_OPTION_FLAGS.map((f) => [f, { type: "string" }])),
501
+ "persist-profile": { type: "boolean" },
502
+ }, -1, "run");
503
+ const secrets = cloudSecrets(v.secret, v["secret-domain"]);
504
+ const runOptions = cloudRunOptions(v);
505
+ let schema;
506
+ if (v.schema) {
507
+ try {
508
+ schema = JSON.parse(readFileSync(v.schema, "utf8"));
509
+ }
510
+ catch (e) {
511
+ cloudFail(`--schema ${v.schema}: ${e.message}`, 2);
512
+ }
513
+ }
514
+ const cloud = new Cloud();
515
+ const run = await cloud.runs.create(pos.join(" "), {
516
+ url: v.url, schema, secrets: Object.keys(secrets).length ? secrets : undefined,
517
+ ...runOptions, onUpdate: runProgress,
518
+ });
519
+ if (v.json)
520
+ dump(run);
521
+ else
522
+ formatRun(run).forEach((l) => out(l));
523
+ return run?.status === "succeeded" ? 0 : 1;
524
+ }
525
+ if (cmd === "sessions") {
526
+ const { values: v } = parse(rest, {}, 0, "sessions");
527
+ const data = (await new Cloud().browsers.list()) ?? {};
528
+ if (v.json) {
529
+ dump(data);
530
+ return 0;
531
+ }
532
+ const sessions = data.sessions ?? [];
533
+ for (const s of sessions)
534
+ out(sessionLine(s));
535
+ if (!sessions.length)
536
+ out("no sessions");
537
+ if (data.balanceEur !== undefined && data.balanceEur !== null)
538
+ out(`balance ${eur(data.balanceEur, 2)}`);
539
+ return 0;
540
+ }
541
+ if (cmd === "stop") {
542
+ const { values: v, pos } = parse(rest, {}, 1, "stop");
543
+ const view = await new Cloud().browsers.stop(pos[0]);
544
+ if (v.json)
545
+ dump(view);
546
+ else
547
+ out(`stopped ${pos[0]} (${str(view?.status)})`);
548
+ return 0;
549
+ }
550
+ if (cmd === "events") {
551
+ const { values: v, pos } = parse(rest, {}, 1, "events");
552
+ const cloud = new Cloud();
553
+ const events = [];
554
+ let after = 0;
555
+ for (;;) {
556
+ const page = (await cloud.browsers.events(pos[0], { after })) ?? {};
557
+ const batch = page.events ?? [];
558
+ events.push(...batch);
559
+ const next = page.next;
560
+ if (!batch.length || next === null || next === undefined || next === after)
561
+ break;
562
+ after = next;
563
+ }
564
+ if (v.json) {
565
+ dump({ events });
566
+ return 0;
567
+ }
568
+ for (const e of events) {
569
+ const data = e?.data;
570
+ const extra = data && Object.keys(data).length ? ` ${JSON.stringify(data)}` : "";
571
+ out(`${str(e?.seq)} ${str(e?.at)} ${str(e?.type)}${extra}`);
572
+ }
573
+ if (!events.length)
574
+ out("no events");
575
+ return 0;
576
+ }
577
+ if (cmd === "recording") {
578
+ const { values: v, pos } = parse(rest, { output: { type: "string", short: "o" } }, 1, "recording");
579
+ const path = v.output || `${pos[0]}.mp4`;
580
+ await new Cloud().browsers.downloadRecording(pos[0], path);
581
+ const size = statSync(path).size;
582
+ if (v.json)
583
+ dump({ path, bytes: size });
584
+ else
585
+ out(`saved ${path} (${size} bytes)`);
586
+ return 0;
587
+ }
588
+ if (cmd === "profile") {
589
+ if (rest[0] !== "sync")
590
+ cloudFail(`unknown profile command '${rest[0]}'. Run \`clearcote cloud --help\`.`, 2);
591
+ const { values: v, pos } = parse(rest.slice(1), {
592
+ "from-profile": { type: "string" }, "from-cdp": { type: "string" }, "from-file": { type: "string" },
593
+ login: { type: "string" }, domain: { type: "string", multiple: true }, "all-domains": { type: "boolean" },
594
+ replace: { type: "boolean" },
595
+ }, 1, "profile sync");
596
+ const sources = [v["from-profile"], v["from-cdp"], v["from-file"], v.login].filter(Boolean);
597
+ if (sources.length !== 1)
598
+ cloudFail("profile sync needs exactly one of --from-profile, --from-cdp, --from-file or --login", 2);
599
+ if (v.domain?.length && v["all-domains"])
600
+ cloudFail("pass --domain or --all-domains, not both", 2);
601
+ if (!v.domain?.length && !v["all-domains"]) {
602
+ cloudFail("refusing to upload every cookie: pass --domain <domain> (repeatable, subdomains included) or --all-domains", 2);
603
+ }
604
+ const res = (await new Cloud().profiles.sync(pos[0], {
605
+ fromProfile: v["from-profile"], fromCdp: v["from-cdp"], fromFile: v["from-file"], loginUrl: v.login,
606
+ domains: v.domain, allDomains: !!v["all-domains"], replace: !!v.replace,
607
+ })) ?? {};
608
+ if (v.json) {
609
+ dump(res);
610
+ return 0;
611
+ }
612
+ out(`imported ${str(res.imported)} cookies into profile ${res.name || pos[0]} (${str(res.cookies)} in total)`);
613
+ if (res.domains?.length)
614
+ out(`domains ${res.domains.join(", ")}`);
615
+ return 0;
616
+ }
617
+ // webhooks
618
+ const action = rest[0];
619
+ if (action === "add") {
620
+ const { values: v, pos } = parse(rest.slice(1), { event: { type: "string", multiple: true } }, 1, "webhooks add");
621
+ const res = (await new Cloud().webhooks.create(pos[0], { events: v.event })) ?? {};
622
+ if (v.json) {
623
+ dump(res);
624
+ return 0;
625
+ }
626
+ out(`webhook ${str(res.id)} -> ${str(res.url)}`);
627
+ out(`events ${(res.events ?? []).join(", ") || "all except ping"}`);
628
+ out(`secret ${str(res.secret)}`);
629
+ out("store the secret now: it is not shown again");
630
+ return 0;
631
+ }
632
+ if (action === "list") {
633
+ const { values: v } = parse(rest.slice(1), {}, 0, "webhooks list");
634
+ const data = (await new Cloud().webhooks.list()) ?? {};
635
+ if (v.json) {
636
+ dump(data);
637
+ return 0;
638
+ }
639
+ const hooks = data.webhooks ?? [];
640
+ for (const h of hooks) {
641
+ const ld = h?.lastDelivery;
642
+ const last = ld ? ` last ${str(ld.status)} ${str(ld.at)}` : "";
643
+ out(`${str(h?.id)} ${str(h?.url)} ${(h?.events ?? []).join(",") || "all"}${last}`);
644
+ }
645
+ if (!hooks.length)
646
+ out("no webhooks");
647
+ return 0;
648
+ }
649
+ if (action === "rm" || action === "test") {
650
+ const { values: v, pos } = parse(rest.slice(1), {}, 1, `webhooks ${action}`);
651
+ const cloud = new Cloud();
652
+ const res = action === "rm" ? await cloud.webhooks.delete(pos[0]) : await cloud.webhooks.test(pos[0]);
653
+ if (v.json)
654
+ dump(res);
655
+ else
656
+ out(action === "rm" ? `removed ${pos[0]}` : `sent a ping to ${pos[0]}`);
657
+ return 0;
658
+ }
659
+ return cloudFail(`unknown webhooks command '${action}'. Run \`clearcote cloud --help\`.`, 2);
660
+ }
320
661
  /** `--proxy scheme://user:pass@host:port` as a launch proxy option with the credentials split out. */
321
662
  export function proxyOption(url) {
322
663
  let spec;