onebrowser 0.4.0 → 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +22 -20
- package/package.json +1 -1
- package/src/cli.js +1 -1
- package/src/cost.js +8 -6
- package/src/index.js +85 -96
- package/src/ledger.js +43 -48
- package/src/providers/anchor.js +1 -0
- package/src/providers/index.js +22 -25
- package/src/session.js +23 -35
package/README.md
CHANGED
|
@@ -15,14 +15,15 @@ const page = await session.newPage(); // Playwright; or any CDP client on sessio
|
|
|
15
15
|
await page.goto('https://example.com');
|
|
16
16
|
await session.close();
|
|
17
17
|
|
|
18
|
-
await browsers.
|
|
19
|
-
browsers.
|
|
20
|
-
|
|
18
|
+
await browsers.readQuota('steel'); // { remaining, unbilled, unit: 'USD', renewsAt: null }
|
|
19
|
+
browsers.quota('steel'); // the same now, from that reading: sessions opened since count at once
|
|
20
|
+
browsers.maxSessionCost('steel'); // what a session with the default options costs at most, in USD
|
|
21
|
+
await browsers.syncUsage(); // stores the providers' records of ended sessions in the ledger
|
|
21
22
|
browsers.usage({ since: Date.parse('2026-11-01') }); // [{ day, provider, client, sessions, minutes, … }]
|
|
22
23
|
await browsers.close();
|
|
23
24
|
```
|
|
24
25
|
|
|
25
|
-
|
|
26
|
+
Choosing a provider, capping sessions and stopping before a quota runs out are the program's policy. onebrowser opens the sessions it is asked for and gives the program what it needs to decide: what the quota has left, what a session costs at most, and `QuotaError` when a provider refuses a session for the account's quota.
|
|
26
27
|
|
|
27
28
|
Built in: [Steel](https://steel.dev), [TinyFish](https://tinyfish.ai), [Hyperbrowser](https://hyperbrowser.ai), [Lightpanda](https://lightpanda.io), [Kernel](https://www.kernel.sh), [Anchor Browser](https://anchorbrowser.io) and [Firecrawl](https://www.firecrawl.dev); a program can add its own (see [Adding a provider](#adding-a-provider)). Node 22.13 or later.
|
|
28
29
|
|
|
@@ -34,25 +35,26 @@ Built in: [Steel](https://steel.dev), [TinyFish](https://tinyfish.ai), [Hyperbro
|
|
|
34
35
|
|---|---|
|
|
35
36
|
| `client` | The program using it, recorded with each session; required |
|
|
36
37
|
| `ledgerPath` | SQLite file of the ledger (default below; `':memory:'` for tests). Opened when first needed |
|
|
37
|
-
| `providers` | Settings per provider name, on top of its adapter's `defaults`: `apiKey`, `price
|
|
38
|
+
| `providers` | Settings per provider name, on top of its adapter's `defaults`: `apiKey`, `price`, `billingDelayMs`, `minIdleTimeoutSec` (see [Quota](#quota)) and the adapter's own (API base URLs, plan facts such as `monthlyHours`) |
|
|
38
39
|
| `adapters` | Further adapters by provider name, or replacements of built-in ones |
|
|
39
40
|
|
|
40
41
|
| Method | |
|
|
41
42
|
|---|---|
|
|
42
43
|
| `providers` | Names of the providers it can open sessions with |
|
|
43
44
|
| `checkReady(provider)` | Throws when the provider is unknown or has no API key |
|
|
44
|
-
| `openSession({ provider, proxyCountry, idleTimeoutSec, maxDurationMs })` | A new session,
|
|
45
|
-
| `
|
|
46
|
-
| `
|
|
47
|
-
| `
|
|
45
|
+
| `openSession({ provider, proxyCountry, idleTimeoutSec, maxDurationMs })` | A new session, in the ledger from when the provider has created it. Throws `QuotaError` when the provider refuses it for the account's quota |
|
|
46
|
+
| `readQuota(provider, { maxAgeMs })` | Reads the provider's quota, unless a reading in progress or started at most `maxAgeMs` ago (default 0) can be used, a failed one too; then answers as `quota()` |
|
|
47
|
+
| `quota(provider)` | What is left now, from the last reading: `{ remaining, unbilled, unit, renewsAt }` (see [Quota](#quota)). Throws until the provider has been read, and while its last reading failed |
|
|
48
|
+
| `maxSessionCost(provider, options)` | What a session with these options costs at most, in the unit of the provider's quota |
|
|
49
|
+
| `syncUsage()` | Stores the providers' records (duration, proxy traffic) of ended sessions: `{ reported, failed: [{ provider, error }] }` |
|
|
48
50
|
| `usage({ since })` | Sessions opened since `since` (ms) per UTC day, provider and client: `sessions`, `minutes` (the provider's record, else the open-to-close timing), `untimed`, `unsynced`, `proxyBytes` |
|
|
49
51
|
| `close()` | Closes the sessions still open and the ledger |
|
|
50
52
|
|
|
51
|
-
A session
|
|
53
|
+
A session has `id`, `provider`, `egress`, `cdpUrl` (it may carry the API key: never log it), `openedAt` (when it was asked for), `closedAt` (when its provider released it), and:
|
|
52
54
|
|
|
53
|
-
- `connect()`: Playwright's `Browser` over its CDP connection, made once. A keep-alive command every 15 s keeps the provider's idle timeout from ending a session
|
|
54
|
-
- `newPage()`:
|
|
55
|
-
- `close()`: disconnects and releases it;
|
|
55
|
+
- `connect()`: Playwright's `Browser` over its CDP connection, made once. A keep-alive command every 15 s keeps the provider's idle timeout from ending a session in use.
|
|
56
|
+
- `newPage()`: a new page in the default context: the page a remote browser starts with cannot always be navigated.
|
|
57
|
+
- `close()`: disconnects and releases it; every call answers as the first. Throws when the provider could not release it: the provider ends it by itself, and the ledger counts it as lasting as long as it can.
|
|
56
58
|
|
|
57
59
|
A session and a `OneBrowser` are async-disposable: `await using session = await browsers.openSession({ provider: 'steel' })` closes the session at the end of its block (Node 24 or later; on Node 22 call `close()`).
|
|
58
60
|
|
|
@@ -61,7 +63,7 @@ A session and a `OneBrowser` are async-disposable: `await using session = await
|
|
|
61
63
|
| Option | Default | Meaning |
|
|
62
64
|
|---|---|---|
|
|
63
65
|
| `proxyCountry` | `null` | A country code: the provider's proxy in that country. `null`: its default egress |
|
|
64
|
-
| `idleTimeoutSec` | 60 | A session whose client is gone (its process was killed) ends this long after at the latest. At least 30 (twice the keep-alive interval) |
|
|
66
|
+
| `idleTimeoutSec` | 60 | A session whose client is gone (its process was killed) ends this long after at the latest, or after the provider's `minIdleTimeoutSec` if longer. At least 30 (twice the keep-alive interval) |
|
|
65
67
|
| `maxDurationMs` | 15 min | The session ends this long after it was opened at the latest: this process closes it then, and the provider's own limit is set to it where its API takes one |
|
|
66
68
|
|
|
67
69
|
`sessionOptions(options)` returns them with the defaults filled in and throws when one is unknown or out of range, so that a program can check its configuration before it opens any session.
|
|
@@ -80,9 +82,9 @@ How each provider meets them:
|
|
|
80
82
|
|
|
81
83
|
### Quota
|
|
82
84
|
|
|
83
|
-
`quota()`
|
|
85
|
+
`quota()` is the last reading less `unbilled`: what the ledger's sessions that the reading does not show cost at the provider's `price`. A reading shows the sessions that ended `billingDelayMs` before it; the others are open ones, counted as lasting as long as they can (`maxDurationMs`, then `idleTimeoutSec`, at least the provider's `minIdleTimeoutSec`), and those ended since, counted as long as they lasted. So each session counts once from when its provider has created it, whichever program on the machine opened it, and a program can check before each session that the account pays for it (`maxSessionCost`). A provider that bills a session while it runs has part of it counted twice, on the safe side. Not counted: sessions opened through another machine's ledger, until the provider bills them, and proxy traffic billed by the GB.
|
|
84
86
|
|
|
85
|
-
`price` is in the quota's unit: `perHour`, `perSession` (default 0),
|
|
87
|
+
`price` is in the quota's unit: `perHour`, `perSession` (default 0), the session time billed in whole `stepSec`, rounded up (default 1), and at least `minSec` (default 0). Where a provider does not document its rounding, the adapter takes the minute. `renewsAt` (ms) is when the quota renews by itself; `null` when only a top-up renews it. `price` and `billingDelayMs` are settings: a program on another plan sets its own.
|
|
86
88
|
|
|
87
89
|
| Provider | Quota, unit | Read from | `price` | `billingDelayMs` |
|
|
88
90
|
|---|---|---|---|---|
|
|
@@ -102,7 +104,7 @@ Every provider read from its API showed a session's cost within about a minute o
|
|
|
102
104
|
|
|
103
105
|
### The ledger
|
|
104
106
|
|
|
105
|
-
One SQLite file per machine by default: `$ONEBROWSER_DB`, else `onebrowser/sessions.db` in `$XDG_DATA_HOME` (default `~/.local/share`)
|
|
107
|
+
One SQLite file per machine by default: `$ONEBROWSER_DB`, else `onebrowser/sessions.db` in `$XDG_DATA_HOME` (default `~/.local/share`), shared by every program on the machine under its own `client`. A session is recorded when its provider has created it (it is billed from then on), with when it ends at the latest; closed when its provider has released it; and completed with the provider's own record by `syncUsage()`. A ledger made by an earlier onebrowser is brought up to date when opened; one made by a later onebrowser is refused.
|
|
106
108
|
|
|
107
109
|
## Command line
|
|
108
110
|
|
|
@@ -115,17 +117,17 @@ npx onebrowser usage --days 7 # the ledger's sessions per UTC day, provid
|
|
|
115
117
|
npx onebrowser usage --json --env-file .env
|
|
116
118
|
```
|
|
117
119
|
|
|
118
|
-
`usage` first fetches the providers' records of the sessions that have ended, as `syncUsage()` does. With `--json`, `quota` prints an array of what `
|
|
120
|
+
`usage` first fetches the providers' records of the sessions that have ended, as `syncUsage()` does. With `--json`, `quota` prints an array of what `readQuota()` returns, with the provider: `{ provider, remaining, unbilled, unit, renewsAt }`, or `{ provider, error }` for a failed reading; `usage` prints `{ ledger, since, sessions, failed }`, where `sessions` is what `usage()` returns and `failed` what `syncUsage()` could not fetch. API keys are read as by the library, from the environment (`--env-file PATH` loads a file into it first). Exit status: 0 done, 1 a reading failed, 2 the command line is wrong.
|
|
119
121
|
|
|
120
122
|
## Adding a provider
|
|
121
123
|
|
|
122
124
|
Write `src/providers/<name>.js` and register it in `ADAPTERS` (`src/providers/index.js`, where the contract is):
|
|
123
125
|
|
|
124
|
-
- `defaults`:
|
|
126
|
+
- `defaults`: `price`, `billingDelayMs`, `minIdleTimeoutSec` if the provider takes no idle timeout as short as asked (see [Quota](#quota)), and its own settings (API base URLs, plan facts).
|
|
125
127
|
- `createSession(settings, apiKey, { proxyCountry, idleTimeoutSec, maxDurationMs }) → { id, cdpUrl, egress }`: maps the session options onto the provider's parameters; throws `QuotaError` when the provider refuses the session for the account's quota.
|
|
126
128
|
- `releaseSession(settings, apiKey, session)`.
|
|
127
129
|
- `sessionUsage(settings, apiKey, [{ id, openedAt, closedAt }]) → [{ id, durationMs, proxyBytes }]`: the record of those sessions that have ended: the provider's own, or the ledger's timing for a provider whose records its API key cannot read.
|
|
128
|
-
- `quota(settings, apiKey, history) → { remaining, unit, renewsAt? }`: `history.endedSessionMsSince(ms)` is the session time of the
|
|
130
|
+
- `quota(settings, apiKey, history) → { remaining, unit, renewsAt? }`: for a provider that cannot report its usage, `history.endedSessionMsSince(ms)` is the session time of the ledger's sessions that the reading shows.
|
|
129
131
|
|
|
130
132
|
Requests go through `src/providers/http.js`: every status other than 2xx is an `HttpError` (with `status`), so an adapter handles only the statuses that mean something of their own, such as a quota refusal.
|
|
131
133
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "onebrowser",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.5.0",
|
|
4
4
|
"description": "One interface to remote browser providers: sessions by provider name, a usage ledger, account quotas",
|
|
5
5
|
"keywords": ["browser", "remote-browser", "cdp", "playwright", "steel", "hyperbrowser", "lightpanda", "kernel", "tinyfish", "anchorbrowser", "firecrawl"],
|
|
6
6
|
"license": "MIT",
|
package/src/cli.js
CHANGED
|
@@ -30,7 +30,7 @@ const COMMANDS = {
|
|
|
30
30
|
if (unknown.length) throw new CommandLineError(`unknown provider ${unknown.join(', ')} (expected ${browsers.providers.join(', ')})`);
|
|
31
31
|
const asked = names.length ? names : browsers.providers.filter((p) => isReady(browsers, p));
|
|
32
32
|
if (!asked.length) throw new Error(`no provider has an API key: set ${browsers.providers.map(apiKeyEnv).join(', ')}`);
|
|
33
|
-
const quotas = await Promise.all(asked.map((provider) => browsers.
|
|
33
|
+
const quotas = await Promise.all(asked.map((provider) => browsers.readQuota(provider).then((q) => ({ provider, ...q }), (e) => ({ provider, error: e.message }))));
|
|
34
34
|
if (json) out.write(`${JSON.stringify(quotas, null, 2)}\n`);
|
|
35
35
|
else {
|
|
36
36
|
for (const q of quotas) {
|
package/src/cost.js
CHANGED
|
@@ -1,20 +1,22 @@
|
|
|
1
1
|
/**
|
|
2
|
-
* A provider's price (its
|
|
2
|
+
* A provider's price (its `price` setting), in the unit its quota is read in:
|
|
3
3
|
* perHour for the session time
|
|
4
4
|
* perSession for each session, however short (default 0)
|
|
5
5
|
* stepSec the session time is billed in whole steps of this, rounded up (default 1)
|
|
6
6
|
* minSec and at least this much of it (default 0)
|
|
7
|
-
*
|
|
8
|
-
* price is used to bound what sessions cost, so it errs on the high side.
|
|
7
|
+
* It bounds what sessions cost, so where a provider does not document its rounding the adapter takes the minute.
|
|
9
8
|
*/
|
|
10
|
-
|
|
9
|
+
|
|
10
|
+
/** The price with its defaults filled in; throws when it is not one */
|
|
11
|
+
export function fullPrice(price) {
|
|
11
12
|
const { perHour, perSession = 0, stepSec = 1, minSec = 0, ...unknown } = price ?? {};
|
|
12
13
|
const fine = [perHour, perSession, minSec].every((x) => Number.isFinite(x) && x >= 0) && Number.isFinite(stepSec) && stepSec > 0;
|
|
13
14
|
if (!fine || Object.keys(unknown).length) throw new Error(`price must be { perHour, perSession, stepSec, minSec } of numbers ≥ 0 (stepSec > 0), not ${JSON.stringify(price)}`);
|
|
15
|
+
return { perHour, perSession, stepSec, minSec };
|
|
14
16
|
}
|
|
15
17
|
|
|
16
|
-
/** What a session that lasts durationMs costs at `price` */
|
|
17
|
-
export function sessionCost({ perHour, perSession
|
|
18
|
+
/** What a session that lasts durationMs costs at `price` (fullPrice) */
|
|
19
|
+
export function sessionCost({ perHour, perSession, stepSec, minSec }, durationMs) {
|
|
18
20
|
const billedSec = Math.max(minSec, Math.ceil(durationMs / 1000 / stepSec) * stepSec);
|
|
19
21
|
return perSession + (perHour * billedSec) / 3600;
|
|
20
22
|
}
|
package/src/index.js
CHANGED
|
@@ -2,7 +2,7 @@ import { sessionCost } from './cost.js';
|
|
|
2
2
|
import { readApiKey } from './credentials.js';
|
|
3
3
|
import { QuotaError } from './errors.js';
|
|
4
4
|
import { defaultLedgerPath, Ledger } from './ledger.js';
|
|
5
|
-
import { ADAPTERS, checkAdapter,
|
|
5
|
+
import { ADAPTERS, checkAdapter, fullSettings } from './providers/index.js';
|
|
6
6
|
import { KEEP_ALIVE_MS, Session } from './session.js';
|
|
7
7
|
|
|
8
8
|
export { apiKeyEnv } from './credentials.js';
|
|
@@ -10,18 +10,18 @@ export { QuotaError };
|
|
|
10
10
|
|
|
11
11
|
/** Session options when openSession() is not given them */
|
|
12
12
|
const SESSION_DEFAULTS = {
|
|
13
|
-
proxyCountry: null,
|
|
14
|
-
idleTimeoutSec: 60,
|
|
13
|
+
proxyCountry: null,
|
|
14
|
+
idleTimeoutSec: 60,
|
|
15
15
|
maxDurationMs: 15 * 60_000,
|
|
16
16
|
};
|
|
17
17
|
|
|
18
18
|
/**
|
|
19
|
-
* The session options of openSession() with the defaults filled in
|
|
20
|
-
*
|
|
19
|
+
* The session options of openSession() with the defaults filled in; throws when one is unknown or out of range,
|
|
20
|
+
* so that a program can check its configuration before it opens any session:
|
|
21
21
|
* proxyCountry country code: the provider's proxy there; null: its default egress
|
|
22
|
-
* idleTimeoutSec a session whose client is gone ends this long after at the latest
|
|
23
|
-
*
|
|
24
|
-
* maxDurationMs
|
|
22
|
+
* idleTimeoutSec a session whose client is gone ends this long after at the latest, or after the provider's
|
|
23
|
+
* minIdleTimeoutSec if longer (≥ 2 × KEEP_ALIVE_MS)
|
|
24
|
+
* maxDurationMs a session ends this long after it was opened at the latest
|
|
25
25
|
*/
|
|
26
26
|
export function sessionOptions(options = {}) {
|
|
27
27
|
const unknown = Object.keys(options).filter((k) => !Object.hasOwn(SESSION_DEFAULTS, k));
|
|
@@ -34,38 +34,24 @@ export function sessionOptions(options = {}) {
|
|
|
34
34
|
return o;
|
|
35
35
|
}
|
|
36
36
|
|
|
37
|
-
/**
|
|
38
|
-
|
|
39
|
-
* process ends it at maxDurationMs, and its provider idleTimeoutSec after its client left, should the process die
|
|
40
|
-
* or its release fail before
|
|
41
|
-
*/
|
|
42
|
-
const sessionMsAtMost = (o) => o.maxDurationMs + o.idleTimeoutSec * 1000;
|
|
37
|
+
/** How long a session lasts at most, whatever becomes of its process: closed at maxDurationMs, else ended by its provider once idle */
|
|
38
|
+
const sessionMsAtMost = (o, settings) => o.maxDurationMs + Math.max(o.idleTimeoutSec, settings.minIdleTimeoutSec) * 1000;
|
|
43
39
|
|
|
44
40
|
/**
|
|
45
|
-
* Remote browsers from several providers behind one interface: a session is asked for by provider name
|
|
46
|
-
*
|
|
47
|
-
*
|
|
48
|
-
*
|
|
49
|
-
* What is left of an account's quota is what its provider reads, less what the sessions it has not billed yet
|
|
50
|
-
* cost (quota()): the ledger knows those, and each provider's price (cost.js) what they cost. So a program can
|
|
51
|
-
* tell, before it opens a session, whether the account can pay for it.
|
|
41
|
+
* Remote browsers from several providers behind one interface: a session is asked for by provider name. Every
|
|
42
|
+
* session is recorded in a ledger (ledger.js) shared by the programs on the machine, so that each provider
|
|
43
|
+
* account's usage, and what its quota has left at any moment, are known. Which provider to use, and when to
|
|
44
|
+
* stop using one, are the caller's decisions.
|
|
52
45
|
*
|
|
53
|
-
*
|
|
54
|
-
*
|
|
55
|
-
*
|
|
56
|
-
* new OneBrowser({ client, ledgerPath, providers, adapters })
|
|
57
|
-
* client the program using it, recorded with each session (several programs may share one ledger)
|
|
58
|
-
* ledgerPath SQLite file of the ledger (default: defaultLedgerPath() in ledger.js; ':memory:' for tests),
|
|
59
|
-
* opened when first needed
|
|
60
|
-
* providers settings per provider name: apiKey (else from the environment, see credentials.js), and the
|
|
61
|
-
* adapter's own settings on top of its defaults (providers/<name>.js)
|
|
62
|
-
* adapters further adapters by provider name, or replacements of built-in ones (providers/index.js)
|
|
46
|
+
* client: the program using it, recorded with each session; ledgerPath: the ledger's SQLite file, opened when
|
|
47
|
+
* first needed; providers: settings per provider name on top of its adapter's defaults (providers/index.js),
|
|
48
|
+
* apiKey among them (else credentials.js); adapters: further or replacement adapters by provider name.
|
|
63
49
|
*/
|
|
64
50
|
export class OneBrowser {
|
|
65
|
-
#providers = new Map(); // name → { adapter, settings
|
|
51
|
+
#providers = new Map(); // name → { adapter, settings, apiKey }
|
|
66
52
|
#ledger = null;
|
|
67
53
|
#open = new Set();
|
|
68
|
-
#readings = new Map(); // provider → its
|
|
54
|
+
#readings = new Map(); // provider → { latest, done }: its latest quota reading, and the latest one done
|
|
69
55
|
|
|
70
56
|
constructor({ client, ledgerPath = defaultLedgerPath(), providers = {}, adapters = {} }) {
|
|
71
57
|
if (!client?.trim()) throw new Error('onebrowser: client must name the program using it');
|
|
@@ -74,15 +60,12 @@ export class OneBrowser {
|
|
|
74
60
|
for (const [name, adapter] of Object.entries({ ...ADAPTERS, ...adapters })) {
|
|
75
61
|
checkAdapter(name, adapter);
|
|
76
62
|
const { apiKey, ...given } = providers[name] ?? {};
|
|
77
|
-
|
|
78
|
-
checkSettings(name, settings);
|
|
79
|
-
this.#providers.set(name, { adapter, settings, apiKey });
|
|
63
|
+
this.#providers.set(name, { adapter, settings: fullSettings(name, { ...adapter.defaults, ...given }), apiKey });
|
|
80
64
|
}
|
|
81
65
|
const unknown = Object.keys(providers).filter((name) => !this.#providers.has(name));
|
|
82
66
|
if (unknown.length) throw new Error(`onebrowser: no adapter for provider ${unknown.join(', ')}`);
|
|
83
67
|
}
|
|
84
68
|
|
|
85
|
-
/** Names of the providers it can open sessions with */
|
|
86
69
|
get providers() {
|
|
87
70
|
return [...this.#providers.keys()];
|
|
88
71
|
}
|
|
@@ -91,14 +74,12 @@ export class OneBrowser {
|
|
|
91
74
|
return (this.#ledger ??= new Ledger(this.ledgerPath));
|
|
92
75
|
}
|
|
93
76
|
|
|
94
|
-
/** The adapter, settings and the API key given of `provider`; throws when it is unknown */
|
|
95
77
|
#known(provider) {
|
|
96
78
|
const p = this.#providers.get(provider);
|
|
97
79
|
if (!p) throw new Error(`unknown provider ${provider} (expected ${this.providers.join(', ')})`);
|
|
98
80
|
return p;
|
|
99
81
|
}
|
|
100
82
|
|
|
101
|
-
/** The adapter, settings and API key of `provider`; throws when it is unknown or has no API key */
|
|
102
83
|
#ready(provider) {
|
|
103
84
|
const p = this.#known(provider);
|
|
104
85
|
return { ...p, apiKey: readApiKey(provider, p.apiKey) };
|
|
@@ -110,14 +91,13 @@ export class OneBrowser {
|
|
|
110
91
|
}
|
|
111
92
|
|
|
112
93
|
/**
|
|
113
|
-
* A new session with `provider
|
|
114
|
-
*
|
|
115
|
-
* quota.
|
|
94
|
+
* A new session with `provider` (options: sessionOptions()), in the ledger from when its provider has created
|
|
95
|
+
* it. Throws when the provider creates none; QuotaError when it refuses for the account's quota.
|
|
116
96
|
*/
|
|
117
97
|
async openSession({ provider, ...options }) {
|
|
118
98
|
const { adapter, settings, apiKey } = this.#ready(provider);
|
|
119
99
|
const o = sessionOptions(options);
|
|
120
|
-
const openedAt = Date.now();
|
|
100
|
+
const openedAt = Date.now();
|
|
121
101
|
let remote;
|
|
122
102
|
try {
|
|
123
103
|
remote = await adapter.createSession(settings, apiKey, o);
|
|
@@ -125,79 +105,91 @@ export class OneBrowser {
|
|
|
125
105
|
if (e instanceof QuotaError) throw e;
|
|
126
106
|
throw new Error(`failed to create a ${provider} session: ${e.message}`);
|
|
127
107
|
}
|
|
128
|
-
const session = new Session({
|
|
129
|
-
provider, remote, openedAt, maxDurationMs: o.maxDurationMs,
|
|
130
|
-
release: async (s) => {
|
|
131
|
-
this.#open.delete(s);
|
|
132
|
-
this.#store.closed(s);
|
|
133
|
-
await adapter.releaseSession(settings, apiKey, s);
|
|
134
|
-
},
|
|
135
|
-
});
|
|
108
|
+
const session = new Session({ provider, remote, openedAt, maxDurationMs: o.maxDurationMs, release: (s) => this.#release(adapter, settings, apiKey, s) });
|
|
136
109
|
try {
|
|
137
|
-
this.#store.opened({ provider, id: session.id, client: this.client, openedAt, endsBy: openedAt + sessionMsAtMost(o) });
|
|
110
|
+
this.#store.opened({ provider, id: session.id, client: this.client, openedAt, endsBy: openedAt + sessionMsAtMost(o, settings) });
|
|
138
111
|
} catch (e) {
|
|
139
|
-
await session.close().catch(() => {}); // a session the ledger
|
|
112
|
+
await session.close().catch(() => {}); // a session the ledger does not know is not handed out
|
|
140
113
|
throw e;
|
|
141
114
|
}
|
|
142
115
|
this.#open.add(session);
|
|
143
116
|
return session;
|
|
144
117
|
}
|
|
145
118
|
|
|
119
|
+
/** Releases `session` at its provider and records it closed: when it was released, so its whole billed time counts */
|
|
120
|
+
async #release(adapter, settings, apiKey, session) {
|
|
121
|
+
try {
|
|
122
|
+
await adapter.releaseSession(settings, apiKey, session);
|
|
123
|
+
const closedAt = Date.now();
|
|
124
|
+
this.#store.closed({ provider: session.provider, id: session.id, closedAt });
|
|
125
|
+
return closedAt;
|
|
126
|
+
} finally {
|
|
127
|
+
this.#open.delete(session);
|
|
128
|
+
}
|
|
129
|
+
}
|
|
130
|
+
|
|
146
131
|
/**
|
|
147
|
-
*
|
|
148
|
-
*
|
|
149
|
-
* reading may not show yet cost, each at the provider's price (cost.js). Those are the sessions that had not
|
|
150
|
-
* ended billingDelayMs before the reading: open ones as if they lasted as long as they can, ended ones as long
|
|
151
|
-
* as they lasted. renewsAt (ms) when the quota renews by itself, null when only a top-up renews it.
|
|
152
|
-
*
|
|
153
|
-
* The provider is read anew unless a reading started at most maxAgeMs ago, or still in progress, can be used
|
|
154
|
-
* (a failed one too: it throws again); what is unbilled is worked out from the ledger every time, so it includes
|
|
155
|
-
* the sessions opened since, by any program sharing the ledger. Throws when the provider cannot be read.
|
|
132
|
+
* Reads the account's quota with `provider` from the provider, unless a reading in progress or started at most
|
|
133
|
+
* maxAgeMs ago can be used (a failed one throws again); then answers as quota().
|
|
156
134
|
*/
|
|
157
|
-
async
|
|
158
|
-
const { settings } = this.#known(provider);
|
|
135
|
+
async readQuota(provider, { maxAgeMs = 0 } = {}) {
|
|
159
136
|
const reading = this.#reading(provider, maxAgeMs);
|
|
160
|
-
|
|
161
|
-
|
|
162
|
-
|
|
163
|
-
return { remaining: answer.remaining - unbilled, unbilled, unit: answer.unit, renewsAt: answer.renewsAt ?? null };
|
|
137
|
+
await reading.settled;
|
|
138
|
+
if (reading.error) throw reading.error;
|
|
139
|
+
return this.quota(provider);
|
|
164
140
|
}
|
|
165
141
|
|
|
166
142
|
/**
|
|
167
|
-
*
|
|
168
|
-
*
|
|
169
|
-
*
|
|
143
|
+
* What is left of the account's quota now, in the provider's unit: { remaining, unbilled, unit, renewsAt }. The
|
|
144
|
+
* last reading shows the sessions that ended billingDelayMs before it; `unbilled` is what the ledger's others
|
|
145
|
+
* cost at the provider's price, open ones as if they lasted as long as they can, and remaining the reading less
|
|
146
|
+
* that. renewsAt (ms): when the quota renews by itself; null: only a top-up renews it. Throws until the
|
|
147
|
+
* provider has been read, and while its last reading failed.
|
|
170
148
|
*/
|
|
149
|
+
quota(provider) {
|
|
150
|
+
const { settings } = this.#known(provider);
|
|
151
|
+
const reading = this.#readings.get(provider)?.done;
|
|
152
|
+
if (!reading) throw new Error(`the quota of ${provider} has not been read yet (readQuota)`);
|
|
153
|
+
if (reading.error) throw reading.error;
|
|
154
|
+
const unbilled = this.#store.openSince(provider, reading.billedBefore).reduce((cost, s) => cost + sessionCost(settings.price, s.durationMs), 0);
|
|
155
|
+
const { remaining, unit, renewsAt = null } = reading.value;
|
|
156
|
+
return { remaining: remaining - unbilled, unbilled, unit, renewsAt };
|
|
157
|
+
}
|
|
158
|
+
|
|
159
|
+
/** The latest reading of the provider's quota if it is in progress or at most maxAgeMs old, else a new one */
|
|
171
160
|
#reading(provider, maxAgeMs) {
|
|
172
|
-
const
|
|
173
|
-
|
|
161
|
+
const readings = this.#readings.get(provider) ?? { latest: null, done: null };
|
|
162
|
+
this.#readings.set(provider, readings);
|
|
163
|
+
const { latest } = readings;
|
|
164
|
+
const inProgress = latest && latest !== readings.done;
|
|
165
|
+
if (inProgress || (latest && Date.now() - latest.startedAt <= maxAgeMs)) return latest;
|
|
174
166
|
const { adapter, settings, apiKey } = this.#ready(provider);
|
|
175
|
-
const
|
|
176
|
-
reading
|
|
177
|
-
|
|
178
|
-
|
|
167
|
+
const startedAt = Date.now();
|
|
168
|
+
const reading = { startedAt, billedBefore: startedAt - settings.billingDelayMs, value: null, error: null };
|
|
169
|
+
readings.latest = reading;
|
|
170
|
+
reading.settled = (async () => {
|
|
171
|
+
try {
|
|
172
|
+
const q = await adapter.quota(settings, apiKey, this.#store.history(provider, reading.billedBefore));
|
|
179
173
|
if (!(Number.isFinite(q?.remaining) && typeof q.unit === 'string')) throw new Error(`${provider} answered no quota: ${JSON.stringify(q)}`);
|
|
180
|
-
|
|
181
|
-
})
|
|
182
|
-
|
|
183
|
-
|
|
184
|
-
|
|
174
|
+
reading.value = q;
|
|
175
|
+
} catch (e) {
|
|
176
|
+
reading.error = e;
|
|
177
|
+
}
|
|
178
|
+
readings.done = reading;
|
|
179
|
+
})();
|
|
185
180
|
return reading;
|
|
186
181
|
}
|
|
187
182
|
|
|
188
|
-
/**
|
|
189
|
-
* What a session with `provider` and these session options (sessionOptions) costs at most, in the unit of its
|
|
190
|
-
* quota: as if it lasted as long as it can, at the provider's price
|
|
191
|
-
*/
|
|
183
|
+
/** What a session with these options costs at most, in the unit of the provider's quota */
|
|
192
184
|
maxSessionCost(provider, options) {
|
|
193
|
-
|
|
185
|
+
const { settings } = this.#known(provider);
|
|
186
|
+
return sessionCost(settings.price, sessionMsAtMost(sessionOptions(options), settings));
|
|
194
187
|
}
|
|
195
188
|
|
|
196
189
|
/**
|
|
197
|
-
* Stores
|
|
198
|
-
*
|
|
199
|
-
*
|
|
200
|
-
* to the programs that have one. Returns { reported: sessions completed, failed: [{ provider, error }] }.
|
|
190
|
+
* Stores the providers' records (duration, proxy traffic) of the ledger's ended sessions that have none yet:
|
|
191
|
+
* { reported, failed: [{ provider, error }] }. What cannot be fetched now, and the sessions of providers it has
|
|
192
|
+
* no adapter for, are left for a later call.
|
|
201
193
|
*/
|
|
202
194
|
async syncUsage() {
|
|
203
195
|
const store = this.#store;
|
|
@@ -218,10 +210,8 @@ export class OneBrowser {
|
|
|
218
210
|
|
|
219
211
|
/**
|
|
220
212
|
* Sessions opened since `since` (ms), per UTC day, provider and client:
|
|
221
|
-
* [{ day, provider, client, sessions, minutes, untimed, unsynced, proxyBytes }].
|
|
222
|
-
*
|
|
223
|
-
* session; untimed: sessions with neither (their process ended before closing them and the provider has not
|
|
224
|
-
* reported them yet); unsynced: sessions without the provider's record.
|
|
213
|
+
* [{ day, provider, client, sessions, minutes, untimed, unsynced, proxyBytes }]. minutes: the provider's record
|
|
214
|
+
* (syncUsage), else the open-to-close timing; untimed: sessions with neither; unsynced: those without the record.
|
|
225
215
|
*/
|
|
226
216
|
usage({ since }) {
|
|
227
217
|
return this.#store.usage(since);
|
|
@@ -234,7 +224,6 @@ export class OneBrowser {
|
|
|
234
224
|
this.#ledger = null;
|
|
235
225
|
}
|
|
236
226
|
|
|
237
|
-
/** close(), for `await using browsers = new OneBrowser(…)` */
|
|
238
227
|
[Symbol.asyncDispose]() {
|
|
239
228
|
return this.close();
|
|
240
229
|
}
|
package/src/ledger.js
CHANGED
|
@@ -14,20 +14,15 @@ export function defaultLedgerPath() {
|
|
|
14
14
|
return path.join(data, 'onebrowser', 'sessions.db');
|
|
15
15
|
}
|
|
16
16
|
|
|
17
|
-
/**
|
|
18
|
-
|
|
19
|
-
* is billed from then on, even if connecting fails), closed when released, and completed with the
|
|
20
|
-
* provider's own record (duration, proxy traffic) once the session has ended and syncUsage has
|
|
21
|
-
* fetched it
|
|
22
|
-
*/
|
|
23
|
-
const SESSIONS = `
|
|
17
|
+
/** Every remote browser session, from when its provider has created it: it is billed from then on, even if connecting fails */
|
|
18
|
+
const SCHEMA = `
|
|
24
19
|
CREATE TABLE sessions (
|
|
25
20
|
provider TEXT NOT NULL,
|
|
26
21
|
id TEXT NOT NULL, -- the session id its provider gave it
|
|
27
22
|
client TEXT NOT NULL, -- the program that opened it (OneBrowser's client)
|
|
28
23
|
opened_at TEXT NOT NULL, -- when it was asked for; its provider created it right after
|
|
29
24
|
ends_by TEXT NOT NULL, -- when it has ended at the latest, whatever becomes of its process
|
|
30
|
-
closed_at TEXT, -- when
|
|
25
|
+
closed_at TEXT, -- when its provider released it; empty until then, and if it never was
|
|
31
26
|
duration_ms INTEGER, -- the provider's record, once the session has ended
|
|
32
27
|
proxy_bytes INTEGER, -- proxy traffic in the provider's record
|
|
33
28
|
synced_at TEXT, -- when the provider's record was stored; empty = not yet
|
|
@@ -37,25 +32,25 @@ CREATE INDEX idx_sessions_opened ON sessions(provider, opened_at);
|
|
|
37
32
|
`;
|
|
38
33
|
|
|
39
34
|
/**
|
|
40
|
-
* The
|
|
41
|
-
*
|
|
42
|
-
* ledger is brought up to date when it is opened.
|
|
35
|
+
* The version of SCHEMA (PRAGMA user_version), and the step to it from each earlier one: a ledger outlives the
|
|
36
|
+
* onebrowser that made it. onebrowser 0.1–0.3 made version 1 and left user_version 0.
|
|
43
37
|
*/
|
|
44
|
-
const
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
38
|
+
const VERSION = 2;
|
|
39
|
+
const UPGRADES = {
|
|
40
|
+
// 1 → 2: ends_by, as if the sessions of 1 had the default session options (15 min, then 60 s idle). Each step
|
|
41
|
+
// makes its version as that version was, whatever SCHEMA has become since.
|
|
42
|
+
1: `ALTER TABLE sessions RENAME TO sessions_1;
|
|
43
|
+
DROP INDEX idx_sessions_opened;
|
|
44
|
+
CREATE TABLE sessions (
|
|
45
|
+
provider TEXT NOT NULL, id TEXT NOT NULL, client TEXT NOT NULL, opened_at TEXT NOT NULL, ends_by TEXT NOT NULL,
|
|
46
|
+
closed_at TEXT, duration_ms INTEGER, proxy_bytes INTEGER, synced_at TEXT, PRIMARY KEY (provider, id)
|
|
47
|
+
);
|
|
48
|
+
CREATE INDEX idx_sessions_opened ON sessions(provider, opened_at);
|
|
49
|
+
INSERT INTO sessions (provider, id, client, opened_at, ends_by, closed_at, duration_ms, proxy_bytes, synced_at)
|
|
50
|
+
SELECT provider, id, client, opened_at, strftime('%Y-%m-%dT%H:%M:%fZ', opened_at, '+16 minutes'), closed_at, duration_ms, proxy_bytes, synced_at
|
|
51
|
+
FROM sessions_1;
|
|
52
|
+
DROP TABLE sessions_1;`,
|
|
53
|
+
};
|
|
59
54
|
|
|
60
55
|
/** ms from one ISO time to another */
|
|
61
56
|
const msBetween = (from, to) => `CAST(ROUND((julianday(${to}) - julianday(${from})) * 86400000) AS INTEGER)`;
|
|
@@ -73,29 +68,31 @@ const iso = (ms) => new Date(ms).toISOString();
|
|
|
73
68
|
const int = (x) => (x === null || x === undefined || !Number.isFinite(Number(x)) ? null : Math.trunc(Number(x)));
|
|
74
69
|
|
|
75
70
|
export class Ledger {
|
|
71
|
+
#file;
|
|
76
72
|
#db;
|
|
77
73
|
#stmts = new Map();
|
|
78
74
|
|
|
79
75
|
constructor(file) {
|
|
80
76
|
if (file !== ':memory:') fs.mkdirSync(path.dirname(file), { recursive: true });
|
|
77
|
+
this.#file = file;
|
|
81
78
|
this.#db = new DatabaseSync(file);
|
|
82
79
|
this.#db.exec('PRAGMA journal_mode=WAL; PRAGMA busy_timeout=5000;'); // several processes share it
|
|
83
80
|
this.#migrate();
|
|
84
81
|
}
|
|
85
82
|
|
|
86
|
-
/**
|
|
83
|
+
/** Makes the ledger, or brings an earlier one up to date, in one transaction: another process may be opening it too */
|
|
87
84
|
#migrate() {
|
|
88
85
|
const version = () => this.#db.prepare('PRAGMA user_version').get().user_version;
|
|
89
|
-
if (version() ===
|
|
90
|
-
this
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
}
|
|
86
|
+
if (version() === VERSION) return;
|
|
87
|
+
this.tx(() => {
|
|
88
|
+
if (!this.#db.prepare("SELECT 1 FROM sqlite_schema WHERE type = 'table' AND name = 'sessions'").get()) this.#db.exec(SCHEMA);
|
|
89
|
+
else {
|
|
90
|
+
const from = version() || 1;
|
|
91
|
+
if (from > VERSION) throw new Error(`onebrowser: the ledger ${this.#file} was made by a later onebrowser (schema ${from}; this one knows ${VERSION})`);
|
|
92
|
+
for (let v = from; v < VERSION; v++) this.#db.exec(UPGRADES[v]);
|
|
93
|
+
}
|
|
94
|
+
this.#db.exec(`PRAGMA user_version = ${VERSION}`);
|
|
95
|
+
});
|
|
99
96
|
}
|
|
100
97
|
|
|
101
98
|
#q(sql) {
|
|
@@ -112,24 +109,21 @@ export class Ledger {
|
|
|
112
109
|
this.#q('UPDATE sessions SET closed_at=? WHERE provider=? AND id=? AND closed_at IS NULL').run(iso(closedAt), provider, id);
|
|
113
110
|
}
|
|
114
111
|
|
|
115
|
-
/** The
|
|
116
|
-
history(provider) {
|
|
112
|
+
/** The provider's sessions that a quota reading shows, those that ended before `billedBefore` (ms); openSince() has the rest */
|
|
113
|
+
history(provider, billedBefore) {
|
|
117
114
|
return {
|
|
118
|
-
/**
|
|
115
|
+
/** Their session time (ms) since `since` (ms), each as long as it can have lasted while it has no timing */
|
|
119
116
|
endedSessionMsSince: (since) =>
|
|
120
|
-
this.#q(`SELECT COALESCE(SUM(${SESSION_MS_AT_MOST}), 0) ms FROM sessions WHERE provider=? AND opened_at >= ? AND ${ENDED_AT} < ?`).get(provider, iso(since), iso(
|
|
117
|
+
this.#q(`SELECT COALESCE(SUM(${SESSION_MS_AT_MOST}), 0) ms FROM sessions WHERE provider=? AND opened_at >= ? AND ${ENDED_AT} < ?`).get(provider, iso(since), iso(billedBefore)).ms,
|
|
121
118
|
};
|
|
122
119
|
}
|
|
123
120
|
|
|
124
|
-
/**
|
|
125
|
-
|
|
126
|
-
* [{ durationMs }]: each as long as it lasted, or can last at most while it has no timing
|
|
127
|
-
*/
|
|
128
|
-
openAfter(provider, at) {
|
|
121
|
+
/** The provider's sessions open at any time since `at` (ms): [{ durationMs }], as long as they can last while untimed */
|
|
122
|
+
openSince(provider, at) {
|
|
129
123
|
return this.#q(`SELECT ${SESSION_MS_AT_MOST} durationMs FROM sessions WHERE provider=? AND ${ENDED_AT} >= ?`).all(provider, iso(at));
|
|
130
124
|
}
|
|
131
125
|
|
|
132
|
-
/** Sessions
|
|
126
|
+
/** Sessions of `providers` without their provider's record yet: [{ provider, id, openedAt, closedAt }] (ISO times) */
|
|
133
127
|
awaitingUsage(providers) {
|
|
134
128
|
return this.#q(
|
|
135
129
|
'SELECT provider, id, opened_at openedAt, closed_at closedAt FROM sessions WHERE synced_at IS NULL AND provider IN (SELECT value FROM json_each(?)) ORDER BY opened_at',
|
|
@@ -140,8 +134,9 @@ export class Ledger {
|
|
|
140
134
|
this.#q('UPDATE sessions SET duration_ms=?, proxy_bytes=?, synced_at=? WHERE provider=? AND id=?').run(int(durationMs), int(proxyBytes), iso(Date.now()), provider, id);
|
|
141
135
|
}
|
|
142
136
|
|
|
137
|
+
/** Runs fn in a transaction that takes the write lock at once: another process may be writing too */
|
|
143
138
|
tx(fn) {
|
|
144
|
-
this.#db.exec('BEGIN');
|
|
139
|
+
this.#db.exec('BEGIN IMMEDIATE');
|
|
145
140
|
try {
|
|
146
141
|
const r = fn();
|
|
147
142
|
this.#db.exec('COMMIT');
|
package/src/providers/anchor.js
CHANGED
|
@@ -16,6 +16,7 @@ export const defaults = {
|
|
|
16
16
|
// (credits_used) bill more, about $0.06 an hour and a step of $0.003 in the second minute, which this covers
|
|
17
17
|
price: { perSession: 0.01, perHour: 0.15, stepSec: 60 },
|
|
18
18
|
billingDelayMs: 5 * 60_000, // seen within 10 s
|
|
19
|
+
minIdleTimeoutSec: 60, // idle_timeout is in whole minutes, at least 1
|
|
19
20
|
};
|
|
20
21
|
|
|
21
22
|
const headers = (apiKey) => ({ 'anchor-api-key': apiKey });
|
package/src/providers/index.js
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
import {
|
|
1
|
+
import { fullPrice } from '../cost.js';
|
|
2
2
|
import * as anchor from './anchor.js';
|
|
3
3
|
import * as firecrawl from './firecrawl.js';
|
|
4
4
|
import * as hyperbrowser from './hyperbrowser.js';
|
|
@@ -8,28 +8,22 @@ import * as steel from './steel.js';
|
|
|
8
8
|
import * as tinyfish from './tinyfish.js';
|
|
9
9
|
|
|
10
10
|
/**
|
|
11
|
-
* Provider adapters
|
|
12
|
-
* defaults
|
|
13
|
-
*
|
|
14
|
-
*
|
|
15
|
-
*
|
|
16
|
-
*
|
|
17
|
-
* createSession(settings, apiKey,
|
|
18
|
-
*
|
|
19
|
-
*
|
|
20
|
-
* session because the account's quota is used up
|
|
21
|
-
* releaseSession(settings, apiKey, session) ends the session; throws on failure
|
|
11
|
+
* Provider adapters by name: the only provider-specific code. Each exports:
|
|
12
|
+
* defaults its settings, which OneBrowser's `providers` option overrides:
|
|
13
|
+
* price what its sessions cost, in the unit of its quota (cost.js)
|
|
14
|
+
* billingDelayMs how long after a session has ended its quota reading shows it, at the latest
|
|
15
|
+
* minIdleTimeoutSec the shortest idle timeout it takes (default 0)
|
|
16
|
+
* and its own API base URLs, plan facts such as monthlyHours
|
|
17
|
+
* createSession(settings, apiKey, { proxyCountry, idleTimeoutSec, maxDurationMs }) → { id, cdpUrl, egress }
|
|
18
|
+
* throws QuotaError (errors.js) when the provider refuses the session for the account's quota
|
|
19
|
+
* releaseSession(settings, apiKey, session)
|
|
22
20
|
* sessionUsage(settings, apiKey, [{ id, openedAt, closedAt }]) → [{ id, durationMs, proxyBytes }]
|
|
23
|
-
* the record of those
|
|
24
|
-
* API key cannot read, the ledger's timing (closedAt is empty for a session no process closed); throws on
|
|
25
|
-
* failure
|
|
21
|
+
* the provider's record of those that have ended, or the ledger's timing where its API key cannot read one
|
|
26
22
|
* quota(settings, apiKey, history) → { remaining, unit, renewsAt? }
|
|
27
|
-
* what
|
|
28
|
-
*
|
|
29
|
-
*
|
|
30
|
-
*
|
|
31
|
-
* Their requests go through http.js, which turns every status other than 2xx into an HttpError: an adapter
|
|
32
|
-
* handles only the statuses that mean something of their own.
|
|
23
|
+
* what the account has left in the provider's unit, and when it renews by itself (ms); history: the
|
|
24
|
+
* ledger's sessions that the reading shows (Ledger.history), for a provider that cannot count its usage
|
|
25
|
+
* Every function throws when it fails. Requests go through http.js, which throws HttpError for a status other
|
|
26
|
+
* than 2xx.
|
|
33
27
|
*/
|
|
34
28
|
export const ADAPTERS = { tinyfish, steel, hyperbrowser, lightpanda, kernel, anchor, firecrawl };
|
|
35
29
|
|
|
@@ -41,12 +35,15 @@ export function checkAdapter(name, adapter) {
|
|
|
41
35
|
if (missing.length) throw new Error(`provider ${name}: its adapter lacks ${missing.join(', ')}`);
|
|
42
36
|
}
|
|
43
37
|
|
|
44
|
-
/**
|
|
45
|
-
export function
|
|
38
|
+
/** An adapter's settings with the defaults of those every adapter has; throws when one of them is missing or wrong */
|
|
39
|
+
export function fullSettings(name, { price, billingDelayMs, minIdleTimeoutSec = 0, ...own }) {
|
|
40
|
+
const atLeast0 = (x) => Number.isFinite(x) && x >= 0;
|
|
46
41
|
try {
|
|
47
|
-
|
|
42
|
+
const full = fullPrice(price);
|
|
43
|
+
if (!atLeast0(billingDelayMs)) throw new Error(`billingDelayMs must be a number ≥ 0, not ${billingDelayMs}`);
|
|
44
|
+
if (!atLeast0(minIdleTimeoutSec)) throw new Error(`minIdleTimeoutSec must be a number ≥ 0, not ${minIdleTimeoutSec}`);
|
|
45
|
+
return { ...own, price: full, billingDelayMs, minIdleTimeoutSec };
|
|
48
46
|
} catch (e) {
|
|
49
47
|
throw new Error(`provider ${name}: ${e.message}`);
|
|
50
48
|
}
|
|
51
|
-
if (!(Number.isFinite(settings.billingDelayMs) && settings.billingDelayMs >= 0)) throw new Error(`provider ${name}: billingDelayMs must be a number ≥ 0, not ${settings.billingDelayMs}`);
|
|
52
49
|
}
|
package/src/session.js
CHANGED
|
@@ -1,31 +1,24 @@
|
|
|
1
|
-
/**
|
|
2
|
-
* A CDP command this often while connected keeps a provider's idle timeout from ending a session that is in use
|
|
3
|
-
* (also while a page waits long for an answer). OneBrowser requires idleTimeoutSec to be at least twice as long.
|
|
4
|
-
*/
|
|
1
|
+
/** A CDP command this often keeps a provider's idle timeout from ending a session in use; idleTimeoutSec must be at least twice as long */
|
|
5
2
|
export const KEEP_ALIVE_MS = 15_000;
|
|
6
3
|
|
|
7
4
|
/** Playwright errors carry a multi-line call log; the first line says what failed */
|
|
8
5
|
const firstLine = (e) => String(e.message).split('\n')[0];
|
|
9
6
|
|
|
10
7
|
/**
|
|
11
|
-
*
|
|
12
|
-
*
|
|
13
|
-
*
|
|
14
|
-
* its
|
|
15
|
-
*
|
|
16
|
-
*
|
|
17
|
-
* id, provider, egress the provider's session id, its name, and the egress the session uses
|
|
18
|
-
* cdpUrl CDP WebSocket URL (it may carry the API key: never log it)
|
|
19
|
-
* openedAt, closedAt ms: when it was asked for (its provider created it right after), when it was
|
|
20
|
-
* closed; closedAt is null while open
|
|
8
|
+
* A remote browser session (OneBrowser.openSession), used over CDP: cdpUrl with any client (it may carry the API
|
|
9
|
+
* key: never log it), or connect() / newPage() with Playwright. This process closes it maxDurationMs after it was
|
|
10
|
+
* opened; one whose process died is ended by its provider idleTimeoutSec (at least its minIdleTimeoutSec) after
|
|
11
|
+
* its client left.
|
|
12
|
+
* openedAt: when it was asked for; closedAt: when its provider released it, null until then.
|
|
21
13
|
*/
|
|
22
14
|
export class Session {
|
|
23
15
|
#release;
|
|
24
|
-
#
|
|
16
|
+
#closing = null;
|
|
17
|
+
#browser = null; // a promise of Playwright's Browser
|
|
25
18
|
#keepAlive = null;
|
|
26
19
|
#deadline;
|
|
27
20
|
|
|
28
|
-
/** release(session)
|
|
21
|
+
/** release(session) → closedAt: releases it at its provider and records it closed (OneBrowser) */
|
|
29
22
|
constructor({ provider, remote, openedAt, maxDurationMs, release }) {
|
|
30
23
|
this.provider = provider;
|
|
31
24
|
this.id = remote.id;
|
|
@@ -34,13 +27,13 @@ export class Session {
|
|
|
34
27
|
this.openedAt = openedAt;
|
|
35
28
|
this.closedAt = null;
|
|
36
29
|
this.#release = release;
|
|
37
|
-
this.#deadline = setTimeout(() => this.close().catch(() => {}), openedAt + maxDurationMs - Date.now());
|
|
30
|
+
this.#deadline = setTimeout(() => this.close().catch(() => {}), openedAt + maxDurationMs - Date.now());
|
|
38
31
|
this.#deadline.unref();
|
|
39
32
|
}
|
|
40
33
|
|
|
41
|
-
/** Playwright's Browser over this session's CDP connection, made once;
|
|
34
|
+
/** Playwright's Browser over this session's CDP connection, made once; a keep-alive runs while it is connected */
|
|
42
35
|
connect() {
|
|
43
|
-
if (this
|
|
36
|
+
if (this.#closing) return Promise.reject(new Error(`${this.provider} session ${this.id} is closed`));
|
|
44
37
|
this.#browser ??= this.#connect();
|
|
45
38
|
return this.#browser;
|
|
46
39
|
}
|
|
@@ -55,7 +48,7 @@ export class Session {
|
|
|
55
48
|
}
|
|
56
49
|
try {
|
|
57
50
|
const cdp = await browser.newBrowserCDPSession();
|
|
58
|
-
if (this
|
|
51
|
+
if (this.#closing) throw new Error(`${this.provider} session ${this.id} is closed`);
|
|
59
52
|
this.#keepAlive = setInterval(() => cdp.send('Browser.getVersion').catch(() => {}), KEEP_ALIVE_MS);
|
|
60
53
|
this.#keepAlive.unref();
|
|
61
54
|
} catch (e) {
|
|
@@ -65,10 +58,7 @@ export class Session {
|
|
|
65
58
|
return browser;
|
|
66
59
|
}
|
|
67
60
|
|
|
68
|
-
/**
|
|
69
|
-
* A new page in the browser's default context. Always a new one: the page a remote browser starts with is not
|
|
70
|
-
* always one that can be navigated.
|
|
71
|
-
*/
|
|
61
|
+
/** A new page in the default context: the page a remote browser starts with cannot always be navigated */
|
|
72
62
|
async newPage() {
|
|
73
63
|
const browser = await this.connect();
|
|
74
64
|
const context = browser.contexts()[0] ?? (await browser.newContext());
|
|
@@ -76,22 +66,20 @@ export class Session {
|
|
|
76
66
|
}
|
|
77
67
|
|
|
78
68
|
/**
|
|
79
|
-
* Disconnects and releases the session
|
|
80
|
-
*
|
|
81
|
-
* idleTimeoutSec after the disconnect.
|
|
69
|
+
* Disconnects and releases the session; every call answers as the first. Throws when its provider could not
|
|
70
|
+
* release it: the ledger then counts it as lasting as long as it can, as its provider ends it by itself.
|
|
82
71
|
*/
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
72
|
+
close() {
|
|
73
|
+
return (this.#closing ??= this.#close());
|
|
74
|
+
}
|
|
75
|
+
|
|
76
|
+
async #close() {
|
|
86
77
|
clearTimeout(this.#deadline);
|
|
87
78
|
clearInterval(this.#keepAlive);
|
|
88
|
-
|
|
89
|
-
this
|
|
90
|
-
await browser?.then((b) => b.close()).catch(() => {});
|
|
91
|
-
await this.#release(this);
|
|
79
|
+
await this.#browser?.then((b) => b.close()).catch(() => {});
|
|
80
|
+
this.closedAt = await this.#release(this);
|
|
92
81
|
}
|
|
93
82
|
|
|
94
|
-
/** close(): `await using session = await browsers.openSession(…)` releases it at the end of the block */
|
|
95
83
|
[Symbol.asyncDispose]() {
|
|
96
84
|
return this.close();
|
|
97
85
|
}
|