@intuned/browser-use-dev 0.0.0-stage → 0.1.2-dev.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/LICENSE ADDED
@@ -0,0 +1,43 @@
1
+ Acceptance
2
+ By using the software, you agree to all of the terms and conditions below.
3
+
4
+ Copyright License
5
+ The licensor grants you a non-exclusive, royalty-free, worldwide, non-sublicensable, non-transferable license to use, copy, distribute, make available, and prepare derivative works of the software, in each case subject to the limitations and conditions below.
6
+
7
+ Limitations
8
+ You may not provide the software to third parties as a hosted or managed service, where the service provides users with access to any substantial set of the features or functionality of the software.
9
+
10
+ You may not move, change, disable, or circumvent the license key functionality in the software, and you may not remove or obscure any functionality in the software that is protected by the license key.
11
+
12
+ You may not alter, remove, or obscure any licensing, copyright, or other notices of the licensor in the software. Any use of the licensor’s trademarks is subject to applicable law.
13
+
14
+ Patents
15
+ The licensor grants you a license, under any patent claims the licensor can license, or becomes able to license, to make, have made, use, sell, offer for sale, import and have imported the software, in each case subject to the limitations and conditions in this license. This license does not cover any patent claims that you cause to be infringed by modifications or additions to the software. If you or your company make any written claim that the software infringes or contributes to infringement of any patent, your patent license for the software granted under these terms ends immediately. If your company makes such a claim, your patent license ends immediately for work on behalf of your company.
16
+
17
+ Notices
18
+ You must ensure that anyone who gets a copy of any part of the software from you also gets a copy of these terms.
19
+
20
+ If you modify the software, you must include in any modified copies of the software prominent notices stating that you have modified the software.
21
+
22
+ No Other Rights
23
+ These terms do not imply any licenses other than those expressly granted in these terms.
24
+
25
+ Termination
26
+ If you use the software in violation of these terms, such use is not licensed, and your licenses will automatically terminate. If the licensor provides you with a notice of your violation, and you cease all violation of this license no later than 30 days after you receive that notice, your licenses will be reinstated retroactively. However, if you violate these terms after such reinstatement, any additional violation of these terms will cause your licenses to terminate automatically and permanently.
27
+
28
+ No Liability
29
+ As far as the law allows, the software comes as is, without any warranty or condition, and the licensor will not be liable to you for any damages arising out of these terms or the use or nature of the software, under any kind of legal claim.
30
+
31
+ Definitions
32
+ The licensor is the entity offering these terms, and the software is the software the licensor makes available under these terms, including any portion of it.
33
+
34
+ you refers to the individual or entity agreeing to these terms.
35
+
36
+ your company is any legal entity, sole proprietorship, or other kind of organization that you work for, plus all organizations that have control over, are under the control of, or are under common control with that organization. control means ownership of substantially all the assets of an entity, or the power to direct its management and policies by vote, contract, or otherwise. Control can be direct or indirect.
37
+
38
+ your licenses are all the licenses granted to you for the software under these terms.
39
+
40
+ use means anything you do with the software requiring one of your licenses.
41
+
42
+ trademark means trademarks, service marks, and similar rights.
43
+
package/README.md CHANGED
@@ -1,3 +1,40 @@
1
- # Temporary Holding Version
1
+ # @intuned/browser-use
2
2
 
3
- This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
3
+ Intuned SDK browser-use agent (BETA): drive a Playwright page with any
4
+ vision-capable model — screenshots in, click/type/scroll actions out, executed
5
+ on the exact page you pass in. Built on the AI SDK, routed through the Intuned
6
+ AI gateway (or a direct provider API key).
7
+
8
+ ```typescript
9
+ import { executeTask } from "@intuned/browser-use";
10
+
11
+ const result = await executeTask({
12
+ page,
13
+ task: "Scroll down until the year 2021 is visible and report the first 2021 entry's title.",
14
+ maxTurns: 20,
15
+ timeoutInMs: 120_000,
16
+ });
17
+ console.log(result.finalText, result.totalCostInCents);
18
+ ```
19
+
20
+ ## Highlights
21
+
22
+ - **Reliable interactions**: every action syncs the viewport to the real
23
+ window size first, waits for the network it triggers to settle, and scrolls
24
+ slowly in steps so lazy-loaded content renders instead of being skipped.
25
+ - **Browser-event awareness**: file downloads, popup tabs opening, and tabs
26
+ closing during an action are detected and reported back to the model,
27
+ alongside the current URL.
28
+ - **Structured results**: pass an `outputSchema` (JSON schema) and get a
29
+ schema-validated `output` object back.
30
+ - **Extensible**: add your own function tools via `tools`; opt into the
31
+ `execute_javascript` escape hatch with `enableJavaScriptTool`.
32
+
33
+ ## Development
34
+
35
+ ```bash
36
+ yarn install
37
+ yarn build # tsc + babel into dist/
38
+ yarn test # vitest
39
+ yarn lint
40
+ ```
@@ -0,0 +1,15 @@
1
+ "use strict";
2
+
3
+ Object.defineProperty(exports, "__esModule", {
4
+ value: true
5
+ });
6
+ exports.BrowserUseError = void 0;
7
+ class BrowserUseError extends Error {
8
+ constructor(message, info) {
9
+ super(message);
10
+ this.name = "BrowserUseError";
11
+ this.totalCostInCents = info.totalCostInCents;
12
+ this.turns = info.turns;
13
+ }
14
+ }
15
+ exports.BrowserUseError = BrowserUseError;
@@ -0,0 +1,96 @@
1
+ "use strict";
2
+
3
+ Object.defineProperty(exports, "__esModule", {
4
+ value: true
5
+ });
6
+ exports.EventCollector = void 0;
7
+ const EVENT_WAIT_TIMEOUT_MS = 3_000;
8
+ function timestamp() {
9
+ return new Date().toISOString().replace("T", " ").slice(0, 19);
10
+ }
11
+ function deferred() {
12
+ const d = {
13
+ done: false
14
+ };
15
+ d.promise = new Promise(resolve => {
16
+ d.resolve = () => {
17
+ d.done = true;
18
+ resolve();
19
+ };
20
+ });
21
+ return d;
22
+ }
23
+ class EventCollector {
24
+ _context = null;
25
+ _events = [];
26
+ _trackedPages = [];
27
+ _newPage = deferred();
28
+ _download = deferred();
29
+ _close = deferred();
30
+ _onNewPage = page => {
31
+ this._newPage.resolve();
32
+ this._events.push(`Open Page Event (In New Tab) Detected with url: ${page.url()} at ${timestamp()}`);
33
+ page.on("close", this._onPageClose);
34
+ page.on("download", this._onDownload);
35
+ this._trackedPages.push(page);
36
+ };
37
+ _onPageClose = page => {
38
+ this._close.resolve();
39
+ this._events.push(`Close Page Event Detected with url: ${page.url()} at ${timestamp()}`);
40
+ };
41
+ _onDownload = download => {
42
+ this._download.resolve();
43
+ this._events.push(`Download Event Detected with file name: ${download.suggestedFilename()} at ${timestamp()}`);
44
+ };
45
+ constructor(page) {
46
+ this._page = page;
47
+ }
48
+ start() {
49
+ this._context = this._page.context();
50
+ this._context.on("page", this._onNewPage);
51
+ this._page.on("close", this._onPageClose);
52
+ this._page.on("download", this._onDownload);
53
+ this._trackedPages.push(this._page);
54
+ }
55
+ async stop() {
56
+ var _this$_context;
57
+ const pending = [this._newPage, this._download, this._close].filter(d => !d.done).map(d => d.promise);
58
+ if (pending.length > 0) {
59
+ let timer;
60
+ await Promise.race([Promise.all(pending), new Promise(resolve => {
61
+ timer = setTimeout(resolve, EVENT_WAIT_TIMEOUT_MS);
62
+ })]);
63
+ if (timer !== undefined) clearTimeout(timer);
64
+ }
65
+ for (const page of this._trackedPages.slice(1)) {
66
+ try {
67
+ const index = this._page.context().pages().indexOf(page);
68
+ if (index !== -1) {
69
+ this._events.push(`New tab available at index ${index} with url: ${page.url()}`);
70
+ }
71
+ } catch {}
72
+ }
73
+ (_this$_context = this._context) === null || _this$_context === void 0 || _this$_context.off("page", this._onNewPage);
74
+ for (const page of this._trackedPages) {
75
+ page.off("close", this._onPageClose);
76
+ page.off("download", this._onDownload);
77
+ }
78
+ }
79
+ getEvents() {
80
+ return [...this._events];
81
+ }
82
+ _eventsString() {
83
+ return this._events.length > 0 ? this._events.join("\n") : "No events detected";
84
+ }
85
+ _currentUrl() {
86
+ try {
87
+ return this._page.url();
88
+ } catch {
89
+ return "unknown";
90
+ }
91
+ }
92
+ formatResponse(actionResult) {
93
+ return `CURRENT URL: ${this._currentUrl()}\nResult: ${actionResult}\nEvents: ${this._eventsString()}`;
94
+ }
95
+ }
96
+ exports.EventCollector = EventCollector;
@@ -0,0 +1,242 @@
1
+ "use strict";
2
+
3
+ Object.defineProperty(exports, "__esModule", {
4
+ value: true
5
+ });
6
+ exports.executeTask = void 0;
7
+ var _dotenv = require("dotenv");
8
+ var _ai = require("ai");
9
+ var _Logger = require("../common/Logger");
10
+ var _aiGateway = require("../common/aiGateway");
11
+ var _parseCostInCents = require("../common/parseCostInCents");
12
+ var _playwrightComputer = require("./playwrightComputer");
13
+ var _messages = require("./messages");
14
+ var _prompts = require("./prompts");
15
+ var _tools = require("./tools");
16
+ var _errors = require("./errors");
17
+ (0, _dotenv.config)();
18
+ const DEFAULT_MODEL = "gpt-5.6-luna";
19
+ const DEFAULT_MAX_TURNS = 100;
20
+ const MAX_IMAGES_IN_CONTEXT = 10;
21
+ const RESULT_NUDGE_TEXT = `You finished without calling the \`${_tools.RESULT_TOOL_NAME}\` tool. ` + `Call \`${_tools.RESULT_TOOL_NAME}\` now with the final result of the task.`;
22
+ function logErrorCauseChain(error) {
23
+ let current = error;
24
+ for (let depth = 0; current instanceof Error && depth < 5; depth++) {
25
+ const e = current;
26
+ const extras = [e.url ? `url=${e.url}` : "", e.statusCode !== undefined ? `status=${e.statusCode}` : "", typeof e.responseBody === "string" ? `body=${e.responseBody.slice(0, 800)}` : ""].filter(Boolean).join(" ");
27
+ _Logger.logger.error(`[browser-use] ${depth === 0 ? "error" : `cause[${depth}]`}: ${e.name}: ${String(e.message).slice(0, 800)}${extras ? ` (${extras})` : ""}`);
28
+ current = e.cause;
29
+ }
30
+ }
31
+ function sanitizeMessages(messages) {
32
+ return messages.map(message => {
33
+ if (!Array.isArray(message.content)) return message;
34
+ return {
35
+ ...message,
36
+ content: message.content.map(part => {
37
+ if (part.type === "image") return {
38
+ ...part,
39
+ image: "[omitted]"
40
+ };
41
+ const output = part.output;
42
+ if (part.type === "tool-result" && (output === null || output === void 0 ? void 0 : output.type) === "content" && Array.isArray(output.value)) {
43
+ return {
44
+ ...part,
45
+ output: {
46
+ ...output,
47
+ value: output.value.map(item => item.type === "media" ? {
48
+ ...item,
49
+ data: "[omitted]"
50
+ } : item)
51
+ }
52
+ };
53
+ }
54
+ return part;
55
+ })
56
+ };
57
+ });
58
+ }
59
+ const executeTask = async options => {
60
+ const {
61
+ page,
62
+ model = DEFAULT_MODEL,
63
+ apiKey,
64
+ tools: extraTools,
65
+ logSteps = true,
66
+ enableJavaScriptTool = false,
67
+ task,
68
+ systemPrompt,
69
+ outputSchema,
70
+ maxTurns = DEFAULT_MAX_TURNS,
71
+ timeoutInMs,
72
+ signal,
73
+ showImages = false
74
+ } = options;
75
+ if (!task || task.trim().length === 0) {
76
+ throw new Error("browser-use: executeTask requires a non-empty task");
77
+ }
78
+ if (logSteps) _Logger.logger.info(`[browser-use] Executing task. Selected model: ${model}`);
79
+ const computer = new _playwrightComputer.PlaywrightComputer(page);
80
+ let outputCaptured = false;
81
+ let capturedOutput = null;
82
+ let costSoFarCents = 0;
83
+ let turnsSoFar = 0;
84
+ const usage = {
85
+ inputTokens: 0,
86
+ outputTokens: 0,
87
+ totalTokens: 0
88
+ };
89
+ const tools = (0, _tools.buildTools)({
90
+ computer,
91
+ extraTools,
92
+ outputSchema,
93
+ logSteps,
94
+ enableJavaScriptTool,
95
+ onResultCaptured: result => {
96
+ outputCaptured = true;
97
+ capturedOutput = result;
98
+ }
99
+ });
100
+ const providerInstance = await (0, _aiGateway.createProviderInstance)({
101
+ apiKey,
102
+ model
103
+ });
104
+ const responses = providerInstance.responses;
105
+ const usesResponsesApi = typeof responses === "function";
106
+ const languageModel = typeof responses === "function" ? responses(model) : providerInstance.chat(model);
107
+ const parts = [(0, _prompts.composeDefaultSystemPrompt)()];
108
+ if (systemPrompt) parts.push(systemPrompt);
109
+ if (outputSchema) {
110
+ parts.push(["<RESULT_SUBMISSION>", `- When the task is complete, call the \`${_tools.RESULT_TOOL_NAME}\` tool exactly once with the final result.`, "- Submit the result via the tool; do not put it only in your text reply.", "</RESULT_SUBMISSION>"].join("\n"));
111
+ }
112
+ const system = parts.join("\n\n");
113
+ const initialScreenshot = await computer.screenshot();
114
+ const initialMessages = [{
115
+ role: "user",
116
+ content: [{
117
+ type: "text",
118
+ text: task
119
+ }, {
120
+ type: "image",
121
+ image: `data:image/png;base64,${initialScreenshot}`
122
+ }]
123
+ }];
124
+ const abort = new AbortController();
125
+ const onExternalAbort = () => abort.abort();
126
+ if (signal) {
127
+ if (signal.aborted) abort.abort();else signal.addEventListener("abort", onExternalAbort, {
128
+ once: true
129
+ });
130
+ }
131
+ const runStep = async (messages, remainingTurns) => (0, _ai.generateText)({
132
+ model: languageModel,
133
+ system,
134
+ messages,
135
+ tools,
136
+ providerOptions: {
137
+ ...(usesResponsesApi ? {
138
+ openai: {
139
+ reasoningSummary: "auto"
140
+ }
141
+ } : {}),
142
+ ...((0, _aiGateway.detectProvider)(model) === "anthropic" ? {
143
+ anthropic: {
144
+ cacheControl: {
145
+ type: "ephemeral"
146
+ }
147
+ }
148
+ } : {})
149
+ },
150
+ stopWhen: [(0, _ai.stepCountIs)(remainingTurns), () => outputCaptured],
151
+ prepareStep: ({
152
+ messages
153
+ }) => {
154
+ const pruned = (0, _messages.pruneImages)(messages, MAX_IMAGES_IN_CONTEXT);
155
+ return pruned === messages ? undefined : {
156
+ messages: pruned
157
+ };
158
+ },
159
+ abortSignal: abort.signal,
160
+ onStepFinish: step => {
161
+ var _step$usage, _step$usage2, _step$usage3, _step$response;
162
+ turnsSoFar += 1;
163
+ usage.inputTokens += ((_step$usage = step.usage) === null || _step$usage === void 0 ? void 0 : _step$usage.inputTokens) ?? 0;
164
+ usage.outputTokens += ((_step$usage2 = step.usage) === null || _step$usage2 === void 0 ? void 0 : _step$usage2.outputTokens) ?? 0;
165
+ usage.totalTokens += ((_step$usage3 = step.usage) === null || _step$usage3 === void 0 ? void 0 : _step$usage3.totalTokens) ?? 0;
166
+ costSoFarCents += (0, _parseCostInCents.parseCostInCents)((_step$response = step.response) === null || _step$response === void 0 || (_step$response = _step$response.headers) === null || _step$response === void 0 ? void 0 : _step$response["x-ai-cost-in-cents"]);
167
+ if (logSteps) {
168
+ var _step$text, _reasoningText;
169
+ const text = (_step$text = step.text) === null || _step$text === void 0 ? void 0 : _step$text.trim();
170
+ if (text) _Logger.logger.info(`[browser-use] agent: ${text}`);
171
+ const reasoning = (_reasoningText = step.reasoningText) === null || _reasoningText === void 0 ? void 0 : _reasoningText.trim();
172
+ if (reasoning && reasoning !== text) {
173
+ _Logger.logger.info(`[browser-use] agent (reasoning): ${reasoning.slice(0, 600)}${reasoning.length > 600 ? "…" : ""}`);
174
+ }
175
+ }
176
+ }
177
+ });
178
+ let timer;
179
+ try {
180
+ const runPromise = runStep(initialMessages, maxTurns);
181
+ let result = timeoutInMs === undefined ? await runPromise : await Promise.race([runPromise, new Promise((_, reject) => {
182
+ timer = setTimeout(() => {
183
+ abort.abort();
184
+ reject(new _errors.BrowserUseError(`browser-use agent timed out after ${timeoutInMs}ms`, {
185
+ totalCostInCents: costSoFarCents,
186
+ turns: turnsSoFar
187
+ }));
188
+ }, timeoutInMs);
189
+ })]);
190
+ const stillWantsToContinue = result.finishReason === "tool-calls";
191
+ if (result.steps.length >= maxTurns && stillWantsToContinue && !outputCaptured) {
192
+ throw new _errors.BrowserUseError(`browser-use agent exceeded ${maxTurns} turns without finishing`, {
193
+ totalCostInCents: costSoFarCents,
194
+ turns: turnsSoFar
195
+ });
196
+ }
197
+ if (outputSchema && !outputCaptured && !stillWantsToContinue && result.steps.length < maxTurns) {
198
+ if (logSteps) _Logger.logger.info(`[browser-use] nudging the agent to submit the structured result`);
199
+ const nudgeMessages = [...initialMessages, ...result.response.messages, {
200
+ role: "user",
201
+ content: [{
202
+ type: "text",
203
+ text: RESULT_NUDGE_TEXT
204
+ }]
205
+ }];
206
+ const remainingTurns = maxTurns - result.steps.length;
207
+ const nudgeResult = await runStep(nudgeMessages, remainingTurns);
208
+ if (nudgeResult.steps.length >= remainingTurns && nudgeResult.finishReason === "tool-calls" && !outputCaptured) {
209
+ throw new _errors.BrowserUseError(`browser-use agent exceeded ${maxTurns} turns without finishing`, {
210
+ totalCostInCents: costSoFarCents,
211
+ turns: turnsSoFar
212
+ });
213
+ }
214
+ result = {
215
+ ...result,
216
+ text: nudgeResult.text,
217
+ finishReason: nudgeResult.finishReason,
218
+ response: nudgeResult.response,
219
+ steps: [...result.steps, ...nudgeResult.steps]
220
+ };
221
+ }
222
+ if (logSteps) {
223
+ _Logger.logger.info(`[browser-use] run finished after ${turnsSoFar} turns. Total AI Cost In Cents: ${costSoFarCents.toFixed(2)}`);
224
+ }
225
+ const allMessages = [...initialMessages, ...result.response.messages];
226
+ return {
227
+ finalText: result.text,
228
+ messages: showImages ? allMessages : sanitizeMessages(allMessages),
229
+ output: outputCaptured ? capturedOutput : null,
230
+ totalCostInCents: costSoFarCents,
231
+ turns: turnsSoFar,
232
+ usage
233
+ };
234
+ } catch (e) {
235
+ if (logSteps && !(e instanceof _errors.BrowserUseError)) logErrorCauseChain(e);
236
+ throw e;
237
+ } finally {
238
+ if (timer !== undefined) clearTimeout(timer);
239
+ if (signal) signal.removeEventListener("abort", onExternalAbort);
240
+ }
241
+ };
242
+ exports.executeTask = executeTask;
@@ -0,0 +1,228 @@
1
+ import { Page } from "playwright";
2
+ import { ModelMessage } from "ai";
3
+
4
+ /**
5
+ * A caller-supplied function tool the browser-use model can invoke during a
6
+ * run, alongside the built-in `computer`, `list_tabs`, `clear_field` and `fill_field`
7
+ * tools. The handler's return value is serialized and fed back to the model
8
+ * as the tool output ("success" when it returns nothing); a thrown error is
9
+ * fed back as an `error: ...` output the model can react to.
10
+ *
11
+ * @interface BrowserUseFunctionTool
12
+ */
13
+ export interface BrowserUseFunctionTool {
14
+ /** Tool name the model calls it by. Must not collide with built-in tools. */
15
+ name: string;
16
+ /** What the tool does and when the model should use it. */
17
+ description: string;
18
+ /** JSON schema object describing the tool's arguments. */
19
+ parameters: Record<string, unknown>;
20
+ /** Executes the tool call with the model-provided arguments. */
21
+ handler: (args: Record<string, unknown>) => unknown | Promise<unknown>;
22
+ }
23
+
24
+ /**
25
+ * A debug frame captured immediately BEFORE a mouse action executes, with the
26
+ * action's target point(s) so a recording shows exactly where each action
27
+ * landed on the state the model acted on.
28
+ *
29
+ * @interface BrowserUseFrame
30
+ */
31
+ export interface BrowserUseFrame {
32
+ /** PNG screenshot, one pixel per CSS pixel. */
33
+ image: Buffer;
34
+ /** Screenshot width in CSS pixels. */
35
+ width: number;
36
+ /** Screenshot height in CSS pixels. */
37
+ height: number;
38
+ /** The action about to run (e.g. "click", "drag", "clear_field"). */
39
+ action: string;
40
+ /** The action's target coordinate(s) in CSS pixels. */
41
+ markers: { x: number; y: number }[];
42
+ /** Extra action context (e.g. mouse button, scroll deltas). */
43
+ label: string;
44
+ }
45
+
46
+ /**
47
+ * Options for {@link executeTask}.
48
+ *
49
+ * @interface ExecuteTaskOptions
50
+ */
51
+ export interface ExecuteTaskOptions {
52
+ /** The Playwright page the agent acts on. The agent stays pinned to this
53
+ * exact page for screenshots and actions (it never switches tabs). */
54
+ page: Page;
55
+ /** Model id to drive the run. Any vision-capable model id supported by the
56
+ * SDK's AI gateway (Anthropic, OpenAI, Google) — the provider is detected
57
+ * from the id. Defaults to "gpt-5.6-luna". */
58
+ model?: string;
59
+ /** Explicit API key for the detected provider. When omitted, the
60
+ * provider's env var (`ANTHROPIC_API_KEY` / `OPENAI_API_KEY` /
61
+ * `GOOGLE_API_KEY`) is used; when that is also absent, the Intuned AI
62
+ * gateway (via `@intuned/runtime`) provides credentials. */
63
+ apiKey?: string;
64
+ /** Extra function tools the model may call during the run. */
65
+ tools?: BrowserUseFunctionTool[];
66
+ /** Log each model message, tool call, and request cost. Defaults to true. */
67
+ logSteps?: boolean;
68
+ /** Expose the built-in `execute_javascript` tool, letting the model run
69
+ * arbitrary JavaScript in the page and get its result back — an escape
70
+ * hatch for cases the standard actions can't reach. Disabled by default. */
71
+ enableJavaScriptTool?: boolean;
72
+ /** The task for the agent, in plain English. Scope it tightly — say exactly
73
+ * what to act on and what outcome is expected. */
74
+ task: string;
75
+ /** Appended (after a blank line) to the built-in system prompt — the place
76
+ * for caller-specific rules the agent must follow at system priority. The
77
+ * built-in prompt (browser capability + viewport size + full-autonomy
78
+ * framing) is always present and never replaced, so callers only supply
79
+ * what's specific to them. */
80
+ systemPrompt?: string;
81
+ /** JSON schema for a structured result of the run. When provided, the
82
+ * model gets a reserved `submit_result` tool with these parameters and is
83
+ * instructed to call it once when the task is done; the submission is
84
+ * validated against the schema (an invalid one is sent back for the model
85
+ * to correct), and the validated object is returned as the result's
86
+ * `output`. If the model finishes without submitting, it is nudged once;
87
+ * `output` is null when it still doesn't submit. Reserves the tool name
88
+ * "submit_result". */
89
+ outputSchema?: Record<string, unknown>;
90
+ /** Model-call cap; the run throws when exceeded (runaway guard).
91
+ * Defaults to 100. */
92
+ maxTurns?: number;
93
+ /** Overall wall-clock cap for the run. On expiry the loop is aborted (no
94
+ * further page actions) and the returned promise rejects. */
95
+ timeoutInMs?: number;
96
+ /** External abort signal; aborting stops the loop between model calls and
97
+ * before each page action. */
98
+ signal?: AbortSignal;
99
+ /** Keep raw screenshots in the returned `messages` instead of replacing
100
+ * them with "[omitted]". Defaults to false. */
101
+ showImages?: boolean;
102
+ }
103
+
104
+ /**
105
+ * Result of a completed {@link executeTask} run.
106
+ *
107
+ * @interface BrowserUseTurnResult
108
+ */
109
+ export interface BrowserUseTurnResult<TOutput = unknown> {
110
+ /** The final text of the model's last assistant message ("" when none). */
111
+ finalText: string;
112
+ /** The full run transcript (screenshots omitted unless `showImages`). */
113
+ messages: ModelMessage[];
114
+ /** The schema-validated structured result the model submitted via the
115
+ * `submit_result` tool (see `outputSchema`). Null when no `outputSchema`
116
+ * was given or the model never submitted a valid result. */
117
+ output: TOutput | null;
118
+ /** Summed per-request cost of the run in cents, read from the Intuned AI
119
+ * gateway's `x-ai-cost-in-cents` response header (0 when unavailable,
120
+ * e.g. when calling a provider directly). */
121
+ totalCostInCents: number;
122
+ /** Number of model calls the run made. */
123
+ turns: number;
124
+ /** Token usage summed over the run's model calls (input, output, and total tokens). */
125
+ usage: { inputTokens: number; outputTokens: number; totalTokens: number };
126
+ }
127
+
128
+ /**
129
+ * Run a browser-use agent (BETA) that drives a Playwright page: screenshots
130
+ * in, click/type/scroll actions out, executed on the exact page you pass in.
131
+ * Built on the AI SDK, so any vision-capable model the SDK's AI gateway
132
+ * supports (Anthropic, OpenAI, Google) can drive the browser — not just
133
+ * models with a provider-specific "computer use" tool.
134
+ *
135
+ * ## Features and limitations
136
+ * - Composes the system and user messages for you — pass a plain-English
137
+ * `task` (and optionally extra system-priority rules via `systemPrompt`).
138
+ * - Structured results: pass an `outputSchema` (JSON schema) and the run
139
+ * returns a schema-validated `output` object the model submits via a
140
+ * reserved `submit_result` tool when the task is done.
141
+ * - Built-in function tools beyond the standard actions (click, double_click,
142
+ * scroll, type, keypress, wait, move, drag): `screenshot` (see the page),
143
+ * `list_tabs` (check whether a popup tab opened), `clear_field`
144
+ * (deterministically empty an input) and `fill_field` (click, clear and
145
+ * type into a field in one action). Add your own via `tools`.
146
+ * - Every action reports the browser events it triggered — file downloads,
147
+ * popup tabs opening or closing — plus the current URL back to the model,
148
+ * so out-of-viewport side effects aren't invisible.
149
+ * - Actions return text and run one at a time in the order the model issued
150
+ * them; the agent calls `screenshot` to see the page (`scroll` and `wait`
151
+ * return one). Screenshots are one pixel per CSS pixel, viewport-synced, so
152
+ * model coordinates and mouse coordinates share one space — safe on HiDPI
153
+ * headful browsers.
154
+ * - Actions wait for the network they trigger to settle, and scrolling is
155
+ * slow and stepped so lazy-loaded content renders instead of being skipped.
156
+ * - Credentials resolve like the rest of the SDK: explicit `apiKey`, else
157
+ * the provider's env var, else the Intuned AI gateway. Per-request gateway
158
+ * cost is summed into the result's `totalCostInCents`.
159
+ * - BETA: the API surface may change between minor versions.
160
+ *
161
+ * @param {ExecuteTaskOptions} options - Task and agent configuration.
162
+ * @returns {Promise<BrowserUseTurnResult>} The completed run's result.
163
+ * @example
164
+ * ```typescript Run a task on the current page
165
+ * import { executeTask } from "@intuned/browser-use";
166
+ * import { BrowserContext, Page } from "playwright";
167
+ *
168
+ * interface Params {}
169
+ *
170
+ * export default async function handler(params: Params, page: Page, context: BrowserContext) {
171
+ * await page.goto("https://sandbox.intuned.dev/infinite-scroll");
172
+ * const result = await executeTask({
173
+ * page,
174
+ * task: "Scroll down until the year 2021 is visible and report the first 2021 entry's title.",
175
+ * maxTurns: 20,
176
+ * timeoutInMs: 120_000,
177
+ * });
178
+ * return { answer: result.finalText, costInCents: result.totalCostInCents };
179
+ * }
180
+ * ```
181
+ * @example
182
+ * ```typescript Custom tool and system prompt
183
+ * import { executeTask } from "@intuned/browser-use";
184
+ * import { BrowserContext, Page } from "playwright";
185
+ *
186
+ * interface Params {}
187
+ *
188
+ * export default async function handler(params: Params, page: Page, context: BrowserContext) {
189
+ * const result = await executeTask({
190
+ * page,
191
+ * task: "Use the get_reference_code tool, type the code into the Lookup field, and click Search.",
192
+ * systemPrompt:
193
+ * "- Act ONLY on the current screen of the active tab. Never navigate to another URL.",
194
+ * tools: [
195
+ * {
196
+ * name: "get_reference_code",
197
+ * description: "Get the reference code to type into the lookup field.",
198
+ * parameters: { type: "object", properties: {}, additionalProperties: false, required: [] },
199
+ * handler: () => ({ code: "REF-1234" }),
200
+ * },
201
+ * ],
202
+ * });
203
+ * return { answer: result.finalText };
204
+ * }
205
+ * ```
206
+ */
207
+ export declare function executeTask<TOutput = unknown>(
208
+ options: ExecuteTaskOptions
209
+ ): Promise<BrowserUseTurnResult<TOutput>>;
210
+
211
+ /**
212
+ * Thrown when a run doesn't finish cleanly (timeout or maxTurns exceeded).
213
+ * Carries spend/progress up to the point of failure — the ONLY way to read
214
+ * it in these cases, since there is no persistent agent object to poll and
215
+ * no successful {@link BrowserUseTurnResult} to return.
216
+ *
217
+ * @class BrowserUseError
218
+ */
219
+ export declare class BrowserUseError extends Error {
220
+ /** Accumulated cost (in cents) up to the point of failure. */
221
+ readonly totalCostInCents: number;
222
+ /** Model calls completed before the failure. */
223
+ readonly turns: number;
224
+ constructor(
225
+ message: string,
226
+ info: { totalCostInCents: number; turns: number }
227
+ );
228
+ }