@boxfab/desktop 0.0.0-stage → 0.1.0-alpha.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/LICENSE ADDED
@@ -0,0 +1,10 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2025 FOUNDRYLABS, INC.
4
+ Copyright (c) 2026 ContextFab, Inc.
5
+
6
+ Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
7
+
8
+ The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
9
+
10
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
package/README.md CHANGED
@@ -1,3 +1,276 @@
1
- # Temporary Holding Version
1
+ # BoxFab Desktop (JavaScript / TypeScript)
2
2
 
3
- This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
3
+ A virtual Linux desktop in a [BoxFab](https://boxfab.io) sandbox, for computer-use agents: mouse
4
+ and keyboard control, screenshots, application windows, and a live stream of the screen in the
5
+ browser.
6
+
7
+ ## Getting started
8
+
9
+ ### 1. Get a BoxFab API key
10
+
11
+ Get a key from the [BoxFab dashboard](https://dashboard.boxfab.io) and set it as
12
+ `BOXFAB_API_KEY`. The SDK reads only `BOXFAB_*` variables.
13
+
14
+ ### 2. Install the SDK
15
+
16
+ ```bash
17
+ npm install @boxfab/desktop
18
+ ```
19
+
20
+ > **Stream access.** `desktop.stream.start()` runs without a password unless you pass
21
+ > `requireAuth: true`. The stream URL contains the sandbox ID, and anyone who has the URL can see
22
+ > and control the desktop. Pass `requireAuth: true` whenever the URL leaves your backend; the auth
23
+ > key is then part of the URL returned by `getUrl({ authKey })`, so share that URL only with the
24
+ > intended viewer.
25
+
26
+ ### 3. Create Desktop Sandbox
27
+
28
+ ```javascript
29
+ import { Sandbox } from '@boxfab/desktop'
30
+
31
+ // Start a new desktop sandbox
32
+ const desktop = await Sandbox.create()
33
+
34
+ // Launch an application
35
+ await desktop.launch('google-chrome') // or vscode, firefox, etc.
36
+
37
+ // Wait 10s for the application to open
38
+ await desktop.wait(10000)
39
+
40
+ // Stream the application's window
41
+ // Note: there can be only one stream at a time
42
+ // You need to stop the current stream before streaming another application
43
+ await desktop.stream.start({
44
+ windowId: await desktop.getCurrentWindowId(), // if not provided the whole desktop will be streamed
45
+ requireAuth: true,
46
+ })
47
+
48
+ // Get the stream auth key
49
+ const authKey = desktop.stream.getAuthKey()
50
+
51
+ // Print the stream URL
52
+ console.log('Stream URL:', desktop.stream.getUrl({ authKey }))
53
+
54
+ // Kill the sandbox after the tasks are finished
55
+ // await desktop.kill()
56
+ ```
57
+
58
+ ### 4. Bind the configuration to a client
59
+
60
+ The top-level `Sandbox` export reads its configuration from the environment variables. To use an explicit configuration — e.g. several API keys or domains in one process — create a `BoxFab` client and use the resources it exposes:
61
+
62
+ ```javascript
63
+ import { BoxFab } from '@boxfab/desktop'
64
+
65
+ const client = new BoxFab({ apiKey: 'e2b_***', domain: 'boxfab.io' })
66
+
67
+ const desktop = await client.Sandbox.create()
68
+ await desktop.stream.start()
69
+
70
+ // The core resources are bound to the client's configuration as well.
71
+ const volume = await client.Volume.create('my-volume')
72
+ const exists = await client.Template.exists('my-template')
73
+ await client.Secret.create('openai-api-key', 'sk-***')
74
+
75
+ // The classes can be destructured and used like the top-level ones.
76
+ const { Sandbox } = client
77
+ const paginator = Sandbox.list()
78
+ ```
79
+
80
+ Per-call options still take precedence over the client's options, and clients are isolated from each other and from the env-configured top-level exports.
81
+
82
+ ## Features
83
+
84
+ ### Streaming desktop's screen
85
+
86
+ ```javascript
87
+ import { Sandbox } from '@boxfab/desktop'
88
+
89
+ const desktop = await Sandbox.create()
90
+
91
+ // Start the stream
92
+ await desktop.stream.start()
93
+
94
+ // Get stream URL
95
+ const url = desktop.stream.getUrl()
96
+ console.log(url)
97
+
98
+ // Stop the stream
99
+ await desktop.stream.stop()
100
+ ```
101
+
102
+ ### Streaming with password protection
103
+
104
+ ```javascript
105
+ import { Sandbox } from '@boxfab/desktop'
106
+
107
+ const desktop = await Sandbox.create()
108
+
109
+ // Start the stream
110
+ await desktop.stream.start({
111
+ requireAuth: true, // Enable authentication with an auto-generated key
112
+ })
113
+
114
+ // Retrieve the authentication key
115
+ const authKey = await desktop.stream.getAuthKey()
116
+
117
+ // Get stream URL
118
+ const url = desktop.stream.getUrl({ authKey })
119
+ console.log(url)
120
+
121
+ // Stop the stream
122
+ await desktop.stream.stop()
123
+ ```
124
+
125
+ ### Streaming specific application
126
+
127
+ > [!WARNING]
128
+ >
129
+ > - Will throw an error if the desired application is not open yet
130
+ > - The stream will close once the application closes
131
+ > - Creating multiple streams at the same time is not supported, you may have to stop the current stream and start a new one for each application
132
+
133
+ ```javascript
134
+ import { Sandbox } from '@boxfab/desktop'
135
+
136
+ const desktop = await Sandbox.create()
137
+
138
+ // Get current (active) window ID
139
+ const windowId = await desktop.getCurrentWindowId()
140
+
141
+ // Get all windows of the application
142
+ const windowIds = await desktop.getApplicationWindows('Firefox')
143
+
144
+ // Start the stream
145
+ await desktop.stream.start({ windowId: windowIds[0] })
146
+
147
+ // Stop the stream
148
+ await desktop.stream.stop()
149
+ ```
150
+
151
+ ### Mouse control
152
+
153
+ ```javascript
154
+ import { Sandbox } from '@boxfab/desktop'
155
+
156
+ const desktop = await Sandbox.create()
157
+
158
+ await desktop.doubleClick()
159
+ await desktop.leftClick()
160
+ await desktop.leftClick(100, 200)
161
+ await desktop.rightClick()
162
+ await desktop.rightClick(100, 200)
163
+ await desktop.middleClick()
164
+ await desktop.middleClick(100, 200)
165
+ await desktop.scroll(10) // Scroll by the amount. Positive for up, negative for down.
166
+ await desktop.moveMouse(100, 200) // Move to x, y coordinates
167
+ await desktop.drag([100, 100], [200, 200]) // Drag using the mouse
168
+ await desktop.mousePress('left') // Press the mouse button
169
+ await desktop.mouseRelease('left') // Release the mouse button
170
+ ```
171
+
172
+ ### Keyboard control
173
+
174
+ ```javascript
175
+ import { Sandbox } from '@boxfab/desktop'
176
+
177
+ const desktop = await Sandbox.create()
178
+
179
+ // Write text at the current cursor position with customizable typing speed
180
+ await desktop.write('Hello, world!')
181
+ await desktop.write('Fast typing!', { chunkSize: 50, delayInMs: 25 }) // Faster typing
182
+
183
+ // Press keys
184
+ await desktop.press('enter')
185
+ await desktop.press('space')
186
+ await desktop.press('backspace')
187
+ await desktop.press(['ctrl', 'c']) // Key combination
188
+ ```
189
+
190
+ ### Window control
191
+
192
+ ```javascript
193
+ import { Sandbox } from '@boxfab/desktop'
194
+
195
+ const desktop = await Sandbox.create()
196
+
197
+ // Get current (active) window ID
198
+ const windowId = await desktop.getCurrentWindowId()
199
+
200
+ // Get all windows of the application
201
+ const windowIds = await desktop.getApplicationWindows('Firefox')
202
+
203
+ // Get window title
204
+ const title = await desktop.getWindowTitle(windowId)
205
+ ```
206
+
207
+ ### Screenshot
208
+
209
+ ```javascript
210
+ import { Sandbox } from '@boxfab/desktop'
211
+
212
+ const desktop = await Sandbox.create()
213
+ const image = await desktop.screenshot()
214
+ // Save the image to a file
215
+ fs.writeFileSync('screenshot.png', image)
216
+ ```
217
+
218
+ ### Open file
219
+
220
+ ```javascript
221
+ import { Sandbox } from '@boxfab/desktop'
222
+
223
+ const desktop = await Sandbox.create()
224
+
225
+ // Open file with default application
226
+ await desktop.files.write('/home/user/index.js', "console.log('hello')") // First create the file
227
+ await desktop.open('/home/user/index.js') // Then open it
228
+ ```
229
+
230
+ ### Launch applications
231
+
232
+ ```javascript
233
+ import { Sandbox } from '@boxfab/desktop'
234
+
235
+ const desktop = await Sandbox.create()
236
+
237
+ // Launch the application
238
+ await desktop.launch('google-chrome')
239
+ ```
240
+
241
+ ### Run any bash commands
242
+
243
+ ```javascript
244
+ import { Sandbox } from '@boxfab/desktop'
245
+
246
+ const desktop = await Sandbox.create()
247
+
248
+ // Run any bash command
249
+ const out = await desktop.commands.run('ls -la /home/user')
250
+ console.log(out)
251
+ ```
252
+
253
+ ### Wait
254
+
255
+ ```javascript
256
+ import { Sandbox } from '@boxfab/desktop'
257
+
258
+ const desktop = await Sandbox.create()
259
+ await desktop.wait(1000) // Wait for 1 second
260
+ ```
261
+
262
+ ## Under the hood
263
+
264
+ The desktop-like environment is based on Linux and [Xfce](https://www.xfce.org/) at the moment. We chose Xfce because it's a fast and lightweight environment that's also popular and actively supported. However, this Sandbox template is fully customizable and you can create your own desktop environment.
265
+ BoxFab's template is in [boxfab/desktop](https://github.com/boxfab/desktop/tree/main/template), generated from [e2b-dev/desktop](https://github.com/e2b-dev/desktop/tree/main/template).
266
+
267
+ BoxFab sandboxes are [Firecracker](https://firecracker-microvm.github.io/) microVMs, run on the
268
+ open-source [E2B](https://github.com/e2b-dev/infra) sandbox infrastructure. This package is derived
269
+ from the open-source E2B Desktop SDK;
270
+ [UPSTREAM.md](https://github.com/boxfab/js-sdk/blob/main/UPSTREAM.md) lists every change.
271
+
272
+ ## License
273
+
274
+ MIT, as the E2B Desktop SDK it is derived from; see [LICENSE](LICENSE).
275
+
276
+ BoxFab is a product of ContextFab, Inc.
@@ -0,0 +1,324 @@
1
+ import { BoxFabClientOpts, CommandResult, Sandbox as Sandbox$1, SandboxOpts, Secret, Template, Volume } from "boxfab";
2
+ export * from "boxfab";
3
+ interface CursorPosition {
4
+ x: number;
5
+ y: number;
6
+ }
7
+ interface ScreenSize {
8
+ width: number;
9
+ height: number;
10
+ }
11
+ /**
12
+ * Configuration options for the Sandbox environment.
13
+ * @interface SandboxOpts
14
+ * @extends {SandboxOptsBase}
15
+ */
16
+ interface SandboxOpts$1 extends SandboxOpts {
17
+ /**
18
+ * The screen resolution in pixels, specified as [width, height].
19
+ * @type {[number, number]}
20
+ */
21
+ resolution?: [number, number];
22
+ /**
23
+ * Dots per inch (DPI) setting for the display.
24
+ * @type {number}
25
+ */
26
+ dpi?: number;
27
+ /**
28
+ * Display identifier.
29
+ * @type {string}
30
+ */
31
+ display?: string;
32
+ }
33
+ declare class Sandbox extends Sandbox$1 {
34
+ protected static readonly defaultTemplate: string;
35
+ display: string;
36
+ stream: VNCServer;
37
+ private lastXfce4Pid;
38
+ /**
39
+ * Use {@link Sandbox.create} to create a new Sandbox instead.
40
+ *
41
+ * @hidden
42
+ * @hide
43
+ * @internal
44
+ * @access protected
45
+ */
46
+ constructor(opts: Omit<SandboxOpts$1, 'timeoutMs' | 'metadata'> & {
47
+ sandboxId: string;
48
+ envdVersion: string;
49
+ });
50
+ /**
51
+ * Create a new sandbox from the default `desktop` sandbox template.
52
+ *
53
+ * @param opts connection options.
54
+ *
55
+ * @returns sandbox instance for the new sandbox.
56
+ *
57
+ * @example
58
+ * ```ts
59
+ * const sandbox = await Sandbox.create()
60
+ * ```
61
+ * @constructs Sandbox
62
+ */
63
+ static create<S extends typeof Sandbox>(this: S, opts?: SandboxOpts$1): Promise<InstanceType<S>>;
64
+ /**
65
+ * Create a new sandbox from the specified sandbox template.
66
+ *
67
+ * @param template sandbox template name or ID.
68
+ * @param opts connection options.
69
+ *
70
+ * @returns sandbox instance for the new sandbox.
71
+ *
72
+ * @example
73
+ * ```ts
74
+ * const sandbox = await Sandbox.create('<template-name-or-id>')
75
+ * ```
76
+ * @constructs Sandbox
77
+ */
78
+ static create<S extends typeof Sandbox>(this: S, template: string, opts?: SandboxOpts$1): Promise<InstanceType<S>>;
79
+ /**
80
+ * Wait for a command to return a specific result.
81
+ * @param cmd - The command to run.
82
+ * @param onResult - The function to check the result of the command.
83
+ * @param timeout - The maximum time to wait for the command to return the result.
84
+ * @param interval - The interval to wait between checks.
85
+ * @returns `true` if the command returned the result within the timeout, otherwise `false`.
86
+ */
87
+ waitAndVerify(cmd: string, onResult: (result: CommandResult) => boolean, timeout?: number, interval?: number): Promise<boolean>;
88
+ /**
89
+ * Take a screenshot and save it to the given name.
90
+ * @param format - The format of the screenshot.
91
+ * @returns A Uint8Array bytes representation of the screenshot.
92
+ */
93
+ screenshot(): Promise<Uint8Array>;
94
+ /**
95
+ * Take a screenshot and save it to the given name.
96
+ * @param format - The format of the screenshot.
97
+ * @returns A Uint8Array bytes representation of the screenshot.
98
+ */
99
+ screenshot(format: 'bytes'): Promise<Uint8Array>;
100
+ /**
101
+ * Take a screenshot and save it to the given name.
102
+ * @returns A Blob representation of the screenshot.
103
+ */
104
+ screenshot(format: 'blob'): Promise<Blob>;
105
+ /**
106
+ * Take a screenshot and save it to the given name.
107
+ * @returns A ReadableStream of bytes representation of the screenshot.
108
+ */
109
+ screenshot(format: 'stream'): Promise<ReadableStream<Uint8Array>>;
110
+ /**
111
+ * Left click on the mouse position.
112
+ */
113
+ leftClick(x?: number, y?: number): Promise<void>;
114
+ /**
115
+ * Double left click on the mouse position.
116
+ */
117
+ doubleClick(x?: number, y?: number): Promise<void>;
118
+ /**
119
+ * Right click on the mouse position.
120
+ */
121
+ rightClick(x?: number, y?: number): Promise<void>;
122
+ /**
123
+ * Middle click on the mouse position.
124
+ */
125
+ middleClick(x?: number, y?: number): Promise<void>;
126
+ /**
127
+ * Scroll the mouse wheel by the given amount.
128
+ * @param direction - The direction to scroll. Can be "up" or "down".
129
+ * @param amount - The amount to scroll.
130
+ */
131
+ scroll(direction?: 'up' | 'down', amount?: number): Promise<void>;
132
+ /**
133
+ * Move the mouse to the given coordinates.
134
+ * @param x - The x coordinate.
135
+ * @param y - The y coordinate.
136
+ */
137
+ moveMouse(x: number, y: number): Promise<void>;
138
+ /**
139
+ * Press the mouse button.
140
+ */
141
+ mousePress(button?: 'left' | 'right' | 'middle'): Promise<void>;
142
+ /**
143
+ * Release the mouse button.
144
+ */
145
+ mouseRelease(button?: 'left' | 'right' | 'middle'): Promise<void>;
146
+ /**
147
+ * Get the current cursor position.
148
+ * @returns A object with the x and y coordinates
149
+ * @throws Error if cursor position cannot be determined
150
+ */
151
+ getCursorPosition(): Promise<CursorPosition>;
152
+ /**
153
+ * Get the current screen size.
154
+ * @returns An {@link ScreenSize} object
155
+ * @throws Error if screen size cannot be determined
156
+ */
157
+ getScreenSize(): Promise<ScreenSize>;
158
+ /**
159
+ * Write the given text at the current cursor position.
160
+ * @param text - The text to write.
161
+ * @param options - An object containing the chunk size and delay between each chunk of text.
162
+ * @param options.chunkSize - The size of each chunk of text to write. Default is 25 characters.
163
+ * @param options.delayInMs - The delay between each chunk of text. Default is 75 ms.
164
+ */
165
+ write(text: string, options?: {
166
+ chunkSize: number;
167
+ delayInMs: number;
168
+ }): Promise<void>;
169
+ /**
170
+ * Press a key.
171
+ * @param key - The key to press (e.g. "enter", "space", "backspace", etc.). Can be a single key or an array of keys.
172
+ */
173
+ press(key: string | string[]): Promise<void>;
174
+ /**
175
+ * Drag the mouse from the given position to the given position.
176
+ * @param from - The starting position as `[x, y]`.
177
+ * @param to - The ending position as `[x, y]`.
178
+ */
179
+ drag(from: [number, number], to: [number, number]): Promise<void>;
180
+ /**
181
+ * Wait for the given amount of time.
182
+ * @param ms - The amount of time to wait in milliseconds.
183
+ */
184
+ wait(ms: number): Promise<void>;
185
+ /**
186
+ * Open a file or a URL in the default application.
187
+ * @param fileOrUrl - The file or URL to open.
188
+ */
189
+ open(fileOrUrl: string): Promise<void>;
190
+ /**
191
+ * Get the current window ID.
192
+ * @returns The ID of the current window.
193
+ */
194
+ getCurrentWindowId(): Promise<string>;
195
+ /**
196
+ * Get the window ID of the window with the given title.
197
+ * @param title - The title of the window.
198
+ * @returns The ID of the window.
199
+ */
200
+ getApplicationWindows(application: string): Promise<string[]>;
201
+ /**
202
+ * Get the title of the window with the given ID.
203
+ * @param windowId - The ID of the window.
204
+ * @returns The title of the window.
205
+ */
206
+ getWindowTitle(windowId: string): Promise<string>;
207
+ /**
208
+ * Launch an application.
209
+ * @param application - The application to launch.
210
+ * @param uri - The URI to open in the application.
211
+ */
212
+ launch(application: string, uri?: string): Promise<void>;
213
+ protected _start(display: string, opts?: SandboxOpts$1): Promise<void>;
214
+ /**
215
+ * Start xfce4 session if logged out or not running.
216
+ */
217
+ private startXfce4;
218
+ private breakIntoChunks;
219
+ private quoteString;
220
+ }
221
+ interface VNCServerOptions {
222
+ vncPort?: number;
223
+ port?: number;
224
+ requireAuth?: boolean;
225
+ windowId?: string;
226
+ }
227
+ interface UrlOptions {
228
+ autoConnect?: boolean;
229
+ viewOnly?: boolean;
230
+ resize?: 'off' | 'scale' | 'remote';
231
+ authKey?: string;
232
+ }
233
+ declare class VNCServer {
234
+ private vncPort;
235
+ private port;
236
+ private novncAuthEnabled;
237
+ private url;
238
+ private novncHandle;
239
+ private password;
240
+ private readonly novncCommand;
241
+ private readonly desktop;
242
+ constructor(desktop: Sandbox);
243
+ getAuthKey(): string;
244
+ /**
245
+ * Get the URL to a web page with a stream of the desktop sandbox.
246
+ * @param autoConnect - Whether to automatically connect to the server after opening the URL.
247
+ * @param viewOnly - Whether to prevent user interaction through the client.
248
+ * @param resize - Whether to resize the view when the window resizes.
249
+ * @param authKey - The password to use to connect to the server.
250
+ * @returns The URL to connect to the VNC server.
251
+ */
252
+ getUrl({ autoConnect, viewOnly, resize, authKey }?: UrlOptions): string;
253
+ /**
254
+ * Start the VNC server.
255
+ */
256
+ start(opts?: VNCServerOptions): Promise<void>;
257
+ /**
258
+ * Stop the VNC server.
259
+ */
260
+ stop(): Promise<void>;
261
+ /**
262
+ * Set the VNC command to start the VNC server.
263
+ */
264
+ private getVNCCommand;
265
+ private waitForPort;
266
+ /**
267
+ * Check if the VNC server is running.
268
+ * @returns Whether the VNC server is running.
269
+ */
270
+ private checkVNCRunning;
271
+ }
272
+ /**
273
+ * BoxFab client with an explicitly bound connection configuration.
274
+ *
275
+ * The resources exposed by the client ({@link BoxFab.Sandbox},
276
+ * {@link BoxFab.Volume}, {@link BoxFab.Template}, {@link BoxFab.Secret}) behave exactly
277
+ * like the top-level `Sandbox` / `Volume` / `Template` / `Secret` exports,
278
+ * except the options passed to the client are used as the defaults instead of
279
+ * the environment variables.
280
+ * Per-call options still take precedence over the client's options.
281
+ *
282
+ * Multiple clients are fully isolated from each other and from the top-level
283
+ * env-configured exports.
284
+ *
285
+ * @example
286
+ * ```ts
287
+ * import { BoxFab } from '@boxfab/desktop'
288
+ *
289
+ * const client = new BoxFab({ apiKey: 'e2b_...', domain: 'boxfab.io' })
290
+ *
291
+ * const desktop = await client.Sandbox.create()
292
+ * await desktop.stream.start()
293
+ * ```
294
+ */
295
+ declare class BoxFab {
296
+ /**
297
+ * Desktop `Sandbox` class bound to this client's connection configuration.
298
+ */
299
+ readonly Sandbox: typeof Sandbox;
300
+ /**
301
+ * `Volume` class bound to this client's connection configuration.
302
+ */
303
+ readonly Volume: typeof Volume;
304
+ /**
305
+ * `Template` bound to this client's connection configuration. Both the
306
+ * builder (`client.Template()`) and the statics
307
+ * (`client.Template.build(...)`, `client.Template.exists(...)`, …) work like
308
+ * the top-level `Template`.
309
+ */
310
+ readonly Template: typeof Template;
311
+ /**
312
+ * `Secret` class bound to this client's connection configuration.
313
+ */
314
+ readonly Secret: typeof Secret;
315
+ /**
316
+ * Create a new client with the connection options bound to it.
317
+ *
318
+ * @param opts connection options used as the defaults for every call made
319
+ * through this client's resource classes.
320
+ */
321
+ constructor(opts?: BoxFabClientOpts);
322
+ }
323
+ export { BoxFab, BoxFab as E2B, type BoxFabClientOpts, type BoxFabClientOpts as E2BClientOpts, Sandbox };
324
+ //# sourceMappingURL=index.d.mts.map