@componentor/quickjs-emscripten 0.31.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md ADDED
@@ -0,0 +1,925 @@
1
+ # quickjs-emscripten
2
+
3
+ Javascript/Typescript bindings for QuickJS, a modern Javascript interpreter,
4
+ compiled to WebAssembly.
5
+
6
+ - Safely evaluate untrusted Javascript (supports [most of ES2023](https://test262.fyi/#|qjs,qjs_ng)).
7
+ - Create and manipulate values inside the QuickJS runtime ([more][values]).
8
+ - Expose host functions to the QuickJS runtime ([more][functions]).
9
+ - Execute synchronous code that uses asynchronous functions, with [asyncify][asyncify].
10
+ - Supports browsers, NodeJS, Deno, Bun, Cloudflare Workers, QuickJS (via [quickjs-for-quickjs][]).
11
+
12
+ [Github] | [NPM] | [API Documentation][api] | [Variants][core] | [Examples][tests]
13
+
14
+ ```typescript
15
+ import { getQuickJS } from "quickjs-emscripten"
16
+
17
+ async function main() {
18
+ const QuickJS = await getQuickJS()
19
+ const vm = QuickJS.newContext()
20
+
21
+ const world = vm.newString("world")
22
+ vm.setProp(vm.global, "NAME", world)
23
+ world.dispose()
24
+
25
+ const result = vm.evalCode(`"Hello " + NAME + "!"`)
26
+ if (result.error) {
27
+ console.log("Execution failed:", vm.dump(result.error))
28
+ result.error.dispose()
29
+ } else {
30
+ console.log("Success:", vm.dump(result.value))
31
+ result.value.dispose()
32
+ }
33
+
34
+ vm.dispose()
35
+ }
36
+
37
+ main()
38
+ ```
39
+
40
+ [github]: https://github.com/componentor/quickjs-emscripten
41
+ [npm]: https://www.npmjs.com/package/quickjs-emscripten
42
+ [api]: https://github.com/componentor/quickjs-emscripten/blob/main/doc/packages.md
43
+ [tests]: https://github.com/componentor/quickjs-emscripten/blob/main/packages/quickjs-emscripten/src/quickjs.test.ts
44
+ [values]: #interfacing-with-the-interpreter
45
+ [asyncify]: #asyncify
46
+ [functions]: #exposing-apis
47
+ [quickjs-for-quickjs]: https://github.com/componentor/quickjs-emscripten/blob/main/packages/quickjs-for-quickjs
48
+
49
+ - [quickjs-emscripten](#quickjs-emscripten)
50
+ - [Usage](#usage)
51
+ - [Safely evaluate Javascript code](#safely-evaluate-javascript-code)
52
+ - [Interfacing with the interpreter](#interfacing-with-the-interpreter)
53
+ - [Runtime](#runtime)
54
+ - [EcmaScript Module Exports](#ecmascript-module-exports)
55
+ - [Memory Management](#memory-management)
56
+ - [`using` statement](#using-statement)
57
+ - [Scope](#scope)
58
+ - [`Lifetime.consume(fn)`](#lifetimeconsumefn)
59
+ - [Exposing APIs](#exposing-apis)
60
+ - [Promises](#promises)
61
+ - [context.getPromiseState(handle)](#contextgetpromisestatehandle)
62
+ - [context.resolvePromise(handle)](#contextresolvepromisehandle)
63
+ - [Asyncify](#asyncify)
64
+ - [Async module loader](#async-module-loader)
65
+ - [Async on host, sync in QuickJS](#async-on-host-sync-in-quickjs)
66
+ - [Testing your code](#testing-your-code)
67
+ - [Packaging](#packaging)
68
+ - [Reducing package size](#reducing-package-size)
69
+ - [WebAssembly loading](#webassembly-loading)
70
+ - [quickjs-ng](#quickjs-ng)
71
+ - [Using in the browser without a build step](#using-in-the-browser-without-a-build-step)
72
+ - [Debugging](#debugging)
73
+ - [Supported Platforms](#supported-platforms)
74
+ - [More Documentation](#more-documentation)
75
+ - [Background](#background)
76
+ - [Status \& Roadmap](#status--roadmap)
77
+ - [Related](#related)
78
+ - [Developing](#developing)
79
+ - [The C parts](#the-c-parts)
80
+ - [The Typescript parts](#the-typescript-parts)
81
+ - [pnpm run updates](#pnpm run-updates)
82
+
83
+ ## Usage
84
+
85
+ Install from `npm`: `npm install --save quickjs-emscripten` or `pnpm run add quickjs-emscripten`.
86
+
87
+ The root entrypoint of this library is the `getQuickJS` function, which returns
88
+ a promise that resolves to a [QuickJSWASMModule](./doc/@componentor/quickjs-emscripten/classes/QuickJSWASMModule.md) when
89
+ the QuickJS WASM module is ready.
90
+
91
+ Once `getQuickJS` has been awaited at least once, you also can use the `getQuickJSSync`
92
+ function to directly access the singleton in your synchronous code.
93
+
94
+ ### Safely evaluate Javascript code
95
+
96
+ See [QuickJSWASMModule.evalCode](https://github.com/componentor/quickjs-emscripten/blob/main/doc/@componentor/quickjs-emscripten/classes/QuickJSWASMModule.md#evalcode)
97
+
98
+ ```typescript
99
+ import { getQuickJS, shouldInterruptAfterDeadline } from "quickjs-emscripten"
100
+
101
+ getQuickJS().then((QuickJS) => {
102
+ const result = QuickJS.evalCode("1 + 1", {
103
+ shouldInterrupt: shouldInterruptAfterDeadline(Date.now() + 1000),
104
+ memoryLimitBytes: 1024 * 1024,
105
+ })
106
+ console.log(result)
107
+ })
108
+ ```
109
+
110
+ ### Interfacing with the interpreter
111
+
112
+ You can use [QuickJSContext](https://github.com/componentor/quickjs-emscripten/blob/main/doc/@componentor/quickjs-emscripten/classes/QuickJSContext.md)
113
+ to build a scripting environment by modifying globals and exposing functions
114
+ into the QuickJS interpreter.
115
+
116
+ Each `QuickJSContext` instance has its own environment -- globals, built-in
117
+ classes -- and actions from one context won't leak into other contexts or
118
+ runtimes (with one exception, see [Asyncify][asyncify]).
119
+
120
+ Every context is created inside a
121
+ [QuickJSRuntime](https://github.com/componentor/quickjs-emscripten/blob/main/doc/@componentor/quickjs-emscripten/classes/QuickJSRuntime.md).
122
+ A runtime represents a Javascript heap, and you can even share values between
123
+ contexts in the same runtime.
124
+
125
+ ```typescript
126
+ const vm = QuickJS.newContext()
127
+ let state = 0
128
+
129
+ const fnHandle = vm.newFunction("nextId", () => {
130
+ return vm.newNumber(++state)
131
+ })
132
+
133
+ vm.setProp(vm.global, "nextId", fnHandle)
134
+ fnHandle.dispose()
135
+
136
+ const nextId = vm.unwrapResult(vm.evalCode(`nextId(); nextId(); nextId()`))
137
+ console.log("vm result:", vm.getNumber(nextId), "native state:", state)
138
+
139
+ nextId.dispose()
140
+ vm.dispose()
141
+ ```
142
+
143
+ When you create a context from a top-level API like in the example above,
144
+ instead of by calling `runtime.newContext()`, a runtime is automatically created
145
+ for the lifetime of the context, and disposed of when you dispose the context.
146
+
147
+ #### Runtime
148
+
149
+ The runtime has APIs for CPU and memory limits that apply to all contexts within
150
+ the runtime in aggregate. You can also use the runtime to configure EcmaScript
151
+ module loading.
152
+
153
+ ```typescript
154
+ const runtime = QuickJS.newRuntime()
155
+ // "Should be enough for everyone" -- attributed to B. Gates
156
+ runtime.setMemoryLimit(1024 * 640)
157
+ // Limit stack size
158
+ runtime.setMaxStackSize(1024 * 320)
159
+ // Interrupt computation after 1024 calls to the interrupt handler
160
+ let interruptCycles = 0
161
+ runtime.setInterruptHandler(() => ++interruptCycles > 1024)
162
+ // Toy module system that always returns the module name
163
+ // as the default export
164
+ runtime.setModuleLoader((moduleName) => `export default '${moduleName}'`)
165
+ const context = runtime.newContext()
166
+ const ok = context.evalCode(`
167
+ import fooName from './foo.js'
168
+ globalThis.result = fooName
169
+ `)
170
+ context.unwrapResult(ok).dispose()
171
+ // logs "foo.js"
172
+ console.log(context.getProp(context.global, "result").consume(context.dump))
173
+ context.dispose()
174
+ runtime.dispose()
175
+ ```
176
+
177
+ #### EcmaScript Module Exports
178
+
179
+ When you evaluate code as an ES Module, the result will be a handle to the
180
+ module's exports, or a handle to a promise that resolves to the module's
181
+ exports if the module depends on a top-level await.
182
+
183
+ ```typescript
184
+ const context = QuickJS.newContext()
185
+ const result = context.evalCode(
186
+ `
187
+ export const name = 'Jake'
188
+ export const favoriteBean = 'wax bean'
189
+ export default 'potato'
190
+ `,
191
+ "jake.js",
192
+ { type: "module" },
193
+ )
194
+ const moduleExports = context.unwrapResult(result)
195
+ console.log(context.dump(moduleExports))
196
+ // -> { name: 'Jake', favoriteBean: 'wax bean', default: 'potato' }
197
+ moduleExports.dispose()
198
+ ```
199
+
200
+ ### Memory Management
201
+
202
+ Many methods in this library return handles to memory allocated inside the
203
+ WebAssembly heap. These types cannot be garbage-collected as usual in
204
+ Javascript. Instead, you must manually manage their memory by calling a
205
+ `.dispose()` method to free the underlying resources. Once a handle has been
206
+ disposed, it cannot be used anymore. Note that in the example above, we call
207
+ `.dispose()` on each handle once it is no longer needed.
208
+
209
+ Calling `QuickJSContext.dispose()` will throw a RuntimeError if you've forgotten to
210
+ dispose any handles associated with that VM, so it's good practice to create a
211
+ new VM instance for each of your tests, and to call `vm.dispose()` at the end
212
+ of every test.
213
+
214
+ ```typescript
215
+ const vm = QuickJS.newContext()
216
+ const numberHandle = vm.newNumber(42)
217
+ // Note: numberHandle not disposed, so it leaks memory.
218
+ vm.dispose()
219
+ // throws RuntimeError: abort(Assertion failed: list_empty(&rt->gc_obj_list), at: quickjs/quickjs.c,1963,JS_FreeRuntime)
220
+ ```
221
+
222
+ Here are some strategies to reduce the toil of calling `.dispose()` on each
223
+ handle you create:
224
+
225
+ #### `using` statement
226
+
227
+ The `using` statement is a Stage 3 (as of 2023-12-29) proposal for Javascript that declares a constant variable and automatically calls the `[Symbol.dispose]()` method of an object when it goes out of scope. Read more [in this Typescript release announcement][using]. Here's the "Interfacing with the interpreter" example re-written using `using`:
228
+
229
+ ```typescript
230
+ using vm = QuickJS.newContext()
231
+ let state = 0
232
+
233
+ // The block here isn't needed for correctness, but it shows
234
+ // how to get a tighter bound on the lifetime of `fnHandle`.
235
+ {
236
+ using fnHandle = vm.newFunction("nextId", () => {
237
+ return vm.newNumber(++state)
238
+ })
239
+
240
+ vm.setProp(vm.global, "nextId", fnHandle)
241
+ // fnHandle.dispose() is called automatically when the block exits
242
+ }
243
+
244
+ using nextId = vm.unwrapResult(vm.evalCode(`nextId(); nextId(); nextId()`))
245
+ console.log("vm result:", vm.getNumber(nextId), "native state:", state)
246
+ // nextId.dispose() is called automatically when the block exits
247
+ // vm.dispose() is called automatically when the block exits
248
+ ```
249
+
250
+ [using]: https://www.typescriptlang.org/docs/handbook/release-notes/typescript-5-2.html#using-declarations-and-explicit-resource-management
251
+
252
+ #### Scope
253
+
254
+ A
255
+ [`Scope`](https://github.com/componentor/quickjs-emscripten/blob/main/doc/@componentor/quickjs-emscripten/classes/Scope.md#class-scope)
256
+ instance manages a set of disposables and calls their `.dispose()`
257
+ method in the reverse order in which they're added to the scope. Here's the
258
+ "Interfacing with the interpreter" example re-written using `Scope`:
259
+
260
+ ```typescript
261
+ Scope.withScope((scope) => {
262
+ const vm = scope.manage(QuickJS.newContext())
263
+ let state = 0
264
+
265
+ const fnHandle = scope.manage(
266
+ vm.newFunction("nextId", () => {
267
+ return vm.newNumber(++state)
268
+ }),
269
+ )
270
+
271
+ vm.setProp(vm.global, "nextId", fnHandle)
272
+
273
+ const nextId = scope.manage(vm.unwrapResult(vm.evalCode(`nextId(); nextId(); nextId()`)))
274
+ console.log("vm result:", vm.getNumber(nextId), "native state:", state)
275
+
276
+ // When the withScope block exits, it calls scope.dispose(), which in turn calls
277
+ // the .dispose() methods of all the disposables managed by the scope.
278
+ })
279
+ ```
280
+
281
+ You can also create `Scope` instances with `new Scope()` if you want to manage
282
+ calling `scope.dispose()` yourself.
283
+
284
+ #### `Lifetime.consume(fn)`
285
+
286
+ [`Lifetime.consume`](https://github.com/componentor/quickjs-emscripten/blob/main/doc/@componentor/quickjs-emscripten/classes/lifetime.md#consume)
287
+ is sugar for the common pattern of using a handle and then
288
+ immediately disposing of it. `Lifetime.consume` takes a `map` function that
289
+ produces a result of any type. The `map` fuction is called with the handle,
290
+ then the handle is disposed, then the result is returned.
291
+
292
+ Here's the "Interfacing with interpreter" example re-written using `.consume()`:
293
+
294
+ ```typescript
295
+ const vm = QuickJS.newContext()
296
+ let state = 0
297
+
298
+ vm.newFunction("nextId", () => {
299
+ return vm.newNumber(++state)
300
+ }).consume((fnHandle) => vm.setProp(vm.global, "nextId", fnHandle))
301
+
302
+ vm.unwrapResult(vm.evalCode(`nextId(); nextId(); nextId()`)).consume((nextId) =>
303
+ console.log("vm result:", vm.getNumber(nextId), "native state:", state),
304
+ )
305
+
306
+ vm.dispose()
307
+ ```
308
+
309
+ Generally working with `Scope` leads to more straight-forward code, but
310
+ `Lifetime.consume` can be handy sugar as part of a method call chain.
311
+
312
+ ### Exposing APIs
313
+
314
+ To add APIs inside the QuickJS environment, you'll need to [create objects][newObject] to
315
+ define the shape of your API, and [add properties][setProp] and [functions][newFunction] to those objects
316
+ to allow code inside QuickJS to call code on the host.
317
+ The [newFunction][] documentation covers writing functions in detail.
318
+
319
+ By default, no host functionality is exposed to code running inside QuickJS.
320
+
321
+ ```typescript
322
+ const vm = QuickJS.newContext()
323
+ // `console.log`
324
+ const logHandle = vm.newFunction("log", (...args) => {
325
+ const nativeArgs = args.map(vm.dump)
326
+ console.log("QuickJS:", ...nativeArgs)
327
+ })
328
+ // Partially implement `console` object
329
+ const consoleHandle = vm.newObject()
330
+ vm.setProp(consoleHandle, "log", logHandle)
331
+ vm.setProp(vm.global, "console", consoleHandle)
332
+ consoleHandle.dispose()
333
+ logHandle.dispose()
334
+
335
+ vm.unwrapResult(vm.evalCode(`console.log("Hello from QuickJS!")`)).dispose()
336
+ ```
337
+
338
+ [newObject]: https://github.com/componentor/quickjs-emscripten/blob/main/doc/@componentor/quickjs-emscripten/classes/QuickJSContext.md#newobject
339
+ [newFunction]: https://github.com/componentor/quickjs-emscripten/blob/main/doc/@componentor/quickjs-emscripten/classes/QuickJSContext.md#newfunction
340
+ [setProp]: https://github.com/componentor/quickjs-emscripten/blob/main/doc/@componentor/quickjs-emscripten/classes/QuickJSContext.md#setprop
341
+
342
+ #### Promises
343
+
344
+ To expose an asynchronous function that _returns a promise_ to callers within
345
+ QuickJS, your function can return the handle of a `QuickJSDeferredPromise`
346
+ created via `context.newPromise()`.
347
+
348
+ When you resolve a `QuickJSDeferredPromise` -- and generally whenever async
349
+ behavior completes for the VM -- pending listeners inside QuickJS may not
350
+ execute immediately. Your code needs to explicitly call
351
+ `runtime.executePendingJobs()` to resume execution inside QuickJS. This API
352
+ gives your code maximum control to _schedule_ when QuickJS will block the host's
353
+ event loop by resuming execution.
354
+
355
+ To work with QuickJS handles that contain a promise inside the environment,
356
+ there are two options:
357
+
358
+ ##### context.getPromiseState(handle)
359
+
360
+ You can synchronously peek into a QuickJS promise handle and get its state
361
+ without introducing asynchronous host code, described by the type [JSPromiseState][]:
362
+
363
+ ```typescript
364
+ type JSPromiseState =
365
+ | { type: "pending"; error: Error }
366
+ | { type: "fulfilled"; value: QuickJSHandle; notAPromise?: boolean }
367
+ | { type: "rejected"; error: QuickJSHandle }
368
+ ```
369
+
370
+ The result conforms to the `SuccessOrFail` type returned by `context.evalCode`, so you can use `context.unwrapResult(context.getPromiseState(promiseHandle))` to assert a promise is settled successfully and retrieve its value. Calling `context.unwrapResult` on a pending or rejected promise will throw an error.
371
+
372
+ ```typescript
373
+ const promiseHandle = context.evalCode(`Promise.resolve(42)`)
374
+ const resultHandle = context.unwrapResult(context.getPromiseState(promiseHandle))
375
+ context.getNumber(resultHandle) === 42 // true
376
+ resultHandle.dispose()
377
+ promiseHandle.dispose()
378
+ ```
379
+
380
+ [JSPromiseState]: https://github.com/componentor/quickjs-emscripten/blob/main/doc/@componentor/quickjs-emscripten/exports.md#jspromisestate
381
+
382
+ ##### context.resolvePromise(handle)
383
+
384
+ You can convert the QuickJSHandle into a native promise using
385
+ `context.resolvePromise()`. Take care with this API to avoid 'deadlocks' where
386
+ the host awaits a guest promise, but the guest cannot make progress until the
387
+ host calls `runtime.executePendingJobs()`. The simplest way to avoid this kind
388
+ of deadlock is to always schedule `executePendingJobs` after any promise is
389
+ settled.
390
+
391
+ ```typescript
392
+ const vm = QuickJS.newContext()
393
+ const fakeFileSystem = new Map([["example.txt", "Example file content"]])
394
+
395
+ // Function that simulates reading data asynchronously
396
+ const readFileHandle = vm.newFunction("readFile", (pathHandle) => {
397
+ const path = vm.getString(pathHandle)
398
+ const promise = vm.newPromise()
399
+ setTimeout(() => {
400
+ const content = fakeFileSystem.get(path)
401
+ promise.resolve(vm.newString(content || ""))
402
+ }, 100)
403
+ // IMPORTANT: Once you resolve an async action inside QuickJS,
404
+ // call runtime.executePendingJobs() to run any code that was
405
+ // waiting on the promise or callback.
406
+ promise.settled.then(vm.runtime.executePendingJobs)
407
+ return promise.handle
408
+ })
409
+ readFileHandle.consume((handle) => vm.setProp(vm.global, "readFile", handle))
410
+
411
+ // Evaluate code that uses `readFile`, which returns a promise
412
+ const result = vm.evalCode(`(async () => {
413
+ const content = await readFile('example.txt')
414
+ return content.toUpperCase()
415
+ })()`)
416
+ const promiseHandle = vm.unwrapResult(result)
417
+
418
+ // Convert the promise handle into a native promise and await it.
419
+ // If code like this deadlocks, make sure you are calling
420
+ // runtime.executePendingJobs appropriately.
421
+ const resolvedResult = await vm.resolvePromise(promiseHandle)
422
+ promiseHandle.dispose()
423
+ const resolvedHandle = vm.unwrapResult(resolvedResult)
424
+ console.log("Result:", vm.getString(resolvedHandle))
425
+ resolvedHandle.dispose()
426
+ ```
427
+
428
+ #### Asyncify
429
+
430
+ Sometimes, we want to create a function that's synchronous from the perspective
431
+ of QuickJS, but prefer to implement that function _asynchronously_ in your host
432
+ code. The most obvious use-case is for EcmaScript module loading. The underlying
433
+ QuickJS C library expects the module loader function to return synchronously,
434
+ but loading data synchronously in the browser or server is somewhere between "a
435
+ bad idea" and "impossible". QuickJS also doesn't expose an API to "pause" the
436
+ execution of a runtime, and adding such an API is tricky due to the VM's
437
+ implementation.
438
+
439
+ As a work-around, we provide an alternate build of QuickJS processed by
440
+ Emscripten/Binaryen's [ASYNCIFY](https://emscripten.org/docs/porting/asyncify.html)
441
+ compiler transform. Here's how Emscripten's documentation describes Asyncify:
442
+
443
+ > Asyncify lets synchronous C or C++ code interact with asynchronous \[host] JavaScript. This allows things like:
444
+ >
445
+ > - A synchronous call in C that yields to the event loop, which allows browser events to be handled.
446
+ > - A synchronous call in C that waits for an asynchronous operation in \[host] JS to complete.
447
+ >
448
+ > Asyncify automatically transforms ... code into a form that can be paused and
449
+ > resumed ..., so that it is asynchronous (hence the name “Asyncify”) even though
450
+ > \[it is written] in a normal synchronous way.
451
+
452
+ This means we can suspend an _entire WebAssembly module_ (which could contain
453
+ multiple runtimes and contexts) while our host Javascript loads data
454
+ asynchronously, and then resume execution once the data load completes. This is
455
+ a very handy superpower, but it comes with a couple of major limitations:
456
+
457
+ 1. _An asyncified WebAssembly module can only suspend to wait for a single
458
+ asynchronous call at a time_. You may call back into a suspended WebAssembly
459
+ module eg. to create a QuickJS value to return a result, but the system will
460
+ crash if this call tries to suspend again. Take a look at Emscripten's documentation
461
+ on [reentrancy](https://emscripten.org/docs/porting/asyncify.html#reentrancy).
462
+
463
+ 2. _Asyncified code is bigger and runs slower_. The asyncified build of
464
+ Quickjs-emscripten library is 1M, 2x larger than the 500K of the default
465
+ version. There may be room for further
466
+ [optimization](https://emscripten.org/docs/porting/asyncify.html#optimizing)
467
+ Of our build in the future.
468
+
469
+ To use asyncify features, use the following functions:
470
+
471
+ - [newAsyncRuntime][]: create a runtime inside a new WebAssembly module.
472
+ - [newAsyncContext][]: create runtime and context together inside a new
473
+ WebAssembly module.
474
+ - [newQuickJSAsyncWASMModule][]: create an empty WebAssembly module.
475
+
476
+ [newasyncruntime]: https://github.com/componentor/quickjs-emscripten/blob/main/doc/@componentor/quickjs-emscripten/exports.md#newasyncruntime
477
+ [newasynccontext]: https://github.com/componentor/quickjs-emscripten/blob/main/doc/@componentor/quickjs-emscripten/exports.md#newasynccontext
478
+ [newquickjsasyncwasmmodule]: https://github.com/componentor/quickjs-emscripten/blob/main/doc/@componentor/quickjs-emscripten/exports.md#newquickjsasyncwasmmodule
479
+
480
+ These functions are asynchronous because they always create a new underlying
481
+ WebAssembly module so that each instance can suspend and resume independently,
482
+ and instantiating a WebAssembly module is an async operation. This also adds
483
+ substantial overhead compared to creating a runtime or context inside an
484
+ existing module; if you only need to wait for a single async action at a time,
485
+ you can create a single top-level module and create runtimes or contexts inside
486
+ of it.
487
+
488
+ ##### Async module loader
489
+
490
+ Here's an example of valuating a script that loads React asynchronously as an ES
491
+ module. In our example, we're loading from the filesystem for reproducibility,
492
+ but you can use this technique to load using `fetch`.
493
+
494
+ ```typescript
495
+ const module = await newQuickJSAsyncWASMModule()
496
+ const runtime = module.newRuntime()
497
+ const path = await import("path")
498
+ const { promises: fs } = await import("fs")
499
+
500
+ const importsPath = path.join(__dirname, "../examples/imports") + "/"
501
+ // Module loaders can return promises.
502
+ // Execution will suspend until the promise resolves.
503
+ runtime.setModuleLoader((moduleName) => {
504
+ const modulePath = path.join(importsPath, moduleName)
505
+ if (!modulePath.startsWith(importsPath)) {
506
+ throw new Error("out of bounds")
507
+ }
508
+ console.log("loading", moduleName, "from", modulePath)
509
+ return fs.readFile(modulePath, "utf-8")
510
+ })
511
+
512
+ // evalCodeAsync is required when execution may suspend.
513
+ const context = runtime.newContext()
514
+ const result = await context.evalCodeAsync(`
515
+ import * as React from 'esm.sh/react@17'
516
+ import * as ReactDOMServer from 'esm.sh/react-dom@17/server'
517
+ const e = React.createElement
518
+ globalThis.html = ReactDOMServer.renderToStaticMarkup(
519
+ e('div', null, e('strong', null, 'Hello world!'))
520
+ )
521
+ `)
522
+ context.unwrapResult(result).dispose()
523
+ const html = context.getProp(context.global, "html").consume(context.getString)
524
+ console.log(html) // <div><strong>Hello world!</strong></div>
525
+ ```
526
+
527
+ ##### Async on host, sync in QuickJS
528
+
529
+ Here's an example of turning an async function into a sync function inside the
530
+ VM.
531
+
532
+ ```typescript
533
+ const context = await newAsyncContext()
534
+ const path = await import("path")
535
+ const { promises: fs } = await import("fs")
536
+
537
+ const importsPath = path.join(__dirname, "../examples/imports") + "/"
538
+ const readFileHandle = context.newAsyncifiedFunction("readFile", async (pathHandle) => {
539
+ const pathString = path.join(importsPath, context.getString(pathHandle))
540
+ if (!pathString.startsWith(importsPath)) {
541
+ throw new Error("out of bounds")
542
+ }
543
+ const data = await fs.readFile(pathString, "utf-8")
544
+ return context.newString(data)
545
+ })
546
+ readFileHandle.consume((fn) => context.setProp(context.global, "readFile", fn))
547
+
548
+ // evalCodeAsync is required when execution may suspend.
549
+ const result = await context.evalCodeAsync(`
550
+ // Not a promise! Sync! vvvvvvvvvvvvvvvvvvvv
551
+ const data = JSON.parse(readFile('data.json'))
552
+ data.map(x => x.toUpperCase()).join(' ')
553
+ `)
554
+ const upperCaseData = context.unwrapResult(result).consume(context.getString)
555
+ console.log(upperCaseData) // 'VERY USEFUL DATA'
556
+ ```
557
+
558
+ ### Testing your code
559
+
560
+ This library is complicated to use, so please consider automated testing your
561
+ implementation. We highly writing your test suite to run with both the "release"
562
+ build variant of quickjs-emscripten, and also the [DEBUG_SYNC] build variant.
563
+ The debug sync build variant has extra instrumentation code for detecting memory
564
+ leaks.
565
+
566
+ The class [TestQuickJSWASMModule] exposes the memory leak detection API,
567
+ although this API is only accurate when using `DEBUG_SYNC` variant. You can also
568
+ enable [debug logging](#debugging) to help diagnose failures.
569
+
570
+ ```typescript
571
+ // Define your test suite in a function, so that you can test against
572
+ // different module loaders.
573
+ function myTests(moduleLoader: () => Promise<QuickJSWASMModule>) {
574
+ let QuickJS: TestQuickJSWASMModule
575
+ beforeEach(async () => {
576
+ // Get a unique TestQuickJSWASMModule instance for each test.
577
+ const wasmModule = await moduleLoader()
578
+ QuickJS = new TestQuickJSWASMModule(wasmModule)
579
+ })
580
+ afterEach(() => {
581
+ // Assert that the test disposed all handles. The DEBUG_SYNC build
582
+ // variant will show detailed traces for each leak.
583
+ QuickJS.assertNoMemoryAllocated()
584
+ })
585
+
586
+ it("works well", () => {
587
+ // TODO: write a test using QuickJS
588
+ const context = QuickJS.newContext()
589
+ context.unwrapResult(context.evalCode("1 + 1")).dispose()
590
+ context.dispose()
591
+ })
592
+ }
593
+
594
+ // Run the test suite against a matrix of module loaders.
595
+ describe("Check for memory leaks with QuickJS DEBUG build", () => {
596
+ const moduleLoader = memoizePromiseFactory(() => newQuickJSWASMModule(DEBUG_SYNC))
597
+ myTests(moduleLoader)
598
+ })
599
+
600
+ describe("Realistic test with QuickJS RELEASE build", () => {
601
+ myTests(getQuickJS)
602
+ })
603
+ ```
604
+
605
+ For more testing examples, please explore the typescript source of [quickjs-emscripten][ts] repository.
606
+
607
+ [ts]: https://github.com/componentor/quickjs-emscripten/blob/main/packages/quickjs-emscripten/src/
608
+ [debug_sync]: https://github.com/componentor/quickjs-emscripten/blob/main/doc/@componentor/quickjs-emscripten/exports.md#debug_sync
609
+ [testquickjswasmmodule]: https://github.com/componentor/quickjs-emscripten/blob/main/doc/@componentor/quickjs-emscripten/classes/TestQuickJSWASMModule.md
610
+
611
+ ### Packaging
612
+
613
+ The main `quickjs-emscripten` package includes several build variants of the WebAssembly module:
614
+
615
+ - `RELEASE...` build variants should be used in production. They offer better performance and smaller file size compared to `DEBUG...` build variants.
616
+ - `RELEASE_SYNC`: This is the default variant used when you don't explicitly provide one. It offers the fastest performance and smallest file size.
617
+ - `RELEASE_ASYNC`: The default variant if you need [asyncify][] magic, which comes at a performance cost. See the asyncify docs for details.
618
+ - `DEBUG...` build variants can be helpful during development and testing. They include source maps and assertions for catching bugs in your code. We recommend running your tests with _both_ a debug build variant and the release build variant you'll use in production.
619
+ - `DEBUG_SYNC`: Instrumented to detect memory leaks, in addition to assertions and source maps.
620
+ - `DEBUG_ASYNC`: An [asyncify][] variant with source maps.
621
+
622
+ To use a variant, call `newQuickJSWASMModule` or `newQuickJSAsyncWASMModule` with the variant object. These functions return a promise that resolves to a [QuickJSWASMModule](./doc/@componentor/quickjs-emscripten/classes/QuickJSWASMModule.md), the same as `getQuickJS`.
623
+
624
+ ```typescript
625
+ import {
626
+ newQuickJSWASMModule,
627
+ newQuickJSAsyncWASMModule,
628
+ RELEASE_SYNC,
629
+ DEBUG_SYNC,
630
+ RELEASE_ASYNC,
631
+ DEBUG_ASYNC,
632
+ } from "quickjs-emscripten"
633
+
634
+ const QuickJSReleaseSync = await newQuickJSWASMModule(RELEASE_SYNC)
635
+ const QuickJSDebugSync = await newQuickJSWASMModule(DEBUG_SYNC)
636
+ const QuickJSReleaseAsync = await newQuickJSAsyncWASMModule(RELEASE_ASYNC)
637
+ const QuickJSDebugAsync = await newQuickJSAsyncWASMModule(DEBUG_ASYNC)
638
+
639
+ for (const quickjs of [
640
+ QuickJSReleaseSync,
641
+ QuickJSDebugSync,
642
+ QuickJSReleaseAsync,
643
+ QuickJSDebugAsync,
644
+ ]) {
645
+ const vm = quickjs.newContext()
646
+ const result = vm.unwrapResult(vm.evalCode("1 + 1")).consume(vm.getNumber)
647
+ console.log(result)
648
+ vm.dispose()
649
+ quickjs.dispose()
650
+ }
651
+ ```
652
+
653
+ #### Reducing package size
654
+
655
+ Including 4 different copies of the WebAssembly module in the main package gives it an install size of [about 9.04mb](https://packagephobia.com/result?p=quickjs-emscripten). If you're building a CLI package or library of your own, or otherwise don't need to include 4 different variants in your `node_modules`, you can switch to the `quickjs-emscripten-core` package, which contains only the Javascript code for this library, and install one (or more) variants a-la-carte as separate packages.
656
+
657
+ The most minimal setup would be to install `quickjs-emscripten-core` and `@componentor/quickjs-wasmfile-release-sync` (1.3mb total):
658
+
659
+ ```bash
660
+ pnpm run add quickjs-emscripten-core @componentor/quickjs-wasmfile-release-sync
661
+ du -h node_modules
662
+ # 640K node_modules/@componentor/quickjs-wasmfile-release-sync
663
+ # 80K node_modules/@componentor/quickjs-ffi-types
664
+ # 588K node_modules/quickjs-emscripten-core
665
+ # 1.3M node_modules
666
+ ```
667
+
668
+ Then, you can use quickjs-emscripten-core's `newQuickJSWASMModuleFromVariant` to create a QuickJS module (see [the minimal example][minimal]):
669
+
670
+ ```typescript
671
+ // src/quickjs.mjs
672
+ import { newQuickJSWASMModuleFromVariant } from "quickjs-emscripten-core"
673
+ import RELEASE_SYNC from "@componentor/quickjs-wasmfile-release-sync"
674
+ export const QuickJS = await newQuickJSWASMModuleFromVariant(RELEASE_SYNC)
675
+
676
+ // src/app.mjs
677
+ import { QuickJS } from "./quickjs.mjs"
678
+ console.log(QuickJS.evalCode("1 + 1"))
679
+ ```
680
+
681
+ See the [documentation of quickjs-emscripten-core][core] for more details and the list of variant packages.
682
+
683
+ [core]: https://github.com/componentor/quickjs-emscripten/blob/main/doc/quickjs-emscripten-core/README.md
684
+
685
+ #### WebAssembly loading
686
+
687
+ To run QuickJS, we need to load a WebAssembly module into the host Javascript runtime's memory (usually as an ArrayBuffer or TypedArray) and [compile it](https://developer.mozilla.org/en-US/docs/WebAssembly/JavaScript_interface/instantiate_static) to a [WebAssembly.Module](https://developer.mozilla.org/en-US/docs/WebAssembly/JavaScript_interface/Module). This means we need to find the file path or URI of the WebAssembly module, and then read it using an API like `fetch` (browser) or `fs.readFile` (NodeJS). `quickjs-emscripten` tries to handle this automatically using patterns like `new URL('./local-path', import.meta.url)` that work in the browser or are handled automatically by bundlers, or `__dirname` in NodeJS, but you may need to configure this manually if these don't work in your environment, or you want more control about how the WebAssembly module is loaded.
688
+
689
+ To customize the loading of an existing variant, create a new variant with your loading settings using `newVariant`, passing [CustomizeVariantOptions][newVariant]. For example, you need to customize loading in Cloudflare Workers (see [the full example][cloudflare]).
690
+
691
+ ```typescript
692
+ import { newQuickJSWASMModule, DEBUG_SYNC as baseVariant, newVariant } from "quickjs-emscripten"
693
+ import cloudflareWasmModule from "./DEBUG_SYNC.wasm"
694
+ import cloudflareWasmModuleSourceMap from "./DEBUG_SYNC.wasm.map.txt"
695
+
696
+ /**
697
+ * We need to make a new variant that directly passes the imported WebAssembly.Module
698
+ * to Emscripten. Normally we'd load the wasm file as bytes from a URL, but
699
+ * that's forbidden in Cloudflare workers.
700
+ */
701
+ const cloudflareVariant = newVariant(baseVariant, {
702
+ wasmModule: cloudflareWasmModule,
703
+ wasmSourceMapData: cloudflareWasmModuleSourceMap,
704
+ })
705
+ ```
706
+
707
+ [newVariant]: https://github.com/componentor/quickjs-emscripten/blob/main/doc/@componentor/quickjs-emscripten/interfaces/CustomizeVariantOptions.md
708
+
709
+ #### quickjs-ng
710
+
711
+ [quickjs-ng/quickjs](https://github.com/quickjs-ng/quickjs) (aka quickjs-ng) is a fork of the original [bellard/quickjs](https://github.com/quickjs-ng/quickjs) under active development. It implements more EcmaScript standards and removes some of quickjs's custom language features like BigFloat.
712
+
713
+ There are several variants of quickjs-ng available, and quickjs-emscripten may switch to using quickjs-ng by default in the future. See [the list of variants][core].
714
+
715
+ #### Using in the browser without a build step
716
+
717
+ You can use quickjs-emscripten directly from an HTML file in two ways:
718
+
719
+ 1. Import it in an ES Module script tag
720
+
721
+ ```html
722
+ <!doctype html>
723
+ <!-- Import from a ES Module CDN -->
724
+ <script type="module">
725
+ import { getQuickJS } from "https://esm.sh/quickjs-emscripten@0.25.0"
726
+ const QuickJS = await getQuickJS()
727
+ console.log(QuickJS.evalCode("1+1"))
728
+ </script>
729
+ ```
730
+
731
+ 1. In edge cases, you might want to use the IIFE build which provides QuickJS as the global `QJS`. You should probably use the ES module though, any recent browser supports it.
732
+
733
+ ```html
734
+ <!doctype html>
735
+ <!-- Add a script tag to load the library as the QJS global -->
736
+ <script
737
+ src="https://cdn.jsdelivr.net/npm/quickjs-emscripten@0.25.0/dist/index.global.js"
738
+ type="text/javascript"
739
+ ></script>
740
+ <!-- Then use the QJS global in a script tag -->
741
+ <script type="text/javascript">
742
+ QJS.getQuickJS().then((QuickJS) => {
743
+ console.log(QuickJS.evalCode("1+1"))
744
+ })
745
+ </script>
746
+ ```
747
+
748
+ ### Debugging
749
+
750
+ Debug logging can be enabled globally, or for specific runtimes. You need to use a DEBUG build variant of the WebAssembly module to see debug log messages from the C part of this library.
751
+
752
+ ```typescript
753
+ import { newQuickJSWASMModule, DEBUG_SYNC } from "quickjs-emscripten"
754
+
755
+ const QuickJS = await newQuickJSWASMModule(DEBUG_SYNC)
756
+ ```
757
+
758
+ With quickjs-emscripten-core:
759
+
760
+ ```typescript
761
+ import { newQuickJSWASMModuleFromVariant } from "quickjs-emscripten-core"
762
+ import DEBUG_SYNC from "@componentor/quickjs-wasmfile-debug-sync"
763
+
764
+ const QuickJS = await newQuickJSWASMModuleFromVariant(DEBUG_SYNC)
765
+ ```
766
+
767
+ To enable debug logging globally, call [setDebugMode][setDebugMode]. This affects global Javascript parts of the library, like the module loader and asyncify internals, and is inherited by runtimes created after the call.
768
+
769
+ ```typescript
770
+ import { setDebugMode } from "quickjs-emscripten"
771
+
772
+ setDebugMode(true)
773
+ ```
774
+
775
+ With quickjs-emscripten-core:
776
+
777
+ ```typescript
778
+ import { setDebugMode } from "quickjs-emscripten-core"
779
+
780
+ setDebugMode(true)
781
+ ```
782
+
783
+ To enable debug logging for a specific runtime, call [setDebugModeRt][setDebugModeRt]. This affects only the runtime and its associated contexts.
784
+
785
+ ```typescript
786
+ const runtime = QuickJS.newRuntime()
787
+ runtime.setDebugMode(true)
788
+
789
+ const context = QuickJS.newContext()
790
+ context.runtime.setDebugMode(true)
791
+ ```
792
+
793
+ [setDebugMode]: doc/@componentor/quickjs-emscripten/README.md#setdebugmode
794
+ [setDebugModeRt]: https://github.com/componentor/quickjs-emscripten/blob/main/doc/@componentor/quickjs-emscripten-core/classes/QuickJSRuntime.md#setdebugmode
795
+
796
+ ### Supported Platforms
797
+
798
+ `quickjs-emscripten` and related packages should work in any environment that supports ES2020.
799
+
800
+ - Browsers: we estimate support for the following browser versions. See the [global-iife][iife] and [esmodule][esm-html] HTML examples.
801
+ - Chrome 63+
802
+ - Edge 79+
803
+ - Safari 11.1+
804
+ - Firefox 58+
805
+ - NodeJS: requires v16.0.0 or later for WebAssembly compatibility. Tested with node@18. See the [node-typescript][tsx-example] and [node-minimal][minimal] examples.
806
+ - Typescript: tested with typescript@4.5.5 and typescript@5.3.3. See the [node-typescript example][tsx-example].
807
+ - Vite: tested with vite@5.0.10. See the [Vite/Vue example][vite].
808
+ - Create react app: tested with react-scripts@5.0.1. See the [create-react-app example][cra].
809
+ - Webpack: tested with webpack@5.89.0 via create-react-app.
810
+ - Cloudflare Workers: tested with wrangler@3.22.1. See the [Cloudflare Workers example][cloudflare].
811
+ - Deno: tested with deno 1.39.1. See the [Deno example][deno].
812
+
813
+ [iife]: https://github.com/componentor/quickjs-emscripten/blob/main/examples/global-iife.html
814
+ [esm-html]: https://github.com/componentor/quickjs-emscripten/blob/main/examples/esmodule.html
815
+ [deno]: https://github.com/componentor/quickjs-emscripten/blob/main/examples/deno
816
+ [vite]: https://github.com/componentor/quickjs-emscripten/blob/main/examples/vite-vue
817
+ [cra]: https://github.com/componentor/quickjs-emscripten/blob/main/examples/create-react-app
818
+ [cloudflare]: https://github.com/componentor/quickjs-emscripten/blob/main/examples/cloudflare-workers
819
+ [tsx-example]: https://github.com/componentor/quickjs-emscripten/blob/main/examples/node-typescript
820
+ [minimal]: https://github.com/componentor/quickjs-emscripten/blob/main/examples/node-minimal
821
+
822
+ ### More Documentation
823
+
824
+ [Github] | [NPM] | [API Documentation][api] | [Variants][core] | [Examples][tests]
825
+
826
+ ## Background
827
+
828
+ This was inspired by seeing https://github.com/maple3142/duktape-eval
829
+ [on Hacker News](https://news.ycombinator.com/item?id=21946565) and Figma's
830
+ blogposts about using building a Javascript plugin runtime:
831
+
832
+ - [How Figma built the Figma plugin system](https://www.figma.com/blog/how-we-built-the-figma-plugin-system/): Describes the LowLevelJavascriptVm interface.
833
+ - [An update on plugin security](https://www.figma.com/blog/an-update-on-plugin-security/): Figma switches to QuickJS.
834
+
835
+ ## Status & Roadmap
836
+
837
+ **Stability**: Because the version number of this project is below `1.0.0`,
838
+ \*expect occasional breaking API changes.
839
+
840
+ **Security**: This project makes every effort to be secure, but has not been
841
+ audited. Please use with care in production settings.
842
+
843
+ **Roadmap**: I work on this project in my free time, for fun. Here's I'm
844
+ thinking comes next. Last updated 2022-03-18.
845
+
846
+ 1. Further work on module loading APIs:
847
+ - Create modules via Javascript, instead of source text.
848
+ - Scan source text for imports, for ahead of time or concurrent loading.
849
+ (This is possible with third-party tools, so lower priority.)
850
+
851
+ 2. Higher-level tools for reading QuickJS values:
852
+ - Type guard functions: `context.isArray(handle)`, `context.isPromise(handle)`, etc.
853
+ - Iteration utilities: `context.getIterable(handle)`, `context.iterateObjectEntries(handle)`.
854
+ This better supports user-level code to deserialize complex handle objects.
855
+
856
+ 3. Higher-level tools for creating QuickJS values:
857
+ - Devise a way to avoid needing to mess around with handles when setting up
858
+ the environment.
859
+ - Consider integrating
860
+ [quickjs-emscripten-sync](https://github.com/reearth/quickjs-emscripten-sync)
861
+ for automatic translation.
862
+ - Consider class-based or interface-type-based marshalling.
863
+
864
+ 4. SQLite integration.
865
+
866
+ ## Related
867
+
868
+ - Duktape wrapped in Wasm: https://github.com/maple3142/duktape-eval/blob/main/src/Makefile
869
+ - QuickJS wrapped in C++: https://github.com/ftk/quickjspp
870
+
871
+ ## Developing
872
+
873
+ This library is implemented in two languages: C (compiled to WASM with
874
+ Emscripten), and Typescript.
875
+
876
+ You will need `node`, `pnpm run`, `make`, and `emscripten` to build this project.
877
+
878
+ ### The C parts
879
+
880
+ The ./c directory contains C code that wraps the QuickJS C library (in ./quickjs).
881
+ Public functions (those starting with `QTS_`) in ./c/interface.c are
882
+ automatically exported to native code (via a generated header) and to
883
+ Typescript (via a generated FFI class). See ./generate.ts for how this works.
884
+
885
+ The C code builds with `emscripten` (using `emcc`), to produce WebAssembly.
886
+ The version of Emscripten used by the project is defined in templates/Variant.mk.
887
+
888
+ - On ARM64, you should install `emscripten` on your machine. For example on macOS, `brew install emscripten`.
889
+ - If _the correct version of emcc_ is not in your PATH, compilation falls back to using Docker.
890
+ On ARM64, this is 10-50x slower than native compilation, but it's just fine on x64.
891
+
892
+ We produce multiple build variants of the C code compiled to WebAssembly using a
893
+ template script the ./packages directory. Each build variant uses its own copy of a Makefile
894
+ to build the C code. The Makefile is generated from a template in ./templates/Variant.mk.
895
+
896
+ Related NPM scripts:
897
+
898
+ - `pnpm run vendor:update` updates vendor/quickjs and vendor/quickjs-ng to the latest versions on Github.
899
+ - `pnpm run build:codegen` updates the ./packages from the template script `./prepareVariants.ts` and Variant.mk.
900
+ - `pnpm run build:packages` builds the variant packages in parallel.
901
+
902
+ ### The Typescript parts
903
+
904
+ The Javascript/Typescript code is also organized into several NPM packages in ./packages:
905
+
906
+ - ./packages/quickjs-ffi-types: Low-level types that define the FFI interface to the C code.
907
+ Each variant exposes an API conforming to these types that's consumed by the higher-level library.
908
+ - ./packages/quickjs-emscripten-core: The higher-level Typescript that implements the user-facing abstractions of the library.
909
+ This package doesn't link directly to the WebAssembly/C code; callers must provide a build variant.
910
+ - ./packages/quicks-emscripten: The main entrypoint of the library, which provides the `getQuickJS` function.
911
+ This package combines quickjs-emscripten-core with platform-appropriate WebAssembly/C code.
912
+
913
+ Related NPM scripts:
914
+
915
+ - `pnpm run check` runs all available checks (build, format, tests, etc).
916
+ - `pnpm run build` builds all the packages and generates the docs.
917
+ - `pnpm run test` runs the tests for all packages.
918
+ - `pnpm run test:fast` runs the tests using only fast build variants.
919
+ - `pnpm run doc` generates the docs into `./doc`.
920
+ - `pnpm run doc:serve` previews the current `./doc` in a browser.
921
+ - `pnpm run prettier` formats the repo.
922
+
923
+ ### pnpm run updates
924
+
925
+ Just run `pnpm run set version from sources` to upgrade the pnpm run release.