@suchipi/quickjs 0.6.0 → 0.7.0

package/README.md

@@ -63,6 +63,7 @@ Fork of the fantastic QuickJS engine by Fabrice Bellard, with many changes.
 - Several C-side helper functions were moved out of quickjs-libc and into quickjs-utils.
 - Most module-related code (setting import.meta, etc) was moved into quickjs-modulesys.
 - Added `std.setExitCode`, `std.getExitCode`, and made `std.exit`'s parameter optional. The value passed to `std.setExitCode` will be used when the process exits normally, or when `std.exit` is called without any arguments. `std.setExitCode`, `std.getExitCode`, and `std.exit` throw if called from a thread other than the main thread (ie. a Worker).
+- The manual garbage collection function `std.gc()` was moved to `"quickjs:engine"`.
 
 ### Changes to the `qjs` binary:
 
@@ -113,6 +114,10 @@ Helper structs, functions, and macros that make it easier to work with QuickJS i
 - Helper function for loading a file from disk into char a buffer
 - Helper functions for printing JS errors to stderr
 
+### New module: "quickjs:engine"
+
+This module contains APIs related to engine internals like script execution, module loading, code eval, filename reflection, and garbage collection. Several parts of quickjs-libc were moved here so that quickjs-libc could be focused on "C standard library" bindings.
+
 ### Changes to the module loader
 
 - `.js` extensions can now be omitted from import specifiers; they're optional.
@@ -123,24 +128,22 @@ Helper structs, functions, and macros that make it easier to work with QuickJS i
   - It's the same as the global `require`; it's just added to import.meta for compatibility with bundlers that output `import.meta.require`, like `bun`.
 - Adds `import.meta.resolve`
   - Similar to [the one in the browser](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/import.meta/resolve#specifications), but it's actually just `require.resolve` exposed via `import.meta`.
-- Module and eval helpers have been moved from "quickjs:std" to the new module "quickjs:module".
-- Makes the module loader's resolution and loading behavior configurable, and allows users to define additional builtin modules
-  - The module "quickjs:module" exports an object called `Module`.
-  - You can specify additional implicit import specifier extensions by adding to the `Module.searchExtensions` array.
-  - You can transform any file prior to evaluating it as a module by adding a function to the `Module.compilers` object. Useful for compile-to-ts languages like TypeScript, Coffeescript, etc.
-  - You can define custom builtin modules using the `Module.define` function.
-  - You can override module name normalization (aka module resolution) by replacing the `Module.resolve` function.
-    - Note that you must handle `Module.searchExtensions` yourself in your replacement implementation.
-  - You can override the method used to load modules by replacing the `Module.read` function.
-    - Note that you must handle `Module.compilers` yourself in your replacement implementation.
+- Module and eval helpers have been moved from "quickjs:std" to the new module "quickjs:engine".
+- Makes the module loader's resolution and loading behavior configurable
+  - The module "quickjs:engine" exports an object called `ModuleDelegate`.
+  - You can specify additional implicit import specifier extensions by adding to the `ModuleDelegate.searchExtensions` array.
+  - You can transform any file prior to evaluating it as a module by adding a function to the `ModuleDelegate.compilers` object. Useful for compile-to-ts languages like TypeScript, Coffeescript, etc.
+  - You can override module name normalization (aka module resolution) by replacing the `ModuleDelegate.resolve` function.
+    - Note that you must handle `ModuleDelegate.searchExtensions` yourself in your replacement implementation.
+  - You can override the method used to load modules by replacing the `ModuleDelegate.read` function.
+    - Note that you must handle `ModuleDelegate.compilers` yourself in your replacement implementation.
 - Makes `import.meta.main` configurable
-  - The module "quickjs:module" exports two functions named `setMainModule` and `isMainModule`.
-  - You can use `setMainModule` to make `import.meta.main` true within that module's code. Note, however, that it does not work retroactively; only modules loaded after the `setMainModule` call will be affected. To defer module load, use `import()`, `importModule` from "quickjs:module", or `require`.
+  - The module "quickjs:engine" exports two functions named `setMainModule` and `isMainModule`.
+  - You can use `setMainModule` to make `import.meta.main` true within that module's code. Note, however, that it does not work retroactively; only modules loaded after the `setMainModule` call will be affected. To defer module load, use `import()`, `importModule` from "quickjs:engine", or `require`.
   - You can use `isMainModule` to check if a given module would be the main module without loading it.
-- Adds a mechanism for identifying module namespace objects
-  - The module "quickjs:module" exports an object called `Module`.
-  - `something instanceof Module` will be true when `something` is a module namespace object.
-- When using `require` to load a module which contains an export named `__cjsExports`, the value of the `__cjsExports` property will be returned from `require` instead of the usual module namespace object. This can be leveraged by users configuring the module loader to add some CommonJS <-> ESM interop. Note, however, that dynamic import and `"quickjs:module"`'s `importModule` always receive the usual module namespace object.
+- New `isModuleNamespace` function lets users identify module namespace objects
+- New `defineBuiltinModule` function lets users add their own builtin modules
+- When using `require` to load a module which contains an export named `__cjsExports`, the value of the `__cjsExports` property will be returned from `require` instead of the usual module namespace object. This can be leveraged by users configuring the module loader to add some CommonJS <-> ESM interop. Note, however, that dynamic import and `"quickjs:engine"`'s `importModule` always receive the usual module namespace object.
 - Synchronous import function added (`importModule`), which provides the same module record object you would get via dynamic (async) import.
 - JS api for using the engine's configured module name normalization function was added (`resolveModule`).
 
@@ -150,7 +153,7 @@ Helper structs, functions, and macros that make it easier to work with QuickJS i
 - Ninja is used instead of make. Ninja build config is generated via `.ninja.js` files which get loaded into [@suchipi/shinobi](https://github.com/suchipi/shinobi).
 - Line endings have been made consistent and trailing whitespace has been removed
 - The tests are authored in a new format which leverages jest snapshot testing.
-- Module-loading code in `quickjs-libc` was moved into `quickjs-modulesys` and `quickjs-libmodule`.
+- Some parts of `quickjs-libc` were moved into `quickjs-modulesys` and `quickjs-libengine`.
 - The `eval_*` functions that were duplicated in each of the programs (`eval_buf`, `eval_file`, and `eval_binary`) were deduplicated and moved into `quickjs-modulesys`.
 
 ### More target OSes/runtimes
@@ -185,7 +188,7 @@ QuickJS itself has no external dependencies outside this repo except pthreads, a
 
 Linux, macOS, iOS, and Windows binaries can be compiled using Docker. Or, you can compile binaries for just your own unix system, without using Docker.
 
-If you're not gonna use Docker, you'll need to install [Ninja](https://ninja-build.org/) and [Node.js](https://nodejs.org/) in order to compile. I use Ninja 1.10.1 and Node.js 18.12.1, but it should work with most versions of both of those.
+If you're not gonna use Docker, you'll need to install [Ninja](https://ninja-build.org/) and [Node.js](https://nodejs.org/) in order to compile. I use Ninja 1.10.1 and Node.js 18.18.0, but it should work with most versions of both of those.
 
 ### Compilation Instructions
 
@@ -198,7 +201,7 @@ To compile binaries for Linux, macOS, iOS, and Windows (using Docker):
 
 Or, to compile binaries for just your own unix system:
 
-- Make sure you have both [Ninja](https://ninja-build.org/) and [Node.js](https://nodejs.org/) installed. I use Ninja 1.10.1 and Node.js 18.12.1, but it should work with most versions of both of those.
+- Make sure you have both [Ninja](https://ninja-build.org/) and [Node.js](https://nodejs.org/) installed. I use Ninja 1.10.1 and Node.js 18.18.0, but it should work with most versions of both of those.
 - Clone the repo and cd to its folder
 - Run `meta/build.sh`
 - Build artifacts will be placed in the `build` folder. You're probably most interested in stuff in the `build/bin` and `build/lib` folders.

package/build/aarch64-apple-darwin/bin/file-to-bytecode.js

@@ -11,7 +11,8 @@ import * as std from "quickjs:std";
 import * as Bytecode from "quickjs:bytecode";
 
 function main() {
-  const [_quickjsRun, _thisScript, inputFile, outputFile] = scriptArgs;
+  let [_quickjsRun, _thisScript, inputFile, outputFile, encodedFileName] =
+    scriptArgs;
 
   if (!(inputFile && outputFile)) {
     throw new Error(
@@ -19,7 +20,11 @@ function main() {
     );
   }
 
-  const bytecode = Bytecode.fromFile(inputFile);
+  if (!encodedFileName) {
+    encodedFileName = inputFile;
+  }
+
+  const bytecode = Bytecode.fromFile(inputFile, { encodedFileName });
   const out = std.open(outputFile, "wb");
   out.write(bytecode, 0, bytecode.byteLength);
 }

package/build/aarch64-unknown-linux-gnu/bin/file-to-bytecode.js

@@ -11,7 +11,8 @@ import * as std from "quickjs:std";
 import * as Bytecode from "quickjs:bytecode";
 
 function main() {
-  const [_quickjsRun, _thisScript, inputFile, outputFile] = scriptArgs;
+  let [_quickjsRun, _thisScript, inputFile, outputFile, encodedFileName] =
+    scriptArgs;
 
   if (!(inputFile && outputFile)) {
     throw new Error(
@@ -19,7 +20,11 @@ function main() {
     );
   }
 
-  const bytecode = Bytecode.fromFile(inputFile);
+  if (!encodedFileName) {
+    encodedFileName = inputFile;
+  }
+
+  const bytecode = Bytecode.fromFile(inputFile, { encodedFileName });
   const out = std.open(outputFile, "wb");
   out.write(bytecode, 0, bytecode.byteLength);
 }

package/build/aarch64-unknown-linux-musl/bin/file-to-bytecode.js

@@ -11,7 +11,8 @@ import * as std from "quickjs:std";
 import * as Bytecode from "quickjs:bytecode";
 
 function main() {
-  const [_quickjsRun, _thisScript, inputFile, outputFile] = scriptArgs;
+  let [_quickjsRun, _thisScript, inputFile, outputFile, encodedFileName] =
+    scriptArgs;
 
   if (!(inputFile && outputFile)) {
     throw new Error(
@@ -19,7 +20,11 @@ function main() {
     );
   }
 
-  const bytecode = Bytecode.fromFile(inputFile);
+  if (!encodedFileName) {
+    encodedFileName = inputFile;
+  }
+
+  const bytecode = Bytecode.fromFile(inputFile, { encodedFileName });
   const out = std.open(outputFile, "wb");
   out.write(bytecode, 0, bytecode.byteLength);
 }

package/build/aarch64-unknown-linux-static/bin/file-to-bytecode.js

@@ -11,7 +11,8 @@ import * as std from "quickjs:std";
 import * as Bytecode from "quickjs:bytecode";
 
 function main() {
-  const [_quickjsRun, _thisScript, inputFile, outputFile] = scriptArgs;
+  let [_quickjsRun, _thisScript, inputFile, outputFile, encodedFileName] =
+    scriptArgs;
 
   if (!(inputFile && outputFile)) {
     throw new Error(
@@ -19,7 +20,11 @@ function main() {
     );
   }
 
-  const bytecode = Bytecode.fromFile(inputFile);
+  if (!encodedFileName) {
+    encodedFileName = inputFile;
+  }
+
+  const bytecode = Bytecode.fromFile(inputFile, { encodedFileName });
   const out = std.open(outputFile, "wb");
   out.write(bytecode, 0, bytecode.byteLength);
 }

package/build/dts/quickjs-libbytecode.d.ts

@@ -6,7 +6,11 @@ declare module "quickjs:bytecode" {
    */
   export function fromFile(
     path: string,
-    options?: { byteSwap?: boolean; sourceType?: "module" | "script" }
+    options?: {
+      byteSwap?: boolean;
+      sourceType?: "module" | "script";
+      encodedFileName?: string;
+    }
   ): ArrayBuffer;
 
   /**

package/build/dts/quickjs-libc.d.ts

@@ -271,9 +271,6 @@ declare module "quickjs:std" {
   /** Constant for {@link FILE.setvbuf}. Declares that the buffer mode should be 'no buffering'. */
   export var _IONBF: number;
 
-  /** Manually invoke the cycle removal algorithm (garbage collector). The cycle removal algorithm is automatically started when needed, so this function is useful in case of specific memory constraints or for testing. */
-  export function gc(): void;
-
   /** Return the value of the environment variable `name` or `undefined` if it is not defined. */
   export function getenv(name: string): string | undefined;
 
@@ -924,7 +921,7 @@ declare module "quickjs:os" {
   export function dup2(oldfd: number, newfd: number): number;
 
   /** `pipe` Unix system call. Return two handles as `[read_fd, write_fd]`. */
-  export function pipe(): null | [number, number];
+  export function pipe(): [number, number];
 
   /** Sleep for `delay_ms` milliseconds. */
   export function sleep(delay_ms: number): void;

package/build/dts/quickjs-libcontext.d.ts

@@ -137,7 +137,7 @@ declare module "quickjs:context" {
       console?: boolean;
       /** Enables `print`. Defaults to `true`. */
       print?: boolean;
-      /** Enables `require` and `Module`. Defaults to `true`. */
+      /** Enables `require`. Defaults to `true`. */
       moduleGlobals?: boolean;
       /**
        * Enables `setTimeout`, `clearTimeout`, `setInterval`, and
@@ -155,8 +155,8 @@ declare module "quickjs:context" {
         "quickjs:bytecode"?: boolean;
         /** Enables the "quickjs:context" module. Defaults to `true`. */
         "quickjs:context"?: boolean;
-        /** Enables the "quickjs:module" module. Defaults to `true`. */
-        "quickjs:module"?: boolean;
+        /** Enables the "quickjs:engine" module. Defaults to `true`. */
+        "quickjs:engine"?: boolean;
       };
     });
 

package/build/dts/{quickjs-libmodule.d.ts → quickjs-libengine.d.ts}

@@ -1,4 +1,4 @@
-declare module "quickjs:module" {
+declare module "quickjs:engine" {
   /**
    * Return whether the provided resolved module path is set as the main module.
    *
@@ -70,12 +70,31 @@ declare module "quickjs:module" {
    */
   export function getFileNameFromStack(stackLevels?: number): string;
 
+  /**
+   * Returns true if `target` is a module namespace object.
+   */
+  export function isModuleNamespace(target: any): boolean;
+
+  /**
+   * Create a virtual built-in module whose exports consist of the own
+   * enumerable properties of `obj`.
+   */
+  export function defineBuiltinModule(
+    name: string,
+    obj: { [key: string]: any }
+  ): void;
+
   /**
    * An object which lets you configure the module loader (import/export/require).
-   * You can use these properties to add support for importing new filetypes.
+   * You can change these properties to add support for importing new filetypes.
+   */
+  export const ModuleDelegate: ModuleDelegate;
+
+  /**
+   * Manually invoke the cycle removal algorithm (garbage collector).
    *
-   * This object can also be used to identify whether an object is a module
-   * namespace record.
+   * The cycle removal algorithm is automatically started when needed, so this
+   * function is useful in case of specific memory constraints or for testing.
    */
-  export const Module: Module;
+  export function gc(): void;
 }

package/build/dts/quickjs-modulesys.d.ts

@@ -1,18 +1,8 @@
 /**
  * An object which lets you configure the module loader (import/export/require).
- * You can use these properties to add support for importing new filetypes.
- *
- * This object can also be used to identify whether an object is a module
- * namespace record.
+ * You can change these properties to add support for importing new filetypes.
  */
-interface Module {
-  /**
-   * Returns true if `target` is a module namespace object.
-   */
-  [Symbol.hasInstance](target: any): target is {
-    [key: string | number | symbol]: any;
-  };
-
+interface ModuleDelegate {
   /**
    * A list of filetype extensions that may be omitted from an import specifier
    * string.
@@ -51,7 +41,7 @@ interface Module {
    * ```js
    * import * as std from "std";
    *
-   * Module.compilers[".txt"] = (filename, content) => {
+   * ModuleDelegate.compilers[".txt"] = (filename, content) => {
    *   return `export default ${JSON.stringify(content)}`;
    * }
    * ```
@@ -71,19 +61,14 @@ interface Module {
     [extensionWithDot: string]: (filename: string, content: string) => string;
   };
 
-  /**
-   * Create a virtual built-in module whose exports consist of the own
-   * enumerable properties of `obj`.
-   */
-  define(name: string, obj: { [key: string]: any }): void;
-
   /**
    * Resolves a require/import request from `fromFile` into a canonicalized
    * path.
    *
    * To change native module resolution behavior, replace this function with
    * your own implementation. Note that you must handle
-   * `Module.searchExtensions` yourself in your replacement implementation.
+   * `ModuleDelegate.searchExtensions` yourself in your replacement
+   * implementation.
    */
   resolve(name: string, fromFile: string): string;
 
@@ -91,8 +76,8 @@ interface Module {
    * Reads the contents of the given resolved module name into a string.
    *
    * To change native module loading behavior, replace this function with your
-   * own implementation. Note that you must handle `Module.compilers` yourself
-   * in your replacement implementation.
+   * own implementation. Note that you must handle `ModuleDelegate.compilers`
+   * yourself in your replacement implementation.
    */
   read(modulePath: string): string;
 }
@@ -105,8 +90,8 @@ interface RequireFunction {
    *
    * If `source` does not have a file extension, and a file without an extension
    * cannot be found, the engine will check for files with the extensions in
-   * {@link Module.searchExtensions}, and use one of those if present. This
-   * behavior also happens when using normal `import` statements.
+   * {@link ModuleDelegate.searchExtensions}, and use one of those if present.
+   * This behavior also happens when using normal `import` statements.
    *
    * For example, if you write:
    *
@@ -115,8 +100,8 @@ interface RequireFunction {
    * ```
    *
    * but there's no file named `somewhere` in the same directory as the file
-   * where that import appears, and `Module.searchExtensions` is the default
-   * value:
+   * where that import appears, and `ModuleDelegate.searchExtensions` is the
+   * default value:
    *
    * ```js
    * [".js"]
@@ -126,9 +111,9 @@ interface RequireFunction {
    * engine will look for `somewhere/index.js`. If *that* doesn't exist, an
    * error will be thrown.
    *
-   * If you add more extensions to `Module.searchExtensions`, then the engine
-   * will use those, too. It will search in the same order as the strings appear
-   * in the `Module.searchExtensions` array.
+   * If you add more extensions to `ModuleDelegate.searchExtensions`, then the
+   * engine will use those, too. It will search in the same order as the strings
+   * appear in the `ModuleDelegate.searchExtensions` array.
    */
   (source: string): any;
 
@@ -168,11 +153,12 @@ interface ImportMeta {
    *
    * Equivalent to `globalThis.require.resolve`.
    *
-   * Behaves similarly to [the browser import.meta.resolve](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/import.meta/resolve),
+   * Behaves similarly to [the browser
+   * import.meta.resolve](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/import.meta/resolve),
    * but it does not ensure that the returned string is a valid URL, because it
-   * delegates directly to {@link Module.resolve} to resolve the name. If you
-   * want this to return URL strings, change `Module.resolve` and `Module.read`
-   * to work with URL strings.
+   * delegates directly to {@link ModuleDelegate.resolve} to resolve the name.
+   * If you want this to return URL strings, change `ModuleDelegate.resolve` and
+   * `ModuleDelegate.read` to work with URL strings.
    */
   resolve: RequireFunction["resolve"];
 }

package/build/x86_64-apple-darwin/bin/file-to-bytecode.js

@@ -11,7 +11,8 @@ import * as std from "quickjs:std";
 import * as Bytecode from "quickjs:bytecode";
 
 function main() {
-  const [_quickjsRun, _thisScript, inputFile, outputFile] = scriptArgs;
+  let [_quickjsRun, _thisScript, inputFile, outputFile, encodedFileName] =
+    scriptArgs;
 
   if (!(inputFile && outputFile)) {
     throw new Error(
@@ -19,7 +20,11 @@ function main() {
     );
   }
 
-  const bytecode = Bytecode.fromFile(inputFile);
+  if (!encodedFileName) {
+    encodedFileName = inputFile;
+  }
+
+  const bytecode = Bytecode.fromFile(inputFile, { encodedFileName });
   const out = std.open(outputFile, "wb");
   out.write(bytecode, 0, bytecode.byteLength);
 }

package/build/x86_64-pc-windows-static/bin/file-to-bytecode.js

@@ -11,7 +11,8 @@ import * as std from "quickjs:std";
 import * as Bytecode from "quickjs:bytecode";
 
 function main() {
-  const [_quickjsRun, _thisScript, inputFile, outputFile] = scriptArgs;
+  let [_quickjsRun, _thisScript, inputFile, outputFile, encodedFileName] =
+    scriptArgs;
 
   if (!(inputFile && outputFile)) {
     throw new Error(
@@ -19,7 +20,11 @@ function main() {
     );
   }
 
-  const bytecode = Bytecode.fromFile(inputFile);
+  if (!encodedFileName) {
+    encodedFileName = inputFile;
+  }
+
+  const bytecode = Bytecode.fromFile(inputFile, { encodedFileName });
   const out = std.open(outputFile, "wb");
   out.write(bytecode, 0, bytecode.byteLength);
 }

package/build/x86_64-unknown-linux-gnu/bin/file-to-bytecode.js

@@ -11,7 +11,8 @@ import * as std from "quickjs:std";
 import * as Bytecode from "quickjs:bytecode";
 
 function main() {
-  const [_quickjsRun, _thisScript, inputFile, outputFile] = scriptArgs;
+  let [_quickjsRun, _thisScript, inputFile, outputFile, encodedFileName] =
+    scriptArgs;
 
   if (!(inputFile && outputFile)) {
     throw new Error(
@@ -19,7 +20,11 @@ function main() {
     );
   }
 
-  const bytecode = Bytecode.fromFile(inputFile);
+  if (!encodedFileName) {
+    encodedFileName = inputFile;
+  }
+
+  const bytecode = Bytecode.fromFile(inputFile, { encodedFileName });
   const out = std.open(outputFile, "wb");
   out.write(bytecode, 0, bytecode.byteLength);
 }

package/build/x86_64-unknown-linux-musl/bin/file-to-bytecode.js

@@ -11,7 +11,8 @@ import * as std from "quickjs:std";
 import * as Bytecode from "quickjs:bytecode";
 
 function main() {
-  const [_quickjsRun, _thisScript, inputFile, outputFile] = scriptArgs;
+  let [_quickjsRun, _thisScript, inputFile, outputFile, encodedFileName] =
+    scriptArgs;
 
   if (!(inputFile && outputFile)) {
     throw new Error(
@@ -19,7 +20,11 @@ function main() {
     );
   }
 
-  const bytecode = Bytecode.fromFile(inputFile);
+  if (!encodedFileName) {
+    encodedFileName = inputFile;
+  }
+
+  const bytecode = Bytecode.fromFile(inputFile, { encodedFileName });
   const out = std.open(outputFile, "wb");
   out.write(bytecode, 0, bytecode.byteLength);
 }

package/build/x86_64-unknown-linux-static/bin/file-to-bytecode.js

@@ -11,7 +11,8 @@ import * as std from "quickjs:std";
 import * as Bytecode from "quickjs:bytecode";
 
 function main() {
-  const [_quickjsRun, _thisScript, inputFile, outputFile] = scriptArgs;
+  let [_quickjsRun, _thisScript, inputFile, outputFile, encodedFileName] =
+    scriptArgs;
 
   if (!(inputFile && outputFile)) {
     throw new Error(
@@ -19,7 +20,11 @@ function main() {
     );
   }
 
-  const bytecode = Bytecode.fromFile(inputFile);
+  if (!encodedFileName) {
+    encodedFileName = inputFile;
+  }
+
+  const bytecode = Bytecode.fromFile(inputFile, { encodedFileName });
   const out = std.open(outputFile, "wb");
   out.write(bytecode, 0, bytecode.byteLength);
 }

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@suchipi/quickjs",
   "description": "Suchipi's QuickJS Fork",
-  "version": "0.6.0",
+  "version": "0.7.0",
   "main": "./npm/index.js",
   "bin": {
     "qjs": "npm/cli/qjs.js",
@@ -13,7 +13,7 @@
     "@babel/preset-env": "^7.20.2",
     "@babel/preset-typescript": "^7.18.6",
     "@suchipi/print": "^2.5.0",
-    "@suchipi/shinobi": "^0.2.0",
+    "@suchipi/shinobi": "^1.0.0",
     "@types/jest": "^29.2.4",
     "@types/node": "^18.11.9",
     "@types/shelljs": "^0.8.11",