@suchipi/quickjs 0.5.0 → 0.6.0

package/README.md

@@ -1,8 +1,21 @@
 # suchipi/quickjs
 
-Fork of the fantastic QuickJS engine by Fabrice Bellard, with the following changes:
+Fork of the fantastic QuickJS engine by Fabrice Bellard, with many changes.
 
-## Changes to `quickjs`:
+## High-level List of Most Notable Changes
+
+- APIs from 'std' and 'os' modules were changed to be more idiomatic-to-JS; notably, they throw Errors on failure instead of returning null or errno numbers.
+- TypeScript-format type interface files (`.d.ts` files) were added for everything.
+- Hooks have been added to the module loader that make its functionality more customizable.
+- Some ES2022 features were added, and some non-standard ECMAScript proposals and extensions were added.
+- The way the project is organized and built was changed dramatically.
+- Several new JS bindings for C APIs have been added, such as `strftime`, `access`, `fsync`, `setvbuf`, `getuid`...
+- Scripts were added that cross-compile binaries for several operating systems and architectures
+- FreeBSD support added
+
+## Detailed List of Changes
+
+### Changes to `quickjs`:
 
 - A TypeScript `.d.ts` file is provided for all QuickJS-specific APIs (operator overloading APIs, BigInt extensions, BigFloat, BigDecimal, etc).
 - Non-standard `Object.toPrimitive` added (static method that invokes ToPrimitive on the given value, using the optionally-provided hint).
@@ -23,8 +36,9 @@ Fork of the fantastic QuickJS engine by Fabrice Bellard, with the following chan
   - `JS_FreezeObjectValue` (performs Object.freeze)
   - `JS_IsPrimitive`
 - ModuleDefs now have an optional "user_data" property (pointer to void) which can be accessed during module initialization (via `JS_GetModuleUserData` and `JS_SetModuleUserData`)
+- Added `JS_SetContextOpaqueValue` and `JS_GetContextOpaqueValue`, which let you associate a JSValue with a JSContext, which will be garbage-collected when that JSContext is garbage-collected.
 
-## Changes to `quickjs-libc`:
+### Changes to `quickjs-libc`:
 
 - `std` and `os` builtin modules are now namespaced under "quickjs:". In other words, you have to import them as "quickjs:std" and "quickjs:os".
 - APIs in `std` and `os` no longer return errno anywhere; instead, Error objects are thrown. `errno` is available as a property on the thrown Error objects.
@@ -35,6 +49,7 @@ Fork of the fantastic QuickJS engine by Fabrice Bellard, with the following chan
 - `os.access` function added (wrapper for libc `access`).
 - `FILE.prototype.sync` method added (wrapper for `fsync`).
 - `FILE.prototype.setvbuf` method added (wrapper for `setvbuf`).
+- `FILE.prototype.writeTo` method added (pipe data from one FILE to another, given a buffer size and limit).
 - `std.isFILE` function added (returns whether the provided object is a `FILE` (via `js_std_file_class_id`)).
 - `os.{WUNTRACED,WEXITSTATUS,WTERMSIG,WSTOPSIG,WIFEXITED,WIFSIGNALED,WIFSTOPPED,WIFCONTINUED}` added, for working with `os.waitpid`.
 - `os.{S_IRWXU,S_IRUSR,S_IWUSR,S_IXUSR,S_IRWXG,S_IRGRP,S_IWGRP,S_IXGRP,S_IRWXO,S_IROTH,S_IWOTH,S_IXOTH}` added, for working with file modes.
@@ -42,8 +57,6 @@ Fork of the fantastic QuickJS engine by Fabrice Bellard, with the following chan
 - `std.strftime` added (wrapper for libc `strftime`).
 - Added `std.getuid`, `std.geteuid`, `std.getgid`, and `std.getegid` (wrappers for the libc functions of the same names).
 - Most module-loading-related code was moved into `quickjs-modulesys`.
-- Synchronous import function added (`std.importModule`), which provides the same module record object you would get via dynamic (async) import.
-- JS api for using the engine's configured module name normalization function was added (`std.resolveModule`).
 - `setTimeout` and `clearTimeout` are now available as globals (previously they were only available as exports).
 - `setInterval` and `clearInterval` are added, available as globals.
 - `String.dedent` added (template tag function, like the proposed [String.dedent](https://github.com/tc39/proposal-string-dedent)).
@@ -51,15 +64,17 @@ Fork of the fantastic QuickJS engine by Fabrice Bellard, with the following chan
 - Most module-related code (setting import.meta, etc) was moved into quickjs-modulesys.
 - Added `std.setExitCode`, `std.getExitCode`, and made `std.exit`'s parameter optional. The value passed to `std.setExitCode` will be used when the process exits normally, or when `std.exit` is called without any arguments. `std.setExitCode`, `std.getExitCode`, and `std.exit` throw if called from a thread other than the main thread (ie. a Worker).
 
-## Changes to the `qjs` repl:
+### Changes to the `qjs` binary:
 
-- The new `inspect` global is used to print results
-- The last error is accessible via the `_error` global
-- If a thrown error has additional properties added onto it, those are printed along with the thrown error
-- Ctrl+W deletes words backwards from the cursor position
 - Blocking the main thread is allowed
+- `-m` (module mode) now affects eval strings passed to `-e`
+- Changes to the repl:
+  - Ctrl+W deletes words backwards from the cursor position
+  - The last error is accessible via the `_error` global
+  - If a thrown error has additional properties added onto it, those are printed along with the thrown error
+  - The new `inspect` global is used to print results
 
-## New binary: `qjsbootstrap`:
+### New binary: `qjsbootstrap`:
 
 Appending some JS code to the end of this binary changes it into a binary that executes that JS code:
 
@@ -74,23 +89,23 @@ You can use this to create distributable binaries that run JS code without needi
 
 > Note: On FreeBSD, `qjsbootstrap` requires procfs. You can mount it with `mount -t procfs proc /proc`. I started some work to use libprocstat instead, but haven't completed it yet.
 
-## New binary: `quickjs-run`:
+### New binary: `quickjs-run`:
 
 Barebones binary for running files, without any of the arg parsing logic from qjs. Good for testing some unusual cases, or writing programs with custom argv parsing logic.
 
-## New module: "quickjs:bytecode"
+### New module: "quickjs:bytecode"
 
 A Module that exposes QuickJS's value <-> bytecode (de)serialization APIs to JavaScript code. Generated bytecode can be combined with `qjsbootstrap-bytecode` in the same way that source code strings can be combined with `qjsbootstrap`.
 
-## New module: "quickjs:context"
+### New module: "quickjs:context"
 
 A Module that allows JS code to create new JS Contexts (Realms). You can create new Contexts and run code inside them. Contexts can have certain features disabled (like eval) for security purposes. You can share values between Contexts. Contexts are destroyed when they get garbage-collected.
 
-## New module: "quickjs:pointer"
+### New module: "quickjs:pointer"
 
 A barebones Module that exports a JS class which can be used to represent an opaque pointer. C modules can use the `js_new_pointer` function provided by this module to pass opaque pointer handles to users without needing to make their own wrapper class for stuff. This is mostly just useful in order to have a codified convention for how FFI libraries and such should represent foreign pointers.
 
-## New library: `quickjs-utils`
+### New library: `quickjs-utils`
 
 Helper structs, functions, and macros that make it easier to work with QuickJS in C code.
 
@@ -98,22 +113,19 @@ Helper structs, functions, and macros that make it easier to work with QuickJS i
 - Helper function for loading a file from disk into char a buffer
 - Helper functions for printing JS errors to stderr
 
-## New library: `quickjs-modulesys`
+### Changes to the module loader
 
-- Module-loading code in `quickjs-libc` was moved into `quickjs-modulesys`, with the following changes:
-  - `.js` extensions can now be omitted from import specifiers; they're optional.
-  - If your import specifier points to a folder, it will attempt to load `index.js` from that folder.
+- `.js` extensions can now be omitted from import specifiers; they're optional.
+- If your import specifier points to a folder, it will attempt to load `index.js` from that folder.
 - Adds the global `require`, a CommonJS-like synchronous module loading function.
   - The `require` function is not fully CommonJS-compliant; for instance, `require.main` is not present. `require.resolve` is, though.
 - Adds `import.meta.require`
   - It's the same as the global `require`; it's just added to import.meta for compatibility with bundlers that output `import.meta.require`, like `bun`.
 - Adds `import.meta.resolve`
   - Similar to [the one in the browser](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/import.meta/resolve#specifications), but it's actually just `require.resolve` exposed via `import.meta`.
-- Makes `import.meta.main` configurable
-  - You can use `std.setMainModule` to make `import.meta.main` true within that module's code. Note, however, that it does not work retroactively; only modules loaded after the `setMainModule` call will be affected. To defer module load, use `import()`, `std.importModule`, or `require`.
-  - You can use `std.isMainModule` to check if a given module would be the main module without loading it.
-- Adds the global object `Module`.
-  - `instanceof Module` can be used to identify module namespace objects.
+- Module and eval helpers have been moved from "quickjs:std" to the new module "quickjs:module".
+- Makes the module loader's resolution and loading behavior configurable, and allows users to define additional builtin modules
+  - The module "quickjs:module" exports an object called `Module`.
   - You can specify additional implicit import specifier extensions by adding to the `Module.searchExtensions` array.
   - You can transform any file prior to evaluating it as a module by adding a function to the `Module.compilers` object. Useful for compile-to-ts languages like TypeScript, Coffeescript, etc.
   - You can define custom builtin modules using the `Module.define` function.
@@ -121,32 +133,41 @@ Helper structs, functions, and macros that make it easier to work with QuickJS i
     - Note that you must handle `Module.searchExtensions` yourself in your replacement implementation.
   - You can override the method used to load modules by replacing the `Module.read` function.
     - Note that you must handle `Module.compilers` yourself in your replacement implementation.
-- The `eval_*` functions that were duplicated in each of the programs (`eval_buf`, `eval_file`, and `eval_binary`) were moved here
-- The default module loader function from quickjs-libc was moved here and augmented to support the features related to the `Module` global as mentioned above
-- When using `require` to load a module which contains an export named `__cjsExports`, the value of the `__cjsExports` property will be returned from `require` instead of the usual module namespace object. This can be leveraged by user-defined properties on `Module` to add some CommonJS <-> ESM interop. Note, however, that dynamic import and `std.importModule` always receive the usual module namespace object.
-
-## Changes to project organization
+- Makes `import.meta.main` configurable
+  - The module "quickjs:module" exports two functions named `setMainModule` and `isMainModule`.
+  - You can use `setMainModule` to make `import.meta.main` true within that module's code. Note, however, that it does not work retroactively; only modules loaded after the `setMainModule` call will be affected. To defer module load, use `import()`, `importModule` from "quickjs:module", or `require`.
+  - You can use `isMainModule` to check if a given module would be the main module without loading it.
+- Adds a mechanism for identifying module namespace objects
+  - The module "quickjs:module" exports an object called `Module`.
+  - `something instanceof Module` will be true when `something` is a module namespace object.
+- When using `require` to load a module which contains an export named `__cjsExports`, the value of the `__cjsExports` property will be returned from `require` instead of the usual module namespace object. This can be leveraged by users configuring the module loader to add some CommonJS <-> ESM interop. Note, however, that dynamic import and `"quickjs:module"`'s `importModule` always receive the usual module namespace object.
+- Synchronous import function added (`importModule`), which provides the same module record object you would get via dynamic (async) import.
+- JS api for using the engine's configured module name normalization function was added (`resolveModule`).
+
+### Changes to project organization
 
 - Stuff is reorganized into separate folders under `src`.
 - Ninja is used instead of make. Ninja build config is generated via `.ninja.js` files which get loaded into [@suchipi/shinobi](https://github.com/suchipi/shinobi).
 - Line endings have been made consistent and trailing whitespace has been removed
 - The tests are authored in a new format which leverages jest snapshot testing.
+- Module-loading code in `quickjs-libc` was moved into `quickjs-modulesys` and `quickjs-libmodule`.
+- The `eval_*` functions that were duplicated in each of the programs (`eval_buf`, `eval_file`, and `eval_binary`) were deduplicated and moved into `quickjs-modulesys`.
 
-## More target OSes/runtimes
+### More target OSes/runtimes
 
 We now include support for more platforms, and cross-compilation scripts to build most of those platforms from any platform where Docker is available.
 
 See the `meta/ninja/envs` folder to see all the supported platforms. The `host` folder represents the machine where compilation is taking place, and the `target` folder represents the target platform for the output binaries.
 
-## Library Archives
+### Library Archives
 
 We create `.a` files containing all of quickjs as part of the build.
 
-## Other changes
+### Other changes
 
 There are also probably some other miscellaneous changes I forgot to write down in the README.
 
-# Compiling
+## Compiling
 
 The repo has stuff set up to compile quickjs binaries for:
 
@@ -166,7 +187,7 @@ Linux, macOS, iOS, and Windows binaries can be compiled using Docker. Or, you ca
 
 If you're not gonna use Docker, you'll need to install [Ninja](https://ninja-build.org/) and [Node.js](https://nodejs.org/) in order to compile. I use Ninja 1.10.1 and Node.js 18.12.1, but it should work with most versions of both of those.
 
-## Compilation Instructions
+### Compilation Instructions
 
 To compile binaries for Linux, macOS, iOS, and Windows (using Docker):
 

package/build/dts/quickjs-libc.d.ts

@@ -98,6 +98,20 @@ declare interface FILE {
   /** Write `length` bytes from the ArrayBuffer `buffer` at byte position `position` into the file (wrapper to the libc `fwrite`). Returns the number of bytes written. */
   write(buffer: ArrayBuffer, position: number, length: number): number;
 
+  /**
+   * Write this file into `target`, using a memory buffer of size `bufferSize`.
+   *
+   * If `limit` is specified, only that amount of bytes will be read and
+   * written. Otherwise, data is read and written until this file reaches EOF.
+   *
+   * A `limit` of 0 is treated the same as not specifying a limit.
+   *
+   * Internally, this function uses libc `fread` and `fwrite` in a loop.
+   *
+   * Returns the number of bytes read and written.
+   */
+  writeTo(target: FILE, bufferSize: number, limit?: number): number;
+
   /**
    * Return the next line from the file, assuming UTF-8 encoding, excluding the trailing line feed or EOF.
    *

package/build/dts/quickjs-libmodule.d.ts

@@ -69,4 +69,13 @@ declare module "quickjs:module" {
    * @param stackLevels - How many levels up the stack to search for a filename. Defaults to 0, which uses the current stack frame.
    */
   export function getFileNameFromStack(stackLevels?: number): string;
+
+  /**
+   * An object which lets you configure the module loader (import/export/require).
+   * You can use these properties to add support for importing new filetypes.
+   *
+   * This object can also be used to identify whether an object is a module
+   * namespace record.
+   */
+  export const Module: Module;
 }

package/build/dts/quickjs-modulesys.d.ts

@@ -1,11 +1,11 @@
 /**
- * A global which lets you configure the module loader (import/export/require).
+ * An object which lets you configure the module loader (import/export/require).
  * You can use these properties to add support for importing new filetypes.
  *
- * This global can also be used to identify whether an object is a module
+ * This object can also be used to identify whether an object is a module
  * namespace record.
  */
-interface ModuleGlobal {
+interface Module {
   /**
    * Returns true if `target` is a module namespace object.
    */
@@ -24,7 +24,7 @@ interface ModuleGlobal {
    * See the doc comment on {@link require} for more information.
    *
    * NOTE: If you add a new extension to this array, you will likely also want
-   * to add to {@link Module.compilers}.
+   * to add to {@link compilers}.
    */
   searchExtensions: Array<string>;
 
@@ -65,7 +65,7 @@ interface ModuleGlobal {
    * And `names` will be a string containing the contents of names.txt.
    *
    * NOTE: When adding to this object, you may also wish to add to
-   * {@link Module.searchExtensions}.
+   * {@link searchExtensions}.
    */
   compilers: {
     [extensionWithDot: string]: (filename: string, content: string) => string;
@@ -78,7 +78,8 @@ interface ModuleGlobal {
   define(name: string, obj: { [key: string]: any }): void;
 
   /**
-   * Resolves a require/import request from `fromFile` into a canonicalized path.
+   * Resolves a require/import request from `fromFile` into a canonicalized
+   * path.
    *
    * To change native module resolution behavior, replace this function with
    * your own implementation. Note that you must handle
@@ -96,9 +97,6 @@ interface ModuleGlobal {
   read(modulePath: string): string;
 }
 
-// global added by QJMS_AddModuleGlobal
-declare var Module: ModuleGlobal;
-
 interface RequireFunction {
   /**
    * Synchronously import a module.
@@ -125,8 +123,8 @@ interface RequireFunction {
    * ```
    *
    * then the engine will look for `somewhere.js`. If that doesn't exist, the
-   * engine will look for `somewhere/index.js`. If *that* doesn't exist, an error
-   * will be thrown.
+   * engine will look for `somewhere/index.js`. If *that* doesn't exist, an
+   * error will be thrown.
    *
    * If you add more extensions to `Module.searchExtensions`, then the engine
    * will use those, too. It will search in the same order as the strings appear
@@ -140,7 +138,7 @@ interface RequireFunction {
   resolve: (source: string) => string;
 }
 
-// global added by QJMS_AddRequireGlobal
+// global added by QJMS_InitContext
 declare var require: RequireFunction;
 
 // gets set per-module by QJMS_SetModuleImportMeta

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@suchipi/quickjs",
   "description": "Suchipi's QuickJS Fork",
-  "version": "0.5.0",
+  "version": "0.6.0",
   "main": "./npm/index.js",
   "bin": {
     "qjs": "npm/cli/qjs.js",

package/.gitmodules

@@ -1,6 +0,0 @@
-[submodule "src/run-test262/test262"]
-	path = src/run-test262/test262
-	url = https://github.com/tc39/test262.git
-[submodule "src/run-test262/test262o"]
-	path = src/run-test262/test262o
-	url = https://github.com/tc39/test262.git