@thi.ng/wasm-api 0.7.0 → 0.10.0

package/CHANGELOG.md

@@ -1,6 +1,6 @@
 # Change Log
 
-- **Last updated**: 2022-08-15T23:41:37Z
+- **Last updated**: 2022-08-24T09:52:32Z
 - **Generator**: [thi.ng/monopub](https://thi.ng/monopub)
 
 All notable changes to this project will be documented in this file.
@@ -9,6 +9,40 @@ See [Conventional Commits](https://conventionalcommits.org/) for commit guidelin
 **Note:** Unlisted _patch_ versions only involve non-code or otherwise excluded changes
 and/or version bumps of transitive dependencies.
 
+## [0.10.0](https://github.com/thi-ng/umbrella/tree/@thi.ng/wasm-api@0.10.0) (2022-08-24)
+
+#### 🚀 Features
+
+- add events, update allocator handling ([89416b1](https://github.com/thi-ng/umbrella/commit/89416b1))
+  - add INotify impl for WasmBridge
+  - emit event when WASM memory has changed (e.g. to recreate user views)
+  - reverse logic so that NO allocator is used by default and instead must
+    be explicitly enabled (rather than disabled)
+  - update Zig & C bindings
+  - add/update docstrings
+
+## [0.9.0](https://github.com/thi-ng/umbrella/tree/@thi.ng/wasm-api@0.9.0) (2022-08-23)
+
+#### 🚀 Features
+
+- add debug() core API function ([ca01978](https://github.com/thi-ng/umbrella/commit/ca01978))
+
+#### 🩹 Bug fixes
+
+- update setString() return value ([116dd0b](https://github.com/thi-ng/umbrella/commit/116dd0b))
+  - always only return num bytes written w/o sentinel
+
+## [0.8.0](https://github.com/thi-ng/umbrella/tree/@thi.ng/wasm-api@0.8.0) (2022-08-16)
+
+#### 🚀 Features
+
+- add preliminary string handling support ([3da4efe](https://github.com/thi-ng/umbrella/commit/3da4efe))
+  - update/rename IWasmMemoryAccess (add string getter/setter)
+  - update StructField.type (add `string`)
+  - add CodeGenOpts.stringType option
+  - update codegen fns
+  - update TS & Zig codegen impls
+
 ## [0.7.0](https://github.com/thi-ng/umbrella/tree/@thi.ng/wasm-api@0.7.0) (2022-08-15)
 
 #### 🚀 Features

package/README.md

@@ -11,11 +11,13 @@ This project is part of the
 
 - [About](#about)
   - [Custom API modules](#custom-api-modules)
-  - [Object indices & handles](#object-indices--handles)
   - [Data bindings & code generators](#data-bindings--code-generators)
     - [CLI generator](#cli-generator)
     - [Data type definitions](#data-type-definitions)
     - [Example usage](#example-usage)
+  - [String handling](#string-handling)
+  - [Memory allocations](#memory-allocations)
+  - [Object indices & handles](#object-indices--handles)
   - [Status](#status)
 - [Installation](#installation)
 - [Dependencies](#dependencies)
@@ -36,9 +38,10 @@ This package provides a the following:
 [`WasmBridge`](https://docs.thi.ng/umbrella/wasm-api/classes/WasmBridge.html)
 class as interop basis and much reduced boilerplate for hybrid JS/WebAssembly
 applications.
-2. A minimal core API for debug output, string, pointer, typed array accessors
-for 8/16/32/64 bit (u)ints and 32/64 bit floats. In the future we aim to also
-supply support modules for DOM manipulation, WebGL, WebGPU, WebAudio etc.
+2. A minimal core API for memory allocation (can be disabled), debug output,
+string/pointer/typedarray accessors for 8/16/32/64 bit (u)ints and 32/64 bit
+floats. In the future we aim to also supply support modules for DOM
+manipulation, WebGL, WebGPU, WebAudio etc.
 3. [Include files for C11/C++ and
    Zig](https://github.com/thi-ng/umbrella/tree/develop/packages/wasm-api/include),
    defining WASM imports of the JS [core
@@ -46,17 +49,20 @@ supply support modules for DOM manipulation, WebGL, WebGPU, WebAudio etc.
    by this package
 4. Extensible shared datatype code generators for (currently)
    [Zig](https://ziglang.org) & TypeScript. The latter also generates fully type
-   checked memory-mapped accessors of WASM-side data. In general, all languages
-with a WebAssembly target are supported, however currently only bindings for
-these few langs are included.
+   checked memory-mapped (zero-copy) accessors of WASM-side data. In general,
+all languages with a WebAssembly target are supported, however currently only
+bindings for these few langs are included.
 5. [CLI frontend/utility](#cli-generator) to invoke the code generator(s)
 
 ### Custom API modules
 
-The [`WasmBridge`](https://docs.thi.ng/umbrella/wasm-api/classes/WasmBridge.html) is extensible via custom defined API modules. Such API extensions
-will consist of a collection of JS/TS functions & variables, their related
-counterparts (import definitions) for the WASM target and (optionally) some
-shared data types ([bindings for which _can_ be generated by this package too](#data-bindings--code-generators)).
+The
+[`WasmBridge`](https://docs.thi.ng/umbrella/wasm-api/classes/WasmBridge.html) is
+extensible via custom defined API modules. Such API extensions will consist of a
+collection of JS/TS functions & variables, their related counterparts (import
+definitions) for the WASM target and (optionally) some shared data types
+([bindings for which _can_ be generated by this package
+too](#data-bindings--code-generators)).
 
 On the JS side, custom API modules can be easily integrated via the [`IWasmAPI`
 interface](https://docs.thi.ng/umbrella/wasm-api/interfaces/IWasmAPI.html). The
@@ -131,53 +137,6 @@ export fn test_randomVec2() void {
 }
 ```
 
-### Object indices & handles
-
-Since only numeric values can be exchanged between the WASM module and the JS
-host, any JS native objects the WASM side might want to be working with must be
-managed manually in JS. For this purpose the [`ObjectIndex`
-class](https://docs.thi.ng/umbrella/wasm-api/classes/ObjectIndex.html) can be
-used by API modules to handle ID generation (incl. recycling, using
-[@thi.ng/idgen](https://github.com/thi-ng/umbrella/tree/develop/packages/idgen))
-and the indexing of different types of JS objects/values. Only the numeric IDs
-(handles) will then need to be exchanged with the WASM module...
-
-```ts
-import { ObjectIndex } from "@thi.ng/wasm-api";
-
-const canvases = new ObjectIndex<HTMLCanvasElement>({ name: "canvas" });
-
-// index item and assign new ID
-canvases.add(document.createElement("canvas"));
-// 0
-
-// look up item by ID
-canvases.get(0);
-// <canvas ...>
-
-// work w/ retrieved item
-canvases.get(0).id = "foo";
-
-// check if item for ID exists (O(1))
-canvases.has(1)
-// false
-
-// by default invalid IDs throw error
-canvases.get(1)
-// Uncaught Error: Assertion failed: missing canvas for ID: 2
-
-// error can be disabled via 2nd arg
-canvases.get(1, false)
-// undefined
-
-// find ID using custom predicate (same failure behavior as .get())
-canvases.find((x) => x.id == "bar")
-// Uncaught Error: Assertion failed: given predicate matched no canvas
-
-canvases.delete(0);
-// true
-```
-
 ### Data bindings & code generators
 
 The package provides an extensible codegeneration framework to simplify the
@@ -193,7 +152,7 @@ to invoke the codegenerator(s) from JSON type definitions and to write the
 generated source code(s) to different files:
 
 ```text
-$ npx run @thi.ng/wasm-api
+$ npx @thi.ng/wasm-api
 
  █ █   █           │
 ██ █               │
@@ -261,7 +220,9 @@ further details:
 Below is an example file with JSON type definitions and the resulting source
 codes:
 
-<details><summary>types.json (Type definitions, click to expand)</summary>
+**⬇︎ CLICK TO EXPAND EACH CODE BLOCK ⬇︎**
+
+<details><summary>types.json (Type definitions)</summary>
 
 ```json
 [
@@ -298,7 +259,7 @@ codes:
 ```
 </details>
 
-<details><summary>generated.ts (generated TypeScript source, click to expand)</summary>
+<details><summary>generated.ts (generated TypeScript source)</summary>
 
 ```ts
 /** Generated by @thi.ng/wasm-api at 2022-08-15T22:32:21.189Z - DO NOT EDIT! */
@@ -373,7 +334,7 @@ export enum Kind {
 ```
 </details>
 
-<details><summary>generated.zig (generated Zig source, click to expand)</summary>
+<details><summary>generated.zig (generated Zig source)</summary>
 
 ```zig
 //! Generated by @thi.ng/wasm-api at 2022-08-15T22:32:21.191Z - DO NOT EDIT!
@@ -422,13 +383,131 @@ foo.color
 // Float32Array(4) [0.1, 0.2, 0.3, 0.4]
 
 // this even applies to arrays using other types
-// (setters are currently only supported for scalar values, incl. enums)
 foo.bars[2].kind = Kind.BEST;
 
 // IMPORTANT: any modifications like this are directly
 // applied to the underlying WASM memory...
 ```
 
+**IMPORTANT:** Struct field setters are currently only supported for single
+values, incl. enums, strings, structs. The latter 2 will always be copied by
+value (mem copy). Arrays or slices of strings do not currently provide write
+access...
+
+### String handling
+
+Most low-level languages deal with strings very differently and alas there's no
+general standard. Some have UTF-8/16 support, others don't. In some languages
+(incl. C & Zig), strings are stored as zero terminated, in others they aren't...
+It's outside the scope of this package to provide an allround out-of-the-box
+solution. The `WasmBridge` provides read & write accessors to obtain JS strings
+from UTF-8 encoded WASM memory. See
+[`getString()`](https://docs.thi.ng/umbrella/wasm-api/classes/WasmBridge.html#getString)
+and
+[`setString()`](https://docs.thi.ng/umbrella/wasm-api/classes/WasmBridge.html#setString)
+for details.
+
+The code generators too provide a global `stringType` option to
+interpret the `string` type of a struct field in different ways:
+
+- `slice` (default): Considers strings as Zig-style slices (i.e. pointer + length)
+- `ptr`: Considers strings as C-style raw `*char` pointer (without any length)
+
+Note: If setting this global option to `ptr`, it also has to be repeated for the
+TypeScript code generator explicitly.
+
+### Memory allocations
+
+If explicitly enabled on the WASM side, the `WasmBridge` includes support for
+malloc/free-style allocations (within the linear WASM memory) from the JS side
+(Note: This is a breaking change in v0.10.0, now using a more flexible approach
+& reverse logic of earlier alpha versions).
+
+The actual allocator is implementation specific and suitable generic mechanisms
+are defined for both the included Zig & C bindings. Please see for further
+reference:
+
+- [`/include/wasmapi.zig`](https://github.com/thi-ng/umbrella/blob/develop/packages/wasm-api/include/wasmapi.zig#L6):
+  comments about WASM-side allocator handling in Zig
+- [`/include/wasmapi.h`](https://github.com/thi-ng/umbrella/blob/develop/packages/wasm-api/include/wasmapi.h#L19):
+  comments about WASM-side allocator handling in C/C++
+- [`WasmBridge.allocate()`](https://docs.thi.ng/umbrella/wasm-api/classes/WasmBridge.html#allocate):
+  allocating memory from JS side
+- [`WasmBridge.free()`](https://docs.thi.ng/umbrella/wasm-api/classes/WasmBridge.html#free):
+  freeing previously allocated memory from JS side
+
+Note: The provided Zig mechanism supports the idiomatic (Zig) pattern of working
+with multiple allocators in different parts of the application and supports
+dynamic assignments/swapping of the exposed allocator. See comments in source
+file for more details...
+
+```ts
+try {
+	// allocate 1KB of memory for passing a string to WASM side
+	const addr = bridge.allocate(256);
+
+	// write string to reserved memory
+	// max. 256 bytes, zero terminated
+	const num = bridge.setString("hello WASM world!", addr, 256, true);
+
+	// call WASM function doing something w/ the string
+	bridge.exports.doSomethingWithString(addr, num);
+
+	// cleanup
+	bridge.free(addr, 256);
+} catch(e) {
+	// deal with allocation error
+	// ...
+}
+```
+
+### Object indices & handles
+
+Since only numeric values can be exchanged between the WASM module and the JS
+host, any JS native objects the WASM side might want to be working with must be
+managed manually in JS. For this purpose the [`ObjectIndex`
+class](https://docs.thi.ng/umbrella/wasm-api/classes/ObjectIndex.html) can be
+used by API modules to handle ID generation (incl. recycling, using
+[@thi.ng/idgen](https://github.com/thi-ng/umbrella/tree/develop/packages/idgen))
+and the indexing of different types of JS objects/values. Only the numeric IDs
+(handles) will then need to be exchanged with the WASM module...
+
+```ts
+import { ObjectIndex } from "@thi.ng/wasm-api";
+
+const canvases = new ObjectIndex<HTMLCanvasElement>({ name: "canvas" });
+
+// index item and assign new ID
+canvases.add(document.createElement("canvas"));
+// 0
+
+// look up item by ID
+canvases.get(0);
+// <canvas ...>
+
+// work w/ retrieved item
+canvases.get(0).id = "foo";
+
+// check if item for ID exists (O(1))
+canvases.has(1)
+// false
+
+// by default invalid IDs throw error
+canvases.get(1)
+// Uncaught Error: Assertion failed: missing canvas for ID: 2
+
+// error can be disabled via 2nd arg
+canvases.get(1, false)
+// undefined
+
+// find ID using custom predicate (same failure behavior as .get())
+canvases.find((x) => x.id == "bar")
+// Uncaught Error: Assertion failed: given predicate matched no canvas
+
+canvases.delete(0);
+// true
+```
+
 ### Status
 
 **ALPHA** - bleeding edge / work-in-progress
@@ -458,7 +537,7 @@ node --experimental-repl-await
 > const wasmApi = await import("@thi.ng/wasm-api");
 ```
 
-Package sizes (gzipped, pre-treeshake): ESM: 4.03 KB
+Package sizes (gzipped, pre-treeshake): ESM: 4.48 KB
 
 **IMPORTANT:** The package includes various code generators and supporting
 functions which are NOT required during runtime. Hence the actual package size

package/api.d.ts

@@ -1,6 +1,7 @@
 import type { BigType, FloatType, Fn, Fn2 } from "@thi.ng/api";
 import type { WasmBridge } from "./bridge.js";
 export declare const PKG_NAME = "@thi.ng/wasm-api";
+export declare const EVENT_MEMORY_CHANGED = "memory-changed";
 export declare type BigIntArray = bigint[] | BigInt64Array | BigUint64Array;
 /**
  * Common interface for WASM/JS child APIs which will be used in combination
@@ -27,9 +28,9 @@ export interface IWasmAPI<T extends WasmExports = WasmExports> {
     getImports(): WebAssembly.ModuleImports;
 }
 /**
- * Base interface of exports declared by the WASM module. At the very least, the
- * module needs to export its memory and the functions defined in this
- * interface.
+ * Base interface of exports declared by the WASM module. At the very least, a
+ * compatible module needs to export its memory and the functions defined in
+ * this interface.
  *
  * @remarks
  * This interface is supposed to be extended with the concrete exports defined
@@ -45,28 +46,41 @@ export interface WasmExports {
      */
     memory: WebAssembly.Memory;
     /**
-     * Implementation specific memory allocation function (likely heap-based).
-     * If successful returns address of new memory block, or zero if
-     * unsuccessful.
+     * Implementation specific WASM memory allocation function. If successful
+     * returns address of new memory block, or zero if unsuccessful.
      *
      * @remarks
-     * In the supplied Zig bindings (see `/zig/core.zig`), by default this is
-     * using the `std.heap.GeneralPurposeAllocator` (which also automatically
-     * handles growing the WASM memory), however as mentioned the underlying
-     * mechanism is purposefully left to the actual WASM-side implementation. In
-     * a C program, this would likely use `malloc()` or similar...
+     * #### Zig
+     *
+     * Using the supplied Zig bindings (see `/include/wasmapi.zig`), it's the
+     * user's responsibility to define a public `WASM_ALLOCATOR` in the root
+     * source file to enable allocations, e.g. using the
+     * [`std.heap.GeneralPurposeAllocator`](https://ziglang.org/documentation/master/#Choosing-an-Allocator)
+     * (which also automatically handles growing the WASM memory). However, as
+     * mentioned, the underlying mechanism is purposefully left to the actual
+     * WASM-side implementation. If no allocator is defined this function
+     * returns zero, which in turn will cause {@link WasmBridge.allocate} to
+     * throw an error.
+     *
+     * #### C/C++
+     *
+     * Using the supplied C bindings (see `/include/wasmapi.h`), it's the user's
+     * responsibility to enable allocation support by defining the
+     * `WASMAPI_MALLOC` symbol (and compiling the WASM module with a malloc
+     * implementation).
      */
     _wasm_allocate(numBytes: number): number;
     /**
      * Implementation specific function to free a previously allocated chunk of
-     * of WASM memory (allocated via {@link WasmExports._wasm_allocate}).
+     * of WASM memory (allocated via {@link WasmExports._wasm_allocate}, also
+     * see remarks for that function).
      *
      * @param addr
      * @param numBytes
      */
     _wasm_free(addr: number, numBytes: number): void;
 }
-export interface WasmMemViews {
+export interface IWasmMemoryAccess {
     i8: Int8Array;
     u8: Uint8Array;
     i16: Int16Array;
@@ -77,6 +91,31 @@ export interface WasmMemViews {
     u64: BigUint64Array;
     f32: Float32Array;
     f64: Float64Array;
+    /**
+     * Reads UTF-8 encoded string from given address and optional byte length.
+     * The default length is 0, which will be interpreted as a zero-terminated
+     * string. Returns string.
+     *
+     * @param addr
+     * @param len
+     */
+    getString(addr: number, len?: number): string;
+    /**
+     * Encodes given string as UTF-8 and writes it to WASM memory starting at
+     * `addr`. By default the string will be zero-terminated and only `maxBytes`
+     * will be written. Returns the number of bytes written.
+     *
+     * @remarks
+     * An error will be thrown if the encoded string doesn't fully fit into the
+     * designated memory region (also note that there might need to be space for
+     * the additional sentinel/termination byte).
+     *
+     * @param str
+     * @param addr
+     * @param maxBytes
+     * @param terminate
+     */
+    setString(str: string, addr: number, maxBytes: number, terminate?: boolean): number;
 }
 /**
  * Core API of WASM imports defined by the {@link WasmBridge}. The same
@@ -110,6 +149,7 @@ export interface CoreAPI extends WebAssembly.ModuleImports {
     _printF64Array: (addr: number, len: number) => void;
     _printStr0: (addr: number) => void;
     _printStr: (addr: number, len: number) => void;
+    debug: () => void;
 }
 export interface WasmTypeBase {
     /**
@@ -126,7 +166,7 @@ export interface WasmType<T> {
     readonly size: number;
     instance: Fn<number, T>;
 }
-export declare type WasmTypeConstructor<T> = Fn<WasmMemViews, WasmType<T>>;
+export declare type WasmTypeConstructor<T> = Fn<IWasmMemoryAccess, WasmType<T>>;
 export declare type WasmInt = "i8" | "i16" | "i32" | "i64";
 export declare type WasmUint = "u8" | "u16" | "u32" | "u64";
 export declare type WasmFloat = FloatType;
@@ -210,13 +250,16 @@ export interface StructField extends TypeInfo {
      */
     tag?: "scalar" | "array" | "ptr" | "slice" | "vec";
     /**
-     * Field base type. If not a {@link WasmPrim} or `opaque`, the value is
-     * interpreted as another type name in the {@link TypeColl}.
+     * Field base type. If not a {@link WasmPrim}, `string` or `opaque`, the
+     * value is interpreted as another type name in the {@link TypeColl}.
+     *
+     * @remarks
+     * Please see {@link CodeGenOpts.stringType} and consult package readme for
+     * further details re: string handling.
      *
      * TODO `opaque` currently unsupported.
-     * TODO add string support (see {@link StructField.sentinel})
      */
-    type: WasmPrim | "opaque" | string;
+    type: WasmPrim | "string" | "opaque" | string;
     /**
      * TODO currently unsupported & ignored!
      */

package/api.js

@@ -1,4 +1,5 @@
 export const PKG_NAME = "@thi.ng/wasm-api";
+export const EVENT_MEMORY_CHANGED = "memory-changed";
 /**
  * WASM usize type. Assuming wasm32 until wasm64 surfaces, then need an option.
  */

package/bridge.d.ts

@@ -1,7 +1,7 @@
 /// <reference types="node" />
-import type { NumericArray } from "@thi.ng/api";
+import type { Event, INotify, Listener, NumericArray } from "@thi.ng/api";
 import type { ILogger } from "@thi.ng/logger";
-import type { BigIntArray, CoreAPI, IWasmAPI, WasmExports, WasmMemViews } from "./api.js";
+import { BigIntArray, CoreAPI, IWasmAPI, WasmExports, IWasmMemoryAccess } from "./api.js";
 export declare const OutOfMemoryError: {
     new (msg?: string | undefined): {
         name: string;
@@ -28,7 +28,7 @@ export declare const OutOfMemoryError: {
  * 64bit integers are handled via JS `BigInt` and hence require the host env to
  * support it. No polyfill is provided.
  */
-export declare class WasmBridge<T extends WasmExports = WasmExports> implements WasmMemViews {
+export declare class WasmBridge<T extends WasmExports = WasmExports> implements IWasmMemoryAccess, INotify {
     modules: Record<string, IWasmAPI<T>>;
     logger: ILogger;
     i8: Int8Array;
@@ -66,14 +66,23 @@ export declare class WasmBridge<T extends WasmExports = WasmExports> implements
      * then initializes all declared bridge child API modules. Returns false if
      * any of the module initializations failed.
      *
+     * @remarks
+     * Emits the {@link EVENT_MEMORY_CHANGED} event just before returning (and
+     * AFTER all child API modules have been initialized).
+     *
      * @param exports
      */
     init(exports: T): Promise<boolean>;
     /**
-     * Called automatically. Initializes and/or updates the various typed WASM
-     * memory views (e.g. after growing the WASM memory).
+     * Called automatically during initialization. Initializes and/or updates
+     * the various typed WASM memory views (e.g. after growing the WASM memory
+     * and the previous buffer becoming detached). Unless `notify` is false,
+     * the {@link EVENT_MEMORY_CHANGED} event will be emitted if the memory
+     * views had to be updated.
+     *
+     * @param notify
      */
-    ensureMemory(): void;
+    ensureMemory(notify?: boolean): void;
     /**
      * Required use for WASM module instantiation to provide JS imports to the
      * module. Returns an object of all WASM imports declared by the bridge core
@@ -137,6 +146,10 @@ export declare class WasmBridge<T extends WasmExports = WasmExports> implements
      * `numBytes` value must be the same as previously given to
      * {@link WasmBridge.allocate}.
      *
+     * @remarks
+     * This function always succeeds, regardless of presence of an active
+     * allocator on the WASM side or validity of given arguments.
+     *
      * @param addr
      * @param numBytes
      */
@@ -193,7 +206,8 @@ export declare class WasmBridge<T extends WasmExports = WasmExports> implements
     /**
      * Encodes given string as UTF-8 and writes it to WASM memory starting at
      * `addr`. By default the string will be zero-terminated and only `maxBytes`
-     * will be written. Returns the number of bytes written.
+     * will be written. Returns the number of bytes written (excluding final
+     * sentinel, if any).
      *
      * @remarks
      * An error will be thrown if the encoded string doesn't fully fit into the
@@ -207,5 +221,11 @@ export declare class WasmBridge<T extends WasmExports = WasmExports> implements
      */
     setString(str: string, addr: number, maxBytes: number, terminate?: boolean): number;
     getElementById(addr: number, len?: number): HTMLElement;
+    /** {@inheritDoc @thi.ng/api#INotify.addListener} */
+    addListener(id: string, fn: Listener, scope?: any): boolean;
+    /** {@inheritDoc @thi.ng/api#INotify.removeListener} */
+    removeListener(id: string, fn: Listener, scope?: any): boolean;
+    /** {@inheritDoc @thi.ng/api#INotify.notify} */
+    notify(event: Event): void;
 }
 //# sourceMappingURL=bridge.d.ts.map