@thi.ng/wasm-api 0.3.1 → 0.6.0

package/CHANGELOG.md

@@ -1,6 +1,6 @@
 # Change Log
 
-- **Last updated**: 2022-08-04T21:21:08Z
+- **Last updated**: 2022-08-15T15:40:55Z
 - **Generator**: [thi.ng/monopub](https://thi.ng/monopub)
 
 All notable changes to this project will be documented in this file.
@@ -9,6 +9,60 @@ See [Conventional Commits](https://conventionalcommits.org/) for commit guidelin
 **Note:** Unlisted _patch_ versions only involve non-code or otherwise excluded changes
 and/or version bumps of transitive dependencies.
 
+## [0.6.0](https://github.com/thi-ng/umbrella/tree/@thi.ng/wasm-api@0.6.0) (2022-08-15)
+
+#### 🚀 Features
+
+- add C11 header/include file, update WasmBridge ([a67dc00](https://github.com/thi-ng/umbrella/commit/a67dc00))
+  - migrate headers/includes to /include
+  - rename "core" import section => "wasmapi"
+  - rename WasmBridge.core => WasmBridge.api
+  - update pkg file
+- update codegens, add opts, fix alignments ([5c1fec5](https://github.com/thi-ng/umbrella/commit/5c1fec5))
+  - add global CodeGenOpts
+  - update generateTypes() to consider new opts
+  - add global USIZE type (for pointer sizes & codegens)
+  - add options for Zig codegen (extra debug helpers)
+  - simplify TS codegen
+  - fix sizeOf() for struct fields
+  - make prepareType() idempotent
+- add bindings code generator framework ([17ee06f](https://github.com/thi-ng/umbrella/commit/17ee06f))
+  - add/update deps
+  - add preliminary codegens for Zig & TS
+  - add supporting types & utils
+  - add generateTypes() codegen facade fn
+- update allocate/free() fns, update Zig core API ([8a55989](https://github.com/thi-ng/umbrella/commit/8a55989))
+  - add _wasm_free() Zig impl
+  - add printFmt() Zig fn
+  - update WasmBridge.allocate() (add clear option)
+  - update WasmBridge.free()
+  - ensure memory in WasmBridge.getString()
+  - add/update docstrings
+
+#### ♻️ Refactoring
+
+- extract WasmMemViews interface, update test WASM ([4c73e65](https://github.com/thi-ng/umbrella/commit/4c73e65))
+
+## [0.5.0](https://github.com/thi-ng/umbrella/tree/@thi.ng/wasm-api@0.5.0) (2022-08-08)
+
+#### 🚀 Features
+
+- add memory allocation ([980c1f2](https://github.com/thi-ng/umbrella/commit/980c1f2))
+  - add WasmBridge.allocate()/free()
+  - add WasmBridge.growMemory()
+  - extract WasmBridge.ensureMemory()
+  - update WasmExports
+  - update Zig bindings (configurable allocator, GPA as default)
+
+## [0.4.0](https://github.com/thi-ng/umbrella/tree/@thi.ng/wasm-api@0.4.0) (2022-08-07)
+
+#### 🚀 Features
+
+- use named import objects ([4965f20](https://github.com/thi-ng/umbrella/commit/4965f20))
+  - switch to name import objects to avoid merging into flat namespace
+  - update externs in core.zig
+  - update docstrings
+
 ## [0.3.0](https://github.com/thi-ng/umbrella/tree/@thi.ng/wasm-api@0.3.0) (2022-08-04)
 
 #### 🚀 Features

package/README.md

@@ -12,30 +12,48 @@ This project is part of the
 - [About](#about)
   - [Custom API modules](#custom-api-modules)
   - [Object indices & handles](#object-indices--handles)
+  - [Data bindings & code generators](#data-bindings--code-generators)
+    - [Data type definitions](#data-type-definitions)
+    - [Code generation](#code-generation)
+    - [Example usage](#example-usage)
   - [Status](#status)
 - [Installation](#installation)
 - [Dependencies](#dependencies)
 - [API](#api)
+- [Basic usage example](#basic-usage-example)
+  - [Zig version](#zig-version)
+  - [C11 version](#c11-version)
 - [Authors](#authors)
 - [License](#license)
 
 ## About
 
-Modular, extensible API bridge and generic glue code between JS & WebAssembly.
+Generic, modular, extensible API bridge, glue code and bindings code generator for hybrid JS & WebAssembly projects.
 
-This package provides a small, generic and modular
-[`WasmBridge`](https://docs.thi.ng/umbrella/wasm-api/classes/WasmBridge.html)
-class as interop basis and a much reduced boilerplate for hybrid JS/WebAssembly
-applications. At the moment only a minimal core API is provided (i.e. for debug
-output, string, pointer, typed array accessors [8/16/32/64 bit (u)ints, 32/64
-bit floats]), but in the future we aim to also supply support modules for DOM
-manipulation, WebGL, WebGPU, WebAudio etc.
+This package provides a the following:
 
-In general, all languages with a WebAssembly target are supported, however
-currently only bindings for [Zig](https://ziglang.org) are included.
+1. A small, generic and modular
+[`WasmBridge`](https://docs.thi.ng/umbrella/wasm-api/classes/WasmBridge.html)
+class as interop basis and much reduced boilerplate for hybrid JS/WebAssembly
+applications.
+2. A minimal core API for debug output, string, pointer, typed array accessors
+(8/16/32/64 bit (u)ints, 32/64 bit floats). In the future we aim to also supply
+support modules for DOM manipulation, WebGL, WebGPU, WebAudio etc.
+3. Extensible shared datatype code generators for (currently)
+   [Zig](https://ziglang.org) & TypeScript. The latter also generates fully type
+   checked memory-mapped accessors of WASM-side data. In general, all languages
+with a WebAssembly target are supported, however currently only bindings for
+these few langs are included.
+4. Include files for C11 and Zig, defining imports for the JS core API defined
+   by this package
 
 ### Custom API modules
 
+The WasmBridge is extensible via custom defined API modules. Such API extensions
+will consist of a collection of JS/TS functions & variables, their related
+counterparts (import definitions) for the WASM target and (optionally) some
+shared data types (bindings for which _can_ be generated by this package too).
+
 On the JS side, custom API modules can be easily integrated via the [`IWasmAPI`
 interface](https://docs.thi.ng/umbrella/wasm-api/interfaces/IWasmAPI.html). The
 following example provides a brief overview:
@@ -57,15 +75,15 @@ export class CustomAPI implements IWasmAPI {
 
 	/**
 	 * Returns object of functions to import as externals into
-	 * the WASM module. These imports are merged with the bridge's
-	 * core API and hence should use naming prefixes...
+	 * the WASM module. These imports are merged into a larger
+	 * imports object alongside the bridge's core API...
 	 */
 	getImports(): WebAssembly.Imports {
 		return {
 			/**
 			 * Writes 2 random float32 numbers to given address
 			 */
-			custom_randomVec2: (addr: number) => {
+			randomVec2: (addr: number) => {
 				this.parent.f32.set(
 					[Math.random(), Math.random()],
 					addr >> 2
@@ -83,14 +101,17 @@ export const bridge = new WasmBridge({ custom: new CustomAPI() });
 ```
 
 In Zig (or any other language of your choice) we can then utilize this custom
-API like so (Please also see example further below in this readme):
+API like so (Please also see /test/index.ts` & the example further below in this
+readme):
 
 ```zig
 // Import JS core API
 const js = @import("wasmapi");
 
 /// JS external to fill vec2 w/ random values
-extern fn custom_randomVec2(addr: usize) void;
+/// Note: Each API module uses a separate import object to avoid naming clashes
+/// Here we declare an external binding belonging to the "custom" import group
+extern "custom" fn randomVec2(addr: usize) void;
 
 export fn test_randomVec2() void {
 	var foo = [2]f32{ 0, 0 };
@@ -99,7 +120,7 @@ export fn test_randomVec2() void {
 	js.printF32Array(foo[0..]);
 
 	// populate foo with random numbers
-	custom_randomVec2(@ptrToInt(&foo));
+	randomVec2(@ptrToInt(&foo));
 
 	// print result
 	js.printF32Array(foo[0..]);
@@ -110,12 +131,12 @@ export fn test_randomVec2() void {
 
 Since only numeric values can be exchanged between the WASM module and the JS
 host, any JS native objects the WASM side might want to be working with must be
-managed in JS. For this purpose the [`ObjectIndex`
+managed manually in JS. For this purpose the [`ObjectIndex`
 class](https://docs.thi.ng/umbrella/wasm-api/classes/ObjectIndex.html) can be
 used by API modules to handle ID generation (incl. recycling, using
 [@thi.ng/idgen](https://github.com/thi-ng/umbrella/tree/develop/packages/idgen))
-& indexing of different types of JS objects/values. Only the numeric IDs will
-then need to be exchanged with the WASM module...
+and the indexing of different types of JS objects/values. Only the numeric IDs
+(handles) will then need to be exchanged with the WASM module...
 
 ```ts
 import { ObjectIndex } from "@thi.ng/wasm-api";
@@ -153,6 +174,29 @@ canvases.delete(0);
 // true
 ```
 
+### Data bindings & code generators
+
+The package provides an extensible codegeneration framework to simplify the
+bilateral design & exchange of data structures shared between the WASM & JS host
+env. Currently, code generators for TypeScript & Zig are supplied (more are
+planned). A CLI wrapper is worked on too.
+
+#### Data type definitions
+
+TODO
+
+##### Struct
+
+##### Enum
+
+#### Code generation
+
+TODO
+
+#### Example usage
+
+TODO
+
 ### Status
 
 **ALPHA** - bleeding edge / work-in-progress
@@ -182,11 +226,19 @@ node --experimental-repl-await
 > const wasmApi = await import("@thi.ng/wasm-api");
 ```
 
-Package sizes (gzipped, pre-treeshake): ESM: 1.63 KB
+Package sizes (gzipped, pre-treeshake): ESM: 3.98 KB
+
+**IMPORTANT:** The package includes various code generators and supporting
+functions which are NOT required during runtime. Hence the actual package size
+in production will be MUCH smaller!
 
 ## Dependencies
 
 - [@thi.ng/api](https://github.com/thi-ng/umbrella/tree/develop/packages/api)
+- [@thi.ng/binary](https://github.com/thi-ng/umbrella/tree/develop/packages/binary)
+- [@thi.ng/checks](https://github.com/thi-ng/umbrella/tree/develop/packages/checks)
+- [@thi.ng/compare](https://github.com/thi-ng/umbrella/tree/develop/packages/compare)
+- [@thi.ng/defmulti](https://github.com/thi-ng/umbrella/tree/develop/packages/defmulti)
 - [@thi.ng/errors](https://github.com/thi-ng/umbrella/tree/develop/packages/errors)
 - [@thi.ng/hex](https://github.com/thi-ng/umbrella/tree/develop/packages/hex)
 - [@thi.ng/idgen](https://github.com/thi-ng/umbrella/tree/develop/packages/idgen)
@@ -196,6 +248,8 @@ Package sizes (gzipped, pre-treeshake): ESM: 1.63 KB
 
 [Generated API docs](https://docs.thi.ng/umbrella/wasm-api/)
 
+## Basic usage example
+
 ```ts
 import { WasmBridge, WasmExports } from "@thi.ng/wasm-api";
 import { readFileSync } from "fs";
@@ -212,6 +266,7 @@ interface App extends WasmExports {
 
 	// instantiate WASM module using imports provided by the bridge
 	// this also initializes any bindings & bridge child APIs (if any)
+	// (also accepts a fetch() `Response` as input)
 	await bridge.instantiate(readFileSync("hello.wasm"));
 
 	// call an exported WASM function
@@ -219,12 +274,20 @@ interface App extends WasmExports {
 })();
 ```
 
+### Zig version
+
+Requires [Zig](https://ziglang.org) to be installed:
+
 ```zig
 //! Example Zig application (hello.zig)
 
 /// import externals
 /// see build command for configuration
 const js = @import("wasmapi");
+const std = @import("std");
+
+// set custom memory allocator (here to disable)
+pub const WASM_ALLOCATOR: ?std.mem.Allocator = null;
 
 export fn start() void {
 	js.printStr("hello world!");
@@ -238,7 +301,7 @@ folder):
 ```bash
 # compile WASM binary
 zig build-lib \
-	--pkg-begin wasmapi node_modules/@thi.ng/wasm-api/zig/core.zig --pkg-end \
+	--pkg-begin wasmapi node_modules/@thi.ng/wasm-api/include/wasmapi.zig --pkg-end \
 	-target wasm32-freestanding \
 	-O ReleaseSmall -dynamic --strip \
 	hello.zig
@@ -253,18 +316,112 @@ The resulting WASM:
 (module
  (type $i32_i32_=>_none (func (param i32 i32)))
  (type $none_=>_none (func))
- (import "env" "_printStr" (func $fimport$0 (param i32 i32)))
+ (type $i32_=>_i32 (func (param i32) (result i32)))
+ (import "wasmapi" "_printStr" (func $fimport$0 (param i32 i32)))
  (global $global$0 (mut i32) (i32.const 65536))
  (memory $0 2)
  (data (i32.const 65536) "hello world!\00")
  (export "memory" (memory $0))
  (export "start" (func $0))
+ (export "_wasm_allocate" (func $1))
+ (export "_wasm_free" (func $2))
  (func $0
   (call $fimport$0
    (i32.const 65536)
    (i32.const 12)
   )
  )
+ (func $1 (param $0 i32) (result i32)
+  (i32.const 0)
+ )
+ (func $2 (param $0 i32) (param $1 i32)
+ )
+)
+```
+
+### C11 version
+
+Requires [Emscripten](https://emscripten.org/) to be installed:
+
+```c
+#include <wasmapi.h>
+
+void WASM_KEEP start() {
+	wasm_printStr0("hello world!");
+}
+```
+
+Building the WASM module:
+
+```bash
+emcc -Os -Inode_modules/@thi.ng/wasm-api/include -DWASMAPI_NO_MALLOC \
+  -sERROR_ON_UNDEFINED_SYMBOLS=0 --no-entry \
+  -o hello.wasm hello.c
+```
+
+Resulting WASM:
+
+```wasm
+(module
+ (type $i32_=>_none (func (param i32)))
+ (type $none_=>_none (func))
+ (type $i32_=>_i32 (func (param i32) (result i32)))
+ (type $none_=>_i32 (func (result i32)))
+ (import "wasmapi" "_printStr0" (func $fimport$0 (param i32)))
+ (global $global$0 (mut i32) (i32.const 5243936))
+ (memory $0 256 256)
+ (data (i32.const 1024) "hello world!")
+ (table $0 2 2 funcref)
+ (elem (i32.const 1) $0)
+ (export "memory" (memory $0))
+ (export "_wasm_allocate" (func $1))
+ (export "_wasm_free" (func $2))
+ (export "start" (func $3))
+ (export "__indirect_function_table" (table $0))
+ (export "_initialize" (func $0))
+ (export "__errno_location" (func $7))
+ (export "stackSave" (func $4))
+ (export "stackRestore" (func $5))
+ (export "stackAlloc" (func $6))
+ (func $0
+  (nop)
+ )
+ (func $1 (param $0 i32) (result i32)
+  (i32.const 0)
+ )
+ (func $2 (param $0 i32)
+  (nop)
+ )
+ (func $3
+  (call $fimport$0
+   (i32.const 1024)
+  )
+ )
+ (func $4 (result i32)
+  (global.get $global$0)
+ )
+ (func $5 (param $0 i32)
+  (global.set $global$0
+   (local.get $0)
+  )
+ )
+ (func $6 (param $0 i32) (result i32)
+  (global.set $global$0
+   (local.tee $0
+    (i32.and
+     (i32.sub
+      (global.get $global$0)
+      (local.get $0)
+     )
+     (i32.const -16)
+    )
+   )
+  )
+  (local.get $0)
+ )
+ (func $7 (result i32)
+  (i32.const 1040)
+ )
 )
 ```
 

package/api.d.ts

@@ -1,5 +1,6 @@
-import type { Fn, Fn2 } from "@thi.ng/api";
+import type { FloatType, Fn, Fn2 } from "@thi.ng/api";
 import type { WasmBridge } from "./bridge.js";
+export declare const PKG_NAME = "@thi.ng/wasm-api";
 export declare type BigIntArray = bigint[] | BigInt64Array | BigUint64Array;
 /**
  * Common interface for WASM/JS child APIs which will be used in combination
@@ -27,7 +28,8 @@ export interface IWasmAPI<T extends WasmExports = WasmExports> {
 }
 /**
  * Base interface of exports declared by the WASM module. At the very least, the
- * module needs to export its memory.
+ * module needs to export its memory and the functions defined in this
+ * interface.
  *
  * @remarks
  * This interface is supposed to be extended with the concrete exports defined
@@ -39,16 +41,49 @@ export interface IWasmAPI<T extends WasmExports = WasmExports> {
 export interface WasmExports {
     /**
      * The WASM module's linear memory buffer. The `WasmBridge` automatically
-     * creates various typed views of that memory.
+     * creates various typed views of that memory (i.e. u8, u16, u32, f32 etc.)
      */
     memory: WebAssembly.Memory;
+    /**
+     * Implementation specific memory allocation function (likely heap-based).
+     * If successful returns address of new memory block, or zero if
+     * unsuccessful.
+     *
+     * @remarks
+     * In the supplied Zig bindings (see `/zig/core.zig`), by default this is
+     * using the `std.heap.GeneralPurposeAllocator` (which also automatically
+     * handles growing the WASM memory), however as mentioned the underlying
+     * mechanism is purposefully left to the actual WASM-side implementation. In
+     * a C program, this would likely use `malloc()` or similar...
+     */
+    _wasm_allocate(numBytes: number): number;
+    /**
+     * Implementation specific function to free a previously allocated chunk of
+     * of WASM memory (allocated via {@link WasmExports._wasm_allocate}).
+     *
+     * @param addr
+     * @param numBytes
+     */
+    _wasm_free(addr: number, numBytes: number): void;
+}
+export interface WasmMemViews {
+    i8: Int8Array;
+    u8: Uint8Array;
+    i16: Int16Array;
+    u16: Uint16Array;
+    i32: Int32Array;
+    u32: Uint32Array;
+    i64: BigInt64Array;
+    u64: BigUint64Array;
+    f32: Float32Array;
+    f64: Float64Array;
 }
 /**
  * Core API of WASM imports defined by the {@link WasmBridge}. The same
  * functions are declared as bindings in `/zig/core.zig`. Also see this file for
  * documentation of each function...
  */
-export interface CoreAPI {
+export interface CoreAPI extends WebAssembly.ModuleImports {
     printI8: Fn<number, void>;
     printU8: Fn<number, void>;
     printU8Hex: Fn<number, void>;
@@ -76,4 +111,131 @@ export interface CoreAPI {
     _printStr0: (addr: number) => void;
     _printStr: (addr: number, len: number) => void;
 }
+export interface WasmTypeBase {
+    /**
+     * Base address in linear WASM memory.
+     */
+    readonly __base: number;
+    /**
+     * Obtain as byte buffer
+     */
+    readonly __bytes: Uint8Array;
+}
+export interface WasmType<T> {
+    readonly align: number;
+    readonly size: number;
+    instance: Fn<number, T>;
+}
+export declare type WasmTypeConstructor<T> = Fn<WasmMemViews, WasmType<T>>;
+export declare type WasmInt = "i8" | "i16" | "i32" | "i64";
+export declare type WasmUint = "u8" | "u16" | "u32" | "u64";
+export declare type WasmFloat = FloatType;
+export declare type WasmPrim = WasmInt | WasmUint | WasmFloat;
+export declare type TypeColl = Record<string, TopLevelType>;
+export interface TypeInfo {
+    /**
+     * Auto-computed size (in bytes)
+     *
+     * @internal
+     */
+    __size?: number;
+    /**
+     * Auto-computed offset (in bytes) in parent struct
+     *
+     * @internal
+     */
+    __offset?: number;
+    /**
+     * Auto-computed alignment (in bytes)
+     *
+     * @internal
+     */
+    __align?: number;
+}
+export interface TopLevelType extends TypeInfo {
+    name: string;
+    doc?: string;
+    type: "struct" | "enum";
+}
+export interface Struct extends TopLevelType {
+    type: "struct";
+    fields: StructField[];
+    /**
+     * If true, struct fields will be re-ordered in descending order based on
+     * their {@link TypeInfo.__align} size.
+     *
+     * @defaultValue false
+     */
+    auto?: boolean;
+}
+export interface StructField extends TypeInfo {
+    /**
+     * Field name (prefix: "__" is reserved)
+     */
+    name: string;
+    /**
+     * Field docstring (can be multiline, will be formatted)
+     */
+    doc?: string;
+    /**
+     * Field type tag/qualifier (note: `slice` & `vec` are only supported by
+     * Zig & TS)
+     *
+     * @defaultValue "scalar"
+     */
+    tag?: "scalar" | "array" | "ptr" | "slice" | "vec";
+    /**
+     * Field base type. If not a {@link WasmPrim} or `opaque`, the value is
+     * interpreted as another type name in the {@link TypeColl}.
+     *
+     * TODO `opaque` currently unsupported.
+     */
+    type: WasmPrim | "opaque" | string;
+    /**
+     * TODO currently unsupported!
+     */
+    sentinel?: number;
+    /**
+     * Array or vector length
+     */
+    len?: number;
+    /**
+     * TODO currently unsupported!
+     */
+    default?: any;
+}
+export interface Enum extends TopLevelType {
+    type: "enum";
+    /**
+     * No u64 support, due to Typescript not supporting bigint enum values
+     */
+    tag: Exclude<WasmUint, "u64">;
+    values: (string | EnumValue)[];
+}
+export interface EnumValue {
+    name: string;
+    value?: number;
+    doc?: string;
+}
+export interface ICodeGen {
+    /**
+     * Optional prelude source, to be prepended before any generated type defs.
+     */
+    pre?: string;
+    /**
+     * Optional source code to be appended after any generated type defs.
+     */
+    post?: string;
+    doc: (doc: string, indent: string, acc: string[], topLevel?: boolean) => void;
+    enum: (type: Enum, types: TypeColl, acc: string[]) => void;
+    struct: (type: Struct, types: TypeColl, acc: string[]) => void;
+}
+/**
+ * WASM usize type. Assuming wasm32 until wasm64 surfaces, then need an option.
+ */
+export declare const USIZE = "u32";
+/**
+ * Byte size of {@link USIZE}.
+ */
+export declare const USIZE_SIZE = 4;
 //# sourceMappingURL=api.d.ts.map

package/api.js

@@ -1 +1,9 @@
-export {};
+export const PKG_NAME = "@thi.ng/wasm-api";
+/**
+ * WASM usize type. Assuming wasm32 until wasm64 surfaces, then need an option.
+ */
+export const USIZE = "u32";
+/**
+ * Byte size of {@link USIZE}.
+ */
+export const USIZE_SIZE = 4;