@thi.ng/wasm-api 0.5.0 → 0.6.0

package/CHANGELOG.md

@@ -1,6 +1,6 @@
 # Change Log
 
-- **Last updated**: 2022-08-08T22:36:17Z
+- **Last updated**: 2022-08-15T15:40:55Z
 - **Generator**: [thi.ng/monopub](https://thi.ng/monopub)
 
 All notable changes to this project will be documented in this file.
@@ -9,6 +9,40 @@ See [Conventional Commits](https://conventionalcommits.org/) for commit guidelin
 **Note:** Unlisted _patch_ versions only involve non-code or otherwise excluded changes
 and/or version bumps of transitive dependencies.
 
+## [0.6.0](https://github.com/thi-ng/umbrella/tree/@thi.ng/wasm-api@0.6.0) (2022-08-15)
+
+#### 🚀 Features
+
+- add C11 header/include file, update WasmBridge ([a67dc00](https://github.com/thi-ng/umbrella/commit/a67dc00))
+  - migrate headers/includes to /include
+  - rename "core" import section => "wasmapi"
+  - rename WasmBridge.core => WasmBridge.api
+  - update pkg file
+- update codegens, add opts, fix alignments ([5c1fec5](https://github.com/thi-ng/umbrella/commit/5c1fec5))
+  - add global CodeGenOpts
+  - update generateTypes() to consider new opts
+  - add global USIZE type (for pointer sizes & codegens)
+  - add options for Zig codegen (extra debug helpers)
+  - simplify TS codegen
+  - fix sizeOf() for struct fields
+  - make prepareType() idempotent
+- add bindings code generator framework ([17ee06f](https://github.com/thi-ng/umbrella/commit/17ee06f))
+  - add/update deps
+  - add preliminary codegens for Zig & TS
+  - add supporting types & utils
+  - add generateTypes() codegen facade fn
+- update allocate/free() fns, update Zig core API ([8a55989](https://github.com/thi-ng/umbrella/commit/8a55989))
+  - add _wasm_free() Zig impl
+  - add printFmt() Zig fn
+  - update WasmBridge.allocate() (add clear option)
+  - update WasmBridge.free()
+  - ensure memory in WasmBridge.getString()
+  - add/update docstrings
+
+#### ♻️ Refactoring
+
+- extract WasmMemViews interface, update test WASM ([4c73e65](https://github.com/thi-ng/umbrella/commit/4c73e65))
+
 ## [0.5.0](https://github.com/thi-ng/umbrella/tree/@thi.ng/wasm-api@0.5.0) (2022-08-08)
 
 #### 🚀 Features

package/README.md

@@ -12,30 +12,48 @@ This project is part of the
 - [About](#about)
   - [Custom API modules](#custom-api-modules)
   - [Object indices & handles](#object-indices--handles)
+  - [Data bindings & code generators](#data-bindings--code-generators)
+    - [Data type definitions](#data-type-definitions)
+    - [Code generation](#code-generation)
+    - [Example usage](#example-usage)
   - [Status](#status)
 - [Installation](#installation)
 - [Dependencies](#dependencies)
 - [API](#api)
+- [Basic usage example](#basic-usage-example)
+  - [Zig version](#zig-version)
+  - [C11 version](#c11-version)
 - [Authors](#authors)
 - [License](#license)
 
 ## About
 
-Modular, extensible API bridge and generic glue code between JS & WebAssembly.
+Generic, modular, extensible API bridge, glue code and bindings code generator for hybrid JS & WebAssembly projects.
 
-This package provides a small, generic and modular
-[`WasmBridge`](https://docs.thi.ng/umbrella/wasm-api/classes/WasmBridge.html)
-class as interop basis and a much reduced boilerplate for hybrid JS/WebAssembly
-applications. At the moment only a minimal core API is provided (i.e. for debug
-output, string, pointer, typed array accessors [8/16/32/64 bit (u)ints, 32/64
-bit floats]), but in the future we aim to also supply support modules for DOM
-manipulation, WebGL, WebGPU, WebAudio etc.
+This package provides a the following:
 
-In general, all languages with a WebAssembly target are supported, however
-currently only bindings for [Zig](https://ziglang.org) are included.
+1. A small, generic and modular
+[`WasmBridge`](https://docs.thi.ng/umbrella/wasm-api/classes/WasmBridge.html)
+class as interop basis and much reduced boilerplate for hybrid JS/WebAssembly
+applications.
+2. A minimal core API for debug output, string, pointer, typed array accessors
+(8/16/32/64 bit (u)ints, 32/64 bit floats). In the future we aim to also supply
+support modules for DOM manipulation, WebGL, WebGPU, WebAudio etc.
+3. Extensible shared datatype code generators for (currently)
+   [Zig](https://ziglang.org) & TypeScript. The latter also generates fully type
+   checked memory-mapped accessors of WASM-side data. In general, all languages
+with a WebAssembly target are supported, however currently only bindings for
+these few langs are included.
+4. Include files for C11 and Zig, defining imports for the JS core API defined
+   by this package
 
 ### Custom API modules
 
+The WasmBridge is extensible via custom defined API modules. Such API extensions
+will consist of a collection of JS/TS functions & variables, their related
+counterparts (import definitions) for the WASM target and (optionally) some
+shared data types (bindings for which _can_ be generated by this package too).
+
 On the JS side, custom API modules can be easily integrated via the [`IWasmAPI`
 interface](https://docs.thi.ng/umbrella/wasm-api/interfaces/IWasmAPI.html). The
 following example provides a brief overview:
@@ -113,12 +131,12 @@ export fn test_randomVec2() void {
 
 Since only numeric values can be exchanged between the WASM module and the JS
 host, any JS native objects the WASM side might want to be working with must be
-managed in JS. For this purpose the [`ObjectIndex`
+managed manually in JS. For this purpose the [`ObjectIndex`
 class](https://docs.thi.ng/umbrella/wasm-api/classes/ObjectIndex.html) can be
 used by API modules to handle ID generation (incl. recycling, using
 [@thi.ng/idgen](https://github.com/thi-ng/umbrella/tree/develop/packages/idgen))
-& indexing of different types of JS objects/values. Only the numeric IDs will
-then need to be exchanged with the WASM module...
+and the indexing of different types of JS objects/values. Only the numeric IDs
+(handles) will then need to be exchanged with the WASM module...
 
 ```ts
 import { ObjectIndex } from "@thi.ng/wasm-api";
@@ -156,6 +174,29 @@ canvases.delete(0);
 // true
 ```
 
+### Data bindings & code generators
+
+The package provides an extensible codegeneration framework to simplify the
+bilateral design & exchange of data structures shared between the WASM & JS host
+env. Currently, code generators for TypeScript & Zig are supplied (more are
+planned). A CLI wrapper is worked on too.
+
+#### Data type definitions
+
+TODO
+
+##### Struct
+
+##### Enum
+
+#### Code generation
+
+TODO
+
+#### Example usage
+
+TODO
+
 ### Status
 
 **ALPHA** - bleeding edge / work-in-progress
@@ -185,11 +226,19 @@ node --experimental-repl-await
 > const wasmApi = await import("@thi.ng/wasm-api");
 ```
 
-Package sizes (gzipped, pre-treeshake): ESM: 1.61 KB
+Package sizes (gzipped, pre-treeshake): ESM: 3.98 KB
+
+**IMPORTANT:** The package includes various code generators and supporting
+functions which are NOT required during runtime. Hence the actual package size
+in production will be MUCH smaller!
 
 ## Dependencies
 
 - [@thi.ng/api](https://github.com/thi-ng/umbrella/tree/develop/packages/api)
+- [@thi.ng/binary](https://github.com/thi-ng/umbrella/tree/develop/packages/binary)
+- [@thi.ng/checks](https://github.com/thi-ng/umbrella/tree/develop/packages/checks)
+- [@thi.ng/compare](https://github.com/thi-ng/umbrella/tree/develop/packages/compare)
+- [@thi.ng/defmulti](https://github.com/thi-ng/umbrella/tree/develop/packages/defmulti)
 - [@thi.ng/errors](https://github.com/thi-ng/umbrella/tree/develop/packages/errors)
 - [@thi.ng/hex](https://github.com/thi-ng/umbrella/tree/develop/packages/hex)
 - [@thi.ng/idgen](https://github.com/thi-ng/umbrella/tree/develop/packages/idgen)
@@ -199,6 +248,8 @@ Package sizes (gzipped, pre-treeshake): ESM: 1.61 KB
 
 [Generated API docs](https://docs.thi.ng/umbrella/wasm-api/)
 
+## Basic usage example
+
 ```ts
 import { WasmBridge, WasmExports } from "@thi.ng/wasm-api";
 import { readFileSync } from "fs";
@@ -223,12 +274,20 @@ interface App extends WasmExports {
 })();
 ```
 
+### Zig version
+
+Requires [Zig](https://ziglang.org) to be installed:
+
 ```zig
 //! Example Zig application (hello.zig)
 
 /// import externals
 /// see build command for configuration
 const js = @import("wasmapi");
+const std = @import("std");
+
+// set custom memory allocator (here to disable)
+pub const WASM_ALLOCATOR: ?std.mem.Allocator = null;
 
 export fn start() void {
 	js.printStr("hello world!");
@@ -242,7 +301,7 @@ folder):
 ```bash
 # compile WASM binary
 zig build-lib \
-	--pkg-begin wasmapi node_modules/@thi.ng/wasm-api/zig/core.zig --pkg-end \
+	--pkg-begin wasmapi node_modules/@thi.ng/wasm-api/include/wasmapi.zig --pkg-end \
 	-target wasm32-freestanding \
 	-O ReleaseSmall -dynamic --strip \
 	hello.zig
@@ -257,18 +316,112 @@ The resulting WASM:
 (module
  (type $i32_i32_=>_none (func (param i32 i32)))
  (type $none_=>_none (func))
- (import "core" "_printStr" (func $fimport$0 (param i32 i32)))
+ (type $i32_=>_i32 (func (param i32) (result i32)))
+ (import "wasmapi" "_printStr" (func $fimport$0 (param i32 i32)))
  (global $global$0 (mut i32) (i32.const 65536))
  (memory $0 2)
  (data (i32.const 65536) "hello world!\00")
  (export "memory" (memory $0))
  (export "start" (func $0))
+ (export "_wasm_allocate" (func $1))
+ (export "_wasm_free" (func $2))
  (func $0
   (call $fimport$0
    (i32.const 65536)
    (i32.const 12)
   )
  )
+ (func $1 (param $0 i32) (result i32)
+  (i32.const 0)
+ )
+ (func $2 (param $0 i32) (param $1 i32)
+ )
+)
+```
+
+### C11 version
+
+Requires [Emscripten](https://emscripten.org/) to be installed:
+
+```c
+#include <wasmapi.h>
+
+void WASM_KEEP start() {
+	wasm_printStr0("hello world!");
+}
+```
+
+Building the WASM module:
+
+```bash
+emcc -Os -Inode_modules/@thi.ng/wasm-api/include -DWASMAPI_NO_MALLOC \
+  -sERROR_ON_UNDEFINED_SYMBOLS=0 --no-entry \
+  -o hello.wasm hello.c
+```
+
+Resulting WASM:
+
+```wasm
+(module
+ (type $i32_=>_none (func (param i32)))
+ (type $none_=>_none (func))
+ (type $i32_=>_i32 (func (param i32) (result i32)))
+ (type $none_=>_i32 (func (result i32)))
+ (import "wasmapi" "_printStr0" (func $fimport$0 (param i32)))
+ (global $global$0 (mut i32) (i32.const 5243936))
+ (memory $0 256 256)
+ (data (i32.const 1024) "hello world!")
+ (table $0 2 2 funcref)
+ (elem (i32.const 1) $0)
+ (export "memory" (memory $0))
+ (export "_wasm_allocate" (func $1))
+ (export "_wasm_free" (func $2))
+ (export "start" (func $3))
+ (export "__indirect_function_table" (table $0))
+ (export "_initialize" (func $0))
+ (export "__errno_location" (func $7))
+ (export "stackSave" (func $4))
+ (export "stackRestore" (func $5))
+ (export "stackAlloc" (func $6))
+ (func $0
+  (nop)
+ )
+ (func $1 (param $0 i32) (result i32)
+  (i32.const 0)
+ )
+ (func $2 (param $0 i32)
+  (nop)
+ )
+ (func $3
+  (call $fimport$0
+   (i32.const 1024)
+  )
+ )
+ (func $4 (result i32)
+  (global.get $global$0)
+ )
+ (func $5 (param $0 i32)
+  (global.set $global$0
+   (local.get $0)
+  )
+ )
+ (func $6 (param $0 i32) (result i32)
+  (global.set $global$0
+   (local.tee $0
+    (i32.and
+     (i32.sub
+      (global.get $global$0)
+      (local.get $0)
+     )
+     (i32.const -16)
+    )
+   )
+  )
+  (local.get $0)
+ )
+ (func $7 (result i32)
+  (i32.const 1040)
+ )
 )
 ```
 

package/api.d.ts

@@ -1,5 +1,6 @@
-import type { Fn, Fn2 } from "@thi.ng/api";
+import type { FloatType, Fn, Fn2 } from "@thi.ng/api";
 import type { WasmBridge } from "./bridge.js";
+export declare const PKG_NAME = "@thi.ng/wasm-api";
 export declare type BigIntArray = bigint[] | BigInt64Array | BigUint64Array;
 /**
  * Common interface for WASM/JS child APIs which will be used in combination
@@ -60,12 +61,22 @@ export interface WasmExports {
      * Implementation specific function to free a previously allocated chunk of
      * of WASM memory (allocated via {@link WasmExports._wasm_allocate}).
      *
-     * @remarks
-     * In the supplied Zig bindings (/zig/core.zig) this is a no-op (currently).
-     *
      * @param addr
+     * @param numBytes
      */
-    _wasm_free(addr: number): void;
+    _wasm_free(addr: number, numBytes: number): void;
+}
+export interface WasmMemViews {
+    i8: Int8Array;
+    u8: Uint8Array;
+    i16: Int16Array;
+    u16: Uint16Array;
+    i32: Int32Array;
+    u32: Uint32Array;
+    i64: BigInt64Array;
+    u64: BigUint64Array;
+    f32: Float32Array;
+    f64: Float64Array;
 }
 /**
  * Core API of WASM imports defined by the {@link WasmBridge}. The same
@@ -100,4 +111,131 @@ export interface CoreAPI extends WebAssembly.ModuleImports {
     _printStr0: (addr: number) => void;
     _printStr: (addr: number, len: number) => void;
 }
+export interface WasmTypeBase {
+    /**
+     * Base address in linear WASM memory.
+     */
+    readonly __base: number;
+    /**
+     * Obtain as byte buffer
+     */
+    readonly __bytes: Uint8Array;
+}
+export interface WasmType<T> {
+    readonly align: number;
+    readonly size: number;
+    instance: Fn<number, T>;
+}
+export declare type WasmTypeConstructor<T> = Fn<WasmMemViews, WasmType<T>>;
+export declare type WasmInt = "i8" | "i16" | "i32" | "i64";
+export declare type WasmUint = "u8" | "u16" | "u32" | "u64";
+export declare type WasmFloat = FloatType;
+export declare type WasmPrim = WasmInt | WasmUint | WasmFloat;
+export declare type TypeColl = Record<string, TopLevelType>;
+export interface TypeInfo {
+    /**
+     * Auto-computed size (in bytes)
+     *
+     * @internal
+     */
+    __size?: number;
+    /**
+     * Auto-computed offset (in bytes) in parent struct
+     *
+     * @internal
+     */
+    __offset?: number;
+    /**
+     * Auto-computed alignment (in bytes)
+     *
+     * @internal
+     */
+    __align?: number;
+}
+export interface TopLevelType extends TypeInfo {
+    name: string;
+    doc?: string;
+    type: "struct" | "enum";
+}
+export interface Struct extends TopLevelType {
+    type: "struct";
+    fields: StructField[];
+    /**
+     * If true, struct fields will be re-ordered in descending order based on
+     * their {@link TypeInfo.__align} size.
+     *
+     * @defaultValue false
+     */
+    auto?: boolean;
+}
+export interface StructField extends TypeInfo {
+    /**
+     * Field name (prefix: "__" is reserved)
+     */
+    name: string;
+    /**
+     * Field docstring (can be multiline, will be formatted)
+     */
+    doc?: string;
+    /**
+     * Field type tag/qualifier (note: `slice` & `vec` are only supported by
+     * Zig & TS)
+     *
+     * @defaultValue "scalar"
+     */
+    tag?: "scalar" | "array" | "ptr" | "slice" | "vec";
+    /**
+     * Field base type. If not a {@link WasmPrim} or `opaque`, the value is
+     * interpreted as another type name in the {@link TypeColl}.
+     *
+     * TODO `opaque` currently unsupported.
+     */
+    type: WasmPrim | "opaque" | string;
+    /**
+     * TODO currently unsupported!
+     */
+    sentinel?: number;
+    /**
+     * Array or vector length
+     */
+    len?: number;
+    /**
+     * TODO currently unsupported!
+     */
+    default?: any;
+}
+export interface Enum extends TopLevelType {
+    type: "enum";
+    /**
+     * No u64 support, due to Typescript not supporting bigint enum values
+     */
+    tag: Exclude<WasmUint, "u64">;
+    values: (string | EnumValue)[];
+}
+export interface EnumValue {
+    name: string;
+    value?: number;
+    doc?: string;
+}
+export interface ICodeGen {
+    /**
+     * Optional prelude source, to be prepended before any generated type defs.
+     */
+    pre?: string;
+    /**
+     * Optional source code to be appended after any generated type defs.
+     */
+    post?: string;
+    doc: (doc: string, indent: string, acc: string[], topLevel?: boolean) => void;
+    enum: (type: Enum, types: TypeColl, acc: string[]) => void;
+    struct: (type: Struct, types: TypeColl, acc: string[]) => void;
+}
+/**
+ * WASM usize type. Assuming wasm32 until wasm64 surfaces, then need an option.
+ */
+export declare const USIZE = "u32";
+/**
+ * Byte size of {@link USIZE}.
+ */
+export declare const USIZE_SIZE = 4;
 //# sourceMappingURL=api.d.ts.map

package/api.js

@@ -1 +1,9 @@
-export {};
+export const PKG_NAME = "@thi.ng/wasm-api";
+/**
+ * WASM usize type. Assuming wasm32 until wasm64 surfaces, then need an option.
+ */
+export const USIZE = "u32";
+/**
+ * Byte size of {@link USIZE}.
+ */
+export const USIZE_SIZE = 4;

package/bridge.d.ts

@@ -1,7 +1,7 @@
 /// <reference types="node" />
 import type { NumericArray } from "@thi.ng/api";
 import type { ILogger } from "@thi.ng/logger";
-import type { BigIntArray, CoreAPI, IWasmAPI, WasmExports } from "./api.js";
+import type { BigIntArray, CoreAPI, IWasmAPI, WasmExports, WasmMemViews } from "./api.js";
 export declare const OutOfMemoryError: {
     new (msg?: string | undefined): {
         name: string;
@@ -28,7 +28,7 @@ export declare const OutOfMemoryError: {
  * 64bit integers are handled via JS `BigInt` and hence require the host env to
  * support it. No polyfill is provided.
  */
-export declare class WasmBridge<T extends WasmExports = WasmExports> {
+export declare class WasmBridge<T extends WasmExports = WasmExports> implements WasmMemViews {
     modules: Record<string, IWasmAPI<T>>;
     logger: ILogger;
     i8: Int8Array;
@@ -45,7 +45,7 @@ export declare class WasmBridge<T extends WasmExports = WasmExports> {
     utf8Encoder: TextEncoder;
     imports: WebAssembly.Imports;
     exports: T;
-    core: CoreAPI;
+    api: CoreAPI;
     constructor(modules?: Record<string, IWasmAPI<T>>, logger?: ILogger);
     /**
      * Instantiates WASM module from given `src` (and optional provided extra
@@ -95,7 +95,7 @@ export declare class WasmBridge<T extends WasmExports = WasmExports> {
      * bridge.getImports();
      * {
      *   // imports defined by the core API of the bridge itself
-     *   core: { ... },
+     *   wasmapi: { ... },
      *   // imports defined by the CustomAPI module
      *   custom: { ... }
      * }
@@ -121,15 +121,26 @@ export declare class WasmBridge<T extends WasmExports = WasmExports> {
      * Attempts to allocate `numBytes` using the exported WASM core API function
      * {@link WasmExports._wasm_allocate} (implementation specific) and returns
      * start address of the new memory block. If unsuccessful, throws an
-     * {@link OutOfMemoryError}.
+     * {@link OutOfMemoryError}. If `clear` is true, the allocated region will
+     * be zero-filled.
      *
      * @remarks
      * See {@link WasmExports._wasm_allocate} docs for further details.
      *
      * @param numBytes
+     * @param clear
      */
-    allocate(numBytes: number): number;
-    free(addr: number): void;
+    allocate(numBytes: number, clear?: boolean): number;
+    /**
+     * Frees a previous allocated memory region using the exported WASM core API
+     * function {@link WasmExports._wasm_free} (implementation specific). The
+     * `numBytes` value must be the same as previously given to
+     * {@link WasmBridge.allocate}.
+     *
+     * @param addr
+     * @param numBytes
+     */
+    free(addr: number, numBytes: number): void;
     getI8(addr: number): number;
     getU8(addr: number): number;
     getI16(addr: number): number;
@@ -170,7 +181,30 @@ export declare class WasmBridge<T extends WasmExports = WasmExports> {
     setU64Array(addr: number, buf: BigIntArray): this;
     setF32Array(addr: number, buf: NumericArray): this;
     setF64Array(addr: number, buf: NumericArray): this;
+    /**
+     * Reads UTF-8 encoded string from given address and optional byte length.
+     * The default length is 0, which will be interpreted as a zero-terminated
+     * string. Returns string.
+     *
+     * @param addr
+     * @param len
+     */
     getString(addr: number, len?: number): string;
+    /**
+     * Encodes given string as UTF-8 and writes it to WASM memory starting at
+     * `addr`. By default the string will be zero-terminated and only `maxBytes`
+     * will be written. Returns the number of bytes written.
+     *
+     * @remarks
+     * An error will be thrown if the encoded string doesn't fully fit into the
+     * designated memory region (also note that there might need to be space for
+     * the additional sentinel/termination byte).
+     *
+     * @param str
+     * @param addr
+     * @param maxBytes
+     * @param terminate
+     */
     setString(str: string, addr: number, maxBytes: number, terminate?: boolean): number;
     getElementById(addr: number, len?: number): HTMLElement;
 }