unwasm 0.1.0 → 0.3.0

package/README.md

@@ -1,68 +1,173 @@
-# 🇼 unwasm
-
 [![npm version][npm-version-src]][npm-version-href]
 [![npm downloads][npm-downloads-src]][npm-downloads-href]
 [![Codecov][codecov-src]][codecov-href]
 
+# unwasm
+
 Universal [WebAssembly](https://webassembly.org/) tools for JavaScript.
 
 ## Goal
 
-This project aims to make a common and future-proof solution for WebAssembly modules support suitable for various JavaScript runtimes, Frameworks, and build Tools following [WebAssembly/ES Module Integration](https://github.com/WebAssembly/esm-integration/tree/main/proposals/esm-integration) proposal from WebAssembly Community Group as much as possible.
+This project aims to make a common and future-proof solution for WebAssembly modules support suitable for various JavaScript runtimes, frameworks, and build Tools following [WebAssembly/ES Module Integration](https://github.com/WebAssembly/esm-integration/tree/main/proposals/esm-integration) proposal from WebAssembly Community Group as much as possible while also trying to keep compatibility with current ecosystem libraries.
 
 ## Roadmap
 
 The development will be split into multiple stages.
 
 > [!IMPORTANT]
-> This Project is under development! Join the linked discussions to be involved!
+> This Project is under development! See the linked discussions to be involved!
 
-- [ ] Universal builder plugins ([unjs/unwasm#2](https://github.com/unjs/unwasm/issues/2)) built with [unjs/unplugin](https://github.com/unjs/unplugin) 
+- [ ] Builder plugin powered by [unjs/unplugin](https://github.com/unjs/unplugin) ([unjs/unwasm#2](https://github.com/unjs/unwasm/issues/2))
   - [x] Rollup
-- [ ] Tools to operate and inspect `.wasm` files ([unjs/unwasm#3](https://github.com/unjs/unwasm/issues/3)) 
-- [ ] Runtime utils ([unjs/unwasm#4](https://github.com/unjs/unwasm/issues/4)) 
-- [ ] ESM loader for Node.js and other JavaScript runtimes ([unjs/unwasm#5](https://github.com/unjs/unwasm/issues/5))
-- [ ] Integration with [Wasmer](https://github.com/wasmerio) ([unjs/unwasm#6](https://github.com/unjs/unwasm/issues/6)) 
-- [ ] Convention and tools for library authors exporting wasm modules ([unjs/unwasm#7](https://github.com/unjs/unwasm/issues/7))
+- [ ] Build Tools ([unjs/unwasm#3](https://github.com/unjs/unwasm/issues/3))
+  - [x] `parseWasm`
+- [ ] Runtime Utils ([unjs/unwasm#4](https://github.com/unjs/unwasm/issues/4))
+- [ ] ESM Loader ([unjs/unwasm#5](https://github.com/unjs/unwasm/issues/5))
+- [ ] Integration with [Wasmer](https://github.com/wasmerio) ([unjs/unwasm#6](https://github.com/unjs/unwasm/issues/6))
+- [ ] Convention for library authors exporting wasm modules ([unjs/unwasm#7](https://github.com/unjs/unwasm/issues/7))
+
+## Bindings API
+
+When importing a `.wasm` module, unwasm resolves, reads, and then parses the module during build process to get the information about imports and exports and generate appropriate code bindings.
+
+If the target environment supports [top level `await`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/await#top_level_await) and also the wasm module requires no imports object (auto-detected after parsing), unwasm generates bindings to allow importing wasm module like any other ESM import.
+
+If the target environment lacks support for top level `await` or the wasm module requires imports object or `lazy` plugin option is set to `true`, unwasm will export a wrapped [Proxy](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Proxy) object which can be called as a function to lazily evaluate the module with custom imports object. This way we still have a simple syntax as close as possible to ESM modules and also we can lazily initialize modules.
+
+**Example:** Using static import
+
+```js
+import { sum } from "unwasm/examples/sum.wasm";
+```
+
+**Example:** Using dynamic import
+
+```js
+const { sum } = await import("unwasm/examples/sum.wasm");
+```
+
+If your WebAssembly module requires an import object (which is likely!), the usage syntax would be slightly different as we need to initiate the module with an import object first.
+
+**Example:** Using dynamic import with imports object
+
+```js
+const { rand } = await import("unwasm/examples/rand.wasm").then((r) =>
+  r.default({
+    env: {
+      seed: () => () => Math.random() * Date.now(),
+    },
+  }),
+);
+```
 
-## Install
+**Example:** Using static import with imports object
 
-Install package from [npm](https://www.npmjs.com/package/unwasm):
+```js
+import initRand, { rand } from "unwasm/examples/rand.wasm";
+
+await initRand({
+  env: {
+    seed: () => () => Math.random() * Date.now(),
+  },
+});
+```
+
+> [!NOTE]
+> When using **static import syntax**, and before initializing the module, the named exports will be wrapped into a function by proxy that waits for the module initialization and if called before init, will immediately try to call init without imports and return a Promise that calls a function after init.
+
+## Integration
+
+Unwasm needs to transform the `.wasm` imports to the compatible bindings. Currently, the only method is using a rollup plugin. In the future, more usage methods will be introduced.
+
+### Install
+
+First, install the [`unwasm`](https://www.npmjs.com/package/unwasm) npm package.
 
 ```sh
 # npm
-npm install unwasm
+npm install --dev unwasm
 
 # yarn
-yarn add unwasm
+yarn add -D unwasm
 
 # pnpm
-pnpm install unwasm
+pnpm i -D unwasm
 
 # bun
-bun install unwasm
+bun i -D unwasm
 ```
 
-## Using build plugin
+### Builder Plugins
 
 ###### Rollup
 
 ```js
-import unwasmPlugin from "unwasm/plugin";
+// rollup.config.js
+import { rollup as unwasm } from "unwasm/plugin";
 
 export default {
   plugins: [
-    unwasmPlugin.rollup({
+    unwasm({
       /* options */
     }),
   ],
 };
 ```
 
-### Options
+### Plugin Options
+
+- `esmImport`: Direct import the wasm file instead of bundling, required in Cloudflare Workers and works with environments that allow natively importing a `.wasm` module (default is `false`)
+- `lazy`: Import `.wasm` files using a lazily evaluated proxy for compatibility with runtimes without top-level await support (default is `false`)
+
+## Tools
 
-- `esmImport`: Direct import the wasm file instead of bundling, required in Cloudflare Workers (default is `false`)
-- `lazy`: Import `.wasm` files using a lazily evaluated promise for compatibility with runtimes without top-level await support (default is `false`)
+unwasm provides useful build tools to operate on `.wasm` modules directly.
+
+**Note:** `unwasm/tools` subpath export is **not** meant or optimized for production runtime. Only rely on it for development and build time.
+
+### `parseWasm`
+
+Parses `wasm` binary format with useful information using [webassemblyjs/wasm-parser](https://github.com/xtuc/webassemblyjs/tree/master/packages/wasm-parser).
+
+```js
+import { readFile } from "node:fs/promises";
+import { parseWasm } from "unwasm/tools";
+
+const source = await readFile(new URL("./examples/sum.wasm", import.meta.url));
+const parsed = parseWasm(source);
+console.log(JSON.stringify(parsed, undefined, 2));
+```
+
+Example parsed result:
+
+```json
+{
+  "modules": [
+    {
+      "exports": [
+        {
+          "id": 5,
+          "name": "rand",
+          "type": "Func"
+        },
+        {
+          "id": 0,
+          "name": "memory",
+          "type": "Memory"
+        }
+      ],
+      "imports": [
+        {
+          "module": "env",
+          "name": "seed",
+          "params": [],
+          "returnType": "f64"
+        }
+      ]
+    }
+  ]
+}
+```
 
 ## Development
 
@@ -71,6 +176,7 @@ export default {
 - Enable [Corepack](https://github.com/nodejs/corepack) using `corepack enable`
 - Install dependencies using `pnpm install`
 - Run interactive tests using `pnpm dev`
+- Optionally install [es6-string-html](https://marketplace.visualstudio.com/items?itemName=Tobermory.es6-string-html) extension to make it easier to work with string templates.
 
 ## License
 

package/dist/plugin.d.mts

@@ -2,20 +2,24 @@ import { Plugin } from 'rollup';
 
 interface UnwasmPluginOptions {
     /**
-     * Direct import the wasm file instead of bundling, required in Cloudflare Workers
+     * Directly import the `.wasm` files instead of bundling as base64 string.
      *
      * @default false
      */
     esmImport?: boolean;
     /**
-     * Import `.wasm` files using a lazily evaluated promise for compatibility with runtimes without top-level await support
+     * Avoid using top level await and always use a proxy.
+     *
+     * Useful for compatibility with environments that don't support top level await.
      *
      * @default false
      */
     lazy?: boolean;
 }
+
+declare const rollup: (opts: UnwasmPluginOptions) => Plugin;
 declare const _default: {
     rollup: (opts: UnwasmPluginOptions) => Plugin<any>;
 };
 
-export { type UnwasmPluginOptions, _default as default };
+export { _default as default, rollup };

package/dist/plugin.d.ts

@@ -2,20 +2,24 @@ import { Plugin } from 'rollup';
 
 interface UnwasmPluginOptions {
     /**
-     * Direct import the wasm file instead of bundling, required in Cloudflare Workers
+     * Directly import the `.wasm` files instead of bundling as base64 string.
      *
      * @default false
      */
     esmImport?: boolean;
     /**
-     * Import `.wasm` files using a lazily evaluated promise for compatibility with runtimes without top-level await support
+     * Avoid using top level await and always use a proxy.
+     *
+     * Useful for compatibility with environments that don't support top level await.
      *
      * @default false
      */
     lazy?: boolean;
 }
+
+declare const rollup: (opts: UnwasmPluginOptions) => Plugin;
 declare const _default: {
     rollup: (opts: UnwasmPluginOptions) => Plugin<any>;
 };
 
-export { type UnwasmPluginOptions, _default as default };
+export { _default as default, rollup };

package/dist/plugin.mjs

@@ -3,19 +3,166 @@ import { basename } from 'pathe';
 import MagicString from 'magic-string';
 import { createUnplugin } from 'unplugin';
 import { createHash } from 'node:crypto';
+import { parseWasm } from './tools.mjs';
+import '@webassemblyjs/wasm-parser';
 
+const UNWASM_EXTERNAL_PREFIX = "\0unwasm:external:";
+const UMWASM_HELPERS_ID = "\0unwasm:helpers";
 function sha1(source) {
   return createHash("sha1").update(source).digest("hex").slice(0, 16);
 }
 
-const WASM_EXTERNAL_ID = "\0unwasm:external:";
+const js = String.raw;
+function getWasmBinding(asset, opts) {
+  const envCode = opts.esmImport ? js`
+async function _instantiate(imports) {
+  const _mod = await import("${UNWASM_EXTERNAL_PREFIX}${asset.id}").then(r => r.default || r);
+  return WebAssembly.instantiate(_mod, imports)
+}
+  ` : js`
+import { base64ToUint8Array } from "${UMWASM_HELPERS_ID}";
+
+function _instantiate(imports) {
+  const _data = base64ToUint8Array("${asset.source.toString("base64")}")
+  return WebAssembly.instantiate(_data, imports)
+}
+  `;
+  const canTopAwait = opts.lazy !== true && Object.keys(asset.imports).length === 0;
+  if (canTopAwait) {
+    return js`
+import { getExports } from "${UMWASM_HELPERS_ID}";
+${envCode}
+
+const $exports = getExports(await _instantiate());
+
+${asset.exports.map((name) => `export const ${name} = $exports.${name};`).join("\n")}
+
+export const $init = () => $exports;
+
+export default $exports;
+    `;
+  } else {
+    return js`
+import { createLazyWasmModule } from "${UMWASM_HELPERS_ID}";
+${envCode}
+
+const _mod = createLazyWasmModule(_instantiate);
+
+${asset.exports.map((name) => `export const ${name} = _mod.${name};`).join("\n")}
+
+export const $init = _mod.$init.bind(_mod);
+
+export default _mod;
+    `;
+  }
+}
+function getPluginUtils() {
+  return js`
+export function debug(...args) {
+  console.log('[wasm] [debug]', ...args);
+}
+
+export function getExports(input) {
+  return input?.instance?.exports || input?.exports || input;
+}
+
+export function base64ToUint8Array(str) {
+  const data = atob(str);
+  const size = data.length;
+  const bytes = new Uint8Array(size);
+  for (let i = 0; i < size; i++) {
+    bytes[i] = data.charCodeAt(i);
+  }
+  return bytes;
+}
+
+export function createLazyWasmModule(_instantiator) {
+  const _exports = Object.create(null);
+  let _loaded;
+  let _promise;
+
+  const init = (imports) => {
+    if (_loaded) {
+      return Promise.resolve(exportsProxy);
+    }
+    if (_promise) {
+      return _promise;
+    }
+    return _promise = _instantiator(imports)
+      .then(r => {
+        Object.assign(_exports, getExports(r));
+        _loaded = true;
+        _promise = undefined;
+        return exportsProxy;
+      })
+      .catch(error => {
+        _promise = undefined;
+        console.error('[wasm] [error]', error);
+        throw error;
+      });
+  }
+
+  const exportsProxy = new Proxy(_exports, {
+    get(_, prop) {
+      if (_loaded) {
+        return _exports[prop];
+      }
+      return (...args) => {
+        return _loaded
+          ? _exports[prop]?.(...args)
+          : init().then(() => _exports[prop]?.(...args));
+      };
+    },
+  });
+
+
+  const lazyProxy = new Proxy(() => {}, {
+    get(_, prop) {
+      return exportsProxy[prop];
+    },
+    apply(_, __, args) {
+      return init(args[0])
+    },
+  });
+
+  return lazyProxy;
+}
+  `;
+}
+
 const unplugin = createUnplugin((opts) => {
   const assets = /* @__PURE__ */ Object.create(null);
+  const _parseCache = /* @__PURE__ */ Object.create(null);
+  function parse(name, source) {
+    if (_parseCache[name]) {
+      return _parseCache[name];
+    }
+    const parsed = parseWasm(source);
+    const imports = /* @__PURE__ */ Object.create(null);
+    const exports = [];
+    for (const mod of parsed.modules) {
+      exports.push(...mod.exports.map((e) => e.name));
+      for (const imp of mod.imports) {
+        if (!imports[imp.module]) {
+          imports[imp.module] = [];
+        }
+        imports[imp.module].push(imp.name);
+      }
+    }
+    _parseCache[name] = {
+      imports,
+      exports
+    };
+    return _parseCache[name];
+  }
   return {
     name: "unwasm",
     rollup: {
       async resolveId(id, importer) {
-        if (id.startsWith(WASM_EXTERNAL_ID)) {
+        if (id === UMWASM_HELPERS_ID) {
+          return id;
+        }
+        if (id.startsWith(UNWASM_EXTERNAL_PREFIX)) {
           return {
             id,
             external: true
@@ -23,12 +170,11 @@ const unplugin = createUnplugin((opts) => {
         }
         if (id.endsWith(".wasm")) {
           const r = await this.resolve(id, importer, { skipSelf: true });
-          if (r?.id && r?.id !== id) {
+          if (r?.id && r.id !== id) {
             return {
               id: r.id.startsWith("file://") ? r.id.slice(7) : r.id,
               external: false,
-              moduleSideEffects: false,
-              syntheticNamedExports: false
+              moduleSideEffects: false
             };
           }
         }
@@ -46,13 +192,23 @@ const unplugin = createUnplugin((opts) => {
       }
     },
     async load(id) {
+      if (id === UMWASM_HELPERS_ID) {
+        return getPluginUtils();
+      }
       if (!id.endsWith(".wasm") || !existsSync(id)) {
         return;
       }
       const source = await promises.readFile(id);
       const name = `wasm/${basename(id, ".wasm")}-${sha1(source)}.wasm`;
-      assets[id] = { name, source };
-      return `export default "WASM";`;
+      const parsed = parse(name, source);
+      assets[id] = {
+        name,
+        id,
+        source,
+        imports: parsed.imports,
+        exports: parsed.exports
+      };
+      return `export default "UNWASM DUMMY EXPORT";`;
     },
     transform(_code, id) {
       if (!id.endsWith(".wasm")) {
@@ -62,25 +218,16 @@ const unplugin = createUnplugin((opts) => {
       if (!asset) {
         return;
       }
-      let _dataStr;
-      if (opts.esmImport) {
-        _dataStr = `await import("${WASM_EXTERNAL_ID}${id}").then(r => r?.default || r)`;
-      } else {
-        const base64Str = asset.source.toString("base64");
-        _dataStr = `(()=>{const d=atob("${base64Str}");const s=d.length;const b=new Uint8Array(s);for(let i=0;i<s;i++)b[i]=d.charCodeAt(i);return b})()`;
-      }
-      let _str = `await WebAssembly.instantiate(${_dataStr}, { env: { "Math.random": () => Math.random, "Math.floor": () => Math.floor } }).then(r => r?.exports||r?.instance?.exports || r);`;
-      if (opts.lazy) {
-        _str = `(()=>{const e=async()=>{return ${_str}};let _p;const p=()=>{if(!_p)_p=e();return _p;};return {then:cb=>p().then(cb),catch:cb=>p().catch(cb)}})()`;
-      }
       return {
-        code: `export default ${_str};`,
-        map: { mappings: "" },
-        syntheticNamedExports: true
+        code: getWasmBinding(asset, opts),
+        map: { mappings: "" }
       };
     },
     renderChunk(code, chunk) {
-      if (!chunk.moduleIds.some((id) => id.endsWith(".wasm")) || !code.includes(WASM_EXTERNAL_ID)) {
+      if (!opts.esmImport) {
+        return;
+      }
+      if (!(chunk.moduleIds.some((id) => id.endsWith(".wasm")) || chunk.imports.some((id) => id.endsWith(".wasm"))) || !code.includes(UNWASM_EXTERNAL_PREFIX)) {
         return;
       }
       const s = new MagicString(code);
@@ -99,7 +246,7 @@ const unplugin = createUnplugin((opts) => {
           asset
         };
       };
-      const ReplaceRE = new RegExp(`${WASM_EXTERNAL_ID}([^"']+)`, "g");
+      const ReplaceRE = new RegExp(`${UNWASM_EXTERNAL_PREFIX}([^"']+)`, "g");
       for (const match of code.matchAll(ReplaceRE)) {
         const resolved = resolveImport(match[1]);
         const index = match.index;
@@ -122,8 +269,8 @@ const unplugin = createUnplugin((opts) => {
   };
 });
 const rollup = unplugin.rollup;
-const plugin = {
+const index = {
   rollup
 };
 
-export { plugin as default };
+export { index as default, rollup };

package/dist/tools.d.mts

@@ -0,0 +1,25 @@
+type ParsedWasmModule = {
+    id?: string;
+    imports: ModuleImport[];
+    exports: ModuleExport[];
+};
+type ModuleImport = {
+    module: string;
+    name: string;
+    returnType?: string;
+    params?: {
+        id?: string;
+        type: string;
+    }[];
+};
+type ModuleExport = {
+    name: string;
+    id: string | number;
+    type: "Func" | "Memory";
+};
+type ParseResult = {
+    modules: ParsedWasmModule[];
+};
+declare function parseWasm(source: Buffer | ArrayBuffer): ParseResult;
+
+export { type ModuleExport, type ModuleImport, type ParseResult, type ParsedWasmModule, parseWasm };

package/dist/tools.d.ts

@@ -0,0 +1,25 @@
+type ParsedWasmModule = {
+    id?: string;
+    imports: ModuleImport[];
+    exports: ModuleExport[];
+};
+type ModuleImport = {
+    module: string;
+    name: string;
+    returnType?: string;
+    params?: {
+        id?: string;
+        type: string;
+    }[];
+};
+type ModuleExport = {
+    name: string;
+    id: string | number;
+    type: "Func" | "Memory";
+};
+type ParseResult = {
+    modules: ParsedWasmModule[];
+};
+declare function parseWasm(source: Buffer | ArrayBuffer): ParseResult;
+
+export { type ModuleExport, type ModuleImport, type ParseResult, type ParsedWasmModule, parseWasm };

package/dist/tools.mjs

@@ -0,0 +1,41 @@
+import { decode } from '@webassemblyjs/wasm-parser';
+
+function parseWasm(source) {
+  const ast = decode(source);
+  const modules = [];
+  for (const body of ast.body) {
+    if (body.type === "Module") {
+      const module = {
+        imports: [],
+        exports: []
+      };
+      modules.push(module);
+      for (const field of body.fields) {
+        if (field.type === "ModuleImport") {
+          module.imports.push({
+            module: field.module,
+            name: field.name,
+            returnType: field.descr?.signature?.results?.[0],
+            params: field.descr.signature.params?.map(
+              (p) => ({
+                id: p.id,
+                type: p.valtype
+              })
+            )
+          });
+        } else if (field.type === "ModuleExport") {
+          module.exports.push({
+            name: field.name,
+            id: field.descr.id.value,
+            type: field.descr.exportType
+          });
+        }
+      }
+    }
+  }
+  return {
+    modules
+  };
+}
+
+export { parseWasm };

package/package.json

@@ -1,22 +1,30 @@
 {
   "name": "unwasm",
-  "version": "0.1.0",
+  "version": "0.3.0",
   "description": "WebAssembly tools for JavaScript",
   "repository": "unjs/unwasm",
   "license": "MIT",
   "sideEffects": false,
   "type": "module",
   "exports": {
+    "./examples/*": "./examples/*",
     "./plugin": {
       "types": "./dist/plugin.d.mts",
       "import": "./dist/plugin.mjs"
+    },
+    "./tools": {
+      "types": "./dist/tools.d.mts",
+      "import": "./dist/tools.mjs"
     }
   },
   "files": [
-    "dist"
+    "dist",
+    "*.d.ts",
+    "examples/*.wasm"
   ],
   "scripts": {
-    "build": "unbuild",
+    "build": "unbuild && pnpm build:examples",
+    "build:examples": "node ./examples/build.mjs",
     "dev": "vitest dev",
     "lint": "eslint --cache --ext .ts,.js,.mjs,.cjs . && prettier -c src test",
     "lint:fix": "eslint --cache --ext .ts,.js,.mjs,.cjs . --fix && prettier -c src test -w",
@@ -26,6 +34,7 @@
     "test:types": "tsc --noEmit --skipLibCheck"
   },
   "dependencies": {
+    "@webassemblyjs/wasm-parser": "^1.11.6",
     "magic-string": "^0.30.5",
     "mlly": "^1.4.2",
     "pathe": "^1.1.1",
@@ -35,10 +44,12 @@
     "@rollup/plugin-node-resolve": "^15.2.3",
     "@types/node": "^20.10.5",
     "@vitest/coverage-v8": "^1.1.0",
+    "assemblyscript": "^0.27.22",
     "changelogen": "^0.5.5",
     "eslint": "^8.56.0",
     "eslint-config-unjs": "^0.2.1",
     "jiti": "^1.21.0",
+    "miniflare": "^3.20231030.4",
     "prettier": "^3.1.1",
     "rollup": "^4.9.1",
     "typescript": "^5.3.3",

package/plugin.d.ts

@@ -0,0 +1,3 @@
+export * from "./dist/plugin";
+
+export { default } from "./dist/plugin";