koffi 0.9.45 → 1.0.0

package/CMakeLists.txt

@@ -37,7 +37,7 @@ set(KOFFI_SRC
 if(CMAKE_SIZEOF_VOID_P EQUAL 8)
     if(WIN32)
         list(APPEND KOFFI_SRC src/call_x64_win_fwd.asm)
-    elseif(CMAKE_SYSTEM_PROCESSOR STREQUAL aarch64)
+    elseif(CMAKE_SYSTEM_PROCESSOR MATCHES "aarch64|arm64")
         list(APPEND KOFFI_SRC src/call_arm64_fwd.S)
     else()
         list(APPEND KOFFI_SRC src/call_x64_sysv_fwd.S)

package/README.md

@@ -1,145 +1,111 @@
 # Table of contents
 
 - [Introduction](#introduction)
+- [Get started](#get-started)
 - [Benchmarks](#benchmarks)
   * [atoi results](#atoi-results)
   * [Raylib results](#raylib-results)
-- [Installation](#installation)
+- [Tests](#tests)
+- [Compilation](#compilation)
   * [Windows](#windows)
   * [Other platforms](#other-platforms)
-- [Get started](#get-started)
-- [Tests](#tests)
 
 # Introduction
 
-Koffi is a fast and easy-to-use FFI module for Node.js, with support for complex data types such as structs.
-
-The following platforms __are officially supported and tested__ at the moment:
-
-* Windows x86 *(cdecl, stdcall, fastcall)*
-* Windows x86_64
-* Linux x86
-* Linux x86_64
-* Linux ARM32+VFP Little Endian
-* Linux ARM64 Little Endian
-* FreeBSD x86
-* FreeBSD x86_64
-* FreeBSD ARM64 Little Endian
-* macOS x86_64
+Koffi is a fast and easy-to-use FFI module for Node.js, with support for primitive and aggregate data types (structs), both by reference (pointer) and by value.
 
-The following platforms will __soon be officially supported__, when I can get my hand on a machine...:
+After the release of version 1.0, the following features are planned:
 
-* macOS ARM64
+* 1.1: C to JS callbacks
+* 1.2: Support fixed array types
 
-The following platforms __may be supported__ but are not tested:
+The following platforms __are officially supported and tested__ at the moment:
 
-* Linux ARM32 (full software FP) Little Endian
-* NetBSD x86_64
-* NetBSD ARM64
-* OpenBSD x86_64
-* OpenBSD ARM64
+Platoform | Architecture                     | JS to C | C to JS (callback) | Pre-built binary
+--------- | -------------------------------- | ------- | ------------------ | ----------------
+Windows   | x86 (cdecl, stdcall, fastcall)   | 🟩      | 🟥                 | Yes ✓
+Windows   | x86_64                           | 🟩      | 🟥                 | Yes ✓
+Linux     | x86                              | 🟩      | 🟥                 | Yes ✓
+Linux     | x86_64                           | 🟩      | 🟥                 | Yes ✓
+Linux     | ARM32+VFP Little Endian          | 🟩      | 🟥                 | Yes ✓
+Linux     | ARM64 Little Endian              | 🟩      | 🟥                 | Yes ✓
+FreeBSD   | x86                              | 🟩      | 🟥                 | Yes ✓
+FreeBSD   | x86_64                           | 🟩      | 🟥                 | Yes ✓
+FreeBSD   | ARM64 Little Endian              | 🟩      | 🟥                 | Yes ✓
+macOS     | x86_64                           | 🟩      | 🟥                 | Yes ✓
+macOS     | ARM64 (M1) Little Endian         | 🟩      | 🟥                 | No x
+NetBSD    | x86_64                           | 🟧      | 🟥                 | No x
+NetBSD    | ARM64 Little Endian              | 🟧      | 🟥                 | No x
+OpenBSD   | x86_64                           | 🟧      | 🟥                 | No x
+OpenBSD   | ARM64 Little Endian              | 🟧      | 🟥                 | No x
+
+🟩 Tested, fully operational
+🟧 May work, but not actively tested
+🟥 Does not work yet
 
 This is still in development, bugs are to expected. More tests will come in the near future.
 
-# Benchmarks
-
-In order to run it, go to `koffi/benchmark` and run `../../cnoke/cnoke.js` (or `node ..\..\cnoke\cnoke.js` on Windows) before doing anything else.
-
-Once this is done, you can execute each implementation, e.g. `build/atoi_cc` or `./atoi_koffi.js`. You can optionally define a custom number of iterations, e.g. `./atoi_koffi.js 10000000`.
-
-## atoi results
-
-This test is based around repeated calls to a simple standard C function atoi, and has three implementations:
-- the first one is the reference, it calls atoi through an N-API module, and is close to the theoretical limit of a perfect (no overhead) Node.js > C FFI implementation.
-- the second one calls atoi through Koffi
-- the third one uses the official Node.js FFI implementation, node-ffi-napi
-
-Because atoi is a small call, the FFI overhead is clearly visible.
-
-### Linux
-
-The results below were measured on my x86_64 Linux machine (AMD® Ryzen™ 7 5800H 16G):
-
-Benchmark     | Iterations | Total time  | Overhead
-------------- | ---------- | ----------- | ----------
-atoi_napi     | 20000000   | 1.10s       | (baseline)
-atoi_koffi    | 20000000   | 1.91s       | x1.73
-atoi_node_ffi | 20000000   | 640.49s     | x582
-
-### Windows
-
-The results below were measured on my x86_64 Windows machine (AMD® Ryzen™ 7 5800H 16G):
-
-Benchmark     | Iterations | Total time  | Overhead
-------------- | ---------- | ----------- | ----------
-atoi_napi     | 20000000   | 1.94s       | (baseline)
-atoi_koffi    | 20000000   | 3.15s       | x1.62
-atoi_node_ffi | 20000000   | 640.49s     | x242
-
-## Raylib results
-
-This benchmark uses the CPU-based image drawing functions in Raylib. The calls are much heavier than in the atoi benchmark, thus the FFI overhead is reduced. In this implemenetation, the baseline is a full C++ version of the code.
-
-### Linux
-
-The results below were measured on my x86_64 Linux machine (AMD® Ryzen™ 7 5800H 16G):
+# Get started
 
-Benchmark       | Iterations | Total time  | Overhead
---------------- | ---------- | ----------- | ----------
-raylib_cc       | 100        | 4.14s       | (baseline)
-raylib_koffi    | 100        | 6.25s       | x1.51
-raylib_node_ffi | 100        | 27.13s      | x6.55
+Once you have installed koffi with `npm install koffi`, you can start by loading it this way:
 
-### Windows
+```js
+const koffi = require('koffi');
+```
 
-The results below were measured on my x86_64 Windows machine (AMD® Ryzen™ 7 5800H 16G):
+Below you can find two examples:
 
-Benchmark       | Iterations | Total time  | Overhead
---------------- | ---------- | ----------- | ----------
-raylib_cc       | 100        | 8.39s       | (baseline)
-raylib_koffi    | 100        | 11.51s      | x1.37
-raylib_node_ffi | 100        | 31.47s      | x3.8
+* The first one runs on Linux. The functions are declared with the C-like prototype language.
+* The second one runs on Windows, and uses the node-ffi like syntax to declare functions.
+* The third one is more complex and uses Raylib to animate "Hello World" in a window.
 
-# Installation
+## Small Linux example
 
-## Windows
+```js
+const koffi = require('koffi');
+const lib = koffi.load('libc.so.6');
 
-First, make sure the following dependencies are met:
+// Declare types
+const timeval = koffi.struct('timeval', {
+    tv_sec: 'unsigned int',
+    tv_usec: 'unsigned int'
+});
+const timezone = koffi.struct('timezone', {
+    tz_minuteswest: 'int',
+    tz_dsttime: 'int'
+});
 
-* The "Desktop development with C++" workload from [Visual Studio 2022 or 2019](https://visualstudio.microsoft.com/downloads/) or the "C++ build tools" workload from the [Build Tools](https://visualstudio.microsoft.com/downloads/#build-tools-for-visual-studio-2022), with the default optional components.
-* [CMake meta build system](https://cmake.org/)
-* [Node 16 LTS](https://nodejs.org/), but a newer version should work too
+// Declare functions
+const gettimeofday = lib.func('int gettimeofday(_Out_ timeval *tv, _Out_ timezone *tz)');
+const printf = lib.func('int printf(const char *format, ...)');
 
-Once this is done, run this command from the project root:
+let tv = {};
+let tz = {};
+gettimeofday(tv, tz);
 
-```sh
-npm install koffi
+printf('Hello World!, it is: %d\n', 'int', tv.tv_sec);
+console.log(tz);
 ```
 
-## Other platforms
-
-Make sure the following dependencies are met:
+## Small Windows example
 
-* `gcc` and `g++` >= 8.3 or newer
-* GNU Make 3.81 or newer
-* [CMake meta build system](https://cmake.org/)
+```js
+const koffi = require('koffi');
+const lib = koffi.load('user32.dll');
 
-Once these dependencies are met, simply run the follow command:
+const MessageBoxA = lib.stdcall('MessageBoxA', 'int', ['void *', 'string', 'string', 'uint']);
+const MB_ICONINFORMATION = 0x40;
 
-```sh
-npm install koffi
+MessageBoxA(null, 'Hello', 'Foobar', MB_ICONINFORMATION);
 ```
 
-# Get started
-
-This section assumes you know how to build C shared libraries.
-
 ## Raylib example
 
-This examples illustrates how to use Koffi with a Raylib shared library:
+This section assumes you know how to build C shared libraries, such as Raylib. You may need to fix the URL to the library before you can do anything.
 
 ```js
 const koffi = require('koffi');
+let lib = koffi.load('raylib.dll'); // Fix path if needed
 
 const Color = koffi.struct('Color', {
     r: 'uchar',
@@ -193,9 +159,6 @@ const Font = koffi.struct('Font', {
     glyphs: koffi.pointer(GlyphInfo)
 });
 
-// Fix the path to Raylib DLL if needed
-let lib = koffi.load('build/raylib' + koffi.extension);
-
 // Classic function declaration
 const InitWindow = lib.func('InitWindow', 'void', ['int', 'int', 'string']);
 const SetTargetFPS = lib.func('SetTargetFPS', 'void', ['int']);
@@ -246,19 +209,84 @@ while (!WindowShouldClose()) {
 
 ```
 
-## Win32 stdcall example
+# Extra features
 
-```js
-const koffi = require('koffi');
+## Variadic functions
 
-let lib = koffi.load('user32.dll');
+Variadic functions are declared with an ellipsis as the last argument.
 
-const MessageBoxA = lib.stdcall('MessageBoxA', 'int', ['void *', 'string', 'string', 'uint']);
-const MB_ICONINFORMATION = 0x40;
+In order to call a variadic function, you must provide two Javascript arguments for each C parameter,
+the first one is the expected type and the second one is the value.
 
-MessageBoxA(null, 'Hello', 'Foobar', MB_ICONINFORMATION);
+```js
+const printf = lib.func('printf', 'int', ['string', '...']);
+
+printf('Integer %d, double %g, string %s', 'int', 6, 'double', 8.5, 'string', 'THE END');
 ```
 
+## Callbacks
+
+Koffi does not yet support passing JS functions as callbacks. This is planned for version 1.1.
+
+# Benchmarks
+
+In order to run it, go to `koffi/benchmark` and run `../../cnoke/cnoke.js` (or `node ..\..\cnoke\cnoke.js` on Windows) before doing anything else.
+
+Once this is done, you can execute each implementation, e.g. `build/atoi_cc` or `./atoi_koffi.js`. You can optionally define a custom number of iterations, e.g. `./atoi_koffi.js 10000000`.
+
+## atoi results
+
+This test is based around repeated calls to a simple standard C function atoi, and has three implementations:
+- the first one is the reference, it calls atoi through an N-API module, and is close to the theoretical limit of a perfect (no overhead) Node.js > C FFI implementation.
+- the second one calls atoi through Koffi
+- the third one uses the official Node.js FFI implementation, node-ffi-napi
+
+Because atoi is a small call, the FFI overhead is clearly visible.
+
+### Linux
+
+The results below were measured on my x86_64 Linux machine (AMD® Ryzen™ 7 5800H 16G):
+
+Benchmark     | Iterations | Total time  | Overhead
+------------- | ---------- | ----------- | ----------
+atoi_napi     | 20000000   | 1.10s       | (baseline)
+atoi_koffi    | 20000000   | 1.91s       | x1.73
+atoi_node_ffi | 20000000   | 640.49s     | x582
+
+### Windows
+
+The results below were measured on my x86_64 Windows machine (AMD® Ryzen™ 7 5800H 16G):
+
+Benchmark     | Iterations | Total time  | Overhead
+------------- | ---------- | ----------- | ----------
+atoi_napi     | 20000000   | 1.94s       | (baseline)
+atoi_koffi    | 20000000   | 3.15s       | x1.62
+atoi_node_ffi | 20000000   | 640.49s     | x242
+
+## Raylib results
+
+This benchmark uses the CPU-based image drawing functions in Raylib. The calls are much heavier than in the atoi benchmark, thus the FFI overhead is reduced. In this implemenetation, the baseline is a full C++ version of the code.
+
+### Linux
+
+The results below were measured on my x86_64 Linux machine (AMD® Ryzen™ 7 5800H 16G):
+
+Benchmark       | Iterations | Total time  | Overhead
+--------------- | ---------- | ----------- | ----------
+raylib_cc       | 100        | 4.14s       | (baseline)
+raylib_koffi    | 100        | 6.25s       | x1.51
+raylib_node_ffi | 100        | 27.13s      | x6.55
+
+### Windows
+
+The results below were measured on my x86_64 Windows machine (AMD® Ryzen™ 7 5800H 16G):
+
+Benchmark       | Iterations | Total time  | Overhead
+--------------- | ---------- | ----------- | ----------
+raylib_cc       | 100        | 8.39s       | (baseline)
+raylib_koffi    | 100        | 11.51s      | x1.37
+raylib_node_ffi | 100        | 31.47s      | x3.8
+
 # Tests
 
 Koffi is tested on multiple architectures using emulated (accelerated when possible) QEMU machines. First, you need to install qemu packages, such as `qemu-system` (or even `qemu-system-gui`) on Ubuntu.
@@ -314,4 +342,36 @@ Each machine is configured to run a VNC server available locally, which you can
 
 ```sh
 node qemu.js info debian_x64
-```
+```
+
+# Compilation
+
+We provide prebuilt binaries, packaged in the NPM archive, so in most cases it should be as simple as `npm install koffi`. If you want to hack Koffi or use a specific platform, follow the instructions below.
+
+## Windows
+
+First, make sure the following dependencies are met:
+
+* The "Desktop development with C++" workload from [Visual Studio 2022 or 2019](https://visualstudio.microsoft.com/downloads/) or the "C++ build tools" workload from the [Build Tools](https://visualstudio.microsoft.com/downloads/#build-tools-for-visual-studio-2022), with the default optional components.
+* [CMake meta build system](https://cmake.org/)
+* [Node 16 LTS](https://nodejs.org/), but a newer version should work too
+
+Once this is done, run this command from the project root:
+
+```sh
+npm install koffi
+```
+
+## Other platforms
+
+Make sure the following dependencies are met:
+
+* `gcc` and `g++` >= 8.3 or newer
+* GNU Make 3.81 or newer
+* [CMake meta build system](https://cmake.org/)
+
+Once these dependencies are met, simply run the follow command:
+
+```sh
+npm install koffi
+```

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "koffi",
-  "version": "0.9.45",
+  "version": "1.0.0",
   "description": "Fast and simple FFI (foreign function interface) for Node.js",
   "keywords": [
     "foreign",
@@ -18,7 +18,7 @@
   "author": "Niels Martignène <niels.martignene@protonmail.com>",
   "main": "src/index.js",
   "scripts": {
-    "install": "cnoke --prebuild qemu/build/{{version}}/koffi_{{platform}}_{{arch}}.tar.gz",
+    "install": "cnoke --prebuild build/qemu/{{version}}/koffi_{{platform}}_{{arch}}.tar.gz",
     "test": "node qemu/qemu.js",
     "prepublishOnly": "node qemu/qemu.js pack"
   },
@@ -48,6 +48,6 @@
     "LICENSE.txt",
     "README.md",
     "CMakeLists.txt",
-    "qemu/build/*/*.tar.gz"
+    "build/qemu/*/*.tar.gz"
   ]
 }

package/src/call_arm32.cc

@@ -39,7 +39,6 @@ extern "C" HfaRet ForwardCallXDDDD(const void *func, uint8_t *sp);
 
 static bool IsHFA(const TypeInfo *type)
 {
-#ifdef __ARM_PCS_VFP
     if (type->primitive != PrimitiveKind::Record)
         return false;
 
@@ -55,9 +54,6 @@ static bool IsHFA(const TypeInfo *type)
     }
 
     return true;
-#else
-    return false;
-#endif
 }
 
 bool AnalyseFunction(InstanceData *, FunctionInfo *func)
@@ -110,12 +106,7 @@ bool AnalyseFunction(InstanceData *, FunctionInfo *func)
             case PrimitiveKind::Float32:
             case PrimitiveKind::Float64: {
                 Size need = param.type->size / 4;
-
-#ifdef __ARM_PCS_VFP
                 bool vfp = !param.variadic;
-#else
-                bool vfp = false;
-#endif
 
                 if (vfp) {
                     if (need <= vec_avail) {

package/src/call_arm64.cc

@@ -88,6 +88,11 @@ bool AnalyseFunction(InstanceData *, FunctionInfo *func)
             case PrimitiveKind::UInt64:
             case PrimitiveKind::String:
             case PrimitiveKind::Pointer: {
+#ifdef __APPLE__
+                if (param.variadic)
+                    break;
+#endif
+
                 if (gpr_avail) {
                     param.gpr_count = 1;
                     gpr_avail--;
@@ -96,6 +101,11 @@ bool AnalyseFunction(InstanceData *, FunctionInfo *func)
 
             case PrimitiveKind::Float32:
             case PrimitiveKind::Float64: {
+#ifdef __APPLE__
+                if (param.variadic)
+                    break;
+#endif
+
                 if (vec_avail) {
                     param.vec_count = 1;
                     vec_avail--;
@@ -103,6 +113,13 @@ bool AnalyseFunction(InstanceData *, FunctionInfo *func)
             } break;
 
             case PrimitiveKind::Record: {
+#ifdef __APPLE__
+                if (param.variadic) {
+                    param.use_memory = (param.type->size > 16);
+                    break;
+                }
+#endif
+
                 if (IsHFA(param.type)) {
                     int vec_count = (int)param.type->members.len;