koffi 2.5.18 → 2.5.20

package/CHANGELOG.md

@@ -4,6 +4,16 @@
 
 ### Koffi 2.5
 
+#### Koffi 2.5.20 (2023-08-31)
+
+- Fix possible crash with async registered callbacks introduced in Koffi 2.5.19
+- Various documentation fixes and improvements
+
+#### Koffi 2.5.19 (2023-08-29)
+
+- Create thread-safe function broker lazily
+- Add [koffi.reset()](misc.md#reset-internal-state) for type names and async broker
+
 #### Koffi 2.5.18 (2023-08-27)
 
 - Fix compatibility with Electron
@@ -327,7 +337,7 @@ Pre-built binaries don't work correctly in Koffi 2.5.13 to 2.5.15, skip those ve
 **Main changes:**
 
 - Add [koffi.as()](polymorphism.md#input-polymorphism) to support polymorphic APIs based on `void *` parameters
-- Add [endian-sensitive integer types](input.md#endian-sensitive-types): `intX_le_t`, `intX_be_t`, `uintX_le_t`, `uintX_be_t`
+- Add [endian-sensitive integer types](input.md#endian-sensitive-integers): `intX_le_t`, `intX_be_t`, `uintX_le_t`, `uintX_be_t`
 - Accept typed arrays for `void *` parameters
 - Introduce `koffi.opaque()` to replace `koffi.handle()` (which remains supported until Koffi 3.0)
 - Support JS Array and TypedArray to fill struct and array pointer members

package/doc/callbacks.md

@@ -88,7 +88,7 @@ console.log(ret);
 
 Use registered callbacks when the function needs to be called at a later time (e.g. log handler, event handler, `fopencookie/funopen`). Call `koffi.register(func, type)` to register a callback function, with two arguments: the JS function, and the callback type.
 
-When you are done, call `koffi.unregister()` (with the value returned by `koffi.register()`) to release the slot. A maximum of 1024 callbacks can exist at the same time. Failure to do so will leak the slot, and subsequent registrations may fail (with an exception) once all slots are used.
+When you are done, call `koffi.unregister()` (with the value returned by `koffi.register()`) to release the slot. A maximum of 8192 callbacks can exist at the same time. Failure to do so will leak the slot, and subsequent registrations may fail (with an exception) once all slots are used.
 
 The example below shows how to register and unregister delayed callbacks.
 

package/doc/contribute.md

@@ -14,7 +14,7 @@ Start by cloning the repository with [Git](https://git-scm.com/):
 
 ```sh
 git clone https://github.com/Koromix/rygel
-cd rygel/src/koffi
+cd rygel
 ```
 
 As said before, this is a monorepository containg multiple projects, hence the name.
@@ -30,8 +30,8 @@ First, make sure the following dependencies are met:
 Once this is done, run this command _from the test or the benchmark directory_ (depending on what you want to build):
 
 ```sh
-cd koffi/test # or cd koffi/benchmark
-node ../../cnoke/cnoke.js
+cd src/koffi
+node ../cnoke/cnoke.js
 ```
 
 ### Other platforms
@@ -46,11 +46,31 @@ Make sure the following dependencies are met:
 Once this is done, run this command _from the test or the benchmark directory_ (depending on what you want to build):
 
 ```sh
-cd koffi/test # or cd koffi/benchmark
+cd src/koffi
+node ../cnoke/cnoke.js
+```
+
+## Run tests
+
+### On your machine
+
+Once Koffi is built, you can build the tests and run them with the following commands:
+
+```sh
+cd src/koffi/test
 node ../../cnoke/cnoke.js
+
+node sync.js # Run synchronous unit tests
+node async.js # Run asynchronous unit tests
+node callbacks.js # Run callback unit tests
+node union.js # Run union unit tests
+node win32.js # Run Windows-specific unit tests (only on Windows)
+
+node sqlite.js # Run SQLite integration tests
+node raylib.js # Run Raylib integration tests
 ```
 
-## Running tests
+### On virtual machines
 
 Koffi is tested on multiple architectures using emulated (accelerated when possible) QEMU machines. First, you need to install qemu packages, such as `qemu-system` (or even `qemu-system-gui`) on Ubuntu.
 
@@ -59,7 +79,7 @@ These machines are not included directly in this repository (for license and siz
 For example, if you want to run the tests on Debian ARM64, run the following commands:
 
 ```sh
-cd rygel/src/koffi/tools/
+cd src/koffi/tools/
 wget -q -O- https://koromix.dev/files/machines/qemu_debian_arm64.tar.zst | zstd -d | tar xv
 sha256sum -c --ignore-missing registry/sha256sum.txt
 ```

package/doc/input.md

@@ -56,26 +56,26 @@ let struct1 = koffi.struct({ dummy: 'long' });
 let struct2 = koffi.struct({ dummy: koffi.types.long });
 ```
 
-### Endian-sensitive types
+### Endian-sensitive integers
 
 *New in Koffi 2.1*
 
 Koffi defines a bunch of endian-sensitive types, which can be used when dealing with binary data (network payloads, binary file formats, etc.).
 
-JS type          | C type                 | Bytes | Signedness | Endianness
----------------- | ---------------------- | ----- | ---------- | -------------
-Number (integer) | int16_le, int16_le_t   | 2     | Signed     | Little Endian
-Number (integer) | int16_be, int16_be_t   | 2     | Signed     | Big Endian
-Number (integer) | uint16_le, uint16_le_t | 2     | Unsigned   | Little Endian
-Number (integer) | uint16_be, uint16_be_t | 2     | Unsigned   | Big Endian
-Number (integer) | int32_le, int32_le_t   | 4     | Signed     | Little Endian
-Number (integer) | int32_be, int32_be_t   | 4     | Signed     | Big Endian
-Number (integer) | uint32_le, uint32_le_t | 4     | Unsigned   | Little Endian
-Number (integer) | uint32_be, uint32_be_t | 4     | Unsigned   | Big Endian
-Number (integer) | int64_le, int64_le_t   | 8     | Signed     | Little Endian
-Number (integer) | int64_be, int64_be_t   | 8     | Signed     | Big Endian
-Number (integer) | uint64_le, uint64_le_t | 8     | Unsigned   | Little Endian
-Number (integer) | uint64_be, uint64_be_t | 8     | Unsigned   | Big Endian
+C type                 | Bytes | Signedness | Endianness
+---------------------- | ----- | ---------- | -------------
+int16_le, int16_le_t   | 2     | Signed     | Little Endian
+int16_be, int16_be_t   | 2     | Signed     | Big Endian
+uint16_le, uint16_le_t | 2     | Unsigned   | Little Endian
+uint16_be, uint16_be_t | 2     | Unsigned   | Big Endian
+int32_le, int32_le_t   | 4     | Signed     | Little Endian
+int32_be, int32_be_t   | 4     | Signed     | Big Endian
+uint32_le, uint32_le_t | 4     | Unsigned   | Little Endian
+uint32_be, uint32_be_t | 4     | Unsigned   | Big Endian
+int64_le, int64_le_t   | 8     | Signed     | Little Endian
+int64_be, int64_be_t   | 8     | Signed     | Big Endian
+uint64_le, uint64_le_t | 8     | Unsigned   | Little Endian
+uint64_be, uint64_be_t | 8     | Unsigned   | Big Endian
 
 ## Struct types
 

package/doc/misc.md

@@ -106,9 +106,9 @@ You can use `koffi.stats()` to get a few statistics related to Koffi.
 
 *New in Koffi 2.3.14*
 
-You can use `koffi.errno()` to the current errno value, and `koffi.errno(value)` to change it.
+You can use `koffi.errno()` to get the current errno value, and `koffi.errno(value)` to change it.
 
-The standard POSIX error codes are available in `koffi.os.errno`, as in the example below:
+The standard POSIX error codes are available in `koffi.os.errno`, as shown below:
 
 ```js
 const assert = require('assert');
@@ -125,3 +125,18 @@ assert.equal(koffi.errno(), koffi.os.errno.EBADF);
 
 console.log('close() with invalid FD is POSIX compliant!');
 ```
+
+## Reset internal state
+
+*New in Koffi 2.5.19*
+
+You can use `koffi.reset()` to clear some Koffi internal state such as:
+
+- Parser type names
+- Asynchronous function broker (useful to avoid false positive with `jest --detectOpenHandles`)
+
+This function is mainly intended for test code, when you execute the same code over and over and you need to reuse type names.
+
+```{warning}
+Trying to use a function or a type that was initially defined before the reset is undefined behavior and will likely lead to a crash!
+```

package/package.json

@@ -1,7 +1,7 @@
 {
     "name": "koffi",
-    "version": "2.5.18",
-    "stable": "2.5.18",
+    "version": "2.5.20",
+    "stable": "2.5.20",
     "description": "Fast and simple C FFI (foreign function interface) for Node.js",
     "keywords": [
         "foreign",

package/src/cnoke/package.json

@@ -1,6 +1,6 @@
 {
   "name": "cnoke",
-  "version": "4.0.1",
+  "version": "4.0.2",
   "description": "Build native Node addons based on CMake, without extra dependency",
   "keywords": [
     "native",

package/src/index.d.ts

@@ -145,6 +145,8 @@ declare module 'koffi' {
     export function errno(): number;
     export function errno(value: number): number;
 
+    export function reset(): void;
+
     export const internal: Boolean;
     export const extension: String;
 

package/src/index.js

@@ -378,8 +378,8 @@ var require_package = __commonJS({
   "build/dist/src/koffi/package.json"(exports2, module2) {
     module2.exports = {
       name: "koffi",
-      version: "2.5.18",
-      stable: "2.5.18",
+      version: "2.5.20",
+      stable: "2.5.20",
       description: "Fast and simple C FFI (foreign function interface) for Node.js",
       keywords: [
         "foreign",

package/src/koffi/CMakeLists.txt

@@ -47,6 +47,10 @@ endif()
 
 # ---- Koffi ----
 
+# Recompute the version string after each commit
+if(EXISTS "${CMAKE_SOURCE_DIR}/../../.git/logs/HEAD")
+    configure_file("${CMAKE_SOURCE_DIR}/../../.git/logs/HEAD" git_logs_HEAD COPYONLY)
+endif()
 if(EXISTS ${CMAKE_CURRENT_SOURCE_DIR}/package.json)
     file(READ ${CMAKE_CURRENT_SOURCE_DIR}/package.json PKG)
 else()

package/src/koffi/src/call.cc

@@ -1236,6 +1236,9 @@ void CallData::PopOutArguments()
 
 void *CallData::ReserveTrampoline(const FunctionInfo *proto, Napi::Function func)
 {
+    if (!InitAsyncBroker(env, instance)) [[unlikely]]
+        return nullptr;
+
     int16_t idx;
     {
         std::lock_guard<std::mutex> lock(shared.mutex);

package/src/koffi/src/ffi.cc

@@ -1656,6 +1656,9 @@ static Napi::Value RegisterCallback(const Napi::CallbackInfo &info)
     Napi::Env env = info.Env();
     InstanceData *instance = env.GetInstanceData<InstanceData>();
 
+    if (!InitAsyncBroker(env, instance)) [[unlikely]]
+        return env.Null();
+
     bool has_recv = (info.Length() >= 3 && info[1].IsFunction());
 
     if (info.Length() < 2u + has_recv) {
@@ -1937,6 +1940,22 @@ InstanceMemory::~InstanceMemory()
 #endif
 }
 
+bool InitAsyncBroker(Napi::Env env, InstanceData *instance)
+{
+    if (!instance->broker) {
+        if (napi_create_threadsafe_function(env, nullptr, nullptr,
+                                            Napi::String::New(env, "Koffi Async Callback Broker"),
+                                            0, 1, nullptr, nullptr, nullptr,
+                                            CallData::RelayAsync, &instance->broker) != napi_ok) {
+            LogError("Failed to create async callback broker");
+            return false;
+        }
+        napi_unref_threadsafe_function(env, instance->broker);
+    }
+
+    return true;
+}
+
 static void RegisterPrimitiveType(Napi::Env env, Napi::Object map, std::initializer_list<const char *> names,
                                   PrimitiveKind primitive, int32_t size, int16_t align, const char *ref = nullptr)
 {
@@ -2068,6 +2087,25 @@ static Napi::Object InitBaseTypes(Napi::Env env)
     return types;
 }
 
+static Napi::Value ResetKoffi(const Napi::CallbackInfo &info)
+{
+    Napi::Env env = info.Env();
+    InstanceData *instance = env.GetInstanceData<InstanceData>();
+
+    if (instance->broker) {
+        napi_release_threadsafe_function(instance->broker, napi_tsfn_abort);
+        instance->broker = nullptr;
+    }
+
+    instance->types.Clear();
+    instance->types_map.Clear();
+    instance->callbacks.Clear();
+
+    InitBaseTypes(env);
+
+    return env.Undefined();
+}
+
 static InstanceData *CreateInstance(Napi::Env env)
 {
     InstanceData *instance = new InstanceData();
@@ -2078,15 +2116,6 @@ static InstanceData *CreateInstance(Napi::Env env)
     instance->debug = GetDebugFlag("DUMP_CALLS");
     FillRandomSafe(&instance->tag_lower, RG_SIZE(instance->tag_lower));
 
-    if (napi_create_threadsafe_function(env, nullptr, nullptr,
-                                        Napi::String::New(env, "Koffi Async Callback Broker"),
-                                        0, 1, nullptr, nullptr, nullptr,
-                                        CallData::RelayAsync, &instance->broker) != napi_ok) {
-        LogError("Failed to create async callback broker");
-        return nullptr;
-    }
-    napi_unref_threadsafe_function(env, instance->broker);
-
 #ifdef _WIN32
     TEB *teb = GetTEB();
 
@@ -2141,6 +2170,8 @@ static Napi::Object InitModule(Napi::Env env, Napi::Object exports)
     exports.Set("address", Napi::Function::New(env, GetPointerAddress));
     exports.Set("call", Napi::Function::New(env, CallPointerSync));
 
+    exports.Set("reset", Napi::Function::New(env, ResetKoffi));
+
     exports.Set("errno", Napi::Function::New(env, GetOrSetErrNo));
 
     Napi::Object os = Napi::Object::New(env);

package/src/koffi/src/ffi.hh

@@ -341,4 +341,6 @@ Napi::Value TranslateNormalCall(const Napi::CallbackInfo &info);
 Napi::Value TranslateVariadicCall(const Napi::CallbackInfo &info);
 Napi::Value TranslateAsyncCall(const Napi::CallbackInfo &info);
 
+bool InitAsyncBroker(Napi::Env env, InstanceData *instance);
+
 }