koffi 2.9.2 → 2.10.0

package/CHANGELOG.md

@@ -5,6 +5,13 @@
 
 ## Koffi 2
 
+### Koffi 2.10
+
+#### Koffi 2.10.0 (2024-12-22)
+
+- Allow [redefinition of opaque types](misc#circular-references) to concrete struct or union
+- Update documentation style
+
 ### Koffi 2.9
 
 #### Koffi 2.9.2 (2024-11-08)
@@ -30,7 +37,7 @@
 - Work around MSVC compiler bug introduced in Visual Studio 17.10
 
 > [!WARNING]
-Use on platforms without pre-built binaries is broken in Koffi 2.8.10, skip this version.
+> Use on platforms without pre-built binaries is broken in Koffi 2.8.10, skip this version.
 
 #### Koffi 2.8.9 (2024-05-17)
 
@@ -700,9 +707,10 @@ This entry documents changes since version 1.1.0.
 
 The following features and improvements are planned, not necessarily in that order:
 
+- Port Koffi to Loong64 ISA and ABI
+- Port Koffi to PowerPC (POWER9+) ISA and ABI
 - Optimize passing of structs and arrays (with auto-generated JS)
 - Automate Windows/AArch64 (qemu) and macOS/AArch64 (how? ... thanks Apple) tests
 - Create a real-world example, using several libraries (Raylib, SQLite, libsodium) to illustrate various C API styles
 - Add simple struct type parser
-- Port Koffi to PowerPC (POWER9+) ABI
 - Fix assembly unwind and CFI directives for better debugging experience

package/doc/flaat/normal.css

@@ -20,8 +20,9 @@
    OTHER DEALINGS IN THE SOFTWARE. */
 
 html {
-    --top_height: 120px;
-    --small_height: 90px;
+    --top_height: 90px;
+    --top_padding: 6px;
+    --small_height: 80px;
 
     height: 100%;
     scroll-padding-top: calc(var(--top_height) + 10px);
@@ -68,7 +69,7 @@ a:has(> img) { text-decoration: none !important; }
     height: var(--top_height);
     box-sizing: border-box;
     margin: 0 auto;
-    padding: 16px 16px 24px 16px;
+    padding: var(--top_padding);
     overflow: visible;
     z-index: 1;
     display: flex;
@@ -132,7 +133,6 @@ a:has(> img) { text-decoration: none !important; }
     height: 100%;
     object-fit: contain;
 }
-#top.border #logo { filter: saturate(0%) brightness(0); }
 
 #side menu {
     margin: 0;
@@ -175,6 +175,16 @@ main {
 }
 #side ~ main { padding-right: 290px; }
 
+footer {
+    padding: 0.5em;
+    background: #f6f6f9;
+    display: flex;
+    gap: 1.5em;
+    align-items: center;
+    justify-content: center;
+}
+footer > img { filter: saturate(0%) brightness(0); }
+
 p { margin: 1em 0 0 0; }
 p:first-child, h1 + p, h2 + p, h3 + p { margin-top: 0; }
 
@@ -268,6 +278,11 @@ code:not(.hljs) {
     font-size: 14px;
     background: #eee;
 }
+pre > code:not(.hljs) {
+    padding: 0;
+    font-size: inherit;
+    background: transparent;
+}
 
 pre {
     position: relative;

package/doc/flaat/small.css

@@ -36,7 +36,6 @@
     #top menu {
         height: var(--small_height);
         padding-left: 60px;
-        padding-bottom: 16px;
         flex-direction: column;
         gap: 0;
         align-items: start;
@@ -61,6 +60,14 @@
         line-height: 1.6em;
     }
 
+    footer {
+        flex-direction: column;
+        padding: 0.5em;
+        gap: 6px;
+        text-align: center;
+    }
+    footer > img { display: none; }
+
     #logo {
         height: 100%;
         margin: 0 auto;

package/doc/pages/index.md

@@ -1,4 +1,4 @@
-# Overview
+# Koffi
 
 Koffi is a **fast and easy-to-use C FFI module for Node.js**, featuring:
 

package/doc/pages/misc.md

@@ -59,6 +59,41 @@ console.log(koffi.sizeof(koffi.types.long));
 
 You can alias a type with `koffi.alias(name, type)`. Aliased types are completely equivalent.
 
+## Circular references
+
+*New in Koffi 2.10.0*
+
+In some cases, composite types can point to each other and thus depend on each other. This can also happen when a function takes a pointer to a struct that also contains a function pointer.
+
+To deal with this, you can create an opaque type and redefine it later to a concrete struct or union type, as shown below.
+
+```js
+const Type1 = koffi.opaque('Type1');
+
+const Type2 = koffi.struct('Type2', {
+    ptr: 'Type1 *',
+    i: 'int'
+});
+
+// Redefine Type1 to a concrete type
+koffi.struct(Type1, {
+    ptr: 'Type2 *',
+    f: 'float'
+});
+```
+
+> [!NOTE]
+> You must use a proper type object when you redefine the type. If you only have the name, use `koffi.resolve()` to get a type object from a type string.
+>
+> ```js
+> const MyType = koffi.opaque('MyType');
+>
+> // This does not work, you must use the MyType object and not a type string
+> koffi.struct('MyType', {
+>      ptr: 'Type2 *',
+>      f: 'float'
+> });
+
 # Settings
 
 ## Memory usage

package/doc/pages/packaging.md

@@ -6,13 +6,16 @@
 
 Koffi uses native modules to work. The NPM package contains binaries for various platforms and architectures, and the appropriate module is selected at runtime.
 
+> [!IMPORTANT]
+> Please note that Koffi is meant for Node.js (or Electron) and not for browsers! It it not possible to load native libraries inside a browser!
+
 In theory, the **packagers/bundlers should be able to find all native modules** because they are explictly listed in the Javascript file (as static strings) and package them somehow.
 
 If that is not the case, you can manually arrange to copy the `node_modules/koffi/build/koffi` directory next to your bundled script.
 
 Here is an example that would work:
 
-```
+```text
 koffi/
     win32_x64/
         koffi.node
@@ -24,7 +27,7 @@ MyBundle.js
 
 When running in Electron, Koffi will also try to find the native module in `process.resourcesPath`. For an Electron app you could do something like this
 
-```
+```text
 locales/
 resources/
     koffi/
@@ -77,3 +80,9 @@ esbuild index.js --platform=node --bundle --loader:.node=copy --outdir=dist/
 ```
 
 You can find a full [working example in the repository](https://github.com/Koromix/rygel/tree/master/src/koffi/examples/node-esbuild).
+
+## Node.js and yao-pkg
+
+Use [yao-pkg](https://github.com/yao-pkg/pkg) to make binary packages of your Node.js-based project.
+
+You can find a full [working example in the repository](https://github.com/Koromix/rygel/tree/master/src/koffi/examples/yao-pkg).

package/doc/pages/platforms.md

@@ -14,16 +14,32 @@ Use [NVM](https://github.com/nvm-sh/nvm) to install more recent Node versions on
 
 The following combinations of OS and architectures __are officially supported and tested__ at the moment:
 
-ISA / OS           | Windows     | Linux    | macOS       | FreeBSD     | OpenBSD
------------------- | ----------- | -------- | ----------- | ----------- | --------
-x86 (IA32) [^1]    | ✅ Yes      | ✅ Yes   | ⬜️ *N/A*    | ✅ Yes      | ✅ Yes
-x86_64 (AMD64)     | ✅ Yes      | ✅ Yes   | ✅ Yes      | ✅ Yes      | ✅ Yes
-ARM32 LE [^2]      | ⬜️ *N/A*    | ✅ Yes   | ⬜️ *N/A*    | 🟨 Probably | 🟨 Probably
-ARM64 (AArch64) LE | ✅ Yes      | ✅ Yes   | ✅ Yes      | ✅ Yes      | 🟨 Probably
-RISC-V 64 [^3]     | ⬜️ *N/A*    | ✅ Yes   | ⬜️ *N/A*    | 🟨 Probably | 🟨 Probably
+ISA / OS           | Windows | Linux (glibc) | Linux (musl)
+------------------ | ------- | ------------- | ------------
+x86 (IA32) [^1]    | ✅      | ✅            | 🟨
+x86_64 (AMD64)     | ✅      | ✅            | ✅
+ARM32 LE [^2]      | ⬜️      | ✅            | 🟨
+ARM64 (AArch64) LE | ✅      | ✅            | 🟨
+RISC-V 64 [^3]     | ⬜️      | ✅            | 🟨
+
+<div class="legend">✅ Yes | 🟨 Probably | ⬜️ Not applicable</div>
+
+ISA / OS           | macOS | FreeBSD     | OpenBSD
+------------------ | ----- | ----------- | --------
+x86 (IA32) [^1]    | ⬜️    | ✅          | ✅
+x86_64 (AMD64)     | ✅    | ✅          | ✅
+ARM32 LE [^2]      | ⬜️    | 🟨          | 🟨
+ARM64 (AArch64) LE | ✅    | ✅          | 🟨
+RISC-V 64 [^3]     | ⬜️    | 🟨          | 🟨
+
+<div class="legend">✅ Yes | 🟨 Probably | ⬜️ Not applicable</div>
 
 [^1]: The following call conventions are supported for forward calls: cdecl, stdcall, MS fastcall, thiscall. Only cdecl and stdcall can be used for C to JS callbacks.
 [^2]: The prebuilt binary uses the hard float ABI and expects a VFP coprocessor. Build from source to use Koffi with a different ABI (softfp, soft).
 [^3]: The prebuilt binary uses the LP64D (double-precision float) ABI. The LP64 ABI is supported in theory if you build Koffi from source (untested), the LP64F ABI is not supported.
 
 For all fully supported platforms (green check marks), a prebuilt binary is included in the NPM package which means you can install Koffi without a C++ compiler.
+
+<style>
+     table td:not(:first-child) { text-align: center; }
+</style>

package/doc/pages.ini

@@ -2,6 +2,7 @@
 Title = Koffi
 Menu = Overview
 Description = Koffi presentation and features
+ToC = Off
 Template = templates/page.html
 
 [pages/platforms.md]
@@ -62,6 +63,12 @@ Title = Packaging | Koffi
 Menu = Documentation / Bundlers and Koffi
 Template = templates/page.html
 
+[pages/migration.md]
+Title = Migration | Koffi
+Menu = Documentation / Migration guide
+Description = Migration between major Koffi versions
+Template = templates/page.html
+
 [pages/benchmarks.md]
 Title = Benchmarks | Koffi
 Menu = Benchmarks
@@ -77,12 +84,6 @@ Template = templates/page.html
 [changelog]
 SourceFile = ../../src/koffi/CHANGELOG.md
 Title = Changelog | Koffi
-Menu = Changelog / Version history
+Menu = Changelog
 Description = List of Koffi versions
 Template = templates/page.html
-
-[pages/migration.md]
-Title = Migration | Koffi
-Menu = Changelog / Migration guide
-Description = Migration between major Koffi versions
-Template = templates/page.html

package/doc/templates/page.html

@@ -20,7 +20,7 @@
 
         <nav id="top">
             <menu>
-                <a id="logo" href="/"><img src="{{ ASSET favicon.png }}" alt="Logo Rekkord" /></a>
+                <a id="logo" href="/"><img src="{{ ASSET static/logo.webp }}" alt="Logo Koffi" /></a>
 
 {{ LINKS }}
 
@@ -36,5 +36,13 @@
         <main>
             {{ CONTENT }}
         </main>
+
+        <footer>
+            <div>Koffi © 2024</div>
+            <div style="font-size: 0.8em;">
+                Niels Martignène (<a href="https://github.com/Koromix/" target="_blank">Koromix</a>)<br>
+                <a href="mailto:niels.martignene@protonmail.com" style="font-weight: bold; color: inherit;">niels.martignene@protonmail.com</a>
+            </div>
+        </footer>
     </body>
 </html>

package/index.js

@@ -363,8 +363,8 @@ var require_package = __commonJS({
   "../../bin/Koffi/package/src/koffi/package.json"(exports2, module2) {
     module2.exports = {
       name: "koffi",
-      version: "2.9.2",
-      stable: "2.9.2",
+      version: "2.10.0",
+      stable: "2.10.0",
       description: "Fast and simple C FFI (foreign function interface) for Node.js",
       keywords: [
         "foreign",

package/indirect.js

@@ -363,8 +363,8 @@ var require_package = __commonJS({
   "../../bin/Koffi/package/src/koffi/package.json"(exports2, module2) {
     module2.exports = {
       name: "koffi",
-      version: "2.9.2",
-      stable: "2.9.2",
+      version: "2.10.0",
+      stable: "2.10.0",
       description: "Fast and simple C FFI (foreign function interface) for Node.js",
       keywords: [
         "foreign",

package/package.json

@@ -1,7 +1,7 @@
 {
     "name": "koffi",
-    "version": "2.9.2",
-    "stable": "2.9.2",
+    "version": "2.10.0",
+    "stable": "2.10.0",
     "description": "Fast and simple C FFI (foreign function interface) for Node.js",
     "keywords": [
         "foreign",

package/src/koffi/CMakeLists.txt

@@ -48,6 +48,12 @@ else()
     endif()
 endif()
 
+if(UNIX AND NOT APPLE)
+    set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -D_FILE_OFFSET_BITS=64")
+    set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -D_FILE_OFFSET_BITS=64")
+    set(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -z noexecstack")
+endif()
+
 # ---- Koffi ----
 
 # Recompute the version string after each commit

package/src/koffi/examples/yao-pkg/README.md

@@ -0,0 +1,17 @@
+This is a simple example to bundle a CLI node.js app that uses Koffi, using [yao-pkg](https://github.com/yao-pkg/pkg).
+
+To run the app, execute the following:
+
+```sh
+cd examples/yao-pkg
+npm install
+npm start
+```
+
+You can bundle the script and the native modules with the following command
+
+```sh
+cd examples/yao-pkg
+npm install
+npm run bundle
+```

package/src/koffi/examples/yao-pkg/index.js

@@ -0,0 +1,2 @@
+const koffi = require('koffi');
+console.log(koffi.config());

package/src/koffi/examples/yao-pkg/package.json

@@ -0,0 +1,22 @@
+{
+  "name": "KoffiConfig",
+  "version": "1.0.0",
+  "description": "",
+  "main": "index.js",
+  "scripts": {
+    "start": "node index.js",
+    "bundle": "pkg ."
+  },
+  "author": "",
+  "license": "ISC",
+  "dependencies": {
+    "koffi": "^2.5.18"
+  },
+  "devDependencies": {
+    "@yao-pkg/pkg": "^6.1.1"
+  },
+  "bin": "index.js",
+  "pkg": {
+    "outputPath": "dist"
+  }
+}

package/src/koffi/src/ffi.cc

@@ -244,8 +244,9 @@ static Napi::Value CreateStructType(const Napi::CallbackInfo &info, bool pad)
     }
 
     bool named = info.Length() > 1;
+    bool redefine = named && CheckValueTag(instance, info[0], &TypeInfoMarker);
 
-    if (named && !info[0].IsString()) {
+    if (named && !info[0].IsString() && !redefine) {
         ThrowError<Napi::TypeError>(env, "Unexpected %1 value for name, expected string", GetValueType(instance, info[0]));
         return env.Null();
     }
@@ -274,8 +275,20 @@ static Napi::Value CreateStructType(const Napi::CallbackInfo &info, bool pad)
     };
 
     TypeInfo *type = instance->types.AppendDefault();
+    TypeInfo *replace = nullptr;
 
-    if (named) {
+    if (redefine) {
+        Napi::External<TypeInfo> external = name.As<Napi::External<TypeInfo>>();
+        const TypeInfo *raw = external.Data();
+
+        replace = (TypeInfo *)AlignDown(raw, 4);
+        type->name = replace->name;
+
+        if (replace->primitive != PrimitiveKind::Void || replace == instance->void_type) {
+            ThrowError<Napi::TypeError>(env, "Cannot redefine non-opaque type %1", replace->name);
+            return env.Null();
+        }
+    } else if (named) {
         type->name = DuplicateString(name.Utf8Value().c_str(), &instance->str_alloc).ptr;
 
         if (!MapType(env, instance, type, type->name))
@@ -366,6 +379,11 @@ static Napi::Value CreateStructType(const Napi::CallbackInfo &info, bool pad)
     type->flags &= ~(int)TypeFlag::IsIncomplete;
     err_guard.Disable();
 
+    if (replace) {
+        std::swap(*type, *replace);
+        type = replace;
+    }
+
     return WrapType(env, instance, type);
 }
 
@@ -390,8 +408,9 @@ static Napi::Value CreateUnionType(const Napi::CallbackInfo &info)
     }
 
     bool named = info.Length() > 1;
+    bool redefine = named && CheckValueTag(instance, info[0], &TypeInfoMarker);
 
-    if (named && !info[0].IsString()) {
+    if (named && !info[0].IsString() && !redefine) {
         ThrowError<Napi::TypeError>(env, "Unexpected %1 value for name, expected string", GetValueType(instance, info[0]));
         return env.Null();
     }
@@ -420,8 +439,20 @@ static Napi::Value CreateUnionType(const Napi::CallbackInfo &info)
     };
 
     TypeInfo *type = instance->types.AppendDefault();
+    TypeInfo *replace = nullptr;
 
-    if (named) {
+    if (redefine) {
+        Napi::External<TypeInfo> external = name.As<Napi::External<TypeInfo>>();
+        const TypeInfo *raw = external.Data();
+
+        replace = (TypeInfo *)AlignDown(raw, 4);
+        type->name = replace->name;
+
+        if (replace->primitive != PrimitiveKind::Void || replace == instance->void_type) {
+            ThrowError<Napi::TypeError>(env, "Cannot redefine non-opaque type %1", replace->name);
+            return env.Null();
+        }
+    } else if (named) {
         type->name = DuplicateString(name.Utf8Value().c_str(), &instance->str_alloc).ptr;
 
         if (!MapType(env, instance, type, type->name))
@@ -507,6 +538,11 @@ static Napi::Value CreateUnionType(const Napi::CallbackInfo &info)
     Napi::Function constructor = MagicUnion::InitClass(env, type);
     type->construct.Reset(constructor, 1);
 
+    if (replace) {
+        std::swap(*type, *replace);
+        type = replace;
+    }
+
     return WrapType(env, instance, type);
 }
 

package/src/koffi/src/util.cc

@@ -146,8 +146,8 @@ const TypeInfo *ResolveType(Napi::Value value, int *out_directions)
         return type;
     } else if (CheckValueTag(instance, value, &TypeInfoMarker)) {
         Napi::External<TypeInfo> external = value.As<Napi::External<TypeInfo>>();
-
         const TypeInfo *raw = external.Data();
+
         const TypeInfo *type = AlignDown(raw, 4);
         RG_ASSERT(type);