koffi 2.5.21-beta.4 → 2.6.1

package/CHANGELOG.md

@@ -2,6 +2,26 @@
 
 ## Version history
 
+### Koffi 2.6
+
+#### Koffi 2.6.1 (2023-09-18)
+
+- Support string direction qualifiers in classic function definitions
+- Fix possible off-by-one array access when parsing type name
+
+#### Koffi 2.6.0 (2023-09-13)
+
+**New features:**
+
+- Use [koffi.symbol()](variables.md#variable-definitions) to make pointers to exported variables (or other symbols)
+- Use [koffi.encode()](variables.md#encode-to-c-memory) to explictly encode data from JS to C memory
+- Use shared library [lazy-loading](functions.md#loading-options) (RTLD_LAZY) on POSIX platforms
+
+**Other changes:**
+
+- Print more helpful error for Win32/Win64 DLL mismatches
+- Add missing 'Union' primitive value in TS definition file
+
 ### Koffi 2.5
 
 #### Koffi 2.5.20 (2023-08-31)

package/doc/callbacks.md

@@ -153,14 +153,9 @@ let cb2 = koffi.register(store, store.get, 'IntCallback *'); // However in this
 
 Koffi does not have enough information to convert callback pointer arguments to an appropriate JS value. In this case, your JS function will receive an opaque *External* object.
 
-You can pass this value through to another C function that expects a pointer of the same type, or you can use `koffi.decode()` to decode it into something you can use in Javascript.
+You can pass this value through to another C function that expects a pointer of the same type, or you can use [koffi.decode()](variables.md#decode-to-js-values) function to decode pointer arguments.
 
-Some arguments are optional and this function can be called in several ways:
-
-- `koffi.decode(value, type)`: no offset, expect NUL-terminated strings
-- `koffi.decode(value, offset, type)`: explicit offset to add to the pointer before decoding
-
-The following example sorts an array of strings (in-place) with `qsort()`:
+The following examples uses it to sort an array of strings in-place with the standard C function `qsort()`:
 
 ```js
 // ES6 syntax: import koffi from 'koffi';
@@ -183,15 +178,6 @@ qsort(koffi.as(array, 'char **'), array.length, koffi.sizeof('void *'), (ptr1, p
 console.log(array); // Prints ['123', 'bar', 'foo', 'foobar']
 ```
 
-There is also an optional ending `length` argument that you can use in two cases:
-
-- Use it to give the number of bytes to decode in non-NUL terminated strings: `koffi.decode(value, 'char *', 5)`
-- Decode consecutive values into an array. For example, here is how you can decode an array with 3 float values: `koffi.decode(value, 'float', 3)`. This is equivalent to `koffi.decode(value, koffi.array('float', 3))`.
-
-```{note}
-In Koffi 2.2 and earlier versions, the length argument is only used to decode strings and is ignored otherwise.
-```
-
 ### Asynchronous callbacks
 
 *New in Koffi 2.2.2*

package/doc/functions.md

@@ -19,6 +19,22 @@ Starting with *Koffi 2.3.20*, you can explicitly unload a library by calling `li
 On some platforms (such as with the [musl C library on Linux](https://wiki.musl-libc.org/functional-differences-from-glibc.html#Unloading-libraries)), shared libraries cannot be unloaded, so the library will remain loaded and memory mapped after the call to `lib.unload()`.
 ```
 
+## Loading options
+
+*New in Koffi 2.6*
+
+The `load` function can take an optional object argument, with the following options:
+
+```js
+const options = {
+    lazy: true // Use RTLD_LAZY (lazy-binding) on POSIX platforms (by default, use RTLD_NOW)
+};
+
+const lib = koffi.load('/path/to/shared/library.so', options);
+```
+
+More options may be added if needed.
+
 ## Function definitions
 
 ### Definition syntax

package/doc/index.rst

@@ -31,6 +31,7 @@ Table of contents
    output
    polymorphism
    unions
+   variables
    callbacks
    misc
    packaging

package/doc/polymorphism.md

@@ -105,4 +105,4 @@ console.log(vec1); // { x: 3, y: 2, z: 1 }
 console.log(vec2); // { x: 1, y: 2, z: 3 }
 ```
 
-See [pointer arguments](callbacks.md#decoding-pointer-arguments) for another example with the decode function.
+See [decoding variables](variables.md#decode-to-js-values) for more information about the decode function.

package/doc/variables.md

@@ -0,0 +1,100 @@
+# Exported variables
+
+## Variable definitions
+
+*New in Koffi 2.6*
+
+To find an exported and  declare a variable, use `lib.symbol(name, type)`. You need to specify its name and its type.
+
+```c
+int my_int = 42;
+const char *my_string = NULL;
+```
+
+```js
+const my_int = lib.symbol('my_int', 'int');
+const my_string = lib.symbol('my_string', 'const char *');
+```
+
+You cannot directly manipulate these variables, use:
+
+- [koffi.decode()](#decode-to-js-values) to read their value
+- [koffi.encode()](#encode-to-c-memory) to change their valuèe
+
+## Decode to JS values
+
+*New in Koffi 2.2, changed in Koffi 2.3*
+
+Use `koffi.decode()` to decode C pointers, wrapped as external objects or as simple numbers.
+
+Some arguments are optional and this function can be called in several ways:
+
+- `koffi.decode(value, type)`: no offset
+- `koffi.decode(value, offset, type)`: explicit offset to add to the pointer before decoding
+
+By default, Koffi expects NUL terminated strings when decoding them. See below if you need to specify the string length.
+
+The following example illustrates how to decode an integer and a C string variable.
+
+```c
+int my_int = 42;
+const char *my_string = "foobar";
+```
+
+```js
+const my_int = lib.symbol('my_int', 'int');
+const my_string = lib.symbol('my_string', 'const char *');
+
+console.log(koffi.decode(my_int, 'int')) // Prints 42
+console.log(koffi.decode(my_string, 'const char *')) // Prints "foobar"
+```
+
+There is also an optional ending `length` argument that you can use in two cases:
+
+- Use it to give the number of bytes to decode in non-NUL terminated strings: `koffi.decode(value, 'char *', 5)`
+- Decode consecutive values into an array. For example, here is how you can decode an array with 3 float values: `koffi.decode(value, 'float', 3)`. This is equivalent to `koffi.decode(value, koffi.array('float', 3))`.
+
+Thge example below will decode the symbol `my_string` defined above but only the first three bytes.
+
+```js
+// Only decode 3 bytes from the C string my_string
+console.log(koffi.decode(my_string, 'const char *', 3)) // Prints "foo"
+```
+
+```{note}
+In Koffi 2.2 and earlier versions, the length argument is only used to decode strings and is ignored otherwise.
+```
+
+## Encode to C memory
+
+*New in Koffi 2.6*
+
+Use `koffi.encode()` to encode C pointers, wrapped as external objects or as simple numbers.
+
+Some arguments are optional and this function can be called in several ways:
+
+- `koffi.encode(ref, type, value)`: no offset
+- `koffi.encode(ref, offset, type, value)`: explicit offset to add to the pointer before encoding
+
+We'll reuse the examples shown above and change the variable values with `koffi.encode()`.
+
+```c
+int my_int = 42;
+const char *my_string = NULL;
+```
+
+```js
+const my_int = lib.symbol('my_int', 'int');
+const my_string = lib.symbol('my_string', 'const char *');
+
+console.log(koffi.decode(my_int, 'int')) // Prints 42
+console.log(koffi.decode(my_string, 'const char *')) // Prints null
+
+koffi.encode(my_int, 'int', -1);
+koffi.encode(my_string, 'const char *', 'Hello World!');
+
+console.log(koffi.decode(my_int, 'int')) // Prints -1
+console.log(koffi.decode(my_string, 'const char *')) // Prints "Hello World!"
+```
+
+When encoding strings (either directly or embedded in arrays or structs), the memory will be bound to the raw pointer value and managed by Koffi. You can assign to the same string again and again without any leak or risk of use-after-free.

package/package.json

@@ -1,7 +1,7 @@
 {
     "name": "koffi",
-    "version": "2.5.21-beta.4",
-    "stable": "2.5.20",
+    "version": "2.6.1",
+    "stable": "2.6.1",
     "description": "Fast and simple C FFI (foreign function interface) for Node.js",
     "keywords": [
         "foreign",

package/src/index.js

@@ -378,8 +378,8 @@ var require_package = __commonJS({
   "build/dist/src/koffi/package.json"(exports2, module2) {
     module2.exports = {
       name: "koffi",
-      version: "2.5.21-beta.4",
-      stable: "2.5.20",
+      version: "2.6.1",
+      stable: "2.6.1",
       description: "Fast and simple C FFI (foreign function interface) for Node.js",
       keywords: [
         "foreign",

package/src/koffi/src/parser.cc

@@ -35,7 +35,7 @@ bool PrototypeParser::Parse(const char *str, FunctionInfo *out_func)
 
     Tokenize(str);
 
-    out_func->ret.type = ParseType();
+    out_func->ret.type = ParseType(nullptr);
     if (!CanReturnType(out_func->ret.type)) {
         MarkError("You are not allowed to directly return %1 values (maybe try %1 *)", out_func->ret.type->name);
         return false;
@@ -62,17 +62,7 @@ bool PrototypeParser::Parse(const char *str, FunctionInfo *out_func)
                 break;
             }
 
-            if (Match("_In_")) {
-                param.directions = 1;
-            } else if (Match("_Out_")) {
-                param.directions = 2;
-            } else if (Match("_Inout_")) {
-                param.directions = 3;
-            } else {
-                param.directions = 1;
-            }
-
-            param.type = ParseType();
+            param.type = ParseType(&param.directions);
 
             if (!CanPassType(param.type, param.directions)) {
                 MarkError("Type %1 cannot be used as a parameter", param.type->name);
@@ -145,7 +135,7 @@ void PrototypeParser::Tokenize(const char *str)
     }
 }
 
-const TypeInfo *PrototypeParser::ParseType()
+const TypeInfo *PrototypeParser::ParseType(int *out_directions)
 {
     Size start = offset;
 
@@ -166,7 +156,7 @@ const TypeInfo *PrototypeParser::ParseType()
 
     while (offset >= start) {
         Span<const char> str = MakeSpan(tokens[start].ptr, tokens[offset].end() - tokens[start].ptr);
-        const TypeInfo *type = ResolveType(env, str);
+        const TypeInfo *type = ResolveType(env, str, out_directions);
 
         if (type) {
             offset++;

package/src/koffi/src/parser.hh

@@ -49,7 +49,7 @@ public:
 private:
     void Tokenize(const char *str);
 
-    const TypeInfo *ParseType();
+    const TypeInfo *ParseType(int *out_directions);
     const char *ParseIdentifier();
 
     bool Consume(const char *expect);

package/src/koffi/src/util.cc

@@ -153,6 +153,22 @@ static inline bool IsIdentifierChar(char c)
     return IsAsciiAlphaOrDigit(c) || c == '_';
 }
 
+static inline Span<const char> SplitIdentifier(Span<const char> str)
+{
+    Size offset = 0;
+
+    if (str.len && IsIdentifierStart(str[0])) {
+        offset++;
+
+        while (offset < str.len && IsIdentifierChar(str[offset])) {
+            offset++;
+        }
+    }
+
+    Span<const char> token = str.Take(0, offset);
+    return token;
+}
+
 const TypeInfo *ResolveType(Napi::Env env, Span<const char> str, int *out_directions)
 {
     InstanceData *instance = env.GetInstanceData<InstanceData>();
@@ -160,8 +176,31 @@ const TypeInfo *ResolveType(Napi::Env env, Span<const char> str, int *out_direct
     int indirect = 0;
     uint8_t disposables = 0;
 
+    // Consume parameter direction qualifier
+    if (out_directions) {
+        if (str.len && str[0] == '_') {
+            Span<const char> qualifier = SplitIdentifier(str);
+
+            if (qualifier == "_In_") {
+                *out_directions = 1;
+                str = str.Take(5, str.len - 5);
+            } else if (qualifier == "_Out_") {
+                *out_directions = 2;
+                str = str.Take(6, str.len - 6);
+            } else if (qualifier == "_Inout_") {
+                *out_directions = 3;
+                str = str.Take(8, str.len - 8);
+            } else {
+                *out_directions = 1;
+            }
+        } else {
+            *out_directions = 1;
+        }
+    }
+
     // Skip initial const qualifiers
-    while (str.len >= 6 && StartsWith(str, "const") && IsAsciiWhite(str[5])) {
+    str = TrimStr(str);
+    while (SplitIdentifier(str) == "const") {
         str = str.Take(6, str.len - 6);
         str = TrimStr(str);
     }
@@ -173,13 +212,10 @@ const TypeInfo *ResolveType(Napi::Env env, Span<const char> str, int *out_direct
     for (;;) {
         remain = TrimStr(remain);
 
-        if (!remain.len || !IsIdentifierStart(remain[0]))
+        Span<const char> token = SplitIdentifier(remain);
+        if (!token.len)
             break;
-
-        do {
-            remain.ptr++;
-            remain.len--;
-        } while (remain.len && IsIdentifierChar(remain[0]));
+        remain = remain.Take(token.len, remain.len - token.len);
     }
 
     Span<const char> name = TrimStr(MakeSpan(str.ptr, remain.ptr - str.ptr));
@@ -197,7 +233,7 @@ const TypeInfo *ResolveType(Napi::Env env, Span<const char> str, int *out_direct
         } else if (remain[0] == '!') {
             remain = remain.Take(1, remain.len - 1);
             disposables |= (1u << indirect);
-        } else if (remain.len >= 6 && StartsWith(remain, "const") && !IsIdentifierStart(remain[6])) {
+        } else if (SplitIdentifier(remain) == "const") {
             remain = remain.Take(6, remain.len - 6);
         } else {
             break;
@@ -263,9 +299,6 @@ const TypeInfo *ResolveType(Napi::Env env, Span<const char> str, int *out_direct
         RG_ASSERT(type);
     }
 
-    if (out_directions) {
-        *out_directions = 1;
-    }
     return type;
 }