koffi 3.0.1 → 3.1.0

package/doc/primitives.md

@@ -0,0 +1,151 @@
+# Number types
+
+While the C standard allows for variation in the size of most integer types, Koffi enforces the same definition for most primitive types, listed below:
+
+C type                        | JS type          | Bytes | Signedness | Note
+----------------------------- | ---------------- | ----- | ---------- | ---------------------------
+void                          | Undefined        | 0     |            | Only valid as a return type
+int8, int8_t                  | Number (integer) | 1     | Signed     |
+uint8, uint8_t                | Number (integer) | 1     | Unsigned   |
+char                          | Number (integer) | 1     | Signed     |
+uchar, unsigned char          | Number (integer) | 1     | Unsigned   |
+char16, char16_t              | Number (integer) | 2     | Signed     |
+int16, int16_t                | Number (integer) | 2     | Signed     |
+uint16, uint16_t              | Number (integer) | 2     | Unsigned   |
+short                         | Number (integer) | 2     | Signed     |
+ushort, unsigned short        | Number (integer) | 2     | Unsigned   |
+char32, char32_t              | Number (integer) | 4     | Signed     |
+int32, int32_t                | Number (integer) | 4     | Signed     |
+uint32, uint32_t              | Number (integer) | 4     | Unsigned   |
+int                           | Number (integer) | 4     | Signed     |
+uint, unsigned int            | Number (integer) | 4     | Unsigned   |
+int64, int64_t                | Number (integer) | 8     | Signed     |
+uint64, uint64_t              | Number (integer) | 8     | Unsigned   |
+longlong, long long           | Number (integer) | 8     | Signed     |
+ulonglong, unsigned long long | Number (integer) | 8     | Unsigned   |
+float32                       | Number (float)   | 4     |            |
+float64                       | Number (float)   | 8     |            |
+float                         | Number (float)   | 4     |            |
+double                        | Number (float)   | 8     |            |
+
+Koffi also accepts BigInt values when converting from JS to C integers. If the value exceeds the range of the C type, Koffi will convert the number to an undefined value. In the reverse direction, BigInt values are automatically used when needed for big 64-bit integers.
+
+Koffi defines a few more types that can change size depending on the OS and the architecture:
+
+C type           | JS type          | Signedness  | Note
+---------------- | ---------------- | ----------- | ------------------------------------------------
+bool             | Boolean          |             | Usually one byte
+long             | Number (integer) | Signed      | 4 or 8 bytes depending on platform (LP64, LLP64)
+ulong            | Number (integer) | Unsigned    | 4 or 8 bytes depending on platform (LP64, LLP64)
+unsigned long    | Number (integer) | Unsigned    | 4 or 8 bytes depending on platform (LP64, LLP64)
+intptr           | Number (integer) | Signed      | 4 or 8 bytes depending on register width
+intptr_t         | Number (integer) | Signed      | 4 or 8 bytes depending on register width
+uintptr          | Number (integer) | Unsigned    | 4 or 8 bytes depending on register width
+uintptr_t        | Number (integer) | Unsigned    | 4 or 8 bytes depending on register width
+wchar_t          | Number (integer) | | 2 bytes on Windows, 4 bytes Linux, macOS and BSD
+
+Primitive types can be specified by name (in a string) or through `koffi.types`:
+
+```js
+// These two lines do the same:
+let struct1 = koffi.struct({ dummy: 'long' });
+let struct2 = koffi.struct({ dummy: koffi.types.long });
+```
+
+# String types
+
+Koffi can convert JS strings to and from UTF-8, UTF-16 or UTF-32.
+
+Type name                      | Aliases         | JS type | Conversion / encoding
+------------------------------ | --------------- | ------- | ---------------------
+const char \*, char \*         | str, string     | String  | UTF-8
+const char16_t \*, char16_t \* | str16, string16 | String  | UTF-16
+const char32_t \*, char32_t \* | str32, string32 | String  | UTF-32
+
+Koffi also supports wide strings (`wchar_t *` or `wstring`), which use:
+
+- UTF-16 on Windows (where wchar_t is 2 bytes)
+- UTF-32 on other platforms (where wchar_t is 4 bytes)
+
+# Enum types
+
+*New in Koffi 3.0*
+
+C enumeration values are stored as integers. The underlying integer type is implementation-defined but Koffi tries to match usual platform behavior.
+
+On POSIX platforms, Koffi follows the following rules:
+
+- If no negative value exists: `unsigned int` by default, `uint64_t` if needed
+- If any negative value exists: `int` by default, `int64_t` if needed
+
+On Windows, things are simpler, and `int` is used all the time. Koffi will throw an error if any enumeration value does not fit in a 32-bit integer.
+
+```js
+// For OpenResult, the underlying type will be unsigned int on POSIX, int on Windows
+const OpenResult = koffi.enumeration('OpenResult', {
+    Success: 0,
+    MissingFile: 1,
+    AccessDenied: 2,
+    OtherError: 3
+});
+
+// For RelativePosition, the underlying type will be int everywhere
+const RelativePosition = koffi.enumeration('RelativePosition', {
+    Left: -1,
+    Center: 0,
+    Right: 1
+});
+
+// For IntLimits, the underlying type will be int64_t on POSIX, and fail on Windows
+const Int64Limits = koffi.enumeration('Int64Limits', {
+    Min: -9223372036854775808n,
+    Max: 9223372036854775807n
+});
+```
+
+> [!WARNING]
+> This behavior may not match your compiler:
+>
+> - On POSIX platforms, GCC and Clang will use a short integer type if `-fshort-enums` is specified and the enumeration values fit in `short` or `unsigned short`.
+> - On Windows, MSVC (and Clang) always use `int` even if some values do not fit, which matches what Koffi does... unless the compiler flag `/Zc:enumTypes` is set, maybe.
+>
+> Use an explicit type specifier to work around these problems, as shown below.
+
+You can access the constants in `values` member of the type object.
+
+```js
+console.log(OpenResult.values.MissingFile); // Prints 1
+console.log(RelativePosition.values.Left); // Prints -1
+```
+
+You can specify the storage type explicitly as the last argument: `koffi.enumeration(name, values, type)`.
+
+```js
+// This one explictly uses int64_t as the underlying type, despite the fact that the values fit inside an int.
+const ExplicitEnum = koffi.enumeration('ExplicitEnum', {
+    Zero: 0,
+    One: 1,
+    Two: 2
+}, 'int64_t');
+```
+
+# Endian-sensitive integers
+
+Koffi defines a bunch of endian-sensitive types, which can be used when dealing with binary data (network payloads, binary file formats, etc.).
+
+C type                 | Bytes | Signedness | Endianness
+---------------------- | ----- | ---------- | -------------
+int16_le, int16_le_t   | 2     | Signed     | Little Endian
+int16_be, int16_be_t   | 2     | Signed     | Big Endian
+uint16_le, uint16_le_t | 2     | Unsigned   | Little Endian
+uint16_be, uint16_be_t | 2     | Unsigned   | Big Endian
+int32_le, int32_le_t   | 4     | Signed     | Little Endian
+int32_be, int32_be_t   | 4     | Signed     | Big Endian
+uint32_le, uint32_le_t | 4     | Unsigned   | Little Endian
+uint32_be, uint32_be_t | 4     | Unsigned   | Big Endian
+int64_le, int64_le_t   | 8     | Signed     | Little Endian
+int64_be, int64_be_t   | 8     | Signed     | Big Endian
+uint64_le, uint64_le_t | 8     | Unsigned   | Little Endian
+uint64_be, uint64_be_t | 8     | Unsigned   | Big Endian
+
+You can find an example that uses these types to [extract information from a PNG header](pointers#handling-void-pointers) with transparent endian conversion.

package/doc/start.md

@@ -16,6 +16,9 @@ import koffi from 'koffi';
 const koffi = require('koffi');
 ```
 
+> [!NOTE]
+> Follow the [build instructions](contribute#build-from-source) if you want to build the native Koffi code yourself, to fix something or simply to study the code.
+
 # Simple examples
 
 Below you can find two examples:
@@ -105,16 +108,3 @@ let ret = MessageBoxA(null, 'Do you want another message box?', 'Koffi', MB_YESN
 if (ret == IDYES)
     MessageBoxW(null, 'Hello World!', 'Koffi', MB_ICONINFORMATION);
 ```
-
-# Bundling Koffi
-
-Please read the [dedicated page](packaging) for information about bundling and packaging applications using Koffi.
-
-# Build manually
-
-Follow the [build instructions](contribute#build-from-source) if you want to build the native Koffi code yourself.
-
-> [!NOTE]
-> This is only needed if you want to hack on Koffi. The official NPM package provides prebuilt binaries and you don't need to compile anything if you only want to use Koffi in Node.js.
->
-> Just run `npm install koffi`!

package/doc/types.md

@@ -0,0 +1,88 @@
+# Type specifiers
+
+*Changed in Koffi 3.0*
+
+> [!NOTE]
+> In Koffi 2.0, types were External values, you had to use `koffi.introspect()` to get type information. In Koffi 3.0, this information is directly available in type objects, and this function is deprecated.
+>
+> Consult the [migration guide](migration) for more information.
+
+You can use strings or type objects to give type information to Koffi (when declaring functions, structs, and so on). Use `koffi.type(spec)` to resolve all accepted type values (strings and type objects) to type objects.
+
+You can inspect the type object for information: name, primitive, size, alignment, members (record types), reference type (array, pointer), length (array), arguments and return type (prototypes).
+
+```js
+const FoobarType = koffi.struct('FoobarType', {
+    a: 'int',
+    b: 'char *',
+    c: 'double'
+});
+
+console.log(FoobarType);
+
+// Expected result on 64-bit machines:
+// {
+//     name: 'FoobarType',
+//     primitive: 'Record',
+//     size: 24,
+//     alignment: 8,
+//     disposable: false,
+//     members: {
+//         a: { name: 'a', type: [Type], offset: 0 },
+//         b: { name: 'b', type: [Type], offset: 8 },
+//         c: { name: 'c', type: [Type], offset: 16 }
+//     }
+// }
+```
+
+Koffi also exposes a few more utility functions to get a subset of this information:
+
+- `koffi.sizeof(type)` to get the size of a type
+- `koffi.alignof(type)` to get the alignment of a type
+- `koffi.offsetof(type, member_name)` to get the offset of a record member
+- `koffi.type(type)` to get the resolved type object from a type string
+
+Just like before, you can refer to primitive types by their name or through `koffi.types`:
+
+```js
+// These two lines do the same:
+console.log(koffi.sizeof('long'));
+console.log(koffi.sizeof(koffi.types.long));
+```
+
+# Aliases
+
+You can alias a type with `koffi.alias(name, type)`. Aliased types are completely equivalent.
+
+# Circular references
+
+In some cases, composite types can point to each other and thus depend on each other. This can also happen when a function takes a pointer to a struct that also contains a function pointer.
+
+To deal with this, you can create an opaque type and redefine it later to a concrete struct or union type, as shown below.
+
+```js
+const Type1 = koffi.opaque('Type1');
+
+const Type2 = koffi.struct('Type2', {
+    ptr: 'Type1 *',
+    i: 'int'
+});
+
+// Redefine Type1 to a concrete type
+koffi.struct(Type1, {
+    ptr: 'Type2 *',
+    f: 'float'
+});
+```
+
+> [!NOTE]
+> You must use a proper type object when you redefine the type. If you only have the name, use `koffi.type()` to get a type object from a type string.
+>
+> ```js
+> const MyType = koffi.opaque('MyType');
+>
+> // This does not work, you must use the MyType object and not a type string
+> koffi.struct('MyType', {
+>      ptr: 'Type2 *',
+>      f: 'float'
+> });

package/doc/unions.md

@@ -1,186 +0,0 @@
-# Union definition
-
-*New in Koffi 2.5*
-
-You can declare unions with a syntax similar to structs, but with the `koffi.union()` function. This function takes two arguments: the first one is the name of the type, and the second one is an object containing the union member names and types. You can omit the first argument to declare an anonymous union.
-
-The following example illustrates how to declare the same union in C and in JS with Koffi:
-
-```c
-typedef union IntOrDouble {
-    int64_t i;
-    double d;
-} IntOrDouble;
-```
-
-```js
-const IntOrDouble = koffi.union('IntOrDouble', {
-    i: 'int64_t',
-    d: 'double'
-});
-```
-
-# Input unions
-
-## Passing union values to C
-
-You can instantiate an union object with `koffi.Union(type)`. This will create a special object that contains at most one active member.
-
-Once you have created an instance of your union, you can simply set the member with the dot operator as you would with a basic object. Then, simply pass your union value to the C function you wish.
-
-```js
-const U = koffi.union('U', { i: 'int', str: 'char *' });
-
-const DoSomething = lib.func('void DoSomething(const char *type, U u)');
-
-const u1 = new koffi.Union('U'); u1.i = 42;
-const u2 = new koffi.Union('U'); u2.str = 'Hello!';
-
-DoSomething('int', u1);
-DoSomething('string', u2);
-```
-
-For simplicity, Koffi also accepts object literals with one property (no more, no less) setting the corresponding union member. The example belows uses this to simplify the code shown above:
-
-```js
-const U = koffi.union('U', { i: 'int', str: 'char *' });
-
-const DoSomething = lib.func('void DoSomething(const char *type, U u)');
-
-DoSomething('int', { i: 42 });
-DoSomething('string', { str: 'Hello!' });
-```
-
-## Win32 example
-
-The following example uses the [SendInput](https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-sendinput) Win32 API to emit the Win+D shortcut and hide windows (show the desktop).
-
-```js
-import koffi from 'koffi';
-// CJS: const koffi = require('koffi');
-
-// Win32 type and functions
-
-const user32 = koffi.load('user32.dll');
-
-const INPUT_MOUSE = 0;
-const INPUT_KEYBOARD = 1;
-const INPUT_HARDWARE = 2;
-
-const KEYEVENTF_KEYUP = 0x2;
-const KEYEVENTF_SCANCODE = 0x8;
-
-const VK_LWIN = 0x5B;
-const VK_D = 0x44;
-
-const MOUSEINPUT = koffi.struct('MOUSEINPUT', {
-    dx: 'long',
-    dy: 'long',
-    mouseData: 'uint32_t',
-    dwFlags: 'uint32_t',
-    time: 'uint32_t',
-    dwExtraInfo: 'uintptr_t'
-});
-const KEYBDINPUT = koffi.struct('KEYBDINPUT', {
-    wVk: 'uint16_t',
-    wScan: 'uint16_t',
-    dwFlags: 'uint32_t',
-    time: 'uint32_t',
-    dwExtraInfo: 'uintptr_t'
-});
-const HARDWAREINPUT = koffi.struct('HARDWAREINPUT', {
-    uMsg: 'uint32_t',
-    wParamL: 'uint16_t',
-    wParamH: 'uint16_t'
-});
-
-const INPUT = koffi.struct('INPUT', {
-    type: 'uint32_t',
-    u: koffi.union({
-        mi: MOUSEINPUT,
-        ki: KEYBDINPUT,
-        hi: HARDWAREINPUT
-    })
-});
-
-const SendInput = user32.func('unsigned int __stdcall SendInput(unsigned int cInputs, INPUT *pInputs, int cbSize)');
-
-// Show/hide desktop with Win+D shortcut
-
-let events = [
-    make_keyboard_event(VK_LWIN, true),
-    make_keyboard_event(VK_D, true),
-    make_keyboard_event(VK_D, false),
-    make_keyboard_event(VK_LWIN, false)
-];
-
-SendInput(events.length, events, koffi.sizeof(INPUT));
-
-// Utility
-
-function make_keyboard_event(vk, down) {
-    let event = {
-        type: INPUT_KEYBOARD,
-        u: {
-            ki: {
-                wVk: vk,
-                wScan: 0,
-                dwFlags: down ? 0 : KEYEVENTF_KEYUP,
-                time: 0,
-                dwExtraInfo: 0
-            }
-        }
-    };
-
-    return event;
-}
-```
-
-# Output unions
-
-Unlike structs, Koffi does not know which union member is valid, and it cannot decode it automatically. You can however use special `koffi.Union` objects for output parameters, and decode the memory after the call.
-
-To decode an output union pointer parameter, create a placeholder object with `new koffi.Union(type)` and pass the resulting object to the function.
-
-After the call, you can dereference the member value you want on this object and Koffi will decode it at this moment.
-
-The following example illustrates the use of `koffi.Union()` to decode output unions after the call.
-
-```c
-#include <stdint.h>
-
-typedef union IntOrDouble {
-    int64_t i;
-    double d;
-} IntOrDouble;
-
-void SetUnionInt(int64_t i, IntOrDouble *out)
-{
-    out->i = i;
-}
-
-void SetUnionDouble(double d, IntOrDouble *out)
-{
-    out->d = d;
-}
-```
-
-```js
-const IntOrDouble = koffi.union('IntOrDouble', {
-    i: 'int64_t',
-    d: 'double',
-    raw: koffi.array('uint8_t', 8)
-});
-
-const SetUnionInt = lib.func('void SetUnionInt(int64_t i, _Out_ IntOrDouble *out)');
-const SetUnionDouble = lib.func('void SetUnionDouble(double d, _Out_ IntOrDouble *out)');
-
-let u1 = new koffi.Union('IntOrDouble');
-let u2 = new koffi.Union('IntOrDouble');
-
-SetUnionInt(123, u1);
-SetUnionDouble(123, u2);
-
-console.log(u1.i, '---', u1.raw); // Prints 123 --- Uint8Array(8) [123, 0, 0, 0, 0, 0, 0, 0]
-console.log(u2.d, '---', u2.raw); // Prints 123 --- Uint8Array(8) [0, 0, 0, 0, 0, 0, 69, 64]
-```

package/doc/values.md

@@ -0,0 +1,134 @@
+# Decode to JS values
+
+## Decode numbers
+
+*New in Koffi 3.1.0*
+
+Use these functions to decode numeric values from a pointer. These functions match the [number types](primitives#number-types) of Koffi:
+
+Type      | Function                     | Result
+--------- | ---------------------------- | ------------------------------------
+char      | `koffi.decode.char(ptr)`     | Number
+uchar     | `koffi.decode.uchar(ptr)`    | Number
+short     | `koffi.decode.short(ptr)`    | Number
+ushort    | `koffi.decode.ushort(ptr)`   | Number
+int       | `koffi.decode.int(ptr)`      | Number
+uint      | `koffi.decode.uint(ptr)`     | Number
+long      | `koffi.decode.long(ptr)`     | Number or BigInt (on LP64 platforms)
+ulong     | `koffi.decode.ulong(ptr)`    | Number or BigInt (on LP64 platforms)
+longlong  | `koffi.decode.longlong(ptr)` | Number or BigInt
+ulonglong | `koffi.decode.ulonglong(ptr)`| Number or BigInt
+int8      | `koffi.decode.int8(ptr)`     | Number
+uint8     | `koffi.decode.uint8(ptr)`    | Number
+int16     | `koffi.decode.int16(ptr)`    | Number
+uint16    | `koffi.decode.uint16(ptr)`   | Number
+int32     | `koffi.decode.int32(ptr)`    | Number
+uint32    | `koffi.decode.uint32(ptr)`   | Number
+int64     | `koffi.decode.int64(ptr)`    | Number or BigInt
+uint64    | `koffi.decode.uint64(ptr)`   | Number or BigInt
+float     | `koffi.decode.float(ptr)`    | Number
+double    | `koffi.decode.double(ptr)`   | Number
+
+You can also decode [endian-sensitive integers](primitives#endian-sensitive-integers) with these functions:
+
+Type      | Function                     | Endianness    | Result
+--------- | ---------------------------- | ------------- | ----------------
+int16_le  | `koffi.decode.int16le(ptr)`  | Little Endian | Number
+uint16_le | `koffi.decode.uint16le(ptr)` | Little Endian | Number
+int32_le  | `koffi.decode.int32le(ptr)`  | Little Endian | Number
+uint32_le | `koffi.decode.uint32le(ptr)` | Little Endian | Number
+int64_le  | `koffi.decode.int64le(ptr)`  | Little Endian | Number or BigInt
+uint64_le | `koffi.decode.uint64le(ptr)` | Little Endian | Number or BigInt
+int16_be  | `koffi.decode.int16be(ptr)`  | Big Endian    | Number
+uint16_be | `koffi.decode.uint16be(ptr)` | Big Endian    | Number
+int32_be  | `koffi.decode.int32be(ptr)`  | Big Endian    | Number
+uint32_be | `koffi.decode.uint32be(ptr)` | Big Endian    | Number
+int64_be  | `koffi.decode.int64be(ptr)`  | Big Endian    | Number or BigInt
+uint64_be | `koffi.decode.uint64be(ptr)` | Big Endian    | Number or BigInt
+
+## Decode strings
+
+You can also decode strings, NUL-terminated or with an explicit length:
+
+Type     | Function                             | Conversion/encoding
+-------- | ------------------------------------ | -------------------------------------
+string   | `koffi.decode.string(ptr)`           | UTF-8 (NUL-terminated)
+string   | `koffi.decode.string(ptr, length)`   | UTF-8 (explicit length)
+string16 | `koffi.decode.string16(ptr)`         | UTF-16 (NUL-terminated)
+string16 | `koffi.decode.string16(ptr, length)` | UTF-16 (explicit length)
+string32 | `koffi.decode.string32(ptr)`         | UTF-32 (NUL-terminated)
+string32 | `koffi.decode.string32(ptr, length)` | UTF-32 (explicit length)
+wstring  | `koffi.decode.wstring(ptr)`          | UTF-16 or UTF-32 (platform-dependent)
+wstring  | `koffi.decode.wstring(ptr, length)`  | UTF-16 or UTF-32 (platform-dependent)
+
+## Generic decode function
+
+Use `koffi.decode()` to decode the data pointed to by a pointer (represented by a BigInt value).
+
+Some arguments are optional and this function can be called in several ways:
+
+- `koffi.decode(ptr, type)`: no offset
+- `koffi.decode(ptr, offset, type)`: explicit offset to add to the pointer before decoding
+
+By default, Koffi expects NUL terminated strings when decoding them. See below if you need to specify the string length.
+
+The following example illustrates how to decode an integer and a C string variable.
+
+```c
+int my_int = 42;
+const char *my_string = "foobar";
+```
+
+```js
+const my_int = lib.symbol('my_int');
+const my_string = lib.symbol('my_string');
+
+console.log(koffi.decode(my_int, 'int')) // Prints 42
+console.log(koffi.decode(my_string, 'const char *')) // Prints "foobar"
+```
+
+There is also an optional ending `length` argument that you can use in two cases:
+
+- Use it to give the number of bytes to decode in non-NUL terminated strings: `koffi.decode(ptr, 'char *', 5)`
+- Decode consecutive values into an array. For example, here is how you can decode an array with 3 float values: `koffi.decode(ptr, 'float', 3)`. This is equivalent to `koffi.decode(ptr, koffi.array('float', 3))`.
+
+The example below will decode the symbol `my_string` defined above but only the first three bytes.
+
+```js
+// Only decode 3 bytes from the C string my_string
+console.log(koffi.decode(my_string, 'const char *', 3)) // Prints "foo"
+```
+
+# Encode to C memory
+
+Use `koffi.encode()` to encode JS values into C symbols or pointers, which are represented by BigInt numbers.
+
+Some arguments are optional and this function can be called in several ways:
+
+- `koffi.encode(ptr, type, value)`: no offset
+- `koffi.encode(ptr, offset, type, value)`: explicit offset to add to the pointer before encoding
+
+We'll reuse the examples shown above and change the variable values with `koffi.encode()`.
+
+```c
+int my_int = 42;
+const char *my_string = NULL;
+```
+
+```js
+const my_int = lib.symbol('my_int');
+const my_string = lib.symbol('my_string');
+
+console.log(koffi.decode(my_int, 'int')) // Prints 42
+console.log(koffi.decode(my_string, 'const char *')) // Prints null
+
+koffi.encode(my_int, 'int', -1);
+koffi.encode(my_string, 'const char *', 'Hello World!');
+
+console.log(koffi.decode(my_int, 'int')) // Prints -1
+console.log(koffi.decode(my_string, 'const char *')) // Prints "Hello World!"
+```
+
+When encoding strings (either directly or embedded in arrays or structs), the memory will be bound to the raw pointer value and managed by Koffi. You can assign to the same string again and again without any leak or risk of use-after-free.
+
+There is also an optional ending `length` argument that you can use to encode an array. For example, here is how you can encode an array with 3 float values: `koffi.encode(symbol, 'float', [1, 2, 3], 3)`. This is equivalent to `koffi.encode(symbol, koffi.array('float', 3), [1, 2, 3])`.