koffi 2.7.4-beta.1 → 2.8.0

package/CHANGELOG.md

@@ -2,8 +2,22 @@
 
 ## Version history
 
+### Koffi 2.8
+
+#### Koffi 2.8.0 (2024-02-12)
+
+- Support pushing pointers for string arguments
+- Add `koffi.alloc()` for [stable pointers](output.md#stable-pointers)
+
 ### Koffi 2.7
 
+#### Koffi 2.7.4 (2024-02-04)
+
+- Support callbacks happening on main thread but outside JS call (such as during event loop)
+- Improve stability after `koffi.reset()`
+- Expose internal Node/NAPI `env` pointer
+- Name main Koffi API functions
+
 #### Koffi 2.7.3 (2024-01-26)
 
 - Support decoding NULL terminated arrays
@@ -123,7 +137,7 @@ Pre-built binaries don't work correctly in Koffi 2.5.13 to 2.5.15, skip those ve
 
 #### Koffi 2.5.11 (2023-08-03)
 
-- Support casting function pointers with [koffi.as()](polymorphism.md#input-polymorphism)
+- Support casting function pointers with [koffi.as()](pointers.md#handling-void-pointers)
 - Build in C++20 mode
 
 #### Koffi 2.5.10 (2023-08-01)
@@ -331,7 +345,7 @@ Pre-built binaries don't work correctly in Koffi 2.5.13 to 2.5.15, skip those ve
 **Main changes:**
 
 - Allow buffers (TypedArray or ArrayBuffer) values for input and/or output pointer arguments (for polymorphic arguments)
-- Support opaque buffers (TypedArray or ArrayBuffer) values in `koffi.decode()` to [decode output buffers](polymorphism.md#output-buffers)
+- Support opaque buffers (TypedArray or ArrayBuffer) values in `koffi.decode()` to [decode output buffers](output.md#output-buffers)
 - Decode non-string types as arrays when an [explicit length is passed to koffi.decode()](callbacks.md#decoding-pointer-arguments)
 
 **Other changes:**
@@ -424,7 +438,7 @@ Pre-built binaries don't work correctly in Koffi 2.5.13 to 2.5.15, skip those ve
 
 **Main changes:**
 
-- Add [koffi.as()](polymorphism.md#input-polymorphism) to support polymorphic APIs based on `void *` parameters
+- Add [koffi.as()](pointers.md#handling-void-pointers) to support polymorphic APIs based on `void *` parameters
 - Add [endian-sensitive integer types](input.md#endian-sensitive-integers): `intX_le_t`, `intX_be_t`, `uintX_le_t`, `uintX_be_t`
 - Accept typed arrays for `void *` parameters
 - Introduce `koffi.opaque()` to replace `koffi.handle()` (which remains supported until Koffi 3.0)

package/doc/functions.md

@@ -251,4 +251,3 @@ Among other thing, in the the following pages you will learn more about:
 - How Koffi translates [input parameters](input.md) to C
 - How you can [define and use pointers](pointers.md)
 - How to deal with [output parameters](output.md)
-- How to handle [polymorphic API](polymorphism.md)

package/doc/index.rst

@@ -29,7 +29,6 @@ Table of contents
    input
    pointers
    output
-   polymorphism
    unions
    variables
    callbacks

package/doc/input.md

@@ -393,7 +393,7 @@ The reverse case is also true, Koffi can convert a C fixed-size buffer to a JS s
 
 ### Dynamic arrays (pointers)
 
-In C, dynamically-sized arrays are usually passed around as pointers. Read more about [array pointers](pointers.md#array-pointers-dynamic-arrays) in the relevant section.
+In C, dynamically-sized arrays are usually passed around as pointers. Read more about [array pointers](pointers.md#dynamic-arrays) in the relevant section.
 
 ## Union types
 

package/doc/output.md

@@ -167,3 +167,111 @@ ConcatToBuffer(str1, str2, out);
 
 console.log(out[0]);
 ```
+
+## Output buffers
+
+In most cases, you can use buffers and typed arrays to provide output buffers. This works as long as the buffer only gets used while the native C function is being called. See [transient pointers](#transient-pointers) below for an example.
+
+```{warning}
+It is unsafe to keep the pointer around in the native code, or to change the contents outside of the function call where it is provided.
+
+If you need to provide a pointer that will be kept around, allocate memory with [koffi.alloc()](#stable-pointers) instead.
+```
+
+### Transient pointers
+
+*New in Koffi 2.3*
+
+You can use buffers and typed arrays for output (and input/output) pointer parameters. Simply pass the buffer as an argument and the native function will receive a pointer to its contents.
+
+Once the native function returns, you can decode the content with `koffi.decode(value, type)` as in the following example:
+
+```js
+// ES6 syntax: import koffi from 'koffi';
+const koffi = require('koffi');
+
+const lib = koffi.load('libc.so.6');
+
+const Vec3 = koffi.struct('Vec3', {
+    x: 'float32',
+    y: 'float32',
+    z: 'float32'
+})
+
+const memcpy = lib.func('void *memcpy(_Out_ void *dest, const void *src, size_t size)');
+
+let vec1 = { x: 1, y: 2, z: 3 };
+let vec2 = null;
+
+// Copy the vector in a convoluted way through memcpy
+{
+    let src = koffi.as(vec1, 'Vec3 *');
+    let dest = Buffer.allocUnsafe(koffi.sizeof(Vec3));
+
+    memcpy(dest, src, koffi.sizeof(Vec3));
+
+    vec2 = koffi.decode(dest, Vec3);
+}
+
+// CHange vector1, leaving copy alone
+[vec1.x, vec1.y, vec1.z] = [vec1.z, vec1.y, vec1.x];
+
+console.log(vec1); // { x: 3, y: 2, z: 1 }
+console.log(vec2); // { x: 1, y: 2, z: 3 }
+```
+
+See [decoding variables](variables.md#decode-to-js-values) for more information about the decode function.
+
+### Stable pointers
+
+*New in Koffi 2.8*
+
+In some cases, the native code may need to change the output buffer at a later time, maybe during a later call or from another thread.
+
+In this case, it is **not safe to use buffers or typed arrays**! 
+
+However, you can use `koffi.alloc(type, len)` to allocate memory and get a pointer that won't move, and can be safely used at any time by the native code. Use [koffi.decode()](variables.md#decode-to-js-values) to read data from the pointer when needed.
+
+The example below sets up some memory to be used as an output buffer where a concatenation function appends a string on each call.
+
+```c
+#include <assert.h>
+#include <stddef.h>
+
+static char *buf_ptr;
+static size_t buf_len;
+static size_t buf_size;
+
+void reset_buffer(char *buf, size_t size)
+{
+    assert(size > 1);
+
+    buf_ptr = buf;
+    buf_len = 0;
+    buf_size = size - 1; // Keep space for trailing NUL
+
+    buf_ptr[0] = 0;
+}
+
+void append_str(const char *str)
+{
+    for (size_t i = 0; str[i] && buf_len < buf_size; i++, buf_len++) {
+        buf_ptr[buf_len] = str[i];
+    }
+    buf_ptr[buf_len] = 0;
+}
+```
+
+```js
+const reset_buffer = lib.func('void reset_buffer(char *buf, size_t size)');
+const append_str = lib.func('void append_str(const char *str)');
+
+let output = koffi.alloc('char', 64);
+reset_buffer(output, 64);
+
+append_str('Hello');
+console.log(koffi.decode(output, 'char', -1)); // Prints Hello
+
+append_str(' World!');
+console.log(koffi.decode(output, 'char', -1)); // Prints Hello World!
+```

package/doc/pointers.md

@@ -88,7 +88,7 @@ AddInt(sum, 6);
 console.log(sum[0]); // Prints 42
 ```
 
-### Array pointers (dynamic arrays)
+### Dynamic arrays
 
 In C, dynamically-sized arrays are usually passed around as pointers. The length is either passed as an additional argument, or inferred from the array content itself, for example with a terminating sentinel value (such as a NULL pointers in the case of an array of strings).
 
@@ -132,6 +132,69 @@ console.log(total); // Prints 14
 
 By default, just like for objects, array arguments are copied from JS to C but not vice-versa. You can however change the direction as documented in the section on [output parameters](output.md).
 
+## Handling void pointers
+
+*New in Koffi 2.1*
+
+Many C functions use `void *` parameters in order to pass polymorphic objects and arrays, meaning that the data format changes can change depending on one other argument, or on some kind of struct tag member.
+
+Koffi provides two features to deal with this:
+
+- You can use `koffi.as(value, type)` to tell Koffi what kind of type is actually expected, as shown in the example below.
+- Buffers and typed JS arrays can be used as values in place everywhere a pointer is expected. See [dynamic arrays](#dynamic-arrays) for more information, for input or output.
+
+The example below shows the use of `koffi.as()` to read the header of a PNG file with `fread()` directly to a JS object.
+
+```js
+// ES6 syntax: import koffi from 'koffi';
+const koffi = require('koffi');
+
+const lib = koffi.load('libc.so.6');
+
+const FILE = koffi.opaque('FILE');
+
+const PngHeader = koffi.pack('PngHeader', {
+    signature: koffi.array('uint8_t', 8),
+    ihdr: koffi.pack({
+        length: 'uint32_be_t',
+        chunk: koffi.array('char', 4),
+        width: 'uint32_be_t',
+        height: 'uint32_be_t',
+        depth: 'uint8_t',
+        color: 'uint8_t',
+        compression: 'uint8_t',
+        filter: 'uint8_t',
+        interlace: 'uint8_t',
+        crc: 'uint32_be_t'
+    })
+});
+
+const fopen = lib.func('FILE *fopen(const char *path, const char *mode)');
+const fclose = lib.func('int fclose(FILE *fp)');
+const fread = lib.func('size_t fread(_Out_ void *ptr, size_t size, size_t nmemb, FILE *fp)');
+
+let filename = process.argv[2];
+if (filename == null)
+    throw new Error('Usage: node png.js <image.png>');
+
+let hdr = {};
+{
+    let fp = fopen(filename, 'rb');
+    if (!fp)
+        throw new Error(`Failed to open '${filename}'`);
+
+    try {
+        let len = fread(koffi.as(hdr, 'PngHeader *'), 1, koffi.sizeof(PngHeader), fp);
+        if (len < koffi.sizeof(PngHeader))
+            throw new Error('Failed to read PNG header');
+    } finally {
+        fclose(fp);
+    }
+}
+
+console.log('PNG header:', hdr);
+```
+
 ## Disposable types
 
 *New in Koffi 2.0*

package/index.js

@@ -378,8 +378,8 @@ var require_package = __commonJS({
   "build/dist/src/koffi/package.json"(exports2, module2) {
     module2.exports = {
       name: "koffi",
-      version: "2.7.4-beta.1",
-      stable: "2.7.3",
+      version: "2.8.0",
+      stable: "2.8.0",
       description: "Fast and simple C FFI (foreign function interface) for Node.js",
       keywords: [
         "foreign",

package/indirect.js

@@ -378,8 +378,8 @@ var require_package = __commonJS({
   "build/dist/src/koffi/package.json"(exports2, module2) {
     module2.exports = {
       name: "koffi",
-      version: "2.7.4-beta.1",
-      stable: "2.7.3",
+      version: "2.8.0",
+      stable: "2.8.0",
       description: "Fast and simple C FFI (foreign function interface) for Node.js",
       keywords: [
         "foreign",

package/package.json

@@ -1,7 +1,7 @@
 {
     "name": "koffi",
-    "version": "2.7.4-beta.1",
-    "stable": "2.7.3",
+    "version": "2.8.0",
+    "stable": "2.8.0",
     "description": "Fast and simple C FFI (foreign function interface) for Node.js",
     "keywords": [
         "foreign",

package/src/cnoke/assets/FindCNoke.cmake

@@ -30,12 +30,16 @@ function(add_node_addon)
     cmake_parse_arguments(ARG "" "NAME" "SOURCES" ${ARGN})
 
     add_library(${ARG_NAME} SHARED ${ARG_SOURCES} ${NODE_JS_SOURCES})
+    target_link_node(${ARG_NAME})
     set_target_properties(${ARG_NAME} PROPERTIES PREFIX "" SUFFIX ".node")
-    target_include_directories(${ARG_NAME} PRIVATE ${NODE_JS_INCLUDE_DIRS})
-    target_link_libraries(${ARG_NAME} PRIVATE ${NODE_JS_LIBRARIES})
-    target_compile_options(${ARG_NAME} PRIVATE ${NODE_JS_COMPILE_FLAGS})
+endfunction()
+
+function(target_link_node TARGET)
+    target_include_directories(${TARGET} PRIVATE ${NODE_JS_INCLUDE_DIRS})
+    target_link_libraries(${TARGET} PRIVATE ${NODE_JS_LIBRARIES})
+    target_compile_options(${TARGET} PRIVATE ${NODE_JS_COMPILE_FLAGS})
     if (NODE_JS_LINK_FLAGS)
-        target_link_options(${ARG_NAME} PRIVATE ${NODE_JS_LINK_FLAGS})
+        target_link_options(${TARGET} PRIVATE ${NODE_JS_LINK_FLAGS})
     endif()
 endfunction()
 

package/src/koffi/CMakeLists.txt

@@ -21,17 +21,16 @@
 
 cmake_minimum_required(VERSION 3.6)
 cmake_policy(SET CMP0091 NEW)
-
 project(koffi C CXX ASM)
 
-include(CheckCXXCompilerFlag)
-
 find_package(CNoke)
 
+include(CheckCXXCompilerFlag)
+
 set(CMAKE_CXX_STANDARD 20)
 if(MSVC)
     set(CMAKE_MSVC_RUNTIME_LIBRARY MultiThreaded)
-    set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /Zc:__cplusplus /W4 /wd4200 /wd4201 /wd4127 /wd4458 /wd4706 /wd4702 /wd4324")
+    set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /Zc:__cplusplus /Zc:preprocessor /W4 /wd4200 /wd4201 /wd4127 /wd4458 /wd4706 /wd4702 /wd4324")
 
     # ASM_MASM does not (yet) work on Windows ARM64
     if(NOT CMAKE_GENERATOR_PLATFORM MATCHES "ARM64")

package/src/koffi/src/call.cc

@@ -209,8 +209,7 @@ bool CallData::PushString(Napi::Value value, int directions, const char **out_st
 
         return true;
     } else {
-        ThrowError<Napi::TypeError>(env, "Unexpected %1 value, expected string", GetValueType(instance, value));
-        return false;
+        return PushPointer(value, instance->str_type, directions, (void **)out_str);
     }
 }
 
@@ -311,8 +310,7 @@ bool CallData::PushString16(Napi::Value value, int directions, const char16_t **
 
         return true;
     } else {
-        ThrowError<Napi::TypeError>(env, "Unexpected %1 value, expected string", GetValueType(instance, value));
-        return false;
+        return PushPointer(value, instance->str16_type, directions, (void **)out_str16);
     }
 }
 
@@ -983,7 +981,9 @@ bool CallData::PushPointer(Napi::Value value, const TypeInfo *type, int directio
         } break;
 
         case napi_external: {
-            RG_ASSERT(type->primitive == PrimitiveKind::Pointer);
+            RG_ASSERT(type->primitive == PrimitiveKind::Pointer ||
+                      type->primitive == PrimitiveKind::String ||
+                      type->primitive == PrimitiveKind::String16);
 
             if (!CheckValueTag(instance, value, type->ref.marker) &&
                     !CheckValueTag(instance, value, instance->void_type) &&

package/src/koffi/src/ffi.cc

@@ -770,6 +770,55 @@ static inline bool GetExternalPointer(Napi::Env env, Napi::Value value, void **o
     }
 }
 
+static Napi::Value CallAlloc(const Napi::CallbackInfo &info)
+{
+    Napi::Env env = info.Env();
+    InstanceData *instance = env.GetInstanceData<InstanceData>();
+
+    if (info.Length() < 2) {
+        ThrowError<Napi::TypeError>(env, "Expected 2 arguments, got %1", info.Length());
+        return env.Null();
+    }
+    if (!info[1].IsNumber()) {
+        ThrowError<Napi::TypeError>(env, "Unexpected %1 value for length, expected number", GetValueType(instance, info[1]));
+        return env.Null();
+    }
+
+    const TypeInfo *type = ResolveType(info[0]);
+    if (!type)
+        return env.Null();
+
+    if (!type->size) [[unlikely]] {
+        ThrowError<Napi::TypeError>(env, "Cannot allocate memory for zero-sized type %1", type->name);
+        return env.Null();
+    }
+
+    int32_t len = info[1].As<Napi::Number>();
+
+    if (len <= 0) [[unlikely]] {
+        ThrowError<Napi::Error>(env, "Size must be greater than 0");
+        return env.Null();
+    }
+    if (len > INT32_MAX / type->size) [[unlikely]] {
+        ThrowError<Napi::Error>(env, "Cannot allocate more than %1 objects of type %2", INT32_MAX / type->size, type->name);
+        return env.Null();
+    }
+
+    void *ptr = calloc((size_t)len, (size_t)type->size);
+
+    if (!ptr) [[unlikely]] {
+        Size size = (Size)(len * type->size);
+
+        ThrowError<Napi::Error>(env, "Failed to allocate %1 of memory", FmtMemSize((Size)size));
+        return env.Null();
+    }
+
+    Napi::External<void> external = Napi::External<void>::New(env, ptr);
+    SetValueTag(instance, external, type);
+
+    return external;
+}
+
 static Napi::Value CallFree(const Napi::CallbackInfo &info)
 {
     Napi::Env env = info.Env();
@@ -1997,6 +2046,40 @@ static Napi::Value EncodeValue(const Napi::CallbackInfo &info)
     return env.Undefined();
 }
 
+static Napi::Value ResetKoffi(const Napi::CallbackInfo &info)
+{
+    Napi::Env env = info.Env();
+    InstanceData *instance = env.GetInstanceData<InstanceData>();
+
+    if (instance->broker) {
+        napi_release_threadsafe_function(instance->broker, napi_tsfn_abort);
+        instance->broker = nullptr;
+    }
+
+    instance->types.RemoveFrom(instance->base_types_len);
+
+    {
+        HashSet<const void *> base_types;
+        HashMap<const char *, const TypeInfo *> new_map;
+
+        for (const TypeInfo &type: instance->types) {
+            base_types.Set(&type);
+        }
+
+        for (const auto &bucket: instance->types_map.table) {
+            if (base_types.Find(bucket.value)) {
+                new_map.Set(bucket.key, bucket.value);
+            }
+        }
+
+        std::swap(instance->types_map, new_map);
+    }
+
+    instance->callbacks.Clear();
+
+    return env.Undefined();
+}
+
 void LibraryHolder::Unload()
 {
 #ifdef _WIN32
@@ -2153,83 +2236,6 @@ static inline PrimitiveKind GetBigEndianPrimitive(PrimitiveKind kind)
 #endif
 }
 
-static Napi::Object InitBaseTypes(Napi::Env env)
-{
-    InstanceData *instance = env.GetInstanceData<InstanceData>();
-
-    Napi::Object types = Napi::Object::New(env);
-
-    RegisterPrimitiveType(env, types, {"void"}, PrimitiveKind::Void, 0, 0);
-    RegisterPrimitiveType(env, types, {"bool"}, PrimitiveKind::Bool, RG_SIZE(bool), alignof(bool));
-    RegisterPrimitiveType(env, types, {"int8_t", "int8"}, PrimitiveKind::Int8, 1, 1);
-    RegisterPrimitiveType(env, types, {"uint8_t", "uint8"}, PrimitiveKind::UInt8, 1, 1);
-    RegisterPrimitiveType(env, types, {"char"}, PrimitiveKind::Int8, 1, 1);
-    RegisterPrimitiveType(env, types, {"unsigned char", "uchar"}, PrimitiveKind::UInt8, 1, 1);
-    RegisterPrimitiveType(env, types, {"char16_t", "char16"}, PrimitiveKind::Int16, 2, 2);
-    RegisterPrimitiveType(env, types, {"int16_t", "int16"}, PrimitiveKind::Int16, 2, 2);
-    RegisterPrimitiveType(env, types, {"int16_le_t", "int16_le"}, GetLittleEndianPrimitive(PrimitiveKind::Int16), 2, 2);
-    RegisterPrimitiveType(env, types, {"int16_be_t", "int16_be"}, GetBigEndianPrimitive(PrimitiveKind::Int16), 2, 2);
-    RegisterPrimitiveType(env, types, {"uint16_t", "uint16"}, PrimitiveKind::UInt16, 2, 2);
-    RegisterPrimitiveType(env, types, {"uint16_le_t", "uint16_le"}, GetLittleEndianPrimitive(PrimitiveKind::UInt16), 2, 2);
-    RegisterPrimitiveType(env, types, {"uint16_be_t", "uint16_be"}, GetBigEndianPrimitive(PrimitiveKind::UInt16), 2, 2);
-    RegisterPrimitiveType(env, types, {"short"}, PrimitiveKind::Int16, 2, 2);
-    RegisterPrimitiveType(env, types, {"unsigned short", "ushort"}, PrimitiveKind::UInt16, 2, 2);
-    RegisterPrimitiveType(env, types, {"int32_t", "int32"}, PrimitiveKind::Int32, 4, 4);
-    RegisterPrimitiveType(env, types, {"int32_le_t", "int32_le"}, GetLittleEndianPrimitive(PrimitiveKind::Int32), 4, 4);
-    RegisterPrimitiveType(env, types, {"int32_be_t", "int32_be"}, GetBigEndianPrimitive(PrimitiveKind::Int32), 4, 4);
-    RegisterPrimitiveType(env, types, {"uint32_t", "uint32"}, PrimitiveKind::UInt32, 4, 4);
-    RegisterPrimitiveType(env, types, {"uint32_le_t", "uint32_le"}, GetLittleEndianPrimitive(PrimitiveKind::UInt32), 4, 4);
-    RegisterPrimitiveType(env, types, {"uint32_be_t", "uint32_be"}, GetBigEndianPrimitive(PrimitiveKind::UInt32), 4, 4);
-    RegisterPrimitiveType(env, types, {"int"}, PrimitiveKind::Int32, 4, 4);
-    RegisterPrimitiveType(env, types, {"unsigned int", "uint"}, PrimitiveKind::UInt32, 4, 4);
-    RegisterPrimitiveType(env, types, {"int64_t", "int64"}, PrimitiveKind::Int64, 8, alignof(int64_t));
-    RegisterPrimitiveType(env, types, {"int64_le_t", "int64_le"}, GetLittleEndianPrimitive(PrimitiveKind::Int64), 8, alignof(int64_t));
-    RegisterPrimitiveType(env, types, {"int64_be_t", "int64_be"}, GetBigEndianPrimitive(PrimitiveKind::Int64), 8, alignof(int64_t));
-    RegisterPrimitiveType(env, types, {"uint64_t", "uint64"}, PrimitiveKind::UInt64, 8, alignof(int64_t));
-    RegisterPrimitiveType(env, types, {"uint64_le_t", "uint64_le"}, GetLittleEndianPrimitive(PrimitiveKind::UInt64), 8, alignof(int64_t));
-    RegisterPrimitiveType(env, types, {"uint64_be_t", "uint64_be"}, GetBigEndianPrimitive(PrimitiveKind::UInt64), 8, alignof(int64_t));
-    RegisterPrimitiveType(env, types, {"intptr_t", "intptr"}, GetSignPrimitive(RG_SIZE(intptr_t), true), RG_SIZE(intptr_t), alignof(intptr_t));
-    RegisterPrimitiveType(env, types, {"uintptr_t", "uintptr"}, GetSignPrimitive(RG_SIZE(intptr_t), false), RG_SIZE(intptr_t), alignof(intptr_t));
-    RegisterPrimitiveType(env, types, {"size_t"}, GetSignPrimitive(RG_SIZE(size_t), false), RG_SIZE(size_t), alignof(size_t));
-    RegisterPrimitiveType(env, types, {"long"}, GetSignPrimitive(RG_SIZE(long), true), RG_SIZE(long), alignof(long));
-    RegisterPrimitiveType(env, types, {"unsigned long", "ulong"}, GetSignPrimitive(RG_SIZE(long), false), RG_SIZE(long), alignof(long));
-    RegisterPrimitiveType(env, types, {"long long", "longlong"}, PrimitiveKind::Int64, RG_SIZE(int64_t), alignof(int64_t));
-    RegisterPrimitiveType(env, types, {"unsigned long long", "ulonglong"}, PrimitiveKind::UInt64, RG_SIZE(uint64_t), alignof(uint64_t));
-    RegisterPrimitiveType(env, types, {"float", "float32"}, PrimitiveKind::Float32, 4, alignof(float));
-    RegisterPrimitiveType(env, types, {"double", "float64"}, PrimitiveKind::Float64, 8, alignof(double));
-    RegisterPrimitiveType(env, types, {"char *", "str", "string"}, PrimitiveKind::String, RG_SIZE(void *), alignof(void *), "char");
-    RegisterPrimitiveType(env, types, {"char16_t *", "char16 *", "str16", "string16"}, PrimitiveKind::String16, RG_SIZE(void *), alignof(void *), "char16_t");
-
-    instance->void_type = instance->types_map.FindValue("void", nullptr);
-    instance->char_type = instance->types_map.FindValue("char", nullptr);
-    instance->char16_type = instance->types_map.FindValue("char16", nullptr);
-
-    instance->active_symbol = Napi::Symbol::New(env, "active");
-
-    types.Freeze();
-
-    return types;
-}
-
-static Napi::Value ResetKoffi(const Napi::CallbackInfo &info)
-{
-    Napi::Env env = info.Env();
-    InstanceData *instance = env.GetInstanceData<InstanceData>();
-
-    if (instance->broker) {
-        napi_release_threadsafe_function(instance->broker, napi_tsfn_abort);
-        instance->broker = nullptr;
-    }
-
-    instance->types.Clear();
-    instance->types_map.Clear();
-    instance->callbacks.Clear();
-
-    InitBaseTypes(env);
-
-    return env.Undefined();
-}
-
 static InstanceData *CreateInstance()
 {
     InstanceData *instance = new InstanceData();
@@ -2258,52 +2264,53 @@ static Napi::Object InitModule(Napi::Env env, Napi::Object exports)
 
     env.SetInstanceData(instance);
 
-    exports.Set("config", Napi::Function::New(env, GetSetConfig));
-    exports.Set("stats", Napi::Function::New(env, GetStats));
-
-    exports.Set("struct", Napi::Function::New(env, CreatePaddedStructType));
-    exports.Set("pack", Napi::Function::New(env, CreatePackedStructType));
-    exports.Set("union", Napi::Function::New(env, CreateUnionType));
-    exports.Set("Union", Napi::Function::New(env, InstantiateUnion));
-    exports.Set("opaque", Napi::Function::New(env, CreateOpaqueType));
-    exports.Set("pointer", Napi::Function::New(env, CreatePointerType));
-    exports.Set("array", Napi::Function::New(env, CreateArrayType));
-    exports.Set("proto", Napi::Function::New(env, CreateFunctionType));
-    exports.Set("alias", Napi::Function::New(env, CreateTypeAlias));
+    exports.Set("config", Napi::Function::New(env, GetSetConfig, "config"));
+    exports.Set("stats", Napi::Function::New(env, GetStats, "stats"));
 
-    exports.Set("sizeof", Napi::Function::New(env, GetTypeSize));
-    exports.Set("alignof", Napi::Function::New(env, GetTypeAlign));
-    exports.Set("offsetof", Napi::Function::New(env, GetMemberOffset));
-    exports.Set("resolve", Napi::Function::New(env, GetResolvedType));
-    exports.Set("introspect", Napi::Function::New(env, GetTypeDefinition));
+    exports.Set("struct", Napi::Function::New(env, CreatePaddedStructType, "struct"));
+    exports.Set("pack", Napi::Function::New(env, CreatePackedStructType, "pack"));
+    exports.Set("union", Napi::Function::New(env, CreateUnionType, "union"));
+    exports.Set("Union", Napi::Function::New(env, InstantiateUnion, "Union"));
+    exports.Set("opaque", Napi::Function::New(env, CreateOpaqueType, "opaque"));
+    exports.Set("pointer", Napi::Function::New(env, CreatePointerType, "pointer"));
+    exports.Set("array", Napi::Function::New(env, CreateArrayType, "array"));
+    exports.Set("proto", Napi::Function::New(env, CreateFunctionType, "proto"));
+    exports.Set("alias", Napi::Function::New(env, CreateTypeAlias, "alias"));
 
-    exports.Set("load", Napi::Function::New(env, LoadSharedLibrary));
+    exports.Set("sizeof", Napi::Function::New(env, GetTypeSize, "sizeof"));
+    exports.Set("alignof", Napi::Function::New(env, GetTypeAlign, "alignof"));
+    exports.Set("offsetof", Napi::Function::New(env, GetMemberOffset, "offsetof"));
+    exports.Set("resolve", Napi::Function::New(env, GetResolvedType, "resolve"));
+    exports.Set("introspect", Napi::Function::New(env, GetTypeDefinition, "introspect"));
 
-    exports.Set("in", Napi::Function::New(env, MarkIn));
-    exports.Set("out", Napi::Function::New(env, MarkOut));
-    exports.Set("inout", Napi::Function::New(env, MarkInOut));
+    exports.Set("load", Napi::Function::New(env, LoadSharedLibrary, "load"));
 
-    exports.Set("disposable", Napi::Function::New(env, CreateDisposableType));
-    exports.Set("free", Napi::Function::New(env, CallFree));
+    exports.Set("in", Napi::Function::New(env, MarkIn, "in"));
+    exports.Set("out", Napi::Function::New(env, MarkOut, "out"));
+    exports.Set("inout", Napi::Function::New(env, MarkInOut, "inout"));
 
-    exports.Set("register", Napi::Function::New(env, RegisterCallback));
-    exports.Set("unregister", Napi::Function::New(env, UnregisterCallback));
+    exports.Set("disposable", Napi::Function::New(env, CreateDisposableType, "disposable"));
+    exports.Set("alloc", Napi::Function::New(env, CallAlloc, "alloc"));
+    exports.Set("free", Napi::Function::New(env, CallFree, "free"));
 
-    exports.Set("as", Napi::Function::New(env, CastValue));
-    exports.Set("decode", Napi::Function::New(env, DecodeValue));
-    exports.Set("address", Napi::Function::New(env, GetPointerAddress));
-    exports.Set("call", Napi::Function::New(env, CallPointerSync));
-    exports.Set("encode", Napi::Function::New(env, EncodeValue));
+    exports.Set("register", Napi::Function::New(env, RegisterCallback, "register"));
+    exports.Set("unregister", Napi::Function::New(env, UnregisterCallback, "unregister"));
 
-    exports.Set("reset", Napi::Function::New(env, ResetKoffi));
+    exports.Set("as", Napi::Function::New(env, CastValue, "as"));
+    exports.Set("decode", Napi::Function::New(env, DecodeValue, "decode"));
+    exports.Set("address", Napi::Function::New(env, GetPointerAddress, "address"));
+    exports.Set("call", Napi::Function::New(env, CallPointerSync, "call"));
+    exports.Set("encode", Napi::Function::New(env, EncodeValue, "encode"));
 
-    exports.Set("errno", Napi::Function::New(env, GetOrSetErrNo));
+    exports.Set("reset", Napi::Function::New(env, ResetKoffi, "reset"));
 
-    Napi::Object os = Napi::Object::New(env);
-    exports.Set("os", os);
+    exports.Set("errno", Napi::Function::New(env, GetOrSetErrNo, "errno"));
 
-    // Init constants mapping
+    // Export useful OS info
     {
+        Napi::Object os = Napi::Object::New(env);
+        exports.Set("os", os);
+
         Napi::Object codes = Napi::Object::New(env);
 
         for (const ErrnoCodeInfo &info: ErrnoCodes) {
@@ -2321,8 +2328,73 @@ static Napi::Object InitModule(Napi::Env env, Napi::Object exports)
     exports.Set("extension", Napi::String::New(env, ".so"));
 #endif
 
-    Napi::Object types = InitBaseTypes(env);
-    exports.Set("types", types);
+    // Init base types
+    {
+        Napi::Object types = Napi::Object::New(env);
+        exports.Set("types", types);
+
+        RegisterPrimitiveType(env, types, {"void"}, PrimitiveKind::Void, 0, 0);
+        RegisterPrimitiveType(env, types, {"bool"}, PrimitiveKind::Bool, RG_SIZE(bool), alignof(bool));
+        RegisterPrimitiveType(env, types, {"int8_t", "int8"}, PrimitiveKind::Int8, 1, 1);
+        RegisterPrimitiveType(env, types, {"uint8_t", "uint8"}, PrimitiveKind::UInt8, 1, 1);
+        RegisterPrimitiveType(env, types, {"char"}, PrimitiveKind::Int8, 1, 1);
+        RegisterPrimitiveType(env, types, {"unsigned char", "uchar"}, PrimitiveKind::UInt8, 1, 1);
+        RegisterPrimitiveType(env, types, {"char16_t", "char16"}, PrimitiveKind::Int16, 2, 2);
+        RegisterPrimitiveType(env, types, {"int16_t", "int16"}, PrimitiveKind::Int16, 2, 2);
+        RegisterPrimitiveType(env, types, {"int16_le_t", "int16_le"}, GetLittleEndianPrimitive(PrimitiveKind::Int16), 2, 2);
+        RegisterPrimitiveType(env, types, {"int16_be_t", "int16_be"}, GetBigEndianPrimitive(PrimitiveKind::Int16), 2, 2);
+        RegisterPrimitiveType(env, types, {"uint16_t", "uint16"}, PrimitiveKind::UInt16, 2, 2);
+        RegisterPrimitiveType(env, types, {"uint16_le_t", "uint16_le"}, GetLittleEndianPrimitive(PrimitiveKind::UInt16), 2, 2);
+        RegisterPrimitiveType(env, types, {"uint16_be_t", "uint16_be"}, GetBigEndianPrimitive(PrimitiveKind::UInt16), 2, 2);
+        RegisterPrimitiveType(env, types, {"short"}, PrimitiveKind::Int16, 2, 2);
+        RegisterPrimitiveType(env, types, {"unsigned short", "ushort"}, PrimitiveKind::UInt16, 2, 2);
+        RegisterPrimitiveType(env, types, {"int32_t", "int32"}, PrimitiveKind::Int32, 4, 4);
+        RegisterPrimitiveType(env, types, {"int32_le_t", "int32_le"}, GetLittleEndianPrimitive(PrimitiveKind::Int32), 4, 4);
+        RegisterPrimitiveType(env, types, {"int32_be_t", "int32_be"}, GetBigEndianPrimitive(PrimitiveKind::Int32), 4, 4);
+        RegisterPrimitiveType(env, types, {"uint32_t", "uint32"}, PrimitiveKind::UInt32, 4, 4);
+        RegisterPrimitiveType(env, types, {"uint32_le_t", "uint32_le"}, GetLittleEndianPrimitive(PrimitiveKind::UInt32), 4, 4);
+        RegisterPrimitiveType(env, types, {"uint32_be_t", "uint32_be"}, GetBigEndianPrimitive(PrimitiveKind::UInt32), 4, 4);
+        RegisterPrimitiveType(env, types, {"int"}, PrimitiveKind::Int32, 4, 4);
+        RegisterPrimitiveType(env, types, {"unsigned int", "uint"}, PrimitiveKind::UInt32, 4, 4);
+        RegisterPrimitiveType(env, types, {"int64_t", "int64"}, PrimitiveKind::Int64, 8, alignof(int64_t));
+        RegisterPrimitiveType(env, types, {"int64_le_t", "int64_le"}, GetLittleEndianPrimitive(PrimitiveKind::Int64), 8, alignof(int64_t));
+        RegisterPrimitiveType(env, types, {"int64_be_t", "int64_be"}, GetBigEndianPrimitive(PrimitiveKind::Int64), 8, alignof(int64_t));
+        RegisterPrimitiveType(env, types, {"uint64_t", "uint64"}, PrimitiveKind::UInt64, 8, alignof(int64_t));
+        RegisterPrimitiveType(env, types, {"uint64_le_t", "uint64_le"}, GetLittleEndianPrimitive(PrimitiveKind::UInt64), 8, alignof(int64_t));
+        RegisterPrimitiveType(env, types, {"uint64_be_t", "uint64_be"}, GetBigEndianPrimitive(PrimitiveKind::UInt64), 8, alignof(int64_t));
+        RegisterPrimitiveType(env, types, {"intptr_t", "intptr"}, GetSignPrimitive(RG_SIZE(intptr_t), true), RG_SIZE(intptr_t), alignof(intptr_t));
+        RegisterPrimitiveType(env, types, {"uintptr_t", "uintptr"}, GetSignPrimitive(RG_SIZE(intptr_t), false), RG_SIZE(intptr_t), alignof(intptr_t));
+        RegisterPrimitiveType(env, types, {"size_t"}, GetSignPrimitive(RG_SIZE(size_t), false), RG_SIZE(size_t), alignof(size_t));
+        RegisterPrimitiveType(env, types, {"long"}, GetSignPrimitive(RG_SIZE(long), true), RG_SIZE(long), alignof(long));
+        RegisterPrimitiveType(env, types, {"unsigned long", "ulong"}, GetSignPrimitive(RG_SIZE(long), false), RG_SIZE(long), alignof(long));
+        RegisterPrimitiveType(env, types, {"long long", "longlong"}, PrimitiveKind::Int64, RG_SIZE(int64_t), alignof(int64_t));
+        RegisterPrimitiveType(env, types, {"unsigned long long", "ulonglong"}, PrimitiveKind::UInt64, RG_SIZE(uint64_t), alignof(uint64_t));
+        RegisterPrimitiveType(env, types, {"float", "float32"}, PrimitiveKind::Float32, 4, alignof(float));
+        RegisterPrimitiveType(env, types, {"double", "float64"}, PrimitiveKind::Float64, 8, alignof(double));
+        RegisterPrimitiveType(env, types, {"char *", "str", "string"}, PrimitiveKind::String, RG_SIZE(void *), alignof(void *), "char");
+        RegisterPrimitiveType(env, types, {"char16_t *", "char16 *", "str16", "string16"}, PrimitiveKind::String16, RG_SIZE(void *), alignof(void *), "char16_t");
+
+        instance->void_type = instance->types_map.FindValue("void", nullptr);
+        instance->char_type = instance->types_map.FindValue("char", nullptr);
+        instance->char16_type = instance->types_map.FindValue("char16", nullptr);
+        instance->str_type = instance->types_map.FindValue("char *", nullptr);
+        instance->str16_type = instance->types_map.FindValue("char16_t *", nullptr);
+
+        instance->active_symbol = Napi::Symbol::New(env, "active");
+
+        instance->base_types_len = instance->types.len;
+    }
+
+    // Expose internal Node stuff
+    {
+        Napi::Object node = Napi::Object::New(env);
+        exports.Set("node", node);
+
+        Napi::External<void> external = Napi::External<void>::New(env, (napi_env)env);
+        SetValueTag(instance, external, instance->void_type);
+
+        node.Set("env", external);
+    }
 
     exports.Set("version", Napi::String::New(env, RG_STRINGIFY(VERSION)));
 

package/src/koffi/src/ffi.hh

@@ -269,6 +269,7 @@ struct InstanceData {
     BucketArray<TypeInfo> types;
     HashMap<const char *, const TypeInfo *> types_map;
     BucketArray<FunctionInfo> callbacks;
+    Size base_types_len;
 
     bool debug;
     uint64_t tag_lower;
@@ -276,6 +277,8 @@ struct InstanceData {
     const TypeInfo *void_type;
     const TypeInfo *char_type;
     const TypeInfo *char16_type;
+    const TypeInfo *str_type;
+    const TypeInfo *str16_type;
 
     Napi::Symbol active_symbol;
 

package/doc/polymorphism.md

@@ -1,108 +0,0 @@
-# Polymorphic arguments
-
-## Input polymorphism
-
-*New in Koffi 2.1*
-
-Many C functions use `void *` parameters in order to pass polymorphic objects and arrays, meaning that the data format changes can change depending on one other argument, or on some kind of struct tag member.
-
-Koffi provides two features to deal with this:
-
-- Buffers and typed JS arrays can be used as values in place everywhere a pointer is expected. See [dynamic arrays](pointers.md#array-pointers-dynamic-arrays) for more information, for input or output.
-- You can use `koffi.as(value, type)` to tell Koffi what kind of type is actually expected.
-
-The example below shows the use of `koffi.as()` to read the header of a PNG file with `fread()`.
-
-```js
-// ES6 syntax: import koffi from 'koffi';
-const koffi = require('koffi');
-
-const lib = koffi.load('libc.so.6');
-
-const FILE = koffi.opaque('FILE');
-
-const PngHeader = koffi.pack('PngHeader', {
-    signature: koffi.array('uint8_t', 8),
-    ihdr: koffi.pack({
-        length: 'uint32_be_t',
-        chunk: koffi.array('char', 4),
-        width: 'uint32_be_t',
-        height: 'uint32_be_t',
-        depth: 'uint8_t',
-        color: 'uint8_t',
-        compression: 'uint8_t',
-        filter: 'uint8_t',
-        interlace: 'uint8_t',
-        crc: 'uint32_be_t'
-    })
-});
-
-const fopen = lib.func('FILE *fopen(const char *path, const char *mode)');
-const fclose = lib.func('int fclose(FILE *fp)');
-const fread = lib.func('size_t fread(_Out_ void *ptr, size_t size, size_t nmemb, FILE *fp)');
-
-let filename = process.argv[2];
-if (filename == null)
-    throw new Error('Usage: node png.js <image.png>');
-
-let hdr = {};
-{
-    let fp = fopen(filename, 'rb');
-    if (!fp)
-        throw new Error(`Failed to open '${filename}'`);
-
-    try {
-        let len = fread(koffi.as(hdr, 'PngHeader *'), 1, koffi.sizeof(PngHeader), fp);
-        if (len < koffi.sizeof(PngHeader))
-            throw new Error('Failed to read PNG header');
-    } finally {
-        fclose(fp);
-    }
-}
-
-console.log('PNG header:', hdr);
-```
-
-## Output buffers
-
-*New in Koffi 2.3*
-
-You can use buffers and typed arrays for output (and input/output) pointer parameters. Simply pass the buffer as an argument and the native function will receive a pointer to its contents.
-
-Once the native function returns, you can decode the content with `koffi.decode(value, type)` as in the following example:
-
-```js
-// ES6 syntax: import koffi from 'koffi';
-const koffi = require('koffi');
-
-const lib = koffi.load('libc.so.6');
-
-const Vec3 = koffi.struct('Vec3', {
-    x: 'float32',
-    y: 'float32',
-    z: 'float32'
-})
-
-const memcpy = lib.func('void *memcpy(_Out_ void *dest, const void *src, size_t size)');
-
-let vec1 = { x: 1, y: 2, z: 3 };
-let vec2 = null;
-
-// Copy the vector in a convoluted way through memcpy
-{
-    let src = koffi.as(vec1, 'Vec3 *');
-    let dest = Buffer.allocUnsafe(koffi.sizeof(Vec3));
-
-    memcpy(dest, src, koffi.sizeof(Vec3));
-
-    vec2 = koffi.decode(dest, Vec3);
-}
-
-// CHange vector1, leaving copy alone
-[vec1.x, vec1.y, vec1.z] = [vec1.z, vec1.y, vec1.x];
-
-console.log(vec1); // { x: 3, y: 2, z: 1 }
-console.log(vec2); // { x: 1, y: 2, z: 3 }
-```
-
-See [decoding variables](variables.md#decode-to-js-values) for more information about the decode function.