lakutata 3.1.1 → 3.1.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +2 -2
- package/README.zh-CN.md +1 -1
- package/doc/en/Command.md +3 -0
- package/doc/en/FFI.md +458 -0
- package/doc/en/NativeAddons.md +2 -1
- package/doc/en/Overview.md +3 -1
- package/doc/en/UpgradingFrom2.md +4 -2
- package/doc/zh/Command.md +2 -0
- package/doc/zh/FFI.md +337 -0
- package/doc/zh/NativeAddons.md +1 -1
- package/doc/zh/Overview.md +2 -1
- package/doc/zh/UpgradingFrom2.md +1 -1
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -144,8 +144,8 @@ on. Each package's README lists its exceptions.
|
|
|
144
144
|
`Entrypoint.md`, `Middlewares.md`, `HTTP.md`, `CLI.md`, `ServiceCalls.md`, `GRPC.md`, `NATS.md`, `Schedules.md`,
|
|
145
145
|
`Queue.md`, `Database.md`, `DatabaseDrivers.md`, `MongoEmbedded.md`, `Entities.md`, `Migrations.md`, `Cache.md`,
|
|
146
146
|
`Mutex.md`, `I18n.md`, `DockerIntegration.md`, `ContainerdIntegration.md`, `Monitor.md`, `Security.md`, `DBus.md`,
|
|
147
|
-
`MCP.md`, `WebRTC.md`, `Command.md`, `Workspaces.md`, `NativeAddons.md`, `Packaging.md`, `Installers.md`;
|
|
148
|
-
examples, in `doc/en/examples` and `doc/zh/examples`: whole applications built from the templates.
|
|
147
|
+
`MCP.md`, `WebRTC.md`, `FFI.md`, `Command.md`, `Workspaces.md`, `NativeAddons.md`, `Packaging.md`, `Installers.md`;
|
|
148
|
+
and the examples, in `doc/en/examples` and `doc/zh/examples`: whole applications built from the templates.
|
|
149
149
|
- `@lakutata/cli`: creating, building and packaging applications.
|
|
150
150
|
- The official Agent Skill for AI coding assistants that work on lakutata projects:
|
|
151
151
|
[lakutata.com/skill](https://lakutata.com/skill/).
|
package/README.zh-CN.md
CHANGED
|
@@ -121,7 +121,7 @@ Application
|
|
|
121
121
|
## 另见
|
|
122
122
|
|
|
123
123
|
- 网站 <https://lakutata.com/zh/>:指南、示例和 API 参考,中英文都有。
|
|
124
|
-
- 指南在这个包的 `doc/zh`(中文)和 `doc/en`(英文)中:`Overview.md`、`GettingStarted.md`、`ProjectStructure.md`、`UpgradingFrom2.md`、`UpgradingFrom1.md`、`Application.md`、`Objects.md`、`Injection.md`、`DTOValidation.md`、`Exceptions.md`、`Logger.md`、`Decorator.md`、`Time.md`、`HelperFunctions.md`、`Controllers.md`、`Entrypoint.md`、`Middlewares.md`、`HTTP.md`、`CLI.md`、`ServiceCalls.md`、`GRPC.md`、`NATS.md`、`Schedules.md`、`Queue.md`、`Database.md`、`DatabaseDrivers.md`、`MongoEmbedded.md`、`Entities.md`、`Migrations.md`、`Cache.md`、`Mutex.md`、`I18n.md`、`DockerIntegration.md`、`ContainerdIntegration.md`、`Monitor.md`、`Security.md`、`DBus.md`、`MCP.md`、`WebRTC.md`、`Command.md`、`Workspaces.md`、`NativeAddons.md`、`Packaging.md`、`Installers.md`;示例在 `doc/zh/examples` 和 `doc/en/examples` 中:从模板构建的完整应用。
|
|
124
|
+
- 指南在这个包的 `doc/zh`(中文)和 `doc/en`(英文)中:`Overview.md`、`GettingStarted.md`、`ProjectStructure.md`、`UpgradingFrom2.md`、`UpgradingFrom1.md`、`Application.md`、`Objects.md`、`Injection.md`、`DTOValidation.md`、`Exceptions.md`、`Logger.md`、`Decorator.md`、`Time.md`、`HelperFunctions.md`、`Controllers.md`、`Entrypoint.md`、`Middlewares.md`、`HTTP.md`、`CLI.md`、`ServiceCalls.md`、`GRPC.md`、`NATS.md`、`Schedules.md`、`Queue.md`、`Database.md`、`DatabaseDrivers.md`、`MongoEmbedded.md`、`Entities.md`、`Migrations.md`、`Cache.md`、`Mutex.md`、`I18n.md`、`DockerIntegration.md`、`ContainerdIntegration.md`、`Monitor.md`、`Security.md`、`DBus.md`、`MCP.md`、`WebRTC.md`、`FFI.md`、`Command.md`、`Workspaces.md`、`NativeAddons.md`、`Packaging.md`、`Installers.md`;示例在 `doc/zh/examples` 和 `doc/en/examples` 中:从模板构建的完整应用。
|
|
125
125
|
- `@lakutata/cli`:创建、构建和打包应用。
|
|
126
126
|
- 官方 Agent Skill(智能体技能),供 AI 编程助手在 lakutata 项目中使用:<https://lakutata.com/zh/skill/>。
|
|
127
127
|
|
package/doc/en/Command.md
CHANGED
|
@@ -81,6 +81,9 @@ packages, run a script in each of them in the order of their dependencies, give
|
|
|
81
81
|
`lakutata add <name> --native gyp|cmake|rust` adds a native addon written in C++ or Rust, and
|
|
82
82
|
`lakutata native <link|include|define> <value>` links libraries to it. See [NativeAddons](NativeAddons.md).
|
|
83
83
|
|
|
84
|
+
`lakutata ffi generate <header>` writes the class of a native library of `@lakutata/ffi` from its C header. See
|
|
85
|
+
[FFI](FFI.md), section 9.
|
|
86
|
+
|
|
84
87
|
## 4. Migrations and translations
|
|
85
88
|
|
|
86
89
|
`lakutata migration <action>` generates, creates, runs, reverts and shows the migrations of the databases of an
|
package/doc/en/FFI.md
ADDED
|
@@ -0,0 +1,458 @@
|
|
|
1
|
+
# Lakutata FFI
|
|
2
|
+
|
|
3
|
+
The package `@lakutata/ffi` calls the C functions of native libraries (`.so`, `.dylib`, `.dll`) from Node.js, the main
|
|
4
|
+
process of Electron and Bun: the APIs of the system, the SDKs of devices, existing C libraries. Nothing is
|
|
5
|
+
compiled and no addon is written: a library is declared with the C declarations of its headers, and each library is a
|
|
6
|
+
component of the application. It is built on [koffi](https://koffi.dev/). It is not part of `lakutata`:
|
|
7
|
+
|
|
8
|
+
```bash
|
|
9
|
+
npm install lakutata @lakutata/ffi
|
|
10
|
+
```
|
|
11
|
+
|
|
12
|
+
koffi comes with a package of prebuilt binaries for each platform, installed with it. npm may warn that the install
|
|
13
|
+
script of koffi was not run: it is not needed.
|
|
14
|
+
|
|
15
|
+
A C function called with a wrong prototype, or given a pointer it should not get, ends the process: no exception can
|
|
16
|
+
catch that. Copy the declarations from the headers of the library, and test them.
|
|
17
|
+
|
|
18
|
+
## 1. A library
|
|
19
|
+
|
|
20
|
+
A library is a subclass of `NativeLibrary`: `@Library` gives its file names and its C declarations, `@Native` binds a
|
|
21
|
+
function to a property, `@NativeVariable` a variable. The TypeScript type of each property is the type of the
|
|
22
|
+
function.
|
|
23
|
+
|
|
24
|
+
```typescript
|
|
25
|
+
import {Application, Component} from 'lakutata'
|
|
26
|
+
import {Inject} from 'lakutata/decorator/di'
|
|
27
|
+
import {buildNativeLibraryOptions, Library, Native, NativeLibrary, NativeVariable, type Out, type Pointer, readArray} from '@lakutata/ffi'
|
|
28
|
+
|
|
29
|
+
type RowCallback = (context: Pointer | null, count: number, values: Pointer, names: Pointer) => number
|
|
30
|
+
|
|
31
|
+
@Library({
|
|
32
|
+
names: {darwin: 'libsqlite3.dylib', linux: 'libsqlite3.so.0', win32: 'winsqlite3.dll'},
|
|
33
|
+
declarations: `
|
|
34
|
+
typedef struct sqlite3 sqlite3;
|
|
35
|
+
typedef int (*sqlite3_callback)(void *context, int count, char **values, char **names);
|
|
36
|
+
#define SQLITE_OK 0
|
|
37
|
+
int sqlite3_close(sqlite3 *db);
|
|
38
|
+
`
|
|
39
|
+
})
|
|
40
|
+
export class SQLite extends NativeLibrary {
|
|
41
|
+
@Native('int sqlite3_open(const char *filename, _Out_ sqlite3 **db)')
|
|
42
|
+
public readonly open: (filename: string, db: Out<Pointer>) => number
|
|
43
|
+
|
|
44
|
+
//Runs in a thread of the pool; the error message it gives is freed by sqlite3_free
|
|
45
|
+
@Native('int sqlite3_exec(sqlite3 *db, const char *sql, sqlite3_callback callback, void *context, _Out_ char **error)', {async: true, free: 'sqlite3_free'})
|
|
46
|
+
public readonly exec: (db: Pointer, sql: string, callback: RowCallback | null, context: null, error: Out<string>) => Promise<number>
|
|
47
|
+
|
|
48
|
+
//The function of the declarations named as the property
|
|
49
|
+
@Native()
|
|
50
|
+
public readonly sqlite3_close: (db: Pointer) => number
|
|
51
|
+
|
|
52
|
+
@NativeVariable('extern const char sqlite3_version[]')
|
|
53
|
+
public readonly version: string
|
|
54
|
+
}
|
|
55
|
+
|
|
56
|
+
class Store extends Component {
|
|
57
|
+
@Inject('sqlite')
|
|
58
|
+
protected readonly sqlite: SQLite
|
|
59
|
+
|
|
60
|
+
public async tables(): Promise<string[]> {
|
|
61
|
+
const db: Out<Pointer> = [null]
|
|
62
|
+
if (this.sqlite.open(':memory:', db) !== Number(this.sqlite.constants.SQLITE_OK)) throw new Error('open failed')
|
|
63
|
+
const names: string[] = []
|
|
64
|
+
const error: Out<string> = [null]
|
|
65
|
+
await this.sqlite.exec(db[0]!, 'SELECT name FROM sqlite_master', (context: Pointer | null, count: number, values: Pointer): number => {
|
|
66
|
+
names.push(...readArray(values, 'char *', count) as string[])
|
|
67
|
+
return 0
|
|
68
|
+
}, null, error)
|
|
69
|
+
this.sqlite.sqlite3_close(db[0]!)
|
|
70
|
+
return names
|
|
71
|
+
}
|
|
72
|
+
}
|
|
73
|
+
|
|
74
|
+
Application.run(() => ({
|
|
75
|
+
id: 'store.app',
|
|
76
|
+
name: 'Store',
|
|
77
|
+
components: {
|
|
78
|
+
sqlite: buildNativeLibraryOptions(SQLite),
|
|
79
|
+
store: {class: Store}
|
|
80
|
+
}
|
|
81
|
+
}))
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
- **The names**: one name (`sqlite3` is completed as each platform names its libraries: `libsqlite3.dylib`,
|
|
85
|
+
`libsqlite3.so`, `sqlite3.dll`), a file, or one by platform (the names of `process.platform`, `default` for the
|
|
86
|
+
others). `c` is the C library and `m` its mathematical functions, found where each system has them: `libSystem` on
|
|
87
|
+
macOS, `libc.so.6` with glibc, the dynamic loader of musl on Alpine, `ucrtbase.dll` on Windows.
|
|
88
|
+
- **The declarations** are written as the headers write them, and can be copied from them: typedefs, structs and unions
|
|
89
|
+
(defined later, referring to themselves, anonymous members), enums (their values can be constant expressions),
|
|
90
|
+
pointers to functions, prototypes, variables, `#define` without arguments (an empty one is removed where it is used,
|
|
91
|
+
as the markers of exported functions; one whose value is an integer is also a constant), `#pragma pack`, the packed
|
|
92
|
+
and aligned attributes. The other attributes, the bodies of inline functions and the `extern "C"` blocks are skipped.
|
|
93
|
+
Each library has its own declarations: two libraries can both declare `HANDLE`.
|
|
94
|
+
- **What is refused**, with its line and its column: `#if` and its kind (run the header through the preprocessor first:
|
|
95
|
+
`cc -E`, `cl /P`), the macros with arguments, `long double`, `__int128`, and the declarations of the old style. The
|
|
96
|
+
types of `stdint.h`, of POSIX (`time_t`, `pid_t`…), of BSD (`u_int`…) and of the Windows API (`DWORD`, `HANDLE`,
|
|
97
|
+
`LPCWSTR`…) are known without being declared, with their size on each platform.
|
|
98
|
+
- **`@Native`** takes a prototype, the name of a function of the declarations, or nothing (the function named as the
|
|
99
|
+
property). The property gets the function when the component is created: the function of koffi itself when nothing
|
|
100
|
+
has to be converted, so a call costs no more than through koffi (about 20 ns).
|
|
101
|
+
- **`@NativeVariable`** takes a declaration, a name, or nothing: reading the property reads the variable, writing it
|
|
102
|
+
writes the variable (refused for a const one). An array of `char` of unknown length is read as a string.
|
|
103
|
+
- A subclass inherits the library of its parent class and can add functions.
|
|
104
|
+
- For a library with many functions, the class can be generated from its header (section 9).
|
|
105
|
+
|
|
106
|
+
## 2. Where the library is found
|
|
107
|
+
|
|
108
|
+
When the component is created, the library is looked for:
|
|
109
|
+
|
|
110
|
+
1. at the `path` of its configuration, when given (the only file tried then);
|
|
111
|
+
2. in the directories of `search` in the configuration, then in the ones of `@Library`;
|
|
112
|
+
3. in a packaged application, in the directory `native` of its resources;
|
|
113
|
+
4. where the system looks for libraries (`dlopen`, `LoadLibrary`).
|
|
114
|
+
|
|
115
|
+
```typescript
|
|
116
|
+
import {Application} from 'lakutata'
|
|
117
|
+
import {buildNativeLibraryOptions, Library, Native, NativeLibrary} from '@lakutata/ffi'
|
|
118
|
+
|
|
119
|
+
@Library({names: {darwin: 'libvendor.dylib', linux: 'libvendor.so.2', win32: 'vendor.dll'}})
|
|
120
|
+
class Vendor extends NativeLibrary {
|
|
121
|
+
@Native('int vendor_version(void)')
|
|
122
|
+
public readonly version: () => number
|
|
123
|
+
}
|
|
124
|
+
|
|
125
|
+
Application.run(() => ({
|
|
126
|
+
id: 'devices.app',
|
|
127
|
+
name: 'Devices',
|
|
128
|
+
components: {
|
|
129
|
+
vendor: buildNativeLibraryOptions(Vendor, {search: ['./native']})
|
|
130
|
+
}
|
|
131
|
+
}))
|
|
132
|
+
```
|
|
133
|
+
|
|
134
|
+
When it cannot be loaded, `NativeLibraryNotFoundException` lists each file tried and why it was not loaded (not found,
|
|
135
|
+
built for another architecture, a library it needs missing). On Linux, the libraries of the system are often installed
|
|
136
|
+
with their version only (`libsqlite3.so.0`): give that name.
|
|
137
|
+
|
|
138
|
+
To ship a library with a packaged application, put it in the resources of the package.json, in `native`:
|
|
139
|
+
|
|
140
|
+
```json
|
|
141
|
+
{
|
|
142
|
+
"lakutata": {
|
|
143
|
+
"resources": [{"from": "vendor/lib", "to": "native"}]
|
|
144
|
+
}
|
|
145
|
+
}
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
The bundles, the executables (of Node.js and of Bun) and the installers of Electron made by `lakutata package` carry it,
|
|
149
|
+
and koffi with its binary for the platform. On Alpine, make the executable with Bun (`--runtime bun`): the executables
|
|
150
|
+
of Node.js do not load native addons on musl. Under the permission model of Node.js, run with `--allow-addons`.
|
|
151
|
+
|
|
152
|
+
## 3. Calling functions
|
|
153
|
+
|
|
154
|
+
| C | JavaScript |
|
|
155
|
+
|---|---|
|
|
156
|
+
| integers, `float`, `double`, `bool`, enums | numbers, booleans |
|
|
157
|
+
| 64-bit integers (`int64_t`, `long long`, `size_t`, `long`, `time_t`…) | bigints (numbers are accepted as arguments) |
|
|
158
|
+
| `const char *`, `char *` | strings (a `Buffer` is accepted too, and the function writes into it); `char16_t *`, `wchar_t *` too |
|
|
159
|
+
| other pointers | bigints (`Pointer`), `null` for NULL; a `Buffer` or a typed array can be given |
|
|
160
|
+
| structs, unions | objects |
|
|
161
|
+
| pointers to functions | functions (see section 5) |
|
|
162
|
+
|
|
163
|
+
- **The 64-bit integers** are bigints everywhere: results, output parameters, members of structs, arguments of
|
|
164
|
+
callbacks. `long` is 64-bit on Linux and macOS, 32-bit on Windows: it is a bigint on every platform. A function
|
|
165
|
+
declared with `{int64: 'number'}` gives numbers instead, and throws a `RangeError` for a value that is not a safe
|
|
166
|
+
integer.
|
|
167
|
+
- **Output parameters**: a parameter marked `_Out_` (or `_Inout_`) is written by the function. An integer or a pointer
|
|
168
|
+
is given as an array of one element (`Out<T>`, `[null]` before the call); a struct as an object, whose members are
|
|
169
|
+
set once the function returns. A pointer without a mark is given to the function only: a struct passed through it is
|
|
170
|
+
not read back.
|
|
171
|
+
- **A buffer given to a function** is valid during the call only: memory the library keeps must be allocated with
|
|
172
|
+
`alloc()` (section 6).
|
|
173
|
+
- **Variadic functions** (`printf`, `curl_easy_setopt`) take the type of each extra argument by its C name, before it:
|
|
174
|
+
|
|
175
|
+
```typescript
|
|
176
|
+
import {loadNativeLibrary} from '@lakutata/ffi'
|
|
177
|
+
|
|
178
|
+
const libc = loadNativeLibrary('c')
|
|
179
|
+
const snprintf = libc.func<(buffer: Buffer, size: number, format: string, ...args: unknown[]) => number>('int snprintf(char *buffer, size_t size, const char *format, ...)')
|
|
180
|
+
const buffer: Buffer = Buffer.alloc(64)
|
|
181
|
+
snprintf(buffer, buffer.length, '%s: %d', 'const char *', 'count', 'int', 42)
|
|
182
|
+
```
|
|
183
|
+
|
|
184
|
+
- **Strings to free**: a function returning a string the caller must free (or giving one through an output parameter)
|
|
185
|
+
is declared with `{free: 'sqlite3_free'}`, the function of the library that frees it, or `{free: true}` for `free()`
|
|
186
|
+
of the C library. The string is read, then freed. On Windows, a library built with another C library must be given its
|
|
187
|
+
own function.
|
|
188
|
+
- **Asynchronous calls**: a function declared with `{async: true}` runs in another thread and returns a promise; the
|
|
189
|
+
event loop goes on during the call. By default it runs in the thread pool of the runtime: Node.js runs them in the
|
|
190
|
+
pool of libuv, 4 threads by default, shared with the file system, DNS and crypto; raise `UV_THREADPOOL_SIZE` for many
|
|
191
|
+
calls, or long ones. A variadic function cannot be asynchronous there. A library can have threads of its own instead
|
|
192
|
+
(section 10).
|
|
193
|
+
|
|
194
|
+
## 4. Structs, unions, enums and constants
|
|
195
|
+
|
|
196
|
+
A struct is an object of its members. Its nested structs are objects; its arrays of `char` are strings, its arrays of
|
|
197
|
+
numbers typed arrays (`Uint8Array`, `Int32Array`…), its arrays of 64-bit integers arrays of bigints. An anonymous
|
|
198
|
+
struct or union inside a struct is the member `_0` (then `_1`…).
|
|
199
|
+
|
|
200
|
+
- **Bitfields** are members like the others, read and written by their names: the package lays them out as the
|
|
201
|
+
compilers of the platform do (the rules of GCC and Clang, of ARM, of Microsoft). A bitfield of a 64-bit type is a
|
|
202
|
+
bigint.
|
|
203
|
+
- **Packed structs** (`#pragma pack`, the packed attribute) and the aligned attribute (1, 2, 4 or 8 bytes) are laid out
|
|
204
|
+
as the compilers do.
|
|
205
|
+
- **A union** is given as an object setting one of its members (`{i: 42}`).
|
|
206
|
+
- **Enums** are integers; their values, the static constants and the integer macros of the declarations are in
|
|
207
|
+
`constants`, as bigints (`Number(sqlite.constants.SQLITE_OK)`).
|
|
208
|
+
|
|
209
|
+
The structs and the memory of a library are read with its types: `read(pointer, 'struct timeval')`,
|
|
210
|
+
`sizeOf('struct timeval')`, `offsetOf('struct timeval', 'tv_usec')`.
|
|
211
|
+
|
|
212
|
+
## 5. Callbacks
|
|
213
|
+
|
|
214
|
+
A parameter whose type is a pointer to a function takes a JS function: a **transient callback**, valid during the call
|
|
215
|
+
only (`qsort`, `sqlite3_exec`). An exception it throws makes the function receive 0 (or NULL), then is thrown by the
|
|
216
|
+
call once it returns.
|
|
217
|
+
|
|
218
|
+
A library that keeps a callback to call it later (a log handler, an event handler) gets a **persistent callback**:
|
|
219
|
+
|
|
220
|
+
```typescript
|
|
221
|
+
import {Library, Native, NativeLibrary, type NativeCallback} from '@lakutata/ffi'
|
|
222
|
+
|
|
223
|
+
@Library({
|
|
224
|
+
names: 'vendor',
|
|
225
|
+
declarations: 'typedef void (*vendor_log)(int level, const char *message);'
|
|
226
|
+
})
|
|
227
|
+
export class Vendor extends NativeLibrary {
|
|
228
|
+
@Native('void vendor_set_logger(vendor_log logger)')
|
|
229
|
+
public readonly setLogger: (logger: bigint | null) => void
|
|
230
|
+
|
|
231
|
+
#logger?: NativeCallback
|
|
232
|
+
|
|
233
|
+
public listen(): void {
|
|
234
|
+
this.#logger = this.callback('vendor_log', (level: number, message: string): void => console.log(level, message))
|
|
235
|
+
this.setLogger(this.#logger.pointer)
|
|
236
|
+
this.on('error', (error: unknown): void => console.error('the logger threw', error))
|
|
237
|
+
}
|
|
238
|
+
|
|
239
|
+
public stop(): void {
|
|
240
|
+
this.setLogger(null)
|
|
241
|
+
this.#logger?.release()
|
|
242
|
+
}
|
|
243
|
+
}
|
|
244
|
+
```
|
|
245
|
+
|
|
246
|
+
- `callback(type, fn)` takes the name of a pointer-to-function type of the declarations, or a prototype. Give its
|
|
247
|
+
`pointer` to the library, and `release()` it when the library no longer calls it; the callbacks of a component are
|
|
248
|
+
released when it is destroyed. A process has at most 8192 at a time.
|
|
249
|
+
- Called from another thread of the library, it runs in the JS thread once the event loop gets to it. A function that
|
|
250
|
+
waits for a thread calling back must be asynchronous: called synchronously, the JS thread would wait for the thread
|
|
251
|
+
that waits for it.
|
|
252
|
+
- An exception it throws is an `error` event of the component (written to the log of the application when nothing
|
|
253
|
+
listens to them), and the library gets 0 (or NULL) as its result.
|
|
254
|
+
|
|
255
|
+
## 6. Memory
|
|
256
|
+
|
|
257
|
+
| Function | What it gives |
|
|
258
|
+
|---|---|
|
|
259
|
+
| `alloc(type, count)`, `free(pointer)` | memory for values of a type of the library, which does not move and which the library can keep; freed when the component is destroyed if not before |
|
|
260
|
+
| `read(pointer, type, count?)`, `write(pointer, type, value)` | a value (or an array) of a type of the library |
|
|
261
|
+
| `readCString(pointer)`, `readWString(pointer)` | a string given as a pointer (a `void *` result, a member, an argument of a callback) |
|
|
262
|
+
| `readBuffer(pointer, length)` | a copy, valid once the native memory is freed (about 10 GB/s) |
|
|
263
|
+
| `viewBuffer(pointer, length)` | the memory itself, without copy |
|
|
264
|
+
| `readValue`, `readArray`, `writeValue`, `allocMemory`, `freeMemory`, `sizeOf` | the same for the built-in types (`int`, `uint64_t`, `char *`…), without a library |
|
|
265
|
+
|
|
266
|
+
A copy is as fast as a view below 64 KiB; a view is worth it for large blocks read often (8 MiB: about 0.13 ms
|
|
267
|
+
rather than 1 ms). A view must not be used once the native memory is freed: the process would crash. Electron forbids
|
|
268
|
+
views of native memory: `viewBuffer` throws a `NativeMemoryException` there.
|
|
269
|
+
|
|
270
|
+
## 7. The errors of the C functions
|
|
271
|
+
|
|
272
|
+
`lastErrno()` (or `errno()` of the component) gives the `errno` of the last call of the thread (for a library with
|
|
273
|
+
threads of its own, see section 10). On Windows, each C library has its own `errno`: the one of the C library of
|
|
274
|
+
Microsoft (`ucrtbase.dll`) is read by default, another one given by `crt` of `@Library` (`msvcrt.dll` for the old
|
|
275
|
+
libraries). `lastWindowsError()` gives the error of the Windows API (`GetLastError`), kept after each call.
|
|
276
|
+
|
|
277
|
+
```typescript
|
|
278
|
+
import {lastErrno, loadNativeLibrary} from '@lakutata/ffi'
|
|
279
|
+
|
|
280
|
+
const libc = loadNativeLibrary('c')
|
|
281
|
+
const open = libc.func<(path: string, flags: number) => number>('int open(const char *path, int flags, ...)')
|
|
282
|
+
if (open('/missing', 0) < 0) console.log('errno', lastErrno())
|
|
283
|
+
```
|
|
284
|
+
|
|
285
|
+
## 8. Without the container
|
|
286
|
+
|
|
287
|
+
Scripts and tests load a library with `loadNativeLibrary(names, declarations, options)`: the same declarations, the
|
|
288
|
+
same rules, the same methods as a component (`func`, `function`, `variable`, `callback`, `read`…), and `close()`.
|
|
289
|
+
|
|
290
|
+
```typescript
|
|
291
|
+
import {loadNativeLibrary} from '@lakutata/ffi'
|
|
292
|
+
|
|
293
|
+
const libc = loadNativeLibrary('c', 'typedef struct { int quot; int rem; } div_t;')
|
|
294
|
+
const div = libc.func<(numerator: number, denominator: number) => {quot: number, rem: number}>('div_t div(int numerator, int denominator)')
|
|
295
|
+
console.log(div(17, 5))
|
|
296
|
+
await libc.close()
|
|
297
|
+
```
|
|
298
|
+
|
|
299
|
+
## 9. Generating the class from the header
|
|
300
|
+
|
|
301
|
+
For a library with many functions, `lakutata ffi generate` writes its class from its header: its declarations, a
|
|
302
|
+
property by function with its TypeScript type, an interface by struct and union, its integer macros as constants.
|
|
303
|
+
|
|
304
|
+
```bash
|
|
305
|
+
lakutata ffi generate /usr/include/sqlite3.h --class-name SQLite3 --out src/native/SQLite3.ts
|
|
306
|
+
lakutata ffi generate sdk/include/vendor.h --library sdk/lib/vendor.dll --out src/native/Vendor.ts
|
|
307
|
+
```
|
|
308
|
+
|
|
309
|
+
- **The header** is run through the preprocessor of the C compiler (`cc -E`, `cl /P`): its `#include`, its `#if` and its
|
|
310
|
+
macros are resolved as for a build on this platform (`--include` and `--define` give the directories and the macros
|
|
311
|
+
of a build). On Windows, run it in the Developer Command Prompt of Visual Studio, which has `cl`; `--compiler` (or
|
|
312
|
+
`CC`) chooses another compiler.
|
|
313
|
+
- **What is kept**: the declarations of the header, and those of the headers of its directory unless it is a directory
|
|
314
|
+
of the system (`--own` adds the directories of the other headers of the library), with the types of the system they
|
|
315
|
+
use. A declaration the package cannot read (`long double`, a macro with arguments) is left out, and listed with its
|
|
316
|
+
file and its line.
|
|
317
|
+
- **Checked against the library**: the library named as the header (`sqlite3.h`: `sqlite3`), or the one given by
|
|
318
|
+
`--library` (a name or a file), is loaded, and the functions it does not export or that cannot be called (a
|
|
319
|
+
`va_list` parameter) are left out and listed. When it cannot be loaded, the class is generated without the check, with
|
|
320
|
+
a warning (zlib needs `--library z`); `--no-check` skips the check.
|
|
321
|
+
- **The class** gets a property by function, named as the function, with its C prototype above it; a function named as
|
|
322
|
+
a member of the component (`free`, `read`) gets another property name. The TypeScript types follow section 3:
|
|
323
|
+
64-bit integers are bigints, strings are `string | null`, other pointers `Pointer | null` (a parameter also takes a
|
|
324
|
+
`Buffer` or a typed array).
|
|
325
|
+
- **What to edit**: a header does not say which pointers are output parameters, nor which functions block, and the
|
|
326
|
+
generated file is meant to be edited. Mark an output parameter `_Out_` in the prototype of its function, in the
|
|
327
|
+
declarations, and change the type of its parameter to `Out<T>` (to the interface of the struct, for a pointer to a
|
|
328
|
+
struct). A function that blocks gets `{async: true}` in `@Native`, which then names the function. For a new version
|
|
329
|
+
of the header, generate the class again and carry those changes over.
|
|
330
|
+
- **For one platform**: the declarations are the ones of the platform the generator ran on; a header whose declarations
|
|
331
|
+
differ by platform (`#ifdef _WIN32`) is generated on each of them. Another build of the library may lack some of the
|
|
332
|
+
functions: the component then refuses to start, naming the missing function. On Linux, the library is found through
|
|
333
|
+
the link its development package installs (`libsqlite3.so`): for the machines without that package, give `names`
|
|
334
|
+
the file with its version (`libsqlite3.so.0`, section 2).
|
|
335
|
+
|
|
336
|
+
What the generator wrote for `sqlite3_open` and `sqlite3_exec`, edited:
|
|
337
|
+
|
|
338
|
+
```typescript
|
|
339
|
+
import {Library, Native, NativeLibrary, type Out, type Pointer} from '@lakutata/ffi'
|
|
340
|
+
|
|
341
|
+
@Library({
|
|
342
|
+
names: 'sqlite3',
|
|
343
|
+
declarations: `
|
|
344
|
+
typedef struct sqlite3 sqlite3;
|
|
345
|
+
int sqlite3_open(const char *filename, _Out_ sqlite3 **ppDb);
|
|
346
|
+
int sqlite3_exec(sqlite3 *, const char *sql, int (*callback)(void *, int, char **, char **), void *, _Out_ char **errmsg);
|
|
347
|
+
`
|
|
348
|
+
})
|
|
349
|
+
export class SQLite3 extends NativeLibrary {
|
|
350
|
+
//Generated with ppDb: Pointer | Buffer | ArrayBufferView | null; ppDb marked _Out_ in the declarations
|
|
351
|
+
@Native()
|
|
352
|
+
public readonly sqlite3_open: (filename: string | null, ppDb: Out<Pointer>) => number
|
|
353
|
+
|
|
354
|
+
//It blocks: asynchronous; errmsg marked _Out_, the message freed by sqlite3_free
|
|
355
|
+
@Native('sqlite3_exec', {async: true, free: 'sqlite3_free'})
|
|
356
|
+
public readonly sqlite3_exec: (db: Pointer | null, sql: string | null, callback: null, context: null, errmsg: Out<string>) => Promise<number>
|
|
357
|
+
}
|
|
358
|
+
```
|
|
359
|
+
|
|
360
|
+
The generator can also be called from a script: `generateNativeLibrary()` of `@lakutata/ffi/generator` returns the
|
|
361
|
+
code of the class, its declarations and the report.
|
|
362
|
+
|
|
363
|
+
## 10. The threads of a library
|
|
364
|
+
|
|
365
|
+
By default, the asynchronous functions run in the thread pool of the runtime. With `threads`, a library has worker
|
|
366
|
+
threads of its own instead: each one loads the library, and the asynchronous functions run in them.
|
|
367
|
+
|
|
368
|
+
```typescript
|
|
369
|
+
import {Library, Native, NativeLibrary} from '@lakutata/ffi'
|
|
370
|
+
|
|
371
|
+
@Library({names: 'vendor', threads: 1})
|
|
372
|
+
export class Vendor extends NativeLibrary {
|
|
373
|
+
@Native('int vendor_read(int device, unsigned char *buffer, size_t size)', {async: true})
|
|
374
|
+
public readonly read: (device: number, buffer: Buffer, size: number) => Promise<number>
|
|
375
|
+
|
|
376
|
+
@Native('int vendor_version(void)')
|
|
377
|
+
public readonly version: () => number
|
|
378
|
+
}
|
|
379
|
+
```
|
|
380
|
+
|
|
381
|
+
- **The number of threads**: with one thread, the asynchronous calls run one after the other, in the order they were
|
|
382
|
+
made: for a library that is not thread-safe, or a device that serves one request at a time. With several, each call
|
|
383
|
+
goes to the least busy thread. The configuration can give another number
|
|
384
|
+
(`buildNativeLibraryOptions(Vendor, {threads: 2})`), `loadNativeLibrary` takes it in its options, and `0` keeps the
|
|
385
|
+
thread pool.
|
|
386
|
+
- **What runs there**: the asynchronous functions only. The synchronous ones still run in the thread calling them, so
|
|
387
|
+
the library is used by two threads at once: for a library that is not thread-safe, declare all its functions
|
|
388
|
+
asynchronous. A variadic function can be asynchronous here. The threads of a library are not taken from the thread
|
|
389
|
+
pool: long calls do not hold up the file system, and `UV_THREADPOOL_SIZE` does not matter for them.
|
|
390
|
+
- **The arguments are copied** to the thread. The ones the function writes are copied back once it returns: the
|
|
391
|
+
`_Out_` and `_Inout_` parameters, and the buffers and typed arrays given for a pointer that is not `const`. A
|
|
392
|
+
buffer is copied to the thread at each call, and back when the function may write it: for large buffers, the thread
|
|
393
|
+
pool costs less. Pointers are passed as they are: the memory of `alloc()` is shared by all the threads.
|
|
394
|
+
- **Callbacks**: a JS function given for a pointer to a function runs in the thread that made the call, while the
|
|
395
|
+
thread of the library waits for it; an exception it throws rejects the promise. Persistent callbacks work as in
|
|
396
|
+
section 5.
|
|
397
|
+
- **Cost**: about 11 µs for a call awaited before the next one, against 7 µs in the thread pool (Node.js on Apple
|
|
398
|
+
silicon).
|
|
399
|
+
- **errno**: `errno()` and `windowsError()` of the library give the values of the thread of the last asynchronous call
|
|
400
|
+
completed. They are reliable with one thread, or when each call is awaited before the next. From the first
|
|
401
|
+
asynchronous call completed on, they no longer give the values of the synchronous calls: `lastErrno()` and
|
|
402
|
+
`lastWindowsError()` read those (section 7).
|
|
403
|
+
- **Starting and closing**: the threads start with the library and take a few milliseconds to load it. A component is
|
|
404
|
+
created once they have loaded it; for a library loaded by `loadNativeLibrary`, `ready` settles then (the calls wait
|
|
405
|
+
for it anyway). When the library is closed, the calls running are awaited, then the threads unload the library and
|
|
406
|
+
end. A thread that ends unexpectedly fails the calls it was running, and the next call starts a new one.
|
|
407
|
+
|
|
408
|
+
## 11. Destruction and worker threads
|
|
409
|
+
|
|
410
|
+
When the application stops, a library component makes its properties throw `NativeLibraryClosedException` (its
|
|
411
|
+
asynchronous functions reject), waits for its asynchronous calls running (`drainTimeout`, 10 seconds by default),
|
|
412
|
+
releases its callbacks, frees the memory of `alloc()` and unloads the library (`keepLoaded` keeps it loaded; musl never
|
|
413
|
+
unloads). A function copied out of its property (`const abs = libc.abs`) must not be called after that: a function of an
|
|
414
|
+
unloaded library ends the process.
|
|
415
|
+
|
|
416
|
+
A native call cannot be interrupted. When calls are still running after `drainTimeout`, the library, its callbacks and
|
|
417
|
+
its memory are left as they are, so that those calls can return safely. The calls in the thread pool go on, and settle
|
|
418
|
+
when they return. The ones in the threads of the library are rejected with `NativeLibraryClosedException`, and each of
|
|
419
|
+
those threads ends once its call returns. Node.js waits for such calls to return even on `process.exit()`;
|
|
420
|
+
`process.exit()` in Bun does not wait for them.
|
|
421
|
+
|
|
422
|
+
A worker thread loads the library for itself (a component created in the worker). Stop the application running in the
|
|
423
|
+
worker before stopping the worker: the application waits for the calls and closes its libraries. Stopping a worker
|
|
424
|
+
while it uses koffi can end the whole process:
|
|
425
|
+
|
|
426
|
+
- while koffi loads in the worker, in Node.js and in Bun: when the worker imports the package, or while the threads of
|
|
427
|
+
a library created in it start (a few milliseconds). koffi 3.3.4 does not survive being stopped while it loads.
|
|
428
|
+
Exiting the process while the threads of a library start can do the same;
|
|
429
|
+
- in Bun, also while asynchronous calls run in the thread pool (a bug of Bun, in 1.4.4 as well), and while the threads
|
|
430
|
+
of a library bind its functions.
|
|
431
|
+
|
|
432
|
+
Stopping a worker while asynchronous calls run in the threads of its libraries does not crash the process.
|
|
433
|
+
|
|
434
|
+
## 12. Runtimes
|
|
435
|
+
|
|
436
|
+
Node.js `^22.13.0 || >=24.11.0`, the main process of Electron and Bun; not the renderers of Electron nor the browsers.
|
|
437
|
+
|
|
438
|
+
The tests of the package, including those of the threads of a library, pass on macOS (arm64: Node.js 24, Bun 1.4,
|
|
439
|
+
Electron 44), on Linux (arm64, glibc and musl: Node.js 24, Bun 1.4) and on Windows 11 (arm64, and x64 under emulation:
|
|
440
|
+
Node.js 24, Bun 1.4). On macOS, an application whose libraries run in threads of their own was also packed by
|
|
441
|
+
`lakutata package` into a bundle, executables of Node.js and of Bun, and an Electron application: each one ran, and the
|
|
442
|
+
threads loaded the library shipped in its resources.
|
|
443
|
+
|
|
444
|
+
The generator of section 9 was run with clang on macOS, gcc on Linux (glibc and musl) and cl on Windows 11 (arm64 and
|
|
445
|
+
x64). The classes it generated from the headers of sqlite3, zlib, pcap and curl on macOS and Linux, and from
|
|
446
|
+
winsqlite3.h and the header of a library built for the test on Windows, compiled under the strict mode of TypeScript
|
|
447
|
+
and ran in Node.js and Bun.
|
|
448
|
+
|
|
449
|
+
## 13. Exceptions
|
|
450
|
+
|
|
451
|
+
| Exception | When |
|
|
452
|
+
|---|---|
|
|
453
|
+
| `NativeDeclarationException` | Declarations or a prototype cannot be read (with the line and the column), a function is not in the library, a type cannot be passed (`va_list`), a class has no `@Library` |
|
|
454
|
+
| `NativeLibraryNotFoundException` | The library cannot be loaded: each file tried and why |
|
|
455
|
+
| `NativeLibraryClosedException` | A function, a variable or a callback of a library used once it is closed; a call still running in the threads of a library when the library is closed |
|
|
456
|
+
| `NativeMemoryException` | A null pointer, a negative length, a view in Electron |
|
|
457
|
+
| `TypeError` (from koffi) | An argument of the wrong type |
|
|
458
|
+
| `RangeError` | A 64-bit integer that is not a safe integer, for a function declared with `{int64: 'number'}` |
|
package/doc/en/NativeAddons.md
CHANGED
|
@@ -3,7 +3,8 @@
|
|
|
3
3
|
A native addon is a library written in C++ or Rust that Node.js loads as a `.node` file: for drivers and devices,
|
|
4
4
|
codecs, system APIs, or code that must run fast. `lakutata add --native` creates an addon package in a workspace,
|
|
5
5
|
ready to build and test; `lakutata native` links libraries to it; and `lakutata package` ships it with the application.
|
|
6
|
-
The addons use Node-API, so one build runs in all the supported versions of Node.js and in Electron.
|
|
6
|
+
The addons use Node-API, so one build runs in all the supported versions of Node.js and in Electron. To call the C
|
|
7
|
+
functions of a library without writing an addon, see [FFI](FFI.md).
|
|
7
8
|
|
|
8
9
|
## 1. Creating an addon
|
|
9
10
|
|
package/doc/en/Overview.md
CHANGED
|
@@ -47,7 +47,8 @@ tool, a web and a desktop application, calls between services, background jobs,
|
|
|
47
47
|
Not part of this package: `@lakutata/cli` (the `lakutata` command for creating projects, running migrations and packing
|
|
48
48
|
applications, see [Command](Command.md)), `@lakutata/grpc` (a gRPC client, server and service transport),
|
|
49
49
|
`@lakutata/nats` (a NATS component and service transport), `@lakutata/dbus` (D-Bus), `@lakutata/mcp` (the Model
|
|
50
|
-
Context Protocol)
|
|
50
|
+
Context Protocol), `@lakutata/webrtc` (WebRTC: the signaling, a TURN server, a media server, peer connections) and
|
|
51
|
+
`@lakutata/ffi` (the C functions of native libraries).
|
|
51
52
|
|
|
52
53
|
## The guides
|
|
53
54
|
|
|
@@ -91,6 +92,7 @@ Context Protocol) and `@lakutata/webrtc` (WebRTC: the signaling, a TURN server,
|
|
|
91
92
|
| [DBus](DBus.md) | A D-Bus service and client (`@lakutata/dbus`): methods, objects, properties, signals |
|
|
92
93
|
| [MCP](MCP.md) | The tools, resources and prompts of the Model Context Protocol (`@lakutata/mcp`), over stdio and HTTP |
|
|
93
94
|
| [WebRTC](WebRTC.md) | Peer-to-peer connections and a media server (`@lakutata/webrtc`): the signaling, the TURN server, browsers and applications, the actions of an application called through data channels, WHIP and WHEP |
|
|
95
|
+
| [FFI](FFI.md) | The C functions of native libraries (`@lakutata/ffi`): declarations copied from headers, structs, callbacks, memory, packaging |
|
|
94
96
|
| [Command](Command.md) | The `lakutata` command: creating projects, migrations, translations, environment variables |
|
|
95
97
|
| [Workspaces](Workspaces.md) | Projects of several packages: running their scripts, giving them a version, adding packages |
|
|
96
98
|
| [NativeAddons](NativeAddons.md) | Native addons in C++ or Rust: creating them, linking libraries, shipping them |
|
package/doc/en/UpgradingFrom2.md
CHANGED
|
@@ -28,8 +28,10 @@ that the project declares and the one installed. There is no upgrade command: th
|
|
|
28
28
|
matrix and the base images of the Dockerfiles.
|
|
29
29
|
- **A TypeScript that knows `"module": "Node20"`** (the templates use TypeScript 6): `lakutata/tsconfig.json` now sets
|
|
30
30
|
it (2.x: `Node16`), and older compilers reject it.
|
|
31
|
-
- **`package.json`**: `"type": "module"`, `lakutata` `^3.0.0`,
|
|
32
|
-
|
|
31
|
+
- **`package.json`**: `"type": "module"`, `lakutata` `^3.0.0`, the `@lakutata/*` packages at `^3.0.0` too, and
|
|
32
|
+
`@lakutata/cli` in `devDependencies`. Do not give the other packages the version of `lakutata`: a patch release
|
|
33
|
+
publishes only the packages it changes, so their latest versions can differ (`lakutata` 3.1.4 with
|
|
34
|
+
`@lakutata/ffi` 3.1.2).
|
|
33
35
|
- **A clean install**: remove `node_modules` and the lock file, then install. Upgrading in place can leave several
|
|
34
36
|
copies of `@lakutata/core`, whose classes and decorator metadata do not match (`instanceof` and injection fail across
|
|
35
37
|
them). `npm ls @lakutata/core` must print one version.
|
package/doc/zh/Command.md
CHANGED
|
@@ -55,6 +55,8 @@ lakutata create --name tool --template cli --package-manager none --no-git
|
|
|
55
55
|
|
|
56
56
|
`lakutata add <name> --native gyp|cmake|rust` 添加一个用 C++ 或 Rust 编写的原生插件,`lakutata native <link|include|define> <value>` 给它链接库。见 [NativeAddons](NativeAddons.md)。
|
|
57
57
|
|
|
58
|
+
`lakutata ffi generate <header>` 根据 C 头文件生成 `@lakutata/ffi` 原生库的类。见 [FFI](FFI.md) 第 9 节。
|
|
59
|
+
|
|
58
60
|
## 4. 迁移与翻译
|
|
59
61
|
|
|
60
62
|
`lakutata migration <action>` 生成、创建、执行、回滚和查看应用的数据库迁移。它用应用的 `dev` 脚本构建并启动应用,此时应用只连接数据库。见 [Migrations](Migrations.md) 第 2 节。
|
package/doc/zh/FFI.md
ADDED
|
@@ -0,0 +1,337 @@
|
|
|
1
|
+
# Lakutata FFI
|
|
2
|
+
|
|
3
|
+
`@lakutata/ffi` 包让 Node.js、Electron 的主进程和 Bun 调用动态库(`.so`、`.dylib`、`.dll`)里的 C 函数:系统 API、设备厂商的 SDK、现成的 C 库。不需要编译,也不需要写扩展:用库的头文件里的 C 声明来描述它,每个库都是应用的一个组件。它基于 [koffi](https://koffi.dev/)。它不属于 `lakutata`:
|
|
4
|
+
|
|
5
|
+
```bash
|
|
6
|
+
npm install lakutata @lakutata/ffi
|
|
7
|
+
```
|
|
8
|
+
|
|
9
|
+
koffi 会连同各平台的预编译文件一起安装(每个平台一个包)。npm 可能会提示 koffi 的安装脚本没有运行:不需要运行它。
|
|
10
|
+
|
|
11
|
+
C 函数的原型写错,或者传给它不该传的指针,整个进程都会退出,任何异常处理都拦不住。请从库的头文件里复制声明,并且测试它们。
|
|
12
|
+
|
|
13
|
+
## 1. 一个库
|
|
14
|
+
|
|
15
|
+
一个库就是 `NativeLibrary` 的一个子类:`@Library` 给出它的文件名和 C 声明,`@Native` 把一个函数绑到一个属性上,`@NativeVariable` 绑定一个变量。每个属性的 TypeScript 类型就是函数的类型。
|
|
16
|
+
|
|
17
|
+
```typescript
|
|
18
|
+
import {Application, Component} from 'lakutata'
|
|
19
|
+
import {Inject} from 'lakutata/decorator/di'
|
|
20
|
+
import {buildNativeLibraryOptions, Library, Native, NativeLibrary, NativeVariable, type Out, type Pointer, readArray} from '@lakutata/ffi'
|
|
21
|
+
|
|
22
|
+
type RowCallback = (context: Pointer | null, count: number, values: Pointer, names: Pointer) => number
|
|
23
|
+
|
|
24
|
+
@Library({
|
|
25
|
+
names: {darwin: 'libsqlite3.dylib', linux: 'libsqlite3.so.0', win32: 'winsqlite3.dll'},
|
|
26
|
+
declarations: `
|
|
27
|
+
typedef struct sqlite3 sqlite3;
|
|
28
|
+
typedef int (*sqlite3_callback)(void *context, int count, char **values, char **names);
|
|
29
|
+
#define SQLITE_OK 0
|
|
30
|
+
int sqlite3_close(sqlite3 *db);
|
|
31
|
+
`
|
|
32
|
+
})
|
|
33
|
+
export class SQLite extends NativeLibrary {
|
|
34
|
+
@Native('int sqlite3_open(const char *filename, _Out_ sqlite3 **db)')
|
|
35
|
+
public readonly open: (filename: string, db: Out<Pointer>) => number
|
|
36
|
+
|
|
37
|
+
//在线程池里执行;它给出的错误信息由 sqlite3_free 释放
|
|
38
|
+
@Native('int sqlite3_exec(sqlite3 *db, const char *sql, sqlite3_callback callback, void *context, _Out_ char **error)', {async: true, free: 'sqlite3_free'})
|
|
39
|
+
public readonly exec: (db: Pointer, sql: string, callback: RowCallback | null, context: null, error: Out<string>) => Promise<number>
|
|
40
|
+
|
|
41
|
+
//声明里和属性同名的函数
|
|
42
|
+
@Native()
|
|
43
|
+
public readonly sqlite3_close: (db: Pointer) => number
|
|
44
|
+
|
|
45
|
+
@NativeVariable('extern const char sqlite3_version[]')
|
|
46
|
+
public readonly version: string
|
|
47
|
+
}
|
|
48
|
+
|
|
49
|
+
class Store extends Component {
|
|
50
|
+
@Inject('sqlite')
|
|
51
|
+
protected readonly sqlite: SQLite
|
|
52
|
+
|
|
53
|
+
public async tables(): Promise<string[]> {
|
|
54
|
+
const db: Out<Pointer> = [null]
|
|
55
|
+
if (this.sqlite.open(':memory:', db) !== Number(this.sqlite.constants.SQLITE_OK)) throw new Error('open failed')
|
|
56
|
+
const names: string[] = []
|
|
57
|
+
const error: Out<string> = [null]
|
|
58
|
+
await this.sqlite.exec(db[0]!, 'SELECT name FROM sqlite_master', (context: Pointer | null, count: number, values: Pointer): number => {
|
|
59
|
+
names.push(...readArray(values, 'char *', count) as string[])
|
|
60
|
+
return 0
|
|
61
|
+
}, null, error)
|
|
62
|
+
this.sqlite.sqlite3_close(db[0]!)
|
|
63
|
+
return names
|
|
64
|
+
}
|
|
65
|
+
}
|
|
66
|
+
|
|
67
|
+
Application.run(() => ({
|
|
68
|
+
id: 'store.app',
|
|
69
|
+
name: 'Store',
|
|
70
|
+
components: {
|
|
71
|
+
sqlite: buildNativeLibraryOptions(SQLite),
|
|
72
|
+
store: {class: Store}
|
|
73
|
+
}
|
|
74
|
+
}))
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
- **文件名**:可以写一个名字(`sqlite3` 会按各平台给库命名的方式补全:`libsqlite3.dylib`、`libsqlite3.so`、`sqlite3.dll`),可以写一个文件,也可以按平台分别写(键是 `process.platform` 的值,`default` 用于其他平台)。`c` 表示 C 标准库,`m` 表示它的数学函数,包会到各系统存放它们的地方去找:macOS 的 `libSystem`、glibc 的 `libc.so.6`、Alpine 上 musl 的动态加载器、Windows 的 `ucrtbase.dll`。
|
|
78
|
+
- **声明**按头文件的写法写,可以直接从头文件里复制:typedef、结构体和联合(可以后定义、可以引用自身、可以有匿名成员)、枚举(值可以是常量表达式)、函数指针、原型、变量、不带参数的 `#define`(空的宏在使用处会被去掉,比如导出函数的标记;值是整数的宏同时也是常量)、`#pragma pack`、`packed` 和 `aligned` 属性。其他属性、inline 函数的函数体和 `extern "C"` 块会被跳过。每个库的声明都是独立的:两个库可以各自声明 `HANDLE`。
|
|
79
|
+
- **不支持的写法**会报错,并给出行号和列号:`#if` 一类条件编译(请先用预处理器展开头文件:`cc -E`、`cl /P`)、带参数的宏、`long double`、`__int128`,以及旧式声明。`stdint.h`、POSIX(`time_t`、`pid_t` 等)、BSD(`u_int` 等)和 Windows API(`DWORD`、`HANDLE`、`LPCWSTR` 等)的类型不用声明就能用,大小按各平台的实际情况。
|
|
80
|
+
- **`@Native`** 的参数可以是原型、声明里某个函数的名字,或者省略(用和属性同名的函数)。组件创建时,属性就会得到这个函数。不需要做任何转换时,它就是 koffi 的函数本身,调用的开销和直接用 koffi 一样(约 20 ns)。
|
|
81
|
+
- **`@NativeVariable`** 的参数可以是声明、名字,或者省略:读属性就是读变量,写属性就是写变量(`const` 变量拒绝写入)。长度未知的 `char` 数组按字符串读取。
|
|
82
|
+
- 子类会继承父类的库,也可以再加函数。
|
|
83
|
+
- 函数很多的库,可以根据它的头文件生成类(第 9 节)。
|
|
84
|
+
|
|
85
|
+
## 2. 到哪里找库
|
|
86
|
+
|
|
87
|
+
组件创建时,按以下顺序找库:
|
|
88
|
+
|
|
89
|
+
1. 配置里给了 `path` 时,只试这一个文件;
|
|
90
|
+
2. 配置里 `search` 的目录,然后是 `@Library` 里的目录;
|
|
91
|
+
3. 打包后的应用里,资源目录下的 `native` 目录;
|
|
92
|
+
4. 系统找库的位置(`dlopen`、`LoadLibrary`)。
|
|
93
|
+
|
|
94
|
+
```typescript
|
|
95
|
+
import {Application} from 'lakutata'
|
|
96
|
+
import {buildNativeLibraryOptions, Library, Native, NativeLibrary} from '@lakutata/ffi'
|
|
97
|
+
|
|
98
|
+
@Library({names: {darwin: 'libvendor.dylib', linux: 'libvendor.so.2', win32: 'vendor.dll'}})
|
|
99
|
+
class Vendor extends NativeLibrary {
|
|
100
|
+
@Native('int vendor_version(void)')
|
|
101
|
+
public readonly version: () => number
|
|
102
|
+
}
|
|
103
|
+
|
|
104
|
+
Application.run(() => ({
|
|
105
|
+
id: 'devices.app',
|
|
106
|
+
name: 'Devices',
|
|
107
|
+
components: {
|
|
108
|
+
vendor: buildNativeLibraryOptions(Vendor, {search: ['./native']})
|
|
109
|
+
}
|
|
110
|
+
}))
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
加载不了时,`NativeLibraryNotFoundException` 会列出试过的每个文件,以及没能加载的原因(找不到、为别的架构编译的、缺少它依赖的库)。Linux 上系统自带的库常常只装了带版本号的文件(`libsqlite3.so.0`),这时请写这个名字。
|
|
114
|
+
|
|
115
|
+
要把库和打包后的应用一起发布,把它放进 package.json 的资源里,放到 `native` 目录:
|
|
116
|
+
|
|
117
|
+
```json
|
|
118
|
+
{
|
|
119
|
+
"lakutata": {
|
|
120
|
+
"resources": [{"from": "vendor/lib", "to": "native"}]
|
|
121
|
+
}
|
|
122
|
+
}
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
`lakutata package` 打出的 bundle、可执行文件(Node.js 的和 Bun 的)和 Electron 安装包都会带上它,也会带上 koffi 和本平台的预编译文件。在 Alpine 上请用 Bun 打可执行文件(`--runtime bun`):Node.js 的可执行文件在 musl 上加载不了原生插件。在 Node.js 的权限模型下运行时,要加 `--allow-addons`。
|
|
126
|
+
|
|
127
|
+
## 3. 调用函数
|
|
128
|
+
|
|
129
|
+
| C | JavaScript |
|
|
130
|
+
|---|---|
|
|
131
|
+
| 整数、`float`、`double`、`bool`、枚举 | number、boolean |
|
|
132
|
+
| 64 位整数(`int64_t`、`long long`、`size_t`、`long`、`time_t` 等) | bigint(作为参数时也接受 number) |
|
|
133
|
+
| `const char *`、`char *` | 字符串(也接受 `Buffer`,函数会直接写进去);`char16_t *`、`wchar_t *` 也一样 |
|
|
134
|
+
| 其他指针 | bigint(`Pointer`),空指针是 `null`;可以传 `Buffer` 或类型化数组 |
|
|
135
|
+
| 结构体、联合 | 对象 |
|
|
136
|
+
| 函数指针 | 函数(见第 5 节) |
|
|
137
|
+
|
|
138
|
+
- **64 位整数**在任何地方都是 bigint:返回值、输出参数、结构体成员、回调参数。`long` 在 Linux 和 macOS 上是 64 位,在 Windows 上是 32 位:它在所有平台上都是 bigint。声明函数时加上 `{int64: 'number'}`,就改为给出 number;值超出安全整数范围时抛出 `RangeError`。
|
|
139
|
+
- **输出参数**:标了 `_Out_`(或 `_Inout_`)的参数由函数写入。整数和指针用只有一个元素的数组传入(`Out<T>`,调用前是 `[null]`);结构体用对象传入,函数返回后,对象的成员会被填上。没有标记的指针只把值传给函数:通过它传入的结构体,调用后不会读回来。
|
|
140
|
+
- **传给函数的缓冲区**只在这次调用期间有效:库要保存起来以后再用的内存,要用 `alloc()` 分配(第 6 节)。
|
|
141
|
+
- **可变参数的函数**(`printf`、`curl_easy_setopt`)的每个额外参数,前面都要用 C 类型名写出它的类型:
|
|
142
|
+
|
|
143
|
+
```typescript
|
|
144
|
+
import {loadNativeLibrary} from '@lakutata/ffi'
|
|
145
|
+
|
|
146
|
+
const libc = loadNativeLibrary('c')
|
|
147
|
+
const snprintf = libc.func<(buffer: Buffer, size: number, format: string, ...args: unknown[]) => number>('int snprintf(char *buffer, size_t size, const char *format, ...)')
|
|
148
|
+
const buffer: Buffer = Buffer.alloc(64)
|
|
149
|
+
snprintf(buffer, buffer.length, '%s: %d', 'const char *', 'count', 'int', 42)
|
|
150
|
+
```
|
|
151
|
+
|
|
152
|
+
- **需要释放的字符串**:函数返回(或通过输出参数给出)一个要由调用方释放的字符串时,声明时加上 `{free: 'sqlite3_free'}`,写库里负责释放的函数;或者加上 `{free: true}`,用 C 标准库的 `free()`。字符串读出来以后就会被释放。在 Windows 上,如果库是用另一套 C 运行库编译的,必须给出它自己的释放函数。
|
|
153
|
+
- **异步调用**:声明时加上 `{async: true}` 的函数在另一个线程里执行,返回一个 Promise;调用期间事件循环照常运行。默认用的是运行时的线程池:Node.js 用的是 libuv 的线程池,默认只有 4 个线程,而且和文件系统、DNS、加解密共用;调用多或者调用时间长时,请调大 `UV_THREADPOOL_SIZE`。在线程池里,可变参数的函数不能异步调用。库也可以改用自己的线程(第 10 节)。
|
|
154
|
+
|
|
155
|
+
## 4. 结构体、联合、枚举和常量
|
|
156
|
+
|
|
157
|
+
结构体是一个由它的成员组成的对象。嵌套的结构体也是对象;`char` 数组是字符串,数字数组是类型化数组(`Uint8Array`、`Int32Array` 等),64 位整数的数组是由 bigint 组成的数组。结构体里的匿名结构体或匿名联合是成员 `_0`(依次是 `_1` 等)。
|
|
158
|
+
|
|
159
|
+
- **位域**和其他成员一样,按名字读写:包会按平台上编译器的规则来排布它们(GCC 和 Clang 的规则、ARM 的规则、微软的规则)。64 位类型的位域是 bigint。
|
|
160
|
+
- **紧凑的结构体**(`#pragma pack`、`packed` 属性)和 `aligned` 属性(1、2、4 或 8 字节)都按编译器的方式排布。
|
|
161
|
+
- **联合**用一个只设置了其中一个成员的对象传入(`{i: 42}`)。
|
|
162
|
+
- **枚举**是整数;声明里枚举的值、`static const` 常量和值是整数的宏都在 `constants` 里,类型是 bigint(`Number(sqlite.constants.SQLITE_OK)`)。
|
|
163
|
+
|
|
164
|
+
库的结构体和内存,要用库自己的类型来读:`read(pointer, 'struct timeval')`、`sizeOf('struct timeval')`、`offsetOf('struct timeval', 'tv_usec')`。
|
|
165
|
+
|
|
166
|
+
## 5. 回调
|
|
167
|
+
|
|
168
|
+
参数类型是函数指针时,可以直接传一个 JS 函数,这就是**临时回调**,只在这次调用期间有效(`qsort`、`sqlite3_exec`)。它抛出的异常会让函数收到 0(或空指针),等这次调用返回后再抛出来。
|
|
169
|
+
|
|
170
|
+
库要把回调保存起来以后再调用时(日志处理函数、事件处理函数),要给它一个**持久回调**:
|
|
171
|
+
|
|
172
|
+
```typescript
|
|
173
|
+
import {Library, Native, NativeLibrary, type NativeCallback} from '@lakutata/ffi'
|
|
174
|
+
|
|
175
|
+
@Library({
|
|
176
|
+
names: 'vendor',
|
|
177
|
+
declarations: 'typedef void (*vendor_log)(int level, const char *message);'
|
|
178
|
+
})
|
|
179
|
+
export class Vendor extends NativeLibrary {
|
|
180
|
+
@Native('void vendor_set_logger(vendor_log logger)')
|
|
181
|
+
public readonly setLogger: (logger: bigint | null) => void
|
|
182
|
+
|
|
183
|
+
#logger?: NativeCallback
|
|
184
|
+
|
|
185
|
+
public listen(): void {
|
|
186
|
+
this.#logger = this.callback('vendor_log', (level: number, message: string): void => console.log(level, message))
|
|
187
|
+
this.setLogger(this.#logger.pointer)
|
|
188
|
+
this.on('error', (error: unknown): void => console.error('the logger threw', error))
|
|
189
|
+
}
|
|
190
|
+
|
|
191
|
+
public stop(): void {
|
|
192
|
+
this.setLogger(null)
|
|
193
|
+
this.#logger?.release()
|
|
194
|
+
}
|
|
195
|
+
}
|
|
196
|
+
```
|
|
197
|
+
|
|
198
|
+
- `callback(type, fn)` 的第一个参数是声明里某个函数指针类型的名字,或者一个原型。把它的 `pointer` 交给库;库不再调用它时,调用 `release()` 释放。组件销毁时,它的回调会全部释放。一个进程同时最多有 8192 个持久回调。
|
|
199
|
+
- 库从别的线程调用它时,它会在事件循环轮到它时,在 JS 线程里执行。如果一个函数会等某个线程回调,就必须把它声明为异步:同步调用的话,JS 线程在等那个线程,而那个线程又在等 JS 线程。
|
|
200
|
+
- 它抛出的异常会成为组件的 `error` 事件(没人监听时写进应用日志),库收到的结果是 0(或空指针)。
|
|
201
|
+
|
|
202
|
+
## 6. 内存
|
|
203
|
+
|
|
204
|
+
| 函数 | 得到什么 |
|
|
205
|
+
|---|---|
|
|
206
|
+
| `alloc(type, count)`、`free(pointer)` | 为库的某个类型分配内存,地址不会变,库可以保存它;没释放的,组件销毁时会释放 |
|
|
207
|
+
| `read(pointer, type, count?)`、`write(pointer, type, value)` | 按库的类型读写一个值(或一个数组) |
|
|
208
|
+
| `readCString(pointer)`、`readWString(pointer)` | 读一个以指针形式给出的字符串(`void *` 返回值、结构体成员、回调参数) |
|
|
209
|
+
| `readBuffer(pointer, length)` | 一份副本,原生内存释放以后也能用(约 10 GB/s) |
|
|
210
|
+
| `viewBuffer(pointer, length)` | 原生内存本身,不复制 |
|
|
211
|
+
| `readValue`、`readArray`、`writeValue`、`allocMemory`、`freeMemory`、`sizeOf` | 内置类型(`int`、`uint64_t`、`char *` 等)的对应操作,不需要库 |
|
|
212
|
+
|
|
213
|
+
64 KiB 以下,复制和不复制一样快;数据块很大、读得又频繁时才值得用视图(8 MiB:约 0.13 ms,复制要约 1 ms)。原生内存释放以后,绝不能再使用视图,否则进程会崩溃。Electron 不允许原生内存的视图:在那里调用 `viewBuffer` 会抛出 `NativeMemoryException`。
|
|
214
|
+
|
|
215
|
+
## 7. C 函数的错误
|
|
216
|
+
|
|
217
|
+
`lastErrno()`(或组件的 `errno()`)给出本线程上一次调用之后的 `errno`(库有自己的线程时,见第 10 节)。在 Windows 上,每个 C 运行库各有一份 `errno`:默认读微软 C 运行库(`ucrtbase.dll`)的,其他运行库在 `@Library` 的 `crt` 里指定(旧的库是 `msvcrt.dll`)。`lastWindowsError()` 给出 Windows API 的错误(`GetLastError`),每次调用之后都会保留下来。
|
|
218
|
+
|
|
219
|
+
```typescript
|
|
220
|
+
import {lastErrno, loadNativeLibrary} from '@lakutata/ffi'
|
|
221
|
+
|
|
222
|
+
const libc = loadNativeLibrary('c')
|
|
223
|
+
const open = libc.func<(path: string, flags: number) => number>('int open(const char *path, int flags, ...)')
|
|
224
|
+
if (open('/missing', 0) < 0) console.log('errno', lastErrno())
|
|
225
|
+
```
|
|
226
|
+
|
|
227
|
+
## 8. 不用容器
|
|
228
|
+
|
|
229
|
+
脚本和测试可以用 `loadNativeLibrary(names, declarations, options)` 加载库:声明一样、规则一样,方法也和组件一样(`func`、`function`、`variable`、`callback`、`read` 等),另外还有 `close()`。
|
|
230
|
+
|
|
231
|
+
```typescript
|
|
232
|
+
import {loadNativeLibrary} from '@lakutata/ffi'
|
|
233
|
+
|
|
234
|
+
const libc = loadNativeLibrary('c', 'typedef struct { int quot; int rem; } div_t;')
|
|
235
|
+
const div = libc.func<(numerator: number, denominator: number) => {quot: number, rem: number}>('div_t div(int numerator, int denominator)')
|
|
236
|
+
console.log(div(17, 5))
|
|
237
|
+
await libc.close()
|
|
238
|
+
```
|
|
239
|
+
|
|
240
|
+
## 9. 根据头文件生成类
|
|
241
|
+
|
|
242
|
+
函数很多的库,可以用 `lakutata ffi generate` 根据它的头文件生成类:包括它的声明、每个函数一个带 TypeScript 类型的属性、每个结构体和联合一个接口,整数宏作为常量。
|
|
243
|
+
|
|
244
|
+
```bash
|
|
245
|
+
lakutata ffi generate /usr/include/sqlite3.h --class-name SQLite3 --out src/native/SQLite3.ts
|
|
246
|
+
lakutata ffi generate sdk/include/vendor.h --library sdk/lib/vendor.dll --out src/native/Vendor.ts
|
|
247
|
+
```
|
|
248
|
+
|
|
249
|
+
- **头文件**先经过 C 编译器的预处理器(`cc -E`、`cl /P`):`#include`、`#if` 和宏按在当前平台上编译时那样展开(编译时用的头文件目录和宏用 `--include`、`--define` 给出)。在 Windows 上,请在 Visual Studio 的开发者命令提示符里运行,那里有 `cl`;用 `--compiler`(或 `CC` 环境变量)可以换别的编译器。
|
|
250
|
+
- **保留哪些声明**:头文件里的声明;它所在的目录不是系统目录时,还有这个目录里其他头文件的声明(库的其他头文件所在的目录用 `--own` 加上);以及这些声明用到的系统类型。这个包读不了的声明(`long double`、带参数的宏)会被去掉,并列出它所在的文件和行号。
|
|
251
|
+
- **用库检查**:加载和头文件同名的库(`sqlite3.h` 对应 `sqlite3`),或者 `--library` 给出的库(库名或文件),库里没有导出的函数和不能调用的函数(参数是 `va_list`)会被去掉并列出来。库加载不了时,照样生成类,只是不做检查,并给出警告(zlib 要写 `--library z`);`--no-check` 跳过检查。
|
|
252
|
+
- **生成的类**:每个函数一个属性,属性名就是函数名,上面注释着它的 C 原型;函数名和组件的成员重名时(`free`、`read`),属性换一个名字。TypeScript 类型按第 3 节的规则:64 位整数是 bigint,字符串是 `string | null`,其他指针是 `Pointer | null`(作为参数时也可以传 `Buffer` 或类型化数组)。
|
|
253
|
+
- **哪些要自己改**:头文件看不出哪些指针是输出参数,也看不出哪些函数会阻塞,生成的文件就是供你修改的。输出参数要在声明里它所属函数的原型上标成 `_Out_`,参数类型改成 `Out<T>`(指向结构体的指针改成那个结构体的接口)。会阻塞的函数在 `@Native` 里加上 `{async: true}`,这时 `@Native` 要写出函数名。头文件出了新版本时,重新生成类,再把这些修改搬过去。
|
|
254
|
+
- **只针对一个平台**:声明是运行生成器的那个平台上的;头文件按平台写了不同声明的(`#ifdef _WIN32`),要在每个平台上各生成一份。换一个版本的库可能少一些函数:组件启动时会报错,并指出缺的是哪个函数。在 Linux 上,库是通过它的开发包装的链接(`libsqlite3.so`)找到的:在没有装开发包的机器上,`names` 要写带版本号的文件名(`libsqlite3.so.0`,见第 2 节)。
|
|
255
|
+
|
|
256
|
+
下面是生成器为 `sqlite3_open` 和 `sqlite3_exec` 生成的代码改过之后的样子:
|
|
257
|
+
|
|
258
|
+
```typescript
|
|
259
|
+
import {Library, Native, NativeLibrary, type Out, type Pointer} from '@lakutata/ffi'
|
|
260
|
+
|
|
261
|
+
@Library({
|
|
262
|
+
names: 'sqlite3',
|
|
263
|
+
declarations: `
|
|
264
|
+
typedef struct sqlite3 sqlite3;
|
|
265
|
+
int sqlite3_open(const char *filename, _Out_ sqlite3 **ppDb);
|
|
266
|
+
int sqlite3_exec(sqlite3 *, const char *sql, int (*callback)(void *, int, char **, char **), void *, _Out_ char **errmsg);
|
|
267
|
+
`
|
|
268
|
+
})
|
|
269
|
+
export class SQLite3 extends NativeLibrary {
|
|
270
|
+
//生成时 ppDb 是 Pointer | Buffer | ArrayBufferView | null;在声明里把 ppDb 标成了 _Out_
|
|
271
|
+
@Native()
|
|
272
|
+
public readonly sqlite3_open: (filename: string | null, ppDb: Out<Pointer>) => number
|
|
273
|
+
|
|
274
|
+
//它会阻塞:改成异步;errmsg 标成 _Out_,消息由 sqlite3_free 释放
|
|
275
|
+
@Native('sqlite3_exec', {async: true, free: 'sqlite3_free'})
|
|
276
|
+
public readonly sqlite3_exec: (db: Pointer | null, sql: string | null, callback: null, context: null, errmsg: Out<string>) => Promise<number>
|
|
277
|
+
}
|
|
278
|
+
```
|
|
279
|
+
|
|
280
|
+
生成器也可以在脚本里调用:`@lakutata/ffi/generator` 的 `generateNativeLibrary()` 返回类的代码、它的声明和报告。
|
|
281
|
+
|
|
282
|
+
## 10. 库的线程
|
|
283
|
+
|
|
284
|
+
异步函数默认在运行时的线程池里执行。设置了 `threads` 之后,库就有自己的工作线程:每个线程各自加载这个库,异步函数在这些线程里执行。
|
|
285
|
+
|
|
286
|
+
```typescript
|
|
287
|
+
import {Library, Native, NativeLibrary} from '@lakutata/ffi'
|
|
288
|
+
|
|
289
|
+
@Library({names: 'vendor', threads: 1})
|
|
290
|
+
export class Vendor extends NativeLibrary {
|
|
291
|
+
@Native('int vendor_read(int device, unsigned char *buffer, size_t size)', {async: true})
|
|
292
|
+
public readonly read: (device: number, buffer: Buffer, size: number) => Promise<number>
|
|
293
|
+
|
|
294
|
+
@Native('int vendor_version(void)')
|
|
295
|
+
public readonly version: () => number
|
|
296
|
+
}
|
|
297
|
+
```
|
|
298
|
+
|
|
299
|
+
- **几个线程**:只有一个线程时,异步调用按发起的先后顺序,一个接一个执行,适合不是线程安全的库,或者一次只能处理一个请求的设备。有多个线程时,每个调用交给最空闲的那个线程。配置里可以改成别的数量(`buildNativeLibraryOptions(Vendor, {threads: 2})`),`loadNativeLibrary` 的选项里也可以设置;`0` 表示仍然用线程池。
|
|
300
|
+
- **哪些函数在线程里执行**:只有异步函数。同步函数仍然在调用它的线程里执行,所以这个库会同时被两个线程使用:如果库不是线程安全的,请把它的函数全部声明为异步。在这里,可变参数的函数也可以异步调用。库的线程不占用线程池:耗时长的调用不会拖慢文件系统,`UV_THREADPOOL_SIZE` 对它们也不起作用。
|
|
301
|
+
- **参数会复制**到线程里。函数会写入的参数,在函数返回后再复制回来:`_Out_` 和 `_Inout_` 参数,以及传给非 `const` 指针的缓冲区和类型化数组。缓冲区每次调用都要复制过去,函数可能写入的还要复制回来:缓冲区很大时,用线程池开销更小。指针原样传过去:`alloc()` 分配的内存所有线程都能用。
|
|
302
|
+
- **回调**:传给函数指针参数的 JS 函数,在发起调用的那个线程里执行,库的线程会等它执行完;它抛出的异常会让 Promise 被拒绝。持久回调和第 5 节说的一样。
|
|
303
|
+
- **开销**:每次调用都等上一次完成时,一次调用约 11 µs,线程池约 7 µs(Node.js,Apple 芯片)。
|
|
304
|
+
- **errno**:库的 `errno()` 和 `windowsError()` 给出最近完成的那次异步调用所在线程的值。只有一个线程,或者每次调用都等上一次完成时,这个值才可靠。从第一次异步调用完成起,它们就不再给出同步调用的值了:同步调用的值用 `lastErrno()` 和 `lastWindowsError()` 读(第 7 节)。
|
|
305
|
+
- **启动和关闭**:线程随库一起启动,加载库要几毫秒。组件要等线程把库加载好才算创建完成;用 `loadNativeLibrary` 加载的库,这时它的 `ready` 才会完成(调用本来也会等它)。关闭库时,先等进行中的调用结束,再让线程卸载库并退出。线程意外退出时,它正在执行的调用会失败,下一次调用会启动一个新线程。
|
|
306
|
+
|
|
307
|
+
## 11. 销毁和工作线程
|
|
308
|
+
|
|
309
|
+
应用停止时,库组件会:让它的属性抛出 `NativeLibraryClosedException`(异步函数返回被拒绝的 Promise);等进行中的异步调用结束(`drainTimeout`,默认 10 秒);释放它的回调;释放 `alloc()` 分配的内存;卸载库(`keepLoaded` 时保留;musl 从不卸载)。如果把属性上的函数另外存了一份(`const abs = libc.abs`),销毁后就不能再调用它:调用已卸载的库里的函数,进程会退出。
|
|
310
|
+
|
|
311
|
+
原生调用没法中途打断。过了 `drainTimeout` 还有调用没结束时,库、它的回调和它的内存都保持原样,让这些调用能安全返回。线程池里的调用继续执行,返回时照常完成;库的线程里的调用会以 `NativeLibraryClosedException` 被拒绝,那些线程在调用返回后退出。即使调用了 `process.exit()`,Node.js 也要等这样的调用返回才退出;Bun 的 `process.exit()` 不等它们。
|
|
312
|
+
|
|
313
|
+
工作线程要自己加载库(在线程里创建组件)。请先停止工作线程里运行的应用,再终止这个线程:应用会等调用结束,并关闭它的库。工作线程正在用 koffi 时被终止,整个进程都可能退出:
|
|
314
|
+
|
|
315
|
+
- koffi 正在这个线程里加载时,Node.js 和 Bun 都会这样:线程正在 import 这个包,或者线程里创建的库正在启动它的线程(几毫秒)。koffi 3.3.4 在加载过程中被终止就会让进程退出。库的线程启动期间退出进程,也可能这样;
|
|
316
|
+
- 在 Bun 里,线程池里还有异步调用没返回时(这是 Bun 的 bug,1.4.4 里也有),以及库的线程正在绑定函数时,也会这样。
|
|
317
|
+
|
|
318
|
+
库的线程里有异步调用正在执行时终止工作线程,进程不会崩溃。
|
|
319
|
+
|
|
320
|
+
## 12. 运行环境
|
|
321
|
+
|
|
322
|
+
Node.js `^22.13.0 || >=24.11.0`、Electron 的主进程和 Bun;不支持 Electron 的渲染进程和浏览器。
|
|
323
|
+
|
|
324
|
+
这个包的测试(包括库的线程)在 macOS(arm64:Node.js 24、Bun 1.4、Electron 44)、Linux(arm64,glibc 和 musl:Node.js 24、Bun 1.4)和 Windows 11(arm64,以及模拟运行的 x64:Node.js 24、Bun 1.4)上都能通过。在 macOS 上,还用 `lakutata package` 把一个使用它的应用(它的库在自己的线程里执行)分别打成了 bundle、Node.js 和 Bun 的可执行文件以及 Electron 应用:每一个都能运行,库的线程也都从应用自己的资源目录里加载了自带的库。
|
|
325
|
+
|
|
326
|
+
第 9 节的生成器在 macOS 上用 clang、在 Linux 上用 gcc(glibc 和 musl)、在 Windows 11 上用 cl(arm64 和 x64)运行过:它根据 sqlite3、zlib、pcap 和 curl 的头文件(在 Windows 上是 winsqlite3 和一个为测试编译的库)生成的类,都能通过 TypeScript 严格模式的编译,并在 Node.js 和 Bun 里运行。
|
|
327
|
+
|
|
328
|
+
## 13. 异常
|
|
329
|
+
|
|
330
|
+
| 异常 | 什么时候 |
|
|
331
|
+
|---|---|
|
|
332
|
+
| `NativeDeclarationException` | 声明或原型读不了(带行号和列号)、库里没有这个函数、某个类型不能传(`va_list`)、类上没有 `@Library` |
|
|
333
|
+
| `NativeLibraryNotFoundException` | 库加载不了:列出试过的每个文件和原因 |
|
|
334
|
+
| `NativeLibraryClosedException` | 库关闭后又使用它的函数、变量或回调;库关闭时,它的线程里还没结束的调用 |
|
|
335
|
+
| `NativeMemoryException` | 空指针、负的长度、在 Electron 里用视图 |
|
|
336
|
+
| `TypeError`(来自 koffi) | 参数类型不对 |
|
|
337
|
+
| `RangeError` | 声明了 `{int64: 'number'}` 的函数遇到超出安全整数范围的 64 位整数 |
|
package/doc/zh/NativeAddons.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Lakutata 原生插件
|
|
2
2
|
|
|
3
|
-
原生插件是用 C++ 或 Rust 编写、由 Node.js 以 `.node` 文件加载的库:用于驱动和设备、编解码、系统 API,或者必须跑得快的代码。`lakutata add --native` 在工作区中创建一个原生插件包,可以直接构建和测试;`lakutata native` 给它链接库;`lakutata package` 把它和应用一起发布。这些插件使用 Node-API,所以同一份构建能在所有受支持的 Node.js 版本和 Electron
|
|
3
|
+
原生插件是用 C++ 或 Rust 编写、由 Node.js 以 `.node` 文件加载的库:用于驱动和设备、编解码、系统 API,或者必须跑得快的代码。`lakutata add --native` 在工作区中创建一个原生插件包,可以直接构建和测试;`lakutata native` 给它链接库;`lakutata package` 把它和应用一起发布。这些插件使用 Node-API,所以同一份构建能在所有受支持的 Node.js 版本和 Electron 中运行。不写插件、直接调用库里的 C 函数,见 [FFI](FFI.md)。
|
|
4
4
|
|
|
5
5
|
## 1. 创建原生插件
|
|
6
6
|
|
package/doc/zh/Overview.md
CHANGED
|
@@ -35,7 +35,7 @@
|
|
|
35
35
|
| `lakutata/security`、`lakutata/provider/passwordHash` | 哈希、加解密、签名、`PasswordHash` 提供者 |
|
|
36
36
|
| `lakutata/utils` | 辅助函数 |
|
|
37
37
|
|
|
38
|
-
不在本包中的:`@lakutata/cli`(`lakutata` 命令:创建项目、执行迁移、打包应用,见 [Command](Command.md))、`@lakutata/grpc`(gRPC 客户端、服务端和服务传输)、`@lakutata/nats`(NATS 组件和服务传输)、`@lakutata/dbus`(D-Bus)、`@lakutata/mcp`(Model Context Protocol
|
|
38
|
+
不在本包中的:`@lakutata/cli`(`lakutata` 命令:创建项目、执行迁移、打包应用,见 [Command](Command.md))、`@lakutata/grpc`(gRPC 客户端、服务端和服务传输)、`@lakutata/nats`(NATS 组件和服务传输)、`@lakutata/dbus`(D-Bus)、`@lakutata/mcp`(Model Context Protocol)、`@lakutata/webrtc`(WebRTC:信令、TURN 服务器、媒体服务器、对等连接)和 `@lakutata/ffi`(调用动态库中的 C 函数)。
|
|
39
39
|
|
|
40
40
|
## 指南
|
|
41
41
|
|
|
@@ -79,6 +79,7 @@
|
|
|
79
79
|
| [DBus](DBus.md) | D-Bus 服务和客户端(`@lakutata/dbus`):方法、对象、属性、信号 |
|
|
80
80
|
| [MCP](MCP.md) | 模型上下文协议(`@lakutata/mcp`)的工具、资源和提示词,基于 stdio 和 HTTP |
|
|
81
81
|
| [WebRTC](WebRTC.md) | 点对点连接和媒体服务器(`@lakutata/webrtc`):信令、TURN 服务器、浏览器和应用、通过数据通道调用应用的动作、WHIP 和 WHEP |
|
|
82
|
+
| [FFI](FFI.md) | 调用动态库中的 C 函数(`@lakutata/ffi`):从头文件复制的声明、结构体、回调、内存、打包 |
|
|
82
83
|
| [Command](Command.md) | `lakutata` 命令:创建项目、迁移、翻译、环境变量 |
|
|
83
84
|
| [Workspaces](Workspaces.md) | 由多个包组成的项目:运行各个包的脚本、统一版本号、添加包 |
|
|
84
85
|
| [NativeAddons](NativeAddons.md) | 用 C++ 或 Rust 编写的原生插件:创建、链接库、发布 |
|
package/doc/zh/UpgradingFrom2.md
CHANGED
|
@@ -22,7 +22,7 @@
|
|
|
22
22
|
|
|
23
23
|
- **Node.js** `^22.13.0 || >=24.11.0`(22.13 或更高的 22.x,或者 24.11 及以上;不包括 23)。同时更新 `engines`、CI 的版本矩阵,以及 Dockerfile 的基础镜像。
|
|
24
24
|
- **支持 `"module": "Node20"` 的 TypeScript**(模板使用 TypeScript 6):`lakutata/tsconfig.json` 现在设置了这个选项(2.x 中是 `Node16`),更旧的编译器不认识它。
|
|
25
|
-
- **`package.json`**:`"type": "module"`,`lakutata` 为 `^3.0.0
|
|
25
|
+
- **`package.json`**:`"type": "module"`,`lakutata` 为 `^3.0.0`,`@lakutata/*` 包也写 `^3.0.0`,`@lakutata/cli` 放在 `devDependencies` 中。不要把 `lakutata` 的版本号照抄给其他包:补丁版只发布改动过的包,各包的最新版本可能不同(比如 `lakutata` 是 3.1.4,`@lakutata/ffi` 是 3.1.2)。
|
|
26
26
|
- **全新安装**:删除 `node_modules` 和锁文件,再重新安装。原地升级可能留下多份 `@lakutata/core`,它们的类和装饰器元数据互不相通(跨副本的 `instanceof` 和注入都会失败)。`npm ls @lakutata/core` 必须只显示一个版本。
|
|
27
27
|
- **`tsconfig.json`** 继承 `lakutata/tsconfig.json`,并设置自己的 `rootDir`、`outDir` 和 `include`。要保留 `include`:发布的选项设置了 `allowJs`,没有它的话,项目根目录下的配置文件(`eslint.config.js`)也会被编译。保留自己选项的项目需要 `experimentalDecorators`、`emitDecoratorMetadata` 和 `useDefineForClassFields: false`。
|
|
28
28
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "lakutata",
|
|
3
|
-
"version": "3.1.
|
|
3
|
+
"version": "3.1.2",
|
|
4
4
|
"description": "An IoC application framework for Node.js and TypeScript: one programming model for web services, CLI tools, desktop (Electron) and embedded applications.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"exports": {
|