tigerbeetle-node 0.9.0 → 0.9.143

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (79) hide show
  1. package/README.md +580 -179
  2. package/dist/benchmark.js +44 -36
  3. package/dist/benchmark.js.map +1 -1
  4. package/dist/bin/aarch64-linux-gnu/client.node +0 -0
  5. package/dist/bin/aarch64-linux-musl/client.node +0 -0
  6. package/dist/bin/aarch64-macos/client.node +0 -0
  7. package/dist/bin/x86_64-linux-gnu/client.node +0 -0
  8. package/dist/bin/x86_64-linux-musl/client.node +0 -0
  9. package/dist/bin/x86_64-macos/client.node +0 -0
  10. package/dist/bin/x86_64-windows/client.node +0 -0
  11. package/dist/bindings.d.ts +141 -0
  12. package/dist/bindings.js +112 -0
  13. package/dist/bindings.js.map +1 -0
  14. package/dist/index.d.ts +2 -125
  15. package/dist/index.js +51 -101
  16. package/dist/index.js.map +1 -1
  17. package/dist/test.js +68 -54
  18. package/dist/test.js.map +1 -1
  19. package/package-lock.json +26 -0
  20. package/package.json +13 -22
  21. package/src/benchmark.ts +58 -49
  22. package/src/bindings.ts +631 -0
  23. package/src/index.ts +71 -163
  24. package/src/node.zig +169 -148
  25. package/src/test.ts +71 -57
  26. package/src/translate.zig +19 -36
  27. package/scripts/download_node_headers.sh +0 -25
  28. package/src/tigerbeetle/scripts/benchmark.bat +0 -46
  29. package/src/tigerbeetle/scripts/benchmark.sh +0 -55
  30. package/src/tigerbeetle/scripts/install.sh +0 -6
  31. package/src/tigerbeetle/scripts/install_zig.bat +0 -109
  32. package/src/tigerbeetle/scripts/install_zig.sh +0 -84
  33. package/src/tigerbeetle/scripts/lint.zig +0 -199
  34. package/src/tigerbeetle/scripts/upgrade_ubuntu_kernel.sh +0 -39
  35. package/src/tigerbeetle/scripts/vopr.bat +0 -48
  36. package/src/tigerbeetle/scripts/vopr.sh +0 -33
  37. package/src/tigerbeetle/scripts/vr_state_enumerate +0 -46
  38. package/src/tigerbeetle/src/benchmark.zig +0 -290
  39. package/src/tigerbeetle/src/cli.zig +0 -244
  40. package/src/tigerbeetle/src/config.zig +0 -239
  41. package/src/tigerbeetle/src/demo.zig +0 -125
  42. package/src/tigerbeetle/src/demo_01_create_accounts.zig +0 -35
  43. package/src/tigerbeetle/src/demo_02_lookup_accounts.zig +0 -7
  44. package/src/tigerbeetle/src/demo_03_create_transfers.zig +0 -24
  45. package/src/tigerbeetle/src/demo_04_create_pending_transfers.zig +0 -61
  46. package/src/tigerbeetle/src/demo_05_post_pending_transfers.zig +0 -37
  47. package/src/tigerbeetle/src/demo_06_void_pending_transfers.zig +0 -24
  48. package/src/tigerbeetle/src/demo_07_lookup_transfers.zig +0 -7
  49. package/src/tigerbeetle/src/fifo.zig +0 -104
  50. package/src/tigerbeetle/src/io/benchmark.zig +0 -213
  51. package/src/tigerbeetle/src/io/darwin.zig +0 -793
  52. package/src/tigerbeetle/src/io/linux.zig +0 -1038
  53. package/src/tigerbeetle/src/io/test.zig +0 -643
  54. package/src/tigerbeetle/src/io/windows.zig +0 -1161
  55. package/src/tigerbeetle/src/io.zig +0 -34
  56. package/src/tigerbeetle/src/main.zig +0 -144
  57. package/src/tigerbeetle/src/message_bus.zig +0 -1000
  58. package/src/tigerbeetle/src/message_pool.zig +0 -142
  59. package/src/tigerbeetle/src/ring_buffer.zig +0 -289
  60. package/src/tigerbeetle/src/simulator.zig +0 -417
  61. package/src/tigerbeetle/src/state_machine.zig +0 -2470
  62. package/src/tigerbeetle/src/storage.zig +0 -308
  63. package/src/tigerbeetle/src/test/cluster.zig +0 -351
  64. package/src/tigerbeetle/src/test/message_bus.zig +0 -93
  65. package/src/tigerbeetle/src/test/network.zig +0 -179
  66. package/src/tigerbeetle/src/test/packet_simulator.zig +0 -387
  67. package/src/tigerbeetle/src/test/state_checker.zig +0 -145
  68. package/src/tigerbeetle/src/test/state_machine.zig +0 -76
  69. package/src/tigerbeetle/src/test/storage.zig +0 -438
  70. package/src/tigerbeetle/src/test/time.zig +0 -84
  71. package/src/tigerbeetle/src/tigerbeetle.zig +0 -222
  72. package/src/tigerbeetle/src/time.zig +0 -113
  73. package/src/tigerbeetle/src/unit_tests.zig +0 -14
  74. package/src/tigerbeetle/src/vsr/client.zig +0 -505
  75. package/src/tigerbeetle/src/vsr/clock.zig +0 -812
  76. package/src/tigerbeetle/src/vsr/journal.zig +0 -2293
  77. package/src/tigerbeetle/src/vsr/marzullo.zig +0 -309
  78. package/src/tigerbeetle/src/vsr/replica.zig +0 -5015
  79. package/src/tigerbeetle/src/vsr.zig +0 -1017
package/README.md CHANGED
@@ -1,249 +1,650 @@
1
+ ---
2
+ title: Node.js
3
+ ---
4
+
5
+ This file is generated by
6
+ [/src/clients/docs_generate.zig](/src/clients/docs_generate.zig).
1
7
  # tigerbeetle-node
2
- [TigerBeetle](https://github.com/coilhq/tigerbeetle) client for Node.js.
3
8
 
4
- ## Installation
9
+ The TigerBeetle client for Node.js.
10
+
11
+ ### Prerequisites
12
+
13
+ Linux >= 5.6 is the only production environment we
14
+ support. But for ease of development we also support macOS and Windows.
15
+ * NodeJS >= `14`
5
16
 
6
- Install the `tigerbeetle-node` module to your current working directory:
17
+ ## Setup
7
18
 
8
- ```shell
19
+ First, create a directory for your project and `cd` into the directory.
20
+
21
+ Then, install the TigerBeetle client:
22
+
23
+ ```console
9
24
  npm install tigerbeetle-node
10
25
  ```
11
26
 
12
- ### Prerequisites
27
+ Now, create `main.js` and copy this into it:
28
+
29
+ ```javascript
30
+ const { createClient } = require("tigerbeetle-node");
31
+ console.log("Import ok!");
32
+ ```
33
+
34
+ Finally, build and run:
35
+
36
+ ```console
37
+ node main.js
38
+ ```
39
+
40
+ Now that all prerequisites and dependencies are correctly set
41
+ up, let's dig into using TigerBeetle.
13
42
 
14
- * NodeJS >= `14.0.0`. _(If the correct version is not installed, an installation error will occur)_
43
+ If you run into issues, check out the distribution-specific install
44
+ steps that are run in CI to test support:
15
45
 
16
- > Your operating system should be Linux (kernel >= v5.6) or macOS.
17
- > Windows support is not yet available.
46
+ * [Alpine](https://github.com/tigerbeetle/tigerbeetle/blob/main/src/clients/node/scripts/test_install_on_alpine.sh)
47
+ * [Amazon Linux](https://github.com/tigerbeetle/tigerbeetle/blob/main/src/clients/node/scripts/test_install_on_amazonlinux.sh)
48
+ * [Debian](https://github.com/tigerbeetle/tigerbeetle/blob/main/src/clients/node/scripts/test_install_on_debian.sh)
49
+ * [Fedora](https://github.com/tigerbeetle/tigerbeetle/blob/main/src/clients/node/scripts/test_install_on_fedora.sh)
50
+ * [Ubuntu](https://github.com/tigerbeetle/tigerbeetle/blob/main/src/clients/node/scripts/test_install_on_ubuntu.sh)
51
+ * [RHEL](https://github.com/tigerbeetle/tigerbeetle/blob/main/src/clients/node/scripts/test_install_on_rhelubi.sh)
18
52
 
19
- ## Usage
53
+ ## Sample projects
20
54
 
21
- A client needs to be configured with a `cluster_id` and `replica_addresses`.
22
- This instantiates the client where memory is allocated to internally buffer events to be sent.
23
- For the moment, only one client can be instantiated globally per process.
24
- Future releases will allow multiple client instantiations.
55
+ This document is primarily a reference guide to
56
+ the client. Below are various sample projects demonstrating
57
+ features of TigerBeetle.
25
58
 
26
- ```js
27
- import { createClient } from 'tigerbeetle-node'
59
+ * [Basic](/src/clients/node/samples/basic/): Create two accounts and transfer an amount between them.
60
+ * [Two-Phase Transfer](/src/clients/node/samples/two-phase/): Create two accounts and start a pending transfer between
61
+ them, then post the transfer.
62
+ * [Many Two-Phase Transfers](/src/clients/node/samples/two-phase-many/): Create two accounts and start a number of pending transfer
63
+ between them, posting and voiding alternating transfers.
64
+ ### Sidenote: `BigInt`
65
+ TigerBeetle uses 64-bit integers for many fields while JavaScript's
66
+ builtin `Number` maximum value is `2^53-1`. The `n` suffix in JavaScript
67
+ means the value is a `BigInt`. This is useful for literal numbers. If
68
+ you already have a `Number` variable though, you can call the `BigInt`
69
+ constructor to get a `BigInt` from it. For example, `1n` is the same as
70
+ `BigInt(1)`.
28
71
 
72
+ ## Creating a Client
73
+
74
+ A client is created with a cluster ID and replica
75
+ addresses for all replicas in the cluster. The cluster
76
+ ID and replica addresses are both chosen by the system that
77
+ starts the TigerBeetle cluster.
78
+
79
+ Clients are thread-safe and a single instance should be shared
80
+ between multiple concurrent tasks.
81
+
82
+ Multiple clients are useful when connecting to more than
83
+ one TigerBeetle cluster.
84
+
85
+ In this example the cluster ID is `0` and there is one
86
+ replica. The address is read from the `TB_ADDRESS`
87
+ environment variable and defaults to port `3000`.
88
+
89
+ ```javascript
29
90
  const client = createClient({
30
91
  cluster_id: 0,
31
- replica_addresses: ['3001', '3002', '3003']
32
- })
33
- ```
34
-
35
- One of the ways TigerBeetle achieves its performance is through batching.
36
- This is reflected in the below function interfaces where each one takes in an array of events.
37
-
38
- ### Account Creation
39
-
40
- ```js
41
- const account = {
42
- id: 137n, // u128
43
- user_data: 0n, // u128, opaque third-party identifier to link this account to an external entity:
44
- reserved: Buffer.alloc(48, 0), // [48]u8
45
- ledger: 1, // u32, ledger value
46
- code: 718, // u16, a chart of accounts code describing the type of account (e.g. clearing, settlement)
47
- flags: 0, // u16
48
- debits_pending: 0n, // u64
49
- debits_posted: 0n, // u64
50
- credits_pending: 0n, // u64
51
- credits_posted: 0n, // u64
52
- timestamp: 0n, // u64, Reserved: This will be set by the server.
53
- }
54
-
55
- const errors = await client.createAccounts([account])
92
+ replica_addresses: [process.env.TB_ADDRESS || '3000']
93
+ });
56
94
  ```
57
- Successfully executed events return an empty array whilst unsuccessful ones return an array with errors for **only the ones that failed**. An error will point to the index in the submitted array of the failed event.
58
- ```js
59
- const errors = await client.createAccounts([account1, account2, account3])
60
95
 
61
- // errors = [{ index: 1, code: 1 }]
62
- const error = errors[0]
63
- switch (error.code) {
64
- case CreateAccountError.exists: {
65
- console.error(`Batch event at ${error.index} already exists.`)
66
- }
67
- }
96
+ The following are valid addresses:
97
+ * `3000` (interpreted as `127.0.0.1:3000`)
98
+ * `127.0.0.1:3000` (interpreted as `127.0.0.1:3000`)
99
+ * `127.0.0.1` (interpreted as `127.0.0.1:3001`, `3001` is the default port)
100
+
101
+ ## Creating Accounts
102
+
103
+ See details for account fields in the [Accounts
104
+ reference](https://docs.tigerbeetle.com/reference/accounts).
105
+
106
+ ```javascript
107
+ let account = {
108
+ id: 137n,
109
+ debits_pending: 0n,
110
+ debits_posted: 0n,
111
+ credits_pending: 0n,
112
+ credits_posted: 0n,
113
+ user_data_128: 0n,
114
+ user_data_64: 0n,
115
+ user_data_32: 0,
116
+ reserved: 0,
117
+ ledger: 1,
118
+ code: 718,
119
+ flags: 0,
120
+ timestamp: 0n,
121
+ };
122
+
123
+ let accountErrors = await client.createAccounts([account]);
68
124
  ```
69
- The example above shows that the event in index 1 failed with error 1. This means that `account1` and `account3` were created successfully but not `account2`.
70
125
 
71
- The `flags` on an account provide a way for you to enforce policies by toggling the bits below.
72
- | bit 0 | bit 1 | bit 2 |
73
- |----------|----------------------------------|-----------------------------------------|
74
- | `linked` | `debits_must_not_exceed_credits` | `credits_must_not_exceed_debits` |
126
+ ### Account Flags
127
+
128
+ The account flags value is a bitfield. See details for
129
+ these flags in the [Accounts
130
+ reference](https://docs.tigerbeetle.com/reference/accounts#flags).
131
+
132
+ To toggle behavior for an account, combine enum values stored in the
133
+ `AccountFlags` object (in TypeScript it is an actual enum) with
134
+ bitwise-or:
135
+
136
+ * `AccountFlags.linked`
137
+ * `AccountFlags.debits_must_not_exceed_credits`
138
+ * `AccountFlags.credits_must_not_exceed_credits`
139
+
140
+
141
+ For example, to link two accounts where the first account
142
+ additionally has the `debits_must_not_exceed_credits` constraint:
143
+
144
+ ```javascript
145
+ let account0 = {
146
+ id: 100n,
147
+ debits_pending: 0n,
148
+ debits_posted: 0n,
149
+ credits_pending: 0n,
150
+ credits_posted: 0n,
151
+ user_data_128: 0n,
152
+ user_data_64: 0n,
153
+ user_data_32: 0,
154
+ reserved: 0,
155
+ ledger: 1,
156
+ code: 1,
157
+ timestamp: 0n,
158
+ flags: 0,
159
+ };
160
+ let account1 = {
161
+ id: 101n,
162
+ debits_pending: 0n,
163
+ debits_posted: 0n,
164
+ credits_pending: 0n,
165
+ credits_posted: 0n,
166
+ user_data_128: 0n,
167
+ user_data_64: 0n,
168
+ user_data_32: 0,
169
+ reserved: 0,
170
+ ledger: 1,
171
+ code: 1,
172
+ timestamp: 0n,
173
+ flags: 0,
174
+ };
175
+ account0.flags = AccountFlags.linked | AccountFlags.debits_must_not_exceed_credits;
176
+ accountErrors = await client.createAccounts([account0, account1]);
177
+ ```
75
178
 
76
- The creation of an account can be linked to the successful creation of another by setting the `linked` flag (see [linked events](#linked-events)). By setting `debits_must_not_exceed_credits`, then any transfer such that `debits_posted + debits_pending + amount > credits_posted` will fail. Similarly for `credits_must_not_exceed_debits`.
77
- ```js
78
- enum CreateAccountFlags {
79
- linked = (1 << 0),
80
- debits_must_not_exceed_credits = (1 << 1),
81
- credits_must_not_exceed_debits = (1 << 2)
179
+ ### Response and Errors
180
+
181
+ The response is an empty array if all accounts were
182
+ created successfully. If the response is non-empty, each
183
+ object in the response array contains error information
184
+ for an account that failed. The error object contains an
185
+ error code and the index of the account in the request
186
+ batch.
187
+
188
+ See all error conditions in the [create_accounts
189
+ reference](https://docs.tigerbeetle.com/reference/operations/create_accounts).
190
+
191
+ ```javascript
192
+ let account2 = {
193
+ id: 102n,
194
+ debits_pending: 0n,
195
+ debits_posted: 0n,
196
+ credits_pending: 0n,
197
+ credits_posted: 0n,
198
+ user_data_128: 0n,
199
+ user_data_64: 0n,
200
+ user_data_32: 0,
201
+ reserved: 0,
202
+ ledger: 1,
203
+ code: 1,
204
+ timestamp: 0n,
205
+ flags: 0,
206
+ };
207
+ let account3 = {
208
+ id: 103n,
209
+ debits_pending: 0n,
210
+ debits_posted: 0n,
211
+ credits_pending: 0n,
212
+ credits_posted: 0n,
213
+ user_data_128: 0n,
214
+ user_data_64: 0n,
215
+ user_data_32: 0,
216
+ reserved: 0,
217
+ ledger: 1,
218
+ code: 1,
219
+ timestamp: 0n,
220
+ flags: 0,
221
+ };
222
+ let account4 = {
223
+ id: 104n,
224
+ debits_pending: 0n,
225
+ debits_posted: 0n,
226
+ credits_pending: 0n,
227
+ credits_posted: 0n,
228
+ user_data_128: 0n,
229
+ user_data_64: 0n,
230
+ user_data_32: 0,
231
+ reserved: 0,
232
+ ledger: 1,
233
+ code: 1,
234
+ timestamp: 0n,
235
+ flags: 0,
236
+ };
237
+ accountErrors = await client.createAccounts([account2, account3, account4]);
238
+ for (const error of accountErrors) {
239
+ switch (error.result) {
240
+ case CreateAccountError.exists:
241
+ console.error(`Batch account at ${error.index} already exists.`);
242
+ break;
243
+ default:
244
+ console.error(`Batch account at ${error.index} failed to create: ${CreateAccountError[error.result]}.`);
82
245
  }
246
+ }
247
+ ```
83
248
 
84
- let flags = 0
85
- flags |= CreateAccountFlags.debits_must_not_exceed_credits
249
+ To handle errors you can either 1) exactly match error codes returned
250
+ from `client.createAccounts` with enum values in the
251
+ `CreateAccountError` object, or you can 2) look up the error code in
252
+ the `CreateAccountError` object for a human-readable string.
253
+
254
+ ## Account Lookup
255
+
256
+ Account lookup is batched, like account creation. Pass
257
+ in all IDs to fetch. The account for each matched ID is returned.
258
+
259
+ If no account matches an ID, no object is returned for
260
+ that account. So the order of accounts in the response is
261
+ not necessarily the same as the order of IDs in the
262
+ request. You can refer to the ID field in the response to
263
+ distinguish accounts.
264
+
265
+ ```javascript
266
+ const accounts = await client.lookupAccounts([137n, 138n]);
267
+ console.log(accounts);
268
+ /*
269
+ * [{
270
+ * id: 137n,
271
+ * debits_pending: 0n,
272
+ * debits_posted: 0n,
273
+ * credits_pending: 0n,
274
+ * credits_posted: 0n,
275
+ * user_data_128: 0n,
276
+ * user_data_64: 0n,
277
+ * user_data_32: 0,
278
+ * reserved: 0,
279
+ * ledger: 1,
280
+ * code: 718,
281
+ * flags: 0,
282
+ * timestamp: 1623062009212508993n,
283
+ * }]
284
+ */
86
285
  ```
87
286
 
88
- ### Account Lookup
287
+ ## Create Transfers
89
288
 
90
- The `id` of the account is used for lookups. Only matched accounts are returned.
91
- ```js
92
- // account 137n exists, 138n does not
93
- const accounts = await client.lookupAccounts([137n, 138n])
289
+ This creates a journal entry between two accounts.
290
+
291
+ See details for transfer fields in the [Transfers
292
+ reference](https://docs.tigerbeetle.com/reference/transfers).
293
+
294
+ ```javascript
295
+ let transfer = {
296
+ id: 1n,
297
+ debit_account_id: 102n,
298
+ credit_account_id: 103n,
299
+ amount: 10n,
300
+ pending_id: 0n,
301
+ user_data_128: 0n,
302
+ user_data_64: 0n,
303
+ user_data_32: 0,
304
+ timeout: 0,
305
+ ledger: 1,
306
+ code: 720,
307
+ flags: 0,
308
+ timestamp: 0n,
309
+ };
310
+ let transferErrors = await client.createTransfers([transfer]);
311
+ ```
94
312
 
95
- /**
96
- * const accounts = [{
97
- * id: 137n,
98
- * user_data: 0n,
99
- * reserved: Buffer,
100
- * ledger: 1,
101
- * code: 718,
102
- * flags: 0,
103
- * debits_pending: 0n,
104
- * debits_posted: 0n,
105
- * credits_pending: 0n,
106
- * credits_posted: 0n,
107
- * timestamp: 1623062009212508993n,
108
- * }]
109
- */
313
+ ### Response and Errors
314
+
315
+ The response is an empty array if all transfers were created
316
+ successfully. If the response is non-empty, each object in the
317
+ response array contains error information for a transfer that
318
+ failed. The error object contains an error code and the index of the
319
+ transfer in the request batch.
320
+
321
+ See all error conditions in the [create_transfers
322
+ reference](https://docs.tigerbeetle.com/reference/operations/create_transfers).
323
+
324
+ ```javascript
325
+ for (const error of transferErrors) {
326
+ switch (error.result) {
327
+ case CreateTransferError.exists:
328
+ console.error(`Batch transfer at ${error.index} already exists.`);
329
+ break;
330
+ default:
331
+ console.error(`Batch transfer at ${error.index} failed to create: ${CreateTransferError[error.result]}.`);
332
+ }
333
+ }
110
334
  ```
111
335
 
112
- ### Creating a Transfer
336
+ To handle errors you can either 1) exactly match error codes returned
337
+ from `client.createTransfers` with enum values in the
338
+ `CreateTransferError` object, or you can 2) look up the error code in
339
+ the `CreateTransferError` object for a human-readable string.
113
340
 
114
- This creates a journal entry between two accounts.
115
- ```js
116
- const transfer = {
117
- id: 1n, // u128
118
- // Double-entry accounting:
119
- debit_account_id: 1n, // u128
120
- credit_account_id: 2n, // u128
121
- // Opaque third-party identifier to link this transfer to an external entity:
122
- user_data: 0n, // u128
123
- reserved: 0n, // u128
124
- // Timeout applicable for a pending/2-phase transfer:
125
- timeout: 0n, // u64, in nano-seconds.
126
- // Collection of accounts usually grouped by the currency:
127
- // You can't transfer money between accounts with different ledgers:
128
- ledger: 720, // u32, ledger for transfer (e.g. currency).
129
- // Chart of accounts code describing the reason for the transfer:
130
- code: 1, // u16, (e.g. deposit, settlement)
131
- flags: 0, // u16
132
- amount: 10n, // u64
133
- timestamp: 0n, //u64, Reserved: This will be set by the server.
341
+ ## Batching
342
+
343
+ TigerBeetle performance is maximized when you batch
344
+ API requests. The client does not do this automatically for
345
+ you. So, for example, you *can* insert 1 million transfers
346
+ one at a time like so:
347
+
348
+ ```javascript
349
+ for (let i = 0; i < transfers.len; i++) {
350
+ const transferErrors = await client.createTransfers(transfers[i]);
351
+ // error handling omitted
134
352
  }
135
- const errors = await client.createTransfers([transfer])
136
353
  ```
137
- Two-phase transfers are supported natively by toggling the appropriate flag. TigerBeetle will then adjust the `credits_pending` and `debits_pending` fields of the appropriate accounts. A corresponding commit transfer then needs to be sent to accept or reject the transfer.
138
354
 
139
- Transfers within a batch may also be linked (see [linked events](#linked-events)).
140
- ```js
141
- enum TransferFlags {
142
- linked = (1 << 0),
143
- pending = (1 << 1),
144
- post_pending_transfer = (1 << 2),
145
- void_pending_transfer = (1 << 3)
146
- }
147
-
148
- // Two-phase transfer (pending):
149
- let flags = 0n
150
- flags |= TransferFlags.pending
151
-
152
- // Linked two-phase transfer (pending):
153
- let flags = 0n
154
- flags |= TransferFlags.linked
155
- flags |= TransferFlags.pending
156
- ```
157
-
158
- ### Post a Pending Transfer (2-Phase)
159
-
160
- With `flags = post_pending_transfer`, TigerBeetle will accept the transfer. TigerBeetle will atomically rollback the changes to `debits_pending` and `credits_pending` of the appropriate accounts and apply them to the `debits_posted` and `credits_posted` balances.
161
- ```js
162
- const post = {
163
- id: 2n, // u128, must correspond to the transfer id
164
- pending_id: 1n, // u128, id of the pending transfer
165
- flags: TransferFlags.post_pending_transfer, // to void, use [void_pending_transfer]
166
- timestamp: 0n, // u64, Reserved: This will be set by the server.
355
+ But the insert rate will be a *fraction* of
356
+ potential. Instead, **always batch what you can**.
357
+
358
+ The maximum batch size is set in the TigerBeetle server. The default
359
+ is 8191.
360
+
361
+ ```javascript
362
+ const BATCH_SIZE = 8191;
363
+ for (let i = 0; i < transfers.length; i += BATCH_SIZE) {
364
+ const transferErrors = await client.createTransfers(transfers.slice(i, Math.min(transfers.length, BATCH_SIZE)));
365
+ // error handling omitted
167
366
  }
168
- const errors = await client.createTransfers([post])
169
367
  ```
170
368
 
171
- ### Linked Events
369
+ ### Queues and Workers
370
+
371
+ If you are making requests to TigerBeetle from workers
372
+ pulling jobs from a queue, you can batch requests to
373
+ TigerBeetle by having the worker act on multiple jobs from
374
+ the queue at once rather than one at a time. i.e. pulling
375
+ multiple jobs from the queue rather than just one.
376
+
377
+ ## Transfer Flags
378
+
379
+ The transfer `flags` value is a bitfield. See details for these flags in
380
+ the [Transfers
381
+ reference](https://docs.tigerbeetle.com/reference/transfers#flags).
382
+
383
+ To toggle behavior for a transfer, combine enum values stored in the
384
+ `TransferFlags` object (in TypeScript it is an actual enum) with
385
+ bitwise-or:
386
+
387
+ * `TransferFlags.linked`
388
+ * `TransferFlags.pending`
389
+ * `TransferFlags.post_pending_transfer`
390
+ * `TransferFlags.void_pending_transfer`
391
+
392
+ For example, to link `transfer0` and `transfer1`:
393
+
394
+ ```javascript
395
+ let transfer0 = {
396
+ id: 2n,
397
+ debit_account_id: 102n,
398
+ credit_account_id: 103n,
399
+ amount: 10n,
400
+ pending_id: 0n,
401
+ user_data_128: 0n,
402
+ user_data_64: 0n,
403
+ user_data_32: 0,
404
+ timeout: 0,
405
+ ledger: 1,
406
+ code: 720,
407
+ flags: 0,
408
+ timestamp: 0n,
409
+ };
410
+ let transfer1 = {
411
+ id: 3n,
412
+ debit_account_id: 102n,
413
+ credit_account_id: 103n,
414
+ amount: 10n,
415
+ pending_id: 0n,
416
+ user_data_128: 0n,
417
+ user_data_64: 0n,
418
+ user_data_32: 0,
419
+ timeout: 0,
420
+ ledger: 1,
421
+ code: 720,
422
+ flags: 0,
423
+ timestamp: 0n,
424
+ };
425
+ transfer0.flags = TransferFlags.linked;
426
+ // Create the transfer
427
+ transferErrors = await client.createTransfers([transfer0, transfer1]);
428
+ ```
429
+
430
+ ### Two-Phase Transfers
431
+
432
+ Two-phase transfers are supported natively by toggling the appropriate
433
+ flag. TigerBeetle will then adjust the `credits_pending` and
434
+ `debits_pending` fields of the appropriate accounts. A corresponding
435
+ post pending transfer then needs to be sent to post or void the
436
+ transfer.
437
+
438
+ #### Post a Pending Transfer
439
+
440
+ With `flags` set to `post_pending_transfer`,
441
+ TigerBeetle will post the transfer. TigerBeetle will atomically roll
442
+ back the changes to `debits_pending` and `credits_pending` of the
443
+ appropriate accounts and apply them to the `debits_posted` and
444
+ `credits_posted` balances.
445
+
446
+ ```javascript
447
+ let transfer2 = {
448
+ id: 4n,
449
+ debit_account_id: 102n,
450
+ credit_account_id: 103n,
451
+ amount: 10n,
452
+ pending_id: 0n,
453
+ user_data_128: 0n,
454
+ user_data_64: 0n,
455
+ user_data_32: 0,
456
+ timeout: 0,
457
+ ledger: 1,
458
+ code: 720,
459
+ flags: TransferFlags.pending,
460
+ timestamp: 0n,
461
+ };
462
+ transferErrors = await client.createTransfers([transfer2]);
463
+
464
+ let transfer3 = {
465
+ id: 5n,
466
+ debit_account_id: 102n,
467
+ credit_account_id: 103n,
468
+ amount: 10n,
469
+ pending_id: 4n,
470
+ user_data_128: 0n,
471
+ user_data_64: 0n,
472
+ user_data_32: 0,
473
+ timeout: 0,
474
+ ledger: 1,
475
+ code: 720,
476
+ flags: TransferFlags.post_pending_transfer,
477
+ timestamp: 0n,
478
+ };
479
+ transferErrors = await client.createTransfers([transfer3]);
480
+ ```
481
+
482
+ #### Void a Pending Transfer
483
+
484
+ In contrast, with `flags` set to `void_pending_transfer`,
485
+ TigerBeetle will void the transfer. TigerBeetle will roll
486
+ back the changes to `debits_pending` and `credits_pending` of the
487
+ appropriate accounts and **not** apply them to the `debits_posted` and
488
+ `credits_posted` balances.
489
+
490
+ ```javascript
491
+ let transfer4 = {
492
+ id: 4n,
493
+ debit_account_id: 102n,
494
+ credit_account_id: 103n,
495
+ amount: 10n,
496
+ pending_id: 0n,
497
+ user_data_128: 0n,
498
+ user_data_64: 0n,
499
+ user_data_32: 0,
500
+ timeout: 0,
501
+ ledger: 1,
502
+ code: 720,
503
+ flags: TransferFlags.pending,
504
+ timestamp: 0n,
505
+ };
506
+ transferErrors = await client.createTransfers([transfer4]);
507
+
508
+ let transfer5 = {
509
+ id: 7n,
510
+ debit_account_id: 102n,
511
+ credit_account_id: 103n,
512
+ amount: 10n,
513
+ pending_id: 6n,
514
+ user_data_128: 0n,
515
+ user_data_64: 0n,
516
+ user_data_32: 0,
517
+ timeout: 0,
518
+ ledger: 1,
519
+ code: 720,
520
+ flags: TransferFlags.void_pending_transfer,
521
+ timestamp: 0n,
522
+ };
523
+ transferErrors = await client.createTransfers([transfer5]);
524
+ ```
172
525
 
173
- When the `linked` flag is specified for the `createAccount`, `createTransfer`, `commitTransfer` event, it links an event with the next event in the batch, to create a chain of events, of arbitrary length, which all succeed or fail together. The tail of a chain is denoted by the first event without this flag. The last event in a batch may therefore never have the `linked` flag set as this would leave a chain open-ended. Multiple chains or individual events may coexist within a batch to succeed or fail independently. Events within a chain are executed within order, or are rolled back on error, so that the effect of each event in the chain is visible to the next, and so that the chain is either visible or invisible as a unit to subsequent events after the chain. The event that was the first to break the chain will have a unique error result. Other events in the chain will have their error result set to `linked_event_failed`.
526
+ ## Transfer Lookup
527
+
528
+ NOTE: While transfer lookup exists, it is not a flexible query API. We
529
+ are developing query APIs and there will be new methods for querying
530
+ transfers in the future.
531
+
532
+ Transfer lookup is batched, like transfer creation. Pass in all `id`s to
533
+ fetch, and matched transfers are returned.
534
+
535
+ If no transfer matches an `id`, no object is returned for that
536
+ transfer. So the order of transfers in the response is not necessarily
537
+ the same as the order of `id`s in the request. You can refer to the
538
+ `id` field in the response to distinguish transfers.
539
+
540
+ ```javascript
541
+ const transfers = await client.lookupTransfers([1n, 2n]);
542
+ console.log(transfers);
543
+ /*
544
+ * [{
545
+ * id: 1n,
546
+ * debit_account_id: 102n,
547
+ * credit_account_id: 103n,
548
+ * amount: 10n,
549
+ * pending_id: 0n,
550
+ * user_data_128: 0n,
551
+ * user_data_64: 0n,
552
+ * user_data_32: 0,
553
+ * timeout: 0,
554
+ * ledger: 1,
555
+ * code: 720,
556
+ * flags: 0,
557
+ * timestamp: 1623062009212508993n,
558
+ * }]
559
+ */
560
+ ```
174
561
 
175
- ```js
176
- let batch = []
177
- let linkedFlag = 0
178
- linkedFlag |= CreateTransferFlags.linked
562
+ ## Linked Events
563
+
564
+ When the `linked` flag is specified for an account when creating accounts or
565
+ a transfer when creating transfers, it links that event with the next event in the
566
+ batch, to create a chain of events, of arbitrary length, which all
567
+ succeed or fail together. The tail of a chain is denoted by the first
568
+ event without this flag. The last event in a batch may therefore never
569
+ have the `linked` flag set as this would leave a chain
570
+ open-ended. Multiple chains or individual events may coexist within a
571
+ batch to succeed or fail independently.
572
+
573
+ Events within a chain are executed within order, or are rolled back on
574
+ error, so that the effect of each event in the chain is visible to the
575
+ next, and so that the chain is either visible or invisible as a unit
576
+ to subsequent events after the chain. The event that was the first to
577
+ break the chain will have a unique error result. Other events in the
578
+ chain will have their error result set to `linked_event_failed`.
579
+
580
+ ```javascript
581
+ const batch = [];
582
+ let linkedFlag = 0;
583
+ linkedFlag |= TransferFlags.linked;
179
584
 
180
585
  // An individual transfer (successful):
181
- batch.push({ id: 1n, ... })
586
+ batch.push({ id: 1n /* , ... */ });
182
587
 
183
588
  // A chain of 4 transfers (the last transfer in the chain closes the chain with linked=false):
184
- batch.push({ id: 2n, ..., flags: linkedFlag }) // Commit/rollback.
185
- batch.push({ id: 3n, ..., flags: linkedFlag }) // Commit/rollback.
186
- batch.push({ id: 2n, ..., flags: linkedFlag }) // Fail with exists
187
- batch.push({ id: 4n, ..., flags: 0 }) // Fail without committing.
589
+ batch.push({ id: 2n, /* ..., */ flags: linkedFlag }); // Commit/rollback.
590
+ batch.push({ id: 3n, /* ..., */ flags: linkedFlag }); // Commit/rollback.
591
+ batch.push({ id: 2n, /* ..., */ flags: linkedFlag }); // Fail with exists
592
+ batch.push({ id: 4n, /* ..., */ flags: 0 }); // Fail without committing.
188
593
 
189
594
  // An individual transfer (successful):
190
595
  // This should not see any effect from the failed chain above.
191
- batch.push({ id: 2n, ..., flags: 0 })
596
+ batch.push({ id: 2n, /* ..., */ flags: 0 });
192
597
 
193
598
  // A chain of 2 transfers (the first transfer fails the chain):
194
- batch.push({ id: 2n, ..., flags: linkedFlag })
195
- batch.push({ id: 3n, ..., flags: 0 })
599
+ batch.push({ id: 2n, /* ..., */ flags: linkedFlag });
600
+ batch.push({ id: 3n, /* ..., */ flags: 0 });
196
601
 
197
602
  // A chain of 2 transfers (successful):
198
- batch.push({ id: 3n, ..., flags: linkedFlag })
199
- batch.push({ id: 4n, ..., flags: 0 })
603
+ batch.push({ id: 3n, /* ..., */ flags: linkedFlag });
604
+ batch.push({ id: 4n, /* ..., */ flags: 0 });
200
605
 
201
- const errors = await client.createTransfers(batch)
606
+ const errors = await client.createTransfers(batch);
202
607
 
203
608
  /**
609
+ * console.log(errors);
204
610
  * [
205
611
  * { index: 1, error: 1 }, // linked_event_failed
206
612
  * { index: 2, error: 1 }, // linked_event_failed
207
613
  * { index: 3, error: 25 }, // exists
208
614
  * { index: 4, error: 1 }, // linked_event_failed
209
- *
615
+ *
210
616
  * { index: 6, error: 17 }, // exists_with_different_flags
211
617
  * { index: 7, error: 1 }, // linked_event_failed
212
618
  * ]
213
619
  */
214
620
  ```
215
621
 
216
- ### Development
622
+ ## Development Setup
217
623
 
218
- To get up and running when cloning the repo:
624
+ ### On Linux and macOS
219
625
 
220
- ```shell
221
- git clone --recurse-submodules https://github.com/coilhq/tigerbeetle-node.git
222
- cd tigerbeetle-node/
223
- npm install --include dev # This will automatically install and build everything you need.
224
- ```
225
-
226
- #### Rebuild
227
-
228
- To rebuild the TypeScript distribution, and to rebuild the native Node library, again after changes:
626
+ In a POSIX shell run:
229
627
 
230
- ```shell
231
- npm run build
628
+ ```console
629
+ git clone https://github.com/tigerbeetle/tigerbeetle
630
+ cd tigerbeetle
631
+ git submodule update --init --recursive
632
+ ./scripts/install_zig.sh
633
+ cd src/clients/node
634
+ npm install --include dev
635
+ npm pack
232
636
  ```
233
637
 
234
- *If you ever run `npm run clean` then you will need to `npm install --include dev` to reinstall
235
- TypeScript within `node_modules`, as TypeScript is required by `npm run prepack` when publishing.*
236
-
237
- #### Benchmark
238
-
239
- ```shell
240
- npm run benchmark
241
- ```
638
+ ### On Windows
242
639
 
243
- #### Test
640
+ In PowerShell run:
244
641
 
245
- ```shell
246
- ./tigerbeetle init --cluster=1 --replica=0 --directory=.
247
- ./tigerbeetle start --cluster=1 --replica=0 --directory=. --addresses=3001
248
- npm run test
642
+ ```console
643
+ git clone https://github.com/tigerbeetle/tigerbeetle
644
+ cd tigerbeetle
645
+ git submodule update --init --recursive
646
+ .\scripts\install_zig.bat
647
+ cd src/clients/node
648
+ npm install --include dev
649
+ npm pack
249
650
  ```