tigerbeetle-node 0.8.1 → 0.9.143

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (83) hide show
  1. package/README.md +584 -184
  2. package/dist/benchmark.js +59 -51
  3. package/dist/benchmark.js.map +1 -1
  4. package/dist/bin/aarch64-linux-gnu/client.node +0 -0
  5. package/dist/bin/aarch64-linux-musl/client.node +0 -0
  6. package/dist/bin/aarch64-macos/client.node +0 -0
  7. package/dist/bin/x86_64-linux-gnu/client.node +0 -0
  8. package/dist/bin/x86_64-linux-musl/client.node +0 -0
  9. package/dist/bin/x86_64-macos/client.node +0 -0
  10. package/dist/bin/x86_64-windows/client.node +0 -0
  11. package/dist/bindings.d.ts +141 -0
  12. package/dist/bindings.js +112 -0
  13. package/dist/bindings.js.map +1 -0
  14. package/dist/index.d.ts +2 -125
  15. package/dist/index.js +51 -101
  16. package/dist/index.js.map +1 -1
  17. package/dist/test.js +69 -55
  18. package/dist/test.js.map +1 -1
  19. package/package-lock.json +26 -0
  20. package/package.json +17 -28
  21. package/src/benchmark.ts +58 -49
  22. package/src/bindings.ts +631 -0
  23. package/src/index.ts +71 -163
  24. package/src/node.zig +169 -148
  25. package/src/test.ts +71 -57
  26. package/src/translate.zig +19 -36
  27. package/.yarn/releases/yarn-berry.cjs +0 -55
  28. package/.yarnrc.yml +0 -1
  29. package/scripts/download_node_headers.sh +0 -25
  30. package/scripts/postinstall.sh +0 -6
  31. package/src/tigerbeetle/scripts/benchmark.bat +0 -46
  32. package/src/tigerbeetle/scripts/benchmark.sh +0 -55
  33. package/src/tigerbeetle/scripts/install.sh +0 -6
  34. package/src/tigerbeetle/scripts/install_zig.bat +0 -109
  35. package/src/tigerbeetle/scripts/install_zig.sh +0 -84
  36. package/src/tigerbeetle/scripts/lint.zig +0 -199
  37. package/src/tigerbeetle/scripts/upgrade_ubuntu_kernel.sh +0 -39
  38. package/src/tigerbeetle/scripts/vopr.bat +0 -48
  39. package/src/tigerbeetle/scripts/vopr.sh +0 -33
  40. package/src/tigerbeetle/scripts/vr_state_enumerate +0 -46
  41. package/src/tigerbeetle/src/benchmark.zig +0 -290
  42. package/src/tigerbeetle/src/cli.zig +0 -244
  43. package/src/tigerbeetle/src/config.zig +0 -239
  44. package/src/tigerbeetle/src/demo.zig +0 -125
  45. package/src/tigerbeetle/src/demo_01_create_accounts.zig +0 -35
  46. package/src/tigerbeetle/src/demo_02_lookup_accounts.zig +0 -7
  47. package/src/tigerbeetle/src/demo_03_create_transfers.zig +0 -24
  48. package/src/tigerbeetle/src/demo_04_create_pending_transfers.zig +0 -61
  49. package/src/tigerbeetle/src/demo_05_post_pending_transfers.zig +0 -37
  50. package/src/tigerbeetle/src/demo_06_void_pending_transfers.zig +0 -24
  51. package/src/tigerbeetle/src/demo_07_lookup_transfers.zig +0 -7
  52. package/src/tigerbeetle/src/fifo.zig +0 -104
  53. package/src/tigerbeetle/src/io/benchmark.zig +0 -213
  54. package/src/tigerbeetle/src/io/darwin.zig +0 -793
  55. package/src/tigerbeetle/src/io/linux.zig +0 -1038
  56. package/src/tigerbeetle/src/io/test.zig +0 -643
  57. package/src/tigerbeetle/src/io/windows.zig +0 -1161
  58. package/src/tigerbeetle/src/io.zig +0 -34
  59. package/src/tigerbeetle/src/main.zig +0 -144
  60. package/src/tigerbeetle/src/message_bus.zig +0 -1000
  61. package/src/tigerbeetle/src/message_pool.zig +0 -142
  62. package/src/tigerbeetle/src/ring_buffer.zig +0 -289
  63. package/src/tigerbeetle/src/simulator.zig +0 -417
  64. package/src/tigerbeetle/src/state_machine.zig +0 -2470
  65. package/src/tigerbeetle/src/storage.zig +0 -308
  66. package/src/tigerbeetle/src/test/cluster.zig +0 -351
  67. package/src/tigerbeetle/src/test/message_bus.zig +0 -93
  68. package/src/tigerbeetle/src/test/network.zig +0 -179
  69. package/src/tigerbeetle/src/test/packet_simulator.zig +0 -387
  70. package/src/tigerbeetle/src/test/state_checker.zig +0 -145
  71. package/src/tigerbeetle/src/test/state_machine.zig +0 -76
  72. package/src/tigerbeetle/src/test/storage.zig +0 -438
  73. package/src/tigerbeetle/src/test/time.zig +0 -84
  74. package/src/tigerbeetle/src/tigerbeetle.zig +0 -222
  75. package/src/tigerbeetle/src/time.zig +0 -113
  76. package/src/tigerbeetle/src/unit_tests.zig +0 -14
  77. package/src/tigerbeetle/src/vsr/client.zig +0 -505
  78. package/src/tigerbeetle/src/vsr/clock.zig +0 -812
  79. package/src/tigerbeetle/src/vsr/journal.zig +0 -2293
  80. package/src/tigerbeetle/src/vsr/marzullo.zig +0 -309
  81. package/src/tigerbeetle/src/vsr/replica.zig +0 -5015
  82. package/src/tigerbeetle/src/vsr.zig +0 -1017
  83. package/yarn.lock +0 -42
package/README.md CHANGED
@@ -1,250 +1,650 @@
1
+ ---
2
+ title: Node.js
3
+ ---
4
+
5
+ This file is generated by
6
+ [/src/clients/docs_generate.zig](/src/clients/docs_generate.zig).
1
7
  # tigerbeetle-node
2
- [TigerBeetle](https://github.com/coilhq/tigerbeetle) client for Node.js.
3
8
 
4
- ## Installation
5
- The following steps will install the `tigerbeetle-node` module to your current working directory.
9
+ The TigerBeetle client for Node.js.
6
10
 
7
- ### Prerequisites
11
+ ### Prerequisites
8
12
 
9
- * NodeJS >= `14.0.0`. _(If the correct version is not installed, an installation error will occur)_
13
+ Linux >= 5.6 is the only production environment we
14
+ support. But for ease of development we also support macOS and Windows.
15
+ * NodeJS >= `14`
10
16
 
11
- > Your operating system should be Linux (kernel >= v5.6) or macOS. Windows support is not yet available but is in the works.
17
+ ## Setup
12
18
 
13
- ### YARN Package Manager Installation
14
- ```shell
15
- yarn add tigerbeetle-node
16
- ```
17
- or
18
- ### NPM Package Manager Installation
19
- ```shell
19
+ First, create a directory for your project and `cd` into the directory.
20
+
21
+ Then, install the TigerBeetle client:
22
+
23
+ ```console
20
24
  npm install tigerbeetle-node
21
25
  ```
22
26
 
23
- ## Usage
24
- A client needs to be configured with a `cluster_id` and `replica_addresses`.
25
- This instantiates the client where memory is allocated to internally buffer events to be sent.
26
- For the moment, only one client can be instantiated globally per process.
27
- Future releases will allow multiple client instantiations.
28
- ```js
29
- import { createClient } from 'tigerbeetle-node'
27
+ Now, create `main.js` and copy this into it:
28
+
29
+ ```javascript
30
+ const { createClient } = require("tigerbeetle-node");
31
+ console.log("Import ok!");
32
+ ```
33
+
34
+ Finally, build and run:
35
+
36
+ ```console
37
+ node main.js
38
+ ```
39
+
40
+ Now that all prerequisites and dependencies are correctly set
41
+ up, let's dig into using TigerBeetle.
42
+
43
+ If you run into issues, check out the distribution-specific install
44
+ steps that are run in CI to test support:
45
+
46
+ * [Alpine](https://github.com/tigerbeetle/tigerbeetle/blob/main/src/clients/node/scripts/test_install_on_alpine.sh)
47
+ * [Amazon Linux](https://github.com/tigerbeetle/tigerbeetle/blob/main/src/clients/node/scripts/test_install_on_amazonlinux.sh)
48
+ * [Debian](https://github.com/tigerbeetle/tigerbeetle/blob/main/src/clients/node/scripts/test_install_on_debian.sh)
49
+ * [Fedora](https://github.com/tigerbeetle/tigerbeetle/blob/main/src/clients/node/scripts/test_install_on_fedora.sh)
50
+ * [Ubuntu](https://github.com/tigerbeetle/tigerbeetle/blob/main/src/clients/node/scripts/test_install_on_ubuntu.sh)
51
+ * [RHEL](https://github.com/tigerbeetle/tigerbeetle/blob/main/src/clients/node/scripts/test_install_on_rhelubi.sh)
52
+
53
+ ## Sample projects
30
54
 
55
+ This document is primarily a reference guide to
56
+ the client. Below are various sample projects demonstrating
57
+ features of TigerBeetle.
58
+
59
+ * [Basic](/src/clients/node/samples/basic/): Create two accounts and transfer an amount between them.
60
+ * [Two-Phase Transfer](/src/clients/node/samples/two-phase/): Create two accounts and start a pending transfer between
61
+ them, then post the transfer.
62
+ * [Many Two-Phase Transfers](/src/clients/node/samples/two-phase-many/): Create two accounts and start a number of pending transfer
63
+ between them, posting and voiding alternating transfers.
64
+ ### Sidenote: `BigInt`
65
+ TigerBeetle uses 64-bit integers for many fields while JavaScript's
66
+ builtin `Number` maximum value is `2^53-1`. The `n` suffix in JavaScript
67
+ means the value is a `BigInt`. This is useful for literal numbers. If
68
+ you already have a `Number` variable though, you can call the `BigInt`
69
+ constructor to get a `BigInt` from it. For example, `1n` is the same as
70
+ `BigInt(1)`.
71
+
72
+ ## Creating a Client
73
+
74
+ A client is created with a cluster ID and replica
75
+ addresses for all replicas in the cluster. The cluster
76
+ ID and replica addresses are both chosen by the system that
77
+ starts the TigerBeetle cluster.
78
+
79
+ Clients are thread-safe and a single instance should be shared
80
+ between multiple concurrent tasks.
81
+
82
+ Multiple clients are useful when connecting to more than
83
+ one TigerBeetle cluster.
84
+
85
+ In this example the cluster ID is `0` and there is one
86
+ replica. The address is read from the `TB_ADDRESS`
87
+ environment variable and defaults to port `3000`.
88
+
89
+ ```javascript
31
90
  const client = createClient({
32
91
  cluster_id: 0,
33
- replica_addresses: ['3001', '3002', '3003']
34
- })
35
- ```
36
-
37
- One of the ways TigerBeetle achieves its performance is through batching.
38
- This is reflected in the below function interfaces where each one takes in an array of events.
39
-
40
- ### Account Creation
41
-
42
- ```js
43
- const account = {
44
- id: 137n, // u128
45
- user_data: 0n, // u128, opaque third-party identifier to link this account to an external entity:
46
- reserved: Buffer.alloc(48, 0), // [48]u8
47
- ledger: 1, // u32, ledger value
48
- code: 718, // u16, a chart of accounts code describing the type of account (e.g. clearing, settlement)
49
- flags: 0, // u16
50
- debits_pending: 0n, // u64
51
- debits_posted: 0n, // u64
52
- credits_pending: 0n, // u64
53
- credits_posted: 0n, // u64
54
- timestamp: 0n, // u64, Reserved: This will be set by the server.
55
- }
56
-
57
- const errors = await client.createAccounts([account])
92
+ replica_addresses: [process.env.TB_ADDRESS || '3000']
93
+ });
58
94
  ```
59
- Successfully executed events return an empty array whilst unsuccessful ones return an array with errors for **only the ones that failed**. An error will point to the index in the submitted array of the failed event.
60
- ```js
61
- const errors = await client.createAccounts([account1, account2, account3])
62
95
 
63
- // errors = [{ index: 1, code: 1 }]
64
- const error = errors[0]
65
- switch (error.code) {
66
- case CreateAccountError.exists: {
67
- console.error(`Batch event at ${error.index} already exists.`)
68
- }
69
- }
96
+ The following are valid addresses:
97
+ * `3000` (interpreted as `127.0.0.1:3000`)
98
+ * `127.0.0.1:3000` (interpreted as `127.0.0.1:3000`)
99
+ * `127.0.0.1` (interpreted as `127.0.0.1:3001`, `3001` is the default port)
100
+
101
+ ## Creating Accounts
102
+
103
+ See details for account fields in the [Accounts
104
+ reference](https://docs.tigerbeetle.com/reference/accounts).
105
+
106
+ ```javascript
107
+ let account = {
108
+ id: 137n,
109
+ debits_pending: 0n,
110
+ debits_posted: 0n,
111
+ credits_pending: 0n,
112
+ credits_posted: 0n,
113
+ user_data_128: 0n,
114
+ user_data_64: 0n,
115
+ user_data_32: 0,
116
+ reserved: 0,
117
+ ledger: 1,
118
+ code: 718,
119
+ flags: 0,
120
+ timestamp: 0n,
121
+ };
122
+
123
+ let accountErrors = await client.createAccounts([account]);
70
124
  ```
71
- The example above shows that the event in index 1 failed with error 1. This means that `account1` and `account3` were created successfully but not `account2`.
72
125
 
73
- The `flags` on an account provide a way for you to enforce policies by toggling the bits below.
74
- | bit 0 | bit 1 | bit 2 |
75
- |----------|----------------------------------|-----------------------------------------|
76
- | `linked` | `debits_must_not_exceed_credits` | `credits_must_not_exceed_debits` |
126
+ ### Account Flags
127
+
128
+ The account flags value is a bitfield. See details for
129
+ these flags in the [Accounts
130
+ reference](https://docs.tigerbeetle.com/reference/accounts#flags).
131
+
132
+ To toggle behavior for an account, combine enum values stored in the
133
+ `AccountFlags` object (in TypeScript it is an actual enum) with
134
+ bitwise-or:
135
+
136
+ * `AccountFlags.linked`
137
+ * `AccountFlags.debits_must_not_exceed_credits`
138
+ * `AccountFlags.credits_must_not_exceed_credits`
139
+
140
+
141
+ For example, to link two accounts where the first account
142
+ additionally has the `debits_must_not_exceed_credits` constraint:
143
+
144
+ ```javascript
145
+ let account0 = {
146
+ id: 100n,
147
+ debits_pending: 0n,
148
+ debits_posted: 0n,
149
+ credits_pending: 0n,
150
+ credits_posted: 0n,
151
+ user_data_128: 0n,
152
+ user_data_64: 0n,
153
+ user_data_32: 0,
154
+ reserved: 0,
155
+ ledger: 1,
156
+ code: 1,
157
+ timestamp: 0n,
158
+ flags: 0,
159
+ };
160
+ let account1 = {
161
+ id: 101n,
162
+ debits_pending: 0n,
163
+ debits_posted: 0n,
164
+ credits_pending: 0n,
165
+ credits_posted: 0n,
166
+ user_data_128: 0n,
167
+ user_data_64: 0n,
168
+ user_data_32: 0,
169
+ reserved: 0,
170
+ ledger: 1,
171
+ code: 1,
172
+ timestamp: 0n,
173
+ flags: 0,
174
+ };
175
+ account0.flags = AccountFlags.linked | AccountFlags.debits_must_not_exceed_credits;
176
+ accountErrors = await client.createAccounts([account0, account1]);
177
+ ```
77
178
 
78
- The creation of an account can be linked to the successful creation of another by setting the `linked` flag (see [linked events](#linked-events)). By setting `debits_must_not_exceed_credits`, then any transfer such that `debits_posted + debits_pending + amount > credits_posted` will fail. Similarly for `credits_must_not_exceed_debits`.
79
- ```js
80
- enum CreateAccountFlags {
81
- linked = (1 << 0),
82
- debits_must_not_exceed_credits = (1 << 1),
83
- credits_must_not_exceed_debits = (1 << 2)
179
+ ### Response and Errors
180
+
181
+ The response is an empty array if all accounts were
182
+ created successfully. If the response is non-empty, each
183
+ object in the response array contains error information
184
+ for an account that failed. The error object contains an
185
+ error code and the index of the account in the request
186
+ batch.
187
+
188
+ See all error conditions in the [create_accounts
189
+ reference](https://docs.tigerbeetle.com/reference/operations/create_accounts).
190
+
191
+ ```javascript
192
+ let account2 = {
193
+ id: 102n,
194
+ debits_pending: 0n,
195
+ debits_posted: 0n,
196
+ credits_pending: 0n,
197
+ credits_posted: 0n,
198
+ user_data_128: 0n,
199
+ user_data_64: 0n,
200
+ user_data_32: 0,
201
+ reserved: 0,
202
+ ledger: 1,
203
+ code: 1,
204
+ timestamp: 0n,
205
+ flags: 0,
206
+ };
207
+ let account3 = {
208
+ id: 103n,
209
+ debits_pending: 0n,
210
+ debits_posted: 0n,
211
+ credits_pending: 0n,
212
+ credits_posted: 0n,
213
+ user_data_128: 0n,
214
+ user_data_64: 0n,
215
+ user_data_32: 0,
216
+ reserved: 0,
217
+ ledger: 1,
218
+ code: 1,
219
+ timestamp: 0n,
220
+ flags: 0,
221
+ };
222
+ let account4 = {
223
+ id: 104n,
224
+ debits_pending: 0n,
225
+ debits_posted: 0n,
226
+ credits_pending: 0n,
227
+ credits_posted: 0n,
228
+ user_data_128: 0n,
229
+ user_data_64: 0n,
230
+ user_data_32: 0,
231
+ reserved: 0,
232
+ ledger: 1,
233
+ code: 1,
234
+ timestamp: 0n,
235
+ flags: 0,
236
+ };
237
+ accountErrors = await client.createAccounts([account2, account3, account4]);
238
+ for (const error of accountErrors) {
239
+ switch (error.result) {
240
+ case CreateAccountError.exists:
241
+ console.error(`Batch account at ${error.index} already exists.`);
242
+ break;
243
+ default:
244
+ console.error(`Batch account at ${error.index} failed to create: ${CreateAccountError[error.result]}.`);
84
245
  }
246
+ }
247
+ ```
85
248
 
86
- let flags = 0
87
- flags |= CreateAccountFlags.debits_must_not_exceed_credits
249
+ To handle errors you can either 1) exactly match error codes returned
250
+ from `client.createAccounts` with enum values in the
251
+ `CreateAccountError` object, or you can 2) look up the error code in
252
+ the `CreateAccountError` object for a human-readable string.
253
+
254
+ ## Account Lookup
255
+
256
+ Account lookup is batched, like account creation. Pass
257
+ in all IDs to fetch. The account for each matched ID is returned.
258
+
259
+ If no account matches an ID, no object is returned for
260
+ that account. So the order of accounts in the response is
261
+ not necessarily the same as the order of IDs in the
262
+ request. You can refer to the ID field in the response to
263
+ distinguish accounts.
264
+
265
+ ```javascript
266
+ const accounts = await client.lookupAccounts([137n, 138n]);
267
+ console.log(accounts);
268
+ /*
269
+ * [{
270
+ * id: 137n,
271
+ * debits_pending: 0n,
272
+ * debits_posted: 0n,
273
+ * credits_pending: 0n,
274
+ * credits_posted: 0n,
275
+ * user_data_128: 0n,
276
+ * user_data_64: 0n,
277
+ * user_data_32: 0,
278
+ * reserved: 0,
279
+ * ledger: 1,
280
+ * code: 718,
281
+ * flags: 0,
282
+ * timestamp: 1623062009212508993n,
283
+ * }]
284
+ */
88
285
  ```
89
286
 
90
- ### Account Lookup
287
+ ## Create Transfers
288
+
289
+ This creates a journal entry between two accounts.
91
290
 
92
- The `id` of the account is used for lookups. Only matched accounts are returned.
93
- ```js
94
- // account 137n exists, 138n does not
95
- const accounts = await client.lookupAccounts([137n, 138n])
291
+ See details for transfer fields in the [Transfers
292
+ reference](https://docs.tigerbeetle.com/reference/transfers).
293
+
294
+ ```javascript
295
+ let transfer = {
296
+ id: 1n,
297
+ debit_account_id: 102n,
298
+ credit_account_id: 103n,
299
+ amount: 10n,
300
+ pending_id: 0n,
301
+ user_data_128: 0n,
302
+ user_data_64: 0n,
303
+ user_data_32: 0,
304
+ timeout: 0,
305
+ ledger: 1,
306
+ code: 720,
307
+ flags: 0,
308
+ timestamp: 0n,
309
+ };
310
+ let transferErrors = await client.createTransfers([transfer]);
311
+ ```
96
312
 
97
- /**
98
- * const accounts = [{
99
- * id: 137n,
100
- * user_data: 0n,
101
- * reserved: Buffer,
102
- * ledger: 1,
103
- * code: 718,
104
- * flags: 0,
105
- * debits_pending: 0n,
106
- * debits_posted: 0n,
107
- * credits_pending: 0n,
108
- * credits_posted: 0n,
109
- * timestamp: 1623062009212508993n,
110
- * }]
111
- */
313
+ ### Response and Errors
314
+
315
+ The response is an empty array if all transfers were created
316
+ successfully. If the response is non-empty, each object in the
317
+ response array contains error information for a transfer that
318
+ failed. The error object contains an error code and the index of the
319
+ transfer in the request batch.
320
+
321
+ See all error conditions in the [create_transfers
322
+ reference](https://docs.tigerbeetle.com/reference/operations/create_transfers).
323
+
324
+ ```javascript
325
+ for (const error of transferErrors) {
326
+ switch (error.result) {
327
+ case CreateTransferError.exists:
328
+ console.error(`Batch transfer at ${error.index} already exists.`);
329
+ break;
330
+ default:
331
+ console.error(`Batch transfer at ${error.index} failed to create: ${CreateTransferError[error.result]}.`);
332
+ }
333
+ }
112
334
  ```
113
335
 
114
- ### Creating a Transfer
336
+ To handle errors you can either 1) exactly match error codes returned
337
+ from `client.createTransfers` with enum values in the
338
+ `CreateTransferError` object, or you can 2) look up the error code in
339
+ the `CreateTransferError` object for a human-readable string.
115
340
 
116
- This creates a journal entry between two accounts.
117
- ```js
118
- const transfer = {
119
- id: 1n, // u128
120
- // Double-entry accounting:
121
- debit_account_id: 1n, // u128
122
- credit_account_id: 2n, // u128
123
- // Opaque third-party identifier to link this transfer to an external entity:
124
- user_data: 0n, // u128
125
- reserved: 0n, // u128
126
- // Timeout applicable for a pending/2-phase transfer:
127
- timeout: 0n, // u64, in nano-seconds.
128
- // Collection of accounts usually grouped by the currency:
129
- // You can't transfer money between accounts with different ledgers:
130
- ledger: 720, // u32, ledger for transfer (e.g. currency).
131
- // Chart of accounts code describing the reason for the transfer:
132
- code: 1, // u16, (e.g. deposit, settlement)
133
- flags: 0, // u16
134
- amount: 10n, // u64
135
- timestamp: 0n, //u64, Reserved: This will be set by the server.
341
+ ## Batching
342
+
343
+ TigerBeetle performance is maximized when you batch
344
+ API requests. The client does not do this automatically for
345
+ you. So, for example, you *can* insert 1 million transfers
346
+ one at a time like so:
347
+
348
+ ```javascript
349
+ for (let i = 0; i < transfers.len; i++) {
350
+ const transferErrors = await client.createTransfers(transfers[i]);
351
+ // error handling omitted
136
352
  }
137
- const errors = await client.createTransfers([transfer])
138
353
  ```
139
- Two-phase transfers are supported natively by toggling the appropriate flag. TigerBeetle will then adjust the `credits_pending` and `debits_pending` fields of the appropriate accounts. A corresponding commit transfer then needs to be sent to accept or reject the transfer.
140
354
 
141
- Transfers within a batch may also be linked (see [linked events](#linked-events)).
142
- ```js
143
- enum TransferFlags {
144
- linked = (1 << 0),
145
- pending = (1 << 1),
146
- post_pending_transfer = (1 << 2),
147
- void_pending_transfer = (1 << 3)
148
- }
149
-
150
- // Two-phase transfer (pending):
151
- let flags = 0n
152
- flags |= TransferFlags.pending
153
-
154
- // Linked two-phase transfer (pending):
155
- let flags = 0n
156
- flags |= TransferFlags.linked
157
- flags |= TransferFlags.pending
158
- ```
159
-
160
- ### Post a Pending Transfer (2-Phase)
161
-
162
- With `flags = post_pending_transfer`, TigerBeetle will accept the transfer. TigerBeetle will atomically rollback the changes to `debits_pending` and `credits_pending` of the appropriate accounts and apply them to the `debits_posted` and `credits_posted` balances.
163
- ```js
164
- const post = {
165
- id: 2n, // u128, must correspond to the transfer id
166
- pending_id: 1n, // u128, id of the pending transfer
167
- flags: TransferFlags.post_pending_transfer, // to void, use [void_pending_transfer]
168
- timestamp: 0n, // u64, Reserved: This will be set by the server.
355
+ But the insert rate will be a *fraction* of
356
+ potential. Instead, **always batch what you can**.
357
+
358
+ The maximum batch size is set in the TigerBeetle server. The default
359
+ is 8191.
360
+
361
+ ```javascript
362
+ const BATCH_SIZE = 8191;
363
+ for (let i = 0; i < transfers.length; i += BATCH_SIZE) {
364
+ const transferErrors = await client.createTransfers(transfers.slice(i, Math.min(transfers.length, BATCH_SIZE)));
365
+ // error handling omitted
169
366
  }
170
- const errors = await client.createTransfers([post])
171
367
  ```
172
368
 
173
- ### Linked Events
369
+ ### Queues and Workers
370
+
371
+ If you are making requests to TigerBeetle from workers
372
+ pulling jobs from a queue, you can batch requests to
373
+ TigerBeetle by having the worker act on multiple jobs from
374
+ the queue at once rather than one at a time. i.e. pulling
375
+ multiple jobs from the queue rather than just one.
376
+
377
+ ## Transfer Flags
378
+
379
+ The transfer `flags` value is a bitfield. See details for these flags in
380
+ the [Transfers
381
+ reference](https://docs.tigerbeetle.com/reference/transfers#flags).
382
+
383
+ To toggle behavior for a transfer, combine enum values stored in the
384
+ `TransferFlags` object (in TypeScript it is an actual enum) with
385
+ bitwise-or:
386
+
387
+ * `TransferFlags.linked`
388
+ * `TransferFlags.pending`
389
+ * `TransferFlags.post_pending_transfer`
390
+ * `TransferFlags.void_pending_transfer`
391
+
392
+ For example, to link `transfer0` and `transfer1`:
393
+
394
+ ```javascript
395
+ let transfer0 = {
396
+ id: 2n,
397
+ debit_account_id: 102n,
398
+ credit_account_id: 103n,
399
+ amount: 10n,
400
+ pending_id: 0n,
401
+ user_data_128: 0n,
402
+ user_data_64: 0n,
403
+ user_data_32: 0,
404
+ timeout: 0,
405
+ ledger: 1,
406
+ code: 720,
407
+ flags: 0,
408
+ timestamp: 0n,
409
+ };
410
+ let transfer1 = {
411
+ id: 3n,
412
+ debit_account_id: 102n,
413
+ credit_account_id: 103n,
414
+ amount: 10n,
415
+ pending_id: 0n,
416
+ user_data_128: 0n,
417
+ user_data_64: 0n,
418
+ user_data_32: 0,
419
+ timeout: 0,
420
+ ledger: 1,
421
+ code: 720,
422
+ flags: 0,
423
+ timestamp: 0n,
424
+ };
425
+ transfer0.flags = TransferFlags.linked;
426
+ // Create the transfer
427
+ transferErrors = await client.createTransfers([transfer0, transfer1]);
428
+ ```
429
+
430
+ ### Two-Phase Transfers
431
+
432
+ Two-phase transfers are supported natively by toggling the appropriate
433
+ flag. TigerBeetle will then adjust the `credits_pending` and
434
+ `debits_pending` fields of the appropriate accounts. A corresponding
435
+ post pending transfer then needs to be sent to post or void the
436
+ transfer.
437
+
438
+ #### Post a Pending Transfer
439
+
440
+ With `flags` set to `post_pending_transfer`,
441
+ TigerBeetle will post the transfer. TigerBeetle will atomically roll
442
+ back the changes to `debits_pending` and `credits_pending` of the
443
+ appropriate accounts and apply them to the `debits_posted` and
444
+ `credits_posted` balances.
445
+
446
+ ```javascript
447
+ let transfer2 = {
448
+ id: 4n,
449
+ debit_account_id: 102n,
450
+ credit_account_id: 103n,
451
+ amount: 10n,
452
+ pending_id: 0n,
453
+ user_data_128: 0n,
454
+ user_data_64: 0n,
455
+ user_data_32: 0,
456
+ timeout: 0,
457
+ ledger: 1,
458
+ code: 720,
459
+ flags: TransferFlags.pending,
460
+ timestamp: 0n,
461
+ };
462
+ transferErrors = await client.createTransfers([transfer2]);
463
+
464
+ let transfer3 = {
465
+ id: 5n,
466
+ debit_account_id: 102n,
467
+ credit_account_id: 103n,
468
+ amount: 10n,
469
+ pending_id: 4n,
470
+ user_data_128: 0n,
471
+ user_data_64: 0n,
472
+ user_data_32: 0,
473
+ timeout: 0,
474
+ ledger: 1,
475
+ code: 720,
476
+ flags: TransferFlags.post_pending_transfer,
477
+ timestamp: 0n,
478
+ };
479
+ transferErrors = await client.createTransfers([transfer3]);
480
+ ```
481
+
482
+ #### Void a Pending Transfer
483
+
484
+ In contrast, with `flags` set to `void_pending_transfer`,
485
+ TigerBeetle will void the transfer. TigerBeetle will roll
486
+ back the changes to `debits_pending` and `credits_pending` of the
487
+ appropriate accounts and **not** apply them to the `debits_posted` and
488
+ `credits_posted` balances.
489
+
490
+ ```javascript
491
+ let transfer4 = {
492
+ id: 4n,
493
+ debit_account_id: 102n,
494
+ credit_account_id: 103n,
495
+ amount: 10n,
496
+ pending_id: 0n,
497
+ user_data_128: 0n,
498
+ user_data_64: 0n,
499
+ user_data_32: 0,
500
+ timeout: 0,
501
+ ledger: 1,
502
+ code: 720,
503
+ flags: TransferFlags.pending,
504
+ timestamp: 0n,
505
+ };
506
+ transferErrors = await client.createTransfers([transfer4]);
507
+
508
+ let transfer5 = {
509
+ id: 7n,
510
+ debit_account_id: 102n,
511
+ credit_account_id: 103n,
512
+ amount: 10n,
513
+ pending_id: 6n,
514
+ user_data_128: 0n,
515
+ user_data_64: 0n,
516
+ user_data_32: 0,
517
+ timeout: 0,
518
+ ledger: 1,
519
+ code: 720,
520
+ flags: TransferFlags.void_pending_transfer,
521
+ timestamp: 0n,
522
+ };
523
+ transferErrors = await client.createTransfers([transfer5]);
524
+ ```
174
525
 
175
- When the `linked` flag is specified for the `createAccount`, `createTransfer`, `commitTransfer` event, it links an event with the next event in the batch, to create a chain of events, of arbitrary length, which all succeed or fail together. The tail of a chain is denoted by the first event without this flag. The last event in a batch may therefore never have the `linked` flag set as this would leave a chain open-ended. Multiple chains or individual events may coexist within a batch to succeed or fail independently. Events within a chain are executed within order, or are rolled back on error, so that the effect of each event in the chain is visible to the next, and so that the chain is either visible or invisible as a unit to subsequent events after the chain. The event that was the first to break the chain will have a unique error result. Other events in the chain will have their error result set to `linked_event_failed`.
526
+ ## Transfer Lookup
527
+
528
+ NOTE: While transfer lookup exists, it is not a flexible query API. We
529
+ are developing query APIs and there will be new methods for querying
530
+ transfers in the future.
531
+
532
+ Transfer lookup is batched, like transfer creation. Pass in all `id`s to
533
+ fetch, and matched transfers are returned.
534
+
535
+ If no transfer matches an `id`, no object is returned for that
536
+ transfer. So the order of transfers in the response is not necessarily
537
+ the same as the order of `id`s in the request. You can refer to the
538
+ `id` field in the response to distinguish transfers.
539
+
540
+ ```javascript
541
+ const transfers = await client.lookupTransfers([1n, 2n]);
542
+ console.log(transfers);
543
+ /*
544
+ * [{
545
+ * id: 1n,
546
+ * debit_account_id: 102n,
547
+ * credit_account_id: 103n,
548
+ * amount: 10n,
549
+ * pending_id: 0n,
550
+ * user_data_128: 0n,
551
+ * user_data_64: 0n,
552
+ * user_data_32: 0,
553
+ * timeout: 0,
554
+ * ledger: 1,
555
+ * code: 720,
556
+ * flags: 0,
557
+ * timestamp: 1623062009212508993n,
558
+ * }]
559
+ */
560
+ ```
176
561
 
177
- ```js
178
- let batch = []
179
- let linkedFlag = 0
180
- linkedFlag |= CreateTransferFlags.linked
562
+ ## Linked Events
563
+
564
+ When the `linked` flag is specified for an account when creating accounts or
565
+ a transfer when creating transfers, it links that event with the next event in the
566
+ batch, to create a chain of events, of arbitrary length, which all
567
+ succeed or fail together. The tail of a chain is denoted by the first
568
+ event without this flag. The last event in a batch may therefore never
569
+ have the `linked` flag set as this would leave a chain
570
+ open-ended. Multiple chains or individual events may coexist within a
571
+ batch to succeed or fail independently.
572
+
573
+ Events within a chain are executed within order, or are rolled back on
574
+ error, so that the effect of each event in the chain is visible to the
575
+ next, and so that the chain is either visible or invisible as a unit
576
+ to subsequent events after the chain. The event that was the first to
577
+ break the chain will have a unique error result. Other events in the
578
+ chain will have their error result set to `linked_event_failed`.
579
+
580
+ ```javascript
581
+ const batch = [];
582
+ let linkedFlag = 0;
583
+ linkedFlag |= TransferFlags.linked;
181
584
 
182
585
  // An individual transfer (successful):
183
- batch.push({ id: 1n, ... })
586
+ batch.push({ id: 1n /* , ... */ });
184
587
 
185
588
  // A chain of 4 transfers (the last transfer in the chain closes the chain with linked=false):
186
- batch.push({ id: 2n, ..., flags: linkedFlag }) // Commit/rollback.
187
- batch.push({ id: 3n, ..., flags: linkedFlag }) // Commit/rollback.
188
- batch.push({ id: 2n, ..., flags: linkedFlag }) // Fail with exists
189
- batch.push({ id: 4n, ..., flags: 0 }) // Fail without committing.
589
+ batch.push({ id: 2n, /* ..., */ flags: linkedFlag }); // Commit/rollback.
590
+ batch.push({ id: 3n, /* ..., */ flags: linkedFlag }); // Commit/rollback.
591
+ batch.push({ id: 2n, /* ..., */ flags: linkedFlag }); // Fail with exists
592
+ batch.push({ id: 4n, /* ..., */ flags: 0 }); // Fail without committing.
190
593
 
191
594
  // An individual transfer (successful):
192
595
  // This should not see any effect from the failed chain above.
193
- batch.push({ id: 2n, ..., flags: 0 })
596
+ batch.push({ id: 2n, /* ..., */ flags: 0 });
194
597
 
195
598
  // A chain of 2 transfers (the first transfer fails the chain):
196
- batch.push({ id: 2n, ..., flags: linkedFlag })
197
- batch.push({ id: 3n, ..., flags: 0 })
599
+ batch.push({ id: 2n, /* ..., */ flags: linkedFlag });
600
+ batch.push({ id: 3n, /* ..., */ flags: 0 });
198
601
 
199
602
  // A chain of 2 transfers (successful):
200
- batch.push({ id: 3n, ..., flags: linkedFlag })
201
- batch.push({ id: 4n, ..., flags: 0 })
603
+ batch.push({ id: 3n, /* ..., */ flags: linkedFlag });
604
+ batch.push({ id: 4n, /* ..., */ flags: 0 });
202
605
 
203
- const errors = await client.createTransfers(batch)
606
+ const errors = await client.createTransfers(batch);
204
607
 
205
608
  /**
609
+ * console.log(errors);
206
610
  * [
207
611
  * { index: 1, error: 1 }, // linked_event_failed
208
612
  * { index: 2, error: 1 }, // linked_event_failed
209
613
  * { index: 3, error: 25 }, // exists
210
614
  * { index: 4, error: 1 }, // linked_event_failed
211
- *
615
+ *
212
616
  * { index: 6, error: 17 }, // exists_with_different_flags
213
617
  * { index: 7, error: 1 }, // linked_event_failed
214
618
  * ]
215
619
  */
216
620
  ```
217
621
 
218
- ### Development
622
+ ## Development Setup
219
623
 
220
- Follow these steps to get up and running when cloning the repo:
221
- ```shell
222
- git clone --recurse-submodules https://github.com/coilhq/tigerbeetle-node.git
223
- cd tigerbeetle-node/
224
- yarn install --immutable
225
- ```
624
+ ### On Linux and macOS
226
625
 
227
- Build locally using `yarn`:
228
- ```shell
229
- # Run the following from this directory:
230
- yarn && yarn build
231
- ```
626
+ In a POSIX shell run:
232
627
 
233
- * **Please note: `yarn clean` will remove Zig and NodeAPI C headers, which mean you need to run:**
234
- ```shell
235
- ./scripts/postinstall.sh #Install Zig and NodeJS C Headers
628
+ ```console
629
+ git clone https://github.com/tigerbeetle/tigerbeetle
630
+ cd tigerbeetle
631
+ git submodule update --init --recursive
632
+ ./scripts/install_zig.sh
633
+ cd src/clients/node
634
+ npm install --include dev
635
+ npm pack
236
636
  ```
237
637
 
238
- ***Yarn - Run Test***
239
- Ensure TigerBeetle (`init` & `start`) is running on the port configured in `test.ts`, then run:
240
- ```shell
241
- ./tigerbeetle init --cluster=1 --replica=0 --directory=.
242
- ./tigerbeetle start --cluster=1 --replica=0 --directory=. --addresses=3001
243
- yarn test
244
- ```
638
+ ### On Windows
639
+
640
+ In PowerShell run:
245
641
 
246
- ***Yarn - Run Benchmark***
247
- Run the benchmark (The `benchmark` will automatically start TigerBeetle on port `3001` _(single replica)_:
248
- ```shell
249
- yarn benchmark
642
+ ```console
643
+ git clone https://github.com/tigerbeetle/tigerbeetle
644
+ cd tigerbeetle
645
+ git submodule update --init --recursive
646
+ .\scripts\install_zig.bat
647
+ cd src/clients/node
648
+ npm install --include dev
649
+ npm pack
250
650
  ```