acebase-server 1.6.1 → 1.7.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -41,6 +41,23 @@ const server = new AceBaseServer(dbname, settings);
41
41
 
42
42
  You can either pass ```certPath``` and ```keyPath```, or ```pfxPath``` and ```passphrase``` - depending on the type of certificate files you have.
43
43
 
44
+ ### Enable transaction logging
45
+ (NEW, BETA)
46
+
47
+ AceBase now supports transaction logging to facilitate sophisticated synchronization options and custom data recovery. Using cursors that indicate certain points in time, this allows for fast and easy synchronization of data between an AceBase server and multiple clients, or other server instances. This functionality is currently in BETA stage and will be tested extensively in the coming weeks.
48
+
49
+ To enable transaction logging on your database, add the `transactions` setting:
50
+ ```js
51
+ const server = new AceBaseServer(dbname, {
52
+ host: 'localhost', port: 443,
53
+ transactions: {
54
+ log: true, // Enable
55
+ maxAge: 30, // Keep logs of last 30 days
56
+ noWait: false // Data changes wait for log to be written
57
+ }
58
+ };
59
+ ```
60
+
44
61
  ### Enable authentication
45
62
 
46
63
  Authentication settings determine who is allowed access to your server. By enabling authentication, the server will allow users to sign in (and signup) and authorization rules to be defined for data being read/written to the database.
@@ -58,22 +75,22 @@ const settings = {
58
75
  }
59
76
  ```
60
77
 
61
- The following ```authentication``` settings are available:
62
- - ```enabled```: whether to enable user authentication (default: ```true```)
63
- - ```allowUserSignup```: whether users can sign up for an account themselves, or if the admin has to (default: ```false```)
64
- - ```defaultAccessRule```: default authorization rule for read/write operations. Either ```deny```, ```allow``` or ```auth```. The first time the server is launched with authentication enabled, it will create a ```rules.json``` file that contains this access rule for any path. ```deny``` will deny all users (except admin) read/write access, ```allow``` will grant access to anyone, ```auth``` will grant access only to authenticated users (default: ```auth```)
65
- - ```defaultAdminPassword```: supply a default password for the admin account that will be used when it is created. If you do not supply this, a generated password will be used instead and displayed only once in the console output.
78
+ The following `authentication` settings are available:
79
+ - `enabled`: whether to enable user authentication (default: `true`)
80
+ - `allowUserSignup`: whether users can sign up for an account themselves, or if the admin has to (default: `false`)
81
+ - `defaultAccessRule`: default authorization rule for read/write operations. Either `deny`, `allow` or `auth`. The first time the server is launched with authentication enabled, it will create a *rules.json* file that contains this access rule for any path. `deny` will deny all users (except admin) read/write access, `allow` will grant access to anyone, `auth` will grant access only to authenticated users (default: `auth`)
82
+ - `defaultAdminPassword`: supply a default password for the admin account that will be used when it is created. If you do not supply this, a generated password will be used instead and displayed only once in the console output.
66
83
 
67
84
  ### Setup authorization rules
68
85
 
69
- If you enabled authentication, you can also define access rules for your data. Using rules, you can allow or deny specific (or anonymous) users read and/or write access to your data. These rules are identical to those used by [Firebase](https://firebase.google.com/docs/database/security/) (Note: ".read" and ".write" only, ".validate" might be implemented in the future, ".schema" implemented as alternative) and are saved in a file called _rules.json_ in your database directory. The default rules written to the file are determined by the ```defaultAccessRule``` authentication setting at the first server launch with ```authentication``` enabled.
86
+ If you enabled authentication, you can also define access rules for your data. Using rules, you can allow or deny specific (or anonymous) users read and/or write access to your data. These rules are identical to those used by [Firebase](https://firebase.google.com/docs/database/security/) (Note: ".read" and ".write" only, ".validate" might be implemented in the future, ".schema" implemented as alternative) and are saved in a file called _rules.json_ in your database directory. The default rules written to the file are determined by the `defaultAccessRule` authentication setting at the first server launch with `authentication` enabled.
70
87
 
71
- The default _rules.json_ file content is based on the value of the ```defaultAccessRule``` setting, possible values are:
72
- * ```"auth"```: Only allow authenticated users read/write access to the database
73
- * ```"allow"```: Allow anyone (including anonymous users) read/write access to the database
74
- * ```"deny"```: Deny anyone (except admin user) read/write access to the database
88
+ The default _rules.json_ file content is based on the value of the `defaultAccessRule` setting, possible values are:
89
+ * `"auth"`: Only allow authenticated users read/write access to the database
90
+ * `"allow"`: Allow anyone (including anonymous users) read/write access to the database
91
+ * `"deny"`: Deny anyone (except admin user) read/write access to the database
75
92
 
76
- When ```defaultAccessRule: "auth"``` is used, it will generate the following _rules.json_ file:
93
+ When `defaultAccessRule: "auth"` is used, it will generate the following _rules.json_ file:
77
94
  ```json
78
95
  {
79
96
  "rules": {
@@ -83,7 +100,7 @@ When ```defaultAccessRule: "auth"``` is used, it will generate the following _ru
83
100
  }
84
101
  ```
85
102
 
86
- When ```"allow"``` or ```"deny"``` is used, the ```".read"``` and ```".write"``` properties will be set to ```true``` or ```false``` respectively.
103
+ When `"allow"` or `"deny"` is used, the `".read"` and `".write"` properties will be set to `true` or `false` respectively.
87
104
 
88
105
  If you want to further restrict what data users can access and/or write to (RECOMMENDED!), you could edit the file as such, granting users read/write access to their own user node:
89
106
  ```json
@@ -117,8 +134,8 @@ NOTE: Just like Firebase, access is denied by default when no rule is found for
117
134
  ```
118
135
  Above rules enforces:
119
136
  * No read or write access to the root node or any child for anyone. (No rule has been set for those nodes, access will be denied)
120
- * Read access to all reviews for specific shops ('shop_reviews/shop1', 'shop_reviews/shop2') for anyone, including unauthenticated clients (```".read"``` rule is set to ```true```)
121
- * Write access to an authenticated user's own review for any shop. (```".write"``` rule is set to ```"auth.uid === $uid"```)
137
+ * Read access to all reviews for specific shops ('shop_reviews/shop1', 'shop_reviews/shop2') for anyone, including unauthenticated clients (`".read"` rule is set to `true`)
138
+ * Write access to an authenticated user's own review for any shop. (`".write"` rule is set to `"auth.uid === $uid"`)
122
139
 
123
140
  ### Schema validation
124
141
 
@@ -128,7 +145,7 @@ There are 2 ways to can add schemas:
128
145
  - In your `rules.json` file, see below.
129
146
  - Programmatically through `db.schema.set`. See the [AceBase documentation](https://github.com/appy-one/acebase#adding-schemas-to-enforce-data-rules) for more info.
130
147
 
131
- To ensure all users have a ```name``` (string), ```email``` (string) and ```language``` (either Dutch, English, German, French or Spanish), optionally a ```birthdate``` (Date) and ```address``` (custom object definition), add the following to your _rules.json_ file:
148
+ To ensure all users have a `name` (string), `email` (string) and `language` (either Dutch, English, German, French or Spanish), optionally a `birthdate` (Date) and `address` (custom object definition), add the following to your _rules.json_ file:
132
149
  ```json
133
150
  {
134
151
  "rules": {
@@ -197,11 +214,11 @@ You can also decide to split the schema up into multiple levels:
197
214
 
198
215
  And, if you prefer, schema definitions can be defined as strings instead:
199
216
  ```json
200
- (...)
217
+ {
201
218
  "address": {
202
219
  ".schema": "{ street: string, city: string, country: string, geo?: { lat: number, lon: number } }"
203
220
  }
204
- (...)
221
+ }
205
222
  ```
206
223
 
207
224
  ## Sending user e-mails [NEW]
@@ -225,39 +242,36 @@ function sendRequestedEmail(request) {
225
242
  }
226
243
  ```
227
244
 
228
- Any links you would add to the e-mails you send out should point to your own website/app, where you would handle account verification / password resets through an ```AceBaseClient``` connected to your server:
245
+ Any links you would add to the e-mails you send out should point to your own website/app, where you would handle account verification / password resets through an `AceBaseClient` connected to your server:
229
246
 
230
- ```javascript
247
+ ```js
231
248
  const resetCode = 'weydgaed7gjsdhfjadbsfadsfasq3w7dtuqwebd'; // eg from your_reset_url?code=weydga...
232
249
  const newPassword = 'MyNewPassword'; // from user input on your page
233
- client.auth.resetPassword(resetCode, newPassword)
234
- .then(() => {
250
+ try {
251
+ await client.auth.resetPassword(resetCode, newPassword);
235
252
  // User can now sign in with their new password
236
- })
237
- .catch(err => {
253
+ }
254
+ catch(err) {
238
255
  // Something went wrong
239
- })
256
+ }
240
257
  ```
241
258
 
242
259
  OR, if you have direct access to your AceBaseServer instance from your website:
243
260
 
244
261
  ```javascript
245
- server.resetPassword(req.ip, resetCode, newPassword)
246
- .then(() => { ... });
262
+ await server.resetPassword(req.ip, resetCode, newPassword);
247
263
  ```
248
264
 
249
265
  To verify a user's email address:
250
266
  ```javascript
251
267
  const verificationCode = 'weydgaed7gjsdhfjadbsfadsfasq3w7dtuqwebd'; // eg from your_verify_url?code=weydga...
252
- client.auth.verifyEmailAddress(verificationCode)
253
- .then(() => { ... })
268
+ await client.auth.verifyEmailAddress(verificationCode);
254
269
  ```
255
270
 
256
271
  OR, directly on your server:
257
272
 
258
273
  ```javascript
259
- server.verifyEmailAddress(req.ip, verificationCode)
260
- .then(() => { ... });
274
+ await server.verifyEmailAddress(req.ip, verificationCode);
261
275
  ```
262
276
 
263
277
  ### E-mail request examples
@@ -317,11 +331,11 @@ Request type *user_reset_password_success*:
317
331
 
318
332
  You can enable users to sign into your app through a third party login provider, such as Facebook, Google, Twitter etc. To enable this, follow these steps for each provider:
319
333
 
320
- * First, get ```client_id``` and ```client_secret``` API keys from the auth providers' developer environment. This will allow your app to use with the provider's auth API.
334
+ * First, get `client_id` and `client_secret` API keys from the auth providers' developer environment. This will allow your app to use with the provider's auth API.
321
335
 
322
- * Most OAuth providers restrict authentication callback uri's to a predefined set of uris, so make sure you add AceBase's callback URL in the provider's API settings: ```"https://your.acebase.server/oauth2/dbname/signin"``` (replace hostname and dbname to your server Url)
336
+ * Most OAuth providers restrict authentication callback uri's to a predefined set of uris, so make sure you add AceBase's callback URL in the provider's API settings: `"https://your.acebase.server/oauth2/dbname/signin"` (replace hostname and dbname to your server Url)
323
337
 
324
- * Then, add your API keys to your ```AceBaseServer``` config with ```configAuthProvider```:
338
+ * Then, add your API keys to your `AceBaseServer` config with `configAuthProvider`:
325
339
 
326
340
  ```javascript
327
341
  server.configAuthProvider('facebook', { client_id: '[your fb app_id]', client_secret: '[your fb app_secret]', scopes: [/* Any additional scopes, such as 'user_birthday' */] });
@@ -347,7 +361,7 @@ client.auth.finishAuthProviderSignIn(callbackResult)
347
361
  })
348
362
  ```
349
363
 
350
- The ```result``` object will also contain the provider's ```access_token``` and ```refresh_token``` in case you want to make custom calls to the provider's API. If you want to keep the provider's access_token active, you will have to call ```client.auth.refreshAuthProviderToken``` before it expires:
364
+ The `result` object will also contain the provider's `access_token` and `refresh_token` in case you want to make custom calls to the provider's API. If you want to keep the provider's access_token active, you will have to call `client.auth.refreshAuthProviderToken` before it expires:
351
365
 
352
366
  ```javascript
353
367
  const keepAlive = (provider) => {
@@ -373,13 +387,13 @@ Currently implemented auth providers are:
373
387
  * Facebook*
374
388
  * Spotify
375
389
 
376
- *NOTE: Facebook access tokens are short-lived by default, but will be exchanged for a long-lived (60 day) access token upon refresh. If you need to keep the Facebook access token active, execute ```refreshAuthProviderToken``` immediately after ```finishAuthProviderSignIn```, and keep refreshing every time the user starts your app. Once a Facebook access token has expired, it cannot be refreshed and the user will have to sign in again.
390
+ *NOTE: Facebook access tokens are short-lived by default, but will be exchanged for a long-lived (60 day) access token upon refresh. If you need to keep the Facebook access token active, execute `refreshAuthProviderToken` immediately after `finishAuthProviderSignIn`, and keep refreshing every time the user starts your app. Once a Facebook access token has expired, it cannot be refreshed and the user will have to sign in again.
377
391
 
378
392
  ## Add cloud functions to handle data changes
379
393
 
380
394
  You can add "cloud functions" to perform custom tasks upon data changes. You can do this in 2 ways:
381
395
 
382
- * In the same process you are running your ```AceBaseServer``` (requires server v1.1+). Make sure you run cpu heavy code in a separate worker thread to keep the server thread available for core tasks:
396
+ * In the same process you are running your `AceBaseServer` (requires server v1.1+). Make sure you run cpu heavy code in a separate worker thread to keep the server thread available for core tasks:
383
397
  ```js
384
398
  const server = new AceBaseServer(dbname, settings);
385
399
  await server.ready();
@@ -411,7 +425,7 @@ db.ref('uploads/images').on('child_added', async snap => {
411
425
 
412
426
  ## Extending the server API
413
427
 
414
- You can add your own custom API functions to the server with the ```server.extend(method, path, handler)``` method:
428
+ You can add your own custom API functions to the server with the `server.extend(method, path, handler)` method:
415
429
 
416
430
  ```javascript
417
431
  const server = new AceBaseServer(...);
@@ -459,6 +473,10 @@ client.callExtension('post', 'quotes/add', {
459
473
  });
460
474
  ```
461
475
 
476
+ ## Running in a cluster
477
+
478
+ It is possible to run your *AceBaseServer* in a cluster so you can utilize more than just the 1 CPU a Node.js application typically uses. To make sure all instances can read and write to the same database files safely they will have to be able to communicate with each other. This is done using a technique called interprocess communication (IPC). Node.js' built-in cluster functionality provides IPC channels to communicate between master and worker processes, but if you want to run a `pm2` cluster or want multiple instances running in the cloud, you'll have to use an external IPC server. See [AceBase IPC Server](https://github.com/appy-one/acebase-ipc-server) for more information about the different clustering solutions and code examples.
479
+
462
480
  ## Connecting to a server
463
481
 
464
482
  See *acebase-client* on [npm](https://www.npmjs.com/package/acebase-client) or [github](https://github.com/appy-one/acebase-client)
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "acebase-server",
3
- "version": "1.6.1",
3
+ "version": "1.7.0",
4
4
  "description": "AceBase realtime database server (webserver endpoint to allow remote connections)",
5
5
  "main": "./src/index.js",
6
6
  "private": false,
@@ -44,7 +44,7 @@
44
44
  "author": "Ewout Stortenbeker <me@appy.one> (http://appy.one)",
45
45
  "license": "MIT",
46
46
  "dependencies": {
47
- "acebase": "^1.11.1",
47
+ "acebase": "^1.12.2",
48
48
  "acebase-core": "^1.10.1",
49
49
  "express": "^4.17.1",
50
50
  "socket.io": "^2.4.1"
@@ -5,22 +5,6 @@ const { ID, Transport, DataSnapshot, PathInfo, Utils, DebugLogger, ColorStyle }
5
5
  const fs = require('fs');
6
6
  const crypto = require('crypto');
7
7
 
8
- class AceBaseClusterSettings {
9
- constructor(settings) {
10
- this.enabled = typeof settings === "object";
11
- this.isMaster = this.enabled && settings.isMaster;
12
- this.master = this.enabled ? settings.master : process;
13
- this.workers = this.enabled ? settings.workers : [process];
14
- }
15
- }
16
-
17
- // class AceBaseClusterSettings {
18
- // constructor(settings) {
19
- // this.enabled = typeof settings === "object";
20
- // this.port = this.enabled ? settings.port : 0;
21
- // }
22
- // }
23
-
24
8
  class AceBaseServerHttpsSettings {
25
9
  /**
26
10
  *
@@ -206,6 +190,15 @@ class AceBaseServerTransactionSettings {
206
190
  }
207
191
  }
208
192
 
193
+ /**
194
+ * @typedef IPCClientSettings
195
+ * @property {string} [host='localhost'] IPC Server host to connect to. Default is `"localhost"`
196
+ * @property {number} port IPC Server port number
197
+ * @property {boolean} [ssl=false] Whether to use a secure connection to the server. Strongly recommended if `host` is not `"localhost"`. Default is `false`
198
+ * @property {string} [token] Token used in the IPC Server configuration (optional). The server will refuse connections using the wrong token.
199
+ * @property {'master'|'worker'} role Determines the role of this IPC client. Only 1 process can be assigned the 'master' role, all other processes must use the role 'worker'
200
+ */
201
+
209
202
  class AceBaseServerSettings {
210
203
 
211
204
  /**
@@ -221,6 +214,7 @@ class AceBaseServerSettings {
221
214
  * @param {AceBaseServerAuthenticationSettings} [settings.auth]
222
215
  * @param {AceBaseServerEmailSettings} [settings.email]
223
216
  * @param {AceBaseServerTransactionSettings} [settings.transactions]
217
+ * @param {IPCClientSettings} [settings.ipc]
224
218
  *
225
219
  */
226
220
  constructor(settings) {
@@ -229,105 +223,17 @@ class AceBaseServerSettings {
229
223
  this.host = settings.host || "localhost";
230
224
  this.port = settings.port || 3000;
231
225
  this.path = settings.path || '.';
232
- this.cluster = new AceBaseClusterSettings(settings.cluster);
233
226
  this.https = new AceBaseServerHttpsSettings(settings.https);
234
227
  this.authentication = new AceBaseServerAuthenticationSettings(settings.authentication);
235
228
  this.maxPayloadSize = settings.maxPayloadSize || '10mb';
236
229
  this.allowOrigin = settings.allowOrigin || '*';
237
230
  this.email = typeof settings.email === 'object' ? new AceBaseServerEmailSettings(settings.email) : null;
238
231
  this.transactions = new AceBaseServerTransactionSettings(settings.transactions);
232
+ this.ipc = settings.ipc;
239
233
  }
240
234
  }
241
235
 
242
236
  class AccessDeniedError extends Error { }
243
-
244
- class ClientSubscription {
245
- constructor(obj) {
246
- this.callback = obj.callback;
247
- this.disconnectedCallback = null;
248
- this.missedEvents = [];
249
- }
250
- }
251
- class MissedClientEvent {
252
- constructor(obj) {
253
- this.time = Date.now();
254
- this.event = obj.event;
255
- this.subscriptionPath = obj.subscriptionPath;
256
- this.path = obj.path;
257
- }
258
- }
259
- class Client {
260
- /**
261
- * Not used - yet
262
- * @param {object} obj
263
- * @param {AceBaseServer} obj.server
264
- * @param {SocketIO.Socket} obj.socket
265
- * @param {DbUserAccountDetails} obj.user
266
- */
267
- constructor(obj) {
268
- this.server = obj.server;
269
- this.socket = obj.socket;
270
- this.id = this.socket.id;
271
- this.user = obj.user;
272
-
273
- /** @type {{ [path: string]: ClientSubscription }} */
274
- this.subscriptions = {};
275
- /** @type {MissedClientEvent[]} */
276
- this.missedEvents = [];
277
- }
278
-
279
- subscribe(requestId, event, subscriptionPath) {
280
- // Check if user has access
281
- if (!this.server.userHasAccess(this.user, subscriptionPath, false)) {
282
- logRef.push({ action: `subscribe`, success: false, code: `access_denied`, uid: this.user ? this.user.uid : '-', path: subscriptionPath });
283
- this.socket.emit('result', {
284
- success: false,
285
- reason: `access_denied`,
286
- req_id: requestId
287
- });
288
- return;
289
- }
290
-
291
- const callback = (err, path, currentValue, previousValue) => {
292
- if (err) {
293
- return;
294
- }
295
- if (!this.socket) {
296
- // Client is currently disconnected. Keep track of missed events in case
297
- // the client comes back again.
298
- // TODO: Move from memory to eventsDb after a while
299
- this.missedEvents.push(new MissedClientEvent({ event, subscriptionPath, path, currentValue, previousValue }));
300
- return;
301
- }
302
- if (!this.server.userHasAccess(this.user, path, false)) {
303
- if (subscriptionPath.indexOf('*') < 0 && subscriptionPath.indexOf('$') < 0) {
304
- // Could potentially be very many callbacks, so
305
- // DISABLED: logRef.push({ action: `access_revoked`, uid: client.user ? client.user.uid : '-', path: subscriptionPath });
306
- // Only log when user subscribes again
307
- this.socket.emit('result', {
308
- success: false,
309
- reason: `access_denied`,
310
- req_id: requestId
311
- });
312
- }
313
- return;
314
- }
315
- let val = Transport.serialize({
316
- current: currentValue,
317
- previous: previousValue
318
- });
319
- this.server.verbose(`Sending data event "${event}" for path "/${path}" to client ${this.id}`);
320
- this.socket.emit("data-event", {
321
- subscr_path: subscriptionPath,
322
- path,
323
- event,
324
- val
325
- });
326
- };
327
-
328
- }
329
- }
330
-
331
237
  /**
332
238
  * @typedef {object} DbUserAccountDetails
333
239
  * @property {string} [uid] uid, not stored in database object (uid is the node's key)
@@ -528,8 +434,25 @@ class AceBaseServer extends EventEmitter {
528
434
  };
529
435
  app.use(corsHandler);
530
436
 
437
+ // TODO: determine max socket payload using options.maxPayloadSize which is now only used for json POST data
438
+ // const maxPayloadBytes = ((payloadStr) => {
439
+ // const match = payloadStr.match(/^([0-9]+)(?:mb|kb|b)$/i);
440
+ // if (!match) { return 10e7; } // Socket.IO 2.x default (100MB), 3.x default is 1MB (1e6)
441
+ // const nr = +match[0], unit = match[1].toLowerCase();
442
+ // switch (unit) {
443
+ // case 'mb': return nr * 1e6;
444
+ // case 'kb': return nr * 1e3;
445
+ // case 'b': return nr;
446
+ // }
447
+ // }, options.maxPayloadSize);
448
+
531
449
  const server = options.https.enabled ? require('https').createServer(options.https, app) : require('http').createServer(app);
532
450
  const io = require('socket.io')(server, {
451
+ // See https://socket.io/docs/v2/server-initialization/ and https://socket.io/docs/v3/server-initialization/
452
+ pingInterval: 5000, // socket.io 2.x default is 25000
453
+ pingTimeout: 5000, // socket.io 2.x default is 5000, 3.x default = 20000
454
+ maxHttpBufferSize: 10e7, // Socket is closed if sent message exceeds this. Socket.io 2.x default is 10e7 (100MB)
455
+
533
456
  // socket.io 2.x:
534
457
  handlePreflightRequest: corsHandler
535
458
  // socket.io 3+:
@@ -552,17 +475,18 @@ class AceBaseServer extends EventEmitter {
552
475
  }
553
476
  };
554
477
  this.url = this.config.url; // Is this used?
555
- this.debug = new DebugLogger(options.logLevel, `[${dbname}]`.colorize(ColorStyle.green)); //`« ${dbname} »`
478
+ this.debug = new DebugLogger(options.logLevel, `[${dbname}]`.colorize(ColorStyle.green));
556
479
 
557
480
  const dbOptions = {
558
481
  logLevel: options.logLevel,
559
482
  info: 'realtime database server',
560
- storage: {
561
- cluster: options.cluster,
483
+ // NEW: Allow storage setting like AceBaseLocalSettings - could allow using other db backend (typed, but undocumented)
484
+ storage: options.storage || {
562
485
  path: options.path,
563
486
  removeVoidProperties: true,
564
- transactions: options.transactions
565
- }
487
+ },
488
+ transactions: options.transactions,
489
+ ipc: options.ipc
566
490
  };
567
491
 
568
492
  const db = new AceBase(dbname, dbOptions);
@@ -617,7 +541,7 @@ class AceBaseServer extends EventEmitter {
617
541
 
618
542
  const setupRules = () => {
619
543
  // Check if there is a rules file, load it or generate default
620
- fs.unwatchFile(rulesFilePath); // If function was triggered because of file change, stop listening now. Listener will be setup again at end of function
544
+ fs.unwatchFile(rulesFilePath, setupRules); // If function was triggered because of file change, stop listening now. Listener will be setup again at end of function
621
545
 
622
546
  const defaultAccessRule = (def => {
623
547
  switch (def) {
@@ -713,10 +637,7 @@ class AceBaseServer extends EventEmitter {
713
637
  processRules('', accessRules.rules, []);
714
638
 
715
639
  // Watch file for changes
716
- fs.watchFile(rulesFilePath, (currStats, prevStats) => {
717
- // rules.json file changed, setup rules again
718
- setupRules();
719
- });
640
+ fs.watchFile(rulesFilePath, setupRules);
720
641
  }
721
642
  setupRules();
722
643
 
@@ -2022,9 +1943,14 @@ class AceBaseServer extends EventEmitter {
2022
1943
  }
2023
1944
  });
2024
1945
 
1946
+ app.get(`/ping/${dbname}`, (req, res) => {
1947
+ // For simple connectivity check
1948
+ res.end('pong');
1949
+ });
1950
+
2025
1951
  app.get(`/info/${dbname}`, (req, res) => {
2026
1952
  const info = {
2027
- version: '1.5.1', // TODO: Load from package.json
1953
+ version: '1.7.0', // TODO: Load from package.json
2028
1954
  time: Date.now(),
2029
1955
  process: process.pid
2030
1956
  };
@@ -2697,7 +2623,7 @@ class AceBaseServer extends EventEmitter {
2697
2623
 
2698
2624
  this.shutdown = async () => {
2699
2625
  this.debug.warn('shutting down server');
2700
- fs.unwatchFile(rulesFilePath);
2626
+ fs.unwatchFile(rulesFilePath, setupRules);
2701
2627
  await new Promise((resolve) => {
2702
2628
  server.close(resolve);
2703
2629
  clients.list.slice().forEach(client => {
@@ -2707,9 +2633,43 @@ class AceBaseServer extends EventEmitter {
2707
2633
  this.debug.warn('closing database');
2708
2634
  await db.close(); // TODO: Check what happens if shutdown was triggered by SIGINT. Maybe already closed?
2709
2635
  this.debug.warn('shutdown complete');
2636
+
2637
+ // Emit events to let the outside world know we shut down.
2638
+ // This is especially important if this instance was running in a Node.js cluster: the process will
2639
+ // not exit automatically after this shutdown because Node.js' IPC channel between worker and master is still open.
2640
+ // By sending these events, the cluster manager can determine if it should (and when to) execute process.exit()
2641
+ this.emit('shutdown'); // Emit on AceBaseServer instance
2642
+ process.emit('acebase-server-shutdown'); // Emit on process
2643
+ try {
2644
+ process.send && process.send('acebase-server-shutdown'); // Send to master process when running in a Node.js cluster
2645
+ }
2646
+ catch(err) {
2647
+ // IPC Channel has apparently been closed already
2648
+ }
2710
2649
  };
2711
2650
  process.on('SIGINT', () => this.shutdown());
2712
2651
 
2652
+ let paused = false;
2653
+ this.pause = async () => {
2654
+ if (paused) { throw new Error('Server is already paused'); }
2655
+ server.close();
2656
+ this.debug.warn(`Paused "${dbname}" database server at ${this.config.url}`);
2657
+ this.emit('pause');
2658
+ paused = true;
2659
+ }
2660
+
2661
+ this.resume = async () => {
2662
+ if (!paused) { throw new Error('Server is not paused'); }
2663
+ return new Promise(resolve => {
2664
+ server.listen(this.config.port, this.config.hostname, () => {
2665
+ this.debug.warn(`Resumed "${dbname}" database server at ${this.config.url}`);
2666
+ this.emit('resume');
2667
+ paused = false;
2668
+ resolve();
2669
+ });
2670
+ })
2671
+ }
2672
+
2713
2673
  /** @type {Map<string, Client>} */
2714
2674
  // const clientsNew = new Map();
2715
2675
 
@@ -2848,6 +2808,7 @@ class AceBaseServer extends EventEmitter {
2848
2808
  previous: previousValue
2849
2809
  });
2850
2810
  this.debug.verbose(`Sending data event "${data.event}" for path "/${path}" to client ${socket.id}`.colorize([ColorStyle.bgWhite, ColorStyle.black]));
2811
+ // TODO: let large data events notify the client, then let them download the data manually so it doesn't have to be transmitted through the websocket
2851
2812
  socket.emit('data-event', {
2852
2813
  subscr_path: subscriptionPath,
2853
2814
  path,
@@ -3050,7 +3011,6 @@ if (__filename.replace(/\\/g,"/").endsWith("/acebase-server.js") && process.argv
3050
3011
  module.exports = {
3051
3012
  AceBaseServer,
3052
3013
  AceBaseServerSettings,
3053
- AceBaseClusterSettings,
3054
3014
  AceBaseEmailRequest,
3055
3015
  AceBaseUserSignupEmailRequest,
3056
3016
  AceBaseUserResetPasswordEmailRequest,
package/src/cluster.js CHANGED
@@ -3,6 +3,15 @@ const { AceBaseServer } = require('./acebase-server');
3
3
  const cluster = require('cluster');
4
4
  const numCPUs = 2; //require('os').cpus().length;
5
5
 
6
+ /**
7
+ * Node.js cluster example. You can use this as a reference how to create an AceBaseServer cluster using Node.js'
8
+ * built-in cluster functionality.
9
+ *
10
+ * If you want to run a pm2 or cloud-based cluster instead, do NOT use this code.
11
+ * To do that, see https://github.com/appy-one/acebase-ipc-server
12
+ *
13
+ */
14
+
6
15
  let dbname = "default";
7
16
  let options = { /* default options */ }; // Load from cluster.config.js!
8
17
  options.authentication = { enabled: false };
@@ -10,52 +19,42 @@ options.https = { enabled: false };
10
19
 
11
20
  if (cluster.isMaster) {
12
21
  // Startup master
13
- console.log(`Master ${process.pid} is running`);
14
-
15
- options.cluster = {
16
- enabled: true,
17
- isMaster: true,
18
- workers: []
19
- };
22
+ console.log(`Starting database server cluster with ${numCPUs} workers`);
23
+ const master = new AceBase(dbname, options);
24
+ master.once('ready', () => {
25
+ console.log(`Master database server started on process ${process.pid}`);
26
+ });
20
27
 
21
28
  for (let i = 0; i < numCPUs; i++) {
22
- let worker = cluster.fork();
23
- worker.on("disconnect", (worker, code, signal) => {
29
+ const worker = cluster.fork();
30
+ worker.on('disconnect', () => {
24
31
  console.error(`worker ${worker.process.pid} disconnected`);
25
- // TODO: restart!
26
32
  });
27
- options.cluster.workers.push(worker);
33
+ worker.on('message', (msg) => {
34
+ if (msg === 'acebase-server-shutdown') {
35
+ // We could handle shutdown message here to kill the worker safely, but we'll do it in the worker threads instead: see below
36
+ // worker.kill();
37
+ }
38
+ });
28
39
  }
29
40
 
30
- process.on("unhandledRejection", (reason, p) => {
31
- console.log("Unhandled Rejection in master ", process.pid, " at: ", reason.stack);
32
- });
33
-
34
- console.log(`Starting database server with ${options.cluster.workers.length} workers`);
35
- const master = new AceBase(dbname, options);
36
- master.once("ready", () => {
37
- console.log(`Master database server started on process ${process.pid}`);
38
- });
39
-
40
- cluster.on("exit", (worker, code, signal) => {
41
+ cluster.on('exit', (worker, code, signal) => {
41
42
  console.error(`worker ${worker.process.pid} died`);
42
43
  });
43
44
  }
44
45
  else {
45
46
  console.log(`Worker ${process.pid} is running`);
46
47
 
47
- options.cluster = {
48
- enabled: true,
49
- isMaster: false,
50
- master: process
51
- };
52
- process.on("unhandledRejection", (reason, p) => {
53
- console.log("Unhandled Rejection in worker ", process.pid, " at: ", reason.stack);
54
- });
55
-
56
- const workerServer = new AceBaseServer(dbname, options);
57
- workerServer.ready(() => {
48
+ const server = new AceBaseServer(dbname, options);
49
+ server.ready(() => {
58
50
  console.log(`Worker database server started on process ${process.pid}`);
59
51
  });
60
-
52
+ // Handle shutdown event to exit the worker process safely
53
+ server.on('shutdown', () => {
54
+ process.exit();
55
+ });
56
+ // Other option:
57
+ // process.on('acebase-server-shutdown', () => {
58
+ // process.exit();
59
+ // });
61
60
  }
package/src/index.d.ts CHANGED
@@ -74,8 +74,17 @@ declare namespace acebaseserver {
74
74
 
75
75
  /**
76
76
  * Shuts down the server. Stops listening for incoming connections, breaks current connections and closes the database.
77
+ * Is automatically executed when a "SIGINT" process event is received.
78
+ *
79
+ * Once the shutdown procedure is completed, it emits a "shutdown" event on the server instance, "acebase-server-shutdown" event on the `process`, and sends an 'acebase-server-shutdown' IPC message if Node.js clustering is used.
80
+ * These events can be handled by cluster managing code to `kill` or `exit` the process safely.
77
81
  */
78
82
  shutdown(): Promise<void>
83
+
84
+ /** Temporarily stops the server from handling incoming connections, but keeps existing connections open */
85
+ pause(): Promise<void>
86
+ /** Resumes handling incoming connections */
87
+ resume(): Promise<void>
79
88
  }
80
89
 
81
90
  interface IOAuthProviderSettings {
@@ -109,6 +118,10 @@ declare namespace acebaseserver {
109
118
  email?: AceBaseServerEmailSettings
110
119
  /** Transaction logging settings. Warning: BETA stage, do NOT use in production yet */
111
120
  transactions?: AceBaseServerTransactionLogSettings
121
+ /** IPC settings for pm2 or cloud-based clusters. BETA stage, see https://github.com/appy-one/acebase-ipc-server */
122
+ ipc?: IPCClientSettings
123
+ /** Allows overriding of default storage settings used by the database. ALPHA stage */
124
+ storage?: any
112
125
  }
113
126
 
114
127
  interface AceBaseServerHttpsSettings {
@@ -154,7 +167,18 @@ declare namespace acebaseserver {
154
167
  maxAge?: number;
155
168
  noWait?: boolean;
156
169
  }
157
-
170
+ interface IPCClientSettings {
171
+ /** IPC Server hostname. Default is "localhost" */
172
+ host?: string
173
+ /** IPC Server port */
174
+ port: number
175
+ /** IPC Server token needed to access the server. Only needed if the server does not use a token */
176
+ token?: string
177
+ /** Whether to use a secure connection to the IPC server, default is `false` */
178
+ ssl?: boolean
179
+ /** Role of the IPC Client. There can only be 1 `master`, all other need to be a `worker`. */
180
+ role: 'master'|'worker'
181
+ }
158
182
  interface AceBaseEmailRequest {
159
183
  readonly type: string
160
184
  readonly ip: string
package/src/start.js CHANGED
@@ -15,11 +15,21 @@ function getVariable(name, defaultValue) {
15
15
  const path = getVariable('DBPATH', '.');
16
16
  const dbname = getVariable('DBNAME', 'default');
17
17
  const host = getVariable('HOST', '0.0.0.0');
18
- const port = parseInt(getVariable('PORT', 3000));
19
- const clusterPort = parseInt(getVariable('CLUSTER_PORT', 0));
18
+ const port = +getVariable('PORT', 3000);
19
+ const ipcPort = +getVariable('IPC_PORT', 0);
20
20
  const options = { host, port, path };
21
- if (clusterPort > 0) {
22
- options.cluster = { enabled: true, port: clusterPort };
21
+ if (ipcPort > 0) {
22
+ const role = getVariable('IPC_ROLE');
23
+ if (!['master','worker'].includes(role)) {
24
+ throw new Error('IPC_ROLE must be either "master" or "worker"');
25
+ }
26
+ options.ipc = {
27
+ host: getVariable('IPC_HOST'),
28
+ port: ipcPort,
29
+ ssl: +getVariable('IPC_SSL', 0) === 1,
30
+ token: getVariable('IPC_TOKEN'),
31
+ role
32
+ };
23
33
  }
24
34
  if (+getVariable('TXLOG', 0) === 1) {
25
35
  options.transactions = {
@@ -276,7 +276,7 @@ function updateBrowsePath(path = '') {
276
276
  return showError('node does not exist');
277
277
  }
278
278
  else if (info.children.list.length === 0) {
279
- return showError('node has no children');
279
+ showError('node has no children');
280
280
  }
281
281
 
282
282
  if (skip === 0 && !info.children.more) {