wingbot-mongodb 2.17.0 → 2.20.0

package/README.md

@@ -32,17 +32,17 @@ Contains storage for tokens, chat states, bot config and chat logs.
 ## Typedefs
 
 <dl>
-<dt><a href="#State">State</a> : <code>Object</code></dt>
+<dt><a href="#State">State</a> : <code>object</code></dt>
 <dd></dd>
-<dt><a href="#StateCondition">StateCondition</a> : <code>Object</code></dt>
+<dt><a href="#StateCondition">StateCondition</a> : <code>object</code></dt>
 <dd></dd>
-<dt><a href="#Token">Token</a> : <code>Object</code></dt>
+<dt><a href="#Token">Token</a> : <code>object</code></dt>
 <dd></dd>
 <dt><a href="#Target">Target</a> : <code>Object</code></dt>
 <dd></dd>
 <dt><a href="#Subscribtion">Subscribtion</a> : <code>Object</code></dt>
 <dd></dd>
-<dt><a href="#Campaign">Campaign</a> : <code>Object</code></dt>
+<dt><a href="#Campaign">Campaign</a> : <code>object</code></dt>
 <dd></dd>
 <dt><a href="#Task">Task</a> : <code>Object</code></dt>
 <dd></dd>
@@ -57,13 +57,11 @@ Storage for chat states
 
 * [StateStorage](#StateStorage)
     * [new StateStorage(mongoDb, collectionName, [log], isCosmo)](#new_StateStorage_new)
-    * [._collection](#StateStorage+_collection) : <code>Promise.&lt;mongodb.Collection&gt;</code>
-    * [.addCustomIndex(index, options)](#StateStorage+addCustomIndex)
-    * [._getCollection()](#StateStorage+_getCollection) ⇒ <code>Promise.&lt;mongodb.Collection&gt;</code>
+    * ~~[.addCustomIndex(index, options)](#StateStorage+addCustomIndex)~~
     * [.getState(senderId, pageId)](#StateStorage+getState) ⇒ <code>Promise.&lt;(State\|null)&gt;</code>
-    * [.getOrCreateAndLock(senderId, pageId, [defaultState], [timeout])](#StateStorage+getOrCreateAndLock) ⇒ <code>Promise.&lt;Object&gt;</code>
+    * [.getOrCreateAndLock(senderId, pageId, [defaultState], [timeout])](#StateStorage+getOrCreateAndLock) ⇒ <code>Promise.&lt;object&gt;</code>
     * [.getStates(condition, limit, lastKey)](#StateStorage+getStates) ⇒ <code>Promise.&lt;{Array.&lt;data:State&gt;, lastKey:string}&gt;</code>
-    * [.saveState(state)](#StateStorage+saveState) ⇒ <code>Promise.&lt;Object&gt;</code>
+    * [.saveState(state)](#StateStorage+saveState) ⇒ <code>Promise.&lt;object&gt;</code>
 
 <a name="new_StateStorage_new"></a>
 
@@ -72,31 +70,25 @@ Storage for chat states
 | Param | Type | Default | Description |
 | --- | --- | --- | --- |
 | mongoDb | <code>mongodb.Db</code> \| <code>Object</code> |  |  |
-| collectionName | <code>string</code> | <code>&quot;states&quot;</code> |  |
+| collectionName | <code>string</code> | <code>&quot;chatlogs&quot;</code> |  |
 | [log] | <code>Object</code> |  | console like logger |
 | isCosmo | <code>boolean</code> | <code>false</code> |  |
 
-<a name="StateStorage+_collection"></a>
-
-### stateStorage.\_collection : <code>Promise.&lt;mongodb.Collection&gt;</code>
-**Kind**: instance property of [<code>StateStorage</code>](#StateStorage)  
 <a name="StateStorage+addCustomIndex"></a>
 
-### stateStorage.addCustomIndex(index, options)
+### ~~stateStorage.addCustomIndex(index, options)~~
+***Deprecated***
+
 Add custom indexing rule
 
 **Kind**: instance method of [<code>StateStorage</code>](#StateStorage)  
 
 | Param | Type |
 | --- | --- |
-| index | <code>Object</code> | 
-| options | <code>Object</code> | 
+| index | <code>object</code> | 
+| options | <code>object</code> | 
 | options.name | <code>string</code> | 
 
-<a name="StateStorage+_getCollection"></a>
-
-### stateStorage.\_getCollection() ⇒ <code>Promise.&lt;mongodb.Collection&gt;</code>
-**Kind**: instance method of [<code>StateStorage</code>](#StateStorage)  
 <a name="StateStorage+getState"></a>
 
 ### stateStorage.getState(senderId, pageId) ⇒ <code>Promise.&lt;(State\|null)&gt;</code>
@@ -109,17 +101,17 @@ Add custom indexing rule
 
 <a name="StateStorage+getOrCreateAndLock"></a>
 
-### stateStorage.getOrCreateAndLock(senderId, pageId, [defaultState], [timeout]) ⇒ <code>Promise.&lt;Object&gt;</code>
+### stateStorage.getOrCreateAndLock(senderId, pageId, [defaultState], [timeout]) ⇒ <code>Promise.&lt;object&gt;</code>
 Load state from database and lock it to prevent another reads
 
 **Kind**: instance method of [<code>StateStorage</code>](#StateStorage)  
-**Returns**: <code>Promise.&lt;Object&gt;</code> - - conversation state  
+**Returns**: <code>Promise.&lt;object&gt;</code> - - conversation state  
 
 | Param | Type | Default | Description |
 | --- | --- | --- | --- |
 | senderId | <code>string</code> |  | sender identifier |
 | pageId | <code>string</code> |  | page identifier |
-| [defaultState] | <code>Object</code> |  | default state of the conversation |
+| [defaultState] | <code>object</code> |  | default state of the conversation |
 | [timeout] | <code>number</code> | <code>300</code> | given default state |
 
 <a name="StateStorage+getStates"></a>
@@ -135,14 +127,14 @@ Load state from database and lock it to prevent another reads
 
 <a name="StateStorage+saveState"></a>
 
-### stateStorage.saveState(state) ⇒ <code>Promise.&lt;Object&gt;</code>
+### stateStorage.saveState(state) ⇒ <code>Promise.&lt;object&gt;</code>
 Save the state to database
 
 **Kind**: instance method of [<code>StateStorage</code>](#StateStorage)  
 
 | Param | Type | Description |
 | --- | --- | --- |
-| state | <code>Object</code> | conversation state |
+| state | <code>object</code> | conversation state |
 
 <a name="BotTokenStorage"></a>
 
@@ -244,9 +236,9 @@ Log single event
 | Param | Type | Description |
 | --- | --- | --- |
 | senderId | <code>string</code> |  |
-| responses | <code>Array.&lt;Object&gt;</code> | list of sent responses |
-| request | <code>Object</code> | event request |
-| [metadata] | <code>Object</code> | request metadata |
+| responses | <code>Array.&lt;object&gt;</code> | list of sent responses |
+| request | <code>object</code> | event request |
+| [metadata] | <code>object</code> | request metadata |
 
 <a name="BotConfigStorage"></a>
 
@@ -262,8 +254,9 @@ Storage for wingbot.ai conversation config
     * [.api([onUpdate], [acl])](#BotConfigStorage+api) ⇒ <code>Object</code>
     * [.invalidateConfig()](#BotConfigStorage+invalidateConfig) ⇒ <code>Promise</code>
     * [.getConfigTimestamp()](#BotConfigStorage+getConfigTimestamp) ⇒ <code>Promise.&lt;number&gt;</code>
-    * [.updateConfig(newConfig)](#BotConfigStorage+updateConfig) ⇒ <code>Promise.&lt;T&gt;</code>
-    * [.getConfig()](#BotConfigStorage+getConfig) ⇒ <code>Promise.&lt;(Object\|null)&gt;</code>
+    * [.updateConfig(newConfig, [id])](#BotConfigStorage+updateConfig) ⇒ <code>Promise.&lt;T&gt;</code>
+    * [.setConfig(id, newConfig)](#BotConfigStorage+setConfig)
+    * [.getConfig([id])](#BotConfigStorage+getConfig) ⇒ <code>Promise.&lt;(object\|null)&gt;</code>
 
 <a name="new_BotConfigStorage_new"></a>
 
@@ -306,17 +299,33 @@ Invalidates current configuration
 **Kind**: instance method of [<code>BotConfigStorage</code>](#BotConfigStorage)  
 <a name="BotConfigStorage+updateConfig"></a>
 
-### botConfigStorage.updateConfig(newConfig) ⇒ <code>Promise.&lt;T&gt;</code>
+### botConfigStorage.updateConfig(newConfig, [id]) ⇒ <code>Promise.&lt;T&gt;</code>
 **Kind**: instance method of [<code>BotConfigStorage</code>](#BotConfigStorage)  
 
 | Param | Type |
 | --- | --- |
 | newConfig | <code>T</code> | 
+| [id] | <code>string</code> | 
+
+<a name="BotConfigStorage+setConfig"></a>
+
+### botConfigStorage.setConfig(id, newConfig)
+**Kind**: instance method of [<code>BotConfigStorage</code>](#BotConfigStorage)  
+
+| Param | Type |
+| --- | --- |
+| id | <code>string</code> | 
+| newConfig | <code>object</code> | 
 
 <a name="BotConfigStorage+getConfig"></a>
 
-### botConfigStorage.getConfig() ⇒ <code>Promise.&lt;(Object\|null)&gt;</code>
+### botConfigStorage.getConfig([id]) ⇒ <code>Promise.&lt;(object\|null)&gt;</code>
 **Kind**: instance method of [<code>BotConfigStorage</code>](#BotConfigStorage)  
+
+| Param | Type |
+| --- | --- |
+| [id] | <code>string</code> | 
+
 <a name="AttachmentCache"></a>
 
 ## AttachmentCache
@@ -428,7 +437,7 @@ Cache storage for Facebook attachments
 
 | Param | Type |
 | --- | --- |
-| tasks | <code>Object</code> | 
+| tasks | <code>object</code> | 
 
 <a name="NotificationsStorage+getUnsuccessfulSubscribersByCampaign"></a>
 
@@ -460,7 +469,7 @@ Return Task By Id
 | Param | Type |
 | --- | --- |
 | taskId | <code>string</code> | 
-| data | <code>Object</code> | 
+| data | <code>object</code> | 
 
 <a name="NotificationsStorage+getSentTask"></a>
 
@@ -506,8 +515,8 @@ Get last sent task from campaign
 
 | Param | Type | Default |
 | --- | --- | --- |
-| campaign | <code>Object</code> |  | 
-| [updateCampaign] | <code>Object</code> | <code></code> | 
+| campaign | <code>object</code> |  | 
+| [updateCampaign] | <code>object</code> | <code></code> | 
 
 <a name="NotificationsStorage+removeCampaign"></a>
 
@@ -526,7 +535,7 @@ Get last sent task from campaign
 | Param | Type |
 | --- | --- |
 | campaignId | <code>string</code> | 
-| increment | <code>Object</code> | 
+| increment | <code>object</code> | 
 
 <a name="NotificationsStorage+updateCampaign"></a>
 
@@ -536,7 +545,7 @@ Get last sent task from campaign
 | Param | Type |
 | --- | --- |
 | campaignId | <code>string</code> | 
-| data | <code>Object</code> | 
+| data | <code>object</code> | 
 
 <a name="NotificationsStorage+popCampaign"></a>
 
@@ -572,9 +581,9 @@ Get last sent task from campaign
 
 | Param | Type | Default |
 | --- | --- | --- |
-| condition | <code>Object</code> |  | 
+| condition | <code>object</code> |  | 
 | [limit] | <code>number</code> | <code></code> | 
-| [lastKey] | <code>Object</code> | <code></code> | 
+| [lastKey] | <code>object</code> | <code></code> | 
 
 <a name="NotificationsStorage+subscribe"></a>
 
@@ -692,9 +701,8 @@ Add custom indexing rule
 
 | Param | Type |
 | --- | --- |
-| index | <code>Object</code> | 
-| options | <code>Object</code> | 
-| options.name | <code>string</code> | 
+| index | <code>object</code> | 
+| options | <code>mongodb.IndexOptions</code> | 
 
 <a name="BaseStorage+_getCollection"></a>
 
@@ -704,7 +712,7 @@ Returns the collection to operate with
 **Kind**: instance method of [<code>BaseStorage</code>](#BaseStorage)  
 <a name="State"></a>
 
-## State : <code>Object</code>
+## State : <code>object</code>
 **Kind**: global typedef  
 **Properties**
 
@@ -712,11 +720,11 @@ Returns the collection to operate with
 | --- | --- |
 | senderId | <code>string</code> | 
 | pageId | <code>string</code> | 
-| state | <code>Object</code> | 
+| state | <code>object</code> | 
 
 <a name="StateCondition"></a>
 
-## StateCondition : <code>Object</code>
+## StateCondition : <code>object</code>
 **Kind**: global typedef  
 **Properties**
 
@@ -726,7 +734,7 @@ Returns the collection to operate with
 
 <a name="Token"></a>
 
-## Token : <code>Object</code>
+## Token : <code>object</code>
 **Kind**: global typedef  
 **Properties**
 
@@ -761,7 +769,7 @@ Returns the collection to operate with
 
 <a name="Campaign"></a>
 
-## Campaign : <code>Object</code>
+## Campaign : <code>object</code>
 **Kind**: global typedef  
 **Properties**
 
@@ -781,7 +789,7 @@ Returns the collection to operate with
 | leaved | <code>number</code> |  |
 | queued | <code>number</code> | Interaction |
 | action | <code>string</code> |  |
-| [data] | <code>Object</code> | Setup |
+| [data] | <code>object</code> | Setup |
 | sliding | <code>boolean</code> |  |
 | slide | <code>number</code> |  |
 | slideRound | <code>number</code> |  |

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "wingbot-mongodb",
-  "version": "2.17.0",
+  "version": "2.20.0",
   "description": "MongoDB storage for wingbot.ai",
   "main": "src/main.js",
   "scripts": {
@@ -46,10 +46,15 @@
     "mocha": "^9.1.3",
     "mongodb": "^3.7.3",
     "nyc": "^15.1.0",
-    "wingbot": "^3.22.4"
+    "wingbot": "^3.25.0"
   },
   "peerDependencies": {
-    "mongodb": "^3.0.0",
+    "mongodb": "^3.0.0"
+  },
+  "optionalDependencies": {
     "wingbot": "^3.0.0"
+  },
+  "dependencies": {
+    "jsonwebtoken": "^8.5.1"
   }
 }

package/src/AuditLogStorage.js

@@ -0,0 +1,364 @@
+/**
+ * @author wingbot.ai
+ */
+'use strict';
+
+const jsonwebtoken = require('jsonwebtoken');
+const BaseStorage = require('./BaseStorage');
+
+/** @typedef {import('mongodb/lib/db')} Db */
+
+const LEVEL_CRITICAL = 'Critical';
+const LEVEL_IMPORTANT = 'Important';
+const LEVEL_DEBUG = 'Debug';
+
+const TYPE_ERROR = 'Error';
+const TYPE_WARN = 'Warn';
+const TYPE_INFO = 'Info';
+
+/**
+ * @typedef {object} TrackingEvent
+ * @prop {string} [type='audit']
+ * @prop {string} category
+ * @prop {string} action
+ * @prop {string} [label]
+ * @prop {object} [payload]
+ */
+
+/**
+ * @typedef {object} User
+ * @prop {string} [id]
+ * @prop {string} [senderId]
+ * @prop {string} [pageId]
+ * @prop {string} [jwt] - jwt to check the authorship
+ */
+
+/**
+ * @typedef {object} Meta
+ * @prop {string} [ip]
+ * @prop {string} [ua]
+ * @prop {string} [ro] - referrer || origin
+ */
+
+/**
+ * @typedef {object} LogEntry
+ * @prop {string} date - ISO date
+ * @prop {number} delta - time skew in ms if there was a write conflict
+ * @prop {string} [eventType='audit']
+ * @prop {string} category
+ * @prop {string} action
+ * @prop {string} [label]
+ * @prop {object} [payload]
+ * @prop {string} level - (Critical|Important|Debug)
+ * @prop {boolean} ok - signature matches
+ * @prop {number} seq - sequence number
+ * @prop {string} type - (Error|Warn|Info)
+ * @prop {User} user
+ * @prop {string} wid - workspace id
+ * @prop {Meta} meta
+ */
+
+/**
+ * JWT Verifier
+ *
+ * @callback JwtVerifier
+ * @param {string} token
+ * @param {string} userId
+ * @param {User} [user]
+ * @returns {Promise<boolean>}
+ */
+
+/**
+ * @typedef {object} AuditLogEntry
+ * @prop {string} date - ISO date
+ * @prop {string} [eventType='audit']
+ * @prop {string} category
+ * @prop {string} action
+ * @prop {string} [label]
+ * @prop {object} [payload]
+ * @prop {string} level - (Critical|Important|Debug)
+ * @prop {string} type - (Error|Warn|Info)
+ * @prop {User} user
+ * @prop {string} wid - workspace id
+ * @prop {Meta} meta
+ */
+
+/**
+ * Audit Log Callback
+ *
+ * @callback AuditLogCallback
+ * @param {AuditLogEntry} entry
+ * @returns {Promise}
+ */
+
+/**
+ * Storage for audit logs with signatures chain
+ */
+class AuditLogStorage extends BaseStorage {
+
+    /**
+     *
+     * @param {Db|{():Promise<Db>}} mongoDb
+     * @param {string} collectionName
+     * @param {{error:Function,log:Function}} [log] - console like logger
+     * @param {boolean} [isCosmo]
+     * @param {string|Promise<string>} [secret]
+     * @param {string|Promise<string>} [jwtVerifier]
+     */
+    constructor (mongoDb, collectionName = 'auditlog', log = console, isCosmo = false, secret = null, jwtVerifier = null) {
+        super(mongoDb, collectionName, log, isCosmo);
+
+        this.addIndex({
+            wid: 1,
+            seq: -1
+        }, {
+            unique: true,
+            name: 'wid_1_seq_-1'
+        });
+
+        if (isCosmo) {
+            this.addIndex({
+                wid: 1
+            }, {
+                name: 'wid_1'
+            });
+
+            this.addIndex({
+                seq: -1
+            }, {
+                name: 'seq_-1'
+            });
+        } else {
+            this.addIndex({
+                wid: 1, date: -1
+            }, {
+                name: 'wid_1_date_-1'
+            });
+        }
+
+        this.defaultWid = '0';
+
+        this.muteErrors = true;
+        this.maxRetries = 4;
+        this._secret = secret;
+
+        this.LEVEL_CRITICAL = LEVEL_CRITICAL;
+        this.LEVEL_IMPORTANT = LEVEL_IMPORTANT;
+        this.LEVEL_DEBUG = LEVEL_DEBUG;
+
+        this.TYPE_ERROR = TYPE_ERROR;
+        this.TYPE_WARN = TYPE_WARN;
+        this.TYPE_INFO = TYPE_INFO;
+
+        /**
+         * @type {JwtVerifier}
+         */
+        // @ts-ignore
+        this._jwtVerify = typeof jwtVerifier === 'function' || jwtVerifier === null
+            ? jwtVerifier
+            : async (token, userId) => {
+                const jwtSec = await Promise.resolve(jwtVerifier);
+                const decoded = await new Promise((resolve) => {
+                    jsonwebtoken.verify(token, jwtSec, (err, res) => {
+                        resolve(res || {});
+                    });
+                });
+                return decoded.id === userId;
+            };
+
+        /** @type {AuditLogCallback} */
+        this.callback = null;
+    }
+
+    /**
+     * Add a log
+     *
+     * @param {TrackingEvent} event
+     * @param {User} user
+     * @param {Meta} [meta]
+     * @param {string} [wid] - workspace ID
+     * @param {string} [type]
+     * @param {string} [level]
+     * @param {Date} [date]
+     * @returns {Promise}
+     */
+    async log (
+        event,
+        user = {},
+        meta = {},
+        wid = this.defaultWid,
+        type = TYPE_INFO,
+        level = LEVEL_IMPORTANT,
+        date = new Date()
+    ) {
+        const {
+            type: eventType = 'audit',
+            ...rest
+        } = event;
+        const entry = {
+            date,
+            eventType,
+            ...rest,
+            level,
+            meta,
+            type,
+            user,
+            wid
+        };
+
+        const secret = await Promise.resolve(this._secret);
+        const stored = await this._storeWithRetry(secret, entry);
+        if (!this.callback) {
+            return;
+        }
+        try {
+            await this.callback(stored);
+        } catch (e) {
+            this._log.error('Failed to send AuditLog', e, entry);
+        }
+
+        // logEvent(level, type, workflowType, workflowInstance, eventData, account)
+        // level is the criticality of the event ('Critical','Important','Debug').
+        // type is the type of the event ('Error','Warn','Info').
+        /**
+         *  - log (N/A)
+            - report (Debug)
+            - conversation (N/A)
+            - audit (Important)
+            - user (N/A)
+         */
+    }
+
+    /**
+     *
+     * @param {string} [wid] - workspace id
+     * @param {number} [fromSeq] - for paging
+     * @param {number} [limit]
+     * @returns {Promise<LogEntry[]>}
+     */
+    async list (wid = this.defaultWid, fromSeq = 0, limit = 40) {
+        const c = await this._getCollection();
+
+        const cond = { wid };
+
+        if (fromSeq) {
+            cond.seq = { $lt: fromSeq };
+        }
+
+        const data = await c.find({ wid })
+            .limit(limit + 1)
+            .sort({ seq: -1 })
+            .project({ _id: 0 })
+            .toArray();
+
+        const secret = await Promise.resolve(this._secret);
+
+        const len = data.length === limit + 1
+            ? data.length - 1
+            : data.length;
+        const ret = new Array(len);
+
+        let verifyUser = false;
+        for (let i = 0; i < len; i++) {
+            const {
+                sign,
+                ...log
+            } = data[i];
+
+            verifyUser = verifyUser || (log.user.id && log.user.jwt);
+
+            if (secret) {
+                const previous = data[i + 1] || { sign: null };
+                const objToSign = this._objectToSign(log);
+                const compare = this._signWithSecret(objToSign, secret, previous.sign);
+                log.ok = compare === sign;
+                if (!log.ok) {
+                    this._log.error(`AuditLog: found wrong signature at wid: "${log.wid}", seq: "${log.seq}"`, log);
+                }
+            } else {
+                log.ok = null;
+            }
+
+            ret[i] = log;
+        }
+
+        if (verifyUser && this._jwtVerify) {
+            return Promise.all(
+                ret.map(async (log) => {
+                    if (!log.user.id || !log.user.jwt) {
+                        return log;
+                    }
+                    const userChecked = await this._jwtVerify(log.user.jwt, log.user.id, log.user);
+                    return Object.assign(log, {
+                        // it's ok, when there was null
+                        ok: log.ok !== false && userChecked
+                    });
+                })
+            );
+        }
+
+        return ret;
+    }
+
+    _wait (ms) {
+        return new Promise((r) => setTimeout(r, ms));
+    }
+
+    async _storeWithRetry (secret, entry, delta = 0, i = 1) {
+        const start = Date.now();
+        Object.assign(entry, { delta });
+        try {
+            await this._store(entry, secret);
+            return entry;
+        } catch (e) {
+            if (e.code === 11000) {
+                // duplicate key
+                if (i >= this.maxRetries) {
+                    throw new Error('AuditLog: cannot store log due to max-retries');
+                } else {
+                    await this._wait((i * 50) + (Math.random() * 100));
+                    const add = Date.now() - start;
+                    return this._storeWithRetry(secret, entry, delta + add, i + 1);
+                }
+            } else if (this.muteErrors) {
+                this._log.error('Audit log store error', e, entry);
+                return entry;
+            } else {
+                throw e;
+            }
+        }
+    }
+
+    async _store (entry, secret) {
+        const c = await this._getCollection();
+
+        const previous = await c.findOne({
+            wid: entry.wid
+        }, {
+            sort: { seq: -1 },
+            projection: { seq: 1, _id: 0, sign: 1 }
+        });
+
+        Object.assign(entry, {
+            seq: previous ? previous.seq + 1 : 0
+        });
+
+        let insert;
+        if (secret) {
+            insert = this._objectToSign(entry);
+            const sign = this._signWithSecret(insert, secret, previous ? previous.sign : null);
+            Object.assign(insert, { sign });
+        } else {
+            insert = {
+                ...entry,
+                sign: null
+            };
+        }
+
+        // @ts-ignore
+        await c.insertOne(insert);
+    }
+
+}
+
+module.exports = AuditLogStorage;