monastery 1.36.0 → 1.36.3

package/.eslintrc.json

@@ -9,7 +9,8 @@
   ],
   "globals": {
     "test": true,
-    "expect": true
+    "expect": true,
+    "oid": true
   },
   "parserOptions": {
     "ecmaFeatures": {

package/.travis.yml

@@ -1,4 +1,4 @@
 language: node_js
 node_js:
-  - "node"
+  - 17
 services: mongodb

package/changelog.md

@@ -2,6 +2,29 @@
 
 All notable changes to this project will be documented in this file. See [standard-version](https://github.com/conventional-changelog/standard-version) for commit guidelines.
 
+### [1.36.3](https://github.com/boycce/monastery/compare/1.36.2...1.36.3) (2022-05-27)
+
+
+### Bug Fixes
+
+* order maintained when mixing of formData/dotNotation with normal key values ([898fa90](https://github.com/boycce/monastery/commit/898fa90a25e81ae1d2413e32ca67173fdef733a9))
+
+### [1.36.2](https://github.com/boycce/monastery/compare/1.36.1...1.36.2) (2022-05-26)
+
+
+### Bug Fixes
+
+* nullObjects were skipping required rules ([315af00](https://github.com/boycce/monastery/commit/315af000ea274a165e58a39b850508bd3a417642))
+* refactored tests (oid) ([439aa27](https://github.com/boycce/monastery/commit/439aa279c5073d1652b8a6b81f2fce7ae403b0d5))
+* tests (oid) ([bd92970](https://github.com/boycce/monastery/commit/bd92970957c9cb0bd5e0a4b4c4433fd4b6c6b7d7))
+
+### [1.36.1](https://github.com/boycce/monastery/compare/1.36.0...1.36.1) (2022-04-16)
+
+
+### Bug Fixes
+
+* findOneAndUpdate population, blacklisting, etc ([19c4fc9](https://github.com/boycce/monastery/commit/19c4fc96d8c94d9dd68af2a74af693c8dc7a4c17))
+
 ## [1.36.0](https://github.com/boycce/monastery/compare/1.35.0...1.36.0) (2022-04-15)
 
 

package/docs/{schema/rules.md → definition/field-rules.md}

@@ -1,12 +1,14 @@
 ---
-title: Rules
+title: Field Rules
 nav_order: 5
-parent: Schema
+parent: Model Definition
 ---
 
-# Validation Rules
+# Field Rules
 
-  All rules except for `type` and `required` are ignored if the data value is a **null** or an **empty string**
+  - Field rules are ignored if the value is **undefined**, **null**, or an **empty string** (except `required` and `type`)
+  - `type` is only ignored if the value is **undefined**
+  - `required` is never ignored
 
   Rule | Rule argument
   - | -

package/docs/definition/index.md

@@ -0,0 +1,338 @@
+---
+title: Model Definition
+nav_order: 4
+has_children: true
+---
+
+# Model Definition
+
+Model definition object.
+
+### Table of Contents
+
+- [Fields](#fields)
+- [Field blacklisting](#field-blacklisting)
+- [Field options](#field-options)
+- [MongoDB indexes](#mongodb-indexes)
+- [Custom field rules](#custom-field-rules)
+- [Custom error messages](#custom-error-messages)
+- [Operation hooks](#operation-hooks)
+- [Full example](#full-example)
+
+### Fields
+
+1. Fields can either be a field-type, embedded document, or an array of field-types or embedded documents
+2. Field-types are recognised by having a `type` property defined as a string
+3. Field-types can contain [custom](#custom-field-rules) and [default field rules](./rules), e.g. `{ minLength: 2 }`
+4. Field-types can contain [field options](#field-options).
+
+```js
+{
+  fields: {
+    name: { // Field-type
+      type: 'string',
+      required: true,
+    },
+    address: { // embedded document
+      line1: { type: 'string', required: true },
+      city: { type: 'string', minLength: 2 },
+    },
+    names: [ // array of field-types
+      { type: 'string' },
+    ],
+    pets: [{ // array of embedded documents
+      name: { type: 'string' },
+      type: { type: 'string' },
+    }]
+    // You can add a rule on the embedded document or array using the following structure
+    pets: {
+      type: [{
+        name: { type: 'string' },
+        type: { type: 'string' },
+      }],
+      minLength: 1,
+    }
+  }
+}
+```
+
+The fields below implicitly get assigned and take presidence over any input data when [`manager.timestamps`](./manager) is true (default). You can override the `timestamps` value per operation, e.g. `db.user.update({ ..., timestamps: false})`. These fields use unix timestamps in seconds (by default), but can be configured to use use milliseconds via the manager [`useMilliseconds` ](./manager) option.
+
+```js
+{
+  fields: {
+    createdAt: {
+      type: 'date',
+      insertOnly: true,
+      default: function() { return Math.floor(Date.now() / 1000) }
+    },
+    updatedAt: {
+      type: 'date',
+      default: function() { return Math.floor(Date.now() / 1000) }
+    }
+  }
+}
+```
+
+### Field blacklisting
+
+You are able to provide a list of fields to blacklist per model operation.
+
+```js
+{
+  // The 'password' field will be removed from the results returned from `model.find`
+  findBL: ['password'],
+
+  // The 'password' field will be removed before inserting via `model.insert`
+  insertBL: ['password'],
+
+  // The 'password' and 'createdAt' fields will be removed before updating via `model.update`
+  updateBL: ['createdAt', 'password'],
+}
+```
+
+You are also able to blacklist fields on embedded documents and arrays of embedded documents.
+
+```js
+{
+  // Embedded document example: `address.city` will be excluded from the response
+  findBL: ['address.city']
+
+  // Array of embedded documents example: `meta` will be removed from each comment in the array
+  findBL: ['addresses.city']
+}
+```
+
+### Field options
+
+Here are some other special field options that can be used alongside field rules.
+
+```js
+fieldType: {
+
+  // Enables population, you would save the foreign document _id on this field.
+  model: 'pet',
+
+  // Field will only be allowed to be set on insert when calling model.insert
+  insertOnly: true,
+
+  // Default will always override any passed value (it has some use-cases)
+  defaultOverride: true,
+
+  // Default value
+  default: 12,
+
+  // Default value can be returned from a function. `this` refers to the data object, but be
+  // sure to pass any referenced default fields along with insert/update/validate, e.g. `this.age`
+  default: function(fieldName, model) { return `I'm ${this.age} years old` },
+
+  // Monastery will automatically create a mongodb index for this field, see "MongoDB indexes"
+  // below for more information
+  index: true|1|-1|'text'|'unique'|Object,
+
+  // The field  won't stored, handy for fields that get populated with documents, see ./find for more details
+  virtual: true
+
+}
+```
+
+### MongoDB indexes
+
+You are able to automatically setup MongoDB indexes via the `index` field option.
+
+```js
+fieldType: {
+
+  // This will create an ascending / descending index for this field
+  index: true|1|-1,
+
+  // This will create an ascending unique index which translates:
+  // { key: { [fieldName]: 1 }, unique: true }
+  index: 'unique',
+
+  // Text indexes are handled a little differently in which all the fields on the model
+  // definition that have a `index: 'text` set are collated into one index, e.g.
+  // { key: { [fieldName1]: 'text', [fieldName2]: 'text', .. }}
+  index: 'text'
+
+  // You can also pass an object if you need to use mongodb's index options
+  // https://docs.mongodb.com/manual/reference/command/createIndexes/
+  // https://mongodb.github.io/node-mongodb-native/2.1/api/Collection.html#createIndexes
+  index: { type: 1, ...(any mongodb index option) },
+
+}
+```
+
+And here's how you would use a 2dsphere index, e.g.
+
+```js
+{
+  fields: {
+    location: {
+      index: '2dsphere',
+      type: { type: 'string', default: 'Point' },
+      coordinates: [{ type: 'number' }] // lng, lat
+    }
+  }
+}
+
+// Inserting a 2dsphere point
+await db.user.insert({
+  data: {
+    location: {
+      coordinates: [170.2628528648167, -43.59467883784971]
+    }
+  }
+}
+```
+
+Since unique indexes by default don't allow multiple documents with `null`, you use a partial index (less performant), e.g.
+
+```js
+{
+  fields: {
+    // So, instead of...
+    email: {
+      type: 'string',
+      index: 'unique',
+    },
+    // You would use...
+    email: {
+      type: 'string',
+      index: {
+        type: 'unique',
+        partialFilterExpression: {
+          email: { $type: 'string' }
+        },
+      },
+    },
+  },
+}
+```
+
+### Custom field rules
+
+You are able to define custom field rules to use. (`this` will refer to the data object passed in)
+
+```js
+{
+  rules: {
+    // Basic definition
+    isGrandMaster: function(value, ruleArgument, path, model) {
+      return (value == 'Martin Luther')? true : false
+    },
+    // Full definition
+    isGrandMaster: {
+      message: function(value, ruleArgument, path, model) {
+        return 'Only grand masters are permitted'
+      },
+      fn: function(value, ruleArgument, path, model) {
+        return (value == 'Martin Luther' || this.age > 100)? true : false
+      },
+    },
+  },
+}
+
+// And referencing is the same as any other builtin rule
+{
+  fields: {
+    user: {
+      name: {
+        type: 'string'
+        isGrandMaster: true, // true is the ruleArgument
+      },
+    },
+  },
+}
+
+// Additionally, you can define custom messages here
+{
+  messages: {
+    'user.name': {
+      isGrandMaster: 'Only grand masters are permitted'
+    }
+  },
+}
+```
+
+### Custom error messages
+
+You are able to define custom error messages for each field rule.
+
+```js
+{
+  messages: {
+    'name': {
+      required: 'Sorry, even a monk cannot be nameless'
+      type: 'Sorry, your name needs to be a string'
+    },
+    'address.city': {
+      minLength: function(value, ruleArgument, path, model) {
+        return `Is your city of residence really only ${ruleArgument} characters long?`
+      }
+    },
+    // Assign custom error messages for arrays
+    'pets': {
+      minLength: `Please add at least one pet pet group.`
+    },
+    // You can assign custom error messages for all fields on embedded documents in an array
+    // e.g. pets = [{ name: { type: 'string' }}]
+    'pets.name': {
+      required: `Your pet's name needs to be a string.`
+    },
+    // To target a specific array item
+    'pets.0.name': {
+      required: `You first pet needs a name`
+    },
+  },
+}
+```
+
+### Operation hooks
+
+You are able provide an array of callbacks to these model operation hooks. If you need to throw an error asynchronously, please pass an error as the first argument to `next()`, e.g. `next(new Error('Your error here'))`. You can also access the operation details via `this` in each callback.
+
+```js
+{
+  afterFind: [function(data, next) {}],
+  afterInsert: [function(data, next) {}],
+  afterInsertUpdate: [function(data, next) {}],
+  afterUpdate: [function(data, next) {}],
+  afterRemove: [function(next) {}],
+  beforeInsert: [function(data, next) {}],
+  beforeInsertUpdate: [function(data, next) {}],
+  beforeUpdate: [function(data, next) {}],
+  beforeRemove: [function(next) {}],
+  beforeValidate: [function(data, next) {}],
+}
+```
+
+### Definition example
+
+```js
+{
+  fields: {
+    email: { type: 'email', required: true, index: 'unique' },
+    firstName: { type: 'string', required: true },
+    lastName: { type: 'string' }
+  },
+
+  messages: {
+    email: { required: 'Please enter an email.' }
+  },
+
+  updateBL: ['email'],
+
+  beforeValidate: [function (data, next) {
+    if (data.firstName) data.firstName = util.ucFirst(data.firstName)
+    if (data.lastName) data.lastName = util.ucFirst(data.lastName)
+    next()
+  }],
+
+  afterFind: [function(data) {// Synchronous
+    data = data || {}
+    data.name = data.firstName + ' ' + data.lastName
+  }],
+}
+
+```

package/docs/image-plugin.md

@@ -7,8 +7,6 @@ nav_order: 7
 
 To use the default image plugin shipped with monastery, you need to use the options below when initialising a manager
 
-
-
 ```js
   let db = monastery('localhost/mydb', {
     imagePlugin: {
@@ -21,22 +19,24 @@ To use the default image plugin shipped with monastery, you need to use the opti
   })
 ```
 
-Then in your model schema, e.g.
+Then in your model definition, e.g.
 
 ```js
-let user = db.model('user', { fields: {
-  logo:  {
-    type: 'image', // required
-    formats: ['bmp', 'gif', 'jpg', 'jpeg', 'png', 'tiff'],
-    filename: 'avatar',
-    filesize: 1000 * 1000 * 5, // max size in bytes
-    getSignedUrl: true, // get a s3 signed url after every `find()` operation (can be overridden per request)
-    params: {}, // upload params, https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html#upload-property
-  },
-  logos: [{
-    type: 'image'
-  }],
-}})
+let user = db.model('user', {
+  fields: {
+    logo:  {
+      type: 'image', // required
+      formats: ['bmp', 'gif', 'jpg', 'jpeg', 'png', 'tiff'],
+      filename: 'avatar',
+      filesize: 1000 * 1000 * 5, // max size in bytes
+      getSignedUrl: true, // get a s3 signed url after every `find()` operation (can be overridden per request)
+      params: {}, // upload params, https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html#upload-property
+    },
+    logos: [{
+      type: 'image'
+    }],
+  }
+})
 ```
 
 Then when inserting or updating a document you need to set `files` to an obkect containing containing your parsed files, [express-fileupload](https://github.com/richardgirges/express-fileupload) works great with an express setup, e.g.

package/docs/manager/index.md

@@ -10,16 +10,16 @@ Monastery constructor, same as the [monk constructor](https://automattic.github.
 
 ### Arguments
 
-1. `uri` *(string\|array)*: A [mongo connection string URI](https://docs.mongodb.com/manual/reference/connection-string/). Replica sets can be an array or comma separated.
+`uri` *(string\|array)*: A [mongo connection string URI](https://docs.mongodb.com/manual/reference/connection-string/). Replica sets can be an array or comma separated.
 
-2. [`options`] *(object)*:
-    - [`defaultObjects = false`] *(boolean)*: when [inserting](../model/insert.html#defaults-example), undefined subdocuments and arrays are defined
-    - [`nullObjects = false`] *(boolean)*: subdocuments and arrays can be set to null or an empty string (which gets converted to null). You can override this per field via `field.schema = { nullObject: true }`.
-    - [`timestamps = true`] *(boolean)*: whether to use [`createdAt` and `updatedAt`](../schema.html#fields), this can be overridden per operation
-    - [`useMilliseconds = false`] *(boolean)*: by default the `createdAt` and `updatedAt` fields that get created automatically use unix timestamps in seconds, set this to true to use milliseconds instead.
-    - [`mongo options`](http://mongodb.github.io/node-mongodb-native/3.2/reference/connecting/connection-settings/)...
+[`options`] *(object)*:
+  - [`defaultObjects=false`] *(boolean)*: when [inserting](../model/insert.html#defaults-example), undefined embedded documents and arrays are defined
+  - [`nullObjects=false`] *(boolean)*: embedded documents and arrays can be set to null or an empty string (which gets converted to null). You can override this per field via `nullObject: true`.
+  - [`timestamps=true`] *(boolean)*: whether to use [`createdAt` and `updatedAt`](../definition), this can be overridden per operation
+  - [`useMilliseconds=false`] *(boolean)*: by default the `createdAt` and `updatedAt` fields that get created automatically use unix timestamps in seconds, set this to true to use milliseconds instead.
+  - [`mongo options`](http://mongodb.github.io/node-mongodb-native/3.2/reference/connecting/connection-settings/)...
 
-3. [`callback`] *(function)*: You may optionally specify a callback which will be called once the connection to the mongo database is opened or throws an error.
+[`callback`] *(function)*: You may optionally specify a callback which will be called once the connection to the mongo database is opened or throws an error.
 
 ### Returns
 

package/docs/manager/model.md

@@ -9,13 +9,14 @@ Sets up a model, retrieves a collection, and sets up any required indexes
 
 ### Arguments
 
-1. `name` *(string)*: name of the mongo collection
+`name` *(string)*: name of the mongo collection
 
-2. [`schema`] *(object)*: [Schema](../schema) (and collection level options)
+[`definition`] *(object)*: [definition](../definition)
 
 ### Returns
 
 A [model](../model) instance, the model instance will also be avaliable at:
+
 ```js
 db.user
 db.model.user
@@ -24,5 +25,5 @@ db.model.user
 ### Example
 
 ```js
-const user = db.model('user', schema)
+const user = db.model('user', definition)
 ```

package/docs/manager/models.md

@@ -9,7 +9,7 @@ Setup model definitions from a folder location
 
 ### Arguments
 
-1. `path` *(string)*: path to model definitions, the filenames are used as the corresponding model name.
+`path` *(string)*: path to model definitions, the filenames are used as the corresponding model name.
 
 ### Returns
 

package/docs/model/find.md

@@ -5,18 +5,18 @@ parent: Model
 
 # `model.find`
 
-Find document(s) in a collection and call related hook: `schema.afterFind`
+Find document(s) in a collection and call the model hook: `afterFind`
 
 ### Arguments
 
 `options` *(object)*
 
-- `options.query` *(object\|id)*
-- [[`options.populate`](#populate)] *(array)*
-- [`options.sort`] *(string\|array\|object)*: same as the mongodb option, but allows string parsing e.g. 'name', 'name:1'
-- [`options.blacklist`] *(array\|string\|false)*: augment `schema.findBL`. `false` will remove all blacklisting
-- [`options.getSignedUrls`] *(boolean)*: get signed urls for all image objects
-- [`options.project`] *(string\|array\|object)*: return only these fields, ignores blacklisting
+- `query` *(object\|id)*
+- [[`blacklist`](#blacklisting)] *(array\|string\|false)*: augment `definition.findBL`. `false` will remove all blacklisting
+- [`getSignedUrls`] *(boolean)*: get signed urls for all image objects
+- [[`populate`](#populate)] *(array)*
+- [`project`] *(string\|array\|object)*: return only these fields, ignores blacklisting
+- [`sort`] *(string\|array\|object)*: same as the mongodb option, but allows string parsing e.g. 'name', 'name:1'
 - [[`any mongodb option`](http://mongodb.github.io/node-mongodb-native/3.2/api/Collection.html#find)] *(any)*
 
 [`callback`] *(function)*: pass instead of return a promise
@@ -28,27 +28,24 @@ A promise if no callback is passed in.
 ### Example
 
 ```js
-user.find({ query: "5ebdd6677466b95109aa278e" }).then(data => {
-  // {..}
-})
+await user.find({ query: "5ebdd6677466b95109aa278e" })
+// {..}
 
-user.find({ query: { name: "Martin Luther" }}).then(data => {
-  // [{..}]
-})
+await user.find({ query: { name: "Martin Luther" }})
+// [{..}]
 
-user.find({ query: { name: "Martin Luther" }, limit: 100 }).then(data => {
-  // [{..}]
-})
+await user.find({ query: { name: "Martin Luther" }, limit: 100 })
+// [{..}]
 ```
 
 ### Blacklisting
 
-You can augment the model's `schema.findBL` blacklist by passing a custom `blacklist`:
+You can augment the model's `definition.findBL` blacklist by passing a custom `blacklist`:
 
 ```js
 // Prevents `name` and `pets.$.name` (array) from being returned.
 user.find({ query: {}, blacklist: ['name', 'pets.name'] })
-// You can also whitelist any blacklisted fields found in schema.findBL
+// You can also whitelist any blacklisted fields found in definition.findBL
 user.find({ query: {}, blacklist: ['-name', '-pet'] })
 ```
 
@@ -57,13 +54,15 @@ user.find({ query: {}, blacklist: ['-name', '-pet'] })
 You are able to populate document references to other collections. Behind the scenes
 this uses mongodb's [$lookup](https://docs.mongodb.com/manual/reference/operator/aggregation/lookup/) aggregation operator which can also be passed full $lookup syntax for more control.
 
-Populate is first enabled by including a reference to another document in the schema via `model`.
+Populate is first enabled by including a reference to another document in the field-type via `model`.
 The value of the model reference should be an ID, e.g. `myBook = id`
 
 ```js
-schema.fields = {
-  myBook: {
-    model: 'book'
+{
+  fields: {
+    myBook: {
+      model: 'book'
+    }
   }
 }
 ```
@@ -74,7 +73,7 @@ You are then able to easily populate returned results via a find operation.
 user.find({ query: {...}, populate: ['myBook'] })
 ```
 
-You can also populate within subdocument fields. Although at this time arrays are not supported,
+You can also populate within embedded document fields. Although at this time arrays are not supported,
 you would need to use the [example below](#more-control).
 ```js
 user.find({ query: {...}, populate: ['myBooks.book'] })
@@ -85,7 +84,8 @@ user.find({ query: {...}, populate: ['myBooks.book'] })
 If you would like more control you can either use monk's native
 [aggregate](https://automattic.github.io/monk/docs/collection/aggregate.html) function via
 `user._aggregate`, or simply pass a MongoDB lookup object to populate. When passing a lookup object, the
-populated field still needs to be defined on the schema. See the examples below,
+populated field still needs to be defined in `definition.fields` if you want to call any related hooks,
+and prune any blacklisted fields. See the examples below,
 
 #### Populate a single document
 
@@ -109,11 +109,13 @@ Below populates all books into `user.myBooks` with a `bookOwnerId` equal to the
 isn't stored on the user, you will need define it as virtual field
 
 ```js
-schema.fields = {
-  myBooks: [{
-    model: 'book',
-    virtual: true
-  }]
+{
+  fields: {
+    myBooks: [{
+      model: 'book',
+      virtual: true
+    }],
+  },
 }
 
 user.find({

package/docs/model/findOne.md

@@ -5,7 +5,7 @@ parent: Model
 
 # `model.findOne`
 
-Find a single document and call related hook: `schema.afterFind`
+Find a single document and call the model hook: `afterFind`
 
 ### Arguments
 
@@ -18,7 +18,6 @@ A promise if no callback is passed in.
 ### Example
 
 ```js
-user.findOne({ query: { name: "Martin Luther" }}).then(data => {
-  // {..}
-})
+await user.findOne({ query: { name: "Martin Luther" }})
+// {..}
 ```