monastery 2.2.3 → 3.0.1

package/docs/readme.md

@@ -2,16 +2,26 @@
 
 [![NPM](https://img.shields.io/npm/v/monastery.svg)](https://www.npmjs.com/package/monastery) [![Build Status](https://travis-ci.com/boycce/monastery.svg?branch=master)](https://app.travis-ci.com/github/boycce/monastery)
 
+> v3.0 has been released 🎉 refer to [breaking changes](#v3-breaking-changes) below when upgrading from v2.x.
+
 ## Features
 
-* User friendly API design, built around the awesome [Monk](https://automattic.github.io/monk/)
-* Simple CRUD operations with model population
-* Model validation deriving from your model definitions
-* Custom error messages can be defined in your model definition
-* Normalised error responses ready for client consumption
-* Automatic mongodb index setup
-* CRUD operations can accept bracket (multipart/form-data) and dot notation data formats, you can also mix these together
+* User friendly API design, *inspired by SailsJS*
+* Simple CRUD operations, with simple but fast model population
+* Model validation controlled by your model definitions
+* Normalized error responses objects ready for client consumption
+* Custom error messages can be defined in your model definitions
+* Blacklist sensitive fields once in your model definition, or per operation
+* Model methods can accept data in bracket (multipart/form) and dot notation, you can also mix these together
+* Automatic Mongo index creation
+
+#### Why Monastery over Mongoose?
 
+* User friendly API designed for busy agencies, allowing you to quickly build projects without distractions
+* Model schema and configurations are all defined within a single object (model definition)
+* You can blacklist/exclude sensitive model fields in the model definition for each CRUD operation
+* Model population uses a single aggregation call instead of multiple queries for faster responses
+* Errors throw normalized error objects that contain the model, field, error message etc, handy in the client
 
 ## Install
 
@@ -26,9 +36,8 @@ $ npm install --save monastery
 ```javascript
 import monastery from 'monastery'
 
-// Initialise a monastery manager
+// Initialize a monastery manager
 const db = monastery('localhost/mydb')
-// const db = monastery('user:pass@localhost:port/mydb')
 
 // Define a model
 db.model('user', {
@@ -41,18 +50,16 @@ db.model('user', {
 })
 
 // Insert some data
-db.user.insert({
-  data: {
-    name: 'Martin Luther',
-    pets: ['sparky', 'tiny'],
-    address: { city: 'Eisleben' },
-    points: [[1, 5], [3, 1]]
-  }
-
-}).then(data => {
-  // valid data..
-
-}).catch(errs => {
+try {
+  const newUser = await db.user.insert({
+    data: {
+      name: 'Martin Luther',
+      pets: ['sparky', 'tiny'],
+      address: { city: 'Eisleben' },
+      points: [[1, 5], [3, 1]]
+    }
+  })
+} catch (errs) {
   // [{
   //   detail: "Value needs to be at least 10 characters long.",
   //   status: "400",
@@ -63,31 +70,26 @@ db.user.insert({
   //     rule: "minLength"
   //   }
   // }]
-})
+}
 ```
-## Versions
-
-- Monk: `v7.3.4`
-- MongoDB NodeJS driver: `v3.7.4` ([MongoDB compatibility](https://www.mongodb.com/docs/drivers/node/current/compatibility/#compatibility))
-- MongoDB: [`v5.0.0`](https://www.mongodb.com/docs/v5.0/reference/) [(`v6.0.0` partial support)](https://www.mongodb.com/docs/v6.0/reference/)
-  
-## Debugging
+## Version Compatibility
 
-This package uses [debug](https://github.com/visionmedia/debug) which allows you to set different levels of output via the `DEBUG` environment variable. Due to known limations `monastery:warning` and `monastery:error` are forced on, you can however disable these via [manager settings](./manager).
-
-```bash
-$ DEBUG=monastery:info # shows operation information
-```
+You can view MongoDB's [compatibility table here](https://www.mongodb.com/docs/drivers/node/current/compatibility/), and see all of MongoDB NodeJS Driver [releases here](https://mongodb.github.io/node-mongodb-native/)
 
-To run isolated tests with Jest:
+| Monastery            | Mongo NodeJS Driver | MongoDB Server    | Node                |
+| :------------------- | :-----------------: | :---------------: | ------------------: |
+| `3.x` | [`5.9.x`](https://mongodb.github.io/node-mongodb-native/5.9/) | `>=3.6  <=7.x` | `>=14.x <=latest` |
+| `2.x` | [`3.7.x`](https://mongodb.github.io/node-mongodb-native/3.7/api/) | `>=2.6  <=6.x` | `>=4.x  <=14.x` |
 
-```bash
-npm run dev -- -t 'Model indexes'
-```
 
-## Contributing
+## v3 Breaking Changes
 
-Coming soon...
+  - Removed callback functions on all model methods, you can use the returned promise instead
+  - model.update() now returns the following _update property: `{ acknowledged: true, matchedCount: 1, modifiedCount: 1, upsertedCount: 0, upsertedId: null }` instead of `{ n: 1, nModified: 1, ok: 1 }`
+  - model.remove() now returns `{ acknowledged: true, deletedCount: 1 }`, instead of `{ results: {n:1, ok:1} }`
+  - Models are now added to db.models instead of db.model, e.g. db.models.user
+  - MongoDB connection can be found here db.db changed from db._db
+  - model._indexes() now returns collection._indexes() not collection._indexInformation()
 
 ## Roadmap
 
@@ -103,19 +105,38 @@ Coming soon...
 - ~~Ability to change ACL default on the manager~~
 - ~~Public db.arrayWithSchema method~~
 - ~~Added support for array population~~
+- ~~MongoClient instances can now be reused when initializing the manager, e.g. `monastery(mongoClient)`, handy for migrate-mongo~~
 - Change population warnings into errors
 - Global after/before hooks
-- before hooks can receive a data array, remove this
-- docs: Make the implicit ID query conversion more apparent
-- Split away from Monk so we can update the MongoDB NodeJS Driver version
+- Before hooks can receive a data array, remove this
+- Docs: Make the implicit ID query conversion more apparent
+- ~~Split away from Monk so we can update the MongoDB NodeJS Driver version~~
 - Add a warning if an invalid model is referenced in jthe schema
 - Remove leading forward slashes from custom image paths (AWS adds this as a seperate folder)
-- double check await db.model.remove({ query: idfromparam }) doesnt cause issues for null, undefined or '', but continue to allow {}
-- ~~can't insert/update model id (maybe we can allow this and add _id to default insert/update blacklists)~~
+- Double check await db.model.remove({ query: idfromparam }) doesnt cause issues for null, undefined or '', but continue to allow {}
+- ~~Can't insert/update model id (maybe we can allow this and add _id to default insert/update blacklists)~~
 - timstamps are blacklisted by default (instead of the `timestamps` opt), and can be switched off via blacklisting
 - Allow rules on image types, e.g. `required`
-- test importing of models
-- Docs: model.methods
+- Test importing of models
+- ~~Docs: model.methods~~
+- ~~Convert hooks to promises~~
+- ~~added `model.count()`~~
+
+## Debugging
+
+This package uses [debug](https://github.com/visionmedia/debug) which allows you to set different levels of output via the `DEBUG` environment variable. Due to known limations `monastery:warning` and `monastery:error` are forced on, you can however disable these via [manager settings](./manager).
+
+```bash
+$ DEBUG=monastery:info # shows operation information
+```
+
+## Contributing
+
+All pull requests are welcome. To run isolated tests with Jest:
+
+```bash
+npm run dev -- -t 'Model indexes'
+```
 
 ## Special Thanks
 
@@ -123,4 +144,4 @@ Coming soon...
 
 ## License
 
-Copyright 2020 Ricky Boyce. Code released under the MIT license.
+Copyright 2024 Ricky Boyce. Code released under the MIT license.

package/lib/collection.js

@@ -0,0 +1,324 @@
+const util = require('./util.js')
+
+function Collection (manager, name, options) {
+  this.col = null
+  this.manager = manager
+  this.name = name
+  this.options = options
+}
+
+Collection.prototype._middleware = async function (args) {
+  /**
+   * Modfy the arguments before passing them to the MongoDB driver collection operations
+   * @return {Object} args
+   */
+  const objectsToCast = ['operations', 'query', 'data', 'update']
+  let { fields, opts, query } = args
+  if (!opts) args.opts = opts = {}
+
+  // Get the collection
+  this.col = this.col || this.manager.db.collection(this.name)
+
+  // Query: convert strings to ObjectIds
+  if (query) {
+    if (typeof query === 'string' || typeof query.toHexString === 'function') {
+      args.query = {_id: args.query}
+    }
+  }
+
+  // Options: setup
+  if (typeof opts === 'string' || Array.isArray(opts)) {
+    throw new Error('You can no longer pass an array or string `projection` to find()')
+  } else {
+    // MongoDB 5.0 docs seem to be a little off, projection is still included in `opts`...
+    if (opts.fields) opts.projection = _fields(opts.fields, 0)
+    if (opts.sort) opts.sort = _fields(opts.sort, -1)
+    delete opts.fields
+  }
+
+  // Options: use collection defaults
+  for (let key in this.options) {
+    if (typeof opts[key] === 'undefined') {
+      opts[key] = this.options[key]
+    }
+  }
+
+  // Options: castIds (defaults on)
+  if (opts.castIds !== false) {
+    for (const key of objectsToCast) {
+      if (args[key]) args[key] = util.cast(args[key])
+    }
+  }
+
+  // Fields: convert strings/arrays to objects, e.g. 'name _id' -> {name: 1, _id: 1}
+  if (fields && !util.isObject(fields)) {
+    let fieldsArray = typeof fields === 'string' ? fields.split(' ') : (fields || [])
+    args.fields = fieldsArray.reduce((acc, fieldName) => {
+      acc[fieldName] = 1
+      return acc
+    }, {})
+  }
+
+  function _fields (obj, numberWhenMinus) {
+    if (!Array.isArray(obj) && typeof obj === 'object') return obj
+    obj = typeof obj === 'string' ? obj.split(' ') : (obj || [])
+    let fields = {}
+    for (let i = 0, l = obj.length; i < l; i++) {
+      if (obj[i][0] === '-') fields[obj[i].substr(1)] = numberWhenMinus
+      else fields[obj[i]] = 1
+    }
+    return fields
+  }
+
+  return args
+}
+
+Collection.prototype.aggregate = async function (stages, opts) {
+  const args = await this._middleware({ stages, opts })
+  return this.col.aggregate(args.stages, args.opts).toArray()
+}
+
+Collection.prototype.bulkWrite = async function (operations, opts) {
+  const args = await this._middleware({ operations, opts })
+  return this.col.bulkWrite(args.operations, args.opts)
+}
+
+Collection.prototype.count = async function (query, opts) {
+  const args = await this._middleware({ query, opts })
+  const { estimate, ..._opts } = args.opts
+  if (estimate) {
+    return this.col.estimatedDocumentCount(_opts)
+  } else {
+    return this.col.countDocuments(args.query, _opts)
+  }
+}
+
+Collection.prototype.createIndex = async function (indexSpec, opts) {
+  const args = await this._middleware({ indexSpec, opts })
+  return await this.col.createIndex(args.indexSpec, args.opts)
+}
+
+Collection.prototype.createIndexes = async function (indexSpecs, opts) {
+  const args = await this._middleware({ indexSpecs, opts }) // doesn't currently accept string or array parsing.
+  return await this.col.createIndexes(args.indexSpecs, args.opts)
+}
+
+Collection.prototype.distinct = async function (field, query, opts) {
+  const args = await this._middleware({ field, query, opts })
+  return this.col.distinct(args.field, args.query, args.opts)
+}
+
+Collection.prototype.drop = async function () {
+  try {
+    await this._middleware({})
+    await this.col.drop()
+    // this.col = null
+  } catch (err) {
+    if (err?.message == 'ns not found') return 'ns not found'
+    throw err
+  }
+}
+
+Collection.prototype.dropIndex = async function (name, opts) {
+  const args = await this._middleware({ name, opts })
+  return await this.col.dropIndex(args.name, args.opts)
+}
+
+Collection.prototype.dropIndexes = async function () {
+  await this._middleware({})
+  return this.col.dropIndexes()
+}
+
+Collection.prototype.find = async function (query, opts) {
+  // v3.0 - removed the abillity to pass an array or string to opts as `opts.projection`
+  // v3.0 - find().each() is removed, use `opts.stream` instead
+  const args = await this._middleware({ query, opts })
+  query = args.query
+  opts = args.opts
+  const rawCursor = opts.rawCursor
+
+  // Get the raw cursor 
+  const cursor = this.col.find(query, opts)
+
+  // If a raw cursor is requested, return it now
+  if (rawCursor) return cursor
+
+  // If no stream is requested, return now the array of results
+  if (!opts.stream) return cursor.toArray()
+
+  if (typeof opts.stream !== 'function') {
+    throw new Error('opts.stream must be a function')
+  }
+
+  const stream = cursor.stream()
+  return new Promise((resolve, reject) => {
+    let closed = false
+    let finished = false
+    let processing = 0
+    function close () {
+      closed = true
+      processing -= 1
+      cursor.close()
+    }
+    function pause () {
+      processing += 1
+      stream.pause()
+    }
+    function resume () {
+      processing -= 1
+      stream.resume()
+      if (processing === 0 && finished) done()
+    }
+    function done () {
+      finished = true
+      if (processing <= 0) resolve()
+    }
+  
+    stream.on('close', done)
+    stream.on('end', done)
+    stream.on('error', (err) => reject(err))
+    stream.on('data', (doc) => {
+      if (!closed) opts.stream(doc, { close, pause, resume })
+    })
+  })
+}
+
+Collection.prototype.findOne = async function (query, opts) {
+  const args = await this._middleware({ query, opts })
+  const docs = await this.col.find(args.query, args.opts).limit(1).toArray()
+  return docs?.[0] || null
+}
+
+Collection.prototype.findOneAndDelete = async function (query, opts) {
+  const args = await this._middleware({ query, opts })
+  const doc = await this.col.findOneAndDelete(args.query, args.opts)
+
+  if (doc && typeof doc.value !== 'undefined') return doc.value
+  if (doc.ok && doc.lastErrorObject && doc.lastErrorObject.n === 0) return null
+  return doc
+}
+
+Collection.prototype.findOneAndUpdate = async function (query, update, opts) {
+  const args = await this._middleware({ query, update, opts })
+  let method = 'findOneAndUpdate'
+
+  if (typeof args.opts?.returnDocument === 'undefined') {
+    args.opts.returnDocument = 'after'
+  }
+  if (typeof args.opts?.returnOriginal !== 'undefined') {
+    this.manager.warn('The `returnOriginal` option is deprecated, use `returnDocument` instead.')
+    args.opts.returnDocument = args.opts.returnOriginal ? 'before' : 'after'
+  }
+  if (args.opts.replaceOne || args.opts.replace) {
+    method = 'findOneAndReplace'
+  }
+
+  const doc = await this.col[method](args.query, args.update, args.opts)
+  if (doc && typeof doc.value !== 'undefined') return doc.value
+  if (doc.ok && doc.lastErrorObject && doc.lastErrorObject.n === 0) return null
+  return doc
+}
+
+Collection.prototype.geoHaystackSearch = async function (x, y, opts) {
+  // https://www.mongodb.com/docs/manual/geospatial-queries/
+  throw new Error('geoHaystackSearch is depreciated in MongoDB 4.0, use geospatial queries instead, e.g. $geoWithin')
+}
+
+Collection.prototype.indexInformation = async function (opts) {
+  try {
+    const args = await this._middleware({ opts })
+    return await this.col.indexInformation(args.opts)
+  } catch (e) {
+    // col.indexInformation() throws an error if the collection is created yet...
+    if (e?.message.match(/ns does not exist/)) return {}
+    else throw new Error(e)
+  } 
+}
+
+Collection.prototype.indexes = async function (opts) {
+  try {
+    const args = await this._middleware({ opts })
+    return await this.col.indexes(args.opts)
+  } catch (e) {
+    // col.indexes() throws an error if the collection is created yet...
+    if (e?.message.match(/ns does not exist/)) return []
+    else throw new Error(e)
+  } 
+}
+
+Collection.prototype.insert = async function (data, opts) {
+  const args = await this._middleware({ data, opts })
+  const arrayInsert = Array.isArray(args.data)
+
+
+  if (arrayInsert && args.data.length === 0) {
+    return Promise.resolve([])
+  }
+
+  const doc = await this.col[`insert${arrayInsert ? 'Many' : 'One'}`](args.data, args.opts)
+  if (!doc) return
+
+  // Starting MongoDB 4 the `insert` method only returns the _id, rather than the whole doc.
+  // We need to return the whole doc for consistency with previous versions.
+  const output = util.deepCopy(args.data)
+  if (arrayInsert) {
+    for (let i=output.length; i--;) {
+      output[i]._id = doc.insertedIds[i]
+    }
+  } else {
+    output._id = doc.insertedId
+  }
+
+  return output
+}
+
+Collection.prototype.mapReduce = async function (map, reduce, opts) {
+  // https://www.mongodb.com/docs/manual/reference/method/db.collection.mapReduce/
+  throw new Error('mapReduce is depreciated in MongoDB 5.0, use aggregation pipeline instead')
+}
+
+Collection.prototype.remove = async function (query, opts) {
+  const args = await this._middleware({ query, opts })
+  const method = args.opts.single || args.opts.multi === false ? 'deleteOne' : 'deleteMany'
+  return this.col[method](args.query, args.opts)
+}
+
+Collection.prototype.stats = async function (opts) {
+  const args = await this._middleware({ opts })
+  return this.col.stats(args.opts)
+}
+
+Collection.prototype.update = async function (query, update, opts) {
+  // v3.0 - returns now an object with the following properties:
+  // {
+  //   acknowledged: true,
+  //   matchedCount: 0,
+  //   modifiedCount: 0,
+  //   upsertedCount: 1,
+  //   upsertedId: expect.any(ObjectId),
+  // }
+  // was:
+  // {
+  //   result: { 
+  //     ok: 1,  
+  //     n: 1, 
+  //     nModified: 1, 
+  //     upserted: [{ _id: expect.any(ObjectId) }],
+  //   }
+  // }
+  const args = await this._middleware({ query, update, opts })
+  let method = args.opts.multi || args.opts.single === false ? 'updateMany' : 'updateOne'
+
+  if (args.opts.replace || args.opts.replaceOne) {
+    if (args.opts.multi || args.opts.single === false) {
+      throw new Error('The `replace` option is only available for single updates.')
+    }
+    method = 'replaceOne'
+  }
+
+  const doc = await this.col[method](args.query, args.update, args.opts)
+  // return doc?.result || doc
+  return doc
+}
+
+module.exports = Collection