@jcbuisson/express-x 1.1.0 → 1.1.3

package/README.md

@@ -1 +1,263 @@
 
+
+# Getting started
+
+## Create a project
+
+Let's create a new folder for our application:
+
+```bash
+mkdir expressx-project
+cd expressx-project
+```
+
+Since any ExpressX application is a Node application, we can create a default package.json using npm:
+
+```bash
+npm init es6 --yes
+```
+
+The `es6` argument adds `"type": "module"` in `package.json`. Beware! All further module imports must made with es6/esm `import` syntax.
+
+
+## Install ExpressX
+
+```bash
+npm install @jcbuisson/express-x
+```
+
+## Our first server application
+
+Now we can create an ExpressX application which will provide a complete REST API on a `user` resource
+backed in a [Prisma](https://www.prisma.io/) database
+
+```js
+// app.js
+import express from 'express'
+import bodyParser from 'body-parser'
+import expressX from '@jcbuisson/express-x'
+
+// `app` is a regular express application, enhanced with express-x features
+const app = expressX(express())
+
+// create two CRUD database services. They provide Prisma methods: `create`, 'createMany', 'find', 'findMany', 'upsert', etc.
+app.createDatabaseService('User')
+app.createDatabaseService('Post')
+
+// add body parsers for http requests
+app.use(bodyParser.json())
+app.use(bodyParser.urlencoded({ extended: false }))
+
+// add http/rest endpoints
+app.addHttpRest('/api/user', app.service('User'))
+app.addHttpRest('/api/post', app.service('Post'))
+
+app.server.listen(8000, () => console.log(`App listening at http://localhost:8000`))
+```
+
+Before running it we need to setup the corresponding database.
+
+
+## Create the database
+
+Presently, ExpressX only handles [Prisma](https://www.prisma.io/). Prisma is able to connect to most brands of relational or NoSQL databases.
+
+First, provide the database schema in `prisma/schema.prisma`:
+```prisma
+generator client {
+   provider = "prisma-client-js"
+}
+
+model User {
+   id          Int       @default(autoincrement()) @id
+   name        String
+   posts       Post[]
+}
+
+model Post {
+   id          Int       @default(autoincrement()) @id
+   text        String
+   author      User      @relation(fields: [authorId], references: [id], onDelete: Cascade)
+   authorId    Int
+}
+
+datasource db {
+   provider = "sqlite"
+   url      = "file:./dev.db"
+}
+```
+
+Then create the database:
+```bash
+npx prisma migrate dev --name init
+```
+
+The sqlite database file is created at `prisma/dev.db`
+
+
+## Run the application
+
+```bash
+node app.js
+```
+
+It prints the following lines in the console:
+```bash
+created service 'user' over table 'User'
+created service 'post' over table 'Post'
+added HTTP endpoints for service 'user' at path '/api/user'
+added HTTP endpoints for service 'post' at path '/api/post'
+App listening at http://localhost:8000
+```
+
+
+## Enjoy your HTTP REST API
+
+Now you can try the HTTP endpoints `/api/user` and `/api/post`; open a new console and run HTTP requests:
+```bash
+curl -X POST -H 'Content-Type: application/json' -d '{"name":"JC"}' http://localhost:8000/api/user
+# --> {"id":1,"name":"JC"}
+curl http://localhost:8000/api/user
+# --> [{"id":1,"name":"JC"}]
+```
+
+With a few lines of code, we got a complete REST API over the database tables. But there is more!
+
+
+## Use it with a websocket client
+
+First install ExpressX client library:
+
+```bash
+npm i @jcbuisson/express-x-client
+```
+
+Create the following client NodeJS script:
+
+```js
+// client.js
+import io from 'socket.io-client'
+import expressxClient from '@jcbuisson/express-x-client'
+
+const socket = io('http://localhost:8000', { transports: ["websocket"] })
+
+const app = expressxClient(socket)
+
+async function main() {
+   const user = await app.service('User').create({
+      name: "Joe",
+   })
+   await app.service('Post').create({
+      authorId: user.id,
+      text: "Post#1"
+   })
+   await app.service('Post').create({
+      authorId: user.id,
+      text: "Post#2"
+   })
+   const joe = await app.service('User').findUnique({
+      where: {
+         id: user.id,
+      },
+      include: {
+         posts: true,
+      },
+   })
+   console.log('joe', joe)
+   process.exit(0)
+}
+main()
+```
+
+For simplicity we use a node client, but you of course you would write something similar with your favorite front-end framework.
+
+You can use on the client side the exact same statements on services as you would on the server side, such as: `app.service('User').create(...)`.
+Of course the `app` object here on the client is quite different that the `app` object on the server; you can find explanations [here]().
+
+Now run the client script:
+```bash
+node client.js
+```
+
+It prints the following lines in the console:
+```
+joe {
+  id: 11,
+  name: 'Joe',
+  posts: [
+    { id: 12, text: 'Post#1', authorId: 11 },
+    { id: 13, text: 'Post#2', authorId: 11 }
+  ]
+}
+```
+
+We have a GraphQL-like experience with the nested posts, thanks to Prisma and its use through ExpressX services.
+
+::: info
+We could have removed from `app.js` all lines related to HTTP, since we are only using the websocket transport.
+:::
+
+
+## Real-time applications
+
+When websocket transport is used (default situation) and when a connected client calls a service method,
+two twings happen on method completion:
+
+- the resulting value is sent to the client
+- an event is emitted, and sent to connected clients we'll call subscribers. The calling client may or not be one of those subscribers.
+
+For example in a medical application, whenever a patients's record is modified, an event could be sent to all his/her caregivers.
+
+***Channels*** are used for this pub/sub mechanism. Service methods ***publish*** events on ***channels***, and clients ***subscribe***
+to channels in order to receive those events. ExpressX provides functions to configure which events are published to which channels.
+A channel is represented by a name and you can create and use as many channels as you need.
+
+In the following example, every time a client connects to the server, it joins (= is subscribed to) the 'anonymous' channel.
+And whenever an event is emited by the `post` or `user` service, this event is published on this channel,
+and then broacasted to all connected clients, leading to real-time updates.
+
+```js
+// app.js
+import express from 'express'
+import expressX from '@jcbuisson/express-x'
+import { PrismaClient } from '@prisma/client'
+
+// `app` is a regular express application, enhanced with express-x features
+const app = expressX(express())
+
+// configure prisma client from schema
+app.set('prisma', new PrismaClient())
+
+// create two CRUD database services. They provide Prisma methods: `create`, 'createMany', 'find', 'findMany', 'upsert', etc.
+app.createDatabaseService('User')
+app.createDatabaseService('Post')
+
+// publish
+app.service('User').publish(async (post, context) => {
+   return ['anonymous']
+})
+app.service('Post').publish(async (post, context) => {
+   return ['anonymous']
+})
+
+// subscribe
+app.on('connection', (connection) => {
+   console.log('connection', connection.id)
+   app.joinChannel('anonymous', connection)
+})
+
+app.server.listen(8000, () => console.log(`App listening at http://localhost:8000`))
+```
+
+Here is how a client may listen to channel events:
+
+```js
+// client.js
+...
+app.service('Post').on('create', post => {
+   console.log('post event created', post)
+})
+```
+
+The listener is triggered whenever the client receives from the server a `create` event from the service `post`.
+This event results from the completion on the server of a call `app.service('Post').create()`

package/package.json

@@ -1,26 +1,32 @@
 {
   "name": "@jcbuisson/express-x",
-  "version": "1.1.0",
+  "version": "1.1.3",
   "description": "",
   "type": "module",
   "main": "src/index.mjs",
   "repository": {
     "type": "git",
-    "url": "git@gitlab.com:buisson31/express-x.git"
+    "url": "git@github.com:jcbuisson/express-x.git"
   },
   "author": "Jean-Christophe Buisson <buisson@enseeiht.fr> (jcbuisson.dev)",
   "license": "MIT",
   "private": false,
   "scripts": {
-  },
+    "test": "mocha --require esm 'test/**/*.test.js'",
+    "generate": "npx prisma generate",
+    "migrate": "npx prisma migrate dev --name init"
+},
   "keywords": [],
   "dependencies": {
     "@prisma/client": "^4.10.1",
+    "axios": "^1.4.0",
     "config": "^3.3.9",
+    "esm": "^3.2.25",
     "express": "^4.18.2",
     "socket.io": "^4.6.0"
   },
   "devDependencies": {
-    "@types/express": "^4.17.17"
+    "chai": "^4.3.7",
+    "mocha": "^10.2.0"
   }
 }

package/prisma/schema.prisma

@@ -0,0 +1,22 @@
+generator client {
+   provider = "prisma-client-js"
+}
+
+model User {
+   id          Int       @default(autoincrement()) @id
+   name        String
+   email       String    @unique
+   posts       Post[]
+}
+
+model Post {
+   id          Int       @default(autoincrement()) @id
+   text        String
+   author      User      @relation(fields: [authorId], references: [id], onDelete: Cascade)
+   authorId    Int
+}
+
+datasource db {
+   provider = "sqlite"
+   url      = "file:./dev.db"
+}

package/src/index.mjs

@@ -8,7 +8,7 @@ import { PrismaClient } from '@prisma/client'
  */
 function expressX(app, options={}) {
 
-   if (options.debug === undefined) options.debug = true
+   if (options.debug === undefined) options.debug = false
    if (options.ws === undefined) options.ws = { ws_prefix: "expressx" }
 
    const services = {}
@@ -19,62 +19,23 @@ function expressX(app, options={}) {
    /*
     * create a service `name` based on Prisma table `entity`
     */
-   function createDatabaseService(name, { entity=name }) {
+   function createDatabaseService(name, prismaOptions = { entity: name }) {
+
       let prisma = app.get('prisma')
       if (!prisma) {
          prisma = new PrismaClient()
          app.set('prisma', prisma)
       }
-      
-      const service = createService(name, {
-         create: (data) => {
-            if (options.debug) console.log('create', name, data)
-            return prisma[entity].create({
-               data,
-            })
-         },
-
-         get: (id) => {
-            if (options.debug) console.log('get', name, id)
-            return prisma[entity].findUnique({
-               where: {
-                  id,
-               },
-            })
-         },
-
-         patch: (id, data) => {
-            if (options.debug) console.log('patch', name, id, data)
-            return prisma[entity].update({
-               where: {
-                  id,
-               },
-               data,
-            })
-         },
-
-         remove: (id) => {
-            if (options.debug) console.log('remove', name, id)
-            return prisma[entity].delete({
-               where: {
-                  id,
-               },
-            })},
 
-         find: (options) => {
-            if (options.debug) console.log('find', name, options)
-            return prisma[entity].findMany(options)
-         },
+      // take all prisma methods on `entity` table
+      const methods = prisma[prismaOptions.entity]
 
-         upsert: (options) => {
-            if (options.debug) console.log('upsert', name, options)
-            return prisma[entity].upsert(options)
-         },
-      })
+      const service = createService(name, methods)
+      
       service.prisma = prisma
-      service.entity = entity
+      service.entity = prismaOptions.entity
       
-      if (options.debug) console.log(`created service '${name}' over table '${entity}'`)
+      if (options.debug) console.log(`created service '${name}' over table '${prismaOptions.entity}'`)
       return service
    }
 
@@ -86,6 +47,7 @@ function expressX(app, options={}) {
 
       for (const methodName in methods) {
          const method = methods[methodName]
+         if (! method instanceof Function) continue
 
          // `context` is the context of execution (transport type, connection, app)
          // `args` is the list of arguments of the method
@@ -151,16 +113,41 @@ function expressX(app, options={}) {
    /*
     * add an HTTP REST endpoint at `path`, based on `service`
     */
-   function addHttpRest(path, service) {
+   async function addHttpRest(path, service) {
       const context = {
          app,
          http: { name: service.name }
       }
 
+      // introspect table schema
+      async function getFieldTypes() {
+         const fieldTypes = {}
+         if (service.prisma._activeProvider === 'sqlite') {
+            const fieldInfo = await service.prisma.$queryRawUnsafe(`
+               PRAGMA table_info(${service.entity})
+            `)
+            fieldInfo.forEach(column => {
+               fieldTypes[column.name] = column.type.toLowerCase()
+            })
+         } else if (service.prisma._activeProvider === 'postgresql') {
+            const fieldInfo = await service.prisma.$queryRawUnsafe(`
+               SELECT column_name, data_type
+               FROM information_schema.columns
+               WHERE table_name = '${service.entity}';
+            `)
+            fieldInfo.forEach(column => {
+               fieldTypes[column.column_name] = column.data_type
+            })
+         }
+         return fieldTypes
+      }
+
+
       app.post(path, async (req, res) => {
          context.http.req = req
          try {
-            const value = await service.__create(context, req.body)
+            const value = await service.__create(context, { data: req.body })
+            publish(service, 'create', value)
             res.json(value)
          } catch(err) {
             console.log('callErr', err)
@@ -171,30 +158,31 @@ function expressX(app, options={}) {
       app.get(path, async (req, res) => {
          context.http.req = req
          const query = { ...req.query }
-         for (const fieldName in query) {
-            const fieldInfo = await service.prisma.$queryRawUnsafe(`
-               SELECT column_name, data_type
-               FROM information_schema.columns
-               WHERE table_name = '${service.entity}' AND column_name = '${fieldName}';
-            `)
-            const fieldType = fieldInfo[0].data_type
-            if (fieldType === 'integer') {
-               query[fieldName] = parseInt(query[fieldName])
-            } else if (fieldType === 'numeric') {
-               query[fieldName] = parseFloat(query[fieldName])
-            } else if (fieldType === 'boolean') {
-               query[fieldName] = (query[fieldName] === 't') ? true : false
-            } else if (fieldType === 'text' || fieldType === 'character varying') {
-               query[fieldName] = query[fieldName]
-            } else {
-               // ?
-               query[fieldName] = query[fieldName]
-            }
-         }
          try {
-            const values = await service.__find(context, {
+            // the values in `req.query` are all strings, but Prisma need proper types
+            // we need to introspect column types and do the proper transtyping
+            for (const fieldName in query) {
+               if (!service.fieldTypes) service.fieldTypes = await getFieldTypes()
+               const fieldType = service.fieldTypes[fieldName]
+
+               if (fieldType === 'integer') {
+                  query[fieldName] = parseInt(query[fieldName])
+               } else if (fieldType === 'numeric') {
+                  query[fieldName] = parseFloat(query[fieldName])
+               } else if (fieldType === 'boolean') {
+                  query[fieldName] = (query[fieldName] === 't') ? true : false
+               } else if (fieldType === 'text' || fieldType === 'character varying') {
+                  query[fieldName] = query[fieldName]
+               } else {
+                  // ?
+                  query[fieldName] = query[fieldName]
+               }
+            }
+
+            const values = await service.__findMany(context, {
                where: query,
             })
+            publish(service, 'findMany', values)
             res.json(values)
          } catch(err) {
             console.log('callErr', err)
@@ -205,7 +193,12 @@ function expressX(app, options={}) {
       app.get(`${path}/:id`, async (req, res) => {
          context.http.req = req
          try {
-            const value = await service.__get(context, parseInt(req.params.id))
+            const value = await service.__findUnique(context, {
+               where: {
+                  id: parseInt(req.params.id)
+               }
+            })
+            publish(service, 'findUnique', value)
             res.json(value)
          } catch(err) {
             console.log('callErr', err)
@@ -216,7 +209,13 @@ function expressX(app, options={}) {
       app.patch(`${path}/:id`, async (req, res) => {
          context.http.req = req
          try {
-            const value = await service.__patch(context, parseInt(req.params.id), req.body)
+            const value = await service.__update(context, {
+               where: {
+                  id: parseInt(req.params.id),
+               },
+               data: req.body,
+            })
+            publish(service, 'update', value)
             res.json(value)
          } catch(err) {
             console.log('callErr', err)
@@ -227,7 +226,12 @@ function expressX(app, options={}) {
       app.delete(`${path}/:id`, async (req, res) => {
          context.http.req = req
          try {
-            const value = await service.__remove(context, parseInt(req.params.id))
+            const value = await service.__delete(context, {
+               where: {
+                  id: parseInt(req.params.id)
+               }
+            })
+            publish(service, 'delete', value)
             res.json(value)
          } catch(err) {
             console.log('callErr', err)
@@ -295,23 +299,7 @@ function expressX(app, options={}) {
                            result,
                         })
                         // pub/sub: send event on associated channels
-                        const publishFunc = service.publishCallback
-                        if (publishFunc) {
-                           const channelNames = await publishFunc(result, app)
-                           if (options.debug) console.log('publish channels', name, action, channelNames)
-                           for (const channelName of channelNames) {
-                              if (options.debug) console.log('service-event', name, action, channelName)
-                              const connectionList = Object.values(connections).filter(cnx => cnx.channelNames.has(channelName))
-                              for (const connection of connectionList) {
-                                 if (options.debug) console.log('emit to', connection.id, name, action, result)
-                                 connection.socket.emit('service-event', {
-                                    name,
-                                    action,
-                                    result,
-                                 })
-                              }
-                           }
-                        }
+                        publish(service, action, result)
                      } catch(err) {
                         console.log('callErr', err)
                         io.emit('client-response', {
@@ -341,6 +329,28 @@ function expressX(app, options={}) {
       })      
    }
 
+   // publish event on associated channels
+   async function publish(service, action, result) {
+      console.log('PUB!')
+      const publishFunc = service.publishCallback
+      if (publishFunc) {
+         const channelNames = await publishFunc(result, app)
+         if (options.debug) console.log('publish channels', service.name, action, channelNames)
+         for (const channelName of channelNames) {
+            if (options.debug) console.log('service-event', service.name, action, channelName)
+            const connectionList = Object.values(connections).filter(cnx => cnx.channelNames.has(channelName))
+            for (const connection of connectionList) {
+               if (options.debug) console.log('emit to', connection.id, service.name, action, result)
+               connection.socket.emit('service-event', {
+                  name: service.name,
+                  action,
+                  result,
+               })
+            }
+         }
+      }
+   }
+
    function joinChannel(channelName, connection) {
       connection.channelNames.add(channelName)
    }

package/test/index.test.js

@@ -0,0 +1,110 @@
+
+import expressX from '../src/index.mjs'
+import express from 'express'
+import bodyParser from 'body-parser'
+import axios from 'axios'
+
+import { expect, assert } from 'chai'
+
+
+// `app` is a regular express application, enhanced with express-x features
+const app = expressX(express(), { debug: false })
+
+app.createDatabaseService('User')
+app.createDatabaseService('Post')
+
+
+
+describe('ExpressX API', () => {
+
+   it("can delete all users", async () => {
+      const res = await app.service('User').deleteMany()
+      assert(res.count === 1)
+   })
+
+   it("can create a user", async () => {
+      const user = await app.service('User').create({
+         data: {
+            name: "chris",
+            email: 'chris@mail.fr'
+         },
+      })
+      assert(user.name === 'chris')
+   })
+
+   it("can find a user by name", async () => {
+      const users = await app.service('User').findMany({
+         where: {
+            name: {
+               startsWith: "ch"
+            }
+         }
+      })
+      assert(users.length > 0)
+   })
+
+   it("can find a unique user by email", async () => {
+      const chris = await app.service('User').findUnique({
+         where: {
+            email: "chris@mail.fr"
+         }
+      })
+      assert(chris.name === 'chris')
+   })
+})
+
+
+describe('HTTP/REST API', () => {
+
+   // add body parsers for http requests
+   app.use(bodyParser.json())
+   app.use(bodyParser.urlencoded({ extended: false }))
+
+   // add http/rest endpoints
+   app.addHttpRest('/api/user', app.service('User'))
+   app.addHttpRest('/api/post', app.service('Post'))
+
+   app.server.listen(8008, () => console.log(`App listening at http://localhost:8008`))
+
+   let chris
+
+   it("can create a user", async () => {
+      const res = await axios.post('http://localhost:8008/api/user', {
+         name: "carole",
+         email: 'carole@mail.fr'
+      })
+      assert(res?.data?.name === 'carole')
+   })
+
+   it("can find users", async () => {
+      const res = await axios.get('http://localhost:8008/api/user')
+      assert(res?.data?.length > 0)
+   })
+
+   it("can find a user by name", async () => {
+      const res = await axios.get('http://localhost:8008/api/user?name=chris')
+      assert(res?.data?.length > 0)
+      chris = res.data[0]
+   })
+
+   it("can find a user by id", async () => {
+      const res = await axios.get(`http://localhost:8008/api/user/${chris.id}`)
+      assert(res?.data?.id === chris.id)
+   })
+
+   it("can patch a user", async () => {
+      const res = await axios.patch(`http://localhost:8008/api/user/${chris.id}`, {
+         name: "Christophe",
+      })
+      assert(res?.data?.name === "Christophe")
+   })
+
+   it("can delete a user", async () => {
+      const res = await axios.delete(`http://localhost:8008/api/user/${chris.id}`)
+      assert(res?.data?.name === "Christophe")
+   })
+
+   it("can stop server", () => {
+      app.server.close()
+   })
+})