scimgateway 4.4.6 → 4.5.0

package/README.md

@@ -16,6 +16,7 @@ Validated through IdP's:
   
 Latest news:  
 
+- Supports stream publishing mode having [SCIM Stream](https://elshaug.xyz/docs/scim-stream) as a prerequisite. In this mode, standard incoming SCIM requests from your Identity Provider (IdP) or API are directed and published to the stream. Subsequently, one of the gateways subscribing to the channel utilized by the publisher will manage the SCIM request, and response back to the publisher. Using SCIM Stream we have `egress/outbound only traffic` and get loadbalancing/failover by adding more gateways subscribing to same channel.
 - **BREAKING**: [SCIM Stream](https://elshaug.xyz/docs/scim-stream) is the modern way of user provisioning letting clients subscribe to messages instead of traditional IGA top-down provisioning. SCIM Gateway now offers enhanced functionality with support for message subscription and automated provisioning using SCIM Stream
 - Authentication PassThrough letting plugin pass authentication directly to endpoint for avoid maintaining secrets at the gateway. Kubernetes health checks and shutdown handler support
 - Supports OAuth Client Credentials authentication
@@ -33,7 +34,9 @@ Latest news:
 
 ## Overview  
 
-With SCIM Gateway, user management is facilitated through the utilization of the REST-based SCIM 1.1 or 2.0 protocol. The Gateway acts as a translator for incoming SCIM requests, seamlessly enabling the exposure of CRUD functionality (create, read, update, and delete user/group) towards destinations. This is achieved through the implementation of endpoint-specific protocols, ensuring precise and efficient provisioning with diverse endpoints.
+With SCIM Gateway, user management is facilitated through the utilization of the REST-based SCIM 1.1 or 2.0 protocol. The gateway acts as a translator for incoming SCIM requests, seamlessly enabling the exposure of CRUD functionality (create, read, update, and delete user/group) towards destinations. This is achieved through the implementation of endpoint-specific protocols, ensuring precise and efficient provisioning with diverse endpoints.  
+
+Using [SCIM Stream](https://elshaug.xyz/docs/scim-stream), gateway may enable Pub/Sub allowing incoming SCIM requests to be published and processed by other gateways acting as subscribers. This extends beyond being Entra ID and HR subscriber for messages published by the SCIM Stream collector.
 
 ![](https://jelhub.github.io/images/ScimGateway.svg)
 
@@ -54,10 +57,10 @@ Same as plugin "Loki", but using external MongoDB
 Shows how to implement a highly configurable multi tenant or multi endpoint solution through `baseEntity` in URL
 
 * **SCIM** (REST Webservice)  
-Demonstrates user provisioning towards REST-Based endpoint (type SCIM) 
-Using plugin "Loki" as SCIM endpoint  
+Demonstrates user provisioning towards REST-Based endpoint (type SCIM)  
+Using plugin Loki as SCIM endpoint  
 Can be used as SCIM version-gateway e.g. 1.1=>2.0 or 2.0=>1.1  
-Can be used to chain several SCIM Gateway's  
+Can be used to chain several gateways  
 
 
 * **Soap** (SOAP Webservice)  
@@ -86,9 +89,9 @@ Using endpointMapper (like plugin-entra-id) for attribute flexibility
 * **API** (REST Webservices)  
 Demonstrates API Gateway/plugin functionality using post/put/patch/get/delete  
 None SCIM plugin, becomes what you want it to become.  
-Methods listed can also be used in standard SCIM plugins  
+Methods included can also be used in standard SCIM plugins  
 Endpoint complexity could be put in this plugin, and client could instead communicate through Gateway using your own simplified REST specification.  
-One example of usage could be creation of tickets in ServiceDesk/HelpDesk and also the other way, closing a ticket could automatically approve/reject corresponding workflow in Identity Manager.  
+One example of usage could be creation of tickets in ServiceDesk and also the other way, closing a ticket could automatically approve/reject corresponding workflow in IdP.  
 
     
 ## Installation  
@@ -384,13 +387,13 @@ Definitions in `endpoint` object are customized according to our plugin code. Pl
 
 - **auth.bearerJwtAzure** - Array of one or more JWT used by Azure SyncFabric. **tenantIdGUID** must be set to Entra ID Tenant ID.  
 
-- **auth.bearerJwt** - Array of one or more standard JWT objects. Using **secret** or **publicKey** for signature verification. publicKey should be set to the filename of public key or certificate pem-file located in `<package-root>\config\certs`. Clear text secret will become encrypted when gateway is started. **options.issuer** is mandatory. Other options may also be included according to jsonwebtoken npm package definition.   
+- **auth.bearerJwt** - Array of one or more standard JWT objects. Using **secret** or **publicKey** for signature verification. publicKey should be set to the filename of public key or certificate pem-file located in `<package-root>\config\certs` or absolute path being used. Clear text secret will become encrypted when gateway is started. **options.issuer** is mandatory. Other options may also be included according to jsonwebtoken npm package definition.   
 
 - **auth.bearerOAuth** - Array of one or more Client Credentials OAuth configuration objects. **`client_id`** and **`client_secret`** are mandatory. client_secret value will become encrypted when gateway is started. OAuth token request url is **/oauth/token** e.g. http://localhost:8880/oauth/token  
 
 - **auth.passThrough** - Setting **auth.passThrough.enabled=true** will bypass SCIM Gateway authentication. Gateway will instead pass ctx containing authentication header to the plugin. Plugin could then use this information for endpoint authentication and we don't have any password/token stored at the gateway. Note, this also requires plugin binary having `scimgateway.authPassThroughAllowed = true` and endpoint logic for handling/passing ctx.request.header.authorization 
 
-- **certificate** - If not using TLS certificate, set "key", "cert" and "ca" to **null**. When using TLS, "key" and "cert" have to be defined with the filename corresponding to the primary-key and public-certificate. Both files must be located in the `<package-root>\config\certs` directory e.g:  
+- **certificate** - If not using TLS certificate, set "key", "cert" and "ca" to **null**. When using TLS, "key" and "cert" have to be defined with the filename corresponding to the primary-key and public-certificate. Both files must be located in the `<package-root>\config\certs` directory unless absolute path being defined e.g:  
   
 		"certificate": {
 		  "key": "key.pem",
@@ -1147,6 +1150,16 @@ MIT © [Jarle Elshaug](https://www.elshaug.xyz)
 
 ## Change log  
 
+### v4.5.0
+
+[Improved]  
+
+- scim-stream, scimgateway now supports stream publishing mode having [SCIM Stream](https://elshaug.xyz/docs/scim-stream) as a prerequisite. In this mode, standard incoming SCIM requests from your Identity Provider (IdP) or API are directed and published to the stream. Subsequently, one of the gateways subscribing to the channel utilized by the publisher will manage the SCIM request, and response back to the publisher. Using SCIM Stream we have `egress/outbound only traffic` and get loadbalancing/failover by adding more gateways subscribing to same channel.
+- scim-stream, subscriber will do automatic retry until connected when plugin not able to connect to endpoint (offline endpoint)
+- plugin-ldap, modifyGroup now supports all attributes and not only add/remove members
+- certificate absolute path may be used in plugin configuration file instead of default relative path 
+- dependencies bump
+
 ### v4.4.6
 
 [Improved]  

package/config/plugin-api.json

@@ -116,6 +116,19 @@
             "replaceDomains": []
           }
         }
+      },
+      "publisher": {
+        "enabled": false,
+        "entity": {
+          "undefined": {
+            "nats": {
+              "tenant": null,
+              "subject": null,
+              "jwt": null,
+              "secret": null
+            }
+          }
+        }
       }
     }
   },

package/config/plugin-entra-id.json

@@ -116,6 +116,19 @@
             "replaceDomains": []
           }
         }
+      },
+      "publisher": {
+        "enabled": false,
+        "entity": {
+          "undefined": {
+            "nats": {
+              "tenant": null,
+              "subject": null,
+              "jwt": null,
+              "secret": null
+            }
+          }
+        }
       }
     }
   },

package/config/plugin-ldap.json

@@ -116,6 +116,19 @@
             "replaceDomains": []
           }
         }
+      },
+      "publisher": {
+        "enabled": false,
+        "entity": {
+          "undefined": {
+            "nats": {
+              "tenant": null,
+              "subject": null,
+              "jwt": null,
+              "secret": null
+            }
+          }
+        }
       }
     }
   },

package/config/plugin-loki.json

@@ -116,6 +116,19 @@
             "replaceDomains": []
           }
         }
+      },
+      "publisher": {
+        "enabled": false,
+        "entity": {
+          "undefined": {
+            "nats": {
+              "tenant": null,
+              "subject": null,
+              "jwt": null,
+              "secret": null
+            }
+          }
+        }
       }
     }
   },

package/config/plugin-mongodb.json

@@ -122,6 +122,19 @@
             "replaceDomains": []
           }
         }
+      },
+      "publisher": {
+        "enabled": false,
+        "entity": {
+          "undefined": {
+            "nats": {
+              "tenant": null,
+              "subject": null,
+              "jwt": null,
+              "secret": null
+            }
+          }
+        }
       }
     }
   },

package/config/plugin-mssql.json

@@ -116,6 +116,19 @@
             "replaceDomains": []
           }
         }
+      },
+      "publisher": {
+        "enabled": false,
+        "entity": {
+          "undefined": {
+            "nats": {
+              "tenant": null,
+              "subject": null,
+              "jwt": null,
+              "secret": null
+            }
+          }
+        }
       }
     }
   },

package/config/plugin-saphana.json

@@ -116,6 +116,19 @@
             "replaceDomains": []
           }
         }
+      },
+      "publisher": {
+        "enabled": false,
+        "entity": {
+          "undefined": {
+            "nats": {
+              "tenant": null,
+              "subject": null,
+              "jwt": null,
+              "secret": null
+            }
+          }
+        }
       }
     }
   },

package/config/plugin-scim.json

@@ -116,6 +116,19 @@
             "replaceDomains": []
           }
         }
+      },
+      "publisher": {
+        "enabled": false,
+        "entity": {
+          "undefined": {
+            "nats": {
+              "tenant": null,
+              "subject": null,
+              "jwt": null,
+              "secret": null
+            }
+          }
+        }
       }
     }
   },

package/config/plugin-soap.json

@@ -116,6 +116,19 @@
             "replaceDomains": []
           }
         }
+      },
+      "publisher": {
+        "enabled": false,
+        "entity": {
+          "undefined": {
+            "nats": {
+              "tenant": null,
+              "subject": null,
+              "jwt": null,
+              "secret": null
+            }
+          }
+        }
       }
     }
   },

package/lib/plugin-api.js

@@ -253,6 +253,7 @@ const getAccessToken = async (baseEntity, ctx) => {
       grant_type: 'client_credentials',
       client_id: username || config.entity[baseEntity].oauth.clientId,
       client_secret: secret || scimgateway.getPassword(`endpoint.entity.${baseEntity}.oauth.clientSecret`, configFile),
+      scope: config.entity[baseEntity].oauth.scope || '',
       resource: resource // "https://graph.microsoft.com"
     }
     options.headers = {

package/lib/plugin-entra-id.js

@@ -1069,6 +1069,7 @@ const getAccessToken = async (baseEntity, ctx) => {
       grant_type: 'client_credentials',
       client_id: username || config.entity[baseEntity].oauth.clientId,
       client_secret: secret || scimgateway.getPassword(`endpoint.entity.${baseEntity}.oauth.clientSecret`, configFile),
+      scope: config.entity[baseEntity].oauth.scope || '',
       resource: resource // "https://graph.microsoft.com"
     }
     options.headers = {

package/lib/plugin-ldap.js

@@ -289,8 +289,7 @@ scimgateway.createUser = async (baseEntity, userObj, ctx) => {
   if (!userBase) userBase = config.entity[baseEntity].ldap.userBase
 
   // convert SCIM attributes to endpoint attributes according to config.map
-  const [endpointObj] = scimgateway.endpointMapper('outbound', userObj, config.map.user)
-  // if (err) throw new Error(`${action} error: ${err.message}`)  // use above [endpointObj, err] to catch non supported attributes
+  const [endpointObj] = scimgateway.endpointMapper('outbound', userObj, config.map.user) // use [endpointObj, err] and if err, throw error to catch non supported attributes
 
   // endpoint spesific attribute handling
   if (endpointObj.sAMAccountName !== undefined) { // Active Directory
@@ -372,18 +371,16 @@ scimgateway.modifyUser = async (baseEntity, id, attrObj, ctx) => {
     delete attrObj.groups // make sure to be removed from attrObj
 
     const [groupsAttr] = scimgateway.endpointMapper('outbound', 'groups.value', config.map.user)
-    // if (err) throw new Error(`${action} error: ${err.message}`)  // use above [endpointObj, err] to catch non supported attributes
-
-    const body = { add: { }, remove: { } }
-    body.add[groupsAttr] = []
-    body.remove[groupsAttr] = []
+    const grp = { add: { }, remove: { } }
+    grp.add[groupsAttr] = []
+    grp.remove[groupsAttr] = []
 
     for (let i = 0; i < groups.length; i++) {
       const el = groups[i]
       if (el.operation && el.operation === 'delete') { // delete from users group attribute
-        body.remove[groupsAttr].push(el.value)
+        grp.remove[groupsAttr].push(el.value)
       } else { // add to users group attribute
-        body.add[groupsAttr].push(el.value)
+        grp.add[groupsAttr].push(el.value)
       }
     }
 
@@ -399,17 +396,17 @@ scimgateway.modifyUser = async (baseEntity, id, attrObj, ctx) => {
     } else base = id // dn
 
     try {
-      if (body.add[groupsAttr].length > 0) {
+      if (grp.add[groupsAttr].length > 0) {
         const ldapOptions = {
           operation: 'add',
-          modification: body.add
+          modification: grp.add
         }
         await doRequest(baseEntity, method, base, ldapOptions, ctx)
       }
-      if (body.remove[groupsAttr].length > 0) {
+      if (grp.remove[groupsAttr].length > 0) {
         const ldapOptions = {
           operation: 'delete',
-          modification: body.remove
+          modification: grp.remove
         }
         await doRequest(baseEntity, method, base, ldapOptions, ctx)
       }
@@ -418,11 +415,10 @@ scimgateway.modifyUser = async (baseEntity, id, attrObj, ctx) => {
     }
   }
 
-  if (JSON.stringify(attrObj) === '{}') return null // only groups included
+  if (Object.keys(attrObj).length < 1) return null // only groups included
 
   // convert SCIM attributes to endpoint attributes according to config.map
   const [endpointObj] = scimgateway.endpointMapper('outbound', attrObj, config.map.user)
-  // if (err) throw new Error(`${action} error: ${err.message}`)  // use above [endpointObj, err] to catch non supported attributes
 
   // endpoint spesific attribute handling
   if (endpointObj.userAccountControl !== undefined) { // SCIM "active" - Active Directory
@@ -527,7 +523,6 @@ scimgateway.getGroups = async (baseEntity, getObj, attributes, ctx) => {
   }
 
   const [attrs] = scimgateway.endpointMapper('outbound', attributes, config.map.group) // SCIM/CustomSCIM => endpoint attribute naming
-
   const method = 'search'
   const scope = 'sub'
   let base = config.entity[baseEntity].ldap.groupBase
@@ -644,8 +639,7 @@ scimgateway.createGroup = async (baseEntity, groupObj, ctx) => {
   if (!config.map.group) throw new Error(`${action} error: missing configuration endpoint.map.group`)
 
   // convert SCIM attributes to endpoint attributes according to config.map
-  const [endpointObj, err] = scimgateway.endpointMapper('outbound', groupObj, config.map.group)
-  if (err) throw new Error(`${action} error: ${err.message}`)
+  const [endpointObj] = scimgateway.endpointMapper('outbound', groupObj, config.map.group)
 
   // endpointObj.objectClass is mandatory and must must match your ldap schema
   endpointObj.objectClass = config.entity[baseEntity].ldap.groupObjectClasses // Active Directory: ["group"]
@@ -700,21 +694,18 @@ scimgateway.modifyGroup = async (baseEntity, id, attrObj, ctx) => {
   scimgateway.logger.debug(`${pluginName}[${baseEntity}] handling "${action}" id=${id} attrObj=${JSON.stringify(attrObj)}`)
 
   if (!config.map.group) throw new Error(`${action} error: missing configuration endpoint.map.group`)
-  if (!attrObj.members) {
-    throw new Error(`${action} error: only supports modification of members`)
-  }
-  if (!Array.isArray(attrObj.members)) {
+  if (attrObj.members && !Array.isArray(attrObj.members)) {
     throw new Error(`${action} error: ${JSON.stringify(attrObj)} - correct syntax is { "members": [...] }`)
   }
 
-  const [memberAttr, err1] = scimgateway.endpointMapper('outbound', 'members.value', config.map.group)
-  if (err1) throw new Error(`${action} error: ${err1.message}`)
+  const [memberAttr] = scimgateway.endpointMapper('outbound', 'members.value', config.map.group)
+  if (!memberAttr && attrObj.members) throw new Error(`${action} error: missing attribute mapping configuration for group members`)
 
-  const body = { add: { }, remove: { } }
-  body.add[memberAttr] = []
-  body.remove[memberAttr] = []
+  const grp = { add: { }, remove: { } }
+  grp.add[memberAttr] = []
+  grp.remove[memberAttr] = []
 
-  for (let i = 0; i < attrObj.members.length; i++) {
+  for (let i = 0; i < attrObj?.members?.length; i++) {
     const el = attrObj.members[i]
     if (config.useSID_id || config.useGUID_id) {
       const dn = await sidGuidToDn(baseEntity, el.value, ctx)
@@ -722,9 +713,9 @@ scimgateway.modifyGroup = async (baseEntity, id, attrObj, ctx) => {
       el.value = dn
     }
     if (el.operation && el.operation === 'delete') { // delete member from group
-      body.remove[memberAttr].push(el.value) // endpointMapper returns URI encoded id because some IdP's don't encode id used in GET url e.g. Symantec/Broadcom/CA
+      grp.remove[memberAttr].push(el.value) // endpointMapper returns URI encoded id because some IdP's don't encode id used in GET url e.g. Symantec/Broadcom/CA
     } else { // add member to group
-      body.add[memberAttr].push(el.value)
+      grp.add[memberAttr].push(el.value)
     }
   }
 
@@ -735,17 +726,26 @@ scimgateway.modifyGroup = async (baseEntity, id, attrObj, ctx) => {
   else base = id // dn
 
   try {
-    if (body.add[memberAttr].length > 0) {
+    delete attrObj.members
+    const [endpointObj] = scimgateway.endpointMapper('outbound', attrObj, config.map.group)
+    if (Object.keys(endpointObj).length > 0) {
+      const ldapOptions = {
+        operation: 'replace',
+        modification: endpointObj
+      }
+      await doRequest(baseEntity, method, base, ldapOptions, ctx)
+    }
+    if (grp.add[memberAttr].length > 0) {
       const ldapOptions = { // using ldap lookup (dn) instead of search
         operation: 'add',
-        modification: body.add
+        modification: grp.add
       }
       await doRequest(baseEntity, method, base, ldapOptions, ctx)
     }
-    if (body.remove[memberAttr].length > 0) {
+    if (grp.remove[memberAttr].length > 0) {
       const ldapOptions = { // using ldap lookup (dn) instead of search
         operation: 'delete',
-        modification: body.remove
+        modification: grp.remove
       }
       await doRequest(baseEntity, method, base, ldapOptions, ctx)
     }

package/lib/plugin-loki.js

@@ -51,7 +51,7 @@ const validFilterOperators = ['eq', 'ne', 'aeq', 'dteq', 'gt', 'gte', 'lt', 'lte
 
 const dbNames = []
 for (const baseEntity in config.entity) {
-  let dbname = (config.entity[baseEntity].dbname ? config.entity[baseEntity].dbname : 'loki.db')
+  let dbname = config.entity[baseEntity].dbname || 'loki.db'
   if (dbNames.includes(dbname)) {
     scimgateway.logger.error(`${pluginName}[${baseEntity}] initialization error: database '${dbname}' is already used by another baseEntity configuration`)
     continue

package/lib/plugin-scim.js

@@ -587,6 +587,7 @@ const getAccessToken = async (baseEntity, ctx) => {
       grant_type: 'client_credentials',
       client_id: username || config.entity[baseEntity].oauth.clientId,
       client_secret: secret || scimgateway.getPassword(`endpoint.entity.${baseEntity}.oauth.clientSecret`, configFile),
+      scope: config.entity[baseEntity].oauth.scope || '',
       resource: resource // "https://graph.microsoft.com"
     }
     options.headers = {