scimgateway 5.3.7 → 5.4.0

package/README.md

@@ -16,12 +16,10 @@ Validated through IdP's:
 
 Latest news:  
  
-- [ETag](https://datatracker.ietf.org/doc/html/rfc7644#section-3.14) now supported
-- [Bulk Operations](https://datatracker.ietf.org/doc/html/rfc7644#section-3.7) now supported
-- Remote real-time log subscription for monitoring and centralized logging  
-using browser and url: `https://<host>/logger`  
-`curl -N https://<host>/logger -u user:password`  
-custom client API, see configuration notes  
+- [Azure Relay](https://learn.microsoft.com/en-us/azure/azure-relay/relay-what-is-it) is now supported for secure and hassle-free outbound communication — with just one minute of configuration
+- [ETag](https://datatracker.ietf.org/doc/html/rfc7644#section-3.14) is now supported
+- [Bulk Operations](https://datatracker.ietf.org/doc/html/rfc7644#section-3.7) is now supported
+- Remote real-time log subscription for centralized logging and monitoring. Using browser `https://<host>/logger`, curl or custom client API - see configuration notes  
 - By configuring the chainingBaseUrl, it is now possible to chain multiple gateways in sequence, such as `gateway1->gateway2->gateway3->endpoint`. In this setup, gateway beave much like a reverse proxy, validating authorization at each step unless PassThrough mode is enabled. Chaining is also supported in stream subscriber mode
 - Email, onError and sendMail() supports more secure RESTful OAuth for Microsoft Exchange Online (ExO) and Google Workspace Gmail, alongside traditional SMTP Auth for all mail systems. HelperRest supports a wide range of common authentication methods, including basicAuth, bearerAuth, tokenAuth, oauth, oauthSamlBearer, oauthJwtBearer and Auth PassTrough 
 - Major version **v5.0.0** marks a shift from JavaScript to native TypeScript and prioritizes [Bun](https://bun.sh/) over Node.js. This upgrade requires some modifications to existing plugins.  
@@ -42,7 +40,7 @@ custom client API, see configuration notes
 
 ## Overview  
 
-SCIM Gateway facilitates user management using the standardized REST-based SCIM 1.1 or 2.0 protocol, offering easier, more powerful, and consistent provisioning while avoiding vendor lock-in. Acting as a translator for incoming SCIM requests, the gateway seamlessly enables CRUD functionality (create, read, update, and delete) for users and groups. By implementing endpoint-specific protocols, it ensures precise and efficient provisioning across diverse destinations. With the gateway, your diverse destinations effectively become SCIM endpoints, streamlining integration and simplifying user management.  
+SCIM Gateway facilitates user management using the standardized REST-based SCIM 1.1 or 2.0 protocol, offering easier, more powerful, and consistent provisioning while avoiding vendor lock-in. Acting as a translator for incoming SCIM requests, the gateway seamlessly enables CRUD functionality (create, read, update, and delete) for users and groups. By implementing endpoint-specific protocols, it ensures provisioning across diverse destinations. With the gateway, your destinations effectively become SCIM endpoints, streamlining integration and simplifying user management.  
 
 
 ![](https://jelhub.github.io/images/ScimGateway.svg)
@@ -129,7 +127,7 @@ If internet connection is blocked, we could install on another machine and copy
 
 	bun c:\my-scimgateway
 	
-	Start a browser (note, Edge do not pop-up logon dialog box when using http)
+	Start a browser
 
 	http://localhost:8880/ping
 	=> Health check with a "hello" response
@@ -138,7 +136,7 @@ If internet connection is blocked, we could install on another machine and copy
 	http://localhost:8880/Groups
 	=> Logon using gwadmin/password and two users and groups should be listed  
 
-	Start a new browser for log monitoring (info level)
+	Start a new browser for remote log monitoring
 	using url: http://localhost:8880/logger
 
 	http://localhost:8880/Users/bjensen
@@ -160,7 +158,7 @@ If internet connection is blocked, we could install on another machine and copy
 >Tip, take a look at bun test scripts located in `node_modules\scimgateway\test\lib`
 
 > If using Node.js instead of Bun, scimgateway must be downloaded from github because Node.js does not support native typescript used by modules. Startup will then be:  
-`node --experimental-strip-types c:\my-scimgateway\index.ts`
+`node --experimental-strip-types c:\scimgateway\index.ts`
 
 #### Upgrade SCIM Gateway  
 
@@ -396,7 +394,7 @@ Definitions in `endpoint` object are customized according to our plugin code. Pl
 
 - **log.loglevel.console** - off, debug, info, warn or error. Default off. Output to stdout and errors to stderr
 
-- **log.loglevel.push** - off, debug, info, warn or error. Default info. Push to stream that can be used by client subscriber
+- **log.loglevel.push** - off, debug, info, warn or error. Default info. Push to stream used by remote real-time log subscription
 
 - **log.logDirectory** - custom defined log directory e.g. `/var/log/scimgateway` that will override default `<scimgateway path>/logs`. If not exist it will be created.
 
@@ -435,7 +433,7 @@ Definitions in `endpoint` object are customized according to our plugin code. Pl
   
 	Example of how to make a self signed certificate:  
 
-		openssl.exe req -nodes -newkey rsa:2048 -x509 -sha256 -days 3650 -keyout key.pem -out cert.pem -subj "/O=My Company/OU=Application/CN=SCIM Gateway" -addext "subjectAltName=DNS:localhost,DNS:127.0.0.1,DNS:*.mycompany.com" -addext "extendedKeyUsage=serverAuth" -addext "keyUsage=digitalSignature"
+		openssl req -nodes -newkey rsa:2048 -x509 -sha256 -days 3650 -keyout key.pem -out cert.pem -subj "/O=My Company/OU=Application/CN=SCIM Gateway" -addext "subjectAltName=DNS:localhost,DNS:127.0.0.1,DNS:*.mycompany.com" -addext "extendedKeyUsage=serverAuth" -addext "keyUsage=digitalSignature"
   
     Note, when using Symantec/Broadcom Provisioning, the "certificate authority - CA" also must be imported on the Connector Server. For self-signed certificate, CA and the certificate (public key) is the same.  
 
@@ -475,12 +473,18 @@ Definitions in `endpoint` object are customized according to our plugin code. Pl
 - **email.proxy.password** - password if authentication is required
 - **email.emailOnError** - Contains configuration for sending error notifications by email. Note, only the first error will be sent until sendInterval have passed
 - **email.emailOnError.enabled** - true or false, value set to true will enable email notifications
-- **email.emailOnError.sendInterval** - Default 15. Mail notifications on error are deferred until sendInterval **minutes** have passed since the last notification.
+- **email.emailOnError.sendInterval** - Default 15. Mail notifications on error are deferred until sendInterval **minutes** have passed since the last notification
 - **email.emailOnError.from** - Sender email addresses e.g: "noreply@example.com". **Mandatory for oauth**. For smtp email.auth.options.username will be used
 - **email.emailOnError.to** - Comma separated list of recipients email addresses e.g: "someone@example.com"
 - **email.emailOnError.cc** - Optional comma separated list of cc mail addresses
 - **email.emailOnError.subject** - Optional mail subject, default `SCIM Gateway error message`
 
+- **azureRelay** - Azure Relay outbound listener 
+- **azureRelay.enabled** - true or false, true will enable the Azure Relay listener
+- **azureRelay.connectionUrl** - `https://<namespace-name>.servicebus.windows.net/<hybrid-connection-name>` - `<namespace-name>` is the name of the Relay created and `<hybrid-connection-name>` is the name of the Hybrid Connection entity created in the Relay
+- **azureRelay.apiKey** - The `Private Key` found in the `Shared access policy` (RootManageSharedaccessKey)
+- **azureRelay.keyRule** - Optional, the `Shared access policy` name - default using `RootManageSharedaccessKey`
+
 - **stream** - See [SCIM Stream](https://elshaug.xyz/docs/scim-stream) for configuration details
 
 - **endpoint** - Contains endpoint specific configuration according to customized **plugin code**. 
@@ -743,9 +747,11 @@ Please see code editor method HelperRest doRequest() IntelliSense for type and o
 Using remote real-time log subscription we may implement custom logic like monitoring and centralized logging
 
 - using browser and url: https://host/logger  
-- curl -N https://host/logger -u gwread:password  
-- curl -N https://host/logger -H "Authorization: Bearer secret"  
-- custom client API
+- curl -Ns https://host/logger -u gwread:password | sed 's/\xE2\x80\x8B//g'  
+- curl -Ns https://host/logger -H "Authorization: Bearer secret" | sed 's/\xE2\x80\x8B//g'  
+	(-s and sed to ignore keep-alive character)
+- custom client API (see example below)
+- not supported by Azure Relay
 
 We may configure read-only user/secret for log collection purpose    
 
@@ -774,8 +780,8 @@ We may configure read-only user/secret for log collection purpose
 	  ...
 	}
 
-push logger using default `info` log level  
-push log level may be customized by configuration  
+Remote log subscription is configured by log.loglevel.push and the push logger has default loglevel set to `info` 
+Example using debug loglevel:
 
 	"log": {
 	  "loglevel": {
@@ -783,57 +789,118 @@ push log level may be customized by configuration
 	  }
 	}
 
-Example code implementing subscriber for real-time log messages collection  
+Example code implementing remote real-time log subscription and custom message handling  
+
+```
+// startup: bun <scriptname.ts>
+// update url (ws or wss) and the auth according to environment used
+const url = 'ws://localhost:8880/logger'
+const auth = 'Basic ' + btoa('gwadmin' + ':' + 'password') // const auth = 'Bearer ' + 'secret'
+
+const tls: any = {}
+if (url.startsWith('wss:')) {
+    tls.ca = [Bun.file('/path/to/self-signed-cert.pem')], // only needed for self-signed certs
+    tls.rejectUnauthorized = false
+}
+
+// messageHandler implements message handling and custom logic
+// could also use JSON.parse(message) and granular filtering on log "level"
+const messageHandler = async (message: string) => {
+    console.log(message)
+}
+
+const startWebSocket = async () => {
+    try {
+        const ws = new WebSocket(url, {
+            headers: {
+                Authorization: auth,
+            },
+            tls,
+        })
+
+        // message is received
+        ws.addEventListener("message", event => {
+            messageHandler(event.data)
+        });
+
+        // socket opened
+        ws.addEventListener("open", event => {
+            console.log('✅ Now awaiting log events...\n')
+        });
+
+        // socket closed
+        ws.addEventListener("close", event => {
+            let addInfo = ''
+            if (event.code === 1002) addInfo = ' => most likely authentication failure?'
+            console.warn(`⚠️ Connection closed (${event.code}): ${event.reason || 'no reason'}${addInfo}`)
+            retry()
+        });
+
+        // error handler
+        ws.addEventListener("error", event => {
+            // console.error('❌ WebSocket error:', event.message)
+        });
+
+    } catch (err: any) {
+        console.error('❌ Unexpected error:', err)
+    }
+}
+
+const retry = async () => {
+    console.log('🔁 Retry in 10 seconds...')
+    await Bun.sleep(10 * 1000)
+    startWebSocket()
+}
+
+startWebSocket()
+```
+
+### Configuration notes - Azure Relay
+
+Using Azure technology we have different options for setting up a communication tunnel to SCIM Gateway:  
+
+- `Microsoft Entra Application Proxy + Microsoft Entra Application Proxy Connector` (SCIM Gateway located on-premises or using Azure private VNet/IP)
+- `Azure Application Gateway` - Layer 7 (SCIM Gateway located in Azure)
+- `Azure Relay` (SCIM Gateway located on-premises or in Azure)
+
+SCIM Gateway have builtin [Azure Relay](https://learn.microsoft.com/en-us/azure/azure-relay/relay-what-is-it) support which gives secure and hassle-free outbound communication — with just one minute of configuration
+
+Azure pricing for using Azure Relay is approx. 10$ per month for each listener (SCIM Gateway plugin)
+
+**Using out-of-the-box Azure Relay:**
+
+- Prerequisite: SCIM Gateway having outbound internet access (https/443)
+- In Azure create a `Relay` - `<namespace-name>`
+- In the Relay, create an entity of type `Hybrid Connection` - `<hybrid-connection-name>` **one for each SCIM Gateway plugin**
+- The `Requires Client Authorization` option **should be unchecked (not activated)**, unless we are using custom IdP/API having logic for including SAS-token in the communication header
+- Shared access policies - RootManageSharedaccessKey - Primary Key (copy this one)  
+	Instead of RootManageSharedaccessKey policy in the `<namespace-name>`, we could create dedicated policy in the sub level `<hybrid-connection-name>` and use this policy name in plugin configuration `scimgateway.azureRelay.keyRule`
+
+SCIM Gateway plugin configuration:
+
+```
+{
+  "scimgateway: {
+    ...
+    "azureRelay": {
+      "enabled": true,
+      "connectionUrl": "https://<namespace-name>.servicebus.windows.net/<hybrid-connection-name>",
+      "apiKey": "<primary-key>"
+    },
+    ...
+  },
+  ...
+}
+````
+
+`connectionUrl` will be the SCIM base URL used by IdP/API for accessing SCIM Gateway
+
+Example:  
+GET `https://<namespace-name>.servicebus.windows.net/<hybrid-connection-name>/Users`  
+GET `https://<namespace-name>.servicebus.windows.net/<hybrid-connection-name>/<baseEntity>/Users`
+
+If several SCIM Gateway´s (same plugin) connect listeners using the same Azure Relay connectionUrl, there will be load-balancing and round-robin distribution
 
-	let headers = new Headers()
-	headers.append('Authorization', 'Basic ' + btoa('gwadmin' + ':' + 'password'))
-	
-	// message handling and custom logic
-	// we could also do JSON.parse(message) and granular filtering on log "level"
-	const messageHandler = async (message: string) => {
-	  console.log(message)
-	}
-	
-	let ignoreCatch = false
-	do { // retry loop when connection closed or service unavailable
-	  if (ignoreCatch) ignoreCatch = false
-	
-	  try {
-	    const resp = await fetch("http://localhost:8880/logger", {
-	      method: "GET",
-	      headers: headers,
-	    })
-	
-	    const reader = resp.body.pipeThrough(new TextDecoderStream()).getReader()
-	    console.log('Now awaiting log events...\n')
-	
-	    while (true) {
-	      const { value, done } = await reader.read()
-	      if (done) break
-	      if (value.at(-1) !== '\n') continue
-	      const message = value.slice(0, -1)
-	      messageHandler(message)
-	    }
-	
-	    // shouldn't be here... authentication failure?
-	    const e = {
- 	     url: resp.url,
-	      status: resp.status,
-	      statusText: resp.statusText
-	    }
-	    console.error('error', e)
-	
-	  } catch (err: any) {
-	    if (['ConnectionClosed', 'ConnectionRefused', 'ECONNRESET'].includes(err.code)) {
-	      console.log('Connection closed or service unavailable')
-	      ignoreCatch = true
-	      await Bun.sleep(10 * 1000)
-	    } else console.error(err)
-	  }
-	
-	} while (ignoreCatch)
-	
-	console.log('\n\ndone!')
 
 ## Manual startup    
 
@@ -1406,7 +1473,56 @@ MIT © [Jarle Elshaug](https://www.elshaug.xyz)
 
 ## Change log  
 
-## Change log  
+### v5.4.0
+
+[Improved]
+
+- Remote real-time log subscription now prioritize using WebSocket over SSE. Using browser will show loglevel colors. If running Node.js, WebSocket is not supported and SSE will be used. Remote logger is not supported by Azure Relay.
+
+### v5.3.8
+
+[Improved]
+
+- [Azure Relay](https://learn.microsoft.com/en-us/azure/azure-relay/relay-what-is-it) is now supported for secure and hassle-free outbound communication — with just one minute of configuration
+
+	Using Azure technology we have different options for setting up a communication tunnel to SCIM Gateway:  
+
+	`Microsoft Entra Application Proxy + Microsoft Entra Application Proxy Connector` (SCIM Gateway located on-premises or using Azure private VNet/IP)  
+	`Azure Application Gateway` - Layer 7 (SCIM Gateway located in Azure)  
+	`Azure Relay` (SCIM Gateway located on-premises or in Azure)  
+   
+	Azure pricing for using Azure Relay is approx. 10$ per month for each listener (SCIM Gateway plugin)
+
+    **Using out-of-the-box Azure Relay:**
+
+	Prerequisite: SCIM Gateway having outbound internet access (https/443)  
+	In Azure create a `Relay` - `<namespace-name>`  
+	In the Relay, create an entity of type `Hybrid Connection` - `<hybrid-connection-name>` **one for each SCIM Gateway plugin**  
+	The `Requires Client Authorization` option **should be unchecked (not activated)**, unless we are using custom IdP/API having logic for including SAS-token in the communication header  
+	Shared access policies - RootManageSharedaccessKey - Primary Key (copy this one)  
+
+	SCIM Gateway plugin configuration:
+
+		{
+		  "scimgateway: {
+		    ...
+		    "azureRelay": {
+		      "enabled": true,
+		      "connectionUrl": "https://<namespace-name>.servicebus.windows.net/<hybrid-connection-name>",
+		      "apiKey": "<primary-key>"
+		    },
+		    ...
+		  },
+		  ...
+		}
+
+	`connectionUrl` will be the SCIM base URL used by IdP/API for accessing SCIM Gateway
+
+	Example:  
+	GET `https://<namespace-name>.servicebus.windows.net/<hybrid-connection-name>/Users`  
+	GET `https://<namespace-name>.servicebus.windows.net/<hybrid-connection-name>/<baseEntity>/Users`
+
+	If several SCIM Gateway´s (same plugin) connect listeners using the same Azure Relay connectionUrl, there will be load-balancing and round-robin distribution
 
 ### v5.3.7