samanbayaka 0.0.20 → 0.0.22

package/README.md

@@ -14,8 +14,16 @@ This project is a modular, production-ready microservices framework built with M
 - [Installation](#installation)
 - [Prerequisites](#prerequisites)
 - [Documentations](#documentations)
+  - [Errors](#errors)
+  - [getConfig](#getConfig)
+  - [loadDemo](#loadDemo)
+  - [loadGatewayService](#loadGatewayService)
+  - [loadFeatureService](#loadFeatureService)
+  - [auxBrokerService](#auxBrokerService)
 - [Usage](#usage)
 - [Demo](#Demo)
+- [Dockers](#dockers)
+  - [Etcd](#etcd)
 
 ## Features
 
@@ -43,14 +51,14 @@ import sbk from "samanbayaka"
 ```
 ## Prerequisites
 It is assumed that NATS, Redpanda, Redis, and Etcd are installed and properly configured. All services should be discoverable through the /etc/hosts file.
-#### Minimum required Node.js version: >= 22.x.x
+### Minimum required Node.js version: >= 22.x.x
 ```bash
 node -v
 nvm use 22
 ```
 Additionally, OpenObserve is used to view logs and telemetry.
 
-#### Environment variables
+### Environment variables
 To configure all required environment variables, create a Bash file:
 ```bash
 source /etc/profile.d/sbk.sh
@@ -84,14 +92,14 @@ Optionally, you may verify the set environment variables.
 ```bash
 echo $SBK_PORT
 ```
-#### Configuration files
+### Configuration files
 Create a directory to store the configuration files, then copy all configuration files from the current project path into this directory and update them as needed. Before copying, ensure that your current working directory is the project directory.
 ```bash
 pwd
 sudo mkdir -p /usr/local/etc/<your_folder>
 sudo cp config/*.*  /usr/local/etc/<your_folder>
 ```
-#### Log files
+### Log files
 For production, it is recommended to create a new directory named "samanbayaka" under /var/log to store logs.
 ```bash
 sudo mkdir -p /var/log/samanbayaka
@@ -102,7 +110,7 @@ Empty your log files if they become very large, as follows:
 ```bash
 truncate -s 0 /var/log/samanbayaka/sbk-*.log
 ```
-#### Setup logrotate (optional)
+### Setup logrotate (optional)
 Create config file:
 ```bash
 sudo nano /etc/logrotate.d/sbk
@@ -123,25 +131,25 @@ Paste this:
 
 ## Documentations
 
-### sbk.Errors
+### Errors
 
 `sbk.Errors` exposes the Moleculer `Errors` object.
 
-### sbk.getConfig
+### getConfig
 
 `sbk.getConfig` is a function that returns a configuration object from a specified file name.
 
 The file is loaded from the configuration directory defined by the environment variable.
 
-### sbk.loadDemo
+### loadDemo
 
 `sbk.loadDemo` is an asynchronous function that creates a demo service to help understand the project.
 
-### sbk.loadGatewayService
+### loadGatewayService
 
 `sbk.loadGatewayService` is an asynchronous function that creates a gateway service to expose REST APIs.
 
-### sbk.loadFeatureService
+### loadFeatureService
 
 `sbk.loadFeatureService` is an asynchronous function that creates a feature service.
 
@@ -173,7 +181,7 @@ Example:
 ```
 
 
-### sbk.auxBrokerService
+### auxBrokerService
 
 `sbk.auxBrokerService` is a function used to initialize and manage message brokers such as Kafka, MQTT, and RabbitMQ.
 
@@ -253,7 +261,7 @@ Example:
 ```
 
 ## Usage
-#### Create gateway service
+### Create gateway service
 The Gateway is a critical service that exposes all endpoints as REST APIs. At least one Gateway service must be running to make the REST APIs accessible.
 ```bash
 mkdir gateway
@@ -269,7 +277,7 @@ import sbk from "samanbayaka"
 await sbk.loadGatewayService()
 ```
 ---
-#### Create feature services
+### Create feature services
 ```bash
 mkdir <your_service_name>
 cd <your_service_name>
@@ -328,7 +336,7 @@ await sbk.loadFeatureService({
 
 ```
 ---
-#### Create auxaliary broker services
+### Create auxaliary broker services
 ##### for producer
 ```bash
 mkdir <your_service_name>-<type>-<topic>
@@ -369,7 +377,7 @@ If you do not expose the service as a REST API and run it as a private/internal
 rest: false
 ```
 
-##### for consumer
+#### for consumer
 ```bash
 mkdir <your_service_name>-<type>-<topic>-<group>
 cd <your_service_name>-<type>-<topic>-<group>
@@ -407,7 +415,7 @@ The `interMessageDelayMs` option introduces a delay between two consecutive mess
 
 ---
 
-#### Run the services 
+### Run the services 
 
  * development/testing
 ```bash
@@ -443,6 +451,134 @@ Run the service, demo
 node demo.mjs
 ```
 
+## Dockers
+
+### Etcd
+To manage configurations centrally, use the docker-compose.yml file to run Etcd.
+```bash
+mkdir -p etcd
+cd etcd
+touch docker-compose.yml
+export ETCD_ROOT_PW=<your_root_pass>
+export ETCD_CONFIG_ADMIN_PW=<your_config_admin_pass>
+```
+
+Open `docker-compose.yml` and paste the following:
+```yml
+services:
+  etcd:
+    image: quay.io/coreos/etcd:v3.5.5
+    container_name: sbk-etcd
+
+    ports:
+      - "<your_api_port>:2379"
+      - "<your_cluster_port>:2380"
+
+    volumes:
+      - /usr/local/etc/samanbayaka:/etcd-data
+
+    command:
+      - /usr/local/bin/etcd
+      - --name=etcd
+      - --data-dir=/etcd-data
+      - --listen-client-urls=http://0.0.0.0:2379
+      - --advertise-client-urls=http://etcd:2379
+      - --listen-peer-urls=http://0.0.0.0:2380
+      - --initial-advertise-peer-urls=http://etcd:2380
+      - --initial-cluster=etcd=http://etcd:2380
+      - --initial-cluster-state=new
+
+    restart: unless-stopped
+
+  etcd-init:
+    image: quay.io/coreos/etcd:v3.5.5
+    depends_on:
+      - etcd
+
+    volumes:
+      - /usr/local/etc/samanbayaka:/etcd-data
+
+    environment:
+      - ETCDCTL_API=3
+
+    entrypoint: 
+      - /bin/sh
+      - -c
+
+    command:
+      - |
+        set -e
+
+        echo "waiting for etcd..."
+
+        for i in $(seq 1 30); do
+          etcdctl --endpoints=http://etcd:2379 endpoint health && break
+          echo "retry $$i..."
+          sleep 2
+        done
+
+        echo "etcd initialized"
+
+        echo "Creating the root role"
+        etcdctl --endpoints=http://etcd:2379 role add root || true
+
+        echo "Creating the root user"
+        etcdctl --endpoints=http://etcd:2379 user add root:${ETCD_ROOT_PW} || true
+
+        echo "Granting the root role to the root user"
+        etcdctl --endpoints=http://etcd:2379 user grant-role root root
+
+        echo "Enable authentication"
+        etcdctl --endpoints=http://etcd:2379 auth enable || true
+
+        # -------------------------------------------------------
+
+        echo "Creating admin role"
+        etcdctl --endpoints=http://etcd:2379 --user=root:${ETCD_ROOT_PW} role add admin
+
+        echo "Granting broad permissions"
+        etcdctl --endpoints=http://etcd:2379 --user=root:${ETCD_ROOT_PW} role grant-permission --prefix=true admin readwrite /
+
+        echo "Creating configadmin user"
+        etcdctl --endpoints=http://etcd:2379 --user=root:${ETCD_ROOT_PW} user add configadmin:${ETCD_CONFIG_ADMIN_PW}
+
+        echo "Assign role"
+        etcdctl --endpoints=http://etcd:2379 --user=root:${ETCD_ROOT_PW} user grant-role configadmin admin
+
+        # ---------------------------------------------------------
+
+        echo "Creating Roles and Granting"
+        etcdctl --endpoints=http://etcd:2379 --user=root:${ETCD_ROOT_PW} role add gateway
+        etcdctl --endpoints=http://etcd:2379 --user=root:${ETCD_ROOT_PW} role grant-permission --prefix=true gateway read /config/yaml/nats
+        etcdctl --endpoints=http://etcd:2379 --user=root:${ETCD_ROOT_PW} role grant-permission --prefix=true gateway read /config/yaml/auth
+        etcdctl --endpoints=http://etcd:2379 --user=root:${ETCD_ROOT_PW} role grant-permission --prefix=true gateway read /config/yaml/catch
+        etcdctl --endpoints=http://etcd:2379 --user=root:${ETCD_ROOT_PW} role grant-permission --prefix=true gateway read /config/yaml/telemetry
+
+        # ---------------------------------------------------------
+        
+        echo "Creating User and Assign Role"
+        etcdctl --endpoints=http://etcd:2379 --user=root:${ETCD_ROOT_PW} user add <your_user>:<your_pass>
+        etcdctl --endpoints=http://etcd:2379 --user=root:${ETCD_ROOT_PW} user grant-role apigtwy gateway
+
+        # ---------------------------------------------------------
+
+        etcdctl --endpoints=http://etcd:2379 --user=configadmin:${ETCD_CONFIG_ADMIN_PW} put /config/yaml/nats "$(cat /etcd-data/nats.yml)" || true
+
+        #etcdctl --endpoints=http://etcd:2379 --user=<your_user>:<your_pass> get /config/yaml/nats
+
+
+        echo "etcd initialization completed"
+```
+Optionally verify:
+```bash
+docker ps | grep 'sbk-etcd'
+```
+Optionally check health:
+```bash
+docker exec -it sbk-etcd etcdctl --endpoints=http://etcd:2379 endpoint health
+```
+---
+
 <p align="center" style="margin-top: 100px;">
   <img src="https://moleculer.services/images/banner.png" alt="Moleculer Logo" width="600">
 </p>

package/commit-hash.mjs

@@ -1 +1 @@
-export const COMMIT_HASH = '8d687db';
+export const COMMIT_HASH = '02c6744';

package/config-manager.mjs

@@ -0,0 +1,21 @@
+import { Etcd3 } from "etcd3"
+import YAML from 'yaml'
+
+const [user, pass] = (process.env.SBK_CONFIG_URPW || "ur:pa").split(":")
+
+const client = new Etcd3({
+  hosts: "http://etcd:12379",
+  auth: {
+    username: user,
+    password: pass
+  }
+})
+
+try {
+  const yamlText = await client.get("/config/yaml/telemetry")
+  const obj = YAML.parse(yamlText);
+
+  console.log(obj);
+} catch (err) {
+  console.error("Configuration reading error: ", err.message)
+}

package/helper/file/esm-loading.mjs

@@ -7,13 +7,24 @@ import os from "os"
 import { createRequire } from 'module'
 
 import chokidar from "chokidar"
-
+import { Etcd3 } from "etcd3"
+import YAML from 'yaml'
 
 const __filename = fileURLToPath(import.meta.url)
 const __dirname = path.dirname(__filename)
 const require = createRequire(import.meta.url)
 
 
+const [user, pass] = (process.env.SBK_CONFIG_URPW || "ur:pa").split(":")
+
+const client = new Etcd3({
+  hosts: "http://etcd:12379",
+  auth: {
+    username: user,
+    password: pass
+  }
+})
+
 /**
  * Configuration path
  * @type {string}
@@ -47,42 +58,51 @@ const SERVICES_DIR = __dirname.split('node_modules')[0]
  * @return {object}
  */
 export const getConfig = async (fileName) => {
-	const configFile = path.
-		join(
-			CONFIG_PATH,
-			`${fileName}.mjs`
-		)
-
-  // /**
-  //  * Validate Configuration 
-  //  */
-  // if(process.env.NODE_ENV){
-  //   console.error("NODE_ENV must be 'production', 'testing', or 'development'")
-  //   process.exit(0)
-  // }
+	// const configFile = path
+	// 	.join(
+	// 		CONFIG_PATH,
+	// 		`${fileName}.mjs`
+	// 	)
+
+  // // /**
+  // //  * Validate Configuration 
+  // //  */
+  // // if(process.env.NODE_ENV){
+  // //   console.error("NODE_ENV must be 'production', 'testing', or 'development'")
+  // //   process.exit(0)
+  // // }
   
 
-	return (await import(pathToFileURL(configFile).href)).default
+	// return (await import(pathToFileURL(configFile).href)).default
+
+  // try {
+    const yamlText = await client.get(`/config/yaml/${fileName}`)
+    return Object.freeze(
+      YAML.parse(yamlText)
+    )
+  // } catch (err) {
+  //   throw new Error(`Configuration reading error: ${err.message}`)
+  // }
 }
 
 
-/**
- * Getting server configuration.
- * @type {string}
- */
-const CONFIG = await getConfig('server')
+// /**
+//  * Getting server configuration.
+//  * @type {string}
+//  */
+// const CONFIG = await getConfig('server')
 
 
-/**
- * Assets path
- */
-export const assetPath = {
-  rootFolder: path
-    .join(
-      ABSOLUTE_PATH,
-      CONFIG.assetPath,
-    ),
-}
+// /**
+//  * Assets path
+//  */
+// export const assetPath = {
+//   rootFolder: path
+//     .join(
+//       ABSOLUTE_PATH,
+//       CONFIG.assetPath,
+//     ),
+// }
 
 
 /**
@@ -139,18 +159,18 @@ export const pkgDtls = {
 }
 
 
-/**
- * OpenAPI configuration
- */
-export const openApiConfig = {
-  info: {
-    title: pkgDtls.name,
-    version: pkgDtls.version,
-    summary: CONFIG.openApiInfo[0].summary,
-    description: CONFIG.openApiInfo[0].description,
-  },
-  servers: CONFIG.openApiInfo[0].url,
-}
+// /**
+//  * OpenAPI configuration
+//  */
+// export const openApiConfig = {
+//   info: {
+//     title: pkgDtls.name,
+//     version: pkgDtls.version,
+//     summary: CONFIG.openApiInfo[0].summary,
+//     description: CONFIG.openApiInfo[0].description,
+//   },
+//   servers: CONFIG.openApiInfo[0].url,
+// }
 
 // /**
 //  * Load modules from path as moleculer-repl is only supports cjs files

package/helper/utility/access-token-validator.mjs

@@ -2,9 +2,25 @@ import ApiGateway from "moleculer-web"
 import jwt from "jsonwebtoken"
 import jwksClient from "jwks-rsa"
 
+// const OPENID_CONFIG = {
+//   url: "https://accounts.google.com/.well-known/openid-configuration",
+//   url: "",
+//   clientId: "184045176764-ugg28aegdro383pintufun14uubtt374.apps.googleusercontent.com",
+//   jwksClient: {
+//     cache: true,
+//     cacheMaxEntries: 5,
+//     cacheMaxAge: 10 * 60 * 1000,
+//     rateLimit: true,
+//     jwksRequestsPerMinute: 10,
+//   },
+//   jwtVerifyAlgo: ["RS256"],
+//   client: {},
+//   issuer: "",
+// }
+
 const OPENID_CONFIG = {
-  url: "https://accounts.google.com/.well-known/openid-configuration",
-  clientId: "184045176764-ugg28aegdro383pintufun14uubtt374.apps.googleusercontent.com",
+  url: "https://auth.wbsedcl.in/realms/redpanda/.well-known/openid-configuration",
+  clientId: "bff-client",
   jwksClient: {
     cache: true,
     cacheMaxEntries: 5,
@@ -159,7 +175,18 @@ export const authorize = async (ctx, route, req) => {
       })
 
     ctx.broker.logger.debug({message: "Decoded Token Object", token: decoded})
-    ctx.meta.user = { name: decoded?.name, email: decoded?.email }
+    ctx.meta.user = {
+      sessionId: decoded?.sid,
+      id: decoded?.sub,
+      userId: (decoded?.sub).split(":").at(-1),
+      username: decoded?.preferred_username,
+      firstName: decoded?.given_name,
+      lastName: decoded?.family_name,
+      fullName: [...new Set(decoded?.name.split(/\s+/))].join(" "),
+      emailIds: decoded?.email,
+      mobileNos: decoded?.mobile_no,
+      userRoles: decoded?.user_role,
+    }
     return decoded
 
   } catch (err) {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "samanbayaka",
-  "version": "0.0.20",
+  "version": "0.0.22",
   "description": "Moleculer Gateway service with kafka transporter",
   "homepage": "https://gitlab.com/dalal.suvendu/samanbayaka#readme",
   "bugs": {
@@ -41,6 +41,7 @@
     "chokidar": "^5.0.0",
     "compression": "^1.8.1",
     "cookie-parser": "^1.4.7",
+    "etcd3": "^1.1.2",
     "helmet": "^8.1.0",
     "ioredis": "^5.10.1",
     "jsonwebtoken": "^9.0.3",
@@ -55,7 +56,8 @@
     "nats": "^2.29.3",
     "pino": "^10.3.1",
     "pnpm": "^10.32.1",
-    "sinon": "^21.1.2"
+    "sinon": "^21.1.2",
+    "yaml": "^2.9.0"
   },
   "devDependencies": {
     "chai": "^6.2.2",

package/services/about/index.mjs

@@ -41,7 +41,8 @@ export default {
       },
 
       handler: async(ctx) => {
-        return ctx.meta.user
+        const { sessionId, id, ...profile } = ctx.meta.user
+        return profile
       }
     }
   },

package/services/gateway/index.mjs

@@ -73,7 +73,7 @@ export default {
        */
       {
         path: "/api",
-        authorization: true,
+        authorization: false,
         whitelist: [
           /**
            * Access any actions except 'gateway' and 'system' service