samanbayaka 0.0.12 → 0.0.14

package/README.md

@@ -30,6 +30,10 @@ This project is a modular, production-ready microservices framework built with M
 ```bash
 pnpm install samanbayaka
 ```
+Once the package is installed, you can import the library using import or require approach:
+```bash
+import sbk from "samanbayaka"
+```
 # Prerequisites
 It is assumed that NATS, Redpanda, and Redis are installed and properly configured. All services should be discoverable through the /etc/hosts file.
 #### Minimum required Node.js version: >= 22.x.x
@@ -37,12 +41,35 @@ It is assumed that NATS, Redpanda, and Redis are installed and properly configur
 node -v
 nvm use 22
 ```
+Additionally, OpenObserve is used to view logs and telemetry.
 
 #### Environment variables
-To configure all required environment variables, copy the <your_project_name>/bash/sbk.sh file into the /etc/profile.d/ directory and update the environment variable values as needed. Before copying, ensure that your current working directory is the project directory.
+To configure all required environment variables, create a Bash file:
+```bash
+sudo nano /etc/logrotate.d/sbk.sh
+```
+Paste this:
+```bash
+## Configurations files path
+export SBK_CONFIG_PATH=/usr/local/etc/<your_folder>
+
+## Web server port 8760-8769
+export SBK_PORT=8765
+
+## Log level allowed values are fatal | error | warn | info | debug | trace
+export SBK_LOG_LEVEL=info
+
+## Maximum allowed TTL for the memoryLRU (L1) cache; the value is capped at 600 seconds.
+export MAX_L1_TTL=120
+
+## Maximum allowed TTL for the redis (L2) cache; the value is capped at 86400 seconds.
+export MAX_L2_TTL=600
+
+## Enabling telemetry middleware – Set the value to true; the default is false.
+export SBK_TELEMETRY_ENABLE=false
+```
+Set environment variables in the current Bash session.
 ```bash
-pwd
-sudo cp bash/sbk.sh /etc/profile.d/
 source /etc/profile.d/sbk.sh
 ```
 Optionally, you may verify the set environment variables.
@@ -53,14 +80,173 @@ echo $SBK_PORT
 Create a directory to store the configuration files, then copy all configuration files from the current project path into this directory and update them as needed. Before copying, ensure that your current working directory is the project directory.
 ```bash
 pwd
-sudo mkdir -p /usr/local/etc/<your_project_name>
-sudo cp config/*.*  /usr/local/etc/<your_project_name>
+sudo mkdir -p /usr/local/etc/<your_folder>
+sudo cp config/*.*  /usr/local/etc/<your_folder>
+```
+#### Log files
+For production, it is recommended to create a new directory named "samanbayaka" under /var/log to store logs.
+```bash
+sudo mkdir -p /var/log/samanbayaka
+sudo chown -R $USER:$(id -gn) /var/log/samanbayaka
+sudo chmod 755 /var/log/samanbayaka
+```
+Empty your log files if they become very large, as follows:
+```bash
+truncate -s 0 /var/log/samanbayaka/sbk-*.log
+```
+#### Setup logrotate (optional)
+Create config file:
+```bash
+sudo nano /etc/logrotate.d/sbk
+```
+Paste this:
+```bash
+/var/log/samanbayaka/sbk-*.log {
+  daily
+  rotate 14
+  compress
+  delaycompress
+  missingok
+  notifempty
+  copytruncate
+  create 0640 node node
+}
+```
+
+## API Docs
+
+### sbk.Errors
+
+`sbk.Errors` exposes the Moleculer `Errors` object.
+
+### sbk.getConfig
+
+`sbk.getConfig` is a function that returns a configuration object from a specified file name.
+
+The file is loaded from the configuration directory defined by the environment variable.
+
+### sbk.loadDemo
+
+`sbk.loadDemo` is an asynchronous function that creates a demo service to help understand the project.
+
+### sbk.loadGatewayService
+
+`sbk.loadGatewayService` is an asynchronous function that creates a gateway service to expose REST APIs.
+
+### sbk.loadFeatureService
+
+`sbk.loadFeatureService` is an asynchronous function that creates a feature service.
+
+##### Function Signature
+
+```js
+sbk.loadFeatureService(schema)
+```
+
+##### Parameter
+
+*  `schema` - An object that follows the Moleculer service schema structure.
+
+Example:
+
+```js
+{
+    name: "math",
+    actions: {
+        add(ctx) {
+            return Number(ctx.params.a) + Number(ctx.params.b);
+        },
+
+        sub(ctx) {
+            return Number(ctx.params.a) - Number(ctx.params.b);
+        }
+    }
+}
+```
+
+
+### sbk.auxBrokerService
+
+`sbk.auxBrokerService` is a function used to initialize and manage message brokers such as Kafka, MQTT, and RabbitMQ.
+
+##### Function Signature
+
+```js
+sbk.auxBrokerService(
+  type,
+  brokerOpts = {},
+  callback = () => {}
+)
+```
+##### Parameters
+
+* `type` *(string)* - Specifies the broker type to initialize. Supported values:
+
+  * `"kafka"`
+  * `"mqtt"`
+  * `"amqp"`
+
+Example:
+
+```js
+"kafka"
+```
+
+
+* `brokerOpts` *(object)* - Configuration object used to initialize the broker client, producer, and consumer. Example structure:
+
+```js
+{
+  name: <your_service_name>
+  client: {},
+  producer: {},
+  consumer: {
+    groupId: "all",
+    interMessageDelayMs: 10,
+  },
+  logLevel: "INFO",
+  gzip: true,
+  msgPack: true,
+  rest: false,
+}
 ```
 
+Configuration Options
+
+| Property | Type | Description |
+|---|---|---|
+| `name` | `string` | Only lowercase letters (a–z), digits (0–9), and hyphens (-) are allowed. |
+| `client` | `object` | Broker client configuration |
+| `producer` | `object` | Producer configuration |
+| `consumer` | `object` | Consumer configuration |
+| `consumer.groupId` | `string` | Consumer group identifier is mandatory for the consumer |
+| `consumer.interMessageDelayMs` | `number` | Optional delay between messages in milliseconds |
+| `logLevel` | `string` | Logging level (`NOTHING`, `ERROR`, `WARN`, `INFO`, `DEBUG`), default `NOTHING` |
+| `gzip` | `boolean` | Enables message compression; only GZIP is supported, default `true` |
+| `msgPack` | `boolean` | Enables MessagePack serialization, default `true` |
+| `rest` | `boolean` | Enables REST end point, default `false` |
+
+
+* `callback` *(function)* - Callback function executed when a message/event is received.
+
+Arguments
+
+| Argument | Description |
+|---|---|
+| `ctx` | Broker context |
+| `payload` | Incoming message payload |
+
+Example:
+
+```js
+(ctx, payload) => {
+  ctx.broker.logger.debug(payload)
+}
+```
 
 # Usage
-#### Create a service, gateway
-The gateway is an important service that exposes all endpoints as REST APIs. At least one Gateway service is mandatory in the project.
+#### Create gateway service
+The Gateway is a critical service that exposes all endpoints as REST APIs. At least one Gateway service must be running to make the REST APIs accessible.
 ```bash
 mkdir gateway
 cd gateway
@@ -69,18 +255,22 @@ pnpm install samanbayaka
 touch index.mjs
 ```
 
-Open and edit the index.mjs file as follows
+Open index.mjs and paste the following:
 ```bash
 import sbk from "samanbayaka"
 await sbk.loadGatewayService()
 ```
-Run the service, gateway
+---
+#### Create feature services
 ```bash
-node index.mjs
+mkdir <your_service_name>
+cd <your_service_name>
+npm init -y
+pnpm install samanbayaka
+touch index.mjs
 ```
 
-
-#### Create your endpoints as a feature service, such as hello
+A feature service, such as `hello`, can be created as follows:
 ```bash
 mkdir hello
 cd hello
@@ -89,10 +279,10 @@ pnpm install samanbayaka
 touch index.mjs
 ```
 
-Open and edit the index.mjs file as follows
+Open index.mjs and paste the following:
 ```bash
 import sbk from "samanbayaka"
-await sbk.loadFeatureService.mainBus({
+await sbk.loadFeatureService({
   name: "hello",
   version: "v1",
 
@@ -109,9 +299,11 @@ await sbk.loadFeatureService.mainBus({
       cache: {
 
         /**
-         * These cache entries will be expired after 5 seconds instead of 30.
+         * Cache entries expire after 30 seconds.
+         * For L1 caching, TTL should be defined as [L2_TTL, L1_TTL].
+         * Example: ttl: [60, 10] → L2 TTL is 60 seconds and L1 TTL is 10 seconds.
          */      
-        ttl: 2*60
+        ttl: 30
       },
       handler(ctx){
         return "Hello Samanbayaka"
@@ -123,17 +315,109 @@ await sbk.loadFeatureService.mainBus({
         return `Welcome, ${ctx.params.query?.name || "Guest"} - ${ctx.broker.nodeID}`
       }
     },
-    createUser
   },
 })
 
 ```
-Run the service, hello
+---
+#### Create auxaliary broker services
+##### for producer
+```bash
+mkdir <your_service_name>-<type>-<topic>
+cd <your_service_name>-<type>-<topic>
+npm init -y
+pnpm install samanbayaka
+touch index.mjs
+```
+An auxiliary service, such as a `producer` that publishes messages to the `STUDENT` topic, can be created as follows:
+```js
+mkdir producer-kafka-student
+cd producer-kafka-student
+npm init -y
+pnpm install samanbayaka
+touch index.mjs
+```
+
+Open index.mjs and paste the following:
+```bash
+import sbk from "samanbayaka"
+await sbk.auxBrokerService(
+  "kafka",
+  {
+    name: "producer-kafka-student", 
+    rest: true,
+    ...
+  }, 
+  () => {}
+)
+```
+In the service configuration `name` end with "`*`" i.e. `producer-kafka-student*` can capable to publish to the all topics start with `STUDENT` like `STUDENT.SCIENCE`, `STUDENT.ARTS` etc.
+```js
+name: "producer-kafka-student*"
+```
+
+If you do not expose the service as a REST API and run it as a private/internal service, either remove the `rest` property or set it to `false`.
+```js
+rest: false
+```
+
+##### for consumer
+```bash
+mkdir <your_service_name>-<type>-<topic>-<group>
+cd <your_service_name>-<type>-<topic>-<group>
+npm init -y
+pnpm install samanbayaka
+touch index.mjs
+```
+An auxiliary service, such as a `consumer` that subscribe messages from the `STUDENT` topic having group name `junior`, can be created as follows:
+```js
+mkdir producer-kafka-student-junior
+cd producer-kafka-student-junior
+npm init -y
+pnpm install samanbayaka
+touch index.mjs
+```
+Open index.mjs and paste the following:
+```bash
+import sbk from "samanbayaka"
+await sbk.auxBrokerService(
+  "kafka",
+  {
+    name: "consumer-kafka-student", 
+    consumer: {
+      groupId: "junior",
+      interMessageDelayMs: 10,
+      ...
+    },
+    ...
+  }, 
+  () => {}
+)
+```
+The `interMessageDelayMs` option introduces a delay between two consecutive message/event consumptions, measured in milliseconds. If you do not need any delay between message consumption, simply remove the `interMessageDelayMs` property from the consumer options.
+
+
+---
+
+#### Run the services 
+
+ * development/testing
 ```bash
+cd <your_path>/gateway
 node index.mjs
+cd <your_path>/hello
+node index.mjs
+```
+ * production
+```bash
+cd <your_path>/gateway
+node index.mjs >> /var/log/samanbayaka/sbk-${HOSTNAME}-$$.log 2>&1
+cd <your_path>/hello
+node index.mjs >> /var/log/samanbayaka/sbk-${HOSTNAME}-$$.log 2>&1
 ```
+
 # Demo
-You can also run the demo service to better understand how the system works.
+You can also run the demo service to better understand how microservices work in the Moleculer ecosystem and how REST APIs are exposed.
 ```bash
 mkdir demo
 cd demo
@@ -151,9 +435,6 @@ Run the service, demo
 node demo.mjs
 ```
 
-
-
-
 <p align="center" style="margin-top: 100px;">
   <img src="https://moleculer.services/images/banner.png" alt="Moleculer Logo" width="600">
 </p>

package/commit-hash.mjs

@@ -1 +1 @@
-export const COMMIT_HASH = '3290420';
+export const COMMIT_HASH = '782e7c8';

package/config/catcher.mjs

@@ -0,0 +1,27 @@
+/*catcher.mjs*/
+export default {
+  /**
+   * Maximum items
+   */
+  max: 200,
+
+  /**
+   * Time-to-Live in seconds
+   */
+  ttl: [600, 120],
+
+  redis: {           
+    /**
+     * Turns Redis client monitoring on.
+     */
+    monitor: false,
+
+    /**
+     * Redis settings like hostname, port, password
+     */
+    host: "redis",
+    port: 6379,
+    // password: "1234",
+    // db: 0,
+  }
+}

package/config/kafkajs.mjs

@@ -1,54 +1,66 @@
-/*kafkajs.mjs*/
-// export default Object.freeze({
-//   prefix: "SBK",
-//   client: {
-//     clientId: "moleculer-kafkajs",
-//     brokers: ["redpanda:9092"],
-//     connectionTimeout: 3000,
-//     requestTimeout: 25000,
-//     enforceRequestTimeout: false,
-//     retry: {
-//       maxRetryTime: 30000,
-//       initialRetryTime: 300,
-//       factor: 0.2,
-//       multiplier: 2,
-//       retries: 5
-//     },
-//     producer: {
-//       idempotent: true,
-//       maxInFlightRequests: 1,
-//     },
-//     logLevel: "INFO",
-//   }
-// })
-
-
-export default Object.freeze({
-  type: "Kafka",
-  options: {
-    // host: "localhost:9092", // Your Kafka broker address
-    client: {
-      clientId: "moleculer-cluster",
-      brokers: ["redpanda:9092"],
-      connectionTimeout: 3000,
-      requestTimeout: 25000,
-      enforceRequestTimeout: false,
-      retry: {
-        maxRetryTime: 30000,
-        initialRetryTime: 300,
-        factor: 0.2,
-        multiplier: 2,
-        retries: 5
-      }      
+export default {
+  client: {
+    clientId: "node-client123",
+    brokers:  ["pnsntst.wbsedcl.in:9092"],
+    connectionTimeout: 20000,
+    requestTimeout: 20000,
+    enforceRequestTimeout: true,
+    retry: {
+      maxRetryTime: 60000,
+      initialRetryTime: 300,
+      factor: 0.2,
+      multiplier: 2,
+      retries: 5
     },
-    producer: {
-      idempotent: true,
-      maxInFlightRequests: 1,
+    ssl: {
+     rejectUnauthorized: true,
     },
-    consumer: {
-      groupId: "moleculer-group"
+    sasl: {
+      mechanism: "scram-sha-256",
+      username:  "bob",
+      password:  "SUhcznwt0xopsTfDXKIPzgw66dzJpZ",
+    }, 
+  },
+  producer: {
+    retry: {
+      initialRetryTime: 300,
+      // retries: Number.MAX_SAFE_INTEGER
+      retries: 8,
+      maxRetryTime: 30000,
+      factor: 0.2,
+      multiplier: 2
     },
-    logLevel: "INFO",
-  }
-  // Ensure protocol v5 is active (default in 0.15.0-beta1)
-})
+    metadataMaxAge: 300000,
+    allowAutoTopicCreation: false,
+    transactionTimeout: 30000,
+    idempotent: true,
+    maxInFlightRequests: 1,
+  },
+  consumer: {
+    groupId: "default",
+    // partitionAssigners: <Array>,
+    sessionTimeout: 30000,
+    rebalanceTimeout: 60000,
+    heartbeatInterval: 5000,
+    metadataMaxAge: 300000,
+    allowAutoTopicCreation: false,
+    maxBytesPerPartition: 1048576,      // 1 MB
+    minBytes: 1,
+    maxBytes: 10485760,                 // 10 MB
+    maxWaitTimeInMs: 5000,
+    retry: {
+      initialRetryTime: 300,
+      retries: 8,
+      maxRetryTime: 30000,
+      factor: 0.2,
+      multiplier: 2
+    },
+    maxInFlightRequests: 1,
+    // rackId: <String>,
+    // interMessageDelayMs: 10,
+  },
+  logLevel: "INFO",
+  gzip: true,
+  msgPack: true,
+  rest: false,
+}