systemlynx 1.18.12 → 1.20.0

package/README.md

@@ -2,7 +2,10 @@
 
 # SystemLynx JS ![GitHub license](https://img.shields.io/badge/license-MIT-blue.svg) ![PRs Welcome](https://img.shields.io/badge/PRs-welcome-blue.svg) ![JS 100%](https://img.shields.io/badge/JavaScript-100%25-green)
 
-SystemLynx is a NodeJS framework for building modular web APIs, built on top of ExpressJS and Socket.io. It allows you to create objects and load them from a server into a client application.
+SystemLynx is a Node.js framework for building modular, distributed web applications using an RPC-style architecture. It enables services to expose structured modules whose methods can be invoked remotely from client applications, abstracting away much of the complexity of client-to-service communication.
+
+Built on top of Express and Socket.io, SystemLynx supports both request-response and real-time communication patterns, allowing developers to build scalable APIs and event-driven systems with a unified interface.
+
 
 SystemLynx comes with the following objects that are used for web app development:
 
@@ -39,7 +42,7 @@ Service.module("Users", Users);
 
 In the code above we assigned an object to the variable `Users` and gave it an add method. The `Service.module(name, constructor/object)` function takes the name assigned to the object as the first argument and the object itself as the second argument.
 
-Alternatively, you can use a constructor function instead of an object as the second argument. In the example below we create another **Module** called "Orders". This time we use a constructor function as the second argument of the to **Service.module** function. The `this` value is the initial instance of the **Module** object. Every method added to the `this` value will be accessible when the object is loaded by a **SystemLynx Client**. Note: **Module** methods can be synchronous or asynchronous functions.
+Alternatively, you can use a constructor function instead of an object as the second argument. In the example below we create another **Module** called "Orders". This time we use a constructor function as the second argument of the to **Service.module** function. The `this` value is the initial instance of the **Module** object. Every method added to the `this` value will be accessible when the object is loaded by a **SystemLynx Client**. Note: **Module** methods can be synchronous or asynchronous functions.
 
 ```javascript
 const { Service } = require("systemlynx");
@@ -156,8 +159,8 @@ const Users = {};
 
 Users.add = function (data) {
   console.log(data);
-  return { message: "You have successfully called the Users.add method" };
   this.emit("new_user", { message: "new_user event test" });
+  return { message: "You have successfully called the Users.add method" };
 };
 
 Service.module("Users", Users);
@@ -173,3 +176,103 @@ Service.module("Orders", function () {
 
 Service.startService({ route: "test/service", port: "4400", host: "localhost" });
 ```
+
+---
+
+# Middleware
+
+SystemLynx supports middleware that runs before or after a module method is called. This is useful for things like authentication, logging, or validating requests.
+
+Use `Service.before` to add a middleware function that runs before any method is invoked, and `Service.after` to run one after the response is ready.
+
+```javascript
+const { Service } = require("systemlynx");
+
+Service.before(function (req, res, next) {
+  console.log("Incoming request:", req.fn);
+  next();
+});
+
+Service.module("Users", Users);
+Service.module("Orders", Orders);
+
+Service.startService({ route: "api", port: 4400, host: "localhost" });
+```
+
+You can also scope middleware to a specific module or method so it only runs where you need it.
+
+```javascript
+// Runs before every method on the Users module
+Service.before("Users", authMiddleware);
+
+// Runs only before Users.delete
+Service.before("Users.delete", requireAdminMiddleware);
+```
+
+See the [API Documentation](https://github.com/Odion100/SystemLynx/blob/master/API.md#tasksjs-api-documentation) for the full middleware API.
+
+---
+
+# Architecture: Monolith to Microservices
+
+One of the core ideas behind SystemLynx is that your module code doesn't change depending on how you deploy it. You can start with everything in a single service and scale it out later — without rewriting anything.
+
+## Start as a monolith
+
+```javascript
+const { Service } = require("systemlynx");
+
+Service.module("Users", Users);
+Service.module("Orders", Orders);
+Service.module("Products", Products);
+
+Service.startService({ route: "api", port: 4400, host: "localhost" });
+```
+
+Your client loads everything from one place:
+
+```javascript
+const { Users, Orders, Products } = await Client.loadService("http://localhost:4400/api");
+```
+
+## Scale out: move modules to their own services
+
+When you're ready to scale, pull any module into its own service. The module code is unchanged — just move it and update the URL the client loads from.
+
+```javascript
+// users-service.js
+Service.module("Users", Users);
+Service.startService({ route: "users", port: 4401, host: "localhost" });
+
+// orders-service.js
+Service.module("Orders", Orders);
+Service.startService({ route: "orders", port: 4402, host: "localhost" });
+```
+
+```javascript
+// client
+const { Users } = await Client.loadService("http://localhost:4401/users");
+const { Orders } = await Client.loadService("http://localhost:4402/orders");
+```
+
+Because every module can be hosted independently, you can scale horizontally (run multiple instances of a service) or vertically (split a busy module into its own service) as your needs grow.
+
+## Load Balancing
+
+The **LoadBalancer** lets you run multiple clones of a service and distribute requests across them. Clones register themselves with the LoadBalancer, which handles routing — your client just talks to the LoadBalancer URL and doesn't need to know how many instances are running behind it.
+
+```javascript
+const { LoadBalancer } = require("systemlynx");
+
+LoadBalancer.startService({ route: "users", port: 4400, host: "localhost" });
+```
+
+Each clone registers on startup:
+
+```javascript
+const { Client } = require("systemlynx");
+
+const { clones } = await Client.loadService("http://localhost:4400/users");
+
+await clones.register({ host: "localhost", port: 4401, route: "/users" });
+```

package/RFCs/001-websocket-named-events-room-scoping.md

@@ -0,0 +1,133 @@
+# RFC: WebSocket Named Events + Room-Based Scoping
+
+## Context
+
+Currently all server-to-client WebSocket traffic flows through a single event named `"dispatch"`. The server uses `socket.emit("dispatch", {id, name, data, type})` on the namespace, which broadcasts to **every connected client** in the namespace. Each client then checks `event.name` and routes or discards client-side. This wastes bandwidth — if 10 clients are connected and only 1 subscribed to `"orderCreated"`, all 10 still receive the packet.
+
+The goal is to use Socket.io's native named events + room-based scoping so that each event is only delivered to clients that actually subscribed to it.
+
+---
+
+## Architecture Change
+
+**Before:**
+```
+Server: namespace.emit("dispatch", { id, name: "orderCreated", data, type })
+          → ALL clients in namespace receive it
+          → each client checks event.name, ignores if not listening
+```
+
+**After:**
+```
+Client subscribes: socket.emit("subscribe", "orderCreated")
+Server joins room: clientSocket.join("orderCreated")
+Server emits:      namespace.to("orderCreated").emit("orderCreated", { id, data, type })
+          → ONLY clients in room "orderCreated" receive it
+```
+
+The user-facing API is **unchanged** — `module.on("eventName", cb)` still works the same.
+
+---
+
+## Files to Change
+
+### 1. `systemlynx/ServerManager/components/SocketEmitter.js`
+
+Two changes:
+- Add a `"connection"` handler on the namespace to manage room membership per connected client
+- Change the emit to target the room instead of broadcasting to all
+
+```javascript
+// Add after creating namespace socket:
+socket.on("connection", (clientSocket) => {
+  clientSocket.on("subscribe", (name) => clientSocket.join(name));
+  clientSocket.on("unsubscribe", (name) => clientSocket.leave(name));
+});
+
+// Change the emit:
+// OLD: socket.emit("dispatch", { id, name, data, type });
+// NEW: socket.to(name).emit(name, { id, data, type });
+```
+
+### 2. `systemlynx/Client/components/SocketDispatcher.js`
+
+Three changes:
+
+**a) Replace `socket.on("dispatch", ...)` with `socket.onAny(...)`** to receive named events. Reconstruct the event object explicitly (no spread — avoids payload fields bleeding into event shape). `name` is kept on the event object since a single callback can handle multiple events and may need to know which fired:
+```javascript
+socket.onAny((name, payload) => {
+  const event = { id: payload.id, name, data: payload.data, type: payload.type };
+  dispatcher.emit(name, payload.data, event);
+});
+```
+
+**b) Track subscription counts and sync with server rooms** by overriding `on`, `once`, `$clearEvent`, and `destroy`. Use a `subscriptionCounts` Map. On first listener for an event → emit `"subscribe"` to server. When last listener removed → emit `"unsubscribe"`. Use a `RESERVED` set for socket.io internal events (`connect`, `disconnect`, `error`, `connect_error`) that must never be treated as subscriptions.
+
+**c) Re-subscribe on reconnect** — inside the `"connect"` handler, re-emit `"subscribe"` for all events currently tracked in `subscriptionCounts`.
+
+Subscription reference-counting sketch:
+```javascript
+const subscriptionCounts = new Map();
+const RESERVED = new Set(["connect", "disconnect", "error", "connect_error"]);
+
+const trackSubscribe = (name) => {
+  const n = (subscriptionCounts.get(name) || 0) + 1;
+  subscriptionCounts.set(name, n);
+  if (n === 1) socket.emit("subscribe", name);
+};
+
+const trackUnsubscribe = (name) => {
+  const n = (subscriptionCounts.get(name) || 0) - 1;
+  if (n <= 0) { subscriptionCounts.delete(name); socket.emit("unsubscribe", name); }
+  else subscriptionCounts.set(name, n);
+};
+```
+
+Override `on`: call original, if not RESERVED → `trackSubscribe`, return unsub that also calls `trackUnsubscribe`.  
+Override `once`: same but auto-`trackUnsubscribe` when the callback fires (use a `done` flag to prevent double-unsubscribe if unsub is called before event fires).  
+Override `$clearEvent`: call original, if tracked → clear count and emit `"unsubscribe"`.  
+Override `destroy`: emit `"unsubscribe"` for all tracked events, clear map, call original.
+
+### 3. `systemlynx/Client/tests/SocketDispatcher.test.js`
+
+Two changes:
+
+**a) Add subscription handling to test server setup** (mirrors what SocketEmitter does internally):
+```javascript
+socket.on("connection", (clientSocket) => {
+  clientSocket.on("subscribe", (name) => clientSocket.join(name));
+  clientSocket.on("unsubscribe", (name) => clientSocket.leave(name));
+});
+```
+
+**b) Change server-side emit in test** from namespace broadcast to room-targeted named event:
+```javascript
+// OLD:
+socket.emit("dispatch", { name: eventName, data: { testPassed: true } })
+// NEW:
+socket.to(eventName).emit(eventName, { id: "test-id", data: { testPassed: true }, type: "WebSocket" })
+```
+
+Update the `event` deep-equal assertion in the `SocketDispatcher.apply()` test to match the new reconstructed shape `{ id, name, data, type }`.
+
+### 4. `/Users/odionedwards/SystemLynx-client/systemlynx/Client/components/SocketDispatcher.mjs`
+
+The `systemlynx-client` package (separate repo at `/Users/odionedwards/SystemLynx-client/`, used by `systemview`) has its own `SocketDispatcher.mjs` that is identical in logic to `SocketDispatcher.js` but uses ES module syntax (`import`/`export default`). Apply the exact same changes — subscription count Map, `on`/`once`/`$clearEvent`/`destroy` overrides, `socket.onAny`, reconnect re-subscription — written with ES module syntax.
+
+---
+
+## Files NOT Changed
+
+- `systemlynx/ServerManager/components/Router.js` — HTTP method calls, unaffected
+- `systemlynx/Client/components/ServiceRequestHandler.js` — HTTP method calls, unaffected
+- `systemlynx/Client/tests/Client.test.js` — event shape `has.all.keys("id", "name", "data", "type")` still matches ✓
+- `systemlynx/App/tests/App.test.js` — event shape not tested here ✓
+
+---
+
+## Verification
+
+1. `npm test` — all existing tests should pass
+2. The `SocketDispatcher.test.js` "emit and handle events" tests confirm room delivery works
+3. The `Client.test.js` "receive events emitted from backend" test verifies end-to-end: server `eventTester.emit(...)` → only subscribed client receives it
+4. Manually: connect two clients to the same service, subscribe one to `"eventA"`, confirm only that client gets the event when the server emits it

package/index.test.js

@@ -34,8 +34,10 @@ describe("SystemLynx Objects", () => {
       .that.has.all.keys(
         "module",
         "on",
+        "once",
         "emit",
         "$clearEvent",
+        "destroy",
         "before",
         "after",
         "use",
@@ -96,8 +98,10 @@ describe("SystemLynx Objects", () => {
         "before",
         "after",
         "on",
+        "once",
         "emit",
         "$clearEvent",
+        "destroy",
         "clones",
         "register",
         "dispatch",

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "systemlynx",
-  "version": "1.18.12",
+  "version": "1.20.0",
   "description": "",
   "main": "index.js",
   "browser": {

package/systemlynx/App/tests/App.test.js

@@ -12,10 +12,12 @@ describe("createApp()", () => {
       .that.has.all.keys(
         "module",
         "on",
+        "once",
         "emit",
         "before",
         "after",
         "$clearEvent",
+        "destroy",
         "use",
         "startService",
         "loadService",
@@ -62,7 +64,9 @@ describe("App: Loading Services", () => {
             .that.has.all.keys(
               "emit",
               "on",
+              "once",
               "$clearEvent",
+              "destroy",
               "resetConnection",
               "disconnect",
               "headers",
@@ -103,7 +107,9 @@ describe("App: Loading Services", () => {
           .that.has.all.keys(
             "emit",
             "on",
+            "once",
             "$clearEvent",
+            "destroy",
             "resetConnection",
             "disconnect",
             "headers",
@@ -136,48 +142,52 @@ describe("App: Loading Services", () => {
 
     await new Promise((resolve) => {
       const App = createApp();
-      App.loadService("test", url)
-        .on("service_loaded", (test) => {
-          expect(test)
-            .to.be.an("object")
-            .that.has.all.keys(
-              "emit",
-              "on",
-              "$clearEvent",
-              "resetConnection",
-              "disconnect",
-              "headers",
-              "setHeaders",
-              "mod"
-            )
-            .that.respondsTo("emit")
-            .that.respondsTo("$clearEvent")
-            .that.respondsTo("on")
-            .that.respondsTo("resetConnection")
-            .that.respondsTo("headers")
-            .that.respondsTo("setHeaders");
-        })
-        .on("service_loaded:test", (test) => {
-          expect(test)
-            .to.be.an("object")
-            .that.has.all.keys(
-              "emit",
-              "on",
-              "$clearEvent",
-              "resetConnection",
-              "disconnect",
-              "headers",
-              "setHeaders",
-              "mod"
-            )
-            .that.respondsTo("emit")
-            .that.respondsTo("$clearEvent")
-            .that.respondsTo("on")
-            .that.respondsTo("resetConnection")
-            .that.respondsTo("headers")
-            .that.respondsTo("setHeaders");
-          resolve();
-        });
+      App.loadService("test", url);
+      App.on("service_loaded", (test) => {
+        expect(test)
+          .to.be.an("object")
+          .that.has.all.keys(
+            "emit",
+            "on",
+            "once",
+            "$clearEvent",
+            "destroy",
+            "resetConnection",
+            "disconnect",
+            "headers",
+            "setHeaders",
+            "mod"
+          )
+          .that.respondsTo("emit")
+          .that.respondsTo("$clearEvent")
+          .that.respondsTo("on")
+          .that.respondsTo("resetConnection")
+          .that.respondsTo("headers")
+          .that.respondsTo("setHeaders");
+      });
+      App.on("service_loaded:test", (test) => {
+        expect(test)
+          .to.be.an("object")
+          .that.has.all.keys(
+            "emit",
+            "on",
+            "once",
+            "$clearEvent",
+            "destroy",
+            "resetConnection",
+            "disconnect",
+            "headers",
+            "setHeaders",
+            "mod"
+          )
+          .that.respondsTo("emit")
+          .that.respondsTo("$clearEvent")
+          .that.respondsTo("on")
+          .that.respondsTo("resetConnection")
+          .that.respondsTo("headers")
+          .that.respondsTo("setHeaders");
+        resolve();
+      });
     });
   });
 
@@ -204,7 +214,9 @@ describe("App: Loading Services", () => {
             .that.has.all.keys(
               "emit",
               "on",
+              "once",
               "$clearEvent",
+              "destroy",
               "resetConnection",
               "disconnect",
               "headers",
@@ -236,8 +248,10 @@ describe("App SystemObjects: Initializing Modules,  Modules and configurations",
             "useService",
             "useConfig",
             "on",
+            "once",
             "emit",
             "$clearEvent",
+            "destroy",
             "before",
             "after"
           )
@@ -257,8 +271,10 @@ describe("App SystemObjects: Initializing Modules,  Modules and configurations",
             "useService",
             "useConfig",
             "on",
+            "once",
             "emit",
             "$clearEvent",
+            "destroy",
             "before",
             "after"
           )
@@ -472,19 +488,19 @@ describe("SystemContext", () => {
           .that.respondsTo("useConfig");
         this.configPassed = true;
         next();
-      })
-      .on("ready", function () {
-        expect(this)
-          .to.be.an("object")
-          .that.respondsTo("useService")
-          .that.respondsTo("useModule")
-          .that.respondsTo("useConfig");
-        const mod1 = this.useModule("mod1");
-        const config = this.useConfig();
-        expect(mod1.testPassed).to.equal(true);
-        expect(config.configPassed).to.equal(true);
-      })
-      .on("ready", function () {
+      });
+    App.on("ready", function () {
+      expect(this)
+        .to.be.an("object")
+        .that.respondsTo("useService")
+        .that.respondsTo("useModule")
+        .that.respondsTo("useConfig");
+      const mod1 = this.useModule("mod1");
+      const config = this.useConfig();
+      expect(mod1.testPassed).to.equal(true);
+      expect(config.configPassed).to.equal(true);
+    });
+    App.on("ready", function () {
         expect(this)
           .to.be.an("object")
           .that.respondsTo("useService")
@@ -543,7 +559,8 @@ describe("SystemContext", () => {
           expect(event.type).to.equal("WebSocket");
           resolve();
         });
-        EventTesterModule.sendEvent(eventName);
+        // small delay so "subscribe" WebSocket message is processed before the HTTP call triggers the emit
+        setTimeout(() => EventTesterModule.sendEvent(eventName), 100);
       })
     );
   });

package/systemlynx/Client/components/ServiceRequestHandler.js

@@ -10,6 +10,32 @@ const makeQuery = (data) =>
     (query, name) => (query += `${name}=${data[name]}&`),
     "?"
   );
+
+const extractFilesFromArguments = (__arguments) => {
+  let foundFile = null;
+  let fileType = null;
+
+  __arguments.forEach((arg) => {
+    if (isObject(arg)) {
+      if (arg.file) {
+        if (foundFile)
+          throw new Error("Only one file or files allowed across arguments.");
+        foundFile = arg.file;
+        fileType = "file";
+        arg.file = "__file__";
+      } else if (arg.files) {
+        if (foundFile)
+          throw new Error("Only one file or files allowed across arguments.");
+        foundFile = arg.files;
+        fileType = "files";
+        arg.files = "__files__";
+      }
+    }
+  });
+
+  return { foundFile, fileType };
+};
+
 module.exports = function ServiceRequestHandler(
   httpClient,
   protocol,
@@ -23,14 +49,20 @@ module.exports = function ServiceRequestHandler(
 
     const tryRequest = (cb, errCount = 0) => {
       const { route, port, host } = this.__connectionData();
-      const singleFileURL = `${protocol}${host}:${port}/sf${route}/${fn}`;
-      const multiFileURL = `${protocol}${host}:${port}/mf${route}/${fn}`;
+      const { foundFile, fileType } = extractFilesFromArguments(__arguments);
+
       const defaultURL = `${protocol}${host}:${port}${route}/${fn}`;
-      const { file, files } = __arguments[0] || {};
-      const url = file ? singleFileURL : files ? multiFileURL : defaultURL;
+      const url =
+        fileType === "file"
+          ? `${protocol}${host}:${port}/sf${route}/${fn}`
+          : fileType === "files"
+          ? `${protocol}${host}:${port}/mf${route}/${fn}`
+          : defaultURL;
+
       const defaultHeaders = this.headers();
       const headers = !isEmpty(defaultHeaders) ? defaultHeaders : Service.headers();
-      if (url === defaultURL) {
+
+      if (!foundFile) {
         const query =
           method === "get" && isObject(__arguments[0]) ? makeQuery(__arguments[0]) : "";
         httpClient
@@ -43,13 +75,12 @@ module.exports = function ServiceRequestHandler(
           .then((results) => cb(null, results))
           .catch((err) => ErrorHandler(err, errCount, cb));
       } else {
-        delete __arguments[0].file;
-        delete __arguments[0].files;
-        const formData = {};
-        formData.__arguments = __arguments;
-        if (file) formData.file = isNode ? convertToReadStream(file) : file;
-        if (Array.isArray(files))
-          formData.files = isNode ? files.map(convertToReadStream) : files;
+        const formData = { __arguments };
+        if (fileType === "file")
+          formData.file = isNode ? convertToReadStream(foundFile) : foundFile;
+        if (fileType === "files")
+          formData.files = isNode ? foundFile.map(convertToReadStream) : foundFile;
+
         httpClient
           .upload({
             url,