systemlynx 1.19.12 → 1.21.0

package/README.md

@@ -2,7 +2,10 @@
 
 # SystemLynx JS ![GitHub license](https://img.shields.io/badge/license-MIT-blue.svg) ![PRs Welcome](https://img.shields.io/badge/PRs-welcome-blue.svg) ![JS 100%](https://img.shields.io/badge/JavaScript-100%25-green)
 
-SystemLynx is a NodeJS framework for building modular web APIs, built on top of ExpressJS and Socket.io. It allows you to create objects and load them from a server into a client application.
+SystemLynx is a Node.js framework for building modular, distributed web applications using an RPC-style architecture. It enables services to expose structured modules whose methods can be invoked remotely from client applications, abstracting away much of the complexity of client-to-service communication.
+
+Built on top of Express and Socket.io, SystemLynx supports both request-response and real-time communication patterns, allowing developers to build scalable APIs and event-driven systems with a unified interface.
+
 
 SystemLynx comes with the following objects that are used for web app development:
 
@@ -39,7 +42,7 @@ Service.module("Users", Users);
 
 In the code above we assigned an object to the variable `Users` and gave it an add method. The `Service.module(name, constructor/object)` function takes the name assigned to the object as the first argument and the object itself as the second argument.
 
-Alternatively, you can use a constructor function instead of an object as the second argument. In the example below we create another **Module** called "Orders". This time we use a constructor function as the second argument of the to **Service.module** function. The `this` value is the initial instance of the **Module** object. Every method added to the `this` value will be accessible when the object is loaded by a **SystemLynx Client**. Note: **Module** methods can be synchronous or asynchronous functions.
+Alternatively, you can use a constructor function instead of an object as the second argument. In the example below we create another **Module** called "Orders". This time we use a constructor function as the second argument of the to **Service.module** function. The `this` value is the initial instance of the **Module** object. Every method added to the `this` value will be accessible when the object is loaded by a **SystemLynx Client**. Note: **Module** methods can be synchronous or asynchronous functions.
 
 ```javascript
 const { Service } = require("systemlynx");
@@ -156,8 +159,8 @@ const Users = {};
 
 Users.add = function (data) {
   console.log(data);
-  return { message: "You have successfully called the Users.add method" };
   this.emit("new_user", { message: "new_user event test" });
+  return { message: "You have successfully called the Users.add method" };
 };
 
 Service.module("Users", Users);
@@ -173,3 +176,103 @@ Service.module("Orders", function () {
 
 Service.startService({ route: "test/service", port: "4400", host: "localhost" });
 ```
+
+---
+
+# Middleware
+
+SystemLynx supports middleware that runs before or after a module method is called. This is useful for things like authentication, logging, or validating requests.
+
+Use `Service.before` to add a middleware function that runs before any method is invoked, and `Service.after` to run one after the response is ready.
+
+```javascript
+const { Service } = require("systemlynx");
+
+Service.before(function (req, res, next) {
+  console.log("Incoming request:", req.fn);
+  next();
+});
+
+Service.module("Users", Users);
+Service.module("Orders", Orders);
+
+Service.startService({ route: "api", port: 4400, host: "localhost" });
+```
+
+You can also scope middleware to a specific module or method so it only runs where you need it.
+
+```javascript
+// Runs before every method on the Users module
+Service.before("Users", authMiddleware);
+
+// Runs only before Users.delete
+Service.before("Users.delete", requireAdminMiddleware);
+```
+
+See the [API Documentation](https://github.com/Odion100/SystemLynx/blob/master/API.md#tasksjs-api-documentation) for the full middleware API.
+
+---
+
+# Architecture: Monolith to Microservices
+
+One of the core ideas behind SystemLynx is that your module code doesn't change depending on how you deploy it. You can start with everything in a single service and scale it out later — without rewriting anything.
+
+## Start as a monolith
+
+```javascript
+const { Service } = require("systemlynx");
+
+Service.module("Users", Users);
+Service.module("Orders", Orders);
+Service.module("Products", Products);
+
+Service.startService({ route: "api", port: 4400, host: "localhost" });
+```
+
+Your client loads everything from one place:
+
+```javascript
+const { Users, Orders, Products } = await Client.loadService("http://localhost:4400/api");
+```
+
+## Scale out: move modules to their own services
+
+When you're ready to scale, pull any module into its own service. The module code is unchanged — just move it and update the URL the client loads from.
+
+```javascript
+// users-service.js
+Service.module("Users", Users);
+Service.startService({ route: "users", port: 4401, host: "localhost" });
+
+// orders-service.js
+Service.module("Orders", Orders);
+Service.startService({ route: "orders", port: 4402, host: "localhost" });
+```
+
+```javascript
+// client
+const { Users } = await Client.loadService("http://localhost:4401/users");
+const { Orders } = await Client.loadService("http://localhost:4402/orders");
+```
+
+Because every module can be hosted independently, you can scale horizontally (run multiple instances of a service) or vertically (split a busy module into its own service) as your needs grow.
+
+## Load Balancing
+
+The **LoadBalancer** lets you run multiple clones of a service and distribute requests across them. Clones register themselves with the LoadBalancer, which handles routing — your client just talks to the LoadBalancer URL and doesn't need to know how many instances are running behind it.
+
+```javascript
+const { LoadBalancer } = require("systemlynx");
+
+LoadBalancer.startService({ route: "users", port: 4400, host: "localhost" });
+```
+
+Each clone registers on startup:
+
+```javascript
+const { Client } = require("systemlynx");
+
+const { clones } = await Client.loadService("http://localhost:4400/users");
+
+await clones.register({ host: "localhost", port: 4401, route: "/users" });
+```

package/RFCs/001-websocket-named-events-room-scoping.md

@@ -0,0 +1,133 @@
+# RFC: WebSocket Named Events + Room-Based Scoping
+
+## Context
+
+Currently all server-to-client WebSocket traffic flows through a single event named `"dispatch"`. The server uses `socket.emit("dispatch", {id, name, data, type})` on the namespace, which broadcasts to **every connected client** in the namespace. Each client then checks `event.name` and routes or discards client-side. This wastes bandwidth — if 10 clients are connected and only 1 subscribed to `"orderCreated"`, all 10 still receive the packet.
+
+The goal is to use Socket.io's native named events + room-based scoping so that each event is only delivered to clients that actually subscribed to it.
+
+---
+
+## Architecture Change
+
+**Before:**
+```
+Server: namespace.emit("dispatch", { id, name: "orderCreated", data, type })
+          → ALL clients in namespace receive it
+          → each client checks event.name, ignores if not listening
+```
+
+**After:**
+```
+Client subscribes: socket.emit("subscribe", "orderCreated")
+Server joins room: clientSocket.join("orderCreated")
+Server emits:      namespace.to("orderCreated").emit("orderCreated", { id, data, type })
+          → ONLY clients in room "orderCreated" receive it
+```
+
+The user-facing API is **unchanged** — `module.on("eventName", cb)` still works the same.
+
+---
+
+## Files to Change
+
+### 1. `systemlynx/ServerManager/components/SocketEmitter.js`
+
+Two changes:
+- Add a `"connection"` handler on the namespace to manage room membership per connected client
+- Change the emit to target the room instead of broadcasting to all
+
+```javascript
+// Add after creating namespace socket:
+socket.on("connection", (clientSocket) => {
+  clientSocket.on("subscribe", (name) => clientSocket.join(name));
+  clientSocket.on("unsubscribe", (name) => clientSocket.leave(name));
+});
+
+// Change the emit:
+// OLD: socket.emit("dispatch", { id, name, data, type });
+// NEW: socket.to(name).emit(name, { id, data, type });
+```
+
+### 2. `systemlynx/Client/components/SocketDispatcher.js`
+
+Three changes:
+
+**a) Replace `socket.on("dispatch", ...)` with `socket.onAny(...)`** to receive named events. Reconstruct the event object explicitly (no spread — avoids payload fields bleeding into event shape). `name` is kept on the event object since a single callback can handle multiple events and may need to know which fired:
+```javascript
+socket.onAny((name, payload) => {
+  const event = { id: payload.id, name, data: payload.data, type: payload.type };
+  dispatcher.emit(name, payload.data, event);
+});
+```
+
+**b) Track subscription counts and sync with server rooms** by overriding `on`, `once`, `$clearEvent`, and `destroy`. Use a `subscriptionCounts` Map. On first listener for an event → emit `"subscribe"` to server. When last listener removed → emit `"unsubscribe"`. Use a `RESERVED` set for socket.io internal events (`connect`, `disconnect`, `error`, `connect_error`) that must never be treated as subscriptions.
+
+**c) Re-subscribe on reconnect** — inside the `"connect"` handler, re-emit `"subscribe"` for all events currently tracked in `subscriptionCounts`.
+
+Subscription reference-counting sketch:
+```javascript
+const subscriptionCounts = new Map();
+const RESERVED = new Set(["connect", "disconnect", "error", "connect_error"]);
+
+const trackSubscribe = (name) => {
+  const n = (subscriptionCounts.get(name) || 0) + 1;
+  subscriptionCounts.set(name, n);
+  if (n === 1) socket.emit("subscribe", name);
+};
+
+const trackUnsubscribe = (name) => {
+  const n = (subscriptionCounts.get(name) || 0) - 1;
+  if (n <= 0) { subscriptionCounts.delete(name); socket.emit("unsubscribe", name); }
+  else subscriptionCounts.set(name, n);
+};
+```
+
+Override `on`: call original, if not RESERVED → `trackSubscribe`, return unsub that also calls `trackUnsubscribe`.  
+Override `once`: same but auto-`trackUnsubscribe` when the callback fires (use a `done` flag to prevent double-unsubscribe if unsub is called before event fires).  
+Override `$clearEvent`: call original, if tracked → clear count and emit `"unsubscribe"`.  
+Override `destroy`: emit `"unsubscribe"` for all tracked events, clear map, call original.
+
+### 3. `systemlynx/Client/tests/SocketDispatcher.test.js`
+
+Two changes:
+
+**a) Add subscription handling to test server setup** (mirrors what SocketEmitter does internally):
+```javascript
+socket.on("connection", (clientSocket) => {
+  clientSocket.on("subscribe", (name) => clientSocket.join(name));
+  clientSocket.on("unsubscribe", (name) => clientSocket.leave(name));
+});
+```
+
+**b) Change server-side emit in test** from namespace broadcast to room-targeted named event:
+```javascript
+// OLD:
+socket.emit("dispatch", { name: eventName, data: { testPassed: true } })
+// NEW:
+socket.to(eventName).emit(eventName, { id: "test-id", data: { testPassed: true }, type: "WebSocket" })
+```
+
+Update the `event` deep-equal assertion in the `SocketDispatcher.apply()` test to match the new reconstructed shape `{ id, name, data, type }`.
+
+### 4. `/Users/odionedwards/SystemLynx-client/systemlynx/Client/components/SocketDispatcher.mjs`
+
+The `systemlynx-client` package (separate repo at `/Users/odionedwards/SystemLynx-client/`, used by `systemview`) has its own `SocketDispatcher.mjs` that is identical in logic to `SocketDispatcher.js` but uses ES module syntax (`import`/`export default`). Apply the exact same changes — subscription count Map, `on`/`once`/`$clearEvent`/`destroy` overrides, `socket.onAny`, reconnect re-subscription — written with ES module syntax.
+
+---
+
+## Files NOT Changed
+
+- `systemlynx/ServerManager/components/Router.js` — HTTP method calls, unaffected
+- `systemlynx/Client/components/ServiceRequestHandler.js` — HTTP method calls, unaffected
+- `systemlynx/Client/tests/Client.test.js` — event shape `has.all.keys("id", "name", "data", "type")` still matches ✓
+- `systemlynx/App/tests/App.test.js` — event shape not tested here ✓
+
+---
+
+## Verification
+
+1. `npm test` — all existing tests should pass
+2. The `SocketDispatcher.test.js` "emit and handle events" tests confirm room delivery works
+3. The `Client.test.js` "receive events emitted from backend" test verifies end-to-end: server `eventTester.emit(...)` → only subscribed client receives it
+4. Manually: connect two clients to the same service, subscribe one to `"eventA"`, confirm only that client gets the event when the server emits it

package/RFCs/002-module-handle-and-internal-error-events.md

@@ -0,0 +1,135 @@
+# RFC: Module Handle + Internal Error Events
+
+## Context
+
+SystemView is a SystemLynx plugin (installed via `App.use`) that wants to act as the
+observability/logging layer for a SystemLynx service — monitoring requests and errors
+under the hood, and letting modules log through SystemView.
+
+Today the plugin gets `(App, system)` but has no *supported* way to reach the live module
+objects, and SystemLynx emits **no internal events** on the request/error lifecycle. This
+RFC adds the minimal SystemLynx-side surface to enable that. **All SystemView-specific
+behavior (the log function, the monitoring sink) lives in the SystemView plugin, not in
+SystemLynx.** SystemLynx only exposes access + an emit point.
+
+---
+
+## Two gaps, two changes
+
+### Gap 1 — No supported handle on the live modules
+
+A plugin receives `App` and `system`, and live modules already exist at
+`system.modules[i].module` after construction. But:
+
+- Reaching into `system.modules[].module` couples the plugin to internal shape.
+- At `plugin.apply` time the modules are **not built yet** — `system.modules` holds only
+  `{ name, __constructor }`. The live `.module` is assigned later in `loadModules`, right
+  before `App.emit("ready", system)`.
+
+**Change:** Add `App.getModules()` (and `App.getModule(name)`) returning the live module
+objects. Returns them after `ready`; empty/undefined before.
+
+### Gap 2 — Errors bypass the lifecycle
+
+`before`/`after` middleware already let a plugin observe the request happy-path. But every
+error to the client funnels through `res.sendError` in `Router.js`, which writes the HTTP
+response and **does not call `next()`** — so afterware and the response middleware are
+skipped. `after` hooks never see failures.
+
+**Change:** Emit a **local-only** `"error"` event on the module from inside `sendError`.
+The failing client already receives the error over HTTP; this event is purely for
+server-side observers. Use `$emit` (local) **not** `emit` (which would broadcast the error
+over websockets to every connected client).
+
+---
+
+## Files to Change
+
+### 1. `systemlynx/App/App.js`
+
+Add module accessors. The live module is at `system.modules[i].module` once built.
+
+```js
+App.getModule = (name) => {
+  const found = system.modules.find((m) => m.name === name);
+  return found ? found.module : undefined;
+};
+
+App.getModules = () =>
+  system.modules.reduce((obj, { name, module }) => {
+    if (module) obj[name] = module;
+    return obj;
+  }, {});
+```
+
+`getModules()` returns an object **keyed by module name** (`{ [name]: module }`) —
+consistent with how SystemLynx exposes service modules elsewhere (`useService(name)`
+returns named module keys). `getModule(name)` returns a single live module. Both only
+return live modules after the `ready` event (before that, `module` is undefined). Plugins
+should call these inside an `App.on("ready", ...)` handler.
+
+### 2. `systemlynx/ServerManager/components/Router.js`
+
+Emit a local `"error"` event from the single error chokepoint, `sendError` (inside
+`parseRequest`). `req.Module` is the live module dispatcher.
+
+```js
+const sendError = (error) => {
+  const status = (error || {}).status || 500;
+  const message = (error || {}).message || unhandledMessage;
+  if (req.Module && typeof req.Module.$emit === "function")
+    req.Module.$emit("error", {
+      module_name,
+      fn,
+      arguments: req.arguments,
+      status,
+      message,
+      error,
+    });
+  res.status(status).json({ ...presets, ...error, status, message, SystemLynxService: true });
+};
+```
+
+The payload carries the call identity (`module_name` + `fn`) and the inputs
+(`arguments`) so an observer can reconstruct what was called and with what. The raw
+Express `req` is intentionally **not** included — args-only keeps the event lean; if a
+consumer later needs headers we can revisit.
+
+`$emit` is the local-only emitter installed by `SocketEmitter` (see `SocketEmitter.js` —
+`Emitter.$emit = Emitter.emit` before `emit` is overridden to broadcast). Guard with a
+typeof check so a module without SocketEmitter applied doesn't throw.
+
+---
+
+## Files NOT Changed
+
+- `systemlynx/ServerManager/components/SocketEmitter.js` — `$emit` already exists; we only
+  consume it.
+- `systemlynx/Service/Service.js` — module construction unchanged.
+- HTTP response shape — unchanged; the `"error"` event is additive and server-side only.
+
+---
+
+## Plugin-side usage (for reference — lives in SystemView, not this repo)
+
+```js
+App.use((App, system) => {
+  App.on("ready", () => {
+    Object.entries(App.getModules()).forEach(([name, module]) => {
+      module.on("error", (info) => { /* SystemView records the failure */ });
+      module.log = (...args) => { /* SystemView log routing */ };
+    });
+  });
+});
+```
+
+---
+
+## Verification
+
+1. `npm test` — existing suite stays green (changes are additive).
+2. New test: register a module whose method throws, call it over HTTP, assert the module's
+   local `"error"` event fired with `{ module_name, fn, status, message, error }` and that
+   the event did **not** go out over the websocket.
+3. New test: `App.getModule(name)` returns the live module after `ready`, undefined before.
+4. Confirm the failing HTTP response body is unchanged from current behavior.

package/index.test.js

@@ -34,8 +34,10 @@ describe("SystemLynx Objects", () => {
       .that.has.all.keys(
         "module",
         "on",
+        "once",
         "emit",
         "$clearEvent",
+        "destroy",
         "before",
         "after",
         "use",
@@ -44,7 +46,9 @@ describe("SystemLynx Objects", () => {
         "onLoad",
         "config",
         "server",
-        "WebSocket"
+        "WebSocket",
+        "getModule",
+        "getModules"
       )
       .that.respondsTo("module")
       .that.respondsTo("on")
@@ -56,7 +60,9 @@ describe("SystemLynx Objects", () => {
       .that.respondsTo("startService")
       .that.respondsTo("loadService")
       .that.respondsTo("onLoad")
-      .that.respondsTo("config");
+      .that.respondsTo("config")
+      .that.respondsTo("getModule")
+      .that.respondsTo("getModules");
   });
 
   it("should return a SystemLynx Client", () => {
@@ -96,8 +102,10 @@ describe("SystemLynx Objects", () => {
         "before",
         "after",
         "on",
+        "once",
         "emit",
         "$clearEvent",
+        "destroy",
         "clones",
         "register",
         "dispatch",

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "systemlynx",
-  "version": "1.19.12",
+  "version": "1.21.0",
   "description": "",
   "main": "index.js",
   "browser": {

package/systemlynx/App/App.js

@@ -43,6 +43,20 @@ module.exports = function createApp(server, WebSocket, customClient) {
     return App;
   };
 
+  // Live module accessors. Modules are constructed during initialization, so the
+  // `module` reference only exists after the "ready" event — before that these
+  // return undefined / an empty list.
+  App.getModule = (name) => {
+    const found = system.modules.find((mod) => mod.name === name);
+    return found ? found.module : undefined;
+  };
+
+  App.getModules = () =>
+    system.modules.reduce((obj, { name, module }) => {
+      if (module) obj[name] = module;
+      return obj;
+    }, {});
+
   App.before = (...args) => {
     system.Service.before(...args);
     return App;