@kadi.build/core 0.0.1-alpha.0

package/.prettierrc

@@ -0,0 +1,6 @@
+{
+  "printWidth": 80,
+  "singleQuote": true,
+  "trailingComma": "none",
+  "semi": true
+}

package/README.md

@@ -0,0 +1,306 @@
+# @kadi.build/core
+
+---
+
+The `@kadi.build/core` module is a comprehensive toolkit for developers integrating with the KADI infrastructure. This module simplifies tasks such as managing `agent.json` files, spawning processes, interacting with brokers, and handling interprocess communication (IPC).
+
+## Features
+
+- **Agent JSON Management**: Manage configurations for tools, agents, or systems using KADI.
+- **Process Management**: Support launching and managing subprocesses.
+- **Multi-Broker Support**: Dynamic broker selection and management with environment/runtime configuration.
+- **Broker Interaction**: Facilitate communications with WebSocket brokers.
+- **IPC Support**: Tools for interprocess communication across various programming languages.
+
+## Installation
+
+Install `@kadi.build/core` using npm:
+
+```bash
+npm install @kadi.build/core
+```
+
+## Documentation
+
+### Agent JSON Functions
+
+- **`getAbilityJSON(abilityName, abilityVersion)`**: Retrieves the `agent.json` for a specified ability.
+- **`getAbilityJSONPath(abilityName, abilityVersion)`**: Provides the file path to the `agent.json` for a specific ability.
+- **`getAbilityVersionFromArray(abilities, name)`**: Searches ability array provided, and returns the version number for name provided.
+- **`getAbilitiesDir()`**: Returns the directory path where abilities are stored.
+- **`getProjectJSON()`**: Fetches the `agent.json` for the current project.
+- **`getProjectJSONPath()`**: Gets the file path for the project's `agent.json`.
+- **`getKadiCoreJSON()`**: Retrieves the `agent.json` for the Kadi core.
+- **`getKadiCoreJSONPath()`**: Provides the file path to the Kadi core's `agent.json`.
+- **`getKadiJSON()`**: Fetches the `agent.json` for the Kadi system.
+- **`getKadiJSONPath()`**: Returns the file path for the Kadi system's `agent.json`.
+
+### Broker Management Functions
+
+The broker management system allows dynamic selection from multiple configured brokers defined in `agent.json`:
+
+```json
+{
+  "brokers": {
+    "local": "ws://127.0.0.1:8080",
+    "remote": "ws://production.example.com:8080",
+    "staging": "ws://staging.example.com:8080"
+  }
+}
+```
+
+Available functions:
+
+- **`KADI_BROKERS`**: Object containing all configured brokers with their parsed URLs.
+- **`KADI_BROKER_URL`**: Default broker URL (first one defined, for backward compatibility).
+- **`getBrokerUrl(brokerName)`**: Get URL for a specific broker by name. Returns `null` if not found.
+- **`getBrokerNames()`**: Get array of all available broker names.
+- **`setActiveBroker(brokerName)`**: Set the active broker for the session. Returns `true` if successful.
+- **`getActiveBrokerName()`**: Get the name of the currently active broker.
+- **`getActiveBrokerUrl()`**: Get the URL of the currently active broker.
+- **`getDefaultBrokerName()`**: Get the name of the default broker (first one defined).
+- **`selectBrokerFromEnv()`**: Set active broker from `KADI_BROKER` environment variable.
+
+### Process Management Functions
+
+- **`runExecCommand(name, version, command)`**: Executes a command for initializing abilities.
+- **`runSpawnCommand(name, version, command)`**: Uses `spawn` to execute commands for subprocesses.
+
+## Usage Examples
+
+```javascript
+import {
+  getProjectJSON,
+  runExecCommand,
+  IPCManager,
+  Broker
+} from '@kadi.build/core';
+
+async function setupProject() {
+  const projectConfig = getProjectJSON();
+  console.log(projectConfig);
+
+  await runExecCommand('example', '1.0', 'npm install');
+
+  // Broker management examples
+  console.log('Available brokers:', getBrokerNames());
+  console.log('All brokers:', KADI_BROKERS);
+
+  // Set active broker
+  setActiveBroker('remote');
+  console.log('Active broker:', getActiveBrokerUrl());
+
+  // IPC setup
+  const ipc = new IPCManager();
+  ipc.createInstance('python', 'pythonScript.py', 'pythonInstance');
+
+  // Traditional broker setup
+  Broker.addBroker('ws://example.com', 'exampleBroker');
+  let broker = Broker.getBroker('default');
+  console.log(`Connected to ${broker.url}`);
+  broker.send(
+    BrokerMessageBuilder.setup(
+      'TestAgent',
+      'A Broker Testing Agent',
+      null,
+      null
+    )
+  );
+}
+```
+
+### Broker Functions
+
+- **`Broker`**: Manages interactions with broker systems.
+- **`IBroker`**: Broker instance interface with methods to manage its lifecycle and communications.
+- **`BrokerMessageBuilder`**: Assists in constructing messages for broker communication.
+
+## Broker Functions
+
+The Broker system facilitates interaction between different agents and services within the Kadi environment.
+
+### `Broker` Interface
+
+- **`addBroker(url, name = 'default')`**: Adds a new broker connection. If a broker with the same name exists, it will be reused.
+- **`disconnect(name = 'default')`**: Disconnects the broker specified by the name.
+- **`deleteBroker(name = 'default')`**: Deletes the broker specified by the name and closes its connection.
+- **`send(message, brokerName = 'default')`**: Sends a message through the broker specified by the name.
+- **`addEventListener(event, listener, brokerName = 'default')`**: Adds an event listener to the specified broker.
+- **`removeEventListener(event, listener, brokerName = 'default')`**: Removes an event listener from the specified broker.
+- **`removeAllListeners(brokerName = 'default')`**: Removes all event listeners from the specified broker.
+- **`getBroker(brokerName = 'default')`**: Retrieves a broker instance by name.
+
+### `IBroker` Instance
+
+This class extends `EventEmitter` and manages individual broker connections:
+
+- **`constructor(url, name)`**: Initializes a new broker connection.
+- **`send(message)`**: Sends a message through the WebSocket connection.
+- **`getConnectedAgents()`**: Retrieves a list of currently connected agents.
+
+### `BrokerMessageBuilder`
+
+Utility class for constructing broker messages:
+
+- **`create_message(type, data)`**: Creates a JSON string with the specified type and data.
+- **`message(to, content)`**: Constructs a message directed to a specific peer.
+- **`setup(name, description, limit, uuid)`**: Creates a setup message for registering the broker.
+- **`suspend()`**: Constructs a suspend message.
+- **`finish()`**: Constructs a finish message.
+- **`list()`**: Constructs a list message to request a list of connected agents.
+
+### IPC Functions
+
+- **`IPCMessageBuilder`**: Constructs messages for IPC interactions.
+- **`IPCManager`**: Manages setup and lifecycle of IPC connections.
+- **`IAbilityIPC`**: Represents an IPC ability instance with methods to manage its lifecycle and communications.
+
+### `IPCManager`
+
+Manages the lifecycle and setup of interprocess communications.
+
+- **`createInstance(language, commandString, name = 'default')`**: Creates and manages a new IPC instance for the specified language.
+- **`getInstance(name)`**: Retrieves an existing IPC instance by name.
+- **`shutdownInstance(name)`**: Shuts down an existing IPC instance by name.
+
+### `IAbilityIPC` Instance
+
+Manages an individual interprocess communication instance:
+
+- **`start()`**: Starts the child process associated with this IPC instance.
+- **`launch()`**: Sends the launch command to the associated process.
+- **`sendMessage(message)`**: Sends a JSON-formatted message to the child process.
+- **`shutdown()`**: Sends a shutdown command to the child process and terminates it.
+
+### `IPCMessageBuilder`
+
+Utility class for constructing IPC messages:
+
+- **`createMessage(type, contents)`**: Creates a JSON string with the specified type and contents.
+- **`launch(commandString)`**: Constructs a launch message with the specified command string.
+- **`shutdown()`**: Constructs a shutdown message.
+- **`message(content)`**: Constructs a general message with the specified content.
+
+## Usage Examples
+
+Here's how you might use the IPC and Broker functionalities:
+
+### Interacting with IPC
+
+```javascript
+console.log('Starting IPC Ability Interface test...');
+
+// Create Python IPC instance
+const pythonIPC = IPCManager.createInstance(
+  'python',
+  'echo "Hello, World!"',
+  'pythonTest'
+);
+
+pythonIPC.on('error', (error) => {
+  console.error('Handled Python Error:', error);
+  // Consider appropriate error handling or recovery actions here
+});
+
+pythonIPC.on('critical-error', (error) => {
+  console.error('Critical Error from Python:', error);
+  // Handle critical errors, potentially restarting the process or alerting administrators
+});
+
+pythonIPC.on('message', (msg) => {
+  console.log('Message from Python:', msg);
+});
+
+pythonIPC.on('broker', (msg) => {
+  console.log('Broker message from Python:', msg);
+});
+
+pythonIPC.on('shutdown', () => {
+  console.log('Python IPC has shut down.');
+});
+
+Broker.addBroker('http://your.broker.com', 'default');
+let brk = Broker.getBroker('default');
+broker = brk;
+
+pythonIPC.attachBroker(brk);
+
+brk.on('open', function open() {
+  console.log(`Connected to ${brk.url}`);
+  const jsonObject = BrokerMessageBuilder.setup(
+    'TestAgent',
+    'A Broker Testing Agent',
+    null,
+    null
+  );
+  brk.send(jsonObject);
+});
+
+brk.on('message', function incoming(data) {
+  let msg = JSON.parse(data);
+  if (msg.type == 'setup') {
+    (async () => {
+      try {
+        await GetLLMProxy();
+      } catch (error) {
+        console.error('Error during GetLLMProxy:', error.message);
+      }
+    })();
+  }
+  // console.log('Received:', formatJSON(data));
+});
+
+brk.on('error', function error(error) {
+  console.error('WebSocket error:', error);
+});
+
+brk.on('close', function close() {
+  console.log('Disconnected from the server');
+  process.exit(0); // Exit the process when the WebSocket connection is closed
+});
+
+// Send launch command
+
+//Utilize log files
+// pythonIPC.sendMessage(IPCMessageBuilder.launch(pythonIPC.commandString, process.cwd() + "/logs"));
+
+//No log files created
+pythonIPC.sendMessage(IPCMessageBuilder.launch(pythonIPC.commandString));
+```
+
+### Handling IPC
+
+```javascript
+const ipcManager = new IPCManager();
+const ipcInstance = ipcManager.createInstance('node', 'app.js', 'nodeApp');
+ipcInstance.on('message', (message) => console.log(message));
+```
+
+---
+
+## TODO:
+
+- Once @kadi.build/core and dependencies are done, @kadi.build/cli (and all commands), need to be rewritten, using the @kadi.build/core module. this will be v1.0 release
+
+* [ ] add getConnectedAgents to the Broker, it should call the IBroker method
+
+* [x] add key to IBroker object
+* [ ] IBroker on 'message' should listen for setup event, and update teh uuid and key for the IBroker object
+
+* [ ] Extract Broker and IPC to their own node module and host on npm
+* [x] Publish @kadi.build/core to npm
+* [x] Setup of python environment needs to happen once an IPC Inteface is created
+      This include setting up the virtual environment and installing the required packages
+* [x] Add method to IPCManger to set IPCInterfaces from agent.json. If IPCManger has empty list, it thorws an error that no IAbilityIPC's can be created. IPCManger passes the proper interface command to IAbilityIPC on creation from this list, IAbilityIPC does not need access directly to agent.json. Controlling app can update this list anytime a new IAbilityInterface is created, allowing the agent.json to be updated during runtime.
+* [x] Need to adapt ipc.js to read from the @kadi.build/core folder when trying to open IPC instances, instead of project folder. @kadi.build/core should be in the npm_modules folder (when deployed)
+* [x] The test code needs to fully test the IPCManger interface, and not just the IPCAbility
+* [x] If node fails, we need to make sure python IPC interface and child processes also close out.
+* [x] Add buffer to error handeling in IAbilityIPC, mimic same process used in std input
+* [x] Verify if Python IAbilityIPC needs to utilize buffering, and node passing '\n' at end of jso
+* [x] Setup passthrough for broker messages to/from Language IPC interface
+* [x] Add broker message builder/parser in python IAbilityIPC
+* [x] Update BrokerMessageBuilder to use same format at IPCMessageBuilder
+* [x] Update Broker so it extends EventEmitter, rather than have a property of event emitter (similar to IAbilityIPC)
+* [x] update python IAbility so that it creates a timestamped command_output.log file, new file for each run, in the project root directory... not the @kadi.build/core directory
+* [x] Comment out debug messages
+* [x] Add ability to allow for launched process to be piped to event handler (launch command should default to process, but allow user to set flag that will save all outputs to file in the event handler)

package/agent.json

@@ -0,0 +1,18 @@
+{
+  "name": "kadi-core",
+  "version": "0.0.1",
+  "license": "MIT",
+  "description": "base install ability for kadi",
+  "repo": "https://gitlab.com/humin-game-lab/agent-abilities/kadi-core.git",
+  "lib": "https://gitlab.com/humin-game-lab/agent-abilities/kadi-core/-/archive/v0.0.1/kadi-core-v0.0.1.zip",
+  "abilities": {},
+
+  "scripts": {
+    "preflight": "echo 'kadi-core v0.0.1 is starting...'",
+    "setup": "echo 'kadi-core v0.0.1 is ready...'",
+    "start": "npm install",
+    "stop": "echo 'kadi-core v0.0.1 is ready...'",
+    "python": "python3 ./node_modules/kadi-core/ipcInterfaces/pythonAbilityIPC.py",
+    "rust": "cargo run --bin path/to/rustAbilityIPC"
+  }
+}

package/broker.js

@@ -0,0 +1,214 @@
+import { EventEmitter } from 'events';
+import { WebSocket } from 'ws';
+import { randomUUID } from 'crypto';
+
+function formatJSON(jsonString) {
+  try {
+    const jsonObj = JSON.parse(jsonString);
+    return JSON.stringify(jsonObj, null, 2);
+  } catch (error) {
+    return `Error parsing JSON: ${error}`;
+  }
+}
+
+//List Message
+// {
+//     "type": "list",
+//     "data": {
+//       "clients": [
+//         {
+//           "id": "9143a10c-5b4f-11ef-bb3a-2250bcde1118",
+//           "name": "TestAgent",
+//           "description": "A Broker Testing Agent"
+//         }
+//       ]
+//     }
+//   }
+
+class IBroker extends EventEmitter {
+  constructor(url, name) {
+    super();
+    this.url = url;
+    this.name = name;
+    this.uuid = null; //public id for the broker
+    this.key = null; //private id for the broker
+    this.queryPending = null; //used for Syncronous queries, disables forwarding to onMessage
+    this.ws = new WebSocket(url);
+
+    this.ws.on('open', () => {
+      this.emit('open');
+      console.log(`Connected to ${this.url}`);
+    });
+
+    this.ws.on('message', (data) => {
+      let msg = JSON.parse(data);
+
+      //Emit messages to listeners
+      if (msg.type != this.queryPending) {
+        //A syncronous query is pending
+        this.emit('message', data);
+      } else {
+        // Emit synQuery to listener, to proces event
+        this.queryPending = null;
+        this.emit('syncQuery', data);
+      }
+
+      // console.log('Received:', data);
+    });
+
+    this.ws.on('error', (error) => {
+      this.emit('error', error);
+      console.error('WebSocket error:', error);
+    });
+
+    this.ws.on('close', () => {
+      this.emit('close');
+      console.log('Disconnected from the server');
+    });
+  }
+
+  getConnectedAgents() {
+    return new Promise((resolve, reject) => {
+      // Set the query pending to list
+      this.queryPending = 'list';
+
+      // Set up a timeout to reject if no response is received in 10 seconds
+      const timeout = setTimeout(() => {
+        reject(new Error('Get Connected Agents timed out after 10 seconds'));
+      }, 10000);
+
+      const handleListRequest = (data) => {
+        const msg = JSON.parse(data);
+
+        if (msg.type === 'list') {
+          clearTimeout(timeout);
+          this.off('syncQuery', handleListRequest);
+          resolve(msg.data.clients);
+        }
+      };
+
+      const jsonObject = BrokerMessageBuilder.list();
+      this.send(jsonObject);
+      // console.log("Requested List of Agents");
+      this.on('syncQuery', handleListRequest);
+    });
+  }
+
+  send(message) {
+    // console.log("WS Ready State: ", this.ws.readyState)
+    // console.log("WS OPEN: ", WebSocket.OPEN)
+    if (this.ws.readyState === WebSocket.OPEN) {
+      // console.log("Sending: ", JSON.stringify(message, null, 2));
+      this.ws.send(message);
+      // console.log(`Sending message to ${this.url}: ${message}`);
+    } else {
+      // console.log('WebSocket is not open.');
+    }
+  }
+}
+
+let brokers = new Map();
+
+export const Broker = {
+  addBroker: function (url, name = 'default') {
+    if (!brokers.has(name)) {
+      let broker = new IBroker(url, name);
+      brokers.set(name, broker);
+    } else {
+      console.log(`Broker ${name} already exists.`);
+    }
+  },
+
+  disconnect: function (name = 'default') {
+    let broker = brokers.get(name);
+    if (broker) {
+      broker.ws.close();
+    } else {
+      console.error(`Broker ${name} not found`);
+    }
+  },
+
+  deleteBroker: function (name = 'default') {
+    let broker = brokers.get(name);
+    if (broker) {
+      broker.ws.close();
+      brokers.delete(name);
+    } else {
+      console.error(`Broker ${name} not found`);
+    }
+  },
+
+  send: function (message, brokerName = 'default') {
+    let broker = brokers.get(brokerName);
+    if (broker) {
+      broker.send(message);
+    } else {
+      console.error(`Broker ${brokerName} not found`);
+    }
+  },
+
+  addEventListener: function (event, listener, brokerName = 'default') {
+    let broker = brokers.get(brokerName);
+    if (broker) {
+      broker.on(event, listener);
+    } else {
+      console.error(`Broker ${brokerName} not found`);
+    }
+  },
+
+  removeEventListener: function (event, listener, brokerName = 'default') {
+    let broker = brokers.get(brokerName);
+    if (broker) {
+      broker.off(event, listener);
+    } else {
+      console.error(`Broker ${brokerName} not found`);
+    }
+  },
+
+  removeAllListeners: function (brokerName = 'default') {
+    let broker = brokers.get(brokerName);
+    if (broker) {
+      broker.removeAllListeners();
+    } else {
+      console.error(`Broker ${brokerName} not found`);
+    }
+  },
+
+  getBroker: function (brokerName = 'default') {
+    return brokers.get(brokerName);
+  }
+};
+
+//Need to add a message that will return a list of agents connected
+//  for a specific name/type
+export const BrokerMessageBuilder = {
+  create_message: function (type, data) {
+    return JSON.stringify({
+      type: type,
+      data: data
+    });
+  },
+  message: function (to, data) {
+    return BrokerMessageBuilder.create_message('sendmessage', {
+      peer: to,
+      data: data
+    });
+  },
+  setup: function (name, description, limit, uuid) {
+    return BrokerMessageBuilder.create_message('setup', {
+      name: name,
+      description: description,
+      limit: limit,
+      key: uuid
+    });
+  },
+  suspend: function () {
+    return BrokerMessageBuilder.create_message('suspend', {});
+  },
+  finish: function () {
+    return BrokerMessageBuilder.create_message('finish', {});
+  },
+  list: function () {
+    return BrokerMessageBuilder.create_message('list', {});
+  }
+};

package/index.js

@@ -0,0 +1,370 @@
+import path from 'path';
+import fs from 'fs';
+// import { promises as fsPromises } from 'fs';
+import { fileURLToPath } from 'url';
+import { exec, execSync, spawn } from 'child_process';
+import { Broker, BrokerMessageBuilder } from './broker.js';
+import { IPCManager, IPCMessageBuilder } from './ipc.js';
+import { createRequire } from 'module'; // Import createRequire to use require in ES Modules
+
+// Create a require function scoped to the current module
+const require = createRequire(import.meta.url);
+
+function resolveKadiExecPath() {
+  const command = process.platform === 'win32' ? 'where kadi' : 'which kadi';
+
+  try {
+    const kadiPath = execSync(command, { encoding: 'utf8' }).trim();
+    if (kadiPath) {
+      return kadiPath;
+    } else {
+      throw new Error('Kadi CLI tool not found in the system PATH');
+    }
+  } catch (error) {
+    console.error('Error resolving Kadi path:', error);
+    return null;
+  }
+}
+
+function resolveKadiInstallPath() {
+  try {
+    // Step 1: Get the path to the kadi binary
+    const command = process.platform === 'win32' ? 'where kadi' : 'which kadi';
+    const kadiBinaryPath = execSync(command, { encoding: 'utf8' }).trim();
+
+    if (!kadiBinaryPath) {
+      throw new Error('Kadi CLI tool not found in the system PATH');
+    }
+    // console.log('Resolved Kadi binary path:', kadiBinaryPath);
+
+    // Step 2: Resolve the symlink to find the actual location of the binary
+    const resolvedBinaryPath = fs.realpathSync(kadiBinaryPath);
+    // console.log('Resolved binary real path:', resolvedBinaryPath);
+
+    // Step 3: Correctly determine the root directory of the kadi package
+    const kadiDir = path.dirname(resolvedBinaryPath); // Move up one level to get the kadi directory
+    // console.log('Kadi directory:', kadiDir);
+
+    // Step 4: Verify if this directory contains the kadi package.json
+    const packageJsonPath = path.join(kadiDir, 'package.json');
+    // console.log('Checking for package.json at:', packageJsonPath);
+
+    if (fs.existsSync(packageJsonPath)) {
+      const packageJson = JSON.parse(fs.readFileSync(packageJsonPath, 'utf8'));
+
+      if (packageJson.name === 'kadi') {
+        // console.log('Kadi package.json found:', packageJsonPath);
+        return kadiDir;
+      } else {
+        throw new Error('Package.json found, but it is not for kadi.');
+      }
+    } else {
+      throw new Error('Kadi package.json not found in the resolved path.');
+    }
+  } catch (error) {
+    console.error('Error resolving Kadi CLI install path:', error);
+    return null;
+  }
+}
+
+const __filename = fileURLToPath(import.meta.url);
+const __dirname = path.dirname(__filename);
+
+const kadiExecDir = resolveKadiExecPath();
+const kadiRootDir = resolveKadiInstallPath(); //path.join(kadiCLIInstallPath);
+const kadiAgentPath = path.join(kadiRootDir, 'agent.json');
+
+const rootDir = process.cwd();
+const abilitiesDir = path.join(rootDir, 'abilities');
+const projectAgentPath = path.join(rootDir, 'agent.json');
+const kadiCoreAgentPath = path.join(__dirname, 'agent.json');
+
+//Gets the agent.json for the ability, so dependencies can be installed
+export function getAbilityJSON(abilityName, abilityVersion) {
+  //Get the agent.json for the ability
+  let abilityPath = path.join(
+    rootDir,
+    'abilities',
+    abilityName,
+    abilityVersion,
+    'agent.json'
+  );
+  let abilityJSON = fs.readFileSync(abilityPath, 'utf8');
+  abilityJSON = JSON.parse(abilityJSON);
+  return abilityJSON;
+}
+
+//Gets the file path for the ability agent.json
+export function getAbilityJSONPath(abilityName, abilityVersion) {
+  return path.join(
+    rootDir,
+    'abilities',
+    abilityName,
+    abilityVersion,
+    'agent.json'
+  );
+}
+
+//Gets the abilities directory
+export function getAbilitiesDir() {
+  return abilitiesDir;
+}
+
+//Gets the agent.json for the ability, so dependencies can be installed
+export function getProjectJSON() {
+  //Get the agent.json for kadi
+  let projectJSON = fs.readFileSync(projectAgentPath, 'utf8');
+  projectJSON = JSON.parse(projectJSON);
+  return projectJSON;
+}
+
+//Get the file path for the project agent.json
+export function getProjectJSONPath() {
+  return projectAgentPath;
+}
+
+//Get Kadi Core agent.json
+export function getKadiCoreJSON() {
+  //Get the agent.json for kadi
+  let kadiCoreJSON = fs.readFileSync(kadiCoreAgentPath, 'utf8');
+  kadiCoreJSON = JSON.parse(kadiCoreJSON);
+  return kadiCoreJSON;
+}
+
+//Get the file path for the KADI Core Agent.json
+export function getKadiCoreJSONPath() {
+  return kadiCoreAgentPath;
+}
+
+//Gets the agent.json for KADI system
+export function getKadiJSON() {
+  //Get the agent.json for kadi
+  let kadiJSON = fs.readFileSync(kadiAgentPath, 'utf8');
+  kadiJSON = JSON.parse(kadiJSON);
+  return kadiJSON;
+}
+
+//Gets the file path for the KADI Agent.json
+export function getKadiJSONPath() {
+  return kadiAgentPath;
+}
+
+//Get the exec path of the KADI system
+export function getKadiExecPath() {
+  return kadiExecDir;
+}
+
+//Get the Install directory of KADI system
+export function getKadiInstallPath() {
+  return kadiRootDir;
+}
+
+//Save agent.json file
+export async function saveAgentJSON(agentJSON, agentJSONPath) {
+  // Convert the JSON object to a string with pretty-print formatting
+  const jsonString = JSON.stringify(agentJSON, null, 2);
+
+  // Write the JSON string to the specified file
+  fs.writeFile(agentJSONPath, jsonString, 'utf8', (err) => {
+    if (err) {
+      console.error('Error saving JSON to file:', err);
+    }
+    // else {
+    //     console.log('JSON saved successfully to', filePath);
+    // }
+  });
+}
+
+export function getAbilityVersionFromArray(abilities, name) {
+  const ability = abilities.find((ability) => ability.name === name);
+  if (ability) {
+    return ability.version;
+  } else {
+    return null;
+  }
+}
+
+// Get the API URL from the agent.json
+let kadijson = getKadiJSON();
+
+// Define the API URL (Adjust according to your API)
+export const KADI_API_URL = kadijson.api;
+export const SEARCH_API_URL = kadijson.api + '/search';
+export const GET_API_URL = kadijson.api + '/get';
+
+// Export all broker URLs
+const brokers = {};
+for (const [name, url] of Object.entries(kadijson.brokers)) {
+  const brokerUrl = new URL(url);
+  brokers[name] =
+    `${brokerUrl.hostname}:${brokerUrl.port}${brokerUrl.pathname}`;
+}
+
+export const KADI_BROKERS = brokers;
+
+// Get default broker (first one defined)
+const defaultBrokerName = Object.keys(kadijson.brokers)[0];
+export const KADI_BROKER_URL = brokers[defaultBrokerName]; // Backward compatibility
+
+// Utility functions
+export function getBrokerUrl(brokerName) {
+  return brokers[brokerName] || null;
+}
+
+export function getBrokerNames() {
+  return Object.keys(brokers);
+}
+
+export function getDefaultBrokerName() {
+  return defaultBrokerName;
+}
+
+// Active broker selection
+let activeBrokerName = defaultBrokerName;
+
+export function setActiveBroker(brokerName) {
+  if (brokers[brokerName]) {
+    activeBrokerName = brokerName;
+    return true;
+  }
+  return false;
+}
+
+export function getActiveBrokerName() {
+  return activeBrokerName;
+}
+
+export function getActiveBrokerUrl() {
+  return brokers[activeBrokerName];
+}
+
+//Runs a command line string, used for abilities init functions
+export async function runExecCommand(name, version, command) {
+  let execDirectory;
+  if (name === '' || version === '') {
+    execDirectory = path.join(rootDir);
+  } else {
+    execDirectory = path.join(rootDir, 'abilities', name, version);
+  }
+  console.log('execDirectory: ', execDirectory);
+  return new Promise((resolve, reject) => {
+    exec(
+      command,
+      { cwd: execDirectory, maxBuffer: 1024 * 1024 },
+      (error, stdout, stderr) => {
+        if (error) {
+          console.error(`exec error: ${error}`);
+          reject(error); // Reject the promise on error
+          return;
+        }
+        if (stderr.trim()) {
+          console.error(stderr);
+        }
+        resolve(stdout.trim()); // Resolve the promise when exec completes successfully
+      }
+    );
+  });
+}
+
+export async function runSpawnCommand(name, version, command) {
+  let execDirectory;
+  if (name === '' || version === '') {
+    execDirectory = path.join(rootDir);
+  } else {
+    execDirectory = path.join(rootDir, 'abilities', name, version);
+  }
+  console.log('execDirectory: ', execDirectory);
+  return new Promise((resolve, reject) => {
+    const child = spawn(command, {
+      cwd: execDirectory,
+      stdio: 'inherit', // This pipes the child process's stdio to the parent
+      shell: true // This ensures that the command is run within a shell
+    });
+
+    child.on('error', (error) => {
+      console.error(`Error: ${error}`);
+      reject(error);
+    });
+
+    child.on('close', (code) => {
+      if (code !== 0) {
+        console.error(`Process exited with code: ${code}`);
+        reject(new Error(`Process exited with code: ${code}`));
+      } else {
+        resolve();
+      }
+    });
+  });
+}
+
+// Function to find the version of an ability by name
+export function findAbilityVersionByName(abilities, name) {
+  const ability = abilities.find((ability) => ability.name === name);
+  if (ability) {
+    return ability.version;
+  } else {
+    return 'No ability found with the specified name.';
+  }
+}
+
+export async function loadAbility(abilityName) {
+  let agentJSON = await getProjectJSON();
+  let abilityConfig = agentJSON.abilities.find(
+    (ability) => ability.name === abilityName
+  );
+  if (!abilityConfig) {
+    //   throw new Error(`Ability ${abilityName} not found in configuration`);
+    console.error(`Ability ${abilityName} not found in Project configuration`);
+    const filePath = __filename;
+    console.log('filePath:', filePath);
+    // Extract the directory path of the file
+    const dirPath = path.dirname(filePath);
+    console.log(' dirPath:', dirPath);
+
+    // Split the directory path into an array of directories
+    const pathParts = dirPath.split(path.sep);
+
+    // Find the index of the "abilities" folder
+    const abilitiesIndex = pathParts.indexOf('abilities');
+
+    // Ensure that the folder above 'phone-ability' is 'abilities'
+    if (abilitiesIndex !== -1 && pathParts.length > abilitiesIndex + 2) {
+      const ability = pathParts[abilitiesIndex + 1]; // Extract 'phone-ability'
+      const version = pathParts[abilitiesIndex + 2]; // Extract '0.0.1'
+
+      console.log('Loading ability dependency from an ability');
+      agentJSON = await getAbilityJSON(ability, version);
+      abilityConfig = agentJSON.abilities.find(
+        (ability) => ability.name === abilityName
+      );
+      if (!abilityConfig) {
+        throw new Error(
+          `Ability ${abilityName} not found in ${ability} : ${version} configuration`
+        );
+      }
+    } else {
+      throw new Error('Ability not found in Project agent.json');
+    }
+  }
+  const moduleJSON = getAbilityJSON(abilityName, abilityConfig.version);
+  const modulePath = path.join(
+    rootDir,
+    `abilities/${abilityName}/${abilityConfig.version}/${moduleJSON.entry}`
+  );
+  const module = await import(modulePath);
+  return module.default || module; // Return the default export directly
+}
+
+export async function launchAbility(language, path = '') {}
+
+export { Broker };
+
+export { BrokerMessageBuilder };
+
+export { IPCManager };
+
+export { IPCMessageBuilder };
+
+//Load the IPC interfaces
+let kadiCoreJSON = getKadiCoreJSON();
+IPCManager.setIPCInterfaces(kadiCoreJSON.IPCInterfaces);

package/ipc.js

@@ -0,0 +1,220 @@
+import { EventEmitter } from 'events';
+import { randomUUID } from 'crypto';
+import { spawn } from 'child_process';
+import fs from 'fs';
+import { log } from 'console';
+
+//IPC JSON Message Schema
+// {
+//     type: 'launch' | 'shutdown' | 'message',
+//     contents: any,
+//     time: string
+// }
+
+export const IPCMessageBuilder = {
+  createMessage: function (type, contents) {
+    return JSON.stringify({
+      type: type,
+      contents: contents,
+      time: new Date().toISOString()
+    });
+  },
+
+  launch: function (commandString, logfilepath = null) {
+    return IPCMessageBuilder.createMessage('launch', {
+      command: commandString,
+      logfilepath: logfilepath
+    });
+  },
+
+  shutdown: function () {
+    return IPCMessageBuilder.createMessage('shutdown', {});
+  },
+
+  message: function (content) {
+    return IPCMessageBuilder.createMessage('message', content);
+  },
+
+  broker: function (content) {
+    return IPCMessageBuilder.createMessage('broker', content);
+  }
+};
+
+export const IPCManager = {
+  instances: new Map(),
+  IPCInterfaces: {},
+  createInstance: function (language, commandString, name = 'default') {
+    if (!IPCManager.IPCInterfaces[language]) {
+      throw new Error('Configuration for this language is not provided');
+    }
+
+    const instance = new IAbilityIPC(
+      language,
+      commandString,
+      IPCManager.IPCInterfaces,
+      name
+    );
+    this.instances.set(name, instance);
+
+    instance.on('shutdown', () => {
+      this.instances.delete(name);
+      console.log(`Instance ${name} removed after shutdown.`);
+    });
+
+    return instance;
+  },
+
+  getInstance: function (name) {
+    return this.instances.get(name);
+  },
+
+  shutdownInstance: function (name) {
+    const instance = this.instances.get(name);
+    if (instance) {
+      instance.shutdown();
+    }
+  },
+
+  setIPCInterfaces: function (interfaces) {
+    IPCManager.IPCInterfaces = interfaces;
+  }
+};
+
+class IAbilityIPC extends EventEmitter {
+  constructor(language, commandString, interfaces, name) {
+    super();
+    this.language = language;
+    this.process = null; // Process is initially null
+    this.commandString = commandString;
+    this.name = name;
+    this.input_buffer = '';
+    this.error_buffer = '';
+    this._broker = null; // Broker used for proxying messages
+
+    // Read configurations from agent.json
+    this.IPCInterfaces = interfaces;
+    this.ipcStartupCommand = this.IPCInterfaces[language]; // Command to start the language IPC
+
+    if (!this.ipcStartupCommand) {
+      throw new Error('Unsupported language or missing command configuration');
+    } else {
+      console.log('IPC Startup Command: ', this.ipcStartupCommand);
+      this.start();
+    }
+  }
+
+  start() {
+    // Starts the child process that handles IPC
+    this.process = spawn(this.ipcStartupCommand, {
+      shell: true
+    });
+
+    this.process.stdout.on('data', (data) => {
+      this.input_buffer += data.toString();
+      let boundary = this.input_buffer.indexOf('\n');
+      while (boundary !== -1) {
+        let message = this.input_buffer.substring(0, boundary).trim();
+        this.input_buffer = this.input_buffer.substring(boundary + 1);
+        if (message) {
+          try {
+            let msg = JSON.parse(message);
+            if (msg.type === 'shutdown') {
+              this.emit('shutdown');
+              if (this._broker) {
+                this._broker.ws.close();
+              }
+            } else if (msg.type === 'broker') {
+              this.emit('broker', msg);
+            } else {
+              this.emit('message', msg);
+            }
+            // this.emit('message', JSON.parse(message));
+          } catch (error) {
+            console.error('Error parsing JSON:', error.message);
+            console.error('Raw data received:', message);
+          }
+        }
+        boundary = this.input_buffer.indexOf('\n');
+      }
+    });
+
+    this.process.stderr.on('data', (data) => {
+      this.error_buffer += data.toString();
+      let boundary = this.error_buffer.indexOf('\n');
+      while (boundary !== -1) {
+        let message = this.error_buffer.substring(0, boundary).trim();
+        this.error_buffer = this.error_buffer.substring(boundary + 1);
+        if (message) {
+          try {
+            this.emit('error', JSON.parse(message));
+          } catch (error) {
+            console.error('Error parsing JSON:', error.message);
+            console.error('Raw data received:', message);
+          }
+        }
+        boundary = this.error_buffer.indexOf('\n');
+      }
+    });
+
+    this.process.on('exit', (code) => {
+      this.emit('exit', code);
+      // this.emit('shutdown', this);  // Emitting a shutdown event for the manager to catch
+    });
+
+    this.process.on('error', (error) => {
+      this.emit('error', error);
+    });
+
+    this.on('broker', (msg) => {
+      if (this._broker) {
+        this._broker.send(JSON.stringify(msg.contents));
+      } else {
+        console.error('No broker available to forward message to.');
+      }
+    });
+
+    this.on('message', (msg) => {
+      // console.log('Message received:', msg);
+    });
+  }
+
+  launch(logfilepath = null) {
+    // Send the command string to the language-specific IPC to launch the actual tool
+    let msg = IPCMessageBuilder.launch(this.commandString, logfilepath);
+    console.log('Sending launch message: ', msg);
+    this.sendMessage(msg);
+  }
+
+  sendMessage(message) {
+    if (this.process && this.process.stdin.writable) {
+      this.process.stdin.write(JSON.stringify(message) + '\n');
+    } else {
+      // throw new Error("Process is not running or stdin is not writable.");
+    }
+  }
+
+  shutdown() {
+    this.sendMessage(IPCMessageBuilder.shutdown());
+  }
+
+  attachBroker(broker) {
+    this._broker = broker;
+
+    this._broker.on('message', (data) => {
+      let msg = JSON.parse(data);
+      console.log('IPC Broker Message Recieved: ', msg);
+      this.sendMessage(IPCMessageBuilder.broker(msg));
+    });
+
+    this._broker.on('error', (error) => {
+      let msg = JSON.parse(error);
+
+      this.sendMessage(IPCMessageBuilder.broker(msg));
+    });
+
+    this._broker.on('close', () => {
+      this.shutdown();
+      // process.exit(0); // Exit the process when the WebSocket connection is closed
+    });
+  }
+}

package/ipcInterfaces/pythonAbilityIPC.py

@@ -0,0 +1,177 @@
+import sys
+import json
+import subprocess
+import select
+import threading
+import os
+from datetime import datetime, timezone
+
+class MessageBuilder:
+    @staticmethod
+    def create_message(type, contents):
+        return json.dumps({
+            "type": type,
+            "contents": contents,
+            "time": datetime.now(timezone.utc).isoformat()
+        })
+
+    @staticmethod
+    def shutdown():
+        return MessageBuilder.create_message('shutdown', {})
+
+    @staticmethod
+    def message(contents):
+        return MessageBuilder.create_message('message', contents)
+    
+    @staticmethod
+    def broker(contents):
+        return MessageBuilder.create_message('broker', contents)
+
+def handle_process_output(ipc_instance, output):
+    """Developer-customizable function to handle stdout data."""
+    # print("Process Output:", output.strip())
+    # ipc_instance.send(MessageBuilder.message({"status": "stdout", "contents": output}))
+
+def handle_process_errors(ipc_instance, output):
+    """Developer-customizable function to handle stderr data."""
+    # print("Process Error:", output.strip())
+    ipc_instance.send(MessageBuilder.message({"status": "stderr", "contents": output}))
+
+def handle_broker(ipc_instance, contents):
+    """Developer-customizable function to handle custom Broker messages."""
+    # ipc_instance.send(MessageBuilder.broker({"status": "message_received", "contents": contents}))
+
+def handle_message(ipc_instance, contents):
+    """Developer-customizable function to handle custom IPC messages."""
+    # Placeholder implementation that simply prints the message
+    # print(f"Received custom message: {contents}")
+    # ipc_instance.send(MessageBuilder.message({"status": "message_received", "contents": contents}))
+    # Developers can add their own logic here depending on their requirements
+
+class PythonAbilityIPC:
+    def __init__(self):
+        self.running = True
+        self.child_process = None
+        self.logfile = False
+
+    def process_stream(self, stream, handler):
+        """Generic method to process stdout or stderr from child process."""
+        for line in iter(stream.readline, ''):
+            handler(line)
+        stream.close()
+
+    def read_input(self):
+        """Read input lines from stdin and process each as a JSON message."""
+        while self.running:
+            try:
+                readable, _, _ = select.select([sys.stdin], [], [], 0.1)
+                if readable:
+                    line = sys.stdin.readline()
+                    if line == '':
+                        print("Node.js process has terminated or closed the pipe.")
+                        self.shutdown()
+                        break
+                    if line.strip():
+                        try:
+                            message = json.loads(line)
+                            # self.send(MessageBuilder.message({"debug": "Parsing Message", "message": message}))
+                            self.parse_message(message)
+                        except json.JSONDecodeError as e:
+                            self.send_error(f"JSON decoding failed: {str(e)}")
+            except Exception as e:
+                self.send_error(f"An unexpected error occurred: {str(e)}")
+                self.shutdown()
+                break
+
+    def parse_message(self, message):
+        """Parse incoming messages and take appropriate actions."""
+        try:
+            message = json.loads(message)
+            message_type = message.get('type')
+            if message_type == 'launch':
+                command = message.get('contents').get('command')
+                logfilepath = message.get('contents').get('logfilepath', None)
+                self.launch_tool(command, logfilepath)
+            elif message_type == 'shutdown':
+                self.shutdown()
+            elif message_type == 'broker':
+                handle_broker(self, message.get('contents'))  # Handle broker messages
+            elif message_type == 'message':
+                handle_message(self, message.get('contents'))  # Handle ipc messages
+            else:
+                self.send(MessageBuilder.message({"status": "unknown_message_type", "contents": message}))
+        except Exception as e:
+            self.send_error(f"An error occurred: {e}")
+
+    def handle_stdout(self, output):
+        """Handles stdout from the child process."""
+        if self.logfile and self.stdout_log_path:
+            with open(self.stdout_log_path, 'a') as f:
+                f.write(output)
+        handle_process_output(self, output)
+
+    def handle_stderr(self, output):
+        """Handles stderr from the child process."""
+        if self.logfile and self.stderr_log_path:
+            with open(self.stderr_log_path, 'a') as f:
+                f.write(output)
+        handle_process_errors(self, output)
+
+    def launch_tool(self, command_string, logfilepath=None):
+        if self.child_process:
+            self.child_process.terminate()
+
+        if logfilepath:
+            self.logfile = True
+            timestamp = datetime.now().strftime("%Y-%m-%d_%H-%M-%S")
+            self.stdout_log_path = os.path.join(logfilepath, f"{timestamp}-stdout.log")
+            self.stderr_log_path = os.path.join(logfilepath, f"{timestamp}-stderr.log")
+            os.makedirs(logfilepath, exist_ok=True)
+
+        self.child_process = subprocess.Popen(
+            command_string, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE,
+            universal_newlines=True, bufsize=1
+        )
+        # Setup stdout and stderr handling threads
+        threading.Thread(target=self.process_stream, args=(self.child_process.stdout, self.handle_stdout)).start()
+        threading.Thread(target=self.process_stream, args=(self.child_process.stderr, self.handle_stderr)).start()
+
+        # Send a message back to KADI using the message builder
+        self.send(MessageBuilder.message({"status": "tool_launched", "command": command_string}))
+
+    def shutdown(self):
+        if self.child_process:
+            self.child_process.terminate()
+            self.child_process = None
+        self.running = False
+        self.send(MessageBuilder.shutdown())
+        sys.exit(0)
+
+    def send_error(self, message):
+        sys.stderr.write(MessageBuilder.message({
+            'type': 'error',
+            'contents': {'message': message, 'type': 'exception'},
+            'time': datetime.now(timezone.utc).isoformat()
+        }) + '\n')
+        sys.stderr.flush()
+
+    def send(self, serializedJSONmessage):
+        """Send a JSON message back to Node.js using standardized format."""
+        sys.stdout.write(serializedJSONmessage + '\n')
+        sys.stdout.flush()
+
+def main():
+    try:
+        ipc = PythonAbilityIPC()
+        ipc.read_input()
+    except Exception as e:
+        # Print exception information to stderr to catch it in Node.js
+        print(json.dumps({'type': 'error', 'contents': {'message': str(e), 'type': 'exception'}, 'time': datetime.now(timezone.utc).isoformat()}), file=sys.stderr)
+        # Here you could also include more sophisticated error handling mechanisms:
+        # - Logging to a file
+        # - Sending alerts to a monitoring system
+        # - Cleanup resources or attempt recovery
+        sys.exit(1)  # Exit with a non-zero status to indicate an error
+
+if __name__ == "__main__":
+    main()

package/package.json

@@ -0,0 +1,26 @@
+{
+  "name": "@kadi.build/core",
+  "version": "0.0.1-alpha.0",
+  "description": "A module that is a comprehensive toolkit for developers integrating with the KADI infrastructure.",
+  "main": "index.js",
+  "type": "module",
+  "scripts": {
+    "format": "prettier --write .",
+    "test": "echo \"Error: no test specified\" && exit 1"
+  },
+  "author": "Corey Clark",
+  "contributors": [
+    {
+      "name": "Kassi Bertrand",
+      "url": "https://github.com/kassi-bertrand"
+    }
+  ],
+  "license": "MIT",
+  "dependencies": {
+    "events": "^3.3.0",
+    "ws": "^8.16.0"
+  },
+  "devDependencies": {
+    "prettier": "^3.6.2"
+  }
+}