@kadi.build/core 0.0.1-alpha.2 → 0.0.1-alpha.4

package/README.md

@@ -1,306 +1,1212 @@
 # @kadi.build/core
 
----
+> A comprehensive toolkit for building and managing KADI abilities with multiple transport protocols
+
+[![npm version](https://img.shields.io/npm/v/@kadi.build/core.svg)](https://www.npmjs.com/package/@kadi.build/core)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+
+## 🎯 Overview
+
+`@kadi.build/core` is the foundational library for creating KADI abilities - modular, protocol-agnostic services that can communicate via multiple transport layers. Whether you're building local tools, distributed microservices, or high-performance native modules, this toolkit provides a unified, developer-friendly API that abstracts away transport complexity.
+
+## 📚 Table of Contents
+
+- [🎯 Overview](#-overview)
+- [📦 Installation](#-installation)
+- [🚀 Quick Start](#-quick-start)
+- [🔌 Transport Protocols](#-transport-protocols)
+- [🛠️ Creating Abilities](#-creating-abilities)
+- [🤖 Creating Agents](#-creating-agents)
+- [🔄 Architecture Deep Dive](#-architecture-deep-dive)
+- [📥 Loading Abilities](#-loading-abilities)
+- [⚙️ Configuration](#-configuration)
+- [🚀 Advanced Usage](#-advanced-usage)
+- [🔧 Development Workflow](#-development-workflow)
+- [📖 API Reference](#-api-reference)
+- [🎮 Running the Examples](#-running-the-examples)
+- [💡 Additional Examples](#-additional-examples)
+- [🐛 Troubleshooting](#-troubleshooting)
+- [🤝 Contributing](#-contributing)
+- [📄 License](#-license)
+- [🔗 Related Projects](#-related-projects)
+- [📚 Resources](#-resources)
+
+## 📦 Installation
+
+```bash
+npm install @kadi.build/core
+```
+
+For global CLI tools:
+
+```bash
+npm install -g @kadi.build/cli
+```
+
+## 🚀 Quick Start
+
+The library provides a unified way to work with KADI through the **KadiClient** class, which can operate in multiple roles:
+
+- **As an Ability**: Serving tools/methods that others can call
+- **As an Agent**: Calling remote tools via broker protocol
+- **As a Service**: Combining both roles - serving and consuming tools
+
+### Creating Your First Service
+
+```javascript
+#!/usr/bin/env node
+import { KadiClient } from '@kadi.build/core';
+
+// Create a service instance
+const mathService = new KadiClient({
+  name: 'math-service',
+  role: 'ability', // 'agent', 'ability', or 'service'
+  protocol: 'stdio' // 'native', 'stdio', or 'broker'
+});
+
+// Register a tool
+mathService.registerTool('add', async ({ a, b }) => {
+  return { result: a + b };
+});
+
+// Start serving requests
+mathService.serve().catch(console.error);
+```
+
+### Loading and Using Abilities
+
+```javascript
+import { loadAbility } from '@kadi.build/core';
+
+async function main() {
+  // Load an ability (uses first interface defined in agent.json)
+  const math = await loadAbility('math-ability');
+
+  // Call methods like regular functions
+  const result = await math.add({ a: 5, b: 3 });
+  console.log(result); // { result: 8 }
+}
+
+main().catch(console.error);
+```
+
+### Creating a Broker-Connected Agent
+
+```javascript
+import { KadiClient } from '@kadi.build/core';
+
+// Create an agent that connects to broker
+const agent = new KadiClient({
+  name: 'my-agent',
+  role: 'agent',
+  protocol: 'broker',
+  brokerUrls: ['ws://localhost:8080'],
+  networks: ['global']
+});
+
+// Register tools that other agents can call
+agent.registerTool(
+  'greet',
+  async ({ name }) => {
+    return { greeting: `Hello, ${name}!` };
+  },
+  {
+    description: 'Greet someone by name',
+    inputSchema: {
+      type: 'object',
+      properties: {
+        name: { type: 'string', description: 'Name to greet' }
+      },
+      required: ['name']
+    }
+  }
+);
+
+// Connect to broker and start serving
+await agent.connectToBrokers();
+
+// Call tools from OTHER agents connected to the same broker
+const result = await agent.callTool('translate', 'text-tool', {
+  text: 'Hello world',
+  language: 'spanish'
+});
+console.log(result);
+```
+
+## 🔌 Transport Protocols
+
+KADI abilities support three transport protocols, each optimized for different use cases:
+
+### 1. Native Protocol (Fastest)
+
+- **Use Case**: High-performance, in-process execution
+- **How it Works**: Loads abilities as ES modules, enabling direct function calls
+- **Performance**: Zero IPC overhead - functions run in the same process
+- **Best For**: Utility libraries, data processing, performance-critical operations
+
+```javascript
+// Loads as native module by default if available
+const ability = await loadAbility('my-ability', 'native');
+```
+
+### 2. Stdio Protocol (Balanced)
 
-The `@kadi.build/core` module is a comprehensive toolkit for developers integrating with the KADI infrastructure. This module simplifies tasks such as managing `agent.json` files, spawning processes, interacting with brokers, and handling interprocess communication (IPC).
+- **Use Case**: Language-agnostic local execution
+- **How it Works**: Spawns abilities as child processes, communicates via LSP-style JSON-RPC over stdin/stdout
+- **Performance**: Minimal overhead, reliable local IPC
+- **Best For**: Python scripts, Go binaries, Node.js services, development/testing
+
+```javascript
+// Force stdio protocol
+const ability = await loadAbility('my-ability', 'stdio');
+```
 
-## Features
+**Environment Variables Passed to Child Process:**
 
-- **Agent JSON Management**: Manage configurations for tools, agents, or systems using KADI.
-- **Process Management**: Support launching and managing subprocesses.
-- **Multi-Broker Support**: Dynamic broker selection and management with environment/runtime configuration.
-- **Broker Interaction**: Facilitate communications with WebSocket brokers.
-- **IPC Support**: Tools for interprocess communication across various programming languages.
+```bash
+KADI_PROTOCOL=stdio  # Identifies the protocol being used
+# Plus all parent process.env variables
+```
 
-## Installation
+### 3. Broker Protocol (Distributed)
 
-Install `@kadi.build/core` using npm:
+- **Use Case**: Distributed services, containerized deployments
+- **How it Works**: Abilities connect to a WebSocket broker for network communication
+- **Performance**: Network overhead, but enables scaling and distribution
+- **Best For**: Microservices, cloud deployments, load-balanced services
+
+```javascript
+// Use broker for distributed execution
+const ability = await loadAbility('my-ability', 'broker');
+```
+
+**Environment Variables Passed to Child Process:**
 
 ```bash
-npm install @kadi.build/core
+KADI_PROTOCOL=broker                        # Identifies the protocol
+KADI_BROKER_URL=ws://localhost:8080        # Broker connection URL
+KADI_SERVICE_NAME=ability.echo.1_0_0       # Service identifier
+KADI_AGENT_SCOPE=uuid-here                 # Agent's scope/namespace
+# Plus all parent process.env variables
 ```
 
-## Documentation
+**Scope/Namespace Visibility:**
 
-### Agent JSON Functions
+The `KADI_AGENT_SCOPE` environment variable represents the agent's namespace. For the ability to be visible to its parent agent via the broker, it must register with the same scope:
 
-- **`getAbilityJSON(abilityName, abilityVersion)`**: Retrieves the `agent.json` for a specified ability.
-- **`getAbilityJSONPath(abilityName, abilityVersion)`**: Provides the file path to the `agent.json` for a specific ability.
-- **`getAbilityVersionFromArray(abilities, name)`**: Searches ability array provided, and returns the version number for name provided.
-- **`getAbilitiesDir()`**: Returns the directory path where abilities are stored.
-- **`getProjectJSON()`**: Fetches the `agent.json` for the current project.
-- **`getProjectJSONPath()`**: Gets the file path for the project's `agent.json`.
-- **`getKadiCoreJSON()`**: Retrieves the `agent.json` for the Kadi core.
-- **`getKadiCoreJSONPath()`**: Provides the file path to the Kadi core's `agent.json`.
-- **`getKadiJSON()`**: Fetches the `agent.json` for the Kadi system.
-- **`getKadiJSONPath()`**: Returns the file path for the Kadi system's `agent.json`.
+```javascript
+// In your ability code:
+const echoAbility = new KadiAbility({
+  name: 'echo-js',
+  version: '0.0.1',
+  description: 'Echo ability with broker support',
+  scope: process.env.KADI_AGENT_SCOPE // Use agent's scope for visibility
+});
 
-### Broker Management Functions
+// Without this, the ability would only be visible in the 'global' scope
+// and the parent agent wouldn't be able to communicate with it
+```
 
-The broker management system allows dynamic selection from multiple configured brokers defined in `agent.json`:
+### Protocol Selection Strategy
+
+When no protocol is specified, `loadAbility` uses the **first interface** defined in the ability's `agent.json`. The KADI team recommends ordering interfaces from fastest to slowest (native → stdio → broker) for optimal default performance.
+
+```mermaid
+graph TD
+    A[Load Ability] --> B{Protocol Specified?}
+    B -->|Yes| C[Use Specified Protocol]
+    B -->|No| D{Check agent.json interfaces}
+    D --> E[Use First Interface Listed]
+    E --> F[native/stdio/broker]
+```
+
+**Recommended Interface Order in agent.json:**
 
 ```json
 {
-  "brokers": {
-    "local": "ws://127.0.0.1:8080",
-    "remote": "ws://production.example.com:8080",
-    "staging": "ws://staging.example.com:8080"
+  "interfaces": {
+    "native": { "entry": "service.js" }, // 1st choice: Fastest
+    "stdio": { "discover": true }, // 2nd choice: Balanced
+    "broker": { "discover": true } // 3rd choice: Distributed
   }
 }
 ```
 
-Available functions:
+## 🛠️ Creating Services with KadiClient
+
+### Basic Service Structure
+
+```javascript
+import { KadiClient } from '@kadi.build/core';
 
-- **`KADI_BROKERS`**: Object containing all configured brokers with their parsed URLs.
-- **`KADI_BROKER_URL`**: Default broker URL (first one defined, for backward compatibility).
-- **`getBrokerUrl(brokerName)`**: Get URL for a specific broker by name. Returns `null` if not found.
-- **`getBrokerNames()`**: Get array of all available broker names.
-- **`setActiveBroker(brokerName)`**: Set the active broker for the session. Returns `true` if successful.
-- **`getActiveBrokerName()`**: Get the name of the currently active broker.
-- **`getActiveBrokerUrl()`**: Get the URL of the currently active broker.
-- **`getDefaultBrokerName()`**: Get the name of the default broker (first one defined).
-- **`selectBrokerFromEnv()`**: Set active broker from `KADI_BROKER` environment variable.
+const service = new KadiClient({
+  name: 'echo-service',
+  role: 'ability',  // 'agent', 'ability', or 'service'
+  protocol: 'stdio', // 'native', 'stdio', or 'broker'
+  brokerUrls: ['ws://localhost:8080'] 
+});
+
+// Method 1: Simple tool registration
+service.registerTool('echo', async ({ message }) => ({
+  echo: message,
+  timestamp: new Date().toISOString()
+}));
+
+// Method 2: Tool with inline schema
+service.registerTool(
+  'format',
+  async ({ text, style }) => ({
+    formatted: style === 'upper' ? text.toUpperCase() : text.toLowerCase()
+  }),
+  {
+    description: 'Format text with specified style',
+    inputSchema: {
+      type: 'object',
+      properties: {
+        text: { type: 'string', description: 'Text to format' },
+        style: {
+          type: 'string',
+          enum: ['upper', 'lower'],
+          description: 'Formatting style'
+        }
+      },
+      required: ['text', 'style']
+    },
+    outputSchema: {
+      type: 'object',
+      properties: {
+        formatted: { type: 'string', description: 'Formatted text' }
+      }
+    }
+  }
+);
+
+// Start serving
+service.serve().catch(console.error);
+```
 
-### Process Management Functions
+### Event Publishing and Subscription
 
-- **`runExecCommand(name, version, command)`**: Executes a command for initializing abilities.
-- **`runSpawnCommand(name, version, command)`**: Uses `spawn` to execute commands for subprocesses.
+**Full Support**: Events work across all protocols - native, stdio, and broker.
 
-## Usage Examples
+KadiClient provides a unified event system that works consistently across all transport protocols:
 
 ```javascript
-import {
-  getProjectJSON,
-  runExecCommand,
-  IPCManager,
-  Broker
-} from '@kadi.build/core';
+// In your service (service.js)
+const service = new KadiClient({ 
+  name: 'my-service',
+  role: 'ability',
+  protocol: 'broker' // Works with all protocols
+});
+
+// Publish events from tool handlers
+service.registerTool('echo', async ({ message }) => {
+  // Publish events during processing
+  await service.publishEvent('echo.processing', { status: 'started' });
+  
+  // Do work...
+  const result = { echo: message, timestamp: new Date().toISOString() };
+  
+  await service.publishEvent('echo.completed', { result });
+  return result;
+});
+
+// In your client (index.js)
+const client = new KadiClient({
+  name: 'my-client',
+  role: 'agent',
+  protocol: 'broker'
+});
+
+// Subscribe to events using patterns
+client.subscribeToEvent('echo.*', (data) => {
+  console.log('Echo event received:', data);
+});
+
+// For loaded abilities with native/stdio protocols
+const ability = await loadAbility('my-service', 'stdio');
+ability.events.on('echo.completed', (data) => {
+  console.log('Completed:', data);
+});
+```
+
+**Event Patterns**: 
+- Use wildcards: `math.*` matches `math.operation`, `math.milestone`
+- Protocol-specific delivery:
+  - **Native**: Direct EventEmitter (synchronous)
+  - **Stdio**: JSON-RPC notifications over stdout
+  - **Broker**: RabbitMQ pub/sub via WebSocket
+
+### Ability Configuration (agent.json)
 
-async function setupProject() {
-  const projectConfig = getProjectJSON();
-  console.log(projectConfig);
-
-  await runExecCommand('example', '1.0', 'npm install');
-
-  // Broker management examples
-  console.log('Available brokers:', getBrokerNames());
-  console.log('All brokers:', KADI_BROKERS);
-
-  // Set active broker
-  setActiveBroker('remote');
-  console.log('Active broker:', getActiveBrokerUrl());
-
-  // IPC setup
-  const ipc = new IPCManager();
-  ipc.createInstance('python', 'pythonScript.py', 'pythonInstance');
-
-  // Traditional broker setup
-  Broker.addBroker('ws://example.com', 'exampleBroker');
-  let broker = Broker.getBroker('default');
-  console.log(`Connected to ${broker.url}`);
-  broker.send(
-    BrokerMessageBuilder.setup(
-      'TestAgent',
-      'A Broker Testing Agent',
-      null,
-      null
-    )
-  );
+Every ability should have an `agent.json` file that defines its capabilities and interfaces:
+
+```json
+{
+  "name": "echo-ability",
+  "kind": "ability",
+  "version": "1.0.0",
+  "license": "MIT",
+  "description": "Echo service with multiple transport support",
+  "scripts": {
+    "setup": "npm install",
+    "start": "node service.js"
+  },
+  "interfaces": {
+    "native": {
+      "entry": "service.js"
+    },
+    "stdio": {
+      "discover": true,
+      "timeoutMs": 10000
+    },
+    "broker": {
+      "discover": true,
+      "timeoutMs": 15000,
+      "serviceName": "echo-service"
+    }
+  },
+  "exports": [
+    {
+      "name": "echo",
+      "description": "Echo the input message",
+      "inputSchema": {
+        "type": "object",
+        "properties": {
+          "message": { "type": "string" }
+        },
+        "required": ["message"]
+      },
+      "outputSchema": {
+        "type": "object",
+        "properties": {
+          "echo": { "type": "string" },
+          "timestamp": { "type": "string" }
+        }
+      }
+    }
+  ],
+  "brokers": {
+    "local": "ws://localhost:8080",
+    "remote": "ws://api.example.com:8080"
+  }
 }
 ```
 
-### Broker Functions
+### Schema Definition Options
 
-- **`Broker`**: Manages interactions with broker systems.
-- **`IBroker`**: Broker instance interface with methods to manage its lifecycle and communications.
-- **`BrokerMessageBuilder`**: Assists in constructing messages for broker communication.
+The ability system provides two ways to define method schemas:
 
-## Broker Functions
+1. **Export Schemas** (defined in `agent.json` exports section)
+2. **Inline Schemas** (passed directly to `.method()`)
 
-The Broker system facilitates interaction between different agents and services within the Kadi environment.
+```javascript
+// Option 1: Define in agent.json exports section
+// The schema is picked up automatically if the method name matches
+// agent.json:
+{
+  "exports": [
+    {
+      "name": "process",
+      "description": "Process data",
+      "inputSchema": { /* ... */ },
+      "outputSchema": { /* ... */ }
+    }
+  ]
+}
 
-### `Broker` Interface
+// Option 2: Define inline when registering the method
+ability.method('process', handler, {
+  description: 'Process data',
+  inputSchema: { /* ... */ },
+  outputSchema: { /* ... */ }
+});
 
-- **`addBroker(url, name = 'default')`**: Adds a new broker connection. If a broker with the same name exists, it will be reused.
-- **`disconnect(name = 'default')`**: Disconnects the broker specified by the name.
-- **`deleteBroker(name = 'default')`**: Deletes the broker specified by the name and closes its connection.
-- **`send(message, brokerName = 'default')`**: Sends a message through the broker specified by the name.
-- **`addEventListener(event, listener, brokerName = 'default')`**: Adds an event listener to the specified broker.
-- **`removeEventListener(event, listener, brokerName = 'default')`**: Removes an event listener from the specified broker.
-- **`removeAllListeners(brokerName = 'default')`**: Removes all event listeners from the specified broker.
-- **`getBroker(brokerName = 'default')`**: Retrieves a broker instance by name.
+// Note: If both are defined, inline schemas take precedence over exports
+```
 
-### `IBroker` Instance
+**Important**: Specifying MCP schemas for ability handlers is particularly important for `broker` communication. If you only need `native` and `stdio` protocols, schemas can be omitted.
 
-This class extends `EventEmitter` and manages individual broker connections:
+## 🤖 Working with KadiClient
 
-- **`constructor(url, name)`**: Initializes a new broker connection.
-- **`send(message)`**: Sends a message through the WebSocket connection.
-- **`getConnectedAgents()`**: Retrieves a list of currently connected agents.
+KadiClient is the unified API for all KADI operations. It replaces the previous separate KadiAbility and KadiAgent classes with a single, powerful interface that can operate in multiple roles.
 
-### `BrokerMessageBuilder`
+### Direct Approach (using loadAbility)
 
-Utility class for constructing broker messages:
+```javascript
+import { loadAbility } from '@kadi.build/core';
 
-- **`create_message(type, data)`**: Creates a JSON string with the specified type and data.
-- **`message(to, content)`**: Constructs a message directed to a specific peer.
-- **`setup(name, description, limit, uuid)`**: Creates a setup message for registering the broker.
-- **`suspend()`**: Constructs a suspend message.
-- **`finish()`**: Constructs a finish message.
-- **`list()`**: Constructs a list message to request a list of connected agents.
+// Load and call abilities directly
+const echoAbility = await loadAbility('echo-js', 'stdio');
+const result = await echoAbility.echo({ message: 'Hello world' });
 
-### IPC Functions
+// Load broker tools directly
+const remoteService = await loadAbility('remote-service', 'broker', {
+  brokerUrl: 'ws://localhost:8080',
+  networks: ['global']
+});
+const brokerResult = await remoteService.process({ data: 'test' });
+```
 
-- **`IPCMessageBuilder`**: Constructs messages for IPC interactions.
-- **`IPCManager`**: Manages setup and lifecycle of IPC connections.
-- **`IAbilityIPC`**: Represents an IPC ability instance with methods to manage its lifecycle and communications.
+### Using KadiClient
 
-### `IPCManager`
+```javascript
+import { KadiClient } from '@kadi.build/core';
+
+// Create a unified client
+const client = new KadiClient({
+  name: 'my-service',
+  role: 'service', // Can both serve and consume
+  protocol: 'broker',
+  brokerUrls: ['ws://localhost:8080'],
+  networks: ['global']
+});
 
-Manages the lifecycle and setup of interprocess communications.
+// Register tools for others to call
+client.registerTool(
+  'greet',
+  async ({ name }) => {
+    return { greeting: `Hello, ${name}!` };
+  },
+  {
+    description: 'Greet someone by name',
+    inputSchema: {
+      type: 'object',
+      properties: { name: { type: 'string' } },
+      required: ['name']
+    }
+  }
+);
 
-- **`createInstance(language, commandString, name = 'default')`**: Creates and manages a new IPC instance for the specified language.
-- **`getInstance(name)`**: Retrieves an existing IPC instance by name.
-- **`shutdownInstance(name)`**: Shuts down an existing IPC instance by name.
+// Connect and serve
+await client.connectToBrokers();
 
-### `IAbilityIPC` Instance
+// Call remote tools
+const result = await client.callTool('translator', 'translate', {
+  text: 'Hello',
+  to: 'es'
+});
 
-Manages an individual interprocess communication instance:
+// Load abilities for compatibility
+const ability = await client.loadAbility('echo-js');
+```
 
-- **`start()`**: Starts the child process associated with this IPC instance.
-- **`launch()`**: Sends the launch command to the associated process.
-- **`sendMessage(message)`**: Sends a JSON-formatted message to the child process.
-- **`shutdown()`**: Sends a shutdown command to the child process and terminates it.
+Use `loadAbility()` for simple consumption, or KadiClient for full-featured service development with tool registration, event publishing, and remote tool invocation.
+
+## 🔄 Architecture Deep Dive
+
+### Transport Architecture and Loading Sequence
+
+The loading process uses modular transports based on the selected protocol:
+
+```mermaid
+sequenceDiagram
+    participant Client
+    participant loadAbility
+    participant Transport
+    participant FileSystem
+    participant ChildProcess
+    participant Service
+    participant Broker
+
+    Client->>loadAbility: loadAbility('echo-js', protocol?)
+    loadAbility->>FileSystem: Read agent.json manifests
+    FileSystem-->>loadAbility: Return config & version
+    
+    alt protocol === 'native'
+        loadAbility->>Transport: new NativeTransport(config)
+        Transport->>FileSystem: import(entry point)
+        FileSystem-->>Transport: Module with KadiClient instance
+        Transport->>Service: Extract tool handlers via getToolNames()
+        Service-->>Transport: Return registered tools
+        Transport->>Transport: Store handlers in Map
+    else protocol === 'stdio'
+        loadAbility->>Transport: new StdioTransport(config)
+        Transport->>ChildProcess: spawn(startCmd, env: {KADI_PROTOCOL})
+        ChildProcess->>Service: KadiClient detects stdio mode
+        Service->>Service: Setup FrameReader/Writer
+        Transport->>Service: Send discovery request
+        Service-->>Transport: Return tool list
+        Transport->>Transport: Setup JSON-RPC proxy
+    else protocol === 'broker'
+        loadAbility->>Transport: new BrokerTransport(config)
+        Transport->>Transport: Create KadiClient wrapper
+        Transport->>Broker: WebSocket connect
+        Transport->>Broker: kadi.session.hello
+        Broker-->>Transport: { nonce, heartbeatIntervalSec }
+        Transport->>Broker: kadi.session.authenticate
+        Broker-->>Transport: { sessionId }
+        Transport->>Broker: kadi.ability.list
+        Broker-->>Transport: Available tools
+        Transport->>Transport: Start heartbeat timer
+    end
+
+    loadAbility->>loadAbility: Create proxy with tool methods
+    loadAbility-->>Client: Return ability proxy
+    
+    Note over Client,Service: Tool invocation
+    Client->>Transport: ability.echo({ message })
+    
+    alt native
+        Transport->>Service: Direct handler call
+    else stdio  
+        Transport->>Service: JSON-RPC over frames
+    else broker
+        Transport->>Broker: kadi.ability.invoke
+        Broker->>Service: Route to target
+        Service->>Broker: kadi.ability.result
+        Broker-->>Transport: Forward result
+    end
+    
+    Transport-->>Client: Return result
+```
 
-### `IPCMessageBuilder`
+### KadiClient Architecture: Tool Registration & Serving
+
+How KadiClient registers tools and starts serving:
+
+```mermaid
+sequenceDiagram
+    participant Dev as Developer
+    participant KC as KadiClient
+    participant Transport
+    participant Network as Network Layer
+    participant RemoteClient
+
+    Note over Dev,KC: Tool Registration Phase
+    Dev->>KC: new KadiClient({ name, role, protocol })
+    KC->>KC: Initialize based on role & protocol
+    
+    Dev->>KC: registerTool('echo', handler, schema?)
+    KC->>KC: Store in toolHandlers Map<name, {handler, schema}>
+    
+    Dev->>KC: registerTool('process', handler, schema)
+    KC->>KC: Add to toolHandlers Map
+
+    Note over KC,Network: Connection & Serving Phase
+    Dev->>KC: serve() or connectToBrokers()
+    
+    alt protocol === 'native'
+        KC->>KC: Tools available via direct reference
+        KC->>KC: Emit 'start' event
+    else protocol === 'stdio'
+        KC->>Transport: Setup StdioTransport
+        Transport->>Transport: new FrameReader(stdin)
+        Transport->>Transport: new FrameWriter(stdout)
+        Transport->>KC: on('message', handleStdioMessage)
+        KC->>KC: Process JSON-RPC requests
+    else protocol === 'broker'
+        KC->>Network: WebSocket.connect(brokerUrl)
+        KC->>Network: kadi.session.hello({ role })
+        Network-->>KC: { nonce, heartbeatIntervalSec: 30 }
+        KC->>KC: Sign nonce with Ed25519
+        KC->>Network: kadi.session.authenticate({ signature })
+        Network-->>KC: { sessionId }
+        KC->>Network: kadi.agent.register({ tools, networks })
+        KC->>KC: Start heartbeat timer (30s interval)
+    end
+
+    Note over RemoteClient,KC: Tool Invocation Flow
+    RemoteClient->>Network: Request tool invocation
+    
+    alt native
+        RemoteClient->>KC: Direct method call
+        KC->>KC: toolHandlers.get(name).handler(params)
+    else stdio
+        Network->>Transport: JSON-RPC frame
+        Transport->>KC: Parsed request
+        KC->>KC: toolHandlers.get(name).handler(params)
+        KC->>Transport: JSON-RPC response
+        Transport->>Network: Framed response
+    else broker
+        Network->>KC: kadi.ability.invoke message
+        KC->>KC: toolHandlers.get(toolName).handler(params)
+        KC->>Network: kadi.ability.result message
+    end
+    
+    Network-->>RemoteClient: Tool result
+```
 
-Utility class for constructing IPC messages:
+### Event System Architecture
+
+KadiClient provides a unified event API that works across all transport protocols:
+
+```mermaid
+sequenceDiagram
+    participant Publisher as Service (Publisher)
+    participant Transport as Transport Layer
+    participant EventBus as Event Bus
+    participant Subscriber as Client (Subscriber)
+    
+    Note over Subscriber: Setup subscription
+    Subscriber->>Subscriber: subscribeToEvent('math.*', callback)
+    
+    alt protocol === 'broker'
+        Subscriber->>EventBus: kadi.event.subscribe({channels: ['math.*']})
+        EventBus-->>Subscriber: {subscribed: ['math.*'], queueName}
+    else protocol === 'stdio'
+        Subscriber->>Transport: Setup event listener
+        Transport->>Subscriber: on('transport:event', dispatcher)
+    else protocol === 'native'
+        Subscriber->>Publisher: Direct EventEmitter reference
+        Publisher->>Subscriber: on('ability:event', dispatcher)
+    end
+    
+    Note over Publisher: Publish event
+    Publisher->>Publisher: publishEvent('math.operation', data)
+    
+    alt protocol === 'native'
+        Publisher->>Publisher: emit('ability:event', {eventName, data})
+        Publisher-->>Subscriber: Direct EventEmitter delivery
+        Subscriber->>Subscriber: Pattern match & invoke callback
+    else protocol === 'stdio'
+        Publisher->>Transport: writer.write(__kadi_event notification)
+        Transport->>Transport: Frame with LSP headers
+        Transport-->>Subscriber: Framed event over stdout
+        Subscriber->>Transport: FrameReader parses
+        Transport->>Subscriber: emit('transport:event', {name, data})
+        Subscriber->>Subscriber: Pattern match & invoke callback  
+    else protocol === 'broker'
+        Publisher->>EventBus: kadi.event.publish({channel, data})
+        EventBus->>EventBus: Route via RabbitMQ
+        EventBus-->>Subscriber: kadi.event.delivery
+        Subscriber->>Subscriber: Pattern match & invoke callback
+    end
+    
+    Note over Subscriber: Cleanup
+    Subscriber->>Subscriber: unsubscribe()
+    alt protocol === 'broker'
+        Subscriber->>EventBus: kadi.event.unsubscribe
+    else
+        Subscriber->>Subscriber: Remove local listener
+    end
+```
 
-- **`createMessage(type, contents)`**: Creates a JSON string with the specified type and contents.
-- **`launch(commandString)`**: Constructs a launch message with the specified command string.
-- **`shutdown()`**: Constructs a shutdown message.
-- **`message(content)`**: Constructs a general message with the specified content.
+### Broker Protocol Communication Flow
+
+Detailed view of broker-based communication with heartbeat:
+
+```mermaid
+sequenceDiagram
+    participant Client as KadiClient
+    participant Broker as KADI Broker
+    participant Target as Target Service
+    
+    Note over Client,Broker: Connection & Authentication
+    Client->>Broker: WebSocket connect
+    Client->>Broker: kadi.session.hello({ role })
+    Broker-->>Client: { nonce, heartbeatIntervalSec: 30 }
+    Client->>Client: Generate Ed25519 keypair
+    Client->>Client: Sign nonce
+    Client->>Broker: kadi.session.authenticate({ signature })
+    Broker-->>Client: { sessionId }
+    
+    Note over Client,Broker: Heartbeat Management
+    Client->>Client: Start heartbeat timer (30s)
+    loop Every 30 seconds
+        Client->>Broker: kadi.session.ping
+        Note over Broker: Reset connection timeout (90s)
+    end
+    
+    Note over Client,Broker: Tool Registration
+    Client->>Broker: kadi.agent.register({ tools, networks })
+    Broker->>Broker: Store in registry
+    Broker-->>Client: { registered: true }
+    
+    Note over Client,Target: Remote Tool Invocation
+    Client->>Client: callTool('service-b', 'process', params)
+    Client->>Broker: kadi.ability.invoke({ 
+        targetAgent: 'service-b',
+        toolName: 'process',
+        toolInput: params
+    })
+    Broker->>Broker: Lookup service-b
+    Broker->>Target: kadi.ability.invoke
+    Target->>Target: Execute tool handler
+    Target->>Broker: JSON-RPC response
+    Broker-->>Client: Forward response
+    Client->>Client: Resolve promise
+    
+    Note over Client,Broker: Event Publishing
+    Client->>Broker: kadi.event.publish({
+        channel: 'system.status',
+        data: { status: 'healthy' }
+    })
+    Broker->>Broker: Publish to RabbitMQ exchange
+    
+    Note over Client,Broker: Graceful Shutdown
+    Client->>Client: Stop heartbeat timer
+    Client->>Broker: kadi.session.goodbye
+    Client->>Client: Close WebSocket
+```
 
-## Usage Examples
+### Stdio Protocol: LSP-Style Frame Processing
+
+How KadiClient handles stdio communication with LSP-style framing:
+
+```mermaid
+sequenceDiagram
+    participant Parent as Parent Process
+    participant SR as StdioFrameReader
+    participant SW as StdioFrameWriter
+    participant Server as StdioRpcServer
+    participant Ability as KadiAbility
+
+    Note over Parent,SR: Incoming Request
+    Parent->>SR: Write to stdin: "Kadi-Content-Length: 52\r\n\r\n{...}"
+    SR->>SR: Buffer incoming data
+    SR->>SR: Look for Content-Length header
+    SR->>SR: Parse header, extract body length
+    SR->>SR: Wait for complete body
+    SR->>SR: Parse JSON from body
+    SR->>Server: onMessage({ success: true, data: request })
+
+    Server->>Server: Validate JSON-RPC format
+    Server->>Ability: getMethodHandler(request.method)
+    Ability-->>Server: Return handler
+    Server->>Server: Execute handler(params) with timeout
+
+    Note over Server,Parent: Response
+    Server->>SW: write(response)
+    SW->>SW: JSON.stringify(response)
+    SW->>SW: Calculate byte length
+    SW->>SW: Create header: "Kadi-Content-Length: N\r\n\r\n"
+    SW->>Parent: Write header + body to stdout
+
+    Note over SR: Error Recovery
+    alt Frame Corruption
+        SR->>SR: Detect invalid frame
+        SR->>SR: Search for next Content-Length marker
+        SR->>SR: Skip corrupted bytes
+        SR->>Server: onMessage({ success: false, error: 'FRAME_CORRUPTION' })
+        Server->>Server: Log error, continue processing
+    end
+```
 
-Here's how you might use the IPC and Broker functionalities:
+## 📥 Loading Abilities
 
-### Interacting with IPC
+### Basic Loading
 
 ```javascript
-console.log('Starting IPC Ability Interface test...');
+import { loadAbility } from '@kadi.build/core';
 
-// Create Python IPC instance
-const pythonIPC = IPCManager.createInstance(
-  'python',
-  'echo "Hello, World!"',
-  'pythonTest'
-);
+// Auto-detect protocol from agent.json
+const ability = await loadAbility('my-ability');
+
+// Explicitly specify protocol
+const stdioAbility = await loadAbility('my-ability', 'stdio');
+const brokerAbility = await loadAbility('my-ability', 'broker');
+const nativeAbility = await loadAbility('my-ability', 'native');
+```
+
+### Working with Loaded Abilities
 
-pythonIPC.on('error', (error) => {
-  console.error('Handled Python Error:', error);
-  // Consider appropriate error handling or recovery actions here
+```javascript
+// Call methods directly - no need to discover them first
+const result = await ability.echo({ message: 'hello' });
+
+// Call methods
+const result = await ability.someMethod({ param: 'value' });
+
+// Direct RPC call (bypasses method validation)
+const response = await ability.call('someMethod', { param: 'value' });
+
+// Subscribe to events
+ability.events.on('custom:event', (data) => {
+  console.log('Event received:', data);
+});
+
+// Note: Events work for native and stdio protocols
+// Subscribe before calling methods that emit events
+```
+
+### Loading Context
+
+The loader automatically resolves ability versions from:
+
+1. **Project context**: Reads from project's `agent.json`
+2. **Nested context**: When loading from within another ability
+3. **Explicit version**: Can be specified in ability name
+
+## ⚙️ Configuration
+
+### Environment Variables
+
+```bash
+# Set default protocol
+export KADI_PROTOCOL=stdio
+
+# Configure broker
+export KADI_BROKER_URL=ws://localhost:8080
+export KADI_SERVICE_NAME=my-service
+export KADI_AGENT_SCOPE=project-123
+```
+
+### Environment Variables for Child Processes
+
+When abilities are spawned as child processes (stdio and broker protocols), the parent process passes down important environment variables:
+
+```javascript
+// In your ability code, you can access these variables:
+const protocol = process.env.KADI_PROTOCOL; // 'stdio' or 'broker'
+const brokerUrl = process.env.KADI_BROKER_URL; 
+const serviceName = process.env.KADI_SERVICE_NAME; 
+const agentScope = process.env.KADI_AGENT_SCOPE;
+
+// Use them to configure your ability behavior
+const ability = new KadiAbility({
+  name: 'my-ability',
+  scope: process.env.KADI_AGENT_SCOPE || 'global'
+  // Ability automatically uses KADI_PROTOCOL to determine transport
+});
+```
+
+### Example Project Structure
+
+```
+my-project/
+├── agent.json              # Project configuration
+├── abilities/              # Installed abilities
+│   └── echo-ability/
+│       └── 1.0.0/
+│           ├── agent.json  # Ability configuration
+│           ├── service.js  # Ability implementation
+│           └── package.json
+├── modules/                # Source modules
+│   └── echo-ability/       # Development version
+└── index.js               # Main entry point
+```
+
+## 🚀 Advanced Usage
+
+### Event System
+
+KadiClient provides a comprehensive event system that works across all protocols:
+
+#### 1. Lifecycle Events
+
+Monitor the service lifecycle:
+
+```javascript
+const client = new KadiClient({ name: 'my-service' });
+
+// Connection events
+client.on('connected', ({ broker }) => {
+  console.log(`Connected to broker: ${broker}`);
 });
 
-pythonIPC.on('critical-error', (error) => {
-  console.error('Critical Error from Python:', error);
-  // Handle critical errors, potentially restarting the process or alerting administrators
+client.on('disconnected', () => {
+  console.log('Disconnected from broker');
 });
 
-pythonIPC.on('message', (msg) => {
-  console.log('Message from Python:', msg);
+// Tool invocation events
+client.on('tool:invoked', ({ toolName, params }) => {
+  console.log(`Tool ${toolName} invoked with:`, params);
 });
 
-pythonIPC.on('broker', (msg) => {
-  console.log('Broker message from Python:', msg);
+client.on('tool:completed', ({ toolName, result }) => {
+  console.log(`Tool ${toolName} completed:`, result);
 });
 
-pythonIPC.on('shutdown', () => {
-  console.log('Python IPC has shut down.');
+client.on('error', (error) => {
+  console.error('Service error:', error);
 });
+```
+
+#### 2. Custom Events (All Protocols)
 
-Broker.addBroker('http://your.broker.com', 'default');
-let brk = Broker.getBroker('default');
-broker = brk;
-
-pythonIPC.attachBroker(brk);
-
-brk.on('open', function open() {
-  console.log(`Connected to ${brk.url}`);
-  const jsonObject = BrokerMessageBuilder.setup(
-    'TestAgent',
-    'A Broker Testing Agent',
-    null,
-    null
-  );
-  brk.send(jsonObject);
+Publish and subscribe to custom events across all transport protocols:
+
+```javascript
+// Publishing events
+const publisher = new KadiClient({
+  name: 'event-publisher',
+  role: 'service',
+  protocol: 'broker' // Works with all protocols
 });
 
-brk.on('message', function incoming(data) {
-  let msg = JSON.parse(data);
-  if (msg.type == 'setup') {
-    (async () => {
-      try {
-        await GetLLMProxy();
-      } catch (error) {
-        console.error('Error during GetLLMProxy:', error.message);
-      }
-    })();
+publisher.registerTool('process', async ({ data }) => {
+  // Publish events during processing
+  await publisher.publishEvent('process.started', {
+    timestamp: Date.now(),
+    data
+  });
+
+  // Do work...
+  const result = await processData(data);
+
+  await publisher.publishEvent('process.completed', {
+    timestamp: Date.now(),
+    result
+  });
+
+  return result;
+});
+
+// Subscribing to events
+const subscriber = new KadiClient({
+  name: 'event-subscriber',
+  role: 'agent',
+  protocol: 'broker'
+});
+
+// Subscribe with wildcards
+subscriber.subscribeToEvent('process.*', (data) => {
+  console.log('Process event:', data);
+});
+
+// One-time subscription
+subscriber.onceEvent('process.completed', (data) => {
+  console.log('First completion:', data);
+});
+
+// Multiple pattern subscription
+subscriber.subscribeToEvents(
+  ['process.*', 'system.*'],
+  (pattern, data) => {
+    console.log(`Event from ${pattern}:`, data);
   }
-  // console.log('Received:', formatJSON(data));
+);
+```
+
+**Protocol-Specific Behavior:**
+
+- **Native**: Direct EventEmitter, synchronous delivery
+- **Stdio**: JSON-RPC notifications with LSP framing
+- **Broker**: RabbitMQ pub/sub via WebSocket, persistent queues available
+
+
+## 🔧 Development Workflow
+
+### Testing Local Changes
+
+When developing new features or testing changes to `@kadi.build/core` before publishing to NPM:
+
+0. **clone the repository**
+
+```bash
+git clone https://gitlab.com/humin-game-lab/kadi/kadi-core.git
+```
+
+1. **Pack the local version** (in the kadi-core directory):
+
+```bash
+cd /path/to/kadi-core
+npm pack
+# This creates: kadi.build-core-X.Y.Z.tgz
+```
+
+1. **Install in your project**:
+
+```bash
+cd /path/to/your-project
+npm install /path/to/kadi-core/kadi.build-core-X.Y.Z.tgz
+```
+
+1. **For ability development**, update the ability's preflight script:
+
+```json
+{
+  "scripts": {
+    "preflight": "npm uninstall @kadi.build/core && npm install /path/to/kadi.build-core-X.Y.Z.tgz"
+  }
+}
+```
+
+## 📖 API Reference
+
+### Core Classes
+
+#### `KadiClient`
+
+The unified class for all KADI operations - serving tools, calling remote services, and managing events.
+
+```javascript
+import { KadiClient } from '@kadi.build/core';
+
+const client = new KadiClient({
+  name: 'my-service',
+  role: 'service', // 'agent', 'ability', or 'service'
+  protocol: 'broker', // 'native', 'stdio', or 'broker'
+  brokerUrls: ['ws://localhost:8080'],
+  networks: ['global', 'custom-network']
 });
+```
+
+**Configuration Options:**
+- `name` - Service/agent name
+- `role` - Operating role: 'agent' (consumer), 'ability' (provider), 'service' (both)
+- `protocol` - Transport protocol to use
+- `brokerUrls` - Array of broker WebSocket URLs (for broker protocol)
+- `networks` - Network namespaces for tool discovery
+- `heartbeatIntervalSec` - Custom heartbeat interval (default: from broker)
+
+**Tool Registration Methods:**
+- `registerTool(name, handler, schema?)` - Register a single tool
+- `getTools()` - Get list of registered tool names
+- `getToolNames()` - Get filtered list of tool names
+- `getToolHandler(name)` - Get a specific tool's handler
+- `getToolSchema(name)` - Get a specific tool's schema
+- `hasTool(name)` - Check if a tool is registered
+
+**Remote Tool Invocation:**
+- `callTool(targetAgent, toolName, params)` - Call a remote tool via broker
+- `discoverRemoteTools(targetAgent)` - List tools from a remote agent
+- `loadAbility(name, protocol?, options?)` - Load an ability (compatibility)
+
+**Event System:**
+- `publishEvent(eventName, data)` - Publish an event
+- `subscribeToEvent(pattern, callback)` - Subscribe with wildcards
+- `subscribeToEvents(patterns[], callback)` - Multiple subscriptions
+- `unsubscribeFromEvent(pattern, callback)` - Remove subscription
+- `onceEvent(pattern, callback)` - One-time subscription
+
+**Connection Management:**
+- `serve()` - Start serving (stdio/native modes)
+- `connectToBrokers(urls?)` - Connect to broker(s)
+- `disconnect()` - Clean disconnect
+- `isConnected` - Check connection status
+
+**Properties:**
+- `agentId` - Unique agent identifier
+- `name` - Service name
+- `role` - Current operating role
+- `protocol` - Active protocol
+
+**Complete Usage Example:**
 
-brk.on('error', function error(error) {
-  console.error('WebSocket error:', error);
+```javascript
+import { KadiClient } from '@kadi.build/core';
+
+// Create a service that can both serve and consume
+const service = new KadiClient({
+  name: 'math-service',
+  role: 'service',
+  protocol: 'broker',
+  brokerUrls: ['ws://localhost:8080'],
+  networks: ['global']
 });
 
-brk.on('close', function close() {
-  console.log('Disconnected from the server');
-  process.exit(0); // Exit the process when the WebSocket connection is closed
+// Register tools with schemas
+service.registerTool(
+  'add',
+  async ({ a, b }) => {
+    // Publish event before processing
+    await service.publishEvent('math.operation', {
+      operation: 'add',
+      inputs: { a, b }
+    });
+    
+    const result = a + b;
+    
+    // Publish completion event
+    await service.publishEvent('math.completed', {
+      operation: 'add',
+      result
+    });
+    
+    return { result };
+  },
+  {
+    description: 'Add two numbers',
+    inputSchema: {
+      type: 'object',
+      properties: {
+        a: { type: 'number', description: 'First number' },
+        b: { type: 'number', description: 'Second number' }
+      },
+      required: ['a', 'b']
+    },
+    outputSchema: {
+      type: 'object',
+      properties: {
+        result: { type: 'number', description: 'Sum of a and b' }
+      }
+    }
+  }
+);
+
+// Subscribe to events from other services
+service.subscribeToEvent('system.*', (data) => {
+  console.log('System event:', data);
 });
 
-// Send launch command
+// Connect to broker
+await service.connectToBrokers();
 
-//Utilize log files
-// pythonIPC.sendMessage(IPCMessageBuilder.launch(pythonIPC.commandString, process.cwd() + "/logs"));
+// Call remote tools
+const result = await service.callTool(
+  'translator-service',
+  'translate',
+  { text: 'Hello', to: 'es' }
+);
 
-//No log files created
-pythonIPC.sendMessage(IPCMessageBuilder.launch(pythonIPC.commandString));
+// Graceful shutdown
+process.on('SIGTERM', async () => {
+  await service.disconnect();
+  process.exit(0);
+});
 ```
 
-### Handling IPC
+#### `loadAbility`
+
+Load an ability by name and protocol.
 
 ```javascript
-const ipcManager = new IPCManager();
-const ipcInstance = ipcManager.createInstance('node', 'app.js', 'nodeApp');
-ipcInstance.on('message', (message) => console.log(message));
+import { loadAbility } from '@kadi.build/core';
+
+const ability = await loadAbility('ability-name', 'protocol');
 ```
 
+**Returns:** A proxy object with:
+
+- Direct method calls (e.g., `ability.echo({ message: 'hello' })`)
+- `ability.events` - EventEmitter for subscribing to events (native/stdio protocols only)
+- `ability.__call(method, params)` - Direct RPC call
+
+### Utility Functions
+
+```javascript
+import {
+  createLogger,
+  getProjectJSON,
+  getAbilityJSON,
+  getBrokerUrl,
+  runExecCommand
+} from '@kadi.build/core';
+```
+
+### Debug Mode
+
+Enable detailed logging:
+
+```bash
+DEBUG=kadi:* node index.js
+```
+
+Check ability logs:
+
+```bash
+tail -f abilities/my-ability/1.0.0/my-ability.log
+```
+
+## 🤝 Contributing
+
+We welcome contributions! Please see our [Contributing Guide](CONTRIBUTING.md) for details.
+
+1. Fork the repository
+2. Create your feature branch (`git checkout -b feature/amazing-feature`)
+3. Commit your changes (`git commit -m 'Add amazing feature'`)
+4. Push to the branch (`git push origin feature/amazing-feature`)
+5. Open a Pull Request
+
+
+
+## 🔗 Related Projects
+
+- [@kadi.build/cli](https://gitlab.com/humin-game-lab/kadi/kadi) - Command-line interface
+- [@kadi.build/broker](https://gitlab.com/humin-game-lab/kadi/kadi-broker) - The KADI broker
+
 ---
 
-## TODO:
-
-- Once @kadi.build/core and dependencies are done, @kadi.build/cli (and all commands), need to be rewritten, using the @kadi.build/core module. this will be v1.0 release
-
-* [ ] add getConnectedAgents to the Broker, it should call the IBroker method
-
-* [x] add key to IBroker object
-* [ ] IBroker on 'message' should listen for setup event, and update teh uuid and key for the IBroker object
-
-* [ ] Extract Broker and IPC to their own node module and host on npm
-* [x] Publish @kadi.build/core to npm
-* [x] Setup of python environment needs to happen once an IPC Inteface is created
-      This include setting up the virtual environment and installing the required packages
-* [x] Add method to IPCManger to set IPCInterfaces from agent.json. If IPCManger has empty list, it thorws an error that no IAbilityIPC's can be created. IPCManger passes the proper interface command to IAbilityIPC on creation from this list, IAbilityIPC does not need access directly to agent.json. Controlling app can update this list anytime a new IAbilityInterface is created, allowing the agent.json to be updated during runtime.
-* [x] Need to adapt ipc.js to read from the @kadi.build/core folder when trying to open IPC instances, instead of project folder. @kadi.build/core should be in the npm_modules folder (when deployed)
-* [x] The test code needs to fully test the IPCManger interface, and not just the IPCAbility
-* [x] If node fails, we need to make sure python IPC interface and child processes also close out.
-* [x] Add buffer to error handeling in IAbilityIPC, mimic same process used in std input
-* [x] Verify if Python IAbilityIPC needs to utilize buffering, and node passing '\n' at end of jso
-* [x] Setup passthrough for broker messages to/from Language IPC interface
-* [x] Add broker message builder/parser in python IAbilityIPC
-* [x] Update BrokerMessageBuilder to use same format at IPCMessageBuilder
-* [x] Update Broker so it extends EventEmitter, rather than have a property of event emitter (similar to IAbilityIPC)
-* [x] update python IAbility so that it creates a timestamped command_output.log file, new file for each run, in the project root directory... not the @kadi.build/core directory
-* [x] Comment out debug messages
-* [x] Add ability to allow for launched process to be piped to event handler (launch command should default to process, but allow user to set flag that will save all outputs to file in the event handler)
+Built with ❤️ by the KADI team