@j-o-r/hello-dave 0.0.7 → 0.0.8

package/docs/generic-toolset.md

@@ -0,0 +1,326 @@
+# Generic Toolset Module (lib/genericToolset.js)
+
+## Overview
+
+The `genericToolset.js` module provides a pre-built collection of 11 utility tools designed for integration with the `ToolSet` class from the hello-dave library. These tools handle common operations like code execution, file I/O, system interactions, and memory management. They are exported as `toolsPool`, an object mapping tool names to `TSTool` objects (containing description, parameters schema, and execution method).
+
+This module is ideal for toolsets or utility libraries, enabling agents to perform real-world tasks without custom implementations. Tools follow OpenAI-style JSON Schema for parameters and support asynchronous execution.
+
+**Key Benefits**:
+- **Ready-to-Use**: Load into any `ToolSet` instance for immediate functionality.
+- **Secure & Sandboxed**: Tools operate in controlled environments (e.g., specific CWD, no escaping in Bash).
+- **Persistent State**: Memory tools allow offloading state from LLM context for efficiency.
+- **Documentation**: Each tool includes JSDoc annotations for IDE support and generated docs.
+
+To generate full JSDoc for this module (and the library):
+1. Install JSDoc: `npm install -g jsdoc`
+2. Run: `jsdoc lib/genericToolset.js -d docs/jsdoc/generic-toolset`
+3. View HTML docs in `docs/jsdoc/generic-toolset/index.html`.
+
+See [ToolSet Documentation](../toolset.md) for integration details.
+
+## Exports
+
+- **`toolsPool`**: `{ [name: string]: TSTool }` – Object with 11 pre-defined tools.
+  - `TSTool`: `{ description: string, parameters: TSSchema, method: async (params: object) => any }`
+  - `TSSchema`: JSON Schema object (e.g., `{ type: "object", properties: {...}, required: [...] }`).
+
+No default export; use named import: `import { toolsPool } from "@j-o-r/hello-dave/lib/genericToolset.js";`
+
+## Usage Examples
+
+### Basic Import and Population of ToolSet
+```javascript
+import { ToolSet } from "@j-o-r/hello-dave";
+import { toolsPool } from "@j-o-r/hello-dave/lib/genericToolset.js";
+
+const toolset = new ToolSet("auto"); // Or "required" for empty start
+
+// Add all generic tools
+Object.entries(toolsPool).forEach(([name, tool]) => {
+  toolset.add(name, tool.description, tool.parameters, tool.method);
+});
+
+// Or selectively
+toolset.add("memory_recall", toolsPool.memory_recall.description, toolsPool.memory_recall.parameters, toolsPool.memory_recall.method);
+
+// List available tools
+console.log(toolset.list());
+```
+
+### Integration with AgentManager
+```javascript
+import { AgentManager } from "@j-o-r/hello-dave";
+
+const agent = new AgentManager("my-agent");
+agent.setup({
+  toolsetMode: "auto" // Automatically loads all tools from toolsPool
+});
+
+// Add specific tools post-setup
+agent.addGenericToolcall("execute_bash_script");
+agent.addGenericToolcall("memory_write");
+```
+
+### Example: Using Memory Tools in a Prompt Workflow
+```javascript
+import { Prompt } from "@j-o-r/hello-dave";
+import { toolsPool } from "@j-o-r/hello-dave/lib/genericToolset.js";
+
+const prompt = new Prompt("You are a task manager. Use memory tools to track progress.");
+const toolset = new ToolSet("required");
+toolset.add("memory_write", ...toolsPool.memory_write); // Spread for destructuring
+
+// In agent loop: prompt.addMessage("Start new task: Implement login.");
+await toolset.execute(prompt); // LLM may call tools based on prompt
+```
+
+## Key Tools
+
+Below is a list of the 11 tools in `toolsPool`, with descriptions, parameter schemas, and usage examples. For full JSDoc details, generate docs as noted above.
+
+### 1. `javascript_interpreter`
+**Description**: Execute ESM ES6 JavaScript in a Node.js environment. Captures `console.log` output. CWD: `~/devpri/js/hello-dave`.
+
+**Parameters Schema**:
+```json
+{
+  "type": "object",
+  "properties": {
+    "code": { "type": "string", "description": "JavaScript code to execute" }
+  },
+  "required": ["code"]
+}
+```
+
+**Usage Example**:
+```javascript
+await toolset.call("javascript_interpreter", { code: "console.log(2 + 2);" });
+// Returns: { output: "4\n", error: null }
+```
+
+### 2. `get_user_env`
+**Description**: Retrieve user environment details (name, system info, location, IP).
+
+**Parameters Schema**:
+```json
+{
+  "type": "object",
+  "properties": {},
+  "required": []
+}
+```
+
+**Usage Example**:
+```javascript
+await toolset.call("get_user_env", {});
+// Returns: { name: "User", system: "Ubuntu 25.10", city: "Example City", ... }
+```
+
+### 3. `execute_bash_script`
+**Description**: Run raw Bash scripts/commands verbatim (no escaping). Supports Ubuntu 25.10 environment.
+
+**Parameters Schema**:
+```json
+{
+  "type": "object",
+  "properties": {
+    "bash_script": { "type": "string", "description": "Raw Bash script (literal text)" }
+  },
+  "required": ["bash_script"]
+}
+```
+
+**Usage Example**:
+```javascript
+await toolset.call("execute_bash_script", { bash_script: "ls -la" });
+// Returns: { stdout: "...", stderr: "", exitCode: 0 }
+```
+
+### 4. `send_email`
+**Description**: Send email using `msmtp` (to, subject, body).
+
+**Parameters Schema**:
+```json
+{
+  "type": "object",
+  "properties": {
+    "to": { "type": "string" },
+    "subject": { "type": "string" },
+    "body": { "type": "string" }
+  },
+  "required": ["to", "subject", "body"]
+}
+```
+
+**Usage Example**:
+```javascript
+await toolset.call("send_email", { to: "user@example.com", subject: "Hello", body: "Test" });
+// Returns: { success: true }
+```
+
+### 5. `open_link`
+**Description**: Open URL or file using `xdg-open`.
+
+**Parameters Schema**:
+```json
+{
+  "type": "object",
+  "properties": {
+    "url": { "type": "string", "description": "URL or file path" }
+  },
+  "required": ["url"]
+}
+```
+
+**Usage Example**:
+```javascript
+await toolset.call("open_link", { url: "https://example.com" });
+// Returns: { opened: true }
+```
+
+### 6. `execute_remote_script`
+**Description**: Execute Bash script on remote server via SSH (`ssh://user@host[:port]`).
+
+**Parameters Schema**:
+```json
+{
+  "type": "object",
+  "properties": {
+    "remote": { "type": "string", "description": "SSH URL" },
+    "script": { "type": "string", "description": "Bash script" }
+  },
+  "required": ["remote", "script"]
+}
+```
+
+**Usage Example**:
+```javascript
+await toolset.call("execute_remote_script", { remote: "ssh://user@host", script: "whoami" });
+// Returns: { stdout: "user\n", ... }
+```
+
+### 7. `history_search`
+**Description**: Search chat history sessions using regex (e.g., in `.cache/[app]/[prompt]/sessions/*.ndjson`).
+
+**Parameters Schema**:
+```json
+{
+  "type": "object",
+  "properties": {
+    "query": { "type": "string", "description": "Regex query" }
+  },
+  "required": ["query"]
+}
+```
+
+**Usage Example**:
+```javascript
+await toolset.call("history_search", { query: package.json });
+// Returns: [{ session: "path", matches: [...] }, ...]
+```
+
+### 8. `read_file`
+**Description**: Read file content from CWD (relative path, no absolute/parent traversal).
+
+**Parameters Schema**:
+```json
+{
+  "type": "object",
+  "properties": {
+    "path": { "type": "string", "description": "Relative file path" }
+  },
+  "required": ["path"]
+}
+```
+
+**Usage Example**:
+```javascript
+await toolset.call("read_file", { path: "example.txt" });
+// Returns: { content: "File text\n", encoding: "utf8" }
+```
+
+### 9. `write_file`
+**Description**: Write raw content to file in CWD (relative path). Returns bytes written.
+
+**Parameters Schema**:
+```json
+{
+  "type": "object",
+  "properties": {
+    "path": { "type": "string" },
+    "content": { "type": "string" }
+  },
+  "required": ["path", "content"]
+}
+```
+
+**Usage Example**:
+```javascript
+await toolset.call("write_file", { path: "output.txt", content: "Hello World" });
+// Returns: { bytesWritten: 11 }
+```
+
+### 10. `memory_recall`
+**Description**: Recall latest values for keys from agent-specific memory (`.cache/[agent]/memory.ndjson`).
+
+**Parameters Schema**:
+```json
+{
+  "type": "object",
+  "properties": {
+    "keys": {
+      "type": "array",
+      "items": { "type": "string" },
+      "description": "List of keys to recall"
+    }
+  },
+  "required": ["keys"]
+}
+```
+
+**Usage Example**:
+```javascript
+await toolset.call("memory_recall", { keys: ["current_task"] });
+// Returns: { current_task: "Implement login" } or {}
+```
+
+### 11. `memory_write`
+**Description**: Append/update key-value pairs in agent memory (overwrites same keys).
+
+**Parameters Schema**:
+```json
+{
+  "type": "object",
+  "properties": {
+    "entries": {
+      "type": "array",
+      "items": {
+        "type": "object",
+        "properties": {
+          "key": { "type": "string" },
+          "value": { "type": "string" }
+        },
+        "required": ["key", "value"]
+      }
+    }
+  },
+  "required": ["entries"]
+}
+```
+
+**Usage Example**:
+```javascript
+await toolset.call("memory_write", { entries: [{ key: "step", value: "1/5" }] });
+// Returns: { stored: [{ key: "step", value: "1/5" }] }
+```
+
+## Best Practices
+- **Security**: Tools like `execute_bash_script` and `write_file` are powerful; use in trusted environments.
+- **Error Handling**: Methods return `{ success: boolean, error?: string, data: any }` for robustness.
+- **Customization**: Extend `toolsPool` by adding your own tools before populating `ToolSet`.
+- **JSDoc Integration**: Annotations include `@param`, `@returns`, `@example` for full type safety in editors like VS Code.
+
+For syntax validation in tools (e.g., JS in `write_file`), see [Tools Syntax Validation](../tools-syntax-validation.md). For path handling, see [Path Resolution Best Practices](../path-resolution-best-practices.md).
+
+**Updated**: April 10, 2026

package/docs/howtos/agent-networking.md

@@ -0,0 +1,253 @@
+# Agent Networking How-Tos
+
+## Table of Contents
+- [Introduction](#introduction)
+- [Current Howtos Files](#current-howtos-files)
+- [Overview of Agent Networking](#overview-of-agent-networking)
+- [Setup Steps](#setup-steps)
+- [Using --serve Flag](#using--serve-flag)
+- [Using --connect Flag](#using--connect-flag)
+- [Multi-Agent Chains via CodeServer](#multi-agent-chains-via-codeserver)
+- [Examples](#examples)
+- [Bash Examples from codeserver.sh](#bash-examples-from-codeserver-sh)
+- [JavaScript Fetch Examples](#javascript-fetch-examples)
+- [Troubleshooting](#troubleshooting)
+- [Related Documentation](#related-documentation)
+
+## Introduction
+This guide provides accurate how-tos for networking agents in the hello-dave project. Agent communication uses WebSocket servers for real-time interaction, primarily through the CodeServer setup. The CLI entrypoint is `bin/dave.js` (or `node bin/dave.js`), with flags like `--serve` (for starting a WebSocket server on individual agents like `code_agent.js`), `--connect` (to chain to a server), and `--ask` (for querying agents locally or remotely). Multi-agent chains are orchestrated via `agents/codeserver.sh`, which launches a PM2 cluster of interconnected agents.
+
+Networking enables specialized agents (e.g., memory_agent, todo_agent, code_agent) to collaborate by connecting to a central server, supporting complex workflows like code generation with memory and testing.
+
+## Current Howtos Files
+After the urgent fix and deletion of the inaccurate previous file:
+- The `./docs/howtos/` directory was recreated and now contains only this file: `agent-networking.md` (newly created based on real project structure).
+- No other `.md` files are present.
+
+## Overview of Agent Networking
+In hello-dave, networking is WebSocket-based:
+- **Server Mode (--serve)**: Starts a WebSocket server (e.g., on `code_agent.js`) for other agents to connect to.
+- **Client Mode (--connect)**: Agents connect to a WebSocket server (e.g., `ws://localhost:8080/ws`) to forward requests and share state.
+- **Primary Example**: `agents/codeserver.sh` spawns a central `code_agent` server and connects helper agents (todo, memory, etc.) to it, forming a multi-agent chain.
+- No standalone `--serve` on `bin/dave.js`; it's used on individual agent scripts (e.g., `node code_agent.js --serve 8080`).
+- Interactions: Use `bin/dave.js --connect ws://... --secret ...` for remote queries, or pipe inputs for one-shot actions.
+- Chaining: Outputs from connected agents (e.g., memory_agent recall) feed into the main agent via WebSocket messages.
+
+This setup is useful for scalable agent collaboration but is centered around the CodeServer pattern. For broader networking, consider extending with custom WebSocket handlers.
+
+## Setup Steps
+1. **Prerequisites**:
+   - Node.js (v18+), PM2 (`npm install -g pm2`), and project dependencies: `npm install`.
+   - Set environment: `export XAIKEY=your_xai_api_key` for Grok API access (required for local asks).
+   - Ensure ports (e.g., 8080) are free and firewall allows localhost connections.
+
+2. **Project Structure**:
+   - CLI: `bin/dave.js` for high-level commands.
+   - Agents: Individual scripts in root (e.g., `code_agent.js`, `memory_agent.js`).
+   - Orchestrator: `agents/codeserver.sh` for multi-agent setup.
+
+3. **Start a Basic Server**:
+   - Run an individual agent in server mode: `node code_agent.js --serve 8080 --secret 123 --tools javascript`.
+
+4. **Connect and Interact**:
+   - Use `bin/dave.js --connect ws://localhost:8080/ws --secret 123` for interactive mode.
+   - Or pipe: `echo "Write a function" | bin/dave.js --connect ws://localhost:8080/ws --secret 123`.
+
+## Using --serve Flag
+The `--serve` flag is supported on individual agent scripts (e.g., `code_agent.js`) to start a WebSocket server. It enables the agent to act as a hub for connected clients.
+
+### Syntax
+```
+node [agent-script].js --serve [PORT] --secret [SECRET] [--tools [TOOLSET]]
+```
+- `PORT`: WebSocket port (default 8080).
+- `SECRET`: Authentication token (min 3 chars, default '123').
+- `--tools`: Optional toolset (e.g., 'javascript' for code_agent).
+
+### Example: Serving Code Agent
+```
+node code_agent.js --serve 8080 --secret mysecret --tools javascript
+```
+Output:
+```
+Code Agent WebSocket server started at ws://localhost:8080/ws
+Waiting for connections...
+```
+The server exposes `/ws` endpoint for WebSocket connections. Connected agents can send actions like `user_request`, `user_info`, `user_reset`.
+
+## Using --connect Flag
+The `--connect` flag (on `bin/dave.js` or agent scripts) connects to a WebSocket server, enabling remote interaction or chaining.
+
+### Syntax
+```
+bin/dave.js --connect [ws://URL] --secret [SECRET]
+```
+Or on agents:
+```
+node [agent-script].js --connect [ws://URL] --secret [SECRET]
+```
+- Supports interactive mode or piped input (e.g., `echo "query" | bin/dave.js --connect ...`).
+- Actions: `user_request` (default for queries), `user_info`, `user_reset`.
+
+### Example: Connecting to CodeServer
+Interactive:
+```
+bin/dave.js --connect ws://localhost:8080/ws --secret 123
+```
+Piped:
+```
+echo "Generate code for a todo app" | bin/dave.js --connect ws://localhost:8080/ws --secret 123
+```
+This forwards the request to the connected server, leveraging any chained agents.
+
+## Multi-Agent Chains via CodeServer
+The primary multi-agent setup is via `agents/codeserver.sh`, which launches a PM2 cluster:
+- Central: `code_agent` in `--serve` mode.
+- Helpers: `todo_agent`, `memory_agent`, `readme_agent`, `npm_agent`, `docs_agent`, `test_agent` all `--connect` to the central WS.
+- Chain Flow: Input → code_agent → delegates to connected agents (e.g., memory for recall, todo for planning, test for validation) → response.
+
+This forms implicit chains, e.g., code_agent queries memory_agent for context before generating code.
+
+### Chain Example: Memory → Todo → Code
+- memory_agent stores/recalls state.
+- todo_agent manages tasks based on memory.
+- code_agent generates code from todo lists, with all connected to the same WS hub.
+
+Orchestration happens internally via WebSocket message passing.
+
+## Examples
+### Basic Server + Connect
+1. Start server:
+   ```
+   node code_agent.js --serve 8080 --secret 123
+   ```
+2. Connect interactively:
+   ```
+   bin/dave.js --connect ws://localhost:8080/ws --secret 123
+   ```
+   Type queries; responses use the code_agent's capabilities.
+
+### Remote Ask via Connect
+```
+bin/dave.js --ask --connect ws://localhost:8080/ws --secret 123
+```
+Combines local asking with remote execution.
+
+## Bash Examples from codeserver.sh
+The `agents/codeserver.sh` script is the canonical multi-agent launcher. It:
+- Validates port (1024-65535) and secret (≥3 chars).
+- Deletes old PM2 processes.
+- Spawns central `code_agent --serve`.
+- Connects helpers: todo, readme, npm, docs, test, memory.
+
+### Full Script Usage
+```
+cd hello-dave
+./agents/codeserver.sh 8080 mysecret
+```
+Output:
+```
+Starting CodeServer on port 8080 in folder 'hello-dave' ... with SECRET 'mysecret'...
+CodeServer processes spawned with prefix 'hello-dave_' and suffix _8080. Check with: pm2 list | grep 'hello-dave_8080'
+dave --connect ws://127.0.0.1:8080/ws --secret 'mysecret'
+```
+
+### Custom Bash Chain Script
+Create `start-chain.sh` (inspired by codeserver.sh):
+```bash
+#!/bin/bash
+PORT=${1:-8080}
+SECRET=${2:-123}
+
+# Start central server
+node code_agent.js --serve $PORT --secret $SECRET --tools javascript &
+SERVER_PID=$!
+
+# Connect helpers
+node memory_agent.js --connect "ws://127.0.0.1:$PORT/ws" --secret $SECRET &
+node todo_agent.js --connect "ws://127.0.0.1:$PORT/ws" --secret $SECRET &
+
+# Interact
+echo "Build a simple app" | bin/dave.js --connect "ws://127.0.0.1:$PORT/ws" --secret $SECRET
+
+# Cleanup (optional)
+# kill $SERVER_PID
+```
+Run: `chmod +x start-chain.sh && ./start-chain.sh 8080 mysecret`
+
+Use PM2 for production: `pm2 start ecosystem.config.js` (define processes similarly).
+
+## JavaScript Fetch Examples
+For programmatic chaining, use native `fetch` (Node.js 18+) to interact with WS (via WebSocket library) or HTTP endpoints if exposed. However, hello-dave uses pure WS; for fetch, assume agent servers expose REST if customized. Here's a basic WS client example using `ws` library (`npm install ws`).
+
+### JS WS Client Chain
+Create `chain-client.js`:
+```javascript
+const WebSocket = require('ws');
+
+async function connectAndChain(url, secret, input) {
+  return new Promise((resolve, reject) => {
+    const ws = new WebSocket(url);
+
+    ws.on('open', () => {
+      // Send user_request (simulates piped input)
+      ws.send(JSON.stringify({ action: 'user_request', input, secret }));
+
+      let response = '';
+      ws.on('message', (data) => {
+        const msg = JSON.parse(data);
+        if (msg.type === 'content') {
+          response += msg.content;
+          if (msg.done) {
+            ws.close();
+            resolve(response);
+          }
+        }
+      });
+
+      ws.on('error', reject);
+    });
+  });
+}
+
+// Chain: Query memory via todo, then code
+async function exampleChain() {
+  const url = 'ws://localhost:8080/ws';
+  const secret = '123';
+
+  // Step 1: Recall memory (via connected memory_agent)
+  const memory = await connectAndChain(url, secret, 'Recall last task');
+  console.log('Memory:', memory);
+
+  // Step 2: Add to todo
+  const todo = await connectAndChain(url, secret, `Add task from memory: ${memory}`);
+  console.log('Todo:', todo);
+
+  // Step 3: Generate code
+  const code = await connectAndChain(url, secret, `Generate code for: ${todo}`);
+  console.log('Code:', code);
+}
+
+exampleChain().catch(console.error);
+```
+Run: `node chain-client.js` (after starting CodeServer).
+
+Note: Actual message format matches project's `wsIO` (action: 'user_request', input).
+
+## Troubleshooting
+- **Connection Refused**: Verify server is running (`node code_agent.js --serve 8080`) and port open (`netstat -tuln | grep 8080`). Check WS URL format: `ws://localhost:8080/ws`.
+- **Secret Mismatch**: Ensure `--secret` matches on connect/serve (case-sensitive, ≥3 chars). Error: "Invalid secret".
+- **PM2 Issues**: List processes: `pm2 list | grep code_agent`. Delete: `pm2 delete hello-dave_code_agent_8080`. Restart: `pm2 restart all`.
+- **No Response in Chain**: Add logging to agents (e.g., `console.log` in handlers). Verify connections: `pm2 logs hello-dave_*_8080`.
+- **Port Conflicts**: Change port in `codeserver.sh` call. Avoid <1024 ports.
+- **WebSocket Errors**: Use `wscat` for testing: `npm install -g wscat; wscat -c ws://localhost:8080/ws` (manual auth may vary).
+- **XAIKEY Missing**: For local `--ask`, set `export XAIKEY=sk-...`.
+- **Version/Deps**: Ensure Node ≥18, `npm update`. Check agent scripts for flag support.
+
+If networking needs extend beyond CodeServer (e.g., peer-to-peer), note in TODO.md for future enhancements. For now, this covers the project's real capabilities.
+
+## Related Documentation
+- [Project README](../README.md) (CLI flags, setup).
+- [Agent Scripts](../ (list individual agents like code_agent.js)).
+- [PM2 Guide](https://pm2.keymetrics.io/docs/usage/quick-start/) (for cluster management).
+- [WebSocket in Node.js](https://nodejs.org/api/websocket.html) (low-level details).