@j-o-r/hello-dave 0.0.7 → 0.0.8

package/docs/multi-agent-clusters.md

@@ -0,0 +1,265 @@
+# Multi-Agent Clusters in Dave: v0.0.9
+
+Welcome to the exciting world of **Multi-Agent Clusters** in the Dave framework! 🚀 If you're ready to scale your AI agents into powerful, collaborative clusters, you've come to the right place. This guide walks you through setting up CodeServer with PM2, spawning agents, launching in server mode, connecting clients, querying sessions, managing with PM2, testing, and more. By the end, you'll have a robust cluster humming along, enabling self-aware, interconnected agents. Let's dive in step-by-step!
+
+Note: Dave v0.0.9 is fully ESM (ECMAScript Modules) consistent. All JavaScript examples use `import`/`export` syntax—no CommonJS `require`/`module.exports`.
+
+## CodeServer PM2 Setup
+
+CodeServer is the backbone of your multi-agent cluster, providing a WebSocket-based server for agent communication. It integrates seamlessly with PM2 for process management, ensuring high availability and easy scaling.
+
+### Prerequisites
+- Node.js (v18+ recommended)
+- PM2 installed globally: `npm install -g pm2`
+- Dave project cloned and dependencies installed: `npm install`
+
+### Step-by-Step Setup
+1. **Choose Your Launch Method**:
+   - Use the binary script: `./bin/codeDave` (convenient for quick starts).
+   - Or the shell script: `./agents/codeserver.sh` (for more customization).
+
+2. **Default Configuration**:
+   - **Port**: 9000 (change with `--port 8080` if needed).
+   - **Secret**: "123" (for secure WebSocket connections; override with `--secret your_secret`).
+
+3. **Launch CodeServer with PM2**:
+   Run the following to start CodeServer as a PM2-managed process:
+   ```
+   pm2 start ./bin/codeDave --name "hello-dave_code_9000" -- --serve --port 9000 --secret 123
+   ```
+   - `--serve`: Enables server mode (detailed below).
+   - This creates a process named `hello-dave_code_9000` for easy identification.
+
+4. **Verify Setup**:
+   - Check PM2 status: `pm2 list` (look for `hello-dave_code_9000` in "online" status).
+   - Test connection: Use a WebSocket client to connect to `ws://127.0.0.1:9000/ws` with secret "123".
+
+## Creating Agents via spawn_agent
+
+Spawning agents is where the magic happens! Use the `spawn_agent` function to dynamically create and integrate new agents into your cluster. In v0.0.9, spawning is primarily via CLI for seamless integration with the ESM project structure.
+
+### Reference to spawn_agent Blueprint
+For a deep dive into `spawn_agent`, check the [spawn_agent blueprint](../blueprints/spawn-agent.md) (or implement based on the core Dave agent manager patterns in [agent-manager.md](./agent-manager.md)).
+
+### Step-by-Step Agent Creation
+1. **CLI Spawning (Recommended)**:
+   Use the `./agents/spawn_agent.js` script to spawn agents directly from the command line, attaching them to the cluster:
+   ```
+   ./agents/spawn_agent.js "Create weather predictor agent with prompt: You are a weather prediction agent. Use tools to fetch data." --connect ws://127.0.0.1:9000/ws --secret 123 --tools web_search,weather_api
+   ```
+   - This spawns the agent (e.g., named "weatherPredictor" based on description), registers it with CodeServer, and enables collaborative querying.
+   - Flags: `--connect` for cluster attachment, `--secret` for authentication, `--tools` for tool assignment.
+
+2. **Integration with Cluster**:
+   - Agents auto-register with CodeServer upon spawn via the CLI.
+   - For programmatic control in ESM scripts, import and use as shown in examples below.
+
+## Launching Server Mode (--serve)
+
+Server mode turns CodeServer into a persistent hub for multi-agent interactions.
+
+1. **Command**:
+   ```
+   ./bin/codeDave --serve --port 9000 --secret 123
+   ```
+
+2. **What Happens**:
+   - Listens on WebSocket endpoint `/ws`.
+   - Handles agent registrations and session management.
+   - Supports multiple concurrent clients.
+
+3. **PM2 Integration**:
+   As shown in setup, wrap it in PM2 for daemonization and auto-restart.
+
+## Attaching Clients (--connect)
+
+Clients (like other Dave instances or tools) connect to the cluster for tool calls and session sharing.
+
+1. **Basic Connection**:
+   ```
+   ./bin/dave.js --connect ws://127.0.0.1:9000/ws --secret 123
+   ```
+   - This attaches the client as a tool-calling endpoint.
+   - Use in scripts for agent-to-agent communication.
+
+2. **Advanced Usage**:
+   - Pipe inputs: `echo "query" | ./bin/dave.js --connect ...`
+   - Handle toolcalls: Clients receive and execute tools on behalf of the cluster.
+
+## Querying via bin/dave.js --connect
+
+For one-shot queries or building sessions, use `bin/dave.js` with connection flags.
+
+### One-Shot Example
+Predict weather with a quick query:
+```
+echo "predict weather in NYC" | ./bin/dave.js --connect ws://127.0.0.1:9000/ws --secret 123
+```
+- Builds a temporary session, routes to relevant agents (e.g., weatherPredictor), and returns results.
+
+### Session Building
+For persistent sessions:
+```
+./bin/dave.js --connect ws://127.0.0.1:9000/ws --secret 123 --session my_weather_session
+```
+- Follow with queries to maintain state across interactions.
+
+## PM2 Start/Stop/Reload
+
+PM2 makes cluster management a breeze!
+
+1. **List Processes**:
+   ```
+   pm2 list
+   ```
+   - Shows status of `hello-dave_code_9000` and other agents.
+
+2. **Start/Stop**:
+   ```
+   pm2 start hello-dave_code_9000  # Start
+   pm2 stop hello-dave_code_9000   # Stop
+   pm2 restart hello-dave_code_9000 # Restart
+   ```
+
+3. **Reload (Zero-Downtime)**:
+   ```
+   pm2 reload hello-dave_code_9000
+   ```
+   - Ideal for updates without interrupting sessions.
+
+4. **Logs and Monitoring**:
+   ```
+   pm2 logs hello-dave_code_9000
+   pm2 monit
+   ```
+
+## Testing One-Shots and Sessions
+
+Ensure your cluster is rock-solid with these tests.
+
+### One-Shot Testing
+1. Spawn a test agent.
+2. Run: `echo "test query" | ./bin/dave.js --connect ...`
+3. Verify output matches expected (e.g., no errors, relevant response).
+
+### Session Testing
+1. Start a session: `./bin/dave.js --connect ... --session test`
+2. Send multiple queries: `echo "first" | ...` then `echo "follow-up" | ...`
+3. Check session persistence (e.g., context retained).
+
+### End-to-End Test Script
+```bash
+#!/bin/bash
+# test_cluster.sh
+pm2 start ./bin/codeDave --name "test_code" -- --serve --port 9000 --secret 123
+sleep 2
+echo "Hello cluster!" | ./bin/dave.js --connect ws://127.0.0.1:9000/ws --secret 123
+pm2 stop test_code
+```
+
+## Self-Awareness: Detecting PM2/CodeServer in Prompts
+
+Make your agents smarter by embedding cluster awareness! In v0.0.9, self-awareness leverages argument detection and initial PM2 inspection for accurate blueprint compliance.
+
+- **In Agent Prompts**:
+  Include detection logic: "If running under PM2/CodeServer (detect via args like --serve/--connect or initial `pm2 list` inspect), coordinate with cluster agents."
+
+- **Implementation** (ESM Syntax):
+  Agents auto-detect via argument parsing and PM2 inspection:
+  ```javascript
+  import { execSync } from 'child_process';
+  import { process } from 'process'; // Built-in
+
+  const args = process.argv.slice(2);
+  const isServeMode = args.includes('--serve');
+  const isConnectMode = args.includes('--connect');
+  const pm2Status = execSync('pm2 list', { encoding: 'utf8' }).includes('hello-dave_code_9000');
+
+  if (isServeMode || isConnectMode || pm2Status) {
+    // Cluster mode: Share context, route to other agents
+    console.log('Detected cluster environment');
+  }
+  ```
+  This enables self-aware routing, e.g., "Delegate weather query to weatherPredictor in cluster." Aligns with blueprint by inspecting PM2 initially for process awareness.
+
+## Mermaid Diagram: Cluster Architecture
+
+Visualize your setup!
+
+```mermaid
+graph TD
+    A[PM2 Manager] --> B[CodeServer<br/>Port 9000 /ws]
+    B --> C[Agent 1<br/>spawn_agent()]
+    B --> D[Agent 2<br/>e.g., WeatherPredictor]
+    E[Client / bin/dave.js] -->|connect| B
+    E -->|one-shot| F[Session Builder]
+    G[Tools: web_search, etc.] <-->|toolcalls| C
+    G <-->|toolcalls| D
+    style B fill:#f9f,stroke:#333,stroke-width:2px
+```
+
+- **Nodes**: PM2 oversees CodeServer, which hubs agents and clients.
+- **Flows**: Connections for spawning, querying, and tool execution.
+
+## Examples
+
+### Full ESM Node Script for Custom Agent Launch (--serve)
+Here&apos;s a complete ESM script (`custom-agent-launch.mjs`) to spawn and launch a custom agent in server mode:
+
+```javascript
+#!/usr/bin/env node
+// custom-agent-launch.mjs - ESM for v0.0.9
+
+import { spawn_agent } from &apos;./agents/spawn.js&apos;; // ESM import
+import { connectToCluster } from &apos;./utils/cluster.js&apos;; // Hypothetical utility
+
+async function main() {
+  const agentConfig = {
+    name: &quot;queryMaster&quot;,
+    prompt: &quot;You are a master query router in the cluster.&quot;,
+    tools: [&apos;web_search&apos;],
+    cluster: &quot;ws://127.0.0.1:9000/ws&quot;
+  };
+
+  // Spawn the agent
+  const agent = await spawn_agent(agentConfig);
+  
+  // Connect to cluster if in server mode
+  if (process.argv.includes(&apos;--serve&apos;)) {
+    await connectToCluster(agent, { secret: &quot;123&quot; });
+    console.log(&apos;Agent launched in server mode!&quot;);
+  }
+}
+
+main().catch(console.error);
+```
+
+Run it: `node custom-agent-launch.mjs --serve`
+
+### Full Cluster Spawn and Query (CLI + ESM)
+1. Start CodeServer: `pm2 start ./bin/codeDave --name code_9000 -- --serve --port 9000 --secret 123`
+2. Spawn Agent (CLI):
+   ```
+   ./agents/spawn_agent.js &quot;Create queryMaster: You are a master query router.&quot; --connect ws://127.0.0.1:9000/ws --secret 123
+   ```
+3. Query: `echo &quot;What&apos;s the plan?&quot; | ./bin/dave.js --connect ws://127.0.0.1:9000/ws --secret 123`
+
+### Multi-Agent Collaboration
+- Spawn multiple: weather, news, summary agents.
+- Query: &quot;Summarize today&apos;s news and weather&quot; → Routes to respective agents via cluster.
+
+## Troubleshooting
+
+- **Connection Refused**: Ensure CodeServer is running (`pm2 list`) and port 9000 is free (`lsof -i :9000`).
+- **Secret Mismatch**: Double-check `--secret` flags match.
+- **PM2 Crashes**: View logs (`pm2 logs`)—common issues: missing deps or port conflicts. Restart: `pm2 restart all`.
+- **Agent Not Attaching**: Verify `spawn_agent` cluster URL and secret. Test WebSocket manually.
+- **Session Loss**: Check PM2 memory limits (`pm2 desc code_9000`); increase if needed.
+- **One-Shot Fails**: Ensure input is piped correctly; debug with `--verbose`.
+- **ESM Import Errors**: Ensure `package.json` has &quot;type&quot;: &quot;module&quot; and files use `.mjs` or `.js` with ESM syntax.
+
+If issues persist, reference [project-overview.md](./project-overview.md) or spawn a debug agent!
+
+---
+
+*Ready for v0.0.9 deployment! Cluster up and conquer those multi-agent challenges. 🎉 Questions? Ping the Dave community.*

package/docs/multi-agent-clusters.md.bak

@@ -0,0 +1,229 @@
+# Multi-Agent Clusters in Dave: v0.0.9
+
+Welcome to the exciting world of **Multi-Agent Clusters** in the Dave framework! 🚀 If you're ready to scale your AI agents into powerful, collaborative clusters, you've come to the right place. This guide walks you through setting up CodeServer with PM2, spawning agents, launching in server mode, connecting clients, querying sessions, managing with PM2, testing, and more. By the end, you'll have a robust cluster humming along, enabling self-aware, interconnected agents. Let's dive in step-by-step!
+
+## CodeServer PM2 Setup
+
+CodeServer is the backbone of your multi-agent cluster, providing a WebSocket-based server for agent communication. It integrates seamlessly with PM2 for process management, ensuring high availability and easy scaling.
+
+### Prerequisites
+- Node.js (v18+ recommended)
+- PM2 installed globally: `npm install -g pm2`
+- Dave project cloned and dependencies installed: `npm install`
+
+### Step-by-Step Setup
+1. **Choose Your Launch Method**:
+   - Use the binary script: `./bin/codeDave` (convenient for quick starts).
+   - Or the shell script: `./agents/codeserver.sh` (for more customization).
+
+2. **Default Configuration**:
+   - **Port**: 9000 (change with `--port 8080` if needed).
+   - **Secret**: "123" (for secure WebSocket connections; override with `--secret your_secret`).
+
+3. **Launch CodeServer with PM2**:
+   Run the following to start CodeServer as a PM2-managed process:
+   ```
+   pm2 start ./bin/codeDave --name "hello-dave_code_9000" -- --serve --port 9000 --secret 123
+   ```
+   - `--serve`: Enables server mode (detailed below).
+   - This creates a process named `hello-dave_code_9000` for easy identification.
+
+4. **Verify Setup**:
+   - Check PM2 status: `pm2 list` (look for `hello-dave_code_9000` in "online" status).
+   - Test connection: Use a WebSocket client to connect to `ws://127.0.0.1:9000/ws` with secret "123".
+
+## Creating Agents via spawn_agent
+
+Spawning agents is where the magic happens! Use the `spawn_agent` function to dynamically create and integrate new agents into your cluster.
+
+### Reference to spawn_agent Blueprint
+For a deep dive into `spawn_agent`, check the [spawn_agent blueprint](../blueprints/spawn-agent.md) (or implement based on the core Dave agent manager patterns in [agent-manager.md](./agent-manager.md)).
+
+### Step-by-Step Agent Creation
+1. **Import and Initialize**:
+   In your Node.js script:
+   ```javascript
+   const { spawn_agent } = require('./agents/spawn'); // Adjust path as needed
+   ```
+
+2. **Spawn an Agent**:
+   ```javascript
+   const newAgent = spawn_agent({
+     name: "weatherPredictor",
+     prompt: "You are a weather prediction agent. Use tools to fetch data.",
+     tools: ['web_search', 'weather_api'],
+     cluster: "ws://127.0.0.1:9000/ws" // Attach to your CodeServer
+   });
+   ```
+   - This generates the agent and attaches it to the cluster for collaborative querying.
+
+3. **Integration with Cluster**:
+   - Agents auto-register with CodeServer upon spawn.
+   - Use `newAgent.connect({ secret: "123" })` to ensure secure attachment.
+
+## Launching Server Mode (--serve)
+
+Server mode turns CodeServer into a persistent hub for multi-agent interactions.
+
+1. **Command**:
+   ```
+   ./bin/codeDave --serve --port 9000 --secret 123
+   ```
+
+2. **What Happens**:
+   - Listens on WebSocket endpoint `/ws`.
+   - Handles agent registrations and session management.
+   - Supports multiple concurrent clients.
+
+3. **PM2 Integration**:
+   As shown in setup, wrap it in PM2 for daemonization and auto-restart.
+
+## Attaching Clients (--connect)
+
+Clients (like other Dave instances or tools) connect to the cluster for tool calls and session sharing.
+
+1. **Basic Connection**:
+   ```
+   ./bin/dave.js --connect ws://127.0.0.1:9000/ws --secret 123
+   ```
+   - This attaches the client as a tool-calling endpoint.
+   - Use in scripts for agent-to-agent communication.
+
+2. **Advanced Usage**:
+   - Pipe inputs: `echo "query" | ./bin/dave.js --connect ...`
+   - Handle toolcalls: Clients receive and execute tools on behalf of the cluster.
+
+## Querying via bin/dave.js --connect
+
+For one-shot queries or building sessions, use `bin/dave.js` with connection flags.
+
+### One-Shot Example
+Predict weather with a quick query:
+```
+echo "predict weather in NYC" | ./bin/dave.js --connect ws://127.0.0.1:9000/ws --secret 123
+```
+- Builds a temporary session, routes to relevant agents (e.g., weatherPredictor), and returns results.
+
+### Session Building
+For persistent sessions:
+```
+./bin/dave.js --connect ws://127.0.0.1:9000/ws --secret 123 --session my_weather_session
+```
+- Follow with queries to maintain state across interactions.
+
+## PM2 Start/Stop/Reload
+
+PM2 makes cluster management a breeze!
+
+1. **List Processes**:
+   ```
+   pm2 list
+   ```
+   - Shows status of `hello-dave_code_9000` and other agents.
+
+2. **Start/Stop**:
+   ```
+   pm2 start hello-dave_code_9000  # Start
+   pm2 stop hello-dave_code_9000   # Stop
+   pm2 restart hello-dave_code_9000 # Restart
+   ```
+
+3. **Reload (Zero-Downtime)**:
+   ```
+   pm2 reload hello-dave_code_9000
+   ```
+   - Ideal for updates without interrupting sessions.
+
+4. **Logs and Monitoring**:
+   ```
+   pm2 logs hello-dave_code_9000
+   pm2 monit
+   ```
+
+## Testing One-Shots and Sessions
+
+Ensure your cluster is rock-solid with these tests.
+
+### One-Shot Testing
+1. Spawn a test agent.
+2. Run: `echo "test query" | ./bin/dave.js --connect ...`
+3. Verify output matches expected (e.g., no errors, relevant response).
+
+### Session Testing
+1. Start a session: `./bin/dave.js --connect ... --session test`
+2. Send multiple queries: `echo "first" | ...` then `echo "follow-up" | ...`
+3. Check session persistence (e.g., context retained).
+
+### End-to-End Test Script
+```bash
+#!/bin/bash
+# test_cluster.sh
+pm2 start ./bin/codeDave --name "test_code" -- --serve --port 9000 --secret 123
+sleep 2
+echo "Hello cluster!" | ./bin/dave.js --connect ws://127.0.0.1:9000/ws --secret 123
+pm2 stop test_code
+```
+
+## Self-Awareness: Detecting PM2/CodeServer in Prompts
+
+Make your agents smarter by embedding cluster awareness!
+
+- **In Agent Prompts**:
+  Include detection logic: "If running under PM2/CodeServer (detect via env vars like PM2_PROCESS_ID or connection to ws://localhost:9000), coordinate with cluster agents."
+
+- **Implementation**:
+  Agents auto-detect via:
+  ```javascript
+  if (process.env.PM2_PROCESS_ID || connection.url.includes('9000')) {
+    // Cluster mode: Share context
+  }
+  ```
+  This enables self-aware routing, e.g., "Delegate weather query to weatherPredictor in cluster."
+
+## Mermaid Diagram: Cluster Architecture
+
+Visualize your setup!
+
+```mermaid
+graph TD
+    A[PM2 Manager] --> B[CodeServer<br/>Port 9000 /ws]
+    B --> C[Agent 1<br/>spawn_agent()]
+    B --> D[Agent 2<br/>e.g., WeatherPredictor]
+    E[Client / bin/dave.js] -->|connect| B
+    E -->|one-shot| F[Session Builder]
+    G[Tools: web_search, etc.] <-->|toolcalls| C
+    G <-->|toolcalls| D
+    style B fill:#f9f,stroke:#333,stroke-width:2px
+```
+
+- **Nodes**: PM2 oversees CodeServer, which hubs agents and clients.
+- **Flows**: Connections for spawning, querying, and tool execution.
+
+## Examples
+
+### Full Cluster Spawn and Query
+1. Start CodeServer: `pm2 start ./bin/codeDave --name code_9000 -- --serve --port 9000 --secret 123`
+2. Spawn Agent (in Node script):
+   ```javascript
+   const agent = spawn_agent({ name: &quot;queryMaster&quot;, cluster: &quot;ws://127.0.0.1:9000/ws&quot; });
+   ```
+3. Query: `echo &quot;What&apos;s the plan?&quot; | ./bin/dave.js --connect ws://127.0.0.1:9000/ws --secret 123`
+
+### Multi-Agent Collaboration
+- Spawn multiple: weather, news, summary agents.
+- Query: &quot;Summarize today&apos;s news and weather&quot; → Routes to respective agents via cluster.
+
+## Troubleshooting
+
+- **Connection Refused**: Ensure CodeServer is running (`pm2 list`) and port 9000 is free (`lsof -i :9000`).
+- **Secret Mismatch**: Double-check `--secret` flags match.
+- **PM2 Crashes**: View logs (`pm2 logs`)—common issues: missing deps or port conflicts. Restart: `pm2 restart all`.
+- **Agent Not Attaching**: Verify `spawn_agent` cluster URL and secret. Test WebSocket manually.
+- **Session Loss**: Check PM2 memory limits (`pm2 desc code_9000`); increase if needed.
+- **One-Shot Fails**: Ensure input is piped correctly; debug with `--verbose`.
+
+If issues persist, reference [project-overview.md](./project-overview.md) or spawn a debug agent!
+
+---
+
+*Ready for v0.0.9 deployment! Cluster up and conquer those multi-agent challenges. 🎉 Questions? Ping the Dave community.*

package/docs/path-resolution-best-practices.md

@@ -0,0 +1,104 @@
+# Path Resolution Best Practices for Generic Tools
+
+## Overview
+In `lib/genericToolset.js` and related tools (e.g., `syntax_check`, `write_file`), **path resolution** distinguishes between:
+- **Module-relative paths** (utils/scripts): Use `__dirname` for portability.
+- **User-provided file paths**: Resolve to `process.cwd()` (CWD sandbox) + strict validation.
+
+This ensures **sandboxing** (no escapes), **portability** (works from any CWD/PM2), and **security**. All tools enforce: no `/`, `..`, `\\\\`; `resolvedPath.startsWith(process.cwd())`.
+
+**ESM Setup** (genericToolset.js):
+```javascript
+const __dirname = path.dirname(fileURLToPath(import.meta.url));  // Polyfill for ESM
+```
+
+## __dirname vs process.cwd(): When to Use What
+
+| Context | Resolution | Example Code | Rationale |
+|---------|------------|--------------|-----------|
+| **Utils/Scripts** (e.g., syntax_check.sh, search_sessions.sh) | `__dirname` (module-rel) | `path.resolve(__dirname, '..', 'utils', 'syntax_check.sh')` | Portable: Locates utils/ from `lib/` regardless of CWD. Works in servers/clients. |
+| **User Files** (read/write/syntax/memory) | `process.cwd()` | `path.resolve(process.cwd(), params.file)` | Sandbox: Pins to project CWD (`~/devpri/js/hello-dave`). Prevents jailbreaks. |
+| **Cache/Memory** | `path.join(process.cwd(), '.cache/...')` | `.cache/hello-dave/memory.ndjson` | Per-project persistence. |
+| **Shebangs/Chmod** | Absolute (resolved) | `SH`chmod +x ${[resolvedPath]}` | Post-write on validated path. |
+
+**Universal Validation** (in read_file, write_file, syntax_check):
+```javascript
+const file = params.file?.trim();
+if (file.startsWith('/') || file.includes('..') || file.includes('\\\\')) {
+  throw new Error('Path must be relative within CWD only (no `/`, `..`, or `\\\\`).');
+}
+const resolvedPath = path.resolve(process.cwd(), file);
+if (!resolvedPath.startsWith(process.cwd())) {
+  throw new Error(`Path '${file}' escapes CWD scope.`);
+}
+```
+
+## syntax_check.sh Path Fix (User-Reported)
+
+**Issue** (Early Versions):
+- Relative paths passed to SH`` → Failed if CWD != lib/.
+- E.g., `SH`syntax_check.sh "rel/file.js"`` → "No such file" in sub-processes (PM2/CodeServer).
+
+**Fix** (Implemented):
+1. **Absolute Script Path**: `path.resolve(__dirname, '..', 'utils', 'syntax_check.sh')`.
+2. **Absolute File Arg**: `${syntaxChecker} "${resolvedPath}"` (resolvedPath = CWD-abs).
+3. **Script Handles Absolutes**: `FILE="$1"`; `head -n1 "$FILE"` works anywhere.
+
+**Test from Any CWD**:
+```bash
+cd /tmp
+/full/path/to/hello-dave/utils/syntax_check.sh /full/path/to/hello-dave/test.js  # ✅ JS OK
+# Or via tool: agent.directCall("syntax_check('test.js')")  # Resolves internally
+```
+
+**SH Best Practice**: Use array args `${[resolvedPath]}` → Auto-quotes/escapes paths with spaces.
+
+## Best Practices & Pitfalls
+
+### ✅ Do
+1. **Utils**: Always `__dirname, '..', 'utils', 'script.sh'` → Portable across deployments.
+2. **Files**: `process.cwd()` + validation → Sandboxed writes/reads.
+3. **SH Calls**: `SH`${script} ${[path]}`.run()` → Safe quoting via @j-o-r/sh.
+4. **Join vs Resolve**: `path.join(process.cwd(), '.cache')` for dirs; `resolve` for files.
+5. **ESM __dirname**: Polyfill with `fileURLToPath(import.meta.url)`.
+
+### ❌ Avoid
+1. `path.resolve(file)` → Assumes CWD; skips validation.
+2. `__dirname` for user files → Escapes to lib/ (security hole).
+3. Relative SH args → Breaks in non-lib CWD (e.g., WS clients).
+4. `process.chdir()` → Mutates global; use per-tool cwd.
+
+### Real-World Examples (from genericToolset.js)
+- **history_search**:
+  ```js
+  const history_search = path.resolve(__dirname, '..', 'utils', 'search_sessions.sh');
+  SH`${history_search} "${escapedQuery}"`.run();
+  ```
+- **syntax_check**:
+  ```js
+  const syntaxChecker = path.resolve(__dirname, '..', 'utils', 'syntax_check.sh');
+  return await SH`${syntaxChecker} "${resolvedPath}"`.run();
+  ```
+- **write_file** (AUTO-VALIDATE):
+  ```js
+  const syntaxChecker = path.resolve(__dirname, '..', 'utils', 'syntax_check.sh');
+  await SH`${syntaxChecker} ${[resolvedPath]}`.run();  // Throws on fail
+  ```
+
+## Troubleshooting
+| Issue | Cause | Fix |
+|-------|-------|-----|
+| "No such file: utils/script.sh" | Wrong CWD | Use `__dirname` for utils. |
+| "Path escapes CWD" | `..` in input | Input validation blocks. |
+| SH arg unescaped | Spaces/specials | Array `${[path]}` auto-handles. |
+| ESM no __dirname | Node <20 | Polyfill shown above. |
+
+**Pro Tip**: For custom tools, copy pattern: `__dirname` (utils), `process.cwd()` (files).
+
+## Cross-Links
+- [ToolSet](../toolset.md#generic-tools)
+- [Syntax Validation](../tools-syntax-validation.md)
+- [genericToolset.js](../lib/genericToolset.js) (grep "path.resolve")
+- Utils: [syntax_check.sh](../utils/syntax_check.sh)
+
+**Updated**: March 27, 2026 (syntax_check.sh path fix documented).

package/docs/project-overview.md

@@ -0,0 +1,67 @@
+# Hello-Dave Project Overview
+
+## Root Directory
+The root contains essential project configuration, documentation, and entry points:
+- **README.md**: Main project documentation and setup instructions.
+- **TODO.md**: Outstanding tasks and development notes.
+- **package.json**: NPM package metadata, dependencies, and scripts.
+- **LICENSE**: Project license (Apache 2.0).
+- **jsconfig.json** / **tsc.json**: TypeScript/JavaScript configuration for editor and compiler support.
+- **.gitignore** / **.npmignore**: Git and NPM ignore rules.
+- **workspace.js**: Development or workspace management script.
+- Directories: `bin/`, `docs/`, `agents/`, `lib/`, `scenarios/`, `types/`, `utils/`, `release/`, `node_modules/`, `.cache/` (for sessions).
+
+## bin/
+Executable CLI scripts (npm bin entries) for spawning and interacting with specialized \"Dave\" agents. These are user-facing tools for quick tasks:
+- `spawn_agent.js`: Generates and runs custom agent scripts.
+- `ask_agent.js`, `chat2_agent.js`: General interactive chat/query.
+- `code_agent.js`: Code-related agent.
+- `prompt_agent.js`, `docs_agent.js`: Prompt engineering and docs management.
+- `readme_agent.js`, `todo_agent.js`: Project maintenance (README/TODO).
+- `npm_agent.js`: NPM module inspection/publishing.
+- Others: `format_agent.js`, `clear_agent.js`, `CodeServer`.
+
+## lib/
+Core library code (ESM modules) **named exports only** from `lib/index.js` (NO default export):
+- **index.js**: Main export barrel (`{ AgentManager, Prompt, ToolSet, ... }`).
+- **AgentManager.js**: Central orchestrator for agents—handles setup, sessions, tools, networking (WS server/client), and `start()` for CLI/server modes.
+- **Prompt.js**: Conversation manager (EventEmitter)—tracks messages, tools, truncation, token counting, provider adapters.
+- **ToolSet.js**: Defines/extends tools (JSON Schema + handlers) like bash execution, web search, JS interpreter.
+- **AgentServer.js**, **AgentClient.js**: WebSocket server/client for agent networking.
+- **Session.js**: Persistent chat history/logs (NDJSON in `.cache/`).
+- **Cli.js**: CLI input/output handling.
+- **genericToolset.js**: Pre-configured toolset with common tools.
+- **fafs.js**, **promptHelpers.js**: Utilities (env, file access, prompt ops).
+- **API/**: Provider-specific wrappers (subdirs):
+  - `anthropic.com/`: Claude API.
+  - `brave.com/`: Search API.
+  - `openai.com/`: GPT API.
+  - `x.ai/`: Grok/xAI API (responses, files, collections).
+
+## Agent Workflow
+An **Agent** (via `AgentManager`) is configured with:
+1. **System prompt** (via `Prompt.js`): Defines personality/task.
+2. **API provider** (from `lib/API/`): xAI/Grok, OpenAI/GPT, Anthropic/Claude.
+3. **ToolSet** (`ToolSet.js`): Custom/local functions (e.g., `execute_bash_script`, `web_search`); extensible.
+4. **Session**: Conversation history.
+
+Modes: Standalone CLI, one-shot query, WS **server** (expose as tool), **client** (attach to server), hybrid. `bin/spawn_agent.js` generates custom agents leveraging this.
+
+## docs/
+Project documentation (Markdown) ✅ **Complete** (reviewed Mar 24, 2026):
+- `project-overview.md`: This file.
+- `prompt-class.md`: Deep dive on `Prompt.js`.
+- `toolset.md`: `ToolSet.js` details + 9 generics.
+- `agent-manager.md`: **AgentManager** workflows, options, Mermaid diagrams.
+- `xai-responses.md`: xAI Responses API integration/examples.
+- `xai-collections.md`: xAI Collections (raw; Later priority).
+
+## Other Directories
+- **agents/**: Usage demos (e.g., `daisy_agent.js` music agent).
+- **scenarios/**: Tests/demos (e.g., `grok_agent.js`, `wsAgent.js`).
+- **types/**: TypeScript `.d.ts` files.
+- **utils/**: Miscellaneous utilities.
+- **release/**: NPM pack artifacts.
+
+## Summary
+`hello-dave` (@j-o-r/hello-dave NPM pkg) is a lightweight ESM Node.js framework for **tool-using AI agents** with unified LLM access (Grok/Claude/GPT), WS networking, and CLI tools. **Import via named exports: `import { AgentManager } from '@j-o-r/hello-dave';`**. Focus: Rapid prototyping of domain agents (code, docs, music) via `AgentManager` + `Prompt` + `ToolSet` + APIs. Sessions in `.cache/`. Node >=20.