@j-o-r/hello-dave 0.0.1 → 0.0.3

package/README.md

@@ -1,207 +1,332 @@
-# hello-dave (ALPHA)
-
-A small, ESM-only Node.js toolkit for building AI “agents” with:
-- one unified Prompt abstraction across multiple LLM providers (x.ai Grok, OpenAI, Anthropic)
-- first-class function calling via a simple ToolSet
-- a tiny AgentManager that wires everything together (CLI, direct calls, WS server/client)
-- optional persisted sessions and records on disk
-
-> **Note:**
-> This is a personal project, created for my own convenience and exclusively used and tested on Debian/Ubuntu.
-> It can be used to create, use and combine  Agents. It has great potential and is very powerful; e.g., `hdCode` has full access to the user environment and is able to execute any command or code.
-> Prompt injection is a RISK so use with care and only if you know what you're doing.
->
-> How-tos and documentation are not finished; you may find rough edges.
->
-> There are no fancy interfaces; everything is kept minimal and functional. 
->
-> It is not yet able to connect to MCP servers; however, it is possible to connect Agents to the main Agent-server and so build a network of Agents.
-> `This is achieved by using function calls and WebSockets.
-> 
-> It is, e.g., possible to run a 'root' Agent with shell access and extend the Agent-server with it.
->
-> Also, it is not tied to localhost. All Agents can run anywhere as long as they can make a connection to the Agent-server.
->
-> The Agent-server can also connect to another Agent-server, running multiple hubs (from different nodes) each with their own specialty.
-
-This repo also ships example CLIs. The most compact example is examples/grok.js which shows how to use:
-- lib/AgentManager.js
-- lib/AgentServer.js
-- lib/AgentClient.js
-- lib/Cli.js
-
-## Requirements
-- Node.js >= 20 (package is ESM only: "type": "module")
-- API keys via environment variables (only set what you use):
-  - XAIKEY for x.ai (Grok)
-  - OPENAIKEY for OpenAI
-  - ANTHKEY for Anthropic
-  - BRAVESEARCHKEY for the Brave Search tool (optional tool)
-
-
-## Install
-
-- As a dependency in your project:
-  - npm i @j-o-r/hello-dave
-  - then use either the programmatic API (AgentManager) or the provided CLIs.
-
-### Bonus (when installed)
-
-Use the AI right away. Make sure you have an x.ai api key available.
-(Grok gives very good result)
+# Updated README.md content
+
+# @j-o-r/hello-dave
+
+[![npm version](https://badge.fury.io/js/%40j-o-r%2Fhello-dave.svg)](https://www.npmjs.com/package/%40j-o-r%2Fhello-dave)
+[![Apache 2.0](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
+[![Node.js >=20](https://img.shields.io/badge/Node.js-%3E=20-green.svg)](https://nodejs.org/)
+
+## Table of Contents
+
+- [Description](#description)
+- [Features](#features)
+- [Installation](#installation)
+- [Quick Start](#quick-start)
+- [Usage](#usage)
+- [Examples](#examples)
+- [Development](#development)
+- [Changelog](#changelog)
+- [Contributing](#contributing)
+- [License](#license)
+- [TODO Alignment](#todo-alignment)
+
+## Description
+
+**@j-o-r/hello-dave** is a lightweight, ESM-only Node.js framework for building intelligent AI agents powered by large language models (LLMs) such as xAI Grok, Anthropic Claude, or OpenAI GPT. It simplifies the creation of custom agents with features like the central `AgentManager` class, WebSocket-based client/server support for networked agents, seamless tool integration (e.g., bash execution, web search), customizable system prompts, and robust session management supporting large context windows (up to millions of tokens).
+
+The framework emphasizes ease of use, allowing developers and non-experts alike to rapidly prototype and deploy specialized agents. Core integrations include unified xAI Grok API access, WebSocket (WS) networking with secret-based authentication, and direct CLI/cron messaging (no heartbeat/ping mechanism required). Example scripts like `readmeDave`, `todoDave`, and others have been reorganized into the `examples/` folder for clarity.
+
+Whether you're building a code reviewer, a research assistant, or a creative collaborator, `@j-o-r/hello-dave` abstracts away API complexities, providing a unified interface across providers. It's alpha-stage but production-ready for personal projects, with persistence for sessions and logs in NDJSON format.
+
+## Features
+
+- **Unified LLM Access**: Seamless integration with xAI Grok API, Anthropic Claude, and OpenAI GPT via a single API.
+- **AgentManager Class**: Core orchestrator for prompts, tools, sessions, and interactions (CLI, direct calls, WebSocket).
+- **Tool Integration**: Built-in tools for bash scripting, web search, JavaScript execution, and more; easily extensible.
+- **WebSocket Networking with Secrets**: Agents can act as servers or clients with optional secret authentication, enabling secure distributed agent networks across machines.
+- **Direct CLI/Cron Messaging**: Simplified interactions without needing heartbeat mechanisms—ideal for scheduled tasks.
+- **Customizable Prompts**: Tailor agent behavior with system prompts for specialized tasks (e.g., music production, code analysis).
+- **Session Management**: Persistent conversations with large context windows; automatic truncation and history search.
+- **CLI Utilities**: Core tool at `bin/dave.js` for cache management, WS connections, and interactive queries; plus agent generator.
+- **Lightweight & TypeScript-Friendly**: ESM-only, no heavy deps; full types included.
+
+## Installation
+
+Install as a dependency in your Node.js project (requires Node.js >= 20 with `"type": "module"` in `package.json`):
 
 ```bash
-export XAIKEY=.....
+npm install @j-o-r/hello-dave
 ```
-Now you can run the hello-dave code AI and the generic hello-dave 'ask' ai from within the current folder.
-Make sure to create a '.cache' folder first.
+
+For global access to CLI tools (including `dave` from `bin/dave.js`):
+
 ```bash
-mkdir .cache
+npm install -g @j-o-r/hello-dave
 ```
-(I am used to run Agents within tmux sessions)
 
+Set your API keys via environment variables (only set what you need):
+- `XAIKEY` for xAI Grok
+- `ANTHKEY` for Anthropic Claude
+- `OPENAIKEY` for OpenAI GPT
+- `BRAVESEARCHKEY` for web search (optional)
+
+Create a cache directory for sessions:
 ```bash
-# Code AI with direct shell access (CAREFULL)
-npx hdCode
+mkdir .cache
+```
+
+## Quick Start
+
+1. Set an API key (e.g., for Grok):
+   ```bash
+   export XAIKEY=your_xai_api_key
+   ```
+
+2. Use the core CLI utility `bin/dave.js` for interactive queries or management:
+   ```bash
+   npx @j-o-r/hello-dave dave --ask
+   ```
+   - Launches an interactive agent session. Use `Alt+d` to exit, `Alt+r` to reset.
+   - Example: Ask questions about anything; it uses xAI Grok by default with web search and X search tools.
+
+3. For one-shot queries via piped input:
+   ```bash
+   echo "Explain quantum computing" | npx @j-o-r/hello-dave dave --ask --model grok-4
+   ```
+   - Provides a direct response using the specified model (e.g., `--model grok-4`).
+
+4. Connect to a WebSocket server (e.g., for networked agents):
+   ```bash
+   npx @j-o-r/hello-dave dave --connect ws://localhost:8080 --secret your_secret
+   ```
+
+5. Generate a custom agent using the portable `spawndave` tool (CLI/WS agents + CodeServer launchers):
+   ```bash
+   npx @j-o-r/hello-dave spawndave
+   ```
+   - Interactive: Follow prompts to create a new agent script in `bin/` (e.g., `bin/myagent.js`).
+   - Portable to any project (auto-installs deps like `@j-o-r/hello-dave`):
+     ```bash
+     npx @j-o-r/hello-dave spawndave --cwd ~/my-project
+     ```
+   - Supports one-shot: `echo "Create code-review agent" | npx @j-o-r/hello-dave spawndave --cwd ~/proj`.
+   - Aligns with CodeServer pattern for multi-agent setups (see Usage/Examples).
+
+`bin/dave.js` also supports cache operations like `--list` (list sessions), `--search "query"`, `--clear`, and `--inspect path/to/log.ndjson`.
+
+## Usage
+
+### Core CLI via `bin/dave.js`
+
+`bin/dave.js` provides essential utilities and examples for agent spawning, WS interactions, and basic querying. Key commands:
+
+- **Ask Mode** (Interactive or Piped One-Shot):
+  ```bash
+  # Interactive
+  npx @j-o-r/hello-dave dave --ask --model grok-4 --temperature 0.2
+
+  # Piped one-shot
+  echo "What is quantum computing?" | npx @j-o-r/hello-dave dave --ask --model grok-4
+  ```
+  - Note: Piped input triggers a direct response via `directCall`; without pipe, it launches an interactive CLI session.
+  - `--model` option works for both modes (e.g., `grok-4`, `grok-4-1-fast-reasoning`).
+  - Uses `AgentManager` with xAI Grok API integration, adds tools like `web_search`, `x_search`, `open_link`, `send_email`, `history_search`.
+  - System prompt emphasizes brief, direct responses with step-by-step reasoning.
+
+- **SpawnDave Mode** (Portable Agent Creator with --cwd Support):
+  ```bash
+  # Interactive in current project
+  npx @j-o-r/hello-dave spawndave --model grok-4-fast-reasoning --temperature 0.7
+
+  # Portable to another project (switches cwd, auto-deps)
+  npx @j-o-r/hello-dave spawndave --cwd ../other-proj
+
+  # One-shot for quick generation
+  echo "name=coderev desc=Git diff analyzer tools=read_file execute_bash_script web_search" | npx @j-o-r/hello-dave spawndave --cwd ~/my-proj
+  ```
+  - Creates production-ready CLI/WS agents (`bin/<name>.js`) supporting CLI, server (`--serve`), client (`--connect`), and hybrid modes.
+  - Generates multi-agent CodeServer launchers (`examples/<Name>Server`) using PM2 for persistent networked setups (main server + sub-agents).
+  - **Portable**: Runs via `npx` anywhere—no local install needed. Auto-inspects project, installs `@j-o-r/hello-dave @j-o-r/sh` if missing, validates ESM code (`node --check`), tests deployment.
+  - Aligns with CodeServer pattern: Self-contained Bash launchers for PM2-managed multi-agents (e.g., code review + TODO + NPM agents).
+  - Requires `XAIKEY`; uses online repo docs (https://codeberg.org/duin/hello-dave) for portability.
+  - Example output: Deploys `bin/coderev.js`, tests it, provides server/client usage guide.
+
+- **WebSocket Client Mode** (with secrets for secure networking):
+  ```bash
+  # Interactive connection
+  npx @j-o-r/hello-dave dave --connect ws://localhost:8080 --secret "mysecret"
+
+  # Piped one-shot (direct CLI/cron messaging, no heartbeat needed)
+  echo "Predict the weather" | npx @j-o-r/hello-dave dave --connect ws://localhost:8080 --secret "mysecret"
+  echo "user_reset" | npx @j-o-r/hello-dave dave --connect ws://localhost:8080 --secret "mysecret"
+  ```
+
+- **Cache Management**:
+  ```bash
+  npx @j-o-r/hello-dave dave --list  # List sessions
+  npx @j-o-r/hello-dave dave --search "AI agents"  # Search history
+  npx @j-o-r/hello-dave dave --clear  # Clear cache
+  ```
+
+For custom agents, use `AgentManager` programmatically (see Examples).
+
+### Programmatic Usage with AgentManager
+
+```javascript
+import AgentManager from '@j-o-r/hello-dave';
+
+const agent = new AgentManager({ name: 'MyAgent', secret: 'optional_secret' });
+agent.setup({
+  prompt: 'You are a helpful assistant...',
+  api: 'xai',  // Uses unified xAI Grok API access
+  options: { model: 'grok-4', temperature: 0.7, tools: [{ type: 'web_search' }] },
+  toolsetMode: 'auto',
+  contextWindow: 500000
+});
 
-# Ask Cli
-npx hdAsk
-# Ask One shot
-echo "Current date time" | npx hdAsk
+agent.addGenericToolcall('execute_bash_script');  // Post-setup generics
+agent.addGenericToolcall('read_file');
+agent.addGenericToolcall('write_file');
 
-# Inspect a ndjson log/session file
-npx hdInspect .cache/hello-dave/...
+await agent.start();  // Interactive CLI
+// Or: await agent.start(undefined, 8000, undefined, 'Intro', 'toolName', 'desc');  // WS server
+// Or: const response = await agent.directCall('Query');  // One-shot
 ```
 
-- Interactive mode (no stdin):
-  - Use keys inside the TUI:
-    - ALT+d: exit
-    - ALT+r: reset conversation
-    - ALT+s: load a previous session
-    - ALT+k: show shortcuts/help
+Direct CLI/cron: Pipe inputs to custom scripts for non-interactive use. WS supports secrets for authentication; no heartbeat required due to event-driven messaging.
 
-- One-shot (pipe stdin):
-  - echo "explain sse vs websocket in 2 lines" | npx hdAsk
+## Examples
 
-- Switch parameters:
-  - node examples/grok.js --model grok-3-mini --tokens 4000 --context 10000 --temperature 0.3 --top_p 0.9
-  - node examples/grok.js --reasoning high
+Example scripts (e.g., `readmeDave.js` for README management, `todoDave.js` for task handling, `spawndave.js` for agent creation) are now organized in the `examples/` folder. These demonstrate specialized agents using `AgentManager`, xAI Grok API integration, and tools. **Note**: Agent names and filenames follow lowercase naming rules per [docs/agent-manager.md](docs/agent-manager.md) (regex `/^[a-z_0-9_]{2,}$/` for safety in folders, tools, and filesystems—no uppercase, hyphens, or specials).
 
-- Enable tools (function calling):
-  - node examples/grok.js --tools javascript,bash
-    - javascript adds javascript_interpreter
-    - bash adds execute_bash_script
+- **Link to Examples Folder**: Explore [examples/](examples/) for full scripts like `daisy.js` (music agent), `codeDave.js` (code review), `npmDave.js` (NPM inspector).
 
-- Run as a local dynamic tool server and connect a client:
-  - Terminal A: node examples/grok.js --serve 8000 --tools javascript,bash
-  - Terminal B: node examples/grok.js --connect ws://127.0.0.1:8000/ws
+### Persistent Multi-Agent CodeServer Pattern
 
-Notes
-- With --serve, the agent starts a WS server on 127.0.0.1:PORT and exposes any connected clients as callable tools.
-- With --connect, an agent registers itself on that WS and will handle queries routed to its tool name.
+The `examples/CodeServer` script provides a persistent multi-agent setup using PM2 to manage long-running processes. It starts a main code agent as a WebSocket server on a specified port, with sub-agents (for TODO, README, NPM, and docs tasks) attached as clients. This pattern enables a networked ecosystem of specialized agents that maintain state across sessions, ideal for extended development workflows where context persistence and task delegation are key.
 
-## Where data is stored
+Benefits include seamless integration for dev sessions—agents stay online, handling queries via WS without restarts, supporting large context windows and tool calls. For full details, see [docs/codeserver-pattern.md](docs/codeserver-pattern.md).
 
-Conversations, logs and request records are written under ./.cache/hello-dave/<agent-name>/
-- sessions/  incremental ndjson of messages
-- records/   per-day ndjson of timing/token/billing metadata
-- logs/      per-day ndjson of internal events
+**Usage:**
 
-These are managed by lib/Session.js and used by lib/Cli.js (ALT+s to load older sessions).
+```bash
+# Start the CodeServer (requires PM2: npm i -g pm2)
+./examples/CodeServer 8080 mysecret
 
-## Programmatic use (AgentManager)
+# Connect via CLI (interactive or piped)
+npx @j-o-r/hello-dave dave --connect ws://127.0.0.1:8080/ws --secret mysecret
+# Or for one-shot: echo "Review this code" | npx @j-o-r/hello-dave dave --connect ws://127.0.0.1:8080/ws --secret mysecret
+```
 
-AgentManager is the easiest way to embed the library in your own code. It creates a Prompt, optional ToolSet, session persistence, and can run as CLI or server/client.
+Monitor with `pm2 list` and stop via `pm2 delete <name>`.
 
-```js
-// ESM
-import AgentManager from '@j-o-r/hello-dave';
+### Snippet: Spawning Agents (Portable via `bin/dave.js` --spawndave)
+
+`spawndave` uses `AgentManager` to interactively generate new agent scripts, now fully aware of WebSocket server/client modes and the CodeServer multi-agent pattern for networked setups.
+
+Now CodeServer-aware: Can generate PM2 multi-agent launchers (e.g., 'Generate CodeServer for coderev + todo'). See updated [examples/spawndave.js](examples/spawndave.js) and [docs/codeserver-pattern.md](docs/codeserver-pattern.md).
 
-// Minimal: direct call against Grok
-const agent = new AgentManager({ name: 'grok' });
+**Portable**: `spawndave` is fully portable—no local docs/examples needed. Uses online repo links (https://codeberg.org/duin/hello-dave). Works in any project via `npx` (auto-deps with `--cwd`).
+
+```javascript
+// Core setup in spawndave
+const agent = new AgentManager({ name: 'spawn_dave', secret });
 agent.setup({
-  prompt: 'Be precise and concise.',
-  api: 'grok',                        // 'gpt' | 'grok' | 'claude'
-  options: {                          // forwarded to the provider adapter
-    model: 'grok-4',
-    search_parameters: { mode: 'auto' }
-  },
-  toolsetMode: 'auto',                // 'auto' | 'required' | undefined
-  contextWindow: 250_000
+  prompt: 'You are AgentCreator... [detailed prompt for portability, validation]',
+  api: 'xai',
+  options: { model: 'grok-4-fast-reasoning', tools: [{ type: 'web_search' }], reasoning: { effort: 'high' } },
+  toolsetMode: 'auto',
+  contextWindow: 500000
 });
+agent.addGenericToolcall('execute_bash_script');  // For project inspection
+agent.addGenericToolcall('read_file');
+agent.addGenericToolcall('write_file');  // For generating bin/ scripts
 
-const answer = await agent.directCall('Two bullet points on tail recursion.');
-console.log(answer);
+await agent.start();  // Guides user to create e.g., bin/myagent.js
 ```
 
-### Adding your own tools (function calling)
-
-```js
-const toolset = agent.getToolset();
-if (toolset) {
-  toolset.add(
-    'say_hello',
-    'Return a friendly greeting',
-    {
-      type: 'object',
-      properties: { name: { type: 'string', description: 'Who to greet' } },
-      required: ['name']
-    },
-    async ({ name }) => `Hello, ${name}!`
-  );
-}
-
-// Now the model can call say_hello when appropriate.
-```
+- Validates generated ESM code with `node --check` and tool rules (xAI natives pre-setup, generics post-setup).
+- Ensures absolute imports for portability.
 
-### Running an agent as a tool server
+**Example Usage:**
 
-```js
-agent.enableServer('search', 'General-purpose search interface', 8000);
-// WS on ws://127.0.0.1:8000/ws; connected clients become callable tools.
-```
+```bash
+# Interactive spawning (runs from npm without cloning; current dir)
+npx @j-o-r/hello-dave spawndave
 
-### Attaching an agent as a WS client (remote tool)
+# Portable to specific project (auto-switches cwd, installs deps if needed)
+npx @j-o-r/hello-dave spawndave --cwd ~/my-proj
 
-```js
-agent.attach(
-  'code',
-  'Run JavaScript snippets safely and return output',
-  'ws://127.0.0.1:8000/ws'
-);
+# Piped one-shot for CodeServer launcher (portable, no cloning)
+echo "Generate CodeServer launcher for coderev + tododave" | npx @j-o-r/hello-dave spawndave --cwd ~/proj
 ```
 
-## Library building blocks (how grok.js uses them)
+### Snippet: Using xAI Grok API Integration
 
-- lib/AgentManager.js: façade that composes Prompt, ToolSet, Session and the provider adaptor (API.text[api]). Also starts CLI, WS server, or WS client.
-- lib/Cli.js: a small TUI with history, session loading, and handy shortcuts; it listens to Prompt events.
-- lib/AgentServer.js: local WS server that turns each attached client into a ToolSet function. Calls are routed over WS and responses returned as tool outputs.
-- lib/AgentClient.js: WS client that introduces itself (name + description) and processes queued requests exactly one at a time.
+The unified xAI Grok API access supports options like reasoning and tools:
 
-Under the hood:
-- lib/Prompt.js: uniform message format (system, user, assistant, tool, reasoning, log), function-call plumbing, truncation, and provider-neutral calls.
-- lib/ToolSet.js: dead-simple JSON-schema + async method registry powering function calling.
-- lib/API/*: adaptors for x.ai (Grok), OpenAI Responses API, Anthropic Messages API, Brave Search tool.
-- lib/Session.js: on-disk persistence for sessions, logs and records.
+```javascript
+// In AgentManager setup
+const options = {
+  model: 'grok-4-1-fast-reasoning',
+  temperature: 0.8,
+  tools: [{ type: 'web_search' }, { type: 'x_search', from_date: '2024-01-01' }],  // Native xAI tools
+  reasoning: { effort: 'medium', summary: 'auto' }  // Chain-of-thought support
+};
+```
 
-## TypeScript
-- Types are bundled. Main types for AgentManager are available via types/AgentManager.d.ts.
+Detailed examples for xAI integration (e.g., advanced tool calls, reasoning) are planned (see TODO).
 
-## Utils
+### Running Examples
 
 ```bash
-# Clear all session and log files
-npx hdClear
-# Read an ndjson file (session, log  etc.)
-npx hdInspect [path/to/file.ndjson]
-# Design / create a prompt. (AI prompt, x.ai key needed)
-npx hdPrompt
-# Ask a generic quesion to openai gpt-5-nano (openai key needed)
-npx hdAsk
-# Cli Connect to a websocket agent server
-npx hdConnect [ws://127.0.0.1:8000/ws]
+# Music agent (examples/daisy.js)
+chmod +x examples/daisy.js
+./examples/daisy.js "Generate lyrics for a pop song"
+
+# TODO manager (examples/todoDave.js)
+./examples/todoDave.js --connect ws://localhost:8080 --secret "secret"
 
+# Server setup for networked examples
+npx @j-o-r/hello-dave dave --ask --serve 8080  # Expose as WS server
 ```
 
+## Development
+
+- **Scripts**: `npm test` (runs scenarios/grok.js), `npm run types` (generate TypeScript defs), `npm run release` (pack for release).
+- **Structure**: Core in `lib/` (AgentManager.js, wsCli.js, etc.), API integrations in `lib/API/`, examples in `examples/`, utils in `utils/`.
+- **Testing**: Basic validation via `npm test`. Unit tests for xAI integration planned (see TODO).
+- **Prep for v0.0.4**: Focus on spawnDave enhancements and release testing.
+
+## Changelog
+
+### v0.0.4 (Prep - March 2026)
+- Reorganized examples to `examples/` folder (readmeDave, todoDave, etc.).
+- Enhanced `bin/dave.js` for core utilities: spawning, WS with secrets, direct messaging.
+- Improved xAI Grok API integration; added native tools/reasoning support.
+- Removed heartbeat needs; direct CLI/cron support.
+- Added piped one-shot support to dave --ask.
+- TODO alignment: High-priority items noted (release prep, spawnDave enhancements).
+
+### v0.0.3 (Previous)
+- Initial WS secrets, generic tools, AgentManager refinements.
+
+Full history in git tags.
+
+## Contributing
+
+Contributions are welcome! Fork the repo at [https://codeberg.org/duin/hello-dave](https://codeberg.org/duin/hello-dave), create a feature branch, and submit a pull request.
+
+1. Install deps: `npm install`
+2. Self dep `npm run link-self`
+3. Run tests: `npm test`
+4. Generate types: `npm run types`
+5. Report issues: [https://codeberg.org/duin/hello-dave/issues](https://codeberg.org/duin/hello-dave/issues)
+
+Follow the [Code of Conduct](CODE_OF_CONDUCT.md) and Apache-2.0 license.
+
 ## License
-Apache-2.0
+
+Apache License 2.0. See [LICENSE](LICENSE) for details.
+
+## TODO Alignment
+
+This README aligns with high-priority items from [TODO.md](TODO.md):
+- **v0.0.4 Prep**: Changelog snippet added; release testing emphasized in Development.
+- **spawnDave Enhancements**: Highlighted in Examples with portability notes.
+- xAI integration tests/examples: Deferred to later (general enhancements).
+- Documentation tasks (e.g., AgentManager internals) integrated into Usage/Examples.
+- Deferred: Other xAI features to later; focus on core release and spawnDave.
+
+For full TODO, see [TODO.md](TODO.md).
+