@j-o-r/hello-dave 0.0.2 → 0.0.4

package/docs.bak.1774780058/path-resolution-best-practices.md

@@ -0,0 +1,104 @@
+# Path Resolution Best Practices for Generic Tools
+
+## Overview
+In `lib/genericToolset.js` and related tools (e.g., `syntax_check`, `write_file`), **path resolution** distinguishes between:
+- **Module-relative paths** (utils/scripts): Use `__dirname` for portability.
+- **User-provided file paths**: Resolve to `process.cwd()` (CWD sandbox) + strict validation.
+
+This ensures **sandboxing** (no escapes), **portability** (works from any CWD/PM2), and **security**. All tools enforce: no `/`, `..`, `\\\\`; `resolvedPath.startsWith(process.cwd())`.
+
+**ESM Setup** (genericToolset.js):
+```javascript
+const __dirname = path.dirname(fileURLToPath(import.meta.url));  // Polyfill for ESM
+```
+
+## __dirname vs process.cwd(): When to Use What
+
+| Context | Resolution | Example Code | Rationale |
+|---------|------------|--------------|-----------|
+| **Utils/Scripts** (e.g., syntax_check.sh, search_sessions.sh) | `__dirname` (module-rel) | `path.resolve(__dirname, '..', 'utils', 'syntax_check.sh')` | Portable: Locates utils/ from `lib/` regardless of CWD. Works in servers/clients. |
+| **User Files** (read/write/syntax/memory) | `process.cwd()` | `path.resolve(process.cwd(), params.file)` | Sandbox: Pins to project CWD (`~/devpri/js/hello-dave`). Prevents jailbreaks. |
+| **Cache/Memory** | `path.join(process.cwd(), '.cache/...')` | `.cache/hello-dave/memory.ndjson` | Per-project persistence. |
+| **Shebangs/Chmod** | Absolute (resolved) | `SH`chmod +x ${[resolvedPath]}` | Post-write on validated path. |
+
+**Universal Validation** (in read_file, write_file, syntax_check):
+```javascript
+const file = params.file?.trim();
+if (file.startsWith('/') || file.includes('..') || file.includes('\\\\')) {
+  throw new Error('Path must be relative within CWD only (no `/`, `..`, or `\\\\`).');
+}
+const resolvedPath = path.resolve(process.cwd(), file);
+if (!resolvedPath.startsWith(process.cwd())) {
+  throw new Error(`Path '${file}' escapes CWD scope.`);
+}
+```
+
+## syntax_check.sh Path Fix (User-Reported)
+
+**Issue** (Early Versions):
+- Relative paths passed to SH`` → Failed if CWD != lib/.
+- E.g., `SH`syntax_check.sh "rel/file.js"`` → "No such file" in sub-processes (PM2/CodeServer).
+
+**Fix** (Implemented):
+1. **Absolute Script Path**: `path.resolve(__dirname, '..', 'utils', 'syntax_check.sh')`.
+2. **Absolute File Arg**: `${syntaxChecker} "${resolvedPath}"` (resolvedPath = CWD-abs).
+3. **Script Handles Absolutes**: `FILE="$1"`; `head -n1 "$FILE"` works anywhere.
+
+**Test from Any CWD**:
+```bash
+cd /tmp
+/full/path/to/hello-dave/utils/syntax_check.sh /full/path/to/hello-dave/test.js  # ✅ JS OK
+# Or via tool: agent.directCall("syntax_check('test.js')")  # Resolves internally
+```
+
+**SH Best Practice**: Use array args `${[resolvedPath]}` → Auto-quotes/escapes paths with spaces.
+
+## Best Practices & Pitfalls
+
+### ✅ Do
+1. **Utils**: Always `__dirname, '..', 'utils', 'script.sh'` → Portable across deployments.
+2. **Files**: `process.cwd()` + validation → Sandboxed writes/reads.
+3. **SH Calls**: `SH`${script} ${[path]}`.run()` → Safe quoting via @j-o-r/sh.
+4. **Join vs Resolve**: `path.join(process.cwd(), '.cache')` for dirs; `resolve` for files.
+5. **ESM __dirname**: Polyfill with `fileURLToPath(import.meta.url)`.
+
+### ❌ Avoid
+1. `path.resolve(file)` → Assumes CWD; skips validation.
+2. `__dirname` for user files → Escapes to lib/ (security hole).
+3. Relative SH args → Breaks in non-lib CWD (e.g., WS clients).
+4. `process.chdir()` → Mutates global; use per-tool cwd.
+
+### Real-World Examples (from genericToolset.js)
+- **history_search**:
+  ```js
+  const history_search = path.resolve(__dirname, '..', 'utils', 'search_sessions.sh');
+  SH`${history_search} "${escapedQuery}"`.run();
+  ```
+- **syntax_check**:
+  ```js
+  const syntaxChecker = path.resolve(__dirname, '..', 'utils', 'syntax_check.sh');
+  return await SH`${syntaxChecker} "${resolvedPath}"`.run();
+  ```
+- **write_file** (AUTO-VALIDATE):
+  ```js
+  const syntaxChecker = path.resolve(__dirname, '..', 'utils', 'syntax_check.sh');
+  await SH`${syntaxChecker} ${[resolvedPath]}`.run();  // Throws on fail
+  ```
+
+## Troubleshooting
+| Issue | Cause | Fix |
+|-------|-------|-----|
+| "No such file: utils/script.sh" | Wrong CWD | Use `__dirname` for utils. |
+| "Path escapes CWD" | `..` in input | Input validation blocks. |
+| SH arg unescaped | Spaces/specials | Array `${[path]}` auto-handles. |
+| ESM no __dirname | Node <20 | Polyfill shown above. |
+
+**Pro Tip**: For custom tools, copy pattern: `__dirname` (utils), `process.cwd()` (files).
+
+## Cross-Links
+- [ToolSet](../toolset.md#generic-tools)
+- [Syntax Validation](../tools-syntax-validation.md)
+- [genericToolset.js](../lib/genericToolset.js) (grep "path.resolve")
+- Utils: [syntax_check.sh](../utils/syntax_check.sh)
+
+**Updated**: March 27, 2026 (syntax_check.sh path fix documented).

package/docs.bak.1774780058/project-overview.md

@@ -0,0 +1,67 @@
+# Hello-Dave Project Overview
+
+## Root Directory
+The root contains essential project configuration, documentation, and entry points:
+- **README.md**: Main project documentation and setup instructions.
+- **TODO.md**: Outstanding tasks and development notes.
+- **package.json**: NPM package metadata, dependencies, and scripts.
+- **LICENSE**: Project license (Apache 2.0).
+- **jsconfig.json** / **tsc.json**: TypeScript/JavaScript configuration for editor and compiler support.
+- **.gitignore** / **.npmignore**: Git and NPM ignore rules.
+- **workspace.js**: Development or workspace management script.
+- Directories: `bin/`, `docs/`, `examples/`, `lib/`, `scenarios/`, `types/`, `utils/`, `release/`, `node_modules/`, `.cache/` (for sessions).
+
+## bin/
+Executable CLI scripts (npm bin entries) for spawning and interacting with specialized \"Dave\" agents. These are user-facing tools for quick tasks:
+- `spawn_agent.js`: Generates and runs custom agent scripts.
+- `ask_agent.js`, `chat2Dave.js`: General interactive chat/query.
+- `code_agent.js`: Code-related agent.
+- `prompt_agent.js`, `docs_agent.js`: Prompt engineering and docs management.
+- `readme_agent.js`, `todo_agent.js`: Project maintenance (README/TODO).
+- `npm_agent.js`: NPM module inspection/publishing.
+- Others: `formatDave.js`, `clearDave.js`, `CodeServer`.
+
+## lib/
+Core library code (ESM modules) **named exports only** from `lib/index.js` (NO default export):
+- **index.js**: Main export barrel (`{ AgentManager, Prompt, ToolSet, ... }`).
+- **AgentManager.js**: Central orchestrator for agents—handles setup, sessions, tools, networking (WS server/client), and `start()` for CLI/server modes.
+- **Prompt.js**: Conversation manager (EventEmitter)—tracks messages, tools, truncation, token counting, provider adapters.
+- **ToolSet.js**: Defines/extends tools (JSON Schema + handlers) like bash execution, web search, JS interpreter.
+- **AgentServer.js**, **AgentClient.js**: WebSocket server/client for agent networking.
+- **Session.js**: Persistent chat history/logs (NDJSON in `.cache/`).
+- **Cli.js**: CLI input/output handling.
+- **genericToolset.js**: Pre-configured toolset with common tools.
+- **fafs.js**, **promptHelpers.js**: Utilities (env, file access, prompt ops).
+- **API/**: Provider-specific wrappers (subdirs):
+  - `anthropic.com/`: Claude API.
+  - `brave.com/`: Search API.
+  - `openai.com/`: GPT API.
+  - `x.ai/`: Grok/xAI API (responses, files, collections).
+
+## Agent Workflow
+An **Agent** (via `AgentManager`) is configured with:
+1. **System prompt** (via `Prompt.js`): Defines personality/task.
+2. **API provider** (from `lib/API/`): xAI/Grok, OpenAI/GPT, Anthropic/Claude.
+3. **ToolSet** (`ToolSet.js`): Custom/local functions (e.g., `execute_bash_script`, `web_search`); extensible.
+4. **Session**: Conversation history.
+
+Modes: Standalone CLI, one-shot query, WS **server** (expose as tool), **client** (attach to server), hybrid. `bin/spawn_agent.js` generates custom agents leveraging this.
+
+## docs/
+Project documentation (Markdown) ✅ **Complete** (reviewed Mar 24, 2026):
+- `project-overview.md`: This file.
+- `prompt-class.md`: Deep dive on `Prompt.js`.
+- `toolset.md`: `ToolSet.js` details + 9 generics.
+- `agent-manager.md`: **AgentManager** workflows, options, Mermaid diagrams.
+- `xai-responses.md`: xAI Responses API integration/examples.
+- `xai-collections.md`: xAI Collections (raw; Later priority).
+
+## Other Directories
+- **examples/**: Usage demos (e.g., `daisy_agent.js` music agent).
+- **scenarios/**: Tests/demos (e.g., `grok_agent.js`, `wsAgent.js`).
+- **types/**: TypeScript `.d.ts` files.
+- **utils/**: Miscellaneous utilities.
+- **release/**: NPM pack artifacts.
+
+## Summary
+`hello-dave` (@j-o-r/hello-dave NPM pkg) is a lightweight ESM Node.js framework for **tool-using AI agents** with unified LLM access (Grok/Claude/GPT), WS networking, and CLI tools. **Import via named exports: `import { AgentManager } from '@j-o-r/hello-dave';`**. Focus: Rapid prototyping of domain agents (code, docs, music) via `AgentManager` + `Prompt` + `ToolSet` + APIs. Sessions in `.cache/`. Node >=20.

package/docs.bak.1774780058/project-overview.md.bak

@@ -0,0 +1,67 @@
+# Hello-Dave Project Overview
+
+## Root Directory
+The root contains essential project configuration, documentation, and entry points:
+- **README.md**: Main project documentation and setup instructions.
+- **TODO.md**: Outstanding tasks and development notes.
+- **package.json**: NPM package metadata, dependencies, and scripts.
+- **LICENSE**: Project license (Apache 2.0).
+- **jsconfig.json** / **tsc.json**: TypeScript/JavaScript configuration for editor and compiler support.
+- **.gitignore** / **.npmignore**: Git and NPM ignore rules.
+- **workspace.js**: Development or workspace management script.
+- Directories: `bin/`, `docs/`, `examples/`, `lib/`, `scenarios/`, `types/`, `utils/`, `release/`, `node_modules/`, `.cache/` (for sessions).
+
+## bin/
+Executable CLI scripts (npm bin entries) for spawning and interacting with specialized \"Dave\" agents. These are user-facing tools for quick tasks:
+- `spawnDave.js`: Generates and runs custom agent scripts.
+- `ask_agent.js`, `chat2Dave.js`: General interactive chat/query.
+- `codeDave.js`: Code-related agent.
+- `prompt_agent.js`, `docsDave.js`: Prompt engineering and docs management.
+- `readmeDave.js`, `todoDave.js`: Project maintenance (README/TODO).
+- `npmDave.js`: NPM module inspection/publishing.
+- Others: `formatDave.js`, `clearDave.js`, `CodeServer`.
+
+## lib/
+Core library code (ESM modules) implementing the agent framework:
+- **index.js**: Main export barrel (AgentManager, Prompt, ToolSet, API wrappers, etc.).
+- **AgentManager.js**: Central orchestrator for agents—handles setup, sessions, tools, networking (WS server/client), and `start()` for CLI/server modes.
+- **Prompt.js**: Conversation manager (EventEmitter)—tracks messages, tools, truncation, token counting, provider adapters.
+- **ToolSet.js**: Defines/extends tools (JSON Schema + handlers) like bash execution, web search, JS interpreter.
+- **AgentServer.js**, **AgentClient.js**: WebSocket server/client for agent networking.
+- **Session.js**: Persistent chat history/logs (NDJSON in `.cache/`).
+- **Cli.js**: CLI input/output handling.
+- **genericToolset.js**: Pre-configured toolset with common tools.
+- **fafs.js**, **promptHelpers.js**: Utilities (env, file access, prompt ops).
+- **API/**: Provider-specific wrappers (subdirs):
+  - `anthropic.com/`: Claude API.
+  - `brave.com/`: Search API.
+  - `openai.com/`: GPT API.
+  - `x.ai/`: Grok/xAI API (responses, files, collections).
+
+## Agent Workflow
+An **Agent** (via `AgentManager`) is configured with:
+1. **System prompt** (via `Prompt.js`): Defines personality/task.
+2. **API provider** (from `lib/API/`): xAI/Grok, OpenAI/GPT, Anthropic/Claude.
+3. **ToolSet** (`ToolSet.js`): Custom/local functions (e.g., `execute_bash_script`, `web_search`); extensible.
+4. **Session**: Conversation history.
+
+Modes: Standalone CLI, one-shot query, WS **server** (expose as tool), **client** (attach to server), hybrid. `bin/spawnDave.js` generates custom agents leveraging this.
+
+## docs/
+Project documentation (Markdown) ✅ **Complete** (reviewed Mar 24, 2026):
+- `project-overview.md`: This file.
+- `prompt-class.md`: Deep dive on `Prompt.js`.
+- `toolset.md`: `ToolSet.js` details + 9 generics.
+- `agent-manager.md`: **AgentManager** workflows, options, Mermaid diagrams.
+- `xai-responses.md`: xAI Responses API integration/examples.
+- `xai-collections.md`: xAI Collections (raw; Later priority).
+
+## Other Directories
+- **examples/**: Usage demos (e.g., `daisy_agent.js` music agent).
+- **scenarios/**: Tests/demos (e.g., `grok_agent.js`, `wsAgent.js`).
+- **types/**: TypeScript `.d.ts` files.
+- **utils/**: Miscellaneous utilities.
+- **release/**: NPM pack artifacts.
+
+## Summary
+`hello-dave` (@j-o-r/hello-dave NPM pkg) is a lightweight ESM Node.js framework for **tool-using AI agents** with unified LLM access (Grok/Claude/GPT), WS networking, and CLI tools. Focus: Rapid prototyping of domain agents (code, docs, music) via `AgentManager` + `Prompt` + `ToolSet` + APIs. Sessions in `.cache/`. Node >=20."

package/docs.bak.1774780058/prompt-class.md

@@ -0,0 +1,141 @@
+# Prompt Class
+
+Source: \`lib/Prompt.js\`
+
+The \`Prompt\` class is an EventEmitter-based manager for standardized AI conversation messages. It supports multi-modal content (text, images, audio), tool/function calls, token counting, automatic truncation, and adapters for different LLM providers (OpenAI, xAI, Anthropic).
+
+## Key Features
+- Uniform message format across providers.
+- Context window management with truncation.
+- Toolset integration for function calling.
+- Event-driven (add, start, finished, tool_request, etc.).
+- Sticky system prompts preserved on reset/truncate.
+
+## Dependencies
+- \`gpt-3-encoder\` for token counting.
+- \`node:events\` EventEmitter.
+- \`./promptHelpers.js\` for tool pruning.
+
+## Message Structure
+```js
+{
+  role: "system" | "assistant" | "user" | "tool" | "log" | "reasoning",
+  content: Content[],
+  name?: string,
+  sticky?: boolean,  // Preserve on reset
+  ts?: number        // Timestamp
+}
+```
+
+### Content Types
+| Type | Role Constraints | Structure |
+|------|------------------|-----------|
+| `text` | system/assistant/user | `{ type: "text", text: string }` |
+| `image_url` | user | `{ type: "image_url", url: string \| data:URL }` |
+| `audio_url` | user | `{ type: "audio_url", url: string \| data:URL }` |
+| `function_request` | assistant | `{ type: "function_request", function_request: { name: string, id: string, call_id?: string, parameters: string (JSON) } }` |
+| `function_response` | tool | `{ type: "function_response", function_response: { name: string, id: string, call_id?: string, response: string (JSON) } }` |
+
+## Record Structure (API Logs)
+```js
+{
+  id: string,
+  isoDate: string,
+  duration: number (ms),
+  environment: string,
+  model: string,
+  tokensIn: number,
+  tokensOut: number
+}
+```
+
+## Constants
+```js
+const MESSAGE_ROLES = { SYSTEM: "system", ASSISTANT: "assistant", ... };
+const MESSAGE_TYPES = { TEXT: "text", IMAGE_URL: "image_url", ... };
+const EVENTS = { start: "start", finished: "finished", tool_request: "tool_request", ... };
+```
+
+## Usage Example
+```js
+import { Prompt, ToolSet } from '@j-o-r/hello-dave';
+import { openaiText } from './API/openai.com/text.js';  // Adapter example
+
+const prompt = new Prompt(128000);  // Context window
+prompt.setAdaptor(openaiText, new ToolSet(), { model: "gpt-4o" });
+
+prompt.add("system", "You are a helpful assistant.", true);
+const response = await prompt.call("Hello!");
+console.log(response);  // Assistant response
+```
+
+## Properties
+| Name | Type | Description |
+|------|------|-------------|
+| `contextLength` | `number` | Max tokens (getter) |
+| `messages` | `Message[]` | Copy of messages (getter/setter validates/imports) |
+| `length` | `number` | Message count |
+| `system_prompt` | `string` | System message text (throws if missing) |
+| `hasSystemprompt` | `boolean` | System prompt present? |
+| `toolset` | `ToolSet \| void` | Current toolset |
+| `EVENTS` | `object` | Frozen event names |
+
+## Methods
+
+### Constructor
+```js
+new Prompt(contextWindow?: number)
+```
+- `contextWindow = 0`: One-shot (no history buildup).
+
+### Setup
+- `setAdaptor(requestHandler, toolset?: ToolSet, options?)`: Bind AI provider.
+- `toolsetFunctions()`: `string[]` of available tools.
+
+### Adding Messages
+- `add(role: Roles, text: string, sticky = false)`: Text message.
+- `addMultiModal(role: Roles, content: Content[], sticky = false)`: Multi-modal.
+  - Validates role/content, order (system first/sticky), emits `message`.
+
+### Conversation Flow
+- `call(content: string \| Content[])`: Add user input, call adapter, return text response. Emits `start`, `finished`.
+- `triggerRequest()`: Auto-trigger if last message is user/tool (for tools first).
+
+### Management
+- `reset()`: Remove non-sticky, return removed. Emits `reset`.
+- `truncate()`: Prune tools, remove old user-assistant blocks to fit context. Emits `truncated`.
+- `countTokens(str?: string)`: Tokens in string or last total.
+- `contentToString(content)`: Concat text from content.
+- `getLastMessage()`: Last message.
+
+### Records & Info
+- `addRecord(record)` / `addRecords(records[])`: Log API usage, update tokens, warn if over limit.
+- `templateRecord()`: Empty record.
+- `info()`: Debug string (cwd, tools, context, tokens).
+
+## Events
+Emitted via `EventEmitter`:
+
+| Event | Triggered When | Data |
+|-------|----------------|------|
+| `message` | Message added | `Message` |
+| `start` / `retrigger` / `finished` | Request lifecycle | `true` |
+| `record` | Record added | `Record` |
+| `truncated` / `reset` | History altered | Event name |
+| `tool_request` / `tool_response` / `tool_error` | Tool lifecycle | Tool data |
+| `error` / `warning` | Issues | Error/Warning |
+
+## Validation
+- Strict checks on message roles/types/content.
+- Records must have all fields.
+- Order: System first/sticky sequential.
+
+## Truncation Details
+1. Prune resolved tool calls (keep last per ID).
+2. Sum sticky tokens.
+3. From end: Remove full user-assistant blocks exceeding window.
+
+## Integration Notes
+- Adapters handle provider-specific formatting (e.g., OpenAI, xAI).
+- Tools execute automatically in `triggerRequest`.
+- Supports reasoning role (xAI-specific).

package/docs.bak.1774780058/prompt-class.md.bak

@@ -0,0 +1,142 @@
+# Prompt Class
+
+Source: \`lib/Prompt.js\`
+
+The \`Prompt\` class is an EventEmitter-based manager for standardized AI conversation messages. It supports multi-modal content (text, images, audio), tool/function calls, token counting, automatic truncation, and adapters for different LLM providers (OpenAI, xAI, Anthropic).
+
+## Key Features
+- Uniform message format across providers.
+- Context window management with truncation.
+- Toolset integration for function calling.
+- Event-driven (add, start, finished, tool_request, etc.).
+- Sticky system prompts preserved on reset/truncate.
+
+## Dependencies
+- \`gpt-3-encoder\` for token counting.
+- \`node:events\` EventEmitter.
+- \`./promptHelpers.js\` for tool pruning.
+
+## Message Structure
+```js
+{
+  role: "system" | "assistant" | "user" | "tool" | "log" | "reasoning",
+  content: Content[],
+  name?: string,
+  sticky?: boolean,  // Preserve on reset
+  ts?: number        // Timestamp
+}
+```
+
+### Content Types
+| Type | Role Constraints | Structure |
+|------|------------------|-----------|
+| `text` | system/assistant/user | `{ type: "text", text: string }` |
+| `image_url` | user | `{ type: "image_url", url: string \| data:URL }` |
+| `audio_url` | user | `{ type: "audio_url", url: string \| data:URL }` |
+| `function_request` | assistant | `{ type: "function_request", function_request: { name: string, id: string, call_id?: string, parameters: string (JSON) } }` |
+| `function_response` | tool | `{ type: "function_response", function_response: { name: string, id: string, call_id?: string, response: string (JSON) } }` |
+
+## Record Structure (API Logs)
+```js
+{
+  id: string,
+  isoDate: string,
+  duration: number (ms),
+  environment: string,
+  model: string,
+  tokensIn: number,
+  tokensOut: number
+}
+```
+
+## Constants
+```js
+const MESSAGE_ROLES = { SYSTEM: "system", ASSISTANT: "assistant", ... };
+const MESSAGE_TYPES = { TEXT: "text", IMAGE_URL: "image_url", ... };
+const EVENTS = { start: "start", finished: "finished", tool_request: "tool_request", ... };
+```
+
+## Usage Example
+```js
+import Prompt from "./Prompt.js";
+import { openaiText } from "./API/openai.com/text.js";  // Adapter example
+import ToolSet from "./ToolSet.js";
+
+const prompt = new Prompt(128000);  // Context window
+prompt.setAdaptor(openaiText, new ToolSet(), { model: "gpt-4o" });
+
+prompt.add("system", "You are a helpful assistant.", true);
+const response = await prompt.call("Hello!");
+console.log(response);  // Assistant response
+```
+
+## Properties
+| Name | Type | Description |
+|------|------|-------------|
+| `contextLength` | `number` | Max tokens (getter) |
+| `messages` | `Message[]` | Copy of messages (getter/setter validates/imports) |
+| `length` | `number` | Message count |
+| `system_prompt` | `string` | System message text (throws if missing) |
+| `hasSystemprompt` | `boolean` | System prompt present? |
+| `toolset` | `ToolSet \| void` | Current toolset |
+| `EVENTS` | `object` | Frozen event names |
+
+## Methods
+
+### Constructor
+```js
+new Prompt(contextWindow?: number)
+```
+- `contextWindow = 0`: One-shot (no history buildup).
+
+### Setup
+- `setAdaptor(requestHandler, toolset?: ToolSet, options?)`: Bind AI provider.
+- `toolsetFunctions()`: `string[]` of available tools.
+
+### Adding Messages
+- `add(role: Roles, text: string, sticky = false)`: Text message.
+- `addMultiModal(role: Roles, content: Content[], sticky = false)`: Multi-modal.
+  - Validates role/content, order (system first/sticky), emits `message`.
+
+### Conversation Flow
+- `call(content: string \| Content[])`: Add user input, call adapter, return text response. Emits `start`, `finished`.
+- `triggerRequest()`: Auto-trigger if last message is user/tool (for tools first).
+
+### Management
+- `reset()`: Remove non-sticky, return removed. Emits `reset`.
+- `truncate()`: Prune tools, remove old user-assistant blocks to fit context. Emits `truncated`.
+- `countTokens(str?: string)`: Tokens in string or last total.
+- `contentToString(content)`: Concat text from content.
+- `getLastMessage()`: Last message.
+
+### Records & Info
+- `addRecord(record)` / `addRecords(records[])`: Log API usage, update tokens, warn if over limit.
+- `templateRecord()`: Empty record.
+- `info()`: Debug string (cwd, tools, context, tokens).
+
+## Events
+Emitted via `EventEmitter`:
+
+| Event | Triggered When | Data |
+|-------|----------------|------|
+| `message` | Message added | `Message` |
+| `start` / `retrigger` / `finished` | Request lifecycle | `true` |
+| `record` | Record added | `Record` |
+| `truncated` / `reset` | History altered | Event name |
+| `tool_request` / `tool_response` / `tool_error` | Tool lifecycle | Tool data |
+| `error` / `warning` | Issues | Error/Warning |
+
+## Validation
+- Strict checks on message roles/types/content.
+- Records must have all fields.
+- Order: System first/sticky sequential.
+
+## Truncation Details
+1. Prune resolved tool calls (keep last per ID).
+2. Sum sticky tokens.
+3. From end: Remove full user-assistant blocks exceeding window.
+
+## Integration Notes
+- Adapters handle provider-specific formatting (e.g., OpenAI, xAI).
+- Tools execute automatically in `triggerRequest`.
+- Supports reasoning role (xAI-specific).

package/docs.bak.1774780058/tools-syntax-validation.md

@@ -0,0 +1,121 @@
+# Syntax Validation in Generic Tools
+
+## Overview
+**Integration Complete**: `lib/genericToolset.js` now includes **automatic JS syntax validation** in `write_file` **and a new `syntax_check` tool** using `./utils/syntax_check.sh` for multi-language support (JS/Python/Bash/JSON). Prevents invalid code, resolves escaping issues, auto `chmod +x` shebangs. Post-write errors prompt LLM retries.
+ **Path Note**: Uses `__dirname` for utils/; resolves files to `process.cwd()`. See [path-resolution-best-practices.md](path-resolution-best-practices.md).
+
+**Key Benefits**:
+- **JS Safety**: `node --check` (in `write_file`); full multi-lang in `syntax_check`.
+ **Path Note**: Uses `__dirname` for utils/; resolves files to `process.cwd()`. See [path-resolution-best-practices.md](path-resolution-best-practices.md).
+- **Executable Prep**: Auto `chmod +x`.
+- **Retry-Friendly**: Descriptive errors + previews.
+- **Generic Access**: `toolsetMode: 'auto'` loads both.
+
+**Commit History** (relevant):
+- `3c8e1ae`: Initial `read_file`/`write_file`.
+- Recent: JS validation + `syntax_check` tool.
+ **Path Note**: Uses `__dirname` for utils/; resolves files to `process.cwd()`. See [path-resolution-best-practices.md](path-resolution-best-practices.md).
+
+## write_file Tool Updates
+
+### Description (Updated)
+```
+Write raw content to a file strictly within the current working directory (CWD). Paths must be relative (no leading /, no ..). Content written as-is (no escaping). **AUTO-VALIDATES JS FILES** with `node --check`; chmod +x shebangs. Retries syntax errors force LLM fix.
+```
+
+### Examples
+**Success** (CLI/Agent):
+```bash
+echo 'const x = \`hi\`; console.log(x);' | agent.directCall("Use write_file on test.js with this content")
+# Returns: "Successfully wrote to 'test.js' (42 bytes). ✓ JS syntax OK"
+```
+
+**Failure + Retry**:
+```javascript
+// LLM generates invalid:
+const msg = `Hello ${name`;  // Missing `
+```
+**Error**:
+```
+❌ JS SYNTAX ERROR in 'test.js' (node --check failed):
+SyntaxError: Unexpected token '}'
+
+PREVIEW:
+const msg = `Hello ${name
+...
+
+... Fix escaping/backticks in template literals (use \\\\` for \` in JS strings) and retry.
+```
+**LLM Sees → Fixes**: Uses `"Hello \${name}"` or proper `` \` ``.
+
+**Shebang**:
+```javascript
+content: '#!/usr/bin/env node\nconst hello = () => console.log("Hi");'
+# Returns: "... ✓ JS syntax OK ✓ chmod +x"
+```
+
+## New: syntax_check Tool
+ **Path Note**: Uses `__dirname` for utils/; resolves files to `process.cwd()`. See [path-resolution-best-practices.md](path-resolution-best-practices.md).
+
+**Description**:
+```
+Validate syntax of a file (JS, Python, Bash/Sh, JSON). Detects language from extension/shebang. Uses native checkers... Exit 1 on fail.
+```
+
+### Parameters
+```json
+{
+  "type": "object",
+  "properties": { "file": { "type": "string" } },
+  "required": ["file"]
+}
+```
+
+### Examples
+**Agent Call**:
+```
+agent.directCall("Use syntax_check on test.js")  → "🔍 Validating test.js (lang: js)\n✅ JS OK\n✅ Syntax validation passed for test.js"
+ **Path Note**: Uses `__dirname` for utils/; resolves files to `process.cwd()`. See [path-resolution-best-practices.md](path-resolution-best-practices.md).
+```
+
+**Multi-Lang**:
+- **Python**: `syntax_check script.py` → `✅ Python OK`
+ **Path Note**: Uses `__dirname` for utils/; resolves files to `process.cwd()`. See [path-resolution-best-practices.md](path-resolution-best-practices.md).
+- **Bash**: `syntax_check script.sh` → `✅ Bash OK` ( + Shellcheck if avail.)
+ **Path Note**: Uses `__dirname` for utils/; resolves files to `process.cwd()`. See [path-resolution-best-practices.md](path-resolution-best-practices.md).
+- **JSON**: `syntax_check config.json` → `✅ JSON OK`
+ **Path Note**: Uses `__dirname` for utils/; resolves files to `process.cwd()`. See [path-resolution-best-practices.md](path-resolution-best-practices.md).
+- **Fail**: Invalid JSON → `❌ JSON invalid`
+
+**Integration**: Post-`write_file`, agent can call `syntax_check(file)` for non-JS.
+ **Path Note**: Uses `__dirname` for utils/; resolves files to `process.cwd()`. See [path-resolution-best-practices.md](path-resolution-best-practices.md).
+
+## utils/syntax_check.sh (Underlying Script)
+ **Path Note**: Uses `__dirname` for utils/; resolves files to `process.cwd()`. See [path-resolution-best-practices.md](path-resolution-best-practices.md).
+[Full source in prior doc]. Detects lang, runs checkers. Standalone/executable.
+
+### Usage (Direct)
+```bash
+./utils/syntax_check.sh test.js      # ✅ JS OK
+ **Path Note**: Uses `__dirname` for utils/; resolves files to `process.cwd()`. See [path-resolution-best-practices.md](path-resolution-best-practices.md).
+./utils/syntax_check.sh bad.py       # ❌ Python syntax error
+ **Path Note**: Uses `__dirname` for utils/; resolves files to `process.cwd()`. See [path-resolution-best-practices.md](path-resolution-best-practices.md).
+```
+
+## Agent Workflow Example
+```
+User: "Create & validate test.js: const add = (a,b) => a+b; export {add};"
+Agent:
+1. write_file(test.js, content)  → ✓ JS OK
+2. syntax_check(test.js)        → ✅ (redundant but confirms)
+ **Path Note**: Uses `__dirname` for utils/; resolves files to `process.cwd()`. See [path-resolution-best-practices.md](path-resolution-best-practices.md).
+```
+
+## Cross-Links
+- [ToolSet](../toolset.md#generic-tools) ← [Linked below](#update-to-toolsetmd)
+- [AgentManager](../agent-manager.md)
+- Source: [lib/genericToolset.js](../lib/genericToolset.js), [utils/syntax_check.sh](../utils/syntax_check.sh)
+ **Path Note**: Uses `__dirname` for utils/; resolves files to `process.cwd()`. See [path-resolution-best-practices.md](path-resolution-best-practices.md).
+
+**Updated**: March 27, 2026 (syntax_check integrated).
+ **Path Note**: Uses `__dirname` for utils/; resolves files to `process.cwd()`. See [path-resolution-best-practices.md](path-resolution-best-practices.md).