@push.rocks/smartagent 1.3.0 → 1.4.1

package/readme.hints.md

@@ -1,15 +1,39 @@
 # Project Readme Hints
 
 ## Overview
-`@push.rocks/smartagent` is an agentic framework built on top of `@push.rocks/smartai`. It provides autonomous AI agent capabilities including tool use, multi-step reasoning, and conversation memory.
+`@push.rocks/smartagent` is a dual-agent agentic framework built on top of `@push.rocks/smartai`. It implements a Driver/Guardian architecture where the Driver proposes tool calls and the Guardian evaluates them against security policies.
 
 ## Architecture
-- **SmartAgent**: Main class that wraps SmartAi and adds agentic behaviors
-- **plugins.ts**: Imports and re-exports smartai
-- **index.ts**: Main entry point, exports SmartAgent class and relevant types
+- **DualAgentOrchestrator**: Main entry point, coordinates Driver and Guardian agents
+- **DriverAgent**: Reasons about tasks, plans steps, proposes tool calls
+- **GuardianAgent**: Evaluates tool calls against configured policies
+- **BaseToolWrapper**: Base class for creating custom tools
+- **plugins.ts**: Imports and re-exports smartai and other dependencies
+
+## Standard Tools
+1. **FilesystemTool** - File operations with scoping and exclusion patterns
+2. **HttpTool** - HTTP requests
+3. **ShellTool** - Secure shell commands (no injection possible)
+4. **BrowserTool** - Web page interaction via Puppeteer
+5. **DenoTool** - Sandboxed TypeScript/JavaScript execution
+6. **JsonValidatorTool** - JSON validation and formatting
+
+## Key Features
+- Token streaming support (`onToken` callback)
+- Vision support (pass images as base64)
+- Progress events (`onProgress` callback)
+- Scoped filesystem with exclusion patterns
+- Result truncation with configurable limits
+- History windowing to manage token usage
 
 ## Key Dependencies
-- `@push.rocks/smartai`: Provides the underlying multi-modal AI provider interface
+- `@push.rocks/smartai`: Multi-provider AI interface
+- `@push.rocks/smartfs`: Filesystem operations
+- `@push.rocks/smartshell`: Shell command execution
+- `@push.rocks/smartbrowser`: Browser automation
+- `@push.rocks/smartdeno`: Deno code execution
+- `@push.rocks/smartrequest`: HTTP requests
+- `minimatch`: Glob pattern matching for exclusions
 
 ## Test Structure
 - Tests use `@git.zone/tstest/tapbundle`

package/readme.md

@@ -50,6 +50,7 @@ flowchart TB
         Shell["Shell"]
         Browser["Browser"]
         Deno["Deno"]
+        JSON["JSON Validator"]
     end
 
     Task --> Orchestrator
@@ -99,7 +100,7 @@ await orchestrator.stop();
 
 ## Standard Tools
 
-SmartAgent comes with five battle-tested tools out of the box:
+SmartAgent comes with six battle-tested tools out of the box:
 
 ### 🗂️ FilesystemTool
 
@@ -117,11 +118,29 @@ File and directory operations powered by `@push.rocks/smartfs`.
 </tool_call>
 ```
 
-**Scoped Filesystem**: Lock file operations to a specific directory:
+**Scoped Filesystem**: Lock file operations to a specific directory with optional exclusion patterns:
 
 ```typescript
 // Only allow access within a specific directory
 orchestrator.registerScopedFilesystemTool('/home/user/workspace');
+
+// With exclusion patterns (glob syntax)
+orchestrator.registerScopedFilesystemTool('/home/user/workspace', [
+  '.nogit/**',
+  'node_modules/**',
+  '*.secret',
+]);
+```
+
+**Line-range Reading**: Read specific portions of large files:
+
+```typescript
+<tool_call>
+  <tool>filesystem</tool>
+  <action>read</action>
+  <params>{"path": "/var/log/app.log", "startLine": 100, "endLine": 150}</params>
+  <reasoning>Reading only the relevant log section to avoid token overload</reasoning>
+</tool_call>
 ```
 
 ### 🌐 HttpTool
@@ -212,6 +231,105 @@ By default, code runs **fully sandboxed with no permissions**. Permissions must
 </tool_call>
 ```
 
+### 📋 JsonValidatorTool
+
+Validate and format JSON data. Perfect for agents to self-check their JSON output before completing tasks.
+
+**Actions**: `validate`, `format`
+
+```typescript
+// Validate JSON with required field checking
+<tool_call>
+  <tool>json</tool>
+  <action>validate</action>
+  <params>{
+    "jsonString": "{\"name\": \"test\", \"version\": \"1.0.0\"}",
+    "requiredFields": ["name", "version", "description"]
+  }</params>
+  <reasoning>Ensuring the config has all required fields before saving</reasoning>
+</tool_call>
+
+// Pretty-print JSON
+<tool_call>
+  <tool>json</tool>
+  <action>format</action>
+  <params>{"jsonString": "{\"compact\":true,\"data\":[1,2,3]}"}</params>
+  <reasoning>Formatting JSON for readable output</reasoning>
+</tool_call>
+```
+
+## 🎥 Streaming Support
+
+SmartAgent supports token-by-token streaming for real-time output during LLM generation:
+
+```typescript
+const orchestrator = new DualAgentOrchestrator({
+  openaiToken: 'sk-...',
+  defaultProvider: 'openai',
+  guardianPolicyPrompt: '...',
+
+  // Token streaming callback
+  onToken: (token, source) => {
+    // source is 'driver' or 'guardian'
+    process.stdout.write(token);
+  },
+});
+```
+
+This is perfect for CLI applications or UIs that need to show progress as the agent thinks.
+
+## 🖼️ Vision Support
+
+Pass images to vision-capable models for multimodal tasks:
+
+```typescript
+import { readFileSync } from 'fs';
+
+// Load image as base64
+const imageBase64 = readFileSync('screenshot.png').toString('base64');
+
+// Run task with images
+const result = await orchestrator.run(
+  'Analyze this UI screenshot and describe any usability issues',
+  { images: [imageBase64] }
+);
+```
+
+## 📊 Progress Events
+
+Get real-time feedback on task execution with the `onProgress` callback:
+
+```typescript
+const orchestrator = new DualAgentOrchestrator({
+  openaiToken: 'sk-...',
+  guardianPolicyPrompt: '...',
+  logPrefix: '[MyAgent]',  // Optional prefix for log messages
+
+  onProgress: (event) => {
+    // Pre-formatted log message ready for output
+    console.log(event.logMessage);
+
+    // Or handle specific event types
+    switch (event.type) {
+      case 'tool_proposed':
+        console.log(`Proposing: ${event.toolName}.${event.action}`);
+        break;
+      case 'tool_approved':
+        console.log(`✓ Approved`);
+        break;
+      case 'tool_rejected':
+        console.log(`✗ Rejected: ${event.reason}`);
+        break;
+      case 'task_completed':
+        console.log(`Done in ${event.iteration} iterations`);
+        break;
+    }
+  },
+});
+```
+
+**Event Types**: `task_started`, `iteration_started`, `tool_proposed`, `guardian_evaluating`, `tool_approved`, `tool_rejected`, `tool_executing`, `tool_completed`, `task_completed`, `clarification_needed`, `max_iterations`, `max_rejections`
+
 ## Guardian Policy Examples
 
 The Guardian's power comes from your policy. Here are battle-tested examples:
@@ -294,10 +412,19 @@ interface IDualAgentOptions {
   // Agent configuration
   driverSystemMessage?: string;     // Custom system message for Driver
   guardianPolicyPrompt: string;     // REQUIRED: Policy for Guardian to enforce
+  name?: string;                    // Agent system name
+  verbose?: boolean;                // Enable verbose logging
 
   // Limits
   maxIterations?: number;           // Max task iterations (default: 20)
   maxConsecutiveRejections?: number; // Abort after N rejections (default: 3)
+  maxResultChars?: number;          // Max chars for tool results before truncation (default: 15000)
+  maxHistoryMessages?: number;      // Max history messages for API (default: 20)
+
+  // Callbacks
+  onProgress?: (event: IProgressEvent) => void;  // Progress event callback
+  onToken?: (token: string, source: 'driver' | 'guardian') => void;  // Streaming callback
+  logPrefix?: string;               // Prefix for log messages
 }
 ```
 
@@ -311,6 +438,10 @@ interface IDualAgentRunResult {
   iterations: number;         // Number of iterations taken
   history: IAgentMessage[];   // Full conversation history
   status: TDualAgentRunStatus; // 'completed' | 'max_iterations_reached' | etc.
+  toolCallCount?: number;     // Number of tool calls made
+  rejectionCount?: number;    // Number of Guardian rejections
+  toolLog?: IToolExecutionLog[]; // Detailed tool execution log
+  error?: string;             // Error message if status is 'error'
 }
 
 type TDualAgentRunStatus =
@@ -365,6 +496,7 @@ class MyCustomTool extends BaseToolWrapper {
       return {
         success: true,
         result: { processed: params.input },
+        summary: `Processed input: ${params.input}`,  // Optional human-readable summary
       };
     }
 
@@ -439,11 +571,11 @@ const orchestrator = new DualAgentOrchestrator({
 |--------|-------------|
 | `start()` | Initialize all tools and AI providers |
 | `stop()` | Cleanup all tools and resources |
-| `run(task: string)` | Execute a task and return result |
-| `continueTask(input: string)` | Continue a task with user input |
+| `run(task, options?)` | Execute a task with optional images for vision |
+| `continueTask(input)` | Continue a task with user input |
 | `registerTool(tool)` | Register a custom tool |
 | `registerStandardTools()` | Register all built-in tools |
-| `registerScopedFilesystemTool(basePath)` | Register filesystem tool with path restriction |
+| `registerScopedFilesystemTool(basePath, excludePatterns?)` | Register filesystem tool with path restriction |
 | `setGuardianPolicy(policy)` | Update Guardian policy at runtime |
 | `getHistory()` | Get conversation history |
 | `getToolNames()` | Get list of registered tool names |
@@ -459,14 +591,18 @@ export { GuardianAgent } from '@push.rocks/smartagent';
 
 // Tools
 export { BaseToolWrapper } from '@push.rocks/smartagent';
-export { FilesystemTool } from '@push.rocks/smartagent';
+export { FilesystemTool, type IFilesystemToolOptions } from '@push.rocks/smartagent';
 export { HttpTool } from '@push.rocks/smartagent';
 export { ShellTool } from '@push.rocks/smartagent';
 export { BrowserTool } from '@push.rocks/smartagent';
-export { DenoTool } from '@push.rocks/smartagent';
+export { DenoTool, type TDenoPermission } from '@push.rocks/smartagent';
+export { JsonValidatorTool } from '@push.rocks/smartagent';
 
 // Types and interfaces
-export * from '@push.rocks/smartagent'; // All interfaces
+export * from '@push.rocks/smartagent';  // All interfaces
+
+// Re-exported from @push.rocks/smartai
+export { type ISmartAiOptions, type TProvider, type ChatMessage, type ChatOptions, type ChatResponse };
 ```
 
 ## License and Legal Information
@@ -483,7 +619,7 @@ Use of these trademarks must comply with Task Venture Capital GmbH's Trademark G
 
 ### Company Information
 
-Task Venture Capital GmbH  
+Task Venture Capital GmbH
 Registered at District Court Bremen HRB 35230 HB, Germany
 
 For any legal inquiries or further information, please contact us via email at hello@task.vc.

package/ts/00_commitinfo_data.ts

@@ -3,6 +3,6 @@
  */
 export const commitinfo = {
   name: '@push.rocks/smartagent',
-  version: '1.3.0',
+  version: '1.4.1',
   description: 'an agentic framework built on top of @push.rocks/smartai'
 }

package/ts/smartagent.classes.dualagent.ts

@@ -319,9 +319,27 @@ export class DualAgentOrchestrator {
       const proposals = this.driver.parseToolCallProposals(driverResponse.content);
 
       if (proposals.length === 0) {
-        // No tool calls, continue the conversation
+        // No tool calls found - remind the model of the exact XML format
         driverResponse = await this.driver.continueWithMessage(
-          'Please either use a tool to make progress on the task, or indicate that the task is complete with <task_complete>summary</task_complete>.'
+          `No valid tool call was found in your response. To use a tool, you MUST output the exact XML format:
+
+<tool_call>
+  <tool>tool_name</tool>
+  <action>action_name</action>
+  <params>{"param1": "value1"}</params>
+</tool_call>
+
+For example, to validate JSON:
+<tool_call>
+  <tool>json</tool>
+  <action>validate</action>
+  <params>{"jsonString": "{\\"key\\":\\"value\\"}", "requiredFields": ["key"]}</params>
+</tool_call>
+
+Or to complete the task:
+<task_complete>your final JSON output here</task_complete>
+
+Please output the exact XML format above.`
         );
         this.conversationHistory.push(driverResponse);
         continue;

package/ts/smartagent.tools.base.ts

@@ -35,6 +35,13 @@ export abstract class BaseToolWrapper implements interfaces.IAgentToolWrapper {
    */
   abstract getCallSummary(action: string, params: Record<string, unknown>): string;
 
+  /**
+   * Get a comprehensive explanation of this tool for LLM consumption.
+   * Tools should implement this to provide detailed usage instructions with examples.
+   * This includes parameter schemas and concrete <tool_call> XML examples.
+   */
+  abstract getToolExplanation(): string;
+
   /**
    * Validate that an action exists for this tool
    * @throws Error if the action is not valid
@@ -60,14 +67,10 @@ export abstract class BaseToolWrapper implements interfaces.IAgentToolWrapper {
 
   /**
    * Get the full tool description including all actions
-   * Used for Driver's tool awareness
+   * Used for Driver's tool awareness - now delegates to getToolExplanation()
    */
   public getFullDescription(): string {
-    const actionDescriptions = this.actions
-      .map((a) => `  - ${a.name}: ${a.description}`)
-      .join('\n');
-
-    return `${this.name}: ${this.description}\nActions:\n${actionDescriptions}`;
+    return this.getToolExplanation();
   }
 
   /**

package/ts/smartagent.tools.browser.ts

@@ -176,6 +176,59 @@ export class BrowserTool extends BaseToolWrapper {
     }
   }
 
+  public getToolExplanation(): string {
+    return `## Tool: browser
+Interact with web pages - take screenshots, generate PDFs, and execute JavaScript on pages.
+
+### Actions:
+
+**screenshot** - Take a screenshot of a webpage
+Parameters:
+  - url (required): URL of the page to screenshot
+
+Example:
+<tool_call>
+  <tool>browser</tool>
+  <action>screenshot</action>
+  <params>{"url": "https://example.com"}</params>
+</tool_call>
+
+**pdf** - Generate a PDF from a webpage
+Parameters:
+  - url (required): URL of the page to convert to PDF
+
+Example:
+<tool_call>
+  <tool>browser</tool>
+  <action>pdf</action>
+  <params>{"url": "https://example.com/report"}</params>
+</tool_call>
+
+**evaluate** - Execute JavaScript code on a webpage and return the result
+Parameters:
+  - url (required): URL of the page to run the script on
+  - script (required): JavaScript code to execute (must return a value)
+
+Example:
+<tool_call>
+  <tool>browser</tool>
+  <action>evaluate</action>
+  <params>{"url": "https://example.com", "script": "document.querySelectorAll('a').length"}</params>
+</tool_call>
+
+**getPageContent** - Get the text content and title of a webpage
+Parameters:
+  - url (required): URL of the page to get content from
+
+Example:
+<tool_call>
+  <tool>browser</tool>
+  <action>getPageContent</action>
+  <params>{"url": "https://example.com"}</params>
+</tool_call>
+`;
+  }
+
   public getCallSummary(action: string, params: Record<string, unknown>): string {
     switch (action) {
       case 'screenshot':

package/ts/smartagent.tools.deno.ts

@@ -164,6 +164,45 @@ export class DenoTool extends BaseToolWrapper {
     }
   }
 
+  public getToolExplanation(): string {
+    return `## Tool: deno
+Execute TypeScript/JavaScript code in a sandboxed Deno environment with fine-grained permission control.
+
+### Actions:
+
+**execute** - Execute TypeScript/JavaScript code and return stdout/stderr
+Parameters:
+  - code (required): TypeScript/JavaScript code to execute
+  - permissions (optional): Array of Deno permissions to grant. Options: "all", "env", "net", "read", "write", "run", "sys", "ffi", "hrtime". Default: none (fully sandboxed)
+
+Example - Simple execution:
+<tool_call>
+  <tool>deno</tool>
+  <action>execute</action>
+  <params>{"code": "console.log('Hello from Deno!');"}</params>
+</tool_call>
+
+Example - With network permission:
+<tool_call>
+  <tool>deno</tool>
+  <action>execute</action>
+  <params>{"code": "const resp = await fetch('https://api.example.com/data');\\nconsole.log(await resp.text());", "permissions": ["net"]}</params>
+</tool_call>
+
+**executeWithResult** - Execute code that outputs JSON on the last line of stdout
+Parameters:
+  - code (required): Code that console.logs a JSON value on the final line
+  - permissions (optional): Array of Deno permissions to grant
+
+Example:
+<tool_call>
+  <tool>deno</tool>
+  <action>executeWithResult</action>
+  <params>{"code": "const result = { sum: 1 + 2, product: 2 * 3 };\\nconsole.log(JSON.stringify(result));"}</params>
+</tool_call>
+`;
+  }
+
   public getCallSummary(action: string, params: Record<string, unknown>): string {
     const code = params.code as string;
     const permissions = (params.permissions as string[]) || [];

package/ts/smartagent.tools.filesystem.ts

@@ -666,6 +666,170 @@ export class FilesystemTool extends BaseToolWrapper {
     }
   }
 
+  public getToolExplanation(): string {
+    return `## Tool: filesystem
+Read, write, list, and delete files and directories.
+
+### Actions:
+
+**read** - Read file contents (full or specific line range)
+Parameters:
+  - path (required): Path to the file
+  - encoding (optional): File encoding - "utf8" (default), "binary", or "base64"
+  - startLine (optional): First line to read (1-indexed, inclusive)
+  - endLine (optional): Last line to read (1-indexed, inclusive)
+
+Example:
+<tool_call>
+  <tool>filesystem</tool>
+  <action>read</action>
+  <params>{"path": "/path/to/file.txt"}</params>
+</tool_call>
+
+Example with line range:
+<tool_call>
+  <tool>filesystem</tool>
+  <action>read</action>
+  <params>{"path": "/path/to/file.txt", "startLine": 10, "endLine": 20}</params>
+</tool_call>
+
+**write** - Write content to a file (creates or overwrites)
+Parameters:
+  - path (required): Absolute path to the file
+  - content (required): Content to write
+  - encoding (optional): File encoding - "utf8" (default), "binary", or "base64"
+
+Example:
+<tool_call>
+  <tool>filesystem</tool>
+  <action>write</action>
+  <params>{"path": "/path/to/output.txt", "content": "Hello, World!"}</params>
+</tool_call>
+
+**list** - List files and directories in a path
+Parameters:
+  - path (required): Directory path to list
+  - recursive (optional): List recursively (default: false)
+  - filter (optional): Glob pattern to filter results (e.g., "*.ts")
+
+Example:
+<tool_call>
+  <tool>filesystem</tool>
+  <action>list</action>
+  <params>{"path": "/path/to/dir", "recursive": true, "filter": "*.ts"}</params>
+</tool_call>
+
+**exists** - Check if a file or directory exists
+Parameters:
+  - path (required): Path to check
+
+Example:
+<tool_call>
+  <tool>filesystem</tool>
+  <action>exists</action>
+  <params>{"path": "/path/to/check"}</params>
+</tool_call>
+
+**mkdir** - Create a directory
+Parameters:
+  - path (required): Directory path to create
+  - recursive (optional): Create parent directories if needed (default: true)
+
+Example:
+<tool_call>
+  <tool>filesystem</tool>
+  <action>mkdir</action>
+  <params>{"path": "/path/to/new/dir"}</params>
+</tool_call>
+
+**delete** - Delete a file or directory
+Parameters:
+  - path (required): Path to delete
+  - recursive (optional): For directories, delete recursively (default: false)
+
+Example:
+<tool_call>
+  <tool>filesystem</tool>
+  <action>delete</action>
+  <params>{"path": "/path/to/delete", "recursive": true}</params>
+</tool_call>
+
+**copy** - Copy a file to a new location
+Parameters:
+  - source (required): Source file path
+  - destination (required): Destination file path
+
+Example:
+<tool_call>
+  <tool>filesystem</tool>
+  <action>copy</action>
+  <params>{"source": "/path/to/source.txt", "destination": "/path/to/dest.txt"}</params>
+</tool_call>
+
+**move** - Move a file to a new location
+Parameters:
+  - source (required): Source file path
+  - destination (required): Destination file path
+
+Example:
+<tool_call>
+  <tool>filesystem</tool>
+  <action>move</action>
+  <params>{"source": "/path/to/old.txt", "destination": "/path/to/new.txt"}</params>
+</tool_call>
+
+**stat** - Get file or directory statistics (size, dates, etc.)
+Parameters:
+  - path (required): Path to get stats for
+
+Example:
+<tool_call>
+  <tool>filesystem</tool>
+  <action>stat</action>
+  <params>{"path": "/path/to/file.txt"}</params>
+</tool_call>
+
+**append** - Append content to a file
+Parameters:
+  - path (required): Absolute path to the file
+  - content (required): Content to append
+
+Example:
+<tool_call>
+  <tool>filesystem</tool>
+  <action>append</action>
+  <params>{"path": "/path/to/log.txt", "content": "New log entry\\n"}</params>
+</tool_call>
+
+**tree** - Show directory structure as a tree
+Parameters:
+  - path (required): Root directory path
+  - maxDepth (optional): Maximum depth to traverse (default: 3)
+  - filter (optional): Glob pattern to filter files
+  - showSizes (optional): Include file sizes in output (default: false)
+  - format (optional): Output format - "string" (default) or "json"
+
+Example:
+<tool_call>
+  <tool>filesystem</tool>
+  <action>tree</action>
+  <params>{"path": "/path/to/dir", "maxDepth": 2}</params>
+</tool_call>
+
+**glob** - Find files matching a glob pattern
+Parameters:
+  - pattern (required): Glob pattern (e.g., "**/*.ts", "src/**/*.js")
+  - path (optional): Base path to search from
+
+Example:
+<tool_call>
+  <tool>filesystem</tool>
+  <action>glob</action>
+  <params>{"pattern": "**/*.ts", "path": "/path/to/project"}</params>
+</tool_call>
+`;
+  }
+
   public getCallSummary(action: string, params: Record<string, unknown>): string {
     switch (action) {
       case 'read': {