@semantictools/ai-toolbox 0.1.0

package/README.md

@@ -0,0 +1,778 @@
+# @semantictools/ai-toolbox
+
+A flexible tool manager for managing and executing AI function calling toolboxes. This package provides a centralized system for registering, organizing, and executing functions that can be used with AI models supporting function calling (like Anthropic Claude, OpenAI GPT, etc.).
+
+**Note:** This package uses a standardized interface format. See the [Interface Standard](#interface-standard) section below for detailed specifications.
+
+## Features
+
+- **Toolbox Registration**: Register multiple toolboxes with their own sets of functions
+- **Hierarchical Organization**: Support for root toolboxes and namespaced function calls
+- **Function Discovery**: List and find functions across all registered toolboxes
+- **Flexible Calling**: Call functions using dot notation (e.g., `toolbox.function`)
+- **Built-in Translators**: Production-ready translator included (`atfunc`) for AI text-based function calling
+- **Custom Translators**: Support for custom function interface translators with configurable case sensitivity
+- **Response Parsing**: Extract and execute function calls from AI model text responses
+- **Control Functions**: Built-in introspection functions (domain: `tools`) for dynamic function discovery
+- **Interface Validation**: Automatic validation of function interfaces during registration
+- **Parameter Validation**: Runtime validation of required parameters with clear error messages
+- **Statistics Tracking**: Monitor tool and function call counts
+- **Result Truncation**: Configurable truncation for long function outputs
+- **Standardized Interface**: Well-defined interface format for consistent toolbox development
+
+## Installation
+
+```bash
+npm install @semantictools/ai-toolbox
+```
+
+## Quick Start
+
+```javascript
+import {
+  register,
+  setRootToolBox,
+  setFunctionTranslator,
+  TRANSLATOR_ATFUNC,
+  call,
+  getFunctions,
+  executeResponseFunctions
+} from '@semantictools/ai-toolbox';
+
+// 1. Set up the built-in translator (do this first!)
+setFunctionTranslator(TRANSLATOR_ATFUNC);
+
+// 2. Create your toolbox
+const myToolBox = {
+  getName: () => 'mytoolbox',
+  getFunction: (fname) => {
+    if (fname === 'greet') {
+      return {
+        f: async (args) => `Hello, ${args.name}!`,
+        interface: {
+          domain: 'mytoolbox',    // Must match getName()
+          name: 'greet',          // Lowercase, snake_case recommended
+          type: 'function',       // 'function' or 'api-wrapper'
+          description: 'Greet a person by name',
+          params: {               // Use 'params', not 'parameters'
+            name: {
+              type: 'string',
+              isRequired: true,   // Use 'isRequired', not 'required'
+              description: 'Person\'s name'
+            }
+          },
+          returns: 'string'
+        }
+      };
+    }
+    return null;
+  },
+  functions: { greet: true }
+};
+
+// 3. Register the toolbox
+register(myToolBox);
+setRootToolBox('mytoolbox');
+
+// 4. Call functions
+const result = await call('greet', { name: 'World' });
+console.log(result); // "Hello, World!"
+```
+
+## API Reference
+
+### Core Functions
+
+#### `register(toolBox)`
+Register a toolbox with the tool manager.
+
+**Parameters:**
+- `toolBox`: Object with `getName()`, `getFunction(fname)`, and `functions` property
+
+#### `setRootToolBox(toolBoxName)`
+Set the root toolbox (functions can be called without namespace prefix).
+
+**Parameters:**
+- `toolBoxName`: String name of the toolbox
+
+#### `call(toolPath, args, truncateLength)`
+Call a function by its path.
+
+**Parameters:**
+- `toolPath`: String in format `'function'` or `'toolbox.function'`
+- `args`: Object with function arguments
+- `truncateLength`: Optional number for result truncation (default: 5000)
+
+**Returns:** Function result
+
+#### `getFunctions(toolBoxName)`
+Get list of all functions or functions in a specific toolbox.
+
+**Parameters:**
+- `toolBoxName`: Optional string to filter by toolbox
+
+**Returns:** Array of function names
+
+#### `findFunction(toolPath)`
+Find a function by its path.
+
+**Parameters:**
+- `toolPath`: String in format `'function'` or `'toolbox.function'`
+
+**Returns:** Function object with `f` (function) and `interface` properties
+
+### Advanced Features
+
+#### `setFunctionTranslator(converterObject)`
+Set a custom function interface converter for AI model compatibility.
+
+**Parameters:**
+- `converterObject`: Translator object, translator name string, or translator constant (e.g., `TRANSLATOR_ATFUNC`)
+
+**Examples:**
+```javascript
+// Option 1: Use translator constant (recommended)
+import { setFunctionTranslator, TRANSLATOR_ATFUNC } from '@semantictools/ai-toolbox';
+setFunctionTranslator(TRANSLATOR_ATFUNC);
+
+// Option 2: Use translator name string
+setFunctionTranslator('atfunc');
+
+// Option 3: Load explicitly with getTranslator
+const translator = getTranslator('atfunc');
+setFunctionTranslator(translator);
+
+// Option 4: Use custom translator object
+import * as myTranslator from './my-translator.mjs';
+setFunctionTranslator(myTranslator);
+```
+
+#### `getTranslator(name)`
+Get a built-in standard translator by name.
+
+**Parameters:**
+- `name`: String name of the translator (e.g., `'atfunc'`)
+
+**Returns:** Translator object with `generateFunctionIfText`, `getFunctionCallsFromText`, and `formatFunctionCallResults`
+
+**Example:**
+```javascript
+import { getTranslator, setFunctionTranslator } from '@semantictools/ai-toolbox';
+
+const translator = getTranslator('atfunc');
+setFunctionTranslator(translator);
+```
+
+#### `getStandardTranslatorList()`
+Get list of all available built-in translators.
+
+**Returns:** Array of translator names
+
+**Example:**
+```javascript
+import { getStandardTranslatorList } from '@semantictools/ai-toolbox';
+
+console.log(getStandardTranslatorList()); // ['atfunc']
+```
+
+### Translator Constants
+
+The package exports constants for easy access to built-in translators:
+
+- **`TRANSLATOR_ATFUNC`** - The atfunc production translator
+
+**Example:**
+```javascript
+import { setFunctionTranslator, TRANSLATOR_ATFUNC } from '@semantictools/ai-toolbox';
+
+// Most convenient way to set a built-in translator
+setFunctionTranslator(TRANSLATOR_ATFUNC);
+```
+
+### Control Functions
+
+The toolmanager provides built-in control functions (domain: `tools`) for introspection and management. These are **registered by default** and can be called like any other function.
+
+#### Available Control Functions
+
+##### `tools.get_toolboxes()`
+Get list of all registered toolboxes.
+
+**Returns:** Array of toolbox names
+
+##### `tools.get_functions(toolbox?)`
+Get all functions, optionally filtered by toolbox.
+
+**Parameters:**
+- `toolbox` (optional): String - Filter by toolbox name
+
+**Returns:** Array of function names
+
+##### `tools.get_function_interface(function_name, separator?)`
+Get the interface definition of a specific function as text.
+
+**Parameters:**
+- `function_name` (required): String - Full function name (e.g., "weather.get_temp")
+- `separator` (optional): Boolean - Add visual separator
+
+**Returns:** String with function interface in text format
+
+##### `tools.set_truncate_length(length)`
+Set the output truncation length for function results.
+
+**Parameters:**
+- `length` (required): Number - Truncation length in characters
+
+**Returns:** Confirmation string
+
+#### Managing Control Functions
+
+```javascript
+import {
+  registerControlFunctions,
+  unregisterControlFunctions,
+  registerControlFunction,
+  unregisterControlFunction
+} from '@semantictools/ai-toolbox';
+
+// Unregister all control functions
+unregisterControlFunctions();
+
+// Re-register all control functions
+registerControlFunctions();
+
+// Register individual control function
+registerControlFunction('get_toolboxes');
+
+// Unregister individual control function
+unregisterControlFunction('get_toolboxes');
+```
+
+#### Usage Examples
+
+**Direct calls:**
+```javascript
+const toolboxes = await call('tools.get_toolboxes');
+const allFunctions = await call('tools.get_functions');
+const weatherFunctions = await call('tools.get_functions', { toolbox: { value: 'weather' } });
+```
+
+**AI text-based calls:**
+```javascript
+const aiResponse = `
+@tools.get_toolboxes()
+@tools.get_functions("weather")
+`;
+const result = await executeResponseFunctions(aiResponse);
+```
+
+**Use cases:**
+- Let AI discover available functions dynamically
+- Allow AI to inspect function interfaces
+- Enable AI to adjust truncation settings
+- Debug and introspection during development
+
+## Validation
+
+The toolmanager includes comprehensive validation to catch errors early and provide clear error messages.
+
+### Interface Validation
+
+Function interfaces are automatically validated when toolboxes are registered. This catches configuration errors before runtime.
+
+**What is validated:**
+- Required fields: `domain`, `name`, `type`, `description`, `returns`
+- `type` must be `'function'` or `'api-wrapper'`
+- `domain` must match toolbox `getName()`
+- Parameter structure: each param must have `type`, `isRequired`, `description`
+- For API wrappers: validates `path_params`, `query_params` structure
+
+**Example error:**
+```javascript
+const badToolbox = {
+  getName: () => 'weather',
+  getFunction: () => ({
+    f: async () => 'result',
+    interface: {
+      domain: 'wrong',  // Doesn't match 'weather'
+      name: 'get_temp',
+      type: 'function',
+      // ... other fields
+    }
+  }),
+  functions: { get_temp: true }
+};
+
+register(badToolbox);
+// Throws: "Validation failed for function 'get_temp' in toolbox 'weather':
+//          Interface domain 'wrong' does not match toolbox name 'weather'"
+```
+
+### Parameter Validation
+
+Required parameters are validated at runtime when functions are called, providing immediate feedback.
+
+**What is validated:**
+- All parameters with `isRequired: true` must be present in function calls
+- Validation happens in both `call()` and `executeResponseFunctions()`
+- Clear error messages identify which parameters are missing
+
+**Example error:**
+```javascript
+// Interface requires 'location' parameter
+await call('weather.get_temp');  // Missing location
+
+// Throws: "Missing required parameter for function 'weather.get_temp': location"
+```
+
+**Handling validation errors in AI responses:**
+```javascript
+const aiResponse = '@weather.get_temp()';  // Missing required param
+const result = await executeResponseFunctions(aiResponse);
+
+// Result includes validation error:
+// "Function call: weather.get_temp()
+//  Returned: Error: Missing required parameter for function 'weather.get_temp': location"
+```
+
+### Benefits
+
+✅ **Fail Fast** - Errors caught at registration/call time, not deep in execution
+✅ **Clear Messages** - Know exactly what's wrong and where
+✅ **Development Aid** - Catch typos and mistakes during development
+✅ **Type Safety** - Ensure interfaces match expected structure
+✅ **AI Safety** - Handle malformed AI function calls gracefully
+
+#### `executeResponseFunctions(promptText)`
+Parse and execute function calls from AI model response text.
+
+**Parameters:**
+- `promptText`: String containing function calls
+
+**Returns:** Object with `functionsOutput` and `functionCalCount`
+
+#### `getRootInterfaceAsText()`
+Get all function interfaces as formatted text.
+
+**Returns:** String with all function interface descriptions
+
+#### `getStats()`
+Get statistics on tool and function calls.
+
+**Returns:** Object with `toolCalls` and `functionCalls` counts
+
+#### `resetStats()`
+Reset call statistics to zero.
+
+## Toolbox Structure
+
+A toolbox must implement:
+
+```javascript
+{
+  getName: () => 'toolbox-name',
+  getFunction: (functionName) => {
+    return {
+      f: async (args, truncateLength) => {
+        // Function implementation
+        return result;
+      },
+      interface: {
+        domain: 'toolbox-name',    // Must match getName()
+        name: 'function_name',     // Lowercase, use snake_case
+        type: 'function',          // 'function' or 'api-wrapper'
+        description: 'What the function does',
+        params: {                  // Use 'params', not 'parameters'
+          param1: {
+            type: 'string',
+            isRequired: true,      // Use 'isRequired', not 'required'
+            description: 'Parameter description'
+          }
+        },
+        returns: 'string' // or 'object', 'array', etc.
+      }
+    };
+  },
+  functions: {
+    'function_name': true
+    // ... list all available functions
+  }
+}
+```
+
+**Important:** Case sensitivity is determined by your translator. The `atfunc` translator is case-insensitive (flexible for AI), while custom translators can be case-sensitive or case-insensitive based on their needs. See the [Case Sensitivity](#case-sensitivity) section for details.
+
+## Interface Standard
+
+### Function Interface Format
+
+All function interfaces must follow this standardized structure:
+
+```javascript
+{
+  domain: 'toolbox-name',    // The toolbox name (must match getName())
+  name: 'function_name',     // Function name (lowercase, snake_case recommended)
+  type: 'function',          // Type: 'function' or 'api-wrapper'
+  description: 'What the function does',
+  params: {                  // Function parameters (note: 'params', not 'parameters')
+    param_name: {
+      type: 'string' | 'number' | 'boolean' | 'object',
+      isRequired: true | false,  // Note: 'isRequired', not 'required'
+      description: 'Parameter description'
+    }
+  },
+  returns: 'string' | 'number' | 'object' | 'array'
+}
+```
+
+#### Required Fields
+
+- **domain**: Must match the toolbox name from `getName()`
+- **name**: Function name (lowercase, preferably snake_case)
+- **type**: Either `'function'` or `'api-wrapper'`
+- **description**: Clear description of what the function does
+- **params**: Object containing parameter definitions
+- **returns**: The return type of the function
+
+#### Naming Conventions
+
+Naming conventions depend on your translator's case sensitivity:
+
+**For case-insensitive translators (like atfunc):**
+- Use any casing style (camelCase, snake_case, PascalCase)
+- `getCurrentWeather`, `get_current_weather`, or `GetCurrentWeather` all work
+- Recommendation: Use snake_case for consistency with the text format
+
+**For case-sensitive translators:**
+- Use exact case matching
+- `getCurrentWeather` ≠ `getcurrentweather`
+- Choose a consistent style (camelCase or snake_case)
+
+**General recommendations:**
+- **Function names**: snake_case (e.g., `get_current_weather`)
+- **Domain/toolbox names**: lowercase or PascalCase
+- **Parameter names**: snake_case for consistency
+
+#### API Wrapper Type
+
+For `type: 'api-wrapper'`, additional fields are supported:
+
+```javascript
+{
+  domain: 'api',
+  name: 'get_user',
+  type: 'api-wrapper',
+  description: 'Get user information',
+  path_params: {
+    user_id: {
+      type: 'string',
+      isRequired: true,
+      description: 'User ID'
+    }
+  },
+  query_params: {
+    include_details: {
+      type: 'boolean',
+      isRequired: false,
+      description: 'Include detailed information'
+    }
+  },
+  request_body: {
+    field_name: {
+      type: 'string',
+      required: true,
+      description: 'Field description'
+    }
+  },
+  response_type: 'application/json',
+  returns: 'object'
+}
+```
+
+### Function Translator Interface
+
+A translator enables text-based function calling for AI model integration. To create a translator, implement these three functions:
+
+#### 1. `generateFunctionIfText(funcInterface, useSeparator)`
+
+Converts a function interface to text format for AI consumption.
+
+**Parameters:**
+- `funcInterface`: The function interface object
+- `useSeparator`: Boolean to add visual separator (optional)
+
+**Returns:** String representation of the function
+
+**Example Output:**
+```
+@weather.get_current_weather( <*location>, <unit> )
+Description: Get current weather information for a location
+```
+
+#### 2. `getFunctionCallsFromText(text, interfaces, rootToolBox)`
+
+Parses text to extract function calls.
+
+**Parameters:**
+- `text`: String containing function calls
+- `interfaces`: Array of all available function interfaces
+- `rootToolBox`: String name of root toolbox (can be null)
+
+**Returns:** Array of function call objects:
+```javascript
+[
+  {
+    name: 'domain.function_name',
+    params: { param_name: value, ... },
+    interface: <matching interface object>
+  }
+]
+```
+
+#### 3. `formatFunctionCallResults(functionCalls)`
+
+Formats function execution results for display.
+
+**Parameters:**
+- `functionCalls`: Array of function call objects with results
+
+**Returns:** Formatted string showing calls and results
+
+### Text Format for Function Calls
+
+When using a translator, functions are represented in this text format:
+
+```
+@domain.function_name( <*required_param>, <optional_param{type}> )
+Description: What the function does
+```
+
+**Format Conventions:**
+- `@` prefix indicates a function call
+- `*` prefix on parameter means required
+- `{type}` suffix shows non-string types (e.g., `{number}`, `{boolean}`)
+- String types have no type suffix
+- Optional `with_body [[ ]]` syntax for API wrappers with request bodies
+
+**Examples:**
+```
+@weather.get_current_weather( <*location>, <unit> )
+Description: Get current weather information for a location
+
+@math.calculate_power( <*base{number}>, <*exponent{number}> )
+Description: Calculate base raised to the power of exponent
+
+@api.create_user( <*username>, <*email> ) with_body [[
+{
+  "username": "john_doe",
+  "email": "john@example.com",
+  "age": 30
+}
+]]
+```
+
+### Setting Up a Translator
+
+#### Using a Built-in Translator (Recommended)
+
+```javascript
+import { setFunctionTranslator, TRANSLATOR_ATFUNC } from '@semantictools/ai-toolbox';
+
+// Set translator BEFORE registering toolboxes (using constant)
+setFunctionTranslator(TRANSLATOR_ATFUNC);
+
+// Alternative: use translator name string
+setFunctionTranslator('atfunc');
+
+// Alternative: load explicitly
+import { getTranslator, setFunctionTranslator } from '@semantictools/ai-toolbox';
+const translator = getTranslator('atfunc');
+setFunctionTranslator(translator);
+```
+
+#### Using a Custom Translator
+
+```javascript
+import * as translator from './my-translator.mjs';
+import { setFunctionTranslator } from '@semantictools/ai-toolbox';
+
+// Set translator BEFORE registering toolboxes
+setFunctionTranslator(translator);
+```
+
+Your custom translator module must export:
+
+```javascript
+// Optional: Declare case sensitivity preference (default: true = case-sensitive)
+export const caseSensitive = false;
+
+export {
+  generateFunctionIfText,
+  getFunctionCallsFromText,
+  formatFunctionCallResults
+};
+```
+
+**Case Sensitivity Property:**
+- `caseSensitive: true` - Exact case matching required (default)
+- `caseSensitive: false` - Case-insensitive matching (like atfunc)
+- If not specified, defaults to `true` (case-sensitive)
+
+### Case Sensitivity
+
+Case sensitivity is **determined by the translator** you use:
+
+- **Default (no translator set)**: Case-sensitive
+- **atfunc translator**: Case-insensitive (handles AI text variability)
+- **Custom translators**: Specify via `caseSensitive` property
+
+**How it works:**
+
+```javascript
+// Custom translator with case sensitivity preference
+export const caseSensitive = false; // or true
+
+export function generateFunctionIfText(...) { ... }
+```
+
+When you set a translator with `setFunctionTranslator()`, the toolmanager automatically adapts to its case sensitivity preference.
+
+**Examples:**
+
+```javascript
+// Using atfunc (case-insensitive)
+setFunctionTranslator(TRANSLATOR_ATFUNC);
+register(myToolbox); // getName() returns 'Weather'
+
+// All of these work:
+call('weather.getTemp');
+call('Weather.getTemp');
+call('WEATHER.GETTEMP');
+```
+
+```javascript
+// Custom case-sensitive translator
+const myTranslator = {
+  caseSensitive: true,
+  generateFunctionIfText: ...,
+  getFunctionCallsFromText: ...,
+  formatFunctionCallResults: ...
+};
+
+setFunctionTranslator(myTranslator);
+register(myToolbox); // getName() returns 'Weather'
+
+// Only exact case works:
+call('Weather.getTemp'); // ✓ Works
+call('weather.getTemp'); // ✗ Error
+```
+
+**Important:** Set the translator **before** registering toolboxes for the case sensitivity to apply correctly.
+
+### Why This Standard?
+
+The interface standard was designed with these principles:
+
+1. **Translator-Driven Case Handling**: Each translator declares its case sensitivity needs, toolmanager adapts
+2. **AI Compatibility**: Text-based format enables natural integration with LLM function calling
+3. **Consistency**: Standardized structure across all toolboxes improves maintainability
+4. **Extensibility**: Supports both simple functions and complex API wrappers
+5. **Type Safety**: Explicit type definitions help with parameter validation and conversion
+6. **Production Tested**: Based on real-world implementations in production systems
+
+### Reference Implementation
+
+The `atfunc` translator is included as a built-in standard translator. You can use it directly with:
+
+```javascript
+import { setFunctionTranslator, TRANSLATOR_ATFUNC } from '@semantictools/ai-toolbox';
+
+setFunctionTranslator(TRANSLATOR_ATFUNC);
+```
+
+This production-ready translator supports:
+- Both `function` and `api-wrapper` types
+- Type conversion (string, number, boolean, object)
+- Quote and escape sequence handling
+- Multi-line request bodies with `with_body [[ ]]` syntax
+- Path parameters, query parameters, and request bodies
+
+The complete interface specification is documented in this README. The reference implementation is available as a built-in translator accessible via `TRANSLATOR_ATFUNC`.
+
+## Migration Guide
+
+If you have existing code using the old interface format, here's how to migrate:
+
+### Old Format (deprecated)
+```javascript
+interface: {
+  name: 'getCurrentWeather',
+  description: 'Get current weather',
+  parameters: {
+    location: {
+      type: 'string',
+      required: true,
+      description: 'City name'
+    }
+  },
+  returns: 'object'
+}
+```
+
+### New Format (standardized)
+```javascript
+interface: {
+  domain: 'weather',           // ← Add domain (must match toolbox name)
+  name: 'get_current_weather', // ← Lowercase with underscores
+  type: 'function',            // ← Add type
+  description: 'Get current weather',
+  params: {                    // ← Changed from 'parameters'
+    location: {
+      type: 'string',
+      isRequired: true,        // ← Changed from 'required'
+      description: 'City name'
+    }
+  },
+  returns: 'object'
+}
+```
+
+**Key Changes:**
+1. Add `domain` field matching your toolbox name
+2. Add `type` field (`'function'` or `'api-wrapper'`)
+3. Convert function names to lowercase snake_case
+4. Rename `parameters` to `params`
+5. Rename `required` to `isRequired`
+
+## Examples
+
+See the [examples](./examples) directory for complete working examples using the standardized interface:
+
+- [Basic Usage](./examples/basic.mjs) - Simple toolbox registration and function calls
+- [Multiple Toolboxes](./examples/multiple-toolboxes.mjs) - Working with multiple toolboxes
+- [AI Integration](./examples/ai-integration.mjs) - Full AI integration with translator
+- [Simple Translator](./examples/simple-translator.mjs) - Basic translator implementation
+
+**Built-in Translator:**
+The `atfunc` translator is included as a built-in and can be used via the `TRANSLATOR_ATFUNC` constant or by name (`'atfunc'`). See the source at [src/translators/atfunc.mjs](./src/translators/atfunc.mjs).
+
+## Use Cases
+
+- **AI Function Calling**: Provide tools for Claude, GPT, or other AI models
+- **Plugin Systems**: Create extensible plugin architectures
+- **Service Orchestration**: Manage and call multiple service APIs
+- **Workflow Automation**: Build flexible automation toolchains
+
+## License
+
+Apache-2.0
+
+Copyright 2026 Dusty Wilhelm Murray / Semantic Tools
+
+## Contributing
+
+Contributions are welcome! Please feel free to submit a Pull Request.
+
+## Author
+
+Dusty Wilhelm Murray / Semantic Tools