toolpack-sdk 2.7.0 → 3.1.0

package/README.md

@@ -19,7 +19,7 @@ The TypeScript SDK for building production AI agents — 100+ built-in tools, 8
 - **Rules** — Always-on behavioral constraints loaded from Markdown files, auto-discovered per mode and globally via `rulesDir`
 - **HITL Confirmation** — Human-in-the-loop approval for high-risk operations with configurable bypass rules
 - **Extensible at Every Layer** — Every built-in component is a plug-in point: custom tools (`ToolDefinition`), custom channels (`BaseChannel`), custom provider adapters (`ProviderAdapter`), custom agents (`BaseAgent`), custom modes (`createMode()`), and custom interceptors — all using the same interfaces as the built-ins
-- **100+ Built-in Tools** across 12 categories:
+- **100+ Built-in Tools** across 15 categories:
 - **MCP Client & Server** — consume external MCP servers via `createMcpToolProject()`, or expose Toolpack as an MCP server via `sdk.startMcpServer()` with static/JWT/custom auth, search mode, and agent exposure.
 
 | Category | Tools | Description |
@@ -196,9 +196,11 @@ const mcpToolProject = await createMcpToolProject({
 const sdk = await Toolpack.init({
   provider: 'openai',
   tools: true,
-  customTools: [mcpToolProject],
 });
 
+// Use MCP tools via ModeConfig.customTools (per-agent or per-request):
+// agent.mode = { ...myMode, customTools: [...mcpToolProject.tools] };
+
 // On shutdown/cold path:
 // await disconnectMcpToolProject(mcpToolProject);
 ```
@@ -344,6 +346,8 @@ sdk.cycleMode(); // Cycles through all registered modes
 | `blockAllTools` | boolean | `false` | If `true`, disables all tools (pure conversation) |
 | `baseContext` | object/false | `undefined` | Controls working directory and tool category injection |
 | `workflow` | WorkflowConfig | `undefined` | Planning, execution mode, and progress configuration |
+| `customTools` | ToolDefinition[] | `[]` | Agent-specific tools — never shared across agents |
+| `toolsConfig` | Partial\<ToolsConfig\> | `{}` | Per-agent tool behavior overrides (merged over global `toolsConfig`) |
 
 ## Workflow Engine
 
@@ -501,7 +505,7 @@ client.on('tool:failed', (event) => { /* ... */ });
 
 ## Custom Tools
 
-In addition to the 100+ built-in tools, you can create and register your own custom tool projects using `createToolProject()`:
+In addition to the 100+ built-in tools, you can create and register your own custom tool projects using `createToolProject()`. Built-in tool categories used for mode filtering include `filesystem`, `coding`, `version-control`, `http`, `web`, `github`, `slack`, `execution`, `system`, and others — there is no combined `network` category.
 
 ```typescript
 import { Toolpack, createToolProject } from 'toolpack-sdk';
@@ -535,12 +539,25 @@ const myToolProject = createToolProject({
   ],
 });
 
-// Register custom tools at init
+// Attach custom tools to a mode — scoped to that agent, not global.
+// To replace built-ins by name at init time, pass toolOverrides: [myToolProject].
 const sdk = await Toolpack.init({
   provider: 'openai',
-  tools: true,              // Load built-in tools
-  customTools: [myToolProject], // Add your custom tools
+  tools: true,
+});
+
+// Option A: set on your agent class
+// agent.mode = { ...agentMode, customTools: [...myToolProject.tools] };
+
+// Option B: pass inline per-request
+const result = await sdk.generate({
+  messages: [{ role: 'user', content: 'Use my tool!' }],
+  model: 'gpt-4.1',
+  mode: { ...sdk.getMode()!, customTools: [...myToolProject.tools] },
 });
+
+// Option C: load projects at runtime (rebuilds the tool search index once)
+await sdk.loadToolProjects([myToolProject]);
 ```
 
 ### Tool Project Structure
@@ -593,11 +610,12 @@ const response = await toolpack.chat('How do I configure authentication?');
 
 - **Multiple Providers**: In-memory (`MemoryProvider`) or persistent SQLite (`PersistentKnowledgeProvider`)
 - **Multiple Embedders**: OpenAI, Ollama (local), or custom embedders
-- **Multiple Sources**: Markdown, JSON, SQLite ingestion
+- **Multiple Sources**: Markdown, text, web, API, JSON, SQLite, and more
 - **Progress Events**: Track embedding progress with `onEmbeddingProgress`
 - **Metadata Filtering**: Query with filters like `{ hasCode: true, category: 'api' }`
+- **Incremental updates**: `ingest()`, `delete()`, and `deleteWhere()` for per-item lifecycle
 
-See the [Knowledge package README](./packages/toolpack-knowledge/README.md) for full documentation.
+See the [Knowledge package README](../toolpack-knowledge/README.md) for full documentation.
 
 ## Skills
 
@@ -608,15 +626,17 @@ The skills system lets you define **reusable behavioral instructions** in `.skil
 ```typescript
 import { Toolpack, createSkillInterceptor, createSkillTools } from 'toolpack-sdk';
 
+const skillTools = createSkillTools({ dir: '.toolpack/skills' });
+
 const toolpack = await Toolpack.init({
   provider: 'anthropic',
   interceptors: [
     createSkillInterceptor({ dir: '.toolpack/skills', maxSkills: 3, minScore: 0.3 }),
   ],
-  customTools: [
-    createSkillTools({ dir: '.toolpack/skills' }),
-  ],
 });
+
+// Attach skill tools per agent via ModeConfig.customTools:
+// agent.mode = { ...agentMode, customTools: [...skillTools.tools] };
 ```
 
 Create a skill file at `.toolpack/skills/code-review.skill.md`:
@@ -1076,148 +1096,104 @@ export ANTHROPIC_API_KEY="sk-ant-..."
 export GOOGLE_GENERATIVE_AI_KEY="AIza..."
 export OPENROUTER_API_KEY="sk-or-..."
 
-# SDK logging (override — prefer toolpack.config.json instead)
+# SDK logging (override programmatic config)
 export TOOLPACK_SDK_LOG_FILE="./toolpack.log"    # Log file path (also enables logging)
-export TOOLPACK_SDK_LOG_LEVEL="debug"            # Log level override (error, warn, info, debug, trace)
+export TOOLPACK_SDK_LOG_LEVEL="debug"            # Log level (error, warn, info, debug, trace)
+export TOOLPACK_SDK_LOG_ENABLED="true"           # Enable/disable logging
+export TOOLPACK_SDK_LOG_CONSOLE="true"           # Mirror log output to console
 ```
 
-## Configuration Architecture
-
-Toolpack uses a hierarchical configuration system that separates build-time (SDK) and runtime (CLI) configurations.
-
-### Configuration Layers
-
-1. **Workspace Local (Highest Priority)**
-   - Location: `<workspace>/.toolpack/config/toolpack.config.json`
-   - Purpose: Project-specific overrides for the CLI tool.
-
-2. **Global Default (CLI First Run)**
-   - Location: `~/.toolpack/config/toolpack.config.json`
-   - Purpose: Global default settings for the CLI tool across all projects. Created automatically on first run.
-
-3. **Build Time / SDK Base**
-   - Location: `toolpack.config.json` in project root.
-   - Purpose: Static configuration used when bundling the SDK or running it directly in an app.
-
-### Settings UI
-
-The CLI includes a settings screen to view the active configuration source and its location. Press `Ctrl+S` from the Home screen to access it.
-
-### Configuration Sections
+### Logging Configuration
 
-The `toolpack.config.json` file supports several sections:
+Pass `logging` to `Toolpack.init()`:
 
-#### Global Options
-
-| Option | Default | Description |
-|--------|---------|-------------|
-| `systemPrompt` | - | Override the base system prompt |
-| `baseContext` | `true` | Agent context configuration (`{ includeWorkingDirectory, includeToolCategories, custom }` or `false`) |
-| `modeOverrides` | `{}` | Mode-specific system prompt and toolSearch overrides |
-
-#### Logging Configuration
-
-Create a `toolpack.config.json` in your project root:
-
-```json
-{
-  "logging": {
-    "enabled": true,
-    "filePath": "./toolpack.log",
-    "level": "info"
-  }
-}
+```typescript
+const sdk = await Toolpack.init({
+  provider: 'anthropic',
+  logging: {
+    enabled: true,
+    filePath: './toolpack.log',
+    level: 'debug',
+    console: false,
+  },
+});
 ```
 
-| Option | Default | Description |
-|--------|---------|-------------|
-| `enabled` | `false` | Enable file logging |
-| `filePath` | `toolpack-sdk.log` | Log file path (relative to CWD) |
-| `level` | `info` | Log level (`error`, `warn`, `info`, `debug`, `trace`) |
+| Option | Type | Default | Description |
+|--------|------|---------|-------------|
+| `enabled` | boolean | `false` | Enable file logging |
+| `filePath` | string | `toolpack-sdk.log` | Log file path (relative to CWD) |
+| `level` | string | `info` | Log level (`error`, `warn`, `info`, `debug`, `trace`) |
+| `console` | boolean | `false` | Mirror log output to console |
+
+Environment variables override programmatic config (highest precedence).
 
 ### Tools Configuration
 
-Create a `toolpack.config.json` in your project root:
-
-```json
-{
-  "tools": {
-    "enabled": true,
-    "autoExecute": true,
-    "maxToolRounds": 5,
-    "toolChoicePolicy": "auto",
-    "resultMaxChars": 20000,
-    "enabledTools": [],
-    "enabledToolCategories": [],
-    "additionalConfigurations": {
-      "webSearch": {
-        "tavilyApiKey": "tvly-...",
-        "braveApiKey": "BSA..."
-      }
+Pass `toolsConfig` to `Toolpack.init()` for global defaults, or set `ModeConfig.toolsConfig` for per-agent overrides:
+
+```typescript
+const sdk = await Toolpack.init({
+  provider: 'anthropic',
+  tools: true,
+  toolsConfig: {
+    autoExecute: true,
+    maxToolRounds: 10,
+    toolChoicePolicy: 'auto',
+    resultMaxChars: 20000,
+    additionalConfigurations: {
+      webSearch: {
+        tavilyApiKey: 'tvly-...',
+        braveApiKey: 'BSA...',
+      },
     },
-    "toolSearch": {
-      "enabled": false,
-      "alwaysLoadedTools": ["fs.read_file", "fs.write_file", "fs.list_dir"],
-      "alwaysLoadedCategories": [],
-      "searchResultLimit": 5,
-      "cacheDiscoveredTools": true
-    }
-  }
-}
+    toolSearch: {
+      enabled: false,
+      alwaysLoadedTools: ['fs.read_file', 'fs.write_file', 'fs.list_dir'],
+      searchResultLimit: 5,
+      cacheDiscoveredTools: true,
+    },
+  },
+});
 ```
 
-#### Configuration Options
-
 | Option | Type | Default | Description |
 |--------|------|---------|-------------|
-| `enabled` | boolean | `true` | Enable/disable tool system |
+| `enabled` | boolean | `true` | Enable/disable the tool system entirely |
 | `autoExecute` | boolean | `true` | Auto-execute tool calls from AI |
 | `maxToolRounds` | number | `5` | Max tool execution rounds per request |
 | `toolChoicePolicy` | string | `"auto"` | `"auto"`, `"required"`, or `"required_for_actions"` |
+| `resultMaxChars` | number | `20000` | Max characters in a tool result |
 | `enabledTools` | string[] | `[]` | Whitelist specific tools (empty = all) |
 | `enabledToolCategories` | string[] | `[]` | Whitelist categories (empty = all) |
 
 ### HITL (Human-in-the-Loop) Configuration
 
-Configure user confirmation for high-risk tool operations:
-
-```json
-{
-  "hitl": {
-    "enabled": true,
-    "confirmationMode": "all",
-    "bypass": {
-      "tools": ["fs.write_file"],
-      "categories": ["filesystem"],
-      "levels": ["medium"]
-    }
-  }
-}
+Configure user confirmation for high-risk tool operations via `Toolpack.init()`:
+
+```typescript
+const sdk = await Toolpack.init({
+  provider: 'anthropic',
+  hitl: {
+    enabled: true,
+    confirmationMode: 'all',
+    bypass: {
+      tools: ['fs.write_file'],
+      categories: ['filesystem'],
+      levels: ['medium'],
+    },
+  },
+});
 ```
 
 | Option | Type | Default | Description |
 |--------|------|---------|-------------|
-| `enabled` | boolean | `true` | Master switch for HITL confirmation |
+| `enabled` | boolean | `false` | Enable HITL. Auto-enabled when `onToolConfirm` is provided |
 | `confirmationMode` | string | `"all"` | `"off"`, `"high-only"`, or `"all"` |
 | `bypass.tools` | string[] | `[]` | Tool names to bypass (e.g., `["fs.write_file"]`) |
 | `bypass.categories` | string[] | `[]` | Categories to bypass (e.g., `["filesystem"]`) |
 | `bypass.levels` | string[] | `[]` | Risk levels to bypass (`["high"]` or `["medium"]`) |
 
-**Programmatic API:**
-
-```typescript
-import { addBypassRule, removeBypassRule } from 'toolpack-sdk';
-
-// Add bypass rule
-await addBypassRule({ type: 'tool', value: 'fs.delete_file' });
-
-// Remove bypass rule
-await removeBypassRule({ type: 'tool', value: 'fs.delete_file' });
-
-// Reload config to apply changes
-toolpack.reloadConfig();
-```
-
 See the [HITL documentation](https://toolpacksdk.com/guides/hitl-confirmation) for detailed configuration options and best practices.
 
 #### Web Search Providers
@@ -1232,17 +1208,19 @@ The `web.search` tool supports multiple search backends with automatic fallback:
 
 When you have many tools (50+), enable tool search to reduce token usage. The AI discovers tools on-demand via a built-in `tool.search` meta-tool using BM25 ranking:
 
-```json
-{
-  "tools": {
-    "toolSearch": {
-      "enabled": true,
-      "alwaysLoadedTools": ["fs.read_file", "fs.write_file", "web.search"],
-      "searchResultLimit": 5,
-      "cacheDiscoveredTools": true
-    }
-  }
-}
+```typescript
+const sdk = await Toolpack.init({
+  provider: 'anthropic',
+  tools: true,
+  toolsConfig: {
+    toolSearch: {
+      enabled: true,
+      alwaysLoadedTools: ['fs.read_file', 'fs.write_file', 'web.search'],
+      searchResultLimit: 5,
+      cacheDiscoveredTools: true,
+    },
+  },
+});
 ```
 
 ## API Reference
@@ -1252,6 +1230,12 @@ When you have many tools (50+), enable tool search to reduce token usage. The AI
 ```typescript
 import { Toolpack } from 'toolpack-sdk';
 
+// ToolpackInitConfig — fields removed in v3.0.0:
+//   customTools   → use ModeConfig.customTools per agent instead
+//   modeOverrides → configure modes directly via registerMode()
+//   configPath    → configuration is now passed inline to Toolpack.init()
+//
+// Fields: logging, hitl, toolsConfig, toolOverrides (v3.1.0)
 const sdk = await Toolpack.init(config: ToolpackInitConfig): Promise<Toolpack>
 
 // Completions (routes through workflow engine if mode has workflow enabled)
@@ -1271,6 +1255,12 @@ sdk.getModes(): ModeConfig[]
 sdk.cycleMode(): ModeConfig
 sdk.registerMode(mode: ModeConfig): void
 
+// Tool projects
+await sdk.loadToolProject(project: ToolProject): Promise<void>
+await sdk.loadToolProjects(projects: ToolProject[]): Promise<void>
+sdk.searchTools(query: string, category?: string)
+sdk.getRegisteredToolNames(): string[]
+
 // Internal access
 sdk.getClient(): AIClient
 sdk.getWorkflowExecutor(): WorkflowExecutor