@supertools-ai/core 0.1.0

package/README.md

@@ -0,0 +1,379 @@
+# Supertools
+
+<p align="center">
+  <img src="assets/banner.svg" alt="Supertools - Let LLMs write code that calls your tools" width="100%">
+</p>
+
+> **🚧 Work in Progress** — This project is under active development. The npm package will be published soon. Contributions are welcome, especially for adding support for other AI providers (OpenAI, Vercel AI SDK, etc.)!
+
+<p align="center">
+  <a href="#quick-start">Quick Start</a> •
+  <a href="#how-it-works">How It Works</a> •
+  <a href="#api">API</a> •
+  <a href="#architecture">Architecture</a> •
+  <a href="#roadmap">Roadmap</a>
+</p>
+
+---
+
+> Inspired by [Anthropic's Programmatic Tool Calling](https://platform.claude.com/docs/en/agents-and-tools/tool-use/programmatic-tool-calling) — the LLM writes code that orchestrates tools, instead of calling them one by one.
+
+## The Problem
+
+Traditional tool calling has limitations:
+
+- **Loops require enumeration** — querying 50 states means 50 explicit tool calls
+- **Results stay in context** — all tool outputs consume tokens on every round-trip
+- **Processing needs LLM** — filtering, aggregating, or transforming data requires another LLM call
+
+## The Solution
+
+Supertools lets the LLM write code that runs in a sandbox:
+
+```
+User Request → LLM generates code → Sandbox executes → Result
+                                         │
+                              for (state of states) {
+                                await query_db(state)
+                              }
+                              // Process locally
+                              return topResults
+```
+
+- **Loops are native** — the LLM writes a `for` loop, not 50 tool calls
+- **Processing is free** — filtering/aggregation runs in sandbox, not LLM
+- **Only final result** — intermediate data never hits the LLM context
+
+## Quick Start
+
+```bash
+bun add @supertools-ai/core
+```
+
+```typescript
+import { supertools, defineTool, z } from '@supertools-ai/core';
+import { Sandbox } from 'e2b';
+import Anthropic from '@anthropic-ai/sdk';
+
+// Define your tools with Zod schemas
+const queryDatabase = defineTool({
+  name: 'queryDatabase',
+  description: 'Execute a SQL query',
+  parameters: z.object({
+    sql: z.string().describe('The SQL query to execute'),
+  }),
+  returns: z.array(z.record(z.unknown())),  // Array of row objects
+  execute: async ({ sql }) => db.query(sql),
+});
+
+const sendEmail = defineTool({
+  name: 'sendEmail',
+  description: 'Send an email',
+  parameters: z.object({
+    to: z.string(),
+    subject: z.string(),
+    body: z.string(),
+  }),
+  returns: z.object({ success: z.boolean(), messageId: z.string() }),
+  execute: async ({ to, subject, body }) => mailer.send({ to, subject, body }),
+});
+
+// Create sandbox and wrap your SDK client
+const sandbox = await Sandbox.create('supertools-bun');
+const client = supertools(new Anthropic(), {
+  tools: [queryDatabase, sendEmail],
+  sandbox,
+});
+
+// Use exactly like the normal Anthropic SDK
+const response = await client.messages.create({
+  model: 'claude-haiku-4-5-20251001',
+  max_tokens: 1024,
+  messages: [{
+    role: 'user',
+    content: 'Query sales for all 50 states, find the top 5, and email a report to the CEO'
+  }],
+});
+
+// Traditional: 2-3 LLM round-trips + tokens for all 50 results in context
+// Supertools: 1 LLM call, loop runs in sandbox, only final result returned
+```
+
+## How It Works
+
+When you ask: *"Query sales for all 50 states, find top 5, email a report"*
+
+The LLM generates JavaScript that runs in a secure sandbox:
+
+```javascript
+// All 50 queries execute without LLM round-trips
+const states = ['AL', 'AK', 'AZ', /* ... all 50 */];
+const results = {};
+
+for (const state of states) {
+  const data = await query_database({
+    sql: `SELECT SUM(revenue) FROM sales WHERE state = '${state}'`
+  });
+  results[state] = data[0].sum;
+}
+
+// Process data locally (no tokens consumed)
+const top5 = Object.entries(results)
+  .sort((a, b) => b[1] - a[1])
+  .slice(0, 5);
+
+// Format and send
+const report = top5.map(([state, rev]) => `${state}: $${rev.toLocaleString()}`).join('\n');
+await send_email({
+  to: 'ceo@company.com',
+  subject: 'Top 5 States Report',
+  body: report
+});
+
+// Return the result as JSON
+return { topStates: top5, reportSent: true };
+```
+
+**Result:** 51 tool calls execute in the sandbox with 1 LLM call. Traditional approach would need multiple round-trips with all results in context.
+
+## Why Supertools?
+
+<p align="center">
+  <img src="assets/benchmark.svg" alt="Benchmark Results" width="100%">
+</p>
+
+The benchmark compares three approaches on the same model (Claude Sonnet 4.5):
+- **Native**: Traditional tool calling with LLM round-trips
+- **Anthropic Beta**: Anthropic's `code_execution` beta feature
+- **Supertools**: Code generation with E2B sandbox execution
+
+> **Note on Anthropic Beta results:** While the `allowed_callers` feature works (tools are called from within the Python code), each tool call still requires a full API round-trip. For N tool calls, you need N+1 API requests - the code execution pauses, returns to your server, you provide the result, and it continues. The only savings are that tool results don't inflate Claude's context. In contrast, Supertools makes 1 API call total - the generated code runs in the sandbox and calls tools via WebSocket without additional API round-trips. This explains the significant performance difference.
+
+> **Note:** Supertools returns raw JSON data, not natural language. The LLM generates code but never sees the execution results. This is ideal for data pipelines and batch operations, but for chatbots needing conversational responses, consider traditional tool calling or add a summarization step.
+
+## API
+
+### `supertools(client, config)`
+
+Wrap any supported LLM SDK client with programmatic tool calling.
+
+```typescript
+import { supertools, defineTool, z } from '@supertools-ai/core';
+import { Sandbox } from 'e2b';
+import Anthropic from '@anthropic-ai/sdk';
+
+const sandbox = await Sandbox.create('supertools-bun');
+const client = supertools(new Anthropic(), {
+  // Required
+  tools: [defineTool({ name, description, parameters, execute })],
+  sandbox,  // E2B sandbox instance
+
+  // Optional
+  debug: false,        // Enable debug logging
+  instructions: '...', // Additional instructions for the LLM
+  onEvent: (event) => {
+    // Available event types:
+    // - 'code_generated': LLM generated the code
+    // - 'sandbox_ready': Sandbox connection established
+    // - 'tool_call': Tool invoked (includes tool name and args)
+    // - 'tool_result': Tool completed (includes result and durationMs)
+    // - 'tool_error': Tool execution failed
+    // - 'result': Final execution result
+    // - 'stdout': Standard output from sandbox
+    // - 'stderr': Standard error from sandbox
+    // - 'complete': Execution finished (success or error)
+    if (event.type === 'tool_call') console.log(`Calling ${event.tool}...`);
+    if (event.type === 'tool_result') console.log(`${event.tool} done in ${event.durationMs}ms`);
+    if (event.type === 'result') console.log('Result:', event.data);
+  },
+});
+
+// Use exactly like the original SDK
+const response = await client.messages.create({
+  model: 'claude-haiku-4-5-20251001',
+  max_tokens: 1024,
+  messages: [{ role: 'user', content: 'Your request here' }],
+});
+```
+
+**Supported SDKs:**
+- ✅ Anthropic SDK (`@anthropic-ai/sdk`)
+- ⏳ OpenAI SDK — [contributions welcome](https://github.com/anthropics/supertools)
+- ⏳ Vercel AI SDK — [contributions welcome](https://github.com/anthropics/supertools)
+- ⏳ Mastra AI — [contributions welcome](https://github.com/anthropics/supertools)
+
+### `defineTool(config)`
+
+```typescript
+const tool = defineTool({
+  name: 'searchUsers',           // Must match /^[a-zA-Z][a-zA-Z0-9_]*$/
+  description: 'Search users',   // Used in LLM prompt (min 5 chars)
+  parameters: z.object({         // Zod schema for inputs
+    query: z.string(),
+    limit: z.number().optional().default(10),
+  }),
+  returns: z.array(UserSchema),  // Optional: Zod schema for return type (improves LLM accuracy)
+  execute: async (params) => {   // Your implementation
+    return db.users.search(params);
+  },
+});
+// Note: Tool names are converted to snake_case in sandbox code
+// e.g., 'searchUsers' becomes 'search_users' when called
+
+// Local tools run entirely in the sandbox (no network round-trip)
+// Use for pure computation when all data is already available
+const calculateStats = defineTool({
+  name: 'calculateStats',
+  description: 'Calculate statistics for numbers',
+  parameters: z.object({ values: z.array(z.number()) }),
+  returns: z.object({ mean: z.number(), sum: z.number() }),
+  local: true,  // Runs in sandbox, not on host
+  execute: async ({ values }) => ({
+    mean: values.reduce((a, b) => a + b, 0) / values.length,
+    sum: values.reduce((a, b) => a + b, 0),
+  }),
+});
+```
+
+### Advanced: Low-level Executor
+
+For more control, use the executor directly:
+
+```typescript
+import { createExecutor, defineTool } from '@supertools-ai/core';
+import { Sandbox } from 'e2b';
+
+// Create your own LLM adapter
+const myAdapter = {
+  async generateCode(request: string, systemPrompt: string) {
+    // Call your LLM
+    return { code: '...', rawResponse: '...' };
+  },
+};
+
+const sandbox = await Sandbox.create('supertools-bun');
+const executor = createExecutor({
+  llm: myAdapter,
+  tools: [/* your tools */],
+  sandbox,
+});
+
+const result = await executor.run('Your natural language request');
+console.log(result.code);           // Generated JavaScript
+console.log(result.result.output);  // stdout from execution
+```
+
+## Architecture
+
+```
+┌───────────────────────────────────────────────────────────────────┐
+│                         Your Application                          │
+│                                                                   │
+│  const client = supertools(new Anthropic(), { tools, sandbox });  │
+│  const response = await client.messages.create({...});            │
+└─────────────────────────────────┬─────────────────────────────────┘
+                                  │
+                                  ▼
+                   ┌────────────────────────────┐
+                   │     Supertools Wrapper     │
+                   │   (intercepts SDK calls)   │
+                   └──────────────┬─────────────┘
+                                  │ generates JavaScript
+                                  ▼
+┌───────────────────────────────────────────────────────────────────┐
+│                        E2B Cloud Sandbox                          │
+│  ┌─────────────────────────────────────────────────────────────┐  │
+│  │                       Generated Code                        │  │
+│  │                                                             │  │
+│  │   for (const r of regions) {                                │  │
+│  │     const data = await query_db({ region: r });             │  │
+│  │     results.push(data);                                     │  │
+│  │   }                                                         │  │
+│  │   await send_email({ to: 'ceo', body: summary });           │  │
+│  │   return { regions: results, emailSent: true };             │  │
+│  │                                                             │  │
+│  └────────────────────────────┬────────────────────────────────┘  │
+│                               │ tool calls via WebSocket          │
+│  ┌────────────────────────────▼────────────────────────────────┐  │
+│  │                    Relay Server (Bun)                       │  │
+│  │                  WebSocket bridge to host                   │  │
+│  └────────────────────────────┬────────────────────────────────┘  │
+└───────────────────────────────┼───────────────────────────────────┘
+                                │ WebSocket (authenticated)
+                                ▼
+                   ┌────────────────────────────┐
+                   │        Relay Client        │
+                   │    (runs on your host)     │
+                   └──────────────┬─────────────┘
+                                  │
+                                  ▼
+                   ┌────────────────────────────┐
+                   │         Your Tools         │
+                   │    query_db, send_email    │
+                   │      (execute locally)     │
+                   └────────────────────────────┘
+```
+
+**How it works:**
+
+1. You wrap your SDK client with `supertools()`
+2. When you call `client.messages.create()`, supertools intercepts it
+3. The LLM generates JavaScript code that calls your tools
+4. Code runs in an isolated E2B sandbox (secure, no host access)
+5. Tool calls are relayed back to your machine via WebSocket
+6. Your tools execute locally with full access to your systems
+7. Results flow back to the sandbox, code continues executing
+8. Final output returns in the expected SDK response format
+
+> **Note:** The Relay Server runs inside the pre-built `supertools-bun` E2B template. The Relay Client is included in the `@supertools-ai/core` package and runs on your host.
+
+**Security:**
+- LLM-generated code runs in isolated cloud containers
+- Your tools run locally — the sandbox never has direct access
+- WebSocket authenticated with cryptographically secure tokens
+- Tokens are single-use and expire with the sandbox
+
+## When to Use
+
+**Use Supertools when:**
+- Calling 3+ tools in sequence
+- Processing data (filter/aggregate before returning)
+- Parallel operations (query 50 endpoints at once)
+- Complex logic (loops, conditionals, early exit)
+
+**Use traditional tool calling when:**
+- Single tool calls
+- User needs to approve each step
+- Tools have dangerous side effects
+
+## Roadmap
+
+**Coming Soon:**
+- [x] Publish npm package (`@supertools-ai/core`)
+- [x] Publish E2B sandbox template for zero-config setup
+
+**Providers:**
+- [x] Anthropic SDK
+- [ ] OpenAI SDK
+- [ ] Vercel AI SDK
+- [ ] Mastra AI
+
+**Future:**
+- [ ] Alternative sandbox providers (??)
+- [ ] Python SDK (1:1 API parity)
+
+## Requirements
+
+- Node.js 18+ or [Bun](https://bun.sh)
+- [E2B](https://e2b.dev) API key (set `E2B_API_KEY` env var)
+- [Anthropic](https://anthropic.com) API key (set `ANTHROPIC_API_KEY` env var)
+
+## License
+
+MIT
+
+---
+
+<p align="center">
+  Secure sandboxing powered by <a href="https://e2b.dev">E2B</a>
+</p>

package/dist/__tests__/protocol.test.d.ts

@@ -0,0 +1,5 @@
+/**
+ * Protocol Validation Tests
+ */
+export {};
+//# sourceMappingURL=protocol.test.d.ts.map

package/dist/__tests__/protocol.test.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"protocol.test.d.ts","sourceRoot":"","sources":["../../src/__tests__/protocol.test.ts"],"names":[],"mappings":"AAAA;;GAEG"}

package/dist/__tests__/security.test.d.ts

@@ -0,0 +1,5 @@
+/**
+ * Security Module Tests
+ */
+export {};
+//# sourceMappingURL=security.test.d.ts.map

package/dist/__tests__/security.test.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"security.test.d.ts","sourceRoot":"","sources":["../../src/__tests__/security.test.ts"],"names":[],"mappings":"AAAA;;GAEG"}

package/dist/__tests__/tool.test.d.ts

@@ -0,0 +1,5 @@
+/**
+ * Tool Module Tests
+ */
+export {};
+//# sourceMappingURL=tool.test.d.ts.map

package/dist/__tests__/tool.test.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"tool.test.d.ts","sourceRoot":"","sources":["../../src/__tests__/tool.test.ts"],"names":[],"mappings":"AAAA;;GAEG"}

package/dist/errors.d.ts

@@ -0,0 +1,64 @@
+export type ErrorCode = 'CODE_GENERATION_ERROR' | 'EXECUTION_ERROR' | 'TOOL_ERROR' | 'RELAY_CONNECTION_ERROR' | 'RELAY_TIMEOUT_ERROR' | 'PROTOCOL_ERROR' | 'SANDBOX_ERROR' | 'AUTHENTICATION_ERROR' | 'VALIDATION_ERROR' | 'CONFIGURATION_ERROR';
+export declare class OPTError extends Error {
+    readonly code: ErrorCode;
+    readonly cause?: Error;
+    constructor(message: string, code: ErrorCode, cause?: Error);
+    toJSON(): {
+        name: string;
+        code: ErrorCode;
+        message: string;
+        cause: string | undefined;
+    };
+}
+export declare class CodeGenerationError extends OPTError {
+    constructor(message: string, cause?: Error);
+}
+export declare class ExecutionError extends OPTError {
+    readonly output?: string;
+    constructor(message: string, output?: string, cause?: Error);
+    toJSON(): {
+        output: string | undefined;
+        name: string;
+        code: ErrorCode;
+        message: string;
+        cause: string | undefined;
+    };
+}
+export declare class ToolError extends OPTError {
+    readonly toolName: string;
+    constructor(message: string, toolName: string, cause?: Error);
+    toJSON(): {
+        toolName: string;
+        name: string;
+        code: ErrorCode;
+        message: string;
+        cause: string | undefined;
+    };
+}
+export declare class RelayConnectionError extends OPTError {
+    constructor(message: string, cause?: Error);
+}
+export declare class RelayTimeoutError extends OPTError {
+    constructor(message: string);
+}
+export declare class SandboxError extends OPTError {
+    constructor(message: string, cause?: Error);
+}
+export declare class AuthenticationError extends OPTError {
+    constructor(message: string);
+}
+export declare class ConfigurationError extends OPTError {
+    constructor(message: string);
+}
+export declare class ValidationError extends OPTError {
+    readonly field?: string;
+    constructor(message: string, field?: string);
+    toJSON(): {
+        field: string | undefined;
+        name: string;
+        code: ErrorCode;
+        message: string;
+        cause: string | undefined;
+    };
+}
+//# sourceMappingURL=errors.d.ts.map

package/dist/errors.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"errors.d.ts","sourceRoot":"","sources":["../src/errors.ts"],"names":[],"mappings":"AAAA,MAAM,MAAM,SAAS,GACjB,uBAAuB,GACvB,iBAAiB,GACjB,YAAY,GACZ,wBAAwB,GACxB,qBAAqB,GACrB,gBAAgB,GAChB,eAAe,GACf,sBAAsB,GACtB,kBAAkB,GAClB,qBAAqB,CAAC;AAE1B,qBAAa,QAAS,SAAQ,KAAK;IACjC,QAAQ,CAAC,IAAI,EAAE,SAAS,CAAC;IACzB,QAAQ,CAAC,KAAK,CAAC,EAAE,KAAK,CAAC;gBAEX,OAAO,EAAE,MAAM,EAAE,IAAI,EAAE,SAAS,EAAE,KAAK,CAAC,EAAE,KAAK;IAQ3D,MAAM;;;;;;CAQP;AAED,qBAAa,mBAAoB,SAAQ,QAAQ;gBACnC,OAAO,EAAE,MAAM,EAAE,KAAK,CAAC,EAAE,KAAK;CAI3C;AAED,qBAAa,cAAe,SAAQ,QAAQ;IAC1C,QAAQ,CAAC,MAAM,CAAC,EAAE,MAAM,CAAC;gBAEb,OAAO,EAAE,MAAM,EAAE,MAAM,CAAC,EAAE,MAAM,EAAE,KAAK,CAAC,EAAE,KAAK;IAMlD,MAAM;;;;;;;CAGhB;AAED,qBAAa,SAAU,SAAQ,QAAQ;IACrC,QAAQ,CAAC,QAAQ,EAAE,MAAM,CAAC;gBAEd,OAAO,EAAE,MAAM,EAAE,QAAQ,EAAE,MAAM,EAAE,KAAK,CAAC,EAAE,KAAK;IAMnD,MAAM;;;;;;;CAGhB;AAED,qBAAa,oBAAqB,SAAQ,QAAQ;gBACpC,OAAO,EAAE,MAAM,EAAE,KAAK,CAAC,EAAE,KAAK;CAI3C;AAED,qBAAa,iBAAkB,SAAQ,QAAQ;gBACjC,OAAO,EAAE,MAAM;CAI5B;AAED,qBAAa,YAAa,SAAQ,QAAQ;gBAC5B,OAAO,EAAE,MAAM,EAAE,KAAK,CAAC,EAAE,KAAK;CAI3C;AAED,qBAAa,mBAAoB,SAAQ,QAAQ;gBACnC,OAAO,EAAE,MAAM;CAI5B;AAED,qBAAa,kBAAmB,SAAQ,QAAQ;gBAClC,OAAO,EAAE,MAAM;CAI5B;AAED,qBAAa,eAAgB,SAAQ,QAAQ;IAC3C,QAAQ,CAAC,KAAK,CAAC,EAAE,MAAM,CAAC;gBAEZ,OAAO,EAAE,MAAM,EAAE,KAAK,CAAC,EAAE,MAAM;IAMlC,MAAM;;;;;;;CAGhB"}

package/dist/executor.d.ts

@@ -0,0 +1,38 @@
+/**
+ * Programmatic Executor
+ *
+ * Orchestrates the complete flow:
+ * 1. LLM generates JavaScript code
+ * 2. Code executes in secure E2B Bun sandbox
+ * 3. Tool calls relay back to host via WebSocket
+ * 4. Results return to user
+ */
+import type { NormalizedTool } from './tool';
+import type { ExecutorConfig, ExecutionResult, ProgrammaticResult, LLMAdapter } from './types';
+export interface CreateExecutorOptions extends ExecutorConfig {
+    readonly llm: LLMAdapter;
+}
+export declare function createExecutor(options: CreateExecutorOptions): ProgrammaticExecutor;
+export declare class ProgrammaticExecutor {
+    private readonly originalTools;
+    private readonly normalizedTools;
+    private readonly toolsMap;
+    private readonly llm;
+    private readonly sandbox;
+    private readonly instructions?;
+    private readonly debug;
+    private readonly onEvent?;
+    constructor(options: CreateExecutorOptions);
+    private emit;
+    run(userRequest: string): Promise<ProgrammaticResult>;
+    executeCode(code: string): Promise<ExecutionResult>;
+    private executeCodeWithRelay;
+    getTools(): readonly NormalizedTool[];
+    getToolDocumentation(): string;
+    private generateCode;
+    private connectRelay;
+    private cleanup;
+    private log;
+    private logTiming;
+}
+//# sourceMappingURL=executor.d.ts.map

package/dist/executor.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"executor.d.ts","sourceRoot":"","sources":["../src/executor.ts"],"names":[],"mappings":"AAAA;;;;;;;;GAQG;AASH,OAAO,KAAK,EAAE,cAAc,EAAW,MAAM,QAAQ,CAAC;AAEtD,OAAO,KAAK,EAAE,cAAc,EAAE,eAAe,EAAE,kBAAkB,EAAE,UAAU,EAAkB,MAAM,SAAS,CAAC;AAE/G,MAAM,WAAW,qBAAsB,SAAQ,cAAc;IAC3D,QAAQ,CAAC,GAAG,EAAE,UAAU,CAAC;CAC1B;AAED,wBAAgB,cAAc,CAAC,OAAO,EAAE,qBAAqB,GAAG,oBAAoB,CAEnF;AAED,qBAAa,oBAAoB;IAC/B,OAAO,CAAC,QAAQ,CAAC,aAAa,CAAqB;IACnD,OAAO,CAAC,QAAQ,CAAC,eAAe,CAAmB;IACnD,OAAO,CAAC,QAAQ,CAAC,QAAQ,CAA8B;IACvD,OAAO,CAAC,QAAQ,CAAC,GAAG,CAAa;IACjC,OAAO,CAAC,QAAQ,CAAC,OAAO,CAAU;IAClC,OAAO,CAAC,QAAQ,CAAC,YAAY,CAAC,CAAS;IACvC,OAAO,CAAC,QAAQ,CAAC,KAAK,CAAU;IAChC,OAAO,CAAC,QAAQ,CAAC,OAAO,CAAC,CAAkC;gBAE/C,OAAO,EAAE,qBAAqB;IAW1C,OAAO,CAAC,IAAI;IAIN,GAAG,CAAC,WAAW,EAAE,MAAM,GAAG,OAAO,CAAC,kBAAkB,CAAC;IA6CrD,WAAW,CAAC,IAAI,EAAE,MAAM,GAAG,OAAO,CAAC,eAAe,CAAC;YAW3C,oBAAoB;IA8ClC,QAAQ,IAAI,SAAS,cAAc,EAAE;IAIrC,oBAAoB,IAAI,MAAM;YAIhB,YAAY;YAWZ,YAAY;YAsBZ,OAAO;IAcrB,OAAO,CAAC,GAAG;IAIX,OAAO,CAAC,SAAS;CAMlB"}