@startsimpli/llm 0.4.11 → 0.4.14

package/README.md

@@ -0,0 +1,178 @@
+# @startsimpli/llm
+
+Provider-agnostic LLM client for the StartSimpli Next.js apps. Wraps **OpenAI** (GPT-5.x, GPT-4o family, fine-tuned `ft:*` models) and **Anthropic** (Claude 4 + Claude 3.x) behind a single `LLMService<TOutput>` that does:
+
+1. provider selection with preferred + fallback,
+2. raw text generation **or** Zod-validated structured output,
+3. JSON extraction from messy LLM responses (direct → markdown code block → brace boundary),
+4. automatic retry with **parse-error correction** and **schema-error correction** prompts when the model returns invalid output,
+5. typed `LLMProviderError` classification (`RATE_LIMITED`, `CONTEXT_LENGTH_EXCEEDED`, `TIMEOUT`, etc.) with a helper to map onto HTTP status codes for Next API routes.
+
+Streaming is **not** supported — every call is a single non-streaming request. Bring-your-own batching (see [Batching](#batching) below).
+
+A `MockLLMProvider` ships in-box for testing.
+
+## Install
+
+Workspace dep:
+
+```jsonc
+"dependencies": {
+  "@startsimpli/llm": "workspace:*"
+}
+```
+
+Only runtime dep is `zod ^4` (for schema validation). HTTP is native `fetch` — no `@anthropic-ai/sdk` or `openai` SDK is pulled in, which keeps the package edge-runtime friendly.
+
+> **Current consumer status.** As of this writing, **no app in the monorepo consumes `@startsimpli/llm`** at runtime — it is transpiled by `market-simpli/next.config.ts` but not imported. `raise-simpli/web-app/src/integrations/` still ships an app-local OpenAI provider that duplicates this package. Migrating that app onto `@startsimpli/llm` is open work; until then, treat the package's own test suite as the canonical usage reference.
+
+## Public surface
+
+| Export | Type | Description |
+| --- | --- | --- |
+| `LLMService<TOutput>` | class | Top-level service: provider selection + Zod-validated generation + retries |
+| `GenerationResult<TOutput>` / `GenerationMetadata` | types | Discriminated `success`/`error` union returned by `.generate()` |
+| `LLMServiceConfig` | type | `{ maxRetries?, preferredProvider?, defaultModel?, defaultTemperature?, defaultMaxTokens? }` |
+| `ILLMProvider` | interface | Contract every provider implements — `generate`, `isAvailable`, optional `estimateCost` |
+| `OpenAIProvider` / `createOpenAIProvider()` | class + factory | OpenAI implementation; factory returns `null` if `OPENAI_API_KEY` is unset |
+| `AnthropicProvider` / `createAnthropicProvider()` | class + factory | Anthropic implementation; factory returns `null` if `ANTHROPIC_API_KEY` is unset |
+| `MockLLMProvider` / `setMockBehavior` / `resetMockBehavior` / `MockBehavior` | class + helpers | Deterministic test provider — control delay, errors, custom responses, invalid-JSON simulation |
+| `LLMProviderError` / `LLMErrorCode` | class + union | Typed error with `code`, `retryable`, `statusCode`, `provider` |
+| `LLMResponse` / `TokenUsage` / `GenerateOptions` / `ProviderConfig` / `JsonExtractionResult` / `ValidationResult<T>` | types | Provider/response types |
+| `LLM_DEFAULTS` | const | `{ openaiModel: 'gpt-4o-mini', anthropicModel: 'claude-sonnet-4-20250514', temperature: 0.7, maxTokens: 8192, timeoutMs: 60000 }` |
+| `extractJson(content)` | function | Robust JSON extractor (direct → ```` ```json ```` block → brace boundary) |
+| `llmErrorToHttpStatus(code)` | function | Maps `LLMErrorCode` → 400/422/429/500/503/504 |
+| `buildSchemaErrorCorrectionPrompt` / `buildParseErrorCorrectionPrompt` / `buildGenericRetryPrompt` | functions | Domain-agnostic retry prompt builders |
+
+### Models recognised for cost estimation
+
+| Provider | Models with cost table baked in |
+| --- | --- |
+| OpenAI | `gpt-5.2`, `gpt-5.1`, `gpt-5`, `gpt-4o`, `gpt-4o-mini`. Base `gpt-5*` are auto-forced to `temperature=1.0`; fine-tuned `ft:*` clones use the configured temperature. |
+| Anthropic | `claude-opus-4-6`, `claude-opus-4-20250514`, `claude-sonnet-4-6`, `claude-sonnet-4-20250514`, `claude-3-5-sonnet-20241022`, `claude-3-5-haiku-20241022`, `claude-3-opus-20240229`, `claude-3-sonnet-20240229`, `claude-3-haiku-20240307`. |
+
+## Configuration
+
+The factories read from `process.env` and return `null` when nothing is configured, so a missing key silently drops a provider from the pool.
+
+| Variable | Used by | Default |
+| --- | --- | --- |
+| `OPENAI_API_KEY` | `createOpenAIProvider()` | unset → provider disabled |
+| `OPENAI_BASE_URL` | `createOpenAIProvider()` | `https://api.openai.com/v1` |
+| `OPENAI_MODEL` | `createOpenAIProvider()` | `gpt-4o-mini` (`LLM_DEFAULTS.openaiModel`) |
+| `ANTHROPIC_API_KEY` | `createAnthropicProvider()` | unset → provider disabled |
+| `ANTHROPIC_BASE_URL` | `createAnthropicProvider()` | `https://api.anthropic.com/v1` |
+| `ANTHROPIC_MODEL` | `createAnthropicProvider()` | `claude-sonnet-4-20250514` (`LLM_DEFAULTS.anthropicModel`) |
+| `LLM_PROVIDER` | `LLMService` constructor | When set to `mock`, only the `MockLLMProvider` is registered |
+
+You can also bypass env wiring entirely by `new OpenAIProvider({ apiKey, … })` + `service.registerProvider('openai', …)`.
+
+### Batching
+
+> **Pack N items into one prompt with echoed ids — never loop `client.create()` per item.**
+
+`LLMService` does **one HTTP round-trip per `.generate()` call** (plus up to `maxRetries` correction attempts on parse/schema failure). It does **not** transparently batch — calling `.generate()` in a loop will issue N requests.
+
+Canonical pattern: instead of `items.map(item => service.generate(item, …))`, build one prompt that contains the full set with stable ids, and a Zod schema for the bulk envelope:
+
+```ts
+const Bulk = z.object({
+  results: z.array(z.object({
+    id: z.string(),         // echo back the input id
+    score: z.number(),
+    label: z.string(),
+  })),
+})
+
+const userPrompt = `Score each item. Echo the id verbatim.\n\n` +
+  items.map(i => `[id=${i.id}] ${i.text}`).join('\n')
+
+const result = await service.generate(userPrompt, systemPrompt, Bulk)
+if (result.success) {
+  for (const row of result.data.results) {
+    // match row.id back to the input
+  }
+}
+```
+
+This collapses N calls into 1, which is usually 5–50× cheaper in latency and dollars and avoids per-call rate-limit headwind.
+
+## Usage
+
+Structured generation with a Zod schema (canonical pattern, lifted from the package's own test suite `src/__tests__/llm-service.test.ts`):
+
+```ts
+import { z } from 'zod'
+import { LLMService } from '@startsimpli/llm'
+
+const Pattern = z.object({
+  name: z.string(),
+  value: z.number(),
+})
+
+const service = new LLMService<z.infer<typeof Pattern>>({
+  preferredProvider: 'anthropic',
+  maxRetries: 2,
+})
+
+const result = await service.generate(
+  'Generate a sample object',
+  'You return JSON matching the requested schema.',
+  Pattern,
+)
+
+if (result.success) {
+  console.log(result.data, result.metadata) // { name, value }, { model, promptTokens, … }
+} else {
+  console.error(result.error.code, result.error.retryable)
+}
+```
+
+Raw chat (no schema):
+
+```ts
+const response = await service.chat(userPrompt, systemPrompt, { model: 'gpt-4o-mini' })
+console.log(response.content, response.usage)
+```
+
+Mapping errors to HTTP in a Next API route:
+
+```ts
+import { llmErrorToHttpStatus, LLMProviderError } from '@startsimpli/llm'
+
+try {
+  // ...
+} catch (e) {
+  if (e instanceof LLMProviderError) {
+    return new Response(e.message, { status: llmErrorToHttpStatus(e.code) })
+  }
+  throw e
+}
+```
+
+Testing with the mock provider:
+
+```ts
+import { LLMService, MockLLMProvider, setMockBehavior } from '@startsimpli/llm'
+
+const mock = new MockLLMProvider()
+const service = new LLMService({ preferredProvider: 'mock' })
+service.registerProvider('mock', mock)
+
+setMockBehavior({ customResponse: JSON.stringify({ name: 'Alice', value: 42 }) })
+const result = await service.generate('prompt', 'system', Pattern)
+```
+
+## Verification
+
+```bash
+cd packages/llm
+pnpm vitest run
+pnpm tsc --noEmit
+```
+
+Test suite ships **32 tests across 4 files**: `json-extraction.test.ts`, `mock-provider.test.ts`, `retry.test.ts`, and `llm-service.test.ts`. They exercise the JSON extraction strategies, the mock provider's behavior matrix, retry-prompt construction, and the success/failure paths through `LLMService.generate`.
+
+## Shared-first
+
+Per CLAUDE.md **rule 9**, LLM client wiring never belongs in app `src/`. If you find a `lib/openai.ts` or an `LLMService` clone inside an app (today: `raise-simpli/web-app/src/integrations/providers/openai/`), the right move is to delete it and import from `@startsimpli/llm`. Domain prompts and Zod schemas stay in the app; provider abstraction, retries, JSON extraction, and cost accounting stay here.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@startsimpli/llm",
-  "version": "0.4.11",
+  "version": "0.4.14",
   "description": "Generic LLM service infrastructure — provider abstraction, retry logic, JSON extraction",
   "main": "./src/index.ts",
   "types": "./src/index.ts",
@@ -14,14 +14,6 @@
   "publishConfig": {
     "access": "public"
   },
-  "scripts": {
-    "build": "tsup",
-    "dev": "tsup --watch",
-    "type-check": "tsc --noEmit",
-    "test": "vitest run",
-    "test:watch": "vitest",
-    "clean": "rm -rf dist"
-  },
   "dependencies": {
     "zod": "^4.3.6"
   },
@@ -30,5 +22,13 @@
     "tsup": "^8.5.1",
     "typescript": "^6.0.3",
     "vitest": "^4.1.5"
+  },
+  "scripts": {
+    "build": "tsup",
+    "dev": "tsup --watch",
+    "type-check": "tsc --noEmit",
+    "test": "vitest run",
+    "test:watch": "vitest",
+    "clean": "rm -rf dist"
   }
-}
+}

package/src/__tests__/agent.test.ts

@@ -0,0 +1,186 @@
+/**
+ * Tool-using agent harness (startsim-2nlo.1). Uses a queued fake provider so the
+ * multi-step tool-use loop is deterministic — no real LLM calls.
+ */
+import { describe, it, expect, vi } from 'vitest';
+import { runAgentTurn, buildToolRegistry } from '../agent/agent';
+import type { ToolProvider } from '../agent/types';
+import type { ILLMProvider } from '../providers/interface';
+import type { LLMResponse, NativeToolResponse } from '../types';
+
+/** Returns the queued responses in order; repeats the last once exhausted. */
+class QueuedProvider implements ILLMProvider {
+  readonly name = 'queued';
+  readonly availableModels = ['q'] as const;
+  readonly defaultModel = 'q';
+  private i = 0;
+  constructor(private readonly responses: string[]) {}
+  async generate(): Promise<LLMResponse> {
+    const content = this.responses[Math.min(this.i, this.responses.length - 1)] ?? '';
+    this.i += 1;
+    return { content, usage: { promptTokens: 0, completionTokens: 0, totalTokens: 0 }, model: 'q' };
+  }
+  isAvailable() {
+    return true;
+  }
+}
+
+const msgs = (text: string) => [{ role: 'user' as const, content: text }];
+
+describe('runAgentTurn', () => {
+  it('returns plain text with no tool calls when the model does not call a tool', async () => {
+    const provider = new QueuedProvider(['Here are some ideas for you.']);
+    const res = await runAgentTurn({ provider, toolProviders: [], messages: msgs('hi') });
+    expect(res.finalText).toBe('Here are some ideas for you.');
+    expect(res.toolCalls).toHaveLength(0);
+    expect(res.stoppedReason).toBe('final');
+    expect(res.steps).toBe(1);
+  });
+
+  it('dispatches a tool call to the owning provider, then returns the final reply', async () => {
+    const handler = vi.fn(async (input: { types: string[] }) => ({ saved: input.types.length }));
+    const provider = new QueuedProvider([
+      JSON.stringify({ tool: 'save_schema', input: { types: ['Creator', 'Campaign'] } }),
+      'Saved your 2 types.',
+    ]);
+    const foundry: ToolProvider = {
+      name: 'foundry',
+      tools: [{ name: 'save_schema', description: 'persist a schema', handler }],
+    };
+    const res = await runAgentTurn({
+      provider,
+      toolProviders: [foundry],
+      messages: msgs('track creators and campaigns'),
+      context: { tenant: 'marketing-agents' },
+    });
+    expect(handler).toHaveBeenCalledWith({ types: ['Creator', 'Campaign'] }, { tenant: 'marketing-agents' });
+    expect(res.toolCalls).toEqual([
+      { tool: 'save_schema', input: { types: ['Creator', 'Campaign'] }, output: { saved: 2 } },
+    ]);
+    expect(res.finalText).toBe('Saved your 2 types.');
+    expect(res.steps).toBe(2);
+  });
+
+  it('records an error (and recovers) when the model calls an unknown tool', async () => {
+    const provider = new QueuedProvider([
+      JSON.stringify({ tool: 'nope', input: {} }),
+      'Sorry, let me try differently.',
+    ]);
+    const res = await runAgentTurn({ provider, toolProviders: [], messages: msgs('hi') });
+    // No tools registered => the JSON is treated as plain text (no protocol advertised).
+    expect(res.toolCalls).toHaveLength(0);
+
+    const withTool: ToolProvider = {
+      name: 'x',
+      tools: [{ name: 'real', description: 'd', handler: async () => 'ok' }],
+    };
+    const provider2 = new QueuedProvider([
+      JSON.stringify({ tool: 'ghost', input: {} }),
+      'recovered',
+    ]);
+    const res2 = await runAgentTurn({ provider: provider2, toolProviders: [withTool], messages: msgs('hi') });
+    expect(res2.toolCalls).toEqual([{ tool: 'ghost', input: {}, error: "unknown tool 'ghost'" }]);
+    expect(res2.finalText).toBe('recovered');
+  });
+
+  it('captures a handler exception as a tool error and keeps going', async () => {
+    const provider = new QueuedProvider([
+      JSON.stringify({ tool: 'boom', input: {} }),
+      'handled the failure',
+    ]);
+    const p: ToolProvider = {
+      name: 'p',
+      tools: [{ name: 'boom', description: 'd', handler: async () => { throw new Error('db down'); } }],
+    };
+    const res = await runAgentTurn({ provider, toolProviders: [p], messages: msgs('go') });
+    expect(res.toolCalls[0].error).toBe('db down');
+    expect(res.finalText).toBe('handled the failure');
+  });
+
+  it('stops at maxSteps when the model keeps calling tools', async () => {
+    const provider = new QueuedProvider([JSON.stringify({ tool: 'loop', input: {} })]); // always a tool call
+    const p: ToolProvider = { name: 'p', tools: [{ name: 'loop', description: 'd', handler: async () => 'again' }] };
+    const res = await runAgentTurn({ provider, toolProviders: [p], messages: msgs('go'), maxSteps: 3 });
+    expect(res.stoppedReason).toBe('max_steps');
+    expect(res.toolCalls).toHaveLength(3);
+  });
+
+  it('rejects duplicate tool names across providers', () => {
+    const a: ToolProvider = { name: 'a', tools: [{ name: 'dup', description: 'd', handler: () => 1 }] };
+    const b: ToolProvider = { name: 'b', tools: [{ name: 'dup', description: 'd', handler: () => 2 }] };
+    expect(() => buildToolRegistry([a, b])).toThrow(/duplicate tool name 'dup'/);
+  });
+});
+
+/** Provider that supports NATIVE tool-calling: returns queued {content, toolCalls}. */
+class NativeQueuedProvider implements ILLMProvider {
+  readonly name = 'native';
+  readonly availableModels = ['n'] as const;
+  readonly defaultModel = 'n';
+  private i = 0;
+  constructor(private readonly turns: NativeToolResponse[]) {}
+  async generate(): Promise<LLMResponse> {
+    return { content: '', usage: { promptTokens: 0, completionTokens: 0, totalTokens: 0 }, model: 'n' };
+  }
+  async generateWithTools(): Promise<NativeToolResponse> {
+    const t = this.turns[Math.min(this.i, this.turns.length - 1)];
+    this.i += 1;
+    return t;
+  }
+  isAvailable() {
+    return true;
+  }
+}
+
+describe('runAgentTurn (native tool-calling)', () => {
+  it('uses the native loop, dispatches the structured call, returns plain-text final', async () => {
+    const handler = vi.fn(async (input: { types: string[] }) => ({ saved: input.types.length }));
+    const provider = new NativeQueuedProvider([
+      { content: null, toolCalls: [{ id: 'c1', name: 'save_schema', arguments: { types: ['Creator', 'Campaign'] } }] },
+      { content: 'All set — I created Creator and Campaign.', toolCalls: [] },
+    ]);
+    const foundry: ToolProvider = {
+      name: 'foundry',
+      tools: [{ name: 'save_schema', description: 'persist', inputSchema: { type: 'object' }, handler }],
+    };
+    const res = await runAgentTurn({
+      provider,
+      toolProviders: [foundry],
+      messages: msgs('track creators and campaigns'),
+      context: { tenant: 'x' },
+    });
+    expect(handler).toHaveBeenCalledWith({ types: ['Creator', 'Campaign'] }, { tenant: 'x' });
+    expect(res.toolCalls).toEqual([
+      { tool: 'save_schema', input: { types: ['Creator', 'Campaign'] }, output: { saved: 2 } },
+    ]);
+    // The final is the model's REAL text — no JSON echo, no synthesized fallback needed.
+    expect(res.finalText).toBe('All set — I created Creator and Campaign.');
+    expect(res.steps).toBe(2);
+  });
+
+  it('does NOT retry a satisfied call (unlike the forced-json string protocol)', async () => {
+    const handler = vi.fn(async () => ({ ok: true }));
+    const provider = new NativeQueuedProvider([
+      { content: null, toolCalls: [{ id: 'c1', name: 'save', arguments: {} }] },
+      { content: 'Done.', toolCalls: [] },
+    ]);
+    const p: ToolProvider = { name: 'p', tools: [{ name: 'save', description: 'd', handler }] };
+    const res = await runAgentTurn({ provider, toolProviders: [p], messages: msgs('go') });
+    expect(handler).toHaveBeenCalledTimes(1); // one clean call, not N retries
+    expect(res.finalText).toBe('Done.');
+  });
+
+  it('captures a handler error in the native loop', async () => {
+    const provider = new NativeQueuedProvider([
+      { content: null, toolCalls: [{ id: 'c1', name: 'boom', arguments: {} }] },
+      { content: 'handled', toolCalls: [] },
+    ]);
+    const p: ToolProvider = {
+      name: 'p',
+      tools: [{ name: 'boom', description: 'd', handler: async () => { throw new Error('db down'); } }],
+    };
+    const res = await runAgentTurn({ provider, toolProviders: [p], messages: msgs('go') });
+    expect(res.toolCalls[0].error).toBe('db down');
+    expect(res.finalText).toBe('handled');
+  });
+});

package/src/agent/agent.ts

@@ -0,0 +1,198 @@
+/**
+ * Generic tool-using agent loop (startsim-2nlo.1).
+ *
+ * runAgentTurn() drives an LLM through a tool-use loop: it advertises the
+ * registered tools to the model, parses a tool call, dispatches it to the owning
+ * provider's handler, feeds the result back, and repeats until the model replies
+ * in plain text (or the step budget is hit). Built on the existing string-based
+ * ILLMProvider via a JSON tool-call protocol, so it works with every provider
+ * (incl. MockLLMProvider in tests) and can later swap to native tool-use without
+ * changing the ToolProvider API.
+ */
+import { extractJson } from '../utils/json-extraction';
+import type { NativeChatMessage, NativeToolSpec } from '../types';
+import type {
+  AgentMessage,
+  AgentToolCall,
+  AgentTurnResult,
+  RunAgentOptions,
+  ToolDefinition,
+  ToolProvider,
+} from './types';
+
+/** Flatten providers into a name->tool map; duplicate tool names are a bug. */
+export function buildToolRegistry(providers: ToolProvider[]): Map<string, ToolDefinition> {
+  const registry = new Map<string, ToolDefinition>();
+  for (const provider of providers) {
+    for (const tool of provider.tools) {
+      if (registry.has(tool.name)) {
+        throw new Error(
+          `duplicate tool name '${tool.name}' (provider '${provider.name}')`,
+        );
+      }
+      registry.set(tool.name, tool);
+    }
+  }
+  return registry;
+}
+
+function toolProtocolPrompt(registry: Map<string, ToolDefinition>): string {
+  if (registry.size === 0) return '';
+  const lines = [...registry.values()].map((t) => {
+    const schema = t.inputSchema ? ` input schema: ${JSON.stringify(t.inputSchema)}` : '';
+    return `- ${t.name}: ${t.description}.${schema}`;
+  });
+  return [
+    'You have tools available:',
+    ...lines,
+    '',
+    'To call a tool, reply with ONLY a JSON object of the form '
+      + '{"tool":"<name>","input":{...}} and nothing else. Call one tool at a '
+      + 'time; you will receive its result and may then call another tool. When '
+      + 'you are finished and ready to answer the user, reply in plain text (NOT JSON).',
+  ].join('\n');
+}
+
+function renderTranscript(messages: AgentMessage[]): string {
+  return messages
+    .map((m) => {
+      if (m.role === 'tool') return `TOOL_RESULT[${m.toolName ?? 'unknown'}]: ${m.content}`;
+      return `${m.role.toUpperCase()}: ${m.content}`;
+    })
+    .join('\n\n');
+}
+
+interface ParsedToolCall {
+  tool: string;
+  input: unknown;
+}
+
+/** A response is a tool call iff it parses to JSON with a string `tool`. */
+function parseToolCall(content: string): ParsedToolCall | null {
+  const res = extractJson(content);
+  if (!res.success || !res.json || typeof res.json !== 'object') return null;
+  const obj = res.json as Record<string, unknown>;
+  if (typeof obj.tool !== 'string') return null;
+  return { tool: obj.tool, input: 'input' in obj ? obj.input : {} };
+}
+
+/** Native (structured) tool loop for providers that implement generateWithTools.
+ *  Tool calls come back structured (no JSON-in-text parsing) and finals are real
+ *  text, so there's no ambiguity and the model doesn't re-emit a satisfied call. */
+async function runNativeToolLoop(
+  opts: RunAgentOptions,
+  registry: Map<string, ToolDefinition>,
+): Promise<AgentTurnResult> {
+  const { provider, context = {}, maxSteps = 6, generateOptions } = opts;
+  const system = opts.systemPrompt?.trim() ?? '';
+  const toolSpecs: NativeToolSpec[] = [...registry.values()].map((t) => ({
+    name: t.name,
+    description: t.description,
+    parameters: t.inputSchema ?? { type: 'object', properties: {} },
+  }));
+
+  const messages: NativeChatMessage[] = opts.messages.map((m) => ({
+    role: m.role === 'tool' ? 'tool' : m.role,
+    content: m.content,
+  }));
+  const toolCalls: AgentToolCall[] = [];
+  const toAgent = (): AgentMessage[] =>
+    messages.map((m) => ({ role: m.role, content: m.content ?? '' }));
+
+  for (let step = 1; step <= maxSteps; step++) {
+    const resp = await provider.generateWithTools!(messages, system, toolSpecs, generateOptions);
+    if (!resp.toolCalls.length) {
+      messages.push({ role: 'assistant', content: resp.content ?? '' });
+      return { finalText: resp.content ?? '', toolCalls, messages: toAgent(), steps: step, stoppedReason: 'final' };
+    }
+    messages.push({ role: 'assistant', content: resp.content, toolCalls: resp.toolCalls });
+    for (const call of resp.toolCalls) {
+      const record: AgentToolCall = { tool: call.name, input: call.arguments };
+      const tool = registry.get(call.name);
+      if (!tool) {
+        record.error = `unknown tool '${call.name}'`;
+        messages.push({ role: 'tool', toolCallId: call.id, content: `ERROR: ${record.error}` });
+      } else {
+        try {
+          const output = await tool.handler(call.arguments, context);
+          record.output = output;
+          messages.push({ role: 'tool', toolCallId: call.id, content: JSON.stringify(output ?? null) });
+        } catch (err) {
+          record.error = err instanceof Error ? err.message : String(err);
+          messages.push({ role: 'tool', toolCallId: call.id, content: `ERROR: ${record.error}` });
+        }
+      }
+      toolCalls.push(record);
+    }
+  }
+
+  // Budget exhausted — ask for a plain-text wrap-up (tool_choice stays 'auto').
+  const resp = await provider.generateWithTools!(
+    messages,
+    `${system}\n\nYou have reached the tool-call limit. Reply to the user now in plain text.`,
+    toolSpecs,
+    generateOptions,
+  );
+  messages.push({ role: 'assistant', content: resp.content ?? '' });
+  return { finalText: resp.content ?? '', toolCalls, messages: toAgent(), steps: maxSteps, stoppedReason: 'max_steps' };
+}
+
+export async function runAgentTurn(opts: RunAgentOptions): Promise<AgentTurnResult> {
+  const { provider, toolProviders, context = {}, maxSteps = 6, generateOptions } = opts;
+  const registry = buildToolRegistry(toolProviders);
+
+  // Prefer native structured tool-calling when the provider supports it.
+  if (typeof provider.generateWithTools === 'function' && registry.size > 0) {
+    return runNativeToolLoop(opts, registry);
+  }
+
+  const protocol = toolProtocolPrompt(registry);
+  const baseSystem = [opts.systemPrompt?.trim(), protocol].filter(Boolean).join('\n\n');
+
+  const messages: AgentMessage[] = [...opts.messages];
+  const toolCalls: AgentToolCall[] = [];
+
+  for (let step = 1; step <= maxSteps; step++) {
+    const resp = await provider.generate(renderTranscript(messages), baseSystem, generateOptions);
+    const content = resp.content ?? '';
+    const call = registry.size > 0 ? parseToolCall(content) : null;
+
+    if (!call) {
+      messages.push({ role: 'assistant', content });
+      return { finalText: content, toolCalls, messages, steps: step, stoppedReason: 'final' };
+    }
+
+    // Record the model's tool-call message, then run the tool and feed the result back.
+    messages.push({ role: 'assistant', content });
+    const record: AgentToolCall = { tool: call.tool, input: call.input };
+    const tool = registry.get(call.tool);
+    if (!tool) {
+      record.error = `unknown tool '${call.tool}'`;
+      messages.push({ role: 'tool', toolName: call.tool, content: `ERROR: ${record.error}` });
+    } else {
+      try {
+        const output = await tool.handler(call.input, context);
+        record.output = output;
+        messages.push({
+          role: 'tool',
+          toolName: call.tool,
+          content: JSON.stringify(output ?? null),
+        });
+      } catch (err) {
+        record.error = err instanceof Error ? err.message : String(err);
+        messages.push({ role: 'tool', toolName: call.tool, content: `ERROR: ${record.error}` });
+      }
+    }
+    toolCalls.push(record);
+  }
+
+  // Step budget exhausted — force one final plain-text reply.
+  const resp = await provider.generate(
+    renderTranscript(messages),
+    `${baseSystem}\n\nYou have reached the tool-call limit. Reply to the user now in plain text.`,
+    generateOptions,
+  );
+  const finalText = resp.content ?? '';
+  messages.push({ role: 'assistant', content: finalText });
+  return { finalText, toolCalls, messages, steps: maxSteps, stoppedReason: 'max_steps' };
+}

package/src/agent/types.ts

@@ -0,0 +1,80 @@
+/**
+ * Generic agent harness types (startsim-2nlo.1).
+ *
+ * The harness runs an LLM tool-use loop over a set of pluggable TOOL PROVIDERS.
+ * Each app registers its own provider(s) declaring the tools it supports — their
+ * input/output data and a handler — so NO app-specific logic lives in the
+ * harness. The foundry schema-builder, recipe/crochet save+export, etc. are all
+ * just providers. The app-facing API here is deliberately stable so the internal
+ * transport (today: a JSON tool-call protocol over the string provider) can later
+ * swap to native Anthropic tool-use without changing how apps register tools.
+ */
+import type { GenerateOptions } from '../types';
+import type { ILLMProvider } from '../providers/interface';
+
+/** Opaque, app-provided context handed to every tool handler (e.g. the calling
+ *  user, the target foundry/tenant, a db client). The harness never inspects it. */
+export type ToolContext = Record<string, unknown>;
+
+/** A single tool an app exposes to the agent. */
+export interface ToolDefinition<I = any, O = any> {
+  /** Unique tool name the model calls (e.g. 'save_schema'). */
+  name: string;
+  /** What the tool does + when to use it — the model reads this. */
+  description: string;
+  /** JSON-schema-ish description of `input` (also drives native tool-use later). */
+  inputSchema?: Record<string, unknown>;
+  /** Executes the tool. Return value is serialized back to the model. */
+  handler: (input: I, ctx: ToolContext) => Promise<O> | O;
+}
+
+/** A named bundle of tools an app contributes. */
+export interface ToolProvider {
+  /** Namespace label for diagnostics (e.g. 'foundry'). */
+  name: string;
+  tools: ToolDefinition[];
+}
+
+export type AgentRole = 'user' | 'assistant' | 'tool';
+
+export interface AgentMessage {
+  role: AgentRole;
+  content: string;
+  /** Set when role === 'tool': which tool produced this result. */
+  toolName?: string;
+}
+
+/** Record of one tool invocation during a turn. */
+export interface AgentToolCall {
+  tool: string;
+  input: unknown;
+  output?: unknown;
+  error?: string;
+}
+
+export interface AgentTurnResult {
+  /** The model's final natural-language reply to the user. */
+  finalText: string;
+  /** Every tool call made during the turn, in order. */
+  toolCalls: AgentToolCall[];
+  /** Full transcript including tool-call + tool-result messages. */
+  messages: AgentMessage[];
+  /** Number of model round-trips used. */
+  steps: number;
+  stoppedReason: 'final' | 'max_steps';
+}
+
+export interface RunAgentOptions {
+  provider: ILLMProvider;
+  /** Tool providers contributed by the calling app. */
+  toolProviders: ToolProvider[];
+  /** Conversation so far, ending with the latest user message. */
+  messages: AgentMessage[];
+  /** App/domain system prompt (the tool protocol is appended automatically). */
+  systemPrompt?: string;
+  /** Passed verbatim to every tool handler. */
+  context?: ToolContext;
+  /** Max model round-trips before forcing a final reply (default 6). */
+  maxSteps?: number;
+  generateOptions?: GenerateOptions;
+}

package/src/index.ts

@@ -11,6 +11,10 @@ export type {
   JsonExtractionResult,
   ValidationResult,
   LLMServiceConfig,
+  NativeToolSpec,
+  NativeToolCall,
+  NativeChatMessage,
+  NativeToolResponse,
 } from './types';
 export { LLM_DEFAULTS } from './types';
 
@@ -35,3 +39,17 @@ export {
 // Service
 export { LLMService } from './service';
 export type { GenerationResult, GenerationMetadata } from './service';
+
+// Generic tool-using agent harness (startsim-2nlo.1): apps register ToolProviders;
+// the harness runs the LLM tool-use loop. No app-specific logic lives here.
+export { runAgentTurn, buildToolRegistry } from './agent/agent';
+export type {
+  ToolContext,
+  ToolDefinition,
+  ToolProvider,
+  AgentRole,
+  AgentMessage,
+  AgentToolCall,
+  AgentTurnResult,
+  RunAgentOptions,
+} from './agent/types';

package/src/providers/interface.ts

@@ -1,4 +1,10 @@
-import type { LLMResponse, GenerateOptions } from '../types';
+import type {
+  LLMResponse,
+  GenerateOptions,
+  NativeToolSpec,
+  NativeChatMessage,
+  NativeToolResponse,
+} from '../types';
 
 /**
  * Contract that every LLM provider must implement.
@@ -29,6 +35,20 @@ export interface ILLMProvider {
     options?: GenerateOptions
   ): Promise<LLMResponse>;
 
+  /**
+   * Optional NATIVE (structured) tool-calling. Providers backed by a real tools
+   * API (OpenAI functions, Anthropic tool blocks) implement this so the agent
+   * harness can use structured tool calls + plain-text finals instead of the
+   * fragile JSON-in-text protocol. If omitted, the harness falls back to the
+   * string protocol — so existing providers (Mock, etc.) keep working unchanged.
+   */
+  generateWithTools?(
+    messages: NativeChatMessage[],
+    systemPrompt: string,
+    tools: NativeToolSpec[],
+    options?: GenerateOptions
+  ): Promise<NativeToolResponse>;
+
   /** Check if the provider is properly configured and available */
   isAvailable(): boolean;
 

package/src/providers/openai.ts

@@ -10,7 +10,14 @@
  */
 
 import type { ILLMProvider } from './interface';
-import type { LLMResponse, GenerateOptions, ProviderConfig } from '../types';
+import type {
+  LLMResponse,
+  GenerateOptions,
+  ProviderConfig,
+  NativeToolSpec,
+  NativeChatMessage,
+  NativeToolResponse,
+} from '../types';
 import { LLM_DEFAULTS } from '../types';
 import { LLMProviderError } from '../errors';
 import type { LLMErrorCode } from '../errors';
@@ -54,6 +61,30 @@ interface OpenAIErrorResponse {
   };
 }
 
+interface OpenAIToolCall {
+  id: string;
+  type: 'function';
+  function: { name: string; arguments: string };
+}
+
+interface OpenAIToolMessage {
+  role: 'system' | 'user' | 'assistant' | 'tool';
+  content: string | null;
+  tool_calls?: OpenAIToolCall[];
+  tool_call_id?: string;
+}
+
+interface OpenAIToolsResponse {
+  id: string;
+  model: string;
+  choices: Array<{
+    index: number;
+    message: { role: string; content: string | null; tool_calls?: OpenAIToolCall[] };
+    finish_reason: string;
+  }>;
+  usage: { prompt_tokens: number; completion_tokens: number; total_tokens: number };
+}
+
 const OPENAI_MODELS = [
   'gpt-5.2',
   'gpt-5.1',
@@ -210,6 +241,136 @@ export class OpenAIProvider implements ILLMProvider {
     }
   }
 
+  /**
+   * Native tool-calling via OpenAI's `tools` API. Unlike generate(), this does
+   * NOT force response_format=json_object — so the model returns structured
+   * tool_calls when calling a tool and plain text when it's done, which the agent
+   * harness can tell apart unambiguously (no JSON-in-text parsing).
+   */
+  async generateWithTools(
+    messages: NativeChatMessage[],
+    systemPrompt: string,
+    tools: NativeToolSpec[],
+    options?: GenerateOptions
+  ): Promise<NativeToolResponse> {
+    if (!this.isAvailable()) {
+      throw new LLMProviderError(
+        'OpenAI API key not configured',
+        'AUTHENTICATION_ERROR',
+        false,
+        undefined,
+        this.name
+      );
+    }
+
+    const model = options?.model ?? this.config.defaultModel;
+    const isBaseGpt5 = model === 'gpt-5' || model === 'gpt-5.1' || model === 'gpt-5.2';
+    const isFineTuned = model.startsWith('ft:');
+    const temperature = isBaseGpt5 && !isFineTuned
+      ? 1.0
+      : (options?.temperature ?? this.config.defaultTemperature);
+    const maxTokens = options?.maxTokens ?? this.config.defaultMaxTokens;
+    const timeout = options?.timeout ?? this.config.timeoutMs;
+
+    const oaiMessages: OpenAIToolMessage[] = [
+      { role: 'system', content: systemPrompt },
+      ...messages.map((m): OpenAIToolMessage => {
+        if (m.role === 'tool') {
+          return { role: 'tool', content: m.content ?? '', tool_call_id: m.toolCallId };
+        }
+        if (m.role === 'assistant' && m.toolCalls?.length) {
+          return {
+            role: 'assistant',
+            content: m.content,
+            tool_calls: m.toolCalls.map((tc) => ({
+              id: tc.id,
+              type: 'function',
+              function: { name: tc.name, arguments: JSON.stringify(tc.arguments ?? {}) },
+            })),
+          };
+        }
+        return { role: m.role, content: m.content ?? '' };
+      }),
+    ];
+
+    const requestBody = {
+      model,
+      messages: oaiMessages,
+      temperature,
+      max_tokens: maxTokens,
+      tools: tools.map((t) => ({
+        type: 'function' as const,
+        function: { name: t.name, description: t.description, parameters: t.parameters },
+      })),
+      tool_choice: 'auto' as const,
+    };
+
+    const controller = new AbortController();
+    const timeoutId = setTimeout(() => controller.abort(), timeout);
+    try {
+      const response = await fetch(`${this.config.baseUrl}/chat/completions`, {
+        method: 'POST',
+        headers: {
+          'Content-Type': 'application/json',
+          Authorization: `Bearer ${this.config.apiKey}`,
+        },
+        body: JSON.stringify(requestBody),
+        signal: controller.signal,
+      });
+      clearTimeout(timeoutId);
+
+      if (!response.ok) {
+        const errorData = (await response.json().catch(() => ({
+          error: { message: 'Unknown error' },
+        }))) as OpenAIErrorResponse;
+        throw this.handleApiError(response.status, errorData);
+      }
+
+      const data = (await response.json()) as OpenAIToolsResponse;
+      const choice = data.choices?.[0];
+      const msg = choice?.message;
+      const toolCalls = (msg?.tool_calls ?? []).map((tc) => {
+        let args: unknown = {};
+        try {
+          args = tc.function.arguments ? JSON.parse(tc.function.arguments) : {};
+        } catch {
+          args = { _raw: tc.function.arguments };
+        }
+        return { id: tc.id, name: tc.function.name, arguments: args };
+      });
+      return {
+        content: msg?.content ?? null,
+        toolCalls,
+        usage: {
+          promptTokens: data.usage?.prompt_tokens ?? 0,
+          completionTokens: data.usage?.completion_tokens ?? 0,
+          totalTokens: data.usage?.total_tokens ?? 0,
+        },
+        model: data.model,
+        finishReason: choice?.finish_reason,
+      };
+    } catch (error) {
+      clearTimeout(timeoutId);
+      if (error instanceof LLMProviderError) throw error;
+      if (error instanceof Error && error.name === 'AbortError') {
+        throw new LLMProviderError(
+          `Request timed out after ${timeout}ms`,
+          'TIMEOUT',
+          true,
+          undefined,
+          this.name
+        );
+      }
+      throw new LLMProviderError(
+        `OpenAI tools request failed: ${error instanceof Error ? error.message : 'Unknown error'}`,
+        'UNKNOWN_ERROR',
+        true,
+        undefined,
+        this.name
+      );
+    }
+  }
+
   estimateCost(
     promptTokens: number,
     completionTokens: number,

package/src/types.ts

@@ -41,6 +41,44 @@ export interface GenerateOptions {
   extra?: Record<string, unknown>;
 }
 
+/**
+ * Native (structured) tool-calling types. Providers that support a real tools API
+ * (OpenAI functions, Anthropic tool blocks) implement ILLMProvider.generateWithTools
+ * so the agent harness can use structured calls instead of the JSON-in-text protocol.
+ */
+export interface NativeToolSpec {
+  name: string;
+  description: string;
+  /** JSON Schema for the tool's arguments. */
+  parameters: Record<string, unknown>;
+}
+
+/** A structured tool call the model emitted (arguments already parsed from JSON). */
+export interface NativeToolCall {
+  id: string;
+  name: string;
+  arguments: unknown;
+}
+
+/** One message in a native tool-calling conversation. */
+export interface NativeChatMessage {
+  role: 'user' | 'assistant' | 'tool';
+  content: string | null;
+  /** Set on assistant turns that called tools. */
+  toolCalls?: NativeToolCall[];
+  /** Set on tool-result turns: which call this answers. */
+  toolCallId?: string;
+}
+
+/** Result of one native tool-calling round-trip. */
+export interface NativeToolResponse {
+  content: string | null;
+  toolCalls: NativeToolCall[];
+  usage?: TokenUsage;
+  model?: string;
+  finishReason?: string;
+}
+
 /**
  * Provider configuration — everything needed to construct a provider instance.
  */