@startsimpli/llm 0.4.11 → 0.4.13

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md ADDED
@@ -0,0 +1,178 @@
1
+ # @startsimpli/llm
2
+
3
+ Provider-agnostic LLM client for the StartSimpli Next.js apps. Wraps **OpenAI** (GPT-5.x, GPT-4o family, fine-tuned `ft:*` models) and **Anthropic** (Claude 4 + Claude 3.x) behind a single `LLMService<TOutput>` that does:
4
+
5
+ 1. provider selection with preferred + fallback,
6
+ 2. raw text generation **or** Zod-validated structured output,
7
+ 3. JSON extraction from messy LLM responses (direct → markdown code block → brace boundary),
8
+ 4. automatic retry with **parse-error correction** and **schema-error correction** prompts when the model returns invalid output,
9
+ 5. typed `LLMProviderError` classification (`RATE_LIMITED`, `CONTEXT_LENGTH_EXCEEDED`, `TIMEOUT`, etc.) with a helper to map onto HTTP status codes for Next API routes.
10
+
11
+ Streaming is **not** supported — every call is a single non-streaming request. Bring-your-own batching (see [Batching](#batching) below).
12
+
13
+ A `MockLLMProvider` ships in-box for testing.
14
+
15
+ ## Install
16
+
17
+ Workspace dep:
18
+
19
+ ```jsonc
20
+ "dependencies": {
21
+ "@startsimpli/llm": "workspace:*"
22
+ }
23
+ ```
24
+
25
+ Only runtime dep is `zod ^4` (for schema validation). HTTP is native `fetch` — no `@anthropic-ai/sdk` or `openai` SDK is pulled in, which keeps the package edge-runtime friendly.
26
+
27
+ > **Current consumer status.** As of this writing, **no app in the monorepo consumes `@startsimpli/llm`** at runtime — it is transpiled by `market-simpli/next.config.ts` but not imported. `raise-simpli/web-app/src/integrations/` still ships an app-local OpenAI provider that duplicates this package. Migrating that app onto `@startsimpli/llm` is open work; until then, treat the package's own test suite as the canonical usage reference.
28
+
29
+ ## Public surface
30
+
31
+ | Export | Type | Description |
32
+ | --- | --- | --- |
33
+ | `LLMService<TOutput>` | class | Top-level service: provider selection + Zod-validated generation + retries |
34
+ | `GenerationResult<TOutput>` / `GenerationMetadata` | types | Discriminated `success`/`error` union returned by `.generate()` |
35
+ | `LLMServiceConfig` | type | `{ maxRetries?, preferredProvider?, defaultModel?, defaultTemperature?, defaultMaxTokens? }` |
36
+ | `ILLMProvider` | interface | Contract every provider implements — `generate`, `isAvailable`, optional `estimateCost` |
37
+ | `OpenAIProvider` / `createOpenAIProvider()` | class + factory | OpenAI implementation; factory returns `null` if `OPENAI_API_KEY` is unset |
38
+ | `AnthropicProvider` / `createAnthropicProvider()` | class + factory | Anthropic implementation; factory returns `null` if `ANTHROPIC_API_KEY` is unset |
39
+ | `MockLLMProvider` / `setMockBehavior` / `resetMockBehavior` / `MockBehavior` | class + helpers | Deterministic test provider — control delay, errors, custom responses, invalid-JSON simulation |
40
+ | `LLMProviderError` / `LLMErrorCode` | class + union | Typed error with `code`, `retryable`, `statusCode`, `provider` |
41
+ | `LLMResponse` / `TokenUsage` / `GenerateOptions` / `ProviderConfig` / `JsonExtractionResult` / `ValidationResult<T>` | types | Provider/response types |
42
+ | `LLM_DEFAULTS` | const | `{ openaiModel: 'gpt-4o-mini', anthropicModel: 'claude-sonnet-4-20250514', temperature: 0.7, maxTokens: 8192, timeoutMs: 60000 }` |
43
+ | `extractJson(content)` | function | Robust JSON extractor (direct → ```` ```json ```` block → brace boundary) |
44
+ | `llmErrorToHttpStatus(code)` | function | Maps `LLMErrorCode` → 400/422/429/500/503/504 |
45
+ | `buildSchemaErrorCorrectionPrompt` / `buildParseErrorCorrectionPrompt` / `buildGenericRetryPrompt` | functions | Domain-agnostic retry prompt builders |
46
+
47
+ ### Models recognised for cost estimation
48
+
49
+ | Provider | Models with cost table baked in |
50
+ | --- | --- |
51
+ | OpenAI | `gpt-5.2`, `gpt-5.1`, `gpt-5`, `gpt-4o`, `gpt-4o-mini`. Base `gpt-5*` are auto-forced to `temperature=1.0`; fine-tuned `ft:*` clones use the configured temperature. |
52
+ | Anthropic | `claude-opus-4-6`, `claude-opus-4-20250514`, `claude-sonnet-4-6`, `claude-sonnet-4-20250514`, `claude-3-5-sonnet-20241022`, `claude-3-5-haiku-20241022`, `claude-3-opus-20240229`, `claude-3-sonnet-20240229`, `claude-3-haiku-20240307`. |
53
+
54
+ ## Configuration
55
+
56
+ The factories read from `process.env` and return `null` when nothing is configured, so a missing key silently drops a provider from the pool.
57
+
58
+ | Variable | Used by | Default |
59
+ | --- | --- | --- |
60
+ | `OPENAI_API_KEY` | `createOpenAIProvider()` | unset → provider disabled |
61
+ | `OPENAI_BASE_URL` | `createOpenAIProvider()` | `https://api.openai.com/v1` |
62
+ | `OPENAI_MODEL` | `createOpenAIProvider()` | `gpt-4o-mini` (`LLM_DEFAULTS.openaiModel`) |
63
+ | `ANTHROPIC_API_KEY` | `createAnthropicProvider()` | unset → provider disabled |
64
+ | `ANTHROPIC_BASE_URL` | `createAnthropicProvider()` | `https://api.anthropic.com/v1` |
65
+ | `ANTHROPIC_MODEL` | `createAnthropicProvider()` | `claude-sonnet-4-20250514` (`LLM_DEFAULTS.anthropicModel`) |
66
+ | `LLM_PROVIDER` | `LLMService` constructor | When set to `mock`, only the `MockLLMProvider` is registered |
67
+
68
+ You can also bypass env wiring entirely by `new OpenAIProvider({ apiKey, … })` + `service.registerProvider('openai', …)`.
69
+
70
+ ### Batching
71
+
72
+ > **Pack N items into one prompt with echoed ids — never loop `client.create()` per item.**
73
+
74
+ `LLMService` does **one HTTP round-trip per `.generate()` call** (plus up to `maxRetries` correction attempts on parse/schema failure). It does **not** transparently batch — calling `.generate()` in a loop will issue N requests.
75
+
76
+ Canonical pattern: instead of `items.map(item => service.generate(item, …))`, build one prompt that contains the full set with stable ids, and a Zod schema for the bulk envelope:
77
+
78
+ ```ts
79
+ const Bulk = z.object({
80
+ results: z.array(z.object({
81
+ id: z.string(), // echo back the input id
82
+ score: z.number(),
83
+ label: z.string(),
84
+ })),
85
+ })
86
+
87
+ const userPrompt = `Score each item. Echo the id verbatim.\n\n` +
88
+ items.map(i => `[id=${i.id}] ${i.text}`).join('\n')
89
+
90
+ const result = await service.generate(userPrompt, systemPrompt, Bulk)
91
+ if (result.success) {
92
+ for (const row of result.data.results) {
93
+ // match row.id back to the input
94
+ }
95
+ }
96
+ ```
97
+
98
+ This collapses N calls into 1, which is usually 5–50× cheaper in latency and dollars and avoids per-call rate-limit headwind.
99
+
100
+ ## Usage
101
+
102
+ Structured generation with a Zod schema (canonical pattern, lifted from the package's own test suite `src/__tests__/llm-service.test.ts`):
103
+
104
+ ```ts
105
+ import { z } from 'zod'
106
+ import { LLMService } from '@startsimpli/llm'
107
+
108
+ const Pattern = z.object({
109
+ name: z.string(),
110
+ value: z.number(),
111
+ })
112
+
113
+ const service = new LLMService<z.infer<typeof Pattern>>({
114
+ preferredProvider: 'anthropic',
115
+ maxRetries: 2,
116
+ })
117
+
118
+ const result = await service.generate(
119
+ 'Generate a sample object',
120
+ 'You return JSON matching the requested schema.',
121
+ Pattern,
122
+ )
123
+
124
+ if (result.success) {
125
+ console.log(result.data, result.metadata) // { name, value }, { model, promptTokens, … }
126
+ } else {
127
+ console.error(result.error.code, result.error.retryable)
128
+ }
129
+ ```
130
+
131
+ Raw chat (no schema):
132
+
133
+ ```ts
134
+ const response = await service.chat(userPrompt, systemPrompt, { model: 'gpt-4o-mini' })
135
+ console.log(response.content, response.usage)
136
+ ```
137
+
138
+ Mapping errors to HTTP in a Next API route:
139
+
140
+ ```ts
141
+ import { llmErrorToHttpStatus, LLMProviderError } from '@startsimpli/llm'
142
+
143
+ try {
144
+ // ...
145
+ } catch (e) {
146
+ if (e instanceof LLMProviderError) {
147
+ return new Response(e.message, { status: llmErrorToHttpStatus(e.code) })
148
+ }
149
+ throw e
150
+ }
151
+ ```
152
+
153
+ Testing with the mock provider:
154
+
155
+ ```ts
156
+ import { LLMService, MockLLMProvider, setMockBehavior } from '@startsimpli/llm'
157
+
158
+ const mock = new MockLLMProvider()
159
+ const service = new LLMService({ preferredProvider: 'mock' })
160
+ service.registerProvider('mock', mock)
161
+
162
+ setMockBehavior({ customResponse: JSON.stringify({ name: 'Alice', value: 42 }) })
163
+ const result = await service.generate('prompt', 'system', Pattern)
164
+ ```
165
+
166
+ ## Verification
167
+
168
+ ```bash
169
+ cd packages/llm
170
+ pnpm vitest run
171
+ pnpm tsc --noEmit
172
+ ```
173
+
174
+ Test suite ships **32 tests across 4 files**: `json-extraction.test.ts`, `mock-provider.test.ts`, `retry.test.ts`, and `llm-service.test.ts`. They exercise the JSON extraction strategies, the mock provider's behavior matrix, retry-prompt construction, and the success/failure paths through `LLMService.generate`.
175
+
176
+ ## Shared-first
177
+
178
+ Per CLAUDE.md **rule 9**, LLM client wiring never belongs in app `src/`. If you find a `lib/openai.ts` or an `LLMService` clone inside an app (today: `raise-simpli/web-app/src/integrations/providers/openai/`), the right move is to delete it and import from `@startsimpli/llm`. Domain prompts and Zod schemas stay in the app; provider abstraction, retries, JSON extraction, and cost accounting stay here.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@startsimpli/llm",
3
- "version": "0.4.11",
3
+ "version": "0.4.13",
4
4
  "description": "Generic LLM service infrastructure — provider abstraction, retry logic, JSON extraction",
5
5
  "main": "./src/index.ts",
6
6
  "types": "./src/index.ts",
@@ -14,14 +14,6 @@
14
14
  "publishConfig": {
15
15
  "access": "public"
16
16
  },
17
- "scripts": {
18
- "build": "tsup",
19
- "dev": "tsup --watch",
20
- "type-check": "tsc --noEmit",
21
- "test": "vitest run",
22
- "test:watch": "vitest",
23
- "clean": "rm -rf dist"
24
- },
25
17
  "dependencies": {
26
18
  "zod": "^4.3.6"
27
19
  },
@@ -30,5 +22,13 @@
30
22
  "tsup": "^8.5.1",
31
23
  "typescript": "^6.0.3",
32
24
  "vitest": "^4.1.5"
25
+ },
26
+ "scripts": {
27
+ "build": "tsup",
28
+ "dev": "tsup --watch",
29
+ "type-check": "tsc --noEmit",
30
+ "test": "vitest run",
31
+ "test:watch": "vitest",
32
+ "clean": "rm -rf dist"
33
33
  }
34
- }
34
+ }
@@ -0,0 +1,113 @@
1
+ /**
2
+ * Tool-using agent harness (startsim-2nlo.1). Uses a queued fake provider so the
3
+ * multi-step tool-use loop is deterministic — no real LLM calls.
4
+ */
5
+ import { describe, it, expect, vi } from 'vitest';
6
+ import { runAgentTurn, buildToolRegistry } from '../agent/agent';
7
+ import type { ToolProvider } from '../agent/types';
8
+ import type { ILLMProvider } from '../providers/interface';
9
+ import type { LLMResponse } from '../types';
10
+
11
+ /** Returns the queued responses in order; repeats the last once exhausted. */
12
+ class QueuedProvider implements ILLMProvider {
13
+ readonly name = 'queued';
14
+ readonly availableModels = ['q'] as const;
15
+ readonly defaultModel = 'q';
16
+ private i = 0;
17
+ constructor(private readonly responses: string[]) {}
18
+ async generate(): Promise<LLMResponse> {
19
+ const content = this.responses[Math.min(this.i, this.responses.length - 1)] ?? '';
20
+ this.i += 1;
21
+ return { content, usage: { promptTokens: 0, completionTokens: 0, totalTokens: 0 }, model: 'q' };
22
+ }
23
+ isAvailable() {
24
+ return true;
25
+ }
26
+ }
27
+
28
+ const msgs = (text: string) => [{ role: 'user' as const, content: text }];
29
+
30
+ describe('runAgentTurn', () => {
31
+ it('returns plain text with no tool calls when the model does not call a tool', async () => {
32
+ const provider = new QueuedProvider(['Here are some ideas for you.']);
33
+ const res = await runAgentTurn({ provider, toolProviders: [], messages: msgs('hi') });
34
+ expect(res.finalText).toBe('Here are some ideas for you.');
35
+ expect(res.toolCalls).toHaveLength(0);
36
+ expect(res.stoppedReason).toBe('final');
37
+ expect(res.steps).toBe(1);
38
+ });
39
+
40
+ it('dispatches a tool call to the owning provider, then returns the final reply', async () => {
41
+ const handler = vi.fn(async (input: { types: string[] }) => ({ saved: input.types.length }));
42
+ const provider = new QueuedProvider([
43
+ JSON.stringify({ tool: 'save_schema', input: { types: ['Creator', 'Campaign'] } }),
44
+ 'Saved your 2 types.',
45
+ ]);
46
+ const foundry: ToolProvider = {
47
+ name: 'foundry',
48
+ tools: [{ name: 'save_schema', description: 'persist a schema', handler }],
49
+ };
50
+ const res = await runAgentTurn({
51
+ provider,
52
+ toolProviders: [foundry],
53
+ messages: msgs('track creators and campaigns'),
54
+ context: { tenant: 'marketing-agents' },
55
+ });
56
+ expect(handler).toHaveBeenCalledWith({ types: ['Creator', 'Campaign'] }, { tenant: 'marketing-agents' });
57
+ expect(res.toolCalls).toEqual([
58
+ { tool: 'save_schema', input: { types: ['Creator', 'Campaign'] }, output: { saved: 2 } },
59
+ ]);
60
+ expect(res.finalText).toBe('Saved your 2 types.');
61
+ expect(res.steps).toBe(2);
62
+ });
63
+
64
+ it('records an error (and recovers) when the model calls an unknown tool', async () => {
65
+ const provider = new QueuedProvider([
66
+ JSON.stringify({ tool: 'nope', input: {} }),
67
+ 'Sorry, let me try differently.',
68
+ ]);
69
+ const res = await runAgentTurn({ provider, toolProviders: [], messages: msgs('hi') });
70
+ // No tools registered => the JSON is treated as plain text (no protocol advertised).
71
+ expect(res.toolCalls).toHaveLength(0);
72
+
73
+ const withTool: ToolProvider = {
74
+ name: 'x',
75
+ tools: [{ name: 'real', description: 'd', handler: async () => 'ok' }],
76
+ };
77
+ const provider2 = new QueuedProvider([
78
+ JSON.stringify({ tool: 'ghost', input: {} }),
79
+ 'recovered',
80
+ ]);
81
+ const res2 = await runAgentTurn({ provider: provider2, toolProviders: [withTool], messages: msgs('hi') });
82
+ expect(res2.toolCalls).toEqual([{ tool: 'ghost', input: {}, error: "unknown tool 'ghost'" }]);
83
+ expect(res2.finalText).toBe('recovered');
84
+ });
85
+
86
+ it('captures a handler exception as a tool error and keeps going', async () => {
87
+ const provider = new QueuedProvider([
88
+ JSON.stringify({ tool: 'boom', input: {} }),
89
+ 'handled the failure',
90
+ ]);
91
+ const p: ToolProvider = {
92
+ name: 'p',
93
+ tools: [{ name: 'boom', description: 'd', handler: async () => { throw new Error('db down'); } }],
94
+ };
95
+ const res = await runAgentTurn({ provider, toolProviders: [p], messages: msgs('go') });
96
+ expect(res.toolCalls[0].error).toBe('db down');
97
+ expect(res.finalText).toBe('handled the failure');
98
+ });
99
+
100
+ it('stops at maxSteps when the model keeps calling tools', async () => {
101
+ const provider = new QueuedProvider([JSON.stringify({ tool: 'loop', input: {} })]); // always a tool call
102
+ const p: ToolProvider = { name: 'p', tools: [{ name: 'loop', description: 'd', handler: async () => 'again' }] };
103
+ const res = await runAgentTurn({ provider, toolProviders: [p], messages: msgs('go'), maxSteps: 3 });
104
+ expect(res.stoppedReason).toBe('max_steps');
105
+ expect(res.toolCalls).toHaveLength(3);
106
+ });
107
+
108
+ it('rejects duplicate tool names across providers', () => {
109
+ const a: ToolProvider = { name: 'a', tools: [{ name: 'dup', description: 'd', handler: () => 1 }] };
110
+ const b: ToolProvider = { name: 'b', tools: [{ name: 'dup', description: 'd', handler: () => 2 }] };
111
+ expect(() => buildToolRegistry([a, b])).toThrow(/duplicate tool name 'dup'/);
112
+ });
113
+ });
@@ -0,0 +1,130 @@
1
+ /**
2
+ * Generic tool-using agent loop (startsim-2nlo.1).
3
+ *
4
+ * runAgentTurn() drives an LLM through a tool-use loop: it advertises the
5
+ * registered tools to the model, parses a tool call, dispatches it to the owning
6
+ * provider's handler, feeds the result back, and repeats until the model replies
7
+ * in plain text (or the step budget is hit). Built on the existing string-based
8
+ * ILLMProvider via a JSON tool-call protocol, so it works with every provider
9
+ * (incl. MockLLMProvider in tests) and can later swap to native tool-use without
10
+ * changing the ToolProvider API.
11
+ */
12
+ import { extractJson } from '../utils/json-extraction';
13
+ import type {
14
+ AgentMessage,
15
+ AgentToolCall,
16
+ AgentTurnResult,
17
+ RunAgentOptions,
18
+ ToolDefinition,
19
+ ToolProvider,
20
+ } from './types';
21
+
22
+ /** Flatten providers into a name->tool map; duplicate tool names are a bug. */
23
+ export function buildToolRegistry(providers: ToolProvider[]): Map<string, ToolDefinition> {
24
+ const registry = new Map<string, ToolDefinition>();
25
+ for (const provider of providers) {
26
+ for (const tool of provider.tools) {
27
+ if (registry.has(tool.name)) {
28
+ throw new Error(
29
+ `duplicate tool name '${tool.name}' (provider '${provider.name}')`,
30
+ );
31
+ }
32
+ registry.set(tool.name, tool);
33
+ }
34
+ }
35
+ return registry;
36
+ }
37
+
38
+ function toolProtocolPrompt(registry: Map<string, ToolDefinition>): string {
39
+ if (registry.size === 0) return '';
40
+ const lines = [...registry.values()].map((t) => {
41
+ const schema = t.inputSchema ? ` input schema: ${JSON.stringify(t.inputSchema)}` : '';
42
+ return `- ${t.name}: ${t.description}.${schema}`;
43
+ });
44
+ return [
45
+ 'You have tools available:',
46
+ ...lines,
47
+ '',
48
+ 'To call a tool, reply with ONLY a JSON object of the form '
49
+ + '{"tool":"<name>","input":{...}} and nothing else. Call one tool at a '
50
+ + 'time; you will receive its result and may then call another tool. When '
51
+ + 'you are finished and ready to answer the user, reply in plain text (NOT JSON).',
52
+ ].join('\n');
53
+ }
54
+
55
+ function renderTranscript(messages: AgentMessage[]): string {
56
+ return messages
57
+ .map((m) => {
58
+ if (m.role === 'tool') return `TOOL_RESULT[${m.toolName ?? 'unknown'}]: ${m.content}`;
59
+ return `${m.role.toUpperCase()}: ${m.content}`;
60
+ })
61
+ .join('\n\n');
62
+ }
63
+
64
+ interface ParsedToolCall {
65
+ tool: string;
66
+ input: unknown;
67
+ }
68
+
69
+ /** A response is a tool call iff it parses to JSON with a string `tool`. */
70
+ function parseToolCall(content: string): ParsedToolCall | null {
71
+ const res = extractJson(content);
72
+ if (!res.success || !res.json || typeof res.json !== 'object') return null;
73
+ const obj = res.json as Record<string, unknown>;
74
+ if (typeof obj.tool !== 'string') return null;
75
+ return { tool: obj.tool, input: 'input' in obj ? obj.input : {} };
76
+ }
77
+
78
+ export async function runAgentTurn(opts: RunAgentOptions): Promise<AgentTurnResult> {
79
+ const { provider, toolProviders, context = {}, maxSteps = 6, generateOptions } = opts;
80
+ const registry = buildToolRegistry(toolProviders);
81
+ const protocol = toolProtocolPrompt(registry);
82
+ const baseSystem = [opts.systemPrompt?.trim(), protocol].filter(Boolean).join('\n\n');
83
+
84
+ const messages: AgentMessage[] = [...opts.messages];
85
+ const toolCalls: AgentToolCall[] = [];
86
+
87
+ for (let step = 1; step <= maxSteps; step++) {
88
+ const resp = await provider.generate(renderTranscript(messages), baseSystem, generateOptions);
89
+ const content = resp.content ?? '';
90
+ const call = registry.size > 0 ? parseToolCall(content) : null;
91
+
92
+ if (!call) {
93
+ messages.push({ role: 'assistant', content });
94
+ return { finalText: content, toolCalls, messages, steps: step, stoppedReason: 'final' };
95
+ }
96
+
97
+ // Record the model's tool-call message, then run the tool and feed the result back.
98
+ messages.push({ role: 'assistant', content });
99
+ const record: AgentToolCall = { tool: call.tool, input: call.input };
100
+ const tool = registry.get(call.tool);
101
+ if (!tool) {
102
+ record.error = `unknown tool '${call.tool}'`;
103
+ messages.push({ role: 'tool', toolName: call.tool, content: `ERROR: ${record.error}` });
104
+ } else {
105
+ try {
106
+ const output = await tool.handler(call.input, context);
107
+ record.output = output;
108
+ messages.push({
109
+ role: 'tool',
110
+ toolName: call.tool,
111
+ content: JSON.stringify(output ?? null),
112
+ });
113
+ } catch (err) {
114
+ record.error = err instanceof Error ? err.message : String(err);
115
+ messages.push({ role: 'tool', toolName: call.tool, content: `ERROR: ${record.error}` });
116
+ }
117
+ }
118
+ toolCalls.push(record);
119
+ }
120
+
121
+ // Step budget exhausted — force one final plain-text reply.
122
+ const resp = await provider.generate(
123
+ renderTranscript(messages),
124
+ `${baseSystem}\n\nYou have reached the tool-call limit. Reply to the user now in plain text.`,
125
+ generateOptions,
126
+ );
127
+ const finalText = resp.content ?? '';
128
+ messages.push({ role: 'assistant', content: finalText });
129
+ return { finalText, toolCalls, messages, steps: maxSteps, stoppedReason: 'max_steps' };
130
+ }
@@ -0,0 +1,80 @@
1
+ /**
2
+ * Generic agent harness types (startsim-2nlo.1).
3
+ *
4
+ * The harness runs an LLM tool-use loop over a set of pluggable TOOL PROVIDERS.
5
+ * Each app registers its own provider(s) declaring the tools it supports — their
6
+ * input/output data and a handler — so NO app-specific logic lives in the
7
+ * harness. The foundry schema-builder, recipe/crochet save+export, etc. are all
8
+ * just providers. The app-facing API here is deliberately stable so the internal
9
+ * transport (today: a JSON tool-call protocol over the string provider) can later
10
+ * swap to native Anthropic tool-use without changing how apps register tools.
11
+ */
12
+ import type { GenerateOptions } from '../types';
13
+ import type { ILLMProvider } from '../providers/interface';
14
+
15
+ /** Opaque, app-provided context handed to every tool handler (e.g. the calling
16
+ * user, the target foundry/tenant, a db client). The harness never inspects it. */
17
+ export type ToolContext = Record<string, unknown>;
18
+
19
+ /** A single tool an app exposes to the agent. */
20
+ export interface ToolDefinition<I = any, O = any> {
21
+ /** Unique tool name the model calls (e.g. 'save_schema'). */
22
+ name: string;
23
+ /** What the tool does + when to use it — the model reads this. */
24
+ description: string;
25
+ /** JSON-schema-ish description of `input` (also drives native tool-use later). */
26
+ inputSchema?: Record<string, unknown>;
27
+ /** Executes the tool. Return value is serialized back to the model. */
28
+ handler: (input: I, ctx: ToolContext) => Promise<O> | O;
29
+ }
30
+
31
+ /** A named bundle of tools an app contributes. */
32
+ export interface ToolProvider {
33
+ /** Namespace label for diagnostics (e.g. 'foundry'). */
34
+ name: string;
35
+ tools: ToolDefinition[];
36
+ }
37
+
38
+ export type AgentRole = 'user' | 'assistant' | 'tool';
39
+
40
+ export interface AgentMessage {
41
+ role: AgentRole;
42
+ content: string;
43
+ /** Set when role === 'tool': which tool produced this result. */
44
+ toolName?: string;
45
+ }
46
+
47
+ /** Record of one tool invocation during a turn. */
48
+ export interface AgentToolCall {
49
+ tool: string;
50
+ input: unknown;
51
+ output?: unknown;
52
+ error?: string;
53
+ }
54
+
55
+ export interface AgentTurnResult {
56
+ /** The model's final natural-language reply to the user. */
57
+ finalText: string;
58
+ /** Every tool call made during the turn, in order. */
59
+ toolCalls: AgentToolCall[];
60
+ /** Full transcript including tool-call + tool-result messages. */
61
+ messages: AgentMessage[];
62
+ /** Number of model round-trips used. */
63
+ steps: number;
64
+ stoppedReason: 'final' | 'max_steps';
65
+ }
66
+
67
+ export interface RunAgentOptions {
68
+ provider: ILLMProvider;
69
+ /** Tool providers contributed by the calling app. */
70
+ toolProviders: ToolProvider[];
71
+ /** Conversation so far, ending with the latest user message. */
72
+ messages: AgentMessage[];
73
+ /** App/domain system prompt (the tool protocol is appended automatically). */
74
+ systemPrompt?: string;
75
+ /** Passed verbatim to every tool handler. */
76
+ context?: ToolContext;
77
+ /** Max model round-trips before forcing a final reply (default 6). */
78
+ maxSteps?: number;
79
+ generateOptions?: GenerateOptions;
80
+ }
package/src/index.ts CHANGED
@@ -35,3 +35,17 @@ export {
35
35
  // Service
36
36
  export { LLMService } from './service';
37
37
  export type { GenerationResult, GenerationMetadata } from './service';
38
+
39
+ // Generic tool-using agent harness (startsim-2nlo.1): apps register ToolProviders;
40
+ // the harness runs the LLM tool-use loop. No app-specific logic lives here.
41
+ export { runAgentTurn, buildToolRegistry } from './agent/agent';
42
+ export type {
43
+ ToolContext,
44
+ ToolDefinition,
45
+ ToolProvider,
46
+ AgentRole,
47
+ AgentMessage,
48
+ AgentToolCall,
49
+ AgentTurnResult,
50
+ RunAgentOptions,
51
+ } from './agent/types';