llmz 0.0.18 → 0.0.20
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/DOCS.md +1836 -0
- package/dist/{chunk-M2MYALUN.cjs → chunk-BEPRLBPK.cjs} +14 -28
- package/dist/{chunk-VIMCBE6B.js → chunk-D3ESDRLH.js} +12 -26
- package/dist/errors.d.ts +2 -2
- package/dist/index.cjs +3 -3
- package/dist/index.js +3 -3
- package/dist/{llmz-HVMGCGKR.cjs → llmz-EUESEPB7.cjs} +2 -2
- package/dist/{llmz-D2R3BIFW.js → llmz-T4DEP7OD.js} +1 -1
- package/dist/prompts/chat-mode/system.md.d.ts +1 -1
- package/dist/prompts/chat-mode/user.md.d.ts +1 -1
- package/dist/prompts/worker-mode/system.md.d.ts +1 -1
- package/dist/prompts/worker-mode/user.md.d.ts +1 -1
- package/dist/{vm-6NC73N5B.cjs → vm-2DLG7V4G.cjs} +2 -2
- package/dist/{vm-DNS3KON5.js → vm-FLBMZUA2.js} +3 -3
- package/dist/vm.d.ts +1 -1
- package/package.json +3 -3
package/DOCS.md
ADDED
|
@@ -0,0 +1,1836 @@
|
|
|
1
|
+
# LLMz Documentation
|
|
2
|
+
|
|
3
|
+
**LLMz: A Revolutionary TypeScript AI Agent Framework**
|
|
4
|
+
|
|
5
|
+
_Stop chaining tools. Start generating real code._
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## Table of Contents
|
|
10
|
+
|
|
11
|
+
1. [Introduction](#introduction)
|
|
12
|
+
2. [Core Philosophy](#core-philosophy)
|
|
13
|
+
3. [Quick Start](#quick-start)
|
|
14
|
+
4. [Core Concepts](#core-concepts)
|
|
15
|
+
5. [Execution Modes](#execution-modes)
|
|
16
|
+
6. [Tools](#tools)
|
|
17
|
+
7. [Objects and Variables](#objects-and-variables)
|
|
18
|
+
8. [Execution Results](#execution-results)
|
|
19
|
+
9. [Hooks System](#hooks-system)
|
|
20
|
+
10. [Advanced Features](#advanced-features)
|
|
21
|
+
11. [API Reference](#api-reference)
|
|
22
|
+
12. [Examples](#examples)
|
|
23
|
+
|
|
24
|
+
---
|
|
25
|
+
|
|
26
|
+
## Introduction
|
|
27
|
+
|
|
28
|
+
LLMz is a revolutionary TypeScript AI agent framework that fundamentally changes how AI agents work. Like other agent frameworks, LLMz calls LLM models in a loop to achieve desired outcomes with access to tools and memory. However, LLMz is **code-first** – meaning it generates and runs TypeScript code in a sandbox rather than using traditional JSON tool calling.
|
|
29
|
+
|
|
30
|
+
### What Makes LLMz Different
|
|
31
|
+
|
|
32
|
+
Traditional agent frameworks rely on JSON tool calling, which has significant limitations:
|
|
33
|
+
|
|
34
|
+
- **Hard-to-parse JSON schemas** for LLMs
|
|
35
|
+
- **Incapable of complex logic** like loops and conditionals
|
|
36
|
+
- **Multiple expensive roundtrips** for each tool call
|
|
37
|
+
- **Unreliable beyond simple scenarios**
|
|
38
|
+
|
|
39
|
+
LLMz leverages the fact that models have been trained extensively on millions of TypeScript codebases, making them incredibly reliable at generating working code. This enables:
|
|
40
|
+
|
|
41
|
+
- **Complex logic and multi-tool orchestration** in **one call**
|
|
42
|
+
- **Native LLM thinking** via comments and code structure
|
|
43
|
+
- **Complete type safety** and predictable schemas
|
|
44
|
+
- **Seamless scaling** in production environments
|
|
45
|
+
|
|
46
|
+
### Battle-Tested at Scale
|
|
47
|
+
|
|
48
|
+
LLMz operates as an LLM-native TypeScript VM built on top of Zui (Botpress's internal schema library), battle-tested in production powering millions of AI agents worldwide.
|
|
49
|
+
|
|
50
|
+
---
|
|
51
|
+
|
|
52
|
+
## Core Philosophy
|
|
53
|
+
|
|
54
|
+
### Code Generation > Tool Calling
|
|
55
|
+
|
|
56
|
+
Traditional tool-calling agents are fundamentally limited by the JSON interface between the LLM and tools. This requires multiple roundtrips for complex tasks and cannot handle conditional logic, loops, or sophisticated data processing.
|
|
57
|
+
|
|
58
|
+
LLMz solves this by letting LLMs do what they do best: **generate code**. Since models are trained extensively on code, they can reliably generate TypeScript that:
|
|
59
|
+
|
|
60
|
+
- Calls multiple tools in sequence
|
|
61
|
+
- Handles conditional logic and error cases
|
|
62
|
+
- Processes and transforms data between tool calls
|
|
63
|
+
- Implements complex business logic
|
|
64
|
+
- Maintains type safety throughout execution
|
|
65
|
+
|
|
66
|
+
### Example: Traditional vs LLMz
|
|
67
|
+
|
|
68
|
+
**Traditional Tool Calling:**
|
|
69
|
+
|
|
70
|
+
```
|
|
71
|
+
LLM → JSON: {"tool": "getPrice", "params": {"from": "quebec", "to": "new york"}}
|
|
72
|
+
System → Response: {"price": 600}
|
|
73
|
+
LLM → JSON: {"tool": "checkBudget", "params": {"amount": 600}}
|
|
74
|
+
System → Response: {"canAfford": false}
|
|
75
|
+
LLM → JSON: {"tool": "notifyUser", "params": {"message": "Price too high"}}
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
**LLMz Code Generation:**
|
|
79
|
+
|
|
80
|
+
```typescript
|
|
81
|
+
// Check the ticket price and user's budget in one go
|
|
82
|
+
const price = await getTicketPrice({ from: 'quebec', to: 'new york' })
|
|
83
|
+
const budget = await getUserBudget()
|
|
84
|
+
|
|
85
|
+
if (price > budget) {
|
|
86
|
+
await notifyUser({ message: `Price $${price} exceeds budget $${budget}` })
|
|
87
|
+
return { action: 'budget_exceeded', price, budget }
|
|
88
|
+
} else {
|
|
89
|
+
const ticketId = await buyTicket({ from: 'quebec', to: 'new york' })
|
|
90
|
+
return { action: 'done', result: ticketId }
|
|
91
|
+
}
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
---
|
|
95
|
+
|
|
96
|
+
## Quick Start
|
|
97
|
+
|
|
98
|
+
### Installation
|
|
99
|
+
|
|
100
|
+
```bash
|
|
101
|
+
npm install llmz @botpress/client
|
|
102
|
+
```
|
|
103
|
+
|
|
104
|
+
### Basic Example (Worker Mode)
|
|
105
|
+
|
|
106
|
+
```typescript
|
|
107
|
+
import { execute } from 'llmz'
|
|
108
|
+
import { Client } from '@botpress/client'
|
|
109
|
+
|
|
110
|
+
const client = new Client({
|
|
111
|
+
botId: process.env.BOTPRESS_BOT_ID!,
|
|
112
|
+
token: process.env.BOTPRESS_TOKEN!,
|
|
113
|
+
})
|
|
114
|
+
|
|
115
|
+
const result = await execute({
|
|
116
|
+
instructions: 'Calculate the sum of numbers 1 to 100',
|
|
117
|
+
client,
|
|
118
|
+
})
|
|
119
|
+
|
|
120
|
+
if (result.isSuccess()) {
|
|
121
|
+
console.log('Result:', result.output)
|
|
122
|
+
console.log('Generated code:', result.iteration.code)
|
|
123
|
+
}
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
### Basic Example (Chat Mode)
|
|
127
|
+
|
|
128
|
+
```typescript
|
|
129
|
+
import { execute } from 'llmz'
|
|
130
|
+
import { Client } from '@botpress/client'
|
|
131
|
+
import { CLIChat } from './utils/cli-chat'
|
|
132
|
+
|
|
133
|
+
const client = new Client({
|
|
134
|
+
botId: process.env.BOTPRESS_BOT_ID!,
|
|
135
|
+
token: process.env.BOTPRESS_TOKEN!,
|
|
136
|
+
})
|
|
137
|
+
|
|
138
|
+
const chat = new CLIChat()
|
|
139
|
+
|
|
140
|
+
while (await chat.iterate()) {
|
|
141
|
+
await execute({
|
|
142
|
+
instructions: 'You are a helpful assistant',
|
|
143
|
+
chat,
|
|
144
|
+
client,
|
|
145
|
+
})
|
|
146
|
+
}
|
|
147
|
+
```
|
|
148
|
+
|
|
149
|
+
---
|
|
150
|
+
|
|
151
|
+
## Core Concepts
|
|
152
|
+
|
|
153
|
+
### Execution Loop
|
|
154
|
+
|
|
155
|
+
At its core, LLMz exposes a single method (`execute`) that runs in a loop until one of these conditions is met:
|
|
156
|
+
|
|
157
|
+
1. **An Exit is returned** - Agent completes with structured result
|
|
158
|
+
2. **Agent waits for user input** (Chat Mode) - Returns control to user
|
|
159
|
+
3. **Maximum iterations reached** - Safety limit to prevent infinite loops
|
|
160
|
+
|
|
161
|
+
The loop automatically handles:
|
|
162
|
+
|
|
163
|
+
- Tool calling and result processing
|
|
164
|
+
- Thinking about outputs and context
|
|
165
|
+
- Error recovery and retry logic
|
|
166
|
+
- Variable state persistence across iterations
|
|
167
|
+
|
|
168
|
+
### Generated Code Structure
|
|
169
|
+
|
|
170
|
+
Every LLMz code block follows a predictable structure that LLMs can reliably generate:
|
|
171
|
+
|
|
172
|
+
#### Return Statement (Required)
|
|
173
|
+
|
|
174
|
+
At minimum, an LLMz response must contain a return statement with an Exit:
|
|
175
|
+
|
|
176
|
+
```typescript
|
|
177
|
+
// Chat mode - give turn back to user
|
|
178
|
+
return { action: 'listen' }
|
|
179
|
+
|
|
180
|
+
// Worker mode - complete with result
|
|
181
|
+
return { action: 'done', result: calculatedValue }
|
|
182
|
+
```
|
|
183
|
+
|
|
184
|
+
#### Tool Calls with Logic
|
|
185
|
+
|
|
186
|
+
Unlike traditional tool calling, LLMz enables complex logic impossible with JSON:
|
|
187
|
+
|
|
188
|
+
```typescript
|
|
189
|
+
// Complex conditional logic and error handling
|
|
190
|
+
const price = await getTicketPrice({ from: 'quebec', to: 'new york' })
|
|
191
|
+
|
|
192
|
+
if (price > 500) {
|
|
193
|
+
throw new Error('Price too high')
|
|
194
|
+
} else {
|
|
195
|
+
const ticketId = await buyTicket({ from: 'quebec', to: 'new york' })
|
|
196
|
+
return { action: 'done', result: ticketId }
|
|
197
|
+
}
|
|
198
|
+
```
|
|
199
|
+
|
|
200
|
+
#### Comments for Planning
|
|
201
|
+
|
|
202
|
+
Comments help LLMs think step-by-step and plan ahead:
|
|
203
|
+
|
|
204
|
+
```typescript
|
|
205
|
+
// Check user's budget first before proceeding with purchase
|
|
206
|
+
const budget = await getUserBudget()
|
|
207
|
+
|
|
208
|
+
// Only proceed if we have enough funds
|
|
209
|
+
if (budget >= price) {
|
|
210
|
+
// Purchase the ticket
|
|
211
|
+
const ticket = await buyTicket(ticketDetails)
|
|
212
|
+
}
|
|
213
|
+
```
|
|
214
|
+
|
|
215
|
+
#### React Components (Chat Mode Only)
|
|
216
|
+
|
|
217
|
+
In Chat Mode, agents can yield React components for rich user interaction:
|
|
218
|
+
|
|
219
|
+
```typescript
|
|
220
|
+
// Multi-line text support
|
|
221
|
+
yield <Text>
|
|
222
|
+
Hello, world!
|
|
223
|
+
This is a second line.
|
|
224
|
+
</Text>
|
|
225
|
+
|
|
226
|
+
// Composed/nested components
|
|
227
|
+
yield <Message>
|
|
228
|
+
<Text>What do you prefer?</Text>
|
|
229
|
+
<Button>Cats</Button>
|
|
230
|
+
<Button>Dogs</Button>
|
|
231
|
+
</Message>
|
|
232
|
+
|
|
233
|
+
return { action: 'listen' }
|
|
234
|
+
```
|
|
235
|
+
|
|
236
|
+
### Compilation Pipeline
|
|
237
|
+
|
|
238
|
+
LLMz uses a sophisticated Babel-based compilation system to transform generated code:
|
|
239
|
+
|
|
240
|
+
1. **AST Parsing**: TypeScript/JSX code parsed into Abstract Syntax Tree
|
|
241
|
+
2. **Plugin Transformation**: Custom plugins modify the AST for execution
|
|
242
|
+
3. **Code Generation**: Modified AST compiled back to executable JavaScript
|
|
243
|
+
4. **Source Maps**: Generated for debugging and error tracking
|
|
244
|
+
|
|
245
|
+
Key transformations include:
|
|
246
|
+
|
|
247
|
+
- Tool call instrumentation for monitoring
|
|
248
|
+
- Variable extraction and tracking
|
|
249
|
+
- JSX component handling
|
|
250
|
+
- Line number preservation for stack traces
|
|
251
|
+
|
|
252
|
+
### Virtual Machine Execution
|
|
253
|
+
|
|
254
|
+
LLMz supports multiple execution environments:
|
|
255
|
+
|
|
256
|
+
- **Production**: Uses `isolated-vm` for security isolation
|
|
257
|
+
- **CI/Development**: Falls back to Node.js VM for compatibility
|
|
258
|
+
- **Browser**: Uses standard JavaScript execution
|
|
259
|
+
|
|
260
|
+
The VM provides:
|
|
261
|
+
|
|
262
|
+
- Memory isolation and limits
|
|
263
|
+
- Execution timeouts
|
|
264
|
+
- Secure context separation
|
|
265
|
+
- Stack trace sanitization
|
|
266
|
+
|
|
267
|
+
---
|
|
268
|
+
|
|
269
|
+
## Execution Modes
|
|
270
|
+
|
|
271
|
+
LLMz operates in two distinct modes depending on whether a chat interface is provided:
|
|
272
|
+
|
|
273
|
+
### Chat Mode
|
|
274
|
+
|
|
275
|
+
**Enabled when**: `chat` parameter is provided to `execute()`
|
|
276
|
+
|
|
277
|
+
Chat Mode is designed for interactive conversational agents that need to:
|
|
278
|
+
|
|
279
|
+
- Maintain conversation history
|
|
280
|
+
- Respond to user messages
|
|
281
|
+
- Yield UI components for rich interaction
|
|
282
|
+
- Handle turn-taking between agent and user
|
|
283
|
+
|
|
284
|
+
Key characteristics:
|
|
285
|
+
|
|
286
|
+
- Agent can yield React components to user
|
|
287
|
+
- Special `ListenExit` automatically available
|
|
288
|
+
- Transcript management for conversation history
|
|
289
|
+
- Turn-based execution flow
|
|
290
|
+
|
|
291
|
+
```typescript
|
|
292
|
+
const result = await execute({
|
|
293
|
+
instructions: 'You are a helpful assistant',
|
|
294
|
+
chat: myChatInstance,
|
|
295
|
+
tools: [searchTool, calculatorTool],
|
|
296
|
+
client,
|
|
297
|
+
})
|
|
298
|
+
|
|
299
|
+
if (result.is(ListenExit)) {
|
|
300
|
+
// Agent is waiting for user input
|
|
301
|
+
}
|
|
302
|
+
```
|
|
303
|
+
|
|
304
|
+
### Worker Mode
|
|
305
|
+
|
|
306
|
+
**Enabled when**: `chat` parameter is omitted from `execute()`
|
|
307
|
+
|
|
308
|
+
Worker Mode is designed for automated execution environments that need to:
|
|
309
|
+
|
|
310
|
+
- Process data and perform computations
|
|
311
|
+
- Execute multi-step workflows
|
|
312
|
+
- Return structured results
|
|
313
|
+
- Run without human interaction
|
|
314
|
+
|
|
315
|
+
Key characteristics:
|
|
316
|
+
|
|
317
|
+
- Focus on computational tasks and data processing
|
|
318
|
+
- Uses `DefaultExit` if no custom exits provided
|
|
319
|
+
- Sandboxed execution with security isolation
|
|
320
|
+
- Automated completion without user interaction
|
|
321
|
+
|
|
322
|
+
```typescript
|
|
323
|
+
const result = await execute({
|
|
324
|
+
instructions: 'Process the customer data and generate insights',
|
|
325
|
+
tools: [dataProcessorTool, analyticseTool],
|
|
326
|
+
exits: [dataProcessedExit],
|
|
327
|
+
client,
|
|
328
|
+
})
|
|
329
|
+
|
|
330
|
+
if (result.is(dataProcessedExit)) {
|
|
331
|
+
console.log('Analysis complete:', result.output)
|
|
332
|
+
}
|
|
333
|
+
```
|
|
334
|
+
|
|
335
|
+
### Mode Comparison
|
|
336
|
+
|
|
337
|
+
| Feature | Chat Mode | Worker Mode |
|
|
338
|
+
| -------------------- | ------------------- | --------------- |
|
|
339
|
+
| User Interaction | ✅ Interactive | ❌ Automated |
|
|
340
|
+
| UI Components | ✅ React components | ❌ No UI |
|
|
341
|
+
| Conversation History | ✅ Full transcript | ❌ No history |
|
|
342
|
+
| Default Exits | `ListenExit` | `DefaultExit` |
|
|
343
|
+
| Primary Use Case | Conversational AI | Data processing |
|
|
344
|
+
| Execution Pattern | Turn-based | Continuous |
|
|
345
|
+
|
|
346
|
+
---
|
|
347
|
+
|
|
348
|
+
## Tools
|
|
349
|
+
|
|
350
|
+
Tools are the primary way to extend LLMz agents with external capabilities. Unlike traditional agent frameworks, LLMz tools are called through generated TypeScript code, enabling complex orchestration and error handling.
|
|
351
|
+
|
|
352
|
+
### Tool Definition
|
|
353
|
+
|
|
354
|
+
Tools are defined using Zui schemas for complete type safety:
|
|
355
|
+
|
|
356
|
+
```typescript
|
|
357
|
+
import { Tool } from 'llmz'
|
|
358
|
+
import { z } from '@bpinternal/zui'
|
|
359
|
+
|
|
360
|
+
const weatherTool = new Tool({
|
|
361
|
+
name: 'getWeather',
|
|
362
|
+
description: 'Get current weather for a location',
|
|
363
|
+
input: z.object({
|
|
364
|
+
location: z.string().describe('City name or coordinates'),
|
|
365
|
+
units: z.enum(['celsius', 'fahrenheit']).optional().default('celsius'),
|
|
366
|
+
}),
|
|
367
|
+
output: z.object({
|
|
368
|
+
temperature: z.number(),
|
|
369
|
+
conditions: z.string(),
|
|
370
|
+
humidity: z.number(),
|
|
371
|
+
}),
|
|
372
|
+
handler: async ({ location, units }) => {
|
|
373
|
+
// Implementation here
|
|
374
|
+
return {
|
|
375
|
+
temperature: 22,
|
|
376
|
+
conditions: 'sunny',
|
|
377
|
+
humidity: 65,
|
|
378
|
+
}
|
|
379
|
+
},
|
|
380
|
+
})
|
|
381
|
+
```
|
|
382
|
+
|
|
383
|
+
### Tool Usage in Generated Code
|
|
384
|
+
|
|
385
|
+
The LLM generates TypeScript code that calls tools naturally:
|
|
386
|
+
|
|
387
|
+
```typescript
|
|
388
|
+
// Simple tool call
|
|
389
|
+
const weather = await getWeather({ location: 'New York' })
|
|
390
|
+
|
|
391
|
+
// Complex logic with multiple tools
|
|
392
|
+
const weather = await getWeather({ location: userLocation })
|
|
393
|
+
if (weather.temperature < 0) {
|
|
394
|
+
const clothing = await getSuggestions({ type: 'winter', temperature: weather.temperature })
|
|
395
|
+
yield <Text>It's {weather.temperature}°C! {clothing.suggestion}</Text>
|
|
396
|
+
} else {
|
|
397
|
+
yield <Text>Nice weather! {weather.conditions} at {weather.temperature}°C</Text>
|
|
398
|
+
}
|
|
399
|
+
|
|
400
|
+
return { action: 'listen' }
|
|
401
|
+
```
|
|
402
|
+
|
|
403
|
+
### Advanced Tool Features
|
|
404
|
+
|
|
405
|
+
#### Tool Aliases
|
|
406
|
+
|
|
407
|
+
Tools can have multiple names for flexible calling:
|
|
408
|
+
|
|
409
|
+
```typescript
|
|
410
|
+
const tool = new Tool({
|
|
411
|
+
name: 'calculatePrice',
|
|
412
|
+
aliases: ['getPrice', 'checkCost'],
|
|
413
|
+
// ... rest of definition
|
|
414
|
+
})
|
|
415
|
+
|
|
416
|
+
// All of these work in generated code:
|
|
417
|
+
// await calculatePrice(params)
|
|
418
|
+
// await getPrice(params)
|
|
419
|
+
// await checkCost(params)
|
|
420
|
+
```
|
|
421
|
+
|
|
422
|
+
#### Static Inputs
|
|
423
|
+
|
|
424
|
+
Force specific inputs to be always included:
|
|
425
|
+
|
|
426
|
+
```typescript
|
|
427
|
+
const tool = new Tool({
|
|
428
|
+
name: 'logEvent',
|
|
429
|
+
input: z.object({
|
|
430
|
+
event: z.string(),
|
|
431
|
+
userId: z.string(),
|
|
432
|
+
timestamp: z.number(),
|
|
433
|
+
}),
|
|
434
|
+
staticInputs: {
|
|
435
|
+
userId: 'user-123',
|
|
436
|
+
timestamp: () => Date.now(), // Dynamic static input
|
|
437
|
+
},
|
|
438
|
+
handler: async ({ event, userId, timestamp }) => {
|
|
439
|
+
// userId and timestamp are automatically provided
|
|
440
|
+
},
|
|
441
|
+
})
|
|
442
|
+
```
|
|
443
|
+
|
|
444
|
+
#### Tool Wrapping and Cloning
|
|
445
|
+
|
|
446
|
+
Clone and modify existing tools:
|
|
447
|
+
|
|
448
|
+
```typescript
|
|
449
|
+
const originalTool = new Tool({
|
|
450
|
+
/* definition */
|
|
451
|
+
})
|
|
452
|
+
|
|
453
|
+
const wrappedTool = originalTool.clone({
|
|
454
|
+
name: 'wrappedVersion',
|
|
455
|
+
description: 'Enhanced version with logging',
|
|
456
|
+
handler: async (input) => {
|
|
457
|
+
console.log('Tool called with:', input)
|
|
458
|
+
const result = await originalTool.execute(input)
|
|
459
|
+
console.log('Tool returned:', result)
|
|
460
|
+
return result
|
|
461
|
+
},
|
|
462
|
+
})
|
|
463
|
+
```
|
|
464
|
+
|
|
465
|
+
### Tool Type Generation
|
|
466
|
+
|
|
467
|
+
Use `tool.getTypings()` to see the TypeScript definitions generated for the LLM:
|
|
468
|
+
|
|
469
|
+
```typescript
|
|
470
|
+
console.log(weatherTool.getTypings())
|
|
471
|
+
// Output:
|
|
472
|
+
// /**
|
|
473
|
+
// * Get current weather for a location
|
|
474
|
+
// */
|
|
475
|
+
// declare function getWeather(input: {
|
|
476
|
+
// location: string; // City name or coordinates
|
|
477
|
+
// units?: "celsius" | "fahrenheit";
|
|
478
|
+
// }): Promise<{
|
|
479
|
+
// temperature: number;
|
|
480
|
+
// conditions: string;
|
|
481
|
+
// humidity: number;
|
|
482
|
+
// }>;
|
|
483
|
+
```
|
|
484
|
+
|
|
485
|
+
### Best Practices
|
|
486
|
+
|
|
487
|
+
1. **Descriptive Schemas**: Detailed descriptions help LLMs generate better code
|
|
488
|
+
2. **Type Safety**: Use strict Zui schemas for predictable behavior
|
|
489
|
+
3. **Error Handling**: Tools should handle errors gracefully
|
|
490
|
+
4. **Performance**: Keep tool execution fast to avoid timeouts
|
|
491
|
+
5. **Documentation**: Clear descriptions improve code generation quality
|
|
492
|
+
|
|
493
|
+
---
|
|
494
|
+
|
|
495
|
+
## Objects and Variables
|
|
496
|
+
|
|
497
|
+
Objects in LLMz provide namespaced containers for related tools and variables, enabling sophisticated state management and data organization.
|
|
498
|
+
|
|
499
|
+
### Object Definition
|
|
500
|
+
|
|
501
|
+
Objects group related functionality and provide scoped variables:
|
|
502
|
+
|
|
503
|
+
```typescript
|
|
504
|
+
import { ObjectInstance } from 'llmz'
|
|
505
|
+
import { z } from '@bpinternal/zui'
|
|
506
|
+
|
|
507
|
+
const userObject = new ObjectInstance({
|
|
508
|
+
name: 'user',
|
|
509
|
+
properties: [
|
|
510
|
+
{
|
|
511
|
+
name: 'name',
|
|
512
|
+
value: 'John Doe',
|
|
513
|
+
writable: true,
|
|
514
|
+
type: z.string(),
|
|
515
|
+
},
|
|
516
|
+
{
|
|
517
|
+
name: 'age',
|
|
518
|
+
value: 30,
|
|
519
|
+
writable: false, // Read-only
|
|
520
|
+
type: z.number(),
|
|
521
|
+
},
|
|
522
|
+
{
|
|
523
|
+
name: 'preferences',
|
|
524
|
+
value: { theme: 'dark', language: 'en' },
|
|
525
|
+
writable: true,
|
|
526
|
+
type: z.object({
|
|
527
|
+
theme: z.enum(['light', 'dark']),
|
|
528
|
+
language: z.string(),
|
|
529
|
+
}),
|
|
530
|
+
},
|
|
531
|
+
],
|
|
532
|
+
tools: [
|
|
533
|
+
new Tool({
|
|
534
|
+
name: 'updateProfile',
|
|
535
|
+
input: z.object({ name: z.string() }),
|
|
536
|
+
handler: async ({ name }) => {
|
|
537
|
+
// This tool is scoped to the user object
|
|
538
|
+
return { success: true }
|
|
539
|
+
},
|
|
540
|
+
}),
|
|
541
|
+
],
|
|
542
|
+
})
|
|
543
|
+
```
|
|
544
|
+
|
|
545
|
+
### Variables in Generated Code
|
|
546
|
+
|
|
547
|
+
The LLM can read and write object properties in generated code:
|
|
548
|
+
|
|
549
|
+
```typescript
|
|
550
|
+
// Reading variables
|
|
551
|
+
const userName = user.name // "John Doe"
|
|
552
|
+
const userAge = user.age // 30
|
|
553
|
+
|
|
554
|
+
// Writing to writable variables
|
|
555
|
+
user.name = 'Jane Smith' // ✅ Succeeds
|
|
556
|
+
user.preferences = { theme: 'light', language: 'es' } // ✅ Succeeds
|
|
557
|
+
|
|
558
|
+
// Attempting to write read-only variables
|
|
559
|
+
user.age = 25 // ❌ Throws AssignmentError
|
|
560
|
+
```
|
|
561
|
+
|
|
562
|
+
### Type Safety and Validation
|
|
563
|
+
|
|
564
|
+
Variables are validated against their schemas:
|
|
565
|
+
|
|
566
|
+
```typescript
|
|
567
|
+
// Valid assignment
|
|
568
|
+
user.preferences = { theme: 'dark', language: 'fr' } // ✅
|
|
569
|
+
|
|
570
|
+
// Invalid assignment - wrong type
|
|
571
|
+
user.preferences = { theme: 'blue', language: 'fr' } // ❌ Throws validation error
|
|
572
|
+
|
|
573
|
+
// Invalid assignment - missing required fields
|
|
574
|
+
user.preferences = { theme: 'dark' } // ❌ Missing language field
|
|
575
|
+
```
|
|
576
|
+
|
|
577
|
+
### Mutation Tracking
|
|
578
|
+
|
|
579
|
+
LLMz automatically tracks changes to object properties:
|
|
580
|
+
|
|
581
|
+
```typescript
|
|
582
|
+
// In generated code
|
|
583
|
+
user.name = 'Updated Name'
|
|
584
|
+
user.preferences.theme = 'light'
|
|
585
|
+
|
|
586
|
+
// After execution, mutations are available
|
|
587
|
+
console.log(result.iteration.mutations)
|
|
588
|
+
// [
|
|
589
|
+
// {
|
|
590
|
+
// object: 'user',
|
|
591
|
+
// property: 'name',
|
|
592
|
+
// before: 'John Doe',
|
|
593
|
+
// after: 'Updated Name'
|
|
594
|
+
// },
|
|
595
|
+
// {
|
|
596
|
+
// object: 'user',
|
|
597
|
+
// property: 'preferences',
|
|
598
|
+
// before: { theme: 'dark', language: 'en' },
|
|
599
|
+
// after: { theme: 'light', language: 'en' }
|
|
600
|
+
// }
|
|
601
|
+
// ]
|
|
602
|
+
```
|
|
603
|
+
|
|
604
|
+
### Namespaced Tools
|
|
605
|
+
|
|
606
|
+
Tools within objects are called with object namespace:
|
|
607
|
+
|
|
608
|
+
```typescript
|
|
609
|
+
// Tool is scoped to the user object
|
|
610
|
+
await user.updateProfile({ name: 'New Name' })
|
|
611
|
+
|
|
612
|
+
// This automatically updates the user object's properties
|
|
613
|
+
// and is tracked as a mutation
|
|
614
|
+
```
|
|
615
|
+
|
|
616
|
+
### Object Sealing and Protection
|
|
617
|
+
|
|
618
|
+
Objects are automatically sealed to prevent unauthorized modifications:
|
|
619
|
+
|
|
620
|
+
```typescript
|
|
621
|
+
// In generated code - these will throw errors
|
|
622
|
+
user.newProperty = 'value' // ❌ Cannot add new properties
|
|
623
|
+
delete user.name // ❌ Cannot delete properties
|
|
624
|
+
|
|
625
|
+
// Only predefined properties can be modified (if writable)
|
|
626
|
+
user.name = 'New Name' // ✅ Allowed if writable: true
|
|
627
|
+
```
|
|
628
|
+
|
|
629
|
+
### Variable Persistence
|
|
630
|
+
|
|
631
|
+
Variables persist across iterations and thinking cycles:
|
|
632
|
+
|
|
633
|
+
```typescript
|
|
634
|
+
// Iteration 1: Set a variable
|
|
635
|
+
user.preferences = { theme: 'dark', language: 'es' }
|
|
636
|
+
return { action: 'think' } // Trigger thinking
|
|
637
|
+
|
|
638
|
+
// Iteration 2: Variable is still available
|
|
639
|
+
const currentTheme = user.preferences.theme // 'dark'
|
|
640
|
+
```
|
|
641
|
+
|
|
642
|
+
### Best Practices
|
|
643
|
+
|
|
644
|
+
1. **Meaningful Names**: Use descriptive object and property names
|
|
645
|
+
2. **Appropriate Scope**: Group related functionality together
|
|
646
|
+
3. **Write Protection**: Mark properties as read-only when appropriate
|
|
647
|
+
4. **Type Safety**: Use strict schemas for predictable behavior
|
|
648
|
+
5. **Mutation Tracking**: Leverage mutation tracking for audit trails
|
|
649
|
+
|
|
650
|
+
---
|
|
651
|
+
|
|
652
|
+
## Execution Results
|
|
653
|
+
|
|
654
|
+
Every call to `execute()` returns an `ExecutionResult` that provides type-safe access to the execution outcome. LLMz execution can result in three different types of results.
|
|
655
|
+
|
|
656
|
+
### Result Types
|
|
657
|
+
|
|
658
|
+
#### SuccessExecutionResult
|
|
659
|
+
|
|
660
|
+
Agent completed successfully with an Exit. Contains the structured data produced by the agent.
|
|
661
|
+
|
|
662
|
+
```typescript
|
|
663
|
+
const result = await execute({
|
|
664
|
+
instructions: 'Calculate the sum',
|
|
665
|
+
client,
|
|
666
|
+
})
|
|
667
|
+
|
|
668
|
+
if (result.isSuccess()) {
|
|
669
|
+
console.log('Output:', result.output)
|
|
670
|
+
console.log('Exit used:', result.exit.name)
|
|
671
|
+
console.log('Generated code:', result.iteration.code)
|
|
672
|
+
}
|
|
673
|
+
```
|
|
674
|
+
|
|
675
|
+
#### ErrorExecutionResult
|
|
676
|
+
|
|
677
|
+
Execution failed with an unrecoverable error:
|
|
678
|
+
|
|
679
|
+
```typescript
|
|
680
|
+
if (result.isError()) {
|
|
681
|
+
console.error('Error:', result.error)
|
|
682
|
+
console.error('Failed iteration:', result.iteration?.error)
|
|
683
|
+
|
|
684
|
+
// Analyze failure progression
|
|
685
|
+
result.iterations.forEach((iter, i) => {
|
|
686
|
+
console.log(`Iteration ${i + 1}: ${iter.status.type}`)
|
|
687
|
+
})
|
|
688
|
+
}
|
|
689
|
+
```
|
|
690
|
+
|
|
691
|
+
#### PartialExecutionResult
|
|
692
|
+
|
|
693
|
+
Execution was interrupted by a SnapshotSignal for pauseable operations:
|
|
694
|
+
|
|
695
|
+
```typescript
|
|
696
|
+
if (result.isInterrupted()) {
|
|
697
|
+
console.log('Interrupted:', result.signal.message)
|
|
698
|
+
|
|
699
|
+
// Save snapshot for later resumption
|
|
700
|
+
const serialized = result.snapshot.toJSON()
|
|
701
|
+
await database.saveSnapshot(serialized)
|
|
702
|
+
}
|
|
703
|
+
```
|
|
704
|
+
|
|
705
|
+
### Type-Safe Exit Checking
|
|
706
|
+
|
|
707
|
+
Use `result.is(exit)` for type-safe access to specific exit data:
|
|
708
|
+
|
|
709
|
+
```typescript
|
|
710
|
+
const successExit = new Exit({
|
|
711
|
+
name: 'success',
|
|
712
|
+
schema: z.object({
|
|
713
|
+
recordsProcessed: z.number(),
|
|
714
|
+
processingTime: z.number(),
|
|
715
|
+
}),
|
|
716
|
+
})
|
|
717
|
+
|
|
718
|
+
const errorExit = new Exit({
|
|
719
|
+
name: 'error',
|
|
720
|
+
schema: z.object({
|
|
721
|
+
errorCode: z.string(),
|
|
722
|
+
details: z.string(),
|
|
723
|
+
}),
|
|
724
|
+
})
|
|
725
|
+
|
|
726
|
+
const result = await execute({
|
|
727
|
+
instructions: 'Process the data',
|
|
728
|
+
exits: [successExit, errorExit],
|
|
729
|
+
client,
|
|
730
|
+
})
|
|
731
|
+
|
|
732
|
+
// Type-safe exit handling with automatic output typing
|
|
733
|
+
if (result.is(successExit)) {
|
|
734
|
+
// TypeScript knows result.output has the success schema
|
|
735
|
+
console.log(`Processed ${result.output.recordsProcessed} records`)
|
|
736
|
+
console.log(`Processing took ${result.output.processingTime}ms`)
|
|
737
|
+
} else if (result.is(errorExit)) {
|
|
738
|
+
// TypeScript knows result.output has the error schema
|
|
739
|
+
console.error(`Error ${result.output.errorCode}: ${result.output.details}`)
|
|
740
|
+
}
|
|
741
|
+
```
|
|
742
|
+
|
|
743
|
+
### Built-in Exits
|
|
744
|
+
|
|
745
|
+
```typescript
|
|
746
|
+
import { ListenExit, DefaultExit, ThinkExit } from 'llmz'
|
|
747
|
+
|
|
748
|
+
// Check for built-in exits
|
|
749
|
+
if (result.is(ListenExit)) {
|
|
750
|
+
console.log('Agent is waiting for user input')
|
|
751
|
+
}
|
|
752
|
+
|
|
753
|
+
if (result.is(DefaultExit)) {
|
|
754
|
+
// DefaultExit has success/failure discriminated union
|
|
755
|
+
if (result.output.success) {
|
|
756
|
+
console.log('Completed successfully:', result.output.result)
|
|
757
|
+
} else {
|
|
758
|
+
console.error('Completed with error:', result.output.error)
|
|
759
|
+
}
|
|
760
|
+
}
|
|
761
|
+
|
|
762
|
+
if (result.is(ThinkExit)) {
|
|
763
|
+
console.log('Agent requested thinking time')
|
|
764
|
+
console.log('Current variables:', result.output.variables)
|
|
765
|
+
}
|
|
766
|
+
```
|
|
767
|
+
|
|
768
|
+
### Accessing Execution Details
|
|
769
|
+
|
|
770
|
+
#### Iterations and Execution Flow
|
|
771
|
+
|
|
772
|
+
```typescript
|
|
773
|
+
// Access the final iteration
|
|
774
|
+
const lastIteration = result.iteration
|
|
775
|
+
if (lastIteration) {
|
|
776
|
+
console.log('Generated code:', lastIteration.code)
|
|
777
|
+
console.log('Status:', lastIteration.status.type)
|
|
778
|
+
console.log('Duration:', lastIteration.duration)
|
|
779
|
+
}
|
|
780
|
+
|
|
781
|
+
// Access all iterations to see full execution flow
|
|
782
|
+
result.iterations.forEach((iteration, index) => {
|
|
783
|
+
console.log(`Iteration ${index + 1}:`)
|
|
784
|
+
console.log(' Status:', iteration.status.type)
|
|
785
|
+
console.log(' Code length:', iteration.code?.length || 0)
|
|
786
|
+
console.log(' Variables:', Object.keys(iteration.variables).length)
|
|
787
|
+
})
|
|
788
|
+
```
|
|
789
|
+
|
|
790
|
+
#### Variables and Declarations
|
|
791
|
+
|
|
792
|
+
```typescript
|
|
793
|
+
// If agent generates: const hello = '1234'
|
|
794
|
+
const lastIteration = result.iteration
|
|
795
|
+
if (lastIteration) {
|
|
796
|
+
console.log(lastIteration.variables.hello) // '1234'
|
|
797
|
+
|
|
798
|
+
// Access all variables from the final iteration
|
|
799
|
+
Object.entries(lastIteration.variables).forEach(([name, value]) => {
|
|
800
|
+
console.log(`Variable ${name}:`, value)
|
|
801
|
+
})
|
|
802
|
+
}
|
|
803
|
+
```
|
|
804
|
+
|
|
805
|
+
#### Tool Calls and Traces
|
|
806
|
+
|
|
807
|
+
```typescript
|
|
808
|
+
// Access tool calls from all iterations
|
|
809
|
+
const allToolCalls = result.iterations.flatMap((iter) => iter.traces.filter((trace) => trace.type === 'tool_call'))
|
|
810
|
+
|
|
811
|
+
console.log('Total tool calls:', allToolCalls.length)
|
|
812
|
+
|
|
813
|
+
// Access other trace types
|
|
814
|
+
const lastIteration = result.iteration
|
|
815
|
+
if (lastIteration) {
|
|
816
|
+
const yields = lastIteration.traces.filter((trace) => trace.type === 'yield')
|
|
817
|
+
const comments = lastIteration.traces.filter((trace) => trace.type === 'comment')
|
|
818
|
+
const propertyAccess = lastIteration.traces.filter((trace) => trace.type === 'property')
|
|
819
|
+
}
|
|
820
|
+
```
|
|
821
|
+
|
|
822
|
+
#### Context and Metadata
|
|
823
|
+
|
|
824
|
+
```typescript
|
|
825
|
+
if (result.isSuccess()) {
|
|
826
|
+
// Access original execution parameters
|
|
827
|
+
console.log('Instructions:', result.context.instructions)
|
|
828
|
+
console.log('Loop limit:', result.context.loop)
|
|
829
|
+
console.log('Temperature:', result.context.temperature)
|
|
830
|
+
console.log('Model:', result.context.model)
|
|
831
|
+
|
|
832
|
+
// Access tools and exits that were available
|
|
833
|
+
console.log(
|
|
834
|
+
'Available tools:',
|
|
835
|
+
result.context.tools?.map((t) => t.name)
|
|
836
|
+
)
|
|
837
|
+
console.log(
|
|
838
|
+
'Available exits:',
|
|
839
|
+
result.context.exits?.map((e) => e.name)
|
|
840
|
+
)
|
|
841
|
+
}
|
|
842
|
+
```
|
|
843
|
+
|
|
844
|
+
### Error Analysis
|
|
845
|
+
|
|
846
|
+
```typescript
|
|
847
|
+
if (result.isError()) {
|
|
848
|
+
console.error('Execution failed:', result.error)
|
|
849
|
+
|
|
850
|
+
// Analyze the failure progression
|
|
851
|
+
const failedIteration = result.iteration
|
|
852
|
+
if (failedIteration) {
|
|
853
|
+
switch (failedIteration.status.type) {
|
|
854
|
+
case 'execution_error':
|
|
855
|
+
console.error('Code execution failed:', failedIteration.status.execution_error.message)
|
|
856
|
+
console.error('Stack trace:', failedIteration.status.execution_error.stack)
|
|
857
|
+
console.error('Failed code:', failedIteration.code)
|
|
858
|
+
break
|
|
859
|
+
|
|
860
|
+
case 'generation_error':
|
|
861
|
+
console.error('LLM generation failed:', failedIteration.status.generation_error.message)
|
|
862
|
+
break
|
|
863
|
+
|
|
864
|
+
case 'invalid_code_error':
|
|
865
|
+
console.error('Invalid code generated:', failedIteration.status.invalid_code_error.message)
|
|
866
|
+
console.error('Invalid code:', failedIteration.code)
|
|
867
|
+
break
|
|
868
|
+
|
|
869
|
+
case 'aborted':
|
|
870
|
+
console.error('Execution aborted:', failedIteration.status.aborted.reason)
|
|
871
|
+
break
|
|
872
|
+
}
|
|
873
|
+
}
|
|
874
|
+
|
|
875
|
+
// Review all iterations to understand failure progression
|
|
876
|
+
console.log('Iterations before failure:', result.iterations.length)
|
|
877
|
+
result.iterations.forEach((iter, i) => {
|
|
878
|
+
console.log(`Iteration ${i + 1}: ${iter.status.type}`)
|
|
879
|
+
})
|
|
880
|
+
}
|
|
881
|
+
```
|
|
882
|
+
|
|
883
|
+
### Snapshot Handling
|
|
884
|
+
|
|
885
|
+
Handle interrupted executions with snapshot resumption:
|
|
886
|
+
|
|
887
|
+
```typescript
|
|
888
|
+
const result = await execute({
|
|
889
|
+
instructions: 'Process large dataset with pauseable operation',
|
|
890
|
+
tools: [snapshotCapableTool],
|
|
891
|
+
client,
|
|
892
|
+
})
|
|
893
|
+
|
|
894
|
+
if (result.isInterrupted()) {
|
|
895
|
+
console.log('Execution paused:', result.signal.message)
|
|
896
|
+
|
|
897
|
+
// Serialize snapshot for persistence
|
|
898
|
+
const serialized = result.snapshot.toJSON()
|
|
899
|
+
await database.saveSnapshot('execution-123', serialized)
|
|
900
|
+
|
|
901
|
+
// Later, resume from snapshot
|
|
902
|
+
const snapshot = Snapshot.fromJSON(serialized)
|
|
903
|
+
snapshot.resolve({ resumeData: 'Operation completed' })
|
|
904
|
+
|
|
905
|
+
const continuation = await execute({
|
|
906
|
+
snapshot,
|
|
907
|
+
instructions: result.context.instructions,
|
|
908
|
+
tools: result.context.tools,
|
|
909
|
+
exits: result.context.exits,
|
|
910
|
+
client,
|
|
911
|
+
})
|
|
912
|
+
|
|
913
|
+
if (continuation.isSuccess()) {
|
|
914
|
+
console.log('Resumed execution completed:', continuation.output)
|
|
915
|
+
}
|
|
916
|
+
}
|
|
917
|
+
```
|
|
918
|
+
|
|
919
|
+
---
|
|
920
|
+
|
|
921
|
+
## Hooks System
|
|
922
|
+
|
|
923
|
+
LLMz provides a comprehensive hook system that allows you to inject custom logic at various points during execution. Hooks are categorized as either blocking (execution waits) or non-blocking, and either mutation (can modify data) or non-mutation.
|
|
924
|
+
|
|
925
|
+
### Hook Types Overview
|
|
926
|
+
|
|
927
|
+
| Hook | Blocking | Mutation | Called When |
|
|
928
|
+
| ------------------- | -------- | -------- | -------------------------- |
|
|
929
|
+
| `onTrace` | ❌ | ❌ | Each trace generated |
|
|
930
|
+
| `onIterationEnd` | ✅ | ❌ | After iteration completion |
|
|
931
|
+
| `onExit` | ✅ | ❌ | When exit is reached |
|
|
932
|
+
| `onBeforeExecution` | ✅ | ✅ | Before code execution |
|
|
933
|
+
| `onBeforeTool` | ✅ | ✅ | Before tool execution |
|
|
934
|
+
| `onAfterTool` | ✅ | ✅ | After tool execution |
|
|
935
|
+
|
|
936
|
+
### onTrace (Non-blocking, Non-mutation)
|
|
937
|
+
|
|
938
|
+
Called for each trace generated during iteration. Useful for logging, debugging, or monitoring execution progress.
|
|
939
|
+
|
|
940
|
+
```typescript
|
|
941
|
+
await execute({
|
|
942
|
+
onTrace: ({ trace, iteration }) => {
|
|
943
|
+
console.log(`Iteration ${iteration}: ${trace.type}`, trace)
|
|
944
|
+
|
|
945
|
+
// Log specific trace types
|
|
946
|
+
if (trace.type === 'tool_call') {
|
|
947
|
+
console.log(`Tool ${trace.tool_name} called with:`, trace.input)
|
|
948
|
+
}
|
|
949
|
+
},
|
|
950
|
+
// ... other props
|
|
951
|
+
})
|
|
952
|
+
```
|
|
953
|
+
|
|
954
|
+
**Available Trace Types:**
|
|
955
|
+
|
|
956
|
+
- `abort_signal`: Abort signal received
|
|
957
|
+
- `comment`: Comment found in generated code
|
|
958
|
+
- `llm_call_success`: LLM generation completed successfully
|
|
959
|
+
- `property`: Object property accessed or modified
|
|
960
|
+
- `think_signal`: ThinkSignal thrown
|
|
961
|
+
- `tool_call`: Tool executed
|
|
962
|
+
- `yield`: Component yielded in chat mode
|
|
963
|
+
- `log`: General logging event
|
|
964
|
+
|
|
965
|
+
### onIterationEnd (Blocking, Non-mutation)
|
|
966
|
+
|
|
967
|
+
Called after each iteration ends, regardless of status. Useful for logging, cleanup, or controlling iteration timing.
|
|
968
|
+
|
|
969
|
+
```typescript
|
|
970
|
+
await execute({
|
|
971
|
+
onIterationEnd: async (iteration, controller) => {
|
|
972
|
+
console.log(`Iteration ${iteration.id} ended with status: ${iteration.status.type}`)
|
|
973
|
+
|
|
974
|
+
// Add delays, cleanup, or conditional logic
|
|
975
|
+
if (iteration.status.type === 'execution_error') {
|
|
976
|
+
await logError(iteration.error)
|
|
977
|
+
|
|
978
|
+
// Add delay before retry
|
|
979
|
+
await new Promise((resolve) => setTimeout(resolve, 1000))
|
|
980
|
+
}
|
|
981
|
+
|
|
982
|
+
// Can use controller to abort execution if needed
|
|
983
|
+
if (shouldAbort(iteration)) {
|
|
984
|
+
controller.abort('Custom abort reason')
|
|
985
|
+
}
|
|
986
|
+
},
|
|
987
|
+
// ... other props
|
|
988
|
+
})
|
|
989
|
+
```
|
|
990
|
+
|
|
991
|
+
### onExit (Blocking, Non-mutation)
|
|
992
|
+
|
|
993
|
+
Called when an exit is reached. Useful for logging, notifications, or implementing guardrails by throwing errors to prevent exit.
|
|
994
|
+
|
|
995
|
+
```typescript
|
|
996
|
+
await execute({
|
|
997
|
+
onExit: async (result) => {
|
|
998
|
+
console.log(`Exiting with: ${result.exit.name}`, result.result)
|
|
999
|
+
|
|
1000
|
+
// Implement guardrails
|
|
1001
|
+
if (result.exit.name === 'approve_loan' && result.result.amount > 10000) {
|
|
1002
|
+
throw new Error('Manager approval required for loans over $10,000')
|
|
1003
|
+
}
|
|
1004
|
+
|
|
1005
|
+
// Send notifications
|
|
1006
|
+
await notifyStakeholders(result)
|
|
1007
|
+
|
|
1008
|
+
// Log to audit trail
|
|
1009
|
+
await auditLog.record({
|
|
1010
|
+
action: result.exit.name,
|
|
1011
|
+
data: result.result,
|
|
1012
|
+
timestamp: Date.now(),
|
|
1013
|
+
})
|
|
1014
|
+
},
|
|
1015
|
+
// ... other props
|
|
1016
|
+
})
|
|
1017
|
+
```
|
|
1018
|
+
|
|
1019
|
+
### onBeforeExecution (Blocking, Mutation)
|
|
1020
|
+
|
|
1021
|
+
Called after LLM generates code but before execution. Allows code modification and guardrails implementation.
|
|
1022
|
+
|
|
1023
|
+
```typescript
|
|
1024
|
+
await execute({
|
|
1025
|
+
onBeforeExecution: async (iteration, controller) => {
|
|
1026
|
+
console.log('Generated code:', iteration.code)
|
|
1027
|
+
|
|
1028
|
+
// Code modification
|
|
1029
|
+
if (iteration.code?.includes('dangerousOperation')) {
|
|
1030
|
+
return {
|
|
1031
|
+
code: iteration.code.replace('dangerousOperation', 'safeOperation'),
|
|
1032
|
+
}
|
|
1033
|
+
}
|
|
1034
|
+
|
|
1035
|
+
// Guardrails - throw to prevent execution
|
|
1036
|
+
if (iteration.code?.includes('forbidden')) {
|
|
1037
|
+
throw new Error('Forbidden operation detected')
|
|
1038
|
+
}
|
|
1039
|
+
|
|
1040
|
+
// Add security checks
|
|
1041
|
+
const securityIssues = await scanCodeForSecurity(iteration.code)
|
|
1042
|
+
if (securityIssues.length > 0) {
|
|
1043
|
+
throw new Error(`Security issues found: ${securityIssues.join(', ')}`)
|
|
1044
|
+
}
|
|
1045
|
+
|
|
1046
|
+
// Log code generation for audit
|
|
1047
|
+
await auditCodeGeneration(iteration.code)
|
|
1048
|
+
},
|
|
1049
|
+
// ... other props
|
|
1050
|
+
})
|
|
1051
|
+
```
|
|
1052
|
+
|
|
1053
|
+
### onBeforeTool (Blocking, Mutation)
|
|
1054
|
+
|
|
1055
|
+
Called before any tool execution. Allows input modification and tool execution control.
|
|
1056
|
+
|
|
1057
|
+
```typescript
|
|
1058
|
+
await execute({
|
|
1059
|
+
onBeforeTool: async ({ iteration, tool, input, controller }) => {
|
|
1060
|
+
console.log(`Executing tool: ${tool.name}`, input)
|
|
1061
|
+
|
|
1062
|
+
// Input modification
|
|
1063
|
+
if (tool.name === 'sendEmail') {
|
|
1064
|
+
return {
|
|
1065
|
+
input: {
|
|
1066
|
+
...input,
|
|
1067
|
+
subject: `[Automated] ${input.subject}`, // Add prefix
|
|
1068
|
+
from: 'noreply@company.com', // Override sender
|
|
1069
|
+
},
|
|
1070
|
+
}
|
|
1071
|
+
}
|
|
1072
|
+
|
|
1073
|
+
// Access control
|
|
1074
|
+
if (tool.name === 'deleteFile' && !hasPermission(input.path)) {
|
|
1075
|
+
throw new Error('Insufficient permissions to delete file')
|
|
1076
|
+
}
|
|
1077
|
+
|
|
1078
|
+
// Rate limiting
|
|
1079
|
+
const rateLimit = await checkRateLimit(tool.name)
|
|
1080
|
+
if (rateLimit.exceeded) {
|
|
1081
|
+
throw new Error(`Rate limit exceeded for ${tool.name}`)
|
|
1082
|
+
}
|
|
1083
|
+
|
|
1084
|
+
// Validation
|
|
1085
|
+
await validateToolUsage(tool, input)
|
|
1086
|
+
},
|
|
1087
|
+
// ... other props
|
|
1088
|
+
})
|
|
1089
|
+
```
|
|
1090
|
+
|
|
1091
|
+
### onAfterTool (Blocking, Mutation)
|
|
1092
|
+
|
|
1093
|
+
Called after tool execution. Allows output modification and post-processing.
|
|
1094
|
+
|
|
1095
|
+
```typescript
|
|
1096
|
+
await execute({
|
|
1097
|
+
onAfterTool: async ({ iteration, tool, input, output, controller }) => {
|
|
1098
|
+
console.log(`Tool ${tool.name} completed`, { input, output })
|
|
1099
|
+
|
|
1100
|
+
// Output modification
|
|
1101
|
+
if (tool.name === 'fetchUserData') {
|
|
1102
|
+
return {
|
|
1103
|
+
output: {
|
|
1104
|
+
...output,
|
|
1105
|
+
// Remove sensitive data before LLM sees it
|
|
1106
|
+
ssn: undefined,
|
|
1107
|
+
creditCard: undefined,
|
|
1108
|
+
// Add metadata
|
|
1109
|
+
fetchedAt: Date.now(),
|
|
1110
|
+
},
|
|
1111
|
+
}
|
|
1112
|
+
}
|
|
1113
|
+
|
|
1114
|
+
// Result enhancement
|
|
1115
|
+
if (tool.name === 'calculatePrice') {
|
|
1116
|
+
return {
|
|
1117
|
+
output: {
|
|
1118
|
+
...output,
|
|
1119
|
+
currency: 'USD',
|
|
1120
|
+
timestamp: Date.now(),
|
|
1121
|
+
exchangeRate: await getCurrentExchangeRate(),
|
|
1122
|
+
},
|
|
1123
|
+
}
|
|
1124
|
+
}
|
|
1125
|
+
|
|
1126
|
+
// Logging and caching
|
|
1127
|
+
await Promise.all([
|
|
1128
|
+
cacheResult(tool.name, input, output),
|
|
1129
|
+
logToolExecution(tool.name, input, output),
|
|
1130
|
+
updateMetrics(tool.name, Date.now() - tool.startTime),
|
|
1131
|
+
])
|
|
1132
|
+
},
|
|
1133
|
+
// ... other props
|
|
1134
|
+
})
|
|
1135
|
+
```
|
|
1136
|
+
|
|
1137
|
+
### Hook Execution Order
|
|
1138
|
+
|
|
1139
|
+
For each iteration:
|
|
1140
|
+
|
|
1141
|
+
1. **onTrace**: Throughout execution (non-blocking)
|
|
1142
|
+
2. **onBeforeExecution**: After code generation, before execution
|
|
1143
|
+
3. **onBeforeTool**: Before each tool call
|
|
1144
|
+
4. **onAfterTool**: After each tool call
|
|
1145
|
+
5. **onExit**: When exit is reached
|
|
1146
|
+
6. **onIterationEnd**: After iteration completes
|
|
1147
|
+
|
|
1148
|
+
### Advanced Hook Patterns
|
|
1149
|
+
|
|
1150
|
+
#### Conditional Hook Logic
|
|
1151
|
+
|
|
1152
|
+
```typescript
|
|
1153
|
+
await execute({
|
|
1154
|
+
onBeforeTool: async ({ tool, input }) => {
|
|
1155
|
+
// Apply different logic based on tool
|
|
1156
|
+
switch (tool.name) {
|
|
1157
|
+
case 'payment':
|
|
1158
|
+
return await handlePaymentValidation(input)
|
|
1159
|
+
case 'notification':
|
|
1160
|
+
return await handleNotificationThrottling(input)
|
|
1161
|
+
default:
|
|
1162
|
+
return // No modification
|
|
1163
|
+
}
|
|
1164
|
+
},
|
|
1165
|
+
})
|
|
1166
|
+
```
|
|
1167
|
+
|
|
1168
|
+
#### Error Recovery in Hooks
|
|
1169
|
+
|
|
1170
|
+
```typescript
|
|
1171
|
+
await execute({
|
|
1172
|
+
onExit: async (result) => {
|
|
1173
|
+
try {
|
|
1174
|
+
await criticalPostProcessing(result)
|
|
1175
|
+
} catch (error) {
|
|
1176
|
+
// Log error but don't fail the entire execution
|
|
1177
|
+
console.error('Post-processing failed:', error)
|
|
1178
|
+
|
|
1179
|
+
// Optionally throw to retry the iteration
|
|
1180
|
+
if (error.retryable) {
|
|
1181
|
+
throw new Error('Retrying due to recoverable error')
|
|
1182
|
+
}
|
|
1183
|
+
}
|
|
1184
|
+
},
|
|
1185
|
+
})
|
|
1186
|
+
```
|
|
1187
|
+
|
|
1188
|
+
#### Hook State Management
|
|
1189
|
+
|
|
1190
|
+
```typescript
|
|
1191
|
+
let executionMetrics = { toolCalls: 0, totalTime: 0 }
|
|
1192
|
+
|
|
1193
|
+
await execute({
|
|
1194
|
+
onBeforeTool: async ({ tool }) => {
|
|
1195
|
+
executionMetrics.toolCalls++
|
|
1196
|
+
tool.startTime = Date.now()
|
|
1197
|
+
},
|
|
1198
|
+
|
|
1199
|
+
onAfterTool: async ({ tool }) => {
|
|
1200
|
+
executionMetrics.totalTime += Date.now() - tool.startTime
|
|
1201
|
+
},
|
|
1202
|
+
|
|
1203
|
+
onIterationEnd: async () => {
|
|
1204
|
+
console.log('Execution metrics:', executionMetrics)
|
|
1205
|
+
},
|
|
1206
|
+
})
|
|
1207
|
+
```
|
|
1208
|
+
|
|
1209
|
+
### Best Practices
|
|
1210
|
+
|
|
1211
|
+
1. **Error Handling**: Always wrap hook logic in try-catch for production
|
|
1212
|
+
2. **Performance**: Keep hooks lightweight, especially `onTrace`
|
|
1213
|
+
3. **Security**: Use `onBeforeExecution` and `onBeforeTool` for security validation
|
|
1214
|
+
4. **Debugging**: Leverage `onTrace` for comprehensive execution monitoring
|
|
1215
|
+
5. **Guardrails**: Implement business logic validation in `onExit`
|
|
1216
|
+
6. **Data Transformation**: Use `onBeforeTool`/`onAfterTool` for input/output processing
|
|
1217
|
+
7. **Async Operations**: All hooks support async/await for external API calls
|
|
1218
|
+
8. **State Management**: Use closures or external state for cross-hook data sharing
|
|
1219
|
+
|
|
1220
|
+
---
|
|
1221
|
+
|
|
1222
|
+
## Advanced Features
|
|
1223
|
+
|
|
1224
|
+
### Snapshots (Pauseable Execution)
|
|
1225
|
+
|
|
1226
|
+
Snapshots allow you to pause and resume LLMz execution, enabling long-running workflows that can be interrupted and continued later.
|
|
1227
|
+
|
|
1228
|
+
#### SnapshotSignal
|
|
1229
|
+
|
|
1230
|
+
Inside a tool, throw a `SnapshotSignal` to halt execution and create a serializable snapshot:
|
|
1231
|
+
|
|
1232
|
+
```typescript
|
|
1233
|
+
import { SnapshotSignal, Tool } from 'llmz'
|
|
1234
|
+
|
|
1235
|
+
const longRunningTool = new Tool({
|
|
1236
|
+
name: 'processLargeDataset',
|
|
1237
|
+
input: z.object({ datasetId: z.string() }),
|
|
1238
|
+
async handler({ datasetId }) {
|
|
1239
|
+
// Start processing
|
|
1240
|
+
const dataset = await loadDataset(datasetId)
|
|
1241
|
+
|
|
1242
|
+
// At any point, pause execution for later resumption
|
|
1243
|
+
if (dataset.size > LARGE_THRESHOLD) {
|
|
1244
|
+
throw new SnapshotSignal(
|
|
1245
|
+
'Dataset is large, pausing for background processing',
|
|
1246
|
+
'Processing will continue once background job completes'
|
|
1247
|
+
)
|
|
1248
|
+
}
|
|
1249
|
+
|
|
1250
|
+
return { processed: true }
|
|
1251
|
+
},
|
|
1252
|
+
})
|
|
1253
|
+
```
|
|
1254
|
+
|
|
1255
|
+
#### Snapshot Handling
|
|
1256
|
+
|
|
1257
|
+
```typescript
|
|
1258
|
+
const result = await execute({
|
|
1259
|
+
instructions: 'Process the uploaded dataset',
|
|
1260
|
+
tools: [longRunningTool],
|
|
1261
|
+
client,
|
|
1262
|
+
})
|
|
1263
|
+
|
|
1264
|
+
if (result.isInterrupted()) {
|
|
1265
|
+
console.log('Execution paused:', result.signal.message)
|
|
1266
|
+
|
|
1267
|
+
// Serialize snapshot for persistence
|
|
1268
|
+
const serialized = result.snapshot.toJSON()
|
|
1269
|
+
await database.saveSnapshot('job-123', serialized)
|
|
1270
|
+
|
|
1271
|
+
// Start background processing
|
|
1272
|
+
await backgroundJobQueue.add('process-dataset', {
|
|
1273
|
+
snapshotId: 'job-123',
|
|
1274
|
+
datasetId: result.signal.toolCall?.input.datasetId,
|
|
1275
|
+
})
|
|
1276
|
+
}
|
|
1277
|
+
```
|
|
1278
|
+
|
|
1279
|
+
#### Resuming from Snapshot
|
|
1280
|
+
|
|
1281
|
+
```typescript
|
|
1282
|
+
// Later, when background job completes
|
|
1283
|
+
const serialized = await database.getSnapshot('job-123')
|
|
1284
|
+
const snapshot = Snapshot.fromJSON(serialized)
|
|
1285
|
+
|
|
1286
|
+
// Resolve the snapshot with the result
|
|
1287
|
+
snapshot.resolve({
|
|
1288
|
+
processed: true,
|
|
1289
|
+
recordCount: 1000000,
|
|
1290
|
+
processingTime: 3600000,
|
|
1291
|
+
})
|
|
1292
|
+
|
|
1293
|
+
// Continue execution from where it left off
|
|
1294
|
+
const continuation = await execute({
|
|
1295
|
+
snapshot,
|
|
1296
|
+
instructions: 'Process the uploaded dataset', // Same as original
|
|
1297
|
+
tools: [longRunningTool], // Same tools
|
|
1298
|
+
client,
|
|
1299
|
+
})
|
|
1300
|
+
|
|
1301
|
+
if (continuation.isSuccess()) {
|
|
1302
|
+
console.log('Processing completed:', continuation.output)
|
|
1303
|
+
}
|
|
1304
|
+
```
|
|
1305
|
+
|
|
1306
|
+
#### Snapshot Rejection
|
|
1307
|
+
|
|
1308
|
+
```typescript
|
|
1309
|
+
// If background processing fails
|
|
1310
|
+
const snapshot = Snapshot.fromJSON(serialized)
|
|
1311
|
+
snapshot.reject(new Error('Background processing failed'))
|
|
1312
|
+
|
|
1313
|
+
const continuation = await execute({
|
|
1314
|
+
snapshot,
|
|
1315
|
+
// ... same parameters
|
|
1316
|
+
})
|
|
1317
|
+
|
|
1318
|
+
// The agent will receive the error and can handle it
|
|
1319
|
+
```
|
|
1320
|
+
|
|
1321
|
+
### Thinking (Agent Reflection)
|
|
1322
|
+
|
|
1323
|
+
The thinking system allows agents to pause and reflect on variables and context before proceeding.
|
|
1324
|
+
|
|
1325
|
+
#### ThinkSignal (Tool-Initiated)
|
|
1326
|
+
|
|
1327
|
+
Tools can force thinking by throwing a `ThinkSignal`:
|
|
1328
|
+
|
|
1329
|
+
```typescript
|
|
1330
|
+
const analysisTool = new Tool({
|
|
1331
|
+
name: 'analyzeData',
|
|
1332
|
+
input: z.object({ data: z.array(z.number()) }),
|
|
1333
|
+
async handler({ data }) {
|
|
1334
|
+
const result = performAnalysis(data)
|
|
1335
|
+
|
|
1336
|
+
// Force the agent to think about the results before responding
|
|
1337
|
+
throw new ThinkSignal(
|
|
1338
|
+
'Analysis complete, consider the implications',
|
|
1339
|
+
`Found ${result.anomalies.length} anomalies and ${result.patterns.length} patterns`
|
|
1340
|
+
)
|
|
1341
|
+
},
|
|
1342
|
+
})
|
|
1343
|
+
```
|
|
1344
|
+
|
|
1345
|
+
#### Agent-Initiated Thinking
|
|
1346
|
+
|
|
1347
|
+
Agents can request thinking time in generated code:
|
|
1348
|
+
|
|
1349
|
+
```typescript
|
|
1350
|
+
// In generated code
|
|
1351
|
+
const analysisResult = await analyzeData({ data: userInputData })
|
|
1352
|
+
|
|
1353
|
+
// Think about the results before responding to user
|
|
1354
|
+
return { action: 'think' }
|
|
1355
|
+
```
|
|
1356
|
+
|
|
1357
|
+
#### Thinking with Variables
|
|
1358
|
+
|
|
1359
|
+
Pass specific variables for reflection:
|
|
1360
|
+
|
|
1361
|
+
```typescript
|
|
1362
|
+
// In generated code
|
|
1363
|
+
const price = await calculatePrice({ items: cartItems })
|
|
1364
|
+
const budget = await getUserBudget()
|
|
1365
|
+
|
|
1366
|
+
// Think about pricing vs budget with specific context
|
|
1367
|
+
return {
|
|
1368
|
+
action: 'think',
|
|
1369
|
+
price,
|
|
1370
|
+
budget,
|
|
1371
|
+
recommendation: price > budget ? 'deny' : 'approve',
|
|
1372
|
+
}
|
|
1373
|
+
```
|
|
1374
|
+
|
|
1375
|
+
#### Handling Think Results
|
|
1376
|
+
|
|
1377
|
+
```typescript
|
|
1378
|
+
const result = await execute({
|
|
1379
|
+
instructions: 'Analyze the user data and provide recommendations',
|
|
1380
|
+
tools: [analysisTool],
|
|
1381
|
+
client,
|
|
1382
|
+
})
|
|
1383
|
+
|
|
1384
|
+
if (result.is(ThinkExit)) {
|
|
1385
|
+
console.log('Agent is thinking about:', result.output.variables)
|
|
1386
|
+
|
|
1387
|
+
// Continue execution after thinking
|
|
1388
|
+
const continuation = await execute({
|
|
1389
|
+
instructions: result.context.instructions,
|
|
1390
|
+
tools: result.context.tools,
|
|
1391
|
+
// Variables from thinking are automatically preserved
|
|
1392
|
+
client,
|
|
1393
|
+
})
|
|
1394
|
+
}
|
|
1395
|
+
```
|
|
1396
|
+
|
|
1397
|
+
### Citations (RAG Support)
|
|
1398
|
+
|
|
1399
|
+
CitationsManager provides standardized source tracking and referencing for RAG (Retrieval-Augmented Generation) systems.
|
|
1400
|
+
|
|
1401
|
+
#### Core Concepts
|
|
1402
|
+
|
|
1403
|
+
Citations use rare Unicode symbols (`【】`) as markers that are unlikely to appear in natural text. The system supports:
|
|
1404
|
+
|
|
1405
|
+
- **Source Registration**: Register any object as a citation source
|
|
1406
|
+
- **Tag Generation**: Automatic creation of unique citation tags like `【0】`, `【1】`
|
|
1407
|
+
- **Content Processing**: Extract and clean citation tags from text
|
|
1408
|
+
- **Multiple Citations**: Support for multi-source citations like `【0,1,3】`
|
|
1409
|
+
|
|
1410
|
+
#### Basic Usage
|
|
1411
|
+
|
|
1412
|
+
```typescript
|
|
1413
|
+
import { CitationsManager } from 'llmz'
|
|
1414
|
+
|
|
1415
|
+
const citations = new CitationsManager()
|
|
1416
|
+
|
|
1417
|
+
// Register sources and get citation tags
|
|
1418
|
+
const source1 = citations.registerSource({
|
|
1419
|
+
file: 'document.pdf',
|
|
1420
|
+
page: 5,
|
|
1421
|
+
title: 'Company Policy',
|
|
1422
|
+
})
|
|
1423
|
+
|
|
1424
|
+
const source2 = citations.registerSource({
|
|
1425
|
+
url: 'https://example.com/article',
|
|
1426
|
+
title: 'Best Practices',
|
|
1427
|
+
})
|
|
1428
|
+
|
|
1429
|
+
console.log(source1.tag) // "【0】"
|
|
1430
|
+
console.log(source2.tag) // "【1】"
|
|
1431
|
+
|
|
1432
|
+
// Use tags in content
|
|
1433
|
+
const content = `The policy states employees must arrive on time${source1.tag}. However, best practices suggest flexibility${source2.tag}.`
|
|
1434
|
+
```
|
|
1435
|
+
|
|
1436
|
+
#### RAG Implementation Example
|
|
1437
|
+
|
|
1438
|
+
```typescript
|
|
1439
|
+
const ragTool = new Tool({
|
|
1440
|
+
name: 'search',
|
|
1441
|
+
description: 'Searches in the knowledge base for relevant information.',
|
|
1442
|
+
input: z.string().describe('The query to search in the knowledge base.'),
|
|
1443
|
+
async handler(query) {
|
|
1444
|
+
// Perform semantic search
|
|
1445
|
+
const { passages } = await client.searchFiles({
|
|
1446
|
+
query,
|
|
1447
|
+
limit: 20,
|
|
1448
|
+
contextDepth: 3,
|
|
1449
|
+
})
|
|
1450
|
+
|
|
1451
|
+
if (!passages.length) {
|
|
1452
|
+
throw new ThinkSignal(
|
|
1453
|
+
'No results found',
|
|
1454
|
+
'No results were found in the knowledge base. Try rephrasing your question.'
|
|
1455
|
+
)
|
|
1456
|
+
}
|
|
1457
|
+
|
|
1458
|
+
// Build response with citations
|
|
1459
|
+
let message: string[] = ['Here are the search results:']
|
|
1460
|
+
let { tag: example } = chat.citations.registerSource({}) // Example citation
|
|
1461
|
+
|
|
1462
|
+
// Register each passage as a source
|
|
1463
|
+
for (const passage of passages) {
|
|
1464
|
+
const { tag } = chat.citations.registerSource({
|
|
1465
|
+
file: passage.file.key,
|
|
1466
|
+
title: passage.file.tags.title,
|
|
1467
|
+
})
|
|
1468
|
+
|
|
1469
|
+
message.push(`<${tag} file="${passage.file.key}">`)
|
|
1470
|
+
message.push(`**${passage.file.tags.title}**`)
|
|
1471
|
+
message.push(passage.content)
|
|
1472
|
+
message.push(`</${tag}>`)
|
|
1473
|
+
}
|
|
1474
|
+
|
|
1475
|
+
// Provide context with citation instructions
|
|
1476
|
+
throw new ThinkSignal(
|
|
1477
|
+
`Got search results. When answering, you MUST add inline citations (eg: "The price is $10${example} ...")`,
|
|
1478
|
+
message.join('\n').trim()
|
|
1479
|
+
)
|
|
1480
|
+
},
|
|
1481
|
+
})
|
|
1482
|
+
```
|
|
1483
|
+
|
|
1484
|
+
#### Chat Integration
|
|
1485
|
+
|
|
1486
|
+
```typescript
|
|
1487
|
+
class CLIChat extends Chat {
|
|
1488
|
+
public citations: CitationsManager = new CitationsManager()
|
|
1489
|
+
|
|
1490
|
+
private async sendMessage(input: RenderedComponent) {
|
|
1491
|
+
if (input.type === 'Text') {
|
|
1492
|
+
let sources: string[] = []
|
|
1493
|
+
|
|
1494
|
+
// Extract citations and format them for display
|
|
1495
|
+
const { cleaned } = this.citations.extractCitations(input.text, (citation) => {
|
|
1496
|
+
let idx = chalk.bgGreenBright.black.bold(` ${sources.length + 1} `)
|
|
1497
|
+
sources.push(`${idx}: ${JSON.stringify(citation.source)}`)
|
|
1498
|
+
return `${idx}` // Replace 【0】 with [1]
|
|
1499
|
+
})
|
|
1500
|
+
|
|
1501
|
+
// Display cleaned text and sources
|
|
1502
|
+
console.log(`🤖 Agent: ${cleaned}`)
|
|
1503
|
+
|
|
1504
|
+
if (sources.length) {
|
|
1505
|
+
console.log(chalk.dim('Citations'))
|
|
1506
|
+
console.log(chalk.dim('========='))
|
|
1507
|
+
console.log(chalk.dim(sources.join('\n')))
|
|
1508
|
+
}
|
|
1509
|
+
}
|
|
1510
|
+
}
|
|
1511
|
+
}
|
|
1512
|
+
```
|
|
1513
|
+
|
|
1514
|
+
#### Advanced Citation Features
|
|
1515
|
+
|
|
1516
|
+
**Multiple Citation Support:**
|
|
1517
|
+
|
|
1518
|
+
```typescript
|
|
1519
|
+
// Agent can reference multiple sources in one citation
|
|
1520
|
+
const content = 'This fact is supported by multiple studies【0,1,3】'
|
|
1521
|
+
|
|
1522
|
+
const { cleaned, citations } = manager.extractCitations(content)
|
|
1523
|
+
// citations array contains entries for sources 0, 1, and 3
|
|
1524
|
+
```
|
|
1525
|
+
|
|
1526
|
+
**Object Citation Processing:**
|
|
1527
|
+
|
|
1528
|
+
```typescript
|
|
1529
|
+
// Remove citations from complex objects
|
|
1530
|
+
const dataWithCitations = {
|
|
1531
|
+
summary: 'The report shows positive trends【0】',
|
|
1532
|
+
details: {
|
|
1533
|
+
revenue: 'Increased by 15%【1】',
|
|
1534
|
+
costs: 'Reduced by 8%【2】',
|
|
1535
|
+
},
|
|
1536
|
+
}
|
|
1537
|
+
|
|
1538
|
+
const [cleanData, extractedCitations] = manager.removeCitationsFromObject(dataWithCitations)
|
|
1539
|
+
// cleanData has citations removed, extractedCitations contains path + citation info
|
|
1540
|
+
```
|
|
1541
|
+
|
|
1542
|
+
**Citation Stripping:**
|
|
1543
|
+
|
|
1544
|
+
```typescript
|
|
1545
|
+
// Remove all citation tags from content
|
|
1546
|
+
const textWithCitations = 'This statement【0】 has multiple【1,2】 citations.'
|
|
1547
|
+
const cleaned = CitationsManager.stripCitationTags(textWithCitations)
|
|
1548
|
+
// Result: "This statement has multiple citations."
|
|
1549
|
+
```
|
|
1550
|
+
|
|
1551
|
+
### Dynamic Context
|
|
1552
|
+
|
|
1553
|
+
LLMz supports dynamic evaluation of most parameters, allowing context-aware configuration:
|
|
1554
|
+
|
|
1555
|
+
```typescript
|
|
1556
|
+
await execute({
|
|
1557
|
+
// Dynamic instructions based on context
|
|
1558
|
+
instructions: (ctx) => {
|
|
1559
|
+
const timeOfDay = new Date().getHours()
|
|
1560
|
+
const greeting = timeOfDay < 12 ? 'Good morning' : 'Good afternoon'
|
|
1561
|
+
return `${greeting}! You are a helpful assistant with access to ${ctx.tools?.length || 0} tools.`
|
|
1562
|
+
},
|
|
1563
|
+
|
|
1564
|
+
// Dynamic tools based on user permissions
|
|
1565
|
+
tools: async (ctx) => {
|
|
1566
|
+
const userPermissions = await getUserPermissions(ctx.userId)
|
|
1567
|
+
return allTools.filter((tool) => userPermissions.includes(tool.permission))
|
|
1568
|
+
},
|
|
1569
|
+
|
|
1570
|
+
// Dynamic objects with current state
|
|
1571
|
+
objects: async (ctx) => {
|
|
1572
|
+
const userPreferences = await loadUserPreferences(ctx.userId)
|
|
1573
|
+
return [
|
|
1574
|
+
new ObjectInstance({
|
|
1575
|
+
name: 'user',
|
|
1576
|
+
properties: [{ name: 'preferences', value: userPreferences, writable: true }],
|
|
1577
|
+
}),
|
|
1578
|
+
]
|
|
1579
|
+
},
|
|
1580
|
+
|
|
1581
|
+
client,
|
|
1582
|
+
})
|
|
1583
|
+
```
|
|
1584
|
+
|
|
1585
|
+
---
|
|
1586
|
+
|
|
1587
|
+
## API Reference
|
|
1588
|
+
|
|
1589
|
+
### Core Functions
|
|
1590
|
+
|
|
1591
|
+
#### execute(props: ExecutionProps): Promise<ExecutionResult>
|
|
1592
|
+
|
|
1593
|
+
Main execution function that runs LLMz agents in either Chat Mode or Worker Mode.
|
|
1594
|
+
|
|
1595
|
+
**Parameters:**
|
|
1596
|
+
|
|
1597
|
+
- `props.client` - Botpress Client or Cognitive Client instance for LLM generation
|
|
1598
|
+
- `props.instructions` - System prompt/instructions for the LLM (static string or dynamic function)
|
|
1599
|
+
- `props.chat` - Optional Chat instance to enable Chat Mode with user interaction
|
|
1600
|
+
- `props.tools` - Array of Tool instances available to the agent (static or dynamic)
|
|
1601
|
+
- `props.objects` - Array of ObjectInstance for namespaced tools and variables (static or dynamic)
|
|
1602
|
+
- `props.exits` - Array of Exit definitions for structured completion (static or dynamic)
|
|
1603
|
+
- `props.snapshot` - Optional Snapshot to resume paused execution
|
|
1604
|
+
- `props.signal` - Optional AbortSignal to cancel execution
|
|
1605
|
+
- `props.options` - Optional execution options (loop limit, temperature, model, timeout)
|
|
1606
|
+
- `props.onTrace` - Optional non-blocking hook for monitoring traces during execution
|
|
1607
|
+
- `props.onIterationEnd` - Optional blocking hook called after each iteration
|
|
1608
|
+
- `props.onExit` - Optional blocking hook called when an exit is reached
|
|
1609
|
+
- `props.onBeforeExecution` - Optional blocking hook to modify code before VM execution
|
|
1610
|
+
- `props.onBeforeTool` - Optional blocking hook to modify tool inputs before execution
|
|
1611
|
+
- `props.onAfterTool` - Optional blocking hook to modify tool outputs after execution
|
|
1612
|
+
|
|
1613
|
+
**Returns:** `Promise<ExecutionResult>` - Result containing success/error/interrupted status with type-safe exit checking
|
|
1614
|
+
|
|
1615
|
+
### Tool Class
|
|
1616
|
+
|
|
1617
|
+
#### new Tool(config: ToolConfig)
|
|
1618
|
+
|
|
1619
|
+
Creates a new tool definition with type-safe schemas.
|
|
1620
|
+
|
|
1621
|
+
**Properties:**
|
|
1622
|
+
|
|
1623
|
+
- `name: string` - Tool name used in generated code
|
|
1624
|
+
- `description?: string` - Description for LLM understanding
|
|
1625
|
+
- `input?: ZuiSchema` - Input validation schema
|
|
1626
|
+
- `output?: ZuiSchema` - Output validation schema
|
|
1627
|
+
- `handler: (input: any) => Promise<any> | any` - Tool implementation
|
|
1628
|
+
- `aliases?: string[]` - Alternative names for the tool
|
|
1629
|
+
- `staticInputs?: Record<string, any>` - Force specific input values
|
|
1630
|
+
|
|
1631
|
+
**Methods:**
|
|
1632
|
+
|
|
1633
|
+
- `execute(input: any, context?: ToolContext): Promise<any>` - Execute the tool
|
|
1634
|
+
- `getTypings(): string` - Get TypeScript definitions for LLM
|
|
1635
|
+
- `clone(overrides: Partial<ToolConfig>): Tool` - Create a modified copy
|
|
1636
|
+
|
|
1637
|
+
### Exit Class
|
|
1638
|
+
|
|
1639
|
+
#### new Exit(config: ExitConfig)
|
|
1640
|
+
|
|
1641
|
+
Defines a structured exit point for agent execution.
|
|
1642
|
+
|
|
1643
|
+
**Properties:**
|
|
1644
|
+
|
|
1645
|
+
- `name: string` - Exit name used in generated code
|
|
1646
|
+
- `description?: string` - Description for LLM understanding
|
|
1647
|
+
- `schema?: ZuiSchema` - Output validation schema
|
|
1648
|
+
- `aliases?: string[]` - Alternative names for the exit
|
|
1649
|
+
|
|
1650
|
+
### ObjectInstance Class
|
|
1651
|
+
|
|
1652
|
+
#### new ObjectInstance(config: ObjectConfig)
|
|
1653
|
+
|
|
1654
|
+
Creates a namespaced container for tools and variables.
|
|
1655
|
+
|
|
1656
|
+
**Properties:**
|
|
1657
|
+
|
|
1658
|
+
- `name: string` - Object name used in generated code
|
|
1659
|
+
- `properties?: PropertyConfig[]` - Object properties/variables
|
|
1660
|
+
- `tools?: Tool[]` - Tools scoped to this object
|
|
1661
|
+
|
|
1662
|
+
**PropertyConfig:**
|
|
1663
|
+
|
|
1664
|
+
- `name: string` - Property name
|
|
1665
|
+
- `value: any` - Initial value
|
|
1666
|
+
- `writable: boolean` - Whether property can be modified
|
|
1667
|
+
- `type?: ZuiSchema` - Validation schema
|
|
1668
|
+
|
|
1669
|
+
### ExecutionResult Types
|
|
1670
|
+
|
|
1671
|
+
#### SuccessExecutionResult
|
|
1672
|
+
|
|
1673
|
+
**Properties:**
|
|
1674
|
+
|
|
1675
|
+
- `isSuccess(): boolean` - Type guard for success
|
|
1676
|
+
- `output: any` - The result data from the exit
|
|
1677
|
+
- `exit: Exit` - The exit that was used
|
|
1678
|
+
- `iteration: Iteration` - Final iteration details
|
|
1679
|
+
- `iterations: Iteration[]` - All iterations
|
|
1680
|
+
- `context: Context` - Execution context
|
|
1681
|
+
- `is(exit: Exit): boolean` - Type-safe exit checking
|
|
1682
|
+
|
|
1683
|
+
#### ErrorExecutionResult
|
|
1684
|
+
|
|
1685
|
+
**Properties:**
|
|
1686
|
+
|
|
1687
|
+
- `isError(): boolean` - Type guard for error
|
|
1688
|
+
- `error: Error | string` - The error that occurred
|
|
1689
|
+
- `iteration?: Iteration` - Failed iteration details
|
|
1690
|
+
- `iterations: Iteration[]` - All iterations before failure
|
|
1691
|
+
- `context: Context` - Execution context
|
|
1692
|
+
|
|
1693
|
+
#### PartialExecutionResult
|
|
1694
|
+
|
|
1695
|
+
**Properties:**
|
|
1696
|
+
|
|
1697
|
+
- `isInterrupted(): boolean` - Type guard for interruption
|
|
1698
|
+
- `signal: SnapshotSignal` - The signal that caused interruption
|
|
1699
|
+
- `snapshot: Snapshot` - Serializable execution state
|
|
1700
|
+
- `iterations: Iteration[]` - All iterations before interruption
|
|
1701
|
+
- `context: Context` - Execution context
|
|
1702
|
+
|
|
1703
|
+
### Chat Class
|
|
1704
|
+
|
|
1705
|
+
Abstract base class for implementing chat interfaces.
|
|
1706
|
+
|
|
1707
|
+
**Abstract Methods:**
|
|
1708
|
+
|
|
1709
|
+
- `getTranscript(): Promise<Transcript.Message[]> | Transcript.Message[]` - Get conversation history
|
|
1710
|
+
- `getComponents(): Promise<ComponentDefinition[]> | ComponentDefinition[]` - Get available UI components
|
|
1711
|
+
- `handler(component: RenderedComponent): Promise<void>` - Handle agent messages
|
|
1712
|
+
|
|
1713
|
+
### CitationsManager Class
|
|
1714
|
+
|
|
1715
|
+
Manages source citations for RAG systems.
|
|
1716
|
+
|
|
1717
|
+
**Methods:**
|
|
1718
|
+
|
|
1719
|
+
- `registerSource(source: any): { tag: string, id: number }` - Register a source and get citation tag
|
|
1720
|
+
- `extractCitations(text: string, replacer?: (citation) => string): { cleaned: string, citations: Citation[] }` - Extract and process citations
|
|
1721
|
+
- `removeCitationsFromObject(obj: any): [cleanedObj: any, citations: Citation[]]` - Remove citations from objects
|
|
1722
|
+
- `static stripCitationTags(text: string): string` - Remove all citation tags
|
|
1723
|
+
|
|
1724
|
+
### Snapshot Class
|
|
1725
|
+
|
|
1726
|
+
Manages pauseable execution state.
|
|
1727
|
+
|
|
1728
|
+
**Methods:**
|
|
1729
|
+
|
|
1730
|
+
- `toJSON(): string` - Serialize snapshot
|
|
1731
|
+
- `static fromJSON(json: string): Snapshot` - Deserialize snapshot
|
|
1732
|
+
- `resolve(data: any): void` - Resume with success
|
|
1733
|
+
- `reject(error: Error): void` - Resume with error
|
|
1734
|
+
|
|
1735
|
+
### Built-in Exits
|
|
1736
|
+
|
|
1737
|
+
- `ListenExit` - Automatically available in Chat Mode for user interaction
|
|
1738
|
+
- `DefaultExit` - Default exit for Worker Mode with success/failure discrimination
|
|
1739
|
+
- `ThinkExit` - Used when agent requests thinking time
|
|
1740
|
+
|
|
1741
|
+
### Signals
|
|
1742
|
+
|
|
1743
|
+
- `SnapshotSignal` - Thrown to pause execution for later resumption
|
|
1744
|
+
- `ThinkSignal` - Thrown to request agent reflection time
|
|
1745
|
+
- `LoopExceededError` - Thrown when maximum iterations reached
|
|
1746
|
+
|
|
1747
|
+
### Environment Variables
|
|
1748
|
+
|
|
1749
|
+
- `VM_DRIVER: 'isolated-vm' | 'node'` - Choose VM execution environment
|
|
1750
|
+
- `CI: boolean` - Automatically detected, affects VM driver selection
|
|
1751
|
+
|
|
1752
|
+
---
|
|
1753
|
+
|
|
1754
|
+
## Examples
|
|
1755
|
+
|
|
1756
|
+
The LLMz repository includes 20 comprehensive examples demonstrating different patterns and capabilities:
|
|
1757
|
+
|
|
1758
|
+
### Chat Examples (Interactive Patterns)
|
|
1759
|
+
|
|
1760
|
+
1. **01_chat_basic** - Basic conversational agent setup
|
|
1761
|
+
2. **02_chat_exits** - Custom exits for structured conversations
|
|
1762
|
+
3. **03_chat_conditional_tool** - Conditional tool usage based on context
|
|
1763
|
+
4. **04_chat_small_models** - Optimizations for smaller language models
|
|
1764
|
+
5. **05_chat_web_search** - Integration with web search capabilities
|
|
1765
|
+
6. **06_chat_confirm_tool** - User confirmation patterns for sensitive operations
|
|
1766
|
+
7. **07_chat_guardrails** - Safety mechanisms and content filtering
|
|
1767
|
+
8. **08_chat_multi_agent** - Multi-agent orchestration and delegation
|
|
1768
|
+
9. **09_chat_variables** - Object variables and state management
|
|
1769
|
+
10. **10_chat_components** - Rich UI components and interactive elements
|
|
1770
|
+
|
|
1771
|
+
### Worker Examples (Automated Patterns)
|
|
1772
|
+
|
|
1773
|
+
11. **11_worker_minimal** - Simplest worker mode execution
|
|
1774
|
+
12. **12_worker_fs** - File system operations and data processing
|
|
1775
|
+
13. **13_worker_sandbox** - Security isolation and sandboxing
|
|
1776
|
+
14. **14_worker_snapshot** - Pauseable execution and resumption
|
|
1777
|
+
15. **15_worker_stacktraces** - Error handling and debugging
|
|
1778
|
+
16. **16_worker_tool_chaining** - Complex multi-tool workflows
|
|
1779
|
+
17. **17_worker_error_recovery** - Graceful error recovery patterns
|
|
1780
|
+
18. **18_worker_security** - Security best practices and validation
|
|
1781
|
+
19. **19_worker_wrap_tool** - Tool modification and enhancement
|
|
1782
|
+
20. **20_chat_rag** - Retrieval-Augmented Generation with citations
|
|
1783
|
+
|
|
1784
|
+
### Running Examples
|
|
1785
|
+
|
|
1786
|
+
```bash
|
|
1787
|
+
# Install dependencies
|
|
1788
|
+
pnpm install
|
|
1789
|
+
|
|
1790
|
+
# Set up environment variables
|
|
1791
|
+
cp .env.example .env
|
|
1792
|
+
# Edit .env with your Botpress credentials
|
|
1793
|
+
|
|
1794
|
+
# Run a specific example
|
|
1795
|
+
pnpm start 01_chat_basic
|
|
1796
|
+
pnpm start chat_basic
|
|
1797
|
+
pnpm start 01
|
|
1798
|
+
|
|
1799
|
+
# List all available examples
|
|
1800
|
+
pnpm start
|
|
1801
|
+
```
|
|
1802
|
+
|
|
1803
|
+
### Example Environment Setup
|
|
1804
|
+
|
|
1805
|
+
Create a `.env` file in the examples directory:
|
|
1806
|
+
|
|
1807
|
+
```env
|
|
1808
|
+
BOTPRESS_BOT_ID=your_bot_id_here
|
|
1809
|
+
BOTPRESS_TOKEN=your_token_here
|
|
1810
|
+
```
|
|
1811
|
+
|
|
1812
|
+
### Key Learning Paths
|
|
1813
|
+
|
|
1814
|
+
**Getting Started:**
|
|
1815
|
+
|
|
1816
|
+
- Start with `01_chat_basic` and `11_worker_minimal`
|
|
1817
|
+
- Understand the difference between Chat and Worker modes
|
|
1818
|
+
- Learn basic tool integration patterns
|
|
1819
|
+
|
|
1820
|
+
**Intermediate Concepts:**
|
|
1821
|
+
|
|
1822
|
+
- Explore `09_chat_variables` for state management
|
|
1823
|
+
- Study `16_worker_tool_chaining` for complex workflows
|
|
1824
|
+
- Review `14_worker_snapshot` for pauseable execution
|
|
1825
|
+
|
|
1826
|
+
**Advanced Patterns:**
|
|
1827
|
+
|
|
1828
|
+
- Examine `08_chat_multi_agent` for orchestration
|
|
1829
|
+
- Learn from `20_chat_rag` for knowledge integration
|
|
1830
|
+
- Study `18_worker_security` for production deployment
|
|
1831
|
+
|
|
1832
|
+
Each example includes detailed comments explaining the concepts and implementation patterns, making them excellent learning resources for understanding LLMz capabilities.
|
|
1833
|
+
|
|
1834
|
+
---
|
|
1835
|
+
|
|
1836
|
+
_This documentation covers the complete LLMz framework. For the latest updates and community contributions, visit the [LLMz repository](https://github.com/botpress/llmz)._
|