llmz 0.0.18 → 0.0.19

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/DOCS.md ADDED
@@ -0,0 +1,1836 @@
1
+ # LLMz Documentation
2
+
3
+ **LLMz: A Revolutionary TypeScript AI Agent Framework**
4
+
5
+ _Stop chaining tools. Start generating real code._
6
+
7
+ ---
8
+
9
+ ## Table of Contents
10
+
11
+ 1. [Introduction](#introduction)
12
+ 2. [Core Philosophy](#core-philosophy)
13
+ 3. [Quick Start](#quick-start)
14
+ 4. [Core Concepts](#core-concepts)
15
+ 5. [Execution Modes](#execution-modes)
16
+ 6. [Tools](#tools)
17
+ 7. [Objects and Variables](#objects-and-variables)
18
+ 8. [Execution Results](#execution-results)
19
+ 9. [Hooks System](#hooks-system)
20
+ 10. [Advanced Features](#advanced-features)
21
+ 11. [API Reference](#api-reference)
22
+ 12. [Examples](#examples)
23
+
24
+ ---
25
+
26
+ ## Introduction
27
+
28
+ LLMz is a revolutionary TypeScript AI agent framework that fundamentally changes how AI agents work. Like other agent frameworks, LLMz calls LLM models in a loop to achieve desired outcomes with access to tools and memory. However, LLMz is **code-first** – meaning it generates and runs TypeScript code in a sandbox rather than using traditional JSON tool calling.
29
+
30
+ ### What Makes LLMz Different
31
+
32
+ Traditional agent frameworks rely on JSON tool calling, which has significant limitations:
33
+
34
+ - **Hard-to-parse JSON schemas** for LLMs
35
+ - **Incapable of complex logic** like loops and conditionals
36
+ - **Multiple expensive roundtrips** for each tool call
37
+ - **Unreliable beyond simple scenarios**
38
+
39
+ LLMz leverages the fact that models have been trained extensively on millions of TypeScript codebases, making them incredibly reliable at generating working code. This enables:
40
+
41
+ - **Complex logic and multi-tool orchestration** in **one call**
42
+ - **Native LLM thinking** via comments and code structure
43
+ - **Complete type safety** and predictable schemas
44
+ - **Seamless scaling** in production environments
45
+
46
+ ### Battle-Tested at Scale
47
+
48
+ LLMz operates as an LLM-native TypeScript VM built on top of Zui (Botpress's internal schema library), battle-tested in production powering millions of AI agents worldwide.
49
+
50
+ ---
51
+
52
+ ## Core Philosophy
53
+
54
+ ### Code Generation > Tool Calling
55
+
56
+ Traditional tool-calling agents are fundamentally limited by the JSON interface between the LLM and tools. This requires multiple roundtrips for complex tasks and cannot handle conditional logic, loops, or sophisticated data processing.
57
+
58
+ LLMz solves this by letting LLMs do what they do best: **generate code**. Since models are trained extensively on code, they can reliably generate TypeScript that:
59
+
60
+ - Calls multiple tools in sequence
61
+ - Handles conditional logic and error cases
62
+ - Processes and transforms data between tool calls
63
+ - Implements complex business logic
64
+ - Maintains type safety throughout execution
65
+
66
+ ### Example: Traditional vs LLMz
67
+
68
+ **Traditional Tool Calling:**
69
+
70
+ ```
71
+ LLM → JSON: {"tool": "getPrice", "params": {"from": "quebec", "to": "new york"}}
72
+ System → Response: {"price": 600}
73
+ LLM → JSON: {"tool": "checkBudget", "params": {"amount": 600}}
74
+ System → Response: {"canAfford": false}
75
+ LLM → JSON: {"tool": "notifyUser", "params": {"message": "Price too high"}}
76
+ ```
77
+
78
+ **LLMz Code Generation:**
79
+
80
+ ```typescript
81
+ // Check the ticket price and user's budget in one go
82
+ const price = await getTicketPrice({ from: 'quebec', to: 'new york' })
83
+ const budget = await getUserBudget()
84
+
85
+ if (price > budget) {
86
+ await notifyUser({ message: `Price $${price} exceeds budget $${budget}` })
87
+ return { action: 'budget_exceeded', price, budget }
88
+ } else {
89
+ const ticketId = await buyTicket({ from: 'quebec', to: 'new york' })
90
+ return { action: 'done', result: ticketId }
91
+ }
92
+ ```
93
+
94
+ ---
95
+
96
+ ## Quick Start
97
+
98
+ ### Installation
99
+
100
+ ```bash
101
+ npm install llmz @botpress/client
102
+ ```
103
+
104
+ ### Basic Example (Worker Mode)
105
+
106
+ ```typescript
107
+ import { execute } from 'llmz'
108
+ import { Client } from '@botpress/client'
109
+
110
+ const client = new Client({
111
+ botId: process.env.BOTPRESS_BOT_ID!,
112
+ token: process.env.BOTPRESS_TOKEN!,
113
+ })
114
+
115
+ const result = await execute({
116
+ instructions: 'Calculate the sum of numbers 1 to 100',
117
+ client,
118
+ })
119
+
120
+ if (result.isSuccess()) {
121
+ console.log('Result:', result.output)
122
+ console.log('Generated code:', result.iteration.code)
123
+ }
124
+ ```
125
+
126
+ ### Basic Example (Chat Mode)
127
+
128
+ ```typescript
129
+ import { execute } from 'llmz'
130
+ import { Client } from '@botpress/client'
131
+ import { CLIChat } from './utils/cli-chat'
132
+
133
+ const client = new Client({
134
+ botId: process.env.BOTPRESS_BOT_ID!,
135
+ token: process.env.BOTPRESS_TOKEN!,
136
+ })
137
+
138
+ const chat = new CLIChat()
139
+
140
+ while (await chat.iterate()) {
141
+ await execute({
142
+ instructions: 'You are a helpful assistant',
143
+ chat,
144
+ client,
145
+ })
146
+ }
147
+ ```
148
+
149
+ ---
150
+
151
+ ## Core Concepts
152
+
153
+ ### Execution Loop
154
+
155
+ At its core, LLMz exposes a single method (`execute`) that runs in a loop until one of these conditions is met:
156
+
157
+ 1. **An Exit is returned** - Agent completes with structured result
158
+ 2. **Agent waits for user input** (Chat Mode) - Returns control to user
159
+ 3. **Maximum iterations reached** - Safety limit to prevent infinite loops
160
+
161
+ The loop automatically handles:
162
+
163
+ - Tool calling and result processing
164
+ - Thinking about outputs and context
165
+ - Error recovery and retry logic
166
+ - Variable state persistence across iterations
167
+
168
+ ### Generated Code Structure
169
+
170
+ Every LLMz code block follows a predictable structure that LLMs can reliably generate:
171
+
172
+ #### Return Statement (Required)
173
+
174
+ At minimum, an LLMz response must contain a return statement with an Exit:
175
+
176
+ ```typescript
177
+ // Chat mode - give turn back to user
178
+ return { action: 'listen' }
179
+
180
+ // Worker mode - complete with result
181
+ return { action: 'done', result: calculatedValue }
182
+ ```
183
+
184
+ #### Tool Calls with Logic
185
+
186
+ Unlike traditional tool calling, LLMz enables complex logic impossible with JSON:
187
+
188
+ ```typescript
189
+ // Complex conditional logic and error handling
190
+ const price = await getTicketPrice({ from: 'quebec', to: 'new york' })
191
+
192
+ if (price > 500) {
193
+ throw new Error('Price too high')
194
+ } else {
195
+ const ticketId = await buyTicket({ from: 'quebec', to: 'new york' })
196
+ return { action: 'done', result: ticketId }
197
+ }
198
+ ```
199
+
200
+ #### Comments for Planning
201
+
202
+ Comments help LLMs think step-by-step and plan ahead:
203
+
204
+ ```typescript
205
+ // Check user's budget first before proceeding with purchase
206
+ const budget = await getUserBudget()
207
+
208
+ // Only proceed if we have enough funds
209
+ if (budget >= price) {
210
+ // Purchase the ticket
211
+ const ticket = await buyTicket(ticketDetails)
212
+ }
213
+ ```
214
+
215
+ #### React Components (Chat Mode Only)
216
+
217
+ In Chat Mode, agents can yield React components for rich user interaction:
218
+
219
+ ```typescript
220
+ // Multi-line text support
221
+ yield <Text>
222
+ Hello, world!
223
+ This is a second line.
224
+ </Text>
225
+
226
+ // Composed/nested components
227
+ yield <Message>
228
+ <Text>What do you prefer?</Text>
229
+ <Button>Cats</Button>
230
+ <Button>Dogs</Button>
231
+ </Message>
232
+
233
+ return { action: 'listen' }
234
+ ```
235
+
236
+ ### Compilation Pipeline
237
+
238
+ LLMz uses a sophisticated Babel-based compilation system to transform generated code:
239
+
240
+ 1. **AST Parsing**: TypeScript/JSX code parsed into Abstract Syntax Tree
241
+ 2. **Plugin Transformation**: Custom plugins modify the AST for execution
242
+ 3. **Code Generation**: Modified AST compiled back to executable JavaScript
243
+ 4. **Source Maps**: Generated for debugging and error tracking
244
+
245
+ Key transformations include:
246
+
247
+ - Tool call instrumentation for monitoring
248
+ - Variable extraction and tracking
249
+ - JSX component handling
250
+ - Line number preservation for stack traces
251
+
252
+ ### Virtual Machine Execution
253
+
254
+ LLMz supports multiple execution environments:
255
+
256
+ - **Production**: Uses `isolated-vm` for security isolation
257
+ - **CI/Development**: Falls back to Node.js VM for compatibility
258
+ - **Browser**: Uses standard JavaScript execution
259
+
260
+ The VM provides:
261
+
262
+ - Memory isolation and limits
263
+ - Execution timeouts
264
+ - Secure context separation
265
+ - Stack trace sanitization
266
+
267
+ ---
268
+
269
+ ## Execution Modes
270
+
271
+ LLMz operates in two distinct modes depending on whether a chat interface is provided:
272
+
273
+ ### Chat Mode
274
+
275
+ **Enabled when**: `chat` parameter is provided to `execute()`
276
+
277
+ Chat Mode is designed for interactive conversational agents that need to:
278
+
279
+ - Maintain conversation history
280
+ - Respond to user messages
281
+ - Yield UI components for rich interaction
282
+ - Handle turn-taking between agent and user
283
+
284
+ Key characteristics:
285
+
286
+ - Agent can yield React components to user
287
+ - Special `ListenExit` automatically available
288
+ - Transcript management for conversation history
289
+ - Turn-based execution flow
290
+
291
+ ```typescript
292
+ const result = await execute({
293
+ instructions: 'You are a helpful assistant',
294
+ chat: myChatInstance,
295
+ tools: [searchTool, calculatorTool],
296
+ client,
297
+ })
298
+
299
+ if (result.is(ListenExit)) {
300
+ // Agent is waiting for user input
301
+ }
302
+ ```
303
+
304
+ ### Worker Mode
305
+
306
+ **Enabled when**: `chat` parameter is omitted from `execute()`
307
+
308
+ Worker Mode is designed for automated execution environments that need to:
309
+
310
+ - Process data and perform computations
311
+ - Execute multi-step workflows
312
+ - Return structured results
313
+ - Run without human interaction
314
+
315
+ Key characteristics:
316
+
317
+ - Focus on computational tasks and data processing
318
+ - Uses `DefaultExit` if no custom exits provided
319
+ - Sandboxed execution with security isolation
320
+ - Automated completion without user interaction
321
+
322
+ ```typescript
323
+ const result = await execute({
324
+ instructions: 'Process the customer data and generate insights',
325
+ tools: [dataProcessorTool, analyticseTool],
326
+ exits: [dataProcessedExit],
327
+ client,
328
+ })
329
+
330
+ if (result.is(dataProcessedExit)) {
331
+ console.log('Analysis complete:', result.output)
332
+ }
333
+ ```
334
+
335
+ ### Mode Comparison
336
+
337
+ | Feature | Chat Mode | Worker Mode |
338
+ | -------------------- | ------------------- | --------------- |
339
+ | User Interaction | ✅ Interactive | ❌ Automated |
340
+ | UI Components | ✅ React components | ❌ No UI |
341
+ | Conversation History | ✅ Full transcript | ❌ No history |
342
+ | Default Exits | `ListenExit` | `DefaultExit` |
343
+ | Primary Use Case | Conversational AI | Data processing |
344
+ | Execution Pattern | Turn-based | Continuous |
345
+
346
+ ---
347
+
348
+ ## Tools
349
+
350
+ Tools are the primary way to extend LLMz agents with external capabilities. Unlike traditional agent frameworks, LLMz tools are called through generated TypeScript code, enabling complex orchestration and error handling.
351
+
352
+ ### Tool Definition
353
+
354
+ Tools are defined using Zui schemas for complete type safety:
355
+
356
+ ```typescript
357
+ import { Tool } from 'llmz'
358
+ import { z } from '@bpinternal/zui'
359
+
360
+ const weatherTool = new Tool({
361
+ name: 'getWeather',
362
+ description: 'Get current weather for a location',
363
+ input: z.object({
364
+ location: z.string().describe('City name or coordinates'),
365
+ units: z.enum(['celsius', 'fahrenheit']).optional().default('celsius'),
366
+ }),
367
+ output: z.object({
368
+ temperature: z.number(),
369
+ conditions: z.string(),
370
+ humidity: z.number(),
371
+ }),
372
+ handler: async ({ location, units }) => {
373
+ // Implementation here
374
+ return {
375
+ temperature: 22,
376
+ conditions: 'sunny',
377
+ humidity: 65,
378
+ }
379
+ },
380
+ })
381
+ ```
382
+
383
+ ### Tool Usage in Generated Code
384
+
385
+ The LLM generates TypeScript code that calls tools naturally:
386
+
387
+ ```typescript
388
+ // Simple tool call
389
+ const weather = await getWeather({ location: 'New York' })
390
+
391
+ // Complex logic with multiple tools
392
+ const weather = await getWeather({ location: userLocation })
393
+ if (weather.temperature < 0) {
394
+ const clothing = await getSuggestions({ type: 'winter', temperature: weather.temperature })
395
+ yield <Text>It's {weather.temperature}°C! {clothing.suggestion}</Text>
396
+ } else {
397
+ yield <Text>Nice weather! {weather.conditions} at {weather.temperature}°C</Text>
398
+ }
399
+
400
+ return { action: 'listen' }
401
+ ```
402
+
403
+ ### Advanced Tool Features
404
+
405
+ #### Tool Aliases
406
+
407
+ Tools can have multiple names for flexible calling:
408
+
409
+ ```typescript
410
+ const tool = new Tool({
411
+ name: 'calculatePrice',
412
+ aliases: ['getPrice', 'checkCost'],
413
+ // ... rest of definition
414
+ })
415
+
416
+ // All of these work in generated code:
417
+ // await calculatePrice(params)
418
+ // await getPrice(params)
419
+ // await checkCost(params)
420
+ ```
421
+
422
+ #### Static Inputs
423
+
424
+ Force specific inputs to be always included:
425
+
426
+ ```typescript
427
+ const tool = new Tool({
428
+ name: 'logEvent',
429
+ input: z.object({
430
+ event: z.string(),
431
+ userId: z.string(),
432
+ timestamp: z.number(),
433
+ }),
434
+ staticInputs: {
435
+ userId: 'user-123',
436
+ timestamp: () => Date.now(), // Dynamic static input
437
+ },
438
+ handler: async ({ event, userId, timestamp }) => {
439
+ // userId and timestamp are automatically provided
440
+ },
441
+ })
442
+ ```
443
+
444
+ #### Tool Wrapping and Cloning
445
+
446
+ Clone and modify existing tools:
447
+
448
+ ```typescript
449
+ const originalTool = new Tool({
450
+ /* definition */
451
+ })
452
+
453
+ const wrappedTool = originalTool.clone({
454
+ name: 'wrappedVersion',
455
+ description: 'Enhanced version with logging',
456
+ handler: async (input) => {
457
+ console.log('Tool called with:', input)
458
+ const result = await originalTool.execute(input)
459
+ console.log('Tool returned:', result)
460
+ return result
461
+ },
462
+ })
463
+ ```
464
+
465
+ ### Tool Type Generation
466
+
467
+ Use `tool.getTypings()` to see the TypeScript definitions generated for the LLM:
468
+
469
+ ```typescript
470
+ console.log(weatherTool.getTypings())
471
+ // Output:
472
+ // /**
473
+ // * Get current weather for a location
474
+ // */
475
+ // declare function getWeather(input: {
476
+ // location: string; // City name or coordinates
477
+ // units?: "celsius" | "fahrenheit";
478
+ // }): Promise<{
479
+ // temperature: number;
480
+ // conditions: string;
481
+ // humidity: number;
482
+ // }>;
483
+ ```
484
+
485
+ ### Best Practices
486
+
487
+ 1. **Descriptive Schemas**: Detailed descriptions help LLMs generate better code
488
+ 2. **Type Safety**: Use strict Zui schemas for predictable behavior
489
+ 3. **Error Handling**: Tools should handle errors gracefully
490
+ 4. **Performance**: Keep tool execution fast to avoid timeouts
491
+ 5. **Documentation**: Clear descriptions improve code generation quality
492
+
493
+ ---
494
+
495
+ ## Objects and Variables
496
+
497
+ Objects in LLMz provide namespaced containers for related tools and variables, enabling sophisticated state management and data organization.
498
+
499
+ ### Object Definition
500
+
501
+ Objects group related functionality and provide scoped variables:
502
+
503
+ ```typescript
504
+ import { ObjectInstance } from 'llmz'
505
+ import { z } from '@bpinternal/zui'
506
+
507
+ const userObject = new ObjectInstance({
508
+ name: 'user',
509
+ properties: [
510
+ {
511
+ name: 'name',
512
+ value: 'John Doe',
513
+ writable: true,
514
+ type: z.string(),
515
+ },
516
+ {
517
+ name: 'age',
518
+ value: 30,
519
+ writable: false, // Read-only
520
+ type: z.number(),
521
+ },
522
+ {
523
+ name: 'preferences',
524
+ value: { theme: 'dark', language: 'en' },
525
+ writable: true,
526
+ type: z.object({
527
+ theme: z.enum(['light', 'dark']),
528
+ language: z.string(),
529
+ }),
530
+ },
531
+ ],
532
+ tools: [
533
+ new Tool({
534
+ name: 'updateProfile',
535
+ input: z.object({ name: z.string() }),
536
+ handler: async ({ name }) => {
537
+ // This tool is scoped to the user object
538
+ return { success: true }
539
+ },
540
+ }),
541
+ ],
542
+ })
543
+ ```
544
+
545
+ ### Variables in Generated Code
546
+
547
+ The LLM can read and write object properties in generated code:
548
+
549
+ ```typescript
550
+ // Reading variables
551
+ const userName = user.name // "John Doe"
552
+ const userAge = user.age // 30
553
+
554
+ // Writing to writable variables
555
+ user.name = 'Jane Smith' // ✅ Succeeds
556
+ user.preferences = { theme: 'light', language: 'es' } // ✅ Succeeds
557
+
558
+ // Attempting to write read-only variables
559
+ user.age = 25 // ❌ Throws AssignmentError
560
+ ```
561
+
562
+ ### Type Safety and Validation
563
+
564
+ Variables are validated against their schemas:
565
+
566
+ ```typescript
567
+ // Valid assignment
568
+ user.preferences = { theme: 'dark', language: 'fr' } // ✅
569
+
570
+ // Invalid assignment - wrong type
571
+ user.preferences = { theme: 'blue', language: 'fr' } // ❌ Throws validation error
572
+
573
+ // Invalid assignment - missing required fields
574
+ user.preferences = { theme: 'dark' } // ❌ Missing language field
575
+ ```
576
+
577
+ ### Mutation Tracking
578
+
579
+ LLMz automatically tracks changes to object properties:
580
+
581
+ ```typescript
582
+ // In generated code
583
+ user.name = 'Updated Name'
584
+ user.preferences.theme = 'light'
585
+
586
+ // After execution, mutations are available
587
+ console.log(result.iteration.mutations)
588
+ // [
589
+ // {
590
+ // object: 'user',
591
+ // property: 'name',
592
+ // before: 'John Doe',
593
+ // after: 'Updated Name'
594
+ // },
595
+ // {
596
+ // object: 'user',
597
+ // property: 'preferences',
598
+ // before: { theme: 'dark', language: 'en' },
599
+ // after: { theme: 'light', language: 'en' }
600
+ // }
601
+ // ]
602
+ ```
603
+
604
+ ### Namespaced Tools
605
+
606
+ Tools within objects are called with object namespace:
607
+
608
+ ```typescript
609
+ // Tool is scoped to the user object
610
+ await user.updateProfile({ name: 'New Name' })
611
+
612
+ // This automatically updates the user object's properties
613
+ // and is tracked as a mutation
614
+ ```
615
+
616
+ ### Object Sealing and Protection
617
+
618
+ Objects are automatically sealed to prevent unauthorized modifications:
619
+
620
+ ```typescript
621
+ // In generated code - these will throw errors
622
+ user.newProperty = 'value' // ❌ Cannot add new properties
623
+ delete user.name // ❌ Cannot delete properties
624
+
625
+ // Only predefined properties can be modified (if writable)
626
+ user.name = 'New Name' // ✅ Allowed if writable: true
627
+ ```
628
+
629
+ ### Variable Persistence
630
+
631
+ Variables persist across iterations and thinking cycles:
632
+
633
+ ```typescript
634
+ // Iteration 1: Set a variable
635
+ user.preferences = { theme: 'dark', language: 'es' }
636
+ return { action: 'think' } // Trigger thinking
637
+
638
+ // Iteration 2: Variable is still available
639
+ const currentTheme = user.preferences.theme // 'dark'
640
+ ```
641
+
642
+ ### Best Practices
643
+
644
+ 1. **Meaningful Names**: Use descriptive object and property names
645
+ 2. **Appropriate Scope**: Group related functionality together
646
+ 3. **Write Protection**: Mark properties as read-only when appropriate
647
+ 4. **Type Safety**: Use strict schemas for predictable behavior
648
+ 5. **Mutation Tracking**: Leverage mutation tracking for audit trails
649
+
650
+ ---
651
+
652
+ ## Execution Results
653
+
654
+ Every call to `execute()` returns an `ExecutionResult` that provides type-safe access to the execution outcome. LLMz execution can result in three different types of results.
655
+
656
+ ### Result Types
657
+
658
+ #### SuccessExecutionResult
659
+
660
+ Agent completed successfully with an Exit. Contains the structured data produced by the agent.
661
+
662
+ ```typescript
663
+ const result = await execute({
664
+ instructions: 'Calculate the sum',
665
+ client,
666
+ })
667
+
668
+ if (result.isSuccess()) {
669
+ console.log('Output:', result.output)
670
+ console.log('Exit used:', result.exit.name)
671
+ console.log('Generated code:', result.iteration.code)
672
+ }
673
+ ```
674
+
675
+ #### ErrorExecutionResult
676
+
677
+ Execution failed with an unrecoverable error:
678
+
679
+ ```typescript
680
+ if (result.isError()) {
681
+ console.error('Error:', result.error)
682
+ console.error('Failed iteration:', result.iteration?.error)
683
+
684
+ // Analyze failure progression
685
+ result.iterations.forEach((iter, i) => {
686
+ console.log(`Iteration ${i + 1}: ${iter.status.type}`)
687
+ })
688
+ }
689
+ ```
690
+
691
+ #### PartialExecutionResult
692
+
693
+ Execution was interrupted by a SnapshotSignal for pauseable operations:
694
+
695
+ ```typescript
696
+ if (result.isInterrupted()) {
697
+ console.log('Interrupted:', result.signal.message)
698
+
699
+ // Save snapshot for later resumption
700
+ const serialized = result.snapshot.toJSON()
701
+ await database.saveSnapshot(serialized)
702
+ }
703
+ ```
704
+
705
+ ### Type-Safe Exit Checking
706
+
707
+ Use `result.is(exit)` for type-safe access to specific exit data:
708
+
709
+ ```typescript
710
+ const successExit = new Exit({
711
+ name: 'success',
712
+ schema: z.object({
713
+ recordsProcessed: z.number(),
714
+ processingTime: z.number(),
715
+ }),
716
+ })
717
+
718
+ const errorExit = new Exit({
719
+ name: 'error',
720
+ schema: z.object({
721
+ errorCode: z.string(),
722
+ details: z.string(),
723
+ }),
724
+ })
725
+
726
+ const result = await execute({
727
+ instructions: 'Process the data',
728
+ exits: [successExit, errorExit],
729
+ client,
730
+ })
731
+
732
+ // Type-safe exit handling with automatic output typing
733
+ if (result.is(successExit)) {
734
+ // TypeScript knows result.output has the success schema
735
+ console.log(`Processed ${result.output.recordsProcessed} records`)
736
+ console.log(`Processing took ${result.output.processingTime}ms`)
737
+ } else if (result.is(errorExit)) {
738
+ // TypeScript knows result.output has the error schema
739
+ console.error(`Error ${result.output.errorCode}: ${result.output.details}`)
740
+ }
741
+ ```
742
+
743
+ ### Built-in Exits
744
+
745
+ ```typescript
746
+ import { ListenExit, DefaultExit, ThinkExit } from 'llmz'
747
+
748
+ // Check for built-in exits
749
+ if (result.is(ListenExit)) {
750
+ console.log('Agent is waiting for user input')
751
+ }
752
+
753
+ if (result.is(DefaultExit)) {
754
+ // DefaultExit has success/failure discriminated union
755
+ if (result.output.success) {
756
+ console.log('Completed successfully:', result.output.result)
757
+ } else {
758
+ console.error('Completed with error:', result.output.error)
759
+ }
760
+ }
761
+
762
+ if (result.is(ThinkExit)) {
763
+ console.log('Agent requested thinking time')
764
+ console.log('Current variables:', result.output.variables)
765
+ }
766
+ ```
767
+
768
+ ### Accessing Execution Details
769
+
770
+ #### Iterations and Execution Flow
771
+
772
+ ```typescript
773
+ // Access the final iteration
774
+ const lastIteration = result.iteration
775
+ if (lastIteration) {
776
+ console.log('Generated code:', lastIteration.code)
777
+ console.log('Status:', lastIteration.status.type)
778
+ console.log('Duration:', lastIteration.duration)
779
+ }
780
+
781
+ // Access all iterations to see full execution flow
782
+ result.iterations.forEach((iteration, index) => {
783
+ console.log(`Iteration ${index + 1}:`)
784
+ console.log(' Status:', iteration.status.type)
785
+ console.log(' Code length:', iteration.code?.length || 0)
786
+ console.log(' Variables:', Object.keys(iteration.variables).length)
787
+ })
788
+ ```
789
+
790
+ #### Variables and Declarations
791
+
792
+ ```typescript
793
+ // If agent generates: const hello = '1234'
794
+ const lastIteration = result.iteration
795
+ if (lastIteration) {
796
+ console.log(lastIteration.variables.hello) // '1234'
797
+
798
+ // Access all variables from the final iteration
799
+ Object.entries(lastIteration.variables).forEach(([name, value]) => {
800
+ console.log(`Variable ${name}:`, value)
801
+ })
802
+ }
803
+ ```
804
+
805
+ #### Tool Calls and Traces
806
+
807
+ ```typescript
808
+ // Access tool calls from all iterations
809
+ const allToolCalls = result.iterations.flatMap((iter) => iter.traces.filter((trace) => trace.type === 'tool_call'))
810
+
811
+ console.log('Total tool calls:', allToolCalls.length)
812
+
813
+ // Access other trace types
814
+ const lastIteration = result.iteration
815
+ if (lastIteration) {
816
+ const yields = lastIteration.traces.filter((trace) => trace.type === 'yield')
817
+ const comments = lastIteration.traces.filter((trace) => trace.type === 'comment')
818
+ const propertyAccess = lastIteration.traces.filter((trace) => trace.type === 'property')
819
+ }
820
+ ```
821
+
822
+ #### Context and Metadata
823
+
824
+ ```typescript
825
+ if (result.isSuccess()) {
826
+ // Access original execution parameters
827
+ console.log('Instructions:', result.context.instructions)
828
+ console.log('Loop limit:', result.context.loop)
829
+ console.log('Temperature:', result.context.temperature)
830
+ console.log('Model:', result.context.model)
831
+
832
+ // Access tools and exits that were available
833
+ console.log(
834
+ 'Available tools:',
835
+ result.context.tools?.map((t) => t.name)
836
+ )
837
+ console.log(
838
+ 'Available exits:',
839
+ result.context.exits?.map((e) => e.name)
840
+ )
841
+ }
842
+ ```
843
+
844
+ ### Error Analysis
845
+
846
+ ```typescript
847
+ if (result.isError()) {
848
+ console.error('Execution failed:', result.error)
849
+
850
+ // Analyze the failure progression
851
+ const failedIteration = result.iteration
852
+ if (failedIteration) {
853
+ switch (failedIteration.status.type) {
854
+ case 'execution_error':
855
+ console.error('Code execution failed:', failedIteration.status.execution_error.message)
856
+ console.error('Stack trace:', failedIteration.status.execution_error.stack)
857
+ console.error('Failed code:', failedIteration.code)
858
+ break
859
+
860
+ case 'generation_error':
861
+ console.error('LLM generation failed:', failedIteration.status.generation_error.message)
862
+ break
863
+
864
+ case 'invalid_code_error':
865
+ console.error('Invalid code generated:', failedIteration.status.invalid_code_error.message)
866
+ console.error('Invalid code:', failedIteration.code)
867
+ break
868
+
869
+ case 'aborted':
870
+ console.error('Execution aborted:', failedIteration.status.aborted.reason)
871
+ break
872
+ }
873
+ }
874
+
875
+ // Review all iterations to understand failure progression
876
+ console.log('Iterations before failure:', result.iterations.length)
877
+ result.iterations.forEach((iter, i) => {
878
+ console.log(`Iteration ${i + 1}: ${iter.status.type}`)
879
+ })
880
+ }
881
+ ```
882
+
883
+ ### Snapshot Handling
884
+
885
+ Handle interrupted executions with snapshot resumption:
886
+
887
+ ```typescript
888
+ const result = await execute({
889
+ instructions: 'Process large dataset with pauseable operation',
890
+ tools: [snapshotCapableTool],
891
+ client,
892
+ })
893
+
894
+ if (result.isInterrupted()) {
895
+ console.log('Execution paused:', result.signal.message)
896
+
897
+ // Serialize snapshot for persistence
898
+ const serialized = result.snapshot.toJSON()
899
+ await database.saveSnapshot('execution-123', serialized)
900
+
901
+ // Later, resume from snapshot
902
+ const snapshot = Snapshot.fromJSON(serialized)
903
+ snapshot.resolve({ resumeData: 'Operation completed' })
904
+
905
+ const continuation = await execute({
906
+ snapshot,
907
+ instructions: result.context.instructions,
908
+ tools: result.context.tools,
909
+ exits: result.context.exits,
910
+ client,
911
+ })
912
+
913
+ if (continuation.isSuccess()) {
914
+ console.log('Resumed execution completed:', continuation.output)
915
+ }
916
+ }
917
+ ```
918
+
919
+ ---
920
+
921
+ ## Hooks System
922
+
923
+ LLMz provides a comprehensive hook system that allows you to inject custom logic at various points during execution. Hooks are categorized as either blocking (execution waits) or non-blocking, and either mutation (can modify data) or non-mutation.
924
+
925
+ ### Hook Types Overview
926
+
927
+ | Hook | Blocking | Mutation | Called When |
928
+ | ------------------- | -------- | -------- | -------------------------- |
929
+ | `onTrace` | ❌ | ❌ | Each trace generated |
930
+ | `onIterationEnd` | ✅ | ❌ | After iteration completion |
931
+ | `onExit` | ✅ | ❌ | When exit is reached |
932
+ | `onBeforeExecution` | ✅ | ✅ | Before code execution |
933
+ | `onBeforeTool` | ✅ | ✅ | Before tool execution |
934
+ | `onAfterTool` | ✅ | ✅ | After tool execution |
935
+
936
+ ### onTrace (Non-blocking, Non-mutation)
937
+
938
+ Called for each trace generated during iteration. Useful for logging, debugging, or monitoring execution progress.
939
+
940
+ ```typescript
941
+ await execute({
942
+ onTrace: ({ trace, iteration }) => {
943
+ console.log(`Iteration ${iteration}: ${trace.type}`, trace)
944
+
945
+ // Log specific trace types
946
+ if (trace.type === 'tool_call') {
947
+ console.log(`Tool ${trace.tool_name} called with:`, trace.input)
948
+ }
949
+ },
950
+ // ... other props
951
+ })
952
+ ```
953
+
954
+ **Available Trace Types:**
955
+
956
+ - `abort_signal`: Abort signal received
957
+ - `comment`: Comment found in generated code
958
+ - `llm_call_success`: LLM generation completed successfully
959
+ - `property`: Object property accessed or modified
960
+ - `think_signal`: ThinkSignal thrown
961
+ - `tool_call`: Tool executed
962
+ - `yield`: Component yielded in chat mode
963
+ - `log`: General logging event
964
+
965
+ ### onIterationEnd (Blocking, Non-mutation)
966
+
967
+ Called after each iteration ends, regardless of status. Useful for logging, cleanup, or controlling iteration timing.
968
+
969
+ ```typescript
970
+ await execute({
971
+ onIterationEnd: async (iteration, controller) => {
972
+ console.log(`Iteration ${iteration.id} ended with status: ${iteration.status.type}`)
973
+
974
+ // Add delays, cleanup, or conditional logic
975
+ if (iteration.status.type === 'execution_error') {
976
+ await logError(iteration.error)
977
+
978
+ // Add delay before retry
979
+ await new Promise((resolve) => setTimeout(resolve, 1000))
980
+ }
981
+
982
+ // Can use controller to abort execution if needed
983
+ if (shouldAbort(iteration)) {
984
+ controller.abort('Custom abort reason')
985
+ }
986
+ },
987
+ // ... other props
988
+ })
989
+ ```
990
+
991
+ ### onExit (Blocking, Non-mutation)
992
+
993
+ Called when an exit is reached. Useful for logging, notifications, or implementing guardrails by throwing errors to prevent exit.
994
+
995
+ ```typescript
996
+ await execute({
997
+ onExit: async (result) => {
998
+ console.log(`Exiting with: ${result.exit.name}`, result.result)
999
+
1000
+ // Implement guardrails
1001
+ if (result.exit.name === 'approve_loan' && result.result.amount > 10000) {
1002
+ throw new Error('Manager approval required for loans over $10,000')
1003
+ }
1004
+
1005
+ // Send notifications
1006
+ await notifyStakeholders(result)
1007
+
1008
+ // Log to audit trail
1009
+ await auditLog.record({
1010
+ action: result.exit.name,
1011
+ data: result.result,
1012
+ timestamp: Date.now(),
1013
+ })
1014
+ },
1015
+ // ... other props
1016
+ })
1017
+ ```
1018
+
1019
+ ### onBeforeExecution (Blocking, Mutation)
1020
+
1021
+ Called after LLM generates code but before execution. Allows code modification and guardrails implementation.
1022
+
1023
+ ```typescript
1024
+ await execute({
1025
+ onBeforeExecution: async (iteration, controller) => {
1026
+ console.log('Generated code:', iteration.code)
1027
+
1028
+ // Code modification
1029
+ if (iteration.code?.includes('dangerousOperation')) {
1030
+ return {
1031
+ code: iteration.code.replace('dangerousOperation', 'safeOperation'),
1032
+ }
1033
+ }
1034
+
1035
+ // Guardrails - throw to prevent execution
1036
+ if (iteration.code?.includes('forbidden')) {
1037
+ throw new Error('Forbidden operation detected')
1038
+ }
1039
+
1040
+ // Add security checks
1041
+ const securityIssues = await scanCodeForSecurity(iteration.code)
1042
+ if (securityIssues.length > 0) {
1043
+ throw new Error(`Security issues found: ${securityIssues.join(', ')}`)
1044
+ }
1045
+
1046
+ // Log code generation for audit
1047
+ await auditCodeGeneration(iteration.code)
1048
+ },
1049
+ // ... other props
1050
+ })
1051
+ ```
1052
+
1053
+ ### onBeforeTool (Blocking, Mutation)
1054
+
1055
+ Called before any tool execution. Allows input modification and tool execution control.
1056
+
1057
+ ```typescript
1058
+ await execute({
1059
+ onBeforeTool: async ({ iteration, tool, input, controller }) => {
1060
+ console.log(`Executing tool: ${tool.name}`, input)
1061
+
1062
+ // Input modification
1063
+ if (tool.name === 'sendEmail') {
1064
+ return {
1065
+ input: {
1066
+ ...input,
1067
+ subject: `[Automated] ${input.subject}`, // Add prefix
1068
+ from: 'noreply@company.com', // Override sender
1069
+ },
1070
+ }
1071
+ }
1072
+
1073
+ // Access control
1074
+ if (tool.name === 'deleteFile' && !hasPermission(input.path)) {
1075
+ throw new Error('Insufficient permissions to delete file')
1076
+ }
1077
+
1078
+ // Rate limiting
1079
+ const rateLimit = await checkRateLimit(tool.name)
1080
+ if (rateLimit.exceeded) {
1081
+ throw new Error(`Rate limit exceeded for ${tool.name}`)
1082
+ }
1083
+
1084
+ // Validation
1085
+ await validateToolUsage(tool, input)
1086
+ },
1087
+ // ... other props
1088
+ })
1089
+ ```
1090
+
1091
+ ### onAfterTool (Blocking, Mutation)
1092
+
1093
+ Called after tool execution. Allows output modification and post-processing.
1094
+
1095
+ ```typescript
1096
+ await execute({
1097
+ onAfterTool: async ({ iteration, tool, input, output, controller }) => {
1098
+ console.log(`Tool ${tool.name} completed`, { input, output })
1099
+
1100
+ // Output modification
1101
+ if (tool.name === 'fetchUserData') {
1102
+ return {
1103
+ output: {
1104
+ ...output,
1105
+ // Remove sensitive data before LLM sees it
1106
+ ssn: undefined,
1107
+ creditCard: undefined,
1108
+ // Add metadata
1109
+ fetchedAt: Date.now(),
1110
+ },
1111
+ }
1112
+ }
1113
+
1114
+ // Result enhancement
1115
+ if (tool.name === 'calculatePrice') {
1116
+ return {
1117
+ output: {
1118
+ ...output,
1119
+ currency: 'USD',
1120
+ timestamp: Date.now(),
1121
+ exchangeRate: await getCurrentExchangeRate(),
1122
+ },
1123
+ }
1124
+ }
1125
+
1126
+ // Logging and caching
1127
+ await Promise.all([
1128
+ cacheResult(tool.name, input, output),
1129
+ logToolExecution(tool.name, input, output),
1130
+ updateMetrics(tool.name, Date.now() - tool.startTime),
1131
+ ])
1132
+ },
1133
+ // ... other props
1134
+ })
1135
+ ```
1136
+
1137
+ ### Hook Execution Order
1138
+
1139
+ For each iteration:
1140
+
1141
+ 1. **onTrace**: Throughout execution (non-blocking)
1142
+ 2. **onBeforeExecution**: After code generation, before execution
1143
+ 3. **onBeforeTool**: Before each tool call
1144
+ 4. **onAfterTool**: After each tool call
1145
+ 5. **onExit**: When exit is reached
1146
+ 6. **onIterationEnd**: After iteration completes
1147
+
1148
+ ### Advanced Hook Patterns
1149
+
1150
+ #### Conditional Hook Logic
1151
+
1152
+ ```typescript
1153
+ await execute({
1154
+ onBeforeTool: async ({ tool, input }) => {
1155
+ // Apply different logic based on tool
1156
+ switch (tool.name) {
1157
+ case 'payment':
1158
+ return await handlePaymentValidation(input)
1159
+ case 'notification':
1160
+ return await handleNotificationThrottling(input)
1161
+ default:
1162
+ return // No modification
1163
+ }
1164
+ },
1165
+ })
1166
+ ```
1167
+
1168
+ #### Error Recovery in Hooks
1169
+
1170
+ ```typescript
1171
+ await execute({
1172
+ onExit: async (result) => {
1173
+ try {
1174
+ await criticalPostProcessing(result)
1175
+ } catch (error) {
1176
+ // Log error but don't fail the entire execution
1177
+ console.error('Post-processing failed:', error)
1178
+
1179
+ // Optionally throw to retry the iteration
1180
+ if (error.retryable) {
1181
+ throw new Error('Retrying due to recoverable error')
1182
+ }
1183
+ }
1184
+ },
1185
+ })
1186
+ ```
1187
+
1188
+ #### Hook State Management
1189
+
1190
+ ```typescript
1191
+ let executionMetrics = { toolCalls: 0, totalTime: 0 }
1192
+
1193
+ await execute({
1194
+ onBeforeTool: async ({ tool }) => {
1195
+ executionMetrics.toolCalls++
1196
+ tool.startTime = Date.now()
1197
+ },
1198
+
1199
+ onAfterTool: async ({ tool }) => {
1200
+ executionMetrics.totalTime += Date.now() - tool.startTime
1201
+ },
1202
+
1203
+ onIterationEnd: async () => {
1204
+ console.log('Execution metrics:', executionMetrics)
1205
+ },
1206
+ })
1207
+ ```
1208
+
1209
+ ### Best Practices
1210
+
1211
+ 1. **Error Handling**: Always wrap hook logic in try-catch for production
1212
+ 2. **Performance**: Keep hooks lightweight, especially `onTrace`
1213
+ 3. **Security**: Use `onBeforeExecution` and `onBeforeTool` for security validation
1214
+ 4. **Debugging**: Leverage `onTrace` for comprehensive execution monitoring
1215
+ 5. **Guardrails**: Implement business logic validation in `onExit`
1216
+ 6. **Data Transformation**: Use `onBeforeTool`/`onAfterTool` for input/output processing
1217
+ 7. **Async Operations**: All hooks support async/await for external API calls
1218
+ 8. **State Management**: Use closures or external state for cross-hook data sharing
1219
+
1220
+ ---
1221
+
1222
+ ## Advanced Features
1223
+
1224
+ ### Snapshots (Pauseable Execution)
1225
+
1226
+ Snapshots allow you to pause and resume LLMz execution, enabling long-running workflows that can be interrupted and continued later.
1227
+
1228
+ #### SnapshotSignal
1229
+
1230
+ Inside a tool, throw a `SnapshotSignal` to halt execution and create a serializable snapshot:
1231
+
1232
+ ```typescript
1233
+ import { SnapshotSignal, Tool } from 'llmz'
1234
+
1235
+ const longRunningTool = new Tool({
1236
+ name: 'processLargeDataset',
1237
+ input: z.object({ datasetId: z.string() }),
1238
+ async handler({ datasetId }) {
1239
+ // Start processing
1240
+ const dataset = await loadDataset(datasetId)
1241
+
1242
+ // At any point, pause execution for later resumption
1243
+ if (dataset.size > LARGE_THRESHOLD) {
1244
+ throw new SnapshotSignal(
1245
+ 'Dataset is large, pausing for background processing',
1246
+ 'Processing will continue once background job completes'
1247
+ )
1248
+ }
1249
+
1250
+ return { processed: true }
1251
+ },
1252
+ })
1253
+ ```
1254
+
1255
+ #### Snapshot Handling
1256
+
1257
+ ```typescript
1258
+ const result = await execute({
1259
+ instructions: 'Process the uploaded dataset',
1260
+ tools: [longRunningTool],
1261
+ client,
1262
+ })
1263
+
1264
+ if (result.isInterrupted()) {
1265
+ console.log('Execution paused:', result.signal.message)
1266
+
1267
+ // Serialize snapshot for persistence
1268
+ const serialized = result.snapshot.toJSON()
1269
+ await database.saveSnapshot('job-123', serialized)
1270
+
1271
+ // Start background processing
1272
+ await backgroundJobQueue.add('process-dataset', {
1273
+ snapshotId: 'job-123',
1274
+ datasetId: result.signal.toolCall?.input.datasetId,
1275
+ })
1276
+ }
1277
+ ```
1278
+
1279
+ #### Resuming from Snapshot
1280
+
1281
+ ```typescript
1282
+ // Later, when background job completes
1283
+ const serialized = await database.getSnapshot('job-123')
1284
+ const snapshot = Snapshot.fromJSON(serialized)
1285
+
1286
+ // Resolve the snapshot with the result
1287
+ snapshot.resolve({
1288
+ processed: true,
1289
+ recordCount: 1000000,
1290
+ processingTime: 3600000,
1291
+ })
1292
+
1293
+ // Continue execution from where it left off
1294
+ const continuation = await execute({
1295
+ snapshot,
1296
+ instructions: 'Process the uploaded dataset', // Same as original
1297
+ tools: [longRunningTool], // Same tools
1298
+ client,
1299
+ })
1300
+
1301
+ if (continuation.isSuccess()) {
1302
+ console.log('Processing completed:', continuation.output)
1303
+ }
1304
+ ```
1305
+
1306
+ #### Snapshot Rejection
1307
+
1308
+ ```typescript
1309
+ // If background processing fails
1310
+ const snapshot = Snapshot.fromJSON(serialized)
1311
+ snapshot.reject(new Error('Background processing failed'))
1312
+
1313
+ const continuation = await execute({
1314
+ snapshot,
1315
+ // ... same parameters
1316
+ })
1317
+
1318
+ // The agent will receive the error and can handle it
1319
+ ```
1320
+
1321
+ ### Thinking (Agent Reflection)
1322
+
1323
+ The thinking system allows agents to pause and reflect on variables and context before proceeding.
1324
+
1325
+ #### ThinkSignal (Tool-Initiated)
1326
+
1327
+ Tools can force thinking by throwing a `ThinkSignal`:
1328
+
1329
+ ```typescript
1330
+ const analysisTool = new Tool({
1331
+ name: 'analyzeData',
1332
+ input: z.object({ data: z.array(z.number()) }),
1333
+ async handler({ data }) {
1334
+ const result = performAnalysis(data)
1335
+
1336
+ // Force the agent to think about the results before responding
1337
+ throw new ThinkSignal(
1338
+ 'Analysis complete, consider the implications',
1339
+ `Found ${result.anomalies.length} anomalies and ${result.patterns.length} patterns`
1340
+ )
1341
+ },
1342
+ })
1343
+ ```
1344
+
1345
+ #### Agent-Initiated Thinking
1346
+
1347
+ Agents can request thinking time in generated code:
1348
+
1349
+ ```typescript
1350
+ // In generated code
1351
+ const analysisResult = await analyzeData({ data: userInputData })
1352
+
1353
+ // Think about the results before responding to user
1354
+ return { action: 'think' }
1355
+ ```
1356
+
1357
+ #### Thinking with Variables
1358
+
1359
+ Pass specific variables for reflection:
1360
+
1361
+ ```typescript
1362
+ // In generated code
1363
+ const price = await calculatePrice({ items: cartItems })
1364
+ const budget = await getUserBudget()
1365
+
1366
+ // Think about pricing vs budget with specific context
1367
+ return {
1368
+ action: 'think',
1369
+ price,
1370
+ budget,
1371
+ recommendation: price > budget ? 'deny' : 'approve',
1372
+ }
1373
+ ```
1374
+
1375
+ #### Handling Think Results
1376
+
1377
+ ```typescript
1378
+ const result = await execute({
1379
+ instructions: 'Analyze the user data and provide recommendations',
1380
+ tools: [analysisTool],
1381
+ client,
1382
+ })
1383
+
1384
+ if (result.is(ThinkExit)) {
1385
+ console.log('Agent is thinking about:', result.output.variables)
1386
+
1387
+ // Continue execution after thinking
1388
+ const continuation = await execute({
1389
+ instructions: result.context.instructions,
1390
+ tools: result.context.tools,
1391
+ // Variables from thinking are automatically preserved
1392
+ client,
1393
+ })
1394
+ }
1395
+ ```
1396
+
1397
+ ### Citations (RAG Support)
1398
+
1399
+ CitationsManager provides standardized source tracking and referencing for RAG (Retrieval-Augmented Generation) systems.
1400
+
1401
+ #### Core Concepts
1402
+
1403
+ Citations use rare Unicode symbols (`【】`) as markers that are unlikely to appear in natural text. The system supports:
1404
+
1405
+ - **Source Registration**: Register any object as a citation source
1406
+ - **Tag Generation**: Automatic creation of unique citation tags like `【0】`, `【1】`
1407
+ - **Content Processing**: Extract and clean citation tags from text
1408
+ - **Multiple Citations**: Support for multi-source citations like `【0,1,3】`
1409
+
1410
+ #### Basic Usage
1411
+
1412
+ ```typescript
1413
+ import { CitationsManager } from 'llmz'
1414
+
1415
+ const citations = new CitationsManager()
1416
+
1417
+ // Register sources and get citation tags
1418
+ const source1 = citations.registerSource({
1419
+ file: 'document.pdf',
1420
+ page: 5,
1421
+ title: 'Company Policy',
1422
+ })
1423
+
1424
+ const source2 = citations.registerSource({
1425
+ url: 'https://example.com/article',
1426
+ title: 'Best Practices',
1427
+ })
1428
+
1429
+ console.log(source1.tag) // "【0】"
1430
+ console.log(source2.tag) // "【1】"
1431
+
1432
+ // Use tags in content
1433
+ const content = `The policy states employees must arrive on time${source1.tag}. However, best practices suggest flexibility${source2.tag}.`
1434
+ ```
1435
+
1436
+ #### RAG Implementation Example
1437
+
1438
+ ```typescript
1439
+ const ragTool = new Tool({
1440
+ name: 'search',
1441
+ description: 'Searches in the knowledge base for relevant information.',
1442
+ input: z.string().describe('The query to search in the knowledge base.'),
1443
+ async handler(query) {
1444
+ // Perform semantic search
1445
+ const { passages } = await client.searchFiles({
1446
+ query,
1447
+ limit: 20,
1448
+ contextDepth: 3,
1449
+ })
1450
+
1451
+ if (!passages.length) {
1452
+ throw new ThinkSignal(
1453
+ 'No results found',
1454
+ 'No results were found in the knowledge base. Try rephrasing your question.'
1455
+ )
1456
+ }
1457
+
1458
+ // Build response with citations
1459
+ let message: string[] = ['Here are the search results:']
1460
+ let { tag: example } = chat.citations.registerSource({}) // Example citation
1461
+
1462
+ // Register each passage as a source
1463
+ for (const passage of passages) {
1464
+ const { tag } = chat.citations.registerSource({
1465
+ file: passage.file.key,
1466
+ title: passage.file.tags.title,
1467
+ })
1468
+
1469
+ message.push(`<${tag} file="${passage.file.key}">`)
1470
+ message.push(`**${passage.file.tags.title}**`)
1471
+ message.push(passage.content)
1472
+ message.push(`</${tag}>`)
1473
+ }
1474
+
1475
+ // Provide context with citation instructions
1476
+ throw new ThinkSignal(
1477
+ `Got search results. When answering, you MUST add inline citations (eg: "The price is $10${example} ...")`,
1478
+ message.join('\n').trim()
1479
+ )
1480
+ },
1481
+ })
1482
+ ```
1483
+
1484
+ #### Chat Integration
1485
+
1486
+ ```typescript
1487
+ class CLIChat extends Chat {
1488
+ public citations: CitationsManager = new CitationsManager()
1489
+
1490
+ private async sendMessage(input: RenderedComponent) {
1491
+ if (input.type === 'Text') {
1492
+ let sources: string[] = []
1493
+
1494
+ // Extract citations and format them for display
1495
+ const { cleaned } = this.citations.extractCitations(input.text, (citation) => {
1496
+ let idx = chalk.bgGreenBright.black.bold(` ${sources.length + 1} `)
1497
+ sources.push(`${idx}: ${JSON.stringify(citation.source)}`)
1498
+ return `${idx}` // Replace 【0】 with [1]
1499
+ })
1500
+
1501
+ // Display cleaned text and sources
1502
+ console.log(`🤖 Agent: ${cleaned}`)
1503
+
1504
+ if (sources.length) {
1505
+ console.log(chalk.dim('Citations'))
1506
+ console.log(chalk.dim('========='))
1507
+ console.log(chalk.dim(sources.join('\n')))
1508
+ }
1509
+ }
1510
+ }
1511
+ }
1512
+ ```
1513
+
1514
+ #### Advanced Citation Features
1515
+
1516
+ **Multiple Citation Support:**
1517
+
1518
+ ```typescript
1519
+ // Agent can reference multiple sources in one citation
1520
+ const content = 'This fact is supported by multiple studies【0,1,3】'
1521
+
1522
+ const { cleaned, citations } = manager.extractCitations(content)
1523
+ // citations array contains entries for sources 0, 1, and 3
1524
+ ```
1525
+
1526
+ **Object Citation Processing:**
1527
+
1528
+ ```typescript
1529
+ // Remove citations from complex objects
1530
+ const dataWithCitations = {
1531
+ summary: 'The report shows positive trends【0】',
1532
+ details: {
1533
+ revenue: 'Increased by 15%【1】',
1534
+ costs: 'Reduced by 8%【2】',
1535
+ },
1536
+ }
1537
+
1538
+ const [cleanData, extractedCitations] = manager.removeCitationsFromObject(dataWithCitations)
1539
+ // cleanData has citations removed, extractedCitations contains path + citation info
1540
+ ```
1541
+
1542
+ **Citation Stripping:**
1543
+
1544
+ ```typescript
1545
+ // Remove all citation tags from content
1546
+ const textWithCitations = 'This statement【0】 has multiple【1,2】 citations.'
1547
+ const cleaned = CitationsManager.stripCitationTags(textWithCitations)
1548
+ // Result: "This statement has multiple citations."
1549
+ ```
1550
+
1551
+ ### Dynamic Context
1552
+
1553
+ LLMz supports dynamic evaluation of most parameters, allowing context-aware configuration:
1554
+
1555
+ ```typescript
1556
+ await execute({
1557
+ // Dynamic instructions based on context
1558
+ instructions: (ctx) => {
1559
+ const timeOfDay = new Date().getHours()
1560
+ const greeting = timeOfDay < 12 ? 'Good morning' : 'Good afternoon'
1561
+ return `${greeting}! You are a helpful assistant with access to ${ctx.tools?.length || 0} tools.`
1562
+ },
1563
+
1564
+ // Dynamic tools based on user permissions
1565
+ tools: async (ctx) => {
1566
+ const userPermissions = await getUserPermissions(ctx.userId)
1567
+ return allTools.filter((tool) => userPermissions.includes(tool.permission))
1568
+ },
1569
+
1570
+ // Dynamic objects with current state
1571
+ objects: async (ctx) => {
1572
+ const userPreferences = await loadUserPreferences(ctx.userId)
1573
+ return [
1574
+ new ObjectInstance({
1575
+ name: 'user',
1576
+ properties: [{ name: 'preferences', value: userPreferences, writable: true }],
1577
+ }),
1578
+ ]
1579
+ },
1580
+
1581
+ client,
1582
+ })
1583
+ ```
1584
+
1585
+ ---
1586
+
1587
+ ## API Reference
1588
+
1589
+ ### Core Functions
1590
+
1591
+ #### execute(props: ExecutionProps): Promise<ExecutionResult>
1592
+
1593
+ Main execution function that runs LLMz agents in either Chat Mode or Worker Mode.
1594
+
1595
+ **Parameters:**
1596
+
1597
+ - `props.client` - Botpress Client or Cognitive Client instance for LLM generation
1598
+ - `props.instructions` - System prompt/instructions for the LLM (static string or dynamic function)
1599
+ - `props.chat` - Optional Chat instance to enable Chat Mode with user interaction
1600
+ - `props.tools` - Array of Tool instances available to the agent (static or dynamic)
1601
+ - `props.objects` - Array of ObjectInstance for namespaced tools and variables (static or dynamic)
1602
+ - `props.exits` - Array of Exit definitions for structured completion (static or dynamic)
1603
+ - `props.snapshot` - Optional Snapshot to resume paused execution
1604
+ - `props.signal` - Optional AbortSignal to cancel execution
1605
+ - `props.options` - Optional execution options (loop limit, temperature, model, timeout)
1606
+ - `props.onTrace` - Optional non-blocking hook for monitoring traces during execution
1607
+ - `props.onIterationEnd` - Optional blocking hook called after each iteration
1608
+ - `props.onExit` - Optional blocking hook called when an exit is reached
1609
+ - `props.onBeforeExecution` - Optional blocking hook to modify code before VM execution
1610
+ - `props.onBeforeTool` - Optional blocking hook to modify tool inputs before execution
1611
+ - `props.onAfterTool` - Optional blocking hook to modify tool outputs after execution
1612
+
1613
+ **Returns:** `Promise<ExecutionResult>` - Result containing success/error/interrupted status with type-safe exit checking
1614
+
1615
+ ### Tool Class
1616
+
1617
+ #### new Tool(config: ToolConfig)
1618
+
1619
+ Creates a new tool definition with type-safe schemas.
1620
+
1621
+ **Properties:**
1622
+
1623
+ - `name: string` - Tool name used in generated code
1624
+ - `description?: string` - Description for LLM understanding
1625
+ - `input?: ZuiSchema` - Input validation schema
1626
+ - `output?: ZuiSchema` - Output validation schema
1627
+ - `handler: (input: any) => Promise<any> | any` - Tool implementation
1628
+ - `aliases?: string[]` - Alternative names for the tool
1629
+ - `staticInputs?: Record<string, any>` - Force specific input values
1630
+
1631
+ **Methods:**
1632
+
1633
+ - `execute(input: any, context?: ToolContext): Promise<any>` - Execute the tool
1634
+ - `getTypings(): string` - Get TypeScript definitions for LLM
1635
+ - `clone(overrides: Partial<ToolConfig>): Tool` - Create a modified copy
1636
+
1637
+ ### Exit Class
1638
+
1639
+ #### new Exit(config: ExitConfig)
1640
+
1641
+ Defines a structured exit point for agent execution.
1642
+
1643
+ **Properties:**
1644
+
1645
+ - `name: string` - Exit name used in generated code
1646
+ - `description?: string` - Description for LLM understanding
1647
+ - `schema?: ZuiSchema` - Output validation schema
1648
+ - `aliases?: string[]` - Alternative names for the exit
1649
+
1650
+ ### ObjectInstance Class
1651
+
1652
+ #### new ObjectInstance(config: ObjectConfig)
1653
+
1654
+ Creates a namespaced container for tools and variables.
1655
+
1656
+ **Properties:**
1657
+
1658
+ - `name: string` - Object name used in generated code
1659
+ - `properties?: PropertyConfig[]` - Object properties/variables
1660
+ - `tools?: Tool[]` - Tools scoped to this object
1661
+
1662
+ **PropertyConfig:**
1663
+
1664
+ - `name: string` - Property name
1665
+ - `value: any` - Initial value
1666
+ - `writable: boolean` - Whether property can be modified
1667
+ - `type?: ZuiSchema` - Validation schema
1668
+
1669
+ ### ExecutionResult Types
1670
+
1671
+ #### SuccessExecutionResult
1672
+
1673
+ **Properties:**
1674
+
1675
+ - `isSuccess(): boolean` - Type guard for success
1676
+ - `output: any` - The result data from the exit
1677
+ - `exit: Exit` - The exit that was used
1678
+ - `iteration: Iteration` - Final iteration details
1679
+ - `iterations: Iteration[]` - All iterations
1680
+ - `context: Context` - Execution context
1681
+ - `is(exit: Exit): boolean` - Type-safe exit checking
1682
+
1683
+ #### ErrorExecutionResult
1684
+
1685
+ **Properties:**
1686
+
1687
+ - `isError(): boolean` - Type guard for error
1688
+ - `error: Error | string` - The error that occurred
1689
+ - `iteration?: Iteration` - Failed iteration details
1690
+ - `iterations: Iteration[]` - All iterations before failure
1691
+ - `context: Context` - Execution context
1692
+
1693
+ #### PartialExecutionResult
1694
+
1695
+ **Properties:**
1696
+
1697
+ - `isInterrupted(): boolean` - Type guard for interruption
1698
+ - `signal: SnapshotSignal` - The signal that caused interruption
1699
+ - `snapshot: Snapshot` - Serializable execution state
1700
+ - `iterations: Iteration[]` - All iterations before interruption
1701
+ - `context: Context` - Execution context
1702
+
1703
+ ### Chat Class
1704
+
1705
+ Abstract base class for implementing chat interfaces.
1706
+
1707
+ **Abstract Methods:**
1708
+
1709
+ - `getTranscript(): Promise<Transcript.Message[]> | Transcript.Message[]` - Get conversation history
1710
+ - `getComponents(): Promise<ComponentDefinition[]> | ComponentDefinition[]` - Get available UI components
1711
+ - `handler(component: RenderedComponent): Promise<void>` - Handle agent messages
1712
+
1713
+ ### CitationsManager Class
1714
+
1715
+ Manages source citations for RAG systems.
1716
+
1717
+ **Methods:**
1718
+
1719
+ - `registerSource(source: any): { tag: string, id: number }` - Register a source and get citation tag
1720
+ - `extractCitations(text: string, replacer?: (citation) => string): { cleaned: string, citations: Citation[] }` - Extract and process citations
1721
+ - `removeCitationsFromObject(obj: any): [cleanedObj: any, citations: Citation[]]` - Remove citations from objects
1722
+ - `static stripCitationTags(text: string): string` - Remove all citation tags
1723
+
1724
+ ### Snapshot Class
1725
+
1726
+ Manages pauseable execution state.
1727
+
1728
+ **Methods:**
1729
+
1730
+ - `toJSON(): string` - Serialize snapshot
1731
+ - `static fromJSON(json: string): Snapshot` - Deserialize snapshot
1732
+ - `resolve(data: any): void` - Resume with success
1733
+ - `reject(error: Error): void` - Resume with error
1734
+
1735
+ ### Built-in Exits
1736
+
1737
+ - `ListenExit` - Automatically available in Chat Mode for user interaction
1738
+ - `DefaultExit` - Default exit for Worker Mode with success/failure discrimination
1739
+ - `ThinkExit` - Used when agent requests thinking time
1740
+
1741
+ ### Signals
1742
+
1743
+ - `SnapshotSignal` - Thrown to pause execution for later resumption
1744
+ - `ThinkSignal` - Thrown to request agent reflection time
1745
+ - `LoopExceededError` - Thrown when maximum iterations reached
1746
+
1747
+ ### Environment Variables
1748
+
1749
+ - `VM_DRIVER: 'isolated-vm' | 'node'` - Choose VM execution environment
1750
+ - `CI: boolean` - Automatically detected, affects VM driver selection
1751
+
1752
+ ---
1753
+
1754
+ ## Examples
1755
+
1756
+ The LLMz repository includes 20 comprehensive examples demonstrating different patterns and capabilities:
1757
+
1758
+ ### Chat Examples (Interactive Patterns)
1759
+
1760
+ 1. **01_chat_basic** - Basic conversational agent setup
1761
+ 2. **02_chat_exits** - Custom exits for structured conversations
1762
+ 3. **03_chat_conditional_tool** - Conditional tool usage based on context
1763
+ 4. **04_chat_small_models** - Optimizations for smaller language models
1764
+ 5. **05_chat_web_search** - Integration with web search capabilities
1765
+ 6. **06_chat_confirm_tool** - User confirmation patterns for sensitive operations
1766
+ 7. **07_chat_guardrails** - Safety mechanisms and content filtering
1767
+ 8. **08_chat_multi_agent** - Multi-agent orchestration and delegation
1768
+ 9. **09_chat_variables** - Object variables and state management
1769
+ 10. **10_chat_components** - Rich UI components and interactive elements
1770
+
1771
+ ### Worker Examples (Automated Patterns)
1772
+
1773
+ 11. **11_worker_minimal** - Simplest worker mode execution
1774
+ 12. **12_worker_fs** - File system operations and data processing
1775
+ 13. **13_worker_sandbox** - Security isolation and sandboxing
1776
+ 14. **14_worker_snapshot** - Pauseable execution and resumption
1777
+ 15. **15_worker_stacktraces** - Error handling and debugging
1778
+ 16. **16_worker_tool_chaining** - Complex multi-tool workflows
1779
+ 17. **17_worker_error_recovery** - Graceful error recovery patterns
1780
+ 18. **18_worker_security** - Security best practices and validation
1781
+ 19. **19_worker_wrap_tool** - Tool modification and enhancement
1782
+ 20. **20_chat_rag** - Retrieval-Augmented Generation with citations
1783
+
1784
+ ### Running Examples
1785
+
1786
+ ```bash
1787
+ # Install dependencies
1788
+ pnpm install
1789
+
1790
+ # Set up environment variables
1791
+ cp .env.example .env
1792
+ # Edit .env with your Botpress credentials
1793
+
1794
+ # Run a specific example
1795
+ pnpm start 01_chat_basic
1796
+ pnpm start chat_basic
1797
+ pnpm start 01
1798
+
1799
+ # List all available examples
1800
+ pnpm start
1801
+ ```
1802
+
1803
+ ### Example Environment Setup
1804
+
1805
+ Create a `.env` file in the examples directory:
1806
+
1807
+ ```env
1808
+ BOTPRESS_BOT_ID=your_bot_id_here
1809
+ BOTPRESS_TOKEN=your_token_here
1810
+ ```
1811
+
1812
+ ### Key Learning Paths
1813
+
1814
+ **Getting Started:**
1815
+
1816
+ - Start with `01_chat_basic` and `11_worker_minimal`
1817
+ - Understand the difference between Chat and Worker modes
1818
+ - Learn basic tool integration patterns
1819
+
1820
+ **Intermediate Concepts:**
1821
+
1822
+ - Explore `09_chat_variables` for state management
1823
+ - Study `16_worker_tool_chaining` for complex workflows
1824
+ - Review `14_worker_snapshot` for pauseable execution
1825
+
1826
+ **Advanced Patterns:**
1827
+
1828
+ - Examine `08_chat_multi_agent` for orchestration
1829
+ - Learn from `20_chat_rag` for knowledge integration
1830
+ - Study `18_worker_security` for production deployment
1831
+
1832
+ Each example includes detailed comments explaining the concepts and implementation patterns, making them excellent learning resources for understanding LLMz capabilities.
1833
+
1834
+ ---
1835
+
1836
+ _This documentation covers the complete LLMz framework. For the latest updates and community contributions, visit the [LLMz repository](https://github.com/botpress/llmz)._