@vibe-agent-toolkit/vat-development-agents 0.1.29 → 0.1.30-rc.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/.claude/plugins/marketplaces/vat-skills/CHANGELOG.md +20 -0
- package/dist/.claude/plugins/marketplaces/vat-skills/plugins/vibe-agent-toolkit/.claude-plugin/plugin.json +1 -1
- package/dist/.claude/plugins/marketplaces/vat-skills/plugins/vibe-agent-toolkit/skills/audit/SKILL.md +21 -16
- package/dist/.claude/plugins/marketplaces/vat-skills/plugins/vibe-agent-toolkit/skills/authoring/SKILL.md +30 -12
- package/dist/.claude/plugins/marketplaces/vat-skills/plugins/vibe-agent-toolkit/skills/authoring/resources/skill-quality-checklist.md +42 -0
- package/dist/.claude/plugins/marketplaces/vat-skills/plugins/vibe-agent-toolkit/skills/debugging/SKILL.md +4 -4
- package/dist/.claude/plugins/marketplaces/vat-skills/plugins/vibe-agent-toolkit/skills/vibe-agent-toolkit/SKILL.md +6 -6
- package/dist/generated/resources/skills/vat-agent-authoring.js +3 -3
- package/dist/generated/resources/skills/vat-audit.js +5 -5
- package/dist/skills/audit/SKILL.md +21 -16
- package/dist/skills/authoring/SKILL.md +30 -12
- package/dist/skills/authoring/resources/skill-quality-checklist.md +42 -0
- package/dist/skills/debugging/SKILL.md +4 -4
- package/dist/skills/vibe-agent-toolkit/SKILL.md +6 -6
- package/package.json +4 -4
- package/dist/.claude/plugins/marketplaces/vat-skills/plugins/vibe-agent-toolkit/skills/debugging/resources/CLAUDE.md +0 -539
- package/dist/.claude/plugins/marketplaces/vat-skills/plugins/vibe-agent-toolkit/skills/debugging/resources/debug-and-test-vat-fixes.md +0 -111
- package/dist/.claude/plugins/marketplaces/vat-skills/plugins/vibe-agent-toolkit/skills/debugging/resources/writing-tests.md +0 -577
- package/dist/.claude/plugins/marketplaces/vat-skills/plugins/vibe-agent-toolkit/skills/vibe-agent-toolkit/resources/adding-runtime-adapters.md +0 -628
- package/dist/.claude/plugins/marketplaces/vat-skills/plugins/vibe-agent-toolkit/skills/vibe-agent-toolkit/resources/agent-authoring.md +0 -905
- package/dist/.claude/plugins/marketplaces/vat-skills/plugins/vibe-agent-toolkit/skills/vibe-agent-toolkit/resources/compiling-markdown-to-typescript.md +0 -501
- package/dist/.claude/plugins/marketplaces/vat-skills/plugins/vibe-agent-toolkit/skills/vibe-agent-toolkit/resources/getting-started.md +0 -360
- package/dist/.claude/plugins/marketplaces/vat-skills/plugins/vibe-agent-toolkit/skills/vibe-agent-toolkit/resources/orchestration.md +0 -859
- package/dist/.claude/plugins/marketplaces/vat-skills/plugins/vibe-agent-toolkit/skills/vibe-agent-toolkit/resources/rag-usage-guide.md +0 -770
- package/dist/skills/debugging/resources/CLAUDE.md +0 -539
- package/dist/skills/debugging/resources/debug-and-test-vat-fixes.md +0 -111
- package/dist/skills/debugging/resources/writing-tests.md +0 -577
- package/dist/skills/vibe-agent-toolkit/resources/adding-runtime-adapters.md +0 -628
- package/dist/skills/vibe-agent-toolkit/resources/agent-authoring.md +0 -905
- package/dist/skills/vibe-agent-toolkit/resources/compiling-markdown-to-typescript.md +0 -501
- package/dist/skills/vibe-agent-toolkit/resources/getting-started.md +0 -360
- package/dist/skills/vibe-agent-toolkit/resources/orchestration.md +0 -859
- package/dist/skills/vibe-agent-toolkit/resources/rag-usage-guide.md +0 -770
|
@@ -1,859 +0,0 @@
|
|
|
1
|
-
# Agent Orchestration Guide
|
|
2
|
-
|
|
3
|
-
## Introduction
|
|
4
|
-
|
|
5
|
-
This guide shows how to orchestrate VAT agents using standardized result envelopes and Railway-Oriented Programming (ROP) patterns. All VAT agents return consistent result types, enabling type-safe composition, error handling, and complex workflows.
|
|
6
|
-
|
|
7
|
-
## Core Concepts
|
|
8
|
-
|
|
9
|
-
### Result Types
|
|
10
|
-
|
|
11
|
-
VAT defines two core result types in `@vibe-agent-toolkit/agent-schema`:
|
|
12
|
-
|
|
13
|
-
#### AgentResult<TData, TError>
|
|
14
|
-
|
|
15
|
-
For single-execution agents (pure functions, one-shot LLM analyzers):
|
|
16
|
-
|
|
17
|
-
```typescript
|
|
18
|
-
type AgentResult<TData, TError> =
|
|
19
|
-
| { status: 'success'; data: TData }
|
|
20
|
-
| { status: 'error'; error: TError };
|
|
21
|
-
```
|
|
22
|
-
|
|
23
|
-
**Examples:**
|
|
24
|
-
- Pure function validators return `AgentResult<ValidationData, ValidationError>`
|
|
25
|
-
- One-shot LLM analyzers return `AgentResult<AnalysisData, LLMError>`
|
|
26
|
-
|
|
27
|
-
#### StatefulAgentResult<TData, TError, TMetadata>
|
|
28
|
-
|
|
29
|
-
For multi-turn conversational agents that maintain state:
|
|
30
|
-
|
|
31
|
-
```typescript
|
|
32
|
-
type StatefulAgentResult<TData, TError, TMetadata> =
|
|
33
|
-
| { status: 'in-progress'; metadata?: TMetadata }
|
|
34
|
-
| { status: 'success'; data: TData }
|
|
35
|
-
| { status: 'error'; error: TError };
|
|
36
|
-
```
|
|
37
|
-
|
|
38
|
-
**Examples:**
|
|
39
|
-
- Conversational assistants return `StatefulAgentResult<FinalData, ConversationError, ProgressMetadata>`
|
|
40
|
-
- Multi-step workflows return `StatefulAgentResult<WorkflowResult, WorkflowError, StepMetadata>`
|
|
41
|
-
|
|
42
|
-
### Output Envelopes
|
|
43
|
-
|
|
44
|
-
Agents return output envelopes that wrap results with additional context:
|
|
45
|
-
|
|
46
|
-
#### OneShotAgentOutput<TData, TError>
|
|
47
|
-
|
|
48
|
-
```typescript
|
|
49
|
-
interface OneShotAgentOutput<TData, TError> {
|
|
50
|
-
result: AgentResult<TData, TError>;
|
|
51
|
-
}
|
|
52
|
-
```
|
|
53
|
-
|
|
54
|
-
**Use for:**
|
|
55
|
-
- Pure function tools (validators, transformers)
|
|
56
|
-
- One-shot LLM analyzers (photo analysis, text parsing)
|
|
57
|
-
- External event integrators (approval gates, webhooks)
|
|
58
|
-
|
|
59
|
-
**Example:**
|
|
60
|
-
```typescript
|
|
61
|
-
const output = await haikuValidator.execute({
|
|
62
|
-
text: 'Ancient pond, frog leaps in, splash',
|
|
63
|
-
syllables: [5, 7, 5],
|
|
64
|
-
});
|
|
65
|
-
|
|
66
|
-
// output.result is AgentResult<ValidationData, ValidationError>
|
|
67
|
-
if (output.result.status === 'success') {
|
|
68
|
-
console.log('Valid:', output.result.data.valid);
|
|
69
|
-
} else {
|
|
70
|
-
console.error('Invalid:', output.result.error);
|
|
71
|
-
}
|
|
72
|
-
```
|
|
73
|
-
|
|
74
|
-
#### ConversationalAgentOutput<TData, TError, TState>
|
|
75
|
-
|
|
76
|
-
```typescript
|
|
77
|
-
interface ConversationalAgentOutput<TData, TError, TState> {
|
|
78
|
-
reply: string; // Natural language response
|
|
79
|
-
sessionState: TState; // Updated session state
|
|
80
|
-
result: StatefulAgentResult<TData, TError, TMetadata>; // Machine-readable result
|
|
81
|
-
}
|
|
82
|
-
```
|
|
83
|
-
|
|
84
|
-
**Use for:**
|
|
85
|
-
- Multi-turn conversational assistants
|
|
86
|
-
- Progressive data collection agents
|
|
87
|
-
- Workflows with user interaction
|
|
88
|
-
|
|
89
|
-
**Example:**
|
|
90
|
-
```typescript
|
|
91
|
-
const output = await breedAdvisor.execute({
|
|
92
|
-
message: 'I love classical music',
|
|
93
|
-
sessionState: { profile: { conversationPhase: 'gathering' } },
|
|
94
|
-
});
|
|
95
|
-
|
|
96
|
-
// output.reply - Natural language for user
|
|
97
|
-
console.log('Agent says:', output.reply);
|
|
98
|
-
|
|
99
|
-
// output.sessionState - Carry to next turn
|
|
100
|
-
const nextInput = { message: 'I live in an apartment', sessionState: output.sessionState };
|
|
101
|
-
|
|
102
|
-
// output.result - Machine-readable status
|
|
103
|
-
if (output.result.status === 'in-progress') {
|
|
104
|
-
console.log('Gathering info, progress:', output.result.metadata);
|
|
105
|
-
} else if (output.result.status === 'success') {
|
|
106
|
-
console.log('Recommendation:', output.result.data);
|
|
107
|
-
}
|
|
108
|
-
```
|
|
109
|
-
|
|
110
|
-
### Standard Error Types
|
|
111
|
-
|
|
112
|
-
#### LLMError
|
|
113
|
-
|
|
114
|
-
For LLM-related failures:
|
|
115
|
-
|
|
116
|
-
```typescript
|
|
117
|
-
type LLMError =
|
|
118
|
-
| 'llm-refusal' // LLM refused to generate output
|
|
119
|
-
| 'llm-invalid-output' // Output didn't match expected format
|
|
120
|
-
| 'llm-timeout' // Request timed out
|
|
121
|
-
| 'llm-rate-limit' // Hit rate limit
|
|
122
|
-
| 'llm-token-limit' // Exceeded token limit
|
|
123
|
-
| 'llm-unavailable'; // Service unavailable
|
|
124
|
-
```
|
|
125
|
-
|
|
126
|
-
#### ExternalEventError
|
|
127
|
-
|
|
128
|
-
For external system integration failures:
|
|
129
|
-
|
|
130
|
-
```typescript
|
|
131
|
-
type ExternalEventError =
|
|
132
|
-
| 'event-timeout' // External event timed out
|
|
133
|
-
| 'event-unavailable' // External system unavailable
|
|
134
|
-
| 'event-rejected' // External system rejected request
|
|
135
|
-
| 'event-invalid-response'; // External system returned invalid data
|
|
136
|
-
```
|
|
137
|
-
|
|
138
|
-
#### Custom Error Types
|
|
139
|
-
|
|
140
|
-
Agents can define domain-specific errors:
|
|
141
|
-
|
|
142
|
-
```typescript
|
|
143
|
-
type ValidationError =
|
|
144
|
-
| 'invalid-syllables'
|
|
145
|
-
| 'missing-kigo'
|
|
146
|
-
| 'missing-kireji'
|
|
147
|
-
| 'too-long'
|
|
148
|
-
| 'too-short';
|
|
149
|
-
```
|
|
150
|
-
|
|
151
|
-
## Result Helpers
|
|
152
|
-
|
|
153
|
-
The `@vibe-agent-toolkit/agent-runtime` package provides helper functions for working with results.
|
|
154
|
-
|
|
155
|
-
### mapResult()
|
|
156
|
-
|
|
157
|
-
Transform success data while preserving errors:
|
|
158
|
-
|
|
159
|
-
```typescript
|
|
160
|
-
import { mapResult } from '@vibe-agent-toolkit/agent-runtime';
|
|
161
|
-
|
|
162
|
-
const result1 = { status: 'success' as const, data: 10 };
|
|
163
|
-
const result2 = mapResult(result1, (n) => n * 2);
|
|
164
|
-
// result2 = { status: 'success', data: 20 }
|
|
165
|
-
|
|
166
|
-
const result3 = { status: 'error' as const, error: 'failed' };
|
|
167
|
-
const result4 = mapResult(result3, (n) => n * 2);
|
|
168
|
-
// result4 = { status: 'error', error: 'failed' } (unchanged)
|
|
169
|
-
```
|
|
170
|
-
|
|
171
|
-
### andThen()
|
|
172
|
-
|
|
173
|
-
Chain operations that return results (monadic bind):
|
|
174
|
-
|
|
175
|
-
```typescript
|
|
176
|
-
import { andThen } from '@vibe-agent-toolkit/agent-runtime';
|
|
177
|
-
|
|
178
|
-
const result1 = { status: 'success' as const, data: 10 };
|
|
179
|
-
const result2 = andThen(result1, (n) => {
|
|
180
|
-
if (n > 5) {
|
|
181
|
-
return { status: 'success' as const, data: n * 2 };
|
|
182
|
-
}
|
|
183
|
-
return { status: 'error' as const, error: 'too-small' };
|
|
184
|
-
});
|
|
185
|
-
// result2 = { status: 'success', data: 20 }
|
|
186
|
-
|
|
187
|
-
const result3 = { status: 'error' as const, error: 'failed' };
|
|
188
|
-
const result4 = andThen(result3, (n) => {
|
|
189
|
-
return { status: 'success' as const, data: n * 2 };
|
|
190
|
-
});
|
|
191
|
-
// result4 = { status: 'error', error: 'failed' } (not called)
|
|
192
|
-
```
|
|
193
|
-
|
|
194
|
-
### match()
|
|
195
|
-
|
|
196
|
-
Pattern match on result status:
|
|
197
|
-
|
|
198
|
-
```typescript
|
|
199
|
-
import { match } from '@vibe-agent-toolkit/agent-runtime';
|
|
200
|
-
|
|
201
|
-
const result = { status: 'success' as const, data: 'hello' };
|
|
202
|
-
|
|
203
|
-
const message = match(result, {
|
|
204
|
-
success: (data) => `Success: ${data}`,
|
|
205
|
-
error: (err) => `Error: ${err}`,
|
|
206
|
-
});
|
|
207
|
-
// message = "Success: hello"
|
|
208
|
-
|
|
209
|
-
// For stateful results, add inProgress handler
|
|
210
|
-
const statefulResult = { status: 'in-progress' as const, metadata: { step: 2 } };
|
|
211
|
-
|
|
212
|
-
const status = match(statefulResult, {
|
|
213
|
-
success: (data) => `Done: ${data}`,
|
|
214
|
-
error: (err) => `Failed: ${err}`,
|
|
215
|
-
inProgress: (meta) => `Working: step ${meta?.step}`,
|
|
216
|
-
});
|
|
217
|
-
// status = "Working: step 2"
|
|
218
|
-
```
|
|
219
|
-
|
|
220
|
-
### unwrap()
|
|
221
|
-
|
|
222
|
-
Extract data, throwing on error:
|
|
223
|
-
|
|
224
|
-
```typescript
|
|
225
|
-
import { unwrap } from '@vibe-agent-toolkit/agent-runtime';
|
|
226
|
-
|
|
227
|
-
const result1 = { status: 'success' as const, data: 'hello' };
|
|
228
|
-
const data = unwrap(result1);
|
|
229
|
-
// data = "hello"
|
|
230
|
-
|
|
231
|
-
const result2 = { status: 'error' as const, error: 'failed' };
|
|
232
|
-
const data2 = unwrap(result2);
|
|
233
|
-
// Throws: Error: Result was error: failed
|
|
234
|
-
```
|
|
235
|
-
|
|
236
|
-
## Orchestration Patterns
|
|
237
|
-
|
|
238
|
-
### Sequential Execution
|
|
239
|
-
|
|
240
|
-
Execute agents in order, passing results forward:
|
|
241
|
-
|
|
242
|
-
```typescript
|
|
243
|
-
import { andThen } from '@vibe-agent-toolkit/agent-runtime';
|
|
244
|
-
|
|
245
|
-
async function analyzeAndNameCat(imagePath: string) {
|
|
246
|
-
// Step 1: Analyze photo
|
|
247
|
-
const analysisOutput = await photoAnalyzer.execute({ imagePath });
|
|
248
|
-
|
|
249
|
-
// Step 2: Generate name (only if analysis succeeded)
|
|
250
|
-
const nameOutput = await andThen(
|
|
251
|
-
analysisOutput.result,
|
|
252
|
-
async (characteristics) => {
|
|
253
|
-
const output = await nameGenerator.execute({ characteristics });
|
|
254
|
-
return output.result;
|
|
255
|
-
}
|
|
256
|
-
);
|
|
257
|
-
|
|
258
|
-
// Step 3: Validate name (only if generation succeeded)
|
|
259
|
-
const validationOutput = await andThen(
|
|
260
|
-
nameOutput,
|
|
261
|
-
async (name) => {
|
|
262
|
-
const output = await nameValidator.execute({
|
|
263
|
-
name: name.name,
|
|
264
|
-
characteristics: name.characteristics,
|
|
265
|
-
});
|
|
266
|
-
return output.result;
|
|
267
|
-
}
|
|
268
|
-
);
|
|
269
|
-
|
|
270
|
-
return validationOutput;
|
|
271
|
-
}
|
|
272
|
-
|
|
273
|
-
// Usage
|
|
274
|
-
const result = await analyzeAndNameCat('photos/orange-tabby.jpg');
|
|
275
|
-
|
|
276
|
-
match(result, {
|
|
277
|
-
success: (data) => console.log('Valid name:', data),
|
|
278
|
-
error: (err) => console.error('Failed:', err),
|
|
279
|
-
});
|
|
280
|
-
```
|
|
281
|
-
|
|
282
|
-
### Parallel Execution
|
|
283
|
-
|
|
284
|
-
Execute multiple agents concurrently:
|
|
285
|
-
|
|
286
|
-
```typescript
|
|
287
|
-
async function analyzeMultipleSources(
|
|
288
|
-
imagePath: string,
|
|
289
|
-
description: string
|
|
290
|
-
) {
|
|
291
|
-
// Run both analyzers in parallel
|
|
292
|
-
const [photoOutput, descOutput] = await Promise.all([
|
|
293
|
-
photoAnalyzer.execute({ imagePath }),
|
|
294
|
-
descriptionParser.execute({ text: description }),
|
|
295
|
-
]);
|
|
296
|
-
|
|
297
|
-
// Combine results
|
|
298
|
-
if (photoOutput.result.status === 'success' &&
|
|
299
|
-
descOutput.result.status === 'success') {
|
|
300
|
-
// Both succeeded - merge characteristics
|
|
301
|
-
return {
|
|
302
|
-
status: 'success' as const,
|
|
303
|
-
data: mergeCharacteristics(
|
|
304
|
-
photoOutput.result.data,
|
|
305
|
-
descOutput.result.data
|
|
306
|
-
),
|
|
307
|
-
};
|
|
308
|
-
} else if (photoOutput.result.status === 'success') {
|
|
309
|
-
// Photo succeeded, use that
|
|
310
|
-
return photoOutput.result;
|
|
311
|
-
} else if (descOutput.result.status === 'success') {
|
|
312
|
-
// Description succeeded, use that
|
|
313
|
-
return descOutput.result;
|
|
314
|
-
} else {
|
|
315
|
-
// Both failed
|
|
316
|
-
return {
|
|
317
|
-
status: 'error' as const,
|
|
318
|
-
error: 'both-analyzers-failed' as const,
|
|
319
|
-
};
|
|
320
|
-
}
|
|
321
|
-
}
|
|
322
|
-
```
|
|
323
|
-
|
|
324
|
-
### Intelligent Retry with Tracking
|
|
325
|
-
|
|
326
|
-
Use the built-in `withRetry` helper for automatic retry with exponential backoff:
|
|
327
|
-
|
|
328
|
-
```typescript
|
|
329
|
-
import { withRetry } from '@vibe-agent-toolkit/agent-runtime';
|
|
330
|
-
import {
|
|
331
|
-
LLM_TIMEOUT,
|
|
332
|
-
LLM_RATE_LIMIT,
|
|
333
|
-
LLM_UNAVAILABLE,
|
|
334
|
-
RETRYABLE_LLM_ERRORS,
|
|
335
|
-
RETRYABLE_EVENT_ERRORS,
|
|
336
|
-
} from '@vibe-agent-toolkit/agent-schema';
|
|
337
|
-
|
|
338
|
-
// Use the withRetry helper (built into agent-runtime)
|
|
339
|
-
const output = await withRetry(
|
|
340
|
-
() => photoAnalyzer.execute({ imagePath }),
|
|
341
|
-
5 // max attempts
|
|
342
|
-
);
|
|
343
|
-
|
|
344
|
-
if (output.result.status === 'success') {
|
|
345
|
-
console.log('Success after', output.result.execution?.retryCount ?? 0, 'retries');
|
|
346
|
-
console.log('Total duration:', output.result.execution?.durationMs, 'ms');
|
|
347
|
-
console.log('Total cost:', output.result.execution?.cost);
|
|
348
|
-
} else {
|
|
349
|
-
console.error('Failed after', output.result.execution?.retryCount ?? 0, 'retries');
|
|
350
|
-
console.error('Error type:', output.result.error);
|
|
351
|
-
}
|
|
352
|
-
```
|
|
353
|
-
|
|
354
|
-
**How it works:**
|
|
355
|
-
|
|
356
|
-
1. **Error classification**: Agents return error constants (e.g., `LLM_TIMEOUT`, `LLM_RATE_LIMIT`)
|
|
357
|
-
2. **Retryability**: Error types imply retryability via exported sets:
|
|
358
|
-
- `RETRYABLE_LLM_ERRORS`: `[LLM_TIMEOUT, LLM_RATE_LIMIT, LLM_UNAVAILABLE]`
|
|
359
|
-
- `RETRYABLE_EVENT_ERRORS`: `[EVENT_TIMEOUT, EVENT_UNAVAILABLE]`
|
|
360
|
-
3. **Backoff delays**: Different error types have different base delays:
|
|
361
|
-
- `LLM_RATE_LIMIT`: 5000ms (rate limits need longer waits)
|
|
362
|
-
- `LLM_TIMEOUT`: 1000ms (timeouts can retry quickly)
|
|
363
|
-
- `LLM_UNAVAILABLE`: 10000ms (service issues need long waits)
|
|
364
|
-
4. **Exponential backoff**: Each retry doubles the delay (capped at 30 seconds)
|
|
365
|
-
5. **Accumulated metrics**: The helper accumulates `durationMs`, `tokensUsed`, and `cost` across all attempts
|
|
366
|
-
6. **Retry count injection**: The orchestrator injects `retryCount` into `ExecutionMetadata`
|
|
367
|
-
|
|
368
|
-
**Custom retry logic:**
|
|
369
|
-
|
|
370
|
-
If you need custom retry behavior, you can use the standard error sets:
|
|
371
|
-
|
|
372
|
-
```typescript
|
|
373
|
-
import { RETRYABLE_LLM_ERRORS } from '@vibe-agent-toolkit/agent-schema';
|
|
374
|
-
|
|
375
|
-
function isRetryable(error: string): boolean {
|
|
376
|
-
return RETRYABLE_LLM_ERRORS.has(error as LLMError) ||
|
|
377
|
-
RETRYABLE_EVENT_ERRORS.has(error as ExternalEventError);
|
|
378
|
-
}
|
|
379
|
-
```
|
|
380
|
-
|
|
381
|
-
### Conversational Multi-Turn
|
|
382
|
-
|
|
383
|
-
Handle multi-turn conversations with state:
|
|
384
|
-
|
|
385
|
-
```typescript
|
|
386
|
-
async function runConversation() {
|
|
387
|
-
let session = {
|
|
388
|
-
history: [],
|
|
389
|
-
state: { profile: { conversationPhase: 'gathering' as const } },
|
|
390
|
-
};
|
|
391
|
-
|
|
392
|
-
while (true) {
|
|
393
|
-
const userMessage = await getUserInput();
|
|
394
|
-
|
|
395
|
-
const output = await breedAdvisor.execute({
|
|
396
|
-
message: userMessage,
|
|
397
|
-
sessionState: session.state,
|
|
398
|
-
});
|
|
399
|
-
|
|
400
|
-
// Show agent's reply
|
|
401
|
-
console.log('Agent:', output.reply);
|
|
402
|
-
|
|
403
|
-
// Update session for next turn
|
|
404
|
-
session = {
|
|
405
|
-
history: [
|
|
406
|
-
...session.history,
|
|
407
|
-
{ role: 'user', content: userMessage },
|
|
408
|
-
{ role: 'assistant', content: output.reply },
|
|
409
|
-
],
|
|
410
|
-
state: output.sessionState,
|
|
411
|
-
};
|
|
412
|
-
|
|
413
|
-
// Check if conversation is complete
|
|
414
|
-
if (output.result.status === 'success') {
|
|
415
|
-
console.log('Final recommendation:', output.result.data);
|
|
416
|
-
break;
|
|
417
|
-
} else if (output.result.status === 'error') {
|
|
418
|
-
console.error('Conversation failed:', output.result.error);
|
|
419
|
-
break;
|
|
420
|
-
}
|
|
421
|
-
|
|
422
|
-
// status === 'in-progress', continue conversation
|
|
423
|
-
}
|
|
424
|
-
}
|
|
425
|
-
```
|
|
426
|
-
|
|
427
|
-
### Fan-Out/Fan-In
|
|
428
|
-
|
|
429
|
-
Execute multiple agents and combine results:
|
|
430
|
-
|
|
431
|
-
```typescript
|
|
432
|
-
async function generateMultipleNames(
|
|
433
|
-
characteristics: CatCharacteristics,
|
|
434
|
-
count: number = 3
|
|
435
|
-
) {
|
|
436
|
-
// Generate multiple names in parallel
|
|
437
|
-
const outputs = await Promise.all(
|
|
438
|
-
Array.from({ length: count }, () =>
|
|
439
|
-
nameGenerator.execute({ characteristics })
|
|
440
|
-
)
|
|
441
|
-
);
|
|
442
|
-
|
|
443
|
-
// Collect successful results
|
|
444
|
-
const successfulNames = outputs
|
|
445
|
-
.map(output => output.result)
|
|
446
|
-
.filter(result => result.status === 'success')
|
|
447
|
-
.map(result => result.data);
|
|
448
|
-
|
|
449
|
-
if (successfulNames.length === 0) {
|
|
450
|
-
return {
|
|
451
|
-
status: 'error' as const,
|
|
452
|
-
error: 'all-generations-failed' as const,
|
|
453
|
-
};
|
|
454
|
-
}
|
|
455
|
-
|
|
456
|
-
return {
|
|
457
|
-
status: 'success' as const,
|
|
458
|
-
data: {
|
|
459
|
-
suggestions: successfulNames,
|
|
460
|
-
count: successfulNames.length,
|
|
461
|
-
},
|
|
462
|
-
};
|
|
463
|
-
}
|
|
464
|
-
```
|
|
465
|
-
|
|
466
|
-
### Validation Loop
|
|
467
|
-
|
|
468
|
-
Generator + Validator feedback loop:
|
|
469
|
-
|
|
470
|
-
```typescript
|
|
471
|
-
async function generateValidName(
|
|
472
|
-
characteristics: CatCharacteristics,
|
|
473
|
-
maxAttempts: number = 5
|
|
474
|
-
): Promise<AgentResult<string, 'max-attempts-exceeded'>> {
|
|
475
|
-
for (let attempt = 0; attempt < maxAttempts; attempt++) {
|
|
476
|
-
// Generate name
|
|
477
|
-
const nameOutput = await nameGenerator.execute({ characteristics });
|
|
478
|
-
if (nameOutput.result.status === 'error') {
|
|
479
|
-
continue; // Try again
|
|
480
|
-
}
|
|
481
|
-
|
|
482
|
-
const { name } = nameOutput.result.data;
|
|
483
|
-
|
|
484
|
-
// Validate name
|
|
485
|
-
const validationOutput = await nameValidator.execute({
|
|
486
|
-
name,
|
|
487
|
-
characteristics,
|
|
488
|
-
});
|
|
489
|
-
|
|
490
|
-
if (validationOutput.result.status === 'success' &&
|
|
491
|
-
validationOutput.result.data.valid) {
|
|
492
|
-
// Found valid name!
|
|
493
|
-
return { status: 'success', data: name };
|
|
494
|
-
}
|
|
495
|
-
|
|
496
|
-
// Invalid, try again with feedback
|
|
497
|
-
console.log('Attempt', attempt + 1, 'invalid:', validationOutput.result.data.reason);
|
|
498
|
-
}
|
|
499
|
-
|
|
500
|
-
return { status: 'error', error: 'max-attempts-exceeded' };
|
|
501
|
-
}
|
|
502
|
-
```
|
|
503
|
-
|
|
504
|
-
### Human-in-the-Loop
|
|
505
|
-
|
|
506
|
-
Integrate human approval into workflows:
|
|
507
|
-
|
|
508
|
-
```typescript
|
|
509
|
-
import { humanApprovalAgent } from '@vibe-agent-toolkit/vat-example-cat-agents';
|
|
510
|
-
|
|
511
|
-
async function generateWithApproval(
|
|
512
|
-
characteristics: CatCharacteristics
|
|
513
|
-
) {
|
|
514
|
-
// Generate name
|
|
515
|
-
const nameOutput = await nameGenerator.execute({ characteristics });
|
|
516
|
-
if (nameOutput.result.status === 'error') {
|
|
517
|
-
return nameOutput.result;
|
|
518
|
-
}
|
|
519
|
-
|
|
520
|
-
const { name, reasoning } = nameOutput.result.data;
|
|
521
|
-
|
|
522
|
-
// Request human approval
|
|
523
|
-
const approvalOutput = await humanApprovalAgent.execute({
|
|
524
|
-
actionType: 'cat-name-generation',
|
|
525
|
-
content: { name, reasoning },
|
|
526
|
-
context: {
|
|
527
|
-
characteristics: {
|
|
528
|
-
physical: characteristics.physical,
|
|
529
|
-
behavioral: characteristics.behavioral,
|
|
530
|
-
},
|
|
531
|
-
},
|
|
532
|
-
});
|
|
533
|
-
|
|
534
|
-
if (approvalOutput.result.status === 'error') {
|
|
535
|
-
return { status: 'error' as const, error: 'approval-failed' as const };
|
|
536
|
-
}
|
|
537
|
-
|
|
538
|
-
const { approved, feedback } = approvalOutput.result.data;
|
|
539
|
-
|
|
540
|
-
if (approved) {
|
|
541
|
-
return { status: 'success' as const, data: name };
|
|
542
|
-
} else {
|
|
543
|
-
return {
|
|
544
|
-
status: 'error' as const,
|
|
545
|
-
error: 'rejected-by-human' as const,
|
|
546
|
-
};
|
|
547
|
-
}
|
|
548
|
-
}
|
|
549
|
-
```
|
|
550
|
-
|
|
551
|
-
## Observability and Production Monitoring
|
|
552
|
-
|
|
553
|
-
Result envelopes support optional observability fields for production monitoring, intelligent orchestration, and debugging.
|
|
554
|
-
|
|
555
|
-
### Confidence
|
|
556
|
-
|
|
557
|
-
Indicates certainty in the result (0-1 scale), enabling intelligent orchestration decisions:
|
|
558
|
-
|
|
559
|
-
```typescript
|
|
560
|
-
import { RESULT_SUCCESS } from '@vibe-agent-toolkit/agent-schema';
|
|
561
|
-
|
|
562
|
-
// Agent returns confidence
|
|
563
|
-
const output = await photoAnalyzer.execute({ imagePath });
|
|
564
|
-
|
|
565
|
-
if (output.result.status === RESULT_SUCCESS) {
|
|
566
|
-
const confidence = output.result.confidence ?? 1.0;
|
|
567
|
-
|
|
568
|
-
if (confidence < 0.8) {
|
|
569
|
-
// Low confidence - verify with another agent
|
|
570
|
-
const secondOpinion = await backupAnalyzer.execute({ imagePath });
|
|
571
|
-
// Compare results...
|
|
572
|
-
}
|
|
573
|
-
}
|
|
574
|
-
```
|
|
575
|
-
|
|
576
|
-
**Use cases:**
|
|
577
|
-
- **Orchestration decisions**: Retry if confidence < threshold
|
|
578
|
-
- **Chain validation**: Verify uncertain results with another agent
|
|
579
|
-
- **User transparency**: Show uncertainty to users ("I'm 70% confident")
|
|
580
|
-
- **Stopping criteria**: Iterate until confidence > 0.9
|
|
581
|
-
|
|
582
|
-
### Warnings
|
|
583
|
-
|
|
584
|
-
Non-fatal issues array for graceful degradation:
|
|
585
|
-
|
|
586
|
-
```typescript
|
|
587
|
-
const output = await photoAnalyzer.execute({ imagePath });
|
|
588
|
-
|
|
589
|
-
if (output.result.status === RESULT_SUCCESS) {
|
|
590
|
-
// Check for warnings
|
|
591
|
-
if (output.result.warnings && output.result.warnings.length > 0) {
|
|
592
|
-
console.warn('Warnings:', output.result.warnings);
|
|
593
|
-
// Example: ['Image quality was poor, confidence may be lower']
|
|
594
|
-
}
|
|
595
|
-
|
|
596
|
-
// Use data despite warnings
|
|
597
|
-
console.log('Characteristics:', output.result.data);
|
|
598
|
-
}
|
|
599
|
-
```
|
|
600
|
-
|
|
601
|
-
**Use cases:**
|
|
602
|
-
- **User transparency**: Show quality issues or caveats
|
|
603
|
-
- **Context for downstream agents**: Pass warnings along pipeline
|
|
604
|
-
- **Logging/debugging**: Track degraded but successful operations
|
|
605
|
-
|
|
606
|
-
### Execution Metadata
|
|
607
|
-
|
|
608
|
-
Performance and cost tracking for production systems:
|
|
609
|
-
|
|
610
|
-
```typescript
|
|
611
|
-
import { withTiming } from '@vibe-agent-toolkit/agent-runtime';
|
|
612
|
-
|
|
613
|
-
// Wrap with timing helper
|
|
614
|
-
const output = await withTiming(() =>
|
|
615
|
-
photoAnalyzer.execute({ imagePath })
|
|
616
|
-
);
|
|
617
|
-
|
|
618
|
-
if (output.result.execution) {
|
|
619
|
-
console.log('Duration:', output.result.execution.durationMs, 'ms');
|
|
620
|
-
console.log('Tokens:', output.result.execution.tokensUsed);
|
|
621
|
-
console.log('Cost:', output.result.execution.cost, 'USD');
|
|
622
|
-
console.log('Model:', output.result.execution.model);
|
|
623
|
-
console.log('Provider:', output.result.execution.provider);
|
|
624
|
-
console.log('Retry count:', output.result.execution.retryCount ?? 0);
|
|
625
|
-
console.log('Timestamp:', output.result.execution.timestamp);
|
|
626
|
-
}
|
|
627
|
-
```
|
|
628
|
-
|
|
629
|
-
**Fields:**
|
|
630
|
-
- `durationMs`: Total execution duration (including retries)
|
|
631
|
-
- `tokensUsed`: LLM tokens consumed (sum across all LLM calls)
|
|
632
|
-
- `cost`: Estimated cost in USD
|
|
633
|
-
- `model`: Model identifier for A/B testing
|
|
634
|
-
- `provider`: Provider for cost attribution
|
|
635
|
-
- `retryCount`: Set by orchestrator's retry wrapper (0 = no retries)
|
|
636
|
-
- `timestamp`: Execution start time (ISO 8601)
|
|
637
|
-
|
|
638
|
-
**Use cases:**
|
|
639
|
-
- **Performance optimization**: Identify slow agents
|
|
640
|
-
- **Cost attribution**: Multi-tenant systems
|
|
641
|
-
- **Usage analytics**: Tokens per request
|
|
642
|
-
- **Debugging**: View retry behavior
|
|
643
|
-
|
|
644
|
-
### Combining Observability Features
|
|
645
|
-
|
|
646
|
-
```typescript
|
|
647
|
-
import { withRetry, withTiming } from '@vibe-agent-toolkit/agent-runtime';
|
|
648
|
-
import { RESULT_SUCCESS } from '@vibe-agent-toolkit/agent-schema';
|
|
649
|
-
|
|
650
|
-
// Combine timing and retry
|
|
651
|
-
const output = await withRetry(
|
|
652
|
-
() => withTiming(() => photoAnalyzer.execute({ imagePath })),
|
|
653
|
-
5
|
|
654
|
-
);
|
|
655
|
-
|
|
656
|
-
if (output.result.status === RESULT_SUCCESS) {
|
|
657
|
-
const { confidence, warnings, execution } = output.result;
|
|
658
|
-
|
|
659
|
-
// Log comprehensive observability data
|
|
660
|
-
console.log({
|
|
661
|
-
confidence: confidence ?? 1.0,
|
|
662
|
-
warnings: warnings ?? [],
|
|
663
|
-
duration: execution?.durationMs,
|
|
664
|
-
retries: execution?.retryCount ?? 0,
|
|
665
|
-
cost: execution?.cost,
|
|
666
|
-
tokens: execution?.tokensUsed,
|
|
667
|
-
});
|
|
668
|
-
|
|
669
|
-
// Make orchestration decisions based on confidence
|
|
670
|
-
if (confidence && confidence < 0.7) {
|
|
671
|
-
console.warn('Low confidence result, consider verification');
|
|
672
|
-
}
|
|
673
|
-
|
|
674
|
-
// Alert on excessive retries
|
|
675
|
-
if (execution?.retryCount && execution.retryCount > 3) {
|
|
676
|
-
console.error('Agent required excessive retries, investigate');
|
|
677
|
-
}
|
|
678
|
-
}
|
|
679
|
-
```
|
|
680
|
-
|
|
681
|
-
## Testing Patterns
|
|
682
|
-
|
|
683
|
-
### Test Helpers
|
|
684
|
-
|
|
685
|
-
Use test helpers from `@vibe-agent-toolkit/agent-runtime`:
|
|
686
|
-
|
|
687
|
-
```typescript
|
|
688
|
-
import { resultMatchers } from '@vibe-agent-toolkit/agent-runtime';
|
|
689
|
-
import { describe, expect, it } from 'vitest';
|
|
690
|
-
|
|
691
|
-
describe('haikuValidator', () => {
|
|
692
|
-
it('should return success for valid haiku', async () => {
|
|
693
|
-
const output = await haikuValidator.execute({
|
|
694
|
-
text: 'Ancient pond, frog leaps in, splash',
|
|
695
|
-
syllables: [5, 7, 5],
|
|
696
|
-
kigo: 'pond',
|
|
697
|
-
kireji: 'splash',
|
|
698
|
-
});
|
|
699
|
-
|
|
700
|
-
// Type-safe assertions
|
|
701
|
-
resultMatchers.expectSuccess(output.result);
|
|
702
|
-
expect(output.result.data.valid).toBe(true);
|
|
703
|
-
});
|
|
704
|
-
|
|
705
|
-
it('should return error for invalid syllables', async () => {
|
|
706
|
-
const output = await haikuValidator.execute({
|
|
707
|
-
text: 'Too many syllables here, not a haiku, nope',
|
|
708
|
-
syllables: [7, 7, 5],
|
|
709
|
-
kigo: 'syllables',
|
|
710
|
-
kireji: 'nope',
|
|
711
|
-
});
|
|
712
|
-
|
|
713
|
-
resultMatchers.expectError(output.result);
|
|
714
|
-
expect(output.result.error).toBe('invalid-syllables');
|
|
715
|
-
});
|
|
716
|
-
});
|
|
717
|
-
```
|
|
718
|
-
|
|
719
|
-
### Mocking External Events
|
|
720
|
-
|
|
721
|
-
Mock external event integrators in tests:
|
|
722
|
-
|
|
723
|
-
```typescript
|
|
724
|
-
import { vi } from 'vitest';
|
|
725
|
-
|
|
726
|
-
describe('workflow with approval', () => {
|
|
727
|
-
it('should handle approved case', async () => {
|
|
728
|
-
// Mock approval agent to auto-approve
|
|
729
|
-
const mockApproval = vi.fn().mockResolvedValue({
|
|
730
|
-
result: {
|
|
731
|
-
status: 'success',
|
|
732
|
-
data: { approved: true, feedback: null },
|
|
733
|
-
},
|
|
734
|
-
});
|
|
735
|
-
|
|
736
|
-
const result = await generateWithApproval(characteristics, {
|
|
737
|
-
approvalAgent: { execute: mockApproval },
|
|
738
|
-
});
|
|
739
|
-
|
|
740
|
-
resultMatchers.expectSuccess(result);
|
|
741
|
-
expect(mockApproval).toHaveBeenCalledOnce();
|
|
742
|
-
});
|
|
743
|
-
});
|
|
744
|
-
```
|
|
745
|
-
|
|
746
|
-
## Best Practices
|
|
747
|
-
|
|
748
|
-
### 1. Always Handle Errors
|
|
749
|
-
|
|
750
|
-
Never assume success - always check result status:
|
|
751
|
-
|
|
752
|
-
```typescript
|
|
753
|
-
// ❌ BAD - Assumes success
|
|
754
|
-
const output = await analyzer.execute(input);
|
|
755
|
-
const data = output.result.data; // TypeScript error if result is error!
|
|
756
|
-
|
|
757
|
-
// ✅ GOOD - Check status first
|
|
758
|
-
const output = await analyzer.execute(input);
|
|
759
|
-
if (output.result.status === 'success') {
|
|
760
|
-
const data = output.result.data; // Type-safe
|
|
761
|
-
console.log(data);
|
|
762
|
-
} else {
|
|
763
|
-
console.error('Failed:', output.result.error);
|
|
764
|
-
}
|
|
765
|
-
```
|
|
766
|
-
|
|
767
|
-
### 2. Use Helpers for Composition
|
|
768
|
-
|
|
769
|
-
Use `andThen()` and `mapResult()` instead of manual checks:
|
|
770
|
-
|
|
771
|
-
```typescript
|
|
772
|
-
// ❌ VERBOSE - Manual checks
|
|
773
|
-
const output1 = await agent1.execute(input);
|
|
774
|
-
let output2;
|
|
775
|
-
if (output1.result.status === 'success') {
|
|
776
|
-
output2 = await agent2.execute(output1.result.data);
|
|
777
|
-
} else {
|
|
778
|
-
output2 = { result: output1.result };
|
|
779
|
-
}
|
|
780
|
-
|
|
781
|
-
// ✅ CONCISE - Use andThen
|
|
782
|
-
const output1 = await agent1.execute(input);
|
|
783
|
-
const output2 = await andThen(output1.result, (data) =>
|
|
784
|
-
agent2.execute(data).then(out => out.result)
|
|
785
|
-
);
|
|
786
|
-
```
|
|
787
|
-
|
|
788
|
-
### 3. Leverage Discriminated Unions
|
|
789
|
-
|
|
790
|
-
TypeScript's discriminated unions provide type safety:
|
|
791
|
-
|
|
792
|
-
```typescript
|
|
793
|
-
function handleResult(result: AgentResult<string, LLMError>) {
|
|
794
|
-
// TypeScript knows which properties are available
|
|
795
|
-
if (result.status === 'success') {
|
|
796
|
-
console.log(result.data); // ✅ data available
|
|
797
|
-
console.log(result.error); // ❌ TypeScript error
|
|
798
|
-
} else {
|
|
799
|
-
console.log(result.error); // ✅ error available
|
|
800
|
-
console.log(result.data); // ❌ TypeScript error
|
|
801
|
-
}
|
|
802
|
-
}
|
|
803
|
-
```
|
|
804
|
-
|
|
805
|
-
### 4. Define Clear Error Types
|
|
806
|
-
|
|
807
|
-
Use enums or literal unions for errors:
|
|
808
|
-
|
|
809
|
-
```typescript
|
|
810
|
-
// ✅ GOOD - Clear error types
|
|
811
|
-
type ValidationError =
|
|
812
|
-
| 'invalid-syllables'
|
|
813
|
-
| 'missing-kigo'
|
|
814
|
-
| 'too-long';
|
|
815
|
-
|
|
816
|
-
// ❌ BAD - Generic string
|
|
817
|
-
type ValidationError = string;
|
|
818
|
-
```
|
|
819
|
-
|
|
820
|
-
### 5. Document Metadata Schemas
|
|
821
|
-
|
|
822
|
-
For conversational agents, document metadata structure:
|
|
823
|
-
|
|
824
|
-
```typescript
|
|
825
|
-
/**
|
|
826
|
-
* Metadata for breed advisor in-progress state
|
|
827
|
-
*/
|
|
828
|
-
interface BreedAdvisorMetadata {
|
|
829
|
-
/** Number of factors collected so far */
|
|
830
|
-
factorsCollected: number;
|
|
831
|
-
/** Minimum factors required for recommendations */
|
|
832
|
-
requiredFactors: number;
|
|
833
|
-
/** Current conversation phase */
|
|
834
|
-
conversationPhase: 'gathering' | 'ready-to-recommend' | 'refining';
|
|
835
|
-
/** Current breed recommendations (if in refining phase) */
|
|
836
|
-
recommendations?: Array<{ breed: string; score: number }>;
|
|
837
|
-
}
|
|
838
|
-
```
|
|
839
|
-
|
|
840
|
-
## Related Documentation
|
|
841
|
-
|
|
842
|
-
- [Agent Authoring Guide](agent-authoring.md) - How to create agents with result envelopes
|
|
843
|
-
- [Architecture Overview]() - Package structure and design principles
|
|
844
|
-
|
|
845
|
-
## Examples
|
|
846
|
-
|
|
847
|
-
See the `@vibe-agent-toolkit/vat-example-cat-agents` package for complete working examples:
|
|
848
|
-
|
|
849
|
-
- **Pure Function Tool**: `haiku-validator` - Validation with typed errors
|
|
850
|
-
- **One-Shot LLM Analyzer**: `photo-analyzer`, `description-parser` - LLM with error handling
|
|
851
|
-
- **Conversational Assistant**: `breed-advisor` - Multi-turn with stateful results
|
|
852
|
-
- **External Event Integrator**: `human-approval` - Event integration with timeouts
|
|
853
|
-
|
|
854
|
-
Run demos:
|
|
855
|
-
```bash
|
|
856
|
-
cd packages/vat-example-cat-agents
|
|
857
|
-
bun run demo:photos # Photo analysis pipeline
|
|
858
|
-
bun run demo:conversation # Interactive breed advisor
|
|
859
|
-
```
|