@vibe-agent-toolkit/vat-development-agents 0.1.29 → 0.1.30-rc.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (33) hide show
  1. package/dist/.claude/plugins/marketplaces/vat-skills/CHANGELOG.md +20 -0
  2. package/dist/.claude/plugins/marketplaces/vat-skills/plugins/vibe-agent-toolkit/.claude-plugin/plugin.json +1 -1
  3. package/dist/.claude/plugins/marketplaces/vat-skills/plugins/vibe-agent-toolkit/skills/audit/SKILL.md +21 -16
  4. package/dist/.claude/plugins/marketplaces/vat-skills/plugins/vibe-agent-toolkit/skills/authoring/SKILL.md +30 -12
  5. package/dist/.claude/plugins/marketplaces/vat-skills/plugins/vibe-agent-toolkit/skills/authoring/resources/skill-quality-checklist.md +42 -0
  6. package/dist/.claude/plugins/marketplaces/vat-skills/plugins/vibe-agent-toolkit/skills/debugging/SKILL.md +4 -4
  7. package/dist/.claude/plugins/marketplaces/vat-skills/plugins/vibe-agent-toolkit/skills/vibe-agent-toolkit/SKILL.md +6 -6
  8. package/dist/generated/resources/skills/vat-agent-authoring.js +3 -3
  9. package/dist/generated/resources/skills/vat-audit.js +5 -5
  10. package/dist/skills/audit/SKILL.md +21 -16
  11. package/dist/skills/authoring/SKILL.md +30 -12
  12. package/dist/skills/authoring/resources/skill-quality-checklist.md +42 -0
  13. package/dist/skills/debugging/SKILL.md +4 -4
  14. package/dist/skills/vibe-agent-toolkit/SKILL.md +6 -6
  15. package/package.json +4 -4
  16. package/dist/.claude/plugins/marketplaces/vat-skills/plugins/vibe-agent-toolkit/skills/debugging/resources/CLAUDE.md +0 -539
  17. package/dist/.claude/plugins/marketplaces/vat-skills/plugins/vibe-agent-toolkit/skills/debugging/resources/debug-and-test-vat-fixes.md +0 -111
  18. package/dist/.claude/plugins/marketplaces/vat-skills/plugins/vibe-agent-toolkit/skills/debugging/resources/writing-tests.md +0 -577
  19. package/dist/.claude/plugins/marketplaces/vat-skills/plugins/vibe-agent-toolkit/skills/vibe-agent-toolkit/resources/adding-runtime-adapters.md +0 -628
  20. package/dist/.claude/plugins/marketplaces/vat-skills/plugins/vibe-agent-toolkit/skills/vibe-agent-toolkit/resources/agent-authoring.md +0 -905
  21. package/dist/.claude/plugins/marketplaces/vat-skills/plugins/vibe-agent-toolkit/skills/vibe-agent-toolkit/resources/compiling-markdown-to-typescript.md +0 -501
  22. package/dist/.claude/plugins/marketplaces/vat-skills/plugins/vibe-agent-toolkit/skills/vibe-agent-toolkit/resources/getting-started.md +0 -360
  23. package/dist/.claude/plugins/marketplaces/vat-skills/plugins/vibe-agent-toolkit/skills/vibe-agent-toolkit/resources/orchestration.md +0 -859
  24. package/dist/.claude/plugins/marketplaces/vat-skills/plugins/vibe-agent-toolkit/skills/vibe-agent-toolkit/resources/rag-usage-guide.md +0 -770
  25. package/dist/skills/debugging/resources/CLAUDE.md +0 -539
  26. package/dist/skills/debugging/resources/debug-and-test-vat-fixes.md +0 -111
  27. package/dist/skills/debugging/resources/writing-tests.md +0 -577
  28. package/dist/skills/vibe-agent-toolkit/resources/adding-runtime-adapters.md +0 -628
  29. package/dist/skills/vibe-agent-toolkit/resources/agent-authoring.md +0 -905
  30. package/dist/skills/vibe-agent-toolkit/resources/compiling-markdown-to-typescript.md +0 -501
  31. package/dist/skills/vibe-agent-toolkit/resources/getting-started.md +0 -360
  32. package/dist/skills/vibe-agent-toolkit/resources/orchestration.md +0 -859
  33. package/dist/skills/vibe-agent-toolkit/resources/rag-usage-guide.md +0 -770
@@ -1,859 +0,0 @@
1
- # Agent Orchestration Guide
2
-
3
- ## Introduction
4
-
5
- This guide shows how to orchestrate VAT agents using standardized result envelopes and Railway-Oriented Programming (ROP) patterns. All VAT agents return consistent result types, enabling type-safe composition, error handling, and complex workflows.
6
-
7
- ## Core Concepts
8
-
9
- ### Result Types
10
-
11
- VAT defines two core result types in `@vibe-agent-toolkit/agent-schema`:
12
-
13
- #### AgentResult<TData, TError>
14
-
15
- For single-execution agents (pure functions, one-shot LLM analyzers):
16
-
17
- ```typescript
18
- type AgentResult<TData, TError> =
19
- | { status: 'success'; data: TData }
20
- | { status: 'error'; error: TError };
21
- ```
22
-
23
- **Examples:**
24
- - Pure function validators return `AgentResult<ValidationData, ValidationError>`
25
- - One-shot LLM analyzers return `AgentResult<AnalysisData, LLMError>`
26
-
27
- #### StatefulAgentResult<TData, TError, TMetadata>
28
-
29
- For multi-turn conversational agents that maintain state:
30
-
31
- ```typescript
32
- type StatefulAgentResult<TData, TError, TMetadata> =
33
- | { status: 'in-progress'; metadata?: TMetadata }
34
- | { status: 'success'; data: TData }
35
- | { status: 'error'; error: TError };
36
- ```
37
-
38
- **Examples:**
39
- - Conversational assistants return `StatefulAgentResult<FinalData, ConversationError, ProgressMetadata>`
40
- - Multi-step workflows return `StatefulAgentResult<WorkflowResult, WorkflowError, StepMetadata>`
41
-
42
- ### Output Envelopes
43
-
44
- Agents return output envelopes that wrap results with additional context:
45
-
46
- #### OneShotAgentOutput<TData, TError>
47
-
48
- ```typescript
49
- interface OneShotAgentOutput<TData, TError> {
50
- result: AgentResult<TData, TError>;
51
- }
52
- ```
53
-
54
- **Use for:**
55
- - Pure function tools (validators, transformers)
56
- - One-shot LLM analyzers (photo analysis, text parsing)
57
- - External event integrators (approval gates, webhooks)
58
-
59
- **Example:**
60
- ```typescript
61
- const output = await haikuValidator.execute({
62
- text: 'Ancient pond, frog leaps in, splash',
63
- syllables: [5, 7, 5],
64
- });
65
-
66
- // output.result is AgentResult<ValidationData, ValidationError>
67
- if (output.result.status === 'success') {
68
- console.log('Valid:', output.result.data.valid);
69
- } else {
70
- console.error('Invalid:', output.result.error);
71
- }
72
- ```
73
-
74
- #### ConversationalAgentOutput<TData, TError, TState>
75
-
76
- ```typescript
77
- interface ConversationalAgentOutput<TData, TError, TState> {
78
- reply: string; // Natural language response
79
- sessionState: TState; // Updated session state
80
- result: StatefulAgentResult<TData, TError, TMetadata>; // Machine-readable result
81
- }
82
- ```
83
-
84
- **Use for:**
85
- - Multi-turn conversational assistants
86
- - Progressive data collection agents
87
- - Workflows with user interaction
88
-
89
- **Example:**
90
- ```typescript
91
- const output = await breedAdvisor.execute({
92
- message: 'I love classical music',
93
- sessionState: { profile: { conversationPhase: 'gathering' } },
94
- });
95
-
96
- // output.reply - Natural language for user
97
- console.log('Agent says:', output.reply);
98
-
99
- // output.sessionState - Carry to next turn
100
- const nextInput = { message: 'I live in an apartment', sessionState: output.sessionState };
101
-
102
- // output.result - Machine-readable status
103
- if (output.result.status === 'in-progress') {
104
- console.log('Gathering info, progress:', output.result.metadata);
105
- } else if (output.result.status === 'success') {
106
- console.log('Recommendation:', output.result.data);
107
- }
108
- ```
109
-
110
- ### Standard Error Types
111
-
112
- #### LLMError
113
-
114
- For LLM-related failures:
115
-
116
- ```typescript
117
- type LLMError =
118
- | 'llm-refusal' // LLM refused to generate output
119
- | 'llm-invalid-output' // Output didn't match expected format
120
- | 'llm-timeout' // Request timed out
121
- | 'llm-rate-limit' // Hit rate limit
122
- | 'llm-token-limit' // Exceeded token limit
123
- | 'llm-unavailable'; // Service unavailable
124
- ```
125
-
126
- #### ExternalEventError
127
-
128
- For external system integration failures:
129
-
130
- ```typescript
131
- type ExternalEventError =
132
- | 'event-timeout' // External event timed out
133
- | 'event-unavailable' // External system unavailable
134
- | 'event-rejected' // External system rejected request
135
- | 'event-invalid-response'; // External system returned invalid data
136
- ```
137
-
138
- #### Custom Error Types
139
-
140
- Agents can define domain-specific errors:
141
-
142
- ```typescript
143
- type ValidationError =
144
- | 'invalid-syllables'
145
- | 'missing-kigo'
146
- | 'missing-kireji'
147
- | 'too-long'
148
- | 'too-short';
149
- ```
150
-
151
- ## Result Helpers
152
-
153
- The `@vibe-agent-toolkit/agent-runtime` package provides helper functions for working with results.
154
-
155
- ### mapResult()
156
-
157
- Transform success data while preserving errors:
158
-
159
- ```typescript
160
- import { mapResult } from '@vibe-agent-toolkit/agent-runtime';
161
-
162
- const result1 = { status: 'success' as const, data: 10 };
163
- const result2 = mapResult(result1, (n) => n * 2);
164
- // result2 = { status: 'success', data: 20 }
165
-
166
- const result3 = { status: 'error' as const, error: 'failed' };
167
- const result4 = mapResult(result3, (n) => n * 2);
168
- // result4 = { status: 'error', error: 'failed' } (unchanged)
169
- ```
170
-
171
- ### andThen()
172
-
173
- Chain operations that return results (monadic bind):
174
-
175
- ```typescript
176
- import { andThen } from '@vibe-agent-toolkit/agent-runtime';
177
-
178
- const result1 = { status: 'success' as const, data: 10 };
179
- const result2 = andThen(result1, (n) => {
180
- if (n > 5) {
181
- return { status: 'success' as const, data: n * 2 };
182
- }
183
- return { status: 'error' as const, error: 'too-small' };
184
- });
185
- // result2 = { status: 'success', data: 20 }
186
-
187
- const result3 = { status: 'error' as const, error: 'failed' };
188
- const result4 = andThen(result3, (n) => {
189
- return { status: 'success' as const, data: n * 2 };
190
- });
191
- // result4 = { status: 'error', error: 'failed' } (not called)
192
- ```
193
-
194
- ### match()
195
-
196
- Pattern match on result status:
197
-
198
- ```typescript
199
- import { match } from '@vibe-agent-toolkit/agent-runtime';
200
-
201
- const result = { status: 'success' as const, data: 'hello' };
202
-
203
- const message = match(result, {
204
- success: (data) => `Success: ${data}`,
205
- error: (err) => `Error: ${err}`,
206
- });
207
- // message = "Success: hello"
208
-
209
- // For stateful results, add inProgress handler
210
- const statefulResult = { status: 'in-progress' as const, metadata: { step: 2 } };
211
-
212
- const status = match(statefulResult, {
213
- success: (data) => `Done: ${data}`,
214
- error: (err) => `Failed: ${err}`,
215
- inProgress: (meta) => `Working: step ${meta?.step}`,
216
- });
217
- // status = "Working: step 2"
218
- ```
219
-
220
- ### unwrap()
221
-
222
- Extract data, throwing on error:
223
-
224
- ```typescript
225
- import { unwrap } from '@vibe-agent-toolkit/agent-runtime';
226
-
227
- const result1 = { status: 'success' as const, data: 'hello' };
228
- const data = unwrap(result1);
229
- // data = "hello"
230
-
231
- const result2 = { status: 'error' as const, error: 'failed' };
232
- const data2 = unwrap(result2);
233
- // Throws: Error: Result was error: failed
234
- ```
235
-
236
- ## Orchestration Patterns
237
-
238
- ### Sequential Execution
239
-
240
- Execute agents in order, passing results forward:
241
-
242
- ```typescript
243
- import { andThen } from '@vibe-agent-toolkit/agent-runtime';
244
-
245
- async function analyzeAndNameCat(imagePath: string) {
246
- // Step 1: Analyze photo
247
- const analysisOutput = await photoAnalyzer.execute({ imagePath });
248
-
249
- // Step 2: Generate name (only if analysis succeeded)
250
- const nameOutput = await andThen(
251
- analysisOutput.result,
252
- async (characteristics) => {
253
- const output = await nameGenerator.execute({ characteristics });
254
- return output.result;
255
- }
256
- );
257
-
258
- // Step 3: Validate name (only if generation succeeded)
259
- const validationOutput = await andThen(
260
- nameOutput,
261
- async (name) => {
262
- const output = await nameValidator.execute({
263
- name: name.name,
264
- characteristics: name.characteristics,
265
- });
266
- return output.result;
267
- }
268
- );
269
-
270
- return validationOutput;
271
- }
272
-
273
- // Usage
274
- const result = await analyzeAndNameCat('photos/orange-tabby.jpg');
275
-
276
- match(result, {
277
- success: (data) => console.log('Valid name:', data),
278
- error: (err) => console.error('Failed:', err),
279
- });
280
- ```
281
-
282
- ### Parallel Execution
283
-
284
- Execute multiple agents concurrently:
285
-
286
- ```typescript
287
- async function analyzeMultipleSources(
288
- imagePath: string,
289
- description: string
290
- ) {
291
- // Run both analyzers in parallel
292
- const [photoOutput, descOutput] = await Promise.all([
293
- photoAnalyzer.execute({ imagePath }),
294
- descriptionParser.execute({ text: description }),
295
- ]);
296
-
297
- // Combine results
298
- if (photoOutput.result.status === 'success' &&
299
- descOutput.result.status === 'success') {
300
- // Both succeeded - merge characteristics
301
- return {
302
- status: 'success' as const,
303
- data: mergeCharacteristics(
304
- photoOutput.result.data,
305
- descOutput.result.data
306
- ),
307
- };
308
- } else if (photoOutput.result.status === 'success') {
309
- // Photo succeeded, use that
310
- return photoOutput.result;
311
- } else if (descOutput.result.status === 'success') {
312
- // Description succeeded, use that
313
- return descOutput.result;
314
- } else {
315
- // Both failed
316
- return {
317
- status: 'error' as const,
318
- error: 'both-analyzers-failed' as const,
319
- };
320
- }
321
- }
322
- ```
323
-
324
- ### Intelligent Retry with Tracking
325
-
326
- Use the built-in `withRetry` helper for automatic retry with exponential backoff:
327
-
328
- ```typescript
329
- import { withRetry } from '@vibe-agent-toolkit/agent-runtime';
330
- import {
331
- LLM_TIMEOUT,
332
- LLM_RATE_LIMIT,
333
- LLM_UNAVAILABLE,
334
- RETRYABLE_LLM_ERRORS,
335
- RETRYABLE_EVENT_ERRORS,
336
- } from '@vibe-agent-toolkit/agent-schema';
337
-
338
- // Use the withRetry helper (built into agent-runtime)
339
- const output = await withRetry(
340
- () => photoAnalyzer.execute({ imagePath }),
341
- 5 // max attempts
342
- );
343
-
344
- if (output.result.status === 'success') {
345
- console.log('Success after', output.result.execution?.retryCount ?? 0, 'retries');
346
- console.log('Total duration:', output.result.execution?.durationMs, 'ms');
347
- console.log('Total cost:', output.result.execution?.cost);
348
- } else {
349
- console.error('Failed after', output.result.execution?.retryCount ?? 0, 'retries');
350
- console.error('Error type:', output.result.error);
351
- }
352
- ```
353
-
354
- **How it works:**
355
-
356
- 1. **Error classification**: Agents return error constants (e.g., `LLM_TIMEOUT`, `LLM_RATE_LIMIT`)
357
- 2. **Retryability**: Error types imply retryability via exported sets:
358
- - `RETRYABLE_LLM_ERRORS`: `[LLM_TIMEOUT, LLM_RATE_LIMIT, LLM_UNAVAILABLE]`
359
- - `RETRYABLE_EVENT_ERRORS`: `[EVENT_TIMEOUT, EVENT_UNAVAILABLE]`
360
- 3. **Backoff delays**: Different error types have different base delays:
361
- - `LLM_RATE_LIMIT`: 5000ms (rate limits need longer waits)
362
- - `LLM_TIMEOUT`: 1000ms (timeouts can retry quickly)
363
- - `LLM_UNAVAILABLE`: 10000ms (service issues need long waits)
364
- 4. **Exponential backoff**: Each retry doubles the delay (capped at 30 seconds)
365
- 5. **Accumulated metrics**: The helper accumulates `durationMs`, `tokensUsed`, and `cost` across all attempts
366
- 6. **Retry count injection**: The orchestrator injects `retryCount` into `ExecutionMetadata`
367
-
368
- **Custom retry logic:**
369
-
370
- If you need custom retry behavior, you can use the standard error sets:
371
-
372
- ```typescript
373
- import { RETRYABLE_LLM_ERRORS } from '@vibe-agent-toolkit/agent-schema';
374
-
375
- function isRetryable(error: string): boolean {
376
- return RETRYABLE_LLM_ERRORS.has(error as LLMError) ||
377
- RETRYABLE_EVENT_ERRORS.has(error as ExternalEventError);
378
- }
379
- ```
380
-
381
- ### Conversational Multi-Turn
382
-
383
- Handle multi-turn conversations with state:
384
-
385
- ```typescript
386
- async function runConversation() {
387
- let session = {
388
- history: [],
389
- state: { profile: { conversationPhase: 'gathering' as const } },
390
- };
391
-
392
- while (true) {
393
- const userMessage = await getUserInput();
394
-
395
- const output = await breedAdvisor.execute({
396
- message: userMessage,
397
- sessionState: session.state,
398
- });
399
-
400
- // Show agent's reply
401
- console.log('Agent:', output.reply);
402
-
403
- // Update session for next turn
404
- session = {
405
- history: [
406
- ...session.history,
407
- { role: 'user', content: userMessage },
408
- { role: 'assistant', content: output.reply },
409
- ],
410
- state: output.sessionState,
411
- };
412
-
413
- // Check if conversation is complete
414
- if (output.result.status === 'success') {
415
- console.log('Final recommendation:', output.result.data);
416
- break;
417
- } else if (output.result.status === 'error') {
418
- console.error('Conversation failed:', output.result.error);
419
- break;
420
- }
421
-
422
- // status === 'in-progress', continue conversation
423
- }
424
- }
425
- ```
426
-
427
- ### Fan-Out/Fan-In
428
-
429
- Execute multiple agents and combine results:
430
-
431
- ```typescript
432
- async function generateMultipleNames(
433
- characteristics: CatCharacteristics,
434
- count: number = 3
435
- ) {
436
- // Generate multiple names in parallel
437
- const outputs = await Promise.all(
438
- Array.from({ length: count }, () =>
439
- nameGenerator.execute({ characteristics })
440
- )
441
- );
442
-
443
- // Collect successful results
444
- const successfulNames = outputs
445
- .map(output => output.result)
446
- .filter(result => result.status === 'success')
447
- .map(result => result.data);
448
-
449
- if (successfulNames.length === 0) {
450
- return {
451
- status: 'error' as const,
452
- error: 'all-generations-failed' as const,
453
- };
454
- }
455
-
456
- return {
457
- status: 'success' as const,
458
- data: {
459
- suggestions: successfulNames,
460
- count: successfulNames.length,
461
- },
462
- };
463
- }
464
- ```
465
-
466
- ### Validation Loop
467
-
468
- Generator + Validator feedback loop:
469
-
470
- ```typescript
471
- async function generateValidName(
472
- characteristics: CatCharacteristics,
473
- maxAttempts: number = 5
474
- ): Promise<AgentResult<string, 'max-attempts-exceeded'>> {
475
- for (let attempt = 0; attempt < maxAttempts; attempt++) {
476
- // Generate name
477
- const nameOutput = await nameGenerator.execute({ characteristics });
478
- if (nameOutput.result.status === 'error') {
479
- continue; // Try again
480
- }
481
-
482
- const { name } = nameOutput.result.data;
483
-
484
- // Validate name
485
- const validationOutput = await nameValidator.execute({
486
- name,
487
- characteristics,
488
- });
489
-
490
- if (validationOutput.result.status === 'success' &&
491
- validationOutput.result.data.valid) {
492
- // Found valid name!
493
- return { status: 'success', data: name };
494
- }
495
-
496
- // Invalid, try again with feedback
497
- console.log('Attempt', attempt + 1, 'invalid:', validationOutput.result.data.reason);
498
- }
499
-
500
- return { status: 'error', error: 'max-attempts-exceeded' };
501
- }
502
- ```
503
-
504
- ### Human-in-the-Loop
505
-
506
- Integrate human approval into workflows:
507
-
508
- ```typescript
509
- import { humanApprovalAgent } from '@vibe-agent-toolkit/vat-example-cat-agents';
510
-
511
- async function generateWithApproval(
512
- characteristics: CatCharacteristics
513
- ) {
514
- // Generate name
515
- const nameOutput = await nameGenerator.execute({ characteristics });
516
- if (nameOutput.result.status === 'error') {
517
- return nameOutput.result;
518
- }
519
-
520
- const { name, reasoning } = nameOutput.result.data;
521
-
522
- // Request human approval
523
- const approvalOutput = await humanApprovalAgent.execute({
524
- actionType: 'cat-name-generation',
525
- content: { name, reasoning },
526
- context: {
527
- characteristics: {
528
- physical: characteristics.physical,
529
- behavioral: characteristics.behavioral,
530
- },
531
- },
532
- });
533
-
534
- if (approvalOutput.result.status === 'error') {
535
- return { status: 'error' as const, error: 'approval-failed' as const };
536
- }
537
-
538
- const { approved, feedback } = approvalOutput.result.data;
539
-
540
- if (approved) {
541
- return { status: 'success' as const, data: name };
542
- } else {
543
- return {
544
- status: 'error' as const,
545
- error: 'rejected-by-human' as const,
546
- };
547
- }
548
- }
549
- ```
550
-
551
- ## Observability and Production Monitoring
552
-
553
- Result envelopes support optional observability fields for production monitoring, intelligent orchestration, and debugging.
554
-
555
- ### Confidence
556
-
557
- Indicates certainty in the result (0-1 scale), enabling intelligent orchestration decisions:
558
-
559
- ```typescript
560
- import { RESULT_SUCCESS } from '@vibe-agent-toolkit/agent-schema';
561
-
562
- // Agent returns confidence
563
- const output = await photoAnalyzer.execute({ imagePath });
564
-
565
- if (output.result.status === RESULT_SUCCESS) {
566
- const confidence = output.result.confidence ?? 1.0;
567
-
568
- if (confidence < 0.8) {
569
- // Low confidence - verify with another agent
570
- const secondOpinion = await backupAnalyzer.execute({ imagePath });
571
- // Compare results...
572
- }
573
- }
574
- ```
575
-
576
- **Use cases:**
577
- - **Orchestration decisions**: Retry if confidence < threshold
578
- - **Chain validation**: Verify uncertain results with another agent
579
- - **User transparency**: Show uncertainty to users ("I'm 70% confident")
580
- - **Stopping criteria**: Iterate until confidence > 0.9
581
-
582
- ### Warnings
583
-
584
- Non-fatal issues array for graceful degradation:
585
-
586
- ```typescript
587
- const output = await photoAnalyzer.execute({ imagePath });
588
-
589
- if (output.result.status === RESULT_SUCCESS) {
590
- // Check for warnings
591
- if (output.result.warnings && output.result.warnings.length > 0) {
592
- console.warn('Warnings:', output.result.warnings);
593
- // Example: ['Image quality was poor, confidence may be lower']
594
- }
595
-
596
- // Use data despite warnings
597
- console.log('Characteristics:', output.result.data);
598
- }
599
- ```
600
-
601
- **Use cases:**
602
- - **User transparency**: Show quality issues or caveats
603
- - **Context for downstream agents**: Pass warnings along pipeline
604
- - **Logging/debugging**: Track degraded but successful operations
605
-
606
- ### Execution Metadata
607
-
608
- Performance and cost tracking for production systems:
609
-
610
- ```typescript
611
- import { withTiming } from '@vibe-agent-toolkit/agent-runtime';
612
-
613
- // Wrap with timing helper
614
- const output = await withTiming(() =>
615
- photoAnalyzer.execute({ imagePath })
616
- );
617
-
618
- if (output.result.execution) {
619
- console.log('Duration:', output.result.execution.durationMs, 'ms');
620
- console.log('Tokens:', output.result.execution.tokensUsed);
621
- console.log('Cost:', output.result.execution.cost, 'USD');
622
- console.log('Model:', output.result.execution.model);
623
- console.log('Provider:', output.result.execution.provider);
624
- console.log('Retry count:', output.result.execution.retryCount ?? 0);
625
- console.log('Timestamp:', output.result.execution.timestamp);
626
- }
627
- ```
628
-
629
- **Fields:**
630
- - `durationMs`: Total execution duration (including retries)
631
- - `tokensUsed`: LLM tokens consumed (sum across all LLM calls)
632
- - `cost`: Estimated cost in USD
633
- - `model`: Model identifier for A/B testing
634
- - `provider`: Provider for cost attribution
635
- - `retryCount`: Set by orchestrator's retry wrapper (0 = no retries)
636
- - `timestamp`: Execution start time (ISO 8601)
637
-
638
- **Use cases:**
639
- - **Performance optimization**: Identify slow agents
640
- - **Cost attribution**: Multi-tenant systems
641
- - **Usage analytics**: Tokens per request
642
- - **Debugging**: View retry behavior
643
-
644
- ### Combining Observability Features
645
-
646
- ```typescript
647
- import { withRetry, withTiming } from '@vibe-agent-toolkit/agent-runtime';
648
- import { RESULT_SUCCESS } from '@vibe-agent-toolkit/agent-schema';
649
-
650
- // Combine timing and retry
651
- const output = await withRetry(
652
- () => withTiming(() => photoAnalyzer.execute({ imagePath })),
653
- 5
654
- );
655
-
656
- if (output.result.status === RESULT_SUCCESS) {
657
- const { confidence, warnings, execution } = output.result;
658
-
659
- // Log comprehensive observability data
660
- console.log({
661
- confidence: confidence ?? 1.0,
662
- warnings: warnings ?? [],
663
- duration: execution?.durationMs,
664
- retries: execution?.retryCount ?? 0,
665
- cost: execution?.cost,
666
- tokens: execution?.tokensUsed,
667
- });
668
-
669
- // Make orchestration decisions based on confidence
670
- if (confidence && confidence < 0.7) {
671
- console.warn('Low confidence result, consider verification');
672
- }
673
-
674
- // Alert on excessive retries
675
- if (execution?.retryCount && execution.retryCount > 3) {
676
- console.error('Agent required excessive retries, investigate');
677
- }
678
- }
679
- ```
680
-
681
- ## Testing Patterns
682
-
683
- ### Test Helpers
684
-
685
- Use test helpers from `@vibe-agent-toolkit/agent-runtime`:
686
-
687
- ```typescript
688
- import { resultMatchers } from '@vibe-agent-toolkit/agent-runtime';
689
- import { describe, expect, it } from 'vitest';
690
-
691
- describe('haikuValidator', () => {
692
- it('should return success for valid haiku', async () => {
693
- const output = await haikuValidator.execute({
694
- text: 'Ancient pond, frog leaps in, splash',
695
- syllables: [5, 7, 5],
696
- kigo: 'pond',
697
- kireji: 'splash',
698
- });
699
-
700
- // Type-safe assertions
701
- resultMatchers.expectSuccess(output.result);
702
- expect(output.result.data.valid).toBe(true);
703
- });
704
-
705
- it('should return error for invalid syllables', async () => {
706
- const output = await haikuValidator.execute({
707
- text: 'Too many syllables here, not a haiku, nope',
708
- syllables: [7, 7, 5],
709
- kigo: 'syllables',
710
- kireji: 'nope',
711
- });
712
-
713
- resultMatchers.expectError(output.result);
714
- expect(output.result.error).toBe('invalid-syllables');
715
- });
716
- });
717
- ```
718
-
719
- ### Mocking External Events
720
-
721
- Mock external event integrators in tests:
722
-
723
- ```typescript
724
- import { vi } from 'vitest';
725
-
726
- describe('workflow with approval', () => {
727
- it('should handle approved case', async () => {
728
- // Mock approval agent to auto-approve
729
- const mockApproval = vi.fn().mockResolvedValue({
730
- result: {
731
- status: 'success',
732
- data: { approved: true, feedback: null },
733
- },
734
- });
735
-
736
- const result = await generateWithApproval(characteristics, {
737
- approvalAgent: { execute: mockApproval },
738
- });
739
-
740
- resultMatchers.expectSuccess(result);
741
- expect(mockApproval).toHaveBeenCalledOnce();
742
- });
743
- });
744
- ```
745
-
746
- ## Best Practices
747
-
748
- ### 1. Always Handle Errors
749
-
750
- Never assume success - always check result status:
751
-
752
- ```typescript
753
- // ❌ BAD - Assumes success
754
- const output = await analyzer.execute(input);
755
- const data = output.result.data; // TypeScript error if result is error!
756
-
757
- // ✅ GOOD - Check status first
758
- const output = await analyzer.execute(input);
759
- if (output.result.status === 'success') {
760
- const data = output.result.data; // Type-safe
761
- console.log(data);
762
- } else {
763
- console.error('Failed:', output.result.error);
764
- }
765
- ```
766
-
767
- ### 2. Use Helpers for Composition
768
-
769
- Use `andThen()` and `mapResult()` instead of manual checks:
770
-
771
- ```typescript
772
- // ❌ VERBOSE - Manual checks
773
- const output1 = await agent1.execute(input);
774
- let output2;
775
- if (output1.result.status === 'success') {
776
- output2 = await agent2.execute(output1.result.data);
777
- } else {
778
- output2 = { result: output1.result };
779
- }
780
-
781
- // ✅ CONCISE - Use andThen
782
- const output1 = await agent1.execute(input);
783
- const output2 = await andThen(output1.result, (data) =>
784
- agent2.execute(data).then(out => out.result)
785
- );
786
- ```
787
-
788
- ### 3. Leverage Discriminated Unions
789
-
790
- TypeScript's discriminated unions provide type safety:
791
-
792
- ```typescript
793
- function handleResult(result: AgentResult<string, LLMError>) {
794
- // TypeScript knows which properties are available
795
- if (result.status === 'success') {
796
- console.log(result.data); // ✅ data available
797
- console.log(result.error); // ❌ TypeScript error
798
- } else {
799
- console.log(result.error); // ✅ error available
800
- console.log(result.data); // ❌ TypeScript error
801
- }
802
- }
803
- ```
804
-
805
- ### 4. Define Clear Error Types
806
-
807
- Use enums or literal unions for errors:
808
-
809
- ```typescript
810
- // ✅ GOOD - Clear error types
811
- type ValidationError =
812
- | 'invalid-syllables'
813
- | 'missing-kigo'
814
- | 'too-long';
815
-
816
- // ❌ BAD - Generic string
817
- type ValidationError = string;
818
- ```
819
-
820
- ### 5. Document Metadata Schemas
821
-
822
- For conversational agents, document metadata structure:
823
-
824
- ```typescript
825
- /**
826
- * Metadata for breed advisor in-progress state
827
- */
828
- interface BreedAdvisorMetadata {
829
- /** Number of factors collected so far */
830
- factorsCollected: number;
831
- /** Minimum factors required for recommendations */
832
- requiredFactors: number;
833
- /** Current conversation phase */
834
- conversationPhase: 'gathering' | 'ready-to-recommend' | 'refining';
835
- /** Current breed recommendations (if in refining phase) */
836
- recommendations?: Array<{ breed: string; score: number }>;
837
- }
838
- ```
839
-
840
- ## Related Documentation
841
-
842
- - [Agent Authoring Guide](agent-authoring.md) - How to create agents with result envelopes
843
- - [Architecture Overview]() - Package structure and design principles
844
-
845
- ## Examples
846
-
847
- See the `@vibe-agent-toolkit/vat-example-cat-agents` package for complete working examples:
848
-
849
- - **Pure Function Tool**: `haiku-validator` - Validation with typed errors
850
- - **One-Shot LLM Analyzer**: `photo-analyzer`, `description-parser` - LLM with error handling
851
- - **Conversational Assistant**: `breed-advisor` - Multi-turn with stateful results
852
- - **External Event Integrator**: `human-approval` - Event integration with timeouts
853
-
854
- Run demos:
855
- ```bash
856
- cd packages/vat-example-cat-agents
857
- bun run demo:photos # Photo analysis pipeline
858
- bun run demo:conversation # Interactive breed advisor
859
- ```