@output.ai/cli 0.3.1-dev.pr156.0 → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (48) hide show
  1. package/README.md +2 -28
  2. package/dist/api/generated/api.d.ts +35 -59
  3. package/dist/api/generated/api.js +4 -13
  4. package/dist/assets/docker/docker-compose-dev.yml +2 -2
  5. package/dist/commands/workflow/debug.d.ts +2 -8
  6. package/dist/commands/workflow/debug.js +24 -164
  7. package/dist/commands/workflow/debug.spec.js +36 -0
  8. package/dist/commands/workflow/generate.js +3 -10
  9. package/dist/commands/workflow/generate.spec.js +6 -4
  10. package/dist/commands/workflow/{output.d.ts → result.d.ts} +1 -1
  11. package/dist/commands/workflow/{output.js → result.js} +8 -8
  12. package/dist/commands/workflow/result.test.js +23 -0
  13. package/dist/commands/workflow/start.js +1 -1
  14. package/dist/services/coding_agents.js +30 -0
  15. package/dist/services/coding_agents.spec.js +36 -61
  16. package/dist/services/messages.d.ts +1 -0
  17. package/dist/services/messages.js +65 -1
  18. package/dist/services/trace_reader.d.ts +14 -0
  19. package/dist/services/trace_reader.js +67 -0
  20. package/dist/services/trace_reader.spec.d.ts +1 -0
  21. package/dist/services/trace_reader.spec.js +164 -0
  22. package/dist/services/workflow_generator.spec.d.ts +1 -0
  23. package/dist/services/workflow_generator.spec.js +77 -0
  24. package/dist/templates/agent_instructions/AGENTS.md.template +209 -19
  25. package/dist/templates/agent_instructions/agents/context_fetcher.md.template +82 -0
  26. package/dist/templates/agent_instructions/agents/prompt_writer.md.template +595 -0
  27. package/dist/templates/agent_instructions/agents/workflow_planner.md.template +13 -4
  28. package/dist/templates/agent_instructions/agents/workflow_quality.md.template +244 -0
  29. package/dist/templates/agent_instructions/commands/build_workflow.md.template +52 -9
  30. package/dist/templates/agent_instructions/commands/plan_workflow.md.template +4 -4
  31. package/dist/templates/project/package.json.template +2 -2
  32. package/dist/templates/project/src/simple/scenarios/question_ada_lovelace.json.template +3 -0
  33. package/dist/templates/workflow/scenarios/test_input.json.template +7 -0
  34. package/dist/types/trace.d.ts +161 -0
  35. package/dist/types/trace.js +18 -0
  36. package/dist/utils/date_formatter.d.ts +8 -0
  37. package/dist/utils/date_formatter.js +19 -0
  38. package/dist/utils/template.spec.js +6 -0
  39. package/dist/utils/trace_formatter.d.ts +11 -61
  40. package/dist/utils/trace_formatter.js +384 -239
  41. package/package.json +2 -2
  42. package/dist/commands/workflow/debug.test.js +0 -107
  43. package/dist/commands/workflow/output.test.js +0 -23
  44. package/dist/utils/s3_downloader.d.ts +0 -49
  45. package/dist/utils/s3_downloader.js +0 -154
  46. /package/dist/commands/workflow/{debug.test.d.ts → debug.spec.d.ts} +0 -0
  47. /package/dist/commands/workflow/{output.test.d.ts → result.test.d.ts} +0 -0
  48. /package/dist/templates/workflow/{prompt@v1.prompt.template → prompts/prompt@v1.prompt.template} +0 -0
@@ -1,30 +1,220 @@
1
+ # Output.ai Based Project Guide
2
+
3
+ ## Overview
4
+
5
+ This project uses Output Framework to build durable, LLM-powered workflows orchestrated by Temporal. Output Framework provides abstractions for creating reliable AI workflows with automatic retry, tracing, and error handling. Developers use it to build workflows like fact checkers, content generators, data extractors, research assistants, and multi-step AI agents.
6
+
7
+ ### Project Overview
8
+
9
+ Each workflow lives in its own folder under `src/workflows/` and follows a consistent structure. Workflows define the orchestration logic, calling steps to perform external operations like API calls, database queries, and LLM inference. The system automatically handles retries, timeouts, and distributed execution through Temporal.
10
+
11
+ ### Key Concepts
12
+
13
+ #### Built on Top of Temporal
14
+
15
+ Temporal provides durable execution guarantees - if a workflow fails mid-execution, it resumes from the last successful step rather than restarting. Output Framework wraps Temporal's workflow and activity primitives with higher-level abstractions (`workflow`, `step`, `evaluator`) that enforce best practices and provide automatic tracing.
16
+
17
+ #### Single Folder Project Structure
18
+
19
+ Each workflow is self-contained in a single folder with a predictable structure: `workflow.ts` contains the deterministic orchestration logic, `steps.ts` contains I/O operations (API calls, LLM inference), `evaluators.ts` contains analysis logic returning confidence-scored results, and `prompts/*.prompt` files define LLM prompts using Liquid.js templates with YAML frontmatter for model configuration.
20
+
21
+ ## Critical Conventions
22
+
23
+ - **HTTP**: Never use axios → use `@output.ai/http` (traced, auto-retry)
24
+ - **LLM**: Never call LLM APIs directly → use `@output.ai/llm`
25
+ - **Workflows**: Must be deterministic - only call steps/evaluators, no direct I/O
26
+ - **Steps**: All external operations (APIs, DBs, LLMs) must be wrapped in steps
27
+ - **Schemas**: Use Zod (`z`) from `@output.ai/core` to define input/output schemas
28
+
29
+ ## Project Structure
30
+
31
+ ```
32
+ src/workflows/{name}/
33
+ workflow.ts # Orchestration logic (deterministic)
34
+ steps.ts # I/O operations (APIs, LLM, DB)
35
+ evaluators.ts # Analysis steps returning EvaluationResult
36
+ prompts/*.prompt # LLM prompts (name@v1.prompt)
37
+ scenarios/*.json # Test scenarios
38
+ ```
39
+
40
+ ## Commands
41
+
42
+ ```bash
43
+ output dev # Start dev (Temporal:8080, API:3001)
44
+ output workflow list # List workflows
45
+
46
+ # Sync execution (waits for result)
47
+ output workflow run <name> --input <JSON|JSON_FILE> # Execute and wait
48
+
49
+ # Async execution
50
+ output workflow start <name> --input <JSON|JSON_FILE> # Start workflow, returns ID
51
+ output workflow status <workflowId> # Check execution status
52
+ output workflow result <workflowId> # Get result when complete
53
+ output workflow stop <workflowId> # Cancel running workflow
54
+ ```
55
+
56
+ ## Workflow Pattern
57
+
58
+ Workflows orchestrate steps. They must be deterministic (no direct I/O).
59
+
60
+ ```typescript
61
+ import { workflow, z } from '@output.ai/core';
62
+ import { processData, callApi } from './steps.js';
63
+
64
+ export default workflow({
65
+ name: 'my-workflow',
66
+ description: 'What this workflow does',
67
+ inputSchema: z.object({ query: z.string() }),
68
+ outputSchema: z.object({ result: z.string() }),
69
+ fn: async (input) => {
70
+ const data = await processData(input);
71
+ const result = await callApi(data);
72
+ return { result };
73
+ }
74
+ });
75
+ ```
76
+
77
+ **Allowed imports**: steps.ts, evaluators.ts, shared_steps.ts, types.ts, consts.ts, utils.ts
78
+
79
+ **Forbidden in workflows**: Direct API calls, Math.random(), Date.now(), dynamic imports
80
+
81
+ ## Step Pattern
82
+
83
+ Steps contain all I/O operations. They are automatically retried on failure.
84
+
85
+ ```typescript
86
+ import { step, z } from '@output.ai/core';
87
+ import { httpClient } from '@output.ai/http';
88
+
89
+ const client = httpClient({ prefixUrl: 'https://api.example.com' });
90
+
91
+ export const fetchData = step({
92
+ name: 'fetchData',
93
+ description: 'Fetch data from external API',
94
+ inputSchema: z.object({ id: z.string() }),
95
+ outputSchema: z.object({ data: z.any() }),
96
+ fn: async ({ id }) => {
97
+ const response = await client.get(`items/${id}`).json();
98
+ return { data: response };
99
+ },
100
+ options: {
101
+ retry: { maximumAttempts: 3 }
102
+ }
103
+ });
104
+ ```
105
+
106
+ ## LLM Pattern
107
+
108
+ Use `@output.ai/llm` for all LLM operations. Prompts are defined in `.prompt` files.
109
+
110
+ **Prompt file** (`summarize@v1.prompt`):
111
+ ```yaml
1
112
  ---
2
- generated_by: "@output.ai/cli"
3
- version: "{{cliVersion}}"
4
- generated_on: "{{date}}"
113
+ provider: anthropic
114
+ model: claude-sonnet-4-20250514
115
+ temperature: 0.7
116
+ maxTokens: 2000
5
117
  ---
118
+ <system>You are a helpful assistant.</system>
119
+ <user>Summarize: \{{ content }}</user>
120
+ ```
121
+
122
+ **Step using prompt**:
123
+ ```typescript
124
+ import { step, z } from '@output.ai/core';
125
+ import { generateText, generateObject } from '@output.ai/llm';
126
+
127
+ export const summarize = step({
128
+ name: 'summarize',
129
+ inputSchema: z.object({ content: z.string() }),
130
+ outputSchema: z.string(),
131
+ fn: async ({ content }) => {
132
+ return generateText({
133
+ prompt: 'summarize@v1',
134
+ variables: { content }
135
+ });
136
+ }
137
+ });
138
+
139
+ // For structured output
140
+ export const extractInfo = step({
141
+ name: 'extractInfo',
142
+ inputSchema: z.object({ text: z.string() }),
143
+ outputSchema: z.object({ title: z.string(), summary: z.string() }),
144
+ fn: async ({ text }) => {
145
+ return generateObject({
146
+ prompt: 'extract@v1',
147
+ variables: { text },
148
+ schema: z.object({ title: z.string(), summary: z.string() })
149
+ });
150
+ }
151
+ });
152
+ ```
153
+
154
+ **Available functions**: `generateText`, `generateObject`, `generateArray`, `generateEnum`
155
+
156
+ **Providers**: anthropic, openai, azure
157
+
158
+ ## HTTP Pattern
159
+
160
+ Use `@output.ai/http` for traced HTTP requests with automatic retry.
161
+
162
+ ```typescript
163
+ import { httpClient } from '@output.ai/http';
164
+
165
+ const client = httpClient({
166
+ prefixUrl: 'https://api.example.com',
167
+ timeout: 30000,
168
+ retry: { limit: 3 }
169
+ });
170
+
171
+ // In a step:
172
+ const data = await client.get('endpoint').json();
173
+ const result = await client.post('endpoint', { json: payload }).json();
174
+ ```
175
+
176
+ ## Evaluator Pattern
177
+
178
+ Evaluators analyze data and return confidence-scored results.
179
+
180
+ ```typescript
181
+ import { evaluator, EvaluationStringResult } from '@output.ai/core';
6
182
 
7
- # Output SDK Workflow Planning Agent Configuration
183
+ export const judgeQuality = evaluator({
184
+ name: 'judgeQuality',
185
+ inputSchema: z.string(),
186
+ fn: async (content) => {
187
+ // Analysis logic
188
+ return new EvaluationStringResult({
189
+ value: 'good',
190
+ confidence: 0.95
191
+ });
192
+ }
193
+ });
194
+ ```
8
195
 
9
- This directory contains AI agent configuration for workflow planning assistance in Output SDK development. Claude Code will automatically detect and use this configuration when planning workflows in your project.
196
+ ## Error Handling
10
197
 
11
- ## Available Agent
198
+ ```typescript
199
+ import { FatalError, ValidationError } from '@output.ai/core';
12
200
 
13
- ### 🤖 workflow_planner
14
- - **Purpose**: Assists with workflow architecture and planning
15
- - **Capabilities**: System design, workflow structure, step organization, requirements analysis
16
- - **Usage**: Planning new workflows, architectural decisions, creating implementation blueprints
201
+ // Non-retryable error (workflow fails immediately)
202
+ throw new FatalError('Critical failure - do not retry');
17
203
 
18
- ## Integration
204
+ // Validation error (input/output schema failure)
205
+ throw new ValidationError('Invalid input format');
206
+ ```
19
207
 
20
- This agent configuration integrates with:
21
- - **Claude Code**: AI-powered development assistant
22
- - **Output SDK**: Temporal-based workflow orchestration framework
23
- - **TypeScript**: Type-safe development patterns
24
- - **XML Process Flow**: Structured planning with validation checkpoints
208
+ ## Agent System
25
209
 
26
- ## Usage
210
+ For workflow planning and implementation:
211
+ - `.claude/agents/workflow_planner.md` - Workflow architecture specialist
212
+ - `.claude/agents/workflow_quality.md` - Workflow quality and best practices specialist
213
+ - `.claude/agents/prompt_writer.md` - Prompt file creation and review specialist
214
+ - `.claude/agents/context_fetcher.md` - Efficient context retrieval (used by other agents)
215
+ - `.claude/commands/plan_workflow.md` - Planning command
216
+ - `.claude/commands/build_workflow.md` - Implementation command
27
217
 
28
- The workflow planning agent is automatically available when using Claude Code in projects with this configuration. It provides specialized expertise for the planning and architecture phase of Output SDK workflow development.
218
+ ## Configuration
29
219
 
30
- To regenerate this configuration: `output-cli agents init --force`
220
+ See `.env` file for required environment variables (API keys, etc.)
@@ -0,0 +1,82 @@
1
+ ---
2
+ name: context-fetcher
3
+ description: Use proactively to retrieve and extract relevant information from Output SDK project documentation files. Checks if content is already in context before returning.
4
+ tools: Read, Grep, Glob
5
+ model: haiku
6
+ color: blue
7
+ ---
8
+
9
+ # Output SDK Context Fetcher Agent
10
+
11
+ You are a specialized information retrieval agent for Output SDK workflows. Your role is to efficiently fetch and extract relevant content from documentation and configuration files while avoiding duplication.
12
+
13
+ ## Core Responsibilities
14
+
15
+ 1. **Context Check First**: Determine if requested information is already in the main agent's context
16
+ 2. **Selective Reading**: Extract only the specific sections or information requested
17
+ 3. **Smart Retrieval**: Use grep to find relevant sections rather than reading entire files
18
+ 4. **Return Efficiently**: Provide only new information not already in context
19
+
20
+ ## Supported File Types
21
+
22
+ ### Configuration & Meta
23
+ - `.outputai/meta/pre_flight.md` - Pre-execution validation rules
24
+ - `.outputai/meta/post_flight.md` - Post-execution validation checks
25
+ - `.outputai/AGENTS.md` - Main project context (CLAUDE.md)
26
+
27
+ ### Agent Instructions
28
+ - `.outputai/agents/*.md` - Specialist agent definitions
29
+ - `.outputai/commands/*.md` - Command definitions
30
+
31
+ ### Workflow Files
32
+ - `src/workflows/*/workflow.ts` - Workflow definitions
33
+ - `src/workflows/*/steps.ts` - Step implementations
34
+ - `src/workflows/*/evaluators.ts` - Evaluator definitions
35
+ - `src/workflows/*/prompts/*.prompt` - LLM prompt templates
36
+ - `src/workflows/*/scenarios/*.json` - Test scenarios
37
+
38
+ ## Workflow
39
+
40
+ 1. Check if the requested information appears to be in context already
41
+ 2. If not in context, locate the requested file(s)
42
+ 3. Extract only the relevant sections
43
+ 4. Return the specific information needed
44
+
45
+ ## Output Format
46
+
47
+ For new information:
48
+ ```
49
+ 📄 Retrieved from [file-path]
50
+
51
+ [Extracted content]
52
+ ```
53
+
54
+ For already-in-context information:
55
+ ```
56
+ ✓ Already in context: [brief description of what was requested]
57
+ ```
58
+
59
+ ## Smart Extraction Examples
60
+
61
+ Request: "Get the pre-flight rules"
62
+ → Extract only `.outputai/meta/pre_flight.md`, not other meta files
63
+
64
+ Request: "Find existing workflow patterns"
65
+ → Scan `src/workflows/*/workflow.ts` for patterns
66
+
67
+ Request: "Get retry policy defaults"
68
+ → Use grep to find retry-related sections in pre_flight.md or AGENTS.md
69
+
70
+ Request: "Find how existing steps use httpClient"
71
+ → Search `src/workflows/*/steps.ts` for httpClient patterns
72
+
73
+ ## Important Constraints
74
+
75
+ - Never return information already visible in current context
76
+ - Extract minimal necessary content
77
+ - Use grep for targeted searches
78
+ - Never modify any files
79
+ - Keep responses concise
80
+
81
+ ---
82
+ *This agent specializes in efficient context retrieval for Output SDK projects.*