@output.ai/cli 0.3.1-dev.pr156.0 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +2 -28
- package/dist/api/generated/api.d.ts +35 -59
- package/dist/api/generated/api.js +4 -13
- package/dist/assets/docker/docker-compose-dev.yml +2 -2
- package/dist/commands/workflow/debug.d.ts +2 -8
- package/dist/commands/workflow/debug.js +24 -164
- package/dist/commands/workflow/debug.spec.js +36 -0
- package/dist/commands/workflow/generate.js +3 -10
- package/dist/commands/workflow/generate.spec.js +6 -4
- package/dist/commands/workflow/{output.d.ts → result.d.ts} +1 -1
- package/dist/commands/workflow/{output.js → result.js} +8 -8
- package/dist/commands/workflow/result.test.js +23 -0
- package/dist/commands/workflow/start.js +1 -1
- package/dist/services/coding_agents.js +30 -0
- package/dist/services/coding_agents.spec.js +36 -61
- package/dist/services/messages.d.ts +1 -0
- package/dist/services/messages.js +65 -1
- package/dist/services/trace_reader.d.ts +14 -0
- package/dist/services/trace_reader.js +67 -0
- package/dist/services/trace_reader.spec.d.ts +1 -0
- package/dist/services/trace_reader.spec.js +164 -0
- package/dist/services/workflow_generator.spec.d.ts +1 -0
- package/dist/services/workflow_generator.spec.js +77 -0
- package/dist/templates/agent_instructions/AGENTS.md.template +209 -19
- package/dist/templates/agent_instructions/agents/context_fetcher.md.template +82 -0
- package/dist/templates/agent_instructions/agents/prompt_writer.md.template +595 -0
- package/dist/templates/agent_instructions/agents/workflow_planner.md.template +13 -4
- package/dist/templates/agent_instructions/agents/workflow_quality.md.template +244 -0
- package/dist/templates/agent_instructions/commands/build_workflow.md.template +52 -9
- package/dist/templates/agent_instructions/commands/plan_workflow.md.template +4 -4
- package/dist/templates/project/package.json.template +2 -2
- package/dist/templates/project/src/simple/scenarios/question_ada_lovelace.json.template +3 -0
- package/dist/templates/workflow/scenarios/test_input.json.template +7 -0
- package/dist/types/trace.d.ts +161 -0
- package/dist/types/trace.js +18 -0
- package/dist/utils/date_formatter.d.ts +8 -0
- package/dist/utils/date_formatter.js +19 -0
- package/dist/utils/template.spec.js +6 -0
- package/dist/utils/trace_formatter.d.ts +11 -61
- package/dist/utils/trace_formatter.js +384 -239
- package/package.json +2 -2
- package/dist/commands/workflow/debug.test.js +0 -107
- package/dist/commands/workflow/output.test.js +0 -23
- package/dist/utils/s3_downloader.d.ts +0 -49
- package/dist/utils/s3_downloader.js +0 -154
- /package/dist/commands/workflow/{debug.test.d.ts → debug.spec.d.ts} +0 -0
- /package/dist/commands/workflow/{output.test.d.ts → result.test.d.ts} +0 -0
- /package/dist/templates/workflow/{prompt@v1.prompt.template → prompts/prompt@v1.prompt.template} +0 -0
|
@@ -1,30 +1,220 @@
|
|
|
1
|
+
# Output.ai Based Project Guide
|
|
2
|
+
|
|
3
|
+
## Overview
|
|
4
|
+
|
|
5
|
+
This project uses Output Framework to build durable, LLM-powered workflows orchestrated by Temporal. Output Framework provides abstractions for creating reliable AI workflows with automatic retry, tracing, and error handling. Developers use it to build workflows like fact checkers, content generators, data extractors, research assistants, and multi-step AI agents.
|
|
6
|
+
|
|
7
|
+
### Project Overview
|
|
8
|
+
|
|
9
|
+
Each workflow lives in its own folder under `src/workflows/` and follows a consistent structure. Workflows define the orchestration logic, calling steps to perform external operations like API calls, database queries, and LLM inference. The system automatically handles retries, timeouts, and distributed execution through Temporal.
|
|
10
|
+
|
|
11
|
+
### Key Concepts
|
|
12
|
+
|
|
13
|
+
#### Built on Top of Temporal
|
|
14
|
+
|
|
15
|
+
Temporal provides durable execution guarantees - if a workflow fails mid-execution, it resumes from the last successful step rather than restarting. Output Framework wraps Temporal's workflow and activity primitives with higher-level abstractions (`workflow`, `step`, `evaluator`) that enforce best practices and provide automatic tracing.
|
|
16
|
+
|
|
17
|
+
#### Single Folder Project Structure
|
|
18
|
+
|
|
19
|
+
Each workflow is self-contained in a single folder with a predictable structure: `workflow.ts` contains the deterministic orchestration logic, `steps.ts` contains I/O operations (API calls, LLM inference), `evaluators.ts` contains analysis logic returning confidence-scored results, and `prompts/*.prompt` files define LLM prompts using Liquid.js templates with YAML frontmatter for model configuration.
|
|
20
|
+
|
|
21
|
+
## Critical Conventions
|
|
22
|
+
|
|
23
|
+
- **HTTP**: Never use axios → use `@output.ai/http` (traced, auto-retry)
|
|
24
|
+
- **LLM**: Never call LLM APIs directly → use `@output.ai/llm`
|
|
25
|
+
- **Workflows**: Must be deterministic - only call steps/evaluators, no direct I/O
|
|
26
|
+
- **Steps**: All external operations (APIs, DBs, LLMs) must be wrapped in steps
|
|
27
|
+
- **Schemas**: Use Zod (`z`) from `@output.ai/core` to define input/output schemas
|
|
28
|
+
|
|
29
|
+
## Project Structure
|
|
30
|
+
|
|
31
|
+
```
|
|
32
|
+
src/workflows/{name}/
|
|
33
|
+
workflow.ts # Orchestration logic (deterministic)
|
|
34
|
+
steps.ts # I/O operations (APIs, LLM, DB)
|
|
35
|
+
evaluators.ts # Analysis steps returning EvaluationResult
|
|
36
|
+
prompts/*.prompt # LLM prompts (name@v1.prompt)
|
|
37
|
+
scenarios/*.json # Test scenarios
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
## Commands
|
|
41
|
+
|
|
42
|
+
```bash
|
|
43
|
+
output dev # Start dev (Temporal:8080, API:3001)
|
|
44
|
+
output workflow list # List workflows
|
|
45
|
+
|
|
46
|
+
# Sync execution (waits for result)
|
|
47
|
+
output workflow run <name> --input <JSON|JSON_FILE> # Execute and wait
|
|
48
|
+
|
|
49
|
+
# Async execution
|
|
50
|
+
output workflow start <name> --input <JSON|JSON_FILE> # Start workflow, returns ID
|
|
51
|
+
output workflow status <workflowId> # Check execution status
|
|
52
|
+
output workflow result <workflowId> # Get result when complete
|
|
53
|
+
output workflow stop <workflowId> # Cancel running workflow
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
## Workflow Pattern
|
|
57
|
+
|
|
58
|
+
Workflows orchestrate steps. They must be deterministic (no direct I/O).
|
|
59
|
+
|
|
60
|
+
```typescript
|
|
61
|
+
import { workflow, z } from '@output.ai/core';
|
|
62
|
+
import { processData, callApi } from './steps.js';
|
|
63
|
+
|
|
64
|
+
export default workflow({
|
|
65
|
+
name: 'my-workflow',
|
|
66
|
+
description: 'What this workflow does',
|
|
67
|
+
inputSchema: z.object({ query: z.string() }),
|
|
68
|
+
outputSchema: z.object({ result: z.string() }),
|
|
69
|
+
fn: async (input) => {
|
|
70
|
+
const data = await processData(input);
|
|
71
|
+
const result = await callApi(data);
|
|
72
|
+
return { result };
|
|
73
|
+
}
|
|
74
|
+
});
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
**Allowed imports**: steps.ts, evaluators.ts, shared_steps.ts, types.ts, consts.ts, utils.ts
|
|
78
|
+
|
|
79
|
+
**Forbidden in workflows**: Direct API calls, Math.random(), Date.now(), dynamic imports
|
|
80
|
+
|
|
81
|
+
## Step Pattern
|
|
82
|
+
|
|
83
|
+
Steps contain all I/O operations. They are automatically retried on failure.
|
|
84
|
+
|
|
85
|
+
```typescript
|
|
86
|
+
import { step, z } from '@output.ai/core';
|
|
87
|
+
import { httpClient } from '@output.ai/http';
|
|
88
|
+
|
|
89
|
+
const client = httpClient({ prefixUrl: 'https://api.example.com' });
|
|
90
|
+
|
|
91
|
+
export const fetchData = step({
|
|
92
|
+
name: 'fetchData',
|
|
93
|
+
description: 'Fetch data from external API',
|
|
94
|
+
inputSchema: z.object({ id: z.string() }),
|
|
95
|
+
outputSchema: z.object({ data: z.any() }),
|
|
96
|
+
fn: async ({ id }) => {
|
|
97
|
+
const response = await client.get(`items/${id}`).json();
|
|
98
|
+
return { data: response };
|
|
99
|
+
},
|
|
100
|
+
options: {
|
|
101
|
+
retry: { maximumAttempts: 3 }
|
|
102
|
+
}
|
|
103
|
+
});
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
## LLM Pattern
|
|
107
|
+
|
|
108
|
+
Use `@output.ai/llm` for all LLM operations. Prompts are defined in `.prompt` files.
|
|
109
|
+
|
|
110
|
+
**Prompt file** (`summarize@v1.prompt`):
|
|
111
|
+
```yaml
|
|
1
112
|
---
|
|
2
|
-
|
|
3
|
-
|
|
4
|
-
|
|
113
|
+
provider: anthropic
|
|
114
|
+
model: claude-sonnet-4-20250514
|
|
115
|
+
temperature: 0.7
|
|
116
|
+
maxTokens: 2000
|
|
5
117
|
---
|
|
118
|
+
<system>You are a helpful assistant.</system>
|
|
119
|
+
<user>Summarize: \{{ content }}</user>
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
**Step using prompt**:
|
|
123
|
+
```typescript
|
|
124
|
+
import { step, z } from '@output.ai/core';
|
|
125
|
+
import { generateText, generateObject } from '@output.ai/llm';
|
|
126
|
+
|
|
127
|
+
export const summarize = step({
|
|
128
|
+
name: 'summarize',
|
|
129
|
+
inputSchema: z.object({ content: z.string() }),
|
|
130
|
+
outputSchema: z.string(),
|
|
131
|
+
fn: async ({ content }) => {
|
|
132
|
+
return generateText({
|
|
133
|
+
prompt: 'summarize@v1',
|
|
134
|
+
variables: { content }
|
|
135
|
+
});
|
|
136
|
+
}
|
|
137
|
+
});
|
|
138
|
+
|
|
139
|
+
// For structured output
|
|
140
|
+
export const extractInfo = step({
|
|
141
|
+
name: 'extractInfo',
|
|
142
|
+
inputSchema: z.object({ text: z.string() }),
|
|
143
|
+
outputSchema: z.object({ title: z.string(), summary: z.string() }),
|
|
144
|
+
fn: async ({ text }) => {
|
|
145
|
+
return generateObject({
|
|
146
|
+
prompt: 'extract@v1',
|
|
147
|
+
variables: { text },
|
|
148
|
+
schema: z.object({ title: z.string(), summary: z.string() })
|
|
149
|
+
});
|
|
150
|
+
}
|
|
151
|
+
});
|
|
152
|
+
```
|
|
153
|
+
|
|
154
|
+
**Available functions**: `generateText`, `generateObject`, `generateArray`, `generateEnum`
|
|
155
|
+
|
|
156
|
+
**Providers**: anthropic, openai, azure
|
|
157
|
+
|
|
158
|
+
## HTTP Pattern
|
|
159
|
+
|
|
160
|
+
Use `@output.ai/http` for traced HTTP requests with automatic retry.
|
|
161
|
+
|
|
162
|
+
```typescript
|
|
163
|
+
import { httpClient } from '@output.ai/http';
|
|
164
|
+
|
|
165
|
+
const client = httpClient({
|
|
166
|
+
prefixUrl: 'https://api.example.com',
|
|
167
|
+
timeout: 30000,
|
|
168
|
+
retry: { limit: 3 }
|
|
169
|
+
});
|
|
170
|
+
|
|
171
|
+
// In a step:
|
|
172
|
+
const data = await client.get('endpoint').json();
|
|
173
|
+
const result = await client.post('endpoint', { json: payload }).json();
|
|
174
|
+
```
|
|
175
|
+
|
|
176
|
+
## Evaluator Pattern
|
|
177
|
+
|
|
178
|
+
Evaluators analyze data and return confidence-scored results.
|
|
179
|
+
|
|
180
|
+
```typescript
|
|
181
|
+
import { evaluator, EvaluationStringResult } from '@output.ai/core';
|
|
6
182
|
|
|
7
|
-
|
|
183
|
+
export const judgeQuality = evaluator({
|
|
184
|
+
name: 'judgeQuality',
|
|
185
|
+
inputSchema: z.string(),
|
|
186
|
+
fn: async (content) => {
|
|
187
|
+
// Analysis logic
|
|
188
|
+
return new EvaluationStringResult({
|
|
189
|
+
value: 'good',
|
|
190
|
+
confidence: 0.95
|
|
191
|
+
});
|
|
192
|
+
}
|
|
193
|
+
});
|
|
194
|
+
```
|
|
8
195
|
|
|
9
|
-
|
|
196
|
+
## Error Handling
|
|
10
197
|
|
|
11
|
-
|
|
198
|
+
```typescript
|
|
199
|
+
import { FatalError, ValidationError } from '@output.ai/core';
|
|
12
200
|
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
- **Capabilities**: System design, workflow structure, step organization, requirements analysis
|
|
16
|
-
- **Usage**: Planning new workflows, architectural decisions, creating implementation blueprints
|
|
201
|
+
// Non-retryable error (workflow fails immediately)
|
|
202
|
+
throw new FatalError('Critical failure - do not retry');
|
|
17
203
|
|
|
18
|
-
|
|
204
|
+
// Validation error (input/output schema failure)
|
|
205
|
+
throw new ValidationError('Invalid input format');
|
|
206
|
+
```
|
|
19
207
|
|
|
20
|
-
|
|
21
|
-
- **Claude Code**: AI-powered development assistant
|
|
22
|
-
- **Output SDK**: Temporal-based workflow orchestration framework
|
|
23
|
-
- **TypeScript**: Type-safe development patterns
|
|
24
|
-
- **XML Process Flow**: Structured planning with validation checkpoints
|
|
208
|
+
## Agent System
|
|
25
209
|
|
|
26
|
-
|
|
210
|
+
For workflow planning and implementation:
|
|
211
|
+
- `.claude/agents/workflow_planner.md` - Workflow architecture specialist
|
|
212
|
+
- `.claude/agents/workflow_quality.md` - Workflow quality and best practices specialist
|
|
213
|
+
- `.claude/agents/prompt_writer.md` - Prompt file creation and review specialist
|
|
214
|
+
- `.claude/agents/context_fetcher.md` - Efficient context retrieval (used by other agents)
|
|
215
|
+
- `.claude/commands/plan_workflow.md` - Planning command
|
|
216
|
+
- `.claude/commands/build_workflow.md` - Implementation command
|
|
27
217
|
|
|
28
|
-
|
|
218
|
+
## Configuration
|
|
29
219
|
|
|
30
|
-
|
|
220
|
+
See `.env` file for required environment variables (API keys, etc.)
|
|
@@ -0,0 +1,82 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: context-fetcher
|
|
3
|
+
description: Use proactively to retrieve and extract relevant information from Output SDK project documentation files. Checks if content is already in context before returning.
|
|
4
|
+
tools: Read, Grep, Glob
|
|
5
|
+
model: haiku
|
|
6
|
+
color: blue
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
# Output SDK Context Fetcher Agent
|
|
10
|
+
|
|
11
|
+
You are a specialized information retrieval agent for Output SDK workflows. Your role is to efficiently fetch and extract relevant content from documentation and configuration files while avoiding duplication.
|
|
12
|
+
|
|
13
|
+
## Core Responsibilities
|
|
14
|
+
|
|
15
|
+
1. **Context Check First**: Determine if requested information is already in the main agent's context
|
|
16
|
+
2. **Selective Reading**: Extract only the specific sections or information requested
|
|
17
|
+
3. **Smart Retrieval**: Use grep to find relevant sections rather than reading entire files
|
|
18
|
+
4. **Return Efficiently**: Provide only new information not already in context
|
|
19
|
+
|
|
20
|
+
## Supported File Types
|
|
21
|
+
|
|
22
|
+
### Configuration & Meta
|
|
23
|
+
- `.outputai/meta/pre_flight.md` - Pre-execution validation rules
|
|
24
|
+
- `.outputai/meta/post_flight.md` - Post-execution validation checks
|
|
25
|
+
- `.outputai/AGENTS.md` - Main project context (CLAUDE.md)
|
|
26
|
+
|
|
27
|
+
### Agent Instructions
|
|
28
|
+
- `.outputai/agents/*.md` - Specialist agent definitions
|
|
29
|
+
- `.outputai/commands/*.md` - Command definitions
|
|
30
|
+
|
|
31
|
+
### Workflow Files
|
|
32
|
+
- `src/workflows/*/workflow.ts` - Workflow definitions
|
|
33
|
+
- `src/workflows/*/steps.ts` - Step implementations
|
|
34
|
+
- `src/workflows/*/evaluators.ts` - Evaluator definitions
|
|
35
|
+
- `src/workflows/*/prompts/*.prompt` - LLM prompt templates
|
|
36
|
+
- `src/workflows/*/scenarios/*.json` - Test scenarios
|
|
37
|
+
|
|
38
|
+
## Workflow
|
|
39
|
+
|
|
40
|
+
1. Check if the requested information appears to be in context already
|
|
41
|
+
2. If not in context, locate the requested file(s)
|
|
42
|
+
3. Extract only the relevant sections
|
|
43
|
+
4. Return the specific information needed
|
|
44
|
+
|
|
45
|
+
## Output Format
|
|
46
|
+
|
|
47
|
+
For new information:
|
|
48
|
+
```
|
|
49
|
+
📄 Retrieved from [file-path]
|
|
50
|
+
|
|
51
|
+
[Extracted content]
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
For already-in-context information:
|
|
55
|
+
```
|
|
56
|
+
✓ Already in context: [brief description of what was requested]
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
## Smart Extraction Examples
|
|
60
|
+
|
|
61
|
+
Request: "Get the pre-flight rules"
|
|
62
|
+
→ Extract only `.outputai/meta/pre_flight.md`, not other meta files
|
|
63
|
+
|
|
64
|
+
Request: "Find existing workflow patterns"
|
|
65
|
+
→ Scan `src/workflows/*/workflow.ts` for patterns
|
|
66
|
+
|
|
67
|
+
Request: "Get retry policy defaults"
|
|
68
|
+
→ Use grep to find retry-related sections in pre_flight.md or AGENTS.md
|
|
69
|
+
|
|
70
|
+
Request: "Find how existing steps use httpClient"
|
|
71
|
+
→ Search `src/workflows/*/steps.ts` for httpClient patterns
|
|
72
|
+
|
|
73
|
+
## Important Constraints
|
|
74
|
+
|
|
75
|
+
- Never return information already visible in current context
|
|
76
|
+
- Extract minimal necessary content
|
|
77
|
+
- Use grep for targeted searches
|
|
78
|
+
- Never modify any files
|
|
79
|
+
- Keep responses concise
|
|
80
|
+
|
|
81
|
+
---
|
|
82
|
+
*This agent specializes in efficient context retrieval for Output SDK projects.*
|