@micdrop/openai 1.0.1 → 1.0.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,30 +1,13 @@
1
1
  # @micdrop/openai
2
2
 
3
- OpenAI implementation for [@micdrop/server](../server/README.md).
3
+ [Micdrop website](https://micdrop.dev) | [Documentation](https://micdrop.dev/docs/ai-integration/provided-integrations/openai)
4
4
 
5
- This package provides AI agent and speech-to-text implementations using OpenAI's API.
6
-
7
- ## Features
8
-
9
- - 🤖 **OpenAI Agent** - GPT-powered conversational AI with advanced features:
10
- - Streaming responses for real-time conversation
11
- - Auto end call detection when user requests to end conversation
12
- - Semantic turn detection to handle incomplete sentences
13
- - User noise filtering to ignore meaningless interjections
14
- - Tool calling support for enhanced interactions
15
- - 🎙️ **OpenAI STT** - Whisper-powered speech-to-text with:
16
- - High-quality transcription using Whisper models
17
- - Multi-language support
18
- - Configurable model selection
5
+ OpenAI implementation for [@micdrop/server](https://micdrop.dev/docs/server).
19
6
 
20
7
  ## Installation
21
8
 
22
9
  ```bash
23
10
  npm install @micdrop/openai
24
- # or
25
- yarn add @micdrop/openai
26
- # or
27
- pnpm add @micdrop/openai
28
11
  ```
29
12
 
30
13
  ## OpenAI Agent
@@ -59,18 +42,6 @@ new MicdropServer(socket, {
59
42
  })
60
43
  ```
61
44
 
62
- ### Options
63
-
64
- | Option | Type | Default | Description |
65
- | --------------------- | ------------------- | ---------- | --------------------------------------- |
66
- | `apiKey` | `string` | Required | Your OpenAI API key |
67
- | `model` | `string` | `'gpt-4o'` | OpenAI model to use |
68
- | `systemPrompt` | `string` | Required | System prompt for the agent |
69
- | `autoEndCall` | `boolean \| string` | `false` | Auto-detect when user wants to end call |
70
- | `autoSemanticTurn` | `boolean \| string` | `false` | Handle incomplete user sentences |
71
- | `autoIgnoreUserNoise` | `boolean \| string` | `false` | Filter meaningless user sounds |
72
- | `settings` | `object` | `{}` | Additional OpenAI API parameters |
73
-
74
45
  ## OpenAI STT (Speech-to-Text)
75
46
 
76
47
  ### Usage
@@ -92,133 +63,9 @@ new MicdropServer(socket, {
92
63
  })
93
64
  ```
94
65
 
95
- ### Options
96
-
97
- | Option | Type | Default | Description |
98
- | ---------- | -------- | ------------- | ------------------------------- |
99
- | `apiKey` | `string` | Required | Your OpenAI API key |
100
- | `model` | `string` | `'whisper-1'` | Whisper model to use |
101
- | `language` | `string` | Optional | Language code for transcription |
102
-
103
- ## Tool Management
104
-
105
- The OpenAI Agent supports adding and removing custom tools to extend its capabilities.
106
-
107
- ### Adding Tools
108
-
109
- Use `addTool(tool: Tool)` to add custom functions that the agent can call during conversations:
110
-
111
- ```typescript
112
- import { z } from 'zod'
113
-
114
- const agent = new OpenaiAgent({
115
- apiKey: process.env.OPENAI_API_KEY || '',
116
- systemPrompt: 'You are a helpful assistant that can manage user information.',
117
- })
118
-
119
- // Add a simple tool without parameters
120
- agent.addTool({
121
- name: 'get_time',
122
- description: 'Get the current time',
123
- callback: () => new Date().toLocaleTimeString(),
124
- })
125
-
126
- // Add a tool with typed parameters using Zod schema
127
- agent.addTool({
128
- name: 'set_user_info',
129
- description: 'Save user information to the database',
130
- parameters: z.object({
131
- city: z.string().describe('City'),
132
- jobTitle: z.string().describe('Job title').nullable(),
133
- experience: z
134
- .number()
135
- .describe('Number of years of experience of the user')
136
- .nullable(),
137
- }),
138
- callback: ({ city, jobTitle, experience }) => {
139
- // Your implementation here
140
- console.log('Saving user:', { city, jobTitle, experience })
141
- return { success: true, message: 'User information saved' }
142
- },
143
- })
144
-
145
- // Add a tool that returns data for the conversation
146
- agent.addTool({
147
- name: 'search_database',
148
- description: 'Search for items in the database',
149
- parameters: z.object({
150
- query: z.string().describe('Search query'),
151
- limit: z.number().default(10).describe('Maximum number of results'),
152
- }),
153
- callback: async ({ query, limit }) => {
154
- // Your search implementation
155
- const results = await searchDatabase(query, limit)
156
- return { results, total: results.length }
157
- },
158
- })
159
- ```
160
-
161
- ### Removing Tools
162
-
163
- Use `removeTool(name: string)` to remove tools by name:
164
-
165
- ```typescript
166
- // Remove a specific tool
167
- agent.removeTool('get_time')
168
- ```
169
-
170
- ### Getting Tools
171
-
172
- Use `getTool(name: string)` to retrieve a tool by name:
66
+ ## Documentation
173
67
 
174
- ```typescript
175
- // Get a specific tool (undefined if not found)
176
- const tool = agent.getTool('get_time')
177
- ```
178
-
179
- ## Advanced Features
180
-
181
- ### Auto End Call
182
-
183
- When enabled, the agent automatically detects when the user wants to end the conversation and triggers the call termination:
184
-
185
- ```typescript
186
- const agent = new OpenaiAgent({
187
- apiKey: process.env.OPENAI_API_KEY || '',
188
- systemPrompt: 'You are a helpful assistant',
189
- autoEndCall: true, // Use default detection
190
- // or provide custom prompt:
191
- // autoEndCall: 'User is saying goodbye or wants to hang up',
192
- })
193
- ```
194
-
195
- ### Semantic Turn Detection
196
-
197
- Handles cases where users speak incomplete sentences, allowing for more natural conversation flow:
198
-
199
- ```typescript
200
- const agent = new OpenaiAgent({
201
- apiKey: process.env.OPENAI_API_KEY || '',
202
- systemPrompt: 'You are a helpful assistant',
203
- autoSemanticTurn: true, // Wait for complete thoughts
204
- // or provide custom prompt:
205
- // autoSemanticTurn: 'Last user message is an incomplete sentence',
206
- })
207
- ```
208
-
209
- ### User Noise Filtering
210
-
211
- Filters out meaningless sounds like "uh", "hmm", "ahem" that don't carry conversational meaning:
212
-
213
- ```typescript
214
- const agent = new OpenaiAgent({
215
- apiKey: process.env.OPENAI_API_KEY || '',
216
- systemPrompt: 'You are a helpful assistant',
217
- autoIgnoreUserNoise: true, // Ignore filler sounds
218
- // or provide custom prompt:
219
- // autoIgnoreUserNoise: 'Last user message is just an interjection',
220
- })
221
- ```
68
+ Read full [documentation of the OpenAI integration for Micdrop](https://micdrop.dev/docs/ai-integration/provided-integrations/openai) on the [website](https://micdrop.dev).
222
69
 
223
70
  ## License
224
71
 
package/dist/index.d.mts CHANGED
@@ -25,10 +25,7 @@ declare class OpenaiAgent extends Agent<OpenaiAgentOptions> {
25
25
  private tools;
26
26
  private abortController?;
27
27
  constructor(options: OpenaiAgentOptions);
28
- answer(): {
29
- message: Promise<string>;
30
- stream: PassThrough;
31
- };
28
+ answer(): PassThrough;
32
29
  private generateAnswer;
33
30
  addTool<Schema extends z.ZodObject>(tool: Tool<Schema>): void;
34
31
  removeTool(name: string): void;
package/dist/index.d.ts CHANGED
@@ -25,10 +25,7 @@ declare class OpenaiAgent extends Agent<OpenaiAgentOptions> {
25
25
  private tools;
26
26
  private abortController?;
27
27
  constructor(options: OpenaiAgentOptions);
28
- answer(): {
29
- message: Promise<string>;
30
- stream: PassThrough;
31
- };
28
+ answer(): PassThrough;
32
29
  private generateAnswer;
33
30
  addTool<Schema extends z.ZodObject>(tool: Tool<Schema>): void;
34
31
  removeTool(name: string): void;
package/dist/index.js CHANGED
@@ -78,19 +78,17 @@ var OpenaiAgent = class extends import_server.Agent {
78
78
  this.cancel();
79
79
  this.log("Start answering");
80
80
  const stream = new import_stream.PassThrough();
81
- const textPromise = this.createTextPromise();
82
- this.generateAnswer(stream, textPromise).finally(() => {
81
+ this.generateAnswer(stream).finally(() => {
83
82
  this.abortController = void 0;
84
83
  if (stream.writable) {
85
84
  stream.end();
86
85
  }
87
86
  });
88
- return { message: textPromise.promise, stream };
87
+ return stream;
89
88
  }
90
- async generateAnswer(stream, textPromise, callCount = 0, toolInputs) {
89
+ async generateAnswer(stream, callCount = 0, toolInputs) {
91
90
  if (callCount >= (this.options.maxCalls || DEFAULT_MAX_CALLS)) {
92
91
  console.error("[OpenaiAgent] Max calls reached");
93
- textPromise.resolve("");
94
92
  return;
95
93
  }
96
94
  this.abortController = new AbortController();
@@ -130,7 +128,6 @@ var OpenaiAgent = class extends import_server.Agent {
130
128
  case "response.output_text.done":
131
129
  this.addAssistantMessage(event.text);
132
130
  stream.end();
133
- textPromise.resolve(event.text);
134
131
  hasAnswer = true;
135
132
  break;
136
133
  case "response.output_item.done":
@@ -147,17 +144,9 @@ var OpenaiAgent = class extends import_server.Agent {
147
144
  }
148
145
  }
149
146
  if (toolOutputs.length > 0 && !hasAnswer && !skipAnswer) {
150
- await this.generateAnswer(
151
- stream,
152
- textPromise,
153
- callCount + 1,
154
- toolOutputs
155
- );
156
- } else {
157
- textPromise.resolve("");
147
+ await this.generateAnswer(stream, callCount + 1, toolOutputs);
158
148
  }
159
149
  } catch (error) {
160
- textPromise.resolve("");
161
150
  if (!(error instanceof import_openai.default.APIUserAbortError)) {
162
151
  console.error("[OpenaiAgent] Error answering:", error);
163
152
  }
package/dist/index.js.map CHANGED
@@ -1 +1 @@
1
- {"version":3,"sources":["../src/index.ts","../src/OpenaiAgent.ts","../src/tools.ts","../src/OpenaiSTT.ts"],"sourcesContent":["export * from './OpenaiAgent'\nexport * from './OpenaiSTT'\n","import { Agent, AgentOptions, TextPromise } from '@micdrop/server'\nimport OpenAI from 'openai'\nimport { PassThrough } from 'stream'\nimport { z } from 'zod'\nimport {\n createToolCallOutput,\n createToolSchema,\n Tool,\n ToolCall,\n ToolCallOutput,\n} from './tools'\n\nexport interface OpenaiAgentOptions extends AgentOptions {\n apiKey: string\n model?: string\n maxCalls?: number\n settings?: Omit<\n OpenAI.Responses.ResponseCreateParamsStreaming,\n 'input' | 'model'\n >\n\n // Enable auto ending of the call when user asks to end the call\n // You can provide a custom prompt to use instead of the default one by passing a string\n autoEndCall?: boolean | string\n\n // Enable detection of an incomplete sentence, and skip the answer (assistant waits)\n // You can provide a custom prompt to use instead of the default one by passing a string\n autoSemanticTurn?: boolean | string\n\n // Ignore of the last user message when it's meaningless\n // You can provide a custom prompt to use instead of the default one by passing a string\n autoIgnoreUserNoise?: boolean | string\n}\n\nconst DEFAULT_MODEL = 'gpt-4o'\nconst DEFAULT_MAX_CALLS = 5\n\nconst AUTO_END_CALL_TOOL_NAME = 'end_call'\nconst AUTO_END_CALL_PROMPT = 'Call this tool only if user asks to end the call'\n\nconst AUTO_SEMANTIC_TURN_TOOL_NAME = 'semantic_turn'\nconst AUTO_SEMANTIC_TURN_PROMPT =\n 'Call this tool only if last user message is obviously an incomplete sentence that you need to wait for the end before answering'\n\nconst AUTO_IGNORE_USER_NOISE_TOOL_NAME = 'ignore_user_noise'\nconst AUTO_IGNORE_USER_NOISE_PROMPT =\n 'Call this tool only if last user message is just an interjection or a sound that expresses emotion, hesitation, or reaction (ex: \"Uh\", \"Ahem\", \"Hmm\", \"Ah\") but doesn\\'t carry any clear meaning like agreeing, refusing, or commanding'\n\nexport class OpenaiAgent extends Agent<OpenaiAgentOptions> {\n private openai: OpenAI\n private tools: Tool[]\n private abortController?: AbortController\n\n constructor(options: OpenaiAgentOptions) {\n super(options)\n this.openai = new OpenAI({ apiKey: options.apiKey })\n this.tools = this.getDefaultTools()\n }\n\n answer() {\n this.cancel()\n this.log('Start answering')\n const stream = new PassThrough()\n const textPromise = this.createTextPromise()\n\n this.generateAnswer(stream, textPromise).finally(() => {\n this.abortController = undefined\n if (stream.writable) {\n stream.end()\n }\n })\n\n return { message: textPromise.promise, stream }\n }\n\n private async generateAnswer(\n stream: PassThrough,\n textPromise: TextPromise,\n callCount = 0,\n toolInputs?: Array<ToolCall | ToolCallOutput>\n ): Promise<void> {\n if (callCount >= (this.options.maxCalls || DEFAULT_MAX_CALLS)) {\n console.error('[OpenaiAgent] Max calls reached')\n textPromise.resolve('')\n return\n }\n\n this.abortController = new AbortController()\n const signal = this.abortController.signal\n\n // Disable tools if first message\n const enableTools = this.conversation.length > 1\n\n // Build input\n const input: OpenAI.Responses.ResponseInput = this.conversation.map(\n (message) => ({\n role: message.role,\n content: message.content,\n })\n )\n if (toolInputs) {\n input.push(...toolInputs)\n }\n\n try {\n // Generate answer\n const response = await this.openai.responses.create(\n {\n model: this.options.model || DEFAULT_MODEL,\n input,\n temperature: 0.5,\n max_output_tokens: 250,\n stream: true,\n tools: enableTools ? this.tools.map(createToolSchema) : undefined,\n ...this.options.settings,\n },\n { signal }\n )\n const toolOutputs: Array<ToolCall | ToolCallOutput> = []\n let hasAnswer = false\n let skipAnswer = false\n\n // Handle response events\n for await (const event of response) {\n switch (event.type) {\n case 'response.output_text.delta':\n this.log(`Answer chunk: \"${event.delta}\"`)\n stream.write(event.delta)\n break\n\n case 'response.output_text.done':\n this.addAssistantMessage(event.text)\n stream.end()\n textPromise.resolve(event.text)\n hasAnswer = true\n break\n\n case 'response.output_item.done':\n if (event.item.type === 'function_call') {\n const result = await this.executeTool(event.item)\n toolOutputs.push(result.toolCall, result.toolCallOutput)\n if (result.skipAnswer) {\n skipAnswer = true\n }\n }\n break\n\n default:\n break\n }\n }\n\n // Query again in case of tool call\n if (toolOutputs.length > 0 && !hasAnswer && !skipAnswer) {\n await this.generateAnswer(\n stream,\n textPromise,\n callCount + 1,\n toolOutputs\n )\n } else {\n // Ensure promise is resolved (if skipAnswer is true in a tool)\n textPromise.resolve('')\n }\n } catch (error) {\n textPromise.resolve('')\n if (!(error instanceof OpenAI.APIUserAbortError)) {\n console.error('[OpenaiAgent] Error answering:', error)\n }\n }\n }\n\n addTool<Schema extends z.ZodObject>(tool: Tool<Schema>) {\n this.tools.push(tool)\n }\n\n removeTool(name: string) {\n this.tools = this.tools.filter((tool) => tool.name !== name)\n }\n\n getTool(name: string): Tool | undefined {\n return this.tools.find((tool) => tool.name === name)\n }\n\n private getDefaultTools() {\n const tools: Tool[] = []\n if (this.options.autoEndCall) {\n tools.push({\n name: AUTO_END_CALL_TOOL_NAME,\n description:\n typeof this.options.autoEndCall === 'string'\n ? this.options.autoEndCall\n : AUTO_END_CALL_PROMPT,\n callback: () => this.endCall(),\n })\n }\n if (this.options.autoSemanticTurn) {\n tools.push({\n name: AUTO_SEMANTIC_TURN_TOOL_NAME,\n description:\n typeof this.options.autoSemanticTurn === 'string'\n ? this.options.autoSemanticTurn\n : AUTO_SEMANTIC_TURN_PROMPT,\n skipAnswer: true,\n callback: () => this.skipAnswer(),\n })\n }\n if (this.options.autoIgnoreUserNoise) {\n tools.push({\n name: AUTO_IGNORE_USER_NOISE_TOOL_NAME,\n description:\n typeof this.options.autoIgnoreUserNoise === 'string'\n ? this.options.autoIgnoreUserNoise\n : AUTO_IGNORE_USER_NOISE_PROMPT,\n skipAnswer: true,\n callback: () => this.cancelLastUserMessage(),\n })\n }\n return tools\n }\n\n private async executeTool(toolCall: ToolCall) {\n try {\n const tool = this.getTool(toolCall.name)\n if (!tool) {\n throw new Error(`Tool not found \"${toolCall.name}\"`)\n }\n\n this.log('Executing tool:', toolCall.name, toolCall.arguments)\n const args = JSON.parse(toolCall.arguments)\n const result = await tool.callback(args)\n\n return {\n toolCall,\n toolCallOutput: createToolCallOutput(toolCall, result || {}),\n skipAnswer: tool.skipAnswer,\n }\n } catch (error: any) {\n console.error('[OpenaiAgent] Error executing tool:', error)\n return {\n toolCall,\n toolCallOutput: createToolCallOutput(toolCall, {\n error: error.message,\n }),\n }\n }\n }\n\n cancel() {\n if (!this.abortController) return\n this.log('Cancel')\n this.abortController.abort()\n this.abortController = undefined\n }\n}\n","import {\n FunctionTool,\n ResponseInputItem,\n Responses,\n} from 'openai/resources/responses/responses'\nimport { toJSONSchema, z } from 'zod'\n\nexport interface Tool<Schema extends z.ZodObject = z.ZodObject> {\n name: string\n description: string\n parameters?: Schema\n callback: (input: z.infer<Schema>) => any | Promise<any>\n skipAnswer?: boolean\n}\n\nexport type ToolCall = Responses.ResponseFunctionToolCall\nexport type ToolCallOutput = ResponseInputItem.FunctionCallOutput\n\nexport function createToolSchema<Schema extends z.ZodObject>(\n tool: Tool<Schema>\n): FunctionTool {\n return {\n type: 'function',\n name: tool.name,\n description: tool.description,\n strict: true,\n parameters: toJSONSchema(tool.parameters || z.object()),\n }\n}\n\nexport function createToolCallOutput(\n toolCall: ToolCall,\n args: any\n): ToolCallOutput {\n return {\n type: 'function_call_output',\n call_id: toolCall.call_id,\n output: JSON.stringify(args),\n }\n}\n","import { FileSTT } from '@micdrop/server'\n\n/**\n * OpenAI Whisper STT\n *\n * @see https://platform.openai.com/docs/guides/speech-to-text?lang=curl\n */\n\nexport interface OpenaiSTTOptions {\n apiKey: string\n model?: string\n language?: string\n}\n\nconst DEFAULT_MODEL = 'whisper-1'\n\nexport class OpenaiSTT extends FileSTT {\n constructor(private options: OpenaiSTTOptions) {\n super()\n }\n\n async transcribeFile(file: File) {\n // Prepare form data\n const formData = new FormData()\n formData.append('file', file)\n formData.append('model', this.options.model || DEFAULT_MODEL)\n formData.append('response_format', 'text')\n\n if (this.options.language) {\n formData.append('language', this.options.language)\n }\n\n const response = await fetch(\n 'https://api.openai.com/v1/audio/transcriptions',\n {\n method: 'POST',\n headers: {\n Authorization: `Bearer ${this.options.apiKey}`,\n },\n body: formData,\n }\n )\n\n if (!response.ok) {\n this.log('Failed to transcribe audio', await response.json())\n throw new Error(`Failed to transcribe audio: ${response.statusText}`)\n }\n\n const transcript = await response.text()\n this.log(`Received transcript: \"${transcript}\"`)\n\n return transcript\n }\n}\n"],"mappings":";;;;;;;;;;;;;;;;;;;;;;;;;;;;;;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;;;ACAA,oBAAiD;AACjD,oBAAmB;AACnB,oBAA4B;;;ACG5B,iBAAgC;AAazB,SAAS,iBACd,MACc;AACd,SAAO;AAAA,IACL,MAAM;AAAA,IACN,MAAM,KAAK;AAAA,IACX,aAAa,KAAK;AAAA,IAClB,QAAQ;AAAA,IACR,gBAAY,yBAAa,KAAK,cAAc,aAAE,OAAO,CAAC;AAAA,EACxD;AACF;AAEO,SAAS,qBACd,UACA,MACgB;AAChB,SAAO;AAAA,IACL,MAAM;AAAA,IACN,SAAS,SAAS;AAAA,IAClB,QAAQ,KAAK,UAAU,IAAI;AAAA,EAC7B;AACF;;;ADLA,IAAM,gBAAgB;AACtB,IAAM,oBAAoB;AAE1B,IAAM,0BAA0B;AAChC,IAAM,uBAAuB;AAE7B,IAAM,+BAA+B;AACrC,IAAM,4BACJ;AAEF,IAAM,mCAAmC;AACzC,IAAM,gCACJ;AAEK,IAAM,cAAN,cAA0B,oBAA0B;AAAA,EAKzD,YAAY,SAA6B;AACvC,UAAM,OAAO;AACb,SAAK,SAAS,IAAI,cAAAA,QAAO,EAAE,QAAQ,QAAQ,OAAO,CAAC;AACnD,SAAK,QAAQ,KAAK,gBAAgB;AAAA,EACpC;AAAA,EAEA,SAAS;AACP,SAAK,OAAO;AACZ,SAAK,IAAI,iBAAiB;AAC1B,UAAM,SAAS,IAAI,0BAAY;AAC/B,UAAM,cAAc,KAAK,kBAAkB;AAE3C,SAAK,eAAe,QAAQ,WAAW,EAAE,QAAQ,MAAM;AACrD,WAAK,kBAAkB;AACvB,UAAI,OAAO,UAAU;AACnB,eAAO,IAAI;AAAA,MACb;AAAA,IACF,CAAC;AAED,WAAO,EAAE,SAAS,YAAY,SAAS,OAAO;AAAA,EAChD;AAAA,EAEA,MAAc,eACZ,QACA,aACA,YAAY,GACZ,YACe;AACf,QAAI,cAAc,KAAK,QAAQ,YAAY,oBAAoB;AAC7D,cAAQ,MAAM,iCAAiC;AAC/C,kBAAY,QAAQ,EAAE;AACtB;AAAA,IACF;AAEA,SAAK,kBAAkB,IAAI,gBAAgB;AAC3C,UAAM,SAAS,KAAK,gBAAgB;AAGpC,UAAM,cAAc,KAAK,aAAa,SAAS;AAG/C,UAAM,QAAwC,KAAK,aAAa;AAAA,MAC9D,CAAC,aAAa;AAAA,QACZ,MAAM,QAAQ;AAAA,QACd,SAAS,QAAQ;AAAA,MACnB;AAAA,IACF;AACA,QAAI,YAAY;AACd,YAAM,KAAK,GAAG,UAAU;AAAA,IAC1B;AAEA,QAAI;AAEF,YAAM,WAAW,MAAM,KAAK,OAAO,UAAU;AAAA,QAC3C;AAAA,UACE,OAAO,KAAK,QAAQ,SAAS;AAAA,UAC7B;AAAA,UACA,aAAa;AAAA,UACb,mBAAmB;AAAA,UACnB,QAAQ;AAAA,UACR,OAAO,cAAc,KAAK,MAAM,IAAI,gBAAgB,IAAI;AAAA,UACxD,GAAG,KAAK,QAAQ;AAAA,QAClB;AAAA,QACA,EAAE,OAAO;AAAA,MACX;AACA,YAAM,cAAgD,CAAC;AACvD,UAAI,YAAY;AAChB,UAAI,aAAa;AAGjB,uBAAiB,SAAS,UAAU;AAClC,gBAAQ,MAAM,MAAM;AAAA,UAClB,KAAK;AACH,iBAAK,IAAI,kBAAkB,MAAM,KAAK,GAAG;AACzC,mBAAO,MAAM,MAAM,KAAK;AACxB;AAAA,UAEF,KAAK;AACH,iBAAK,oBAAoB,MAAM,IAAI;AACnC,mBAAO,IAAI;AACX,wBAAY,QAAQ,MAAM,IAAI;AAC9B,wBAAY;AACZ;AAAA,UAEF,KAAK;AACH,gBAAI,MAAM,KAAK,SAAS,iBAAiB;AACvC,oBAAM,SAAS,MAAM,KAAK,YAAY,MAAM,IAAI;AAChD,0BAAY,KAAK,OAAO,UAAU,OAAO,cAAc;AACvD,kBAAI,OAAO,YAAY;AACrB,6BAAa;AAAA,cACf;AAAA,YACF;AACA;AAAA,UAEF;AACE;AAAA,QACJ;AAAA,MACF;AAGA,UAAI,YAAY,SAAS,KAAK,CAAC,aAAa,CAAC,YAAY;AACvD,cAAM,KAAK;AAAA,UACT;AAAA,UACA;AAAA,UACA,YAAY;AAAA,UACZ;AAAA,QACF;AAAA,MACF,OAAO;AAEL,oBAAY,QAAQ,EAAE;AAAA,MACxB;AAAA,IACF,SAAS,OAAO;AACd,kBAAY,QAAQ,EAAE;AACtB,UAAI,EAAE,iBAAiB,cAAAA,QAAO,oBAAoB;AAChD,gBAAQ,MAAM,kCAAkC,KAAK;AAAA,MACvD;AAAA,IACF;AAAA,EACF;AAAA,EAEA,QAAoC,MAAoB;AACtD,SAAK,MAAM,KAAK,IAAI;AAAA,EACtB;AAAA,EAEA,WAAW,MAAc;AACvB,SAAK,QAAQ,KAAK,MAAM,OAAO,CAAC,SAAS,KAAK,SAAS,IAAI;AAAA,EAC7D;AAAA,EAEA,QAAQ,MAAgC;AACtC,WAAO,KAAK,MAAM,KAAK,CAAC,SAAS,KAAK,SAAS,IAAI;AAAA,EACrD;AAAA,EAEQ,kBAAkB;AACxB,UAAM,QAAgB,CAAC;AACvB,QAAI,KAAK,QAAQ,aAAa;AAC5B,YAAM,KAAK;AAAA,QACT,MAAM;AAAA,QACN,aACE,OAAO,KAAK,QAAQ,gBAAgB,WAChC,KAAK,QAAQ,cACb;AAAA,QACN,UAAU,MAAM,KAAK,QAAQ;AAAA,MAC/B,CAAC;AAAA,IACH;AACA,QAAI,KAAK,QAAQ,kBAAkB;AACjC,YAAM,KAAK;AAAA,QACT,MAAM;AAAA,QACN,aACE,OAAO,KAAK,QAAQ,qBAAqB,WACrC,KAAK,QAAQ,mBACb;AAAA,QACN,YAAY;AAAA,QACZ,UAAU,MAAM,KAAK,WAAW;AAAA,MAClC,CAAC;AAAA,IACH;AACA,QAAI,KAAK,QAAQ,qBAAqB;AACpC,YAAM,KAAK;AAAA,QACT,MAAM;AAAA,QACN,aACE,OAAO,KAAK,QAAQ,wBAAwB,WACxC,KAAK,QAAQ,sBACb;AAAA,QACN,YAAY;AAAA,QACZ,UAAU,MAAM,KAAK,sBAAsB;AAAA,MAC7C,CAAC;AAAA,IACH;AACA,WAAO;AAAA,EACT;AAAA,EAEA,MAAc,YAAY,UAAoB;AAC5C,QAAI;AACF,YAAM,OAAO,KAAK,QAAQ,SAAS,IAAI;AACvC,UAAI,CAAC,MAAM;AACT,cAAM,IAAI,MAAM,mBAAmB,SAAS,IAAI,GAAG;AAAA,MACrD;AAEA,WAAK,IAAI,mBAAmB,SAAS,MAAM,SAAS,SAAS;AAC7D,YAAM,OAAO,KAAK,MAAM,SAAS,SAAS;AAC1C,YAAM,SAAS,MAAM,KAAK,SAAS,IAAI;AAEvC,aAAO;AAAA,QACL;AAAA,QACA,gBAAgB,qBAAqB,UAAU,UAAU,CAAC,CAAC;AAAA,QAC3D,YAAY,KAAK;AAAA,MACnB;AAAA,IACF,SAAS,OAAY;AACnB,cAAQ,MAAM,uCAAuC,KAAK;AAC1D,aAAO;AAAA,QACL;AAAA,QACA,gBAAgB,qBAAqB,UAAU;AAAA,UAC7C,OAAO,MAAM;AAAA,QACf,CAAC;AAAA,MACH;AAAA,IACF;AAAA,EACF;AAAA,EAEA,SAAS;AACP,QAAI,CAAC,KAAK,gBAAiB;AAC3B,SAAK,IAAI,QAAQ;AACjB,SAAK,gBAAgB,MAAM;AAC3B,SAAK,kBAAkB;AAAA,EACzB;AACF;;;AE9PA,IAAAC,iBAAwB;AAcxB,IAAMC,iBAAgB;AAEf,IAAM,YAAN,cAAwB,uBAAQ;AAAA,EACrC,YAAoB,SAA2B;AAC7C,UAAM;AADY;AAAA,EAEpB;AAAA,EAEA,MAAM,eAAe,MAAY;AAE/B,UAAM,WAAW,IAAI,SAAS;AAC9B,aAAS,OAAO,QAAQ,IAAI;AAC5B,aAAS,OAAO,SAAS,KAAK,QAAQ,SAASA,cAAa;AAC5D,aAAS,OAAO,mBAAmB,MAAM;AAEzC,QAAI,KAAK,QAAQ,UAAU;AACzB,eAAS,OAAO,YAAY,KAAK,QAAQ,QAAQ;AAAA,IACnD;AAEA,UAAM,WAAW,MAAM;AAAA,MACrB;AAAA,MACA;AAAA,QACE,QAAQ;AAAA,QACR,SAAS;AAAA,UACP,eAAe,UAAU,KAAK,QAAQ,MAAM;AAAA,QAC9C;AAAA,QACA,MAAM;AAAA,MACR;AAAA,IACF;AAEA,QAAI,CAAC,SAAS,IAAI;AAChB,WAAK,IAAI,8BAA8B,MAAM,SAAS,KAAK,CAAC;AAC5D,YAAM,IAAI,MAAM,+BAA+B,SAAS,UAAU,EAAE;AAAA,IACtE;AAEA,UAAM,aAAa,MAAM,SAAS,KAAK;AACvC,SAAK,IAAI,yBAAyB,UAAU,GAAG;AAE/C,WAAO;AAAA,EACT;AACF;","names":["OpenAI","import_server","DEFAULT_MODEL"]}
1
+ {"version":3,"sources":["../src/index.ts","../src/OpenaiAgent.ts","../src/tools.ts","../src/OpenaiSTT.ts"],"sourcesContent":["export * from './OpenaiAgent'\nexport * from './OpenaiSTT'\n","import { Agent, AgentOptions } from '@micdrop/server'\nimport OpenAI from 'openai'\nimport { PassThrough } from 'stream'\nimport { z } from 'zod'\nimport {\n createToolCallOutput,\n createToolSchema,\n Tool,\n ToolCall,\n ToolCallOutput,\n} from './tools'\n\nexport interface OpenaiAgentOptions extends AgentOptions {\n apiKey: string\n model?: string\n maxCalls?: number\n settings?: Omit<\n OpenAI.Responses.ResponseCreateParamsStreaming,\n 'input' | 'model'\n >\n\n // Enable auto ending of the call when user asks to end the call\n // You can provide a custom prompt to use instead of the default one by passing a string\n autoEndCall?: boolean | string\n\n // Enable detection of an incomplete sentence, and skip the answer (assistant waits)\n // You can provide a custom prompt to use instead of the default one by passing a string\n autoSemanticTurn?: boolean | string\n\n // Ignore of the last user message when it's meaningless\n // You can provide a custom prompt to use instead of the default one by passing a string\n autoIgnoreUserNoise?: boolean | string\n}\n\nconst DEFAULT_MODEL = 'gpt-4o'\nconst DEFAULT_MAX_CALLS = 5\n\nconst AUTO_END_CALL_TOOL_NAME = 'end_call'\nconst AUTO_END_CALL_PROMPT = 'Call this tool only if user asks to end the call'\n\nconst AUTO_SEMANTIC_TURN_TOOL_NAME = 'semantic_turn'\nconst AUTO_SEMANTIC_TURN_PROMPT =\n 'Call this tool only if last user message is obviously an incomplete sentence that you need to wait for the end before answering'\n\nconst AUTO_IGNORE_USER_NOISE_TOOL_NAME = 'ignore_user_noise'\nconst AUTO_IGNORE_USER_NOISE_PROMPT =\n 'Call this tool only if last user message is just an interjection or a sound that expresses emotion, hesitation, or reaction (ex: \"Uh\", \"Ahem\", \"Hmm\", \"Ah\") but doesn\\'t carry any clear meaning like agreeing, refusing, or commanding'\n\nexport class OpenaiAgent extends Agent<OpenaiAgentOptions> {\n private openai: OpenAI\n private tools: Tool[]\n private abortController?: AbortController\n\n constructor(options: OpenaiAgentOptions) {\n super(options)\n this.openai = new OpenAI({ apiKey: options.apiKey })\n this.tools = this.getDefaultTools()\n }\n\n answer() {\n this.cancel()\n this.log('Start answering')\n const stream = new PassThrough()\n\n this.generateAnswer(stream).finally(() => {\n this.abortController = undefined\n if (stream.writable) {\n stream.end()\n }\n })\n\n return stream\n }\n\n private async generateAnswer(\n stream: PassThrough,\n callCount = 0,\n toolInputs?: Array<ToolCall | ToolCallOutput>\n ): Promise<void> {\n if (callCount >= (this.options.maxCalls || DEFAULT_MAX_CALLS)) {\n console.error('[OpenaiAgent] Max calls reached')\n return\n }\n\n this.abortController = new AbortController()\n const signal = this.abortController.signal\n\n // Disable tools if first message\n const enableTools = this.conversation.length > 1\n\n // Build input\n const input: OpenAI.Responses.ResponseInput = this.conversation.map(\n (message) => ({\n role: message.role,\n content: message.content,\n })\n )\n if (toolInputs) {\n input.push(...toolInputs)\n }\n\n try {\n // Generate answer\n const response = await this.openai.responses.create(\n {\n model: this.options.model || DEFAULT_MODEL,\n input,\n temperature: 0.5,\n max_output_tokens: 250,\n stream: true,\n tools: enableTools ? this.tools.map(createToolSchema) : undefined,\n ...this.options.settings,\n },\n { signal }\n )\n const toolOutputs: Array<ToolCall | ToolCallOutput> = []\n let hasAnswer = false\n let skipAnswer = false\n\n // Handle response events\n for await (const event of response) {\n switch (event.type) {\n case 'response.output_text.delta':\n this.log(`Answer chunk: \"${event.delta}\"`)\n stream.write(event.delta)\n break\n\n case 'response.output_text.done':\n this.addAssistantMessage(event.text)\n stream.end()\n hasAnswer = true\n break\n\n case 'response.output_item.done':\n if (event.item.type === 'function_call') {\n const result = await this.executeTool(event.item)\n toolOutputs.push(result.toolCall, result.toolCallOutput)\n if (result.skipAnswer) {\n skipAnswer = true\n }\n }\n break\n\n default:\n break\n }\n }\n\n // Query again in case of tool call\n if (toolOutputs.length > 0 && !hasAnswer && !skipAnswer) {\n await this.generateAnswer(stream, callCount + 1, toolOutputs)\n }\n } catch (error) {\n if (!(error instanceof OpenAI.APIUserAbortError)) {\n console.error('[OpenaiAgent] Error answering:', error)\n }\n }\n }\n\n addTool<Schema extends z.ZodObject>(tool: Tool<Schema>) {\n this.tools.push(tool)\n }\n\n removeTool(name: string) {\n this.tools = this.tools.filter((tool) => tool.name !== name)\n }\n\n getTool(name: string): Tool | undefined {\n return this.tools.find((tool) => tool.name === name)\n }\n\n private getDefaultTools() {\n const tools: Tool[] = []\n if (this.options.autoEndCall) {\n tools.push({\n name: AUTO_END_CALL_TOOL_NAME,\n description:\n typeof this.options.autoEndCall === 'string'\n ? this.options.autoEndCall\n : AUTO_END_CALL_PROMPT,\n callback: () => this.endCall(),\n })\n }\n if (this.options.autoSemanticTurn) {\n tools.push({\n name: AUTO_SEMANTIC_TURN_TOOL_NAME,\n description:\n typeof this.options.autoSemanticTurn === 'string'\n ? this.options.autoSemanticTurn\n : AUTO_SEMANTIC_TURN_PROMPT,\n skipAnswer: true,\n callback: () => this.skipAnswer(),\n })\n }\n if (this.options.autoIgnoreUserNoise) {\n tools.push({\n name: AUTO_IGNORE_USER_NOISE_TOOL_NAME,\n description:\n typeof this.options.autoIgnoreUserNoise === 'string'\n ? this.options.autoIgnoreUserNoise\n : AUTO_IGNORE_USER_NOISE_PROMPT,\n skipAnswer: true,\n callback: () => this.cancelLastUserMessage(),\n })\n }\n return tools\n }\n\n private async executeTool(toolCall: ToolCall) {\n try {\n const tool = this.getTool(toolCall.name)\n if (!tool) {\n throw new Error(`Tool not found \"${toolCall.name}\"`)\n }\n\n this.log('Executing tool:', toolCall.name, toolCall.arguments)\n const args = JSON.parse(toolCall.arguments)\n const result = await tool.callback(args)\n\n return {\n toolCall,\n toolCallOutput: createToolCallOutput(toolCall, result || {}),\n skipAnswer: tool.skipAnswer,\n }\n } catch (error: any) {\n console.error('[OpenaiAgent] Error executing tool:', error)\n return {\n toolCall,\n toolCallOutput: createToolCallOutput(toolCall, {\n error: error.message,\n }),\n }\n }\n }\n\n cancel() {\n if (!this.abortController) return\n this.log('Cancel')\n this.abortController.abort()\n this.abortController = undefined\n }\n}\n","import {\n FunctionTool,\n ResponseInputItem,\n Responses,\n} from 'openai/resources/responses/responses'\nimport { toJSONSchema, z } from 'zod'\n\nexport interface Tool<Schema extends z.ZodObject = z.ZodObject> {\n name: string\n description: string\n parameters?: Schema\n callback: (input: z.infer<Schema>) => any | Promise<any>\n skipAnswer?: boolean\n}\n\nexport type ToolCall = Responses.ResponseFunctionToolCall\nexport type ToolCallOutput = ResponseInputItem.FunctionCallOutput\n\nexport function createToolSchema<Schema extends z.ZodObject>(\n tool: Tool<Schema>\n): FunctionTool {\n return {\n type: 'function',\n name: tool.name,\n description: tool.description,\n strict: true,\n parameters: toJSONSchema(tool.parameters || z.object()),\n }\n}\n\nexport function createToolCallOutput(\n toolCall: ToolCall,\n args: any\n): ToolCallOutput {\n return {\n type: 'function_call_output',\n call_id: toolCall.call_id,\n output: JSON.stringify(args),\n }\n}\n","import { FileSTT } from '@micdrop/server'\n\n/**\n * OpenAI Whisper STT\n *\n * @see https://platform.openai.com/docs/guides/speech-to-text?lang=curl\n */\n\nexport interface OpenaiSTTOptions {\n apiKey: string\n model?: string\n language?: string\n}\n\nconst DEFAULT_MODEL = 'whisper-1'\n\nexport class OpenaiSTT extends FileSTT {\n constructor(private options: OpenaiSTTOptions) {\n super()\n }\n\n async transcribeFile(file: File) {\n // Prepare form data\n const formData = new FormData()\n formData.append('file', file)\n formData.append('model', this.options.model || DEFAULT_MODEL)\n formData.append('response_format', 'text')\n\n if (this.options.language) {\n formData.append('language', this.options.language)\n }\n\n const response = await fetch(\n 'https://api.openai.com/v1/audio/transcriptions',\n {\n method: 'POST',\n headers: {\n Authorization: `Bearer ${this.options.apiKey}`,\n },\n body: formData,\n }\n )\n\n if (!response.ok) {\n this.log('Failed to transcribe audio', await response.json())\n throw new Error(`Failed to transcribe audio: ${response.statusText}`)\n }\n\n const transcript = await response.text()\n this.log(`Received transcript: \"${transcript}\"`)\n\n return transcript\n }\n}\n"],"mappings":";;;;;;;;;;;;;;;;;;;;;;;;;;;;;;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;;;ACAA,oBAAoC;AACpC,oBAAmB;AACnB,oBAA4B;;;ACG5B,iBAAgC;AAazB,SAAS,iBACd,MACc;AACd,SAAO;AAAA,IACL,MAAM;AAAA,IACN,MAAM,KAAK;AAAA,IACX,aAAa,KAAK;AAAA,IAClB,QAAQ;AAAA,IACR,gBAAY,yBAAa,KAAK,cAAc,aAAE,OAAO,CAAC;AAAA,EACxD;AACF;AAEO,SAAS,qBACd,UACA,MACgB;AAChB,SAAO;AAAA,IACL,MAAM;AAAA,IACN,SAAS,SAAS;AAAA,IAClB,QAAQ,KAAK,UAAU,IAAI;AAAA,EAC7B;AACF;;;ADLA,IAAM,gBAAgB;AACtB,IAAM,oBAAoB;AAE1B,IAAM,0BAA0B;AAChC,IAAM,uBAAuB;AAE7B,IAAM,+BAA+B;AACrC,IAAM,4BACJ;AAEF,IAAM,mCAAmC;AACzC,IAAM,gCACJ;AAEK,IAAM,cAAN,cAA0B,oBAA0B;AAAA,EAKzD,YAAY,SAA6B;AACvC,UAAM,OAAO;AACb,SAAK,SAAS,IAAI,cAAAA,QAAO,EAAE,QAAQ,QAAQ,OAAO,CAAC;AACnD,SAAK,QAAQ,KAAK,gBAAgB;AAAA,EACpC;AAAA,EAEA,SAAS;AACP,SAAK,OAAO;AACZ,SAAK,IAAI,iBAAiB;AAC1B,UAAM,SAAS,IAAI,0BAAY;AAE/B,SAAK,eAAe,MAAM,EAAE,QAAQ,MAAM;AACxC,WAAK,kBAAkB;AACvB,UAAI,OAAO,UAAU;AACnB,eAAO,IAAI;AAAA,MACb;AAAA,IACF,CAAC;AAED,WAAO;AAAA,EACT;AAAA,EAEA,MAAc,eACZ,QACA,YAAY,GACZ,YACe;AACf,QAAI,cAAc,KAAK,QAAQ,YAAY,oBAAoB;AAC7D,cAAQ,MAAM,iCAAiC;AAC/C;AAAA,IACF;AAEA,SAAK,kBAAkB,IAAI,gBAAgB;AAC3C,UAAM,SAAS,KAAK,gBAAgB;AAGpC,UAAM,cAAc,KAAK,aAAa,SAAS;AAG/C,UAAM,QAAwC,KAAK,aAAa;AAAA,MAC9D,CAAC,aAAa;AAAA,QACZ,MAAM,QAAQ;AAAA,QACd,SAAS,QAAQ;AAAA,MACnB;AAAA,IACF;AACA,QAAI,YAAY;AACd,YAAM,KAAK,GAAG,UAAU;AAAA,IAC1B;AAEA,QAAI;AAEF,YAAM,WAAW,MAAM,KAAK,OAAO,UAAU;AAAA,QAC3C;AAAA,UACE,OAAO,KAAK,QAAQ,SAAS;AAAA,UAC7B;AAAA,UACA,aAAa;AAAA,UACb,mBAAmB;AAAA,UACnB,QAAQ;AAAA,UACR,OAAO,cAAc,KAAK,MAAM,IAAI,gBAAgB,IAAI;AAAA,UACxD,GAAG,KAAK,QAAQ;AAAA,QAClB;AAAA,QACA,EAAE,OAAO;AAAA,MACX;AACA,YAAM,cAAgD,CAAC;AACvD,UAAI,YAAY;AAChB,UAAI,aAAa;AAGjB,uBAAiB,SAAS,UAAU;AAClC,gBAAQ,MAAM,MAAM;AAAA,UAClB,KAAK;AACH,iBAAK,IAAI,kBAAkB,MAAM,KAAK,GAAG;AACzC,mBAAO,MAAM,MAAM,KAAK;AACxB;AAAA,UAEF,KAAK;AACH,iBAAK,oBAAoB,MAAM,IAAI;AACnC,mBAAO,IAAI;AACX,wBAAY;AACZ;AAAA,UAEF,KAAK;AACH,gBAAI,MAAM,KAAK,SAAS,iBAAiB;AACvC,oBAAM,SAAS,MAAM,KAAK,YAAY,MAAM,IAAI;AAChD,0BAAY,KAAK,OAAO,UAAU,OAAO,cAAc;AACvD,kBAAI,OAAO,YAAY;AACrB,6BAAa;AAAA,cACf;AAAA,YACF;AACA;AAAA,UAEF;AACE;AAAA,QACJ;AAAA,MACF;AAGA,UAAI,YAAY,SAAS,KAAK,CAAC,aAAa,CAAC,YAAY;AACvD,cAAM,KAAK,eAAe,QAAQ,YAAY,GAAG,WAAW;AAAA,MAC9D;AAAA,IACF,SAAS,OAAO;AACd,UAAI,EAAE,iBAAiB,cAAAA,QAAO,oBAAoB;AAChD,gBAAQ,MAAM,kCAAkC,KAAK;AAAA,MACvD;AAAA,IACF;AAAA,EACF;AAAA,EAEA,QAAoC,MAAoB;AACtD,SAAK,MAAM,KAAK,IAAI;AAAA,EACtB;AAAA,EAEA,WAAW,MAAc;AACvB,SAAK,QAAQ,KAAK,MAAM,OAAO,CAAC,SAAS,KAAK,SAAS,IAAI;AAAA,EAC7D;AAAA,EAEA,QAAQ,MAAgC;AACtC,WAAO,KAAK,MAAM,KAAK,CAAC,SAAS,KAAK,SAAS,IAAI;AAAA,EACrD;AAAA,EAEQ,kBAAkB;AACxB,UAAM,QAAgB,CAAC;AACvB,QAAI,KAAK,QAAQ,aAAa;AAC5B,YAAM,KAAK;AAAA,QACT,MAAM;AAAA,QACN,aACE,OAAO,KAAK,QAAQ,gBAAgB,WAChC,KAAK,QAAQ,cACb;AAAA,QACN,UAAU,MAAM,KAAK,QAAQ;AAAA,MAC/B,CAAC;AAAA,IACH;AACA,QAAI,KAAK,QAAQ,kBAAkB;AACjC,YAAM,KAAK;AAAA,QACT,MAAM;AAAA,QACN,aACE,OAAO,KAAK,QAAQ,qBAAqB,WACrC,KAAK,QAAQ,mBACb;AAAA,QACN,YAAY;AAAA,QACZ,UAAU,MAAM,KAAK,WAAW;AAAA,MAClC,CAAC;AAAA,IACH;AACA,QAAI,KAAK,QAAQ,qBAAqB;AACpC,YAAM,KAAK;AAAA,QACT,MAAM;AAAA,QACN,aACE,OAAO,KAAK,QAAQ,wBAAwB,WACxC,KAAK,QAAQ,sBACb;AAAA,QACN,YAAY;AAAA,QACZ,UAAU,MAAM,KAAK,sBAAsB;AAAA,MAC7C,CAAC;AAAA,IACH;AACA,WAAO;AAAA,EACT;AAAA,EAEA,MAAc,YAAY,UAAoB;AAC5C,QAAI;AACF,YAAM,OAAO,KAAK,QAAQ,SAAS,IAAI;AACvC,UAAI,CAAC,MAAM;AACT,cAAM,IAAI,MAAM,mBAAmB,SAAS,IAAI,GAAG;AAAA,MACrD;AAEA,WAAK,IAAI,mBAAmB,SAAS,MAAM,SAAS,SAAS;AAC7D,YAAM,OAAO,KAAK,MAAM,SAAS,SAAS;AAC1C,YAAM,SAAS,MAAM,KAAK,SAAS,IAAI;AAEvC,aAAO;AAAA,QACL;AAAA,QACA,gBAAgB,qBAAqB,UAAU,UAAU,CAAC,CAAC;AAAA,QAC3D,YAAY,KAAK;AAAA,MACnB;AAAA,IACF,SAAS,OAAY;AACnB,cAAQ,MAAM,uCAAuC,KAAK;AAC1D,aAAO;AAAA,QACL;AAAA,QACA,gBAAgB,qBAAqB,UAAU;AAAA,UAC7C,OAAO,MAAM;AAAA,QACf,CAAC;AAAA,MACH;AAAA,IACF;AAAA,EACF;AAAA,EAEA,SAAS;AACP,QAAI,CAAC,KAAK,gBAAiB;AAC3B,SAAK,IAAI,QAAQ;AACjB,SAAK,gBAAgB,MAAM;AAC3B,SAAK,kBAAkB;AAAA,EACzB;AACF;;;AEjPA,IAAAC,iBAAwB;AAcxB,IAAMC,iBAAgB;AAEf,IAAM,YAAN,cAAwB,uBAAQ;AAAA,EACrC,YAAoB,SAA2B;AAC7C,UAAM;AADY;AAAA,EAEpB;AAAA,EAEA,MAAM,eAAe,MAAY;AAE/B,UAAM,WAAW,IAAI,SAAS;AAC9B,aAAS,OAAO,QAAQ,IAAI;AAC5B,aAAS,OAAO,SAAS,KAAK,QAAQ,SAASA,cAAa;AAC5D,aAAS,OAAO,mBAAmB,MAAM;AAEzC,QAAI,KAAK,QAAQ,UAAU;AACzB,eAAS,OAAO,YAAY,KAAK,QAAQ,QAAQ;AAAA,IACnD;AAEA,UAAM,WAAW,MAAM;AAAA,MACrB;AAAA,MACA;AAAA,QACE,QAAQ;AAAA,QACR,SAAS;AAAA,UACP,eAAe,UAAU,KAAK,QAAQ,MAAM;AAAA,QAC9C;AAAA,QACA,MAAM;AAAA,MACR;AAAA,IACF;AAEA,QAAI,CAAC,SAAS,IAAI;AAChB,WAAK,IAAI,8BAA8B,MAAM,SAAS,KAAK,CAAC;AAC5D,YAAM,IAAI,MAAM,+BAA+B,SAAS,UAAU,EAAE;AAAA,IACtE;AAEA,UAAM,aAAa,MAAM,SAAS,KAAK;AACvC,SAAK,IAAI,yBAAyB,UAAU,GAAG;AAE/C,WAAO;AAAA,EACT;AACF;","names":["OpenAI","import_server","DEFAULT_MODEL"]}
package/dist/index.mjs CHANGED
@@ -41,19 +41,17 @@ var OpenaiAgent = class extends Agent {
41
41
  this.cancel();
42
42
  this.log("Start answering");
43
43
  const stream = new PassThrough();
44
- const textPromise = this.createTextPromise();
45
- this.generateAnswer(stream, textPromise).finally(() => {
44
+ this.generateAnswer(stream).finally(() => {
46
45
  this.abortController = void 0;
47
46
  if (stream.writable) {
48
47
  stream.end();
49
48
  }
50
49
  });
51
- return { message: textPromise.promise, stream };
50
+ return stream;
52
51
  }
53
- async generateAnswer(stream, textPromise, callCount = 0, toolInputs) {
52
+ async generateAnswer(stream, callCount = 0, toolInputs) {
54
53
  if (callCount >= (this.options.maxCalls || DEFAULT_MAX_CALLS)) {
55
54
  console.error("[OpenaiAgent] Max calls reached");
56
- textPromise.resolve("");
57
55
  return;
58
56
  }
59
57
  this.abortController = new AbortController();
@@ -93,7 +91,6 @@ var OpenaiAgent = class extends Agent {
93
91
  case "response.output_text.done":
94
92
  this.addAssistantMessage(event.text);
95
93
  stream.end();
96
- textPromise.resolve(event.text);
97
94
  hasAnswer = true;
98
95
  break;
99
96
  case "response.output_item.done":
@@ -110,17 +107,9 @@ var OpenaiAgent = class extends Agent {
110
107
  }
111
108
  }
112
109
  if (toolOutputs.length > 0 && !hasAnswer && !skipAnswer) {
113
- await this.generateAnswer(
114
- stream,
115
- textPromise,
116
- callCount + 1,
117
- toolOutputs
118
- );
119
- } else {
120
- textPromise.resolve("");
110
+ await this.generateAnswer(stream, callCount + 1, toolOutputs);
121
111
  }
122
112
  } catch (error) {
123
- textPromise.resolve("");
124
113
  if (!(error instanceof OpenAI.APIUserAbortError)) {
125
114
  console.error("[OpenaiAgent] Error answering:", error);
126
115
  }
@@ -1 +1 @@
1
- {"version":3,"sources":["../src/OpenaiAgent.ts","../src/tools.ts","../src/OpenaiSTT.ts"],"sourcesContent":["import { Agent, AgentOptions, TextPromise } from '@micdrop/server'\nimport OpenAI from 'openai'\nimport { PassThrough } from 'stream'\nimport { z } from 'zod'\nimport {\n createToolCallOutput,\n createToolSchema,\n Tool,\n ToolCall,\n ToolCallOutput,\n} from './tools'\n\nexport interface OpenaiAgentOptions extends AgentOptions {\n apiKey: string\n model?: string\n maxCalls?: number\n settings?: Omit<\n OpenAI.Responses.ResponseCreateParamsStreaming,\n 'input' | 'model'\n >\n\n // Enable auto ending of the call when user asks to end the call\n // You can provide a custom prompt to use instead of the default one by passing a string\n autoEndCall?: boolean | string\n\n // Enable detection of an incomplete sentence, and skip the answer (assistant waits)\n // You can provide a custom prompt to use instead of the default one by passing a string\n autoSemanticTurn?: boolean | string\n\n // Ignore of the last user message when it's meaningless\n // You can provide a custom prompt to use instead of the default one by passing a string\n autoIgnoreUserNoise?: boolean | string\n}\n\nconst DEFAULT_MODEL = 'gpt-4o'\nconst DEFAULT_MAX_CALLS = 5\n\nconst AUTO_END_CALL_TOOL_NAME = 'end_call'\nconst AUTO_END_CALL_PROMPT = 'Call this tool only if user asks to end the call'\n\nconst AUTO_SEMANTIC_TURN_TOOL_NAME = 'semantic_turn'\nconst AUTO_SEMANTIC_TURN_PROMPT =\n 'Call this tool only if last user message is obviously an incomplete sentence that you need to wait for the end before answering'\n\nconst AUTO_IGNORE_USER_NOISE_TOOL_NAME = 'ignore_user_noise'\nconst AUTO_IGNORE_USER_NOISE_PROMPT =\n 'Call this tool only if last user message is just an interjection or a sound that expresses emotion, hesitation, or reaction (ex: \"Uh\", \"Ahem\", \"Hmm\", \"Ah\") but doesn\\'t carry any clear meaning like agreeing, refusing, or commanding'\n\nexport class OpenaiAgent extends Agent<OpenaiAgentOptions> {\n private openai: OpenAI\n private tools: Tool[]\n private abortController?: AbortController\n\n constructor(options: OpenaiAgentOptions) {\n super(options)\n this.openai = new OpenAI({ apiKey: options.apiKey })\n this.tools = this.getDefaultTools()\n }\n\n answer() {\n this.cancel()\n this.log('Start answering')\n const stream = new PassThrough()\n const textPromise = this.createTextPromise()\n\n this.generateAnswer(stream, textPromise).finally(() => {\n this.abortController = undefined\n if (stream.writable) {\n stream.end()\n }\n })\n\n return { message: textPromise.promise, stream }\n }\n\n private async generateAnswer(\n stream: PassThrough,\n textPromise: TextPromise,\n callCount = 0,\n toolInputs?: Array<ToolCall | ToolCallOutput>\n ): Promise<void> {\n if (callCount >= (this.options.maxCalls || DEFAULT_MAX_CALLS)) {\n console.error('[OpenaiAgent] Max calls reached')\n textPromise.resolve('')\n return\n }\n\n this.abortController = new AbortController()\n const signal = this.abortController.signal\n\n // Disable tools if first message\n const enableTools = this.conversation.length > 1\n\n // Build input\n const input: OpenAI.Responses.ResponseInput = this.conversation.map(\n (message) => ({\n role: message.role,\n content: message.content,\n })\n )\n if (toolInputs) {\n input.push(...toolInputs)\n }\n\n try {\n // Generate answer\n const response = await this.openai.responses.create(\n {\n model: this.options.model || DEFAULT_MODEL,\n input,\n temperature: 0.5,\n max_output_tokens: 250,\n stream: true,\n tools: enableTools ? this.tools.map(createToolSchema) : undefined,\n ...this.options.settings,\n },\n { signal }\n )\n const toolOutputs: Array<ToolCall | ToolCallOutput> = []\n let hasAnswer = false\n let skipAnswer = false\n\n // Handle response events\n for await (const event of response) {\n switch (event.type) {\n case 'response.output_text.delta':\n this.log(`Answer chunk: \"${event.delta}\"`)\n stream.write(event.delta)\n break\n\n case 'response.output_text.done':\n this.addAssistantMessage(event.text)\n stream.end()\n textPromise.resolve(event.text)\n hasAnswer = true\n break\n\n case 'response.output_item.done':\n if (event.item.type === 'function_call') {\n const result = await this.executeTool(event.item)\n toolOutputs.push(result.toolCall, result.toolCallOutput)\n if (result.skipAnswer) {\n skipAnswer = true\n }\n }\n break\n\n default:\n break\n }\n }\n\n // Query again in case of tool call\n if (toolOutputs.length > 0 && !hasAnswer && !skipAnswer) {\n await this.generateAnswer(\n stream,\n textPromise,\n callCount + 1,\n toolOutputs\n )\n } else {\n // Ensure promise is resolved (if skipAnswer is true in a tool)\n textPromise.resolve('')\n }\n } catch (error) {\n textPromise.resolve('')\n if (!(error instanceof OpenAI.APIUserAbortError)) {\n console.error('[OpenaiAgent] Error answering:', error)\n }\n }\n }\n\n addTool<Schema extends z.ZodObject>(tool: Tool<Schema>) {\n this.tools.push(tool)\n }\n\n removeTool(name: string) {\n this.tools = this.tools.filter((tool) => tool.name !== name)\n }\n\n getTool(name: string): Tool | undefined {\n return this.tools.find((tool) => tool.name === name)\n }\n\n private getDefaultTools() {\n const tools: Tool[] = []\n if (this.options.autoEndCall) {\n tools.push({\n name: AUTO_END_CALL_TOOL_NAME,\n description:\n typeof this.options.autoEndCall === 'string'\n ? this.options.autoEndCall\n : AUTO_END_CALL_PROMPT,\n callback: () => this.endCall(),\n })\n }\n if (this.options.autoSemanticTurn) {\n tools.push({\n name: AUTO_SEMANTIC_TURN_TOOL_NAME,\n description:\n typeof this.options.autoSemanticTurn === 'string'\n ? this.options.autoSemanticTurn\n : AUTO_SEMANTIC_TURN_PROMPT,\n skipAnswer: true,\n callback: () => this.skipAnswer(),\n })\n }\n if (this.options.autoIgnoreUserNoise) {\n tools.push({\n name: AUTO_IGNORE_USER_NOISE_TOOL_NAME,\n description:\n typeof this.options.autoIgnoreUserNoise === 'string'\n ? this.options.autoIgnoreUserNoise\n : AUTO_IGNORE_USER_NOISE_PROMPT,\n skipAnswer: true,\n callback: () => this.cancelLastUserMessage(),\n })\n }\n return tools\n }\n\n private async executeTool(toolCall: ToolCall) {\n try {\n const tool = this.getTool(toolCall.name)\n if (!tool) {\n throw new Error(`Tool not found \"${toolCall.name}\"`)\n }\n\n this.log('Executing tool:', toolCall.name, toolCall.arguments)\n const args = JSON.parse(toolCall.arguments)\n const result = await tool.callback(args)\n\n return {\n toolCall,\n toolCallOutput: createToolCallOutput(toolCall, result || {}),\n skipAnswer: tool.skipAnswer,\n }\n } catch (error: any) {\n console.error('[OpenaiAgent] Error executing tool:', error)\n return {\n toolCall,\n toolCallOutput: createToolCallOutput(toolCall, {\n error: error.message,\n }),\n }\n }\n }\n\n cancel() {\n if (!this.abortController) return\n this.log('Cancel')\n this.abortController.abort()\n this.abortController = undefined\n }\n}\n","import {\n FunctionTool,\n ResponseInputItem,\n Responses,\n} from 'openai/resources/responses/responses'\nimport { toJSONSchema, z } from 'zod'\n\nexport interface Tool<Schema extends z.ZodObject = z.ZodObject> {\n name: string\n description: string\n parameters?: Schema\n callback: (input: z.infer<Schema>) => any | Promise<any>\n skipAnswer?: boolean\n}\n\nexport type ToolCall = Responses.ResponseFunctionToolCall\nexport type ToolCallOutput = ResponseInputItem.FunctionCallOutput\n\nexport function createToolSchema<Schema extends z.ZodObject>(\n tool: Tool<Schema>\n): FunctionTool {\n return {\n type: 'function',\n name: tool.name,\n description: tool.description,\n strict: true,\n parameters: toJSONSchema(tool.parameters || z.object()),\n }\n}\n\nexport function createToolCallOutput(\n toolCall: ToolCall,\n args: any\n): ToolCallOutput {\n return {\n type: 'function_call_output',\n call_id: toolCall.call_id,\n output: JSON.stringify(args),\n }\n}\n","import { FileSTT } from '@micdrop/server'\n\n/**\n * OpenAI Whisper STT\n *\n * @see https://platform.openai.com/docs/guides/speech-to-text?lang=curl\n */\n\nexport interface OpenaiSTTOptions {\n apiKey: string\n model?: string\n language?: string\n}\n\nconst DEFAULT_MODEL = 'whisper-1'\n\nexport class OpenaiSTT extends FileSTT {\n constructor(private options: OpenaiSTTOptions) {\n super()\n }\n\n async transcribeFile(file: File) {\n // Prepare form data\n const formData = new FormData()\n formData.append('file', file)\n formData.append('model', this.options.model || DEFAULT_MODEL)\n formData.append('response_format', 'text')\n\n if (this.options.language) {\n formData.append('language', this.options.language)\n }\n\n const response = await fetch(\n 'https://api.openai.com/v1/audio/transcriptions',\n {\n method: 'POST',\n headers: {\n Authorization: `Bearer ${this.options.apiKey}`,\n },\n body: formData,\n }\n )\n\n if (!response.ok) {\n this.log('Failed to transcribe audio', await response.json())\n throw new Error(`Failed to transcribe audio: ${response.statusText}`)\n }\n\n const transcript = await response.text()\n this.log(`Received transcript: \"${transcript}\"`)\n\n return transcript\n }\n}\n"],"mappings":";AAAA,SAAS,aAAwC;AACjD,OAAO,YAAY;AACnB,SAAS,mBAAmB;;;ACG5B,SAAS,cAAc,SAAS;AAazB,SAAS,iBACd,MACc;AACd,SAAO;AAAA,IACL,MAAM;AAAA,IACN,MAAM,KAAK;AAAA,IACX,aAAa,KAAK;AAAA,IAClB,QAAQ;AAAA,IACR,YAAY,aAAa,KAAK,cAAc,EAAE,OAAO,CAAC;AAAA,EACxD;AACF;AAEO,SAAS,qBACd,UACA,MACgB;AAChB,SAAO;AAAA,IACL,MAAM;AAAA,IACN,SAAS,SAAS;AAAA,IAClB,QAAQ,KAAK,UAAU,IAAI;AAAA,EAC7B;AACF;;;ADLA,IAAM,gBAAgB;AACtB,IAAM,oBAAoB;AAE1B,IAAM,0BAA0B;AAChC,IAAM,uBAAuB;AAE7B,IAAM,+BAA+B;AACrC,IAAM,4BACJ;AAEF,IAAM,mCAAmC;AACzC,IAAM,gCACJ;AAEK,IAAM,cAAN,cAA0B,MAA0B;AAAA,EAKzD,YAAY,SAA6B;AACvC,UAAM,OAAO;AACb,SAAK,SAAS,IAAI,OAAO,EAAE,QAAQ,QAAQ,OAAO,CAAC;AACnD,SAAK,QAAQ,KAAK,gBAAgB;AAAA,EACpC;AAAA,EAEA,SAAS;AACP,SAAK,OAAO;AACZ,SAAK,IAAI,iBAAiB;AAC1B,UAAM,SAAS,IAAI,YAAY;AAC/B,UAAM,cAAc,KAAK,kBAAkB;AAE3C,SAAK,eAAe,QAAQ,WAAW,EAAE,QAAQ,MAAM;AACrD,WAAK,kBAAkB;AACvB,UAAI,OAAO,UAAU;AACnB,eAAO,IAAI;AAAA,MACb;AAAA,IACF,CAAC;AAED,WAAO,EAAE,SAAS,YAAY,SAAS,OAAO;AAAA,EAChD;AAAA,EAEA,MAAc,eACZ,QACA,aACA,YAAY,GACZ,YACe;AACf,QAAI,cAAc,KAAK,QAAQ,YAAY,oBAAoB;AAC7D,cAAQ,MAAM,iCAAiC;AAC/C,kBAAY,QAAQ,EAAE;AACtB;AAAA,IACF;AAEA,SAAK,kBAAkB,IAAI,gBAAgB;AAC3C,UAAM,SAAS,KAAK,gBAAgB;AAGpC,UAAM,cAAc,KAAK,aAAa,SAAS;AAG/C,UAAM,QAAwC,KAAK,aAAa;AAAA,MAC9D,CAAC,aAAa;AAAA,QACZ,MAAM,QAAQ;AAAA,QACd,SAAS,QAAQ;AAAA,MACnB;AAAA,IACF;AACA,QAAI,YAAY;AACd,YAAM,KAAK,GAAG,UAAU;AAAA,IAC1B;AAEA,QAAI;AAEF,YAAM,WAAW,MAAM,KAAK,OAAO,UAAU;AAAA,QAC3C;AAAA,UACE,OAAO,KAAK,QAAQ,SAAS;AAAA,UAC7B;AAAA,UACA,aAAa;AAAA,UACb,mBAAmB;AAAA,UACnB,QAAQ;AAAA,UACR,OAAO,cAAc,KAAK,MAAM,IAAI,gBAAgB,IAAI;AAAA,UACxD,GAAG,KAAK,QAAQ;AAAA,QAClB;AAAA,QACA,EAAE,OAAO;AAAA,MACX;AACA,YAAM,cAAgD,CAAC;AACvD,UAAI,YAAY;AAChB,UAAI,aAAa;AAGjB,uBAAiB,SAAS,UAAU;AAClC,gBAAQ,MAAM,MAAM;AAAA,UAClB,KAAK;AACH,iBAAK,IAAI,kBAAkB,MAAM,KAAK,GAAG;AACzC,mBAAO,MAAM,MAAM,KAAK;AACxB;AAAA,UAEF,KAAK;AACH,iBAAK,oBAAoB,MAAM,IAAI;AACnC,mBAAO,IAAI;AACX,wBAAY,QAAQ,MAAM,IAAI;AAC9B,wBAAY;AACZ;AAAA,UAEF,KAAK;AACH,gBAAI,MAAM,KAAK,SAAS,iBAAiB;AACvC,oBAAM,SAAS,MAAM,KAAK,YAAY,MAAM,IAAI;AAChD,0BAAY,KAAK,OAAO,UAAU,OAAO,cAAc;AACvD,kBAAI,OAAO,YAAY;AACrB,6BAAa;AAAA,cACf;AAAA,YACF;AACA;AAAA,UAEF;AACE;AAAA,QACJ;AAAA,MACF;AAGA,UAAI,YAAY,SAAS,KAAK,CAAC,aAAa,CAAC,YAAY;AACvD,cAAM,KAAK;AAAA,UACT;AAAA,UACA;AAAA,UACA,YAAY;AAAA,UACZ;AAAA,QACF;AAAA,MACF,OAAO;AAEL,oBAAY,QAAQ,EAAE;AAAA,MACxB;AAAA,IACF,SAAS,OAAO;AACd,kBAAY,QAAQ,EAAE;AACtB,UAAI,EAAE,iBAAiB,OAAO,oBAAoB;AAChD,gBAAQ,MAAM,kCAAkC,KAAK;AAAA,MACvD;AAAA,IACF;AAAA,EACF;AAAA,EAEA,QAAoC,MAAoB;AACtD,SAAK,MAAM,KAAK,IAAI;AAAA,EACtB;AAAA,EAEA,WAAW,MAAc;AACvB,SAAK,QAAQ,KAAK,MAAM,OAAO,CAAC,SAAS,KAAK,SAAS,IAAI;AAAA,EAC7D;AAAA,EAEA,QAAQ,MAAgC;AACtC,WAAO,KAAK,MAAM,KAAK,CAAC,SAAS,KAAK,SAAS,IAAI;AAAA,EACrD;AAAA,EAEQ,kBAAkB;AACxB,UAAM,QAAgB,CAAC;AACvB,QAAI,KAAK,QAAQ,aAAa;AAC5B,YAAM,KAAK;AAAA,QACT,MAAM;AAAA,QACN,aACE,OAAO,KAAK,QAAQ,gBAAgB,WAChC,KAAK,QAAQ,cACb;AAAA,QACN,UAAU,MAAM,KAAK,QAAQ;AAAA,MAC/B,CAAC;AAAA,IACH;AACA,QAAI,KAAK,QAAQ,kBAAkB;AACjC,YAAM,KAAK;AAAA,QACT,MAAM;AAAA,QACN,aACE,OAAO,KAAK,QAAQ,qBAAqB,WACrC,KAAK,QAAQ,mBACb;AAAA,QACN,YAAY;AAAA,QACZ,UAAU,MAAM,KAAK,WAAW;AAAA,MAClC,CAAC;AAAA,IACH;AACA,QAAI,KAAK,QAAQ,qBAAqB;AACpC,YAAM,KAAK;AAAA,QACT,MAAM;AAAA,QACN,aACE,OAAO,KAAK,QAAQ,wBAAwB,WACxC,KAAK,QAAQ,sBACb;AAAA,QACN,YAAY;AAAA,QACZ,UAAU,MAAM,KAAK,sBAAsB;AAAA,MAC7C,CAAC;AAAA,IACH;AACA,WAAO;AAAA,EACT;AAAA,EAEA,MAAc,YAAY,UAAoB;AAC5C,QAAI;AACF,YAAM,OAAO,KAAK,QAAQ,SAAS,IAAI;AACvC,UAAI,CAAC,MAAM;AACT,cAAM,IAAI,MAAM,mBAAmB,SAAS,IAAI,GAAG;AAAA,MACrD;AAEA,WAAK,IAAI,mBAAmB,SAAS,MAAM,SAAS,SAAS;AAC7D,YAAM,OAAO,KAAK,MAAM,SAAS,SAAS;AAC1C,YAAM,SAAS,MAAM,KAAK,SAAS,IAAI;AAEvC,aAAO;AAAA,QACL;AAAA,QACA,gBAAgB,qBAAqB,UAAU,UAAU,CAAC,CAAC;AAAA,QAC3D,YAAY,KAAK;AAAA,MACnB;AAAA,IACF,SAAS,OAAY;AACnB,cAAQ,MAAM,uCAAuC,KAAK;AAC1D,aAAO;AAAA,QACL;AAAA,QACA,gBAAgB,qBAAqB,UAAU;AAAA,UAC7C,OAAO,MAAM;AAAA,QACf,CAAC;AAAA,MACH;AAAA,IACF;AAAA,EACF;AAAA,EAEA,SAAS;AACP,QAAI,CAAC,KAAK,gBAAiB;AAC3B,SAAK,IAAI,QAAQ;AACjB,SAAK,gBAAgB,MAAM;AAC3B,SAAK,kBAAkB;AAAA,EACzB;AACF;;;AE9PA,SAAS,eAAe;AAcxB,IAAMA,iBAAgB;AAEf,IAAM,YAAN,cAAwB,QAAQ;AAAA,EACrC,YAAoB,SAA2B;AAC7C,UAAM;AADY;AAAA,EAEpB;AAAA,EAEA,MAAM,eAAe,MAAY;AAE/B,UAAM,WAAW,IAAI,SAAS;AAC9B,aAAS,OAAO,QAAQ,IAAI;AAC5B,aAAS,OAAO,SAAS,KAAK,QAAQ,SAASA,cAAa;AAC5D,aAAS,OAAO,mBAAmB,MAAM;AAEzC,QAAI,KAAK,QAAQ,UAAU;AACzB,eAAS,OAAO,YAAY,KAAK,QAAQ,QAAQ;AAAA,IACnD;AAEA,UAAM,WAAW,MAAM;AAAA,MACrB;AAAA,MACA;AAAA,QACE,QAAQ;AAAA,QACR,SAAS;AAAA,UACP,eAAe,UAAU,KAAK,QAAQ,MAAM;AAAA,QAC9C;AAAA,QACA,MAAM;AAAA,MACR;AAAA,IACF;AAEA,QAAI,CAAC,SAAS,IAAI;AAChB,WAAK,IAAI,8BAA8B,MAAM,SAAS,KAAK,CAAC;AAC5D,YAAM,IAAI,MAAM,+BAA+B,SAAS,UAAU,EAAE;AAAA,IACtE;AAEA,UAAM,aAAa,MAAM,SAAS,KAAK;AACvC,SAAK,IAAI,yBAAyB,UAAU,GAAG;AAE/C,WAAO;AAAA,EACT;AACF;","names":["DEFAULT_MODEL"]}
1
+ {"version":3,"sources":["../src/OpenaiAgent.ts","../src/tools.ts","../src/OpenaiSTT.ts"],"sourcesContent":["import { Agent, AgentOptions } from '@micdrop/server'\nimport OpenAI from 'openai'\nimport { PassThrough } from 'stream'\nimport { z } from 'zod'\nimport {\n createToolCallOutput,\n createToolSchema,\n Tool,\n ToolCall,\n ToolCallOutput,\n} from './tools'\n\nexport interface OpenaiAgentOptions extends AgentOptions {\n apiKey: string\n model?: string\n maxCalls?: number\n settings?: Omit<\n OpenAI.Responses.ResponseCreateParamsStreaming,\n 'input' | 'model'\n >\n\n // Enable auto ending of the call when user asks to end the call\n // You can provide a custom prompt to use instead of the default one by passing a string\n autoEndCall?: boolean | string\n\n // Enable detection of an incomplete sentence, and skip the answer (assistant waits)\n // You can provide a custom prompt to use instead of the default one by passing a string\n autoSemanticTurn?: boolean | string\n\n // Ignore of the last user message when it's meaningless\n // You can provide a custom prompt to use instead of the default one by passing a string\n autoIgnoreUserNoise?: boolean | string\n}\n\nconst DEFAULT_MODEL = 'gpt-4o'\nconst DEFAULT_MAX_CALLS = 5\n\nconst AUTO_END_CALL_TOOL_NAME = 'end_call'\nconst AUTO_END_CALL_PROMPT = 'Call this tool only if user asks to end the call'\n\nconst AUTO_SEMANTIC_TURN_TOOL_NAME = 'semantic_turn'\nconst AUTO_SEMANTIC_TURN_PROMPT =\n 'Call this tool only if last user message is obviously an incomplete sentence that you need to wait for the end before answering'\n\nconst AUTO_IGNORE_USER_NOISE_TOOL_NAME = 'ignore_user_noise'\nconst AUTO_IGNORE_USER_NOISE_PROMPT =\n 'Call this tool only if last user message is just an interjection or a sound that expresses emotion, hesitation, or reaction (ex: \"Uh\", \"Ahem\", \"Hmm\", \"Ah\") but doesn\\'t carry any clear meaning like agreeing, refusing, or commanding'\n\nexport class OpenaiAgent extends Agent<OpenaiAgentOptions> {\n private openai: OpenAI\n private tools: Tool[]\n private abortController?: AbortController\n\n constructor(options: OpenaiAgentOptions) {\n super(options)\n this.openai = new OpenAI({ apiKey: options.apiKey })\n this.tools = this.getDefaultTools()\n }\n\n answer() {\n this.cancel()\n this.log('Start answering')\n const stream = new PassThrough()\n\n this.generateAnswer(stream).finally(() => {\n this.abortController = undefined\n if (stream.writable) {\n stream.end()\n }\n })\n\n return stream\n }\n\n private async generateAnswer(\n stream: PassThrough,\n callCount = 0,\n toolInputs?: Array<ToolCall | ToolCallOutput>\n ): Promise<void> {\n if (callCount >= (this.options.maxCalls || DEFAULT_MAX_CALLS)) {\n console.error('[OpenaiAgent] Max calls reached')\n return\n }\n\n this.abortController = new AbortController()\n const signal = this.abortController.signal\n\n // Disable tools if first message\n const enableTools = this.conversation.length > 1\n\n // Build input\n const input: OpenAI.Responses.ResponseInput = this.conversation.map(\n (message) => ({\n role: message.role,\n content: message.content,\n })\n )\n if (toolInputs) {\n input.push(...toolInputs)\n }\n\n try {\n // Generate answer\n const response = await this.openai.responses.create(\n {\n model: this.options.model || DEFAULT_MODEL,\n input,\n temperature: 0.5,\n max_output_tokens: 250,\n stream: true,\n tools: enableTools ? this.tools.map(createToolSchema) : undefined,\n ...this.options.settings,\n },\n { signal }\n )\n const toolOutputs: Array<ToolCall | ToolCallOutput> = []\n let hasAnswer = false\n let skipAnswer = false\n\n // Handle response events\n for await (const event of response) {\n switch (event.type) {\n case 'response.output_text.delta':\n this.log(`Answer chunk: \"${event.delta}\"`)\n stream.write(event.delta)\n break\n\n case 'response.output_text.done':\n this.addAssistantMessage(event.text)\n stream.end()\n hasAnswer = true\n break\n\n case 'response.output_item.done':\n if (event.item.type === 'function_call') {\n const result = await this.executeTool(event.item)\n toolOutputs.push(result.toolCall, result.toolCallOutput)\n if (result.skipAnswer) {\n skipAnswer = true\n }\n }\n break\n\n default:\n break\n }\n }\n\n // Query again in case of tool call\n if (toolOutputs.length > 0 && !hasAnswer && !skipAnswer) {\n await this.generateAnswer(stream, callCount + 1, toolOutputs)\n }\n } catch (error) {\n if (!(error instanceof OpenAI.APIUserAbortError)) {\n console.error('[OpenaiAgent] Error answering:', error)\n }\n }\n }\n\n addTool<Schema extends z.ZodObject>(tool: Tool<Schema>) {\n this.tools.push(tool)\n }\n\n removeTool(name: string) {\n this.tools = this.tools.filter((tool) => tool.name !== name)\n }\n\n getTool(name: string): Tool | undefined {\n return this.tools.find((tool) => tool.name === name)\n }\n\n private getDefaultTools() {\n const tools: Tool[] = []\n if (this.options.autoEndCall) {\n tools.push({\n name: AUTO_END_CALL_TOOL_NAME,\n description:\n typeof this.options.autoEndCall === 'string'\n ? this.options.autoEndCall\n : AUTO_END_CALL_PROMPT,\n callback: () => this.endCall(),\n })\n }\n if (this.options.autoSemanticTurn) {\n tools.push({\n name: AUTO_SEMANTIC_TURN_TOOL_NAME,\n description:\n typeof this.options.autoSemanticTurn === 'string'\n ? this.options.autoSemanticTurn\n : AUTO_SEMANTIC_TURN_PROMPT,\n skipAnswer: true,\n callback: () => this.skipAnswer(),\n })\n }\n if (this.options.autoIgnoreUserNoise) {\n tools.push({\n name: AUTO_IGNORE_USER_NOISE_TOOL_NAME,\n description:\n typeof this.options.autoIgnoreUserNoise === 'string'\n ? this.options.autoIgnoreUserNoise\n : AUTO_IGNORE_USER_NOISE_PROMPT,\n skipAnswer: true,\n callback: () => this.cancelLastUserMessage(),\n })\n }\n return tools\n }\n\n private async executeTool(toolCall: ToolCall) {\n try {\n const tool = this.getTool(toolCall.name)\n if (!tool) {\n throw new Error(`Tool not found \"${toolCall.name}\"`)\n }\n\n this.log('Executing tool:', toolCall.name, toolCall.arguments)\n const args = JSON.parse(toolCall.arguments)\n const result = await tool.callback(args)\n\n return {\n toolCall,\n toolCallOutput: createToolCallOutput(toolCall, result || {}),\n skipAnswer: tool.skipAnswer,\n }\n } catch (error: any) {\n console.error('[OpenaiAgent] Error executing tool:', error)\n return {\n toolCall,\n toolCallOutput: createToolCallOutput(toolCall, {\n error: error.message,\n }),\n }\n }\n }\n\n cancel() {\n if (!this.abortController) return\n this.log('Cancel')\n this.abortController.abort()\n this.abortController = undefined\n }\n}\n","import {\n FunctionTool,\n ResponseInputItem,\n Responses,\n} from 'openai/resources/responses/responses'\nimport { toJSONSchema, z } from 'zod'\n\nexport interface Tool<Schema extends z.ZodObject = z.ZodObject> {\n name: string\n description: string\n parameters?: Schema\n callback: (input: z.infer<Schema>) => any | Promise<any>\n skipAnswer?: boolean\n}\n\nexport type ToolCall = Responses.ResponseFunctionToolCall\nexport type ToolCallOutput = ResponseInputItem.FunctionCallOutput\n\nexport function createToolSchema<Schema extends z.ZodObject>(\n tool: Tool<Schema>\n): FunctionTool {\n return {\n type: 'function',\n name: tool.name,\n description: tool.description,\n strict: true,\n parameters: toJSONSchema(tool.parameters || z.object()),\n }\n}\n\nexport function createToolCallOutput(\n toolCall: ToolCall,\n args: any\n): ToolCallOutput {\n return {\n type: 'function_call_output',\n call_id: toolCall.call_id,\n output: JSON.stringify(args),\n }\n}\n","import { FileSTT } from '@micdrop/server'\n\n/**\n * OpenAI Whisper STT\n *\n * @see https://platform.openai.com/docs/guides/speech-to-text?lang=curl\n */\n\nexport interface OpenaiSTTOptions {\n apiKey: string\n model?: string\n language?: string\n}\n\nconst DEFAULT_MODEL = 'whisper-1'\n\nexport class OpenaiSTT extends FileSTT {\n constructor(private options: OpenaiSTTOptions) {\n super()\n }\n\n async transcribeFile(file: File) {\n // Prepare form data\n const formData = new FormData()\n formData.append('file', file)\n formData.append('model', this.options.model || DEFAULT_MODEL)\n formData.append('response_format', 'text')\n\n if (this.options.language) {\n formData.append('language', this.options.language)\n }\n\n const response = await fetch(\n 'https://api.openai.com/v1/audio/transcriptions',\n {\n method: 'POST',\n headers: {\n Authorization: `Bearer ${this.options.apiKey}`,\n },\n body: formData,\n }\n )\n\n if (!response.ok) {\n this.log('Failed to transcribe audio', await response.json())\n throw new Error(`Failed to transcribe audio: ${response.statusText}`)\n }\n\n const transcript = await response.text()\n this.log(`Received transcript: \"${transcript}\"`)\n\n return transcript\n }\n}\n"],"mappings":";AAAA,SAAS,aAA2B;AACpC,OAAO,YAAY;AACnB,SAAS,mBAAmB;;;ACG5B,SAAS,cAAc,SAAS;AAazB,SAAS,iBACd,MACc;AACd,SAAO;AAAA,IACL,MAAM;AAAA,IACN,MAAM,KAAK;AAAA,IACX,aAAa,KAAK;AAAA,IAClB,QAAQ;AAAA,IACR,YAAY,aAAa,KAAK,cAAc,EAAE,OAAO,CAAC;AAAA,EACxD;AACF;AAEO,SAAS,qBACd,UACA,MACgB;AAChB,SAAO;AAAA,IACL,MAAM;AAAA,IACN,SAAS,SAAS;AAAA,IAClB,QAAQ,KAAK,UAAU,IAAI;AAAA,EAC7B;AACF;;;ADLA,IAAM,gBAAgB;AACtB,IAAM,oBAAoB;AAE1B,IAAM,0BAA0B;AAChC,IAAM,uBAAuB;AAE7B,IAAM,+BAA+B;AACrC,IAAM,4BACJ;AAEF,IAAM,mCAAmC;AACzC,IAAM,gCACJ;AAEK,IAAM,cAAN,cAA0B,MAA0B;AAAA,EAKzD,YAAY,SAA6B;AACvC,UAAM,OAAO;AACb,SAAK,SAAS,IAAI,OAAO,EAAE,QAAQ,QAAQ,OAAO,CAAC;AACnD,SAAK,QAAQ,KAAK,gBAAgB;AAAA,EACpC;AAAA,EAEA,SAAS;AACP,SAAK,OAAO;AACZ,SAAK,IAAI,iBAAiB;AAC1B,UAAM,SAAS,IAAI,YAAY;AAE/B,SAAK,eAAe,MAAM,EAAE,QAAQ,MAAM;AACxC,WAAK,kBAAkB;AACvB,UAAI,OAAO,UAAU;AACnB,eAAO,IAAI;AAAA,MACb;AAAA,IACF,CAAC;AAED,WAAO;AAAA,EACT;AAAA,EAEA,MAAc,eACZ,QACA,YAAY,GACZ,YACe;AACf,QAAI,cAAc,KAAK,QAAQ,YAAY,oBAAoB;AAC7D,cAAQ,MAAM,iCAAiC;AAC/C;AAAA,IACF;AAEA,SAAK,kBAAkB,IAAI,gBAAgB;AAC3C,UAAM,SAAS,KAAK,gBAAgB;AAGpC,UAAM,cAAc,KAAK,aAAa,SAAS;AAG/C,UAAM,QAAwC,KAAK,aAAa;AAAA,MAC9D,CAAC,aAAa;AAAA,QACZ,MAAM,QAAQ;AAAA,QACd,SAAS,QAAQ;AAAA,MACnB;AAAA,IACF;AACA,QAAI,YAAY;AACd,YAAM,KAAK,GAAG,UAAU;AAAA,IAC1B;AAEA,QAAI;AAEF,YAAM,WAAW,MAAM,KAAK,OAAO,UAAU;AAAA,QAC3C;AAAA,UACE,OAAO,KAAK,QAAQ,SAAS;AAAA,UAC7B;AAAA,UACA,aAAa;AAAA,UACb,mBAAmB;AAAA,UACnB,QAAQ;AAAA,UACR,OAAO,cAAc,KAAK,MAAM,IAAI,gBAAgB,IAAI;AAAA,UACxD,GAAG,KAAK,QAAQ;AAAA,QAClB;AAAA,QACA,EAAE,OAAO;AAAA,MACX;AACA,YAAM,cAAgD,CAAC;AACvD,UAAI,YAAY;AAChB,UAAI,aAAa;AAGjB,uBAAiB,SAAS,UAAU;AAClC,gBAAQ,MAAM,MAAM;AAAA,UAClB,KAAK;AACH,iBAAK,IAAI,kBAAkB,MAAM,KAAK,GAAG;AACzC,mBAAO,MAAM,MAAM,KAAK;AACxB;AAAA,UAEF,KAAK;AACH,iBAAK,oBAAoB,MAAM,IAAI;AACnC,mBAAO,IAAI;AACX,wBAAY;AACZ;AAAA,UAEF,KAAK;AACH,gBAAI,MAAM,KAAK,SAAS,iBAAiB;AACvC,oBAAM,SAAS,MAAM,KAAK,YAAY,MAAM,IAAI;AAChD,0BAAY,KAAK,OAAO,UAAU,OAAO,cAAc;AACvD,kBAAI,OAAO,YAAY;AACrB,6BAAa;AAAA,cACf;AAAA,YACF;AACA;AAAA,UAEF;AACE;AAAA,QACJ;AAAA,MACF;AAGA,UAAI,YAAY,SAAS,KAAK,CAAC,aAAa,CAAC,YAAY;AACvD,cAAM,KAAK,eAAe,QAAQ,YAAY,GAAG,WAAW;AAAA,MAC9D;AAAA,IACF,SAAS,OAAO;AACd,UAAI,EAAE,iBAAiB,OAAO,oBAAoB;AAChD,gBAAQ,MAAM,kCAAkC,KAAK;AAAA,MACvD;AAAA,IACF;AAAA,EACF;AAAA,EAEA,QAAoC,MAAoB;AACtD,SAAK,MAAM,KAAK,IAAI;AAAA,EACtB;AAAA,EAEA,WAAW,MAAc;AACvB,SAAK,QAAQ,KAAK,MAAM,OAAO,CAAC,SAAS,KAAK,SAAS,IAAI;AAAA,EAC7D;AAAA,EAEA,QAAQ,MAAgC;AACtC,WAAO,KAAK,MAAM,KAAK,CAAC,SAAS,KAAK,SAAS,IAAI;AAAA,EACrD;AAAA,EAEQ,kBAAkB;AACxB,UAAM,QAAgB,CAAC;AACvB,QAAI,KAAK,QAAQ,aAAa;AAC5B,YAAM,KAAK;AAAA,QACT,MAAM;AAAA,QACN,aACE,OAAO,KAAK,QAAQ,gBAAgB,WAChC,KAAK,QAAQ,cACb;AAAA,QACN,UAAU,MAAM,KAAK,QAAQ;AAAA,MAC/B,CAAC;AAAA,IACH;AACA,QAAI,KAAK,QAAQ,kBAAkB;AACjC,YAAM,KAAK;AAAA,QACT,MAAM;AAAA,QACN,aACE,OAAO,KAAK,QAAQ,qBAAqB,WACrC,KAAK,QAAQ,mBACb;AAAA,QACN,YAAY;AAAA,QACZ,UAAU,MAAM,KAAK,WAAW;AAAA,MAClC,CAAC;AAAA,IACH;AACA,QAAI,KAAK,QAAQ,qBAAqB;AACpC,YAAM,KAAK;AAAA,QACT,MAAM;AAAA,QACN,aACE,OAAO,KAAK,QAAQ,wBAAwB,WACxC,KAAK,QAAQ,sBACb;AAAA,QACN,YAAY;AAAA,QACZ,UAAU,MAAM,KAAK,sBAAsB;AAAA,MAC7C,CAAC;AAAA,IACH;AACA,WAAO;AAAA,EACT;AAAA,EAEA,MAAc,YAAY,UAAoB;AAC5C,QAAI;AACF,YAAM,OAAO,KAAK,QAAQ,SAAS,IAAI;AACvC,UAAI,CAAC,MAAM;AACT,cAAM,IAAI,MAAM,mBAAmB,SAAS,IAAI,GAAG;AAAA,MACrD;AAEA,WAAK,IAAI,mBAAmB,SAAS,MAAM,SAAS,SAAS;AAC7D,YAAM,OAAO,KAAK,MAAM,SAAS,SAAS;AAC1C,YAAM,SAAS,MAAM,KAAK,SAAS,IAAI;AAEvC,aAAO;AAAA,QACL;AAAA,QACA,gBAAgB,qBAAqB,UAAU,UAAU,CAAC,CAAC;AAAA,QAC3D,YAAY,KAAK;AAAA,MACnB;AAAA,IACF,SAAS,OAAY;AACnB,cAAQ,MAAM,uCAAuC,KAAK;AAC1D,aAAO;AAAA,QACL;AAAA,QACA,gBAAgB,qBAAqB,UAAU;AAAA,UAC7C,OAAO,MAAM;AAAA,QACf,CAAC;AAAA,MACH;AAAA,IACF;AAAA,EACF;AAAA,EAEA,SAAS;AACP,QAAI,CAAC,KAAK,gBAAiB;AAC3B,SAAK,IAAI,QAAQ;AACjB,SAAK,gBAAgB,MAAM;AAC3B,SAAK,kBAAkB;AAAA,EACzB;AACF;;;AEjPA,SAAS,eAAe;AAcxB,IAAMA,iBAAgB;AAEf,IAAM,YAAN,cAAwB,QAAQ;AAAA,EACrC,YAAoB,SAA2B;AAC7C,UAAM;AADY;AAAA,EAEpB;AAAA,EAEA,MAAM,eAAe,MAAY;AAE/B,UAAM,WAAW,IAAI,SAAS;AAC9B,aAAS,OAAO,QAAQ,IAAI;AAC5B,aAAS,OAAO,SAAS,KAAK,QAAQ,SAASA,cAAa;AAC5D,aAAS,OAAO,mBAAmB,MAAM;AAEzC,QAAI,KAAK,QAAQ,UAAU;AACzB,eAAS,OAAO,YAAY,KAAK,QAAQ,QAAQ;AAAA,IACnD;AAEA,UAAM,WAAW,MAAM;AAAA,MACrB;AAAA,MACA;AAAA,QACE,QAAQ;AAAA,QACR,SAAS;AAAA,UACP,eAAe,UAAU,KAAK,QAAQ,MAAM;AAAA,QAC9C;AAAA,QACA,MAAM;AAAA,MACR;AAAA,IACF;AAEA,QAAI,CAAC,SAAS,IAAI;AAChB,WAAK,IAAI,8BAA8B,MAAM,SAAS,KAAK,CAAC;AAC5D,YAAM,IAAI,MAAM,+BAA+B,SAAS,UAAU,EAAE;AAAA,IACtE;AAEA,UAAM,aAAa,MAAM,SAAS,KAAK;AACvC,SAAK,IAAI,yBAAyB,UAAU,GAAG;AAE/C,WAAO;AAAA,EACT;AACF;","names":["DEFAULT_MODEL"]}
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@micdrop/openai",
3
- "version": "1.0.1",
3
+ "version": "1.0.2",
4
4
  "description": "OpenAI implementation for @micdrop/server",
5
5
  "author": "Lonestone",
6
6
  "license": "MIT",