@cogitator-ai/openai-compat 0.1.0 → 2.0.0

package/README.md

@@ -1,6 +1,6 @@
 # @cogitator-ai/openai-compat
 
-OpenAI Assistants API compatibility layer for Cogitator. Use OpenAI SDK clients with Cogitator backend.
+OpenAI Assistants API compatibility layer for Cogitator. Use OpenAI SDK clients with Cogitator backend, or integrate Cogitator with existing OpenAI-based applications.
 
 ## Installation
 
@@ -8,70 +8,733 @@ OpenAI Assistants API compatibility layer for Cogitator. Use OpenAI SDK clients
 pnpm add @cogitator-ai/openai-compat
 ```
 
-## Usage
+## Features
 
-### Start the Server
+- **OpenAI Server** - Expose Cogitator as OpenAI-compatible REST API
+- **OpenAI Adapter** - In-process adapter for programmatic access
+- **Thread Manager** - Manage conversations, messages, and assistants
+- **File Operations** - Upload and manage files for assistants
+- **Full Assistants API** - Create, update, delete assistants
+- **Run Management** - Execute runs with tool support
+- **Authentication** - Optional API key authentication
+- **CORS Support** - Configurable cross-origin requests
+
+---
+
+## Quick Start
+
+### Server Mode
 
 ```typescript
 import { createOpenAIServer } from '@cogitator-ai/openai-compat';
-import { Cogitator } from '@cogitator-ai/core';
+import { Cogitator, tool } from '@cogitator-ai/core';
+import { z } from 'zod';
+
+const calculator = tool({
+  name: 'calculator',
+  description: 'Perform calculations',
+  parameters: z.object({
+    expression: z.string(),
+  }),
+  execute: async ({ expression }) => eval(expression).toString(),
+});
+
+const cogitator = new Cogitator({
+  defaultModel: 'openai/gpt-4o-mini',
+});
 
-const cogitator = new Cogitator();
 const server = createOpenAIServer(cogitator, {
-  port: 3001,
-  apiKey: 'your-api-key', // Optional auth
+  port: 8080,
+  tools: [calculator],
+  apiKeys: ['sk-my-secret-key'],
 });
 
 await server.start();
+// Server is now available at http://localhost:8080
 ```
 
-### Use with OpenAI SDK
+### Client Mode
 
 ```typescript
 import OpenAI from 'openai';
 
 const openai = new OpenAI({
-  baseURL: 'http://localhost:3001/v1',
-  apiKey: 'your-api-key',
+  baseURL: 'http://localhost:8080/v1',
+  apiKey: 'sk-my-secret-key',
 });
 
-// Create an assistant
 const assistant = await openai.beta.assistants.create({
-  name: 'My Assistant',
-  instructions: 'You are a helpful assistant',
+  name: 'Math Tutor',
+  instructions: 'You help with math problems',
   model: 'ollama/llama3.2:3b',
 });
 
-// Create a thread
 const thread = await openai.beta.threads.create();
 
-// Add a message
 await openai.beta.threads.messages.create(thread.id, {
   role: 'user',
-  content: 'Hello!',
+  content: 'What is 2 + 2?',
 });
 
-// Run the assistant
 const run = await openai.beta.threads.runs.create(thread.id, {
   assistant_id: assistant.id,
 });
+
+// Poll for completion
+let status = run.status;
+while (status === 'queued' || status === 'in_progress') {
+  await new Promise((r) => setTimeout(r, 1000));
+  const updated = await openai.beta.threads.runs.retrieve(thread.id, run.id);
+  status = updated.status;
+}
+
+const messages = await openai.beta.threads.messages.list(thread.id);
+console.log(messages.data[0].content);
+```
+
+---
+
+## OpenAI Server
+
+The `OpenAIServer` exposes Cogitator as an OpenAI-compatible REST API.
+
+### Configuration
+
+```typescript
+import { OpenAIServer, createOpenAIServer } from '@cogitator-ai/openai-compat';
+
+const server = new OpenAIServer(cogitator, {
+  port: 8080,
+  host: '0.0.0.0',
+
+  apiKeys: ['sk-key1', 'sk-key2'],
+
+  tools: [calculator, datetime, webSearch],
+
+  logging: true,
+
+  cors: {
+    origin: ['http://localhost:3000', 'https://myapp.com'],
+    methods: ['GET', 'POST', 'DELETE', 'OPTIONS'],
+  },
+});
+```
+
+### Configuration Options
+
+| Option         | Type                            | Default                                | Description                                      |
+| -------------- | ------------------------------- | -------------------------------------- | ------------------------------------------------ |
+| `port`         | `number`                        | `8080`                                 | Port to listen on                                |
+| `host`         | `string`                        | `'0.0.0.0'`                            | Host to bind to                                  |
+| `apiKeys`      | `string[]`                      | `[]`                                   | API keys for authentication. Empty disables auth |
+| `tools`        | `Tool[]`                        | `[]`                                   | Tools available to assistants                    |
+| `logging`      | `boolean`                       | `false`                                | Enable request logging                           |
+| `cors.origin`  | `string \| string[] \| boolean` | `true`                                 | CORS origin configuration                        |
+| `cors.methods` | `string[]`                      | `['GET', 'POST', 'DELETE', 'OPTIONS']` | Allowed HTTP methods                             |
+
+### Server Lifecycle
+
+```typescript
+await server.start();
+
+console.log(server.getUrl());
+console.log(server.getBaseUrl());
+
+console.log(server.isRunning());
+
+const adapter = server.getAdapter();
+
+await server.stop();
+```
+
+### Health Check
+
+The server provides a health endpoint:
+
+```bash
+curl http://localhost:8080/health
+# {"status":"ok"}
+```
+
+---
+
+## OpenAI Adapter
+
+The `OpenAIAdapter` provides in-process access without running a server.
+
+```typescript
+import { OpenAIAdapter, createOpenAIAdapter } from '@cogitator-ai/openai-compat';
+
+const adapter = createOpenAIAdapter(cogitator, {
+  tools: [calculator],
+});
+```
+
+### Assistant Management
+
+```typescript
+const assistant = adapter.createAssistant({
+  model: 'openai/gpt-4o',
+  name: 'Code Helper',
+  instructions: 'You help write code',
+  temperature: 0.7,
+  tools: [{ type: 'code_interpreter' }],
+  metadata: { category: 'development' },
+});
+
+const fetched = adapter.getAssistant(assistant.id);
+
+const updated = adapter.updateAssistant(assistant.id, {
+  name: 'Code Expert',
+  temperature: 0.5,
+});
+
+const all = adapter.listAssistants();
+
+const deleted = adapter.deleteAssistant(assistant.id);
+```
+
+### Thread Operations
+
+```typescript
+const thread = adapter.createThread({ project: 'demo' });
+
+const fetched = adapter.getThread(thread.id);
+
+const message = adapter.addMessage(thread.id, {
+  role: 'user',
+  content: 'Hello, how are you?',
+  metadata: { source: 'web' },
+});
+
+const messages = adapter.listMessages(thread.id, {
+  limit: 20,
+  order: 'asc',
+  after: 'msg_abc123',
+  before: 'msg_xyz789',
+  run_id: 'run_123',
+});
+
+const msg = adapter.getMessage(thread.id, 'msg_abc123');
+
+adapter.deleteThread(thread.id);
+```
+
+### Run Execution
+
+```typescript
+const run = await adapter.createRun(thread.id, {
+  assistant_id: assistant.id,
+  model: 'openai/gpt-4o',
+  instructions: 'Be concise',
+  temperature: 0.5,
+  additional_messages: [{ role: 'user', content: 'Extra context' }],
+  metadata: { source: 'api' },
+});
+
+const status = adapter.getRun(thread.id, run.id);
+
+const cancelled = adapter.cancelRun(thread.id, run.id);
+```
+
+### Tool Outputs
+
+```typescript
+const run = adapter.getRun(thread.id, runId);
+
+if (run?.status === 'requires_action') {
+  const toolCalls = run.required_action?.submit_tool_outputs.tool_calls;
+
+  const outputs = await Promise.all(
+    toolCalls!.map(async (call) => ({
+      tool_call_id: call.id,
+      output: await executeMyTool(call.function.name, call.function.arguments),
+    }))
+  );
+
+  await adapter.submitToolOutputs(thread.id, runId, {
+    tool_outputs: outputs,
+  });
+}
+```
+
+---
+
+## Thread Manager
+
+The `ThreadManager` handles storage for threads, messages, assistants, and files.
+
+```typescript
+import { ThreadManager } from '@cogitator-ai/openai-compat';
+
+const manager = new ThreadManager();
+```
+
+### Assistant Storage
+
+```typescript
+interface StoredAssistant {
+  id: string;
+  name: string | null;
+  model: string;
+  instructions: string | null;
+  tools: AssistantTool[];
+  metadata: Record<string, string>;
+  temperature?: number;
+  created_at: number;
+}
+
+const assistant = manager.createAssistant({
+  model: 'gpt-4o',
+  name: 'Helper',
+  instructions: 'Be helpful',
+});
+
+const fetched = manager.getAssistant(assistant.id);
+const updated = manager.updateAssistant(assistant.id, { name: 'Expert' });
+const all = manager.listAssistants();
+manager.deleteAssistant(assistant.id);
+```
+
+### Thread Storage
+
+```typescript
+const thread = manager.createThread({ key: 'value' });
+const fetched = manager.getThread(thread.id);
+manager.deleteThread(thread.id);
+```
+
+### Message Operations
+
+```typescript
+const message = manager.addMessage(thread.id, {
+  role: 'user',
+  content: 'Hello!',
+});
+
+const assistantMsg = manager.addAssistantMessage(thread.id, 'Hi there!', assistant.id, run.id);
+
+const messages = manager.listMessages(thread.id, {
+  limit: 50,
+  order: 'desc',
+});
+
+const llmMessages = manager.getMessagesForLLM(thread.id);
+```
+
+### File Management
+
+```typescript
+const file = manager.addFile(Buffer.from('file content'), 'document.txt');
+
+const fetched = manager.getFile(file.id);
+
+const all = manager.listFiles();
+
+manager.deleteFile(file.id);
+```
+
+---
+
+## Supported Endpoints
+
+### Models
+
+| Method | Endpoint     | Description           |
+| ------ | ------------ | --------------------- |
+| GET    | `/v1/models` | List available models |
+
+### Assistants
+
+| Method | Endpoint             | Description      |
+| ------ | -------------------- | ---------------- |
+| POST   | `/v1/assistants`     | Create assistant |
+| GET    | `/v1/assistants`     | List assistants  |
+| GET    | `/v1/assistants/:id` | Get assistant    |
+| POST   | `/v1/assistants/:id` | Update assistant |
+| DELETE | `/v1/assistants/:id` | Delete assistant |
+
+### Threads
+
+| Method | Endpoint          | Description   |
+| ------ | ----------------- | ------------- |
+| POST   | `/v1/threads`     | Create thread |
+| GET    | `/v1/threads/:id` | Get thread    |
+| DELETE | `/v1/threads/:id` | Delete thread |
+
+### Messages
+
+| Method | Endpoint                           | Description   |
+| ------ | ---------------------------------- | ------------- |
+| POST   | `/v1/threads/:id/messages`         | Add message   |
+| GET    | `/v1/threads/:id/messages`         | List messages |
+| GET    | `/v1/threads/:id/messages/:msg_id` | Get message   |
+
+### Runs
+
+| Method | Endpoint                                           | Description         |
+| ------ | -------------------------------------------------- | ------------------- |
+| POST   | `/v1/threads/:id/runs`                             | Create run          |
+| GET    | `/v1/threads/:id/runs/:run_id`                     | Get run status      |
+| POST   | `/v1/threads/:id/runs/:run_id/cancel`              | Cancel run          |
+| POST   | `/v1/threads/:id/runs/:run_id/submit_tool_outputs` | Submit tool outputs |
+
+### Files
+
+| Method | Endpoint                | Description           |
+| ------ | ----------------------- | --------------------- |
+| POST   | `/v1/files`             | Upload file           |
+| GET    | `/v1/files`             | List files            |
+| GET    | `/v1/files/:id`         | Get file metadata     |
+| GET    | `/v1/files/:id/content` | Download file content |
+| DELETE | `/v1/files/:id`         | Delete file           |
+
+---
+
+## Error Handling
+
+The server returns OpenAI-compatible error responses:
+
+```typescript
+interface OpenAIError {
+  error: {
+    message: string;
+    type: string;
+    param?: string;
+    code?: string;
+  };
+}
+```
+
+### Error Types
+
+| HTTP Status | Type                    | Description                |
+| ----------- | ----------------------- | -------------------------- |
+| 400         | `invalid_request_error` | Invalid request parameters |
+| 401         | `authentication_error`  | Invalid or missing API key |
+| 404         | `invalid_request_error` | Resource not found         |
+| 500         | `server_error`          | Internal server error      |
+
+### Client-Side Error Handling
+
+```typescript
+try {
+  const run = await openai.beta.threads.runs.create(threadId, {
+    assistant_id: 'invalid-id',
+  });
+} catch (error) {
+  if (error instanceof OpenAI.APIError) {
+    console.log(error.status);
+    console.log(error.message);
+    console.log(error.code);
+  }
+}
+```
+
+---
+
+## Run Status
+
+Runs go through the following states:
+
+```typescript
+type RunStatus =
+  | 'queued'
+  | 'in_progress'
+  | 'requires_action'
+  | 'cancelling'
+  | 'cancelled'
+  | 'failed'
+  | 'completed'
+  | 'incomplete'
+  | 'expired';
+```
+
+### Status Flow
+
+```
+queued → in_progress → completed
+                     → failed
+                     → requires_action → in_progress → ...
+
+in_progress → cancelling → cancelled
+```
+
+### Polling for Completion
+
+```typescript
+async function waitForRun(openai: OpenAI, threadId: string, runId: string): Promise<Run> {
+  const terminalStates = ['completed', 'failed', 'cancelled', 'expired'];
+
+  while (true) {
+    const run = await openai.beta.threads.runs.retrieve(threadId, runId);
+
+    if (terminalStates.includes(run.status)) {
+      return run;
+    }
+
+    if (run.status === 'requires_action') {
+      return run;
+    }
+
+    await new Promise((r) => setTimeout(r, 1000));
+  }
+}
+```
+
+---
+
+## Stream Events
+
+The package defines stream event types for future streaming support:
+
+```typescript
+type StreamEvent =
+  | { event: 'thread.created'; data: Thread }
+  | { event: 'thread.run.created'; data: Run }
+  | { event: 'thread.run.queued'; data: Run }
+  | { event: 'thread.run.in_progress'; data: Run }
+  | { event: 'thread.run.requires_action'; data: Run }
+  | { event: 'thread.run.completed'; data: Run }
+  | { event: 'thread.run.failed'; data: Run }
+  | { event: 'thread.run.cancelled'; data: Run }
+  | { event: 'thread.message.created'; data: Message }
+  | { event: 'thread.message.delta'; data: MessageDelta }
+  | { event: 'thread.message.completed'; data: Message }
+  | { event: 'done'; data: '[DONE]' };
+```
+
+---
+
+## Type Reference
+
+### Core Types
+
+```typescript
+import type {
+  OpenAIError,
+  ListResponse,
+  Assistant,
+  AssistantTool,
+  FunctionDefinition,
+  ResponseFormat,
+  CreateAssistantRequest,
+  UpdateAssistantRequest,
+} from '@cogitator-ai/openai-compat';
+```
+
+### Thread Types
+
+```typescript
+import type { Thread, ToolResources, CreateThreadRequest } from '@cogitator-ai/openai-compat';
+```
+
+### Message Types
+
+```typescript
+import type {
+  Message,
+  MessageContent,
+  TextContent,
+  TextAnnotation,
+  Attachment,
+  CreateMessageRequest,
+  MessageContentPart,
+  MessageDelta,
+} from '@cogitator-ai/openai-compat';
+```
+
+### Run Types
+
+```typescript
+import type {
+  Run,
+  RunStatus,
+  RequiredAction,
+  ToolCall,
+  RunError,
+  Usage,
+  ToolChoice,
+  CreateRunRequest,
+  SubmitToolOutputsRequest,
+  ToolOutput,
+} from '@cogitator-ai/openai-compat';
+```
+
+### Run Step Types
+
+```typescript
+import type { RunStep, StepDetails, StepToolCall, RunStepDelta } from '@cogitator-ai/openai-compat';
+```
+
+### File Types
+
+```typescript
+import type { FileObject, FilePurpose, UploadFileRequest } from '@cogitator-ai/openai-compat';
+```
+
+### Stream Types
+
+```typescript
+import type { StreamEvent, MessageDelta, RunStepDelta } from '@cogitator-ai/openai-compat';
+```
+
+---
+
+## Examples
+
+### Chat Bot with Memory
+
+```typescript
+import { createOpenAIServer } from '@cogitator-ai/openai-compat';
+import { Cogitator } from '@cogitator-ai/core';
+import OpenAI from 'openai';
+
+const cogitator = new Cogitator({
+  defaultModel: 'ollama/llama3.2:3b',
+});
+
+const server = createOpenAIServer(cogitator, { port: 8080 });
+await server.start();
+
+const openai = new OpenAI({
+  baseURL: server.getBaseUrl(),
+  apiKey: 'not-needed',
+});
+
+const assistant = await openai.beta.assistants.create({
+  name: 'Chat Bot',
+  instructions: 'You are a friendly chat bot. Remember previous messages.',
+  model: 'ollama/llama3.2:3b',
+});
+
+const thread = await openai.beta.threads.create();
+
+async function chat(message: string): Promise<string> {
+  await openai.beta.threads.messages.create(thread.id, {
+    role: 'user',
+    content: message,
+  });
+
+  const run = await openai.beta.threads.runs.create(thread.id, {
+    assistant_id: assistant.id,
+  });
+
+  while (true) {
+    const status = await openai.beta.threads.runs.retrieve(thread.id, run.id);
+    if (status.status === 'completed') break;
+    if (status.status === 'failed') throw new Error(status.last_error?.message);
+    await new Promise((r) => setTimeout(r, 500));
+  }
+
+  const messages = await openai.beta.threads.messages.list(thread.id, {
+    limit: 1,
+    order: 'desc',
+  });
+
+  const content = messages.data[0].content[0];
+  return content.type === 'text' ? content.text.value : '';
+}
+
+console.log(await chat('Hi, my name is Alex'));
+console.log(await chat('What is my name?'));
 ```
 
-### Supported Endpoints
+### Code Assistant with Tools
+
+```typescript
+import { createOpenAIServer } from '@cogitator-ai/openai-compat';
+import { Cogitator, tool } from '@cogitator-ai/core';
+import { z } from 'zod';
+import OpenAI from 'openai';
 
-- `POST /v1/assistants` - Create assistant
-- `GET /v1/assistants` - List assistants
-- `DELETE /v1/assistants/:id` - Delete assistant
-- `POST /v1/threads` - Create thread
-- `GET /v1/threads/:id` - Get thread
-- `POST /v1/threads/:id/messages` - Add message
-- `GET /v1/threads/:id/messages` - List messages
-- `POST /v1/threads/:id/runs` - Create run
-- `GET /v1/threads/:id/runs/:id` - Get run status
+const runCode = tool({
+  name: 'run_code',
+  description: 'Execute Python code',
+  parameters: z.object({
+    code: z.string().describe('Python code to execute'),
+  }),
+  execute: async ({ code }) => {
+    return `Output: ${code.length} characters`;
+  },
+});
+
+const cogitator = new Cogitator({
+  defaultModel: 'openai/gpt-4o',
+});
+
+const server = createOpenAIServer(cogitator, {
+  port: 8080,
+  tools: [runCode],
+});
+
+await server.start();
 
-## Documentation
+const openai = new OpenAI({
+  baseURL: server.getBaseUrl(),
+  apiKey: process.env.OPENAI_API_KEY,
+});
+
+const assistant = await openai.beta.assistants.create({
+  name: 'Code Runner',
+  instructions: 'You can run Python code using the run_code tool.',
+  model: 'openai/gpt-4o',
+  tools: [{ type: 'function', function: { name: 'run_code' } }],
+});
+```
+
+### File Upload
+
+```typescript
+import OpenAI from 'openai';
+
+const openai = new OpenAI({
+  baseURL: 'http://localhost:8080/v1',
+  apiKey: 'key',
+});
+
+const file = await openai.files.create({
+  file: fs.createReadStream('data.csv'),
+  purpose: 'assistants',
+});
+
+console.log('Uploaded:', file.id);
+
+const content = await openai.files.content(file.id);
+console.log('Content:', await content.text());
+
+await openai.files.del(file.id);
+```
+
+### Multi-Model Setup
+
+```typescript
+const cogitator = new Cogitator({
+  defaultModel: 'ollama/llama3.2:3b',
+});
+
+const server = createOpenAIServer(cogitator, { port: 8080 });
+await server.start();
+
+const openai = new OpenAI({
+  baseURL: server.getBaseUrl(),
+  apiKey: 'not-needed',
+});
+
+const localAssistant = await openai.beta.assistants.create({
+  name: 'Local Assistant',
+  model: 'ollama/llama3.2:3b',
+  instructions: 'Fast local responses',
+});
+
+const cloudAssistant = await openai.beta.assistants.create({
+  name: 'Cloud Assistant',
+  model: 'openai/gpt-4o',
+  instructions: 'Complex reasoning tasks',
+});
+```
 
-See the [Cogitator documentation](https://github.com/eL1fe/cogitator) for full API reference.
+---
 
 ## License