@hebo-ai/gateway 0.7.0 → 0.8.0-rc0

package/README.md

@@ -13,6 +13,7 @@ Learn more in our blog post: [Yet Another AI Gateway?](https://hebo.ai/blog/2601
 ## 🍌 Features
 
 - 🌐 OpenAI-compatible /chat/completions, /embeddings & /models endpoints.
+- 💬 /conversations endpoint built on top of the Responses API.
 - 🔌 Integrate into your existing Hono, Elysia, Next.js & TanStack apps.
 - 🧩 Provider registry compatible with Vercel AI SDK providers.
 - 🧭 Canonical model IDs and parameter naming across providers.
@@ -32,7 +33,7 @@ bun install @hebo-ai/gateway
 - Quickstart
   - [Setup A Gateway Instance](#setup-a-gateway-instance) | [Mount Route Handlers](#mount-route-handlers) | [Call the Gateway](#call-the-gateway)
 - Configuration Reference
-  - [Providers](#providers) | [Models](#models) | [Hooks](#hooks) | [Logger](#logger-settings) | [Observability](#observability) | [Timeouts](#timeout-settings)
+  - [Providers](#providers) | [Models](#models) | [Hooks](#hooks) | [Storage](#storage) | [Logger](#logger-settings) | [Observability](#observability) | [Timeouts](#timeout-settings)
 - Framework Support
   - [ElysiaJS](#elysiajs) | [Hono](#hono) | [Next.js](#nextjs) | [TanStack Start](#tanstack-start)
 - Runtime Support
@@ -343,9 +344,7 @@ const gw = gateway({
      */
     after: async (ctx: {
       result: ChatCompletions | ChatCompletionsStream | Embeddings;
-    }): Promise<
-      ChatCompletions | ChatCompletionsStream | Embeddings | void
-    > => {
+    }): Promise<ChatCompletions | ChatCompletionsStream | Embeddings | void> => {
       // Example Use Cases:
       // - Transform result
       // - Result logging
@@ -371,6 +370,52 @@ The `ctx` object is **readonly for core fields**. Use return values to override
 > [!TIP]
 > To pass data between hooks, use `ctx.state`. It’s a per-request mutable bag in which you can stash things like auth info, routing decisions, timers, or trace IDs and read them later again in any of the other hooks.
 
+### Storage
+
+The `/conversations` endpoint stores conversation history and associated items. By default, the gateway uses an in-memory storage, which is suitable for development but not for production as data is lost when the server restarts.
+
+#### In-Memory Storage
+
+You can configure the size of the in-memory storage (default is 256MB).
+
+```ts
+import { gateway } from "@hebo-ai/gateway";
+import { InMemoryStorage } from "@hebo-ai/gateway/storage/memory";
+
+const gw = gateway({
+  // ...
+  storage: new InMemoryStorage({
+    maxSize: 512 * 1024 * 1024, // 512MB
+  }),
+});
+```
+
+#### SQL Storage
+
+Hebo Gateway provides high-performance SQL adapters for **PostgreSQL**, **SQLite**, **MySQL**, and **GrepTimeDB**. It supports common drivers like `pg`, `postgres.js`, `mysql2`, `better-sqlite3`, `@libsql/client`, and `Bun.SQL`.
+
+```ts
+import { gateway } from "@hebo-ai/gateway";
+import { SqlStorage, PostgresDialect } from "@hebo-ai/gateway/storage/sql";
+import { Pool } from "pg";
+
+// 1. Setup dialect-specific client (e.g. pg, mysql2, sqlite, bun)
+const client = new Pool({ connectionString: process.env.DATABASE_URL });
+
+// 2. Setup storage with matching dialect (PostgresDialect, SqliteDialect, MysqlDialect, ...)
+const storage = new SqlStorage({
+  dialect: new PostgresDialect({ client }),
+});
+
+// 3. Run migrations
+await storage.migrate();
+
+const gw = gateway({ storage });
+```
+
+> [!TIP]
+> The `PostgresDialect` includes optimized `JSONB` storage and high-performance `BRIN` indexing for time-ordered data by default.
+
 ## 🧩 Framework Support
 
 Hebo Gateway exposes **WinterCG-compatible** handlers that integrate with almost any existing framework.
@@ -580,6 +625,38 @@ Provider-specific mapping:
 
 When available, the resolved value is echoed back on response as `service_tier`.
 
+### Conversations
+
+Hebo Gateway provides a dedicated `/conversations` endpoint for managing persistent conversation state. It is designed as an extension of the [OpenAI Conversations API](https://developers.openai.com/api/reference/typescript/resources/conversations) and supports standard CRUD operations alongside advanced listing with metadata filtering.
+
+#### List & Filter Conversations
+
+You can list conversations with standard cursor-based pagination and filter by any metadata key using the `metadata.KEY=VALUE` pattern.
+
+```bash
+# List conversations for a specific user
+curl "https://api.gateway.com/conversations?limit=10&metadata.user_id=123"
+```
+
+The response follows the standard OpenAI list object:
+
+```json
+{
+  "object": "list",
+  "data": [
+    {
+      "id": "conv_abc123",
+      "object": "conversation",
+      "created_at": 1678531200,
+      "metadata": { "user_id": "123" }
+    }
+  ],
+  "first_id": "conv_abc123",
+  "last_id": "conv_abc123",
+  "has_more": false
+}
+```
+
 ### Prompt Caching
 
 The chat completions endpoint supports both implicit (provider-managed) and explicit prompt caching across OpenAI-compatible providers.
@@ -888,7 +965,7 @@ export async function handler(req: Request): Promise<Response> {
 }
 ```
 
-Non-streaming versions are available via `toChatCompletionsResponse`. Equivalent schemas and helpers are available in the `embeddings` and `models` endpoints.
+Non-streaming versions are available via `toChatCompletionsResponse`. Equivalent schemas and helpers are available in the `conversations`, `embeddings` and `models` endpoints.
 
 > [!TIP]
 > Since Zod v4.3 you can generate a JSON Schema from any zod object by calling `z.toJSONSchema(...)`. This is useful for producing OpenAPI documentation from the same source of truth.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@hebo-ai/gateway",
-  "version": "0.7.0",
+  "version": "0.8.0-rc0",
   "description": "AI gateway as a framework. For full control over models, routing & lifecycle. OpenAI-compatible /chat/completions, /embeddings & /models.",
   "keywords": [
     "ai",
@@ -53,6 +53,11 @@
       "types": "./dist/endpoints/chat-completions/index.d.ts",
       "import": "./dist/endpoints/chat-completions/index.js"
     },
+    "./endpoints/conversations": {
+      "types": "./dist/endpoints/conversations/index.d.ts",
+      "import": "./dist/endpoints/conversations/index.js",
+      "dev-source": "./src/endpoints/conversations/index.ts"
+    },
     "./endpoints/embeddings": {
       "types": "./dist/endpoints/embeddings/index.d.ts",
       "import": "./dist/endpoints/embeddings/index.js"
@@ -61,6 +66,16 @@
       "types": "./dist/endpoints/models/index.d.ts",
       "import": "./dist/endpoints/models/index.js"
     },
+    "./storage/memory": {
+      "types": "./dist/endpoints/conversations/storage/memory.d.ts",
+      "import": "./dist/endpoints/conversations/storage/memory.js",
+      "dev-source": "./src/endpoints/conversations/storage/memory.ts"
+    },
+    "./storage/sql": {
+      "types": "./dist/endpoints/conversations/storage/sql.d.ts",
+      "import": "./dist/endpoints/conversations/storage/sql.js",
+      "dev-source": "./src/endpoints/conversations/storage/sql.ts"
+    },
     "./errors/openai": {
       "types": "./dist/errors/openai.d.ts",
       "import": "./dist/errors/openai.js"
@@ -142,6 +157,8 @@
   "dependencies": {
     "@ai-sdk/provider": "^3.0.8",
     "ai": "^6.0.116",
+    "lru-cache": "^11.2.6",
+    "uuid": "^13.0.0",
     "zod": "^4.3.6"
   },
   "devDependencies": {
@@ -153,23 +170,31 @@
     "@ai-sdk/openai": "^3.0.41",
     "@aws-sdk/credential-providers": "^3.1004.0",
     "@langfuse/otel": "^4.6.1",
+    "@libsql/client": "^0.17.0",
     "@mjackson/node-fetch-server": "^0.7.0",
     "@opentelemetry/api": "^1.9.0",
     "@opentelemetry/context-async-hooks": "^2.6.0",
     "@opentelemetry/sdk-trace-base": "^2.6.0",
     "@tanstack/react-router": "^1.166.3",
     "@tanstack/react-start": "^1.166.3",
+    "@types/better-sqlite3": "^7.6.13",
     "@types/bun": "latest",
+    "@types/pg": "^8.18.0",
     "@types/react": "^19.2.14",
     "@types/react-dom": "^19.2.3",
+    "@types/uuid": "^11.0.0",
+    "better-sqlite3": "^12.6.2",
     "elysia": "^1.4.27",
     "hono": "^4.12.5",
     "lefthook": "^2.1.3",
+    "mysql2": "^3.18.2",
     "next": "^16.1.6",
     "oxfmt": "^0.24.0",
     "oxlint": "^1.51.0",
     "oxlint-tsgolint": "^0.16.0",
+    "pg": "^8.19.0",
     "pino": "^10.3.1",
+    "postgres": "^3.4.8",
     "typescript": "^5.9.3",
     "vite": "^7.3.1",
     "vite-tsconfig-paths": "^6.1.1",
@@ -183,7 +208,12 @@
     "@ai-sdk/google-vertex": "^4.0.80",
     "@ai-sdk/groq": "^3.0.29",
     "@ai-sdk/openai": "^3.0.41",
+    "@libsql/client": "^0.14.0",
     "@opentelemetry/api": "^1.9.0",
+    "better-sqlite3": "^11.0.0",
+    "mysql2": "^3.11.0",
+    "pg": "^8.13.0",
+    "postgres": "^3.4.0",
     "typescript": "^5.9.3",
     "voyage-ai-provider": "^3.0.0"
   },
@@ -211,6 +241,21 @@
     },
     "voyage-ai-provider": {
       "optional": true
+    },
+    "pg": {
+      "optional": true
+    },
+    "postgres": {
+      "optional": true
+    },
+    "mysql2": {
+      "optional": true
+    },
+    "better-sqlite3": {
+      "optional": true
+    },
+    "@libsql/client": {
+      "optional": true
     }
   }
 }

package/dist/config.d.ts

@@ -1,2 +0,0 @@
-import { type GatewayConfig, type GatewayConfigParsed } from "./types";
-export declare const parseConfig: (config: GatewayConfig) => GatewayConfigParsed;

package/dist/config.js

@@ -1,108 +0,0 @@
-import { isLogger, logger, setLoggerInstance } from "./logger";
-import { createDefaultLogger } from "./logger/default";
-import { installAiSdkWarningLogger } from "./telemetry/ai-sdk";
-import { DEFAULT_CHAT_TIMEOUT_MS, kParsed, } from "./types";
-export const parseConfig = (config) => {
-    // If it has been parsed before, just return.
-    if (kParsed in config)
-        return config;
-    const providers = config.providers ?? {};
-    const parsedProviders = {};
-    const models = config.models ?? {};
-    // Set the global logger instance.
-    if (config.logger === undefined) {
-        setLoggerInstance(createDefaultLogger({}));
-    }
-    else if (config.logger !== null) {
-        setLoggerInstance(isLogger(config.logger) ? config.logger : createDefaultLogger(config.logger));
-        logger.info(isLogger(config.logger)
-            ? `[logger] custom logger configured`
-            : `[logger] logger configured: level=${config.logger.level}`);
-    }
-    // Strip providers that are not configured.
-    for (const id in providers) {
-        const provider = providers[id];
-        if (provider === undefined) {
-            logger.warn(`[config] ${id} provider removed (undefined)`);
-            continue;
-        }
-        parsedProviders[id] = provider;
-    }
-    if (Object.keys(parsedProviders).length === 0) {
-        throw new Error("No providers configured (config.providers is empty)");
-    }
-    // Strip providers that are not configured from models.
-    const parsedModels = {};
-    const warnings = new Set();
-    for (const id in models) {
-        const model = models[id];
-        const kept = [];
-        for (const p of model.providers) {
-            if (p in parsedProviders)
-                kept.push(p);
-            else
-                warnings.add(p);
-        }
-        if (kept.length > 0)
-            parsedModels[id] = { ...model, providers: kept };
-    }
-    for (const warning of warnings) {
-        logger.warn(`[config] ${warning} provider removed (not configured)`);
-    }
-    if (Object.keys(parsedModels).length === 0) {
-        throw new Error("No models configured (config.models is empty)");
-    }
-    // Default for the telemetry settings.
-    const telemetryEnabled = config.telemetry?.enabled ?? false;
-    const telemetrySignals = telemetryEnabled
-        ? {
-            http: config.telemetry?.signals?.http ?? "recommended",
-            gen_ai: config.telemetry?.signals?.gen_ai ?? "full",
-            hebo: config.telemetry?.signals?.hebo ?? "off",
-        }
-        : {
-            http: "off",
-            gen_ai: "off",
-            hebo: "off",
-        };
-    installAiSdkWarningLogger(telemetrySignals.gen_ai);
-    // Default timeouts
-    let normal;
-    let flex;
-    const t = config.timeouts;
-    if (t === null) {
-        normal = flex = undefined;
-    }
-    else if (typeof t === "number") {
-        normal = t;
-        flex = t * 3;
-    }
-    else {
-        if (t?.normal === null)
-            normal = undefined;
-        else if (t?.normal === undefined)
-            normal = DEFAULT_CHAT_TIMEOUT_MS;
-        else
-            normal = t.normal;
-        if (t?.flex === null)
-            flex = undefined;
-        else if (t?.flex === undefined)
-            flex = normal === undefined ? undefined : normal * 3;
-        else
-            flex = t.flex;
-    }
-    const parsedTimeouts = { normal, flex };
-    // Return parsed config.
-    return {
-        ...config,
-        timeouts: parsedTimeouts,
-        telemetry: {
-            ...config.telemetry,
-            enabled: telemetryEnabled,
-            signals: telemetrySignals,
-        },
-        providers: parsedProviders,
-        models: parsedModels,
-        [kParsed]: true,
-    };
-};

package/dist/endpoints/chat-completions/converters.d.ts

@@ -1,43 +0,0 @@
-import type { SharedV3ProviderOptions, SharedV3ProviderMetadata } from "@ai-sdk/provider";
-import type { GenerateTextResult, StreamTextResult, FinishReason, ToolChoice, ToolSet, ModelMessage, UserContent, LanguageModelUsage, TextStreamPart, ReasoningOutput, AssistantModelMessage, ToolModelMessage, UserModelMessage } from "ai";
-import { Output } from "ai";
-import type { ChatCompletionsToolCall, ChatCompletionsTool, ChatCompletionsToolChoice, ChatCompletionsStream, ChatCompletionsContentPart, ChatCompletionsMessage, ChatCompletionsUserMessage, ChatCompletionsAssistantMessage, ChatCompletionsToolMessage, ChatCompletionsFinishReason, ChatCompletionsUsage, ChatCompletionsInputs, ChatCompletions, ChatCompletionsChunk, ChatCompletionsReasoningDetail } from "./schema";
-import type { SseErrorFrame, SseFrame } from "../../utils/stream";
-export type TextCallOptions = {
-    messages: ModelMessage[];
-    tools?: ToolSet;
-    toolChoice?: ToolChoice<ToolSet>;
-    activeTools?: Array<keyof ToolSet>;
-    output?: Output.Output;
-    temperature?: number;
-    maxOutputTokens?: number;
-    frequencyPenalty?: number;
-    presencePenalty?: number;
-    seed?: number;
-    stopSequences?: string[];
-    topP?: number;
-    providerOptions: SharedV3ProviderOptions;
-};
-export declare function convertToTextCallOptions(params: ChatCompletionsInputs): TextCallOptions;
-export declare function convertToModelMessages(messages: ChatCompletionsMessage[]): ModelMessage[];
-export declare function fromChatCompletionsUserMessage(message: ChatCompletionsUserMessage): UserModelMessage;
-export declare function fromChatCompletionsAssistantMessage(message: ChatCompletionsAssistantMessage): AssistantModelMessage;
-export declare function fromChatCompletionsToolResultMessage(message: ChatCompletionsAssistantMessage, toolById: Map<string, ChatCompletionsToolMessage>): ToolModelMessage | undefined;
-export declare function fromChatCompletionsContent(content: ChatCompletionsContentPart[]): UserContent;
-export declare const convertToToolSet: (tools: ChatCompletionsTool[] | undefined) => ToolSet | undefined;
-export declare const convertToToolChoiceOptions: (toolChoice: ChatCompletionsToolChoice | undefined) => {
-    toolChoice?: ToolChoice<ToolSet>;
-    activeTools?: Array<keyof ToolSet>;
-};
-export declare function toChatCompletions(result: GenerateTextResult<ToolSet, Output.Output>, model: string): ChatCompletions;
-export declare function toChatCompletionsResponse(result: GenerateTextResult<ToolSet, Output.Output>, model: string, responseInit?: ResponseInit): Response;
-export declare function toChatCompletionsStream(result: StreamTextResult<ToolSet, Output.Output>, model: string): ChatCompletionsStream;
-export declare function toChatCompletionsStreamResponse(result: StreamTextResult<ToolSet, Output.Output>, model: string, responseInit?: ResponseInit): Response;
-export declare class ChatCompletionsTransformStream extends TransformStream<TextStreamPart<ToolSet>, SseFrame<ChatCompletionsChunk> | SseErrorFrame> {
-    constructor(model: string);
-}
-export declare const toChatCompletionsAssistantMessage: (result: GenerateTextResult<ToolSet, Output.Output>) => ChatCompletionsAssistantMessage;
-export declare function toReasoningDetail(reasoning: ReasoningOutput, id: string, index: number): ChatCompletionsReasoningDetail;
-export declare function toChatCompletionsUsage(usage: LanguageModelUsage): ChatCompletionsUsage;
-export declare function toChatCompletionsToolCall(id: string, name: string, args: unknown, providerMetadata?: SharedV3ProviderMetadata): ChatCompletionsToolCall;
-export declare const toChatCompletionsFinishReason: (finishReason: FinishReason) => ChatCompletionsFinishReason;