@hebo-ai/gateway 0.9.0 → 0.9.2

package/README.md

@@ -42,7 +42,7 @@ bun install @hebo-ai/gateway
 - Endpoints
   - [/chat/completions](#chatcompletions) | [/embeddings](#embeddings) | [/models](#models) | [/responses](#responses) | [/conversations](#conversations)
 - OpenAI Extensions
-  - [Reasoning](#reasoning) | [Service Tier](#service-tier) | [Prompt Caching](#prompt-caching)
+  - [Reasoning](#reasoning) | [Service Tier](#service-tier) | [Prompt Caching](#prompt-caching) | [Compressed Requests](#compressed-requests)
 - Advanced Usage
   - [Passing Framework State to Hooks](#passing-framework-state-to-hooks) | [Selective Route Mounting](#selective-route-mounting) | [Low-level Schemas & Converters](#low-level-schemas--converters)
 
@@ -792,6 +792,36 @@ Provider behavior:
 - **Google Gemini**: maps `cached_content` to Gemini `cachedContent`.
 - **Amazon Nova (Bedrock)**: maps `cache_control` to Bedrock `cachePoints` and inserts an automatic cache point on a stable prefix when none is provided.
 
+
+### Compressed Requests
+
+The gateway supports gzip and deflate compressed request bodies via the Web Compression Streams API. The `maxBodySize` option controls the maximum *decompressed* body size for these compressed requests, protecting against gzip bombs and oversized payloads.
+
+```ts
+import { gateway } from "@hebo-ai/gateway";
+
+const gw = gateway({
+  // ...
+  // Maximum decompressed body size in bytes (default: 10 MB).
+  // Set to 0 to disable the decompressed size limit.
+  maxBodySize: 10 * 1024 * 1024,
+});
+```
+
+Compressed requests that exceed this limit after decompression receive an HTTP `413 Payload Too Large` response. Unsupported `Content-Encoding` values return HTTP `415 Unsupported Media Type`.
+
+> [!IMPORTANT]
+> **Plain (uncompressed) request body size limits** are *not* enforced by the gateway — they should be configured at the framework or server level. The gateway only enforces `maxBodySize` on decompressed output, since the framework cannot know the decompressed size ahead of time.
+>
+> Framework-level configuration examples:
+>
+> - **Bun** — [`Bun.serve({ maxRequestBodySize: 10_485_760 })`](https://bun.sh/docs/api/http#bun-serve)
+> - **Elysia** — inherits from Bun's `maxRequestBodySize`
+> - **Hono** — [`bodyLimit` middleware](https://hono.dev/docs/middleware/builtin/body-limit): `app.use(bodyLimit({ maxSize: 10 * 1024 * 1024 }))`
+> - **Express** — [`express.json({ limit: '10mb' })`](https://expressjs.com/en/api.html#express.json)
+> - **Fastify** — [`fastify({ bodyLimit: 10485760 })`](https://fastify.dev/docs/latest/Reference/Server/#bodylimit)
+> - **Node.js `http`** — [`server.maxRequestSize`](https://nodejs.org/api/http.html) (v22.6+), or use a reverse proxy like nginx (`client_max_body_size 10m`)
+
 ## 🧪 Advanced Usage
 
 ### Logger Settings
@@ -863,19 +893,37 @@ Attribute names and span & metrics semantics follow OpenTelemetry GenAI semantic
 https://opentelemetry.io/docs/specs/semconv/gen-ai/gen-ai-spans/
 https://opentelemetry.io/docs/specs/semconv/gen-ai/gen-ai-metrics/
 
+For observability integration that is not otel compliant, you can disable built-in telemetry and manually instrument requests during `before` / `after` hooks.
+
+#### Custom Telemetry Attributes
+
+Use `ctx.otel` in any hook to attach attributes to both spans and metrics:
+
+```ts
+hooks: {
+  onRequest: (ctx) => {
+    ctx.otel["app.tenant.id"] = tenantId;
+    ctx.otel["app.user.id"] = userId;
+  },
+}
+```
+
+These attributes appear on the active span and on all metric instruments (request duration, token usage, TPOT, TTFT).
+
 > [!TIP]
 > To populate custom span attributes, the inbound W3C `baggage` header is supported. Keys in the `hebo.` namespace are mapped to span attributes, with the namespace stripped. For example: `baggage: hebo.user_id=u-123` becomes span attribute `user_id=u-123`.  
 > For `/chat/completions` and `/embeddings`, request `metadata` (`Record<string, string>`, key 1-64 chars, value up to 512 chars) is also forwarded to spans as `gen_ai.request.metadata.<key>`.
 
-For observability integration that is not otel compliant, you can disable built-in telemetry and manually instrument requests during `before` / `after` hooks.
-
 #### Metrics
 
 The Gateway also emits `gen_ai` metrics:
 
 - `gen_ai.server.request.duration` (histogram, seconds)
 - `gen_ai.server.time_per_output_token` (histogram, seconds)
-- `gen_ai.client.token.usage` (histogram, tokens; tagged with `gen_ai.token.type=input|output`)
+- `gen_ai.server.time_to_first_token` (histogram, seconds)
+- `gen_ai.client.token.usage` (histogram, tokens; tagged with `gen_ai.token.type=input|output|cached|reasoning`)
+
+Metric names and attributes follow OpenTelemetry GenAI semantic conventions. Histogram bucket boundaries are tuned for practical dashboards and alerting rather than copied verbatim from upstream recommendations.
 
 To capture them, configure a global `MeterProvider` before creating the gateway:
 
@@ -1073,3 +1121,33 @@ Non-streaming versions are available via `toChatCompletionsResponse`. Equivalent
 
 > [!TIP]
 > Since Zod v4.3 you can generate a JSON Schema from any zod object by calling `z.toJSONSchema(...)`. This is useful for producing OpenAPI documentation from the same source of truth.
+
+
+### Request Body Size
+
+The gateway supports gzip and deflate compressed request bodies via the Web Compression Streams API. The `maxBodySize` option controls the maximum *decompressed* body size for these compressed requests, protecting against gzip bombs and oversized payloads.
+
+```ts
+import { gateway } from "@hebo-ai/gateway";
+
+const gw = gateway({
+  // ...
+  // Maximum decompressed body size in bytes (default: 10 MB).
+  // Set to 0 to disable the decompressed size limit.
+  maxBodySize: 10 * 1024 * 1024,
+});
+```
+
+Compressed requests that exceed this limit after decompression receive an HTTP `413 Payload Too Large` response. Unsupported `Content-Encoding` values return HTTP `415 Unsupported Media Type`.
+
+> [!IMPORTANT]
+> **Plain (uncompressed) request body size limits** are *not* enforced by the gateway — they should be configured at the framework or server level. The gateway only enforces `maxBodySize` on decompressed output, since the framework cannot know the decompressed size ahead of time.
+>
+> Framework-level configuration examples:
+>
+> - **Bun** — [`Bun.serve({ maxRequestBodySize: 10_485_760 })`](https://bun.sh/docs/api/http#bun-serve)
+> - **Elysia** — inherits from Bun's `maxRequestBodySize`
+> - **Hono** — [`bodyLimit` middleware](https://hono.dev/docs/middleware/builtin/body-limit): `app.use(bodyLimit({ maxSize: 10 * 1024 * 1024 }))`
+> - **Express** — [`express.json({ limit: '10mb' })`](https://expressjs.com/en/api.html#express.json)
+> - **Fastify** — [`fastify({ bodyLimit: 10485760 })`](https://fastify.dev/docs/latest/Reference/Server/#bodylimit)
+> - **Node.js `http`** — [`server.maxRequestSize`](https://nodejs.org/api/http.html) (v22.6+), or use a reverse proxy like nginx (`client_max_body_size 10m`)

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@hebo-ai/gateway",
-  "version": "0.9.0",
+  "version": "0.9.2",
   "description": "AI gateway as a framework. For full control over models, routing & lifecycle. OpenAI-compatible /chat/completions, /embeddings & /models.",
   "keywords": [
     "ai",
@@ -160,51 +160,51 @@
     "typecheck": "oxlint --type-check",
     "test": "bun test",
     "check": "bun lint && bun typecheck",
-    "fix": "bun lint:staged && bun fmt:staged"
+    "fix": "bun lint:staged && bun format:staged"
   },
   "dependencies": {
     "@ai-sdk/provider": "^3.0.8",
-    "ai": "^6.0.116",
-    "lru-cache": "^11.2.6",
+    "ai": "^6.0.154",
+    "lru-cache": "^11.3.3",
     "uuid": "^13.0.0",
     "zod": "^4.3.6"
   },
   "devDependencies": {
-    "@ai-sdk/amazon-bedrock": "^4.0.77",
-    "@ai-sdk/anthropic": "^3.0.58",
-    "@ai-sdk/cohere": "^3.0.25",
-    "@ai-sdk/google-vertex": "^4.0.80",
-    "@ai-sdk/groq": "^3.0.29",
-    "@ai-sdk/openai": "^3.0.41",
-    "@aws-sdk/credential-providers": "^3.1004.0",
-    "@langfuse/otel": "^4.6.1",
-    "@libsql/client": "^0.17.0",
+    "@ai-sdk/amazon-bedrock": "^4.0.92",
+    "@ai-sdk/anthropic": "^3.0.68",
+    "@ai-sdk/cohere": "^3.0.30",
+    "@ai-sdk/google-vertex": "^4.0.105",
+    "@ai-sdk/groq": "^3.0.35",
+    "@ai-sdk/openai": "^3.0.52",
+    "@aws-sdk/credential-providers": "^3.1027.0",
+    "@langfuse/otel": "^5.0.2",
+    "@libsql/client": "^0.17.2",
     "@mjackson/node-fetch-server": "^0.7.0",
-    "@opentelemetry/api": "^1.9.0",
-    "@opentelemetry/context-async-hooks": "^2.6.0",
-    "@opentelemetry/sdk-trace-base": "^2.6.0",
-    "@tanstack/react-router": "^1.166.3",
-    "@tanstack/react-start": "^1.166.3",
+    "@opentelemetry/api": "^1.9.1",
+    "@opentelemetry/context-async-hooks": "^2.6.1",
+    "@opentelemetry/sdk-trace-base": "^2.6.1",
+    "@tanstack/react-router": "^1.168.10",
+    "@tanstack/react-start": "^1.167.16",
     "@types/better-sqlite3": "^7.6.13",
-    "@types/bun": "latest",
-    "@types/pg": "^8.18.0",
+    "@types/bun": "1.3.11",
+    "@types/pg": "^8.20.0",
     "@types/react": "^19.2.14",
     "@types/react-dom": "^19.2.3",
     "@types/uuid": "^11.0.0",
-    "better-sqlite3": "^12.6.2",
-    "elysia": "^1.4.27",
-    "hono": "^4.12.5",
-    "lefthook": "^2.1.3",
-    "mysql2": "^3.18.2",
-    "next": "^16.1.6",
-    "oxfmt": "^0.24.0",
-    "oxlint": "^1.51.0",
-    "oxlint-tsgolint": "^0.16.0",
+    "better-sqlite3": "^12.8.0",
+    "elysia": "^1.4.28",
+    "hono": "^4.12.12",
+    "lefthook": "^2.1.5",
+    "mysql2": "^3.21.0",
+    "next": "^16.2.3",
+    "oxfmt": "^0.44.0",
+    "oxlint": "^1.59.0",
+    "oxlint-tsgolint": "^0.20.0",
     "pg": "^8.19.0",
     "pino": "^10.3.1",
-    "postgres": "^3.4.8",
-    "typescript": "^5.9.3",
-    "vite": "^7.3.1",
+    "postgres": "^3.4.9",
+    "typescript": "^6.0.2",
+    "vite": "^7.3.2",
     "vite-tsconfig-paths": "^6.1.1",
     "voyage-ai-provider": "^3.0.0"
   },
@@ -217,12 +217,12 @@
     "@ai-sdk/groq": "^3.0.29",
     "@ai-sdk/openai": "^3.0.41",
     "@libsql/client": "^0.14.0",
-    "@opentelemetry/api": "^1.9.0",
+    "@opentelemetry/api": "^1.9.1",
     "better-sqlite3": "^11.0.0",
     "mysql2": "^3.11.0",
     "pg": "^8.13.0",
     "postgres": "^3.4.0",
-    "typescript": "^5.9.3",
+    "typescript": ">=5.9.3",
     "voyage-ai-provider": "^3.0.0"
   },
   "peerDependenciesMeta": {

package/dist/config.d.ts

@@ -1,2 +0,0 @@
-import { type GatewayConfig, type GatewayConfigParsed } from "./types";
-export declare const parseConfig: (config: GatewayConfig) => GatewayConfigParsed;

package/dist/config.js

@@ -1,111 +0,0 @@
-import { InMemoryStorage } from "./endpoints/conversations/storage/memory";
-import { isLogger, logger, setLoggerInstance } from "./logger";
-import { createDefaultLogger } from "./logger/default";
-import { installAiSdkWarningLogger } from "./telemetry/ai-sdk";
-import { DEFAULT_CHAT_TIMEOUT_MS, kParsed, } from "./types";
-export const parseConfig = (config) => {
-    // If it has been parsed before, just return.
-    if (kParsed in config)
-        return config;
-    const providers = config.providers ?? {};
-    const parsedProviders = {};
-    const models = config.models ?? {};
-    const storage = config.storage ?? new InMemoryStorage();
-    // Set the global logger instance.
-    if (config.logger === undefined) {
-        setLoggerInstance(createDefaultLogger({}));
-    }
-    else if (config.logger !== null) {
-        setLoggerInstance(isLogger(config.logger) ? config.logger : createDefaultLogger(config.logger));
-        logger.info(isLogger(config.logger)
-            ? `[logger] custom logger configured`
-            : `[logger] logger configured: level=${config.logger.level}`);
-    }
-    // Strip providers that are not configured.
-    for (const id in providers) {
-        const provider = providers[id];
-        if (provider === undefined) {
-            logger.warn(`[config] ${id} provider removed (undefined)`);
-            continue;
-        }
-        parsedProviders[id] = provider;
-    }
-    if (Object.keys(parsedProviders).length === 0) {
-        throw new Error("No providers configured (config.providers is empty)");
-    }
-    // Strip providers that are not configured from models.
-    const parsedModels = {};
-    const warnings = new Set();
-    for (const id in models) {
-        const model = models[id];
-        const kept = [];
-        for (const p of model.providers) {
-            if (p in parsedProviders)
-                kept.push(p);
-            else
-                warnings.add(p);
-        }
-        if (kept.length > 0)
-            parsedModels[id] = { ...model, providers: kept };
-    }
-    for (const warning of warnings) {
-        logger.warn(`[config] ${warning} provider removed (not configured)`);
-    }
-    if (Object.keys(parsedModels).length === 0) {
-        throw new Error("No models configured (config.models is empty)");
-    }
-    // Default for the telemetry settings.
-    const telemetryEnabled = config.telemetry?.enabled ?? false;
-    const telemetrySignals = telemetryEnabled
-        ? {
-            http: config.telemetry?.signals?.http ?? "recommended",
-            gen_ai: config.telemetry?.signals?.gen_ai ?? "full",
-            hebo: config.telemetry?.signals?.hebo ?? "off",
-        }
-        : {
-            http: "off",
-            gen_ai: "off",
-            hebo: "off",
-        };
-    installAiSdkWarningLogger(telemetrySignals.gen_ai);
-    // Default timeouts
-    let normal;
-    let flex;
-    const t = config.timeouts;
-    if (t === null) {
-        normal = flex = undefined;
-    }
-    else if (typeof t === "number") {
-        normal = t;
-        flex = t * 3;
-    }
-    else {
-        if (t?.normal === null)
-            normal = undefined;
-        else if (t?.normal === undefined)
-            normal = DEFAULT_CHAT_TIMEOUT_MS;
-        else
-            normal = t.normal;
-        if (t?.flex === null)
-            flex = undefined;
-        else if (t?.flex === undefined)
-            flex = normal === undefined ? undefined : normal * 3;
-        else
-            flex = t.flex;
-    }
-    const parsedTimeouts = { normal, flex };
-    // Return parsed config.
-    return {
-        ...config,
-        timeouts: parsedTimeouts,
-        telemetry: {
-            ...config.telemetry,
-            enabled: telemetryEnabled,
-            signals: telemetrySignals,
-        },
-        providers: parsedProviders,
-        models: parsedModels,
-        storage,
-        [kParsed]: true,
-    };
-};

package/dist/endpoints/chat-completions/converters.d.ts

@@ -1,26 +0,0 @@
-import type { SharedV3ProviderMetadata } from "@ai-sdk/provider";
-import type { GenerateTextResult, StreamTextResult, FinishReason, ToolSet, ModelMessage, UserContent, LanguageModelUsage, TextStreamPart, ReasoningOutput, AssistantModelMessage, ToolModelMessage, UserModelMessage } from "ai";
-import { Output } from "ai";
-import type { ChatCompletionsToolCall, ChatCompletionsTool, ChatCompletionsToolChoice, ChatCompletionsStream, ChatCompletionsContentPart, ChatCompletionsMessage, ChatCompletionsUserMessage, ChatCompletionsAssistantMessage, ChatCompletionsToolMessage, ChatCompletionsFinishReason, ChatCompletionsUsage, ChatCompletionsInputs, ChatCompletions, ChatCompletionsChunk, ChatCompletionsReasoningDetail } from "./schema";
-import type { SseErrorFrame, SseFrame } from "../../utils/stream";
-import { type TextCallOptions, type ToolChoiceOptions } from "../shared/converters";
-export declare function convertToTextCallOptions(params: ChatCompletionsInputs): TextCallOptions;
-export declare function convertToModelMessages(messages: ChatCompletionsMessage[]): ModelMessage[];
-export declare function fromChatCompletionsUserMessage(message: ChatCompletionsUserMessage): UserModelMessage;
-export declare function fromChatCompletionsAssistantMessage(message: ChatCompletionsAssistantMessage): AssistantModelMessage;
-export declare function fromChatCompletionsToolResultMessage(message: ChatCompletionsAssistantMessage, toolById: Map<string, ChatCompletionsToolMessage>): ToolModelMessage | undefined;
-export declare function fromChatCompletionsContent(content: ChatCompletionsContentPart[]): UserContent;
-export declare const convertToToolSet: (tools: ChatCompletionsTool[] | undefined) => ToolSet | undefined;
-export declare const convertToToolChoiceOptions: (toolChoice: ChatCompletionsToolChoice | undefined) => ToolChoiceOptions;
-export declare function toChatCompletions(result: GenerateTextResult<ToolSet, Output.Output>, model: string): ChatCompletions;
-export declare function toChatCompletionsResponse(result: GenerateTextResult<ToolSet, Output.Output>, model: string, responseInit?: ResponseInit): Response;
-export declare function toChatCompletionsStream(result: StreamTextResult<ToolSet, Output.Output>, model: string): ChatCompletionsStream;
-export declare function toChatCompletionsStreamResponse(result: StreamTextResult<ToolSet, Output.Output>, model: string, responseInit?: ResponseInit): Response;
-export declare class ChatCompletionsTransformStream extends TransformStream<TextStreamPart<ToolSet>, SseFrame<ChatCompletionsChunk> | SseErrorFrame> {
-    constructor(model: string);
-}
-export declare const toChatCompletionsAssistantMessage: (result: GenerateTextResult<ToolSet, Output.Output>) => ChatCompletionsAssistantMessage;
-export declare function toReasoningDetail(reasoning: ReasoningOutput, id: string, index: number): ChatCompletionsReasoningDetail;
-export declare function toChatCompletionsUsage(usage: LanguageModelUsage): ChatCompletionsUsage;
-export declare function toChatCompletionsToolCall(id: string, name: string, args: unknown, providerMetadata?: SharedV3ProviderMetadata): ChatCompletionsToolCall;
-export declare const toChatCompletionsFinishReason: (finishReason: FinishReason) => ChatCompletionsFinishReason;