flexinference 1.5.1 → 1.5.3

package/CHANGELOG.md

@@ -1,5 +1,18 @@
 # Changelog
 
+## 1.5.3
+
+Relaxes Messages `max_tokens` on the FlexInference side. The SDK no longer requires or
+runtime-validates it before sending a `/messages` request; FlexInference forwards it when
+set and lets Anthropic return its own request error when it is omitted.
+
+## 1.5.2
+
+Adds zero-dependency runtime request validation matching the Python SDK: a request is now
+checked for `model` and the per-endpoint required fields before it is sent, with the two SDKs
+held to identical behavior by shared parity fixtures. The generated `Error` type also gains a
+`doc_url` field, so an error now carries a link to its documentation.
+
 ## 1.5.1
 
 Small robustness fixes for the transport, no API changes. A non-streaming request that hits
@@ -7,8 +20,8 @@ its total `timeout` now throws a clear timeout error instead of a bare abort tha
 a caller cancellation. A streamed response flushes its decoder and delivers the final frame
 even when the stream ends without a trailing blank line, so the last event is never dropped,
 and the streaming buffer is capped so a stream that never sends a separator cannot grow
-without bound. The `start_within` type now matches the runtime rule, so a value the client
-would reject at run time no longer type-checks.
+without bound. The `start_within` type now requires the runtime's two-digit duration
+shape, so values like `0h-5m-0s` no longer type-check.
 
 ## 1.5.0
 
@@ -82,11 +95,11 @@ required and aligns its values with OpenAI's `service_tier` vocabulary.
 
 ### Breaking changes
 
-- **`start_within` is now required on every request.** It is typed as
-  `StartWithin = "default" | "priority" | "auto" | \`${number}h-${number}m-${number}s\``,
-so omitting it (or passing a value outside those shapes) is a **compile-time** error.
-A duration that fits the shape but is malformed or out of range is caught at **runtime**;
-plain-JS callers get every check at **runtime** (a thrown `Error` before any network call).
+- **`start_within` is now required on every request.** It is typed as `"default"`,
+  `"priority"`, `"auto"`, or a duration shaped like `"00h-05m-00s"`, so omitting it
+  (or passing a value outside those shapes) is a **compile-time** error.
+  A duration that fits the shape but is malformed or out of range is caught at **runtime**;
+  plain-JS callers get every check at **runtime** (a thrown `Error` before any network call).
 - **Accepted values changed.** `start_within` now takes `"default"`, `"priority"`,
   `"auto"`, or a duration `"HHh-MMm-SSs"` (5s-10m). The old `"standard"` value is
   removed; use `"default"` (it maps to the same OpenAI service tier).

package/README.md

@@ -34,7 +34,7 @@ FlexInference routes to **OpenAI**, **Google Gemini**, and **Anthropic**. Send t
 
 - **OpenAI:** `default` (the standard tier), `priority`, `auto`, and flex on flex-capable models. You reach flex by passing a duration.
 - **Gemini:** `default` maps to Gemini's standard tier. You also get `priority` and flex on the Gemini flex models (`gemini-3.5-flash`, `gemini-3.1-flash-lite`, `gemini-3.1-pro-preview`, `gemini-3-flash-preview`, `gemini-2.5-pro`, `gemini-2.5-flash`, `gemini-2.5-flash-lite`). Gemini has no `auto` tier, so `start_within: "auto"` on a Gemini model returns `400`.
-- **Anthropic (Claude):** proxy-only. `default`, `priority`, and `auto` work; there is **no flex tier**, so a duration `start_within` on a `claude-*` model returns `400 flex_unsupported_for_anthropic`. Anthropic requires a token cap, so set `max_output_tokens` (`max_completion_tokens` on Chat, `max_tokens` on Messages) or you get `400 missing_max_tokens`. You keep the unified API and tier control, and draw down your own Anthropic credits.
+- **Anthropic (Claude):** proxy-only. `default`, `priority`, and `auto` work; there is **no flex tier**, so a duration `start_within` on a `claude-*` model returns `400 flex_unsupported_for_anthropic`. Anthropic requires an output-token field upstream. FlexInference forwards `max_output_tokens` (`max_completion_tokens` on Chat, `max_tokens` on Messages) when set and does not synthesize a default, so omitting it returns Anthropic's own request error. You keep the unified API and tier control, and draw down your own Anthropic credits.
 
 Add the provider key you'll use (OpenAI, Gemini, and/or Anthropic) in the [dashboard](https://www.flexinference.com/dashboard). Text, streaming, structured outputs, function calling, image input, and web search work across providers (send a Responses `web_search` tool; we map it to Gemini's `google_search`).
 
@@ -85,7 +85,7 @@ console.log(interactionText(res));
 
 ## Messages (Anthropic shape)
 
-Speak Anthropic's Messages shape and reach any model. `max_tokens` is required (Anthropic requires it). `messageText(res)` pulls the assistant text out of the message `content`.
+Speak Anthropic's Messages shape and reach any model. Anthropic requires `max_tokens` upstream. FlexInference forwards it when set and does not synthesize a default, so omitting it returns Anthropic's own request error. `messageText(res)` pulls the assistant text out of the message `content`.
 
 ```ts
 import { messageText } from "flexinference";
@@ -104,7 +104,7 @@ console.log(messageText(res));
 
 Streaming and non-streaming requests are timed differently, so a long generation is never cut off yet a hung request never hangs forever:
 
-- **Non-streaming** requests have a total wall-clock budget: `timeout`, default **600000 ms (10 min)**, from send through the full body.
+- **Non-streaming** requests have a body-read budget after response headers arrive: `timeout`, default **600000 ms (10 min)**. The worst case is roughly `firstByteTimeout + timeout`.
 - **Streaming** requests have **no total cap** (a healthy stream runs as long as tokens keep arriving). They are bounded by two clocks: `firstByteTimeout` (wait for the first response, default **60000 ms**) and `idleTimeout` (max silence between chunks before the stream is treated as hung, default **60000 ms**).
 
 The first-byte wait is **auto-raised for a flex `start_within`** (the router withholds response headers until the flex race resolves), so a long deadline just works. Raise `firstByteTimeout` yourself only for very large non-flex contexts whose first token is slow.

package/dist/index.d.ts

@@ -5,7 +5,12 @@ type Schemas = components["schemas"];
  * those OpenAI service tiers and proxy any model; a duration "HHh-MMm-SSs" (5s-10m)
  * runs the flex race on a flex-capable model. An arbitrary string needs `as StartWithin`.
  */
-export type StartWithin = "default" | "priority" | "auto" | `${number}h-${number}m-${number}s`;
+type Digit = "0" | "1" | "2" | "3" | "4" | "5" | "6" | "7" | "8" | "9";
+type Second00To59 = "00" | "01" | "02" | "03" | "04" | "05" | "06" | "07" | "08" | "09" | `1${Digit}` | `2${Digit}` | `3${Digit}` | `4${Digit}` | `5${Digit}`;
+type Second05To59 = Exclude<Second00To59, "00" | "01" | "02" | "03" | "04">;
+type Minute01To09 = `0${Exclude<Digit, "0">}`;
+type Duration = `00h-00m-${Second05To59}s` | `00h-${Minute01To09}m-${Second00To59}s` | "00h-10m-00s";
+export type StartWithin = "default" | "priority" | "auto" | Duration;
 /** Make `start_within` a required, typed field on a generated request body. */
 type WithStartWithin<T> = Omit<T, "start_within"> & {
     start_within: StartWithin;
@@ -78,6 +83,7 @@ export declare class FlexInferenceError extends Error {
  * FlexInferenceError)` handlers keep working; narrow to this to handle 402 specially.
  */
 export declare class PaymentRequiredError extends FlexInferenceError {
+    constructor(status: number, body: unknown, fallback: string);
 }
 export interface ClientOptions {
     apiKey: string;
@@ -86,9 +92,10 @@ export interface ClientOptions {
     /** Override the global fetch (e.g. for tests or non-standard runtimes). */
     fetch?: typeof fetch;
     /**
-     * Non-streaming wall-clock budget in ms (from send through the full body). Defaults to
-     * 600000 (10 min). Streaming requests ignore this: they are governed by `firstByteTimeout`
-     * and `idleTimeout` and have no total cap, so a healthy long stream is never cut off.
+     * Non-streaming body-read budget in ms, after response headers arrive. Defaults to
+     * 600000 (10 min). The worst case is roughly `firstByteTimeout + timeout`.
+     * Streaming requests ignore this: they are governed by `firstByteTimeout` and
+     * `idleTimeout` and have no total cap, so a healthy long stream is never cut off.
      */
     timeout?: number;
     /**

package/dist/index.js

@@ -120,6 +120,10 @@ export class FlexInferenceError extends Error {
  * FlexInferenceError)` handlers keep working; narrow to this to handle 402 specially.
  */
 export class PaymentRequiredError extends FlexInferenceError {
+    constructor(status, body, fallback) {
+        super(status, body, fallback);
+        this.name = "PaymentRequiredError";
+    }
 }
 const DEFAULT_BASE_URL = "https://api.flexinference.com/v1";
 // A streaming request is governed by two clocks and no total: how long we wait for the
@@ -134,43 +138,102 @@ const DEFAULT_TOTAL_MS = 600_000; // non-streaming wall-clock budget (streaming
 const ERROR_BODY_READ_MS = 30_000; // bound the read of a non-2xx error body
 // Cap the streaming buffer above the largest legitimate single frame (a terminal
 // response object is well under this). A stream that never sends a blank-line separator
-// would otherwise grow the buffer without bound; error clearly instead. See T2.
+// would otherwise grow the buffer without bound; error clearly instead.
 const MAX_SSE_BUFFER_CHARS = 8 * 1024 * 1024;
 const MIN_DEADLINE_SECONDS = 5;
 const MAX_DEADLINE_SECONDS = 600;
 const SECONDS_PER_HOUR = 3600;
 const SECONDS_PER_MINUTE = 60;
-// One or two digits per field so the runtime accepts the same shapes the StartWithin
-// template-literal type does (e.g. "0h-5m-0s"): `${number}` in the type imposes no digit
-// count, so a two-digit-only regex would reject values that type-check. See T4.
-const DURATION_RE = /^(\d{1,2})h-(\d{1,2})m-(\d{1,2})s$/;
+// Two digits per field so the runtime matches the router, Python SDK, and StartWithin type.
+const DURATION_RE = /^(\d{2})h-(\d{2})m-(\d{2})s$/;
 const START_WITHIN_VALUES = '"default", "priority", "auto", or a duration "HHh-MMm-SSs"';
+function startWithinValue(body) {
+    const value = typeof body === "object" && body !== null
+        ? body["start_within"]
+        : undefined;
+    return value === undefined ? { kind: "missing" } : { kind: "present", value };
+}
+function parseStartWithin(value) {
+    if (typeof value !== "string")
+        return { kind: "invalid", reason: "type", value };
+    if (value === "default" || value === "priority" || value === "auto") {
+        return { kind: "tier", value };
+    }
+    const match = DURATION_RE.exec(value);
+    if (match === null)
+        return { kind: "invalid", reason: "malformed", value };
+    const seconds = Number(match[1]) * SECONDS_PER_HOUR + Number(match[2]) * SECONDS_PER_MINUTE + Number(match[3]);
+    return { kind: "duration", value, seconds };
+}
 // Runtime guard so plain-JS callers (who get no compile-time check) still fail
 // locally with a clear message instead of a provider round-trip. TypeScript callers
 // are already protected by the required StartWithin field above.
 function assertStartWithin(body) {
-    const value = typeof body === "object" && body !== null
-        ? body["start_within"]
-        : undefined;
-    if (value === undefined) {
+    const sw = startWithinValue(body);
+    if (sw.kind === "missing") {
         throw new Error(`FlexInference: \`start_within\` is required on every request. Set it to ${START_WITHIN_VALUES}, ` +
             `for example "start_within": "default" for standard-tier behavior or "00h-00m-30s" for a 30-second flex race.`);
     }
-    if (typeof value !== "string") {
+    const parsed = parseStartWithin(sw.value);
+    if (parsed.kind === "invalid" && parsed.reason === "type") {
         throw new Error(`FlexInference: \`start_within\` must be a string (${START_WITHIN_VALUES}), for example "default".`);
     }
-    if (value === "default" || value === "priority" || value === "auto")
-        return;
-    const match = DURATION_RE.exec(value);
-    if (match === null) {
-        throw new Error(`FlexInference: \`start_within\` "${value}" is not ${START_WITHIN_VALUES}. ` +
-            `A duration is one or two digits each for hours, minutes, and seconds, for example "00h-05m-00s" for a 5-minute deadline.`);
+    if (parsed.kind === "invalid") {
+        throw new Error(`FlexInference: \`start_within\` "${parsed.value}" is not ${START_WITHIN_VALUES}. ` +
+            `A duration is two digits each for hours, minutes, and seconds, for example "00h-05m-00s" for a 5-minute deadline.`);
     }
-    const seconds = Number(match[1]) * SECONDS_PER_HOUR + Number(match[2]) * SECONDS_PER_MINUTE + Number(match[3]);
-    if (seconds < MIN_DEADLINE_SECONDS || seconds > MAX_DEADLINE_SECONDS) {
+    if (parsed.kind === "duration" &&
+        (parsed.seconds < MIN_DEADLINE_SECONDS || parsed.seconds > MAX_DEADLINE_SECONDS)) {
         throw new Error("FlexInference: `start_within` duration must be between 5s and 10m.");
     }
 }
+// Per-endpoint required fields beyond the model/start_within envelope. Mirrors the Python
+// _RequestSpec table in _validation.py: responses/interactions require `input` (presence only);
+// chat/messages require `messages` (array).
+const RESPONSES_REQUEST_SPEC = {
+    required: [{ name: "input", kind: "present" }],
+};
+const CHAT_REQUEST_SPEC = {
+    required: [{ name: "messages", kind: "array" }],
+};
+const INTERACTIONS_REQUEST_SPEC = {
+    required: [{ name: "input", kind: "present" }],
+};
+const MESSAGES_REQUEST_SPEC = {
+    required: [{ name: "messages", kind: "array" }],
+};
+// Read one field by reference off the raw body without copying it. `undefined` (or a non-object
+// body) is treated as missing; `null` counts as present, matching Python's `name in body`
+// presence check (a JSON `null` is a present key, and JSON serialization drops `undefined`).
+function requestField(body, name) {
+    if (!isRecord(body))
+        return { kind: "missing" };
+    const value = body[name];
+    return value === undefined ? { kind: "missing" } : { kind: "present", value };
+}
+// Pre-flight guard: model, then start_within, then the endpoint's required fields in spec order.
+// Pure - it only throws earlier for requests the router/Python already reject; it never mutates,
+// clones, or reshapes the body, so unknown/extra fields pass straight through to `send`.
+function validateRequest(spec, body) {
+    const model = requestField(body, "model");
+    if (model.kind === "missing" || typeof model.value !== "string" || model.value === "") {
+        throw new Error("FlexInference: `model` is required and must be a non-empty string.");
+    }
+    assertStartWithin(body);
+    for (const field of spec.required) {
+        const value = requestField(body, field.name);
+        if (value.kind === "missing") {
+            throw new Error(`FlexInference: \`${field.name}\` is required for this endpoint.`);
+        }
+        if (field.kind === "array" && !Array.isArray(value.value)) {
+            throw new Error(`FlexInference: \`${field.name}\` must be an array.`);
+        }
+        if (field.kind === "finiteNumber" &&
+            (typeof value.value !== "number" || !Number.isFinite(value.value))) {
+            throw new Error(`FlexInference: \`${field.name}\` must be a finite number.`);
+        }
+    }
+}
 // Distinct abort reasons so an aborted fetch/read can be told apart: a timeout we fired
 // (turned into a clear message) versus the caller's own AbortSignal (propagated as-is).
 const FIRST_BYTE_TIMEOUT = Symbol("first-byte-timeout");
@@ -183,15 +246,11 @@ const TOTAL_TIMEOUT = Symbol("total-timeout");
  * clock must cover the whole deadline plus the fallback path.
  */
 function flexDeadlineSeconds(body) {
-    const value = typeof body === "object" && body !== null
-        ? body["start_within"]
-        : undefined;
-    if (typeof value !== "string")
+    const sw = startWithinValue(body);
+    if (sw.kind === "missing")
         return null;
-    const match = DURATION_RE.exec(value);
-    if (match === null)
-        return null;
-    return (Number(match[1]) * SECONDS_PER_HOUR + Number(match[2]) * SECONDS_PER_MINUTE + Number(match[3]));
+    const parsed = parseStartWithin(sw.value);
+    return parsed.kind === "duration" ? parsed.seconds : null;
 }
 // Read a non-streamed JSON body under the total wall-clock budget, then release the timer
 // and the abort listener. Non-streaming only: a stream is drained by streamSSE instead.
@@ -199,7 +258,7 @@ async function readJson(open) {
     const { res, controller, cleanup, totalMs } = open;
     // Abort with a dedicated reason so a fired total timeout can be told apart from the
     // caller cancelling on purpose, and translated into a clear message (parity with the
-    // first-byte and idle timeouts, and with the Python SDK). See T1.
+    // first-byte and idle timeouts, and with the Python SDK).
     const totalTimer = setTimeout(() => {
         controller.abort(TOTAL_TIMEOUT);
     }, totalMs);
@@ -219,6 +278,13 @@ async function readJson(open) {
         cleanup();
     }
 }
+async function createEndpoint(send, path, spec, body, options) {
+    validateRequest(spec, body);
+    const open = await send(path, body, options);
+    if (body.stream)
+        return streamSSE(open);
+    return readJson(open);
+}
 export class FlexInference {
     responses;
     chat;
@@ -345,11 +411,7 @@ class Responses {
         this.send = send;
     }
     async create(body, options) {
-        assertStartWithin(body);
-        const open = await this.send("/responses", body, options);
-        if (body.stream)
-            return streamSSE(open);
-        return readJson(open);
+        return createEndpoint(this.send, "/responses", RESPONSES_REQUEST_SPEC, body, options);
     }
 }
 class Chat {
@@ -364,11 +426,7 @@ class ChatCompletions {
         this.send = send;
     }
     async create(body, options) {
-        assertStartWithin(body);
-        const open = await this.send("/chat/completions", body, options);
-        if (body.stream)
-            return streamSSE(open);
-        return readJson(open);
+        return createEndpoint(this.send, "/chat/completions", CHAT_REQUEST_SPEC, body, options);
     }
 }
 class Interactions {
@@ -377,11 +435,7 @@ class Interactions {
         this.send = send;
     }
     async create(body, options) {
-        assertStartWithin(body);
-        const open = await this.send("/interactions", body, options);
-        if (body.stream)
-            return streamSSE(open);
-        return readJson(open);
+        return createEndpoint(this.send, "/interactions", INTERACTIONS_REQUEST_SPEC, body, options);
     }
 }
 class Messages {
@@ -390,11 +444,7 @@ class Messages {
         this.send = send;
     }
     async create(body, options) {
-        assertStartWithin(body);
-        const open = await this.send("/messages", body, options);
-        if (body.stream)
-            return streamSSE(open);
-        return readJson(open);
+        return createEndpoint(this.send, "/messages", MESSAGES_REQUEST_SPEC, body, options);
     }
 }
 function parseSSEFrame(frame) {
@@ -445,7 +495,7 @@ async function* streamSSE(open) {
             if (done) {
                 // The stream ended. Flush any bytes the decoder held back for an incomplete
                 // multibyte sequence, then emit a final frame that arrived without its trailing
-                // blank line so the last piece of the answer is not dropped. See T3.
+                // blank line so the last piece of the answer is not dropped.
                 exhausted = true;
                 buffer += decoder.decode();
                 const frame = buffer;
@@ -473,7 +523,7 @@ async function* streamSSE(open) {
             }
             // With every complete frame drained, whatever remains is a single unterminated
             // frame. If it grows past the cap the stream is never sending a separator; error
-            // clearly instead of buffering without bound. See T2.
+            // clearly instead of buffering without bound.
             if (buffer.length > MAX_SSE_BUFFER_CHARS) {
                 throw new Error(`FlexInference: stream buffer exceeded ${String(MAX_SSE_BUFFER_CHARS)} characters ` +
                     "without a frame boundary; aborting to avoid unbounded memory.");

package/dist/types.d.ts

@@ -90,7 +90,7 @@ export interface paths {
         };
         get?: never;
         put?: never;
-        /** Create a message. The fourth FlexInference caller format (Anthropic Messages shape in and out); works for any model via the canonical Responses path. Send start_within and do not send service_tier. max_tokens is required. */
+        /** Create a message. The fourth FlexInference caller format (Anthropic Messages shape in and out); works for any model via the canonical Responses path. Send start_within and do not send service_tier. Anthropic requires max_tokens upstream. FlexInference forwards it when set and does not synthesize a default, so omitting it returns Anthropic's own request error. */
         post: operations["createMessage"];
         delete?: never;
         options?: never;
@@ -5398,6 +5398,8 @@ export interface components {
                 type: string;
                 code?: string | null;
                 param?: string | null;
+                /** @description Additive deep link into the docs for this exact code. Present (a URL or null) on FlexInference-layer errors; absent on verbatim upstream errors. */
+                doc_url?: string | null;
             };
         };
         /** Interaction Text Part */
@@ -5792,11 +5794,11 @@ export interface components {
         };
         /**
          * Message Create Params
-         * @description Request body for POST /v1/messages (Anthropic Messages shape). Works for any model via the canonical Responses path. max_tokens is required (Anthropic requires it and the router rejects a claude-* request that reaches the Anthropic upstream without a canonical max_output_tokens). Does not accept service_tier (the router owns the tier). start_within is required at runtime but left optional here.
+         * @description Request body for POST /v1/messages (Anthropic Messages shape). Works for any model via the canonical Responses path. Anthropic requires max_tokens upstream. FlexInference forwards it when set and does not synthesize a default, so omitting it returns Anthropic's own request error. Does not accept service_tier (the router owns the tier). start_within is required at runtime but left optional here.
          */
         MessageCreateParams: {
             model: string;
-            max_tokens: number;
+            max_tokens?: number;
             /** @description System prompt as a string or an array of text blocks. */
             system?: string | components["schemas"]["MessageTextBlock"][];
             messages: components["schemas"]["Message"][];
@@ -6134,7 +6136,7 @@ export interface operations {
                     "text/event-stream": components["schemas"]["MessageStreamEvent"];
                 };
             };
-            /** @description Bad request. A FlexInference-layer rejection (type flexinference_error, e.g. missing_start_within, invalid_start_within, model_not_flex_capable, flex_unsupported_for_anthropic, missing_max_tokens, service_tier_not_allowed) or an upstream provider rejection passed through unchanged. */
+            /** @description Bad request. A FlexInference-layer rejection (type flexinference_error, e.g. missing_start_within, invalid_start_within, model_not_flex_capable, flex_unsupported_for_anthropic, service_tier_not_allowed) or an upstream provider rejection passed through unchanged. */
             400: {
                 headers: {
                     [name: string]: unknown;

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "flexinference",
-  "version": "1.5.1",
+  "version": "1.5.3",
   "description": "Official TypeScript SDK for FlexInference - a deadline-aware, OpenAI-compatible inference router.",
   "license": "MIT",
   "author": "Aditya Perswal <adityaperswal@gmail.com>",
@@ -34,7 +34,7 @@
     "build": "tsc -p tsconfig.json",
     "typecheck": "tsc -p tsconfig.json --noEmit",
     "test:types": "tsc -p tsconfig.test.json --noEmit",
-    "test": "node test/smoke.mjs && node test/fixes.test.mjs && npm run test:types",
+    "test": "node test/smoke.mjs && node test/fixes.test.mjs && node test/parity.test.mjs && npm run test:types",
     "lint": "eslint .",
     "lint:fix": "eslint . --fix",
     "format": "prettier --write .",