zengate 1.0.4 → 1.1.0

package/README.md

@@ -36,7 +36,8 @@ OpenCode's free Zen models only work from inside OpenCode. So this gateway does
 Trade-offs to know about:
 
 - **Prompt overhead.** Each request carries OpenCode's system prompt and tool list (roughly 6–9k input tokens, largely cache hits), and adds about 1–3 s of latency.
-- **Sampling parameters are ignored.** Settings like `temperature` and `max_tokens` cannot be forwarded through OpenCode. The gateway accepts them and lists them in an `x-gateway-ignored-params` response header.
+- **Sampling parameters are ignored.** Settings like `temperature` and `top_p` cannot be forwarded through OpenCode. The gateway accepts them and lists them in an `x-gateway-ignored-params` response header.
+- **`max_tokens` is approximate.** OpenCode has no output limit either, so the gateway counts about four characters as a token, cuts the answer there, stops the model and reports `finish_reason: "length"`. As in OpenAI's API, `max_completion_tokens` and `max_output_tokens` include reasoning text and `max_tokens` covers only the answer. Usage for a cut answer is estimated, because OpenCode reports none for a stopped turn.
 - **Upstream terms apply.** Availability, rate limits and the model list are set by OpenCode Zen and can change at any time. Free models may have their own data policies; see the [Zen docs](https://opencode.ai/docs/zen/).
 
 > **Fair use.** zengate talks to Zen only through the official OpenCode CLI and never bypasses its limits or free-tier checks. You are responsible for following OpenCode's terms and fair-use expectations. Run it for yourself or your team, not as a public or resold service.
@@ -78,6 +79,13 @@ services:
       - "127.0.0.1:8083:8083"
     volumes:
       - zengate:/data
+    # Longer than SHUTDOWN_TIMEOUT_MS plus 5s, so in-flight requests can finish.
+    stop_grace_period: 20s
+    cap_drop: [ALL]
+    security_opt: ["no-new-privileges:true"]
+    # OpenCode writes only to /tmp and the gateway only to /data.
+    read_only: true
+    tmpfs: [/tmp]
 volumes:
   zengate:
 ```
@@ -182,10 +190,12 @@ curl http://127.0.0.1:8083/v1/chat/completions \
 |---|---|
 | `POST /v1/chat/completions` | Streaming (SSE, `stream_options.include_usage`), `n` 1–4, `stop`, `tools` / `tool_choice` / `parallel_tool_calls`, legacy `functions`, `response_format` (`json_object`, `json_schema`), `reasoning_effort`, `developer` / `system` / `tool` roles, `reasoning_content` in responses |
 | `POST /v1/responses` | Streaming with the standard event sequence, `instructions`, `previous_response_id`, `item_reference`, `function` and `custom` tools, `text.format`, `reasoning.effort` / `reasoning` summary items, `store` |
-| `GET` / `DELETE /v1/responses/{id}` | Stored in memory for 1 hour, up to `RESPONSES_STORE_MAX` entries |
+| `GET` / `DELETE /v1/responses/{id}` | Stored in memory for 1 hour, within `RESPONSES_STORE_MAX` entries and `RESPONSES_STORE_MB` |
 | `GET /v1/models`, `GET /v1/models/{id}` | Live list from OpenCode |
-| `GET /health`, `GET /ready` | Public liveness and readiness probes |
-| `GET /metrics` | Request counters and slot usage, as JSON (needs the key) |
+| `GET /health`, `GET /ready` | Public liveness and readiness probes (`/ready` answers `503` while starting or shutting down) |
+| `POST /v1/completions` | Legacy text completions: `prompt` as a string or list of strings, `n`, `stop`, `max_tokens`, `echo`, streaming. The model is asked to continue the text, so it behaves like a chat model, not a base model. Without `max_tokens` the length is not limited (OpenAI defaults to 16) |
+| `POST /v1/embeddings` | `404 unsupported_endpoint`: Zen serves chat models only |
+| `GET /metrics` | Request counters, latency and time-to-first-token, slot usage and the OpenCode version (needs the key). JSON by default; Prometheus text format for `?format=prometheus` or an `Accept: text/plain` scrape |
 
 Routes also work without the `/v1` prefix. Errors use the standard OpenAI envelope, `{"error": {"message", "type", "param", "code"}}`, with matching HTTP status codes (400, 401, 404, 413, 429 with `Retry-After`, 502, 503, 504).
 
@@ -201,9 +211,11 @@ Routes also work without the `/v1` prefix. Errors use the standard OpenAI envelo
 
 Text files (plain text, markdown, JSON, CSV and so on) are inlined as text, so every model can read them.
 
+The gateway downloads `https` attachment URLs itself (at most 16 in one request, 30 seconds each, up to 3 redirects) and passes the content on, so remote files obey `MAX_MEDIA_MB` and `MAX_BODY_MB` like inline ones. The server's `Content-Type` must match the part: an `image_url` must return an image. This direct connection does not use `HTTPS_PROXY`.
+
 **Function calling** is emulated. OpenCode sessions cannot register your functions natively, so their schemas go into the prompt and the model's calls come back as standard `tool_calls` or `function_call` items. It works reliably with `big-pickle`. If a model tries to call a function the wrong way, or ignores `tool_choice: "required"`, the gateway retries once with a correction.
 
-**Not supported** (clear 400 error): `logprobs`, audio output, `file_id` references (there is no Files API), `background` responses, the Conversations API and stored prompts. Hosted tools such as `web_search` in Responses are ignored and listed in `x-gateway-ignored-params`.
+**Not supported** (clear 400 error): `logprobs`, `suffix`, audio output, `file_id` references (there is no Files API), `background` responses, the Conversations API and stored prompts. Hosted tools such as `web_search` in Responses are ignored and listed in `x-gateway-ignored-params`.
 
 ## Configuration
 
@@ -212,13 +224,16 @@ Set values as environment variables or in the config file (same names; see [`con
 | Setting | Default | Meaning |
 |---|---|---|
 | `API_KEY` / `API_KEYS` | generated | Gateway key(s), at least 16 characters. `API_KEYS` takes a list for rotation. |
-| `ALLOW_NO_AUTH` | `false` | Serve without any key (explicit opt-out). |
+| `ALLOW_NO_AUTH` | `false` | Serve without any key (explicit opt-out). Requests must then use an IP address, `localhost` or a name in `ALLOWED_HOSTS` as their Host, which stops web pages from reaching the gateway through DNS rebinding. |
+| `ALLOWED_HOSTS` | none | Extra Host names accepted while `ALLOW_NO_AUTH` is on, such as the name a reverse proxy forwards. |
 | `HOST` / `PORT` | `127.0.0.1` / `8083` | Listen address. |
 | `MAX_CONCURRENT` / `MAX_QUEUE` | `8` / `32` | Parallel generations, and how many requests may wait (then `429`). |
+| `QUEUE_TIMEOUT_MS` | `30000` | How long a request may wait for a slot (then `429`). Keep it below your proxy's response timeout. |
 | `RATE_LIMIT_PER_MINUTE` | `120` | Per client IP (`0` turns it off). |
-| `REQUEST_TIMEOUT_MS` | `300000` | Per generation (then `504`). |
+| `REQUEST_TIMEOUT_MS` | `300000` | Per generation, counted from when it gets a slot (then `504`). |
+| `SHUTDOWN_TIMEOUT_MS` | `10000` | How long a shutdown waits for in-flight requests. Give Docker or systemd at least 5 seconds more. |
 | `MAX_BODY_MB` / `MAX_MEDIA_MB` | `25` / `20` | Request body limit, and the limit per attachment. |
-| `RESPONSES_STORE_MAX` | `500` | Stored responses for `previous_response_id` (`0` turns storage off). |
+| `RESPONSES_STORE_MAX` / `RESPONSES_STORE_MB` | `500` / `256` | Stored responses for `previous_response_id`, by count and approximate memory (`RESPONSES_STORE_MAX=0` turns storage off). |
 | `CORS_ORIGINS` | none | Browser origins allowed to call the API (explicit list; `*` is refused). |
 | `TRUST_PROXY` | `0` | Number of reverse-proxy hops to trust for client IPs. |
 | `LOG_LEVEL` / `LOG_JSON` | `info` / `false` | Log verbosity, and JSON-lines output. |
@@ -229,13 +244,13 @@ Set values as environment variables or in the config file (same names; see [`con
 | `ALLOW_INSECURE_BACKEND_HTTP` | `false` | Allow a non-loopback `http://` server URL. |
 | `CONFIG_FILE` | see [First start](#first-start) | Where the config file lives (environment variable only). |
 
-**Attach mode.** `OPENCODE_SERVER_URL` uses a server you run yourself. The gateway can only reject tool calls that server *asks* permission for; that happens in its own sessions and their subagents. It trusts the server's own permission config, plugins and MCP servers. If that config allows tools without asking, they will run. Use it only with a server configured like the managed one (every permission set to `"ask"`). The managed default is safer. Remote servers must use `https://` unless you set `ALLOW_INSECURE_BACKEND_HTTP`.
+**Attach mode.** `OPENCODE_SERVER_URL` uses a server you run yourself. The gateway can only reject tool calls that server *asks* permission for; that happens in its own sessions and their subagents. It trusts the server's own permission config, plugins and MCP servers. If that config allows tools without asking, they will run. Use it only with a server configured like the managed one: every permission set to `"ask"`, and `experimental.continue_loop_on_deny` set to `true` so a rejected tool does not end the turn with an empty reply. The managed default is safer. Remote servers must use `https://` unless you set `ALLOW_INSECURE_BACKEND_HTTP`.
 
 ## Deployment
 
-**Docker.** The key persists in the `/data` volume. You can pass `-e API_KEY=...` instead. Publish the port on `127.0.0.1`, or put a TLS reverse proxy in front before exposing it to a network.
+**Docker.** The key persists in the `/data` volume. You can pass `-e API_KEY=...` instead. Publish the port on `127.0.0.1`, or put a TLS reverse proxy in front before exposing it to a network. `docker stop` waits only 10 seconds before killing the container, so pass `--stop-timeout 20` (or `stop_grace_period` in Compose) to let in-flight requests finish. The container also runs with a read-only root (`--read-only --tmpfs /tmp`), `--cap-drop ALL` and `--security-opt no-new-privileges:true`. On shutdown, `/ready` answers `503` for up to 2 seconds before the listener closes, so a load balancer polling it stops sending traffic first.
 
-**systemd.** See [`deploy/zengate.service`](deploy/zengate.service). It runs as an unprivileged user with a hardened sandbox, and the key goes in `/var/lib/zengate/config.json`.
+**systemd.** See [`deploy/zengate.service`](deploy/zengate.service). It runs as an unprivileged user with a hardened sandbox and a system-call filter (`@system-service`), and the key goes in `/var/lib/zengate/config.json`.
 
 **Behind a reverse proxy.** Set `TRUST_PROXY=1` (or however many hops you have) so rate limits apply per real client. Disable response buffering for streaming. The gateway already sends `X-Accel-Buffering: no` for nginx.
 
@@ -263,8 +278,9 @@ sha256sum -c checksums.txt
 - The API key is required unless you set `ALLOW_NO_AUTH`. Keys are compared in constant time, the config file is written with mode `0600`, and keys are never logged.
 - The gateway binds to loopback by default and warns when it listens elsewhere, because traffic is plain HTTP, so put TLS in front.
 - OpenCode tools are never executed, and OpenCode runs isolated from your home directory and configuration.
-- Attachment URLs must be `https` and resolve to public addresses. Loopback, private, link-local and similar ranges are refused, which blocks SSRF into your network.
-- Rate limiting, a bounded queue, body and attachment size limits, and per-request timeouts all apply. A request with `n` choices uses `n` concurrency slots, and slots are always released on disconnect or timeout.
+- Attachment URLs must be `https` and resolve to public addresses. Loopback, private, link-local and similar ranges (including IPv6 forms that embed them) are refused, which blocks SSRF into your network. The gateway fetches them itself and connects only to the address it checked, including on every redirect, so DNS rebinding cannot reach an internal host. OpenCode never sees the URL.
+- Conversation history is sent to the model as a tagged transcript. Message text, file names and tool names are escaped so they cannot fake another turn, a tool result or a function call.
+- Rate limiting, a bounded queue with a wait limit, body and attachment size limits, and per-request timeouts all apply. Requests that are uploading, queued or running are capped at `MAX_CONCURRENT + MAX_QUEUE`, which bounds the memory held by request bodies, and one client address may be uploading at most a quarter of that at once. Attachment URLs appear in errors and logs without their query string or credentials. A request with `n` choices uses `n` concurrency slots, and slots are always released on disconnect or timeout.
 - Stored responses (`previous_response_id`, `GET /v1/responses/{id}`) are visible only to the API key that created them.
 - The gateway sends no telemetry and never logs request bodies. OpenCode's auto-update and session sharing are disabled.
 
@@ -277,7 +293,8 @@ To report a vulnerability, see [SECURITY.md](SECURITY.md).
 | `401 invalid_api_key` | Send `Authorization: Bearer <key from the config file>`. |
 | `404 model_not_found` | Use an id from `GET /v1/models`. The free model list changes over time. |
 | `400 unsupported_modality` | That model can't take this input kind. Pick one that does. |
-| `429 server_busy` / `rate_limit_exceeded` | Raise `MAX_CONCURRENT` / `MAX_QUEUE` / `RATE_LIMIT_PER_MINUTE`, or slow down. |
+| `429 server_busy` / `rate_limit_exceeded` | Raise `MAX_CONCURRENT` / `MAX_QUEUE` / `QUEUE_TIMEOUT_MS` / `RATE_LIMIT_PER_MINUTE`, or slow down. |
+| `400 invalid_attachment_url` | The attachment URL is not public, failed to download, or returned the wrong type. Send it as a base64 data URI instead. |
 | `429 upstream_rate_limited`, `502 upstream_*` | OpenCode Zen is limiting or failing. Retry later or try another model. |
 | `503 backend_unavailable` | OpenCode is still starting or restarting. Check `GET /ready`. Run with `LOG_LEVEL=debug` to see OpenCode's own logs. |
 | Startup: `Port ... in use` | Another process has the port. Set `PORT`. |

package/config.json.example

@@ -7,9 +7,12 @@
     "MAX_QUEUE": 32,
     "RATE_LIMIT_PER_MINUTE": 120,
     "REQUEST_TIMEOUT_MS": 300000,
+    "QUEUE_TIMEOUT_MS": 30000,
+    "SHUTDOWN_TIMEOUT_MS": 10000,
     "MAX_BODY_MB": 25,
     "MAX_MEDIA_MB": 20,
     "RESPONSES_STORE_MAX": 500,
+    "RESPONSES_STORE_MB": 256,
     "CORS_ORIGINS": [],
     "TRUST_PROXY": 0,
     "LOG_LEVEL": "info",

package/index.js

@@ -1,4 +1,5 @@
 #!/usr/bin/env node
+import './src/node-version.js';
 import fs from 'node:fs';
 import path from 'node:path';
 import { fileURLToPath } from 'node:url';

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "zengate",
-  "version": "1.0.4",
+  "version": "1.1.0",
   "description": "Keyless OpenAI-compatible API (Chat Completions + Responses) for OpenCode's free Zen models, served through the real OpenCode CLI.",
   "author": {
     "name": "developingchet",
@@ -52,7 +52,7 @@
   "license": "MIT",
   "dependencies": {
     "express": "^5.1.0",
-    "opencode-ai": "^1.18.32"
+    "opencode-ai": "1.18.32"
   },
   "overrides": {
     "es-define-property": "npm:@socketregistry/es-define-property@^1",

package/src/bootstrap.js

@@ -13,7 +13,10 @@ export function generateApiKey() {
 export function writeConfigFile(filePath, data) {
     fs.mkdirSync(path.dirname(filePath), { recursive: true, mode: 0o700 });
     const temp = `${filePath}.${process.pid}.tmp`;
-    fs.writeFileSync(temp, `${JSON.stringify(data, null, 4)}\n`, { mode: 0o600 });
+    // Never write through a file or symlink already at the temp path: unlink
+    // removes a link itself, and 'wx' creates the file fresh with mode 0600.
+    fs.rmSync(temp, { force: true });
+    fs.writeFileSync(temp, `${JSON.stringify(data, null, 4)}\n`, { mode: 0o600, flag: 'wx' });
     fs.renameSync(temp, filePath);
 }
 

package/src/config.js

@@ -21,10 +21,14 @@ const SPEC = Object.freeze({
     MAX_QUEUE: { type: 'int', default: 32, min: 0, max: 1000 },
     RATE_LIMIT_PER_MINUTE: { type: 'int', default: 120, min: 0, max: 100000 },
     REQUEST_TIMEOUT_MS: { type: 'int', default: 300000, min: 10000, max: 3600000 },
+    QUEUE_TIMEOUT_MS: { type: 'int', default: 30000, min: 1000, max: 3600000 },
+    SHUTDOWN_TIMEOUT_MS: { type: 'int', default: 10000, min: 1000, max: 3600000 },
     MAX_BODY_MB: { type: 'int', default: 25, min: 1, max: 200 },
     MAX_MEDIA_MB: { type: 'int', default: 20, min: 1, max: 100 },
     RESPONSES_STORE_MAX: { type: 'int', default: 500, min: 0, max: 100000 },
+    RESPONSES_STORE_MB: { type: 'int', default: 256, min: 1, max: 65536 },
     CORS_ORIGINS: { type: 'list', default: [] },
+    ALLOWED_HOSTS: { type: 'list', default: [] },
     TRUST_PROXY: { type: 'hops', default: 0 },
     LOG_LEVEL: { type: 'enum', default: 'info', values: ['debug', 'info', 'warn', 'error'] },
     LOG_JSON: { type: 'bool', default: false },

package/src/gateway.js

@@ -6,7 +6,11 @@ import { createResponsesStore } from './openai/responses-store.js';
 import { createApp } from './server/app.js';
 
 const HUB_CONNECT_TIMEOUT_MS = 10000;
-const DRAIN_TIMEOUT_MS = 10000;
+const MB = 1024 * 1024;
+// Time allowed to receive a whole request (headers and body). Generation
+// time is not included; REQUEST_TIMEOUT_MS bounds that.
+const BODY_TIMEOUT_MS = 120000;
+const SHUTDOWN_NOTICE_MS = 2000;
 
 export function createBackend(config, logger) {
     return config.OPENCODE_SERVER_URL
@@ -29,9 +33,25 @@ function listen(app, { PORT, HOST }) {
     });
 }
 
-function drain(server) {
+const openConnections = (server) => new Promise((resolve) => {
+    server.getConnections((error, count) => resolve(error ? 0 : count));
+});
+
+/**
+ * Keep listening for `noticeMs` while /ready reports "stopping", so a load
+ * balancer that polls it stops routing here before the listener closes.
+ * Skipped when nobody is connected, so a local Ctrl+C stays instant.
+ * @returns {Promise<number>} how long it waited
+ */
+async function announceShutdown(server, noticeMs) {
+    if (noticeMs <= 0 || (await openConnections(server)) === 0) return 0;
+    await new Promise((resolve) => setTimeout(resolve, noticeMs));
+    return noticeMs;
+}
+
+function drain(server, timeoutMs) {
     return new Promise((resolve) => {
-        const force = setTimeout(() => server.closeAllConnections(), DRAIN_TIMEOUT_MS);
+        const force = setTimeout(() => server.closeAllConnections(), timeoutMs);
         force.unref();
         server.close(() => { clearTimeout(force); resolve(); });
         server.closeIdleConnections();
@@ -42,15 +62,15 @@ function drain(server) {
  * Wires the OpenCode backend, event hub, catalog and HTTP app together and
  * starts listening.
  * @param {object} config validated config from loadConfig
- * @param {{ logger: object, backend?: object }} deps backend is injectable for tests
+ * @param {{ logger: object, backend?: object, shutdownNoticeMs?: number }} deps backend and notice are injectable for tests
  */
-export async function startGateway(config, { logger, backend = createBackend(config, logger) }) {
+export async function startGateway(config, { logger, backend = createBackend(config, logger), shutdownNoticeMs = SHUTDOWN_NOTICE_MS }) {
     const getClient = () => backend.getClient();
     const hub = createEventHub({ getClient, logger, ownsAllSessions: backend.ownsAllSessions });
     const catalog = createCatalog({ getClient, logger });
     const runner = createRunner({ getClient, hub, logger, agent: config.OPENCODE_AGENT });
-    const store = createResponsesStore({ maxEntries: config.RESPONSES_STORE_MAX });
-    const { app, close: closeApp } = createApp({ config, logger, backend, hub, catalog, runner, store });
+    const store = createResponsesStore({ maxEntries: config.RESPONSES_STORE_MAX, maxChars: config.RESPONSES_STORE_MB * MB });
+    const { app, close: closeApp, startDraining } = createApp({ config, logger, backend, hub, catalog, runner, store });
 
     await backend.start();
     let server;
@@ -70,15 +90,17 @@ export async function startGateway(config, { logger, backend = createBackend(con
         await backend.stop();
         throw error;
     }
-    // Generations can legitimately run for minutes; REQUEST_TIMEOUT_MS bounds them instead.
-    server.requestTimeout = 0;
+    server.requestTimeout = BODY_TIMEOUT_MS;
     server.headersTimeout = 30000;
     server.keepAliveTimeout = 65000;
 
     let stopping = null;
     const stop = () => {
         stopping ??= (async () => {
-            await drain(server);
+            startDraining();
+            // In-flight requests keep running during the notice, so it counts towards SHUTDOWN_TIMEOUT_MS.
+            const noticed = await announceShutdown(server, Math.min(shutdownNoticeMs, config.SHUTDOWN_TIMEOUT_MS / 2));
+            await drain(server, config.SHUTDOWN_TIMEOUT_MS - noticed);
             closeApp();
             await hub.stop();
             await backend.stop();

package/src/node-version.js

@@ -0,0 +1,11 @@
+/**
+ * Imported first by index.js, before any module that needs Node 24, so an
+ * older runtime gets one clear message instead of an obscure failure later.
+ */
+const REQUIRED_MAJOR = 24;
+const major = Number(process.versions.node.split('.')[0]);
+
+if (major < REQUIRED_MAJOR) {
+    process.stderr.write(`zengate needs Node.js ${REQUIRED_MAJOR} or newer; this is ${process.version}.\n`);
+    process.exit(1);
+}

package/src/openai/chat-request.js

@@ -1,5 +1,7 @@
 import { invalidRequest, unsupported } from '../server/errors.js';
 import { audioFromBase64, fileAttachment, imageFromUrl, videoFromUrl } from './media.js';
+import { parseMaxTokens } from './length-limit.js';
+import { escapeFileTags, fileBlock } from './markup.js';
 import { newCallId } from './tool-calls.js';
 
 /**
@@ -8,24 +10,29 @@ import { newCallId } from './tool-calls.js';
  * They are reported back in the `x-gateway-ignored-params` header.
  */
 const IGNORED = new Set([
-    'temperature', 'top_p', 'max_tokens', 'max_completion_tokens', 'presence_penalty', 'frequency_penalty',
+    'temperature', 'top_p', 'presence_penalty', 'frequency_penalty',
     'seed', 'logit_bias', 'user', 'metadata', 'store', 'service_tier', 'prompt_cache_key', 'prompt_cache_retention',
     'safety_identifier', 'verbosity', 'prediction', 'web_search_options', 'top_k', 'min_p', 'repetition_penalty',
 ]);
 const HANDLED = new Set([
     'model', 'messages', 'stream', 'stream_options', 'n', 'stop', 'response_format', 'reasoning_effort',
     'tools', 'tool_choice', 'parallel_tool_calls', 'functions', 'function_call', 'logprobs', 'top_logprobs',
-    'modalities', 'audio',
+    'modalities', 'audio', 'max_tokens', 'max_completion_tokens',
 ]);
 const ROLES = new Set(['system', 'developer', 'user', 'assistant', 'tool', 'function']);
 const MAX_MESSAGES = 2000;
 export const MAX_CHOICES = 4;
+/** OpenAI's own limit on functions in one request. */
+export const MAX_TOOLS = 128;
+/** Long enough for any real stop sequence; the stop filter rescans up to this much text. */
+const MAX_STOP_CHARS = 1000;
 
 const isObject = (value) => Boolean(value) && typeof value === 'object' && !Array.isArray(value);
 
 export function parseTools(tools, param = 'tools') {
     if (tools === undefined || tools === null) return [];
     if (!Array.isArray(tools)) throw invalidRequest('tools must be an array.', param);
+    if (tools.length > MAX_TOOLS) throw invalidRequest(`${param} may contain at most ${MAX_TOOLS} entries.`, param);
     const seen = new Set();
     return tools.map((tool, index) => {
         const fn = tool?.type === 'function' ? tool.function : tool?.type === undefined ? tool : null;
@@ -60,7 +67,7 @@ function textOf(parts, param) {
 }
 
 function parseUserContent(content, param, media) {
-    if (typeof content === 'string') return { text: content, media: [] };
+    if (typeof content === 'string') return { text: escapeFileTags(content), media: [] };
     if (content === null || content === undefined) return { text: '', media: [] };
     if (!Array.isArray(content)) throw invalidRequest('content must be a string or an array of content parts.', param);
     const texts = [];
@@ -69,18 +76,18 @@ function parseUserContent(content, param, media) {
         const where = `${param}[${index}]`;
         const options = { ...media, param: where };
         switch (part?.type) {
-            case 'text': texts.push(String(part.text ?? '')); break;
+            case 'text': texts.push(escapeFileTags(part.text)); break;
             case 'image_url': attachments.push(imageFromUrl(typeof part.image_url === 'string' ? part.image_url : part.image_url?.url, options)); break;
             case 'input_audio': attachments.push(audioFromBase64(part.input_audio?.data, part.input_audio?.format, options)); break;
             case 'file': attachments.push(fileAttachment(part.file || {}, options)); break;
             case 'video_url': attachments.push(videoFromUrl(typeof part.video_url === 'string' ? part.video_url : part.video_url?.url, options)); break;
-            case 'refusal': texts.push(String(part.refusal ?? '')); break;
+            case 'refusal': texts.push(escapeFileTags(part.refusal)); break;
             default:
                 throw unsupported(`Unsupported content part type '${part?.type}'. Use text, image_url, input_audio, file or video_url.`, where);
         }
     });
     const inlined = attachments.filter((a) => a.kind === 'text');
-    for (const file of inlined) texts.push(`\n<file name="${file.filename || 'attachment'}">\n${file.text}\n</file>\n`);
+    for (const file of inlined) texts.push(fileBlock(file.filename, file.text));
     return { text: texts.join(''), media: attachments.filter((a) => a.kind !== 'text') };
 }
 
@@ -139,10 +146,11 @@ export function parseResponseFormat(format, param = 'response_format') {
     throw invalidRequest('response_format.type must be text, json_object or json_schema.', param);
 }
 
-function parseStop(stop) {
+export function parseStop(stop) {
     if (stop === undefined || stop === null) return [];
     const list = Array.isArray(stop) ? stop : [stop];
     if (list.length > 4 || list.some((s) => typeof s !== 'string')) throw invalidRequest('stop must be a string or up to 4 strings.', 'stop');
+    if (list.some((s) => s.length > MAX_STOP_CHARS)) throw invalidRequest(`Each stop sequence may be at most ${MAX_STOP_CHARS} characters.`, 'stop');
     return list.filter(Boolean);
 }
 
@@ -176,6 +184,8 @@ export function parseChatRequest(body, media) {
         reasoningEffort: typeof body.reasoning_effort === 'string' ? body.reasoning_effort : null,
         stop: parseStop(body.stop),
         n,
+        maxTokens: parseMaxTokens(body, ['max_completion_tokens', 'max_tokens']),
+        limitsReasoning: body.max_completion_tokens !== undefined && body.max_completion_tokens !== null,
         stream: body.stream === true,
         includeUsage: body.stream_options?.include_usage === true,
         ignored,

package/src/openai/chat.js

@@ -2,7 +2,10 @@ import crypto from 'node:crypto';
 import { toApiError } from '../server/errors.js';
 import { parseChatRequest } from './chat-request.js';
 import { chatUsage, generate } from './generate.js';
+import { setIgnoredParams } from './ignored-params.js';
+import { mapLimited } from './map-limited.js';
 import { buildPrompt } from './prompt.js';
+import { inlineRemoteAttachments } from './remote-media.js';
 import { openSse } from './sse-writer.js';
 import { assertPublicUrls } from './url-guard.js';
 
@@ -38,15 +41,19 @@ export function chatCompletionsHandler({ runner, catalog, limits }) {
         const model = await catalog.resolve(request.model);
         const prompt = buildPrompt(request, model);
         await assertPublicUrls(prompt.parts);
-        if (request.ignored.length) res.set('x-gateway-ignored-params', request.ignored.join(','));
-        await req.withSlot((signal) => (request.stream
-            ? streamChat({ req, res, runner, request, prompt, model, signal })
-            : jsonChat({ res, runner, request, prompt, model, signal })), request.n);
+        setIgnoredParams(res, request.ignored);
+        await req.withSlot(async (signal, parallel) => {
+            const ready = await inlineRemoteAttachments(prompt, { ...limits, signal });
+            const context = { req, res, runner, request, prompt: ready, model, signal, parallel };
+            return request.stream ? streamChat(context) : jsonChat(context);
+        }, request.n);
     };
 }
 
-async function jsonChat({ res, runner, request, prompt, model, signal }) {
-    const results = await Promise.all(Array.from({ length: request.n }, () => generate({ runner, prompt, request, signal })));
+const choiceIndexes = (n) => Array.from({ length: n }, (_, index) => index);
+
+async function jsonChat({ req, res, runner, request, prompt, model, signal, parallel }) {
+    const results = await mapLimited(choiceIndexes(request.n), parallel, () => generate({ runner, prompt, request, signal, onFirstToken: req.markFirstToken }));
     res.json({
         id: completionId(),
         object: 'chat.completion',
@@ -60,7 +67,7 @@ async function jsonChat({ res, runner, request, prompt, model, signal }) {
     });
 }
 
-async function streamChat({ req, res, runner, request, prompt, model, signal }) {
+async function streamChat({ req, res, runner, request, prompt, model, signal, parallel }) {
     const id = completionId();
     const created = now();
     const sse = openSse(res);
@@ -72,8 +79,8 @@ async function streamChat({ req, res, runner, request, prompt, model, signal })
 
     for (let index = 0; index < request.n; index += 1) chunk(index, { role: 'assistant', content: '', refusal: null });
     try {
-        const results = await Promise.all(Array.from({ length: request.n }, (_, index) => generate({
-            runner, prompt, request, signal,
+        const results = await mapLimited(choiceIndexes(request.n), parallel, (index) => generate({
+            runner, prompt, request, signal, onFirstToken: req.markFirstToken,
             onText: (text) => chunk(index, { content: text }),
             onReasoning: (text) => chunk(index, { reasoning_content: text }),
         }).then((result) => {
@@ -82,7 +89,7 @@ async function streamChat({ req, res, runner, request, prompt, model, signal })
             else if (calls.length) chunk(index, { tool_calls: calls.map((call, position) => ({ index: position, ...call })) });
             chunk(index, {}, finishReason(result, request.legacyFunctions));
             return result;
-        })));
+        }));
         if (request.includeUsage) {
             sse.send({ id, object: 'chat.completion.chunk', created, model: model.id, system_fingerprint: null, choices: [], usage: chatUsage(results.map((r) => r.usage)) });
         }

package/src/openai/completions.js

@@ -0,0 +1,140 @@
+import crypto from 'node:crypto';
+import { invalidRequest, toApiError, unsupported } from '../server/errors.js';
+import { MAX_CHOICES, parseStop } from './chat-request.js';
+import { chatUsage, generate } from './generate.js';
+import { setIgnoredParams } from './ignored-params.js';
+import { mapLimited } from './map-limited.js';
+import { parseMaxTokens } from './length-limit.js';
+import { buildPrompt } from './prompt.js';
+import { openSse } from './sse-writer.js';
+
+/**
+ * Legacy text completions (POST /v1/completions) on top of chat models: the
+ * model is asked to continue the prompt and reply with the continuation only.
+ */
+const CONTINUE_INSTRUCTION = 'Continue the text in the user message from exactly where it stops. '
+    + 'Reply with only the continuation: no preamble, no quotes, and do not repeat the given text.';
+const IGNORED = new Set(['temperature', 'top_p', 'presence_penalty', 'frequency_penalty', 'seed', 'logit_bias', 'user', 'best_of']);
+const HANDLED = new Set(['model', 'prompt', 'stream', 'stream_options', 'n', 'stop', 'max_tokens', 'echo', 'suffix', 'logprobs']);
+const MAX_PROMPT_CHARS = 4 * 1024 * 1024;
+
+const completionId = () => `cmpl-${crypto.randomBytes(12).toString('hex')}`;
+const now = () => Math.floor(Date.now() / 1000);
+
+function parsePrompts(prompt) {
+    const list = Array.isArray(prompt) ? prompt : [prompt];
+    if (list.length === 0 || list.some((item) => typeof item !== 'string')) {
+        throw invalidRequest('prompt must be a string or an array of strings (token arrays are not supported).', 'prompt');
+    }
+    if (list.reduce((sum, item) => sum + item.length, 0) > MAX_PROMPT_CHARS) throw invalidRequest('prompt is too long.', 'prompt');
+    return list;
+}
+
+/**
+ * Validate a completions request. Each prompt becomes its own single-turn
+ * chat request; `n` completions are made for each.
+ * @param {unknown} body
+ */
+export function parseCompletionRequest(body) {
+    if (!body || typeof body !== 'object' || Array.isArray(body)) throw invalidRequest('Request body must be a JSON object.');
+    if (typeof body.model !== 'string' || !body.model.trim()) throw invalidRequest('model is required (see GET /v1/models).', 'model');
+    if (body.prompt === undefined) throw invalidRequest('prompt is required.', 'prompt');
+    if (body.suffix !== undefined && body.suffix !== null && body.suffix !== '') throw unsupported('suffix (fill-in-the-middle) is not supported.', 'suffix');
+    if (body.logprobs !== undefined && body.logprobs !== null) throw unsupported('logprobs are not available through OpenCode.', 'logprobs');
+    const prompts = parsePrompts(body.prompt);
+    const n = body.n ?? 1;
+    if (!Number.isInteger(n) || n < 1 || n * prompts.length > MAX_CHOICES) {
+        throw invalidRequest(`n times the number of prompts must be from 1 to ${MAX_CHOICES}.`, 'n');
+    }
+    const shared = {
+        model: body.model.trim(),
+        tools: [],
+        toolChoice: 'auto',
+        parallelToolCalls: false,
+        format: null,
+        reasoningEffort: null,
+        stop: parseStop(body.stop),
+        maxTokens: parseMaxTokens(body, ['max_tokens']),
+    };
+    return {
+        ...shared,
+        prompts,
+        requests: prompts.map((text) => ({
+            ...shared,
+            messages: [{ role: 'system', content: CONTINUE_INSTRUCTION, media: [] }, { role: 'user', content: text, media: [] }],
+        })),
+        n,
+        echo: body.echo === true,
+        stream: body.stream === true,
+        includeUsage: body.stream_options?.include_usage === true,
+        ignored: Object.keys(body).filter((key) => IGNORED.has(key) || !HANDLED.has(key)),
+    };
+}
+
+/**
+ * POST /v1/completions
+ * @param {{ runner: object, catalog: object }} deps
+ */
+export function completionsHandler({ runner, catalog }) {
+    return async (req, res) => {
+        const parsed = parseCompletionRequest(req.body);
+        const model = await catalog.resolve(parsed.model);
+        const jobs = parsed.requests.flatMap((request, promptIndex) => Array.from({ length: parsed.n }, () => ({
+            request, prompt: buildPrompt(request, model), echo: parsed.echo ? parsed.prompts[promptIndex] : '',
+        })));
+        setIgnoredParams(res, parsed.ignored);
+        await req.withSlot((signal, parallel) => (parsed.stream
+            ? streamCompletion({ req, res, runner, parsed, jobs, model, signal, parallel })
+            : jsonCompletion({ req, res, runner, jobs, model, signal, parallel })), jobs.length);
+    };
+}
+
+const run = ({ req, runner, job, signal, onText }) => generate({
+    runner, prompt: job.prompt, request: job.request, signal, onText, onFirstToken: req.markFirstToken,
+});
+
+async function jsonCompletion({ req, res, runner, jobs, model, signal, parallel }) {
+    const results = await mapLimited(jobs, parallel, (job) => run({ req, runner, job, signal }));
+    res.json({
+        id: completionId(),
+        object: 'text_completion',
+        created: now(),
+        model: model.id,
+        system_fingerprint: null,
+        choices: results.map((result, index) => ({
+            text: `${jobs[index].echo}${result.content}`, index, logprobs: null, finish_reason: result.finish,
+        })),
+        usage: chatUsage(results.map((r) => r.usage)),
+    });
+}
+
+async function streamCompletion({ req, res, runner, parsed, jobs, model, signal, parallel }) {
+    const id = completionId();
+    const created = now();
+    const sse = openSse(res);
+    const chunk = (choices, usage) => sse.send({
+        id, object: 'text_completion', created, model: model.id, system_fingerprint: null, choices,
+        ...(usage ? { usage } : {}),
+    });
+    const text = (index, value, finishReason = null) => chunk([{ text: value, index, logprobs: null, finish_reason: finishReason }]);
+    try {
+        const results = await mapLimited(jobs, parallel, (job, index) => {
+            if (job.echo) text(index, job.echo);
+            return run({ req, runner, job, signal, onText: (delta) => text(index, delta) }).then((result) => {
+                text(index, '', result.finish);
+                return result;
+            });
+        });
+        if (parsed.includeUsage) chunk([], chatUsage(results.map((r) => r.usage)));
+        sse.raw('data: [DONE]\n\n');
+    } catch (error) {
+        if (!signal.aborted || error?.status) {
+            const apiError = toApiError(error);
+            req.log.warn('Completion stream failed', { status: apiError.status, code: apiError.code, error: apiError.message });
+            sse.send(apiError.toJSON());
+            sse.raw('data: [DONE]\n\n');
+        }
+    } finally {
+        sse.end();
+    }
+}