@cldmv/llm-translator 0.11.0 → 0.12.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -17,19 +17,18 @@ Use it as a library when you want the conversion without any I/O (`request`, `re
17
17
 
18
18
  ## ✨ What's New
19
19
 
20
- ### Latest: v0.11.0 (November 2026)
20
+ ### Latest: v0.12.0 (November 2026)
21
21
 
22
- - **Constrained decoding** — a neutral `constraint` (JSON Schema, regex, choice list or grammar) is turned into each backend's native field: vLLM `structured_outputs` (or `guided_*` before 0.11.0), SGLang `regex` and `ebnf`, llama.cpp `grammar` and `json_schema`, and TGI's grammar. A kind a backend can't enforce is dropped and recorded, or rejected.
23
- - **`clear_tool_uses` context edits** — `context_management` edits are applied locally before an OpenAI-protocol upstream sees the conversation, and the reply reports them in `applied_edits`.
24
- - **Prompt-cache hints** — `cache_salt` (set by the server, vLLM only) and `prompt_cache_key` (OpenAI only) are mapped, with docs on what is and isn't cached. Hosted APIs such as `api.openai.com` now bind by host name before any driver probes them.
25
- - [View full v0.11.0 Changelog](https://github.com/CLDMV/llm-translator/blob/master/docs/changelog/v0/v0.11.0.md)
22
+ - **Production server features** — an optional Prometheus `/metrics` endpoint (request counts and durations, tokens, errors and upstream health), plus HTTPS, per-key rate limits with a front-shaped 429, and CORS for browser clients. All are off unless set, with CLI flags and environment variables for each.
23
+ - **Failed tool results marked by default** — a `tool_result` with `is_error: true` sent to a Chat Completions upstream now carries `toolErrorMarker`, so the model knows the tool failed. Set `toolErrors: false` to turn it off.
24
+ - [View full v0.12.0 Changelog](https://github.com/CLDMV/llm-translator/blob/master/docs/changelog/v0/v0.12.0.md)
26
25
 
27
26
  ### Recent Releases
28
27
 
28
+ - **v0.11.0** (November 2026) — native constrained decoding (JSON Schema, regex, choice, grammar), `clear_tool_uses` context edits, `cache_salt` and `prompt_cache_key` hints, and host-first binding for hosted APIs ([Changelog](https://github.com/CLDMV/llm-translator/blob/master/docs/changelog/v0/v0.11.0.md))
29
29
  - **v0.10.0** (November 2026) — native `/v1/messages` for vLLM, readiness probes and LoRA adapters, Groq, Together, Fireworks, xAI and Gemini drivers, and Ollama's native `/api/chat` ([Changelog](https://github.com/CLDMV/llm-translator/blob/master/docs/changelog/v0/v0.10.0.md))
30
30
  - **v0.9.0** (November 2026) — Azure OpenAI, OpenRouter, Hugging Face TGI and LocalAI drivers, with Azure's `apiVersion` and `deployment` upstream fields and more startup probes ([Changelog](https://github.com/CLDMV/llm-translator/blob/master/docs/changelog/v0/v0.9.0.md))
31
31
  - **v0.8.0** (November 2026) — Mistral and LM Studio drivers, an OpenAI client's `n` above 1 answered by parallel requests to Anthropic-protocol upstreams, and `stop_details.category` from content-filter results ([Changelog](https://github.com/CLDMV/llm-translator/blob/master/docs/changelog/v0/v0.8.0.md))
32
- - **v0.7.0** (November 2026) — Anthropic-protocol servers detected by probe on any host, tool-call ids rewritten for strict upstream formats, embeddings usage through `onUsage`, and capture records for every failed-over attempt ([Changelog](https://github.com/CLDMV/llm-translator/blob/master/docs/changelog/v0/v0.7.0.md))
33
32
 
34
33
  📚 **For complete version history and detailed release notes, see the [docs/changelog/](https://github.com/CLDMV/llm-translator/tree/master/docs/changelog/) folder.**
35
34
 
@@ -1,2 +1,2 @@
1
- var w=Object.defineProperty;var n=(o,t)=>w(o,"name",{value:t,configurable:!0});import{context as s,self as u}from"../../lib/runtime.mjs";import{TranslatorError as d}from"../../lib/errors.mjs";import{register as k}from"../../lib/routes.mjs";import{capture as x}from"../../lib/capture.mjs";import{resolve as y}from"../../lib/route.mjs";import{resolve as I}from"../../lib/settings.mjs";import{emit as T}from"../../lib/events.mjs";async function N(o,t={}){const{config:r}=s;if(t.front!==void 0&&t.front!=="anthropic"&&t.front!=="openai")throw new d(`front must be "anthropic" or "openai" (got ${JSON.stringify(t.front)})`,{code:"invalid_config",status:500});for(const e of["authorize","onUsage"])if(t[e]!==void 0&&typeof t[e]!="function")throw new d(`${e} must be a function`,{code:"invalid_config",status:500});const{send:m,route:p,token:h,model:c,error:v,sse:f,embeddings:b}=u,l=t.front??r.front,{state:g}=s,{events:i}=g;i.logger=o.log,o.addHook("onClose",async()=>{i.logger===o.log&&(i.logger=null)});const a=x(u,s,l);k(o,t,{front:l,filter:r.settings.passthroughFilter!==!1,interval:n(e=>{try{return a.settings(y(a,e).endpoint).pingInterval}catch{return I(r.settings).pingInterval}},"interval"),send:m,embed:b.send,route:p,count:h.count,list:c.list,get:c.get,emit:n(e=>T({config:r,state:g},e),"emit"),build:v.build,encode:f.encode,write:f.write})}n(N,"plugin");export{N as plugin};
1
+ var w=Object.defineProperty;var n=(r,e)=>w(r,"name",{value:e,configurable:!0});import{context as c,self as g}from"../../lib/runtime.mjs";import{TranslatorError as f}from"../../lib/errors.mjs";import{register as y}from"../../lib/routes.mjs";import{capture as _}from"../../lib/capture.mjs";import{resolve as k}from"../../lib/route.mjs";import{resolve as x}from"../../lib/settings.mjs";import{emit as I}from"../../lib/events.mjs";async function N(r,e={}){const{config:o}=c;if(e.front!==void 0&&e.front!=="anthropic"&&e.front!=="openai")throw new f(`front must be "anthropic" or "openai" (got ${JSON.stringify(e.front)})`,{code:"invalid_config",status:500});for(const t of["authorize","onUsage"])if(e[t]!==void 0&&typeof e[t]!="function")throw new f(`${t} must be a function`,{code:"invalid_config",status:500});if(e.metrics!==void 0&&typeof e.metrics!="boolean")throw new f("metrics must be true or false",{code:"invalid_config",status:500});const{send:m,route:p,token:h,model:l,error:v,sse:a,embeddings:b}=g,d=e.front??o.front,{state:i}=c,{events:s}=i;s.logger=r.log,r.addHook("onClose",async()=>{s.logger===r.log&&(s.logger=null)});const u=_(g,c,d);y(r,e,{front:d,filter:o.settings.passthroughFilter!==!1,interval:n(t=>{try{return u.settings(k(u,t).endpoint).pingInterval}catch{return x(o.settings).pingInterval}},"interval"),send:m,embed:b.send,route:p,count:h.count,list:l.list,get:l.get,emit:n(t=>I({config:o,state:i},t),"emit"),endpoints:n(()=>i.endpoints,"endpoints"),build:v.build,encode:a.encode,write:a.write})}n(N,"plugin");export{N as plugin};
2
2
  //# sourceMappingURL=plugin.mjs.map
@@ -1,2 +1,2 @@
1
- var m=Object.defineProperty;var s=(t,r)=>m(t,"name",{value:r,configurable:!0});import{context as g,self as u}from"../../lib/runtime.mjs";import{TranslatorError as c}from"../../lib/errors.mjs";import{isLoopback as b}from"../../lib/http.mjs";async function L(t={}){const{state:r}=g,{plugin:d}=u.server,{build:h}=u.error;let i;try{({default:i}=await import("fastify"))}catch(o){throw new c('server.start needs the optional peer dependency "fastify" (npm install fastify)',{code:"fastify_missing",status:500,details:{cause:o}})}const e=t.host??"127.0.0.1";if(!b(e)&&[t.apiKey??[]].flat().filter(Boolean).length===0&&typeof t.authorize!="function"&&t.allowUnauthenticated!==!0)throw new c(`Refusing to listen on ${e} without an apiKey or an authorize hook: anyone who can reach the port could use the upstream. Set \`apiKey\` (CLI: --api-key) or \`authorize\`, bind to 127.0.0.1, or opt in with \`allowUnauthenticated: true\` (CLI: --allow-unauthenticated).`,{code:"unauthenticated_bind",status:500});const a=i({logger:t.logger??!1,forceCloseConnections:!0});a.setNotFoundHandler((o,f)=>{const{status:y,headers:p,body:w}=h(404,`Not found: ${o.method} ${o.url}`);return f.code(y).headers(p).send(w)}),await a.register(d,{front:t.front,apiKey:t.apiKey,bodyLimit:t.bodyLimit,authorize:t.authorize,onUsage:t.onUsage}),await a.listen({host:e,port:t.port??8787});const l=a.server.address().port,n={url:`http://${e.includes(":")?`[${e}]`:e}:${l}`,host:e,port:l,app:a,close:s(async()=>{r.servers.delete(n),await a.close()},"close")};return r.servers.add(n),n}s(L,"start");export{L as start};
1
+ var g=Object.defineProperty;var i=(t,a)=>g(t,"name",{value:a,configurable:!0});import{context as b,self as u}from"../../lib/runtime.mjs";import{TranslatorError as d}from"../../lib/errors.mjs";import{isLoopback as L}from"../../lib/http.mjs";import{tlsOptions as $}from"../../lib/guard.mjs";async function k(t={}){const{state:a}=b,{plugin:h}=u.server,{build:f}=u.error;let c;try{({default:c}=await import("fastify"))}catch(o){throw new d('server.start needs the optional peer dependency "fastify" (npm install fastify)',{code:"fastify_missing",status:500,details:{cause:o}})}const s=await $(t.tls),e=t.host??"127.0.0.1";if(!L(e)&&[t.apiKey??[]].flat().filter(Boolean).length===0&&typeof t.authorize!="function"&&t.allowUnauthenticated!==!0)throw new d(`Refusing to listen on ${e} without an apiKey or an authorize hook: anyone who can reach the port could use the upstream. Set \`apiKey\` (CLI: --api-key) or \`authorize\`, bind to 127.0.0.1, or opt in with \`allowUnauthenticated: true\` (CLI: --allow-unauthenticated).`,{code:"unauthenticated_bind",status:500});const r=c({logger:t.logger??!1,forceCloseConnections:!0,...s?{https:s}:{}});r.setNotFoundHandler((o,m)=>{const{status:y,headers:p,body:w}=f(404,`Not found: ${o.method} ${o.url}`);return m.code(y).headers(p).send(w)}),await r.register(h,{front:t.front,apiKey:t.apiKey,bodyLimit:t.bodyLimit,authorize:t.authorize,onUsage:t.onUsage,metrics:t.metrics,cors:t.cors,rateLimit:t.rateLimit}),await r.listen({host:e,port:t.port??8787});const l=r.server.address().port,n={url:`${s?"https":"http"}://${e.includes(":")?`[${e}]`:e}:${l}`,host:e,port:l,app:r,close:i(async()=>{a.servers.delete(n),await r.close()},"close")};return a.servers.add(n),n}i(k,"start");export{k as start};
2
2
  //# sourceMappingURL=start.mjs.map
package/dist/lib/cli.mjs CHANGED
@@ -1,4 +1,4 @@
1
- var _=Object.defineProperty;var c=(t,e)=>_(t,"name",{value:e,configurable:!0});import{readFile as d}from"node:fs/promises";import{parseArgs as g}from"node:util";const u=`llm-translator - run Claude Code (or any Anthropic/OpenAI client) against local LLM servers
1
+ var O=Object.defineProperty;var p=(t,e)=>O(t,"name",{value:e,configurable:!0});import{readFile as d}from"node:fs/promises";import{parseArgs as N}from"node:util";const m=`llm-translator - run Claude Code (or any Anthropic/OpenAI client) against local LLM servers
2
2
 
3
3
  Usage: llm-translator --upstream <url> [options]
4
4
 
@@ -12,6 +12,14 @@ Options:
12
12
  --port <port> Listen port (default 8787)
13
13
  --api-key <key> Key clients must send (x-api-key or Authorization: Bearer)
14
14
  --allow-unauthenticated Allow a non-loopback --host without --api-key
15
+ --metrics Serve Prometheus metrics at GET /metrics (same authentication as the other routes)
16
+ --tls-key <file> TLS private key (PEM); with --tls-cert the server speaks HTTPS
17
+ --tls-cert <file> TLS certificate (PEM)
18
+ --tls-ca <file> Extra CA certificate (PEM)
19
+ --tls-passphrase <text> Passphrase of the TLS key
20
+ --cors-origin <origin> Origin browsers may call from (repeatable, * for any); CORS is off without it
21
+ --rate-limit <n> Allow n requests per window for each API key (remote address without one); off without it
22
+ --rate-limit-window <ms> Rate limit window (default 60000)
15
23
  --config <file> JSON config file (translator config plus a "server" object)
16
24
  --logs [path] Log warnings and errors to the console (default), or append every event as a JSON line to <path>
17
25
  --silent Same as logs: false (no console output, no file)
@@ -20,11 +28,12 @@ Options:
20
28
  --help Print this help
21
29
 
22
30
  Environment: LLM_TRANSLATOR_FRONT, LLM_TRANSLATOR_UPSTREAM, LLM_TRANSLATOR_UPSTREAM_KEY, LLM_TRANSLATOR_BACKEND,
23
- LLM_TRANSLATOR_HOST, LLM_TRANSLATOR_PORT, LLM_TRANSLATOR_API_KEY, LLM_TRANSLATOR_ALLOW_UNAUTHENTICATED (1), LLM_TRANSLATOR_CONFIG. Flags win over the
24
- environment, which wins over the config file.
25
- `;function S(t){const e=t.map((o,l)=>o==="--logs"&&(l===t.length-1||t[l+1].startsWith("-"))?"--logs=":o);return g({args:e,allowPositionals:!1,options:{front:{type:"string"},upstream:{type:"string",multiple:!0},"upstream-key":{type:"string"},backend:{type:"string"},model:{type:"string",multiple:!0},host:{type:"string"},port:{type:"string"},"api-key":{type:"string"},"allow-unauthenticated":{type:"boolean"},config:{type:"string"},logs:{type:"string"},silent:{type:"boolean"},"no-detect":{type:"boolean"},version:{type:"boolean"},help:{type:"boolean"}}}).values}c(S,"parse");function h(t,e,o){const{server:l,...p}=o,s=t.upstream??(e.LLM_TRANSLATOR_UPSTREAM?e.LLM_TRANSLATOR_UPSTREAM.split(","):void 0),r={...p};if(s){const i=t["upstream-key"]??e.LLM_TRANSLATOR_UPSTREAM_KEY,a=t.backend??e.LLM_TRANSLATOR_BACKEND;r.upstreams=s.map(n=>({url:n,apiKey:i,...a?{backend:a}:{}}))}const T=t.front??e.LLM_TRANSLATOR_FRONT;if(T&&(r.front=T),t.model){r.aliases={...r.aliases};for(const i of t.model){const a=i.indexOf("=");if(a<1)throw new Error(`--model expects alias=target (got "${i}")`);r.aliases[i.slice(0,a)]=i.slice(a+1)}}t.silent&&(r.logs=!1),t.logs!==void 0&&(r.logs=t.logs===""?!0:t.logs),r.logs??=r.silent!==!0,delete r.silent,t["no-detect"]&&(r.detect=!1);const A=t.port??e.LLM_TRANSLATOR_PORT,L={...l,...t.host??e.LLM_TRANSLATOR_HOST?{host:t.host??e.LLM_TRANSLATOR_HOST}:{},...A?{port:Number(A)}:{},...t["api-key"]??e.LLM_TRANSLATOR_API_KEY?{apiKey:t["api-key"]??e.LLM_TRANSLATOR_API_KEY}:{},...t["allow-unauthenticated"]||e.LLM_TRANSLATOR_ALLOW_UNAUTHENTICATED==="1"?{allowUnauthenticated:!0}:{}};return{translator:r,server:L}}c(h,"merge");const y={out:c(t=>process.stdout.write(`${t}
26
- `),"out"),err:c(t=>process.stderr.write(`${t}
27
- `),"err"),onSignal:c(t=>{process.once("SIGINT",t),process.once("SIGTERM",t)},"onSignal")};async function w(t,e=process.env,o=y,l={}){let p;try{p=S(t)}catch(s){return o.err(`${s.message}
31
+ LLM_TRANSLATOR_HOST, LLM_TRANSLATOR_PORT, LLM_TRANSLATOR_API_KEY, LLM_TRANSLATOR_ALLOW_UNAUTHENTICATED (1), LLM_TRANSLATOR_METRICS (1), LLM_TRANSLATOR_CONFIG,
32
+ LLM_TRANSLATOR_TLS_KEY, LLM_TRANSLATOR_TLS_CERT, LLM_TRANSLATOR_TLS_CA, LLM_TRANSLATOR_TLS_PASSPHRASE, LLM_TRANSLATOR_CORS_ORIGINS (comma separated),
33
+ LLM_TRANSLATOR_RATE_LIMIT, LLM_TRANSLATOR_RATE_LIMIT_WINDOW. Flags win over the environment, which wins over the config file.
34
+ `;function g(t){const e=t.map((s,n)=>s==="--logs"&&(n===t.length-1||t[n+1].startsWith("-"))?"--logs=":s);return N({args:e,allowPositionals:!1,options:{front:{type:"string"},upstream:{type:"string",multiple:!0},"upstream-key":{type:"string"},backend:{type:"string"},model:{type:"string",multiple:!0},host:{type:"string"},port:{type:"string"},"api-key":{type:"string"},"allow-unauthenticated":{type:"boolean"},metrics:{type:"boolean"},"tls-key":{type:"string"},"tls-cert":{type:"string"},"tls-ca":{type:"string"},"tls-passphrase":{type:"string"},"cors-origin":{type:"string",multiple:!0},"rate-limit":{type:"string"},"rate-limit-window":{type:"string"},config:{type:"string"},logs:{type:"string"},silent:{type:"boolean"},"no-detect":{type:"boolean"},version:{type:"boolean"},help:{type:"boolean"}}}).values}p(g,"parse");function M(t,e,s){const{server:n,...T}=s,a=t.upstream??(e.LLM_TRANSLATOR_UPSTREAM?e.LLM_TRANSLATOR_UPSTREAM.split(","):void 0),r={...T};if(a){const i=t["upstream-key"]??e.LLM_TRANSLATOR_UPSTREAM_KEY,L=t.backend??e.LLM_TRANSLATOR_BACKEND;r.upstreams=a.map(u=>({url:u,apiKey:i,...L?{backend:L}:{}}))}const R=t.front??e.LLM_TRANSLATOR_FRONT;if(R&&(r.front=R),t.model){r.aliases={...r.aliases};for(const i of t.model){const L=i.indexOf("=");if(L<1)throw new Error(`--model expects alias=target (got "${i}")`);r.aliases[i.slice(0,L)]=i.slice(L+1)}}t.silent&&(r.logs=!1),t.logs!==void 0&&(r.logs=t.logs===""?!0:t.logs),r.logs??=r.silent!==!0,delete r.silent,t["no-detect"]&&(r.detect=!1);const S=t.port??e.LLM_TRANSLATOR_PORT,l={...n,...t.host??e.LLM_TRANSLATOR_HOST?{host:t.host??e.LLM_TRANSLATOR_HOST}:{},...S?{port:Number(S)}:{},...t["api-key"]??e.LLM_TRANSLATOR_API_KEY?{apiKey:t["api-key"]??e.LLM_TRANSLATOR_API_KEY}:{},...t["allow-unauthenticated"]||e.LLM_TRANSLATOR_ALLOW_UNAUTHENTICATED==="1"?{allowUnauthenticated:!0}:{},...t.metrics||e.LLM_TRANSLATOR_METRICS==="1"?{metrics:!0}:{}},_={...n?.tls,...f({key:t["tls-key"]??e.LLM_TRANSLATOR_TLS_KEY,cert:t["tls-cert"]??e.LLM_TRANSLATOR_TLS_CERT}),...f({ca:t["tls-ca"]??e.LLM_TRANSLATOR_TLS_CA,passphrase:t["tls-passphrase"]??e.LLM_TRANSLATOR_TLS_PASSPHRASE})};Object.keys(_).length>0&&(l.tls=_);const c=t["cors-origin"]??(e.LLM_TRANSLATOR_CORS_ORIGINS?e.LLM_TRANSLATOR_CORS_ORIGINS.split(","):void 0);c&&(l.cors=c.map(i=>i.trim()).filter(Boolean));const o=t["rate-limit"]??e.LLM_TRANSLATOR_RATE_LIMIT,A=t["rate-limit-window"]??e.LLM_TRANSLATOR_RATE_LIMIT_WINDOW;return(o||A)&&(l.rateLimit={...n?.rateLimit,...o?{requests:Number(o)}:{},...A?{windowMs:Number(A)}:{}}),{translator:r,server:l}}p(M,"merge");function f(t){return Object.fromEntries(Object.entries(t).filter(([,e])=>e))}p(f,"pick");const h={out:p(t=>process.stdout.write(`${t}
35
+ `),"out"),err:p(t=>process.stderr.write(`${t}
36
+ `),"err"),onSignal:p(t=>{process.once("SIGINT",t),process.once("SIGTERM",t)},"onSignal")};async function I(t,e=process.env,s=h,n={}){let T;try{T=g(t)}catch(a){return s.err(`${a.message}
28
37
 
29
- ${u}`),2}if(p.help)return o.out(u),0;if(p.version)return o.out(l.version??(await import("../../package.json",{with:{type:"json"}})).default.version),0;try{const s=p.config??e.LLM_TRANSLATOR_CONFIG,{translator:r,server:T}=h(p,e,s?JSON.parse(await d(s,"utf8")):{}),L=await(l.createTranslator??(await import("../index.mjs")).createTranslator)(r),i=await L.server.start(T),a=L.status();o.out(`llm-translator (${a.front} front) listening on ${i.url}`);for(const n of a.upstreams??[]){const f=n.version??"unknown",m=n.probed?.inlineSystem??"auto",R=n.probed?.images??"default";o.out(` upstream ${n.name} ${n.url}: driver=${n.driver} version=${f} inlineSystem=${m} images=${R}`)}return await new Promise(n=>o.onSignal(()=>n())),await L.server.stop(),await L.slothlet.shutdown(),0}catch(s){return o.err(`llm-translator: ${s.message}`),1}}c(w,"main");export{w as main,h as merge,S as parse,y as processIo};
38
+ ${m}`),2}if(T.help)return s.out(m),0;if(T.version)return s.out(n.version??(await import("../../package.json",{with:{type:"json"}})).default.version),0;try{const a=T.config??e.LLM_TRANSLATOR_CONFIG,{translator:r,server:R}=M(T,e,a?JSON.parse(await d(a,"utf8")):{}),l=await(n.createTranslator??(await import("../index.mjs")).createTranslator)(r),_=await l.server.start(R),c=l.status();s.out(`llm-translator (${c.front} front) listening on ${_.url}`);for(const o of c.upstreams??[]){const A=o.version??"unknown",i=o.probed?.inlineSystem??"auto",L=o.probed?.images??"default";s.out(` upstream ${o.name} ${o.url}: driver=${o.driver} version=${A} inlineSystem=${i} images=${L}`)}return await new Promise(o=>s.onSignal(()=>o())),await l.server.stop(),await l.slothlet.shutdown(),0}catch(a){return s.err(`llm-translator: ${a.message}`),1}}p(I,"main");export{I as main,M as merge,g as parse,h as processIo};
30
39
  //# sourceMappingURL=cli.mjs.map
@@ -0,0 +1,2 @@
1
+ var y=Object.defineProperty;var c=(t,s)=>y(t,"name",{value:s,configurable:!0});import{readFile as h}from"node:fs/promises";import{TranslatorError as m}from"./errors.mjs";const u=c(t=>new m(t,{code:"invalid_config",status:500}),"invalid");async function x(t){if(t==null)return null;if(typeof t!="object"||Array.isArray(t))throw u("tls must be an object with key and cert file paths");const{key:s,cert:i,ca:n,passphrase:a}=t;if(!s&&!i&&!n&&!a)return null;if(!s||!i)throw u(`tls needs both key and cert (got ${s?"key":"cert"} only)`);const e=c(async(r,o)=>{if(typeof o!="string")throw u(`tls.${r} must be a file path`);try{return await h(o)}catch(f){throw u(`tls.${r}: cannot read ${o} (${f.code??f})`)}},"read"),l={key:await e("key",s),cert:await e("cert",i)};if(n&&(l.ca=await Promise.all([n].flat().map(r=>e("ca",r)))),a!==void 0){if(typeof a!="string")throw u("tls.passphrase must be a string");l.passphrase=a}return l}c(x,"tlsOptions");const g="authorization, x-api-key, content-type, anthropic-version, anthropic-beta",d=c(t=>t.toLowerCase().replace(/\/+$/,""),"comparable");function $(t){if(t==null||t===!1)return null;const s=[Array.isArray(t)||typeof t=="string"?t:t.origins??[]].flat();if(s.some(e=>typeof e!="string"||e===""))throw u("cors origins must be non-empty strings");if(s.length===0)return null;const i=s.includes("*"),n=new Set(s.map(d)),a=c(e=>e?!i&&!n.has(d(e))?{vary:"origin"}:{"access-control-allow-origin":i?"*":e,"access-control-expose-headers":"retry-after, request-id, x-request-id",vary:"origin"}:{},"headers");return{headers:a,preflight:c((e,l)=>{const r=a(e);return r["access-control-allow-origin"]?{...r,"access-control-allow-methods":"GET, HEAD, POST, OPTIONS","access-control-allow-headers":l||g,"access-control-max-age":"600"}:r},"preflight")}}c($,"cors");function v(t,s=Date.now){if(t==null||t===!1)return null;const i=t.requests,n=t.windowMs??6e4;if(!Number.isInteger(i)||i<1)throw u(`rateLimit.requests must be a positive integer (got ${JSON.stringify(i)})`);if(!Number.isInteger(n)||n<1)throw u(`rateLimit.windowMs must be a positive integer (got ${JSON.stringify(n)})`);const a=new Map;let e=s();return{take:c(l=>{const r=s();if(r-e>=n){for(const[f,w]of a)r-w.start>=n&&a.delete(f);e=r}let o=a.get(l);return(!o||r-o.start>=n)&&(o={start:r,count:0},a.set(l,o)),o.count>=i?Math.max(1,Math.ceil((o.start+n-r)/1e3)):(o.count++,0)},"take"),size:c(()=>a.size,"size")}}c(v,"rateLimit");export{$ as cors,v as rateLimit,x as tlsOptions};
2
+ //# sourceMappingURL=guard.mjs.map
@@ -0,0 +1,4 @@
1
+ var x=Object.defineProperty;var c=(t,r)=>x(t,"name",{value:r,configurable:!0});import{describe as I}from"./pool.mjs";const A="text/plain; version=0.0.4; charset=utf-8",N=[.005,.01,.025,.05,.1,.25,.5,1,2.5,5,10,30,60,120,300],R=1e4,m="llm_translator_",T=`${m}requests_total`,w=`${m}request_duration_seconds`,v=`${m}tokens_total`,S=`${m}errors_total`,U=`${m}upstream_up`;function j(t){return String(t).replace(/\\/g,"\\\\").replace(/"/g,'\\"').replace(/\n/g,"\\n")}c(j,"escapeLabel");function P(t){return t.replace(/\\/g,"\\\\").replace(/\n/g,"\\n")}c(P,"escapeHelp");function $(t){return t===1/0?"+Inf":t===-1/0?"-Inf":String(t)}c($,"format");function f(t){const r=Object.entries(t).map(([i,n])=>`${i}="${j(n)}"`);return r.length===0?"":`{${r.join(",")}}`}c(f,"braces");function F(){const t=new Map,r=c(i=>JSON.stringify(Object.entries(i)),"key");return{counter(i,n){t.set(i,{type:"counter",help:n,series:new Map})},histogram(i,n,e=N){t.set(i,{type:"histogram",help:n,buckets:e,series:new Map})},inc(i,n,e=1){const{series:o}=t.get(i),s=r(n),a=o.get(s)??{labels:n,value:0};a.value+=e,o.set(s,a)},observe(i,n,e){const{series:o,buckets:s}=t.get(i),a=r(n),l=o.get(a)??{labels:n,counts:s.map(()=>0),sum:0,count:0};s.forEach((h,u)=>{e<=h&&l.counts[u]++}),l.sum+=e,l.count++,o.set(a,l)},render(i=[]){const n=[];for(const[e,o]of t){n.push(`# HELP ${e} ${P(o.help)}`,`# TYPE ${e} ${o.type}`);for(const s of o.series.values()){if(o.type==="counter"){n.push(`${e}${f(s.labels)} ${$(s.value)}`);continue}o.buckets.forEach((a,l)=>{n.push(`${e}_bucket${f({...s.labels,le:$(a)})} ${s.counts[l]}`)}),n.push(`${e}_bucket${f({...s.labels,le:"+Inf"})} ${s.count}`),n.push(`${e}_sum${f(s.labels)} ${$(s.sum)}`),n.push(`${e}_count${f(s.labels)} ${s.count}`)}}for(const e of i){n.push(`# HELP ${e.name} ${P(e.help)}`,`# TYPE ${e.name} gauge`);for(const o of e.samples)n.push(`${e.name}${f(o.labels)} ${$(o.value)}`)}return`${n.join(`
2
+ `)}
3
+ `}}}c(F,"create");function H(t,r){return t===499||r?.reason==="aborted"?null:r?.reason==="upstream_error"?"upstream":r?.reason==="error"?t>=400&&t<500?"client":"translator":t>=500?"translator":t>=400?"client":null}c(H,"kind");function C(t){const r=F();r.counter(T,"Requests handled, by front protocol, route, upstream driver and HTTP status code (499 when the client left before the reply finished)."),r.histogram(w,"Seconds from the request arriving to the end of the response, streams included."),r.counter(v,"Tokens reported by upstreams, by driver and type (input, output, cache_read, cache_creation)."),r.counter(S,"Failed requests by kind: upstream, translator or client.");const i=new WeakMap,n=c(async(e,o,s)=>{const a=o.raw,l=!a.writableFinished,h=i.get(e);let u=null;if(h){let p;const d=new Promise(g=>{p=setTimeout(g,R,null),p.unref()});u=await Promise.race([h.then(g=>g.usage,()=>null),d]),clearTimeout(p)}const O=Number(process.hrtime.bigint()-s)/1e9,b=l?499:a.statusCode,_=u?.upstream?t.endpoints().find(p=>p.name===u.upstream):void 0,y=_?String(_.driver??"unknown"):"none",k={front:t.front,route:e.routeOptions?.url??"unmatched",driver:y,code:b};r.inc(T,k),r.observe(w,k,O);const E=H(b,u);if(E&&r.inc(S,{kind:E}),u?.reported)for(const[p,d]of[["input","input_tokens"],["output","output_tokens"],["cache_read","cache_read_input_tokens"],["cache_creation","cache_creation_input_tokens"]])Number.isFinite(u[d])&&u[d]>0&&r.inc(v,{driver:y,type:p},u[d])},"finish");return{hook:c(async(e,o)=>{if(e.routeOptions?.config?.metrics)return;const s=process.hrtime.bigint();o.raw.once("close",()=>{n(e,o,s).catch(()=>{})})},"hook"),attach:c((e,o)=>{i.set(e,o)},"attach"),render:c(()=>r.render([{name:U,help:"1 while an upstream is healthy (not backing off after a failure and not reported not ready by its readiness probe), else 0.",samples:t.endpoints().map(e=>({labels:{upstream:e.name,driver:String(e.driver??"unknown")},value:I(e).healthy?1:0}))}]),"render")}}c(C,"recorder");export{N as BUCKETS,A as CONTENT_TYPE,F as create,P as escapeHelp,j as escapeLabel,H as kind,C as recorder};
4
+ //# sourceMappingURL=metrics.mjs.map
@@ -1,2 +1,2 @@
1
- var H=Object.defineProperty;var c=(e,o)=>H(e,"name",{value:o,configurable:!0});import{timingSafeEqual as O}from"node:crypto";import{TranslatorError as p}from"./errors.mjs";import{messageId as T}from"./ids.mjs";import{sign as j}from"./thinking.mjs";const C=64*1024*1024,D={"content-type":"text/event-stream; charset=utf-8","cache-control":"no-cache","x-accel-buffering":"no"},v=Symbol("stalled"),x=Symbol("gone");async function I(e,o){if(!(o>0))return e;let s;const d=new Promise(f=>{s=setTimeout(f,o,v)});try{return await Promise.race([e,d])}finally{clearTimeout(s)}}c(I,"settle");async function*L(e,o,s){let d=!1;try{const f=o.front==="anthropic";f&&(yield{event:"message_start",data:{type:"message_start",message:{id:T(),type:"message",role:"assistant",model:s,content:[],stop_reason:null,stop_sequence:null,usage:{input_tokens:0,output_tokens:0,cache_creation_input_tokens:0,cache_read_input_tokens:0}}}},yield{event:"content_block_start",data:{type:"content_block_start",index:0,content_block:{type:"thinking",thinking:"",signature:""}}});const a=await e,b=c(function*(){yield{event:"content_block_delta",data:{type:"content_block_delta",index:0,delta:{type:"signature_delta",signature:j()}}},yield{event:"content_block_stop",data:{type:"content_block_stop",index:0}}},"closeEarly");if(a.stream){d=!0;let u=null;for await(const i of a.stream){if(!f){yield i;continue}if(i.event!=="message_start"){if(u===null&&i.event==="content_block_start"&&i.data.content_block.type==="thinking"){u=0;continue}u===null&&(yield*b(),u=1),yield u>0&&typeof i.data?.index=="number"?{...i,data:{...i.data,index:i.data.index+u}}:i}}return}f&&(yield*b()),yield*S(o,a.body)}finally{d||e.then(f=>f.stream?.return?.(),()=>{})}}c(L,"late");function*S(e,o){if(e.front==="anthropic"){yield{event:"error",data:o};return}yield{data:o},yield{data:"[DONE]"}}c(S,"failure");async function E(e,o,s,d,f,a=200,b=D){e.hijack();const u=e.raw;u.writeHead(a,b);try{await o.write(u,d,{front:o.front,interval:f})}catch(i){if(!s.signal.aborted){const y=i instanceof p;y||(e.log.error(i),o.emit({type:"error",kind:"route",level:"error",message:String(i?.message??i),log:!1}));const{body:g}=o.build(y?i.status:502,y?i.message:"Internal server error",y?{type:i.type,code:i.code}:{});for(const w of S(o,g))u.write(o.encode(w))}}return u.end(),e}c(E,"stream");function U(e,o){const s=Buffer.from(e),d=Buffer.from(o);return s.length===d.length&&O(s,d)}c(U,"same");function z(e){const o=/^Bearer\s+(.+)$/i.exec(e.authorization??"");return e["x-api-key"]||o?.[1]||""}c(z,"keyOf");function B(e){const o={};for(const[s,d]of Object.entries(e))(s==="anthropic-beta"||s==="anthropic-version"||s.startsWith("x-claude-code-"))&&(o[s]=String(d));return o}c(B,"forwardable");function P(e){if(e===!0)return{ok:!0,context:void 0};if(e&&typeof e=="object"&&e.ok===!0)return{ok:!0,context:e.context,cacheSalt:e.cacheSalt};const o=e&&typeof e=="object"&&e.ok===!1?e:{};return{ok:!1,status:Number.isInteger(o.status)&&o.status>=400&&o.status<=599?o.status:401,message:typeof o.message=="string"&&o.message!==""?o.message:"unauthorized"}}c(P,"verdictOf");function R(e,o,s){const{front:d,filter:f}=s,a={...s,build:c((t,n,r={})=>s.build(t,n,{...r,front:d}),"build"),send:c((t,n)=>s.send(t,{...n,front:d,filter:f}),"send"),route:c(t=>s.route(t,{front:d}),"route"),count:c((t,n)=>s.count(t,{...n,front:d}),"count"),embed:c((t,n)=>s.embed(t,n),"embed"),list:c(t=>s.list({...t,front:d}),"list"),get:c((t,n)=>s.get(t,{...n,front:d}),"get")},b=[o.apiKey??[]].flat().filter(Boolean),u=o.bodyLimit??C,i=new WeakMap,y=new WeakMap,g=c((t,n)=>t.code(n.status).headers(n.headers).send(n.body),"reject");e.setErrorHandler((t,n,r)=>{if(t instanceof p)return g(r,a.build(t.status,t.message,{type:t.type,code:t.code,param:t.param}));const l=t.statusCode??500;return l<500?g(r,a.build(l,t.message)):(n.log.error(t),a.emit({type:"error",kind:"route",level:"error",message:String(t.message),status:l,log:!1}),g(r,a.build(l,"Internal server error")))}),e.addHook("onRequest",async(t,n)=>{if(b.length===0||t.routeOptions.config?.public)return;const r=z(t.headers);if(!(r&&b.some(l=>U(r,l))))return g(n,a.build(401,"invalid x-api-key"))}),typeof o.authorize=="function"&&e.addHook("preHandler",async(t,n)=>{if(t.routeOptions.config?.public)return;let r=null;if(t.body&&typeof t.body=="object"&&!Array.isArray(t.body))try{r=a.route(t.body)}catch{}let l;try{l=P(await o.authorize({method:t.method,url:t.url,headers:t.headers,body:t.body,front:a.front,route:r}))}catch(m){t.log.error(m),a.emit({type:"error",kind:"route",level:"error",message:String(m?.message??m),log:!1}),l={ok:!1,status:401,message:"unauthorized"}}if(!l.ok)return g(n,a.build(l.status,l.message));i.set(t,l.context),typeof l.cacheSalt=="string"&&l.cacheSalt!==""&&y.set(t,l.cacheSalt)}),e.route({method:["GET","HEAD"],url:"/",config:{public:!0},handler:c(async(t,n)=>n.send({ok:!0}),"handler")}),e.route({method:["GET","HEAD"],url:"/api/hello",config:{public:!0},handler:c(async(t,n)=>n.send({ok:!0}),"handler")}),e.get("/v1/models",async(t,n)=>{const r=new AbortController;return n.raw.on("close",()=>r.abort()),a.list({signal:r.signal})}),e.get("/v1/models/*",async(t,n)=>{const r=new AbortController;return n.raw.on("close",()=>r.abort()),a.get(t.params["*"],{signal:r.signal})});const w=a.front==="anthropic"?"/v1/messages":"/v1/chat/completions";e.post(w,{bodyLimit:u},async(t,n)=>{if(n.raw.destroyed)return n;const r=new AbortController;n.raw.on("close",()=>{n.raw.writableFinished||r.abort()});const l=typeof o.onUsage=="function"?k=>o.onUsage(k,i.get(t)):void 0,m=a.interval(t.body),_=a.send(t.body,{signal:r.signal,clientHeaders:B(t.headers),cacheSalt:y.get(t),onUsage:l,log:t.log});_.catch(()=>{});const A=new Promise(k=>r.signal.addEventListener("abort",()=>k(x),{once:!0})),h=await I(Promise.race([_,A]),t.body?.stream===!0?m:0);return h===x?(_.then(k=>k.stream?.return?.(),()=>{}),n):h===v?E(n,a,r,L(_,a,t.body?.model),m):h.stream?E(n,a,r,h.stream,m,h.status,h.headers):g(n,h)}),a.front==="openai"&&e.post("/v1/embeddings",{bodyLimit:u},async(t,n)=>{const r=new AbortController;n.raw.on("close",()=>{n.raw.writableFinished||r.abort()});const l=typeof o.onUsage=="function"?m=>o.onUsage(m,i.get(t)):void 0;return g(n,await a.embed(t.body,{signal:r.signal,onUsage:l,log:t.log}))}),a.front==="anthropic"&&e.post("/v1/messages/count_tokens",{bodyLimit:u},async(t,n)=>{const r=new AbortController;return n.raw.on("close",()=>r.abort()),a.count(t.body,{signal:r.signal})})}c(R,"register");export{L as late,R as register};
1
+ var C=Object.defineProperty;var l=(n,o)=>C(n,"name",{value:o,configurable:!0});import{timingSafeEqual as I}from"node:crypto";import{TranslatorError as x}from"./errors.mjs";import{messageId as P}from"./ids.mjs";import{sign as D}from"./thinking.mjs";import{CONTENT_TYPE as N,recorder as U}from"./metrics.mjs";import{cors as z,rateLimit as B}from"./guard.mjs";const R=64*1024*1024,G={"content-type":"text/event-stream; charset=utf-8","cache-control":"no-cache","x-accel-buffering":"no"},S=Symbol("stalled"),E=Symbol("gone");async function M(n,o){if(!(o>0))return n;let i;const d=new Promise(g=>{i=setTimeout(g,o,S)});try{return await Promise.race([n,d])}finally{clearTimeout(i)}}l(M,"settle");async function*W(n,o,i){let d=!1;try{const g=o.front==="anthropic";g&&(yield{event:"message_start",data:{type:"message_start",message:{id:P(),type:"message",role:"assistant",model:i,content:[],stop_reason:null,stop_sequence:null,usage:{input_tokens:0,output_tokens:0,cache_creation_input_tokens:0,cache_read_input_tokens:0}}}},yield{event:"content_block_start",data:{type:"content_block_start",index:0,content_block:{type:"thinking",thinking:"",signature:""}}});const a=await n,h=l(function*(){yield{event:"content_block_delta",data:{type:"content_block_delta",index:0,delta:{type:"signature_delta",signature:D()}}},yield{event:"content_block_stop",data:{type:"content_block_stop",index:0}}},"closeEarly");if(a.stream){d=!0;let u=null;for await(const c of a.stream){if(!g){yield c;continue}if(c.event!=="message_start"){if(u===null&&c.event==="content_block_start"&&c.data.content_block.type==="thinking"){u=0;continue}u===null&&(yield*h(),u=1),yield u>0&&typeof c.data?.index=="number"?{...c,data:{...c.data,index:c.data.index+u}}:c}}return}g&&(yield*h()),yield*O(o,a.body)}finally{d||n.then(g=>g.stream?.return?.(),()=>{})}}l(W,"late");function*O(n,o){if(n.front==="anthropic"){yield{event:"error",data:o};return}yield{data:o},yield{data:"[DONE]"}}l(O,"failure");async function H(n,o,i,d,g,a=200,h=G){n.hijack();const u=n.raw;u.writeHead(a,h);try{await o.write(u,d,{front:o.front,interval:g})}catch(c){if(!i.signal.aborted){const k=c instanceof x;k||(n.log.error(c),o.emit({type:"error",kind:"route",level:"error",message:String(c?.message??c),log:!1}));const{body:y}=o.build(k?c.status:502,k?c.message:"Internal server error",k?{type:c.type,code:c.code}:{});for(const m of O(o,y))u.write(o.encode(m))}}return u.end(),n}l(H,"stream");function A(n,o){const i=Buffer.from(n),d=Buffer.from(o);return i.length===d.length&&I(i,d)}l(A,"same");function T(n){const o=/^Bearer\s+(.+)$/i.exec(n.authorization??"");return n["x-api-key"]||o?.[1]||""}l(T,"keyOf");function $(n){const o={};for(const[i,d]of Object.entries(n))(i==="anthropic-beta"||i==="anthropic-version"||i.startsWith("x-claude-code-"))&&(o[i]=String(d));return o}l($,"forwardable");function F(n){if(n===!0)return{ok:!0,context:void 0};if(n&&typeof n=="object"&&n.ok===!0)return{ok:!0,context:n.context,cacheSalt:n.cacheSalt};const o=n&&typeof n=="object"&&n.ok===!1?n:{};return{ok:!1,status:Number.isInteger(o.status)&&o.status>=400&&o.status<=599?o.status:401,message:typeof o.message=="string"&&o.message!==""?o.message:"unauthorized"}}l(F,"verdictOf");function q(n,o,i){const{front:d,filter:g}=i,a={...i,build:l((e,t,r={})=>i.build(e,t,{...r,front:d}),"build"),send:l((e,t)=>i.send(e,{...t,front:d,filter:g}),"send"),route:l(e=>i.route(e,{front:d}),"route"),count:l((e,t)=>i.count(e,{...t,front:d}),"count"),embed:l((e,t)=>i.embed(e,t),"embed"),list:l(e=>i.list({...e,front:d}),"list"),get:l((e,t)=>i.get(e,{...t,front:d}),"get")},h=[o.apiKey??[]].flat().filter(Boolean),u=o.bodyLimit??R,c=new WeakMap,k=new WeakMap,y=o.metrics===!0?U({front:d,endpoints:i.endpoints}):null,m=l((e,t)=>e.code(t.status).headers(t.headers).send(t.body),"reject");n.setErrorHandler((e,t,r)=>{if(e instanceof x)return m(r,a.build(e.status,e.message,{type:e.type,code:e.code,param:e.param}));const s=e.statusCode??500;return s<500?m(r,a.build(s,e.message)):(t.log.error(e),a.emit({type:"error",kind:"route",level:"error",message:String(e.message),status:s,log:!1}),m(r,a.build(s,"Internal server error")))});const w=z(o.cors),v=B(o.rateLimit);w&&n.addHook("onRequest",async(e,t)=>{for(const[r,s]of Object.entries(w.headers(e.headers.origin)))t.raw.setHeader(r,s)}),y&&n.addHook("onRequest",y.hook),v&&n.addHook("onRequest",async(e,t)=>{if(e.routeOptions.config?.public||e.routeOptions.config?.metrics)return;const r=T(e.headers),s=v.take(r&&h.some(b=>A(r,b))?`key:${r}`:`ip:${e.ip}`);if(s===0)return;const f=a.build(429,"rate limit exceeded",{type:"rate_limit_error"});return t.code(429).headers({...f.headers,"retry-after":String(s)}).send(f.body)}),w&&n.route({method:"OPTIONS",url:"/*",config:{public:!0,metrics:!0},handler:l((e,t)=>{const r=w.preflight(e.headers.origin,e.headers["access-control-request-headers"]);for(const[s,f]of Object.entries(r))t.raw.setHeader(s,f);return t.code(204).send()},"handler")}),n.addHook("onRequest",async(e,t)=>{if(h.length===0||e.routeOptions.config?.public)return;const r=T(e.headers);if(!(r&&h.some(s=>A(r,s))))return m(t,a.build(401,"invalid x-api-key"))}),typeof o.authorize=="function"&&n.addHook("preHandler",async(e,t)=>{if(e.routeOptions.config?.public)return;let r=null;if(e.body&&typeof e.body=="object"&&!Array.isArray(e.body))try{r=a.route(e.body)}catch{}let s;try{s=F(await o.authorize({method:e.method,url:e.url,headers:e.headers,body:e.body,front:a.front,route:r}))}catch(f){e.log.error(f),a.emit({type:"error",kind:"route",level:"error",message:String(f?.message??f),log:!1}),s={ok:!1,status:401,message:"unauthorized"}}if(!s.ok)return m(t,a.build(s.status,s.message));c.set(e,s.context),typeof s.cacheSalt=="string"&&s.cacheSalt!==""&&k.set(e,s.cacheSalt)}),n.route({method:["GET","HEAD"],url:"/",config:{public:!0},handler:l(async(e,t)=>t.send({ok:!0}),"handler")}),n.route({method:["GET","HEAD"],url:"/api/hello",config:{public:!0},handler:l(async(e,t)=>t.send({ok:!0}),"handler")}),y&&n.get("/metrics",{config:{metrics:!0}},async(e,t)=>t.header("content-type",N).send(y.render())),n.get("/v1/models",async(e,t)=>{const r=new AbortController;return t.raw.on("close",()=>r.abort()),a.list({signal:r.signal})}),n.get("/v1/models/*",async(e,t)=>{const r=new AbortController;return t.raw.on("close",()=>r.abort()),a.get(e.params["*"],{signal:r.signal})});const j=a.front==="anthropic"?"/v1/messages":"/v1/chat/completions";n.post(j,{bodyLimit:u},async(e,t)=>{if(t.raw.destroyed)return t;const r=new AbortController;t.raw.on("close",()=>{t.raw.writableFinished||r.abort()});const s=typeof o.onUsage=="function"?_=>o.onUsage(_,c.get(e)):void 0,f=a.interval(e.body),b=a.send(e.body,{signal:r.signal,clientHeaders:$(e.headers),cacheSalt:k.get(e),onUsage:s,log:e.log});b.catch(()=>{}),y?.attach(e,b);const L=new Promise(_=>r.signal.addEventListener("abort",()=>_(E),{once:!0})),p=await M(Promise.race([b,L]),e.body?.stream===!0?f:0);return p===E?(b.then(_=>_.stream?.return?.(),()=>{}),t):p===S?H(t,a,r,W(b,a,e.body?.model),f):p.stream?H(t,a,r,p.stream,f,p.status,p.headers):m(t,p)}),a.front==="openai"&&n.post("/v1/embeddings",{bodyLimit:u},async(e,t)=>{const r=new AbortController;t.raw.on("close",()=>{t.raw.writableFinished||r.abort()});const s=typeof o.onUsage=="function"?b=>o.onUsage(b,c.get(e)):void 0,f=a.embed(e.body,{signal:r.signal,onUsage:s,log:e.log});return y?.attach(e,f),m(t,await f)}),a.front==="anthropic"&&n.post("/v1/messages/count_tokens",{bodyLimit:u},async(e,t)=>{const r=new AbortController;return t.raw.on("close",()=>r.abort()),a.count(e.body,{signal:r.signal})})}l(q,"register");export{W as late,q as register};
2
2
  //# sourceMappingURL=routes.mjs.map
@@ -1,2 +1,2 @@
1
- var f=Object.defineProperty;var l=(o,e)=>f(o,"name",{value:e,configurable:!0});import{TranslatorError as n}from"./errors.mjs";import{check as c}from"./constraint.mjs";import{valid as d}from"./cache.mjs";const r=Object.freeze({inlineSystem:"auto",stripBillingHeader:!0,reasoningField:"reasoning",thinkTags:!0,thinkOpen:!1,reasoningRoundTrip:!1,thinking:"chat_template_kwargs",thinkingBudget:!1,effort:!1,images:!0,imageFallback:!0,toolFallback:!1,documents:"extract",documentConverter:null,pdf:Object.freeze({pages:!0,maxPages:100,maxBytes:32*1024*1024,maxImages:20,scale:1.5}),toolSchema:"pass",deferredTools:"forward",toolSearch:"expand",jsonObject:"schema",jsonObjectFallback:!0,toolErrors:!1,toolErrorMarker:"[error] ",toolErrorInference:!1,maxTokensField:"max_completion_tokens",streamUsage:!0,continuousUsage:!1,prefill:"continue_final_message",topK:!0,responseFormat:!0,parallelToolCalls:!0,unknownFields:"drop",sampling:Object.freeze({}),constraint:null,cacheSalt:null,promptCacheKey:null,pingInterval:1e4,toolStreaming:"buffer",effortPassthrough:!1,passthroughFilter:!1,nativeMessages:!1,contextWindow:0,defaultMaxTokens:8192}),g={inlineSystem:["auto","keep","hoist","user","drop"],reasoningField:["reasoning","reasoning_content"],thinking:["chat_template_kwargs","reasoning_effort","none"],documents:["extract","note","error"],toolSchema:["pass","sanitize"],deferredTools:["forward","drop"],toolSearch:["expand","inline","native"],jsonObject:["schema","instruction","drop"],maxTokensField:["max_completion_tokens","max_tokens"],prefill:["continue_final_message","none"],unknownFields:["drop","error"],toolStreaming:["buffer","stream"]},p=["effortPassthrough","passthroughFilter","imageFallback","toolFallback","jsonObjectFallback","thinkOpen","reasoningRoundTrip","toolErrorInference"];function y(...o){const e={...r,pdf:{...r.pdf},sampling:{...r.sampling}};for(const t of o)if(t)for(const[s,a]of Object.entries(t))a!==void 0&&(s==="pdf"?Object.assign(e.pdf,a):s==="sampling"&&a!==null&&typeof a=="object"&&!Array.isArray(a)?Object.assign(e.sampling,a):e[s]=a);for(const[t,s]of Object.entries(g))if(!s.includes(e[t]))throw new n(`Invalid setting ${t}: ${JSON.stringify(e[t])} (expected one of ${s.join(", ")})`,{code:"invalid_setting",status:500,details:{key:t,allowed:s}});for(const t of p)if(typeof e[t]!="boolean")throw new n(`Invalid setting ${t}: ${JSON.stringify(e[t])} (expected true or false)`,{code:"invalid_setting",status:500,details:{key:t}});if(e.nativeMessages!==!0&&e.nativeMessages!==!1&&e.nativeMessages!=="auto")throw new n(`Invalid setting nativeMessages: ${JSON.stringify(e.nativeMessages)} (expected true, false or "auto")`,{code:"invalid_setting",status:500,details:{key:"nativeMessages"}});if(typeof e.toolErrorMarker!="string"||e.toolErrorMarker.trim()==="")throw new n(`Invalid setting toolErrorMarker: ${JSON.stringify(e.toolErrorMarker)} (expected a non-empty string)`,{code:"invalid_setting",status:500,details:{key:"toolErrorMarker"}});if(e.sampling===null||typeof e.sampling!="object"||Array.isArray(e.sampling))throw new n(`Invalid setting sampling: ${JSON.stringify(e.sampling)} (expected an object of field names and values)`,{code:"invalid_setting",status:500,details:{key:"sampling"}});const i=c(e.constraint);if(i!==null)throw new n(`Invalid setting constraint: ${JSON.stringify(e.constraint)} (${i})`,{code:"invalid_setting",status:500,details:{key:"constraint"}});for(const t of["cacheSalt","promptCacheKey"])if(!d(e[t]))throw new n(`Invalid setting ${t}: ${JSON.stringify(e[t])} (expected a non-empty string or null)`,{code:"invalid_setting",status:500,details:{key:t}});if(!(Number.isFinite(e.pdf.scale)&&e.pdf.scale>0))throw new n(`Invalid setting pdf.scale: ${JSON.stringify(e.pdf.scale)} (expected a positive number)`,{code:"invalid_setting",status:500,details:{key:"pdf.scale"}});return e}l(y,"resolve");export{r as DEFAULTS,y as resolve};
1
+ var c=Object.defineProperty;var l=(o,e)=>c(o,"name",{value:e,configurable:!0});import{TranslatorError as n}from"./errors.mjs";import{check as f}from"./constraint.mjs";import{valid as d}from"./cache.mjs";const r=Object.freeze({inlineSystem:"auto",stripBillingHeader:!0,reasoningField:"reasoning",thinkTags:!0,thinkOpen:!1,reasoningRoundTrip:!1,thinking:"chat_template_kwargs",thinkingBudget:!1,effort:!1,images:!0,imageFallback:!0,toolFallback:!1,documents:"extract",documentConverter:null,pdf:Object.freeze({pages:!0,maxPages:100,maxBytes:32*1024*1024,maxImages:20,scale:1.5}),toolSchema:"pass",deferredTools:"forward",toolSearch:"expand",jsonObject:"schema",jsonObjectFallback:!0,toolErrors:!0,toolErrorMarker:"[error] ",toolErrorInference:!1,maxTokensField:"max_completion_tokens",streamUsage:!0,continuousUsage:!1,prefill:"continue_final_message",topK:!0,responseFormat:!0,parallelToolCalls:!0,unknownFields:"drop",sampling:Object.freeze({}),constraint:null,cacheSalt:null,promptCacheKey:null,pingInterval:1e4,toolStreaming:"buffer",effortPassthrough:!1,passthroughFilter:!1,nativeMessages:!1,contextWindow:0,defaultMaxTokens:8192}),g={inlineSystem:["auto","keep","hoist","user","drop"],reasoningField:["reasoning","reasoning_content"],thinking:["chat_template_kwargs","reasoning_effort","none"],documents:["extract","note","error"],toolSchema:["pass","sanitize"],deferredTools:["forward","drop"],toolSearch:["expand","inline","native"],jsonObject:["schema","instruction","drop"],maxTokensField:["max_completion_tokens","max_tokens"],prefill:["continue_final_message","none"],unknownFields:["drop","error"],toolStreaming:["buffer","stream"]},p=["effortPassthrough","passthroughFilter","imageFallback","toolFallback","jsonObjectFallback","thinkOpen","reasoningRoundTrip","toolErrorInference"];function y(...o){const e={...r,pdf:{...r.pdf},sampling:{...r.sampling}};for(const t of o)if(t)for(const[s,a]of Object.entries(t))a!==void 0&&(s==="pdf"?Object.assign(e.pdf,a):s==="sampling"&&a!==null&&typeof a=="object"&&!Array.isArray(a)?Object.assign(e.sampling,a):e[s]=a);for(const[t,s]of Object.entries(g))if(!s.includes(e[t]))throw new n(`Invalid setting ${t}: ${JSON.stringify(e[t])} (expected one of ${s.join(", ")})`,{code:"invalid_setting",status:500,details:{key:t,allowed:s}});for(const t of p)if(typeof e[t]!="boolean")throw new n(`Invalid setting ${t}: ${JSON.stringify(e[t])} (expected true or false)`,{code:"invalid_setting",status:500,details:{key:t}});if(e.nativeMessages!==!0&&e.nativeMessages!==!1&&e.nativeMessages!=="auto")throw new n(`Invalid setting nativeMessages: ${JSON.stringify(e.nativeMessages)} (expected true, false or "auto")`,{code:"invalid_setting",status:500,details:{key:"nativeMessages"}});if(typeof e.toolErrorMarker!="string"||e.toolErrorMarker.trim()==="")throw new n(`Invalid setting toolErrorMarker: ${JSON.stringify(e.toolErrorMarker)} (expected a non-empty string)`,{code:"invalid_setting",status:500,details:{key:"toolErrorMarker"}});if(e.sampling===null||typeof e.sampling!="object"||Array.isArray(e.sampling))throw new n(`Invalid setting sampling: ${JSON.stringify(e.sampling)} (expected an object of field names and values)`,{code:"invalid_setting",status:500,details:{key:"sampling"}});const i=f(e.constraint);if(i!==null)throw new n(`Invalid setting constraint: ${JSON.stringify(e.constraint)} (${i})`,{code:"invalid_setting",status:500,details:{key:"constraint"}});for(const t of["cacheSalt","promptCacheKey"])if(!d(e[t]))throw new n(`Invalid setting ${t}: ${JSON.stringify(e[t])} (expected a non-empty string or null)`,{code:"invalid_setting",status:500,details:{key:t}});if(!(Number.isFinite(e.pdf.scale)&&e.pdf.scale>0))throw new n(`Invalid setting pdf.scale: ${JSON.stringify(e.pdf.scale)} (expected a positive number)`,{code:"invalid_setting",status:500,details:{key:"pdf.scale"}});return e}l(y,"resolve");export{r as DEFAULTS,y as resolve};
2
2
  //# sourceMappingURL=settings.mjs.map
package/dist/lib/vllm.mjs CHANGED
@@ -1,4 +1,4 @@
1
- var c=Object.defineProperty;var s=(t,r)=>c(t,"name",{value:r,configurable:!0});import{atLeast as f,semver as l}from"./normalize.mjs";import{sign as a}from"./thinking.mjs";const u="0.11.1",y="x-anthropic-billing-header:";function A(t,r){if(t.driver!=="vllm"||r.nativeMessages===!1)return!1;if(r.nativeMessages===!0)return!0;const n=t.facts?.version??t.version;return l(n)!==null&&f(n,u)}s(A,"wants");function*p(t){for(const r of t)if(Array.isArray(r?.content))for(const n of r.content)yield n,n?.type==="tool_result"&&Array.isArray(n.content)&&(yield*n.content)}s(p,"blocks");function v(t,r){const n=Array.isArray(t.messages)?t.messages:[];if(n.at(-1)?.role==="assistant")return"prefill";const e=[...p(n)];return e.some(o=>o?.type==="document")?"document":r.toolSearch!=="native"&&(Array.isArray(t.tools)&&t.tools.some(i=>i?.defer_loading!==void 0)||e.some(i=>["tool_reference","tool_addition","tool_removal"].includes(i?.type)))?"toolsearch":null}s(v,"unsuited");function d(t,r){return(typeof t=="string"?[{type:"text",text:t}]:t).filter(e=>e?.type==="text"&&typeof e.text=="string").filter(e=>!(r.stripBillingHeader&&e.text.trimStart().startsWith(y))).map(e=>e.text).join(`
1
+ var c=Object.defineProperty;var o=(t,n)=>c(t,"name",{value:n,configurable:!0});import{atLeast as f,semver as l}from"./normalize.mjs";import{sign as a}from"./thinking.mjs";const u="0.11.1",y="x-anthropic-billing-header:";function A(t,n){if(t.driver!=="vllm"||n.nativeMessages===!1)return!1;if(n.nativeMessages===!0)return!0;const e=t.facts?.version??t.version;return l(e)!==null&&f(e,u)}o(A,"wants");function*p(t){for(const n of t)if(Array.isArray(n?.content))for(const e of n.content)yield e,e?.type==="tool_result"&&Array.isArray(e.content)&&(yield*e.content)}o(p,"blocks");function v(t,n){const e=Array.isArray(t.messages)?t.messages:[];if(e.at(-1)?.role==="assistant")return"prefill";const r=[...p(e)];return r.some(s=>s?.type==="document")?"document":n.toolSearch!=="native"&&(Array.isArray(t.tools)&&t.tools.some(i=>i?.defer_loading!==void 0)||r.some(i=>["tool_reference","tool_addition","tool_removal"].includes(i?.type)))?"toolsearch":null}o(v,"unsuited");function d(t,n){return(typeof t=="string"?[{type:"text",text:t}]:t).filter(r=>r?.type==="text"&&typeof r.text=="string").filter(r=>!(n.stripBillingHeader&&r.text.trimStart().startsWith(y))).map(r=>r.text).join(`
2
2
 
3
- `)}s(d,"systemText");function m(t,r){const n=r.toolErrorMarker;return typeof t.content=="string"?{...t,content:`${n}${t.content}`}:Array.isArray(t.content)?{...t,content:[{type:"text",text:n.trimEnd()},...t.content]}:{...t,content:n.trimEnd()}}s(m,"marked");function h(t,r){const n={...t};if(n.system!==void 0&&n.system!==null){const e=d(n.system,r);e===""?delete n.system:n.system=e}return r.toolErrors&&Array.isArray(n.messages)&&(n.messages=n.messages.map(e=>Array.isArray(e?.content)?{...e,content:e.content.map(o=>o?.type==="tool_result"&&o.is_error===!0?m(o,r):o)}:e)),n}s(h,"request");function E(t){return Array.isArray(t?.content)?{...t,content:t.content.map(r=>r?.type==="thinking"&&typeof r.signature!="string"?{...r,signature:a()}:r)}:t}s(E,"response");async function*M(t){const r=new Map;for await(const n of t){const e=n.data;e?.type==="content_block_start"&&e.content_block?.type==="thinking"?r.set(e.index,typeof e.content_block.signature=="string"):e?.type==="content_block_delta"&&e.delta?.type==="signature_delta"?r.set(e.index,!0):e?.type==="content_block_stop"&&r.get(e.index)===!1&&(yield{event:"content_block_delta",data:{type:"content_block_delta",index:e.index,delta:{type:"signature_delta",signature:a()}}}),e?.type==="content_block_stop"&&r.delete(e.index),yield n,e?.type==="message_start"&&(yield{event:"ping",data:{type:"ping"}})}}s(M,"stream");export{u as MIN_VERSION,h as request,E as response,M as stream,v as unsuited,A as wants};
3
+ `)}o(d,"systemText");function m(t,n){const e=n.toolErrorMarker;return typeof t.content=="string"?{...t,content:`${e}${t.content}`}:Array.isArray(t.content)?{...t,content:[{type:"text",text:e.trimEnd()},...t.content]}:{...t,content:e.trimEnd()}}o(m,"marked");function h(t,n){const e={...t};if(e.system!==void 0&&e.system!==null){const s=d(e.system,n);s===""?delete e.system:e.system=s}const r=o(s=>s?.type==="tool_result"&&s.is_error===!0,"failed");return n.toolErrors&&Array.isArray(e.messages)&&e.messages.some(s=>Array.isArray(s?.content)&&s.content.some(r))&&(e.messages=e.messages.map(s=>Array.isArray(s?.content)&&s.content.some(r)?{...s,content:s.content.map(i=>r(i)?m(i,n):i)}:s)),e}o(h,"request");function E(t){return Array.isArray(t?.content)?{...t,content:t.content.map(n=>n?.type==="thinking"&&typeof n.signature!="string"?{...n,signature:a()}:n)}:t}o(E,"response");async function*M(t){const n=new Map;for await(const e of t){const r=e.data;r?.type==="content_block_start"&&r.content_block?.type==="thinking"?n.set(r.index,typeof r.content_block.signature=="string"):r?.type==="content_block_delta"&&r.delta?.type==="signature_delta"?n.set(r.index,!0):r?.type==="content_block_stop"&&n.get(r.index)===!1&&(yield{event:"content_block_delta",data:{type:"content_block_delta",index:r.index,delta:{type:"signature_delta",signature:a()}}}),r?.type==="content_block_stop"&&n.delete(r.index),yield e,r?.type==="message_start"&&(yield{event:"ping",data:{type:"ping"}})}}o(M,"stream");export{u as MIN_VERSION,h as request,E as response,M as stream,v as unsuited,A as wants};
4
4
  //# sourceMappingURL=vllm.mjs.map
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@cldmv/llm-translator",
3
- "version": "0.11.0",
3
+ "version": "0.12.0",
4
4
  "description": "Bidirectional Anthropic Messages ⇄ OpenAI Chat Completions translator with backend auto-detection (vLLM and other OpenAI-compatible servers), built on slothlet, with an optional standalone server for running Claude Code against local models.",
5
5
  "type": "module",
6
6
  "license": "Apache-2.0",
@@ -5,7 +5,7 @@
5
5
  * `app.register(api.server.plugin, { apiKey })`. `authorize(request)` (sync or async) runs after `apiKey` and the body parse and decides
6
6
  * with `true` / `{ ok: true, context }` or `false` / `{ ok: false, status, message }`; `onUsage(report, context)` receives each request's usage report. When the Fastify `logger` is enabled, events also go to it as structured lines (`{ event }` plus the message), through the request's logger for events of a request.
7
7
  * @param {any} fastify - Fastify instance.
8
- * @param {any} [options] - `{ front, apiKey, bodyLimit, authorize, onUsage }`. `authorize(request)` and `onUsage(report, context)` are described above. `front` sets the protocol this plugin instance serves (default: the translator's front, other plugin instances and `api.send` are unaffected); `apiKey` (string or array) is required from clients. Forwarded OpenAI bodies are restricted to known fields unless the `passthroughFilter` setting is `false`.
8
+ * @param {any} [options] - `{ front, apiKey, bodyLimit, authorize, onUsage, metrics, cors, rateLimit }`. `cors` (a list of origins, `"*"` for any) and `rateLimit` (`{ requests, windowMs }`) are described in docs/server.md and are off unless set. `metrics: true` adds `GET /metrics` (Prometheus text format, behind the same authentication as every other route; off by default, then an unknown route). `authorize(request)` and `onUsage(report, context)` are described above. `front` sets the protocol this plugin instance serves (default: the translator's front, other plugin instances and `api.send` are unaffected); `apiKey` (string or array) is required from clients. Forwarded OpenAI bodies are restricted to known fields unless the `passthroughFilter` setting is `false`.
9
9
  * @returns {Promise<void>} Resolves once the routes are registered.
10
10
  */
11
11
  export function plugin(fastify: any, options?: any): Promise<void>;
@@ -1,6 +1,6 @@
1
1
  /**
2
2
  * Start an HTTP server in front of the translator. Fastify is an optional peer dependency and is imported here.
3
- * @param {any} [options] - `{ host = "127.0.0.1", port = 8787, front, apiKey, bodyLimit, logger, allowUnauthenticated, authorize, onUsage }`. `authorize(request)` and `onUsage(report, context)` are described at `server.plugin`. Listening on a non-loopback address without an `apiKey` or an `authorize` hook is refused unless `allowUnauthenticated` is true.
3
+ * @param {any} [options] - `{ host = "127.0.0.1", port = 8787, front, apiKey, bodyLimit, logger, allowUnauthenticated, authorize, onUsage, metrics, tls, cors, rateLimit }`. `authorize(request)` and `onUsage(report, context)` are described at `server.plugin`. `tls` is `{ key, cert, ca, passphrase }` (file paths; `key` and `cert` together) and serves HTTPS; `cors` is a list of allowed origins (`"*"` for any); `rateLimit` is `{ requests, windowMs = 60000 }` per API key (remote address without one). Each is off unless set; see docs/server.md. Listening on a non-loopback address without an `apiKey` or an `authorize` hook is refused unless `allowUnauthenticated` is true.
4
4
  * @returns {Promise<{url: string, host: string, port: number, app: any, close: () => Promise<void>}>} Running server handle.
5
5
  */
6
6
  export function start(options?: any): Promise<{
@@ -0,0 +1,25 @@
1
+ /**
2
+ * Read the TLS files named by `options.tls` into the `https` options of Fastify / `node:https`.
3
+ * @param {any} tls - `{ key, cert, ca, passphrase }`: file paths (`ca` may be a list), `passphrase` a string; or undefined for plain HTTP.
4
+ * @returns {Promise<object | null>} `https` options, or null when TLS is not configured.
5
+ */
6
+ export function tlsOptions(tls: any): Promise<object | null>;
7
+ /**
8
+ * Build the CORS policy.
9
+ * @param {any} option - A list of origins, or `{ origins }`. `"*"` allows every origin. Undefined, null or an empty list turns CORS off.
10
+ * @returns {{ headers: (origin: string | undefined) => Record<string,string>, preflight: (origin: string | undefined, requested: string | undefined) => Record<string,string> } | null} Policy, or null when off.
11
+ */
12
+ export function cors(option: any): {
13
+ headers: (origin: string | undefined) => Record<string, string>;
14
+ preflight: (origin: string | undefined, requested: string | undefined) => Record<string, string>;
15
+ } | null;
16
+ /**
17
+ * Build a fixed-window rate limiter.
18
+ * @param {any} option - `{ requests, windowMs = 60000 }` (requests per window per client); undefined, null or false turns it off.
19
+ * @param {() => number} [now] - Clock in ms (tests).
20
+ * @returns {{ take: (key: string) => number, size: () => number } | null} Limiter, or null when off. `take(key)` returns 0 when the request may go on, else the seconds to wait.
21
+ */
22
+ export function rateLimit(option: any, now?: () => number): {
23
+ take: (key: string) => number;
24
+ size: () => number;
25
+ } | null;
@@ -0,0 +1,58 @@
1
+ /**
2
+ * Escape a label value: backslash, double quote and line feed.
3
+ * @param {any} value - Label value.
4
+ * @returns {string} Escaped text.
5
+ */
6
+ export function escapeLabel(value: any): string;
7
+ /**
8
+ * Escape `# HELP` text: backslash and line feed.
9
+ * @param {string} text - Help text.
10
+ * @returns {string} Escaped text.
11
+ */
12
+ export function escapeHelp(text: string): string;
13
+ /**
14
+ * Create an empty registry.
15
+ * @returns {{ counter: (name: string, help: string) => void, histogram: (name: string, help: string, buckets?: number[]) => void, inc: (name: string, labels: Record<string, any>, by?: number) => void, observe: (name: string, labels: Record<string, any>, value: number) => void, render: (gauges?: {name: string, help: string, samples: {labels: Record<string, any>, value: number}[]}[]) => string }} Registry.
16
+ */
17
+ export function create(): {
18
+ counter: (name: string, help: string) => void;
19
+ histogram: (name: string, help: string, buckets?: number[]) => void;
20
+ inc: (name: string, labels: Record<string, any>, by?: number) => void;
21
+ observe: (name: string, labels: Record<string, any>, value: number) => void;
22
+ render: (gauges?: {
23
+ name: string;
24
+ help: string;
25
+ samples: {
26
+ labels: Record<string, any>;
27
+ value: number;
28
+ }[];
29
+ }[]) => string;
30
+ };
31
+ /**
32
+ * The kind of an error, or null when the request did not fail: `upstream` (the upstream answered with an error or could not be reached
33
+ * as far as the usage report knows), `translator` (a failure in the translator itself, a 5xx or an in-band stream error) and `client`
34
+ * (a refused request: bad body, authentication, unknown model or route). A client that left is not an error.
35
+ * @param {number} status - HTTP status as written (499 when the client left).
36
+ * @param {any} report - The request's usage report, or null.
37
+ * @returns {"upstream" | "translator" | "client" | null} Kind.
38
+ */
39
+ export function kind(status: number, report: any): "upstream" | "translator" | "client" | null;
40
+ /**
41
+ * Create the metrics of one server: a registry plus the request recorder and the scrape renderer. Label values stay bounded: `front` is
42
+ * one of two protocols, `route` is the route pattern (never the URL), `driver` is a driver name, `upstream` a configured upstream name
43
+ * and `code` an HTTP status. The model a client asked for, request ids and API keys are never labels.
44
+ * @param {{ front: "anthropic" | "openai", endpoints: () => any[] }} source - The server's front and its runtime endpoints.
45
+ * @returns {{ hook: (request: any, reply: any) => Promise<void>, attach: (request: any, sending: Promise<any>) => void, render: () => string }} Recorder.
46
+ */
47
+ export function recorder(source: {
48
+ front: "anthropic" | "openai";
49
+ endpoints: () => any[];
50
+ }): {
51
+ hook: (request: any, reply: any) => Promise<void>;
52
+ attach: (request: any, sending: Promise<any>) => void;
53
+ render: () => string;
54
+ };
55
+ /** `Content-Type` of the text exposition format. */
56
+ export const CONTENT_TYPE: "text/plain; version=0.0.4; charset=utf-8";
57
+ /** Request duration buckets in seconds: LLM replies run from milliseconds (errors, embeddings) to minutes (long streams). */
58
+ export const BUCKETS: number[];
@@ -10,7 +10,7 @@ export function late(sending: Promise<any>, api: any, model?: string): AsyncGene
10
10
  /**
11
11
  * Register the routes for one front on a Fastify instance.
12
12
  * @param {any} fastify - Fastify instance (usually an encapsulated plugin scope).
13
- * @param {any} options - `{ apiKey, bodyLimit, authorize, onUsage }`.
13
+ * @param {any} options - `{ apiKey, bodyLimit, authorize, onUsage, metrics, cors, rateLimit }`.
14
14
  * @param {any} captured - Captured translator parts `{ front, interval(body), filter, send, route, count, list, get, build, encode, write }`; `front` applies to this instance only.
15
15
  * @returns {void}
16
16
  */
@@ -28,9 +28,9 @@ export function resolve(...layers: (Partial<Settings> | null | undefined)[]): Se
28
28
  * @property {"expand"|"inline"|"native"} toolSearch - Claude Code's tool search. `expand` offers a deferred tool once a `tool_reference` (or a `ToolSearch` result) loads it; `inline` offers every deferred tool up front and hides `ToolSearch` (it stays when `DeferredToolPlaceholder` shows Claude Code holds tools whose definitions it did not send; `expand` does the same when the request has no `ToolSearch` at all); `native` passes `defer_loading` and `tool_reference` through to a server that understands them.
29
29
  * @property {"schema"|"instruction"|"drop"} jsonObject - OpenAI to Anthropic: how `response_format: json_object` is sent. `schema` sends `output_config.format` with the permissive schema `{ "type": "object" }`, `instruction` appends a short system instruction asking for one JSON object, `drop` omits it and records the drop.
30
30
  * @property {boolean} jsonObjectFallback - Retry a request once with `jsonObject: "instruction"` when an Anthropic-protocol upstream answers 400 or 422 to the schema `jsonObject: "schema"` sent. The endpoint and model are remembered until the process ends only when the retry succeeds.
31
- * @property {boolean} toolErrors - Prefix `is_error` tool results with `toolErrorMarker`. Off by default for every driver.
31
+ * @property {boolean} toolErrors - Prefix `is_error` tool results with `toolErrorMarker` when converting to Chat Completions (which has no error flag) and on vLLM native passthrough. On by default for every driver; set false to send the text unchanged. Upstreams with a native error flag (Anthropic-protocol) never go through this.
32
32
  * @property {string} toolErrorMarker - Text that marks a failed tool result when `toolErrors` is on, and the text `toolErrorInference` looks for. Must contain a non-whitespace character.
33
- * @property {boolean} toolErrorInference - OpenAI to Anthropic: a `tool` message whose text starts with `toolErrorMarker` becomes a `tool_result` with `is_error: true`, with the marker removed from the text.
33
+ * @property {boolean} toolErrorInference - OpenAI to Anthropic: a `tool` message whose text starts with `toolErrorMarker` becomes a `tool_result` with `is_error: true`, with the marker removed from the text. Off by default: a tool that merely prints the marker would be taken for a failure.
34
34
  * @property {"max_completion_tokens"|"max_tokens"} maxTokensField - Field used for the output cap.
35
35
  * @property {boolean} streamUsage - Request `stream_options.include_usage` on streamed requests.
36
36
  * @property {boolean} continuousUsage - Also request `stream_options.continuous_usage_stats` (vLLM extension; real OpenAI may reject it).
@@ -144,7 +144,7 @@ export type Settings = {
144
144
  */
145
145
  jsonObjectFallback: boolean;
146
146
  /**
147
- * - Prefix `is_error` tool results with `toolErrorMarker`. Off by default for every driver.
147
+ * - Prefix `is_error` tool results with `toolErrorMarker` when converting to Chat Completions (which has no error flag) and on vLLM native passthrough. On by default for every driver; set false to send the text unchanged. Upstreams with a native error flag (Anthropic-protocol) never go through this.
148
148
  */
149
149
  toolErrors: boolean;
150
150
  /**
@@ -152,7 +152,7 @@ export type Settings = {
152
152
  */
153
153
  toolErrorMarker: string;
154
154
  /**
155
- * - OpenAI to Anthropic: a `tool` message whose text starts with `toolErrorMarker` becomes a `tool_result` with `is_error: true`, with the marker removed from the text.
155
+ * - OpenAI to Anthropic: a `tool` message whose text starts with `toolErrorMarker` becomes a `tool_result` with `is_error: true`, with the marker removed from the text. Off by default: a tool that merely prints the marker would be taken for a failure.
156
156
  */
157
157
  toolErrorInference: boolean;
158
158
  /**