@cldmv/llm-translator 0.10.0 → 0.12.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (46) hide show
  1. package/README.md +6 -8
  2. package/dist/api/convert/anthropic/messages/to/openai/chat/request.mjs +10 -10
  3. package/dist/api/convert/openai/chat/to/anthropic/messages/response.mjs +1 -1
  4. package/dist/api/convert/openai/chat/to/anthropic/messages/stream.mjs +1 -1
  5. package/dist/api/detect.mjs +1 -1
  6. package/dist/api/driver/azure/detect.mjs +1 -1
  7. package/dist/api/driver/azure/info.mjs +1 -1
  8. package/dist/api/driver/deepseek/info.mjs +1 -1
  9. package/dist/api/driver/llamacpp/info.mjs +1 -1
  10. package/dist/api/driver/mistral/info.mjs +1 -1
  11. package/dist/api/driver/openai/info.mjs +1 -1
  12. package/dist/api/driver/openrouter/info.mjs +1 -1
  13. package/dist/api/request.mjs +2 -2
  14. package/dist/api/send.mjs +1 -1
  15. package/dist/api/server/plugin.mjs +1 -1
  16. package/dist/api/server/start.mjs +1 -1
  17. package/dist/lib/cache.mjs +2 -0
  18. package/dist/lib/cli.mjs +16 -7
  19. package/dist/lib/constraint.mjs +2 -0
  20. package/dist/lib/context.mjs +2 -0
  21. package/dist/lib/guard.mjs +2 -0
  22. package/dist/lib/metrics.mjs +4 -0
  23. package/dist/lib/passthrough.mjs +1 -1
  24. package/dist/lib/presets.mjs +1 -1
  25. package/dist/lib/routes.mjs +1 -1
  26. package/dist/lib/settings.mjs +1 -1
  27. package/dist/lib/vllm.mjs +2 -2
  28. package/package.json +1 -1
  29. package/types/src/api/detect.d.mts +1 -1
  30. package/types/src/api/driver/azure/info.d.mts +2 -1
  31. package/types/src/api/driver/deepseek/info.d.mts +2 -1
  32. package/types/src/api/driver/mistral/info.d.mts +2 -1
  33. package/types/src/api/driver/openai/info.d.mts +3 -1
  34. package/types/src/api/driver/openrouter/info.d.mts +2 -1
  35. package/types/src/api/request.d.mts +1 -1
  36. package/types/src/api/send.d.mts +1 -1
  37. package/types/src/api/server/plugin.d.mts +1 -1
  38. package/types/src/api/server/start.d.mts +1 -1
  39. package/types/src/lib/cache.d.mts +30 -0
  40. package/types/src/lib/constraint.d.mts +36 -0
  41. package/types/src/lib/context.d.mts +34 -0
  42. package/types/src/lib/guard.d.mts +25 -0
  43. package/types/src/lib/metrics.d.mts +58 -0
  44. package/types/src/lib/presets.d.mts +2 -1
  45. package/types/src/lib/routes.d.mts +1 -1
  46. package/types/src/lib/settings.d.mts +24 -4
package/README.md CHANGED
@@ -17,20 +17,18 @@ Use it as a library when you want the conversion without any I/O (`request`, `re
17
17
 
18
18
  ## ✨ What's New
19
19
 
20
- ### Latest: v0.10.0 (November 2026)
20
+ ### Latest: v0.12.0 (November 2026)
21
21
 
22
- - **Native `/v1/messages` for vLLM** — set `nativeMessages` (`true` or `"auto"`) and Anthropic requests go to vLLM's own endpoint with only normalisation. Prefills, document blocks and deferred tools are still converted, and an upstream without the route falls back to conversion.
23
- - **Readiness probes and LoRA adapters** — pools check vLLM's `/health` and try ready upstreams first (`pool.healthInterval`, `pool.healthTimeout`, a `health` event, `health.check()`). Adapters are routable and carry `parent` in model lists.
24
- - **Hosted provider drivers** — Groq, Together, Fireworks, xAI and Gemini are bound by host name, with their token fields, reasoning fields and rejected request fields handled from one table.
25
- - **Ollama's native `/api/chat`** — set `api: "native"` on an Ollama upstream to use `think`, `options` and `format`, with NDJSON streaming converted for both fronts.
26
- - [View full v0.10.0 Changelog](https://github.com/CLDMV/llm-translator/blob/master/docs/changelog/v0/v0.10.0.md)
22
+ - **Production server features** — an optional Prometheus `/metrics` endpoint (request counts and durations, tokens, errors and upstream health), plus HTTPS, per-key rate limits with a front-shaped 429, and CORS for browser clients. All are off unless set, with CLI flags and environment variables for each.
23
+ - **Failed tool results marked by default** — a `tool_result` with `is_error: true` sent to a Chat Completions upstream now carries `toolErrorMarker`, so the model knows the tool failed. Set `toolErrors: false` to turn it off.
24
+ - [View full v0.12.0 Changelog](https://github.com/CLDMV/llm-translator/blob/master/docs/changelog/v0/v0.12.0.md)
27
25
 
28
26
  ### Recent Releases
29
27
 
28
+ - **v0.11.0** (November 2026) — native constrained decoding (JSON Schema, regex, choice, grammar), `clear_tool_uses` context edits, `cache_salt` and `prompt_cache_key` hints, and host-first binding for hosted APIs ([Changelog](https://github.com/CLDMV/llm-translator/blob/master/docs/changelog/v0/v0.11.0.md))
29
+ - **v0.10.0** (November 2026) — native `/v1/messages` for vLLM, readiness probes and LoRA adapters, Groq, Together, Fireworks, xAI and Gemini drivers, and Ollama's native `/api/chat` ([Changelog](https://github.com/CLDMV/llm-translator/blob/master/docs/changelog/v0/v0.10.0.md))
30
30
  - **v0.9.0** (November 2026) — Azure OpenAI, OpenRouter, Hugging Face TGI and LocalAI drivers, with Azure's `apiVersion` and `deployment` upstream fields and more startup probes ([Changelog](https://github.com/CLDMV/llm-translator/blob/master/docs/changelog/v0/v0.9.0.md))
31
31
  - **v0.8.0** (November 2026) — Mistral and LM Studio drivers, an OpenAI client's `n` above 1 answered by parallel requests to Anthropic-protocol upstreams, and `stop_details.category` from content-filter results ([Changelog](https://github.com/CLDMV/llm-translator/blob/master/docs/changelog/v0/v0.8.0.md))
32
- - **v0.7.0** (November 2026) — Anthropic-protocol servers detected by probe on any host, tool-call ids rewritten for strict upstream formats, embeddings usage through `onUsage`, and capture records for every failed-over attempt ([Changelog](https://github.com/CLDMV/llm-translator/blob/master/docs/changelog/v0/v0.7.0.md))
33
- - **v0.6.0** (November 2026) — `POST /v1/embeddings` pass-through, sampling extensions per upstream and per request, `strict` tool use end to end with a schema check, and the `capture` option for debugging ([Changelog](https://github.com/CLDMV/llm-translator/blob/master/docs/changelog/v0/v0.6.0.md))
34
32
 
35
33
  📚 **For complete version history and detailed release notes, see the [docs/changelog/](https://github.com/CLDMV/llm-translator/tree/master/docs/changelog/) folder.**
36
34
 
@@ -1,17 +1,17 @@
1
- var M=Object.defineProperty;var g=(e,t)=>M(e,"name",{value:t,configurable:!0});import{resolve as O}from"../../../../../../../lib/settings.mjs";import{TranslatorError as j}from"../../../../../../../lib/errors.mjs";import{createIdFitter as q,createNameFitter as B}from"../../../../../../../lib/ids.mjs";import{sanitize as C}from"../../../../../../../lib/schema.mjs";import{NOTE as A}from"../../../../../../../lib/images.mjs";import{expand as L}from"../../../../../../../lib/documents.mjs";import{createBudget as z,unsign as D}from"../../../../../../../lib/thinking.mjs";import{plan as P,scan as W}from"../../../../../../../lib/toolsearch.mjs";const H=new Set(["model","messages","system","max_tokens","stream","temperature","top_p","top_k","stop_sequences","tools","tool_choice","thinking","output_config","output_format","context_management"]),K=new Set(["metadata","cache_control","service_tier"]),R=new Set(["mcp_servers","container","inference_geo"]),U="x-anthropic-billing-header:";async function pe(e,t,o){const n=O(t);if(e===null||typeof e!="object"||!Array.isArray(e.messages))throw new j("messages: Field required",{code:"invalid_request",param:"messages"});const s={front:"anthropic",model:e.model,stream:e.stream===!0,thinking:e.thinking?.type??null,reasoningField:n.reasoningField,toolNames:{},toolIds:{},dropped:[]};e.thinking?.type==="enabled"&&Number.isInteger(e.thinking.budget_tokens)&&e.output_config?.effort===void 0&&(s.thinkingBudget=e.thinking.budget_tokens),o?.producer!==void 0&&(s.producer=o.producer);const i=B({max:64,map:s.toolNames}),d=o?.toolCallIds?q({policy:o.toolCallIds,ids:G(e.messages),map:s.toolIds}):c=>c;for(const c of e.tools??[])(c.type===void 0||c.type==="custom")&&i(c.name);const m=W(e.messages,new Set((e.tools??[]).map(c=>c.name))),a=P(e.tools??[],m,n,e.tool_choice);s.toolSearch={mode:n.toolSearch,loaded:a.loaded,hidden:a.hidden,stubbed:a.stubbed,capped:m.capped};for(const c of Object.keys(e))if(!(H.has(c)||K.has(c)))if(R.has(c))x(s,c);else{if(n.unknownFields==="error")throw new j(`${c}: Extra inputs are not permitted`,{code:"unsupported_field",param:c});x(s,c)}const u={model:e.model},$=E(e.system,n),f=await J(e.messages,n,s,i,d,$,Q(e.context_management,s),o?.budget??z());u.messages=f.messages,f.prefill&&(u.continue_final_message=!0,u.add_generation_prompt=!1),e.max_tokens!==void 0&&(u[n.maxTokensField]=e.max_tokens),e.temperature!==void 0&&(u.temperature=e.temperature),e.top_p!==void 0&&(u.top_p=e.top_p),e.top_k!==void 0&&(n.topK?u.top_k=e.top_k:x(s,"top_k")),e.stop_sequences?.length>0&&(u.stop=e.stop_sequences),s.stream&&(u.stream=!0,n.streamUsage&&(u.stream_options={include_usage:!0},n.continuousUsage&&(u.stream_options.continuous_usage_stats=!0)));const h=b(a.offer,n,s,i);if(h.length>0){u.tools=h;const c=ee(e.tool_choice,n,s,i);c!==void 0&&(u.tool_choice=c),e.tool_choice?.disable_parallel_tool_use===!0&&n.parallelToolCalls&&(u.parallel_tool_calls=!1)}return te(e,n,u,o?.listed===!0),ne(e,n,s,u),{body:u,state:s}}g(pe,"request");function x(e,t){e.dropped.includes(t)||e.dropped.push(t)}g(x,"drop");function F(e,t,o){if(t.unknownFields==="error")throw new j(`${o}: is not supported by this translator`,{code:"unsupported_content",param:o});x(e,o)}g(F,"unsupported");function E(e,t){return e==null?"":(typeof e=="string"?[{type:"text",text:e}]:e).filter(n=>n.type==="text"&&!(t.stripBillingHeader&&n.text.trimStart().startsWith(U))).map(n=>n.text).join(`
1
+ var O=Object.defineProperty;var d=(e,n)=>O(e,"name",{value:n,configurable:!0});import{resolve as q}from"../../../../../../../lib/settings.mjs";import{TranslatorError as j}from"../../../../../../../lib/errors.mjs";import{createIdFitter as B,createNameFitter as C}from"../../../../../../../lib/ids.mjs";import{sanitize as A}from"../../../../../../../lib/schema.mjs";import{NOTE as L}from"../../../../../../../lib/images.mjs";import{expand as U}from"../../../../../../../lib/documents.mjs";import{createBudget as W,unsign as z}from"../../../../../../../lib/thinking.mjs";import{clearToolUses as D,valid as P}from"../../../../../../../lib/context.mjs";import{plan as H,scan as K}from"../../../../../../../lib/toolsearch.mjs";const R=new Set(["model","messages","system","max_tokens","stream","temperature","top_p","top_k","stop_sequences","tools","tool_choice","thinking","output_config","output_format","context_management"]),G=new Set(["metadata","cache_control","service_tier"]),J=new Set(["mcp_servers","container","inference_geo"]),Q="x-anthropic-billing-header:";async function he(e,n,o){const t=q(n);if(e===null||typeof e!="object"||!Array.isArray(e.messages))throw new j("messages: Field required",{code:"invalid_request",param:"messages"});const s={front:"anthropic",model:e.model,stream:e.stream===!0,thinking:e.thinking?.type??null,reasoningField:t.reasoningField,toolNames:{},toolIds:{},dropped:[]};e.thinking?.type==="enabled"&&Number.isInteger(e.thinking.budget_tokens)&&e.output_config?.effort===void 0&&(s.thinkingBudget=e.thinking.budget_tokens),o?.producer!==void 0&&(s.producer=o.producer);const i=C({max:64,map:s.toolNames}),m=o?.toolCallIds?B({policy:o.toolCallIds,ids:V(e.messages),map:s.toolIds}):c=>c;for(const c of e.tools??[])(c.type===void 0||c.type==="custom")&&i(c.name);const g=K(e.messages,new Set((e.tools??[]).map(c=>c.name))),a=H(e.tools??[],g,t,e.tool_choice);s.toolSearch={mode:t.toolSearch,loaded:a.loaded,hidden:a.hidden,stubbed:a.stubbed,capped:g.capped};for(const c of Object.keys(e))if(!(R.has(c)||G.has(c)))if(J.has(c))h(s,c);else{if(t.unknownFields==="error")throw new j(`${c}: Extra inputs are not permitted`,{code:"unsupported_field",param:c});h(s,c)}const l={model:e.model},$=I(e.system,t),u=await X(e.messages,t,s,i,m,$,Y(e.context_management,s),o?.budget??W());l.messages=u.messages,u.prefill&&(l.continue_final_message=!0,l.add_generation_prompt=!1),e.max_tokens!==void 0&&(l[t.maxTokensField]=e.max_tokens),e.temperature!==void 0&&(l.temperature=e.temperature),e.top_p!==void 0&&(l.top_p=e.top_p),e.top_k!==void 0&&(t.topK?l.top_k=e.top_k:h(s,"top_k")),e.stop_sequences?.length>0&&(l.stop=e.stop_sequences),s.stream&&(l.stream=!0,t.streamUsage&&(l.stream_options={include_usage:!0},t.continuousUsage&&(l.stream_options.continuous_usage_stats=!0)));const x=oe(a.offer,t,s,i);if(x.length>0){l.tools=x;const c=se(e.tool_choice,t,s,i);c!==void 0&&(l.tool_choice=c),e.tool_choice?.disable_parallel_tool_use===!0&&t.parallelToolCalls&&(l.parallel_tool_calls=!1)}return Z(e.context_management,l,s),ie(e,t,l,o?.listed===!0),re(e,t,s,l),{body:l,state:s}}d(he,"request");function h(e,n){e.dropped.includes(n)||e.dropped.push(n)}d(h,"drop");function v(e,n,o){if(n.unknownFields==="error")throw new j(`${o}: is not supported by this translator`,{code:"unsupported_content",param:o});h(e,o)}d(v,"unsupported");function I(e,n){return e==null?"":(typeof e=="string"?[{type:"text",text:e}]:e).filter(t=>t.type==="text"&&!(n.stripBillingHeader&&t.text.trimStart().startsWith(Q))).map(t=>t.text).join(`
2
2
 
3
- `)}g(E,"textOf");function T(e){return typeof e=="string"?[{type:"text",text:e}]:e??[]}g(T,"asBlocks");function G(e){const t=[];for(const o of e)for(const n of T(o.content))n.type==="tool_use"?t.push(n.id):n.type==="tool_result"&&t.push(n.tool_use_id);return t}g(G,"idsOf");async function J(e,t,o,n,s,i,d,m){const a=[],u=[],$=new Map,f=t.inlineSystem==="auto"?"user":t.inlineSystem,h=new Map;if(t.reasoningRoundTrip){for(let r=e.length-1;r>=0;r--)if(e[r].role==="assistant")for(const l of[...T(e[r].content)].reverse())l.type==="thinking"&&h.set(l,await D(l.signature,{producer:o.producer,budget:m}))}const c=g(r=>{const l=a.at(-1);l?.role==="user"?l.parts.push(...r):a.push({role:"user",parts:r})},"pushUser"),v=g(async(r,l)=>{switch(r.type){case"text":return r.text===""?[]:[{type:"text",text:r.text}];case"image":return Z(r,t,o);case"document":{const _=[];for(const p of await L(r,{settings:t,toolInputs:$,toolUseId:l}))_.push(...await v(p,l));return _}case"search_result":{const _=(r.content??[]).map(p=>p.text).join(`
3
+ `)}d(I,"textOf");function S(e){return typeof e=="string"?[{type:"text",text:e}]:e??[]}d(S,"asBlocks");function V(e){const n=[];for(const o of e)for(const t of S(o.content))t.type==="tool_use"?n.push(t.id):t.type==="tool_result"&&n.push(t.tool_use_id);return n}d(V,"idsOf");async function X(e,n,o,t,s,i,m,g){const a=[],l=[],$=new Map,u=n.inlineSystem==="auto"?"user":n.inlineSystem,x=new Map;if(n.reasoningRoundTrip){for(let r=e.length-1;r>=0;r--)if(e[r].role==="assistant")for(const f of[...S(e[r].content)].reverse())f.type==="thinking"&&x.set(f,await z(f.signature,{producer:o.producer,budget:g}))}const c=d(r=>{const f=a.at(-1);f?.role==="user"?f.parts.push(...r):a.push({role:"user",parts:r})},"pushUser"),E=d(async(r,f)=>{switch(r.type){case"text":return r.text===""?[]:[{type:"text",text:r.text}];case"image":return ne(r,n,o);case"document":{const _=[];for(const p of await U(r,{settings:n,toolInputs:$,toolUseId:f}))_.push(...await E(p,f));return _}case"search_result":{const _=(r.content??[]).map(p=>p.text).join(`
4
4
  `);return[{type:"text",text:`[Search result: ${r.title??""} (${r.source??""})]
5
- ${_}`}]}case"tool_reference":return t.toolSearch==="native"&&t.deferredTools!=="drop"?[{type:"tool_reference",tool_name:r.tool_name}]:[{type:"text",text:o.toolSearch.loaded.includes(r.tool_name)?`[Tool loaded: ${r.tool_name}]`:`[Tool reference: ${r.tool_name}]`}];default:return F(o,t,`block:${r.type}`),[]}},"toParts");for(const r of e)if(r.role==="system"){const l=E(r.content,t);if(l===""||f==="drop")continue;if(f==="keep")a.push({role:"system",content:l});else if(f==="hoist")u.push(l);else{const _=l.trimStart().startsWith("<system-reminder>")?l:`<system-reminder>
6
- ${l}
7
- </system-reminder>`;c([{type:"text",text:_}])}}else if(r.role==="user"){const l=[],_=[];for(const y of T(r.content)){if(y.type!=="tool_result"){l.push(...await v(y,null));continue}const S=[];for(const k of T(y.content))S.push(...await v(k,y.tool_use_id));const N=S.filter(k=>k.type==="text").map(k=>k.text),I=S.filter(k=>k.type==="image_url");let w=S.some(k=>k.type==="tool_reference")?S.filter(k=>k.type!=="image_url"):N.join(`
8
- `);I.length>0&&(_.push({type:"text",text:`Image output of tool call ${s(y.tool_use_id)}:`},...I),w===""&&(w="(image output attached in the next message)")),y.is_error===!0&&t.toolErrors&&(w=typeof w=="string"?`${t.toolErrorMarker}${w}`:[{type:"text",text:t.toolErrorMarker.trimEnd()},...w]),a.push({role:"tool",tool_call_id:s(y.tool_use_id),content:w})}const p=[..._,...l];p.length>0&&c(p)}else if(r.role==="assistant"){const l={role:"assistant",texts:[],parts:[],tool_calls:[]};for(const p of T(r.content))if(p.type==="text")p.text!==""&&l.texts.push(p.text);else if(p.type==="thinking"){const y=h.get(p)??null;y?l.parts.push({restored:y}):p.thinking&&l.parts.push({text:p.thinking})}else p.type==="redacted_thinking"?x(o,"redacted_thinking"):p.type==="tool_use"?($.set(p.id,p.input),l.tool_calls.push({id:s(p.id),type:"function",function:{name:n(p.name),arguments:JSON.stringify(p.input??{})}})):F(o,t,`block:${p.type}`);const _=a.at(-1);_?.role==="assistant"?(_.texts.push(...l.texts),_.parts.push(...l.parts),_.tool_calls.push(...l.tool_calls)):a.push(l)}else throw new j(`messages: unsupported role "${r.role}"`,{code:"invalid_request",param:"messages"});return Y(a,t,i,u,d,o)}g(J,"convertMessages");function Q(e,t){let o=1/0;for(const n of e?.edits??[])typeof n.type=="string"&&n.type.startsWith("clear_thinking")?n.keep==="all"?o=1/0:o=n.keep?.value??1:x(t,`context_management.${n.type}`);return o}g(Q,"thinkingTurnsToKeep");function V(e,t){const o=e.find(a=>a.restored&&(a.restored.reasoning_content!==void 0||a.restored.reasoning!==void 0)),n=o?o.restored.reasoning_content!==void 0?"reasoning_content":"reasoning":t;let s="",i,d=null;for(const a of e){const u=a.restored?a.restored.reasoning_content??a.restored.reasoning:a.text;a.restored?.reasoning_details&&(i=[...i??[],...a.restored.reasoning_details]),u!==void 0&&(s+=(d!==null&&!(d.restored&&a.restored)?`
5
+ ${_}`}]}case"tool_reference":return n.toolSearch==="native"&&n.deferredTools!=="drop"?[{type:"tool_reference",tool_name:r.tool_name}]:[{type:"text",text:o.toolSearch.loaded.includes(r.tool_name)?`[Tool loaded: ${r.tool_name}]`:`[Tool reference: ${r.tool_name}]`}];default:return v(o,n,`block:${r.type}`),[]}},"toParts");for(const r of e)if(r.role==="system"){const f=I(r.content,n);if(f===""||u==="drop")continue;if(u==="keep")a.push({role:"system",content:f});else if(u==="hoist")l.push(f);else{const _=f.trimStart().startsWith("<system-reminder>")?f:`<system-reminder>
6
+ ${f}
7
+ </system-reminder>`;c([{type:"text",text:_}])}}else if(r.role==="user"){const f=[],_=[];for(const y of S(r.content)){if(y.type!=="tool_result"){f.push(...await E(y,null));continue}const T=[];for(const k of S(y.content))T.push(...await E(k,y.tool_use_id));const M=T.filter(k=>k.type==="text").map(k=>k.text),F=T.filter(k=>k.type==="image_url");let w=T.some(k=>k.type==="tool_reference")?T.filter(k=>k.type!=="image_url"):M.join(`
8
+ `);F.length>0&&(_.push({type:"text",text:`Image output of tool call ${s(y.tool_use_id)}:`},...F),w===""&&(w="(image output attached in the next message)")),y.is_error===!0&&n.toolErrors&&(w=typeof w=="string"?`${n.toolErrorMarker}${w}`:[{type:"text",text:n.toolErrorMarker.trimEnd()},...w]),a.push({role:"tool",tool_call_id:s(y.tool_use_id),content:w})}const p=[..._,...f];p.length>0&&c(p)}else if(r.role==="assistant"){const f={role:"assistant",texts:[],parts:[],tool_calls:[]};for(const p of S(r.content))if(p.type==="text")p.text!==""&&f.texts.push(p.text);else if(p.type==="thinking"){const y=x.get(p)??null;y?f.parts.push({restored:y}):p.thinking&&f.parts.push({text:p.thinking})}else p.type==="redacted_thinking"?h(o,"redacted_thinking"):p.type==="tool_use"?($.set(p.id,p.input),f.tool_calls.push({id:s(p.id),type:"function",function:{name:t(p.name),arguments:JSON.stringify(p.input??{})}})):v(o,n,`block:${p.type}`);const _=a.at(-1);_?.role==="assistant"?(_.texts.push(...f.texts),_.parts.push(...f.parts),_.tool_calls.push(...f.tool_calls)):a.push(f)}else throw new j(`messages: unsupported role "${r.role}"`,{code:"invalid_request",param:"messages"});return te(a,n,i,l,m,o)}d(X,"convertMessages");function Y(e,n){let o=1/0;for(const t of e?.edits??[])typeof t.type=="string"&&t.type.startsWith("clear_thinking")?t.keep==="all"?o=1/0:o=t.keep?.value??1:N(t)||h(n,`context_management.${t.type}`);return o}d(Y,"thinkingTurnsToKeep");function N(e){return typeof e.type=="string"&&e.type.startsWith("clear_tool_uses")}d(N,"isClearToolUses");function Z(e,n,o){for(const t of e?.edits??[]){if(!N(t))continue;if(!P(t)){h(o,`context_management.${t.type}`);continue}o.appliedEdits??=[];const s=D(t,n,o.toolNames);s&&o.appliedEdits.push(s)}}d(Z,"applyContextEdits");function b(e,n){const o=e.find(a=>a.restored&&(a.restored.reasoning_content!==void 0||a.restored.reasoning!==void 0)),t=o?o.restored.reasoning_content!==void 0?"reasoning_content":"reasoning":n;let s="",i,m=null;for(const a of e){const l=a.restored?a.restored.reasoning_content??a.restored.reasoning:a.text;a.restored?.reasoning_details&&(i=[...i??[],...a.restored.reasoning_details]),l!==void 0&&(s+=(m!==null&&!(m.restored&&a.restored)?`
9
9
 
10
- `:"")+u,d=a)}const m={};return s!==""&&(m[n]=s),i&&(m.reasoning_details=i),m}g(V,"reasoningFields");function X(e){return!!(e.reasoning_content||e.reasoning||e.reasoning_details)}g(X,"reasoningSent");function Y(e,t,o,n,s,i){let d=s;for(let f=e.length-1;f>=0;f--)e[f].role!=="assistant"||e[f].parts.length===0||(d>0?d--:e[f].parts=[]);const m=[],a=[o,...n].filter(f=>f!=="").join(`
10
+ `:"")+l,m=a)}const g={};return s!==""&&(g[t]=s),i&&(g.reasoning_details=i),g}d(b,"reasoningFields");function ee(e){return!!(e.reasoning_content||e.reasoning||e.reasoning_details)}d(ee,"reasoningSent");function te(e,n,o,t,s,i){let m=s;for(let u=e.length-1;u>=0;u--)e[u].role!=="assistant"||e[u].parts.length===0||(m>0?m--:e[u].parts=[]);const g=[],a=[o,...t].filter(u=>u!=="").join(`
11
11
 
12
- `);a!==""&&m.push({role:"system",content:a});for(const f of e)if(f.role==="user"){const h=f.parts.every(c=>c.type==="text");m.push({role:"user",content:h?f.parts.map(c=>c.text).join(`
12
+ `);a!==""&&g.push({role:"system",content:a});for(const u of e)if(u.role==="user"){const x=u.parts.every(c=>c.type==="text");g.push({role:"user",content:x?u.parts.map(c=>c.text).join(`
13
13
 
14
- `):f.parts})}else if(f.role==="assistant"){const h={role:"assistant",content:f.texts.join(`
14
+ `):u.parts})}else if(u.role==="assistant"){const x={role:"assistant",content:u.texts.join(`
15
15
 
16
- `)};f.parts.length>0&&Object.assign(h,V(f.parts,t.reasoningField)),t.reasoningRoundTrip&&f.tool_calls.length>0&&!X(h)&&(i.reasoningMissing=(i.reasoningMissing??0)+1),f.tool_calls.length>0&&(h.tool_calls=f.tool_calls),m.push(h)}else m.push(f);const u=m.at(-1),$=t.prefill==="continue_final_message"&&u?.role==="assistant"&&!u.tool_calls&&u.content!=="";return{messages:m,prefill:$}}g(Y,"finalize");function Z(e,t,o){if(!t.images)return x(o,"image"),[{type:"text",text:A}];const n=e.source;return n.type==="base64"?[{type:"image_url",image_url:{url:`data:${n.media_type};base64,${n.data}`}}]:n.type==="url"?[{type:"image_url",image_url:{url:n.url}}]:(x(o,"image.file"),[{type:"text",text:"[An image supplied as an uploaded file could not be forwarded.]"}])}g(Z,"imageParts");function b(e,t,o,n){const s=[];for(const i of e){if(i.type!==void 0&&i.type!=="custom"){x(o,`tool:${i.name}`);continue}if(i.defer_loading===!0&&t.deferredTools==="drop"){x(o,"deferred_tools");continue}const d=i.input_schema??{type:"object",properties:{}},m={name:n(i.name),parameters:t.toolSchema==="sanitize"?C(d):d};i.description!==void 0&&(m.description=i.description),i.strict!==void 0&&(m.strict=i.strict),s.push(i.defer_loading===!0&&t.toolSearch==="native"?{type:"function",function:m,defer_loading:!0}:{type:"function",function:m})}return s}g(b,"convertTools");function ee(e,t,o,n){switch(e?.type){case void 0:return;case"auto":return"auto";case"any":return"required";case"none":return"none";case"tool":return{type:"function",function:{name:n(e.name)}};default:F(o,t,`tool_choice:${e.type}`);return}}g(ee,"convertToolChoice");function te(e,t,o,n){const s=e.thinking,i=s?.type;let d;i==="disabled"?(t.thinking==="reasoning_effort"&&(d="none"),t.thinking==="chat_template_kwargs"&&(o.chat_template_kwargs={enable_thinking:!1})):(t.effort&&e.output_config?.effort!==void 0&&(d=e.output_config.effort),i==="enabled"&&(t.thinking==="chat_template_kwargs"&&(o.chat_template_kwargs={enable_thinking:!0},t.thinkingBudget&&s.budget_tokens!==void 0&&e.max_tokens!==void 0&&(o.thinking_token_budget=Math.max(0,Math.min(s.budget_tokens,e.max_tokens-1)))),t.thinking==="reasoning_effort"&&d===void 0&&!n&&(d="medium"))),d!==void 0&&(o.reasoning_effort=d)}g(te,"applyThinking");function ne(e,t,o,n){const s=e.output_config?.format??e.output_format;s!==void 0&&(t.responseFormat&&s.type==="json_schema"?n.response_format={type:"json_schema",json_schema:{name:"output",schema:s.schema}}:x(o,"output_config.format"));for(const i of Object.keys(e.output_config??{}))i!=="effort"&&i!=="format"&&x(o,`output_config.${i}`)}g(ne,"applyOutputConfig");export{pe as request};
16
+ `)};u.parts.length>0&&Object.assign(x,b(u.parts,n.reasoningField)),n.reasoningRoundTrip&&u.tool_calls.length>0&&!ee(x)&&(i.reasoningMissing=(i.reasoningMissing??0)+1),u.tool_calls.length>0&&(x.tool_calls=u.tool_calls),g.push(x)}else g.push(u);const l=g.at(-1),$=n.prefill==="continue_final_message"&&l?.role==="assistant"&&!l.tool_calls&&l.content!=="";return{messages:g,prefill:$}}d(te,"finalize");function ne(e,n,o){if(!n.images)return h(o,"image"),[{type:"text",text:L}];const t=e.source;return t.type==="base64"?[{type:"image_url",image_url:{url:`data:${t.media_type};base64,${t.data}`}}]:t.type==="url"?[{type:"image_url",image_url:{url:t.url}}]:(h(o,"image.file"),[{type:"text",text:"[An image supplied as an uploaded file could not be forwarded.]"}])}d(ne,"imageParts");function oe(e,n,o,t){const s=[];for(const i of e){if(i.type!==void 0&&i.type!=="custom"){h(o,`tool:${i.name}`);continue}if(i.defer_loading===!0&&n.deferredTools==="drop"){h(o,"deferred_tools");continue}const m=i.input_schema??{type:"object",properties:{}},g={name:t(i.name),parameters:n.toolSchema==="sanitize"?A(m):m};i.description!==void 0&&(g.description=i.description),i.strict!==void 0&&(g.strict=i.strict),s.push(i.defer_loading===!0&&n.toolSearch==="native"?{type:"function",function:g,defer_loading:!0}:{type:"function",function:g})}return s}d(oe,"convertTools");function se(e,n,o,t){switch(e?.type){case void 0:return;case"auto":return"auto";case"any":return"required";case"none":return"none";case"tool":return{type:"function",function:{name:t(e.name)}};default:v(o,n,`tool_choice:${e.type}`);return}}d(se,"convertToolChoice");function ie(e,n,o,t){const s=e.thinking,i=s?.type;let m;i==="disabled"?(n.thinking==="reasoning_effort"&&(m="none"),n.thinking==="chat_template_kwargs"&&(o.chat_template_kwargs={enable_thinking:!1})):(n.effort&&e.output_config?.effort!==void 0&&(m=e.output_config.effort),i==="enabled"&&(n.thinking==="chat_template_kwargs"&&(o.chat_template_kwargs={enable_thinking:!0},n.thinkingBudget&&s.budget_tokens!==void 0&&e.max_tokens!==void 0&&(o.thinking_token_budget=Math.max(0,Math.min(s.budget_tokens,e.max_tokens-1)))),n.thinking==="reasoning_effort"&&m===void 0&&!t&&(m="medium"))),m!==void 0&&(o.reasoning_effort=m)}d(ie,"applyThinking");function re(e,n,o,t){const s=e.output_config?.format??e.output_format;s!==void 0&&(n.responseFormat&&s.type==="json_schema"?t.response_format={type:"json_schema",json_schema:{name:"output",schema:s.schema}}:h(o,"output_config.format"));for(const i of Object.keys(e.output_config??{}))i!=="effort"&&i!=="format"&&h(o,`output_config.${i}`)}d(re,"applyOutputConfig");export{he as request};
17
17
  //# sourceMappingURL=request.mjs.map
@@ -1,2 +1,2 @@
1
- var _=Object.defineProperty;var p=(o,t)=>_(o,"name",{value:t,configurable:!0});import{resolve as k}from"../../../../../../../lib/settings.mjs";import{TranslatorError as u}from"../../../../../../../lib/errors.mjs";import{messageId as T,anthropicToolId as x,restoreName as A}from"../../../../../../../lib/ids.mjs";import{sign as w,reasoningOf as C,reasoningKey as I,detailsText as N,splitThink as O}from"../../../../../../../lib/thinking.mjs";import{isComplete as S,parseArguments as U}from"../../../../../../../lib/arguments.mjs";import{filterCategory as v,finishError as E,toAnthropicStop as b,toAnthropicUsage as j}from"../../../../../../../lib/finish.mjs";function G(o,t,m){const c=k(m),i=o?.choices?.[0];if(!i)throw new u("Upstream response contained no choices",{code:"invalid_upstream_response",status:502});const g=E(i.finish_reason);if(g)throw new u(g.message,{code:g.code,status:g.status,type:g.type});const e=i.message??{},a=[];let l=C(e),s=typeof e.content=="string"?e.content:"";const n={};if(c.reasoningRoundTrip){const r=I(e);r&&(n[r]=e[r]),Array.isArray(e.reasoning_details)&&e.reasoning_details.length>0&&(n.reasoning_details=e.reasoning_details)}l===""&&c.thinkTags&&({reasoning:l,text:s}=O(s,c.thinkOpen)),l===""&&n.reasoning_details&&(l=N(n.reasoning_details)),(l!==""||n.reasoning_details)&&(a.push({type:"thinking",thinking:l,signature:w(Object.keys(n).length>0?n:void 0,t?.producer)}),s=s.trimStart()),s===""&&typeof e.refusal=="string"&&(s=e.refusal),s!==""&&a.push({type:"text",text:s});const d=t?.toolNames??{},h=t?.toolIds??{},f=e.tool_calls??[];for(const r of f)a.push({type:"tool_use",id:x(r.id,h),name:A(d,r.function.name),input:U(r.function.arguments)});a.length===0&&a.push({type:"text",text:""});const y=b(i.finish_reason,{hasToolCalls:f.length>0,matchedStop:i.stop_reason,refusal:e.refusal,category:v(i.content_filter_results),argumentsComplete:f.length===0||S(f.at(-1).function.arguments)});return t&&o.usage&&(t.upstreamUsage=o.usage),{id:T(),type:"message",role:"assistant",model:t?.model??o.model,content:a,...y,usage:j(o.usage)}}p(G,"response");export{G as response};
1
+ var y=Object.defineProperty;var f=(n,t)=>y(n,"name",{value:t,configurable:!0});import{resolve as k}from"../../../../../../../lib/settings.mjs";import{TranslatorError as d}from"../../../../../../../lib/errors.mjs";import{messageId as x,anthropicToolId as T,restoreName as A}from"../../../../../../../lib/ids.mjs";import{sign as w,reasoningOf as C,reasoningKey as E,detailsText as I,splitThink as N}from"../../../../../../../lib/thinking.mjs";import{isComplete as O,parseArguments as S}from"../../../../../../../lib/arguments.mjs";import{filterCategory as U,finishError as v,toAnthropicStop as b,toAnthropicUsage as j}from"../../../../../../../lib/finish.mjs";function G(n,t,m){const c=k(m),r=n?.choices?.[0];if(!r)throw new d("Upstream response contained no choices",{code:"invalid_upstream_response",status:502});const p=v(r.finish_reason);if(p)throw new d(p.message,{code:p.code,status:p.status,type:p.type});const e=r.message??{},a=[];let l=C(e),s=typeof e.content=="string"?e.content:"";const o={};if(c.reasoningRoundTrip){const i=E(e);i&&(o[i]=e[i]),Array.isArray(e.reasoning_details)&&e.reasoning_details.length>0&&(o.reasoning_details=e.reasoning_details)}l===""&&c.thinkTags&&({reasoning:l,text:s}=N(s,c.thinkOpen)),l===""&&o.reasoning_details&&(l=I(o.reasoning_details)),(l!==""||o.reasoning_details)&&(a.push({type:"thinking",thinking:l,signature:w(Object.keys(o).length>0?o:void 0,t?.producer)}),s=s.trimStart()),s===""&&typeof e.refusal=="string"&&(s=e.refusal),s!==""&&a.push({type:"text",text:s});const u=t?.toolNames??{},h=t?.toolIds??{},g=e.tool_calls??[];for(const i of g)a.push({type:"tool_use",id:T(i.id,h),name:A(u,i.function.name),input:S(i.function.arguments)});a.length===0&&a.push({type:"text",text:""});const _=b(r.finish_reason,{hasToolCalls:g.length>0,matchedStop:r.stop_reason,refusal:e.refusal,category:U(r.content_filter_results),argumentsComplete:g.length===0||O(g.at(-1).function.arguments)});return t&&n.usage&&(t.upstreamUsage=n.usage),{id:x(),type:"message",role:"assistant",model:t?.model??n.model,content:a,..._,usage:j(n.usage),...t?.appliedEdits?{context_management:{applied_edits:t.appliedEdits}}:{}}}f(G,"response");export{G as response};
2
2
  //# sourceMappingURL=response.mjs.map
@@ -1,2 +1,2 @@
1
- var Q=Object.defineProperty;var s=(b,r)=>Q(b,"name",{value:r,configurable:!0});import{resolve as V}from"../../../../../../../lib/settings.mjs";import{TranslatorError as W,openaiToAnthropicType as X,rewriteContextOverflow as Y}from"../../../../../../../lib/errors.mjs";import{isComplete as Z}from"../../../../../../../lib/arguments.mjs";import{messageId as $,anthropicToolId as ee,restoreName as te}from"../../../../../../../lib/ids.mjs";import{sign as ne,reasoningOf as ie,reasoningKey as oe,detailsText as le,mergeDetail as se,createThinkSplitter as re}from"../../../../../../../lib/thinking.mjs";import{filterCategory as ae,finishError as de,toAnthropicStop as fe,toAnthropicUsage as B}from"../../../../../../../lib/finish.mjs";import{messages as ce}from"../../../../../../../lib/sse.mjs";async function*xe(b,r,F){const u=V(F),J=r?.toolNames??{},K=r?.toolIds??{},d=u.thinkTags?re(u.thinkOpen):null,c=new Map;let m=!1,j=0,i=null,N=null,k=null,O=null,h="",T=null,v=!1,A=!1,I=0,w=!1,U=!1,E=!1;const S=[],l=s((e,t)=>({event:e,data:{type:e,...t}}),"event"),p=s((e,t)=>l("error",{error:{type:e,message:t}}),"fail"),M=s(e=>{const t={};return u.reasoningRoundTrip&&(e.key!==null&&(t[e.key]=e.text),e.details.length>0&&(t.reasoning_details=e.details)),Object.keys(t).length>0?t:void 0},"carried");function*y(){if(i!==null){i.kind==="thinking"?yield l("content_block_delta",{index:i.index,delta:{type:"signature_delta",signature:ne(M(i.carry),r?.producer)}}):i.kind==="tool"&&(i.entry.state="closed",i.entry.sent||(yield l("content_block_delta",{index:i.index,delta:{type:"input_json_delta",partial_json:"{}"}}))),yield l("content_block_stop",{index:i.index}),i=null;for(const e of c.values()){if(e.state!=="pending")continue;e.state="closed";const t=j++,n=e.args.join("");yield l("content_block_start",{index:t,content_block:e.block}),yield l("content_block_delta",{index:t,delta:{type:"input_json_delta",partial_json:n===""?"{}":n}}),yield l("content_block_stop",{index:t})}}}s(y,"close");function*G(){for(const e of R()){if(e.state!=="buffered")continue;yield*g("tool",e.block,e);const t=e.args.join("");t!==""&&(e.sent=!0,yield l("content_block_delta",{index:i.index,delta:{type:"input_json_delta",partial_json:t}})),yield*y()}}s(G,"flushBuffered");function R(){return[...c.entries()].sort((e,t)=>e[0]-t[0]).map(e=>e[1])}s(R,"ordered");function*g(e,t,n){yield*y(),i={kind:e,index:j++,entry:n,carry:{key:null,text:"",details:[]}},yield l("content_block_start",{index:i.index,content_block:t})}s(g,"begin");function*a(e,t,n=null){if(e==="text"&&E&&(t=t.trimStart()),t==="")return;if(E=e==="thinking",e==="text"&&(v=!0),i?.kind==="tool"){S.push({kind:e,text:t,key:n});return}i?.kind!==e&&(yield*g(e,e==="text"?{type:"text",text:""}:{type:"thinking",thinking:"",signature:""})),n!==null&&(i.carry.key??=n,i.carry.text+=t);const o=e==="text"?{type:"text_delta",text:t}:{type:"thinking_delta",thinking:t};yield l("content_block_delta",{index:i.index,delta:o})}s(a,"piece");function*z(e){if(i?.kind==="tool"){S.push({kind:"details",items:e});return}i?.kind!=="thinking"&&(yield*g("thinking",{type:"thinking",thinking:"",signature:""}));for(const t of e)se(i.carry.details,t);i.carry.key===null&&(yield*a("thinking",le(e)))}s(z,"details");function*D(e){yield*a("thinking",e.reasoning),yield*a("text",e.text)}s(D,"pieces");function*H(e){let t=e.index;t===void 0&&(t=e.id===void 0||e.id===null?I:c.size),I=t;let n=c.get(t);if(n===void 0){const x=e.function?.name??"",_={type:"tool_use",id:ee(e.id,K),name:te(J,x),input:{}};d?.pending()&&(U=!0);const f=u.toolStreaming==="buffer"||U;n={state:f?"buffered":"pending",sent:!1,args:[],block:_},c.set(t,n),!f&&i?.kind!=="tool"&&(n.state="open",yield*g("tool",_,n))}let o=e.function?.arguments;o!==null&&typeof o=="object"&&(o=JSON.stringify(o)),!(o==null||o==="")&&(n.args.push(o),n.state==="open"&&(n.sent=!0,yield l("content_block_delta",{index:i.index,delta:{type:"input_json_delta",partial_json:o}})))}s(H,"toolCall");try{for await(const e of ce(b)){if(e.data==="[DONE]"){A=!0;break}const t=e.data;if(t.error){const f=t.error.message??"Upstream error";yield p(X(t.error),Y(f)??f);return}m||(m=!0,yield l("message_start",{message:{id:$(),type:"message",role:"assistant",model:r?.model??t.model,content:[],stop_reason:null,stop_sequence:null,usage:B(t.usage)}})),t.usage&&(N=t.usage,r&&(r.upstreamUsage=t.usage));const n=t.choices?.[0];if(!n)continue;const o=n.delta??{},x=ie(o);let _="";x!==""&&!w&&(w=!0,d&&(_=d.release().text)),yield*a("thinking",x,oe(o)),yield*a("text",_),u.reasoningRoundTrip&&Array.isArray(o.reasoning_details)&&o.reasoning_details.length>0&&(yield*z(o.reasoning_details)),typeof o.content=="string"&&o.content!==""&&(d&&!w?yield*D(d.push(o.content)):yield*a("text",o.content)),typeof o.refusal=="string"&&(h+=o.refusal),T=ae(n.content_filter_results)??T;for(const f of o.tool_calls??[])yield*H(f);n.finish_reason&&(k=n.finish_reason),typeof n.stop_reason=="string"&&(O=n.stop_reason)}}catch(e){yield e instanceof W?p(e.type,e.message):p("api_error",e.message);return}if(!m||k===null&&!A){yield p("api_error",m?"Upstream stream ended before completion":"Upstream stream ended without data");return}const C=de(k);if(C){yield p(C.type,C.message);return}d&&(yield*D(d.end())),yield*G();const q=R().at(-1),L=q===void 0||Z(q.args.join(""));yield*y();for(const e of S)yield*e.kind==="details"?z(e.items):a(e.kind,e.text,e.key);!v&&h!==""&&(yield*a("text",h)),yield*y(),j===0&&(yield*g("text",{type:"text",text:""}),yield*y());const P=fe(k,{hasToolCalls:c.size>0,matchedStop:O,argumentsComplete:L,refusal:h,category:T});yield l("message_delta",{delta:P,usage:B(N)}),yield l("message_stop",{})}s(xe,"stream");export{xe as stream};
1
+ var V=Object.defineProperty;var s=(b,r)=>V(b,"name",{value:r,configurable:!0});import{resolve as W}from"../../../../../../../lib/settings.mjs";import{TranslatorError as X,openaiToAnthropicType as Y,rewriteContextOverflow as Z}from"../../../../../../../lib/errors.mjs";import{isComplete as $}from"../../../../../../../lib/arguments.mjs";import{messageId as ee,anthropicToolId as te,restoreName as ne}from"../../../../../../../lib/ids.mjs";import{sign as ie,reasoningOf as oe,reasoningKey as le,detailsText as se,mergeDetail as re,createThinkSplitter as ae}from"../../../../../../../lib/thinking.mjs";import{filterCategory as de,finishError as fe,toAnthropicStop as ce,toAnthropicUsage as B}from"../../../../../../../lib/finish.mjs";import{messages as ue}from"../../../../../../../lib/sse.mjs";async function*be(b,r,F){const u=W(F),J=r?.toolNames??{},K=r?.toolIds??{},d=u.thinkTags?ae(u.thinkOpen):null,c=new Map;let m=!1,j=0,i=null,N=null,k=null,O=null,h="",T=null,v=!1,A=!1,E=0,w=!1,I=!1,U=!1;const S=[],l=s((e,t)=>({event:e,data:{type:e,...t}}),"event"),p=s((e,t)=>l("error",{error:{type:e,message:t}}),"fail"),M=s(e=>{const t={};return u.reasoningRoundTrip&&(e.key!==null&&(t[e.key]=e.text),e.details.length>0&&(t.reasoning_details=e.details)),Object.keys(t).length>0?t:void 0},"carried");function*y(){if(i!==null){i.kind==="thinking"?yield l("content_block_delta",{index:i.index,delta:{type:"signature_delta",signature:ie(M(i.carry),r?.producer)}}):i.kind==="tool"&&(i.entry.state="closed",i.entry.sent||(yield l("content_block_delta",{index:i.index,delta:{type:"input_json_delta",partial_json:"{}"}}))),yield l("content_block_stop",{index:i.index}),i=null;for(const e of c.values()){if(e.state!=="pending")continue;e.state="closed";const t=j++,n=e.args.join("");yield l("content_block_start",{index:t,content_block:e.block}),yield l("content_block_delta",{index:t,delta:{type:"input_json_delta",partial_json:n===""?"{}":n}}),yield l("content_block_stop",{index:t})}}}s(y,"close");function*G(){for(const e of R()){if(e.state!=="buffered")continue;yield*g("tool",e.block,e);const t=e.args.join("");t!==""&&(e.sent=!0,yield l("content_block_delta",{index:i.index,delta:{type:"input_json_delta",partial_json:t}})),yield*y()}}s(G,"flushBuffered");function R(){return[...c.entries()].sort((e,t)=>e[0]-t[0]).map(e=>e[1])}s(R,"ordered");function*g(e,t,n){yield*y(),i={kind:e,index:j++,entry:n,carry:{key:null,text:"",details:[]}},yield l("content_block_start",{index:i.index,content_block:t})}s(g,"begin");function*a(e,t,n=null){if(e==="text"&&U&&(t=t.trimStart()),t==="")return;if(U=e==="thinking",e==="text"&&(v=!0),i?.kind==="tool"){S.push({kind:e,text:t,key:n});return}i?.kind!==e&&(yield*g(e,e==="text"?{type:"text",text:""}:{type:"thinking",thinking:"",signature:""})),n!==null&&(i.carry.key??=n,i.carry.text+=t);const o=e==="text"?{type:"text_delta",text:t}:{type:"thinking_delta",thinking:t};yield l("content_block_delta",{index:i.index,delta:o})}s(a,"piece");function*z(e){if(i?.kind==="tool"){S.push({kind:"details",items:e});return}i?.kind!=="thinking"&&(yield*g("thinking",{type:"thinking",thinking:"",signature:""}));for(const t of e)re(i.carry.details,t);i.carry.key===null&&(yield*a("thinking",se(e)))}s(z,"details");function*D(e){yield*a("thinking",e.reasoning),yield*a("text",e.text)}s(D,"pieces");function*H(e){let t=e.index;t===void 0&&(t=e.id===void 0||e.id===null?E:c.size),E=t;let n=c.get(t);if(n===void 0){const x=e.function?.name??"",_={type:"tool_use",id:te(e.id,K),name:ne(J,x),input:{}};d?.pending()&&(I=!0);const f=u.toolStreaming==="buffer"||I;n={state:f?"buffered":"pending",sent:!1,args:[],block:_},c.set(t,n),!f&&i?.kind!=="tool"&&(n.state="open",yield*g("tool",_,n))}let o=e.function?.arguments;o!==null&&typeof o=="object"&&(o=JSON.stringify(o)),!(o==null||o==="")&&(n.args.push(o),n.state==="open"&&(n.sent=!0,yield l("content_block_delta",{index:i.index,delta:{type:"input_json_delta",partial_json:o}})))}s(H,"toolCall");try{for await(const e of ue(b)){if(e.data==="[DONE]"){A=!0;break}const t=e.data;if(t.error){const f=t.error.message??"Upstream error";yield p(Y(t.error),Z(f)??f);return}m||(m=!0,yield l("message_start",{message:{id:ee(),type:"message",role:"assistant",model:r?.model??t.model,content:[],stop_reason:null,stop_sequence:null,usage:B(t.usage)}})),t.usage&&(N=t.usage,r&&(r.upstreamUsage=t.usage));const n=t.choices?.[0];if(!n)continue;const o=n.delta??{},x=oe(o);let _="";x!==""&&!w&&(w=!0,d&&(_=d.release().text)),yield*a("thinking",x,le(o)),yield*a("text",_),u.reasoningRoundTrip&&Array.isArray(o.reasoning_details)&&o.reasoning_details.length>0&&(yield*z(o.reasoning_details)),typeof o.content=="string"&&o.content!==""&&(d&&!w?yield*D(d.push(o.content)):yield*a("text",o.content)),typeof o.refusal=="string"&&(h+=o.refusal),T=de(n.content_filter_results)??T;for(const f of o.tool_calls??[])yield*H(f);n.finish_reason&&(k=n.finish_reason),typeof n.stop_reason=="string"&&(O=n.stop_reason)}}catch(e){yield e instanceof X?p(e.type,e.message):p("api_error",e.message);return}if(!m||k===null&&!A){yield p("api_error",m?"Upstream stream ended before completion":"Upstream stream ended without data");return}const C=fe(k);if(C){yield p(C.type,C.message);return}d&&(yield*D(d.end())),yield*G();const q=R().at(-1),L=q===void 0||$(q.args.join(""));yield*y();for(const e of S)yield*e.kind==="details"?z(e.items):a(e.kind,e.text,e.key);!v&&h!==""&&(yield*a("text",h)),yield*y(),j===0&&(yield*g("text",{type:"text",text:""}),yield*y());const P=ce(k,{hasToolCalls:c.size>0,matchedStop:O,argumentsComplete:L,refusal:h,category:T}),Q=r?.appliedEdits?{context_management:{applied_edits:r.appliedEdits}}:{};yield l("message_delta",{delta:P,usage:B(N),...Q}),yield l("message_stop",{})}s(be,"stream");export{be as stream};
2
2
  //# sourceMappingURL=stream.mjs.map
@@ -1,2 +1,2 @@
1
- var h=Object.defineProperty;var o=(t,e)=>h(t,"name",{value:e,configurable:!0});import{context as g,self as u}from"../lib/runtime.mjs";import{capture as w}from"../lib/capture.mjs";import{TranslatorError as m}from"../lib/errors.mjs";import{warn as p}from"../lib/warn.mjs";import{emit as y}from"../lib/events.mjs";import*as k from"../lib/http.mjs";import{validToolIdPolicy as $}from"../lib/ids.mjs";async function j(t={}){const e=w(u,g),{status:l}=u,f=t.network!==!1;for(const a of e.drivers){const s=a.info.fields;if(s!==void 0&&!(Array.isArray(s)&&s.every(i=>typeof i=="string")))throw new m(`Driver ${a.name}: info().fields must be an array of field names`,{code:"invalid_driver",status:500})}for(const a of e.drivers){const s=a.info.toolCallIds;if(s!==void 0&&!$(s))throw new m(`Driver ${a.name}: info().toolCallIds must be { alphabet, min?, max? } with a character-class alphabet and positive integer bounds`,{code:"invalid_driver",status:500})}const r={transport:e.state.transport,signal:t.signal,timeout:e.config.detectTimeout};return await Promise.all(e.state.endpoints.map(a=>U(e,a,f,r))),l()}o(j,"detect");async function U(t,e,l,f){let r,a=null;if(e.backend!=="auto"){if(r=t.drivers.find(s=>s.name===e.backend),!r)throw new m(`Unknown backend "${e.backend}" for upstream ${e.name}`,{code:"unknown_backend",status:500});a={version:e.version}}else if(l){const s=URL.canParse(e.url)?new URL(e.url).hostname:"",i={endpoint:e,...f,probes:new Map};let n=!1;for(const[v,c]of t.drivers.entries()){if(!n&&c.info.probes?.length>0&&!c.info.hosts?.includes(s)){n=!0;for(const d of t.drivers.slice(v))for(const b of d.info.probes??[])k.probe(i,b,d.info)}if(a=await x(c,i),a){r=c;break}}}if(r||(r=t.drivers.find(s=>s.name==="compatible"),a={reachable:null,models:[]}),t.state.noMessages.delete(e.name),Object.assign(e.health,{ready:null,readyReason:null,checkedAt:null,unsupported:!1}),e.driver=r.name,e.protocol=r.info.protocol,e.api==="native"){if(!r.info.native){const s=t.drivers.filter(i=>i.info.native).map(i=>i.name);throw new m(`Upstream ${e.name}: api "native" needs a driver with a native API (${s.join(", ")}), but the upstream is bound to ${r.name}; set backend to one of them`,{code:"unsupported_api",status:500})}e.protocol=r.info.native.protocol}if(e.facts=a,e.driverSettings=await r.settings?.(a)??{},e.probed={},l&&e.backend==="auto"&&r.probe){const s=o(n=>e.settings[n]!==void 0||t.config.settings[n]!==void 0,"given"),i={inlineSystem:t.settings(e).inlineSystem==="auto",images:!s("images")};(i.inlineSystem||i.images)&&(e.probed=await r.probe({endpoint:e,facts:a,want:i,...f}))}a.reachable===!1?p(t.config,t.state,`Upstream ${e.name} (${e.url}) did not answer detection; using the generic OpenAI-compatible driver`,{type:"detect",kind:"unreachable",endpoint:e.name,url:e.url,driver:r.name}):y({config:t.config,state:t.state},{type:"detect",kind:"driver",level:"info",message:`Upstream ${e.name} (${e.url}): driver ${r.name}`,endpoint:e.name,url:e.url,driver:r.name})}o(U,"bind");async function x(t,e){try{return await t.detect?.(e)??null}catch{return null}}o(x,"detectWith");export{j as detect};
1
+ var w=Object.defineProperty;var f=(t,e)=>w(t,"name",{value:e,configurable:!0});import{context as p,self as b}from"../lib/runtime.mjs";import{capture as y}from"../lib/capture.mjs";import{TranslatorError as d}from"../lib/errors.mjs";import{warn as k}from"../lib/warn.mjs";import{emit as $}from"../lib/events.mjs";import*as x from"../lib/http.mjs";import{validToolIdPolicy as U}from"../lib/ids.mjs";async function D(t={}){const e=y(b,p),{status:n}=b,l=t.network!==!1;for(const a of e.drivers){const s=a.info.fields;if(s!==void 0&&!(Array.isArray(s)&&s.every(i=>typeof i=="string")))throw new d(`Driver ${a.name}: info().fields must be an array of field names`,{code:"invalid_driver",status:500})}for(const a of e.drivers){const s=a.info.toolCallIds;if(s!==void 0&&!U(s))throw new d(`Driver ${a.name}: info().toolCallIds must be { alphabet, min?, max? } with a character-class alphabet and positive integer bounds`,{code:"invalid_driver",status:500})}const r={transport:e.state.transport,signal:t.signal,timeout:e.config.detectTimeout};return await Promise.all(e.state.endpoints.map(a=>A(e,a,l,r))),n()}f(D,"detect");async function A(t,e,n,l){let r,a=null;if(e.backend!=="auto"){if(r=t.drivers.find(s=>s.name===e.backend),!r)throw new d(`Unknown backend "${e.backend}" for upstream ${e.name}`,{code:"unknown_backend",status:500});a={version:e.version}}else if(n){const s=URL.canParse(e.url)?new URL(e.url).hostname:"",i={endpoint:e,...l,probes:new Map},c=t.drivers.filter(o=>I(o.info,s));for(const o of c)if(a=await h(o,i),a){r=o;break}let u=!1;for(const[o,m]of r?[]:t.drivers.entries()){if(!u&&m.info.probes?.length>0&&!m.info.hosts?.includes(s)){u=!0;for(const v of t.drivers.slice(o))for(const g of v.info.probes??[])x.probe(i,g,v.info)}if(a=await h(m,i),a){r=m;break}}}if(r||(r=t.drivers.find(s=>s.name==="compatible"),a={reachable:null,models:[]}),t.state.noMessages.delete(e.name),Object.assign(e.health,{ready:null,readyReason:null,checkedAt:null,unsupported:!1}),e.driver=r.name,e.protocol=r.info.protocol,e.api==="native"){if(!r.info.native){const s=t.drivers.filter(i=>i.info.native).map(i=>i.name);throw new d(`Upstream ${e.name}: api "native" needs a driver with a native API (${s.join(", ")}), but the upstream is bound to ${r.name}; set backend to one of them`,{code:"unsupported_api",status:500})}e.protocol=r.info.native.protocol}if(e.facts=a,e.driverSettings=await r.settings?.(a)??{},e.probed={},n&&e.backend==="auto"&&r.probe){const s=f(c=>e.settings[c]!==void 0||t.config.settings[c]!==void 0,"given"),i={inlineSystem:t.settings(e).inlineSystem==="auto",images:!s("images")};(i.inlineSystem||i.images)&&(e.probed=await r.probe({endpoint:e,facts:a,want:i,...l}))}a.reachable===!1?k(t.config,t.state,`Upstream ${e.name} (${e.url}) did not answer detection; using the generic OpenAI-compatible driver`,{type:"detect",kind:"unreachable",endpoint:e.name,url:e.url,driver:r.name}):$({config:t.config,state:t.state},{type:"detect",kind:"driver",level:"info",message:`Upstream ${e.name} (${e.url}): driver ${r.name}`,endpoint:e.name,url:e.url,driver:r.name})}f(A,"bind");function I(t,e){return e!==""&&(t.hosts?.includes(e)===!0||t.hostSuffixes?.some(n=>e.endsWith(n))===!0)}f(I,"matchesHost");async function h(t,e){try{return await t.detect?.(e)??null}catch{return null}}f(h,"detectWith");export{D as detect};
2
2
  //# sourceMappingURL=detect.mjs.map
@@ -1,2 +1,2 @@
1
- var s=Object.defineProperty;var o=(e,c)=>s(e,"name",{value:c,configurable:!0});const n=[".openai.azure.com",".cognitiveservices.azure.com",".services.ai.azure.com"];function a({endpoint:e}){const c=URL.canParse(e.url)?new URL(e.url).hostname:"";return n.some(r=>c.endsWith(r))?{host:c}:null}o(a,"detect");export{a as detect};
1
+ var r=Object.defineProperty;var o=(t,e)=>r(t,"name",{value:e,configurable:!0});import{info as s}from"./info.mjs";function c({endpoint:t}){const e=URL.canParse(t.url)?new URL(t.url).hostname:"";return s().hostSuffixes.some(n=>e.endsWith(n))?{host:e}:null}o(c,"detect");export{c as detect};
2
2
  //# sourceMappingURL=detect.mjs.map
@@ -1,2 +1,2 @@
1
- var a=Object.defineProperty;var e=(o,t)=>a(o,"name",{value:t,configurable:!0});function i(){return{name:"azure",protocol:"openai.chat",order:32,auth:"api-key",strict:!0,paths:{chat:"/openai/v1/chat/completions",embeddings:"/openai/v1/embeddings",models:"/openai/v1/models"}}}e(i,"info");export{i as info};
1
+ var a=Object.defineProperty;var e=(o,i)=>a(o,"name",{value:i,configurable:!0});function n(){return{name:"azure",protocol:"openai.chat",order:32,auth:"api-key",strict:!0,hostSuffixes:[".openai.azure.com",".cognitiveservices.azure.com",".services.ai.azure.com"],paths:{chat:"/openai/v1/chat/completions",embeddings:"/openai/v1/embeddings",models:"/openai/v1/models"}}}e(n,"info");export{n as info};
2
2
  //# sourceMappingURL=info.mjs.map
@@ -1,2 +1,2 @@
1
- var a=Object.defineProperty;var e=(o,t)=>a(o,"name",{value:t,configurable:!0});function c(){return{name:"deepseek",protocol:"openai.chat",order:35,auth:"bearer",strict:!1,paths:{chat:"/v1/chat/completions",models:"/v1/models"}}}e(c,"info");export{c as info};
1
+ var a=Object.defineProperty;var e=(o,t)=>a(o,"name",{value:t,configurable:!0});function r(){return{name:"deepseek",protocol:"openai.chat",order:35,auth:"bearer",strict:!1,hosts:["api.deepseek.com"],paths:{chat:"/v1/chat/completions",models:"/v1/models"}}}e(r,"info");export{r as info};
2
2
  //# sourceMappingURL=info.mjs.map
@@ -1,2 +1,2 @@
1
- var a=Object.defineProperty;var e=(p,t)=>a(p,"name",{value:t,configurable:!0});function r(){return{name:"llamacpp",protocol:"openai.chat",order:40,auth:"bearer",strict:!1,probes:["/props"],fields:["top_k","min_p","chat_template_kwargs","repetition_penalty","repeat_penalty","grammar"],paths:{chat:"/v1/chat/completions",models:"/v1/models",props:"/props"}}}e(r,"info");export{r as info};
1
+ var a=Object.defineProperty;var e=(p,t)=>a(p,"name",{value:t,configurable:!0});function r(){return{name:"llamacpp",protocol:"openai.chat",order:40,auth:"bearer",strict:!1,probes:["/props"],fields:["top_k","min_p","chat_template_kwargs","repetition_penalty","repeat_penalty","grammar","json_schema"],paths:{chat:"/v1/chat/completions",models:"/v1/models",props:"/props"}}}e(r,"info");export{r as info};
2
2
  //# sourceMappingURL=info.mjs.map
@@ -1,2 +1,2 @@
1
- var e=Object.defineProperty;var a=(t,o)=>e(t,"name",{value:o,configurable:!0});function r(){return{name:"mistral",protocol:"openai.chat",order:36,auth:"bearer",strict:!1,toolCallIds:{alphabet:"a-zA-Z0-9",min:9,max:9},paths:{chat:"/v1/chat/completions",models:"/v1/models"}}}a(r,"info");export{r as info};
1
+ var e=Object.defineProperty;var a=(t,o)=>e(t,"name",{value:o,configurable:!0});function r(){return{name:"mistral",protocol:"openai.chat",order:36,auth:"bearer",strict:!1,hosts:["api.mistral.ai"],toolCallIds:{alphabet:"a-zA-Z0-9",min:9,max:9},paths:{chat:"/v1/chat/completions",models:"/v1/models"}}}a(r,"info");export{r as info};
2
2
  //# sourceMappingURL=info.mjs.map
@@ -1,2 +1,2 @@
1
- var r=Object.defineProperty;var o=(t,e)=>r(t,"name",{value:e,configurable:!0});function n(){return{name:"openai",protocol:"openai.chat",order:30,auth:"bearer",strict:!0,paths:{chat:"/v1/chat/completions",models:"/v1/models"}}}o(n,"info");export{n as info};
1
+ var a=Object.defineProperty;var o=(e,t)=>a(e,"name",{value:t,configurable:!0});function c(){return{name:"openai",protocol:"openai.chat",order:30,auth:"bearer",strict:!0,hosts:["api.openai.com"],fields:["prompt_cache_key"],paths:{chat:"/v1/chat/completions",models:"/v1/models"}}}o(c,"info");export{c as info};
2
2
  //# sourceMappingURL=info.mjs.map
@@ -1,2 +1,2 @@
1
- var r=Object.defineProperty;var o=(e,t)=>r(e,"name",{value:t,configurable:!0});function n(){return{name:"openrouter",protocol:"openai.chat",order:37,auth:"bearer",strict:!1,paths:{chat:"/v1/chat/completions",models:"/v1/models"}}}o(n,"info");export{n as info};
1
+ var r=Object.defineProperty;var o=(e,t)=>r(e,"name",{value:t,configurable:!0});function n(){return{name:"openrouter",protocol:"openai.chat",order:37,auth:"bearer",strict:!1,hosts:["openrouter.ai"],paths:{chat:"/v1/chat/completions",models:"/v1/models"}}}o(n,"info");export{n as info};
2
2
  //# sourceMappingURL=info.mjs.map
@@ -1,3 +1,3 @@
1
- var I=Object.defineProperty;var u=(e,o)=>I(e,"name",{value:o,configurable:!0});import{context as C,self as L}from"../lib/runtime.mjs";import{capture as M}from"../lib/capture.mjs";import{resolve as B}from"../lib/route.mjs";import*as y from"../lib/http.mjs";import{filterChatBody as E}from"../lib/passthrough.mjs";import{addSampling as P}from"../lib/sampling.mjs";import*as _ from"../lib/images.mjs";import*as x from"../lib/tools.mjs";import{LIMITS as R}from"../lib/toolsearch.mjs";import{warn as v}from"../lib/warn.mjs";import{emit as H}from"../lib/events.mjs";import{fit as z,pinBody as A}from"../lib/effort.mjs";import{relax as D,strictTools as G}from"../lib/strict.mjs";import{TranslatorError as J}from"../lib/errors.mjs";import*as K from"../lib/vllm.mjs";function N(e){return`${e.driver}:${e.protocol}:${URL.canParse(e.url)?new URL(e.url).host:e.url}`}u(N,"producerOf");async function ie(e,o={}){const n=M(L,C,o.front),m=B(n,e,{endpoint:o.endpoint,advance:o.hold===!0&&o.endpoint===void 0});if(o.hold!==!0)return T(n,e,o,m);m.endpoint.health.inflight++;try{return await T(n,e,o,m)}catch(t){throw m.endpoint.health.inflight--,t}}u(ie,"request");function Q(e,o,n,m,t,l){const j=G(o);if(j.length===0)return o;const a=j.filter(s=>s.problems.length>0);if(a.length>0&&t.unknownFields==="error")throw new J(`tools: ${a[0].name}: schema is outside the strict subset (${a[0].problems.join("; ")})`,{code:"unsupported_content",param:"tools"});const b=D(o);if(l.retry===!0)return b;const h=u((s,g)=>v(e.config,e.state,s,{kind:g,endpoint:n.name,log:l.log}),"tell");for(const s of a)h(`Upstream ${n.name}: strict tool ${s.name} has a schema outside the strict subset: ${s.problems.join("; ")} (sent without strict)`,"strict");const d=j.filter(s=>s.problems.length===0);return m.info.strict!==!0&&d.length>0&&h(`Upstream ${n.name}: the ${m.name} backend does not enforce strict tool use (${d.map(s=>s.name).join(", ")})`,"strict"),b}u(Q,"checkStrict");async function T(e,o,n,m){const{endpoint:t,model:l,unserved:j,convert:a,pool:b,pin:h}=m,d=Object.hasOwn(e.config.efforts,l)?e.config.efforts[l]:void 0;h&&(o=A(o,h,e.front));const s=e.driver(t),g=`${t.name}
2
- ${l}`,U=n.images===!1||e.state.textOnly.has(g)&&e.settings(t).imageFallback,$=e.settings(t),q=n.jsonObject==="instruction"||e.state.jsonInstruction.has(g)&&$.jsonObjectFallback,c={...U?{...$,images:!1}:$,...q?{jsonObject:"instruction"}:{},...h?.effort===void 0?{}:{effort:!0}},w=o.stream===!0,p=a===null?e.front==="anthropic"?"anthropic.messages":"openai.chat":t.protocol,O=p==="anthropic.messages"&&t.protocol==="openai.chat";let r,i;a?({body:r,state:i}=await a.request(o,c,{producer:N(t),budget:n.budget,listed:d!==void 0,toolCallIds:s.info.toolCallIds}),i.toolSearch?.capped&&v(e.config,e.state,`Tool search: more than ${R.definitions} tool definitions came back in ToolSearch results for ${t.name}; the rest are not offered.`,{kind:"toolsearch",endpoint:t.name,log:n.log})):O?r=K.request(o,c):r=e.front==="openai"&&(n.filter??c.passthroughFilter===!0)?E(o,s.info.fields):{...o};const F=e.front==="anthropic"?o.metadata?.sampling:void 0;if(p==="openai.chat"){const f=[];r=P(r,F,c,s.info.fields,f),i&&i.dropped.push(...f.filter(k=>!i.dropped.includes(k)))}else if(!a&&F!==void 0&&r.metadata?.sampling!==void 0){const{sampling:f,...k}=r.metadata;r={...r,metadata:k}}if(n.usage===!0&&w&&e.front==="openai"){if(a)i.includeUsage=!0;else if(c.streamUsage){const f=r.stream_options&&typeof r.stream_options=="object"?r.stream_options:{};r.stream_options={...f,include_usage:!0}}}U&&!a&&e.front==="openai"&&(r=_.strip(r)),n.tools===!1&&(r=x.strip(r)),r.model=l,r=Q(e,r,t,s,c,n),d&&p==="openai.chat"&&z(r,d,l,(f,k)=>v(e.config,e.state,f,{type:"effort",endpoint:t.name,model:l,...k,log:n.log})),i?.reasoningMissing&&v(e.config,e.state,`Upstream ${t.name}: assistant turns with tool calls have no reasoning to send back (the client dropped its thinking blocks or their signatures), which servers that need it with tools will refuse.`,{kind:"reasoning",endpoint:t.name,log:n.log}),s.request&&!O&&(r=await s.request(r,{endpoint:t,settings:c,levels:d,state:i,warn:u(f=>v(e.config,e.state,f,{kind:"driver",endpoint:t.name,log:n.log}),"warn")})),i?.dropped?.length>0&&n.retry!==!0&&H(e,{type:"dropped",level:"debug",message:`Upstream ${t.name}: dropped ${i.dropped.join(", ")}`,endpoint:t.name,model:l,fields:[...i.dropped],log:n.log});const S=await y.pathOf(s,p==="anthropic.messages"?"messages":"chat",t,l);return{upstream:t.name,pool:b.map(f=>f.name),method:"POST",url:y.join(t.url,S),path:S,headers:{"content-type":"application/json",accept:w?"text/event-stream":"application/json",...s.info.headers,...a===null&&t.protocol==="anthropic.messages"?n.clientHeaders:void 0,...y.headers(t,s.info.auth,n.headers)},body:r,state:{v:1,upstream:t.name,front:e.front,native:a===null,messages:O,model:o.model,stream:w,unserved:j,convert:i},images:{sent:_.carries(r),retry:c.imageFallback&&p==="openai.chat",key:g},tools:{sent:x.carries(r),retry:c.toolFallback&&p==="openai.chat"},json:{sent:i?.jsonSchemaEmulated===!0,retry:c.jsonObjectFallback&&p==="anthropic.messages",key:g}}}u(T,"prepare");export{ie as request};
1
+ var M=Object.defineProperty;var h=(e,a)=>M(e,"name",{value:a,configurable:!0});import{context as B,self as E}from"../lib/runtime.mjs";import{capture as K}from"../lib/capture.mjs";import{resolve as P}from"../lib/route.mjs";import*as S from"../lib/http.mjs";import{filterChatBody as R}from"../lib/passthrough.mjs";import{addSampling as H}from"../lib/sampling.mjs";import{addConstraint as z}from"../lib/constraint.mjs";import{addCache as A}from"../lib/cache.mjs";import*as C from"../lib/images.mjs";import*as T from"../lib/tools.mjs";import{LIMITS as D}from"../lib/toolsearch.mjs";import{warn as v}from"../lib/warn.mjs";import{emit as G}from"../lib/events.mjs";import{fit as J,pinBody as N}from"../lib/effort.mjs";import{relax as Q,strictTools as V}from"../lib/strict.mjs";import{TranslatorError as W}from"../lib/errors.mjs";import*as X from"../lib/vllm.mjs";function Y(e){return`${e.driver}:${e.protocol}:${URL.canParse(e.url)?new URL(e.url).host:e.url}`}h(Y,"producerOf");async function ue(e,a={}){const r=K(E,B,a.front),f=P(r,e,{endpoint:a.endpoint,advance:a.hold===!0&&a.endpoint===void 0});if(a.hold!==!0)return q(r,e,a,f);f.endpoint.health.inflight++;try{return await q(r,e,a,f)}catch(n){throw f.endpoint.health.inflight--,n}}h(ue,"request");function Z(e,a,r,f,n,p){const k=V(a);if(k.length===0)return a;const l=k.filter(s=>s.problems.length>0);if(l.length>0&&n.unknownFields==="error")throw new W(`tools: ${l[0].name}: schema is outside the strict subset (${l[0].problems.join("; ")})`,{code:"unsupported_content",param:"tools"});const _=Q(a);if(p.retry===!0)return _;const g=h((s,j)=>v(e.config,e.state,s,{kind:j,endpoint:r.name,log:p.log}),"tell");for(const s of l)g(`Upstream ${r.name}: strict tool ${s.name} has a schema outside the strict subset: ${s.problems.join("; ")} (sent without strict)`,"strict");const u=k.filter(s=>s.problems.length===0);return f.info.strict!==!0&&u.length>0&&g(`Upstream ${r.name}: the ${f.name} backend does not enforce strict tool use (${u.map(s=>s.name).join(", ")})`,"strict"),_}h(Z,"checkStrict");async function q(e,a,r,f){const{endpoint:n,model:p,unserved:k,convert:l,pool:_,pin:g}=f,u=Object.hasOwn(e.config.efforts,p)?e.config.efforts[p]:void 0;g&&(a=N(a,g,e.front));const s=e.driver(n),j=e.front==="anthropic"?a.metadata?.prompt_cache_key:void 0;let w=e.front==="anthropic"?a.metadata?.constraint:void 0;e.front==="openai"&&a.constraint!==void 0&&({constraint:w,...a}=a);const $=`${n.name}
2
+ ${p}`,U=r.images===!1||e.state.textOnly.has($)&&e.settings(n).imageFallback,b=e.settings(n),I=r.jsonObject==="instruction"||e.state.jsonInstruction.has($)&&b.jsonObjectFallback,c={...U?{...b,images:!1}:b,...I?{jsonObject:"instruction"}:{},...g?.effort===void 0?{}:{effort:!0}},y=a.stream===!0,m=l===null?e.front==="anthropic"?"anthropic.messages":"openai.chat":n.protocol,O=m==="anthropic.messages"&&n.protocol==="openai.chat";let t,o;l?({body:t,state:o}=await l.request(a,c,{producer:Y(n),budget:r.budget,listed:u!==void 0,toolCallIds:s.info.toolCallIds}),o.toolSearch?.capped&&v(e.config,e.state,`Tool search: more than ${D.definitions} tool definitions came back in ToolSearch results for ${n.name}; the rest are not offered.`,{kind:"toolsearch",endpoint:n.name,log:r.log})):O?t=X.request(a,c):t=e.front==="openai"&&(r.filter??c.passthroughFilter===!0)?R(a,s.info.fields):{...a};const F=e.front==="anthropic"?a.metadata?.sampling:void 0;if(m==="openai.chat"){const i=[];t=H(t,F,c,s.info.fields,i),o&&o.dropped.push(...i.filter(d=>!o.dropped.includes(d)))}else if(!l&&F!==void 0&&t.metadata?.sampling!==void 0){const{sampling:i,...d}=t.metadata;t={...t,metadata:d}}if(m==="openai.chat"){const i=[],d={name:s.info.name,version:n.facts?.version??n.version};t=z(t,w,c,d,i),t=A(t,{salt:r.cacheSalt,key:j},c,d.name,i),o&&o.dropped.push(...i.filter(L=>!o.dropped.includes(L)))}else{if(o&&!o.dropped.includes("cache_salt")&&(r.cacheSalt??c.cacheSalt)!=null&&o.dropped.push("cache_salt"),o&&!o.dropped.includes("prompt_cache_key")&&(j??c.promptCacheKey)!=null&&o.dropped.push("prompt_cache_key"),!l&&t.metadata?.prompt_cache_key!==void 0){const{prompt_cache_key:i,...d}=t.metadata;t={...t,metadata:d}}if(o&&(w??c.constraint)!=null&&!o.dropped.includes("constraint")&&o.dropped.push("constraint"),!l&&t.metadata?.constraint!==void 0){const{constraint:i,...d}=t.metadata;t={...t,metadata:d}}}if(r.usage===!0&&y&&e.front==="openai"){if(l)o.includeUsage=!0;else if(c.streamUsage){const i=t.stream_options&&typeof t.stream_options=="object"?t.stream_options:{};t.stream_options={...i,include_usage:!0}}}U&&!l&&e.front==="openai"&&(t=C.strip(t)),r.tools===!1&&(t=T.strip(t)),t.model=p,t=Z(e,t,n,s,c,r),u&&m==="openai.chat"&&J(t,u,p,(i,d)=>v(e.config,e.state,i,{type:"effort",endpoint:n.name,model:p,...d,log:r.log})),o?.reasoningMissing&&v(e.config,e.state,`Upstream ${n.name}: assistant turns with tool calls have no reasoning to send back (the client dropped its thinking blocks or their signatures), which servers that need it with tools will refuse.`,{kind:"reasoning",endpoint:n.name,log:r.log}),s.request&&!O&&(t=await s.request(t,{endpoint:n,settings:c,levels:u,state:o,warn:h(i=>v(e.config,e.state,i,{kind:"driver",endpoint:n.name,log:r.log}),"warn")})),o?.dropped?.length>0&&r.retry!==!0&&G(e,{type:"dropped",level:"debug",message:`Upstream ${n.name}: dropped ${o.dropped.join(", ")}`,endpoint:n.name,model:p,fields:[...o.dropped],log:r.log});const x=await S.pathOf(s,m==="anthropic.messages"?"messages":"chat",n,p);return{upstream:n.name,pool:_.map(i=>i.name),method:"POST",url:S.join(n.url,x),path:x,headers:{"content-type":"application/json",accept:y?"text/event-stream":"application/json",...s.info.headers,...l===null&&n.protocol==="anthropic.messages"?r.clientHeaders:void 0,...S.headers(n,s.info.auth,r.headers)},body:t,state:{v:1,upstream:n.name,front:e.front,native:l===null,messages:O,model:a.model,stream:y,unserved:k,convert:o},images:{sent:C.carries(t),retry:c.imageFallback&&m==="openai.chat",key:$},tools:{sent:T.carries(t),retry:c.toolFallback&&m==="openai.chat"},json:{sent:o?.jsonSchemaEmulated===!0,retry:c.jsonObjectFallback&&m==="anthropic.messages",key:$}}}h(q,"prepare");export{ue as request};
3
3
  //# sourceMappingURL=request.mjs.map
package/dist/api/send.mjs CHANGED
@@ -1,2 +1,2 @@
1
- var le=Object.defineProperty;var u=(c,r)=>le(c,"name",{value:r,configurable:!0});import{context as K,self as ee}from"../lib/runtime.mjs";import{TranslatorError as z}from"../lib/errors.mjs";import*as te from"../lib/http.mjs";import*as ue from"../lib/images.mjs";import*as ce from"../lib/tools.mjs";import{parse as ae}from"../lib/upstream-error.mjs";import{meter as me,normalize as de}from"../lib/usage.mjs";import{capture as fe}from"../lib/capture.mjs";import{estimate as pe,bodyLength as ge,output as he,patch as ye,stream as we}from"../lib/estimate.mjs";import{warn as se}from"../lib/warn.mjs";import{start as be}from"../lib/record.mjs";import{emit as re}from"../lib/events.mjs";import*as ne from"../lib/pool.mjs";import{createBudget as ke}from"../lib/thinking.mjs";import*as oe from"../lib/fanout.mjs";import{refresh as je}from"../lib/health.mjs";const ve={"content-type":"text/event-stream; charset=utf-8","cache-control":"no-cache","x-accel-buffering":"no"};async function _e(c,r={}){const{transport:w,textOnly:f,jsonInstruction:v}=K.state,{request:_,response:$,stream:o,error:p}=ee,d=p.build,{config:i,state:b}=K,g=fe(ee,K,r.front),U=c&&typeof c=="object"&&!Array.isArray(c),T={model:null,requested:U&&typeof c.model=="string"?c.model:null,upstream:null,front:r.front??i.front,stream:U&&c.stream===!0},k=me(T,r.onUsage,(m,A)=>se(i,b,m,{type:"usage",...A,log:r.log})),q=u(m=>Object.defineProperty(m,"usage",{value:k.usage,enumerable:!1,configurable:!0}),"attach");let J=!1;try{const m=oe.wants(g,c);if(m>0){J=!0;const A=u((n,S)=>se(i,b,n,{type:"usage",...S,log:r.log}),"warning");return await oe.run({body:c,count:m,options:r,send:_e,warn:A})}return q(await $e({transport:w,textOnly:f,jsonInstruction:v,request:_,response:$,stream:o,error:p,captured:g},c,r,k))}catch(m){if(J)throw m;if(!(m instanceof z))throw r.signal?.aborted||re(g,{type:"error",kind:"exception",level:"error",message:String(m?.message??m),log:r.log}),k.finish(r.signal?.aborted?"aborted":"error"),m;return k.finish("error"),q(d(m.status,m.message,{type:m.type,code:m.code,param:m.param,front:r.front}))}}u(_e,"send");async function $e({transport:c,textOnly:r,jsonInstruction:w,request:f,response:v,stream:_,error:$,captured:o},p,d,i){const b={front:d.front,headers:d.headers,clientHeaders:d.clientHeaders,filter:d.filter,usage:!0,budget:ke(),log:d.log},g=u(t=>{re(o,{...t,log:d.log})},"announce"),U=i.meta.front==="openai"&&p?.stream_options?.include_usage!==!0,T=u(t=>{i.meta.model=t.body.model??null,i.meta.upstream=t.upstream,t.state.native?i.source(null):i.source(()=>t.state.convert?.upstreamUsage,t.state.front==="anthropic"?"openai":"folded")},"note"),k=u(t=>{if(t.state.native)return{input:!1,output:!1};const e=de(t.state.front==="anthropic"?"openai":"folded",t.state.convert?.upstreamUsage);return e===null?{input:!0,output:!0}:{input:e.input_tokens===null&&e.cache_read_input_tokens===null&&e.cache_creation_input_tokens===null,output:e.output_tokens===null}},"gaps"),q=u(async t=>{const e=o.endpoint(t.upstream),a=o.driver(e);return(a.count&&!(U&&t.state.stream)?await a.count({endpoint:e,request:t.body,transport:c,signal:d.signal,timeout:o.config.detectTimeout}):null)??pe(t.body)},"input"),J=u(async(t,e)=>{const a=k(t);return!a.input&&!a.output?null:{input_tokens:a.input?await q(t):null,output_tokens:a.output?he(e):null}},"counts"),m=u((t,e,a,s,R)=>(i.finish(R),$(t,e,a,s)),"fail"),A=u(async t=>{let e=t,a=await Q(c,e,d.signal,I());const s=a.status;if(s===404&&e.state.messages){const y=await C(a);return y?.type==="error"?{upstream:e,result:a,failure:y}:(o.state.noMessages.add(e.upstream),g({type:"fallback",kind:"messages",level:"warn",message:`Upstream ${e.upstream}: /v1/messages answered 404; sent the request converted to Chat Completions, and will do so for this upstream until the next detection. Set nativeMessages: false on this upstream to skip the first attempt.`,endpoint:e.upstream,model:e.body.model,status:s,reason:String(ae(y,s).message).trim().slice(0,500),action:"conversion"}),e=await f(p,{...b,endpoint:e.upstream,retry:!0}),T(e),{upstream:e,result:await Q(c,e,d.signal,I())})}const R=s===400&&[e.images,e.tools].some(y=>y.sent&&y.retry),F=(s===400||s===422)&&e.json.sent&&e.json.retry;if(!a.ok&&(R||F)){const y=await C(a),O=ae(y,s).message,h={};if(s===400&&e.images.sent&&e.images.retry&&ue.refused(O)?(r.add(e.images.key),h.images=!1):s===400&&e.tools.sent&&e.tools.retry&&ce.refused(O)?h.tools=!1:F&&(h.jsonObject="instruction"),!("images"in h||"tools"in h||"jsonObject"in h))return{upstream:e,result:a,failure:y};const j={upstream:e,result:a,failure:y};if("images"in h||"tools"in h){const G="images"in h?"images":"tools";g({type:"fallback",kind:G,level:"warn",message:`Upstream ${e.upstream}: ${e.state.model} refused ${G==="images"?"images":"tools"} (${s}: "${String(O).trim().slice(0,500)}"); sent the request again without them.`,endpoint:e.upstream,model:e.body.model,status:s,reason:String(O).trim().slice(0,500),action:G==="images"?"text-only":"without-tools"})}if(e=await f(p,{...b,...h,endpoint:e.upstream,retry:!0}),T(e),a=await Q(c,e,d.signal,I()),F&&"jsonObject"in h){if(!a.ok){try{await a.body?.cancel?.()}catch{}return T(j.upstream),j}w.add(j.upstream.json.key),g({type:"fallback",kind:"json_object",level:"warn",message:`Upstream ${j.upstream.upstream}: ${j.upstream.state.model} rejected the json_object schema emulation (${s}: "${String(O).trim().slice(0,500)}"); sent it as a system instruction instead. Set jsonObject: "instruction" on this upstream to skip the first attempt.`,endpoint:j.upstream.upstream,model:j.upstream.body.model,status:s,reason:String(O).trim().slice(0,500),action:"instruction"})}}return{upstream:e,result:a}},"attempt");await je(o,p,d.signal);let n=await f(p,{...b,hold:!0});const S=n.pool;let B=0;const D=S.length>1?o.config.pool:null,I=u(()=>D&&{connectTimeout:D.connectTimeout,headersTimeout:B+1<S.length?D.headersTimeout:0},"limitsNow");let E=o.endpoint(n.upstream);const V=u(()=>E.health.inflight--,"release");let l,x,W=null,X=0;const Y=u(t=>{const e=be(o,{body:p,sent:n,front:i.meta.front,requested:i.meta.requested,headers:t,log:d.log,exchange:W,attempt:X});return W??=e.id,e},"begin"),Z=u(async t=>{const e=Y();if(!e.on)return;if(l===void 0)return e.attempt({error:x.message},t);let a=null;try{a=x??await C(l)}catch{}e.attempt({status:l.status,headers:te.plain(l.headers),body:a},t)},"keep");try{for(;;){const t=E;t.health.requests++,X++,T(n);let e=null;try{({upstream:n,result:l,failure:x}=await A(n)),!l.ok&&l.status>=500&&(e=`HTTP ${l.status}`,g({type:"error",kind:"upstream",level:"error",message:`Upstream ${t.name} answered HTTP ${l.status}`,endpoint:t.name,model:n.body.model,status:l.status}))}catch(s){if(!(s instanceof z)||s.code!=="upstream_unreachable")throw s;e=s.message,l=void 0,x=s,g({type:"error",kind:"transport",level:"error",message:s.message,endpoint:t.name,model:n.body.model})}if(e===null){ne.succeeded(t);break}ne.failed(o.config,t,e);let a=null;for(;a===null&&!d.signal?.aborted&&B+1<S.length;)try{a=await f(p,{...b,endpoint:S[++B],hold:!0})}catch(s){if(!(s instanceof z)||s.code!=="endpoint_not_in_pool")throw s}if(a===null){if(l===void 0)throw await Z(),x;break}await Z(e),g({type:"failover",level:"warn",message:`Upstream ${t.name} failed (${e}); trying ${a.upstream}`,from:t.name,to:a.upstream,model:a.body.model,reason:e});try{await l?.body?.cancel?.()}catch{}V(),E=o.endpoint(a.upstream),n=a}}catch(t){throw V(),t}const ie=E;E=null,i.usage.then(()=>ie.health.inflight--);const H=te.plain(l.headers),M=Y(H);if(!l.ok){const t=x??await C(l);return M.reply(l.status,H,t),m(l.status,t,H,n.state,"upstream_error")}if(n.state.stream){const t=_(M.raw(l.body),n.state),e=n.state.native?t:we(i.meta.front,t,{gaps:u(()=>k(n),"gaps"),input:u(()=>q(n),"input"),report:u(a=>i.estimate(a),"report")});return{status:200,headers:ve,stream:M.events(i.track(e,d.signal,U))}}const P=await C(l);if(M.reply(l.status,H,P),P===null||typeof P!="object")return g({type:"error",kind:"upstream",level:"error",message:`Upstream ${n.upstream} returned a body that is not JSON`,endpoint:n.upstream,model:n.body.model,status:502}),m(502,"The upstream returned a body that is not JSON",H,n.state,"error");const N=await v(P,n.state),L=await J(n,ge(i.meta.front,N));return L&&(N.usage=ye(i.meta.front,N.usage,L),i.estimate(L)),i.json(N),i.finish("done"),{status:200,headers:{"content-type":"application/json"},body:N}}u($e,"round");async function Q(c,r,w,f=null){let v,_=!1,$=w;if(f&&f.headersTimeout>0){const o=new AbortController;$=w?AbortSignal.any([w,o.signal]):o.signal,v=setTimeout(()=>{_=!0,o.abort()},f.headersTimeout)}try{return await c(r.url,{method:r.method,headers:r.headers,body:JSON.stringify(r.body),signal:$,redirect:"error",...f&&f.connectTimeout>0?{connectTimeout:f.connectTimeout}:{}})}catch(o){if(w?.aborted)throw o;const p=_?`no response headers within ${f.headersTimeout} ms`:o.message;throw new z(`Could not reach the upstream (${r.upstream}): ${p}`,{code:"upstream_unreachable",status:502,details:{cause:o}})}finally{clearTimeout(v)}}u(Q,"call");async function C(c){const r=await c.text();try{return JSON.parse(r)}catch{return r===""?null:r}}u(C,"parse");export{_e as send};
1
+ var le=Object.defineProperty;var u=(c,s)=>le(c,"name",{value:s,configurable:!0});import{context as K,self as ee}from"../lib/runtime.mjs";import{TranslatorError as z}from"../lib/errors.mjs";import*as te from"../lib/http.mjs";import*as ue from"../lib/images.mjs";import*as ce from"../lib/tools.mjs";import{parse as ae}from"../lib/upstream-error.mjs";import{meter as me,normalize as de}from"../lib/usage.mjs";import{capture as fe}from"../lib/capture.mjs";import{estimate as pe,bodyLength as ge,output as he,patch as ye,stream as we}from"../lib/estimate.mjs";import{warn as re}from"../lib/warn.mjs";import{start as be}from"../lib/record.mjs";import{emit as se}from"../lib/events.mjs";import*as ne from"../lib/pool.mjs";import{createBudget as ke}from"../lib/thinking.mjs";import*as oe from"../lib/fanout.mjs";import{refresh as je}from"../lib/health.mjs";const ve={"content-type":"text/event-stream; charset=utf-8","cache-control":"no-cache","x-accel-buffering":"no"};async function _e(c,s={}){const{transport:w,textOnly:f,jsonInstruction:v}=K.state,{request:_,response:$,stream:o,error:p}=ee,d=p.build,{config:i,state:b}=K,g=fe(ee,K,s.front),U=c&&typeof c=="object"&&!Array.isArray(c),T={model:null,requested:U&&typeof c.model=="string"?c.model:null,upstream:null,front:s.front??i.front,stream:U&&c.stream===!0},k=me(T,s.onUsage,(m,A)=>re(i,b,m,{type:"usage",...A,log:s.log})),q=u(m=>Object.defineProperty(m,"usage",{value:k.usage,enumerable:!1,configurable:!0}),"attach");let J=!1;try{const m=oe.wants(g,c);if(m>0){J=!0;const A=u((n,S)=>re(i,b,n,{type:"usage",...S,log:s.log}),"warning");return await oe.run({body:c,count:m,options:s,send:_e,warn:A})}return q(await $e({transport:w,textOnly:f,jsonInstruction:v,request:_,response:$,stream:o,error:p,captured:g},c,s,k))}catch(m){if(J)throw m;if(!(m instanceof z))throw s.signal?.aborted||se(g,{type:"error",kind:"exception",level:"error",message:String(m?.message??m),log:s.log}),k.finish(s.signal?.aborted?"aborted":"error"),m;return k.finish("error"),q(d(m.status,m.message,{type:m.type,code:m.code,param:m.param,front:s.front}))}}u(_e,"send");async function $e({transport:c,textOnly:s,jsonInstruction:w,request:f,response:v,stream:_,error:$,captured:o},p,d,i){const b={front:d.front,headers:d.headers,clientHeaders:d.clientHeaders,filter:d.filter,cacheSalt:d.cacheSalt,usage:!0,budget:ke(),log:d.log},g=u(t=>{se(o,{...t,log:d.log})},"announce"),U=i.meta.front==="openai"&&p?.stream_options?.include_usage!==!0,T=u(t=>{i.meta.model=t.body.model??null,i.meta.upstream=t.upstream,t.state.native?i.source(null):i.source(()=>t.state.convert?.upstreamUsage,t.state.front==="anthropic"?"openai":"folded")},"note"),k=u(t=>{if(t.state.native)return{input:!1,output:!1};const e=de(t.state.front==="anthropic"?"openai":"folded",t.state.convert?.upstreamUsage);return e===null?{input:!0,output:!0}:{input:e.input_tokens===null&&e.cache_read_input_tokens===null&&e.cache_creation_input_tokens===null,output:e.output_tokens===null}},"gaps"),q=u(async t=>{const e=o.endpoint(t.upstream),a=o.driver(e);return(a.count&&!(U&&t.state.stream)?await a.count({endpoint:e,request:t.body,transport:c,signal:d.signal,timeout:o.config.detectTimeout}):null)??pe(t.body)},"input"),J=u(async(t,e)=>{const a=k(t);return!a.input&&!a.output?null:{input_tokens:a.input?await q(t):null,output_tokens:a.output?he(e):null}},"counts"),m=u((t,e,a,r,R)=>(i.finish(R),$(t,e,a,r)),"fail"),A=u(async t=>{let e=t,a=await Q(c,e,d.signal,I());const r=a.status;if(r===404&&e.state.messages){const y=await C(a);return y?.type==="error"?{upstream:e,result:a,failure:y}:(o.state.noMessages.add(e.upstream),g({type:"fallback",kind:"messages",level:"warn",message:`Upstream ${e.upstream}: /v1/messages answered 404; sent the request converted to Chat Completions, and will do so for this upstream until the next detection. Set nativeMessages: false on this upstream to skip the first attempt.`,endpoint:e.upstream,model:e.body.model,status:r,reason:String(ae(y,r).message).trim().slice(0,500),action:"conversion"}),e=await f(p,{...b,endpoint:e.upstream,retry:!0}),T(e),{upstream:e,result:await Q(c,e,d.signal,I())})}const R=r===400&&[e.images,e.tools].some(y=>y.sent&&y.retry),F=(r===400||r===422)&&e.json.sent&&e.json.retry;if(!a.ok&&(R||F)){const y=await C(a),O=ae(y,r).message,h={};if(r===400&&e.images.sent&&e.images.retry&&ue.refused(O)?(s.add(e.images.key),h.images=!1):r===400&&e.tools.sent&&e.tools.retry&&ce.refused(O)?h.tools=!1:F&&(h.jsonObject="instruction"),!("images"in h||"tools"in h||"jsonObject"in h))return{upstream:e,result:a,failure:y};const j={upstream:e,result:a,failure:y};if("images"in h||"tools"in h){const G="images"in h?"images":"tools";g({type:"fallback",kind:G,level:"warn",message:`Upstream ${e.upstream}: ${e.state.model} refused ${G==="images"?"images":"tools"} (${r}: "${String(O).trim().slice(0,500)}"); sent the request again without them.`,endpoint:e.upstream,model:e.body.model,status:r,reason:String(O).trim().slice(0,500),action:G==="images"?"text-only":"without-tools"})}if(e=await f(p,{...b,...h,endpoint:e.upstream,retry:!0}),T(e),a=await Q(c,e,d.signal,I()),F&&"jsonObject"in h){if(!a.ok){try{await a.body?.cancel?.()}catch{}return T(j.upstream),j}w.add(j.upstream.json.key),g({type:"fallback",kind:"json_object",level:"warn",message:`Upstream ${j.upstream.upstream}: ${j.upstream.state.model} rejected the json_object schema emulation (${r}: "${String(O).trim().slice(0,500)}"); sent it as a system instruction instead. Set jsonObject: "instruction" on this upstream to skip the first attempt.`,endpoint:j.upstream.upstream,model:j.upstream.body.model,status:r,reason:String(O).trim().slice(0,500),action:"instruction"})}}return{upstream:e,result:a}},"attempt");await je(o,p,d.signal);let n=await f(p,{...b,hold:!0});const S=n.pool;let B=0;const D=S.length>1?o.config.pool:null,I=u(()=>D&&{connectTimeout:D.connectTimeout,headersTimeout:B+1<S.length?D.headersTimeout:0},"limitsNow");let E=o.endpoint(n.upstream);const V=u(()=>E.health.inflight--,"release");let l,x,W=null,X=0;const Y=u(t=>{const e=be(o,{body:p,sent:n,front:i.meta.front,requested:i.meta.requested,headers:t,log:d.log,exchange:W,attempt:X});return W??=e.id,e},"begin"),Z=u(async t=>{const e=Y();if(!e.on)return;if(l===void 0)return e.attempt({error:x.message},t);let a=null;try{a=x??await C(l)}catch{}e.attempt({status:l.status,headers:te.plain(l.headers),body:a},t)},"keep");try{for(;;){const t=E;t.health.requests++,X++,T(n);let e=null;try{({upstream:n,result:l,failure:x}=await A(n)),!l.ok&&l.status>=500&&(e=`HTTP ${l.status}`,g({type:"error",kind:"upstream",level:"error",message:`Upstream ${t.name} answered HTTP ${l.status}`,endpoint:t.name,model:n.body.model,status:l.status}))}catch(r){if(!(r instanceof z)||r.code!=="upstream_unreachable")throw r;e=r.message,l=void 0,x=r,g({type:"error",kind:"transport",level:"error",message:r.message,endpoint:t.name,model:n.body.model})}if(e===null){ne.succeeded(t);break}ne.failed(o.config,t,e);let a=null;for(;a===null&&!d.signal?.aborted&&B+1<S.length;)try{a=await f(p,{...b,endpoint:S[++B],hold:!0})}catch(r){if(!(r instanceof z)||r.code!=="endpoint_not_in_pool")throw r}if(a===null){if(l===void 0)throw await Z(),x;break}await Z(e),g({type:"failover",level:"warn",message:`Upstream ${t.name} failed (${e}); trying ${a.upstream}`,from:t.name,to:a.upstream,model:a.body.model,reason:e});try{await l?.body?.cancel?.()}catch{}V(),E=o.endpoint(a.upstream),n=a}}catch(t){throw V(),t}const ie=E;E=null,i.usage.then(()=>ie.health.inflight--);const H=te.plain(l.headers),M=Y(H);if(!l.ok){const t=x??await C(l);return M.reply(l.status,H,t),m(l.status,t,H,n.state,"upstream_error")}if(n.state.stream){const t=_(M.raw(l.body),n.state),e=n.state.native?t:we(i.meta.front,t,{gaps:u(()=>k(n),"gaps"),input:u(()=>q(n),"input"),report:u(a=>i.estimate(a),"report")});return{status:200,headers:ve,stream:M.events(i.track(e,d.signal,U))}}const P=await C(l);if(M.reply(l.status,H,P),P===null||typeof P!="object")return g({type:"error",kind:"upstream",level:"error",message:`Upstream ${n.upstream} returned a body that is not JSON`,endpoint:n.upstream,model:n.body.model,status:502}),m(502,"The upstream returned a body that is not JSON",H,n.state,"error");const N=await v(P,n.state),L=await J(n,ge(i.meta.front,N));return L&&(N.usage=ye(i.meta.front,N.usage,L),i.estimate(L)),i.json(N),i.finish("done"),{status:200,headers:{"content-type":"application/json"},body:N}}u($e,"round");async function Q(c,s,w,f=null){let v,_=!1,$=w;if(f&&f.headersTimeout>0){const o=new AbortController;$=w?AbortSignal.any([w,o.signal]):o.signal,v=setTimeout(()=>{_=!0,o.abort()},f.headersTimeout)}try{return await c(s.url,{method:s.method,headers:s.headers,body:JSON.stringify(s.body),signal:$,redirect:"error",...f&&f.connectTimeout>0?{connectTimeout:f.connectTimeout}:{}})}catch(o){if(w?.aborted)throw o;const p=_?`no response headers within ${f.headersTimeout} ms`:o.message;throw new z(`Could not reach the upstream (${s.upstream}): ${p}`,{code:"upstream_unreachable",status:502,details:{cause:o}})}finally{clearTimeout(v)}}u(Q,"call");async function C(c){const s=await c.text();try{return JSON.parse(s)}catch{return s===""?null:s}}u(C,"parse");export{_e as send};
2
2
  //# sourceMappingURL=send.mjs.map
@@ -1,2 +1,2 @@
1
- var w=Object.defineProperty;var n=(o,t)=>w(o,"name",{value:t,configurable:!0});import{context as s,self as u}from"../../lib/runtime.mjs";import{TranslatorError as d}from"../../lib/errors.mjs";import{register as k}from"../../lib/routes.mjs";import{capture as x}from"../../lib/capture.mjs";import{resolve as y}from"../../lib/route.mjs";import{resolve as I}from"../../lib/settings.mjs";import{emit as T}from"../../lib/events.mjs";async function N(o,t={}){const{config:r}=s;if(t.front!==void 0&&t.front!=="anthropic"&&t.front!=="openai")throw new d(`front must be "anthropic" or "openai" (got ${JSON.stringify(t.front)})`,{code:"invalid_config",status:500});for(const e of["authorize","onUsage"])if(t[e]!==void 0&&typeof t[e]!="function")throw new d(`${e} must be a function`,{code:"invalid_config",status:500});const{send:m,route:p,token:h,model:c,error:v,sse:f,embeddings:b}=u,l=t.front??r.front,{state:g}=s,{events:i}=g;i.logger=o.log,o.addHook("onClose",async()=>{i.logger===o.log&&(i.logger=null)});const a=x(u,s,l);k(o,t,{front:l,filter:r.settings.passthroughFilter!==!1,interval:n(e=>{try{return a.settings(y(a,e).endpoint).pingInterval}catch{return I(r.settings).pingInterval}},"interval"),send:m,embed:b.send,route:p,count:h.count,list:c.list,get:c.get,emit:n(e=>T({config:r,state:g},e),"emit"),build:v.build,encode:f.encode,write:f.write})}n(N,"plugin");export{N as plugin};
1
+ var w=Object.defineProperty;var n=(r,e)=>w(r,"name",{value:e,configurable:!0});import{context as c,self as g}from"../../lib/runtime.mjs";import{TranslatorError as f}from"../../lib/errors.mjs";import{register as y}from"../../lib/routes.mjs";import{capture as _}from"../../lib/capture.mjs";import{resolve as k}from"../../lib/route.mjs";import{resolve as x}from"../../lib/settings.mjs";import{emit as I}from"../../lib/events.mjs";async function N(r,e={}){const{config:o}=c;if(e.front!==void 0&&e.front!=="anthropic"&&e.front!=="openai")throw new f(`front must be "anthropic" or "openai" (got ${JSON.stringify(e.front)})`,{code:"invalid_config",status:500});for(const t of["authorize","onUsage"])if(e[t]!==void 0&&typeof e[t]!="function")throw new f(`${t} must be a function`,{code:"invalid_config",status:500});if(e.metrics!==void 0&&typeof e.metrics!="boolean")throw new f("metrics must be true or false",{code:"invalid_config",status:500});const{send:m,route:p,token:h,model:l,error:v,sse:a,embeddings:b}=g,d=e.front??o.front,{state:i}=c,{events:s}=i;s.logger=r.log,r.addHook("onClose",async()=>{s.logger===r.log&&(s.logger=null)});const u=_(g,c,d);y(r,e,{front:d,filter:o.settings.passthroughFilter!==!1,interval:n(t=>{try{return u.settings(k(u,t).endpoint).pingInterval}catch{return x(o.settings).pingInterval}},"interval"),send:m,embed:b.send,route:p,count:h.count,list:l.list,get:l.get,emit:n(t=>I({config:o,state:i},t),"emit"),endpoints:n(()=>i.endpoints,"endpoints"),build:v.build,encode:a.encode,write:a.write})}n(N,"plugin");export{N as plugin};
2
2
  //# sourceMappingURL=plugin.mjs.map
@@ -1,2 +1,2 @@
1
- var m=Object.defineProperty;var s=(t,r)=>m(t,"name",{value:r,configurable:!0});import{context as g,self as u}from"../../lib/runtime.mjs";import{TranslatorError as c}from"../../lib/errors.mjs";import{isLoopback as b}from"../../lib/http.mjs";async function L(t={}){const{state:r}=g,{plugin:d}=u.server,{build:h}=u.error;let i;try{({default:i}=await import("fastify"))}catch(o){throw new c('server.start needs the optional peer dependency "fastify" (npm install fastify)',{code:"fastify_missing",status:500,details:{cause:o}})}const e=t.host??"127.0.0.1";if(!b(e)&&[t.apiKey??[]].flat().filter(Boolean).length===0&&typeof t.authorize!="function"&&t.allowUnauthenticated!==!0)throw new c(`Refusing to listen on ${e} without an apiKey or an authorize hook: anyone who can reach the port could use the upstream. Set \`apiKey\` (CLI: --api-key) or \`authorize\`, bind to 127.0.0.1, or opt in with \`allowUnauthenticated: true\` (CLI: --allow-unauthenticated).`,{code:"unauthenticated_bind",status:500});const a=i({logger:t.logger??!1,forceCloseConnections:!0});a.setNotFoundHandler((o,f)=>{const{status:y,headers:p,body:w}=h(404,`Not found: ${o.method} ${o.url}`);return f.code(y).headers(p).send(w)}),await a.register(d,{front:t.front,apiKey:t.apiKey,bodyLimit:t.bodyLimit,authorize:t.authorize,onUsage:t.onUsage}),await a.listen({host:e,port:t.port??8787});const l=a.server.address().port,n={url:`http://${e.includes(":")?`[${e}]`:e}:${l}`,host:e,port:l,app:a,close:s(async()=>{r.servers.delete(n),await a.close()},"close")};return r.servers.add(n),n}s(L,"start");export{L as start};
1
+ var g=Object.defineProperty;var i=(t,a)=>g(t,"name",{value:a,configurable:!0});import{context as b,self as u}from"../../lib/runtime.mjs";import{TranslatorError as d}from"../../lib/errors.mjs";import{isLoopback as L}from"../../lib/http.mjs";import{tlsOptions as $}from"../../lib/guard.mjs";async function k(t={}){const{state:a}=b,{plugin:h}=u.server,{build:f}=u.error;let c;try{({default:c}=await import("fastify"))}catch(o){throw new d('server.start needs the optional peer dependency "fastify" (npm install fastify)',{code:"fastify_missing",status:500,details:{cause:o}})}const s=await $(t.tls),e=t.host??"127.0.0.1";if(!L(e)&&[t.apiKey??[]].flat().filter(Boolean).length===0&&typeof t.authorize!="function"&&t.allowUnauthenticated!==!0)throw new d(`Refusing to listen on ${e} without an apiKey or an authorize hook: anyone who can reach the port could use the upstream. Set \`apiKey\` (CLI: --api-key) or \`authorize\`, bind to 127.0.0.1, or opt in with \`allowUnauthenticated: true\` (CLI: --allow-unauthenticated).`,{code:"unauthenticated_bind",status:500});const r=c({logger:t.logger??!1,forceCloseConnections:!0,...s?{https:s}:{}});r.setNotFoundHandler((o,m)=>{const{status:y,headers:p,body:w}=f(404,`Not found: ${o.method} ${o.url}`);return m.code(y).headers(p).send(w)}),await r.register(h,{front:t.front,apiKey:t.apiKey,bodyLimit:t.bodyLimit,authorize:t.authorize,onUsage:t.onUsage,metrics:t.metrics,cors:t.cors,rateLimit:t.rateLimit}),await r.listen({host:e,port:t.port??8787});const l=r.server.address().port,n={url:`${s?"https":"http"}://${e.includes(":")?`[${e}]`:e}:${l}`,host:e,port:l,app:r,close:i(async()=>{a.servers.delete(n),await r.close()},"close")};return a.servers.add(n),n}i(k,"start");export{k as start};
2
2
  //# sourceMappingURL=start.mjs.map
@@ -0,0 +1,2 @@
1
+ var h=Object.defineProperty;var a=(e,t)=>h(e,"name",{value:t,configurable:!0});import{TranslatorError as i}from"./errors.mjs";const f=["vllm"],_=["openai"];function d(e){return e==null||typeof e=="string"&&e!==""}a(d,"valid");function w(e,t,r,l,s=[]){if(!d(t.salt))throw new i("cacheSalt: expected a non-empty string",{code:"invalid_request",param:"cacheSalt"});if(!d(t.key))throw new i("metadata.prompt_cache_key: expected a non-empty string",{code:"invalid_request",param:"metadata.prompt_cache_key"});let c=e;const u=a(n=>{if(r.unknownFields==="error")throw new i(`${n}: is not accepted by this upstream`,{code:"unsupported_field",param:n});s.includes(n)||s.push(n)},"refuse"),o=t.salt??r.cacheSalt;o!=null&&(f.includes(l)?c={...c,cache_salt:o}:u("cache_salt"));const p=t.key??r.promptCacheKey;return p!=null&&!Object.hasOwn(e,"prompt_cache_key")&&(_.includes(l)?c={...c,prompt_cache_key:p}:u("prompt_cache_key")),c}a(w,"addCache");export{_ as KEY_DRIVERS,f as SALT_DRIVERS,w as addCache,d as valid};
2
+ //# sourceMappingURL=cache.mjs.map
package/dist/lib/cli.mjs CHANGED
@@ -1,4 +1,4 @@
1
- var _=Object.defineProperty;var c=(t,e)=>_(t,"name",{value:e,configurable:!0});import{readFile as d}from"node:fs/promises";import{parseArgs as g}from"node:util";const u=`llm-translator - run Claude Code (or any Anthropic/OpenAI client) against local LLM servers
1
+ var O=Object.defineProperty;var p=(t,e)=>O(t,"name",{value:e,configurable:!0});import{readFile as d}from"node:fs/promises";import{parseArgs as N}from"node:util";const m=`llm-translator - run Claude Code (or any Anthropic/OpenAI client) against local LLM servers
2
2
 
3
3
  Usage: llm-translator --upstream <url> [options]
4
4
 
@@ -12,6 +12,14 @@ Options:
12
12
  --port <port> Listen port (default 8787)
13
13
  --api-key <key> Key clients must send (x-api-key or Authorization: Bearer)
14
14
  --allow-unauthenticated Allow a non-loopback --host without --api-key
15
+ --metrics Serve Prometheus metrics at GET /metrics (same authentication as the other routes)
16
+ --tls-key <file> TLS private key (PEM); with --tls-cert the server speaks HTTPS
17
+ --tls-cert <file> TLS certificate (PEM)
18
+ --tls-ca <file> Extra CA certificate (PEM)
19
+ --tls-passphrase <text> Passphrase of the TLS key
20
+ --cors-origin <origin> Origin browsers may call from (repeatable, * for any); CORS is off without it
21
+ --rate-limit <n> Allow n requests per window for each API key (remote address without one); off without it
22
+ --rate-limit-window <ms> Rate limit window (default 60000)
15
23
  --config <file> JSON config file (translator config plus a "server" object)
16
24
  --logs [path] Log warnings and errors to the console (default), or append every event as a JSON line to <path>
17
25
  --silent Same as logs: false (no console output, no file)
@@ -20,11 +28,12 @@ Options:
20
28
  --help Print this help
21
29
 
22
30
  Environment: LLM_TRANSLATOR_FRONT, LLM_TRANSLATOR_UPSTREAM, LLM_TRANSLATOR_UPSTREAM_KEY, LLM_TRANSLATOR_BACKEND,
23
- LLM_TRANSLATOR_HOST, LLM_TRANSLATOR_PORT, LLM_TRANSLATOR_API_KEY, LLM_TRANSLATOR_ALLOW_UNAUTHENTICATED (1), LLM_TRANSLATOR_CONFIG. Flags win over the
24
- environment, which wins over the config file.
25
- `;function S(t){const e=t.map((o,l)=>o==="--logs"&&(l===t.length-1||t[l+1].startsWith("-"))?"--logs=":o);return g({args:e,allowPositionals:!1,options:{front:{type:"string"},upstream:{type:"string",multiple:!0},"upstream-key":{type:"string"},backend:{type:"string"},model:{type:"string",multiple:!0},host:{type:"string"},port:{type:"string"},"api-key":{type:"string"},"allow-unauthenticated":{type:"boolean"},config:{type:"string"},logs:{type:"string"},silent:{type:"boolean"},"no-detect":{type:"boolean"},version:{type:"boolean"},help:{type:"boolean"}}}).values}c(S,"parse");function h(t,e,o){const{server:l,...p}=o,s=t.upstream??(e.LLM_TRANSLATOR_UPSTREAM?e.LLM_TRANSLATOR_UPSTREAM.split(","):void 0),r={...p};if(s){const i=t["upstream-key"]??e.LLM_TRANSLATOR_UPSTREAM_KEY,a=t.backend??e.LLM_TRANSLATOR_BACKEND;r.upstreams=s.map(n=>({url:n,apiKey:i,...a?{backend:a}:{}}))}const T=t.front??e.LLM_TRANSLATOR_FRONT;if(T&&(r.front=T),t.model){r.aliases={...r.aliases};for(const i of t.model){const a=i.indexOf("=");if(a<1)throw new Error(`--model expects alias=target (got "${i}")`);r.aliases[i.slice(0,a)]=i.slice(a+1)}}t.silent&&(r.logs=!1),t.logs!==void 0&&(r.logs=t.logs===""?!0:t.logs),r.logs??=r.silent!==!0,delete r.silent,t["no-detect"]&&(r.detect=!1);const A=t.port??e.LLM_TRANSLATOR_PORT,L={...l,...t.host??e.LLM_TRANSLATOR_HOST?{host:t.host??e.LLM_TRANSLATOR_HOST}:{},...A?{port:Number(A)}:{},...t["api-key"]??e.LLM_TRANSLATOR_API_KEY?{apiKey:t["api-key"]??e.LLM_TRANSLATOR_API_KEY}:{},...t["allow-unauthenticated"]||e.LLM_TRANSLATOR_ALLOW_UNAUTHENTICATED==="1"?{allowUnauthenticated:!0}:{}};return{translator:r,server:L}}c(h,"merge");const y={out:c(t=>process.stdout.write(`${t}
26
- `),"out"),err:c(t=>process.stderr.write(`${t}
27
- `),"err"),onSignal:c(t=>{process.once("SIGINT",t),process.once("SIGTERM",t)},"onSignal")};async function w(t,e=process.env,o=y,l={}){let p;try{p=S(t)}catch(s){return o.err(`${s.message}
31
+ LLM_TRANSLATOR_HOST, LLM_TRANSLATOR_PORT, LLM_TRANSLATOR_API_KEY, LLM_TRANSLATOR_ALLOW_UNAUTHENTICATED (1), LLM_TRANSLATOR_METRICS (1), LLM_TRANSLATOR_CONFIG,
32
+ LLM_TRANSLATOR_TLS_KEY, LLM_TRANSLATOR_TLS_CERT, LLM_TRANSLATOR_TLS_CA, LLM_TRANSLATOR_TLS_PASSPHRASE, LLM_TRANSLATOR_CORS_ORIGINS (comma separated),
33
+ LLM_TRANSLATOR_RATE_LIMIT, LLM_TRANSLATOR_RATE_LIMIT_WINDOW. Flags win over the environment, which wins over the config file.
34
+ `;function g(t){const e=t.map((s,n)=>s==="--logs"&&(n===t.length-1||t[n+1].startsWith("-"))?"--logs=":s);return N({args:e,allowPositionals:!1,options:{front:{type:"string"},upstream:{type:"string",multiple:!0},"upstream-key":{type:"string"},backend:{type:"string"},model:{type:"string",multiple:!0},host:{type:"string"},port:{type:"string"},"api-key":{type:"string"},"allow-unauthenticated":{type:"boolean"},metrics:{type:"boolean"},"tls-key":{type:"string"},"tls-cert":{type:"string"},"tls-ca":{type:"string"},"tls-passphrase":{type:"string"},"cors-origin":{type:"string",multiple:!0},"rate-limit":{type:"string"},"rate-limit-window":{type:"string"},config:{type:"string"},logs:{type:"string"},silent:{type:"boolean"},"no-detect":{type:"boolean"},version:{type:"boolean"},help:{type:"boolean"}}}).values}p(g,"parse");function M(t,e,s){const{server:n,...T}=s,a=t.upstream??(e.LLM_TRANSLATOR_UPSTREAM?e.LLM_TRANSLATOR_UPSTREAM.split(","):void 0),r={...T};if(a){const i=t["upstream-key"]??e.LLM_TRANSLATOR_UPSTREAM_KEY,L=t.backend??e.LLM_TRANSLATOR_BACKEND;r.upstreams=a.map(u=>({url:u,apiKey:i,...L?{backend:L}:{}}))}const R=t.front??e.LLM_TRANSLATOR_FRONT;if(R&&(r.front=R),t.model){r.aliases={...r.aliases};for(const i of t.model){const L=i.indexOf("=");if(L<1)throw new Error(`--model expects alias=target (got "${i}")`);r.aliases[i.slice(0,L)]=i.slice(L+1)}}t.silent&&(r.logs=!1),t.logs!==void 0&&(r.logs=t.logs===""?!0:t.logs),r.logs??=r.silent!==!0,delete r.silent,t["no-detect"]&&(r.detect=!1);const S=t.port??e.LLM_TRANSLATOR_PORT,l={...n,...t.host??e.LLM_TRANSLATOR_HOST?{host:t.host??e.LLM_TRANSLATOR_HOST}:{},...S?{port:Number(S)}:{},...t["api-key"]??e.LLM_TRANSLATOR_API_KEY?{apiKey:t["api-key"]??e.LLM_TRANSLATOR_API_KEY}:{},...t["allow-unauthenticated"]||e.LLM_TRANSLATOR_ALLOW_UNAUTHENTICATED==="1"?{allowUnauthenticated:!0}:{},...t.metrics||e.LLM_TRANSLATOR_METRICS==="1"?{metrics:!0}:{}},_={...n?.tls,...f({key:t["tls-key"]??e.LLM_TRANSLATOR_TLS_KEY,cert:t["tls-cert"]??e.LLM_TRANSLATOR_TLS_CERT}),...f({ca:t["tls-ca"]??e.LLM_TRANSLATOR_TLS_CA,passphrase:t["tls-passphrase"]??e.LLM_TRANSLATOR_TLS_PASSPHRASE})};Object.keys(_).length>0&&(l.tls=_);const c=t["cors-origin"]??(e.LLM_TRANSLATOR_CORS_ORIGINS?e.LLM_TRANSLATOR_CORS_ORIGINS.split(","):void 0);c&&(l.cors=c.map(i=>i.trim()).filter(Boolean));const o=t["rate-limit"]??e.LLM_TRANSLATOR_RATE_LIMIT,A=t["rate-limit-window"]??e.LLM_TRANSLATOR_RATE_LIMIT_WINDOW;return(o||A)&&(l.rateLimit={...n?.rateLimit,...o?{requests:Number(o)}:{},...A?{windowMs:Number(A)}:{}}),{translator:r,server:l}}p(M,"merge");function f(t){return Object.fromEntries(Object.entries(t).filter(([,e])=>e))}p(f,"pick");const h={out:p(t=>process.stdout.write(`${t}
35
+ `),"out"),err:p(t=>process.stderr.write(`${t}
36
+ `),"err"),onSignal:p(t=>{process.once("SIGINT",t),process.once("SIGTERM",t)},"onSignal")};async function I(t,e=process.env,s=h,n={}){let T;try{T=g(t)}catch(a){return s.err(`${a.message}
28
37
 
29
- ${u}`),2}if(p.help)return o.out(u),0;if(p.version)return o.out(l.version??(await import("../../package.json",{with:{type:"json"}})).default.version),0;try{const s=p.config??e.LLM_TRANSLATOR_CONFIG,{translator:r,server:T}=h(p,e,s?JSON.parse(await d(s,"utf8")):{}),L=await(l.createTranslator??(await import("../index.mjs")).createTranslator)(r),i=await L.server.start(T),a=L.status();o.out(`llm-translator (${a.front} front) listening on ${i.url}`);for(const n of a.upstreams??[]){const f=n.version??"unknown",m=n.probed?.inlineSystem??"auto",R=n.probed?.images??"default";o.out(` upstream ${n.name} ${n.url}: driver=${n.driver} version=${f} inlineSystem=${m} images=${R}`)}return await new Promise(n=>o.onSignal(()=>n())),await L.server.stop(),await L.slothlet.shutdown(),0}catch(s){return o.err(`llm-translator: ${s.message}`),1}}c(w,"main");export{w as main,h as merge,S as parse,y as processIo};
38
+ ${m}`),2}if(T.help)return s.out(m),0;if(T.version)return s.out(n.version??(await import("../../package.json",{with:{type:"json"}})).default.version),0;try{const a=T.config??e.LLM_TRANSLATOR_CONFIG,{translator:r,server:R}=M(T,e,a?JSON.parse(await d(a,"utf8")):{}),l=await(n.createTranslator??(await import("../index.mjs")).createTranslator)(r),_=await l.server.start(R),c=l.status();s.out(`llm-translator (${c.front} front) listening on ${_.url}`);for(const o of c.upstreams??[]){const A=o.version??"unknown",i=o.probed?.inlineSystem??"auto",L=o.probed?.images??"default";s.out(` upstream ${o.name} ${o.url}: driver=${o.driver} version=${A} inlineSystem=${i} images=${L}`)}return await new Promise(o=>s.onSignal(()=>o())),await l.server.stop(),await l.slothlet.shutdown(),0}catch(a){return s.err(`llm-translator: ${a.message}`),1}}p(I,"main");export{I as main,M as merge,g as parse,h as processIo};
30
39
  //# sourceMappingURL=cli.mjs.map
@@ -0,0 +1,2 @@
1
+ var g=Object.defineProperty;var i=(r,o)=>g(r,"name",{value:o,configurable:!0});import{atLeast as h}from"./normalize.mjs";import{TranslatorError as l}from"./errors.mjs";const u=["json","regex","choice","grammar"],j="0.11.0",x=i(r=>r.replace(/[.*+?^${}()|[\]\\]/g,"\\$&"),"escape");function y(r){if(r==null)return null;if(typeof r!="object"||Array.isArray(r))return`expected an object with one of ${u.join(", ")}`;const o=Object.keys(r).filter(t=>r[t]!==void 0&&r[t]!==null);if(o.length!==1||!u.includes(o[0]))return`expected exactly one of ${u.join(", ")}`;const[n]=o,e=r[n];return n==="json"&&(typeof e!="object"||Array.isArray(e))?"json must be a JSON Schema object":(n==="regex"||n==="grammar")&&(typeof e!="string"||e==="")?`${n} must be a non-empty string`:n==="choice"&&!(Array.isArray(e)&&e.length>0&&e.every(t=>typeof t=="string"&&t!==""))?"choice must be a non-empty list of non-empty strings":null}i(y,"check");function _(r,o,n,e){const t=i(()=>`(?:${e.map(x).join("|")})`,"alternation"),f={type:"json_schema",json_schema:{name:"constraint",schema:e,strict:!0}};switch(r){case"vllm":case"compatible":return h(o,j)?{structured_outputs:{[n]:e}}:{[`guided_${n}`]:e};case"sglang":return n==="json"?{response_format:f}:n==="regex"?{regex:e}:n==="choice"?{regex:t()}:{ebnf:e};case"llamacpp":return n==="json"?{json_schema:e}:n==="grammar"?{grammar:e}:n==="choice"?{grammar:`root ::= ${e.map(s=>JSON.stringify(s)).join(" | ")}`}:null;case"tgi":return n==="json"?{response_format:{type:"json",value:e}}:n==="regex"?{response_format:{type:"regex",value:e}}:n==="choice"?{response_format:{type:"regex",value:t()}}:null;default:return n==="json"?{response_format:f}:null}}i(_,"fields");function A(r,o,n,e,t=[]){const f=y(o);if(f!==null)throw new l(`constraint: ${f}`,{code:"invalid_request",param:"constraint"});const s=o??n.constraint;if(s==null)return r;const a=u.find(c=>s[c]!==void 0&&s[c]!==null),p=i(c=>{t.includes(c)||t.push(c)},"note"),m=_(e.name,e.version,a,s[a]);if(m===null){if(n.unknownFields==="error")throw new l(`constraint.${a}: is not accepted by this upstream`,{code:"unsupported_field",param:`constraint.${a}`});return p(`constraint.${a}`),r}return["vllm","compatible"].includes(e.name)&&Array.isArray(r.tools)&&r.tools.length>0&&r.tool_choice!=="none"?(p(`constraint.${a}`),r):Object.keys(m).some(c=>Object.hasOwn(r,c))?r:{...r,...m}}i(A,"addConstraint");export{u as KINDS,j as STRUCTURED_OUTPUTS_VERSION,A as addConstraint,y as check,_ as fields};
2
+ //# sourceMappingURL=constraint.mjs.map
@@ -0,0 +1,2 @@
1
+ var m=Object.defineProperty;var u=(e,o)=>m(e,"name",{value:o,configurable:!0});import{estimate as x,tokens as d}from"./estimate.mjs";import{restoreName as y}from"./ids.mjs";const v=1e5,k=3,a="[Tool result cleared to save context]",c=u((e,o)=>e!==null&&typeof e=="object"&&o.includes(e.type)&&Number.isInteger(e.value)&&e.value>=0,"threshold");function N(e){return e.trigger!==void 0&&!c(e.trigger,["input_tokens","tool_uses"])||e.keep!==void 0&&!c(e.keep,["tool_uses"])||e.clear_at_least!==void 0&&!c(e.clear_at_least,["input_tokens"])||e.exclude_tools!==void 0&&!(Array.isArray(e.exclude_tools)&&e.exclude_tools.every(o=>typeof o=="string"))?!1:e.clear_tool_inputs===void 0||typeof e.clear_tool_inputs=="boolean"}u(N,"valid");function D(e,o,p){const l=[];for(const t of o.messages)if(t.role==="assistant")for(const n of t.tool_calls??[])l.push({call:n,result:o.messages.find(r=>r.role==="tool"&&r.tool_call_id===n.id)});const f=e.trigger??{type:"input_tokens",value:v};if((f.type==="tool_uses"?l.length:x(o))<=f.value)return null;const g=e.keep?.value??k,h=new Set(e.exclude_tools??[]),s=l.slice(0,Math.max(0,l.length-g)).filter(t=>t.result!==void 0&&t.result.content!==a&&!h.has(y(p,t.call.function.name))),i=[];for(const t of s){const n=JSON.stringify([t.result.content,e.clear_tool_inputs===!0?t.call.function.arguments:""]).length,r=JSON.stringify([a,e.clear_tool_inputs===!0?"{}":""]).length;i.push(Math.max(0,n-r))}const _=d(i.reduce((t,n)=>t+n,0));if(s.length===0||_<(e.clear_at_least?.value??0))return null;for(const t of s)t.result.content=a,e.clear_tool_inputs===!0&&(t.call.function.arguments="{}");return{type:e.type,cleared_tool_uses:s.length,cleared_input_tokens:_}}u(D,"clearToolUses");export{k as DEFAULT_KEEP,v as DEFAULT_TRIGGER,a as PLACEHOLDER,D as clearToolUses,N as valid};
2
+ //# sourceMappingURL=context.mjs.map
@@ -0,0 +1,2 @@
1
+ var y=Object.defineProperty;var c=(t,s)=>y(t,"name",{value:s,configurable:!0});import{readFile as h}from"node:fs/promises";import{TranslatorError as m}from"./errors.mjs";const u=c(t=>new m(t,{code:"invalid_config",status:500}),"invalid");async function x(t){if(t==null)return null;if(typeof t!="object"||Array.isArray(t))throw u("tls must be an object with key and cert file paths");const{key:s,cert:i,ca:n,passphrase:a}=t;if(!s&&!i&&!n&&!a)return null;if(!s||!i)throw u(`tls needs both key and cert (got ${s?"key":"cert"} only)`);const e=c(async(r,o)=>{if(typeof o!="string")throw u(`tls.${r} must be a file path`);try{return await h(o)}catch(f){throw u(`tls.${r}: cannot read ${o} (${f.code??f})`)}},"read"),l={key:await e("key",s),cert:await e("cert",i)};if(n&&(l.ca=await Promise.all([n].flat().map(r=>e("ca",r)))),a!==void 0){if(typeof a!="string")throw u("tls.passphrase must be a string");l.passphrase=a}return l}c(x,"tlsOptions");const g="authorization, x-api-key, content-type, anthropic-version, anthropic-beta",d=c(t=>t.toLowerCase().replace(/\/+$/,""),"comparable");function $(t){if(t==null||t===!1)return null;const s=[Array.isArray(t)||typeof t=="string"?t:t.origins??[]].flat();if(s.some(e=>typeof e!="string"||e===""))throw u("cors origins must be non-empty strings");if(s.length===0)return null;const i=s.includes("*"),n=new Set(s.map(d)),a=c(e=>e?!i&&!n.has(d(e))?{vary:"origin"}:{"access-control-allow-origin":i?"*":e,"access-control-expose-headers":"retry-after, request-id, x-request-id",vary:"origin"}:{},"headers");return{headers:a,preflight:c((e,l)=>{const r=a(e);return r["access-control-allow-origin"]?{...r,"access-control-allow-methods":"GET, HEAD, POST, OPTIONS","access-control-allow-headers":l||g,"access-control-max-age":"600"}:r},"preflight")}}c($,"cors");function v(t,s=Date.now){if(t==null||t===!1)return null;const i=t.requests,n=t.windowMs??6e4;if(!Number.isInteger(i)||i<1)throw u(`rateLimit.requests must be a positive integer (got ${JSON.stringify(i)})`);if(!Number.isInteger(n)||n<1)throw u(`rateLimit.windowMs must be a positive integer (got ${JSON.stringify(n)})`);const a=new Map;let e=s();return{take:c(l=>{const r=s();if(r-e>=n){for(const[f,w]of a)r-w.start>=n&&a.delete(f);e=r}let o=a.get(l);return(!o||r-o.start>=n)&&(o={start:r,count:0},a.set(l,o)),o.count>=i?Math.max(1,Math.ceil((o.start+n-r)/1e3)):(o.count++,0)},"take"),size:c(()=>a.size,"size")}}c(v,"rateLimit");export{$ as cors,v as rateLimit,x as tlsOptions};
2
+ //# sourceMappingURL=guard.mjs.map
@@ -0,0 +1,4 @@
1
+ var x=Object.defineProperty;var c=(t,r)=>x(t,"name",{value:r,configurable:!0});import{describe as I}from"./pool.mjs";const A="text/plain; version=0.0.4; charset=utf-8",N=[.005,.01,.025,.05,.1,.25,.5,1,2.5,5,10,30,60,120,300],R=1e4,m="llm_translator_",T=`${m}requests_total`,w=`${m}request_duration_seconds`,v=`${m}tokens_total`,S=`${m}errors_total`,U=`${m}upstream_up`;function j(t){return String(t).replace(/\\/g,"\\\\").replace(/"/g,'\\"').replace(/\n/g,"\\n")}c(j,"escapeLabel");function P(t){return t.replace(/\\/g,"\\\\").replace(/\n/g,"\\n")}c(P,"escapeHelp");function $(t){return t===1/0?"+Inf":t===-1/0?"-Inf":String(t)}c($,"format");function f(t){const r=Object.entries(t).map(([i,n])=>`${i}="${j(n)}"`);return r.length===0?"":`{${r.join(",")}}`}c(f,"braces");function F(){const t=new Map,r=c(i=>JSON.stringify(Object.entries(i)),"key");return{counter(i,n){t.set(i,{type:"counter",help:n,series:new Map})},histogram(i,n,e=N){t.set(i,{type:"histogram",help:n,buckets:e,series:new Map})},inc(i,n,e=1){const{series:o}=t.get(i),s=r(n),a=o.get(s)??{labels:n,value:0};a.value+=e,o.set(s,a)},observe(i,n,e){const{series:o,buckets:s}=t.get(i),a=r(n),l=o.get(a)??{labels:n,counts:s.map(()=>0),sum:0,count:0};s.forEach((h,u)=>{e<=h&&l.counts[u]++}),l.sum+=e,l.count++,o.set(a,l)},render(i=[]){const n=[];for(const[e,o]of t){n.push(`# HELP ${e} ${P(o.help)}`,`# TYPE ${e} ${o.type}`);for(const s of o.series.values()){if(o.type==="counter"){n.push(`${e}${f(s.labels)} ${$(s.value)}`);continue}o.buckets.forEach((a,l)=>{n.push(`${e}_bucket${f({...s.labels,le:$(a)})} ${s.counts[l]}`)}),n.push(`${e}_bucket${f({...s.labels,le:"+Inf"})} ${s.count}`),n.push(`${e}_sum${f(s.labels)} ${$(s.sum)}`),n.push(`${e}_count${f(s.labels)} ${s.count}`)}}for(const e of i){n.push(`# HELP ${e.name} ${P(e.help)}`,`# TYPE ${e.name} gauge`);for(const o of e.samples)n.push(`${e.name}${f(o.labels)} ${$(o.value)}`)}return`${n.join(`
2
+ `)}
3
+ `}}}c(F,"create");function H(t,r){return t===499||r?.reason==="aborted"?null:r?.reason==="upstream_error"?"upstream":r?.reason==="error"?t>=400&&t<500?"client":"translator":t>=500?"translator":t>=400?"client":null}c(H,"kind");function C(t){const r=F();r.counter(T,"Requests handled, by front protocol, route, upstream driver and HTTP status code (499 when the client left before the reply finished)."),r.histogram(w,"Seconds from the request arriving to the end of the response, streams included."),r.counter(v,"Tokens reported by upstreams, by driver and type (input, output, cache_read, cache_creation)."),r.counter(S,"Failed requests by kind: upstream, translator or client.");const i=new WeakMap,n=c(async(e,o,s)=>{const a=o.raw,l=!a.writableFinished,h=i.get(e);let u=null;if(h){let p;const d=new Promise(g=>{p=setTimeout(g,R,null),p.unref()});u=await Promise.race([h.then(g=>g.usage,()=>null),d]),clearTimeout(p)}const O=Number(process.hrtime.bigint()-s)/1e9,b=l?499:a.statusCode,_=u?.upstream?t.endpoints().find(p=>p.name===u.upstream):void 0,y=_?String(_.driver??"unknown"):"none",k={front:t.front,route:e.routeOptions?.url??"unmatched",driver:y,code:b};r.inc(T,k),r.observe(w,k,O);const E=H(b,u);if(E&&r.inc(S,{kind:E}),u?.reported)for(const[p,d]of[["input","input_tokens"],["output","output_tokens"],["cache_read","cache_read_input_tokens"],["cache_creation","cache_creation_input_tokens"]])Number.isFinite(u[d])&&u[d]>0&&r.inc(v,{driver:y,type:p},u[d])},"finish");return{hook:c(async(e,o)=>{if(e.routeOptions?.config?.metrics)return;const s=process.hrtime.bigint();o.raw.once("close",()=>{n(e,o,s).catch(()=>{})})},"hook"),attach:c((e,o)=>{i.set(e,o)},"attach"),render:c(()=>r.render([{name:U,help:"1 while an upstream is healthy (not backing off after a failure and not reported not ready by its readiness probe), else 0.",samples:t.endpoints().map(e=>({labels:{upstream:e.name,driver:String(e.driver??"unknown")},value:I(e).healthy?1:0}))}]),"render")}}c(C,"recorder");export{N as BUCKETS,A as CONTENT_TYPE,F as create,P as escapeHelp,j as escapeLabel,H as kind,C as recorder};
4
+ //# sourceMappingURL=metrics.mjs.map
@@ -1,2 +1,2 @@
1
- var i=Object.defineProperty;var a=(t,o)=>i(t,"name",{value:o,configurable:!0});const p=["model","messages","stream","stream_options","max_tokens","max_completion_tokens","temperature","top_p","n","stop","presence_penalty","frequency_penalty","logit_bias","logprobs","top_logprobs","seed","response_format","tools","tool_choice","parallel_tool_calls","reasoning_effort","user","metadata","store","service_tier"],m=["top_k","min_p","repetition_penalty","length_penalty","min_tokens","stop_token_ids","include_stop_str_in_output","ignore_eos","skip_special_tokens","spaces_between_special_tokens","allowed_token_ids","bad_words","prompt_logprobs","truncate_prompt_tokens","add_generation_prompt","continue_final_message","add_special_tokens","chat_template_kwargs","thinking_token_budget","include_reasoning","structured_outputs"],l=new Set(p),c=new Set(["include_usage","continuous_usage_stats"]);function d(t,o=[]){const s={};for(const[e,n]of Object.entries(t))if(!(!l.has(e)&&!o.includes(e)))if(e==="stream_options"){const _=Object.fromEntries(Object.entries(n??{}).filter(([r])=>c.has(r)));Object.keys(_).length>0&&(s.stream_options=_)}else s[e]=n;return s}a(d,"filterChatBody");export{m as VLLM_FIELDS,d as filterChatBody};
1
+ var a=Object.defineProperty;var i=(t,o)=>a(t,"name",{value:o,configurable:!0});const p=["model","messages","stream","stream_options","max_tokens","max_completion_tokens","temperature","top_p","n","stop","presence_penalty","frequency_penalty","logit_bias","logprobs","top_logprobs","seed","response_format","tools","tool_choice","parallel_tool_calls","reasoning_effort","user","metadata","store","service_tier"],d=["top_k","min_p","repetition_penalty","length_penalty","min_tokens","stop_token_ids","include_stop_str_in_output","ignore_eos","skip_special_tokens","spaces_between_special_tokens","allowed_token_ids","bad_words","prompt_logprobs","truncate_prompt_tokens","add_generation_prompt","continue_final_message","add_special_tokens","chat_template_kwargs","thinking_token_budget","include_reasoning","structured_outputs","guided_json","guided_regex","guided_choice","guided_grammar"],c=new Set(p),l=new Set(["include_usage","continuous_usage_stats"]);function g(t,o=[]){const s={};for(const[e,n]of Object.entries(t))if(!(!c.has(e)&&!o.includes(e)))if(e==="stream_options"){const _=Object.fromEntries(Object.entries(n??{}).filter(([r])=>l.has(r)));Object.keys(_).length>0&&(s.stream_options=_)}else s[e]=n;return s}i(g,"filterChatBody");export{d as VLLM_FIELDS,g as filterChatBody};
2
2
  //# sourceMappingURL=passthrough.mjs.map
@@ -1,2 +1,2 @@
1
- var f=Object.defineProperty;var s=(e,t)=>f(e,"name",{value:t,configurable:!0});import{noEmptyTools as d}from"./normalize.mjs";const p={chat:"/v1/chat/completions",embeddings:"/v1/embeddings",models:"/v1/models"},c={chat:"/chat/completions",embeddings:"/embeddings",models:"/models"};function m(e){return/^grok-(3-mini|4|code)/.test(String(e))&&!/non-reasoning/.test(String(e))}s(m,"grokReasons");const l=Object.freeze({groq:{order:21,hosts:["api.groq.com"],root:"/openai",paths:p,settings:{reasoningField:"reasoning",maxTokensField:"max_completion_tokens",thinking:"none",thinkingBudget:!1,effort:!0,topK:!1,prefill:"none",streamUsage:!0,continuousUsage:!1},drop:["logprobs","top_logprobs","logit_bias"],dropName:!0},together:{order:22,hosts:["api.together.ai","api.together.xyz","us-api.together.ai"],paths:p,fields:["top_k","min_p","repetition_penalty"],settings:{reasoningField:"reasoning",reasoningRoundTrip:!0,maxTokensField:"max_tokens",thinking:"none",thinkingBudget:!1,effort:!0,topK:!0,prefill:"none",parallelToolCalls:!1,streamUsage:!1,continuousUsage:!1}},fireworks:{order:23,hosts:["api.fireworks.ai"],root:"/inference",paths:p,fields:["top_k","min_p","repetition_penalty"],settings:{reasoningField:"reasoning_content",reasoningRoundTrip:!0,maxTokensField:"max_tokens",thinking:"reasoning_effort",thinkingBudget:!1,effort:!0,topK:!0,prefill:"none",streamUsage:!0,continuousUsage:!1}},xai:{order:24,hosts:["api.x.ai"],paths:p,settings:{reasoningField:"reasoning_content",maxTokensField:"max_completion_tokens",thinking:"none",thinkingBudget:!1,effort:!0,topK:!1,prefill:"none",streamUsage:!0,continuousUsage:!1},dropWhen:s(e=>m(e.model)?["stop","presence_penalty","frequency_penalty"]:[],"dropWhen")},gemini:{order:25,hosts:["generativelanguage.googleapis.com"],root:"/v1beta/openai",paths:c,settings:{maxTokensField:"max_tokens",thinking:"none",thinkingBudget:!1,effort:!0,topK:!1,prefill:"none",streamUsage:!0,continuousUsage:!1}}});function _(e){const t=l[e],o={name:e,protocol:"openai.chat",order:t.order,auth:"bearer",strict:!1,paths:{...t.paths}};return t.fields?{...o,fields:[...t.fields]}:o}s(_,"describe");function x(e,t){const o=URL.canParse(t.url)?new URL(t.url).hostname:"";return l[e].hosts.includes(o)?{host:o}:null}s(x,"match");function U(e){return{...l[e].settings}}s(U,"settingsOf");function b(e,t,o){const i=l[e],a=i.root??"";return(o.url.endsWith(a)?"":a)+i.paths[t]}s(b,"route");function y(e,t,o){const i=l[e],a=o?.state?.dropped,g=s(n=>{Array.isArray(a)&&!a.includes(n)&&a.push(n)},"note"),r={...d(t)};for(const n of[...i.drop??[],...i.dropWhen?.(t)??[]])r[n]!==void 0&&(delete r[n],g(n));return i.dropName&&Array.isArray(r.messages)&&r.messages.some(n=>n&&"name"in n)&&(r.messages=r.messages.map(n=>{const u={...n};return delete u.name,u}),g("messages[].name")),r}s(y,"trim");export{l as PRESETS,_ as describe,x as match,b as route,U as settingsOf,y as trim};
1
+ var f=Object.defineProperty;var s=(e,t)=>f(e,"name",{value:t,configurable:!0});import{noEmptyTools as d}from"./normalize.mjs";const p={chat:"/v1/chat/completions",embeddings:"/v1/embeddings",models:"/v1/models"},c={chat:"/chat/completions",embeddings:"/embeddings",models:"/models"};function m(e){return/^grok-(3-mini|4|code)/.test(String(e))&&!/non-reasoning/.test(String(e))}s(m,"grokReasons");const l=Object.freeze({groq:{order:21,hosts:["api.groq.com"],root:"/openai",paths:p,settings:{reasoningField:"reasoning",maxTokensField:"max_completion_tokens",thinking:"none",thinkingBudget:!1,effort:!0,topK:!1,prefill:"none",streamUsage:!0,continuousUsage:!1},drop:["logprobs","top_logprobs","logit_bias"],dropName:!0},together:{order:22,hosts:["api.together.ai","api.together.xyz","us-api.together.ai"],paths:p,fields:["top_k","min_p","repetition_penalty"],settings:{reasoningField:"reasoning",reasoningRoundTrip:!0,maxTokensField:"max_tokens",thinking:"none",thinkingBudget:!1,effort:!0,topK:!0,prefill:"none",parallelToolCalls:!1,streamUsage:!1,continuousUsage:!1}},fireworks:{order:23,hosts:["api.fireworks.ai"],root:"/inference",paths:p,fields:["top_k","min_p","repetition_penalty"],settings:{reasoningField:"reasoning_content",reasoningRoundTrip:!0,maxTokensField:"max_tokens",thinking:"reasoning_effort",thinkingBudget:!1,effort:!0,topK:!0,prefill:"none",streamUsage:!0,continuousUsage:!1}},xai:{order:24,hosts:["api.x.ai"],paths:p,settings:{reasoningField:"reasoning_content",maxTokensField:"max_completion_tokens",thinking:"none",thinkingBudget:!1,effort:!0,topK:!1,prefill:"none",streamUsage:!0,continuousUsage:!1},dropWhen:s(e=>m(e.model)?["stop","presence_penalty","frequency_penalty"]:[],"dropWhen")},gemini:{order:25,hosts:["generativelanguage.googleapis.com"],root:"/v1beta/openai",paths:c,settings:{maxTokensField:"max_tokens",thinking:"none",thinkingBudget:!1,effort:!0,topK:!1,prefill:"none",streamUsage:!0,continuousUsage:!1}}});function _(e){const t=l[e],o={name:e,protocol:"openai.chat",order:t.order,auth:"bearer",strict:!1,hosts:[...t.hosts],paths:{...t.paths}};return t.fields?{...o,fields:[...t.fields]}:o}s(_,"describe");function x(e,t){const o=URL.canParse(t.url)?new URL(t.url).hostname:"";return l[e].hosts.includes(o)?{host:o}:null}s(x,"match");function U(e){return{...l[e].settings}}s(U,"settingsOf");function b(e,t,o){const i=l[e],a=i.root??"";return(o.url.endsWith(a)?"":a)+i.paths[t]}s(b,"route");function y(e,t,o){const i=l[e],a=o?.state?.dropped,g=s(n=>{Array.isArray(a)&&!a.includes(n)&&a.push(n)},"note"),r={...d(t)};for(const n of[...i.drop??[],...i.dropWhen?.(t)??[]])r[n]!==void 0&&(delete r[n],g(n));return i.dropName&&Array.isArray(r.messages)&&r.messages.some(n=>n&&"name"in n)&&(r.messages=r.messages.map(n=>{const u={...n};return delete u.name,u}),g("messages[].name")),r}s(y,"trim");export{l as PRESETS,_ as describe,x as match,b as route,U as settingsOf,y as trim};
2
2
  //# sourceMappingURL=presets.mjs.map
@@ -1,2 +1,2 @@
1
- var S=Object.defineProperty;var c=(e,o)=>S(e,"name",{value:o,configurable:!0});import{timingSafeEqual as H}from"node:crypto";import{TranslatorError as w}from"./errors.mjs";import{messageId as O}from"./ids.mjs";import{sign as T}from"./thinking.mjs";const j=64*1024*1024,C={"content-type":"text/event-stream; charset=utf-8","cache-control":"no-cache","x-accel-buffering":"no"},p=Symbol("stalled"),v=Symbol("gone");async function D(e,o){if(!(o>0))return e;let a;const l=new Promise(g=>{a=setTimeout(g,o,p)});try{return await Promise.race([e,l])}finally{clearTimeout(a)}}c(D,"settle");async function*I(e,o,a){let l=!1;try{const g=o.front==="anthropic";g&&(yield{event:"message_start",data:{type:"message_start",message:{id:O(),type:"message",role:"assistant",model:a,content:[],stop_reason:null,stop_sequence:null,usage:{input_tokens:0,output_tokens:0,cache_creation_input_tokens:0,cache_read_input_tokens:0}}}},yield{event:"content_block_start",data:{type:"content_block_start",index:0,content_block:{type:"thinking",thinking:"",signature:""}}});const s=await e,b=c(function*(){yield{event:"content_block_delta",data:{type:"content_block_delta",index:0,delta:{type:"signature_delta",signature:T()}}},yield{event:"content_block_stop",data:{type:"content_block_stop",index:0}}},"closeEarly");if(s.stream){l=!0;let u=null;for await(const i of s.stream){if(!g){yield i;continue}if(i.event!=="message_start"){if(u===null&&i.event==="content_block_start"&&i.data.content_block.type==="thinking"){u=0;continue}u===null&&(yield*b(),u=1),yield u>0&&typeof i.data?.index=="number"?{...i,data:{...i.data,index:i.data.index+u}}:i}}return}g&&(yield*b()),yield*x(o,s.body)}finally{l||e.then(g=>g.stream?.return?.(),()=>{})}}c(I,"late");function*x(e,o){if(e.front==="anthropic"){yield{event:"error",data:o};return}yield{data:o},yield{data:"[DONE]"}}c(x,"failure");async function E(e,o,a,l,g,s=200,b=C){e.hijack();const u=e.raw;u.writeHead(s,b);try{await o.write(u,l,{front:o.front,interval:g})}catch(i){if(!a.signal.aborted){const f=i instanceof w;f||(e.log.error(i),o.emit({type:"error",kind:"route",level:"error",message:String(i?.message??i),log:!1}));const{body:_}=o.build(f?i.status:502,f?i.message:"Internal server error",f?{type:i.type,code:i.code}:{});for(const t of x(o,_))u.write(o.encode(t))}}return u.end(),e}c(E,"stream");function L(e,o){const a=Buffer.from(e),l=Buffer.from(o);return a.length===l.length&&H(a,l)}c(L,"same");function U(e){const o=/^Bearer\s+(.+)$/i.exec(e.authorization??"");return e["x-api-key"]||o?.[1]||""}c(U,"keyOf");function z(e){const o={};for(const[a,l]of Object.entries(e))(a==="anthropic-beta"||a==="anthropic-version"||a.startsWith("x-claude-code-"))&&(o[a]=String(l));return o}c(z,"forwardable");function B(e){if(e===!0)return{ok:!0,context:void 0};if(e&&typeof e=="object"&&e.ok===!0)return{ok:!0,context:e.context};const o=e&&typeof e=="object"&&e.ok===!1?e:{};return{ok:!1,status:Number.isInteger(o.status)&&o.status>=400&&o.status<=599?o.status:401,message:typeof o.message=="string"&&o.message!==""?o.message:"unauthorized"}}c(B,"verdictOf");function R(e,o,a){const{front:l,filter:g}=a,s={...a,build:c((t,n,r={})=>a.build(t,n,{...r,front:l}),"build"),send:c((t,n)=>a.send(t,{...n,front:l,filter:g}),"send"),route:c(t=>a.route(t,{front:l}),"route"),count:c((t,n)=>a.count(t,{...n,front:l}),"count"),embed:c((t,n)=>a.embed(t,n),"embed"),list:c(t=>a.list({...t,front:l}),"list"),get:c((t,n)=>a.get(t,{...n,front:l}),"get")},b=[o.apiKey??[]].flat().filter(Boolean),u=o.bodyLimit??j,i=new WeakMap,f=c((t,n)=>t.code(n.status).headers(n.headers).send(n.body),"reject");e.setErrorHandler((t,n,r)=>{if(t instanceof w)return f(r,s.build(t.status,t.message,{type:t.type,code:t.code,param:t.param}));const d=t.statusCode??500;return d<500?f(r,s.build(d,t.message)):(n.log.error(t),s.emit({type:"error",kind:"route",level:"error",message:String(t.message),status:d,log:!1}),f(r,s.build(d,"Internal server error")))}),e.addHook("onRequest",async(t,n)=>{if(b.length===0||t.routeOptions.config?.public)return;const r=U(t.headers);if(!(r&&b.some(d=>L(r,d))))return f(n,s.build(401,"invalid x-api-key"))}),typeof o.authorize=="function"&&e.addHook("preHandler",async(t,n)=>{if(t.routeOptions.config?.public)return;let r=null;if(t.body&&typeof t.body=="object"&&!Array.isArray(t.body))try{r=s.route(t.body)}catch{}let d;try{d=B(await o.authorize({method:t.method,url:t.url,headers:t.headers,body:t.body,front:s.front,route:r}))}catch(m){t.log.error(m),s.emit({type:"error",kind:"route",level:"error",message:String(m?.message??m),log:!1}),d={ok:!1,status:401,message:"unauthorized"}}if(!d.ok)return f(n,s.build(d.status,d.message));i.set(t,d.context)}),e.route({method:["GET","HEAD"],url:"/",config:{public:!0},handler:c(async(t,n)=>n.send({ok:!0}),"handler")}),e.route({method:["GET","HEAD"],url:"/api/hello",config:{public:!0},handler:c(async(t,n)=>n.send({ok:!0}),"handler")}),e.get("/v1/models",async(t,n)=>{const r=new AbortController;return n.raw.on("close",()=>r.abort()),s.list({signal:r.signal})}),e.get("/v1/models/*",async(t,n)=>{const r=new AbortController;return n.raw.on("close",()=>r.abort()),s.get(t.params["*"],{signal:r.signal})});const _=s.front==="anthropic"?"/v1/messages":"/v1/chat/completions";e.post(_,{bodyLimit:u},async(t,n)=>{if(n.raw.destroyed)return n;const r=new AbortController;n.raw.on("close",()=>{n.raw.writableFinished||r.abort()});const d=typeof o.onUsage=="function"?h=>o.onUsage(h,i.get(t)):void 0,m=s.interval(t.body),k=s.send(t.body,{signal:r.signal,clientHeaders:z(t.headers),onUsage:d,log:t.log});k.catch(()=>{});const A=new Promise(h=>r.signal.addEventListener("abort",()=>h(v),{once:!0})),y=await D(Promise.race([k,A]),t.body?.stream===!0?m:0);return y===v?(k.then(h=>h.stream?.return?.(),()=>{}),n):y===p?E(n,s,r,I(k,s,t.body?.model),m):y.stream?E(n,s,r,y.stream,m,y.status,y.headers):f(n,y)}),s.front==="openai"&&e.post("/v1/embeddings",{bodyLimit:u},async(t,n)=>{const r=new AbortController;n.raw.on("close",()=>{n.raw.writableFinished||r.abort()});const d=typeof o.onUsage=="function"?m=>o.onUsage(m,i.get(t)):void 0;return f(n,await s.embed(t.body,{signal:r.signal,onUsage:d,log:t.log}))}),s.front==="anthropic"&&e.post("/v1/messages/count_tokens",{bodyLimit:u},async(t,n)=>{const r=new AbortController;return n.raw.on("close",()=>r.abort()),s.count(t.body,{signal:r.signal})})}c(R,"register");export{I as late,R as register};
1
+ var C=Object.defineProperty;var l=(n,o)=>C(n,"name",{value:o,configurable:!0});import{timingSafeEqual as I}from"node:crypto";import{TranslatorError as x}from"./errors.mjs";import{messageId as P}from"./ids.mjs";import{sign as D}from"./thinking.mjs";import{CONTENT_TYPE as N,recorder as U}from"./metrics.mjs";import{cors as z,rateLimit as B}from"./guard.mjs";const R=64*1024*1024,G={"content-type":"text/event-stream; charset=utf-8","cache-control":"no-cache","x-accel-buffering":"no"},S=Symbol("stalled"),E=Symbol("gone");async function M(n,o){if(!(o>0))return n;let i;const d=new Promise(g=>{i=setTimeout(g,o,S)});try{return await Promise.race([n,d])}finally{clearTimeout(i)}}l(M,"settle");async function*W(n,o,i){let d=!1;try{const g=o.front==="anthropic";g&&(yield{event:"message_start",data:{type:"message_start",message:{id:P(),type:"message",role:"assistant",model:i,content:[],stop_reason:null,stop_sequence:null,usage:{input_tokens:0,output_tokens:0,cache_creation_input_tokens:0,cache_read_input_tokens:0}}}},yield{event:"content_block_start",data:{type:"content_block_start",index:0,content_block:{type:"thinking",thinking:"",signature:""}}});const a=await n,h=l(function*(){yield{event:"content_block_delta",data:{type:"content_block_delta",index:0,delta:{type:"signature_delta",signature:D()}}},yield{event:"content_block_stop",data:{type:"content_block_stop",index:0}}},"closeEarly");if(a.stream){d=!0;let u=null;for await(const c of a.stream){if(!g){yield c;continue}if(c.event!=="message_start"){if(u===null&&c.event==="content_block_start"&&c.data.content_block.type==="thinking"){u=0;continue}u===null&&(yield*h(),u=1),yield u>0&&typeof c.data?.index=="number"?{...c,data:{...c.data,index:c.data.index+u}}:c}}return}g&&(yield*h()),yield*O(o,a.body)}finally{d||n.then(g=>g.stream?.return?.(),()=>{})}}l(W,"late");function*O(n,o){if(n.front==="anthropic"){yield{event:"error",data:o};return}yield{data:o},yield{data:"[DONE]"}}l(O,"failure");async function H(n,o,i,d,g,a=200,h=G){n.hijack();const u=n.raw;u.writeHead(a,h);try{await o.write(u,d,{front:o.front,interval:g})}catch(c){if(!i.signal.aborted){const k=c instanceof x;k||(n.log.error(c),o.emit({type:"error",kind:"route",level:"error",message:String(c?.message??c),log:!1}));const{body:y}=o.build(k?c.status:502,k?c.message:"Internal server error",k?{type:c.type,code:c.code}:{});for(const m of O(o,y))u.write(o.encode(m))}}return u.end(),n}l(H,"stream");function A(n,o){const i=Buffer.from(n),d=Buffer.from(o);return i.length===d.length&&I(i,d)}l(A,"same");function T(n){const o=/^Bearer\s+(.+)$/i.exec(n.authorization??"");return n["x-api-key"]||o?.[1]||""}l(T,"keyOf");function $(n){const o={};for(const[i,d]of Object.entries(n))(i==="anthropic-beta"||i==="anthropic-version"||i.startsWith("x-claude-code-"))&&(o[i]=String(d));return o}l($,"forwardable");function F(n){if(n===!0)return{ok:!0,context:void 0};if(n&&typeof n=="object"&&n.ok===!0)return{ok:!0,context:n.context,cacheSalt:n.cacheSalt};const o=n&&typeof n=="object"&&n.ok===!1?n:{};return{ok:!1,status:Number.isInteger(o.status)&&o.status>=400&&o.status<=599?o.status:401,message:typeof o.message=="string"&&o.message!==""?o.message:"unauthorized"}}l(F,"verdictOf");function q(n,o,i){const{front:d,filter:g}=i,a={...i,build:l((e,t,r={})=>i.build(e,t,{...r,front:d}),"build"),send:l((e,t)=>i.send(e,{...t,front:d,filter:g}),"send"),route:l(e=>i.route(e,{front:d}),"route"),count:l((e,t)=>i.count(e,{...t,front:d}),"count"),embed:l((e,t)=>i.embed(e,t),"embed"),list:l(e=>i.list({...e,front:d}),"list"),get:l((e,t)=>i.get(e,{...t,front:d}),"get")},h=[o.apiKey??[]].flat().filter(Boolean),u=o.bodyLimit??R,c=new WeakMap,k=new WeakMap,y=o.metrics===!0?U({front:d,endpoints:i.endpoints}):null,m=l((e,t)=>e.code(t.status).headers(t.headers).send(t.body),"reject");n.setErrorHandler((e,t,r)=>{if(e instanceof x)return m(r,a.build(e.status,e.message,{type:e.type,code:e.code,param:e.param}));const s=e.statusCode??500;return s<500?m(r,a.build(s,e.message)):(t.log.error(e),a.emit({type:"error",kind:"route",level:"error",message:String(e.message),status:s,log:!1}),m(r,a.build(s,"Internal server error")))});const w=z(o.cors),v=B(o.rateLimit);w&&n.addHook("onRequest",async(e,t)=>{for(const[r,s]of Object.entries(w.headers(e.headers.origin)))t.raw.setHeader(r,s)}),y&&n.addHook("onRequest",y.hook),v&&n.addHook("onRequest",async(e,t)=>{if(e.routeOptions.config?.public||e.routeOptions.config?.metrics)return;const r=T(e.headers),s=v.take(r&&h.some(b=>A(r,b))?`key:${r}`:`ip:${e.ip}`);if(s===0)return;const f=a.build(429,"rate limit exceeded",{type:"rate_limit_error"});return t.code(429).headers({...f.headers,"retry-after":String(s)}).send(f.body)}),w&&n.route({method:"OPTIONS",url:"/*",config:{public:!0,metrics:!0},handler:l((e,t)=>{const r=w.preflight(e.headers.origin,e.headers["access-control-request-headers"]);for(const[s,f]of Object.entries(r))t.raw.setHeader(s,f);return t.code(204).send()},"handler")}),n.addHook("onRequest",async(e,t)=>{if(h.length===0||e.routeOptions.config?.public)return;const r=T(e.headers);if(!(r&&h.some(s=>A(r,s))))return m(t,a.build(401,"invalid x-api-key"))}),typeof o.authorize=="function"&&n.addHook("preHandler",async(e,t)=>{if(e.routeOptions.config?.public)return;let r=null;if(e.body&&typeof e.body=="object"&&!Array.isArray(e.body))try{r=a.route(e.body)}catch{}let s;try{s=F(await o.authorize({method:e.method,url:e.url,headers:e.headers,body:e.body,front:a.front,route:r}))}catch(f){e.log.error(f),a.emit({type:"error",kind:"route",level:"error",message:String(f?.message??f),log:!1}),s={ok:!1,status:401,message:"unauthorized"}}if(!s.ok)return m(t,a.build(s.status,s.message));c.set(e,s.context),typeof s.cacheSalt=="string"&&s.cacheSalt!==""&&k.set(e,s.cacheSalt)}),n.route({method:["GET","HEAD"],url:"/",config:{public:!0},handler:l(async(e,t)=>t.send({ok:!0}),"handler")}),n.route({method:["GET","HEAD"],url:"/api/hello",config:{public:!0},handler:l(async(e,t)=>t.send({ok:!0}),"handler")}),y&&n.get("/metrics",{config:{metrics:!0}},async(e,t)=>t.header("content-type",N).send(y.render())),n.get("/v1/models",async(e,t)=>{const r=new AbortController;return t.raw.on("close",()=>r.abort()),a.list({signal:r.signal})}),n.get("/v1/models/*",async(e,t)=>{const r=new AbortController;return t.raw.on("close",()=>r.abort()),a.get(e.params["*"],{signal:r.signal})});const j=a.front==="anthropic"?"/v1/messages":"/v1/chat/completions";n.post(j,{bodyLimit:u},async(e,t)=>{if(t.raw.destroyed)return t;const r=new AbortController;t.raw.on("close",()=>{t.raw.writableFinished||r.abort()});const s=typeof o.onUsage=="function"?_=>o.onUsage(_,c.get(e)):void 0,f=a.interval(e.body),b=a.send(e.body,{signal:r.signal,clientHeaders:$(e.headers),cacheSalt:k.get(e),onUsage:s,log:e.log});b.catch(()=>{}),y?.attach(e,b);const L=new Promise(_=>r.signal.addEventListener("abort",()=>_(E),{once:!0})),p=await M(Promise.race([b,L]),e.body?.stream===!0?f:0);return p===E?(b.then(_=>_.stream?.return?.(),()=>{}),t):p===S?H(t,a,r,W(b,a,e.body?.model),f):p.stream?H(t,a,r,p.stream,f,p.status,p.headers):m(t,p)}),a.front==="openai"&&n.post("/v1/embeddings",{bodyLimit:u},async(e,t)=>{const r=new AbortController;t.raw.on("close",()=>{t.raw.writableFinished||r.abort()});const s=typeof o.onUsage=="function"?b=>o.onUsage(b,c.get(e)):void 0,f=a.embed(e.body,{signal:r.signal,onUsage:s,log:e.log});return y?.attach(e,f),m(t,await f)}),a.front==="anthropic"&&n.post("/v1/messages/count_tokens",{bodyLimit:u},async(e,t)=>{const r=new AbortController;return t.raw.on("close",()=>r.abort()),a.count(e.body,{signal:r.signal})})}l(q,"register");export{W as late,q as register};
2
2
  //# sourceMappingURL=routes.mjs.map
@@ -1,2 +1,2 @@
1
- var l=Object.defineProperty;var i=(o,e)=>l(o,"name",{value:e,configurable:!0});import{TranslatorError as a}from"./errors.mjs";const r=Object.freeze({inlineSystem:"auto",stripBillingHeader:!0,reasoningField:"reasoning",thinkTags:!0,thinkOpen:!1,reasoningRoundTrip:!1,thinking:"chat_template_kwargs",thinkingBudget:!1,effort:!1,images:!0,imageFallback:!0,toolFallback:!1,documents:"extract",documentConverter:null,pdf:Object.freeze({pages:!0,maxPages:100,maxBytes:32*1024*1024,maxImages:20,scale:1.5}),toolSchema:"pass",deferredTools:"forward",toolSearch:"expand",jsonObject:"schema",jsonObjectFallback:!0,toolErrors:!1,toolErrorMarker:"[error] ",toolErrorInference:!1,maxTokensField:"max_completion_tokens",streamUsage:!0,continuousUsage:!1,prefill:"continue_final_message",topK:!0,responseFormat:!0,parallelToolCalls:!0,unknownFields:"drop",sampling:Object.freeze({}),pingInterval:1e4,toolStreaming:"buffer",effortPassthrough:!1,passthroughFilter:!1,nativeMessages:!1,contextWindow:0,defaultMaxTokens:8192}),f={inlineSystem:["auto","keep","hoist","user","drop"],reasoningField:["reasoning","reasoning_content"],thinking:["chat_template_kwargs","reasoning_effort","none"],documents:["extract","note","error"],toolSchema:["pass","sanitize"],deferredTools:["forward","drop"],toolSearch:["expand","inline","native"],jsonObject:["schema","instruction","drop"],maxTokensField:["max_completion_tokens","max_tokens"],prefill:["continue_final_message","none"],unknownFields:["drop","error"],toolStreaming:["buffer","stream"]},d=["effortPassthrough","passthroughFilter","imageFallback","toolFallback","jsonObjectFallback","thinkOpen","reasoningRoundTrip","toolErrorInference"];function p(...o){const e={...r,pdf:{...r.pdf},sampling:{...r.sampling}};for(const t of o)if(t)for(const[n,s]of Object.entries(t))s!==void 0&&(n==="pdf"?Object.assign(e.pdf,s):n==="sampling"&&s!==null&&typeof s=="object"&&!Array.isArray(s)?Object.assign(e.sampling,s):e[n]=s);for(const[t,n]of Object.entries(f))if(!n.includes(e[t]))throw new a(`Invalid setting ${t}: ${JSON.stringify(e[t])} (expected one of ${n.join(", ")})`,{code:"invalid_setting",status:500,details:{key:t,allowed:n}});for(const t of d)if(typeof e[t]!="boolean")throw new a(`Invalid setting ${t}: ${JSON.stringify(e[t])} (expected true or false)`,{code:"invalid_setting",status:500,details:{key:t}});if(e.nativeMessages!==!0&&e.nativeMessages!==!1&&e.nativeMessages!=="auto")throw new a(`Invalid setting nativeMessages: ${JSON.stringify(e.nativeMessages)} (expected true, false or "auto")`,{code:"invalid_setting",status:500,details:{key:"nativeMessages"}});if(typeof e.toolErrorMarker!="string"||e.toolErrorMarker.trim()==="")throw new a(`Invalid setting toolErrorMarker: ${JSON.stringify(e.toolErrorMarker)} (expected a non-empty string)`,{code:"invalid_setting",status:500,details:{key:"toolErrorMarker"}});if(e.sampling===null||typeof e.sampling!="object"||Array.isArray(e.sampling))throw new a(`Invalid setting sampling: ${JSON.stringify(e.sampling)} (expected an object of field names and values)`,{code:"invalid_setting",status:500,details:{key:"sampling"}});if(!(Number.isFinite(e.pdf.scale)&&e.pdf.scale>0))throw new a(`Invalid setting pdf.scale: ${JSON.stringify(e.pdf.scale)} (expected a positive number)`,{code:"invalid_setting",status:500,details:{key:"pdf.scale"}});return e}i(p,"resolve");export{r as DEFAULTS,p as resolve};
1
+ var c=Object.defineProperty;var l=(o,e)=>c(o,"name",{value:e,configurable:!0});import{TranslatorError as n}from"./errors.mjs";import{check as f}from"./constraint.mjs";import{valid as d}from"./cache.mjs";const r=Object.freeze({inlineSystem:"auto",stripBillingHeader:!0,reasoningField:"reasoning",thinkTags:!0,thinkOpen:!1,reasoningRoundTrip:!1,thinking:"chat_template_kwargs",thinkingBudget:!1,effort:!1,images:!0,imageFallback:!0,toolFallback:!1,documents:"extract",documentConverter:null,pdf:Object.freeze({pages:!0,maxPages:100,maxBytes:32*1024*1024,maxImages:20,scale:1.5}),toolSchema:"pass",deferredTools:"forward",toolSearch:"expand",jsonObject:"schema",jsonObjectFallback:!0,toolErrors:!0,toolErrorMarker:"[error] ",toolErrorInference:!1,maxTokensField:"max_completion_tokens",streamUsage:!0,continuousUsage:!1,prefill:"continue_final_message",topK:!0,responseFormat:!0,parallelToolCalls:!0,unknownFields:"drop",sampling:Object.freeze({}),constraint:null,cacheSalt:null,promptCacheKey:null,pingInterval:1e4,toolStreaming:"buffer",effortPassthrough:!1,passthroughFilter:!1,nativeMessages:!1,contextWindow:0,defaultMaxTokens:8192}),g={inlineSystem:["auto","keep","hoist","user","drop"],reasoningField:["reasoning","reasoning_content"],thinking:["chat_template_kwargs","reasoning_effort","none"],documents:["extract","note","error"],toolSchema:["pass","sanitize"],deferredTools:["forward","drop"],toolSearch:["expand","inline","native"],jsonObject:["schema","instruction","drop"],maxTokensField:["max_completion_tokens","max_tokens"],prefill:["continue_final_message","none"],unknownFields:["drop","error"],toolStreaming:["buffer","stream"]},p=["effortPassthrough","passthroughFilter","imageFallback","toolFallback","jsonObjectFallback","thinkOpen","reasoningRoundTrip","toolErrorInference"];function y(...o){const e={...r,pdf:{...r.pdf},sampling:{...r.sampling}};for(const t of o)if(t)for(const[s,a]of Object.entries(t))a!==void 0&&(s==="pdf"?Object.assign(e.pdf,a):s==="sampling"&&a!==null&&typeof a=="object"&&!Array.isArray(a)?Object.assign(e.sampling,a):e[s]=a);for(const[t,s]of Object.entries(g))if(!s.includes(e[t]))throw new n(`Invalid setting ${t}: ${JSON.stringify(e[t])} (expected one of ${s.join(", ")})`,{code:"invalid_setting",status:500,details:{key:t,allowed:s}});for(const t of p)if(typeof e[t]!="boolean")throw new n(`Invalid setting ${t}: ${JSON.stringify(e[t])} (expected true or false)`,{code:"invalid_setting",status:500,details:{key:t}});if(e.nativeMessages!==!0&&e.nativeMessages!==!1&&e.nativeMessages!=="auto")throw new n(`Invalid setting nativeMessages: ${JSON.stringify(e.nativeMessages)} (expected true, false or "auto")`,{code:"invalid_setting",status:500,details:{key:"nativeMessages"}});if(typeof e.toolErrorMarker!="string"||e.toolErrorMarker.trim()==="")throw new n(`Invalid setting toolErrorMarker: ${JSON.stringify(e.toolErrorMarker)} (expected a non-empty string)`,{code:"invalid_setting",status:500,details:{key:"toolErrorMarker"}});if(e.sampling===null||typeof e.sampling!="object"||Array.isArray(e.sampling))throw new n(`Invalid setting sampling: ${JSON.stringify(e.sampling)} (expected an object of field names and values)`,{code:"invalid_setting",status:500,details:{key:"sampling"}});const i=f(e.constraint);if(i!==null)throw new n(`Invalid setting constraint: ${JSON.stringify(e.constraint)} (${i})`,{code:"invalid_setting",status:500,details:{key:"constraint"}});for(const t of["cacheSalt","promptCacheKey"])if(!d(e[t]))throw new n(`Invalid setting ${t}: ${JSON.stringify(e[t])} (expected a non-empty string or null)`,{code:"invalid_setting",status:500,details:{key:t}});if(!(Number.isFinite(e.pdf.scale)&&e.pdf.scale>0))throw new n(`Invalid setting pdf.scale: ${JSON.stringify(e.pdf.scale)} (expected a positive number)`,{code:"invalid_setting",status:500,details:{key:"pdf.scale"}});return e}l(y,"resolve");export{r as DEFAULTS,y as resolve};
2
2
  //# sourceMappingURL=settings.mjs.map
package/dist/lib/vllm.mjs CHANGED
@@ -1,4 +1,4 @@
1
- var c=Object.defineProperty;var s=(t,r)=>c(t,"name",{value:r,configurable:!0});import{atLeast as f,semver as l}from"./normalize.mjs";import{sign as a}from"./thinking.mjs";const u="0.11.1",y="x-anthropic-billing-header:";function A(t,r){if(t.driver!=="vllm"||r.nativeMessages===!1)return!1;if(r.nativeMessages===!0)return!0;const n=t.facts?.version??t.version;return l(n)!==null&&f(n,u)}s(A,"wants");function*p(t){for(const r of t)if(Array.isArray(r?.content))for(const n of r.content)yield n,n?.type==="tool_result"&&Array.isArray(n.content)&&(yield*n.content)}s(p,"blocks");function v(t,r){const n=Array.isArray(t.messages)?t.messages:[];if(n.at(-1)?.role==="assistant")return"prefill";const e=[...p(n)];return e.some(o=>o?.type==="document")?"document":r.toolSearch!=="native"&&(Array.isArray(t.tools)&&t.tools.some(i=>i?.defer_loading!==void 0)||e.some(i=>["tool_reference","tool_addition","tool_removal"].includes(i?.type)))?"toolsearch":null}s(v,"unsuited");function d(t,r){return(typeof t=="string"?[{type:"text",text:t}]:t).filter(e=>e?.type==="text"&&typeof e.text=="string").filter(e=>!(r.stripBillingHeader&&e.text.trimStart().startsWith(y))).map(e=>e.text).join(`
1
+ var c=Object.defineProperty;var o=(t,n)=>c(t,"name",{value:n,configurable:!0});import{atLeast as f,semver as l}from"./normalize.mjs";import{sign as a}from"./thinking.mjs";const u="0.11.1",y="x-anthropic-billing-header:";function A(t,n){if(t.driver!=="vllm"||n.nativeMessages===!1)return!1;if(n.nativeMessages===!0)return!0;const e=t.facts?.version??t.version;return l(e)!==null&&f(e,u)}o(A,"wants");function*p(t){for(const n of t)if(Array.isArray(n?.content))for(const e of n.content)yield e,e?.type==="tool_result"&&Array.isArray(e.content)&&(yield*e.content)}o(p,"blocks");function v(t,n){const e=Array.isArray(t.messages)?t.messages:[];if(e.at(-1)?.role==="assistant")return"prefill";const r=[...p(e)];return r.some(s=>s?.type==="document")?"document":n.toolSearch!=="native"&&(Array.isArray(t.tools)&&t.tools.some(i=>i?.defer_loading!==void 0)||r.some(i=>["tool_reference","tool_addition","tool_removal"].includes(i?.type)))?"toolsearch":null}o(v,"unsuited");function d(t,n){return(typeof t=="string"?[{type:"text",text:t}]:t).filter(r=>r?.type==="text"&&typeof r.text=="string").filter(r=>!(n.stripBillingHeader&&r.text.trimStart().startsWith(y))).map(r=>r.text).join(`
2
2
 
3
- `)}s(d,"systemText");function m(t,r){const n=r.toolErrorMarker;return typeof t.content=="string"?{...t,content:`${n}${t.content}`}:Array.isArray(t.content)?{...t,content:[{type:"text",text:n.trimEnd()},...t.content]}:{...t,content:n.trimEnd()}}s(m,"marked");function h(t,r){const n={...t};if(n.system!==void 0&&n.system!==null){const e=d(n.system,r);e===""?delete n.system:n.system=e}return r.toolErrors&&Array.isArray(n.messages)&&(n.messages=n.messages.map(e=>Array.isArray(e?.content)?{...e,content:e.content.map(o=>o?.type==="tool_result"&&o.is_error===!0?m(o,r):o)}:e)),n}s(h,"request");function E(t){return Array.isArray(t?.content)?{...t,content:t.content.map(r=>r?.type==="thinking"&&typeof r.signature!="string"?{...r,signature:a()}:r)}:t}s(E,"response");async function*M(t){const r=new Map;for await(const n of t){const e=n.data;e?.type==="content_block_start"&&e.content_block?.type==="thinking"?r.set(e.index,typeof e.content_block.signature=="string"):e?.type==="content_block_delta"&&e.delta?.type==="signature_delta"?r.set(e.index,!0):e?.type==="content_block_stop"&&r.get(e.index)===!1&&(yield{event:"content_block_delta",data:{type:"content_block_delta",index:e.index,delta:{type:"signature_delta",signature:a()}}}),e?.type==="content_block_stop"&&r.delete(e.index),yield n,e?.type==="message_start"&&(yield{event:"ping",data:{type:"ping"}})}}s(M,"stream");export{u as MIN_VERSION,h as request,E as response,M as stream,v as unsuited,A as wants};
3
+ `)}o(d,"systemText");function m(t,n){const e=n.toolErrorMarker;return typeof t.content=="string"?{...t,content:`${e}${t.content}`}:Array.isArray(t.content)?{...t,content:[{type:"text",text:e.trimEnd()},...t.content]}:{...t,content:e.trimEnd()}}o(m,"marked");function h(t,n){const e={...t};if(e.system!==void 0&&e.system!==null){const s=d(e.system,n);s===""?delete e.system:e.system=s}const r=o(s=>s?.type==="tool_result"&&s.is_error===!0,"failed");return n.toolErrors&&Array.isArray(e.messages)&&e.messages.some(s=>Array.isArray(s?.content)&&s.content.some(r))&&(e.messages=e.messages.map(s=>Array.isArray(s?.content)&&s.content.some(r)?{...s,content:s.content.map(i=>r(i)?m(i,n):i)}:s)),e}o(h,"request");function E(t){return Array.isArray(t?.content)?{...t,content:t.content.map(n=>n?.type==="thinking"&&typeof n.signature!="string"?{...n,signature:a()}:n)}:t}o(E,"response");async function*M(t){const n=new Map;for await(const e of t){const r=e.data;r?.type==="content_block_start"&&r.content_block?.type==="thinking"?n.set(r.index,typeof r.content_block.signature=="string"):r?.type==="content_block_delta"&&r.delta?.type==="signature_delta"?n.set(r.index,!0):r?.type==="content_block_stop"&&n.get(r.index)===!1&&(yield{event:"content_block_delta",data:{type:"content_block_delta",index:r.index,delta:{type:"signature_delta",signature:a()}}}),r?.type==="content_block_stop"&&n.delete(r.index),yield e,r?.type==="message_start"&&(yield{event:"ping",data:{type:"ping"}})}}o(M,"stream");export{u as MIN_VERSION,h as request,E as response,M as stream,v as unsuited,A as wants};
4
4
  //# sourceMappingURL=vllm.mjs.map
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@cldmv/llm-translator",
3
- "version": "0.10.0",
3
+ "version": "0.12.0",
4
4
  "description": "Bidirectional Anthropic Messages ⇄ OpenAI Chat Completions translator with backend auto-detection (vLLM and other OpenAI-compatible servers), built on slothlet, with an optional standalone server for running Claude Code against local models.",
5
5
  "type": "module",
6
6
  "license": "Apache-2.0",
@@ -1,5 +1,5 @@
1
1
  /**
2
- * Bind every endpoint to a driver. `backend: "auto"` endpoints are probed (drivers tried in `info().order`, the generic
2
+ * Bind every endpoint to a driver. `backend: "auto"` endpoints are probed (drivers tried in `info().order` after any driver whose `info().hosts` names the endpoint host, the generic
3
3
  * driver matches last); endpoints with an explicit `backend` skip detection. Call it again to re-probe.
4
4
  * @param {any} [options] - `{ network, signal }`. `network: false` skips all probing (auto endpoints fall back to the generic driver).
5
5
  * @returns {Promise<any>} The `status()` result.
@@ -19,7 +19,7 @@
19
19
  /**
20
20
  * Describe the Azure OpenAI driver. The key goes in the `api-key` header. The real paths come from the `path` leaf, because they depend on the
21
21
  * upstream's `apiVersion` and `deployment`; `paths` holds the v1 API paths for a base URL without `/openai`.
22
- * @returns {{name: string, protocol: string, order: number, auth: string, strict: boolean, paths: Record<string,string>}} Driver facts.
22
+ * @returns {{name: string, protocol: string, order: number, auth: string, strict: boolean, hostSuffixes: string[], paths: Record<string,string>}} Driver facts.
23
23
  */
24
24
  export function info(): {
25
25
  name: string;
@@ -27,5 +27,6 @@ export function info(): {
27
27
  order: number;
28
28
  auth: string;
29
29
  strict: boolean;
30
+ hostSuffixes: string[];
30
31
  paths: Record<string, string>;
31
32
  };
@@ -18,7 +18,7 @@
18
18
  */
19
19
  /**
20
20
  * Describe the DeepSeek driver.
21
- * @returns {{name: string, protocol: string, order: number, auth: string, strict: boolean, paths: Record<string,string>}} Driver facts.
21
+ * @returns {{name: string, protocol: string, order: number, auth: string, strict: boolean, hosts: string[], paths: Record<string,string>}} Driver facts.
22
22
  */
23
23
  export function info(): {
24
24
  name: string;
@@ -26,5 +26,6 @@ export function info(): {
26
26
  order: number;
27
27
  auth: string;
28
28
  strict: boolean;
29
+ hosts: string[];
29
30
  paths: Record<string, string>;
30
31
  };
@@ -18,7 +18,7 @@
18
18
  */
19
19
  /**
20
20
  * Describe the Mistral driver. Mistral accepts only 9-character alphanumeric tool-call ids, so the translator rewrites the client's ids.
21
- * @returns {{name: string, protocol: string, order: number, auth: string, strict: boolean, toolCallIds: {alphabet: string, min: number, max: number}, paths: Record<string,string>}} Driver facts.
21
+ * @returns {{name: string, protocol: string, order: number, auth: string, strict: boolean, hosts: string[], toolCallIds: {alphabet: string, min: number, max: number}, paths: Record<string,string>}} Driver facts.
22
22
  */
23
23
  export function info(): {
24
24
  name: string;
@@ -26,6 +26,7 @@ export function info(): {
26
26
  order: number;
27
27
  auth: string;
28
28
  strict: boolean;
29
+ hosts: string[];
29
30
  toolCallIds: {
30
31
  alphabet: string;
31
32
  min: number;
@@ -18,7 +18,7 @@
18
18
  */
19
19
  /**
20
20
  * Describe the OpenAI driver.
21
- * @returns {{name: string, protocol: string, order: number, auth: string, strict: boolean, paths: Record<string,string>}} Driver facts.
21
+ * @returns {{name: string, protocol: string, order: number, auth: string, strict: boolean, hosts: string[], fields: string[], paths: Record<string,string>}} Driver facts.
22
22
  */
23
23
  export function info(): {
24
24
  name: string;
@@ -26,5 +26,7 @@ export function info(): {
26
26
  order: number;
27
27
  auth: string;
28
28
  strict: boolean;
29
+ hosts: string[];
30
+ fields: string[];
29
31
  paths: Record<string, string>;
30
32
  };
@@ -18,7 +18,7 @@
18
18
  */
19
19
  /**
20
20
  * Describe the OpenRouter driver. The API lives under `/api`; the `path` leaf adds it, so `paths` holds the usual `/v1/...` ones.
21
- * @returns {{name: string, protocol: string, order: number, auth: string, strict: boolean, paths: Record<string,string>}} Driver facts.
21
+ * @returns {{name: string, protocol: string, order: number, auth: string, strict: boolean, hosts: string[], paths: Record<string,string>}} Driver facts.
22
22
  */
23
23
  export function info(): {
24
24
  name: string;
@@ -26,5 +26,6 @@ export function info(): {
26
26
  order: number;
27
27
  auth: string;
28
28
  strict: boolean;
29
+ hosts: string[];
29
30
  paths: Record<string, string>;
30
31
  };
@@ -2,7 +2,7 @@
2
2
  * Route a client request body (in the configured `front` protocol) to an upstream and convert it to that upstream's
3
3
  * protocol. Does no I/O.
4
4
  * @param {any} body - Client request body (Anthropic Messages or OpenAI Chat Completions, per `front`).
5
- * @param {any} [options] - `{ front, headers, clientHeaders, filter, images, usage }`. `front` overrides the configured front for this call, `headers` are extra upstream headers, `clientHeaders` are client headers forwarded only to a native Anthropic upstream, `filter` restricts a forwarded OpenAI body to known fields (default: the `passthroughFilter` setting, off), `images: false` sends the request without images (used by the image fallback), `endpoint` sends to that member of the model's pool (used for failover; default: the first in the pool's order), `hold: true` (used by `send`) advances a round robin and takes the chosen endpoint's in-flight slot, which the caller then releases, `jsonObject: "instruction"` sends a `json_object` response format as a system instruction (used by the JSON fallback, and applied by itself once a refusal is remembered for the endpoint and model), `tools: false` sends the request without tools (used by the tool fallback), `usage: true` (used by `send`) makes a streaming OpenAI-front request always ask the upstream for its final usage chunk, `retry: true` (used by `send` for the same request sent again) keeps the `dropped` event from repeating, `log` is the logger events of this request also go to (used by the server), `budget` (from `createBudget`, one per `send` for all its attempts) limits the cost of decoding reasoning signatures.
5
+ * @param {any} [options] - `{ front, headers, clientHeaders, filter, images, usage }`. `front` overrides the configured front for this call, `headers` are extra upstream headers, `clientHeaders` are client headers forwarded only to a native Anthropic upstream, `filter` restricts a forwarded OpenAI body to known fields (default: the `passthroughFilter` setting, off), `images: false` sends the request without images (used by the image fallback), `endpoint` sends to that member of the model's pool (used for failover; default: the first in the pool's order), `hold: true` (used by `send`) advances a round robin and takes the chosen endpoint's in-flight slot, which the caller then releases, `jsonObject: "instruction"` sends a `json_object` response format as a system instruction (used by the JSON fallback, and applied by itself once a refusal is remembered for the endpoint and model), `tools: false` sends the request without tools (used by the tool fallback), `usage: true` (used by `send`) makes a streaming OpenAI-front request always ask the upstream for its final usage chunk, `cacheSalt` (a trusted per-call vLLM `cache_salt`, which replaces the upstream's `cacheSalt` setting), `retry: true` (used by `send` for the same request sent again) keeps the `dropped` event from repeating, `log` is the logger events of this request also go to (used by the server), `budget` (from `createBudget`, one per `send` for all its attempts) limits the cost of decoding reasoning signatures.
6
6
  * @returns {Promise<{upstream: string, method: string, url: string, path: string, headers: Record<string,string>, body: any, state: any, pool: string[], images: {sent: boolean, retry: boolean, key: string}, tools: {sent: boolean, retry: boolean}, json: {sent: boolean, retry: boolean, key: string}}>} The upstream request and the state `response`/`stream`/`error` need. `pool` lists the names of the endpoints serving the model in the order they are tried (`upstream` is the first); `images` tells `send` whether the body carries images, whether a refusal may be retried without them and the key under which a refusal is remembered; `tools` tells it whether the body carries tools and whether a tool-parser refusal may be retried without them; `json` tells it whether the body carries the schema emulation of `json_object`, whether a refusal may be retried with the instruction and the key under which a refusal is remembered.
7
7
  */
8
8
  export function request(body: any, options?: any): Promise<{
@@ -25,7 +25,7 @@
25
25
  * a stream has started, an error chunk and `[DONE]`. Aborting the caller's signal aborts every branch. An OpenAI-protocol upstream
26
26
  * receives `n` unchanged and answers it itself.
27
27
  * @param {any} body - Client request body (per `front`).
28
- * @param {any} [options] - `{ signal, front, headers, clientHeaders, filter, onUsage, log }`: abort signal for the upstream call, per-call front, extra upstream headers, client headers forwarded to a native Anthropic upstream, the pass-through body filter (see `request`) and the usage callback (an exception in it is reported as a `usage` event and never breaks the reply) and the logger this request's events also go to (used by the server).
28
+ * @param {any} [options] - `{ signal, front, headers, clientHeaders, filter, cacheSalt, onUsage, log }`: abort signal for the upstream call, per-call front, extra upstream headers, client headers forwarded to a native Anthropic upstream, the pass-through body filter (see `request`) and the usage callback (an exception in it is reported as a `usage` event and never breaks the reply) and the logger this request's events also go to (used by the server).
29
29
  * @returns {Promise<{status: number, headers: Record<string,string>, body: any, usage: Promise<any>} | {status: number, headers: Record<string,string>, stream: AsyncIterable<{event?: string, data: *}>, usage: Promise<any>}>} A JSON reply, or a stream reply when the client asked for streaming and the upstream accepted.
30
30
  */
31
31
  export function send(body: any, options?: any): Promise<{
@@ -5,7 +5,7 @@
5
5
  * `app.register(api.server.plugin, { apiKey })`. `authorize(request)` (sync or async) runs after `apiKey` and the body parse and decides
6
6
  * with `true` / `{ ok: true, context }` or `false` / `{ ok: false, status, message }`; `onUsage(report, context)` receives each request's usage report. When the Fastify `logger` is enabled, events also go to it as structured lines (`{ event }` plus the message), through the request's logger for events of a request.
7
7
  * @param {any} fastify - Fastify instance.
8
- * @param {any} [options] - `{ front, apiKey, bodyLimit, authorize, onUsage }`. `authorize(request)` and `onUsage(report, context)` are described above. `front` sets the protocol this plugin instance serves (default: the translator's front, other plugin instances and `api.send` are unaffected); `apiKey` (string or array) is required from clients. Forwarded OpenAI bodies are restricted to known fields unless the `passthroughFilter` setting is `false`.
8
+ * @param {any} [options] - `{ front, apiKey, bodyLimit, authorize, onUsage, metrics, cors, rateLimit }`. `cors` (a list of origins, `"*"` for any) and `rateLimit` (`{ requests, windowMs }`) are described in docs/server.md and are off unless set. `metrics: true` adds `GET /metrics` (Prometheus text format, behind the same authentication as every other route; off by default, then an unknown route). `authorize(request)` and `onUsage(report, context)` are described above. `front` sets the protocol this plugin instance serves (default: the translator's front, other plugin instances and `api.send` are unaffected); `apiKey` (string or array) is required from clients. Forwarded OpenAI bodies are restricted to known fields unless the `passthroughFilter` setting is `false`.
9
9
  * @returns {Promise<void>} Resolves once the routes are registered.
10
10
  */
11
11
  export function plugin(fastify: any, options?: any): Promise<void>;
@@ -1,6 +1,6 @@
1
1
  /**
2
2
  * Start an HTTP server in front of the translator. Fastify is an optional peer dependency and is imported here.
3
- * @param {any} [options] - `{ host = "127.0.0.1", port = 8787, front, apiKey, bodyLimit, logger, allowUnauthenticated, authorize, onUsage }`. `authorize(request)` and `onUsage(report, context)` are described at `server.plugin`. Listening on a non-loopback address without an `apiKey` or an `authorize` hook is refused unless `allowUnauthenticated` is true.
3
+ * @param {any} [options] - `{ host = "127.0.0.1", port = 8787, front, apiKey, bodyLimit, logger, allowUnauthenticated, authorize, onUsage, metrics, tls, cors, rateLimit }`. `authorize(request)` and `onUsage(report, context)` are described at `server.plugin`. `tls` is `{ key, cert, ca, passphrase }` (file paths; `key` and `cert` together) and serves HTTPS; `cors` is a list of allowed origins (`"*"` for any); `rateLimit` is `{ requests, windowMs = 60000 }` per API key (remote address without one). Each is off unless set; see docs/server.md. Listening on a non-loopback address without an `apiKey` or an `authorize` hook is refused unless `allowUnauthenticated` is true.
4
4
  * @returns {Promise<{url: string, host: string, port: number, app: any, close: () => Promise<void>}>} Running server handle.
5
5
  */
6
6
  export function start(options?: any): Promise<{
@@ -0,0 +1,30 @@
1
+ /**
2
+ * Is a salt or key value usable?
3
+ * @param {any} value - Setting or request value.
4
+ * @returns {boolean} True for a non-empty string, or an absent value.
5
+ */
6
+ export function valid(value: any): boolean;
7
+ /**
8
+ * Add the cache hints to an OpenAI-protocol body.
9
+ *
10
+ * - `cache_salt` comes only from trusted places: the `cacheSalt` call option (the server passes the `authorize` hook's `cacheSalt`) or the upstream's `cacheSalt` setting,
11
+ * in that order. It always replaces a `cache_salt` the body carries, so a client cannot choose its own tenant. A backend without the feature
12
+ * follows `unknownFields`: dropped (listed in `dropped`) or rejected.
13
+ * - `prompt_cache_key` comes from the client (`metadata.prompt_cache_key` on the Anthropic front; the OpenAI front's own field is kept) or the upstream's `promptCacheKey` setting. A
14
+ * key the body already carries wins over the setting.
15
+ * @param {any} body - Converted or forwarded Chat Completions body.
16
+ * @param {{ salt?: any, key?: any }} asked - The call's `cacheSalt` option and the request's prompt cache key.
17
+ * @param {any} settings - Resolved settings (`cacheSalt`, `promptCacheKey`, `unknownFields`).
18
+ * @param {string} driver - Driver name of the upstream.
19
+ * @param {string[]} [dropped] - Receives the names that were left out.
20
+ * @returns {any} The body, with the fields added (a copy when anything was added).
21
+ * @throws {TranslatorError} `invalid_request` for a malformed salt or key; `unsupported_field` when the backend lacks the feature and `unknownFields` is `"error"`.
22
+ */
23
+ export function addCache(body: any, asked: {
24
+ salt?: any;
25
+ key?: any;
26
+ }, settings: any, driver: string, dropped?: string[]): any;
27
+ /** Drivers whose servers take `cache_salt` (vLLM's per-tenant prefix-cache isolation). */
28
+ export const SALT_DRIVERS: string[];
29
+ /** Drivers whose servers take `prompt_cache_key` (OpenAI's cache routing hint). */
30
+ export const KEY_DRIVERS: string[];
@@ -0,0 +1,36 @@
1
+ /**
2
+ * What is wrong with a constraint value, if anything.
3
+ * @param {any} value - A `constraint` setting or request value.
4
+ * @returns {string | null} A message, or null when the value is a valid constraint (or absent).
5
+ */
6
+ export function check(value: any): string | null;
7
+ /**
8
+ * The fields to add to a Chat Completions body for one constraint, or null when the backend cannot enforce that kind.
9
+ * @param {string} driver - Driver name.
10
+ * @param {string | undefined} version - Server version (vLLM), when known.
11
+ * @param {string} kind - One of {@link KINDS}.
12
+ * @param {any} value - The constraint's value.
13
+ * @returns {Record<string, any> | null} Body fields, or null when unsupported.
14
+ */
15
+ export function fields(driver: string, version: string | undefined, kind: string, value: any): Record<string, any> | null;
16
+ /**
17
+ * Add a constraint to an OpenAI-protocol body in the backend's own fields. The request's constraint (`constraint` on the OpenAI front,
18
+ * `metadata.constraint` on the Anthropic front) replaces the upstream's `constraint` setting. A body that already carries the native field
19
+ * keeps it. vLLM applies a constraint to the whole generation, so with tools on offer the constraint is dropped and recorded (the tools win).
20
+ * A kind the backend cannot enforce follows `unknownFields`: dropped (listed in `dropped`) or rejected.
21
+ * @param {any} body - Converted or forwarded Chat Completions body.
22
+ * @param {any} asked - The request's constraint, if any.
23
+ * @param {any} settings - Resolved settings (`constraint`, `unknownFields`).
24
+ * @param {{ name: string, version?: string }} target - The upstream's driver name and server version.
25
+ * @param {string[]} [dropped] - Receives `constraint` names that were left out.
26
+ * @returns {any} The body, with the fields added (a copy when anything was added).
27
+ * @throws {TranslatorError} `invalid_request` for a malformed request constraint; `unsupported_field` when the backend cannot enforce the kind and `unknownFields` is `"error"`.
28
+ */
29
+ export function addConstraint(body: any, asked: any, settings: any, target: {
30
+ name: string;
31
+ version?: string;
32
+ }, dropped?: string[]): any;
33
+ /** The constraint kinds a client can ask for. */
34
+ export const KINDS: string[];
35
+ /** First vLLM release whose Chat Completions endpoint takes `structured_outputs`; older releases take `guided_json`, `guided_regex`, `guided_choice` and `guided_grammar`. */
36
+ export const STRUCTURED_OUTPUTS_VERSION: "0.11.0";
@@ -0,0 +1,34 @@
1
+ /**
2
+ * Whether an edit's parameters are all usable. Anything else cannot be honoured and the edit is dropped.
3
+ * @param {any} edit - A `clear_tool_uses` edit.
4
+ * @returns {boolean} True when every present parameter has its documented shape.
5
+ */
6
+ export function valid(edit: any): boolean;
7
+ /**
8
+ * Clear the results (and optionally the inputs) of the oldest tool uses of a converted conversation, as Anthropic's
9
+ * `clear_tool_uses_20250919` edit does. Messages are changed in place.
10
+ *
11
+ * Tool uses are counted in conversation order across every assistant turn. The edit applies when the trigger is exceeded (default: more
12
+ * than 100000 estimated input tokens). The most recent `keep` tool uses (default 3, whatever tool they are) are never cleared, nor are
13
+ * uses of a tool named in `exclude_tools`. The rest, oldest first, have their result replaced by a placeholder (and, with
14
+ * `clear_tool_inputs`, their arguments by `{}`). When fewer than `clear_at_least` input tokens would be freed nothing changes. Input
15
+ * tokens are estimated at one per four characters (`lib/estimate`), for the trigger and for the amount cleared.
16
+ * @param {any} edit - A valid `clear_tool_uses` edit (see `valid`).
17
+ * @param {{ messages: any[], tools?: any[] }} body - Converted OpenAI body; its messages are edited.
18
+ * @param {Record<string, string>} toolNames - `fitted -> original` tool names, to match `exclude_tools`.
19
+ * @returns {{ type: string, cleared_tool_uses: number, cleared_input_tokens: number } | null} The applied edit, or null when nothing was cleared.
20
+ */
21
+ export function clearToolUses(edit: any, body: {
22
+ messages: any[];
23
+ tools?: any[];
24
+ }, toolNames: Record<string, string>): {
25
+ type: string;
26
+ cleared_tool_uses: number;
27
+ cleared_input_tokens: number;
28
+ } | null;
29
+ /** Input tokens above which an edit with no `trigger` applies. */
30
+ export const DEFAULT_TRIGGER: 100000;
31
+ /** Recent tool uses an edit with no `keep` leaves alone. */
32
+ export const DEFAULT_KEEP: 3;
33
+ /** What a cleared tool result says. */
34
+ export const PLACEHOLDER: "[Tool result cleared to save context]";
@@ -0,0 +1,25 @@
1
+ /**
2
+ * Read the TLS files named by `options.tls` into the `https` options of Fastify / `node:https`.
3
+ * @param {any} tls - `{ key, cert, ca, passphrase }`: file paths (`ca` may be a list), `passphrase` a string; or undefined for plain HTTP.
4
+ * @returns {Promise<object | null>} `https` options, or null when TLS is not configured.
5
+ */
6
+ export function tlsOptions(tls: any): Promise<object | null>;
7
+ /**
8
+ * Build the CORS policy.
9
+ * @param {any} option - A list of origins, or `{ origins }`. `"*"` allows every origin. Undefined, null or an empty list turns CORS off.
10
+ * @returns {{ headers: (origin: string | undefined) => Record<string,string>, preflight: (origin: string | undefined, requested: string | undefined) => Record<string,string> } | null} Policy, or null when off.
11
+ */
12
+ export function cors(option: any): {
13
+ headers: (origin: string | undefined) => Record<string, string>;
14
+ preflight: (origin: string | undefined, requested: string | undefined) => Record<string, string>;
15
+ } | null;
16
+ /**
17
+ * Build a fixed-window rate limiter.
18
+ * @param {any} option - `{ requests, windowMs = 60000 }` (requests per window per client); undefined, null or false turns it off.
19
+ * @param {() => number} [now] - Clock in ms (tests).
20
+ * @returns {{ take: (key: string) => number, size: () => number } | null} Limiter, or null when off. `take(key)` returns 0 when the request may go on, else the seconds to wait.
21
+ */
22
+ export function rateLimit(option: any, now?: () => number): {
23
+ take: (key: string) => number;
24
+ size: () => number;
25
+ } | null;
@@ -0,0 +1,58 @@
1
+ /**
2
+ * Escape a label value: backslash, double quote and line feed.
3
+ * @param {any} value - Label value.
4
+ * @returns {string} Escaped text.
5
+ */
6
+ export function escapeLabel(value: any): string;
7
+ /**
8
+ * Escape `# HELP` text: backslash and line feed.
9
+ * @param {string} text - Help text.
10
+ * @returns {string} Escaped text.
11
+ */
12
+ export function escapeHelp(text: string): string;
13
+ /**
14
+ * Create an empty registry.
15
+ * @returns {{ counter: (name: string, help: string) => void, histogram: (name: string, help: string, buckets?: number[]) => void, inc: (name: string, labels: Record<string, any>, by?: number) => void, observe: (name: string, labels: Record<string, any>, value: number) => void, render: (gauges?: {name: string, help: string, samples: {labels: Record<string, any>, value: number}[]}[]) => string }} Registry.
16
+ */
17
+ export function create(): {
18
+ counter: (name: string, help: string) => void;
19
+ histogram: (name: string, help: string, buckets?: number[]) => void;
20
+ inc: (name: string, labels: Record<string, any>, by?: number) => void;
21
+ observe: (name: string, labels: Record<string, any>, value: number) => void;
22
+ render: (gauges?: {
23
+ name: string;
24
+ help: string;
25
+ samples: {
26
+ labels: Record<string, any>;
27
+ value: number;
28
+ }[];
29
+ }[]) => string;
30
+ };
31
+ /**
32
+ * The kind of an error, or null when the request did not fail: `upstream` (the upstream answered with an error or could not be reached
33
+ * as far as the usage report knows), `translator` (a failure in the translator itself, a 5xx or an in-band stream error) and `client`
34
+ * (a refused request: bad body, authentication, unknown model or route). A client that left is not an error.
35
+ * @param {number} status - HTTP status as written (499 when the client left).
36
+ * @param {any} report - The request's usage report, or null.
37
+ * @returns {"upstream" | "translator" | "client" | null} Kind.
38
+ */
39
+ export function kind(status: number, report: any): "upstream" | "translator" | "client" | null;
40
+ /**
41
+ * Create the metrics of one server: a registry plus the request recorder and the scrape renderer. Label values stay bounded: `front` is
42
+ * one of two protocols, `route` is the route pattern (never the URL), `driver` is a driver name, `upstream` a configured upstream name
43
+ * and `code` an HTTP status. The model a client asked for, request ids and API keys are never labels.
44
+ * @param {{ front: "anthropic" | "openai", endpoints: () => any[] }} source - The server's front and its runtime endpoints.
45
+ * @returns {{ hook: (request: any, reply: any) => Promise<void>, attach: (request: any, sending: Promise<any>) => void, render: () => string }} Recorder.
46
+ */
47
+ export function recorder(source: {
48
+ front: "anthropic" | "openai";
49
+ endpoints: () => any[];
50
+ }): {
51
+ hook: (request: any, reply: any) => Promise<void>;
52
+ attach: (request: any, sending: Promise<any>) => void;
53
+ render: () => string;
54
+ };
55
+ /** `Content-Type` of the text exposition format. */
56
+ export const CONTENT_TYPE: "text/plain; version=0.0.4; charset=utf-8";
57
+ /** Request duration buckets in seconds: LLM replies run from milliseconds (errors, embeddings) to minutes (long streams). */
58
+ export const BUCKETS: number[];
@@ -1,7 +1,7 @@
1
1
  /**
2
2
  * The driver facts of a preset.
3
3
  * @param {string} name - Preset name.
4
- * @returns {{name: string, protocol: string, order: number, auth: string, strict: boolean, fields?: string[], paths: Record<string,string>}} Driver facts.
4
+ * @returns {{name: string, protocol: string, order: number, auth: string, strict: boolean, hosts: string[], fields?: string[], paths: Record<string,string>}} Driver facts.
5
5
  */
6
6
  export function describe(name: string): {
7
7
  name: string;
@@ -9,6 +9,7 @@ export function describe(name: string): {
9
9
  order: number;
10
10
  auth: string;
11
11
  strict: boolean;
12
+ hosts: string[];
12
13
  fields?: string[];
13
14
  paths: Record<string, string>;
14
15
  };
@@ -10,7 +10,7 @@ export function late(sending: Promise<any>, api: any, model?: string): AsyncGene
10
10
  /**
11
11
  * Register the routes for one front on a Fastify instance.
12
12
  * @param {any} fastify - Fastify instance (usually an encapsulated plugin scope).
13
- * @param {any} options - `{ apiKey, bodyLimit, authorize, onUsage }`.
13
+ * @param {any} options - `{ apiKey, bodyLimit, authorize, onUsage, metrics, cors, rateLimit }`.
14
14
  * @param {any} captured - Captured translator parts `{ front, interval(body), filter, send, route, count, list, get, build, encode, write }`; `front` applies to this instance only.
15
15
  * @returns {void}
16
16
  */
@@ -28,9 +28,9 @@ export function resolve(...layers: (Partial<Settings> | null | undefined)[]): Se
28
28
  * @property {"expand"|"inline"|"native"} toolSearch - Claude Code's tool search. `expand` offers a deferred tool once a `tool_reference` (or a `ToolSearch` result) loads it; `inline` offers every deferred tool up front and hides `ToolSearch` (it stays when `DeferredToolPlaceholder` shows Claude Code holds tools whose definitions it did not send; `expand` does the same when the request has no `ToolSearch` at all); `native` passes `defer_loading` and `tool_reference` through to a server that understands them.
29
29
  * @property {"schema"|"instruction"|"drop"} jsonObject - OpenAI to Anthropic: how `response_format: json_object` is sent. `schema` sends `output_config.format` with the permissive schema `{ "type": "object" }`, `instruction` appends a short system instruction asking for one JSON object, `drop` omits it and records the drop.
30
30
  * @property {boolean} jsonObjectFallback - Retry a request once with `jsonObject: "instruction"` when an Anthropic-protocol upstream answers 400 or 422 to the schema `jsonObject: "schema"` sent. The endpoint and model are remembered until the process ends only when the retry succeeds.
31
- * @property {boolean} toolErrors - Prefix `is_error` tool results with `toolErrorMarker`. Off by default for every driver.
31
+ * @property {boolean} toolErrors - Prefix `is_error` tool results with `toolErrorMarker` when converting to Chat Completions (which has no error flag) and on vLLM native passthrough. On by default for every driver; set false to send the text unchanged. Upstreams with a native error flag (Anthropic-protocol) never go through this.
32
32
  * @property {string} toolErrorMarker - Text that marks a failed tool result when `toolErrors` is on, and the text `toolErrorInference` looks for. Must contain a non-whitespace character.
33
- * @property {boolean} toolErrorInference - OpenAI to Anthropic: a `tool` message whose text starts with `toolErrorMarker` becomes a `tool_result` with `is_error: true`, with the marker removed from the text.
33
+ * @property {boolean} toolErrorInference - OpenAI to Anthropic: a `tool` message whose text starts with `toolErrorMarker` becomes a `tool_result` with `is_error: true`, with the marker removed from the text. Off by default: a tool that merely prints the marker would be taken for a failure.
34
34
  * @property {"max_completion_tokens"|"max_tokens"} maxTokensField - Field used for the output cap.
35
35
  * @property {boolean} streamUsage - Request `stream_options.include_usage` on streamed requests.
36
36
  * @property {boolean} continuousUsage - Also request `stream_options.continuous_usage_stats` (vLLM extension; real OpenAI may reject it).
@@ -40,6 +40,9 @@ export function resolve(...layers: (Partial<Settings> | null | undefined)[]): Se
40
40
  * @property {boolean} parallelToolCalls - Map `disable_parallel_tool_use` to `parallel_tool_calls: false`.
41
41
  * @property {"drop"|"error"} unknownFields - Unknown request fields are dropped (and listed in `state.dropped`) or rejected.
42
42
  * @property {Record<string, any>} sampling - Sampling extension fields (`min_p`, `repetition_penalty`, `seed`, ...) added to requests for an OpenAI-protocol upstream when the request does not carry them, merged key by key. A request may set them itself (OpenAI front, or `metadata.sampling` on the Anthropic front). A field the upstream's driver does not accept follows `unknownFields`.
43
+ * @property {{ json?: object, regex?: string, choice?: string[], grammar?: string } | null} constraint - Neutral constrained-decoding description sent with every request to an OpenAI-protocol upstream that does not carry its own: exactly one of `json` (a JSON Schema), `regex`, `choice` (a list of strings) or `grammar` (GBNF for llama.cpp, EBNF for SGLang, a vLLM grammar). Each driver turns it into its backend's native field. Replaced whole, not merged. A request overrides it (`constraint` on the OpenAI front, `metadata.constraint` on the Anthropic front).
44
+ * @property {string | null} cacheSalt - vLLM `cache_salt` sent with every request to this upstream: requests with different salts never share cached prefixes. A non-empty string, or null for none. Backends without the feature follow `unknownFields`. A trusted caller overrides it per call (`cacheSalt` option, or the `cacheSalt` an `authorize` hook returns); it is never read from the client.
45
+ * @property {string | null} promptCacheKey - OpenAI `prompt_cache_key` sent with requests to this upstream that do not carry their own. A non-empty string, or null for none.
43
46
  * @property {number} pingInterval - Milliseconds of upstream silence before the SSE writer emits a `ping` (0 disables).
44
47
  * @property {"buffer"|"stream"} toolStreaming - OpenAI to Anthropic streams: `buffer` holds each tool call until its arguments are complete and emits one whole `tool_use` block (text and thinking keep streaming and appear before the block); `stream` forwards argument fragments incrementally and holds text that arrives meanwhile until the tool blocks are done.
45
48
  * @property {boolean} effortPassthrough - vLLM driver: send `xhigh` and `max` reasoning effort unchanged instead of clamping them to `high`.
@@ -141,7 +144,7 @@ export type Settings = {
141
144
  */
142
145
  jsonObjectFallback: boolean;
143
146
  /**
144
- * - Prefix `is_error` tool results with `toolErrorMarker`. Off by default for every driver.
147
+ * - Prefix `is_error` tool results with `toolErrorMarker` when converting to Chat Completions (which has no error flag) and on vLLM native passthrough. On by default for every driver; set false to send the text unchanged. Upstreams with a native error flag (Anthropic-protocol) never go through this.
145
148
  */
146
149
  toolErrors: boolean;
147
150
  /**
@@ -149,7 +152,7 @@ export type Settings = {
149
152
  */
150
153
  toolErrorMarker: string;
151
154
  /**
152
- * - OpenAI to Anthropic: a `tool` message whose text starts with `toolErrorMarker` becomes a `tool_result` with `is_error: true`, with the marker removed from the text.
155
+ * - OpenAI to Anthropic: a `tool` message whose text starts with `toolErrorMarker` becomes a `tool_result` with `is_error: true`, with the marker removed from the text. Off by default: a tool that merely prints the marker would be taken for a failure.
153
156
  */
154
157
  toolErrorInference: boolean;
155
158
  /**
@@ -188,6 +191,23 @@ export type Settings = {
188
191
  * - Sampling extension fields (`min_p`, `repetition_penalty`, `seed`, ...) added to requests for an OpenAI-protocol upstream when the request does not carry them, merged key by key. A request may set them itself (OpenAI front, or `metadata.sampling` on the Anthropic front). A field the upstream's driver does not accept follows `unknownFields`.
189
192
  */
190
193
  sampling: Record<string, any>;
194
+ /**
195
+ * - Neutral constrained-decoding description sent with every request to an OpenAI-protocol upstream that does not carry its own: exactly one of `json` (a JSON Schema), `regex`, `choice` (a list of strings) or `grammar` (GBNF for llama.cpp, EBNF for SGLang, a vLLM grammar). Each driver turns it into its backend's native field. Replaced whole, not merged. A request overrides it (`constraint` on the OpenAI front, `metadata.constraint` on the Anthropic front).
196
+ */
197
+ constraint: {
198
+ json?: object;
199
+ regex?: string;
200
+ choice?: string[];
201
+ grammar?: string;
202
+ } | null;
203
+ /**
204
+ * - vLLM `cache_salt` sent with every request to this upstream: requests with different salts never share cached prefixes. A non-empty string, or null for none. Backends without the feature follow `unknownFields`. A trusted caller overrides it per call (`cacheSalt` option, or the `cacheSalt` an `authorize` hook returns); it is never read from the client.
205
+ */
206
+ cacheSalt: string | null;
207
+ /**
208
+ * - OpenAI `prompt_cache_key` sent with requests to this upstream that do not carry their own. A non-empty string, or null for none.
209
+ */
210
+ promptCacheKey: string | null;
191
211
  /**
192
212
  * - Milliseconds of upstream silence before the SSE writer emits a `ping` (0 disables).
193
213
  */