@cldmv/llm-translator 0.5.0 → 0.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (36) hide show
  1. package/README.md +6 -6
  2. package/dist/api/convert/openai/chat/to/anthropic/messages/request.mjs +3 -3
  3. package/dist/api/driver/anthropic/info.mjs +1 -1
  4. package/dist/api/driver/compatible/info.mjs +1 -1
  5. package/dist/api/driver/deepseek/info.mjs +1 -1
  6. package/dist/api/driver/llamacpp/info.mjs +1 -1
  7. package/dist/api/driver/ollama/info.mjs +1 -1
  8. package/dist/api/driver/openai/info.mjs +1 -1
  9. package/dist/api/driver/sglang/info.mjs +1 -1
  10. package/dist/api/driver/vllm/info.mjs +1 -1
  11. package/dist/api/embeddings/send.mjs +2 -0
  12. package/dist/api/request.mjs +2 -2
  13. package/dist/api/send.mjs +1 -1
  14. package/dist/api/server/plugin.mjs +1 -1
  15. package/dist/lib/config.mjs +1 -1
  16. package/dist/lib/record.mjs +3 -0
  17. package/dist/lib/routes.mjs +1 -1
  18. package/dist/lib/sampling.mjs +2 -0
  19. package/dist/lib/settings.mjs +1 -1
  20. package/dist/lib/strict.mjs +2 -0
  21. package/package.json +1 -1
  22. package/types/src/api/driver/anthropic/info.d.mts +2 -1
  23. package/types/src/api/driver/compatible/info.d.mts +2 -1
  24. package/types/src/api/driver/deepseek/info.d.mts +2 -1
  25. package/types/src/api/driver/llamacpp/info.d.mts +2 -1
  26. package/types/src/api/driver/ollama/info.d.mts +2 -1
  27. package/types/src/api/driver/openai/info.d.mts +2 -1
  28. package/types/src/api/driver/sglang/info.d.mts +2 -1
  29. package/types/src/api/driver/vllm/info.d.mts +2 -1
  30. package/types/src/api/embeddings/send.d.mts +15 -0
  31. package/types/src/api/server/plugin.d.mts +1 -1
  32. package/types/src/index.d.mts +5 -0
  33. package/types/src/lib/record.d.mts +36 -0
  34. package/types/src/lib/sampling.d.mts +13 -0
  35. package/types/src/lib/settings.d.mts +5 -0
  36. package/types/src/lib/strict.d.mts +26 -0
package/README.md CHANGED
@@ -17,18 +17,18 @@ Use it as a library when you want the conversion without any I/O (`request`, `re
17
17
 
18
18
  ## ✨ What's New
19
19
 
20
- ### Latest: v0.5.0 (November 2026)
20
+ ### Latest: v0.6.0 (November 2026)
21
21
 
22
- - **Events and opt-in logging** — everything the library logs is now an event: `translator.events.on(name, fn)`, `"*"` and an `onEvent` config callback, with general `log`/`warning`/`error` events alongside specific ones such as `fallback` and `usage`. Logging is off by default as a library; `logs: true` prints to the console and `logs: "<path>"` writes JSON lines.
23
- - **Also:** `response_format: json_object` reaches Anthropic upstreams as a schema, falling back to a system instruction when an upstream refuses it; refusals carry `stop_details`.
24
- - [View full v0.5.0 Changelog](https://github.com/CLDMV/llm-translator/blob/master/docs/changelog/v0/v0.5.0.md)
22
+ - **Embeddings pass-through** — `POST /v1/embeddings` on the OpenAI front, and `api.embeddings.send()` as a library call, forward to an OpenAI-protocol upstream that lists the model. Aliases, pools, failover, `apiKey` and `authorize` work as they do for chat.
23
+ - **Sampling, strict tools and capture** — the `sampling` setting and `metadata.sampling` send `min_p`, `repetition_penalty` and similar fields to upstreams that accept them. `strict` now round trips: a tool whose schema is outside the strict subset is sent without `strict` and reported, and a backend that does not enforce strict use gets a warning. The `capture` option records redacted request and response pairs to a directory or a callback.
24
+ - [View full v0.6.0 Changelog](https://github.com/CLDMV/llm-translator/blob/master/docs/changelog/v0/v0.6.0.md)
25
25
 
26
26
  ### Recent Releases
27
27
 
28
+ - **v0.5.0** (November 2026) — everything the library logs is available as events, logging is opt-in through `logs`, `json_object` reaches Anthropic upstreams as a schema with a fallback, and refusals carry `stop_details` ([Changelog](https://github.com/CLDMV/llm-translator/blob/master/docs/changelog/v0/v0.5.0.md))
28
29
  - **v0.4.0** (November 2026) — per-model effort levels and aliases that pin effort and thinking, non-standard finish reasons, a configurable tool-error marker, per-upstream `pingInterval` and a Node.js 22.15 floor ([Changelog](https://github.com/CLDMV/llm-translator/blob/master/docs/changelog/v0/v0.4.0.md))
29
30
  - **v0.3.1** (November 2026) — patched the dev-only `minimatch` and `katex` lint dependencies and corrected the v0.3.0 issue links; no runtime changes ([Changelog](https://github.com/CLDMV/llm-translator/blob/master/docs/changelog/v0/v0.3.1.md))
30
31
  - **v0.3.0** (November 2026) — usage reporting from `send()`, the `route()` leaf and the server `authorize` hook; drivers for llama.cpp, Ollama, SGLang and the DeepSeek API, endpoint pools, `reasoningRoundTrip` and `toolSearch`; requires Node.js 24 ([Changelog](https://github.com/CLDMV/llm-translator/blob/master/docs/changelog/v0/v0.3.0.md))
31
- - **v0.2.0** (October 2026) — the first working translator: six Anthropic Messages and OpenAI Chat Completions converters, backend detection (vLLM, OpenAI, Anthropic, generic), model aliases, the optional Fastify server and the `llm-translator` CLI for Claude Code ([Changelog](https://github.com/CLDMV/llm-translator/blob/master/docs/changelog/v0/v0.2.0.md))
32
32
 
33
33
  📚 **For complete version history and detailed release notes, see the [docs/changelog/](https://github.com/CLDMV/llm-translator/tree/master/docs/changelog/) folder.**
34
34
 
@@ -127,7 +127,7 @@ Then point Claude Code at it with `ANTHROPIC_BASE_URL=http://127.0.0.1:8787`. Th
127
127
 
128
128
  ## 📚 Configuration
129
129
 
130
- `createTranslator(config)` takes `front`, `upstreams` (URLs or objects with `url`, `apiKey`, `backend`, `models`, `default` and `settings`), `aliases` (a model name, or `{ model, effort, thinking }` to pin settings), `efforts` (the reasoning levels each model supports, see [docs/settings.md](https://github.com/CLDMV/llm-translator/blob/master/docs/settings.md#effort-levels-and-pinned-aliases-not-converter-settings)), global `settings`, `pool` (failover and load balancing between upstreams that serve the same model, see [docs/server.md](https://github.com/CLDMV/llm-translator/blob/master/docs/server.md#endpoint-pools-failover-and-load-balancing)), `detect`, `logs` (off by default; `true` prints warnings, a path appends JSON lines), `events` and `onEvent` (see [docs/events.md](https://github.com/CLDMV/llm-translator/blob/master/docs/events.md)) and a custom `transport`. `silent: true` is a deprecated alias for `logs: false`. The CLI reads the same keys from flags, `LLM_TRANSLATOR_*` environment variables and a JSON file, in that order of precedence.
130
+ `createTranslator(config)` takes `front`, `upstreams` (URLs or objects with `url`, `apiKey`, `backend`, `models`, `default` and `settings`), `aliases` (a model name, or `{ model, effort, thinking }` to pin settings), `efforts` (the reasoning levels each model supports, see [docs/settings.md](https://github.com/CLDMV/llm-translator/blob/master/docs/settings.md#effort-levels-and-pinned-aliases-not-converter-settings)), global `settings`, `pool` (failover and load balancing between upstreams that serve the same model, see [docs/server.md](https://github.com/CLDMV/llm-translator/blob/master/docs/server.md#endpoint-pools-failover-and-load-balancing)), `detect`, `logs` (off by default; `true` prints warnings, a path appends JSON lines), `events` and `onEvent` (see [docs/events.md](https://github.com/CLDMV/llm-translator/blob/master/docs/events.md)), `capture` (off by default; a directory or a callback receives redacted request and response records for debugging, see [docs/events.md](https://github.com/CLDMV/llm-translator/blob/master/docs/events.md#capture-for-debugging)) and a custom `transport`. `silent: true` is a deprecated alias for `logs: false`. The CLI reads the same keys from flags, `LLM_TRANSLATOR_*` environment variables and a JSON file, in that order of precedence.
131
131
 
132
132
  Settings tune how the converters map one protocol onto the other (`inlineSystem`, `images`, `imageFallback`, `toolStreaming`, `pingInterval` and more); they are resolved per upstream from defaults, probed values, driver defaults and your overrides. The full table is in [docs/settings.md](https://github.com/CLDMV/llm-translator/blob/master/docs/settings.md).
133
133
 
@@ -1,8 +1,8 @@
1
- var v=Object.defineProperty;var f=(e,r)=>v(e,"name",{value:r,configurable:!0});import{resolve as N}from"../../../../../../../lib/settings.mjs";import{TranslatorError as b}from"../../../../../../../lib/errors.mjs";import{anthropicToolId as w,createNameFitter as O}from"../../../../../../../lib/ids.mjs";import{parseArguments as S}from"../../../../../../../lib/arguments.mjs";import{sanitize as q}from"../../../../../../../lib/schema.mjs";import{blocksOf as E}from"../../../../../../../lib/thinking.mjs";const I=new Set(["model","messages","max_tokens","max_completion_tokens","temperature","top_p","top_k","stop","stream","stream_options","tools","tool_choice","parallel_tool_calls","reasoning_effort","response_format","user","n"]),$=new Set(["metadata","store","service_tier"]),x=/^data:([^;,]+);base64,(.*)$/s;async function P(e,r){const i=N(r);if(e===null||typeof e!="object"||!Array.isArray(e.messages))throw new b("messages is a required property",{code:"invalid_request",param:"messages"});if(e.n!==void 0&&e.n!==1)throw new b("n: only a single choice (n = 1) is supported",{code:"unsupported_field",param:"n"});const o={front:"openai",model:e.model,stream:e.stream===!0,includeUsage:e.stream_options?.include_usage===!0,reasoningField:i.reasoningField,toolNames:{},dropped:[]},a=O({max:128,map:o.toolNames});for(const t of Object.keys(e))if(!(I.has(t)||$.has(t))){if(i.unknownFields==="error")throw new b(`${t}: Unrecognized request argument`,{code:"unsupported_field",param:t});g(o,t)}const c=[],p=[],m=f((t,n)=>{const l=p.at(-1);l?.role===t?l.blocks.push(...n):p.push({role:t,blocks:[...n]})},"push");for(const t of e.messages)switch(t.role){case"system":case"developer":{const n=j(t.content);n!==""&&c.push(n);break}case"user":m("user",F(t.content,o));break;case"assistant":{const n=i.reasoningRoundTrip?E(t.reasoning_details):[],l=[],u=j(t.content);u!==""?l.push({type:"text",text:u}):typeof t.refusal=="string"&&t.refusal!==""&&l.push({type:"text",text:t.refusal}),M(t)&&!n.some(d=>d.block.type==="thinking")&&g(o,"reasoning");for(const d of t.tool_calls??[])l.push({type:"tool_use",id:w(d.id),name:a(d.function.name),input:S(d.function.arguments)});const k=A(n,l);k.length>0&&m("assistant",k);break}case"tool":{const n={type:"tool_result",tool_use_id:w(t.tool_call_id),content:j(t.content)};if(i.toolErrorInference){const u=i.toolErrorMarker.trimEnd();n.content.startsWith(u)&&(n.is_error=!0,n.content=n.content.slice(u.length).replace(/^\s/,""))}const l=p.at(-1);if(l?.role==="user"){const u=l.blocks.findIndex(k=>k.type!=="tool_result");l.blocks.splice(u===-1?l.blocks.length:u,0,n)}else p.push({role:"user",blocks:[n]});break}default:throw new b(`messages: unsupported role "${t.role}"`,{code:"invalid_request",param:"messages"})}const s={model:e.model,max_tokens:e.max_completion_tokens??e.max_tokens??i.defaultMaxTokens};c.length>0&&(s.system=c.join(`
1
+ var v=Object.defineProperty;var f=(e,r)=>v(e,"name",{value:r,configurable:!0});import{resolve as N}from"../../../../../../../lib/settings.mjs";import{TranslatorError as b}from"../../../../../../../lib/errors.mjs";import{anthropicToolId as w,createNameFitter as O}from"../../../../../../../lib/ids.mjs";import{parseArguments as S}from"../../../../../../../lib/arguments.mjs";import{sanitize as q}from"../../../../../../../lib/schema.mjs";import{blocksOf as E}from"../../../../../../../lib/thinking.mjs";const I=new Set(["model","messages","max_tokens","max_completion_tokens","temperature","top_p","top_k","stop","stream","stream_options","tools","tool_choice","parallel_tool_calls","reasoning_effort","response_format","user","n"]),$=new Set(["metadata","store","service_tier"]),x=/^data:([^;,]+);base64,(.*)$/s;async function P(e,r){const i=N(r);if(e===null||typeof e!="object"||!Array.isArray(e.messages))throw new b("messages is a required property",{code:"invalid_request",param:"messages"});if(e.n!==void 0&&e.n!==1)throw new b("n: only a single choice (n = 1) is supported",{code:"unsupported_field",param:"n"});const o={front:"openai",model:e.model,stream:e.stream===!0,includeUsage:e.stream_options?.include_usage===!0,reasoningField:i.reasoningField,toolNames:{},dropped:[]},a=O({max:128,map:o.toolNames});for(const t of Object.keys(e))if(!(I.has(t)||$.has(t))){if(i.unknownFields==="error")throw new b(`${t}: Unrecognized request argument`,{code:"unsupported_field",param:t});g(o,t)}const c=[],p=[],m=f((t,s)=>{const l=p.at(-1);l?.role===t?l.blocks.push(...s):p.push({role:t,blocks:[...s]})},"push");for(const t of e.messages)switch(t.role){case"system":case"developer":{const s=j(t.content);s!==""&&c.push(s);break}case"user":m("user",F(t.content,o));break;case"assistant":{const s=i.reasoningRoundTrip?E(t.reasoning_details):[],l=[],u=j(t.content);u!==""?l.push({type:"text",text:u}):typeof t.refusal=="string"&&t.refusal!==""&&l.push({type:"text",text:t.refusal}),M(t)&&!s.some(d=>d.block.type==="thinking")&&g(o,"reasoning");for(const d of t.tool_calls??[])l.push({type:"tool_use",id:w(d.id),name:a(d.function.name),input:S(d.function.arguments)});const k=A(s,l);k.length>0&&m("assistant",k);break}case"tool":{const s={type:"tool_result",tool_use_id:w(t.tool_call_id),content:j(t.content)};if(i.toolErrorInference){const u=i.toolErrorMarker.trimEnd();s.content.startsWith(u)&&(s.is_error=!0,s.content=s.content.slice(u.length).replace(/^\s/,""))}const l=p.at(-1);if(l?.role==="user"){const u=l.blocks.findIndex(k=>k.type!=="tool_result");l.blocks.splice(u===-1?l.blocks.length:u,0,s)}else p.push({role:"user",blocks:[s]});break}default:throw new b(`messages: unsupported role "${t.role}"`,{code:"invalid_request",param:"messages"})}const n={model:e.model,max_tokens:e.max_completion_tokens??e.max_tokens??i.defaultMaxTokens};c.length>0&&(n.system=c.join(`
2
2
 
3
- `)),s.messages=p.map(t=>({role:t.role,content:t.blocks})),e.temperature!==void 0&&(s.temperature=Math.min(1,e.temperature)),e.top_p!==void 0&&(s.top_p=e.top_p),e.top_k!==void 0&&(s.top_k=e.top_k),e.stop!==void 0&&e.stop!==null&&(s.stop_sequences=[].concat(e.stop)),o.stream&&(s.stream=!0),typeof e.user=="string"&&(s.metadata={user_id:e.user});const y=[];for(const t of e.tools??[]){if(t.type!=="function"){g(o,`tool:${t.type}`);continue}const n=t.function,l=n.parameters??{type:"object",properties:{}},u={name:a(n.name),input_schema:i.toolSchema==="sanitize"?q(l):l};n.description!==void 0&&(u.description=n.description),y.push(u)}if(y.length>0){s.tools=y;let t=R(e.tool_choice,a);e.parallel_tool_calls===!1&&t.type!=="none"&&(t={...t,disable_parallel_tool_use:!0}),(e.tool_choice!==void 0||e.parallel_tool_calls===!1)&&(s.tool_choice=t)}const _=e.reasoning_effort;_==="none"?s.thinking={type:"disabled"}:_!=null&&(s.thinking={type:"adaptive"},s.output_config={effort:_==="minimal"?"low":_});const h=e.response_format;return h?.type==="json_schema"?s.output_config={...s.output_config,format:{type:"json_schema",schema:h.json_schema.schema}}:h?.type==="json_object"&&(i.jsonObject==="schema"?(s.output_config={...s.output_config,format:{type:"json_schema",schema:{type:"object"}}},o.jsonSchemaEmulated=!0):i.jsonObject==="instruction"?s.system=s.system===void 0?T:`${s.system}
3
+ `)),n.messages=p.map(t=>({role:t.role,content:t.blocks})),e.temperature!==void 0&&(n.temperature=Math.min(1,e.temperature)),e.top_p!==void 0&&(n.top_p=e.top_p),e.top_k!==void 0&&(n.top_k=e.top_k),e.stop!==void 0&&e.stop!==null&&(n.stop_sequences=[].concat(e.stop)),o.stream&&(n.stream=!0),typeof e.user=="string"&&(n.metadata={user_id:e.user});const y=[];for(const t of e.tools??[]){if(t.type!=="function"){g(o,`tool:${t.type}`);continue}const s=t.function,l=s.parameters??{type:"object",properties:{}},u={name:a(s.name),input_schema:i.toolSchema==="sanitize"?q(l):l};s.description!==void 0&&(u.description=s.description),s.strict!==void 0&&(u.strict=s.strict),y.push(u)}if(y.length>0){n.tools=y;let t=R(e.tool_choice,a);e.parallel_tool_calls===!1&&t.type!=="none"&&(t={...t,disable_parallel_tool_use:!0}),(e.tool_choice!==void 0||e.parallel_tool_calls===!1)&&(n.tool_choice=t)}const _=e.reasoning_effort;_==="none"?n.thinking={type:"disabled"}:_!=null&&(n.thinking={type:"adaptive"},n.output_config={effort:_==="minimal"?"low":_});const h=e.response_format;return h?.type==="json_schema"?n.output_config={...n.output_config,format:{type:"json_schema",schema:h.json_schema.schema}}:h?.type==="json_object"&&(i.jsonObject==="schema"?(n.output_config={...n.output_config,format:{type:"json_schema",schema:{type:"object"}}},o.jsonSchemaEmulated=!0):i.jsonObject==="instruction"?n.system=n.system===void 0?T:`${n.system}
4
4
 
5
- ${T}`:g(o,"response_format")),{body:s,state:o}}f(P,"request");const T="Respond with a single valid JSON object and nothing else.";function g(e,r){e.dropped.includes(r)||e.dropped.push(r)}f(g,"drop");function A(e,r){const i=[...e].sort((p,m)=>p.at-m.at),o=[];let a=0;const c=r.findIndex(p=>p.type==="tool_use");for(const p of i){const m=p.at<=1&&c!==-1?Math.min(p.at,c):p.at;for(;a<Math.min(m,r.length);)o.push(r[a++]);o.push(p.block)}return[...o,...r.slice(a)]}f(A,"interleave");function M(e){return!!(e.reasoning||e.reasoning_content)}f(M,"reasoningPresent");function j(e){return typeof e=="string"?e:(e??[]).filter(r=>r.type==="text").map(r=>r.text).join(`
5
+ ${T}`:g(o,"response_format")),{body:n,state:o}}f(P,"request");const T="Respond with a single valid JSON object and nothing else.";function g(e,r){e.dropped.includes(r)||e.dropped.push(r)}f(g,"drop");function A(e,r){const i=[...e].sort((p,m)=>p.at-m.at),o=[];let a=0;const c=r.findIndex(p=>p.type==="tool_use");for(const p of i){const m=p.at<=1&&c!==-1?Math.min(p.at,c):p.at;for(;a<Math.min(m,r.length);)o.push(r[a++]);o.push(p.block)}return[...o,...r.slice(a)]}f(A,"interleave");function M(e){return!!(e.reasoning||e.reasoning_content)}f(M,"reasoningPresent");function j(e){return typeof e=="string"?e:(e??[]).filter(r=>r.type==="text").map(r=>r.text).join(`
6
6
 
7
7
  `)}f(j,"plainText");function F(e,r){if(typeof e=="string")return e===""?[]:[{type:"text",text:e}];const i=[];for(const o of e??[])if(o.type==="text")i.push({type:"text",text:o.text});else if(o.type==="image_url"){const a=x.exec(o.image_url.url);i.push({type:"image",source:a?{type:"base64",media_type:a[1],data:a[2]}:{type:"url",url:o.image_url.url}})}else if(o.type==="file"&&typeof o.file?.file_data=="string"&&x.test(o.file.file_data)){const a=x.exec(o.file.file_data);i.push({type:"document",source:{type:"base64",media_type:a[1],data:a[2]}})}else g(r,`content:${o.type}`);return i}f(F,"userBlocks");function R(e,r){return e==="none"?{type:"none"}:e==="required"?{type:"any"}:typeof e=="object"&&e!==null?{type:"tool",name:r(e.function.name)}:{type:"auto"}}f(R,"toolChoice");export{P as request};
8
8
  //# sourceMappingURL=request.mjs.map
@@ -1,2 +1,2 @@
1
- var a=Object.defineProperty;var e=(s,o)=>a(s,"name",{value:o,configurable:!0});function t(){return{name:"anthropic",protocol:"anthropic.messages",order:20,auth:"x-api-key",headers:{"anthropic-version":"2023-06-01"},paths:{messages:"/v1/messages",models:"/v1/models",count:"/v1/messages/count_tokens"}}}e(t,"info");export{t as info};
1
+ var t=Object.defineProperty;var e=(s,o)=>t(s,"name",{value:o,configurable:!0});function a(){return{name:"anthropic",protocol:"anthropic.messages",order:20,auth:"x-api-key",strict:!0,headers:{"anthropic-version":"2023-06-01"},paths:{messages:"/v1/messages",models:"/v1/models",count:"/v1/messages/count_tokens"}}}e(a,"info");export{a as info};
2
2
  //# sourceMappingURL=info.mjs.map
@@ -1,2 +1,2 @@
1
- var r=Object.defineProperty;var o=(e,t)=>r(e,"name",{value:t,configurable:!0});import{VLLM_FIELDS as a}from"../../../lib/passthrough.mjs";function m(){return{name:"compatible",protocol:"openai.chat",order:100,auth:"bearer",fields:a,paths:{chat:"/v1/chat/completions",models:"/v1/models"}}}o(m,"info");export{m as info};
1
+ var r=Object.defineProperty;var o=(t,e)=>r(t,"name",{value:e,configurable:!0});import{VLLM_FIELDS as a}from"../../../lib/passthrough.mjs";function l(){return{name:"compatible",protocol:"openai.chat",order:100,auth:"bearer",strict:!1,fields:a,paths:{chat:"/v1/chat/completions",models:"/v1/models"}}}o(l,"info");export{l as info};
2
2
  //# sourceMappingURL=info.mjs.map
@@ -1,2 +1,2 @@
1
- var a=Object.defineProperty;var e=(o,t)=>a(o,"name",{value:t,configurable:!0});function n(){return{name:"deepseek",protocol:"openai.chat",order:35,auth:"bearer",paths:{chat:"/v1/chat/completions",models:"/v1/models"}}}e(n,"info");export{n as info};
1
+ var a=Object.defineProperty;var e=(o,t)=>a(o,"name",{value:t,configurable:!0});function c(){return{name:"deepseek",protocol:"openai.chat",order:35,auth:"bearer",strict:!1,paths:{chat:"/v1/chat/completions",models:"/v1/models"}}}e(c,"info");export{c as info};
2
2
  //# sourceMappingURL=info.mjs.map
@@ -1,2 +1,2 @@
1
- var a=Object.defineProperty;var p=(e,t)=>a(e,"name",{value:t,configurable:!0});function r(){return{name:"llamacpp",protocol:"openai.chat",order:40,auth:"bearer",probes:["/props"],fields:["top_k","min_p","chat_template_kwargs","repetition_penalty","repeat_penalty","grammar"],paths:{chat:"/v1/chat/completions",models:"/v1/models",props:"/props"}}}p(r,"info");export{r as info};
1
+ var a=Object.defineProperty;var e=(p,t)=>a(p,"name",{value:t,configurable:!0});function r(){return{name:"llamacpp",protocol:"openai.chat",order:40,auth:"bearer",strict:!1,probes:["/props"],fields:["top_k","min_p","chat_template_kwargs","repetition_penalty","repeat_penalty","grammar"],paths:{chat:"/v1/chat/completions",models:"/v1/models",props:"/props"}}}e(r,"info");export{r as info};
2
2
  //# sourceMappingURL=info.mjs.map
@@ -1,2 +1,2 @@
1
- var r=Object.defineProperty;var o=(e,a)=>r(e,"name",{value:a,configurable:!0});function t(){return{name:"ollama",protocol:"openai.chat",order:60,auth:"bearer",probes:["/api/version"],paths:{chat:"/v1/chat/completions",models:"/v1/models",version:"/api/version"}}}o(t,"info");export{t as info};
1
+ var r=Object.defineProperty;var o=(e,a)=>r(e,"name",{value:a,configurable:!0});function i(){return{name:"ollama",protocol:"openai.chat",order:60,auth:"bearer",strict:!1,probes:["/api/version"],paths:{chat:"/v1/chat/completions",models:"/v1/models",version:"/api/version"}}}o(i,"info");export{i as info};
2
2
  //# sourceMappingURL=info.mjs.map
@@ -1,2 +1,2 @@
1
- var a=Object.defineProperty;var o=(e,t)=>a(e,"name",{value:t,configurable:!0});function r(){return{name:"openai",protocol:"openai.chat",order:30,auth:"bearer",paths:{chat:"/v1/chat/completions",models:"/v1/models"}}}o(r,"info");export{r as info};
1
+ var r=Object.defineProperty;var o=(t,e)=>r(t,"name",{value:e,configurable:!0});function n(){return{name:"openai",protocol:"openai.chat",order:30,auth:"bearer",strict:!0,paths:{chat:"/v1/chat/completions",models:"/v1/models"}}}o(n,"info");export{n as info};
2
2
  //# sourceMappingURL=info.mjs.map
@@ -1,2 +1,2 @@
1
- var t=Object.defineProperty;var e=(o,r)=>t(o,"name",{value:r,configurable:!0});function a(){return{name:"sglang",protocol:"openai.chat",order:50,auth:"bearer",probes:["/get_server_info"],fields:["top_k","min_p","repetition_penalty","chat_template_kwargs","skip_special_tokens","ignore_eos","regex","ebnf","separate_reasoning"],paths:{chat:"/v1/chat/completions",models:"/v1/models",serverInfo:"/get_server_info"}}}e(a,"info");export{a as info};
1
+ var r=Object.defineProperty;var e=(t,o)=>r(t,"name",{value:o,configurable:!0});function s(){return{name:"sglang",protocol:"openai.chat",order:50,auth:"bearer",strict:!1,probes:["/get_server_info"],fields:["top_k","min_p","repetition_penalty","chat_template_kwargs","skip_special_tokens","ignore_eos","regex","ebnf","separate_reasoning"],paths:{chat:"/v1/chat/completions",models:"/v1/models",serverInfo:"/get_server_info"}}}e(s,"info");export{s as info};
2
2
  //# sourceMappingURL=info.mjs.map
@@ -1,2 +1,2 @@
1
- var r=Object.defineProperty;var o=(e,t)=>r(e,"name",{value:t,configurable:!0});import{VLLM_FIELDS as n}from"../../../lib/passthrough.mjs";function l(){return{name:"vllm",protocol:"openai.chat",order:10,auth:"bearer",fields:n,paths:{chat:"/v1/chat/completions",models:"/v1/models",tokenize:"/tokenize",version:"/version"}}}o(l,"info");export{l as info};
1
+ var r=Object.defineProperty;var e=(o,t)=>r(o,"name",{value:t,configurable:!0});import{VLLM_FIELDS as i}from"../../../lib/passthrough.mjs";function s(){return{name:"vllm",protocol:"openai.chat",order:10,auth:"bearer",strict:!0,fields:i,paths:{chat:"/v1/chat/completions",models:"/v1/models",tokenize:"/tokenize",version:"/version"}}}e(s,"info");export{s as info};
2
2
  //# sourceMappingURL=info.mjs.map
@@ -0,0 +1,2 @@
1
+ var _=Object.defineProperty;var f=(e,s)=>_(e,"name",{value:s,configurable:!0});import{context as g,self as b}from"../../lib/runtime.mjs";import{capture as $}from"../../lib/capture.mjs";import{resolve as w,serves as A}from"../../lib/route.mjs";import*as h from"../../lib/http.mjs";import*as v from"../../lib/pool.mjs";import{emit as O}from"../../lib/events.mjs";const P="/v1/embeddings";async function J(e,s={}){const{transport:y}=g.state,{error:m}=b,a=$(b,g,"openai"),n=f((r,t,l={})=>m.build(r,t,{...l,front:"openai"}),"fail");if(!e||typeof e!="object"||Array.isArray(e))return n(400,"The request body must be a JSON object",{code:"invalid_body"});if(typeof e.model!="string"||e.model==="")return n(400,"model is required",{code:"invalid_body",param:"model"});const T=Object.hasOwn(a.config.aliases,e.model)?a.config.aliases[e.model]:e.model;if(!a.state.endpoints.some(r=>A(r,T)))return n(404,`The model "${e.model}" is not served by any configured upstream.`,{code:"model_not_found",param:"model"});const u=w(a,e,{advance:!0}),c=u.pool.filter(r=>r.protocol==="openai.chat");if(c.length===0)return n(400,`The model "${e.model}" is served by an Anthropic-protocol upstream, which has no embeddings API.`,{code:"embeddings_unsupported",param:"model"});const j={...e,model:u.model};let i;for(let r=0;r<c.length;r++){const t=c[r],l=a.driver(t);t.health.inflight++,t.health.requests++;let o;try{o=await h.json(y,h.join(t.url,P),{method:"POST",headers:{"content-type":"application/json",accept:"application/json",...l.info.headers,...h.headers(t,l.info.auth,s.headers)},body:j,signal:s.signal})}finally{t.health.inflight--}if(s.signal?.aborted)return n(499,"The request was aborted",{code:"aborted"});const d=o.status===0;if(!d&&o.status<500)return v.succeeded(t),o.ok?o.body===null||typeof o.body!="object"?n(502,"The upstream returned a body that is not JSON",{code:"bad_gateway"}):{status:200,headers:{"content-type":"application/json"},body:o.body}:m(o.status,o.body,{},{front:"openai",upstream:t.name,native:!0,model:e.model});const p=d?`Could not reach the upstream (${t.name}): ${o.error?.message}`:`HTTP ${o.status}`;v.failed(a.config,t,p),O(a,{type:"error",kind:d?"transport":"upstream",level:"error",message:d?p:`Upstream ${t.name} answered HTTP ${o.status}`,endpoint:t.name,model:u.model,log:s.log}),i={endpoint:t,result:o,reason:p}}return i.result.status===0?n(502,i.reason,{code:"upstream_unreachable"}):m(i.result.status,i.result.body,{},{front:"openai",upstream:i.endpoint.name,native:!0,model:e.model})}f(J,"send");export{J as send};
2
+ //# sourceMappingURL=send.mjs.map
@@ -1,3 +1,3 @@
1
- var _=Object.defineProperty;var p=(e,o)=>_(e,"name",{value:o,configurable:!0});import{context as q,self as T}from"../lib/runtime.mjs";import{capture as I}from"../lib/capture.mjs";import{resolve as L}from"../lib/route.mjs";import*as w from"../lib/http.mjs";import{filterChatBody as B}from"../lib/passthrough.mjs";import*as $ from"../lib/images.mjs";import*as y from"../lib/tools.mjs";import{LIMITS as M}from"../lib/toolsearch.mjs";import{warn as h}from"../lib/warn.mjs";import{emit as P}from"../lib/events.mjs";import{fit as R,pinBody as C}from"../lib/effort.mjs";function E(e){return`${e.driver}:${e.protocol}:${URL.canParse(e.url)?new URL(e.url).host:e.url}`}p(E,"producerOf");async function V(e,o={}){const r=I(T,q,o.front),c=L(r,e,{endpoint:o.endpoint,advance:o.hold===!0&&o.endpoint===void 0});if(o.hold!==!0)return O(r,e,o,c);c.endpoint.health.inflight++;try{return await O(r,e,o,c)}catch(t){throw c.endpoint.health.inflight--,t}}p(V,"request");async function O(e,o,r,c){const{endpoint:t,model:s,unserved:F,convert:m,pool:U,pin:g}=c,d=Object.hasOwn(e.config.efforts,s)?e.config.efforts[s]:void 0;g&&(o=C(o,g,e.front));const i=e.driver(t),u=`${t.name}
2
- ${s}`,k=r.images===!1||e.state.textOnly.has(u)&&e.settings(t).imageFallback,v=e.settings(t),x=r.jsonObject==="instruction"||e.state.jsonInstruction.has(u)&&v.jsonObjectFallback,l={...k?{...v,images:!1}:v,...x?{jsonObject:"instruction"}:{},...g?.effort===void 0?{}:{effort:!0}},j=o.stream===!0;let n,a;if(m?({body:n,state:a}=await m.request(o,l,{producer:E(t),budget:r.budget,listed:d!==void 0}),a.toolSearch?.capped&&h(e.config,e.state,`Tool search: more than ${M.definitions} tool definitions came back in ToolSearch results for ${t.name}; the rest are not offered.`,{kind:"toolsearch",endpoint:t.name,log:r.log})):n=e.front==="openai"&&(r.filter??l.passthroughFilter===!0)?B(o,i.info.fields):{...o},r.usage===!0&&j&&e.front==="openai"){if(m)a.includeUsage=!0;else if(l.streamUsage){const f=n.stream_options&&typeof n.stream_options=="object"?n.stream_options:{};n.stream_options={...f,include_usage:!0}}}k&&!m&&e.front==="openai"&&(n=$.strip(n)),r.tools===!1&&(n=y.strip(n)),n.model=s,d&&t.protocol==="openai.chat"&&R(n,d,s,(f,S)=>h(e.config,e.state,f,{type:"effort",endpoint:t.name,model:s,...S,log:r.log})),a?.reasoningMissing&&h(e.config,e.state,`Upstream ${t.name}: assistant turns with tool calls have no reasoning to send back (the client dropped its thinking blocks or their signatures), which servers that need it with tools will refuse.`,{kind:"reasoning",endpoint:t.name,log:r.log}),i.request&&(n=await i.request(n,{endpoint:t,settings:l,levels:d,state:a,warn:p(f=>h(e.config,e.state,f,{kind:"driver",endpoint:t.name,log:r.log}),"warn")})),a?.dropped?.length>0&&r.retry!==!0&&P(e,{type:"dropped",level:"debug",message:`Upstream ${t.name}: dropped ${a.dropped.join(", ")}`,endpoint:t.name,model:s,fields:[...a.dropped],log:r.log});const b=t.protocol==="openai.chat"?i.info.paths.chat:i.info.paths.messages;return{upstream:t.name,pool:U.map(f=>f.name),method:"POST",url:w.join(t.url,b),path:b,headers:{"content-type":"application/json",accept:j?"text/event-stream":"application/json",...i.info.headers,...m===null&&t.protocol==="anthropic.messages"?r.clientHeaders:void 0,...w.headers(t,i.info.auth,r.headers)},body:n,state:{v:1,upstream:t.name,front:e.front,native:m===null,model:o.model,stream:j,unserved:F,convert:a},images:{sent:$.carries(n),retry:l.imageFallback&&t.protocol==="openai.chat",key:u},tools:{sent:y.carries(n),retry:l.toolFallback&&t.protocol==="openai.chat"},json:{sent:a?.jsonSchemaEmulated===!0,retry:l.jsonObjectFallback&&t.protocol==="anthropic.messages",key:u}}}p(O,"prepare");export{V as request};
1
+ var T=Object.defineProperty;var p=(e,n)=>T(e,"name",{value:n,configurable:!0});import{context as q,self as I}from"../lib/runtime.mjs";import{capture as L}from"../lib/capture.mjs";import{resolve as B}from"../lib/route.mjs";import*as U from"../lib/http.mjs";import{filterChatBody as E}from"../lib/passthrough.mjs";import{addSampling as M}from"../lib/sampling.mjs";import*as F from"../lib/images.mjs";import*as S from"../lib/tools.mjs";import{LIMITS as P}from"../lib/toolsearch.mjs";import{warn as b}from"../lib/warn.mjs";import{emit as R}from"../lib/events.mjs";import{fit as C,pinBody as H}from"../lib/effort.mjs";import{relax as z,strictTools as A}from"../lib/strict.mjs";import{TranslatorError as D}from"../lib/errors.mjs";function G(e){return`${e.driver}:${e.protocol}:${URL.canParse(e.url)?new URL(e.url).host:e.url}`}p(G,"producerOf");async function ne(e,n={}){const r=L(I,q,n.front),c=B(r,e,{endpoint:n.endpoint,advance:n.hold===!0&&n.endpoint===void 0});if(n.hold!==!0)return _(r,e,n,c);c.endpoint.health.inflight++;try{return await _(r,e,n,c)}catch(t){throw c.endpoint.health.inflight--,t}}p(ne,"request");function J(e,n,r,c,t,f){const g=A(n);if(g.length===0)return n;const i=g.filter(a=>a.problems.length>0);if(i.length>0&&t.unknownFields==="error")throw new D(`tools: ${i[0].name}: schema is outside the strict subset (${i[0].problems.join("; ")})`,{code:"unsupported_content",param:"tools"});const k=z(n);if(f.retry===!0)return k;const u=p((a,h)=>b(e.config,e.state,a,{kind:h,endpoint:r.name,log:f.log}),"tell");for(const a of i)u(`Upstream ${r.name}: strict tool ${a.name} has a schema outside the strict subset: ${a.problems.join("; ")} (sent without strict)`,"strict");const d=g.filter(a=>a.problems.length===0);return c.info.strict!==!0&&d.length>0&&u(`Upstream ${r.name}: the ${c.name} backend does not enforce strict tool use (${d.map(a=>a.name).join(", ")})`,"strict"),k}p(J,"checkStrict");async function _(e,n,r,c){const{endpoint:t,model:f,unserved:g,convert:i,pool:k,pin:u}=c,d=Object.hasOwn(e.config.efforts,f)?e.config.efforts[f]:void 0;u&&(n=H(n,u,e.front));const a=e.driver(t),h=`${t.name}
2
+ ${f}`,w=r.images===!1||e.state.textOnly.has(h)&&e.settings(t).imageFallback,$=e.settings(t),x=r.jsonObject==="instruction"||e.state.jsonInstruction.has(h)&&$.jsonObjectFallback,m={...w?{...$,images:!1}:$,...x?{jsonObject:"instruction"}:{},...u?.effort===void 0?{}:{effort:!0}},v=n.stream===!0;let o,s;i?({body:o,state:s}=await i.request(n,m,{producer:G(t),budget:r.budget,listed:d!==void 0}),s.toolSearch?.capped&&b(e.config,e.state,`Tool search: more than ${P.definitions} tool definitions came back in ToolSearch results for ${t.name}; the rest are not offered.`,{kind:"toolsearch",endpoint:t.name,log:r.log})):o=e.front==="openai"&&(r.filter??m.passthroughFilter===!0)?E(n,a.info.fields):{...n};const y=e.front==="anthropic"?n.metadata?.sampling:void 0;if(t.protocol==="openai.chat"){const l=[];o=M(o,y,m,a.info.fields,l),s&&s.dropped.push(...l.filter(j=>!s.dropped.includes(j)))}else if(!i&&y!==void 0&&o.metadata?.sampling!==void 0){const{sampling:l,...j}=o.metadata;o={...o,metadata:j}}if(r.usage===!0&&v&&e.front==="openai"){if(i)s.includeUsage=!0;else if(m.streamUsage){const l=o.stream_options&&typeof o.stream_options=="object"?o.stream_options:{};o.stream_options={...l,include_usage:!0}}}w&&!i&&e.front==="openai"&&(o=F.strip(o)),r.tools===!1&&(o=S.strip(o)),o.model=f,o=J(e,o,t,a,m,r),d&&t.protocol==="openai.chat"&&C(o,d,f,(l,j)=>b(e.config,e.state,l,{type:"effort",endpoint:t.name,model:f,...j,log:r.log})),s?.reasoningMissing&&b(e.config,e.state,`Upstream ${t.name}: assistant turns with tool calls have no reasoning to send back (the client dropped its thinking blocks or their signatures), which servers that need it with tools will refuse.`,{kind:"reasoning",endpoint:t.name,log:r.log}),a.request&&(o=await a.request(o,{endpoint:t,settings:m,levels:d,state:s,warn:p(l=>b(e.config,e.state,l,{kind:"driver",endpoint:t.name,log:r.log}),"warn")})),s?.dropped?.length>0&&r.retry!==!0&&R(e,{type:"dropped",level:"debug",message:`Upstream ${t.name}: dropped ${s.dropped.join(", ")}`,endpoint:t.name,model:f,fields:[...s.dropped],log:r.log});const O=t.protocol==="openai.chat"?a.info.paths.chat:a.info.paths.messages;return{upstream:t.name,pool:k.map(l=>l.name),method:"POST",url:U.join(t.url,O),path:O,headers:{"content-type":"application/json",accept:v?"text/event-stream":"application/json",...a.info.headers,...i===null&&t.protocol==="anthropic.messages"?r.clientHeaders:void 0,...U.headers(t,a.info.auth,r.headers)},body:o,state:{v:1,upstream:t.name,front:e.front,native:i===null,model:n.model,stream:v,unserved:g,convert:s},images:{sent:F.carries(o),retry:m.imageFallback&&t.protocol==="openai.chat",key:h},tools:{sent:S.carries(o),retry:m.toolFallback&&t.protocol==="openai.chat"},json:{sent:s?.jsonSchemaEmulated===!0,retry:m.jsonObjectFallback&&t.protocol==="anthropic.messages",key:h}}}p(_,"prepare");export{ne as request};
3
3
  //# sourceMappingURL=request.mjs.map
package/dist/api/send.mjs CHANGED
@@ -1,2 +1,2 @@
1
- var Z=Object.defineProperty;var l=(u,r)=>Z(u,"name",{value:r,configurable:!0});import{context as L,self as K}from"../lib/runtime.mjs";import{TranslatorError as J}from"../lib/errors.mjs";import*as tt from"../lib/http.mjs";import*as et from"../lib/images.mjs";import*as at from"../lib/tools.mjs";import{parse as nt}from"../lib/upstream-error.mjs";import{meter as rt,normalize as st}from"../lib/usage.mjs";import{capture as ot}from"../lib/capture.mjs";import{estimate as it,bodyLength as lt,output as ut,patch as ct,stream as mt}from"../lib/estimate.mjs";import{warn as dt}from"../lib/warn.mjs";import{emit as Q}from"../lib/events.mjs";import*as V from"../lib/pool.mjs";import{createBudget as pt}from"../lib/thinking.mjs";const ft={"content-type":"text/event-stream; charset=utf-8","cache-control":"no-cache","x-accel-buffering":"no"};async function St(u,r={}){const{transport:y,textOnly:d,jsonInstruction:k}=L.state,{request:j,response:_,stream:o,error:g}=K,p=g.build,{config:i,state:v}=L,h=ot(K,L,r.front),S=u&&typeof u=="object"&&!Array.isArray(u),x={model:null,requested:S&&typeof u.model=="string"?u.model:null,upstream:null,front:r.front??i.front,stream:S&&u.stream===!0},b=rt(x,r.onUsage,(c,E)=>dt(i,v,c,{type:"usage",...E,log:r.log})),O=l(c=>Object.defineProperty(c,"usage",{value:b.usage,enumerable:!1,configurable:!0}),"attach");try{return O(await gt({transport:y,textOnly:d,jsonInstruction:k,request:j,response:_,stream:o,error:g,captured:h},u,r,b))}catch(c){if(!(c instanceof J))throw r.signal?.aborted||Q(h,{type:"error",kind:"exception",level:"error",message:String(c?.message??c),log:r.log}),b.finish(r.signal?.aborted?"aborted":"error"),c;return b.finish("error"),O(p(c.status,c.message,{type:c.type,code:c.code,param:c.param,front:r.front}))}}l(St,"send");async function gt({transport:u,textOnly:r,jsonInstruction:y,request:d,response:k,stream:j,error:_,captured:o},g,p,i){const v={front:p.front,headers:p.headers,clientHeaders:p.clientHeaders,filter:p.filter,usage:!0,budget:pt(),log:p.log},h=l(e=>{Q(o,{...e,log:p.log})},"announce"),S=i.meta.front==="openai"&&g?.stream_options?.include_usage!==!0,x=l(e=>{i.meta.model=e.body.model??null,i.meta.upstream=e.upstream,e.state.native?i.source(null):i.source(()=>e.state.convert?.upstreamUsage,e.state.front==="anthropic"?"openai":"folded")},"note"),b=l(e=>{if(e.state.native)return{input:!1,output:!1};const t=st(e.state.front==="anthropic"?"openai":"folded",e.state.convert?.upstreamUsage);return t===null?{input:!0,output:!0}:{input:t.input_tokens===null&&t.cache_read_input_tokens===null&&t.cache_creation_input_tokens===null,output:t.output_tokens===null}},"gaps"),O=l(async e=>{const t=o.endpoint(e.upstream),a=o.driver(t);return(a.count&&!(S&&e.state.stream)?await a.count({endpoint:t,request:e.body,transport:u,signal:p.signal,timeout:o.config.detectTimeout}):null)??it(e.body)},"input"),c=l(async(e,t)=>{const a=b(e);return!a.input&&!a.output?null:{input_tokens:a.input?await O(e):null,output_tokens:a.output?ut(t):null}},"counts"),E=l((e,t,a,n,B)=>(i.finish(B),_(e,t,a,n)),"fail"),X=l(async e=>{let t=e,a=await W(u,t,p.signal,R());const n=a.status,B=n===400&&[t.images,t.tools].some(T=>T.sent&&T.retry),D=(n===400||n===422)&&t.json.sent&&t.json.retry;if(!a.ok&&(B||D)){const T=await M(a),$=nt(T,n).message,f={};if(n===400&&t.images.sent&&t.images.retry&&et.refused($)?(r.add(t.images.key),f.images=!1):n===400&&t.tools.sent&&t.tools.retry&&at.refused($)?f.tools=!1:D&&(f.jsonObject="instruction"),!("images"in f||"tools"in f||"jsonObject"in f))return{upstream:t,result:a,failure:T};const w={upstream:t,result:a,failure:T};if("images"in f||"tools"in f){const I="images"in f?"images":"tools";h({type:"fallback",kind:I,level:"warn",message:`Upstream ${t.upstream}: ${t.state.model} refused ${I==="images"?"images":"tools"} (${n}: "${String($).trim().slice(0,500)}"); sent the request again without them.`,endpoint:t.upstream,model:t.body.model,status:n,reason:String($).trim().slice(0,500),action:I==="images"?"text-only":"without-tools"})}if(t=await d(g,{...v,...f,endpoint:t.upstream,retry:!0}),x(t),a=await W(u,t,p.signal,R()),D&&"jsonObject"in f){if(!a.ok){try{await a.body?.cancel?.()}catch{}return x(w.upstream),w}y.add(w.upstream.json.key),h({type:"fallback",kind:"json_object",level:"warn",message:`Upstream ${w.upstream.upstream}: ${w.upstream.state.model} rejected the json_object schema emulation (${n}: "${String($).trim().slice(0,500)}"); sent it as a system instruction instead. Set jsonObject: "instruction" on this upstream to skip the first attempt.`,endpoint:w.upstream.upstream,model:w.upstream.body.model,status:n,reason:String($).trim().slice(0,500),action:"instruction"})}}return{upstream:t,result:a}},"attempt");let s=await d(g,{...v,hold:!0});const H=s.pool;let q=0;const P=H.length>1?o.config.pool:null,R=l(()=>P&&{connectTimeout:P.connectTimeout,headersTimeout:q+1<H.length?P.headersTimeout:0},"limitsNow");let U=o.endpoint(s.upstream);const F=l(()=>U.health.inflight--,"release");let m,N;try{for(;;){const e=U;e.health.requests++,x(s);let t=null;try{({upstream:s,result:m,failure:N}=await X(s)),!m.ok&&m.status>=500&&(t=`HTTP ${m.status}`,h({type:"error",kind:"upstream",level:"error",message:`Upstream ${e.name} answered HTTP ${m.status}`,endpoint:e.name,model:s.body.model,status:m.status}))}catch(n){if(!(n instanceof J)||n.code!=="upstream_unreachable")throw n;t=n.message,m=void 0,N=n,h({type:"error",kind:"transport",level:"error",message:n.message,endpoint:e.name,model:s.body.model})}if(t===null){V.succeeded(e);break}V.failed(o.config,e,t);let a=null;for(;a===null&&!p.signal?.aborted&&q+1<H.length;)try{a=await d(g,{...v,endpoint:H[++q],hold:!0})}catch(n){if(!(n instanceof J)||n.code!=="endpoint_not_in_pool")throw n}if(a===null){if(m===void 0)throw N;break}h({type:"failover",level:"warn",message:`Upstream ${e.name} failed (${t}); trying ${a.upstream}`,from:e.name,to:a.upstream,model:a.body.model,reason:t});try{await m?.body?.cancel?.()}catch{}F(),U=o.endpoint(a.upstream),s=a}}catch(e){throw F(),e}const Y=U;U=null,i.usage.then(()=>Y.health.inflight--);const G=tt.plain(m.headers);if(!m.ok)return E(m.status,N??await M(m),G,s.state,"upstream_error");if(s.state.stream){const e=j(m.body,s.state),t=s.state.native?e:mt(i.meta.front,e,{gaps:l(()=>b(s),"gaps"),input:l(()=>O(s),"input"),report:l(a=>i.estimate(a),"report")});return{status:200,headers:ft,stream:i.track(t,p.signal,S)}}const C=await M(m);if(C===null||typeof C!="object")return h({type:"error",kind:"upstream",level:"error",message:`Upstream ${s.upstream} returned a body that is not JSON`,endpoint:s.upstream,model:s.body.model,status:502}),E(502,"The upstream returned a body that is not JSON",G,s.state,"error");const A=await k(C,s.state),z=await c(s,lt(i.meta.front,A));return z&&(A.usage=ct(i.meta.front,A.usage,z),i.estimate(z)),i.json(A),i.finish("done"),{status:200,headers:{"content-type":"application/json"},body:A}}l(gt,"round");async function W(u,r,y,d=null){let k,j=!1,_=y;if(d&&d.headersTimeout>0){const o=new AbortController;_=y?AbortSignal.any([y,o.signal]):o.signal,k=setTimeout(()=>{j=!0,o.abort()},d.headersTimeout)}try{return await u(r.url,{method:r.method,headers:r.headers,body:JSON.stringify(r.body),signal:_,redirect:"error",...d&&d.connectTimeout>0?{connectTimeout:d.connectTimeout}:{}})}catch(o){if(y?.aborted)throw o;const g=j?`no response headers within ${d.headersTimeout} ms`:o.message;throw new J(`Could not reach the upstream (${r.upstream}): ${g}`,{code:"upstream_unreachable",status:502,details:{cause:o}})}finally{clearTimeout(k)}}l(W,"call");async function M(u){const r=await u.text();try{return JSON.parse(r)}catch{return r===""?null:r}}l(M,"parse");export{St as send};
1
+ var tt=Object.defineProperty;var l=(u,s)=>tt(u,"name",{value:s,configurable:!0});import{context as R,self as Q}from"../lib/runtime.mjs";import{TranslatorError as C}from"../lib/errors.mjs";import*as et from"../lib/http.mjs";import*as at from"../lib/images.mjs";import*as rt from"../lib/tools.mjs";import{parse as st}from"../lib/upstream-error.mjs";import{meter as nt,normalize as ot}from"../lib/usage.mjs";import{capture as it}from"../lib/capture.mjs";import{estimate as lt,bodyLength as ut,output as ct,patch as mt,stream as dt}from"../lib/estimate.mjs";import{warn as ft}from"../lib/warn.mjs";import{start as pt}from"../lib/record.mjs";import{emit as V}from"../lib/events.mjs";import*as W from"../lib/pool.mjs";import{createBudget as gt}from"../lib/thinking.mjs";const ht={"content-type":"text/event-stream; charset=utf-8","cache-control":"no-cache","x-accel-buffering":"no"};async function Ut(u,s={}){const{transport:y,textOnly:f,jsonInstruction:k}=R.state,{request:j,response:_,stream:i,error:g}=Q,d=g.build,{config:o,state:$}=R,h=it(Q,R,s.front),S=u&&typeof u=="object"&&!Array.isArray(u),x={model:null,requested:S&&typeof u.model=="string"?u.model:null,upstream:null,front:s.front??o.front,stream:S&&u.stream===!0},w=nt(x,s.onUsage,(m,E)=>ft(o,$,m,{type:"usage",...E,log:s.log})),O=l(m=>Object.defineProperty(m,"usage",{value:w.usage,enumerable:!1,configurable:!0}),"attach");try{return O(await yt({transport:y,textOnly:f,jsonInstruction:k,request:j,response:_,stream:i,error:g,captured:h},u,s,w))}catch(m){if(!(m instanceof C))throw s.signal?.aborted||V(h,{type:"error",kind:"exception",level:"error",message:String(m?.message??m),log:s.log}),w.finish(s.signal?.aborted?"aborted":"error"),m;return w.finish("error"),O(d(m.status,m.message,{type:m.type,code:m.code,param:m.param,front:s.front}))}}l(Ut,"send");async function yt({transport:u,textOnly:s,jsonInstruction:y,request:f,response:k,stream:j,error:_,captured:i},g,d,o){const $={front:d.front,headers:d.headers,clientHeaders:d.clientHeaders,filter:d.filter,usage:!0,budget:gt(),log:d.log},h=l(e=>{V(i,{...e,log:d.log})},"announce"),S=o.meta.front==="openai"&&g?.stream_options?.include_usage!==!0,x=l(e=>{o.meta.model=e.body.model??null,o.meta.upstream=e.upstream,e.state.native?o.source(null):o.source(()=>e.state.convert?.upstreamUsage,e.state.front==="anthropic"?"openai":"folded")},"note"),w=l(e=>{if(e.state.native)return{input:!1,output:!1};const t=ot(e.state.front==="anthropic"?"openai":"folded",e.state.convert?.upstreamUsage);return t===null?{input:!0,output:!0}:{input:t.input_tokens===null&&t.cache_read_input_tokens===null&&t.cache_creation_input_tokens===null,output:t.output_tokens===null}},"gaps"),O=l(async e=>{const t=i.endpoint(e.upstream),a=i.driver(t);return(a.count&&!(S&&e.state.stream)?await a.count({endpoint:t,request:e.body,transport:u,signal:d.signal,timeout:i.config.detectTimeout}):null)??lt(e.body)},"input"),m=l(async(e,t)=>{const a=w(e);return!a.input&&!a.output?null:{input_tokens:a.input?await O(e):null,output_tokens:a.output?ct(t):null}},"counts"),E=l((e,t,a,r,I)=>(o.finish(I),_(e,t,a,r)),"fail"),Y=l(async e=>{let t=e,a=await X(u,t,d.signal,G());const r=a.status,I=r===400&&[t.images,t.tools].some(v=>v.sent&&v.retry),L=(r===400||r===422)&&t.json.sent&&t.json.retry;if(!a.ok&&(I||L)){const v=await F(a),T=st(v,r).message,p={};if(r===400&&t.images.sent&&t.images.retry&&at.refused(T)?(s.add(t.images.key),p.images=!1):r===400&&t.tools.sent&&t.tools.retry&&rt.refused(T)?p.tools=!1:L&&(p.jsonObject="instruction"),!("images"in p||"tools"in p||"jsonObject"in p))return{upstream:t,result:a,failure:v};const b={upstream:t,result:a,failure:v};if("images"in p||"tools"in p){const M="images"in p?"images":"tools";h({type:"fallback",kind:M,level:"warn",message:`Upstream ${t.upstream}: ${t.state.model} refused ${M==="images"?"images":"tools"} (${r}: "${String(T).trim().slice(0,500)}"); sent the request again without them.`,endpoint:t.upstream,model:t.body.model,status:r,reason:String(T).trim().slice(0,500),action:M==="images"?"text-only":"without-tools"})}if(t=await f(g,{...$,...p,endpoint:t.upstream,retry:!0}),x(t),a=await X(u,t,d.signal,G()),L&&"jsonObject"in p){if(!a.ok){try{await a.body?.cancel?.()}catch{}return x(b.upstream),b}y.add(b.upstream.json.key),h({type:"fallback",kind:"json_object",level:"warn",message:`Upstream ${b.upstream.upstream}: ${b.upstream.state.model} rejected the json_object schema emulation (${r}: "${String(T).trim().slice(0,500)}"); sent it as a system instruction instead. Set jsonObject: "instruction" on this upstream to skip the first attempt.`,endpoint:b.upstream.upstream,model:b.upstream.body.model,status:r,reason:String(T).trim().slice(0,500),action:"instruction"})}}return{upstream:t,result:a}},"attempt");let n=await f(g,{...$,hold:!0});const H=n.pool;let z=0;const B=H.length>1?i.config.pool:null,G=l(()=>B&&{connectTimeout:B.connectTimeout,headersTimeout:z+1<H.length?B.headersTimeout:0},"limitsNow");let U=i.endpoint(n.upstream);const K=l(()=>U.health.inflight--,"release");let c,N;try{for(;;){const e=U;e.health.requests++,x(n);let t=null;try{({upstream:n,result:c,failure:N}=await Y(n)),!c.ok&&c.status>=500&&(t=`HTTP ${c.status}`,h({type:"error",kind:"upstream",level:"error",message:`Upstream ${e.name} answered HTTP ${c.status}`,endpoint:e.name,model:n.body.model,status:c.status}))}catch(r){if(!(r instanceof C)||r.code!=="upstream_unreachable")throw r;t=r.message,c=void 0,N=r,h({type:"error",kind:"transport",level:"error",message:r.message,endpoint:e.name,model:n.body.model})}if(t===null){W.succeeded(e);break}W.failed(i.config,e,t);let a=null;for(;a===null&&!d.signal?.aborted&&z+1<H.length;)try{a=await f(g,{...$,endpoint:H[++z],hold:!0})}catch(r){if(!(r instanceof C)||r.code!=="endpoint_not_in_pool")throw r}if(a===null){if(c===void 0)throw N;break}h({type:"failover",level:"warn",message:`Upstream ${e.name} failed (${t}); trying ${a.upstream}`,from:e.name,to:a.upstream,model:a.body.model,reason:t});try{await c?.body?.cancel?.()}catch{}K(),U=i.endpoint(a.upstream),n=a}}catch(e){throw K(),e}const Z=U;U=null,o.usage.then(()=>Z.health.inflight--);const q=et.plain(c.headers),J=pt(i,{body:g,sent:n,front:o.meta.front,requested:o.meta.requested,headers:q,log:d.log});if(!c.ok){const e=N??await F(c);return J.reply(c.status,q,e),E(c.status,e,q,n.state,"upstream_error")}if(n.state.stream){const e=j(J.raw(c.body),n.state),t=n.state.native?e:dt(o.meta.front,e,{gaps:l(()=>w(n),"gaps"),input:l(()=>O(n),"input"),report:l(a=>o.estimate(a),"report")});return{status:200,headers:ht,stream:J.events(o.track(t,d.signal,S))}}const P=await F(c);if(J.reply(c.status,q,P),P===null||typeof P!="object")return h({type:"error",kind:"upstream",level:"error",message:`Upstream ${n.upstream} returned a body that is not JSON`,endpoint:n.upstream,model:n.body.model,status:502}),E(502,"The upstream returned a body that is not JSON",q,n.state,"error");const A=await k(P,n.state),D=await m(n,ut(o.meta.front,A));return D&&(A.usage=mt(o.meta.front,A.usage,D),o.estimate(D)),o.json(A),o.finish("done"),{status:200,headers:{"content-type":"application/json"},body:A}}l(yt,"round");async function X(u,s,y,f=null){let k,j=!1,_=y;if(f&&f.headersTimeout>0){const i=new AbortController;_=y?AbortSignal.any([y,i.signal]):i.signal,k=setTimeout(()=>{j=!0,i.abort()},f.headersTimeout)}try{return await u(s.url,{method:s.method,headers:s.headers,body:JSON.stringify(s.body),signal:_,redirect:"error",...f&&f.connectTimeout>0?{connectTimeout:f.connectTimeout}:{}})}catch(i){if(y?.aborted)throw i;const g=j?`no response headers within ${f.headersTimeout} ms`:i.message;throw new C(`Could not reach the upstream (${s.upstream}): ${g}`,{code:"upstream_unreachable",status:502,details:{cause:i}})}finally{clearTimeout(k)}}l(X,"call");async function F(u){const s=await u.text();try{return JSON.parse(s)}catch{return s===""?null:s}}l(F,"parse");export{Ut as send};
2
2
  //# sourceMappingURL=send.mjs.map
@@ -1,2 +1,2 @@
1
- var w=Object.defineProperty;var n=(o,t)=>w(o,"name",{value:t,configurable:!0});import{context as s,self as u}from"../../lib/runtime.mjs";import{TranslatorError as m}from"../../lib/errors.mjs";import{register as b}from"../../lib/routes.mjs";import{capture as k}from"../../lib/capture.mjs";import{resolve as x}from"../../lib/route.mjs";import{resolve as y}from"../../lib/settings.mjs";import{emit as I}from"../../lib/events.mjs";async function J(o,t={}){const{config:r}=s;if(t.front!==void 0&&t.front!=="anthropic"&&t.front!=="openai")throw new m(`front must be "anthropic" or "openai" (got ${JSON.stringify(t.front)})`,{code:"invalid_config",status:500});for(const e of["authorize","onUsage"])if(t[e]!==void 0&&typeof t[e]!="function")throw new m(`${e} must be a function`,{code:"invalid_config",status:500});const{send:d,route:p,token:h,model:c,error:v,sse:f}=u,l=t.front??r.front,{state:a}=s,{events:i}=a;i.logger=o.log,o.addHook("onClose",async()=>{i.logger===o.log&&(i.logger=null)});const g=k(u,s,l);b(o,t,{front:l,filter:r.settings.passthroughFilter!==!1,interval:n(e=>{try{return g.settings(x(g,e).endpoint).pingInterval}catch{return y(r.settings).pingInterval}},"interval"),send:d,route:p,count:h.count,list:c.list,get:c.get,emit:n(e=>I({config:r,state:a},e),"emit"),build:v.build,encode:f.encode,write:f.write})}n(J,"plugin");export{J as plugin};
1
+ var w=Object.defineProperty;var n=(o,t)=>w(o,"name",{value:t,configurable:!0});import{context as s,self as u}from"../../lib/runtime.mjs";import{TranslatorError as d}from"../../lib/errors.mjs";import{register as k}from"../../lib/routes.mjs";import{capture as x}from"../../lib/capture.mjs";import{resolve as y}from"../../lib/route.mjs";import{resolve as I}from"../../lib/settings.mjs";import{emit as T}from"../../lib/events.mjs";async function N(o,t={}){const{config:r}=s;if(t.front!==void 0&&t.front!=="anthropic"&&t.front!=="openai")throw new d(`front must be "anthropic" or "openai" (got ${JSON.stringify(t.front)})`,{code:"invalid_config",status:500});for(const e of["authorize","onUsage"])if(t[e]!==void 0&&typeof t[e]!="function")throw new d(`${e} must be a function`,{code:"invalid_config",status:500});const{send:m,route:p,token:h,model:c,error:v,sse:f,embeddings:b}=u,l=t.front??r.front,{state:g}=s,{events:i}=g;i.logger=o.log,o.addHook("onClose",async()=>{i.logger===o.log&&(i.logger=null)});const a=x(u,s,l);k(o,t,{front:l,filter:r.settings.passthroughFilter!==!1,interval:n(e=>{try{return a.settings(y(a,e).endpoint).pingInterval}catch{return I(r.settings).pingInterval}},"interval"),send:m,embed:b.send,route:p,count:h.count,list:c.list,get:c.get,emit:n(e=>T({config:r,state:g},e),"emit"),build:v.build,encode:f.encode,write:f.write})}n(N,"plugin");export{N as plugin};
2
2
  //# sourceMappingURL=plugin.mjs.map
@@ -1,2 +1,2 @@
1
- var i=Object.defineProperty;var n=(t,r)=>i(t,"name",{value:r,configurable:!0});import{TranslatorError as o}from"./errors.mjs";import*as a from"./pool.mjs";import{normalizeAliases as l,normalizeEfforts as u}from"./effort.mjs";function p(t){let r=t.length;for(;t[r-1]==="/";)r--;const e=t.slice(0,r);return e.endsWith("/v1")?e.slice(0,-3):e}n(p,"baseUrl");function d(t,r){const e=typeof t=="string"?{url:t}:t;if(!e||typeof e!="object"||typeof e.url!="string"||e.url==="")throw new o(`upstreams[${r}] needs a "url"`,{code:"INVALID_CONFIG",status:500});return{name:e.name??`upstream-${r}`,url:p(e.url),apiKey:e.apiKey,headers:{...e.headers},backend:e.backend??"auto",models:Array.isArray(e.models)?[...e.models]:void 0,default:e.default===!0,settings:{...e.settings},version:e.version,driver:null,facts:null,driverSettings:{},probed:{},health:a.health()}}n(d,"endpoint");function I(t={}){const r=t.front??"anthropic";if(r!=="anthropic"&&r!=="openai")throw new o(`front must be "anthropic" or "openai" (got ${JSON.stringify(r)})`,{code:"INVALID_CONFIG",status:500});let e=t.upstreams??t.upstream;if(e!==void 0&&!Array.isArray(e)&&(e=[e]),!e||e.length===0)throw new o("at least one upstream is required (config.upstreams)",{code:"INVALID_CONFIG",status:500});const s=t.logs??!1;if(typeof s!="boolean"&&(typeof s!="string"||s===""))throw new o("logs must be true, false or a file path (got "+JSON.stringify(s)+")",{code:"INVALID_CONFIG",status:500});if(t.onEvent!==void 0&&typeof t.onEvent!="function")throw new o("onEvent must be a function",{code:"INVALID_CONFIG",status:500});return{front:r,endpoints:e.map(d),...l(t.aliases),efforts:u(t.efforts),pool:a.normalize(t.pool),settings:{...t.settings},detect:t.detect!==!1,detectTimeout:t.detectTimeout??5e3,transport:t.transport,logs:s,events:t.events!==!1,onEvent:t.onEvent}}n(I,"normalize");export{d as endpoint,I as normalize};
1
+ var i=Object.defineProperty;var a=(t,r)=>i(t,"name",{value:r,configurable:!0});import{TranslatorError as s}from"./errors.mjs";import*as n from"./pool.mjs";import{valid as l}from"./record.mjs";import{normalizeAliases as u,normalizeEfforts as p}from"./effort.mjs";function d(t){let r=t.length;for(;t[r-1]==="/";)r--;const e=t.slice(0,r);return e.endsWith("/v1")?e.slice(0,-3):e}a(d,"baseUrl");function f(t,r){const e=typeof t=="string"?{url:t}:t;if(!e||typeof e!="object"||typeof e.url!="string"||e.url==="")throw new s(`upstreams[${r}] needs a "url"`,{code:"INVALID_CONFIG",status:500});return{name:e.name??`upstream-${r}`,url:d(e.url),apiKey:e.apiKey,headers:{...e.headers},backend:e.backend??"auto",models:Array.isArray(e.models)?[...e.models]:void 0,default:e.default===!0,settings:{...e.settings},version:e.version,driver:null,facts:null,driverSettings:{},probed:{},health:n.health()}}a(f,"endpoint");function N(t={}){const r=t.front??"anthropic";if(r!=="anthropic"&&r!=="openai")throw new s(`front must be "anthropic" or "openai" (got ${JSON.stringify(r)})`,{code:"INVALID_CONFIG",status:500});let e=t.upstreams??t.upstream;if(e!==void 0&&!Array.isArray(e)&&(e=[e]),!e||e.length===0)throw new s("at least one upstream is required (config.upstreams)",{code:"INVALID_CONFIG",status:500});const o=t.logs??!1;if(typeof o!="boolean"&&(typeof o!="string"||o===""))throw new s("logs must be true, false or a file path (got "+JSON.stringify(o)+")",{code:"INVALID_CONFIG",status:500});if(t.onEvent!==void 0&&typeof t.onEvent!="function")throw new s("onEvent must be a function",{code:"INVALID_CONFIG",status:500});if(!l(t.capture))throw new s("capture must be false, a directory path or a function (got "+JSON.stringify(t.capture)+")",{code:"INVALID_CONFIG",status:500});return{front:r,endpoints:e.map(f),...u(t.aliases),efforts:p(t.efforts),pool:n.normalize(t.pool),settings:{...t.settings},detect:t.detect!==!1,detectTimeout:t.detectTimeout??5e3,transport:t.transport,logs:o,capture:t.capture||!1,events:t.events!==!1,onEvent:t.onEvent}}a(N,"normalize");export{f as endpoint,N as normalize};
2
2
  //# sourceMappingURL=config.mjs.map
@@ -0,0 +1,3 @@
1
+ var E=Object.defineProperty;var s=(e,c)=>E(e,"name",{value:c,configurable:!0});import{randomBytes as F}from"node:crypto";import{mkdirSync as O,writeFileSync as T}from"node:fs";import y from"node:path";import{items as v}from"./source.mjs";import{warn as $}from"./warn.mjs";const u=1e6,j=/key|token|secret|auth|cookie|password/i;function m(e){const c={};for(const[r,i]of Object.entries(e??{}))c[r.toLowerCase()]=j.test(r)?"[redacted]":i;return c}s(m,"redact");function L(e){return e===void 0||e===!1||typeof e=="function"||typeof e=="string"&&e!==""}s(L,"valid");const C={raw:s(e=>e,"raw"),events:s(e=>e,"events"),reply(){}};function M({config:e,state:c},r){const i=e.capture;if(!i)return C;const{sent:f}=r,h=F(6).toString("hex"),w=Date.now(),d={time:w,id:h,front:r.front,requested:r.requested,model:f.body?.model??null,endpoint:f.upstream,stream:f.state.stream,client:{body:r.body},upstream:{url:f.url,headers:m(f.headers),body:f.body},response:{}};let a="",g=!1,l=0,p=!1;const S=[],b=s(()=>{try{if(typeof i=="function"){const t=i(d);typeof t?.then=="function"&&t.then(void 0,n=>k(n))}else O(y.resolve(i),{recursive:!0}),T(y.join(y.resolve(i),`${w}-${h}.json`),`${JSON.stringify(d,null,2)}
2
+ `)}catch(t){k(t)}},"emit"),k=s(t=>{$(e,c,`Could not capture the exchange (${t instanceof Error?t.message:String(t)}); the reply is not affected`,{type:"warning",kind:"capture",log:r.log})},"problem");return{raw(t){return(async function*(){const n=new TextDecoder;for await(const o of v(t))g||(a+=typeof o=="string"?o:n.decode(o,{stream:!0}),a.length>u&&(a=a.slice(0,u),g=!0,p=!0)),yield o})()},events(t){return(async function*(){try{for await(const n of v(t)){if(l<=u){const o=JSON.stringify(n).length;l+=o,l>u?p=!0:S.push(n)}yield n}}finally{d.response={status:200,headers:m(r.headers),raw:a,events:S,truncated:p},b()}})()},reply(t,n,o){d.response={status:t,headers:m(n),body:o},b()}}}s(M,"start");export{u as STREAM_LIMIT,m as redact,M as start,L as valid};
3
+ //# sourceMappingURL=record.mjs.map
@@ -1,2 +1,2 @@
1
- var A=Object.defineProperty;var c=(e,n)=>A(e,"name",{value:n,configurable:!0});import{timingSafeEqual as H}from"node:crypto";import{TranslatorError as p}from"./errors.mjs";import{messageId as O}from"./ids.mjs";import{sign as T}from"./thinking.mjs";const j=64*1024*1024,D={"content-type":"text/event-stream; charset=utf-8","cache-control":"no-cache","x-accel-buffering":"no"},w=Symbol("stalled"),v=Symbol("gone");async function I(e,n){if(!(n>0))return e;let a;const l=new Promise(f=>{a=setTimeout(f,n,w)});try{return await Promise.race([e,l])}finally{clearTimeout(a)}}c(I,"settle");async function*L(e,n,a){let l=!1;try{const f=n.front==="anthropic";f&&(yield{event:"message_start",data:{type:"message_start",message:{id:O(),type:"message",role:"assistant",model:a,content:[],stop_reason:null,stop_sequence:null,usage:{input_tokens:0,output_tokens:0,cache_creation_input_tokens:0,cache_read_input_tokens:0}}}},yield{event:"content_block_start",data:{type:"content_block_start",index:0,content_block:{type:"thinking",thinking:"",signature:""}}});const s=await e,m=c(function*(){yield{event:"content_block_delta",data:{type:"content_block_delta",index:0,delta:{type:"signature_delta",signature:T()}}},yield{event:"content_block_stop",data:{type:"content_block_stop",index:0}}},"closeEarly");if(s.stream){l=!0;let u=null;for await(const i of s.stream){if(!f){yield i;continue}if(i.event!=="message_start"){if(u===null&&i.event==="content_block_start"&&i.data.content_block.type==="thinking"){u=0;continue}u===null&&(yield*m(),u=1),yield u>0&&typeof i.data?.index=="number"?{...i,data:{...i.data,index:i.data.index+u}}:i}}return}f&&(yield*m()),yield*x(n,s.body)}finally{l||e.then(f=>f.stream?.return?.(),()=>{})}}c(L,"late");function*x(e,n){if(e.front==="anthropic"){yield{event:"error",data:n};return}yield{data:n},yield{data:"[DONE]"}}c(x,"failure");async function E(e,n,a,l,f,s=200,m=D){e.hijack();const u=e.raw;u.writeHead(s,m);try{await n.write(u,l,{front:n.front,interval:f})}catch(i){if(!a.signal.aborted){const g=i instanceof p;g||(e.log.error(i),n.emit({type:"error",kind:"route",level:"error",message:String(i?.message??i),log:!1}));const{body:_}=n.build(g?i.status:502,g?i.message:"Internal server error",g?{type:i.type,code:i.code}:{});for(const t of x(n,_))u.write(n.encode(t))}}return u.end(),e}c(E,"stream");function z(e,n){const a=Buffer.from(e),l=Buffer.from(n);return a.length===l.length&&H(a,l)}c(z,"same");function B(e){const n=/^Bearer\s+(.+)$/i.exec(e.authorization??"");return e["x-api-key"]||n?.[1]||""}c(B,"keyOf");function C(e){const n={};for(const[a,l]of Object.entries(e))(a==="anthropic-beta"||a==="anthropic-version"||a.startsWith("x-claude-code-"))&&(n[a]=String(l));return n}c(C,"forwardable");function P(e){if(e===!0)return{ok:!0,context:void 0};if(e&&typeof e=="object"&&e.ok===!0)return{ok:!0,context:e.context};const n=e&&typeof e=="object"&&e.ok===!1?e:{};return{ok:!1,status:Number.isInteger(n.status)&&n.status>=400&&n.status<=599?n.status:401,message:typeof n.message=="string"&&n.message!==""?n.message:"unauthorized"}}c(P,"verdictOf");function W(e,n,a){const{front:l,filter:f}=a,s={...a,build:c((t,o,r={})=>a.build(t,o,{...r,front:l}),"build"),send:c((t,o)=>a.send(t,{...o,front:l,filter:f}),"send"),route:c(t=>a.route(t,{front:l}),"route"),count:c((t,o)=>a.count(t,{...o,front:l}),"count"),list:c(t=>a.list({...t,front:l}),"list"),get:c((t,o)=>a.get(t,{...o,front:l}),"get")},m=[n.apiKey??[]].flat().filter(Boolean),u=n.bodyLimit??j,i=new WeakMap,g=c((t,o)=>t.code(o.status).headers(o.headers).send(o.body),"reject");e.setErrorHandler((t,o,r)=>{if(t instanceof p)return g(r,s.build(t.status,t.message,{type:t.type,code:t.code,param:t.param}));const d=t.statusCode??500;return d<500?g(r,s.build(d,t.message)):(o.log.error(t),s.emit({type:"error",kind:"route",level:"error",message:String(t.message),status:d,log:!1}),g(r,s.build(d,"Internal server error")))}),e.addHook("onRequest",async(t,o)=>{if(m.length===0||t.routeOptions.config?.public)return;const r=B(t.headers);if(!(r&&m.some(d=>z(r,d))))return g(o,s.build(401,"invalid x-api-key"))}),typeof n.authorize=="function"&&e.addHook("preHandler",async(t,o)=>{if(t.routeOptions.config?.public)return;let r=null;if(t.body&&typeof t.body=="object"&&!Array.isArray(t.body))try{r=s.route(t.body)}catch{}let d;try{d=P(await n.authorize({method:t.method,url:t.url,headers:t.headers,body:t.body,front:s.front,route:r}))}catch(b){t.log.error(b),s.emit({type:"error",kind:"route",level:"error",message:String(b?.message??b),log:!1}),d={ok:!1,status:401,message:"unauthorized"}}if(!d.ok)return g(o,s.build(d.status,d.message));i.set(t,d.context)}),e.route({method:["GET","HEAD"],url:"/",config:{public:!0},handler:c(async(t,o)=>o.send({ok:!0}),"handler")}),e.route({method:["GET","HEAD"],url:"/api/hello",config:{public:!0},handler:c(async(t,o)=>o.send({ok:!0}),"handler")}),e.get("/v1/models",async(t,o)=>{const r=new AbortController;return o.raw.on("close",()=>r.abort()),s.list({signal:r.signal})}),e.get("/v1/models/*",async(t,o)=>{const r=new AbortController;return o.raw.on("close",()=>r.abort()),s.get(t.params["*"],{signal:r.signal})});const _=s.front==="anthropic"?"/v1/messages":"/v1/chat/completions";e.post(_,{bodyLimit:u},async(t,o)=>{if(o.raw.destroyed)return o;const r=new AbortController;o.raw.on("close",()=>{o.raw.writableFinished||r.abort()});const d=typeof n.onUsage=="function"?h=>n.onUsage(h,i.get(t)):void 0,b=s.interval(t.body),k=s.send(t.body,{signal:r.signal,clientHeaders:C(t.headers),onUsage:d,log:t.log});k.catch(()=>{});const S=new Promise(h=>r.signal.addEventListener("abort",()=>h(v),{once:!0})),y=await I(Promise.race([k,S]),t.body?.stream===!0?b:0);return y===v?(k.then(h=>h.stream?.return?.(),()=>{}),o):y===w?E(o,s,r,L(k,s,t.body?.model),b):y.stream?E(o,s,r,y.stream,b,y.status,y.headers):g(o,y)}),s.front==="anthropic"&&e.post("/v1/messages/count_tokens",{bodyLimit:u},async(t,o)=>{const r=new AbortController;return o.raw.on("close",()=>r.abort()),s.count(t.body,{signal:r.signal})})}c(W,"register");export{L as late,W as register};
1
+ var S=Object.defineProperty;var c=(e,o)=>S(e,"name",{value:o,configurable:!0});import{timingSafeEqual as H}from"node:crypto";import{TranslatorError as w}from"./errors.mjs";import{messageId as O}from"./ids.mjs";import{sign as T}from"./thinking.mjs";const j=64*1024*1024,C={"content-type":"text/event-stream; charset=utf-8","cache-control":"no-cache","x-accel-buffering":"no"},p=Symbol("stalled"),v=Symbol("gone");async function D(e,o){if(!(o>0))return e;let a;const l=new Promise(g=>{a=setTimeout(g,o,p)});try{return await Promise.race([e,l])}finally{clearTimeout(a)}}c(D,"settle");async function*I(e,o,a){let l=!1;try{const g=o.front==="anthropic";g&&(yield{event:"message_start",data:{type:"message_start",message:{id:O(),type:"message",role:"assistant",model:a,content:[],stop_reason:null,stop_sequence:null,usage:{input_tokens:0,output_tokens:0,cache_creation_input_tokens:0,cache_read_input_tokens:0}}}},yield{event:"content_block_start",data:{type:"content_block_start",index:0,content_block:{type:"thinking",thinking:"",signature:""}}});const s=await e,m=c(function*(){yield{event:"content_block_delta",data:{type:"content_block_delta",index:0,delta:{type:"signature_delta",signature:T()}}},yield{event:"content_block_stop",data:{type:"content_block_stop",index:0}}},"closeEarly");if(s.stream){l=!0;let d=null;for await(const i of s.stream){if(!g){yield i;continue}if(i.event!=="message_start"){if(d===null&&i.event==="content_block_start"&&i.data.content_block.type==="thinking"){d=0;continue}d===null&&(yield*m(),d=1),yield d>0&&typeof i.data?.index=="number"?{...i,data:{...i.data,index:i.data.index+d}}:i}}return}g&&(yield*m()),yield*x(o,s.body)}finally{l||e.then(g=>g.stream?.return?.(),()=>{})}}c(I,"late");function*x(e,o){if(e.front==="anthropic"){yield{event:"error",data:o};return}yield{data:o},yield{data:"[DONE]"}}c(x,"failure");async function E(e,o,a,l,g,s=200,m=C){e.hijack();const d=e.raw;d.writeHead(s,m);try{await o.write(d,l,{front:o.front,interval:g})}catch(i){if(!a.signal.aborted){const f=i instanceof w;f||(e.log.error(i),o.emit({type:"error",kind:"route",level:"error",message:String(i?.message??i),log:!1}));const{body:_}=o.build(f?i.status:502,f?i.message:"Internal server error",f?{type:i.type,code:i.code}:{});for(const t of x(o,_))d.write(o.encode(t))}}return d.end(),e}c(E,"stream");function L(e,o){const a=Buffer.from(e),l=Buffer.from(o);return a.length===l.length&&H(a,l)}c(L,"same");function z(e){const o=/^Bearer\s+(.+)$/i.exec(e.authorization??"");return e["x-api-key"]||o?.[1]||""}c(z,"keyOf");function B(e){const o={};for(const[a,l]of Object.entries(e))(a==="anthropic-beta"||a==="anthropic-version"||a.startsWith("x-claude-code-"))&&(o[a]=String(l));return o}c(B,"forwardable");function P(e){if(e===!0)return{ok:!0,context:void 0};if(e&&typeof e=="object"&&e.ok===!0)return{ok:!0,context:e.context};const o=e&&typeof e=="object"&&e.ok===!1?e:{};return{ok:!1,status:Number.isInteger(o.status)&&o.status>=400&&o.status<=599?o.status:401,message:typeof o.message=="string"&&o.message!==""?o.message:"unauthorized"}}c(P,"verdictOf");function R(e,o,a){const{front:l,filter:g}=a,s={...a,build:c((t,n,r={})=>a.build(t,n,{...r,front:l}),"build"),send:c((t,n)=>a.send(t,{...n,front:l,filter:g}),"send"),route:c(t=>a.route(t,{front:l}),"route"),count:c((t,n)=>a.count(t,{...n,front:l}),"count"),embed:c((t,n)=>a.embed(t,n),"embed"),list:c(t=>a.list({...t,front:l}),"list"),get:c((t,n)=>a.get(t,{...n,front:l}),"get")},m=[o.apiKey??[]].flat().filter(Boolean),d=o.bodyLimit??j,i=new WeakMap,f=c((t,n)=>t.code(n.status).headers(n.headers).send(n.body),"reject");e.setErrorHandler((t,n,r)=>{if(t instanceof w)return f(r,s.build(t.status,t.message,{type:t.type,code:t.code,param:t.param}));const u=t.statusCode??500;return u<500?f(r,s.build(u,t.message)):(n.log.error(t),s.emit({type:"error",kind:"route",level:"error",message:String(t.message),status:u,log:!1}),f(r,s.build(u,"Internal server error")))}),e.addHook("onRequest",async(t,n)=>{if(m.length===0||t.routeOptions.config?.public)return;const r=z(t.headers);if(!(r&&m.some(u=>L(r,u))))return f(n,s.build(401,"invalid x-api-key"))}),typeof o.authorize=="function"&&e.addHook("preHandler",async(t,n)=>{if(t.routeOptions.config?.public)return;let r=null;if(t.body&&typeof t.body=="object"&&!Array.isArray(t.body))try{r=s.route(t.body)}catch{}let u;try{u=P(await o.authorize({method:t.method,url:t.url,headers:t.headers,body:t.body,front:s.front,route:r}))}catch(b){t.log.error(b),s.emit({type:"error",kind:"route",level:"error",message:String(b?.message??b),log:!1}),u={ok:!1,status:401,message:"unauthorized"}}if(!u.ok)return f(n,s.build(u.status,u.message));i.set(t,u.context)}),e.route({method:["GET","HEAD"],url:"/",config:{public:!0},handler:c(async(t,n)=>n.send({ok:!0}),"handler")}),e.route({method:["GET","HEAD"],url:"/api/hello",config:{public:!0},handler:c(async(t,n)=>n.send({ok:!0}),"handler")}),e.get("/v1/models",async(t,n)=>{const r=new AbortController;return n.raw.on("close",()=>r.abort()),s.list({signal:r.signal})}),e.get("/v1/models/*",async(t,n)=>{const r=new AbortController;return n.raw.on("close",()=>r.abort()),s.get(t.params["*"],{signal:r.signal})});const _=s.front==="anthropic"?"/v1/messages":"/v1/chat/completions";e.post(_,{bodyLimit:d},async(t,n)=>{if(n.raw.destroyed)return n;const r=new AbortController;n.raw.on("close",()=>{n.raw.writableFinished||r.abort()});const u=typeof o.onUsage=="function"?h=>o.onUsage(h,i.get(t)):void 0,b=s.interval(t.body),k=s.send(t.body,{signal:r.signal,clientHeaders:B(t.headers),onUsage:u,log:t.log});k.catch(()=>{});const A=new Promise(h=>r.signal.addEventListener("abort",()=>h(v),{once:!0})),y=await D(Promise.race([k,A]),t.body?.stream===!0?b:0);return y===v?(k.then(h=>h.stream?.return?.(),()=>{}),n):y===p?E(n,s,r,I(k,s,t.body?.model),b):y.stream?E(n,s,r,y.stream,b,y.status,y.headers):f(n,y)}),s.front==="openai"&&e.post("/v1/embeddings",{bodyLimit:d},async(t,n)=>{const r=new AbortController;return n.raw.on("close",()=>{n.raw.writableFinished||r.abort()}),f(n,await s.embed(t.body,{signal:r.signal,log:t.log}))}),s.front==="anthropic"&&e.post("/v1/messages/count_tokens",{bodyLimit:d},async(t,n)=>{const r=new AbortController;return n.raw.on("close",()=>r.abort()),s.count(t.body,{signal:r.signal})})}c(R,"register");export{I as late,R as register};
2
2
  //# sourceMappingURL=routes.mjs.map
@@ -0,0 +1,2 @@
1
+ var a=Object.defineProperty;var u=(e,i)=>a(e,"name",{value:i,configurable:!0});import{TranslatorError as p}from"./errors.mjs";const f=["seed","frequency_penalty","presence_penalty"];function h(e,i,l,c=[],o=[]){const s={...l.sampling,...i!==null&&typeof i=="object"&&!Array.isArray(i)?i:{}};let r=e;for(const[n,t]of Object.entries(s))if(!(t==null||Object.hasOwn(e,n))){if(!f.includes(n)&&!c.includes(n)){if(l.unknownFields==="error")throw new p(`sampling.${n}: is not accepted by this upstream`,{code:"unsupported_field",param:`sampling.${n}`});o.includes(n)||o.push(n);continue}r===e&&(r={...e}),r[n]=t}return r}u(h,"addSampling");export{h as addSampling};
2
+ //# sourceMappingURL=sampling.mjs.map
@@ -1,2 +1,2 @@
1
- var l=Object.defineProperty;var s=(n,e)=>l(n,"name",{value:e,configurable:!0});import{TranslatorError as r}from"./errors.mjs";const i=Object.freeze({inlineSystem:"auto",stripBillingHeader:!0,reasoningField:"reasoning",thinkTags:!0,thinkOpen:!1,reasoningRoundTrip:!1,thinking:"chat_template_kwargs",thinkingBudget:!1,effort:!1,images:!0,imageFallback:!0,toolFallback:!1,documents:"extract",documentConverter:null,pdf:Object.freeze({pages:!0,maxPages:100,maxBytes:32*1024*1024,maxImages:20,scale:1.5}),toolSchema:"pass",deferredTools:"forward",toolSearch:"expand",jsonObject:"schema",jsonObjectFallback:!0,toolErrors:!1,toolErrorMarker:"[error] ",toolErrorInference:!1,maxTokensField:"max_completion_tokens",streamUsage:!0,continuousUsage:!1,prefill:"continue_final_message",topK:!0,responseFormat:!0,parallelToolCalls:!0,unknownFields:"drop",pingInterval:1e4,toolStreaming:"buffer",effortPassthrough:!1,passthroughFilter:!1,contextWindow:0,defaultMaxTokens:8192}),f={inlineSystem:["auto","keep","hoist","user","drop"],reasoningField:["reasoning","reasoning_content"],thinking:["chat_template_kwargs","reasoning_effort","none"],documents:["extract","note","error"],toolSchema:["pass","sanitize"],deferredTools:["forward","drop"],toolSearch:["expand","inline","native"],jsonObject:["schema","instruction","drop"],maxTokensField:["max_completion_tokens","max_tokens"],prefill:["continue_final_message","none"],unknownFields:["drop","error"],toolStreaming:["buffer","stream"]},c=["effortPassthrough","passthroughFilter","imageFallback","toolFallback","jsonObjectFallback","thinkOpen","reasoningRoundTrip","toolErrorInference"];function u(...n){const e={...i,pdf:{...i.pdf}};for(const t of n)if(t)for(const[o,a]of Object.entries(t))a!==void 0&&(o==="pdf"?Object.assign(e.pdf,a):e[o]=a);for(const[t,o]of Object.entries(f))if(!o.includes(e[t]))throw new r(`Invalid setting ${t}: ${JSON.stringify(e[t])} (expected one of ${o.join(", ")})`,{code:"invalid_setting",status:500,details:{key:t,allowed:o}});for(const t of c)if(typeof e[t]!="boolean")throw new r(`Invalid setting ${t}: ${JSON.stringify(e[t])} (expected true or false)`,{code:"invalid_setting",status:500,details:{key:t}});if(typeof e.toolErrorMarker!="string"||e.toolErrorMarker.trim()==="")throw new r(`Invalid setting toolErrorMarker: ${JSON.stringify(e.toolErrorMarker)} (expected a non-empty string)`,{code:"invalid_setting",status:500,details:{key:"toolErrorMarker"}});if(!(Number.isFinite(e.pdf.scale)&&e.pdf.scale>0))throw new r(`Invalid setting pdf.scale: ${JSON.stringify(e.pdf.scale)} (expected a positive number)`,{code:"invalid_setting",status:500,details:{key:"pdf.scale"}});return e}s(u,"resolve");export{i as DEFAULTS,u as resolve};
1
+ var l=Object.defineProperty;var i=(a,e)=>l(a,"name",{value:e,configurable:!0});import{TranslatorError as r}from"./errors.mjs";const s=Object.freeze({inlineSystem:"auto",stripBillingHeader:!0,reasoningField:"reasoning",thinkTags:!0,thinkOpen:!1,reasoningRoundTrip:!1,thinking:"chat_template_kwargs",thinkingBudget:!1,effort:!1,images:!0,imageFallback:!0,toolFallback:!1,documents:"extract",documentConverter:null,pdf:Object.freeze({pages:!0,maxPages:100,maxBytes:32*1024*1024,maxImages:20,scale:1.5}),toolSchema:"pass",deferredTools:"forward",toolSearch:"expand",jsonObject:"schema",jsonObjectFallback:!0,toolErrors:!1,toolErrorMarker:"[error] ",toolErrorInference:!1,maxTokensField:"max_completion_tokens",streamUsage:!0,continuousUsage:!1,prefill:"continue_final_message",topK:!0,responseFormat:!0,parallelToolCalls:!0,unknownFields:"drop",sampling:Object.freeze({}),pingInterval:1e4,toolStreaming:"buffer",effortPassthrough:!1,passthroughFilter:!1,contextWindow:0,defaultMaxTokens:8192}),f={inlineSystem:["auto","keep","hoist","user","drop"],reasoningField:["reasoning","reasoning_content"],thinking:["chat_template_kwargs","reasoning_effort","none"],documents:["extract","note","error"],toolSchema:["pass","sanitize"],deferredTools:["forward","drop"],toolSearch:["expand","inline","native"],jsonObject:["schema","instruction","drop"],maxTokensField:["max_completion_tokens","max_tokens"],prefill:["continue_final_message","none"],unknownFields:["drop","error"],toolStreaming:["buffer","stream"]},c=["effortPassthrough","passthroughFilter","imageFallback","toolFallback","jsonObjectFallback","thinkOpen","reasoningRoundTrip","toolErrorInference"];function p(...a){const e={...s,pdf:{...s.pdf},sampling:{...s.sampling}};for(const t of a)if(t)for(const[n,o]of Object.entries(t))o!==void 0&&(n==="pdf"?Object.assign(e.pdf,o):n==="sampling"&&o!==null&&typeof o=="object"&&!Array.isArray(o)?Object.assign(e.sampling,o):e[n]=o);for(const[t,n]of Object.entries(f))if(!n.includes(e[t]))throw new r(`Invalid setting ${t}: ${JSON.stringify(e[t])} (expected one of ${n.join(", ")})`,{code:"invalid_setting",status:500,details:{key:t,allowed:n}});for(const t of c)if(typeof e[t]!="boolean")throw new r(`Invalid setting ${t}: ${JSON.stringify(e[t])} (expected true or false)`,{code:"invalid_setting",status:500,details:{key:t}});if(typeof e.toolErrorMarker!="string"||e.toolErrorMarker.trim()==="")throw new r(`Invalid setting toolErrorMarker: ${JSON.stringify(e.toolErrorMarker)} (expected a non-empty string)`,{code:"invalid_setting",status:500,details:{key:"toolErrorMarker"}});if(e.sampling===null||typeof e.sampling!="object"||Array.isArray(e.sampling))throw new r(`Invalid setting sampling: ${JSON.stringify(e.sampling)} (expected an object of field names and values)`,{code:"invalid_setting",status:500,details:{key:"sampling"}});if(!(Number.isFinite(e.pdf.scale)&&e.pdf.scale>0))throw new r(`Invalid setting pdf.scale: ${JSON.stringify(e.pdf.scale)} (expected a positive number)`,{code:"invalid_setting",status:500,details:{key:"pdf.scale"}});return e}i(p,"resolve");export{s as DEFAULTS,p as resolve};
2
2
  //# sourceMappingURL=settings.mjs.map
@@ -0,0 +1,2 @@
1
+ var l=Object.defineProperty;var p=(r,n)=>l(r,"name",{value:n,configurable:!0});const a=["allOf","not","if","then","else","dependentRequired","dependentSchemas","patternProperties","propertyNames","unevaluatedProperties","contains","minProperties","maxProperties","uniqueItems"],y=["items","additionalProperties"],d=["anyOf","oneOf","prefixItems"],$=["properties","$defs","definitions"];function f(r,n="$",i=[]){if(r===null||typeof r!="object"||Array.isArray(r))return i;const t=r.properties!==null&&typeof r.properties=="object"?Object.keys(r.properties):[];if(r.type==="object"||t.length>0){r.additionalProperties!==!1&&i.push(`${n}: object without additionalProperties: false`);const e=Array.isArray(r.required)?r.required:[],o=t.filter(s=>!e.includes(s));o.length>0&&i.push(`${n}: properties not in required: ${o.join(", ")}`)}for(const e of a)e in r&&i.push(`${n}: unsupported keyword ${e}`);for(const e of y)f(r[e],`${n}.${e}`,i);for(const e of d)Array.isArray(r[e])&&r[e].forEach((o,s)=>f(o,`${n}.${e}[${s}]`,i));for(const e of $){const o=r[e];if(!(o===null||typeof o!="object"))for(const[s,u]of Object.entries(o))f(u,`${n}.${e}.${s}`,i)}return i}p(f,"problems");function P(r){const n=[];for(const i of Array.isArray(r?.tools)?r.tools:[]){const t=i?.function??i;t?.strict===!0&&n.push({name:String(t.name),problems:f(t.parameters??t.input_schema)})}return n}p(P,"strictTools");function j(r){if(!Array.isArray(r?.tools))return r;let n=!1;const i=r.tools.map(t=>{const e=t?.function??t;if(e?.strict!==!0||f(e.parameters??e.input_schema).length===0)return t;if(n=!0,t.function){const{strict:u,...c}=t.function;return{...t,function:c}}const{strict:o,...s}=t;return s});return n?{...r,tools:i}:r}p(j,"relax");export{f as problems,j as relax,P as strictTools};
2
+ //# sourceMappingURL=strict.mjs.map
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@cldmv/llm-translator",
3
- "version": "0.5.0",
3
+ "version": "0.6.0",
4
4
  "description": "Bidirectional Anthropic Messages ⇄ OpenAI Chat Completions translator with backend auto-detection (vLLM and other OpenAI-compatible servers), built on slothlet, with an optional standalone server for running Claude Code against local models.",
5
5
  "type": "module",
6
6
  "license": "Apache-2.0",
@@ -18,13 +18,14 @@
18
18
  */
19
19
  /**
20
20
  * Describe the Anthropic driver (native Messages API upstream).
21
- * @returns {{name: string, protocol: string, order: number, auth: string, headers: Record<string,string>, paths: Record<string,string>}} Driver facts.
21
+ * @returns {{name: string, protocol: string, order: number, auth: string, strict: boolean, headers: Record<string,string>, paths: Record<string,string>}} Driver facts.
22
22
  */
23
23
  export function info(): {
24
24
  name: string;
25
25
  protocol: string;
26
26
  order: number;
27
27
  auth: string;
28
+ strict: boolean;
28
29
  headers: Record<string, string>;
29
30
  paths: Record<string, string>;
30
31
  };
@@ -1,12 +1,13 @@
1
1
  /**
2
2
  * Describe the generic OpenAI-compatible driver (matches last, after every specific driver).
3
- * @returns {{name: string, protocol: string, order: number, auth: string, fields: string[], paths: Record<string,string>}} Driver facts.
3
+ * @returns {{name: string, protocol: string, order: number, auth: string, strict: boolean, fields: string[], paths: Record<string,string>}} Driver facts.
4
4
  */
5
5
  export function info(): {
6
6
  name: string;
7
7
  protocol: string;
8
8
  order: number;
9
9
  auth: string;
10
+ strict: boolean;
10
11
  fields: string[];
11
12
  paths: Record<string, string>;
12
13
  };
@@ -18,12 +18,13 @@
18
18
  */
19
19
  /**
20
20
  * Describe the DeepSeek driver.
21
- * @returns {{name: string, protocol: string, order: number, auth: string, paths: Record<string,string>}} Driver facts.
21
+ * @returns {{name: string, protocol: string, order: number, auth: string, strict: boolean, paths: Record<string,string>}} Driver facts.
22
22
  */
23
23
  export function info(): {
24
24
  name: string;
25
25
  protocol: string;
26
26
  order: number;
27
27
  auth: string;
28
+ strict: boolean;
28
29
  paths: Record<string, string>;
29
30
  };
@@ -18,13 +18,14 @@
18
18
  */
19
19
  /**
20
20
  * Describe the llama.cpp driver.
21
- * @returns {{name: string, protocol: string, order: number, auth: string, probes: string[], fields: string[], paths: Record<string,string>}} Driver facts.
21
+ * @returns {{name: string, protocol: string, order: number, auth: string, strict: boolean, probes: string[], fields: string[], paths: Record<string,string>}} Driver facts.
22
22
  */
23
23
  export function info(): {
24
24
  name: string;
25
25
  protocol: string;
26
26
  order: number;
27
27
  auth: string;
28
+ strict: boolean;
28
29
  probes: string[];
29
30
  fields: string[];
30
31
  paths: Record<string, string>;
@@ -18,13 +18,14 @@
18
18
  */
19
19
  /**
20
20
  * Describe the Ollama driver.
21
- * @returns {{name: string, protocol: string, order: number, auth: string, probes: string[], paths: Record<string,string>}} Driver facts.
21
+ * @returns {{name: string, protocol: string, order: number, auth: string, strict: boolean, probes: string[], paths: Record<string,string>}} Driver facts.
22
22
  */
23
23
  export function info(): {
24
24
  name: string;
25
25
  protocol: string;
26
26
  order: number;
27
27
  auth: string;
28
+ strict: boolean;
28
29
  probes: string[];
29
30
  paths: Record<string, string>;
30
31
  };
@@ -18,12 +18,13 @@
18
18
  */
19
19
  /**
20
20
  * Describe the OpenAI driver.
21
- * @returns {{name: string, protocol: string, order: number, auth: string, paths: Record<string,string>}} Driver facts.
21
+ * @returns {{name: string, protocol: string, order: number, auth: string, strict: boolean, paths: Record<string,string>}} Driver facts.
22
22
  */
23
23
  export function info(): {
24
24
  name: string;
25
25
  protocol: string;
26
26
  order: number;
27
27
  auth: string;
28
+ strict: boolean;
28
29
  paths: Record<string, string>;
29
30
  };
@@ -18,13 +18,14 @@
18
18
  */
19
19
  /**
20
20
  * Describe the SGLang driver.
21
- * @returns {{name: string, protocol: string, order: number, auth: string, probes: string[], fields: string[], paths: Record<string,string>}} Driver facts.
21
+ * @returns {{name: string, protocol: string, order: number, auth: string, strict: boolean, probes: string[], fields: string[], paths: Record<string,string>}} Driver facts.
22
22
  */
23
23
  export function info(): {
24
24
  name: string;
25
25
  protocol: string;
26
26
  order: number;
27
27
  auth: string;
28
+ strict: boolean;
28
29
  probes: string[];
29
30
  fields: string[];
30
31
  paths: Record<string, string>;
@@ -1,12 +1,13 @@
1
1
  /**
2
2
  * Describe the vLLM driver.
3
- * @returns {{name: string, protocol: string, order: number, auth: string, fields: string[], paths: Record<string,string>}} Driver facts.
3
+ * @returns {{name: string, protocol: string, order: number, auth: string, strict: boolean, fields: string[], paths: Record<string,string>}} Driver facts.
4
4
  */
5
5
  export function info(): {
6
6
  name: string;
7
7
  protocol: string;
8
8
  order: number;
9
9
  auth: string;
10
+ strict: boolean;
10
11
  fields: string[];
11
12
  paths: Record<string, string>;
12
13
  };
@@ -0,0 +1,15 @@
1
+ /**
2
+ * Forward `POST /v1/embeddings` to an OpenAI-protocol upstream that lists the model. The body and the reply pass through unchanged
3
+ * apart from `model`, which is the upstream's name for it (after `aliases`). The model is routed like a chat model: aliases, then the
4
+ * pool of endpoints serving it, which fails over on a connection error or a 5xx. Errors are replies in the OpenAI error shape (the
5
+ * return value, not a throw): `400` for a body that is not an object or has no `model`, `404` when no upstream serves the model and `400`
6
+ * (`embeddings_unsupported`) when the model is served only by Anthropic-protocol upstreams, which have no embeddings API, and `499` (`aborted`) when `signal` aborts.
7
+ * @param {any} body - OpenAI embeddings request body (`{ model, input, ... }`).
8
+ * @param {any} [options] - `{ signal, headers, log }`: abort signal for the upstream call, extra upstream headers and the logger events of this request also go to (used by the server).
9
+ * @returns {Promise<{status: number, headers: Record<string,string>, body: any}>} The upstream's JSON reply, or an OpenAI-shaped error reply.
10
+ */
11
+ export function send(body: any, options?: any): Promise<{
12
+ status: number;
13
+ headers: Record<string, string>;
14
+ body: any;
15
+ }>;
@@ -1,6 +1,6 @@
1
1
  /**
2
2
  * Fastify plugin. Anthropic front: `POST /v1/messages`, `POST /v1/messages/count_tokens`, `GET /v1/models`, `GET /v1/models/{id}`,
3
- * `GET|HEAD /` and `/api/hello`. OpenAI front: `POST /v1/chat/completions`, `GET /v1/models`, `GET /v1/models/{id}`, `GET|HEAD /`. Responses stream as
3
+ * `GET|HEAD /` and `/api/hello`. OpenAI front: `POST /v1/chat/completions`, `POST /v1/embeddings`, `GET /v1/models`, `GET /v1/models/{id}`, `GET|HEAD /`. Responses stream as
4
4
  * `text/event-stream` with keep-alive pings; a client disconnect aborts the upstream request. Register it with
5
5
  * `app.register(api.server.plugin, { apiKey })`. `authorize(request)` (sync or async) runs after `apiKey` and the body parse and decides
6
6
  * with `true` / `{ ok: true, context }` or `false` / `{ ok: false, status, message }`; `onUsage(report, context)` receives each request's usage report. When the Fastify `logger` is enabled, events also go to it as structured lines (`{ event }` plus the message), through the request's logger for events of a request.
@@ -10,6 +10,7 @@
10
10
  * @property {number} [detectTimeout] Probe timeout in ms (default 5000).
11
11
  * @property {(url: string, init?: any) => Promise<Response>} [transport] fetch-compatible function used for all upstream calls.
12
12
  * @property {boolean | string} [logs] Logging (default `false`: no console output, no file). `true` prints warnings and errors on the console, once per message; a path appends every event as a JSON line to that file (relative paths resolve against `process.cwd()`, a missing directory is created, a file that cannot be opened is reported once on the console and logging stays off).
13
+ * @property {false | string | ((record: object) => any)} [capture] Request and response capture for debugging (default `false`: off). A directory path writes one JSON file per exchange (`<ms>-<id>.json`, the directory is created); a function is called with each record. A record is `{ time, id, front, requested, model, endpoint, stream, client: { body }, upstream: { url, headers, body }, response }`; credential headers (names containing key, token, secret, auth, cookie or password) are replaced by `[redacted]`. A failing sink is reported once as a warning and never affects the reply (see docs/events.md).
13
14
  * @property {boolean} [events] Emit events (default `true`); `false` turns emitting off. Logging does not depend on it.
14
15
  * @property {(event: import("./lib/events.mjs").TranslatorEvent) => any} [onEvent] Callback for every event, registered before startup so it also sees startup events (see docs/events.md).
15
16
  * @property {boolean} [silent] Deprecated alias for `logs: false` (the default); `logs` wins when both are given.
@@ -72,6 +73,10 @@ export type TranslatorConfig = {
72
73
  * Logging (default `false`: no console output, no file). `true` prints warnings and errors on the console, once per message; a path appends every event as a JSON line to that file (relative paths resolve against `process.cwd()`, a missing directory is created, a file that cannot be opened is reported once on the console and logging stays off).
73
74
  */
74
75
  logs?: boolean | string;
76
+ /**
77
+ * Request and response capture for debugging (default `false`: off). A directory path writes one JSON file per exchange (`<ms>-<id>.json`, the directory is created); a function is called with each record. A record is `{ time, id, front, requested, model, endpoint, stream, client: { body }, upstream: { url, headers, body }, response }`; credential headers (names containing key, token, secret, auth, cookie or password) are replaced by `[redacted]`. A failing sink is reported once as a warning and never affects the reply (see docs/events.md).
78
+ */
79
+ capture?: false | string | ((record: object) => any);
75
80
  /**
76
81
  * Emit events (default `true`); `false` turns emitting off. Logging does not depend on it.
77
82
  */
@@ -0,0 +1,36 @@
1
+ /**
2
+ * Copy a header map with the value of every credential header replaced. A header is a credential when its name contains `key`,
3
+ * `token`, `secret`, `auth`, `cookie` or `password` (this covers `authorization`, `proxy-authorization`, `x-api-key`, `api-key`).
4
+ * @param {Record<string, any> | undefined} headers - Header map.
5
+ * @returns {Record<string, any>} Copy with lowercase names.
6
+ */
7
+ export function redact(headers: Record<string, any> | undefined): Record<string, any>;
8
+ /**
9
+ * Validate the `capture` option.
10
+ * @param {any} value - Raw option.
11
+ * @returns {boolean} True when the value is `false`/`undefined`, a non-empty directory path or a function.
12
+ */
13
+ export function valid(value: any): boolean;
14
+ /**
15
+ * Begin the record of one exchange. With `capture` unset this returns a stand-in that does nothing.
16
+ * @param {{config: any, state: any}} ctx - Translator `{ config, state }`.
17
+ * @param {{body: any, sent: any, front: string, requested: string | null, headers?: Record<string, string>, log?: any}} exchange - Client body, the upstream request (result of `request`), front and the logger of the request.
18
+ * @returns {{raw: (source: any) => any, events: (source: any) => any, reply: (status: number, headers: Record<string, string>, body: any) => void}} Recorder: `raw` taps the upstream stream bytes, `events` taps the client events and writes the record when the stream ends, `reply` writes the record of a non-stream reply.
19
+ */
20
+ export function start({ config, state }: {
21
+ config: any;
22
+ state: any;
23
+ }, exchange: {
24
+ body: any;
25
+ sent: any;
26
+ front: string;
27
+ requested: string | null;
28
+ headers?: Record<string, string>;
29
+ log?: any;
30
+ }): {
31
+ raw: (source: any) => any;
32
+ events: (source: any) => any;
33
+ reply: (status: number, headers: Record<string, string>, body: any) => void;
34
+ };
35
+ /** Most characters of a stream kept per exchange, for the raw upstream text and for the client events each. */
36
+ export const STREAM_LIMIT: 1000000;
@@ -0,0 +1,13 @@
1
+ /**
2
+ * Sampling fields to add to an OpenAI-protocol request: the `sampling` setting for the upstream, overridden by `metadata.sampling` of an
3
+ * Anthropic request. Fields the body already carries are kept. A field the upstream's driver does not accept follows `unknownFields`:
4
+ * dropped (listed in `dropped`) or rejected. Names are the canonical vLLM ones; a driver such as llama.cpp renames them in its `request`.
5
+ * @param {any} body - Converted or forwarded Chat Completions body.
6
+ * @param {any} extras - `metadata.sampling` of the client request (ignored unless a plain object).
7
+ * @param {any} settings - Resolved settings (`sampling`, `unknownFields`).
8
+ * @param {string[]} [fields] - Extension fields the upstream's driver accepts.
9
+ * @param {string[]} [dropped] - Receives the names that were left out.
10
+ * @returns {any} The body, with the fields added (a copy when anything was added).
11
+ * @throws {TranslatorError} When a field is not accepted and `unknownFields` is `"error"`.
12
+ */
13
+ export function addSampling(body: any, extras: any, settings: any, fields?: string[], dropped?: string[]): any;
@@ -39,6 +39,7 @@ export function resolve(...layers: (Partial<Settings> | null | undefined)[]): Se
39
39
  * @property {boolean} responseFormat - Map `output_config.format` to `response_format`.
40
40
  * @property {boolean} parallelToolCalls - Map `disable_parallel_tool_use` to `parallel_tool_calls: false`.
41
41
  * @property {"drop"|"error"} unknownFields - Unknown request fields are dropped (and listed in `state.dropped`) or rejected.
42
+ * @property {Record<string, any>} sampling - Sampling extension fields (`min_p`, `repetition_penalty`, `seed`, ...) added to requests for an OpenAI-protocol upstream when the request does not carry them, merged key by key. A request may set them itself (OpenAI front, or `metadata.sampling` on the Anthropic front). A field the upstream's driver does not accept follows `unknownFields`.
42
43
  * @property {number} pingInterval - Milliseconds of upstream silence before the SSE writer emits a `ping` (0 disables).
43
44
  * @property {"buffer"|"stream"} toolStreaming - OpenAI to Anthropic streams: `buffer` holds each tool call until its arguments are complete and emits one whole `tool_use` block (text and thinking keep streaming and appear before the block); `stream` forwards argument fragments incrementally and holds text that arrives meanwhile until the tool blocks are done.
44
45
  * @property {boolean} effortPassthrough - vLLM driver: send `xhigh` and `max` reasoning effort unchanged instead of clamping them to `high`.
@@ -182,6 +183,10 @@ export type Settings = {
182
183
  * - Unknown request fields are dropped (and listed in `state.dropped`) or rejected.
183
184
  */
184
185
  unknownFields: "drop" | "error";
186
+ /**
187
+ * - Sampling extension fields (`min_p`, `repetition_penalty`, `seed`, ...) added to requests for an OpenAI-protocol upstream when the request does not carry them, merged key by key. A request may set them itself (OpenAI front, or `metadata.sampling` on the Anthropic front). A field the upstream's driver does not accept follows `unknownFields`.
188
+ */
189
+ sampling: Record<string, any>;
185
190
  /**
186
191
  * - Milliseconds of upstream silence before the SSE writer emits a `ping` (0 disables).
187
192
  */
@@ -0,0 +1,26 @@
1
+ /**
2
+ * Walk a schema and list what falls outside the strict subset: an object schema must say `additionalProperties: false` and list every
3
+ * property in `required`, and the keywords in `UNSUPPORTED` are not allowed anywhere.
4
+ * @param {any} schema - A tool's JSON Schema.
5
+ * @param {string} [at] - Path of `schema` within the tool's schema.
6
+ * @param {string[]} [out] - Problems found so far.
7
+ * @returns {string[]} One line per problem, each starting with its path (`$` is the schema's root).
8
+ */
9
+ export function problems(schema: any, at?: string, out?: string[]): string[];
10
+ /**
11
+ * The tools of an upstream body that ask for strict use, with the problems of their schemas. Reads both wire shapes: Chat Completions
12
+ * (`function.strict`, `function.parameters`) and Anthropic Messages (`strict`, `input_schema`).
13
+ * @param {any} body - Upstream request body.
14
+ * @returns {{name: string, problems: string[]}[]} Strict tools in request order.
15
+ */
16
+ export function strictTools(body: any): {
17
+ name: string;
18
+ problems: string[];
19
+ }[];
20
+ /**
21
+ * Send the tools whose schema is outside the strict subset without `strict` (the key is removed, not set to false). Reads both wire shapes
22
+ * like `strictTools`. Nothing in `body` is modified: tools that change are rebuilt, and the body is copied only when one did.
23
+ * @param {any} body - Upstream request body.
24
+ * @returns {any} `body` itself when every strict tool conforms, otherwise a copy with the rebuilt `tools`.
25
+ */
26
+ export function relax(body: any): any;