@earendil-works/pi-ai 0.79.10 → 0.80.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +633 -521
- package/dist/api/anthropic-messages.d.ts +71 -0
- package/dist/api/anthropic-messages.d.ts.map +1 -0
- package/dist/api/anthropic-messages.js +965 -0
- package/dist/api/anthropic-messages.js.map +1 -0
- package/dist/api/anthropic-messages.lazy.d.ts +3 -0
- package/dist/api/anthropic-messages.lazy.d.ts.map +1 -0
- package/dist/api/anthropic-messages.lazy.js +3 -0
- package/dist/api/anthropic-messages.lazy.js.map +1 -0
- package/dist/api/azure-openai-responses.d.ts +15 -0
- package/dist/api/azure-openai-responses.d.ts.map +1 -0
- package/dist/api/azure-openai-responses.js +225 -0
- package/dist/api/azure-openai-responses.js.map +1 -0
- package/dist/api/azure-openai-responses.lazy.d.ts +3 -0
- package/dist/api/azure-openai-responses.lazy.d.ts.map +1 -0
- package/dist/api/azure-openai-responses.lazy.js +3 -0
- package/dist/api/azure-openai-responses.lazy.js.map +1 -0
- package/dist/api/bedrock-converse-stream.d.ts +38 -0
- package/dist/api/bedrock-converse-stream.d.ts.map +1 -0
- package/dist/api/bedrock-converse-stream.js +863 -0
- package/dist/api/bedrock-converse-stream.js.map +1 -0
- package/dist/api/bedrock-converse-stream.lazy.d.ts +9 -0
- package/dist/api/bedrock-converse-stream.lazy.d.ts.map +1 -0
- package/dist/api/bedrock-converse-stream.lazy.js +30 -0
- package/dist/api/bedrock-converse-stream.lazy.js.map +1 -0
- package/dist/{providers → api}/cloudflare.d.ts +0 -4
- package/dist/api/cloudflare.d.ts.map +1 -0
- package/dist/{providers → api}/cloudflare.js +0 -18
- package/dist/api/cloudflare.js.map +1 -0
- package/dist/api/github-copilot-headers.d.ts.map +1 -0
- package/dist/api/github-copilot-headers.js.map +1 -0
- package/dist/api/google-generative-ai.d.ts +13 -0
- package/dist/api/google-generative-ai.d.ts.map +1 -0
- package/dist/api/google-generative-ai.js +405 -0
- package/dist/api/google-generative-ai.js.map +1 -0
- package/dist/api/google-generative-ai.lazy.d.ts +3 -0
- package/dist/api/google-generative-ai.lazy.d.ts.map +1 -0
- package/dist/api/google-generative-ai.lazy.js +3 -0
- package/dist/api/google-generative-ai.lazy.js.map +1 -0
- package/dist/api/google-shared.d.ts.map +1 -0
- package/dist/api/google-shared.js.map +1 -0
- package/dist/api/google-vertex.d.ts +15 -0
- package/dist/api/google-vertex.d.ts.map +1 -0
- package/dist/api/google-vertex.js +454 -0
- package/dist/api/google-vertex.js.map +1 -0
- package/dist/api/google-vertex.lazy.d.ts +3 -0
- package/dist/api/google-vertex.lazy.d.ts.map +1 -0
- package/dist/api/google-vertex.lazy.js +3 -0
- package/dist/api/google-vertex.lazy.js.map +1 -0
- package/dist/api/lazy.d.ts +15 -0
- package/dist/api/lazy.d.ts.map +1 -0
- package/dist/api/lazy.js +59 -0
- package/dist/api/lazy.js.map +1 -0
- package/dist/api/mistral-conversations.d.ts +25 -0
- package/dist/api/mistral-conversations.d.ts.map +1 -0
- package/dist/api/mistral-conversations.js +555 -0
- package/dist/api/mistral-conversations.js.map +1 -0
- package/dist/api/mistral-conversations.lazy.d.ts +3 -0
- package/dist/api/mistral-conversations.lazy.d.ts.map +1 -0
- package/dist/api/mistral-conversations.lazy.js +3 -0
- package/dist/api/mistral-conversations.lazy.js.map +1 -0
- package/dist/{providers → api}/openai-codex-responses.d.ts +2 -3
- package/dist/api/openai-codex-responses.d.ts.map +1 -0
- package/dist/{providers → api}/openai-codex-responses.js +66 -45
- package/dist/api/openai-codex-responses.js.map +1 -0
- package/dist/api/openai-codex-responses.lazy.d.ts +3 -0
- package/dist/api/openai-codex-responses.lazy.d.ts.map +1 -0
- package/dist/api/openai-codex-responses.lazy.js +3 -0
- package/dist/api/openai-codex-responses.lazy.js.map +1 -0
- package/dist/{providers → api}/openai-completions.d.ts +2 -3
- package/dist/api/openai-completions.d.ts.map +1 -0
- package/dist/{providers → api}/openai-completions.js +68 -128
- package/dist/api/openai-completions.js.map +1 -0
- package/dist/api/openai-completions.lazy.d.ts +3 -0
- package/dist/api/openai-completions.lazy.d.ts.map +1 -0
- package/dist/api/openai-completions.lazy.js +3 -0
- package/dist/api/openai-completions.lazy.js.map +1 -0
- package/dist/api/openai-prompt-cache.d.ts.map +1 -0
- package/dist/api/openai-prompt-cache.js.map +1 -0
- package/dist/api/openai-responses-shared.d.ts.map +1 -0
- package/dist/{providers → api}/openai-responses-shared.js +37 -29
- package/dist/api/openai-responses-shared.js.map +1 -0
- package/dist/{providers → api}/openai-responses.d.ts +2 -3
- package/dist/api/openai-responses.d.ts.map +1 -0
- package/dist/{providers → api}/openai-responses.js +27 -32
- package/dist/api/openai-responses.js.map +1 -0
- package/dist/api/openai-responses.lazy.d.ts +3 -0
- package/dist/api/openai-responses.lazy.d.ts.map +1 -0
- package/dist/api/openai-responses.lazy.js +3 -0
- package/dist/api/openai-responses.lazy.js.map +1 -0
- package/dist/api/openrouter-images.d.ts +3 -0
- package/dist/api/openrouter-images.d.ts.map +1 -0
- package/dist/{providers/images/openrouter.js → api/openrouter-images.js} +5 -15
- package/dist/api/openrouter-images.js.map +1 -0
- package/dist/api/openrouter-images.lazy.d.ts +3 -0
- package/dist/api/openrouter-images.lazy.d.ts.map +1 -0
- package/dist/api/openrouter-images.lazy.js +4 -0
- package/dist/api/openrouter-images.lazy.js.map +1 -0
- package/dist/api/simple-options.d.ts.map +1 -0
- package/dist/api/simple-options.js.map +1 -0
- package/dist/api/transform-messages.d.ts.map +1 -0
- package/dist/api/transform-messages.js.map +1 -0
- package/dist/auth/context.d.ts +7 -0
- package/dist/auth/context.d.ts.map +1 -0
- package/dist/auth/context.js +42 -0
- package/dist/auth/context.js.map +1 -0
- package/dist/auth/credential-store.d.ts +16 -0
- package/dist/auth/credential-store.d.ts.map +1 -0
- package/dist/auth/credential-store.js +37 -0
- package/dist/auth/credential-store.js.map +1 -0
- package/dist/auth/helpers.d.ts +20 -0
- package/dist/auth/helpers.d.ts.map +1 -0
- package/dist/auth/helpers.js +46 -0
- package/dist/auth/helpers.js.map +1 -0
- package/dist/auth/resolve.d.ts +22 -0
- package/dist/auth/resolve.d.ts.map +1 -0
- package/dist/auth/resolve.js +86 -0
- package/dist/auth/resolve.js.map +1 -0
- package/dist/auth/types.d.ts +180 -0
- package/dist/auth/types.d.ts.map +1 -0
- package/dist/auth/types.js +2 -0
- package/dist/auth/types.js.map +1 -0
- package/dist/bedrock-provider.d.ts +2 -4
- package/dist/bedrock-provider.d.ts.map +1 -1
- package/dist/bedrock-provider.js +3 -4
- package/dist/bedrock-provider.js.map +1 -1
- package/dist/compat.d.ts +64 -0
- package/dist/compat.d.ts.map +1 -0
- package/dist/compat.js +181 -0
- package/dist/compat.js.map +1 -0
- package/dist/images-api-registry.d.ts +0 -1
- package/dist/images-api-registry.d.ts.map +1 -1
- package/dist/images-api-registry.js +0 -3
- package/dist/images-api-registry.js.map +1 -1
- package/dist/images-models.d.ts +93 -0
- package/dist/images-models.d.ts.map +1 -0
- package/dist/images-models.js +138 -0
- package/dist/images-models.js.map +1 -0
- package/dist/images.d.ts +1 -0
- package/dist/images.d.ts.map +1 -1
- package/dist/images.js +1 -0
- package/dist/images.js.map +1 -1
- package/dist/index.d.ts +29 -2
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +17 -4
- package/dist/index.js.map +1 -1
- package/dist/models.d.ts +133 -9
- package/dist/models.d.ts.map +1 -1
- package/dist/models.generated.d.ts +1766 -156
- package/dist/models.generated.d.ts.map +1 -1
- package/dist/models.generated.js +70 -17172
- package/dist/models.generated.js.map +1 -1
- package/dist/models.js +178 -17
- package/dist/models.js.map +1 -1
- package/dist/providers/all.d.ts +21 -0
- package/dist/providers/all.d.ts.map +1 -0
- package/dist/providers/all.js +114 -0
- package/dist/providers/all.js.map +1 -0
- package/dist/providers/amazon-bedrock.d.ts +2 -38
- package/dist/providers/amazon-bedrock.d.ts.map +1 -1
- package/dist/providers/amazon-bedrock.js +35 -865
- package/dist/providers/amazon-bedrock.js.map +1 -1
- package/dist/providers/amazon-bedrock.models.d.ts +1718 -0
- package/dist/providers/amazon-bedrock.models.d.ts.map +1 -0
- package/dist/providers/amazon-bedrock.models.js +1675 -0
- package/dist/providers/amazon-bedrock.models.js.map +1 -0
- package/dist/providers/ant-ling.d.ts +3 -0
- package/dist/providers/ant-ling.d.ts.map +1 -0
- package/dist/providers/ant-ling.js +15 -0
- package/dist/providers/ant-ling.js.map +1 -0
- package/dist/providers/ant-ling.models.d.ts +86 -0
- package/dist/providers/ant-ling.models.d.ts.map +1 -0
- package/dist/providers/ant-ling.models.js +60 -0
- package/dist/providers/ant-ling.models.js.map +1 -0
- package/dist/providers/anthropic.d.ts +2 -71
- package/dist/providers/anthropic.d.ts.map +1 -1
- package/dist/providers/anthropic.js +17 -973
- package/dist/providers/anthropic.js.map +1 -1
- package/dist/providers/anthropic.models.d.ts +458 -0
- package/dist/providers/anthropic.models.d.ts.map +1 -0
- package/dist/providers/anthropic.models.js +439 -0
- package/dist/providers/anthropic.models.js.map +1 -0
- package/dist/providers/azure-openai-responses.d.ts +2 -15
- package/dist/providers/azure-openai-responses.d.ts.map +1 -1
- package/dist/providers/azure-openai-responses.js +11 -230
- package/dist/providers/azure-openai-responses.js.map +1 -1
- package/dist/providers/azure-openai-responses.models.d.ts +804 -0
- package/dist/providers/azure-openai-responses.models.d.ts.map +1 -0
- package/dist/providers/azure-openai-responses.models.js +743 -0
- package/dist/providers/azure-openai-responses.models.js.map +1 -0
- package/dist/providers/cerebras.d.ts +3 -0
- package/dist/providers/cerebras.d.ts.map +1 -0
- package/dist/providers/cerebras.js +15 -0
- package/dist/providers/cerebras.js.map +1 -0
- package/dist/providers/cerebras.models.d.ts +45 -0
- package/dist/providers/cerebras.models.d.ts.map +1 -0
- package/dist/providers/cerebras.models.js +41 -0
- package/dist/providers/cerebras.models.js.map +1 -0
- package/dist/providers/cloudflare-ai-gateway.d.ts +3 -0
- package/dist/providers/cloudflare-ai-gateway.d.ts.map +1 -0
- package/dist/providers/cloudflare-ai-gateway.js +20 -0
- package/dist/providers/cloudflare-ai-gateway.js.map +1 -0
- package/dist/providers/cloudflare-ai-gateway.models.d.ts +765 -0
- package/dist/providers/cloudflare-ai-gateway.models.d.ts.map +1 -0
- package/dist/providers/cloudflare-ai-gateway.models.js +666 -0
- package/dist/providers/cloudflare-ai-gateway.models.js.map +1 -0
- package/dist/providers/cloudflare-auth.d.ts +4 -0
- package/dist/providers/cloudflare-auth.d.ts.map +1 -0
- package/dist/providers/cloudflare-auth.js +87 -0
- package/dist/providers/cloudflare-auth.js.map +1 -0
- package/dist/providers/cloudflare-workers-ai.d.ts +3 -0
- package/dist/providers/cloudflare-workers-ai.d.ts.map +1 -0
- package/dist/providers/cloudflare-workers-ai.js +14 -0
- package/dist/providers/cloudflare-workers-ai.js.map +1 -0
- package/dist/providers/cloudflare-workers-ai.models.d.ts +302 -0
- package/dist/providers/cloudflare-workers-ai.models.d.ts.map +1 -0
- package/dist/providers/cloudflare-workers-ai.models.js +239 -0
- package/dist/providers/cloudflare-workers-ai.models.js.map +1 -0
- package/dist/providers/deepseek.d.ts +3 -0
- package/dist/providers/deepseek.d.ts.map +1 -0
- package/dist/providers/deepseek.js +15 -0
- package/dist/providers/deepseek.js.map +1 -0
- package/dist/providers/deepseek.models.d.ts +63 -0
- package/dist/providers/deepseek.models.d.ts.map +1 -0
- package/dist/providers/deepseek.models.js +43 -0
- package/dist/providers/deepseek.models.js.map +1 -0
- package/dist/providers/faux.d.ts +43 -2
- package/dist/providers/faux.d.ts.map +1 -1
- package/dist/providers/faux.js +34 -7
- package/dist/providers/faux.js.map +1 -1
- package/dist/providers/fireworks.d.ts +3 -0
- package/dist/providers/fireworks.d.ts.map +1 -0
- package/dist/providers/fireworks.js +19 -0
- package/dist/providers/fireworks.js.map +1 -0
- package/dist/providers/fireworks.models.d.ts +353 -0
- package/dist/providers/fireworks.models.d.ts.map +1 -0
- package/dist/providers/fireworks.models.js +276 -0
- package/dist/providers/fireworks.models.js.map +1 -0
- package/dist/providers/github-copilot.d.ts +3 -0
- package/dist/providers/github-copilot.d.ts.map +1 -0
- package/dist/providers/github-copilot.js +25 -0
- package/dist/providers/github-copilot.js.map +1 -0
- package/dist/providers/github-copilot.models.d.ts +616 -0
- package/dist/providers/github-copilot.models.d.ts.map +1 -0
- package/dist/providers/github-copilot.models.js +426 -0
- package/dist/providers/github-copilot.models.js.map +1 -0
- package/dist/providers/google-vertex.d.ts +2 -15
- package/dist/providers/google-vertex.d.ts.map +1 -1
- package/dist/providers/google-vertex.js +30 -455
- package/dist/providers/google-vertex.js.map +1 -1
- package/dist/providers/google-vertex.models.d.ts +202 -0
- package/dist/providers/google-vertex.models.d.ts.map +1 -0
- package/dist/providers/google-vertex.models.js +182 -0
- package/dist/providers/google-vertex.models.js.map +1 -0
- package/dist/providers/google.d.ts +2 -13
- package/dist/providers/google.d.ts.map +1 -1
- package/dist/providers/google.js +12 -408
- package/dist/providers/google.js.map +1 -1
- package/dist/providers/google.models.d.ts +328 -0
- package/dist/providers/google.models.d.ts.map +1 -0
- package/dist/providers/google.models.js +288 -0
- package/dist/providers/google.models.js.map +1 -0
- package/dist/providers/groq.d.ts +3 -0
- package/dist/providers/groq.d.ts.map +1 -0
- package/dist/providers/groq.js +15 -0
- package/dist/providers/groq.js.map +1 -0
- package/dist/providers/groq.models.d.ts +128 -0
- package/dist/providers/groq.models.d.ts.map +1 -0
- package/dist/providers/groq.models.js +125 -0
- package/dist/providers/groq.models.js.map +1 -0
- package/dist/providers/huggingface.d.ts +3 -0
- package/dist/providers/huggingface.d.ts.map +1 -0
- package/dist/providers/huggingface.js +15 -0
- package/dist/providers/huggingface.js.map +1 -0
- package/dist/providers/huggingface.models.d.ts +883 -0
- package/dist/providers/huggingface.models.d.ts.map +1 -0
- package/dist/providers/huggingface.models.js +797 -0
- package/dist/providers/huggingface.models.js.map +1 -0
- package/dist/providers/images/{openrouter.d.ts → register-builtins.d.ts} +2 -2
- package/dist/providers/images/register-builtins.d.ts.map +1 -0
- package/dist/providers/images/register-builtins.js +34 -0
- package/dist/providers/images/register-builtins.js.map +1 -0
- package/dist/providers/kimi-coding.d.ts +3 -0
- package/dist/providers/kimi-coding.d.ts.map +1 -0
- package/dist/providers/kimi-coding.js +15 -0
- package/dist/providers/kimi-coding.js.map +1 -0
- package/dist/providers/kimi-coding.models.d.ts +63 -0
- package/dist/providers/kimi-coding.models.d.ts.map +1 -0
- package/dist/providers/kimi-coding.models.js +59 -0
- package/dist/providers/kimi-coding.models.js.map +1 -0
- package/dist/providers/minimax-cn.d.ts +3 -0
- package/dist/providers/minimax-cn.d.ts.map +1 -0
- package/dist/providers/minimax-cn.js +15 -0
- package/dist/providers/minimax-cn.js.map +1 -0
- package/dist/providers/minimax-cn.models.d.ts +54 -0
- package/dist/providers/minimax-cn.models.d.ts.map +1 -0
- package/dist/providers/minimax-cn.models.js +56 -0
- package/dist/providers/minimax-cn.models.js.map +1 -0
- package/dist/providers/minimax.d.ts +3 -0
- package/dist/providers/minimax.d.ts.map +1 -0
- package/dist/providers/minimax.js +15 -0
- package/dist/providers/minimax.js.map +1 -0
- package/dist/providers/minimax.models.d.ts +54 -0
- package/dist/providers/minimax.models.d.ts.map +1 -0
- package/dist/providers/minimax.models.js +56 -0
- package/dist/providers/minimax.models.js.map +1 -0
- package/dist/providers/mistral.d.ts +2 -25
- package/dist/providers/mistral.d.ts.map +1 -1
- package/dist/providers/mistral.js +12 -560
- package/dist/providers/mistral.js.map +1 -1
- package/dist/providers/mistral.models.d.ts +513 -0
- package/dist/providers/mistral.models.d.ts.map +1 -0
- package/dist/providers/mistral.models.js +515 -0
- package/dist/providers/mistral.models.js.map +1 -0
- package/dist/providers/moonshotai-cn.d.ts +3 -0
- package/dist/providers/moonshotai-cn.d.ts.map +1 -0
- package/dist/providers/moonshotai-cn.js +15 -0
- package/dist/providers/moonshotai-cn.js.map +1 -0
- package/dist/providers/moonshotai-cn.models.d.ts +234 -0
- package/dist/providers/moonshotai-cn.models.d.ts.map +1 -0
- package/dist/providers/moonshotai-cn.models.js +169 -0
- package/dist/providers/moonshotai-cn.models.js.map +1 -0
- package/dist/providers/moonshotai.d.ts +3 -0
- package/dist/providers/moonshotai.d.ts.map +1 -0
- package/dist/providers/moonshotai.js +15 -0
- package/dist/providers/moonshotai.js.map +1 -0
- package/dist/providers/moonshotai.models.d.ts +234 -0
- package/dist/providers/moonshotai.models.d.ts.map +1 -0
- package/dist/providers/moonshotai.models.js +169 -0
- package/dist/providers/moonshotai.models.js.map +1 -0
- package/dist/providers/nvidia.d.ts +3 -0
- package/dist/providers/nvidia.d.ts.map +1 -0
- package/dist/providers/nvidia.js +15 -0
- package/dist/providers/nvidia.js.map +1 -0
- package/dist/providers/nvidia.models.d.ts +535 -0
- package/dist/providers/nvidia.models.d.ts.map +1 -0
- package/dist/providers/nvidia.models.js +366 -0
- package/dist/providers/nvidia.models.js.map +1 -0
- package/dist/providers/openai-codex.d.ts +3 -0
- package/dist/providers/openai-codex.d.ts.map +1 -0
- package/dist/providers/openai-codex.js +18 -0
- package/dist/providers/openai-codex.js.map +1 -0
- package/dist/providers/openai-codex.models.d.ts +87 -0
- package/dist/providers/openai-codex.models.d.ts.map +1 -0
- package/dist/providers/openai-codex.models.js +77 -0
- package/dist/providers/openai-codex.models.js.map +1 -0
- package/dist/providers/openai.d.ts +3 -0
- package/dist/providers/openai.d.ts.map +1 -0
- package/dist/providers/openai.js +15 -0
- package/dist/providers/openai.js.map +1 -0
- package/dist/providers/openai.models.d.ts +805 -0
- package/dist/providers/openai.models.d.ts.map +1 -0
- package/dist/providers/openai.models.js +743 -0
- package/dist/providers/openai.models.js.map +1 -0
- package/dist/providers/opencode-go.d.ts +3 -0
- package/dist/providers/opencode-go.d.ts.map +1 -0
- package/dist/providers/opencode-go.js +18 -0
- package/dist/providers/opencode-go.js.map +1 -0
- package/dist/providers/opencode-go.models.d.ts +309 -0
- package/dist/providers/opencode-go.models.d.ts.map +1 -0
- package/dist/providers/opencode-go.models.js +240 -0
- package/dist/providers/opencode-go.models.js.map +1 -0
- package/dist/providers/opencode.d.ts +3 -0
- package/dist/providers/opencode.d.ts.map +1 -0
- package/dist/providers/opencode.js +22 -0
- package/dist/providers/opencode.js.map +1 -0
- package/dist/providers/opencode.models.d.ts +976 -0
- package/dist/providers/opencode.models.d.ts.map +1 -0
- package/dist/providers/opencode.models.js +815 -0
- package/dist/providers/opencode.models.js.map +1 -0
- package/dist/providers/openrouter-images.d.ts +3 -0
- package/dist/providers/openrouter-images.d.ts.map +1 -0
- package/dist/providers/openrouter-images.js +14 -0
- package/dist/providers/openrouter-images.js.map +1 -0
- package/dist/providers/openrouter.d.ts +3 -0
- package/dist/providers/openrouter.d.ts.map +1 -0
- package/dist/providers/openrouter.js +15 -0
- package/dist/providers/openrouter.js.map +1 -0
- package/dist/providers/openrouter.models.d.ts +5405 -0
- package/dist/providers/openrouter.models.d.ts.map +1 -0
- package/dist/providers/openrouter.models.js +4635 -0
- package/dist/providers/openrouter.models.js.map +1 -0
- package/dist/providers/together.d.ts +3 -0
- package/dist/providers/together.d.ts.map +1 -0
- package/dist/providers/together.js +15 -0
- package/dist/providers/together.js.map +1 -0
- package/dist/providers/together.models.d.ts +567 -0
- package/dist/providers/together.models.d.ts.map +1 -0
- package/dist/providers/together.models.js +361 -0
- package/dist/providers/together.models.js.map +1 -0
- package/dist/providers/vercel-ai-gateway.d.ts +3 -0
- package/dist/providers/vercel-ai-gateway.d.ts.map +1 -0
- package/dist/providers/vercel-ai-gateway.js +15 -0
- package/dist/providers/vercel-ai-gateway.js.map +1 -0
- package/dist/providers/vercel-ai-gateway.models.d.ts +2938 -0
- package/dist/providers/vercel-ai-gateway.models.d.ts.map +1 -0
- package/dist/providers/vercel-ai-gateway.models.js +2897 -0
- package/dist/providers/vercel-ai-gateway.models.js.map +1 -0
- package/dist/providers/xai.d.ts +3 -0
- package/dist/providers/xai.d.ts.map +1 -0
- package/dist/providers/xai.js +15 -0
- package/dist/providers/xai.js.map +1 -0
- package/dist/providers/xai.models.d.ts +157 -0
- package/dist/providers/xai.models.d.ts.map +1 -0
- package/dist/providers/xai.models.js +131 -0
- package/dist/providers/xai.models.js.map +1 -0
- package/dist/providers/xiaomi-token-plan-ams.d.ts +3 -0
- package/dist/providers/xiaomi-token-plan-ams.d.ts.map +1 -0
- package/dist/providers/xiaomi-token-plan-ams.js +15 -0
- package/dist/providers/xiaomi-token-plan-ams.js.map +1 -0
- package/dist/providers/xiaomi-token-plan-ams.models.d.ts +108 -0
- package/dist/providers/xiaomi-token-plan-ams.models.d.ts.map +1 -0
- package/dist/providers/xiaomi-token-plan-ams.models.js +95 -0
- package/dist/providers/xiaomi-token-plan-ams.models.js.map +1 -0
- package/dist/providers/xiaomi-token-plan-cn.d.ts +3 -0
- package/dist/providers/xiaomi-token-plan-cn.d.ts.map +1 -0
- package/dist/providers/xiaomi-token-plan-cn.js +15 -0
- package/dist/providers/xiaomi-token-plan-cn.js.map +1 -0
- package/dist/providers/xiaomi-token-plan-cn.models.d.ts +108 -0
- package/dist/providers/xiaomi-token-plan-cn.models.d.ts.map +1 -0
- package/dist/providers/xiaomi-token-plan-cn.models.js +95 -0
- package/dist/providers/xiaomi-token-plan-cn.models.js.map +1 -0
- package/dist/providers/xiaomi-token-plan-sgp.d.ts +3 -0
- package/dist/providers/xiaomi-token-plan-sgp.d.ts.map +1 -0
- package/dist/providers/xiaomi-token-plan-sgp.js +15 -0
- package/dist/providers/xiaomi-token-plan-sgp.js.map +1 -0
- package/dist/providers/xiaomi-token-plan-sgp.models.d.ts +108 -0
- package/dist/providers/xiaomi-token-plan-sgp.models.d.ts.map +1 -0
- package/dist/providers/xiaomi-token-plan-sgp.models.js +95 -0
- package/dist/providers/xiaomi-token-plan-sgp.models.js.map +1 -0
- package/dist/providers/xiaomi.d.ts +3 -0
- package/dist/providers/xiaomi.d.ts.map +1 -0
- package/dist/providers/xiaomi.js +15 -0
- package/dist/providers/xiaomi.js.map +1 -0
- package/dist/providers/xiaomi.models.d.ts +129 -0
- package/dist/providers/xiaomi.models.d.ts.map +1 -0
- package/dist/providers/xiaomi.models.js +113 -0
- package/dist/providers/xiaomi.models.js.map +1 -0
- package/dist/providers/zai-coding-cn.d.ts +3 -0
- package/dist/providers/zai-coding-cn.d.ts.map +1 -0
- package/dist/providers/zai-coding-cn.js +15 -0
- package/dist/providers/zai-coding-cn.js.map +1 -0
- package/dist/providers/zai-coding-cn.models.d.ts +153 -0
- package/dist/providers/zai-coding-cn.models.d.ts.map +1 -0
- package/dist/providers/zai-coding-cn.models.js +114 -0
- package/dist/providers/zai-coding-cn.models.js.map +1 -0
- package/dist/providers/zai.d.ts +3 -0
- package/dist/providers/zai.d.ts.map +1 -0
- package/dist/providers/zai.js +15 -0
- package/dist/providers/zai.js.map +1 -0
- package/dist/providers/zai.models.d.ts +153 -0
- package/dist/providers/zai.models.d.ts.map +1 -0
- package/dist/providers/zai.models.js +114 -0
- package/dist/providers/zai.models.js.map +1 -0
- package/dist/types.d.ts +67 -8
- package/dist/types.d.ts.map +1 -1
- package/dist/types.js.map +1 -1
- package/dist/utils/headers.d.ts +2 -0
- package/dist/utils/headers.d.ts.map +1 -1
- package/dist/utils/headers.js +10 -0
- package/dist/utils/headers.js.map +1 -1
- package/dist/utils/oauth/anthropic.d.ts +2 -0
- package/dist/utils/oauth/anthropic.d.ts.map +1 -1
- package/dist/utils/oauth/anthropic.js +31 -0
- package/dist/utils/oauth/anthropic.js.map +1 -1
- package/dist/utils/oauth/github-copilot.d.ts +2 -0
- package/dist/utils/oauth/github-copilot.d.ts.map +1 -1
- package/dist/utils/oauth/github-copilot.js +33 -2
- package/dist/utils/oauth/github-copilot.js.map +1 -1
- package/dist/utils/oauth/load.d.ts +5 -0
- package/dist/utils/oauth/load.d.ts.map +1 -0
- package/dist/utils/oauth/load.js +22 -0
- package/dist/utils/oauth/load.js.map +1 -0
- package/dist/utils/oauth/openai-codex.d.ts +2 -0
- package/dist/utils/oauth/openai-codex.d.ts.map +1 -1
- package/dist/utils/oauth/openai-codex.js +49 -0
- package/dist/utils/oauth/openai-codex.js.map +1 -1
- package/package.json +15 -42
- package/dist/api-registry.d.ts +0 -20
- package/dist/api-registry.d.ts.map +0 -1
- package/dist/api-registry.js +0 -44
- package/dist/api-registry.js.map +0 -1
- package/dist/base.d.ts +0 -30
- package/dist/base.d.ts.map +0 -1
- package/dist/base.js +0 -18
- package/dist/base.js.map +0 -1
- package/dist/providers/cloudflare.d.ts.map +0 -1
- package/dist/providers/cloudflare.js.map +0 -1
- package/dist/providers/github-copilot-headers.d.ts.map +0 -1
- package/dist/providers/github-copilot-headers.js.map +0 -1
- package/dist/providers/google-shared.d.ts.map +0 -1
- package/dist/providers/google-shared.js.map +0 -1
- package/dist/providers/images/openrouter.d.ts.map +0 -1
- package/dist/providers/images/openrouter.js.map +0 -1
- package/dist/providers/openai-codex-responses.d.ts.map +0 -1
- package/dist/providers/openai-codex-responses.js.map +0 -1
- package/dist/providers/openai-completions.d.ts.map +0 -1
- package/dist/providers/openai-completions.js.map +0 -1
- package/dist/providers/openai-prompt-cache.d.ts.map +0 -1
- package/dist/providers/openai-prompt-cache.js.map +0 -1
- package/dist/providers/openai-responses-shared.d.ts.map +0 -1
- package/dist/providers/openai-responses-shared.js.map +0 -1
- package/dist/providers/openai-responses.d.ts.map +0 -1
- package/dist/providers/openai-responses.js.map +0 -1
- package/dist/providers/register-builtins.d.ts +0 -37
- package/dist/providers/register-builtins.d.ts.map +0 -1
- package/dist/providers/register-builtins.js +0 -191
- package/dist/providers/register-builtins.js.map +0 -1
- package/dist/providers/simple-options.d.ts.map +0 -1
- package/dist/providers/simple-options.js.map +0 -1
- package/dist/providers/transform-messages.d.ts.map +0 -1
- package/dist/providers/transform-messages.js.map +0 -1
- package/dist/stream.d.ts +0 -6
- package/dist/stream.d.ts.map +0 -1
- package/dist/stream.js +0 -37
- package/dist/stream.js.map +0 -1
- /package/dist/{providers → api}/github-copilot-headers.d.ts +0 -0
- /package/dist/{providers → api}/github-copilot-headers.js +0 -0
- /package/dist/{providers → api}/google-shared.d.ts +0 -0
- /package/dist/{providers → api}/google-shared.js +0 -0
- /package/dist/{providers → api}/openai-prompt-cache.d.ts +0 -0
- /package/dist/{providers → api}/openai-prompt-cache.js +0 -0
- /package/dist/{providers → api}/openai-responses-shared.d.ts +0 -0
- /package/dist/{providers → api}/simple-options.d.ts +0 -0
- /package/dist/{providers → api}/simple-options.js +0 -0
- /package/dist/{providers → api}/transform-messages.d.ts +0 -0
- /package/dist/{providers → api}/transform-messages.js +0 -0
package/README.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# @earendil-works/pi-ai
|
|
2
2
|
|
|
3
|
-
Unified LLM API with
|
|
3
|
+
Unified LLM API with provider collections, automatic auth resolution, token and cost tracking, and simple context persistence and hand-off to other models mid-session.
|
|
4
4
|
|
|
5
5
|
**Note**: This library only includes models that support tool calling (function calling), as this is essential for agentic workflows.
|
|
6
6
|
|
|
@@ -8,8 +8,17 @@ Unified LLM API with automatic model discovery, provider configuration, token an
|
|
|
8
8
|
|
|
9
9
|
- [Supported Providers](#supported-providers)
|
|
10
10
|
- [Installation](#installation)
|
|
11
|
-
- [Base Entry Point](#base-entry-point)
|
|
12
11
|
- [Quick Start](#quick-start)
|
|
12
|
+
- [Providers and Models](#providers-and-models)
|
|
13
|
+
- [Provider Factories](#provider-factories)
|
|
14
|
+
- [All Built-in Providers](#all-built-in-providers)
|
|
15
|
+
- [Querying Models](#querying-models)
|
|
16
|
+
- [Static Catalog Reads](#static-catalog-reads)
|
|
17
|
+
- [Dynamic Providers](#dynamic-providers)
|
|
18
|
+
- [Auth](#auth)
|
|
19
|
+
- [How Auth Resolves](#how-auth-resolves)
|
|
20
|
+
- [Credential Store](#credential-store)
|
|
21
|
+
- [Environment Variables](#environment-variables)
|
|
13
22
|
- [Tools](#tools)
|
|
14
23
|
- [Defining Tools](#defining-tools)
|
|
15
24
|
- [Handling Tool Calls](#handling-tool-calls)
|
|
@@ -18,8 +27,6 @@ Unified LLM API with automatic model discovery, provider configuration, token an
|
|
|
18
27
|
- [Complete Event Reference](#complete-event-reference)
|
|
19
28
|
- [Image Input](#image-input)
|
|
20
29
|
- [Image Generation](#image-generation)
|
|
21
|
-
- [Basic Image Generation](#basic-image-generation)
|
|
22
|
-
- [Notes and Limitations](#notes-and-limitations)
|
|
23
30
|
- [Thinking/Reasoning](#thinkingreasoning)
|
|
24
31
|
- [Unified Interface](#unified-interface-streamsimplecompletesimple)
|
|
25
32
|
- [Provider-Specific Options](#provider-specific-options-streamcomplete)
|
|
@@ -28,26 +35,22 @@ Unified LLM API with automatic model discovery, provider configuration, token an
|
|
|
28
35
|
- [Error Handling](#error-handling)
|
|
29
36
|
- [Aborting Requests](#aborting-requests)
|
|
30
37
|
- [Continuing After Abort](#continuing-after-abort)
|
|
31
|
-
- [
|
|
32
|
-
|
|
33
|
-
- [
|
|
34
|
-
- [
|
|
38
|
+
- [Debugging Provider Payloads](#debugging-provider-payloads)
|
|
39
|
+
- [Custom Providers](#custom-providers)
|
|
40
|
+
- [createProvider()](#createprovider)
|
|
41
|
+
- [Calling API Implementations Directly](#calling-api-implementations-directly)
|
|
35
42
|
- [OpenAI Compatibility Settings](#openai-compatibility-settings)
|
|
36
|
-
|
|
43
|
+
- [Faux Provider for Tests](#faux-provider-for-tests)
|
|
37
44
|
- [Cross-Provider Handoffs](#cross-provider-handoffs)
|
|
38
45
|
- [Context Serialization](#context-serialization)
|
|
39
46
|
- [Browser Usage](#browser-usage)
|
|
40
|
-
|
|
41
|
-
- [Environment Variables](#environment-variables-nodejs-only)
|
|
42
|
-
- [Provider-Scoped Environment Overrides](#provider-scoped-environment-overrides)
|
|
43
|
-
- [Checking Environment Variables](#checking-environment-variables)
|
|
47
|
+
- [Bundling and Tree Shaking](#bundling-and-tree-shaking)
|
|
44
48
|
- [OAuth Providers](#oauth-providers)
|
|
45
49
|
- [Vertex AI](#vertex-ai)
|
|
46
50
|
- [CLI Login](#cli-login)
|
|
47
51
|
- [Programmatic OAuth](#programmatic-oauth)
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
- [Provider Notes](#provider-notes)
|
|
52
|
+
- [Migrating from the Old Global API](#migrating-from-the-old-global-api)
|
|
53
|
+
- [Development](#development)
|
|
51
54
|
- [License](#license)
|
|
52
55
|
|
|
53
56
|
## Supported Providers
|
|
@@ -69,16 +72,18 @@ Unified LLM API with automatic model discovery, provider configuration, token an
|
|
|
69
72
|
- **xAI**
|
|
70
73
|
- **OpenRouter**
|
|
71
74
|
- **Vercel AI Gateway**
|
|
72
|
-
- **ZAI** (with separate
|
|
73
|
-
- **MiniMax**
|
|
75
|
+
- **ZAI Coding Plan (Global)** (with separate China provider)
|
|
76
|
+
- **MiniMax** (with separate China provider)
|
|
74
77
|
- **Together AI**
|
|
78
|
+
- **Hugging Face**
|
|
79
|
+
- **Moonshot AI** (with separate China provider)
|
|
75
80
|
- **GitHub Copilot** (requires OAuth, see below)
|
|
76
81
|
- **Amazon Bedrock**
|
|
77
82
|
- **OpenCode Zen**
|
|
78
83
|
- **OpenCode Go**
|
|
79
|
-
- **Fireworks** (uses Anthropic-compatible
|
|
80
|
-
- **Kimi For Coding** (Moonshot AI, uses Anthropic-compatible API)
|
|
81
|
-
- **Xiaomi MiMo** (
|
|
84
|
+
- **Fireworks** (uses OpenAI- and Anthropic-compatible APIs)
|
|
85
|
+
- **Kimi For Coding** (Moonshot AI subscription endpoint, uses Anthropic-compatible API)
|
|
86
|
+
- **Xiaomi MiMo** (defaults to API billing endpoint, with separate Token Plan providers for `cn`/`ams`/`sgp` regions)
|
|
82
87
|
- **Any OpenAI-compatible API**: Ollama, vLLM, LM Studio, etc.
|
|
83
88
|
|
|
84
89
|
## Installation
|
|
@@ -89,37 +94,19 @@ npm install @earendil-works/pi-ai
|
|
|
89
94
|
|
|
90
95
|
TypeBox exports are re-exported from `@earendil-works/pi-ai`: `Type`, `Static`, and `TSchema`.
|
|
91
96
|
|
|
92
|
-
##
|
|
93
|
-
|
|
94
|
-
Use `@earendil-works/pi-ai/base` when a bundler should include only explicitly selected transport implementations. The base entry point exposes model discovery, registries, generic dispatch, environment-key helpers, and provider-neutral utilities without registering built-in transports during module initialization. Generic dispatch still resolves configured environment keys when called.
|
|
97
|
+
## Quick Start
|
|
95
98
|
|
|
96
|
-
|
|
99
|
+
You build a `Models` collection of providers and stream through it. The quickest start registers every built-in provider; apps that care about bundle size register individual providers instead (see [Provider Factories](#provider-factories) and [Bundling and Tree Shaking](#bundling-and-tree-shaking)).
|
|
97
100
|
|
|
98
101
|
```typescript
|
|
99
|
-
import {
|
|
100
|
-
import {
|
|
101
|
-
|
|
102
|
-
registerAnthropic();
|
|
103
|
-
|
|
104
|
-
const model = getModel('anthropic', 'claude-sonnet-4-6');
|
|
105
|
-
const response = await streamSimple(model, {
|
|
106
|
-
messages: [{ role: 'user', content: 'Hello!' }]
|
|
107
|
-
}, {
|
|
108
|
-
apiKey: process.env.ANTHROPIC_API_KEY
|
|
109
|
-
}).result();
|
|
110
|
-
```
|
|
111
|
-
|
|
112
|
-
Direct transport imports bundle their implementation and SDK code. Register only the transports used by the application. For example, OpenRouter, Groq, xAI, and DeepSeek chat models share the `@earendil-works/pi-ai/openai-completions` transport.
|
|
102
|
+
import { Type, type Context, type Tool } from '@earendil-works/pi-ai';
|
|
103
|
+
import { builtinModels } from '@earendil-works/pi-ai/providers/all';
|
|
113
104
|
|
|
114
|
-
|
|
105
|
+
// A Models collection with every built-in provider registered
|
|
106
|
+
const models = builtinModels();
|
|
115
107
|
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
```typescript
|
|
119
|
-
import { Type, getModel, stream, complete, Context, Tool, StringEnum } from '@earendil-works/pi-ai';
|
|
120
|
-
|
|
121
|
-
// Fully typed with auto-complete support for both providers and models
|
|
122
|
-
const model = getModel('openai', 'gpt-4o-mini');
|
|
108
|
+
// Sync lookup against the collection
|
|
109
|
+
const model = models.getModel('openai', 'gpt-4o-mini')!;
|
|
123
110
|
|
|
124
111
|
// Define tools with TypeBox schemas for type safety and validation
|
|
125
112
|
const tools: Tool[] = [{
|
|
@@ -133,12 +120,13 @@ const tools: Tool[] = [{
|
|
|
133
120
|
// Build a conversation context (easily serializable and transferable between models)
|
|
134
121
|
const context: Context = {
|
|
135
122
|
systemPrompt: 'You are a helpful assistant.',
|
|
136
|
-
messages: [{ role: 'user', content: 'What time is it?' }],
|
|
123
|
+
messages: [{ role: 'user', content: 'What time is it?', timestamp: Date.now() }],
|
|
137
124
|
tools
|
|
138
125
|
};
|
|
139
126
|
|
|
140
|
-
// Option 1: Streaming with all event types
|
|
141
|
-
|
|
127
|
+
// Option 1: Streaming with all event types.
|
|
128
|
+
// Auth resolves through the provider (OPENAI_API_KEY from the environment here).
|
|
129
|
+
const s = models.stream(model, context);
|
|
142
130
|
|
|
143
131
|
for await (const event of s) {
|
|
144
132
|
switch (event.type) {
|
|
@@ -181,7 +169,7 @@ for await (const event of s) {
|
|
|
181
169
|
console.log(`\nFinished: ${event.reason}`);
|
|
182
170
|
break;
|
|
183
171
|
case 'error':
|
|
184
|
-
console.error(`Error: ${event.error}`);
|
|
172
|
+
console.error(`Error: ${event.error.errorMessage}`);
|
|
185
173
|
break;
|
|
186
174
|
}
|
|
187
175
|
}
|
|
@@ -193,7 +181,6 @@ context.messages.push(finalMessage);
|
|
|
193
181
|
// Handle tool calls if any
|
|
194
182
|
const toolCalls = finalMessage.content.filter(b => b.type === 'toolCall');
|
|
195
183
|
for (const call of toolCalls) {
|
|
196
|
-
// Execute the tool
|
|
197
184
|
const result = call.name === 'get_time'
|
|
198
185
|
? new Date().toLocaleString('en-US', {
|
|
199
186
|
timeZone: call.arguments.timezone || 'UTC',
|
|
@@ -215,7 +202,7 @@ for (const call of toolCalls) {
|
|
|
215
202
|
|
|
216
203
|
// Continue if there were tool calls
|
|
217
204
|
if (toolCalls.length > 0) {
|
|
218
|
-
const continuation = await complete(model, context);
|
|
205
|
+
const continuation = await models.complete(model, context);
|
|
219
206
|
context.messages.push(continuation);
|
|
220
207
|
console.log('After tool execution:', continuation.content);
|
|
221
208
|
}
|
|
@@ -224,7 +211,7 @@ console.log(`Total tokens: ${finalMessage.usage.input} in, ${finalMessage.usage.
|
|
|
224
211
|
console.log(`Cost: $${finalMessage.usage.cost.total.toFixed(4)}`);
|
|
225
212
|
|
|
226
213
|
// Option 2: Get complete response without streaming
|
|
227
|
-
const response = await complete(model, context);
|
|
214
|
+
const response = await models.complete(model, context);
|
|
228
215
|
|
|
229
216
|
for (const block of response.content) {
|
|
230
217
|
if (block.type === 'text') {
|
|
@@ -235,6 +222,185 @@ for (const block of response.content) {
|
|
|
235
222
|
}
|
|
236
223
|
```
|
|
237
224
|
|
|
225
|
+
Snippets in the rest of this README assume a `models` collection set up like this (with the relevant providers registered).
|
|
226
|
+
|
|
227
|
+
## Providers and Models
|
|
228
|
+
|
|
229
|
+
A **provider** is the runtime unit: it owns its model catalog, its auth (API key resolution, OAuth flows), and its stream behavior. A `Models` collection holds providers and routes every request to the provider that owns the model.
|
|
230
|
+
|
|
231
|
+
Providers internally share **API implementations** (the wire protocols): Anthropic models use `anthropic-messages`, OpenAI uses `openai-responses`, while xAI, Groq, Cerebras, OpenRouter, and most others share `openai-completions`. Mixed-API providers (GitHub Copilot, OpenCode Zen) dispatch per model.
|
|
232
|
+
|
|
233
|
+
### Provider Factories
|
|
234
|
+
|
|
235
|
+
For apps that only need specific providers, there is one factory per built-in provider, each a subpath import that pulls only that provider's catalog:
|
|
236
|
+
|
|
237
|
+
```typescript
|
|
238
|
+
import { anthropicProvider } from '@earendil-works/pi-ai/providers/anthropic';
|
|
239
|
+
import { openaiProvider } from '@earendil-works/pi-ai/providers/openai';
|
|
240
|
+
import { openrouterProvider } from '@earendil-works/pi-ai/providers/openrouter';
|
|
241
|
+
import { amazonBedrockProvider } from '@earendil-works/pi-ai/providers/amazon-bedrock';
|
|
242
|
+
// ...one module per provider in the Supported Providers list
|
|
243
|
+
|
|
244
|
+
const models = createModels();
|
|
245
|
+
models.setProvider(anthropicProvider());
|
|
246
|
+
models.setProvider(openrouterProvider());
|
|
247
|
+
```
|
|
248
|
+
|
|
249
|
+
Provider factories import their model catalog and a lazy API wrapper. They do not import other providers. With bundler code splitting, SDK implementations (`@anthropic-ai/sdk`, `openai`, `@google/genai`, etc.) stay in lazy chunks loaded on the first request to a model of that API.
|
|
250
|
+
|
|
251
|
+
### All Built-in Providers
|
|
252
|
+
|
|
253
|
+
For apps that want everything (as in Quick Start):
|
|
254
|
+
|
|
255
|
+
```typescript
|
|
256
|
+
import { builtinModels } from '@earendil-works/pi-ai/providers/all';
|
|
257
|
+
|
|
258
|
+
const models = builtinModels(); // a Models collection with every built-in provider registered
|
|
259
|
+
```
|
|
260
|
+
|
|
261
|
+
This imports all catalogs and every built-in provider factory. It is the heavy, explicit entrypoint. `builtinModels()` accepts the same options as `createModels()` (`credentials`, `authContext`); `builtinProviders()` returns the provider array if you want to register them on your own collection.
|
|
262
|
+
|
|
263
|
+
### Querying Models
|
|
264
|
+
|
|
265
|
+
Reads are synchronous and return the last-known lists:
|
|
266
|
+
|
|
267
|
+
```typescript
|
|
268
|
+
const providers = models.getProviders(); // registered Provider objects
|
|
269
|
+
const provider = models.getProvider('anthropic'); // one provider
|
|
270
|
+
|
|
271
|
+
const all = models.getModels(); // every model across providers
|
|
272
|
+
const anthropicModels = models.getModels('anthropic');
|
|
273
|
+
const model = models.getModel('anthropic', 'claude-sonnet-4-5');
|
|
274
|
+
|
|
275
|
+
for (const m of anthropicModels) {
|
|
276
|
+
console.log(`${m.id}: ${m.name}`);
|
|
277
|
+
console.log(` API: ${m.api}`);
|
|
278
|
+
console.log(` Context: ${m.contextWindow} tokens`);
|
|
279
|
+
console.log(` Vision: ${m.input.includes('image')}`);
|
|
280
|
+
console.log(` Reasoning: ${m.reasoning}`);
|
|
281
|
+
}
|
|
282
|
+
```
|
|
283
|
+
|
|
284
|
+
Dynamically listed models are typed `Model<Api>`. Narrow with the `hasApi()` guard when you need API-specific option typing:
|
|
285
|
+
|
|
286
|
+
```typescript
|
|
287
|
+
import { hasApi } from '@earendil-works/pi-ai';
|
|
288
|
+
|
|
289
|
+
const m = models.getModel('anthropic', 'claude-sonnet-4-5');
|
|
290
|
+
if (m && hasApi(m, 'anthropic-messages')) {
|
|
291
|
+
// m: Model<'anthropic-messages'> — stream options fully typed
|
|
292
|
+
models.stream(m, context, { thinkingEnabled: true, thinkingBudgetTokens: 2048 });
|
|
293
|
+
}
|
|
294
|
+
```
|
|
295
|
+
|
|
296
|
+
### Static Catalog Reads
|
|
297
|
+
|
|
298
|
+
For tooling that wants the generated built-in catalog with full literal typing (provider and model IDs auto-complete), independent of any collection:
|
|
299
|
+
|
|
300
|
+
```typescript
|
|
301
|
+
import { getBuiltinModel, getBuiltinModels, getBuiltinProviders } from '@earendil-works/pi-ai/providers/all';
|
|
302
|
+
|
|
303
|
+
const model = getBuiltinModel('openai', 'gpt-4o-mini'); // typed Model<'openai-responses'>
|
|
304
|
+
const providers = getBuiltinProviders();
|
|
305
|
+
const anthropic = getBuiltinModels('anthropic');
|
|
306
|
+
```
|
|
307
|
+
|
|
308
|
+
### Dynamic Providers
|
|
309
|
+
|
|
310
|
+
Providers may have dynamic model lists (a llama.cpp server, a live OpenRouter listing). Reads stay sync; fetching is an explicit async verb:
|
|
311
|
+
|
|
312
|
+
```typescript
|
|
313
|
+
// getModels() returns the last-known list (empty before the first refresh)
|
|
314
|
+
await models.refresh('llamacpp'); // fetch one provider's list; rejects on failure
|
|
315
|
+
await models.refresh(); // refresh all providers concurrently, best-effort
|
|
316
|
+
const fresh = models.getModel('llamacpp', 'qwen3-30b');
|
|
317
|
+
```
|
|
318
|
+
|
|
319
|
+
Static built-in providers are no-ops for `refresh()`. See [createProvider()](#createprovider) for building a dynamic provider.
|
|
320
|
+
|
|
321
|
+
## Auth
|
|
322
|
+
|
|
323
|
+
Every provider owns its auth: how API keys resolve (stored credentials, environment variables, ambient sources like AWS profiles or gcloud ADC) and, where supported, OAuth login/refresh flows.
|
|
324
|
+
|
|
325
|
+
### How Auth Resolves
|
|
326
|
+
|
|
327
|
+
When you call `models.stream()`, the collection resolves auth through the owning provider and merges it into the request. Explicit per-request values always win:
|
|
328
|
+
|
|
329
|
+
```typescript
|
|
330
|
+
// Resolved through the provider (env var, stored credential, OAuth token):
|
|
331
|
+
await models.complete(model, context);
|
|
332
|
+
|
|
333
|
+
// Explicit key wins over anything the provider would resolve:
|
|
334
|
+
await models.complete(model, context, { apiKey: 'sk-explicit' });
|
|
335
|
+
```
|
|
336
|
+
|
|
337
|
+
You can inspect resolution without making a request — useful for status UIs:
|
|
338
|
+
|
|
339
|
+
```typescript
|
|
340
|
+
const auth = await models.getAuth(model);
|
|
341
|
+
if (auth) {
|
|
342
|
+
console.log(`configured via ${auth.source}`); // e.g. "ANTHROPIC_API_KEY", "OAuth", "stored credential"
|
|
343
|
+
} else {
|
|
344
|
+
console.log('not configured');
|
|
345
|
+
}
|
|
346
|
+
```
|
|
347
|
+
|
|
348
|
+
`getAuth()` resolves `undefined` for unconfigured providers and rejects with `ModelsError` when something is actually broken (`"oauth"`: token refresh failed, credential preserved for re-login; `"auth"`: key resolution or credential store failure). Request paths surface the same failures as stream errors.
|
|
349
|
+
|
|
350
|
+
### Credential Store
|
|
351
|
+
|
|
352
|
+
Stored credentials (API keys entered interactively, OAuth tokens) live in a `CredentialStore` — one type-tagged credential per provider. pi-ai ships an in-memory default; apps inject persistent storage:
|
|
353
|
+
|
|
354
|
+
```typescript
|
|
355
|
+
import { createModels, type CredentialStore } from '@earendil-works/pi-ai';
|
|
356
|
+
|
|
357
|
+
const models = createModels({ credentials: myFileBackedStore });
|
|
358
|
+
// builtinModels() takes the same options:
|
|
359
|
+
// const models = builtinModels({ credentials: myFileBackedStore });
|
|
360
|
+
```
|
|
361
|
+
|
|
362
|
+
The contract is small: `read(providerId)`, `modify(providerId, fn)` (the only write path — a serialized read-modify-write), and `delete(providerId)`. OAuth token refresh runs inside `modify`, so concurrent requests and processes cannot double-refresh a rotated token. A stored credential *owns* its provider: environment variables are only consulted when nothing is stored, and a failed refresh never silently falls back to an env key.
|
|
363
|
+
|
|
364
|
+
### Environment Variables
|
|
365
|
+
|
|
366
|
+
Built-in providers resolve these env vars (Node.js; in browsers pass `apiKey` explicitly):
|
|
367
|
+
|
|
368
|
+
| Provider | Environment Variable(s) |
|
|
369
|
+
|----------|------------------------|
|
|
370
|
+
| OpenAI | `OPENAI_API_KEY` |
|
|
371
|
+
| Ant Ling | `ANT_LING_API_KEY` |
|
|
372
|
+
| Azure OpenAI | `AZURE_OPENAI_API_KEY` + `AZURE_OPENAI_BASE_URL` (e.g. `https://{resource}.openai.azure.com`) or `AZURE_OPENAI_RESOURCE_NAME`. Supports `*.openai.azure.com` and `*.cognitiveservices.azure.com`; root endpoints auto-normalize to `/openai/v1`. Optional: `AZURE_OPENAI_API_VERSION` (default `v1`), `AZURE_OPENAI_DEPLOYMENT_NAME_MAP`. |
|
|
373
|
+
| Anthropic | `ANTHROPIC_API_KEY` or `ANTHROPIC_OAUTH_TOKEN` |
|
|
374
|
+
| DeepSeek | `DEEPSEEK_API_KEY` |
|
|
375
|
+
| NVIDIA NIM | `NVIDIA_API_KEY` |
|
|
376
|
+
| Google | `GEMINI_API_KEY` |
|
|
377
|
+
| Vertex AI | `GOOGLE_CLOUD_API_KEY` or `GOOGLE_CLOUD_PROJECT` (or `GCLOUD_PROJECT`) + `GOOGLE_CLOUD_LOCATION` + ADC |
|
|
378
|
+
| Mistral | `MISTRAL_API_KEY` |
|
|
379
|
+
| Groq | `GROQ_API_KEY` |
|
|
380
|
+
| Cerebras | `CEREBRAS_API_KEY` |
|
|
381
|
+
| Cloudflare AI Gateway | `CLOUDFLARE_API_KEY` + `CLOUDFLARE_ACCOUNT_ID` + `CLOUDFLARE_GATEWAY_ID` |
|
|
382
|
+
| Cloudflare Workers AI | `CLOUDFLARE_API_KEY` + `CLOUDFLARE_ACCOUNT_ID` |
|
|
383
|
+
| xAI | `XAI_API_KEY` |
|
|
384
|
+
| Fireworks | `FIREWORKS_API_KEY` |
|
|
385
|
+
| Together AI | `TOGETHER_API_KEY` |
|
|
386
|
+
| OpenRouter | `OPENROUTER_API_KEY` |
|
|
387
|
+
| Vercel AI Gateway | `AI_GATEWAY_API_KEY` |
|
|
388
|
+
| ZAI Coding Plan (Global) | `ZAI_API_KEY` |
|
|
389
|
+
| ZAI Coding Plan (China) | `ZAI_CODING_CN_API_KEY` |
|
|
390
|
+
| MiniMax (Global) | `MINIMAX_API_KEY` |
|
|
391
|
+
| MiniMax (China) | `MINIMAX_CN_API_KEY` |
|
|
392
|
+
| Moonshot AI / Moonshot AI (China) | `MOONSHOT_API_KEY` |
|
|
393
|
+
| Hugging Face | `HF_TOKEN` |
|
|
394
|
+
| OpenCode Zen / OpenCode Go | `OPENCODE_API_KEY` |
|
|
395
|
+
| Kimi For Coding | `KIMI_API_KEY` |
|
|
396
|
+
| Xiaomi MiMo (API billing) | `XIAOMI_API_KEY` |
|
|
397
|
+
| Xiaomi MiMo Token Plan (China) | `XIAOMI_TOKEN_PLAN_CN_API_KEY` |
|
|
398
|
+
| Xiaomi MiMo Token Plan (Amsterdam) | `XIAOMI_TOKEN_PLAN_AMS_API_KEY` |
|
|
399
|
+
| Xiaomi MiMo Token Plan (Singapore) | `XIAOMI_TOKEN_PLAN_SGP_API_KEY` |
|
|
400
|
+
| GitHub Copilot | `COPILOT_GITHUB_TOKEN` |
|
|
401
|
+
|
|
402
|
+
Amazon Bedrock resolves ambient AWS credentials (`AWS_PROFILE`, access key pairs, `AWS_BEARER_TOKEN_BEDROCK`, ECS task roles, web identity tokens). Vertex AI resolves either an explicit key or gcloud Application Default Credentials plus project/location.
|
|
403
|
+
|
|
238
404
|
## Tools
|
|
239
405
|
|
|
240
406
|
Tools enable LLMs to interact with external systems. This library uses TypeBox schemas for type-safe tool definitions with automatic validation using TypeBox's built-in validator and value conversion utilities. TypeBox schemas can be serialized and deserialized as plain JSON, making them ideal for distributed systems.
|
|
@@ -242,7 +408,7 @@ Tools enable LLMs to interact with external systems. This library uses TypeBox s
|
|
|
242
408
|
### Defining Tools
|
|
243
409
|
|
|
244
410
|
```typescript
|
|
245
|
-
import { Type, Tool, StringEnum } from '@earendil-works/pi-ai';
|
|
411
|
+
import { Type, type Tool, StringEnum } from '@earendil-works/pi-ai';
|
|
246
412
|
|
|
247
413
|
// Define tool parameters with TypeBox
|
|
248
414
|
const weatherTool: Tool = {
|
|
@@ -277,11 +443,11 @@ Tool results use content blocks and can include both text and images:
|
|
|
277
443
|
import { readFileSync } from 'fs';
|
|
278
444
|
|
|
279
445
|
const context: Context = {
|
|
280
|
-
messages: [{ role: 'user', content: 'What is the weather in London?' }],
|
|
446
|
+
messages: [{ role: 'user', content: 'What is the weather in London?', timestamp: Date.now() }],
|
|
281
447
|
tools: [weatherTool]
|
|
282
448
|
};
|
|
283
449
|
|
|
284
|
-
const response = await complete(model, context);
|
|
450
|
+
const response = await models.complete(model, context);
|
|
285
451
|
|
|
286
452
|
// Check for tool calls in the response
|
|
287
453
|
for (const block of response.content) {
|
|
@@ -322,7 +488,7 @@ context.messages.push({
|
|
|
322
488
|
During streaming, tool call arguments are progressively parsed as they arrive. This enables real-time UI updates before the complete arguments are available:
|
|
323
489
|
|
|
324
490
|
```typescript
|
|
325
|
-
const s = stream(model, context);
|
|
491
|
+
const s = models.stream(model, context);
|
|
326
492
|
|
|
327
493
|
for await (const event of s) {
|
|
328
494
|
if (event.type === 'toolcall_delta') {
|
|
@@ -363,15 +529,13 @@ for await (const event of s) {
|
|
|
363
529
|
|
|
364
530
|
### Validating Tool Arguments
|
|
365
531
|
|
|
366
|
-
When
|
|
367
|
-
|
|
368
|
-
When implementing your own tool execution loop with `stream()` or `complete()`, use `validateToolCall` to validate arguments before passing them to your tools:
|
|
532
|
+
When implementing your own tool execution loop, use `validateToolCall` to validate arguments before passing them to your tools:
|
|
369
533
|
|
|
370
534
|
```typescript
|
|
371
|
-
import {
|
|
535
|
+
import { validateToolCall, type Tool } from '@earendil-works/pi-ai';
|
|
372
536
|
|
|
373
537
|
const tools: Tool[] = [weatherTool, calculatorTool];
|
|
374
|
-
const s = stream(model, { messages, tools });
|
|
538
|
+
const s = models.stream(model, { messages, tools });
|
|
375
539
|
|
|
376
540
|
for await (const event of s) {
|
|
377
541
|
if (event.type === 'toolcall_end') {
|
|
@@ -424,9 +588,8 @@ Models with vision capabilities can process images. You can check if a model sup
|
|
|
424
588
|
|
|
425
589
|
```typescript
|
|
426
590
|
import { readFileSync } from 'fs';
|
|
427
|
-
import { getModel, complete } from '@earendil-works/pi-ai';
|
|
428
591
|
|
|
429
|
-
const model = getModel('openai', 'gpt-4o-mini')
|
|
592
|
+
const model = models.getModel('openai', 'gpt-4o-mini')!;
|
|
430
593
|
|
|
431
594
|
// Check if model supports images
|
|
432
595
|
if (model.input.includes('image')) {
|
|
@@ -436,13 +599,14 @@ if (model.input.includes('image')) {
|
|
|
436
599
|
const imageBuffer = readFileSync('image.png');
|
|
437
600
|
const base64Image = imageBuffer.toString('base64');
|
|
438
601
|
|
|
439
|
-
const response = await complete(model, {
|
|
602
|
+
const response = await models.complete(model, {
|
|
440
603
|
messages: [{
|
|
441
604
|
role: 'user',
|
|
442
605
|
content: [
|
|
443
606
|
{ type: 'text', text: 'What is in this image?' },
|
|
444
607
|
{ type: 'image', data: base64Image, mimeType: 'image/png' }
|
|
445
|
-
]
|
|
608
|
+
],
|
|
609
|
+
timestamp: Date.now()
|
|
446
610
|
}]
|
|
447
611
|
});
|
|
448
612
|
|
|
@@ -456,21 +620,21 @@ for (const block of response.content) {
|
|
|
456
620
|
|
|
457
621
|
## Image Generation
|
|
458
622
|
|
|
459
|
-
Image generation uses a separate API surface from text/chat generation
|
|
460
|
-
|
|
461
|
-
Do not use `stream()` or `complete()` for image generation. Image generation is a one-shot API: `generateImages()` waits for the provider response and returns the final `AssistantImages` result.
|
|
623
|
+
Image generation uses a separate API surface from text/chat generation, mirroring the chat-side design: an `ImagesModels` collection holds `ImagesProvider`s, reads are sync, and auth resolves through the owning provider. Image generation is a one-shot API: `generateImages()` waits for the provider response and returns the final `AssistantImages` result — do not use the chat/stream APIs for it.
|
|
462
624
|
|
|
463
625
|
### Basic Image Generation
|
|
464
626
|
|
|
465
627
|
```typescript
|
|
466
|
-
import {
|
|
628
|
+
import { builtinImagesModels } from '@earendil-works/pi-ai/providers/all';
|
|
467
629
|
|
|
468
|
-
|
|
630
|
+
// Every built-in image-generation provider; accepts the same options as createModels()
|
|
631
|
+
const imagesModels = builtinImagesModels();
|
|
469
632
|
|
|
470
|
-
const
|
|
633
|
+
const model = imagesModels.getModel('openrouter', 'google/gemini-2.5-flash-image')!;
|
|
634
|
+
|
|
635
|
+
// Auth resolves through the provider (OPENROUTER_API_KEY here); explicit apiKey wins
|
|
636
|
+
const result = await imagesModels.generateImages(model, {
|
|
471
637
|
input: [{ type: 'text', text: 'Generate a red circle on a plain white background.' }]
|
|
472
|
-
}, {
|
|
473
|
-
apiKey: process.env.OPENROUTER_API_KEY
|
|
474
638
|
});
|
|
475
639
|
|
|
476
640
|
for (const block of result.output) {
|
|
@@ -483,19 +647,32 @@ for (const block of result.output) {
|
|
|
483
647
|
}
|
|
484
648
|
```
|
|
485
649
|
|
|
650
|
+
Like the chat side, you can build the collection from parts: `createImagesModels({ credentials?, authContext? })`, the `openrouterImagesProvider()` factory from `@earendil-works/pi-ai/providers/openrouter-images`, and `createImagesProvider({ id, auth, models, refreshModels?, api })` for custom image providers (with `imagesModels.refresh(provider?)` for dynamic lists). Failures never reject — they return an `AssistantImages` with `stopReason: "error"`. The collection's `getAuth(model)` works exactly like the chat-side one.
|
|
651
|
+
|
|
652
|
+
The old global API (`getImageModel()` / `getImageModels()` / `getImageProviders()` / `generateImages()`) remains available on the [compat entrypoint](#migrating-from-the-old-global-api):
|
|
653
|
+
|
|
654
|
+
```typescript
|
|
655
|
+
import { getImageModel, generateImages } from '@earendil-works/pi-ai/compat';
|
|
656
|
+
|
|
657
|
+
const model = getImageModel('openrouter', 'google/gemini-2.5-flash-image');
|
|
658
|
+
const result = await generateImages(model, {
|
|
659
|
+
input: [{ type: 'text', text: 'Generate a red circle on a plain white background.' }]
|
|
660
|
+
}, {
|
|
661
|
+
apiKey: process.env.OPENROUTER_API_KEY
|
|
662
|
+
});
|
|
663
|
+
```
|
|
664
|
+
|
|
486
665
|
Some models also support image input:
|
|
487
666
|
|
|
488
667
|
```typescript
|
|
489
668
|
import { readFileSync } from 'fs';
|
|
490
669
|
|
|
491
670
|
const imageBuffer = readFileSync('input.png');
|
|
492
|
-
const result = await generateImages(model, {
|
|
671
|
+
const result = await imagesModels.generateImages(model, {
|
|
493
672
|
input: [
|
|
494
673
|
{ type: 'text', text: 'Create a variation of this image with a blue background.' },
|
|
495
674
|
{ type: 'image', data: imageBuffer.toString('base64'), mimeType: 'image/png' }
|
|
496
675
|
]
|
|
497
|
-
}, {
|
|
498
|
-
apiKey: process.env.OPENROUTER_API_KEY
|
|
499
676
|
});
|
|
500
677
|
```
|
|
501
678
|
|
|
@@ -508,14 +685,14 @@ console.log(model.output); // ['image'] or ['image', 'text']
|
|
|
508
685
|
|
|
509
686
|
### Notes and Limitations
|
|
510
687
|
|
|
511
|
-
-
|
|
512
|
-
- Use `generateImages()`, not
|
|
688
|
+
- Image models live in `ImagesModels` collections, chat models in `Models` collections; the two are separate surfaces.
|
|
689
|
+
- Use `generateImages()`, not the chat/stream APIs.
|
|
513
690
|
- Image-generation models do not participate in tool calling.
|
|
514
691
|
- Outputs are returned in `AssistantImages.output` and can include both base64-encoded `ImageContent` blocks and `TextContent` blocks.
|
|
515
692
|
- Some models return only images, others return images plus text. Check `model.output`.
|
|
516
693
|
- Some models accept image input, others are text-to-image only. Check `model.input`.
|
|
517
694
|
- Like the streaming APIs, image generation supports options such as `apiKey`, `signal`, `headers`, `onPayload`, and `onResponse`, and results may include `stopReason`, `responseId`, and `usage`.
|
|
518
|
-
- If you want a model to analyze images in a conversation or call tools, use the regular
|
|
695
|
+
- If you want a model to analyze images in a conversation or call tools, use the regular chat APIs with a model that supports image input.
|
|
519
696
|
- At the moment, image generation is available through only one provider, OpenRouter.
|
|
520
697
|
|
|
521
698
|
## Thinking/Reasoning
|
|
@@ -525,16 +702,11 @@ Many models support thinking/reasoning capabilities where they can show their in
|
|
|
525
702
|
### Unified Interface (streamSimple/completeSimple)
|
|
526
703
|
|
|
527
704
|
```typescript
|
|
528
|
-
import { getModel, streamSimple, completeSimple } from '@earendil-works/pi-ai';
|
|
529
|
-
|
|
530
705
|
// Many models across providers support thinking/reasoning
|
|
531
|
-
const model = getModel('anthropic', 'claude-sonnet-4-
|
|
532
|
-
// or getModel('openai', 'gpt-5-mini');
|
|
533
|
-
// or getModel('google', 'gemini-2.5-flash');
|
|
534
|
-
// or getModel('xai', 'grok-code-fast-1');
|
|
535
|
-
// or getModel('groq', 'openai/gpt-oss-20b');
|
|
536
|
-
// or getModel('cerebras', 'gpt-oss-120b');
|
|
537
|
-
// or getModel('openrouter', 'z-ai/glm-4.5v');
|
|
706
|
+
const model = models.getModel('anthropic', 'claude-sonnet-4-5')!;
|
|
707
|
+
// or models.getModel('openai', 'gpt-5-mini');
|
|
708
|
+
// or models.getModel('google', 'gemini-2.5-flash');
|
|
709
|
+
// or models.getModel('xai', 'grok-code-fast-1');
|
|
538
710
|
|
|
539
711
|
// Check if model supports reasoning
|
|
540
712
|
if (model.reasoning) {
|
|
@@ -542,8 +714,8 @@ if (model.reasoning) {
|
|
|
542
714
|
}
|
|
543
715
|
|
|
544
716
|
// Use the simplified reasoning option
|
|
545
|
-
const response = await completeSimple(model, {
|
|
546
|
-
messages: [{ role: 'user', content: 'Solve: 2x + 5 = 13' }]
|
|
717
|
+
const response = await models.completeSimple(model, {
|
|
718
|
+
messages: [{ role: 'user', content: 'Solve: 2x + 5 = 13', timestamp: Date.now() }]
|
|
547
719
|
}, {
|
|
548
720
|
reasoning: 'medium' // 'minimal' | 'low' | 'medium' | 'high' | 'xhigh'
|
|
549
721
|
});
|
|
@@ -560,33 +732,39 @@ for (const block of response.content) {
|
|
|
560
732
|
|
|
561
733
|
### Provider-Specific Options (stream/complete)
|
|
562
734
|
|
|
563
|
-
|
|
735
|
+
`models.stream()`/`complete()` accept the owning API's full option set. Use `hasApi()` to narrow a dynamically looked-up model to its API for full option typing:
|
|
564
736
|
|
|
565
737
|
```typescript
|
|
566
|
-
import {
|
|
738
|
+
import { hasApi } from '@earendil-works/pi-ai';
|
|
567
739
|
|
|
568
740
|
// OpenAI Reasoning (o1, o3, gpt-5)
|
|
569
|
-
const openaiModel = getModel('openai', 'gpt-5-mini')
|
|
570
|
-
|
|
571
|
-
|
|
572
|
-
|
|
573
|
-
|
|
741
|
+
const openaiModel = models.getModel('openai', 'gpt-5-mini')!;
|
|
742
|
+
if (hasApi(openaiModel, 'openai-responses')) {
|
|
743
|
+
await models.complete(openaiModel, context, {
|
|
744
|
+
reasoningEffort: 'medium',
|
|
745
|
+
reasoningSummary: 'detailed' // OpenAI Responses API only
|
|
746
|
+
});
|
|
747
|
+
}
|
|
574
748
|
|
|
575
|
-
// Anthropic Thinking
|
|
576
|
-
const anthropicModel = getModel('anthropic', 'claude-sonnet-4-
|
|
577
|
-
|
|
578
|
-
|
|
579
|
-
|
|
580
|
-
|
|
749
|
+
// Anthropic Thinking
|
|
750
|
+
const anthropicModel = models.getModel('anthropic', 'claude-sonnet-4-5')!;
|
|
751
|
+
if (hasApi(anthropicModel, 'anthropic-messages')) {
|
|
752
|
+
await models.complete(anthropicModel, context, {
|
|
753
|
+
thinkingEnabled: true,
|
|
754
|
+
thinkingBudgetTokens: 8192 // Optional token limit
|
|
755
|
+
});
|
|
756
|
+
}
|
|
581
757
|
|
|
582
758
|
// Google Gemini Thinking
|
|
583
|
-
const googleModel = getModel('google', 'gemini-2.5-flash')
|
|
584
|
-
|
|
585
|
-
|
|
586
|
-
|
|
587
|
-
|
|
588
|
-
|
|
589
|
-
}
|
|
759
|
+
const googleModel = models.getModel('google', 'gemini-2.5-flash')!;
|
|
760
|
+
if (hasApi(googleModel, 'google-generative-ai')) {
|
|
761
|
+
await models.complete(googleModel, context, {
|
|
762
|
+
thinking: {
|
|
763
|
+
enabled: true,
|
|
764
|
+
budgetTokens: 8192 // -1 for dynamic, 0 to disable
|
|
765
|
+
}
|
|
766
|
+
});
|
|
767
|
+
}
|
|
590
768
|
```
|
|
591
769
|
|
|
592
770
|
### Streaming Thinking Content
|
|
@@ -594,7 +772,7 @@ await complete(googleModel, context, {
|
|
|
594
772
|
When streaming, thinking content is delivered through specific events:
|
|
595
773
|
|
|
596
774
|
```typescript
|
|
597
|
-
const s = streamSimple(model, context, { reasoning: 'high' });
|
|
775
|
+
const s = models.streamSimple(model, context, { reasoning: 'high' });
|
|
598
776
|
|
|
599
777
|
for await (const event of s) {
|
|
600
778
|
switch (event.type) {
|
|
@@ -625,11 +803,11 @@ Every `AssistantMessage` includes a `stopReason` field that indicates how the ge
|
|
|
625
803
|
|
|
626
804
|
## Error Handling
|
|
627
805
|
|
|
628
|
-
|
|
806
|
+
Request failures never throw out of the stream functions: when a request ends with an error (including aborts and tool call validation errors), the streaming API emits an error event and the final message carries the details:
|
|
629
807
|
|
|
630
808
|
```typescript
|
|
631
809
|
// In streaming
|
|
632
|
-
for await (const event of
|
|
810
|
+
for await (const event of s) {
|
|
633
811
|
if (event.type === 'error') {
|
|
634
812
|
// event.reason is either "error" or "aborted"
|
|
635
813
|
// event.error is the AssistantMessage with partial content
|
|
@@ -639,7 +817,7 @@ for await (const event of stream) {
|
|
|
639
817
|
}
|
|
640
818
|
|
|
641
819
|
// The final message will have the error details
|
|
642
|
-
const message = await
|
|
820
|
+
const message = await s.result();
|
|
643
821
|
if (message.stopReason === 'error' || message.stopReason === 'aborted') {
|
|
644
822
|
console.error('Request failed:', message.errorMessage);
|
|
645
823
|
// message.content contains any partial content received before the error
|
|
@@ -647,21 +825,20 @@ if (message.stopReason === 'error' || message.stopReason === 'aborted') {
|
|
|
647
825
|
}
|
|
648
826
|
```
|
|
649
827
|
|
|
828
|
+
Auth failures (no key configured, OAuth refresh failed, unknown provider) surface the same way: as a stream error with `stopReason: "error"`.
|
|
829
|
+
|
|
650
830
|
### Aborting Requests
|
|
651
831
|
|
|
652
832
|
The abort signal allows you to cancel in-progress requests. Aborted requests have `stopReason === 'aborted'`:
|
|
653
833
|
|
|
654
834
|
```typescript
|
|
655
|
-
import { getModel, stream } from '@earendil-works/pi-ai';
|
|
656
|
-
|
|
657
|
-
const model = getModel('openai', 'gpt-4o-mini');
|
|
658
835
|
const controller = new AbortController();
|
|
659
836
|
|
|
660
837
|
// Abort after 2 seconds
|
|
661
838
|
setTimeout(() => controller.abort(), 2000);
|
|
662
839
|
|
|
663
|
-
const s = stream(model, {
|
|
664
|
-
messages: [{ role: 'user', content: 'Write a long story' }]
|
|
840
|
+
const s = models.stream(model, {
|
|
841
|
+
messages: [{ role: 'user', content: 'Write a long story', timestamp: Date.now() }]
|
|
665
842
|
}, {
|
|
666
843
|
signal: controller.signal
|
|
667
844
|
});
|
|
@@ -691,7 +868,7 @@ Aborted messages can be added to the conversation context and continued in subse
|
|
|
691
868
|
```typescript
|
|
692
869
|
const context = {
|
|
693
870
|
messages: [
|
|
694
|
-
{ role: 'user', content: 'Explain quantum computing in detail' }
|
|
871
|
+
{ role: 'user', content: 'Explain quantum computing in detail', timestamp: Date.now() }
|
|
695
872
|
]
|
|
696
873
|
};
|
|
697
874
|
|
|
@@ -699,14 +876,14 @@ const context = {
|
|
|
699
876
|
const controller1 = new AbortController();
|
|
700
877
|
setTimeout(() => controller1.abort(), 2000);
|
|
701
878
|
|
|
702
|
-
const partial = await complete(model, context, { signal: controller1.signal });
|
|
879
|
+
const partial = await models.complete(model, context, { signal: controller1.signal });
|
|
703
880
|
|
|
704
881
|
// Add the partial response to context
|
|
705
882
|
context.messages.push(partial);
|
|
706
|
-
context.messages.push({ role: 'user', content: 'Please continue' });
|
|
883
|
+
context.messages.push({ role: 'user', content: 'Please continue', timestamp: Date.now() });
|
|
707
884
|
|
|
708
885
|
// Continue the conversation
|
|
709
|
-
const continuation = await complete(model, context);
|
|
886
|
+
const continuation = await models.complete(model, context);
|
|
710
887
|
```
|
|
711
888
|
|
|
712
889
|
### Debugging Provider Payloads
|
|
@@ -714,7 +891,7 @@ const continuation = await complete(model, context);
|
|
|
714
891
|
Use the `onPayload` callback to inspect the request payload sent to the provider. This is useful for debugging request formatting issues or provider validation errors.
|
|
715
892
|
|
|
716
893
|
```typescript
|
|
717
|
-
const response = await complete(model, context, {
|
|
894
|
+
const response = await models.complete(model, context, {
|
|
718
895
|
onPayload: (payload) => {
|
|
719
896
|
console.log('Provider payload:', JSON.stringify(payload, null, 2));
|
|
720
897
|
}
|
|
@@ -723,206 +900,93 @@ const response = await complete(model, context, {
|
|
|
723
900
|
|
|
724
901
|
The callback is supported by `stream`, `complete`, `streamSimple`, and `completeSimple`.
|
|
725
902
|
|
|
726
|
-
##
|
|
903
|
+
## Custom Providers
|
|
727
904
|
|
|
728
|
-
|
|
905
|
+
### createProvider()
|
|
729
906
|
|
|
730
|
-
|
|
731
|
-
- **`google-generative-ai`**: Google Generative AI API (`streamGoogle`, `GoogleOptions`)
|
|
732
|
-
- **`google-vertex`**: Google Vertex AI API (`streamGoogleVertex`, `GoogleVertexOptions`)
|
|
733
|
-
- **`mistral-conversations`**: Mistral Conversations API (`streamMistral`, `MistralOptions`)
|
|
734
|
-
- **`openai-completions`**: OpenAI Chat Completions API (`streamOpenAICompletions`, `OpenAICompletionsOptions`)
|
|
735
|
-
- **`openai-responses`**: OpenAI Responses API (`streamOpenAIResponses`, `OpenAIResponsesOptions`)
|
|
736
|
-
- **`openai-codex-responses`**: OpenAI Codex Responses API (`streamOpenAICodexResponses`, `OpenAICodexResponsesOptions`)
|
|
737
|
-
- **`azure-openai-responses`**: Azure OpenAI Responses API (`streamAzureOpenAIResponses`, `AzureOpenAIResponsesOptions`)
|
|
738
|
-
- **`bedrock-converse-stream`**: Amazon Bedrock Converse API (`streamBedrock`, `BedrockOptions`)
|
|
739
|
-
|
|
740
|
-
### Faux provider for tests
|
|
741
|
-
|
|
742
|
-
`registerFauxProvider()` registers a temporary in-memory provider for tests and demos. It is opt-in and not part of the built-in provider set.
|
|
907
|
+
`createProvider()` builds a provider from parts: identity, auth, a model list, and an API implementation. Use it for local inference servers, proxies, or any OpenAI/Anthropic-compatible endpoint:
|
|
743
908
|
|
|
744
909
|
```typescript
|
|
745
|
-
import {
|
|
746
|
-
|
|
747
|
-
fauxAssistantMessage,
|
|
748
|
-
fauxText,
|
|
749
|
-
fauxThinking,
|
|
750
|
-
fauxToolCall,
|
|
751
|
-
registerFauxProvider,
|
|
752
|
-
stream,
|
|
753
|
-
} from '@earendil-works/pi-ai';
|
|
754
|
-
|
|
755
|
-
const registration = registerFauxProvider({
|
|
756
|
-
tokensPerSecond: 50 // optional
|
|
757
|
-
});
|
|
910
|
+
import { createModels, createProvider, envApiKeyAuth, type Model } from '@earendil-works/pi-ai';
|
|
911
|
+
import { openAICompletionsApi } from '@earendil-works/pi-ai/api/openai-completions.lazy';
|
|
758
912
|
|
|
759
|
-
const
|
|
760
|
-
|
|
761
|
-
|
|
913
|
+
const ollamaModel: Model<'openai-completions'> = {
|
|
914
|
+
id: 'llama-3.1-8b',
|
|
915
|
+
name: 'Llama 3.1 8B (Ollama)',
|
|
916
|
+
api: 'openai-completions',
|
|
917
|
+
provider: 'ollama',
|
|
918
|
+
baseUrl: 'http://localhost:11434/v1',
|
|
919
|
+
reasoning: false,
|
|
920
|
+
input: ['text'],
|
|
921
|
+
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
|
|
922
|
+
contextWindow: 128000,
|
|
923
|
+
maxTokens: 32000
|
|
762
924
|
};
|
|
763
925
|
|
|
764
|
-
|
|
765
|
-
|
|
766
|
-
|
|
767
|
-
|
|
768
|
-
|
|
769
|
-
|
|
770
|
-
|
|
771
|
-
|
|
772
|
-
sessionId: 'session-1',
|
|
773
|
-
cacheRetention: 'short'
|
|
926
|
+
const ollama = createProvider({
|
|
927
|
+
id: 'ollama',
|
|
928
|
+
name: 'Ollama',
|
|
929
|
+
baseUrl: 'http://localhost:11434/v1',
|
|
930
|
+
// Every provider declares auth; keyless local servers resolve as configured with no key.
|
|
931
|
+
auth: { apiKey: { name: 'Ollama', resolve: async () => ({ auth: {} }) } },
|
|
932
|
+
models: [ollamaModel],
|
|
933
|
+
api: openAICompletionsApi(),
|
|
774
934
|
});
|
|
775
|
-
context.messages.push(first);
|
|
776
935
|
|
|
777
|
-
|
|
778
|
-
|
|
779
|
-
toolCallId: first.content.find((block) => block.type === 'toolCall')!.id,
|
|
780
|
-
toolName: 'echo',
|
|
781
|
-
content: [{ type: 'text', text: 'package.json contents here' }],
|
|
782
|
-
isError: false,
|
|
783
|
-
timestamp: Date.now()
|
|
784
|
-
});
|
|
936
|
+
const models = createModels();
|
|
937
|
+
models.setProvider(ollama);
|
|
785
938
|
|
|
786
|
-
|
|
787
|
-
|
|
788
|
-
fauxThinking('Now I can summarize the tool output.'),
|
|
789
|
-
fauxText('Here is the summary.')
|
|
790
|
-
])
|
|
791
|
-
]);
|
|
939
|
+
await models.complete(models.getModel('ollama', 'llama-3.1-8b')!, context);
|
|
940
|
+
```
|
|
792
941
|
|
|
793
|
-
|
|
794
|
-
for await (const event of s) {
|
|
795
|
-
console.log(event.type);
|
|
796
|
-
}
|
|
942
|
+
For providers with real keys, `envApiKeyAuth(displayName, envVars)` gives the standard behavior (stored credential wins, then the first set env var):
|
|
797
943
|
|
|
798
|
-
|
|
799
|
-
const
|
|
800
|
-
|
|
801
|
-
|
|
802
|
-
|
|
803
|
-
|
|
944
|
+
```typescript
|
|
945
|
+
const proxy = createProvider({
|
|
946
|
+
id: 'my-proxy',
|
|
947
|
+
auth: { apiKey: envApiKeyAuth('My proxy API key', ['MY_PROXY_API_KEY']) },
|
|
948
|
+
models: [/* ... */],
|
|
949
|
+
api: openAICompletionsApi(),
|
|
804
950
|
});
|
|
805
|
-
const thinker = multiModel.getModel('faux-thinker');
|
|
806
|
-
|
|
807
|
-
console.log(thinker?.reasoning);
|
|
808
|
-
console.log(registration.getPendingResponseCount());
|
|
809
|
-
console.log(registration.state.callCount);
|
|
810
|
-
registration.unregister();
|
|
811
|
-
multiModel.unregister();
|
|
812
951
|
```
|
|
813
952
|
|
|
814
|
-
|
|
815
|
-
- Responses are consumed from a queue in request start order.
|
|
816
|
-
- If the queue is empty, the faux provider returns an assistant error message with `errorMessage: "No more faux responses queued"`.
|
|
817
|
-
- Use `registration.setResponses([...])` to replace the remaining queue and `registration.appendResponses([...])` to add more responses.
|
|
818
|
-
- `registration.models` exposes all registered faux models. `registration.getModel()` returns the first one, and `registration.getModel(id)` returns a specific one.
|
|
819
|
-
- Use `fauxAssistantMessage(...)` for scripted assistant replies. Use `fauxText(...)`, `fauxThinking(...)`, and `fauxToolCall(...)` to build content blocks without filling in low-level fields manually.
|
|
820
|
-
- `registration.unregister()` removes the temporary provider from the global API registry.
|
|
821
|
-
- Usage is estimated at roughly 1 token per 4 characters. When `sessionId` is present and `cacheRetention` is not `"none"`, prompt cache reads and writes are simulated automatically.
|
|
822
|
-
- Tool call arguments stream incrementally via `toolcall_delta` chunks.
|
|
823
|
-
- By default, each streamed chunk is emitted on its own microtask. Set `tokensPerSecond` to pace chunk delivery in real time.
|
|
824
|
-
- The intended use is one deterministic scripted flow per registration. If you need independent concurrent flows, register separate faux providers.
|
|
825
|
-
|
|
826
|
-
### Providers and Models
|
|
827
|
-
|
|
828
|
-
A **provider** offers models through a specific API. For example:
|
|
829
|
-
- **Anthropic** models use the `anthropic-messages` API
|
|
830
|
-
- **Google** models use the `google-generative-ai` API
|
|
831
|
-
- **OpenAI** models use the `openai-responses` API
|
|
832
|
-
- **Mistral** models use the `mistral-conversations` API
|
|
833
|
-
- **xAI, Cerebras, Groq, NVIDIA NIM, Together AI, etc.** models use the `openai-completions` API (OpenAI-compatible)
|
|
834
|
-
|
|
835
|
-
### Querying Providers and Models
|
|
953
|
+
Mixed-API providers pass a map keyed by `model.api`; each model dispatches to its API's implementation:
|
|
836
954
|
|
|
837
955
|
```typescript
|
|
838
|
-
import {
|
|
839
|
-
|
|
840
|
-
|
|
841
|
-
const
|
|
842
|
-
|
|
843
|
-
|
|
844
|
-
|
|
845
|
-
|
|
846
|
-
|
|
847
|
-
|
|
848
|
-
|
|
849
|
-
|
|
850
|
-
console.log(` Vision: ${model.input.includes('image')}`);
|
|
851
|
-
console.log(` Reasoning: ${model.reasoning}`);
|
|
852
|
-
}
|
|
853
|
-
|
|
854
|
-
// Get a specific model (both provider and model ID are auto-completed in IDEs)
|
|
855
|
-
const model = getModel('openai', 'gpt-4o-mini');
|
|
856
|
-
console.log(`Using ${model.name} via ${model.api} API`);
|
|
956
|
+
import { anthropicMessagesApi } from '@earendil-works/pi-ai/api/anthropic-messages.lazy';
|
|
957
|
+
import { openAIResponsesApi } from '@earendil-works/pi-ai/api/openai-responses.lazy';
|
|
958
|
+
|
|
959
|
+
const gateway = createProvider({
|
|
960
|
+
id: 'my-gateway',
|
|
961
|
+
auth: { apiKey: envApiKeyAuth('Gateway key', ['GATEWAY_API_KEY']) },
|
|
962
|
+
models: [/* models with api: 'anthropic-messages' or 'openai-responses' */],
|
|
963
|
+
api: {
|
|
964
|
+
'anthropic-messages': anthropicMessagesApi(),
|
|
965
|
+
'openai-responses': openAIResponsesApi(),
|
|
966
|
+
},
|
|
967
|
+
});
|
|
857
968
|
```
|
|
858
969
|
|
|
859
|
-
|
|
860
|
-
|
|
861
|
-
You can create custom models for local inference servers or custom endpoints:
|
|
970
|
+
Dynamic model lists use `refreshModels`; the provider lists empty until the first `models.refresh()`:
|
|
862
971
|
|
|
863
972
|
```typescript
|
|
864
|
-
|
|
865
|
-
|
|
866
|
-
|
|
867
|
-
|
|
868
|
-
|
|
869
|
-
|
|
870
|
-
api: 'openai-completions',
|
|
871
|
-
provider: 'ollama',
|
|
872
|
-
baseUrl: 'http://localhost:11434/v1',
|
|
873
|
-
reasoning: false,
|
|
874
|
-
input: ['text'],
|
|
875
|
-
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
|
|
876
|
-
contextWindow: 128000,
|
|
877
|
-
maxTokens: 32000
|
|
878
|
-
};
|
|
879
|
-
|
|
880
|
-
// Example: LiteLLM proxy with explicit compat settings
|
|
881
|
-
const litellmModel: Model<'openai-completions'> = {
|
|
882
|
-
id: 'gpt-4o',
|
|
883
|
-
name: 'GPT-4o (via LiteLLM)',
|
|
884
|
-
api: 'openai-completions',
|
|
885
|
-
provider: 'litellm',
|
|
886
|
-
baseUrl: 'http://localhost:4000/v1',
|
|
887
|
-
reasoning: false,
|
|
888
|
-
input: ['text', 'image'],
|
|
889
|
-
cost: { input: 2.5, output: 10, cacheRead: 0, cacheWrite: 0 },
|
|
890
|
-
contextWindow: 128000,
|
|
891
|
-
maxTokens: 16384,
|
|
892
|
-
compat: {
|
|
893
|
-
supportsStore: false, // LiteLLM doesn't support the store field
|
|
894
|
-
}
|
|
895
|
-
};
|
|
896
|
-
|
|
897
|
-
// Example: Custom endpoint with headers (bypassing Cloudflare bot detection)
|
|
898
|
-
const proxyModel: Model<'anthropic-messages'> = {
|
|
899
|
-
id: 'claude-sonnet-4',
|
|
900
|
-
name: 'Claude Sonnet 4 (Proxied)',
|
|
901
|
-
api: 'anthropic-messages',
|
|
902
|
-
provider: 'custom-proxy',
|
|
903
|
-
baseUrl: 'https://proxy.example.com/v1',
|
|
904
|
-
reasoning: true,
|
|
905
|
-
input: ['text', 'image'],
|
|
906
|
-
cost: { input: 3, output: 15, cacheRead: 0.3, cacheWrite: 3.75 },
|
|
907
|
-
contextWindow: 200000,
|
|
908
|
-
maxTokens: 8192,
|
|
909
|
-
headers: {
|
|
910
|
-
'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36',
|
|
911
|
-
'X-Custom-Auth': 'bearer-token-here'
|
|
912
|
-
}
|
|
913
|
-
};
|
|
914
|
-
|
|
915
|
-
// Use the custom model
|
|
916
|
-
const response = await stream(ollamaModel, context, {
|
|
917
|
-
apiKey: 'dummy' // Ollama doesn't need a real key
|
|
973
|
+
const llamacpp = createProvider({
|
|
974
|
+
id: 'llamacpp',
|
|
975
|
+
auth: { apiKey: { name: 'llama.cpp', resolve: async () => ({ auth: {} }) } },
|
|
976
|
+
models: [],
|
|
977
|
+
refreshModels: async () => fetchModelsFromServer('http://localhost:8080'),
|
|
978
|
+
api: openAICompletionsApi(),
|
|
918
979
|
});
|
|
980
|
+
|
|
981
|
+
models.setProvider(llamacpp);
|
|
982
|
+
await models.refresh('llamacpp');
|
|
919
983
|
```
|
|
920
984
|
|
|
921
|
-
|
|
985
|
+
Custom models can carry `headers` (e.g. proxies behind bot detection) and `compat` flags — see [OpenAI Compatibility Settings](#openai-compatibility-settings).
|
|
922
986
|
|
|
923
|
-
|
|
987
|
+
Some OpenAI-compatible servers do not understand the `developer` role used for reasoning-capable models. For those providers, set `compat.supportsDeveloperRole` to `false` so the system prompt is sent as a `system` message instead. If the server also does not support `reasoning_effort`, set `compat.supportsReasoningEffort` to `false` too. This commonly applies to Ollama, vLLM, SGLang, and similar OpenAI-compatible servers.
|
|
924
988
|
|
|
925
|
-
|
|
989
|
+
Use model-level `thinkingLevelMap` to describe model-specific thinking controls. Keys are pi thinking levels (`off`, `minimal`, `low`, `medium`, `high`, `xhigh`). Missing keys use provider defaults, string values are sent to the provider, and `null` marks a level unsupported.
|
|
926
990
|
|
|
927
991
|
```typescript
|
|
928
992
|
const ollamaReasoningModel: Model<'openai-completions'> = {
|
|
@@ -950,6 +1014,36 @@ const ollamaReasoningModel: Model<'openai-completions'> = {
|
|
|
950
1014
|
};
|
|
951
1015
|
```
|
|
952
1016
|
|
|
1017
|
+
### Calling API Implementations Directly
|
|
1018
|
+
|
|
1019
|
+
The API implementations are importable on their own. Each module exports exactly `stream` and `streamSimple` with that API's full option typing. Direct calls bypass provider auth — pass `apiKey` explicitly:
|
|
1020
|
+
|
|
1021
|
+
```typescript
|
|
1022
|
+
import { stream } from '@earendil-works/pi-ai/api/anthropic-messages';
|
|
1023
|
+
|
|
1024
|
+
const s = stream(claudeModel, context, {
|
|
1025
|
+
apiKey: process.env.ANTHROPIC_API_KEY,
|
|
1026
|
+
thinkingEnabled: true,
|
|
1027
|
+
thinkingBudgetTokens: 2048,
|
|
1028
|
+
});
|
|
1029
|
+
```
|
|
1030
|
+
|
|
1031
|
+
Built-in API implementations live under `./api/<api-id>`:
|
|
1032
|
+
|
|
1033
|
+
| API id | Options type |
|
|
1034
|
+
|--------|--------------|
|
|
1035
|
+
| `anthropic-messages` | `AnthropicOptions` |
|
|
1036
|
+
| `openai-completions` | `OpenAICompletionsOptions` |
|
|
1037
|
+
| `openai-responses` | `OpenAIResponsesOptions` |
|
|
1038
|
+
| `openai-codex-responses` | `OpenAICodexResponsesOptions` |
|
|
1039
|
+
| `azure-openai-responses` | `AzureOpenAIResponsesOptions` |
|
|
1040
|
+
| `google-generative-ai` | `GoogleOptions` |
|
|
1041
|
+
| `google-vertex` | `GoogleVertexOptions` |
|
|
1042
|
+
| `mistral-conversations` | `MistralOptions` |
|
|
1043
|
+
| `bedrock-converse-stream` | `BedrockOptions` |
|
|
1044
|
+
|
|
1045
|
+
Importing an implementation module loads its SDK. The `./api/<id>.lazy` wrappers (used by the provider factories) defer that load to the first request when the runtime or bundler supports dynamic import chunking. Legacy raw API subpaths from older releases (`./anthropic`, `./google`, `./mistral`, `./openai-completions`, ...) were removed; use `@earendil-works/pi-ai/api/<api-id>`.
|
|
1046
|
+
|
|
953
1047
|
### OpenAI Compatibility Settings
|
|
954
1048
|
|
|
955
1049
|
The `openai-completions` API is implemented by many providers with minor differences. By default, the library auto-detects compatibility settings based on `baseUrl` for a small set of known OpenAI-compatible providers (Cerebras, xAI, Chutes, DeepSeek, NVIDIA NIM, Together AI, zAi, OpenCode, Cloudflare Workers AI, etc.). For custom proxies or unknown endpoints, you can override these settings via the `compat` field. For `openai-responses` models, the compat field supports Responses-specific flags.
|
|
@@ -987,30 +1081,97 @@ If `compat` is not set, the library falls back to URL-based detection. If `compa
|
|
|
987
1081
|
- **Custom inference servers**: May use non-standard field names
|
|
988
1082
|
- **Self-hosted endpoints**: May have different feature support
|
|
989
1083
|
|
|
990
|
-
|
|
1084
|
+
## Faux Provider for Tests
|
|
991
1085
|
|
|
992
|
-
|
|
1086
|
+
`fauxProvider()` builds an in-memory provider with scripted responses for tests and demos:
|
|
993
1087
|
|
|
994
1088
|
```typescript
|
|
995
|
-
import {
|
|
1089
|
+
import {
|
|
1090
|
+
createModels,
|
|
1091
|
+
fauxAssistantMessage,
|
|
1092
|
+
fauxProvider,
|
|
1093
|
+
fauxText,
|
|
1094
|
+
fauxThinking,
|
|
1095
|
+
fauxToolCall,
|
|
1096
|
+
} from '@earendil-works/pi-ai';
|
|
996
1097
|
|
|
997
|
-
|
|
998
|
-
|
|
1098
|
+
const faux = fauxProvider({
|
|
1099
|
+
tokensPerSecond: 50 // optional
|
|
1100
|
+
});
|
|
999
1101
|
|
|
1000
|
-
const
|
|
1001
|
-
|
|
1002
|
-
|
|
1102
|
+
const models = createModels();
|
|
1103
|
+
models.setProvider(faux.provider);
|
|
1104
|
+
|
|
1105
|
+
const model = faux.getModel();
|
|
1106
|
+
const context = {
|
|
1107
|
+
messages: [{ role: 'user', content: 'Summarize package.json and then call echo', timestamp: Date.now() }]
|
|
1003
1108
|
};
|
|
1004
1109
|
|
|
1005
|
-
|
|
1110
|
+
faux.setResponses([
|
|
1111
|
+
fauxAssistantMessage([
|
|
1112
|
+
fauxThinking('Need to inspect package metadata first.'),
|
|
1113
|
+
fauxToolCall('echo', { text: 'package.json' })
|
|
1114
|
+
], { stopReason: 'toolUse' })
|
|
1115
|
+
]);
|
|
1116
|
+
|
|
1117
|
+
const first = await models.complete(model, context, {
|
|
1118
|
+
sessionId: 'session-1',
|
|
1119
|
+
cacheRetention: 'short'
|
|
1120
|
+
});
|
|
1121
|
+
context.messages.push(first);
|
|
1122
|
+
|
|
1123
|
+
context.messages.push({
|
|
1124
|
+
role: 'toolResult',
|
|
1125
|
+
toolCallId: first.content.find((block) => block.type === 'toolCall')!.id,
|
|
1126
|
+
toolName: 'echo',
|
|
1127
|
+
content: [{ type: 'text', text: 'package.json contents here' }],
|
|
1128
|
+
isError: false,
|
|
1129
|
+
timestamp: Date.now()
|
|
1130
|
+
});
|
|
1131
|
+
|
|
1132
|
+
faux.setResponses([
|
|
1133
|
+
fauxAssistantMessage([
|
|
1134
|
+
fauxThinking('Now I can summarize the tool output.'),
|
|
1135
|
+
fauxText('Here is the summary.')
|
|
1136
|
+
])
|
|
1137
|
+
]);
|
|
1138
|
+
|
|
1139
|
+
const s = models.stream(model, context);
|
|
1140
|
+
for await (const event of s) {
|
|
1141
|
+
console.log(event.type);
|
|
1142
|
+
}
|
|
1143
|
+
|
|
1144
|
+
// Optional: multiple faux models for model-switching tests
|
|
1145
|
+
const multiModel = fauxProvider({
|
|
1146
|
+
provider: 'faux-multi',
|
|
1147
|
+
models: [
|
|
1148
|
+
{ id: 'faux-fast', reasoning: false },
|
|
1149
|
+
{ id: 'faux-thinker', reasoning: true }
|
|
1150
|
+
]
|
|
1151
|
+
});
|
|
1152
|
+
models.setProvider(multiModel.provider);
|
|
1153
|
+
const thinker = multiModel.getModel('faux-thinker');
|
|
1154
|
+
|
|
1155
|
+
console.log(thinker?.reasoning);
|
|
1156
|
+
console.log(faux.getPendingResponseCount());
|
|
1157
|
+
console.log(faux.state.callCount);
|
|
1006
1158
|
```
|
|
1007
1159
|
|
|
1160
|
+
Notes:
|
|
1161
|
+
- Responses are consumed from a queue in request start order.
|
|
1162
|
+
- If the queue is empty, the faux provider returns an assistant error message with `errorMessage: "No more faux responses queued"`.
|
|
1163
|
+
- Use `faux.setResponses([...])` to replace the remaining queue and `faux.appendResponses([...])` to add more responses.
|
|
1164
|
+
- `faux.models` exposes all faux models. `faux.getModel()` returns the first one, and `faux.getModel(id)` returns a specific one.
|
|
1165
|
+
- Use `fauxAssistantMessage(...)` for scripted assistant replies. Use `fauxText(...)`, `fauxThinking(...)`, and `fauxToolCall(...)` to build content blocks without filling in low-level fields manually.
|
|
1166
|
+
- Usage is estimated at roughly 1 token per 4 characters. When `sessionId` is present and `cacheRetention` is not `"none"`, prompt cache reads and writes are simulated automatically.
|
|
1167
|
+
- Tool call arguments stream incrementally via `toolcall_delta` chunks.
|
|
1168
|
+
- By default, each streamed chunk is emitted on its own microtask. Set `tokensPerSecond` to pace chunk delivery in real time.
|
|
1169
|
+
- The intended use is one deterministic scripted flow per handle. If you need independent concurrent flows, create separate faux providers with distinct `provider` ids.
|
|
1170
|
+
|
|
1008
1171
|
## Cross-Provider Handoffs
|
|
1009
1172
|
|
|
1010
1173
|
The library supports seamless handoffs between different LLM providers within the same conversation. This allows you to switch models mid-conversation while preserving context, including thinking blocks, tool calls, and tool results.
|
|
1011
1174
|
|
|
1012
|
-
### How It Works
|
|
1013
|
-
|
|
1014
1175
|
When messages from one provider are sent to a different provider, the library automatically transforms them for compatibility:
|
|
1015
1176
|
|
|
1016
1177
|
- **User and tool result messages** are passed through unchanged
|
|
@@ -1018,98 +1179,86 @@ When messages from one provider are sent to a different provider, the library au
|
|
|
1018
1179
|
- **Assistant messages from different providers** have their thinking blocks converted to text with `<thinking>` tags
|
|
1019
1180
|
- **Tool calls and regular text** are preserved unchanged
|
|
1020
1181
|
|
|
1021
|
-
### Example: Multi-Provider Conversation
|
|
1022
|
-
|
|
1023
1182
|
```typescript
|
|
1024
|
-
import {
|
|
1183
|
+
import { createModels, type Context } from '@earendil-works/pi-ai';
|
|
1184
|
+
import { anthropicProvider } from '@earendil-works/pi-ai/providers/anthropic';
|
|
1185
|
+
import { openaiProvider } from '@earendil-works/pi-ai/providers/openai';
|
|
1186
|
+
import { googleProvider } from '@earendil-works/pi-ai/providers/google';
|
|
1025
1187
|
|
|
1026
|
-
|
|
1027
|
-
|
|
1028
|
-
|
|
1029
|
-
|
|
1030
|
-
};
|
|
1188
|
+
const models = createModels();
|
|
1189
|
+
models.setProvider(anthropicProvider());
|
|
1190
|
+
models.setProvider(openaiProvider());
|
|
1191
|
+
models.setProvider(googleProvider());
|
|
1031
1192
|
|
|
1032
|
-
context
|
|
1033
|
-
|
|
1034
|
-
|
|
1035
|
-
|
|
1036
|
-
context.messages.push(
|
|
1193
|
+
const context: Context = { messages: [] };
|
|
1194
|
+
|
|
1195
|
+
// Start with Claude
|
|
1196
|
+
const claude = models.getModel('anthropic', 'claude-sonnet-4-5')!;
|
|
1197
|
+
context.messages.push({ role: 'user', content: 'What is 25 * 18?', timestamp: Date.now() });
|
|
1198
|
+
context.messages.push(await models.completeSimple(claude, context, { reasoning: 'medium' }));
|
|
1037
1199
|
|
|
1038
1200
|
// Switch to GPT-5 - it will see Claude's thinking as <thinking> tagged text
|
|
1039
|
-
const gpt5 = getModel('openai', 'gpt-5-mini')
|
|
1040
|
-
context.messages.push({ role: 'user', content: 'Is that calculation correct?' });
|
|
1041
|
-
|
|
1042
|
-
context.messages.push(gptResponse);
|
|
1201
|
+
const gpt5 = models.getModel('openai', 'gpt-5-mini')!;
|
|
1202
|
+
context.messages.push({ role: 'user', content: 'Is that calculation correct?', timestamp: Date.now() });
|
|
1203
|
+
context.messages.push(await models.complete(gpt5, context));
|
|
1043
1204
|
|
|
1044
1205
|
// Switch to Gemini
|
|
1045
|
-
const gemini = getModel('google', 'gemini-2.5-flash')
|
|
1046
|
-
context.messages.push({ role: 'user', content: 'What was the original question?' });
|
|
1047
|
-
const geminiResponse = await complete(gemini, context);
|
|
1206
|
+
const gemini = models.getModel('google', 'gemini-2.5-flash')!;
|
|
1207
|
+
context.messages.push({ role: 'user', content: 'What was the original question?', timestamp: Date.now() });
|
|
1208
|
+
const geminiResponse = await models.complete(gemini, context);
|
|
1048
1209
|
```
|
|
1049
1210
|
|
|
1050
|
-
|
|
1051
|
-
|
|
1052
|
-
All providers can handle messages from other providers, including:
|
|
1053
|
-
- Text content
|
|
1054
|
-
- Tool calls and tool results (including images in tool results)
|
|
1055
|
-
- Thinking/reasoning blocks (transformed to tagged text for cross-provider compatibility)
|
|
1056
|
-
- Aborted messages with partial content
|
|
1057
|
-
|
|
1058
|
-
This enables flexible workflows where you can:
|
|
1059
|
-
- Start with a fast model for initial responses
|
|
1060
|
-
- Switch to a more capable model for complex reasoning
|
|
1061
|
-
- Use specialized models for specific tasks
|
|
1062
|
-
- Maintain conversation continuity across provider outages
|
|
1211
|
+
All providers can handle messages from other providers — text, tool calls and results (including images), thinking blocks (transformed to tagged text), and aborted messages with partial content. This enables flexible workflows: start with a fast model, switch to a more capable one for complex reasoning, or maintain continuity across provider outages.
|
|
1063
1212
|
|
|
1064
1213
|
## Context Serialization
|
|
1065
1214
|
|
|
1066
1215
|
The `Context` object can be easily serialized and deserialized using standard JSON methods, making it simple to persist conversations, implement chat history, or transfer contexts between services:
|
|
1067
1216
|
|
|
1068
1217
|
```typescript
|
|
1069
|
-
import { Context, getModel, complete } from '@earendil-works/pi-ai';
|
|
1070
|
-
|
|
1071
|
-
// Create and use a context
|
|
1072
1218
|
const context: Context = {
|
|
1073
1219
|
systemPrompt: 'You are a helpful assistant.',
|
|
1074
1220
|
messages: [
|
|
1075
|
-
{ role: 'user', content: 'What is TypeScript?' }
|
|
1221
|
+
{ role: 'user', content: 'What is TypeScript?', timestamp: Date.now() }
|
|
1076
1222
|
]
|
|
1077
1223
|
};
|
|
1078
1224
|
|
|
1079
|
-
const model = getModel('openai', 'gpt-4o-mini')
|
|
1080
|
-
const response = await complete(model, context);
|
|
1225
|
+
const model = models.getModel('openai', 'gpt-4o-mini')!;
|
|
1226
|
+
const response = await models.complete(model, context);
|
|
1081
1227
|
context.messages.push(response);
|
|
1082
1228
|
|
|
1083
1229
|
// Serialize the entire context
|
|
1084
1230
|
const serialized = JSON.stringify(context);
|
|
1085
|
-
console.log('Serialized context size:', serialized.length, 'bytes');
|
|
1086
1231
|
|
|
1087
1232
|
// Save to database, localStorage, file, etc.
|
|
1088
1233
|
localStorage.setItem('conversation', serialized);
|
|
1089
1234
|
|
|
1090
1235
|
// Later: deserialize and continue the conversation
|
|
1091
1236
|
const restored: Context = JSON.parse(localStorage.getItem('conversation')!);
|
|
1092
|
-
restored.messages.push({ role: 'user', content: 'Tell me more about its type system' });
|
|
1237
|
+
restored.messages.push({ role: 'user', content: 'Tell me more about its type system', timestamp: Date.now() });
|
|
1093
1238
|
|
|
1094
1239
|
// Continue with any model
|
|
1095
|
-
const newModel = getModel('anthropic', 'claude-3-5-haiku-20241022')
|
|
1096
|
-
const continuation = await complete(newModel, restored);
|
|
1240
|
+
const newModel = models.getModel('anthropic', 'claude-3-5-haiku-20241022')!;
|
|
1241
|
+
const continuation = await models.complete(newModel, restored);
|
|
1097
1242
|
```
|
|
1098
1243
|
|
|
1244
|
+
Models are plain serializable data too — no functions or implementations attached — so persisting "which model was this conversation using" is a `JSON.stringify` away.
|
|
1245
|
+
|
|
1099
1246
|
> **Note**: If the context contains images (encoded as base64 as shown in the Image Input section), those will also be serialized.
|
|
1100
1247
|
|
|
1101
1248
|
## Browser Usage
|
|
1102
1249
|
|
|
1103
|
-
The library supports browser environments.
|
|
1250
|
+
The library supports browser environments. The core entrypoint and provider factories are side-effect free and bundle cleanly. Environment variables are not available in browsers, so pass API keys explicitly — or inject a `CredentialStore` (e.g. localStorage-backed) and let provider auth resolve from stored credentials:
|
|
1104
1251
|
|
|
1105
1252
|
```typescript
|
|
1106
|
-
import {
|
|
1253
|
+
import { createModels } from '@earendil-works/pi-ai';
|
|
1254
|
+
import { anthropicProvider } from '@earendil-works/pi-ai/providers/anthropic';
|
|
1107
1255
|
|
|
1108
|
-
|
|
1109
|
-
|
|
1256
|
+
const models = createModels();
|
|
1257
|
+
models.setProvider(anthropicProvider());
|
|
1110
1258
|
|
|
1111
|
-
const
|
|
1112
|
-
|
|
1259
|
+
const model = models.getModel('anthropic', 'claude-3-5-haiku-20241022')!;
|
|
1260
|
+
const response = await models.complete(model, {
|
|
1261
|
+
messages: [{ role: 'user', content: 'Hello!', timestamp: Date.now() }]
|
|
1113
1262
|
}, {
|
|
1114
1263
|
apiKey: 'your-api-key'
|
|
1115
1264
|
});
|
|
@@ -1117,70 +1266,75 @@ const response = await complete(model, {
|
|
|
1117
1266
|
|
|
1118
1267
|
> **Security Warning**: Exposing API keys in frontend code is dangerous. Anyone can extract and abuse your keys. Only use this approach for internal tools or demos. For production applications, use a backend proxy that keeps your API keys secure.
|
|
1119
1268
|
|
|
1120
|
-
|
|
1269
|
+
Browser compatibility notes:
|
|
1121
1270
|
|
|
1122
|
-
- Amazon Bedrock (`bedrock-converse-stream`) is not supported in browser environments.
|
|
1123
|
-
- OAuth login flows are
|
|
1124
|
-
- In browser builds, Bedrock can still appear in model lists. Calls to Bedrock models fail at runtime.
|
|
1271
|
+
- Amazon Bedrock (`bedrock-converse-stream`) is not supported in browser environments. It can still appear in model lists; calls fail at runtime.
|
|
1272
|
+
- OAuth login flows are Node-only. They are lazy-loaded behind bundler-opaque imports, so registering an OAuth-capable provider does not pull Node-only code into a browser bundle — only actually logging in would.
|
|
1125
1273
|
- Use a server-side proxy or backend service if you need Bedrock or OAuth-based auth from a web app.
|
|
1126
|
-
- Use `@earendil-works/pi-ai/base` plus explicit direct transport registration when a browser bundle should exclude unused provider SDK implementations.
|
|
1127
1274
|
|
|
1128
|
-
|
|
1275
|
+
## Bundling and Tree Shaking
|
|
1129
1276
|
|
|
1130
|
-
|
|
1277
|
+
For small bundles, import only the providers you need:
|
|
1131
1278
|
|
|
1132
|
-
|
|
1133
|
-
|
|
1134
|
-
|
|
1135
|
-
|
|
1136
|
-
|
|
1137
|
-
|
|
1138
|
-
|
|
1139
|
-
|
|
1140
|
-
|
|
1141
|
-
|
|
1142
|
-
|
|
1143
|
-
|
|
1144
|
-
|
|
1145
|
-
|
|
1146
|
-
|
|
1147
|
-
|
|
1148
|
-
|
|
1149
|
-
|
|
1150
|
-
|
|
1151
|
-
|
|
1152
|
-
|
|
1153
|
-
|
|
1154
|
-
|
|
1155
|
-
|
|
1156
|
-
|
|
1157
|
-
|
|
1158
|
-
|
|
1159
|
-
|
|
1160
|
-
| Xiaomi MiMo Token Plan (Singapore) | `XIAOMI_TOKEN_PLAN_SGP_API_KEY` |
|
|
1161
|
-
| GitHub Copilot | `COPILOT_GITHUB_TOKEN` |
|
|
1279
|
+
```typescript
|
|
1280
|
+
import { createModels } from '@earendil-works/pi-ai';
|
|
1281
|
+
import { openaiProvider } from '@earendil-works/pi-ai/providers/openai';
|
|
1282
|
+
|
|
1283
|
+
const models = createModels();
|
|
1284
|
+
models.setProvider(openaiProvider());
|
|
1285
|
+
```
|
|
1286
|
+
|
|
1287
|
+
Rules:
|
|
1288
|
+
|
|
1289
|
+
- `@earendil-works/pi-ai` is the core entrypoint and does not import built-in catalogs, provider factories, or SDK implementations.
|
|
1290
|
+
- `@earendil-works/pi-ai/providers/<provider>` imports that provider's catalog and lazy API wrapper only.
|
|
1291
|
+
- `@earendil-works/pi-ai/providers/all` imports every built-in provider factory and all catalogs. Use it only when you want the full built-in set.
|
|
1292
|
+
- With code splitting, provider SDKs stay in lazy chunks and load on first request.
|
|
1293
|
+
- Without code splitting, bundlers fold reachable lazy API implementations into the single bundle. A single-provider bundle then includes that provider's SDK; `providers/all` includes all statically visible SDKs. Bedrock is the exception: its AWS SDK implementation is loaded through a bundler-opaque Node-only import.
|
|
1294
|
+
- Importing `@earendil-works/pi-ai/api/<api-id>` directly loads that API implementation and its SDK immediately.
|
|
1295
|
+
|
|
1296
|
+
Avoid `@earendil-works/pi-ai/compat` in new bundled apps; it preserves the old global API and imports the full built-in catalog surface.
|
|
1297
|
+
|
|
1298
|
+
For single-file Node ESM bundles, some SDK dependencies may still use dynamic CommonJS `require()` internally. If you see errors such as `Dynamic require of "child_process" is not supported`, add a Node `require` shim to the bundle. With esbuild:
|
|
1299
|
+
|
|
1300
|
+
```bash
|
|
1301
|
+
esbuild app.js --bundle --platform=node --format=esm \
|
|
1302
|
+
--banner:js='import { createRequire } from "module";const require = createRequire(import.meta.url);' \
|
|
1303
|
+
--outfile=app.bundle.js
|
|
1304
|
+
```
|
|
1305
|
+
|
|
1306
|
+
This is only for Node bundles; it is not a browser or Cloudflare Workers workaround.
|
|
1162
1307
|
|
|
1163
|
-
|
|
1308
|
+
Bedrock is Node-only. Add it like any other provider:
|
|
1164
1309
|
|
|
1165
1310
|
```typescript
|
|
1166
|
-
|
|
1167
|
-
|
|
1168
|
-
const response = await complete(model, context);
|
|
1311
|
+
import { createModels } from '@earendil-works/pi-ai';
|
|
1312
|
+
import { amazonBedrockProvider } from '@earendil-works/pi-ai/providers/amazon-bedrock';
|
|
1169
1313
|
|
|
1170
|
-
|
|
1171
|
-
|
|
1172
|
-
apiKey: 'sk-different-key'
|
|
1173
|
-
});
|
|
1314
|
+
const models = createModels();
|
|
1315
|
+
models.setProvider(amazonBedrockProvider());
|
|
1174
1316
|
```
|
|
1175
1317
|
|
|
1318
|
+
In normal Node package usage and code-split bundles, Bedrock loads its AWS SDK implementation lazily. For a standalone single-file bundle that must include Bedrock support, register the implementation module explicitly:
|
|
1319
|
+
|
|
1320
|
+
```typescript
|
|
1321
|
+
import { setBedrockProviderModule } from '@earendil-works/pi-ai/api/bedrock-converse-stream.lazy';
|
|
1322
|
+
import { bedrockProviderModule } from '@earendil-works/pi-ai/bedrock-provider';
|
|
1323
|
+
|
|
1324
|
+
setBedrockProviderModule(bedrockProviderModule);
|
|
1325
|
+
```
|
|
1326
|
+
|
|
1327
|
+
That explicit override bundles the AWS SDK. Without it, Bedrock's opaque runtime import expects the package's Bedrock implementation file to be available at runtime.
|
|
1328
|
+
|
|
1176
1329
|
### Provider-Scoped Environment Overrides
|
|
1177
1330
|
|
|
1178
|
-
Pass `env` in stream options to scope provider configuration to a request. Values in `env` are used before process environment variables for
|
|
1331
|
+
Pass `env` in stream options to scope provider configuration to a request. Values in `env` are used before process environment variables for provider auth and configuration such as Cloudflare account IDs, Azure OpenAI settings, Vertex project/location, Bedrock settings, `PI_CACHE_RETENTION`, and `HTTP_PROXY`/`HTTPS_PROXY`.
|
|
1179
1332
|
|
|
1180
1333
|
```typescript
|
|
1181
|
-
const
|
|
1334
|
+
const models = builtinModels();
|
|
1335
|
+
const model = models.getModel('cloudflare-ai-gateway', 'workers-ai/@cf/moonshotai/kimi-k2.6')!;
|
|
1182
1336
|
|
|
1183
|
-
const response = await complete(model, context, {
|
|
1337
|
+
const response = await models.complete(model, context, {
|
|
1184
1338
|
env: {
|
|
1185
1339
|
CLOUDFLARE_API_KEY: '...',
|
|
1186
1340
|
CLOUDFLARE_ACCOUNT_ID: 'account-id',
|
|
@@ -1191,24 +1345,47 @@ const response = await complete(model, context, {
|
|
|
1191
1345
|
|
|
1192
1346
|
Use this when one process needs different provider settings per request, or when ambient environment variables should not leak into a provider call.
|
|
1193
1347
|
|
|
1194
|
-
### Checking Environment Variables
|
|
1195
|
-
|
|
1196
|
-
```typescript
|
|
1197
|
-
import { getEnvApiKey } from '@earendil-works/pi-ai';
|
|
1198
|
-
|
|
1199
|
-
// Check if an API key is set in environment variables
|
|
1200
|
-
const key = getEnvApiKey('openai'); // checks OPENAI_API_KEY
|
|
1201
|
-
```
|
|
1202
|
-
|
|
1203
1348
|
## OAuth Providers
|
|
1204
1349
|
|
|
1205
|
-
Several providers
|
|
1350
|
+
Several providers support OAuth authentication instead of static API keys:
|
|
1206
1351
|
|
|
1207
1352
|
- **Anthropic** (Claude Pro/Max subscription)
|
|
1208
1353
|
- **OpenAI Codex** (ChatGPT Plus/Pro subscription, access to GPT-5.x Codex models)
|
|
1209
1354
|
- **GitHub Copilot** (Copilot subscription)
|
|
1210
1355
|
|
|
1211
|
-
|
|
1356
|
+
Each of these providers carries an `OAuthAuth` on `provider.auth.oauth` with three operations: `login(callbacks)` runs the interactive flow and returns a credential, `refresh(credential)` exchanges the refresh token, and `toAuth(credential)` derives request auth (GitHub Copilot's per-account base URL comes from here). Refresh is automatic: `models.getAuth()` and the request paths refresh expired tokens under a credential-store lock, so concurrent requests and processes cannot double-refresh.
|
|
1357
|
+
|
|
1358
|
+
```typescript
|
|
1359
|
+
import { createModels } from '@earendil-works/pi-ai';
|
|
1360
|
+
import { anthropicProvider } from '@earendil-works/pi-ai/providers/anthropic';
|
|
1361
|
+
|
|
1362
|
+
const models = createModels({ credentials: myStore }); // persistent CredentialStore
|
|
1363
|
+
models.setProvider(anthropicProvider());
|
|
1364
|
+
|
|
1365
|
+
// Login: drive the flow with prompt()/notify() callbacks, persist the credential
|
|
1366
|
+
const provider = models.getProvider('anthropic')!;
|
|
1367
|
+
const credential = await provider.auth.oauth!.login({
|
|
1368
|
+
prompt: async (p) => {
|
|
1369
|
+
// p.type: 'text' | 'secret' | 'select' | 'manual_code'
|
|
1370
|
+
// manual_code prompts race a local callback server; p.signal aborts them when the server wins
|
|
1371
|
+
return await askUser(p.message);
|
|
1372
|
+
},
|
|
1373
|
+
notify: (event) => {
|
|
1374
|
+
// event.type: 'auth_url' | 'device_code' | 'progress'
|
|
1375
|
+
if (event.type === 'auth_url') console.log(`Open: ${event.url}`);
|
|
1376
|
+
if (event.type === 'device_code') console.log(`Code: ${event.userCode} at ${event.verificationUri}`);
|
|
1377
|
+
if (event.type === 'progress') console.log(event.message);
|
|
1378
|
+
},
|
|
1379
|
+
});
|
|
1380
|
+
await myStore.modify('anthropic', async () => credential);
|
|
1381
|
+
|
|
1382
|
+
// From here on, requests resolve and refresh the token automatically
|
|
1383
|
+
const model = models.getModel('anthropic', 'claude-sonnet-4-5')!;
|
|
1384
|
+
await models.complete(model, context);
|
|
1385
|
+
|
|
1386
|
+
// Logout
|
|
1387
|
+
await myStore.delete('anthropic');
|
|
1388
|
+
```
|
|
1212
1389
|
|
|
1213
1390
|
### Vertex AI
|
|
1214
1391
|
|
|
@@ -1220,8 +1397,6 @@ Vertex AI models support either a Google Cloud API key or Application Default Cr
|
|
|
1220
1397
|
|
|
1221
1398
|
When using ADC, also set `GOOGLE_CLOUD_PROJECT` (or `GCLOUD_PROJECT`) and `GOOGLE_CLOUD_LOCATION`. You can also pass `project`/`location` in the call options. When using `GOOGLE_CLOUD_API_KEY`, `project` and `location` are not required.
|
|
1222
1399
|
|
|
1223
|
-
Example:
|
|
1224
|
-
|
|
1225
1400
|
```bash
|
|
1226
1401
|
# Local (uses your user credentials)
|
|
1227
1402
|
gcloud auth application-default login
|
|
@@ -1232,23 +1407,6 @@ export GOOGLE_CLOUD_LOCATION="us-central1"
|
|
|
1232
1407
|
export GOOGLE_APPLICATION_CREDENTIALS="/path/to/service-account.json"
|
|
1233
1408
|
```
|
|
1234
1409
|
|
|
1235
|
-
```typescript
|
|
1236
|
-
import { getModel, complete } from '@earendil-works/pi-ai';
|
|
1237
|
-
|
|
1238
|
-
(async () => {
|
|
1239
|
-
const model = getModel('google-vertex', 'gemini-2.5-flash');
|
|
1240
|
-
const response = await complete(model, {
|
|
1241
|
-
messages: [{ role: 'user', content: 'Hello from Vertex AI' }]
|
|
1242
|
-
}, {
|
|
1243
|
-
apiKey: process.env.GOOGLE_CLOUD_API_KEY,
|
|
1244
|
-
});
|
|
1245
|
-
|
|
1246
|
-
for (const block of response.content) {
|
|
1247
|
-
if (block.type === 'text') console.log(block.text);
|
|
1248
|
-
}
|
|
1249
|
-
})().catch(console.error);
|
|
1250
|
-
```
|
|
1251
|
-
|
|
1252
1410
|
Official docs: [Application Default Credentials](https://cloud.google.com/docs/authentication/application-default-credentials)
|
|
1253
1411
|
|
|
1254
1412
|
### CLI Login
|
|
@@ -1265,124 +1423,77 @@ Credentials are saved to `auth.json` in the current directory.
|
|
|
1265
1423
|
|
|
1266
1424
|
### Programmatic OAuth
|
|
1267
1425
|
|
|
1268
|
-
The
|
|
1269
|
-
|
|
1270
|
-
```typescript
|
|
1271
|
-
import {
|
|
1272
|
-
// Login functions (return credentials, do not store)
|
|
1273
|
-
loginAnthropic,
|
|
1274
|
-
loginOpenAICodex,
|
|
1275
|
-
loginGitHubCopilot,
|
|
1276
|
-
loginGeminiCli,
|
|
1277
|
-
|
|
1278
|
-
// Token management
|
|
1279
|
-
refreshOAuthToken, // (provider, credentials) => new credentials
|
|
1280
|
-
getOAuthApiKey, // (provider, credentialsMap) => { newCredentials, apiKey } | null
|
|
1281
|
-
|
|
1282
|
-
// Types
|
|
1283
|
-
type OAuthProvider,
|
|
1284
|
-
type OAuthCredentials,
|
|
1285
|
-
} from '@earendil-works/pi-ai/oauth';
|
|
1286
|
-
```
|
|
1426
|
+
The legacy flow functions remain available via the `@earendil-works/pi-ai/oauth` entry point (`loginAnthropic`, `loginOpenAICodex`, `loginGitHubCopilot`, `refreshOAuthToken`, `getOAuthApiKey`); credential storage is the caller's responsibility there. New code should prefer the provider-owned `OAuthAuth` shown above — it composes with the credential store and gets locked auto-refresh for free.
|
|
1287
1427
|
|
|
1288
|
-
|
|
1428
|
+
Provider notes:
|
|
1289
1429
|
|
|
1290
|
-
|
|
1291
|
-
import { loginGitHubCopilot } from '@earendil-works/pi-ai/oauth';
|
|
1292
|
-
import { writeFileSync } from 'fs';
|
|
1430
|
+
**OpenAI Codex**: Requires a ChatGPT Plus or Pro subscription. Provides access to GPT-5.x Codex models with extended context windows and reasoning capabilities. The library automatically handles session-based prompt caching when `sessionId` is provided in stream options. You can set `transport` in stream options to `"sse"`, `"websocket"`, or `"auto"` for Codex Responses transport selection. When using WebSocket with a `sessionId`, connections are reused per session and expire after 5 minutes of inactivity.
|
|
1293
1431
|
|
|
1294
|
-
|
|
1295
|
-
onAuth: (url, instructions) => {
|
|
1296
|
-
console.log(`Open: ${url}`);
|
|
1297
|
-
if (instructions) console.log(instructions);
|
|
1298
|
-
},
|
|
1299
|
-
onPrompt: async (prompt) => {
|
|
1300
|
-
return await getUserInput(prompt.message);
|
|
1301
|
-
},
|
|
1302
|
-
onProgress: (message) => console.log(message)
|
|
1303
|
-
});
|
|
1432
|
+
**Azure OpenAI (Responses)**: Uses the Responses API only. Set `AZURE_OPENAI_API_KEY` and either `AZURE_OPENAI_BASE_URL` or `AZURE_OPENAI_RESOURCE_NAME`. `AZURE_OPENAI_BASE_URL` supports both `https://<resource>.openai.azure.com` and `https://<resource>.cognitiveservices.azure.com`; root endpoints are normalized to `.../openai/v1` automatically. Use `AZURE_OPENAI_API_VERSION` (defaults to `v1`) to override the API version if needed. Deployment names are treated as model IDs by default, override with `azureDeploymentName` or `AZURE_OPENAI_DEPLOYMENT_NAME_MAP` using comma-separated `model-id=deployment` pairs (for example `gpt-4o-mini=my-deployment,gpt-4o=prod`). Legacy deployment-based URLs are intentionally unsupported.
|
|
1304
1433
|
|
|
1305
|
-
|
|
1306
|
-
const auth = { 'github-copilot': { type: 'oauth', ...credentials } };
|
|
1307
|
-
writeFileSync('auth.json', JSON.stringify(auth, null, 2));
|
|
1308
|
-
```
|
|
1434
|
+
**GitHub Copilot**: If you get "The requested model is not supported" error, enable the model manually in VS Code: open Copilot Chat, click the model selector, select the model (warning icon), and click "Enable".
|
|
1309
1435
|
|
|
1310
|
-
|
|
1436
|
+
## Migrating from the Old Global API
|
|
1311
1437
|
|
|
1312
|
-
|
|
1438
|
+
Older versions exposed a global API: `stream()`/`complete()` dispatching on `model.api` via a global registry, sync `getModel()`/`getModels()`/`getProviders()` catalog reads, `registerApiProvider()`, `getEnvApiKey()`, and per-API lazy stream functions. That surface lives unchanged on the **compat entrypoint**:
|
|
1313
1439
|
|
|
1314
1440
|
```typescript
|
|
1441
|
+
// Before
|
|
1315
1442
|
import { getModel, complete } from '@earendil-works/pi-ai';
|
|
1316
|
-
import { getOAuthApiKey } from '@earendil-works/pi-ai/oauth';
|
|
1317
|
-
import { readFileSync, writeFileSync } from 'fs';
|
|
1318
|
-
|
|
1319
|
-
// Load your stored credentials
|
|
1320
|
-
const auth = JSON.parse(readFileSync('auth.json', 'utf-8'));
|
|
1321
|
-
|
|
1322
|
-
// Get API key (refreshes if expired)
|
|
1323
|
-
const result = await getOAuthApiKey('github-copilot', auth);
|
|
1324
|
-
if (!result) throw new Error('Not logged in');
|
|
1325
1443
|
|
|
1326
|
-
//
|
|
1327
|
-
|
|
1328
|
-
writeFileSync('auth.json', JSON.stringify(auth, null, 2));
|
|
1329
|
-
|
|
1330
|
-
// Use the API key
|
|
1331
|
-
const model = getModel('github-copilot', 'gpt-4o');
|
|
1332
|
-
const response = await complete(model, {
|
|
1333
|
-
messages: [{ role: 'user', content: 'Hello!' }]
|
|
1334
|
-
}, { apiKey: result.apiKey });
|
|
1444
|
+
// After (verbatim behavior, one import-path change)
|
|
1445
|
+
import { getModel, complete } from '@earendil-works/pi-ai/compat';
|
|
1335
1446
|
```
|
|
1336
1447
|
|
|
1337
|
-
|
|
1448
|
+
Compat is a strict superset of the root entrypoint, so a file can switch its import path wholesale. It will be removed in a future release; migrate to `createModels()` + provider factories:
|
|
1338
1449
|
|
|
1339
|
-
|
|
1340
|
-
|
|
1341
|
-
|
|
1342
|
-
|
|
1343
|
-
|
|
1450
|
+
| Old | New |
|
|
1451
|
+
|-----|-----|
|
|
1452
|
+
| `getModel('openai', 'gpt-4o-mini')` | `models.getModel('openai', 'gpt-4o-mini')` or `getBuiltinModel()` from `providers/all` |
|
|
1453
|
+
| `getModels('anthropic')` / `getProviders()` | `models.getModels('anthropic')` / `models.getProviders()` or `getBuiltin*` |
|
|
1454
|
+
| `stream(model, ctx, opts)` (env-key injection) | `models.stream(model, ctx, opts)` (provider auth resolution) |
|
|
1455
|
+
| `registerApiProvider({ api, stream, streamSimple })` | `createProvider({ id, auth, models, api })` + `models.setProvider()` |
|
|
1456
|
+
| `getEnvApiKey('openai')` | `await models.getAuth(model)` |
|
|
1457
|
+
| `streamAnthropic(model, ctx, opts)` | `stream` from `@earendil-works/pi-ai/api/anthropic-messages`, or a provider in a collection |
|
|
1458
|
+
| `registerFauxProvider()` | `fauxProvider()` + `models.setProvider()` |
|
|
1344
1459
|
|
|
1345
1460
|
## Development
|
|
1346
1461
|
|
|
1347
1462
|
### Adding a New Provider
|
|
1348
1463
|
|
|
1349
|
-
Adding a new LLM provider requires changes across multiple files. This checklist covers all necessary steps:
|
|
1464
|
+
Adding a new LLM provider requires changes across multiple files. The layered layout: API implementations live in `src/api/`, provider factories in `src/providers/`, generated catalogs in `src/providers/<id>.models.ts`. This checklist covers all necessary steps:
|
|
1350
1465
|
|
|
1351
1466
|
#### 1. Core Types (`src/types.ts`)
|
|
1352
1467
|
|
|
1353
|
-
- Add the API identifier to `KnownApi` (for example `"bedrock-converse-stream"`)
|
|
1354
|
-
- Create an options interface extending `StreamOptions` (for example `BedrockOptions`)
|
|
1468
|
+
- Add the API identifier to `KnownApi` (for example `"bedrock-converse-stream"`), if it is a new API
|
|
1355
1469
|
- Add the provider name to `KnownProvider` (for example `"amazon-bedrock"`)
|
|
1470
|
+
- Add the options type to `ApiOptionsMap`
|
|
1356
1471
|
|
|
1357
|
-
#### 2.
|
|
1472
|
+
#### 2. API Implementation (`src/api/<api-id>.ts`, only for a new API)
|
|
1358
1473
|
|
|
1359
|
-
Create a new
|
|
1474
|
+
Create a new API implementation file (for example `bedrock-converse-stream.ts`) that exports exactly `stream` and `streamSimple`, plus:
|
|
1360
1475
|
|
|
1361
|
-
- `
|
|
1362
|
-
- `streamSimple<Provider>()` for `SimpleStreamOptions` mapping
|
|
1363
|
-
- `register()` for explicit direct transport registration from `@earendil-works/pi-ai/base`
|
|
1364
|
-
- Provider-specific options interface
|
|
1476
|
+
- An options interface extending `StreamOptions` (for example `BedrockOptions`)
|
|
1365
1477
|
- Message conversion functions to transform `Context` to provider format
|
|
1366
1478
|
- Tool conversion if the provider supports tools
|
|
1367
1479
|
- Response parsing to emit standardized events (`text`, `tool_call`, `thinking`, `usage`, `stop`)
|
|
1368
1480
|
|
|
1369
|
-
|
|
1481
|
+
Add a lazy wrapper `src/api/<api-id>.lazy.ts` (`<name>Api()` via `lazyApi()`) so providers can reference the implementation without importing its SDK. Add any root-level `export type` re-exports in `src/index.ts` that should remain available from `@earendil-works/pi-ai`.
|
|
1370
1482
|
|
|
1371
|
-
|
|
1372
|
-
- Add a package subpath export in `package.json` for the provider module (`./dist/providers/<provider>.js`)
|
|
1373
|
-
- Add lazy loader wrappers in `src/providers/register-builtins.ts`, do not statically import provider implementation modules there
|
|
1374
|
-
- Add any root-level `export type` re-exports in `src/index.ts` and `src/base.ts` that should remain available from `@earendil-works/pi-ai` and `@earendil-works/pi-ai/base`
|
|
1375
|
-
- Keep `src/base.ts` free of built-in registration imports
|
|
1376
|
-
- Add credential detection in `env-api-keys.ts` for the new provider
|
|
1377
|
-
- Ensure `streamSimple` handles auth lookup via `getEnvApiKey()` or provider-specific auth
|
|
1378
|
-
|
|
1379
|
-
#### 4. Model Generation (`scripts/generate-models.ts`, `scripts/generate-image-models.ts`)
|
|
1483
|
+
#### 3. Model Generation (`scripts/generate-models.ts`, `scripts/generate-image-models.ts`)
|
|
1380
1484
|
|
|
1381
1485
|
- Add logic to fetch and parse models from the provider's source (e.g., models.dev API)
|
|
1382
|
-
- Map chat/tool-capable provider model data to the standardized `Model` interface via `scripts/generate-models.ts`
|
|
1486
|
+
- Map chat/tool-capable provider model data to the standardized `Model` interface via `scripts/generate-models.ts`; regeneration emits `src/providers/<id>.models.ts` and the aggregator
|
|
1383
1487
|
- Map image-generation provider model data to the standardized `ImagesModel` interface via `scripts/generate-image-models.ts`
|
|
1384
1488
|
- Handle provider-specific quirks (pricing format, capability flags, model ID transformations)
|
|
1385
1489
|
|
|
1490
|
+
#### 4. Provider Factory (`src/providers/<id>.ts`)
|
|
1491
|
+
|
|
1492
|
+
- `createProvider()` wiring catalog + auth + the lazy API wrapper
|
|
1493
|
+
- Auth: `envApiKeyAuth` for standard key providers, a custom `ApiKeyAuth` for ambient auth (AWS profiles, ADC), `lazyOAuth` where an OAuth flow exists
|
|
1494
|
+
- Register the factory in `src/providers/all.ts`
|
|
1495
|
+
- If it is a new API: register it in the builtin list in `src/compat.ts` and add the package subpath export in `package.json`
|
|
1496
|
+
|
|
1386
1497
|
#### 5. Tests (`test/`)
|
|
1387
1498
|
|
|
1388
1499
|
Create or update test files to cover the new provider:
|
|
@@ -1398,6 +1509,7 @@ Create or update test files to cover the new provider:
|
|
|
1398
1509
|
- `image-tool-result.test.ts` - Images in tool results
|
|
1399
1510
|
- `total-tokens.test.ts` - Token counting accuracy
|
|
1400
1511
|
- `cross-provider-handoff.test.ts` - Cross-provider context replay
|
|
1512
|
+
- `providers.test.ts` - Provider listing and auth resolution
|
|
1401
1513
|
|
|
1402
1514
|
For `cross-provider-handoff.test.ts`, add at least one provider/model pair. If the provider exposes multiple model families (for example GPT and Claude), add at least one pair per family.
|
|
1403
1515
|
|