@aryee337/aery-ai 0.1.148 → 0.2.10
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +2914 -0
- package/README.md +614 -813
- package/dist/types/api-registry.d.ts +30 -0
- package/dist/types/auth-broker/client.d.ts +66 -0
- package/dist/types/auth-broker/index.d.ts +5 -0
- package/dist/types/auth-broker/refresher.d.ts +25 -0
- package/dist/types/auth-broker/remote-store.d.ts +96 -0
- package/dist/types/auth-broker/server.d.ts +32 -0
- package/dist/types/auth-broker/types.d.ts +105 -0
- package/dist/types/auth-broker/wire-schemas.d.ts +412 -0
- package/dist/types/auth-gateway/http.d.ts +39 -0
- package/dist/types/auth-gateway/index.d.ts +3 -0
- package/dist/types/auth-gateway/server.d.ts +36 -0
- package/dist/types/auth-gateway/types.d.ts +117 -0
- package/dist/types/auth-storage.d.ts +739 -0
- package/dist/types/index.d.ts +49 -0
- package/dist/types/model-cache.d.ts +17 -0
- package/dist/types/model-manager.d.ts +64 -0
- package/dist/types/model-thinking.d.ts +100 -0
- package/dist/types/models.d.ts +12 -0
- package/dist/types/provider-details.d.ts +24 -0
- package/dist/types/provider-models/bundled-references.d.ts +4 -0
- package/dist/types/provider-models/descriptors.d.ts +50 -0
- package/dist/types/provider-models/google.d.ts +24 -0
- package/dist/types/provider-models/index.d.ts +5 -0
- package/dist/types/provider-models/ollama.d.ts +7 -0
- package/dist/types/provider-models/openai-compat.d.ts +296 -0
- package/dist/types/provider-models/special.d.ts +16 -0
- package/dist/types/providers/aery-native-client.d.ts +13 -0
- package/dist/types/providers/aery-native-server.d.ts +68 -0
- package/dist/types/providers/amazon-bedrock.d.ts +38 -0
- package/dist/types/providers/anthropic-client.d.ts +99 -0
- package/dist/types/providers/anthropic-messages-server-schema.d.ts +465 -0
- package/dist/types/providers/anthropic-messages-server.d.ts +17 -0
- package/dist/types/providers/anthropic-wire.d.ts +262 -0
- package/dist/types/providers/anthropic.d.ts +206 -0
- package/dist/types/providers/aws-credentials.d.ts +43 -0
- package/dist/types/providers/aws-eventstream.d.ts +38 -0
- package/dist/types/providers/aws-sigv4.d.ts +55 -0
- package/dist/types/providers/azure-openai-responses.d.ts +15 -0
- package/dist/types/providers/cursor/gen/agent_pb.d.ts +13022 -0
- package/dist/types/providers/cursor.d.ts +43 -0
- package/dist/types/providers/error-message.d.ts +27 -0
- package/dist/types/providers/github-copilot-headers.d.ts +40 -0
- package/dist/types/providers/gitlab-duo.d.ts +27 -0
- package/dist/types/providers/google-auth.d.ts +24 -0
- package/dist/types/providers/google-gemini-cli.d.ts +81 -0
- package/dist/types/providers/google-gemini-headers.d.ts +18 -0
- package/dist/types/providers/google-shared.d.ts +171 -0
- package/dist/types/providers/google-types.d.ts +138 -0
- package/dist/types/providers/google-vertex.d.ts +7 -0
- package/dist/types/providers/google.d.ts +4 -0
- package/dist/types/providers/grammar.d.ts +1 -0
- package/dist/types/providers/kimi.d.ts +27 -0
- package/dist/types/providers/mock.d.ts +173 -0
- package/dist/types/providers/ollama.d.ts +6 -0
- package/dist/types/providers/openai-anthropic-shim.d.ts +31 -0
- package/dist/types/providers/openai-chat-server-schema.d.ts +817 -0
- package/dist/types/providers/openai-chat-server.d.ts +16 -0
- package/dist/types/providers/openai-codex/constants.d.ts +26 -0
- package/dist/types/providers/openai-codex/request-transformer.d.ts +49 -0
- package/dist/types/providers/openai-codex/response-handler.d.ts +17 -0
- package/dist/types/providers/openai-codex-responses.d.ts +67 -0
- package/dist/types/providers/openai-completions-compat.d.ts +25 -0
- package/dist/types/providers/openai-completions.d.ts +54 -0
- package/dist/types/providers/openai-responses-server-schema.d.ts +392 -0
- package/dist/types/providers/openai-responses-server.d.ts +17 -0
- package/dist/types/providers/openai-responses-shared.d.ts +100 -0
- package/dist/types/providers/openai-responses.d.ts +66 -0
- package/dist/types/providers/register-builtins.d.ts +31 -0
- package/dist/types/providers/synthetic.d.ts +26 -0
- package/dist/{providers → types/providers}/transform-messages.d.ts +6 -2
- package/dist/types/providers/vision-guard.d.ts +8 -0
- package/dist/types/providers/xai-responses.d.ts +23 -0
- package/dist/types/rate-limit-utils.d.ts +19 -0
- package/dist/types/stream.d.ts +28 -0
- package/dist/types/types.d.ts +801 -0
- package/dist/types/usage/claude.d.ts +4 -0
- package/dist/types/usage/gemini.d.ts +2 -0
- package/dist/types/usage/github-copilot.d.ts +7 -0
- package/dist/types/usage/google-antigravity.d.ts +2 -0
- package/dist/types/usage/kimi.d.ts +2 -0
- package/dist/types/usage/minimax-code.d.ts +2 -0
- package/dist/types/usage/openai-codex.d.ts +3 -0
- package/dist/types/usage/shared.d.ts +1 -0
- package/dist/types/usage/zai.d.ts +2 -0
- package/dist/types/usage.d.ts +260 -0
- package/dist/types/utils/abort.d.ts +19 -0
- package/dist/types/utils/abortable-iterator.d.ts +4 -0
- package/dist/types/utils/anthropic-auth.d.ts +35 -0
- package/dist/types/utils/discovery/antigravity.d.ts +61 -0
- package/dist/types/utils/discovery/codex.d.ts +38 -0
- package/dist/types/utils/discovery/cursor.d.ts +23 -0
- package/dist/types/utils/discovery/gemini.d.ts +25 -0
- package/dist/types/utils/discovery/index.d.ts +4 -0
- package/dist/types/utils/discovery/openai-compatible.d.ts +72 -0
- package/dist/types/utils/event-stream.d.ts +28 -0
- package/dist/types/utils/fireworks-model-id.d.ts +10 -0
- package/dist/types/utils/foundry.d.ts +1 -0
- package/dist/types/utils/http-inspector.d.ts +31 -0
- package/dist/types/utils/idle-iterator.d.ts +78 -0
- package/dist/types/utils/json-parse.d.ts +37 -0
- package/dist/types/utils/oauth/__tests__/xai-oauth.test.d.ts +1 -0
- package/dist/types/utils/oauth/alibaba-coding-plan.d.ts +18 -0
- package/dist/types/utils/oauth/anthropic.d.ts +22 -0
- package/dist/types/utils/oauth/api-key-login.d.ts +35 -0
- package/dist/types/utils/oauth/api-key-validation.d.ts +27 -0
- package/dist/types/utils/oauth/callback-server.d.ts +57 -0
- package/dist/types/utils/oauth/cerebras.d.ts +1 -0
- package/dist/types/utils/oauth/cloudflare-ai-gateway.d.ts +18 -0
- package/dist/types/utils/oauth/cursor.d.ts +15 -0
- package/dist/types/utils/oauth/deepseek.d.ts +10 -0
- package/dist/types/utils/oauth/firepass.d.ts +1 -0
- package/dist/types/utils/oauth/fireworks.d.ts +1 -0
- package/dist/types/utils/oauth/github-copilot.d.ts +38 -0
- package/dist/types/utils/oauth/gitlab-duo.d.ts +3 -0
- package/dist/types/utils/oauth/google-antigravity.d.ts +11 -0
- package/dist/types/utils/oauth/google-gemini-cli.d.ts +10 -0
- package/dist/types/utils/oauth/google-oauth-shared.d.ts +28 -0
- package/dist/types/utils/oauth/huggingface.d.ts +19 -0
- package/dist/types/utils/oauth/index.d.ts +38 -0
- package/dist/types/utils/oauth/kagi.d.ts +17 -0
- package/dist/types/utils/oauth/kilo.d.ts +5 -0
- package/dist/types/utils/oauth/kimi.d.ts +21 -0
- package/dist/types/utils/oauth/litellm.d.ts +18 -0
- package/dist/types/utils/oauth/lm-studio.d.ts +17 -0
- package/dist/types/utils/oauth/minimax-code.d.ts +28 -0
- package/dist/types/utils/oauth/moonshot.d.ts +1 -0
- package/dist/types/utils/oauth/nanogpt.d.ts +1 -0
- package/dist/types/utils/oauth/nvidia.d.ts +18 -0
- package/dist/types/utils/oauth/ollama-cloud.d.ts +2 -0
- package/dist/types/utils/oauth/ollama.d.ts +18 -0
- package/dist/types/utils/oauth/openai-codex.d.ts +21 -0
- package/dist/types/utils/oauth/opencode.d.ts +18 -0
- package/dist/types/utils/oauth/openrouter.d.ts +1 -0
- package/dist/types/utils/oauth/parallel.d.ts +17 -0
- package/dist/types/utils/oauth/perplexity.d.ts +9 -0
- package/dist/{utils → types/utils}/oauth/pkce.d.ts +0 -5
- package/dist/types/utils/oauth/qianfan.d.ts +17 -0
- package/dist/types/utils/oauth/qwen-portal.d.ts +19 -0
- package/dist/types/utils/oauth/synthetic.d.ts +1 -0
- package/dist/types/utils/oauth/tavily.d.ts +17 -0
- package/dist/types/utils/oauth/together.d.ts +1 -0
- package/dist/types/utils/oauth/types.d.ts +44 -0
- package/dist/types/utils/oauth/venice.d.ts +18 -0
- package/dist/types/utils/oauth/vercel-ai-gateway.d.ts +18 -0
- package/dist/types/utils/oauth/vllm.d.ts +16 -0
- package/dist/types/utils/oauth/wafer.d.ts +2 -0
- package/dist/types/utils/oauth/xai-oauth.d.ts +60 -0
- package/dist/types/utils/oauth/xiaomi.d.ts +19 -0
- package/dist/types/utils/oauth/zai.d.ts +18 -0
- package/dist/types/utils/oauth/zenmux.d.ts +1 -0
- package/dist/types/utils/oauth/zhipu.d.ts +18 -0
- package/dist/{utils → types/utils}/overflow.d.ts +9 -11
- package/dist/types/utils/parse-bind.d.ts +23 -0
- package/dist/types/utils/provider-response.d.ts +3 -0
- package/dist/types/utils/request-debug.d.ts +29 -0
- package/dist/types/utils/retry-after.d.ts +3 -0
- package/dist/types/utils/retry.d.ts +26 -0
- package/dist/types/utils/schema/adapt.d.ts +24 -0
- package/dist/types/utils/schema/compatibility.d.ts +30 -0
- package/dist/types/utils/schema/dereference.d.ts +11 -0
- package/dist/types/utils/schema/draft.d.ts +10 -0
- package/dist/types/utils/schema/equality.d.ts +4 -0
- package/dist/types/utils/schema/fields.d.ts +49 -0
- package/dist/types/utils/schema/index.d.ts +13 -0
- package/dist/types/utils/schema/json-schema-validator.d.ts +12 -0
- package/dist/types/utils/schema/meta-validator.d.ts +2 -0
- package/dist/types/utils/schema/normalize.d.ts +93 -0
- package/dist/types/utils/schema/spill.d.ts +8 -0
- package/dist/types/utils/schema/stamps.d.ts +25 -0
- package/dist/types/utils/schema/types.d.ts +4 -0
- package/dist/types/utils/schema/wire.d.ts +53 -0
- package/dist/types/utils/schema/zod-decontaminate.d.ts +31 -0
- package/dist/types/utils/sdk-stream-timeout.d.ts +33 -0
- package/dist/types/utils/sse-debug.d.ts +10 -0
- package/dist/types/utils/stream-markup-healing.d.ts +80 -0
- package/dist/types/utils/tool-choice.d.ts +50 -0
- package/dist/types/utils/validation.d.ts +17 -0
- package/dist/types/utils.d.ts +28 -0
- package/package.json +139 -105
- package/src/api-registry.ts +96 -0
- package/src/auth-broker/client.ts +358 -0
- package/src/auth-broker/index.ts +5 -0
- package/src/auth-broker/refresher.ts +117 -0
- package/src/auth-broker/remote-store.ts +623 -0
- package/src/auth-broker/server.ts +644 -0
- package/src/auth-broker/types.ts +127 -0
- package/src/auth-broker/wire-schemas.ts +200 -0
- package/src/auth-gateway/http.ts +194 -0
- package/src/auth-gateway/index.ts +3 -0
- package/src/auth-gateway/server.ts +818 -0
- package/src/auth-gateway/types.ts +143 -0
- package/src/auth-storage.ts +4422 -0
- package/src/index.ts +54 -0
- package/src/model-cache.ts +129 -0
- package/src/model-manager.ts +469 -0
- package/src/model-thinking.ts +782 -0
- package/src/models.json +83530 -0
- package/src/models.json.d.ts +9 -0
- package/src/models.ts +56 -0
- package/src/prompts/turn-aborted-guidance.md +4 -0
- package/src/provider-details.ts +90 -0
- package/src/provider-models/bundled-references.ts +38 -0
- package/src/provider-models/descriptors.ts +355 -0
- package/src/provider-models/google.ts +88 -0
- package/src/provider-models/index.ts +5 -0
- package/src/provider-models/ollama.ts +153 -0
- package/src/provider-models/openai-compat.ts +2817 -0
- package/src/provider-models/special.ts +67 -0
- package/src/providers/aery-native-client.ts +228 -0
- package/src/providers/aery-native-server.ts +212 -0
- package/src/providers/amazon-bedrock.ts +873 -0
- package/src/providers/anthropic-client.ts +318 -0
- package/src/providers/anthropic-messages-server-schema.ts +243 -0
- package/src/providers/anthropic-messages-server.ts +683 -0
- package/src/providers/anthropic-wire.ts +268 -0
- package/src/providers/anthropic.ts +3094 -0
- package/src/providers/aws-credentials.ts +501 -0
- package/src/providers/aws-eventstream.ts +185 -0
- package/src/providers/aws-sigv4.ts +218 -0
- package/src/providers/azure-openai-responses.ts +361 -0
- package/src/providers/cursor/gen/agent_pb.ts +15274 -0
- package/src/providers/cursor/proto/agent.proto +3526 -0
- package/src/providers/cursor/proto/buf.gen.yaml +6 -0
- package/src/providers/cursor/proto/buf.yaml +17 -0
- package/src/providers/cursor.ts +2621 -0
- package/src/providers/error-message.ts +21 -0
- package/src/providers/github-copilot-headers.ts +140 -0
- package/src/providers/gitlab-duo.ts +372 -0
- package/src/providers/google-auth.ts +252 -0
- package/src/providers/google-gemini-cli.ts +809 -0
- package/src/providers/google-gemini-headers.ts +41 -0
- package/src/providers/google-shared.ts +917 -0
- package/src/providers/google-types.ts +167 -0
- package/src/providers/google-vertex.ts +91 -0
- package/src/providers/google.ts +41 -0
- package/src/providers/grammar.ts +70 -0
- package/src/providers/kimi.ts +52 -0
- package/src/providers/mock.ts +496 -0
- package/src/providers/ollama.ts +644 -0
- package/src/providers/openai-anthropic-shim.ts +138 -0
- package/src/providers/openai-chat-server-schema.ts +252 -0
- package/src/providers/openai-chat-server.ts +647 -0
- package/src/providers/openai-codex/constants.ts +43 -0
- package/src/providers/openai-codex/request-transformer.ts +161 -0
- package/src/providers/openai-codex/response-handler.ts +81 -0
- package/src/providers/openai-codex-responses.ts +3018 -0
- package/src/providers/openai-completions-compat.ts +300 -0
- package/src/providers/openai-completions.ts +1979 -0
- package/src/providers/openai-responses-server-schema.ts +290 -0
- package/src/providers/openai-responses-server.ts +1183 -0
- package/src/providers/openai-responses-shared.ts +873 -0
- package/src/providers/openai-responses.ts +679 -0
- package/src/providers/register-builtins.ts +436 -0
- package/src/providers/synthetic.ts +50 -0
- package/src/providers/transform-messages.ts +382 -0
- package/src/providers/vision-guard.ts +31 -0
- package/src/providers/xai-responses.ts +82 -0
- package/src/rate-limit-utils.ts +84 -0
- package/src/stream.ts +1065 -0
- package/src/types.ts +944 -0
- package/src/usage/claude.ts +482 -0
- package/src/usage/gemini.ts +250 -0
- package/src/usage/github-copilot.ts +421 -0
- package/src/usage/google-antigravity.ts +201 -0
- package/src/usage/kimi.ts +271 -0
- package/src/usage/minimax-code.ts +31 -0
- package/src/usage/openai-codex.ts +503 -0
- package/src/usage/shared.ts +10 -0
- package/src/usage/zai.ts +247 -0
- package/src/usage.ts +185 -0
- package/src/utils/abort.ts +51 -0
- package/src/utils/abortable-iterator.ts +69 -0
- package/src/utils/anthropic-auth.ts +93 -0
- package/src/utils/discovery/antigravity.ts +261 -0
- package/src/utils/discovery/codex.ts +371 -0
- package/src/utils/discovery/cursor.ts +306 -0
- package/src/utils/discovery/gemini.ts +248 -0
- package/src/utils/discovery/index.ts +4 -0
- package/src/utils/discovery/openai-compatible.ts +224 -0
- package/src/utils/event-stream.ts +142 -0
- package/src/utils/fireworks-model-id.ts +30 -0
- package/src/utils/foundry.ts +8 -0
- package/src/utils/http-inspector.ts +176 -0
- package/src/utils/idle-iterator.ts +267 -0
- package/src/utils/json-parse.ts +182 -0
- package/src/utils/oauth/__tests__/xai-oauth.test.ts +107 -0
- package/src/utils/oauth/alibaba-coding-plan.ts +59 -0
- package/src/utils/oauth/anthropic.ts +273 -0
- package/src/utils/oauth/api-key-login.ts +87 -0
- package/src/utils/oauth/api-key-validation.ts +92 -0
- package/src/utils/oauth/callback-server.ts +276 -0
- package/src/utils/oauth/cerebras.ts +16 -0
- package/src/utils/oauth/cloudflare-ai-gateway.ts +48 -0
- package/src/utils/oauth/cursor.ts +157 -0
- package/src/utils/oauth/deepseek.ts +53 -0
- package/src/utils/oauth/firepass.ts +24 -0
- package/src/utils/oauth/fireworks.ts +15 -0
- package/src/utils/oauth/github-copilot.ts +362 -0
- package/src/utils/oauth/gitlab-duo.ts +123 -0
- package/src/utils/oauth/google-antigravity.ts +200 -0
- package/src/utils/oauth/google-gemini-cli.ts +256 -0
- package/src/utils/oauth/google-oauth-shared.ts +110 -0
- package/src/utils/oauth/huggingface.ts +62 -0
- package/src/utils/oauth/index.ts +484 -0
- package/src/utils/oauth/kagi.ts +47 -0
- package/src/utils/oauth/kilo.ts +87 -0
- package/src/utils/oauth/kimi.ts +254 -0
- package/src/utils/oauth/litellm.ts +47 -0
- package/src/utils/oauth/lm-studio.ts +38 -0
- package/src/utils/oauth/minimax-code.ts +78 -0
- package/src/utils/oauth/moonshot.ts +23 -0
- package/src/utils/oauth/nanogpt.ts +15 -0
- package/src/utils/oauth/nvidia.ts +70 -0
- package/src/utils/oauth/oauth.html +203 -0
- package/src/utils/oauth/ollama-cloud.ts +28 -0
- package/src/utils/oauth/ollama.ts +47 -0
- package/src/utils/oauth/openai-codex.ts +299 -0
- package/src/utils/oauth/opencode.ts +49 -0
- package/src/utils/oauth/openrouter.ts +20 -0
- package/src/utils/oauth/parallel.ts +46 -0
- package/src/utils/oauth/perplexity.ts +206 -0
- package/src/utils/oauth/pkce.ts +18 -0
- package/src/utils/oauth/qianfan.ts +58 -0
- package/src/utils/oauth/qwen-portal.ts +60 -0
- package/src/utils/oauth/synthetic.ts +15 -0
- package/src/utils/oauth/tavily.ts +46 -0
- package/src/utils/oauth/together.ts +16 -0
- package/src/utils/oauth/types.ts +99 -0
- package/src/utils/oauth/venice.ts +59 -0
- package/src/utils/oauth/vercel-ai-gateway.ts +47 -0
- package/src/utils/oauth/vllm.ts +40 -0
- package/src/utils/oauth/wafer.ts +50 -0
- package/src/utils/oauth/xai-oauth.ts +342 -0
- package/src/utils/oauth/xiaomi.ts +139 -0
- package/src/utils/oauth/zai.ts +60 -0
- package/src/utils/oauth/zenmux.ts +15 -0
- package/src/utils/oauth/zhipu.ts +60 -0
- package/src/utils/overflow.ts +137 -0
- package/src/utils/parse-bind.ts +54 -0
- package/src/utils/provider-response.ts +30 -0
- package/src/utils/request-debug.ts +336 -0
- package/src/utils/retry-after.ts +110 -0
- package/src/utils/retry.ts +54 -0
- package/src/utils/schema/CONSTRAINTS.md +164 -0
- package/src/utils/schema/adapt.ts +36 -0
- package/src/utils/schema/compatibility.ts +435 -0
- package/src/utils/schema/dereference.ts +98 -0
- package/src/utils/schema/draft.ts +341 -0
- package/src/utils/schema/equality.ts +97 -0
- package/src/utils/schema/fields.ts +191 -0
- package/src/utils/schema/index.ts +13 -0
- package/src/utils/schema/json-schema-validator.ts +577 -0
- package/src/utils/schema/meta-validator.ts +167 -0
- package/src/utils/schema/normalize.ts +1588 -0
- package/src/utils/schema/spill.ts +43 -0
- package/src/utils/schema/stamps.ts +97 -0
- package/src/utils/schema/types.ts +10 -0
- package/src/utils/schema/wire.ts +293 -0
- package/src/utils/schema/zod-decontaminate.ts +331 -0
- package/src/utils/sdk-stream-timeout.ts +43 -0
- package/src/utils/sse-debug.ts +289 -0
- package/src/utils/stream-markup-healing.ts +612 -0
- package/src/utils/tool-choice.ts +99 -0
- package/src/utils/validation.ts +1024 -0
- package/src/utils.ts +166 -0
- package/dist/api-registry.d.ts +0 -20
- package/dist/api-registry.d.ts.map +0 -1
- package/dist/api-registry.js +0 -44
- package/dist/api-registry.js.map +0 -1
- package/dist/bedrock-provider.d.ts +0 -5
- package/dist/bedrock-provider.d.ts.map +0 -1
- package/dist/bedrock-provider.js +0 -6
- package/dist/bedrock-provider.js.map +0 -1
- package/dist/cli.d.ts +0 -3
- package/dist/cli.d.ts.map +0 -1
- package/dist/cli.js +0 -130
- package/dist/cli.js.map +0 -1
- package/dist/env-api-keys.d.ts +0 -18
- package/dist/env-api-keys.d.ts.map +0 -1
- package/dist/env-api-keys.js +0 -178
- package/dist/env-api-keys.js.map +0 -1
- package/dist/image-models.d.ts +0 -10
- package/dist/image-models.d.ts.map +0 -1
- package/dist/image-models.generated.d.ts +0 -440
- package/dist/image-models.generated.d.ts.map +0 -1
- package/dist/image-models.generated.js +0 -442
- package/dist/image-models.generated.js.map +0 -1
- package/dist/image-models.js +0 -23
- package/dist/image-models.js.map +0 -1
- package/dist/images-api-registry.d.ts +0 -14
- package/dist/images-api-registry.d.ts.map +0 -1
- package/dist/images-api-registry.js +0 -22
- package/dist/images-api-registry.js.map +0 -1
- package/dist/images.d.ts +0 -4
- package/dist/images.d.ts.map +0 -1
- package/dist/images.js +0 -14
- package/dist/images.js.map +0 -1
- package/dist/index.d.ts +0 -32
- package/dist/index.d.ts.map +0 -1
- package/dist/index.js +0 -20
- package/dist/index.js.map +0 -1
- package/dist/models.d.ts +0 -18
- package/dist/models.d.ts.map +0 -1
- package/dist/models.generated.d.ts +0 -17480
- package/dist/models.generated.d.ts.map +0 -1
- package/dist/models.generated.js +0 -16339
- package/dist/models.generated.js.map +0 -1
- package/dist/models.js +0 -71
- package/dist/models.js.map +0 -1
- package/dist/oauth.d.ts +0 -2
- package/dist/oauth.d.ts.map +0 -1
- package/dist/oauth.js +0 -2
- package/dist/oauth.js.map +0 -1
- package/dist/providers/aery-error-formatting.d.ts +0 -13
- package/dist/providers/aery-error-formatting.d.ts.map +0 -1
- package/dist/providers/aery-error-formatting.js +0 -112
- package/dist/providers/aery-error-formatting.js.map +0 -1
- package/dist/providers/amazon-bedrock.d.ts +0 -38
- package/dist/providers/amazon-bedrock.d.ts.map +0 -1
- package/dist/providers/amazon-bedrock.js +0 -763
- package/dist/providers/amazon-bedrock.js.map +0 -1
- package/dist/providers/anthropic.d.ts +0 -71
- package/dist/providers/anthropic.d.ts.map +0 -1
- package/dist/providers/anthropic.js +0 -949
- package/dist/providers/anthropic.js.map +0 -1
- package/dist/providers/azure-openai-responses.d.ts +0 -15
- package/dist/providers/azure-openai-responses.d.ts.map +0 -1
- package/dist/providers/azure-openai-responses.js +0 -225
- package/dist/providers/azure-openai-responses.js.map +0 -1
- package/dist/providers/cloudflare.d.ts +0 -13
- package/dist/providers/cloudflare.d.ts.map +0 -1
- package/dist/providers/cloudflare.js +0 -26
- package/dist/providers/cloudflare.js.map +0 -1
- package/dist/providers/faux.d.ts +0 -56
- package/dist/providers/faux.d.ts.map +0 -1
- package/dist/providers/faux.js +0 -368
- package/dist/providers/faux.js.map +0 -1
- package/dist/providers/github-copilot-headers.d.ts +0 -8
- package/dist/providers/github-copilot-headers.d.ts.map +0 -1
- package/dist/providers/github-copilot-headers.js +0 -29
- package/dist/providers/github-copilot-headers.js.map +0 -1
- package/dist/providers/google-gemini-cli.d.ts +0 -74
- package/dist/providers/google-gemini-cli.d.ts.map +0 -1
- package/dist/providers/google-gemini-cli.js +0 -779
- package/dist/providers/google-gemini-cli.js.map +0 -1
- package/dist/providers/google-shared.d.ts +0 -70
- package/dist/providers/google-shared.d.ts.map +0 -1
- package/dist/providers/google-shared.js +0 -329
- package/dist/providers/google-shared.js.map +0 -1
- package/dist/providers/google-vertex.d.ts +0 -15
- package/dist/providers/google-vertex.d.ts.map +0 -1
- package/dist/providers/google-vertex.js +0 -442
- package/dist/providers/google-vertex.js.map +0 -1
- package/dist/providers/google.d.ts +0 -13
- package/dist/providers/google.d.ts.map +0 -1
- package/dist/providers/google.js +0 -400
- package/dist/providers/google.js.map +0 -1
- package/dist/providers/images/openrouter.d.ts +0 -3
- package/dist/providers/images/openrouter.d.ts.map +0 -1
- package/dist/providers/images/openrouter.js +0 -129
- package/dist/providers/images/openrouter.js.map +0 -1
- package/dist/providers/images/register-builtins.d.ts +0 -4
- package/dist/providers/images/register-builtins.d.ts.map +0 -1
- package/dist/providers/images/register-builtins.js +0 -34
- package/dist/providers/images/register-builtins.js.map +0 -1
- package/dist/providers/mistral.d.ts +0 -25
- package/dist/providers/mistral.d.ts.map +0 -1
- package/dist/providers/mistral.js +0 -535
- package/dist/providers/mistral.js.map +0 -1
- package/dist/providers/openai-codex-responses.d.ts +0 -30
- package/dist/providers/openai-codex-responses.d.ts.map +0 -1
- package/dist/providers/openai-codex-responses.js +0 -1090
- package/dist/providers/openai-codex-responses.js.map +0 -1
- package/dist/providers/openai-completions.d.ts +0 -19
- package/dist/providers/openai-completions.d.ts.map +0 -1
- package/dist/providers/openai-completions.js +0 -950
- package/dist/providers/openai-completions.js.map +0 -1
- package/dist/providers/openai-prompt-cache.d.ts +0 -3
- package/dist/providers/openai-prompt-cache.d.ts.map +0 -1
- package/dist/providers/openai-prompt-cache.js +0 -10
- package/dist/providers/openai-prompt-cache.js.map +0 -1
- package/dist/providers/openai-responses-shared.d.ts +0 -18
- package/dist/providers/openai-responses-shared.d.ts.map +0 -1
- package/dist/providers/openai-responses-shared.js +0 -492
- package/dist/providers/openai-responses-shared.js.map +0 -1
- package/dist/providers/openai-responses.d.ts +0 -13
- package/dist/providers/openai-responses.d.ts.map +0 -1
- package/dist/providers/openai-responses.js +0 -237
- package/dist/providers/openai-responses.js.map +0 -1
- package/dist/providers/register-builtins.d.ts +0 -38
- package/dist/providers/register-builtins.d.ts.map +0 -1
- package/dist/providers/register-builtins.js +0 -278
- package/dist/providers/register-builtins.js.map +0 -1
- package/dist/providers/simple-options.d.ts +0 -8
- package/dist/providers/simple-options.d.ts.map +0 -1
- package/dist/providers/simple-options.js +0 -41
- package/dist/providers/simple-options.js.map +0 -1
- package/dist/providers/transform-messages.d.ts.map +0 -1
- package/dist/providers/transform-messages.js +0 -184
- package/dist/providers/transform-messages.js.map +0 -1
- package/dist/session-resources.d.ts +0 -4
- package/dist/session-resources.d.ts.map +0 -1
- package/dist/session-resources.js +0 -22
- package/dist/session-resources.js.map +0 -1
- package/dist/stream.d.ts +0 -8
- package/dist/stream.d.ts.map +0 -1
- package/dist/stream.js +0 -27
- package/dist/stream.js.map +0 -1
- package/dist/types.d.ts +0 -498
- package/dist/types.d.ts.map +0 -1
- package/dist/types.js +0 -2
- package/dist/types.js.map +0 -1
- package/dist/utils/diagnostics.d.ts +0 -19
- package/dist/utils/diagnostics.d.ts.map +0 -1
- package/dist/utils/diagnostics.js +0 -25
- package/dist/utils/diagnostics.js.map +0 -1
- package/dist/utils/event-stream.d.ts +0 -21
- package/dist/utils/event-stream.d.ts.map +0 -1
- package/dist/utils/event-stream.js +0 -81
- package/dist/utils/event-stream.js.map +0 -1
- package/dist/utils/hash.d.ts +0 -3
- package/dist/utils/hash.d.ts.map +0 -1
- package/dist/utils/hash.js +0 -14
- package/dist/utils/hash.js.map +0 -1
- package/dist/utils/headers.d.ts +0 -2
- package/dist/utils/headers.d.ts.map +0 -1
- package/dist/utils/headers.js +0 -8
- package/dist/utils/headers.js.map +0 -1
- package/dist/utils/json-parse.d.ts +0 -16
- package/dist/utils/json-parse.d.ts.map +0 -1
- package/dist/utils/json-parse.js +0 -113
- package/dist/utils/json-parse.js.map +0 -1
- package/dist/utils/node-http-proxy.d.ts +0 -10
- package/dist/utils/node-http-proxy.d.ts.map +0 -1
- package/dist/utils/node-http-proxy.js +0 -97
- package/dist/utils/node-http-proxy.js.map +0 -1
- package/dist/utils/oauth/anthropic.d.ts +0 -25
- package/dist/utils/oauth/anthropic.d.ts.map +0 -1
- package/dist/utils/oauth/anthropic.js +0 -335
- package/dist/utils/oauth/anthropic.js.map +0 -1
- package/dist/utils/oauth/device-code.d.ts +0 -19
- package/dist/utils/oauth/device-code.d.ts.map +0 -1
- package/dist/utils/oauth/device-code.js +0 -55
- package/dist/utils/oauth/device-code.js.map +0 -1
- package/dist/utils/oauth/github-copilot.d.ts +0 -30
- package/dist/utils/oauth/github-copilot.d.ts.map +0 -1
- package/dist/utils/oauth/github-copilot.js +0 -268
- package/dist/utils/oauth/github-copilot.js.map +0 -1
- package/dist/utils/oauth/google-antigravity.d.ts +0 -26
- package/dist/utils/oauth/google-antigravity.d.ts.map +0 -1
- package/dist/utils/oauth/google-antigravity.js +0 -377
- package/dist/utils/oauth/google-antigravity.js.map +0 -1
- package/dist/utils/oauth/google-gemini-cli.d.ts +0 -26
- package/dist/utils/oauth/google-gemini-cli.d.ts.map +0 -1
- package/dist/utils/oauth/google-gemini-cli.js +0 -482
- package/dist/utils/oauth/google-gemini-cli.js.map +0 -1
- package/dist/utils/oauth/index.d.ts +0 -63
- package/dist/utils/oauth/index.d.ts.map +0 -1
- package/dist/utils/oauth/index.js +0 -131
- package/dist/utils/oauth/index.js.map +0 -1
- package/dist/utils/oauth/oauth-page.d.ts +0 -3
- package/dist/utils/oauth/oauth-page.d.ts.map +0 -1
- package/dist/utils/oauth/oauth-page.js +0 -105
- package/dist/utils/oauth/oauth-page.js.map +0 -1
- package/dist/utils/oauth/openai-codex.d.ts +0 -34
- package/dist/utils/oauth/openai-codex.d.ts.map +0 -1
- package/dist/utils/oauth/openai-codex.js +0 -385
- package/dist/utils/oauth/openai-codex.js.map +0 -1
- package/dist/utils/oauth/pkce.d.ts.map +0 -1
- package/dist/utils/oauth/pkce.js +0 -31
- package/dist/utils/oauth/pkce.js.map +0 -1
- package/dist/utils/oauth/types.d.ts +0 -64
- package/dist/utils/oauth/types.d.ts.map +0 -1
- package/dist/utils/oauth/types.js +0 -2
- package/dist/utils/oauth/types.js.map +0 -1
- package/dist/utils/overflow.d.ts.map +0 -1
- package/dist/utils/overflow.js +0 -151
- package/dist/utils/overflow.js.map +0 -1
- package/dist/utils/sanitize-unicode.d.ts +0 -22
- package/dist/utils/sanitize-unicode.d.ts.map +0 -1
- package/dist/utils/sanitize-unicode.js +0 -26
- package/dist/utils/sanitize-unicode.js.map +0 -1
- package/dist/utils/typebox-helpers.d.ts +0 -17
- package/dist/utils/typebox-helpers.d.ts.map +0 -1
- package/dist/utils/typebox-helpers.js +0 -21
- package/dist/utils/typebox-helpers.js.map +0 -1
- package/dist/utils/validation.d.ts +0 -18
- package/dist/utils/validation.d.ts.map +0 -1
- package/dist/utils/validation.js +0 -281
- package/dist/utils/validation.js.map +0 -1
package/README.md
CHANGED
|
@@ -16,9 +16,6 @@ Unified LLM API with automatic model discovery, provider configuration, token an
|
|
|
16
16
|
- [Validating Tool Arguments](#validating-tool-arguments)
|
|
17
17
|
- [Complete Event Reference](#complete-event-reference)
|
|
18
18
|
- [Image Input](#image-input)
|
|
19
|
-
- [Image Generation](#image-generation)
|
|
20
|
-
- [Basic Image Generation](#basic-image-generation)
|
|
21
|
-
- [Notes and Limitations](#notes-and-limitations)
|
|
22
19
|
- [Thinking/Reasoning](#thinkingreasoning)
|
|
23
20
|
- [Unified Interface](#unified-interface-streamsimplecompletesimple)
|
|
24
21
|
- [Provider-Specific Options](#provider-specific-options-streamcomplete)
|
|
@@ -36,11 +33,10 @@ Unified LLM API with automatic model discovery, provider configuration, token an
|
|
|
36
33
|
- [Cross-Provider Handoffs](#cross-provider-handoffs)
|
|
37
34
|
- [Context Serialization](#context-serialization)
|
|
38
35
|
- [Browser Usage](#browser-usage)
|
|
39
|
-
- [Browser Compatibility Notes](#browser-compatibility-notes)
|
|
40
36
|
- [Environment Variables](#environment-variables-nodejs-only)
|
|
41
37
|
- [Checking Environment Variables](#checking-environment-variables)
|
|
42
38
|
- [OAuth Providers](#oauth-providers)
|
|
43
|
-
- [Vertex AI](#vertex-ai)
|
|
39
|
+
- [Vertex AI (ADC)](#vertex-ai-adc)
|
|
44
40
|
- [CLI Login](#cli-login)
|
|
45
41
|
- [Programmatic OAuth](#programmatic-oauth)
|
|
46
42
|
- [Login Flow Example](#login-flow-example)
|
|
@@ -51,30 +47,40 @@ Unified LLM API with automatic model discovery, provider configuration, token an
|
|
|
51
47
|
## Supported Providers
|
|
52
48
|
|
|
53
49
|
- **OpenAI**
|
|
54
|
-
- **Azure OpenAI (Responses)**
|
|
55
50
|
- **OpenAI Codex** (ChatGPT Plus/Pro subscription, requires OAuth, see below)
|
|
56
|
-
- **DeepSeek**
|
|
57
51
|
- **Anthropic**
|
|
58
52
|
- **Google**
|
|
59
53
|
- **Vertex AI** (Gemini via Vertex AI)
|
|
60
54
|
- **Mistral**
|
|
61
55
|
- **Groq**
|
|
62
56
|
- **Cerebras**
|
|
63
|
-
- **
|
|
64
|
-
- **
|
|
57
|
+
- **Together**
|
|
58
|
+
- **Moonshot** (requires `MOONSHOT_API_KEY`)
|
|
59
|
+
- **Qianfan** (requires `QIANFAN_API_KEY`)
|
|
60
|
+
- **NVIDIA** (requires `NVIDIA_API_KEY`)
|
|
61
|
+
- **NanoGPT** (requires `NANO_GPT_API_KEY`)
|
|
62
|
+
- **Hugging Face Inference**
|
|
65
63
|
- **xAI**
|
|
64
|
+
- **Venice** (requires `VENICE_API_KEY`)
|
|
65
|
+
- **Wafer Pass** (requires `WAFER_PASS_API_KEY`; flat-rate subscription, includes GLM-5.1 and Qwen3.5-397B-A17B)
|
|
66
|
+
- **Wafer Serverless** (requires `WAFER_SERVERLESS_API_KEY`; pay-as-you-go)
|
|
66
67
|
- **OpenRouter**
|
|
67
|
-
- **
|
|
68
|
-
- **
|
|
69
|
-
- **
|
|
68
|
+
- **Kilo Gateway** (supports OAuth `/login kilo` or `KILO_API_KEY`)
|
|
69
|
+
- **LiteLLM** (requires `LITELLM_API_KEY`)
|
|
70
|
+
- **zAI** (requires `ZAI_API_KEY`)
|
|
71
|
+
- **MiniMax Coding Plan** (requires `MINIMAX_CODE_API_KEY` or `MINIMAX_CODE_CN_API_KEY`)
|
|
72
|
+
- **Xiaomi MiMo** (requires `XIAOMI_API_KEY`)
|
|
73
|
+
- **ZenMux** (requires `ZENMUX_API_KEY`)
|
|
74
|
+
- **Qwen Portal** (supports `QWEN_OAUTH_TOKEN` or `QWEN_PORTAL_API_KEY`)
|
|
75
|
+
- **Cloudflare AI Gateway** (requires `CLOUDFLARE_AI_GATEWAY_API_KEY` and provider-specific gateway base URL)
|
|
76
|
+
- **Ollama** (local OpenAI-compatible runtime; optional `OLLAMA_API_KEY`)
|
|
77
|
+
- **Ollama Cloud** (hosted native Ollama API; requires `OLLAMA_CLOUD_API_KEY`)
|
|
78
|
+
- **llama.cpp** (local OpenAI and Anthropic compatible inference server)
|
|
79
|
+
- **vLLM** (OpenAI-compatible server; `VLLM_API_KEY` for secured deployments)
|
|
70
80
|
- **GitHub Copilot** (requires OAuth, see below)
|
|
71
|
-
- **
|
|
72
|
-
- **
|
|
73
|
-
- **
|
|
74
|
-
- **Fireworks** (uses Anthropic-compatible API)
|
|
75
|
-
- **Kimi For Coding** (Moonshot AI, uses Anthropic-compatible API)
|
|
76
|
-
- **Xiaomi MiMo** (uses Anthropic-compatible API; defaults to API billing endpoint, with separate Token Plan providers for `cn`/`ams`/`sgp` regions)
|
|
77
|
-
- **Any OpenAI-compatible API**: Ollama, vLLM, LM Studio, etc.
|
|
81
|
+
- **Google Gemini CLI** (requires OAuth, see below)
|
|
82
|
+
- **Antigravity** (requires OAuth, see below)
|
|
83
|
+
- **Any OpenAI-compatible API**: LM Studio, custom proxies, etc.
|
|
78
84
|
|
|
79
85
|
## Installation
|
|
80
86
|
|
|
@@ -82,79 +88,82 @@ Unified LLM API with automatic model discovery, provider configuration, token an
|
|
|
82
88
|
npm install @aryee337/aery-ai
|
|
83
89
|
```
|
|
84
90
|
|
|
85
|
-
TypeBox exports are re-exported from `@aryee337/aery-ai`: `Type`, `Static`, and `TSchema`.
|
|
86
|
-
|
|
87
91
|
## Quick Start
|
|
88
92
|
|
|
89
93
|
```typescript
|
|
90
|
-
import {
|
|
94
|
+
import { z, getModel, stream, complete, Context, Tool } from "@aryee337/aery-ai";
|
|
91
95
|
|
|
92
96
|
// Fully typed with auto-complete support for both providers and models
|
|
93
|
-
const model = getModel(
|
|
94
|
-
|
|
95
|
-
// Define tools with
|
|
96
|
-
const tools: Tool[] = [
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
97
|
+
const model = getModel("openai", "gpt-4o-mini");
|
|
98
|
+
|
|
99
|
+
// Define tools with Zod schemas for type safety and validation
|
|
100
|
+
const tools: Tool[] = [
|
|
101
|
+
{
|
|
102
|
+
name: "get_time",
|
|
103
|
+
description: "Get the current time",
|
|
104
|
+
parameters: z.object({
|
|
105
|
+
timezone: z
|
|
106
|
+
.string()
|
|
107
|
+
.optional()
|
|
108
|
+
.describe("Optional timezone (e.g., America/New_York)"),
|
|
109
|
+
}),
|
|
110
|
+
},
|
|
111
|
+
];
|
|
103
112
|
|
|
104
113
|
// Build a conversation context (easily serializable and transferable between models)
|
|
105
114
|
const context: Context = {
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
115
|
+
systemPrompt: ["You are a helpful assistant."],
|
|
116
|
+
messages: [{ role: "user", content: "What time is it?" }],
|
|
117
|
+
tools,
|
|
109
118
|
};
|
|
110
119
|
|
|
111
120
|
// Option 1: Streaming with all event types
|
|
112
121
|
const s = stream(model, context);
|
|
113
122
|
|
|
114
123
|
for await (const event of s) {
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
|
|
124
|
+
switch (event.type) {
|
|
125
|
+
case "start":
|
|
126
|
+
console.log(`Starting with ${event.partial.model}`);
|
|
127
|
+
break;
|
|
128
|
+
case "text_start":
|
|
129
|
+
console.log("\n[Text started]");
|
|
130
|
+
break;
|
|
131
|
+
case "text_delta":
|
|
132
|
+
process.stdout.write(event.delta);
|
|
133
|
+
break;
|
|
134
|
+
case "text_end":
|
|
135
|
+
console.log("\n[Text ended]");
|
|
136
|
+
break;
|
|
137
|
+
case "thinking_start":
|
|
138
|
+
console.log("[Model is thinking...]");
|
|
139
|
+
break;
|
|
140
|
+
case "thinking_delta":
|
|
141
|
+
process.stdout.write(event.delta);
|
|
142
|
+
break;
|
|
143
|
+
case "thinking_end":
|
|
144
|
+
console.log("[Thinking complete]");
|
|
145
|
+
break;
|
|
146
|
+
case "toolcall_start":
|
|
147
|
+
console.log(`\n[Tool call started: index ${event.contentIndex}]`);
|
|
148
|
+
break;
|
|
149
|
+
case "toolcall_delta":
|
|
150
|
+
// Partial tool arguments are being streamed
|
|
151
|
+
const partialCall = event.partial.content[event.contentIndex];
|
|
152
|
+
if (partialCall.type === "toolCall") {
|
|
153
|
+
console.log(`[Streaming args for ${partialCall.name}]`);
|
|
154
|
+
}
|
|
155
|
+
break;
|
|
156
|
+
case "toolcall_end":
|
|
157
|
+
console.log(`\nTool called: ${event.toolCall.name}`);
|
|
158
|
+
console.log(`Arguments: ${JSON.stringify(event.toolCall.arguments)}`);
|
|
159
|
+
break;
|
|
160
|
+
case "done":
|
|
161
|
+
console.log(`\nFinished: ${event.reason}`);
|
|
162
|
+
break;
|
|
163
|
+
case "error":
|
|
164
|
+
console.error(`Error: ${event.error}`);
|
|
165
|
+
break;
|
|
166
|
+
}
|
|
158
167
|
}
|
|
159
168
|
|
|
160
169
|
// Get the final message after streaming, add it to the context
|
|
@@ -162,33 +171,34 @@ const finalMessage = await s.result();
|
|
|
162
171
|
context.messages.push(finalMessage);
|
|
163
172
|
|
|
164
173
|
// Handle tool calls if any
|
|
165
|
-
const toolCalls = finalMessage.content.filter(b => b.type ===
|
|
174
|
+
const toolCalls = finalMessage.content.filter((b) => b.type === "toolCall");
|
|
166
175
|
for (const call of toolCalls) {
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
|
|
173
|
-
|
|
174
|
-
|
|
175
|
-
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
|
|
183
|
-
|
|
184
|
-
|
|
176
|
+
// Execute the tool
|
|
177
|
+
const result =
|
|
178
|
+
call.name === "get_time"
|
|
179
|
+
? new Date().toLocaleString("en-US", {
|
|
180
|
+
timeZone: call.arguments.timezone || "UTC",
|
|
181
|
+
dateStyle: "full",
|
|
182
|
+
timeStyle: "long",
|
|
183
|
+
})
|
|
184
|
+
: "Unknown tool";
|
|
185
|
+
|
|
186
|
+
// Add tool result to context (supports text and images)
|
|
187
|
+
context.messages.push({
|
|
188
|
+
role: "toolResult",
|
|
189
|
+
toolCallId: call.id,
|
|
190
|
+
toolName: call.name,
|
|
191
|
+
content: [{ type: "text", text: result }],
|
|
192
|
+
isError: false,
|
|
193
|
+
timestamp: Date.now(),
|
|
194
|
+
});
|
|
185
195
|
}
|
|
186
196
|
|
|
187
197
|
// Continue if there were tool calls
|
|
188
198
|
if (toolCalls.length > 0) {
|
|
189
|
-
|
|
190
|
-
|
|
191
|
-
|
|
199
|
+
const continuation = await complete(model, context);
|
|
200
|
+
context.messages.push(continuation);
|
|
201
|
+
console.log("After tool execution:", continuation.content);
|
|
192
202
|
}
|
|
193
203
|
|
|
194
204
|
console.log(`Total tokens: ${finalMessage.usage.input} in, ${finalMessage.usage.output} out`);
|
|
@@ -198,45 +208,42 @@ console.log(`Cost: $${finalMessage.usage.cost.total.toFixed(4)}`);
|
|
|
198
208
|
const response = await complete(model, context);
|
|
199
209
|
|
|
200
210
|
for (const block of response.content) {
|
|
201
|
-
|
|
202
|
-
|
|
203
|
-
|
|
204
|
-
|
|
205
|
-
|
|
211
|
+
if (block.type === "text") {
|
|
212
|
+
console.log(block.text);
|
|
213
|
+
} else if (block.type === "toolCall") {
|
|
214
|
+
console.log(`Tool: ${block.name}(${JSON.stringify(block.arguments)})`);
|
|
215
|
+
}
|
|
206
216
|
}
|
|
207
217
|
```
|
|
208
218
|
|
|
209
219
|
## Tools
|
|
210
220
|
|
|
211
|
-
Tools enable LLMs to interact with external systems. This library uses
|
|
221
|
+
Tools enable LLMs to interact with external systems. This library uses **Zod** schemas for type-safe tool definitions with automatic validation. Schemas are converted to JSON Schema for providers as needed.
|
|
212
222
|
|
|
213
223
|
### Defining Tools
|
|
214
224
|
|
|
215
225
|
```typescript
|
|
216
|
-
import {
|
|
226
|
+
import { z, Tool } from "@aryee337/aery-ai";
|
|
217
227
|
|
|
218
|
-
// Define tool parameters with
|
|
228
|
+
// Define tool parameters with Zod
|
|
219
229
|
const weatherTool: Tool = {
|
|
220
|
-
|
|
221
|
-
|
|
222
|
-
|
|
223
|
-
|
|
224
|
-
|
|
225
|
-
|
|
230
|
+
name: "get_weather",
|
|
231
|
+
description: "Get current weather for a location",
|
|
232
|
+
parameters: z.object({
|
|
233
|
+
location: z.string().describe("City name or coordinates"),
|
|
234
|
+
units: z.enum(["celsius", "fahrenheit"]).default("celsius"),
|
|
235
|
+
}),
|
|
226
236
|
};
|
|
227
237
|
|
|
228
|
-
// Note: For Google API compatibility, use StringEnum helper instead of Type.Enum
|
|
229
|
-
// Type.Enum generates anyOf/const patterns that Google doesn't support
|
|
230
|
-
|
|
231
238
|
const bookMeetingTool: Tool = {
|
|
232
|
-
|
|
233
|
-
|
|
234
|
-
|
|
235
|
-
|
|
236
|
-
|
|
237
|
-
|
|
238
|
-
|
|
239
|
-
|
|
239
|
+
name: "book_meeting",
|
|
240
|
+
description: "Schedule a meeting",
|
|
241
|
+
parameters: z.object({
|
|
242
|
+
title: z.string().min(1),
|
|
243
|
+
startTime: z.string().describe("ISO 8601 date-time"),
|
|
244
|
+
endTime: z.string().describe("ISO 8601 date-time"),
|
|
245
|
+
attendees: z.array(z.email()).min(1),
|
|
246
|
+
}),
|
|
240
247
|
};
|
|
241
248
|
```
|
|
242
249
|
|
|
@@ -245,46 +252,46 @@ const bookMeetingTool: Tool = {
|
|
|
245
252
|
Tool results use content blocks and can include both text and images:
|
|
246
253
|
|
|
247
254
|
```typescript
|
|
248
|
-
import
|
|
255
|
+
import * as fs from "node:fs";
|
|
249
256
|
|
|
250
257
|
const context: Context = {
|
|
251
|
-
|
|
252
|
-
|
|
258
|
+
messages: [{ role: "user", content: "What is the weather in London?" }],
|
|
259
|
+
tools: [weatherTool],
|
|
253
260
|
};
|
|
254
261
|
|
|
255
262
|
const response = await complete(model, context);
|
|
256
263
|
|
|
257
264
|
// Check for tool calls in the response
|
|
258
265
|
for (const block of response.content) {
|
|
259
|
-
|
|
260
|
-
|
|
261
|
-
|
|
262
|
-
|
|
263
|
-
|
|
264
|
-
|
|
265
|
-
|
|
266
|
-
|
|
267
|
-
|
|
268
|
-
|
|
269
|
-
|
|
270
|
-
|
|
271
|
-
|
|
272
|
-
|
|
273
|
-
|
|
266
|
+
if (block.type === "toolCall") {
|
|
267
|
+
// Execute your tool with the arguments
|
|
268
|
+
// See "Validating Tool Arguments" section for validation
|
|
269
|
+
const result = await executeWeatherApi(block.arguments);
|
|
270
|
+
|
|
271
|
+
// Add tool result with text content
|
|
272
|
+
context.messages.push({
|
|
273
|
+
role: "toolResult",
|
|
274
|
+
toolCallId: block.id,
|
|
275
|
+
toolName: block.name,
|
|
276
|
+
content: [{ type: "text", text: JSON.stringify(result) }],
|
|
277
|
+
isError: false,
|
|
278
|
+
timestamp: Date.now(),
|
|
279
|
+
});
|
|
280
|
+
}
|
|
274
281
|
}
|
|
275
282
|
|
|
276
283
|
// Tool results can also include images (for vision-capable models)
|
|
277
|
-
const imageBuffer = readFileSync(
|
|
284
|
+
const imageBuffer = fs.readFileSync("chart.png");
|
|
278
285
|
context.messages.push({
|
|
279
|
-
|
|
280
|
-
|
|
281
|
-
|
|
282
|
-
|
|
283
|
-
|
|
284
|
-
|
|
285
|
-
|
|
286
|
-
|
|
287
|
-
|
|
286
|
+
role: "toolResult",
|
|
287
|
+
toolCallId: "tool_xyz",
|
|
288
|
+
toolName: "generate_chart",
|
|
289
|
+
content: [
|
|
290
|
+
{ type: "text", text: "Generated chart showing temperature trends" },
|
|
291
|
+
{ type: "image", data: imageBuffer.toBase64(), mimeType: "image/png" },
|
|
292
|
+
],
|
|
293
|
+
isError: false,
|
|
294
|
+
timestamp: Date.now(),
|
|
288
295
|
});
|
|
289
296
|
```
|
|
290
297
|
|
|
@@ -296,34 +303,35 @@ During streaming, tool call arguments are progressively parsed as they arrive. T
|
|
|
296
303
|
const s = stream(model, context);
|
|
297
304
|
|
|
298
305
|
for await (const event of s) {
|
|
299
|
-
|
|
300
|
-
|
|
301
|
-
|
|
302
|
-
|
|
303
|
-
|
|
304
|
-
|
|
305
|
-
|
|
306
|
-
|
|
307
|
-
|
|
308
|
-
|
|
309
|
-
|
|
310
|
-
|
|
311
|
-
|
|
312
|
-
|
|
313
|
-
|
|
314
|
-
|
|
315
|
-
|
|
316
|
-
|
|
317
|
-
|
|
318
|
-
|
|
319
|
-
|
|
320
|
-
|
|
321
|
-
|
|
322
|
-
|
|
306
|
+
if (event.type === "toolcall_delta") {
|
|
307
|
+
const toolCall = event.partial.content[event.contentIndex];
|
|
308
|
+
|
|
309
|
+
// toolCall.arguments contains partially parsed JSON during streaming
|
|
310
|
+
// This allows for progressive UI updates
|
|
311
|
+
if (toolCall.type === "toolCall" && toolCall.arguments) {
|
|
312
|
+
// BE DEFENSIVE: arguments may be incomplete
|
|
313
|
+
// Example: Show file path being written even before content is complete
|
|
314
|
+
if (toolCall.name === "write_file" && toolCall.arguments.path) {
|
|
315
|
+
console.log(`Writing to: ${toolCall.arguments.path}`);
|
|
316
|
+
|
|
317
|
+
// Content might be partial or missing
|
|
318
|
+
if (toolCall.arguments.content) {
|
|
319
|
+
console.log(`Content preview: ${toolCall.arguments.content.substring(0, 100)}...`);
|
|
320
|
+
}
|
|
321
|
+
}
|
|
322
|
+
}
|
|
323
|
+
}
|
|
324
|
+
|
|
325
|
+
if (event.type === "toolcall_end") {
|
|
326
|
+
// Here toolCall.arguments is complete (but not yet validated)
|
|
327
|
+
const toolCall = event.toolCall;
|
|
328
|
+
console.log(`Tool completed: ${toolCall.name}`, toolCall.arguments);
|
|
329
|
+
}
|
|
323
330
|
}
|
|
324
331
|
```
|
|
325
332
|
|
|
326
333
|
**Important notes about partial tool arguments:**
|
|
334
|
+
|
|
327
335
|
- During `toolcall_delta` events, `arguments` contains the best-effort parse of partial JSON
|
|
328
336
|
- Fields may be missing or incomplete - always check for existence before use
|
|
329
337
|
- String values may be truncated mid-word
|
|
@@ -334,37 +342,37 @@ for await (const event of s) {
|
|
|
334
342
|
|
|
335
343
|
### Validating Tool Arguments
|
|
336
344
|
|
|
337
|
-
When using `agentLoop`, tool arguments are automatically validated against your
|
|
345
|
+
When using `agentLoop`, tool arguments are automatically validated against your Zod parameter schemas before execution. If validation fails, the error is returned to the model as a tool result, allowing it to retry.
|
|
338
346
|
|
|
339
347
|
When implementing your own tool execution loop with `stream()` or `complete()`, use `validateToolCall` to validate arguments before passing them to your tools:
|
|
340
348
|
|
|
341
349
|
```typescript
|
|
342
|
-
import { stream, validateToolCall, Tool } from
|
|
350
|
+
import { stream, validateToolCall, Tool } from "@aryee337/aery-ai";
|
|
343
351
|
|
|
344
352
|
const tools: Tool[] = [weatherTool, calculatorTool];
|
|
345
353
|
const s = stream(model, { messages, tools });
|
|
346
354
|
|
|
347
355
|
for await (const event of s) {
|
|
348
|
-
|
|
349
|
-
|
|
350
|
-
|
|
351
|
-
|
|
352
|
-
|
|
353
|
-
|
|
354
|
-
|
|
355
|
-
|
|
356
|
-
|
|
357
|
-
|
|
358
|
-
|
|
359
|
-
|
|
360
|
-
|
|
361
|
-
|
|
362
|
-
|
|
363
|
-
|
|
364
|
-
|
|
365
|
-
|
|
366
|
-
|
|
367
|
-
|
|
356
|
+
if (event.type === "toolcall_end") {
|
|
357
|
+
const toolCall = event.toolCall;
|
|
358
|
+
|
|
359
|
+
try {
|
|
360
|
+
// Validate arguments against the tool's schema (throws on invalid args)
|
|
361
|
+
const validatedArgs = validateToolCall(tools, toolCall);
|
|
362
|
+
const result = await executeMyTool(toolCall.name, validatedArgs);
|
|
363
|
+
// ... add tool result to context
|
|
364
|
+
} catch (error) {
|
|
365
|
+
// Validation failed - return error as tool result so model can retry
|
|
366
|
+
context.messages.push({
|
|
367
|
+
role: "toolResult",
|
|
368
|
+
toolCallId: toolCall.id,
|
|
369
|
+
toolName: toolCall.name,
|
|
370
|
+
content: [{ type: "text", text: error.message }],
|
|
371
|
+
isError: true,
|
|
372
|
+
timestamp: Date.now(),
|
|
373
|
+
});
|
|
374
|
+
}
|
|
375
|
+
}
|
|
368
376
|
}
|
|
369
377
|
```
|
|
370
378
|
|
|
@@ -372,123 +380,59 @@ for await (const event of s) {
|
|
|
372
380
|
|
|
373
381
|
All streaming events emitted during assistant message generation:
|
|
374
382
|
|
|
375
|
-
| Event Type
|
|
376
|
-
|
|
377
|
-
| `start`
|
|
378
|
-
| `text_start`
|
|
379
|
-
| `text_delta`
|
|
380
|
-
| `text_end`
|
|
381
|
-
| `thinking_start` | Thinking block starts
|
|
382
|
-
| `thinking_delta` | Thinking chunk received
|
|
383
|
-
| `thinking_end`
|
|
384
|
-
| `toolcall_start` | Tool call begins
|
|
385
|
-
| `toolcall_delta` | Tool arguments streaming | `delta`: JSON chunk, `partial.content[contentIndex].arguments`: Partial parsed args
|
|
386
|
-
| `toolcall_end`
|
|
387
|
-
| `done`
|
|
388
|
-
| `error`
|
|
389
|
-
|
|
390
|
-
Streaming events for different content blocks are not guaranteed to be contiguous. Providers may emit deltas for text, thinking, and tool calls in the same upstream chunk, and pi may surface corresponding events interleaved, for example `text_start`, `text_delta`, `toolcall_start`, `text_delta`, `toolcall_delta`. Consumers must use `contentIndex` to associate each delta/end event with its block and must not assume that a block's `*_start`/`*_delta`/`*_end` sequence is uninterrupted by events for other blocks.
|
|
383
|
+
| Event Type | Description | Key Properties |
|
|
384
|
+
| ---------------- | ------------------------ | ------------------------------------------------------------------------------------------- |
|
|
385
|
+
| `start` | Stream begins | `partial`: Initial assistant message structure |
|
|
386
|
+
| `text_start` | Text block starts | `contentIndex`: Position in content array |
|
|
387
|
+
| `text_delta` | Text chunk received | `delta`: New text, `contentIndex`: Position |
|
|
388
|
+
| `text_end` | Text block complete | `content`: Full text, `contentIndex`: Position |
|
|
389
|
+
| `thinking_start` | Thinking block starts | `contentIndex`: Position in content array |
|
|
390
|
+
| `thinking_delta` | Thinking chunk received | `delta`: New text, `contentIndex`: Position |
|
|
391
|
+
| `thinking_end` | Thinking block complete | `content`: Full thinking, `contentIndex`: Position |
|
|
392
|
+
| `toolcall_start` | Tool call begins | `contentIndex`: Position in content array |
|
|
393
|
+
| `toolcall_delta` | Tool arguments streaming | `delta`: JSON chunk, `partial.content[contentIndex].arguments`: Partial parsed args |
|
|
394
|
+
| `toolcall_end` | Tool call complete | `toolCall`: Complete validated tool call with `id`, `name`, `arguments` |
|
|
395
|
+
| `done` | Stream complete | `reason`: Stop reason ("stop", "length", "toolUse"), `message`: Final assistant message |
|
|
396
|
+
| `error` | Error occurred | `reason`: Error type ("error" or "aborted"), `error`: AssistantMessage with partial content |
|
|
391
397
|
|
|
392
398
|
## Image Input
|
|
393
399
|
|
|
394
400
|
Models with vision capabilities can process images. You can check if a model supports images via the `input` property. If you pass images to a non-vision model, they are silently ignored.
|
|
395
401
|
|
|
396
402
|
```typescript
|
|
397
|
-
import
|
|
398
|
-
import { getModel, complete } from
|
|
403
|
+
import * as fs from "node:fs";
|
|
404
|
+
import { getModel, complete } from "@aryee337/aery-ai";
|
|
399
405
|
|
|
400
|
-
const model = getModel(
|
|
406
|
+
const model = getModel("openai", "gpt-4o-mini");
|
|
401
407
|
|
|
402
408
|
// Check if model supports images
|
|
403
|
-
if (model.input.includes(
|
|
404
|
-
|
|
409
|
+
if (model.input.includes("image")) {
|
|
410
|
+
console.log("Model supports vision");
|
|
405
411
|
}
|
|
406
412
|
|
|
407
|
-
const imageBuffer = readFileSync(
|
|
408
|
-
const base64Image = imageBuffer.
|
|
413
|
+
const imageBuffer = fs.readFileSync("image.png");
|
|
414
|
+
const base64Image = imageBuffer.toBase64();
|
|
409
415
|
|
|
410
416
|
const response = await complete(model, {
|
|
411
|
-
|
|
412
|
-
|
|
413
|
-
|
|
414
|
-
|
|
415
|
-
|
|
416
|
-
|
|
417
|
-
|
|
417
|
+
messages: [
|
|
418
|
+
{
|
|
419
|
+
role: "user",
|
|
420
|
+
content: [
|
|
421
|
+
{ type: "text", text: "What is in this image?" },
|
|
422
|
+
{ type: "image", data: base64Image, mimeType: "image/png" },
|
|
423
|
+
],
|
|
424
|
+
},
|
|
425
|
+
],
|
|
418
426
|
});
|
|
419
427
|
|
|
420
428
|
// Access the response
|
|
421
429
|
for (const block of response.content) {
|
|
422
|
-
|
|
423
|
-
|
|
424
|
-
|
|
425
|
-
}
|
|
426
|
-
```
|
|
427
|
-
|
|
428
|
-
## Image Generation
|
|
429
|
-
|
|
430
|
-
Image generation uses a separate API surface from text/chat generation. Use `getImageModel()` / `getImageModels()` / `getImageProviders()` to discover image-generation models, and `generateImages()` to get the final result.
|
|
431
|
-
|
|
432
|
-
Do not use `stream()` or `complete()` for image generation. Image generation is a one-shot API: `generateImages()` waits for the provider response and returns the final `AssistantImages` result.
|
|
433
|
-
|
|
434
|
-
### Basic Image Generation
|
|
435
|
-
|
|
436
|
-
```typescript
|
|
437
|
-
import { getImageModel, generateImages } from '@aryee337/aery-ai';
|
|
438
|
-
|
|
439
|
-
const model = getImageModel('openrouter', 'google/gemini-2.5-flash-image');
|
|
440
|
-
|
|
441
|
-
const result = await generateImages(model, {
|
|
442
|
-
input: [{ type: 'text', text: 'Generate a red circle on a plain white background.' }]
|
|
443
|
-
}, {
|
|
444
|
-
apiKey: process.env.OPENROUTER_API_KEY
|
|
445
|
-
});
|
|
446
|
-
|
|
447
|
-
for (const block of result.output) {
|
|
448
|
-
if (block.type === 'text') {
|
|
449
|
-
console.log(block.text);
|
|
450
|
-
} else if (block.type === 'image') {
|
|
451
|
-
console.log(block.mimeType);
|
|
452
|
-
console.log(block.data.substring(0, 32));
|
|
453
|
-
}
|
|
430
|
+
if (block.type === "text") {
|
|
431
|
+
console.log(block.text);
|
|
432
|
+
}
|
|
454
433
|
}
|
|
455
434
|
```
|
|
456
435
|
|
|
457
|
-
Some models also support image input:
|
|
458
|
-
|
|
459
|
-
```typescript
|
|
460
|
-
import { readFileSync } from 'fs';
|
|
461
|
-
|
|
462
|
-
const imageBuffer = readFileSync('input.png');
|
|
463
|
-
const result = await generateImages(model, {
|
|
464
|
-
input: [
|
|
465
|
-
{ type: 'text', text: 'Create a variation of this image with a blue background.' },
|
|
466
|
-
{ type: 'image', data: imageBuffer.toString('base64'), mimeType: 'image/png' }
|
|
467
|
-
]
|
|
468
|
-
}, {
|
|
469
|
-
apiKey: process.env.OPENROUTER_API_KEY
|
|
470
|
-
});
|
|
471
|
-
```
|
|
472
|
-
|
|
473
|
-
Check capabilities on the model metadata:
|
|
474
|
-
|
|
475
|
-
```typescript
|
|
476
|
-
console.log(model.input); // ['text', 'image']
|
|
477
|
-
console.log(model.output); // ['image'] or ['image', 'text']
|
|
478
|
-
```
|
|
479
|
-
|
|
480
|
-
### Notes and Limitations
|
|
481
|
-
|
|
482
|
-
- Use `getImageModel(...)`, not `getModel(...)`.
|
|
483
|
-
- Use `generateImages()`, not `stream()` / `complete()`.
|
|
484
|
-
- Image-generation models do not participate in tool calling.
|
|
485
|
-
- Outputs are returned in `AssistantImages.output` and can include both base64-encoded `ImageContent` blocks and `TextContent` blocks.
|
|
486
|
-
- Some models return only images, others return images plus text. Check `model.output`.
|
|
487
|
-
- Some models accept image input, others are text-to-image only. Check `model.input`.
|
|
488
|
-
- Like the streaming APIs, image generation supports options such as `apiKey`, `signal`, `headers`, `onPayload`, and `onResponse`, and results may include `stopReason`, `responseId`, and `usage`.
|
|
489
|
-
- If you want a model to analyze images in a conversation or call tools, use the regular `stream()` / `complete()` APIs with a model that supports image input.
|
|
490
|
-
- At the moment, image generation is available through only one provider, OpenRouter.
|
|
491
|
-
|
|
492
436
|
## Thinking/Reasoning
|
|
493
437
|
|
|
494
438
|
Many models support thinking/reasoning capabilities where they can show their internal thought process. You can check if a model supports reasoning via the `reasoning` property. If you pass reasoning options to a non-reasoning model, they are silently ignored.
|
|
@@ -496,10 +440,10 @@ Many models support thinking/reasoning capabilities where they can show their in
|
|
|
496
440
|
### Unified Interface (streamSimple/completeSimple)
|
|
497
441
|
|
|
498
442
|
```typescript
|
|
499
|
-
import { getModel, streamSimple, completeSimple } from
|
|
443
|
+
import { getModel, streamSimple, completeSimple } from "@aryee337/aery-ai";
|
|
500
444
|
|
|
501
445
|
// Many models across providers support thinking/reasoning
|
|
502
|
-
const model = getModel(
|
|
446
|
+
const model = getModel("anthropic", "claude-sonnet-4-20250514");
|
|
503
447
|
// or getModel('openai', 'gpt-5-mini');
|
|
504
448
|
// or getModel('google', 'gemini-2.5-flash');
|
|
505
449
|
// or getModel('xai', 'grok-code-fast-1');
|
|
@@ -509,23 +453,27 @@ const model = getModel('anthropic', 'claude-sonnet-4-20250514');
|
|
|
509
453
|
|
|
510
454
|
// Check if model supports reasoning
|
|
511
455
|
if (model.reasoning) {
|
|
512
|
-
|
|
456
|
+
console.log("Model supports reasoning/thinking");
|
|
513
457
|
}
|
|
514
458
|
|
|
515
459
|
// Use the simplified reasoning option
|
|
516
|
-
const response = await completeSimple(
|
|
517
|
-
|
|
518
|
-
|
|
519
|
-
|
|
520
|
-
}
|
|
460
|
+
const response = await completeSimple(
|
|
461
|
+
model,
|
|
462
|
+
{
|
|
463
|
+
messages: [{ role: "user", content: "Solve: 2x + 5 = 13" }],
|
|
464
|
+
},
|
|
465
|
+
{
|
|
466
|
+
reasoning: "medium", // 'minimal' | 'low' | 'medium' | 'high' | 'xhigh' (xhigh maps to high on non-OpenAI providers)
|
|
467
|
+
}
|
|
468
|
+
);
|
|
521
469
|
|
|
522
470
|
// Access thinking and text blocks
|
|
523
471
|
for (const block of response.content) {
|
|
524
|
-
|
|
525
|
-
|
|
526
|
-
|
|
527
|
-
|
|
528
|
-
|
|
472
|
+
if (block.type === "thinking") {
|
|
473
|
+
console.log("Thinking:", block.thinking);
|
|
474
|
+
} else if (block.type === "text") {
|
|
475
|
+
console.log("Response:", block.text);
|
|
476
|
+
}
|
|
529
477
|
}
|
|
530
478
|
```
|
|
531
479
|
|
|
@@ -534,29 +482,29 @@ for (const block of response.content) {
|
|
|
534
482
|
For fine-grained control, use the provider-specific options:
|
|
535
483
|
|
|
536
484
|
```typescript
|
|
537
|
-
import { getModel, complete } from
|
|
485
|
+
import { getModel, complete } from "@aryee337/aery-ai";
|
|
538
486
|
|
|
539
487
|
// OpenAI Reasoning (o1, o3, gpt-5)
|
|
540
|
-
const openaiModel = getModel(
|
|
488
|
+
const openaiModel = getModel("openai", "gpt-5-mini");
|
|
541
489
|
await complete(openaiModel, context, {
|
|
542
|
-
|
|
543
|
-
|
|
490
|
+
reasoningEffort: "medium",
|
|
491
|
+
reasoningSummary: "detailed", // OpenAI Responses API only
|
|
544
492
|
});
|
|
545
493
|
|
|
546
494
|
// Anthropic Thinking (Claude Sonnet 4)
|
|
547
|
-
const anthropicModel = getModel(
|
|
495
|
+
const anthropicModel = getModel("anthropic", "claude-sonnet-4-20250514");
|
|
548
496
|
await complete(anthropicModel, context, {
|
|
549
|
-
|
|
550
|
-
|
|
497
|
+
thinkingEnabled: true,
|
|
498
|
+
thinkingBudgetTokens: 8192, // Optional token limit
|
|
551
499
|
});
|
|
552
500
|
|
|
553
501
|
// Google Gemini Thinking
|
|
554
|
-
const googleModel = getModel(
|
|
502
|
+
const googleModel = getModel("google", "gemini-2.5-flash");
|
|
555
503
|
await complete(googleModel, context, {
|
|
556
|
-
|
|
557
|
-
|
|
558
|
-
|
|
559
|
-
|
|
504
|
+
thinking: {
|
|
505
|
+
enabled: true,
|
|
506
|
+
budgetTokens: 8192, // -1 for dynamic, 0 to disable
|
|
507
|
+
},
|
|
560
508
|
});
|
|
561
509
|
```
|
|
562
510
|
|
|
@@ -565,20 +513,20 @@ await complete(googleModel, context, {
|
|
|
565
513
|
When streaming, thinking content is delivered through specific events:
|
|
566
514
|
|
|
567
515
|
```typescript
|
|
568
|
-
const s = streamSimple(model, context, { reasoning:
|
|
516
|
+
const s = streamSimple(model, context, { reasoning: "high" });
|
|
569
517
|
|
|
570
518
|
for await (const event of s) {
|
|
571
|
-
|
|
572
|
-
|
|
573
|
-
|
|
574
|
-
|
|
575
|
-
|
|
576
|
-
|
|
577
|
-
|
|
578
|
-
|
|
579
|
-
|
|
580
|
-
|
|
581
|
-
|
|
519
|
+
switch (event.type) {
|
|
520
|
+
case "thinking_start":
|
|
521
|
+
console.log("[Model started thinking]");
|
|
522
|
+
break;
|
|
523
|
+
case "thinking_delta":
|
|
524
|
+
process.stdout.write(event.delta); // Stream thinking content
|
|
525
|
+
break;
|
|
526
|
+
case "thinking_end":
|
|
527
|
+
console.log("\n[Thinking complete]");
|
|
528
|
+
break;
|
|
529
|
+
}
|
|
582
530
|
}
|
|
583
531
|
```
|
|
584
532
|
|
|
@@ -592,8 +540,6 @@ Every `AssistantMessage` includes a `stopReason` field that indicates how the ge
|
|
|
592
540
|
- `"error"` - An error occurred during generation
|
|
593
541
|
- `"aborted"` - Request was cancelled via abort signal
|
|
594
542
|
|
|
595
|
-
`AssistantMessage` may also include `responseId`, a provider-specific upstream response or message identifier when the underlying API exposes one. Do not assume it is always present across providers.
|
|
596
|
-
|
|
597
543
|
## Error Handling
|
|
598
544
|
|
|
599
545
|
When a request ends with an error (including aborts and tool call validation errors), the streaming API emits an error event:
|
|
@@ -601,20 +547,20 @@ When a request ends with an error (including aborts and tool call validation err
|
|
|
601
547
|
```typescript
|
|
602
548
|
// In streaming
|
|
603
549
|
for await (const event of stream) {
|
|
604
|
-
|
|
605
|
-
|
|
606
|
-
|
|
607
|
-
|
|
608
|
-
|
|
609
|
-
|
|
550
|
+
if (event.type === "error") {
|
|
551
|
+
// event.reason is either "error" or "aborted"
|
|
552
|
+
// event.error is the AssistantMessage with partial content
|
|
553
|
+
console.error(`Error (${event.reason}):`, event.error.errorMessage);
|
|
554
|
+
console.log("Partial content:", event.error.content);
|
|
555
|
+
}
|
|
610
556
|
}
|
|
611
557
|
|
|
612
558
|
// The final message will have the error details
|
|
613
559
|
const message = await stream.result();
|
|
614
|
-
if (message.stopReason ===
|
|
615
|
-
|
|
616
|
-
|
|
617
|
-
|
|
560
|
+
if (message.stopReason === "error" || message.stopReason === "aborted") {
|
|
561
|
+
console.error("Request failed:", message.errorMessage);
|
|
562
|
+
// message.content contains any partial content received before the error
|
|
563
|
+
// message.usage contains partial token counts and costs
|
|
618
564
|
}
|
|
619
565
|
```
|
|
620
566
|
|
|
@@ -623,35 +569,38 @@ if (message.stopReason === 'error' || message.stopReason === 'aborted') {
|
|
|
623
569
|
The abort signal allows you to cancel in-progress requests. Aborted requests have `stopReason === 'aborted'`:
|
|
624
570
|
|
|
625
571
|
```typescript
|
|
626
|
-
import { getModel, stream } from
|
|
572
|
+
import { getModel, stream } from "@aryee337/aery-ai";
|
|
627
573
|
|
|
628
|
-
const model = getModel(
|
|
629
|
-
const controller = new AbortController();
|
|
574
|
+
const model = getModel("openai", "gpt-4o-mini");
|
|
630
575
|
|
|
631
576
|
// Abort after 2 seconds
|
|
632
|
-
|
|
633
|
-
|
|
634
|
-
const s = stream(
|
|
635
|
-
|
|
636
|
-
|
|
637
|
-
|
|
638
|
-
}
|
|
577
|
+
const signal = AbortSignal.timeout(2000);
|
|
578
|
+
|
|
579
|
+
const s = stream(
|
|
580
|
+
model,
|
|
581
|
+
{
|
|
582
|
+
messages: [{ role: "user", content: "Write a long story" }],
|
|
583
|
+
},
|
|
584
|
+
{
|
|
585
|
+
signal,
|
|
586
|
+
}
|
|
587
|
+
);
|
|
639
588
|
|
|
640
589
|
for await (const event of s) {
|
|
641
|
-
|
|
642
|
-
|
|
643
|
-
|
|
644
|
-
|
|
645
|
-
|
|
646
|
-
|
|
590
|
+
if (event.type === "text_delta") {
|
|
591
|
+
process.stdout.write(event.delta);
|
|
592
|
+
} else if (event.type === "error") {
|
|
593
|
+
// event.reason tells you if it was "error" or "aborted"
|
|
594
|
+
console.log(`${event.reason === "aborted" ? "Aborted" : "Error"}:`, event.error.errorMessage);
|
|
595
|
+
}
|
|
647
596
|
}
|
|
648
597
|
|
|
649
598
|
// Get results (may be partial if aborted)
|
|
650
599
|
const response = await s.result();
|
|
651
|
-
if (response.stopReason ===
|
|
652
|
-
|
|
653
|
-
|
|
654
|
-
|
|
600
|
+
if (response.stopReason === "aborted") {
|
|
601
|
+
console.log("Request was aborted:", response.errorMessage);
|
|
602
|
+
console.log("Partial content received:", response.content);
|
|
603
|
+
console.log("Tokens used:", response.usage);
|
|
655
604
|
}
|
|
656
605
|
```
|
|
657
606
|
|
|
@@ -661,9 +610,7 @@ Aborted messages can be added to the conversation context and continued in subse
|
|
|
661
610
|
|
|
662
611
|
```typescript
|
|
663
612
|
const context = {
|
|
664
|
-
|
|
665
|
-
{ role: 'user', content: 'Explain quantum computing in detail' }
|
|
666
|
-
]
|
|
613
|
+
messages: [{ role: "user", content: "Explain quantum computing in detail" }],
|
|
667
614
|
};
|
|
668
615
|
|
|
669
616
|
// First request gets aborted after 2 seconds
|
|
@@ -674,278 +621,168 @@ const partial = await complete(model, context, { signal: controller1.signal });
|
|
|
674
621
|
|
|
675
622
|
// Add the partial response to context
|
|
676
623
|
context.messages.push(partial);
|
|
677
|
-
context.messages.push({ role:
|
|
624
|
+
context.messages.push({ role: "user", content: "Please continue" });
|
|
678
625
|
|
|
679
626
|
// Continue the conversation
|
|
680
627
|
const continuation = await complete(model, context);
|
|
681
628
|
```
|
|
682
629
|
|
|
683
|
-
###
|
|
630
|
+
### Common Stream Options
|
|
631
|
+
|
|
632
|
+
All providers accept the base `StreamOptions` (in addition to provider-specific options):
|
|
684
633
|
|
|
685
|
-
|
|
634
|
+
- `apiKey`: Override the provider API key
|
|
635
|
+
- `headers`: Extra request headers merged on top of model-defined headers
|
|
636
|
+
- `sessionId`: Provider-specific session identifier (prompt caching/routing)
|
|
637
|
+
- `signal`: Abort in-flight requests
|
|
638
|
+
- `onPayload`: Callback invoked with the provider request payload just before sending
|
|
639
|
+
|
|
640
|
+
Example:
|
|
686
641
|
|
|
687
642
|
```typescript
|
|
688
643
|
const response = await complete(model, context, {
|
|
689
|
-
|
|
690
|
-
|
|
691
|
-
|
|
644
|
+
apiKey: "sk-live",
|
|
645
|
+
headers: { "X-Debug-Trace": "true" },
|
|
646
|
+
onPayload: (payload) => {
|
|
647
|
+
console.log("request payload", payload);
|
|
648
|
+
},
|
|
692
649
|
});
|
|
693
650
|
```
|
|
694
651
|
|
|
695
|
-
The callback is supported by `stream`, `complete`, `streamSimple`, and `completeSimple`.
|
|
696
|
-
|
|
697
652
|
## APIs, Models, and Providers
|
|
698
653
|
|
|
699
|
-
The library
|
|
700
|
-
|
|
701
|
-
- **`anthropic-messages`**: Anthropic Messages API (`streamAnthropic`, `AnthropicOptions`)
|
|
702
|
-
- **`google-generative-ai`**: Google Generative AI API (`streamGoogle`, `GoogleOptions`)
|
|
703
|
-
- **`google-vertex`**: Google Vertex AI API (`streamGoogleVertex`, `GoogleVertexOptions`)
|
|
704
|
-
- **`mistral-conversations`**: Mistral Conversations API (`streamMistral`, `MistralOptions`)
|
|
705
|
-
- **`openai-completions`**: OpenAI Chat Completions API (`streamOpenAICompletions`, `OpenAICompletionsOptions`)
|
|
706
|
-
- **`openai-responses`**: OpenAI Responses API (`streamOpenAIResponses`, `OpenAIResponsesOptions`)
|
|
707
|
-
- **`openai-codex-responses`**: OpenAI Codex Responses API (`streamOpenAICodexResponses`, `OpenAICodexResponsesOptions`)
|
|
708
|
-
- **`azure-openai-responses`**: Azure OpenAI Responses API (`streamAzureOpenAIResponses`, `AzureOpenAIResponsesOptions`)
|
|
709
|
-
- **`bedrock-converse-stream`**: Amazon Bedrock Converse API (`streamBedrock`, `BedrockOptions`)
|
|
710
|
-
|
|
711
|
-
### Faux provider for tests
|
|
712
|
-
|
|
713
|
-
`registerFauxProvider()` registers a temporary in-memory provider for tests and demos. It is opt-in and not part of the built-in provider set.
|
|
714
|
-
|
|
715
|
-
```typescript
|
|
716
|
-
import {
|
|
717
|
-
complete,
|
|
718
|
-
fauxAssistantMessage,
|
|
719
|
-
fauxText,
|
|
720
|
-
fauxThinking,
|
|
721
|
-
fauxToolCall,
|
|
722
|
-
registerFauxProvider,
|
|
723
|
-
stream,
|
|
724
|
-
} from '@aryee337/aery-ai';
|
|
725
|
-
|
|
726
|
-
const registration = registerFauxProvider({
|
|
727
|
-
tokensPerSecond: 50 // optional
|
|
728
|
-
});
|
|
729
|
-
|
|
730
|
-
const model = registration.getModel();
|
|
731
|
-
const context = {
|
|
732
|
-
messages: [{ role: 'user', content: 'Summarize package.json and then call echo', timestamp: Date.now() }]
|
|
733
|
-
};
|
|
734
|
-
|
|
735
|
-
registration.setResponses([
|
|
736
|
-
fauxAssistantMessage([
|
|
737
|
-
fauxThinking('Need to inspect package metadata first.'),
|
|
738
|
-
fauxToolCall('echo', { text: 'package.json' })
|
|
739
|
-
], { stopReason: 'toolUse' })
|
|
740
|
-
]);
|
|
654
|
+
The library implements 4 API interfaces, each with its own streaming function and options:
|
|
741
655
|
|
|
742
|
-
|
|
743
|
-
|
|
744
|
-
|
|
745
|
-
|
|
746
|
-
context.messages.push(first);
|
|
747
|
-
|
|
748
|
-
context.messages.push({
|
|
749
|
-
role: 'toolResult',
|
|
750
|
-
toolCallId: first.content.find((block) => block.type === 'toolCall')!.id,
|
|
751
|
-
toolName: 'echo',
|
|
752
|
-
content: [{ type: 'text', text: 'package.json contents here' }],
|
|
753
|
-
isError: false,
|
|
754
|
-
timestamp: Date.now()
|
|
755
|
-
});
|
|
756
|
-
|
|
757
|
-
registration.setResponses([
|
|
758
|
-
fauxAssistantMessage([
|
|
759
|
-
fauxThinking('Now I can summarize the tool output.'),
|
|
760
|
-
fauxText('Here is the summary.')
|
|
761
|
-
])
|
|
762
|
-
]);
|
|
763
|
-
|
|
764
|
-
const s = stream(model, context);
|
|
765
|
-
for await (const event of s) {
|
|
766
|
-
console.log(event.type);
|
|
767
|
-
}
|
|
768
|
-
|
|
769
|
-
// Optional: register multiple faux models for model-switching tests
|
|
770
|
-
const multiModel = registerFauxProvider({
|
|
771
|
-
models: [
|
|
772
|
-
{ id: 'faux-fast', reasoning: false },
|
|
773
|
-
{ id: 'faux-thinker', reasoning: true }
|
|
774
|
-
]
|
|
775
|
-
});
|
|
776
|
-
const thinker = multiModel.getModel('faux-thinker');
|
|
777
|
-
|
|
778
|
-
console.log(thinker?.reasoning);
|
|
779
|
-
console.log(registration.getPendingResponseCount());
|
|
780
|
-
console.log(registration.state.callCount);
|
|
781
|
-
registration.unregister();
|
|
782
|
-
multiModel.unregister();
|
|
783
|
-
```
|
|
784
|
-
|
|
785
|
-
Notes:
|
|
786
|
-
- Responses are consumed from a queue in request start order.
|
|
787
|
-
- If the queue is empty, the faux provider returns an assistant error message with `errorMessage: "No more faux responses queued"`.
|
|
788
|
-
- Use `registration.setResponses([...])` to replace the remaining queue and `registration.appendResponses([...])` to add more responses.
|
|
789
|
-
- `registration.models` exposes all registered faux models. `registration.getModel()` returns the first one, and `registration.getModel(id)` returns a specific one.
|
|
790
|
-
- Use `fauxAssistantMessage(...)` for scripted assistant replies. Use `fauxText(...)`, `fauxThinking(...)`, and `fauxToolCall(...)` to build content blocks without filling in low-level fields manually.
|
|
791
|
-
- `registration.unregister()` removes the temporary provider from the global API registry.
|
|
792
|
-
- Usage is estimated at roughly 1 token per 4 characters. When `sessionId` is present and `cacheRetention` is not `"none"`, prompt cache reads and writes are simulated automatically.
|
|
793
|
-
- Tool call arguments stream incrementally via `toolcall_delta` chunks.
|
|
794
|
-
- By default, each streamed chunk is emitted on its own microtask. Set `tokensPerSecond` to pace chunk delivery in real time.
|
|
795
|
-
- The intended use is one deterministic scripted flow per registration. If you need independent concurrent flows, register separate faux providers.
|
|
656
|
+
- **`anthropic-messages`**: Anthropic's Messages API (`streamAnthropic`, `AnthropicOptions`)
|
|
657
|
+
- **`google-generative-ai`**: Google's Generative AI API (`streamGoogle`, `GoogleOptions`)
|
|
658
|
+
- **`openai-completions`**: OpenAI's Chat Completions API (`streamOpenAICompletions`, `OpenAICompletionsOptions`)
|
|
659
|
+
- **`openai-responses`**: OpenAI's Responses API (`streamOpenAIResponses`, `OpenAIResponsesOptions`)
|
|
796
660
|
|
|
797
661
|
### Providers and Models
|
|
798
662
|
|
|
799
663
|
A **provider** offers models through a specific API. For example:
|
|
664
|
+
|
|
800
665
|
- **Anthropic** models use the `anthropic-messages` API
|
|
801
666
|
- **Google** models use the `google-generative-ai` API
|
|
802
667
|
- **OpenAI** models use the `openai-responses` API
|
|
803
|
-
- **Mistral
|
|
804
|
-
- **xAI, Cerebras, Groq, Together AI, etc.** models use the `openai-completions` API (OpenAI-compatible)
|
|
668
|
+
- **Mistral, xAI, Cerebras, Groq, etc.** models use the `openai-completions` API (OpenAI-compatible)
|
|
805
669
|
|
|
806
670
|
### Querying Providers and Models
|
|
807
671
|
|
|
808
672
|
```typescript
|
|
809
|
-
import { getProviders, getModels, getModel } from
|
|
673
|
+
import { getProviders, getModels, getModel } from "@aryee337/aery-ai";
|
|
810
674
|
|
|
811
675
|
// Get all available providers
|
|
812
676
|
const providers = getProviders();
|
|
813
677
|
console.log(providers); // ['openai', 'anthropic', 'google', 'xai', 'groq', ...]
|
|
814
678
|
|
|
815
679
|
// Get all models from a provider (fully typed)
|
|
816
|
-
const anthropicModels = getModels(
|
|
680
|
+
const anthropicModels = getModels("anthropic");
|
|
817
681
|
for (const model of anthropicModels) {
|
|
818
|
-
|
|
819
|
-
|
|
820
|
-
|
|
821
|
-
|
|
822
|
-
|
|
682
|
+
console.log(`${model.id}: ${model.name}`);
|
|
683
|
+
console.log(` API: ${model.api}`); // 'anthropic-messages'
|
|
684
|
+
console.log(` Context: ${model.contextWindow} tokens`);
|
|
685
|
+
console.log(` Vision: ${model.input.includes("image")}`);
|
|
686
|
+
console.log(` Reasoning: ${model.reasoning}`);
|
|
823
687
|
}
|
|
824
688
|
|
|
825
689
|
// Get a specific model (both provider and model ID are auto-completed in IDEs)
|
|
826
|
-
const model = getModel(
|
|
690
|
+
const model = getModel("openai", "gpt-4o-mini");
|
|
827
691
|
console.log(`Using ${model.name} via ${model.api} API`);
|
|
828
692
|
```
|
|
829
693
|
|
|
830
694
|
### Custom Models
|
|
831
695
|
|
|
832
|
-
You can create custom models for local inference servers or custom endpoints
|
|
696
|
+
You can create custom models for local inference servers or custom endpoints.
|
|
697
|
+
|
|
698
|
+
For local Ollama, `OLLAMA_API_KEY` is optional and mainly needed for authenticated/self-hosted gateways. `ollama` remains the local OpenAI-compatible runtime integration.
|
|
833
699
|
|
|
834
700
|
```typescript
|
|
835
|
-
import { Model, stream } from
|
|
836
|
-
|
|
837
|
-
// Example: Ollama using OpenAI-compatible API
|
|
838
|
-
const ollamaModel: Model<
|
|
839
|
-
|
|
840
|
-
|
|
841
|
-
|
|
842
|
-
|
|
843
|
-
|
|
844
|
-
|
|
845
|
-
|
|
846
|
-
|
|
847
|
-
|
|
848
|
-
|
|
701
|
+
import { Model, stream } from "@aryee337/aery-ai";
|
|
702
|
+
|
|
703
|
+
// Example: local Ollama using the OpenAI-compatible API
|
|
704
|
+
const ollamaModel: Model<"openai-completions"> = {
|
|
705
|
+
id: "llama-3.1-8b",
|
|
706
|
+
name: "Llama 3.1 8B (Ollama)",
|
|
707
|
+
api: "openai-completions",
|
|
708
|
+
provider: "ollama",
|
|
709
|
+
baseUrl: "http://localhost:11434/v1",
|
|
710
|
+
reasoning: false,
|
|
711
|
+
input: ["text"],
|
|
712
|
+
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
|
|
713
|
+
contextWindow: 128000,
|
|
714
|
+
maxTokens: 32000,
|
|
849
715
|
};
|
|
850
716
|
|
|
851
|
-
|
|
852
|
-
|
|
853
|
-
|
|
854
|
-
name: 'GPT-4o (via LiteLLM)',
|
|
855
|
-
api: 'openai-completions',
|
|
856
|
-
provider: 'litellm',
|
|
857
|
-
baseUrl: 'http://localhost:4000/v1',
|
|
858
|
-
reasoning: false,
|
|
859
|
-
input: ['text', 'image'],
|
|
860
|
-
cost: { input: 2.5, output: 10, cacheRead: 0, cacheWrite: 0 },
|
|
861
|
-
contextWindow: 128000,
|
|
862
|
-
maxTokens: 16384,
|
|
863
|
-
compat: {
|
|
864
|
-
supportsStore: false, // LiteLLM doesn't support the store field
|
|
865
|
-
}
|
|
866
|
-
};
|
|
717
|
+
const localResponse = await stream(ollamaModel, context, {
|
|
718
|
+
apiKey: process.env.OLLAMA_API_KEY, // Optional; local Ollama usually runs without auth
|
|
719
|
+
});
|
|
867
720
|
|
|
868
|
-
// Example:
|
|
869
|
-
const
|
|
870
|
-
|
|
871
|
-
|
|
872
|
-
|
|
873
|
-
|
|
874
|
-
|
|
875
|
-
|
|
876
|
-
|
|
877
|
-
|
|
878
|
-
|
|
879
|
-
|
|
880
|
-
headers: {
|
|
881
|
-
'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36',
|
|
882
|
-
'X-Custom-Auth': 'bearer-token-here'
|
|
883
|
-
}
|
|
721
|
+
// Example: Ollama Cloud using the native /api/chat transport
|
|
722
|
+
const ollamaCloudModel: Model<"ollama-chat"> = {
|
|
723
|
+
id: "gpt-oss:120b",
|
|
724
|
+
name: "GPT OSS 120B (Ollama Cloud)",
|
|
725
|
+
api: "ollama-chat",
|
|
726
|
+
provider: "ollama-cloud",
|
|
727
|
+
baseUrl: "https://ollama.com",
|
|
728
|
+
reasoning: true,
|
|
729
|
+
input: ["text", "image"],
|
|
730
|
+
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
|
|
731
|
+
contextWindow: 262144,
|
|
732
|
+
maxTokens: 8192,
|
|
884
733
|
};
|
|
885
734
|
|
|
886
|
-
|
|
887
|
-
|
|
888
|
-
apiKey: 'dummy' // Ollama doesn't need a real key
|
|
735
|
+
const cloudResponse = await stream(ollamaCloudModel, context, {
|
|
736
|
+
apiKey: process.env.OLLAMA_CLOUD_API_KEY,
|
|
889
737
|
});
|
|
890
|
-
```
|
|
891
738
|
|
|
892
|
-
|
|
893
|
-
|
|
894
|
-
|
|
895
|
-
|
|
896
|
-
|
|
739
|
+
// Example: LiteLLM proxy with explicit compat settings
|
|
740
|
+
const litellmModel: Model<"openai-completions"> = {
|
|
741
|
+
id: "gpt-4o",
|
|
742
|
+
name: "GPT-4o (via LiteLLM)",
|
|
743
|
+
api: "openai-completions",
|
|
744
|
+
provider: "litellm",
|
|
745
|
+
baseUrl: "http://localhost:4000/v1",
|
|
746
|
+
reasoning: false,
|
|
747
|
+
input: ["text", "image"],
|
|
748
|
+
cost: { input: 2.5, output: 10, cacheRead: 0, cacheWrite: 0 },
|
|
749
|
+
contextWindow: 128000,
|
|
750
|
+
maxTokens: 16384,
|
|
751
|
+
compat: {
|
|
752
|
+
supportsStore: false, // LiteLLM doesn't support the store field
|
|
753
|
+
},
|
|
754
|
+
};
|
|
897
755
|
|
|
898
|
-
|
|
899
|
-
const
|
|
900
|
-
|
|
901
|
-
|
|
902
|
-
|
|
903
|
-
|
|
904
|
-
|
|
905
|
-
|
|
906
|
-
|
|
907
|
-
|
|
908
|
-
|
|
909
|
-
|
|
910
|
-
|
|
911
|
-
|
|
912
|
-
|
|
913
|
-
|
|
914
|
-
high: 'high',
|
|
915
|
-
xhigh: null,
|
|
916
|
-
},
|
|
917
|
-
compat: {
|
|
918
|
-
supportsDeveloperRole: false,
|
|
919
|
-
supportsReasoningEffort: false,
|
|
920
|
-
}
|
|
756
|
+
// Example: Custom endpoint with headers (bypassing Cloudflare bot detection)
|
|
757
|
+
const proxyModel: Model<"anthropic-messages"> = {
|
|
758
|
+
id: "claude-sonnet-4",
|
|
759
|
+
name: "Claude Sonnet 4 (Proxied)",
|
|
760
|
+
api: "anthropic-messages",
|
|
761
|
+
provider: "custom-proxy",
|
|
762
|
+
baseUrl: "https://proxy.example.com/v1",
|
|
763
|
+
reasoning: true,
|
|
764
|
+
input: ["text", "image"],
|
|
765
|
+
cost: { input: 3, output: 15, cacheRead: 0.3, cacheWrite: 3.75 },
|
|
766
|
+
contextWindow: 200000,
|
|
767
|
+
maxTokens: 8192,
|
|
768
|
+
headers: {
|
|
769
|
+
"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36",
|
|
770
|
+
"X-Custom-Auth": "bearer-token-here",
|
|
771
|
+
},
|
|
921
772
|
};
|
|
922
773
|
```
|
|
923
774
|
|
|
924
775
|
### OpenAI Compatibility Settings
|
|
925
776
|
|
|
926
|
-
The `openai-completions` API is implemented by many providers with minor differences. By default, the library auto-detects compatibility settings based on `baseUrl` for
|
|
777
|
+
The `openai-completions` API is implemented by many providers with minor differences. By default, the library auto-detects compatibility settings based on `baseUrl` for known providers (Cerebras, xAI, Mistral, Chutes, etc.). For custom proxies or unknown endpoints, you can override these settings via the `compat` field:
|
|
927
778
|
|
|
928
779
|
```typescript
|
|
929
|
-
interface
|
|
930
|
-
|
|
931
|
-
|
|
932
|
-
|
|
933
|
-
|
|
934
|
-
|
|
935
|
-
sendSessionAffinityHeaders?: boolean; // Whether to send `session_id`, `x-client-request-id`, and `x-session-affinity` from `sessionId` when caching is enabled (default: false)
|
|
936
|
-
maxTokensField?: 'max_completion_tokens' | 'max_tokens'; // Which field name to use (default: max_completion_tokens)
|
|
937
|
-
requiresToolResultName?: boolean; // Whether tool results require the `name` field (default: false)
|
|
938
|
-
requiresAssistantAfterToolResult?: boolean; // Whether tool results must be followed by an assistant message (default: false)
|
|
939
|
-
requiresThinkingAsText?: boolean; // Whether thinking blocks must be converted to text (default: false)
|
|
940
|
-
requiresReasoningContentOnAssistantMessages?: boolean; // Whether all replayed assistant messages must include empty reasoning_content when reasoning is enabled (default: auto-detected for DeepSeek)
|
|
941
|
-
thinkingFormat?: 'openai' | 'openrouter' | 'deepseek' | 'together' | 'zai' | 'qwen' | 'qwen-chat-template'; // Format for reasoning param: 'openai' uses reasoning_effort, 'openrouter' uses reasoning: { effort }, 'deepseek' uses thinking: { type } plus reasoning_effort, 'together' uses reasoning: { enabled } plus reasoning_effort when supported, 'zai' uses enable_thinking, 'qwen' uses enable_thinking, 'qwen-chat-template' uses chat_template_kwargs.enable_thinking (default: openai)
|
|
942
|
-
cacheControlFormat?: 'anthropic'; // Anthropic-style cache_control on system prompt, last tool, and last user/assistant text content
|
|
943
|
-
openRouterRouting?: OpenRouterRouting; // OpenRouter routing preferences (default: {})
|
|
944
|
-
vercelGatewayRouting?: VercelGatewayRouting; // Vercel AI Gateway routing preferences (default: {})
|
|
945
|
-
}
|
|
946
|
-
|
|
947
|
-
interface OpenAIResponsesCompat {
|
|
948
|
-
// Reserved for future use
|
|
780
|
+
interface OpenAICompat {
|
|
781
|
+
supportsStore?: boolean; // Whether provider supports the `store` field (default: true)
|
|
782
|
+
supportsDeveloperRole?: boolean; // Whether provider supports `developer` role vs `system` (default: true)
|
|
783
|
+
supportsReasoningEffort?: boolean; // Whether provider supports `reasoning_effort` (default: true)
|
|
784
|
+
maxTokensField?: "max_completion_tokens" | "max_tokens"; // Which field name to use (default: max_completion_tokens)
|
|
785
|
+
extraBody?: Record<string, unknown>; // Extra request-body fields for custom proxy routing or provider-specific options
|
|
949
786
|
}
|
|
950
787
|
```
|
|
951
788
|
|
|
@@ -957,20 +794,18 @@ If `compat` is not set, the library falls back to URL-based detection. If `compa
|
|
|
957
794
|
|
|
958
795
|
### Type Safety
|
|
959
796
|
|
|
960
|
-
Models are typed by their API,
|
|
797
|
+
Models are typed by their API, ensuring type-safe options:
|
|
961
798
|
|
|
962
799
|
```typescript
|
|
963
|
-
import { streamAnthropic, type AnthropicOptions } from '@aryee337/aery-ai';
|
|
964
|
-
|
|
965
800
|
// TypeScript knows this is an Anthropic model
|
|
966
|
-
const claude = getModel(
|
|
967
|
-
|
|
968
|
-
const options: AnthropicOptions = {
|
|
969
|
-
thinkingEnabled: true,
|
|
970
|
-
thinkingBudgetTokens: 2048
|
|
971
|
-
};
|
|
801
|
+
const claude = getModel("anthropic", "claude-sonnet-4-20250514");
|
|
972
802
|
|
|
973
|
-
|
|
803
|
+
// So these options are type-checked for AnthropicOptions
|
|
804
|
+
await stream(claude, context, {
|
|
805
|
+
thinkingEnabled: true, // ✓ Valid for anthropic-messages
|
|
806
|
+
thinkingBudgetTokens: 2048, // ✓ Valid for anthropic-messages
|
|
807
|
+
// reasoningEffort: 'high' // ✗ TypeScript error: not valid for anthropic-messages
|
|
808
|
+
});
|
|
974
809
|
```
|
|
975
810
|
|
|
976
811
|
## Cross-Provider Handoffs
|
|
@@ -989,41 +824,43 @@ When messages from one provider are sent to a different provider, the library au
|
|
|
989
824
|
### Example: Multi-Provider Conversation
|
|
990
825
|
|
|
991
826
|
```typescript
|
|
992
|
-
import { getModel, complete, Context } from
|
|
827
|
+
import { getModel, complete, Context } from "@aryee337/aery-ai";
|
|
993
828
|
|
|
994
829
|
// Start with Claude
|
|
995
|
-
const claude = getModel(
|
|
830
|
+
const claude = getModel("anthropic", "claude-sonnet-4-20250514");
|
|
996
831
|
const context: Context = {
|
|
997
|
-
|
|
832
|
+
messages: [],
|
|
998
833
|
};
|
|
999
834
|
|
|
1000
|
-
context.messages.push({ role:
|
|
835
|
+
context.messages.push({ role: "user", content: "What is 25 * 18?" });
|
|
1001
836
|
const claudeResponse = await complete(claude, context, {
|
|
1002
|
-
|
|
837
|
+
thinkingEnabled: true,
|
|
1003
838
|
});
|
|
1004
839
|
context.messages.push(claudeResponse);
|
|
1005
840
|
|
|
1006
841
|
// Switch to GPT-5 - it will see Claude's thinking as <thinking> tagged text
|
|
1007
|
-
const gpt5 = getModel(
|
|
1008
|
-
context.messages.push({ role:
|
|
842
|
+
const gpt5 = getModel("openai", "gpt-5-mini");
|
|
843
|
+
context.messages.push({ role: "user", content: "Is that calculation correct?" });
|
|
1009
844
|
const gptResponse = await complete(gpt5, context);
|
|
1010
845
|
context.messages.push(gptResponse);
|
|
1011
846
|
|
|
1012
847
|
// Switch to Gemini
|
|
1013
|
-
const gemini = getModel(
|
|
1014
|
-
context.messages.push({ role:
|
|
848
|
+
const gemini = getModel("google", "gemini-2.5-flash");
|
|
849
|
+
context.messages.push({ role: "user", content: "What was the original question?" });
|
|
1015
850
|
const geminiResponse = await complete(gemini, context);
|
|
1016
851
|
```
|
|
1017
852
|
|
|
1018
853
|
### Provider Compatibility
|
|
1019
854
|
|
|
1020
855
|
All providers can handle messages from other providers, including:
|
|
856
|
+
|
|
1021
857
|
- Text content
|
|
1022
858
|
- Tool calls and tool results (including images in tool results)
|
|
1023
859
|
- Thinking/reasoning blocks (transformed to tagged text for cross-provider compatibility)
|
|
1024
860
|
- Aborted messages with partial content
|
|
1025
861
|
|
|
1026
862
|
This enables flexible workflows where you can:
|
|
863
|
+
|
|
1027
864
|
- Start with a fast model for initial responses
|
|
1028
865
|
- Switch to a more capable model for complex reasoning
|
|
1029
866
|
- Use specialized models for specific tasks
|
|
@@ -1034,33 +871,31 @@ This enables flexible workflows where you can:
|
|
|
1034
871
|
The `Context` object can be easily serialized and deserialized using standard JSON methods, making it simple to persist conversations, implement chat history, or transfer contexts between services:
|
|
1035
872
|
|
|
1036
873
|
```typescript
|
|
1037
|
-
import { Context, getModel, complete } from
|
|
874
|
+
import { Context, getModel, complete } from "@aryee337/aery-ai";
|
|
1038
875
|
|
|
1039
876
|
// Create and use a context
|
|
1040
877
|
const context: Context = {
|
|
1041
|
-
|
|
1042
|
-
|
|
1043
|
-
{ role: 'user', content: 'What is TypeScript?' }
|
|
1044
|
-
]
|
|
878
|
+
systemPrompt: ["You are a helpful assistant."],
|
|
879
|
+
messages: [{ role: "user", content: "What is TypeScript?" }],
|
|
1045
880
|
};
|
|
1046
881
|
|
|
1047
|
-
const model = getModel(
|
|
882
|
+
const model = getModel("openai", "gpt-4o-mini");
|
|
1048
883
|
const response = await complete(model, context);
|
|
1049
884
|
context.messages.push(response);
|
|
1050
885
|
|
|
1051
886
|
// Serialize the entire context
|
|
1052
887
|
const serialized = JSON.stringify(context);
|
|
1053
|
-
console.log(
|
|
888
|
+
console.log("Serialized context size:", serialized.length, "bytes");
|
|
1054
889
|
|
|
1055
890
|
// Save to database, localStorage, file, etc.
|
|
1056
|
-
localStorage.setItem(
|
|
891
|
+
localStorage.setItem("conversation", serialized);
|
|
1057
892
|
|
|
1058
893
|
// Later: deserialize and continue the conversation
|
|
1059
|
-
const restored: Context = JSON.parse(localStorage.getItem(
|
|
1060
|
-
restored.messages.push({ role:
|
|
894
|
+
const restored: Context = JSON.parse(localStorage.getItem("conversation")!);
|
|
895
|
+
restored.messages.push({ role: "user", content: "Tell me more about its type system" });
|
|
1061
896
|
|
|
1062
897
|
// Continue with any model
|
|
1063
|
-
const newModel = getModel(
|
|
898
|
+
const newModel = getModel("anthropic", "claude-haiku-4-5-20251001");
|
|
1064
899
|
const continuation = await complete(newModel, restored);
|
|
1065
900
|
```
|
|
1066
901
|
|
|
@@ -1071,100 +906,127 @@ const continuation = await complete(newModel, restored);
|
|
|
1071
906
|
The library supports browser environments. You must pass the API key explicitly since environment variables are not available in browsers:
|
|
1072
907
|
|
|
1073
908
|
```typescript
|
|
1074
|
-
import { getModel, complete } from
|
|
909
|
+
import { getModel, complete } from "@aryee337/aery-ai";
|
|
1075
910
|
|
|
1076
911
|
// API key must be passed explicitly in browser
|
|
1077
|
-
const model = getModel(
|
|
1078
|
-
|
|
1079
|
-
const response = await complete(
|
|
1080
|
-
|
|
1081
|
-
|
|
1082
|
-
|
|
1083
|
-
}
|
|
912
|
+
const model = getModel("anthropic", "claude-haiku-4-5-20251001");
|
|
913
|
+
|
|
914
|
+
const response = await complete(
|
|
915
|
+
model,
|
|
916
|
+
{
|
|
917
|
+
messages: [{ role: "user", content: "Hello!" }],
|
|
918
|
+
},
|
|
919
|
+
{
|
|
920
|
+
apiKey: "your-api-key",
|
|
921
|
+
}
|
|
922
|
+
);
|
|
1084
923
|
```
|
|
1085
924
|
|
|
1086
925
|
> **Security Warning**: Exposing API keys in frontend code is dangerous. Anyone can extract and abuse your keys. Only use this approach for internal tools or demos. For production applications, use a backend proxy that keeps your API keys secure.
|
|
1087
926
|
|
|
1088
|
-
### Browser Compatibility Notes
|
|
1089
|
-
|
|
1090
|
-
- Amazon Bedrock (`bedrock-converse-stream`) is not supported in browser environments.
|
|
1091
|
-
- OAuth login flows are not supported in browser environments. Use the `@aryee337/aery-ai/oauth` entry point in Node.js.
|
|
1092
|
-
- In browser builds, Bedrock can still appear in model lists. Calls to Bedrock models fail at runtime.
|
|
1093
|
-
- Use a server-side proxy or backend service if you need Bedrock or OAuth-based auth from a web app.
|
|
1094
|
-
|
|
1095
927
|
### Environment Variables (Node.js only)
|
|
1096
928
|
|
|
1097
929
|
In Node.js environments, you can set environment variables to avoid passing API keys:
|
|
1098
930
|
|
|
1099
|
-
| Provider
|
|
1100
|
-
|
|
1101
|
-
| OpenAI
|
|
1102
|
-
|
|
|
1103
|
-
|
|
|
1104
|
-
|
|
|
1105
|
-
|
|
|
1106
|
-
|
|
|
1107
|
-
|
|
|
1108
|
-
|
|
|
1109
|
-
|
|
|
1110
|
-
|
|
|
1111
|
-
|
|
|
1112
|
-
|
|
|
1113
|
-
|
|
|
1114
|
-
|
|
|
1115
|
-
|
|
|
1116
|
-
|
|
|
1117
|
-
|
|
|
1118
|
-
|
|
|
1119
|
-
|
|
|
1120
|
-
|
|
|
1121
|
-
|
|
|
1122
|
-
|
|
|
1123
|
-
|
|
|
1124
|
-
| Xiaomi MiMo
|
|
1125
|
-
|
|
|
1126
|
-
|
|
931
|
+
| Provider | Environment Variable(s) |
|
|
932
|
+
| -------------- | ---------------------------------------------------------------------------- |
|
|
933
|
+
| OpenAI | `OPENAI_API_KEY` |
|
|
934
|
+
| Anthropic | `ANTHROPIC_API_KEY` or `ANTHROPIC_OAUTH_TOKEN` (or `ANTHROPIC_FOUNDRY_API_KEY` when `CLAUDE_CODE_USE_FOUNDRY=true`) |
|
|
935
|
+
| Google | `GEMINI_API_KEY` |
|
|
936
|
+
| Vertex AI | `GOOGLE_CLOUD_PROJECT` (or `GCLOUD_PROJECT`) + `GOOGLE_CLOUD_LOCATION` + ADC |
|
|
937
|
+
| Mistral | `MISTRAL_API_KEY` |
|
|
938
|
+
| Groq | `GROQ_API_KEY` |
|
|
939
|
+
| Cerebras | `CEREBRAS_API_KEY` |
|
|
940
|
+
| Together | `TOGETHER_API_KEY` |
|
|
941
|
+
| Qianfan | `QIANFAN_API_KEY` |
|
|
942
|
+
| Hugging Face | `HUGGINGFACE_HUB_TOKEN` or `HF_TOKEN` |
|
|
943
|
+
| Synthetic | `SYNTHETIC_API_KEY` |
|
|
944
|
+
| NVIDIA | `NVIDIA_API_KEY` |
|
|
945
|
+
| NanoGPT | `NANO_GPT_API_KEY` |
|
|
946
|
+
| Venice | `VENICE_API_KEY` |
|
|
947
|
+
| Moonshot | `MOONSHOT_API_KEY` |
|
|
948
|
+
| xAI | `XAI_API_KEY` |
|
|
949
|
+
| OpenRouter | `OPENROUTER_API_KEY` |
|
|
950
|
+
| LiteLLM | `LITELLM_API_KEY` |
|
|
951
|
+
| Ollama | `OLLAMA_API_KEY` (optional for local deployments) |
|
|
952
|
+
| Ollama Cloud | `OLLAMA_CLOUD_API_KEY` |
|
|
953
|
+
| Qwen Portal | `QWEN_OAUTH_TOKEN` or `QWEN_PORTAL_API_KEY` |
|
|
954
|
+
| zAI | `ZAI_API_KEY` |
|
|
955
|
+
| MiniMax Code | `MINIMAX_CODE_API_KEY` (international) or `MINIMAX_CODE_CN_API_KEY` (China) |
|
|
956
|
+
| Xiaomi MiMo | `XIAOMI_API_KEY` |
|
|
957
|
+
| ZenMux | `ZENMUX_API_KEY` |
|
|
958
|
+
| vLLM | `VLLM_API_KEY` |
|
|
959
|
+
| Cloudflare AI Gateway | `CLOUDFLARE_AI_GATEWAY_API_KEY` |
|
|
960
|
+
| GitHub Copilot | `COPILOT_GITHUB_TOKEN` or `GH_TOKEN` or `GITHUB_TOKEN` |
|
|
961
|
+
|
|
962
|
+
For Cloudflare AI Gateway models, use provider base URL format
|
|
963
|
+
`https://gateway.ai.cloudflare.com/v1/<account>/<gateway>/anthropic`.
|
|
964
|
+
|
|
965
|
+
For Anthropic Foundry routing, set `CLAUDE_CODE_USE_FOUNDRY=true` plus:
|
|
966
|
+
`FOUNDRY_BASE_URL`, `ANTHROPIC_FOUNDRY_API_KEY`, optional `ANTHROPIC_CUSTOM_HEADERS`,
|
|
967
|
+
and optional mTLS material (`CLAUDE_CODE_CLIENT_CERT`, `CLAUDE_CODE_CLIENT_KEY`, `NODE_EXTRA_CA_CERTS`).
|
|
968
|
+
|
|
969
|
+
Provider endpoint defaults for the current OpenAI-compatible integrations:
|
|
970
|
+
|
|
971
|
+
- Together: `https://api.together.xyz/v1`
|
|
972
|
+
- Moonshot: `https://api.moonshot.ai/v1`
|
|
973
|
+
- Qianfan: `https://qianfan.baidubce.com/v2`
|
|
974
|
+
- NVIDIA: `https://integrate.api.nvidia.com/v1`
|
|
975
|
+
- NanoGPT: `https://nano-gpt.com/api/v1`
|
|
976
|
+
- Hugging Face Inference: `https://router.huggingface.co/v1`
|
|
977
|
+
- Venice: `https://api.venice.ai/api/v1`
|
|
978
|
+
- Xiaomi MiMo: `https://api.xiaomimimo.com/anthropic`
|
|
979
|
+
- ZenMux (OpenAI): `https://zenmux.ai/api/v1`
|
|
980
|
+
- ZenMux (Anthropic models): `https://zenmux.ai/api/anthropic`
|
|
981
|
+
- vLLM: `http://127.0.0.1:8000/v1`
|
|
982
|
+
- Ollama: local OpenAI-compatible runtime (`http://127.0.0.1:11434/v1`)
|
|
983
|
+
- Ollama Cloud: native Ollama API host (`https://ollama.com/api`, configured here as base URL `https://ollama.com`)
|
|
984
|
+
- LiteLLM: `http://localhost:4000/v1`
|
|
985
|
+
- Cloudflare AI Gateway: `https://gateway.ai.cloudflare.com/v1/<account>/<gateway>/anthropic`
|
|
986
|
+
- Qwen Portal: `https://portal.qwen.ai/v1`
|
|
1127
987
|
When set, the library automatically uses these keys:
|
|
1128
988
|
|
|
1129
989
|
```typescript
|
|
1130
990
|
// Uses OPENAI_API_KEY from environment
|
|
1131
|
-
const model = getModel(
|
|
991
|
+
const model = getModel("openai", "gpt-4o-mini");
|
|
1132
992
|
const response = await complete(model, context);
|
|
1133
993
|
|
|
1134
994
|
// Or override with explicit key
|
|
1135
995
|
const response = await complete(model, context, {
|
|
1136
|
-
|
|
996
|
+
apiKey: "sk-different-key",
|
|
1137
997
|
});
|
|
1138
998
|
```
|
|
1139
999
|
|
|
1140
1000
|
### Checking Environment Variables
|
|
1141
1001
|
|
|
1142
1002
|
```typescript
|
|
1143
|
-
import { getEnvApiKey } from
|
|
1003
|
+
import { getEnvApiKey } from "@aryee337/aery-ai";
|
|
1144
1004
|
|
|
1145
1005
|
// Check if an API key is set in environment variables
|
|
1146
|
-
const key = getEnvApiKey(
|
|
1006
|
+
const key = getEnvApiKey("openai"); // checks OPENAI_API_KEY
|
|
1147
1007
|
```
|
|
1148
1008
|
|
|
1149
1009
|
## OAuth Providers
|
|
1150
1010
|
|
|
1151
|
-
Several providers
|
|
1011
|
+
Several providers support OAuth authentication (some also support static API keys):
|
|
1152
1012
|
|
|
1153
1013
|
- **Anthropic** (Claude Pro/Max subscription)
|
|
1154
1014
|
- **OpenAI Codex** (ChatGPT Plus/Pro subscription, access to GPT-5.x Codex models)
|
|
1155
1015
|
- **GitHub Copilot** (Copilot subscription)
|
|
1016
|
+
- **Google Gemini CLI** (Gemini 2.0/2.5 via Google Cloud Code Assist; free tier or paid subscription)
|
|
1017
|
+
- **Antigravity** (Free Gemini 3, Claude, GPT-OSS via Google Cloud)
|
|
1018
|
+
- **Qwen Portal** (Qwen OAuth token or API key)
|
|
1156
1019
|
|
|
1157
1020
|
For paid Cloud Code Assist subscriptions, set `GOOGLE_CLOUD_PROJECT` or `GOOGLE_CLOUD_PROJECT_ID` to your project ID.
|
|
1158
1021
|
|
|
1159
|
-
### Vertex AI
|
|
1022
|
+
### Vertex AI (ADC)
|
|
1160
1023
|
|
|
1161
|
-
Vertex AI models
|
|
1024
|
+
Vertex AI models use Application Default Credentials (ADC):
|
|
1162
1025
|
|
|
1163
|
-
- **
|
|
1164
|
-
- **
|
|
1165
|
-
- **CI/Production (ADC)**: Set `GOOGLE_APPLICATION_CREDENTIALS` to point to a service account JSON key file
|
|
1026
|
+
- **Local development**: Run `gcloud auth application-default login`
|
|
1027
|
+
- **CI/Production**: Set `GOOGLE_APPLICATION_CREDENTIALS` to point to a service account JSON key file
|
|
1166
1028
|
|
|
1167
|
-
|
|
1029
|
+
Also set `GOOGLE_CLOUD_PROJECT` (or `GCLOUD_PROJECT`) and `GOOGLE_CLOUD_LOCATION`. You can also pass `project`/`location` in the call options.
|
|
1168
1030
|
|
|
1169
1031
|
Example:
|
|
1170
1032
|
|
|
@@ -1179,19 +1041,17 @@ export GOOGLE_APPLICATION_CREDENTIALS="/path/to/service-account.json"
|
|
|
1179
1041
|
```
|
|
1180
1042
|
|
|
1181
1043
|
```typescript
|
|
1182
|
-
import { getModel, complete } from
|
|
1044
|
+
import { getModel, complete } from "@aryee337/aery-ai";
|
|
1183
1045
|
|
|
1184
1046
|
(async () => {
|
|
1185
|
-
|
|
1186
|
-
|
|
1187
|
-
|
|
1188
|
-
|
|
1189
|
-
|
|
1190
|
-
|
|
1191
|
-
|
|
1192
|
-
|
|
1193
|
-
if (block.type === 'text') console.log(block.text);
|
|
1194
|
-
}
|
|
1047
|
+
const model = getModel("google-vertex", "gemini-2.5-flash");
|
|
1048
|
+
const response = await complete(model, {
|
|
1049
|
+
messages: [{ role: "user", content: "Hello from Vertex AI" }],
|
|
1050
|
+
});
|
|
1051
|
+
|
|
1052
|
+
for (const block of response.content) {
|
|
1053
|
+
if (block.type === "text") console.log(block.text);
|
|
1054
|
+
}
|
|
1195
1055
|
})().catch(console.error);
|
|
1196
1056
|
```
|
|
1197
1057
|
|
|
@@ -1199,58 +1059,86 @@ Official docs: [Application Default Credentials](https://cloud.google.com/docs/a
|
|
|
1199
1059
|
|
|
1200
1060
|
### CLI Login
|
|
1201
1061
|
|
|
1202
|
-
|
|
1062
|
+
Authenticate via the [`aery`](https://aery.dev) coding-agent CLI, which drives this library's OAuth/API-key flows in-process and persists into `agent.db`:
|
|
1203
1063
|
|
|
1204
1064
|
```bash
|
|
1205
|
-
|
|
1206
|
-
|
|
1207
|
-
|
|
1065
|
+
aery auth-broker login # interactive provider selection
|
|
1066
|
+
aery auth-broker login anthropic # login to a specific provider
|
|
1067
|
+
aery auth-broker login vllm # store vLLM API key (or placeholder for local no-auth)
|
|
1068
|
+
aery auth-broker list # list supported providers
|
|
1069
|
+
aery auth-broker logout # interactive — pick a stored credential to remove
|
|
1208
1070
|
```
|
|
1209
1071
|
|
|
1210
|
-
Credentials are saved to `
|
|
1072
|
+
Credentials are saved to `agent.db` in the agent directory. `/login qianfan` opens the Qianfan console and stores the pasted API key.
|
|
1073
|
+
|
|
1074
|
+
`login` supports OAuth providers (Anthropic, OpenAI Codex, GitHub Copilot, Gemini CLI, Antigravity) and API-key onboarding flows.
|
|
1075
|
+
|
|
1076
|
+
For the current API-key onboarding flows, the library covers Together, Moonshot, Qianfan, NVIDIA, NanoGPT, Hugging Face, Venice, Xiaomi, vLLM, LiteLLM, Cloudflare AI Gateway, Qwen Portal, and Ollama Cloud. Ollama remains the local runtime integration; set `OLLAMA_API_KEY` only when your local or self-hosted deployment enforces bearer auth.
|
|
1211
1077
|
|
|
1212
1078
|
### Programmatic OAuth
|
|
1213
1079
|
|
|
1214
|
-
The library provides login and token refresh functions
|
|
1080
|
+
The library provides login and token refresh functions. Credential storage is the caller's responsibility.
|
|
1215
1081
|
|
|
1216
1082
|
```typescript
|
|
1217
1083
|
import {
|
|
1218
|
-
|
|
1219
|
-
|
|
1220
|
-
|
|
1221
|
-
|
|
1222
|
-
|
|
1223
|
-
|
|
1224
|
-
|
|
1225
|
-
|
|
1226
|
-
|
|
1227
|
-
|
|
1228
|
-
|
|
1229
|
-
|
|
1230
|
-
|
|
1231
|
-
|
|
1084
|
+
// Login functions (return credentials, do not store)
|
|
1085
|
+
loginAnthropic,
|
|
1086
|
+
loginOpenAICodex,
|
|
1087
|
+
loginGitHubCopilot,
|
|
1088
|
+
loginGeminiCli,
|
|
1089
|
+
loginAntigravity,
|
|
1090
|
+
loginCloudflareAiGateway,
|
|
1091
|
+
loginHuggingface,
|
|
1092
|
+
loginLiteLLM,
|
|
1093
|
+
loginMoonshot,
|
|
1094
|
+
loginNvidia,
|
|
1095
|
+
loginNanoGPT,
|
|
1096
|
+
loginQianfan,
|
|
1097
|
+
loginQwenPortal,
|
|
1098
|
+
loginTogether,
|
|
1099
|
+
loginVenice,
|
|
1100
|
+
loginVllm,
|
|
1101
|
+
loginXiaomi,
|
|
1102
|
+
|
|
1103
|
+
// Token management
|
|
1104
|
+
refreshOAuthToken, // (provider, credentials) => new credentials
|
|
1105
|
+
getOAuthApiKey, // (provider, credentialsMap) => { newCredentials, apiKey } | null
|
|
1106
|
+
|
|
1107
|
+
// Types
|
|
1108
|
+
type OAuthProvider, // includes 'anthropic', 'openai-codex', 'github-copilot', 'google-gemini-cli', 'google-antigravity', 'together', 'moonshot', 'qianfan', 'nvidia', 'nanogpt', 'huggingface', 'venice', 'xiaomi', 'vllm', 'litellm', 'cloudflare-ai-gateway', 'qwen-portal', ...
|
|
1109
|
+
type OAuthCredentials,
|
|
1110
|
+
} from "@aryee337/aery-ai";
|
|
1111
|
+
```
|
|
1112
|
+
|
|
1113
|
+
`loginOpenAICodex` accepts an optional `originator` value used in the OAuth flow:
|
|
1114
|
+
|
|
1115
|
+
```typescript
|
|
1116
|
+
await loginOpenAICodex({
|
|
1117
|
+
onAuth: ({ url }) => console.log(url),
|
|
1118
|
+
originator: "my-cli",
|
|
1119
|
+
});
|
|
1232
1120
|
```
|
|
1233
1121
|
|
|
1234
1122
|
### Login Flow Example
|
|
1235
1123
|
|
|
1236
1124
|
```typescript
|
|
1237
|
-
import { loginGitHubCopilot } from
|
|
1238
|
-
import
|
|
1125
|
+
import { loginGitHubCopilot } from "@aryee337/aery-ai";
|
|
1126
|
+
import * as fs from "node:fs";
|
|
1239
1127
|
|
|
1240
1128
|
const credentials = await loginGitHubCopilot({
|
|
1241
|
-
|
|
1242
|
-
|
|
1243
|
-
|
|
1244
|
-
|
|
1245
|
-
|
|
1246
|
-
|
|
1247
|
-
|
|
1248
|
-
|
|
1129
|
+
onAuth: (url, instructions) => {
|
|
1130
|
+
console.log(`Open: ${url}`);
|
|
1131
|
+
if (instructions) console.log(instructions);
|
|
1132
|
+
},
|
|
1133
|
+
onPrompt: async (prompt) => {
|
|
1134
|
+
return await getUserInput(prompt.message);
|
|
1135
|
+
},
|
|
1136
|
+
onProgress: (message) => console.log(message),
|
|
1249
1137
|
});
|
|
1250
1138
|
|
|
1251
1139
|
// Store credentials yourself
|
|
1252
|
-
const auth = {
|
|
1253
|
-
writeFileSync(
|
|
1140
|
+
const auth = { "github-copilot": { type: "oauth", ...credentials } };
|
|
1141
|
+
fs.writeFileSync("credentials.json", JSON.stringify(auth, null, 2));
|
|
1254
1142
|
```
|
|
1255
1143
|
|
|
1256
1144
|
### Using OAuth Tokens
|
|
@@ -1258,125 +1146,38 @@ writeFileSync('auth.json', JSON.stringify(auth, null, 2));
|
|
|
1258
1146
|
Use `getOAuthApiKey()` to get an API key, automatically refreshing if expired:
|
|
1259
1147
|
|
|
1260
1148
|
```typescript
|
|
1261
|
-
import { getModel, complete } from
|
|
1262
|
-
import
|
|
1263
|
-
import { readFileSync, writeFileSync } from 'fs';
|
|
1149
|
+
import { getModel, complete, getOAuthApiKey } from "@aryee337/aery-ai";
|
|
1150
|
+
import * as fs from "node:fs";
|
|
1264
1151
|
|
|
1265
1152
|
// Load your stored credentials
|
|
1266
|
-
const auth = JSON.parse(readFileSync(
|
|
1153
|
+
const auth = JSON.parse(fs.readFileSync("credentials.json", "utf-8"));
|
|
1267
1154
|
|
|
1268
1155
|
// Get API key (refreshes if expired)
|
|
1269
|
-
const result = await getOAuthApiKey(
|
|
1270
|
-
if (!result) throw new Error(
|
|
1156
|
+
const result = await getOAuthApiKey("github-copilot", auth);
|
|
1157
|
+
if (!result) throw new Error("Not logged in");
|
|
1271
1158
|
|
|
1272
1159
|
// Save refreshed credentials
|
|
1273
|
-
auth[
|
|
1274
|
-
writeFileSync(
|
|
1160
|
+
auth["github-copilot"] = { type: "oauth", ...result.newCredentials };
|
|
1161
|
+
fs.writeFileSync("credentials.json", JSON.stringify(auth, null, 2));
|
|
1275
1162
|
|
|
1276
1163
|
// Use the API key
|
|
1277
|
-
const model = getModel(
|
|
1278
|
-
const response = await complete(
|
|
1279
|
-
|
|
1280
|
-
|
|
1164
|
+
const model = getModel("github-copilot", "gpt-4o");
|
|
1165
|
+
const response = await complete(
|
|
1166
|
+
model,
|
|
1167
|
+
{
|
|
1168
|
+
messages: [{ role: "user", content: "Hello!" }],
|
|
1169
|
+
},
|
|
1170
|
+
{ apiKey: result.apiKey }
|
|
1171
|
+
);
|
|
1281
1172
|
```
|
|
1282
1173
|
|
|
1283
1174
|
### Provider Notes
|
|
1284
1175
|
|
|
1285
|
-
**OpenAI Codex**: Requires a ChatGPT Plus or Pro subscription. Provides access to GPT-5.x Codex models with extended context windows and reasoning capabilities. The library automatically handles session-based prompt caching when `sessionId` is provided in stream options.
|
|
1286
|
-
|
|
1287
|
-
**Azure OpenAI (Responses)**: Uses the Responses API only. Set `AZURE_OPENAI_API_KEY` and either `AZURE_OPENAI_BASE_URL` or `AZURE_OPENAI_RESOURCE_NAME`. `AZURE_OPENAI_BASE_URL` supports both `https://<resource>.openai.azure.com` and `https://<resource>.cognitiveservices.azure.com`; root endpoints are normalized to `.../openai/v1` automatically. Use `AZURE_OPENAI_AAERY_VERSION` (defaults to `v1`) to override the API version if needed. Deployment names are treated as model IDs by default, override with `azureDeploymentName` or `AZURE_OPENAI_DEPLOYMENT_NAME_MAP` using comma-separated `model-id=deployment` pairs (for example `gpt-4o-mini=my-deployment,gpt-4o=prod`). Legacy deployment-based URLs are intentionally unsupported.
|
|
1176
|
+
**OpenAI Codex**: Requires a ChatGPT Plus or Pro subscription. Provides access to GPT-5.x Codex models with extended context windows and reasoning capabilities. The library automatically handles session-based prompt caching when `sessionId` is provided in stream options.
|
|
1288
1177
|
|
|
1289
1178
|
**GitHub Copilot**: If you get "The requested model is not supported" error, enable the model manually in VS Code: open Copilot Chat, click the model selector, select the model (warning icon), and click "Enable".
|
|
1290
1179
|
|
|
1291
|
-
|
|
1292
|
-
|
|
1293
|
-
### Adding a New Provider
|
|
1294
|
-
|
|
1295
|
-
Adding a new LLM provider requires changes across multiple files. This checklist covers all necessary steps:
|
|
1296
|
-
|
|
1297
|
-
#### 1. Core Types (`src/types.ts`)
|
|
1298
|
-
|
|
1299
|
-
- Add the API identifier to `KnownApi` (for example `"bedrock-converse-stream"`)
|
|
1300
|
-
- Create an options interface extending `StreamOptions` (for example `BedrockOptions`)
|
|
1301
|
-
- Add the provider name to `KnownProvider` (for example `"amazon-bedrock"`)
|
|
1302
|
-
|
|
1303
|
-
#### 2. Provider Implementation (`src/providers/`)
|
|
1304
|
-
|
|
1305
|
-
Create a new provider file (for example `amazon-bedrock.ts`) that exports:
|
|
1306
|
-
|
|
1307
|
-
- `stream<Provider>()` function returning `AssistantMessageEventStream`
|
|
1308
|
-
- `streamSimple<Provider>()` for `SimpleStreamOptions` mapping
|
|
1309
|
-
- Provider-specific options interface
|
|
1310
|
-
- Message conversion functions to transform `Context` to provider format
|
|
1311
|
-
- Tool conversion if the provider supports tools
|
|
1312
|
-
- Response parsing to emit standardized events (`text`, `tool_call`, `thinking`, `usage`, `stop`)
|
|
1313
|
-
|
|
1314
|
-
#### 3. API Registry Integration (`src/providers/register-builtins.ts`)
|
|
1315
|
-
|
|
1316
|
-
- Register the API with `registerApiProvider()`
|
|
1317
|
-
- Add a package subpath export in `package.json` for the provider module (`./dist/providers/<provider>.js`)
|
|
1318
|
-
- Add lazy loader wrappers in `src/providers/register-builtins.ts`, do not statically import provider implementation modules there
|
|
1319
|
-
- Add any root-level `export type` re-exports in `src/index.ts` that should remain available from `@aryee337/aery-ai`
|
|
1320
|
-
- Add credential detection in `env-api-keys.ts` for the new provider
|
|
1321
|
-
- Ensure `streamSimple` handles auth lookup via `getEnvApiKey()` or provider-specific auth
|
|
1322
|
-
|
|
1323
|
-
#### 4. Model Generation (`scripts/generate-models.ts`, `scripts/generate-image-models.ts`)
|
|
1324
|
-
|
|
1325
|
-
- Add logic to fetch and parse models from the provider's source (e.g., models.dev API)
|
|
1326
|
-
- Map chat/tool-capable provider model data to the standardized `Model` interface via `scripts/generate-models.ts`
|
|
1327
|
-
- Map image-generation provider model data to the standardized `ImagesModel` interface via `scripts/generate-image-models.ts`
|
|
1328
|
-
- Handle provider-specific quirks (pricing format, capability flags, model ID transformations)
|
|
1329
|
-
|
|
1330
|
-
#### 5. Tests (`test/`)
|
|
1331
|
-
|
|
1332
|
-
Create or update test files to cover the new provider:
|
|
1333
|
-
|
|
1334
|
-
- `stream.test.ts` - Basic streaming and tool use
|
|
1335
|
-
- `tokens.test.ts` - Token usage reporting
|
|
1336
|
-
- `abort.test.ts` - Request cancellation
|
|
1337
|
-
- `empty.test.ts` - Empty message handling
|
|
1338
|
-
- `context-overflow.test.ts` - Context limit errors
|
|
1339
|
-
- `image-limits.test.ts` - Image support (if applicable)
|
|
1340
|
-
- `unicode-surrogate.test.ts` - Unicode handling
|
|
1341
|
-
- `tool-call-without-result.test.ts` - Orphaned tool calls
|
|
1342
|
-
- `image-tool-result.test.ts` - Images in tool results
|
|
1343
|
-
- `total-tokens.test.ts` - Token counting accuracy
|
|
1344
|
-
- `cross-provider-handoff.test.ts` - Cross-provider context replay
|
|
1345
|
-
|
|
1346
|
-
For `cross-provider-handoff.test.ts`, add at least one provider/model pair. If the provider exposes multiple model families (for example GPT and Claude), add at least one pair per family.
|
|
1347
|
-
|
|
1348
|
-
For providers with non-standard auth (AWS, Google Vertex), create a utility like `bedrock-utils.ts` with credential detection helpers.
|
|
1349
|
-
|
|
1350
|
-
#### 6. Coding Agent Integration (`../coding-agent/`)
|
|
1351
|
-
|
|
1352
|
-
Update `src/core/model-resolver.ts`:
|
|
1353
|
-
|
|
1354
|
-
- Add a default model ID for the provider in `DEFAULT_MODELS`
|
|
1355
|
-
|
|
1356
|
-
Update `src/cli/args.ts`:
|
|
1357
|
-
|
|
1358
|
-
- Add environment variable documentation in the help text
|
|
1359
|
-
|
|
1360
|
-
Update `README.md`:
|
|
1361
|
-
|
|
1362
|
-
- Add the provider to the providers section with setup instructions
|
|
1363
|
-
|
|
1364
|
-
#### 7. Documentation
|
|
1365
|
-
|
|
1366
|
-
Update `packages/ai/README.md`:
|
|
1367
|
-
|
|
1368
|
-
- Add to the Supported Providers table
|
|
1369
|
-
- Document any provider-specific options or authentication requirements
|
|
1370
|
-
- Add environment variable to the Environment Variables section
|
|
1371
|
-
|
|
1372
|
-
#### 8. Changelog
|
|
1373
|
-
|
|
1374
|
-
Add an entry to `packages/ai/CHANGELOG.md` under `## [Unreleased]`:
|
|
1375
|
-
|
|
1376
|
-
```markdown
|
|
1377
|
-
### Added
|
|
1378
|
-
- Added support for [Provider Name] provider ([#PR](link) by [@author](link))
|
|
1379
|
-
```
|
|
1180
|
+
**Google Gemini CLI / Antigravity**: These use Google Cloud OAuth. The `apiKey` returned by `getOAuthApiKey()` is a JSON string containing both the token and project ID, which the library handles automatically.
|
|
1380
1181
|
|
|
1381
1182
|
## License
|
|
1382
1183
|
|