zuplo 7.6.10 → 7.7.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (5) hide show
  1. package/docs/ai-gateway/providers.mdx +10 -6
  2. package/docs/ai-gateway/universal-api.mdx +6 -6
  3. package/docs/ai-gateway/vertex-ai.mdx +124 -30
  4. package/docs/policies/ai-gateway-model-filtering-v2-inbound/doc.md +2 -1
  5. package/package.json +5 -5
package/docs/ai-gateway/providers.mdx CHANGED
@@ -37,7 +37,7 @@ The following capabilities are supported across providers:
37
37
  | Mistral | ✅ | ✅ | ❌ | ❌ |
38
38
  | xAI | ✅ | ✅ | ❌ | ❌ |
39
39
  | Bedrock Mantle | ✅ | ❌ | ✅ | ✅ |
40
- | Vertex AI | ✅ | ✅ | ❌ | ❌ |
40
+ | Vertex AI | ✅ | ✅ | ❌ | ✅ |
41
41
  | Zuplo Demo | ✅ | ❌ | ❌ | ❌ |
42
42
  | OpenAI-compatible (Custom) | ✅ | ✅ | ❌ | ❌ |
43
43
 
@@ -97,11 +97,15 @@ Two things make its setup different from every other provider:
97
97
  asks for both.
98
98
 
99
99
  Every Vertex model ID is publisher-qualified, so a model reference has two
100
- slashes: a provider named `vertexai` serves `vertexai/google/gemini-3.7-flash`
101
- and `vertexai/qwen/qwen3-coder-480b-a35b-instruct-maas`. Model availability
102
- varies by location—most Model Garden models are served only from the `global`
103
- location. Vertex also hosts Anthropic's Claude models, but on a different API
104
- surface that the gateway doesn't route to.
100
+ slashes: a provider named `vertexai` serves `vertexai/google/gemini-3.7-flash`,
101
+ `vertexai/qwen/qwen3-coder-480b-a35b-instruct-maas`, and
102
+ `vertexai/anthropic/claude-haiku-4-5@20251001`. Model availability varies by
103
+ location—most Model Garden models are served only from the `global` location,
104
+ and Google's regional endpoints serve Claude Sonnet 4.6 and earlier only, so
105
+ `global` is the location that serves every Claude model. Vertex's Claude models
106
+ serve the native Anthropic Messages API (`/v1/messages`) and nothing else: they
107
+ aren't available on `/v1/chat/completions`, and each one must be enabled in
108
+ Model Garden for your Google Cloud project before it answers.
105
109
 
106
110
  For prerequisites, Google Cloud setup, provider steps, code examples, and
107
111
  troubleshooting, see [Using Vertex AI](./vertex-ai.mdx).
package/docs/ai-gateway/universal-api.mdx CHANGED
@@ -63,9 +63,9 @@ list—so clients that can't set a model still work.
63
63
 
64
64
  ## Supported endpoints
65
65
 
66
- | Endpoint | Notes |
67
- | ---------------------- | --------------------------------------------------------------------------------------------------------------- |
68
- | `/v1/chat/completions` | Chat completions for every provider |
69
- | `/v1/embeddings` | Embeddings for every provider except Anthropic |
70
- | `/v1/responses` | OpenAI Responses API: OpenAI, and [Bedrock Mantle](./bedrock-mantle.mdx) OpenAI-compatible models that serve it |
71
- | `/v1/messages` | Anthropic Messages API: Anthropic, and [Bedrock Mantle](./bedrock-mantle.mdx) Claude models |
66
+ | Endpoint | Notes |
67
+ | ---------------------- | ----------------------------------------------------------------------------------------------------------------------------------- |
68
+ | `/v1/chat/completions` | Chat completions for every provider |
69
+ | `/v1/embeddings` | Embeddings for every provider except Anthropic |
70
+ | `/v1/responses` | OpenAI Responses API: OpenAI, and [Bedrock Mantle](./bedrock-mantle.mdx) OpenAI-compatible models that serve it |
71
+ | `/v1/messages` | Anthropic Messages API: Anthropic, and the Claude models of [Bedrock Mantle](./bedrock-mantle.mdx) and [Vertex AI](./vertex-ai.mdx) |
package/docs/ai-gateway/vertex-ai.mdx CHANGED
@@ -9,16 +9,19 @@ description:
9
9
  ---
10
10
 
11
11
  **Vertex AI** is Google Cloud's managed model platform. Adding it as a provider
12
- serves both the Gemini family and Vertex's Model Garden partner models—DeepSeek,
13
- Qwen, GLM, Kimi, MiniMax, Gemma, and GPT-OSS—to your [apps](./apps.mdx) through
14
- the [Universal API](./universal-api.mdx), running on your Google Cloud project
15
- and your Google Cloud billing.
12
+ serves the Gemini family, Vertex's Model Garden partner models—DeepSeek, Qwen,
13
+ GLM, Kimi, MiniMax, Gemma, and GPT-OSS—and Anthropic's Claude models to your
14
+ [apps](./apps.mdx) through the [Universal API](./universal-api.mdx), running on
15
+ your Google Cloud project and your Google Cloud billing.
16
16
 
17
17
  Chat completions go to Vertex's **OpenAI-compatible endpoint**, so the gateway
18
18
  forwards OpenAI-shaped requests without translating them. Embeddings work too,
19
19
  though Vertex has no OpenAI-compatible embeddings endpoint—the gateway
20
20
  translates those onto Vertex's own embedding APIs, which is invisible from your
21
- app. What sets this provider apart is how you connect it: the endpoint is
21
+ app. Claude models are different again: Vertex serves them on the native
22
+ **Anthropic Messages API**, so they're available on `/v1/messages` only—see
23
+ [Call Claude models on the Messages API](#call-claude-models-on-the-messages-api).
24
+ What sets this provider apart is how you connect it: the endpoint is
22
25
  per-location, the Google Cloud project is a separate setting, and the credential
23
26
  is a **service account key file**—not an API key.
24
27
 
@@ -53,9 +56,11 @@ provider named `vertexai` serves all of these:
53
56
  - `vertexai/qwen/qwen3-coder-480b-a35b-instruct-maas`
54
57
  - `vertexai/deepseek-ai/deepseek-v3.2-maas`
55
58
  - `vertexai/zai-org/glm-5-maas`
59
+ - `vertexai/anthropic/claude-haiku-4-5@20251001`
56
60
 
57
- Copy model IDs from the model picker rather than typing them—the `-maas` suffix
58
- and the publisher prefix are both part of the ID.
61
+ Copy model IDs from the model picker rather than typing them—the `-maas` suffix,
62
+ the `@` version on the dated Claude models, and the publisher prefix are all
63
+ part of the ID.
59
64
 
60
65
  ## How the gateway calls Vertex AI
61
66
 
@@ -79,6 +84,9 @@ bearer credential.
79
84
  <DiagramNode id="embed" variant="green">
80
85
  Vertex embedding APIs
81
86
  </DiagramNode>
87
+ <DiagramNode id="messages" variant="blue">
88
+ Anthropic Messages API (Claude)
89
+ </DiagramNode>
82
90
  </DiagramGroup>
83
91
  <DiagramEdge from="app" to="gateway" label="Universal API" />
84
92
  <DiagramEdge
@@ -89,6 +97,7 @@ bearer credential.
89
97
  />
90
98
  <DiagramEdge from="gateway" to="compat" label="chat completions" />
91
99
  <DiagramEdge from="gateway" to="embed" label="embeddings" />
100
+ <DiagramEdge from="gateway" to="messages" label="/v1/messages" />
92
101
  </Diagram>
93
102
 
94
103
  The request URL the gateway builds combines both settings you enter:
@@ -98,6 +107,14 @@ https://us-central1-aiplatform.googleapis.com
98
107
  /v1/projects/my-project-123/locations/us-central1/endpoints/openapi/chat/completions
99
108
  ```
100
109
 
110
+ For a Claude model the gateway calls Vertex's Anthropic publisher surface
111
+ instead, with the model in the URL:
112
+
113
+ ```
114
+ https://aiplatform.googleapis.com
115
+ /v1/projects/my-project-123/locations/global/publishers/anthropic/models/claude-haiku-4-5@20251001:rawPredict
116
+ ```
117
+
101
118
  The **location** lives in the hostname, which is why the provider dialog asks
102
119
  for a location instead of an endpoint URL. The **project** doesn't appear in the
103
120
  hostname at all—it appears only in the path—so it can't be derived from the
@@ -107,25 +124,30 @@ send your traffic and your spend to someone else's Google Cloud project.
107
124
 
108
125
  ## Supported endpoints
109
126
 
110
- | Endpoint | Vertex AI models |
111
- | ---------------------- | ---------------- |
112
- | `/v1/chat/completions` | ✅ Forwarded |
113
- | `/v1/embeddings` | ✅ Translated |
114
- | `/v1/responses` | ❌ |
115
- | `/v1/messages` | ❌ |
127
+ | Endpoint | Gemini and Model Garden models | Claude models |
128
+ | ---------------------- | ------------------------------ | ------------- |
129
+ | `/v1/chat/completions` | ✅ Forwarded | ❌ |
130
+ | `/v1/embeddings` | ✅ Translated | ❌ |
131
+ | `/v1/responses` | ❌ | ❌ |
132
+ | `/v1/messages` | ❌ | ✅ Native |
116
133
 
117
- Streaming (`stream: true`) works on chat completions.
134
+ Streaming (`stream: true`) works on chat completions and on Messages.
118
135
 
119
- The gateway rejects `/v1/responses` and `/v1/messages` requests for Vertex
120
- models with a `400` error before any upstream call, because Vertex serves
121
- neither the OpenAI Responses API nor the Anthropic Messages API.
136
+ The gateway's model catalog records which API serves each model, and the gateway
137
+ routes every request accordingly. It rejects a request for a model on an
138
+ endpoint that doesn't serve it with a `400` before any upstream call:
139
+ `/v1/responses` for every Vertex model, `/v1/messages` for Gemini and Model
140
+ Garden models, and `/v1/chat/completions` or `/v1/embeddings` for Claude
141
+ models—that last error names `/v1/messages` as the endpoint to use.
122
142
 
123
- :::note{title="Anthropic models on Vertex aren't supported"}
143
+ :::note{title="Claude models are Messages-only on Vertex"}
124
144
 
125
- Vertex also hosts Anthropic's Claude models, but on a different API surface than
126
- the one this provider uses. The gateway can't route to them. To use Claude
127
- models, add the [Anthropic](./providers.mdx) provider or
128
- [Bedrock Mantle](./bedrock-mantle.mdx).
145
+ Vertex serves Anthropic's Claude models on the native Anthropic Messages API,
146
+ not on its OpenAI-compatible endpoint, so the gateway doesn't translate chat
147
+ completions for them the way it does for Claude models on
148
+ [Bedrock Mantle](./bedrock-mantle.mdx). Use an Anthropic client and
149
+ `/v1/messages`—see
150
+ [Call Claude models on the Messages API](#call-claude-models-on-the-messages-api).
129
151
 
130
152
  :::
131
153
 
@@ -145,7 +167,14 @@ is not available in region 'us-central1'
145
167
  ```
146
168
 
147
169
  Set the provider's **Location** to `global` if you plan to use Model Garden
148
- models, or add a second Vertex provider for a specific region and route to
170
+ models or Claude models. Google's regional endpoints serve Claude Sonnet 4.6 and
171
+ earlier, only in the regions listed on each model's Model Garden card, and only
172
+ where the model is enabled for your project. Claude models newer than Sonnet 4.6
173
+ are served by the `global` endpoint (and by Google's `us` and `eu` multi-region
174
+ endpoints, which the gateway doesn't accept as a location yet). For a Claude
175
+ model in a location that doesn't serve it, or where it isn't enabled for your
176
+ project, Vertex answers a `400` saying the model "is not servable in region".
177
+ Alternatively, add a second Vertex provider for a specific region and route to
149
178
  whichever one serves the model you want.
150
179
 
151
180
  ## Before you begin
@@ -179,6 +208,16 @@ what the provider dialog asks for.
179
208
  1. Enable the **Vertex AI API** on the project, under **APIs & Services →
180
209
  Library**. Confirm billing is enabled for the project as well.
181
210
 
211
+ 1. To use Claude models, enable each one in **Vertex AI → Model Garden**: search
212
+ for "Claude", open the model card, and click **Enable** for the locations
213
+ your providers use—a model enabled for `global` only answers a `400` saying
214
+ it "is not servable in region" from a regional provider. Until a model is
215
+ enabled, Vertex answers every request for it with a `404` saying the model
216
+ "was not found or your project does not have access to it"—the same message
217
+ it gives a misspelled model ID. Some Claude models also require the project
218
+ to turn on data sharing for the `anthropic` publisher; the `403` Vertex
219
+ returns in that case names the setting.
220
+
182
221
  1. Create a service account under **IAM & Admin → Service Accounts**.
183
222
 
184
223
  1. Grant that service account the **Vertex AI User** (`roles/aiplatform.user`)
@@ -335,6 +374,41 @@ than a partial charge:
335
374
  API also accepts—is rejected, because Vertex has no way to represent it. Send
336
375
  the original text.
337
376
 
377
+ ### Call Claude models on the Messages API
378
+
379
+ Claude models serve the native
380
+ [Anthropic Messages API](./universal-api.mdx#supported-endpoints) at
381
+ `/v1/messages`, including streaming and `/v1/messages/count_tokens`. With the
382
+ Anthropic SDK, set `baseURL` to the app's URL _without_ `/v1`—the SDK appends
383
+ `/v1/messages` itself—and pass the app's API key as `authToken`, not `apiKey`:
384
+
385
+ ```ts
386
+ import Anthropic from "@anthropic-ai/sdk";
387
+
388
+ const client = new Anthropic({
389
+ baseURL:
390
+ "https://my-gateway-main-2e18f50.zuplo.app/config_fe0a04972d2848e0a94ae4b8bcd1497e",
391
+ authToken: process.env.ZUPLO_APP_API_KEY, // apiKey would send x-api-key, which the gateway ignores
392
+ });
393
+
394
+ const message = await client.messages.create({
395
+ model: "vertexai/anthropic/claude-haiku-4-5@20251001",
396
+ max_tokens: 1024,
397
+ messages: [{ role: "user", content: "Say hi" }],
398
+ });
399
+ ```
400
+
401
+ The gateway takes care of what Vertex does differently from Anthropic's own API:
402
+ it addresses the model in the request URL and drops it from the body, adds the
403
+ `anthropic_version` field Vertex requires, and authenticates with the access
404
+ token it minted from your service account. Your request body is otherwise
405
+ forwarded as-is, so system prompts, tool use, and streaming work as they do
406
+ against Anthropic directly. Client headers such as `anthropic-beta` pass
407
+ through, narrowed to the values Vertex accepts.
408
+
409
+ Claude models aren't available on `/v1/chat/completions`—a request there returns
410
+ a `400` that points at `/v1/messages`.
411
+
338
412
  ## Troubleshooting
339
413
 
340
414
  **The dialog rejects your credential.** Vertex uses a service account, not an
@@ -359,9 +433,11 @@ permissions, so it's easy to mistake for a credential problem.
359
433
  project than the one configured on the provider. Grant the role on the same
360
434
  project you entered as the **Google Cloud Project ID**.
361
435
 
362
- **A request fails with `400` saying the model isn't available in the region.**
363
- The model isn't served from your provider's location. Most Model Garden models
364
- are global-only—see
436
+ **A request fails with `400` saying the model isn't available in the region, or
437
+ that a Claude model "is not servable in region".** The model isn't served from
438
+ your provider's location, or isn't enabled for that location in your project.
439
+ Most Model Garden models are global-only, and Claude models are served from
440
+ `global` and the regions on each model's card—see
365
441
  [Model availability](#model-availability-depends-on-the-location). Set the
366
442
  provider's **Location** to `global`, or use a provider that points at a location
367
443
  serving that model.
@@ -370,10 +446,28 @@ serving that model.
370
446
  at most 250 inputs per request and text input only. See
371
447
  [Embeddings](#embeddings).
372
448
 
373
- **A `/v1/responses` or `/v1/messages` request fails with `400`.** Vertex's
374
- OpenAI-compatible endpoint serves neither API. Use `/v1/chat/completions`, or
375
- another provider for those endpoints—see
376
- [Supported endpoints](#supported-endpoints).
449
+ **A `/v1/responses` request fails with `400`.** Vertex serves the OpenAI
450
+ Responses API for none of its models. Use `/v1/chat/completions`, or another
451
+ provider—see [Supported endpoints](#supported-endpoints).
452
+
453
+ **A `/v1/messages` request for a Gemini or Model Garden model fails with
454
+ `400`.** Only Claude models serve the Messages API on Vertex. Use
455
+ `/v1/chat/completions` for the others.
456
+
457
+ **A Claude model fails with `400` on `/v1/chat/completions`.** Claude models are
458
+ Messages-only on Vertex. Send the request to `/v1/messages` with an Anthropic
459
+ client—see
460
+ [Call Claude models on the Messages API](#call-claude-models-on-the-messages-api).
461
+
462
+ **A Claude request fails with `404` saying the model "was not found or your
463
+ project does not have access to it".** Either the model isn't enabled in Model
464
+ Garden for your project (see [Set up Google Cloud](#set-up-google-cloud)) or the
465
+ model ID is wrong. Enabling a model applies to every location; a location that
466
+ doesn't serve the model is the separate `400` described above.
467
+
468
+ **A Claude request fails with `403` about data sharing.** Some Claude models
469
+ require the project to turn on data sharing for the `anthropic` publisher. The
470
+ error names the `setPublisherModelConfig` setting to change.
377
471
 
378
472
  ## Next steps
379
473
 
package/docs/policies/ai-gateway-model-filtering-v2-inbound/doc.md CHANGED
@@ -160,7 +160,8 @@ served on the OpenAI API, and `/v1/messages` one served on the Anthropic API. A
160
160
  selection qualifies when its Provider Assignment is backed by that provider
161
161
  type, or when the provider catalog declares the selected model's dialect as that
162
162
  format — Bedrock Mantle serves its `openai`-dialect models on `/v1/responses`
163
- and its `anthropic`-dialect models on `/v1/messages`. Provider Names may be
163
+ and its `anthropic`-dialect models on `/v1/messages`, and Vertex AI serves its
164
+ `anthropic`-dialect Claude models on `/v1/messages`. Provider Names may be
164
165
  custom labels; validation never infers anything from the label or the model
165
166
  name, and a model the catalog does not declare is rejected.
166
167
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "zuplo",
3
- "version": "7.6.10",
3
+ "version": "7.7.0",
4
4
  "type": "module",
5
5
  "description": "The official Zuplo CLI for local development and platform management",
6
6
  "homepage": "https://zuplo.com/docs/cli/overview",
@@ -32,9 +32,9 @@
32
32
  "zuplo": "zuplo.js"
33
33
  },
34
34
  "dependencies": {
35
- "@zuplo/cli": "7.6.10",
36
- "@zuplo/core": "7.6.10",
37
- "@zuplo/runtime": "7.6.10",
38
- "@zuplo/test": "7.6.10"
35
+ "@zuplo/cli": "7.7.0",
36
+ "@zuplo/core": "7.7.0",
37
+ "@zuplo/runtime": "7.7.0",
38
+ "@zuplo/test": "7.7.0"
39
39
  }
40
40
  }