zuplo 7.12.4 → 7.12.5

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (4) hide show
  1. package/docs/articles/api-key-metadata.mdx +380 -0
  2. package/docs/articles/limits.mdx +112 -22
  3. package/docs/articles/streaming-responses-sse.mdx +244 -0
  4. package/package.json +5 -5
package/docs/articles/api-key-metadata.mdx ADDED
@@ -0,0 +1,380 @@
1
+ ---
2
+ title: Working with API Key and Consumer Metadata
3
+ sidebar_label: Consumer Metadata
4
+ description:
5
+ Set, read, and forward consumer metadata on API key requests to gate features
6
+ by plan, route to tenant-specific backends, and pass identity to your upstream
7
+ services.
8
+ ---
9
+
10
+ Consumer metadata is a JSON object (up to 1 KB) attached to each
11
+ [API key consumer](./api-key-management.mdx) that is available at runtime on
12
+ every authenticated request. It is the primary mechanism for passing
13
+ tenant-specific data — such as a plan tier, customer ID, or organization —
14
+ through the Zuplo gateway to your policies and backend.
15
+
16
+ This guide covers how to set, read, and forward metadata, and shows the most
17
+ common patterns for using it in production.
18
+
19
+ ## Prerequisites
20
+
21
+ - A Zuplo project with the
22
+ [API Key Authentication policy](../policies/api-key-inbound.mdx) configured on
23
+ at least one route.
24
+ - At least one API key consumer. See
25
+ [Add API Key Authentication](../getting-started/gateway/add-api-key-auth/index.mdx)
26
+ if you haven't created one yet.
27
+
28
+ ## How metadata flows
29
+
30
+ When a request arrives with a valid API key, the API Key Authentication policy
31
+ validates the key and populates `request.user`:
32
+
33
+ - **`request.user.sub`** — the consumer's name (its unique identifier within the
34
+ bucket).
35
+ - **`request.user.data`** — the consumer's metadata object.
36
+
37
+ Every policy and handler that runs after authentication can read these values.
38
+ Metadata is set on the **consumer**, not on individual keys. All keys belonging
39
+ to the same consumer share the same identity and metadata. For more on the
40
+ consumer/key data model, see [API Keys](../concepts/api-keys.md).
41
+
42
+ ## Metadata vs tags
43
+
44
+ Consumers support two kinds of key-value data. Mixing them up is a common
45
+ mistake:
46
+
47
+ |   | Metadata | Tags |
48
+ | ------------------- | ------------------------------------------------- | --------------------------------------------- |
49
+ | **Available at** | Runtime (`request.user.data`) | Management API only |
50
+ | **Purpose** | Authorization, routing, rate limiting, metering | Filtering, querying, and organizing consumers |
51
+ | **Size limit** | 1 KB JSON object | Up to 5 key-value pairs per consumer |
52
+ | **Sent to runtime** | Yes — on every authenticated request | No — never reaches your policies or handlers |
53
+ | **Set via** | Portal, Developer API, on-login consumer creation | Portal, Developer API |
54
+
55
+ :::caution{title="Tags are never sent to the runtime"}
56
+
57
+ If you store data in `tags` expecting to read it in a policy or handler, it will
58
+ not be there. Tags are for management operations only (for example, querying all
59
+ consumers on a specific plan via the [Developer API](./api-key-api.mdx)). Use
60
+ `metadata` for anything your code needs at request time.
61
+
62
+ :::
63
+
64
+ ## Setting metadata
65
+
66
+ You can set metadata when creating or updating a consumer through three
67
+ channels.
68
+
69
+ ### Zuplo Portal
70
+
71
+ Navigate to **Services → API Keys → Consumers** in your project. When creating
72
+ or editing a consumer, enter a JSON object in the **Metadata** field:
73
+
74
+ ```json
75
+ {
76
+ "plan": "gold",
77
+ "customerId": "cust_123",
78
+ "orgId": "org_456"
79
+ }
80
+ ```
81
+
82
+ ### Developer API
83
+
84
+ Use the [Zuplo Developer API](./api-key-api.mdx) to create or update consumers
85
+ programmatically. Pass the `metadata` field in the request body:
86
+
87
+ ```bash
88
+ curl -X POST "https://dev.zuplo.com/v1/projects/$PROJECT_ID/buckets/$BUCKET_NAME/consumers" \
89
+ -H "Authorization: Bearer $ZUPLO_API_KEY" \
90
+ -H "Content-Type: application/json" \
91
+ -d '{
92
+ "name": "my-consumer",
93
+ "metadata": {
94
+ "plan": "gold",
95
+ "customerId": "cust_123",
96
+ "orgId": "org_456"
97
+ }
98
+ }'
99
+ ```
100
+
101
+ Replace `$PROJECT_ID` with your project's ID (visible in the Portal URL) and
102
+ `$BUCKET_NAME` with your API key bucket name (visible under **Services → API
103
+ Keys** in the Portal). `$ZUPLO_API_KEY` is a
104
+ [Zuplo Developer API key](./api-key-api.mdx).
105
+
106
+ To update metadata on an existing consumer, use a `PATCH` request to the same
107
+ endpoint with the consumer name appended. See the
108
+ [API reference](../api/api-keys-consumers) for the full endpoint documentation.
109
+
110
+ ### On-login consumer creation
111
+
112
+ If you use the [Developer Portal](../dev-portal/overview.mdx) (the self-serve
113
+ portal where your API consumers manage their keys) with an identity provider
114
+ like Auth0, you can create consumers automatically when a user signs in and set
115
+ metadata from the identity token claims. See
116
+ [Create Consumer on Auth](../dev-portal/dev-portal-create-consumer-on-auth.mdx)
117
+ for the setup.
118
+
119
+ ## Reading metadata at runtime
120
+
121
+ After the API Key Authentication policy runs, access metadata in any downstream
122
+ policy or handler through `request.user.data`:
123
+
124
+ ```ts
125
+ import { ZuploContext, ZuploRequest } from "@zuplo/runtime";
126
+
127
+ export default async function handler(
128
+ request: ZuploRequest,
129
+ context: ZuploContext,
130
+ ) {
131
+ const plan = request.user?.data?.plan;
132
+ const customerId = request.user?.data?.customerId;
133
+
134
+ context.log.info("Authenticated request", {
135
+ consumer: request.user?.sub,
136
+ plan,
137
+ customerId,
138
+ });
139
+
140
+ return new Response(`Plan: ${plan}, Customer: ${customerId}`);
141
+ }
142
+ ```
143
+
144
+ For type safety, define an interface for your metadata shape and pass it under
145
+ the `UserData` key in the `ZuploRequest` type parameter. `UserData` is the key
146
+ name the generic expects; it maps to `request.user.data`. See
147
+ [ZuploRequest](../programmable-api/zuplo-request.mdx) for all available type
148
+ parameters.
149
+
150
+ ```ts
151
+ import { ZuploContext, ZuploRequest } from "@zuplo/runtime";
152
+
153
+ interface ConsumerMetadata {
154
+ plan: string;
155
+ customerId: string;
156
+ orgId?: string;
157
+ }
158
+
159
+ export default async function handler(
160
+ request: ZuploRequest<{ UserData: ConsumerMetadata }>,
161
+ context: ZuploContext,
162
+ ) {
163
+ // TypeScript knows the shape of request.user.data
164
+ const { plan, customerId } = request.user!.data;
165
+
166
+ return new Response(`Plan: ${plan}, Customer: ${customerId}`);
167
+ }
168
+ ```
169
+
170
+ See [RequestUser](../programmable-api/request-user.mdx) for the full type
171
+ reference.
172
+
173
+ ## Forwarding metadata to your backend
174
+
175
+ A common pattern is to extract metadata values and forward them as HTTP headers
176
+ so your backend receives tenant identity without needing to validate API keys
177
+ itself. Create a [custom inbound policy](../policies/custom-code-inbound.mdx)
178
+ that runs after authentication:
179
+
180
+ <Stepper>
181
+
182
+ 1. **Create the policy module**
183
+
184
+ Create a new module file at `modules/forward-metadata.ts`:
185
+
186
+ ```ts title="modules/forward-metadata.ts"
187
+ import { ZuploContext, ZuploRequest } from "@zuplo/runtime";
188
+
189
+ export default async function forwardMetadata(
190
+ request: ZuploRequest,
191
+ context: ZuploContext,
192
+ ) {
193
+ // Strip any client-supplied headers to prevent spoofing
194
+ request.headers.delete("x-consumer-name");
195
+ request.headers.delete("x-customer-id");
196
+ request.headers.delete("x-org-id");
197
+ request.headers.delete("x-plan");
198
+
199
+ if (!request.user) {
200
+ return request;
201
+ }
202
+
203
+ const { customerId, orgId, plan } = request.user.data;
204
+
205
+ // Set trusted headers for the upstream backend
206
+ request.headers.set("x-consumer-name", request.user.sub);
207
+
208
+ if (customerId) {
209
+ request.headers.set("x-customer-id", customerId);
210
+ }
211
+ if (orgId) {
212
+ request.headers.set("x-org-id", orgId);
213
+ }
214
+ if (plan) {
215
+ request.headers.set("x-plan", plan);
216
+ }
217
+
218
+ return request;
219
+ }
220
+ ```
221
+
222
+ :::warning
223
+
224
+ Always delete client-supplied copies of metadata headers **before** setting
225
+ them from `request.user.data`. If a metadata field is absent and you skip the
226
+ delete, the client's spoofed value passes through to your backend. Your
227
+ backend should trust these headers only because the gateway sets them.
228
+
229
+ :::
230
+
231
+ 1. **Register the policy**
232
+
233
+ Add the policy to your `config/policies.json`:
234
+
235
+ ```json
236
+ {
237
+ "name": "forward-metadata",
238
+ "policyType": "custom-code-inbound",
239
+ "handler": {
240
+ "export": "default",
241
+ "module": "$import(./modules/forward-metadata)"
242
+ }
243
+ }
244
+ ```
245
+
246
+ 1. **Attach the policy to your route**
247
+
248
+ Add `forward-metadata` to the route's inbound policy array **after**
249
+ `api-key-inbound` so that `request.user` is populated before the policy runs.
250
+ See [Custom Code Inbound Policy](../policies/custom-code-inbound.mdx) for the
251
+ full policy configuration reference.
252
+
253
+ </Stepper>
254
+
255
+ ## Common patterns
256
+
257
+ ### Feature gating by plan
258
+
259
+ Check `request.user.data.plan` to allow or deny access to specific routes:
260
+
261
+ ```ts
262
+ import { ZuploContext, ZuploRequest, HttpProblems } from "@zuplo/runtime";
263
+
264
+ export default async function requirePlan(
265
+ request: ZuploRequest,
266
+ context: ZuploContext,
267
+ ) {
268
+ const allowedPlans = ["gold", "enterprise"];
269
+
270
+ if (!allowedPlans.includes(request.user?.data?.plan)) {
271
+ return HttpProblems.forbidden(request, context, {
272
+ detail: "This endpoint requires a Gold or Enterprise plan.",
273
+ });
274
+ }
275
+
276
+ return request;
277
+ }
278
+ ```
279
+
280
+ ### Multi-tenant backend routing
281
+
282
+ Route each customer to a dedicated backend based on metadata. The following
283
+ policy sets `context.custom.downstreamUrl`, which the URL Rewrite handler reads
284
+ to determine the upstream destination. See
285
+ [Route to Backends Based on User Identity](../guides/user-based-backend-routing.mdx)
286
+ for the full setup including handler configuration.
287
+
288
+ ```ts
289
+ import { ZuploContext, ZuploRequest } from "@zuplo/runtime";
290
+
291
+ export default async function tenantRouting(
292
+ request: ZuploRequest,
293
+ context: ZuploContext,
294
+ ) {
295
+ const { customerId } = request.user!.data;
296
+
297
+ // The URL Rewrite handler reads this to route the request
298
+ context.custom.downstreamUrl = `https://${customerId}.api.example.com`;
299
+
300
+ return request;
301
+ }
302
+ ```
303
+
304
+ ### Dynamic rate limiting
305
+
306
+ Use metadata to apply different rate limits per plan tier. See
307
+ [Dynamic Rate Limiting](../rate-limiting/dynamic-rate-limiting.mdx) for the full
308
+ setup, which reads `request.user.data` to select the rate limit policy to apply.
309
+
310
+ ## Metadata scope and per-key behavior
311
+
312
+ Each consumer can have any number of API keys. All keys for a consumer share the
313
+ same identity (`request.user.sub`) and metadata (`request.user.data`). This is
314
+ by design — a consumer represents an identity, and metadata describes that
315
+ identity.
316
+
317
+ **Per-key metadata is not supported.** There is no way to attach different
318
+ metadata to different keys under the same consumer. If you need different
319
+ metadata for different keys — for example, different permissions or tenant
320
+ identifiers — create separate consumers. Each consumer is an identity, so
321
+ creating one consumer per identity that needs distinct metadata is the intended
322
+ pattern.
323
+
324
+ For example, if a single organization has two teams that need different access
325
+ levels, create two consumers (`acme-team-a` and `acme-team-b`) with appropriate
326
+ metadata on each, rather than two keys on a single `acme` consumer.
327
+
328
+ :::tip
329
+
330
+ If two keys need different runtime behavior, create separate consumers.
331
+
332
+ :::
333
+
334
+ ## Visibility and security
335
+
336
+ ### Who can see consumer metadata
337
+
338
+ Understanding who has access to metadata is important when deciding what to
339
+ store in it:
340
+
341
+ | Audience | Can see metadata? | Details |
342
+ | -------------------------------------------------------------- | ----------------- | ------------------------------------------------------------------------------------------- |
343
+ | **Your policies and handlers** | Yes | Via `request.user.data` on every authenticated request |
344
+ | **Zuplo Portal admins** | Yes | Via **Services → API Keys → Consumers** in the project |
345
+ | **[Developer API](./api-key-api.mdx) callers** | Yes | Via the [Consumers API](../api/api-keys-consumers) (requires a Zuplo API key) |
346
+ | **[Developer Portal](../dev-portal/overview.mdx) key holders** | No | Key managers can view, copy, and rotate their keys but do not see consumer metadata or tags |
347
+
348
+ ### What to store (and what not to store)
349
+
350
+ Metadata is available to your runtime code and to anyone with Developer API
351
+ access. Follow these guidelines:
352
+
353
+ - **Good for metadata**: plan tier, customer ID, organization ID, feature flags,
354
+ environment identifiers, external system references.
355
+ - **Not for metadata**: secrets, passwords, tokens, PII (email addresses, phone
356
+ numbers), or any data that would cause harm if exposed through the Developer
357
+ API.
358
+
359
+ Keep metadata small. The maximum size is **1 KB** per consumer.
360
+
361
+ ## API key format
362
+
363
+ Zuplo API keys use the `zpka_<random>_<checksum>` format, enabling
364
+ [leak detection](./api-key-leak-detection.mdx) via GitHub secret scanning. For
365
+ the full format breakdown and service limits, see
366
+ [API Keys concepts](../concepts/api-keys.md#api-key-format) and
367
+ [API Key Service Limits](./api-key-service-limits.mdx).
368
+
369
+ ## Next steps
370
+
371
+ - [API Key Authentication](./api-key-authentication.mdx) — how the
372
+ authentication policy sets `request.user`
373
+ - [Route to Backends Based on User Identity](../guides/user-based-backend-routing.mdx)
374
+ — full guide on metadata-driven routing
375
+ - [Dynamic Rate Limiting](../rate-limiting/dynamic-rate-limiting.mdx) — apply
376
+ different rate limits based on metadata
377
+ - [Developer API](./api-key-api.mdx) — manage consumers and metadata
378
+ programmatically
379
+ - [API Key Best Practices](./api-key-best-practices.mdx) — security and
380
+ operational guidance
package/docs/articles/limits.mdx CHANGED
@@ -18,18 +18,18 @@ fully customizable to meet your specific requirements.
18
18
  Below are general limits that apply to deployments on Zuplo's managed edge
19
19
  environments.
20
20
 
21
- | Feature | Description | Limit |
22
- | ----------------------------- | --------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
23
- | Environment Variables | The number of environment variables you can create | Free/Builder Plans: 50 variables per environment, Other Plans: 100 per environment |
24
- | Environment Variable size | The size of an environment variable | ~5kb per variable |
25
- | Request Duration | The maximum time a request can take to complete | Free/Builder Plans: 30 seconds, Other Plans: No Limit |
26
- | Memory | The maximum memory available when processing a request | Free/Builder Plans: 128mb, Enterprise Plans: Custom |
27
- | Requests per second | The maximum number of requests per second | Development Environments (Working Copy): 1000 requests per minute. No Limit outside of plan monthly limit |
28
- | Log Size | The size of a log entry | Zuplo Portal live logs are limited to ~10kB. Excess will be truncated. For third-party logging providers, the value depends on the provider. Zuplo imposes no limit. |
29
- | Request Body Size | The maximum size of an incoming request body | 500 MB |
30
- | Response Headers | The total size of all response headers | 128 KB |
31
- | Subrequests | The maximum number of subrequests per request | 1,000 |
32
- | Simultaneous Open Connections | The maximum number of simultaneous open connections per request | 6 |
21
+ | Feature | Description | Limit |
22
+ | ----------------------------- | ------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
23
+ | Environment Variables | The number of environment variables you can create | Free/Builder Plans: 50 variables per environment, Other Plans: 100 per environment |
24
+ | Environment Variable size | The size of an environment variable | ~5kb per variable |
25
+ | Request Duration | The maximum time a request can take to complete. See [Request duration](#request-duration). | Free/Builder Plans: 30 seconds, Other Plans: No Limit |
26
+ | Memory | The maximum memory available when processing a request | Free/Builder Plans: 128mb, Enterprise Plans: Custom |
27
+ | Requests per second | The maximum number of requests per second | Development Environments (Working Copy): 1000 requests per minute. No Limit outside of plan monthly limit |
28
+ | Log Size | The size of a log entry | Zuplo Portal live logs are limited to ~10kB. Excess will be truncated. For third-party logging providers, the value depends on the provider. Zuplo imposes no limit. |
29
+ | Request Body Size | The maximum size of an incoming request body | 500 MB |
30
+ | Response Headers | The total size of all response headers | 128 KB |
31
+ | Subrequests | The maximum number of subrequests per request | 1,000 |
32
+ | Simultaneous Open Connections | The maximum number of simultaneous open connections per request | 6 |
33
33
 
34
34
  ## Logs and tracing
35
35
 
@@ -50,8 +50,44 @@ efficient use of resources. These timeouts apply to both incoming connections
50
50
  from clients to Zuplo and outgoing connections from Zuplo to your origin
51
51
  servers.
52
52
 
53
+ ### Request duration
54
+
55
+ The request duration limit is the maximum wall-clock time a single request can
56
+ take, from the moment Zuplo receives it until the response is fully delivered to
57
+ the client.
58
+
59
+ | Plan | Limit |
60
+ | -------------------- | ---------- |
61
+ | Free / Builder | 30 seconds |
62
+ | All other paid plans | No limit |
63
+
64
+ On paid plans above Builder, the request duration limit doesn't cap your
65
+ requests. However, the connection timeouts below still apply, so the request
66
+ duration limit isn't the only factor that determines whether a long-running
67
+ request completes.
68
+
69
+ ### Preview and working-copy environments
70
+
71
+ The request duration limit and the connection timeouts on this page apply to
72
+ **all** managed edge environments: production, preview (branch deployments), and
73
+ working copy. Timeout behavior doesn't differ between environment types.
74
+
75
+ :::note
76
+
77
+ Working-copy environments have a separate rate limit of 1,000 requests per
78
+ minute (see [General](#general)). It doesn't affect request duration, but it can
79
+ limit throughput during testing.
80
+
81
+ :::
82
+
53
83
  ## Between client and Zuplo
54
84
 
85
+ These timeouts measure how long an established connection between the calling
86
+ client and Zuplo can remain idle (no data in either direction) before Zuplo
87
+ closes it. The client's own network stack, VPN, or load balancer may impose
88
+ shorter idle timeouts. If requests drop before the limits below, check for
89
+ intermediate network devices that close idle connections.
90
+
55
91
  | Type | Limit (seconds) | HTTP status code at limit |
56
92
  | ------------------------------ | --------------- | ------------------------- |
57
93
  | Connection Keep-Alive HTTP/1.1 | 400 | TCP connection closed |
@@ -59,16 +95,70 @@ servers.
59
95
 
60
96
  ## Between Zuplo and origin server
61
97
 
62
- | Type | Limit (seconds) | HTTP status code at limit |
63
- | ----------------------- | --------------- | ------------------------- |
64
- | Complete TCP Connection | 19 | 522 |
65
- | TCP ACK Timeout | 90 | 522 |
66
- | TCP Keep-Alive Interval | 30 | 520 |
67
- | Proxy Idle Timeout | 900 | 520 |
68
- | Proxy Read Timeout | 180 | 524 |
69
- | Proxy Write Timeout | 30 | 524 |
70
- | HTTP/2 Pings to Origin | Off | - |
71
- | HTTP/2 Connection Idle | 900 | No |
98
+ Zuplo's managed edge environment is routed through Cloudflare's global network
99
+ and is subject to the following origin connection timeouts. These timeouts
100
+ govern the connection between Zuplo and your origin (backend) server, not the
101
+ connection between the client and Zuplo.
102
+
103
+ | Type | Limit (seconds) | What it measures | HTTP status code at limit |
104
+ | ----------------------- | --------------- | ---------------------------------------------------------------------------- | -------------------------- |
105
+ | Complete TCP Connection | 19 | Time to complete the TCP handshake with the origin | 522 |
106
+ | TCP ACK Timeout | 90 | Time waiting for the origin to acknowledge a TCP segment | 522 |
107
+ | TCP Keep-Alive Interval | 30 | Interval between TCP keep-alive probes on an idle connection | 520 |
108
+ | Proxy Idle Timeout | 900 | Time an established connection can sit idle with no data in either direction | 520 |
109
+ | Proxy Read Timeout | 180 | Time between consecutive data transfers from the origin (see below) | 524 |
110
+ | Proxy Write Timeout | 30 | Time to send data to the origin after the connection is established | 524 |
111
+ | HTTP/2 Pings to Origin | Off | - | - |
112
+ | HTTP/2 Connection Idle | 900 | Time an HTTP/2 connection can sit idle | Connection closed silently |
113
+
114
+ ### The 180-second buffered response ceiling
115
+
116
+ The **proxy read timeout** (180 seconds) is the limit that most often affects
117
+ long-running requests. It measures the time between consecutive data transfers
118
+ from the origin. The clock starts after Zuplo sends the request to the origin,
119
+ and it resets every time the origin sends data, including the HTTP response
120
+ headers and every subsequent body chunk.
121
+
122
+ For **buffered (non-streaming) responses**, your origin must begin sending the
123
+ response (at minimum, the status line and headers) within 180 seconds of
124
+ receiving the request. If the origin takes longer than 180 seconds to produce
125
+ any output, Zuplo returns an HTTP 524 error to the client, even if your plan has
126
+ a higher request duration limit.
127
+
128
+ For **streaming responses**, where the origin sends data incrementally, the
129
+ 180-second clock restarts with every chunk, so the total request can run much
130
+ longer than 180 seconds as long as data keeps flowing.
131
+
132
+ :::caution{title="Raised duration limits don't override the proxy read timeout"}
133
+
134
+ If you purchased a raised request duration limit, the 180-second proxy read
135
+ timeout still applies. The two limits are independent, and a request must
136
+ satisfy both to complete.
137
+
138
+ :::
139
+
140
+ ### Strategies for long-running requests
141
+
142
+ If your origin needs more than 180 seconds to produce a buffered response, use
143
+ one of these strategies:
144
+
145
+ - **Stream the response.** Send data incrementally with chunked transfer
146
+ encoding or server-sent events. The proxy read timeout resets with every
147
+ chunk, so the origin only needs to send at least one byte every 180 seconds.
148
+ - **Use an asynchronous pattern.** Return an HTTP 202 (Accepted) immediately
149
+ with a status URL or job identifier, and have the client poll or receive a
150
+ webhook with the result. This avoids long-held connections entirely.
151
+ - **Optimize the origin.** Reduce processing time through caching,
152
+ precomputation, or breaking large operations into smaller steps.
153
+
154
+ For more on diagnosing latency, see
155
+ [Troubleshooting Slow API Response Times](./troubleshooting-slow-responses.mdx).
156
+
157
+ ### Requesting a limit change
158
+
159
+ To raise the 30-second request duration limit on Free and Builder plans, upgrade
160
+ to a higher plan. For custom timeout adjustments, contact
161
+ [sales@zuplo.com](mailto:sales@zuplo.com).
72
162
 
73
163
  ## API keys
74
164
 
package/docs/articles/streaming-responses-sse.mdx ADDED
@@ -0,0 +1,244 @@
1
+ ---
2
+ title: Streaming responses and Server-Sent Events
3
+ sidebar_label: Streaming & SSE
4
+ description:
5
+ Stream responses through Zuplo with ReadableStream and Server-Sent Events,
6
+ proxy upstream streams unbuffered, and stream LLM tokens through the AI
7
+ Gateway.
8
+ tags:
9
+ - request-handling
10
+ - backends
11
+ ---
12
+
13
+ Zuplo's runtime is built on web-standard APIs, so streaming responses work the
14
+ same way they do in any standards-compliant JavaScript environment. You can
15
+ return a `ReadableStream` as a response body from a
16
+ [custom handler](../handlers/custom-handler.mdx), proxy an upstream streaming
17
+ response through the gateway unbuffered, or stream LLM tokens through Zuplo's
18
+ [AI Gateway](../ai-gateway/overview.mdx). The runtime supports the full
19
+ [Streams API](../programmable-api/web-standard-apis.mdx) — `ReadableStream`,
20
+ `WritableStream`, and `TransformStream` — so when a handler returns a `Response`
21
+ whose body is a `ReadableStream`, Zuplo streams the bytes to the client as they
22
+ become available rather than buffering the entire response.
23
+
24
+ ## Return an SSE response from a custom handler
25
+
26
+ To emit Server-Sent Events from a Zuplo handler, create a `ReadableStream` that
27
+ writes SSE-formatted messages and return it with the `text/event-stream` content
28
+ type.
29
+
30
+ ```ts
31
+ import { ZuploContext, ZuploRequest } from "@zuplo/runtime";
32
+
33
+ export default async function (request: ZuploRequest, context: ZuploContext) {
34
+ const stream = new ReadableStream({
35
+ start(controller) {
36
+ const encoder = new TextEncoder();
37
+ let count = 0;
38
+ const interval = setInterval(() => {
39
+ count++;
40
+ const message = `data: ${JSON.stringify({ count, time: Date.now() })}\n\n`;
41
+ controller.enqueue(encoder.encode(message));
42
+
43
+ if (count >= 5) {
44
+ clearInterval(interval);
45
+ controller.close();
46
+ }
47
+ }, 1000);
48
+ },
49
+ });
50
+
51
+ return new Response(stream, {
52
+ headers: {
53
+ "content-type": "text/event-stream",
54
+ "cache-control": "no-cache",
55
+ },
56
+ });
57
+ }
58
+ ```
59
+
60
+ Each SSE message ends with two newline characters (`\n\n`). The
61
+ `text/event-stream` content type tells the client to interpret the response as
62
+ an event stream.
63
+
64
+ ## Proxy an upstream streaming response
65
+
66
+ When your upstream API returns a streaming response — for example, an SSE
67
+ endpoint or a chunked JSON stream — you can pass it through the gateway
68
+ unbuffered. Return the upstream `Response` directly from your handler without
69
+ reading its body:
70
+
71
+ ```ts
72
+ import { ZuploContext, ZuploRequest } from "@zuplo/runtime";
73
+
74
+ export default async function (request: ZuploRequest, context: ZuploContext) {
75
+ const upstreamResponse = await fetch("https://api.example.com/events", {
76
+ headers: {
77
+ accept: "text/event-stream",
78
+ authorization: `Bearer ${context.env.UPSTREAM_API_KEY}`,
79
+ },
80
+ });
81
+
82
+ // Return the response directly — the body streams through unbuffered
83
+ return upstreamResponse;
84
+ }
85
+ ```
86
+
87
+ Set `UPSTREAM_API_KEY` in your project's
88
+ [environment variables](./environment-variables.mdx).
89
+
90
+ :::caution{title="Avoid reading the body"}
91
+
92
+ Calling `.text()`, `.json()`, or `.arrayBuffer()` on the upstream response
93
+ consumes the entire `ReadableStream` into memory. If you need to inspect the
94
+ body _and_ return it, use `.clone()` first — see
95
+ [Safely clone a request or response](../programmable-api/safely-clone-a-request-or-response.mdx).
96
+
97
+ :::
98
+
99
+ You can also use the built-in [URL Rewrite](../handlers/url-rewrite.mdx) or
100
+ [URL Forward](../handlers/url-forward.mdx) handlers to proxy upstream responses.
101
+ These handlers pass the response body through as a stream without buffering.
102
+
103
+ ## Streaming and policies
104
+
105
+ Policies in the Zuplo [request pipeline](../concepts/request-lifecycle.mdx)
106
+ interact with streaming responses differently depending on whether they read the
107
+ response body:
108
+
109
+ - **Policies that do not read the body** — such as
110
+ [Set Headers](../policies/set-headers-outbound.mdx) or
111
+ [Remove Headers](../policies/remove-headers-outbound.mdx) — pass the stream
112
+ through without buffering.
113
+ - **Policies that read the body** — such as
114
+ [Replace String](../policies/replace-string-outbound.mdx) or
115
+ [XML to JSON](../policies/xml-to-json-outbound.mdx) — consume the entire
116
+ response into memory before returning a new response. This effectively
117
+ disables streaming for that request.
118
+
119
+ :::tip{title="Keep streaming intact"}
120
+
121
+ If streaming performance is critical, place only header-modifying policies on
122
+ the outbound pipeline of streaming routes. Avoid outbound policies that inspect
123
+ or transform the response body.
124
+
125
+ :::
126
+
127
+ **Inbound policies** run before the handler executes and do not affect response
128
+ streaming. Authentication, rate limiting, and request validation all complete
129
+ before the handler returns a streaming response.
130
+
131
+ ## Stream LLM tokens through the AI Gateway
132
+
133
+ The [Zuplo AI Gateway](../ai-gateway/overview.mdx) preserves streaming behavior
134
+ from LLM providers. When you send a request with `"stream": true`, the gateway
135
+ proxies the provider's token-by-token response back to your application as it
136
+ arrives.
137
+
138
+ To call the AI Gateway, copy the **API URL** and **API Key** from your app's
139
+ page in the AI Gateway. Append `/v1/chat/completions` to the API URL and send
140
+ the app's API key as a bearer token. Name the model as `providerName/model`,
141
+ where `providerName` is the provider name configured in your gateway. See
142
+ [Universal API](../ai-gateway/universal-api.mdx) for details.
143
+
144
+ ```ts
145
+ const response = await fetch(
146
+ "https://my-gateway-main-2e18f50.zuplo.app/config_fe0a04972d2848e0a94ae4b8bcd1497e/v1/chat/completions",
147
+ {
148
+ method: "POST",
149
+ headers: {
150
+ "content-type": "application/json",
151
+ authorization: `Bearer ${process.env.ZUPLO_AI_GATEWAY_API_KEY}`,
152
+ },
153
+ body: JSON.stringify({
154
+ model: "openai/gpt-6-luna",
155
+ stream: true,
156
+ messages: [{ role: "user", content: "Hello" }],
157
+ }),
158
+ },
159
+ );
160
+
161
+ // The response is an SSE stream of token chunks
162
+ const reader = response.body!.getReader(); // body is non-null for streamed responses
163
+ const decoder = new TextDecoder();
164
+ while (true) {
165
+ const { done, value } = await reader.read();
166
+ if (done) break;
167
+ console.log(decoder.decode(value, { stream: true }));
168
+ }
169
+ ```
170
+
171
+ The AI Gateway applies usage limits, policies, and observability to streamed
172
+ requests. Token usage and cost are tracked for each request regardless of
173
+ whether streaming is enabled. See
174
+ [AI Gateway overview](../ai-gateway/overview.mdx) for more on usage limits and
175
+ provider configuration.
176
+
177
+ ## Cache streaming responses
178
+
179
+ Use [StreamingZoneCache](../programmable-api/streaming-zone-cache.mdx) to cache
180
+ `ReadableStream` responses across Zuplo's edge network. Split the stream with
181
+ `tee()` to return one copy to the client while caching the other:
182
+
183
+ ```ts
184
+ import { StreamingZoneCache, ZuploContext, ZuploRequest } from "@zuplo/runtime";
185
+
186
+ export default async function (request: ZuploRequest, context: ZuploContext) {
187
+ const cache = new StreamingZoneCache("sse-cache", context);
188
+ const cacheKey = `stream:${request.url}`;
189
+
190
+ const cached = await cache.get(cacheKey);
191
+ if (cached) {
192
+ return new Response(cached, {
193
+ headers: { "content-type": "text/event-stream" },
194
+ });
195
+ }
196
+
197
+ const upstream = await fetch("https://api.example.com/events");
198
+
199
+ // Only cache successful responses that have a body
200
+ if (!upstream.ok || !upstream.body) {
201
+ return upstream;
202
+ }
203
+
204
+ const [forCache, forClient] = upstream.body.tee();
205
+
206
+ // TTL is in seconds — cache for 5 minutes
207
+ context.waitUntil(cache.put(cacheKey, forCache, 300));
208
+
209
+ return new Response(forClient, {
210
+ headers: { "content-type": "text/event-stream" },
211
+ });
212
+ }
213
+ ```
214
+
215
+ :::note
216
+
217
+ Caching is best suited for streams that produce a finite, repeatable result (for
218
+ example, a pre-generated report). Infinite or user-specific event streams are
219
+ generally not good candidates for caching.
220
+
221
+ :::
222
+
223
+ ## When to use WebSockets instead
224
+
225
+ SSE is one-directional: the server pushes events to the client. If you need
226
+ bidirectional communication — where the client also sends messages over the same
227
+ connection — use Zuplo's [WebSocket Handler](../handlers/websocket-handler.mdx)
228
+ instead. WebSockets support full-duplex communication and can be proxied to an
229
+ upstream WebSocket server.
230
+
231
+ ## Next steps
232
+
233
+ - [Function Handler](../handlers/custom-handler.mdx) — build custom request
234
+ handlers
235
+ - [Web Standard APIs](../programmable-api/web-standard-apis.mdx) — the full list
236
+ of supported web APIs, including the Streams API
237
+ - [StreamingZoneCache](../programmable-api/streaming-zone-cache.mdx) — cache
238
+ streaming responses at the edge
239
+ - [Safely clone a request or response](../programmable-api/safely-clone-a-request-or-response.mdx)
240
+ — avoid `body-used` errors when inspecting streamed bodies
241
+ - [AI Gateway](../ai-gateway/overview.mdx) — stream LLM tokens with cost
242
+ controls and observability
243
+ - [WebSocket Handler](../handlers/websocket-handler.mdx) — bidirectional
244
+ real-time communication
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "zuplo",
3
- "version": "7.12.4",
3
+ "version": "7.12.5",
4
4
  "type": "module",
5
5
  "description": "The official Zuplo CLI for local development and platform management",
6
6
  "homepage": "https://zuplo.com/docs/cli/overview",
@@ -32,9 +32,9 @@
32
32
  "zuplo": "zuplo.js"
33
33
  },
34
34
  "dependencies": {
35
- "@zuplo/cli": "7.12.4",
36
- "@zuplo/core": "7.12.4",
37
- "@zuplo/runtime": "7.12.4",
38
- "@zuplo/test": "7.12.4"
35
+ "@zuplo/cli": "7.12.5",
36
+ "@zuplo/core": "7.12.5",
37
+ "@zuplo/runtime": "7.12.5",
38
+ "@zuplo/test": "7.12.5"
39
39
  }
40
40
  }