zuplo 6.70.69 → 6.70.71

package/docs/rate-limiting/monitoring-and-troubleshooting.mdx

@@ -0,0 +1,243 @@
+---
+title: Monitoring and troubleshooting rate limits
+sidebar_label: Monitoring & troubleshooting
+description:
+  Monitor rate limit events, debug unexpected 429 responses, and understand
+  failure modes.
+---
+
+Rate limiting only delivers value when you can observe it in action. Without
+visibility into which consumers hit limits, how often requests are rejected, and
+whether the rate limit service itself is healthy, you are operating blind. This
+guide covers how to monitor rate limit activity, understand failure modes,
+choose the right enforcement mode, and diagnose common issues.
+
+## Monitoring rate limit events
+
+Zuplo produces structured logs for every request, including those rejected with
+a `429 Too Many Requests` status code. Ship these logs to an external provider
+to build dashboards and alerts around rate limit activity.
+
+### Setting up log shipping
+
+Configure a [logging plugin](../articles/logging.mdx) in your `zuplo.runtime.ts`
+file to send logs to your observability platform. Zuplo supports AWS CloudWatch,
+Datadog, Dynatrace, Google Cloud Logging, Loki, New Relic, Splunk, Sumo Logic,
+and VMware Log Insight. You can also build a
+[custom logging plugin](../articles/custom-logging-example.mdx) for unsupported
+providers.
+
+### Filtering for rate-limited requests
+
+Every log entry includes default fields you can filter on:
+
+- **`requestId`** -- Correlate a specific rejected request end-to-end using the
+  `zp-rid` response header.
+- **`environment`** and **`environmentStage`** -- Distinguish between
+  `production`, `preview`, and `working-copy` environments.
+
+To break down rate-limited requests by consumer or IP, add custom log properties
+in a policy that runs before or alongside the rate limit check:
+
+```ts
+import { ZuploContext, ZuploRequest } from "@zuplo/runtime";
+
+export default async function policy(
+  request: ZuploRequest,
+  context: ZuploContext,
+) {
+  // Tag every log entry with the consumer identity for filtering
+  context.log.setLogProperties!({
+    rateLimitIdentity:
+      request.user?.sub ?? request.headers.get("true-client-ip") ?? "unknown",
+  });
+  return request;
+}
+```
+
+This adds a `rateLimitIdentity` field to all log entries for the request, making
+it straightforward to group 429 responses by consumer in your logging dashboard.
+
+### Setting up alerts
+
+Configure alerts in your logging provider for the following conditions:
+
+- **Spike in 429 responses** -- A sudden increase may indicate a
+  misconfiguration, an attack, or a legitimate traffic surge.
+- **429 rate exceeding a threshold** -- If more than a small percentage of
+  requests return 429, the rate limit may be set too low for normal traffic.
+- **Zero 429 responses over an extended period** -- If you expect rate limiting
+  to be active but see no rejections, the policy may not be attached to the
+  correct routes.
+
+### Metrics plugins
+
+For quantitative monitoring, Zuplo supports
+[metrics plugins](../articles/metrics-plugins.mdx) that send request latency,
+request size, and response size data to Datadog, Dynatrace, New Relic, or any
+OpenTelemetry-compatible collector. While these metrics do not track rate limit
+counters directly, the `statusCode` dimension (when enabled) allows you to chart
+429 response rates alongside overall request volume.
+
+## Understanding failure modes
+
+The rate limiting policies depend on a globally distributed rate limit service
+to track request counters. Understanding what happens when that service is
+unreachable helps you make the right availability tradeoff.
+
+### Fail-open (default)
+
+By default, `throwOnFailure` is set to `false`. If the rate limit service is
+unreachable, the policy allows the request through. This fail-open behavior
+prevents a rate limit service outage from blocking all traffic to your API.
+
+The tradeoff is that during an outage, rate limits are not enforced and clients
+can exceed their configured thresholds.
+
+### Fail-closed
+
+Set `throwOnFailure` to `true` to return an error when the rate limit service is
+unreachable. This guarantees that no request bypasses rate limiting, but it
+means a service disruption blocks all traffic on routes using that policy.
+
+```json
+{
+  "options": {
+    "rateLimitBy": "user",
+    "requestsAllowed": 100,
+    "timeWindowMinutes": 1,
+    "throwOnFailure": true
+  }
+}
+```
+
+:::warning
+
+Only use `throwOnFailure: true` when allowing unlimited traffic is more
+dangerous than rejecting all traffic. For most APIs, the fail-open default is
+the safer choice.
+
+:::
+
+### Detecting fail-open conditions
+
+Because fail-open requests succeed with a `200` (or other normal status code),
+they do not produce a 429 log entry. To detect when the rate limit service is
+unreachable, monitor for a sudden drop in 429 responses during periods when you
+expect rate limiting to be active. A complete absence of 429s alongside steady
+or increasing traffic volume is a strong signal that the service is in fail-open
+mode.
+
+## Strict vs. async mode in production
+
+The `mode` option controls whether the rate limit check blocks the request or
+runs in parallel with it.
+
+### Strict mode (default)
+
+In `strict` mode, every request waits for the rate limit service to confirm
+whether the request is within limits before proceeding to the backend. This
+provides exact enforcement -- no request exceeds the configured threshold.
+
+The tradeoff is added latency on every request due to the round-trip to the rate
+limit service.
+
+### Async mode
+
+In `async` mode, the request proceeds to the backend immediately while the rate
+limit check runs in parallel. If the check determines the limit is exceeded, the
+result applies to the _next_ request, not the current one.
+
+This means some requests may get through after the limit is reached. In
+practice, the overshoot depends on your request rate and the latency of the rate
+limit check. For an API receiving 100 requests per second with a 10ms check
+time, approximately one extra request may slip through per window.
+
+:::tip
+
+Use `async` mode when low latency matters more than exact enforcement -- for
+example, on high-throughput public endpoints where a few extra requests over the
+limit are acceptable. Use `strict` mode when precise enforcement is required,
+such as billing-sensitive endpoints or APIs with hard backend capacity limits.
+
+:::
+
+## Common troubleshooting scenarios
+
+### Unexpected 429 responses
+
+**Shared IP addresses.** When `rateLimitBy` is set to `"ip"`, multiple clients
+behind the same corporate proxy, cloud NAT, or shared Wi-Fi share a single rate
+limit bucket. One heavy user exhausts the limit for everyone on that IP. Switch
+to `rateLimitBy: "user"` for authenticated APIs to avoid this.
+
+**Missing authentication policy.** The `"user"` mode requires an authentication
+policy (such as API Key Authentication or JWT) earlier in the policy pipeline to
+populate `request.user`. If no authentication policy runs first, the rate limit
+policy returns an error instead of applying per-user limits. Verify that
+authentication appears before rate limiting in the route's inbound policy list.
+
+**Multiple rate limit policies on the same route.** If a route has both a
+per-minute and a per-hour rate limit policy, a request can be rejected by either
+one. Check all rate limit policies attached to the route, and verify the
+ordering (longest time window first, then shorter durations).
+
+**Lower limits than expected.** If you use a custom `rateLimitBy: "function"`,
+verify that the function returns the expected `requestsAllowed` and
+`timeWindowMinutes` values. Log the returned values during development to
+confirm the function resolves correctly for each consumer.
+
+### Rate limits not applying
+
+**Policy not attached to the route.** Defining a rate limit policy in
+`policies.json` does not activate it. The policy name must appear in the
+`policies.inbound` array of each route in `routes.oas.json` where you want it
+enforced. Verify the route configuration.
+
+**Typo in the policy name.** The policy name in `routes.oas.json` must exactly
+match the `name` field in `policies.json`. A mismatched name silently skips the
+policy. Check for case sensitivity and extra whitespace.
+
+**Custom function returning `undefined`.** When `rateLimitBy` is set to
+`"function"` and the identifier function returns `undefined`, rate limiting is
+skipped for that request entirely. This is by design -- it allows you to
+selectively exempt certain requests -- but it can cause confusion if the
+function has an unhandled code path that returns `undefined` unintentionally.
+
+### Different behavior across environments
+
+Rate limit counters are scoped per environment. Production, preview, and
+working-copy environments each maintain their own separate counters. A request
+that is rate-limited in production does not affect the counter in a preview
+environment, and vice versa.
+
+This means:
+
+- Testing rate limits in a preview branch does not interfere with production
+  traffic.
+- Rate limit thresholds you observe in a low-traffic preview environment may
+  behave differently under production load.
+- After deploying a new environment, counters start fresh.
+
+:::note
+
+If you observe rate limits triggering in one environment but not another,
+confirm that both environments use the same policy configuration and that the
+traffic volume is comparable.
+
+:::
+
+## Related resources
+
+- [Rate Limit Exceeded error](../errors/rate-limit-exceeded.mdx) --
+  Understanding the 429 response format and client-side remediation
+- [How rate limiting works](./how-it-works.md) -- Algorithm details,
+  `rateLimitBy` modes, and combining policies
+- [Logging](../articles/logging.mdx) -- Configuring log shipping to external
+  providers
+- [Metrics Plugins](../articles/metrics-plugins.mdx) -- Sending request metrics
+  to Datadog, Dynatrace, New Relic, or OpenTelemetry
+- [Proactive monitoring](../articles/monitoring-your-gateway.mdx) -- Health
+  checks and end-to-end gateway monitoring
+- [Troubleshooting](../articles/troubleshooting.md) -- General gateway
+  troubleshooting guide

package/docs/{articles → rate-limiting}/per-user-rate-limits-using-db.mdx

@@ -1,6 +1,6 @@
 ---
-title: Per user rate-limiting using a database and the ZoneCache
-sidebar_label: "Per-User Rate Limits"
+title: Per-user rate limiting using a database and the ZoneCache
+sidebar_label: "Per-user rate limits"
 description:
   Learn how to implement advanced dynamic rate limiting with database lookups
   and ZoneCache for improved performance.
@@ -9,24 +9,22 @@ tags:
   - caching
 ---
 
-In this example we show a more advanced implementation of
-[dynamic rate limiting](../articles/per-user-rate-limits-using-db.mdx). It uses
-a database lookup to get the customer details and combines that with the
-ZoneCache to improve performance, reduce latency and lower the load on the
-database.
+This example shows a more advanced implementation of
+[dynamic rate limiting](./dynamic-rate-limiting.mdx). It uses a database lookup
+to get the customer details and combines that with the ZoneCache to improve
+performance, reduce latency and lower the load on the database.
 
-In this example we use [Supabase](https://supabase.com) as the database but you
-could use your own API, [Xata](https://xata.io),
-[Firebase](https://firebase.com) etc. The implementation will be similar for
-all.
+This example uses [Supabase](https://supabase.com) as the database, but you
+could use your own API, [Xata](https://xata.io), or
+[Firebase](https://firebase.com). The implementation is similar for all.
 
 If you haven't already, check out the
 [rate-limiting policy](../policies/rate-limit-inbound.mdx) and the
-[dynamic rate limiting quickstart](../articles/per-user-rate-limits-using-db.mdx).
-Then you should be oriented to how dynamic rate limiting works.
+[dynamic rate limiting guide](./dynamic-rate-limiting.mdx). Then you should be
+oriented to how dynamic rate limiting works.
 
-Below is a full implementation of a custom rate limiting function. In our
-example this is a module called `per-user-rate-limiting.ts`.
+Below is a full implementation of a custom rate limiting function. In this
+example it is a module called `per-user-rate-limiting.ts`.
 
 ```ts
 import {
@@ -48,9 +46,18 @@ export async function rateLimitKey(
   context: ZuploContext,
   policyName: string,
 ): Promise<CustomRateLimitDetails> {
-  // We'll get the customer ID from the user data.
-  // This might be from a JWT or API Key metadata
-  const customerId = request.user.data.customerId;
+  // Get the customer ID from the user data.
+  // This might be from a JWT or API Key metadata.
+  // Ensure an authentication policy runs before this.
+  const customerId = request.user?.data?.customerId;
+  if (!customerId) {
+    context.log.error("No customerId found on request.user.data");
+    return {
+      key: request.user?.sub ?? "unknown",
+      requestsAllowed: FALLBACK_REQUESTS_ALLOWED,
+      timeWindowMinutes: 1,
+    };
+  }
 
   // We don't want to hit the database on every request
   // So we'll use the fast zone cache to cache this data
@@ -95,17 +102,21 @@ export async function rateLimitKey(
 The above function can be applied to a rate limiter with the following
 configuration in policies
 
-```json
+```json title="config/policies.json"
 {
-  "export": "RateLimitInboundPolicy",
-  "module": "$import(@zuplo/runtime)",
-  "options": {
-    "rateLimitBy": "function",
-    "requestsAllowed": 2,
-    "timeWindowMinutes": 1,
-    "identifier": {
-      "export": "rateLimitKey",
-      "module": "$import(./modules/per-user-rate-limiting)"
+  "name": "my-per-user-rate-limit-policy",
+  "policyType": "rate-limit-inbound",
+  "handler": {
+    "export": "RateLimitInboundPolicy",
+    "module": "$import(@zuplo/runtime)",
+    "options": {
+      "rateLimitBy": "function",
+      "requestsAllowed": 100,
+      "timeWindowMinutes": 1,
+      "identifier": {
+        "export": "rateLimitKey",
+        "module": "$import(./modules/per-user-rate-limiting)"
+      }
     }
   }
 }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "zuplo",
-  "version": "6.70.69",
+  "version": "6.70.71",
   "type": "module",
   "description": "The programmable API Gateway",
   "author": "Zuplo, Inc.",
@@ -19,9 +19,9 @@
     "zuplo": "zuplo.js"
   },
   "dependencies": {
-    "@zuplo/cli": "6.70.69",
-    "@zuplo/core": "6.70.69",
-    "@zuplo/runtime": "6.70.69",
+    "@zuplo/cli": "6.70.71",
+    "@zuplo/core": "6.70.71",
+    "@zuplo/runtime": "6.70.71",
     "@zuplo/test": "1.4.0"
   }
 }

package/docs/concepts/rate-limiting.md

@@ -1,246 +0,0 @@
----
-title: Rate Limiting
----
-
-Rate limiting controls how many requests a client can make to your API within a
-given time window. It protects your backend from traffic spikes, enforces fair
-usage across consumers, and enables tiered access for different customer plans.
-
-Zuplo's rate limiter uses a **sliding window algorithm** enforced globally
-across all edge locations. When a client exceeds the limit, they receive a
-`429 Too Many Requests` response with a `retry-after` header indicating when
-they can retry.
-
-## Rate limiting policies
-
-Zuplo provides two rate limiting policies, each suited to different levels of
-complexity.
-
-### Rate Limiting policy
-
-The [Rate Limiting policy](../policies/rate-limit-inbound.mdx) enforces a single
-request counter per time window. Configure a maximum number of requests, a time
-window, and how to identify callers.
-
-```json
-{
-  "name": "my-rate-limit-policy",
-  "policyType": "rate-limit-inbound",
-  "handler": {
-    "export": "RateLimitInboundPolicy",
-    "module": "$import(@zuplo/runtime)",
-    "options": {
-      "rateLimitBy": "user",
-      "requestsAllowed": 100,
-      "timeWindowMinutes": 1
-    }
-  }
-}
-```
-
-Use this policy when you need a straightforward "X requests per Y minutes"
-limit.
-
-### Complex Rate Limiting policy
-
-The [Complex Rate Limiting policy](../policies/complex-rate-limit-inbound.mdx)
-supports **multiple named counters** in a single policy. Each counter tracks a
-different resource or unit of work.
-
-```json
-{
-  "name": "my-complex-rate-limit-policy",
-  "policyType": "complex-rate-limit-inbound",
-  "handler": {
-    "export": "ComplexRateLimitInboundPolicy",
-    "module": "$import(@zuplo/runtime)",
-    "options": {
-      "rateLimitBy": "user",
-      "timeWindowMinutes": 1,
-      "limits": {
-        "requests": 100,
-        "compute": 500
-      }
-    }
-  }
-}
-```
-
-You can override counter increments programmatically per request using
-`ComplexRateLimitInboundPolicy.setIncrements()`. This is useful for usage-based
-pricing where different endpoints consume different amounts of a resource (for
-example, counting compute units or tokens instead of raw requests).
-
-## Choosing a policy
-
-| Scenario                                               | Policy                                        |
-| ------------------------------------------------------ | --------------------------------------------- |
-| Fixed requests-per-minute limit for all callers        | Rate Limiting                                 |
-| Different limits per customer tier (free vs. paid)     | Rate Limiting with a custom function          |
-| Counting multiple resources (requests + compute units) | Complex Rate Limiting                         |
-| Usage-based billing with variable cost per request     | Complex Rate Limiting with dynamic increments |
-
-## How `rateLimitBy` works
-
-The `rateLimitBy` option determines how the rate limiter groups requests into
-buckets. Both policies support the same four modes.
-
-### `ip`
-
-Groups requests by the client's IP address. No authentication is required. This
-is the simplest option and works well for public APIs or as a first layer of
-protection.
-
-### `user`
-
-Groups requests by the authenticated user's identity (`request.user.sub`). When
-using [API key authentication](../articles/api-key-authentication.mdx), the
-`sub` value is the consumer name you assigned when creating the API key. When
-using JWT authentication, it comes from the token's `sub` claim.
-
-This is the recommended mode for authenticated APIs because it ties limits to
-the actual consumer rather than a shared IP address.
-
-### `function`
-
-Groups requests using a custom TypeScript function that you provide. The
-function returns a `CustomRateLimitDetails` object containing a grouping key
-and, optionally, overridden values for `requestsAllowed` and
-`timeWindowMinutes`.
-
-This mode enables dynamic rate limiting where limits vary based on customer
-tier, route parameters, or any other request property.
-
-### `all`
-
-Applies a single shared counter across all requests to the route, regardless of
-who makes them. Use this for global rate limits on endpoints that call
-resource-constrained backends.
-
-## Dynamic rate limiting with custom functions
-
-When `rateLimitBy` is set to `"function"`, you provide a TypeScript module that
-determines the rate limit at request time. The function signature is:
-
-```ts
-import {
-  CustomRateLimitDetails,
-  ZuploContext,
-  ZuploRequest,
-} from "@zuplo/runtime";
-
-export function rateLimit(
-  request: ZuploRequest,
-  context: ZuploContext,
-  policyName: string,
-): CustomRateLimitDetails | undefined {
-  const user = request.user;
-
-  if (user.data.customerType === "premium") {
-    return {
-      key: user.sub,
-      requestsAllowed: 1000,
-      timeWindowMinutes: 1,
-    };
-  }
-
-  return {
-    key: user.sub,
-    requestsAllowed: 50,
-    timeWindowMinutes: 1,
-  };
-}
-```
-
-The `CustomRateLimitDetails` object has the following properties:
-
-- `key` - The string used to group requests into rate limit buckets
-- `requestsAllowed` (optional) - Overrides the policy's `requestsAllowed` value
-- `timeWindowMinutes` (optional) - Overrides the policy's `timeWindowMinutes`
-  value
-
-Returning `undefined` skips rate limiting for that request entirely.
-
-The function can also be `async` if you need to look up limits from a database
-or external service. See
-[Per-user rate limiting using a database](../articles/per-user-rate-limits-using-db.mdx)
-for a complete example using the ZoneCache for performance.
-
-Wire the function into the policy configuration using the `identifier` option:
-
-```json
-{
-  "export": "RateLimitInboundPolicy",
-  "module": "$import(@zuplo/runtime)",
-  "options": {
-    "rateLimitBy": "function",
-    "requestsAllowed": 50,
-    "timeWindowMinutes": 1,
-    "identifier": {
-      "export": "rateLimit",
-      "module": "$import(./modules/rate-limit)"
-    }
-  }
-}
-```
-
-:::note
-
-The `requestsAllowed` and `timeWindowMinutes` values in the policy configuration
-serve as defaults. The custom function can override them per request.
-
-:::
-
-## Combining rate limiting with authentication
-
-Rate limiting works best when combined with authentication so that limits apply
-per consumer rather than per IP. A typical policy pipeline is:
-
-1. **Authentication** (e.g., API Key Authentication) -- validates credentials
-   and populates `request.user`
-2. **Rate Limiting** with `rateLimitBy: "user"` -- enforces per-consumer limits
-   using `request.user.sub`
-
-With API key authentication, the consumer's metadata (stored when creating the
-key) is available at `request.user.data`. A custom rate limit function can read
-fields like `customerType` or `plan` from the metadata to apply tiered limits.
-
-## Rate limiting and monetization
-
-If you use Zuplo's
-[Monetization](../articles/monetization/monetization-policy.md) feature, the
-monetization policy handles quota enforcement based on subscription plans. You
-can still add a rate limiting policy after the monetization policy to provide
-per-second or per-minute spike protection on top of monthly billing quotas.
-These serve different purposes:
-
-- **Monetization quotas** enforce monthly or billing-period usage limits tied to
-  a subscription plan
-- **Rate limiting** protects against short-duration traffic spikes that could
-  overwhelm your backend
-
-## Combining multiple rate limit policies
-
-You can apply multiple rate limiting policies to the same route. For example,
-you might enforce both a per-minute and a per-hour limit. When using multiple
-policies, apply the longest time window first, followed by shorter durations.
-
-## Additional options
-
-Both rate limiting policies support the following additional options:
-
-- `headerMode` - Set to `"retry-after"` (default) to include the `retry-after`
-  header in 429 responses, or `"none"` to omit it
-- `mode` - Set to `"strict"` (default) for synchronous enforcement, or `"async"`
-  for non-blocking checks that may allow some requests over the limit
-- `throwOnFailure` - Set to `true` to return an error if the rate limit service
-  is unreachable, or `false` (default) to allow the request through
-
-## Related resources
-
-- [Rate Limiting policy reference](../policies/rate-limit-inbound.mdx)
-- [Complex Rate Limiting policy reference](../policies/complex-rate-limit-inbound.mdx)
-- [Dynamic Rate Limiting tutorial](../articles/step-5-dynamic-rate-limiting.mdx)
-- [Per-user rate limiting with a database](../articles/per-user-rate-limits-using-db.mdx)
-- [Quota policy](../policies/quota-inbound.mdx)
-- [Monetization policy](../articles/monetization/monetization-policy.md)

package/docs/errors/get-head-body-error.mdx

@@ -1,41 +0,0 @@
----
-title: GET/HEAD Body Error (GET_HEAD_BODY_ERROR)
----
-
-A GET or HEAD request included a body, which is not allowed. The
-[Fetch specification](https://fetch.spec.whatwg.org/) defines that GET and HEAD
-requests must not have a request body.
-
-## Why this happens
-
-The HTTP specification states that GET and HEAD requests are intended for
-retrieving resources and should not include a request body. While some HTTP
-clients allow sending a body with GET requests, the Zuplo runtime enforces the
-specification and rejects these requests.
-
-## How to fix
-
-- **Use a different HTTP method** - If the request needs to send data in the
-  body, use `POST`, `PUT`, or `PATCH` instead of `GET` or `HEAD`.
-- **Move data to query parameters** - If the request must remain a `GET`,
-  convert the body data to URL query parameters.
-- **Remove the body** - If the body was included unintentionally, remove it from
-  the request.
-
-## Common causes
-
-- **HTTP client defaults** - Some HTTP client libraries or frameworks
-  automatically attach a body to requests, even for GET methods. Check the
-  client configuration.
-- **Copied request configuration** - A request configuration copied from a POST
-  endpoint may still include a body when adapted for a GET endpoint.
-- **Framework behavior** - Certain frontend frameworks or API testing tools may
-  silently include an empty body or content-type header on GET requests.
-
-:::note
-
-This is a client-side issue. Update the request on the calling side to remove
-the body or change the HTTP method. No changes are needed on the Zuplo gateway
-configuration.
-
-:::