zuplo 6.70.70 → 6.70.71

package/docs/rate-limiting/how-it-works.md

@@ -0,0 +1,225 @@
+---
+title: How rate limiting works
+sidebar_label: How it works
+description:
+  Understand Zuplo's sliding window rate limiter — how requests are counted,
+  what each rateLimitBy mode does, the Complex Rate Limiting policy, and every
+  configuration option.
+---
+
+This page covers the mechanics behind Zuplo's rate limiter: how requests are
+counted, what each `rateLimitBy` mode does in detail, and every configuration
+option available. If you just want to add a rate limit to your API, start with
+the [Getting Started guide](./getting-started.mdx) instead — this page is the
+deep dive you can read alongside or after it.
+
+Zuplo's rate limiter uses a **sliding window algorithm** enforced globally
+across all edge locations. Unlike a fixed window algorithm (which resets
+counters at fixed intervals and can allow bursts at window boundaries), the
+sliding window continuously tracks requests over a rolling time period. This
+produces smoother, more predictable throttling behavior.
+
+## Key terms
+
+A few terms show up repeatedly in the rate limiting docs. They are related but
+not interchangeable.
+
+- **Counter (or bucket)** — The running tally Zuplo keeps for a single caller
+  and a single policy. Each unique combination of policy `name` and caller
+  identifier gets its own counter. Two different policies tracking the same
+  caller do _not_ share a counter; two different callers under the same policy
+  do not share a counter either.
+- **Rate limit key** — The string value that identifies a caller for bucketing.
+  For `rateLimitBy: "ip"` the key is the client's IP address; for `"user"` it is
+  `request.user.sub`; for `"function"` it is whatever your custom function
+  returns as `CustomRateLimitDetails.key`; for `"all"` there is a single
+  implicit key shared by every request to the route.
+- **`identifier` option** — A field in the policy's configuration that points
+  Zuplo at your custom TypeScript function when `rateLimitBy` is `"function"`.
+  Zuplo calls that function on each request, and the function returns a
+  `CustomRateLimitDetails` object whose `key` property becomes the rate limit
+  key. In short: `identifier` is _where the function lives_; `key` is _what the
+  function returns_.
+
+## How `rateLimitBy` works
+
+The `rateLimitBy` option determines how the rate limiter groups requests into
+buckets. Both the standard
+[Rate Limiting policy](../policies/rate-limit-inbound.mdx) and the
+[Complex Rate Limiting policy](../policies/complex-rate-limit-inbound.mdx)
+support the same four modes.
+
+### `ip`
+
+Groups requests by the client's IP address. No authentication is required. This
+is the simplest option and works well for public APIs or as a first layer of
+protection.
+
+:::caution
+
+Multiple clients behind the same corporate proxy, cloud NAT, or shared Wi-Fi
+network can share a single IP address. In these cases, IP-based rate limiting
+can unfairly throttle unrelated users. For authenticated APIs, prefer
+`rateLimitBy: "user"` instead.
+
+:::
+
+### `user`
+
+Groups requests by the authenticated user's identity (`request.user.sub`). When
+using [API key authentication](../articles/api-key-authentication.mdx), the
+`sub` value is the consumer name you assigned when creating the API key. When
+using JWT authentication, it comes from the token's `sub` claim.
+
+This is the recommended mode for authenticated APIs because it ties limits to
+the actual consumer rather than a shared IP address.
+
+:::note
+
+The `user` mode requires an authentication policy (such as API key or JWT
+authentication) earlier in the policy pipeline. If no authenticated user is
+present on the request, the policy returns an error. See
+[Getting Started §5](./getting-started.mdx#5-rate-limit-authenticated-users) for
+a full authenticated pipeline example.
+
+:::
+
+### `function`
+
+Groups requests using a custom TypeScript function that you provide. The
+function returns a `CustomRateLimitDetails` object containing a grouping key
+and, optionally, overridden values for `requestsAllowed` and
+`timeWindowMinutes`. See
+[Custom rate limit functions](#custom-rate-limit-functions) below for the
+function signature and field reference.
+
+### `all`
+
+Applies a single shared counter across all requests to the route, regardless of
+who makes them. Use this for global rate limits on endpoints that call
+resource-constrained backends.
+
+## Custom rate limit functions
+
+When `rateLimitBy` is set to `"function"`, Zuplo calls a TypeScript function you
+provide on every request. The function receives the request, context, and policy
+name, and returns a `CustomRateLimitDetails` object describing how to count that
+request.
+
+```ts
+import {
+  CustomRateLimitDetails,
+  ZuploContext,
+  ZuploRequest,
+} from "@zuplo/runtime";
+
+export function rateLimit(
+  request: ZuploRequest,
+  context: ZuploContext,
+  policyName: string,
+): CustomRateLimitDetails | undefined {
+  return {
+    key: request.user.sub,
+    requestsAllowed: 100,
+    timeWindowMinutes: 1,
+  };
+}
+```
+
+### `CustomRateLimitDetails`
+
+- `key` (required) — The string used to group requests into rate limit buckets.
+- `requestsAllowed` (optional) — Overrides the policy's `requestsAllowed` value
+  for this request.
+- `timeWindowMinutes` (optional) — Overrides the policy's `timeWindowMinutes`
+  value for this request.
+
+Returning `undefined` skips rate limiting for the request entirely — useful for
+health checks or privileged callers. The function can also be `async` if you
+need to await a database lookup or external service call.
+
+Wire the function into the policy using the `identifier` option. The policy's
+configured `requestsAllowed` and `timeWindowMinutes` serve as defaults; the
+function can override them per request.
+
+For concrete walkthroughs (tier-based, route-based, method-based,
+database-backed, selective bypass), see
+[Dynamic Rate Limiting](./dynamic-rate-limiting.mdx). For an advanced
+database-backed example with caching, see
+[Per-user rate limiting with a database](./per-user-rate-limits-using-db.mdx).
+
+## Additional options
+
+Both rate limiting policies support the following additional options:
+
+- `headerMode` — Set to `"retry-after"` (default) to include the `Retry-After`
+  header in 429 responses, or `"none"` to omit it. The `Retry-After` value is
+  returned as a number of seconds (delay-seconds format).
+- `mode` — Set to `"strict"` (default) or `"async"`. In **strict** mode, the
+  request is held until the rate limit check completes — the backend is never
+  called if the limit is exceeded. This adds some latency to every request
+  because the check hits a globally distributed rate limit service. In **async**
+  mode, the request proceeds to the backend in parallel with the rate limit
+  check. This minimizes added latency but means some requests may get through
+  even after the limit is exceeded. Async mode is a good fit when low latency
+  matters more than exact enforcement.
+- `throwOnFailure` — Controls behavior when the rate limit service is
+  unreachable. When set to `false` (default), requests are allowed through
+  (fail-open). When set to `true`, the policy returns an error to the client.
+  The fail-open default prevents a rate limit service outage from blocking all
+  traffic to your API.
+
+## Complex Rate Limiting policy
+
+The [Complex Rate Limiting policy](../policies/complex-rate-limit-inbound.mdx)
+supports **multiple named counters** in a single policy. Each counter tracks a
+different resource or unit of work.
+
+```json
+{
+  "name": "my-complex-rate-limit-policy",
+  "policyType": "complex-rate-limit-inbound",
+  "handler": {
+    "export": "ComplexRateLimitInboundPolicy",
+    "module": "$import(@zuplo/runtime)",
+    "options": {
+      "rateLimitBy": "user",
+      "timeWindowMinutes": 1,
+      "limits": {
+        "requests": 100,
+        "compute": 500
+      }
+    }
+  }
+}
+```
+
+Override counter increments programmatically per request with
+`ComplexRateLimitInboundPolicy.setIncrements()`. This suits usage-based pricing,
+where different endpoints consume different amounts of a resource (for example,
+counting compute units or tokens instead of raw requests).
+
+## Related resources
+
+**Go deeper on configuration:**
+
+- [Rate Limiting policy reference](../policies/rate-limit-inbound.mdx) — Every
+  option for the standard policy.
+- [Complex Rate Limiting policy reference](../policies/complex-rate-limit-inbound.mdx)
+  — Multi-counter limits for usage-based pricing (enterprise).
+
+**Learn by example:**
+
+- [Dynamic Rate Limiting](./dynamic-rate-limiting.mdx) — Tiered limits by
+  customer type.
+- [Per-user rate limiting with a database](./per-user-rate-limits-using-db.mdx)
+  — Look up limits at request time using ZoneCache and a database.
+
+**Combine with other policies:**
+
+- [Combining Policies](./combining-policies.mdx) — Stack multiple rate limits,
+  and pair rate limiting with quotas or monetization.
+- [Quota policy](../policies/quota-inbound.mdx) — Monthly or billing-period
+  usage caps.
+- [Monetization policy](../articles/monetization/monetization-policy.md) —
+  Subscription-based access control and metering.

package/docs/rate-limiting/monitoring-and-troubleshooting.mdx

@@ -0,0 +1,243 @@
+---
+title: Monitoring and troubleshooting rate limits
+sidebar_label: Monitoring & troubleshooting
+description:
+  Monitor rate limit events, debug unexpected 429 responses, and understand
+  failure modes.
+---
+
+Rate limiting only delivers value when you can observe it in action. Without
+visibility into which consumers hit limits, how often requests are rejected, and
+whether the rate limit service itself is healthy, you are operating blind. This
+guide covers how to monitor rate limit activity, understand failure modes,
+choose the right enforcement mode, and diagnose common issues.
+
+## Monitoring rate limit events
+
+Zuplo produces structured logs for every request, including those rejected with
+a `429 Too Many Requests` status code. Ship these logs to an external provider
+to build dashboards and alerts around rate limit activity.
+
+### Setting up log shipping
+
+Configure a [logging plugin](../articles/logging.mdx) in your `zuplo.runtime.ts`
+file to send logs to your observability platform. Zuplo supports AWS CloudWatch,
+Datadog, Dynatrace, Google Cloud Logging, Loki, New Relic, Splunk, Sumo Logic,
+and VMware Log Insight. You can also build a
+[custom logging plugin](../articles/custom-logging-example.mdx) for unsupported
+providers.
+
+### Filtering for rate-limited requests
+
+Every log entry includes default fields you can filter on:
+
+- **`requestId`** -- Correlate a specific rejected request end-to-end using the
+  `zp-rid` response header.
+- **`environment`** and **`environmentStage`** -- Distinguish between
+  `production`, `preview`, and `working-copy` environments.
+
+To break down rate-limited requests by consumer or IP, add custom log properties
+in a policy that runs before or alongside the rate limit check:
+
+```ts
+import { ZuploContext, ZuploRequest } from "@zuplo/runtime";
+
+export default async function policy(
+  request: ZuploRequest,
+  context: ZuploContext,
+) {
+  // Tag every log entry with the consumer identity for filtering
+  context.log.setLogProperties!({
+    rateLimitIdentity:
+      request.user?.sub ?? request.headers.get("true-client-ip") ?? "unknown",
+  });
+  return request;
+}
+```
+
+This adds a `rateLimitIdentity` field to all log entries for the request, making
+it straightforward to group 429 responses by consumer in your logging dashboard.
+
+### Setting up alerts
+
+Configure alerts in your logging provider for the following conditions:
+
+- **Spike in 429 responses** -- A sudden increase may indicate a
+  misconfiguration, an attack, or a legitimate traffic surge.
+- **429 rate exceeding a threshold** -- If more than a small percentage of
+  requests return 429, the rate limit may be set too low for normal traffic.
+- **Zero 429 responses over an extended period** -- If you expect rate limiting
+  to be active but see no rejections, the policy may not be attached to the
+  correct routes.
+
+### Metrics plugins
+
+For quantitative monitoring, Zuplo supports
+[metrics plugins](../articles/metrics-plugins.mdx) that send request latency,
+request size, and response size data to Datadog, Dynatrace, New Relic, or any
+OpenTelemetry-compatible collector. While these metrics do not track rate limit
+counters directly, the `statusCode` dimension (when enabled) allows you to chart
+429 response rates alongside overall request volume.
+
+## Understanding failure modes
+
+The rate limiting policies depend on a globally distributed rate limit service
+to track request counters. Understanding what happens when that service is
+unreachable helps you make the right availability tradeoff.
+
+### Fail-open (default)
+
+By default, `throwOnFailure` is set to `false`. If the rate limit service is
+unreachable, the policy allows the request through. This fail-open behavior
+prevents a rate limit service outage from blocking all traffic to your API.
+
+The tradeoff is that during an outage, rate limits are not enforced and clients
+can exceed their configured thresholds.
+
+### Fail-closed
+
+Set `throwOnFailure` to `true` to return an error when the rate limit service is
+unreachable. This guarantees that no request bypasses rate limiting, but it
+means a service disruption blocks all traffic on routes using that policy.
+
+```json
+{
+  "options": {
+    "rateLimitBy": "user",
+    "requestsAllowed": 100,
+    "timeWindowMinutes": 1,
+    "throwOnFailure": true
+  }
+}
+```
+
+:::warning
+
+Only use `throwOnFailure: true` when allowing unlimited traffic is more
+dangerous than rejecting all traffic. For most APIs, the fail-open default is
+the safer choice.
+
+:::
+
+### Detecting fail-open conditions
+
+Because fail-open requests succeed with a `200` (or other normal status code),
+they do not produce a 429 log entry. To detect when the rate limit service is
+unreachable, monitor for a sudden drop in 429 responses during periods when you
+expect rate limiting to be active. A complete absence of 429s alongside steady
+or increasing traffic volume is a strong signal that the service is in fail-open
+mode.
+
+## Strict vs. async mode in production
+
+The `mode` option controls whether the rate limit check blocks the request or
+runs in parallel with it.
+
+### Strict mode (default)
+
+In `strict` mode, every request waits for the rate limit service to confirm
+whether the request is within limits before proceeding to the backend. This
+provides exact enforcement -- no request exceeds the configured threshold.
+
+The tradeoff is added latency on every request due to the round-trip to the rate
+limit service.
+
+### Async mode
+
+In `async` mode, the request proceeds to the backend immediately while the rate
+limit check runs in parallel. If the check determines the limit is exceeded, the
+result applies to the _next_ request, not the current one.
+
+This means some requests may get through after the limit is reached. In
+practice, the overshoot depends on your request rate and the latency of the rate
+limit check. For an API receiving 100 requests per second with a 10ms check
+time, approximately one extra request may slip through per window.
+
+:::tip
+
+Use `async` mode when low latency matters more than exact enforcement -- for
+example, on high-throughput public endpoints where a few extra requests over the
+limit are acceptable. Use `strict` mode when precise enforcement is required,
+such as billing-sensitive endpoints or APIs with hard backend capacity limits.
+
+:::
+
+## Common troubleshooting scenarios
+
+### Unexpected 429 responses
+
+**Shared IP addresses.** When `rateLimitBy` is set to `"ip"`, multiple clients
+behind the same corporate proxy, cloud NAT, or shared Wi-Fi share a single rate
+limit bucket. One heavy user exhausts the limit for everyone on that IP. Switch
+to `rateLimitBy: "user"` for authenticated APIs to avoid this.
+
+**Missing authentication policy.** The `"user"` mode requires an authentication
+policy (such as API Key Authentication or JWT) earlier in the policy pipeline to
+populate `request.user`. If no authentication policy runs first, the rate limit
+policy returns an error instead of applying per-user limits. Verify that
+authentication appears before rate limiting in the route's inbound policy list.
+
+**Multiple rate limit policies on the same route.** If a route has both a
+per-minute and a per-hour rate limit policy, a request can be rejected by either
+one. Check all rate limit policies attached to the route, and verify the
+ordering (longest time window first, then shorter durations).
+
+**Lower limits than expected.** If you use a custom `rateLimitBy: "function"`,
+verify that the function returns the expected `requestsAllowed` and
+`timeWindowMinutes` values. Log the returned values during development to
+confirm the function resolves correctly for each consumer.
+
+### Rate limits not applying
+
+**Policy not attached to the route.** Defining a rate limit policy in
+`policies.json` does not activate it. The policy name must appear in the
+`policies.inbound` array of each route in `routes.oas.json` where you want it
+enforced. Verify the route configuration.
+
+**Typo in the policy name.** The policy name in `routes.oas.json` must exactly
+match the `name` field in `policies.json`. A mismatched name silently skips the
+policy. Check for case sensitivity and extra whitespace.
+
+**Custom function returning `undefined`.** When `rateLimitBy` is set to
+`"function"` and the identifier function returns `undefined`, rate limiting is
+skipped for that request entirely. This is by design -- it allows you to
+selectively exempt certain requests -- but it can cause confusion if the
+function has an unhandled code path that returns `undefined` unintentionally.
+
+### Different behavior across environments
+
+Rate limit counters are scoped per environment. Production, preview, and
+working-copy environments each maintain their own separate counters. A request
+that is rate-limited in production does not affect the counter in a preview
+environment, and vice versa.
+
+This means:
+
+- Testing rate limits in a preview branch does not interfere with production
+  traffic.
+- Rate limit thresholds you observe in a low-traffic preview environment may
+  behave differently under production load.
+- After deploying a new environment, counters start fresh.
+
+:::note
+
+If you observe rate limits triggering in one environment but not another,
+confirm that both environments use the same policy configuration and that the
+traffic volume is comparable.
+
+:::
+
+## Related resources
+
+- [Rate Limit Exceeded error](../errors/rate-limit-exceeded.mdx) --
+  Understanding the 429 response format and client-side remediation
+- [How rate limiting works](./how-it-works.md) -- Algorithm details,
+  `rateLimitBy` modes, and combining policies
+- [Logging](../articles/logging.mdx) -- Configuring log shipping to external
+  providers
+- [Metrics Plugins](../articles/metrics-plugins.mdx) -- Sending request metrics
+  to Datadog, Dynatrace, New Relic, or OpenTelemetry
+- [Proactive monitoring](../articles/monitoring-your-gateway.mdx) -- Health
+  checks and end-to-end gateway monitoring
+- [Troubleshooting](../articles/troubleshooting.md) -- General gateway
+  troubleshooting guide

package/docs/{articles → rate-limiting}/per-user-rate-limits-using-db.mdx

@@ -1,6 +1,6 @@
 ---
-title: Per user rate-limiting using a database and the ZoneCache
-sidebar_label: "Per-User Rate Limits"
+title: Per-user rate limiting using a database and the ZoneCache
+sidebar_label: "Per-user rate limits"
 description:
   Learn how to implement advanced dynamic rate limiting with database lookups
   and ZoneCache for improved performance.
@@ -9,23 +9,22 @@ tags:
   - caching
 ---
 
-In this example we show a more advanced implementation of
-[dynamic rate limiting](./step-5-dynamic-rate-limiting.mdx). It uses a database
-lookup to get the customer details and combines that with the ZoneCache to
-improve performance, reduce latency and lower the load on the database.
+This example shows a more advanced implementation of
+[dynamic rate limiting](./dynamic-rate-limiting.mdx). It uses a database lookup
+to get the customer details and combines that with the ZoneCache to improve
+performance, reduce latency and lower the load on the database.
 
-In this example we use [Supabase](https://supabase.com) as the database but you
-could use your own API, [Xata](https://xata.io),
-[Firebase](https://firebase.com) etc. The implementation will be similar for
-all.
+This example uses [Supabase](https://supabase.com) as the database, but you
+could use your own API, [Xata](https://xata.io), or
+[Firebase](https://firebase.com). The implementation is similar for all.
 
 If you haven't already, check out the
 [rate-limiting policy](../policies/rate-limit-inbound.mdx) and the
-[dynamic rate limiting quickstart](./step-5-dynamic-rate-limiting.mdx). Then you
-should be oriented to how dynamic rate limiting works.
+[dynamic rate limiting guide](./dynamic-rate-limiting.mdx). Then you should be
+oriented to how dynamic rate limiting works.
 
-Below is a full implementation of a custom rate limiting function. In our
-example this is a module called `per-user-rate-limiting.ts`.
+Below is a full implementation of a custom rate limiting function. In this
+example it is a module called `per-user-rate-limiting.ts`.
 
 ```ts
 import {
@@ -47,9 +46,18 @@ export async function rateLimitKey(
   context: ZuploContext,
   policyName: string,
 ): Promise<CustomRateLimitDetails> {
-  // We'll get the customer ID from the user data.
-  // This might be from a JWT or API Key metadata
-  const customerId = request.user.data.customerId;
+  // Get the customer ID from the user data.
+  // This might be from a JWT or API Key metadata.
+  // Ensure an authentication policy runs before this.
+  const customerId = request.user?.data?.customerId;
+  if (!customerId) {
+    context.log.error("No customerId found on request.user.data");
+    return {
+      key: request.user?.sub ?? "unknown",
+      requestsAllowed: FALLBACK_REQUESTS_ALLOWED,
+      timeWindowMinutes: 1,
+    };
+  }
 
   // We don't want to hit the database on every request
   // So we'll use the fast zone cache to cache this data
@@ -94,17 +102,21 @@ export async function rateLimitKey(
 The above function can be applied to a rate limiter with the following
 configuration in policies
 
-```json
+```json title="config/policies.json"
 {
-  "export": "RateLimitInboundPolicy",
-  "module": "$import(@zuplo/runtime)",
-  "options": {
-    "rateLimitBy": "function",
-    "requestsAllowed": 2,
-    "timeWindowMinutes": 1,
-    "identifier": {
-      "export": "rateLimitKey",
-      "module": "$import(./modules/per-user-rate-limiting)"
+  "name": "my-per-user-rate-limit-policy",
+  "policyType": "rate-limit-inbound",
+  "handler": {
+    "export": "RateLimitInboundPolicy",
+    "module": "$import(@zuplo/runtime)",
+    "options": {
+      "rateLimitBy": "function",
+      "requestsAllowed": 100,
+      "timeWindowMinutes": 1,
+      "identifier": {
+        "export": "rateLimitKey",
+        "module": "$import(./modules/per-user-rate-limiting)"
+      }
     }
   }
 }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "zuplo",
-  "version": "6.70.70",
+  "version": "6.70.71",
   "type": "module",
   "description": "The programmable API Gateway",
   "author": "Zuplo, Inc.",
@@ -19,9 +19,9 @@
     "zuplo": "zuplo.js"
   },
   "dependencies": {
-    "@zuplo/cli": "6.70.70",
-    "@zuplo/core": "6.70.70",
-    "@zuplo/runtime": "6.70.70",
+    "@zuplo/cli": "6.70.71",
+    "@zuplo/core": "6.70.71",
+    "@zuplo/runtime": "6.70.71",
     "@zuplo/test": "1.4.0"
   }
 }