zuplo 7.4.4 → 7.4.6

package/docs/articles/testing.mdx

@@ -1,65 +1,97 @@
 ---
 title: Testing Your API
-sidebar_label: Testing
+sidebar_label: Overview
+description:
+  Learn how `zuplo test` runs the same TypeScript test files against your local
+  dev server, preview deployments, and production to catch configuration defects
+  that unit tests miss.
 ---
 
-Zuplo provides multiple ways to test your gateway at every stage of development.
-Whether you are iterating locally, reviewing a pull request in a preview
-environment, or gating production deployments in CI/CD, the
-[`zuplo test`](../cli/test.mdx) command and the `@zuplo/test` library give you a
-consistent testing experience.
+Your gateway holds authentication, authorization, rate limits, routing, request
+validation, and the shape of every error your consumers see. All of that is
+configuration, and configuration is exactly what unit tests cannot reach. A unit
+test can prove a JWT validation function rejects an expired token, but it cannot
+prove the policy that calls that function is attached to the route your
+customers hit.
 
-## Testing strategies overview
+`zuplo test` closes that gap. It runs plain TypeScript test files from your
+repository against a live gateway, and it takes the target as a flag. The same
+files run against your local dev server, against the preview deployment of your
+branch, and against production.
 
-| Strategy                                       | When to use                                                        | Endpoint target                      |
-| ---------------------------------------------- | ------------------------------------------------------------------ | ------------------------------------ |
-| [Local testing](#local-testing)                | Fast feedback while developing                                     | `http://localhost:9000`              |
-| [Preview environments](#preview-environments)  | Validate changes on a real deployment before merging               | `https://<branch>-<id>.zuplo.app`    |
-| [CI/CD integration](#cicd-integration-testing) | Automated gate that blocks broken changes from reaching production | Deployment URL from your CI provider |
+```bash
+npx zuplo test --endpoint http://localhost:9000
+npx zuplo test --endpoint https://your-branch-abc123.zuplo.app
+npx zuplo test --endpoint https://api.example.com --filter "smoke"
+```
 
-All three strategies use the same test files and the same `zuplo test` command.
-The only thing that changes is the `--endpoint` value.
+New to `zuplo test`? Start with
+[Get started with zuplo test](./testing-getting-started.mdx), which takes a
+fresh project to a green run.
 
-## Local testing
+## Same assertions, rising fidelity
 
-Running tests against a local development server gives the fastest feedback
-loop. Start the server with [`zuplo dev`](../cli/dev.mdx), then run your test
-suite against it.
+Structure the suite as one set of assertions run at rising levels of fidelity.
+Maintaining different tests per stage means the stage you trust least gets the
+tests nobody maintains.
 
-### Starting the local server
+| Stage                                                     | When to use                                              | Endpoint target                      |
+| --------------------------------------------------------- | -------------------------------------------------------- | ------------------------------------ |
+| Local dev                                                 | Fast feedback while developing                           | `http://localhost:9000`              |
+| [Preview environment](./testing-preview-environments.mdx) | Validate a change on a real deployment before merging    | `https://<branch>-<id>.zuplo.app`    |
+| [CI/CD gate](./github-deployment-testing.mdx)             | Block a merge when gateway behavior regresses            | Deployment URL from your CI provider |
+| Production smoke checks                                   | A small, read-only subset after deploy and on a schedule | Your production URL                  |
 
-```bash
-npx zuplo dev
-```
+All of them use the same test files and the same command. The only thing that
+changes is `--endpoint` — and, for production, a `--filter` that narrows the run
+to the read-only subset.
 
-The gateway starts on `http://localhost:9000` by default. You can change the
-port with the `--port` flag. See the [`zuplo dev` reference](../cli/dev.mdx) for
-all available options.
+:::tip
 
-### Running tests locally
+Nothing with side effects belongs in the production subset. Name those tests
+consistently (a `smoke:` prefix works well) and select them by name.
 
-With the dev server running, open a second terminal and run:
+:::
 
-```bash
-npx zuplo test --endpoint http://localhost:9000
-```
+## What to test
 
-The command discovers every `*.test.ts` file under the `tests/` folder and
-executes them against the provided endpoint.
+The interesting assertions at a gateway are the rejections, because the happy
+path hides the worst defects. A policy attached to the wrong route does not fail
+— it succeeds for the wrong caller.
 
-:::tip
+[Gateway test recipes](./testing-recipes.mdx) has a complete, copy-pasteable
+file for each of these:
 
-You can filter which tests run with the `--filter` flag. For example,
-`npx zuplo test --endpoint http://localhost:9000 --filter 'auth'` runs only
-tests whose name contains "auth".
+- **Auth rejections** — missing token, expired token, wrong audience, and a
+  valid token from one tenant asking for another tenant's object
+- **Rate limits, deterministically** — a fresh bucket per test, a sequential
+  loop, and an assertion on `429` plus `Retry-After`
+- **OpenAPI conformance** — validate live responses against the
+  `routes.oas.json` that defines the routes
+- **The error contract** — `application/problem+json` and the RFC 9457 members
+- **Routing and CORS** — unknown paths, legacy aliases, preflight responses
+- **Test data hygiene** — keep a parallel suite from corrupting itself
 
-:::
+## Local testing
+
+Running against a local development server gives the fastest feedback loop.
+Start the server with [`zuplo dev`](../cli/dev.mdx):
+
+```bash
+npx zuplo dev
+```
 
-### Testing with Zuplo services locally
+The gateway starts on `http://localhost:9000` by default. In a second terminal:
+
+```bash
+npx zuplo test --endpoint http://localhost:9000
+```
+
+### Test with Zuplo services locally
 
 Some features, such as API key authentication and rate limiting, require a
-connection to Zuplo cloud services. To use these features in local development,
-link your local project to an existing Zuplo project with
+connection to Zuplo cloud services. To exercise those policies in local
+development, link your local project to an existing Zuplo project with
 [`zuplo link`](../cli/link.mdx):
 
 ```bash
@@ -67,8 +99,8 @@ npx zuplo link
 ```
 
 Follow the prompts to select your account, project, and environment. This
-creates a `.env.zuplo` file that the dev server reads automatically. For local
-development, selecting the **development** environment is recommended.
+creates a `.env.zuplo` file that the dev server reads automatically. We
+recommend selecting the **development** environment for local development.
 
 :::warning
 
@@ -77,10 +109,10 @@ The `.env.zuplo` file can contain sensitive information. Add it to your
 
 :::
 
-Once linked, services like the API Key Authentication policy work locally using
-the same API key bucket as the linked environment. In the Zuplo Portal, open the
-[**Services**](https://portal.zuplo.com/+/account/project/services) tab in your
-project and select **API Key Service** to create API key consumers, then call
+Once linked, the API Key Authentication policy works locally using the same API
+key bucket as the linked environment. In the Zuplo Portal, open
+[**Services**](https://portal.zuplo.com/+/account/project/services) in your
+project and select **API Key Service** to create API key consumers. Then call
 your local gateway with the generated key:
 
 ```bash
@@ -88,393 +120,83 @@ curl http://localhost:9000/your-route \
   -H "Authorization: Bearer YOUR_API_KEY"
 ```
 
-For more details see
-[Connecting to Zuplo Services Locally](./local-development-services.mdx).
+For more details, see
+[connecting to Zuplo services locally](./local-development-services.mdx).
 
-### Setting environment variables locally
+### Set environment variables locally
 
 Your local dev server does not have access to the environment variables
-configured in the Zuplo Portal. Instead, create a `.env` file in your project
-root:
+configured in the Zuplo Portal. Create a `.env` file in your project root
+instead:
 
 ```text title=".env"
 MY_BACKEND_URL=https://api.example.com
 MY_SECRET=supersecret
 ```
 
-The Zuplo CLI loads these variables automatically when you run `npx zuplo dev`.
-See
-[Configuring Environment Variables Locally](./local-development-env-variables.mdx)
-for more information.
+The Zuplo CLI loads these variables automatically when you run `npx zuplo dev`,
+and the same file supplies fixture values to `zuplo test`. See
+[configuring environment variables locally](./local-development-env-variables.mdx).
 
-## Preview environments
+## Test before merge
 
-Every branch pushed to your connected source control provider can create an
-isolated [preview environment](./environments.mdx). Preview environments are
-full Zuplo deployments that behave the same as production, making them ideal for
-testing pull requests before merging.
+Every branch pushed to your connected source control provider deploys as a
+preview environment: a full Zuplo deployment on the same edge network as
+production, at its own URL. Running your suite there is the highest-fidelity
+check available before a change merges, and it needs no shared staging
+environment.
 
-### Running tests against a preview environment
+- [Testing preview environments](./testing-preview-environments.mdx) — getting
+  the URL into your tests, per-environment services and secrets, and what to
+  scope where
+- [Testing GitHub deployments](./github-deployment-testing.mdx) — trigger on the
+  `deployment_status` event and make the workflow a required status check
+- [Custom CI/CD](./custom-ci-cd.mdx) — the same pattern for GitLab, Bitbucket,
+  Azure DevOps, and CircleCI
 
-Pass the preview environment URL as the endpoint:
+:::caution{title="Poll for readiness, never sleep"}
 
-```bash
-npx zuplo test --endpoint https://your-branch-abc123.zuplo.app
-```
-
-Because preview environments run the full Zuplo runtime, including edge
-deployment, policies, and connected services, tests that pass here give high
-confidence that the changes work in production.
-
-### Combining local and remote testing
-
-For maximum coverage, test locally first for fast iteration, then run the same
-test suite against the deployed preview environment:
-
-1. Start local development and run tests against `http://localhost:9000`.
-2. Push your branch. Zuplo deploys a preview environment automatically.
-3. Run `npx zuplo test --endpoint <preview-url>` to verify behavior on the real
-   edge deployment.
-
-## CI/CD integration testing
-
-Automated tests in your CI/CD pipeline ensure that every deployment is validated
-before changes reach production. The `zuplo test` command works with any CI
-system.
-
-:::tip
-
-The examples below use the Zuplo GitHub integration. If you prefer setting up
-your own CI/CD for more fine-grained control, see
-[Custom CI/CD](./custom-ci-cd.mdx).
+Most deployment APIs report success when a deployment is _accepted_, not when
+the gateway is serving traffic. Tests fired into that gap fail, get retried, and
+the pipeline race gets misfiled as flaky tests. Poll a health route with a
+bounded deadline before running the suite — the CI guides above all show the
+loop.
 
 :::
 
-### Testing after deployment with GitHub Actions
-
-Using the Zuplo GitHub integration, tests can run after a deployment and block
-pull requests from being merged. The Zuplo Git Integration sets
-[Deployments](https://docs.github.com/en/rest/deployments/deployments) and
-[Deployment Statuses](https://docs.github.com/en/rest/deployments/statuses) for
-any push to a GitHub branch.
-
-Here is a simple GitHub Action that uses the Zuplo CLI to run the tests after
-the deployment is successful. Notice how the property
-`github.event.deployment_status.environment_url` is set to the `API_URL`
-environment variable. This is one way you can pass the URL where the preview
-environment is deployed into your tests.
-
-```yaml title="/.github/workflows/main.yaml"
-name: Main
-on: [deployment_status]
-
-jobs:
-  test:
-    name: Test API Gateway
-    runs-on: ubuntu-latest
-
-    steps:
-      - uses: actions/checkout@v4
-      - name: Use Node.js
-        uses: actions/setup-node@v4
-        with:
-          node-version-file: ".nvmrc"
-
-      - name: Run Tests
-        # Useful properties 'environment', 'state', and 'environment_url'
-        run:
-          API_URL=${{ toJson(github.event.deployment_status.environment_url) }}
-          npx zuplo test --endpoint $API_URL
-```
-
-### Requiring status checks
-
-[GitHub Branch protection](https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/defining-the-mergeability-of-pull-requests/about-protected-branches)
-can be set in order to enforce policies on when a Pull Request can be merged.
-The example below sets the "Zuplo Deployment" and "Test API Gateway" as required
-status that must pass.
-
-![Require status checks](../../public/media/testing/a1d7c322-125d-4d80-add0-fbfb65ccfea1.png)
-
-When a developer tries to merge their pull request, they will see that the tests
-haven't passed and the pull request can't be merged.
-
-![Test failure](../../public/media/testing/3f3292a3-075c-4568-afb2-00c24e704f03.png)
-
-### Local testing in CI
-
-You can also run tests against a local Zuplo server inside your CI pipeline
-before deploying anywhere. This catches issues earlier and avoids deploying
-broken changes.
-
-```yaml title="/.github/workflows/local-test-then-deploy.yaml"
-name: Local Test Then Deploy
-on:
-  push:
-    branches:
-      - main
-  pull_request:
-
-jobs:
-  local-test:
-    runs-on: ubuntu-latest
-    steps:
-      - uses: actions/checkout@v4
-      - uses: actions/setup-node@v4
-        with:
-          node-version: 24
-      - name: Install dependencies
-        run: npm install
-      - name: Start local server and run tests
-        run: |
-          npx zuplo dev &
-          DEV_PID=$!
-          echo "Waiting for local server to start..."
-          sleep 10
-          npx zuplo test --endpoint http://localhost:9000
-          kill $DEV_PID
-
-  deploy:
-    needs: local-test
-    runs-on: ubuntu-latest
-    if: github.event_name == 'push' && github.ref == 'refs/heads/main'
-    env:
-      ZUPLO_API_KEY: ${{ secrets.ZUPLO_API_KEY }}
-    steps:
-      - uses: actions/checkout@v4
-      - uses: actions/setup-node@v4
-        with:
-          node-version: 24
-      - name: Install dependencies
-        run: npm install
-      - name: Deploy to Zuplo
-        run: npx zuplo deploy --api-key "$ZUPLO_API_KEY"
-```
-
-For CI/CD examples with other providers, see
-[Custom GitHub Actions](./custom-ci-cd-github.mdx),
-[GitLab CI/CD](./custom-ci-cd-gitlab.mdx), and
-[CircleCI](./custom-ci-cd-circleci.mdx).
-
-## Writing tests
-
-Using Node.js and the Zuplo CLI, it's very easy to write tests that make
-requests to your API using `fetch` and then validate expectations with `expect`
-from [chai](https://www.chaijs.com/api/bdd/).
+## Unit tests and mocking
 
-```js title="/tests/my-test.test.ts"
-import { describe, it, TestHelper } from "@zuplo/test";
-import { expect } from "chai";
-
-describe("API", () => {
-  it("should have a body", async () => {
-    const response = await fetch(TestHelper.TEST_URL);
-    const result = await response.text();
-    expect(result).to.equal(JSON.stringify("What zup?"));
-  });
-});
-```
-
-Check out our
-[other sample tests](https://github.com/zuplo/zup-cli-example-project/tree/main/tests)
-to find one that matches your use-case.
-
-:::tip
-
-Your test files need to be under the `tests` folder and end with `.test.ts` to
-be picked up by the Zuplo CLI.
-
-:::
-
-## Tips for writing tests
-
-This section highlights some of the features of the Zuplo CLI that can help you
-write and structure your tests. Check out our
-[other sample tests](https://github.com/zuplo/zup-cli-example-project/tree/main/tests)
-to find one that matches your use-case.
-
-### Ignoring tests
-
-You can use `.ignore` and `.only` to ignore or run only specific test. The full
-example is at
-[ignore-only.test.ts](https://github.com/zuplo/zup-cli-example-project/blob/main/tests/ignore-only.test.ts)
-
-```js title="/tests/ignore-only.test.ts"
-import { describe, it } from "@zuplo/test";
-import { expect } from "chai";
-
-/**
- * This example how to use ignore and only.
- */
-describe("Ignore and only test example", () => {
-  it.ignore("This is a failing test but it's been ignored", () => {
-    expect(1 + 4).to.equals(6);
-  });
-
-  //   it.only("This is the only test that would run if it weren't commented out", () => {
-  //     expect(1 + 4).to.equals(5);
-  //   });
-});
-```
-
-### Filtering tests
-
-You can use the CLI to filter tests by name or regex. The full example is at
-[filter.test.ts](https://github.com/zuplo/zup-cli-example-project/blob/main/tests/filter.test.ts)
-
-```js title="/tests/filter.test.ts"
-import { describe, it } from "@zuplo/test";
-import { expect } from "chai";
-
-/**
- * This example shows how to filter the test by the name in the describe() function.
- * You can run `zuplo test --filter '#labelA'`
- * If you want to use regex, you can do `zuplo test --filter '/#label[Aa]/'`
- */
-describe("[#labelA #labelB] Addition", () => {
-  it("should add positive numbers", () => {
-    expect(1 + 4).to.equals(5);
-  });
-});
-```
-
-### Using environment variables in tests
-
-You can pass environment variables to your tests by setting them before the
-`zuplo test` command. Inside your test files, access them with `process.env`:
-
-```bash
-MY_API_KEY=zpka_abc123 npx zuplo test --endpoint http://localhost:9000
-```
-
-```ts title="/tests/auth.test.ts"
-import { describe, it, TestHelper } from "@zuplo/test";
-import { expect } from "chai";
-
-describe("Authentication", () => {
-  it("should reject requests without an API key", async () => {
-    const response = await fetch(`${TestHelper.TEST_URL}/my-route`);
-    expect(response.status).to.equal(401);
-  });
-
-  it("should accept requests with a valid API key", async () => {
-    const response = await fetch(`${TestHelper.TEST_URL}/my-route`, {
-      headers: {
-        Authorization: `Bearer ${process.env.MY_API_KEY}`,
-      },
-    });
-    expect(response.status).to.equal(200);
-  });
-});
-```
-
-## Writing integration tests
-
-Integration tests verify that your gateway behaves correctly end-to-end,
-including routing, policies, and backend connectivity. Because `zuplo test` runs
-against a live endpoint (local or deployed), every test is inherently an
-integration test.
-
-### Testing response status and headers
-
-```ts title="/tests/headers.test.ts"
-import { describe, it, TestHelper } from "@zuplo/test";
-import { expect } from "chai";
-
-describe("Response headers", () => {
-  it("should include CORS headers", async () => {
-    const response = await fetch(`${TestHelper.TEST_URL}/my-route`, {
-      method: "OPTIONS",
-      headers: {
-        Origin: "https://example.com",
-        "Access-Control-Request-Method": "GET",
-      },
-    });
-    expect(response.headers.get("access-control-allow-origin")).to.exist;
-  });
-
-  it("should return JSON content type", async () => {
-    const response = await fetch(`${TestHelper.TEST_URL}/my-route`);
-    expect(response.headers.get("content-type")).to.include("application/json");
-  });
-});
-```
-
-### Testing rate limiting
-
-```ts title="/tests/rate-limit.test.ts"
-import { describe, it, TestHelper } from "@zuplo/test";
-import { expect } from "chai";
-
-describe("Rate limiting", () => {
-  it("should include rate limit headers", async () => {
-    const response = await fetch(`${TestHelper.TEST_URL}/my-route`, {
-      headers: {
-        Authorization: `Bearer ${process.env.MY_API_KEY}`,
-      },
-    });
-    expect(response.headers.get("ratelimit-limit")).to.exist;
-    expect(response.headers.get("ratelimit-remaining")).to.exist;
-  });
-});
-```
-
-### Testing request validation
-
-```ts title="/tests/validation.test.ts"
-import { describe, it, TestHelper } from "@zuplo/test";
-import { expect } from "chai";
-
-describe("Request validation", () => {
-  it("should reject invalid request bodies", async () => {
-    const response = await fetch(`${TestHelper.TEST_URL}/my-route`, {
-      method: "POST",
-      headers: { "Content-Type": "application/json" },
-      body: JSON.stringify({ invalid: "data" }),
-    });
-    expect(response.status).to.equal(400);
-  });
-
-  it("should accept valid request bodies", async () => {
-    const response = await fetch(`${TestHelper.TEST_URL}/my-route`, {
-      method: "POST",
-      headers: { "Content-Type": "application/json" },
-      body: JSON.stringify({ name: "Test", email: "test@example.com" }),
-    });
-    expect(response.status).to.equal(200);
-  });
-});
-```
-
-## Unit Tests & Mocking
+`zuplo test` suites and the gateway are separate processes that communicate over
+HTTP. A test can mock anything in its own process — a helper that wraps `fetch`,
+a clock, a fixture module — but nothing inside the gateway. To control what the
+gateway sees, change the gateway instead: return canned responses with the
+[Mock API policy](../policies/mock-api-inbound.mdx), disable a policy for test
+traffic with the [Testing bypass policy](./bypass-policy-for-testing.mdx), or
+point the route at your own stub server.
 
 :::caution{title="Advanced"}
 
-Custom testing can be complicated and is best used only to test your own logic
-rather than trying to mock large portions of your gateway.
+Custom testing can be complicated. Use it to test your own logic rather than
+trying to mock large portions of your gateway.
 
 :::
 
-It's usually possible to use test frameworks like
+You can use test frameworks like
 [Mocha](https://github.com/zuplo/zuplo/tree/main/examples/test-mocks) and
-mocking tools like [Sinon](https://sinonjs.org/) to unit tests handlers,
-policies, or other modules. To see an example of how that works see this sample
-on GitHub: https://github.com/zuplo/zuplo/tree/main/examples/test-mocks
+mocking tools like [Sinon](https://sinonjs.org/) to unit test handlers,
+policies, or other modules. For an example, see this
+[sample on GitHub](https://github.com/zuplo/zuplo/tree/main/examples/test-mocks).
 
-Do note though that not everything in the Zuplo runtime can be mocked.
-Additionally, internal implementation changes might cause mocking behavior to
-change or break without notice. Unlike our public API we don't guarantee that
-mocking will remain stable between versions.
+Not everything in the Zuplo runtime can be mocked, and internal implementation
+changes might cause mocking behavior to change or break without notice. Unlike
+the public API, mocking is not guaranteed to remain stable between versions.
 
-Generally speaking, if you must write unit tests, it's best to test your logic
-separately from the Zuplo runtime. For example, write modules and functions that
-take all the arguments as input and return a result, but don't depend on any
-Zuplo runtime code.
+If you must write unit tests, test your logic separately from the Zuplo runtime.
+Write modules and functions that take all their arguments as input and return a
+result, without depending on Zuplo runtime code.
 
 For example, if you have a function that uses an environment variable and want
-to unit test it.
-
-Don't do this:
+to unit test it, don't do this:
 
 ```ts
 import { environment } from "@zuplo/runtime";
@@ -485,7 +207,7 @@ export function myFunction() {
 }
 ```
 
-Instead do this:
+Do this instead:
 
 ```ts
 export function myFunction(myVar: string) {
@@ -493,13 +215,13 @@ export function myFunction(myVar: string) {
 }
 ```
 
-Then write your test like this:
+Then write your test:
 
 ```ts
 import { myFunction } from "./myFunction";
 
 describe("myFunction", () => {
-  it("should return Hello World", () => {
+  it("returns Hello World", () => {
     expect(myFunction("World")).to.equal("Hello World");
   });
 });
@@ -509,7 +231,7 @@ describe("myFunction", () => {
 
 If you are running unit tests in a Node.js environment, you may need to polyfill
 some globals. Zuplo itself doesn't run on Node.js, but because Zuplo is built on
-standard API, testing in Node.js is possible.
+standard APIs, testing in Node.js is possible.
 
 Node.js 24, the minimum version the Zuplo CLI supports, includes the `webcrypto`
 module, which polyfills the `crypto` global. You must register this polyfill
@@ -521,3 +243,11 @@ if (typeof crypto === "undefined") {
   globalThis.crypto = webcrypto;
 }
 ```
+
+## Related
+
+- [Get started with zuplo test](./testing-getting-started.mdx)
+- [Gateway test recipes](./testing-recipes.mdx)
+- [Testing preview environments](./testing-preview-environments.mdx)
+- [Testing GitHub deployments](./github-deployment-testing.mdx)
+- [`zuplo test` CLI reference](../cli/test.mdx)