zuplo 6.67.32 → 6.68.0

package/docs/articles/s3-signed-url-uploads.mdx

@@ -0,0 +1,521 @@
+---
+title: Generating S3 Signed URLs for Large File Uploads
+sidebar_label: S3 Signed URL Uploads
+description:
+  Learn how to bypass Zuplo's 500MB limit by generating pre-signed S3 URLs for
+  direct client uploads to Amazon S3.
+tags:
+  - integrations
+  - backends
+---
+
+Zuplo's managed edge deployment has a 500MB request body size limit. For
+applications that need to handle larger files, you can generate pre-signed S3
+URLs that allow clients to upload directly to Amazon S3, bypassing the gateway
+entirely.
+
+:::tip{title="Managed Dedicated"}
+
+If you require larger request sizes you can consider Zuplo's
+[Managed Dedicated](../dedicated/overview.mdx) offering which allows custom
+request size limits. Contact your Zuplo offering which allows custom request
+size limits. Contact your Zuplo representative for more information.
+
+:::
+
+This approach offers several benefits:
+
+- Upload files larger than 500MB
+- Reduce bandwidth costs and latency
+- Offload file transfer from your gateway
+- Maintain security through temporary, scoped upload permissions
+
+## Prerequisites
+
+Before you begin, you need:
+
+- An AWS account with S3 access
+- An S3 bucket configured for your uploads
+- AWS credentials (Access Key ID and Secret Access Key) with S3 write
+  permissions
+- The AWS region where your bucket is located
+
+Store your AWS credentials securely in Zuplo environment variables:
+
+- `AWS_ACCESS_KEY_ID` - Your AWS access key
+- `AWS_SECRET_ACCESS_KEY` - Your AWS secret key
+- `AWS_REGION` - Your S3 bucket region (for example, `us-east-1`)
+- `AWS_S3_BUCKET` - Your S3 bucket name
+
+## Installing Dependencies
+
+If you are developing locally and want code completion, etc., in your project,
+install the AWS SDK for S3 to your project. These
+dependencies](../programmable-api/node-modules.mdx) are already available in the
+Zuplo runtime.
+
+```bash
+npm install @aws-sdk/client-s3 @aws-sdk/s3-request-presigner
+```
+
+## Creating the Handler
+
+Create a new module in your Zuplo project that generates pre-signed URLs. This
+handler accepts file metadata and returns a signed URL that clients can use to
+upload directly to S3.
+
+```ts title="modules/s3-signed-url.ts"
+import { S3Client, PutObjectCommand } from "@aws-sdk/client-s3";
+import { getSignedUrl } from "@aws-sdk/s3-request-presigner";
+import { ZuploContext, ZuploRequest, environment } from "@zuplo/runtime";
+
+interface UploadRequest {
+  fileName: string;
+  contentType: string;
+  // Optional: add custom metadata fields
+  metadata?: Record<string, string>;
+}
+
+interface UploadResponse {
+  uploadUrl: string;
+  key: string;
+  expiresIn: number;
+}
+
+export default async function (
+  request: ZuploRequest,
+  context: ZuploContext,
+): Promise<Response> {
+  // Parse request body
+  const body = (await request.json()) as UploadRequest;
+
+  if (!body.fileName || !body.contentType) {
+    return new Response(
+      JSON.stringify({
+        error: "fileName and contentType are required",
+      }),
+      {
+        status: 400,
+        headers: { "content-type": "application/json" },
+      },
+    );
+  }
+
+  // Configure S3 client
+  const s3Client = new S3Client({
+    region: environment.AWS_REGION,
+    credentials: {
+      accessKeyId: environment.AWS_ACCESS_KEY_ID!,
+      secretAccessKey: environment.AWS_SECRET_ACCESS_KEY!,
+    },
+  });
+
+  // Generate a unique key for the file
+  // Consider adding user ID or other identifiers to organize uploads
+  const timestamp = Date.now();
+  const key = `uploads/${timestamp}-${body.fileName}`;
+
+  // Create the put object command
+  const command = new PutObjectCommand({
+    Bucket: environment.AWS_S3_BUCKET,
+    Key: key,
+    ContentType: body.contentType,
+    Metadata: body.metadata,
+  });
+
+  try {
+    // Generate pre-signed URL that expires in 1 hour
+    const expiresIn = 3600;
+    const uploadUrl = await getSignedUrl(s3Client, command, { expiresIn });
+
+    const response: UploadResponse = {
+      uploadUrl,
+      key,
+      expiresIn,
+    };
+
+    return new Response(JSON.stringify(response), {
+      status: 200,
+      headers: { "content-type": "application/json" },
+    });
+  } catch (error) {
+    context.log.error("Failed to generate signed URL", error);
+
+    return new Response(
+      JSON.stringify({
+        error: "Failed to generate upload URL",
+      }),
+      {
+        status: 500,
+        headers: { "content-type": "application/json" },
+      },
+    );
+  }
+}
+```
+
+## Configuring the Route
+
+Add a route in your `routes.oas.json` file to expose this handler:
+
+```json
+{
+  "paths": {
+    "/uploads/request-url": {
+      "post": {
+        "summary": "Request pre-signed S3 upload URL",
+        "x-zuplo-route": {
+          "handler": {
+            "export": "default",
+            "module": "$import(@/modules/s3-signed-url)"
+          },
+          "corsPolicy": "anything-goes"
+        },
+        "requestBody": {
+          "required": true,
+          "content": {
+            "application/json": {
+              "schema": {
+                "type": "object",
+                "properties": {
+                  "fileName": {
+                    "type": "string"
+                  },
+                  "contentType": {
+                    "type": "string"
+                  },
+                  "metadata": {
+                    "type": "object"
+                  }
+                },
+                "required": ["fileName", "contentType"]
+              }
+            }
+          }
+        },
+        "responses": {
+          "200": {
+            "description": "Pre-signed upload URL"
+          }
+        }
+      }
+    }
+  }
+}
+```
+
+:::note
+
+Consider adding authentication policies to your route to ensure only authorized
+users can request upload URLs.
+
+:::
+
+## Client Implementation
+
+Here's how clients can use the generated signed URL to upload files:
+
+### JavaScript/TypeScript
+
+```ts
+// Step 1: Request the signed URL from your API
+async function requestUploadUrl(
+  fileName: string,
+  contentType: string,
+): Promise<{ uploadUrl: string; key: string }> {
+  const response = await fetch(
+    "https://your-api.zuplo.app/uploads/request-url",
+    {
+      method: "POST",
+      headers: {
+        "content-type": "application/json",
+        // Add authentication headers as needed
+        authorization: "Bearer YOUR_TOKEN",
+      },
+      body: JSON.stringify({
+        fileName,
+        contentType,
+        metadata: {
+          // Optional: add custom metadata
+          userId: "user123",
+        },
+      }),
+    },
+  );
+
+  if (!response.ok) {
+    throw new Error("Failed to get upload URL");
+  }
+
+  return response.json();
+}
+
+// Step 2: Upload the file directly to S3
+async function uploadFile(file: File): Promise<string> {
+  // Get the signed URL
+  const { uploadUrl, key } = await requestUploadUrl(file.name, file.type);
+
+  // Upload directly to S3
+  const uploadResponse = await fetch(uploadUrl, {
+    method: "PUT",
+    body: file,
+    headers: {
+      "content-type": file.type,
+    },
+  });
+
+  if (!uploadResponse.ok) {
+    throw new Error("Failed to upload file");
+  }
+
+  // Return the S3 key for reference
+  return key;
+}
+
+// Usage
+const fileInput = document.querySelector('input[type="file"]');
+fileInput.addEventListener("change", async (event) => {
+  const file = event.target.files[0];
+  if (file) {
+    try {
+      const s3Key = await uploadFile(file);
+      console.log("File uploaded successfully:", s3Key);
+    } catch (error) {
+      console.error("Upload failed:", error);
+    }
+  }
+});
+```
+
+### React Example
+
+```tsx
+import { useState } from "react";
+
+function FileUploader() {
+  const [uploading, setUploading] = useState(false);
+  const [progress, setProgress] = useState(0);
+
+  const handleUpload = async (file: File) => {
+    setUploading(true);
+    setProgress(0);
+
+    try {
+      // Request signed URL
+      const response = await fetch(
+        "https://your-api.zuplo.app/uploads/request-url",
+        {
+          method: "POST",
+          headers: {
+            "content-type": "application/json",
+            authorization: `Bearer ${getAuthToken()}`,
+          },
+          body: JSON.stringify({
+            fileName: file.name,
+            contentType: file.type,
+          }),
+        },
+      );
+
+      const { uploadUrl, key } = await response.json();
+
+      // Upload to S3 with progress tracking
+      const xhr = new XMLHttpRequest();
+
+      xhr.upload.addEventListener("progress", (e) => {
+        if (e.lengthComputable) {
+          setProgress((e.loaded / e.total) * 100);
+        }
+      });
+
+      await new Promise((resolve, reject) => {
+        xhr.addEventListener("load", () => {
+          if (xhr.status === 200) {
+            resolve(key);
+          } else {
+            reject(new Error("Upload failed"));
+          }
+        });
+
+        xhr.addEventListener("error", () => reject(new Error("Upload failed")));
+
+        xhr.open("PUT", uploadUrl);
+        xhr.setRequestHeader("Content-Type", file.type);
+        xhr.send(file);
+      });
+
+      alert("File uploaded successfully!");
+    } catch (error) {
+      console.error("Upload failed:", error);
+      alert("Upload failed. Please try again.");
+    } finally {
+      setUploading(false);
+    }
+  };
+
+  return (
+    <div>
+      <input
+        type="file"
+        onChange={(e) => {
+          const file = e.target.files?.[0];
+          if (file) handleUpload(file);
+        }}
+        disabled={uploading}
+      />
+      {uploading && <progress value={progress} max="100" />}
+    </div>
+  );
+}
+```
+
+## Security Considerations
+
+### Time-Limited URLs
+
+Pre-signed URLs expire after the specified duration (default: 1 hour in the
+example). Adjust the `expiresIn` parameter based on your needs:
+
+```ts
+// Shorter expiration for sensitive uploads
+const expiresIn = 600; // 10 minutes
+
+// Longer expiration for large files
+const expiresIn = 7200; // 2 hours
+```
+
+### File Organization
+
+Consider organizing uploads by user or purpose to simplify management:
+
+```ts
+// Organize by user and date
+const userId = request.user.sub; // From authentication
+const date = new Date().toISOString().split("T")[0];
+const key = `uploads/${userId}/${date}/${timestamp}-${body.fileName}`;
+```
+
+### Content Type Validation
+
+Validate file types before generating signed URLs:
+
+```ts
+const allowedTypes = [
+  "image/jpeg",
+  "image/png",
+  "image/gif",
+  "application/pdf",
+  "video/mp4",
+];
+
+if (!allowedTypes.includes(body.contentType)) {
+  return new Response(
+    JSON.stringify({
+      error: "File type not allowed",
+    }),
+    {
+      status: 400,
+      headers: { "content-type": "application/json" },
+    },
+  );
+}
+```
+
+### File Size Limits
+
+While S3 can handle files up to 5TB, you may want to enforce size limits. Add
+validation on the client side and consider implementing S3 bucket policies to
+enforce maximum object sizes.
+
+## Advanced Features
+
+### Multipart Upload for Very Large Files
+
+For files larger than 5GB, use multipart uploads. This requires generating
+signed URLs for each part:
+
+```ts
+import {
+  CreateMultipartUploadCommand,
+  UploadPartCommand,
+} from "@aws-sdk/client-s3";
+
+// Create multipart upload
+const multipartCommand = new CreateMultipartUploadCommand({
+  Bucket: environment.AWS_S3_BUCKET,
+  Key: key,
+  ContentType: body.contentType,
+});
+
+const multipartUpload = await s3Client.send(multipartCommand);
+const uploadId = multipartUpload.UploadId;
+
+// Generate signed URLs for each part
+// Client uploads each part separately, then completes the upload
+```
+
+### Upload Notifications
+
+Set up S3 event notifications to trigger actions when uploads complete:
+
+1. Configure S3 bucket notifications to send events to SQS, SNS, or Lambda
+2. Process uploaded files asynchronously
+3. Update your database with file metadata
+4. Run virus scanning or other validations
+
+### Pre-signed POST URLs
+
+For browser uploads with additional security, use pre-signed POST URLs instead
+of PUT:
+
+```ts
+import { createPresignedPost } from "@aws-sdk/s3-presigned-post";
+
+const { url, fields } = await createPresignedPost(s3Client, {
+  Bucket: environment.AWS_S3_BUCKET,
+  Key: key,
+  Conditions: [
+    ["content-length-range", 0, 10485760], // 10MB max
+    ["starts-with", "$Content-Type", "image/"],
+  ],
+  Expires: 3600,
+});
+
+// Client submits multipart/form-data with the fields
+```
+
+## Troubleshooting
+
+### CORS Issues
+
+If clients receive CORS errors when uploading to S3, configure CORS on your S3
+bucket:
+
+```json
+[
+  {
+    "AllowedHeaders": ["*"],
+    "AllowedMethods": ["PUT", "POST"],
+    "AllowedOrigins": ["https://your-domain.com"],
+    "ExposeHeaders": ["ETag"],
+    "MaxAgeSeconds": 3000
+  }
+]
+```
+
+### Invalid Signature Errors
+
+Ensure your AWS credentials are correct and have the necessary permissions. The
+IAM user or role needs `s3:PutObject` permission for the bucket.
+
+### Clock Skew
+
+Pre-signed URLs are sensitive to time differences. Ensure your systems have
+accurate time synchronization via NTP.
+
+## Related Resources
+
+- [Custom Handler Documentation](../handlers/custom-handler.mdx)
+- [Environment Variables](./environment-variables.mdx)
+- [AWS S3 Documentation](https://docs.aws.amazon.com/s3/)
+- [AWS SDK for JavaScript v3](https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/)

package/docs/articles/secure-tunnel.mdx

@@ -0,0 +1,84 @@
+---
+title: Secure Tunnel
+---
+
+For customers running on bare metal, on-premises, or other non-cloud providers
+tunnels provides a way to secure your backend without mTLS or IAM.
+
+The way this system works is by deploying a small service inside your network or
+VPC that makes a secure outbound connection to Zuplo's infrastructure. Your
+Zuplo API Gateway can then use this tunnel to securely route traffic to your
+private API. The benefits of a secure tunnel are:
+
+1. Because the tunnel makes an outbound connection, there is no need for your
+   API to be exposed on the internet at all.
+2. All traffic between Zuplo and your API is fully encrypted.
+3. You eliminate the need to configure complex ingress, firewall, or other types
+   of policies to route traffic into your API. Simply install the tunnel and
+   Zuplo takes care of the rest.
+
+## How does the Tunnel Work?
+
+The Zuplo tunnel can run on virtually any infrastructure. The most common way
+users install the tunnel is as a Docker container, but we can provide you a
+build in virtually any format (for example an Azure VM, etc.). The tunnel itself
+is a lightweight service that when started makes an outbound connection to the
+Zuplo network and then through to your Zuplo Gateway.
+
+When the tunnel service connects to the Zuplo network, traffic from your gateway
+can be routed to internal services running in your network or VPC. For example,
+if your API is running on the internal DNS address `external-api.local`, the
+tunnel will route traffic from the Zuplo API Gateway to your internal service
+based only on the code and policies you have set up in your Zuplo Gateway.
+
+The example below illustrates how the Zuplo tunnel would be configured in an AWS
+ECS Cluster. Notice that there is no public IP address or ingress traffic in
+this configuration. This is a completely private VPC. The tunnel makes an
+outbound connection to the Zuplo Gateway and then uses internal DNS to route
+requests to the Private API.
+
+![System diagram](../../public/media/secure-tunnel/fefdc7fb-f3b6-4908-8485-3d20cb769cfd.png)
+
+## Is this Secure?
+
+Zuplo builds on top of many different tools to ensure that your gateway and API
+stay secure. Each tunnel uses a secret key that allows it to securely connect to
+Zuplo's network. Under the hood, Zuplo relies on Cloudflare's network for
+establishing secure and reliable tunnel connections. Each tunnel is configured
+with unique access policies that allow only the Zuplo Gateway that you have
+authorized to make connections over that tunnel. Every incoming request is
+terminated using Cloudflare's network which provides sophisticated DDoS, bot,
+and threat protections. Next, the request is routed through your Zuplo Gateway
+which can be configured with all policies and code you require to control access
+to your API. By default, no requests will be routed from your gateway to your
+API until you configure routes, URL rewrites, and policies in your Gateway.
+
+All traffic is terminated at the edge with SSL certificates and encrypted
+through the entire route to your API. All requests that your gateway makes to
+Zuplo's internal services like API Key Management or Rate Limiting are also
+transported over secure and encrypted tunnels.
+
+Every request is logged and you can configure Zuplo's logs to push to the log
+service of your choice.
+
+## How will tunnels perform?
+
+Most customers are fine running two instances of the tunnel service for
+redundancy in the event one pod/service fails. Each tunnel is able to handle
+millions of requests per minute. For customers that require additional scale,
+simply increase the number of tunnel instances you are running or configure auto
+scaling on your deployment. It's unlikely that the tunnel will become the
+bottleneck in your traffic before other factors, but if you do run into any
+issues [contact support](mailto:support@zuplo.com) and we will work out a
+solution that meets your scale requirements.
+
+## How should I Configure the Tunnel?
+
+You should run your tunnel with an IAM role or other network policies that only
+allow the tunnel to make requests to the network services that you want your
+Gateway to access. This can be done in a variety of ways depending on your
+setup. IAM roles, network segregation, and internal service meshes are common
+means of controlling which services the tunnel can access.
+
+For details on how to configure a tunnel on your network see
+[the setup guide](tunnel-setup.mdx).