@lobehub/chat 1.0.6 → 1.0.8

package/docs/self-hosting/advanced/server-database.mdx

@@ -0,0 +1,387 @@
+# Deploying Server-Side Database
+
+LobeChat defaults to using a client-side database (IndexedDB) but also supports using a server-side database. LobeChat uses Postgres as the backend storage database. PostgreSQL is a powerful open-source relational database management system with high scalability and standard SQL support. It provides rich data types, concurrency control, data integrity, security, and programmability, making it suitable for complex applications and large-scale data management.
+
+This article will detail how to deploy the server-side database version of LobeChat on Vercel, including: 1) database configuration; 2) identity authentication service configuration; 3) steps for setting up the S3 storage service.
+
+<Callout type={'info'}>
+  Due to workload constraints, currently only deployment on Vercel using the server-side database
+  version is supported, with Docker version support planned for future iterations.
+</Callout>
+
+## 1. Configure the Database
+
+<Steps>
+
+### Prepare a Server-Side Database Instance and Obtain the Connection URL
+
+Before deployment, make sure you have prepared a Postgres database instance. You can choose either of the following methods:
+
+- `A.` Use Serverless Postgres instances like Vercel/Neon;
+- `B.` Use self-deployed Postgres instances like Docker.
+
+The configuration for both methods is slightly different, which will be distinguished in the next step.
+
+### Add Environment Variables in Vercel
+
+In Vercel's deployment environment variables, add the `DATABASE_URL` and other environment variables. Fill in the prepared Postgres database connection URL. The typical format for the database connection URL is `postgres://username:password@host:port/database`.
+
+<Tabs items={['Serverless Postgres', 'Node Postgres']}>
+
+<Tab>
+
+Variables required for Serverless Postgres are as follows:
+
+```shell
+# Serverless Postgres DB Url
+DATABASE_URL=
+
+# Specify the service mode as server, otherwise it won't enter the server-side database
+NEXT_PUBLIC_SERVICE_MODE=server
+```
+
+An example of how to fill in Vercel is shown below:
+
+<Image
+  alt={'Add Serverless Postgres DATABASE_URL'}
+  src={'https://github.com/lobehub/lobe-chat/assets/28616219/d4a710cd-6404-4196-90d0-cd08ca385074'}
+></Image>
+
+</Tab>
+
+<Tab>
+  Variables required for Node Postgres are as follows:
+
+```shell
+# Node Postgres DB Url
+DATABASE_URL=
+
+# Specify the Postgres database driver as node
+DATABASE_DRIVER=node
+
+# Specify the service mode as server, otherwise it won't enter the server-side database
+NEXT_PUBLIC_SERVICE_MODE=server
+```
+
+An example of how to fill in Vercel is shown below:
+
+<Image
+  alt={'Add Node Postgres DATABASE_URL'}
+  src={'https://github.com/lobehub/lobe-chat/assets/28616219/1c689738-809b-4199-b305-ba5770d39da7'}
+></Image>
+
+</Tab>
+
+</Tabs>
+
+### Add the `KEY_VAULTS_SECRET` Environment Variable
+
+After adding the `DATABASE_URL` environment variable, you need to add a `KEY_VAULTS_SECRET` environment variable. This variable is used to encrypt sensitive information like user-stored API keys. You can generate a random 32-character string as the key using `openssl rand -base64 32`.
+
+```shell
+KEY_VAULTS_SECRET=jgwsK28dspyVQoIf8/M3IIHl1h6LYYceSYNXeLpy6uk=
+```
+
+Add this to the Vercel environment variables as well.
+
+</Steps>
+
+## 2. Configure the Identity Authentication Service
+
+A server-side database needs to be paired with an identity authentication service to function properly. Therefore, the corresponding identity authentication service needs to be configured.
+
+<Callout type={'warning'}>
+  Due to workload constraints, currently only Clerk is supported as an identity authentication
+  service solution. Integration with Next-Auth for server-side database is under development.
+</Callout>
+
+<Steps>
+
+### Prepare the Clerk Identity Authentication Service
+
+Go to [Clerk](https://clerk.com?utm_source=lobehub&utm_medium=docs) to register and create an application to obtain the corresponding Public Key and Secret Key.
+
+<Callout type={'info'}>
+  If you are unfamiliar with Clerk, you can refer to [Authentication
+  Service-Clerk](/en/docs/self-hosting/advanced/authentication#clerk) for details on using Clerk.
+</Callout>
+
+### Add Public and Private Key Environment Variables in Vercel
+
+In Vercel's deployment environment variables, add the `NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY` and `CLERK_SECRET_KEY` environment variables. You can click on "API Keys" in the menu, then copy the corresponding values and paste them into Vercel's environment variables.
+
+<Image
+  alt={'Find the corresponding public and private key environment variables in Clerk'}
+  src={'https://github.com/lobehub/lobe-chat/assets/28616219/89883703-7a1a-4a11-b944-5d804544e57c'}
+></Image>
+
+The environment variables required for this step are as follows:
+
+```shell
+NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY=pk_live_xxxxxxxxxxx
+CLERK_SECRET_KEY=sk_live_xxxxxxxxxxxxxxxxxxxxxx
+```
+
+Add these variables to Vercel:
+
+<Image
+  alt={'Add Clerk public and private key environment variables in Vercel'}
+  src={'https://github.com/lobehub/lobe-chat/assets/28616219/2bfa13df-6e20-4768-97c0-4dad06c85a2f'}
+></Image>
+
+### Create and Configure a Webhook in Clerk
+
+Since we let Clerk handle user authentication and management entirely, we need Clerk to notify our application and store data in the database when there are changes in the user lifecycle (create, update, delete). We achieve this using the Webhook provided by Clerk.
+
+We need to add an endpoint in Clerk's Webhooks to inform Clerk to send notifications to this endpoint when a user's status changes.
+
+<Image
+  alt={'Add an endpoint in Clerk Webhooks'}
+  src={'https://github.com/lobehub/lobe-chat/assets/28616219/f50f47fb-5e8e-4930-bf4e-8cf6f5b8afb9'}
+>
+
+</Image>
+
+Fill in your Vercel project's URL in the endpoint, such as `https://your-project.vercel.app/api/webhooks/clerk`. Then, in the Subscribe to events section, check the three user events (`user.created`, `user.deleted`, `user.updated`), and click create.
+
+<Image
+  alt={'Configure URL and user events when adding Clerk Webhooks'}
+  src={'https://github.com/lobehub/lobe-chat/assets/28616219/0249ea56-ab17-4aa9-a56c-9ebd556c2645'}
+></Image>
+
+### Add the Webhook Secret to Vercel Environment Variables
+
+After creation, you can find the secret of this Webhook in the bottom right corner:
+
+<Image
+  alt={'View Clerk Webhook secret'}
+  src={'https://github.com/lobehub/lobe-chat/assets/28616219/fab4abb2-584b-49de-9340-813382951635'}
+></Image>
+
+The corresponding environment variable name for this secret is `CLERK_WEBHOOK_SECRET`:
+
+```shell
+CLERK_WEBHOOK_SECRET=whsec_xxxxxxxxxxxxxxxxxxxxxx
+```
+
+Add this to Vercel's environment variables:
+
+<Image
+alt={'Add Clerk Webhook secret in Vercel'}
+src={'https://github.com/lobehub/lobe-chat/assets/28616219/5fdc9479-007f-46ab-9d6e-a9603e949116'}>
+
+</Image>
+
+</Steps>
+
+You have now successfully configured the Clerk identity authentication service. Next, we will configure the S3 storage service.
+
+## 3. Configure S3 Storage Service
+
+LobeChat has long supported multimodal AI conversations, involving the function of uploading images to AI. In the client-side database solution, image files are stored as binary data in the browser's indexedDB database. However, this solution is not feasible in the server-side database. We need to configure the S3 storage service to store a large number of image files, and S3 can also serve as a storage solution for file uploads.
+
+<Callout type={'info'}>
+    In this article, S3 refers to a compatible S3 storage solution, which supports object storage systems
+  that comply with the Amazon S3 API. Common examples include Cloudflare R2 etc., all of which support
+  S3-compatible APIs.
+</Callout>
+
+<Steps>
+
+### Configure and Obtain S3 Bucket
+
+You need to go to your S3 service provider (such as AWS S3, Cloudflare R2, etc.) and create a new storage bucket. Below is an example of the creation process using Cloudflare R2.
+
+The interface of Cloudflare R2 is shown below:
+
+<Image
+  alt={'Cloudflare R2 Storage Interface'}
+  src={'https://github.com/lobehub/lobe-chat/assets/28616219/41f7f677-0153-4a96-b849-5ac9b7ebefee'}
+></Image>
+
+When creating the bucket, specify its name and then click create.
+
+<Image
+  alt={'Create Bucket in R2'}
+  src={'https://github.com/lobehub/lobe-chat/assets/28616219/9c0d184c-3169-40fa-9115-011cfffb9ca7'}
+></Image>
+
+### Obtain Environment Variables for the Bucket
+
+In the settings of the R2 storage bucket, you can view the bucket configuration information:
+
+<Image
+  alt={'View Bucket Information'}
+  src={'https://github.com/lobehub/lobe-chat/assets/28616219/2ceb210c-eca0-4439-ba27-8734d4ebb3ee'}
+></Image>
+
+The corresponding environment variables are:
+
+```shell
+# Bucket name
+S3_BUCKET=lobechat
+# Bucket request endpoint
+S3_ENDPOINT=https://0b33a03b5c993fd2f453379dc36558e5.r2.cloudflarestorage.com
+# Public domain for accessing the bucket
+NEXT_PUBLIC_S3_DOMAIN=https://s3-for-lobechat.your-domain.com
+```
+
+### Obtain S3 Key Environment Variables
+
+You need to obtain the access key for S3 so that the LobeChat server has permission to access the S3 storage service. In R2, you can configure the access key in the account details:
+
+<Image
+  alt={'View Access Key for Bucket'}
+  src={'https://github.com/lobehub/lobe-chat/assets/28616219/be0c95c0-6693-44ee-a490-7e8dfaa8b34d'}
+></Image>
+
+Click the button in the upper right corner to create an API token, then enter the API Token creation page.
+
+<Image
+  alt={'Create Corresponding API Token'}
+  src={'https://github.com/lobehub/lobe-chat/assets/28616219/7b0ea46c-5157-40a8-888f-f47664a4884f'}
+></Image>
+
+Since our server-side database needs to read and write to the S3 storage service, the permissions need to be set to read and write, and only select the bucket we created, then click create.
+
+<Image
+  alt={'Configure API Token Permissions'}
+  src={'https://github.com/lobehub/lobe-chat/assets/28616219/d6f5a918-7b50-4d6e-83a6-3894ab930ddf'}
+></Image>
+
+After creation, you can see the corresponding S3 API token.
+
+<Image
+  alt={'Copy API Token'}
+  src={'https://github.com/lobehub/lobe-chat/assets/28616219/763b18f9-2b5f-44bb-a479-9b56d46f7397'}
+></Image>
+
+The corresponding environment variables are:
+
+```shell
+S3_ACCESS_KEY_ID=9998d6757e276cf9f1edbd325b7083a6
+S3_SECRET_ACCESS_KEY=55af75d8eb6b99f189f6a35f855336ea62cd9c4751a5cf4337c53c1d3f497ac2
+```
+
+### Add the Corresponding Environment Variables in Vercel
+
+The steps to obtain the required environment variables may vary for different S3 service providers, but the obtained environment variables should be consistent in the end:
+
+```shell
+# S3 Key
+S3_ACCESS_KEY_ID=9998d6757e276cf9f1edbd325b7083a6
+S3_SECRET_ACCESS_KEY=55af75d8eb6b99f189f6a35f855336ea62cd9c4751a5cf4337c53c1d3f497ac2
+
+# Bucket name
+S3_BUCKET=lobechat
+# Bucket request endpoint
+S3_ENDPOINT=https://0b33a03b5c993fd2f453379dc36558e5.r2.cloudflarestorage.com
+# Public domain for accessing the bucket
+NEXT_PUBLIC_S3_DOMAIN=https://s3-for-lobechat.your-domain.com
+
+# Bucket region, such as us-west-1, generally not required to add, but some service providers may need to configure
+# S3_REGION=us-west-1
+```
+
+Then enter the above environment variables into Vercel's environment variables:
+
+<Image
+  alt={'Add S3 Environment Variables in Vercel'}
+  src={'https://github.com/lobehub/lobe-chat/assets/28616219/cd74152d-0ae8-44fd-b815-3307c56a3c18'}
+></Image>
+
+### Configure Cross-Origin Resource Sharing (CORS)
+
+Since S3 storage services are often on a separate domain, cross-origin access needs to be configured.
+
+In R2, you can find the CORS configuration in the settings of the storage bucket:
+
+<Image
+  alt={'Cloudflare R2 CORS Settings'}
+  src={'https://github.com/lobehub/lobe-chat/assets/28616219/ab008be7-26b2-4b78-8bd9-24301bf34d23'}
+></Image>
+
+Add a CORS rule to allow requests from your domain (in this case, `https://your-project.vercel.app`):
+
+<Image
+  alt={'Configure Allowed Site Domain'}
+  src={'https://github.com/lobehub/lobe-chat/assets/28616219/dfcc2cb3-2958-4498-a8a4-51bec584fe7d'}
+></Image>
+
+Example configuration:
+
+```json
+[
+  {
+    "AllowedOrigins": ["https://your-project.vercel.app"],
+    "AllowedMethods": ["GET", "PUT", "HEAD", "POST", "DELETE"],
+    "AllowedHeaders": ["*"]
+  }
+]
+```
+
+After configuration, click save.
+
+</Steps>
+
+## 4. Deployment and Verification
+
+After completing the above steps, the configuration of the server database should be done. Next, we can deploy LobeChat to Vercel and then visit your Vercel link to verify if the server database is working correctly.
+
+<Steps>
+  ### Redeploy the latest commit
+
+After configuring the environment variables, you need to redeploy the latest commit and wait for the deployment to complete.
+
+<Image
+  alt={'Redeploy the latest commit'}
+  src={'https://github.com/lobehub/lobe-chat/assets/28616219/b3a78112-adc8-4837-b4e3-48f67058f16e'}
+></Image>
+
+### Check if the features are working properly
+
+If you click on the login button in the top left corner and the login popup appears normally, then you have configured it successfully. Enjoy using it\~
+
+<Image alt={'User login popup'} src={'https://github.com/lobehub/lobe-chat/assets/28616219/da84edc3-46f7-4e2b-a0cd-dc33a98bf5cb'}>
+
+</Image>
+
+<Image alt={'Login success status'} src={'https://github.com/lobehub/lobe-chat/assets/28616219/9cb5150d-6e1e-4c59-9a18-4e418dce1a5d'}>
+
+</Image>
+
+</Steps>
+
+## Appendix
+
+### Overview of Server Database Environment Variables
+
+For easy copying, here is a summary of the environment variables required to configure the server data:
+
+```shell
+# Specify the service mode as server
+NEXT_PUBLIC_SERVICE_MODE=server
+
+# Postgres database URL
+DATABASE_URL=
+KEY_VAULTS_SECRET=jgwsK28dspyVQoIf8/M3IIHl1h6LYYceSYNXeLpy6uk=
+
+# Clerk related configurations
+NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY=pk_live_xxxxxxxxxxx
+CLERK_SECRET_KEY=sk_live_xxxxxxxxxxxxxxxxxxxxxx
+CLERK_WEBHOOK_SECRET=whsec_xxxxxxxxxxxxxxxxxxxxxx
+
+# S3 related configurations
+# S3 keys
+S3_ACCESS_KEY_ID=9998d6757e276cf9f1edbd325b7083a6
+S3_SECRET_ACCESS_KEY=55af75d8eb6b99f189f6a35f855336ea62cd9c4751a5cf4337c53c1d3f497ac2
+
+# Bucket name
+S3_BUCKET=lobechat
+# Bucket request endpoint
+S3_ENDPOINT=https://0b33a03b5c993fd2f453379dc36558e5.r2.cloudflarestorage.com
+# Public access domain for the bucket
+NEXT_PUBLIC_S3_DOMAIN=https://s3-for-lobechat.your-domain.com
+# Bucket region, such as us-west-1, generally not needed to add, but some service providers may require configuration
+# S3_REGION=us-west-1
+```