@lobehub/chat 1.68.0 → 1.68.2

package/CHANGELOG.md

@@ -2,6 +2,56 @@
 
 # Changelog
 
+### [Version 1.68.2](https://github.com/lobehub/lobe-chat/compare/v1.68.1...v1.68.2)
+
+<sup>Released on **2025-03-03**</sup>
+
+#### 💄 Styles
+
+- **misc**: Add build-in web search support for Wenxin & Hunyuan.
+
+<br/>
+
+<details>
+<summary><kbd>Improvements and Fixes</kbd></summary>
+
+#### Styles
+
+- **misc**: Add build-in web search support for Wenxin & Hunyuan, closes [#6617](https://github.com/lobehub/lobe-chat/issues/6617) ([dfd1f09](https://github.com/lobehub/lobe-chat/commit/dfd1f09))
+
+</details>
+
+<div align="right">
+
+[![](https://img.shields.io/badge/-BACK_TO_TOP-151515?style=flat-square)](#readme-top)
+
+</div>
+
+### [Version 1.68.1](https://github.com/lobehub/lobe-chat/compare/v1.68.0...v1.68.1)
+
+<sup>Released on **2025-03-03**</sup>
+
+#### 🐛 Bug Fixes
+
+- **misc**: Fix page crash with crawler error.
+
+<br/>
+
+<details>
+<summary><kbd>Improvements and Fixes</kbd></summary>
+
+#### What's fixed
+
+- **misc**: Fix page crash with crawler error, closes [#6662](https://github.com/lobehub/lobe-chat/issues/6662) ([0c24251](https://github.com/lobehub/lobe-chat/commit/0c24251))
+
+</details>
+
+<div align="right">
+
+[![](https://img.shields.io/badge/-BACK_TO_TOP-151515?style=flat-square)](#readme-top)
+
+</div>
+
 ## [Version 1.68.0](https://github.com/lobehub/lobe-chat/compare/v1.67.2...v1.68.0)
 
 <sup>Released on **2025-03-03**</sup>

package/README.md

@@ -191,14 +191,14 @@ We have implemented support for the following model service providers:
 - **[Bedrock](https://lobechat.com/discover/provider/bedrock)**: Bedrock is a service provided by Amazon AWS, focusing on delivering advanced AI language and visual models for enterprises. Its model family includes Anthropic's Claude series, Meta's Llama 3.1 series, and more, offering a range of options from lightweight to high-performance, supporting tasks such as text generation, conversation, and image processing for businesses of varying scales and needs.
 - **[Google](https://lobechat.com/discover/provider/google)**: Google's Gemini series represents its most advanced, versatile AI models, developed by Google DeepMind, designed for multimodal capabilities, supporting seamless understanding and processing of text, code, images, audio, and video. Suitable for various environments from data centers to mobile devices, it significantly enhances the efficiency and applicability of AI models.
 - **[DeepSeek](https://lobechat.com/discover/provider/deepseek)**: DeepSeek is a company focused on AI technology research and application, with its latest model DeepSeek-V2.5 integrating general dialogue and code processing capabilities, achieving significant improvements in human preference alignment, writing tasks, and instruction following.
+- **[PPIO](https://lobechat.com/discover/provider/ppio)**: PPIO supports stable and cost-efficient open-source LLM APIs, such as DeepSeek, Llama, Qwen etc.
 - **[HuggingFace](https://lobechat.com/discover/provider/huggingface)**: The HuggingFace Inference API provides a fast and free way for you to explore thousands of models for various tasks. Whether you are prototyping for a new application or experimenting with the capabilities of machine learning, this API gives you instant access to high-performance models across multiple domains.
 - **[OpenRouter](https://lobechat.com/discover/provider/openrouter)**: OpenRouter is a service platform providing access to various cutting-edge large model interfaces, supporting OpenAI, Anthropic, LLaMA, and more, suitable for diverse development and application needs. Users can flexibly choose the optimal model and pricing based on their requirements, enhancing the AI experience.
 - **[Cloudflare Workers AI](https://lobechat.com/discover/provider/cloudflare)**: Run serverless GPU-powered machine learning models on Cloudflare's global network.
-- **[GitHub](https://lobechat.com/discover/provider/github)**: With GitHub Models, developers can become AI engineers and leverage the industry's leading AI models.
 
-<details><summary><kbd>See more providers (+26)</kbd></summary>
+<details><summary><kbd>See more providers (+27)</kbd></summary>
 
-- **[PPIO](https://lobechat.com/discover/provider/ppio)**: PPIO supports stable and cost-efficient open-source LLM APIs, such as DeepSeek, Llama, Qwen etc. [Learn more](https://ppinfra.com/llm-api?utm_source=github_lobe-chat&utm_medium=github_readme&utm_campaign=link)
+- **[GitHub](https://lobechat.com/discover/provider/github)**: With GitHub Models, developers can become AI engineers and leverage the industry's leading AI models.
 - **[Novita](https://lobechat.com/discover/provider/novita)**: Novita AI is a platform providing a variety of large language models and AI image generation API services, flexible, reliable, and cost-effective. It supports the latest open-source models like Llama3 and Mistral, offering a comprehensive, user-friendly, and auto-scaling API solution for generative AI application development, suitable for the rapid growth of AI startups.
 - **[Together AI](https://lobechat.com/discover/provider/togetherai)**: Together AI is dedicated to achieving leading performance through innovative AI models, offering extensive customization capabilities, including rapid scaling support and intuitive deployment processes to meet various enterprise needs.
 - **[Fireworks AI](https://lobechat.com/discover/provider/fireworksai)**: Fireworks AI is a leading provider of advanced language model services, focusing on functional calling and multimodal processing. Its latest model, Firefunction V2, is based on Llama-3, optimized for function calling, conversation, and instruction following. The visual language model FireLLaVA-13B supports mixed input of images and text. Other notable models include the Llama series and Mixtral series, providing efficient multilingual instruction following and generation support.
@@ -228,7 +228,7 @@ We have implemented support for the following model service providers:
 
 </details>
 
-> 📊 Total providers: [<kbd>**36**</kbd>](https://lobechat.com/discover/providers)
+> 📊 Total providers: [<kbd>**37**</kbd>](https://lobechat.com/discover/providers)
 
  <!-- PROVIDER LIST -->
 

package/README.zh-CN.md

@@ -191,13 +191,14 @@ LobeChat 支持文件上传与知识库功能，你可以上传文件、图片
 - **[Bedrock](https://lobechat.com/discover/provider/bedrock)**: Bedrock 是亚马逊 AWS 提供的一项服务，专注于为企业提供先进的 AI 语言模型和视觉模型。其模型家族包括 Anthropic 的 Claude 系列、Meta 的 Llama 3.1 系列等，涵盖从轻量级到高性能的多种选择，支持文本生成、对话、图像处理等多种任务，适用于不同规模和需求的企业应用。
 - **[Google](https://lobechat.com/discover/provider/google)**: Google 的 Gemini 系列是其最先进、通用的 AI 模型，由 Google DeepMind 打造，专为多模态设计，支持文本、代码、图像、音频和视频的无缝理解与处理。适用于从数据中心到移动设备的多种环境，极大提升了 AI 模型的效率与应用广泛性。
 - **[DeepSeek](https://lobechat.com/discover/provider/deepseek)**: DeepSeek 是一家专注于人工智能技术研究和应用的公司，其最新模型 DeepSeek-V3 多项评测成绩超越 Qwen2.5-72B 和 Llama-3.1-405B 等开源模型，性能对齐领军闭源模型 GPT-4o 与 Claude-3.5-Sonnet。
+- **[PPIO](https://lobechat.com/discover/provider/ppio)**: PPIO 派欧云提供稳定、高性价比的开源模型 API 服务，支持 DeepSeek 全系列、Llama、Qwen 等行业领先大模型。
 - **[HuggingFace](https://lobechat.com/discover/provider/huggingface)**: HuggingFace Inference API 提供了一种快速且免费的方式，让您可以探索成千上万种模型，适用于各种任务。无论您是在为新应用程序进行原型设计，还是在尝试机器学习的功能，这个 API 都能让您即时访问多个领域的高性能模型。
 - **[OpenRouter](https://lobechat.com/discover/provider/openrouter)**: OpenRouter 是一个提供多种前沿大模型接口的服务平台，支持 OpenAI、Anthropic、LLaMA 及更多，适合多样化的开发和应用需求。用户可根据自身需求灵活选择最优的模型和价格，助力 AI 体验的提升。
 - **[Cloudflare Workers AI](https://lobechat.com/discover/provider/cloudflare)**: 在 Cloudflare 的全球网络上运行由无服务器 GPU 驱动的机器学习模型。
-- **[GitHub](https://lobechat.com/discover/provider/github)**: 通过 GitHub 模型，开发人员可以成为 AI 工程师，并使用行业领先的 AI 模型进行构建。
 
-<details><summary><kbd>See more providers (+26)</kbd></summary>
-- **[PPIO](https://lobechat.com/discover/provider/ppio)**: PPIO 派欧云提供稳定、高性价比的开源模型 API 服务，支持 DeepSeek 全系列、Llama、Qwen 等行业领先大模型。[了解更多](https://ppinfra.com/llm-api?utm_source=github_lobe-chat&utm_medium=github_readme&utm_campaign=link)
+<details><summary><kbd>See more providers (+27)</kbd></summary>
+
+- **[GitHub](https://lobechat.com/discover/provider/github)**: 通过 GitHub 模型，开发人员可以成为 AI 工程师，并使用行业领先的 AI 模型进行构建。
 - **[Novita](https://lobechat.com/discover/provider/novita)**: Novita AI 是一个提供多种大语言模型与 AI 图像生成的 API 服务的平台，灵活、可靠且具有成本效益。它支持 Llama3、Mistral 等最新的开源模型，并为生成式 AI 应用开发提供了全面、用户友好且自动扩展的 API 解决方案，适合 AI 初创公司的快速发展。
 - **[Together AI](https://lobechat.com/discover/provider/togetherai)**: Together AI 致力于通过创新的 AI 模型实现领先的性能，提供广泛的自定义能力，包括快速扩展支持和直观的部署流程，满足企业的各种需求。
 - **[Fireworks AI](https://lobechat.com/discover/provider/fireworksai)**: Fireworks AI 是一家领先的高级语言模型服务商，专注于功能调用和多模态处理。其最新模型 Firefunction V2 基于 Llama-3，优化用于函数调用、对话及指令跟随。视觉语言模型 FireLLaVA-13B 支持图像和文本混合输入。其他 notable 模型包括 Llama 系列和 Mixtral 系列，提供高效的多语言指令跟随与生成支持。
@@ -227,7 +228,7 @@ LobeChat 支持文件上传与知识库功能，你可以上传文件、图片
 
 </details>
 
-> 📊 Total providers: [<kbd>**36**</kbd>](https://lobechat.com/discover/providers)
+> 📊 Total providers: [<kbd>**37**</kbd>](https://lobechat.com/discover/providers)
 
  <!-- PROVIDER LIST -->
 

package/changelog/v1.json

@@ -1,4 +1,22 @@
 [
+  {
+    "children": {
+      "improvements": [
+        "Add build-in web search support for Wenxin & Hunyuan."
+      ]
+    },
+    "date": "2025-03-03",
+    "version": "1.68.2"
+  },
+  {
+    "children": {
+      "fixes": [
+        "Fix page crash with crawler error."
+      ]
+    },
+    "date": "2025-03-03",
+    "version": "1.68.1"
+  },
   {
     "children": {
       "features": [

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@lobehub/chat",
-  "version": "1.68.0",
+  "version": "1.68.2",
   "description": "Lobe Chat - an open-source, high-performance chatbot framework that supports speech synthesis, multimodal, and extensible Function Call plugin system. Supports one-click free deployment of your private ChatGPT/LLM web application.",
   "keywords": [
     "framework",

package/packages/web-crawler/README.md

@@ -1,34 +1,61 @@
 # @lobechat/web-crawler
 
-LobeChat 内置的网页抓取模块，用于从网页中提取结构化内容，并转换为 Markdown 格式。
+LobeChat's built-in web crawling module for intelligent extraction of web content and conversion to Markdown format.
 
-## 📝 简介
+## 📝 Introduction
 
-`@lobechat/web-crawler` 是 LobeChat 项目的内部组件，专门负责网页内容的抓取和处理。它能够智能地从各种网页中提取有意义的内容，剔除广告、导航栏等干扰元素，并将结果转换为结构良好的 Markdown 文本。
+`@lobechat/web-crawler` is a core component of LobeChat responsible for intelligent web content crawling and processing. It extracts valuable content from various webpages, filters out distracting elements, and generates structured Markdown text.
 
-## 🔍 主要功能
+## 🛠️ Core Features
 
-- **网页内容抓取**：支持从各类网站获取原始 HTML 内容
-- **智能内容提取**：使用 Mozilla 的 Readability 算法识别页面中的主要内容
-- **降级处理机制**：当标准抓取失败时，自动切换到 Browserless.io 服务进行渲染抓取（需要自行配置环境变量）
-- **Markdown 转换**：将提取的 HTML 内容转换为易于 AI 处理的 Markdown 格式
+- **Intelligent Content Extraction**: Identifies main content based on Mozilla Readability algorithm
+- **Multi-level Crawling Strategy**: Supports multiple crawling implementations including basic crawling, Jina, and Browserless rendering
+- **Custom URL Rules**: Handles specific website crawling logic through a flexible rule system
 
-## 🛠️ 技术实现
+## 🤝 Contribution
 
-该模块主要依赖以下技术：
+Web structures are diverse and complex. We welcome community contributions for specific website crawling rules. You can participate in improvements through:
 
-- **@mozilla/readability**：提供了强大的内容提取算法
-- **happy-dom**：轻量级的服务端 DOM 实现
-- **node-html-markdown**：高效的 HTML 到 Markdown 转换工具
+### How to Contribute URL Rules
 
-## 🤝 共建改进
+1. Add new rules to the [urlRules.ts](https://github.com/lobehub/lobe-chat/blob/main/packages/web-crawler/src/urlRules.ts) file
+2. Rule example:
 
-由于网页结构的多样性和复杂性，内容提取可能会遇到各种挑战。如果您发现某些网站的抓取效果不佳，欢迎通过以下方式参与改进：
+```typescript
+// Example: handling specific websites
+const url = [
+  // ... other URL matching rules
+  {
+    // URL matching pattern, supports regex
+    urlPattern: 'https://example.com/articles/(.*)',
 
-1. 提交具体的问题网址和期望的输出结果
-2. 分享您对特定网站类型的处理经验
-3. 提出针对性的算法或配置调整建议
+    // Optional: URL transformation, redirects to an easier-to-crawl version
+    urlTransform: 'https://example.com/print/$1',
 
-## 📌 注意事项
+    // Optional: specify crawling implementation, supports 'naive', 'jina', and 'browserless'
+    impls: ['naive', 'jina', 'browserless'],
 
-这是 LobeHub 的内部模块（`"private": true`），不作为独立包发布使用。它专为 LobeChat 的特定需求设计，与其他系统组件紧密集成。
+    // Optional: content filtering configuration
+    filterOptions: {
+      // Whether to enable Readability algorithm for filtering distracting elements
+      enableReadability: true,
+      // Whether to convert to plain text
+      pureText: false,
+    },
+  },
+];
+```
+
+### Rule Submission Process
+
+1. Fork the [LobeChat repository](https://github.com/lobehub/lobe-chat)
+2. Add or modify URL rules
+3. Submit a Pull Request describing:
+
+- Target website characteristics
+- Problems solved by the rule
+- Test cases (example URLs)
+
+## 📌 Note
+
+This is an internal module of LobeHub (`"private": true`), designed specifically for LobeChat and not published as a standalone package.

package/packages/web-crawler/README.zh-CN.md

@@ -0,0 +1,61 @@
+# @lobechat/web-crawler
+
+LobeChat 内置的网页抓取模块，用于智能提取网页内容并转换为 Markdown 格式。
+
+## 📝 简介
+
+`@lobechat/web-crawler` 是 LobeChat 的核心组件，负责网页内容的智能抓取与处理。它能够从各类网页中提取有价值的内容，过滤掉干扰元素，并生成结构化的 Markdown 文本。
+
+## 🛠️ 核心功能
+
+- **智能内容提取**：基于 Mozilla Readability 算法识别主要内容
+- **多级抓取策略**：支持多种抓取实现，包括基础抓取、Jina 和 Browserless 渲染抓取
+- **自定义 URL 规则**：通过灵活的规则系统处理特定网站的抓取逻辑
+
+## 🤝 参与共建
+
+网页结构多样复杂，我们欢迎社区贡献特定网站的抓取规则。您可以通过以下方式参与改进：
+
+### 如何贡献 URL 规则
+
+1. 在 [urlRules.ts](https://github.com/lobehub/lobe-chat/blob/main/packages/web-crawler/src/urlRules.ts) 文件中添加新规则
+2. 规则示例：
+
+```typescript
+// 示例：处理特定网站
+const url = [
+  // ... 其他 url 匹配规则
+  {
+    // URL 匹配模式，仅支持正则表达式
+    urlPattern: 'https://example.com/articles/(.*)',
+
+    // 可选：URL 转换，用于重定向到更易抓取的版本
+    urlTransform: 'https://example.com/print/$1',
+
+    // 可选：指定抓取实现方式，支持 'naive'、'jina' 和 'browserless' 三种
+    impls: ['naive', 'jina', 'browserless'],
+
+    // 可选：内容过滤配置
+    filterOptions: {
+      // 是否启用 Readability 算法，用于过滤干扰元素
+      enableReadability: true,
+      // 是否转换为纯文本
+      pureText: false,
+    },
+  },
+];
+```
+
+### 规则提交流程
+
+1. Fork [LobeChat 仓库](https://github.com/lobehub/lobe-chat)
+2. 添加或修改 URL 规则
+3. 提交 Pull Request 并描述：
+
+- 目标网站特点
+- 规则解决的问题
+- 测试用例（示例 URL）
+
+## 📌 注意事项
+
+这是 LobeHub 的内部模块（`"private": true`），专为 LobeChat 设计，不作为独立包发布使用。

package/packages/web-crawler/src/__test__/crawler.test.ts

@@ -80,9 +80,12 @@ describe('Crawler', () => {
     });
 
     expect(result).toEqual({
-      content: 'Fail to crawl the page. Error type: CrawlError, error message: Crawl failed',
-      errorMessage: 'Crawl failed',
-      errorType: 'CrawlError',
+      crawler: 'browserless',
+      data: {
+        content: 'Fail to crawl the page. Error type: CrawlError, error message: Crawl failed',
+        errorMessage: 'Crawl failed',
+        errorType: 'CrawlError',
+      },
       originalUrl: 'https://example.com',
       transformedUrl: undefined,
     });

package/packages/web-crawler/src/crawler.ts

@@ -32,6 +32,7 @@ export class Crawler {
       ...userFilterOptions,
     };
 
+    let finalCrawler: string | undefined;
     let finalError: Error | undefined;
 
     const systemImpls = (ruleImpls ?? this.impls) as CrawlImplType[];
@@ -55,6 +56,7 @@ export class Crawler {
       } catch (error) {
         console.error(error);
         finalError = error as Error;
+        finalCrawler = impl;
       }
     }
 
@@ -62,9 +64,12 @@ export class Crawler {
     const errorMessage = finalError?.message;
 
     return {
-      content: `Fail to crawl the page. Error type: ${errorType}, error message: ${errorMessage}`,
-      errorMessage: errorMessage,
-      errorType,
+      crawler: finalCrawler,
+      data: {
+        content: `Fail to crawl the page. Error type: ${errorType}, error message: ${errorMessage}`,
+        errorMessage: errorMessage,
+        errorType,
+      },
       originalUrl: url,
       transformedUrl: transformedUrl !== url ? transformedUrl : undefined,
     };

package/packages/web-crawler/src/type.ts

@@ -11,6 +11,7 @@ export interface CrawlSuccessResult {
 export interface CrawlErrorResult {
   content: string;
   errorMessage: string;
+  errorType: string;
   url: string;
 }
 
@@ -36,9 +37,7 @@ export interface CrawlUrlRule {
   // 内容过滤配置（可选）
   filterOptions?: FilterOptions;
   impls?: CrawlImplType[];
-  // 是否使用正则表达式匹配（默认为glob模式）
-  isRegex?: boolean;
-  // URL匹配模式，支持glob模式或正则表达式
+  // URL匹配模式，仅支持正则表达式
   urlPattern: string;
   // URL转换模板（可选），如果提供则进行URL转换
   urlTransform?: string;

package/packages/web-crawler/src/urlRules.ts

@@ -22,6 +22,11 @@ export const crawUrlRules: CrawlUrlRule[] = [
     impls: ['jina'],
     urlPattern: 'https://(.*).pdf',
   },
+  // arxiv PDF use jina
+  {
+    impls: ['jina'],
+    urlPattern: 'https://arxiv.org/pdf/(.*)',
+  },
   // 知乎有爬虫防护，使用 jina
   {
     impls: ['jina'],

package/src/config/aiModels/wenxin.ts

@@ -4,6 +4,7 @@ const wenxinChatModels: AIChatModelCard[] = [
   {
     abilities: {
       functionCall: true,
+      search: true,
     },
     contextWindowTokens: 8192,
     description:
@@ -16,11 +17,15 @@ const wenxinChatModels: AIChatModelCard[] = [
       input: 0.8,
       output: 2,
     },
+    settings: {
+      searchImpl: 'params',
+    },
     type: 'chat',
   },
   {
     abilities: {
       functionCall: true,
+      search: true,
     },
     contextWindowTokens: 8192,
     description:
@@ -32,11 +37,15 @@ const wenxinChatModels: AIChatModelCard[] = [
       input: 0.8,
       output: 2,
     },
+    settings: {
+      searchImpl: 'params',
+    },
     type: 'chat',
   },
   {
     abilities: {
       functionCall: true,
+      search: true,
     },
     contextWindowTokens: 128_000,
     description:
@@ -49,11 +58,15 @@ const wenxinChatModels: AIChatModelCard[] = [
       input: 0.8,
       output: 2,
     },
+    settings: {
+      searchImpl: 'params',
+    },
     type: 'chat',
   },
   {
     abilities: {
       functionCall: true,
+      search: true,
     },
     contextWindowTokens: 8192,
     description:
@@ -66,11 +79,15 @@ const wenxinChatModels: AIChatModelCard[] = [
       input: 30,
       output: 90,
     },
+    settings: {
+      searchImpl: 'params',
+    },
     type: 'chat',
   },
   {
     abilities: {
       functionCall: true,
+      search: true,
     },
     contextWindowTokens: 8192,
     description:
@@ -82,11 +99,15 @@ const wenxinChatModels: AIChatModelCard[] = [
       input: 30,
       output: 90,
     },
+    settings: {
+      searchImpl: 'params',
+    },
     type: 'chat',
   },
   {
     abilities: {
       functionCall: true,
+      search: true,
     },
     contextWindowTokens: 8192,
     description:
@@ -99,11 +120,15 @@ const wenxinChatModels: AIChatModelCard[] = [
       input: 20,
       output: 60,
     },
+    settings: {
+      searchImpl: 'params',
+    },
     type: 'chat',
   },
   {
     abilities: {
       functionCall: true,
+      search: true,
     },
     contextWindowTokens: 128_000,
     description:
@@ -116,11 +141,15 @@ const wenxinChatModels: AIChatModelCard[] = [
       input: 20,
       output: 60,
     },
+    settings: {
+      searchImpl: 'params',
+    },
     type: 'chat',
   },
   {
     abilities: {
       functionCall: true,
+      search: true,
     },
     contextWindowTokens: 8192,
     description:
@@ -132,6 +161,9 @@ const wenxinChatModels: AIChatModelCard[] = [
       input: 20,
       output: 60,
     },
+    settings: {
+      searchImpl: 'params',
+    },
     type: 'chat',
   },
   {

package/src/libs/agent-runtime/hunyuan/index.test.ts

@@ -1,7 +1,9 @@
 // @vitest-environment node
-import { ModelProvider } from '@/libs/agent-runtime';
+import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
+
+import { LobeOpenAICompatibleRuntime, ModelProvider } from '@/libs/agent-runtime';
+import { testProvider } from '@/libs/agent-runtime/providerTestUtils';
 
-import { testProvider } from '../providerTestUtils';
 import { LobeHunyuanAI } from './index';
 
 testProvider({
@@ -11,3 +13,136 @@ testProvider({
   chatDebugEnv: 'DEBUG_HUNYUAN_CHAT_COMPLETION',
   chatModel: 'hunyuan-lite',
 });
+
+// Mock the console.error to avoid polluting test output
+vi.spyOn(console, 'error').mockImplementation(() => {});
+
+let instance: LobeOpenAICompatibleRuntime;
+
+beforeEach(() => {
+  instance = new LobeHunyuanAI({ apiKey: 'test' });
+
+  // 使用 vi.spyOn 来模拟 chat.completions.create 方法
+  vi.spyOn(instance['client'].chat.completions, 'create').mockResolvedValue(
+    new ReadableStream() as any,
+  );
+});
+
+describe('LobeHunyuanAI', () => {
+  describe('chat', () => {
+    it('should with search citations', async () => {
+const data = [
+        {
+          id: "939fbdb8dbb9b4c5944cbbe687c977c2",
+          object: "chat.completion.chunk",
+          created: 1741000456,
+          model: "hunyuan-turbo",
+          system_fingerprint: "",
+          choices: [
+            {
+              index: 0,
+              delta: { role: "assistant", content: "为您" },
+              finish_reason: null
+            }
+          ],
+          note: "以上内容为AI生成，不代表开发者立场，请勿删除或修改本标记",
+          search_info: {
+            search_results: [
+              {
+                index: 1,
+                title: "公务员考试时政热点【2025年3月3日】_公务员考试网_华图教育",
+                url: "http://www.huatu.com/2025/0303/2803685.html",
+                icon: "https://hunyuan-img-1251316161.cos.ap-guangzhou.myqcloud.com/%2Fpublic/img/63ce96deffe0119827f12deaa5ffe7ef.jpg",
+                text: "华图教育官网"
+              },
+              {
+                index: 2,
+                title: "外交部新闻（2025年3月3日）",
+                url: "https://view.inews.qq.com/a/20250303A02NLC00?scene=qqsearch",
+                icon: "https://hunyuan-img-1251316161.cos.ap-guangzhou.myqcloud.com/%2Fpublic/img/00ce40298870d1accb7920d641152722.jpg",
+                text: "腾讯网"
+              }
+            ]
+          }
+        },
+        {
+          id: "939fbdb8dbb9b4c5944cbbe687c977c2",
+          object: "chat.completion.chunk",
+          created: 1741000456,
+          model: "hunyuan-turbo",
+          system_fingerprint: "",
+          choices: [
+            {
+              index: 0,
+              delta: { role: "assistant", content: "找到" },
+              finish_reason: null
+            }
+          ],
+          note: "以上内容为AI生成，不代表开发者立场，请勿删除或修改本标记",
+          search_info: {
+            search_results: [
+              {
+                index: 1,
+                title: "公务员考试时政热点【2025年3月3日】_公务员考试网_华图教育",
+                url: "http://www.huatu.com/2025/0303/2803685.html",
+                icon: "https://hunyuan-img-1251316161.cos.ap-guangzhou.myqcloud.com/%2Fpublic/img/63ce96deffe0119827f12deaa5ffe7ef.jpg",
+                text: "华图教育官网"
+              },
+              {
+                index: 2,
+                title: "外交部新闻（2025年3月3日）",
+                url: "https://view.inews.qq.com/a/20250303A02NLC00?scene=qqsearch",
+                icon: "https://hunyuan-img-1251316161.cos.ap-guangzhou.myqcloud.com/%2Fpublic/img/00ce40298870d1accb7920d641152722.jpg",
+                text: "腾讯网"
+              }
+            ]
+          }
+        }
+      ];
+
+      const mockStream = new ReadableStream({
+        start(controller) {
+          data.forEach((chunk) => {
+            controller.enqueue(chunk);
+          });
+
+          controller.close();
+        },
+      });
+
+      vi.spyOn(instance['client'].chat.completions, 'create').mockResolvedValue(mockStream as any);
+
+      const result = await instance.chat({
+        messages: [{ content: 'Hello', role: 'user' }],
+        model: 'mistralai/mistral-7b-instruct:free',
+        temperature: 0,
+      });
+
+      const decoder = new TextDecoder();
+      const reader = result.body!.getReader();
+      const stream: string[] = [];
+
+      while (true) {
+        const { value, done } = await reader.read();
+        if (done) break;
+        stream.push(decoder.decode(value));
+      }
+
+      expect(stream).toEqual(
+        [
+          'id: 939fbdb8dbb9b4c5944cbbe687c977c2',
+          'event: grounding',
+          'data: {"citations":[{"title":"公务员考试时政热点【2025年3月3日】_公务员考试网_华图教育","url":"http://www.huatu.com/2025/0303/2803685.html"},{"title":"外交部新闻（2025年3月3日）","url":"https://view.inews.qq.com/a/20250303A02NLC00?scene=qqsearch"}]}\n',
+          'id: 939fbdb8dbb9b4c5944cbbe687c977c2',
+          'event: text',
+          'data: "为您"\n',
+          'id: 939fbdb8dbb9b4c5944cbbe687c977c2',
+          'event: text',
+          'data: "找到"\n',
+        ].map((line) => `${line}\n`),
+      );
+
+      expect((await reader.read()).done).toBe(true);
+    });
+  });
+});

package/src/libs/agent-runtime/hunyuan/index.ts

@@ -15,14 +15,15 @@ export const LobeHunyuanAI = LobeOpenAICompatibleFactory({
 
       return {
         ...rest,
+        stream: true,
         ...(enabledSearch && {
-          /*
           citation: true,
+          enable_enhancement: true,
+          /*
           enable_multimedia: true,
-          search_info: true
           */
-          enable_enhancement: true,
           enable_speed_search: process.env.HUNYUAN_ENABLE_SPEED_SEARCH === '1',
+          search_info: true,
         }),
       } as any;
     },

package/src/libs/agent-runtime/qwen/index.ts

@@ -35,7 +35,7 @@ export const LobeQwenAI = LobeOpenAICompatibleFactory({
           : presence_penalty !== undefined && presence_penalty >= -2 && presence_penalty <= 2
             ? presence_penalty
             : undefined,
-        stream: !payload.tools,
+        stream: true,
         temperature:
           temperature !== undefined && temperature >= 0 && temperature < 2
             ? temperature

package/src/libs/agent-runtime/utils/streams/openai.ts

@@ -127,19 +127,35 @@ export const transformOpenAIStream = (
       }
 
       if (typeof content === 'string') {
-        // in Perplexity api, the citation is in every chunk, but we only need to return it once
-
-        if ('citations' in chunk && !!chunk.citations && !streamContext?.returnedPplxCitation) {
-          streamContext.returnedPplxCitation = true;
-
-          const citations = (chunk.citations as any[]).map((item) =>
-            typeof item === 'string' ? ({ title: item, url: item } as CitationItem) : item,
-          );
-
-          return [
-            { data: { citations }, id: chunk.id, type: 'grounding' },
-            { data: content, id: chunk.id, type: 'text' },
-          ];
+        if (!streamContext?.returnedCitation) {
+          const citations =
+            // in Perplexity api, the citation is in every chunk, but we only need to return it once
+            ('citations' in chunk && chunk.citations) ||
+            // in Hunyuan api, the citation is in every chunk
+            ('search_info' in chunk && (chunk.search_info as any)?.search_results) ||
+            // in Wenxin api, the citation is in the first and last chunk
+            ('search_results' in chunk && chunk.search_results);
+
+          if (citations) {
+            streamContext.returnedCitation = true;
+
+            return [
+              {
+                data: {
+                  citations: (citations as any[]).map(
+                    (item) =>
+                      ({
+                        title: typeof item === 'string' ? item : item.title,
+                        url: typeof item === 'string' ? item : item.url,
+                      }) as CitationItem
+                  ),
+                },
+                id: chunk.id,
+                type: 'grounding',
+              },
+              { data: content, id: chunk.id, type: 'text' },
+            ];
+          }
         }
 
         return { data: content, id: chunk.id, type: 'text' };

package/src/libs/agent-runtime/utils/streams/protocol.ts

@@ -9,9 +9,10 @@ export interface StreamContext {
   id: string;
   /**
    * As pplx citations is in every chunk, but we only need to return it once
-   * this flag is used to check if the pplx citation is returned,and then not return it again
+   * this flag is used to check if the pplx citation is returned,and then not return it again.
+   * Same as Hunyuan and Wenxin
    */
-  returnedPplxCitation?: boolean;
+  returnedCitation?: boolean;
   thinking?: {
     id: string;
     name: string;

package/src/libs/agent-runtime/wenxin/index.test.ts

@@ -0,0 +1,113 @@
+// @vitest-environment node
+import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
+
+import { LobeOpenAICompatibleRuntime, ModelProvider } from '@/libs/agent-runtime';
+import { testProvider } from '@/libs/agent-runtime/providerTestUtils';
+
+import { LobeWenxinAI } from './index';
+
+testProvider({
+  Runtime: LobeWenxinAI,
+  provider: ModelProvider.Wenxin,
+  defaultBaseURL: 'https://qianfan.baidubce.com/v2',
+  chatDebugEnv: 'DEBUG_WENXIN_CHAT_COMPLETION',
+  chatModel: 'ernie-speed-128k',
+});
+
+// Mock the console.error to avoid polluting test output
+vi.spyOn(console, 'error').mockImplementation(() => {});
+
+let instance: LobeOpenAICompatibleRuntime;
+
+beforeEach(() => {
+  instance = new LobeWenxinAI({ apiKey: 'test' });
+
+  // 使用 vi.spyOn 来模拟 chat.completions.create 方法
+  vi.spyOn(instance['client'].chat.completions, 'create').mockResolvedValue(
+    new ReadableStream() as any,
+  );
+});
+
+describe('LobeWenxinAI', () => {
+  describe('chat', () => {
+    it('should with search citations', async () => {
+      const data = [
+        {
+          id: "as-bhrxwy5fq1",
+          object: "chat.completion.chunk",
+          created: 1741000028,
+          model: "ernie-4.0-8k-latest",
+          choices: [
+            {
+              index: 0,
+              delta: { content: "今天是**", role: "assistant" },
+              flag: 0
+            }
+          ],
+          search_results: [
+            { index: 1, url: "http://www.mnw.cn/news/shehui/", title: "社会新闻" },
+            { index: 2, url: "https://www.chinanews.com.cn/sh/2025/03-01/10376297.shtml", title: "中越边民共庆“春龙节”" },
+            { index: 3, url: "https://www.chinanews.com/china/index.shtml", title: "中国新闻网_时政" }
+          ]
+        },
+        {
+          id: "as-bhrxwy5fq1",
+          object: "chat.completion.chunk",
+          created: 1741000028,
+          model: "ernie-4.0-8k-latest",
+          choices: [
+            {
+              index: 0,
+              delta: { content: "20" },
+              flag: 0
+            }
+          ]
+        }
+      ];
+
+      const mockStream = new ReadableStream({
+        start(controller) {
+          data.forEach((chunk) => {
+            controller.enqueue(chunk);
+          });
+
+          controller.close();
+        },
+      });
+
+      vi.spyOn(instance['client'].chat.completions, 'create').mockResolvedValue(mockStream as any);
+
+      const result = await instance.chat({
+        messages: [{ content: 'Hello', role: 'user' }],
+        model: 'mistralai/mistral-7b-instruct:free',
+        temperature: 0,
+      });
+
+      const decoder = new TextDecoder();
+      const reader = result.body!.getReader();
+      const stream: string[] = [];
+
+      while (true) {
+        const { value, done } = await reader.read();
+        if (done) break;
+        stream.push(decoder.decode(value));
+      }
+
+      expect(stream).toEqual(
+        [
+          'id: as-bhrxwy5fq1',
+          'event: grounding',
+          'data: {"citations":[{"title":"社会新闻","url":"http://www.mnw.cn/news/shehui/"},{"title":"中越边民共庆“春龙节”","url":"https://www.chinanews.com.cn/sh/2025/03-01/10376297.shtml"},{"title":"中国新闻网_时政","url":"https://www.chinanews.com/china/index.shtml"}]}\n',
+          'id: as-bhrxwy5fq1',
+          'event: text',
+          'data: "今天是**"\n',
+          'id: as-bhrxwy5fq1',
+          'event: text',
+          'data: "20"\n',
+        ].map((line) => `${line}\n`),
+      );
+
+      expect((await reader.read()).done).toBe(true);
+    });
+  });
+});

package/src/libs/agent-runtime/wenxin/index.ts

@@ -3,6 +3,23 @@ import { LobeOpenAICompatibleFactory } from '../utils/openaiCompatibleFactory';
 
 export const LobeWenxinAI = LobeOpenAICompatibleFactory({
   baseURL: 'https://qianfan.baidubce.com/v2',
+  chatCompletion: {
+    handlePayload: (payload) => {
+      const { enabledSearch, ...rest } = payload;
+
+      return {
+        ...rest,
+        stream: true,
+        ...(enabledSearch && {
+          web_search: {
+            enable: true,
+            enable_citation: true,
+            enable_trace: true,
+          }
+        }),
+      } as any;
+    },
+  },
   debug: {
     chatCompletion: () => process.env.DEBUG_WENXIN_CHAT_COMPLETION === '1',
   },

package/src/tools/web-browsing/Portal/PageContent/index.tsx

@@ -1,4 +1,4 @@
-import { Alert, CopyButton, Icon, Markdown } from '@lobehub/ui';
+import { Alert, CopyButton, Highlighter, Icon, Markdown } from '@lobehub/ui';
 import { Descriptions, Segmented, Typography } from 'antd';
 import { createStyles } from 'antd-style';
 import { ExternalLink } from 'lucide-react';
@@ -90,7 +90,42 @@ const PageContent = memo<PageContentProps>(({ result }) => {
   const { styles } = useStyles();
   const [display, setDisplay] = useState('render');
 
-  if (!result) return undefined;
+  if (!result || !result.data) return undefined;
+
+  if ('errorType' in result.data) {
+    return (
+      <Flexbox className={styles.footer} gap={4}>
+        <div>
+          <Descriptions
+            classNames={{
+              content: styles.footerText,
+            }}
+            column={1}
+            items={[
+              {
+                children: result.crawler,
+                label: t('search.crawPages.meta.crawler'),
+              },
+            ]}
+            size="small"
+          />
+        </div>
+        <Alert
+          extra={
+            <div style={{ maxWidth: 500, overflowX: 'scroll' }}>
+              <Highlighter language={'json'}>{JSON.stringify(result.data, null, 2)}</Highlighter>
+            </div>
+          }
+          message={
+            <div style={{ textAlign: 'start' }}>
+              {result.data.errorMessage || result.data.content}
+            </div>
+          }
+          type={'error'}
+        />
+      </Flexbox>
+    );
+  }
 
   const { url, title, description, content } = result.data;
   return (

package/src/tools/web-browsing/Render/PageContent/Result.tsx

@@ -1,7 +1,7 @@
 'use client';
 
-import { CrawlSuccessResult } from '@lobechat/web-crawler';
-import { Icon } from '@lobehub/ui';
+import { CrawlErrorResult, CrawlSuccessResult } from '@lobechat/web-crawler';
+import { Alert, Highlighter, Icon } from '@lobehub/ui';
 import { Descriptions, Typography } from 'antd';
 import { createStyles } from 'antd-style';
 import { ExternalLink } from 'lucide-react';
@@ -82,7 +82,7 @@ interface CrawlerData {
   crawler: string;
   messageId: string;
   originalUrl: string;
-  result: CrawlSuccessResult;
+  result: CrawlSuccessResult | CrawlErrorResult;
 }
 
 const CrawlerResultCard = memo<CrawlerData>(({ result, messageId, crawler, originalUrl }) => {
@@ -90,6 +90,39 @@ const CrawlerResultCard = memo<CrawlerData>(({ result, messageId, crawler, origi
   const { styles } = useStyles();
   const [openToolUI, togglePageContent] = useChatStore((s) => [s.openToolUI, s.togglePageContent]);
 
+  if ('errorType' in result) {
+    return (
+      <Flexbox className={styles.footer} gap={4}>
+        <div>
+          <Descriptions
+            classNames={{
+              content: styles.footerText,
+            }}
+            column={1}
+            items={[
+              {
+                children: crawler,
+                label: t('search.crawPages.meta.crawler'),
+              },
+            ]}
+            size="small"
+          />
+        </div>
+        <Alert
+          extra={
+            <div style={{ maxWidth: 500, overflowX: 'scroll' }}>
+              <Highlighter language={'json'}>{JSON.stringify(result, null, 2)}</Highlighter>
+            </div>
+          }
+          message={
+            <div style={{ textAlign: 'start' }}>{result.errorMessage || result.content}</div>
+          }
+          type={'error'}
+        />
+      </Flexbox>
+    );
+  }
+
   const { url, title, description } = result;
 
   return (

package/src/tools/web-browsing/Render/PageContent/index.tsx

@@ -1,3 +1,4 @@
+import { CrawlErrorResult } from '@lobechat/web-crawler';
 import { memo } from 'react';
 import { Flexbox } from 'react-layout-kit';
 
@@ -31,7 +32,16 @@ const PagesContent = memo<PagesContentProps>(({ results, messageId, urls }) => {
           key={result.originalUrl}
           messageId={messageId}
           originalUrl={result.originalUrl}
-          result={result.data}
+          result={
+            result.data ||
+            // TODO: Remove this in v2 as it's deprecated
+            ({
+              content: (result as any)?.content,
+              errorMessage: (result as any)?.errorMessage,
+              errorType: (result as any)?.errorType,
+              url: result.originalUrl,
+            } as CrawlErrorResult)
+          }
         />
       ))}
     </Flexbox>

package/src/types/tool/crawler.ts

@@ -1,4 +1,4 @@
-import { CrawlSuccessResult } from '@lobechat/web-crawler';
+import { CrawlErrorResult, CrawlSuccessResult } from '@lobechat/web-crawler';
 
 export interface CrawlSinglePageQuery {
   url: string;
@@ -10,7 +10,7 @@ export interface CrawlMultiPagesQuery {
 
 export interface CrawlResult {
   crawler: string;
-  data: CrawlSuccessResult;
+  data: CrawlSuccessResult | CrawlErrorResult;
   originalUrl: string;
 }