dsh-free-search 0.2.0 → 0.4.0

package/README.en.md

@@ -27,6 +27,7 @@ This plugin provides multiple free engines with automatic failover, fully indepe
 - **FREE / API KEY badges** — green FREE badge for free engines, orange API KEY badge for paid ones
 - **Clean integration** — implements the official `WebSearchProvider` seam, coexists with official plugins
 - **web_fetch** — agent can fetch page content (official `dsh-web-fetch-http` provider, pure JS, zero extra deps)
+- **platform_search** — search GitHub / V2EX / Bilibili (public APIs, zero deps)
 
 ## Engines
 
@@ -35,8 +36,9 @@ This plugin provides multiple free engines with automatic failover, fully indepe
 | `ddg` | DuckDuckGo HTML | Free | Occasionally rate-limited (anti-bot), recovers automatically |
 | `ddg-lite` | DuckDuckGo Lite | Free | Lightweight variant, same caveat |
 | `bing` | Bing | Free | **Default engine**, most stable, zh-CN optimized |
+| `anysearch` | AnySearch AI | Free | AI search, no key (anonymous quota) |
 | `searxng` | SearXNG meta-search | Free | Multi-instance failover, custom instances supported |
-| `exa` | Exa | Paid | Requires `EXA_API_KEY` |
+| `exa` | Exa | Free | **Works keyless** (anonymous MCP), add key for higher limits |
 | `perplexity` | Perplexity | Paid | Requires `PERPLEXITY_API_KEY` |
 | `deepseek-official` | DeepSeek Official | Paid | Requires `DEEPSEEK_API_KEY` |
 
@@ -105,6 +107,18 @@ After search, ask the agent to **read a page** (e.g. "open the first link and su
 - Timeout and size limits
 - ⚠️ Note: `web_fetch` has no SSRF protection — the agent could reach internal addresses. Use deliberately.
 
+### Platform search (platform_search)
+
+Ask the agent to search a specific platform, e.g. "search GitHub for deepseek harness", "find Bilibili videos about this", "V2EX threads about dsh". The `platform_search` tool supports:
+
+| Platform | Purpose |
+|---|---|
+| `github` | GitHub repository search (public API, free, no key) |
+| `v2ex` | V2EX hot/relevant topics |
+| `bilibili` | Bilibili video/content search (public endpoint) |
+
+All use public APIs with zero external dependencies — works out of the box.
+
 ## Local engine switcher (tools/)
 
 Prefer a local tool over the web UI? The `tools/` directory ships a zero-dependency switcher:

package/README.md

@@ -26,6 +26,7 @@ dsh 默认的搜索 provider 依赖 DeepSeek 官方 API key（`DEEPSEEK_API_KEY`
 - **系统提示词注入** —— agent 知道当前用哪个引擎、哪些需要 key
 - **免费标注** —— 设置页中免费引擎带绿色 `FREE` 徽章，付费引擎带橙色 `API KEY` 徽章
 - **网页抓取（web_fetch）** —— 让 agent 抓取网页内容（官方 `dsh-web-fetch-http` provider，纯 JS，零额外依赖）
+- **平台搜索（platform_search）** —— 搜 GitHub / V2EX / B站（公开 API，零依赖）
 - **干净集成** —— 实现官方 `WebSearchProvider` seam 接口，与官方插件共存
 
 ## 引擎列表
@@ -35,8 +36,9 @@ dsh 默认的搜索 provider 依赖 DeepSeek 官方 API key（`DEEPSEEK_API_KEY`
 | `ddg` | DuckDuckGo HTML | 免费 | 偶发限流（反爬），解封自动恢复 |
 | `ddg-lite` | DuckDuckGo Lite | 免费 | 轻量版，同上 |
 | `bing` | Bing | 免费 | **默认引擎**，最稳定，中文优化（zh-CN） |
+| `anysearch` | AnySearch AI | 免费 | AI 搜索，无 key（匿名额度） |
 | `searxng` | SearXNG 元搜索 | 免费 | 多实例自动切换，支持自定义实例 |
-| `exa` | Exa | 付费 | 需 `EXA_API_KEY` |
+| `exa` | Exa | 免费 | **无 key 也可用**（MCP 匿名），配 key 提升额度 |
 | `perplexity` | Perplexity | 付费 | 需 `PERPLEXITY_API_KEY` |
 | `deepseek-official` | DeepSeek 官方 | 付费 | 需 `DEEPSEEK_API_KEY` |
 
@@ -104,6 +106,18 @@ Search engine test:
 - 支持超时和大小限制
 - ⚠️ 注意：`web_fetch` 无 SSRF 防护，agent 理论上可访问内网地址——按需使用
 
+### 平台搜索（platform_search）
+
+让 agent 搜特定平台，如"在 GitHub 上搜 deepseek harness"、"看看 B站有什么相关视频"、"V2EX 上关于 dsh 的讨论"。`platform_search` 工具支持：
+
+| 平台 | 用途 |
+|---|---|
+| `github` | GitHub 仓库搜索（API，免费无 key） |
+| `v2ex` | V2EX 热门/相关主题 |
+| `bilibili` | B站视频/内容搜索（公开接口） |
+
+全部走公开 API，零外部依赖，开箱即用。
+
 ## 本地引擎切换工具（tools/）
 
 不想用网页设置页？`tools/` 目录附带了一个本地切换小工具（零依赖）：

package/lib/client.js

@@ -57,8 +57,9 @@ window.__ModuleLoader__.load({
       { id: "ddg", label: "DuckDuckGo · HTML", badge: "FREE", link: "https://duckduckgo.com" },
       { id: "ddg-lite", label: "DuckDuckGo · Lite", badge: "FREE", link: "https://duckduckgo.com" },
       { id: "bing", label: "Bing", badge: "FREE", link: "https://www.bing.com" },
+      { id: "anysearch", label: "AnySearch · AI", badge: "FREE", link: "https://anysearch.com" },
       { id: "searxng", label: "SearXNG · 元搜索", badge: "FREE", link: "https://github.com/searxng/searxng" },
-      { id: "exa", label: "Exa", badge: "API KEY", link: "https://dashboard.exa.ai/api-keys" },
+      { id: "exa", label: "Exa", badge: "FREE", link: "https://dashboard.exa.ai/api-keys" },
       { id: "perplexity", label: "Perplexity", badge: "API KEY", link: "https://www.perplexity.ai/settings/api" },
       { id: "deepseek-official", label: "DeepSeek Official", badge: "API KEY", link: "https://platform.deepseek.com/api_keys" },
     ];
@@ -251,7 +252,7 @@ window.__ModuleLoader__.load({
                       react_jsx_runtime.jsx("input", {
                         className: "dshfs-input",
                         type: "password",
-                        placeholder: keysConfigured.exa ? "Exa API key (configured)" : "Exa API key (sk-...)",
+                        placeholder: keysConfigured.exa ? "Exa API key (configured)" : "Exa API key (optional, free without)",
                         value: exaKey,
                         disabled: !ready || saving,
                         onChange: (e) => {

package/lib/index.js

@@ -11,8 +11,8 @@ const ACCEPT_LANG = "zh-CN,zh;q=0.9,en;q=0.8";
 
 const FREE_SEARCH_NS = settingsNamespace("free-search");
 const BRIDGE_PREFIX = "/api/dsh-free-search-settings";
-const FREE_ENGINES = ["ddg", "ddg-lite", "bing", "searxng"];
-const ALL_ENGINES = ["ddg", "ddg-lite", "bing", "searxng", "exa", "perplexity", "deepseek-official"];
+const FREE_ENGINES = ["ddg", "ddg-lite", "bing", "searxng", "anysearch"];
+const ALL_ENGINES = ["ddg", "ddg-lite", "bing", "searxng", "anysearch", "exa", "perplexity", "deepseek-official"];
 
 function decodeEntities(text) {
   return String(text)
@@ -224,6 +224,231 @@ async function searchSearxng(query, maxResults, options, signal) {
 }
 //#endregion
 
+//#region keyless engines (AnySearch / Exa MCP - free, no API key)
+const ANYSEARCH_URL = "https://api.anysearch.com/v1/search";
+const EXA_MCP_URL = "https://mcp.exa.ai/mcp";
+
+// AnySearch: 免费匿名额度（无 key），结构化 JSON 结果
+async function searchAnysearch(query, maxResults, signal) {
+  const controller = new AbortController();
+  const timer = setTimeout(() => controller.abort(), 12000);
+  const onAbort = () => controller.abort();
+  signal?.addEventListener("abort", onAbort);
+  let response;
+  try {
+    response = await fetch(ANYSEARCH_URL, {
+      method: "POST",
+      headers: { "content-type": "application/json" },
+      body: JSON.stringify({ query, max_results: maxResults ?? 5 }),
+      signal: controller.signal,
+    });
+  } catch (error) {
+    if (signal?.aborted) throw error;
+    throw new Error(`AnySearch request failed: ${error?.message ?? String(error)}`);
+  } finally {
+    clearTimeout(timer);
+    signal?.removeEventListener("abort", onAbort);
+  }
+  if (!response.ok) throw new Error(`AnySearch API error (HTTP ${response.status})`);
+  const data = await response.json();
+  if (data.code !== 0) throw new Error(`AnySearch API error: ${data.message ?? data.code}`);
+  const results = data.data?.results ?? [];
+  return {
+    sources: results
+      .filter((r) => r.url)
+      .map((r) => ({
+        url: r.url,
+        ...(r.title ? { title: String(r.title) } : {}),
+        ...(r.snippet ? { snippet: String(r.snippet).slice(0, 300) } : {}),
+      })),
+    truncated: false,
+  };
+}
+
+// Exa MCP: 匿名公开 MCP（无 key），web_search_exa 工具
+async function searchExaMCP(query, maxResults, signal) {
+  const controller = new AbortController();
+  const timer = setTimeout(() => controller.abort(), 20000);
+  const onAbort = () => controller.abort();
+  signal?.addEventListener("abort", onAbort);
+  let response;
+  try {
+    response = await fetch(EXA_MCP_URL, {
+      method: "POST",
+      headers: { "content-type": "application/json", accept: "application/json, text/event-stream" },
+      body: JSON.stringify({
+        jsonrpc: "2.0",
+        id: Date.now(),
+        method: "tools/call",
+        params: { name: "web_search_exa", arguments: { query, numResults: maxResults ?? 5 } },
+      }),
+      signal: controller.signal,
+    });
+  } catch (error) {
+    if (signal?.aborted) throw error;
+    throw new Error(`Exa MCP request failed: ${error?.message ?? String(error)}`);
+  } finally {
+    clearTimeout(timer);
+    signal?.removeEventListener("abort", onAbort);
+  }
+  if (!response.ok) throw new Error(`Exa MCP error (HTTP ${response.status})`);
+  const text = await response.text();
+  // 解析 SSE 格式：event: message\ndata: {...}
+  const lines = text.split("\n");
+  let json = null;
+  for (const line of lines) {
+    if (line.startsWith("data: ")) {
+      try {
+        json = JSON.parse(line.slice(6));
+        break;
+      } catch {}
+    }
+  }
+  if (!json || json.error) {
+    throw new Error(`Exa MCP error: ${json?.error?.message ?? "no data"}`);
+  }
+  const content = json.result?.content ?? [];
+  const sources = [];
+  const textBlocks = content
+    .filter((b) => b.type === "text")
+    .map((b) => b.text)
+    .join("\n");
+  // 解析 "Title: X\nURL: Y\nPublished: Z\nHighlights:\n..."
+  const blocks = textBlocks.split(/\n(?=Title:)/);
+  for (const block of blocks) {
+    const title = block.match(/^Title: (.+)$/m)?.[1];
+    const url = block.match(/^URL: (\S+)$/m)?.[1];
+    const published = block.match(/^Published: (.+)$/m)?.[1];
+    const highlights = block.split(/^Highlights:$/m)[1]?.split("\n").filter((l) => l.trim() && !l.trim().startsWith("...")).slice(0, 3).join(" ");
+    if (!url) continue;
+    sources.push({
+      url,
+      ...(title ? { title } : {}),
+      ...(highlights ? { snippet: highlights.slice(0, 300) } : {}),
+      ...(published ? { publishedAt: published } : {}),
+    });
+  }
+  return { sources, truncated: false };
+}
+//#endregion
+
+//#region platform search (GitHub / V2EX / Bilibili / Reddit)
+const PLATFORMS = {
+  github: { name: "GitHub" },
+  v2ex: { name: "V2EX" },
+  bilibili: { name: "Bilibili" },
+};
+
+async function searchGithub(query, maxResults, signal) {
+  const response = await fetch(
+    `https://api.github.com/search/repositories?q=${encodeURIComponent(query)}&per_page=${maxResults ?? 5}`,
+    {
+      headers: { "user-agent": USER_AGENT, accept: "application/vnd.github+json" },
+      ...(signal !== undefined ? { signal } : {}),
+    }
+  );
+  if (!response.ok) throw new Error(`GitHub API error (HTTP ${response.status})`);
+  const data = await response.json();
+  return {
+    sources: (data.items ?? []).map((item) => ({
+      url: item.html_url,
+      title: item.full_name ?? item.name,
+      snippet: `${item.description ?? ""}${item.stargazers_count ? ` ⭐${item.stargazers_count}` : ""}`.trim(),
+    })),
+    truncated: false,
+  };
+}
+
+async function searchV2ex(query, maxResults, signal) {
+  const response = await fetch("https://www.v2ex.com/api/topics/hot.json", {
+    headers: { "user-agent": USER_AGENT },
+    ...(signal !== undefined ? { signal } : {}),
+  });
+  if (!response.ok) throw new Error(`V2EX API error (HTTP ${response.status})`);
+  const topics = await response.json();
+  const q = query.toLowerCase();
+  const matched = Array.isArray(topics)
+    ? topics.filter((t) => (t.title ?? "").toLowerCase().includes(q) || (t.content ?? "").toLowerCase().includes(q))
+    : [];
+  return {
+    sources: matched.slice(0, maxResults ?? 5).map((t) => ({
+      url: `https://www.v2ex.com/t/${t.id}`,
+      title: t.title,
+      ...(t.content ? { snippet: String(t.content).slice(0, 200) } : {}),
+    })),
+    truncated: false,
+  };
+}
+
+async function searchBilibili(query, maxResults, signal) {
+  const response = await fetch(
+    `https://api.bilibili.com/x/web-interface/search/all/v2?keyword=${encodeURIComponent(query)}`,
+    {
+      headers: { "user-agent": USER_AGENT, referer: "https://www.bilibili.com" },
+      ...(signal !== undefined ? { signal } : {}),
+    }
+  );
+  if (!response.ok) throw new Error(`Bilibili API error (HTTP ${response.status})`);
+  const data = await response.json();
+  if (data.code !== 0) throw new Error(`Bilibili API error: ${data.message ?? data.code}`);
+  const sources = [];
+  for (const section of data.data?.result ?? []) {
+    for (const item of section.data ?? []) {
+      if (!item.arcurl) continue;
+      sources.push({
+        url: item.arcurl,
+        title: item.title ? String(item.title).replace(/<[^>]+>/g, "") : item.bvid,
+        ...(item.desc ? { snippet: String(item.desc).slice(0, 200) } : {}),
+      });
+      if (sources.length >= (maxResults ?? 5)) break;
+    }
+    if (sources.length >= (maxResults ?? 5)) break;
+  }
+  return { sources, truncated: false };
+}
+
+async function searchReddit(query, maxResults, signal) {
+  const response = await fetch(
+    `https://old.reddit.com/search.json?q=${encodeURIComponent(query)}&limit=${maxResults ?? 5}&sort=relevance`,
+    {
+      headers: {
+        "user-agent": `${USER_AGENT} (dsh-free-search; contact: github.com/DDDMUC)`,
+        accept: "application/json",
+      },
+      ...(signal !== undefined ? { signal } : {}),
+    }
+  );
+  if (!response.ok) throw new Error(`Reddit API error (HTTP ${response.status})`);
+  const data = await response.json();
+  return {
+    sources: (data.data?.children ?? [])
+      .map((c) => c.data)
+      .filter((p) => p && p.url)
+      .map((p) => ({
+        url: p.url,
+        title: p.title ?? "",
+        ...(p.selftext ? { snippet: String(p.selftext).slice(0, 200) } : {}),
+      })),
+    truncated: false,
+  };
+}
+
+async function searchPlatform(platform, query, maxResults, signal) {
+  switch (platform) {
+    case "github":
+      return searchGithub(query, maxResults, signal);
+    case "v2ex":
+      return searchV2ex(query, maxResults, signal);
+    case "bilibili":
+      return searchBilibili(query, maxResults, signal);
+    case "reddit":
+      return searchReddit(query, maxResults, signal);
+    default:
+      throw new Error(`unknown platform: ${platform}`);
+  }
+}
+//#endregion
+
 //#region paid engines (exa / perplexity / deepseek-official)
 async function searchExa(query, maxResults, apiKey, signal) {
   if (!apiKey) throw new Error("Exa search requires EXA_API_KEY");
@@ -551,10 +776,11 @@ function apply(ctx, config) {
       const cfg = current();
       const preferred = cfg.provider ?? "bing";
 
-      // 付费引擎：直接调用，缺 key 报错
+      // 付费引擎：直接调用，缺 key 报错（exa 无 key 时走 Exa MCP keyless）
       if (preferred === "exa") {
         const key = await resolveApiKey("EXA_API_KEY", "exaApiKey");
-        return searchExa(request.query, request.maxResults, key, signal);
+        if (key) return searchExa(request.query, request.maxResults, key, signal);
+        return searchExaMCP(request.query, request.maxResults, signal);
       }
       if (preferred === "perplexity") {
         const key = await resolveApiKey("PERPLEXITY_API_KEY", "perplexityApiKey");
@@ -565,15 +791,17 @@ function apply(ctx, config) {
         return searchDeepSeekOfficial(request.query, request.maxResults, key, signal);
       }
 
-      // 免费引擎：首选 + 自动回退（searxng 元搜索多实例自动切换）
+      // 免费引擎：首选 + 自动回退（anysearch / searxng 无 key）
       const chain =
         preferred === "ddg-lite"
-          ? ["ddg-lite", "ddg", "searxng", "bing"]
+          ? ["ddg-lite", "ddg", "anysearch", "searxng", "bing"]
           : preferred === "bing"
-            ? ["bing", "searxng", "ddg", "ddg-lite"]
+            ? ["bing", "anysearch", "searxng", "ddg", "ddg-lite"]
             : preferred === "searxng"
-              ? ["searxng", "bing", "ddg", "ddg-lite"]
-              : ["ddg", "searxng", "ddg-lite", "bing"];
+              ? ["searxng", "anysearch", "bing", "ddg", "ddg-lite"]
+              : preferred === "anysearch"
+                ? ["anysearch", "bing", "searxng", "ddg", "ddg-lite"]
+                : ["ddg", "anysearch", "searxng", "ddg-lite", "bing"];
       let lastError = null;
       for (const engine of chain) {
         try {
@@ -591,6 +819,9 @@ function apply(ctx, config) {
             case "searxng":
               result = await searchSearxng(request.query, request.maxResults, cfg, signal);
               break;
+            case "anysearch":
+              result = await searchAnysearch(request.query, request.maxResults, signal);
+              break;
             default:
               result = await searchDdgHtml(request.query, request.maxResults, cfg, signal);
           }
@@ -639,10 +870,12 @@ function apply(ctx, config) {
           return await searchBing(q, 2, cfg);
         case "searxng":
           return await searchSearxng(q, 2, cfg);
+        case "anysearch":
+          return await searchAnysearch(q, 2);
         case "exa": {
           const key = await resolveApiKey("EXA_API_KEY", "exaApiKey");
-          if (!key) return { ok: false, error: "EXA_API_KEY not configured" };
-          return await searchExa(q, 2, key);
+          if (key) return await searchExa(q, 2, key);
+          return await searchExaMCP(q, 2);
         }
         case "perplexity": {
           const key = await resolveApiKey("PERPLEXITY_API_KEY", "perplexityApiKey");
@@ -759,6 +992,70 @@ function apply(ctx, config) {
     }, "free-search: test engines tool");
   });
 
+  // 平台搜索工具：GitHub / V2EX / Bilibili（公开 API，零依赖）
+  ctx.inject(["tools"], (sctx) => {
+    sctx.effect(() => {
+      const dispose = sctx.tools.register(
+        defineTool({
+          name: "platform_search",
+          description:
+            "Search a specific platform (GitHub / V2EX / Bilibili) for a query. Returns source URLs with titles and snippets. Use this when the user asks about repos, code, forum threads, or videos.",
+          parameters: {
+            platform: {
+              type: "string",
+              description: "Platform to search: github, v2ex, bilibili",
+            },
+            query: {
+              type: "string",
+              description: "The search query.",
+            },
+            maxResults: {
+              type: "number",
+              description: "Optional result count (default 5, max 10).",
+            },
+          },
+          output: {
+            schema: {
+              type: "object",
+              additionalProperties: false,
+              properties: {
+                platform: { type: "string" },
+                sources: {
+                  type: "array",
+                  items: {
+                    type: "object",
+                    additionalProperties: false,
+                    properties: {
+                      url: { type: "string" },
+                      title: { type: "string" },
+                      snippet: { type: "string" },
+                    },
+                  },
+                },
+              },
+            },
+            render(args, value) {
+              const lines = value.sources.map((s, i) => `- [${s.title ?? s.url}](${s.url})${s.snippet ? ` - ${s.snippet.slice(0, 120)}` : ""}`);
+              return `Platform search (${value.platform}):\n${lines.join("\n") || "No results found."}`;
+            },
+          },
+          async execute(args) {
+            const platform = args.platform;
+            if (!PLATFORMS[platform]) {
+              throw new Error(`unknown platform "${platform}" - use one of: ${Object.keys(PLATFORMS).join(", ")}`);
+            }
+            const limit = Math.min(args.maxResults ?? 5, 10);
+            const result = await searchPlatform(platform, args.query, limit);
+            return { platform, sources: result.sources };
+          },
+        })
+      );
+      return () => {
+        dispose();
+      };
+    }, "free-search: platform search tool");
+  });
+
   // 让 agent 知道可用搜索引擎（动态生成，随 key 配置变化）
   ctx.inject(["systemPrompt"], (sctx) => {
     sctx.effect(() => {
@@ -776,13 +1073,16 @@ function apply(ctx, config) {
           "- ddg-lite (DuckDuckGo Lite) - FREE, no key (may be rate-limited)",
           "- bing (Bing) - FREE, no key (most stable)",
           "- searxng (meta-search, multi-instance) - FREE, no key",
-          "- exa - requires EXA_API_KEY",
+          "- anysearch (AI search) - FREE, no key",
+          "- exa - FREE keyless (MCP) or EXA_API_KEY for higher limits",
           "- perplexity - requires PERPLEXITY_API_KEY",
           "- deepseek-official - requires DEEPSEEK_API_KEY",
           "",
           "FREE engines auto-fallback to another FREE engine on failure. Paid engines fail with a clear error when their key is missing - tell the user which key to configure.",
           "",
           "Use the free_search_test tool to test which engines actually work right now.",
+          "",
+          "For platform-specific searches (GitHub repos, V2EX threads, Bilibili videos), use the platform_search tool with platform: github|v2ex|bilibili.",
         ].join("\n"),
       };
       const dispose = sctx.systemPrompt.section(section);
@@ -795,21 +1095,31 @@ function apply(ctx, config) {
 
 export {
   ALL_ENGINES,
+  ANYSEARCH_URL,
   BING_URL,
   Config,
   DDG_HTML_URL,
   DDG_LITE_URL,
+  EXA_MCP_URL,
   FREE_ENGINES,
   FREE_SEARCH_NS,
+  PLATFORMS,
   SEARXNG_INSTANCES,
   apply,
   inject,
   name,
+  searchAnysearch,
   searchBing,
+  searchBilibili,
   searchDeepSeekOfficial,
   searchDdgHtml,
   searchDdgLite,
   searchExa,
+  searchExaMCP,
+  searchGithub,
   searchPerplexity,
+  searchPlatform,
+  searchReddit,
   searchSearxng,
+  searchV2ex,
 };

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "dsh-free-search",
-  "version": "0.2.0",
-  "description": "Free web search for DeepSeek Harness: DuckDuckGo/Bing (FREE) + Exa/Perplexity/DeepSeek (API key) engines, with web settings UI and engine test tool.",
+  "version": "0.4.0",
+  "description": "Free web search for DeepSeek Harness: 8 engines (Bing/DuckDuckGo/AnySearch/SearXNG/Exa keyless) + platform search + web_fetch, with web settings UI.",
   "type": "module",
   "main": "lib/index.js",
   "exports": {