@juicesharp/rpiv-web-tools 1.17.0 → 1.17.1

package/README.md

@@ -8,7 +8,7 @@
   </a>
 </div>
 
-Let the model search the web and read pages. `rpiv-web-tools` adds `web_search` and `web_fetch` tools to [Pi Agent](https://github.com/badlogic/pi-mono) with pluggable providers (Brave, Tavily, Serper, Exa, Jina, Firecrawl, [SearXNG](https://docs.searxng.org/), [Ollama](https://ollama.com)), plus `/web-tools` for interactive provider selection and API-key setup.
+Let the model search the web and read pages. `rpiv-web-tools` adds `web_search` and `web_fetch` tools to [Pi Agent](https://github.com/badlogic/pi-mono) with pluggable providers (Brave, Tavily, Serper, Exa, You.com, Jina, Firecrawl, Perplexity, [SearXNG](https://docs.searxng.org/), [Ollama](https://ollama.com)), plus `/web-tools` for interactive provider selection and API-key setup.
 
 ![Provider selection prompt](https://raw.githubusercontent.com/juicesharp/rpiv-mono/main/packages/rpiv-web-tools/docs/config.jpg)
 
@@ -22,14 +22,16 @@ Pick one as the active backend; switch any time without losing the others' keys.
 | Tavily | `TAVILY_API_KEY` | [tavily.com](https://tavily.com) | native extraction (plain text) | |
 | Serper | `SERPER_API_KEY` | [serper.dev](https://serper.dev) | raw HTTP → htmlToText, `raw: true` available | |
 | Exa | `EXA_API_KEY` | [exa.ai](https://exa.ai) | native extraction (plain text) | |
+| You.com | `YOUCOM_API_KEY` | [you.com](https://you.com) | native extraction (markdown) | $5/1K search + $1/1K fetch |
 | Jina | `JINA_API_KEY` | [jina.ai/reader](https://jina.ai/reader) | native extraction (markdown) | |
 | Firecrawl | `FIRECRAWL_API_KEY` | [firecrawl.dev](https://firecrawl.dev) | native extraction (markdown) | |
+| Perplexity | `PERPLEXITY_API_KEY` | [docs.perplexity.ai](https://docs.perplexity.ai/) | raw HTTP → htmlToText, `raw: true` available | |
 | SearXNG | `SEARXNG_URL` (+ optional `SEARXNG_API_KEY`) | self-hosted | raw HTTP → htmlToText, `raw: true` available | see [§SearXNG](#searxng-self-hosted) |
 | Ollama | `OLLAMA_HOST` / `OLLAMA_API_KEY` | local or [ollama.com](https://ollama.com) | native extraction | see [§Ollama](#ollama-local-or-cloud) |
 
 ## Features
 
-- **Read any URL** - fetch http/https pages with HTML-to-text extraction, or get the raw response with `raw: true` (honoured by Brave/Serper/SearXNG; extraction providers — Tavily/Exa/Jina/Firecrawl/Ollama — always return their parsed text).
+- **Read any URL** - fetch http/https pages with HTML-to-text extraction, or get the raw response with `raw: true` (honoured by Brave/Serper/Perplexity/SearXNG; extraction providers — Tavily/Exa/You.com/Jina/Firecrawl/Ollama — always return their parsed text).
 - **GitHub URL interceptor (opt-in)** - github.com URLs route through `gh`/`git` for full repository content (file tree, README, individual file contents) instead of the rendered HTML page. Off by default; enable per-user via config or per-consumer at registration time. See [§GitHub URL interceptor](#github-url-interceptor).
 - **Large-page spillover** - oversized responses truncate inline and spill the full body to a temp file the model can read on demand.
 - **SSRF guard** - refuses loopback, RFC 1918, link-local, and cloud-metadata addresses (`localhost`, `127.0.0.0/8`, `10.0.0.0/8`, `169.254.0.0/16`, `172.16.0.0/12`, `192.168.0.0/16`, `::1`, `fc00::/7`, `fe80::/10`).
@@ -49,8 +51,8 @@ Then restart your Pi session.
   1–10 results per call.
 - **`web_fetch`** - read an http/https URL. Lookup order: opt-in URL interceptors
   (see [§GitHub URL interceptor](#github-url-interceptor)), then the active provider's native
-  fetch endpoint when it has one (Tavily/Exa/Jina/Firecrawl/Ollama → vendor extraction;
-  Brave/Serper/SearXNG → shared raw HTTP + HTML-to-text fallback). Large responses truncate
+  fetch endpoint when it has one (Tavily/Exa/You.com/Jina/Firecrawl/Ollama → vendor extraction;
+  Brave/Serper/Perplexity/SearXNG → shared raw HTTP + HTML-to-text fallback). Large responses truncate
   inline and spill the full body to a temp file the model can read on demand.
 
 ### Schema - `web_search`
@@ -69,7 +71,7 @@ Returns:
   content: [{ type: "text", text: string }], // markdown list of "**title**\n url\n snippet"
   details: {
     query: string,
-    backend: "brave" | "tavily" | "serper" | "exa" | "jina" | "firecrawl" | "searxng" | "ollama",
+    backend: "brave" | "tavily" | "serper" | "exa" | "youcom" | "jina" | "firecrawl" | "perplexity" | "searxng" | "ollama",
     resultCount: number,
     results?: Array<{ title: string, url: string, snippet: string }>,
   }
@@ -103,7 +105,7 @@ Returns:
 }
 ```
 
-Throws on invalid URL, non-http(s) protocol, private/loopback hostnames (SSRF guard), non-2xx response, or `image/` / `video/` / `audio/` content types. Extraction providers (Tavily/Exa/Jina/Firecrawl) additionally throw when the API returns an empty body or a vendor-level failure (e.g. Firecrawl `success: false`, Tavily `failed_results`).
+Throws on invalid URL, non-http(s) protocol, private/loopback hostnames (SSRF guard), non-2xx response, or `image/` / `video/` / `audio/` content types. Extraction providers (Tavily/Exa/You.com/Jina/Firecrawl) additionally throw when the API returns an empty body or a vendor-level failure (e.g. Firecrawl `success: false`, Tavily `failed_results`).
 
 ## Commands
 
@@ -117,7 +119,7 @@ Throws on invalid URL, non-http(s) protocol, private/loopback hostnames (SSRF gu
 
 First match wins:
 
-1. The active provider's environment variable: `BRAVE_SEARCH_API_KEY`, `TAVILY_API_KEY`, `SERPER_API_KEY`, `EXA_API_KEY`, `JINA_API_KEY`, `FIRECRAWL_API_KEY`, `SEARXNG_API_KEY`, or `OLLAMA_API_KEY`
+1. The active provider's environment variable: `BRAVE_SEARCH_API_KEY`, `TAVILY_API_KEY`, `SERPER_API_KEY`, `EXA_API_KEY`, `YOUCOM_API_KEY`, `JINA_API_KEY`, `FIRECRAWL_API_KEY`, `PERPLEXITY_API_KEY`, `SEARXNG_API_KEY`, or `OLLAMA_API_KEY`
 2. `apiKeys.<provider>` field in `~/.config/rpiv-web-tools/config.json`
 3. Legacy `apiKey` field (Brave only — auto-migrated to the new shape on next save)
 

package/package.json

@@ -1,7 +1,7 @@
 {
 	"name": "@juicesharp/rpiv-web-tools",
-	"version": "1.17.0",
-	"description": "Pi extension. Web search and fetch for the model with pluggable providers (Brave, Tavily, Serper, Exa, Jina, Firecrawl, SearXNG, Ollama).",
+	"version": "1.17.1",
+	"description": "Pi extension. Web search and fetch for the model with pluggable providers (Brave, Tavily, Serper, Exa, You.com, Jina, Firecrawl, Perplexity, SearXNG, Ollama).",
 	"keywords": [
 		"pi-package",
 		"pi-extension",
@@ -21,8 +21,10 @@
 		"tavily",
 		"serper",
 		"exa",
+		"youcom",
 		"jina",
 		"firecrawl",
+		"perplexity",
 		"searxng",
 		"ollama"
 	],
@@ -57,7 +59,7 @@
 		]
 	},
 	"dependencies": {
-		"@juicesharp/rpiv-config": "^1.17.0"
+		"@juicesharp/rpiv-config": "^1.17.1"
 	},
 	"peerDependencies": {
 		"@earendil-works/pi-coding-agent": "*",

package/providers/factory.ts

@@ -3,10 +3,12 @@ import { ExaProvider } from "./exa.js";
 import { FirecrawlProvider } from "./firecrawl.js";
 import { JinaProvider } from "./jina.js";
 import { OllamaProvider } from "./ollama.js";
+import { PerplexityProvider } from "./perplexity.js";
 import { SearxngProvider } from "./searxng.js";
 import { SerperProvider } from "./serper.js";
 import { TavilyProvider } from "./tavily.js";
 import type { FullProvider, SearchProvider } from "./types.js";
+import { YouComProvider } from "./youcom.js";
 
 export interface ProviderCredentials {
 	apiKey?: string;
@@ -28,10 +30,14 @@ export function createSearchProvider(name: string, creds: ProviderCredentials):
 			return new SerperProvider(apiKey);
 		case "exa":
 			return new ExaProvider(apiKey);
+		case "youcom":
+			return new YouComProvider(apiKey);
 		case "jina":
 			return new JinaProvider(apiKey);
 		case "firecrawl":
 			return new FirecrawlProvider(apiKey);
+		case "perplexity":
+			return new PerplexityProvider(apiKey);
 		case "searxng":
 			return new SearxngProvider({ apiKey: creds.apiKey, baseUrl: creds.baseUrl ?? "" });
 		case "ollama":

package/providers/index.ts

@@ -3,10 +3,12 @@ import { EXA_PROVIDER_META } from "./exa.js";
 import { FIRECRAWL_PROVIDER_META } from "./firecrawl.js";
 import { JINA_PROVIDER_META } from "./jina.js";
 import { OLLAMA_PROVIDER_META } from "./ollama.js";
+import { PERPLEXITY_PROVIDER_META } from "./perplexity.js";
 import { SEARXNG_PROVIDER_META } from "./searxng.js";
 import { SERPER_PROVIDER_META } from "./serper.js";
 import { TAVILY_PROVIDER_META } from "./tavily.js";
 import type { ProviderMeta } from "./types.js";
+import { YOUCOM_PROVIDER_META } from "./youcom.js";
 
 export { BRAVE_API_KEY_ENV_VAR, BRAVE_PROVIDER_META, BraveProvider } from "./brave.js";
 export { EXA_API_KEY_ENV_VAR, EXA_PROVIDER_META, ExaProvider } from "./exa.js";
@@ -32,6 +34,7 @@ export {
 	OLLAMA_PROVIDER_META,
 	OllamaProvider,
 } from "./ollama.js";
+export { PERPLEXITY_API_KEY_ENV_VAR, PERPLEXITY_PROVIDER_META, PerplexityProvider } from "./perplexity.js";
 export {
 	configureSearxng,
 	SEARXNG_API_KEY_ENV_VAR,
@@ -58,6 +61,7 @@ export type {
 	SearchResponse,
 	SearchResult,
 } from "./types.js";
+export { YOUCOM_API_KEY_ENV_VAR, YOUCOM_PROVIDER_META, YouComProvider } from "./youcom.js";
 
 // Typed as readonly ProviderMeta[] (not `as const`) so iterators can access
 // the optional META fields (baseUrlEnvVar, defaultBaseUrl, configure) without
@@ -68,8 +72,10 @@ export const PROVIDERS: readonly ProviderMeta[] = [
 	TAVILY_PROVIDER_META,
 	SERPER_PROVIDER_META,
 	EXA_PROVIDER_META,
+	YOUCOM_PROVIDER_META,
 	JINA_PROVIDER_META,
 	FIRECRAWL_PROVIDER_META,
+	PERPLEXITY_PROVIDER_META,
 	SEARXNG_PROVIDER_META,
 	OLLAMA_PROVIDER_META,
 ];

package/providers/perplexity.ts

@@ -0,0 +1,67 @@
+import type { SearchProvider, SearchResponse, SearchResult } from "./types.js";
+
+const PERPLEXITY_API_URL = "https://api.perplexity.ai/search";
+export const PERPLEXITY_API_KEY_ENV_VAR = "PERPLEXITY_API_KEY";
+export const PERPLEXITY_PROVIDER_META = {
+	name: "perplexity",
+	label: "Perplexity",
+	envVar: PERPLEXITY_API_KEY_ENV_VAR,
+	roles: ["search"] as const,
+} as const;
+
+interface PerplexityRawResult {
+	title?: string;
+	url?: string;
+	snippet?: string;
+	date?: string | null;
+	last_updated?: string | null;
+}
+
+interface PerplexityRawResponse {
+	results?: PerplexityRawResult[];
+	id?: string;
+	server_time?: string | null;
+}
+
+function normalizePerplexityResults(results: PerplexityRawResult[]): SearchResult[] {
+	return results.map((r) => ({
+		title: r.title ?? "",
+		url: r.url ?? "",
+		snippet: r.snippet ?? "",
+	}));
+}
+
+export class PerplexityProvider implements SearchProvider {
+	readonly name = PERPLEXITY_PROVIDER_META.name;
+	readonly label = PERPLEXITY_PROVIDER_META.label;
+	readonly envVar = PERPLEXITY_PROVIDER_META.envVar;
+
+	constructor(private readonly apiKey: string) {}
+
+	async search(query: string, maxResults: number, signal?: AbortSignal): Promise<SearchResponse> {
+		if (!this.apiKey) {
+			throw new Error(`${this.envVar} is not set. Run /web-tools to configure, or export the env var.`);
+		}
+
+		const res = await fetch(PERPLEXITY_API_URL, {
+			method: "POST",
+			headers: {
+				"Content-Type": "application/json",
+				Authorization: `Bearer ${this.apiKey}`,
+			},
+			body: JSON.stringify({
+				query,
+				max_results: maxResults,
+			}),
+			signal,
+		});
+
+		if (!res.ok) {
+			const text = await res.text();
+			throw new Error(`${this.label} Search API error (${res.status}): ${text}`);
+		}
+
+		const raw = (await res.json()) as PerplexityRawResponse;
+		return { query, results: normalizePerplexityResults(raw.results ?? []) };
+	}
+}

package/providers/youcom.ts

@@ -0,0 +1,110 @@
+import type { FetchResponse, FullProvider, SearchResponse, SearchResult } from "./types.js";
+
+const YOUCOM_SEARCH_URL = "https://ydc-index.io/v1/search";
+const YOUCOM_CONTENTS_URL = "https://ydc-index.io/v1/contents";
+export const YOUCOM_API_KEY_ENV_VAR = "YOUCOM_API_KEY";
+export const YOUCOM_PROVIDER_META = {
+	name: "youcom",
+	label: "You.com",
+	envVar: YOUCOM_API_KEY_ENV_VAR,
+	roles: ["search", "fetch"] as const,
+} as const;
+
+interface YouComWebResult {
+	url?: string;
+	title?: string;
+	description?: string;
+	snippets?: string[];
+}
+
+interface YouComSearchResponse {
+	results?: {
+		web?: YouComWebResult[];
+	};
+}
+
+interface YouComContentsResponseItem {
+	url: string;
+	title?: string;
+	markdown?: string | null;
+}
+
+function normalizeYouComResults(results: YouComWebResult[]): SearchResult[] {
+	return results.map((r) => ({
+		title: r.title ?? "",
+		url: r.url ?? "",
+		snippet: r.snippets?.[0] ?? r.description ?? "",
+	}));
+}
+
+export class YouComProvider implements FullProvider {
+	readonly name = YOUCOM_PROVIDER_META.name;
+	readonly label = YOUCOM_PROVIDER_META.label;
+	readonly envVar = YOUCOM_API_KEY_ENV_VAR;
+
+	constructor(private readonly apiKey: string) {}
+
+	async search(query: string, maxResults: number, signal?: AbortSignal): Promise<SearchResponse> {
+		if (!this.apiKey) {
+			throw new Error(`${this.envVar} is not set. Run /web-tools to configure, or export the env var.`);
+		}
+
+		const res = await fetch(YOUCOM_SEARCH_URL, {
+			method: "POST",
+			headers: {
+				"Content-Type": "application/json",
+				"X-API-Key": this.apiKey,
+			},
+			body: JSON.stringify({
+				query,
+				count: maxResults,
+			}),
+			signal,
+		});
+
+		if (!res.ok) {
+			const text = await res.text();
+			throw new Error(`${this.label} Search API error (${res.status}): ${text}`);
+		}
+
+		const raw = (await res.json()) as YouComSearchResponse;
+		return { query, results: normalizeYouComResults(raw.results?.web ?? []) };
+	}
+
+	async fetch(url: string, _raw: boolean, signal?: AbortSignal): Promise<FetchResponse> {
+		if (!this.apiKey) {
+			throw new Error(`${this.envVar} is not set. Run /web-tools to configure, or export the env var.`);
+		}
+
+		const res = await fetch(YOUCOM_CONTENTS_URL, {
+			method: "POST",
+			headers: {
+				"Content-Type": "application/json",
+				"X-API-Key": this.apiKey,
+			},
+			body: JSON.stringify({
+				urls: [url],
+				formats: ["markdown"],
+			}),
+			signal,
+		});
+
+		if (!res.ok) {
+			const text = await res.text();
+			throw new Error(`${this.label} Fetch API error (${res.status}): ${text}`);
+		}
+
+		const raw = (await res.json()) as YouComContentsResponseItem[];
+		const item = raw[0];
+
+		if (!item?.markdown) {
+			throw new Error(`${this.label} Fetch API error: no content returned for ${url}`);
+		}
+
+		return {
+			text: item.markdown,
+			title: item.title || undefined,
+			contentType: "text/markdown",
+		};
+	}
+}