@specstream/openclaw-plugin 0.1.2 → 0.2.1

package/README.md

@@ -1,9 +1,18 @@
 # @specstream/openclaw-plugin
 
-Ask your OpenClaw agent for a list and get the whole list back. "Dental clinics in Lisbon with online booking, with their website and phone." Every row comes from the live web with the page it was read from, so you can open it and look. You pay for each result delivered, and a search that finds nothing costs nothing.
+[SpecStream](https://specstream.ai) is the live web data engine. One platform to build, search, extract, enrich, map, and monitor data across the entire web.
 
-[SpecStream](https://specstream.ai) is the live web data engine for teams and agents. This plugin gives OpenClaw two tools for it and, if you want the full conversation with SpecStream, four skills.
+Ask for any dataset in plain words, and SpecStream builds it from the entire live web: every match found, extracted and enriched into structured records, then monitored for change. It runs through the API and MCP, and inside Sheets, CRMs, Slack, n8n, Zapier and the other tools teams already use.
 
+Source URL and read time on every value.
+
+Three products:
+
+1. Search API: `GET /v1/web-search`, $5 per 1,000 searches. The most accurate web search API in the world (scores and method: https://specstream.ai/benchmarks).
+2. Extract API: `POST /v1/extract`, up to 20 URLs a call (web pages, PDF, Word, Excel), $0.80 per 1,000 pages. A page with no text is not charged.
+3. Deep Search: ask in plain words and get every match on the live web, in structured rows, priced per row (5 cents, 15 cents or $1 a row, plus 5 cents for a person's email and 15 cents for a phone). A run that finds nothing costs nothing.
+
+This plugin gives OpenClaw four tools (`specstream_web_search`, `specstream_extract_pages`, `specstream_find_list`, `specstream_get_list_rows`), registers SpecStream as an OpenClaw web search and web fetch provider, and ships six skills for the hosted MCP server.
 ## Install
 
 From ClawHub:
@@ -22,7 +31,7 @@ Needs OpenClaw 2026.10.3 or newer.
 
 ## Set it up
 
-Create an API key under **Agents & API** in your [SpecStream workspace](https://specstream.ai/workspace). New accounts start with $10 of work free for 30 days, no card.
+Create an API key under **Agents & API** in your [SpecStream workspace](https://specstream.ai/workspace). New accounts start with $10 of credit for 30 days, no card.
 
 The plugin reads the key from `SPECSTREAM_API_KEY`. To keep it in your OpenClaw config instead, store a SecretRef that points at the variable:
 
@@ -47,6 +56,24 @@ Give the key only what the task needs: its scope, projects, an expiry and a spen
 
 ## The tools
 
+### `specstream_web_search`
+
+| Input | |
+| --- | --- |
+| `q` | The search question or request in everyday words (1 to 2,000 characters). Quoted phrases are searched as phrases |
+| `passages` | Optional. Set `true` to include the main text passages of the top matching pages |
+
+Calls `GET /v1/web-search` ($5 per 1,000 searches, no quote needed). Returns `kind` (`fact` or `list`), `answer` (with `value`, `page` and `seen_at`), `rows`, ranked `sources` and optional `passages`.
+
+### `specstream_extract_pages`
+
+| Input | |
+| --- | --- |
+| `urls` | 1 to 20 `http` or `https` URLs (web pages, PDF, Word, Excel) |
+| `format` | Optional. `"markdown"` (default) or `"text"` |
+
+Calls `POST /v1/extract` ($0.80 per 1,000 pages, a page with no text is free, no quote needed). Returns `results` with `url`, `status`, `title`, `published`, `read_from` and `text` for each URL.
+
 ### `specstream_find_list`
 
 | Input | |
@@ -55,7 +82,7 @@ Give the key only what the task needs: its scope, projects, an expiry and a spen
 | `max_results` | How many rows, 1 to 1000. Default 25 |
 | `max_price_usd` | Optional. A lower ceiling for this call |
 
-The tool reads the request, gets the price, and starts the run only if the quoted maximum is within your ceiling. The agent can ask for a lower ceiling on a call and can never raise yours. If the price is above the ceiling, nothing runs and the agent is told so. Results cost 10 cents, 30 cents or $1 each depending on the list, plus 10 cents for a person's email and 25 cents for a phone number, and nothing when nothing matches.
+The tool reads the request, gets the price, and starts the run only if the quoted maximum is within your ceiling. The agent can ask for a lower ceiling on a call and can never raise yours. If the price is above the ceiling, nothing runs and the agent is told so. Results cost 5 cents, 15 cents or $1 each depending on the list, plus 5 cents for a person's email and 15 cents for a phone number, and nothing when nothing matches.
 
 It waits for the list, then returns JSON: `state`, `list_id`, `list_link` and `rows`. Each row has its columns plus `source_url` and `checked_at`. A retry of the same request with the same limits within the hour reaches the run it already started, so a flaky turn does not buy the list twice.
 
@@ -65,7 +92,7 @@ Reads every row of a list you already own, by `list_id`. Free. Use it when a big
 
 ## Skills and the SpecStream server (optional)
 
-The plugin ships four skills: `specstream-compile` (read the request back), `specstream-review-work` (get the price), `specstream-approve-work` (approve it) and `specstream-continue-work` (read the rows, send them to Sheets, Notion, HubSpot and more, keep them current). They use the hosted SpecStream server and stay hidden until you add it:
+The plugin ships six skills: `specstream-web-search` (Search API), `specstream-extract-pages` (Extract API), `specstream-compile` (show how SpecStream read a dataset request), `specstream-review-work` (get the price), `specstream-approve-work` (approve it) and `specstream-continue-work` (read the rows, send them to Sheets, Notion, HubSpot and more, keep them up to date). They use the hosted SpecStream server and stay hidden until you add it:
 
 ```sh
 openclaw mcp set specstream '{"url":"https://api.specstream.ai/mcp","transport":"streamable-http","auth":"oauth"}'
@@ -74,12 +101,6 @@ openclaw mcp login specstream
 
 The tools above work without it.
 
-## Example
-
-> Find food banks in Texas with their annual revenue. Keep it under $3.
-
-The agent calls `specstream_find_list` with `max_price_usd: 3`, and you get a table of food banks, each row with the page it came from, plus a link to open the list in your workspace.
-
 ## Develop
 
 ```sh

package/dist/client.js

@@ -1,6 +1,6 @@
 /** The SpecStream REST calls the tools share: read a request, price it, start it within a maximum, read its rows. */
 export const BASE_URL = "https://api.specstream.ai";
-export const USER_AGENT = "@specstream/openclaw-plugin/0.1.1";
+export const USER_AGENT = "@specstream/openclaw-plugin/0.2.0";
 /** A list runs for up to about five minutes, so the default wait is just under that. */
 export const DEFAULT_WAIT_SECONDS = 300;
 export const DEFAULT_POLL_INTERVAL_SECONDS = 3;
@@ -146,6 +146,27 @@ export class SpecStreamClient {
             throw new SpecStreamError("SpecStream returned a response this plugin could not read.");
         return data;
     }
+    /** Search the live web with GET /v1/web-search ($5 per 1,000 searches, no quote needed). */
+    async webSearch(query, options = {}) {
+        const q = (query ?? "").trim();
+        if (!q)
+            throw new Error("Provide a search query in q.");
+        if (q.length > 2000)
+            throw new Error("q must be at most 2,000 characters.");
+        const params = new URLSearchParams({ q });
+        if (options.passages)
+            params.set("passages", "true");
+        return (await this.request("GET", `/v1/web-search?${params}`, undefined, { abortSignal: options.abortSignal })) ?? {};
+    }
+    /** Extract the main text of up to 20 URLs live with POST /v1/extract ($0.80 per 1,000 pages, empty pages free). */
+    async extractPages(urls, options = {}) {
+        const cleaned = (Array.isArray(urls) ? urls : []).map((u) => String(u ?? "").trim()).filter(Boolean);
+        if (cleaned.length < 1 || cleaned.length > 20) {
+            throw new Error("Provide 1 to 20 http or https URLs in urls.");
+        }
+        const format = options.format === "text" ? "text" : "markdown";
+        return ((await this.request("POST", "/v1/extract", { urls: cleaned, format }, { abortSignal: options.abortSignal })) ?? {});
+    }
     /** Every row found so far for a list, following the pages. Reading is free. */
     async results(findId, abortSignal) {
         const rows = [];

package/dist/index.js

@@ -12,19 +12,67 @@ const configSchema = Type.Object({
     })),
     waitSeconds: Type.Optional(Type.Integer({ minimum: 10, maximum: 900, description: "How long specstream_find_list waits for a running list. Defaults to 300." })),
 }, { additionalProperties: false });
-export default defineToolPlugin({
+const entry = defineToolPlugin({
     id: "specstream",
     name: "SpecStream",
-    description: "Ask for any list in plain words and get every matching row from the live web, each with its source page, priced per match.",
+    description: "The live web data engine. One platform to build, search, extract, enrich, map, and monitor data across the entire web. Source URL and read time on every value.",
     activation: { onStartup: false },
     configSchema,
     tools: (tool) => [
+        tool({
+            name: "specstream_web_search",
+            label: "Search the web",
+            description: [
+                "Search the live web in seconds with the SpecStream Search API (GET /v1/web-search, $5 per 1,000 searches, no quote needed).",
+                "Use it for a single fact, a question with sources, or to see the first rows of a request before pricing a full Deep Search dataset.",
+                "Returns JSON with kind ('fact' or 'list'), answer (value, page and seen_at), rows, ranked sources, and optional page passages.",
+            ].join(" "),
+            parameters: Type.Object({
+                q: Type.String({
+                    minLength: 1,
+                    maxLength: 2000,
+                    description: "The search question or request in everyday words (1 to 2,000 characters). Quoted phrases are searched as phrases.",
+                }),
+                passages: Type.Optional(Type.Boolean({
+                    description: "Set true to include the main text passages of the top matching pages in the response.",
+                })),
+            }),
+            execute: async ({ q, passages }, config, { signal }) => {
+                const settings = resolveConfig(config);
+                const client = new SpecStreamClient({ apiKey: settings.apiKey });
+                return client.webSearch(q, { passages, abortSignal: signal });
+            },
+        }),
+        tool({
+            name: "specstream_extract_pages",
+            label: "Extract pages",
+            description: [
+                "Read the main text, title and published date of 1 to 20 web pages, PDFs, Word documents or Excel spreadsheets live with the SpecStream Extract API (POST /v1/extract, $0.80 per 1,000 pages).",
+                "A page that returns no text is not charged, and no quote or approval is needed.",
+                "Returns JSON with results (one entry per URL: url, status, title, published, read_from and text), took_ms and price_usd.",
+            ].join(" "),
+            parameters: Type.Object({
+                urls: Type.Array(Type.String({ minLength: 1 }), {
+                    minItems: 1,
+                    maxItems: 20,
+                    description: "1 to 20 http or https URLs to read live.",
+                }),
+                format: Type.Optional(Type.Union([Type.Literal("markdown"), Type.Literal("text")], {
+                    description: 'Output format: "markdown" (default, keeps headings, lists and links) or "text".',
+                })),
+            }),
+            execute: async ({ urls, format }, config, { signal }) => {
+                const settings = resolveConfig(config);
+                const client = new SpecStreamClient({ apiKey: settings.apiKey });
+                return client.extractPages(urls, { format, abortSignal: signal });
+            },
+        }),
         tool({
             name: "specstream_find_list",
             label: "Find a list",
             description: [
                 "Find a list of real things on the live web and return every matching row, each with the page it was read from.",
-                "Use it when the answer is many things that meet conditions: companies in a city, local businesses with a service, funding filings, open grants, tenders, clinical trials, repositories, people in a role.",
+                "Use it when the answer is many things that meet conditions.",
                 "For one fact or a summary of one page, use a search tool instead.",
                 "It is priced per match. Rows that could not be confirmed are left out and not charged. The run starts only if the quoted maximum is within the user's price ceiling.",
                 "Returns JSON with state, list_id, list_link and rows (one object per match: its columns, source_url and checked_at).",
@@ -35,8 +83,7 @@ export default defineToolPlugin({
             parameters: Type.Object({
                 request: Type.String({
                     minLength: 1,
-                    description: "The list in plain words: what to find, where, what every row must meet, and the facts to bring back. " +
-                        'Example: "Dental clinics in Lisbon with online booking, with their website and phone".',
+                    description: "The list in plain words: what to find, where, what every row must meet, and the facts to bring back.",
                 }),
                 max_results: Type.Optional(Type.Integer({
                     minimum: 1,
@@ -79,3 +126,93 @@ export default defineToolPlugin({
         }),
     ],
 });
+/** The plugin config a provider sees: the host passes the whole config, and the plugin entry holds ours. */
+function providerConfig(host, fallback) {
+    const entries = host?.plugins?.entries;
+    return entries?.specstream?.config ?? fallback;
+}
+const credentialAccess = {
+    getCredentialValue: (config) => config?.apiKey,
+    setCredentialValue: (target, value) => {
+        target.apiKey = value;
+    },
+};
+const baseRegister = entry.register.bind(entry);
+entry.register = (api) => {
+    baseRegister(api);
+    if (typeof api?.registerWebSearchProvider === "function") {
+        const search = {
+            id: "specstream",
+            label: "SpecStream Search",
+            hint: "The most accurate web search API in the world ($5 per 1,000 searches, source URL and read time on every value)",
+            onboardingScopes: ["text-inference"],
+            credentialLabel: "SpecStream API key",
+            envVars: ["SPECSTREAM_API_KEY"],
+            placeholder: "ss_live_...",
+            signupUrl: "https://specstream.ai/search-api?utm_source=openclaw",
+            docsUrl: "https://specstream.ai/docs/api?utm_source=openclaw",
+            autoDetectOrder: 55,
+            credentialPath: "plugins.entries.specstream.config.apiKey",
+            ...credentialAccess,
+            createTool: (ctx) => ({
+                description: "Search the live web with the SpecStream Search API. Returns an answer with its source URL and read time, matching rows, ranked sources and optional page passages.",
+                parameters: {
+                    type: "object",
+                    properties: {
+                        query: { type: "string", description: "Search query in everyday words." },
+                        passages: { type: "boolean", description: "Include main text passages from the closest pages." },
+                    },
+                    required: ["query"],
+                    additionalProperties: false,
+                },
+                execute: async (args, context) => {
+                    context?.signal?.throwIfAborted();
+                    const settings = resolveConfig(providerConfig(ctx?.config, api?.pluginConfig));
+                    const client = new SpecStreamClient({ apiKey: settings.apiKey });
+                    return client.webSearch(String(args?.query ?? ""), {
+                        passages: args?.passages === true,
+                        abortSignal: context?.signal,
+                    });
+                },
+            }),
+        };
+        api.registerWebSearchProvider(search);
+    }
+    if (typeof api?.registerWebFetchProvider === "function") {
+        const fetchProvider = {
+            id: "specstream",
+            label: "SpecStream Extract",
+            hint: "Read the main text of web pages, PDFs, Word and Excel files live ($0.80 per 1,000 pages, empty pages free)",
+            credentialLabel: "SpecStream API key",
+            envVars: ["SPECSTREAM_API_KEY"],
+            placeholder: "ss_live_...",
+            signupUrl: "https://specstream.ai/extract-api?utm_source=openclaw",
+            docsUrl: "https://specstream.ai/docs/api?utm_source=openclaw",
+            credentialPath: "plugins.entries.specstream.config.apiKey",
+            ...credentialAccess,
+            createTool: (ctx) => ({
+                description: "Extract the clean main text, title and published date of a URL using the SpecStream Extract API.",
+                parameters: {
+                    type: "object",
+                    properties: {
+                        url: { type: "string", description: "HTTP or HTTPS URL to read live." },
+                        extractMode: { type: "string", enum: ["markdown", "text"], description: 'Output format ("markdown" or "text").' },
+                    },
+                    required: ["url"],
+                    additionalProperties: false,
+                },
+                execute: async (args, context) => {
+                    context?.signal?.throwIfAborted();
+                    const settings = resolveConfig(providerConfig(ctx?.config, api?.pluginConfig));
+                    const client = new SpecStreamClient({ apiKey: settings.apiKey });
+                    return client.extractPages([String(args?.url ?? "")], {
+                        format: args?.extractMode === "text" ? "text" : "markdown",
+                        abortSignal: context?.signal,
+                    });
+                },
+            }),
+        };
+        api.registerWebFetchProvider(fetchProvider);
+    }
+};
+export default entry;

package/openclaw.plugin.json

@@ -1,11 +1,11 @@
 {
   "id": "specstream",
   "name": "SpecStream",
-  "description": "Ask for any list in plain words and get every matching row from the live web, each with its source page, priced per match.",
+  "description": "The live web data engine. One platform to build, search, extract, enrich, map, and monitor data across the entire web. Source URL and read time on every value.",
   "categories": [
     "data-analytics"
   ],
-  "version": "0.1.2",
+  "version": "0.2.1",
   "configSchema": {
     "type": "object",
     "properties": {
@@ -73,8 +73,16 @@
   },
   "contracts": {
     "tools": [
+      "specstream_web_search",
+      "specstream_extract_pages",
       "specstream_find_list",
       "specstream_get_list_rows"
+    ],
+    "webSearchProviders": [
+      "specstream"
+    ],
+    "webFetchProviders": [
+      "specstream"
     ]
   }
 }

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@specstream/openclaw-plugin",
-  "version": "0.1.2",
-  "description": "SpecStream plugin for OpenClaw: ask for any list in plain words and get every matching row from the live web, each with its source page, priced per match.",
+  "version": "0.2.1",
+  "description": "The live web data engine. One platform to build, search, extract, enrich, map, and monitor data across the entire web.",
   "type": "module",
   "files": [
     "dist",
@@ -15,7 +15,9 @@
     "openclaw",
     "openclaw-plugin",
     "specstream",
-    "lead-lists",
+    "search-api",
+    "extract-api",
+    "web-search",
     "web-data",
     "live-web",
     "agent-tools",

package/skills/specstream-extract-pages/SKILL.md

@@ -0,0 +1,11 @@
+---
+name: specstream-extract-pages
+description: Read the main text of up to 20 web pages, PDFs, Word or Excel files live with the Extract API ($0.80 per 1,000 pages).
+metadata: { "openclaw": { "requires": { "config": ["mcp.servers.specstream"] } } }
+---
+
+# Read pages and documents live
+
+SpecStream is the live web data engine for teams and agents. Use the configured MCP server at `https://api.specstream.ai/mcp`.
+
+Use `read_pages` with `{"urls": ["<url>", ...]}` (1 to 20 `http` or `https` URLs) to read the clean main text, title and published date of web pages, PDFs, Word documents or Excel spreadsheets. Each page that returns text costs $0.0008 ($0.80 per 1,000 pages) from the account balance, and a page with no text is not charged. No quote or approval is needed.

package/skills/specstream-web-search/SKILL.md

@@ -0,0 +1,13 @@
+---
+name: specstream-web-search
+description: Answer a question from the live web in seconds with the Search API ($5 per 1,000 searches, no quote needed).
+metadata: { "openclaw": { "requires": { "config": ["mcp.servers.specstream"] } } }
+---
+
+# Search the live web
+
+SpecStream is the live web data engine for teams and agents. Use the configured MCP server at `https://api.specstream.ai/mcp`.
+
+Use `search_web` with `{"q": "<the question>"}` when the person needs one fact, a fast web search with sources, or a quick look at the first rows of a request before pricing a full Deep Search run. Set `"passages": true` when you want the main text passages of the closest pages included in the reply.
+
+Each call costs $0.005 ($5 per 1,000 searches) from the account balance and needs no quote or approval. The reply returns `kind` (`fact` or `list`), `answer` (with `value`, `page` and `seen_at`), `rows` and ranked `sources`. When `kind` is `list`, `handoff` gives the estimated size and the `compile` call to price and run the full dataset.