llama-index-tools-specstream 0.1.0__tar.gz

llama_index_tools_specstream-0.1.0/.gitignore

@@ -0,0 +1,14 @@
+node_modules/
+dist/
+build/
+*.egg-info/
+__pycache__/
+.venv/
+.pytest_cache/
+*.tsbuildinfo
+*.tgz
+*.whl
+.env
+.env.*
+!.env.example
+.DS_Store

llama_index_tools_specstream-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 SpecStream SAS
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

llama_index_tools_specstream-0.1.0/PKG-INFO

@@ -0,0 +1,134 @@
+Metadata-Version: 2.5
+Name: llama-index-tools-specstream
+Version: 0.1.0
+Summary: Give a LlamaIndex agent any list from the live web: ask in plain words, get every matching row with the page it came from, within a price you set in code.
+Project-URL: Homepage, https://specstream.ai
+Project-URL: Documentation, https://specstream.ai/docs/llamaindex
+Project-URL: Repository, https://github.com/Specstreamai/specstream-tools/tree/main/packages/llama-index-tools-specstream
+Project-URL: Issues, https://github.com/Specstreamai/specstream-tools/issues
+Project-URL: Pricing, https://specstream.ai/pricing
+Author-email: SpecStream <support@specstream.ai>
+Maintainer-email: SpecStream <support@specstream.ai>
+License-Expression: MIT
+License-File: LICENSE
+Keywords: agents,lead-lists,live-web,llama-index,llamaindex,research,retriever,specstream,tools,web-data
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Internet :: WWW/HTTP
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Typing :: Typed
+Requires-Python: <4.0,>=3.10
+Requires-Dist: llama-index-core<0.15,>=0.13.0
+Provides-Extra: test
+Requires-Dist: pytest>=8; extra == 'test'
+Description-Content-Type: text/markdown
+
+# llama-index-tools-specstream
+
+Give your LlamaIndex agent any list from the live web. Search tools hand an agent pages to read; SpecStream hands it the finished list. It asks in plain words – "dental clinics in Lisbon with online booking", "US startups that filed a $5M+ Form D this week" – and gets back every matching row, each with the page it was read from. The list starts only within a price you set in code, and the model can't raise it.
+
+```bash
+pip install -U llama-index-tools-specstream
+export SPECSTREAM_API_KEY=ss_live_...   # Agents & API in your SpecStream workspace
+```
+
+A free account comes with $10 of work to use within 30 days and needs no card: [specstream.ai](https://specstream.ai/signup).
+
+## An agent that can find lists
+
+```python
+from llama_index.core.agent.workflow import FunctionAgent
+from llama_index.llms.anthropic import Anthropic
+from llama_index.tools.specstream import SpecStreamToolSpec
+
+agent = FunctionAgent(
+    tools=SpecStreamToolSpec(max_price_usd=5).to_tool_list(),
+    llm=Anthropic(model="claude-sonnet-5-5"),
+)
+response = await agent.run(
+    "Find US startups that raised $5M or more in a Form D filed after October 8, 2026. Which industries lead?"
+)
+```
+
+The agent writes the request and picks how many rows. You decide what one list may cost.
+
+## What comes back
+
+One flat row per match: the list's own columns, then where the row was read. This is a real row from the request above.
+
+```json
+{
+  "list_id": "fnd_gk5y3d0r97bvyz0t08zzh75kr6",
+  "position": 26,
+  "Organization": "RapidCanvas Inc",
+  "Published address": "801 BARTON SPRINGS RD, AUSTIN, TEXAS, 78704",
+  "Source industry classification": "Other Technology",
+  "Amount sold (USD)": "28999988",
+  "Funding filing date": "2026-10-14",
+  "source_url": "https://www.sec.gov/Archives/edgar/data/2052031/000205203126000006/primary_doc.xml",
+  "checked_at": "2026-10-14T19:56:02.016693Z",
+  "list_link": "https://specstream.ai/workspace/collections/fnd_gk5y3d0r97bvyz0t08zzh75kr6"
+}
+```
+
+Columns follow the request. Ask for "breweries in Portland, Oregon that host events, with their website" and the rows carry the brewery and its website instead. A fact a source doesn't publish comes back empty, never guessed.
+
+## The two tools
+
+| Tool | The model gives | What happens | Cost |
+| --- | --- | --- | --- |
+| `specstream_find_list` | `request`, `max_results` | Reads the request, prices it, starts it only within `max_price_usd`, waits up to `wait_seconds` (7 minutes by default) and returns every row | Only the rows that match |
+| `specstream_get_list_results` | `list_id` | Every row found so far, for a list that was still running | Free |
+
+When the price is above your maximum, nothing runs: the tool returns the price, so the agent can ask for fewer rows or tell the user. When a request is too vague to price ("clinics"), it returns how SpecStream read it, so the agent can add the place or the conditions.
+
+Settings live on the spec, never in the tool schema:
+
+```python
+spec = SpecStreamToolSpec(
+    max_price_usd=20,     # the most one list may cost
+    wait_seconds=600,     # how long find_list waits before handing back what it has
+    locale="en-GB",       # the language the requests are written in
+)
+```
+
+## As a retriever
+
+```python
+from llama_index.tools.specstream import SpecStreamRetriever
+
+retriever = SpecStreamRetriever(max_price_usd=3, max_results=25)
+nodes = retriever.retrieve("Breweries in Portland, Oregon that host events, with their website")
+
+nodes[0].text                      # "Business location: Occidental Brewing Company\nPublished business website: http://occidentalbeers.com ..."
+nodes[0].metadata["source_url"]    # the page that row was read from
+```
+
+One node per row, its facts as text and its source in the metadata. The model sees each row with the page behind it, so a query engine can cite every claim; the list's bookkeeping stays out of the prompt and out of embeddings. Above the maximum, the retriever raises `SpecStreamError` and nothing runs.
+
+## Money, plainly
+
+- `max_price_usd` lives on the spec, in your code. The model never sees it and can't change it.
+- You pay for rows that match. Candidates SpecStream couldn't confirm are left out and aren't billed.
+- A retried step asking for the same list within the hour gets the list it already started, not a second bill.
+- Rates and plans: [specstream.ai/pricing](https://specstream.ai/pricing).
+
+## When to reach for it
+
+When the answer is many things that meet conditions: companies in a city, clinics with a service, funding filings, open grants, tenders, repositories, people in a role. For a single fact or a summary of one page, a search tool fits better.
+
+## More
+
+- [SpecStream docs](https://specstream.ai/docs) and the [LlamaIndex page](https://specstream.ai/docs/llamaindex)
+- Agents outside LlamaIndex: the [MCP server](https://specstream.ai/docs/mcp) at `https://api.specstream.ai/mcp`
+- The [Python SDK](https://pypi.org/project/specstream-client/) for everything else in the API
+- Questions: support@specstream.ai
+
+MIT licensed. Made by [SpecStream](https://specstream.ai), the live web data engine for teams and agents.

llama_index_tools_specstream-0.1.0/README.md

@@ -0,0 +1,102 @@
+# llama-index-tools-specstream
+
+Give your LlamaIndex agent any list from the live web. Search tools hand an agent pages to read; SpecStream hands it the finished list. It asks in plain words – "dental clinics in Lisbon with online booking", "US startups that filed a $5M+ Form D this week" – and gets back every matching row, each with the page it was read from. The list starts only within a price you set in code, and the model can't raise it.
+
+```bash
+pip install -U llama-index-tools-specstream
+export SPECSTREAM_API_KEY=ss_live_...   # Agents & API in your SpecStream workspace
+```
+
+A free account comes with $10 of work to use within 30 days and needs no card: [specstream.ai](https://specstream.ai/signup).
+
+## An agent that can find lists
+
+```python
+from llama_index.core.agent.workflow import FunctionAgent
+from llama_index.llms.anthropic import Anthropic
+from llama_index.tools.specstream import SpecStreamToolSpec
+
+agent = FunctionAgent(
+    tools=SpecStreamToolSpec(max_price_usd=5).to_tool_list(),
+    llm=Anthropic(model="claude-sonnet-5-5"),
+)
+response = await agent.run(
+    "Find US startups that raised $5M or more in a Form D filed after October 8, 2026. Which industries lead?"
+)
+```
+
+The agent writes the request and picks how many rows. You decide what one list may cost.
+
+## What comes back
+
+One flat row per match: the list's own columns, then where the row was read. This is a real row from the request above.
+
+```json
+{
+  "list_id": "fnd_gk5y3d0r97bvyz0t08zzh75kr6",
+  "position": 26,
+  "Organization": "RapidCanvas Inc",
+  "Published address": "801 BARTON SPRINGS RD, AUSTIN, TEXAS, 78704",
+  "Source industry classification": "Other Technology",
+  "Amount sold (USD)": "28999988",
+  "Funding filing date": "2026-10-14",
+  "source_url": "https://www.sec.gov/Archives/edgar/data/2052031/000205203126000006/primary_doc.xml",
+  "checked_at": "2026-10-14T19:56:02.016693Z",
+  "list_link": "https://specstream.ai/workspace/collections/fnd_gk5y3d0r97bvyz0t08zzh75kr6"
+}
+```
+
+Columns follow the request. Ask for "breweries in Portland, Oregon that host events, with their website" and the rows carry the brewery and its website instead. A fact a source doesn't publish comes back empty, never guessed.
+
+## The two tools
+
+| Tool | The model gives | What happens | Cost |
+| --- | --- | --- | --- |
+| `specstream_find_list` | `request`, `max_results` | Reads the request, prices it, starts it only within `max_price_usd`, waits up to `wait_seconds` (7 minutes by default) and returns every row | Only the rows that match |
+| `specstream_get_list_results` | `list_id` | Every row found so far, for a list that was still running | Free |
+
+When the price is above your maximum, nothing runs: the tool returns the price, so the agent can ask for fewer rows or tell the user. When a request is too vague to price ("clinics"), it returns how SpecStream read it, so the agent can add the place or the conditions.
+
+Settings live on the spec, never in the tool schema:
+
+```python
+spec = SpecStreamToolSpec(
+    max_price_usd=20,     # the most one list may cost
+    wait_seconds=600,     # how long find_list waits before handing back what it has
+    locale="en-GB",       # the language the requests are written in
+)
+```
+
+## As a retriever
+
+```python
+from llama_index.tools.specstream import SpecStreamRetriever
+
+retriever = SpecStreamRetriever(max_price_usd=3, max_results=25)
+nodes = retriever.retrieve("Breweries in Portland, Oregon that host events, with their website")
+
+nodes[0].text                      # "Business location: Occidental Brewing Company\nPublished business website: http://occidentalbeers.com ..."
+nodes[0].metadata["source_url"]    # the page that row was read from
+```
+
+One node per row, its facts as text and its source in the metadata. The model sees each row with the page behind it, so a query engine can cite every claim; the list's bookkeeping stays out of the prompt and out of embeddings. Above the maximum, the retriever raises `SpecStreamError` and nothing runs.
+
+## Money, plainly
+
+- `max_price_usd` lives on the spec, in your code. The model never sees it and can't change it.
+- You pay for rows that match. Candidates SpecStream couldn't confirm are left out and aren't billed.
+- A retried step asking for the same list within the hour gets the list it already started, not a second bill.
+- Rates and plans: [specstream.ai/pricing](https://specstream.ai/pricing).
+
+## When to reach for it
+
+When the answer is many things that meet conditions: companies in a city, clinics with a service, funding filings, open grants, tenders, repositories, people in a role. For a single fact or a summary of one page, a search tool fits better.
+
+## More
+
+- [SpecStream docs](https://specstream.ai/docs) and the [LlamaIndex page](https://specstream.ai/docs/llamaindex)
+- Agents outside LlamaIndex: the [MCP server](https://specstream.ai/docs/mcp) at `https://api.specstream.ai/mcp`
+- The [Python SDK](https://pypi.org/project/specstream-client/) for everything else in the API
+- Questions: support@specstream.ai
+
+MIT licensed. Made by [SpecStream](https://specstream.ai), the live web data engine for teams and agents.

llama_index_tools_specstream-0.1.0/llama_index/tools/specstream/__init__.py

@@ -0,0 +1,8 @@
+"""SpecStream for LlamaIndex: any list from the live web, every row with its source, within a price set in code."""
+
+from llama_index.tools.specstream._api import SpecStreamError
+from llama_index.tools.specstream.base import SpecStreamToolSpec
+from llama_index.tools.specstream.retriever import SpecStreamRetriever
+
+__all__ = ["SpecStreamError", "SpecStreamRetriever", "SpecStreamToolSpec"]
+__version__ = "0.1.0"

llama_index_tools_specstream-0.1.0/llama_index/tools/specstream/_api.py

@@ -0,0 +1,206 @@
+"""The SpecStream REST calls the tools share: read a request, price it, start it within a maximum, read its rows."""
+
+from __future__ import annotations
+
+import hashlib
+import json
+import time
+import urllib.error
+import urllib.parse
+import urllib.request
+from dataclasses import dataclass
+from typing import Any, Callable
+
+DEFAULT_BASE_URL = "https://api.specstream.ai"
+USER_AGENT = "llama-index-tools-specstream/0.1.0"
+# A list runs for up to about five minutes, so the tools wait a little longer than that by default.
+DEFAULT_WAIT_SECONDS = 420.0
+FINISHED = {"completed", "partial"}
+STOPPED = {"failed", "cancelled", "canceled", "expired"}
+# Fields every row carries; a column with the same name is renamed so it can't overwrite them.
+RESERVED = {"list_id", "list_link", "position", "source_url", "checked_at"}
+
+Opener = Callable[[urllib.request.Request, float], Any]
+
+
+class SpecStreamError(RuntimeError):
+    """SpecStream refused a call or answered with something this package can't read."""
+
+
+def _dollars(cents: int) -> str:
+    return f"${cents / 100:.2f}"
+
+
+def list_link(find_id: str) -> str:
+    return f"https://specstream.ai/workspace/collections/{urllib.parse.quote(find_id, safe='')}"
+
+
+def flatten_row(find_id: str, record: dict[str, Any]) -> dict[str, Any]:
+    """One row as flat fields: each column's label holds what the list shows, plus where the row was read."""
+    row: dict[str, Any] = {"list_id": find_id, "position": record.get("ordinal")}
+    source: dict[str, Any] | None = None
+    for cell in record.get("details") or record.get("cells") or []:
+        label = str(cell.get("label") or cell.get("column_id") or "").strip()
+        if not label:
+            continue
+        if label in RESERVED:
+            label = f"{label} (column)"
+        if cell.get("absence"):
+            value: Any = ""
+        elif cell.get("display") is not None:
+            value = cell["display"]
+        else:
+            value = cell.get("value") if cell.get("value") is not None else ""
+        row[label] = value
+        provenance = cell.get("provenance") or {}
+        if source is None and str(provenance.get("source_url") or "").startswith("http"):
+            source = provenance
+    row["source_url"] = source.get("source_url", "") if source else ""
+    row["checked_at"] = source.get("observed_at", "") if source else ""
+    row["list_link"] = list_link(find_id)
+    return row
+
+
+@dataclass
+class SpecStreamAPI:
+    api_key: str
+    base_url: str = DEFAULT_BASE_URL
+    timeout: float = 30.0
+    opener: Opener | None = None
+    sleep: Callable[[float], None] = time.sleep
+    clock: Callable[[], float] = time.monotonic
+
+    def request(self, method: str, path: str, body: dict[str, Any] | None = None, *, allow_404: bool = False) -> dict[str, Any] | None:
+        key = (self.api_key or "").strip()
+        if not key:
+            raise SpecStreamError("Set SPECSTREAM_API_KEY or pass api_key. Create a key under Agents & API in your SpecStream workspace.")
+        request = urllib.request.Request(
+            self.base_url.rstrip("/") + path,
+            data=None if body is None else json.dumps(body).encode(),
+            method=method,
+            headers={
+                "Authorization": key if key.lower().startswith("bearer ") else f"Bearer {key}",
+                "Accept": "application/json",
+                "Content-Type": "application/json",
+                "User-Agent": USER_AGENT,
+            },
+        )
+        open_ = self.opener or (lambda req, timeout: urllib.request.urlopen(req, timeout=timeout))
+        try:
+            response = open_(request, self.timeout)
+            try:
+                raw = response.read()
+            finally:
+                response.close()
+        except urllib.error.HTTPError as error:
+            if allow_404 and error.code == 404:
+                return None
+            raise SpecStreamError(_error_message(error)) from None
+        except urllib.error.URLError as error:
+            raise SpecStreamError(f"Couldn't reach SpecStream: {error.reason}") from None
+        try:
+            payload = json.loads(raw or b"{}")
+        except ValueError:
+            payload = None
+        if not isinstance(payload, dict) or not isinstance(payload.get("data"), dict):
+            raise SpecStreamError("SpecStream returned a response this package couldn't read.")
+        return payload["data"]
+
+    def results(self, find_id: str) -> dict[str, Any]:
+        """Every row found so far for a list, following the pages."""
+        rows: list[dict[str, Any]] = []
+        cursor: str | None = None
+        status = "running"
+        while True:
+            query = {"limit": "100", **({"cursor": cursor} if cursor else {})}
+            page = self.request("GET", f"/v1/finds/{urllib.parse.quote(find_id, safe='')}/rows?{urllib.parse.urlencode(query)}") or {}
+            status = page.get("status") or status
+            rows.extend(flatten_row(find_id, record) for record in page.get("rows") or [])
+            cursor = page.get("next_cursor")
+            if not cursor:
+                break
+        return {"list_id": find_id, "status": status, "list_link": list_link(find_id), "row_count": len(rows), "rows": rows}
+
+    def find_list(
+        self,
+        request: str,
+        *,
+        max_results: int,
+        max_price_usd: float,
+        wait_seconds: float = DEFAULT_WAIT_SECONDS,
+        poll_interval: float = 3.0,
+        locale: str = "en-US",
+        key_prefix: str = "llamaindex",
+    ) -> dict[str, Any]:
+        q = (request or "").strip()
+        if not q:
+            raise ValueError("Describe the list to find.")
+        if not isinstance(max_results, int) or max_results < 1:
+            raise ValueError("max_results must be 1 or more.")
+        max_cents = round(float(max_price_usd) * 100)
+        if max_cents <= 0:
+            raise ValueError("max_price_usd must be greater than 0.")
+
+        # The same request with the same limits gets the run it already started in this hour or the one before,
+        # so a retried agent step never buys the list twice, even across the turn of the hour.
+        def key_for(at: float) -> str:
+            hour = time.strftime("%Y-%m-%dT%H", time.gmtime(at))
+            return f"{key_prefix}-" + hashlib.sha256(f"{q}|{max_results}|{max_cents}|{hour}".encode()).hexdigest()[:32]
+
+        now = time.time()
+        request_key = key_for(now)
+        existing = None
+        for candidate in (request_key, key_for(now - 3600)):
+            existing = self.request("GET", f"/v1/capacity/requests/{urllib.parse.quote(candidate, safe='')}", allow_404=True)
+            if existing and existing.get("find_id"):
+                break
+        if existing and existing.get("find_id"):
+            find_id = existing["find_id"]
+            maximum_cents = int(existing.get("maximum_cents") or max_cents)
+        else:
+            reading = self.request("POST", "/v1/parse", {"q": q, "locale": locale}) or {}
+            if reading.get("status") != "runnable":
+                read_as = reading.get("canonical_sentence") or q
+                return {
+                    "status": "needs_edit",
+                    "request": q,
+                    "message": f'SpecStream needs a clearer request before it can price it. It read: "{read_as}". Add the place or what every row must meet.',
+                }
+            quote = self.request(
+                "POST", "/v1/capacity/quotes", {"ast_hash": reading["ast_hash"], "operation": "find", "requested_units": max_results}
+            ) or {}
+            maximum_cents = int(quote.get("maximum_cents") or 0)
+            if maximum_cents > max_cents:
+                return {
+                    "status": "over_maximum",
+                    "request": q,
+                    "maximum_price_usd": maximum_cents / 100,
+                    "message": f"Nothing ran: this list can cost up to {_dollars(maximum_cents)}, above the maximum of {_dollars(max_cents)}. Ask for fewer results or raise max_price_usd.",
+                }
+            run = self.request(
+                "POST",
+                "/v1/capacity/executions",
+                {"quote_id": quote["quote_id"], "quote_hash": quote["quote_hash"], "confirm_max_cents": maximum_cents, "idempotency_key": request_key},
+            ) or {}
+            find_id = run["find_id"]
+
+        deadline = self.clock() + max(0.0, wait_seconds)
+        while True:
+            result = self.results(find_id)
+            if result["status"] in FINISHED | STOPPED or self.clock() >= deadline:
+                break
+            self.sleep(poll_interval)
+        result.update({"request": q, "maximum_price_usd": maximum_cents / 100})
+        if result["status"] not in FINISHED | STOPPED:
+            result["message"] = "Still running. Call specstream_get_list_results with this list_id for the rest."
+        return result
+
+
+def _error_message(error: urllib.error.HTTPError) -> str:
+    try:
+        payload = json.loads(error.read() or b"{}")
+    except ValueError:
+        payload = {}
+    detail = payload.get("error") if isinstance(payload, dict) else None
+    message = detail.get("message") if isinstance(detail, dict) else detail if isinstance(detail, str) else None
+    return f"SpecStream answered {error.code}: {message or error.reason}"

llama_index_tools_specstream-0.1.0/llama_index/tools/specstream/base.py

@@ -0,0 +1,116 @@
+"""The SpecStream tool spec: any list from the live web for a LlamaIndex agent, within a price set in code."""
+
+# No `from __future__ import annotations`: LlamaIndex reads these signatures at runtime to build the tool schema.
+import json
+import os
+from typing import Annotated, Optional
+
+from llama_index.core.tools.tool_spec.base import BaseToolSpec
+from pydantic import Field
+
+from llama_index.tools.specstream._api import DEFAULT_BASE_URL, DEFAULT_WAIT_SECONDS, SpecStreamAPI
+
+MAX_RESULTS = 1000
+
+
+def _api(api_key: Optional[str], base_url: str) -> SpecStreamAPI:
+    return SpecStreamAPI(api_key=api_key or os.environ.get("SPECSTREAM_API_KEY", ""), base_url=base_url)
+
+
+class SpecStreamToolSpec(BaseToolSpec):
+    """Two tools that let an agent find a list on the live web and read its rows.
+
+    The price cap is set here, in code, and is not a tool argument, so the model can't raise it.
+
+    Example:
+        .. code-block:: python
+
+            from llama_index.core.agent.workflow import FunctionAgent
+            from llama_index.tools.specstream import SpecStreamToolSpec
+
+            tools = SpecStreamToolSpec(max_price_usd=5).to_tool_list()
+            agent = FunctionAgent(tools=tools, llm=llm)
+    """
+
+    spec_functions = ["specstream_find_list", "specstream_get_list_results"]
+
+    def __init__(
+        self,
+        *,
+        max_price_usd: float,
+        api_key: Optional[str] = None,
+        base_url: str = DEFAULT_BASE_URL,
+        wait_seconds: float = DEFAULT_WAIT_SECONDS,
+        poll_interval: float = 3.0,
+        locale: str = "en-US",
+    ) -> None:
+        """
+        Args:
+            max_price_usd: The most one list may cost, in US dollars. Above it, nothing runs and the tool returns the price.
+            api_key: A SpecStream API key. Defaults to the SPECSTREAM_API_KEY environment variable.
+            base_url: The SpecStream API address.
+            wait_seconds: How long find_list waits for the list before returning the rows found so far.
+            poll_interval: Seconds between checks while a list is running.
+            locale: The language and region the request is written in.
+        """
+        if not max_price_usd or float(max_price_usd) <= 0:
+            raise ValueError("max_price_usd must be greater than 0.")
+        self.max_price_usd = float(max_price_usd)
+        self.wait_seconds = wait_seconds
+        self.poll_interval = poll_interval
+        self.locale = locale
+        self._client = _api(api_key, base_url)
+        if not self._client.api_key.strip():
+            raise ValueError("Set SPECSTREAM_API_KEY or pass api_key. Create a key under Agents & API in your SpecStream workspace.")
+
+    def specstream_find_list(
+        self,
+        request: Annotated[
+            str,
+            Field(
+                description=(
+                    "The list in plain words: what to find, where, what every row must meet, and the facts to bring back. "
+                    'Example: "Dental clinics in Lisbon with online booking, with their website and phone".'
+                )
+            ),
+        ],
+        max_results: Annotated[
+            int,
+            Field(description="How many rows to find, from 1 to 1000. Ask for what the task needs; the price grows with the matches."),
+        ] = 25,
+    ) -> str:
+        """Find a list of real things on the live web and return every matching row, each with the page it was read from.
+
+        Use it when the answer is many things that meet conditions: companies in a city, local businesses with a service,
+        funding filings, open grants, tenders, clinical trials, repositories, people in a role. For one fact or a summary
+        of one page, use a search tool instead. Rows that couldn't be confirmed are left out and aren't charged.
+
+        Returns JSON with status, list_id, list_link and rows (one object per match: its columns, source_url and
+        checked_at). status "over_maximum" means nothing ran because the price was above the cap: ask for fewer rows or
+        tell the user. status "needs_edit" means the request was too vague to price: add the place or the conditions.
+        If the list is still running, call specstream_get_list_results with its list_id for the rest.
+        """
+        count = max(1, min(int(max_results), MAX_RESULTS))
+        result = self._client.find_list(
+            request,
+            max_results=count,
+            max_price_usd=self.max_price_usd,
+            wait_seconds=self.wait_seconds,
+            poll_interval=self.poll_interval,
+            locale=self.locale,
+            key_prefix="llamaindex",
+        )
+        return json.dumps(result, ensure_ascii=False)
+
+    def specstream_get_list_results(
+        self,
+        list_id: Annotated[
+            str,
+            Field(description='The list_id that specstream_find_list returned, like "fnd_gk5y3d0r97bvyz0t08zzh75kr6".'),
+        ],
+    ) -> str:
+        """Read every row found so far for a SpecStream list, such as one that was still running. Reading is free.
+
+        Returns JSON with status, list_id, list_link and rows, each with its columns, source_url and checked_at.
+        """
+        return json.dumps(self._client.results(list_id.strip()), ensure_ascii=False)

llama_index_tools_specstream-0.1.0/llama_index/tools/specstream/retriever.py

@@ -0,0 +1,79 @@
+"""A retriever whose nodes are the rows of a SpecStream list."""
+
+from __future__ import annotations
+
+from typing import Any, List, Optional
+
+from llama_index.core.retrievers import BaseRetriever
+from llama_index.core.schema import NodeWithScore, QueryBundle, TextNode
+
+from llama_index.tools.specstream._api import DEFAULT_BASE_URL, DEFAULT_WAIT_SECONDS, SpecStreamError
+from llama_index.tools.specstream.base import MAX_RESULTS, _api
+
+# Row fields that say where the row came from rather than what it says.
+_META = ("list_id", "list_link", "position", "source_url", "checked_at")
+
+
+class SpecStreamRetriever(BaseRetriever):
+    """Turn a plain-English request into nodes, one per matching row, with its source in the metadata.
+
+    Example:
+        .. code-block:: python
+
+            from llama_index.tools.specstream import SpecStreamRetriever
+
+            retriever = SpecStreamRetriever(max_price_usd=3, max_results=25)
+            nodes = retriever.retrieve("Breweries in Portland, Oregon that host events, with their website")
+    """
+
+    def __init__(
+        self,
+        *,
+        max_price_usd: float,
+        max_results: int = 25,
+        api_key: Optional[str] = None,
+        base_url: str = DEFAULT_BASE_URL,
+        wait_seconds: float = DEFAULT_WAIT_SECONDS,
+        poll_interval: float = 3.0,
+        locale: str = "en-US",
+        **kwargs: Any,
+    ) -> None:
+        if not max_price_usd or float(max_price_usd) <= 0:
+            raise ValueError("max_price_usd must be greater than 0.")
+        if not 1 <= int(max_results) <= MAX_RESULTS:
+            raise ValueError(f"max_results must be from 1 to {MAX_RESULTS}.")
+        self.max_price_usd = float(max_price_usd)
+        self.max_results = int(max_results)
+        self.wait_seconds = wait_seconds
+        self.poll_interval = poll_interval
+        self.locale = locale
+        self._client = _api(api_key, base_url)
+        if not self._client.api_key.strip():
+            raise ValueError("Set SPECSTREAM_API_KEY or pass api_key. Create a key under Agents & API in your SpecStream workspace.")
+        super().__init__(**kwargs)
+
+    def _retrieve(self, query_bundle: QueryBundle) -> List[NodeWithScore]:
+        result = self._client.find_list(
+            query_bundle.query_str,
+            max_results=self.max_results,
+            max_price_usd=self.max_price_usd,
+            wait_seconds=self.wait_seconds,
+            poll_interval=self.poll_interval,
+            locale=self.locale,
+            key_prefix="llamaindex-retriever",
+        )
+        if result["status"] in ("needs_edit", "over_maximum"):
+            raise SpecStreamError(result["message"])
+        nodes = []
+        for row in result["rows"]:
+            facts = "\n".join(f"{label}: {value}" for label, value in row.items() if label not in _META and value not in ("", None))
+            node = TextNode(
+                id_=f"{row['list_id']}:{row['position']}",
+                text=facts,
+                metadata={key: row.get(key) for key in _META} | {"request": result.get("request")},
+                # The model sees each row's facts and the page behind them; the bookkeeping stays out of its prompt.
+                excluded_llm_metadata_keys=["list_id", "list_link", "position", "request"],
+                excluded_embed_metadata_keys=list(_META) + ["request"],
+            )
+            nodes.append(NodeWithScore(node=node, score=None))
+        return nodes

llama_index_tools_specstream-0.1.0/pyproject.toml

@@ -0,0 +1,49 @@
+[project]
+name = "llama-index-tools-specstream"
+version = "0.1.0"
+description = "Give a LlamaIndex agent any list from the live web: ask in plain words, get every matching row with the page it came from, within a price you set in code."
+readme = "README.md"
+requires-python = ">=3.10,<4.0"
+license = "MIT"
+license-files = ["LICENSE"]
+authors = [{ name = "SpecStream", email = "support@specstream.ai" }]
+maintainers = [{ name = "SpecStream", email = "support@specstream.ai" }]
+keywords = ["llama-index", "llamaindex", "specstream", "agents", "tools", "retriever", "web-data", "live-web", "lead-lists", "research"]
+classifiers = [
+  "Development Status :: 4 - Beta",
+  "Intended Audience :: Developers",
+  "Operating System :: OS Independent",
+  "Programming Language :: Python :: 3",
+  "Programming Language :: Python :: 3 :: Only",
+  "Programming Language :: Python :: 3.10",
+  "Programming Language :: Python :: 3.11",
+  "Programming Language :: Python :: 3.12",
+  "Programming Language :: Python :: 3.13",
+  "Topic :: Scientific/Engineering :: Artificial Intelligence",
+  "Topic :: Internet :: WWW/HTTP",
+  "Typing :: Typed",
+]
+dependencies = ["llama-index-core>=0.13.0,<0.15"]
+
+[project.optional-dependencies]
+test = ["pytest>=8"]
+
+[project.urls]
+Homepage = "https://specstream.ai"
+Documentation = "https://specstream.ai/docs/llamaindex"
+Repository = "https://github.com/Specstreamai/specstream-tools/tree/main/packages/llama-index-tools-specstream"
+Issues = "https://github.com/Specstreamai/specstream-tools/issues"
+Pricing = "https://specstream.ai/pricing"
+
+[build-system]
+requires = ["hatchling>=1.27,<2"]
+build-backend = "hatchling.build"
+
+[tool.hatch.build.targets.wheel]
+packages = ["llama_index"]
+
+[tool.hatch.build.targets.sdist]
+include = ["llama_index", "tests", "README.md", "LICENSE"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]

llama_index_tools_specstream-0.1.0/tests/test_specstream.py

@@ -0,0 +1,214 @@
+import io
+import json
+import urllib.error
+import urllib.parse
+
+import pytest
+
+from llama_index.tools.specstream import SpecStreamError, SpecStreamRetriever, SpecStreamToolSpec
+
+KEY = "ss_live_test_key"
+
+
+def cell(label, value, source=None):
+    provenance = {"source_url": source, "observed_at": "2026-10-14T08:00:00Z"} if source else None
+    return {"label": label, "value": value, "display": value, "absence": None, "provenance": provenance}
+
+
+ROWS = [
+    {"ordinal": 1, "cells": [cell("Organization", "Clínica Sorriso", "https://sorriso.pt/marcacoes"), cell("Website", "https://sorriso.pt")]},
+    {"ordinal": 2, "cells": [cell("Organization", "Dentes de Lisboa", "https://dentesdelisboa.pt"), cell("Website", None) | {"absence": "Not published"}]},
+]
+
+
+class FakeSpecStream:
+    """Answers like the SpecStream API: the request, its quote, the run, then rows over two polls and two pages."""
+
+    def __init__(self, maximum_cents=250, runnable=True, existing=None):
+        self.maximum_cents = maximum_cents
+        self.runnable = runnable
+        self.existing = existing
+        self.calls = []
+        self.polls = 0
+        self.misses = 0
+
+    def __call__(self, request, timeout=None):
+        url = urllib.parse.urlsplit(request.full_url)
+        body = json.loads(request.data) if request.data else None
+        self.calls.append((request.get_method(), url.path, body, dict(request.header_items())))
+        query = dict(urllib.parse.parse_qsl(url.query))
+        if url.path.startswith("/v1/capacity/requests/"):
+            if self.misses:
+                self.misses -= 1
+                raise urllib.error.HTTPError(request.full_url, 404, "Not Found", {}, io.BytesIO(b"{}"))
+            if self.existing:
+                return self.reply(self.existing)
+            raise urllib.error.HTTPError(request.full_url, 404, "Not Found", {}, io.BytesIO(b"{}"))
+        if url.path == "/v1/parse":
+            return self.reply({"status": "runnable" if self.runnable else "needs_input", "ast_hash": "a" * 64, "canonical_sentence": "25 dental clinics in Lisbon"})
+        if url.path == "/v1/capacity/quotes":
+            return self.reply({"quote_id": "cqt_1", "quote_hash": "b" * 64, "maximum_cents": self.maximum_cents})
+        if url.path == "/v1/capacity/executions":
+            return self.reply({"find_id": "fnd_1", "state": "running"})
+        if url.path == "/v1/finds/fnd_1/rows":
+            if query.get("cursor") == "page-2":
+                return self.reply({"status": "completed", "rows": ROWS[1:], "next_cursor": None})
+            self.polls += 1
+            if self.polls == 1:
+                return self.reply({"status": "running", "rows": [], "next_cursor": None})
+            return self.reply({"status": "completed", "rows": ROWS[:1], "next_cursor": "page-2"})
+        raise AssertionError(f"unexpected call {request.get_method()} {url.path}")
+
+    @staticmethod
+    def reply(data):
+        return io.BytesIO(json.dumps({"data": data}).encode())
+
+    def paths(self):
+        return [path for _, path, _, _ in self.calls]
+
+
+@pytest.fixture
+def api(monkeypatch):
+    def install(**kwargs):
+        fake = FakeSpecStream(**kwargs)
+        monkeypatch.setattr("urllib.request.urlopen", fake)
+        return fake
+
+    return install
+
+
+def tools(**kwargs):
+    spec = SpecStreamToolSpec(api_key=KEY, max_price_usd=kwargs.pop("max_price_usd", 5), poll_interval=0, **kwargs)
+    return {tool.metadata.name: tool for tool in spec.to_tool_list()}
+
+
+def test_finds_a_list_within_the_maximum_and_returns_every_row(api):
+    fake = api()
+    output = tools()["specstream_find_list"](request="Dental clinics in Lisbon with online booking", max_results=25)
+    result = json.loads(output.content)
+
+    assert result["status"] == "completed"
+    assert result["row_count"] == 2
+    assert result["list_link"] == "https://specstream.ai/workspace/collections/fnd_1"
+    first, second = result["rows"]
+    assert first["Organization"] == "Clínica Sorriso"
+    assert first["source_url"] == "https://sorriso.pt/marcacoes"
+    assert second["Website"] == ""
+    run = next(body for method, path, body, _ in fake.calls if path == "/v1/capacity/executions")
+    assert run["confirm_max_cents"] == 250
+    assert run["idempotency_key"].startswith("llamaindex-")
+
+
+def test_nothing_runs_when_the_price_is_above_the_maximum(api):
+    fake = api(maximum_cents=900)
+    result = json.loads(tools()["specstream_find_list"](request="Dental clinics in Lisbon").content)
+
+    assert result["status"] == "over_maximum"
+    assert "$9.00" in result["message"] and "$5.00" in result["message"]
+    assert "/v1/capacity/executions" not in fake.paths()
+
+
+def test_an_unclear_request_comes_back_with_how_it_was_read(api):
+    fake = api(runnable=False)
+    result = json.loads(tools()["specstream_find_list"](request="clinics").content)
+
+    assert result["status"] == "needs_edit"
+    assert "25 dental clinics in Lisbon" in result["message"]
+    assert "/v1/capacity/quotes" not in fake.paths()
+
+
+def test_the_model_sees_two_tools_and_cannot_set_the_price():
+    found = tools()
+    assert set(found) == {"specstream_find_list", "specstream_get_list_results"}
+    schema = found["specstream_find_list"].metadata.get_parameters_dict()
+    assert set(schema["properties"]) == {"request", "max_results"}
+    assert schema["required"] == ["request"]
+    assert "live web" in found["specstream_find_list"].metadata.description
+
+
+def test_max_results_is_held_to_what_one_list_can_hold(api):
+    fake = api()
+    tools()["specstream_find_list"](request="Dental clinics in Lisbon", max_results=50000)
+    quote = next(body for method, path, body, _ in fake.calls if path == "/v1/capacity/quotes")
+    assert quote["requested_units"] == 1000
+
+
+def test_a_retried_step_reuses_the_run_it_already_started(api):
+    fake = api(existing={"find_id": "fnd_1", "maximum_cents": 250})
+    result = json.loads(tools()["specstream_find_list"](request="Dental clinics in Lisbon").content)
+
+    assert result["row_count"] == 2
+    assert "/v1/parse" not in fake.paths() and "/v1/capacity/executions" not in fake.paths()
+
+
+def test_reading_results_follows_every_page(api):
+    fake = api()
+    fake.polls = 1
+    result = json.loads(tools()["specstream_get_list_results"](list_id="fnd_1").content)
+
+    assert [row["position"] for row in result["rows"]] == [1, 2]
+
+
+def test_the_key_goes_in_the_authorization_header_and_never_in_the_result(api):
+    fake = api()
+    output = tools()["specstream_find_list"](request="Dental clinics in Lisbon").content
+
+    headers = fake.calls[0][3]
+    assert headers["Authorization"] == f"Bearer {KEY}"
+    assert headers["User-agent"].startswith("llama-index-tools-specstream/")
+    assert KEY not in output
+
+
+def test_the_tools_work_from_an_async_agent(api):
+    import asyncio
+
+    api()
+    output = asyncio.run(tools()["specstream_find_list"].acall(request="Dental clinics in Lisbon"))
+    assert json.loads(output.content)["row_count"] == 2
+
+
+def test_the_retriever_returns_one_node_per_row_with_its_source(api):
+    api()
+    nodes = SpecStreamRetriever(api_key=KEY, max_price_usd=5, poll_interval=0).retrieve("Dental clinics in Lisbon")
+
+    assert len(nodes) == 2
+    first = nodes[0].node
+    assert first.text == "Organization: Clínica Sorriso\nWebsite: https://sorriso.pt"
+    assert first.metadata["source_url"] == "https://sorriso.pt/marcacoes"
+    assert first.node_id == "fnd_1:1"
+    assert "source_url" in first.get_content(metadata_mode="llm")
+    assert "list_link" not in first.get_content(metadata_mode="llm")
+    assert nodes[1].node.text == "Organization: Dentes de Lisboa"
+
+
+def test_the_retriever_raises_instead_of_spending_above_the_maximum(api):
+    api(maximum_cents=900)
+    with pytest.raises(SpecStreamError, match="above the maximum"):
+        SpecStreamRetriever(api_key=KEY, max_price_usd=5).retrieve("Dental clinics in Lisbon")
+
+
+def test_a_missing_key_says_where_to_get_one(monkeypatch):
+    monkeypatch.delenv("SPECSTREAM_API_KEY", raising=False)
+    with pytest.raises(ValueError, match="SPECSTREAM_API_KEY"):
+        SpecStreamToolSpec(max_price_usd=5)
+
+
+def test_the_key_can_come_from_the_environment(monkeypatch):
+    monkeypatch.setenv("SPECSTREAM_API_KEY", KEY)
+    assert SpecStreamToolSpec(max_price_usd=5)._client.api_key == KEY
+
+
+def test_a_price_cap_is_required():
+    with pytest.raises(ValueError, match="max_price_usd"):
+        SpecStreamToolSpec(api_key=KEY, max_price_usd=0)
+
+
+def test_a_retry_after_the_turn_of_the_hour_reuses_the_run_from_the_hour_before(api):
+    fake = api(existing={"find_id": "fnd_1", "maximum_cents": 250})
+    fake.misses = 1
+    result = json.loads(tools()["specstream_find_list"](request="Dental clinics in Lisbon").content)
+
+    lookups = [path for path in fake.paths() if path.startswith("/v1/capacity/requests/")]
+    assert len(lookups) == 2 and lookups[0] != lookups[1]
+    assert result["row_count"] == 2
+    assert "/v1/capacity/executions" not in fake.paths()