@tduyng/pi-web 0.0.3 → 0.0.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +32 -32
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -1,55 +1,48 @@
|
|
|
1
1
|
# @tduyng/pi-web
|
|
2
2
|
|
|
3
|
-
Web search and fetch
|
|
3
|
+
Web search and fetch with no API key, no setup, and no MCP tool schemas sitting in the context. Two tools, 326 tokens a request.
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
A model without web access answers from memory, and the reply does not say which parts are memory. Two ways out: paste pages by hand, or install a search MCP server whose tool schemas then ride in every request, including the ones that never search.
|
|
6
6
|
|
|
7
|
-
|
|
7
|
+
## Two tools, one hidden server
|
|
8
8
|
|
|
9
|
-
|
|
9
|
+
pi registers Exa's MCP server with exposure `codemode`, so its tools never enter the prompt. A codemode script calls them through `ctx.executeTool`, and a model that never writes such a script never pays for them.
|
|
10
10
|
|
|
11
|
-
|
|
11
|
+
```text
|
|
12
|
+
web_search({queries: ["typed decision model", "pi coding agent extensions"], numResults: 5})
|
|
13
|
+
web_fetch({urls: ["https://example.com/post"], maxCharacters: 12000})
|
|
14
|
+
```
|
|
12
15
|
|
|
13
|
-
`web_search` takes up to six queries
|
|
14
|
-
|
|
15
|
-
| Backend | When | What it returns |
|
|
16
|
-
| --- | --- | --- |
|
|
17
|
-
| Exa MCP | always, keyless | prose answer plus highlights |
|
|
18
|
-
| DuckDuckGo | Exa is unconnected, `--no-mcp`, or empty | result list |
|
|
19
|
-
| arXiv Atom API | `source: arxiv` | abstracts, authors, dates, newest first |
|
|
20
|
-
| Hacker News (Algolia) | `source: hn` | threads with points and comment count |
|
|
16
|
+
`web_search` takes up to six queries and groups the results under the query that produced them. Each group carries `title`, `url`, `snippet`, plus `published` and `authors` where the backend has them, and `points` and `comments` on Hacker News. The results are a declared output schema, so a codemode script gets them as data in `structuredContent.groups` and can compare values without parsing text.
|
|
21
17
|
|
|
22
18
|
`web_fetch` reads a URL as Markdown. `x.com` and `twitter.com` links go to Fixupx, which answers a page X refuses, and it returns the post together with the long-form Article it links to.
|
|
23
19
|
|
|
24
|
-
## Typed results
|
|
25
|
-
|
|
26
|
-
`web_search` declares an output schema, so a codemode script gets the answer as data in `structuredContent.groups`: `title`, `url`, `snippet`, plus `published` and `authors` where the backend has them, and `points` and `comments` on Hacker News. Compare values across results without parsing text. Exa's answer is prose, and `parseExa` reads it back into the same shape, so the typed view holds for every backend. The text still reaches the model and the data is not sent twice.
|
|
27
|
-
|
|
28
20
|
## Search is a step, not an answer
|
|
29
21
|
|
|
30
|
-
Results are excerpts, so the reply ends with the ranked URLs to read. Without that list the model answers from a snippet
|
|
22
|
+
Results are excerpts, so the reply ends with the ranked URLs to read. Without that list the model answers from a snippet, and a snippet is where a wrong answer sounds most confident.
|
|
31
23
|
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
24
|
+
| Backend | When | What it returns |
|
|
25
|
+
| --- | --- | --- |
|
|
26
|
+
| Exa MCP | always, keyless | prose answer plus highlights |
|
|
27
|
+
| DuckDuckGo | Exa unconnected, `--no-mcp`, or empty | result list |
|
|
28
|
+
| arXiv Atom API | `source: "arxiv"` | abstracts, authors, dates, newest first |
|
|
29
|
+
| Hacker News (Algolia) | `source: "hn"` | threads with points and comment count |
|
|
37
30
|
|
|
38
|
-
|
|
31
|
+
A miss on arXiv or Hacker News falls back to the open web, because arXiv indexes preprints and nothing else.
|
|
39
32
|
|
|
40
|
-
|
|
33
|
+
## Measurements
|
|
41
34
|
|
|
42
|
-
|
|
35
|
+
**DuckDuckGo queue.** DuckDuckGo answers a burst from one address with a bot check and keeps that address locked out for 10 to 15 minutes, with each retry extending it. Measured on 2026-10-26: six requests at once got three pages of results and three challenge pages at 14 KB apiece, and the challenge arrives as HTTP 202 as often as 403. So searches run one at a time, two seconds apart. A bot check pauses the provider for five minutes, doubling to an hour while it keeps answering that way, and a paused search fails at once instead of holding the tool open.
|
|
43
36
|
|
|
44
|
-
arXiv ANDs nothing on its own. Measured on the live API: `typed decision model` returns 1,470,125 rows because the words are read as alternatives, and `all:"typed decision model"` returns 3 because the phrase is too strict. A plain query therefore has its words ANDed one at a time, which is the only form that ranks usefully. A query already written with arXiv field prefixes (`ti:`, `cat:`, `au
|
|
37
|
+
**arXiv query form.** arXiv ANDs nothing on its own. Measured on the live API: `typed decision model` returns 1,470,125 rows because the words are read as alternatives, and `all:"typed decision model"` returns 3 because the phrase is too strict. A plain query therefore has its words ANDed one at a time, which is the only form that ranks usefully. A query already written with arXiv field prefixes (`ti:`, `cat:`, `au:`) passes through untouched.
|
|
45
38
|
|
|
46
|
-
|
|
39
|
+
**Links in a fetched page.** Exa's extraction returns plain text. Measured on 2026-09-27, `arxiv.org/abs/1706.03762` came back with no links at all, so a citation cannot be read out of the page that cites it. Jina Reader returns Markdown with links intact, 295 of them on a Hacker News thread, but it blocks a domain that hammered it. Exa answers first for that reason, and links survive only when Jina does. To follow a citation, search for the paper's title.
|
|
47
40
|
|
|
48
|
-
|
|
41
|
+
**Reddit and X.** Neither has a keyless search. Measured on 2026-09-27, `reddit.com/search.json` answers 403 on `www`, `old` and `api`, with a browser user agent and a plain one. DuckDuckGo's `site:` operator does reach both, so naming the forum in the query is the way in.
|
|
49
42
|
|
|
50
|
-
|
|
43
|
+
## Cost
|
|
51
44
|
|
|
52
|
-
|
|
45
|
+
`web_search` is 219 tokens a request and `web_fetch` is 107. The Exa tools never enter the prompt, so the cold start is 326 tokens on every session, with no key, account or handshake.
|
|
53
46
|
|
|
54
47
|
## Install
|
|
55
48
|
|
|
@@ -59,4 +52,11 @@ pi install npm:@tduyng/pi-web
|
|
|
59
52
|
|
|
60
53
|
Local checkout: `pi install /path/to/dotfiles/pi/agent/extensions/pi-web`
|
|
61
54
|
|
|
62
|
-
|
|
55
|
+
Optionally set `EXA_API_KEY` to use an own key. The extension reads it at load and puts it on the registered server URL as the `exaApiKey` parameter. Exa's free plan includes 1,000 requests a month, so a batch of six queries costs six of them.
|
|
56
|
+
|
|
57
|
+
## Limits
|
|
58
|
+
|
|
59
|
+
- The keyless Exa server takes a 60 second request timeout covering the connect handshake. Pi retries only HTTP 408, 429 and 5xx, so a handshake that overruns marks the server failed for the session and `/mcp` shows why.
|
|
60
|
+
- `numResults` is 1 to 10, default 5. `maxCharacters` defaults to 8,000.
|
|
61
|
+
- Fetch has a 20 second timeout, and at most six queries run per call.
|
|
62
|
+
- Exa's free tier, with an own key, is 1,000 requests a month.
|
package/package.json
CHANGED