@mrclrchtr/supi-web 4.3.0 → 4.5.0

package/README.md

@@ -6,101 +6,89 @@
   </a>
 </div>
 
-# @mrclrchtr/supi-web
+# @mrclrchtr/supi-web — Web Fetch and Context7 for Pi
 
-Adds web-page fetching and library-documentation lookup tools to the [pi coding agent](https://github.com/earendil-works/pi).
+Adds web fetch and Context7 documentation tools to the [Pi coding agent](https://github.com/earendil-works/pi), without making you paste sources into chat.
 
-## Install
-
-```bash
-pi install npm:@mrclrchtr/supi-web
-```
-
-For local development:
-
-```bash
-pi install ./packages/supi-web
-```
-
-![Web tools in action](https://raw.githubusercontent.com/mrclrchtr/supi/main/screenshots/supi-web.png)
-
-## What you get
+## What your agent gets
 
-After install, pi gets three tools.
+After installation, keep asking Pi normal questions. The agent can:
 
-In PI's TUI, these tools now render a collapsed summary by default; expand the tool row to reveal the full inline output.
+- **Read public web pages as clean Markdown** — extract the main article instead of returning navigation, scripts, styles, and raw HTML.
+- **Prefer source Markdown when available** — detect Markdown responses and common Markdown siblings before converting HTML.
+- **Handle plain-text and source files** — wrap them in fenced code blocks with a language hint when the URL provides one.
+- **Look up current library docs** — search Context7 for the right project and version, then retrieve documentation focused on the task at hand.
+- **Protect the context window** — return small results inline, move long pages to temporary files, and preserve full output when model-visible text must be truncated.
 
+Relative page links and images become absolute by default, so the agent can follow them without reconstructing URLs.
 
-| Tool | Purpose |
-|------|---------|
-| `web_fetch_md` | Fetch a web page and convert it to clean Markdown for LLM ingestion |
-| `web_docs_search` | Search Context7 for library IDs and metadata before fetching docs |
-| `web_docs_fetch` | Fetch up-to-date documentation for a known Context7 library |
+## Example requests
 
-## `web_fetch_md`
+You do not need to learn the tool-call syntax. Try asking Pi:
 
-Fetches a public URL and returns clean Markdown.
+- “Read this public design document and summarize the decisions: `https://example.com/spec`.”
+- “Check the current React documentation for effect cleanup.”
+- “Find the FastAPI docs for lifespan events and show the recommended pattern.”
+- “Fetch this long reference page to a file, then read only the relevant section.”
+- “Compare this library's current docs with the code in the repository.”
 
-### Parameters
+## Agent tools
 
-| Parameter | Type | Required | Default | Description |
-|-----------|------|----------|---------|-------------|
-| `url` | string | ✓ | — | `http://` or `https://` URL to fetch |
-| `output_mode` | `"auto"` \| `"inline"` \| `"file"` | — | `"auto"` | How to return the result |
-| `abs_links` | boolean | — | `true` | Absolutize relative links and images |
-| `timeout_ms` | number | — | `30000` | Fetch timeout in milliseconds |
+The package adds three tools that Pi selects as needed:
 
-### Output modes
+| Tool | What it lets the agent do |
+|---|---|
+| `web_fetch_md` | Fetch a public HTTP(S) page and return readable Markdown, fenced plain text, or a temporary file |
+| `web_docs_search` | Find matching Context7 library IDs, versions, trust scores, benchmark scores, and snippet counts |
+| `web_docs_fetch` | Retrieve focused documentation for a chosen Context7 library ID |
 
-- **`auto`** — returns Markdown inline if ≤15,000 characters; otherwise writes to a temporary file and returns the path
-- **`inline`** — returns Markdown inline, truncated to PI's model-visible output limit when necessary
-- **`file`** — always writes to a temporary file and returns the path
+Pi searches Context7 first when the library ID is unknown, then fetches docs with the selected ID. Narrow questions produce more useful documentation context than broad requests. Documentation returns as Markdown by default; the agent can request structured JSON snippets when needed.
 
-### Behavior
+## How page fetching works
 
-- Only accepts real `http://` or `https://` URLs
-- Access-controlled pages (login, paywall) should be skipped — ask the user for an allowed source instead
-- Plain-text responses are wrapped in fenced code blocks
-- Model-visible inline output is truncated to 2,000 lines or 50KB; full truncated output is saved to a temp file
-- Links and images are absolutized by default; set `abs_links: false` to keep them relative
+`web_fetch_md` chooses the cleanest representation it can find:
 
-## `web_docs_search`
+1. Use Markdown returned directly by the server.
+2. Detect Markdown or plain text from the response content.
+3. Check common `.md`, `.markdown`, `index.md`, and `README.md` alternatives.
+4. For HTML, extract the primary readable content and convert it to GitHub-flavored Markdown.
 
-Searches Context7 for library IDs before fetching documentation.
+It follows redirects, removes scripts and styles, preserves useful structures such as headings, links, lists, code blocks, and tables, and makes relative links absolute by default.
 
-### Parameters
+### Large pages
 
-| Parameter | Type | Required | Description |
-|-----------|------|----------|-------------|
-| `library_name` | string | ✓ | Library name to search for (e.g. `"react"`, `"next.js"`) |
-| `query` | string | ✓ | What you're trying to do — used for relevance ranking |
+The default `auto` mode returns content inline up to 15,000 characters and writes larger pages to a temporary Markdown file. You can ask Pi to force inline or file output.
 
-Results return as a compact Markdown table with library ID, name, description, trust score, benchmark score, snippet count, and shortened version lists (top 10 matches shown).
+All three tools cap model-visible inline output at Pi's 2,000-line / 50KB limit. If output is truncated, the complete result is saved to a temporary file for follow-up reads.
 
-## `web_docs_fetch`
+Tool rows stay collapsed in Pi's TUI by default. Press `Ctrl+O` to expand their inline output.
 
-Retrieves documentation context for a known Context7 library.
+## Install
 
-### Parameters
+```bash
+pi install npm:@mrclrchtr/supi-web
+```
 
-| Parameter | Type | Required | Default | Description |
-|-----------|------|----------|---------|-------------|
-| `library_id` | string | ✓ | — | Context7 library ID (e.g. `/facebook/react`, `/vercel/next.js`) |
-| `query` | string | ✓ | — | Specific question about the library |
-| `raw` | boolean | — | `false` | When `true`, returns JSON-serialized snippet objects instead of Markdown |
+To try it for one run without installing:
 
-Default mode returns pre-formatted Markdown. Set `raw: true` when you need structured JSON for programmatic use. Large model-visible responses are truncated to 2,000 lines or 50KB with the full response saved to a temp file.
+```bash
+pi -e npm:@mrclrchtr/supi-web
+```
 
-## Typical workflow
+## Context7 API key
 
-1. **`web_docs_search`** — find the right library ID
-2. **Pick a `library_id`** from the results
-3. **`web_docs_fetch`** — retrieve focused, version-aware docs
+The documentation tools call the [Context7 API](https://context7.com/). Create a free key at <https://context7.com/dashboard>, export it before starting Pi, and keep it out of repository files:
 
-Skip step 1 if you already know the exact Context7 `library_id`.
+```bash
+export CONTEXT7_API_KEY="ctx7sk-..."
+pi
+```
 
-## Context7 API key
+`web_fetch_md` does not use Context7 and needs no API key.
 
-`web_docs_search` and `web_docs_fetch` call the [Context7 REST API](https://context7.com/) directly.
+## Boundaries
 
-Set `CONTEXT7_API_KEY` in your environment to authenticate with the Context7 API. Without a key, the tools will return an authentication error when called. Get a free API key at https://context7.com/dashboard.
+- `web_fetch_md` is for public `http://` and `https://` sources. It does not authenticate to login-protected, private, or paywalled pages.
+- It fetches HTTP responses and does not run a browser, page JavaScript, or interactive flows; client-rendered pages may expose little useful content.
+- When the `gh` CLI is available, Pi is guided to use it instead for GitHub URLs.
+- Fetched pages are external, untrusted content. Treat them as source material, not as repository instructions.

package/node_modules/@mrclrchtr/supi-core/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@mrclrchtr/supi-core",
-  "version": "4.3.0",
+  "version": "4.5.0",
   "description": "SuPi core — shared infrastructure for SuPi extensions (XML context tags, config system)",
   "license": "MIT",
   "repository": {

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@mrclrchtr/supi-web",
-  "version": "4.3.0",
-  "description": "SuPi Web extension — fetch web pages as clean Markdown (web_fetch_md) and library docs via Context7 (web_docs_search, web_docs_fetch)",
+  "version": "4.5.0",
+  "description": "Web fetch and Context7 documentation extension for the Pi coding agent",
   "license": "MIT",
   "repository": {
     "type": "git",
@@ -32,11 +32,11 @@
     "README.md"
   ],
   "dependencies": {
-    "jsdom": "^29.0.0",
+    "jsdom": "^30.0.0",
     "@mozilla/readability": "^0.6.0",
     "turndown": "^7.2.0",
     "turndown-plugin-gfm": "^1.0.2",
-    "@mrclrchtr/supi-core": "4.3.0"
+    "@mrclrchtr/supi-core": "4.5.0"
   },
   "bundledDependencies": [
     "@mrclrchtr/supi-core"