@sasasin/fetch-url 0.0.0-stage → 0.1.1

package/CHANGELOG.md

@@ -0,0 +1,26 @@
+# Changelog
+
+## 0.1.1
+
+- `tests/` is no longer shipped in the npm tarball. Node refuses to strip TypeScript types for
+  files under `node_modules`, so `node tests/selftest.mjs` could never work from an `npm:`-installed
+  copy (`ERR_UNSUPPORTED_NODE_MODULES_TYPE_STRIPPING`), and `tests/test-load.mjs` needs jiti, which
+  lives in `devDependencies`. The scripts stay in the repository and run from a git clone
+  (`~/.pi/agent/git/github.com/sasasin/fetch_url`) or the source tree.
+- README: the self-test section now says where the scripts can run, and notes that the extension
+  itself is unaffected because pi loads it through jiti.
+
+## 0.1.0
+
+Initial public release.
+
+- `fetch_url` tool: fetch up to 16 URLs in one call through a single Chromium
+  using one isolated `BrowserContext` per URL (default 4 concurrent workers, max 8).
+- Output modes: `aria` (accessibility snapshot), `text` (rendered innerText), `html` (page source).
+- Per-page headers with HTTP status, redirect target, title and character count.
+- Retries transient navigation failures once in a fresh context; permanent failures
+  (DNS, invalid URL, certificate errors) stop after a single attempt.
+- Browser pick order `chrome` → `msedge` → embedded `chromium`, overridable with
+  `PI_FETCH_URL_CHANNEL`.
+- Idle browser reuse with a 3-minute auto-close, closed on `session_shutdown`.
+- Requires Node 22.19+ (the same floor as `@earendil-works/pi-coding-agent`); Node 20 is end-of-life since 2026-04-30 and is not tested.

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Shinnosuke Suzuki
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -1,3 +1,217 @@
-# Temporary Holding Version
+# @sasasin/fetch-url
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+A [pi](https://github.com/earendil-works/pi) coding-agent extension that adds a `fetch_url` tool:
+it fetches many web pages **at the same time** by driving Playwright directly —
+one Chromium process, one isolated `BrowserContext` per URL.
+No MCP, no API billing, no external service.
+
+## Install
+
+Requires Node **22.19+** (the same floor as pi itself) and a browser: system Chrome/Edge, or
+Playwright's Chromium — see [Browser requirement](#browser-requirement).
+
+```bash
+# from npm
+pi install npm:@sasasin/fetch-url
+
+# or straight from this repository (updated by `pi update --extensions`)
+pi install git:github.com/sasasin/fetch_url
+```
+
+Both sources install the extension's `dependencies` (`playwright-core`) automatically.
+Restart pi, or run `/reload` in a running session.
+
+> If you previously dropped this folder into `~/.pi/agent/extensions/fetch-url`,
+> remove that copy first — loading both registers the tool name `fetch_url` twice.
+> The same applies to installing the npm and the git source side by side.
+
+### Where it is installed
+
+pi puts the package (and its `node_modules`) under its agent directory, or under `.pi/` for
+project scope:
+
+| Install command | Extension folder (the place to `cd` into for maintenance) |
+|---|---|
+| `pi install npm:@sasasin/fetch-url` | `~/.pi/agent/npm/node_modules/@sasasin/fetch-url` |
+| `pi install -l npm:@sasasin/fetch-url` (project) | `<project>/.pi/npm/node_modules/@sasasin/fetch-url` |
+| `pi install git:github.com/sasasin/fetch_url` | `~/.pi/agent/git/github.com/sasasin/fetch_url` |
+| `pi install -l git:github.com/sasasin/fetch_url` (project) | `<project>/.pi/git/github.com/sasasin/fetch_url` |
+| `pi -e npm:…` / `pi -e git:…` (one run, not persisted) | `~/.pi/agent/tmp/extensions/npm-<hash>/node_modules/@sasasin/fetch-url` / `~/.pi/agent/tmp/extensions/git-github.com-<hash>` |
+| manual local extension folder | `~/.pi/agent/extensions/fetch-url` |
+
+`-e` installs are throwaway and get a fresh hash per run.
+
+### Browser requirement
+
+`playwright-core` ships **without** browser binaries. The extension launches, in order:
+
+1. system Google Chrome (`channel: "chrome"`)
+2. Microsoft Edge (`channel: "msedge"`)
+3. Playwright's bundled Chromium (`channel: "chromium"`) — to add it, run the installer
+   **from the package folder**, so the build matches this extension's `playwright-core` version:
+
+   ```bash
+   cd ~/.pi/agent/git/github.com/sasasin/fetch_url      # npm source: ~/.pi/agent/npm/node_modules/@sasasin/fetch-url
+   npx --no-install playwright-core install chromium    # → ~/.cache/ms-playwright/chromium-<build>
+   ```
+
+   A bare `npx playwright install chromium` resolves `playwright` from the registry instead and
+   may install a different Chromium build that this extension will not use.
+
+If none of the three is available you get `browser launch failed: could not launch a browser (tried …)`.
+The download directory honours `PLAYWRIGHT_BROWSERS_PATH`; it is shared with other Playwright users
+on the machine, not inside the extension folder.
+
+## Usage
+
+```ts
+fetch_url({
+  urls: ["https://a.example", "https://b.example", "https://c.example"],
+  mode: "text",              // default aria
+  concurrency: 6,            // 1-8, default 4
+  max_chars_per_page: 12000, // default 8000
+  timeout_ms: 45000          // default 30000
+})
+```
+
+Up to **16 URLs per call**; extras are reported as skipped. Results come back in input order:
+
+```
+--- https://example.com/ ---
+[HTTP 200 | title: Example Domain | 170 chars]
+# Example Domain
+...
+```
+
+The header carries the HTTP status, the final URL when a redirect happened, the page title and
+the character count.
+
+## Behaviour
+
+- **Permanent failures stop after one attempt** (`ERR_NAME_NOT_RESOLVED`, invalid URL,
+  certificate errors, …). Transient ones retry once in a brand-new context (2 attempts by default).
+- **Aborting a tool call** stops every worker within ~1.5 s; untouched URLs are reported as
+  `cancelled (aborted)`.
+- The Chromium instance is **reused** and closed after 3 idle minutes, on `session_shutdown`,
+  and after a crash it is relaunched on the next call.
+- SIGINT/SIGTERM/SIGHUP handlers are not installed by Playwright, so Ctrl-C in pi does not
+  take the browser down with it.
+
+## Configuration (environment variables)
+
+| Variable | Effect |
+|---|---|
+| `PI_FETCH_URL_HEADLESS=0` | run headed (default: headless) |
+| `PI_FETCH_URL_CHANNEL=chrome\|msedge\|chromium` | force one channel; `""` = embedded Chromium only |
+| `PI_FETCH_URL_NO_SANDBOX=1` | `chromiumSandbox: false` (some containers / distros) |
+
+## Keeping `playwright-core` current
+
+The published range is `^1.64.1` and `package-lock.json` records the resolved version. There is no
+in-extension self-update command:
+
+- **Renovate** opens PRs for the dependency range, the lockfile (weekly `lockFileMaintenance`) and
+  the Actions in `.github/workflows/`; **tagpr** turns merged work into a tagged release.
+- Installed with `pi install npm:` or `git:` → **`pi update --extensions`** reinstalls this package,
+  so `playwright-core` is fetched again. Then restart pi or run `/reload` so the extension module is
+  re-imported. For a git source pi also runs `git clean -fdx` before reinstalling, which is exactly
+  why an in-place `npm install` from inside the extension would not survive the next update.
+- If you keep an unpacked development copy at `~/.pi/agent/extensions/fetch-url`, pi never touches
+  its dependencies (`docs/packages.md`: local packages are not installed or modified). Update that
+  copy yourself:
+
+  ```bash
+  npm install --omit=dev --prefix ~/.pi/agent/extensions/fetch-url
+  ```
+
+## Self-test without pi
+
+Run these from a git clone (the git source lives at `~/.pi/agent/git/github.com/sasasin/fetch_url`)
+or from the source tree — **not** from the `npm:` installed copy: Node refuses to strip types for
+files under `node_modules`, so `selftest.mjs` cannot import `fetch-core.ts` there. The extension
+itself is unaffected, because pi loads it through jiti. Hence `tests/` is kept out of the npm
+tarball.
+
+Needs Node 22.19+ (the same floor as pi itself — `selftest.mjs` relies on native type stripping):
+
+```bash
+npm install --omit=dev
+node tests/selftest.mjs --concurrency 4 --mode text https://news.ycombinator.com/ https://quotes.toscrape.com/js/
+node tests/selftest.mjs --abort-after 1500 --concurrency 2 https://httpbin.org/delay/8 https://example.com/
+```
+
+To check tool registration without starting a browser (uses jiti from `devDependencies`):
+
+```bash
+npm install
+npm run test:load
+```
+
+## Releases and automated updates (tagpr + Renovate + Actions lockfile)
+
+This repository keeps its dependencies fresh with three mechanisms:
+
+- **tagpr** (`.github/workflows/tagpr.yml` + `.tagpr`) watches `main`, keeps one open release pull
+  request with the unreleased changes, and tags it (plus creates a GitHub Release) when you merge.
+  `tagpr.versionFile = package.json`, so the git tag and the npm version stay in sync; the tag is
+  what you publish from. `tagpr.changelog = false` because `CHANGELOG.md` is written by hand.
+- **Renovate** (`renovate.json`) updates `dependencies` / `devDependencies` in `package.json`,
+  runs `lockFileMaintenance` weekly for `package-lock.json`, and bumps the Actions in
+  `.github/workflows/`. Renovate itself has to be installed as a GitHub App on the repository
+  (`github.com/apps/renovate`); `renovate.json` is only its configuration, and
+  `.github/workflows/renovate-config.yml` validates that file.
+- **GitHub Actions workflow lockfile** (`.github/workflows/actions.lock`, GA since 2026-10-28).
+  Workflows keep readable references such as `actions/checkout@v7.0.1`, while the lockfile records
+  the commit that actually runs — including transitive action dependencies.
+  It is generated by the `gh actions-lock` CLI extension and must **never** be edited by hand:
+
+  ```bash
+  gh extension install github/gh-actions-lock --pin v0.4.1
+  gh actions-lock                    # onboard / refresh after changing any uses:
+  gh actions-lock --verify           # the same check CI runs
+  ```
+
+  When Renovate bumps an action it regenerates the matching lockfile entry in the same PR, so the
+  workflow file and `actions.lock` cannot drift apart (the extension version is fixed with
+  `constraints.ghActionsLock`). `.github/workflows/actions-lockfile.yml` fails CI if the lockfile
+  is stale or a pin no longer verifies upstream.
+
+## Development
+
+```bash
+npm install      # devDependencies (typescript, pi types, jiti) are not published
+npm run check    # typecheck + jiti load test
+npm run selftest # optional live fetch
+```
+
+CI (`.github/workflows/ci.yml`) runs typecheck, the jiti load test and the live self-test on
+Node 22 and 24.
+
+Layout: `index.ts` is the thin pi tool-registration layer, `fetch-core.ts` holds the Playwright
+lifecycle and the worker pool, and `tests/` carries the two test scripts (`selftest.mjs` for a
+live parallel fetch, `test-load.mjs` for loading the extension through jiti the way pi does).
+
+## 日本語
+
+pi に `fetch_url` ツールを追加する拡張です。
+専用の Chromium を起動して **URL ごとに独立した BrowserContext** を切るため、最大 16 URL を
+既定 4（最大 8）ワーカーで同時並列に取得できます。課金なし・外部サービスなし。
+
+インストールは `pi install npm:@sasasin/fetch-url`、またはこのリポジトリを直接
+`pi install git:github.com/sasasin/fetch_url`（`pi update --extensions` で追随）。
+設置先は `npm:` なら `~/.pi/agent/npm/node_modules/@sasasin/fetch-url`、
+`git:` なら `~/.pi/agent/git/github.com/sasasin/fetch_url`（上に一覧表あり）。
+`playwright-core` にはブラウザバイナリが含まれないため、システムの Chrome / Edge も無い環境では、
+パッケージフォルダで `npx --no-install playwright-core install chromium` を実行して
+`~/.cache/ms-playwright/` にブラウザを追加してください。素の `npx playwright install chromium` は
+別バージョンの Chromium を入れることがあるので注意。
+出力は `aria`（アクセシビリティスナップショット）/ `text`（innerText）/ `html`（生 HTML）。
+依存更新は Renovate（`renovate.json`）と GitHub Actions の workflow lockfile
+（`.github/workflows/actions.lock`、`gh actions-lock` で生成）で自動化しています。
+リリースは tagpr（`.github/workflows/tagpr.yml`）が行い、`main` への push で release PR が
+更新され、マージでタグと GitHub Release が作られます（`versionFile = package.json`）。
+挙動・環境変数・`playwright-core` の追随（Renovate と `pi update --extensions`）の詳細は上の各節を参照してください。
+
+## License
+
+MIT

package/fetch-core.ts

@@ -0,0 +1,411 @@
+/**
+ * fetch-url コア: MCP を使わず playwright-core を直接呼び、複数 URL を並列取得する。
+ *
+ * ── MCP 経由の URL 取得との違い ──
+ *   Playwright MCP はブラウザ 1 個・タブ 1 個しか持たないため、navigation+snapshot を
+ *   プロセス内で直列化するしかない（実測: 10 並列で 9 失敗、6 並列で 5 失敗）。
+ *   ここでは自分自身で chromium.launch() し、URL ごとに BrowserContext を作るので
+ *   1 browser / N context / N page で並列 navigation できる。
+ *   context は cookie・localStorage・cache を共有しない独立性を持つ
+ *   （「1 タブを順番に使い回す」という制約がない）。
+ *
+ * ── 並列設計 ──
+ *   - URL リストは worker プール（既定 4、最大 8）で回す。
+ *   - navigation 中の待機は I/O 待ちなので worker 数だけ同時に進行する。
+ *   - 直列化はしない。ブラウザの再利用のみ getBrowser() で共有し、
+ *     launch 直後の二重 launch は launching プロミスで束ねる。
+ *   - 使わない間は IDLE_CLOSE_MS でブラウザを閉じる（pi が TUI で生きていても
+ *     Chromium を常駐させない）。unref 済みタイマーなので process を留めない。
+ *
+ * ── 失敗処理 ──
+ *   恒久的に失敗する URL（DNS 解決不可・不正 URL・証明書エラー等）は
+ *   context を作り直しても無駄なので即打ち切る。
+ *   一時的な失敗のみ、新しい context で 1 回だけ再試行する（既定 2 試行）。
+ *   ブラウザがクラッシュして切断されている場合は getBrowser() が再接続（launch し直し）する。
+ *
+ * ── ブラウザバイナリ ──
+ *   channel "chrome" でシステムの Google Chrome を使う。
+ *   playwright-core 単体はブラウザをダウンロードしないため、無い環境では
+ *   msedge → 内蔵 chromium にフォールバックし、それも無ければ
+ *   `npx playwright install chromium` を案内する。
+ *   env で上書き: PI_FETCH_URL_CHANNEL=chrome|msedge|chromium|"" ("" = 内蔵 chromium),
+ *                 PI_FETCH_URL_HEADLESS=0 で headed,
+ *                 PI_FETCH_URL_NO_SANDBOX=1 で chromiumSandbox: false。
+ */
+import { dirname } from "node:path";
+import { fileURLToPath } from "node:url";
+import type { Browser, Page } from "playwright-core";
+
+export type Mode = "aria" | "text" | "html";
+
+export const MAX_URLS = 16;
+export const MAX_CONCURRENCY = 8;
+export const NAV_ATTEMPTS = 2;
+export const IDLE_CLOSE_MS = 3 * 60 * 1000;
+
+export const DEFAULTS = {
+	mode: "aria" as Mode,
+	maxCharsPerPage: 8000,
+	concurrency: 4,
+	navTimeoutMs: 30000,
+	settleMs: 600,
+};
+
+/** 作り直しても解決しない失敗（即打ち切る） */
+const PERMANENT_NAV_ERROR =
+	/net::ERR_(NAME_NOT_RESOLVED|INTERNET_DISCONNECTED|CONNECTION_REFUSED|CONNECTION_RESET|ADDRESS_UNREACHABLE|UNREACHABLE_NETWORK|UNKNOWN_URL_SCHEME|INVALID_URL|CERT_|BLOCKED_BY|FILE_NOT_FOUND|ACCESS_DENIED|SCHEME_NOT_SUPPORTED)|ERR_(INVALID_URL|UNKNOWN_URL_SCHEME)|Cannot navigate to invalid URL/i;
+
+function isPermanentFailure(message: string): boolean {
+	return PERMANENT_NAV_ERROR.test(message);
+}
+
+function oneLine(message: string): string {
+	return message.replace(/\s+/g, " ").slice(0, 240);
+}
+
+function messageOf(error: unknown): string {
+	if (error instanceof Error) return error.message;
+	return String(error);
+}
+
+// ─────────────────────────── playwright / browser lifecycle ───────────────────────────
+
+type PlaywrightModule = typeof import("playwright-core");
+
+let pw: PlaywrightModule | null = null;
+let browser: Browser | null = null;
+let browserKey = "";
+let launching: Promise<Browser> | null = null;
+let idleTimer: ReturnType<typeof setTimeout> | null = null;
+
+/**
+ * このパッケージ自身のルートディレクトリ（エラー案内の文言に出す）。
+ * ローカル拡張として ~/.pi/agent/extensions/ に置いても、
+ * pi install された npm / git パッケージでも、import 元の位置から求める。
+ */
+export function packageDir(): string {
+	try {
+		return dirname(fileURLToPath(import.meta.url));
+	} catch {
+		return process.cwd();
+	}
+}
+
+/** 重い依存の読み込みはツール実行時まで遅らせる（拡張の factory では何もしない） */
+async function loadPlaywright(): Promise<PlaywrightModule> {
+	if (pw) return pw;
+	try {
+		pw = await import("playwright-core");
+	} catch (error) {
+		throw new Error(
+			`cannot import playwright-core (${oneLine(messageOf(error))}). ` +
+				`Fix: npm install --omit=dev --prefix ${packageDir()}`
+		);
+	}
+	return pw;
+}
+
+function headlessDefault(): boolean {
+	return process.env.PI_FETCH_URL_HEADLESS !== "0";
+}
+
+function channelCandidates(): Array<string | undefined> {
+	const fromEnv = process.env.PI_FETCH_URL_CHANNEL;
+	if (fromEnv !== undefined) return [fromEnv === "" ? undefined : fromEnv];
+	// playwright-core にはブラウザバイナリが同梱されないため、内蔵 chromium は最後に試す。
+	return ["chrome", "msedge", "chromium"];
+}
+
+async function launchBrowser(headless: boolean): Promise<Browser> {
+	const module = await loadPlaywright();
+	const options = {
+		headless,
+		// 共通の launch 引数。AutomationControlled を伏せて bot 判定の足場を減らす。
+		args: ["--disable-blink-features=AutomationControlled"],
+		chromiumSandbox: process.env.PI_FETCH_URL_NO_SANDBOX !== "1",
+		// pi プロセスの SIGINT/SIGTERM で Chromium を巻き込まない
+		handleSIGINT: false,
+		handleSIGTERM: false,
+		handleSIGHUP: false,
+	};
+
+	const tried: string[] = [];
+	let lastError = "";
+	for (const channel of channelCandidates()) {
+		tried.push(channel ?? "embedded chromium");
+		try {
+			return await module.chromium.launch({ ...options, channel });
+		} catch (error) {
+			lastError = oneLine(messageOf(error));
+		}
+	}
+	throw new Error(
+		`could not launch a browser (tried ${tried.join(", ")}): ${lastError}. ` +
+			`Fallback: run \`npx playwright install chromium\` in ${packageDir()}, ` +
+			`or point PI_FETCH_URL_CHANNEL at an installed browser.`
+	);
+}
+
+function clearIdleTimer(): void {
+	if (!idleTimer) return;
+	clearTimeout(idleTimer);
+	idleTimer = null;
+}
+
+function scheduleIdleClose(): void {
+	clearIdleTimer();
+	idleTimer = setTimeout(() => {
+		void closeBrowser();
+	}, IDLE_CLOSE_MS);
+	idleTimer.unref?.();
+}
+
+/** 冪等。reload / session_shutdown / cancel が同じ経路に converge しても安全 */
+export async function closeBrowser(): Promise<void> {
+	clearIdleTimer();
+	const current = browser;
+	const pending = launching;
+	browser = null;
+	browserKey = "";
+	launching = null;
+	if (pending) {
+		try {
+			await pending;
+		} catch {
+			/* launch 失敗は握りつぶす */
+		}
+	}
+	if (current) {
+		try {
+			await current.close();
+		} catch {
+			/* もう閉じている */
+		}
+	}
+}
+
+/**
+ * ブラウザ共有。headless 設定が変わったら作り直す。
+ * クラッシュで切断されている場合は launch し直す（MCP 版での tab reset 相当）。
+ */
+async function getBrowser(headless: boolean): Promise<Browser> {
+	clearIdleTimer();
+	const key = headless ? "headless" : "headed";
+	if (browser && browserKey === key && browser.isConnected()) return browser;
+	if (browser) await closeBrowser();
+	if (!launching) {
+		const started = launchBrowser(headless).then(
+			(launched) => {
+				browser = launched;
+				browserKey = key;
+				launching = null;
+				scheduleIdleClose();
+				return launched;
+			},
+			(error) => {
+				launching = null;
+				throw error;
+			}
+		);
+		launching = started;
+	}
+	return launching;
+}
+
+/** headless Chromium の UA に HeadlessChrome と出て bot 判定されやすいので整える */
+function userAgentFor(instance: Browser): string {
+	const version = instance.version?.() ?? "";
+	const major = version.split(".")[0] || "131";
+	const platform =
+		process.platform === "darwin"
+			? "Macintosh; Intel Mac OS X 10_15_7"
+			: process.platform === "win32"
+				? "Windows NT 10.0; Win64; x64"
+				: "X11; Linux x86_64";
+	return `Mozilla/5.0 (${platform}) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/${major}.0.0.0 Safari/537.36`;
+}
+
+// ─────────────────────────── fetch ───────────────────────────
+
+export type PageFetch = {
+	url: string;
+	ok: boolean;
+	status?: number;
+	finalUrl?: string;
+	title?: string;
+	body: string;
+	characters: number;
+	attempts: number;
+	note: string;
+	ms: number;
+};
+
+type Resolved = typeof DEFAULTS & { signal?: AbortSignal };
+
+async function extract(page: Page, mode: Mode): Promise<string> {
+	if (mode === "html") return await page.content();
+	if (mode === "text") {
+		return await page.evaluate(() => document.body?.innerText ?? document.documentElement?.innerText ?? "");
+	}
+	// aria: Playwright 内蔵の accessibility ツリー YAML。MCP の browser_snapshot とほぼ同じ形。
+	try {
+		return await page.locator("body").ariaSnapshot();
+	} catch {
+		return await page.evaluate(() => document.body?.innerText ?? "");
+	}
+}
+
+type SingleOutcome =
+	| { ok: true; status?: number; finalUrl: string; title: string; body: string }
+	| { ok: false; message: string };
+
+/** 1 URL を独立した context で 1 回。context は最後に必ず閉じる（ページと storage が残らないようにする） */
+async function fetchOnce(instance: Browser, url: string, o: Resolved): Promise<SingleOutcome> {
+	const context = await instance.newContext({
+		userAgent: userAgentFor(instance),
+		viewport: null,
+	});
+	const page = await context.newPage();
+	const signal = o.signal;
+	const onAbort = () => {
+		void page.close({ runBeforeUnload: false }).catch(() => {});
+	};
+	if (signal) {
+		if (signal.aborted) onAbort();
+		else signal.addEventListener("abort", onAbort, { once: true });
+	}
+	try {
+		const response = await page.goto(url, { waitUntil: "domcontentloaded", timeout: o.navTimeoutMs });
+		// SPA は domcontentloaded 時点では空なので load まで待つ（ハングしても落とさない）。
+		await page.waitForLoadState("load", { timeout: Math.min(o.navTimeoutMs, 10000) }).catch(() => {});
+		if (o.settleMs > 0) await page.waitForTimeout(o.settleMs).catch(() => {});
+		const body = await extract(page, o.mode);
+		const title = await page.title().catch(() => "");
+		return {
+			ok: true,
+			status: response?.status(),
+			finalUrl: page.url(),
+			title,
+			body,
+		};
+	} catch (error) {
+		return { ok: false, message: oneLine(messageOf(error)) };
+	} finally {
+		if (signal) signal.removeEventListener("abort", onAbort);
+		await context.close().catch(() => {});
+	}
+}
+
+async function fetchOne(url: string, o: Resolved): Promise<PageFetch> {
+	const startedAt = Date.now();
+	const steps: string[] = [];
+	let lastError = "";
+
+	for (let attempt = 1; attempt <= NAV_ATTEMPTS; attempt++) {
+		let instance: Browser;
+		try {
+			instance = await getBrowser(headlessDefault());
+		} catch (error) {
+			return {
+				url,
+				ok: false,
+				body: "",
+				characters: 0,
+				attempts: attempt,
+				note: `browser launch failed: ${oneLine(messageOf(error))}`,
+				ms: Date.now() - startedAt,
+			};
+		}
+
+		const outcome = await fetchOnce(instance, url, o);
+		if (outcome.ok) {
+			const characters = outcome.body.length;
+			const body =
+				characters > o.maxCharsPerPage
+					? `${outcome.body.slice(0, o.maxCharsPerPage)}\n[trimmed: ${characters} characters total]`
+					: outcome.body || "(empty page)";
+			return {
+				url,
+				ok: true,
+				status: outcome.status,
+				finalUrl: outcome.finalUrl,
+				title: outcome.title,
+				body,
+				characters,
+				attempts: attempt,
+				note: steps.length ? `recovered on attempt ${attempt}: ${steps.join(", ")}` : "",
+				ms: Date.now() - startedAt,
+			};
+		}
+
+		const aborted = Boolean(o.signal?.aborted);
+		const permanent = isPermanentFailure(outcome.message) || aborted;
+		if (permanent) {
+			return {
+				url,
+				ok: false,
+				body: "",
+				characters: 0,
+				attempts: attempt,
+				note: aborted
+					? "cancelled (aborted)"
+					: `navigation failed (permanent, no retry): ${outcome.message}`,
+				ms: Date.now() - startedAt,
+			};
+		}
+		lastError = outcome.message;
+		if (attempt < NAV_ATTEMPTS) steps.push(`fresh context (attempt ${attempt + 1})`);
+	}
+
+	return {
+		url,
+		ok: false,
+		body: "",
+		characters: 0,
+		attempts: NAV_ATTEMPTS,
+		note: `navigation failed after ${NAV_ATTEMPTS} attempts (${steps.join(" -> ") || "no retry"}): ${lastError}`,
+		ms: Date.now() - startedAt,
+	};
+}
+
+/** worker プール。結果は入力順のまま返す（完了順ではない） */
+export async function fetchPages(
+	urls: string[],
+	options: Partial<typeof DEFAULTS> = {},
+	hooks: {
+		signal?: AbortSignal;
+		onPage?: (result: PageFetch, index: number, total: number) => void;
+	} = {}
+): Promise<PageFetch[]> {
+	const o: Resolved = { ...DEFAULTS, ...options, signal: hooks.signal };
+	const total = urls.length;
+	const results: Array<PageFetch | undefined> = new Array(total).fill(undefined);
+	if (total === 0) return [];
+
+	const limit = Math.max(1, Math.min(o.concurrency, MAX_CONCURRENCY, total));
+	let cursor = 0;
+	const worker = async () => {
+		for (;;) {
+			if (hooks.signal?.aborted) return;
+			const index = cursor++;
+			if (index >= total) return;
+			const result = await fetchOne(urls[index], o);
+			results[index] = result;
+			hooks.onPage?.(result, index, total);
+		}
+	};
+	await Promise.all(Array.from({ length: limit }, worker));
+
+	return results.map(
+		(result, index) =>
+			result ?? {
+				url: urls[index],
+				ok: false,
+				body: "",
+				characters: 0,
+				attempts: 0,
+				note: "cancelled (aborted)",
+				ms: 0,
+			}
+	);
+}

package/index.ts

@@ -0,0 +1,144 @@
+/**
+ * fetch-url: 複数 URL を並列で取得する pi 拡張ツール。
+ *
+ * 実装は fetch-core.ts（playwright-core を直接操作）。このファイルは
+ * ツール登録と結果レンダリングだけの薄い層。
+ *
+ * MCP 経由の URL 取得（タブ 1 個を直列に使う）に対して、
+ * fetch_url は自分専用の Chromium を launch し、URL ごとに独立した
+ * BrowserContext を開くので同時並列で取得できる。
+ */
+import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
+import { Type } from "typebox";
+
+import {
+	closeBrowser,
+	DEFAULTS,
+	MAX_CONCURRENCY,
+	MAX_URLS,
+	fetchPages,
+	type PageFetch,
+} from "./fetch-core.ts";
+
+function renderOne(result: PageFetch): string {
+	const header = `--- ${result.url} ---`;
+	if (!result.ok) {
+		return `${header}\nFETCH FAILED (${result.ms}ms) ${result.note}`;
+	}
+	const meta = [
+		result.status !== undefined ? `HTTP ${result.status}` : "",
+		result.finalUrl && result.finalUrl !== result.url ? `redirected to ${result.finalUrl}` : "",
+		result.title ? `title: ${result.title}` : "",
+		`${result.characters} chars`,
+		result.note,
+	]
+		.filter(Boolean)
+		.join(" | ");
+	return `${header}\n[${meta}]\n${result.body}`;
+}
+
+export default function (pi: ExtensionAPI) {
+	pi.registerTool({
+		name: "fetch_url",
+		label: "Fetch URLs (parallel Playwright)",
+		description:
+			"Fetch several web pages at once by driving Playwright directly (no MCP, no API billing). " +
+			"One Chromium serves an isolated BrowserContext per URL, so navigation runs concurrently " +
+			"(default 4 workers, max 8) instead of being serialized through one shared browser tab. " +
+			"Returns an accessibility snapshot per page by default; mode \"text\" gives rendered inner text.",
+		promptSnippet: "Fetch multiple web pages concurrently via a local Playwright browser",
+		promptGuidelines: [
+			"Use fetch_url when you need several URLs at once: it fetches concurrently (up to 8 contexts), unlike single-tab MCP browser tools which serialize navigation.",
+			"mode \"text\" is best for article prose, \"aria\" keeps structure and links, \"html\" returns raw markup; raise max_chars_per_page when output is trimmed.",
+			"fetch_url launches its own Chromium and is independent of any Playwright MCP browser; set PI_FETCH_URL_HEADLESS=0 for headed browsing or PI_FETCH_URL_CHANNEL to pick another browser.",
+		],
+		parameters: Type.Object({
+			urls: Type.Array(Type.String(), {
+				description: `URLs to fetch concurrently. At most ${MAX_URLS} per call; extras are skipped.`,
+			}),
+			mode: Type.Optional(
+				Type.Union([Type.Literal("aria"), Type.Literal("text"), Type.Literal("html")], {
+					description:
+						`"aria" (default) = accessibility snapshot YAML (same shape as Playwright MCP browser_snapshot), "text" = rendered innerText, "html" = page source.`,
+				})
+			),
+			max_chars_per_page: Type.Optional(
+				Type.Number({
+					description: `Trim each page to this many characters (default ${DEFAULTS.maxCharsPerPage}).`,
+				})
+			),
+			concurrency: Type.Optional(
+				Type.Number({
+					description: `How many pages to load at once (1-${MAX_CONCURRENCY}, default ${DEFAULTS.concurrency}).`,
+				})
+			),
+			timeout_ms: Type.Optional(
+				Type.Number({
+					description: `Per-URL navigation timeout in milliseconds (default ${DEFAULTS.navTimeoutMs}).`,
+				})
+			),
+		}),
+		async execute(_toolCallId, params, signal, onUpdate, _ctx) {
+			const requested: string[] = params.urls ?? [];
+			const urls = requested.slice(0, MAX_URLS);
+			const startedAt = Date.now();
+			let done = 0;
+
+			const results = await fetchPages(
+				urls,
+				{
+					mode: params.mode ?? DEFAULTS.mode,
+					maxCharsPerPage: params.max_chars_per_page ?? DEFAULTS.maxCharsPerPage,
+					concurrency: params.concurrency ?? DEFAULTS.concurrency,
+					navTimeoutMs: params.timeout_ms ?? DEFAULTS.navTimeoutMs,
+				},
+				{
+					signal,
+					onPage: (result) => {
+						done += 1;
+						onUpdate?.({
+							content: [
+								{
+									type: "text",
+									text: `${result.ok ? "ok" : "failed"} ${done}/${urls.length}: ${result.url} (${result.ms}ms)`,
+								},
+							],
+							details: {
+								provider: "playwright",
+								product: "chromium",
+								mode: params.mode ?? DEFAULTS.mode,
+								fetched: done,
+								total: urls.length,
+							},
+						});
+					},
+				}
+			);
+
+			const sections = results.map(renderOne);
+			if (requested.length > urls.length) {
+				sections.push(`[skipped ${requested.length - urls.length} URLs: over the ${MAX_URLS} per-call limit]`);
+			}
+			const okCount = results.filter((result) => result.ok).length;
+
+			return {
+				content: [{ type: "text", text: sections.join("\n\n") || "No URLs were fetched." }],
+				details: {
+					provider: "playwright",
+					product: "chromium",
+					mode: params.mode ?? DEFAULTS.mode,
+					concurrency: Math.min(params.concurrency ?? DEFAULTS.concurrency, MAX_CONCURRENCY),
+					urls,
+					ok: okCount,
+					failed: results.length - okCount,
+					ms: Date.now() - startedAt,
+				},
+			};
+		},
+	});
+
+	// reload / session 終了 / プロセス終了で Chromium を残さない。冪等。
+	pi.on("session_shutdown", async () => {
+		await closeBrowser();
+	});
+}

package/package.json

@@ -1,6 +1,67 @@
 {
   "name": "@sasasin/fetch-url",
-  "version": "0.0.0-stage",
-  "stub": true,
-  "description": "Temporary package placeholder for staged publishing"
-}
+  "version": "0.1.1",
+  "description": "Pi extension that fetches many web pages at once by driving Playwright directly: one Chromium, one BrowserContext per URL, concurrent navigation, no MCP and no API billing.",
+  "type": "module",
+  "license": "MIT",
+  "keywords": [
+    "pi-package",
+    "pi",
+    "pi-extension",
+    "pi-coding-agent",
+    "fetch",
+    "web",
+    "url",
+    "scraping",
+    "playwright",
+    "parallel",
+    "accessibility-snapshot"
+  ],
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/sasasin/fetch_url.git"
+  },
+  "bugs": {
+    "url": "https://github.com/sasasin/fetch_url/issues"
+  },
+  "homepage": "https://github.com/sasasin/fetch_url#readme",
+  "engines": {
+    "node": ">=22.19.0"
+  },
+  "pi": {
+    "extensions": [
+      "./index.ts"
+    ]
+  },
+  "files": [
+    "index.ts",
+    "fetch-core.ts",
+    "README.md",
+    "CHANGELOG.md",
+    "LICENSE"
+  ],
+  "publishConfig": {
+    "access": "public"
+  },
+  "scripts": {
+    "typecheck": "tsc -p tsconfig.json",
+    "test:load": "node tests/test-load.mjs",
+    "selftest": "node tests/selftest.mjs --concurrency 4 --mode text https://example.com/ https://news.ycombinator.com/",
+    "test": "npm run typecheck && npm run test:load",
+    "check": "npm run test"
+  },
+  "dependencies": {
+    "playwright-core": "^1.64.1"
+  },
+  "peerDependencies": {
+    "@earendil-works/pi-coding-agent": "*",
+    "typebox": "*"
+  },
+  "devDependencies": {
+    "@earendil-works/pi-coding-agent": "^1.4.0",
+    "@types/node": "^24.0.0",
+    "typebox": "^1.3.27",
+    "typescript": "^5.9.0",
+    "jiti": "2.7.0"
+  }
+}