mcp-scraper 0.3.26 → 0.3.28

package/dist/{worker-UC6D2756.js → worker-NV4GTIRG.js}

@@ -4,10 +4,10 @@ import {
   createHarvestAttemptRecorder,
   harvestProblemResponse,
   serializeHarvestProblem
-} from "./chunk-WYCER2HW.js";
+} from "./chunk-H74L743B.js";
 import {
   harvest
-} from "./chunk-G7PQ64LM.js";
+} from "./chunk-QSSH4RCX.js";
 import {
   browserServiceApiKey
 } from "./chunk-3PRO376E.js";
@@ -125,4 +125,4 @@ export {
   startWorker,
   tickOnce
 };
-//# sourceMappingURL=worker-UC6D2756.js.map
+//# sourceMappingURL=worker-NV4GTIRG.js.map

package/docs/spec-kernel-computer-controls.md

@@ -0,0 +1,166 @@
+# Spec — Tighten Kernel Computer Controls usage (human-like input on the hosted browser)
+
+## Correction to the original premise
+An earlier assessment claimed "we drive the Kernel browser over CDP with raw Playwright — we never use the Computer Controls API." **That was wrong** (a bad grep — searched raw HTTP paths + `page.mouse`, missed the SDK method names). Ground truth in `src/services/browser-agent/browser-agent-service.ts`:
+
+| Action | Implementation today | API used |
+|---|---|---|
+| screenshot | `k.browsers.computer.captureScreenshot` | ✅ Computer Controls |
+| click | `k.browsers.computer.clickMouse` | ✅ Computer Controls |
+| type | `k.browsers.computer.typeText` | ✅ Computer Controls |
+| scroll | `k.browsers.computer.scroll` | ✅ Computer Controls |
+| press | `k.browsers.computer.pressKey` | ✅ Computer Controls |
+| moveMouse | `k.browsers.computer.moveMouse({smooth:true})` | ✅ defined… but never called |
+| goto / read / locate / captureFanout | `playwrightChromium.connectOverCDP` | Playwright/CDP (DOM access — required, kept) |
+
+So the interactive agent's **input + screenshot already follow Kernel's API**. The DOM-read path uses Playwright-over-CDP because Computer Controls cannot read the DOM / element coordinates — that path stays (and is the same free Playwright code used off-Kernel).
+
+## Dual-path principle (the rule, stated)
+- **On Kernel (hosted browser):** drive *input* (move/click/type/scroll/press/screenshot) through **Computer Controls** (`k.browsers.computer.*`) — native OS input, no CDP input fingerprint, bezier curves, inter-keystroke delays. Read the DOM via Playwright-over-CDP (no Computer Controls equivalent).
+- **Off Kernel (free local/HTTP path, `BrowserDriver` non-hosted, future local mode):** keep Playwright. It's free and there's no Computer Controls layer to use.
+
+This spec changes only the Kernel input path. It does not touch Playwright scraping (`BrowserDriver`, reddit/serp/maps/fb deterministic tools) — their bottleneck is IP/challenge, not input cadence, and they barely click.
+
+## The two real gaps (verified)
+1. **Clicks teleport.** `click()` calls `clickMouse(x,y)` with no preceding move. `moveMouse({smooth:true})` exists (bezier) but is **called from nowhere** (`grep` confirms one definition, zero call sites). So every click is an instant cursor jump to the target — a behavioral bot signal Kernel's docs flag ("identical cursor paths, rapid mouse movements"). The bezier capability is built and unused.
+2. **Typing has no human cadence by default.** `typeText(id, text, delayMs?)` only passes `delay` if the *caller* supplies `body.delay` (`browser-agent-routes.ts:486`). The model never sends it → `delay` is `undefined` → instant uniform typing ("uniform typing speeds" — another flagged signal).
+
+## Changes
+
+### 1. `src/services/browser-agent/browser-agent-service.ts`
+
+**Add a constant + jitter helper near the top of the action section (after `client()`):**
+```ts
+const HUMAN_TYPE_DELAY_MS = 55
+function jitter(base: number, spread: number): number {
+  return Math.max(1, Math.round(base + (Math.random() * 2 - 1) * spread))
+}
+```
+
+**`click()` — move (bezier) to the target before clicking.** Replace the body:
+```ts
+export async function click(
+  runtimeSessionId: string,
+  x: number,
+  y: number,
+  opts: { button?: 'left' | 'right' | 'middle'; numClicks?: number } = {},
+): Promise<void> {
+  const k = client()
+  await k.browsers.computer.moveMouse(runtimeSessionId, { x, y, smooth: true })
+  await k.browsers.computer.clickMouse(runtimeSessionId, {
+    x,
+    y,
+    button: opts.button ?? 'left',
+    ...(opts.numClicks ? { num_clicks: opts.numClicks } : {}),
+  })
+}
+```
+(Optional optimization — collapse the two round-trips into one with `computer.batch` if the installed SDK exposes it: `await k.browsers.computer.batch(runtimeSessionId, { actions: [{ type: 'move_mouse', x, y, smooth: true }, { type: 'click_mouse', x, y, button: opts.button ?? 'left', num_clicks: opts.numClicks ?? 1 }] })`. Verify the SDK method/shape before using; if unavailable, keep the two sequential calls above.)
+
+**`typeText()` — default a jittered human delay when the caller gives none:**
+```ts
+export async function typeText(runtimeSessionId: string, text: string, delayMs?: number): Promise<void> {
+  const k = client()
+  const delay = typeof delayMs === 'number' ? delayMs : jitter(HUMAN_TYPE_DELAY_MS, 30)
+  await k.browsers.computer.typeText(runtimeSessionId, { text, delay })
+}
+```
+
+No route changes required — `browser-agent-routes.ts` already forwards `x/y`, `button`, `num_clicks`, and an optional `delay`; the new behavior is purely in the service defaults. The MCP tool schemas/descriptions need no change (the human-like behavior is automatic; callers still just pass x/y/text).
+
+### 2. No change to
+- `goto` / `readPage` / `locatePageTargets` / `captureFanout` — DOM access over CDP is required and free-path-shared. Leave as-is.
+- `BrowserDriver` and all deterministic scrapers (reddit, serp, maps, fb, instagram, youtube) — Playwright stays.
+- The free/local browser path — Playwright stays.
+
+## Verification
+1. `grep -n "moveMouse" src/services/browser-agent/browser-agent-service.ts` → now **2** hits (definition + the new call in `click`).
+2. `npm run build` clean.
+3. Deploy prod; open a session and drive it:
+   - `POST /agent/sessions {url:"https://example.com"}` → `POST /agent/sessions/:id/screenshot` (note an element's x/y) → `POST /agent/sessions/:id/click {x,y}` → screenshot again: the click lands (move+click path works).
+   - `POST /agent/sessions/:id/type {text:"hello world"}` (no `delay`) succeeds — typing now carries the default jittered cadence server-side.
+   - `DELETE /agent/sessions/:id`.
+4. Behavioral check on a detection-sensitive target (manual): drive a known bot-walled login form; clicks should approach via curve and typing should be paced (observable in the live-view / replay).
+
+## Release
+- One file changed (`browser-agent-service.ts`). The behavior ships server-side → live on Vercel deploy; **no npm republish required** (the stdio package is a thin client; tool schemas/descriptions are unchanged).
+- Commit on `main`: `feat(browser): human-like input on Kernel — bezier move-before-click + default typing cadence`.
+
+---
+
+# Phase 2 — Move the DOM-read path to the Playwright Execution API (in-VM + Patchright)
+
+## Why (revised from "defer")
+The read path (`goto`, `readPage`, `locatePageTargets`, `captureFanout`) uses `playwrightChromium.connectOverCDP(cdpWsUrl)` from our serverless function — **outside** the Kernel VM. Kernel's recommended-practices table explicitly says *"Prefer our Playwright Execution API or Computer Controls API over self-hosted Playwright/Puppeteer,"* and the bot-detection guide states the Execution API *"runs in the same VM as the browser, ensuring headers, user-agent strings, and environment match. Kernel automatically applies Patchright to remove automation fingerprints, including headless indicators."* So external CDP is the residual detection signal, and this is the documented fix — not optional polish.
+
+## The API
+`POST /browsers/{id}/playwright/execute` — body `{ code: string, timeout_sec?: 1..300 }`. The `code` runs **in-VM** inside a function with `page`, `context`, `browser` in scope; `return` a value. Response `{ success, result, error, stdout, stderr }`. SDK call (verify exact name against installed `@onkernel/sdk`): `k.browsers.playwright.execute(runtimeSessionId, { code, timeout_sec })`.
+
+## Mechanic
+Our read functions already do their work via `page.evaluate(FN)` over a `connectOverCDP` page. The migration is **mechanical**: take the existing in-page extraction JS (the function bodies passed to `page.evaluate`) and ship them as the `code` string to `execute`, returning the same shape — but now running in-VM with Patchright instead of over external CDP.
+
+### `src/services/browser-agent/browser-agent-service.ts`
+Add a helper:
+```ts
+async function runInVm<T>(runtimeSessionId: string, code: string, timeoutSec = 30): Promise<T> {
+  const k = client()
+  const res = await k.browsers.playwright.execute(runtimeSessionId, { code, timeout_sec: timeoutSec })
+  if (!res.success) throw new Error(res.error || 'playwright/execute failed')
+  return res.result as T
+}
+```
+Then rewrite each read function to call `runInVm` with its existing extraction logic as a code string instead of `connectOverCDP`:
+- **`goto(runtimeSessionId, url)`** → `code`: `` await page.goto(${JSON.stringify(url)}, { waitUntil: 'domcontentloaded', timeout: 45000 }); return { url: page.url(), title: await page.title() }; ``  (signature changes from `cdpWsUrl` to `runtimeSessionId`).
+- **`readPage`** → move the current element/text-collection `page.evaluate` body into the `code` string; `return` the same `{ url, title, text, elements }` object.
+- **`locatePageTargets`** → same: the selector/text-bounds logic becomes the `code` body, `return` the bounds array.
+- **`captureFanout`** → port its capture logic; if it relies on response listeners over CDP, evaluate whether it can run in-VM or must stay on CDP (the fan-out hook listens to network events — this one may legitimately stay on CDP; verify before migrating).
+
+### Routes (`src/api/browser-agent-routes.ts`)
+The read/locate/goto routes currently pass `row.cdp_ws_url` to the service. Change those call sites to pass `row.runtime_session_id`. No other route logic changes.
+
+### Keep CDP only where required
+If `captureFanout` needs live network-event listeners (CDP-only), leave it on `connectOverCDP`. Everything else moves in-VM.
+
+## Dual-path still holds
+- **On Kernel:** input → Computer Controls (Phase 1); DOM read → Playwright Execution API (Phase 2). Zero external CDP except any network-listener edge case.
+- **Off Kernel (free/local):** unchanged local Playwright.
+
+## Verification (Phase 2)
+1. `grep -c connectOverCDP src/services/browser-agent/browser-agent-service.ts` → drops to 0 (or 1 if captureFanout stays).
+2. Build clean.
+3. Drive a session: `goto` → `read` → `locate` return the same shapes as before (regression check against current output).
+4. On a detection-sensitive page, confirm reads no longer attach an external CDP client (the session shows only in-VM execution + Computer Controls input).
+
+## Release (Phase 2)
+- Same file (+ route call-site edits). Server-side → ships on deploy; no npm republish.
+- Separate commit from Phase 1 so each is independently revertible: `feat(browser): read DOM via Kernel Playwright Execution API (in-VM, Patchright) instead of external CDP`.
+
+## Sequencing
+Ship **Phase 1 first** (tiny, pure win), validate, then **Phase 2** (mechanical but touches 3–4 functions + their routes; needs a regression pass on read/locate output shape). Don't bundle — Phase 2 has real surface area.
+
+## Still out of scope
+- Migrating the **deterministic scrapers** (`BrowserDriver` reddit/serp/maps/fb) from `connectOverCDP` to Playwright Execution API. Same benefit would apply, but their bottleneck was IP/challenge (solved) and they're a larger blast radius. Flag as Phase 3 if detection on those tools resurfaces.
+- **Batch beyond move+click** — only if a concrete need appears.
+
+---
+
+# Other Kernel APIs — assessment + findings (2026-06-30)
+
+Reviewed the full Kernel API reference (browsers, playwright, computer-controls, filesystem, processes). Findings:
+
+## TESTED & REJECTED — browser curl for Reddit
+**Hypothesis:** `client.browsers.curl(id, {url})` (`POST /browsers/{id}/curl`) sends an HTTP request through Chrome's stack (TLS fingerprint + cookies + proxy), so curling `old.reddit/...json` through a residential-proxied browser would return the comment tree cheaply, no render.
+**Result: FAILED.** Both residential-proxied and no-proxy curl returned **HTTP 403 / Reddit block page** (no JSON). Reason: curl does NOT execute JavaScript, and Reddit's wall is a **JS challenge** that only a *rendered* browser solves. `reddit_thread` therefore MUST stay headful-render (the render is what beats the challenge; the ~$0.0009 cost is justified). **Do not re-attempt curl for Reddit.**
+**Curl is still valid** for genuine JSON/REST APIs with no JS challenge (inherits the browser's authenticated session + residential IP). Keep as a back-pocket tier; not a current build item.
+
+## REJECTED — do NOT run ffmpeg/media in the Kernel VM (cost-wrong)
+Initially flagged process-exec/filesystem for in-VM ffmpeg muxing. **Retracted — it's the wrong place cost-wise.** A Kernel browser VM bills $0.0001333/s (headful); muxing is pure CPU work unrelated to browsing, so running it there is the most expensive option. And we don't need it for product value:
+- **Transcription already runs outside Kernel** via fal.ai wizper, which takes an `audio_url` and fetches it itself (`media-transcription.ts`) — no download, no ffmpeg, no Kernel. Cheap and correct.
+- **Muxing** (`VideoMixer.ts`) is an optional convenience that gracefully degrades to returning separate track URLs when ffmpeg is absent.
+- The `instagram_media` ENOENT was the **Vercel serverless read-only filesystem** (`mkdir ~/Downloads/...`), NOT a Kernel gap. Fix = write to `/tmp` or return track URLs; if muxing must be reliable, run ffmpeg on a cheap non-serverless worker — never in Kernel.
+**Conclusion: keep all media transcode/transcription OUTSIDE Kernel.** Process-exec/filesystem APIs have no compelling MCP Scraper use today.
+
+## LOW / skip unless needed
+- computer-controls extras: `drag` (slider captchas), `clipboard` read/write, `set_cursor`, `get_cursor_position` — situational agent polish.
+- browsers: create/get/list/delete/update (already used), ad-hoc extension upload (niche), load-extensions.
+- filesystem watch/stream, process async/PTY/signals — only for long-running in-VM jobs.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "mcp-scraper",
-  "version": "0.3.26",
+  "version": "0.3.28",
   "description": "MCP server for MCP Scraper web intelligence tools",
   "type": "module",
   "main": "./dist/index.cjs",