mcp-scraper 0.3.2 → 0.3.4

package/README.md

@@ -9,7 +9,7 @@ Use the MCPB Desktop Extension for the branded Claude Desktop install, or use th
 MCP Scraper ships three local stdio entrypoints plus human-facing helper CLIs:
 
 - `mcp-scraper` — live web intelligence, SERP, PAA, site extraction, YouTube, Facebook ads and organic video transcripts, Maps, directory, rank tracker blueprint, and credit tools.
-- `browser-agent` — agent-controlled browser tools for hosted cloud sessions or local Chrome profile mode, with screenshots, clicks, typing, scrolling, watch URLs, replay links, MP4 replay download, and local profile import/sync helpers.
+- `browser-agent` — agent-controlled direct/no-proxy hosted browser sessions with screenshots, clicks, typing, scrolling, watch URLs, replay links, MP4 replay download, and saved profile setup for authenticated AI visibility workflows.
 - `mcp-scraper-combined` — one context-aware command. In a human terminal it prints the branded ASCII install card; in an MCP client it runs the combined stdio server with both tool sets. This is the entrypoint used by the MCPB Desktop Extension.
 - `mcp-scraper-install` — explicit alias for the human-facing terminal installer card with the branded ASCII intro and copyable install commands.
 - `mcp-scraper-cli` — a human-facing CLI for setup checks, AI-agent config generation, workflow prompts, local SEO workflow runs, and local HTML reports. This command is safe to print because it is not an MCP stdio server.
@@ -120,7 +120,11 @@ Claude Desktop:
 
 Existing MCP configs that use only `npx -y mcp-scraper` still work for the web intelligence server, but they do not automatically add browser tools. Switch to `mcp-scraper-combined` or add the second `browser-agent` config entry if you want browser tools. Use `mcp-scraper@latest` to force npm to resolve the newest published package whenever the MCP client starts a fresh `npx` process.
 
-For local Chrome state, first clone a Chrome profile into MCP Scraper managed storage:
+For customer AI visibility setup, use the hosted profile connection flow. Open the setup browser at Google, give the user the returned `watch_url`, have them click Sign in and complete login, then call `browser_close` to save cookies and local storage into the saved MCP Scraper profile. If the profile later disconnects or shows login controls, run the same setup flow again.
+
+Browser-agent hosted sessions are direct/no-proxy by default. They do not inherit the configured SERP/Maps proxy. Use proxies for SERP/Maps tools when those tools need location or anti-bot handling, not for the interactive login browser.
+
+Advanced local/developer mode can clone a Chrome profile into MCP Scraper managed storage:
 
 ```bash
 mcp-scraper-cli browser import-chrome --email you@example.com --name work-accounts
@@ -131,9 +135,9 @@ MCP_SCRAPER_API_KEY=sk_live_your_key mcp-scraper-cli agent install claude --appl
 
 Set `MCP_SCRAPER_BROWSER_MODE=local` and `MCP_SCRAPER_BROWSER_PROFILE=<name>` when you want `browser_open` to launch local Google Chrome against that managed clone. A `profile` argument passed directly to `browser_open` overrides the default for that one session. Instagram tools can use the same managed profile by passing `browserProfile=<name>` or by running the API with local browser mode configured.
 
-Hosted browser profiles are still available. `BROWSER_AGENT_PROFILE_NAME` sets the default saved Kernel browser profile for hosted sessions. For first-time hosted setup, set `BROWSER_AGENT_PROFILE_SAVE_CHANGES=true`, open a browser with the profile, complete login through the watch URL, then call `browser_close` to persist cookies/local storage into the hosted Kernel profile.
+`BROWSER_AGENT_PROFILE_NAME` sets the default saved hosted browser profile for future sessions. For first-time setup, use `browser_profile_onboard`, open the returned `watch_url`, complete login in the hosted browser, then call `browser_close` to persist cookies/local storage into the saved profile.
 
-Inside MCP clients, use `browser_profile_list` to inspect local Chrome account/profile metadata, `browser_profile_import` to clone a profile for local browser mode, `browser_profile_sync` to refresh an existing clone, and `browser_profile_onboard` only for hosted Kernel setup.
+Inside MCP clients, use `browser_profile_onboard` for customer setup and reconnects. `browser_profile_list`, `browser_profile_import`, and `browser_profile_sync` are local/developer helpers, not the normal customer auth path.
 
 Claude Code one-command setup:
 
@@ -213,11 +217,11 @@ env = { MCP_SCRAPER_API_KEY = "sk_live_your_key" }
 
 ### Browser-agent tools
 
-- `browser_open` — open a browser session. Hosted mode returns a human `watch_url`; local mode opens Google Chrome on this machine against an imported managed profile.
+- `browser_open` — open a direct/no-proxy hosted browser session. Pass a saved profile name to load an authenticated session. Hosted mode returns a human `watch_url`.
 - `browser_profile_list` — list local Chrome account/profile metadata and suggested managed profile names. This does not read cookies, passwords, browsing history, or copy local Chrome state.
 - `browser_profile_import` — clone a local Chrome profile into `~/.mcp-scraper/browser-profiles` for local browser mode. This copies browser state files but skips cache and locks.
 - `browser_profile_sync` — refresh an existing managed local profile clone from its recorded source Chrome profile.
-- `browser_profile_onboard` — create or load a hosted Kernel browser profile and open a setup browser with profile saving enabled. The user logs in through the `watch_url`, then `browser_close` persists cookies/local storage into the hosted Kernel profile.
+- `browser_profile_onboard` — create or load a saved hosted browser profile and open a direct/no-proxy setup browser at Google with profile saving enabled. The user clicks Sign in through the `watch_url`, then `browser_close` persists cookies/local storage into the saved profile. Use this again whenever the profile disconnects.
 - `browser_screenshot` — capture a screenshot plus visible text and clickable element center coordinates and DOM bounds.
 - `browser_read` — read the current page text and elements with center coordinates and DOM bounds, without an image.
 - `browser_locate` — locate exact visible DOM elements or text ranges and return screenshot-pixel bounds.
@@ -263,7 +267,7 @@ The `mcp-scraper` and `mcp-scraper-combined` NPX stdio servers also expose saved
 - `MCP_SCRAPER_BROWSER_EXECUTABLE` optionally points local browser mode at a Chrome executable. On macOS, Google Chrome is recommended so Keychain-backed Chrome state works.
 - `MCP_SCRAPER_BROWSER_PROFILE_DIR` optionally points local browser mode at a direct Chrome-compatible user data directory instead of a managed profile manifest.
 - Instagram tools accept `browserMode` (`auto`, `hosted`, `local`) and `browserProfile`; `browserProfile` uses the same managed local profiles created by `browser_profile_import/sync`.
-- `BROWSER_AGENT_PROFILE_NAME` is optional and sets the default saved Kernel browser profile for hosted-mode `browser-agent` and `mcp-scraper-combined` stdio sessions. Aliases: `BROWSER_SERVICE_PROFILE_NAME`, `KERNEL_BROWSER_PROFILE_NAME`, `KERNEL_PROFILE_NAME`.
+- `BROWSER_AGENT_PROFILE_NAME` is optional and sets the default saved hosted browser profile for `browser-agent` and `mcp-scraper-combined` stdio sessions. Aliases: `BROWSER_SERVICE_PROFILE_NAME`, `KERNEL_BROWSER_PROFILE_NAME`, `KERNEL_PROFILE_NAME`.
 - `BROWSER_AGENT_PROFILE_SAVE_CHANGES=true` is optional hosted setup mode. It persists cookies and local storage back to the named profile when `browser_close` deletes the hosted browser session. Aliases: `BROWSER_SERVICE_PROFILE_SAVE_CHANGES`, `KERNEL_BROWSER_PROFILE_SAVE_CHANGES`, `KERNEL_PROFILE_SAVE_CHANGES`.
 
 Every web intelligence tool call made through `mcp-scraper` or `mcp-scraper-combined` saves a full Markdown report locally by default and returns the file path in the MCP response. The hosted `/mcp` endpoint returns reports inline only and never writes files. Browser replay downloads are saved by `browser_replay_download` under `MCP_SCRAPER_OUTPUT_DIR/browser-replays`.

package/dist/bin/api-server.cjs

@@ -15265,7 +15265,9 @@ ${headingSection}${kpoSection}${brandingSection}${bodySection}${screenshotSectio
     entityTypes: kpo?.type ?? [],
     napScore: kpo?.napScore ?? null,
     missingSchemaFields: kpo?.missingFields ?? [],
-    screenshotSaved: screenshotPath ?? null
+    screenshotSaved: screenshotPath ?? null,
+    branding: branding ?? null,
+    mediaAssets: media?.assets ?? null
   };
   if (screenshotMeta?.base64) {
     return {
@@ -22078,7 +22080,7 @@ var PACKAGE_VERSION;
 var init_version = __esm({
   "src/version.ts"() {
     "use strict";
-    PACKAGE_VERSION = "0.3.2";
+    PACKAGE_VERSION = "0.3.4";
   }
 });
 
@@ -24798,11 +24800,16 @@ async function ensureProfile(k, name) {
     if (!isProfileConflict(err)) throw err;
   }
 }
+function explicitProxyId(value) {
+  const trimmed = value?.trim();
+  return trimmed || void 0;
+}
 async function createSession(opts = {}) {
   const k = client();
-  const resolvedProxyId = opts.proxyId ?? browserServiceProxyId();
+  const resolvedProxyId = explicitProxyId(opts.proxyId);
   const resolvedProfileName = opts.profileName ?? browserServiceProfileName();
   const resolvedSaveProfileChanges = opts.saveProfileChanges ?? browserServiceProfileSaveChanges();
+  const disableDefaultProxy = opts.disableDefaultProxy ?? !resolvedProxyId;
   if (resolvedProfileName && resolvedSaveProfileChanges === true) {
     await ensureProfile(k, resolvedProfileName);
   }
@@ -24818,7 +24825,7 @@ async function createSession(opts = {}) {
     } : {}
   });
   const runtimeSessionId = browser.session_id;
-  if (opts.disableDefaultProxy) {
+  if (disableDefaultProxy) {
     try {
       await k.browsers.update(runtimeSessionId, { disable_default_proxy: true });
     } catch {
@@ -25219,7 +25226,7 @@ function buildBrowserAgentRoutes() {
         proxyId: typeof body.proxy_id === "string" ? body.proxy_id : void 0,
         profileName: typeof body.profile === "string" ? body.profile : void 0,
         saveProfileChanges: typeof body.save_profile_changes === "boolean" ? body.save_profile_changes : void 0,
-        disableDefaultProxy: body.disable_default_proxy === true,
+        disableDefaultProxy: typeof body.disable_default_proxy === "boolean" ? body.disable_default_proxy : void 0,
         viewport: body.viewport && typeof body.viewport === "object" ? body.viewport : void 0
       });
       runtimeSessionId = created.runtimeSessionId;
@@ -25651,7 +25658,7 @@ async function selectSession(id) {
 }
 
 async function openSession() {
-  const r = await api('POST', '/sessions', { label: 'console' });
+  const r = await api('POST', '/sessions', { label: 'console', disable_default_proxy: true });
   if (r.ok) { await refreshSessions(); selectSession(r.data.session_id); }
   else alert('Open failed: ' + JSON.stringify(r.data));
 }