gologin-web-access 0.3.0 → 0.3.1

package/CHANGELOG.md

@@ -14,6 +14,6 @@ Highlights:
 - Unified CLI entry point for Gologin Web Unlocker and Gologin Cloud Browser workflows
 - Scraping commands: `scrape`, `scrape-markdown`, `scrape-text`, `scrape-json`, `batch-scrape`
 - Browser commands: `open`, `snapshot`, `click`, `type`, `screenshot`, `close`, `sessions`, `current`
-- Clear two-key configuration model with `GOLOGIN_WEB_UNLOCKER_API_KEY` and `GOLOGIN_CLOUD_TOKEN`
+- Clear two-key configuration model with `GOLOGIN_WEB_UNLOCKER_API_KEY` and `GOLOGIN_TOKEN`
 - `doctor`, `config show`, and `config init` to reduce setup friction
 - Compatibility support for legacy env names used by existing Gologin tools

package/README.md

@@ -26,6 +26,13 @@ The point of the unified CLI is that both modes live in one product with one com
 
 ## Command Groups
 
+### Quick Picks
+
+- `read` for "read this docs page/article" or "tell me what is on this page"
+- `scrape-text` for plain text from one known page when you do not need headings/links metadata
+- `scrape-json` for structured title, description, headings, and links from one known page
+- `batch-scrape` for many known URLs at once; add `--output <path>` when the JSON may be large and add `--strict` only if partial success should fail the command
+
 ### Scraping / Read
 
 These commands use Gologin Web Unlocker:
@@ -35,7 +42,7 @@ These commands use Gologin Web Unlocker:
 - `gologin-web-access scrape-markdown <url> [--source auto|unlocker|browser]`
 - `gologin-web-access scrape-text <url> [--source auto|unlocker|browser]`
 - `gologin-web-access scrape-json <url> [--fallback none|browser]`
-- `gologin-web-access batch-scrape <url...> [--format html|markdown|text|json] [--fallback none|browser] [--source auto|unlocker|browser] [--only-main-content] [--retry <n>] [--backoff-ms <ms>] [--summary]`
+- `gologin-web-access batch-scrape <url...> [--format html|markdown|text|json] [--fallback none|browser] [--source auto|unlocker|browser] [--only-main-content] [--retry <n>] [--backoff-ms <ms>] [--summary] [--output <path>] [--strict]`
 - `gologin-web-access batch-extract <url...> --schema <schema.json> [--source auto|unlocker|browser] [--retry <n>] [--backoff-ms <ms>] [--summary] [--output <path>]`
 - `gologin-web-access search <query> [--limit <n>] [--country <cc>] [--language <lang>] [--source auto|unlocker|browser]`
 - `gologin-web-access map <url> [--limit <n>] [--max-depth <n>] [--concurrency <n>] [--strict]`
@@ -104,6 +111,9 @@ Use these when you need state, interaction, or multi-step browser flows.
 ## When To Use `scrape` vs `browser`
 
 - Use `scrape` commands when you need page content, extracted text, markdown, or simple structured output.
+- Use `read` as the default for docs and article reading when you want one high-level main-content command rather than choosing HTML/text/markdown yourself.
+- Use `scrape-text` when you already know you want plain text.
+- Use `scrape-json` when you want structured metadata and headings instead of full prose.
 - Use `search` when you need web discovery or SERP results before deciding what to scrape. It now tries multiple search paths automatically, validates that the response is a real SERP, and reuses a short local cache for repeated queries.
 - Use `map` when you need internal link discovery or a site inventory.
 - Use `crawl` when you need multi-page read-only extraction across a site.
@@ -153,18 +163,18 @@ If the browser surface grows substantially later, a nested namespace may become
 
 ## Credentials And Config
 
-This CLI uses two different Gologin credentials on purpose, because the underlying products are different.
+This CLI uses two different GoLogin credentials on purpose, because the underlying products are different.
 
 - `GOLOGIN_WEB_UNLOCKER_API_KEY`
   Required for Scraping / Read commands.
-- `GOLOGIN_CLOUD_TOKEN`
+- `GOLOGIN_TOKEN`
   Required for `gologin-web-access open` and for profile validation in `gologin-web-access doctor`.
 - `GOLOGIN_DEFAULT_PROFILE_ID`
   Optional default profile for browser flows.
 - `GOLOGIN_DAEMON_PORT`
   Optional local daemon port for browser workflows.
 
-Recommended full setup for agents is to configure both `GOLOGIN_WEB_UNLOCKER_API_KEY` and `GOLOGIN_CLOUD_TOKEN` before starting work, even if the current task looks read-only or browser-only.
+Recommended full setup for agents is to configure both `GOLOGIN_WEB_UNLOCKER_API_KEY` and `GOLOGIN_TOKEN` before starting work, even if the current task looks read-only or browser-only.
 
 Missing-key errors are command-group specific. Example:
 
@@ -174,7 +184,7 @@ Environment variables are the primary configuration mechanism:
 
 ```bash
 export GOLOGIN_WEB_UNLOCKER_API_KEY="wu_..."
-export GOLOGIN_CLOUD_TOKEN="gl_..."
+export GOLOGIN_TOKEN="gl_..."
 export GOLOGIN_DEFAULT_PROFILE_ID="profile_123"
 export GOLOGIN_DAEMON_PORT="4590"
 ```
@@ -188,8 +198,8 @@ gologin-web-access config init
 Useful variants:
 
 ```bash
-gologin-web-access config init --web-unlocker-api-key wu_... --cloud-token gl_...
-gologin-web-access config init --web-unlocker-key wu_... --cloud-token gl_...
+gologin-web-access config init --web-unlocker-api-key wu_... --token gl_...
+gologin-web-access config init --web-unlocker-key wu_... --token gl_...
 ```
 
 That writes `~/.gologin-web-access/config.json` once and the CLI will keep reading it on later runs.
@@ -211,7 +221,7 @@ Gologin Web Access will also read the older path `~/.gologin-web/config.json` if
 Backward-compatible aliases are also accepted for existing setups:
 
 - `GOLOGIN_WEBUNLOCKER_API_KEY`
-- `GOLOGIN_TOKEN`
+- `GOLOGIN_CLOUD_TOKEN`
 - `GOLOGIN_PROFILE_ID`
 
 Useful config commands:
@@ -259,7 +269,7 @@ gologin-web-access parse-document ./example.pdf
 ### Interact With A Site
 
 ```bash
-export GOLOGIN_CLOUD_TOKEN="gl_..."
+export GOLOGIN_TOKEN="gl_..."
 export GOLOGIN_DEFAULT_PROFILE_ID="profile_123"
 
 gologin-web-access open https://example.com
@@ -280,7 +290,7 @@ gologin-web-access close
 ### Search In A Real Browser
 
 ```bash
-export GOLOGIN_CLOUD_TOKEN="gl_..."
+export GOLOGIN_TOKEN="gl_..."
 
 gologin-web-access search-browser "gologin antidetect browser"
 gologin-web-access snapshot -i
@@ -299,7 +309,10 @@ gologin-web-access snapshot -i
 - `batch-scrape --only-main-content` lets markdown, text, and html batch runs use the same readable-content isolation path as `read`.
 - `crawl --only-main-content` uses the same readable-fragment extraction strategy for html, markdown, and text crawl output, but stays on the stateless unlocker path.
 - `batch-scrape --summary` prints a one-line success/failure summary to `stderr` after the JSON payload.
+- `batch-scrape` now returns exit code `0` on partial success by default and only fails the command when every URL failed. Add `--strict` if any single failed URL should make the whole batch exit non-zero.
+- `batch-scrape --output <path>` writes the full JSON to disk so shells and agent consoles cannot truncate a large payload silently.
 - `batch-scrape --format json` now returns the same structured scrape envelope as `scrape-json`, including `renderSource`, `fallbackAttempted`, `fallbackUsed`, and `request.attemptCount/retryCount/attempts`.
+- `scrape-json` now surfaces explicit `BLOCKED_PAGE` failures when structured output clearly matches a challenge or block page, instead of silently looking like a valid empty result.
 - `search` now returns `requestedLimit`, `returnedCount`, `warnings`, `cacheTtlMs`, and per-result `position`.
 - `search` may return fewer results than the requested `--limit` when the upstream SERP contains fewer valid results; inspect `returnedCount`, `warnings`, and `attempts`.
 - `change-track` now accepts `--retry`, `--backoff-ms`, and `--timeout-ms`, and JSON output includes request metadata.

package/dist/cli.js

@@ -154,6 +154,12 @@ async function main() {
     configGroup.addCommand((0, configShow_1.buildConfigShowCommand)());
     configGroup.addCommand((0, configInit_1.buildConfigInitCommand)());
     program.addHelpText("after", `
+Quick picks:
+  read          Best default for "read this docs page/article" and main-content extraction
+  scrape-text   Plain text from one known page when you do not need headings/links metadata
+  scrape-json   Structured title, description, headings, and links from one known page
+  batch-scrape  Fetch many known URLs at once; add --output <path> for large results and --strict only when partial success should fail the command
+
 Command groups:
   Scraping: gologin-web-access scrape|read|scrape-markdown|scrape-text|scrape-json|batch-scrape|batch-extract|search|map|crawl|crawl-start|crawl-status|crawl-result|crawl-errors|extract|change-track|batch-change-track|parse-document
   Browser:  gologin-web-access open|search-browser|scrape-screenshot|tabs|tabopen|tabfocus|tabclose|snapshot|click|dblclick|focus|type|fill|hover|select|check|uncheck|press|scroll|scrollintoview|wait|get|back|forward|reload|find|cookies|cookies-import|cookies-clear|storage-export|storage-import|storage-clear|eval|upload|pdf|screenshot|close|sessions|current
@@ -161,7 +167,7 @@ Command groups:
 
 Key model:
   ${"GOLOGIN_WEB_UNLOCKER_API_KEY"} powers scraping commands.
-  ${"GOLOGIN_CLOUD_TOKEN"} powers browser commands.
+  ${"GOLOGIN_TOKEN"} powers browser commands.
   Recommended setup: configure both keys up front, even if the current task only needs one path.
 `);
     await program.parseAsync(process.argv);

package/dist/commands/batchScrape.js

@@ -1,6 +1,13 @@
 "use strict";
+var __importDefault = (this && this.__importDefault) || function (mod) {
+    return (mod && mod.__esModule) ? mod : { "default": mod };
+};
 Object.defineProperty(exports, "__esModule", { value: true });
 exports.buildBatchScrapeCommand = buildBatchScrapeCommand;
+exports.resolveBatchScrapeExitCode = resolveBatchScrapeExitCode;
+exports.shouldWarnAboutLargeBatchOutput = shouldWarnAboutLargeBatchOutput;
+const fs_1 = require("fs");
+const path_1 = __importDefault(require("path"));
 const commander_1 = require("commander");
 const config_1 = require("../config");
 const output_1 = require("../lib/output");
@@ -17,7 +24,9 @@ function buildBatchScrapeCommand() {
         .option("--fallback <mode>", "Structured scrape fallback: none or browser", "none")
         .option("--source <source>", "Read source for --only-main-content mode: auto, unlocker, or browser", "auto")
         .option("--only-main-content", "For html, markdown, or text formats, isolate the most readable content block per page")
+        .option("--output <path>", "Write the full batch result JSON to a file")
         .option("--summary", "Print one-line summary stats to stderr after the JSON output")
+        .option("--strict", "Exit non-zero if any URL in the batch fails")
         .action(async (urls, options) => {
         const config = await (0, config_1.loadConfig)();
         const format = normalizeFormat(options.format);
@@ -48,18 +57,29 @@ function buildBatchScrapeCommand() {
                     ok: false,
                     format,
                     error: error instanceof Error ? error.message : "Unknown error",
+                    code: extractErrorCode(error),
                     status: extractStatusCode(error),
                     request,
                 };
             }
         });
-        (0, output_1.printJson)(results);
+        const payload = `${JSON.stringify(results, null, 2)}\n`;
+        if (options.output) {
+            const outputPath = (0, shared_1.resolveOutputPath)(options.output);
+            await fs_1.promises.mkdir(path_1.default.dirname(outputPath), { recursive: true });
+            await fs_1.promises.writeFile(outputPath, payload, "utf8");
+            (0, output_1.printText)(outputPath);
+        }
+        else {
+            (0, output_1.printText)(payload);
+            if (shouldWarnAboutLargeBatchOutput(payload)) {
+                process.stderr.write("Batch output is large. If your shell or agent truncates stdout, rerun with --output <path> to keep the full JSON.\n");
+            }
+        }
         if (options.summary) {
             process.stderr.write(formatBatchSummary(results) + "\n");
         }
-        if (results.some((result) => !result.ok)) {
-            process.exitCode = 1;
-        }
+        process.exitCode = resolveBatchScrapeExitCode(results, Boolean(options.strict));
     })));
 }
 function normalizeFormat(value) {
@@ -120,6 +140,19 @@ function formatBatchSummary(results) {
     const failed = requested - ok;
     return `Summary: ${requested} requested, ${ok} ok, ${failed} failed.`;
 }
+function resolveBatchScrapeExitCode(results, strict) {
+    const okCount = results.filter((result) => result.ok).length;
+    if (okCount === 0) {
+        return 1;
+    }
+    if (strict && okCount !== results.length) {
+        return 1;
+    }
+    return 0;
+}
+function shouldWarnAboutLargeBatchOutput(payload) {
+    return payload.length >= 100_000;
+}
 function extractStatusCode(error) {
     if (typeof error === "object" &&
         error !== null &&
@@ -138,3 +171,12 @@ function extractRequestMeta(error) {
     }
     return undefined;
 }
+function extractErrorCode(error) {
+    if (typeof error === "object" &&
+        error !== null &&
+        "code" in error &&
+        typeof error.code === "string") {
+        return error.code;
+    }
+    return undefined;
+}

package/dist/commands/close.js

@@ -8,9 +8,13 @@ function buildCloseCommand() {
     return new commander_1.Command("close")
         .description("Close the current browser session or a specific session.")
         .option("--session <id>", "Session ID. Defaults to the current session.")
+        .option("--all", "Close every tracked browser session in the current daemon.")
         .action(async (options) => {
         const config = await (0, config_1.loadConfig)();
         const args = ["close"];
+        if (options.all) {
+            args.push("--all");
+        }
         if (options.session) {
             args.push("--session", options.session);
         }

package/dist/commands/configInit.js

@@ -8,10 +8,11 @@ const output_1 = require("../lib/output");
 const unlocker_1 = require("../lib/unlocker");
 function buildConfigInitCommand() {
     return new commander_1.Command("init")
-        .description("Write ~/.gologin-web-access/config.json with current values or placeholders. Recommended: persist both Web Unlocker and Cloud Browser credentials.")
+        .description("Write ~/.gologin-web-access/config.json with current values or placeholders. Recommended: persist both the Web Unlocker key and the GoLogin token.")
         .option("--web-unlocker-api-key <key>", "Persist a Web Unlocker API key")
         .option("--web-unlocker-key <key>", "Alias for --web-unlocker-api-key")
-        .option("--cloud-token <token>", "Persist a Cloud Browser token")
+        .option("--token <token>", "Persist a GoLogin token")
+        .option("--cloud-token <token>", "Backward-compatible alias for --token")
         .option("--default-profile-id <id>", "Persist a default Gologin profile ID")
         .option("--daemon-port <port>", "Persist a daemon port", String(config_1.DEFAULT_DAEMON_PORT))
         .option("--no-validate", "Skip live key validation after writing config")
@@ -20,7 +21,10 @@ function buildConfigInitCommand() {
         const webUnlockerApiKey = options.webUnlockerApiKey ?? options.webUnlockerKey ?? process.env[config_1.ENV_NAMES.webUnlockerApiKey];
         const result = await (0, config_1.initConfigFile)({
             webUnlockerApiKey,
-            cloudToken: options.cloudToken ?? process.env[config_1.ENV_NAMES.cloudToken],
+            cloudToken: options.token ??
+                options.cloudToken ??
+                process.env[config_1.ENV_NAMES.cloudToken] ??
+                process.env.GOLOGIN_CLOUD_TOKEN,
             defaultProfileId: options.defaultProfileId ?? process.env[config_1.ENV_NAMES.defaultProfileId],
             daemonPort: Number(options.daemonPort ?? process.env[config_1.ENV_NAMES.daemonPort] ?? config_1.DEFAULT_DAEMON_PORT),
         }, {
@@ -37,7 +41,7 @@ function buildConfigInitCommand() {
                 value: result.config.webUnlockerApiKey ? "written" : "left empty",
             },
             {
-                label: "Cloud token",
+                label: "GoLogin token",
                 value: result.config.cloudToken ? "written" : "left empty",
             },
             {
@@ -50,7 +54,7 @@ function buildConfigInitCommand() {
             },
         ]);
         if (!result.config.webUnlockerApiKey || !result.config.cloudToken) {
-            (0, output_1.printText)("Recommended next step: configure both GOLOGIN_WEB_UNLOCKER_API_KEY and GOLOGIN_CLOUD_TOKEN so agents can use scraping and browser flows without asking again.");
+            (0, output_1.printText)("Recommended next step: configure both GOLOGIN_WEB_UNLOCKER_API_KEY and GOLOGIN_TOKEN so agents can use scraping and browser flows without asking again.");
         }
         if (options.validate === false) {
             return;
@@ -66,7 +70,7 @@ function buildConfigInitCommand() {
         if (result.config.cloudToken) {
             const validation = await (0, cloudApi_1.validateCloudToken)(result.config.cloudToken);
             validationRows.push({
-                label: "Cloud token validation",
+                label: "GoLogin token validation",
                 value: validation.ok ? "ok" : `failed${validation.status ? ` (${validation.status})` : ""}: ${validation.detail}`,
             });
         }

package/dist/commands/scrapeJson.js

@@ -19,6 +19,15 @@ function buildScrapeJsonCommand() {
             profile: options.profile,
             request: (0, shared_1.normalizeUnlockerRequestOptions)(options),
         });
+        if (envelope.fallbackAttempted) {
+            const fallbackStatus = envelope.fallbackUsed
+                ? "Browser fallback succeeded and replaced the unlocker result."
+                : `Browser fallback was attempted but not used. ${envelope.fallbackReason ?? "It did not improve the structured output."}`;
+            process.stderr.write(`${fallbackStatus}\n`);
+        }
+        if (envelope.warning) {
+            process.stderr.write(`${envelope.warning}\n`);
+        }
         (0, output_1.printJson)(envelope);
     })));
 }

package/dist/commands/sessions.js

@@ -7,8 +7,17 @@ const agentCli_1 = require("../lib/agentCli");
 function buildSessionsCommand() {
     return new commander_1.Command("sessions")
         .description("List active daemon-backed browser sessions.")
-        .action(async () => {
+        .option("--prune", "Close tracked sessions idle for too long before listing.")
+        .option("--older-than-ms <ms>", "Idle threshold used with --prune.")
+        .action(async (options) => {
         const config = await (0, config_1.loadConfig)();
-        await (0, agentCli_1.runAgentCommand)(["sessions"], config);
+        const args = ["sessions"];
+        if (options.prune) {
+            args.push("--prune");
+        }
+        if (options.olderThanMs) {
+            args.push("--older-than-ms", options.olderThanMs);
+        }
+        await (0, agentCli_1.runAgentCommand)(args, config);
     });
 }

package/dist/config.js

@@ -23,13 +23,13 @@ const CONFIG_FILENAME = "config.json";
 exports.DEFAULT_DAEMON_PORT = 4590;
 exports.ENV_NAMES = {
     webUnlockerApiKey: "GOLOGIN_WEB_UNLOCKER_API_KEY",
-    cloudToken: "GOLOGIN_CLOUD_TOKEN",
+    cloudToken: "GOLOGIN_TOKEN",
     defaultProfileId: "GOLOGIN_DEFAULT_PROFILE_ID",
     daemonPort: "GOLOGIN_DAEMON_PORT",
 };
 const LEGACY_ENV_NAMES = {
     webUnlockerApiKey: ["GOLOGIN_WEBUNLOCKER_API_KEY"],
-    cloudToken: ["GOLOGIN_TOKEN"],
+    cloudToken: ["GOLOGIN_CLOUD_TOKEN"],
     defaultProfileId: ["GOLOGIN_PROFILE_ID"],
     daemonPort: [],
 };

package/dist/doctor.js

@@ -17,7 +17,7 @@ async function runDoctor(options = {}) {
         detail: config.webUnlockerApiKey ? `configured via ${config.sources.webUnlockerApiKey}` : "missing",
     });
     checks.push({
-        name: "Cloud Browser token",
+        name: "GoLogin token",
         status: config.cloudToken ? "ok" : "warn",
         detail: config.cloudToken ? `configured via ${config.sources.cloudToken}` : "missing",
     });
@@ -25,7 +25,7 @@ async function runDoctor(options = {}) {
         name: "Recommended full setup",
         status: recommended.ready ? "ok" : "warn",
         detail: recommended.ready
-            ? "both GOLOGIN_WEB_UNLOCKER_API_KEY and GOLOGIN_CLOUD_TOKEN are configured"
+            ? "both GOLOGIN_WEB_UNLOCKER_API_KEY and GOLOGIN_TOKEN are configured"
             : `missing ${recommended.missing.join(" and ")}`,
     });
     checks.push({
@@ -46,7 +46,7 @@ async function runDoctor(options = {}) {
         checks.push({
             name: "Default profile",
             status: "warn",
-            detail: `${config.defaultProfileId} configured, but Cloud Browser token is missing so existence could not be verified`,
+            detail: `${config.defaultProfileId} configured, but GOLOGIN_TOKEN is missing so existence could not be verified`,
         });
     }
     else {

package/dist/internal-agent/commands/close.js

@@ -1,10 +1,20 @@
 "use strict";
 Object.defineProperty(exports, "__esModule", { value: true });
 exports.runCloseCommand = runCloseCommand;
+const errors_1 = require("../lib/errors");
 const utils_1 = require("../lib/utils");
 async function runCloseCommand(context, argv) {
     const parsed = (0, utils_1.parseArgs)(argv);
+    const closeAll = (0, utils_1.getFlagBoolean)(parsed, "all");
     const sessionId = (0, utils_1.getFlagString)(parsed, "session");
+    if (closeAll) {
+        if (sessionId) {
+            throw new errors_1.AppError("BAD_REQUEST", "--all cannot be combined with --session", 400);
+        }
+        const response = await context.client.request("POST", "/sessions/close-all");
+        context.stdout.write(`closed ${response.closed} session(s)\n`);
+        return;
+    }
     const resolvedSessionId = sessionId ??
         (await context.client.request("GET", "/sessions/current")).sessionId;
     const response = await context.client.request("POST", `/sessions/${resolvedSessionId}/close`);

package/dist/internal-agent/commands/sessions.js

@@ -1,9 +1,27 @@
 "use strict";
 Object.defineProperty(exports, "__esModule", { value: true });
 exports.runSessionsCommand = runSessionsCommand;
+const errors_1 = require("../lib/errors");
 const utils_1 = require("../lib/utils");
+function parseOlderThanMs(value) {
+    if (!value) {
+        return undefined;
+    }
+    const parsed = Number(value);
+    if (!Number.isInteger(parsed) || parsed < 0) {
+        throw new errors_1.AppError("BAD_REQUEST", "--older-than-ms must be a non-negative integer", 400);
+    }
+    return parsed;
+}
 async function runSessionsCommand(context, argv) {
-    (0, utils_1.parseArgs)(argv);
+    const parsed = (0, utils_1.parseArgs)(argv);
+    if ((0, utils_1.getFlagBoolean)(parsed, "prune")) {
+        const olderThanMs = parseOlderThanMs((0, utils_1.getFlagString)(parsed, "older-than-ms"));
+        const prune = await context.client.request("POST", "/sessions/prune", {
+            maxIdleMs: olderThanMs,
+        });
+        context.stderr.write(`pruned ${prune.closed} session(s) idle for at least ${prune.maxIdleMs}ms\n`);
+    }
     const response = await context.client.request("GET", "/sessions");
     if (response.sessions.length === 0) {
         context.stdout.write("no sessions\n");

package/dist/internal-agent/daemon/server.js

@@ -53,6 +53,15 @@ async function handleRequest(request, response) {
             (0, utils_1.writeJsonResponse)(response, 200, await sessionManager.currentSession());
             return;
         }
+        if (method === "POST" && pathname === "/sessions/close-all") {
+            (0, utils_1.writeJsonResponse)(response, 200, await sessionManager.closeAll());
+            return;
+        }
+        if (method === "POST" && pathname === "/sessions/prune") {
+            const body = (await (0, utils_1.readJsonBody)(request));
+            (0, utils_1.writeJsonResponse)(response, 200, await sessionManager.pruneSessions(body?.maxIdleMs));
+            return;
+        }
         if (method === "POST" && pathname === "/sessions/open") {
             const body = (await (0, utils_1.readJsonBody)(request));
             (0, utils_1.writeJsonResponse)(response, 200, await sessionManager.open(body));

package/dist/internal-agent/daemon/sessionManager.js

@@ -13,6 +13,8 @@ const refStore_1 = require("./refStore");
 const snapshot_1 = require("./snapshot");
 class SessionManager {
     config;
+    static DEFAULT_PRUNE_IDLE_MS = 10 * 60 * 1000;
+    static CLOUD_SLOT_RELEASE_WAIT_MS = 3_000;
     sessions = new Map();
     activeSessionId;
     refStore = new refStore_1.RefStore();
@@ -38,6 +40,32 @@ class SessionManager {
         }
         return Date.now() - lastActivityAt > session.idleTimeoutMs;
     }
+    sessionIdleMs(session) {
+        const lastActivityAt = Date.parse(session.lastActivityAt);
+        if (Number.isNaN(lastActivityAt)) {
+            return 0;
+        }
+        return Math.max(0, Date.now() - lastActivityAt);
+    }
+    isCloudSlotLimitError(error) {
+        return (error instanceof errors_1.AppError &&
+            error.code === "BROWSER_CONNECTION_FAILED" &&
+            /max parallel cloud launches limit/i.test(error.message));
+    }
+    async pruneInactiveSessions(maxIdleMs = SessionManager.DEFAULT_PRUNE_IDLE_MS) {
+        const closedSessionIds = [];
+        for (const session of Array.from(this.sessions.values())) {
+            if (this.sessionIdleMs(session) < maxIdleMs) {
+                continue;
+            }
+            closedSessionIds.push(session.sessionId);
+            await this.destroySession(session);
+        }
+        return closedSessionIds;
+    }
+    async waitForCloudSlotRelease() {
+        await new Promise((resolve) => setTimeout(resolve, SessionManager.CLOUD_SLOT_RELEASE_WAIT_MS));
+    }
     async destroySession(session) {
         await (0, browser_1.closeSessionHandles)(session).catch(() => undefined);
         this.sessions.delete(session.sessionId);
@@ -116,6 +144,30 @@ class SessionManager {
             throw new errors_1.AppError("BAD_REQUEST", "--idle-timeout-ms must be a positive integer", 400);
         }
     }
+    async createSessionRecord(token, sessionId, profileId, request, createdAt, resolvedProxy, autoCreatedProfile) {
+        const connection = await (0, browser_1.connectToBrowser)(this.config, token, profileId);
+        const currentUrl = await (0, browser_1.navigatePage)(connection.page, request.url, this.config.navigationTimeoutMs);
+        const lastActivityAt = this.nowIso();
+        if (!resolvedProxy && profileId) {
+            resolvedProxy = await (0, browser_1.getCloudProfileProxy)(token, profileId).catch(() => undefined);
+        }
+        return {
+            sessionId,
+            profileId,
+            autoCreatedProfile,
+            connectUrl: connection.connectUrl,
+            browser: connection.browser,
+            context: connection.context,
+            page: connection.page,
+            currentUrl,
+            hasSnapshot: false,
+            staleSnapshot: false,
+            proxy: resolvedProxy,
+            createdAt,
+            lastActivityAt,
+            idleTimeoutMs: request.idleTimeoutMs
+        };
+    }
     async resolveTargetLocator(session, target) {
         if ((0, utils_1.isRefTarget)(target)) {
             const descriptor = this.refStore.get(session.sessionId, target);
@@ -142,6 +194,7 @@ class SessionManager {
     async open(request) {
         const token = this.requireToken();
         this.validateIdleTimeout(request.idleTimeoutMs);
+        await this.pruneInactiveSessions();
         if (request.profileId && request.proxy) {
             throw new errors_1.AppError("BAD_REQUEST", "proxy flags cannot be combined with --profile", 400);
         }
@@ -180,35 +233,36 @@ class SessionManager {
             autoCreatedProfile = true;
         }
         try {
-            const connection = await (0, browser_1.connectToBrowser)(this.config, token, profileId);
-            const currentUrl = await (0, browser_1.navigatePage)(connection.page, request.url, this.config.navigationTimeoutMs);
-            const lastActivityAt = this.nowIso();
-            if (!resolvedProxy && profileId) {
-                resolvedProxy = await (0, browser_1.getCloudProfileProxy)(token, profileId).catch(() => undefined);
+            let session;
+            try {
+                session = await this.createSessionRecord(token, sessionId, profileId, request, createdAt, resolvedProxy, autoCreatedProfile);
+            }
+            catch (error) {
+                if (!this.isCloudSlotLimitError(error)) {
+                    throw error;
+                }
+                if (this.sessions.size === 0) {
+                    throw new errors_1.AppError("BROWSER_CONNECTION_FAILED", `${error.message}. No tracked local sessions were available to close. Wait for cloud slots to free up or close stale sessions from another daemon, then retry.`, error.status, error.details);
+                }
+                const closedSessionIds = (await this.closeAll()).closedSessionIds;
+                await this.waitForCloudSlotRelease();
+                try {
+                    session = await this.createSessionRecord(token, sessionId, profileId, request, createdAt, resolvedProxy, autoCreatedProfile);
+                }
+                catch (retryError) {
+                    if (retryError instanceof errors_1.AppError && retryError.code === "BROWSER_CONNECTION_FAILED") {
+                        throw new errors_1.AppError(retryError.code, `${retryError.message}. Closed tracked sessions (${closedSessionIds.join(", ")}) and retried once, but the cloud slot was still unavailable.`, retryError.status, retryError.details);
+                    }
+                    throw retryError;
+                }
             }
-            const session = {
-                sessionId,
-                profileId,
-                autoCreatedProfile,
-                connectUrl: connection.connectUrl,
-                browser: connection.browser,
-                context: connection.context,
-                page: connection.page,
-                currentUrl,
-                hasSnapshot: false,
-                staleSnapshot: false,
-                proxy: resolvedProxy,
-                createdAt,
-                lastActivityAt,
-                idleTimeoutMs: request.idleTimeoutMs
-            };
             this.sessions.set(sessionId, session);
             this.activeSessionId = sessionId;
             this.refStore.clear(sessionId);
             return {
                 sessionId,
                 profileId,
-                url: currentUrl,
+                url: session.currentUrl,
                 proxy: session.proxy,
                 idleTimeoutMs: session.idleTimeoutMs
             };
@@ -436,10 +490,11 @@ class SessionManager {
                 value
             };
         }
-        if (!target) {
+        const resolvedTarget = target ?? (kind === "text" || kind === "html" ? "body" : undefined);
+        if (!resolvedTarget) {
             throw new errors_1.AppError("BAD_REQUEST", `get ${kind} requires a target`, 400);
         }
-        const locator = await this.resolveTargetLocator(session, target);
+        const locator = await this.resolveTargetLocator(session, resolvedTarget);
         const value = await (0, browser_1.readLocatorValue)(locator, kind, this.config.actionTimeoutMs);
         this.markSessionState(session);
         return {
@@ -673,12 +728,26 @@ class SessionManager {
     async currentSession() {
         return this.toSummary(await this.getSessionOrThrow());
     }
+    async pruneSessions(maxIdleMs = SessionManager.DEFAULT_PRUNE_IDLE_MS) {
+        const closedSessionIds = await this.pruneInactiveSessions(maxIdleMs);
+        return {
+            closed: closedSessionIds.length,
+            closedSessionIds,
+            maxIdleMs,
+        };
+    }
     async closeAll() {
+        const closedSessionIds = [];
         for (const session of Array.from(this.sessions.values())) {
+            closedSessionIds.push(session.sessionId);
             await this.destroySession(session);
         }
         this.sessions.clear();
         this.activeSessionId = undefined;
+        return {
+            closed: closedSessionIds.length,
+            closedSessionIds,
+        };
     }
 }
 exports.SessionManager = SessionManager;

package/dist/internal-agent/lib/utils.js

@@ -26,7 +26,7 @@ const errors_1 = require("./errors");
 function parseArgs(argv) {
     const positional = [];
     const flags = {};
-    const booleanFlags = new Set(["interactive", "exact", "annotate", "press-escape", "json", "clear"]);
+    const booleanFlags = new Set(["interactive", "exact", "annotate", "press-escape", "json", "clear", "all", "prune"]);
     for (let index = 0; index < argv.length; index += 1) {
         const token = argv[index];
         if (token === "-i") {

package/dist/lib/browserRead.js

@@ -43,7 +43,7 @@ function ensureBrowserCommandOk(step, response, url) {
         return;
     }
     const message = response.stderr.trim() || response.stdout.trim() || `Browser command failed for ${url}`;
-    throw new errors_1.CliError(`Browser read ${step} failed.`, 1, message);
+    throw (0, errors_1.createBrowserCommandError)(step, url, message);
 }
 function buildReadableExtractionExpression() {
     return `(() => {

package/dist/lib/browserStructured.js

@@ -30,7 +30,7 @@ function ensureBrowserCommandOk(step, response, url) {
         return;
     }
     const message = response.stderr.trim() || response.stdout.trim() || `Browser command failed for ${url}`;
-    throw new errors_1.CliError(`Browser structured extraction ${step} failed.`, 1, message);
+    throw (0, errors_1.createBrowserCommandError)(step, url, message);
 }
 function buildStructuredExtractionExpression() {
     return `(() => {

package/dist/lib/errors.js

@@ -1,15 +1,18 @@
 "use strict";
 Object.defineProperty(exports, "__esModule", { value: true });
 exports.DaemonError = exports.HttpError = exports.MissingCredentialError = exports.SilentExitError = exports.CliError = void 0;
+exports.createBrowserCommandError = createBrowserCommandError;
 exports.toCliError = toCliError;
 class CliError extends Error {
     exitCode;
     hint;
-    constructor(message, exitCode = 1, hint) {
+    code;
+    constructor(message, exitCode = 1, hint, code) {
         super(message);
         this.name = new.target.name;
         this.exitCode = exitCode;
         this.hint = hint;
+        this.code = code;
     }
 }
 exports.CliError = CliError;
@@ -23,7 +26,7 @@ class MissingCredentialError extends CliError {
     constructor(envName, commandGroup) {
         super(`Missing ${envName}. This is required for ${commandGroup}.`, 1, [
             "This CLI only reads credentials from environment variables or ~/.gologin-web-access/config.json.",
-            "Recommended setup: configure both GOLOGIN_WEB_UNLOCKER_API_KEY and GOLOGIN_CLOUD_TOKEN up front so agents do not stop to ask again.",
+            "Recommended setup: configure both GOLOGIN_WEB_UNLOCKER_API_KEY and GOLOGIN_TOKEN up front so agents do not stop to ask again.",
             `Set ${envName} in your environment or add it to ~/.gologin-web-access/config.json.`,
             "Helpful commands: gologin-web-access config init, gologin-web-access config show, gologin-web-access doctor.",
         ].join("\n"));
@@ -44,6 +47,31 @@ class DaemonError extends CliError {
     }
 }
 exports.DaemonError = DaemonError;
+function createBrowserCommandError(step, url, rawMessage) {
+    const normalized = rawMessage.trim() || `Browser command failed for ${url}`;
+    if (/max parallel cloud launches limit/i.test(normalized)) {
+        return new CliError(`Cloud Browser ${step} failed: max parallel cloud launches limit reached.`, 1, [
+            "Close stale cloud sessions, run sessions --prune, or switch to gologin-local-agent-browser if the task can run locally.",
+            normalized,
+        ].join("\n"), "CLOUD_SLOT_EXHAUSTED");
+    }
+    if (/BROWSER_CONNECTION_FAILED|connect ECONNREFUSED|connection failed/i.test(normalized)) {
+        return new CliError(`Cloud Browser ${step} failed: browser connection could not be established.`, 1, normalized, "BROWSER_CONNECTION_FAILED");
+    }
+    if (/(^|\D)403(\D|$)/.test(normalized)) {
+        return new CliError(`Cloud Browser ${step} failed with 403.`, 1, [
+            "Cloud Browser rejected the session. This can mean missing access, plan restrictions, or stale backend state.",
+            normalized,
+        ].join("\n"), "CLOUD_BROWSER_403");
+    }
+    if (/(^|\D)503(\D|$)/.test(normalized)) {
+        return new CliError(`Cloud Browser ${step} failed with 503.`, 1, [
+            "Cloud Browser is temporarily unavailable or overloaded. Retry shortly or switch to a local GoLogin profile if the task can run locally.",
+            normalized,
+        ].join("\n"), "CLOUD_BROWSER_503");
+    }
+    return new CliError(`Cloud Browser ${step} failed for ${url}.`, 1, normalized, "CLOUD_BROWSER_FAILED");
+}
 function toCliError(error) {
     if (error instanceof CliError) {
         return error;

package/dist/lib/output.js

@@ -22,6 +22,9 @@ function printError(error) {
     if (error.message) {
         write(process.stderr, error.message);
     }
+    if (error.code) {
+        write(process.stderr, `Code: ${error.code}`);
+    }
     if (error.hint) {
         write(process.stderr, error.hint);
     }

package/dist/lib/readSource.js

@@ -79,7 +79,7 @@ async function readRenderedHtmlContent(url, config, apiKey, options = {}) {
             renderSource: "unlocker",
             fallbackAttempted: true,
             fallbackUsed: false,
-            fallbackReason: `${assessment.reason}; GOLOGIN_CLOUD_TOKEN is not configured`,
+            fallbackReason: `${assessment.reason}; GOLOGIN_TOKEN is not configured`,
             request: unlocker.request,
         };
     }
@@ -180,7 +180,7 @@ async function readReadableContent(url, config, apiKey, options) {
             renderSource: "unlocker",
             fallbackAttempted: true,
             fallbackUsed: false,
-            fallbackReason: `${assessment.reason}; GOLOGIN_CLOUD_TOKEN is not configured`,
+            fallbackReason: `${assessment.reason}; GOLOGIN_TOKEN is not configured`,
             request: unlocker.request,
         };
     }

package/dist/lib/search.js

@@ -317,7 +317,7 @@ async function searchViaUnlocker(query, config, options, engine) {
 }
 async function searchViaBrowser(query, config, options, engine) {
     if (!config.cloudToken) {
-        throw new errors_1.CliError("Missing GOLOGIN_CLOUD_TOKEN for browser search fallback.");
+        throw new errors_1.CliError("Missing GOLOGIN_TOKEN for browser search fallback.");
     }
     const sessionId = `search-${(0, crypto_1.randomUUID)()}`;
     const searchUrl = buildSearchUrl(engine, query, options);

package/dist/lib/structuredScrape.js

@@ -4,9 +4,28 @@ exports.scrapeStructuredJson = scrapeStructuredJson;
 exports.makeStructuredScrapeEnvelope = makeStructuredScrapeEnvelope;
 exports.normalizeStructuredFallbackMode = normalizeStructuredFallbackMode;
 exports.shouldUseBrowserFallback = shouldUseBrowserFallback;
+exports.buildStructuredFallbackAdvisory = buildStructuredFallbackAdvisory;
+exports.detectStructuredBlockReason = detectStructuredBlockReason;
 const config_1 = require("../config");
+const errors_1 = require("./errors");
 const browserStructured_1 = require("./browserStructured");
 const unlocker_1 = require("./unlocker");
+class StructuredBlockedPageError extends errors_1.CliError {
+    status;
+    request;
+    constructor(url, status, request, reason, options) {
+        super(`Structured scrape returned a likely blocked or challenge page for ${url}.`, 1, [
+            `Reason: ${reason}.`,
+            options.fallbackAttempted
+                ? options.fallbackUsed
+                    ? "Browser fallback was used but the page still looked blocked."
+                    : `Browser fallback was attempted but not used. ${options.fallbackReason ?? "It did not improve the result."}`
+                : "Retry with --fallback browser, use read --source browser, or switch to gologin-local-agent-browser for full rendered DOM.",
+        ].join("\n"), "BLOCKED_PAGE");
+        this.status = status;
+        this.request = request;
+    }
+}
 async function scrapeStructuredJson(url, config, apiKey, options = {}) {
     const result = await (0, unlocker_1.scrapeJson)(url, apiKey, options.request);
     const fallbackMode = options.fallback ?? "none";
@@ -15,6 +34,7 @@ async function scrapeStructuredJson(url, config, apiKey, options = {}) {
     let fallbackAttempted = false;
     let fallbackUsed = false;
     let fallbackReason;
+    let { browserRecommended, warning } = buildStructuredFallbackAdvisory(data);
     if (fallbackMode === "browser" && shouldUseBrowserFallback(data)) {
         fallbackAttempted = true;
         (0, config_1.requireCloudToken)(config);
@@ -26,16 +46,28 @@ async function scrapeStructuredJson(url, config, apiKey, options = {}) {
             renderSource = "browser";
             fallbackUsed = true;
             fallbackReason = "unlocker structured data looked incomplete";
+            browserRecommended = false;
+            warning = undefined;
         }
         else {
             fallbackReason = "browser fallback did not improve structured output";
         }
     }
+    const blockedReason = detectStructuredBlockReason(data);
+    if (blockedReason) {
+        throw new StructuredBlockedPageError(url, result.status, result.request, blockedReason, {
+            fallbackAttempted,
+            fallbackUsed,
+            fallbackReason,
+        });
+    }
     return makeStructuredScrapeEnvelope(url, result, data, {
         renderSource,
         fallbackAttempted,
         fallbackUsed,
         fallbackReason,
+        browserRecommended,
+        warning,
     });
 }
 function makeStructuredScrapeEnvelope(url, result, data, options = {}) {
@@ -46,6 +78,8 @@ function makeStructuredScrapeEnvelope(url, result, data, options = {}) {
         fallbackAttempted: options.fallbackAttempted ?? false,
         fallbackUsed: options.fallbackUsed ?? false,
         fallbackReason: options.fallbackReason,
+        browserRecommended: options.browserRecommended,
+        warning: options.warning,
         request: result.request,
         data,
     };
@@ -60,16 +94,40 @@ function normalizeStructuredFallbackMode(value) {
     throw new Error(`Unsupported scrape-json fallback mode: ${value}`);
 }
 function shouldUseBrowserFallback(data) {
+    if (detectStructuredBlockReason(data)) {
+        return true;
+    }
     const firstH1 = data.headingsByLevel.h1[0];
     if (!firstH1) {
         return true;
     }
     return looksSuspiciousHeadingText(firstH1);
 }
+function buildStructuredFallbackAdvisory(data) {
+    const blockedReason = detectStructuredBlockReason(data);
+    if (blockedReason) {
+        return {
+            browserRecommended: true,
+            warning: `Structured output looks blocked or challenge-gated (${blockedReason}). Retry with --fallback browser or use a rendered browser path.`,
+        };
+    }
+    if (!shouldUseBrowserFallback(data)) {
+        return { browserRecommended: false };
+    }
+    return {
+        browserRecommended: true,
+        warning: "Structured output looks incomplete or client-rendered. Retry with --fallback browser or use read/open for rendered DOM.",
+    };
+}
 function looksSuspiciousHeadingText(value) {
     return /function\s*\(|window\.|document\.|const\s+|let\s+|var\s+|=>|import\s+/i.test(value) || value.length > 240;
 }
 function isBrowserDataBetter(current, candidate) {
+    const currentBlocked = Boolean(detectStructuredBlockReason(current));
+    const candidateBlocked = Boolean(detectStructuredBlockReason(candidate));
+    if (currentBlocked !== candidateBlocked) {
+        return currentBlocked && !candidateBlocked;
+    }
     if (candidate.headingsByLevel.h1.length > current.headingsByLevel.h1.length) {
         return true;
     }
@@ -81,3 +139,28 @@ function isBrowserDataBetter(current, candidate) {
     }
     return false;
 }
+function detectStructuredBlockReason(data) {
+    const candidates = [
+        data.title,
+        data.description,
+        ...data.headingsByLevel.h1.slice(0, 2),
+        ...data.headingsByLevel.h2.slice(0, 2),
+    ].filter((value) => Boolean(value && value.trim()));
+    for (const candidate of candidates) {
+        const reason = classifyBlockedText(candidate);
+        if (reason) {
+            return reason;
+        }
+    }
+    return undefined;
+}
+function classifyBlockedText(value) {
+    const text = value.trim();
+    if (/(verify you are human|verify you are a human|are you human|captcha|security check|attention required|just a moment|checking your browser|enable javascript and cookies to continue|one more step)/i.test(text)) {
+        return "challenge markers matched the page title or heading";
+    }
+    if (/(access denied|forbidden|blocked request|request blocked|request unsuccessful|temporarily blocked|temporarily unavailable|you have been blocked|access to this page has been denied)/i.test(text)) {
+        return "blocked-page markers matched the page title or heading";
+    }
+    return undefined;
+}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "gologin-web-access",
-  "version": "0.3.0",
+  "version": "0.3.1",
   "description": "Unified web access CLI for developers and AI agents to read and interact with the web using Gologin Web Unlocker and Cloud Browser.",
   "main": "dist/cli.js",
   "bin": {