app-manager-edge-worker 1.0.0

package/README.md

@@ -0,0 +1,168 @@
+# app-manager-edge-worker
+
+> **Distributed Scraper Edge Worker** for Apps Manager. Run on your local laptop, desktop, VPS, or Android phone (via Termux) to offload web scraping, visual element selection, and bypass Cloudflare WAF restrictions with zero proxy cost.
+
+[![npm version](https://img.shields.io/npm/v/app-manager-edge-worker.svg)](https://www.npmjs.com/package/app-manager-edge-worker)
+[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)
+[![Platform Support](https://img.shields.io/badge/Platform-Windows%20%7C%20macOS%20%7C%20Linux%20%7C%20Termux-brightgreen)](https://apps-manager.onrender.com)
+[![Hosted Platform](https://img.shields.io/badge/Hosted%20Platform-apps--manager.onrender.com-blue)](https://apps-manager.onrender.com)
+
+---
+
+## ✨ Why Distributed Edge Workers?
+
+Cloud datacenter IPs (Render, AWS, DigitalOcean, Hetzner) are frequently blocked or challenged by Cloudflare, Akamai, and anti-bot systems. **`app-manager-edge-worker`** turns your existing hardware into an autonomous residential scraping cluster:
+
+- 🌐 **True Residential IP Routing**: Requests originate directly from your home/office broadband or mobile data (Airtel, Jio, Comcast, etc.).
+- 🔒 **Zero Port Forwarding**: Persistent outbound WebSockets (`wss://.../workers`) effortlessly traverse home Wi-Fi routers, firewalls, and carrier-grade NAT (CGNAT).
+- ⚡ **Autonomous Chromium Engine**: Checks your system for Chrome/Edge or automatically installs stable Chromium without any manual intervention.
+- 📱 **Android Phone Support**: Transform an old spare Android phone into a 24/7 scraping powerhouse using Termux and Wake Lock.
+- 🚀 **Smart Single-Flight Deduplication**: Concurrent scrapes for the exact same target URL coalesce into a single execution flight.
+
+---
+
+## ⚡ Quick Start
+
+### Option 1: Run Instantly via NPX (Recommended)
+
+```bash
+npx app-manager-edge-worker
+```
+
+On first run, the CLI interactively asks for your **Worker Key**. Paste it from your Apps Manager dashboard, hit `Enter`, and you are immediately connected! Credentials are automatically saved to `~/.app-manager-edge-worker/config.json` for subsequent runs.
+
+---
+
+### Option 2: Run with Flags Directly
+
+```bash
+npx app-manager-edge-worker --key wkey_live_your_key_here --server https://apps-manager.onrender.com
+```
+
+---
+
+### Option 3: Global NPM Installation
+
+```bash
+npm install -g app-manager-edge-worker
+
+# Run directly
+app-manager-edge-worker
+```
+
+---
+
+## 📱 Turn Any Android Phone into a 24/7 Scraping Node (Termux)
+
+You can turn any old Android phone into an ultra-low-power, 24/7 residential scraping node using **Termux** (available free from F-Droid):
+
+```bash
+# 1. Update Termux packages & install Node.js + Chromium
+pkg update -y && pkg install -y nodejs-lts chromium
+
+# 2. Point Puppeteer to Termux's native Chromium ARM64 binary
+export PUPPETEER_EXECUTABLE_PATH=/data/data/com.termux/files/usr/bin/chromium-browser
+
+# 3. Launch the edge worker
+npx app-manager-edge-worker --server https://apps-manager.onrender.com --key <YOUR_WORKER_KEY>
+```
+
+> [!TIP]
+> In Termux, swipe down your Android notification shade and tap **"Acquire Wake Lock"**. Disable Android OS battery optimization for Termux so your phone runs uninterrupted in the background.
+
+---
+
+## 🔄 Production Background Service (PM2)
+
+To run the worker continuously as a daemon on Linux, macOS, or Windows:
+
+```bash
+# 1. Install PM2 and the worker CLI globally
+npm install -g pm2 app-manager-edge-worker
+
+# 2. Start the scraper worker as a managed process
+pm2 start app-manager-edge-worker --name "apps-worker" -- --server https://apps-manager.onrender.com --key <YOUR_WORKER_KEY>
+
+# 3. Configure auto-start on machine boot
+pm2 save
+pm2 startup
+```
+
+Useful PM2 commands:
+- Check status: `pm2 status`
+- View live stream logs: `pm2 logs apps-worker`
+- Restart worker: `pm2 restart apps-worker`
+
+---
+
+## 🐳 Docker Container
+
+Run in an isolated container on any VPS or local Docker host:
+
+```bash
+docker run -d \
+  --name app-manager-edge-worker \
+  --restart unless-stopped \
+  -e SERVER_URL="https://apps-manager.onrender.com" \
+  -e WORKER_KEY="<YOUR_WORKER_KEY>" \
+  node:20-slim sh -c "apt-get update && apt-get install -y chromium && npx app-manager-edge-worker"
+```
+
+---
+
+## 🌐 Autonomous Chromium Detection & Self-Installation
+
+When `app-manager-edge-worker` initializes, it executes a multi-layer verification routine to detect a usable Chromium executable:
+
+1. **Explicit Environment Variables**: Checks `PUPPETEER_EXECUTABLE_PATH`, `CHROME_BIN`, `CHROME_PATH`, `CHROMIUM_PATH`.
+2. **Pre-Installed System Browsers**: Scans standard OS locations for Google Chrome, Microsoft Edge, Brave, or Chromium, and checks system `PATH`.
+3. **Bundled/Cached Browsers**: Scans `~/.cache/puppeteer` using `@puppeteer/browsers`.
+4. **Android Termux Detection**: Automatically detects `/data/data/com.termux/files/usr/bin/chromium-browser`.
+
+### What happens if NO Chromium is found?
+The worker **automatically downloads and installs it** according to your operating system:
+
+| Operating System | Auto-Installation Action |
+|------------------|--------------------------|
+| **Windows** (x64/x86) | Downloads official stable Chrome for Testing from Google CDN with a progress bar into `~/.cache/puppeteer`. |
+| **macOS** (Intel / Apple Silicon M1-M4) | Detects system architecture (`mac` or `mac_arm`) and downloads matching stable Chrome from Google CDN. |
+| **Linux Desktop / VPS** (Ubuntu, Debian, Fedora, Arch) | Downloads official Linux x64/arm64 Chrome for Testing via `@puppeteer/browsers`. |
+| **Android (Termux)** | Automatically invokes Termux package manager (`pkg install -y chromium`). |
+| **Alpine Linux** | Automatically invokes Alpine package manager (`apk add --no-cache chromium`). |
+
+No manual setup required!
+
+---
+
+## ⚙️ CLI Options & Flags
+
+| Flag | Description | Default |
+|------|-------------|---------|
+| `-k, --key <key>` | Worker authorization key (`wkey_live_...`) | Prompted / Stored |
+| `-s, --server <url>` | Apps Manager Server URL | `https://apps-manager.onrender.com` |
+| `-c, --concurrency <n>` | Max concurrent Chromium browser tabs | `3` |
+| `-n, --name <name>` | Custom display name for this worker | Device hostname |
+| `--reset` | Clear stored credentials and prompt again | - |
+| `-h, --help` | Display help screen | - |
+
+---
+
+## 🔒 Security & Sandboxing
+
+- **Outbound Only**: All socket tunnels are initiated outbound to the central Apps Manager server over TLS (`wss://`). No public IP, dynamic DNS, or port forwarding required.
+- **Granular Scoping**: Workers can be dedicated to a specific application or organization team, or shared with the public pool.
+- **Incognito Sandboxing**: Every scraping job and visual selector preview is executed within an isolated browser context that is cleanly destroyed upon task completion.
+
+---
+
+## 👤 Author
+
+**Chetan Mohite**
+- GitHub: [@codes-by-chetan](https://github.com/codes-by-chetan)
+- Platform: [Apps Manager](https://apps-manager.onrender.com)
+
+---
+
+## 📄 License
+
+MIT © [Chetan Mohite](https://github.com/codes-by-chetan/Apps_Manager)

package/bin/worker.js

@@ -0,0 +1,16 @@
+#!/usr/bin/env node
+
+import { resolveWorkerConfig } from "../src/config.js";
+import { startWorkerClient } from "../src/client.js";
+
+async function main() {
+    try {
+        const config = await resolveWorkerConfig();
+        await startWorkerClient(config);
+    } catch (err) {
+        console.error("Fatal Worker Error:", err);
+        process.exit(1);
+    }
+}
+
+main();

package/package.json

@@ -0,0 +1,45 @@
+{
+  "name": "app-manager-edge-worker",
+  "version": "1.0.0",
+  "description": "Distributed Scraper Edge Worker for Apps Manager. Run on any local machine, VPS, or Termux Android device to serve browser and HTTP scraping jobs.",
+  "type": "module",
+  "bin": {
+    "app-manager-edge-worker": "./bin/worker.js",
+    "apps-manager-worker": "./bin/worker.js"
+  },
+  "main": "./src/client.js",
+  "scripts": {
+    "start": "node ./bin/worker.js"
+  },
+  "keywords": [
+    "app-manager-edge-worker",
+    "apps-manager",
+    "scraper",
+    "edge-worker",
+    "puppeteer",
+    "residential-proxy",
+    "distributed-scraping",
+    "termux"
+  ],
+  "author": "Chetan Mohite <chetanmohite2128@gmail.com> (https://github.com/codes-by-chetan)",
+  "license": "MIT",
+  "homepage": "https://apps-manager.onrender.com",
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/codes-by-chetan/Apps_Manager.git",
+    "directory": "packages/worker"
+  },
+  "publishConfig": {
+    "access": "public"
+  },
+  "dependencies": {
+    "@puppeteer/browsers": "^3.2.2",
+    "axios": "^1.8.1",
+    "cheerio": "^1.0.0",
+    "puppeteer": "^24.3.0",
+    "puppeteer-core": "^24.3.0",
+    "puppeteer-extra": "^3.3.6",
+    "puppeteer-extra-plugin-stealth": "^2.11.2",
+    "socket.io-client": "^4.8.1"
+  }
+}

package/src/banner.js

@@ -0,0 +1,55 @@
+// ANSI color helpers
+const colors = {
+    reset: "\x1b[0m",
+    bright: "\x1b[1m",
+    dim: "\x1b[2m",
+    cyan: "\x1b[36m",
+    green: "\x1b[32m",
+    yellow: "\x1b[33m",
+    red: "\x1b[31m",
+    magenta: "\x1b[35m",
+    gray: "\x1b[90m",
+};
+
+export function printBanner({ workerName, serverUrl, concurrency, platform, isTermux }) {
+    console.clear();
+    console.log(`${colors.cyan}${colors.bright}`);
+    console.log("   █████╗ ██████╗ ██████╗ ███████╗   ███╗   ███╗ █████╗ ███╗   ██╗ █████╗  ██████╗ ███████╗██████╗ ");
+    console.log("  ██╔══██╗██╔══██╗██╔══██╗██╔════╝   ████╗ ████║██╔══██╗████╗  ██║██╔══██╗██╔════╝ ██╔════╝██╔══██╗");
+    console.log("  ███████║██████╔╝██████╔╝███████╗   ██╔████╔██║███████║██╔██╗ ██║███████║██║  ███╗█████╗  ██████╔╝");
+    console.log("  ██╔══██║██╔═══╝ ██╔═══╝ ╚════██║   ██║╚██╔╝██║██╔══██║██║╚██╗██║██╔══██║██║   ██║██╔══╝  ██╔══██╗");
+    console.log("  ██║  ██║██║     ██║     ███████║██╗██║ ╚═╝ ██║██║  ██║██║ ╚████║██║  ██║╚██████╔╝███████╗██║  ██║");
+    console.log("  ╚═╝  ╚═╝╚═╝     ╚═╝     ╚══════╝╚═╝╚═╝     ╚═╝╚═╝  ╚═╝╚═╝  ╚═══╝╚═╝  ╚═╝ ╚═════╝ ╚══════╝╚═╝  ╚═╝");
+    console.log(`${colors.reset}`);
+    console.log(`${colors.bright}  🌐 Distributed Scraper Edge Worker Client v1.0.0${colors.reset}`);
+    console.log(`${colors.dim}  Zero-cost, persistent tunneling & residential edge scraping${colors.reset}`);
+    console.log("--------------------------------------------------------------------------------");
+    console.log(`  ${colors.bright}Device Name  :${colors.reset} ${colors.green}${workerName}${colors.reset}`);
+    console.log(`  ${colors.bright}Server URL   :${colors.reset} ${colors.cyan}${serverUrl}${colors.reset}`);
+    console.log(`  ${colors.bright}Concurrency  :${colors.reset} ${colors.yellow}${concurrency} concurrent tabs${colors.reset}`);
+    console.log(`  ${colors.bright}Environment  :${colors.reset} ${platform} ${isTermux ? "(Termux Android)" : ""}`);
+    console.log("--------------------------------------------------------------------------------\n");
+}
+
+export function logInfo(msg) {
+    console.log(`${colors.cyan}[INFO]${colors.reset} ${msg}`);
+}
+
+export function logSuccess(msg) {
+    console.log(`${colors.green}[SUCCESS]${colors.reset} ${msg}`);
+}
+
+export function logWarn(msg) {
+    console.log(`${colors.yellow}[WARN]${colors.reset} ${msg}`);
+}
+
+export function logError(msg) {
+    console.log(`${colors.red}[ERROR]${colors.reset} ${msg}`);
+}
+
+export function logJob(jobType, url, durationMs = null) {
+    const timeStr = durationMs ? ` ${colors.gray}(${durationMs}ms)${colors.reset}` : "";
+    console.log(
+        `${colors.magenta}[TASK]${colors.reset} ${colors.bright}${jobType}${colors.reset} -> ${url}${timeStr}`
+    );
+}

package/src/browser.js

@@ -0,0 +1,360 @@
+import puppeteer from "puppeteer-extra";
+import StealthPlugin from "puppeteer-extra-plugin-stealth";
+import fs from "fs";
+import path from "path";
+import os from "os";
+import { execSync } from "child_process";
+import {
+    Browser,
+    BrowserTag,
+    detectBrowserPlatform,
+    resolveBuildId,
+    install,
+    getInstalledBrowsers,
+} from "@puppeteer/browsers";
+import { logInfo, logSuccess, logWarn, logError } from "./banner.js";
+
+// Apply stealth plugin globally
+puppeteer.use(StealthPlugin());
+
+class LocalBrowserManager {
+    constructor() {
+        this.browser = null;
+        this.isLaunching = false;
+        this.launchPromise = null;
+        this.resolvedExecutablePath = null;
+    }
+
+    /**
+     * Detects if running inside Android Termux environment.
+     */
+    isTermux() {
+        return Boolean(
+            process.env.TERMUX_VERSION ||
+            (process.env.PREFIX && process.env.PREFIX.includes("com.termux"))
+        );
+    }
+
+    /**
+     * Checks if a given file path exists and is executable.
+     */
+    isValidExecutable(filePath) {
+        if (!filePath || typeof filePath !== "string") return false;
+        try {
+            if (!fs.existsSync(filePath)) return false;
+            const stat = fs.statSync(filePath);
+            if (!stat.isFile()) return false;
+            if (process.platform !== "win32") {
+                fs.accessSync(filePath, fs.constants.X_OK);
+            }
+            return true;
+        } catch {
+            return false;
+        }
+    }
+
+    /**
+     * Scans known system paths for pre-installed browsers (Chrome, Edge, Brave, Chromium).
+     */
+    findPreInstalledSystemBrowser() {
+        if (this.isTermux()) {
+            const termuxCandidates = [
+                "/data/data/com.termux/files/usr/bin/chromium-browser",
+                "/data/data/com.termux/files/usr/bin/chromium",
+            ];
+            for (const p of termuxCandidates) {
+                if (this.isValidExecutable(p)) return p;
+            }
+        }
+
+        const platform = os.platform();
+        const candidates = [];
+
+        if (platform === "win32") {
+            const progFiles = process.env["ProgramFiles"] || "C:\\Program Files";
+            const progFiles86 = process.env["ProgramFiles(x86)"] || "C:\\Program Files (x86)";
+            const localApp = process.env["LOCALAPPDATA"] || path.join(os.homedir(), "AppData", "Local");
+
+            candidates.push(
+                path.join(progFiles, "Google", "Chrome", "Application", "chrome.exe"),
+                path.join(progFiles86, "Google", "Chrome", "Application", "chrome.exe"),
+                path.join(localApp, "Google", "Chrome", "Application", "chrome.exe"),
+                path.join(progFiles86, "Microsoft", "Edge", "Application", "msedge.exe"),
+                path.join(progFiles, "Microsoft", "Edge", "Application", "msedge.exe"),
+                path.join(progFiles, "BraveSoftware", "Brave-Browser", "Application", "brave.exe")
+            );
+        } else if (platform === "darwin") {
+            candidates.push(
+                "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome",
+                "/Applications/Chromium.app/Contents/MacOS/Chromium",
+                "/Applications/Microsoft Edge.app/Contents/MacOS/Microsoft Edge",
+                "/Applications/Brave Browser.app/Contents/MacOS/Brave Browser"
+            );
+        } else if (platform === "linux") {
+            candidates.push(
+                "/usr/bin/google-chrome-stable",
+                "/usr/bin/google-chrome",
+                "/usr/bin/chromium-browser",
+                "/usr/bin/chromium",
+                "/snap/bin/chromium"
+            );
+        }
+
+        for (const p of candidates) {
+            if (this.isValidExecutable(p)) return p;
+        }
+
+        // System PATH lookup fallback
+        try {
+            const cmd = platform === "win32" ? "where chrome" : "which google-chrome || which chromium || which chromium-browser";
+            const output = execSync(cmd, { encoding: "utf-8", stdio: ["ignore", "pipe", "ignore"] }).trim();
+            const firstLine = output.split(/[\r\n]+/)[0]?.trim();
+            if (this.isValidExecutable(firstLine)) {
+                return firstLine;
+            }
+        } catch {
+            // Ignore which/where command failure
+        }
+
+        return null;
+    }
+
+    /**
+     * Checks if a bundled or cached Puppeteer Chromium installation exists.
+     */
+    async findCachedPuppeteerBrowser() {
+        // 1. Try Puppeteer's native resolution
+        try {
+            const nativePath = await puppeteer.executablePath();
+            if (this.isValidExecutable(nativePath)) {
+                return nativePath;
+            }
+        } catch {
+            // Native check failed, inspect cache directories
+        }
+
+        // 2. Inspect ~/.cache/puppeteer and local .cache/puppeteer
+        const cacheDirs = [
+            path.join(os.homedir(), ".cache", "puppeteer"),
+            path.resolve(".cache", "puppeteer"),
+            process.env.PUPPETEER_CACHE_DIR,
+        ].filter(Boolean);
+
+        for (const cacheDir of cacheDirs) {
+            if (!fs.existsSync(cacheDir)) continue;
+            try {
+                const installed = await getInstalledBrowsers({ cacheDir });
+                for (const b of installed) {
+                    if (this.isValidExecutable(b.executablePath)) {
+                        return b.executablePath;
+                    }
+                }
+            } catch {
+                // Ignore cache read errors
+            }
+        }
+
+        return null;
+    }
+
+    /**
+     * Automatically downloads or installs Chromium appropriate for the current OS.
+     * - Termux: runs `pkg install -y chromium`
+     * - Alpine Linux: runs `apk add --no-cache chromium`
+     * - Windows / macOS / Linux: downloads official Chrome via @puppeteer/browsers with terminal progress
+     */
+    async installBrowserForCurrentOS() {
+        const platform = os.platform();
+
+        // 1. Android Termux
+        if (this.isTermux()) {
+            logWarn("Termux Android environment detected without Chromium.");
+            logInfo("Attempting auto-install via Termux package manager (pkg install -y chromium)...");
+            try {
+                execSync("pkg update -y && pkg install -y chromium", { stdio: "inherit" });
+                const termuxPaths = [
+                    "/data/data/com.termux/files/usr/bin/chromium-browser",
+                    "/data/data/com.termux/files/usr/bin/chromium",
+                ];
+                for (const p of termuxPaths) {
+                    if (this.isValidExecutable(p)) {
+                        logSuccess(`Termux Chromium installed successfully at: ${p}`);
+                        return p;
+                    }
+                }
+            } catch (err) {
+                logError(`Termux auto-install failed: ${err.message}`);
+                logWarn("Please run manually in Termux: pkg install chromium -y");
+            }
+        }
+
+        // 2. Alpine Linux
+        if (fs.existsSync("/etc/alpine-release")) {
+            logWarn("Alpine Linux detected without Chromium.");
+            logInfo("Attempting auto-install via apk add --no-cache chromium...");
+            try {
+                execSync("apk add --no-cache chromium", { stdio: "inherit" });
+                if (this.isValidExecutable("/usr/bin/chromium-browser")) {
+                    logSuccess("Alpine Chromium installed successfully!");
+                    return "/usr/bin/chromium-browser";
+                }
+            } catch (err) {
+                logError(`Alpine apk install failed: ${err.message}`);
+            }
+        }
+
+        // 3. Desktop / Standard Linux (Windows, macOS, Linux x64/arm64) via @puppeteer/browsers
+        try {
+            const browserPlatform = detectBrowserPlatform();
+            if (!browserPlatform) {
+                throw new Error(`Unsupported OS platform for automated Chrome download: ${platform} ${os.arch()}`);
+            }
+
+            logWarn("No system or cached Chromium found on this machine.");
+            logInfo(`Resolving latest stable Chrome build for ${browserPlatform}...`);
+
+            const buildId = await resolveBuildId(Browser.CHROME, browserPlatform, BrowserTag.STABLE);
+            const cacheDir = path.join(os.homedir(), ".cache", "puppeteer");
+
+            logInfo(`Downloading Chrome v${buildId} to ${cacheDir}...`);
+
+            let lastPercent = -1;
+            const installed = await install({
+                browser: Browser.CHROME,
+                platform: browserPlatform,
+                buildId,
+                cacheDir,
+                downloadProgressCallback: (downloadedBytes, totalBytes) => {
+                    if (totalBytes > 0) {
+                        const percent = Math.round((downloadedBytes / totalBytes) * 100);
+                        if (percent % 10 === 0 && percent !== lastPercent) {
+                            lastPercent = percent;
+                            process.stdout.write(
+                                `\r[DOWNLOAD] Chrome: ${percent}% (${(downloadedBytes / 1024 / 1024).toFixed(1)}MB / ${(totalBytes / 1024 / 1024).toFixed(1)}MB)`
+                            );
+                        }
+                    }
+                },
+            });
+
+            console.log(""); // New line after progress finishes
+            logSuccess(`Chrome installed and verified at: ${installed.executablePath}`);
+            return installed.executablePath;
+        } catch (err) {
+            logError(`Automated Chrome download failed: ${err.message}`);
+            throw new Error(`Failed to find or install Chromium executable on ${platform}: ${err.message}`);
+        }
+    }
+
+    /**
+     * Resolves the Chromium executable path or auto-installs if missing.
+     */
+    async ensureChromiumExecutable() {
+        if (this.resolvedExecutablePath && this.isValidExecutable(this.resolvedExecutablePath)) {
+            return this.resolvedExecutablePath;
+        }
+
+        // 1. Environment Variable check
+        const envCandidates = [
+            process.env.PUPPETEER_EXECUTABLE_PATH,
+            process.env.CHROME_BIN,
+            process.env.CHROME_PATH,
+            process.env.CHROMIUM_PATH,
+        ];
+        for (const p of envCandidates) {
+            if (this.isValidExecutable(p)) {
+                logInfo(`Using Chromium from environment variable: ${p}`);
+                this.resolvedExecutablePath = p;
+                return p;
+            }
+        }
+
+        // 2. Bundled / Cached Puppeteer Browser check
+        const cachedBrowser = await this.findCachedPuppeteerBrowser();
+        if (cachedBrowser) {
+            this.resolvedExecutablePath = cachedBrowser;
+            return cachedBrowser;
+        }
+
+        // 3. Pre-installed System Browser check (Chrome, Edge, Brave, etc.)
+        const systemBrowser = this.findPreInstalledSystemBrowser();
+        if (systemBrowser) {
+            logInfo(`Found system browser at: ${systemBrowser}`);
+            this.resolvedExecutablePath = systemBrowser;
+            return systemBrowser;
+        }
+
+        // 4. If neither bundled nor system browser exists, auto-install for this OS!
+        const newlyInstalledPath = await this.installBrowserForCurrentOS();
+        this.resolvedExecutablePath = newlyInstalledPath;
+        return newlyInstalledPath;
+    }
+
+    /**
+     * Launches or returns existing healthy Puppeteer browser instance.
+     */
+    async getBrowser() {
+        if (this.browser && this.browser.connected) {
+            return this.browser;
+        }
+
+        if (this.isLaunching && this.launchPromise) {
+            return this.launchPromise;
+        }
+
+        this.isLaunching = true;
+        this.launchPromise = (async () => {
+            try {
+                // Ensure executable path is resolved or auto-installed
+                const executablePath = await this.ensureChromiumExecutable();
+
+                const launchArgs = [
+                    "--no-sandbox",
+                    "--disable-setuid-sandbox",
+                    "--disable-dev-shm-usage",
+                    "--disable-accelerated-2d-canvas",
+                    "--no-first-run",
+                    "--no-zygote",
+                    "--disable-gpu",
+                    "--window-size=1366,900",
+                ];
+
+                const launchOptions = {
+                    headless: "new",
+                    args: launchArgs,
+                    executablePath,
+                };
+
+                this.browser = await puppeteer.launch(launchOptions);
+
+                this.browser.on("disconnected", () => {
+                    this.browser = null;
+                });
+
+                return this.browser;
+            } finally {
+                this.isLaunching = false;
+                this.launchPromise = null;
+            }
+        })();
+
+        return this.launchPromise;
+    }
+
+    /**
+     * Safely closes browser if running.
+     */
+    async close() {
+        if (this.browser) {
+            try {
+                await this.browser.close();
+            } catch {
+                // Ignore error on shutdown
+            }
+            this.browser = null;
+        }
+    }
+}
+
+const localBrowserManager = new LocalBrowserManager();
+export default localBrowserManager;

package/src/client.js

@@ -0,0 +1,149 @@
+import { io } from "socket.io-client";
+import os from "os";
+import path from "path";
+import localBrowserManager from "./browser.js";
+import { handleChromiumScrape, handleHttpScrape } from "./handlers/scrapeHandler.js";
+import { handlePreviewPage, handleInspectElement } from "./handlers/previewHandler.js";
+import { printBanner, logInfo, logSuccess, logWarn, logError, logJob } from "./banner.js";
+
+export async function startWorkerClient(config) {
+    const { workerKey, serverUrl, concurrency, workerName } = config;
+    const isTermux = localBrowserManager.isTermux();
+
+    printBanner({
+        workerName,
+        serverUrl,
+        concurrency,
+        platform: `${os.platform()} ${os.arch()}`,
+        isTermux,
+    });
+
+    logInfo("Checking Chromium browser availability...");
+    let chromiumExecutable = null;
+    try {
+        chromiumExecutable = await localBrowserManager.ensureChromiumExecutable();
+        logSuccess(`Chromium ready: ${chromiumExecutable}`);
+    } catch (browserErr) {
+        logWarn(`Chromium initialization warning: ${browserErr.message}`);
+    }
+
+    logInfo("Connecting to Apps Manager cluster...");
+
+    const socket = io(`${serverUrl}/workers`, {
+        transports: ["websocket", "polling"],
+        reconnection: true,
+        reconnectionAttempts: Infinity,
+        reconnectionDelay: 1000,
+        reconnectionDelayMax: 5000,
+        auth: {
+            workerKey,
+            metadata: {
+                hostname: workerName || os.hostname(),
+                platform: os.platform(),
+                arch: os.arch(),
+                osVersion: os.release(),
+                nodeVersion: process.version,
+                chromiumVersion: chromiumExecutable ? path.basename(chromiumExecutable) : "",
+                chromiumPath: chromiumExecutable || "",
+                isTermux,
+            },
+        },
+    });
+
+    let activeJobs = 0;
+
+    socket.on("connect", () => {
+        logSuccess(`Connected to cluster! (Socket ID: ${socket.id})`);
+        logInfo(`Ready to accept scraping and visual preview jobs (Concurrency: ${concurrency}).\n`);
+    });
+
+    socket.on("connect_error", (err) => {
+        if (err.message && err.message.includes("Authentication failed")) {
+            logError(`Authentication Failed: ${err.message}`);
+            logWarn("Please verify your worker key or run with '--reset' to enter a new key.");
+        } else {
+            logWarn(`Connection failed (${err.message}). Retrying in a few seconds...`);
+        }
+    });
+
+    socket.on("disconnect", (reason) => {
+        logWarn(`Disconnected from cluster: ${reason}`);
+    });
+
+    // Handle incoming job dispatch from server
+    socket.on("job:dispatch", async (job) => {
+        const jobId = job.jobId;
+        const jobType = job.jobType || job.type;
+        const payload = job.payload || {};
+        const targetUrl = payload.url || "(no url)";
+
+        logJob(jobType, targetUrl);
+        activeJobs++;
+
+        const startTime = Date.now();
+
+        try {
+            let resultData = null;
+
+            if (jobType === "SCRAPE_CHROMIUM") {
+                resultData = await handleChromiumScrape(payload);
+            } else if (jobType === "SCRAPE_HTTP") {
+                resultData = await handleHttpScrape(payload);
+            } else if (jobType === "PREVIEW_PAGE") {
+                resultData = await handlePreviewPage(payload);
+            } else if (jobType === "INSPECT_ELEMENT") {
+                resultData = await handleInspectElement(payload);
+            } else {
+                throw new Error(`Unsupported job type: ${jobType}`);
+            }
+
+            const durationMs = Date.now() - startTime;
+            logSuccess(`Completed ${jobType} for ${targetUrl} in ${durationMs}ms`);
+
+            socket.emit("job:result", {
+                jobId,
+                success: true,
+                data: resultData,
+                durationMs,
+            });
+        } catch (err) {
+            const durationMs = Date.now() - startTime;
+            logError(`Failed ${jobType} for ${targetUrl}: ${err.message}`);
+
+            socket.emit("job:result", {
+                jobId,
+                success: false,
+                error: {
+                    message: err.message,
+                    code: err.code || "JOB_EXECUTION_ERROR",
+                },
+                durationMs,
+            });
+        } finally {
+            activeJobs = Math.max(0, activeJobs - 1);
+        }
+    });
+
+    // Handle cluster heartbeat ping
+    socket.on("ping", () => {
+        socket.emit("pong", {
+            activeJobs,
+            memory: process.memoryUsage(),
+            uptime: process.uptime(),
+        });
+    });
+
+    // Graceful shutdown handling
+    const shutdown = async () => {
+        logInfo("\nGracefully shutting down edge worker...");
+        socket.disconnect();
+        await localBrowserManager.close();
+        logSuccess("Edge worker stopped cleanly.");
+        process.exit(0);
+    };
+
+    process.on("SIGINT", shutdown);
+    process.on("SIGTERM", shutdown);
+
+    return socket;
+}

package/src/config.js

@@ -0,0 +1,201 @@
+import fs from "fs";
+import path from "path";
+import os from "os";
+import readline from "readline";
+
+const LEGACY_CONFIG_DIR = path.join(os.homedir(), ".apps-manager-worker");
+const NEW_CONFIG_DIR = path.join(os.homedir(), ".app-manager-edge-worker");
+const CONFIG_DIR = fs.existsSync(NEW_CONFIG_DIR)
+    ? NEW_CONFIG_DIR
+    : fs.existsSync(LEGACY_CONFIG_DIR)
+    ? LEGACY_CONFIG_DIR
+    : NEW_CONFIG_DIR;
+const CONFIG_FILE = path.join(CONFIG_DIR, "config.json");
+
+/**
+ * Ensures the config directory exists.
+ */
+function ensureConfigDir() {
+    if (!fs.existsSync(CONFIG_DIR)) {
+        fs.mkdirSync(CONFIG_DIR, { recursive: true });
+    }
+}
+
+/**
+ * Loads stored configuration from disk.
+ */
+export function loadStoredConfig() {
+    try {
+        if (fs.existsSync(CONFIG_FILE)) {
+            const raw = fs.readFileSync(CONFIG_FILE, "utf-8");
+            return JSON.parse(raw);
+        }
+    } catch {
+        // Fall back to empty object on read/parse error
+    }
+    return {};
+}
+
+/**
+ * Saves configuration to disk.
+ */
+export function saveStoredConfig(cfg) {
+    try {
+        ensureConfigDir();
+        fs.writeFileSync(CONFIG_FILE, JSON.stringify(cfg, null, 2), "utf-8");
+    } catch (err) {
+        console.warn(`[Config] Failed to save config to ${CONFIG_FILE}:`, err.message);
+    }
+}
+
+/**
+ * Parses command-line arguments.
+ */
+export function parseCliArgs() {
+    const args = process.argv.slice(2);
+    const parsed = {};
+
+    for (let i = 0; i < args.length; i++) {
+        const arg = args[i];
+        if (arg === "--key" || arg === "-k") {
+            parsed.workerKey = args[++i];
+        } else if (arg === "--server" || arg === "-s") {
+            parsed.serverUrl = args[++i];
+        } else if (arg === "--concurrency" || arg === "-c") {
+            parsed.concurrency = parseInt(args[++i], 10);
+        } else if (arg === "--name" || arg === "-n") {
+            parsed.name = args[++i];
+        } else if (arg === "--reset") {
+            parsed.reset = true;
+        } else if (arg === "--help" || arg === "-h") {
+            parsed.help = true;
+        }
+    }
+
+    return parsed;
+}
+
+/**
+ * Prompts the user interactively in the terminal via native readline.
+ * @param {string} query
+ * @returns {Promise<string>}
+ */
+function promptUser(query) {
+    const rl = readline.createInterface({
+        input: process.stdin,
+        output: process.stdout,
+    });
+
+    return new Promise((resolve) => {
+        rl.question(query, (answer) => {
+            rl.close();
+            resolve(answer.trim());
+        });
+    });
+}
+
+/**
+ * Resolves the active configuration, prompting interactively if workerKey is missing.
+ */
+export async function resolveWorkerConfig() {
+    const cli = parseCliArgs();
+
+    if (cli.help) {
+        console.log(`
+Apps Manager Scraper Edge Worker CLI
+
+Usage:
+  npx app-manager-edge-worker [options]
+
+Options:
+  -k, --key <workerKey>       Your worker secret key (wkey_live_...)
+  -s, --server <url>          Apps Manager Server URL (e.g. https://apps-manager.onrender.com)
+  -c, --concurrency <number>  Maximum concurrent browser tabs (default: 3)
+  -n, --name <name>           Custom display name for this worker device
+      --reset                 Clear stored credentials and prompt again
+  -h, --help                  Display this help message
+        `);
+        process.exit(0);
+    }
+
+    if (cli.reset) {
+        if (fs.existsSync(CONFIG_FILE)) {
+            fs.unlinkSync(CONFIG_FILE);
+            console.log(`[Config] Cleared saved credentials in ${CONFIG_FILE}`);
+        }
+    }
+
+    const stored = loadStoredConfig();
+
+    // Priority: CLI > Env > Stored > Default
+    let workerKey =
+        cli.workerKey ||
+        process.env.APPS_MANAGER_WORKER_KEY ||
+        process.env.WORKER_KEY ||
+        stored.workerKey;
+
+    let serverUrl =
+        cli.serverUrl ||
+        process.env.APPS_MANAGER_SERVER_URL ||
+        process.env.SERVER_URL ||
+        stored.serverUrl ||
+        "https://apps-manager.onrender.com";
+
+    const concurrency =
+        cli.concurrency ||
+        parseInt(process.env.CONCURRENCY_LIMIT || "3", 10) ||
+        stored.concurrency ||
+        3;
+
+    const workerName =
+        cli.name ||
+        process.env.WORKER_NAME ||
+        stored.workerName ||
+        `${os.hostname()} (${os.platform()})`;
+
+    // Interactive prompt if workerKey is still missing
+    if (!workerKey) {
+        console.log("\n==================================================");
+        console.log("🚀 Apps Manager Edge Worker Setup");
+        console.log("==================================================");
+        console.log("No worker key found on this device.\n");
+        console.log("To get a Worker Key:");
+        console.log("1. Go to your Apps Manager Dashboard (Admin or App Detail > Workers)");
+        console.log("2. Click 'Register New Worker'");
+        console.log("3. Copy the generated worker key\n");
+
+        workerKey = await promptUser("🔑 Paste your Worker Key (wkey_live_...): ");
+
+        if (!workerKey) {
+            console.error("❌ Error: Worker key is required to connect to the cluster.");
+            process.exit(1);
+        }
+
+        // Also prompt for server URL if default might not be what they want (e.g., local dev)
+        const customServer = await promptUser(
+            `🌐 Server URL (Press Enter for default: ${serverUrl}): `
+        );
+        if (customServer) {
+            serverUrl = customServer.replace(/\/+$/, "");
+        }
+
+        // Save for subsequent runs
+        saveStoredConfig({
+            workerKey,
+            serverUrl,
+            concurrency,
+            workerName,
+        });
+        console.log(`\n✅ Saved configuration to: ${CONFIG_FILE}\n`);
+    }
+
+    // Clean up trailing slash
+    serverUrl = serverUrl.replace(/\/+$/, "");
+
+    return {
+        workerKey: workerKey.trim(),
+        serverUrl,
+        concurrency,
+        workerName,
+    };
+}

package/src/handlers/previewHandler.js

@@ -0,0 +1,176 @@
+import localBrowserManager from "../browser.js";
+
+/**
+ * Handles PREVIEW_PAGE task: captures full-page screenshot and dimensions on the edge worker.
+ */
+export async function handlePreviewPage(payload) {
+    const { url, timeout = 25000 } = payload;
+    const browser = await localBrowserManager.getBrowser();
+    const page = await browser.newPage();
+
+    try {
+        await page.setViewport({ width: 1366, height: 900, deviceScaleFactor: 1 });
+
+        await page.goto(url, { waitUntil: "domcontentloaded", timeout });
+        await new Promise((r) => setTimeout(r, 1500));
+
+        const title = (await page.title()) || new URL(url).hostname;
+        const currentUrl = page.url();
+
+        const dimensions = await page.evaluate(() => ({
+            width: document.documentElement.scrollWidth || 1366,
+            height: document.documentElement.scrollHeight || 900,
+        }));
+
+        // Hide scrollbars before screenshot
+        await page
+            .addStyleTag({
+                content: `
+                ::-webkit-scrollbar { display: none !important; width: 0 !important; height: 0 !important; }
+                html, body, * { scrollbar-width: none !important; -ms-overflow-style: none !important; }
+            `,
+            })
+            .catch(() => {});
+
+        const screenshotBuffer = await page.screenshot({
+            encoding: "binary",
+            type: "jpeg",
+            quality: 85,
+            fullPage: true,
+        });
+
+        const screenshotBase64 = Buffer.from(screenshotBuffer).toString("base64");
+
+        return {
+            title,
+            url: currentUrl,
+            dimensions,
+            screenshotBase64: `data:image/jpeg;base64,${screenshotBase64}`,
+        };
+    } finally {
+        await page.close().catch(() => {});
+    }
+}
+
+/**
+ * Handles INSPECT_ELEMENT task: resolves DOM element coordinates or evaluates candidate selectors on the edge worker.
+ */
+export async function handleInspectElement(payload) {
+    const { url, selector, x, y, timeout = 20000 } = payload;
+    const browser = await localBrowserManager.getBrowser();
+    const page = await browser.newPage();
+
+    try {
+        await page.setViewport({ width: 1366, height: 900, deviceScaleFactor: 1 });
+        await page.goto(url, { waitUntil: "domcontentloaded", timeout });
+        await new Promise((r) => setTimeout(r, 1000));
+
+        if (x !== undefined && y !== undefined) {
+            const elementInfo = await page.evaluate((px, py) => {
+                const scrollY = Math.max(0, py - 250);
+                const scrollX = Math.max(0, px - 250);
+                window.scrollTo(scrollX, scrollY);
+
+                const clientX = px - window.scrollX;
+                const clientY = py - window.scrollY;
+
+                const el = document.elementFromPoint(clientX, clientY);
+                if (!el) return null;
+
+                const rect = el.getBoundingClientRect();
+                const boundingRect = {
+                    left: Math.round(rect.left + window.scrollX),
+                    top: Math.round(rect.top + window.scrollY),
+                    width: Math.round(rect.width),
+                    height: Math.round(rect.height),
+                };
+
+                const candidates = [];
+                if (el.id) {
+                    candidates.push({ type: "css", value: `#${el.id}`, attribute: "text" });
+                }
+
+                for (const attr of el.attributes) {
+                    if (attr.name.startsWith("data-") && attr.value) {
+                        candidates.push({
+                            type: "css",
+                            value: `[${attr.name}='${attr.value}']`,
+                            attribute: "text",
+                        });
+                    }
+                }
+
+                if (el.className && typeof el.className === "string") {
+                    const classes = el.className
+                        .split(/\s+/)
+                        .filter((c) => c && !c.includes(":") && !c.includes("/") && !c.includes("["));
+                    if (classes.length > 0) {
+                        candidates.push({
+                            type: "css",
+                            value: `${el.tagName.toLowerCase()}.${classes.slice(0, 2).join(".")}`,
+                            attribute: "text",
+                        });
+                    }
+                }
+
+                const tagName = el.tagName.toLowerCase();
+                candidates.push({ type: "css", value: tagName, attribute: "text" });
+
+                const text = el.innerText ? el.innerText.trim() : el.textContent?.trim() || "";
+
+                return {
+                    found: true,
+                    tagName,
+                    text,
+                    attributes: Array.from(el.attributes).reduce((acc, a) => {
+                        acc[a.name] = a.value;
+                        return acc;
+                    }, {}),
+                    candidates,
+                    boundingRect,
+                };
+            }, x, y);
+
+            return elementInfo || { found: false, message: "No element found at specified coordinates." };
+        }
+
+        if (selector) {
+            const testResult = await page.evaluate((sel) => {
+                try {
+                    const el = document.querySelector(sel);
+                    if (!el) return { found: false, count: 0 };
+
+                    const rect = el.getBoundingClientRect();
+                    const boundingRect = {
+                        left: Math.round(rect.left + window.scrollX),
+                        top: Math.round(rect.top + window.scrollY),
+                        width: Math.round(rect.width),
+                        height: Math.round(rect.height),
+                    };
+
+                    const count = document.querySelectorAll(sel).length;
+
+                    return {
+                        found: true,
+                        count,
+                        tagName: el.tagName.toLowerCase(),
+                        text: el.innerText ? el.innerText.trim() : el.textContent?.trim() || "",
+                        attributes: Array.from(el.attributes).reduce((acc, a) => {
+                            acc[a.name] = a.value;
+                            return acc;
+                        }, {}),
+                        boundingRect,
+                    };
+                } catch (e) {
+                    return { found: false, error: e.message };
+                }
+            }, selector);
+
+            return testResult;
+        }
+
+        return { message: "Specify coordinates (x, y) or selector to inspect." };
+    } finally {
+        await page.close().catch(() => {});
+    }
+}

package/src/handlers/scrapeHandler.js

@@ -0,0 +1,190 @@
+import axios from "axios";
+import * as cheerio from "cheerio";
+import localBrowserManager from "../browser.js";
+
+/**
+ * Handles SCRAPE_CHROMIUM task via local stealth Puppeteer.
+ */
+export async function handleChromiumScrape(payload) {
+    const startTime = Date.now();
+    const {
+        url,
+        timeout = 30000,
+        waitForSelector = null,
+        waitForTimeout = 1000,
+        actions = [],
+        extractSelectors = [],
+        headers = {},
+        cookies = [],
+        viewport = { width: 1366, height: 900 },
+    } = payload;
+
+    const browser = await localBrowserManager.getBrowser();
+    const page = await browser.newPage();
+
+    try {
+        await page.setViewport({
+            width: viewport.width || 1366,
+            height: viewport.height || 900,
+            deviceScaleFactor: 1,
+        });
+
+        if (Object.keys(headers).length > 0) {
+            await page.setExtraHTTPHeaders(headers);
+        }
+
+        if (Array.isArray(cookies) && cookies.length > 0) {
+            await page.setCookie(...cookies);
+        }
+
+        // Navigate
+        const response = await page.goto(url, {
+            waitUntil: "domcontentloaded",
+            timeout,
+        });
+
+        const status = response ? response.status() : 200;
+
+        // Custom selector wait
+        if (waitForSelector) {
+            try {
+                await page.waitForSelector(waitForSelector, { timeout: 10000 });
+            } catch {
+                // Ignore timeout on optional selector wait
+            }
+        }
+
+        // Brief delay for dynamic content
+        if (waitForTimeout > 0) {
+            await new Promise((r) => setTimeout(r, Math.min(waitForTimeout, 10000)));
+        }
+
+        // Execute sequential interactive actions if configured
+        if (Array.isArray(actions) && actions.length > 0) {
+            for (const action of actions) {
+                try {
+                    if (action.type === "click" && action.selector) {
+                        await page.click(action.selector);
+                    } else if (action.type === "type" && action.selector && action.text) {
+                        await page.type(action.selector, action.text);
+                    } else if (action.type === "scroll") {
+                        await page.evaluate((y) => window.scrollBy(0, y || 500), action.distance);
+                    } else if (action.type === "wait") {
+                        await new Promise((r) => setTimeout(r, action.duration || 1000));
+                    }
+                } catch (actionErr) {
+                    console.warn(`[ScrapeHandler] Action ${action.type} failed:`, actionErr.message);
+                }
+            }
+        }
+
+        const pageTitle = (await page.title()) || "";
+        const finalUrl = page.url();
+        const html = await page.content();
+
+        // Extract structured data if extraction rules are defined
+        const extractedData = {};
+        if (Array.isArray(extractSelectors) && extractSelectors.length > 0) {
+            for (const item of extractSelectors) {
+                const { field, selector, type = "text", attribute = null } = item;
+                if (!field || !selector) continue;
+
+                try {
+                    const values = await page.evaluate(
+                        (sel, t, attr) => {
+                            const elements = Array.from(document.querySelectorAll(sel));
+                            return elements.map((el) => {
+                                if (t === "attribute" && attr) {
+                                    return el.getAttribute(attr) || "";
+                                }
+                                if (t === "html") {
+                                    return el.innerHTML;
+                                }
+                                return el.innerText ? el.innerText.trim() : el.textContent?.trim() || "";
+                            });
+                        },
+                        selector,
+                        type,
+                        attribute
+                    );
+
+                    extractedData[field] = values.length <= 1 ? (values[0] ?? null) : values;
+                } catch {
+                    extractedData[field] = null;
+                }
+            }
+        }
+
+        const executionTimeMs = Date.now() - startTime;
+
+        return {
+            status,
+            title: pageTitle,
+            url: finalUrl,
+            html,
+            extractedData,
+            executionTimeMs,
+        };
+    } finally {
+        await page.close().catch(() => {});
+    }
+}
+
+/**
+ * Handles fast SCRAPE_HTTP task via Cheerio.
+ */
+export async function handleHttpScrape(payload) {
+    const startTime = Date.now();
+    const { url, timeout = 15000, headers = {}, extractSelectors = [] } = payload;
+
+    const response = await axios.get(url, {
+        timeout,
+        headers: {
+            "User-Agent":
+                "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36",
+            Accept: "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
+            "Accept-Language": "en-US,en;q=0.9",
+            ...headers,
+        },
+    });
+
+    const html = typeof response.data === "string" ? response.data : "";
+    const $ = cheerio.load(html);
+    const title = $("title").first().text().trim() || "";
+
+    const extractedData = {};
+    if (Array.isArray(extractSelectors) && extractSelectors.length > 0) {
+        for (const item of extractSelectors) {
+            const { field, selector, type = "text", attribute = null } = item;
+            if (!field || !selector) continue;
+
+            try {
+                const values = [];
+                $(selector).each((_, el) => {
+                    if (type === "attribute" && attribute) {
+                        values.push($(el).attr(attribute) || "");
+                    } else if (type === "html") {
+                        values.push($(el).html() || "");
+                    } else {
+                        values.push($(el).text().trim());
+                    }
+                });
+
+                extractedData[field] = values.length <= 1 ? (values[0] ?? null) : values;
+            } catch {
+                extractedData[field] = null;
+            }
+        }
+    }
+
+    const executionTimeMs = Date.now() - startTime;
+
+    return {
+        status: response.status,
+        title,
+        url,
+        html,
+        extractedData,
+        executionTimeMs,
+    };
+}