@romanmatena/browsermonitor 2.0.1 → 2.1.1

package/README.md

@@ -58,9 +58,9 @@ pnpm add -g @romanmatena/browsermonitor
 
 **Note:** Chromium download is skipped — browsermonitor uses your system Chrome/Chromium. No extra 300 MB download.
 
-**First run:** When you run `browsermonitor` in a project directory for the first time, it auto-creates `.browsermonitor/` with `settings.json` and optionally updates agent files (`CLAUDE.md`, `AGENTS.md`, `memory.md`).
+**First run (interactive):** When you run `browsermonitor` for the first time in a project directory, interactive mode asks for HTTP API port, saves `settings.json`, and updates agent files (`CLAUDE.md`, `AGENTS.md`, `memory.md`). When you press `o` (open), it asks for the default URL.
 
-**Manual setup:** Run `browsermonitor init` to re-run the setup wizard.
+**Re-init:** Run `browsermonitor init` to recreate settings and update agent files.
 
 ## Quick Start
 
@@ -74,7 +74,7 @@ browsermonitor --join=9222        # Join mode: attach to existing Chrome on port
 
 | Mode        | How to run              | When to use |
 |------------|--------------------------|-------------|
-| **Interactive** | `browsermonitor` (no flags) | Menu asks for project root, then: **o** = open Chrome, **j** = join running Chrome, **q** = quit. |
+| **Interactive** | `browsermonitor` (no flags) | First run asks HTTP port. Then menu: **o** = open Chrome (asks URL on first use), **j** = join running Chrome, **q** = quit. |
 | **Open**   | `browsermonitor --open [url]` | Launch a new Chrome and monitor it. Uses current dir for logs. |
 | **Join**   | `browsermonitor --join=PORT`  | Attach to an existing Chrome with remote debugging on PORT (e.g. 9222). Port is required. |
 
@@ -132,7 +132,7 @@ For **join mode** (attach to existing Chrome): start Chrome manually with `--rem
 
 Each project gets its own Chrome profile:
 - **WSL:** `%LOCALAPPDATA%\browsermonitor\{project}_{hash}` (Windows path)
-- **Native:** `.browsermonitor-profile/` in project dir
+- **Native:** `.browsermonitor/.chrome-profile/` in project dir
 
 Separate cookies and logins per project; won't interfere with your regular Chrome.
 
@@ -172,8 +172,8 @@ When browsermonitor runs in a project directory, it creates:
 │       ├── cookies/           # Per-domain cookie JSONs
 │       ├── dom.html
 │       └── screenshot.png
-└── .browsermonitor-profile/   # Chrome profile (native) or
-                               # %LOCALAPPDATA%\browsermonitor\ (WSL)
+│   └── .chrome-profile/       # Chrome profile (native) or
+│                              # %LOCALAPPDATA%\browsermonitor\ (WSL)
 ```
 
 ## Keyboard Controls (open/join mode)
@@ -208,8 +208,8 @@ Use `curl` to communicate with the HTTP API over REST. Default URL: `http://loca
 
 | Endpoint | Description |
 |----------|-------------|
-| `GET /dump` | Dump logs, DOM, cookies, screenshot to files; returns output paths |
-| `GET /status` | Current status, monitored URLs, stats, output file paths |
+| `GET /dump` | Dump logs, DOM, cookies, screenshot to files; returns output file paths |
+| `GET /status` | Current state: status, mode, monitored URLs, collecting flag, stats |
 | `GET /stop` | Pause collecting (console/network) |
 | `GET /start` | Resume collecting |
 | `GET /clear` | Clear in-memory buffers |
@@ -341,7 +341,7 @@ Second launch: chrome.exe --remote-debugging-port=9223 --user-data-dir=Y
 
 **Consequence:** If user has Chrome open for regular browsing, launching a new Chrome with debug flags does nothing - the existing Chrome (without debugging) handles it.
 
-**Detection method (used in monitor.mjs):**
+**Detection method (used in open-mode.mjs):**
 ```bash
 # From WSL, query Windows WMI for Chrome processes:
 wmic.exe process where "name='chrome.exe'" get processid,commandline
@@ -553,6 +553,7 @@ netstat -ano | findstr "9222.*LISTEN"
 
 - **Entry point:** [cli.mjs](src/cli.mjs) – argument parsing, mode dispatch
 - **Settings:** [settings.mjs](src/settings.mjs) – project paths, config loading
-- **Modes:** [monitor.mjs](src/monitor.mjs) re-exports; implementations in [monitor/join-mode.mjs](src/monitor/join-mode.mjs), [monitor/open-mode.mjs](src/monitor/open-mode.mjs), [monitor/interactive-mode.mjs](src/monitor/interactive-mode.mjs)
+- **Modes:** [monitor/open-mode.mjs](src/monitor/open-mode.mjs), [monitor/join-mode.mjs](src/monitor/join-mode.mjs)
+- **Shared modules:** [monitor/shared/](src/monitor/shared/) – cleanup, keyboard, HTTP state, tab switching, help, monitoring wrapper, user page filter
 - **WSL:** [os/wsl/index.mjs](src/os/wsl/index.mjs) – `runWslDiagnostics()`, `scanChromeInstances()`, Chrome launch and port proxy helpers
 - **Chrome launch (WSL):** [os/wsl/chrome.mjs](src/os/wsl/chrome.mjs) – `startChromeOnWindows()`, port proxy, profile path

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@romanmatena/browsermonitor",
-  "version": "2.0.1",
+  "version": "2.1.1",
   "description": "Browser console, network, DOM monitoring for debugging and LLM workflows. Captures console, network, cookies, screenshot, DOM. HTTP API for LLM/scripts. WSL+Windows, Linux, Open/Join modes.",
   "type": "module",
   "bin": {

package/src/agents.llm/browser-monitor-section.md

@@ -3,16 +3,18 @@
 
 **Why use this:** Do not ask the user to copy or paste content from the browser (DOM, console, network). You can obtain the current page state yourself: the project has browsermonitor; after a dump you read the generated files.
 
+**Monitored URL:** `{{DEFAULT_URL}}`
+
 **What to do when you need the live page (elements, console, requests):**
-1. Trigger a dump: `curl -s http://localhost:60001/dump` (monitor must be running; user runs `browsermonitor` or `browsermonitor --open` / `--join=9222` to start it).
+1. Trigger a dump: `curl -s http://localhost:{{HTTP_PORT}}/dump` (monitor must be running; user runs `browsermonitor` or `browsermonitor --open` / `--join=9222` to start it).
 2. Read the written files in `.browsermonitor/.puppeteer/`:
    - **.browsermonitor/.puppeteer/dom.html** – current page HTML (JS-modified DOM). Use this for the element tree and structure.
    - **.browsermonitor/.puppeteer/screenshot.png** – screenshot of the current tab viewport.
    - **.browsermonitor/.puppeteer/console.log** – browser console output.
    - **.browsermonitor/.puppeteer/network.log** and **.browsermonitor/.puppeteer/network-log/*.json** – network requests and responses.
    - **.browsermonitor/.puppeteer/cookies/** – cookies per domain (JSON files).
-3. Optional: `curl -s http://localhost:60001/status` for buffer status.
-4. Optional: `curl -s http://localhost:60001/tabs` to list tabs; `curl -s "http://localhost:60001/tab?index=2"` to switch tab.
-5. Optional: `curl -s http://localhost:60001/clear` to clear buffers.
+3. Optional: `curl -s http://localhost:{{HTTP_PORT}}/status` for buffer status.
+4. Optional: `curl -s http://localhost:{{HTTP_PORT}}/tabs` to list tabs; `curl -s "http://localhost:{{HTTP_PORT}}/tab?index=2"` to switch tab.
+5. Optional: `curl -s http://localhost:{{HTTP_PORT}}/clear` to clear buffers.
 
 <!-- END browser-monitor-llm-section - LLM: do not remove this block; required for automatic section updates -->

package/src/cli.mjs

@@ -6,7 +6,7 @@
  * Configuration in .browsermonitor/settings.json (created on first run).
  *
  * Subcommands:
- *   init         → Run setup (create .browsermonitor/, update agent files)
+ *   init         → Create .browsermonitor/, settings.json, update agent files
  *
  * Mode is chosen by arguments:
  *   (none)       → Interactive: menu (o = open, j = join, q = quit)
@@ -21,29 +21,49 @@
  *   --help           Show help
  */
 
-import { runInteractiveMode, runJoinMode, runOpenMode } from './monitor.mjs';
+import { parseArgs } from 'node:util';
+import { runJoinMode } from './monitor/join-mode.mjs';
+import { runOpenMode } from './monitor/open-mode.mjs';
 import { printAppIntro } from './intro.mjs';
 import { createHttpServer } from './http-server.mjs';
 import { printApiHelpTable } from './templates/api-help.mjs';
 import { printCliCommandsTable } from './templates/cli-commands.mjs';
-import { askHttpPort } from './utils/ask.mjs';
-import { loadSettings, isInitialized, getPaths, ensureDirectories } from './settings.mjs';
+import { printModeHeading } from './templates/section-heading.mjs';
+import { loadSettings, getPaths, ensureDirectories, isInitialized, DEFAULT_SETTINGS, saveSettings } from './settings.mjs';
 import { runInit } from './init.mjs';
-
-// Project root = current working directory
-const projectRoot = process.cwd();
-
-// Parse arguments
-const args = process.argv.slice(2);
-
-// Handle `browsermonitor init` subcommand
-if (args[0] === 'init') {
-  await runInit(projectRoot, { askForUrl: true, updateAgentFiles: true });
+import { resolveHttpPort, resolveDefaultUrl, askMode } from './utils/ask.mjs';
+
+// ---- Parse CLI arguments ----
+const { values: flags, positionals } = parseArgs({
+  args: process.argv.slice(2),
+  options: {
+    open:          { type: 'boolean', default: false },
+    join:          { type: 'string' },
+    port:          { type: 'string' },
+    realtime:      { type: 'boolean', default: false },
+    headless:      { type: 'boolean', default: false },
+    timeout:       { type: 'string', default: '0' },
+    'nav-timeout': { type: 'string' },
+    help:          { type: 'boolean', short: 'h', default: false },
+  },
+  allowPositionals: true,
+  strict: false,
+});
+
+const subcommand = positionals[0];
+const urlFromArgs = positionals.find((a) => /^https?:\/\//.test(a) || a.includes('localhost'));
+const joinPort = flags.join ? parseInt(flags.join, 10) : null;
+const httpPortFromArgs = flags.port ? parseInt(flags.port, 10) : null;
+const hardTimeout = parseInt(flags.timeout, 10) || 0;
+
+// Handle `browsermonitor init`
+if (subcommand === 'init') {
+  await runInit(process.cwd());
   process.exit(0);
 }
 
 // Show help
-if (args.includes('--help') || args.includes('-h')) {
+if (flags.help) {
   console.log(`
 Browser Monitor – capture browser console, network, and DOM for debugging and LLM workflows.
 
@@ -57,29 +77,23 @@ What it does:
   printCliCommandsTable({ showEntry: true, showUsage: true });
   console.log(`
 Subcommands:
-  init                  Run setup: create .browsermonitor/, settings.json, update agent files
+  init                  Create .browsermonitor/, settings.json with defaults, update agent files
 
 Modes (chosen by flags; only one applies):
-  INTERACTIVE (default)   No flag. Asks for project root, then menu:
-                            o = open Chrome (launch and monitor)
-                            j = join running Chrome (pick instance/tab)
-                            q = quit
+  INTERACTIVE (default)   No flag. First run asks HTTP port and URL. Then menu:
+                            o = open Chrome, j = join running Chrome, q = quit
 
   OPEN (--open)           Launch a new Chrome and monitor it. URL = first positional or config.
-                          Uses current directory for logs. Good for local dev with a fresh profile.
 
-  JOIN (--join=PORT)      Attach to an existing Chrome with remote debugging on PORT.
-                          Port is required (e.g. --join=9222). Use when Chrome is already
-                          running (e.g. started by a script or on another machine via tunnel).
+  JOIN (--join=PORT)      Attach to existing Chrome with remote debugging on PORT.
+                          If PORT omitted, scans for running instances.
 
 Options:
   --port=PORT             HTTP API port (default: from settings or 60001)
-  --realtime              Write each event to files immediately (default: lazy, buffer in memory)
+  --realtime              Write each event to files immediately (default: lazy)
   --headless              Run Chrome without GUI
-  --open                  Go directly to open mode
-  --join=PORT             Go directly to join mode (PORT required)
   --timeout=MS            Hard timeout in ms; process exits after (0 = disabled)
-  --nav-timeout=MS        Navigation timeout in ms (default: from settings, 0 = no limit)
+  --nav-timeout=MS        Navigation timeout in ms (default: from settings)
   --help, -h              Show this help
 
 Config (.browsermonitor/settings.json):
@@ -90,67 +104,42 @@ Config (.browsermonitor/settings.json):
   process.exit(0);
 }
 
-// Auto-init on first run
-if (!isInitialized(projectRoot)) {
-  console.log('[browsermonitor] First run detected. Setting up .browsermonitor/ ...');
-  await runInit(projectRoot, { askForUrl: process.stdin.isTTY, updateAgentFiles: true });
-}
-
-// Ensure directories exist (in case user deleted .puppeteer/ subdir)
-ensureDirectories(projectRoot);
-
-// Load settings from .browsermonitor/settings.json
-const config = loadSettings(projectRoot);
-const paths = getPaths(projectRoot);
-
-// ---- Mode dispatch: --open | --join=PORT | interactive ----
-const openMode = args.some((a) => a === '--open' || a.startsWith('--open='));
-const joinArg = args.find((a) => a.startsWith('--join'));
-let joinPort = null;
-if (joinArg) {
-  if (joinArg === '--join' || !joinArg.includes('=')) {
-    console.error('Error: --join requires a port (e.g. --join=9222)');
-    process.exit(1);
-  }
-  const portStr = joinArg.split('=')[1];
-  joinPort = parseInt(portStr, 10);
-  if (Number.isNaN(joinPort) || joinPort < 1 || joinPort > 65535) {
-    console.error(`Error: invalid port for --join: ${portStr}`);
-    process.exit(1);
-  }
-}
-
-// Shared options (CLI args override settings.json)
-const realtimeMode = args.includes('--realtime') || config.realtime;
-const headlessCli = args.includes('--headless');
-const timeoutArg = args.find((a) => a.startsWith('--timeout='));
-const hardTimeout = timeoutArg ? parseInt(timeoutArg.split('=')[1], 10) : 0;
-const navTimeoutArg = args.find((a) => a.startsWith('--nav-timeout='));
-const navigationTimeout = navTimeoutArg
-  ? parseInt(navTimeoutArg.split('=')[1], 10)
-  : (config.navigationTimeout !== undefined ? config.navigationTimeout : 60_000);
-const urlFromArgs = args.find((a) => !a.startsWith('--'));
-const url = urlFromArgs || config.defaultUrl || 'https://localhost:4000/';
-const headless = headlessCli || config.headless || false;
-const ignorePatterns = config.ignorePatterns || [];
-const outputDir = projectRoot;
-
-const DEFAULT_HTTP_PORT = config.httpPort || 60001;
-const portArg = args.find((a) => a === '--port' || a.startsWith('--port='));
-let httpPortFromArgs = null;
-if (portArg) {
-  const val = portArg.includes('=') ? portArg.split('=')[1] : '';
-  const num = parseInt(val, 10);
-  if (!Number.isNaN(num) && num >= 1 && num <= 65535) httpPortFromArgs = num;
-}
-
 (async () => {
+  // 1. Intro
   printAppIntro();
 
-  printApiHelpTable({ port: DEFAULT_HTTP_PORT, showApi: true, showInteractive: false, showOutputFiles: true, noLeadingNewline: true });
+  // 2. Project root = cwd, load existing settings (may be empty/missing)
+  const projectRoot = process.cwd();
+  ensureDirectories(projectRoot);
+  let config = loadSettings(projectRoot);
+  const paths = getPaths(projectRoot);
+
+  // 3. CLI args override settings.json
+  const realtimeMode = flags.realtime || config.realtime;
+  const navTimeoutFromArgs = flags['nav-timeout'] ? parseInt(flags['nav-timeout'], 10) : null;
+  const navigationTimeout = navTimeoutFromArgs
+    ?? (config.navigationTimeout !== undefined ? config.navigationTimeout : 60_000);
+  const headless = flags.headless || config.headless || false;
+  const httpPort = await resolveHttpPort(httpPortFromArgs ?? config.httpPort, DEFAULT_SETTINGS.httpPort);
+  const url = await resolveDefaultUrl(urlFromArgs || config.defaultUrl, DEFAULT_SETTINGS.defaultUrl);
+  
+  // 4. Need to initialize project (create .browsermonitor/, settings.json) before showing API info, because API port is part of config
+  if (!isInitialized(projectRoot)) {
+    saveSettings(projectRoot, { ...DEFAULT_SETTINGS, httpPort, defaultUrl: url, headless, navigationTimeout, realtime: realtimeMode });
+    config = loadSettings(projectRoot);
+    await runInit(projectRoot, config);
+  }
 
-  const httpPort =
-    httpPortFromArgs ?? (process.stdin.isTTY ? await askHttpPort(DEFAULT_HTTP_PORT) : DEFAULT_HTTP_PORT);
+  // 5. Show API/output info (now httpPort is known)
+  printApiHelpTable({
+    url: config.defaultUrl,
+    port: config.httpPort,
+    showApi: true,
+    showInteractive: false,
+    showOutputFiles: true,
+    noLeadingNewline: true,
+    context: paths,
+  });
 
   const sharedHttpState = {
     mode: 'interactive',
@@ -162,41 +151,50 @@ if (portArg) {
     getAllTabs: async () => [],
   };
   const sharedHttpServer = createHttpServer({
-    port: httpPort,
-    defaultPort: DEFAULT_HTTP_PORT,
+    port: config.httpPort,
+    defaultPort: config.httpPort,
     getState: () => sharedHttpState,
   });
 
   const commonOptions = {
-    realtime: realtimeMode,
-    outputDir,
+    outputDir: projectRoot,
     paths,
-    ignorePatterns,
+    realtime: realtimeMode,
+    ignorePatterns: config.ignorePatterns,
     hardTimeout,
-    httpPort,
+    httpPort: config.httpPort,
+    joinPort,
     sharedHttpState,
     sharedHttpServer,
   };
 
-  if (openMode) {
-    console.log(`  [CLI] Open mode → ${url}`);
-    await runOpenMode(url, {
+  // 8. Dispatch to mode
+  if (flags.open) {
+    await runOpenMode(config.defaultUrl, {
       ...commonOptions,
       headless,
       navigationTimeout,
     });
   } else if (joinPort !== null) {
-    console.log(`  [CLI] Join mode → localhost:${joinPort}`);
-    await runJoinMode(joinPort, {
-      ...commonOptions,
-      defaultUrl: url,
-    });
+    await runJoinMode(config.defaultUrl, commonOptions);
   } else {
-    await runInteractiveMode({
-      ...commonOptions,
-      defaultUrl: url,
-      headless,
-      navigationTimeout,
-    });
+    // No mode flag → ask user
+    printModeHeading('Choose mode');
+    const mode = await askMode();
+    if (mode === 'q') process.exit(0);
+
+    if (mode === 'o') {
+      await runOpenMode(config.defaultUrl, {
+        ...commonOptions,
+        headless,
+        navigationTimeout,
+        skipModeHeading: true,
+      });
+    } else if (mode === 'j') {
+      await runJoinMode(config.defaultUrl, {
+        ...commonOptions,
+        skipModeHeading: true,
+      });
+    }
   }
 })();

package/src/http-server.mjs

@@ -12,8 +12,8 @@ import http from 'http';
 import { C, log } from './utils/colors.mjs';
 import { getFullTimestamp } from './logging/index.mjs';
 import { getComputedStylesFromPage } from './logging/dump.mjs';
-import { printApiHelpTable, API_ENDPOINTS } from './templates/api-help.mjs';
-import { printSectionHeading } from './templates/section-heading.mjs';
+import { API_ENDPOINTS } from './templates/api-help.mjs';
+
 
 /** Default timeout for Puppeteer operations (ms). */
 const PUPPETEER_CALL_TIMEOUT_MS = 30_000;
@@ -155,7 +155,6 @@ export function createHttpServer(options) {
         return;
       }
       try {
-        const statsBeforeDump = s.logBuffer.getStats();
         const pages = s.getPages();
         const page = pages.length > 0 ? pages[0] : null;
 
@@ -172,26 +171,14 @@ export function createHttpServer(options) {
           success: true,
           timestamp: getFullTimestamp(),
           message: 'Dump completed. Read the files below.',
-          stats: statsBeforeDump,
-          outputFiles: {
-            consoleLog: s.logBuffer.CONSOLE_LOG,
-            networkLog: s.logBuffer.NETWORK_LOG,
-            networkDir: s.logBuffer.NETWORK_DIR,
-            cookiesDir: s.logBuffer.COOKIES_DIR,
-            domHtml: s.logBuffer.DOM_HTML,
-            screenshot: s.logBuffer.SCREENSHOT,
-          },
-          llm: {
-            instruction: 'Read or download these files to get the current browser state. Do not ask the user to copy/paste from the browser.',
-            files: [
-              { path: s.logBuffer.DOM_HTML, what: 'Current page HTML (JS-modified DOM). Use for element tree and structure.' },
-              { path: s.logBuffer.SCREENSHOT, what: 'Screenshot of the current tab viewport (PNG).' },
-              { path: s.logBuffer.CONSOLE_LOG, what: 'Browser console output (logs, errors, warnings).' },
-              { path: s.logBuffer.NETWORK_LOG, what: 'Network requests overview (one line per request with ID).' },
-              { path: s.logBuffer.NETWORK_DIR, what: 'Directory with one JSON per request: full headers, payload, response (see IDs in network log).' },
-              { path: s.logBuffer.COOKIES_DIR, what: 'Directory with cookies per domain (JSON files).' },
-            ],
-          },
+          files: [
+            { path: s.logBuffer.DOM_HTML, what: 'Current page HTML (JS-modified DOM). Use for element tree and structure.' },
+            { path: s.logBuffer.SCREENSHOT, what: 'Screenshot of the current tab viewport (PNG).' },
+            { path: s.logBuffer.CONSOLE_LOG, what: 'Browser console output (logs, errors, warnings).' },
+            { path: s.logBuffer.NETWORK_LOG, what: 'Network requests overview (one line per request with ID).' },
+            { path: s.logBuffer.NETWORK_DIR, what: 'Directory with one JSON per request: full headers, payload, response (see IDs in network log).' },
+            { path: s.logBuffer.COOKIES_DIR, what: 'Directory with cookies per domain (JSON files).' },
+          ],
         }, null, 2));
       } catch (error) {
         res.writeHead(500, { 'Content-Type': 'application/json' });
@@ -200,7 +187,7 @@ export function createHttpServer(options) {
       return;
     }
 
-    // GET /status
+    // GET /status — state only (no file paths; use GET /dump for that)
     if (req.url === '/status' && req.method === 'GET') {
       const pages = s.getPages();
       const collectingPaused = s.getCollectingPaused();
@@ -215,14 +202,6 @@ export function createHttpServer(options) {
       if (!noBrowser) {
         payload.collecting = collectingPaused ? 'paused' : 'running';
         payload.stats = s.logBuffer.getStats();
-        payload.outputFiles = {
-          consoleLog: s.logBuffer.CONSOLE_LOG,
-          networkLog: s.logBuffer.NETWORK_LOG,
-          networkDir: s.logBuffer.NETWORK_DIR,
-          cookiesDir: s.logBuffer.COOKIES_DIR,
-          domHtml: s.logBuffer.DOM_HTML,
-          screenshot: s.logBuffer.SCREENSHOT,
-        };
       } else {
         payload.message = 'No browser. Use interactive (o/j) or --open / --join.';
       }
@@ -500,12 +479,7 @@ export function createHttpServer(options) {
     }, null, 2));
   });
 
-  server.listen(port, host, () => {
-    const url = `http://${host}:${port}`;
-    const changed = port !== defaultPort ? ` ${C.red}(changed)${C.reset}` : '';
-    const title = `HTTP API URL: ${url}${changed}`;
-    printSectionHeading(title, '  ');
-  });
+  server.listen(port, host);
 
   server.on('error', (err) => {
     if (err.code === 'EADDRINUSE') {