browser4-cli 0.1.21 → 0.1.23

package/README.md

@@ -25,19 +25,7 @@ npm install browser4-cli
 
 Then use via `package.json` scripts or by invoking `browser4-cli` directly.
 
-### Homebrew (macOS)
-
-```bash
-brew install browser4-cli
-```
-
-### Cargo (Rust)
-
-```bash
-cargo install browser4-cli
-```
-
-### Standalone Installer Scripts (no npm / Rust / Homebrew needed)
+### Standalone Installer Scripts (no npm needed)
 
 Bootstrap the native binary directly with a single command:
 
@@ -155,7 +143,7 @@ The tables below mirror the commands surfaced by the global `browser4-cli help`
 | `check <ref>` | Check a checkbox or radio button |
 | `uncheck <ref>` | Uncheck a checkbox or radio button |
 | `drag <startRef> <endRef>` | Drag and drop between two elements |
-| `snapshot` | Capture accessibility snapshot. Supports `--filename=<path>` to save snapshot to file. |
+| `snapshot` | Capture accessibility snapshot. Supports `--filename=<path>` to save to file, `--boxes` for bounding boxes, `-i`/`--interactive` for interactive-only, `-u`/`--urls` for link URLs, `-c`/`--compact` for compact output, `-d`/`--depth <n>` for depth limiting, `-s`/`--selector <sel>` for CSS scoping, and `--raw` for raw output without page info. |
 | `eval <expression> [ref]` | Evaluate JavaScript on the page or a target element. Use `--file=<path>` to read the expression from a file. |
 | `get <mode> <selector> [name]` | Extract data from a page element. Modes: `text`, `html`, `box`, `styles`, `property`, `attr`. `name` is required for `property` and `attr`. |
 | `scroll <direction> <pixels>` | Scroll the page. Direction: `up`, `down`, `left`, or `right`. |
@@ -246,7 +234,7 @@ Use `close-all` for session cleanup when you want to keep the current Browser4 s
 |---|---|
 | `install` | Download the Browser4 runtime bundle. Supports `--tag=<version>` to pin a release and `--force` to reinstall even when already present. |
 | `upgrade` | Upgrade the Browser4 runtime bundle to the latest version or a specified `--tag` |
-| `uninstall` | Remove globally installed browser4-cli (npm, cargo) and its runtime data. Supports `-y` / `--yes` (skip confirmation) and `--dry-run` (preview). |
+| `uninstall` | Remove globally installed browser4-cli (npm) and its runtime data. Supports `-y` / `--yes` (skip confirmation) and `--dry-run` (preview). |
 | `stop` | Kill the Browser4 backend after closing all sessions |
 | `status` | Check whether the Browser4 backend is reachable and healthy |
 
@@ -262,8 +250,8 @@ When a local Browser4 checkout is detected with the `browser4-bundle` module pre
 `install` and `upgrade` auto-build the runtime bundle from source (via Maven) instead
 of downloading.
 
-`uninstall` attempts to remove browser4-cli from npm global packages and cargo
-installs, and deletes the Browser4 runtime data and cache directories. It prompts
+`uninstall` attempts to remove browser4-cli from npm global packages
+and deletes the Browser4 runtime data and cache directories. It prompts
 for confirmation unless `-y` / `--yes` is passed. Use `--dry-run` to preview what
 would be removed without making changes. Does not require a running server.
 
@@ -292,6 +280,7 @@ Query `browser4-cli help <command>` for the exact syntax when you need them.
 | `swarm query <url>` | Run an X-SQL query against a loaded webpage |
 | `swarm status <id>` | Check the status of a scrape or query job |
 | `swarm result <id>` | Get the result of a completed job |
+| `crawl <url>` | Crawl a website from a seed URL, following links up to a configurable depth |
 
 ## Agent task workflow (`agent <subcommand>`)
 
@@ -393,6 +382,8 @@ coordinates multiple browser contexts in the Browser4 backend.
 | `swarm query <url>` | Run X-SQL queries against loaded pages | `POST /api/swarm/query` |
 | `swarm status <id>` | Poll job status | `GET /api/swarm/{id}/status` |
 | `swarm result <id>` | Fetch completed job result | `GET /api/swarm/{id}/result` |
+| `crawl <url>` | Start a website crawl | `POST /api/crawl` |
+| (poll) | Track crawl progress | `GET /api/crawl/{id}/result` |
 
 ### URL scraping with `swarm submit`
 
@@ -450,6 +441,57 @@ browser4-cli swarm query --sql @query.sql --seed-file=./urls.txt --refresh
 - Prefix the `--sql` value with `@` to read from a file (e.g. `--sql @query.sql`).
 - All commands return a task ID; use `swarm status` / `swarm result` to track progress.
 
+## Crawl (`crawl`)
+
+Recursive website crawling from a seed URL.
+
+### Command overview
+
+| Command | Purpose | Backend endpoint |
+|---|---|---|
+| `crawl <url>` | Crawl a website from a seed URL | `POST /api/crawl` |
+| (poll) | Track crawl progress | `GET /api/crawl/{id}/result` |
+
+### Basic crawling
+
+```shell
+# depth=1: extract all links from homepage, load each linked page
+browser4-cli crawl "https://example.com" --out-link-selector "a[href]"
+
+# depth=2: follow links two levels deep, filter by pattern
+browser4-cli crawl "https://shop.example.com" \
+  --depth 2 \
+  --out-link-selector "a.product-link" \
+  --out-link-pattern "/product/" \
+  --top-links 10
+
+# with LoadOptions passthrough
+browser4-cli crawl "https://example.com" -ol "a[href]" -a "-refresh -nMaxRetry 5"
+```
+
+### Key flags
+
+| Flag | Default | Description |
+|---|---|---|
+| `-d`, `--depth` | `1` | Maximum crawl depth |
+| `-ol`, `--out-link-selector` | — | CSS selector to extract links from each page |
+| `-olp`, `--out-link-pattern` | `.+` | Regex pattern to filter extracted links |
+| `-tl`, `--top-links` | `20` | Maximum links to extract per page |
+| `-a`, `--args` | — | Additional LoadOptions passthrough |
+| `--refresh` | — | Force a fresh fetch, ignoring cache |
+| `--parse` | — | Parse each page immediately after fetching |
+| `--expires` | — | Cache expiration duration |
+| `--store-content` | — | Persist page content to storage |
+| `--page-load-timeout` | — | Maximum time to wait for page load |
+| `--readonly` | — | Non-destructive mode |
+
+### Notes
+
+- All LoadOptions flags available via `--args` passthrough (e.g. `-a "-nMaxRetry 5 -lazyFlush"`).
+- Depth=1 reuses `PulsarSession.submitForOutPages`; depth>1 uses BFS continuous crawl with visited-URL dedup.
+- CLI timeout: 600s default, configurable via `BROWSER4_CLI_CRAWL_TIMEOUT_SECS`.
+- Duplicate URLs are skipped (normalized: lowercase, no trailing slash, no query string).
+
 ## DOM Snapshot commands (`domsnapshot <subcommand>`)
 
 The `domsnapshot` commands capture and query a **static DOM snapshot** of the
@@ -474,6 +516,37 @@ browser4-cli domsnapshot export --file=page.html
 | `domsnapshot get <field> [selector] [name]` | Extract elements from the static DOM snapshot |
 | `domsnapshot query [url] --sql=<query>` | Run X-SQL against the DOM snapshot via the scrape API |
 | `domsnapshot export [--file=<path>]` | Save full snapshot HTML content to a local file |
+| `domsnapshot grep [OPTIONS] <pattern>` | Search snapshot HTML with regex and grep-style output |
+
+### `domsnapshot grep` flags
+
+Line numbers are shown by default (unlike GNU grep). Use `--no-line-number` to suppress them.
+
+| Flag | Description |
+|---|---|
+| `-i` | Case-insensitive matching |
+| `-A N`, `-B N`, `-C N` | Context lines after / before / around each match |
+| `-v` | Invert match (select non-matching lines) |
+| `-c` | Print only count of matching lines |
+| `-l` | Print only whether matches exist (prints "domsnapshot" if matches found; use with `\| grep -q domsnapshot` for CI pass/fail) |
+| `-F` | Treat pattern as a literal string |
+| `-w` | Match whole words only |
+| `--no-line-number` | Suppress line numbers in output |
+| `--selector <CSS>` | Scope search to a CSS selector |
+
+```shell
+# Basic search
+browser4-cli domsnapshot grep -i error
+
+# With context and literal match
+browser4-cli domsnapshot grep -F -C 2 "404 Not Found"
+
+# Count matches
+browser4-cli domsnapshot grep -c "div"
+
+# Search within a specific element
+browser4-cli domsnapshot grep --selector main "Submit"
+```
 
 ### `domsnapshot get` fields
 
@@ -612,6 +685,18 @@ browser4-cli goto https://browser4.io
 # Inspect the page — note the eN labels on interactive nodes
 browser4-cli snapshot
 
+# Include element bounding boxes in the snapshot
+browser4-cli snapshot --boxes
+
+# Show only interactive elements with compact output, depth-limited
+browser4-cli snapshot -i -c -d 5
+
+# Scope snapshot to a specific CSS selector
+browser4-cli snapshot -s "#main-content"
+
+# Raw output — no page info, just the snapshot content (useful for piping)
+browser4-cli snapshot --raw | grep "button"
+
 # Capture a static DOM snapshot and extract data
 browser4-cli domsnapshot
 browser4-cli domsnapshot get text "#productTitle"

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "browser4-cli",
-  "version": "0.1.21",
+  "version": "0.1.23",
   "description": "Browser automation CLI for AI agents",
   "type": "module",
   "files": [