browser4-cli 0.1.8 → 0.1.9

package/README.md

@@ -68,6 +68,16 @@ browser4-cli -s=<session> <command> [args] [options]
 | `--version`        | Print version                                  |
 | `-s=<name>`        | Named session label                            |
 | `--server=<url>`   | Override Browser4 server URL                   |
+| `--json`           | Emit machine-parseable JSON to stdout          |
+| `-q`, `--quiet`    | Suppress normal output, only show errors       |
+
+`--json` switches every command's stdout from human-readable text to a
+single-line JSON envelope (`{"status":"ok","command":"<name>","output":{...}}`).
+Omit `--json` for the default human-readable output.
+
+`-q` / `--quiet` suppresses all normal stdout output.  Errors and
+progress messages still go to stderr.  Combine with `--json` for
+silent-on-success scripting: `browser4-cli --json -q open`.
 
 Sessions are persisted independently per name. Omitting `-s` uses the
 default session (`~/.browser4/cli-state.json`). With `-s=<name>`, a
@@ -83,9 +93,9 @@ The tables below mirror the commands surfaced by the global `browser4-cli help`
 
 | Command | Description |
 |---|---|
-| `open [url]` | Open or switch to a browser session (optionally navigate to URL) |
+| `open [url]` | Open or switch to a browser session. Supports `--headed` (force visible window) and `--headless` (force headless). |
 | `close` | Close the active session |
-| `goto <url>` | Navigate to a URL using the current active session |
+| `goto <url>` | Navigate to a URL, auto-opening or refreshing the session if needed |
 | `click <ref> [button]` | Click an element |
 | `dblclick <ref> [button]` | Double-click an element |
 | `type <text> [ref]` | Type text into the focused element or an optional target element |
@@ -128,11 +138,11 @@ The tables below mirror the commands surfaced by the global `browser4-cli help`
 | `mouseup [button]` | Release mouse button |
 | `mousewheel <dx> <dy>` | Scroll the mouse wheel |
 
-#### Save as
+#### Screenshots
 
 | Command | Description |
 |---|---|
-| `screenshot [ref]` | Take a screenshot |
+| `screenshot [ref]` | Take a screenshot (optionally of a specific element) |
 
 #### Tabs
 
@@ -181,13 +191,22 @@ Use `close-all` for session cleanup when you want to keep the current Browser4 s
 
 | Command | Description |
 |---|---|
-| `install` | Download the self-contained Browser4 runtime bundle (JAR + bundled JRE) from GitHub Releases |
-| `upgrade` | Upgrade `browser4-cli` itself to the latest release (requires `cargo`) |
+| `install` | Download the Browser4 runtime bundle. Supports `--tag=<version>` to pin a release and `--force` to reinstall even when already present. |
+| `upgrade` | Upgrade the Browser4 runtime bundle to the latest version or a specified `--tag` |
 | `stop` | Kill the Browser4 backend after closing all sessions |
 | `status` | Check whether the Browser4 backend is reachable and healthy |
 
+`install` and `upgrade` both manage the Browser4 runtime bundle — a self-contained
+distribution that includes all dependency jars, a minimal `jlink`-built JRE, and
+platform launcher scripts. Neither requires `cargo` or a Rust toolchain; the runtime
+is a Java application downloaded from GitHub Releases.
+
+Use `--tag=<version>` to pin a specific release (e.g. `--tag=v4.9.3`). Use `--force`
+to reinstall even when the same version is already present.
+
 When a local Browser4 checkout is detected with the `browser4-bundle` module present,
-`install` auto-builds the runtime bundle from source instead of downloading.
+`install` and `upgrade` auto-build the runtime bundle from source (via Maven) instead
+of downloading.
 
 ### Advanced commands
 
@@ -204,9 +223,10 @@ Query `browser4-cli help <command>` for the exact syntax when you need them.
 | `agent status <id>` | Check the status of a running agent task |
 | `agent result <id>` | Get the result of a completed agent task |
 | `swarm create` | Create a swarm scrape session with parallel browser contexts |
-| `swarm submit [url]` | Submit URL(s) or X-SQL payloads as scrape jobs |
-| `swarm status <id>` | Check the status of a scrape job |
-| `swarm result <id>` | Get the result of a completed scrape job |
+| `swarm submit [url]` | Submit URL(s) or raw X-SQL payloads as scrape jobs |
+| `swarm query <url>` | Run an X-SQL query against a loaded webpage |
+| `swarm status <id>` | Check the status of a scrape or query job |
+| `swarm result <id>` | Get the result of a completed job |
 
 ## Agent task workflow (`agent <subcommand>`)
 
@@ -259,7 +279,7 @@ browser4-cli agent result agent-task-1
 #### 1. Submit an autonomous agent task
 
 ```shell
-browser4-cli agent run "Open example.com and summarize the hero section"
+browser4-cli agent run "Open browser4.io and summarize the hero section"
 ```
 
 Typical output:
@@ -299,93 +319,78 @@ If the backend returns a structured `CommandResult`, expect fields such as
 The `swarm` subcommands support a swarm scrape workflow where one CLI session
 coordinates multiple browser contexts in the Browser4 backend.
 
-Use the spaced `swarm <subcommand>` form:
-
-```shell
-browser4-cli swarm create
-browser4-cli swarm submit https://example.com
-```
-
-### Command lifecycle
+### Command overview
 
-| Step | Command | What it does |
+| Command | Purpose | Backend endpoint |
 |---|---|---|
-| 1 | `swarm create` | Opens a swarm scrape session and persists the returned session ID in the current CLI slot |
-| 2 | `swarm submit [url]` | Submits one direct URL plus any URLs from `--seed-file` as scrape jobs through `ScrapeController.submit(payload)` |
-| 3 | `swarm status <id>` | Calls `ScrapeController.getStatus(id)` and prints the returned scrape job status JSON |
-| 4 | `swarm result <id>` | Calls `ScrapeController.getResult(id)` and prints the returned scrape job result JSON |
+| `swarm create` | Create a swarm scrape session | `POST /api/swarm` |
+| `swarm submit <url>` | Scrape URLs or submit raw X-SQL | `POST /api/swarm/submit` |
+| `swarm query <url>` | Run X-SQL queries against loaded pages | `POST /api/swarm/query` |
+| `swarm status <id>` | Poll job status | `GET /api/swarm/{id}/status` |
+| `swarm result <id>` | Fetch completed job result | `GET /api/swarm/{id}/result` |
 
-### Notes
-
-- `swarm create` accepts backend capability hints such as `--profile-mode`,
-  `--max-open-tabs`, `--max-browser-contexts`, and `--display-mode`.
-- `swarm submit` accepts either a direct positional URL, `--seed-file`, or both.
-  Seed files are plain text files with one URL per line; blank lines and lines
-  starting with `#` are ignored.
-- `swarm submit` maps CLI flags like `--deadline`, `--expires`, `--refresh`,
-  `--parse`, and `--store-content` into the raw submission payload sent to the
-  scrape REST API.
-- `swarm status` and `swarm result` are read-only follow-up commands; keep the job ID
-  printed by `swarm submit`.
-
-### Use cases
-
-#### 1. Create a supervised swarm scrape session for manual monitoring
+### URL scraping with `swarm submit`
 
 ```shell
+# create a session
 browser4-cli swarm create \
   --profile-mode=TEMPORARY \
   --max-open-tabs=12 \
   --max-browser-contexts=3 \
   --display-mode=HEADLESS
-```
-
-Use this when you want multiple isolated browser contexts and you still want to
-watch the run visually.
 
-#### 2. Submit a seed crawl as scrape jobs
-
-```shell
+# submit URLs as scrape jobs
 browser4-cli swarm submit https://example.com/direct \
   --seed-file=./swarm-seeds.txt \
   --deadline=2026-03-30T00:00:00Z \
   --expires=1d \
-  --refresh \
-  --parse \
-  --store-content
-```
+  --refresh --parse --store-content
 
-Example `swarm-seeds.txt`:
-
-```text
-# campaign landing pages
-https://example.com/seed-1
-https://example.com/seed-2
+# poll and fetch the result
+browser4-cli swarm status scrape-task-4
+browser4-cli swarm result scrape-task-4
 ```
 
-This pattern is useful for warming caches, refreshing a URL list, or launching
-parallel collection across a curated seed set.
+### X-SQL queries with `swarm query`
 
-#### 3. Poll and fetch the result
+Run structured X-SQL queries against loaded webpages to extract data.
 
 ```shell
-browser4-cli swarm status scrape-task-4
-browser4-cli swarm result scrape-task-4
+# Inline query:
+browser4-cli swarm query "https://www.amazon.com/dp/B08PP5MSVB" --sql "
+  SELECT
+    dom_base_uri(dom) AS url,
+    dom_first_text(dom, '#productTitle') AS title,
+    dom_first_slim_html(dom, 'img:expr(width > 400)') AS img
+  FROM load_and_select(@url, 'body');
+"
+
+# From a file:
+browser4-cli swarm query "https://www.amazon.com/dp/B08PP5MSVB" --sql @query.sql
+
+# With seed file and load options:
+browser4-cli swarm query --sql @query.sql --seed-file=./urls.txt --refresh --parse
 ```
 
-The status and result commands print the scrape job response payload as-is. In
-the current backend, `getResult(id)` returns the same response envelope type as
-`getStatus(id)`.
+### Notes
+
+- `swarm create` accepts backend capability hints: `--profile-mode`, `--max-open-tabs`,
+  `--max-browser-contexts`, `--display-mode`.
+- `swarm submit` and `swarm query` both accept a positional URL, `--seed-file`, or both.
+  Seed files use one URL per line; `#` comments and blank lines are ignored.
+- Both commands support load-option flags: `--deadline`, `--expires`, `--refresh`,
+  `--parse`, `--store-content`.
+- `swarm query --sql` is **required**; `swarm submit --sql` also works as a convenience.
+  Use `@url` in the X-SQL template; it is replaced with the target URL server-side.
+- Prefix the `--sql` value with `@` to read from a file (e.g. `--sql @query.sql`).
+- All commands return a task ID; use `swarm status` / `swarm result` to track progress.
 
 ## Element References
 
 The `snapshot` command returns an accessibility tree where every interactive
 node is labeled with a short identifier such as `e15`. Pass this identifier
-directly to commands like `click`, `type`, or `press`; the CLI automatically
-converts it to the `backend:15` selector format required by the server.
-
-You can also pass plain CSS selectors (e.g. `.my-button`, `#search-input`) or
-fully-qualified `backend:<N>` refs directly.
+directly to commands like `click`, `type`, or `press`. You can also use plain
+CSS selectors (e.g. `.my-button`, `#search-input`).
 
 ## State Persistence
 
@@ -406,41 +411,23 @@ fields such as:
 
 ### Session state transitions
 
-The `with_session()` helper in `src/main.rs` is the central session lifecycle
-gate for commands that require an active Browser4 session.
+| Command | Behavior |
+|---|---|
+| `open` | Creates a new session, or reuses an existing active one. Stale sessions are automatically refreshed. |
+| `open -s=<name>` | Same as `open` but scoped to a named session slot. |
+| `goto <url>` | Reuses the current session if active; otherwise opens a fresh one before navigating. |
+| `close` | Closes the current session (no-op if none active). |
+| `close-all` / `kill-all` / `stop` | Clears all persisted session state. |
 
-| Situation | Persisted state transition | Result |
-|---|---|---|
-| No persisted session | No state change | `require_session()` fails with `No active session. Run "browser4-cli open" first.` |
-| `open` succeeds (no existing session) | `create_session()` writes a fresh state file with new `sessionId`, current `baseUrl`, and clears `activeSelector` / `lastMousePosition` | A new active session becomes the current CLI session |
-| `open` when a saved session exists and the backend still reports it `active` | No state change — keeps the existing `sessionId` | The existing session is reused; subsequent commands target the same session |
-| `open` when a saved session exists but is missing or no longer `active` in the backend | `invalidate_session()` clears the stale saved `sessionId`, `activeSelector`, and `lastMousePosition`, then `create_session()` writes a fresh session | The stale session is refreshed automatically by opening a new one |
-| `open -s=<name>` | Reads/writes the named session state file | Opens, reuses, or refreshes the named session for that slot; subsequent `-s=<name>` commands use the same slot |
-| Command succeeds through `with_session()` | `sessionId` stays unchanged | The command uses the persisted session normally |
-| Command fails because the server reports a stale / expired session and `recover_stale = false` | `invalidate_session()` clears `sessionId`, `activeSelector`, and `lastMousePosition`, while keeping `baseUrl` | The command fails with `Saved session expired. Run "browser4-cli open" first.` |
-| `goto` is invoked but the saved session is missing or no longer `active` in the backend | `invalidate_session()` clears any stale saved `sessionId`, then `create_session()` writes a fresh session before navigation continues | `goto` automatically refreshes the session and proceeds to the requested URL |
-| `close` with an active session | `clear_state()` removes only the current session state file after best-effort remote close | The selected default or named session is fully cleared |
-| `close` with no persisted `sessionId` | `clear_state()` best-effort removes the current session slot | Prints `No active session. Run "browser4-cli open" first.` and exits successfully as a no-op |
-| `close-all` / `kill-all` | `clear_all_state()` removes the default state file and all named session files | All persisted CLI session files are cleared |
-
-Notes:
-
-- `goto` first tries to reuse the current backend-`active` session. If the saved
-  session is missing, stale, or the backend had been stopped, it automatically
-  opens a fresh session for the current slot before navigating.
-- `open` first checks whether the saved session for the current slot is still
-  backend-`active`. It reuses active sessions and refreshes stale ones by
-  creating a new session for the same slot.
-- `list` reads persisted session files and compares them with live backend
-  sessions to show both the current status (`Active`, `Stale`, or `Unknown`)
-  and whether the next `open` will `Reuse` or `Refresh` that slot.
+The `list` command shows each session's status: **Active** (backend confirms),
+**Stale** (backend has stopped it), or **Unknown** (backend unreachable).
 
 ## Runtime Temp Files
 
 `browser4-cli` keeps ephemeral runtime artifacts under the system temp directory:
 
-- Windows: `%TEMP%\.browser4\browser4-cli`
-- Linux/macOS: `${TMPDIR:-/tmp}/.browser4/browser4-cli`
+- Windows: `%TEMP%\browser4\browser4-cli`
+- Linux/macOS: `${TMPDIR:-/tmp}/browser4/browser4-cli`
 
 This temp subtree contains items such as:
 
@@ -448,7 +435,20 @@ This temp subtree contains items such as:
 - staged Maven wrapper launchers
 - Rust test scratch directories used by `browser4-cli` tests
 
-Persistent CLI state and the fallback `Browser4.jar` remain under `~/.browser4` by default.
+Persistent CLI state remains under `~/.browser4` by default.  The Browser4 runtime
+bundle (JRE, JARs, launchers) is stored separately in a platform-conventional
+data directory so that clearing CLI session state does not require re-downloading
+the ~200 MB runtime:
+
+- Linux:   `~/.local/share/browser4/runtime/<version>/`
+- macOS:   `~/Library/Application Support/browser4/runtime/<version>/`
+- Windows: `%APPDATA%/browser4/runtime/<version>/`
+
+The `current.tag` file in the `runtime/` directory records the active version.
+Override the runtime data root with the `BROWSER4_RUNTIME_DIR` environment variable.
+Downloaded archives are cached under the platform cache directory
+(`~/.cache/browser4/downloads/` on Linux, `~/Library/Caches/browser4/downloads/`
+on macOS, `%LOCALAPPDATA%/browser4/downloads/` on Windows).
 
 ## Snapshots
 
@@ -462,10 +462,14 @@ After each command that modifies browser state, the CLI automatically:
 ## Examples
 
 ```shell
-# Open a new browser window
+# Open a new browser window (defaults to headed)
 browser4-cli open
 
-# Navigate to a page with the current active session
+# Open in headed or headless mode
+browser4-cli open --headed https://browser4.io
+browser4-cli open --headless https://browser4.io
+
+# Navigate to a page — auto-opens a session if none is active
 browser4-cli goto https://playwright.dev
 
 # Inspect the page — note the eN labels on interactive nodes

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "browser4-cli",
-  "version": "0.1.8",
+  "version": "0.1.9",
   "description": "Browser automation CLI for AI agents",
   "type": "module",
   "files": [
@@ -20,7 +20,7 @@
     "build:linux": "npm run version:sync && docker compose -f docker/docker-compose.yml run --rm build-linux",
     "build:macos": "npm run version:sync && (cargo build --release --manifest-path browser4-cli/Cargo.toml --target aarch64-apple-darwin & cargo build --release --manifest-path browser4-cli/Cargo.toml --target x86_64-apple-darwin & wait) && cp cli/target/aarch64-apple-darwin/release/browser4 bin/browser4-darwin-arm64 && cp cli/target/x86_64-apple-darwin/release/browser4 bin/browser4-darwin-x64",
     "build:windows": "npm run version:sync && docker compose -f docker/docker-compose.yml run --rm build-windows",
-    "build:all-platforms": "npm run version:sync && (npm run build:linux & npm run build:windows & wait)",
+    "build:all-platforms": "npm run version:sync && (npm run build:linux & npm run build:windows & npm run build:macos & wait)",
     "build:docker": "docker build -t browser4-builder -f docker/Dockerfile.build .",
     "publish:if-needed": "node scripts/publish-if-needed.js",
     "release": "npm run publish:if-needed",