@omniaibot/win-x64 1.6.8 → 1.6.12

package/bin/omnibot-windows-x64/VERSION

@@ -1 +1 @@
-1.6.8 
+1.6.12 

package/bin/omnibot-windows-x64/omnibot/skills/omnibot/SKILL.md

@@ -57,9 +57,15 @@ omnibot --version
 If the current workspace contains `pyproject.toml` and the global command is not the workspace version, run the test commands as `uv run omnibot ...` (or use the workspace `.venv/bin/omnibot`) so the agent exercises the code under test. Do not infer command support from memory or from a different packaged version; run the relevant subcommand `--help` after selecting the runtime.
 
 For a source-checkout test, make this concrete before the first browser action:
+define the command prefix once:
+
+```bash
+export OMNIBOT="uv run omnibot --no-start"
+```
+
 stop any existing daemon, start the workspace daemon with
 `uv run python -m omnibot --api-port 18764 --ws-port 18765 daemon run`, and
-invoke every subsequent CLI command as `uv run omnibot --no-start ...` (or the
+invoke every subsequent CLI command as `$OMNIBOT ...` (or the
 workspace `.venv/bin/omnibot --no-start ...`). Never use bare `omnibot` in that
 workflow, because it can auto-start the globally installed packaged daemon and
 silently test different code.
@@ -71,6 +77,16 @@ Before any page-state operation, choose the narrowest native command that expres
 | Intent | Native command path | Do not start with |
 | --- | --- | --- |
 | Current page identity | `get title`, `get url`, then `snapshot -i` or `read` if needed | `execute-js "return document.title/location.href"` |
+| Browser history | `browser history [text]` (also `history search [text]`) | reading browser history through page JavaScript |
+| Bookmarks | `browser bookmarks` (also `bookmarks tree`) | reading the browser bookmark tree |
+| Downloads | `browser downloads [terms]` (also `downloads search [terms]`, `downloads open <id>`) | inspecting/opening browser downloads |
+| Recently closed | `browser recently-closed` | reading recently closed tabs/windows |
+| Top sites | `browser top-sites` | reading browser frequent/top sites |
+| Installed extensions | `browser extensions` | listing extension name, ID, enabled state, type, and version (read-only) |
+| Content settings | `browser content-settings [TYPE] [URL]` | reading site permission state, default `automaticDownloads` (read-only) |
+| Mouse visual state | `browser mouse-visual-state --tab-id <TAB_ID>` | verifying the extension cursor overlay and asset load state (read-only diagnostic) |
+| Notifications | `browser notify <title> [message]` | showing a browser notification |
+| Tab groups | `tab group <TAB_ID> [NAME]`, `tab group-info <TAB_ID>`, `tab ungroup <TAB_ID>` | grouping and inspecting browser tabs |
 | Page text/content | `read --screens N`, `get text`, `get html`, `get attr`, `get count` | `execute-js "document.body.innerText"` |
 | Iframe context | `frame <iframe-id/name/title/src>`, then `snapshot`/page-state commands | Assuming host selectors reach child frames |
 | Click | `find ... --action click` -> `snapshot -i` + `click @eN` -> `click selector` -> `dom click` -> `mouse click` | `document.querySelector(...).click()` |
@@ -82,6 +98,14 @@ If you are about to write `document.querySelector`, `document.body.innerText`, `
 
 Text entry rule: when the user asks to enter, type, fill, or replace text, use `fill` or `type`. `keyboard` and `press` dispatch key events for shortcuts and key semantics; they are not text insertion commands and must not be substituted for text entry.
 
+Form commit rule: after `fill`, `type`, `select`, `check`, `uncheck`, or `upload`, verify the control itself with `get value`, `get attr`, or `is` before judging the workflow. Many pages keep a separate saved/rendered status that does not change until a visible `Save`, `Apply`, `Submit`, or equivalent control is activated. If such a commit control exists and the task expects business-state verification, observe it, activate it with a native command, then verify the resulting status. Do not report a form action as failed merely because an unrelated status element is still stale, and do not report it as complete without verifying both the control value and any required commit result.
+
+Form terminal-timeout rule: keep form workflows as short observe → one control action → one verification steps. If a terminal call times out after `fill`, `select`, `check`, or `Save`, query the browser state first; do not repeat the action blindly because the browser may already have applied it. Continue from the first unverified control and finish with one commit click plus one final status read.
+
+Mutation retry rule: issue at most one mutating browser command (`fill`, `type`, `select`, `check`, `uncheck`, `click`, `dblclick`, `drag`, `upload`, or `navigate`) per agent request. If its terminal output is empty, delayed, or times out, the next request must be read-only (`get`, `is`, `snapshot`, `read`, `tabs`, or `wait`) to determine whether the mutation already happened. Never replay the mutation merely because the tool response was missing; duplicate fills can append text and duplicate clicks can change business state.
+
+Drag performance rule: selector-based `drag` may take longer than a terminal tool's short default timeout because it dispatches a human-like event sequence. For agent workflows where exact trajectory is not required, prefer `snapshot -i` plus coordinate `mouse drag --fast` after verifying source/target boxes; then verify the target state. If semantic `drag` is required, allow a longer command timeout and do not retry blindly while the first drag may still be running.
+
 Clipboard rule: both `clipboard read` and `clipboard write` require the explicit
 `--tab-id <TAB_ID>` on every invocation, even though the clipboard is shared by
 the browser extension. If no tab id is known, discover a safe target tab first;
@@ -144,6 +168,7 @@ Targeting categories:
 Navigation safety invariant:
 
 - When the task says "open", "open a new tab", or "create a test page", use `open <URL>` or `tab new <URL>` and use the returned tab id. Do not substitute `goto`, plain `navigate`, or the currently active tab.
+- `navigate <URL>` also opens a new tab by default; `navigate <URL> --new-tab` is an explicit equivalent accepted for agent compatibility. Use `navigate <URL> --same-tab --tab-id <TAB_ID>` only for deliberate same-tab reuse.
 - Use `goto --tab-id <TAB_ID>` or `navigate --same-tab --tab-id <TAB_ID>` only when the task explicitly asks to reuse an existing tab.
 - After creating a tab, verify `get url --tab-id <TAB_ID>` before any clipboard, upload, form, or other page-state operation. This prevents a failed or misinterpreted tab-creation result from mutating a user-owned page.
 - Tool-created tabs are temporary workflow resources and may be auto-closed by cleanup after inactivity. Complete the intended tab workflow in the same token/context, keep the returned tab id, and do not assume it will still exist after a long model pause or a separate one-shot agent process. If a later close reports `not found`, re-list tabs before retrying; the cleanup may already have closed it.
@@ -189,12 +214,28 @@ Choose by intent before choosing by command name:
 | Intent | Prefer | Why |
 | --- | --- | --- |
 | Summarize or extract page content, article text, search results, feeds, or long/lazy pages | `read` | Returns clean text/Markdown for agent reasoning. |
-| Quickly observe current visible UI structure before deciding what to do next | `snapshot -i` | Returns actionable refs and page structure for routing. |
+| Quickly observe current visible UI structure before deciding what to do next | `snapshot -i` | Returns interactive refs plus visual-region refs for screenshot targeting. |
 | Find buttons, links, inputs, or refs for a later action | `find` or `snapshot -i` | Produces actionable targets; `read` does not. |
 | Verify one concrete condition | `get`, `is`, or `wait` | Narrow evidence is more reliable than dumping a page. |
 
 Do not use `read` as the first step for click/fill workflows unless the user explicitly asked for page content first.
 
+## Screenshots And Visual Regions
+
+`screenshot --tab-id <TAB_ID>` without `--ref` remains a full current-viewport screenshot.
+For a specific visual region, run `snapshot -i` and choose a ref marked `[visual=true]`, such as
+`article`, `region`, `dialog`, `listitem`, `image`, or `video`:
+
+```bash
+OMNIBOT_SESSION_TOKEN=visual omnibot snapshot -i --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=visual omnibot screenshot --ref @e146 --tab-id <TAB_ID> -o /tmp/region.png
+```
+
+Visual-region refs are intended for complete content containers, cards, posts, dialogs, media, and similar areas.
+Do not screenshot a button, link, or textbox as a standalone target unless the user explicitly asks for that control.
+The ref screenshot path re-resolves the live DOM node, scrolls it into view, waits for layout to settle, and then captures its region.
+After `read` or scrolling, refresh the snapshot before using a stale ref. Raw `get box` plus `cdp Page.captureScreenshot` is not the normal screenshot workflow.
+
 ## Dropdowns And Custom Comboboxes
 
 `snapshot -i` may auto-probe visible custom comboboxes and append their options as refs. If the output lists `@eN [option] "..."`, click that option ref directly; Omnibot will reopen the owning combobox through `openerSelector` if the dropdown closed during snapshot.

package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/command-reference.md

@@ -112,7 +112,7 @@ Fallback relation: Prefer for page content. Use `snapshot -i` when the next step
 
 ### snapshot -i
 
-Purpose: Read accessibility tree and produce actionable `@eN` refs.
+Purpose: Read accessibility tree and produce actionable `@eN` refs. With `-i`, it includes both interactive refs and visual-region refs marked `[visual=true]` for targeted screenshots.
 
 For visible custom comboboxes, `snapshot -i` may auto-probe options and append `[option]` refs under DOM Popup Controls. These refs remain actionable after the dropdown closes because they carry `openerSelector` internally.
 
@@ -351,6 +351,13 @@ Fallback relation: Tier 1. Explain failure before using refs, selectors, DOM, mo
 ### mouse
 
 Purpose: Coordinate-based mouse operations.
+
+All coordinate mouse operations are mirrored in the connected extension's
+visible page: a cursor, movement trail, pressed state, drag path, and click
+wave are shown at the same viewport coordinates as the dispatched input. The
+visual layer is best-effort and never changes whether the underlying action
+succeeds. It is recreated after a page reload when the last mouse position is
+known.
 Usage tier: Tier 5 fallback only.
 
 > **Development status:** The realistic trajectory mode (`mouse drag` without `--fast`) is experimental and has not been validated in production workflows. The trajectory parameters, step count, and timing are still being tuned. Use `--fast` for stable linear drags until the realistic mode is proven.
@@ -556,9 +563,16 @@ Purpose: Capture visual evidence from a browser tab.
 
 ```bash
 OMNIBOT_SESSION_TOKEN=debug-checkout omnibot screenshot --tab-id <TAB_ID> -o /tmp/omni-shot.png
+OMNIBOT_SESSION_TOKEN=debug-checkout omnibot snapshot -i --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=debug-checkout omnibot screenshot --ref @e146 --tab-id <TAB_ID> -o /tmp/omni-region.png
 OMNIBOT_SESSION_TOKEN=debug-checkout omnibot screenshot --annotate --tab-id <TAB_ID> -o /tmp/omni-annotated.png
 ```
 
+Without `--ref`, screenshot captures the current viewport. `--ref` accepts a visual ref marked
+`[visual=true]`, such as `[article]`, `[region]`, `[dialog]`, `[listitem]`, `[image]`, or `[video]`.
+Do not use ordinary button/link/textbox refs as standalone screenshot targets. Re-run `snapshot -i`
+after scrolling or `read` before using a ref that may be stale.
+
 Preferred pattern: See `debugging-and-evidence.md#screenshot`.
 Fallback relation: Evidence collection, not operation fallback.
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@omniaibot/win-x64",
-  "version": "1.6.8",
+  "version": "1.6.12",
   "description": "omnibot executable for Windows x64",
   "os": [
     "win32"