@omniaibot/win-x64 1.6.13 → 1.6.15

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1 +1 @@
1
- 1.6.13
1
+ 1.6.15
@@ -40,6 +40,8 @@ This skill is an execution specification for agents. It is not a human command m
40
40
  - For workflows with more than five meaningful browser actions, split the work into multiple bounded turns. Each turn should finish its own observe -> act -> verify cycle and report the retained tab id before the next turn; do not put a long mutation chain and all final checks into one model request.
41
41
  - Raw `cdp` calls are one-shot. Do not assume `Network.enable` alone creates readable logs. For API/request evidence, use `network clear -> network start -> one action -> network stop -> network logs/summary`.
42
42
  - After a failed click by ref, re-observe or use a higher-evidence fallback. Do not keep using stale `@eN` refs from a previous snapshot.
43
+ - Semantic and ref clicks automatically re-resolve the live target, scroll it into view, wait for stable geometry, and verify that it receives pointer events before clicking. Do not manually scroll merely because a known target's snapshot box is off-screen.
44
+ - Manually scroll before target discovery only when the target is not yet represented in the DOM/snapshot, such as virtualized lists, infinite feeds, collapsed regions, or lazy-rendered content. After it appears, use a normal atomic click rather than coordinate clicking.
43
45
  - For shopping, checkout, payment, banking, or irreversible workflows: Do not click final submit/pay/place-order controls unless the user explicitly confirms that final action in the current turn.
44
46
  - Prefer condition-based `wait` over shell sleep.
45
47
  - Timeout values for commands that expose a timeout option, such as `wait`, are seconds, not milliseconds. Use `--timeout 5` for a five-second wait; never convert milliseconds into a raw CLI timeout value. `dialog logs` and `dialog clear` do not accept `--timeout`.
@@ -47,30 +49,19 @@ This skill is an execution specification for agents. It is not a human command m
47
49
  - Parse JSON output from commands that return JSON.
48
50
  - Do not claim success without verification evidence.
49
51
 
50
- ### Runtime Provenance
52
+ ### Runtime
51
53
 
52
- Before an agent-driven test in a source checkout, verify the command provenance:
54
+ Use the installed `omnibot` command that is available to the user. Before an
55
+ agent-driven browser workflow, verify that the CLI is available and inspect the
56
+ relevant command help when needed:
53
57
 
54
58
  ```bash
55
59
  command -v omnibot
56
60
  omnibot --version
57
61
  ```
58
62
 
59
- If the current workspace contains `pyproject.toml` and the global command is not the workspace version, run the test commands as `uv run omnibot ...` (or use the workspace `.venv/bin/omnibot`) so the agent exercises the code under test. Do not infer command support from memory or from a different packaged version; run the relevant subcommand `--help` after selecting the runtime.
60
-
61
- For a source-checkout test, make this concrete before the first browser action:
62
- define the command prefix once:
63
-
64
- ```bash
65
- export OMNIBOT="uv run omnibot --no-start"
66
- ```
67
-
68
- stop any existing daemon, start the workspace daemon with
69
- `uv run python -m omnibot --api-port 18764 --ws-port 18765 daemon run`, and
70
- invoke every subsequent CLI command as `$OMNIBOT ...` (or the
71
- workspace `.venv/bin/omnibot --no-start ...`). Never use bare `omnibot` in that
72
- workflow, because it can auto-start the globally installed packaged daemon and
73
- silently test different code.
63
+ The CLI manages its local daemon automatically. Do not depend on a source
64
+ checkout, Python environment, repository files, or developer-only commands.
74
65
 
75
66
  ## Native Command Router
76
67
 
@@ -203,6 +194,8 @@ OMNIBOT_SESSION_TOKEN=checkout omnibot get url --tab-id <TAB_ID>
203
194
 
204
195
  If verification fails, do not repeat blindly. Re-observe, choose the next fallback tier, and verify again.
205
196
 
197
+ Click refs are live-node actions, not cached-coordinate actions. A successful `click @eN` may report `auto_scrolled`, `before_box`, `clicked_box`, and `hit_test`; use those fields as evidence that an off-screen or nested-container target was brought into the viewport safely. Do not insert a manual `scroll -> snapshot` sequence between selecting a known ref and clicking it.
198
+
206
199
  ## Pattern > Command
207
200
 
208
201
  Start from the task, not the command name. Use operation patterns for read, click, fill, select/check, navigation, wait, extraction, and batch.
@@ -32,6 +32,7 @@
32
32
  | Unsafe checkout capture | Clicking through all checkout controls until an order is placed is irreversible and may charge the user. | Capture only the transition into confirmation/preview pages, stop before final submit/pay/place-order controls, and report only non-sensitive request metadata. |
33
33
  | Revealing sensitive network data | Cookies, auth headers, full addresses, phone numbers, and payment data must not be pasted into reports. | Report host, path, method, status, and non-sensitive payload shape only. |
34
34
  | Using stale `@eN` refs after a failed click | Refs may have expired or the page may have changed. | Re-observe with `snapshot -i` or use a higher-evidence fallback. |
35
+ | Manually scrolling a known ref before every click | It adds an unnecessary mutation, can invalidate refs, and may scroll the wrong nested container. | Click the semantic target or `@eN` directly; Omnibot atomically scrolls, reboxes, hit-tests, and clicks live DOM targets. Manually scroll only to discover virtualized or lazy-rendered content that is not yet in the snapshot. |
35
36
  | `Network.enable` without log retrieval | Raw `cdp` calls are one-shot and do not persist events for later reading. | Use `network clear -> network start -> one action -> network stop -> network logs/summary`. |
36
37
  | Writing the article body into the title field | Rich text article body editors are often missing from the AX tree; filling the only visible `[textbox]` (title) overwrites the wrong control. | Run `snapshot -i`, find the body under `# DOM Rich Text Editors` as `@eN [richtext]`, then `fill @richtext "..."`. |
37
38
  | `execute-js` for rich text body before `fill @richtext` | ContentEditable writes need event dispatch and framework integration; ad-hoc `innerHTML` bypasses Omnibot's supported path. | Use `fill @richtext "p1\n\np2"` or `type @richtext " text"` first. |
@@ -49,6 +50,7 @@ Do not manually reopen a dropdown just to discover options if `snapshot -i` alre
49
50
  - "Clicking with JS is equivalent to a user click."
50
51
  - "Scrolling with JS is harmless."
51
52
  - "The ref probably still points to the same element."
53
+ - "The target is off-screen, so I must manually scroll before clicking its ref."
52
54
  - "A screenshot is enough even though I need text."
53
55
  - "Sleep should be fine."
54
56
  - "The trigger clicked, so another `find text` should eventually select the option."
@@ -103,6 +103,10 @@ OMNIBOT_SESSION_TOKEN=checkout omnibot get url --tab-id <TAB_ID>
103
103
  OMNIBOT_SESSION_TOKEN=checkout omnibot is visible ".success" --tab-id <TAB_ID>
104
104
  ```
105
105
 
106
+ `find ... --action click`, `click @eN`, and selector clicks own target preparation. For a ref backed by a live DOM node, Omnibot scrolls the correct page or nested scroll container, waits for stable geometry, reacquires the click box, and performs a pointer hit-test before dispatching the click. Do not manually scroll a known target merely because its snapshot box is outside the current viewport.
107
+
108
+ When returned, `auto_scrolled`, `before_box`, `clicked_box`, and `hit_test` are click evidence. `auto_scrolled=true` means target preparation changed its viewport geometry; `hit_test=true` means the final point was verified to receive pointer events.
109
+
106
110
  ### Fallback entry point
107
111
 
108
112
  If semantic and refs fail, document why and continue at `fallback-operations.md#selector-fallback`.
@@ -254,6 +258,8 @@ If native select/check fails, use `fallback-operations.md#selector-fallback`. Us
254
258
 
255
259
  Use Scroll when content, controls, lazy-loaded lists, dropdown panels, or editor areas are off-screen.
256
260
 
261
+ Do not use Scroll as a mandatory pre-step for a target that is already represented by a semantic locator, selector, or `@eN` ref. Normal click preparation scrolls that known live target atomically. Use explicit scrolling to discover content that is not yet present in the DOM/snapshot, or when the task itself is scrolling rather than clicking.
262
+
257
263
  ### Preferred sequence
258
264
 
259
265
  1. `scroll down|up|left|right <pixels>` for page scroll.
@@ -263,6 +269,8 @@ Use Scroll when content, controls, lazy-loaded lists, dropdown panels, or editor
263
269
  5. `mouse scroll` only when element/container scroll commands fail.
264
270
  6. `execute-js` scroll only after the native tiers fail.
265
271
 
272
+ For virtualized lists, infinite feeds, or lazy-rendered content where the target does not exist yet, scroll the relevant container by about 70% of its visible height, take a fresh snapshot, and stop as soon as the target appears. Keep this loop bounded. Once discovered, click the new target normally and let click preparation handle final positioning and hit-testing.
273
+
266
274
  ### Example
267
275
 
268
276
  ```bash
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@omniaibot/win-x64",
3
- "version": "1.6.13",
3
+ "version": "1.6.15",
4
4
  "description": "omnibot executable for Windows x64",
5
5
  "os": [
6
6
  "win32"