@omniaibot/win-x64 1.6.6 → 1.6.8
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/bin/omnibot-windows-x64/VERSION +1 -1
- package/bin/omnibot-windows-x64/_cffi_backend.pyd +0 -0
- package/bin/omnibot-windows-x64/ada92cb5d92a588d1b93__mypyc.pyd +0 -0
- package/bin/omnibot-windows-x64/charset_normalizer/cd.pyd +0 -0
- package/bin/omnibot-windows-x64/charset_normalizer/md.pyd +0 -0
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/SKILL.md +66 -42
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/anti-patterns.md +8 -0
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/command-reference.md +30 -49
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/debugging-and-evidence.md +6 -0
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/fallback-operations.md +19 -0
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/operation-patterns.md +67 -3
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/runtime-and-status.md +4 -1
- package/bin/omnibot-windows-x64/omnibot-windows-x64.exe +0 -0
- package/package.json +1 -1
- package/bin/omnibot-windows-x64/81d243bd2c585b0f4821__mypyc.pyd +0 -0
|
@@ -1 +1 @@
|
|
|
1
|
-
1.6.
|
|
1
|
+
1.6.8
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
@@ -29,17 +29,66 @@ This skill is an execution specification for agents. It is not a human command m
|
|
|
29
29
|
- Pattern > Command.
|
|
30
30
|
- Observe -> Act -> Verify.
|
|
31
31
|
- Fallback is allowed, but never first.
|
|
32
|
-
- Do not use `execute-js` first.
|
|
32
|
+
- Native Omnibot commands before scripts. Do not use `execute-js` first.
|
|
33
33
|
- Run `omnibot <command> --help` before using uncommon or newly introduced commands, especially `batch`, `network`, `cdp`, `record`, `trace`, and `skills`.
|
|
34
34
|
- If a command from memory fails with parser errors, stop using that remembered syntax and re-check `omnibot --help` plus the subcommand help.
|
|
35
35
|
- Prefer semantic locators and `snapshot -i` refs before selectors.
|
|
36
|
+
- When the user has explicitly requested a native Omnibot command and the terminal tool is available, execute it immediately; do not ask for confirmation or claim that it was not run. If dispatch genuinely fails, report the concrete tool error after one bounded retry and do not fabricate browser results.
|
|
37
|
+
- A multi-step browser task is incomplete until its requested final verification command has actually run. After the last mutation, always issue the concrete `get`/`is`/`wait` verification, inspect its result, and only then report completion; never stop after reporting that the mutation itself succeeded.
|
|
38
|
+
- For workflows with more than five meaningful browser actions, split the work into multiple bounded turns. Each turn should finish its own observe -> act -> verify cycle and report the retained tab id before the next turn; do not put a long mutation chain and all final checks into one model request.
|
|
36
39
|
- Raw `cdp` calls are one-shot. Do not assume `Network.enable` alone creates readable logs. For API/request evidence, use `network clear -> network start -> one action -> network stop -> network logs/summary`.
|
|
37
40
|
- After a failed click by ref, re-observe or use a higher-evidence fallback. Do not keep using stale `@eN` refs from a previous snapshot.
|
|
38
41
|
- For shopping, checkout, payment, banking, or irreversible workflows: Do not click final submit/pay/place-order controls unless the user explicitly confirms that final action in the current turn.
|
|
39
42
|
- Prefer condition-based `wait` over shell sleep.
|
|
43
|
+
- Timeout values for commands that expose a timeout option, such as `wait`, are seconds, not milliseconds. Use `--timeout 5` for a five-second wait; never convert milliseconds into a raw CLI timeout value. `dialog logs` and `dialog clear` do not accept `--timeout`.
|
|
44
|
+
- Native dialog workflows are session-token scoped: use the exact same `OMNIBOT_SESSION_TOKEN` for the triggering click, `dialog logs`, and `dialog handle`. A different token cannot observe or handle the original dialog.
|
|
40
45
|
- Parse JSON output from commands that return JSON.
|
|
41
46
|
- Do not claim success without verification evidence.
|
|
42
47
|
|
|
48
|
+
### Runtime Provenance
|
|
49
|
+
|
|
50
|
+
Before an agent-driven test in a source checkout, verify the command provenance:
|
|
51
|
+
|
|
52
|
+
```bash
|
|
53
|
+
command -v omnibot
|
|
54
|
+
omnibot --version
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
If the current workspace contains `pyproject.toml` and the global command is not the workspace version, run the test commands as `uv run omnibot ...` (or use the workspace `.venv/bin/omnibot`) so the agent exercises the code under test. Do not infer command support from memory or from a different packaged version; run the relevant subcommand `--help` after selecting the runtime.
|
|
58
|
+
|
|
59
|
+
For a source-checkout test, make this concrete before the first browser action:
|
|
60
|
+
stop any existing daemon, start the workspace daemon with
|
|
61
|
+
`uv run python -m omnibot --api-port 18764 --ws-port 18765 daemon run`, and
|
|
62
|
+
invoke every subsequent CLI command as `uv run omnibot --no-start ...` (or the
|
|
63
|
+
workspace `.venv/bin/omnibot --no-start ...`). Never use bare `omnibot` in that
|
|
64
|
+
workflow, because it can auto-start the globally installed packaged daemon and
|
|
65
|
+
silently test different code.
|
|
66
|
+
|
|
67
|
+
## Native Command Router
|
|
68
|
+
|
|
69
|
+
Before any page-state operation, choose the narrowest native command that expresses the intent. JavaScript is not a convenience shortcut; it is a fallback that requires evidence that native commands could not reach the state.
|
|
70
|
+
|
|
71
|
+
| Intent | Native command path | Do not start with |
|
|
72
|
+
| --- | --- | --- |
|
|
73
|
+
| Current page identity | `get title`, `get url`, then `snapshot -i` or `read` if needed | `execute-js "return document.title/location.href"` |
|
|
74
|
+
| Page text/content | `read --screens N`, `get text`, `get html`, `get attr`, `get count` | `execute-js "document.body.innerText"` |
|
|
75
|
+
| Iframe context | `frame <iframe-id/name/title/src>`, then `snapshot`/page-state commands | Assuming host selectors reach child frames |
|
|
76
|
+
| Click | `find ... --action click` -> `snapshot -i` + `click @eN` -> `click selector` -> `dom click` -> `mouse click` | `document.querySelector(...).click()` |
|
|
77
|
+
| Scroll | `scroll`, `scrollintoview`, `dom scroll`, then `mouse scroll` | `window.scrollTo`, `element.scrollIntoView()` |
|
|
78
|
+
| Fill/type | Text entry: `find ... --action fill`, `fill`, or `type`; key semantics: `press`/`keyboard` | setting `.value` or `innerHTML` in JS |
|
|
79
|
+
| Wait | `wait` by selector/text/url/load/fn | shell sleep or JS timers |
|
|
80
|
+
|
|
81
|
+
If you are about to write `document.querySelector`, `document.body.innerText`, `.click()`, `.value =`, `innerHTML`, `scrollTo`, or `scrollIntoView`, stop and identify the native command you will try first. Only use `execute-js` after documenting which native tiers failed and how you will verify the result.
|
|
82
|
+
|
|
83
|
+
Text entry rule: when the user asks to enter, type, fill, or replace text, use `fill` or `type`. `keyboard` and `press` dispatch key events for shortcuts and key semantics; they are not text insertion commands and must not be substituted for text entry.
|
|
84
|
+
|
|
85
|
+
Clipboard rule: both `clipboard read` and `clipboard write` require the explicit
|
|
86
|
+
`--tab-id <TAB_ID>` on every invocation, even though the clipboard is shared by
|
|
87
|
+
the browser extension. If no tab id is known, discover a safe target tab first;
|
|
88
|
+
never omit the flag or infer a default tab.
|
|
89
|
+
|
|
90
|
+
Cross-origin frame rule: snapshot visibility does not guarantee write access. After targeting a cross-origin iframe, verify `fill`/`type` focus and the resulting value; if focus fails, report the limitation explicitly and do not use JavaScript as a bypass.
|
|
91
|
+
|
|
43
92
|
## Workflow Context + Tab Target
|
|
44
93
|
|
|
45
94
|
Every page-state workflow must use both controls:
|
|
@@ -71,6 +120,10 @@ The agent dispatch pattern is token + tab-id:
|
|
|
71
120
|
4. Pass `--tab-id <TAB_ID>` on every command that touches page state.
|
|
72
121
|
5. Run observe -> act -> verify on that same tab.
|
|
73
122
|
|
|
123
|
+
Tab-id identity rule: use the exact `id` field returned by Omnibot `tabs`/`tab list` or by a tab-creation command. Browser-control surfaces may expose a shorter numeric `tab_id`; that value is not necessarily the Omnibot session id. When the discovery output contains both `tab_id` and `id`, pass the full namespaced `id` (for example, `edge-...:12345`) to `--tab-id`. Never reconstruct or strip the namespace.
|
|
124
|
+
|
|
125
|
+
Phased workflow rule: do not bundle navigation and several page actions into one unverified sequence. After creating or navigating a tab, first run `get url` (and `snapshot -i` when controls are involved); then perform one meaningful action; then verify its concrete state before issuing the next action. For a form or multi-control task, repeat this observe -> one action -> verify cycle for each control.
|
|
126
|
+
|
|
74
127
|
Examples of tab-locked page-state commands:
|
|
75
128
|
|
|
76
129
|
```bash
|
|
@@ -88,6 +141,14 @@ Targeting categories:
|
|
|
88
141
|
| Discovery or tab creation: `tabs`, `tab list`, `tab new <URL>`, `open <URL>`, `navigate <URL>` | Return or discover the tab id; save it for later page-state commands. |
|
|
89
142
|
| Read URL exception: `read <URL>` | Opens a temporary read tab and does not need `--tab-id`; reading an existing tab still requires `read --tab-id <TAB_ID>`. |
|
|
90
143
|
|
|
144
|
+
Navigation safety invariant:
|
|
145
|
+
|
|
146
|
+
- When the task says "open", "open a new tab", or "create a test page", use `open <URL>` or `tab new <URL>` and use the returned tab id. Do not substitute `goto`, plain `navigate`, or the currently active tab.
|
|
147
|
+
- Use `goto --tab-id <TAB_ID>` or `navigate --same-tab --tab-id <TAB_ID>` only when the task explicitly asks to reuse an existing tab.
|
|
148
|
+
- After creating a tab, verify `get url --tab-id <TAB_ID>` before any clipboard, upload, form, or other page-state operation. This prevents a failed or misinterpreted tab-creation result from mutating a user-owned page.
|
|
149
|
+
- Tool-created tabs are temporary workflow resources and may be auto-closed by cleanup after inactivity. Complete the intended tab workflow in the same token/context, keep the returned tab id, and do not assume it will still exist after a long model pause or a separate one-shot agent process. If a later close reports `not found`, re-list tabs before retrying; the cleanup may already have closed it.
|
|
150
|
+
- If `open` or `tab new` times out, loses its final response, or is retried after an agent/tool interruption, first run `tabs` or `tab list` and match the requested URL. Reuse one matching tab id instead of issuing another create command. Only create again after confirming that no matching tab from the interrupted attempt exists.
|
|
151
|
+
|
|
91
152
|
## Observe -> Act -> Verify
|
|
92
153
|
|
|
93
154
|
All page operations must follow this loop:
|
|
@@ -160,6 +221,7 @@ Do not use `execute-js` first for rich text writes; `fill`/`type` on the `[richt
|
|
|
160
221
|
| Reading pages | `references/operation-patterns.md#read` |
|
|
161
222
|
| Clicking | `references/operation-patterns.md#click` |
|
|
162
223
|
| Filling forms | `references/operation-patterns.md#fill` |
|
|
224
|
+
| Scrolling | `references/operation-patterns.md#scroll` |
|
|
163
225
|
| Select / check | `references/operation-patterns.md#select--check` |
|
|
164
226
|
| Navigation | `references/operation-patterns.md#navigation` |
|
|
165
227
|
| Waiting | `references/operation-patterns.md#wait` |
|
|
@@ -186,47 +248,6 @@ omnibot license status
|
|
|
186
248
|
|
|
187
249
|
If the extension is not connected, open Chrome or Edge with the omnibot extension loaded and keep an HTTP/HTTPS tab open.
|
|
188
250
|
|
|
189
|
-
## Site Memory Reuse
|
|
190
|
-
|
|
191
|
-
Before operating a complex site, check whether a generated site skill is installed for the current domain.
|
|
192
|
-
|
|
193
|
-
1. Run `omnibot skills list-sites` to see available generated site skills.
|
|
194
|
-
2. If a matching site skill is installed (e.g., `omnibot-xhs`), read its `SKILL.md` and use it for the operation.
|
|
195
|
-
3. If no matching site skill is installed, create one with:
|
|
196
|
-
```bash
|
|
197
|
-
omnibot explore run <url> --name <site> --goal "<explicit goal>"
|
|
198
|
-
omnibot skills install --agent opencode --site <site>
|
|
199
|
-
```
|
|
200
|
-
4. Generated site skills are not auto-loaded from `~/.omnibot/skills/`; they must be installed into the current agent's skill directory.
|
|
201
|
-
|
|
202
|
-
## Generated Site Skill Installation
|
|
203
|
-
|
|
204
|
-
Generated site skills are stored under `~/.omnibot/skills/omnibot-<name>/` but must be installed into each agent's native skill path to be auto-loaded.
|
|
205
|
-
|
|
206
|
-
List available generated site skills:
|
|
207
|
-
|
|
208
|
-
```bash
|
|
209
|
-
omnibot skills list-sites
|
|
210
|
-
```
|
|
211
|
-
|
|
212
|
-
Install one generated site skill:
|
|
213
|
-
|
|
214
|
-
```bash
|
|
215
|
-
omnibot skills install --agent opencode --site xhs
|
|
216
|
-
omnibot skills install --agent claude --site xhs
|
|
217
|
-
omnibot skills install --agent codex --site xhs
|
|
218
|
-
```
|
|
219
|
-
|
|
220
|
-
Install all generated site skills:
|
|
221
|
-
|
|
222
|
-
```bash
|
|
223
|
-
omnibot skills install --agent opencode --all-sites
|
|
224
|
-
```
|
|
225
|
-
|
|
226
|
-
Default `omnibot skills install --agent <agent>` installs only the official packaged omnibot skill, not generated site skills.
|
|
227
|
-
|
|
228
|
-
Local site memory is reusable strategy, not user task output. Do not write user inputs, scraped records, or private task results into generated site packages.
|
|
229
|
-
|
|
230
251
|
## Fallback Discipline
|
|
231
252
|
|
|
232
253
|
Fallback tiers are for completing operations after better patterns fail. They are not debugging shortcuts.
|
|
@@ -243,6 +264,8 @@ Tier order:
|
|
|
243
264
|
|
|
244
265
|
When entering a lower tier, state why the higher tier failed. After any fallback action, verify with `snapshot`, `get`, `is`, `wait`, or equivalent evidence.
|
|
245
266
|
|
|
267
|
+
Before `execute-js`, the agent must have native-attempt evidence: the exact `find`/`snapshot`/`click`/`fill`/`scroll`/`get`/`read` commands tried, their result, why they were insufficient, and the post-JS verification command to run. Without that evidence, use a native command instead.
|
|
268
|
+
|
|
246
269
|
## Debug Evidence
|
|
247
270
|
|
|
248
271
|
Debugging is for evidence. Fallback is for completing operations.
|
|
@@ -254,6 +277,7 @@ Use screenshots, annotated screenshots, console logs, network logs, trace, recor
|
|
|
254
277
|
- Missing `OMNIBOT_SESSION_TOKEN` or missing `--tab-id` on page-state commands.
|
|
255
278
|
- Acting without verify or claiming success without evidence.
|
|
256
279
|
- Using `execute-js` first, raw CSS before semantic find, `@eN` across tabs, implicit tab targeting, or shell sleep instead of `omnibot wait`.
|
|
280
|
+
- Using JavaScript read/click/scroll shortcuts when native `get`/`read`/`find`/`snapshot`/`click`/`scroll` can express the task.
|
|
257
281
|
- Using removed commands: `switch-tab`, `focus-tab`, `tab switch`, `tab focus`. These are no longer available.
|
|
258
282
|
- Using `tabs[0]` or the first tab as a target. The first tab is often a user tab or transport tab.
|
|
259
283
|
- Closing user tabs discovered during cleanup. Only close tabs created by the current workflow.
|
|
@@ -5,6 +5,11 @@
|
|
|
5
5
|
| Missing `OMNIBOT_SESSION_TOKEN` | Workflows can share implicit state across agents. | `OMNIBOT_SESSION_TOKEN=research omnibot snapshot -i --tab-id <TAB_ID>` |
|
|
6
6
|
| Missing `--tab-id` | The command may target the wrong page. | Pass `--tab-id <TAB_ID>` on every page-state command. |
|
|
7
7
|
| `execute-js` first | JavaScript bypasses standard browser interaction and hides better evidence. | Try semantic find, snapshot refs, selectors, DOM, and mouse first. For CSS-only dropdowns, use JavaScript only after the trigger clicked, options remain absent from `snapshot -i`, and higher tiers cannot complete open-then-select. |
|
|
8
|
+
| JS page detection (`document.title`, `location.href`) | It teaches agents to treat scripts as normal reads. | Use `get title` and `get url`. |
|
|
9
|
+
| JS text extraction (`document.body.innerText`) | It bypasses `read` cleanup, lazy-page handling, and targeted evidence. | Use `read --screens N` or `get text`. |
|
|
10
|
+
| JS click shortcut (`querySelector(...).click()`) | It can bypass pointer events, overlays, disabled state, and framework/user-intent safety checks. | Use `find ... --action click`, `snapshot -i` + `click @eN`, then selector/DOM/mouse fallback. |
|
|
11
|
+
| JS scroll shortcut (`window.scrollTo`, `scrollIntoView`) | It hides whether the scroll target/container is correct and bypasses Omnibot's action evidence path. | Use `scroll`, `scrollintoview`, `dom scroll`, or `mouse scroll`. |
|
|
12
|
+
| JS value or richtext writes | App frameworks may ignore value/HTML mutation without trusted events and editor integration. | Use `fill`, `type`, `press`, `keyboard`, and `[richtext]` refs. |
|
|
8
13
|
| Reusing `@eN` across tabs | Refs are tab-scoped and may point to a different element elsewhere. | Take a fresh `snapshot -i --tab-id <TAB_ID>` per tab. |
|
|
9
14
|
| Relying on implicit tab targeting | It is unavailable and unsafe for concurrent workflows. | Use explicit `--tab-id` on every page-state command. |
|
|
10
15
|
| Using sleep instead of wait | Fixed sleeps are flaky and either too short or too slow. | Use `omnibot wait --text`, `--url`, selector state, load state, or `--fn`. |
|
|
@@ -40,6 +45,9 @@ Do not manually reopen a dropdown just to discover options if `snapshot -i` alre
|
|
|
40
45
|
- "I can just use the current tab."
|
|
41
46
|
- "This is only one click, no need to verify."
|
|
42
47
|
- "JavaScript is faster."
|
|
48
|
+
- "I just need title/url/text, so JS is fine."
|
|
49
|
+
- "Clicking with JS is equivalent to a user click."
|
|
50
|
+
- "Scrolling with JS is harmless."
|
|
43
51
|
- "The ref probably still points to the same element."
|
|
44
52
|
- "A screenshot is enough even though I need text."
|
|
45
53
|
- "Sleep should be fine."
|
|
@@ -90,7 +90,7 @@ omnibot --version
|
|
|
90
90
|
omnibot -V
|
|
91
91
|
```
|
|
92
92
|
|
|
93
|
-
Output: a single line in the form `omnibot <version
|
|
93
|
+
Output: a single line in the form `omnibot <version>`.
|
|
94
94
|
Preferred pattern: See `runtime-and-status.md#version`.
|
|
95
95
|
Fallback relation: Not a fallback command.
|
|
96
96
|
|
|
@@ -150,6 +150,7 @@ Purpose: Check element state.
|
|
|
150
150
|
|
|
151
151
|
```bash
|
|
152
152
|
OMNIBOT_SESSION_TOKEN=checkout omnibot is visible "#submit" --tab-id <TAB_ID>
|
|
153
|
+
OMNIBOT_SESSION_TOKEN=checkout omnibot is hidden "#spinner" --tab-id <TAB_ID>
|
|
153
154
|
OMNIBOT_SESSION_TOKEN=checkout omnibot is enabled "#submit" --tab-id <TAB_ID>
|
|
154
155
|
OMNIBOT_SESSION_TOKEN=checkout omnibot is checked "#agree" --tab-id <TAB_ID>
|
|
155
156
|
```
|
|
@@ -325,6 +326,8 @@ OMNIBOT_SESSION_TOKEN=form omnibot keyup Shift --tab-id <TAB_ID>
|
|
|
325
326
|
Preferred pattern: Prefer `fill`, `type`, and `press` first.
|
|
326
327
|
Fallback relation: Keyboard commands are a focused fallback before mouse or JavaScript.
|
|
327
328
|
|
|
329
|
+
`keyboard type` preserves keyboard events and updates the focused editable when the browser does not apply the default text insertion. `keyboard inserttext` inserts text directly and intentionally does not emit per-character key events.
|
|
330
|
+
|
|
328
331
|
## Semantic Find
|
|
329
332
|
|
|
330
333
|
Semantic find is Tier 1 for operating by meaning.
|
|
@@ -434,7 +437,7 @@ Purpose: Run arbitrary JavaScript in the page.
|
|
|
434
437
|
Usage tier: Tier 6 fallback only. Do not use first.
|
|
435
438
|
|
|
436
439
|
```bash
|
|
437
|
-
OMNIBOT_SESSION_TOKEN=repair omnibot execute-js "return
|
|
440
|
+
OMNIBOT_SESSION_TOKEN=repair omnibot execute-js "return window.appState?.status" --tab-id <TAB_ID>
|
|
438
441
|
OMNIBOT_SESSION_TOKEN=repair omnibot execute-js --file /tmp/repair.js --tab-id <TAB_ID>
|
|
439
442
|
```
|
|
440
443
|
|
|
@@ -470,6 +473,8 @@ OMNIBOT_SESSION_TOKEN=research omnibot goto https://example.com --tab-id <TAB_ID
|
|
|
470
473
|
Preferred pattern: See `operation-patterns.md#navigation` and `session-and-tabs.md#tab-explicit`.
|
|
471
474
|
Fallback relation: Verify with `get url --tab-id <TAB_ID>` before continuing.
|
|
472
475
|
|
|
476
|
+
Safety rule: treat `open` and `tab new` as the only new-tab commands. Never replace a requested new-tab operation with `goto`, plain `navigate`, or an active-tab action. Use `goto --tab-id` or `navigate --same-tab --tab-id` only when reusing an explicitly selected existing tab, and verify the returned tab's URL before mutating page state.
|
|
477
|
+
|
|
473
478
|
### close / back / forward / reload / pushstate
|
|
474
479
|
|
|
475
480
|
Purpose: Change tab lifecycle or history state.
|
|
@@ -504,9 +509,13 @@ Purpose: Manage windows and frames.
|
|
|
504
509
|
|
|
505
510
|
```bash
|
|
506
511
|
OMNIBOT_SESSION_TOKEN=research omnibot window new
|
|
507
|
-
OMNIBOT_SESSION_TOKEN=research omnibot frame
|
|
512
|
+
OMNIBOT_SESSION_TOKEN=research omnibot frame "#payment-frame" --tab-id <TAB_ID>
|
|
513
|
+
OMNIBOT_SESSION_TOKEN=research omnibot snapshot -i --tab-id <TAB_ID>
|
|
514
|
+
OMNIBOT_SESSION_TOKEN=research omnibot frame main --tab-id <TAB_ID>
|
|
508
515
|
```
|
|
509
516
|
|
|
517
|
+
`frame` targets an iframe by id, name, title, or matching source URL. Set it before child-frame page-state commands and use `frame main` before reading host-page state.
|
|
518
|
+
|
|
510
519
|
## Waiting & Batch
|
|
511
520
|
|
|
512
521
|
### wait
|
|
@@ -566,6 +575,21 @@ OMNIBOT_SESSION_TOKEN=debug-checkout omnibot console clear --tab-id <TAB_ID>
|
|
|
566
575
|
Preferred pattern: See `debugging-and-evidence.md#console`.
|
|
567
576
|
Fallback relation: Evidence collection, not operation fallback. `console errors` currently uses the console log collection path; verify level filtering in output before treating it as error-only evidence.
|
|
568
577
|
|
|
578
|
+
### dialog
|
|
579
|
+
|
|
580
|
+
Purpose: Read, clear, or handle native browser JavaScript dialogs (`alert`, `confirm`, `prompt`).
|
|
581
|
+
|
|
582
|
+
```bash
|
|
583
|
+
OMNIBOT_SESSION_TOKEN=debug-checkout omnibot dialog logs --tab-id <TAB_ID>
|
|
584
|
+
OMNIBOT_SESSION_TOKEN=debug-checkout omnibot dialog clear --tab-id <TAB_ID>
|
|
585
|
+
OMNIBOT_SESSION_TOKEN=debug-checkout omnibot dialog handle accept --tab-id <TAB_ID>
|
|
586
|
+
OMNIBOT_SESSION_TOKEN=debug-checkout omnibot dialog handle dismiss --tab-id <TAB_ID>
|
|
587
|
+
OMNIBOT_SESSION_TOKEN=debug-checkout omnibot dialog handle accept --text "prompt text" --tab-id <TAB_ID>
|
|
588
|
+
```
|
|
589
|
+
|
|
590
|
+
Preferred pattern: Use `dialog logs` or `dialog clear` before triggering a browser-native dialog so the extension has debugger capture ready. Trigger through normal page actions when possible; use `execute-js` only when the task is explicitly to test a native JS dialog.
|
|
591
|
+
Fallback relation: Evidence and dialog handling, not a general page-operation fallback.
|
|
592
|
+
|
|
569
593
|
### network
|
|
570
594
|
|
|
571
595
|
Purpose: Capture and inspect CDP Network events buffered by the Omnibot browser extension.
|
|
@@ -691,6 +715,9 @@ omnibot skills install --agent hermes --profile nuwa
|
|
|
691
715
|
omnibot skills install --agent opencode
|
|
692
716
|
omnibot skills install --agent claude
|
|
693
717
|
omnibot skills install --agent codex
|
|
718
|
+
omnibot skills install --agent openclaw
|
|
719
|
+
omnibot skills install --agent workbuddy
|
|
720
|
+
omnibot skills install --agent trae
|
|
694
721
|
omnibot skills path
|
|
695
722
|
```
|
|
696
723
|
|
|
@@ -707,49 +734,3 @@ omnibot license status
|
|
|
707
734
|
|
|
708
735
|
Preferred pattern: See `runtime-and-status.md#startup-check`.
|
|
709
736
|
Fallback relation: Not a fallback command.
|
|
710
|
-
|
|
711
|
-
## Site Memory
|
|
712
|
-
|
|
713
|
-
### explore
|
|
714
|
-
|
|
715
|
-
Purpose: Explore one explicit site operation goal and generate reusable installable site skill under `~/.omnibot/skills/omnibot-<name>/`.
|
|
716
|
-
|
|
717
|
-
```bash
|
|
718
|
-
omnibot explore run https://www.xiaohongshu.com --name xhs --goal "发布图文笔记"
|
|
719
|
-
omnibot explore run https://x.com --goal "post a tweet"
|
|
720
|
-
omnibot explore list
|
|
721
|
-
omnibot explore show xhs
|
|
722
|
-
omnibot explore promote xhs --goal publish-note --version 2026-06-17T21-30-00Z
|
|
723
|
-
```
|
|
724
|
-
|
|
725
|
-
Rules:
|
|
726
|
-
|
|
727
|
-
- `--goal` is required for a new exploration.
|
|
728
|
-
- Same site and different goals are stored in the same site package.
|
|
729
|
-
- Same site and same goal create a new preserved version.
|
|
730
|
-
- Safe verified versions become active automatically.
|
|
731
|
-
- Dangerous verified versions remain candidates until promoted.
|
|
732
|
-
|
|
733
|
-
### skills list-sites
|
|
734
|
-
|
|
735
|
-
Purpose: List generated site skills available for installation.
|
|
736
|
-
|
|
737
|
-
```bash
|
|
738
|
-
omnibot skills list-sites
|
|
739
|
-
```
|
|
740
|
-
|
|
741
|
-
### skills install --site
|
|
742
|
-
|
|
743
|
-
Purpose: Install one generated site skill into an agent's skill directory.
|
|
744
|
-
|
|
745
|
-
```bash
|
|
746
|
-
omnibot skills install --agent opencode --site xhs
|
|
747
|
-
omnibot skills install --agent claude --site xhs
|
|
748
|
-
omnibot skills install --agent opencode --all-sites
|
|
749
|
-
```
|
|
750
|
-
|
|
751
|
-
Rules:
|
|
752
|
-
|
|
753
|
-
- `--site` installs one generated site skill by name.
|
|
754
|
-
- `--all-sites` installs all generated site skills.
|
|
755
|
-
- Without `--site` or `--all-sites`, only the official packaged omnibot skill is installed.
|
|
@@ -93,6 +93,12 @@ Trace is evidence. It does not replace operation verification.
|
|
|
93
93
|
|
|
94
94
|
Use record/replay when a flow must be captured or reproduced.
|
|
95
95
|
|
|
96
|
+
When a user asks for a complete recording or trace workflow, treat `start` as an
|
|
97
|
+
internal setup step: continue with the requested browser actions, run the
|
|
98
|
+
matching `stop -o <path>`, verify the artifact exists and contains events, and
|
|
99
|
+
only then report the result. Do not stop after `record start` or `trace start`
|
|
100
|
+
to ask what should be recorded when the user already specified the workflow.
|
|
101
|
+
|
|
96
102
|
```bash
|
|
97
103
|
OMNIBOT_SESSION_TOKEN=debug-checkout omnibot record start
|
|
98
104
|
OMNIBOT_SESSION_TOKEN=debug-checkout omnibot click --tab-id <TAB_ID> @e4
|
|
@@ -7,6 +7,15 @@ Every fallback escalation must answer two questions:
|
|
|
7
7
|
- Which higher tier failed, and why?
|
|
8
8
|
- How will the result be verified?
|
|
9
9
|
|
|
10
|
+
Before using `execute-js`, the answer must include native-attempt evidence:
|
|
11
|
+
|
|
12
|
+
- The native command path already tried (`find`, `snapshot -i`, `click`, `fill`, `scroll`, `get`, `read`, `wait`, `dom`, or `mouse`).
|
|
13
|
+
- The observed failure or missing state.
|
|
14
|
+
- Why the next native tier is insufficient.
|
|
15
|
+
- The exact post-JS verification command.
|
|
16
|
+
|
|
17
|
+
If the task is normal page identity, text extraction, click, scroll, fill/type, or wait, and you do not have that evidence, go back to `operation-patterns.md#native-first-decision-gate`.
|
|
18
|
+
|
|
10
19
|
## Fallback Tier Model
|
|
11
20
|
|
|
12
21
|
### Tier 1 Semantic
|
|
@@ -99,6 +108,16 @@ Mouse fallback is fragile across viewport size, scroll position, and responsive
|
|
|
99
108
|
|
|
100
109
|
Do not use `execute-js` first.
|
|
101
110
|
|
|
111
|
+
Do not use JavaScript as a shortcut for native commands:
|
|
112
|
+
|
|
113
|
+
| Shortcut | Native command instead |
|
|
114
|
+
| --- | --- |
|
|
115
|
+
| `return document.title` / `location.href` | `get title`, `get url` |
|
|
116
|
+
| `document.body.innerText` | `read --screens N`, `get text` |
|
|
117
|
+
| `querySelector(...).click()` | `find ... --action click`, `click @eN`, `click selector` |
|
|
118
|
+
| `.value = ...` or `innerHTML = ...` | `fill`, `type`, richtext refs |
|
|
119
|
+
| `window.scrollTo` / `scrollIntoView()` | `scroll`, `scrollintoview`, `dom scroll`, `mouse scroll` |
|
|
120
|
+
|
|
102
121
|
Use `execute-js` only when:
|
|
103
122
|
|
|
104
123
|
- Semantic find fails.
|
|
@@ -2,6 +2,22 @@
|
|
|
2
2
|
|
|
3
3
|
Use this file to choose behavior by task. Every page-state workflow must set `OMNIBOT_SESSION_TOKEN=<workflow-name>` and pass `--tab-id <TAB_ID>` on each page-state command.
|
|
4
4
|
|
|
5
|
+
## Native-First Decision Gate
|
|
6
|
+
|
|
7
|
+
Use native Omnibot commands for normal browser operations. Treat `execute-js` as a repair tool only after native attempts fail with evidence.
|
|
8
|
+
|
|
9
|
+
| If you need to... | Use first | JS shortcut to avoid |
|
|
10
|
+
| --- | --- | --- |
|
|
11
|
+
| Know where you are | `get title`, `get url` | `return document.title`, `location.href` |
|
|
12
|
+
| Read visible or rendered content | `read`, `get text`, `get html`, `get attr`, `get count` | `document.body.innerText` |
|
|
13
|
+
| Click a control | `find ... --action click`, `snapshot -i` + `click @eN`, `click selector` | `querySelector(...).click()` |
|
|
14
|
+
| Scroll | `scroll`, `scrollintoview`, `dom scroll`, `mouse scroll` | `window.scrollTo`, `scrollIntoView()` |
|
|
15
|
+
| Enter text | `find ... --action fill`, `fill`, `type` | `.value = ...`, `innerHTML = ...`, `keyboard`, `press` |
|
|
16
|
+
|
|
17
|
+
If a page task can be phrased as click, fill, type, press, scroll, wait, read, get, or is, use that command. Do not translate user-visible actions into JavaScript unless the fallback file says the higher tiers have failed.
|
|
18
|
+
|
|
19
|
+
Text versus keys: `fill`/`type` are the only native commands for inserting text and dispatching the page's input events. Use `press`/`keyboard` only for a key such as Enter, Escape, Tab, arrows, or a shortcut; raw keydown/keyup events do not guarantee that a focused text control receives characters.
|
|
20
|
+
|
|
5
21
|
## Read
|
|
6
22
|
|
|
7
23
|
### When to use
|
|
@@ -16,7 +32,7 @@ Use targeted state commands when the agent needs one value, visibility, enabled/
|
|
|
16
32
|
2. `read --screens N --tab-id <TAB_ID>` for longer or lazy-loaded pages.
|
|
17
33
|
3. `read <URL>` when opening a temporary tab only for reading that URL.
|
|
18
34
|
4. `get title/url/text/html/value/attr/count/box/styles` for narrow evidence.
|
|
19
|
-
5. `is visible/enabled/checked` for boolean state.
|
|
35
|
+
5. `is visible/hidden/enabled/checked` for boolean state.
|
|
20
36
|
5. `snapshot -i` when actionable refs are needed.
|
|
21
37
|
6. `dom visible` or `execute-js` fallback only after standard reads cannot access the state.
|
|
22
38
|
|
|
@@ -25,8 +41,12 @@ Use targeted state commands when the agent needs one value, visibility, enabled/
|
|
|
25
41
|
```bash
|
|
26
42
|
OMNIBOT_SESSION_TOKEN=research omnibot read --screens 5 --tab-id <TAB_ID>
|
|
27
43
|
OMNIBOT_SESSION_TOKEN=research omnibot read --screens 3 https://example.com/article
|
|
44
|
+
OMNIBOT_SESSION_TOKEN=research omnibot get title --tab-id <TAB_ID>
|
|
45
|
+
OMNIBOT_SESSION_TOKEN=research omnibot get url --tab-id <TAB_ID>
|
|
28
46
|
```
|
|
29
47
|
|
|
48
|
+
Do not use `execute-js "return document.title + location.href"` or `execute-js "return document.body.innerText"` for normal page detection or text extraction. Use `get` or `read`; they are the native evidence path.
|
|
49
|
+
|
|
30
50
|
### Verification
|
|
31
51
|
|
|
32
52
|
Use the narrowest read that proves the claim:
|
|
@@ -34,6 +54,7 @@ Use the narrowest read that proves the claim:
|
|
|
34
54
|
```bash
|
|
35
55
|
OMNIBOT_SESSION_TOKEN=research omnibot get title --tab-id <TAB_ID>
|
|
36
56
|
OMNIBOT_SESSION_TOKEN=research omnibot is visible "#main" --tab-id <TAB_ID>
|
|
57
|
+
OMNIBOT_SESSION_TOKEN=research omnibot is hidden "#spinner" --tab-id <TAB_ID>
|
|
37
58
|
OMNIBOT_SESSION_TOKEN=research omnibot get count ".result" --tab-id <TAB_ID>
|
|
38
59
|
```
|
|
39
60
|
|
|
@@ -86,6 +107,8 @@ OMNIBOT_SESSION_TOKEN=checkout omnibot is visible ".success" --tab-id <TAB_ID>
|
|
|
86
107
|
|
|
87
108
|
If semantic and refs fail, document why and continue at `fallback-operations.md#selector-fallback`.
|
|
88
109
|
|
|
110
|
+
Do not use `execute-js` for a visible button, link, menu item, tab, card, or modal control until the native click tiers above have failed and you have re-observed the page. A JS `.click()` can bypass pointer events, framework guards, disabled state, overlays, and user-intent safety checks.
|
|
111
|
+
|
|
89
112
|
### Static-text refs
|
|
90
113
|
|
|
91
114
|
When `snapshot -i` shows only `StaticText` for a visible control label, do not assume the text ref itself is clickable. Prefer, in order:
|
|
@@ -159,7 +182,7 @@ Insert an image after a specific paragraph:
|
|
|
159
182
|
```bash
|
|
160
183
|
OMNIBOT_SESSION_TOKEN=editor omnibot find text "第二段锚点文字" --action click --tab-id <TAB_ID>
|
|
161
184
|
OMNIBOT_SESSION_TOKEN=editor omnibot click ".syl-toolbar-tool.image .syl-toolbar-button" --tab-id <TAB_ID>
|
|
162
|
-
OMNIBOT_SESSION_TOKEN=editor omnibot wait
|
|
185
|
+
OMNIBOT_SESSION_TOKEN=editor omnibot wait ".upload-image-panel input[type=file]" --tab-id <TAB_ID>
|
|
163
186
|
OMNIBOT_SESSION_TOKEN=editor omnibot upload ".upload-image-panel input[type=file]" "/path/to/image.png" --tab-id <TAB_ID>
|
|
164
187
|
OMNIBOT_SESSION_TOKEN=editor omnibot find role button --name "确定" --action click --tab-id <TAB_ID>
|
|
165
188
|
```
|
|
@@ -170,6 +193,8 @@ Verify the inserted image by counting visible image cards, not raw `<img>` nodes
|
|
|
170
193
|
|
|
171
194
|
If fill does not update the application state, use `fallback-operations.md#javascript-fallback` only after selector/focus/type attempts fail. JavaScript must dispatch real `input` and `change` events.
|
|
172
195
|
|
|
196
|
+
Do not use JS `.value = ...`, `innerText`, or `innerHTML` for normal form fields or rich text. Native `fill`/`type` paths already handle events and rich text editor integration.
|
|
197
|
+
|
|
173
198
|
## Select / Check
|
|
174
199
|
|
|
175
200
|
### When to use
|
|
@@ -223,6 +248,31 @@ Do not loop on manual open -> snapshot -> open -> snapshot when `snapshot -i` al
|
|
|
223
248
|
|
|
224
249
|
If native select/check fails, use `fallback-operations.md#selector-fallback`. Use JavaScript only if events must be synthesized or if evidence shows a CSS-only multi-step widget cannot be completed through semantic, snapshot, selector, DOM, or mouse tiers.
|
|
225
250
|
|
|
251
|
+
## Scroll
|
|
252
|
+
|
|
253
|
+
### When to use
|
|
254
|
+
|
|
255
|
+
Use Scroll when content, controls, lazy-loaded lists, dropdown panels, or editor areas are off-screen.
|
|
256
|
+
|
|
257
|
+
### Preferred sequence
|
|
258
|
+
|
|
259
|
+
1. `scroll down|up|left|right <pixels>` for page scroll.
|
|
260
|
+
2. `scroll down|up|left|right <pixels> --selector <container>` for scrollable containers.
|
|
261
|
+
3. `scrollintoview @eN` or `scrollintoview selector` for a known target.
|
|
262
|
+
4. `dom scroll` when visible DOM node fallback is needed.
|
|
263
|
+
5. `mouse scroll` only when element/container scroll commands fail.
|
|
264
|
+
6. `execute-js` scroll only after the native tiers fail.
|
|
265
|
+
|
|
266
|
+
### Example
|
|
267
|
+
|
|
268
|
+
```bash
|
|
269
|
+
OMNIBOT_SESSION_TOKEN=read omnibot scroll down 800 --tab-id <TAB_ID>
|
|
270
|
+
OMNIBOT_SESSION_TOKEN=read omnibot scrollintoview @e7 --tab-id <TAB_ID>
|
|
271
|
+
OMNIBOT_SESSION_TOKEN=read omnibot snapshot -i --tab-id <TAB_ID>
|
|
272
|
+
```
|
|
273
|
+
|
|
274
|
+
Do not start with `execute-js "window.scrollTo(...)"` or `element.scrollIntoView()`. Native scroll commands preserve the agent's observe -> act -> verify loop and produce clearer failure modes.
|
|
275
|
+
|
|
226
276
|
## Navigation
|
|
227
277
|
|
|
228
278
|
### When to use
|
|
@@ -236,6 +286,8 @@ Use Navigation to open pages, create tabs, change history, reload, close tabs, m
|
|
|
236
286
|
3. Verify with `get url --tab-id <TAB_ID>`.
|
|
237
287
|
4. Run all later page-state commands with `--tab-id <TAB_ID>`.
|
|
238
288
|
|
|
289
|
+
Do not chain navigation, discovery, and multiple mutations before observing the result. After navigation, verify the URL and snapshot first; then issue one action and verify its state before continuing. Multi-control workflows must use one observe -> act -> verify cycle per control.
|
|
290
|
+
|
|
239
291
|
### Example
|
|
240
292
|
|
|
241
293
|
```bash
|
|
@@ -270,6 +322,18 @@ OMNIBOT_SESSION_TOKEN=research omnibot window new
|
|
|
270
322
|
OMNIBOT_SESSION_TOKEN=research omnibot frame main
|
|
271
323
|
```
|
|
272
324
|
|
|
325
|
+
Iframe workflow:
|
|
326
|
+
|
|
327
|
+
1. Verify the host tab URL and snapshot first.
|
|
328
|
+
2. Set the frame target explicitly, for example `frame #payment-frame --tab-id <TAB_ID>` (an iframe `id`, `name`, `title`, or matching `src` is accepted).
|
|
329
|
+
3. Re-observe the child context with `snapshot -i --tab-id <TAB_ID>` before filling or clicking.
|
|
330
|
+
4. Perform one child-frame action and verify its child-frame state with `get`, `is`, or `wait`.
|
|
331
|
+
5. Return to the host with `frame main --tab-id <TAB_ID>` before reading host-page state.
|
|
332
|
+
|
|
333
|
+
Do not assume a host-page selector reaches an iframe. If a child selector returns `null`, establish the frame target and re-observe; for cross-origin frames, report the explicit inaccessible-frame result rather than using JavaScript to bypass it.
|
|
334
|
+
|
|
335
|
+
Cross-origin caveat: a CDP snapshot may expose remote-frame refs for observation or coordinate actions, but `fill`/`type` may still fail to focus a child control. Verify the child value after every write; if focus fails, report cross-origin text entry as unsupported instead of claiming success or retrying with JavaScript.
|
|
336
|
+
|
|
273
337
|
### Verification
|
|
274
338
|
|
|
275
339
|
Always verify the final target:
|
|
@@ -388,7 +452,7 @@ OMNIBOT_SESSION_TOKEN=form omnibot is enabled "button[type=submit]" --tab-id <TA
|
|
|
388
452
|
Confirm the input accepted the file:
|
|
389
453
|
|
|
390
454
|
```bash
|
|
391
|
-
OMNIBOT_SESSION_TOKEN=form omnibot get "input[type=file]"
|
|
455
|
+
OMNIBOT_SESSION_TOKEN=form omnibot get attr "input[type=file]" value --tab-id <TAB_ID>
|
|
392
456
|
OMNIBOT_SESSION_TOKEN=form omnibot is enabled "button[type=submit]" --tab-id <TAB_ID>
|
|
393
457
|
```
|
|
394
458
|
|
|
@@ -99,7 +99,7 @@ omnibot --version
|
|
|
99
99
|
omnibot -V
|
|
100
100
|
```
|
|
101
101
|
|
|
102
|
-
Use `version` to print the installed omnibot CLI version. Output is a single line in the form `omnibot <version>`
|
|
102
|
+
Use `version` to print the installed omnibot CLI version. Output is a single line in the form `omnibot <version>` with no banner, JSON, or daemon calls. Prefer this over `omnibot --help` or `doctor` when you only need the version string.
|
|
103
103
|
|
|
104
104
|
## skills path
|
|
105
105
|
|
|
@@ -116,6 +116,9 @@ omnibot skills install --agent hermes --profile nuwa
|
|
|
116
116
|
omnibot skills install --agent opencode
|
|
117
117
|
omnibot skills install --agent claude
|
|
118
118
|
omnibot skills install --agent codex
|
|
119
|
+
omnibot skills install --agent openclaw
|
|
120
|
+
omnibot skills install --agent workbuddy
|
|
121
|
+
omnibot skills install --agent trae
|
|
119
122
|
```
|
|
120
123
|
|
|
121
124
|
Install only when setting up or repairing an agent integration. It is not part of normal page operations.
|
|
Binary file
|
package/package.json
CHANGED
|
Binary file
|