@omniaibot/win-x64 1.6.7 → 1.6.11
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/bin/omnibot-windows-x64/VERSION +1 -1
- package/bin/omnibot-windows-x64/_asyncio.pyd +0 -0
- package/bin/omnibot-windows-x64/_bz2.pyd +0 -0
- package/bin/omnibot-windows-x64/_cffi_backend.pyd +0 -0
- package/bin/omnibot-windows-x64/_ctypes.pyd +0 -0
- package/bin/omnibot-windows-x64/_decimal.pyd +0 -0
- package/bin/omnibot-windows-x64/_hashlib.pyd +0 -0
- package/bin/omnibot-windows-x64/_lzma.pyd +0 -0
- package/bin/omnibot-windows-x64/_multiprocessing.pyd +0 -0
- package/bin/omnibot-windows-x64/_overlapped.pyd +0 -0
- package/bin/omnibot-windows-x64/_queue.pyd +0 -0
- package/bin/omnibot-windows-x64/_socket.pyd +0 -0
- package/bin/omnibot-windows-x64/_ssl.pyd +0 -0
- package/bin/omnibot-windows-x64/_uuid.pyd +0 -0
- package/bin/omnibot-windows-x64/_wmi.pyd +0 -0
- package/bin/omnibot-windows-x64/ada92cb5d92a588d1b93__mypyc.pyd +0 -0
- package/bin/omnibot-windows-x64/certifi/cacert.pem +0 -70
- package/bin/omnibot-windows-x64/charset_normalizer/cd.pyd +0 -0
- package/bin/omnibot-windows-x64/charset_normalizer/md.pyd +0 -0
- package/bin/omnibot-windows-x64/cryptography/hazmat/bindings/_rust.pyd +0 -0
- package/bin/omnibot-windows-x64/libcrypto-3-x64.dll +0 -0
- package/bin/omnibot-windows-x64/libffi-8.dll +0 -0
- package/bin/omnibot-windows-x64/libssl-3-x64.dll +0 -0
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/SKILL.md +310 -220
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/anti-patterns.md +65 -57
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/command-reference.md +743 -709
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/css-only-dropdown-js.md +50 -50
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/debugging-and-evidence.md +135 -129
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/fallback-operations.md +196 -177
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/operation-patterns.md +496 -432
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/runtime-and-status.md +124 -121
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/session-and-tabs.md +164 -164
- package/bin/omnibot-windows-x64/omnibot/sop/tmwebdriver_sop.md +128 -128
- package/bin/omnibot-windows-x64/omnibot/sop/vue3_component_sop.md +163 -163
- package/bin/omnibot-windows-x64/omnibot-windows-x64.exe +0 -0
- package/bin/omnibot-windows-x64/pyexpat.pyd +0 -0
- package/bin/omnibot-windows-x64/python3.dll +0 -0
- package/bin/omnibot-windows-x64/python313.dll +0 -0
- package/bin/omnibot-windows-x64/select.pyd +0 -0
- package/bin/omnibot-windows-x64/unicodedata.pyd +0 -0
- package/bin/omnibot-windows-x64/vcruntime140.dll +0 -0
- package/bin/omnibot-windows-x64/vcruntime140_1.dll +0 -0
- package/package.json +1 -1
- package/bin/omnibot-windows-x64/81d243bd2c585b0f4821__mypyc.pyd +0 -0
- package/bin/omnibot-windows-x64/libcrypto-3.dll +0 -0
- package/bin/omnibot-windows-x64/libssl-3.dll +0 -0
- package/bin/omnibot-windows-x64/python312.dll +0 -0
|
@@ -1,220 +1,310 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: omnibot
|
|
3
|
-
description: Use when AI agents need to read, inspect, operate, navigate, debug, or verify browser state through the omnibot CLI and connected Chromium extension.
|
|
4
|
-
compatibility: Requires omnibot v2 CLI daemon and the omnibot Chromium extension connected to 127.0.0.1:18765.
|
|
5
|
-
allowed-tools: Bash
|
|
6
|
-
metadata:
|
|
7
|
-
author: omnibot
|
|
8
|
-
version: "2.4.0"
|
|
9
|
-
---
|
|
10
|
-
|
|
11
|
-
# Omnibot
|
|
12
|
-
|
|
13
|
-
Omnibot is Browser Infrastructure for AI Agents.
|
|
14
|
-
|
|
15
|
-
It connects Hermes, Claude Code, Codex, OpenCode, and other agent systems to a real Chromium browser through the local omnibot daemon and CLI.
|
|
16
|
-
|
|
17
|
-
This skill is an execution specification for agents. It is not a human command manual.
|
|
18
|
-
|
|
19
|
-
## When to Use
|
|
20
|
-
|
|
21
|
-
- Use Omnibot when browser runtime state matters: login, cookies, storage, client-side rendering, extensions, or visible user tabs.
|
|
22
|
-
- Use Omnibot when an agent must read rendered pages, click controls, fill forms, navigate, extract content, or collect evidence.
|
|
23
|
-
- Do not use Omnibot when static files or plain HTTP content already answer the task.
|
|
24
|
-
|
|
25
|
-
## Core Rules
|
|
26
|
-
|
|
27
|
-
- Reliability > Convenience.
|
|
28
|
-
- Explicit State > Implicit State.
|
|
29
|
-
- Pattern > Command.
|
|
30
|
-
- Observe -> Act -> Verify.
|
|
31
|
-
- Fallback is allowed, but never first.
|
|
32
|
-
- Do not use `execute-js` first.
|
|
33
|
-
- Run `omnibot <command> --help` before using uncommon or newly introduced commands, especially `batch`, `network`, `cdp`, `record`, `trace`, and `skills`.
|
|
34
|
-
- If a command from memory fails with parser errors, stop using that remembered syntax and re-check `omnibot --help` plus the subcommand help.
|
|
35
|
-
- Prefer semantic locators and `snapshot -i` refs before selectors.
|
|
36
|
-
-
|
|
37
|
-
-
|
|
38
|
-
- For
|
|
39
|
-
-
|
|
40
|
-
-
|
|
41
|
-
- Do not
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
```
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
|
86
|
-
|
|
|
87
|
-
|
|
|
88
|
-
|
|
|
89
|
-
|
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
-
|
|
148
|
-
|
|
149
|
-
-
|
|
150
|
-
|
|
151
|
-
-
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
|
|
162
|
-
|
|
|
163
|
-
|
|
|
164
|
-
|
|
|
165
|
-
|
|
|
166
|
-
|
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
|
|
173
|
-
|
|
174
|
-
|
|
175
|
-
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
|
|
183
|
-
|
|
184
|
-
|
|
185
|
-
|
|
186
|
-
|
|
187
|
-
|
|
188
|
-
|
|
189
|
-
|
|
190
|
-
|
|
191
|
-
|
|
192
|
-
|
|
193
|
-
|
|
194
|
-
|
|
195
|
-
|
|
196
|
-
|
|
197
|
-
|
|
198
|
-
|
|
199
|
-
|
|
200
|
-
|
|
201
|
-
|
|
202
|
-
|
|
203
|
-
|
|
204
|
-
|
|
205
|
-
|
|
206
|
-
|
|
207
|
-
|
|
208
|
-
|
|
209
|
-
|
|
210
|
-
|
|
211
|
-
|
|
212
|
-
|
|
213
|
-
|
|
214
|
-
|
|
215
|
-
|
|
216
|
-
|
|
217
|
-
|
|
218
|
-
|
|
219
|
-
|
|
220
|
-
|
|
1
|
+
---
|
|
2
|
+
name: omnibot
|
|
3
|
+
description: Use when AI agents need to read, inspect, operate, navigate, debug, or verify browser state through the omnibot CLI and connected Chromium extension.
|
|
4
|
+
compatibility: Requires omnibot v2 CLI daemon and the omnibot Chromium extension connected to 127.0.0.1:18765.
|
|
5
|
+
allowed-tools: Bash
|
|
6
|
+
metadata:
|
|
7
|
+
author: omnibot
|
|
8
|
+
version: "2.4.0"
|
|
9
|
+
---
|
|
10
|
+
|
|
11
|
+
# Omnibot
|
|
12
|
+
|
|
13
|
+
Omnibot is Browser Infrastructure for AI Agents.
|
|
14
|
+
|
|
15
|
+
It connects Hermes, Claude Code, Codex, OpenCode, and other agent systems to a real Chromium browser through the local omnibot daemon and CLI.
|
|
16
|
+
|
|
17
|
+
This skill is an execution specification for agents. It is not a human command manual.
|
|
18
|
+
|
|
19
|
+
## When to Use
|
|
20
|
+
|
|
21
|
+
- Use Omnibot when browser runtime state matters: login, cookies, storage, client-side rendering, extensions, or visible user tabs.
|
|
22
|
+
- Use Omnibot when an agent must read rendered pages, click controls, fill forms, navigate, extract content, or collect evidence.
|
|
23
|
+
- Do not use Omnibot when static files or plain HTTP content already answer the task.
|
|
24
|
+
|
|
25
|
+
## Core Rules
|
|
26
|
+
|
|
27
|
+
- Reliability > Convenience.
|
|
28
|
+
- Explicit State > Implicit State.
|
|
29
|
+
- Pattern > Command.
|
|
30
|
+
- Observe -> Act -> Verify.
|
|
31
|
+
- Fallback is allowed, but never first.
|
|
32
|
+
- Native Omnibot commands before scripts. Do not use `execute-js` first.
|
|
33
|
+
- Run `omnibot <command> --help` before using uncommon or newly introduced commands, especially `batch`, `network`, `cdp`, `record`, `trace`, and `skills`.
|
|
34
|
+
- If a command from memory fails with parser errors, stop using that remembered syntax and re-check `omnibot --help` plus the subcommand help.
|
|
35
|
+
- Prefer semantic locators and `snapshot -i` refs before selectors.
|
|
36
|
+
- When the user has explicitly requested a native Omnibot command and the terminal tool is available, execute it immediately; do not ask for confirmation or claim that it was not run. If dispatch genuinely fails, report the concrete tool error after one bounded retry and do not fabricate browser results.
|
|
37
|
+
- A multi-step browser task is incomplete until its requested final verification command has actually run. After the last mutation, always issue the concrete `get`/`is`/`wait` verification, inspect its result, and only then report completion; never stop after reporting that the mutation itself succeeded.
|
|
38
|
+
- For workflows with more than five meaningful browser actions, split the work into multiple bounded turns. Each turn should finish its own observe -> act -> verify cycle and report the retained tab id before the next turn; do not put a long mutation chain and all final checks into one model request.
|
|
39
|
+
- Raw `cdp` calls are one-shot. Do not assume `Network.enable` alone creates readable logs. For API/request evidence, use `network clear -> network start -> one action -> network stop -> network logs/summary`.
|
|
40
|
+
- After a failed click by ref, re-observe or use a higher-evidence fallback. Do not keep using stale `@eN` refs from a previous snapshot.
|
|
41
|
+
- For shopping, checkout, payment, banking, or irreversible workflows: Do not click final submit/pay/place-order controls unless the user explicitly confirms that final action in the current turn.
|
|
42
|
+
- Prefer condition-based `wait` over shell sleep.
|
|
43
|
+
- Timeout values for commands that expose a timeout option, such as `wait`, are seconds, not milliseconds. Use `--timeout 5` for a five-second wait; never convert milliseconds into a raw CLI timeout value. `dialog logs` and `dialog clear` do not accept `--timeout`.
|
|
44
|
+
- Native dialog workflows are session-token scoped: use the exact same `OMNIBOT_SESSION_TOKEN` for the triggering click, `dialog logs`, and `dialog handle`. A different token cannot observe or handle the original dialog.
|
|
45
|
+
- Parse JSON output from commands that return JSON.
|
|
46
|
+
- Do not claim success without verification evidence.
|
|
47
|
+
|
|
48
|
+
### Runtime Provenance
|
|
49
|
+
|
|
50
|
+
Before an agent-driven test in a source checkout, verify the command provenance:
|
|
51
|
+
|
|
52
|
+
```bash
|
|
53
|
+
command -v omnibot
|
|
54
|
+
omnibot --version
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
If the current workspace contains `pyproject.toml` and the global command is not the workspace version, run the test commands as `uv run omnibot ...` (or use the workspace `.venv/bin/omnibot`) so the agent exercises the code under test. Do not infer command support from memory or from a different packaged version; run the relevant subcommand `--help` after selecting the runtime.
|
|
58
|
+
|
|
59
|
+
For a source-checkout test, make this concrete before the first browser action:
|
|
60
|
+
define the command prefix once:
|
|
61
|
+
|
|
62
|
+
```bash
|
|
63
|
+
export OMNIBOT="uv run omnibot --no-start"
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
stop any existing daemon, start the workspace daemon with
|
|
67
|
+
`uv run python -m omnibot --api-port 18764 --ws-port 18765 daemon run`, and
|
|
68
|
+
invoke every subsequent CLI command as `$OMNIBOT ...` (or the
|
|
69
|
+
workspace `.venv/bin/omnibot --no-start ...`). Never use bare `omnibot` in that
|
|
70
|
+
workflow, because it can auto-start the globally installed packaged daemon and
|
|
71
|
+
silently test different code.
|
|
72
|
+
|
|
73
|
+
## Native Command Router
|
|
74
|
+
|
|
75
|
+
Before any page-state operation, choose the narrowest native command that expresses the intent. JavaScript is not a convenience shortcut; it is a fallback that requires evidence that native commands could not reach the state.
|
|
76
|
+
|
|
77
|
+
| Intent | Native command path | Do not start with |
|
|
78
|
+
| --- | --- | --- |
|
|
79
|
+
| Current page identity | `get title`, `get url`, then `snapshot -i` or `read` if needed | `execute-js "return document.title/location.href"` |
|
|
80
|
+
| Browser history | `browser history [text]` (also `history search [text]`) | reading browser history through page JavaScript |
|
|
81
|
+
| Bookmarks | `browser bookmarks` (also `bookmarks tree`) | reading the browser bookmark tree |
|
|
82
|
+
| Downloads | `browser downloads [terms]` (also `downloads search [terms]`, `downloads open <id>`) | inspecting/opening browser downloads |
|
|
83
|
+
| Recently closed | `browser recently-closed` | reading recently closed tabs/windows |
|
|
84
|
+
| Top sites | `browser top-sites` | reading browser frequent/top sites |
|
|
85
|
+
| Installed extensions | `browser extensions` | listing extension name, ID, enabled state, type, and version (read-only) |
|
|
86
|
+
| Content settings | `browser content-settings [TYPE] [URL]` | reading site permission state, default `automaticDownloads` (read-only) |
|
|
87
|
+
| Mouse visual state | `browser mouse-visual-state --tab-id <TAB_ID>` | verifying the extension cursor overlay and asset load state (read-only diagnostic) |
|
|
88
|
+
| Notifications | `browser notify <title> [message]` | showing a browser notification |
|
|
89
|
+
| Tab groups | `tab group <TAB_ID> [NAME]`, `tab group-info <TAB_ID>`, `tab ungroup <TAB_ID>` | grouping and inspecting browser tabs |
|
|
90
|
+
| Page text/content | `read --screens N`, `get text`, `get html`, `get attr`, `get count` | `execute-js "document.body.innerText"` |
|
|
91
|
+
| Iframe context | `frame <iframe-id/name/title/src>`, then `snapshot`/page-state commands | Assuming host selectors reach child frames |
|
|
92
|
+
| Click | `find ... --action click` -> `snapshot -i` + `click @eN` -> `click selector` -> `dom click` -> `mouse click` | `document.querySelector(...).click()` |
|
|
93
|
+
| Scroll | `scroll`, `scrollintoview`, `dom scroll`, then `mouse scroll` | `window.scrollTo`, `element.scrollIntoView()` |
|
|
94
|
+
| Fill/type | Text entry: `find ... --action fill`, `fill`, or `type`; key semantics: `press`/`keyboard` | setting `.value` or `innerHTML` in JS |
|
|
95
|
+
| Wait | `wait` by selector/text/url/load/fn | shell sleep or JS timers |
|
|
96
|
+
|
|
97
|
+
If you are about to write `document.querySelector`, `document.body.innerText`, `.click()`, `.value =`, `innerHTML`, `scrollTo`, or `scrollIntoView`, stop and identify the native command you will try first. Only use `execute-js` after documenting which native tiers failed and how you will verify the result.
|
|
98
|
+
|
|
99
|
+
Text entry rule: when the user asks to enter, type, fill, or replace text, use `fill` or `type`. `keyboard` and `press` dispatch key events for shortcuts and key semantics; they are not text insertion commands and must not be substituted for text entry.
|
|
100
|
+
|
|
101
|
+
Form commit rule: after `fill`, `type`, `select`, `check`, `uncheck`, or `upload`, verify the control itself with `get value`, `get attr`, or `is` before judging the workflow. Many pages keep a separate saved/rendered status that does not change until a visible `Save`, `Apply`, `Submit`, or equivalent control is activated. If such a commit control exists and the task expects business-state verification, observe it, activate it with a native command, then verify the resulting status. Do not report a form action as failed merely because an unrelated status element is still stale, and do not report it as complete without verifying both the control value and any required commit result.
|
|
102
|
+
|
|
103
|
+
Form terminal-timeout rule: keep form workflows as short observe → one control action → one verification steps. If a terminal call times out after `fill`, `select`, `check`, or `Save`, query the browser state first; do not repeat the action blindly because the browser may already have applied it. Continue from the first unverified control and finish with one commit click plus one final status read.
|
|
104
|
+
|
|
105
|
+
Mutation retry rule: issue at most one mutating browser command (`fill`, `type`, `select`, `check`, `uncheck`, `click`, `dblclick`, `drag`, `upload`, or `navigate`) per agent request. If its terminal output is empty, delayed, or times out, the next request must be read-only (`get`, `is`, `snapshot`, `read`, `tabs`, or `wait`) to determine whether the mutation already happened. Never replay the mutation merely because the tool response was missing; duplicate fills can append text and duplicate clicks can change business state.
|
|
106
|
+
|
|
107
|
+
Drag performance rule: selector-based `drag` may take longer than a terminal tool's short default timeout because it dispatches a human-like event sequence. For agent workflows where exact trajectory is not required, prefer `snapshot -i` plus coordinate `mouse drag --fast` after verifying source/target boxes; then verify the target state. If semantic `drag` is required, allow a longer command timeout and do not retry blindly while the first drag may still be running.
|
|
108
|
+
|
|
109
|
+
Clipboard rule: both `clipboard read` and `clipboard write` require the explicit
|
|
110
|
+
`--tab-id <TAB_ID>` on every invocation, even though the clipboard is shared by
|
|
111
|
+
the browser extension. If no tab id is known, discover a safe target tab first;
|
|
112
|
+
never omit the flag or infer a default tab.
|
|
113
|
+
|
|
114
|
+
Cross-origin frame rule: snapshot visibility does not guarantee write access. After targeting a cross-origin iframe, verify `fill`/`type` focus and the resulting value; if focus fails, report the limitation explicitly and do not use JavaScript as a bypass.
|
|
115
|
+
|
|
116
|
+
## Workflow Context + Tab Target
|
|
117
|
+
|
|
118
|
+
Every page-state workflow must use both controls:
|
|
119
|
+
|
|
120
|
+
- `OMNIBOT_SESSION_TOKEN=<workflow-name>` for workflow isolation.
|
|
121
|
+
- `--tab-id <TAB_ID>` for page targeting.
|
|
122
|
+
|
|
123
|
+
OMNIBOT_SESSION_TOKEN is a workflow/context token, not a browser session and not a tab target. It isolates agent state such as refs, trace, recording, aliases, and temporary resources. Tab ID controls which page receives the command.
|
|
124
|
+
|
|
125
|
+
Do not rely on default tab, active tab, current tab, or prior targeting state. Every page-state command requires explicit `--tab-id` except when the command is only discovering or creating a tab.
|
|
126
|
+
|
|
127
|
+
Use the same token inside one workflow. Use different tokens for independent workflows.
|
|
128
|
+
|
|
129
|
+
Standard form:
|
|
130
|
+
|
|
131
|
+
```bash
|
|
132
|
+
OMNIBOT_SESSION_TOKEN=research omnibot snapshot -i --tab-id <TAB_ID>
|
|
133
|
+
OMNIBOT_SESSION_TOKEN=research omnibot click --tab-id <TAB_ID> @e4
|
|
134
|
+
OMNIBOT_SESSION_TOKEN=research omnibot snapshot -i --tab-id <TAB_ID>
|
|
135
|
+
```
|
|
136
|
+
|
|
137
|
+
`@eN` refs are tab-scoped. Refs rule: never reuse `@eN` refs across tabs.
|
|
138
|
+
|
|
139
|
+
The agent dispatch pattern is token + tab-id:
|
|
140
|
+
|
|
141
|
+
1. Set a stable `OMNIBOT_SESSION_TOKEN`.
|
|
142
|
+
2. Discover or create the target tab.
|
|
143
|
+
3. Save the returned tab id.
|
|
144
|
+
4. Pass `--tab-id <TAB_ID>` on every command that touches page state.
|
|
145
|
+
5. Run observe -> act -> verify on that same tab.
|
|
146
|
+
|
|
147
|
+
Tab-id identity rule: use the exact `id` field returned by Omnibot `tabs`/`tab list` or by a tab-creation command. Browser-control surfaces may expose a shorter numeric `tab_id`; that value is not necessarily the Omnibot session id. When the discovery output contains both `tab_id` and `id`, pass the full namespaced `id` (for example, `edge-...:12345`) to `--tab-id`. Never reconstruct or strip the namespace.
|
|
148
|
+
|
|
149
|
+
Phased workflow rule: do not bundle navigation and several page actions into one unverified sequence. After creating or navigating a tab, first run `get url` (and `snapshot -i` when controls are involved); then perform one meaningful action; then verify its concrete state before issuing the next action. For a form or multi-control task, repeat this observe -> one action -> verify cycle for each control.
|
|
150
|
+
|
|
151
|
+
Examples of tab-locked page-state commands:
|
|
152
|
+
|
|
153
|
+
```bash
|
|
154
|
+
OMNIBOT_SESSION_TOKEN=checkout omnibot get value "input[name=email]" --tab-id <TAB_ID>
|
|
155
|
+
OMNIBOT_SESSION_TOKEN=checkout omnibot find placeholder "Search" --action type --action-value "omnibot" --tab-id <TAB_ID>
|
|
156
|
+
OMNIBOT_SESSION_TOKEN=checkout omnibot dom dblclick n1 --tab-id <TAB_ID>
|
|
157
|
+
OMNIBOT_SESSION_TOKEN=checkout omnibot clipboard read --tab-id <TAB_ID>
|
|
158
|
+
```
|
|
159
|
+
|
|
160
|
+
Targeting categories:
|
|
161
|
+
|
|
162
|
+
| Command category | Rule |
|
|
163
|
+
| --- | --- |
|
|
164
|
+
| Page-state commands: `snapshot`, `click`, `fill`, `type`, `get`, `is`, `wait`, `screenshot`, `console`, `network`, `execute-js`, `dom`, `mouse`, `clipboard`, `viewport`, `assets`, `goto`, same-tab `navigate` | Require `--tab-id <TAB_ID>`. |
|
|
165
|
+
| Discovery or tab creation: `tabs`, `tab list`, `tab new <URL>`, `open <URL>`, `navigate <URL>` | Return or discover the tab id; save it for later page-state commands. |
|
|
166
|
+
| Read URL exception: `read <URL>` | Opens a temporary read tab and does not need `--tab-id`; reading an existing tab still requires `read --tab-id <TAB_ID>`. |
|
|
167
|
+
|
|
168
|
+
Navigation safety invariant:
|
|
169
|
+
|
|
170
|
+
- When the task says "open", "open a new tab", or "create a test page", use `open <URL>` or `tab new <URL>` and use the returned tab id. Do not substitute `goto`, plain `navigate`, or the currently active tab.
|
|
171
|
+
- `navigate <URL>` also opens a new tab by default; `navigate <URL> --new-tab` is an explicit equivalent accepted for agent compatibility. Use `navigate <URL> --same-tab --tab-id <TAB_ID>` only for deliberate same-tab reuse.
|
|
172
|
+
- Use `goto --tab-id <TAB_ID>` or `navigate --same-tab --tab-id <TAB_ID>` only when the task explicitly asks to reuse an existing tab.
|
|
173
|
+
- After creating a tab, verify `get url --tab-id <TAB_ID>` before any clipboard, upload, form, or other page-state operation. This prevents a failed or misinterpreted tab-creation result from mutating a user-owned page.
|
|
174
|
+
- Tool-created tabs are temporary workflow resources and may be auto-closed by cleanup after inactivity. Complete the intended tab workflow in the same token/context, keep the returned tab id, and do not assume it will still exist after a long model pause or a separate one-shot agent process. If a later close reports `not found`, re-list tabs before retrying; the cleanup may already have closed it.
|
|
175
|
+
- If `open` or `tab new` times out, loses its final response, or is retried after an agent/tool interruption, first run `tabs` or `tab list` and match the requested URL. Reuse one matching tab id instead of issuing another create command. Only create again after confirming that no matching tab from the interrupted attempt exists.
|
|
176
|
+
|
|
177
|
+
## Observe -> Act -> Verify
|
|
178
|
+
|
|
179
|
+
All page operations must follow this loop:
|
|
180
|
+
|
|
181
|
+
1. Observe the current page state.
|
|
182
|
+
2. Act once using the highest reliable pattern available.
|
|
183
|
+
3. Verify the expected state change.
|
|
184
|
+
|
|
185
|
+
Click example:
|
|
186
|
+
|
|
187
|
+
```bash
|
|
188
|
+
OMNIBOT_SESSION_TOKEN=research omnibot snapshot -i --tab-id <TAB_ID>
|
|
189
|
+
OMNIBOT_SESSION_TOKEN=research omnibot click --tab-id <TAB_ID> @e4
|
|
190
|
+
OMNIBOT_SESSION_TOKEN=research omnibot snapshot -i --tab-id <TAB_ID>
|
|
191
|
+
```
|
|
192
|
+
|
|
193
|
+
Read/verify example:
|
|
194
|
+
|
|
195
|
+
```bash
|
|
196
|
+
OMNIBOT_SESSION_TOKEN=checkout omnibot is enabled "button[type=submit]" --tab-id <TAB_ID>
|
|
197
|
+
OMNIBOT_SESSION_TOKEN=checkout omnibot find role button --name "Submit" --action click --tab-id <TAB_ID>
|
|
198
|
+
OMNIBOT_SESSION_TOKEN=checkout omnibot wait --url "/dashboard" --tab-id <TAB_ID>
|
|
199
|
+
OMNIBOT_SESSION_TOKEN=checkout omnibot get url --tab-id <TAB_ID>
|
|
200
|
+
```
|
|
201
|
+
|
|
202
|
+
If verification fails, do not repeat blindly. Re-observe, choose the next fallback tier, and verify again.
|
|
203
|
+
|
|
204
|
+
## Pattern > Command
|
|
205
|
+
|
|
206
|
+
Start from the task, not the command name. Use operation patterns for read, click, fill, select/check, navigation, wait, extraction, and batch.
|
|
207
|
+
|
|
208
|
+
Use fallback tiers only after standard patterns fail. Collect evidence separately from fallback execution. The command reference is only a lookup table; it must not decide behavior.
|
|
209
|
+
|
|
210
|
+
## Read vs Snapshot Routing
|
|
211
|
+
|
|
212
|
+
Choose by intent before choosing by command name:
|
|
213
|
+
|
|
214
|
+
| Intent | Prefer | Why |
|
|
215
|
+
| --- | --- | --- |
|
|
216
|
+
| Summarize or extract page content, article text, search results, feeds, or long/lazy pages | `read` | Returns clean text/Markdown for agent reasoning. |
|
|
217
|
+
| Quickly observe current visible UI structure before deciding what to do next | `snapshot -i` | Returns actionable refs and page structure for routing. |
|
|
218
|
+
| Find buttons, links, inputs, or refs for a later action | `find` or `snapshot -i` | Produces actionable targets; `read` does not. |
|
|
219
|
+
| Verify one concrete condition | `get`, `is`, or `wait` | Narrow evidence is more reliable than dumping a page. |
|
|
220
|
+
|
|
221
|
+
Do not use `read` as the first step for click/fill workflows unless the user explicitly asked for page content first.
|
|
222
|
+
|
|
223
|
+
## Dropdowns And Custom Comboboxes
|
|
224
|
+
|
|
225
|
+
`snapshot -i` may auto-probe visible custom comboboxes and append their options as refs. If the output lists `@eN [option] "..."`, click that option ref directly; Omnibot will reopen the owning combobox through `openerSelector` if the dropdown closed during snapshot.
|
|
226
|
+
|
|
227
|
+
Use `select @combobox "value"` for native selects or simple controls. Use `click @option` when `snapshot -i` exposes auto-probed combobox option refs. Do not manually reopen every dropdown just to discover options unless `snapshot -i` did not expose option refs.
|
|
228
|
+
|
|
229
|
+
## Rich Text Article Editors
|
|
230
|
+
|
|
231
|
+
ContentEditable rich text editors (ProseMirror, Quill, Draft.js, Slate, ByteDance/Toutiao article body, etc.) are often missing from the AX tree. `snapshot -i` appends them under `# DOM Rich Text Editors` as `@eN [richtext] "..." [contenteditable=true]` refs.
|
|
232
|
+
|
|
233
|
+
- Detect the body editor: `snapshot -i` → look for `@eN [richtext]` under `# DOM Rich Text Editors`. Do not fill the article body through a `[textbox]` title ref.
|
|
234
|
+
- Replace the whole body: `fill @richtext "paragraph 1\n\nparagraph 2"`. Empty lines split paragraphs; the previous content is cleared and `input`/`change` events are dispatched.
|
|
235
|
+
- Append to the body: `type @richtext " additional text"`.
|
|
236
|
+
- Focus before keyboard input: click `@richtext` first, then `press`/`keyboard`.
|
|
237
|
+
- Raw selector fallback: `omnibot fill '[contenteditable="true"]' "..."` also routes through the rich text path when the selector looks like a rich editor.
|
|
238
|
+
|
|
239
|
+
Do not use `execute-js` first for rich text writes; `fill`/`type` on the `[richtext]` ref is the supported path.
|
|
240
|
+
|
|
241
|
+
## Quick Routing
|
|
242
|
+
|
|
243
|
+
| Need | Read |
|
|
244
|
+
| --- | --- |
|
|
245
|
+
| Runtime check | `references/runtime-and-status.md` |
|
|
246
|
+
| Reading pages | `references/operation-patterns.md#read` |
|
|
247
|
+
| Clicking | `references/operation-patterns.md#click` |
|
|
248
|
+
| Filling forms | `references/operation-patterns.md#fill` |
|
|
249
|
+
| Scrolling | `references/operation-patterns.md#scroll` |
|
|
250
|
+
| Select / check | `references/operation-patterns.md#select--check` |
|
|
251
|
+
| Navigation | `references/operation-patterns.md#navigation` |
|
|
252
|
+
| Waiting | `references/operation-patterns.md#wait` |
|
|
253
|
+
| Extraction | `references/operation-patterns.md#extraction` |
|
|
254
|
+
| Batch | `references/operation-patterns.md#batch` |
|
|
255
|
+
| Fallback | `references/fallback-operations.md` |
|
|
256
|
+
| Tabs and sessions | `references/session-and-tabs.md` |
|
|
257
|
+
| Debug evidence | `references/debugging-and-evidence.md` |
|
|
258
|
+
| Network/API capture | `references/debugging-and-evidence.md#network-capture` |
|
|
259
|
+
| Commands | `references/command-reference.md` |
|
|
260
|
+
| Anti-patterns | `references/anti-patterns.md` |
|
|
261
|
+
|
|
262
|
+
## Runtime First
|
|
263
|
+
|
|
264
|
+
Before troubleshooting browser behavior, check runtime state:
|
|
265
|
+
|
|
266
|
+
```bash
|
|
267
|
+
omnibot doctor
|
|
268
|
+
omnibot tabs
|
|
269
|
+
omnibot visibility status
|
|
270
|
+
omnibot browser current
|
|
271
|
+
omnibot license status
|
|
272
|
+
```
|
|
273
|
+
|
|
274
|
+
If the extension is not connected, open Chrome or Edge with the omnibot extension loaded and keep an HTTP/HTTPS tab open.
|
|
275
|
+
|
|
276
|
+
## Fallback Discipline
|
|
277
|
+
|
|
278
|
+
Fallback tiers are for completing operations after better patterns fail. They are not debugging shortcuts.
|
|
279
|
+
|
|
280
|
+
Tier order:
|
|
281
|
+
|
|
282
|
+
1. Semantic `find`.
|
|
283
|
+
2. `snapshot -i` refs.
|
|
284
|
+
3. Selectors.
|
|
285
|
+
4. `dom` fallback.
|
|
286
|
+
5. `mouse` fallback.
|
|
287
|
+
6. `execute-js` fallback.
|
|
288
|
+
7. Raw `cdp` fallback.
|
|
289
|
+
|
|
290
|
+
When entering a lower tier, state why the higher tier failed. After any fallback action, verify with `snapshot`, `get`, `is`, `wait`, or equivalent evidence.
|
|
291
|
+
|
|
292
|
+
Before `execute-js`, the agent must have native-attempt evidence: the exact `find`/`snapshot`/`click`/`fill`/`scroll`/`get`/`read` commands tried, their result, why they were insufficient, and the post-JS verification command to run. Without that evidence, use a native command instead.
|
|
293
|
+
|
|
294
|
+
## Debug Evidence
|
|
295
|
+
|
|
296
|
+
Debugging is for evidence. Fallback is for completing operations.
|
|
297
|
+
|
|
298
|
+
Use screenshots, annotated screenshots, console logs, network logs, trace, record/replay, and CDP inspection to explain failures or prove state. Do not treat `execute-js` as ordinary debug output.
|
|
299
|
+
|
|
300
|
+
## Absolute Prohibitions
|
|
301
|
+
|
|
302
|
+
- Missing `OMNIBOT_SESSION_TOKEN` or missing `--tab-id` on page-state commands.
|
|
303
|
+
- Acting without verify or claiming success without evidence.
|
|
304
|
+
- Using `execute-js` first, raw CSS before semantic find, `@eN` across tabs, implicit tab targeting, or shell sleep instead of `omnibot wait`.
|
|
305
|
+
- Using JavaScript read/click/scroll shortcuts when native `get`/`read`/`find`/`snapshot`/`click`/`scroll` can express the task.
|
|
306
|
+
- Using removed commands: `switch-tab`, `focus-tab`, `tab switch`, `tab focus`. These are no longer available.
|
|
307
|
+
- Using `tabs[0]` or the first tab as a target. The first tab is often a user tab or transport tab.
|
|
308
|
+
- Closing user tabs discovered during cleanup. Only close tabs created by the current workflow.
|
|
309
|
+
|
|
310
|
+
See `references/anti-patterns.md` before using shortcuts.
|