@omniaibot/win-x64 1.6.7 → 1.6.8
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/bin/omnibot-windows-x64/VERSION +1 -1
- package/bin/omnibot-windows-x64/_asyncio.pyd +0 -0
- package/bin/omnibot-windows-x64/_bz2.pyd +0 -0
- package/bin/omnibot-windows-x64/_cffi_backend.pyd +0 -0
- package/bin/omnibot-windows-x64/_ctypes.pyd +0 -0
- package/bin/omnibot-windows-x64/_decimal.pyd +0 -0
- package/bin/omnibot-windows-x64/_hashlib.pyd +0 -0
- package/bin/omnibot-windows-x64/_lzma.pyd +0 -0
- package/bin/omnibot-windows-x64/_multiprocessing.pyd +0 -0
- package/bin/omnibot-windows-x64/_overlapped.pyd +0 -0
- package/bin/omnibot-windows-x64/_queue.pyd +0 -0
- package/bin/omnibot-windows-x64/_socket.pyd +0 -0
- package/bin/omnibot-windows-x64/_ssl.pyd +0 -0
- package/bin/omnibot-windows-x64/_uuid.pyd +0 -0
- package/bin/omnibot-windows-x64/_wmi.pyd +0 -0
- package/bin/omnibot-windows-x64/ada92cb5d92a588d1b93__mypyc.pyd +0 -0
- package/bin/omnibot-windows-x64/certifi/cacert.pem +0 -70
- package/bin/omnibot-windows-x64/charset_normalizer/cd.pyd +0 -0
- package/bin/omnibot-windows-x64/charset_normalizer/md.pyd +0 -0
- package/bin/omnibot-windows-x64/cryptography/hazmat/bindings/_rust.pyd +0 -0
- package/bin/omnibot-windows-x64/libcrypto-3-x64.dll +0 -0
- package/bin/omnibot-windows-x64/libffi-8.dll +0 -0
- package/bin/omnibot-windows-x64/libssl-3-x64.dll +0 -0
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/SKILL.md +285 -220
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/anti-patterns.md +65 -57
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/command-reference.md +736 -709
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/css-only-dropdown-js.md +50 -50
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/debugging-and-evidence.md +135 -129
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/fallback-operations.md +196 -177
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/operation-patterns.md +496 -432
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/runtime-and-status.md +124 -121
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/session-and-tabs.md +164 -164
- package/bin/omnibot-windows-x64/omnibot/sop/tmwebdriver_sop.md +128 -128
- package/bin/omnibot-windows-x64/omnibot/sop/vue3_component_sop.md +163 -163
- package/bin/omnibot-windows-x64/omnibot-windows-x64.exe +0 -0
- package/bin/omnibot-windows-x64/pyexpat.pyd +0 -0
- package/bin/omnibot-windows-x64/python3.dll +0 -0
- package/bin/omnibot-windows-x64/python313.dll +0 -0
- package/bin/omnibot-windows-x64/select.pyd +0 -0
- package/bin/omnibot-windows-x64/unicodedata.pyd +0 -0
- package/bin/omnibot-windows-x64/vcruntime140.dll +0 -0
- package/bin/omnibot-windows-x64/vcruntime140_1.dll +0 -0
- package/package.json +1 -1
- package/bin/omnibot-windows-x64/81d243bd2c585b0f4821__mypyc.pyd +0 -0
- package/bin/omnibot-windows-x64/libcrypto-3.dll +0 -0
- package/bin/omnibot-windows-x64/libssl-3.dll +0 -0
- package/bin/omnibot-windows-x64/python312.dll +0 -0
|
@@ -1,220 +1,285 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: omnibot
|
|
3
|
-
description: Use when AI agents need to read, inspect, operate, navigate, debug, or verify browser state through the omnibot CLI and connected Chromium extension.
|
|
4
|
-
compatibility: Requires omnibot v2 CLI daemon and the omnibot Chromium extension connected to 127.0.0.1:18765.
|
|
5
|
-
allowed-tools: Bash
|
|
6
|
-
metadata:
|
|
7
|
-
author: omnibot
|
|
8
|
-
version: "2.4.0"
|
|
9
|
-
---
|
|
10
|
-
|
|
11
|
-
# Omnibot
|
|
12
|
-
|
|
13
|
-
Omnibot is Browser Infrastructure for AI Agents.
|
|
14
|
-
|
|
15
|
-
It connects Hermes, Claude Code, Codex, OpenCode, and other agent systems to a real Chromium browser through the local omnibot daemon and CLI.
|
|
16
|
-
|
|
17
|
-
This skill is an execution specification for agents. It is not a human command manual.
|
|
18
|
-
|
|
19
|
-
## When to Use
|
|
20
|
-
|
|
21
|
-
- Use Omnibot when browser runtime state matters: login, cookies, storage, client-side rendering, extensions, or visible user tabs.
|
|
22
|
-
- Use Omnibot when an agent must read rendered pages, click controls, fill forms, navigate, extract content, or collect evidence.
|
|
23
|
-
- Do not use Omnibot when static files or plain HTTP content already answer the task.
|
|
24
|
-
|
|
25
|
-
## Core Rules
|
|
26
|
-
|
|
27
|
-
- Reliability > Convenience.
|
|
28
|
-
- Explicit State > Implicit State.
|
|
29
|
-
- Pattern > Command.
|
|
30
|
-
- Observe -> Act -> Verify.
|
|
31
|
-
- Fallback is allowed, but never first.
|
|
32
|
-
- Do not use `execute-js` first.
|
|
33
|
-
- Run `omnibot <command> --help` before using uncommon or newly introduced commands, especially `batch`, `network`, `cdp`, `record`, `trace`, and `skills`.
|
|
34
|
-
- If a command from memory fails with parser errors, stop using that remembered syntax and re-check `omnibot --help` plus the subcommand help.
|
|
35
|
-
- Prefer semantic locators and `snapshot -i` refs before selectors.
|
|
36
|
-
-
|
|
37
|
-
-
|
|
38
|
-
- For
|
|
39
|
-
-
|
|
40
|
-
-
|
|
41
|
-
- Do not
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
OMNIBOT_SESSION_TOKEN=
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
-
|
|
148
|
-
-
|
|
149
|
-
-
|
|
150
|
-
-
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
|
|
162
|
-
|
|
163
|
-
|
|
164
|
-
|
|
165
|
-
|
|
166
|
-
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
|
|
173
|
-
|
|
174
|
-
|
|
175
|
-
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
|
|
183
|
-
|
|
184
|
-
|
|
185
|
-
|
|
186
|
-
|
|
187
|
-
|
|
188
|
-
|
|
189
|
-
|
|
190
|
-
|
|
191
|
-
|
|
192
|
-
|
|
193
|
-
|
|
194
|
-
|
|
195
|
-
|
|
196
|
-
|
|
197
|
-
|
|
198
|
-
|
|
199
|
-
|
|
200
|
-
|
|
201
|
-
|
|
202
|
-
|
|
203
|
-
|
|
204
|
-
|
|
205
|
-
|
|
206
|
-
|
|
207
|
-
|
|
208
|
-
|
|
209
|
-
|
|
210
|
-
|
|
211
|
-
|
|
212
|
-
|
|
213
|
-
|
|
214
|
-
-
|
|
215
|
-
|
|
216
|
-
|
|
217
|
-
|
|
218
|
-
|
|
219
|
-
|
|
220
|
-
|
|
1
|
+
---
|
|
2
|
+
name: omnibot
|
|
3
|
+
description: Use when AI agents need to read, inspect, operate, navigate, debug, or verify browser state through the omnibot CLI and connected Chromium extension.
|
|
4
|
+
compatibility: Requires omnibot v2 CLI daemon and the omnibot Chromium extension connected to 127.0.0.1:18765.
|
|
5
|
+
allowed-tools: Bash
|
|
6
|
+
metadata:
|
|
7
|
+
author: omnibot
|
|
8
|
+
version: "2.4.0"
|
|
9
|
+
---
|
|
10
|
+
|
|
11
|
+
# Omnibot
|
|
12
|
+
|
|
13
|
+
Omnibot is Browser Infrastructure for AI Agents.
|
|
14
|
+
|
|
15
|
+
It connects Hermes, Claude Code, Codex, OpenCode, and other agent systems to a real Chromium browser through the local omnibot daemon and CLI.
|
|
16
|
+
|
|
17
|
+
This skill is an execution specification for agents. It is not a human command manual.
|
|
18
|
+
|
|
19
|
+
## When to Use
|
|
20
|
+
|
|
21
|
+
- Use Omnibot when browser runtime state matters: login, cookies, storage, client-side rendering, extensions, or visible user tabs.
|
|
22
|
+
- Use Omnibot when an agent must read rendered pages, click controls, fill forms, navigate, extract content, or collect evidence.
|
|
23
|
+
- Do not use Omnibot when static files or plain HTTP content already answer the task.
|
|
24
|
+
|
|
25
|
+
## Core Rules
|
|
26
|
+
|
|
27
|
+
- Reliability > Convenience.
|
|
28
|
+
- Explicit State > Implicit State.
|
|
29
|
+
- Pattern > Command.
|
|
30
|
+
- Observe -> Act -> Verify.
|
|
31
|
+
- Fallback is allowed, but never first.
|
|
32
|
+
- Native Omnibot commands before scripts. Do not use `execute-js` first.
|
|
33
|
+
- Run `omnibot <command> --help` before using uncommon or newly introduced commands, especially `batch`, `network`, `cdp`, `record`, `trace`, and `skills`.
|
|
34
|
+
- If a command from memory fails with parser errors, stop using that remembered syntax and re-check `omnibot --help` plus the subcommand help.
|
|
35
|
+
- Prefer semantic locators and `snapshot -i` refs before selectors.
|
|
36
|
+
- When the user has explicitly requested a native Omnibot command and the terminal tool is available, execute it immediately; do not ask for confirmation or claim that it was not run. If dispatch genuinely fails, report the concrete tool error after one bounded retry and do not fabricate browser results.
|
|
37
|
+
- A multi-step browser task is incomplete until its requested final verification command has actually run. After the last mutation, always issue the concrete `get`/`is`/`wait` verification, inspect its result, and only then report completion; never stop after reporting that the mutation itself succeeded.
|
|
38
|
+
- For workflows with more than five meaningful browser actions, split the work into multiple bounded turns. Each turn should finish its own observe -> act -> verify cycle and report the retained tab id before the next turn; do not put a long mutation chain and all final checks into one model request.
|
|
39
|
+
- Raw `cdp` calls are one-shot. Do not assume `Network.enable` alone creates readable logs. For API/request evidence, use `network clear -> network start -> one action -> network stop -> network logs/summary`.
|
|
40
|
+
- After a failed click by ref, re-observe or use a higher-evidence fallback. Do not keep using stale `@eN` refs from a previous snapshot.
|
|
41
|
+
- For shopping, checkout, payment, banking, or irreversible workflows: Do not click final submit/pay/place-order controls unless the user explicitly confirms that final action in the current turn.
|
|
42
|
+
- Prefer condition-based `wait` over shell sleep.
|
|
43
|
+
- Timeout values for commands that expose a timeout option, such as `wait`, are seconds, not milliseconds. Use `--timeout 5` for a five-second wait; never convert milliseconds into a raw CLI timeout value. `dialog logs` and `dialog clear` do not accept `--timeout`.
|
|
44
|
+
- Native dialog workflows are session-token scoped: use the exact same `OMNIBOT_SESSION_TOKEN` for the triggering click, `dialog logs`, and `dialog handle`. A different token cannot observe or handle the original dialog.
|
|
45
|
+
- Parse JSON output from commands that return JSON.
|
|
46
|
+
- Do not claim success without verification evidence.
|
|
47
|
+
|
|
48
|
+
### Runtime Provenance
|
|
49
|
+
|
|
50
|
+
Before an agent-driven test in a source checkout, verify the command provenance:
|
|
51
|
+
|
|
52
|
+
```bash
|
|
53
|
+
command -v omnibot
|
|
54
|
+
omnibot --version
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
If the current workspace contains `pyproject.toml` and the global command is not the workspace version, run the test commands as `uv run omnibot ...` (or use the workspace `.venv/bin/omnibot`) so the agent exercises the code under test. Do not infer command support from memory or from a different packaged version; run the relevant subcommand `--help` after selecting the runtime.
|
|
58
|
+
|
|
59
|
+
For a source-checkout test, make this concrete before the first browser action:
|
|
60
|
+
stop any existing daemon, start the workspace daemon with
|
|
61
|
+
`uv run python -m omnibot --api-port 18764 --ws-port 18765 daemon run`, and
|
|
62
|
+
invoke every subsequent CLI command as `uv run omnibot --no-start ...` (or the
|
|
63
|
+
workspace `.venv/bin/omnibot --no-start ...`). Never use bare `omnibot` in that
|
|
64
|
+
workflow, because it can auto-start the globally installed packaged daemon and
|
|
65
|
+
silently test different code.
|
|
66
|
+
|
|
67
|
+
## Native Command Router
|
|
68
|
+
|
|
69
|
+
Before any page-state operation, choose the narrowest native command that expresses the intent. JavaScript is not a convenience shortcut; it is a fallback that requires evidence that native commands could not reach the state.
|
|
70
|
+
|
|
71
|
+
| Intent | Native command path | Do not start with |
|
|
72
|
+
| --- | --- | --- |
|
|
73
|
+
| Current page identity | `get title`, `get url`, then `snapshot -i` or `read` if needed | `execute-js "return document.title/location.href"` |
|
|
74
|
+
| Page text/content | `read --screens N`, `get text`, `get html`, `get attr`, `get count` | `execute-js "document.body.innerText"` |
|
|
75
|
+
| Iframe context | `frame <iframe-id/name/title/src>`, then `snapshot`/page-state commands | Assuming host selectors reach child frames |
|
|
76
|
+
| Click | `find ... --action click` -> `snapshot -i` + `click @eN` -> `click selector` -> `dom click` -> `mouse click` | `document.querySelector(...).click()` |
|
|
77
|
+
| Scroll | `scroll`, `scrollintoview`, `dom scroll`, then `mouse scroll` | `window.scrollTo`, `element.scrollIntoView()` |
|
|
78
|
+
| Fill/type | Text entry: `find ... --action fill`, `fill`, or `type`; key semantics: `press`/`keyboard` | setting `.value` or `innerHTML` in JS |
|
|
79
|
+
| Wait | `wait` by selector/text/url/load/fn | shell sleep or JS timers |
|
|
80
|
+
|
|
81
|
+
If you are about to write `document.querySelector`, `document.body.innerText`, `.click()`, `.value =`, `innerHTML`, `scrollTo`, or `scrollIntoView`, stop and identify the native command you will try first. Only use `execute-js` after documenting which native tiers failed and how you will verify the result.
|
|
82
|
+
|
|
83
|
+
Text entry rule: when the user asks to enter, type, fill, or replace text, use `fill` or `type`. `keyboard` and `press` dispatch key events for shortcuts and key semantics; they are not text insertion commands and must not be substituted for text entry.
|
|
84
|
+
|
|
85
|
+
Clipboard rule: both `clipboard read` and `clipboard write` require the explicit
|
|
86
|
+
`--tab-id <TAB_ID>` on every invocation, even though the clipboard is shared by
|
|
87
|
+
the browser extension. If no tab id is known, discover a safe target tab first;
|
|
88
|
+
never omit the flag or infer a default tab.
|
|
89
|
+
|
|
90
|
+
Cross-origin frame rule: snapshot visibility does not guarantee write access. After targeting a cross-origin iframe, verify `fill`/`type` focus and the resulting value; if focus fails, report the limitation explicitly and do not use JavaScript as a bypass.
|
|
91
|
+
|
|
92
|
+
## Workflow Context + Tab Target
|
|
93
|
+
|
|
94
|
+
Every page-state workflow must use both controls:
|
|
95
|
+
|
|
96
|
+
- `OMNIBOT_SESSION_TOKEN=<workflow-name>` for workflow isolation.
|
|
97
|
+
- `--tab-id <TAB_ID>` for page targeting.
|
|
98
|
+
|
|
99
|
+
OMNIBOT_SESSION_TOKEN is a workflow/context token, not a browser session and not a tab target. It isolates agent state such as refs, trace, recording, aliases, and temporary resources. Tab ID controls which page receives the command.
|
|
100
|
+
|
|
101
|
+
Do not rely on default tab, active tab, current tab, or prior targeting state. Every page-state command requires explicit `--tab-id` except when the command is only discovering or creating a tab.
|
|
102
|
+
|
|
103
|
+
Use the same token inside one workflow. Use different tokens for independent workflows.
|
|
104
|
+
|
|
105
|
+
Standard form:
|
|
106
|
+
|
|
107
|
+
```bash
|
|
108
|
+
OMNIBOT_SESSION_TOKEN=research omnibot snapshot -i --tab-id <TAB_ID>
|
|
109
|
+
OMNIBOT_SESSION_TOKEN=research omnibot click --tab-id <TAB_ID> @e4
|
|
110
|
+
OMNIBOT_SESSION_TOKEN=research omnibot snapshot -i --tab-id <TAB_ID>
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
`@eN` refs are tab-scoped. Refs rule: never reuse `@eN` refs across tabs.
|
|
114
|
+
|
|
115
|
+
The agent dispatch pattern is token + tab-id:
|
|
116
|
+
|
|
117
|
+
1. Set a stable `OMNIBOT_SESSION_TOKEN`.
|
|
118
|
+
2. Discover or create the target tab.
|
|
119
|
+
3. Save the returned tab id.
|
|
120
|
+
4. Pass `--tab-id <TAB_ID>` on every command that touches page state.
|
|
121
|
+
5. Run observe -> act -> verify on that same tab.
|
|
122
|
+
|
|
123
|
+
Tab-id identity rule: use the exact `id` field returned by Omnibot `tabs`/`tab list` or by a tab-creation command. Browser-control surfaces may expose a shorter numeric `tab_id`; that value is not necessarily the Omnibot session id. When the discovery output contains both `tab_id` and `id`, pass the full namespaced `id` (for example, `edge-...:12345`) to `--tab-id`. Never reconstruct or strip the namespace.
|
|
124
|
+
|
|
125
|
+
Phased workflow rule: do not bundle navigation and several page actions into one unverified sequence. After creating or navigating a tab, first run `get url` (and `snapshot -i` when controls are involved); then perform one meaningful action; then verify its concrete state before issuing the next action. For a form or multi-control task, repeat this observe -> one action -> verify cycle for each control.
|
|
126
|
+
|
|
127
|
+
Examples of tab-locked page-state commands:
|
|
128
|
+
|
|
129
|
+
```bash
|
|
130
|
+
OMNIBOT_SESSION_TOKEN=checkout omnibot get value "input[name=email]" --tab-id <TAB_ID>
|
|
131
|
+
OMNIBOT_SESSION_TOKEN=checkout omnibot find placeholder "Search" --action type --action-value "omnibot" --tab-id <TAB_ID>
|
|
132
|
+
OMNIBOT_SESSION_TOKEN=checkout omnibot dom dblclick n1 --tab-id <TAB_ID>
|
|
133
|
+
OMNIBOT_SESSION_TOKEN=checkout omnibot clipboard read --tab-id <TAB_ID>
|
|
134
|
+
```
|
|
135
|
+
|
|
136
|
+
Targeting categories:
|
|
137
|
+
|
|
138
|
+
| Command category | Rule |
|
|
139
|
+
| --- | --- |
|
|
140
|
+
| Page-state commands: `snapshot`, `click`, `fill`, `type`, `get`, `is`, `wait`, `screenshot`, `console`, `network`, `execute-js`, `dom`, `mouse`, `clipboard`, `viewport`, `assets`, `goto`, same-tab `navigate` | Require `--tab-id <TAB_ID>`. |
|
|
141
|
+
| Discovery or tab creation: `tabs`, `tab list`, `tab new <URL>`, `open <URL>`, `navigate <URL>` | Return or discover the tab id; save it for later page-state commands. |
|
|
142
|
+
| Read URL exception: `read <URL>` | Opens a temporary read tab and does not need `--tab-id`; reading an existing tab still requires `read --tab-id <TAB_ID>`. |
|
|
143
|
+
|
|
144
|
+
Navigation safety invariant:
|
|
145
|
+
|
|
146
|
+
- When the task says "open", "open a new tab", or "create a test page", use `open <URL>` or `tab new <URL>` and use the returned tab id. Do not substitute `goto`, plain `navigate`, or the currently active tab.
|
|
147
|
+
- Use `goto --tab-id <TAB_ID>` or `navigate --same-tab --tab-id <TAB_ID>` only when the task explicitly asks to reuse an existing tab.
|
|
148
|
+
- After creating a tab, verify `get url --tab-id <TAB_ID>` before any clipboard, upload, form, or other page-state operation. This prevents a failed or misinterpreted tab-creation result from mutating a user-owned page.
|
|
149
|
+
- Tool-created tabs are temporary workflow resources and may be auto-closed by cleanup after inactivity. Complete the intended tab workflow in the same token/context, keep the returned tab id, and do not assume it will still exist after a long model pause or a separate one-shot agent process. If a later close reports `not found`, re-list tabs before retrying; the cleanup may already have closed it.
|
|
150
|
+
- If `open` or `tab new` times out, loses its final response, or is retried after an agent/tool interruption, first run `tabs` or `tab list` and match the requested URL. Reuse one matching tab id instead of issuing another create command. Only create again after confirming that no matching tab from the interrupted attempt exists.
|
|
151
|
+
|
|
152
|
+
## Observe -> Act -> Verify
|
|
153
|
+
|
|
154
|
+
All page operations must follow this loop:
|
|
155
|
+
|
|
156
|
+
1. Observe the current page state.
|
|
157
|
+
2. Act once using the highest reliable pattern available.
|
|
158
|
+
3. Verify the expected state change.
|
|
159
|
+
|
|
160
|
+
Click example:
|
|
161
|
+
|
|
162
|
+
```bash
|
|
163
|
+
OMNIBOT_SESSION_TOKEN=research omnibot snapshot -i --tab-id <TAB_ID>
|
|
164
|
+
OMNIBOT_SESSION_TOKEN=research omnibot click --tab-id <TAB_ID> @e4
|
|
165
|
+
OMNIBOT_SESSION_TOKEN=research omnibot snapshot -i --tab-id <TAB_ID>
|
|
166
|
+
```
|
|
167
|
+
|
|
168
|
+
Read/verify example:
|
|
169
|
+
|
|
170
|
+
```bash
|
|
171
|
+
OMNIBOT_SESSION_TOKEN=checkout omnibot is enabled "button[type=submit]" --tab-id <TAB_ID>
|
|
172
|
+
OMNIBOT_SESSION_TOKEN=checkout omnibot find role button --name "Submit" --action click --tab-id <TAB_ID>
|
|
173
|
+
OMNIBOT_SESSION_TOKEN=checkout omnibot wait --url "/dashboard" --tab-id <TAB_ID>
|
|
174
|
+
OMNIBOT_SESSION_TOKEN=checkout omnibot get url --tab-id <TAB_ID>
|
|
175
|
+
```
|
|
176
|
+
|
|
177
|
+
If verification fails, do not repeat blindly. Re-observe, choose the next fallback tier, and verify again.
|
|
178
|
+
|
|
179
|
+
## Pattern > Command
|
|
180
|
+
|
|
181
|
+
Start from the task, not the command name. Use operation patterns for read, click, fill, select/check, navigation, wait, extraction, and batch.
|
|
182
|
+
|
|
183
|
+
Use fallback tiers only after standard patterns fail. Collect evidence separately from fallback execution. The command reference is only a lookup table; it must not decide behavior.
|
|
184
|
+
|
|
185
|
+
## Read vs Snapshot Routing
|
|
186
|
+
|
|
187
|
+
Choose by intent before choosing by command name:
|
|
188
|
+
|
|
189
|
+
| Intent | Prefer | Why |
|
|
190
|
+
| --- | --- | --- |
|
|
191
|
+
| Summarize or extract page content, article text, search results, feeds, or long/lazy pages | `read` | Returns clean text/Markdown for agent reasoning. |
|
|
192
|
+
| Quickly observe current visible UI structure before deciding what to do next | `snapshot -i` | Returns actionable refs and page structure for routing. |
|
|
193
|
+
| Find buttons, links, inputs, or refs for a later action | `find` or `snapshot -i` | Produces actionable targets; `read` does not. |
|
|
194
|
+
| Verify one concrete condition | `get`, `is`, or `wait` | Narrow evidence is more reliable than dumping a page. |
|
|
195
|
+
|
|
196
|
+
Do not use `read` as the first step for click/fill workflows unless the user explicitly asked for page content first.
|
|
197
|
+
|
|
198
|
+
## Dropdowns And Custom Comboboxes
|
|
199
|
+
|
|
200
|
+
`snapshot -i` may auto-probe visible custom comboboxes and append their options as refs. If the output lists `@eN [option] "..."`, click that option ref directly; Omnibot will reopen the owning combobox through `openerSelector` if the dropdown closed during snapshot.
|
|
201
|
+
|
|
202
|
+
Use `select @combobox "value"` for native selects or simple controls. Use `click @option` when `snapshot -i` exposes auto-probed combobox option refs. Do not manually reopen every dropdown just to discover options unless `snapshot -i` did not expose option refs.
|
|
203
|
+
|
|
204
|
+
## Rich Text Article Editors
|
|
205
|
+
|
|
206
|
+
ContentEditable rich text editors (ProseMirror, Quill, Draft.js, Slate, ByteDance/Toutiao article body, etc.) are often missing from the AX tree. `snapshot -i` appends them under `# DOM Rich Text Editors` as `@eN [richtext] "..." [contenteditable=true]` refs.
|
|
207
|
+
|
|
208
|
+
- Detect the body editor: `snapshot -i` → look for `@eN [richtext]` under `# DOM Rich Text Editors`. Do not fill the article body through a `[textbox]` title ref.
|
|
209
|
+
- Replace the whole body: `fill @richtext "paragraph 1\n\nparagraph 2"`. Empty lines split paragraphs; the previous content is cleared and `input`/`change` events are dispatched.
|
|
210
|
+
- Append to the body: `type @richtext " additional text"`.
|
|
211
|
+
- Focus before keyboard input: click `@richtext` first, then `press`/`keyboard`.
|
|
212
|
+
- Raw selector fallback: `omnibot fill '[contenteditable="true"]' "..."` also routes through the rich text path when the selector looks like a rich editor.
|
|
213
|
+
|
|
214
|
+
Do not use `execute-js` first for rich text writes; `fill`/`type` on the `[richtext]` ref is the supported path.
|
|
215
|
+
|
|
216
|
+
## Quick Routing
|
|
217
|
+
|
|
218
|
+
| Need | Read |
|
|
219
|
+
| --- | --- |
|
|
220
|
+
| Runtime check | `references/runtime-and-status.md` |
|
|
221
|
+
| Reading pages | `references/operation-patterns.md#read` |
|
|
222
|
+
| Clicking | `references/operation-patterns.md#click` |
|
|
223
|
+
| Filling forms | `references/operation-patterns.md#fill` |
|
|
224
|
+
| Scrolling | `references/operation-patterns.md#scroll` |
|
|
225
|
+
| Select / check | `references/operation-patterns.md#select--check` |
|
|
226
|
+
| Navigation | `references/operation-patterns.md#navigation` |
|
|
227
|
+
| Waiting | `references/operation-patterns.md#wait` |
|
|
228
|
+
| Extraction | `references/operation-patterns.md#extraction` |
|
|
229
|
+
| Batch | `references/operation-patterns.md#batch` |
|
|
230
|
+
| Fallback | `references/fallback-operations.md` |
|
|
231
|
+
| Tabs and sessions | `references/session-and-tabs.md` |
|
|
232
|
+
| Debug evidence | `references/debugging-and-evidence.md` |
|
|
233
|
+
| Network/API capture | `references/debugging-and-evidence.md#network-capture` |
|
|
234
|
+
| Commands | `references/command-reference.md` |
|
|
235
|
+
| Anti-patterns | `references/anti-patterns.md` |
|
|
236
|
+
|
|
237
|
+
## Runtime First
|
|
238
|
+
|
|
239
|
+
Before troubleshooting browser behavior, check runtime state:
|
|
240
|
+
|
|
241
|
+
```bash
|
|
242
|
+
omnibot doctor
|
|
243
|
+
omnibot tabs
|
|
244
|
+
omnibot visibility status
|
|
245
|
+
omnibot browser current
|
|
246
|
+
omnibot license status
|
|
247
|
+
```
|
|
248
|
+
|
|
249
|
+
If the extension is not connected, open Chrome or Edge with the omnibot extension loaded and keep an HTTP/HTTPS tab open.
|
|
250
|
+
|
|
251
|
+
## Fallback Discipline
|
|
252
|
+
|
|
253
|
+
Fallback tiers are for completing operations after better patterns fail. They are not debugging shortcuts.
|
|
254
|
+
|
|
255
|
+
Tier order:
|
|
256
|
+
|
|
257
|
+
1. Semantic `find`.
|
|
258
|
+
2. `snapshot -i` refs.
|
|
259
|
+
3. Selectors.
|
|
260
|
+
4. `dom` fallback.
|
|
261
|
+
5. `mouse` fallback.
|
|
262
|
+
6. `execute-js` fallback.
|
|
263
|
+
7. Raw `cdp` fallback.
|
|
264
|
+
|
|
265
|
+
When entering a lower tier, state why the higher tier failed. After any fallback action, verify with `snapshot`, `get`, `is`, `wait`, or equivalent evidence.
|
|
266
|
+
|
|
267
|
+
Before `execute-js`, the agent must have native-attempt evidence: the exact `find`/`snapshot`/`click`/`fill`/`scroll`/`get`/`read` commands tried, their result, why they were insufficient, and the post-JS verification command to run. Without that evidence, use a native command instead.
|
|
268
|
+
|
|
269
|
+
## Debug Evidence
|
|
270
|
+
|
|
271
|
+
Debugging is for evidence. Fallback is for completing operations.
|
|
272
|
+
|
|
273
|
+
Use screenshots, annotated screenshots, console logs, network logs, trace, record/replay, and CDP inspection to explain failures or prove state. Do not treat `execute-js` as ordinary debug output.
|
|
274
|
+
|
|
275
|
+
## Absolute Prohibitions
|
|
276
|
+
|
|
277
|
+
- Missing `OMNIBOT_SESSION_TOKEN` or missing `--tab-id` on page-state commands.
|
|
278
|
+
- Acting without verify or claiming success without evidence.
|
|
279
|
+
- Using `execute-js` first, raw CSS before semantic find, `@eN` across tabs, implicit tab targeting, or shell sleep instead of `omnibot wait`.
|
|
280
|
+
- Using JavaScript read/click/scroll shortcuts when native `get`/`read`/`find`/`snapshot`/`click`/`scroll` can express the task.
|
|
281
|
+
- Using removed commands: `switch-tab`, `focus-tab`, `tab switch`, `tab focus`. These are no longer available.
|
|
282
|
+
- Using `tabs[0]` or the first tab as a target. The first tab is often a user tab or transport tab.
|
|
283
|
+
- Closing user tabs discovered during cleanup. Only close tabs created by the current workflow.
|
|
284
|
+
|
|
285
|
+
See `references/anti-patterns.md` before using shortcuts.
|