@omniaibot/win-x64 1.6.5 → 1.6.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/bin/omnibot-windows-x64/81d243bd2c585b0f4821__mypyc.pyd +0 -0
- package/bin/omnibot-windows-x64/VERSION +1 -1
- package/bin/omnibot-windows-x64/_asyncio.pyd +0 -0
- package/bin/omnibot-windows-x64/_bz2.pyd +0 -0
- package/bin/omnibot-windows-x64/_cffi_backend.pyd +0 -0
- package/bin/omnibot-windows-x64/_ctypes.pyd +0 -0
- package/bin/omnibot-windows-x64/_decimal.pyd +0 -0
- package/bin/omnibot-windows-x64/_hashlib.pyd +0 -0
- package/bin/omnibot-windows-x64/_lzma.pyd +0 -0
- package/bin/omnibot-windows-x64/_multiprocessing.pyd +0 -0
- package/bin/omnibot-windows-x64/_overlapped.pyd +0 -0
- package/bin/omnibot-windows-x64/_queue.pyd +0 -0
- package/bin/omnibot-windows-x64/_socket.pyd +0 -0
- package/bin/omnibot-windows-x64/_ssl.pyd +0 -0
- package/bin/omnibot-windows-x64/_uuid.pyd +0 -0
- package/bin/omnibot-windows-x64/_wmi.pyd +0 -0
- package/bin/omnibot-windows-x64/certifi/cacert.pem +70 -0
- package/bin/omnibot-windows-x64/charset_normalizer/cd.pyd +0 -0
- package/bin/omnibot-windows-x64/charset_normalizer/md.pyd +0 -0
- package/bin/omnibot-windows-x64/cryptography/hazmat/bindings/_rust.pyd +0 -0
- package/bin/omnibot-windows-x64/libcrypto-3.dll +0 -0
- package/bin/omnibot-windows-x64/libffi-8.dll +0 -0
- package/bin/omnibot-windows-x64/libssl-3.dll +0 -0
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/SKILL.md +220 -261
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/anti-patterns.md +57 -57
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/command-reference.md +709 -755
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/css-only-dropdown-js.md +50 -50
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/debugging-and-evidence.md +129 -129
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/fallback-operations.md +177 -177
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/operation-patterns.md +432 -432
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/runtime-and-status.md +121 -121
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/session-and-tabs.md +164 -164
- package/bin/omnibot-windows-x64/omnibot/sop/tmwebdriver_sop.md +128 -128
- package/bin/omnibot-windows-x64/omnibot/sop/vue3_component_sop.md +163 -163
- package/bin/omnibot-windows-x64/omnibot-windows-x64.exe +0 -0
- package/bin/omnibot-windows-x64/pyexpat.pyd +0 -0
- package/bin/omnibot-windows-x64/python3.dll +0 -0
- package/bin/omnibot-windows-x64/python312.dll +0 -0
- package/bin/omnibot-windows-x64/select.pyd +0 -0
- package/bin/omnibot-windows-x64/unicodedata.pyd +0 -0
- package/bin/omnibot-windows-x64/vcruntime140.dll +0 -0
- package/bin/omnibot-windows-x64/vcruntime140_1.dll +0 -0
- package/package.json +1 -1
- package/bin/omnibot-windows-x64/libcrypto-3-x64.dll +0 -0
- package/bin/omnibot-windows-x64/libssl-3-x64.dll +0 -0
- package/bin/omnibot-windows-x64/python313.dll +0 -0
|
@@ -1,432 +1,432 @@
|
|
|
1
|
-
# Operation Patterns
|
|
2
|
-
|
|
3
|
-
Use this file to choose behavior by task. Every page-state workflow must set `OMNIBOT_SESSION_TOKEN=<workflow-name>` and pass `--tab-id <TAB_ID>` on each page-state command.
|
|
4
|
-
|
|
5
|
-
## Read
|
|
6
|
-
|
|
7
|
-
### When to use
|
|
8
|
-
|
|
9
|
-
Use Read when the agent needs clean rendered page content: article text, search results, feed items, long pages, lazy-loaded text, or a human-readable summary source.
|
|
10
|
-
|
|
11
|
-
Use targeted state commands when the agent needs one value, visibility, enabled/checked state, layout box, or computed styles. Use `snapshot -i` when the agent needs current visible UI structure for deciding the next action.
|
|
12
|
-
|
|
13
|
-
### Preferred sequence
|
|
14
|
-
|
|
15
|
-
1. `read --tab-id <TAB_ID>` for clean page text on an existing tab.
|
|
16
|
-
2. `read --screens N --tab-id <TAB_ID>` for longer or lazy-loaded pages.
|
|
17
|
-
3. `read <URL>` when opening a temporary tab only for reading that URL.
|
|
18
|
-
4. `get title/url/text/html/value/attr/count/box/styles` for narrow evidence.
|
|
19
|
-
5. `is visible/enabled/checked` for boolean state.
|
|
20
|
-
5. `snapshot -i` when actionable refs are needed.
|
|
21
|
-
6. `dom visible` or `execute-js` fallback only after standard reads cannot access the state.
|
|
22
|
-
|
|
23
|
-
### Example
|
|
24
|
-
|
|
25
|
-
```bash
|
|
26
|
-
OMNIBOT_SESSION_TOKEN=research omnibot read --screens 5 --tab-id <TAB_ID>
|
|
27
|
-
OMNIBOT_SESSION_TOKEN=research omnibot read --screens 3 https://example.com/article
|
|
28
|
-
```
|
|
29
|
-
|
|
30
|
-
### Verification
|
|
31
|
-
|
|
32
|
-
Use the narrowest read that proves the claim:
|
|
33
|
-
|
|
34
|
-
```bash
|
|
35
|
-
OMNIBOT_SESSION_TOKEN=research omnibot get title --tab-id <TAB_ID>
|
|
36
|
-
OMNIBOT_SESSION_TOKEN=research omnibot is visible "#main" --tab-id <TAB_ID>
|
|
37
|
-
OMNIBOT_SESSION_TOKEN=research omnibot get count ".result" --tab-id <TAB_ID>
|
|
38
|
-
```
|
|
39
|
-
|
|
40
|
-
### Fallback entry point
|
|
41
|
-
|
|
42
|
-
If `read` or targeted reads omit needed runtime state, go to `fallback-operations.md#fallback-tier-model`. Start with DOM fallback before JavaScript.
|
|
43
|
-
|
|
44
|
-
## Click
|
|
45
|
-
|
|
46
|
-
### When to use
|
|
47
|
-
|
|
48
|
-
Use Click for buttons, links, menu items, cards, checkouts, modal actions, and any user-like click target.
|
|
49
|
-
|
|
50
|
-
### Preferred sequence
|
|
51
|
-
|
|
52
|
-
1. `find role/text/testid --action click`
|
|
53
|
-
2. `snapshot -i` -> `click @eN`
|
|
54
|
-
3. `click selector`
|
|
55
|
-
4. `dom click`
|
|
56
|
-
5. `mouse click`
|
|
57
|
-
6. `execute-js` click
|
|
58
|
-
7. `cdp` fallback
|
|
59
|
-
|
|
60
|
-
### Example
|
|
61
|
-
|
|
62
|
-
```bash
|
|
63
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot snapshot -i --tab-id <TAB_ID>
|
|
64
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot find role button --name "Submit" --action click --tab-id <TAB_ID>
|
|
65
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot wait --url "/dashboard" --tab-id <TAB_ID>
|
|
66
|
-
```
|
|
67
|
-
|
|
68
|
-
If semantic find is ambiguous, use refs from the same tab:
|
|
69
|
-
|
|
70
|
-
```bash
|
|
71
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot snapshot -i --tab-id <TAB_ID>
|
|
72
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot click --tab-id <TAB_ID> @e4
|
|
73
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot snapshot -i --tab-id <TAB_ID>
|
|
74
|
-
```
|
|
75
|
-
|
|
76
|
-
### Verification
|
|
77
|
-
|
|
78
|
-
Verify the expected state, not just command success:
|
|
79
|
-
|
|
80
|
-
```bash
|
|
81
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot get url --tab-id <TAB_ID>
|
|
82
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot is visible ".success" --tab-id <TAB_ID>
|
|
83
|
-
```
|
|
84
|
-
|
|
85
|
-
### Fallback entry point
|
|
86
|
-
|
|
87
|
-
If semantic and refs fail, document why and continue at `fallback-operations.md#selector-fallback`.
|
|
88
|
-
|
|
89
|
-
### Static-text refs
|
|
90
|
-
|
|
91
|
-
When `snapshot -i` shows only `StaticText` for a visible control label, do not assume the text ref itself is clickable. Prefer, in order:
|
|
92
|
-
|
|
93
|
-
1. `find role button --name "<label>" --action click`.
|
|
94
|
-
2. `click "text=<label>"`.
|
|
95
|
-
3. `get box "text=<label>"` followed by `mouse click <center-x> <center-y>`.
|
|
96
|
-
4. CDP/JS fallback only after verifying the semantic and coordinate paths failed.
|
|
97
|
-
|
|
98
|
-
## Fill
|
|
99
|
-
|
|
100
|
-
### When to use
|
|
101
|
-
|
|
102
|
-
Use Fill for text inputs, textareas, search boxes, login fields, form fields where the final value must be controlled, and rich text article body editors (contenteditable ProseMirror/Quill/Draft.js/Slate/Toutiao article body).
|
|
103
|
-
|
|
104
|
-
### Preferred sequence
|
|
105
|
-
|
|
106
|
-
1. `find label/placeholder --action fill`
|
|
107
|
-
2. `snapshot -i` -> `fill @eN`
|
|
108
|
-
3. `fill selector`
|
|
109
|
-
4. `focus` + `type`
|
|
110
|
-
5. `execute-js` set value and dispatch `input`/`change` events
|
|
111
|
-
|
|
112
|
-
### Example
|
|
113
|
-
|
|
114
|
-
```bash
|
|
115
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot find label "Email" --action fill --action-value "a@b.com" --tab-id <TAB_ID>
|
|
116
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot get value "input[name=email]" --tab-id <TAB_ID>
|
|
117
|
-
```
|
|
118
|
-
|
|
119
|
-
With refs:
|
|
120
|
-
|
|
121
|
-
```bash
|
|
122
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot snapshot -i --tab-id <TAB_ID>
|
|
123
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot fill --tab-id <TAB_ID> @e2 "a@b.com"
|
|
124
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot get value "input[name=email]" --tab-id <TAB_ID>
|
|
125
|
-
```
|
|
126
|
-
|
|
127
|
-
### Rich text article body
|
|
128
|
-
|
|
129
|
-
For article body editors exposed as `@eN [richtext] "..." [contenteditable=true]` under `# DOM Rich Text Editors`:
|
|
130
|
-
|
|
131
|
-
```bash
|
|
132
|
-
OMNIBOT_SESSION_TOKEN=editor omnibot snapshot -i --tab-id <TAB_ID>
|
|
133
|
-
# Body appears under "# DOM Rich Text Editors": @eN [richtext] "..."
|
|
134
|
-
OMNIBOT_SESSION_TOKEN=editor omnibot fill --tab-id <TAB_ID> @eN "First paragraph.\n\nSecond paragraph."
|
|
135
|
-
OMNIBOT_SESSION_TOKEN=editor omnibot snapshot -i --tab-id <TAB_ID> # verify body content replaced
|
|
136
|
-
```
|
|
137
|
-
|
|
138
|
-
- `\n\n` splits paragraphs; previous content is cleared.
|
|
139
|
-
- `type @richtext " text"` appends at the cursor (click the ref first to focus).
|
|
140
|
-
- Do NOT write the article body into a `[textbox]` title field. Title and richtext body are separate refs.
|
|
141
|
-
|
|
142
|
-
### Verification
|
|
143
|
-
|
|
144
|
-
Use `get value`, visible validation text, enabled state, or submit readiness:
|
|
145
|
-
|
|
146
|
-
```bash
|
|
147
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot get value "input[name=email]" --tab-id <TAB_ID>
|
|
148
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot is enabled "button[type=submit]" --tab-id <TAB_ID>
|
|
149
|
-
```
|
|
150
|
-
|
|
151
|
-
For richtext bodies, re-run `snapshot -i` and confirm the `@eN [richtext]` name matches the intended content.
|
|
152
|
-
|
|
153
|
-
### Cursor positioning and inline image insertion
|
|
154
|
-
|
|
155
|
-
For `contenteditable` article bodies (ProseMirror/Quill/Draft.js/Slate/Toutiao article body), `find text "<anchor phrase>" --action click` collapses the editor selection at the end of the matched phrase and focuses the editor. Use this to place the caret at a specific paragraph before inserting an image, typing, or pressing keys. Do not rely on a bare `click` of the editor followed by `press End`, because the caret would stay on the first line.
|
|
156
|
-
|
|
157
|
-
Insert an image after a specific paragraph:
|
|
158
|
-
|
|
159
|
-
```bash
|
|
160
|
-
OMNIBOT_SESSION_TOKEN=editor omnibot find text "第二段锚点文字" --action click --tab-id <TAB_ID>
|
|
161
|
-
OMNIBOT_SESSION_TOKEN=editor omnibot click ".syl-toolbar-tool.image .syl-toolbar-button" --tab-id <TAB_ID>
|
|
162
|
-
OMNIBOT_SESSION_TOKEN=editor omnibot wait --selector ".upload-image-panel input[type=file]" --tab-id <TAB_ID>
|
|
163
|
-
OMNIBOT_SESSION_TOKEN=editor omnibot upload ".upload-image-panel input[type=file]" "/path/to/image.png" --tab-id <TAB_ID>
|
|
164
|
-
OMNIBOT_SESSION_TOKEN=editor omnibot find role button --name "确定" --action click --tab-id <TAB_ID>
|
|
165
|
-
```
|
|
166
|
-
|
|
167
|
-
Verify the inserted image by counting visible image cards, not raw `<img>` nodes. Toutiao/ProseMirror image cards keep a hidden `<templ>` template and a visible `<mask>` copy of the same `<img>`, so `.ProseMirror img` returns 2 for one logical image. Count the image card container instead (e.g. `.ProseMirror [__syl_tag=true]` or `.ProseMirror .pgc-image`), which reports one per inserted image.
|
|
168
|
-
|
|
169
|
-
### Fallback entry point
|
|
170
|
-
|
|
171
|
-
If fill does not update the application state, use `fallback-operations.md#javascript-fallback` only after selector/focus/type attempts fail. JavaScript must dispatch real `input` and `change` events.
|
|
172
|
-
|
|
173
|
-
## Select / Check
|
|
174
|
-
|
|
175
|
-
### When to use
|
|
176
|
-
|
|
177
|
-
Use this pattern for dropdowns, comboboxes, checkboxes, toggles, agreement boxes, filters, and boolean controls.
|
|
178
|
-
|
|
179
|
-
### Preferred sequence
|
|
180
|
-
|
|
181
|
-
1. Use `snapshot -i` first; visible comboboxes may include auto-probed option refs.
|
|
182
|
-
2. Use `click @option` if `snapshot -i` lists auto-probed combobox option refs.
|
|
183
|
-
3. Use `select @combobox "value"` for native selects or simple select-like controls.
|
|
184
|
-
4. Use `check` / `uncheck` for boolean controls.
|
|
185
|
-
5. Verify selected label, value, URL query, result count, or dependent page state.
|
|
186
|
-
|
|
187
|
-
`snapshot -i` can list options for visible custom comboboxes even though the dropdown closes during observation. These refs are still actionable because Omnibot stores the option's `openerSelector` and can reopen the owning combobox before clicking the option.
|
|
188
|
-
|
|
189
|
-
### Example
|
|
190
|
-
|
|
191
|
-
```bash
|
|
192
|
-
OMNIBOT_SESSION_TOKEN=form omnibot snapshot -i --tab-id <TAB_ID>
|
|
193
|
-
OMNIBOT_SESSION_TOKEN=form omnibot click @e22 --tab-id <TAB_ID> # @e22 [option] "OpenAI Chat"
|
|
194
|
-
OMNIBOT_SESSION_TOKEN=form omnibot get url --tab-id <TAB_ID>
|
|
195
|
-
```
|
|
196
|
-
|
|
197
|
-
Checkbox example:
|
|
198
|
-
|
|
199
|
-
```bash
|
|
200
|
-
OMNIBOT_SESSION_TOKEN=form omnibot is checked "#agree" --tab-id <TAB_ID>
|
|
201
|
-
OMNIBOT_SESSION_TOKEN=form omnibot check "#agree" --tab-id <TAB_ID>
|
|
202
|
-
OMNIBOT_SESSION_TOKEN=form omnibot is checked "#agree" --tab-id <TAB_ID>
|
|
203
|
-
```
|
|
204
|
-
|
|
205
|
-
### Verification
|
|
206
|
-
|
|
207
|
-
For checkboxes, verify boolean state. For selects, verify value or dependent page state.
|
|
208
|
-
|
|
209
|
-
### CSS-only dropdowns and custom menus
|
|
210
|
-
|
|
211
|
-
Some dropdowns, hover menus, tooltips, and custom selects are CSS-only or portal-based widgets. The visible trigger may be accessible, while the option nodes are hidden `<div>` elements without `role`, accessible name, or stable refs. Start with `snapshot -i`; visible comboboxes may include auto-probed option refs before any manual dropdown opening.
|
|
212
|
-
|
|
213
|
-
Escalation order:
|
|
214
|
-
|
|
215
|
-
1. Run `snapshot -i` and look for `[option]` refs near the combobox.
|
|
216
|
-
2. If an option ref exists, click the option ref directly and verify.
|
|
217
|
-
3. If options are absent, click/hover the trigger once, run `snapshot -i` again, then click the option ref if it appears.
|
|
218
|
-
4. If trigger interaction works but options remain absent or option refs fail verification, use `fallback-operations.md#css-only-dropdown-javascript-fallback`.
|
|
219
|
-
|
|
220
|
-
Do not loop on manual open -> snapshot -> open -> snapshot when `snapshot -i` already lists auto-probed combobox options.
|
|
221
|
-
|
|
222
|
-
### Fallback entry point
|
|
223
|
-
|
|
224
|
-
If native select/check fails, use `fallback-operations.md#selector-fallback`. Use JavaScript only if events must be synthesized or if evidence shows a CSS-only multi-step widget cannot be completed through semantic, snapshot, selector, DOM, or mouse tiers.
|
|
225
|
-
|
|
226
|
-
## Navigation
|
|
227
|
-
|
|
228
|
-
### When to use
|
|
229
|
-
|
|
230
|
-
Use Navigation to open pages, create tabs, change history, reload, close tabs, move between pages, create windows, or select frames.
|
|
231
|
-
|
|
232
|
-
### Preferred sequence
|
|
233
|
-
|
|
234
|
-
1. Create or discover a tab with `tabs`, `tab list`, `tab new`, `open`, or `navigate`.
|
|
235
|
-
2. Save the returned tab id.
|
|
236
|
-
3. Verify with `get url --tab-id <TAB_ID>`.
|
|
237
|
-
4. Run all later page-state commands with `--tab-id <TAB_ID>`.
|
|
238
|
-
|
|
239
|
-
### Example
|
|
240
|
-
|
|
241
|
-
```bash
|
|
242
|
-
OMNIBOT_SESSION_TOKEN=research omnibot tab new https://example.com --label research
|
|
243
|
-
OMNIBOT_SESSION_TOKEN=research omnibot get url --tab-id <TAB_ID>
|
|
244
|
-
OMNIBOT_SESSION_TOKEN=research omnibot snapshot -i --tab-id <TAB_ID>
|
|
245
|
-
```
|
|
246
|
-
|
|
247
|
-
History and lifecycle:
|
|
248
|
-
|
|
249
|
-
```bash
|
|
250
|
-
OMNIBOT_SESSION_TOKEN=research omnibot back --tab-id <TAB_ID>
|
|
251
|
-
OMNIBOT_SESSION_TOKEN=research omnibot get url --tab-id <TAB_ID>
|
|
252
|
-
OMNIBOT_SESSION_TOKEN=research omnibot forward --tab-id <TAB_ID>
|
|
253
|
-
OMNIBOT_SESSION_TOKEN=research omnibot get url --tab-id <TAB_ID>
|
|
254
|
-
OMNIBOT_SESSION_TOKEN=research omnibot reload --tab-id <TAB_ID>
|
|
255
|
-
OMNIBOT_SESSION_TOKEN=research omnibot wait --load domcontentloaded --tab-id <TAB_ID>
|
|
256
|
-
```
|
|
257
|
-
|
|
258
|
-
Other navigation commands:
|
|
259
|
-
|
|
260
|
-
```bash
|
|
261
|
-
OMNIBOT_SESSION_TOKEN=research omnibot navigate https://example.com
|
|
262
|
-
OMNIBOT_SESSION_TOKEN=research omnibot navigate https://example.com --same-tab --tab-id <TAB_ID>
|
|
263
|
-
OMNIBOT_SESSION_TOKEN=research omnibot open https://example.com
|
|
264
|
-
OMNIBOT_SESSION_TOKEN=research omnibot goto https://example.com --tab-id <TAB_ID>
|
|
265
|
-
OMNIBOT_SESSION_TOKEN=research omnibot pushstate /dashboard --tab-id <TAB_ID>
|
|
266
|
-
OMNIBOT_SESSION_TOKEN=research omnibot close <TAB_ID>
|
|
267
|
-
OMNIBOT_SESSION_TOKEN=research omnibot tab list
|
|
268
|
-
OMNIBOT_SESSION_TOKEN=research omnibot tab close research
|
|
269
|
-
OMNIBOT_SESSION_TOKEN=research omnibot window new
|
|
270
|
-
OMNIBOT_SESSION_TOKEN=research omnibot frame main
|
|
271
|
-
```
|
|
272
|
-
|
|
273
|
-
### Verification
|
|
274
|
-
|
|
275
|
-
Always verify the final target:
|
|
276
|
-
|
|
277
|
-
```bash
|
|
278
|
-
OMNIBOT_SESSION_TOKEN=research omnibot get url --tab-id <TAB_ID>
|
|
279
|
-
OMNIBOT_SESSION_TOKEN=research omnibot wait --load networkidle --tab-id <TAB_ID>
|
|
280
|
-
OMNIBOT_SESSION_TOKEN=research omnibot snapshot -i --tab-id <TAB_ID>
|
|
281
|
-
```
|
|
282
|
-
|
|
283
|
-
### Fallback entry point
|
|
284
|
-
|
|
285
|
-
Same-tab navigation must include `--tab-id <TAB_ID>`. If the target tab id is unknown, run `omnibot tabs` first and choose the intended tab explicitly.
|
|
286
|
-
|
|
287
|
-
## Wait
|
|
288
|
-
|
|
289
|
-
### When to use
|
|
290
|
-
|
|
291
|
-
Use Wait after navigation, clicks, form submissions, reloads, dynamic rendering, lazy loading, or any action that changes asynchronous state.
|
|
292
|
-
|
|
293
|
-
### Preferred sequence
|
|
294
|
-
|
|
295
|
-
1. Wait for selector visibility or hidden state.
|
|
296
|
-
2. Wait for text.
|
|
297
|
-
3. Wait for URL.
|
|
298
|
-
4. Wait for load state.
|
|
299
|
-
5. Wait for a JavaScript condition with `--fn`.
|
|
300
|
-
6. Avoid shell sleep.
|
|
301
|
-
|
|
302
|
-
### Example
|
|
303
|
-
|
|
304
|
-
```bash
|
|
305
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot click --tab-id <TAB_ID> @e4
|
|
306
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot wait "#spinner" --state hidden --tab-id <TAB_ID>
|
|
307
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot wait --text "Welcome" --tab-id <TAB_ID>
|
|
308
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot get text "#main" --tab-id <TAB_ID>
|
|
309
|
-
```
|
|
310
|
-
|
|
311
|
-
Other waits:
|
|
312
|
-
|
|
313
|
-
```bash
|
|
314
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot wait --url "/dashboard" --tab-id <TAB_ID>
|
|
315
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot wait --load domcontentloaded --tab-id <TAB_ID>
|
|
316
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot wait --load networkidle --tab-id <TAB_ID>
|
|
317
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot wait --fn "window.appReady === true" --tab-id <TAB_ID>
|
|
318
|
-
```
|
|
319
|
-
|
|
320
|
-
### Verification
|
|
321
|
-
|
|
322
|
-
After waiting, read the final condition with `get`, `is`, `wait`, or `snapshot`.
|
|
323
|
-
|
|
324
|
-
### Fallback entry point
|
|
325
|
-
|
|
326
|
-
If waits time out, collect evidence with `debugging-and-evidence.md`; do not replace them with shell sleep.
|
|
327
|
-
|
|
328
|
-
## Extraction
|
|
329
|
-
|
|
330
|
-
### When to use
|
|
331
|
-
|
|
332
|
-
Use Extraction for retrieving page text, HTML fragments, links, counts, assets, or clipboard data.
|
|
333
|
-
|
|
334
|
-
### Preferred sequence
|
|
335
|
-
|
|
336
|
-
1. `get text/html/attr/count` for targeted extraction.
|
|
337
|
-
2. `read --screens N` for clean page text, long pages, or lazy-loaded content.
|
|
338
|
-
3. `snapshot -i` for structured page observation before action planning.
|
|
339
|
-
4. `assets list/export` for resources.
|
|
340
|
-
5. `clipboard read` only when clipboard content is part of the task.
|
|
341
|
-
|
|
342
|
-
### Example
|
|
343
|
-
|
|
344
|
-
```bash
|
|
345
|
-
OMNIBOT_SESSION_TOKEN=extract omnibot get text "#article" --tab-id <TAB_ID>
|
|
346
|
-
OMNIBOT_SESSION_TOKEN=extract omnibot get attr "a.next" href --tab-id <TAB_ID>
|
|
347
|
-
OMNIBOT_SESSION_TOKEN=extract omnibot get count ".result" --tab-id <TAB_ID>
|
|
348
|
-
```
|
|
349
|
-
|
|
350
|
-
Long page example:
|
|
351
|
-
|
|
352
|
-
```bash
|
|
353
|
-
OMNIBOT_SESSION_TOKEN=extract omnibot read --screens 5 --tab-id <TAB_ID>
|
|
354
|
-
OMNIBOT_SESSION_TOKEN=extract omnibot assets list --tab-id <TAB_ID>
|
|
355
|
-
```
|
|
356
|
-
|
|
357
|
-
### Verification
|
|
358
|
-
|
|
359
|
-
Verify extraction completeness with expected markers in `read`, element counts, or targeted reads for known selectors.
|
|
360
|
-
|
|
361
|
-
### Fallback entry point
|
|
362
|
-
|
|
363
|
-
If content exists only in runtime objects, use `fallback-operations.md#javascript-fallback` after explaining why `read`, `snapshot`, and `get` could not access it.
|
|
364
|
-
|
|
365
|
-
## Upload
|
|
366
|
-
|
|
367
|
-
### When to use
|
|
368
|
-
|
|
369
|
-
Use `omnibot upload` to attach a local file to an `<input type="file">` element. The command speaks CDP (`DOM.setFileInputFiles`) through the extension debugger transport, with a JS `DataTransfer` fallback for pages that reject programmatic file assignment.
|
|
370
|
-
|
|
371
|
-
### Preferred sequence
|
|
372
|
-
|
|
373
|
-
1. Observe the upload UI and confirm the task requires a local file.
|
|
374
|
-
2. Locate the file input via `snapshot -i` or a CSS selector.
|
|
375
|
-
3. Run `omnibot upload <selector> <local-file> --tab-id <TAB_ID>`.
|
|
376
|
-
4. Verify the file was accepted (filename, count, enabled submit state, or application-specific confirmation).
|
|
377
|
-
|
|
378
|
-
### Example
|
|
379
|
-
|
|
380
|
-
```bash
|
|
381
|
-
OMNIBOT_SESSION_TOKEN=form omnibot snapshot -i --tab-id <TAB_ID>
|
|
382
|
-
OMNIBOT_SESSION_TOKEN=form omnibot upload "input[type=file]" /path/to/cover.png --tab-id <TAB_ID>
|
|
383
|
-
OMNIBOT_SESSION_TOKEN=form omnibot is enabled "button[type=submit]" --tab-id <TAB_ID>
|
|
384
|
-
```
|
|
385
|
-
|
|
386
|
-
### Verification
|
|
387
|
-
|
|
388
|
-
Confirm the input accepted the file:
|
|
389
|
-
|
|
390
|
-
```bash
|
|
391
|
-
OMNIBOT_SESSION_TOKEN=form omnibot get "input[type=file]" attr value --tab-id <TAB_ID>
|
|
392
|
-
OMNIBOT_SESSION_TOKEN=form omnibot is enabled "button[type=submit]" --tab-id <TAB_ID>
|
|
393
|
-
```
|
|
394
|
-
|
|
395
|
-
### Fallback entry point
|
|
396
|
-
|
|
397
|
-
The CLI already falls back from CDP `DOM.setFileInputFiles` (by objectId, then nodeId) to a JS `DataTransfer` assignment. Do not add further JavaScript fallbacks on top; file inputs are security-sensitive and page-specific. If all three transports fail, report the failure to the user instead of scripting around it.
|
|
398
|
-
|
|
399
|
-
## Batch
|
|
400
|
-
|
|
401
|
-
### When to use
|
|
402
|
-
|
|
403
|
-
Use Batch for short, known-safe extension/CDP command chains where the target tab and verification are already explicit. Batch is not a wrapper for normal CLI commands such as `snapshot` or `click`.
|
|
404
|
-
|
|
405
|
-
### Preferred sequence
|
|
406
|
-
|
|
407
|
-
1. Keep batches short.
|
|
408
|
-
2. Include only deterministic operations.
|
|
409
|
-
3. Verify after the batch with a separate command.
|
|
410
|
-
4. Use `--file` for complex JSON quoting.
|
|
411
|
-
|
|
412
|
-
### Example
|
|
413
|
-
|
|
414
|
-
```bash
|
|
415
|
-
OMNIBOT_SESSION_TOKEN=research omnibot batch '[{"cmd":"cdp","method":"Runtime.evaluate","params":{"expression":"document.title","returnByValue":true}}]' --tab-id <TAB_ID>
|
|
416
|
-
OMNIBOT_SESSION_TOKEN=research omnibot get title --tab-id <TAB_ID>
|
|
417
|
-
```
|
|
418
|
-
|
|
419
|
-
File example:
|
|
420
|
-
|
|
421
|
-
```bash
|
|
422
|
-
OMNIBOT_SESSION_TOKEN=research omnibot batch --file /tmp/omnibot-batch.json --tab-id <TAB_ID>
|
|
423
|
-
OMNIBOT_SESSION_TOKEN=research omnibot get url --tab-id <TAB_ID>
|
|
424
|
-
```
|
|
425
|
-
|
|
426
|
-
### Verification
|
|
427
|
-
|
|
428
|
-
Do not treat batch success as task success. Verify final page state separately.
|
|
429
|
-
|
|
430
|
-
### Fallback entry point
|
|
431
|
-
|
|
432
|
-
Batch is not a fallback tier. If a batched operation fails, re-run the workflow step-by-step and enter the normal fallback model.
|
|
1
|
+
# Operation Patterns
|
|
2
|
+
|
|
3
|
+
Use this file to choose behavior by task. Every page-state workflow must set `OMNIBOT_SESSION_TOKEN=<workflow-name>` and pass `--tab-id <TAB_ID>` on each page-state command.
|
|
4
|
+
|
|
5
|
+
## Read
|
|
6
|
+
|
|
7
|
+
### When to use
|
|
8
|
+
|
|
9
|
+
Use Read when the agent needs clean rendered page content: article text, search results, feed items, long pages, lazy-loaded text, or a human-readable summary source.
|
|
10
|
+
|
|
11
|
+
Use targeted state commands when the agent needs one value, visibility, enabled/checked state, layout box, or computed styles. Use `snapshot -i` when the agent needs current visible UI structure for deciding the next action.
|
|
12
|
+
|
|
13
|
+
### Preferred sequence
|
|
14
|
+
|
|
15
|
+
1. `read --tab-id <TAB_ID>` for clean page text on an existing tab.
|
|
16
|
+
2. `read --screens N --tab-id <TAB_ID>` for longer or lazy-loaded pages.
|
|
17
|
+
3. `read <URL>` when opening a temporary tab only for reading that URL.
|
|
18
|
+
4. `get title/url/text/html/value/attr/count/box/styles` for narrow evidence.
|
|
19
|
+
5. `is visible/enabled/checked` for boolean state.
|
|
20
|
+
5. `snapshot -i` when actionable refs are needed.
|
|
21
|
+
6. `dom visible` or `execute-js` fallback only after standard reads cannot access the state.
|
|
22
|
+
|
|
23
|
+
### Example
|
|
24
|
+
|
|
25
|
+
```bash
|
|
26
|
+
OMNIBOT_SESSION_TOKEN=research omnibot read --screens 5 --tab-id <TAB_ID>
|
|
27
|
+
OMNIBOT_SESSION_TOKEN=research omnibot read --screens 3 https://example.com/article
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
### Verification
|
|
31
|
+
|
|
32
|
+
Use the narrowest read that proves the claim:
|
|
33
|
+
|
|
34
|
+
```bash
|
|
35
|
+
OMNIBOT_SESSION_TOKEN=research omnibot get title --tab-id <TAB_ID>
|
|
36
|
+
OMNIBOT_SESSION_TOKEN=research omnibot is visible "#main" --tab-id <TAB_ID>
|
|
37
|
+
OMNIBOT_SESSION_TOKEN=research omnibot get count ".result" --tab-id <TAB_ID>
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
### Fallback entry point
|
|
41
|
+
|
|
42
|
+
If `read` or targeted reads omit needed runtime state, go to `fallback-operations.md#fallback-tier-model`. Start with DOM fallback before JavaScript.
|
|
43
|
+
|
|
44
|
+
## Click
|
|
45
|
+
|
|
46
|
+
### When to use
|
|
47
|
+
|
|
48
|
+
Use Click for buttons, links, menu items, cards, checkouts, modal actions, and any user-like click target.
|
|
49
|
+
|
|
50
|
+
### Preferred sequence
|
|
51
|
+
|
|
52
|
+
1. `find role/text/testid --action click`
|
|
53
|
+
2. `snapshot -i` -> `click @eN`
|
|
54
|
+
3. `click selector`
|
|
55
|
+
4. `dom click`
|
|
56
|
+
5. `mouse click`
|
|
57
|
+
6. `execute-js` click
|
|
58
|
+
7. `cdp` fallback
|
|
59
|
+
|
|
60
|
+
### Example
|
|
61
|
+
|
|
62
|
+
```bash
|
|
63
|
+
OMNIBOT_SESSION_TOKEN=checkout omnibot snapshot -i --tab-id <TAB_ID>
|
|
64
|
+
OMNIBOT_SESSION_TOKEN=checkout omnibot find role button --name "Submit" --action click --tab-id <TAB_ID>
|
|
65
|
+
OMNIBOT_SESSION_TOKEN=checkout omnibot wait --url "/dashboard" --tab-id <TAB_ID>
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
If semantic find is ambiguous, use refs from the same tab:
|
|
69
|
+
|
|
70
|
+
```bash
|
|
71
|
+
OMNIBOT_SESSION_TOKEN=checkout omnibot snapshot -i --tab-id <TAB_ID>
|
|
72
|
+
OMNIBOT_SESSION_TOKEN=checkout omnibot click --tab-id <TAB_ID> @e4
|
|
73
|
+
OMNIBOT_SESSION_TOKEN=checkout omnibot snapshot -i --tab-id <TAB_ID>
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
### Verification
|
|
77
|
+
|
|
78
|
+
Verify the expected state, not just command success:
|
|
79
|
+
|
|
80
|
+
```bash
|
|
81
|
+
OMNIBOT_SESSION_TOKEN=checkout omnibot get url --tab-id <TAB_ID>
|
|
82
|
+
OMNIBOT_SESSION_TOKEN=checkout omnibot is visible ".success" --tab-id <TAB_ID>
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
### Fallback entry point
|
|
86
|
+
|
|
87
|
+
If semantic and refs fail, document why and continue at `fallback-operations.md#selector-fallback`.
|
|
88
|
+
|
|
89
|
+
### Static-text refs
|
|
90
|
+
|
|
91
|
+
When `snapshot -i` shows only `StaticText` for a visible control label, do not assume the text ref itself is clickable. Prefer, in order:
|
|
92
|
+
|
|
93
|
+
1. `find role button --name "<label>" --action click`.
|
|
94
|
+
2. `click "text=<label>"`.
|
|
95
|
+
3. `get box "text=<label>"` followed by `mouse click <center-x> <center-y>`.
|
|
96
|
+
4. CDP/JS fallback only after verifying the semantic and coordinate paths failed.
|
|
97
|
+
|
|
98
|
+
## Fill
|
|
99
|
+
|
|
100
|
+
### When to use
|
|
101
|
+
|
|
102
|
+
Use Fill for text inputs, textareas, search boxes, login fields, form fields where the final value must be controlled, and rich text article body editors (contenteditable ProseMirror/Quill/Draft.js/Slate/Toutiao article body).
|
|
103
|
+
|
|
104
|
+
### Preferred sequence
|
|
105
|
+
|
|
106
|
+
1. `find label/placeholder --action fill`
|
|
107
|
+
2. `snapshot -i` -> `fill @eN`
|
|
108
|
+
3. `fill selector`
|
|
109
|
+
4. `focus` + `type`
|
|
110
|
+
5. `execute-js` set value and dispatch `input`/`change` events
|
|
111
|
+
|
|
112
|
+
### Example
|
|
113
|
+
|
|
114
|
+
```bash
|
|
115
|
+
OMNIBOT_SESSION_TOKEN=checkout omnibot find label "Email" --action fill --action-value "a@b.com" --tab-id <TAB_ID>
|
|
116
|
+
OMNIBOT_SESSION_TOKEN=checkout omnibot get value "input[name=email]" --tab-id <TAB_ID>
|
|
117
|
+
```
|
|
118
|
+
|
|
119
|
+
With refs:
|
|
120
|
+
|
|
121
|
+
```bash
|
|
122
|
+
OMNIBOT_SESSION_TOKEN=checkout omnibot snapshot -i --tab-id <TAB_ID>
|
|
123
|
+
OMNIBOT_SESSION_TOKEN=checkout omnibot fill --tab-id <TAB_ID> @e2 "a@b.com"
|
|
124
|
+
OMNIBOT_SESSION_TOKEN=checkout omnibot get value "input[name=email]" --tab-id <TAB_ID>
|
|
125
|
+
```
|
|
126
|
+
|
|
127
|
+
### Rich text article body
|
|
128
|
+
|
|
129
|
+
For article body editors exposed as `@eN [richtext] "..." [contenteditable=true]` under `# DOM Rich Text Editors`:
|
|
130
|
+
|
|
131
|
+
```bash
|
|
132
|
+
OMNIBOT_SESSION_TOKEN=editor omnibot snapshot -i --tab-id <TAB_ID>
|
|
133
|
+
# Body appears under "# DOM Rich Text Editors": @eN [richtext] "..."
|
|
134
|
+
OMNIBOT_SESSION_TOKEN=editor omnibot fill --tab-id <TAB_ID> @eN "First paragraph.\n\nSecond paragraph."
|
|
135
|
+
OMNIBOT_SESSION_TOKEN=editor omnibot snapshot -i --tab-id <TAB_ID> # verify body content replaced
|
|
136
|
+
```
|
|
137
|
+
|
|
138
|
+
- `\n\n` splits paragraphs; previous content is cleared.
|
|
139
|
+
- `type @richtext " text"` appends at the cursor (click the ref first to focus).
|
|
140
|
+
- Do NOT write the article body into a `[textbox]` title field. Title and richtext body are separate refs.
|
|
141
|
+
|
|
142
|
+
### Verification
|
|
143
|
+
|
|
144
|
+
Use `get value`, visible validation text, enabled state, or submit readiness:
|
|
145
|
+
|
|
146
|
+
```bash
|
|
147
|
+
OMNIBOT_SESSION_TOKEN=checkout omnibot get value "input[name=email]" --tab-id <TAB_ID>
|
|
148
|
+
OMNIBOT_SESSION_TOKEN=checkout omnibot is enabled "button[type=submit]" --tab-id <TAB_ID>
|
|
149
|
+
```
|
|
150
|
+
|
|
151
|
+
For richtext bodies, re-run `snapshot -i` and confirm the `@eN [richtext]` name matches the intended content.
|
|
152
|
+
|
|
153
|
+
### Cursor positioning and inline image insertion
|
|
154
|
+
|
|
155
|
+
For `contenteditable` article bodies (ProseMirror/Quill/Draft.js/Slate/Toutiao article body), `find text "<anchor phrase>" --action click` collapses the editor selection at the end of the matched phrase and focuses the editor. Use this to place the caret at a specific paragraph before inserting an image, typing, or pressing keys. Do not rely on a bare `click` of the editor followed by `press End`, because the caret would stay on the first line.
|
|
156
|
+
|
|
157
|
+
Insert an image after a specific paragraph:
|
|
158
|
+
|
|
159
|
+
```bash
|
|
160
|
+
OMNIBOT_SESSION_TOKEN=editor omnibot find text "第二段锚点文字" --action click --tab-id <TAB_ID>
|
|
161
|
+
OMNIBOT_SESSION_TOKEN=editor omnibot click ".syl-toolbar-tool.image .syl-toolbar-button" --tab-id <TAB_ID>
|
|
162
|
+
OMNIBOT_SESSION_TOKEN=editor omnibot wait --selector ".upload-image-panel input[type=file]" --tab-id <TAB_ID>
|
|
163
|
+
OMNIBOT_SESSION_TOKEN=editor omnibot upload ".upload-image-panel input[type=file]" "/path/to/image.png" --tab-id <TAB_ID>
|
|
164
|
+
OMNIBOT_SESSION_TOKEN=editor omnibot find role button --name "确定" --action click --tab-id <TAB_ID>
|
|
165
|
+
```
|
|
166
|
+
|
|
167
|
+
Verify the inserted image by counting visible image cards, not raw `<img>` nodes. Toutiao/ProseMirror image cards keep a hidden `<templ>` template and a visible `<mask>` copy of the same `<img>`, so `.ProseMirror img` returns 2 for one logical image. Count the image card container instead (e.g. `.ProseMirror [__syl_tag=true]` or `.ProseMirror .pgc-image`), which reports one per inserted image.
|
|
168
|
+
|
|
169
|
+
### Fallback entry point
|
|
170
|
+
|
|
171
|
+
If fill does not update the application state, use `fallback-operations.md#javascript-fallback` only after selector/focus/type attempts fail. JavaScript must dispatch real `input` and `change` events.
|
|
172
|
+
|
|
173
|
+
## Select / Check
|
|
174
|
+
|
|
175
|
+
### When to use
|
|
176
|
+
|
|
177
|
+
Use this pattern for dropdowns, comboboxes, checkboxes, toggles, agreement boxes, filters, and boolean controls.
|
|
178
|
+
|
|
179
|
+
### Preferred sequence
|
|
180
|
+
|
|
181
|
+
1. Use `snapshot -i` first; visible comboboxes may include auto-probed option refs.
|
|
182
|
+
2. Use `click @option` if `snapshot -i` lists auto-probed combobox option refs.
|
|
183
|
+
3. Use `select @combobox "value"` for native selects or simple select-like controls.
|
|
184
|
+
4. Use `check` / `uncheck` for boolean controls.
|
|
185
|
+
5. Verify selected label, value, URL query, result count, or dependent page state.
|
|
186
|
+
|
|
187
|
+
`snapshot -i` can list options for visible custom comboboxes even though the dropdown closes during observation. These refs are still actionable because Omnibot stores the option's `openerSelector` and can reopen the owning combobox before clicking the option.
|
|
188
|
+
|
|
189
|
+
### Example
|
|
190
|
+
|
|
191
|
+
```bash
|
|
192
|
+
OMNIBOT_SESSION_TOKEN=form omnibot snapshot -i --tab-id <TAB_ID>
|
|
193
|
+
OMNIBOT_SESSION_TOKEN=form omnibot click @e22 --tab-id <TAB_ID> # @e22 [option] "OpenAI Chat"
|
|
194
|
+
OMNIBOT_SESSION_TOKEN=form omnibot get url --tab-id <TAB_ID>
|
|
195
|
+
```
|
|
196
|
+
|
|
197
|
+
Checkbox example:
|
|
198
|
+
|
|
199
|
+
```bash
|
|
200
|
+
OMNIBOT_SESSION_TOKEN=form omnibot is checked "#agree" --tab-id <TAB_ID>
|
|
201
|
+
OMNIBOT_SESSION_TOKEN=form omnibot check "#agree" --tab-id <TAB_ID>
|
|
202
|
+
OMNIBOT_SESSION_TOKEN=form omnibot is checked "#agree" --tab-id <TAB_ID>
|
|
203
|
+
```
|
|
204
|
+
|
|
205
|
+
### Verification
|
|
206
|
+
|
|
207
|
+
For checkboxes, verify boolean state. For selects, verify value or dependent page state.
|
|
208
|
+
|
|
209
|
+
### CSS-only dropdowns and custom menus
|
|
210
|
+
|
|
211
|
+
Some dropdowns, hover menus, tooltips, and custom selects are CSS-only or portal-based widgets. The visible trigger may be accessible, while the option nodes are hidden `<div>` elements without `role`, accessible name, or stable refs. Start with `snapshot -i`; visible comboboxes may include auto-probed option refs before any manual dropdown opening.
|
|
212
|
+
|
|
213
|
+
Escalation order:
|
|
214
|
+
|
|
215
|
+
1. Run `snapshot -i` and look for `[option]` refs near the combobox.
|
|
216
|
+
2. If an option ref exists, click the option ref directly and verify.
|
|
217
|
+
3. If options are absent, click/hover the trigger once, run `snapshot -i` again, then click the option ref if it appears.
|
|
218
|
+
4. If trigger interaction works but options remain absent or option refs fail verification, use `fallback-operations.md#css-only-dropdown-javascript-fallback`.
|
|
219
|
+
|
|
220
|
+
Do not loop on manual open -> snapshot -> open -> snapshot when `snapshot -i` already lists auto-probed combobox options.
|
|
221
|
+
|
|
222
|
+
### Fallback entry point
|
|
223
|
+
|
|
224
|
+
If native select/check fails, use `fallback-operations.md#selector-fallback`. Use JavaScript only if events must be synthesized or if evidence shows a CSS-only multi-step widget cannot be completed through semantic, snapshot, selector, DOM, or mouse tiers.
|
|
225
|
+
|
|
226
|
+
## Navigation
|
|
227
|
+
|
|
228
|
+
### When to use
|
|
229
|
+
|
|
230
|
+
Use Navigation to open pages, create tabs, change history, reload, close tabs, move between pages, create windows, or select frames.
|
|
231
|
+
|
|
232
|
+
### Preferred sequence
|
|
233
|
+
|
|
234
|
+
1. Create or discover a tab with `tabs`, `tab list`, `tab new`, `open`, or `navigate`.
|
|
235
|
+
2. Save the returned tab id.
|
|
236
|
+
3. Verify with `get url --tab-id <TAB_ID>`.
|
|
237
|
+
4. Run all later page-state commands with `--tab-id <TAB_ID>`.
|
|
238
|
+
|
|
239
|
+
### Example
|
|
240
|
+
|
|
241
|
+
```bash
|
|
242
|
+
OMNIBOT_SESSION_TOKEN=research omnibot tab new https://example.com --label research
|
|
243
|
+
OMNIBOT_SESSION_TOKEN=research omnibot get url --tab-id <TAB_ID>
|
|
244
|
+
OMNIBOT_SESSION_TOKEN=research omnibot snapshot -i --tab-id <TAB_ID>
|
|
245
|
+
```
|
|
246
|
+
|
|
247
|
+
History and lifecycle:
|
|
248
|
+
|
|
249
|
+
```bash
|
|
250
|
+
OMNIBOT_SESSION_TOKEN=research omnibot back --tab-id <TAB_ID>
|
|
251
|
+
OMNIBOT_SESSION_TOKEN=research omnibot get url --tab-id <TAB_ID>
|
|
252
|
+
OMNIBOT_SESSION_TOKEN=research omnibot forward --tab-id <TAB_ID>
|
|
253
|
+
OMNIBOT_SESSION_TOKEN=research omnibot get url --tab-id <TAB_ID>
|
|
254
|
+
OMNIBOT_SESSION_TOKEN=research omnibot reload --tab-id <TAB_ID>
|
|
255
|
+
OMNIBOT_SESSION_TOKEN=research omnibot wait --load domcontentloaded --tab-id <TAB_ID>
|
|
256
|
+
```
|
|
257
|
+
|
|
258
|
+
Other navigation commands:
|
|
259
|
+
|
|
260
|
+
```bash
|
|
261
|
+
OMNIBOT_SESSION_TOKEN=research omnibot navigate https://example.com
|
|
262
|
+
OMNIBOT_SESSION_TOKEN=research omnibot navigate https://example.com --same-tab --tab-id <TAB_ID>
|
|
263
|
+
OMNIBOT_SESSION_TOKEN=research omnibot open https://example.com
|
|
264
|
+
OMNIBOT_SESSION_TOKEN=research omnibot goto https://example.com --tab-id <TAB_ID>
|
|
265
|
+
OMNIBOT_SESSION_TOKEN=research omnibot pushstate /dashboard --tab-id <TAB_ID>
|
|
266
|
+
OMNIBOT_SESSION_TOKEN=research omnibot close <TAB_ID>
|
|
267
|
+
OMNIBOT_SESSION_TOKEN=research omnibot tab list
|
|
268
|
+
OMNIBOT_SESSION_TOKEN=research omnibot tab close research
|
|
269
|
+
OMNIBOT_SESSION_TOKEN=research omnibot window new
|
|
270
|
+
OMNIBOT_SESSION_TOKEN=research omnibot frame main
|
|
271
|
+
```
|
|
272
|
+
|
|
273
|
+
### Verification
|
|
274
|
+
|
|
275
|
+
Always verify the final target:
|
|
276
|
+
|
|
277
|
+
```bash
|
|
278
|
+
OMNIBOT_SESSION_TOKEN=research omnibot get url --tab-id <TAB_ID>
|
|
279
|
+
OMNIBOT_SESSION_TOKEN=research omnibot wait --load networkidle --tab-id <TAB_ID>
|
|
280
|
+
OMNIBOT_SESSION_TOKEN=research omnibot snapshot -i --tab-id <TAB_ID>
|
|
281
|
+
```
|
|
282
|
+
|
|
283
|
+
### Fallback entry point
|
|
284
|
+
|
|
285
|
+
Same-tab navigation must include `--tab-id <TAB_ID>`. If the target tab id is unknown, run `omnibot tabs` first and choose the intended tab explicitly.
|
|
286
|
+
|
|
287
|
+
## Wait
|
|
288
|
+
|
|
289
|
+
### When to use
|
|
290
|
+
|
|
291
|
+
Use Wait after navigation, clicks, form submissions, reloads, dynamic rendering, lazy loading, or any action that changes asynchronous state.
|
|
292
|
+
|
|
293
|
+
### Preferred sequence
|
|
294
|
+
|
|
295
|
+
1. Wait for selector visibility or hidden state.
|
|
296
|
+
2. Wait for text.
|
|
297
|
+
3. Wait for URL.
|
|
298
|
+
4. Wait for load state.
|
|
299
|
+
5. Wait for a JavaScript condition with `--fn`.
|
|
300
|
+
6. Avoid shell sleep.
|
|
301
|
+
|
|
302
|
+
### Example
|
|
303
|
+
|
|
304
|
+
```bash
|
|
305
|
+
OMNIBOT_SESSION_TOKEN=checkout omnibot click --tab-id <TAB_ID> @e4
|
|
306
|
+
OMNIBOT_SESSION_TOKEN=checkout omnibot wait "#spinner" --state hidden --tab-id <TAB_ID>
|
|
307
|
+
OMNIBOT_SESSION_TOKEN=checkout omnibot wait --text "Welcome" --tab-id <TAB_ID>
|
|
308
|
+
OMNIBOT_SESSION_TOKEN=checkout omnibot get text "#main" --tab-id <TAB_ID>
|
|
309
|
+
```
|
|
310
|
+
|
|
311
|
+
Other waits:
|
|
312
|
+
|
|
313
|
+
```bash
|
|
314
|
+
OMNIBOT_SESSION_TOKEN=checkout omnibot wait --url "/dashboard" --tab-id <TAB_ID>
|
|
315
|
+
OMNIBOT_SESSION_TOKEN=checkout omnibot wait --load domcontentloaded --tab-id <TAB_ID>
|
|
316
|
+
OMNIBOT_SESSION_TOKEN=checkout omnibot wait --load networkidle --tab-id <TAB_ID>
|
|
317
|
+
OMNIBOT_SESSION_TOKEN=checkout omnibot wait --fn "window.appReady === true" --tab-id <TAB_ID>
|
|
318
|
+
```
|
|
319
|
+
|
|
320
|
+
### Verification
|
|
321
|
+
|
|
322
|
+
After waiting, read the final condition with `get`, `is`, `wait`, or `snapshot`.
|
|
323
|
+
|
|
324
|
+
### Fallback entry point
|
|
325
|
+
|
|
326
|
+
If waits time out, collect evidence with `debugging-and-evidence.md`; do not replace them with shell sleep.
|
|
327
|
+
|
|
328
|
+
## Extraction
|
|
329
|
+
|
|
330
|
+
### When to use
|
|
331
|
+
|
|
332
|
+
Use Extraction for retrieving page text, HTML fragments, links, counts, assets, or clipboard data.
|
|
333
|
+
|
|
334
|
+
### Preferred sequence
|
|
335
|
+
|
|
336
|
+
1. `get text/html/attr/count` for targeted extraction.
|
|
337
|
+
2. `read --screens N` for clean page text, long pages, or lazy-loaded content.
|
|
338
|
+
3. `snapshot -i` for structured page observation before action planning.
|
|
339
|
+
4. `assets list/export` for resources.
|
|
340
|
+
5. `clipboard read` only when clipboard content is part of the task.
|
|
341
|
+
|
|
342
|
+
### Example
|
|
343
|
+
|
|
344
|
+
```bash
|
|
345
|
+
OMNIBOT_SESSION_TOKEN=extract omnibot get text "#article" --tab-id <TAB_ID>
|
|
346
|
+
OMNIBOT_SESSION_TOKEN=extract omnibot get attr "a.next" href --tab-id <TAB_ID>
|
|
347
|
+
OMNIBOT_SESSION_TOKEN=extract omnibot get count ".result" --tab-id <TAB_ID>
|
|
348
|
+
```
|
|
349
|
+
|
|
350
|
+
Long page example:
|
|
351
|
+
|
|
352
|
+
```bash
|
|
353
|
+
OMNIBOT_SESSION_TOKEN=extract omnibot read --screens 5 --tab-id <TAB_ID>
|
|
354
|
+
OMNIBOT_SESSION_TOKEN=extract omnibot assets list --tab-id <TAB_ID>
|
|
355
|
+
```
|
|
356
|
+
|
|
357
|
+
### Verification
|
|
358
|
+
|
|
359
|
+
Verify extraction completeness with expected markers in `read`, element counts, or targeted reads for known selectors.
|
|
360
|
+
|
|
361
|
+
### Fallback entry point
|
|
362
|
+
|
|
363
|
+
If content exists only in runtime objects, use `fallback-operations.md#javascript-fallback` after explaining why `read`, `snapshot`, and `get` could not access it.
|
|
364
|
+
|
|
365
|
+
## Upload
|
|
366
|
+
|
|
367
|
+
### When to use
|
|
368
|
+
|
|
369
|
+
Use `omnibot upload` to attach a local file to an `<input type="file">` element. The command speaks CDP (`DOM.setFileInputFiles`) through the extension debugger transport, with a JS `DataTransfer` fallback for pages that reject programmatic file assignment.
|
|
370
|
+
|
|
371
|
+
### Preferred sequence
|
|
372
|
+
|
|
373
|
+
1. Observe the upload UI and confirm the task requires a local file.
|
|
374
|
+
2. Locate the file input via `snapshot -i` or a CSS selector.
|
|
375
|
+
3. Run `omnibot upload <selector> <local-file> --tab-id <TAB_ID>`.
|
|
376
|
+
4. Verify the file was accepted (filename, count, enabled submit state, or application-specific confirmation).
|
|
377
|
+
|
|
378
|
+
### Example
|
|
379
|
+
|
|
380
|
+
```bash
|
|
381
|
+
OMNIBOT_SESSION_TOKEN=form omnibot snapshot -i --tab-id <TAB_ID>
|
|
382
|
+
OMNIBOT_SESSION_TOKEN=form omnibot upload "input[type=file]" /path/to/cover.png --tab-id <TAB_ID>
|
|
383
|
+
OMNIBOT_SESSION_TOKEN=form omnibot is enabled "button[type=submit]" --tab-id <TAB_ID>
|
|
384
|
+
```
|
|
385
|
+
|
|
386
|
+
### Verification
|
|
387
|
+
|
|
388
|
+
Confirm the input accepted the file:
|
|
389
|
+
|
|
390
|
+
```bash
|
|
391
|
+
OMNIBOT_SESSION_TOKEN=form omnibot get "input[type=file]" attr value --tab-id <TAB_ID>
|
|
392
|
+
OMNIBOT_SESSION_TOKEN=form omnibot is enabled "button[type=submit]" --tab-id <TAB_ID>
|
|
393
|
+
```
|
|
394
|
+
|
|
395
|
+
### Fallback entry point
|
|
396
|
+
|
|
397
|
+
The CLI already falls back from CDP `DOM.setFileInputFiles` (by objectId, then nodeId) to a JS `DataTransfer` assignment. Do not add further JavaScript fallbacks on top; file inputs are security-sensitive and page-specific. If all three transports fail, report the failure to the user instead of scripting around it.
|
|
398
|
+
|
|
399
|
+
## Batch
|
|
400
|
+
|
|
401
|
+
### When to use
|
|
402
|
+
|
|
403
|
+
Use Batch for short, known-safe extension/CDP command chains where the target tab and verification are already explicit. Batch is not a wrapper for normal CLI commands such as `snapshot` or `click`.
|
|
404
|
+
|
|
405
|
+
### Preferred sequence
|
|
406
|
+
|
|
407
|
+
1. Keep batches short.
|
|
408
|
+
2. Include only deterministic operations.
|
|
409
|
+
3. Verify after the batch with a separate command.
|
|
410
|
+
4. Use `--file` for complex JSON quoting.
|
|
411
|
+
|
|
412
|
+
### Example
|
|
413
|
+
|
|
414
|
+
```bash
|
|
415
|
+
OMNIBOT_SESSION_TOKEN=research omnibot batch '[{"cmd":"cdp","method":"Runtime.evaluate","params":{"expression":"document.title","returnByValue":true}}]' --tab-id <TAB_ID>
|
|
416
|
+
OMNIBOT_SESSION_TOKEN=research omnibot get title --tab-id <TAB_ID>
|
|
417
|
+
```
|
|
418
|
+
|
|
419
|
+
File example:
|
|
420
|
+
|
|
421
|
+
```bash
|
|
422
|
+
OMNIBOT_SESSION_TOKEN=research omnibot batch --file /tmp/omnibot-batch.json --tab-id <TAB_ID>
|
|
423
|
+
OMNIBOT_SESSION_TOKEN=research omnibot get url --tab-id <TAB_ID>
|
|
424
|
+
```
|
|
425
|
+
|
|
426
|
+
### Verification
|
|
427
|
+
|
|
428
|
+
Do not treat batch success as task success. Verify final page state separately.
|
|
429
|
+
|
|
430
|
+
### Fallback entry point
|
|
431
|
+
|
|
432
|
+
Batch is not a fallback tier. If a batched operation fails, re-run the workflow step-by-step and enter the normal fallback model.
|