@omniaibot/win-x64 1.6.4 → 1.6.5

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (38) hide show
  1. package/bin/omnibot-windows-x64/81d243bd2c585b0f4821__mypyc.pyd +0 -0
  2. package/bin/omnibot-windows-x64/VERSION +1 -1
  3. package/bin/omnibot-windows-x64/_asyncio.pyd +0 -0
  4. package/bin/omnibot-windows-x64/_bz2.pyd +0 -0
  5. package/bin/omnibot-windows-x64/_cffi_backend.pyd +0 -0
  6. package/bin/omnibot-windows-x64/_ctypes.pyd +0 -0
  7. package/bin/omnibot-windows-x64/_decimal.pyd +0 -0
  8. package/bin/omnibot-windows-x64/_hashlib.pyd +0 -0
  9. package/bin/omnibot-windows-x64/_lzma.pyd +0 -0
  10. package/bin/omnibot-windows-x64/_multiprocessing.pyd +0 -0
  11. package/bin/omnibot-windows-x64/_overlapped.pyd +0 -0
  12. package/bin/omnibot-windows-x64/_queue.pyd +0 -0
  13. package/bin/omnibot-windows-x64/_socket.pyd +0 -0
  14. package/bin/omnibot-windows-x64/_ssl.pyd +0 -0
  15. package/bin/omnibot-windows-x64/_uuid.pyd +0 -0
  16. package/bin/omnibot-windows-x64/_wmi.pyd +0 -0
  17. package/bin/omnibot-windows-x64/charset_normalizer/cd.pyd +0 -0
  18. package/bin/omnibot-windows-x64/charset_normalizer/md.pyd +0 -0
  19. package/bin/omnibot-windows-x64/libcrypto-3-x64.dll +0 -0
  20. package/bin/omnibot-windows-x64/libffi-8.dll +0 -0
  21. package/bin/omnibot-windows-x64/libssl-3-x64.dll +0 -0
  22. package/bin/omnibot-windows-x64/omnibot/skills/omnibot/SKILL.md +12 -0
  23. package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/anti-patterns.md +8 -0
  24. package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/command-reference.md +17 -3
  25. package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/fallback-operations.md +2 -0
  26. package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/operation-patterns.md +43 -7
  27. package/bin/omnibot-windows-x64/omnibot-windows-x64.exe +0 -0
  28. package/bin/omnibot-windows-x64/pyexpat.pyd +0 -0
  29. package/bin/omnibot-windows-x64/python3.dll +0 -0
  30. package/bin/omnibot-windows-x64/python313.dll +0 -0
  31. package/bin/omnibot-windows-x64/select.pyd +0 -0
  32. package/bin/omnibot-windows-x64/unicodedata.pyd +0 -0
  33. package/package.json +1 -1
  34. package/bin/omnibot-win-x64/VERSION +0 -1
  35. package/bin/omnibot-win-x64/omnibot/skills/omnibot/SKILL.md +0 -261
  36. package/bin/omnibot-win-x64/omnibot/skills/omnibot/references/command-reference.md +0 -755
  37. package/bin/omnibot-win-x64/omnibot-bin.exe +0 -0
  38. package/bin/omnibot-windows-x64/python312.dll +0 -0
@@ -1 +1 @@
1
- 1.6.3
1
+ 1.6.5
Binary file
Binary file
Binary file
Binary file
Binary file
Binary file
Binary file
Binary file
@@ -140,6 +140,18 @@ Do not use `read` as the first step for click/fill workflows unless the user exp
140
140
 
141
141
  Use `select @combobox "value"` for native selects or simple controls. Use `click @option` when `snapshot -i` exposes auto-probed combobox option refs. Do not manually reopen every dropdown just to discover options unless `snapshot -i` did not expose option refs.
142
142
 
143
+ ## Rich Text Article Editors
144
+
145
+ ContentEditable rich text editors (ProseMirror, Quill, Draft.js, Slate, ByteDance/Toutiao article body, etc.) are often missing from the AX tree. `snapshot -i` appends them under `# DOM Rich Text Editors` as `@eN [richtext] "..." [contenteditable=true]` refs.
146
+
147
+ - Detect the body editor: `snapshot -i` → look for `@eN [richtext]` under `# DOM Rich Text Editors`. Do not fill the article body through a `[textbox]` title ref.
148
+ - Replace the whole body: `fill @richtext "paragraph 1\n\nparagraph 2"`. Empty lines split paragraphs; the previous content is cleared and `input`/`change` events are dispatched.
149
+ - Append to the body: `type @richtext " additional text"`.
150
+ - Focus before keyboard input: click `@richtext` first, then `press`/`keyboard`.
151
+ - Raw selector fallback: `omnibot fill '[contenteditable="true"]' "..."` also routes through the rich text path when the selector looks like a rich editor.
152
+
153
+ Do not use `execute-js` first for rich text writes; `fill`/`type` on the `[richtext]` ref is the supported path.
154
+
143
155
  ## Quick Routing
144
156
 
145
157
  | Need | Read |
@@ -28,6 +28,10 @@
28
28
  | Revealing sensitive network data | Cookies, auth headers, full addresses, phone numbers, and payment data must not be pasted into reports. | Report host, path, method, status, and non-sensitive payload shape only. |
29
29
  | Using stale `@eN` refs after a failed click | Refs may have expired or the page may have changed. | Re-observe with `snapshot -i` or use a higher-evidence fallback. |
30
30
  | `Network.enable` without log retrieval | Raw `cdp` calls are one-shot and do not persist events for later reading. | Use `network clear -> network start -> one action -> network stop -> network logs/summary`. |
31
+ | Writing the article body into the title field | Rich text article body editors are often missing from the AX tree; filling the only visible `[textbox]` (title) overwrites the wrong control. | Run `snapshot -i`, find the body under `# DOM Rich Text Editors` as `@eN [richtext]`, then `fill @richtext "..."`. |
32
+ | `execute-js` for rich text body before `fill @richtext` | ContentEditable writes need event dispatch and framework integration; ad-hoc `innerHTML` bypasses Omnibot's supported path. | Use `fill @richtext "p1\n\np2"` or `type @richtext " text"` first. |
33
+ | Counting `<img>` nodes to detect duplicate images in a rich editor | ProseMirror/Toutiao image cards keep a hidden `<templ>` template and a visible `<mask>` copy of the same `<img>`, so `.ProseMirror img` returns 2 for one logical image and falsely reports duplication. | Count image card containers (`.ProseMirror [__syl_tag=true]` or `.ProseMirror .pgc-image`), one per inserted image. |
34
+ | Inserting an image via `click @richtext` + `press End` | The caret stays on the first line; the image lands at the editor root instead of the target paragraph. | Use `find text "<anchor phrase>" --action click` to collapse the selection at the end of the matched phrase, then open the image tool and upload. |
31
35
 
32
36
  Do not manually reopen a dropdown just to discover options if `snapshot -i` already lists them.
33
37
 
@@ -45,5 +49,9 @@ Do not manually reopen a dropdown just to discover options if `snapshot -i` alre
45
49
  - "I remember the command syntax from before."
46
50
  - "`Network.enable` should be enough to capture requests."
47
51
  - "It is just a checkout preview, clicking pay should be fine."
52
+ - "The article body is not in `snapshot -i`, so I will fill the title textbox."
53
+ - "I will just set `innerHTML` on the contenteditable with `execute-js`."
54
+ - "`.ProseMirror img` returned 2, so the image was inserted twice."
55
+ - "I clicked the editor and pressed End, so the caret must be after the last paragraph."
48
56
 
49
57
  All of these mean: stop, re-enter the standard pattern, and make state explicit.
@@ -174,6 +174,8 @@ Fallback relation: Tier 2 with refs, Tier 3 with selector.
174
174
 
175
175
  Combobox option refs: if `snapshot -i` shows `@eN [option] "OpenAI Chat"`, prefer `click @eN` over JavaScript fallback. Omnibot will reopen the owning combobox when needed.
176
176
 
177
+ Rich text editor refs: if `snapshot -i` shows `@eN [richtext] "..." [contenteditable=true]` under `# DOM Rich Text Editors`, treat it as the article body editor. Use `fill @richtext` to replace the whole body (supports `\n\n` for paragraphs) and `type @richtext` to append. Do not fall back to `execute-js` first.
178
+
177
179
  ### dblclick
178
180
 
179
181
  Purpose: Double-click an element by `@eN` ref or selector.
@@ -197,6 +199,14 @@ OMNIBOT_SESSION_TOKEN=checkout omnibot fill "input[name=email]" "a@b.com" --tab-
197
199
  Preferred pattern: See `operation-patterns.md#fill`.
198
200
  Fallback relation: Tier 2 with refs, Tier 3 with selector.
199
201
 
202
+ Rich text body editors (ProseMirror/Quill/Draft.js/Slate/Toutiao article body): if the target is an `@eN [richtext]` ref, `fill` writes through a dedicated contenteditable path that clears the previous body and dispatches `input`/`change` events. Split paragraphs with an empty line:
203
+
204
+ ```bash
205
+ OMNIBOT_SESSION_TOKEN=editor omnibot fill --tab-id <TAB_ID> @eN "First paragraph.\n\nSecond paragraph."
206
+ ```
207
+
208
+ Raw selector fallback is supported for selectors that look like a rich editor (e.g. `[contenteditable="true"]`, `.ProseMirror`, `.ql-editor`).
209
+
200
210
  ### type
201
211
 
202
212
  Purpose: Type text into an element.
@@ -208,6 +218,8 @@ OMNIBOT_SESSION_TOKEN=search omnibot type --tab-id <TAB_ID> @e3 "omnibot"
208
218
  Preferred pattern: See `operation-patterns.md#fill`.
209
219
  Fallback relation: Use after fill is not appropriate or focus is needed.
210
220
 
221
+ For `@eN [richtext]` article body editors, `type` appends at the cursor position. Click the `@richtext` ref first to focus the editor.
222
+
211
223
  ### press
212
224
 
213
225
  Purpose: Press a key in the target tab.
@@ -290,14 +302,14 @@ Fallback relation: Mouse drag is Tier 5 if element drag fails.
290
302
 
291
303
  ### upload
292
304
 
293
- Purpose: Reserved command for local file upload. Native upload is currently unavailable in the extension transport.
305
+ Purpose: Attach a local file to an `<input type="file">` element via CDP (`DOM.setFileInputFiles`), with nodeId and JS `DataTransfer` fallbacks.
294
306
 
295
307
  ```bash
296
- # Do not use in agent workflows until file chooser support is implemented.
308
+ OMNIBOT_SESSION_TOKEN=form omnibot upload "input[type=file]" /path/to/file.png --tab-id <TAB_ID>
297
309
  ```
298
310
 
299
311
  Preferred pattern: See `operation-patterns.md#upload`.
300
- Fallback relation: Do not treat JavaScript as a normal upload fallback.
312
+ Fallback relation: The CLI falls back from CDP objectId CDP nodeId → JS DataTransfer. Do not stack additional JavaScript fallbacks.
301
313
 
302
314
  ### keyboard / keydown / keyup
303
315
 
@@ -326,6 +338,8 @@ OMNIBOT_SESSION_TOKEN=checkout omnibot find testid submit-button --action click
326
338
  OMNIBOT_SESSION_TOKEN=checkout omnibot find nth ".card" 2 --action click --tab-id <TAB_ID>
327
339
  ```
328
340
 
341
+ Text-node precision: `find text` walks the DOM text nodes and marks the nearest visible container of the matching text, not the first ancestor whose `textContent` happens to include the needle. The matched offset/length is stored on the element, so `--action click` positions the mouse at the end of the matched text. Inside a `contenteditable` body (ProseMirror/Quill/etc.), the selection is collapsed right after the matched text and the editor is focused, so a follow-up image insertion lands at the intended paragraph instead of the editor root.
342
+
329
343
  Preferred pattern: See `operation-patterns.md#click` and `operation-patterns.md#fill`.
330
344
  Fallback relation: Tier 1. Explain failure before using refs, selectors, DOM, mouse, JavaScript, or CDP.
331
345
 
@@ -109,6 +109,8 @@ Use `execute-js` only when:
109
109
  - A CSS-only dropdown or hover menu requires an atomic open-then-select sequence, and `snapshot -i` confirms the option nodes are absent from the accessibility tree.
110
110
  - Runtime state cannot be read through `read`, `snapshot`, `get`, or `is`.
111
111
 
112
+ Rich text article body editors (`@eN [richtext]`, `.ProseMirror`, `.ql-editor`, `[contenteditable="true"]`) have native `fill`/`type` support. Do not fall back to JavaScript for richtext writes; use `fill @richtext "..."` first.
113
+
112
114
  Read-only JavaScript fallback:
113
115
 
114
116
  ```bash
@@ -99,7 +99,7 @@ When `snapshot -i` shows only `StaticText` for a visible control label, do not a
99
99
 
100
100
  ### When to use
101
101
 
102
- Use Fill for text inputs, textareas, search boxes, login fields, and form fields where the final value must be controlled.
102
+ Use Fill for text inputs, textareas, search boxes, login fields, form fields where the final value must be controlled, and rich text article body editors (contenteditable ProseMirror/Quill/Draft.js/Slate/Toutiao article body).
103
103
 
104
104
  ### Preferred sequence
105
105
 
@@ -124,6 +124,21 @@ OMNIBOT_SESSION_TOKEN=checkout omnibot fill --tab-id <TAB_ID> @e2 "a@b.com"
124
124
  OMNIBOT_SESSION_TOKEN=checkout omnibot get value "input[name=email]" --tab-id <TAB_ID>
125
125
  ```
126
126
 
127
+ ### Rich text article body
128
+
129
+ For article body editors exposed as `@eN [richtext] "..." [contenteditable=true]` under `# DOM Rich Text Editors`:
130
+
131
+ ```bash
132
+ OMNIBOT_SESSION_TOKEN=editor omnibot snapshot -i --tab-id <TAB_ID>
133
+ # Body appears under "# DOM Rich Text Editors": @eN [richtext] "..."
134
+ OMNIBOT_SESSION_TOKEN=editor omnibot fill --tab-id <TAB_ID> @eN "First paragraph.\n\nSecond paragraph."
135
+ OMNIBOT_SESSION_TOKEN=editor omnibot snapshot -i --tab-id <TAB_ID> # verify body content replaced
136
+ ```
137
+
138
+ - `\n\n` splits paragraphs; previous content is cleared.
139
+ - `type @richtext " text"` appends at the cursor (click the ref first to focus).
140
+ - Do NOT write the article body into a `[textbox]` title field. Title and richtext body are separate refs.
141
+
127
142
  ### Verification
128
143
 
129
144
  Use `get value`, visible validation text, enabled state, or submit readiness:
@@ -133,6 +148,24 @@ OMNIBOT_SESSION_TOKEN=checkout omnibot get value "input[name=email]" --tab-id <T
133
148
  OMNIBOT_SESSION_TOKEN=checkout omnibot is enabled "button[type=submit]" --tab-id <TAB_ID>
134
149
  ```
135
150
 
151
+ For richtext bodies, re-run `snapshot -i` and confirm the `@eN [richtext]` name matches the intended content.
152
+
153
+ ### Cursor positioning and inline image insertion
154
+
155
+ For `contenteditable` article bodies (ProseMirror/Quill/Draft.js/Slate/Toutiao article body), `find text "<anchor phrase>" --action click` collapses the editor selection at the end of the matched phrase and focuses the editor. Use this to place the caret at a specific paragraph before inserting an image, typing, or pressing keys. Do not rely on a bare `click` of the editor followed by `press End`, because the caret would stay on the first line.
156
+
157
+ Insert an image after a specific paragraph:
158
+
159
+ ```bash
160
+ OMNIBOT_SESSION_TOKEN=editor omnibot find text "第二段锚点文字" --action click --tab-id <TAB_ID>
161
+ OMNIBOT_SESSION_TOKEN=editor omnibot click ".syl-toolbar-tool.image .syl-toolbar-button" --tab-id <TAB_ID>
162
+ OMNIBOT_SESSION_TOKEN=editor omnibot wait --selector ".upload-image-panel input[type=file]" --tab-id <TAB_ID>
163
+ OMNIBOT_SESSION_TOKEN=editor omnibot upload ".upload-image-panel input[type=file]" "/path/to/image.png" --tab-id <TAB_ID>
164
+ OMNIBOT_SESSION_TOKEN=editor omnibot find role button --name "确定" --action click --tab-id <TAB_ID>
165
+ ```
166
+
167
+ Verify the inserted image by counting visible image cards, not raw `<img>` nodes. Toutiao/ProseMirror image cards keep a hidden `<templ>` template and a visible `<mask>` copy of the same `<img>`, so `.ProseMirror img` returns 2 for one logical image. Count the image card container instead (e.g. `.ProseMirror [__syl_tag=true]` or `.ProseMirror .pgc-image`), which reports one per inserted image.
168
+
136
169
  ### Fallback entry point
137
170
 
138
171
  If fill does not update the application state, use `fallback-operations.md#javascript-fallback` only after selector/focus/type attempts fail. JavaScript must dispatch real `input` and `change` events.
@@ -333,32 +366,35 @@ If content exists only in runtime objects, use `fallback-operations.md#javascrip
333
366
 
334
367
  ### When to use
335
368
 
336
- Native upload is currently unavailable in the extension transport. Do not use `omnibot upload` as an agent workflow step until file chooser support is implemented.
369
+ Use `omnibot upload` to attach a local file to an `<input type="file">` element. The command speaks CDP (`DOM.setFileInputFiles`) through the extension debugger transport, with a JS `DataTransfer` fallback for pages that reject programmatic file assignment.
337
370
 
338
371
  ### Preferred sequence
339
372
 
340
373
  1. Observe the upload UI and confirm the task requires a local file.
341
- 2. Report that native upload is currently unavailable.
342
- 3. If the user still wants a workaround, ask for approval before using page-specific JavaScript or manual browser interaction.
374
+ 2. Locate the file input via `snapshot -i` or a CSS selector.
375
+ 3. Run `omnibot upload <selector> <local-file> --tab-id <TAB_ID>`.
376
+ 4. Verify the file was accepted (filename, count, enabled submit state, or application-specific confirmation).
343
377
 
344
378
  ### Example
345
379
 
346
380
  ```bash
347
381
  OMNIBOT_SESSION_TOKEN=form omnibot snapshot -i --tab-id <TAB_ID>
348
- # upload is currently unavailable; do not run omnibot upload here.
382
+ OMNIBOT_SESSION_TOKEN=form omnibot upload "input[type=file]" /path/to/cover.png --tab-id <TAB_ID>
383
+ OMNIBOT_SESSION_TOKEN=form omnibot is enabled "button[type=submit]" --tab-id <TAB_ID>
349
384
  ```
350
385
 
351
386
  ### Verification
352
387
 
353
- If upload support is implemented later, verify with visible filename, count, enabled submit state, or application-specific confirmation:
388
+ Confirm the input accepted the file:
354
389
 
355
390
  ```bash
391
+ OMNIBOT_SESSION_TOKEN=form omnibot get "input[type=file]" attr value --tab-id <TAB_ID>
356
392
  OMNIBOT_SESSION_TOKEN=form omnibot is enabled "button[type=submit]" --tab-id <TAB_ID>
357
393
  ```
358
394
 
359
395
  ### Fallback entry point
360
396
 
361
- Do not treat JavaScript as a normal upload fallback. File inputs and browser file chooser behavior are security-sensitive and page-specific.
397
+ The CLI already falls back from CDP `DOM.setFileInputFiles` (by objectId, then nodeId) to a JS `DataTransfer` assignment. Do not add further JavaScript fallbacks on top; file inputs are security-sensitive and page-specific. If all three transports fail, report the failure to the user instead of scripting around it.
362
398
 
363
399
  ## Batch
364
400
 
Binary file
Binary file
Binary file
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@omniaibot/win-x64",
3
- "version": "1.6.4",
3
+ "version": "1.6.5",
4
4
  "description": "omnibot executable for Windows x64",
5
5
  "os": [
6
6
  "win32"
@@ -1 +0,0 @@
1
- 1.6.4
@@ -1,261 +0,0 @@
1
- ---
2
- name: omnibot
3
- description: Use when AI agents need to read, inspect, operate, navigate, debug, or verify browser state through the omnibot CLI and connected Chromium extension.
4
- compatibility: Requires omnibot v2 CLI daemon and the omnibot Chromium extension connected to 127.0.0.1:18765.
5
- allowed-tools: Bash
6
- metadata:
7
- author: omnibot
8
- version: "2.4.0"
9
- ---
10
-
11
- # Omnibot
12
-
13
- Omnibot is Browser Infrastructure for AI Agents.
14
-
15
- It connects Hermes, Claude Code, Codex, OpenCode, and other agent systems to a real Chromium browser through the local omnibot daemon and CLI.
16
-
17
- This skill is an execution specification for agents. It is not a human command manual.
18
-
19
- ## When to Use
20
-
21
- - Use Omnibot when browser runtime state matters: login, cookies, storage, client-side rendering, extensions, or visible user tabs.
22
- - Use Omnibot when an agent must read rendered pages, click controls, fill forms, navigate, extract content, or collect evidence.
23
- - Do not use Omnibot when static files or plain HTTP content already answer the task.
24
-
25
- ## Core Rules
26
-
27
- - Reliability > Convenience.
28
- - Explicit State > Implicit State.
29
- - Pattern > Command.
30
- - Observe -> Act -> Verify.
31
- - Fallback is allowed, but never first.
32
- - Do not use `execute-js` first.
33
- - Run `omnibot <command> --help` before using uncommon or newly introduced commands, especially `batch`, `network`, `cdp`, `record`, `trace`, and `skills`.
34
- - If a command from memory fails with parser errors, stop using that remembered syntax and re-check `omnibot --help` plus the subcommand help.
35
- - Prefer semantic locators and `snapshot -i` refs before selectors.
36
- - Raw `cdp` calls are one-shot. Do not assume `Network.enable` alone creates readable logs. For API/request evidence, use `network clear -> network start -> one action -> network stop -> network logs/summary`.
37
- - After a failed click by ref, re-observe or use a higher-evidence fallback. Do not keep using stale `@eN` refs from a previous snapshot.
38
- - For shopping, checkout, payment, banking, or irreversible workflows: Do not click final submit/pay/place-order controls unless the user explicitly confirms that final action in the current turn.
39
- - Prefer condition-based `wait` over shell sleep.
40
- - Parse JSON output from commands that return JSON.
41
- - Do not claim success without verification evidence.
42
-
43
- ## Workflow Context + Tab Target
44
-
45
- Every page-state workflow must use both controls:
46
-
47
- - `OMNIBOT_SESSION_TOKEN=<workflow-name>` for workflow isolation.
48
- - `--tab-id <TAB_ID>` for page targeting.
49
-
50
- OMNIBOT_SESSION_TOKEN is a workflow/context token, not a browser session and not a tab target. It isolates agent state such as refs, trace, recording, aliases, and temporary resources. Tab ID controls which page receives the command.
51
-
52
- Do not rely on default tab, active tab, current tab, or prior targeting state. Every page-state command requires explicit `--tab-id` except when the command is only discovering or creating a tab.
53
-
54
- Use the same token inside one workflow. Use different tokens for independent workflows.
55
-
56
- Standard form:
57
-
58
- ```bash
59
- OMNIBOT_SESSION_TOKEN=research omnibot snapshot -i --tab-id <TAB_ID>
60
- OMNIBOT_SESSION_TOKEN=research omnibot click --tab-id <TAB_ID> @e4
61
- OMNIBOT_SESSION_TOKEN=research omnibot snapshot -i --tab-id <TAB_ID>
62
- ```
63
-
64
- `@eN` refs are tab-scoped. Refs rule: never reuse `@eN` refs across tabs.
65
-
66
- The agent dispatch pattern is token + tab-id:
67
-
68
- 1. Set a stable `OMNIBOT_SESSION_TOKEN`.
69
- 2. Discover or create the target tab.
70
- 3. Save the returned tab id.
71
- 4. Pass `--tab-id <TAB_ID>` on every command that touches page state.
72
- 5. Run observe -> act -> verify on that same tab.
73
-
74
- Examples of tab-locked page-state commands:
75
-
76
- ```bash
77
- OMNIBOT_SESSION_TOKEN=checkout omnibot get value "input[name=email]" --tab-id <TAB_ID>
78
- OMNIBOT_SESSION_TOKEN=checkout omnibot find placeholder "Search" --action type --action-value "omnibot" --tab-id <TAB_ID>
79
- OMNIBOT_SESSION_TOKEN=checkout omnibot dom dblclick n1 --tab-id <TAB_ID>
80
- OMNIBOT_SESSION_TOKEN=checkout omnibot clipboard read --tab-id <TAB_ID>
81
- ```
82
-
83
- Targeting categories:
84
-
85
- | Command category | Rule |
86
- | --- | --- |
87
- | Page-state commands: `snapshot`, `click`, `fill`, `type`, `get`, `is`, `wait`, `screenshot`, `console`, `network`, `execute-js`, `dom`, `mouse`, `clipboard`, `viewport`, `assets`, `goto`, same-tab `navigate` | Require `--tab-id <TAB_ID>`. |
88
- | Discovery or tab creation: `tabs`, `tab list`, `tab new <URL>`, `open <URL>`, `navigate <URL>` | Return or discover the tab id; save it for later page-state commands. |
89
- | Read URL exception: `read <URL>` | Opens a temporary read tab and does not need `--tab-id`; reading an existing tab still requires `read --tab-id <TAB_ID>`. |
90
-
91
- ## Observe -> Act -> Verify
92
-
93
- All page operations must follow this loop:
94
-
95
- 1. Observe the current page state.
96
- 2. Act once using the highest reliable pattern available.
97
- 3. Verify the expected state change.
98
-
99
- Click example:
100
-
101
- ```bash
102
- OMNIBOT_SESSION_TOKEN=research omnibot snapshot -i --tab-id <TAB_ID>
103
- OMNIBOT_SESSION_TOKEN=research omnibot click --tab-id <TAB_ID> @e4
104
- OMNIBOT_SESSION_TOKEN=research omnibot snapshot -i --tab-id <TAB_ID>
105
- ```
106
-
107
- Read/verify example:
108
-
109
- ```bash
110
- OMNIBOT_SESSION_TOKEN=checkout omnibot is enabled "button[type=submit]" --tab-id <TAB_ID>
111
- OMNIBOT_SESSION_TOKEN=checkout omnibot find role button --name "Submit" --action click --tab-id <TAB_ID>
112
- OMNIBOT_SESSION_TOKEN=checkout omnibot wait --url "/dashboard" --tab-id <TAB_ID>
113
- OMNIBOT_SESSION_TOKEN=checkout omnibot get url --tab-id <TAB_ID>
114
- ```
115
-
116
- If verification fails, do not repeat blindly. Re-observe, choose the next fallback tier, and verify again.
117
-
118
- ## Pattern > Command
119
-
120
- Start from the task, not the command name. Use operation patterns for read, click, fill, select/check, navigation, wait, extraction, and batch.
121
-
122
- Use fallback tiers only after standard patterns fail. Collect evidence separately from fallback execution. The command reference is only a lookup table; it must not decide behavior.
123
-
124
- ## Read vs Snapshot Routing
125
-
126
- Choose by intent before choosing by command name:
127
-
128
- | Intent | Prefer | Why |
129
- | --- | --- | --- |
130
- | Summarize or extract page content, article text, search results, feeds, or long/lazy pages | `read` | Returns clean text/Markdown for agent reasoning. |
131
- | Quickly observe current visible UI structure before deciding what to do next | `snapshot -i` | Returns actionable refs and page structure for routing. |
132
- | Find buttons, links, inputs, or refs for a later action | `find` or `snapshot -i` | Produces actionable targets; `read` does not. |
133
- | Verify one concrete condition | `get`, `is`, or `wait` | Narrow evidence is more reliable than dumping a page. |
134
-
135
- Do not use `read` as the first step for click/fill workflows unless the user explicitly asked for page content first.
136
-
137
- ## Dropdowns And Custom Comboboxes
138
-
139
- `snapshot -i` may auto-probe visible custom comboboxes and append their options as refs. If the output lists `@eN [option] "..."`, click that option ref directly; Omnibot will reopen the owning combobox through `openerSelector` if the dropdown closed during snapshot.
140
-
141
- Use `select @combobox "value"` for native selects or simple controls. Use `click @option` when `snapshot -i` exposes auto-probed combobox option refs. Do not manually reopen every dropdown just to discover options unless `snapshot -i` did not expose option refs.
142
-
143
- ## Rich Text Article Editors
144
-
145
- ContentEditable rich text editors (ProseMirror, Quill, Draft.js, Slate, ByteDance/Toutiao article body, etc.) are often missing from the AX tree. `snapshot -i` appends them under `# DOM Rich Text Editors` as `@eN [richtext] "..." [contenteditable=true]` refs.
146
-
147
- - Detect the body editor: `snapshot -i` → look for `@eN [richtext]` under `# DOM Rich Text Editors`. Do not fill the article body through a `[textbox]` title ref.
148
- - Replace the whole body: `fill @richtext "paragraph 1\n\nparagraph 2"`. Empty lines split paragraphs; the previous content is cleared and `input`/`change` events are dispatched.
149
- - Append to the body: `type @richtext " additional text"`.
150
- - Focus before keyboard input: click `@richtext` first, then `press`/`keyboard`.
151
- - Raw selector fallback: `omnibot fill '[contenteditable="true"]' "..."` also routes through the rich text path when the selector looks like a rich editor.
152
-
153
- Do not use `execute-js` first for rich text writes; `fill`/`type` on the `[richtext]` ref is the supported path.
154
-
155
- ## Quick Routing
156
-
157
- | Need | Read |
158
- | --- | --- |
159
- | Runtime check | `references/runtime-and-status.md` |
160
- | Reading pages | `references/operation-patterns.md#read` |
161
- | Clicking | `references/operation-patterns.md#click` |
162
- | Filling forms | `references/operation-patterns.md#fill` |
163
- | Select / check | `references/operation-patterns.md#select--check` |
164
- | Navigation | `references/operation-patterns.md#navigation` |
165
- | Waiting | `references/operation-patterns.md#wait` |
166
- | Extraction | `references/operation-patterns.md#extraction` |
167
- | Batch | `references/operation-patterns.md#batch` |
168
- | Fallback | `references/fallback-operations.md` |
169
- | Tabs and sessions | `references/session-and-tabs.md` |
170
- | Debug evidence | `references/debugging-and-evidence.md` |
171
- | Network/API capture | `references/debugging-and-evidence.md#network-capture` |
172
- | Commands | `references/command-reference.md` |
173
- | Anti-patterns | `references/anti-patterns.md` |
174
-
175
- ## Runtime First
176
-
177
- Before troubleshooting browser behavior, check runtime state:
178
-
179
- ```bash
180
- omnibot doctor
181
- omnibot tabs
182
- omnibot visibility status
183
- omnibot browser current
184
- omnibot license status
185
- ```
186
-
187
- If the extension is not connected, open Chrome or Edge with the omnibot extension loaded and keep an HTTP/HTTPS tab open.
188
-
189
- ## Site Memory Reuse
190
-
191
- Before operating a complex site, check whether a generated site skill is installed for the current domain.
192
-
193
- 1. Run `omnibot skills list-sites` to see available generated site skills.
194
- 2. If a matching site skill is installed (e.g., `omnibot-xhs`), read its `SKILL.md` and use it for the operation.
195
- 3. If no matching site skill is installed, create one with:
196
- ```bash
197
- omnibot explore run <url> --name <site> --goal "<explicit goal>"
198
- omnibot skills install --agent opencode --site <site>
199
- ```
200
- 4. Generated site skills are not auto-loaded from `~/.omnibot/skills/`; they must be installed into the current agent's skill directory.
201
-
202
- ## Generated Site Skill Installation
203
-
204
- Generated site skills are stored under `~/.omnibot/skills/omnibot-<name>/` but must be installed into each agent's native skill path to be auto-loaded.
205
-
206
- List available generated site skills:
207
-
208
- ```bash
209
- omnibot skills list-sites
210
- ```
211
-
212
- Install one generated site skill:
213
-
214
- ```bash
215
- omnibot skills install --agent opencode --site xhs
216
- omnibot skills install --agent claude --site xhs
217
- omnibot skills install --agent codex --site xhs
218
- ```
219
-
220
- Install all generated site skills:
221
-
222
- ```bash
223
- omnibot skills install --agent opencode --all-sites
224
- ```
225
-
226
- Default `omnibot skills install --agent <agent>` installs only the official packaged omnibot skill, not generated site skills.
227
-
228
- Local site memory is reusable strategy, not user task output. Do not write user inputs, scraped records, or private task results into generated site packages.
229
-
230
- ## Fallback Discipline
231
-
232
- Fallback tiers are for completing operations after better patterns fail. They are not debugging shortcuts.
233
-
234
- Tier order:
235
-
236
- 1. Semantic `find`.
237
- 2. `snapshot -i` refs.
238
- 3. Selectors.
239
- 4. `dom` fallback.
240
- 5. `mouse` fallback.
241
- 6. `execute-js` fallback.
242
- 7. Raw `cdp` fallback.
243
-
244
- When entering a lower tier, state why the higher tier failed. After any fallback action, verify with `snapshot`, `get`, `is`, `wait`, or equivalent evidence.
245
-
246
- ## Debug Evidence
247
-
248
- Debugging is for evidence. Fallback is for completing operations.
249
-
250
- Use screenshots, annotated screenshots, console logs, network logs, trace, record/replay, and CDP inspection to explain failures or prove state. Do not treat `execute-js` as ordinary debug output.
251
-
252
- ## Absolute Prohibitions
253
-
254
- - Missing `OMNIBOT_SESSION_TOKEN` or missing `--tab-id` on page-state commands.
255
- - Acting without verify or claiming success without evidence.
256
- - Using `execute-js` first, raw CSS before semantic find, `@eN` across tabs, implicit tab targeting, or shell sleep instead of `omnibot wait`.
257
- - Using removed commands: `switch-tab`, `focus-tab`, `tab switch`, `tab focus`. These are no longer available.
258
- - Using `tabs[0]` or the first tab as a target. The first tab is often a user tab or transport tab.
259
- - Closing user tabs discovered during cleanup. Only close tabs created by the current workflow.
260
-
261
- See `references/anti-patterns.md` before using shortcuts.