@omniaibot/win-x64 1.1.2 → 1.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/bin/omnibot-windows-x64/81d243bd2c585b0f4821__mypyc.pyd +0 -0
- package/bin/omnibot-windows-x64/VERSION +1 -1
- package/bin/omnibot-windows-x64/_asyncio.pyd +0 -0
- package/bin/omnibot-windows-x64/_bz2.pyd +0 -0
- package/bin/omnibot-windows-x64/_cffi_backend.pyd +0 -0
- package/bin/omnibot-windows-x64/_ctypes.pyd +0 -0
- package/bin/omnibot-windows-x64/_decimal.pyd +0 -0
- package/bin/omnibot-windows-x64/_hashlib.pyd +0 -0
- package/bin/omnibot-windows-x64/_lzma.pyd +0 -0
- package/bin/omnibot-windows-x64/_multiprocessing.pyd +0 -0
- package/bin/omnibot-windows-x64/_overlapped.pyd +0 -0
- package/bin/omnibot-windows-x64/_queue.pyd +0 -0
- package/bin/omnibot-windows-x64/_socket.pyd +0 -0
- package/bin/omnibot-windows-x64/_ssl.pyd +0 -0
- package/bin/omnibot-windows-x64/_uuid.pyd +0 -0
- package/bin/omnibot-windows-x64/_wmi.pyd +0 -0
- package/bin/omnibot-windows-x64/charset_normalizer/cd.pyd +0 -0
- package/bin/omnibot-windows-x64/charset_normalizer/md.pyd +0 -0
- package/bin/omnibot-windows-x64/{libcrypto-3-x64.dll → libcrypto-3.dll} +0 -0
- package/bin/omnibot-windows-x64/libffi-8.dll +0 -0
- package/bin/omnibot-windows-x64/libssl-3.dll +0 -0
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/SKILL.md +20 -13
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/anti-patterns.md +4 -2
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/command-reference.md +48 -44
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/css-only-dropdown-js.md +50 -0
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/debugging-and-evidence.md +24 -22
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/fallback-operations.md +19 -5
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/operation-patterns.md +34 -22
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/session-and-tabs.md +14 -14
- package/bin/omnibot-windows-x64/omnibot-windows-x64.exe +0 -0
- package/bin/omnibot-windows-x64/pyexpat.pyd +0 -0
- package/bin/omnibot-windows-x64/python3.dll +0 -0
- package/bin/omnibot-windows-x64/python312.dll +0 -0
- package/bin/omnibot-windows-x64/select.pyd +0 -0
- package/bin/omnibot-windows-x64/unicodedata.pyd +0 -0
- package/bin/omnibot-windows-x64/vcruntime140.dll +0 -0
- package/bin/omnibot-windows-x64/vcruntime140_1.dll +0 -0
- package/package.json +29 -29
- package/bin/omnibot-windows-x64/ffi.dll +0 -0
- package/bin/omnibot-windows-x64/libbz2.dll +0 -0
- package/bin/omnibot-windows-x64/libexpat.dll +0 -0
- package/bin/omnibot-windows-x64/liblzma.dll +0 -0
- package/bin/omnibot-windows-x64/libmpdec-4.dll +0 -0
- package/bin/omnibot-windows-x64/libssl-3-x64.dll +0 -0
- package/bin/omnibot-windows-x64/python313.dll +0 -0
- package/bin/omnibot-windows-x64/zlib.dll +0 -0
|
Binary file
|
|
@@ -1 +1 @@
|
|
|
1
|
-
|
|
1
|
+
v1.3.0
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
@@ -5,21 +5,21 @@ compatibility: Requires omnibot v2 CLI daemon and the omnibot Chromium extension
|
|
|
5
5
|
allowed-tools: Bash
|
|
6
6
|
metadata:
|
|
7
7
|
author: omnibot
|
|
8
|
-
version: "2.
|
|
8
|
+
version: "2.2.0"
|
|
9
9
|
---
|
|
10
10
|
|
|
11
11
|
# Omnibot
|
|
12
12
|
|
|
13
13
|
Omnibot is Browser Infrastructure for AI Agents.
|
|
14
14
|
|
|
15
|
-
It connects Hermes, Claude Code, Codex, OpenCode,
|
|
15
|
+
It connects Hermes, Claude Code, Codex, OpenCode, and other agent systems to a real Chromium browser through the local omnibot daemon and CLI.
|
|
16
16
|
|
|
17
17
|
This skill is an execution specification for agents. It is not a human command manual.
|
|
18
18
|
|
|
19
19
|
## When to Use
|
|
20
20
|
|
|
21
21
|
- Use Omnibot when browser runtime state matters: login, cookies, storage, client-side rendering, extensions, or visible user tabs.
|
|
22
|
-
- Use Omnibot when an agent must read rendered pages, click controls, fill forms,
|
|
22
|
+
- Use Omnibot when an agent must read rendered pages, click controls, fill forms, navigate, extract content, or collect evidence.
|
|
23
23
|
- Do not use Omnibot when static files or plain HTTP content already answer the task.
|
|
24
24
|
|
|
25
25
|
## Core Rules
|
|
@@ -35,16 +35,16 @@ This skill is an execution specification for agents. It is not a human command m
|
|
|
35
35
|
- Parse JSON output from commands that return JSON.
|
|
36
36
|
- Do not claim success without verification evidence.
|
|
37
37
|
|
|
38
|
-
##
|
|
38
|
+
## Workflow Context + Tab Target
|
|
39
39
|
|
|
40
40
|
Every page-state workflow must use both controls:
|
|
41
41
|
|
|
42
42
|
- `OMNIBOT_SESSION_TOKEN=<workflow-name>` for workflow isolation.
|
|
43
43
|
- `--tab-id <TAB_ID>` for page targeting.
|
|
44
44
|
|
|
45
|
-
|
|
45
|
+
OMNIBOT_SESSION_TOKEN is a workflow/context token, not a browser session and not a tab target. It isolates agent state such as refs, trace, recording, aliases, and temporary resources. Tab ID controls which page receives the command.
|
|
46
46
|
|
|
47
|
-
Do not rely on default tab, active tab, current tab, or prior targeting state. Every page command requires explicit `--tab-id
|
|
47
|
+
Do not rely on default tab, active tab, current tab, or prior targeting state. Every page-state command requires explicit `--tab-id` except when the command is only discovering or creating a tab.
|
|
48
48
|
|
|
49
49
|
Use the same token inside one workflow. Use different tokens for independent workflows.
|
|
50
50
|
|
|
@@ -56,7 +56,7 @@ OMNIBOT_SESSION_TOKEN=research omnibot click --tab-id <TAB_ID> @e4
|
|
|
56
56
|
OMNIBOT_SESSION_TOKEN=research omnibot snapshot -i --tab-id <TAB_ID>
|
|
57
57
|
```
|
|
58
58
|
|
|
59
|
-
`@eN` refs are tab-scoped. never reuse `@eN` refs across tabs.
|
|
59
|
+
`@eN` refs are tab-scoped. Refs rule: never reuse `@eN` refs across tabs.
|
|
60
60
|
|
|
61
61
|
The agent dispatch pattern is token + tab-id:
|
|
62
62
|
|
|
@@ -75,6 +75,14 @@ OMNIBOT_SESSION_TOKEN=checkout omnibot dom dblclick n1 --tab-id <TAB_ID>
|
|
|
75
75
|
OMNIBOT_SESSION_TOKEN=checkout omnibot clipboard read --tab-id <TAB_ID>
|
|
76
76
|
```
|
|
77
77
|
|
|
78
|
+
Targeting categories:
|
|
79
|
+
|
|
80
|
+
| Command category | Rule |
|
|
81
|
+
| --- | --- |
|
|
82
|
+
| Page-state commands: `snapshot`, `click`, `fill`, `type`, `get`, `is`, `wait`, `screenshot`, `console`, `network`, `execute-js`, `dom`, `mouse`, `clipboard`, `viewport`, `assets`, `goto`, same-tab `navigate` | Require `--tab-id <TAB_ID>`. |
|
|
83
|
+
| Discovery or tab creation: `tabs`, `tab list`, `tab new <URL>`, `open <URL>`, `navigate <URL>` | Return or discover the tab id; save it for later page-state commands. |
|
|
84
|
+
| Read URL exception: `read <URL>` | Opens a temporary read tab and does not need `--tab-id`; reading an existing tab still requires `read --tab-id <TAB_ID>`. |
|
|
85
|
+
|
|
78
86
|
## Observe -> Act -> Verify
|
|
79
87
|
|
|
80
88
|
All page operations must follow this loop:
|
|
@@ -104,22 +112,22 @@ If verification fails, do not repeat blindly. Re-observe, choose the next fallba
|
|
|
104
112
|
|
|
105
113
|
## Pattern > Command
|
|
106
114
|
|
|
107
|
-
Start from the task, not the command name. Use operation patterns for read, click, fill, select/check, navigation, wait, extraction,
|
|
115
|
+
Start from the task, not the command name. Use operation patterns for read, click, fill, select/check, navigation, wait, extraction, and batch.
|
|
108
116
|
|
|
109
117
|
Use fallback tiers only after standard patterns fail. Collect evidence separately from fallback execution. The command reference is only a lookup table; it must not decide behavior.
|
|
110
118
|
|
|
111
|
-
## Read vs
|
|
119
|
+
## Read vs Snapshot Routing
|
|
112
120
|
|
|
113
121
|
Choose by intent before choosing by command name:
|
|
114
122
|
|
|
115
123
|
| Intent | Prefer | Why |
|
|
116
124
|
| --- | --- | --- |
|
|
117
125
|
| Summarize or extract page content, article text, search results, feeds, or long/lazy pages | `read` | Returns clean text/Markdown for agent reasoning. |
|
|
118
|
-
| Quickly observe
|
|
126
|
+
| Quickly observe current visible UI structure before deciding what to do next | `snapshot -i` | Returns actionable refs and page structure for routing. |
|
|
119
127
|
| Find buttons, links, inputs, or refs for a later action | `find` or `snapshot -i` | Produces actionable targets; `read` does not. |
|
|
120
128
|
| Verify one concrete condition | `get`, `is`, or `wait` | Narrow evidence is more reliable than dumping a page. |
|
|
121
129
|
|
|
122
|
-
Do not use `
|
|
130
|
+
Do not use `read` as the first step for click/fill workflows unless the user explicitly asked for page content first.
|
|
123
131
|
|
|
124
132
|
## Quick Routing
|
|
125
133
|
|
|
@@ -133,7 +141,6 @@ Do not use `scan` as a long-form reader. Do not use `read` as the first step for
|
|
|
133
141
|
| Navigation | `references/operation-patterns.md#navigation` |
|
|
134
142
|
| Waiting | `references/operation-patterns.md#wait` |
|
|
135
143
|
| Extraction | `references/operation-patterns.md#extraction` |
|
|
136
|
-
| Upload | `references/operation-patterns.md#upload` |
|
|
137
144
|
| Batch | `references/operation-patterns.md#batch` |
|
|
138
145
|
| Fallback | `references/fallback-operations.md` |
|
|
139
146
|
| Tabs and sessions | `references/session-and-tabs.md` |
|
|
@@ -182,7 +189,7 @@ Use screenshots, annotated screenshots, console logs, network logs, trace, recor
|
|
|
182
189
|
- Missing `OMNIBOT_SESSION_TOKEN` or missing `--tab-id` on page-state commands.
|
|
183
190
|
- Acting without verify or claiming success without evidence.
|
|
184
191
|
- Using `execute-js` first, raw CSS before semantic find, `@eN` across tabs, implicit tab targeting, or shell sleep instead of `omnibot wait`.
|
|
185
|
-
- Using removed commands: `switch-tab`, `focus-tab`, `tab switch`. These are no longer available.
|
|
192
|
+
- Using removed commands: `switch-tab`, `focus-tab`, `tab switch`, `tab focus`. These are no longer available.
|
|
186
193
|
- Using `tabs[0]` or the first tab as a target. The first tab is often a user tab or transport tab.
|
|
187
194
|
- Closing user tabs discovered during cleanup. Only close tabs created by the current workflow.
|
|
188
195
|
|
|
@@ -4,7 +4,7 @@
|
|
|
4
4
|
| --- | --- | --- |
|
|
5
5
|
| Missing `OMNIBOT_SESSION_TOKEN` | Workflows can share implicit state across agents. | `OMNIBOT_SESSION_TOKEN=research omnibot snapshot -i --tab-id <TAB_ID>` |
|
|
6
6
|
| Missing `--tab-id` | The command may target the wrong page. | Pass `--tab-id <TAB_ID>` on every page-state command. |
|
|
7
|
-
| `execute-js` first | JavaScript bypasses standard browser interaction and hides better evidence. | Try semantic find, snapshot refs, selectors, DOM, and mouse first. |
|
|
7
|
+
| `execute-js` first | JavaScript bypasses standard browser interaction and hides better evidence. | Try semantic find, snapshot refs, selectors, DOM, and mouse first. For CSS-only dropdowns, use JavaScript only after the trigger clicked, options remain absent from `snapshot -i`, and higher tiers cannot complete open-then-select. |
|
|
8
8
|
| Reusing `@eN` across tabs | Refs are tab-scoped and may point to a different element elsewhere. | Take a fresh `snapshot -i --tab-id <TAB_ID>` per tab. |
|
|
9
9
|
| Relying on implicit tab targeting | It is unavailable and unsafe for concurrent workflows. | Use explicit `--tab-id` on every page-state command. |
|
|
10
10
|
| Using sleep instead of wait | Fixed sleeps are flaky and either too short or too slow. | Use `omnibot wait --text`, `--url`, selector state, load state, or `--fn`. |
|
|
@@ -12,7 +12,6 @@
|
|
|
12
12
|
| Claiming success without evidence | The browser may not have changed as expected. | Cite verified URL, text, element state, screenshot, console, or network evidence. |
|
|
13
13
|
| Raw CSS before semantic find | CSS is more brittle and less tied to user intent. | Start with `find role/text/label/placeholder/testid`. |
|
|
14
14
|
| Screenshot when text extraction is enough | Screenshots cost more and are harder to parse. | Use `read` for page content or `get text` for a selector. |
|
|
15
|
-
| Using `scan` for long-form reading | `scan` is viewport observation and may miss long/lazy content. | Use `read --screens N --tab-id <TAB_ID>`. |
|
|
16
15
|
| Using `read` before an action workflow | `read` returns text, not stable action targets. | Use `find`, `snapshot -i`, `get`, or `is` for click/fill/verify workflows. |
|
|
17
16
|
| Headless expecting existing user login | Headless does not automatically share the user's visible browser session. | Use visible/background mode for existing login or explicitly log in headless. |
|
|
18
17
|
| Mixing multiple workflows in one session token | Agents can overwrite each other's default state. | Use different tokens: `research`, `checkout`, `debug`. |
|
|
@@ -31,5 +30,8 @@
|
|
|
31
30
|
- "The ref probably still points to the same element."
|
|
32
31
|
- "A screenshot is enough even though I need text."
|
|
33
32
|
- "Sleep should be fine."
|
|
33
|
+
- "The trigger clicked, so another `find text` should eventually select the option."
|
|
34
|
+
- "`snapshot -i` cannot see the option, so the element must not exist."
|
|
35
|
+
- "This dropdown is custom, so I can start with JavaScript."
|
|
34
36
|
|
|
35
37
|
All of these mean: stop, re-enter the standard pattern, and make state explicit.
|
|
@@ -2,6 +2,8 @@
|
|
|
2
2
|
|
|
3
3
|
This is a lookup table, not a behavior guide. Choose behavior from `operation-patterns.md`, use `session-and-tabs.md` for targeting, and use `fallback-operations.md` before dropping to lower tiers.
|
|
4
4
|
|
|
5
|
+
Unless marked as discovery, runtime, or tab creation, page-state examples assume `OMNIBOT_SESSION_TOKEN=<workflow>` and `--tab-id <TAB_ID>`.
|
|
6
|
+
|
|
5
7
|
## Runtime & Status
|
|
6
8
|
|
|
7
9
|
### doctor
|
|
@@ -80,30 +82,19 @@ Fallback relation: Not a fallback command.
|
|
|
80
82
|
|
|
81
83
|
## Reading
|
|
82
84
|
|
|
83
|
-
### scan
|
|
84
|
-
|
|
85
|
-
Purpose: Observe the current rendered viewport, especially before choosing a follow-up action.
|
|
86
|
-
|
|
87
|
-
```bash
|
|
88
|
-
OMNIBOT_SESSION_TOKEN=research omnibot scan --json --tab-id <TAB_ID>
|
|
89
|
-
OMNIBOT_SESSION_TOKEN=research omnibot scan --text-only --tab-id <TAB_ID>
|
|
90
|
-
```
|
|
91
|
-
|
|
92
|
-
Preferred pattern: See `operation-patterns.md#read` and `operation-patterns.md#extraction`.
|
|
93
|
-
Fallback relation: Use before `dom visible` or `execute-js` when structured viewport state is needed. Use `read` instead for long-form page text.
|
|
94
|
-
|
|
95
85
|
### read
|
|
96
86
|
|
|
97
87
|
Purpose: Read clean rendered page text/Markdown from an existing tab or a temporary URL tab.
|
|
98
88
|
|
|
99
89
|
```bash
|
|
100
90
|
OMNIBOT_SESSION_TOKEN=research omnibot read --screens 5 --tab-id <TAB_ID>
|
|
91
|
+
OMNIBOT_SESSION_TOKEN=research omnibot read --screens 5 --timeout 120 --tab-id <TAB_ID>
|
|
101
92
|
OMNIBOT_SESSION_TOKEN=research omnibot read --screens 3 https://example.com/article
|
|
102
93
|
OMNIBOT_SESSION_TOKEN=research omnibot read --json --screens 5 --tab-id <TAB_ID>
|
|
103
94
|
```
|
|
104
95
|
|
|
105
96
|
Preferred pattern: See `operation-patterns.md#read` and `operation-patterns.md#extraction`.
|
|
106
|
-
Fallback relation: Prefer for page content. Use `
|
|
97
|
+
Fallback relation: Prefer for page content. Use `snapshot -i` when the next step is action planning, and `get`/`is`/`wait` for narrow state verification.
|
|
107
98
|
|
|
108
99
|
### snapshot -i
|
|
109
100
|
|
|
@@ -241,7 +232,7 @@ OMNIBOT_SESSION_TOKEN=form omnibot select @e5 "US" --tab-id <TAB_ID>
|
|
|
241
232
|
```
|
|
242
233
|
|
|
243
234
|
Preferred pattern: See `operation-patterns.md#select--check`.
|
|
244
|
-
Fallback relation: Prefer refs/selectors before JavaScript value changes.
|
|
235
|
+
Fallback relation: Prefer refs/selectors before JavaScript value changes. For CSS-only dropdowns whose options are absent from `snapshot -i`, see `fallback-operations.md#css-only-dropdown-javascript-fallback`.
|
|
245
236
|
|
|
246
237
|
### check / uncheck
|
|
247
238
|
|
|
@@ -281,14 +272,28 @@ Fallback relation: Mouse drag is Tier 5 if element drag fails.
|
|
|
281
272
|
|
|
282
273
|
### upload
|
|
283
274
|
|
|
284
|
-
Purpose:
|
|
275
|
+
Purpose: Reserved command for local file upload. Native upload is currently unavailable in the extension transport.
|
|
285
276
|
|
|
286
277
|
```bash
|
|
287
|
-
|
|
278
|
+
# Do not use in agent workflows until file chooser support is implemented.
|
|
288
279
|
```
|
|
289
280
|
|
|
290
281
|
Preferred pattern: See `operation-patterns.md#upload`.
|
|
291
|
-
Fallback relation:
|
|
282
|
+
Fallback relation: Do not treat JavaScript as a normal upload fallback.
|
|
283
|
+
|
|
284
|
+
### keyboard / keydown / keyup
|
|
285
|
+
|
|
286
|
+
Purpose: Send keyboard text or key state to the target tab when element-specific `type` or `press` is insufficient.
|
|
287
|
+
|
|
288
|
+
```bash
|
|
289
|
+
OMNIBOT_SESSION_TOKEN=form omnibot keyboard type "hello" --tab-id <TAB_ID>
|
|
290
|
+
OMNIBOT_SESSION_TOKEN=form omnibot keyboard inserttext "hello" --tab-id <TAB_ID>
|
|
291
|
+
OMNIBOT_SESSION_TOKEN=form omnibot keydown Shift --tab-id <TAB_ID>
|
|
292
|
+
OMNIBOT_SESSION_TOKEN=form omnibot keyup Shift --tab-id <TAB_ID>
|
|
293
|
+
```
|
|
294
|
+
|
|
295
|
+
Preferred pattern: Prefer `fill`, `type`, and `press` first.
|
|
296
|
+
Fallback relation: Keyboard commands are a focused fallback before mouse or JavaScript.
|
|
292
297
|
|
|
293
298
|
## Semantic Find
|
|
294
299
|
|
|
@@ -350,7 +355,7 @@ OMNIBOT_SESSION_TOKEN=repair omnibot execute-js --file /tmp/repair.js --tab-id <
|
|
|
350
355
|
```
|
|
351
356
|
|
|
352
357
|
Preferred pattern: See `fallback-operations.md#javascript-fallback`.
|
|
353
|
-
Fallback relation: After semantic, refs, selector, DOM, and mouse are insufficient.
|
|
358
|
+
Fallback relation: After semantic, refs, selector, DOM, and mouse are insufficient. Valid examples include CSS-only dropdowns where the trigger exists but option refs are absent from `snapshot -i`.
|
|
354
359
|
|
|
355
360
|
### cdp
|
|
356
361
|
|
|
@@ -358,8 +363,8 @@ Purpose: Send raw Chrome DevTools Protocol commands.
|
|
|
358
363
|
Usage tier: Tier 7 fallback and inspection only.
|
|
359
364
|
|
|
360
365
|
```bash
|
|
361
|
-
OMNIBOT_SESSION_TOKEN=debug omnibot cdp Runtime.evaluate '{"expression":"document.title"}' --tab-id <TAB_ID>
|
|
362
|
-
OMNIBOT_SESSION_TOKEN=debug omnibot cdp DOM.getDocument '{"depth":1}' --tab-id <TAB_ID>
|
|
366
|
+
OMNIBOT_SESSION_TOKEN=debug-checkout omnibot cdp Runtime.evaluate '{"expression":"document.title"}' --tab-id <TAB_ID>
|
|
367
|
+
OMNIBOT_SESSION_TOKEN=debug-checkout omnibot cdp DOM.getDocument '{"depth":1}' --tab-id <TAB_ID>
|
|
363
368
|
```
|
|
364
369
|
|
|
365
370
|
Preferred pattern: See `fallback-operations.md#raw-cdp-fallback` and `debugging-and-evidence.md#cdp-inspection`.
|
|
@@ -369,13 +374,13 @@ Fallback relation: Last resort.
|
|
|
369
374
|
|
|
370
375
|
### navigate / open / goto
|
|
371
376
|
|
|
372
|
-
Purpose: Navigate or open pages.
|
|
377
|
+
Purpose: Navigate or open pages. New-tab forms return a tab id. Same-tab navigation requires an explicit target tab id. There is no CLI current-tab state.
|
|
373
378
|
|
|
374
379
|
```bash
|
|
375
380
|
OMNIBOT_SESSION_TOKEN=research omnibot navigate https://example.com
|
|
376
|
-
OMNIBOT_SESSION_TOKEN=research omnibot navigate --same-tab
|
|
381
|
+
OMNIBOT_SESSION_TOKEN=research omnibot navigate https://example.com --same-tab --tab-id <TAB_ID>
|
|
377
382
|
OMNIBOT_SESSION_TOKEN=research omnibot open https://example.com
|
|
378
|
-
OMNIBOT_SESSION_TOKEN=research omnibot goto https://example.com
|
|
383
|
+
OMNIBOT_SESSION_TOKEN=research omnibot goto https://example.com --tab-id <TAB_ID>
|
|
379
384
|
```
|
|
380
385
|
|
|
381
386
|
Preferred pattern: See `operation-patterns.md#navigation` and `session-and-tabs.md#tab-explicit`.
|
|
@@ -425,7 +430,6 @@ OMNIBOT_SESSION_TOKEN=research omnibot frame main
|
|
|
425
430
|
Purpose: Wait for time, selector, text, URL, load state, or JavaScript condition.
|
|
426
431
|
|
|
427
432
|
```bash
|
|
428
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot wait 500 --tab-id <TAB_ID>
|
|
429
433
|
OMNIBOT_SESSION_TOKEN=checkout omnibot wait "#ready" --tab-id <TAB_ID>
|
|
430
434
|
OMNIBOT_SESSION_TOKEN=checkout omnibot wait "#spinner" --state hidden --tab-id <TAB_ID>
|
|
431
435
|
OMNIBOT_SESSION_TOKEN=checkout omnibot wait --text "Welcome" --tab-id <TAB_ID>
|
|
@@ -433,17 +437,18 @@ OMNIBOT_SESSION_TOKEN=checkout omnibot wait --url "/dashboard" --tab-id <TAB_ID>
|
|
|
433
437
|
OMNIBOT_SESSION_TOKEN=checkout omnibot wait --load domcontentloaded --tab-id <TAB_ID>
|
|
434
438
|
OMNIBOT_SESSION_TOKEN=checkout omnibot wait --load networkidle --tab-id <TAB_ID>
|
|
435
439
|
OMNIBOT_SESSION_TOKEN=checkout omnibot wait --fn "window.appReady === true" --tab-id <TAB_ID>
|
|
440
|
+
OMNIBOT_SESSION_TOKEN=checkout omnibot wait 500 --tab-id <TAB_ID>
|
|
436
441
|
```
|
|
437
442
|
|
|
438
443
|
Preferred pattern: See `operation-patterns.md#wait`.
|
|
439
|
-
Fallback relation: Prefer over shell sleep.
|
|
444
|
+
Fallback relation: Prefer condition waits over shell sleep. Fixed-time waits like `wait 500` are diagnostic or last resort only.
|
|
440
445
|
|
|
441
446
|
### batch
|
|
442
447
|
|
|
443
|
-
Purpose: Send short command arrays as JSON or from a file.
|
|
448
|
+
Purpose: Send short extension/CDP command arrays as JSON or from a file. Batch is not a wrapper for normal CLI commands such as `snapshot` or `click`.
|
|
444
449
|
|
|
445
450
|
```bash
|
|
446
|
-
OMNIBOT_SESSION_TOKEN=research omnibot batch '[{"cmd":"
|
|
451
|
+
OMNIBOT_SESSION_TOKEN=research omnibot batch '[{"cmd":"cdp","method":"Runtime.evaluate","params":{"expression":"document.title","returnByValue":true}}]' --tab-id <TAB_ID>
|
|
447
452
|
OMNIBOT_SESSION_TOKEN=research omnibot batch --file /tmp/omnibot-batch.json --tab-id <TAB_ID>
|
|
448
453
|
```
|
|
449
454
|
|
|
@@ -457,8 +462,8 @@ Fallback relation: Not a substitute for verification.
|
|
|
457
462
|
Purpose: Capture visual evidence from a browser tab.
|
|
458
463
|
|
|
459
464
|
```bash
|
|
460
|
-
OMNIBOT_SESSION_TOKEN=debug omnibot screenshot --tab-id <TAB_ID> -o /tmp/omni-shot.png
|
|
461
|
-
OMNIBOT_SESSION_TOKEN=debug omnibot screenshot --annotate --tab-id <TAB_ID> -o /tmp/omni-annotated.png
|
|
465
|
+
OMNIBOT_SESSION_TOKEN=debug-checkout omnibot screenshot --tab-id <TAB_ID> -o /tmp/omni-shot.png
|
|
466
|
+
OMNIBOT_SESSION_TOKEN=debug-checkout omnibot screenshot --annotate --tab-id <TAB_ID> -o /tmp/omni-annotated.png
|
|
462
467
|
```
|
|
463
468
|
|
|
464
469
|
Preferred pattern: See `debugging-and-evidence.md#screenshot`.
|
|
@@ -469,21 +474,21 @@ Fallback relation: Evidence collection, not operation fallback.
|
|
|
469
474
|
Purpose: Read or clear browser console evidence.
|
|
470
475
|
|
|
471
476
|
```bash
|
|
472
|
-
OMNIBOT_SESSION_TOKEN=debug omnibot console logs --tab-id <TAB_ID>
|
|
473
|
-
OMNIBOT_SESSION_TOKEN=debug omnibot console errors --tab-id <TAB_ID>
|
|
474
|
-
OMNIBOT_SESSION_TOKEN=debug omnibot console clear --tab-id <TAB_ID>
|
|
477
|
+
OMNIBOT_SESSION_TOKEN=debug-checkout omnibot console logs --tab-id <TAB_ID>
|
|
478
|
+
OMNIBOT_SESSION_TOKEN=debug-checkout omnibot console errors --tab-id <TAB_ID>
|
|
479
|
+
OMNIBOT_SESSION_TOKEN=debug-checkout omnibot console clear --tab-id <TAB_ID>
|
|
475
480
|
```
|
|
476
481
|
|
|
477
482
|
Preferred pattern: See `debugging-and-evidence.md#console`.
|
|
478
|
-
Fallback relation: Evidence collection, not operation fallback.
|
|
483
|
+
Fallback relation: Evidence collection, not operation fallback. `console errors` currently uses the console log collection path; verify level filtering in output before treating it as error-only evidence.
|
|
479
484
|
|
|
480
485
|
### network
|
|
481
486
|
|
|
482
487
|
Purpose: Read browser network evidence.
|
|
483
488
|
|
|
484
489
|
```bash
|
|
485
|
-
OMNIBOT_SESSION_TOKEN=debug omnibot network logs --tab-id <TAB_ID>
|
|
486
|
-
OMNIBOT_SESSION_TOKEN=debug omnibot network summary --tab-id <TAB_ID>
|
|
490
|
+
OMNIBOT_SESSION_TOKEN=debug-checkout omnibot network logs --tab-id <TAB_ID>
|
|
491
|
+
OMNIBOT_SESSION_TOKEN=debug-checkout omnibot network summary --tab-id <TAB_ID>
|
|
487
492
|
```
|
|
488
493
|
|
|
489
494
|
Preferred pattern: See `debugging-and-evidence.md#network`.
|
|
@@ -531,18 +536,17 @@ Fallback relation: Use after page content indicates assets are needed.
|
|
|
531
536
|
|
|
532
537
|
### visibility
|
|
533
538
|
|
|
534
|
-
Purpose: Inspect
|
|
539
|
+
Purpose: Inspect or set automation browser visibility mode. `visibility launch` currently reports planned configuration only; it does not launch a browser.
|
|
535
540
|
|
|
536
541
|
```bash
|
|
537
542
|
omnibot visibility status
|
|
538
543
|
omnibot visibility set background
|
|
539
544
|
omnibot visibility set visible
|
|
540
|
-
omnibot visibility set headless --user-data-dir /tmp/omnibot-headless
|
|
541
545
|
omnibot visibility launch headless --user-data-dir /tmp/omnibot-headless
|
|
542
546
|
```
|
|
543
547
|
|
|
544
548
|
Preferred pattern: See `session-and-tabs.md#visibility`.
|
|
545
|
-
Fallback relation: Headless does not inherit existing user login state.
|
|
549
|
+
Fallback relation: Headless does not inherit existing user login state. Treat `visibility launch` status `planned` as not launched.
|
|
546
550
|
|
|
547
551
|
### browser
|
|
548
552
|
|
|
@@ -560,7 +564,7 @@ Fallback relation: Use in multi-browser or multi-runtime scenarios.
|
|
|
560
564
|
|
|
561
565
|
### session
|
|
562
566
|
|
|
563
|
-
Purpose: Name or list
|
|
567
|
+
Purpose: Name or list Omnibot workflow context state for the current `OMNIBOT_SESSION_TOKEN`. This does not switch tabs, preserve login, or replace `--tab-id`.
|
|
564
568
|
|
|
565
569
|
```bash
|
|
566
570
|
OMNIBOT_SESSION_TOKEN=checkout omnibot session name "checkout"
|
|
@@ -577,11 +581,11 @@ Fallback relation: Not a fallback command.
|
|
|
577
581
|
Purpose: Capture, replay, or trace workflows for evidence.
|
|
578
582
|
|
|
579
583
|
```bash
|
|
580
|
-
OMNIBOT_SESSION_TOKEN=debug omnibot record start
|
|
581
|
-
OMNIBOT_SESSION_TOKEN=debug omnibot record stop -o flow.json
|
|
582
|
-
OMNIBOT_SESSION_TOKEN=debug omnibot replay flow.json
|
|
583
|
-
OMNIBOT_SESSION_TOKEN=debug omnibot trace start
|
|
584
|
-
OMNIBOT_SESSION_TOKEN=debug omnibot trace stop -o trace.zip
|
|
584
|
+
OMNIBOT_SESSION_TOKEN=debug-checkout omnibot record start
|
|
585
|
+
OMNIBOT_SESSION_TOKEN=debug-checkout omnibot record stop -o flow.json
|
|
586
|
+
OMNIBOT_SESSION_TOKEN=debug-checkout omnibot replay flow.json
|
|
587
|
+
OMNIBOT_SESSION_TOKEN=debug-checkout omnibot trace start
|
|
588
|
+
OMNIBOT_SESSION_TOKEN=debug-checkout omnibot trace stop -o trace.zip
|
|
585
589
|
```
|
|
586
590
|
|
|
587
591
|
Preferred pattern: See `debugging-and-evidence.md#record-and-replay` and `debugging-and-evidence.md#trace`.
|
|
@@ -0,0 +1,50 @@
|
|
|
1
|
+
# CSS-Only Dropdown JavaScript Recipe
|
|
2
|
+
|
|
3
|
+
Use only after semantic find, refs, selectors, DOM, and mouse fallback cannot expose or select the dropdown option.
|
|
4
|
+
|
|
5
|
+
Example for a visible trigger like `按留言时间排序` and a target option text:
|
|
6
|
+
|
|
7
|
+
```js
|
|
8
|
+
const triggerText = '按留言时间排序';
|
|
9
|
+
const optionText = '目标选项文本';
|
|
10
|
+
|
|
11
|
+
const visibleElements = [...document.querySelectorAll('button, [role="button"], [aria-haspopup], div, span, li')]
|
|
12
|
+
.filter((el) => {
|
|
13
|
+
const style = getComputedStyle(el);
|
|
14
|
+
const box = el.getBoundingClientRect();
|
|
15
|
+
return style.display !== 'none' && style.visibility !== 'hidden' && box.width > 0 && box.height > 0;
|
|
16
|
+
});
|
|
17
|
+
|
|
18
|
+
const byExactText = (text) => visibleElements.find((el) => el.textContent?.trim() === text);
|
|
19
|
+
const trigger = byExactText(triggerText);
|
|
20
|
+
if (!trigger) {
|
|
21
|
+
return { ok: false, reason: 'trigger-not-found', triggerText, optionText };
|
|
22
|
+
}
|
|
23
|
+
|
|
24
|
+
trigger.click();
|
|
25
|
+
await new Promise((resolve) => requestAnimationFrame(() => requestAnimationFrame(resolve)));
|
|
26
|
+
|
|
27
|
+
const option = [...document.querySelectorAll('div, span, li, button, [role="option"], [role="menuitem"]')]
|
|
28
|
+
.filter((el) => {
|
|
29
|
+
const style = getComputedStyle(el);
|
|
30
|
+
const box = el.getBoundingClientRect();
|
|
31
|
+
return style.display !== 'none' && style.visibility !== 'hidden' && box.width > 0 && box.height > 0;
|
|
32
|
+
})
|
|
33
|
+
.find((el) => el.textContent?.trim() === optionText);
|
|
34
|
+
|
|
35
|
+
if (!option) {
|
|
36
|
+
return { ok: false, reason: 'option-not-found-after-trigger-click', triggerText, optionText };
|
|
37
|
+
}
|
|
38
|
+
|
|
39
|
+
option.click();
|
|
40
|
+
await new Promise((resolve) => requestAnimationFrame(() => requestAnimationFrame(resolve)));
|
|
41
|
+
|
|
42
|
+
return {
|
|
43
|
+
ok: true,
|
|
44
|
+
triggerText,
|
|
45
|
+
optionText,
|
|
46
|
+
selectedText: trigger.textContent?.trim(),
|
|
47
|
+
};
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
Verify with page state after running the script: selected label, sorted result order, result count, URL/query state, or another user-visible condition.
|
|
@@ -4,13 +4,15 @@ Debugging is for evidence. Fallback is for completing operations.
|
|
|
4
4
|
|
|
5
5
|
Use this file when you need to prove browser state, explain a failure, or collect artifacts for review. Do not use `execute-js` as the first debug tool.
|
|
6
6
|
|
|
7
|
+
Use a unique workflow/debug token per task, such as `debug-checkout` or `debug-research`. Do not share one global `debug` token across independent agents.
|
|
8
|
+
|
|
7
9
|
## Screenshot
|
|
8
10
|
|
|
9
11
|
Use screenshots when visual layout, canvas, dialogs, or visual regressions matter.
|
|
10
12
|
|
|
11
13
|
```bash
|
|
12
|
-
OMNIBOT_SESSION_TOKEN=debug omnibot screenshot --tab-id <TAB_ID> -o /tmp/omni-shot.png
|
|
13
|
-
OMNIBOT_SESSION_TOKEN=debug omnibot screenshot --annotate --tab-id <TAB_ID> -o /tmp/omni-annotated.png
|
|
14
|
+
OMNIBOT_SESSION_TOKEN=debug-checkout omnibot screenshot --tab-id <TAB_ID> -o /tmp/omni-shot.png
|
|
15
|
+
OMNIBOT_SESSION_TOKEN=debug-checkout omnibot screenshot --annotate --tab-id <TAB_ID> -o /tmp/omni-annotated.png
|
|
14
16
|
```
|
|
15
17
|
|
|
16
18
|
Prefer text extraction when text is enough. Screenshot is evidence, not extraction.
|
|
@@ -20,26 +22,26 @@ Prefer text extraction when text is enough. Screenshot is evidence, not extracti
|
|
|
20
22
|
Use console logs for runtime JavaScript errors, frontend warnings, or app-level diagnostics.
|
|
21
23
|
|
|
22
24
|
```bash
|
|
23
|
-
OMNIBOT_SESSION_TOKEN=debug omnibot console errors --tab-id <TAB_ID>
|
|
24
|
-
OMNIBOT_SESSION_TOKEN=debug omnibot console logs --tab-id <TAB_ID>
|
|
25
|
-
OMNIBOT_SESSION_TOKEN=debug omnibot console clear --tab-id <TAB_ID>
|
|
25
|
+
OMNIBOT_SESSION_TOKEN=debug-checkout omnibot console errors --tab-id <TAB_ID>
|
|
26
|
+
OMNIBOT_SESSION_TOKEN=debug-checkout omnibot console logs --tab-id <TAB_ID>
|
|
27
|
+
OMNIBOT_SESSION_TOKEN=debug-checkout omnibot console clear --tab-id <TAB_ID>
|
|
26
28
|
```
|
|
27
29
|
|
|
28
|
-
Collect console
|
|
30
|
+
Collect console diagnostics before and after the action when debugging regressions. `console errors` currently uses the console log collection path; verify level filtering in output before treating it as error-only evidence.
|
|
29
31
|
|
|
30
32
|
## Network
|
|
31
33
|
|
|
32
34
|
Use network logs for failed requests, redirects, blocked assets, API errors, or loading stalls.
|
|
33
35
|
|
|
34
36
|
```bash
|
|
35
|
-
OMNIBOT_SESSION_TOKEN=debug omnibot network summary --tab-id <TAB_ID>
|
|
36
|
-
OMNIBOT_SESSION_TOKEN=debug omnibot network logs --tab-id <TAB_ID>
|
|
37
|
+
OMNIBOT_SESSION_TOKEN=debug-checkout omnibot network summary --tab-id <TAB_ID>
|
|
38
|
+
OMNIBOT_SESSION_TOKEN=debug-checkout omnibot network logs --tab-id <TAB_ID>
|
|
37
39
|
```
|
|
38
40
|
|
|
39
41
|
Pair network evidence with page verification:
|
|
40
42
|
|
|
41
43
|
```bash
|
|
42
|
-
OMNIBOT_SESSION_TOKEN=debug omnibot get text "#main" --tab-id <TAB_ID>
|
|
44
|
+
OMNIBOT_SESSION_TOKEN=debug-checkout omnibot get text "#main" --tab-id <TAB_ID>
|
|
43
45
|
```
|
|
44
46
|
|
|
45
47
|
## Trace
|
|
@@ -47,10 +49,10 @@ OMNIBOT_SESSION_TOKEN=debug omnibot get text "#main" --tab-id <TAB_ID>
|
|
|
47
49
|
Use trace for a reproducible artifact around a failing workflow.
|
|
48
50
|
|
|
49
51
|
```bash
|
|
50
|
-
OMNIBOT_SESSION_TOKEN=debug omnibot trace start
|
|
51
|
-
OMNIBOT_SESSION_TOKEN=debug omnibot click --tab-id <TAB_ID> @e4
|
|
52
|
-
OMNIBOT_SESSION_TOKEN=debug omnibot snapshot -i --tab-id <TAB_ID>
|
|
53
|
-
OMNIBOT_SESSION_TOKEN=debug omnibot trace stop -o trace.zip
|
|
52
|
+
OMNIBOT_SESSION_TOKEN=debug-checkout omnibot trace start
|
|
53
|
+
OMNIBOT_SESSION_TOKEN=debug-checkout omnibot click --tab-id <TAB_ID> @e4
|
|
54
|
+
OMNIBOT_SESSION_TOKEN=debug-checkout omnibot snapshot -i --tab-id <TAB_ID>
|
|
55
|
+
OMNIBOT_SESSION_TOKEN=debug-checkout omnibot trace stop -o trace.zip
|
|
54
56
|
```
|
|
55
57
|
|
|
56
58
|
Trace is evidence. It does not replace operation verification.
|
|
@@ -60,17 +62,17 @@ Trace is evidence. It does not replace operation verification.
|
|
|
60
62
|
Use record/replay when a flow must be captured or reproduced.
|
|
61
63
|
|
|
62
64
|
```bash
|
|
63
|
-
OMNIBOT_SESSION_TOKEN=debug omnibot record start
|
|
64
|
-
OMNIBOT_SESSION_TOKEN=debug omnibot click --tab-id <TAB_ID> @e4
|
|
65
|
-
OMNIBOT_SESSION_TOKEN=debug omnibot snapshot -i --tab-id <TAB_ID>
|
|
66
|
-
OMNIBOT_SESSION_TOKEN=debug omnibot record stop -o flow.json
|
|
67
|
-
OMNIBOT_SESSION_TOKEN=debug omnibot replay flow.json
|
|
65
|
+
OMNIBOT_SESSION_TOKEN=debug-checkout omnibot record start
|
|
66
|
+
OMNIBOT_SESSION_TOKEN=debug-checkout omnibot click --tab-id <TAB_ID> @e4
|
|
67
|
+
OMNIBOT_SESSION_TOKEN=debug-checkout omnibot snapshot -i --tab-id <TAB_ID>
|
|
68
|
+
OMNIBOT_SESSION_TOKEN=debug-checkout omnibot record stop -o flow.json
|
|
69
|
+
OMNIBOT_SESSION_TOKEN=debug-checkout omnibot replay flow.json
|
|
68
70
|
```
|
|
69
71
|
|
|
70
72
|
After replay, verify final state:
|
|
71
73
|
|
|
72
74
|
```bash
|
|
73
|
-
OMNIBOT_SESSION_TOKEN=debug omnibot get url --tab-id <TAB_ID>
|
|
75
|
+
OMNIBOT_SESSION_TOKEN=debug-checkout omnibot get url --tab-id <TAB_ID>
|
|
74
76
|
```
|
|
75
77
|
|
|
76
78
|
## CDP Inspection
|
|
@@ -78,8 +80,8 @@ OMNIBOT_SESSION_TOKEN=debug omnibot get url --tab-id <TAB_ID>
|
|
|
78
80
|
Use CDP for low-level inspection when normal evidence is insufficient.
|
|
79
81
|
|
|
80
82
|
```bash
|
|
81
|
-
OMNIBOT_SESSION_TOKEN=debug omnibot cdp Runtime.evaluate '{"expression":"document.title"}' --tab-id <TAB_ID>
|
|
82
|
-
OMNIBOT_SESSION_TOKEN=debug omnibot cdp DOM.getDocument '{"depth":1}' --tab-id <TAB_ID>
|
|
83
|
+
OMNIBOT_SESSION_TOKEN=debug-checkout omnibot cdp Runtime.evaluate '{"expression":"document.title"}' --tab-id <TAB_ID>
|
|
84
|
+
OMNIBOT_SESSION_TOKEN=debug-checkout omnibot cdp DOM.getDocument '{"depth":1}' --tab-id <TAB_ID>
|
|
83
85
|
```
|
|
84
86
|
|
|
85
87
|
CDP inspection is not a first-choice operation path. If CDP changes state, it becomes Tier 7 fallback and must follow `fallback-operations.md#raw-cdp-fallback`.
|
|
@@ -89,7 +91,7 @@ CDP inspection is not a first-choice operation path. If CDP changes state, it be
|
|
|
89
91
|
Before reporting a browser issue, collect the smallest evidence set that explains it:
|
|
90
92
|
|
|
91
93
|
- Runtime: `doctor`, `tabs`, `visibility status`.
|
|
92
|
-
- Page state: `get`, `is`, `
|
|
94
|
+
- Page state: `get`, `is`, `snapshot`, or `wait`.
|
|
93
95
|
- Visual proof: `screenshot --annotate` only if visual state matters.
|
|
94
96
|
- Browser diagnostics: `console errors` and `network summary`.
|
|
95
97
|
- Reproduction artifact: `trace` or `record` when the flow is unstable.
|
|
@@ -106,7 +106,8 @@ Use `execute-js` only when:
|
|
|
106
106
|
- CSS selector operation fails.
|
|
107
107
|
- DOM fallback fails.
|
|
108
108
|
- A complex frontend event must be triggered.
|
|
109
|
-
-
|
|
109
|
+
- A CSS-only dropdown or hover menu requires an atomic open-then-select sequence, and `snapshot -i` confirms the option nodes are absent from the accessibility tree.
|
|
110
|
+
- Runtime state cannot be read through `read`, `snapshot`, `get`, or `is`.
|
|
110
111
|
|
|
111
112
|
Read-only JavaScript fallback:
|
|
112
113
|
|
|
@@ -129,6 +130,19 @@ OMNIBOT_SESSION_TOKEN=repair omnibot execute-js "document.querySelector('button[
|
|
|
129
130
|
OMNIBOT_SESSION_TOKEN=repair omnibot wait --url "/dashboard" --tab-id <TAB_ID>
|
|
130
131
|
```
|
|
131
132
|
|
|
133
|
+
### CSS-only dropdown JavaScript fallback
|
|
134
|
+
|
|
135
|
+
Use this only after the trigger is known and higher tiers failed: semantic `find` clicked the trigger or found the text, `snapshot -i` could not expose option refs, selector/DOM/mouse could not complete the two-step interaction, and verification showed the selection did not change.
|
|
136
|
+
|
|
137
|
+
Use a short `execute-js --file` script and verify the user-visible result, not just `{ ok: true }`: selected label, sorted result order, result count, URL/query state, or another page-specific condition.
|
|
138
|
+
|
|
139
|
+
Detailed recipe: `css-only-dropdown-js.md`.
|
|
140
|
+
|
|
141
|
+
```bash
|
|
142
|
+
OMNIBOT_SESSION_TOKEN=repair omnibot execute-js --file /tmp/omnibot-css-dropdown.js --tab-id <TAB_ID>
|
|
143
|
+
OMNIBOT_SESSION_TOKEN=repair omnibot snapshot -i --tab-id <TAB_ID>
|
|
144
|
+
```
|
|
145
|
+
|
|
132
146
|
Prefer `--file` for longer scripts:
|
|
133
147
|
|
|
134
148
|
```bash
|
|
@@ -143,15 +157,15 @@ OMNIBOT_SESSION_TOKEN=repair omnibot snapshot -i --tab-id <TAB_ID>
|
|
|
143
157
|
Raw CDP is the last resort. Use it only when standard commands and JavaScript fallback cannot complete or inspect the task.
|
|
144
158
|
|
|
145
159
|
```bash
|
|
146
|
-
OMNIBOT_SESSION_TOKEN=debug omnibot cdp Runtime.evaluate '{"expression":"document.title"}' --tab-id <TAB_ID>
|
|
147
|
-
OMNIBOT_SESSION_TOKEN=debug omnibot cdp DOM.getDocument '{"depth":1}' --tab-id <TAB_ID>
|
|
160
|
+
OMNIBOT_SESSION_TOKEN=debug-checkout omnibot cdp Runtime.evaluate '{"expression":"document.title"}' --tab-id <TAB_ID>
|
|
161
|
+
OMNIBOT_SESSION_TOKEN=debug-checkout omnibot cdp DOM.getDocument '{"depth":1}' --tab-id <TAB_ID>
|
|
148
162
|
```
|
|
149
163
|
|
|
150
164
|
After CDP fallback, verify using normal Omnibot reads whenever possible:
|
|
151
165
|
|
|
152
166
|
```bash
|
|
153
|
-
OMNIBOT_SESSION_TOKEN=debug omnibot get title --tab-id <TAB_ID>
|
|
154
|
-
OMNIBOT_SESSION_TOKEN=debug omnibot snapshot -i --tab-id <TAB_ID>
|
|
167
|
+
OMNIBOT_SESSION_TOKEN=debug-checkout omnibot get title --tab-id <TAB_ID>
|
|
168
|
+
OMNIBOT_SESSION_TOKEN=debug-checkout omnibot snapshot -i --tab-id <TAB_ID>
|
|
155
169
|
```
|
|
156
170
|
|
|
157
171
|
CDP can bypass application-level event semantics. Treat it as expert-only infrastructure access.
|
|
@@ -8,7 +8,7 @@ Use this file to choose behavior by task. Every page-state workflow must set `OM
|
|
|
8
8
|
|
|
9
9
|
Use Read when the agent needs clean rendered page content: article text, search results, feed items, long pages, lazy-loaded text, or a human-readable summary source.
|
|
10
10
|
|
|
11
|
-
Use targeted state commands when the agent needs one value, visibility, enabled/checked state, layout box, or computed styles. Use
|
|
11
|
+
Use targeted state commands when the agent needs one value, visibility, enabled/checked state, layout box, or computed styles. Use `snapshot -i` when the agent needs current visible UI structure for deciding the next action.
|
|
12
12
|
|
|
13
13
|
### Preferred sequence
|
|
14
14
|
|
|
@@ -17,9 +17,8 @@ Use targeted state commands when the agent needs one value, visibility, enabled/
|
|
|
17
17
|
3. `read <URL>` when opening a temporary tab only for reading that URL.
|
|
18
18
|
4. `get title/url/text/html/value/attr/count/box/styles` for narrow evidence.
|
|
19
19
|
5. `is visible/enabled/checked` for boolean state.
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
8. `dom visible` or `execute-js` fallback only after standard reads cannot access the state.
|
|
20
|
+
5. `snapshot -i` when actionable refs are needed.
|
|
21
|
+
6. `dom visible` or `execute-js` fallback only after standard reads cannot access the state.
|
|
23
22
|
|
|
24
23
|
### Example
|
|
25
24
|
|
|
@@ -40,7 +39,7 @@ OMNIBOT_SESSION_TOKEN=research omnibot get count ".result" --tab-id <TAB_ID>
|
|
|
40
39
|
|
|
41
40
|
### Fallback entry point
|
|
42
41
|
|
|
43
|
-
If `read
|
|
42
|
+
If `read` or targeted reads omit needed runtime state, go to `fallback-operations.md#fallback-tier-model`. Start with DOM fallback before JavaScript.
|
|
44
43
|
|
|
45
44
|
## Click
|
|
46
45
|
|
|
@@ -161,9 +160,23 @@ OMNIBOT_SESSION_TOKEN=form omnibot is checked "#agree" --tab-id <TAB_ID>
|
|
|
161
160
|
|
|
162
161
|
For checkboxes, verify boolean state. For selects, verify value or dependent page state.
|
|
163
162
|
|
|
163
|
+
### CSS-only dropdowns and custom menus
|
|
164
|
+
|
|
165
|
+
Some dropdowns, hover menus, tooltips, and custom selects are CSS-only widgets. The visible trigger may be accessible, while the option nodes are hidden `<div>` elements without `role`, accessible name, or stable refs.
|
|
166
|
+
|
|
167
|
+
Use this evidence path before JavaScript:
|
|
168
|
+
|
|
169
|
+
1. Try semantic `find` or a normal `select`/`click` against the trigger.
|
|
170
|
+
2. Verify whether the selected label, result order, or dependent state changed.
|
|
171
|
+
3. Run `snapshot -i` after opening the control.
|
|
172
|
+
4. If the trigger clicked but options are absent from the accessibility tree, do not keep retrying `find` or stale refs.
|
|
173
|
+
5. Escalate to `fallback-operations.md#css-only-dropdown-javascript-fallback` for an atomic open-then-select script.
|
|
174
|
+
|
|
175
|
+
`snapshot -i` not showing an option does not prove the DOM node is missing. It may only prove the widget is not exposed through the accessibility tree.
|
|
176
|
+
|
|
164
177
|
### Fallback entry point
|
|
165
178
|
|
|
166
|
-
If native select/check fails, use `fallback-operations.md#selector-fallback
|
|
179
|
+
If native select/check fails, use `fallback-operations.md#selector-fallback`. Use JavaScript only if events must be synthesized or if evidence shows a CSS-only multi-step widget cannot be completed through semantic, snapshot, selector, DOM, or mouse tiers.
|
|
167
180
|
|
|
168
181
|
## Navigation
|
|
169
182
|
|
|
@@ -201,9 +214,9 @@ Other navigation commands:
|
|
|
201
214
|
|
|
202
215
|
```bash
|
|
203
216
|
OMNIBOT_SESSION_TOKEN=research omnibot navigate https://example.com
|
|
204
|
-
OMNIBOT_SESSION_TOKEN=research omnibot navigate --same-tab
|
|
217
|
+
OMNIBOT_SESSION_TOKEN=research omnibot navigate https://example.com --same-tab --tab-id <TAB_ID>
|
|
205
218
|
OMNIBOT_SESSION_TOKEN=research omnibot open https://example.com
|
|
206
|
-
OMNIBOT_SESSION_TOKEN=research omnibot goto https://example.com
|
|
219
|
+
OMNIBOT_SESSION_TOKEN=research omnibot goto https://example.com --tab-id <TAB_ID>
|
|
207
220
|
OMNIBOT_SESSION_TOKEN=research omnibot pushstate /dashboard --tab-id <TAB_ID>
|
|
208
221
|
OMNIBOT_SESSION_TOKEN=research omnibot close <TAB_ID>
|
|
209
222
|
OMNIBOT_SESSION_TOKEN=research omnibot tab list
|
|
@@ -261,7 +274,7 @@ OMNIBOT_SESSION_TOKEN=checkout omnibot wait --fn "window.appReady === true" --ta
|
|
|
261
274
|
|
|
262
275
|
### Verification
|
|
263
276
|
|
|
264
|
-
After waiting, read the final condition with `get`, `is`, `
|
|
277
|
+
After waiting, read the final condition with `get`, `is`, `wait`, or `snapshot`.
|
|
265
278
|
|
|
266
279
|
### Fallback entry point
|
|
267
280
|
|
|
@@ -277,7 +290,7 @@ Use Extraction for retrieving page text, HTML fragments, links, counts, assets,
|
|
|
277
290
|
|
|
278
291
|
1. `get text/html/attr/count` for targeted extraction.
|
|
279
292
|
2. `read --screens N` for clean page text, long pages, or lazy-loaded content.
|
|
280
|
-
3. `
|
|
293
|
+
3. `snapshot -i` for structured page observation before action planning.
|
|
281
294
|
4. `assets list/export` for resources.
|
|
282
295
|
5. `clipboard read` only when clipboard content is part of the task.
|
|
283
296
|
|
|
@@ -302,31 +315,30 @@ Verify extraction completeness with expected markers in `read`, element counts,
|
|
|
302
315
|
|
|
303
316
|
### Fallback entry point
|
|
304
317
|
|
|
305
|
-
If content exists only in runtime objects, use `fallback-operations.md#javascript-fallback` after explaining why `read`, `
|
|
318
|
+
If content exists only in runtime objects, use `fallback-operations.md#javascript-fallback` after explaining why `read`, `snapshot`, and `get` could not access it.
|
|
306
319
|
|
|
307
320
|
## Upload
|
|
308
321
|
|
|
309
322
|
### When to use
|
|
310
323
|
|
|
311
|
-
|
|
324
|
+
Native upload is currently unavailable in the extension transport. Do not use `omnibot upload` as an agent workflow step until file chooser support is implemented.
|
|
312
325
|
|
|
313
326
|
### Preferred sequence
|
|
314
327
|
|
|
315
|
-
1. Observe the
|
|
316
|
-
2.
|
|
317
|
-
3.
|
|
328
|
+
1. Observe the upload UI and confirm the task requires a local file.
|
|
329
|
+
2. Report that native upload is currently unavailable.
|
|
330
|
+
3. If the user still wants a workaround, ask for approval before using page-specific JavaScript or manual browser interaction.
|
|
318
331
|
|
|
319
332
|
### Example
|
|
320
333
|
|
|
321
334
|
```bash
|
|
322
335
|
OMNIBOT_SESSION_TOKEN=form omnibot snapshot -i --tab-id <TAB_ID>
|
|
323
|
-
|
|
324
|
-
OMNIBOT_SESSION_TOKEN=form omnibot wait --text "file.png" --tab-id <TAB_ID>
|
|
336
|
+
# upload is currently unavailable; do not run omnibot upload here.
|
|
325
337
|
```
|
|
326
338
|
|
|
327
339
|
### Verification
|
|
328
340
|
|
|
329
|
-
|
|
341
|
+
If upload support is implemented later, verify with visible filename, count, enabled submit state, or application-specific confirmation:
|
|
330
342
|
|
|
331
343
|
```bash
|
|
332
344
|
OMNIBOT_SESSION_TOKEN=form omnibot is enabled "button[type=submit]" --tab-id <TAB_ID>
|
|
@@ -334,13 +346,13 @@ OMNIBOT_SESSION_TOKEN=form omnibot is enabled "button[type=submit]" --tab-id <TA
|
|
|
334
346
|
|
|
335
347
|
### Fallback entry point
|
|
336
348
|
|
|
337
|
-
|
|
349
|
+
Do not treat JavaScript as a normal upload fallback. File inputs and browser file chooser behavior are security-sensitive and page-specific.
|
|
338
350
|
|
|
339
351
|
## Batch
|
|
340
352
|
|
|
341
353
|
### When to use
|
|
342
354
|
|
|
343
|
-
Use Batch for short, known-safe command chains where the target tab and verification are already explicit.
|
|
355
|
+
Use Batch for short, known-safe extension/CDP command chains where the target tab and verification are already explicit. Batch is not a wrapper for normal CLI commands such as `snapshot` or `click`.
|
|
344
356
|
|
|
345
357
|
### Preferred sequence
|
|
346
358
|
|
|
@@ -352,8 +364,8 @@ Use Batch for short, known-safe command chains where the target tab and verifica
|
|
|
352
364
|
### Example
|
|
353
365
|
|
|
354
366
|
```bash
|
|
355
|
-
OMNIBOT_SESSION_TOKEN=research omnibot batch '[{"cmd":"
|
|
356
|
-
OMNIBOT_SESSION_TOKEN=research omnibot
|
|
367
|
+
OMNIBOT_SESSION_TOKEN=research omnibot batch '[{"cmd":"cdp","method":"Runtime.evaluate","params":{"expression":"document.title","returnByValue":true}}]' --tab-id <TAB_ID>
|
|
368
|
+
OMNIBOT_SESSION_TOKEN=research omnibot get title --tab-id <TAB_ID>
|
|
357
369
|
```
|
|
358
370
|
|
|
359
371
|
File example:
|
|
@@ -1,17 +1,17 @@
|
|
|
1
|
-
#
|
|
1
|
+
# Workflow Context and Tabs
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
Workflow context tokens and tab ids are core reliability concepts, not advanced usage.
|
|
4
4
|
|
|
5
|
-
##
|
|
5
|
+
## Context First
|
|
6
6
|
|
|
7
|
-
`OMNIBOT_SESSION_TOKEN` is mandatory for every workflow.
|
|
7
|
+
`OMNIBOT_SESSION_TOKEN` is mandatory for every workflow. It is a workflow/context token, not a browser session, login session, or tab target.
|
|
8
8
|
|
|
9
9
|
```bash
|
|
10
10
|
OMNIBOT_SESSION_TOKEN=research omnibot tabs
|
|
11
11
|
OMNIBOT_SESSION_TOKEN=research omnibot snapshot -i --tab-id <TAB_ID>
|
|
12
12
|
```
|
|
13
13
|
|
|
14
|
-
The same token keeps one workflow isolated. Different tokens prevent independent agents from sharing
|
|
14
|
+
The same token keeps one workflow isolated. Different tokens prevent independent agents from sharing refs, trace, recording, temporary resources, and other workflow state.
|
|
15
15
|
|
|
16
16
|
## Tab Explicit
|
|
17
17
|
|
|
@@ -41,6 +41,7 @@ Safe sequence:
|
|
|
41
41
|
|
|
42
42
|
```bash
|
|
43
43
|
OMNIBOT_SESSION_TOKEN=research omnibot navigate https://example.com
|
|
44
|
+
# Save the returned full tab id exactly, for example edge-client:123.
|
|
44
45
|
OMNIBOT_SESSION_TOKEN=research omnibot snapshot -i --tab-id edge-client:123
|
|
45
46
|
OMNIBOT_SESSION_TOKEN=research omnibot click --tab-id edge-client:123 @e4
|
|
46
47
|
OMNIBOT_SESSION_TOKEN=research omnibot wait --text "Saved" --tab-id edge-client:123
|
|
@@ -68,10 +69,10 @@ Browser integration tests must only close tabs they created:
|
|
|
68
69
|
|
|
69
70
|
## Why Both
|
|
70
71
|
|
|
71
|
-
-
|
|
72
|
+
- Workflow context token = agent/workflow state isolation.
|
|
72
73
|
- Tab ID = page targeting.
|
|
73
74
|
|
|
74
|
-
|
|
75
|
+
The workflow token alone does not prove which page receives a command. Tab ID alone does not isolate concurrent agents. Use both.
|
|
75
76
|
|
|
76
77
|
## Ref Scope
|
|
77
78
|
|
|
@@ -102,7 +103,7 @@ OMNIBOT_SESSION_TOKEN=checkout
|
|
|
102
103
|
Agent C:
|
|
103
104
|
|
|
104
105
|
```bash
|
|
105
|
-
OMNIBOT_SESSION_TOKEN=debug
|
|
106
|
+
OMNIBOT_SESSION_TOKEN=debug-checkout
|
|
106
107
|
```
|
|
107
108
|
|
|
108
109
|
Each agent must:
|
|
@@ -143,13 +144,13 @@ Claiming does not replace `--tab-id`. It only documents or coordinates ownership
|
|
|
143
144
|
|
|
144
145
|
## visibility
|
|
145
146
|
|
|
146
|
-
Visibility controls where automation
|
|
147
|
+
Visibility controls where automation is intended to run:
|
|
147
148
|
|
|
148
149
|
- `visible`: controls the user's already-open real browser tabs and preserves visible session state.
|
|
149
150
|
- `background`: avoids foregrounding tabs unless requested.
|
|
150
|
-
- `dedicated-profile`:
|
|
151
|
-
- `headless`:
|
|
152
|
-
- `launch`:
|
|
151
|
+
- `dedicated-profile`: planned separate-profile mode.
|
|
152
|
+
- `headless`: planned headless browser context.
|
|
153
|
+
- `launch`: currently reports planned configuration only; it does not launch a browser.
|
|
153
154
|
|
|
154
155
|
Examples:
|
|
155
156
|
|
|
@@ -157,8 +158,7 @@ Examples:
|
|
|
157
158
|
omnibot visibility status
|
|
158
159
|
omnibot visibility set visible
|
|
159
160
|
omnibot visibility set background
|
|
160
|
-
omnibot visibility set headless --user-data-dir /tmp/omnibot-headless
|
|
161
161
|
omnibot visibility launch headless --user-data-dir /tmp/omnibot-headless
|
|
162
162
|
```
|
|
163
163
|
|
|
164
|
-
Headless and dedicated-profile modes do not automatically share the user's current tabs or login state.
|
|
164
|
+
Headless and dedicated-profile modes do not automatically share the user's current tabs or login state. Because `visibility launch` is currently planned-only, verify actual browser availability with `doctor` and `tabs` before assuming access.
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
package/package.json
CHANGED
|
@@ -1,29 +1,29 @@
|
|
|
1
|
-
{
|
|
2
|
-
"name": "@omniaibot/win-x64",
|
|
3
|
-
"version": "1.
|
|
4
|
-
"description": "omnibot executable for Windows x64",
|
|
5
|
-
"os": [
|
|
6
|
-
"win32"
|
|
7
|
-
],
|
|
8
|
-
"cpu": [
|
|
9
|
-
"x64"
|
|
10
|
-
],
|
|
11
|
-
"bin": {
|
|
12
|
-
"omnibot": "bin/omnibot-windows-x64/omnibot-windows-x64.exe"
|
|
13
|
-
},
|
|
14
|
-
"files": [
|
|
15
|
-
"bin/"
|
|
16
|
-
],
|
|
17
|
-
"keywords": [
|
|
18
|
-
"
|
|
19
|
-
"
|
|
20
|
-
"
|
|
21
|
-
"omnibot"
|
|
22
|
-
],
|
|
23
|
-
"author": "Unagi-cq",
|
|
24
|
-
"license": "MIT",
|
|
25
|
-
"repository": {
|
|
26
|
-
"type": "git",
|
|
27
|
-
"url": "git+https://github.com/DennisJcy/omnibot.git"
|
|
28
|
-
}
|
|
29
|
-
}
|
|
1
|
+
{
|
|
2
|
+
"name": "@omniaibot/win-x64",
|
|
3
|
+
"version": "1.3.0",
|
|
4
|
+
"description": "omnibot executable for Windows x64",
|
|
5
|
+
"os": [
|
|
6
|
+
"win32"
|
|
7
|
+
],
|
|
8
|
+
"cpu": [
|
|
9
|
+
"x64"
|
|
10
|
+
],
|
|
11
|
+
"bin": {
|
|
12
|
+
"omnibot": "bin/omnibot-windows-x64/omnibot-windows-x64.exe"
|
|
13
|
+
},
|
|
14
|
+
"files": [
|
|
15
|
+
"bin/"
|
|
16
|
+
],
|
|
17
|
+
"keywords": [
|
|
18
|
+
"browser",
|
|
19
|
+
"automation",
|
|
20
|
+
"agent",
|
|
21
|
+
"omnibot"
|
|
22
|
+
],
|
|
23
|
+
"author": "Unagi-cq",
|
|
24
|
+
"license": "MIT",
|
|
25
|
+
"repository": {
|
|
26
|
+
"type": "git",
|
|
27
|
+
"url": "git+https://github.com/DennisJcy/omnibot.git"
|
|
28
|
+
}
|
|
29
|
+
}
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|