@omniaibot/win-x64 1.1.6 → 1.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/bin/omnibot-windows-x64/81d243bd2c585b0f4821__mypyc.pyd +0 -0
- package/bin/omnibot-windows-x64/VERSION +1 -1
- package/bin/omnibot-windows-x64/_asyncio.pyd +0 -0
- package/bin/omnibot-windows-x64/_bz2.pyd +0 -0
- package/bin/omnibot-windows-x64/_cffi_backend.pyd +0 -0
- package/bin/omnibot-windows-x64/_ctypes.pyd +0 -0
- package/bin/omnibot-windows-x64/_decimal.pyd +0 -0
- package/bin/omnibot-windows-x64/_hashlib.pyd +0 -0
- package/bin/omnibot-windows-x64/_lzma.pyd +0 -0
- package/bin/omnibot-windows-x64/_multiprocessing.pyd +0 -0
- package/bin/omnibot-windows-x64/_overlapped.pyd +0 -0
- package/bin/omnibot-windows-x64/_queue.pyd +0 -0
- package/bin/omnibot-windows-x64/_socket.pyd +0 -0
- package/bin/omnibot-windows-x64/_ssl.pyd +0 -0
- package/bin/omnibot-windows-x64/_uuid.pyd +0 -0
- package/bin/omnibot-windows-x64/_wmi.pyd +0 -0
- package/bin/omnibot-windows-x64/charset_normalizer/cd.pyd +0 -0
- package/bin/omnibot-windows-x64/charset_normalizer/md.pyd +0 -0
- package/bin/omnibot-windows-x64/{libcrypto-3-x64.dll → libcrypto-3.dll} +0 -0
- package/bin/omnibot-windows-x64/libffi-8.dll +0 -0
- package/bin/omnibot-windows-x64/libssl-3.dll +0 -0
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/SKILL.md +16 -9
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/command-reference.md +44 -29
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/css-only-dropdown-js.md +50 -0
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/debugging-and-evidence.md +23 -21
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/fallback-operations.md +7 -52
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/operation-patterns.md +12 -13
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/session-and-tabs.md +14 -14
- package/bin/omnibot-windows-x64/omnibot-windows-x64.exe +0 -0
- package/bin/omnibot-windows-x64/pyexpat.pyd +0 -0
- package/bin/omnibot-windows-x64/python3.dll +0 -0
- package/bin/omnibot-windows-x64/python312.dll +0 -0
- package/bin/omnibot-windows-x64/select.pyd +0 -0
- package/bin/omnibot-windows-x64/unicodedata.pyd +0 -0
- package/bin/omnibot-windows-x64/vcruntime140.dll +0 -0
- package/bin/omnibot-windows-x64/vcruntime140_1.dll +0 -0
- package/package.json +29 -29
- package/bin/omnibot-windows-x64/ffi.dll +0 -0
- package/bin/omnibot-windows-x64/libbz2.dll +0 -0
- package/bin/omnibot-windows-x64/libexpat.dll +0 -0
- package/bin/omnibot-windows-x64/liblzma.dll +0 -0
- package/bin/omnibot-windows-x64/libmpdec-4.dll +0 -0
- package/bin/omnibot-windows-x64/libssl-3-x64.dll +0 -0
- package/bin/omnibot-windows-x64/omnibot/SKILL.md +0 -189
- package/bin/omnibot-windows-x64/omnibot/references/anti-patterns.md +0 -37
- package/bin/omnibot-windows-x64/omnibot/references/command-reference.md +0 -605
- package/bin/omnibot-windows-x64/omnibot/references/debugging-and-evidence.md +0 -95
- package/bin/omnibot-windows-x64/omnibot/references/fallback-operations.md +0 -216
- package/bin/omnibot-windows-x64/omnibot/references/operation-patterns.md +0 -385
- package/bin/omnibot-windows-x64/omnibot/references/runtime-and-status.md +0 -111
- package/bin/omnibot-windows-x64/omnibot/references/session-and-tabs.md +0 -164
- package/bin/omnibot-windows-x64/python313.dll +0 -0
- package/bin/omnibot-windows-x64/zlib.dll +0 -0
|
@@ -1,216 +0,0 @@
|
|
|
1
|
-
# Fallback Operations
|
|
2
|
-
|
|
3
|
-
Fallback is for completing operations after preferred patterns fail. It is not debugging. Debugging collects evidence; fallback changes page state.
|
|
4
|
-
|
|
5
|
-
Every fallback escalation must answer two questions:
|
|
6
|
-
|
|
7
|
-
- Which higher tier failed, and why?
|
|
8
|
-
- How will the result be verified?
|
|
9
|
-
|
|
10
|
-
## Fallback Tier Model
|
|
11
|
-
|
|
12
|
-
### Tier 1 Semantic
|
|
13
|
-
|
|
14
|
-
Use semantic intent first:
|
|
15
|
-
|
|
16
|
-
```bash
|
|
17
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot find role button --name "Submit" --action click --tab-id <TAB_ID>
|
|
18
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot find text "Sign in" --action click --tab-id <TAB_ID>
|
|
19
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot find label "Email" --action fill --action-value "a@b.com" --tab-id <TAB_ID>
|
|
20
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot find placeholder "Search" --action type --action-value "omnibot" --tab-id <TAB_ID>
|
|
21
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot find testid submit-button --action click --tab-id <TAB_ID>
|
|
22
|
-
```
|
|
23
|
-
|
|
24
|
-
Verify:
|
|
25
|
-
|
|
26
|
-
```bash
|
|
27
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot snapshot -i --tab-id <TAB_ID>
|
|
28
|
-
```
|
|
29
|
-
|
|
30
|
-
### Tier 2 Snapshot Refs
|
|
31
|
-
|
|
32
|
-
Use refs when semantic targeting is missing or ambiguous.
|
|
33
|
-
|
|
34
|
-
```bash
|
|
35
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot snapshot -i --tab-id <TAB_ID>
|
|
36
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot click --tab-id <TAB_ID> @e4
|
|
37
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot snapshot -i --tab-id <TAB_ID>
|
|
38
|
-
```
|
|
39
|
-
|
|
40
|
-
Refs are tab-scoped. Never reuse `@eN` across tabs or after large DOM changes without a fresh snapshot.
|
|
41
|
-
|
|
42
|
-
## Selector Fallback
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
### Tier 3 Selector
|
|
46
|
-
|
|
47
|
-
Use selectors only after semantic find and refs are unavailable or unreliable.
|
|
48
|
-
|
|
49
|
-
```bash
|
|
50
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot click "button[type=submit]" --tab-id <TAB_ID>
|
|
51
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot fill "input[name=email]" "a@b.com" --tab-id <TAB_ID>
|
|
52
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot focus "input[name=email]" --tab-id <TAB_ID>
|
|
53
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot get value "input[name=email]" --tab-id <TAB_ID>
|
|
54
|
-
```
|
|
55
|
-
|
|
56
|
-
Verify with `get`, `is`, `wait`, or `snapshot`.
|
|
57
|
-
|
|
58
|
-
## DOM Fallback
|
|
59
|
-
|
|
60
|
-
### Tier 4 DOM
|
|
61
|
-
|
|
62
|
-
Use DOM node fallback when selectors exist but standard commands fail due overlays, custom event routing, or unstable selectors.
|
|
63
|
-
|
|
64
|
-
```bash
|
|
65
|
-
OMNIBOT_SESSION_TOKEN=repair omnibot dom visible --tab-id <TAB_ID>
|
|
66
|
-
OMNIBOT_SESSION_TOKEN=repair omnibot dom click n1 --tab-id <TAB_ID>
|
|
67
|
-
OMNIBOT_SESSION_TOKEN=repair omnibot dom dblclick n1 --tab-id <TAB_ID>
|
|
68
|
-
OMNIBOT_SESSION_TOKEN=repair omnibot dom scroll n1 --dy 800 --tab-id <TAB_ID>
|
|
69
|
-
OMNIBOT_SESSION_TOKEN=repair omnibot snapshot -i --tab-id <TAB_ID>
|
|
70
|
-
```
|
|
71
|
-
|
|
72
|
-
Do not use `dom` as the first click path. It bypasses higher-level intent and can target the wrong node if the visible-node list changes.
|
|
73
|
-
|
|
74
|
-
## Mouse Fallback
|
|
75
|
-
|
|
76
|
-
### Tier 5 Mouse
|
|
77
|
-
|
|
78
|
-
Use coordinate mouse operations only when element-based actions cannot reach the target.
|
|
79
|
-
|
|
80
|
-
```bash
|
|
81
|
-
OMNIBOT_SESSION_TOKEN=repair omnibot get box "#canvas-button" --tab-id <TAB_ID>
|
|
82
|
-
OMNIBOT_SESSION_TOKEN=repair omnibot mouse click --x 100 --y 200 --tab-id <TAB_ID>
|
|
83
|
-
OMNIBOT_SESSION_TOKEN=repair omnibot snapshot -i --tab-id <TAB_ID>
|
|
84
|
-
```
|
|
85
|
-
|
|
86
|
-
Other mouse operations:
|
|
87
|
-
|
|
88
|
-
```bash
|
|
89
|
-
OMNIBOT_SESSION_TOKEN=repair omnibot mouse move --x 50 --y 60 --tab-id <TAB_ID>
|
|
90
|
-
OMNIBOT_SESSION_TOKEN=repair omnibot mouse scroll --x 100 --y 200 --dy 500 --tab-id <TAB_ID>
|
|
91
|
-
OMNIBOT_SESSION_TOKEN=repair omnibot mouse drag --from-x 10 --from-y 20 --to-x 100 --to-y 200 --tab-id <TAB_ID>
|
|
92
|
-
```
|
|
93
|
-
|
|
94
|
-
Mouse fallback is fragile across viewport size, scroll position, and responsive layouts. Verify immediately.
|
|
95
|
-
|
|
96
|
-
## JavaScript Fallback
|
|
97
|
-
|
|
98
|
-
### Tier 6 JavaScript
|
|
99
|
-
|
|
100
|
-
Do not use `execute-js` first.
|
|
101
|
-
|
|
102
|
-
Use `execute-js` only when:
|
|
103
|
-
|
|
104
|
-
- Semantic find fails.
|
|
105
|
-
- Snapshot refs are unstable.
|
|
106
|
-
- CSS selector operation fails.
|
|
107
|
-
- DOM fallback fails.
|
|
108
|
-
- A complex frontend event must be triggered.
|
|
109
|
-
- A CSS-only dropdown or hover menu requires an atomic open-then-select sequence, and `snapshot -i` confirms the option nodes are absent from the accessibility tree.
|
|
110
|
-
- Runtime state cannot be read through `read`, `snapshot`, `get`, or `is`.
|
|
111
|
-
|
|
112
|
-
Read-only JavaScript fallback:
|
|
113
|
-
|
|
114
|
-
```bash
|
|
115
|
-
OMNIBOT_SESSION_TOKEN=repair omnibot execute-js "return window.appState?.status" --tab-id <TAB_ID>
|
|
116
|
-
OMNIBOT_SESSION_TOKEN=repair omnibot get text "#status" --tab-id <TAB_ID>
|
|
117
|
-
```
|
|
118
|
-
|
|
119
|
-
Input JavaScript fallback must trigger real browser events:
|
|
120
|
-
|
|
121
|
-
```bash
|
|
122
|
-
OMNIBOT_SESSION_TOKEN=repair omnibot execute-js "const el=document.querySelector('input[name=email]'); el.value='a@b.com'; el.dispatchEvent(new Event('input',{bubbles:true})); el.dispatchEvent(new Event('change',{bubbles:true})); return el.value;" --tab-id <TAB_ID>
|
|
123
|
-
OMNIBOT_SESSION_TOKEN=repair omnibot get value "input[name=email]" --tab-id <TAB_ID>
|
|
124
|
-
```
|
|
125
|
-
|
|
126
|
-
Click JavaScript fallback should call the element click only after higher tiers fail:
|
|
127
|
-
|
|
128
|
-
```bash
|
|
129
|
-
OMNIBOT_SESSION_TOKEN=repair omnibot execute-js "document.querySelector('button[type=submit]')?.click(); return true;" --tab-id <TAB_ID>
|
|
130
|
-
OMNIBOT_SESSION_TOKEN=repair omnibot wait --url "/dashboard" --tab-id <TAB_ID>
|
|
131
|
-
```
|
|
132
|
-
|
|
133
|
-
### CSS-only dropdown JavaScript fallback
|
|
134
|
-
|
|
135
|
-
Use this only after the trigger is known and higher tiers failed: semantic `find` clicked the trigger or found the text, `snapshot -i` could not expose option refs, selector/DOM/mouse could not complete the two-step interaction, and verification showed the selection did not change.
|
|
136
|
-
|
|
137
|
-
Example for a visible trigger like `按留言时间排序` and a target option text:
|
|
138
|
-
|
|
139
|
-
```bash
|
|
140
|
-
OMNIBOT_SESSION_TOKEN=repair omnibot execute-js --file /tmp/omnibot-css-dropdown.js --tab-id <TAB_ID>
|
|
141
|
-
OMNIBOT_SESSION_TOKEN=repair omnibot snapshot -i --tab-id <TAB_ID>
|
|
142
|
-
```
|
|
143
|
-
|
|
144
|
-
`/tmp/omnibot-css-dropdown.js`:
|
|
145
|
-
|
|
146
|
-
```js
|
|
147
|
-
const triggerText = '按留言时间排序';
|
|
148
|
-
const optionText = '目标选项文本';
|
|
149
|
-
|
|
150
|
-
const visibleElements = [...document.querySelectorAll('button, [role="button"], [aria-haspopup], div, span, li')]
|
|
151
|
-
.filter((el) => {
|
|
152
|
-
const style = getComputedStyle(el);
|
|
153
|
-
const box = el.getBoundingClientRect();
|
|
154
|
-
return style.display !== 'none' && style.visibility !== 'hidden' && box.width > 0 && box.height > 0;
|
|
155
|
-
});
|
|
156
|
-
|
|
157
|
-
const byExactText = (text) => visibleElements.find((el) => el.textContent?.trim() === text);
|
|
158
|
-
const trigger = byExactText(triggerText);
|
|
159
|
-
if (!trigger) {
|
|
160
|
-
return { ok: false, reason: 'trigger-not-found', triggerText, optionText };
|
|
161
|
-
}
|
|
162
|
-
|
|
163
|
-
trigger.click();
|
|
164
|
-
await new Promise((resolve) => requestAnimationFrame(() => requestAnimationFrame(resolve)));
|
|
165
|
-
|
|
166
|
-
const option = [...document.querySelectorAll('div, span, li, button, [role="option"], [role="menuitem"]')]
|
|
167
|
-
.filter((el) => {
|
|
168
|
-
const style = getComputedStyle(el);
|
|
169
|
-
const box = el.getBoundingClientRect();
|
|
170
|
-
return style.display !== 'none' && style.visibility !== 'hidden' && box.width > 0 && box.height > 0;
|
|
171
|
-
})
|
|
172
|
-
.find((el) => el.textContent?.trim() === optionText);
|
|
173
|
-
|
|
174
|
-
if (!option) {
|
|
175
|
-
return { ok: false, reason: 'option-not-found-after-trigger-click', triggerText, optionText };
|
|
176
|
-
}
|
|
177
|
-
|
|
178
|
-
option.click();
|
|
179
|
-
await new Promise((resolve) => requestAnimationFrame(() => requestAnimationFrame(resolve)));
|
|
180
|
-
|
|
181
|
-
return {
|
|
182
|
-
ok: true,
|
|
183
|
-
triggerText,
|
|
184
|
-
optionText,
|
|
185
|
-
selectedText: trigger.textContent?.trim(),
|
|
186
|
-
};
|
|
187
|
-
```
|
|
188
|
-
|
|
189
|
-
Verify with the user-visible result, not just `{ ok: true }`: selected label, sorted result order, result count, URL/query state, or another page-specific condition.
|
|
190
|
-
|
|
191
|
-
Prefer `--file` for longer scripts:
|
|
192
|
-
|
|
193
|
-
```bash
|
|
194
|
-
OMNIBOT_SESSION_TOKEN=repair omnibot execute-js --file /tmp/omnibot-repair.js --tab-id <TAB_ID>
|
|
195
|
-
OMNIBOT_SESSION_TOKEN=repair omnibot snapshot -i --tab-id <TAB_ID>
|
|
196
|
-
```
|
|
197
|
-
|
|
198
|
-
## Raw CDP Fallback
|
|
199
|
-
|
|
200
|
-
### Tier 7 Raw CDP
|
|
201
|
-
|
|
202
|
-
Raw CDP is the last resort. Use it only when standard commands and JavaScript fallback cannot complete or inspect the task.
|
|
203
|
-
|
|
204
|
-
```bash
|
|
205
|
-
OMNIBOT_SESSION_TOKEN=debug omnibot cdp Runtime.evaluate '{"expression":"document.title"}' --tab-id <TAB_ID>
|
|
206
|
-
OMNIBOT_SESSION_TOKEN=debug omnibot cdp DOM.getDocument '{"depth":1}' --tab-id <TAB_ID>
|
|
207
|
-
```
|
|
208
|
-
|
|
209
|
-
After CDP fallback, verify using normal Omnibot reads whenever possible:
|
|
210
|
-
|
|
211
|
-
```bash
|
|
212
|
-
OMNIBOT_SESSION_TOKEN=debug omnibot get title --tab-id <TAB_ID>
|
|
213
|
-
OMNIBOT_SESSION_TOKEN=debug omnibot snapshot -i --tab-id <TAB_ID>
|
|
214
|
-
```
|
|
215
|
-
|
|
216
|
-
CDP can bypass application-level event semantics. Treat it as expert-only infrastructure access.
|
|
@@ -1,385 +0,0 @@
|
|
|
1
|
-
# Operation Patterns
|
|
2
|
-
|
|
3
|
-
Use this file to choose behavior by task. Every page-state workflow must set `OMNIBOT_SESSION_TOKEN=<workflow-name>` and pass `--tab-id <TAB_ID>` on each page-state command.
|
|
4
|
-
|
|
5
|
-
## Read
|
|
6
|
-
|
|
7
|
-
### When to use
|
|
8
|
-
|
|
9
|
-
Use Read when the agent needs clean rendered page content: article text, search results, feed items, long pages, lazy-loaded text, or a human-readable summary source.
|
|
10
|
-
|
|
11
|
-
Use targeted state commands when the agent needs one value, visibility, enabled/checked state, layout box, or computed styles. Use `snapshot -i` when the agent needs current visible UI structure for deciding the next action.
|
|
12
|
-
|
|
13
|
-
### Preferred sequence
|
|
14
|
-
|
|
15
|
-
1. `read --tab-id <TAB_ID>` for clean page text on an existing tab.
|
|
16
|
-
2. `read --screens N --tab-id <TAB_ID>` for longer or lazy-loaded pages.
|
|
17
|
-
3. `read <URL>` when opening a temporary tab only for reading that URL.
|
|
18
|
-
4. `get title/url/text/html/value/attr/count/box/styles` for narrow evidence.
|
|
19
|
-
5. `is visible/enabled/checked` for boolean state.
|
|
20
|
-
5. `snapshot -i` when actionable refs are needed.
|
|
21
|
-
6. `dom visible` or `execute-js` fallback only after standard reads cannot access the state.
|
|
22
|
-
|
|
23
|
-
### Example
|
|
24
|
-
|
|
25
|
-
```bash
|
|
26
|
-
OMNIBOT_SESSION_TOKEN=research omnibot read --screens 5 --tab-id <TAB_ID>
|
|
27
|
-
OMNIBOT_SESSION_TOKEN=research omnibot read --screens 3 https://example.com/article
|
|
28
|
-
```
|
|
29
|
-
|
|
30
|
-
### Verification
|
|
31
|
-
|
|
32
|
-
Use the narrowest read that proves the claim:
|
|
33
|
-
|
|
34
|
-
```bash
|
|
35
|
-
OMNIBOT_SESSION_TOKEN=research omnibot get title --tab-id <TAB_ID>
|
|
36
|
-
OMNIBOT_SESSION_TOKEN=research omnibot is visible "#main" --tab-id <TAB_ID>
|
|
37
|
-
OMNIBOT_SESSION_TOKEN=research omnibot get count ".result" --tab-id <TAB_ID>
|
|
38
|
-
```
|
|
39
|
-
|
|
40
|
-
### Fallback entry point
|
|
41
|
-
|
|
42
|
-
If `read` or targeted reads omit needed runtime state, go to `fallback-operations.md#fallback-tier-model`. Start with DOM fallback before JavaScript.
|
|
43
|
-
|
|
44
|
-
## Click
|
|
45
|
-
|
|
46
|
-
### When to use
|
|
47
|
-
|
|
48
|
-
Use Click for buttons, links, menu items, cards, checkouts, modal actions, and any user-like click target.
|
|
49
|
-
|
|
50
|
-
### Preferred sequence
|
|
51
|
-
|
|
52
|
-
1. `find role/text/testid --action click`
|
|
53
|
-
2. `snapshot -i` -> `click @eN`
|
|
54
|
-
3. `click selector`
|
|
55
|
-
4. `dom click`
|
|
56
|
-
5. `mouse click`
|
|
57
|
-
6. `execute-js` click
|
|
58
|
-
7. `cdp` fallback
|
|
59
|
-
|
|
60
|
-
### Example
|
|
61
|
-
|
|
62
|
-
```bash
|
|
63
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot snapshot -i --tab-id <TAB_ID>
|
|
64
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot find role button --name "Submit" --action click --tab-id <TAB_ID>
|
|
65
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot wait --url "/dashboard" --tab-id <TAB_ID>
|
|
66
|
-
```
|
|
67
|
-
|
|
68
|
-
If semantic find is ambiguous, use refs from the same tab:
|
|
69
|
-
|
|
70
|
-
```bash
|
|
71
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot snapshot -i --tab-id <TAB_ID>
|
|
72
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot click --tab-id <TAB_ID> @e4
|
|
73
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot snapshot -i --tab-id <TAB_ID>
|
|
74
|
-
```
|
|
75
|
-
|
|
76
|
-
### Verification
|
|
77
|
-
|
|
78
|
-
Verify the expected state, not just command success:
|
|
79
|
-
|
|
80
|
-
```bash
|
|
81
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot get url --tab-id <TAB_ID>
|
|
82
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot is visible ".success" --tab-id <TAB_ID>
|
|
83
|
-
```
|
|
84
|
-
|
|
85
|
-
### Fallback entry point
|
|
86
|
-
|
|
87
|
-
If semantic and refs fail, document why and continue at `fallback-operations.md#selector-fallback`.
|
|
88
|
-
|
|
89
|
-
## Fill
|
|
90
|
-
|
|
91
|
-
### When to use
|
|
92
|
-
|
|
93
|
-
Use Fill for text inputs, textareas, search boxes, login fields, and form fields where the final value must be controlled.
|
|
94
|
-
|
|
95
|
-
### Preferred sequence
|
|
96
|
-
|
|
97
|
-
1. `find label/placeholder --action fill`
|
|
98
|
-
2. `snapshot -i` -> `fill @eN`
|
|
99
|
-
3. `fill selector`
|
|
100
|
-
4. `focus` + `type`
|
|
101
|
-
5. `execute-js` set value and dispatch `input`/`change` events
|
|
102
|
-
|
|
103
|
-
### Example
|
|
104
|
-
|
|
105
|
-
```bash
|
|
106
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot find label "Email" --action fill --action-value "a@b.com" --tab-id <TAB_ID>
|
|
107
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot get value "input[name=email]" --tab-id <TAB_ID>
|
|
108
|
-
```
|
|
109
|
-
|
|
110
|
-
With refs:
|
|
111
|
-
|
|
112
|
-
```bash
|
|
113
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot snapshot -i --tab-id <TAB_ID>
|
|
114
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot fill --tab-id <TAB_ID> @e2 "a@b.com"
|
|
115
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot get value "input[name=email]" --tab-id <TAB_ID>
|
|
116
|
-
```
|
|
117
|
-
|
|
118
|
-
### Verification
|
|
119
|
-
|
|
120
|
-
Use `get value`, visible validation text, enabled state, or submit readiness:
|
|
121
|
-
|
|
122
|
-
```bash
|
|
123
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot get value "input[name=email]" --tab-id <TAB_ID>
|
|
124
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot is enabled "button[type=submit]" --tab-id <TAB_ID>
|
|
125
|
-
```
|
|
126
|
-
|
|
127
|
-
### Fallback entry point
|
|
128
|
-
|
|
129
|
-
If fill does not update the application state, use `fallback-operations.md#javascript-fallback` only after selector/focus/type attempts fail. JavaScript must dispatch real `input` and `change` events.
|
|
130
|
-
|
|
131
|
-
## Select / Check
|
|
132
|
-
|
|
133
|
-
### When to use
|
|
134
|
-
|
|
135
|
-
Use this pattern for dropdowns, checkboxes, toggles, agreement boxes, filters, and boolean controls.
|
|
136
|
-
|
|
137
|
-
### Preferred sequence
|
|
138
|
-
|
|
139
|
-
1. Observe state with `snapshot -i` and `is checked` when relevant.
|
|
140
|
-
2. Use `select`, `check`, or `uncheck` with refs or selectors.
|
|
141
|
-
3. Verify with `get value`, `is checked`, visible text, or result count.
|
|
142
|
-
|
|
143
|
-
### Example
|
|
144
|
-
|
|
145
|
-
```bash
|
|
146
|
-
OMNIBOT_SESSION_TOKEN=form omnibot snapshot -i --tab-id <TAB_ID>
|
|
147
|
-
OMNIBOT_SESSION_TOKEN=form omnibot select @e5 "US" --tab-id <TAB_ID>
|
|
148
|
-
OMNIBOT_SESSION_TOKEN=form omnibot get value "select[name=country]" --tab-id <TAB_ID>
|
|
149
|
-
```
|
|
150
|
-
|
|
151
|
-
Checkbox example:
|
|
152
|
-
|
|
153
|
-
```bash
|
|
154
|
-
OMNIBOT_SESSION_TOKEN=form omnibot is checked "#agree" --tab-id <TAB_ID>
|
|
155
|
-
OMNIBOT_SESSION_TOKEN=form omnibot check "#agree" --tab-id <TAB_ID>
|
|
156
|
-
OMNIBOT_SESSION_TOKEN=form omnibot is checked "#agree" --tab-id <TAB_ID>
|
|
157
|
-
```
|
|
158
|
-
|
|
159
|
-
### Verification
|
|
160
|
-
|
|
161
|
-
For checkboxes, verify boolean state. For selects, verify value or dependent page state.
|
|
162
|
-
|
|
163
|
-
### CSS-only dropdowns and custom menus
|
|
164
|
-
|
|
165
|
-
Some dropdowns, hover menus, tooltips, and custom selects are CSS-only widgets. The visible trigger may be accessible, while the option nodes are hidden `<div>` elements without `role`, accessible name, or stable refs.
|
|
166
|
-
|
|
167
|
-
Use this evidence path before JavaScript:
|
|
168
|
-
|
|
169
|
-
1. Try semantic `find` or a normal `select`/`click` against the trigger.
|
|
170
|
-
2. Verify whether the selected label, result order, or dependent state changed.
|
|
171
|
-
3. Run `snapshot -i` after opening the control.
|
|
172
|
-
4. If the trigger clicked but options are absent from the accessibility tree, do not keep retrying `find` or stale refs.
|
|
173
|
-
5. Escalate to `fallback-operations.md#css-only-dropdown-javascript-fallback` for an atomic open-then-select script.
|
|
174
|
-
|
|
175
|
-
`snapshot -i` not showing an option does not prove the DOM node is missing. It may only prove the widget is not exposed through the accessibility tree.
|
|
176
|
-
|
|
177
|
-
### Fallback entry point
|
|
178
|
-
|
|
179
|
-
If native select/check fails, use `fallback-operations.md#selector-fallback`. Use JavaScript only if events must be synthesized or if evidence shows a CSS-only multi-step widget cannot be completed through semantic, snapshot, selector, DOM, or mouse tiers.
|
|
180
|
-
|
|
181
|
-
## Navigation
|
|
182
|
-
|
|
183
|
-
### When to use
|
|
184
|
-
|
|
185
|
-
Use Navigation to open pages, create tabs, change history, reload, close tabs, move between pages, create windows, or select frames.
|
|
186
|
-
|
|
187
|
-
### Preferred sequence
|
|
188
|
-
|
|
189
|
-
1. Create or discover a tab with `tabs`, `tab list`, `tab new`, `open`, or `navigate`.
|
|
190
|
-
2. Save the returned tab id.
|
|
191
|
-
3. Verify with `get url --tab-id <TAB_ID>`.
|
|
192
|
-
4. Run all later page-state commands with `--tab-id <TAB_ID>`.
|
|
193
|
-
|
|
194
|
-
### Example
|
|
195
|
-
|
|
196
|
-
```bash
|
|
197
|
-
OMNIBOT_SESSION_TOKEN=research omnibot tab new https://example.com --label research
|
|
198
|
-
OMNIBOT_SESSION_TOKEN=research omnibot get url --tab-id <TAB_ID>
|
|
199
|
-
OMNIBOT_SESSION_TOKEN=research omnibot snapshot -i --tab-id <TAB_ID>
|
|
200
|
-
```
|
|
201
|
-
|
|
202
|
-
History and lifecycle:
|
|
203
|
-
|
|
204
|
-
```bash
|
|
205
|
-
OMNIBOT_SESSION_TOKEN=research omnibot back --tab-id <TAB_ID>
|
|
206
|
-
OMNIBOT_SESSION_TOKEN=research omnibot get url --tab-id <TAB_ID>
|
|
207
|
-
OMNIBOT_SESSION_TOKEN=research omnibot forward --tab-id <TAB_ID>
|
|
208
|
-
OMNIBOT_SESSION_TOKEN=research omnibot get url --tab-id <TAB_ID>
|
|
209
|
-
OMNIBOT_SESSION_TOKEN=research omnibot reload --tab-id <TAB_ID>
|
|
210
|
-
OMNIBOT_SESSION_TOKEN=research omnibot wait --load domcontentloaded --tab-id <TAB_ID>
|
|
211
|
-
```
|
|
212
|
-
|
|
213
|
-
Other navigation commands:
|
|
214
|
-
|
|
215
|
-
```bash
|
|
216
|
-
OMNIBOT_SESSION_TOKEN=research omnibot navigate https://example.com
|
|
217
|
-
OMNIBOT_SESSION_TOKEN=research omnibot navigate --same-tab https://example.com
|
|
218
|
-
OMNIBOT_SESSION_TOKEN=research omnibot open https://example.com
|
|
219
|
-
OMNIBOT_SESSION_TOKEN=research omnibot goto https://example.com
|
|
220
|
-
OMNIBOT_SESSION_TOKEN=research omnibot pushstate /dashboard --tab-id <TAB_ID>
|
|
221
|
-
OMNIBOT_SESSION_TOKEN=research omnibot close <TAB_ID>
|
|
222
|
-
OMNIBOT_SESSION_TOKEN=research omnibot tab list
|
|
223
|
-
OMNIBOT_SESSION_TOKEN=research omnibot tab close research
|
|
224
|
-
OMNIBOT_SESSION_TOKEN=research omnibot window new
|
|
225
|
-
OMNIBOT_SESSION_TOKEN=research omnibot frame main
|
|
226
|
-
```
|
|
227
|
-
|
|
228
|
-
### Verification
|
|
229
|
-
|
|
230
|
-
Always verify the final target:
|
|
231
|
-
|
|
232
|
-
```bash
|
|
233
|
-
OMNIBOT_SESSION_TOKEN=research omnibot get url --tab-id <TAB_ID>
|
|
234
|
-
OMNIBOT_SESSION_TOKEN=research omnibot wait --load networkidle --tab-id <TAB_ID>
|
|
235
|
-
OMNIBOT_SESSION_TOKEN=research omnibot snapshot -i --tab-id <TAB_ID>
|
|
236
|
-
```
|
|
237
|
-
|
|
238
|
-
### Fallback entry point
|
|
239
|
-
|
|
240
|
-
Same-tab navigation must include `--tab-id <TAB_ID>`. If the target tab id is unknown, run `omnibot tabs` first and choose the intended tab explicitly.
|
|
241
|
-
|
|
242
|
-
## Wait
|
|
243
|
-
|
|
244
|
-
### When to use
|
|
245
|
-
|
|
246
|
-
Use Wait after navigation, clicks, form submissions, reloads, dynamic rendering, lazy loading, or any action that changes asynchronous state.
|
|
247
|
-
|
|
248
|
-
### Preferred sequence
|
|
249
|
-
|
|
250
|
-
1. Wait for selector visibility or hidden state.
|
|
251
|
-
2. Wait for text.
|
|
252
|
-
3. Wait for URL.
|
|
253
|
-
4. Wait for load state.
|
|
254
|
-
5. Wait for a JavaScript condition with `--fn`.
|
|
255
|
-
6. Avoid shell sleep.
|
|
256
|
-
|
|
257
|
-
### Example
|
|
258
|
-
|
|
259
|
-
```bash
|
|
260
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot click --tab-id <TAB_ID> @e4
|
|
261
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot wait "#spinner" --state hidden --tab-id <TAB_ID>
|
|
262
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot wait --text "Welcome" --tab-id <TAB_ID>
|
|
263
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot get text "#main" --tab-id <TAB_ID>
|
|
264
|
-
```
|
|
265
|
-
|
|
266
|
-
Other waits:
|
|
267
|
-
|
|
268
|
-
```bash
|
|
269
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot wait --url "/dashboard" --tab-id <TAB_ID>
|
|
270
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot wait --load domcontentloaded --tab-id <TAB_ID>
|
|
271
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot wait --load networkidle --tab-id <TAB_ID>
|
|
272
|
-
OMNIBOT_SESSION_TOKEN=checkout omnibot wait --fn "window.appReady === true" --tab-id <TAB_ID>
|
|
273
|
-
```
|
|
274
|
-
|
|
275
|
-
### Verification
|
|
276
|
-
|
|
277
|
-
After waiting, read the final condition with `get`, `is`, `wait`, or `snapshot`.
|
|
278
|
-
|
|
279
|
-
### Fallback entry point
|
|
280
|
-
|
|
281
|
-
If waits time out, collect evidence with `debugging-and-evidence.md`; do not replace them with shell sleep.
|
|
282
|
-
|
|
283
|
-
## Extraction
|
|
284
|
-
|
|
285
|
-
### When to use
|
|
286
|
-
|
|
287
|
-
Use Extraction for retrieving page text, HTML fragments, links, counts, assets, or clipboard data.
|
|
288
|
-
|
|
289
|
-
### Preferred sequence
|
|
290
|
-
|
|
291
|
-
1. `get text/html/attr/count` for targeted extraction.
|
|
292
|
-
2. `read --screens N` for clean page text, long pages, or lazy-loaded content.
|
|
293
|
-
3. `snapshot -i` for structured page observation before action planning.
|
|
294
|
-
4. `assets list/export` for resources.
|
|
295
|
-
5. `clipboard read` only when clipboard content is part of the task.
|
|
296
|
-
|
|
297
|
-
### Example
|
|
298
|
-
|
|
299
|
-
```bash
|
|
300
|
-
OMNIBOT_SESSION_TOKEN=extract omnibot get text "#article" --tab-id <TAB_ID>
|
|
301
|
-
OMNIBOT_SESSION_TOKEN=extract omnibot get attr "a.next" href --tab-id <TAB_ID>
|
|
302
|
-
OMNIBOT_SESSION_TOKEN=extract omnibot get count ".result" --tab-id <TAB_ID>
|
|
303
|
-
```
|
|
304
|
-
|
|
305
|
-
Long page example:
|
|
306
|
-
|
|
307
|
-
```bash
|
|
308
|
-
OMNIBOT_SESSION_TOKEN=extract omnibot read --screens 5 --tab-id <TAB_ID>
|
|
309
|
-
OMNIBOT_SESSION_TOKEN=extract omnibot assets list --tab-id <TAB_ID>
|
|
310
|
-
```
|
|
311
|
-
|
|
312
|
-
### Verification
|
|
313
|
-
|
|
314
|
-
Verify extraction completeness with expected markers in `read`, element counts, or targeted reads for known selectors.
|
|
315
|
-
|
|
316
|
-
### Fallback entry point
|
|
317
|
-
|
|
318
|
-
If content exists only in runtime objects, use `fallback-operations.md#javascript-fallback` after explaining why `read`, `snapshot`, and `get` could not access it.
|
|
319
|
-
|
|
320
|
-
## Upload
|
|
321
|
-
|
|
322
|
-
### When to use
|
|
323
|
-
|
|
324
|
-
Use Upload when a browser form needs a local file attached through a file input.
|
|
325
|
-
|
|
326
|
-
### Preferred sequence
|
|
327
|
-
|
|
328
|
-
1. Observe the file input or upload control.
|
|
329
|
-
2. Use `upload` against the input selector.
|
|
330
|
-
3. Verify that the uploaded file appeared or form state changed.
|
|
331
|
-
|
|
332
|
-
### Example
|
|
333
|
-
|
|
334
|
-
```bash
|
|
335
|
-
OMNIBOT_SESSION_TOKEN=form omnibot snapshot -i --tab-id <TAB_ID>
|
|
336
|
-
OMNIBOT_SESSION_TOKEN=form omnibot upload "input[type=file]" /path/to/file.png --tab-id <TAB_ID>
|
|
337
|
-
OMNIBOT_SESSION_TOKEN=form omnibot wait --text "file.png" --tab-id <TAB_ID>
|
|
338
|
-
```
|
|
339
|
-
|
|
340
|
-
### Verification
|
|
341
|
-
|
|
342
|
-
Use visible filename, count, enabled submit state, or application-specific confirmation:
|
|
343
|
-
|
|
344
|
-
```bash
|
|
345
|
-
OMNIBOT_SESSION_TOKEN=form omnibot is enabled "button[type=submit]" --tab-id <TAB_ID>
|
|
346
|
-
```
|
|
347
|
-
|
|
348
|
-
### Fallback entry point
|
|
349
|
-
|
|
350
|
-
If upload input is hidden behind custom UI, click the semantic upload control first. Use JavaScript only after standard upload cannot reach the input.
|
|
351
|
-
|
|
352
|
-
## Batch
|
|
353
|
-
|
|
354
|
-
### When to use
|
|
355
|
-
|
|
356
|
-
Use Batch for short, known-safe command chains where the target tab and verification are already explicit.
|
|
357
|
-
|
|
358
|
-
### Preferred sequence
|
|
359
|
-
|
|
360
|
-
1. Keep batches short.
|
|
361
|
-
2. Include only deterministic operations.
|
|
362
|
-
3. Verify after the batch with a separate command.
|
|
363
|
-
4. Use `--file` for complex JSON quoting.
|
|
364
|
-
|
|
365
|
-
### Example
|
|
366
|
-
|
|
367
|
-
```bash
|
|
368
|
-
OMNIBOT_SESSION_TOKEN=research omnibot batch '[{"cmd":"snapshot","interactive":true}]' --tab-id <TAB_ID>
|
|
369
|
-
OMNIBOT_SESSION_TOKEN=research omnibot snapshot -i --tab-id <TAB_ID>
|
|
370
|
-
```
|
|
371
|
-
|
|
372
|
-
File example:
|
|
373
|
-
|
|
374
|
-
```bash
|
|
375
|
-
OMNIBOT_SESSION_TOKEN=research omnibot batch --file /tmp/omnibot-batch.json --tab-id <TAB_ID>
|
|
376
|
-
OMNIBOT_SESSION_TOKEN=research omnibot get url --tab-id <TAB_ID>
|
|
377
|
-
```
|
|
378
|
-
|
|
379
|
-
### Verification
|
|
380
|
-
|
|
381
|
-
Do not treat batch success as task success. Verify final page state separately.
|
|
382
|
-
|
|
383
|
-
### Fallback entry point
|
|
384
|
-
|
|
385
|
-
Batch is not a fallback tier. If a batched operation fails, re-run the workflow step-by-step and enter the normal fallback model.
|