@omniaibot/win-x64 1.1.2 → 1.1.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/bin/omnibot-windows-x64/VERSION +1 -1
- package/bin/omnibot-windows-x64/omnibot/SKILL.md +189 -0
- package/bin/omnibot-windows-x64/omnibot/references/anti-patterns.md +37 -0
- package/bin/omnibot-windows-x64/omnibot/references/command-reference.md +605 -0
- package/bin/omnibot-windows-x64/omnibot/references/debugging-and-evidence.md +95 -0
- package/bin/omnibot-windows-x64/omnibot/references/fallback-operations.md +216 -0
- package/bin/omnibot-windows-x64/omnibot/references/operation-patterns.md +385 -0
- package/bin/omnibot-windows-x64/omnibot/references/runtime-and-status.md +111 -0
- package/bin/omnibot-windows-x64/omnibot/references/session-and-tabs.md +164 -0
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/SKILL.md +4 -4
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/anti-patterns.md +4 -2
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/command-reference.md +5 -16
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/debugging-and-evidence.md +1 -1
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/fallback-operations.md +60 -1
- package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/operation-patterns.md +23 -10
- package/bin/omnibot-windows-x64/omnibot-windows-x64.exe +0 -0
- package/package.json +2 -2
|
@@ -0,0 +1,111 @@
|
|
|
1
|
+
# Runtime and Status
|
|
2
|
+
|
|
3
|
+
Use this file before troubleshooting. Runtime checks establish whether the daemon, extension, browser, visibility mode, license, and packaged skills are healthy.
|
|
4
|
+
|
|
5
|
+
## Startup Check
|
|
6
|
+
|
|
7
|
+
Run this sequence before diagnosing browser failures:
|
|
8
|
+
|
|
9
|
+
```bash
|
|
10
|
+
omnibot doctor
|
|
11
|
+
omnibot status
|
|
12
|
+
omnibot tabs
|
|
13
|
+
omnibot visibility status
|
|
14
|
+
omnibot browser current
|
|
15
|
+
omnibot license status
|
|
16
|
+
```
|
|
17
|
+
|
|
18
|
+
Interpretation:
|
|
19
|
+
|
|
20
|
+
- `doctor` checks daemon and extension health.
|
|
21
|
+
- `status` checks daemon status only (lighter than `doctor`).
|
|
22
|
+
- `tabs` proves connected tabs are visible and provides tab ids.
|
|
23
|
+
- `visibility status` shows whether automation is visible, background, dedicated-profile, or headless.
|
|
24
|
+
- `browser current` shows current browser runtime ownership.
|
|
25
|
+
- `license status` verifies license state.
|
|
26
|
+
|
|
27
|
+
If the extension is not connected, open Chrome or Edge with the omnibot extension loaded and keep an HTTP/HTTPS page open.
|
|
28
|
+
|
|
29
|
+
## Daemon Lifecycle
|
|
30
|
+
|
|
31
|
+
Top-level shortcuts:
|
|
32
|
+
|
|
33
|
+
```bash
|
|
34
|
+
omnibot status # Show daemon status
|
|
35
|
+
omnibot start # Start the daemon
|
|
36
|
+
omnibot stop # Stop the daemon
|
|
37
|
+
omnibot run # Run daemon in foreground
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
Full form (also supported):
|
|
41
|
+
|
|
42
|
+
```bash
|
|
43
|
+
omnibot daemon run
|
|
44
|
+
omnibot daemon start
|
|
45
|
+
omnibot daemon stop
|
|
46
|
+
omnibot daemon status
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
## doctor
|
|
50
|
+
|
|
51
|
+
```bash
|
|
52
|
+
omnibot doctor
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
Use `doctor` first when commands fail, tabs are empty, screenshots fail, or the agent cannot reach the browser.
|
|
56
|
+
|
|
57
|
+
## tabs
|
|
58
|
+
|
|
59
|
+
```bash
|
|
60
|
+
omnibot tabs
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
Use `tabs` to discover tab ids. Save the target tab id and use it on every page-state command:
|
|
64
|
+
|
|
65
|
+
```bash
|
|
66
|
+
OMNIBOT_SESSION_TOKEN=research omnibot snapshot -i --tab-id <TAB_ID>
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
## browser list/current
|
|
70
|
+
|
|
71
|
+
```bash
|
|
72
|
+
omnibot browser list
|
|
73
|
+
omnibot browser current
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
Use browser status when multiple browser runtimes or agents may be active.
|
|
77
|
+
|
|
78
|
+
## visibility status
|
|
79
|
+
|
|
80
|
+
```bash
|
|
81
|
+
omnibot visibility status
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
Use visibility status to confirm whether automation should share the user's visible browser state. Headless and dedicated-profile modes do not automatically inherit user login.
|
|
85
|
+
|
|
86
|
+
## license status
|
|
87
|
+
|
|
88
|
+
```bash
|
|
89
|
+
omnibot license status
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
Use license status when runtime checks pass but features appear unavailable.
|
|
93
|
+
|
|
94
|
+
## skills path
|
|
95
|
+
|
|
96
|
+
```bash
|
|
97
|
+
omnibot skills path
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
Use skills path to locate packaged skills for installation or inspection.
|
|
101
|
+
|
|
102
|
+
## Skills Install
|
|
103
|
+
|
|
104
|
+
```bash
|
|
105
|
+
omnibot skills install --agent hermes --profile nuwa
|
|
106
|
+
omnibot skills install --agent opencode
|
|
107
|
+
omnibot skills install --agent claude
|
|
108
|
+
omnibot skills install --agent codex
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
Install only when setting up or repairing an agent integration. It is not part of normal page operations.
|
|
@@ -0,0 +1,164 @@
|
|
|
1
|
+
# Sessions and Tabs
|
|
2
|
+
|
|
3
|
+
Sessions and tabs are core reliability concepts, not advanced usage.
|
|
4
|
+
|
|
5
|
+
## Session First
|
|
6
|
+
|
|
7
|
+
`OMNIBOT_SESSION_TOKEN` is mandatory for every workflow.
|
|
8
|
+
|
|
9
|
+
```bash
|
|
10
|
+
OMNIBOT_SESSION_TOKEN=research omnibot tabs
|
|
11
|
+
OMNIBOT_SESSION_TOKEN=research omnibot snapshot -i --tab-id <TAB_ID>
|
|
12
|
+
```
|
|
13
|
+
|
|
14
|
+
The same token keeps one workflow isolated. Different tokens prevent independent agents from sharing default state.
|
|
15
|
+
|
|
16
|
+
## Tab Explicit
|
|
17
|
+
|
|
18
|
+
`--tab-id <TAB_ID>` is mandatory for every page-state command.
|
|
19
|
+
|
|
20
|
+
```bash
|
|
21
|
+
OMNIBOT_SESSION_TOKEN=checkout omnibot get url --tab-id <TAB_ID>
|
|
22
|
+
OMNIBOT_SESSION_TOKEN=checkout omnibot click --tab-id <TAB_ID> @e4
|
|
23
|
+
OMNIBOT_SESSION_TOKEN=checkout omnibot get url --tab-id <TAB_ID>
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
Do not rely on default tab, active tab, current tab, or previous targeting state.
|
|
27
|
+
|
|
28
|
+
## Explicit Targeting Model
|
|
29
|
+
|
|
30
|
+
Page-state commands never infer a target tab. Always pass `--tab-id <TAB_ID>` for commands that read or mutate page state.
|
|
31
|
+
|
|
32
|
+
Transport selection is internal. Omnibot may use any connected extension WebSocket to deliver extension-level commands, but that transport tab is never the page target.
|
|
33
|
+
|
|
34
|
+
Do not use the first returned tab, `tabs[0]`, browser active tab, newest tab, or prior command state as a target.
|
|
35
|
+
|
|
36
|
+
## Transport Is Not Target
|
|
37
|
+
|
|
38
|
+
Omnibot can send extension commands through any connected extension WebSocket tab. That transport tab is not the page target. The target is only the tab id passed in the command, for example `--tab-id edge-client:123`.
|
|
39
|
+
|
|
40
|
+
Safe sequence:
|
|
41
|
+
|
|
42
|
+
```bash
|
|
43
|
+
OMNIBOT_SESSION_TOKEN=research omnibot navigate https://example.com
|
|
44
|
+
OMNIBOT_SESSION_TOKEN=research omnibot snapshot -i --tab-id edge-client:123
|
|
45
|
+
OMNIBOT_SESSION_TOKEN=research omnibot click --tab-id edge-client:123 @e4
|
|
46
|
+
OMNIBOT_SESSION_TOKEN=research omnibot wait --text "Saved" --tab-id edge-client:123
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
Unsafe sequence:
|
|
50
|
+
|
|
51
|
+
```bash
|
|
52
|
+
OMNIBOT_SESSION_TOKEN=research omnibot tabs
|
|
53
|
+
OMNIBOT_SESSION_TOKEN=research omnibot snapshot -i
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
The unsafe sequence has no target. Do not infer target from first tab, active tab, or prior operation.
|
|
57
|
+
|
|
58
|
+
## Safe Browser Test Cleanup
|
|
59
|
+
|
|
60
|
+
Browser integration tests must only close tabs they created:
|
|
61
|
+
|
|
62
|
+
1. Record the list of existing tabs before creating fixtures.
|
|
63
|
+
2. Create a unique fixture tab per case.
|
|
64
|
+
3. Store the returned full tab id.
|
|
65
|
+
4. Pass that id to every page-state command.
|
|
66
|
+
5. Close only stored fixture ids in cleanup.
|
|
67
|
+
6. Verify user tabs remain present after the test.
|
|
68
|
+
|
|
69
|
+
## Why Both
|
|
70
|
+
|
|
71
|
+
- Session = workflow isolation.
|
|
72
|
+
- Tab ID = page targeting.
|
|
73
|
+
|
|
74
|
+
Session alone does not prove which page receives a command. Tab ID alone does not isolate concurrent agents. Use both.
|
|
75
|
+
|
|
76
|
+
## Ref Scope
|
|
77
|
+
|
|
78
|
+
`@eN` refs are tab-scoped.
|
|
79
|
+
|
|
80
|
+
Never reuse `@eN` across tabs. Never assume a ref remains valid after navigation, reload, major DOM changes, or a different snapshot.
|
|
81
|
+
|
|
82
|
+
```bash
|
|
83
|
+
OMNIBOT_SESSION_TOKEN=checkout omnibot snapshot -i --tab-id <TAB_ID>
|
|
84
|
+
OMNIBOT_SESSION_TOKEN=checkout omnibot click --tab-id <TAB_ID> @e4
|
|
85
|
+
OMNIBOT_SESSION_TOKEN=checkout omnibot snapshot -i --tab-id <TAB_ID>
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
## Multi-Agent Pattern
|
|
89
|
+
|
|
90
|
+
Agent A:
|
|
91
|
+
|
|
92
|
+
```bash
|
|
93
|
+
OMNIBOT_SESSION_TOKEN=research
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
Agent B:
|
|
97
|
+
|
|
98
|
+
```bash
|
|
99
|
+
OMNIBOT_SESSION_TOKEN=checkout
|
|
100
|
+
```
|
|
101
|
+
|
|
102
|
+
Agent C:
|
|
103
|
+
|
|
104
|
+
```bash
|
|
105
|
+
OMNIBOT_SESSION_TOKEN=debug
|
|
106
|
+
```
|
|
107
|
+
|
|
108
|
+
Each agent must:
|
|
109
|
+
|
|
110
|
+
1. Set a stable token.
|
|
111
|
+
2. Discover or create a tab.
|
|
112
|
+
3. Save the tab id.
|
|
113
|
+
4. Pass `--tab-id <TAB_ID>` on every page-state command.
|
|
114
|
+
5. Run `snapshot` -> action -> verify, or `get/is` -> action -> `get/is`.
|
|
115
|
+
|
|
116
|
+
Example:
|
|
117
|
+
|
|
118
|
+
```bash
|
|
119
|
+
OMNIBOT_SESSION_TOKEN=research omnibot tab new https://example.com --label research
|
|
120
|
+
OMNIBOT_SESSION_TOKEN=research omnibot get url --tab-id <TAB_ID>
|
|
121
|
+
OMNIBOT_SESSION_TOKEN=research omnibot snapshot -i --tab-id <TAB_ID>
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
## No Implicit Targeting
|
|
125
|
+
|
|
126
|
+
Omnibot does not provide a workflow-level current tab command. Every command that reads or changes page state must receive `--tab-id <TAB_ID>` in that command invocation.
|
|
127
|
+
|
|
128
|
+
This prevents stale target state and cross-agent tab confusion.
|
|
129
|
+
|
|
130
|
+
## browser claim/release
|
|
131
|
+
|
|
132
|
+
Use `browser claim` and `browser release` in multi-browser or multi-runtime scenarios where an agent must explicitly own or release a browser tab.
|
|
133
|
+
|
|
134
|
+
```bash
|
|
135
|
+
OMNIBOT_SESSION_TOKEN=research omnibot browser list
|
|
136
|
+
OMNIBOT_SESSION_TOKEN=research omnibot browser current
|
|
137
|
+
OMNIBOT_SESSION_TOKEN=research omnibot browser claim 123
|
|
138
|
+
OMNIBOT_SESSION_TOKEN=research omnibot snapshot -i --tab-id 123
|
|
139
|
+
OMNIBOT_SESSION_TOKEN=research omnibot browser release 123
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
Claiming does not replace `--tab-id`. It only documents or coordinates ownership.
|
|
143
|
+
|
|
144
|
+
## visibility
|
|
145
|
+
|
|
146
|
+
Visibility controls where automation runs:
|
|
147
|
+
|
|
148
|
+
- `visible`: controls the user's already-open real browser tabs and preserves visible session state.
|
|
149
|
+
- `background`: avoids foregrounding tabs unless requested.
|
|
150
|
+
- `dedicated-profile`: launches or uses a separate profile.
|
|
151
|
+
- `headless`: uses a headless browser context.
|
|
152
|
+
- `launch`: starts a dedicated-profile or headless browser with an explicit user data dir.
|
|
153
|
+
|
|
154
|
+
Examples:
|
|
155
|
+
|
|
156
|
+
```bash
|
|
157
|
+
omnibot visibility status
|
|
158
|
+
omnibot visibility set visible
|
|
159
|
+
omnibot visibility set background
|
|
160
|
+
omnibot visibility set headless --user-data-dir /tmp/omnibot-headless
|
|
161
|
+
omnibot visibility launch headless --user-data-dir /tmp/omnibot-headless
|
|
162
|
+
```
|
|
163
|
+
|
|
164
|
+
Headless and dedicated-profile modes do not automatically share the user's current tabs or login state. Verify login state before assuming access.
|
|
@@ -12,7 +12,7 @@ metadata:
|
|
|
12
12
|
|
|
13
13
|
Omnibot is Browser Infrastructure for AI Agents.
|
|
14
14
|
|
|
15
|
-
It connects Hermes, Claude Code, Codex, OpenCode,
|
|
15
|
+
It connects Hermes, Claude Code, Codex, OpenCode, and other agent systems to a real Chromium browser through the local omnibot daemon and CLI.
|
|
16
16
|
|
|
17
17
|
This skill is an execution specification for agents. It is not a human command manual.
|
|
18
18
|
|
|
@@ -108,18 +108,18 @@ Start from the task, not the command name. Use operation patterns for read, clic
|
|
|
108
108
|
|
|
109
109
|
Use fallback tiers only after standard patterns fail. Collect evidence separately from fallback execution. The command reference is only a lookup table; it must not decide behavior.
|
|
110
110
|
|
|
111
|
-
## Read vs
|
|
111
|
+
## Read vs Snapshot Routing
|
|
112
112
|
|
|
113
113
|
Choose by intent before choosing by command name:
|
|
114
114
|
|
|
115
115
|
| Intent | Prefer | Why |
|
|
116
116
|
| --- | --- | --- |
|
|
117
117
|
| Summarize or extract page content, article text, search results, feeds, or long/lazy pages | `read` | Returns clean text/Markdown for agent reasoning. |
|
|
118
|
-
| Quickly observe
|
|
118
|
+
| Quickly observe current visible UI structure before deciding what to do next | `snapshot -i` | Returns actionable refs and page structure for routing. |
|
|
119
119
|
| Find buttons, links, inputs, or refs for a later action | `find` or `snapshot -i` | Produces actionable targets; `read` does not. |
|
|
120
120
|
| Verify one concrete condition | `get`, `is`, or `wait` | Narrow evidence is more reliable than dumping a page. |
|
|
121
121
|
|
|
122
|
-
Do not use `
|
|
122
|
+
Do not use `read` as the first step for click/fill workflows unless the user explicitly asked for page content first.
|
|
123
123
|
|
|
124
124
|
## Quick Routing
|
|
125
125
|
|
|
@@ -4,7 +4,7 @@
|
|
|
4
4
|
| --- | --- | --- |
|
|
5
5
|
| Missing `OMNIBOT_SESSION_TOKEN` | Workflows can share implicit state across agents. | `OMNIBOT_SESSION_TOKEN=research omnibot snapshot -i --tab-id <TAB_ID>` |
|
|
6
6
|
| Missing `--tab-id` | The command may target the wrong page. | Pass `--tab-id <TAB_ID>` on every page-state command. |
|
|
7
|
-
| `execute-js` first | JavaScript bypasses standard browser interaction and hides better evidence. | Try semantic find, snapshot refs, selectors, DOM, and mouse first. |
|
|
7
|
+
| `execute-js` first | JavaScript bypasses standard browser interaction and hides better evidence. | Try semantic find, snapshot refs, selectors, DOM, and mouse first. For CSS-only dropdowns, use JavaScript only after the trigger clicked, options remain absent from `snapshot -i`, and higher tiers cannot complete open-then-select. |
|
|
8
8
|
| Reusing `@eN` across tabs | Refs are tab-scoped and may point to a different element elsewhere. | Take a fresh `snapshot -i --tab-id <TAB_ID>` per tab. |
|
|
9
9
|
| Relying on implicit tab targeting | It is unavailable and unsafe for concurrent workflows. | Use explicit `--tab-id` on every page-state command. |
|
|
10
10
|
| Using sleep instead of wait | Fixed sleeps are flaky and either too short or too slow. | Use `omnibot wait --text`, `--url`, selector state, load state, or `--fn`. |
|
|
@@ -12,7 +12,6 @@
|
|
|
12
12
|
| Claiming success without evidence | The browser may not have changed as expected. | Cite verified URL, text, element state, screenshot, console, or network evidence. |
|
|
13
13
|
| Raw CSS before semantic find | CSS is more brittle and less tied to user intent. | Start with `find role/text/label/placeholder/testid`. |
|
|
14
14
|
| Screenshot when text extraction is enough | Screenshots cost more and are harder to parse. | Use `read` for page content or `get text` for a selector. |
|
|
15
|
-
| Using `scan` for long-form reading | `scan` is viewport observation and may miss long/lazy content. | Use `read --screens N --tab-id <TAB_ID>`. |
|
|
16
15
|
| Using `read` before an action workflow | `read` returns text, not stable action targets. | Use `find`, `snapshot -i`, `get`, or `is` for click/fill/verify workflows. |
|
|
17
16
|
| Headless expecting existing user login | Headless does not automatically share the user's visible browser session. | Use visible/background mode for existing login or explicitly log in headless. |
|
|
18
17
|
| Mixing multiple workflows in one session token | Agents can overwrite each other's default state. | Use different tokens: `research`, `checkout`, `debug`. |
|
|
@@ -31,5 +30,8 @@
|
|
|
31
30
|
- "The ref probably still points to the same element."
|
|
32
31
|
- "A screenshot is enough even though I need text."
|
|
33
32
|
- "Sleep should be fine."
|
|
33
|
+
- "The trigger clicked, so another `find text` should eventually select the option."
|
|
34
|
+
- "`snapshot -i` cannot see the option, so the element must not exist."
|
|
35
|
+
- "This dropdown is custom, so I can start with JavaScript."
|
|
34
36
|
|
|
35
37
|
All of these mean: stop, re-enter the standard pattern, and make state explicit.
|
|
@@ -80,30 +80,19 @@ Fallback relation: Not a fallback command.
|
|
|
80
80
|
|
|
81
81
|
## Reading
|
|
82
82
|
|
|
83
|
-
### scan
|
|
84
|
-
|
|
85
|
-
Purpose: Observe the current rendered viewport, especially before choosing a follow-up action.
|
|
86
|
-
|
|
87
|
-
```bash
|
|
88
|
-
OMNIBOT_SESSION_TOKEN=research omnibot scan --json --tab-id <TAB_ID>
|
|
89
|
-
OMNIBOT_SESSION_TOKEN=research omnibot scan --text-only --tab-id <TAB_ID>
|
|
90
|
-
```
|
|
91
|
-
|
|
92
|
-
Preferred pattern: See `operation-patterns.md#read` and `operation-patterns.md#extraction`.
|
|
93
|
-
Fallback relation: Use before `dom visible` or `execute-js` when structured viewport state is needed. Use `read` instead for long-form page text.
|
|
94
|
-
|
|
95
83
|
### read
|
|
96
84
|
|
|
97
85
|
Purpose: Read clean rendered page text/Markdown from an existing tab or a temporary URL tab.
|
|
98
86
|
|
|
99
87
|
```bash
|
|
100
88
|
OMNIBOT_SESSION_TOKEN=research omnibot read --screens 5 --tab-id <TAB_ID>
|
|
89
|
+
OMNIBOT_SESSION_TOKEN=research omnibot read --screens 5 --timeout 120 --tab-id <TAB_ID>
|
|
101
90
|
OMNIBOT_SESSION_TOKEN=research omnibot read --screens 3 https://example.com/article
|
|
102
91
|
OMNIBOT_SESSION_TOKEN=research omnibot read --json --screens 5 --tab-id <TAB_ID>
|
|
103
92
|
```
|
|
104
93
|
|
|
105
94
|
Preferred pattern: See `operation-patterns.md#read` and `operation-patterns.md#extraction`.
|
|
106
|
-
Fallback relation: Prefer for page content. Use `
|
|
95
|
+
Fallback relation: Prefer for page content. Use `snapshot -i` when the next step is action planning, and `get`/`is`/`wait` for narrow state verification.
|
|
107
96
|
|
|
108
97
|
### snapshot -i
|
|
109
98
|
|
|
@@ -241,7 +230,7 @@ OMNIBOT_SESSION_TOKEN=form omnibot select @e5 "US" --tab-id <TAB_ID>
|
|
|
241
230
|
```
|
|
242
231
|
|
|
243
232
|
Preferred pattern: See `operation-patterns.md#select--check`.
|
|
244
|
-
Fallback relation: Prefer refs/selectors before JavaScript value changes.
|
|
233
|
+
Fallback relation: Prefer refs/selectors before JavaScript value changes. For CSS-only dropdowns whose options are absent from `snapshot -i`, see `fallback-operations.md#css-only-dropdown-javascript-fallback`.
|
|
245
234
|
|
|
246
235
|
### check / uncheck
|
|
247
236
|
|
|
@@ -350,7 +339,7 @@ OMNIBOT_SESSION_TOKEN=repair omnibot execute-js --file /tmp/repair.js --tab-id <
|
|
|
350
339
|
```
|
|
351
340
|
|
|
352
341
|
Preferred pattern: See `fallback-operations.md#javascript-fallback`.
|
|
353
|
-
Fallback relation: After semantic, refs, selector, DOM, and mouse are insufficient.
|
|
342
|
+
Fallback relation: After semantic, refs, selector, DOM, and mouse are insufficient. Valid examples include CSS-only dropdowns where the trigger exists but option refs are absent from `snapshot -i`.
|
|
354
343
|
|
|
355
344
|
### cdp
|
|
356
345
|
|
|
@@ -443,7 +432,7 @@ Fallback relation: Prefer over shell sleep.
|
|
|
443
432
|
Purpose: Send short command arrays as JSON or from a file.
|
|
444
433
|
|
|
445
434
|
```bash
|
|
446
|
-
OMNIBOT_SESSION_TOKEN=research omnibot batch '[{"cmd":"
|
|
435
|
+
OMNIBOT_SESSION_TOKEN=research omnibot batch '[{"cmd":"snapshot","interactive":true}]' --tab-id <TAB_ID>
|
|
447
436
|
OMNIBOT_SESSION_TOKEN=research omnibot batch --file /tmp/omnibot-batch.json --tab-id <TAB_ID>
|
|
448
437
|
```
|
|
449
438
|
|
|
@@ -89,7 +89,7 @@ CDP inspection is not a first-choice operation path. If CDP changes state, it be
|
|
|
89
89
|
Before reporting a browser issue, collect the smallest evidence set that explains it:
|
|
90
90
|
|
|
91
91
|
- Runtime: `doctor`, `tabs`, `visibility status`.
|
|
92
|
-
- Page state: `get`, `is`, `
|
|
92
|
+
- Page state: `get`, `is`, `snapshot`, or `wait`.
|
|
93
93
|
- Visual proof: `screenshot --annotate` only if visual state matters.
|
|
94
94
|
- Browser diagnostics: `console errors` and `network summary`.
|
|
95
95
|
- Reproduction artifact: `trace` or `record` when the flow is unstable.
|
|
@@ -106,7 +106,8 @@ Use `execute-js` only when:
|
|
|
106
106
|
- CSS selector operation fails.
|
|
107
107
|
- DOM fallback fails.
|
|
108
108
|
- A complex frontend event must be triggered.
|
|
109
|
-
-
|
|
109
|
+
- A CSS-only dropdown or hover menu requires an atomic open-then-select sequence, and `snapshot -i` confirms the option nodes are absent from the accessibility tree.
|
|
110
|
+
- Runtime state cannot be read through `read`, `snapshot`, `get`, or `is`.
|
|
110
111
|
|
|
111
112
|
Read-only JavaScript fallback:
|
|
112
113
|
|
|
@@ -129,6 +130,64 @@ OMNIBOT_SESSION_TOKEN=repair omnibot execute-js "document.querySelector('button[
|
|
|
129
130
|
OMNIBOT_SESSION_TOKEN=repair omnibot wait --url "/dashboard" --tab-id <TAB_ID>
|
|
130
131
|
```
|
|
131
132
|
|
|
133
|
+
### CSS-only dropdown JavaScript fallback
|
|
134
|
+
|
|
135
|
+
Use this only after the trigger is known and higher tiers failed: semantic `find` clicked the trigger or found the text, `snapshot -i` could not expose option refs, selector/DOM/mouse could not complete the two-step interaction, and verification showed the selection did not change.
|
|
136
|
+
|
|
137
|
+
Example for a visible trigger like `按留言时间排序` and a target option text:
|
|
138
|
+
|
|
139
|
+
```bash
|
|
140
|
+
OMNIBOT_SESSION_TOKEN=repair omnibot execute-js --file /tmp/omnibot-css-dropdown.js --tab-id <TAB_ID>
|
|
141
|
+
OMNIBOT_SESSION_TOKEN=repair omnibot snapshot -i --tab-id <TAB_ID>
|
|
142
|
+
```
|
|
143
|
+
|
|
144
|
+
`/tmp/omnibot-css-dropdown.js`:
|
|
145
|
+
|
|
146
|
+
```js
|
|
147
|
+
const triggerText = '按留言时间排序';
|
|
148
|
+
const optionText = '目标选项文本';
|
|
149
|
+
|
|
150
|
+
const visibleElements = [...document.querySelectorAll('button, [role="button"], [aria-haspopup], div, span, li')]
|
|
151
|
+
.filter((el) => {
|
|
152
|
+
const style = getComputedStyle(el);
|
|
153
|
+
const box = el.getBoundingClientRect();
|
|
154
|
+
return style.display !== 'none' && style.visibility !== 'hidden' && box.width > 0 && box.height > 0;
|
|
155
|
+
});
|
|
156
|
+
|
|
157
|
+
const byExactText = (text) => visibleElements.find((el) => el.textContent?.trim() === text);
|
|
158
|
+
const trigger = byExactText(triggerText);
|
|
159
|
+
if (!trigger) {
|
|
160
|
+
return { ok: false, reason: 'trigger-not-found', triggerText, optionText };
|
|
161
|
+
}
|
|
162
|
+
|
|
163
|
+
trigger.click();
|
|
164
|
+
await new Promise((resolve) => requestAnimationFrame(() => requestAnimationFrame(resolve)));
|
|
165
|
+
|
|
166
|
+
const option = [...document.querySelectorAll('div, span, li, button, [role="option"], [role="menuitem"]')]
|
|
167
|
+
.filter((el) => {
|
|
168
|
+
const style = getComputedStyle(el);
|
|
169
|
+
const box = el.getBoundingClientRect();
|
|
170
|
+
return style.display !== 'none' && style.visibility !== 'hidden' && box.width > 0 && box.height > 0;
|
|
171
|
+
})
|
|
172
|
+
.find((el) => el.textContent?.trim() === optionText);
|
|
173
|
+
|
|
174
|
+
if (!option) {
|
|
175
|
+
return { ok: false, reason: 'option-not-found-after-trigger-click', triggerText, optionText };
|
|
176
|
+
}
|
|
177
|
+
|
|
178
|
+
option.click();
|
|
179
|
+
await new Promise((resolve) => requestAnimationFrame(() => requestAnimationFrame(resolve)));
|
|
180
|
+
|
|
181
|
+
return {
|
|
182
|
+
ok: true,
|
|
183
|
+
triggerText,
|
|
184
|
+
optionText,
|
|
185
|
+
selectedText: trigger.textContent?.trim(),
|
|
186
|
+
};
|
|
187
|
+
```
|
|
188
|
+
|
|
189
|
+
Verify with the user-visible result, not just `{ ok: true }`: selected label, sorted result order, result count, URL/query state, or another page-specific condition.
|
|
190
|
+
|
|
132
191
|
Prefer `--file` for longer scripts:
|
|
133
192
|
|
|
134
193
|
```bash
|
|
@@ -8,7 +8,7 @@ Use this file to choose behavior by task. Every page-state workflow must set `OM
|
|
|
8
8
|
|
|
9
9
|
Use Read when the agent needs clean rendered page content: article text, search results, feed items, long pages, lazy-loaded text, or a human-readable summary source.
|
|
10
10
|
|
|
11
|
-
Use targeted state commands when the agent needs one value, visibility, enabled/checked state, layout box, or computed styles. Use
|
|
11
|
+
Use targeted state commands when the agent needs one value, visibility, enabled/checked state, layout box, or computed styles. Use `snapshot -i` when the agent needs current visible UI structure for deciding the next action.
|
|
12
12
|
|
|
13
13
|
### Preferred sequence
|
|
14
14
|
|
|
@@ -17,9 +17,8 @@ Use targeted state commands when the agent needs one value, visibility, enabled/
|
|
|
17
17
|
3. `read <URL>` when opening a temporary tab only for reading that URL.
|
|
18
18
|
4. `get title/url/text/html/value/attr/count/box/styles` for narrow evidence.
|
|
19
19
|
5. `is visible/enabled/checked` for boolean state.
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
8. `dom visible` or `execute-js` fallback only after standard reads cannot access the state.
|
|
20
|
+
5. `snapshot -i` when actionable refs are needed.
|
|
21
|
+
6. `dom visible` or `execute-js` fallback only after standard reads cannot access the state.
|
|
23
22
|
|
|
24
23
|
### Example
|
|
25
24
|
|
|
@@ -40,7 +39,7 @@ OMNIBOT_SESSION_TOKEN=research omnibot get count ".result" --tab-id <TAB_ID>
|
|
|
40
39
|
|
|
41
40
|
### Fallback entry point
|
|
42
41
|
|
|
43
|
-
If `read
|
|
42
|
+
If `read` or targeted reads omit needed runtime state, go to `fallback-operations.md#fallback-tier-model`. Start with DOM fallback before JavaScript.
|
|
44
43
|
|
|
45
44
|
## Click
|
|
46
45
|
|
|
@@ -161,9 +160,23 @@ OMNIBOT_SESSION_TOKEN=form omnibot is checked "#agree" --tab-id <TAB_ID>
|
|
|
161
160
|
|
|
162
161
|
For checkboxes, verify boolean state. For selects, verify value or dependent page state.
|
|
163
162
|
|
|
163
|
+
### CSS-only dropdowns and custom menus
|
|
164
|
+
|
|
165
|
+
Some dropdowns, hover menus, tooltips, and custom selects are CSS-only widgets. The visible trigger may be accessible, while the option nodes are hidden `<div>` elements without `role`, accessible name, or stable refs.
|
|
166
|
+
|
|
167
|
+
Use this evidence path before JavaScript:
|
|
168
|
+
|
|
169
|
+
1. Try semantic `find` or a normal `select`/`click` against the trigger.
|
|
170
|
+
2. Verify whether the selected label, result order, or dependent state changed.
|
|
171
|
+
3. Run `snapshot -i` after opening the control.
|
|
172
|
+
4. If the trigger clicked but options are absent from the accessibility tree, do not keep retrying `find` or stale refs.
|
|
173
|
+
5. Escalate to `fallback-operations.md#css-only-dropdown-javascript-fallback` for an atomic open-then-select script.
|
|
174
|
+
|
|
175
|
+
`snapshot -i` not showing an option does not prove the DOM node is missing. It may only prove the widget is not exposed through the accessibility tree.
|
|
176
|
+
|
|
164
177
|
### Fallback entry point
|
|
165
178
|
|
|
166
|
-
If native select/check fails, use `fallback-operations.md#selector-fallback
|
|
179
|
+
If native select/check fails, use `fallback-operations.md#selector-fallback`. Use JavaScript only if events must be synthesized or if evidence shows a CSS-only multi-step widget cannot be completed through semantic, snapshot, selector, DOM, or mouse tiers.
|
|
167
180
|
|
|
168
181
|
## Navigation
|
|
169
182
|
|
|
@@ -261,7 +274,7 @@ OMNIBOT_SESSION_TOKEN=checkout omnibot wait --fn "window.appReady === true" --ta
|
|
|
261
274
|
|
|
262
275
|
### Verification
|
|
263
276
|
|
|
264
|
-
After waiting, read the final condition with `get`, `is`, `
|
|
277
|
+
After waiting, read the final condition with `get`, `is`, `wait`, or `snapshot`.
|
|
265
278
|
|
|
266
279
|
### Fallback entry point
|
|
267
280
|
|
|
@@ -277,7 +290,7 @@ Use Extraction for retrieving page text, HTML fragments, links, counts, assets,
|
|
|
277
290
|
|
|
278
291
|
1. `get text/html/attr/count` for targeted extraction.
|
|
279
292
|
2. `read --screens N` for clean page text, long pages, or lazy-loaded content.
|
|
280
|
-
3. `
|
|
293
|
+
3. `snapshot -i` for structured page observation before action planning.
|
|
281
294
|
4. `assets list/export` for resources.
|
|
282
295
|
5. `clipboard read` only when clipboard content is part of the task.
|
|
283
296
|
|
|
@@ -302,7 +315,7 @@ Verify extraction completeness with expected markers in `read`, element counts,
|
|
|
302
315
|
|
|
303
316
|
### Fallback entry point
|
|
304
317
|
|
|
305
|
-
If content exists only in runtime objects, use `fallback-operations.md#javascript-fallback` after explaining why `read`, `
|
|
318
|
+
If content exists only in runtime objects, use `fallback-operations.md#javascript-fallback` after explaining why `read`, `snapshot`, and `get` could not access it.
|
|
306
319
|
|
|
307
320
|
## Upload
|
|
308
321
|
|
|
@@ -352,7 +365,7 @@ Use Batch for short, known-safe command chains where the target tab and verifica
|
|
|
352
365
|
### Example
|
|
353
366
|
|
|
354
367
|
```bash
|
|
355
|
-
OMNIBOT_SESSION_TOKEN=research omnibot batch '[{"cmd":"
|
|
368
|
+
OMNIBOT_SESSION_TOKEN=research omnibot batch '[{"cmd":"snapshot","interactive":true}]' --tab-id <TAB_ID>
|
|
356
369
|
OMNIBOT_SESSION_TOKEN=research omnibot snapshot -i --tab-id <TAB_ID>
|
|
357
370
|
```
|
|
358
371
|
|
|
Binary file
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@omniaibot/win-x64",
|
|
3
|
-
"version": "1.1.
|
|
3
|
+
"version": "1.1.6",
|
|
4
4
|
"description": "omnibot executable for Windows x64",
|
|
5
5
|
"os": [
|
|
6
6
|
"win32"
|
|
@@ -15,9 +15,9 @@
|
|
|
15
15
|
"bin/"
|
|
16
16
|
],
|
|
17
17
|
"keywords": [
|
|
18
|
-
"mcp",
|
|
19
18
|
"browser",
|
|
20
19
|
"automation",
|
|
20
|
+
"agent",
|
|
21
21
|
"omnibot"
|
|
22
22
|
],
|
|
23
23
|
"author": "Unagi-cq",
|