@omniaibot/win-x64 0.3.11 → 1.1.0

package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/fallback-operations.md

@@ -0,0 +1,157 @@
+# Fallback Operations
+
+Fallback is for completing operations after preferred patterns fail. It is not debugging. Debugging collects evidence; fallback changes page state.
+
+Every fallback escalation must answer two questions:
+
+- Which higher tier failed, and why?
+- How will the result be verified?
+
+## Fallback Tier Model
+
+### Tier 1 Semantic
+
+Use semantic intent first:
+
+```bash
+OMNIBOT_SESSION_TOKEN=checkout omnibot find role button --name "Submit" --action click --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=checkout omnibot find text "Sign in" --action click --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=checkout omnibot find label "Email" --action fill --action-value "a@b.com" --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=checkout omnibot find placeholder "Search" --action type --action-value "omnibot" --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=checkout omnibot find testid submit-button --action click --tab-id <TAB_ID>
+```
+
+Verify:
+
+```bash
+OMNIBOT_SESSION_TOKEN=checkout omnibot snapshot -i --tab-id <TAB_ID>
+```
+
+### Tier 2 Snapshot Refs
+
+Use refs when semantic targeting is missing or ambiguous.
+
+```bash
+OMNIBOT_SESSION_TOKEN=checkout omnibot snapshot -i --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=checkout omnibot click --tab-id <TAB_ID> @e4
+OMNIBOT_SESSION_TOKEN=checkout omnibot snapshot -i --tab-id <TAB_ID>
+```
+
+Refs are tab-scoped. Never reuse `@eN` across tabs or after large DOM changes without a fresh snapshot.
+
+## Selector Fallback
+
+
+### Tier 3 Selector
+
+Use selectors only after semantic find and refs are unavailable or unreliable.
+
+```bash
+OMNIBOT_SESSION_TOKEN=checkout omnibot click "button[type=submit]" --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=checkout omnibot fill "input[name=email]" "a@b.com" --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=checkout omnibot focus "input[name=email]" --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=checkout omnibot get value "input[name=email]" --tab-id <TAB_ID>
+```
+
+Verify with `get`, `is`, `wait`, or `snapshot`.
+
+## DOM Fallback
+
+### Tier 4 DOM
+
+Use DOM node fallback when selectors exist but standard commands fail due overlays, custom event routing, or unstable selectors.
+
+```bash
+OMNIBOT_SESSION_TOKEN=repair omnibot dom visible --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=repair omnibot dom click n1 --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=repair omnibot dom dblclick n1 --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=repair omnibot dom scroll n1 --dy 800 --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=repair omnibot snapshot -i --tab-id <TAB_ID>
+```
+
+Do not use `dom` as the first click path. It bypasses higher-level intent and can target the wrong node if the visible-node list changes.
+
+## Mouse Fallback
+
+### Tier 5 Mouse
+
+Use coordinate mouse operations only when element-based actions cannot reach the target.
+
+```bash
+OMNIBOT_SESSION_TOKEN=repair omnibot get box "#canvas-button" --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=repair omnibot mouse click --x 100 --y 200 --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=repair omnibot snapshot -i --tab-id <TAB_ID>
+```
+
+Other mouse operations:
+
+```bash
+OMNIBOT_SESSION_TOKEN=repair omnibot mouse move --x 50 --y 60 --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=repair omnibot mouse scroll --x 100 --y 200 --dy 500 --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=repair omnibot mouse drag --from-x 10 --from-y 20 --to-x 100 --to-y 200 --tab-id <TAB_ID>
+```
+
+Mouse fallback is fragile across viewport size, scroll position, and responsive layouts. Verify immediately.
+
+## JavaScript Fallback
+
+### Tier 6 JavaScript
+
+Do not use `execute-js` first.
+
+Use `execute-js` only when:
+
+- Semantic find fails.
+- Snapshot refs are unstable.
+- CSS selector operation fails.
+- DOM fallback fails.
+- A complex frontend event must be triggered.
+- Runtime state cannot be read through `scan`, `full-scan`, `get`, or `is`.
+
+Read-only JavaScript fallback:
+
+```bash
+OMNIBOT_SESSION_TOKEN=repair omnibot execute-js "return window.appState?.status" --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=repair omnibot get text "#status" --tab-id <TAB_ID>
+```
+
+Input JavaScript fallback must trigger real browser events:
+
+```bash
+OMNIBOT_SESSION_TOKEN=repair omnibot execute-js "const el=document.querySelector('input[name=email]'); el.value='a@b.com'; el.dispatchEvent(new Event('input',{bubbles:true})); el.dispatchEvent(new Event('change',{bubbles:true})); return el.value;" --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=repair omnibot get value "input[name=email]" --tab-id <TAB_ID>
+```
+
+Click JavaScript fallback should call the element click only after higher tiers fail:
+
+```bash
+OMNIBOT_SESSION_TOKEN=repair omnibot execute-js "document.querySelector('button[type=submit]')?.click(); return true;" --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=repair omnibot wait --url "/dashboard" --tab-id <TAB_ID>
+```
+
+Prefer `--file` for longer scripts:
+
+```bash
+OMNIBOT_SESSION_TOKEN=repair omnibot execute-js --file /tmp/omnibot-repair.js --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=repair omnibot snapshot -i --tab-id <TAB_ID>
+```
+
+## Raw CDP Fallback
+
+### Tier 7 Raw CDP
+
+Raw CDP is the last resort. Use it only when standard commands and JavaScript fallback cannot complete or inspect the task.
+
+```bash
+OMNIBOT_SESSION_TOKEN=debug omnibot cdp Runtime.evaluate '{"expression":"document.title"}' --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=debug omnibot cdp DOM.getDocument '{"depth":1}' --tab-id <TAB_ID>
+```
+
+After CDP fallback, verify using normal Omnibot reads whenever possible:
+
+```bash
+OMNIBOT_SESSION_TOKEN=debug omnibot get title --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=debug omnibot snapshot -i --tab-id <TAB_ID>
+```
+
+CDP can bypass application-level event semantics. Treat it as expert-only infrastructure access.

package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/operation-patterns.md

@@ -0,0 +1,369 @@
+# Operation Patterns
+
+Use this file to choose behavior by task. Every page-state workflow must set `OMNIBOT_SESSION_TOKEN=<workflow-name>` and pass `--tab-id <TAB_ID>` on each page-state command.
+
+## Read
+
+### When to use
+
+Use Read when the agent needs current browser state, page text, a single value, visibility, enabled/checked state, layout box, or computed styles.
+
+### Preferred sequence
+
+1. `get title/url/text/html/value/attr/count/box/styles`
+2. `is visible/enabled/checked`
+3. `scan --json`
+4. `full-scan`
+5. `snapshot -i`
+6. `dom visible`
+7. `execute-js` fallback
+
+### Example
+
+```bash
+OMNIBOT_SESSION_TOKEN=research omnibot get title --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=research omnibot get url --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=research omnibot get text "#main" --tab-id <TAB_ID>
+```
+
+### Verification
+
+Use the narrowest read that proves the claim:
+
+```bash
+OMNIBOT_SESSION_TOKEN=research omnibot is visible "#main" --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=research omnibot get count ".result" --tab-id <TAB_ID>
+```
+
+### Fallback entry point
+
+If targeted reads fail or omit needed runtime state, go to `fallback-operations.md#fallback-tier-model`. Start with DOM fallback before JavaScript.
+
+## Click
+
+### When to use
+
+Use Click for buttons, links, menu items, cards, checkouts, modal actions, and any user-like click target.
+
+### Preferred sequence
+
+1. `find role/text/testid --action click`
+2. `snapshot -i` -> `click @eN`
+3. `click selector`
+4. `dom click`
+5. `mouse click`
+6. `execute-js` click
+7. `cdp` fallback
+
+### Example
+
+```bash
+OMNIBOT_SESSION_TOKEN=checkout omnibot snapshot -i --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=checkout omnibot find role button --name "Submit" --action click --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=checkout omnibot wait --url "/dashboard" --tab-id <TAB_ID>
+```
+
+If semantic find is ambiguous, use refs from the same tab:
+
+```bash
+OMNIBOT_SESSION_TOKEN=checkout omnibot snapshot -i --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=checkout omnibot click --tab-id <TAB_ID> @e4
+OMNIBOT_SESSION_TOKEN=checkout omnibot snapshot -i --tab-id <TAB_ID>
+```
+
+### Verification
+
+Verify the expected state, not just command success:
+
+```bash
+OMNIBOT_SESSION_TOKEN=checkout omnibot get url --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=checkout omnibot is visible ".success" --tab-id <TAB_ID>
+```
+
+### Fallback entry point
+
+If semantic and refs fail, document why and continue at `fallback-operations.md#selector-fallback`.
+
+## Fill
+
+### When to use
+
+Use Fill for text inputs, textareas, search boxes, login fields, and form fields where the final value must be controlled.
+
+### Preferred sequence
+
+1. `find label/placeholder --action fill`
+2. `snapshot -i` -> `fill @eN`
+3. `fill selector`
+4. `focus` + `type`
+5. `execute-js` set value and dispatch `input`/`change` events
+
+### Example
+
+```bash
+OMNIBOT_SESSION_TOKEN=checkout omnibot find label "Email" --action fill --action-value "a@b.com" --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=checkout omnibot get value "input[name=email]" --tab-id <TAB_ID>
+```
+
+With refs:
+
+```bash
+OMNIBOT_SESSION_TOKEN=checkout omnibot snapshot -i --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=checkout omnibot fill --tab-id <TAB_ID> @e2 "a@b.com"
+OMNIBOT_SESSION_TOKEN=checkout omnibot get value "input[name=email]" --tab-id <TAB_ID>
+```
+
+### Verification
+
+Use `get value`, visible validation text, enabled state, or submit readiness:
+
+```bash
+OMNIBOT_SESSION_TOKEN=checkout omnibot get value "input[name=email]" --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=checkout omnibot is enabled "button[type=submit]" --tab-id <TAB_ID>
+```
+
+### Fallback entry point
+
+If fill does not update the application state, use `fallback-operations.md#javascript-fallback` only after selector/focus/type attempts fail. JavaScript must dispatch real `input` and `change` events.
+
+## Select / Check
+
+### When to use
+
+Use this pattern for dropdowns, checkboxes, toggles, agreement boxes, filters, and boolean controls.
+
+### Preferred sequence
+
+1. Observe state with `snapshot -i` and `is checked` when relevant.
+2. Use `select`, `check`, or `uncheck` with refs or selectors.
+3. Verify with `get value`, `is checked`, visible text, or result count.
+
+### Example
+
+```bash
+OMNIBOT_SESSION_TOKEN=form omnibot snapshot -i --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=form omnibot select @e5 "US" --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=form omnibot get value "select[name=country]" --tab-id <TAB_ID>
+```
+
+Checkbox example:
+
+```bash
+OMNIBOT_SESSION_TOKEN=form omnibot is checked "#agree" --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=form omnibot check "#agree" --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=form omnibot is checked "#agree" --tab-id <TAB_ID>
+```
+
+### Verification
+
+For checkboxes, verify boolean state. For selects, verify value or dependent page state.
+
+### Fallback entry point
+
+If native select/check fails, use `fallback-operations.md#selector-fallback`, then JavaScript only if events must be synthesized.
+
+## Navigation
+
+### When to use
+
+Use Navigation to open pages, create tabs, change history, reload, close tabs, move between pages, create windows, or select frames.
+
+### Preferred sequence
+
+1. Create or discover a tab with `tabs`, `tab list`, `tab new`, `open`, or `navigate`.
+2. Save the returned tab id.
+3. Verify with `get url --tab-id <TAB_ID>`.
+4. Run all later page-state commands with `--tab-id <TAB_ID>`.
+
+### Example
+
+```bash
+OMNIBOT_SESSION_TOKEN=research omnibot tab new https://example.com --label research
+OMNIBOT_SESSION_TOKEN=research omnibot get url --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=research omnibot snapshot -i --tab-id <TAB_ID>
+```
+
+History and lifecycle:
+
+```bash
+OMNIBOT_SESSION_TOKEN=research omnibot back --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=research omnibot get url --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=research omnibot forward --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=research omnibot get url --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=research omnibot reload --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=research omnibot wait --load domcontentloaded --tab-id <TAB_ID>
+```
+
+Other navigation commands:
+
+```bash
+OMNIBOT_SESSION_TOKEN=research omnibot navigate https://example.com
+OMNIBOT_SESSION_TOKEN=research omnibot navigate --same-tab https://example.com
+OMNIBOT_SESSION_TOKEN=research omnibot open https://example.com
+OMNIBOT_SESSION_TOKEN=research omnibot goto https://example.com
+OMNIBOT_SESSION_TOKEN=research omnibot pushstate /dashboard --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=research omnibot close <TAB_ID>
+OMNIBOT_SESSION_TOKEN=research omnibot tab list
+OMNIBOT_SESSION_TOKEN=research omnibot tab close research
+OMNIBOT_SESSION_TOKEN=research omnibot window new
+OMNIBOT_SESSION_TOKEN=research omnibot frame main
+```
+
+### Verification
+
+Always verify the final target:
+
+```bash
+OMNIBOT_SESSION_TOKEN=research omnibot get url --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=research omnibot wait --load networkidle --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=research omnibot snapshot -i --tab-id <TAB_ID>
+```
+
+### Fallback entry point
+
+`switch-tab` cannot be used as a reliable base for multi-step workflows. Use it only for casual single-step work. If targeted same-tab navigation is unavailable, state the limitation, re-observe the tab list, and verify before continuing.
+
+## Wait
+
+### When to use
+
+Use Wait after navigation, clicks, form submissions, reloads, dynamic rendering, lazy loading, or any action that changes asynchronous state.
+
+### Preferred sequence
+
+1. Wait for selector visibility or hidden state.
+2. Wait for text.
+3. Wait for URL.
+4. Wait for load state.
+5. Wait for a JavaScript condition with `--fn`.
+6. Avoid shell sleep.
+
+### Example
+
+```bash
+OMNIBOT_SESSION_TOKEN=checkout omnibot click --tab-id <TAB_ID> @e4
+OMNIBOT_SESSION_TOKEN=checkout omnibot wait "#spinner" --state hidden --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=checkout omnibot wait --text "Welcome" --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=checkout omnibot get text "#main" --tab-id <TAB_ID>
+```
+
+Other waits:
+
+```bash
+OMNIBOT_SESSION_TOKEN=checkout omnibot wait --url "/dashboard" --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=checkout omnibot wait --load domcontentloaded --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=checkout omnibot wait --load networkidle --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=checkout omnibot wait --fn "window.appReady === true" --tab-id <TAB_ID>
+```
+
+### Verification
+
+After waiting, read the final condition with `get`, `is`, `scan`, or `snapshot`.
+
+### Fallback entry point
+
+If waits time out, collect evidence with `debugging-and-evidence.md`; do not replace them with shell sleep.
+
+## Extraction
+
+### When to use
+
+Use Extraction for retrieving page text, HTML fragments, links, counts, assets, or clipboard data.
+
+### Preferred sequence
+
+1. `get text/html/attr/count` for targeted extraction.
+2. `scan --json` for structured viewport extraction.
+3. `full-scan` for long or lazy pages.
+4. `assets list/export` for resources.
+5. `clipboard read` only when clipboard content is part of the task.
+
+### Example
+
+```bash
+OMNIBOT_SESSION_TOKEN=extract omnibot get text "#article" --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=extract omnibot get attr "a.next" href --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=extract omnibot get count ".result" --tab-id <TAB_ID>
+```
+
+Long page example:
+
+```bash
+OMNIBOT_SESSION_TOKEN=extract omnibot full-scan --duration 5 --text-only --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=extract omnibot assets list --tab-id <TAB_ID>
+```
+
+### Verification
+
+Verify extraction completeness with count, repeated scan after scroll, or targeted reads for expected markers.
+
+### Fallback entry point
+
+If content exists only in runtime objects, use `fallback-operations.md#javascript-fallback` after explaining why `scan`, `full-scan`, and `get` could not access it.
+
+## Upload
+
+### When to use
+
+Use Upload when a browser form needs a local file attached through a file input.
+
+### Preferred sequence
+
+1. Observe the file input or upload control.
+2. Use `upload` against the input selector.
+3. Verify that the uploaded file appeared or form state changed.
+
+### Example
+
+```bash
+OMNIBOT_SESSION_TOKEN=form omnibot snapshot -i --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=form omnibot upload "input[type=file]" /path/to/file.png --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=form omnibot wait --text "file.png" --tab-id <TAB_ID>
+```
+
+### Verification
+
+Use visible filename, count, enabled submit state, or application-specific confirmation:
+
+```bash
+OMNIBOT_SESSION_TOKEN=form omnibot is enabled "button[type=submit]" --tab-id <TAB_ID>
+```
+
+### Fallback entry point
+
+If upload input is hidden behind custom UI, click the semantic upload control first. Use JavaScript only after standard upload cannot reach the input.
+
+## Batch
+
+### When to use
+
+Use Batch for short, known-safe command chains where the target tab and verification are already explicit.
+
+### Preferred sequence
+
+1. Keep batches short.
+2. Include only deterministic operations.
+3. Verify after the batch with a separate command.
+4. Use `--file` for complex JSON quoting.
+
+### Example
+
+```bash
+OMNIBOT_SESSION_TOKEN=research omnibot batch '[{"cmd":"scan"},{"cmd":"snapshot","interactive":true}]' --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=research omnibot snapshot -i --tab-id <TAB_ID>
+```
+
+File example:
+
+```bash
+OMNIBOT_SESSION_TOKEN=research omnibot batch --file /tmp/omnibot-batch.json --tab-id <TAB_ID>
+OMNIBOT_SESSION_TOKEN=research omnibot get url --tab-id <TAB_ID>
+```
+
+### Verification
+
+Do not treat batch success as task success. Verify final page state separately.
+
+### Fallback entry point
+
+Batch is not a fallback tier. If a batched operation fails, re-run the workflow step-by-step and enter the normal fallback model.

package/bin/omnibot-windows-x64/omnibot/skills/omnibot/references/runtime-and-status.md

@@ -0,0 +1,89 @@
+# Runtime and Status
+
+Use this file before troubleshooting. Runtime checks establish whether the daemon, extension, browser, visibility mode, license, and packaged skills are healthy.
+
+## Startup Check
+
+Run this sequence before diagnosing browser failures:
+
+```bash
+omnibot doctor
+omnibot tabs
+omnibot visibility status
+omnibot browser current
+omnibot license status
+```
+
+Interpretation:
+
+- `doctor` checks daemon and extension health.
+- `tabs` proves connected tabs are visible and provides tab ids.
+- `visibility status` shows whether automation is visible, background, dedicated-profile, or headless.
+- `browser current` shows current browser runtime ownership.
+- `license status` verifies license state.
+
+If the extension is not connected, open Chrome or Edge with the omnibot extension loaded and keep an HTTP/HTTPS page open.
+
+## doctor
+
+```bash
+omnibot doctor
+```
+
+Use `doctor` first when commands fail, tabs are empty, screenshots fail, or the agent cannot reach the browser.
+
+## tabs
+
+```bash
+omnibot tabs
+```
+
+Use `tabs` to discover tab ids. Save the target tab id and use it on every page-state command:
+
+```bash
+OMNIBOT_SESSION_TOKEN=research omnibot snapshot -i --tab-id <TAB_ID>
+```
+
+## browser list/current
+
+```bash
+omnibot browser list
+omnibot browser current
+```
+
+Use browser status when multiple browser runtimes or agents may be active.
+
+## visibility status
+
+```bash
+omnibot visibility status
+```
+
+Use visibility status to confirm whether automation should share the user's visible browser state. Headless and dedicated-profile modes do not automatically inherit user login.
+
+## license status
+
+```bash
+omnibot license status
+```
+
+Use license status when runtime checks pass but features appear unavailable.
+
+## skills path
+
+```bash
+omnibot skills path
+```
+
+Use skills path to locate packaged skills for installation or inspection.
+
+## Skills Install
+
+```bash
+omnibot skills install --agent hermes --profile nuwa
+omnibot skills install --agent opencode
+omnibot skills install --agent claude
+omnibot skills install --agent codex
+```
+
+Install only when setting up or repairing an agent integration. It is not part of normal page operations.