@weavetab/skills 1.0.3 → 2.5.0-beta.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +108 -60
- package/ROUTER.md +43 -0
- package/bin/cli.js +232 -132
- package/browser/dom-strategy.md +119 -0
- package/browser/form-automation.md +87 -0
- package/browser/interaction-pipeline.md +216 -0
- package/browser/tab-and-window.md +136 -0
- package/desktop/desktop-targeting.md +85 -0
- package/desktop/electron-discovery.md +115 -0
- package/developer/github-forensics.md +74 -0
- package/dist/bundled/weavetab-all.md +1910 -0
- package/dist/bundled/weavetab-browser.md +501 -0
- package/dist/bundled/weavetab-desktop.md +176 -0
- package/dist/bundled/weavetab-developer.md +67 -0
- package/dist/bundled/weavetab-documents.md +142 -0
- package/dist/bundled/weavetab-memory.md +359 -0
- package/dist/bundled/weavetab-network-and-perf.md +223 -0
- package/dist/bundled/weavetab-plugins.md +83 -0
- package/dist/bundled/weavetab-resilience.md +216 -0
- package/dist/bundled/weavetab-security.md +87 -0
- package/dist/granular/browser_dom-strategy.md +119 -0
- package/dist/granular/browser_form-automation.md +87 -0
- package/dist/granular/browser_interaction-pipeline.md +216 -0
- package/dist/granular/browser_tab-and-window.md +136 -0
- package/dist/granular/desktop_desktop-targeting.md +85 -0
- package/dist/granular/desktop_electron-discovery.md +115 -0
- package/dist/granular/developer_github-forensics.md +74 -0
- package/dist/granular/documents_canvas-and-visual-capture.md +103 -0
- package/dist/granular/documents_office-document-ingestion.md +63 -0
- package/dist/granular/memory_selector-memory-system.md +127 -0
- package/dist/granular/memory_session-trail.md +95 -0
- package/dist/granular/memory_site-learning.md +180 -0
- package/dist/granular/network-and-perf_device-and-environment.md +87 -0
- package/dist/granular/network-and-perf_network-intercept-and-mock.md +84 -0
- package/dist/granular/network-and-perf_performance-and-vitals.md +84 -0
- package/dist/granular/plugins_hooks-and-overrides.md +35 -0
- package/dist/granular/plugins_plugin-runtime.md +63 -0
- package/dist/granular/resilience_loop-breaker.md +67 -0
- package/dist/granular/resilience_session-time-travel.md +131 -0
- package/dist/granular/resilience_stale-ref-recovery.md +53 -0
- package/dist/granular/security_rbac-and-policies.md +47 -0
- package/dist/granular/security_secret-containment-protocol.md +59 -0
- package/documents/canvas-and-visual-capture.md +103 -0
- package/documents/office-document-ingestion.md +63 -0
- package/evals/evaluator.js +146 -0
- package/evals/scenarios.json +239 -0
- package/memory/selector-memory-system.md +127 -0
- package/memory/session-trail.md +95 -0
- package/memory/site-learning.md +180 -0
- package/network-and-perf/device-and-environment.md +87 -0
- package/network-and-perf/network-intercept-and-mock.md +84 -0
- package/network-and-perf/performance-and-vitals.md +84 -0
- package/package.json +31 -9
- package/plugins/hooks-and-overrides.md +35 -0
- package/plugins/plugin-runtime.md +63 -0
- package/resilience/loop-breaker.md +67 -0
- package/resilience/session-time-travel.md +131 -0
- package/resilience/stale-ref-recovery.md +53 -0
- package/security/rbac-and-policies.md +47 -0
- package/security/secret-containment-protocol.md +59 -0
- package/skills.json +321 -203
- package/src/api.js +239 -128
- package/advanced/README.md +0 -2
- package/automation/README.md +0 -2
- package/general/captcha-evasion.md +0 -13
- package/general/context-awareness.md +0 -13
- package/general/cookie-consent.md +0 -13
- package/general/data-extraction.md +0 -13
- package/general/dynamic-interaction.md +0 -13
- package/general/error-recovery.md +0 -13
- package/general/file-upload.md +0 -13
- package/general/form-filling.md +0 -16
- package/general/guardrails.md +0 -13
- package/general/iframe-handling.md +0 -13
- package/general/infinite-scroll.md +0 -13
- package/general/login-flow.md +0 -16
- package/general/multi-tab-management.md +0 -13
- package/general/spa-navigation.md +0 -13
- package/general/table-scraping.md +0 -13
|
@@ -0,0 +1,119 @@
|
|
|
1
|
+
---
|
|
2
|
+
id: dom-strategy
|
|
3
|
+
domain: browser
|
|
4
|
+
triggers:
|
|
5
|
+
- "read page"
|
|
6
|
+
- "inspect dom"
|
|
7
|
+
- "find element"
|
|
8
|
+
- "parse webpage"
|
|
9
|
+
- "locate button"
|
|
10
|
+
- "extract text"
|
|
11
|
+
tools:
|
|
12
|
+
- "browser_map"
|
|
13
|
+
- "browser_find"
|
|
14
|
+
- "browser_scrape"
|
|
15
|
+
- "browser_snapshot"
|
|
16
|
+
- "browser_eval"
|
|
17
|
+
weavetab: ">=2.5.0-beta.5"
|
|
18
|
+
---
|
|
19
|
+
|
|
20
|
+
# DOM Strategy & Token Conservation Engine
|
|
21
|
+
|
|
22
|
+
When parsing modern web applications, naive DOM dumps can consume 50,000+ tokens and cause agent degradation. Weavetab provides a high-efficiency 4-tier reading architecture.
|
|
23
|
+
|
|
24
|
+
---
|
|
25
|
+
|
|
26
|
+
## 1. The 4-Tier Reading Filter
|
|
27
|
+
|
|
28
|
+
Always select the narrowest tool that satisfies your immediate objective:
|
|
29
|
+
|
|
30
|
+
```
|
|
31
|
+
┌───────────────────────────────┐
|
|
32
|
+
│ Do you need full structural │
|
|
33
|
+
│ context or just a target? │
|
|
34
|
+
└──────────────┬────────────────┘
|
|
35
|
+
│
|
|
36
|
+
┌───────────────┴───────────────┐
|
|
37
|
+
▼ ▼
|
|
38
|
+
[Specific Target] [Whole Document]
|
|
39
|
+
│ │
|
|
40
|
+
┌────────┴────────┐ ┌───────┴────────┐
|
|
41
|
+
▼ ▼ ▼ ▼
|
|
42
|
+
Target by text Target by role Read articles Interactive
|
|
43
|
+
or keywords or hierarchy or prose data UI layout
|
|
44
|
+
│ │ │ │
|
|
45
|
+
▼ ▼ ▼ ▼
|
|
46
|
+
`browser_find` `browser_map` `browser_scrape` `browser_map`
|
|
47
|
+
(lite: true) (lite: true)
|
|
48
|
+
│
|
|
49
|
+
[Sparse / Canvas / WebGL / <5 elements?]
|
|
50
|
+
│
|
|
51
|
+
▼
|
|
52
|
+
`browser_map`
|
|
53
|
+
(visual: "auto")
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
### Tier 1: Targeted Keyword Jump (`browser_find`)
|
|
57
|
+
Use `browser_find` when you already know the label, text snippet, placeholder, or aria-label of the target element.
|
|
58
|
+
- **Latency**: Sub-5ms CDP search.
|
|
59
|
+
- **Token Cost**: ~50–100 tokens (returns only matching candidate nodes).
|
|
60
|
+
|
|
61
|
+
```json
|
|
62
|
+
{
|
|
63
|
+
"query": "Sign In",
|
|
64
|
+
"matchType": "contains",
|
|
65
|
+
"visibleOnly": true
|
|
66
|
+
}
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
### Tier 2: Token-Optimized DOM Walk (`browser_map` with `lite: true`)
|
|
70
|
+
Use `browser_map` when you must understand the interactive structure of the page.
|
|
71
|
+
- **Rule**: ALWAYS pass `"lite": true` unless you strictly need deeply nested non-interactive DOM trees.
|
|
72
|
+
- **Efficiency**: Strips decorative wrappers, SVG paths, hidden containers, and scripts. Reduces payload by **85% to 95%**.
|
|
73
|
+
- **Viewport Filtering**: Set `"includeOffscreen": false` to only inspect elements currently visible in the active viewport.
|
|
74
|
+
|
|
75
|
+
```json
|
|
76
|
+
{
|
|
77
|
+
"lite": true,
|
|
78
|
+
"includeOffscreen": false,
|
|
79
|
+
"maxDepth": 6
|
|
80
|
+
}
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
### Tier 3: Markdown Prose & Data Ingestion (`browser_scrape`)
|
|
84
|
+
Use `browser_scrape` when your objective is reading articles, blog posts, documentation, or legal terms rather than clicking buttons.
|
|
85
|
+
- Converts HTML to clean, semantically structured Markdown.
|
|
86
|
+
- Automatically handles readability filtering and removes advertising/cookie banners.
|
|
87
|
+
|
|
88
|
+
### Tier 4: Set-of-Marks Visual Grounding (`browser_map` with `visual: "auto"`)
|
|
89
|
+
Use when target pages are canvas-heavy, WebGL, Figma, Google Sheets, or when standard `browser_map` returns < 5 interactive elements.
|
|
90
|
+
- Injects temporary numbered badges `[1]…[N]` over interactive elements & canvas regions.
|
|
91
|
+
- Captures an annotated screenshot + returns a structured mark list.
|
|
92
|
+
- **Zero Page Pollution**: Strips badges immediately after capture.
|
|
93
|
+
- **Action**: Pass `"mark": N` to `browser_click` to click the exact element without re-scanning the DOM.
|
|
94
|
+
|
|
95
|
+
```json
|
|
96
|
+
{
|
|
97
|
+
"visual": "auto"
|
|
98
|
+
}
|
|
99
|
+
```
|
|
100
|
+
|
|
101
|
+
### Tier 5: Runtime State & JavaScript Evaluation (`browser_eval`)
|
|
102
|
+
When DOM nodes are occluded or you need direct access to in-memory window state (e.g. Redux store, localStorage keys, or complex canvas dimensions), use `browser_eval` as an escape hatch:
|
|
103
|
+
```json
|
|
104
|
+
{
|
|
105
|
+
"expression": "window.__INITIAL_STATE__ || document.title"
|
|
106
|
+
}
|
|
107
|
+
```
|
|
108
|
+
|
|
109
|
+
---
|
|
110
|
+
|
|
111
|
+
## 2. Volatile Ref ID & Visual Mark Navigation
|
|
112
|
+
|
|
113
|
+
Weavetab maps elements to ephemeral handles (`w:NN`) and visual marks (`[1]…[N]`):
|
|
114
|
+
|
|
115
|
+
### Operational Rules:
|
|
116
|
+
1. **Never Construct Speculative CSS Selectors**: If an element has ref `w:14`, pass `"ref": "w:14"` to `browser_click`, `browser_fill`, `browser_type`.
|
|
117
|
+
2. **Visual Mark Targeting**: On canvas or unlabelled elements from `visual: "auto"`, pass `"mark": N` directly to `browser_click({ mark: N })`.
|
|
118
|
+
3. **Lifespan of Handles**: Ref IDs and marks invalidate on page navigation (`browser_navigate`) or heavy SPA re-renders.
|
|
119
|
+
4. **Recovery on Invalidation**: If an action fails with `ELEMENT_NOT_FOUND` or `MARK_NOT_FOUND`, re-run `browser_map({ visual: "auto" })` to refresh active handles.
|
|
@@ -0,0 +1,87 @@
|
|
|
1
|
+
---
|
|
2
|
+
id: form-automation
|
|
3
|
+
domain: browser
|
|
4
|
+
triggers:
|
|
5
|
+
- "fill form"
|
|
6
|
+
- "submit form"
|
|
7
|
+
- "select dropdown"
|
|
8
|
+
- "multi field input"
|
|
9
|
+
- "radio button"
|
|
10
|
+
- "checkbox"
|
|
11
|
+
tools:
|
|
12
|
+
- "browser_fill"
|
|
13
|
+
- "browser_select"
|
|
14
|
+
weavetab: ">=2.5.0-beta.5"
|
|
15
|
+
---
|
|
16
|
+
|
|
17
|
+
# Form Automation: Atomic Multi-Field Execution
|
|
18
|
+
|
|
19
|
+
Submitting complex web forms one field at a time wastes agent turns and increases the probability of layout shifts. Weavetab provides atomic multi-field batching and native dropdown selectors.
|
|
20
|
+
|
|
21
|
+
---
|
|
22
|
+
|
|
23
|
+
## 1. Atomic Multi-Field Filling (`browser_fill`)
|
|
24
|
+
|
|
25
|
+
`browser_fill` populates multiple inputs, textareas, checkboxes, and radio buttons in a single CDP roundtrip.
|
|
26
|
+
|
|
27
|
+
```json
|
|
28
|
+
{
|
|
29
|
+
"fields": [
|
|
30
|
+
{
|
|
31
|
+
"ref": "w:12",
|
|
32
|
+
"value": "Anas"
|
|
33
|
+
},
|
|
34
|
+
{
|
|
35
|
+
"ref": "w:14",
|
|
36
|
+
"value": "Khezaz"
|
|
37
|
+
},
|
|
38
|
+
{
|
|
39
|
+
"ref": "w:16",
|
|
40
|
+
"value": "contact@weavetab.com"
|
|
41
|
+
},
|
|
42
|
+
{
|
|
43
|
+
"ref": "w:19",
|
|
44
|
+
"value": true
|
|
45
|
+
}
|
|
46
|
+
]
|
|
47
|
+
}
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
### Supported Field Value Types:
|
|
51
|
+
- **Text / Search / Email / Tel**: Provide a `string`.
|
|
52
|
+
- **Checkbox**: Provide a `boolean` (`true` to check, `false` to uncheck).
|
|
53
|
+
- **Radio Buttons**: Provide `true` on the matching option ref.
|
|
54
|
+
- **ContentEditable**: Provide a `string` (automatically handles inner text formatting).
|
|
55
|
+
|
|
56
|
+
---
|
|
57
|
+
|
|
58
|
+
## 2. Dropdown & Select Automation (`browser_select`)
|
|
59
|
+
|
|
60
|
+
Standard `<select>` elements and custom dropdowns often fail under simple click-and-type automation. `browser_select` dispatches native CDP change events.
|
|
61
|
+
|
|
62
|
+
```json
|
|
63
|
+
{
|
|
64
|
+
"ref": "w:31",
|
|
65
|
+
"values": ["enterprise_tier"]
|
|
66
|
+
}
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
### Multi-Select Support:
|
|
70
|
+
For `<select multiple>` elements, pass an array of option values:
|
|
71
|
+
|
|
72
|
+
```json
|
|
73
|
+
{
|
|
74
|
+
"ref": "w:35",
|
|
75
|
+
"values": ["us-east-1", "eu-central-1"]
|
|
76
|
+
}
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
---
|
|
80
|
+
|
|
81
|
+
## 3. Form Submission Best Practices
|
|
82
|
+
|
|
83
|
+
1. **Pre-Validate State**: Before clicking Submit, confirm that required fields have passed HTML5/client-side validation (no active red borders or validation popups).
|
|
84
|
+
2. **Submit via Enter Key vs Click**:
|
|
85
|
+
- If the submit button has a dynamic ref or changes during typing, send `"key": "Enter"` via `browser_key` inside the last filled text field.
|
|
86
|
+
- Otherwise, locate the submit button via `browser_find` and click via `browser_click`.
|
|
87
|
+
3. **Wait for Post-Submission Route**: Always follow form submission with `browser_wait` for URL changes or target DOM confirmation rather than immediately firing subsequent actions.
|
|
@@ -0,0 +1,216 @@
|
|
|
1
|
+
---
|
|
2
|
+
id: interaction-pipeline
|
|
3
|
+
domain: browser
|
|
4
|
+
triggers:
|
|
5
|
+
- "click element"
|
|
6
|
+
- "type text"
|
|
7
|
+
- "press key"
|
|
8
|
+
- "drag and drop"
|
|
9
|
+
- "hover"
|
|
10
|
+
- "upload file"
|
|
11
|
+
tools:
|
|
12
|
+
- "browser_click"
|
|
13
|
+
- "browser_type"
|
|
14
|
+
- "browser_key"
|
|
15
|
+
- "browser_pointer"
|
|
16
|
+
- "browser_upload"
|
|
17
|
+
- "browser_scroll"
|
|
18
|
+
- "browser_dialog"
|
|
19
|
+
- "browser_burst"
|
|
20
|
+
- "browser_highlight"
|
|
21
|
+
- "browser_clipboard"
|
|
22
|
+
weavetab: ">=2.5.0-beta.5"
|
|
23
|
+
---
|
|
24
|
+
|
|
25
|
+
# Interaction Pipeline: Physical & Synthetic Input Engine
|
|
26
|
+
|
|
27
|
+
Weavetab executes browser interactions via direct Chrome DevTools Protocol (CDP) dispatch, bypassing synthetic JavaScript event limitations and anti-automation tripwires.
|
|
28
|
+
|
|
29
|
+
---
|
|
30
|
+
|
|
31
|
+
## 1. Clicking Elements (`browser_click`)
|
|
32
|
+
|
|
33
|
+
Execute deterministic clicks on DOM elements using volatile handles (`ref`), CSS selectors, or exact coordinates.
|
|
34
|
+
|
|
35
|
+
```json
|
|
36
|
+
{
|
|
37
|
+
"ref": "w:23",
|
|
38
|
+
"clickCount": 1,
|
|
39
|
+
"button": "left"
|
|
40
|
+
}
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
### Advanced Modifiers:
|
|
44
|
+
- **Double Click**: Set `"clickCount": 2` for inline editing or selecting text blocks.
|
|
45
|
+
- **Context Menu**: Set `"button": "right"` to trigger native context menus.
|
|
46
|
+
- **Modifier Keys**: Pass `"modifiers": ["Control"]` or `["Meta"]` for multi-selection or opening links in background tabs.
|
|
47
|
+
|
|
48
|
+
---
|
|
49
|
+
|
|
50
|
+
## 2. Text Input (`browser_type`)
|
|
51
|
+
|
|
52
|
+
Simulate natural keystrokes into input fields, textareas, and `contenteditable` elements.
|
|
53
|
+
|
|
54
|
+
```json
|
|
55
|
+
{
|
|
56
|
+
"ref": "w:18",
|
|
57
|
+
"text": "Enterprise Search Query",
|
|
58
|
+
"clear": true,
|
|
59
|
+
"delay": 20
|
|
60
|
+
}
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
> [!CAUTION]
|
|
64
|
+
> **Secret Boundary Enforcement**:
|
|
65
|
+
> NEVER use `browser_type` for passwords, API tokens, or payment card data. Always route sensitive credentials through [`security/secret-containment-protocol.md`](file:///c:/Users/fy2ne/Music/Weavetab/skills/security/secret-containment-protocol.md) via `browser_type_secret`.
|
|
66
|
+
|
|
67
|
+
---
|
|
68
|
+
|
|
69
|
+
## 3. Physical Pointer Trajectories (`browser_pointer`)
|
|
70
|
+
|
|
71
|
+
Handle complex drag-and-drop interactions, canvas sketching, hover-activated navigation flyouts, and custom sliders.
|
|
72
|
+
|
|
73
|
+
### Hover / Mouse Move:
|
|
74
|
+
```json
|
|
75
|
+
{
|
|
76
|
+
"action": "move",
|
|
77
|
+
"x": 450,
|
|
78
|
+
"y": 180
|
|
79
|
+
}
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
### Drag & Drop Pipeline:
|
|
83
|
+
```json
|
|
84
|
+
// Step 1: Move to origin & press mouse
|
|
85
|
+
{ "action": "move", "x": 200, "y": 300 }
|
|
86
|
+
{ "action": "down", "button": "left" }
|
|
87
|
+
|
|
88
|
+
// Step 2: Smooth trajectory to destination
|
|
89
|
+
{ "action": "move", "x": 600, "y": 300 }
|
|
90
|
+
|
|
91
|
+
// Step 3: Release
|
|
92
|
+
{ "action": "up", "button": "left" }
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
---
|
|
96
|
+
|
|
97
|
+
## 4. Keystrokes & Shortcuts (`browser_key`)
|
|
98
|
+
|
|
99
|
+
Trigger keyboard shortcuts, navigation keys, and dialog dismissals.
|
|
100
|
+
|
|
101
|
+
```json
|
|
102
|
+
{
|
|
103
|
+
"key": "Enter"
|
|
104
|
+
}
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
### High-Frequency Key Combos:
|
|
108
|
+
- **Select All & Delete**: Send `"key": "a"`, `"modifiers": ["Control"]` (or `["Meta"]` on macOS), followed by `"key": "Backspace"`.
|
|
109
|
+
- **Dismiss Overlays**: Send `"key": "Escape"`.
|
|
110
|
+
- **Form Tab Navigation**: Send `"key": "Tab"`.
|
|
111
|
+
|
|
112
|
+
---
|
|
113
|
+
|
|
114
|
+
## 5. OS File Upload Bypass (`browser_upload`)
|
|
115
|
+
|
|
116
|
+
Native operating system file picker dialogs block autonomous agent execution because they operate outside the browser DOM. Weavetab bypasses this entirely using CDP's `DOM.setFileInputFiles`.
|
|
117
|
+
|
|
118
|
+
```json
|
|
119
|
+
{
|
|
120
|
+
"ref": "w:41",
|
|
121
|
+
"files": [
|
|
122
|
+
"C:/Users/fy2ne/Documents/contract.pdf",
|
|
123
|
+
"C:/Users/fy2ne/Documents/financial_report.xlsx"
|
|
124
|
+
]
|
|
125
|
+
}
|
|
126
|
+
```
|
|
127
|
+
|
|
128
|
+
- **Targeting**: Supply the volatile handle `ref` or CSS selector for the `<input type="file">`.
|
|
129
|
+
- **Hidden Inputs**: `browser_upload` automatically handles `display: none` file inputs commonly used in custom drag-and-drop upload widgets.
|
|
130
|
+
|
|
131
|
+
---
|
|
132
|
+
|
|
133
|
+
## 6. Viewport & Element Scrolling (`browser_scroll`)
|
|
134
|
+
|
|
135
|
+
Scroll the entire page viewport or scroll specific scrollable containers:
|
|
136
|
+
|
|
137
|
+
```json
|
|
138
|
+
{
|
|
139
|
+
"direction": "down",
|
|
140
|
+
"amount": 500
|
|
141
|
+
}
|
|
142
|
+
```
|
|
143
|
+
|
|
144
|
+
Or scroll a specific element container:
|
|
145
|
+
```json
|
|
146
|
+
{
|
|
147
|
+
"ref": "w:12",
|
|
148
|
+
"direction": "down",
|
|
149
|
+
"amount": 300
|
|
150
|
+
}
|
|
151
|
+
```
|
|
152
|
+
|
|
153
|
+
---
|
|
154
|
+
|
|
155
|
+
## 7. Native Dialog Handling (`browser_dialog`)
|
|
156
|
+
|
|
157
|
+
Handle native JavaScript `alert`, `confirm`, `prompt`, or `beforeunload` dialogs that block page execution:
|
|
158
|
+
|
|
159
|
+
```json
|
|
160
|
+
{
|
|
161
|
+
"action": "accept",
|
|
162
|
+
"promptText": "Confirmed"
|
|
163
|
+
}
|
|
164
|
+
```
|
|
165
|
+
|
|
166
|
+
---
|
|
167
|
+
|
|
168
|
+
## 8. High-Speed Action Batches (`browser_burst`)
|
|
169
|
+
|
|
170
|
+
Execute multiple sequential micro-actions in a single CDP roundtrip without waiting for intermediate LLM turns:
|
|
171
|
+
|
|
172
|
+
```json
|
|
173
|
+
{
|
|
174
|
+
"steps": [
|
|
175
|
+
{ "tool": "browser_fill", "ref": "w:10", "value": "search query" },
|
|
176
|
+
{ "tool": "click_and_wait", "ref": "w:15" }
|
|
177
|
+
]
|
|
178
|
+
}
|
|
179
|
+
```
|
|
180
|
+
*(Accepts both `steps` and `actions`, with arguments either nested under `args` or specified flat on the step object).*
|
|
181
|
+
|
|
182
|
+
---
|
|
183
|
+
|
|
184
|
+
## 9. Visual Element Highlighting & Consideration (`browser_highlight`)
|
|
185
|
+
|
|
186
|
+
Draw visual consideration, target lock, danger/success, or shimmer overlays without mutating the DOM. Supports multi-element candidate evaluation:
|
|
187
|
+
|
|
188
|
+
```json
|
|
189
|
+
{
|
|
190
|
+
"refs": ["w:12", "w:15", "w:23"],
|
|
191
|
+
"style": "consideration",
|
|
192
|
+
"label": "Evaluating form options",
|
|
193
|
+
"duration": 2000
|
|
194
|
+
}
|
|
195
|
+
```
|
|
196
|
+
|
|
197
|
+
### Presentation Styles:
|
|
198
|
+
- `"consideration"`: Soft pulsing cyan/blue outline (`#38bdf8`) with ambient glow — ideal when weighing multiple interactive candidates.
|
|
199
|
+
- `"target"`: High-contrast amber lock-on ring (`#f59e0b`) before executing high-impact clicks.
|
|
200
|
+
- `"danger"`: Red warning border (`#ef4444`) for destructive buttons (Delete, Reset, Purge).
|
|
201
|
+
- `"success"`: Emerald glow (`#10b981`) confirming completed actions.
|
|
202
|
+
- `"shimmer"`: Rapid purple accent wave (default for quick inspection).
|
|
203
|
+
|
|
204
|
+
---
|
|
205
|
+
|
|
206
|
+
## 10. System Clipboard Bridging (`browser_clipboard`)
|
|
207
|
+
|
|
208
|
+
Read or write system clipboard text during automation:
|
|
209
|
+
|
|
210
|
+
```json
|
|
211
|
+
{
|
|
212
|
+
"action": "write",
|
|
213
|
+
"text": "AuthToken_xyz987"
|
|
214
|
+
}
|
|
215
|
+
```
|
|
216
|
+
|
|
@@ -0,0 +1,136 @@
|
|
|
1
|
+
---
|
|
2
|
+
id: tab-and-window
|
|
3
|
+
domain: browser
|
|
4
|
+
triggers:
|
|
5
|
+
- "switch tab"
|
|
6
|
+
- "open tab"
|
|
7
|
+
- "close tab"
|
|
8
|
+
- "list tabs"
|
|
9
|
+
- "tab crash recovery"
|
|
10
|
+
- "heal tabs"
|
|
11
|
+
- "desktop app tabs"
|
|
12
|
+
tools:
|
|
13
|
+
- "browser_tabs"
|
|
14
|
+
weavetab: ">=2.5.0-beta.5"
|
|
15
|
+
---
|
|
16
|
+
|
|
17
|
+
# Tab & Window Lifecycle Management
|
|
18
|
+
|
|
19
|
+
Multi-tab workflows (OAuth popups, payment gateways, documentation references, desktop application windows) require deterministic tab switching, window focusing, and failure recovery. `browser_tabs` controls all CDP page and window targets.
|
|
20
|
+
|
|
21
|
+
---
|
|
22
|
+
|
|
23
|
+
## 1. Tab Discovery & Target Enumeration
|
|
24
|
+
|
|
25
|
+
Inspect all active targets, open URLs, and page titles:
|
|
26
|
+
|
|
27
|
+
```json
|
|
28
|
+
{
|
|
29
|
+
"action": "list"
|
|
30
|
+
}
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
To list targets inside a specific desktop app (e.g. Discord, Slack):
|
|
34
|
+
```json
|
|
35
|
+
{
|
|
36
|
+
"action": "list",
|
|
37
|
+
"app": "discord"
|
|
38
|
+
}
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
> [!TIP]
|
|
42
|
+
> Always pass `"app": "<name>"` when inspecting desktop applications. This guarantees that Weavetab does not launch or query the web browser.
|
|
43
|
+
|
|
44
|
+
---
|
|
45
|
+
|
|
46
|
+
## 2. Opening Tabs & Smart Reuse
|
|
47
|
+
|
|
48
|
+
Open a target URL. By default, `reuse: true` will focus an existing tab if the URL is already loaded:
|
|
49
|
+
|
|
50
|
+
```json
|
|
51
|
+
{
|
|
52
|
+
"action": "open",
|
|
53
|
+
"url": "https://github.com"
|
|
54
|
+
}
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
To force opening a brand-new duplicate tab instead of reusing:
|
|
58
|
+
```json
|
|
59
|
+
{
|
|
60
|
+
"action": "open",
|
|
61
|
+
"url": "https://github.com",
|
|
62
|
+
"reuse": false
|
|
63
|
+
}
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
---
|
|
67
|
+
|
|
68
|
+
## 3. Context Switching
|
|
69
|
+
|
|
70
|
+
Switch active agent focus between open tabs:
|
|
71
|
+
|
|
72
|
+
```json
|
|
73
|
+
{
|
|
74
|
+
"action": "switch",
|
|
75
|
+
"tabId": "TARGET_PAGE_ID"
|
|
76
|
+
}
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
---
|
|
80
|
+
|
|
81
|
+
## 4. Closing Tabs & Batch Cleanup
|
|
82
|
+
|
|
83
|
+
Close a single tab:
|
|
84
|
+
```json
|
|
85
|
+
{
|
|
86
|
+
"action": "close",
|
|
87
|
+
"tabId": "TARGET_PAGE_ID"
|
|
88
|
+
}
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
Close multiple tabs simultaneously in a single round-trip:
|
|
92
|
+
```json
|
|
93
|
+
{
|
|
94
|
+
"action": "close",
|
|
95
|
+
"tabIds": ["TARGET_1", "TARGET_2", "TARGET_3"]
|
|
96
|
+
}
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
---
|
|
100
|
+
|
|
101
|
+
## 5. Native Window Management
|
|
102
|
+
|
|
103
|
+
List native OS windows:
|
|
104
|
+
```json
|
|
105
|
+
{
|
|
106
|
+
"action": "list_windows"
|
|
107
|
+
}
|
|
108
|
+
```
|
|
109
|
+
|
|
110
|
+
Bring a specific native window to the foreground:
|
|
111
|
+
```json
|
|
112
|
+
{
|
|
113
|
+
"action": "focus_window",
|
|
114
|
+
"windowId": 1
|
|
115
|
+
}
|
|
116
|
+
```
|
|
117
|
+
|
|
118
|
+
---
|
|
119
|
+
|
|
120
|
+
## 6. Duplicate Consolidation & Auto-Heal
|
|
121
|
+
|
|
122
|
+
Deduplicate redundant tabs across the session:
|
|
123
|
+
```json
|
|
124
|
+
{
|
|
125
|
+
"action": "consolidate"
|
|
126
|
+
}
|
|
127
|
+
```
|
|
128
|
+
|
|
129
|
+
If a target page crashes or detaches due to memory limits:
|
|
130
|
+
```json
|
|
131
|
+
{
|
|
132
|
+
"action": "heal"
|
|
133
|
+
}
|
|
134
|
+
```
|
|
135
|
+
|
|
136
|
+
The auto-heal routine re-establishes CDP protocol sessions, reconnects event listeners, and synchronizes the active page target.
|
|
@@ -0,0 +1,85 @@
|
|
|
1
|
+
---
|
|
2
|
+
id: desktop-targeting
|
|
3
|
+
domain: desktop
|
|
4
|
+
triggers:
|
|
5
|
+
- "automate desktop app"
|
|
6
|
+
- "send message in slack"
|
|
7
|
+
- "edit in vscode"
|
|
8
|
+
- "switch to discord"
|
|
9
|
+
- "desktop app routing"
|
|
10
|
+
tools:
|
|
11
|
+
- "browser_map"
|
|
12
|
+
- "browser_click"
|
|
13
|
+
- "browser_type"
|
|
14
|
+
- "browser_fill"
|
|
15
|
+
- "browser_detect"
|
|
16
|
+
- "browser_tabs"
|
|
17
|
+
- "browser_navigate"
|
|
18
|
+
weavetab: ">=2.5.0-beta.5"
|
|
19
|
+
---
|
|
20
|
+
|
|
21
|
+
# Desktop App Targeting & Multi-Window Routing
|
|
22
|
+
|
|
23
|
+
Once a desktop application is detected via `browser_detect`, every standard Weavetab tool can be scoped directly to that desktop application without opening or disturbing the user's web browser.
|
|
24
|
+
|
|
25
|
+
---
|
|
26
|
+
|
|
27
|
+
## 1. Direct App Parameter Routing
|
|
28
|
+
|
|
29
|
+
Pass `"app": "<application_name>"` to target the Electron application.
|
|
30
|
+
|
|
31
|
+
### Example: Reading Discord Channels
|
|
32
|
+
```json
|
|
33
|
+
// Tool: browser_map
|
|
34
|
+
{
|
|
35
|
+
"app": "discord",
|
|
36
|
+
"scope": "main_content"
|
|
37
|
+
}
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
### Example: Sending a Message in Discord
|
|
41
|
+
```json
|
|
42
|
+
// Tool: browser_type
|
|
43
|
+
{
|
|
44
|
+
"app": "discord",
|
|
45
|
+
"ref": "w:88",
|
|
46
|
+
"text": "Deployment v2.5.0-beta.3 completed successfully."
|
|
47
|
+
}
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
### Example: Listing Discord Tabs/Windows
|
|
51
|
+
```json
|
|
52
|
+
// Tool: browser_tabs
|
|
53
|
+
{
|
|
54
|
+
"action": "list",
|
|
55
|
+
"app": "discord"
|
|
56
|
+
}
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
> [!IMPORTANT]
|
|
60
|
+
> Always pass `"app": "discord"` (or the target app name) to `browser_tabs` when working with desktop apps. This prevents Weavetab from accidentally launching or switching to the web browser.
|
|
61
|
+
|
|
62
|
+
---
|
|
63
|
+
|
|
64
|
+
## 2. Smart Domain Auto-Routing & Sticky Context
|
|
65
|
+
|
|
66
|
+
Weavetab features automatic target resolution to eliminate accidental browser popups:
|
|
67
|
+
|
|
68
|
+
### A. Automatic Domain Matching
|
|
69
|
+
If you call `browser_navigate` or `browser_tabs(action: "open")` with a known domain (e.g. `discord.com`, `slack.com`, `linear.app`, `notion.so`), Weavetab automatically routes the command to the configured Electron desktop application.
|
|
70
|
+
|
|
71
|
+
### B. Sticky App Sessions
|
|
72
|
+
When you interact with an Electron application (e.g., clicking or typing in Discord), Weavetab remembers the active application context. If a subsequent tool call omits `app` and no web browser instance is active, Weavetab stays attached to the active desktop app rather than spawning an unwanted Chrome window.
|
|
73
|
+
|
|
74
|
+
### C. Explicit Browser Override
|
|
75
|
+
If you specifically want to open a URL in the web browser instead of a desktop application, pass `"app": "browser"`.
|
|
76
|
+
|
|
77
|
+
---
|
|
78
|
+
|
|
79
|
+
## 3. Multi-Window Focus Management
|
|
80
|
+
|
|
81
|
+
Some Electron apps open secondary windows (e.g. popout editors, settings panels, or call windows).
|
|
82
|
+
|
|
83
|
+
1. Use `browser_tabs(action: "list", app: "discord")` to view all open windows and page targets for that specific app.
|
|
84
|
+
2. Switch focus between windows using `browser_tabs(action: "switch", tabId: "<id>", app: "discord")`.
|
|
85
|
+
3. To bring a specific native window to the foreground, use `browser_tabs(action: "focus_window", windowId: <id>, app: "discord")`.
|