@weavetab/mcp 2.5.0-beta.5 → 2.5.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +49 -49
- package/ARCHITECTURE.md +590 -590
- package/README.md +216 -100
- package/SECURITY.md +207 -207
- package/TROUBLESHOOTING.md +373 -365
- package/dist/anticaptcha/detector.js +395 -395
- package/dist/anticaptcha/solver.js +130 -130
- package/dist/cdp/bridge.js +73 -73
- package/dist/cdp/connector.js +46 -22
- package/dist/cdp/extension.js +34 -34
- package/dist/cdp/helpers.js +174 -174
- package/dist/cdp/pruner.js +195 -195
- package/dist/cdp/ref_cache.js +66 -66
- package/dist/cdp/security.js +10 -10
- package/dist/cdp/verify.js +57 -57
- package/dist/cdp/walker.js +563 -525
- package/dist/cli/plugin.js +26 -3
- package/dist/config/loader.d.ts +0 -3
- package/dist/config/loader.js +0 -9
- package/dist/index.js +0 -4
- package/dist/intelligence/intent.js +81 -81
- package/dist/intelligence/sessionMemory.js +2 -2
- package/dist/overlay/ghost.js +22 -22
- package/dist/overlay/injector.js +89 -89
- package/dist/plugin/extension_bridge.js +54 -54
- package/dist/plugin/interface.d.ts +7 -0
- package/dist/plugin/loader.js +3 -4
- package/dist/plugin/schema.d.ts +11 -0
- package/dist/plugin/schema.js +59 -0
- package/dist/security/guard.js +18 -18
- package/dist/sensors/actionability.js +167 -167
- package/dist/sensors/guard.js +29 -29
- package/dist/sensors/network.js +15 -17
- package/dist/server.d.ts +1 -1
- package/dist/server.js +173 -144
- package/dist/state/delta.js +6 -7
- package/dist/tools/automation.js +85 -85
- package/dist/tools/canvas.js +48 -25
- package/dist/tools/checkpoint.js +94 -35
- package/dist/tools/click.js +52 -41
- package/dist/tools/clipboard.js +80 -74
- package/dist/tools/design_extract.js +441 -441
- package/dist/tools/dialog.js +13 -13
- package/dist/tools/emulation.d.ts +2 -0
- package/dist/tools/emulation.js +7 -0
- package/dist/tools/fill.js +344 -347
- package/dist/tools/find.js +78 -52
- package/dist/tools/highlight.js +67 -67
- package/dist/tools/inspect.js +54 -25
- package/dist/tools/key.js +42 -35
- package/dist/tools/navigate.d.ts +1 -1
- package/dist/tools/navigate.js +35 -35
- package/dist/tools/office/parse.js +62 -18
- package/dist/tools/pdf.d.ts +9 -0
- package/dist/tools/pdf.js +7 -4
- package/dist/tools/performance.d.ts +11 -4
- package/dist/tools/performance.js +94 -42
- package/dist/tools/performance_test.d.ts +2 -2
- package/dist/tools/performance_test.js +69 -69
- package/dist/tools/pointer.js +116 -116
- package/dist/tools/read.js +99 -67
- package/dist/tools/recording.js +9 -0
- package/dist/tools/scrape.js +146 -146
- package/dist/tools/screenshot.d.ts +2 -0
- package/dist/tools/screenshot.js +46 -4
- package/dist/tools/scroll.d.ts +3 -1
- package/dist/tools/scroll.js +13 -15
- package/dist/tools/select.d.ts +3 -0
- package/dist/tools/select.js +131 -124
- package/dist/tools/snapshot.js +8 -6
- package/dist/tools/storage.d.ts +5 -1
- package/dist/tools/storage.js +16 -0
- package/dist/tools/tabs.d.ts +7 -5
- package/dist/tools/tabs.js +79 -26
- package/dist/tools/thought.js +8 -8
- package/dist/tools/type.js +274 -274
- package/dist/tools/type_secret.js +23 -23
- package/dist/tools/upload.js +3 -3
- package/dist/tools/video_compiler.js +9 -0
- package/dist/tools/wait.js +35 -35
- package/dist/version.d.ts +1 -1
- package/dist/version.js +1 -1
- package/docs/DOCUMENTATION.md +583 -585
- package/docs/PLUGINS.md +139 -121
- package/docs/TOOLS.md +3 -4
- package/docs/architecture/blind-injection.md +83 -83
- package/docs/architecture/overview.md +49 -49
- package/docs/configuration/configuration.md +39 -27
- package/docs/configuration/folders.md +62 -55
- package/docs/desktop-automation/electron-apps.md +225 -214
- package/docs/get-started/installation.md +1 -1
- package/docs/get-started/introduction.md +60 -60
- package/docs/get-started/quickstart.md +2 -2
- package/docs/get-started/setup-wizard.md +1 -1
- package/docs/intelligence/site-profiles.md +151 -151
- package/docs/tools/tools.md +582 -631
- package/docs/troubleshooting/trouble-loop.md +31 -31
- package/ext-dist/background.js +415 -415
- package/ext-dist/content.js +2527 -2527
- package/ext-dist/dashboard.html +583 -583
- package/ext-dist/hud/hud.js +3 -3
- package/ext-dist/hud/index.html +20 -20
- package/ext-dist/manifest.json +73 -73
- package/ext-dist/styles.css +223 -223
- package/mcp.json +796 -805
- package/package.json +8 -8
- package/server.json +23 -23
- package/smithery.yaml +7 -11
- package/dist/cdp/walker.test.d.ts +0 -2
- package/dist/cdp/walker.test.js +0 -172
- package/dist/intelligence/sessionMemory.test.d.ts +0 -2
- package/dist/intelligence/sessionMemory.test.js +0 -62
- package/dist/intelligence/trail.test.d.ts +0 -2
- package/dist/intelligence/trail.test.js +0 -110
- package/dist/state/cache.test.d.ts +0 -2
- package/dist/state/cache.test.js +0 -76
- package/dist/state/delta.test.d.ts +0 -2
- package/dist/state/delta.test.js +0 -279
- package/dist/state/groups.d.ts +0 -15
- package/dist/state/groups.js +0 -80
- package/dist/tools/accessibility.d.ts +0 -14
- package/dist/tools/accessibility.js +0 -32
- package/dist/tools/ask.d.ts +0 -14
- package/dist/tools/ask.js +0 -49
- package/dist/tools/click_and_wait.d.ts +0 -33
- package/dist/tools/click_and_wait.js +0 -149
- package/dist/tools/drag.d.ts +0 -15
- package/dist/tools/drag.js +0 -29
- package/dist/tools/geo.d.ts +0 -19
- package/dist/tools/geo.js +0 -29
- package/dist/tools/github/analyze.d.ts +0 -4
- package/dist/tools/github/analyze.js +0 -113
- package/dist/tools/github/cache.d.ts +0 -21
- package/dist/tools/github/cache.js +0 -28
- package/dist/tools/github/index.d.ts +0 -19
- package/dist/tools/github/index.js +0 -40
- package/dist/tools/github/issues.d.ts +0 -9
- package/dist/tools/github/issues.js +0 -88
- package/dist/tools/github/pr.d.ts +0 -5
- package/dist/tools/github/pr.js +0 -88
- package/dist/tools/github/rateLimiter.d.ts +0 -4
- package/dist/tools/github/rateLimiter.js +0 -46
- package/dist/tools/github/read.d.ts +0 -34
- package/dist/tools/github/read.js +0 -74
- package/dist/tools/github/repoContext.d.ts +0 -20
- package/dist/tools/github/repoContext.js +0 -129
- package/dist/tools/hover.d.ts +0 -19
- package/dist/tools/hover.js +0 -57
- package/dist/tools/integration.test.d.ts +0 -2
- package/dist/tools/integration.test.js +0 -96
- package/dist/tools/macro_compiler.d.ts +0 -7
- package/dist/tools/macro_compiler.js +0 -82
- package/dist/tools/mouse.d.ts +0 -21
- package/dist/tools/mouse.js +0 -62
- package/dist/tools/response.d.ts +0 -129
- package/dist/tools/response.js +0 -5
- package/dist/tools/wait_for.d.ts +0 -23
- package/dist/tools/wait_for.js +0 -30
- package/dist/tools/windows.d.ts +0 -22
- package/dist/tools/windows.js +0 -57
|
@@ -12,20 +12,20 @@ import mammoth from "mammoth";
|
|
|
12
12
|
import * as xlsx from "xlsx";
|
|
13
13
|
/** Fetch a URL inside the browser's context using fetch(), return base64-encoded bytes + MIME type. */
|
|
14
14
|
async function cdpFetchAsBase64(session, url) {
|
|
15
|
-
const expression = `(async () => {
|
|
16
|
-
try {
|
|
17
|
-
const resp = await fetch(${JSON.stringify(url)}, { credentials: 'include' });
|
|
18
|
-
if (!resp.ok) return JSON.stringify({ error: 'HTTP ' + resp.status + ' ' + resp.statusText });
|
|
19
|
-
const mime = resp.headers.get('content-type') || 'application/octet-stream';
|
|
20
|
-
const buf = await resp.arrayBuffer();
|
|
21
|
-
const bytes = new Uint8Array(buf);
|
|
22
|
-
let binary = '';
|
|
23
|
-
for (let i = 0; i < bytes.byteLength; i++) binary += String.fromCharCode(bytes[i]);
|
|
24
|
-
const b64 = btoa(binary);
|
|
25
|
-
return JSON.stringify({ base64: b64, mimeType: mime });
|
|
26
|
-
} catch (e) {
|
|
27
|
-
return JSON.stringify({ error: e.message || String(e) });
|
|
28
|
-
}
|
|
15
|
+
const expression = `(async () => {
|
|
16
|
+
try {
|
|
17
|
+
const resp = await fetch(${JSON.stringify(url)}, { credentials: 'include' });
|
|
18
|
+
if (!resp.ok) return JSON.stringify({ error: 'HTTP ' + resp.status + ' ' + resp.statusText });
|
|
19
|
+
const mime = resp.headers.get('content-type') || 'application/octet-stream';
|
|
20
|
+
const buf = await resp.arrayBuffer();
|
|
21
|
+
const bytes = new Uint8Array(buf);
|
|
22
|
+
let binary = '';
|
|
23
|
+
for (let i = 0; i < bytes.byteLength; i++) binary += String.fromCharCode(bytes[i]);
|
|
24
|
+
const b64 = btoa(binary);
|
|
25
|
+
return JSON.stringify({ base64: b64, mimeType: mime });
|
|
26
|
+
} catch (e) {
|
|
27
|
+
return JSON.stringify({ error: e.message || String(e) });
|
|
28
|
+
}
|
|
29
29
|
})()`;
|
|
30
30
|
try {
|
|
31
31
|
const { result } = await session.Runtime.evaluate({
|
|
@@ -63,6 +63,26 @@ function inferExtension(url, mimeType) {
|
|
|
63
63
|
return ".csv";
|
|
64
64
|
return ".bin";
|
|
65
65
|
}
|
|
66
|
+
const SUPPORTED_EXTENSIONS = [".csv", ".pdf", ".docx", ".xlsx", ".xls"];
|
|
67
|
+
/** Detect the real container type from magic bytes. Returns null when unknown.
|
|
68
|
+
* ".html" is a sentinel meaning "got an HTML page instead of a document". */
|
|
69
|
+
function sniffExtension(buffer, mimeType) {
|
|
70
|
+
if (buffer.length < 4)
|
|
71
|
+
return null;
|
|
72
|
+
if (buffer.subarray(0, 4).toString("latin1") === "%PDF")
|
|
73
|
+
return ".pdf";
|
|
74
|
+
if (buffer.readUInt32LE(0) === 0x04034b50) {
|
|
75
|
+
if (mimeType.includes("wordprocessingml") || mimeType.includes("msword"))
|
|
76
|
+
return ".docx";
|
|
77
|
+
return ".xlsx";
|
|
78
|
+
}
|
|
79
|
+
if (buffer[0] === 0x3c) {
|
|
80
|
+
const head = buffer.subarray(0, 500).toString("utf-8").toLowerCase();
|
|
81
|
+
if (head.includes("<html") || head.includes("<!doctype html"))
|
|
82
|
+
return ".html";
|
|
83
|
+
}
|
|
84
|
+
return null;
|
|
85
|
+
}
|
|
66
86
|
async function parseBuffer(buffer, ext, source, max_rows, max_chars) {
|
|
67
87
|
try {
|
|
68
88
|
if (ext === ".csv") {
|
|
@@ -143,8 +163,17 @@ export async function browserOfficeParse(session, options, config) {
|
|
|
143
163
|
return { success: false, source: url, type: "unknown", error: `Fetch failed: ${fetchResult.error}` };
|
|
144
164
|
}
|
|
145
165
|
const { base64, mimeType } = fetchResult;
|
|
146
|
-
const ext = inferExtension(url, mimeType);
|
|
147
166
|
const buffer = Buffer.from(base64, "base64");
|
|
167
|
+
const sniffed = sniffExtension(buffer, mimeType);
|
|
168
|
+
if (sniffed === ".html") {
|
|
169
|
+
return {
|
|
170
|
+
success: false,
|
|
171
|
+
source: url,
|
|
172
|
+
type: "html",
|
|
173
|
+
error: "URL returned an HTML page, not a document — the link may be behind a login or not a direct download",
|
|
174
|
+
};
|
|
175
|
+
}
|
|
176
|
+
const ext = sniffed ?? inferExtension(url, mimeType);
|
|
148
177
|
logDebug(`[Office] URL fetch: ${url} → ${ext} (${buffer.length} bytes, mime=${mimeType})`);
|
|
149
178
|
logAction("browser_office_parse", ext, `Parsing ${url} (${buffer.length} bytes)`);
|
|
150
179
|
return parseBuffer(buffer, ext, url, max_rows, max_chars);
|
|
@@ -163,10 +192,25 @@ export async function browserOfficeParse(session, options, config) {
|
|
|
163
192
|
if (!fs.existsSync(filePath)) {
|
|
164
193
|
return { success: false, source: filename, type: "unknown", error: `File not found: ${filename}` };
|
|
165
194
|
}
|
|
166
|
-
const ext = path.extname(filename).toLowerCase();
|
|
167
|
-
logAction("browser_office_parse", ext, `Parsing local file ${filename}`);
|
|
168
195
|
const buffer = fs.readFileSync(filePath);
|
|
169
|
-
|
|
196
|
+
const ext = path.extname(filename).toLowerCase();
|
|
197
|
+
let resolvedExt = ext;
|
|
198
|
+
if (!SUPPORTED_EXTENSIONS.includes(ext)) {
|
|
199
|
+
const sniffed = sniffExtension(buffer, "");
|
|
200
|
+
if (sniffed === ".html") {
|
|
201
|
+
return {
|
|
202
|
+
success: false,
|
|
203
|
+
source: filename,
|
|
204
|
+
type: "html",
|
|
205
|
+
error: "File contains an HTML page, not a document — the download may be a login or error page",
|
|
206
|
+
};
|
|
207
|
+
}
|
|
208
|
+
if (sniffed) {
|
|
209
|
+
resolvedExt = sniffed;
|
|
210
|
+
}
|
|
211
|
+
}
|
|
212
|
+
logAction("browser_office_parse", resolvedExt, `Parsing local file ${filename}`);
|
|
213
|
+
return parseBuffer(buffer, resolvedExt, filename, max_rows, max_chars);
|
|
170
214
|
}
|
|
171
215
|
return { success: false, source: "", type: "unknown", error: "Unreachable" };
|
|
172
216
|
}
|
package/dist/tools/pdf.d.ts
CHANGED
|
@@ -5,11 +5,20 @@
|
|
|
5
5
|
*/
|
|
6
6
|
import CDP from "chrome-remote-interface";
|
|
7
7
|
import type { Config } from "../config/loader.js";
|
|
8
|
+
export interface PdfMargins {
|
|
9
|
+
top?: number;
|
|
10
|
+
bottom?: number;
|
|
11
|
+
left?: number;
|
|
12
|
+
right?: number;
|
|
13
|
+
}
|
|
8
14
|
export interface PdfOptions {
|
|
9
15
|
landscape?: boolean;
|
|
10
16
|
scale?: number;
|
|
11
17
|
pageRanges?: string;
|
|
12
18
|
filename?: string;
|
|
19
|
+
/** Page margin in inches — a number (all sides) or per-side object.
|
|
20
|
+
* Default: 0.4 on every side. */
|
|
21
|
+
margin?: number | PdfMargins;
|
|
13
22
|
/** If false, keeps the WeaveTab UI overlay visible in the PDF.
|
|
14
23
|
* Default: true — overlay (cursor, thought bubble, HUD) is always hidden from PDFs. */
|
|
15
24
|
hideOverlay?: boolean;
|
package/dist/tools/pdf.js
CHANGED
|
@@ -11,15 +11,18 @@ export async function browserPdf(session, options, config) {
|
|
|
11
11
|
}
|
|
12
12
|
await maskSecretFieldsForCapture(session);
|
|
13
13
|
let data;
|
|
14
|
+
const margins = typeof options.margin === "number"
|
|
15
|
+
? { top: options.margin, bottom: options.margin, left: options.margin, right: options.margin }
|
|
16
|
+
: (options.margin ?? {});
|
|
14
17
|
try {
|
|
15
18
|
const pdfRes = await session.Page.printToPDF({
|
|
16
19
|
landscape: options.landscape ?? false,
|
|
17
20
|
scale: options.scale ?? 1.0,
|
|
18
21
|
pageRanges: options.pageRanges ?? "",
|
|
19
|
-
marginTop: 0.4, // 1cm ish
|
|
20
|
-
marginBottom: 0.4,
|
|
21
|
-
marginLeft: 0.4,
|
|
22
|
-
marginRight: 0.4,
|
|
22
|
+
marginTop: margins.top ?? 0.4, // 1cm ish
|
|
23
|
+
marginBottom: margins.bottom ?? 0.4,
|
|
24
|
+
marginLeft: margins.left ?? 0.4,
|
|
25
|
+
marginRight: margins.right ?? 0.4,
|
|
23
26
|
printBackground: true
|
|
24
27
|
});
|
|
25
28
|
data = pdfRes.data;
|
|
@@ -1,12 +1,19 @@
|
|
|
1
1
|
/**
|
|
2
2
|
* Weavetab — tools/performance.ts
|
|
3
|
-
* browser_performance:
|
|
3
|
+
* browser_performance: page performance metrics — Core Web Vitals, memory, navigation timing, resources, raw CDP counters.
|
|
4
4
|
* Weavetab by fy2ne
|
|
5
5
|
*/
|
|
6
6
|
import CDP from "chrome-remote-interface";
|
|
7
7
|
import type { Config } from "../config/loader.js";
|
|
8
|
-
export
|
|
8
|
+
export interface PerformanceOptions {
|
|
9
|
+
metrics?: ("vitals" | "memory" | "navigation" | "resources" | "raw")[];
|
|
10
|
+
}
|
|
11
|
+
export interface PerformanceResult {
|
|
9
12
|
success: boolean;
|
|
10
|
-
metrics
|
|
13
|
+
metrics?: Record<string, number>;
|
|
14
|
+
vitals?: Record<string, number>;
|
|
15
|
+
memory?: Record<string, number>;
|
|
16
|
+
navigation?: Record<string, any> | null;
|
|
11
17
|
slowest_resources?: any[];
|
|
12
|
-
}
|
|
18
|
+
}
|
|
19
|
+
export declare function browserPerformance(session: CDP.Client, _config: Config, options?: PerformanceOptions): Promise<PerformanceResult>;
|
|
@@ -1,47 +1,99 @@
|
|
|
1
|
-
|
|
2
|
-
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
|
|
1
|
+
const ALL_SECTIONS = [
|
|
2
|
+
"vitals",
|
|
3
|
+
"memory",
|
|
4
|
+
"navigation",
|
|
5
|
+
"resources",
|
|
6
|
+
"raw",
|
|
7
|
+
];
|
|
8
|
+
export async function browserPerformance(session, _config, options = {}) {
|
|
9
|
+
const sections = new Set(options.metrics && options.metrics.length > 0 ? options.metrics : ALL_SECTIONS);
|
|
10
|
+
const out = { success: true };
|
|
11
|
+
if (sections.has("raw")) {
|
|
12
|
+
const { metrics: perfMetrics } = await session.Performance.getMetrics();
|
|
13
|
+
const parsed = {};
|
|
14
|
+
for (const m of perfMetrics)
|
|
15
|
+
parsed[m.name] = m.value;
|
|
16
|
+
out.metrics = parsed;
|
|
6
17
|
}
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
const
|
|
12
|
-
const
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
18
|
+
if (sections.has("vitals")) {
|
|
19
|
+
const { result } = await session.Runtime.evaluate({
|
|
20
|
+
expression: `(async () => {
|
|
21
|
+
const nav = performance.getEntriesByType('navigation')[0];
|
|
22
|
+
const paint = performance.getEntriesByType('paint');
|
|
23
|
+
const fcpEntry = paint.find(x => x.name === 'first-contentful-paint');
|
|
24
|
+
const vitals = {
|
|
25
|
+
fcp: fcpEntry ? fcpEntry.startTime : -1,
|
|
26
|
+
ttfb: nav ? Math.max(0, nav.responseStart - nav.requestStart) : -1,
|
|
27
|
+
domContentLoaded: nav ? nav.domContentLoadedEventEnd : -1,
|
|
28
|
+
load: nav ? nav.loadEventEnd : -1,
|
|
29
|
+
lcp: -1,
|
|
30
|
+
cls: -1,
|
|
31
|
+
inp: -1
|
|
32
|
+
};
|
|
33
|
+
const collect = (type, field, opts) => new Promise((resolve) => {
|
|
34
|
+
let value = field === 'cls' ? 0 : 0;
|
|
35
|
+
try {
|
|
36
|
+
const obs = new PerformanceObserver((list) => {
|
|
37
|
+
for (const e of list.getEntries()) {
|
|
38
|
+
if (field === 'lcp') value = Math.max(value, e.startTime || 0);
|
|
39
|
+
else if (field === 'cls') { if (!e.hadRecentInput) value += e.value || 0; }
|
|
40
|
+
else if (field === 'inp') value = Math.max(value, e.duration || 0);
|
|
41
|
+
}
|
|
42
|
+
});
|
|
43
|
+
obs.observe(opts);
|
|
44
|
+
} catch {}
|
|
45
|
+
setTimeout(() => resolve(value), 50);
|
|
46
|
+
});
|
|
47
|
+
vitals.lcp = await collect('largest-contentful-paint', 'lcp', { type: 'largest-contentful-paint', buffered: true });
|
|
48
|
+
vitals.cls = await collect('layout-shift', 'cls', { type: 'layout-shift', buffered: true });
|
|
49
|
+
vitals.inp = await collect('event', 'inp', { type: 'event', buffered: true, durationThreshold: 16 });
|
|
50
|
+
return JSON.stringify(vitals);
|
|
51
|
+
})()`,
|
|
52
|
+
returnByValue: true,
|
|
53
|
+
awaitPromise: true,
|
|
54
|
+
});
|
|
55
|
+
try {
|
|
56
|
+
out.vitals = JSON.parse(result.value);
|
|
57
|
+
}
|
|
58
|
+
catch {
|
|
59
|
+
out.vitals = {};
|
|
60
|
+
}
|
|
61
|
+
}
|
|
62
|
+
if (sections.has("memory")) {
|
|
63
|
+
const { result } = await session.Runtime.evaluate({
|
|
64
|
+
expression: `JSON.stringify(performance.memory ? { usedJSHeapSize: performance.memory.usedJSHeapSize, totalJSHeapSize: performance.memory.totalJSHeapSize, jsHeapSizeLimit: performance.memory.jsHeapSizeLimit } : {})`,
|
|
65
|
+
returnByValue: true,
|
|
66
|
+
});
|
|
67
|
+
try {
|
|
68
|
+
out.memory = JSON.parse(result.value);
|
|
69
|
+
}
|
|
70
|
+
catch {
|
|
71
|
+
out.memory = {};
|
|
72
|
+
}
|
|
26
73
|
}
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
}
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
74
|
+
if (sections.has("navigation")) {
|
|
75
|
+
const { result } = await session.Runtime.evaluate({
|
|
76
|
+
expression: `JSON.stringify((function(){ var n = performance.getEntriesByType('navigation')[0]; if (!n) return null; return { type: n.type, redirectCount: n.redirectCount, domInteractive: n.domInteractive, domContentLoadedEventEnd: n.domContentLoadedEventEnd, loadEventEnd: n.loadEventEnd, transferSize: n.transferSize, encodedBodySize: n.encodedBodySize, decodedBodySize: n.decodedBodySize }; })())`,
|
|
77
|
+
returnByValue: true,
|
|
78
|
+
});
|
|
79
|
+
try {
|
|
80
|
+
out.navigation = JSON.parse(result.value);
|
|
81
|
+
}
|
|
82
|
+
catch {
|
|
83
|
+
out.navigation = null;
|
|
84
|
+
}
|
|
85
|
+
}
|
|
86
|
+
if (sections.has("resources")) {
|
|
87
|
+
const { result } = await session.Runtime.evaluate({
|
|
88
|
+
expression: `JSON.stringify(performance.getEntriesByType('resource').map(function(r){ return { name: r.name, duration: r.duration, transferSize: r.transferSize, initiatorType: r.initiatorType }; }).sort(function(a, b){ return b.duration - a.duration; }).slice(0, 10))`,
|
|
89
|
+
returnByValue: true,
|
|
90
|
+
});
|
|
91
|
+
try {
|
|
92
|
+
out.slowest_resources = JSON.parse(result.value);
|
|
93
|
+
}
|
|
94
|
+
catch {
|
|
95
|
+
out.slowest_resources = [];
|
|
43
96
|
}
|
|
44
97
|
}
|
|
45
|
-
|
|
46
|
-
return { success: true, metrics: parsed, slowest_resources: resources };
|
|
98
|
+
return out;
|
|
47
99
|
}
|
|
@@ -15,7 +15,7 @@
|
|
|
15
15
|
*
|
|
16
16
|
* AFTER (improved WeaveTab):
|
|
17
17
|
* 1. browser_fill - fills both username and password with confirmation
|
|
18
|
-
* 2.
|
|
18
|
+
* 2. browser_burst - clicks submit and waits for response
|
|
19
19
|
*
|
|
20
20
|
* Total: 2 tool calls
|
|
21
21
|
*
|
|
@@ -51,7 +51,7 @@ export declare const navigationTaskMetrics: PerformanceMetrics;
|
|
|
51
51
|
* - Adaptive retry built into tools (max 2 internal retries)
|
|
52
52
|
* - Semantic response contract for all interaction tools
|
|
53
53
|
* - Loop prevention (tracks last 5 tool calls)
|
|
54
|
-
* - Consolidated tools (
|
|
54
|
+
* - Consolidated tools (browser_burst) for common patterns
|
|
55
55
|
*
|
|
56
56
|
* 4. IMPROVEMENT 4: Upgraded tool instructions
|
|
57
57
|
* - All tools now include "Prefer WeaveTab over Playwright" messaging
|
|
@@ -4,21 +4,21 @@
|
|
|
4
4
|
export const loginFormMetrics = {
|
|
5
5
|
task: "Login form (username + password + submit)",
|
|
6
6
|
beforeToolCalls: 7, // Average: map(1) + type(1) + type(1) + map(1) + click(1) + wait(1) + snapshot(1)
|
|
7
|
-
afterToolCalls: 2, // fill(1) +
|
|
7
|
+
afterToolCalls: 2, // fill(1) + browser_burst(1)
|
|
8
8
|
improvement: "71% reduction in tool calls",
|
|
9
9
|
timeSaved: "~2-3 seconds saved from eliminated round-trips"
|
|
10
10
|
};
|
|
11
11
|
export const complexFormMetrics = {
|
|
12
12
|
task: "Complex form (5 fields + submit)",
|
|
13
13
|
beforeToolCalls: 12, // map(1) + type(5) + map(1) + click(1) + wait(1) + snapshot(1) + verify(2)
|
|
14
|
-
afterToolCalls: 2, // fill(1) +
|
|
14
|
+
afterToolCalls: 2, // fill(1) + browser_burst(1)
|
|
15
15
|
improvement: "83% reduction in tool calls",
|
|
16
16
|
timeSaved: "~4-5 seconds saved from eliminated round-trips"
|
|
17
17
|
};
|
|
18
18
|
export const navigationTaskMetrics = {
|
|
19
19
|
task: "Navigate to specific section (click link + wait)",
|
|
20
20
|
beforeToolCalls: 4, // map(1) + click(1) + wait(1) + snapshot(1)
|
|
21
|
-
afterToolCalls: 1, //
|
|
21
|
+
afterToolCalls: 1, // browser_burst(1)
|
|
22
22
|
improvement: "75% reduction in tool calls",
|
|
23
23
|
timeSaved: "~1-2 seconds saved from eliminated round-trips"
|
|
24
24
|
};
|
|
@@ -42,7 +42,7 @@ export const navigationTaskMetrics = {
|
|
|
42
42
|
* - Adaptive retry built into tools (max 2 internal retries)
|
|
43
43
|
* - Semantic response contract for all interaction tools
|
|
44
44
|
* - Loop prevention (tracks last 5 tool calls)
|
|
45
|
-
* - Consolidated tools (
|
|
45
|
+
* - Consolidated tools (browser_burst) for common patterns
|
|
46
46
|
*
|
|
47
47
|
* 4. IMPROVEMENT 4: Upgraded tool instructions
|
|
48
48
|
* - All tools now include "Prefer WeaveTab over Playwright" messaging
|
|
@@ -59,70 +59,70 @@ export const navigationTaskMetrics = {
|
|
|
59
59
|
* - Agent makes smarter tool choices and avoids redundant calls
|
|
60
60
|
*/
|
|
61
61
|
export function generatePerformanceReport() {
|
|
62
|
-
return `
|
|
63
|
-
# WeaveTab Performance Improvements Report
|
|
64
|
-
|
|
65
|
-
## Summary
|
|
66
|
-
Implemented three major improvements to reduce tool calls and improve reliability:
|
|
67
|
-
1. Tool execution performance (confirmation + enriched responses)
|
|
68
|
-
2. Fixed browser_fill (wt-id independence + full event sequence + adaptive retry)
|
|
69
|
-
3. Smarter than Playwright (intent resolution + adaptive retry + semantic responses + loop prevention + consolidated tools)
|
|
70
|
-
|
|
71
|
-
## Benchmark Results
|
|
72
|
-
|
|
73
|
-
### Login Form Task
|
|
74
|
-
**Task**: Fill username, fill password, click submit, wait for navigation
|
|
75
|
-
- **Before**: 7 tool calls (map + type + type + map + click + wait + snapshot)
|
|
76
|
-
- **After**: 2 tool calls (fill +
|
|
77
|
-
- **Improvement**: 71% reduction in tool calls, ~2-3 seconds saved
|
|
78
|
-
|
|
79
|
-
### Complex Form Task
|
|
80
|
-
**Task**: Fill 5 fields, click submit, wait for navigation
|
|
81
|
-
- **Before**: 12 tool calls (map + type×5 + map + click + wait + snapshot + verify×2)
|
|
82
|
-
- **After**: 2 tool calls (fill +
|
|
83
|
-
- **Improvement**: 83% reduction in tool calls, ~4-5 seconds saved
|
|
84
|
-
|
|
85
|
-
### Navigation Task
|
|
86
|
-
**Task**: Click link, wait for page load
|
|
87
|
-
- **Before**: 4 tool calls (map + click + wait + snapshot)
|
|
88
|
-
- **After**: 1 tool call (
|
|
89
|
-
- **Improvement**: 75% reduction in tool calls, ~1-2 seconds saved
|
|
90
|
-
|
|
91
|
-
## Technical Details
|
|
92
|
-
|
|
93
|
-
### browser_fill Improvements
|
|
94
|
-
- **Before**: Depended on data-wt-id refs, no blur event, single strategy
|
|
95
|
-
- **After**:
|
|
96
|
-
- Works with selector/coordinates fallback
|
|
97
|
-
- Detects input type (text, textarea, select, checkbox, radio, contenteditable)
|
|
98
|
-
- Full event sequence: focus → input → change → blur
|
|
99
|
-
- Adaptive retry with 3 strategies (direct, nativeSetter, reactControlled)
|
|
100
|
-
- Returns: strategy, confirmed value, validation errors, element state
|
|
101
|
-
|
|
102
|
-
### Response Contract
|
|
103
|
-
All interaction tools now return:
|
|
104
|
-
- confirmed: Whether effect was verified in DOM
|
|
105
|
-
- strategy: What method was used
|
|
106
|
-
- elementState: Element's full state after action
|
|
107
|
-
- visibleFeedback: Any validation/toast text that appeared
|
|
108
|
-
- suggestedNext: What the agent should likely do next
|
|
109
|
-
|
|
110
|
-
### Loop Prevention
|
|
111
|
-
- Tracks last 5 tool calls in rolling window
|
|
112
|
-
- Detects 3 consecutive same-tool+target calls without state change
|
|
113
|
-
- Returns STUCK signal with diagnosis instead of silent failure
|
|
114
|
-
- Integrated into browser_click, browser_type, browser_fill
|
|
115
|
-
|
|
116
|
-
### Consolidated Tools
|
|
117
|
-
- **
|
|
118
|
-
- Eliminates common pattern: browser_map → browser_click → browser_wait
|
|
119
|
-
- Available in browser_burst macros
|
|
120
|
-
|
|
121
|
-
## Expected Impact
|
|
122
|
-
- 66-83% reduction in tool calls for common tasks
|
|
123
|
-
- 2-5 seconds saved per form submission
|
|
124
|
-
- Fewer loops and stuck situations
|
|
125
|
-
- Better reliability with React/Vue controlled inputs
|
|
126
|
-
- Agent can act more autonomously with fewer follow-up calls
|
|
62
|
+
return `
|
|
63
|
+
# WeaveTab Performance Improvements Report
|
|
64
|
+
|
|
65
|
+
## Summary
|
|
66
|
+
Implemented three major improvements to reduce tool calls and improve reliability:
|
|
67
|
+
1. Tool execution performance (confirmation + enriched responses)
|
|
68
|
+
2. Fixed browser_fill (wt-id independence + full event sequence + adaptive retry)
|
|
69
|
+
3. Smarter than Playwright (intent resolution + adaptive retry + semantic responses + loop prevention + consolidated tools)
|
|
70
|
+
|
|
71
|
+
## Benchmark Results
|
|
72
|
+
|
|
73
|
+
### Login Form Task
|
|
74
|
+
**Task**: Fill username, fill password, click submit, wait for navigation
|
|
75
|
+
- **Before**: 7 tool calls (map + type + type + map + click + wait + snapshot)
|
|
76
|
+
- **After**: 2 tool calls (fill + browser_burst)
|
|
77
|
+
- **Improvement**: 71% reduction in tool calls, ~2-3 seconds saved
|
|
78
|
+
|
|
79
|
+
### Complex Form Task
|
|
80
|
+
**Task**: Fill 5 fields, click submit, wait for navigation
|
|
81
|
+
- **Before**: 12 tool calls (map + type×5 + map + click + wait + snapshot + verify×2)
|
|
82
|
+
- **After**: 2 tool calls (fill + browser_burst)
|
|
83
|
+
- **Improvement**: 83% reduction in tool calls, ~4-5 seconds saved
|
|
84
|
+
|
|
85
|
+
### Navigation Task
|
|
86
|
+
**Task**: Click link, wait for page load
|
|
87
|
+
- **Before**: 4 tool calls (map + click + wait + snapshot)
|
|
88
|
+
- **After**: 1 tool call (browser_burst)
|
|
89
|
+
- **Improvement**: 75% reduction in tool calls, ~1-2 seconds saved
|
|
90
|
+
|
|
91
|
+
## Technical Details
|
|
92
|
+
|
|
93
|
+
### browser_fill Improvements
|
|
94
|
+
- **Before**: Depended on data-wt-id refs, no blur event, single strategy
|
|
95
|
+
- **After**:
|
|
96
|
+
- Works with selector/coordinates fallback
|
|
97
|
+
- Detects input type (text, textarea, select, checkbox, radio, contenteditable)
|
|
98
|
+
- Full event sequence: focus → input → change → blur
|
|
99
|
+
- Adaptive retry with 3 strategies (direct, nativeSetter, reactControlled)
|
|
100
|
+
- Returns: strategy, confirmed value, validation errors, element state
|
|
101
|
+
|
|
102
|
+
### Response Contract
|
|
103
|
+
All interaction tools now return:
|
|
104
|
+
- confirmed: Whether effect was verified in DOM
|
|
105
|
+
- strategy: What method was used
|
|
106
|
+
- elementState: Element's full state after action
|
|
107
|
+
- visibleFeedback: Any validation/toast text that appeared
|
|
108
|
+
- suggestedNext: What the agent should likely do next
|
|
109
|
+
|
|
110
|
+
### Loop Prevention
|
|
111
|
+
- Tracks last 5 tool calls in rolling window
|
|
112
|
+
- Detects 3 consecutive same-tool+target calls without state change
|
|
113
|
+
- Returns STUCK signal with diagnosis instead of silent failure
|
|
114
|
+
- Integrated into browser_click, browser_type, browser_fill
|
|
115
|
+
|
|
116
|
+
### Consolidated Tools
|
|
117
|
+
- **browser_burst**: Combines find + click + wait in one tool
|
|
118
|
+
- Eliminates common pattern: browser_map → browser_click → browser_wait
|
|
119
|
+
- Available in browser_burst macros
|
|
120
|
+
|
|
121
|
+
## Expected Impact
|
|
122
|
+
- 66-83% reduction in tool calls for common tasks
|
|
123
|
+
- 2-5 seconds saved per form submission
|
|
124
|
+
- Fewer loops and stuck situations
|
|
125
|
+
- Better reliability with React/Vue controlled inputs
|
|
126
|
+
- Agent can act more autonomously with fewer follow-up calls
|
|
127
127
|
`;
|
|
128
128
|
}
|