@caiqy/browser-skill-dsh-plugin 0.3.3-dsh.2 → 0.3.3-dsh.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -21
- package/README.md +301 -301
- package/cordis.patch.yml +5 -5
- package/lib/client.cjs +50 -50
- package/package.json +136 -136
- package/skill/references/debugging.md +119 -119
- package/skill/references/help-and-recovery.md +47 -47
- package/skill/references/interaction-details.md +139 -139
- package/skill/references/screenshots-and-canvas.md +63 -63
- package/skill/references/tabs-and-profiles.md +53 -53
- package/skill/references/video.md +76 -76
|
@@ -1,119 +1,119 @@
|
|
|
1
|
-
# Website debugging
|
|
2
|
-
|
|
3
|
-
Use `bsk_inspect` with `action: "debug"` to record and investigate a website
|
|
4
|
-
problem. Keep ordinary browsing capture-free. All examples use the owning
|
|
5
|
-
`session`; use returned IDs and the owned `tabId` when targeting a specific tab.
|
|
6
|
-
Read loaded schemas for parameters; `debugAction: "capabilities"` reports supported
|
|
7
|
-
actions, limits, filters and build information.
|
|
8
|
-
|
|
9
|
-
## Capture, reproduce, inspect, save
|
|
10
|
-
|
|
11
|
-
1. Record the problem and expected result. Start a session or borrow the user's tab
|
|
12
|
-
following [tabs and profiles](tabs-and-profiles.md). Start capture **before**
|
|
13
|
-
navigating or reproducing; earlier traffic cannot be recovered.
|
|
14
|
-
|
|
15
|
-
```text
|
|
16
|
-
bsk_inspect({ action: "debug", debugAction: "start", session: "<id>", name: "Nickname does not persist" })
|
|
17
|
-
bsk_page({ action: "navigate", session: "<id>", url: "https://example.com/profile" })
|
|
18
|
-
bsk_inspect({ action: "observe", session: "<id>" })
|
|
19
|
-
```
|
|
20
|
-
|
|
21
|
-
2. Retain the returned run ID; pass it as `runId` for subsequent reads. Reproduce
|
|
22
|
-
with ordinary interaction tools, or let the user act (manual capture covers the
|
|
23
|
-
main frame). Observe the outcome; allow expected delayed work to finish.
|
|
24
|
-
3. Read `operations`, then `operation` with its `id` to inspect associated requests,
|
|
25
|
-
console messages, field values and later page changes. Use `requests`, then
|
|
26
|
-
`request` with its `id` and `part: "request"` or `"response"` for body text;
|
|
27
|
-
metadata is the default. `console` and `pages` expose other retained context.
|
|
28
|
-
4. Use `stop` to save the final record, then export before stopping the session:
|
|
29
|
-
|
|
30
|
-
```text
|
|
31
|
-
bsk_inspect({ action: "debug", debugAction: "stop", session: "<id>" })
|
|
32
|
-
bsk_inspect({ action: "debug", debugAction: "export", session: "<id>", runId: "<run-id>", output: "<new-file.json>" })
|
|
33
|
-
bsk_session({ action: "stop", session: "<id>" })
|
|
34
|
-
```
|
|
35
|
-
|
|
36
|
-
`export` writes a new file without overwriting and returns its path/size. Exporting
|
|
37
|
-
while capturing gives a point-in-time record. Browser-local history survives task
|
|
38
|
-
completion within retention limits; inspect/export it through the extension after
|
|
39
|
-
session teardown. A new session cannot read another task's history. Give exported
|
|
40
|
-
JSON to an agent for analysis/comparison; there is no repair or comparison action.
|
|
41
|
-
`pin`/`unpin` with a completed request `id` protect it from capacity eviction,
|
|
42
|
-
not record expiry.
|
|
43
|
-
|
|
44
|
-
## Decide whether the evidence supports a conclusion
|
|
45
|
-
|
|
46
|
-
Operation links are time associations, not proof of causation. Distinguish original,
|
|
47
|
-
submitted, response and later page values only where recorded; do not invent missing
|
|
48
|
-
steps. HTTP 200 is not business success. Console source labels distinguish website,
|
|
49
|
-
extension, browser and unknown errors.
|
|
50
|
-
`payload_partial` means a field scan was incomplete; inspect the body directly
|
|
51
|
-
rather than assuming a unique field match.
|
|
52
|
-
|
|
53
|
-
Check `run.coverage`, `run.storage`, body states/reasons and `output.omitted`.
|
|
54
|
-
Late start, unsupported capture, pending work, truncation, eviction and storage
|
|
55
|
-
failure can leave gaps: missing evidence does not mean an event did not happen.
|
|
56
|
-
State the supported finding and the specific gap preventing a stronger conclusion.
|
|
57
|
-
Evidence may be redacted; never treat displayable text as an exact original request.
|
|
58
|
-
|
|
59
|
-
## Read within a budget; wait without repeating commands
|
|
60
|
-
|
|
61
|
-
Default output budget is 64 KiB; `budget` accepts 4096..262144 bytes, export exempt.
|
|
62
|
-
For `requests`, filter with `url`, `method`, `resourceType`, `status`, `state` or
|
|
63
|
-
`kind`; `kind: "business"` reduces resource noise. Use a comma-separated `fields`
|
|
64
|
-
string from capabilities for optional projections. `limit` is 1..100. Requests/operations return `next_since`:
|
|
65
|
-
pass it as `since` and merge updates by ID. Body slices return the body's
|
|
66
|
-
`next_offset`; console/pages, performance and analysis return top-level `next_offset`. Pass it
|
|
67
|
-
as `offset`. Body reads also accept `maxChars` and a JSON `pointer` for complete bodies.
|
|
68
|
-
Pagination does not imply completeness: inspect omissions and availability flags.
|
|
69
|
-
|
|
70
|
-
When busy, read `activity`; use `wait` with `commandId` from activity or
|
|
71
|
-
`session_busy`, and optional `waitMs` up to 60000. Omitting `commandId` waits for
|
|
72
|
-
idle. Waiting neither resends nor cancels a command; completion is not success.
|
|
73
|
-
Inspect the result/page before retrying an operation with uncertain effects.
|
|
74
|
-
|
|
75
|
-
## Performance and request summaries
|
|
76
|
-
|
|
77
|
-
`performance` reports main-frame navigation/paint, CLS and long tasks, not INP or
|
|
78
|
-
CPU profiles. Read each metric's state/reason and visibility history; hidden-page
|
|
79
|
-
or late-start metrics may be invalid.
|
|
80
|
-
`aggregate` groups method/path with latency percentiles, errors and slow counts
|
|
81
|
-
(`slowMs`). `duplicates` reports suspected equal URL/body/document bursts
|
|
82
|
-
(`windowMs`); valid retries or polling can look repetitive. Both default to business
|
|
83
|
-
traffic and exclude rule/replay experiments; `includeControlled: true` includes them.
|
|
84
|
-
Follow linked request IDs and coverage gaps. Stop capture for stable summary
|
|
85
|
-
pagination, using `offset`/`next_offset`, not `since`.
|
|
86
|
-
|
|
87
|
-
## Explicit network experiments
|
|
88
|
-
|
|
89
|
-
Capture alone does not alter traffic. Use rules/replay only within the user's
|
|
90
|
-
requested experiment: they can block or change live requests and server data.
|
|
91
|
-
They require active capture in the original owning task; history never reactivates controls.
|
|
92
|
-
|
|
93
|
-
`rule_add` takes a **JSON-string** `rule` containing `match`, `effect` and optional
|
|
94
|
-
`times`. Match an absolute URL; `*` matches path/query. Effects are `block`,
|
|
95
|
-
`modify` (same-origin URL, method, headers, text body or top-level JSON edits) and
|
|
96
|
-
`mock` (status, text body, optional headers/delay). Default: one Fetch/XHR match;
|
|
97
|
-
first matching rule wins. `times: 0` lasts until disabled/capture ends. Read `rules`
|
|
98
|
-
for IDs, hits and failures; `rule_enable`, `rule_disable`, `rule_remove` take `id`.
|
|
99
|
-
Capture end clears execution. Check recorded `intervention`; a mock is not a server
|
|
100
|
-
response. Active rules also apply to matching replays.
|
|
101
|
-
|
|
102
|
-
`replay` takes the source request `id` and a **JSON-string** `replay`, for example
|
|
103
|
-
`"{\"key\":\"attempt-1\"}"`; add explicit `url`/`body` replacements when needed.
|
|
104
|
-
It sends a new request using current cookies. Source, destination and current page
|
|
105
|
-
must share an HTTP(S) origin; redirects, cross-origin and binary/multipart replay
|
|
106
|
-
are unsupported. Reuse the **same key** after an uncertain result to avoid sending
|
|
107
|
-
again; a new key deliberately creates a new attempt. Inspect the linked request
|
|
108
|
-
for outcome; replay does not invoke the original page handler or guarantee UI updates.
|
|
109
|
-
|
|
110
|
-
Reusing retained values requires `integrity.url: "complete"`, complete request metadata
|
|
111
|
-
and `request_body.replay_safe: true`. Body `available` only means displayable.
|
|
112
|
-
`integrity.metadata` covers the request; `integrity.response_headers` is independent
|
|
113
|
-
and does not restrict replay.
|
|
114
|
-
Changed, truncated or unverified URL/body require explicit complete replacements;
|
|
115
|
-
old records without these markers do too. Never copy a truncated evidence draft as
|
|
116
|
-
its replacement. Replacing URL/body cannot fix incomplete metadata. Replace known
|
|
117
|
-
redacted headers with authorized values or remove them with `null`; never send
|
|
118
|
-
redacted placeholders or guess secrets. Preserve exact numeric tokens when supplying
|
|
119
|
-
text bodies; JSON edits must not round large numeric IDs.
|
|
1
|
+
# Website debugging
|
|
2
|
+
|
|
3
|
+
Use `bsk_inspect` with `action: "debug"` to record and investigate a website
|
|
4
|
+
problem. Keep ordinary browsing capture-free. All examples use the owning
|
|
5
|
+
`session`; use returned IDs and the owned `tabId` when targeting a specific tab.
|
|
6
|
+
Read loaded schemas for parameters; `debugAction: "capabilities"` reports supported
|
|
7
|
+
actions, limits, filters and build information.
|
|
8
|
+
|
|
9
|
+
## Capture, reproduce, inspect, save
|
|
10
|
+
|
|
11
|
+
1. Record the problem and expected result. Start a session or borrow the user's tab
|
|
12
|
+
following [tabs and profiles](tabs-and-profiles.md). Start capture **before**
|
|
13
|
+
navigating or reproducing; earlier traffic cannot be recovered.
|
|
14
|
+
|
|
15
|
+
```text
|
|
16
|
+
bsk_inspect({ action: "debug", debugAction: "start", session: "<id>", name: "Nickname does not persist" })
|
|
17
|
+
bsk_page({ action: "navigate", session: "<id>", url: "https://example.com/profile" })
|
|
18
|
+
bsk_inspect({ action: "observe", session: "<id>" })
|
|
19
|
+
```
|
|
20
|
+
|
|
21
|
+
2. Retain the returned run ID; pass it as `runId` for subsequent reads. Reproduce
|
|
22
|
+
with ordinary interaction tools, or let the user act (manual capture covers the
|
|
23
|
+
main frame). Observe the outcome; allow expected delayed work to finish.
|
|
24
|
+
3. Read `operations`, then `operation` with its `id` to inspect associated requests,
|
|
25
|
+
console messages, field values and later page changes. Use `requests`, then
|
|
26
|
+
`request` with its `id` and `part: "request"` or `"response"` for body text;
|
|
27
|
+
metadata is the default. `console` and `pages` expose other retained context.
|
|
28
|
+
4. Use `stop` to save the final record, then export before stopping the session:
|
|
29
|
+
|
|
30
|
+
```text
|
|
31
|
+
bsk_inspect({ action: "debug", debugAction: "stop", session: "<id>" })
|
|
32
|
+
bsk_inspect({ action: "debug", debugAction: "export", session: "<id>", runId: "<run-id>", output: "<new-file.json>" })
|
|
33
|
+
bsk_session({ action: "stop", session: "<id>" })
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
`export` writes a new file without overwriting and returns its path/size. Exporting
|
|
37
|
+
while capturing gives a point-in-time record. Browser-local history survives task
|
|
38
|
+
completion within retention limits; inspect/export it through the extension after
|
|
39
|
+
session teardown. A new session cannot read another task's history. Give exported
|
|
40
|
+
JSON to an agent for analysis/comparison; there is no repair or comparison action.
|
|
41
|
+
`pin`/`unpin` with a completed request `id` protect it from capacity eviction,
|
|
42
|
+
not record expiry.
|
|
43
|
+
|
|
44
|
+
## Decide whether the evidence supports a conclusion
|
|
45
|
+
|
|
46
|
+
Operation links are time associations, not proof of causation. Distinguish original,
|
|
47
|
+
submitted, response and later page values only where recorded; do not invent missing
|
|
48
|
+
steps. HTTP 200 is not business success. Console source labels distinguish website,
|
|
49
|
+
extension, browser and unknown errors.
|
|
50
|
+
`payload_partial` means a field scan was incomplete; inspect the body directly
|
|
51
|
+
rather than assuming a unique field match.
|
|
52
|
+
|
|
53
|
+
Check `run.coverage`, `run.storage`, body states/reasons and `output.omitted`.
|
|
54
|
+
Late start, unsupported capture, pending work, truncation, eviction and storage
|
|
55
|
+
failure can leave gaps: missing evidence does not mean an event did not happen.
|
|
56
|
+
State the supported finding and the specific gap preventing a stronger conclusion.
|
|
57
|
+
Evidence may be redacted; never treat displayable text as an exact original request.
|
|
58
|
+
|
|
59
|
+
## Read within a budget; wait without repeating commands
|
|
60
|
+
|
|
61
|
+
Default output budget is 64 KiB; `budget` accepts 4096..262144 bytes, export exempt.
|
|
62
|
+
For `requests`, filter with `url`, `method`, `resourceType`, `status`, `state` or
|
|
63
|
+
`kind`; `kind: "business"` reduces resource noise. Use a comma-separated `fields`
|
|
64
|
+
string from capabilities for optional projections. `limit` is 1..100. Requests/operations return `next_since`:
|
|
65
|
+
pass it as `since` and merge updates by ID. Body slices return the body's
|
|
66
|
+
`next_offset`; console/pages, performance and analysis return top-level `next_offset`. Pass it
|
|
67
|
+
as `offset`. Body reads also accept `maxChars` and a JSON `pointer` for complete bodies.
|
|
68
|
+
Pagination does not imply completeness: inspect omissions and availability flags.
|
|
69
|
+
|
|
70
|
+
When busy, read `activity`; use `wait` with `commandId` from activity or
|
|
71
|
+
`session_busy`, and optional `waitMs` up to 60000. Omitting `commandId` waits for
|
|
72
|
+
idle. Waiting neither resends nor cancels a command; completion is not success.
|
|
73
|
+
Inspect the result/page before retrying an operation with uncertain effects.
|
|
74
|
+
|
|
75
|
+
## Performance and request summaries
|
|
76
|
+
|
|
77
|
+
`performance` reports main-frame navigation/paint, CLS and long tasks, not INP or
|
|
78
|
+
CPU profiles. Read each metric's state/reason and visibility history; hidden-page
|
|
79
|
+
or late-start metrics may be invalid.
|
|
80
|
+
`aggregate` groups method/path with latency percentiles, errors and slow counts
|
|
81
|
+
(`slowMs`). `duplicates` reports suspected equal URL/body/document bursts
|
|
82
|
+
(`windowMs`); valid retries or polling can look repetitive. Both default to business
|
|
83
|
+
traffic and exclude rule/replay experiments; `includeControlled: true` includes them.
|
|
84
|
+
Follow linked request IDs and coverage gaps. Stop capture for stable summary
|
|
85
|
+
pagination, using `offset`/`next_offset`, not `since`.
|
|
86
|
+
|
|
87
|
+
## Explicit network experiments
|
|
88
|
+
|
|
89
|
+
Capture alone does not alter traffic. Use rules/replay only within the user's
|
|
90
|
+
requested experiment: they can block or change live requests and server data.
|
|
91
|
+
They require active capture in the original owning task; history never reactivates controls.
|
|
92
|
+
|
|
93
|
+
`rule_add` takes a **JSON-string** `rule` containing `match`, `effect` and optional
|
|
94
|
+
`times`. Match an absolute URL; `*` matches path/query. Effects are `block`,
|
|
95
|
+
`modify` (same-origin URL, method, headers, text body or top-level JSON edits) and
|
|
96
|
+
`mock` (status, text body, optional headers/delay). Default: one Fetch/XHR match;
|
|
97
|
+
first matching rule wins. `times: 0` lasts until disabled/capture ends. Read `rules`
|
|
98
|
+
for IDs, hits and failures; `rule_enable`, `rule_disable`, `rule_remove` take `id`.
|
|
99
|
+
Capture end clears execution. Check recorded `intervention`; a mock is not a server
|
|
100
|
+
response. Active rules also apply to matching replays.
|
|
101
|
+
|
|
102
|
+
`replay` takes the source request `id` and a **JSON-string** `replay`, for example
|
|
103
|
+
`"{\"key\":\"attempt-1\"}"`; add explicit `url`/`body` replacements when needed.
|
|
104
|
+
It sends a new request using current cookies. Source, destination and current page
|
|
105
|
+
must share an HTTP(S) origin; redirects, cross-origin and binary/multipart replay
|
|
106
|
+
are unsupported. Reuse the **same key** after an uncertain result to avoid sending
|
|
107
|
+
again; a new key deliberately creates a new attempt. Inspect the linked request
|
|
108
|
+
for outcome; replay does not invoke the original page handler or guarantee UI updates.
|
|
109
|
+
|
|
110
|
+
Reusing retained values requires `integrity.url: "complete"`, complete request metadata
|
|
111
|
+
and `request_body.replay_safe: true`. Body `available` only means displayable.
|
|
112
|
+
`integrity.metadata` covers the request; `integrity.response_headers` is independent
|
|
113
|
+
and does not restrict replay.
|
|
114
|
+
Changed, truncated or unverified URL/body require explicit complete replacements;
|
|
115
|
+
old records without these markers do too. Never copy a truncated evidence draft as
|
|
116
|
+
its replacement. Replacing URL/body cannot fix incomplete metadata. Replace known
|
|
117
|
+
redacted headers with authorized values or remove them with `null`; never send
|
|
118
|
+
redacted placeholders or guess secrets. Preserve exact numeric tokens when supplying
|
|
119
|
+
text bodies; JSON edits must not round large numeric IDs.
|
|
@@ -1,47 +1,47 @@
|
|
|
1
|
-
# Human help and recovery
|
|
2
|
-
|
|
3
|
-
With help enabled, use `bsk_assist` action `request-help` for login, CAPTCHA,
|
|
4
|
-
OTP, payment confirmation, consent, or after two attempts without progress. Supply
|
|
5
|
-
a precise prompt and fresh targets; completion criteria need a stable success signal.
|
|
6
|
-
`all` and `any` support at most eight conditions in total. `urlMatches` accepts
|
|
7
|
-
RE2-compatible regular expressions of at most 128 characters. A compiled pattern
|
|
8
|
-
may contain at most 4096 instructions, and all URL-regex conditions together at most
|
|
9
|
-
8192, counting repeated patterns each time. Invalid or over-budget patterns return
|
|
10
|
-
`invalid_params` before help starts.
|
|
11
|
-
URL conditions do not auto-complete on URLs longer than 8192 characters; the user
|
|
12
|
-
can still finish manually. The protocol treats empty, null, and omitted
|
|
13
|
-
`url_matches` values as no URL-regex condition.
|
|
14
|
-
|
|
15
|
-
RE2 rejects JavaScript patterns such as `\u0061`, `a{1001}`, `[^]`, lookahead,
|
|
16
|
-
lookbehind, and backreferences. Some accepted patterns differ in meaning: `\a`
|
|
17
|
-
is a control character in RE2 rather than `a`, and RE2's `\s` does not include
|
|
18
|
-
Unicode whitespace. Rewrite patterns in RE2 syntax; use `urlContains` when a
|
|
19
|
-
substring is enough.
|
|
20
|
-
Resume only on `continued` / `completed`, then observe. Cancellation/timeout blocks
|
|
21
|
-
the step; do not repeat the request. Navigation alone is not success.
|
|
22
|
-
|
|
23
|
-
With help disabled, neither request help nor re-enable it. `disabled` grants no human
|
|
24
|
-
action or permission. Re-observe; use existing logins, authorized inputs and alternatives
|
|
25
|
-
within task/host rules. Vision models may try authorized graphical verification.
|
|
26
|
-
Phone-only QR scans, face verification, missing SMS codes and image-only tasks for
|
|
27
|
-
text-only models may stay blocked. Report missing inputs/capabilities or exhausted
|
|
28
|
-
alternatives; continue independent work. Never repeat unknown effects or switch
|
|
29
|
-
backends to bypass limits. Borrow confirmation still applies.
|
|
30
|
-
|
|
31
|
-
## Recover
|
|
32
|
-
|
|
33
|
-
- Unknown browser tool after plugin reload: invoke `skill` with `name: "browser-skill"`
|
|
34
|
-
again (users can enter `/browser-skill`), then retry the intended browser tool once
|
|
35
|
-
after its schema appears. If it remains unavailable, report the failure.
|
|
36
|
-
- Stale ref: observe, then retry the intended action once.
|
|
37
|
-
- Unknown tab/session: list owned resources or start a session with the required
|
|
38
|
-
browser selector, if any; never guess IDs.
|
|
39
|
-
- Failed/interrupted stop: accepted cleanup continues in the background. Retry the
|
|
40
|
-
same stop; completed cleanup returns `alreadyClosed: true`. For multiple pending
|
|
41
|
-
stops, specify `session` or the owned `requestId` from the result/list/error,
|
|
42
|
-
never both. The request ID identifies the original operation even if its short
|
|
43
|
-
session ID is reused. Never switch sessions to retry cleanup.
|
|
44
|
-
- Timeout/unknown effect: inspect before retrying; the action may have happened.
|
|
45
|
-
- Unconfirmed fill: read the field. Formatting may satisfy the goal; correct only a
|
|
46
|
-
remaining difference instead of blindly refilling or requesting help.
|
|
47
|
-
- Other errors: follow the hint; on unrecoverable failure, report and stop the owned session.
|
|
1
|
+
# Human help and recovery
|
|
2
|
+
|
|
3
|
+
With help enabled, use `bsk_assist` action `request-help` for login, CAPTCHA,
|
|
4
|
+
OTP, payment confirmation, consent, or after two attempts without progress. Supply
|
|
5
|
+
a precise prompt and fresh targets; completion criteria need a stable success signal.
|
|
6
|
+
`all` and `any` support at most eight conditions in total. `urlMatches` accepts
|
|
7
|
+
RE2-compatible regular expressions of at most 128 characters. A compiled pattern
|
|
8
|
+
may contain at most 4096 instructions, and all URL-regex conditions together at most
|
|
9
|
+
8192, counting repeated patterns each time. Invalid or over-budget patterns return
|
|
10
|
+
`invalid_params` before help starts.
|
|
11
|
+
URL conditions do not auto-complete on URLs longer than 8192 characters; the user
|
|
12
|
+
can still finish manually. The protocol treats empty, null, and omitted
|
|
13
|
+
`url_matches` values as no URL-regex condition.
|
|
14
|
+
|
|
15
|
+
RE2 rejects JavaScript patterns such as `\u0061`, `a{1001}`, `[^]`, lookahead,
|
|
16
|
+
lookbehind, and backreferences. Some accepted patterns differ in meaning: `\a`
|
|
17
|
+
is a control character in RE2 rather than `a`, and RE2's `\s` does not include
|
|
18
|
+
Unicode whitespace. Rewrite patterns in RE2 syntax; use `urlContains` when a
|
|
19
|
+
substring is enough.
|
|
20
|
+
Resume only on `continued` / `completed`, then observe. Cancellation/timeout blocks
|
|
21
|
+
the step; do not repeat the request. Navigation alone is not success.
|
|
22
|
+
|
|
23
|
+
With help disabled, neither request help nor re-enable it. `disabled` grants no human
|
|
24
|
+
action or permission. Re-observe; use existing logins, authorized inputs and alternatives
|
|
25
|
+
within task/host rules. Vision models may try authorized graphical verification.
|
|
26
|
+
Phone-only QR scans, face verification, missing SMS codes and image-only tasks for
|
|
27
|
+
text-only models may stay blocked. Report missing inputs/capabilities or exhausted
|
|
28
|
+
alternatives; continue independent work. Never repeat unknown effects or switch
|
|
29
|
+
backends to bypass limits. Borrow confirmation still applies.
|
|
30
|
+
|
|
31
|
+
## Recover
|
|
32
|
+
|
|
33
|
+
- Unknown browser tool after plugin reload: invoke `skill` with `name: "browser-skill"`
|
|
34
|
+
again (users can enter `/browser-skill`), then retry the intended browser tool once
|
|
35
|
+
after its schema appears. If it remains unavailable, report the failure.
|
|
36
|
+
- Stale ref: observe, then retry the intended action once.
|
|
37
|
+
- Unknown tab/session: list owned resources or start a session with the required
|
|
38
|
+
browser selector, if any; never guess IDs.
|
|
39
|
+
- Failed/interrupted stop: accepted cleanup continues in the background. Retry the
|
|
40
|
+
same stop; completed cleanup returns `alreadyClosed: true`. For multiple pending
|
|
41
|
+
stops, specify `session` or the owned `requestId` from the result/list/error,
|
|
42
|
+
never both. The request ID identifies the original operation even if its short
|
|
43
|
+
session ID is reused. Never switch sessions to retry cleanup.
|
|
44
|
+
- Timeout/unknown effect: inspect before retrying; the action may have happened.
|
|
45
|
+
- Unconfirmed fill: read the field. Formatting may satisfy the goal; correct only a
|
|
46
|
+
remaining difference instead of blindly refilling or requesting help.
|
|
47
|
+
- Other errors: follow the hint; on unrecoverable failure, report and stop the owned session.
|
|
@@ -1,139 +1,139 @@
|
|
|
1
|
-
# Interaction details
|
|
2
|
-
|
|
3
|
-
## Element waits
|
|
4
|
-
|
|
5
|
-
`bsk_page` supports `action: "wait-for-element"` for an explicit element
|
|
6
|
-
condition. For example, if
|
|
7
|
-
`@e2` is an observed loading indicator after the triggering action, use
|
|
8
|
-
`bsk_page({ action: "wait-for-element", target: "@e2", state: "hidden" })`,
|
|
9
|
-
then check `satisfied` and observe the result. Replace the example ref with the
|
|
10
|
-
actual one. Skip waiting when the page is already ready.
|
|
11
|
-
|
|
12
|
-
Prefer a fresh ref for an existing node, including frame/shadow-root targets; do
|
|
13
|
-
not fetch HTML just to replace a usable ref with a selector. A known
|
|
14
|
-
CSS selector can find a node that will appear or be replaced; it checks only the
|
|
15
|
-
first match in the main document, so scope it to the intended component. Do not
|
|
16
|
-
invent selectors: inspect bounded HTML if the current observation lacks one.
|
|
17
|
-
Only CSS selectors are supported, not `text=...`, XPath, or Playwright locators.
|
|
18
|
-
|
|
19
|
-
States are `visible` (default), `hidden` (absent or not visible), `attached`, and
|
|
20
|
-
`detached` (absent). Visibility does not guarantee enabled state or actionability.
|
|
21
|
-
An already absent loading mask satisfies `hidden` immediately, possibly before
|
|
22
|
-
loading starts; prefer a result marker when that race is possible. Invalid refs
|
|
23
|
-
remain errors, even for `hidden`/`detached`. Navigation invalidates refs.
|
|
24
|
-
|
|
25
|
-
The default timeout is 10000 ms (range 1..300000); `pollMs` is optional, defaults
|
|
26
|
-
to 100, and accepts 16..2000. The wait stops as soon as the condition matches.
|
|
27
|
-
Always check `satisfied`: false means timeout, not success. `attached`/`visible`
|
|
28
|
-
describe the last completed probe and are unknown if no probe completed.
|
|
29
|
-
Waiting does not refresh refs: observe before choosing a new action. For a
|
|
30
|
-
timeout, stale ref, or an older installation without this command, observe the
|
|
31
|
-
current page and continue with supported single actions when possible. Do not
|
|
32
|
-
repeat the triggering action without checking its effects or keep retrying an
|
|
33
|
-
unsupported wait. If blocked, follow [help and recovery](help-and-recovery.md).
|
|
34
|
-
|
|
35
|
-
## Page-script evaluation
|
|
36
|
-
|
|
37
|
-
Use `bsk_interact` with `action: "evaluate"` and a nonempty JavaScript `expression`
|
|
38
|
-
for authorized page work. Use an IIFE for statements. `awaitPromise` and
|
|
39
|
-
`returnByValue` default to true; explicit false is preserved. `timeoutMs` controls
|
|
40
|
-
execution. All page actions here accept an owned `session` and optional `tabId`.
|
|
41
|
-
|
|
42
|
-
```text
|
|
43
|
-
bsk_interact({ action: "evaluate", session: "<id>", expression: "document.title" })
|
|
44
|
-
bsk_interact({ action: "evaluate", session: "<id>", tabId: 123, expression: "Promise.resolve({ ready: true })", awaitPromise: true, returnByValue: true, timeoutMs: 10000 })
|
|
45
|
-
bsk_interact({ action: "evaluate", session: "<id>", expression: "window", awaitPromise: false, returnByValue: false })
|
|
46
|
-
```
|
|
47
|
-
|
|
48
|
-
Replace example IDs with actual results. Scripts may mutate the page or send
|
|
49
|
-
requests; use the same authorization and ownership limits as other interactions.
|
|
50
|
-
Never extract secrets. Page exceptions (`ok:false`) are failures even if the
|
|
51
|
-
underlying process exits successfully; retain their details. Timeout/cancellation
|
|
52
|
-
is not success and may leave effects: inspect before retrying. Observe after changes.
|
|
53
|
-
|
|
54
|
-
## Upload and download
|
|
55
|
-
|
|
56
|
-
`upload` requires an observed `target` and nonempty `files` array of readable local
|
|
57
|
-
paths. `mode` defaults to `input` for a file input/chooser; `drop` targets a drop zone.
|
|
58
|
-
Multiple paths use the existing transfer pipeline, including staging and cleanup.
|
|
59
|
-
|
|
60
|
-
```text
|
|
61
|
-
bsk_interact({ action: "upload", session: "<id>", target: "@e3", files: ["C:/task/sample.txt"] })
|
|
62
|
-
bsk_interact({ action: "upload", session: "<id>", target: "@e4", files: ["C:/task/a.txt", "C:/task/b.txt"], mode: "drop", timeoutMs: 120000 })
|
|
63
|
-
bsk_interact({ action: "download", session: "<id>", target: "@e5", output: "C:/task/report.csv", overwrite: false, timeoutMs: 120000 })
|
|
64
|
-
```
|
|
65
|
-
|
|
66
|
-
`download` clicks the target and requires `output`. Existing destinations are
|
|
67
|
-
refused by default; use `overwrite:true` only when replacement is authorized.
|
|
68
|
-
Downloads use temporary files, atomic commit and transfer release. Paths refer to
|
|
69
|
-
the computer running the plugin/CLI, not a remote browser; remote browser file
|
|
70
|
-
transfer is unsupported. Empty lists, unreadable files, invalid modes, failed
|
|
71
|
-
transfers and cancellation must be reported as failures. Inspect uncertain browser
|
|
72
|
-
effects before repeating a click/upload. Verify the actual upload/download result.
|
|
73
|
-
|
|
74
|
-
## Fixed delays
|
|
75
|
-
|
|
76
|
-
`wait` waits for navigation lifecycle events; `wait-for-element` checks a DOM
|
|
77
|
-
condition. Use `wait-ms` only when a fixed delay is needed, then observe the result:
|
|
78
|
-
|
|
79
|
-
```text
|
|
80
|
-
bsk_page({ action: "wait-ms", durationMs: 250 })
|
|
81
|
-
```
|
|
82
|
-
|
|
83
|
-
`durationMs` must be an integer in 0..300000. This is a cancellable daemon-level
|
|
84
|
-
delay and needs no session or tab. Completion means elapsed time, not page readiness;
|
|
85
|
-
cancellation must not be reported as completion. Prefer an element condition when available.
|
|
86
|
-
|
|
87
|
-
## Interactions
|
|
88
|
-
|
|
89
|
-
- Hover markers like `[hover first: Shoes | Bags]` list labels, not refs. Hover the
|
|
90
|
-
trigger, observe, then use the item's ref. Click the trigger only if its action is wanted.
|
|
91
|
-
- `scroll-to` returns ancestor-clipped bounds in top-level viewport CSS pixels.
|
|
92
|
-
Partial visibility suffices; hidden/fully clipped targets fail. It does not test occlusion.
|
|
93
|
-
- `wheel` uses signed `deltaX`/`deltaY`, at least one nonzero. Optional `target` is
|
|
94
|
-
scrolled into view first; otherwise it uses the viewport centre. It reports input,
|
|
95
|
-
not scrolling success: observe afterwards. Focus/blur change focus states.
|
|
96
|
-
|
|
97
|
-
## Observation continuation
|
|
98
|
-
|
|
99
|
-
No default token cap. With `maxTokens`, pass `nextCursor` as observe's `cursor` to
|
|
100
|
-
continue. Each page replaces refs; use them before continuing, never reuse old ones.
|
|
101
|
-
Continuation uses the same capture without refresh/depth changes. New
|
|
102
|
-
observe/snapshot or changed page identity invalidates it.
|
|
103
|
-
|
|
104
|
-
`probeHover` and `debugSurfaces` default to false. If static observation misses
|
|
105
|
-
hover content, explicitly request a probe:
|
|
106
|
-
|
|
107
|
-
```text
|
|
108
|
-
bsk_inspect({ action: "observe", session: "<id>", probeHover: true, debugSurfaces: true, maxTokens: 3000 })
|
|
109
|
-
```
|
|
110
|
-
|
|
111
|
-
Active probing moves the real cursor and can trigger page events; it uses the
|
|
112
|
-
session queue and must not run concurrently with page operations. Read returned
|
|
113
|
-
`hoverProbe`, `debug` and `dialogs` when present; missing fields are not evidence
|
|
114
|
-
of a successful probe. `debugSurfaces` requests diagnostics without enabling active
|
|
115
|
-
hover by itself. Do not combine `cursor` with `maxDepth` or enabled probe flags;
|
|
116
|
-
continue the captured observation or start a new one. Console/network remain bounded
|
|
117
|
-
read-only diagnostics; follow sequence cursors.
|
|
118
|
-
|
|
119
|
-
## Window and device emulation
|
|
120
|
-
|
|
121
|
-
`bsk_assist` with `action:"resize"` changes the window; `emulate` applies per-tab
|
|
122
|
-
overrides, not inherited by new tabs. Device presets supply viewport, DPR, UA and
|
|
123
|
-
touch defaults; explicit values override the preset, including false.
|
|
124
|
-
|
|
125
|
-
```text
|
|
126
|
-
bsk_assist({ action: "emulate", session: "<id>", device: "iphone-14", width: 420, dpr: 2, mobile: false, touch: false })
|
|
127
|
-
bsk_assist({ action: "emulate", session: "<id>", tabId: 123, width: 800, height: 600, dpr: 1.5, mobile: false, ua: "Mozilla/5.0 task-test", acceptLanguage: "zh-CN,en;q=0.8", touch: true, maxTouchPoints: 5 })
|
|
128
|
-
bsk_assist({ action: "emulate", session: "<id>", off: true })
|
|
129
|
-
```
|
|
130
|
-
|
|
131
|
-
Dimensions are integers 1..7680; without a preset give width and height together.
|
|
132
|
-
`dpr` must be positive and finite; `dpr`/`mobile` require dimensions or a preset.
|
|
133
|
-
UA and touch can be set alone; `acceptLanguage` requires `ua` or a device preset.
|
|
134
|
-
`maxTouchPoints` is a positive integer (official unsigned 32-bit range) and implies
|
|
135
|
-
touch unless explicit `touch:false` overrides it. `mobile:false` and `touch:false`
|
|
136
|
-
explicitly disable those features; `touch:false` takes priority over both preset
|
|
137
|
-
touch defaults and `maxTouchPoints`. `off:true` clears emulation and must be used
|
|
138
|
-
without any preset or override, including false values. Invalid values/combinations
|
|
139
|
-
fail before execution. Inspect the page after applying or clearing emulation.
|
|
1
|
+
# Interaction details
|
|
2
|
+
|
|
3
|
+
## Element waits
|
|
4
|
+
|
|
5
|
+
`bsk_page` supports `action: "wait-for-element"` for an explicit element
|
|
6
|
+
condition. For example, if
|
|
7
|
+
`@e2` is an observed loading indicator after the triggering action, use
|
|
8
|
+
`bsk_page({ action: "wait-for-element", target: "@e2", state: "hidden" })`,
|
|
9
|
+
then check `satisfied` and observe the result. Replace the example ref with the
|
|
10
|
+
actual one. Skip waiting when the page is already ready.
|
|
11
|
+
|
|
12
|
+
Prefer a fresh ref for an existing node, including frame/shadow-root targets; do
|
|
13
|
+
not fetch HTML just to replace a usable ref with a selector. A known
|
|
14
|
+
CSS selector can find a node that will appear or be replaced; it checks only the
|
|
15
|
+
first match in the main document, so scope it to the intended component. Do not
|
|
16
|
+
invent selectors: inspect bounded HTML if the current observation lacks one.
|
|
17
|
+
Only CSS selectors are supported, not `text=...`, XPath, or Playwright locators.
|
|
18
|
+
|
|
19
|
+
States are `visible` (default), `hidden` (absent or not visible), `attached`, and
|
|
20
|
+
`detached` (absent). Visibility does not guarantee enabled state or actionability.
|
|
21
|
+
An already absent loading mask satisfies `hidden` immediately, possibly before
|
|
22
|
+
loading starts; prefer a result marker when that race is possible. Invalid refs
|
|
23
|
+
remain errors, even for `hidden`/`detached`. Navigation invalidates refs.
|
|
24
|
+
|
|
25
|
+
The default timeout is 10000 ms (range 1..300000); `pollMs` is optional, defaults
|
|
26
|
+
to 100, and accepts 16..2000. The wait stops as soon as the condition matches.
|
|
27
|
+
Always check `satisfied`: false means timeout, not success. `attached`/`visible`
|
|
28
|
+
describe the last completed probe and are unknown if no probe completed.
|
|
29
|
+
Waiting does not refresh refs: observe before choosing a new action. For a
|
|
30
|
+
timeout, stale ref, or an older installation without this command, observe the
|
|
31
|
+
current page and continue with supported single actions when possible. Do not
|
|
32
|
+
repeat the triggering action without checking its effects or keep retrying an
|
|
33
|
+
unsupported wait. If blocked, follow [help and recovery](help-and-recovery.md).
|
|
34
|
+
|
|
35
|
+
## Page-script evaluation
|
|
36
|
+
|
|
37
|
+
Use `bsk_interact` with `action: "evaluate"` and a nonempty JavaScript `expression`
|
|
38
|
+
for authorized page work. Use an IIFE for statements. `awaitPromise` and
|
|
39
|
+
`returnByValue` default to true; explicit false is preserved. `timeoutMs` controls
|
|
40
|
+
execution. All page actions here accept an owned `session` and optional `tabId`.
|
|
41
|
+
|
|
42
|
+
```text
|
|
43
|
+
bsk_interact({ action: "evaluate", session: "<id>", expression: "document.title" })
|
|
44
|
+
bsk_interact({ action: "evaluate", session: "<id>", tabId: 123, expression: "Promise.resolve({ ready: true })", awaitPromise: true, returnByValue: true, timeoutMs: 10000 })
|
|
45
|
+
bsk_interact({ action: "evaluate", session: "<id>", expression: "window", awaitPromise: false, returnByValue: false })
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
Replace example IDs with actual results. Scripts may mutate the page or send
|
|
49
|
+
requests; use the same authorization and ownership limits as other interactions.
|
|
50
|
+
Never extract secrets. Page exceptions (`ok:false`) are failures even if the
|
|
51
|
+
underlying process exits successfully; retain their details. Timeout/cancellation
|
|
52
|
+
is not success and may leave effects: inspect before retrying. Observe after changes.
|
|
53
|
+
|
|
54
|
+
## Upload and download
|
|
55
|
+
|
|
56
|
+
`upload` requires an observed `target` and nonempty `files` array of readable local
|
|
57
|
+
paths. `mode` defaults to `input` for a file input/chooser; `drop` targets a drop zone.
|
|
58
|
+
Multiple paths use the existing transfer pipeline, including staging and cleanup.
|
|
59
|
+
|
|
60
|
+
```text
|
|
61
|
+
bsk_interact({ action: "upload", session: "<id>", target: "@e3", files: ["C:/task/sample.txt"] })
|
|
62
|
+
bsk_interact({ action: "upload", session: "<id>", target: "@e4", files: ["C:/task/a.txt", "C:/task/b.txt"], mode: "drop", timeoutMs: 120000 })
|
|
63
|
+
bsk_interact({ action: "download", session: "<id>", target: "@e5", output: "C:/task/report.csv", overwrite: false, timeoutMs: 120000 })
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
`download` clicks the target and requires `output`. Existing destinations are
|
|
67
|
+
refused by default; use `overwrite:true` only when replacement is authorized.
|
|
68
|
+
Downloads use temporary files, atomic commit and transfer release. Paths refer to
|
|
69
|
+
the computer running the plugin/CLI, not a remote browser; remote browser file
|
|
70
|
+
transfer is unsupported. Empty lists, unreadable files, invalid modes, failed
|
|
71
|
+
transfers and cancellation must be reported as failures. Inspect uncertain browser
|
|
72
|
+
effects before repeating a click/upload. Verify the actual upload/download result.
|
|
73
|
+
|
|
74
|
+
## Fixed delays
|
|
75
|
+
|
|
76
|
+
`wait` waits for navigation lifecycle events; `wait-for-element` checks a DOM
|
|
77
|
+
condition. Use `wait-ms` only when a fixed delay is needed, then observe the result:
|
|
78
|
+
|
|
79
|
+
```text
|
|
80
|
+
bsk_page({ action: "wait-ms", durationMs: 250 })
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
`durationMs` must be an integer in 0..300000. This is a cancellable daemon-level
|
|
84
|
+
delay and needs no session or tab. Completion means elapsed time, not page readiness;
|
|
85
|
+
cancellation must not be reported as completion. Prefer an element condition when available.
|
|
86
|
+
|
|
87
|
+
## Interactions
|
|
88
|
+
|
|
89
|
+
- Hover markers like `[hover first: Shoes | Bags]` list labels, not refs. Hover the
|
|
90
|
+
trigger, observe, then use the item's ref. Click the trigger only if its action is wanted.
|
|
91
|
+
- `scroll-to` returns ancestor-clipped bounds in top-level viewport CSS pixels.
|
|
92
|
+
Partial visibility suffices; hidden/fully clipped targets fail. It does not test occlusion.
|
|
93
|
+
- `wheel` uses signed `deltaX`/`deltaY`, at least one nonzero. Optional `target` is
|
|
94
|
+
scrolled into view first; otherwise it uses the viewport centre. It reports input,
|
|
95
|
+
not scrolling success: observe afterwards. Focus/blur change focus states.
|
|
96
|
+
|
|
97
|
+
## Observation continuation
|
|
98
|
+
|
|
99
|
+
No default token cap. With `maxTokens`, pass `nextCursor` as observe's `cursor` to
|
|
100
|
+
continue. Each page replaces refs; use them before continuing, never reuse old ones.
|
|
101
|
+
Continuation uses the same capture without refresh/depth changes. New
|
|
102
|
+
observe/snapshot or changed page identity invalidates it.
|
|
103
|
+
|
|
104
|
+
`probeHover` and `debugSurfaces` default to false. If static observation misses
|
|
105
|
+
hover content, explicitly request a probe:
|
|
106
|
+
|
|
107
|
+
```text
|
|
108
|
+
bsk_inspect({ action: "observe", session: "<id>", probeHover: true, debugSurfaces: true, maxTokens: 3000 })
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
Active probing moves the real cursor and can trigger page events; it uses the
|
|
112
|
+
session queue and must not run concurrently with page operations. Read returned
|
|
113
|
+
`hoverProbe`, `debug` and `dialogs` when present; missing fields are not evidence
|
|
114
|
+
of a successful probe. `debugSurfaces` requests diagnostics without enabling active
|
|
115
|
+
hover by itself. Do not combine `cursor` with `maxDepth` or enabled probe flags;
|
|
116
|
+
continue the captured observation or start a new one. Console/network remain bounded
|
|
117
|
+
read-only diagnostics; follow sequence cursors.
|
|
118
|
+
|
|
119
|
+
## Window and device emulation
|
|
120
|
+
|
|
121
|
+
`bsk_assist` with `action:"resize"` changes the window; `emulate` applies per-tab
|
|
122
|
+
overrides, not inherited by new tabs. Device presets supply viewport, DPR, UA and
|
|
123
|
+
touch defaults; explicit values override the preset, including false.
|
|
124
|
+
|
|
125
|
+
```text
|
|
126
|
+
bsk_assist({ action: "emulate", session: "<id>", device: "iphone-14", width: 420, dpr: 2, mobile: false, touch: false })
|
|
127
|
+
bsk_assist({ action: "emulate", session: "<id>", tabId: 123, width: 800, height: 600, dpr: 1.5, mobile: false, ua: "Mozilla/5.0 task-test", acceptLanguage: "zh-CN,en;q=0.8", touch: true, maxTouchPoints: 5 })
|
|
128
|
+
bsk_assist({ action: "emulate", session: "<id>", off: true })
|
|
129
|
+
```
|
|
130
|
+
|
|
131
|
+
Dimensions are integers 1..7680; without a preset give width and height together.
|
|
132
|
+
`dpr` must be positive and finite; `dpr`/`mobile` require dimensions or a preset.
|
|
133
|
+
UA and touch can be set alone; `acceptLanguage` requires `ua` or a device preset.
|
|
134
|
+
`maxTouchPoints` is a positive integer (official unsigned 32-bit range) and implies
|
|
135
|
+
touch unless explicit `touch:false` overrides it. `mobile:false` and `touch:false`
|
|
136
|
+
explicitly disable those features; `touch:false` takes priority over both preset
|
|
137
|
+
touch defaults and `maxTouchPoints`. `off:true` clears emulation and must be used
|
|
138
|
+
without any preset or override, including false values. Invalid values/combinations
|
|
139
|
+
fail before execution. Inspect the page after applying or clearing emulation.
|