@caiqy/browser-skill-dsh-plugin 0.3.3-dsh.2 → 0.3.3-dsh.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,63 +1,63 @@
1
- # Screenshots, HTML export and Canvas
2
-
3
- ## Viewport and full-page screenshots
4
-
5
- Without new options, `screenshot` captures the viewport; a fresh `ref` crops to
6
- that element. For a full-page PNG:
7
-
8
- ```text
9
- bsk_inspect({ action: "screenshot", session: "<id>", fullPage: true, scope: "current", timeoutMs: 120000, output: "C:/task/loaded-page.png" })
10
- bsk_inspect({ action: "screenshot", session: "<id>", fullPage: true, scope: "follow" })
11
- ```
12
-
13
- `fullPage` scrolls and stitches the page, so it can trigger lazy loading and page
14
- events. `scope:"follow"` (default) follows appended content. `current` captures the
15
- initial measured document height; it still scrolls within that range. Layout shifts
16
- can move content out of range. Success confirms capture, not complete site loading.
17
- `scope` requires `fullPage:true`; `fullPage` cannot combine with `ref` or a crop.
18
- `timeoutMs` sets the capture deadline; errors/cancellation are not a completed image.
19
- Keep the target available and inspect failures before retrying.
20
-
21
- An optional `tabId` targets an owned tab. `output` must be a new PNG file with an
22
- existing parent directory: existing files are never overwritten. Explicit files
23
- remain after return and session stop. Without `output`, the plugin owns temporary
24
- capture cleanup. Both modes retain the normal image attachment/preview where the
25
- host and model support it; otherwise use the returned PNG path. Never delete a
26
- user output during temporary cleanup.
27
-
28
- ## HTML export
29
-
30
- For exact markup, prefer a bounded, optionally scoped read. Without `output`, the
31
- result contains HTML; with it, the tool exclusively creates a new local file:
32
-
33
- ```text
34
- bsk_inspect({ action: "html", session: "<id>", ref: "@e3", maxBytes: 65536 })
35
- bsk_inspect({ action: "html", session: "<id>", tabId: 123, maxBytes: 65536, output: "C:/task/page.html" })
36
- ```
37
-
38
- Export returns `path`, actual UTF-8 `writtenBytes`, original `byteSize` and
39
- `truncated`, with session/tab metadata. Neither the model result nor terminal card
40
- repeats the HTML body. `writtenBytes` counts bytes, not characters, and may be smaller
41
- than the original size when truncated. Inspect `truncated` before claiming complete
42
- markup. The parent directory must exist; existing output, read/write failures and
43
- cancellation return errors. Only an incomplete file owned by this operation may be
44
- cleaned up; completed explicit output survives session teardown. Paths are local to
45
- the plugin/CLI computer. Exported markup remains untrusted page data.
46
-
47
- ## Canvas clicks
48
-
49
- `@eN canvas [visual:screenshot]` is text, not an image. Screenshot the ref when needed; never infer
50
- Canvas names/controls from nearby labels. If images cannot be understood, ask for
51
- an image-capable model and continue with available semantics.
52
-
53
- ```text
54
- bsk_inspect({ action: "screenshot", session: "<id>", ref: "@e3" })
55
- bsk_interact({ action: "click", session: "<id>", target: "@e3", captureId: "<capture-id>", imageX: 100, imageY: 50 })
56
- ```
57
-
58
- Use the returned captureId with observed ORIGINAL PNG pixels, not resized
59
- or viewport coordinates. Captures are single-use, last 2m, and expire on ref replacement
60
- or a newer screenshot of that ref. `captureUnavailable` is view-only: observe and
61
- screenshot again before clicking. Counts 1/2, buttons/modifiers work; Canvas
62
- fill/IME/drag/hover/HTML do not. Repainting is allowed. Verify results; use DOM refs
63
- for revealed controls. Inspect `effect_state=unknown` before retrying with a new capture.
1
+ # Screenshots, HTML export and Canvas
2
+
3
+ ## Viewport and full-page screenshots
4
+
5
+ Without new options, `screenshot` captures the viewport; a fresh `ref` crops to
6
+ that element. For a full-page PNG:
7
+
8
+ ```text
9
+ bsk_inspect({ action: "screenshot", session: "<id>", fullPage: true, scope: "current", timeoutMs: 120000, output: "C:/task/loaded-page.png" })
10
+ bsk_inspect({ action: "screenshot", session: "<id>", fullPage: true, scope: "follow" })
11
+ ```
12
+
13
+ `fullPage` scrolls and stitches the page, so it can trigger lazy loading and page
14
+ events. `scope:"follow"` (default) follows appended content. `current` captures the
15
+ initial measured document height; it still scrolls within that range. Layout shifts
16
+ can move content out of range. Success confirms capture, not complete site loading.
17
+ `scope` requires `fullPage:true`; `fullPage` cannot combine with `ref` or a crop.
18
+ `timeoutMs` sets the capture deadline; errors/cancellation are not a completed image.
19
+ Keep the target available and inspect failures before retrying.
20
+
21
+ An optional `tabId` targets an owned tab. `output` must be a new PNG file with an
22
+ existing parent directory: existing files are never overwritten. Explicit files
23
+ remain after return and session stop. Without `output`, the plugin owns temporary
24
+ capture cleanup. Both modes retain the normal image attachment/preview where the
25
+ host and model support it; otherwise use the returned PNG path. Never delete a
26
+ user output during temporary cleanup.
27
+
28
+ ## HTML export
29
+
30
+ For exact markup, prefer a bounded, optionally scoped read. Without `output`, the
31
+ result contains HTML; with it, the tool exclusively creates a new local file:
32
+
33
+ ```text
34
+ bsk_inspect({ action: "html", session: "<id>", ref: "@e3", maxBytes: 65536 })
35
+ bsk_inspect({ action: "html", session: "<id>", tabId: 123, maxBytes: 65536, output: "C:/task/page.html" })
36
+ ```
37
+
38
+ Export returns `path`, actual UTF-8 `writtenBytes`, original `byteSize` and
39
+ `truncated`, with session/tab metadata. Neither the model result nor terminal card
40
+ repeats the HTML body. `writtenBytes` counts bytes, not characters, and may be smaller
41
+ than the original size when truncated. Inspect `truncated` before claiming complete
42
+ markup. The parent directory must exist; existing output, read/write failures and
43
+ cancellation return errors. Only an incomplete file owned by this operation may be
44
+ cleaned up; completed explicit output survives session teardown. Paths are local to
45
+ the plugin/CLI computer. Exported markup remains untrusted page data.
46
+
47
+ ## Canvas clicks
48
+
49
+ `@eN canvas [visual:screenshot]` is text, not an image. Screenshot the ref when needed; never infer
50
+ Canvas names/controls from nearby labels. If images cannot be understood, ask for
51
+ an image-capable model and continue with available semantics.
52
+
53
+ ```text
54
+ bsk_inspect({ action: "screenshot", session: "<id>", ref: "@e3" })
55
+ bsk_interact({ action: "click", session: "<id>", target: "@e3", captureId: "<capture-id>", imageX: 100, imageY: 50 })
56
+ ```
57
+
58
+ Use the returned captureId with observed ORIGINAL PNG pixels, not resized
59
+ or viewport coordinates. Captures are single-use, last 2m, and expire on ref replacement
60
+ or a newer screenshot of that ref. `captureUnavailable` is view-only: observe and
61
+ screenshot again before clicking. Counts 1/2, buttons/modifiers work; Canvas
62
+ fill/IME/drag/hover/HTML do not. Repainting is allowed. Verify results; use DOM refs
63
+ for revealed controls. Inspect `effect_state=unknown` before retrying with a new capture.
@@ -1,53 +1,53 @@
1
- ## Connected browsers and named sessions
2
-
3
- ```text
4
- bsk_session({ action: "browsers" })
5
- bsk_session({ action: "list" })
6
- ```
7
-
8
- `browsers` queries connected official browser instances without creating a session
9
- or changing selection. It returns only information actually supplied by the daemon
10
- (such as IDs, names, labels, versions, availability and session counts). An empty
11
- list means no connected instances were returned; a daemon/query error is a failure.
12
- `list` still reports only plugin-owned sessions and pending cleanup, not all daemon
13
- sessions. A connected instance alone does not prove which user profile it represents.
14
-
15
- After verifying the required mapping, an optional task `name` labels a new session:
16
-
17
- ```text
18
- bsk_session({ action: "start", browser: "<verified-instance-id>", name: "Check account layout" })
19
- ```
20
-
21
- Names do not replace returned session IDs or ownership. Named starts use the same
22
- tracked start journal, recovery and failure cleanup as ordinary starts. If a reply
23
- is lost, inspect owned sessions/pending cleanup before repeating a start.
24
-
25
- ## Required browser profiles
26
-
27
- If the user or workspace requires a specific profile, confirm its instance ID before
28
- starting, even with only one connected browser. If unknown, ask the user to open the
29
- intended profile, check **Profile Path** at `chrome://version` if a directory was
30
- specified, and copy the **Instance ID** or **Copy profile instructions** from the
31
- connected BrowserSkill popup in that same profile. Connected alone and Chrome's
32
- process arguments do not prove the profile.
33
-
34
- Use the verified ID (or verified unique BrowserSkill label) on every new session:
35
-
36
- ```text
37
- bsk_session({ action: "start", browser: "<verified-instance-id>" })
38
- ```
39
-
40
- For copied command-line examples, use the instance ID in
41
- this tool call, without running the command. A Chrome profile name, directory,
42
- or extension ID is not an instance ID. If the mapping is unclear/ambiguous or the
43
- target unavailable, stop and ask the user to confirm/reconnect. Never omit
44
- `browser` or substitute another instance to recover.
45
-
46
- ## Borrowing and settings
47
-
48
- Use `bsk_tabs` to list IDs before acting. Borrow for the immediate step and
49
- return promptly. Browser Automation settings
50
- govern confirmation and help; never change them to bypass a prompt or repeat
51
- pending/denied/expired borrows. Inspect unknown outcomes; follow version-error hints.
52
- Remote reads/actions require task-created or borrowed tabs; popups gain no control.
53
- An unowned tab inside the Agent Window needs a user move to a user window before borrowing.
1
+ ## Connected browsers and named sessions
2
+
3
+ ```text
4
+ bsk_session({ action: "browsers" })
5
+ bsk_session({ action: "list" })
6
+ ```
7
+
8
+ `browsers` queries connected official browser instances without creating a session
9
+ or changing selection. It returns only information actually supplied by the daemon
10
+ (such as IDs, names, labels, versions, availability and session counts). An empty
11
+ list means no connected instances were returned; a daemon/query error is a failure.
12
+ `list` still reports only plugin-owned sessions and pending cleanup, not all daemon
13
+ sessions. A connected instance alone does not prove which user profile it represents.
14
+
15
+ After verifying the required mapping, an optional task `name` labels a new session:
16
+
17
+ ```text
18
+ bsk_session({ action: "start", browser: "<verified-instance-id>", name: "Check account layout" })
19
+ ```
20
+
21
+ Names do not replace returned session IDs or ownership. Named starts use the same
22
+ tracked start journal, recovery and failure cleanup as ordinary starts. If a reply
23
+ is lost, inspect owned sessions/pending cleanup before repeating a start.
24
+
25
+ ## Required browser profiles
26
+
27
+ If the user or workspace requires a specific profile, confirm its instance ID before
28
+ starting, even with only one connected browser. If unknown, ask the user to open the
29
+ intended profile, check **Profile Path** at `chrome://version` if a directory was
30
+ specified, and copy the **Instance ID** or **Copy profile instructions** from the
31
+ connected BrowserSkill popup in that same profile. Connected alone and Chrome's
32
+ process arguments do not prove the profile.
33
+
34
+ Use the verified ID (or verified unique BrowserSkill label) on every new session:
35
+
36
+ ```text
37
+ bsk_session({ action: "start", browser: "<verified-instance-id>" })
38
+ ```
39
+
40
+ For copied command-line examples, use the instance ID in
41
+ this tool call, without running the command. A Chrome profile name, directory,
42
+ or extension ID is not an instance ID. If the mapping is unclear/ambiguous or the
43
+ target unavailable, stop and ask the user to confirm/reconnect. Never omit
44
+ `browser` or substitute another instance to recover.
45
+
46
+ ## Borrowing and settings
47
+
48
+ Use `bsk_tabs` to list IDs before acting. Borrow for the immediate step and
49
+ return promptly. Browser Automation settings
50
+ govern confirmation and help; never change them to bypass a prompt or repeat
51
+ pending/denied/expired borrows. Inspect unknown outcomes; follow version-error hints.
52
+ Remote reads/actions require task-created or borrowed tabs; popups gain no control.
53
+ An unowned tab inside the Agent Window needs a user move to a user window before borrowing.
@@ -1,76 +1,76 @@
1
- # Video recording
2
-
3
- When the user asks to record a task, control recording yourself through
4
- `bsk_inspect` with `action: "video"`. The user does not need to race to the
5
- extension's Start button. Record only when requested. Video is separate from
6
- semantic interaction recording, which remains unsupported by these tools.
7
-
8
- 1. Start a session without an initial URL. For an existing page, borrow its tab
9
- first using the normal authorization flow. Retain the session and tab IDs.
10
- 2. Start recording and wait for `recording.state: "recording"` before navigating
11
- or performing task operations. A start error must be reported before proceeding
12
- without video; never imply that capture is active after an error.
13
- 3. Execute the task on that fixed tab. Navigation/reload remain recorded; switching
14
- tabs does not move capture. Start is nonblocking after the first encoded frame.
15
- 4. Before stopping, wait for navigation to settle and the expected final page
16
- state (such as its result element), then observe to confirm the outcome. An
17
- action returning does not guarantee that asynchronous page updates have finished.
18
- Stopping during unresolved navigation produces a partial result even if the
19
- task action succeeded. If the page fails to settle, stop and report incomplete
20
- capture rather than waiting indefinitely.
21
- 5. Stop recording before stopping the session, on success or failure. Inspect
22
- `state`, `completeness`, `stop_reason` and `error`, then report the result.
23
-
24
- ```text
25
- bsk_session({ action: "start" })
26
- bsk_inspect({ action: "video", videoAction: "start", session: "<sessionId>", durationMs: 60000, quality: "standard" })
27
- // Retain recording.recording_id. Navigate and perform the task only after start succeeds.
28
- bsk_inspect({ action: "video", videoAction: "status", recordingId: "<recording_id>" })
29
- // Wait for navigation and the expected final page state before stopping.
30
- bsk_inspect({ action: "video", videoAction: "stop", recordingId: "<recording_id>" })
31
- bsk_session({ action: "stop", session: "<sessionId>" })
32
- ```
33
-
34
- The result is a silent H.264 MP4. Default limit: 60 seconds; range: 1 second to
35
- 10 minutes. `standard` targets 1280 pixels / 15 fps; `clear` targets 1920 / 30 fps.
36
- Choose a limit appropriate to the requested task. A duration/size limit stops the
37
- video, not the task: do not call a capped video a recording of the entire task.
38
- One video can record per browser. A start error includes a stable `requestId` for
39
- retrying the same session/options if the reply was lost. A recovered recording
40
- that has already stopped is not an active recording; inspect it before continuing.
41
-
42
- ## Preview and explicit export
43
-
44
- Tell the user to open Features → Video recording → Recent recordings in the
45
- extension to preview, seek, and Save MP4 as. Stopping video leaves the task running;
46
- closing the popup does not stop recording. No save-location question is needed to
47
- start recording or keep its result for preview.
48
-
49
- Export through the tool only when the user specifies a destination; ask if needed:
50
-
51
- ```text
52
- bsk_inspect({ action: "video", videoAction: "save", recordingId: "<recording_id>", output: "<explicit-path.mp4>" })
53
- ```
54
-
55
- Tool export writes on the harness host, which may be remote. The extension's Save
56
- As writes on the browser computer. Replacing an existing file requires explicit
57
- permission and `overwrite: true`. A partial stop/save is a tool error containing
58
- the recording metadata and any saved path. Use status to inspect it; a file's
59
- existence does not make the recording complete. Unsupported capture/encoding fails
60
- explicitly; do not silently switch to another browser or recording method.
61
-
62
- Status, stop and list remain available during navigation or human help. After
63
- session teardown, use the recording ID to inspect or save the preserved video:
64
- artifact access is independent of the live task. `videoAction: "list"` lists only
65
- recordings started by this plugin instance; optional `session` filters that list.
66
- `videoAction: "discard"` with `recordingId` deletes a stopped browser copy only when
67
- requested. Never delete unsaved evidence merely to start another capture.
68
-
69
- Browser copies last 24 hours, including after task teardown. Closing the tab,
70
- disconnects or ending the session before stopping video may preserve a partial
71
- result. Network error pages or PDF viewers may also end as partial if clean
72
- capture cannot resume. Unresolved navigation has a 60-second fallback; the selected
73
- duration limit can stop it sooner. A crash may lose the last incomplete fragment.
74
- After a plugin reload, or for recordings started manually, use the extension
75
- library to preview and save.
76
- Interactive confirmation/help UI is replaced by a neutral waiting screen in video.
1
+ # Video recording
2
+
3
+ When the user asks to record a task, control recording yourself through
4
+ `bsk_inspect` with `action: "video"`. The user does not need to race to the
5
+ extension's Start button. Record only when requested. Video is separate from
6
+ semantic interaction recording, which remains unsupported by these tools.
7
+
8
+ 1. Start a session without an initial URL. For an existing page, borrow its tab
9
+ first using the normal authorization flow. Retain the session and tab IDs.
10
+ 2. Start recording and wait for `recording.state: "recording"` before navigating
11
+ or performing task operations. A start error must be reported before proceeding
12
+ without video; never imply that capture is active after an error.
13
+ 3. Execute the task on that fixed tab. Navigation/reload remain recorded; switching
14
+ tabs does not move capture. Start is nonblocking after the first encoded frame.
15
+ 4. Before stopping, wait for navigation to settle and the expected final page
16
+ state (such as its result element), then observe to confirm the outcome. An
17
+ action returning does not guarantee that asynchronous page updates have finished.
18
+ Stopping during unresolved navigation produces a partial result even if the
19
+ task action succeeded. If the page fails to settle, stop and report incomplete
20
+ capture rather than waiting indefinitely.
21
+ 5. Stop recording before stopping the session, on success or failure. Inspect
22
+ `state`, `completeness`, `stop_reason` and `error`, then report the result.
23
+
24
+ ```text
25
+ bsk_session({ action: "start" })
26
+ bsk_inspect({ action: "video", videoAction: "start", session: "<sessionId>", durationMs: 60000, quality: "standard" })
27
+ // Retain recording.recording_id. Navigate and perform the task only after start succeeds.
28
+ bsk_inspect({ action: "video", videoAction: "status", recordingId: "<recording_id>" })
29
+ // Wait for navigation and the expected final page state before stopping.
30
+ bsk_inspect({ action: "video", videoAction: "stop", recordingId: "<recording_id>" })
31
+ bsk_session({ action: "stop", session: "<sessionId>" })
32
+ ```
33
+
34
+ The result is a silent H.264 MP4. Default limit: 60 seconds; range: 1 second to
35
+ 10 minutes. `standard` targets 1280 pixels / 15 fps; `clear` targets 1920 / 30 fps.
36
+ Choose a limit appropriate to the requested task. A duration/size limit stops the
37
+ video, not the task: do not call a capped video a recording of the entire task.
38
+ One video can record per browser. A start error includes a stable `requestId` for
39
+ retrying the same session/options if the reply was lost. A recovered recording
40
+ that has already stopped is not an active recording; inspect it before continuing.
41
+
42
+ ## Preview and explicit export
43
+
44
+ Tell the user to open Features → Video recording → Recent recordings in the
45
+ extension to preview, seek, and Save MP4 as. Stopping video leaves the task running;
46
+ closing the popup does not stop recording. No save-location question is needed to
47
+ start recording or keep its result for preview.
48
+
49
+ Export through the tool only when the user specifies a destination; ask if needed:
50
+
51
+ ```text
52
+ bsk_inspect({ action: "video", videoAction: "save", recordingId: "<recording_id>", output: "<explicit-path.mp4>" })
53
+ ```
54
+
55
+ Tool export writes on the harness host, which may be remote. The extension's Save
56
+ As writes on the browser computer. Replacing an existing file requires explicit
57
+ permission and `overwrite: true`. A partial stop/save is a tool error containing
58
+ the recording metadata and any saved path. Use status to inspect it; a file's
59
+ existence does not make the recording complete. Unsupported capture/encoding fails
60
+ explicitly; do not silently switch to another browser or recording method.
61
+
62
+ Status, stop and list remain available during navigation or human help. After
63
+ session teardown, use the recording ID to inspect or save the preserved video:
64
+ artifact access is independent of the live task. `videoAction: "list"` lists only
65
+ recordings started by this plugin instance; optional `session` filters that list.
66
+ `videoAction: "discard"` with `recordingId` deletes a stopped browser copy only when
67
+ requested. Never delete unsaved evidence merely to start another capture.
68
+
69
+ Browser copies last 24 hours, including after task teardown. Closing the tab,
70
+ disconnects or ending the session before stopping video may preserve a partial
71
+ result. Network error pages or PDF viewers may also end as partial if clean
72
+ capture cannot resume. Unresolved navigation has a 60-second fallback; the selected
73
+ duration limit can stop it sooner. A crash may lose the last incomplete fragment.
74
+ After a plugin reload, or for recordings started manually, use the extension
75
+ library to preview and save.
76
+ Interactive confirmation/help UI is replaced by a neutral waiting screen in video.