dsh-chrome-control 0.1.5

package/README.md

@@ -0,0 +1,97 @@
+# dsh-chrome
+
+A DeepSeek Harness bundle that gives the agent control of the user's **real
+Chrome** — their profile, their logins, their open tabs — as `mcp__chrome__*`
+tools.
+
+## How it fits together
+
+```
+agent ──MCP Streamable HTTP──▶ chrome-daemon ──WebSocket──▶ extension ──CDP──▶ page
+```
+
+The daemon is a standard MCP server, so this bundle needs **no tool code of its
+own**. Its patch contributes three rows:
+
+| Row | Job |
+|---|---|
+| `chrome-daemon` | Installs the daemon binary if needed and starts it. |
+| `chrome-mcp` | The in-box `@deepseek-ai/dsh-mcp-client` bridge, pointed at the daemon. This is the entire tool surface. |
+| `chrome-skills` | Registers the bundled skill that teaches the agent how to drive those tools. |
+
+## Install
+
+```bash
+dsh plugin --profile web add dsh-chrome-control     # or a link: dependency for local dev
+```
+
+Then **install the browser extension yourself** — see
+`extension/dsh-chrome/README.md`. Loading an unpacked extension is a decision
+only the browser's owner can make, so nothing does it for you. Until it is
+loaded and enabled, every tool returns a message saying exactly that.
+
+## Configuration
+
+```yaml
+- id: chrome-daemon
+  name: 'dsh-chrome/daemon'
+  config:
+    port: 10086       # must match the mcp-client row's url
+    autoStart: true   # false: never launch the daemon, just use one if present
+```
+
+The default port is **10086**, because the stock Kimi WebBridge
+daemon uses 10086 and the two are allowed to coexist. If you change `port`,
+change the `chrome-mcp` row's `url` to match — a test enforces that they agree.
+
+### How the binary is found
+
+In order: an existing `~/.dsh-chrome/bin/chrome-daemon`; a prebuilt binary in
+this package's `bin/chrome-daemon-<platform>-<arch>`; otherwise a
+`cargo build --release` of the sibling `crates/chrome-daemon` checkout. If none
+of those work the bundle logs one actionable line and **loads anyway** — a
+missing browser bridge must never stop a session from starting.
+
+## Behaviour worth knowing
+
+- **Startup order does not matter.** The mcp-client row sets
+  `failOnStartupError: false`, so a daemon that is not up yet does not abort the
+  profile; the bridge's reconnect policy attaches as soon as it answers. No
+  harness restart is needed.
+- **Someone else's daemon is never touched.** If the port answers but does not
+  identify itself as `dsh-chrome`, the bundle logs a warning telling you to pick
+  another port and stops. It never kills a process it does not own.
+- **The daemon is never stopped or restarted automatically.**
+
+## Tools
+
+`navigate`, `find_tab`, `list_tabs`, `close_tab`, `close_session`,
+`snapshot`, `click`, `fill`, `evaluate`, `screenshot`, `save_as_pdf`,
+`mouse_click`, `key_type`, `send_keys` — each prefixed `mcp__chrome__`.
+
+These are distinct from the harness's built-in `browser_*` tools, which drive a
+separate WebKit panel sharing no cookies or logins with Chrome. The bundled
+skill tells the agent when to use which.
+
+## What this bundle does and does not do
+
+| Does | Does not |
+|---|---|
+| Install and start the daemon | Install the browser extension |
+| Point the in-box MCP bridge at it | Implement any tool itself |
+| Register a skill describing the tools | Download binaries from the network |
+| Refuse to disturb a foreign process on the port | Stop or restart anything automatically |
+
+## Security
+
+While the extension is connected the agent acts with **your logged-in
+sessions**, and `mouse_click`/`key_type` send *trusted* input that pages cannot
+distinguish from your own. The extension popup has an **Allow agent control**
+toggle that severs this immediately; `autoStart: false` keeps the daemon from
+launching on its own. The daemon binds loopback only and sends no telemetry.
+
+## Tests
+
+```bash
+pnpm test        # patch contract, skill resolution, daemon supervision
+```

package/cordis.patch.yml

@@ -0,0 +1,32 @@
+# dsh-chrome bundle: three rows that together give the agent control of the
+# user's real Chrome.
+#
+#   1. the daemon row makes sure the local MCP server binary exists and runs;
+#   2. the mcp-client row is the *only* tool surface — the daemon speaks MCP
+#      Streamable HTTP, so the in-box bridge publishes its catalog as
+#      mcp__chrome__* with no adapter code of our own;
+#   3. the skills row teaches the agent how to drive those tools.
+#
+# The browser extension is installed by the user, by design: loading an
+# unpacked extension is a decision only the browser's owner can make.
+- insert:
+    - id: chrome-daemon
+      name: 'dsh-chrome/daemon'
+      config:
+        port: 10086
+        autoStart: true
+
+    - id: chrome-mcp
+      name: '@deepseek-ai/dsh-mcp-client'
+      config:
+        serverName: chrome
+        transport: streamable-http
+        url: http://127.0.0.1:10086/mcp
+        toolCallTimeoutMs: 30000
+        # The daemon may still be starting, and the extension may be attached
+        # later; a failed first connect must not abort the profile. The bridge's
+        # own reconnect policy then attaches as soon as the daemon answers.
+        failOnStartupError: false
+
+    - id: chrome-skills
+      name: 'dsh-chrome/skills'

package/lib/daemon.js

@@ -0,0 +1,146 @@
+import { createRequire } from "node:module";
+import { spawn } from "node:child_process";
+import { chmod, copyFile, mkdir, stat } from "node:fs/promises";
+import { dirname, join, resolve } from "node:path";
+import { homedir } from "node:os";
+//#region src/daemon.ts
+/**
+* Daemon lifecycle for the dsh-chrome bundle.
+*
+* This plugin owns *installation and supervision only*. The agent-facing tools
+* come from the in-box `@deepseek-ai/dsh-mcp-client` row in this bundle's
+* patch, because `chrome-daemon` speaks MCP Streamable HTTP directly.
+*
+* The daemon binary is resolved in three steps, each preferred over the next:
+* a binary already installed under the harness home, a prebuilt binary shipped
+* inside this package, or a local `cargo build` of the sibling crate. When none
+* of those work the plugin logs one actionable line and loads anyway — a
+* missing browser bridge must never stop a session from starting.
+*
+* @module dsh-chrome/daemon
+*/
+/** Stable Cordis plugin name. */
+const name = "chrome-daemon";
+/** The daemon's own `serverInfo.name`, used to recognize our own process. */
+const DAEMON_IDENTITY = "dsh-chrome";
+/** Where the installed binary lives. */
+function installedBinaryPath(home = homedir()) {
+	return join(home, ".dsh-chrome", "bin", "chrome-daemon");
+}
+/** Platform token used to name prebuilt binaries shipped in this package. */
+function platformToken(platform = process.platform, arch = process.arch) {
+	return `${platform}-${arch}`;
+}
+/** This package's own directory, resolved from its installed location. */
+function packageRoot() {
+	const require = createRequire(import.meta.url);
+	return dirname(require.resolve("../package.json"));
+}
+async function isFile(path) {
+	try {
+		return (await stat(path)).isFile();
+	} catch {
+		return false;
+	}
+}
+/** Ask a running daemon who it is. */
+async function probe(port, signal) {
+	try {
+		const response = await fetch(`http://127.0.0.1:${port}/status`, { signal: signal ?? AbortSignal.timeout(1500) });
+		if (!response.ok) return void 0;
+		return await response.json();
+	} catch {
+		return;
+	}
+}
+/**
+* Locate a usable daemon binary, installing or building one when needed.
+* @returns the binary's path, or `undefined` when none could be produced.
+*/
+async function resolveBinary(logger) {
+	const installed = installedBinaryPath();
+	if (await isFile(installed)) return installed;
+	const root = packageRoot();
+	const prebuilt = join(root, "bin", `chrome-daemon-${platformToken()}`);
+	if (await isFile(prebuilt)) {
+		await mkdir(dirname(installed), { recursive: true });
+		await copyFile(prebuilt, installed);
+		await chmod(installed, 493);
+		return installed;
+	}
+	const crate = resolve(root, "..", "..", "crates", "chrome-daemon");
+	if (await isFile(join(crate, "Cargo.toml"))) {
+		const built = await cargoBuild(crate, logger);
+		if (built) {
+			await mkdir(dirname(installed), { recursive: true });
+			await copyFile(built, installed);
+			await chmod(installed, 493);
+			return installed;
+		}
+	}
+}
+/** Build the crate in release mode and return the artifact path. */
+async function cargoBuild(crate, logger) {
+	logger?.info("building chrome-daemon from source (first run only)");
+	if (!await new Promise((settle) => {
+		const child = spawn("cargo", ["build", "--release"], {
+			cwd: crate,
+			stdio: "ignore"
+		});
+		child.once("error", () => settle(false));
+		child.once("exit", (code) => settle(code === 0));
+	})) {
+		logger?.warn("cargo build failed; install Rust or ship a prebuilt binary");
+		return;
+	}
+	const artifact = join(crate, "target", "release", "chrome-daemon");
+	return await isFile(artifact) ? artifact : void 0;
+}
+/** Start the daemon and wait until it answers, or report why it did not. */
+async function start(binary, port, logger) {
+	await new Promise((settle) => {
+		const child = spawn(binary, [
+			"start",
+			"--port",
+			String(port)
+		], { stdio: "ignore" });
+		child.once("error", () => settle());
+		child.once("exit", () => settle());
+	});
+	for (let attempt = 0; attempt < 30; attempt += 1) {
+		if ((await probe(port))?.name === "dsh-chrome") return true;
+		await new Promise((wait) => setTimeout(wait, 100));
+	}
+	logger?.warn("chrome-daemon did not become ready; see ~/.dsh-chrome/logs/daemon.log");
+	return false;
+}
+/**
+* Ensure a daemon is serving `port`, installing and starting one if allowed.
+* @param ctx - plugin context, used only for logging.
+* @param config - port and auto-start policy.
+*/
+async function apply(ctx, config = {}) {
+	const port = config.port ?? 10086;
+	const autoStart = config.autoStart !== false;
+	const running = await probe(port);
+	if (running) {
+		if (running.name === "dsh-chrome") {
+			ctx.logger?.info(`chrome-daemon already running on ${port}`);
+			return;
+		}
+		ctx.logger?.warn(`port ${port} is served by "${running.name ?? "unknown"}", not chrome-daemon; set this row's port (and the mcp-client url) to a free port`);
+		return;
+	}
+	if (!autoStart) {
+		ctx.logger?.info("chrome-daemon autoStart is off; start it yourself when needed");
+		return;
+	}
+	const binary = await resolveBinary(ctx.logger);
+	if (!binary) {
+		ctx.logger?.warn("chrome-daemon is not installed. Build it with: cargo build --release (in crates/chrome-daemon), or ship a prebuilt binary in this package's bin/ directory.");
+		return;
+	}
+	await start(binary, port, ctx.logger);
+}
+//#endregion
+export { DAEMON_IDENTITY, apply, installedBinaryPath, name, platformToken, probe, resolveBinary };

package/lib/skills.js

@@ -0,0 +1,40 @@
+import { createRequire } from "node:module";
+import { dirname, join } from "node:path";
+import * as SkillFilesystem from "@deepseek-ai/dsh-skill-filesystem";
+//#region src/skills.ts
+/**
+* dsh-chrome companion plugin: mounts a dsh-skill-filesystem provider scoped to
+* this package's own bundled `skills/` directory, so the Chrome skill is
+* discovered without a manual copy step. Resolves its own installed location
+* through `createRequire`, the technique DeepSeek Harness's own bundles use to
+* find their installed assets.
+*
+* @module dsh-chrome/skills
+*/
+/** Stable Cordis plugin name. */
+const name = "chrome-skills";
+/** Provider name registered on `ctx.skills`. */
+const SKILL_PROVIDER_NAME = "chrome";
+/**
+* Resolve this package's bundled `skills/` directory from its own installed
+* location, independent of the caller's working directory.
+* @returns the absolute path to the bundled skills directory.
+*/
+function resolveBundledSkillsDir() {
+	const require = createRequire(import.meta.url);
+	return join(dirname(require.resolve("../package.json")), "skills");
+}
+/**
+* Mount the bundled skill directory as an isolated provider, contributing only
+* this package's own skill and no default roots.
+* @param ctx - plugin context; skill-filesystem injects `skills` from it.
+*/
+function apply(ctx) {
+	ctx.plugin(SkillFilesystem, {
+		providerName: SKILL_PROVIDER_NAME,
+		includeDefaultRoots: false,
+		customSkillDirs: [resolveBundledSkillsDir()]
+	});
+}
+//#endregion
+export { SKILL_PROVIDER_NAME, apply, name, resolveBundledSkillsDir };

package/package.json

@@ -0,0 +1,54 @@
+{
+  "name": "dsh-chrome-control",
+  "version": "0.1.5",
+  "description": "DeepSeek Harness (DSH) bundle: installs and supervises the chrome-daemon MCP server, exposing the user's real Chrome to the agent as mcp__chrome__* tools.",
+  "type": "module",
+  "license": "MIT",
+  "publishConfig": { "access": "public" },
+  "engines": { "node": ">=22.19" },
+  "main": "lib/daemon.js",
+  "exports": {
+    ".": "./lib/daemon.js",
+    "./daemon": "./lib/daemon.js",
+    "./skills": "./lib/skills.js",
+    "./cordis.patch.yml": "./cordis.patch.yml",
+    "./package.json": "./package.json"
+  },
+  "files": [
+    "lib/**/*.js",
+    "cordis.patch.yml",
+    "skills/**/*"
+  ],
+  "dsh": {
+    "bundle": {
+      "patch": "./cordis.patch.yml"
+    }
+  },
+  "peerDependencies": {
+    "@deepseek-ai/dsh": ">=0.1.1-rc.2"
+  },
+  "peerDependenciesMeta": {
+    "@deepseek-ai/dsh": { "optional": true }
+  },
+  "devDependencies": {
+    "@deepseek-ai/cordis": "^4.0.1",
+    "@deepseek-ai/cordis-plugin-include": "^1.0.6",
+    "@deepseek-ai/cordis-plugin-loader": "^1.0.2",
+    "@deepseek-ai/dsh-app-boot": "^0.1.1-rc.2",
+    "@deepseek-ai/dsh-mcp-client": "^0.1.1-rc.2",
+    "@deepseek-ai/dsh-skill": "^0.1.1-rc.2",
+    "@deepseek-ai/dsh-skill-filesystem": "^0.1.1-rc.2",
+    "@deepseek-ai/dsh-tools": "^0.1.1-rc.2",
+    "@types/node": "^22.20.1",
+    "tsdown": "^0.22.14",
+    "typescript": "^5.7.0",
+    "vitest": "^4.1.11"
+  },
+  "scripts": {
+    "build": "tsdown",
+    "prepare": "tsdown",
+    "typecheck": "tsc --noEmit -p tsconfig.json",
+    "test": "vitest run",
+    "check": "npm run typecheck && npm run build && npm run test"
+  }
+}

package/skills/chrome/SKILL.md

@@ -0,0 +1,125 @@
+---
+name: chrome
+description: |
+  Drive the user's real Chrome browser — navigate, click, type, read pages, screenshot — using their actual logged-in sessions. Use this skill whenever the user wants to act on a website that needs their login, automate a browser task, or read a page that the harness's own browser cannot reach. Also use when the user says "my browser", "my Chrome", "the tab I have open", or names a site they are signed in to.
+---
+
+# Chrome (real browser, real sessions)
+
+These tools drive the **user's own Chrome**, with their cookies and logins, through a local daemon and a browser extension. Tools appear as `mcp__chrome__*`.
+
+## Sessions: one task, one session, one tab group
+
+Every tool takes a required `session`. Pick one name at the start of a task, in kebab-case, describing the **task** and not the site (`flight-compare`, not `united`). Reuse it for every call in that task, even across different sites. Switching names mid-task is the single most common cause of tabs scattering across groups.
+
+On the **first** `navigate` of a task, also pass `group_title` — a short human-readable label in the user's language. Tell the user once that the task's pages are collected under that group and that you will close them whenever they ask.
+
+Only call `close_session` when the user explicitly asks ("close those tabs"). Never close a group on your own initiative.
+
+## The loop
+
+1. `navigate` to open a page (`newTab: true` when pages must coexist).
+2. `snapshot` to read it. This returns an accessibility outline, not HTML — one element per line, indented by nesting:
+
+   ```
+   heading "Sign in"
+   @e46 textbox "Email" val="you@example.com"
+   @e50 checkbox "Remember me" [x]
+   @e51 button "Sign in"
+   ```
+
+   A leading `@e…` is the ref you pass to `click`/`fill`; a line without one is context you can read but not act on. Trailing markers: `[x]`/`[ ]`/`[/]` for checked, unchecked, and mixed, `*` focused, `-` disabled, `×N` for N identical rows collapsed into one.
+3. Act with `click` / `fill` using an `@e` ref from that snapshot.
+4. `snapshot` again to confirm the result before the next action.
+
+Take a fresh snapshot after anything that changes the page. A ref names the underlying element rather than its position in the outline, so it usually survives an unrelated change — but re-reading is what tells you the action worked.
+
+### Snapshot options
+
+- `mode` — `interactive` (default) keeps the controls plus headings, table cells, and images. `full` keeps every node, for when the default has filtered away something you needed. `text` keeps prose and issues no refs.
+- `maxTokens` — ceiling on the outline, 2000 by default. Output stops at the last element that fits and ends with `# truncated: N more nodes`; when you see that, narrow the page (scroll, open a subsection) rather than raising the ceiling.
+- `maxDepth` — drop elements nested deeper than this, to see a large page's shape first.
+- `diff: true` — mark what changed since this tab's previous snapshot: `[+]` added, `[~]` changed, and a trailing `# removed:` line. Every current element is still listed, so all refs stay usable. Use it after an action on a page whose bulk does not change.
+
+On a large page where you want one specific control, `find` is cheaper than a snapshot: give it a plain-language query ("sign in button") and it returns the best-matching `@e` refs. It adds to the ref table rather than replacing it, so refs from your last snapshot stay usable. Use `snapshot` when you need the page's structure, `find` when you already know what you are looking for.
+
+## Prefer @e refs over CSS selectors
+
+`@e` refs identify the element itself, so they survive both the hashed class names of modern frameworks and unrelated edits elsewhere on the page. Reach for a CSS selector only when the target has no ref, and for `evaluate` only when you need something the snapshot cannot express — an attribute like `href`, a scroll, or a complex event sequence.
+
+## Text input
+
+`fill` handles `<input>`, `<textarea>`, and `contenteditable` rich editors (ProseMirror, Lexical, Slate, Quill), firing the input events those editors listen for. It is **clear-and-insert**: existing content is replaced. To append, read the current value with `evaluate`, concatenate, then `fill` the result.
+
+## Dropdowns and focus
+
+`fill` cannot drive a native `<select>`. Use `select` instead: it matches the option's `value` first, then its exact visible text, then a substring, so both `"CA"` and `"California"` can work. When nothing matches, the error lists the available options — read it rather than guessing again.
+
+`focus` gives an element keyboard focus without clicking it, for fields that react badly to a click (date pickers that open a popup, inputs that select-all). Follow it with `key_type` or `send_keys`.
+
+## Waiting
+
+Prefer `wait_for_selector`: it polls until an element is visible, or gone with `state: "hidden"`, and returns the moment the condition holds. `navigate` already waits for load, so do not add a blind wait after it.
+
+`wait` is a fixed sleep capped at 3000ms. Reach for it only when there is nothing to wait *for* — a settling animation, a debounce — never as a substitute for `wait_for_selector`.
+
+## Scrolling
+
+`scroll` takes a `direction` ("down" by default) with optional `pixels`, or an explicit `deltaY`. It reports `moved` and `atEnd`, so you can tell an infinite feed from the bottom of the page instead of scrolling blindly. Pass a `selector` to scroll one pane rather than the window.
+
+`scroll_into_view` scrolls an element into view and returns its geometry. Call it before `mouse_click` or `hover`, whose coordinates only mean anything for a visible element.
+
+## Reading page text
+
+To read an article, use `get_text`, not `snapshot`: a snapshot describes structure and controls, while `get_text` returns prose. By default it strips navigation, headers, footers, sidebars, and cookie banners. If that removes something you needed, retry with `raw: true` for the body's verbatim `innerText`. Output is capped (`maxChars`, default 20000) and reports `truncated`.
+
+## Network and dialogs
+
+`network` lists the tab's requests with method, URL, status, and a `requestId`; filter by `method`, `status`, or a URL substring. Use it to find the API call behind a page, or to see why something failed. `network_detail` returns one request's headers, and with `body: true` its response body.
+
+Requests are only recorded from the moment the tools attached to that tab, so a request made during the very first `navigate` may be missing — reload if you need it.
+
+`dialog` answers a native `alert`, `confirm`, or `prompt` with `action: "accept"` or `"dismiss"`, passing `text` for a prompt. **An open dialog blocks every other tool on that tab** — if calls suddenly start timing out after a click, an unanswered dialog is the likeliest cause. Call it only once a dialog is actually open; it errors when there is none.
+
+## Submitting and special keys
+
+Prefer clicking the submit button with `click`. When there is no button, use `send_keys` with `Enter`. `send_keys` also takes chords such as `Control+A` or a bare `Escape` to dismiss a modal.
+
+## When a page ignores clicks
+
+Some pages (banking portals, captchas) check `event.isTrusted` and ignore the synthetic events `click` and `fill` produce. For those, use the trusted-input tools, which generate real browser input:
+
+- `mouse_click` — click at viewport coordinates. Get coordinates from a `screenshot`, from `scroll_into_view`, or from `evaluate` returning `getBoundingClientRect()`.
+- `hover` — move the pointer over an element or point, for menus and tooltips that only open on hover.
+- `key_type` — type into whatever is focused.
+- `send_keys` — press one key or chord.
+
+Reach for these only after a normal `click` or `fill` has failed; coordinates are far more brittle than `@e` refs.
+
+## Screenshots
+
+`screenshot` returns the image directly, so you can simply look at it. Pass `selector` to capture one element, or `format: "jpeg"` with `quality` to shrink a large capture.
+prefer snapshot over screenshot use screenshot when snapshot not work.
+
+## Evaluate tips
+
+- Return compact data. Use `JSON.stringify(value)` without indentation — pretty-printing inflates a large result until it is truncated.
+- Calls share the page's JavaScript realm, so re-declaring the same `const` twice throws. Wrap each call's body in an IIFE: `(() => { const x = 1; return x })()`.
+
+## Reading the user's open tab
+
+To act on a page the user is already looking at ("the invoice I have open"), call `find_tab` with `active: true`. That borrows the tab they are viewing rather than opening a new one. Otherwise `find_tab` takes a full `url` and searches this session's own tabs.
+
+## Known limits
+
+- **Cross-origin iframes**: tools act on the top frame. If the target lives in an iframe from another origin, navigate directly to the iframe's URL.
+- **Trusted input activates the tab.** `hover`, `mouse_click`, `key_type`, and `send_keys` bring their tab to the foreground, because Chrome only delivers real input to a focused tab. The DOM-level tools work fine on a hidden tab.
+- **Network history starts at attach.** `network` cannot show requests made before the tools attached to that tab.
+- **A debugging banner is normal.** Chrome shows a "being debugged" notice while these tools are attached, and DevTools cannot be open on the same tab at the same time. This is how the bridge works, not a fault — mention it if the user is surprised.
+
+## When a tool fails
+
+The error text names the fix. Two cases matter:
+
+- **"No Chrome extension is attached"** — Chrome is closed, the extension is not loaded, or its popup toggle is off. Ask the user to open Chrome, load the extension at `chrome://extensions` (Developer mode → Load unpacked), and check the toggle in its popup. Do not try to install it yourself; loading an unpacked extension is the browser owner's decision.
+- **The tools are missing entirely** — the daemon is not running. It normally starts with the harness; if it did not, tell the user to run `~/.dsh-chrome/bin/chrome-daemon start`. Never run `stop` or `restart` on your own.