@momasiku/dsh-pilot 0.5.14 → 0.5.16

package/README.md

@@ -2,14 +2,16 @@
 
 # dsh-pilot
 
+## 让 DSH 看见并高效地操作你的电脑
+
 **Give DeepSeek Harness hands and eyes on your desktop.**
 
+[![awesome · DSH plugin](https://awesome-dsh-plugin.com/badge.svg)](https://awesome-dsh-plugin.com/p/momasiku/dsh-pilot/)
 [![npm](https://img.shields.io/npm/v/@momasiku/dsh-pilot?label=npm)](https://www.npmjs.com/package/@momasiku/dsh-pilot)
 [![compat](https://github.com/momasiku/dsh-pilot/actions/workflows/compat.yml/badge.svg)](https://github.com/momasiku/dsh-pilot/actions/workflows/compat.yml)
 [![license](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
 
 ![platform](https://img.shields.io/badge/platform-Windows-0078d4.svg)
-![DSH](https://img.shields.io/badge/DSH-0.1.5--rc.2%20%E2%86%92%200.2.x-5b50ed.svg)
 
 [English](#why-this-exists) · [中文说明](#中文说明)
 
@@ -232,6 +234,18 @@ and what `dsh plugin add` checks before it installs anything. Requirements:
 
 ### Recent changes
 
+**0.5.14** — the coordinate contract stops guessing how big the image you see is.
+
+- **The request pipeline resizes again after capture**, and this plugin never saw
+  that step: on a 2560x1600 display the frame reached the model as 1610x1006 while
+  the contract said "delivered at capture size … do not rescale", so every reading
+  taken off it was 1.59x off. The contract now **predicts** the projected size from
+  the image budget the deployment publishes, and prints the size and the multiplier:
+  `2560x1600 capture -> 1609x1006 preview, multiply every reading by 1.5911`. A frame
+  whose area fits the budget arrives untouched, so framing the window or a region
+  gives a true 1:1 frame — 1365x960 does, 1024x768 does, 4K and 5K say 2.2642x and
+  3.0189x instead.
+
 **0.5.12** — a cold tree, and an empty list that explained nothing.
 
 - **Chromium builds its accessibility tree lazily**, and the first caller pays for it:
@@ -628,6 +642,10 @@ dsh plugin --profile <你的profile> add "file:E:\path\to\dsh-pilot"
 
 ### 最近改动
 
+**0.5.14** —— 坐标契约不再猜「你看到的那张图有多大」。
+
+- **请求管线会在抓取之后再缩一次**，而插件看不到这一步：2560×1600 的屏幕上，帧送到模型时是 1610×1006，契约却写着「交付即抓取尺寸……不要缩放」，于是每个读数都偏 1.59 倍。现在契约用部署公布的图片预算**预测**投影尺寸，并把尺寸和乘数一起写出来：`2560x1600 抓取 → 1609x1006 预览，把每个读数乘 1.5911`。面积落在预算内的帧会原样送达，所以抓窗口或抓区域能得到真正的 1:1 —— 1365×960 可以，1024×768 可以；4K 和 5K 则分别写 2.2642 倍和 3.0189 倍。
+
 **0.5.12** —— 冷树，以及一个什么都没说的空列表。
 
 - **Chromium 的无障碍树是懒加载的**，而且由第一个来问的人付账：实测重启后在 1.2 秒时只有 3 个元素，下一次调用 0.3 秒时是 77 个。信任第一个答案的人会认定「这窗口没有树」，白白退回像素。现在 `controls` 在「worker 还没见过完整树、且结果异常少」时会等 300 ms 重试一次。

package/lib/index.js

@@ -789,36 +789,28 @@ function coordinateContract(value) {
 	const height = Number(capture?.imageHeight);
 	const sizeKnown = Number.isFinite(width) && width > 0 && Number.isFinite(height) && height > 0;
 	const frame = sizeKnown ? `${width}x${height}` : 'the captured frame';
-	const delivered = capture?.image;
-	const preview = capture?.requestPreview;
-	const predicted = Number.isFinite(Number(preview?.width)) && Number(preview.width) > 0 && Number.isFinite(Number(preview?.height)) && Number(preview.height) > 0
-		? { width: Number(preview.width), height: Number(preview.height), maxPixels: Number(preview.maxPixels) }
-		: undefined;
-	const final = predicted ?? (delivered === undefined || delivered === null ? undefined : { width: Number(delivered.width), height: Number(delivered.height) });
-	const kx = final !== undefined && sizeKnown && final.width > 0 ? width / final.width : 1;
-	const ky = final !== undefined && sizeKnown && final.height > 0 ? height / final.height : 1;
-	const oneToOne = Math.abs(kx - 1) < 0.002 && Math.abs(ky - 1) < 0.002;
-	const budgetLine = predicted?.maxPixels > 0 ? `This deployment's image budget is ${predicted.maxPixels} px. ` : '';
-	const guard = 'The message carrying this frame states its own preview size; if it differs from the size named here, multiply every reading by capture_width / preview_width before passing it to desktop_control, because a step after capture decides it.';
-	if (oneToOne) {
-		return [
-			`Image pixel (0,0) is the top-left of this frame, which was captured at ${frame} physical screen pixels. Both halves of this device are per-monitor DPI aware.`,
-			`${budgetLine}At that size one image pixel is one real screen pixel: to act on a feature at image (x,y), call desktop_control with exactly x and y.`,
-			guard
-		].join(' ');
+	const scale = deliveredScale(capture);
+	const storeShrank = !scaleIsOne(scale);
+	// No prediction. The request pipeline resizes a frame to fit the model's image
+	// budget, after this plugin has let go of it, and the budget it uses is not the
+	// deployment's admission limit the plugin can read: measured 64,000,000 in
+	// imageLimits.maxImagePixels against an actual projection of ~1,640,000. A
+	// predicted size built from the wrong number reads as a promise, so the contract
+	// stops predicting and points at the one size that is not a guess - the preview
+	// size the message itself states.
+	const lines = [
+		`Image pixel (0,0) is the top-left of this frame, which was captured at ${frame} physical screen pixels; both halves of this device are per-monitor DPI aware.`
+	];
+	if (storeShrank) {
+		lines.push(axesAgree(scale)
+			? `The attachment store already delivered it at ${printed((scale.x + scale.y) / 2)}x that.`
+			: `The attachment store already delivered it at ${printed(scale.x)}x horizontally and ${printed(scale.y)}x vertically.`);
 	}
-	if (!sizeKnown || final === undefined) return guard;
-	const axes = Math.abs(kx - ky) <= 0.005 * Math.max(kx, ky);
-	const multiply = axes
-		? `multiply every reading by ${printed(kx)}`
-		: `multiply x readings by ${printed(kx)} and y readings by ${printed(ky)}`;
-	const shown = predicted === undefined ? 'delivered' : 'the request pipeline will show it to you';
-	return [
-		`This frame was captured at ${frame} physical screen pixels and ${shown} at ${final.width}x${final.height} (${printed(final.width / width)}x the capture), so image pixels are NOT screen pixels.`,
-		`${budgetLine}To act on a feature at image (x,y), ${multiply} and pass those physical screen pixels to desktop_control, or read the ruler labels, which are already screen coordinates.`,
-		'A capture whose area is at or under the budget arrives at capture size: framing the window or a region instead of the whole screen gives a true 1:1 frame.',
-		guard
-	].join(' ');
+	lines.push('The image you are looking at may be smaller again: the request pipeline resizes a frame to fit the model image budget, and this plugin cannot see that step, so do not trust the capture size as the size on screen. This same message states what it settled on as "request preview WxH" - that number is authoritative.');
+	lines.push(sizeKnown
+		? `If the preview is not ${frame}, multiply every reading by ${width} / preview_width (and ${height} / preview_height) before passing it to desktop_control; if it is, call desktop_control with exactly x and y. Ruler labels, when drawn, are already screen coordinates.`
+		: 'If the preview is smaller than the capture, multiply every reading by capture_width / preview_width before passing it to desktop_control.');
+	return lines.join(' ');
 }
 
 /**
@@ -1453,6 +1445,10 @@ function apply(ctx, config) {
 			// Copy the declared fields only: the helper's own bookkeeping keys
 			// must not reach the closed output schema.
 			const value = { action: args.action };
+			// A one-element list can still arrive unrolled from the helper; the output
+			// schema wants an array, and a string here is a rejected result, not a bug the
+			// caller can see.
+			if (typeof acted.controls === 'string') acted.controls = [acted.controls];
 			for (const field of ['cursor', 'foregroundTitle', 'foregroundClass', 'foregroundProcess', 'notes', 'windowCount', 'actedAt', 'controls', 'controlCount', 'controlsTruncated', 'controlsWindow']) {
 				if (acted[field] !== undefined) value[field] = acted[field];
 			}

package/lib/scripts/_dsh-uia.ps1

@@ -124,7 +124,11 @@ function Format-DshUiaNodes {
         $lines.Add(('{0} [{1}] ''{2}'' @ {3}{4}' -f $n.index, $n.role, $name, $rect, $tail))
     }
     if ($Truncated) { $lines.Add(('... truncated at the node cap ({0} elements in total)' -f $Total)) }
-    return $lines.ToArray()
+    # The unary comma stops PowerShell unrolling a one-element array on return:
+    # without it a window that exposes a single control comes back as a bare
+    # string, and the result then fails its own schema ("controls must be an
+    # array"). Measured on a LibreOffice window that publishes exactly one.
+    return ,($lines.ToArray())
 }
 
 <# Resolve an index back to an element by re-reading the same listing. #>

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@momasiku/dsh-pilot",
-  "version": "0.5.14",
+  "version": "0.5.16",
   "description": "Desktop vision and control for DeepSeek Harness - the whole Windows machine, any application, not browser automation: screen_view captures the live screen as an image the model can see, desktop_control drives the mouse, keyboard, and windows against the exact same pixel space; desktop_sequence runs a whole batch of actions in one call and looks once at the end.",
   "license": "MIT",
   "author": "momasiku",