dsh-click 0.1.3 → 0.2.0

package/CHANGELOG.md

@@ -5,6 +5,18 @@ All notable changes to this project are documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.2.0] - 2026-08-26
+
+### Added
+
+- OCR and visual grounding providers for no-UIA windows.
+
+## [0.1.4] - 2026-08-23
+
+### Added
+
+- Three-interface assertions (model schema + canonical value + content blocks) for `screen_shot` and `click`, completing the U2 tool-surface coverage alongside the existing `screen_read` assertion.
+
 ## [0.1.3] - 2026-08-22
 
 ### Changed

package/README.es.md

@@ -103,6 +103,7 @@ Todos los ajustes son campos `Config` de Schemastery (modificables desde cordis.
 | `maxTreeDepth` | `32` | Profundidad máxima del recorrido del árbol de accesibilidad (1..64) |
 | `maxTextLength` | `200` | Longitud de truncado de las cadenas visibles para el modelo (16..10000) |
 | `rollbackEnabled` | `true` | Respaldar y restaurar el texto del control cuando `type` falla |
+| `ocr.enabled` / `command` / `language` | `true` / `tesseract` / `eng` | OCR opcional para `screen_find` (probado al montar; degrada a no disponible sin tesseract) |
 
 Ejemplo de sobrescritura en el parche de tu perfil:
 
@@ -157,7 +158,7 @@ Ejemplo de sobrescritura en el parche de tu perfil:
 pnpm install        # node ^22.19 || >=24
 pnpm run typecheck  # tsc: src + tests contra el checkout local del harness
 pnpm run typecheck:ci  # tsc contra los tipos publicados 0.1.1-rc.2 (sin paths)
-pnpm test           # vitest: 56 tests, 8 suites (el smoke del helper corre en Windows)
+pnpm test           # vitest: 66 tests, 11 archivos (el smoke del helper corre en Windows)
 pnpm run build      # bundle tsdown + declaraciones tsc (lib/)
 pnpm run verify:self-contained  # las especificaciones de dependencias resuelven desde el registry
 pnpm run verify:artifacts       # cara ESM construida + helper nativo presentes
@@ -171,6 +172,7 @@ pnpm pack           # el tarball publicado
 ## Contributors
 
 - [@PerryLink](https://github.com/PerryLink) — creador y mantenedor: superficie de herramientas, frontera de seguridad de acciones, helper nativo de Windows, sanitizadores y la documentación en cinco idiomas.
+- [@Mchsd](https://github.com/Mchsd) — añadió la opción `auditSessionEvents` para harnesses cuyo lector de sesión rechaza los eventos de auditoría de `dsh-click` (#2).
 
 ## PerryLink DSH Plugin Family
 

package/README.hi.md

@@ -103,6 +103,7 @@ dsh --profile web --dump-config | grep -A2 'id: dsh-click'
 | `maxTreeDepth` | `32` | एक्सेसिबिलिटी ट्री-वॉक की अधिकतम गहराई (1..64) |
 | `maxTextLength` | `200` | सैनिटाइज़्ड मॉडल-दृश्य स्ट्रिंग्स की काट-छाँट लंबाई (16..10000) |
 | `rollbackEnabled` | `true` | `type` विफल होने पर नियंत्रण टेक्स्ट का बैकअप और पुनर्स्थापन |
+| `ocr.enabled` / `command` / `language` | `true` / `tesseract` / `eng` | `screen_find` के लिए वैकल्पिक OCR (माउंट पर जाँच; tesseract अनुपस्थित होने पर अनुपलब्ध) |
 
 आपके प्रोफ़ाइल पैच में ओवरराइड उदाहरण:
 
@@ -157,7 +158,7 @@ dsh --profile web --dump-config | grep -A2 'id: dsh-click'
 pnpm install        # node ^22.19 || >=24
 pnpm run typecheck  # tsc: src + tests स्थानीय हार्नेस चेकआउट के विरुद्ध
 pnpm run typecheck:ci  # tsc प्रकाशित 0.1.1-rc.2 प्रकारों के विरुद्ध (बिना paths)
-pnpm test           # vitest: 56 टेस्ट, 8 सुइट (हेल्पर स्मोक Windows पर चलता है)
+pnpm test           # vitest: 66 टेस्ट, 11 फ़ाइलें (हेल्पर स्मोक Windows पर चलता है)
 pnpm run build      # tsdown बंडल + tsc घोषणाएँ (lib/)
 pnpm run verify:self-contained  # निर्भरता स्पेक registry से हल होती हैं
 pnpm run verify:artifacts       # निर्मित ESM फ़ेस + नेटिव हेल्पर मौजूद
@@ -171,6 +172,7 @@ pnpm pack           # प्रकाशित tarball
 ## Contributors
 
 - [@PerryLink](https://github.com/PerryLink) — निर्माता और मेंटेनर: टूल सतह, क्रिया सुरक्षा सीमा, Windows नेटिव हेल्पर, सैनिटाइज़र और पाँच-भाषा दस्तावेज़।
+- [@Mchsd](https://github.com/Mchsd) — उन harnesses के लिए `auditSessionEvents` ऑप्ट-आउट जोड़ा जिनका सत्र रीडर `dsh-click` ऑडिट इवेंट को अस्वीकार करता है (#2)।
 
 ## PerryLink DSH Plugin Family
 

package/README.md

@@ -104,6 +104,7 @@ All tunables are Schemastery `Config` fields (changeable from cordis.yml). An id
 | `maxTreeDepth` | `32` | Maximum accessibility tree-walk depth (1..64) |
 | `maxTextLength` | `200` | Truncation length for sanitized model-visible strings (16..10000) |
 | `rollbackEnabled` | `true` | Back up and restore control text when `type` fails |
+| `ocr.enabled` / `command` / `language` | `true` / `tesseract` / `eng` | Optional OCR for the `screen_find` path (probed at mount; degrades to unavailable when tesseract is absent) |
 
 Example override in your profile patch:
 
@@ -158,7 +159,7 @@ Example override in your profile patch:
 pnpm install        # node ^22.19 || >=24
 pnpm run typecheck  # tsc: src + tests against the local harness checkout
 pnpm run typecheck:ci  # tsc against the published 0.1.1-rc.2 types (no paths)
-pnpm test           # vitest: 56 tests, 8 suites (helper smoke runs on Windows)
+pnpm test           # vitest: 66 tests, 11 files (helper smoke runs on Windows)
 pnpm run build      # tsdown bundle + tsc declarations (lib/)
 pnpm run verify:self-contained  # dependency specs resolve from the registry
 pnpm run verify:artifacts       # built ESM face + native helper present
@@ -172,6 +173,7 @@ pnpm pack           # the published tarball
 ## Contributors
 
 - [@PerryLink](https://github.com/PerryLink) — creator and maintainer: tool surface, action safety boundary, Windows native helper, sanitizers, and the five-language docs.
+- [@Mchsd](https://github.com/Mchsd) — added the `auditSessionEvents` opt-out for harnesses whose session reader rejects the `dsh-click` audit events (#2).
 
 ## PerryLink DSH Plugin Family
 

package/README.pt.md

@@ -103,6 +103,7 @@ Todos os ajustes são campos `Config` do Schemastery (alteráveis pelo cordis.ym
 | `maxTreeDepth` | `32` | Profundidade máxima do percurso da árvore de acessibilidade (1..64) |
 | `maxTextLength` | `200` | Comprimento de truncamento das strings visíveis ao modelo (16..10000) |
 | `rollbackEnabled` | `true` | Fazer backup e restaurar o texto do controle quando `type` falha |
+| `ocr.enabled` / `command` / `language` | `true` / `tesseract` / `eng` | OCR opcional para `screen_find` (detectado na montagem; degrada para indisponível sem tesseract) |
 
 Exemplo de sobrescrita no patch do seu perfil:
 
@@ -157,7 +158,7 @@ Exemplo de sobrescrita no patch do seu perfil:
 pnpm install        # node ^22.19 || >=24
 pnpm run typecheck  # tsc: src + tests contra o checkout local do harness
 pnpm run typecheck:ci  # tsc contra os tipos publicados 0.1.1-rc.2 (sem paths)
-pnpm test           # vitest: 56 testes, 8 suítes (o smoke do helper roda no Windows)
+pnpm test           # vitest: 66 testes, 11 arquivos (o smoke do helper roda no Windows)
 pnpm run build      # bundle tsdown + declarações tsc (lib/)
 pnpm run verify:self-contained  # especificações de dependências resolvem pelo registry
 pnpm run verify:artifacts       # face ESM construída + helper nativo presentes
@@ -171,6 +172,7 @@ pnpm pack           # o tarball publicado
 ## Contributors
 
 - [@PerryLink](https://github.com/PerryLink) — criador e mantenedor: superfície de ferramentas, fronteira de segurança das ações, helper nativo do Windows, sanitizadores e a documentação em cinco idiomas.
+- [@Mchsd](https://github.com/Mchsd) — adicionou a opção `auditSessionEvents` para harnesses cujo leitor de sessão rejeita os eventos de auditoria do `dsh-click` (#2).
 
 ## PerryLink DSH Plugin Family
 

package/README.zh.md

@@ -103,6 +103,7 @@ dsh --profile web --dump-config | grep -A2 'id: dsh-click'
 | `maxTreeDepth` | `32` | 无障碍树遍历最大深度（1..64） |
 | `maxTextLength` | `200` | 脱敏后模型可见字符串的截断长度（16..10000） |
 | `rollbackEnabled` | `true` | `type` 失败时备份并还原控件文本 |
+| `ocr.enabled` / `command` / `language` | `true` / `tesseract` / `eng` | `screen_find` 的可选 OCR（挂载时探测；无 tesseract 时降级为不可用） |
 
 profile patch 中的覆盖示例：
 
@@ -157,7 +158,7 @@ profile patch 中的覆盖示例：
 pnpm install        # node ^22.19 || >=24
 pnpm run typecheck  # tsc：src + tests，对照本地 harness checkout
 pnpm run typecheck:ci  # tsc：对照已发布的 0.1.1-rc.2 类型（无 paths）
-pnpm test           # vitest：56 个测试、8 个套件（helper 冒烟在 Windows 上运行）
+pnpm test           # vitest：66 个测试、11 个文件（helper 冒烟在 Windows 上运行）
 pnpm run build      # tsdown bundle + tsc 声明（lib/）
 pnpm run verify:self-contained  # 依赖声明全部来自 registry
 pnpm run verify:artifacts       # 构建产物 ESM 面 + 原生 helper 齐全
@@ -171,6 +172,7 @@ pnpm pack           # 发布用 tarball
 ## Contributors
 
 - [@PerryLink](https://github.com/PerryLink) —— 创建者与维护者：工具面、操作安全边界、Windows 原生 helper、脱敏层与五语文档。
+- [@Mchsd](https://github.com/Mchsd) —— 新增 `auditSessionEvents` 开关，供会话读取器拒绝 `dsh-click` 审计事件的主机退出审计追加（#2）。
 
 ## PerryLink DSH Plugin Family
 

package/THIRD_PARTY_NOTICES.md

@@ -14,7 +14,7 @@ published tarball; these are install-time dependencies:
 | [typescript](https://github.com/microsoft/TypeScript) | `^5.9.0` | Apache-2.0 | Build-time declaration emission (`lib/types/`) |
 | [@deepseek-ai/cordis](https://www.npmjs.com/package/@deepseek-ai/cordis) | `^4.0.1` (peer) | See package | The plugin runtime |
 | [@deepseek-ai/schemastery](https://www.npmjs.com/package/@deepseek-ai/schemastery) | `^3.18.0` (peer) | See package | Configuration schema |
-| `@deepseek-ai/dsh-*` peers | `0.1.1-rc.2` (peer) | See packages | Official harness seams (`dsh-tools`, `dsh-session`, `dsh-subprocess`, `dsh-llm`, `dsh-attachment`, `dsh-agent`, `dsh-user-approval`) |
+| `@deepseek-ai/dsh-*` peers | `>=0.1.0-rc.8 <0.2.0` | See packages | Official harness seams (`dsh-tools`, `dsh-session`, `dsh-subprocess`, `dsh-llm`, `dsh-attachment`, `dsh-agent`, `dsh-user-approval`) |
 
 At runtime the plugin only talks to the harness services listed as
 peerDependencies; it performs no network requests of its own.

package/cordis.patch.yml

@@ -51,3 +51,12 @@
         # Back up and restore control text when `type` fails (true = restore
         # the original text; false = leave the partially typed text).
         rollbackEnabled: true
+        # Optical character recognition for windows without a UIA tree.
+        ocr:
+          # Enable the OCR path in screen_find (true = probe tesseract at
+          # mount and use it when available).
+          enabled: true
+          # tesseract executable name.
+          command: tesseract
+          # tesseract language tag.
+          language: eng

package/lib/index.js

@@ -1,9 +1,11 @@
 import z from "@deepseek-ai/schemastery";
 import { createHash, randomUUID } from "node:crypto";
 import { existsSync } from "node:fs";
-import path from "node:path";
+import path, { join } from "node:path";
 import { fileURLToPath } from "node:url";
 import { defineTool } from "@deepseek-ai/dsh-tools";
+import { mkdtemp, rm, writeFile } from "node:fs/promises";
+import { tmpdir } from "node:os";
 //#region src/events.ts
 /** The observation audit event type. */
 const OBSERVED_EVENT = "dsh-click/observed";
@@ -348,7 +350,12 @@ const Config = z.object({
 	maxElements: z.number().min(1).max(MAX_ELEMENTS).default(500),
 	maxTreeDepth: z.number().min(1).max(64).default(32),
 	maxTextLength: z.number().min(16).max(MAX_TEXT_LENGTH).default(200),
-	rollbackEnabled: z.boolean().default(true)
+	rollbackEnabled: z.boolean().default(true),
+	ocr: z.object({
+		enabled: z.boolean().default(true),
+		command: z.string().min(1).max(256).default("tesseract"),
+		language: z.string().min(1).max(32).default("eng")
+	})
 });
 /** Throw the standard fail-loud config error for one invalid field. */
 function invalid(field, detail) {
@@ -398,6 +405,12 @@ function resolveConfig(config) {
 	if (!Number.isInteger(maxTextLength) || maxTextLength < 16 || maxTextLength > 1e4) invalid("maxTextLength", `must be an integer between 16 and ${MAX_TEXT_LENGTH}`);
 	const rollbackEnabled = config?.rollbackEnabled ?? true;
 	if (typeof rollbackEnabled !== "boolean") invalid("rollbackEnabled", "must be a boolean");
+	const ocrEnabled = config?.ocr?.enabled ?? true;
+	if (typeof ocrEnabled !== "boolean") invalid("ocr.enabled", "must be a boolean");
+	const ocrCommand = config?.ocr?.command ?? "tesseract";
+	if (typeof ocrCommand !== "string" || ocrCommand.length === 0 || ocrCommand.length > 256) invalid("ocr.command", "must be a non-empty string of at most 256 characters");
+	const ocrLanguage = config?.ocr?.language ?? "eng";
+	if (typeof ocrLanguage !== "string" || ocrLanguage.length === 0 || ocrLanguage.length > 32) invalid("ocr.language", "must be a non-empty string of at most 32 characters");
 	return Object.freeze({
 		requireApproval,
 		autoApproveMatchers: Object.freeze(autoApproveMatchers),
@@ -413,7 +426,12 @@ function resolveConfig(config) {
 		maxElements,
 		maxTreeDepth,
 		maxTextLength,
-		rollbackEnabled
+		rollbackEnabled,
+		ocr: Object.freeze({
+			enabled: ocrEnabled,
+			command: ocrCommand,
+			language: ocrLanguage
+		})
 	});
 }
 //#endregion
@@ -538,7 +556,7 @@ var ObservationStore = class {
 * @module dsh-click/version
 */
 /** Plugin version advertised in the helper-protocol handshake. */
-const VERSION = "0.1.3";
+const VERSION = "0.2.0";
 /** The helper protocol revision; bump only on incompatible wire changes. */
 const HELPER_PROTOCOL_VERSION = 1;
 //#endregion
@@ -922,6 +940,209 @@ async function sessionAcceptsImages(agent, ctx, signal) {
 	}
 }
 //#endregion
+//#region src/vision/ocr.ts
+/**
+* Optical character recognition seam for windows without a UIA tree. OCR is
+* optional and probed at mount: the `tesseract` provider is used only when the
+* binary resolves through the subprocess service, otherwise an unavailable
+* provider fails closed with a model-readable reason. Recognized words carry
+* their pixel boxes, which `screen_find` maps to screen coordinates for the
+* coordinate-based `click` path.
+*
+* @module dsh-click/vision/ocr
+*/
+/** Bytes cap for one tesseract TSV stdout. */
+const OCR_STDOUT_BYTES = 16777216;
+/**
+* Build the tesseract command line: write words as TSV to stdout.
+* @param command - tesseract executable.
+* @param imagePath - absolute PNG path.
+* @param language - tesseract language tag.
+* @returns the argv (argv[0] is the executable).
+*/
+function tesseractArgv(command, imagePath, language) {
+	return [
+		command,
+		imagePath,
+		"stdout",
+		"-l",
+		language,
+		"tsv"
+	];
+}
+/**
+* Parse tesseract TSV output into word boxes. Non-word rows, empty text, and
+* malformed numeric fields are skipped; a hostile or truncated stdout cannot
+* produce bogus coordinates.
+* @param tsv - tesseract TSV text from stdout.
+* @returns the recognized words.
+*/
+function parseTesseractTsv(tsv) {
+	const words = [];
+	for (const line of tsv.split(/\r?\n/u)) {
+		if (line === "" || line.startsWith("level")) continue;
+		const columns = line.split("	");
+		if (columns.length < 12 || columns[0] !== "5") continue;
+		const left = Number(columns[6]);
+		const top = Number(columns[7]);
+		const width = Number(columns[8]);
+		const height = Number(columns[9]);
+		const confidence = Number(columns[10]);
+		const text = columns[11] ?? "";
+		if (text.trim() === "") continue;
+		if (!Number.isFinite(left) || !Number.isFinite(top) || !Number.isFinite(width) || !Number.isFinite(height)) continue;
+		words.push({
+			text: text.trim(),
+			box: {
+				x: left,
+				y: top,
+				width,
+				height
+			},
+			confidence: Number.isFinite(confidence) ? confidence : 0
+		});
+	}
+	return words;
+}
+/** A provider that cannot run: `recognize` fails closed with the given reason. */
+var UnavailableOcrProvider = class {
+	id;
+	reason;
+	available = false;
+	/** @param id - provider id. @param reason - model-readable unavailability reason. */
+	constructor(id, reason) {
+		this.id = id;
+		this.reason = reason;
+	}
+	get unavailableReason() {
+		return this.reason;
+	}
+	recognize(_pngBase64, _signal) {
+		return Promise.reject(new DshClickError(`OCR unavailable: ${this.reason}`, "OCR_UNAVAILABLE"));
+	}
+};
+/** The tesseract OCR provider over `ctx.subprocess`. */
+var TesseractOcrProvider = class {
+	subprocess;
+	command;
+	language;
+	id = "tesseract";
+	available = true;
+	/**
+	* @param subprocess - the subprocess service that runs tesseract.
+	* @param command - tesseract executable name.
+	* @param language - tesseract language tag.
+	*/
+	constructor(subprocess, command, language) {
+		this.subprocess = subprocess;
+		this.command = command;
+		this.language = language;
+	}
+	async recognize(pngBase64, signal) {
+		const dir = await mkdtemp(join(tmpdir(), "dsh-click-ocr-"));
+		const imagePath = join(dir, "shot.png");
+		try {
+			await writeFile(imagePath, Buffer.from(pngBase64, "base64"));
+			const handle = this.subprocess.spawn({
+				argv: tesseractArgv(this.command, imagePath, this.language),
+				cwd: tmpdir(),
+				stdio: {
+					stdin: "ignore",
+					stdout: { maxBytes: OCR_STDOUT_BYTES },
+					stderr: { maxBytes: 65536 }
+				},
+				graceMs: 5e3,
+				...signal === void 0 ? {} : { signal }
+			});
+			const outcome = await handle.done;
+			const stdout = handle.collected.stdout?.readFrom(0).text ?? "";
+			const stderr = handle.collected.stderr?.readFrom(0).text ?? "";
+			if (outcome.exitCode !== 0) throw new DshClickError(`tesseract exited ${String(outcome.exitCode)}: ${sanitizeText(stderr, 500)}`, "OCR_FAILED");
+			return parseTesseractTsv(stdout);
+		} finally {
+			await rm(dir, {
+				recursive: true,
+				force: true
+			}).catch(() => {});
+		}
+	}
+};
+/**
+* Probe tesseract through the subprocess service and return a provider.
+* A missing binary degrades to an unavailable provider (fail closed), never a
+* crash, so the zero-dependency desktop path keeps working.
+*
+* @param subprocess - the subprocess service.
+* @param command - tesseract executable name.
+* @param language - tesseract language tag.
+* @returns the probed OCR provider.
+*/
+async function probeTesseract(subprocess, command, language) {
+	try {
+		await subprocess.resolveExecutable(command);
+		return new TesseractOcrProvider(subprocess, command, language);
+	} catch {
+		return new UnavailableOcrProvider("tesseract", `tesseract (${command}) is not installed`);
+	}
+}
+//#endregion
+//#region src/vision/grounding.ts
+/**
+* Visual grounding seam: turning a screenshot plus a natural-language prompt
+* into labeled bounding boxes. Kept as an independent provider so a vision
+* model (or an external grounding service) can plug in later without touching
+* the desktop backend or the tool surface. The default is unavailable — the
+* OCR path (`screen_find`) already yields text targets, and a grounding
+* provider is optional and probed through the `dsh-click/grounding` seam.
+*
+* @module dsh-click/vision/grounding
+*/
+/** A provider that cannot run: `ground` fails closed with the given reason. */
+var UnavailableGroundingProvider = class {
+	id;
+	reason;
+	available = false;
+	/** @param id - provider id. @param reason - model-readable unavailability reason. */
+	constructor(id = "unavailable", reason = "no visual grounding provider configured") {
+		this.id = id;
+		this.reason = reason;
+	}
+	get unavailableReason() {
+		return this.reason;
+	}
+	ground(_pngBase64, _width, _height, _prompt, _signal) {
+		return Promise.reject(new DshClickError(`visual grounding unavailable: ${this.reason}`, "GROUNDING_UNAVAILABLE"));
+	}
+};
+//#endregion
+//#region src/vision/index.ts
+/**
+* Map one screenshot box (possibly downscaled by `maxScreenshotSide`) to its
+* screen-space box, centering on the window's screen rect origin.
+* @param rect - the window's screen rectangle.
+* @param imageWidth - screenshot width in pixels.
+* @param imageHeight - screenshot height in pixels.
+* @param box - the screenshot-space box.
+* @returns the screen-space box.
+*/
+function screenBoxFor(rect, imageWidth, imageHeight, box) {
+	const scaleX = rect.width > 0 && imageWidth > 0 ? rect.width / imageWidth : 1;
+	const scaleY = rect.height > 0 && imageHeight > 0 ? rect.height / imageHeight : 1;
+	return {
+		x: rect.x + box.x * scaleX,
+		y: rect.y + box.y * scaleY,
+		width: box.width * scaleX,
+		height: box.height * scaleY
+	};
+}
+/** Center point of a screen-space box (the coordinate `click` addresses). */
+function centerOf(box) {
+	return {
+		x: Math.round(box.x + box.width / 2),
+		y: Math.round(box.y + box.height / 2)
+	};
+}
+//#endregion
 //#region src/tools.ts
 /** Sanitize one snapshot into the model-visible window value. */
 function observedWindow(snapshot, maxTextLength) {
@@ -1291,6 +1512,177 @@ function screenReadTool(services) {
 		}
 	});
 }
+/**
+* `screen_find` — find clickable text/visual targets in a window whose
+* accessibility tree is empty (no UIA elements). It captures the window,
+* runs OCR (when the probed provider is available) and, when a grounding
+* provider is mounted and a query is given, visual grounding, then returns
+* screen-coordinate targets plus an observationId for the coordinate-based
+* `click` path. Read-only: never needs approval.
+*
+* @param services - runtime services.
+* @returns the tool definition.
+*/
+function screenFindTool(services) {
+	const { config, backend, observations, ocr, grounding } = services;
+	return defineTool({
+		name: "screen_find",
+		description: "Find clickable text (or a described visual target) in a desktop window that exposes no accessibility tree, via OCR and optional visual grounding. Returns screen-coordinate targets plus an observationId; later `click` calls address a target by its (x, y) coordinates and cite this observationId in `basedOn`. Read-only: never needs approval.",
+		parameters: {
+			target: {
+				type: "object",
+				description: "Which window to search (windowId, windowTitle, or processId); omitted = the foreground window.",
+				properties: {
+					windowId: {
+						type: "integer",
+						description: "Native window handle from app_list."
+					},
+					windowTitle: {
+						type: "string",
+						description: "Visible window title (matched case-insensitively by substring)."
+					},
+					processId: {
+						type: "integer",
+						description: "Owning process id."
+					}
+				},
+				additionalProperties: false
+			},
+			query: {
+				type: "string",
+				description: "Optional substring to filter OCR text by, or the visual target description for grounding."
+			},
+			maxSide: {
+				type: "integer",
+				description: "Longest side in pixels; larger captures are downscaled."
+			}
+		},
+		output: {
+			schema: {
+				type: "object",
+				properties: {
+					ok: {
+						type: "boolean",
+						const: true
+					},
+					observationId: { type: "string" },
+					window: {
+						type: "object",
+						properties: {
+							windowId: { type: "integer" },
+							processId: { type: "integer" },
+							title: { type: "string" },
+							className: { type: "string" },
+							executablePath: { oneOf: [{ type: "string" }, { type: "null" }] },
+							rect: {
+								type: "object",
+								properties: {
+									x: { type: "integer" },
+									y: { type: "integer" },
+									width: { type: "integer" },
+									height: { type: "integer" }
+								},
+								additionalProperties: false
+							},
+							foreground: { type: "boolean" }
+						},
+						additionalProperties: false
+					},
+					ocrAvailable: { type: "boolean" },
+					targets: {
+						type: "array",
+						items: {
+							type: "object",
+							properties: {
+								text: { type: "string" },
+								x: { type: "integer" },
+								y: { type: "integer" },
+								width: { type: "integer" },
+								height: { type: "integer" },
+								confidence: { type: "number" },
+								source: {
+									type: "string",
+									enum: ["ocr", "grounding"]
+								}
+							},
+							additionalProperties: false
+						}
+					}
+				},
+				additionalProperties: false
+			},
+			render(_args, value) {
+				const result = value;
+				const lines = [
+					`Text/visual targets in ${windowLine(result.window)}:`,
+					`observationId: ${result.observationId}`,
+					`OCR available: ${result.ocrAvailable}`,
+					`${result.targets.length} target(s):`
+				];
+				for (const target of result.targets) lines.push(`- ${JSON.stringify(target.text)} at (${target.x}, ${target.y}) ${target.width}x${target.height} [${target.source}]`);
+				lines.push("cite this observationId in click `basedOn` and address a target by its (x, y) coordinates.");
+				return [{
+					type: "text",
+					text: lines.join("\n")
+				}];
+			}
+		},
+		timeoutMs: config.helperTimeoutMs + 3e4,
+		async execute(args, exec) {
+			backendGuard(services);
+			const parsed = args;
+			const requestedSide = parsed.maxSide;
+			const maxSide = requestedSide === void 0 ? config.maxScreenshotSide : Math.min(requestedSide, config.maxScreenshotSide);
+			const shot = await backend.shot(parsed.target ?? {}, maxSide, exec.signal);
+			const sanitized = observedWindow(shot.snapshot, config.maxTextLength);
+			const record = observations.record(shot.snapshot);
+			const targets = [];
+			if (ocr.available) for (const word of await ocr.recognize(shot.pngBase64, exec.signal)) {
+				const box = screenBoxFor(shot.snapshot.rect, shot.width, shot.height, word.box);
+				const center = centerOf(box);
+				targets.push({
+					text: sanitizeVisible(word.text, config.maxTextLength),
+					x: center.x,
+					y: center.y,
+					width: Math.round(box.width),
+					height: Math.round(box.height),
+					confidence: word.confidence / 100,
+					source: "ocr"
+				});
+			}
+			const query = parsed.query?.trim() ?? "";
+			if (grounding.available && query !== "") for (const target of await grounding.ground(shot.pngBase64, shot.width, shot.height, query, exec.signal)) {
+				const box = screenBoxFor(shot.snapshot.rect, shot.width, shot.height, target.box);
+				const center = centerOf(box);
+				targets.push({
+					text: sanitizeVisible(target.label, config.maxTextLength),
+					x: center.x,
+					y: center.y,
+					width: Math.round(box.width),
+					height: Math.round(box.height),
+					confidence: target.confidence,
+					source: "grounding"
+				});
+			}
+			const filtered = query === "" ? targets : targets.filter((target) => target.text.toLowerCase().includes(query.toLowerCase()));
+			auditObservation(exec, {
+				observationId: record.id,
+				windowId: shot.snapshot.windowId,
+				processId: shot.snapshot.processId,
+				executablePath: sanitized.executablePath,
+				windowTitle: sanitized.title,
+				elementCount: 0
+			}, config.auditSessionEvents);
+			return {
+				ok: true,
+				observationId: record.id,
+				window: sanitized,
+				ocrAvailable: ocr.available,
+				targets: filtered
+			};
+		}
+	});
+}
 /** The shared window-action output schema (click/scroll/key). */
 function actionOutputSchema(withRestored) {
 	return {
@@ -1773,6 +2165,7 @@ function allTools(services) {
 	return [
 		screenShotTool(services),
 		screenReadTool(services),
+		screenFindTool(services),
 		clickTool(services),
 		typeTool(services),
 		scrollTool(services),
@@ -1787,30 +2180,35 @@ const name = "dsh-click";
 /** Hard services: the tool registry every contribution lands in. */
 const inject = ["tools"];
 /**
-* Mount the plugin: resolve config, select the platform backend, and register
+* Mount the plugin: resolve config, select the platform backend, probe the
+* optional OCR provider, read the optional visual-grounding seam, and register
 * every tool through `ctx.tools.register` (each registration is an effect
 * whose disposer removes exactly that tool on stop/HMR).
 *
 * @param ctx - context carrying the tools registry.
 * @param config - raw loader config; defaults applied through {@link resolveConfig}.
 */
-function apply(ctx, config) {
+async function apply(ctx, config) {
 	const resolved = resolveConfig(config);
 	const backend = ctx.get("dsh-click/backend") ?? createBackend(ctx, resolved);
 	const observations = new ObservationStore(resolved);
+	const actions = new ActionExecutor({
+		ctx,
+		config: resolved,
+		backend,
+		observations
+	});
+	const subprocess = ctx.get("subprocess");
 	const services = {
 		ctx,
 		config: resolved,
 		backend,
 		observations,
-		actions: new ActionExecutor({
-			ctx,
-			config: resolved,
-			backend,
-			observations
-		})
+		actions,
+		ocr: resolved.ocr.enabled && subprocess !== void 0 ? await probeTesseract(subprocess, resolved.ocr.command, resolved.ocr.language) : new UnavailableOcrProvider("ocr-disabled", resolved.ocr.enabled ? "the subprocess service is not mounted" : "OCR is disabled by config"),
+		grounding: ctx.get("dsh-click/grounding") ?? new UnavailableGroundingProvider()
 	};
 	for (const tool of allTools(services)) ctx.effect(() => ctx.tools.register(tool), `dsh-click: ${tool.name} tool`);
 }
 //#endregion
-export { ACTION_EVENT, ActionExecutor, Config, DshClickError, HELPER_PROTOCOL_VERSION, HelperBackend, OBSERVED_EVENT, ObservationStore, UnavailableBackend, VERSION, allTools, apply, createBackend, inject, name, observationIdOf, redactSensitive, resolveConfig, sanitizePath, sanitizeText, sanitizeVisible };
+export { ACTION_EVENT, ActionExecutor, Config, DshClickError, HELPER_PROTOCOL_VERSION, HelperBackend, OBSERVED_EVENT, ObservationStore, TesseractOcrProvider, UnavailableBackend, UnavailableGroundingProvider, UnavailableOcrProvider, VERSION, allTools, apply, centerOf, createBackend, inject, name, observationIdOf, parseTesseractTsv, probeTesseract, redactSensitive, resolveConfig, sanitizePath, sanitizeText, sanitizeVisible, screenBoxFor, tesseractArgv };

package/lib/types/config.d.ts

@@ -45,6 +45,19 @@ export declare const MAX_TREE_DEPTH = 64;
 export declare const DEFAULT_MAX_TEXT_LENGTH = 200;
 /** Ceiling on the sanitization truncation length. */
 export declare const MAX_TEXT_LENGTH = 10000;
+/** Default tesseract executable for the optional OCR path. */
+export declare const DEFAULT_OCR_COMMAND = "tesseract";
+/** Default OCR language (tesseract language data). */
+export declare const DEFAULT_OCR_LANGUAGE = "eng";
+/** Optical character recognition (optional; probed at mount). */
+export interface OcrConfig {
+    /** Enable OCR-based text finding when a window has no UIA tree (default true). */
+    enabled?: boolean;
+    /** tesseract executable name (default "tesseract"). */
+    command?: string;
+    /** tesseract language tag (default "eng"). */
+    language?: string;
+}
 /** Configuration for the dsh-click desktop-control tools. */
 export interface Config {
     /** Gate every mutating action (click/type/scroll/key/app_launch) behind approval (default true). */
@@ -84,6 +97,8 @@ export interface Config {
     maxTextLength?: number;
     /** Back up and restore control text when `type` fails (default true). */
     rollbackEnabled?: boolean;
+    /** Optical character recognition (optional; probed at mount). */
+    ocr?: OcrConfig;
 }
 /** Fully resolved configuration captured at plugin load. */
 export interface ResolvedConfig {
@@ -117,6 +132,12 @@ export interface ResolvedConfig {
     maxTextLength: number;
     /** Whether `type` backs up and restores control text on failure. */
     rollbackEnabled: boolean;
+    /** OCR configuration (probed at mount). */
+    ocr: {
+        enabled: boolean;
+        command: string;
+        language: string;
+    };
 }
 /** Schemastery schema for loader-validated configuration. */
 export declare const Config: z<Config>;

package/lib/types/config.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"config.d.ts","sourceRoot":"","sources":["../../src/config.ts"],"names":[],"mappings":"AAAA;;;;;;;GAOG;AAEH,OAAO,CAAC,MAAM,0BAA0B,CAAA;AAExC,+EAA+E;AAC/E,MAAM,MAAM,SAAS,GAAG,MAAM,GAAG,MAAM,CAAA;AAEvC,mFAAmF;AACnF,MAAM,MAAM,aAAa,GAAG,OAAO,GAAG,OAAO,CAAA;AAE7C,uDAAuD;AACvD,eAAO,MAAM,yBAAyB,QAAS,CAAA;AAE/C,qFAAqF;AACrF,eAAO,MAAM,qBAAqB,SAAU,CAAA;AAE5C,sFAAsF;AACtF,eAAO,MAAM,+BAA+B,QAAmB,CAAA;AAE/D,sCAAsC;AACtC,eAAO,MAAM,uBAAuB,QAAmB,CAAA;AAEvD,2FAA2F;AAC3F,eAAO,MAAM,2BAA2B,OAAO,CAAA;AAE/C,sFAAsF;AACtF,eAAO,MAAM,mBAAmB,MAAM,CAAA;AACtC,eAAO,MAAM,mBAAmB,OAAO,CAAA;AAEvC,8EAA8E;AAC9E,eAAO,MAAM,8BAA8B,QAAS,CAAA;AAEpD,sCAAsC;AACtC,eAAO,MAAM,sBAAsB,OAAQ,CAAA;AAC3C,eAAO,MAAM,sBAAsB,SAAU,CAAA;AAE7C,4EAA4E;AAC5E,eAAO,MAAM,+BAA+B,IAAI,CAAA;AAEhD,sCAAsC;AACtC,eAAO,MAAM,uBAAuB,KAAK,CAAA;AAEzC,8EAA8E;AAC9E,eAAO,MAAM,oBAAoB,MAAM,CAAA;AAEvC,kCAAkC;AAClC,eAAO,MAAM,YAAY,OAAQ,CAAA;AAEjC,4DAA4D;AAC5D,eAAO,MAAM,sBAAsB,KAAK,CAAA;AAExC,iCAAiC;AACjC,eAAO,MAAM,cAAc,KAAK,CAAA;AAEhC,qEAAqE;AACrE,eAAO,MAAM,uBAAuB,MAAM,CAAA;AAE1C,qDAAqD;AACrD,eAAO,MAAM,eAAe,QAAS,CAAA;AAErC,6DAA6D;AAC7D,MAAM,WAAW,MAAM;IACrB,oGAAoG;IACpG,eAAe,CAAC,EAAE,OAAO,CAAA;IACzB,sGAAsG;IACtG,kBAAkB,CAAC,EAAE,MAAM,EAAE,CAAA;IAC7B;;;;;;;OAOG;IACH,kBAAkB,CAAC,EAAE,OAAO,CAAA;IAC5B,8GAA8G;IAC9G,aAAa,CAAC,EAAE,aAAa,CAAA;IAC7B,iGAAiG;IACjG,SAAS,CAAC,EAAE,SAAS,CAAA;IACrB,+DAA+D;IAC/D,eAAe,CAAC,EAAE,MAAM,CAAA;IACxB,8DAA8D;IAC9D,oBAAoB,CAAC,EAAE,MAAM,CAAA;IAC7B,sEAAsE;IACtE,iBAAiB,CAAC,EAAE,MAAM,CAAA;IAC1B,+FAA+F;IAC/F,gBAAgB,CAAC,EAAE,OAAO,CAAA;IAC1B,4FAA4F;IAC5F,mBAAmB,CAAC,EAAE,MAAM,CAAA;IAC5B,8CAA8C;IAC9C,qBAAqB,CAAC,EAAE,MAAM,CAAA;IAC9B,oEAAoE;IACpE,WAAW,CAAC,EAAE,MAAM,CAAA;IACpB,iEAAiE;IACjE,YAAY,CAAC,EAAE,MAAM,CAAA;IACrB,2EAA2E;IAC3E,aAAa,CAAC,EAAE,MAAM,CAAA;IACtB,yEAAyE;IACzE,eAAe,CAAC,EAAE,OAAO,CAAA;CAC1B;AAED,4DAA4D;AAC5D,MAAM,WAAW,cAAc;IAC7B,iDAAiD;IACjD,eAAe,EAAE,OAAO,CAAA;IACxB,qEAAqE;IACrE,mBAAmB,EAAE,aAAa,CAAC,MAAM,CAAC,CAAA;IAC1C,oEAAoE;IACpE,kBAAkB,EAAE,OAAO,CAAA;IAC3B,2CAA2C;IAC3C,aAAa,EAAE,aAAa,CAAA;IAC5B,8BAA8B;IAC9B,SAAS,EAAE,SAAS,CAAA;IACpB,+CAA+C;IAC/C,eAAe,EAAE,MAAM,CAAA;IACvB,2CAA2C;IAC3C,oBAAoB,EAAE,MAAM,CAAA;IAC5B,yCAAyC;IACzC,iBAAiB,EAAE,MAAM,CAAA;IACzB,gDAAgD;IAChD,gBAAgB,EAAE,OAAO,CAAA;IACzB,+CAA+C;IAC/C,mBAAmB,EAAE,MAAM,CAAA;IAC3B,kCAAkC;IAClC,qBAAqB,EAAE,MAAM,CAAA;IAC7B,uCAAuC;IACvC,WAAW,EAAE,MAAM,CAAA;IACnB,+BAA+B;IAC/B,YAAY,EAAE,MAAM,CAAA;IACpB,+CAA+C;IAC/C,aAAa,EAAE,MAAM,CAAA;IACrB,oEAAoE;IACpE,eAAe,EAAE,OAAO,CAAA;CACzB;AAED,6DAA6D;AAC7D,eAAO,MAAM,MAAM,EAAE,CAAC,CAAC,MAAM,CAgB3B,CAAA;AAgBF;;;;;GAKG;AACH,wBAAgB,aAAa,CAAC,MAAM,EAAE,MAAM,GAAG,SAAS,GAAG,cAAc,CA6ExE"}
+{"version":3,"file":"config.d.ts","sourceRoot":"","sources":["../../src/config.ts"],"names":[],"mappings":"AAAA;;;;;;;GAOG;AAEH,OAAO,CAAC,MAAM,0BAA0B,CAAA;AAExC,+EAA+E;AAC/E,MAAM,MAAM,SAAS,GAAG,MAAM,GAAG,MAAM,CAAA;AAEvC,mFAAmF;AACnF,MAAM,MAAM,aAAa,GAAG,OAAO,GAAG,OAAO,CAAA;AAE7C,uDAAuD;AACvD,eAAO,MAAM,yBAAyB,QAAS,CAAA;AAE/C,qFAAqF;AACrF,eAAO,MAAM,qBAAqB,SAAU,CAAA;AAE5C,sFAAsF;AACtF,eAAO,MAAM,+BAA+B,QAAmB,CAAA;AAE/D,sCAAsC;AACtC,eAAO,MAAM,uBAAuB,QAAmB,CAAA;AAEvD,2FAA2F;AAC3F,eAAO,MAAM,2BAA2B,OAAO,CAAA;AAE/C,sFAAsF;AACtF,eAAO,MAAM,mBAAmB,MAAM,CAAA;AACtC,eAAO,MAAM,mBAAmB,OAAO,CAAA;AAEvC,8EAA8E;AAC9E,eAAO,MAAM,8BAA8B,QAAS,CAAA;AAEpD,sCAAsC;AACtC,eAAO,MAAM,sBAAsB,OAAQ,CAAA;AAC3C,eAAO,MAAM,sBAAsB,SAAU,CAAA;AAE7C,4EAA4E;AAC5E,eAAO,MAAM,+BAA+B,IAAI,CAAA;AAEhD,sCAAsC;AACtC,eAAO,MAAM,uBAAuB,KAAK,CAAA;AAEzC,8EAA8E;AAC9E,eAAO,MAAM,oBAAoB,MAAM,CAAA;AAEvC,kCAAkC;AAClC,eAAO,MAAM,YAAY,OAAQ,CAAA;AAEjC,4DAA4D;AAC5D,eAAO,MAAM,sBAAsB,KAAK,CAAA;AAExC,iCAAiC;AACjC,eAAO,MAAM,cAAc,KAAK,CAAA;AAEhC,qEAAqE;AACrE,eAAO,MAAM,uBAAuB,MAAM,CAAA;AAE1C,qDAAqD;AACrD,eAAO,MAAM,eAAe,QAAS,CAAA;AAErC,8DAA8D;AAC9D,eAAO,MAAM,mBAAmB,cAAc,CAAA;AAE9C,sDAAsD;AACtD,eAAO,MAAM,oBAAoB,QAAQ,CAAA;AAEzC,iEAAiE;AACjE,MAAM,WAAW,SAAS;IACxB,kFAAkF;IAClF,OAAO,CAAC,EAAE,OAAO,CAAA;IACjB,uDAAuD;IACvD,OAAO,CAAC,EAAE,MAAM,CAAA;IAChB,8CAA8C;IAC9C,QAAQ,CAAC,EAAE,MAAM,CAAA;CAClB;AAED,6DAA6D;AAC7D,MAAM,WAAW,MAAM;IACrB,oGAAoG;IACpG,eAAe,CAAC,EAAE,OAAO,CAAA;IACzB,sGAAsG;IACtG,kBAAkB,CAAC,EAAE,MAAM,EAAE,CAAA;IAC7B;;;;;;;OAOG;IACH,kBAAkB,CAAC,EAAE,OAAO,CAAA;IAC5B,8GAA8G;IAC9G,aAAa,CAAC,EAAE,aAAa,CAAA;IAC7B,iGAAiG;IACjG,SAAS,CAAC,EAAE,SAAS,CAAA;IACrB,+DAA+D;IAC/D,eAAe,CAAC,EAAE,MAAM,CAAA;IACxB,8DAA8D;IAC9D,oBAAoB,CAAC,EAAE,MAAM,CAAA;IAC7B,sEAAsE;IACtE,iBAAiB,CAAC,EAAE,MAAM,CAAA;IAC1B,+FAA+F;IAC/F,gBAAgB,CAAC,EAAE,OAAO,CAAA;IAC1B,4FAA4F;IAC5F,mBAAmB,CAAC,EAAE,MAAM,CAAA;IAC5B,8CAA8C;IAC9C,qBAAqB,CAAC,EAAE,MAAM,CAAA;IAC9B,oEAAoE;IACpE,WAAW,CAAC,EAAE,MAAM,CAAA;IACpB,iEAAiE;IACjE,YAAY,CAAC,EAAE,MAAM,CAAA;IACrB,2EAA2E;IAC3E,aAAa,CAAC,EAAE,MAAM,CAAA;IACtB,yEAAyE;IACzE,eAAe,CAAC,EAAE,OAAO,CAAA;IACzB,iEAAiE;IACjE,GAAG,CAAC,EAAE,SAAS,CAAA;CAChB;AAED,4DAA4D;AAC5D,MAAM,WAAW,cAAc;IAC7B,iDAAiD;IACjD,eAAe,EAAE,OAAO,CAAA;IACxB,qEAAqE;IACrE,mBAAmB,EAAE,aAAa,CAAC,MAAM,CAAC,CAAA;IAC1C,oEAAoE;IACpE,kBAAkB,EAAE,OAAO,CAAA;IAC3B,2CAA2C;IAC3C,aAAa,EAAE,aAAa,CAAA;IAC5B,8BAA8B;IAC9B,SAAS,EAAE,SAAS,CAAA;IACpB,+CAA+C;IAC/C,eAAe,EAAE,MAAM,CAAA;IACvB,2CAA2C;IAC3C,oBAAoB,EAAE,MAAM,CAAA;IAC5B,yCAAyC;IACzC,iBAAiB,EAAE,MAAM,CAAA;IACzB,gDAAgD;IAChD,gBAAgB,EAAE,OAAO,CAAA;IACzB,+CAA+C;IAC/C,mBAAmB,EAAE,MAAM,CAAA;IAC3B,kCAAkC;IAClC,qBAAqB,EAAE,MAAM,CAAA;IAC7B,uCAAuC;IACvC,WAAW,EAAE,MAAM,CAAA;IACnB,+BAA+B;IAC/B,YAAY,EAAE,MAAM,CAAA;IACpB,+CAA+C;IAC/C,aAAa,EAAE,MAAM,CAAA;IACrB,oEAAoE;IACpE,eAAe,EAAE,OAAO,CAAA;IACxB,2CAA2C;IAC3C,GAAG,EAAE;QAAE,OAAO,EAAE,OAAO,CAAC;QAAC,OAAO,EAAE,MAAM,CAAC;QAAC,QAAQ,EAAE,MAAM,CAAA;KAAE,CAAA;CAC7D;AAED,6DAA6D;AAC7D,eAAO,MAAM,MAAM,EAAE,CAAC,CAAC,MAAM,CAqB3B,CAAA;AAgBF;;;;;GAKG;AACH,wBAAgB,aAAa,CAAC,MAAM,EAAE,MAAM,GAAG,SAAS,GAAG,cAAc,CAqFxE"}

package/lib/types/index.d.ts

@@ -27,13 +27,17 @@ export { HelperBackend } from './platform/runner.js';
 export { allTools, type ToolServices } from './tools.js';
 export { OBSERVED_EVENT, ACTION_EVENT, type ObservedEvent, type ActionEvent, type ProcessFacts } from './events.js';
 export { sanitizeText, sanitizePath, redactSensitive, sanitizeVisible } from './sanitize.js';
+export { TesseractOcrProvider, UnavailableOcrProvider, parseTesseractTsv, probeTesseract, tesseractArgv, type OcrBox, type OcrProvider, type OcrWord, } from './vision/ocr.js';
+export { UnavailableGroundingProvider, type GroundingBox, type GroundingTarget, type VisualGroundingProvider, } from './vision/grounding.js';
+export { centerOf, screenBoxFor } from './vision/index.js';
 /**
- * Mount the plugin: resolve config, select the platform backend, and register
+ * Mount the plugin: resolve config, select the platform backend, probe the
+ * optional OCR provider, read the optional visual-grounding seam, and register
  * every tool through `ctx.tools.register` (each registration is an effect
  * whose disposer removes exactly that tool on stop/HMR).
  *
  * @param ctx - context carrying the tools registry.
  * @param config - raw loader config; defaults applied through {@link resolveConfig}.
  */
-export declare function apply(ctx: Context, config: Config): void;
+export declare function apply(ctx: Context, config: Config): Promise<void>;
 //# sourceMappingURL=index.d.ts.map

package/lib/types/index.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../../src/index.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;GAaG;AAEH,OAAO,KAAK,EAAE,OAAO,EAAE,MAAM,qBAAqB,CAAA;AAIlD,OAAO,EAAE,MAAM,EAAiB,MAAM,aAAa,CAAA;AAMnD,eAAO,MAAM,IAAI,cAAc,CAAA;AAE/B,oEAAoE;AACpE,eAAO,MAAM,MAAM,UAAY,CAAA;AAE/B,OAAO,EAAE,MAAM,EAAE,aAAa,EAAE,MAAM,aAAa,CAAA;AACnD,OAAO,EAAE,OAAO,EAAE,uBAAuB,EAAE,MAAM,cAAc,CAAA;AAC/D,OAAO,EAAE,aAAa,EAAE,MAAM,qBAAqB,CAAA;AACnD,OAAO,EAAE,gBAAgB,EAAE,eAAe,EAAE,KAAK,iBAAiB,EAAE,KAAK,gBAAgB,EAAE,MAAM,cAAc,CAAA;AAC/G,OAAO,EAAE,cAAc,EAAE,KAAK,kBAAkB,EAAE,KAAK,YAAY,EAAE,MAAM,cAAc,CAAA;AACzF,OAAO,EAAE,aAAa,EAAE,kBAAkB,EAAE,MAAM,yBAAyB,CAAA;AAC3E,OAAO,EAAE,aAAa,EAAE,MAAM,sBAAsB,CAAA;AACpD,OAAO,EAAE,QAAQ,EAAE,KAAK,YAAY,EAAE,MAAM,YAAY,CAAA;AACxD,OAAO,EAAE,cAAc,EAAE,YAAY,EAAE,KAAK,aAAa,EAAE,KAAK,WAAW,EAAE,KAAK,YAAY,EAAE,MAAM,aAAa,CAAA;AACnH,OAAO,EAAE,YAAY,EAAE,YAAY,EAAE,eAAe,EAAE,eAAe,EAAE,MAAM,eAAe,CAAA;AAE5F;;;;;;;GAOG;AACH,wBAAgB,KAAK,CAAC,GAAG,EAAE,OAAO,EAAE,MAAM,EAAE,MAAM,GAAG,IAAI,CAYxD"}
+{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../../src/index.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;GAaG;AAEH,OAAO,KAAK,EAAE,OAAO,EAAE,MAAM,qBAAqB,CAAA;AAKlD,OAAO,EAAE,MAAM,EAAiB,MAAM,aAAa,CAAA;AAQnD,eAAO,MAAM,IAAI,cAAc,CAAA;AAE/B,oEAAoE;AACpE,eAAO,MAAM,MAAM,UAAY,CAAA;AAE/B,OAAO,EAAE,MAAM,EAAE,aAAa,EAAE,MAAM,aAAa,CAAA;AACnD,OAAO,EAAE,OAAO,EAAE,uBAAuB,EAAE,MAAM,cAAc,CAAA;AAC/D,OAAO,EAAE,aAAa,EAAE,MAAM,qBAAqB,CAAA;AACnD,OAAO,EAAE,gBAAgB,EAAE,eAAe,EAAE,KAAK,iBAAiB,EAAE,KAAK,gBAAgB,EAAE,MAAM,cAAc,CAAA;AAC/G,OAAO,EAAE,cAAc,EAAE,KAAK,kBAAkB,EAAE,KAAK,YAAY,EAAE,MAAM,cAAc,CAAA;AACzF,OAAO,EAAE,aAAa,EAAE,kBAAkB,EAAE,MAAM,yBAAyB,CAAA;AAC3E,OAAO,EAAE,aAAa,EAAE,MAAM,sBAAsB,CAAA;AACpD,OAAO,EAAE,QAAQ,EAAE,KAAK,YAAY,EAAE,MAAM,YAAY,CAAA;AACxD,OAAO,EAAE,cAAc,EAAE,YAAY,EAAE,KAAK,aAAa,EAAE,KAAK,WAAW,EAAE,KAAK,YAAY,EAAE,MAAM,aAAa,CAAA;AACnH,OAAO,EAAE,YAAY,EAAE,YAAY,EAAE,eAAe,EAAE,eAAe,EAAE,MAAM,eAAe,CAAA;AAC5F,OAAO,EACL,oBAAoB,EACpB,sBAAsB,EACtB,iBAAiB,EACjB,cAAc,EACd,aAAa,EACb,KAAK,MAAM,EACX,KAAK,WAAW,EAChB,KAAK,OAAO,GACb,MAAM,iBAAiB,CAAA;AACxB,OAAO,EACL,4BAA4B,EAC5B,KAAK,YAAY,EACjB,KAAK,eAAe,EACpB,KAAK,uBAAuB,GAC7B,MAAM,uBAAuB,CAAA;AAC9B,OAAO,EAAE,QAAQ,EAAE,YAAY,EAAE,MAAM,mBAAmB,CAAA;AAE1D;;;;;;;;GAQG;AACH,wBAAsB,KAAK,CAAC,GAAG,EAAE,OAAO,EAAE,MAAM,EAAE,MAAM,GAAG,OAAO,CAAC,IAAI,CAAC,CA4BvE"}

package/lib/types/tools.d.ts

@@ -14,6 +14,7 @@ import type { Context } from '@deepseek-ai/cordis';
 import { ActionExecutor } from './actions.js';
 import type { ResolvedConfig } from './config.js';
 import { ObservationStore } from './observe.js';
+import { type OcrProvider, type VisualGroundingProvider } from './vision/index.js';
 import type { DesktopBackend, Rect } from './platform/types.js';
 /** Everything one tool needs at runtime; injected by `src/index.ts`. */
 export interface ToolServices {
@@ -27,6 +28,10 @@ export interface ToolServices {
     readonly observations: ObservationStore;
     /** The action executor. */
     readonly actions: ActionExecutor;
+    /** The probed OCR provider (optional). */
+    readonly ocr: OcrProvider;
+    /** The optional visual grounding provider. */
+    readonly grounding: VisualGroundingProvider;
 }
 /** The sanitized, model-visible window facts shared by observer outputs. */
 export interface ObservedWindowValue {
@@ -57,6 +62,18 @@ export declare function screenShotTool(services: ToolServices): import("@deepsee
  * @returns the tool definition.
  */
 export declare function screenReadTool(services: ToolServices): import("@deepseek-ai/dsh-tools").ToolDefinition;
+/**
+ * `screen_find` — find clickable text/visual targets in a window whose
+ * accessibility tree is empty (no UIA elements). It captures the window,
+ * runs OCR (when the probed provider is available) and, when a grounding
+ * provider is mounted and a query is given, visual grounding, then returns
+ * screen-coordinate targets plus an observationId for the coordinate-based
+ * `click` path. Read-only: never needs approval.
+ *
+ * @param services - runtime services.
+ * @returns the tool definition.
+ */
+export declare function screenFindTool(services: ToolServices): import("@deepseek-ai/dsh-tools").ToolDefinition;
 /**
  * `click` — click an element (by elementId) or a coordinate inside the
  * observed window. Never steals foreground focus; delivery is UIA invoke

package/lib/types/tools.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"tools.d.ts","sourceRoot":"","sources":["../../src/tools.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;GAWG;AAEH,OAAO,KAAK,EAAE,OAAO,EAAE,MAAM,qBAAqB,CAAA;AAKlD,OAAO,EAAE,cAAc,EAAE,MAAM,cAAc,CAAA;AAC7C,OAAO,KAAK,EAAE,cAAc,EAAE,MAAM,aAAa,CAAA;AAEjD,OAAO,EAAE,gBAAgB,EAAE,MAAM,cAAc,CAAA;AAG/C,OAAO,KAAK,EAAE,cAAc,EAA0B,IAAI,EAAyC,MAAM,qBAAqB,CAAA;AAE9H,wEAAwE;AACxE,MAAM,WAAW,YAAY;IAC3B,4BAA4B;IAC5B,QAAQ,CAAC,GAAG,EAAE,OAAO,CAAA;IACrB,8BAA8B;IAC9B,QAAQ,CAAC,MAAM,EAAE,cAAc,CAAA;IAC/B,kCAAkC;IAClC,QAAQ,CAAC,OAAO,EAAE,cAAc,CAAA;IAChC,6BAA6B;IAC7B,QAAQ,CAAC,YAAY,EAAE,gBAAgB,CAAA;IACvC,2BAA2B;IAC3B,QAAQ,CAAC,OAAO,EAAE,cAAc,CAAA;CACjC;AAED,4EAA4E;AAC5E,MAAM,WAAW,mBAAmB;IAClC,QAAQ,EAAE,MAAM,CAAA;IAChB,SAAS,EAAE,MAAM,CAAA;IACjB,KAAK,EAAE,MAAM,CAAA;IACb,SAAS,EAAE,MAAM,CAAA;IACjB,cAAc,EAAE,MAAM,GAAG,IAAI,CAAA;IAC7B,IAAI,EAAE,IAAI,CAAA;IACV,UAAU,EAAE,OAAO,CAAA;CACpB;AAgFD;;;;;;;;GAQG;AACH,wBAAgB,cAAc,CAAC,QAAQ,EAAE,YAAY,mDAoIpD;AAED;;;;;;;GAOG;AACH,wBAAgB,cAAc,CAAC,QAAQ,EAAE,YAAY,mDA+IpD;AAiED;;;;;;;GAOG;AACH,wBAAgB,SAAS,CAAC,QAAQ,EAAE,YAAY,mDAoD/C;AAED;;;;;;;GAOG;AACH,wBAAgB,QAAQ,CAAC,QAAQ,EAAE,YAAY,mDAyC9C;AAED;;;;;;GAMG;AACH,wBAAgB,UAAU,CAAC,QAAQ,EAAE,YAAY,mDA0ChD;AAED;;;;;;;GAOG;AACH,wBAAgB,OAAO,CAAC,QAAQ,EAAE,YAAY,mDAkC7C;AAED;;;;;;GAMG;AACH,wBAAgB,WAAW,CAAC,QAAQ,EAAE,YAAY,mDA+EjD;AAED;;;;;;GAMG;AACH,wBAAgB,aAAa,CAAC,QAAQ,EAAE,YAAY,mDAwCnD;AAED,oDAAoD;AACpD,wBAAgB,QAAQ,CAAC,QAAQ,EAAE,YAAY,qDAW9C"}
+{"version":3,"file":"tools.d.ts","sourceRoot":"","sources":["../../src/tools.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;GAWG;AAEH,OAAO,KAAK,EAAE,OAAO,EAAE,MAAM,qBAAqB,CAAA;AAKlD,OAAO,EAAE,cAAc,EAAE,MAAM,cAAc,CAAA;AAC7C,OAAO,KAAK,EAAE,cAAc,EAAE,MAAM,aAAa,CAAA;AAEjD,OAAO,EAAE,gBAAgB,EAAE,MAAM,cAAc,CAAA;AAG/C,OAAO,EAA0B,KAAK,WAAW,EAAE,KAAK,uBAAuB,EAAE,MAAM,mBAAmB,CAAA;AAC1G,OAAO,KAAK,EAAE,cAAc,EAA0B,IAAI,EAAyC,MAAM,qBAAqB,CAAA;AAE9H,wEAAwE;AACxE,MAAM,WAAW,YAAY;IAC3B,4BAA4B;IAC5B,QAAQ,CAAC,GAAG,EAAE,OAAO,CAAA;IACrB,8BAA8B;IAC9B,QAAQ,CAAC,MAAM,EAAE,cAAc,CAAA;IAC/B,kCAAkC;IAClC,QAAQ,CAAC,OAAO,EAAE,cAAc,CAAA;IAChC,6BAA6B;IAC7B,QAAQ,CAAC,YAAY,EAAE,gBAAgB,CAAA;IACvC,2BAA2B;IAC3B,QAAQ,CAAC,OAAO,EAAE,cAAc,CAAA;IAChC,0CAA0C;IAC1C,QAAQ,CAAC,GAAG,EAAE,WAAW,CAAA;IACzB,8CAA8C;IAC9C,QAAQ,CAAC,SAAS,EAAE,uBAAuB,CAAA;CAC5C;AAED,4EAA4E;AAC5E,MAAM,WAAW,mBAAmB;IAClC,QAAQ,EAAE,MAAM,CAAA;IAChB,SAAS,EAAE,MAAM,CAAA;IACjB,KAAK,EAAE,MAAM,CAAA;IACb,SAAS,EAAE,MAAM,CAAA;IACjB,cAAc,EAAE,MAAM,GAAG,IAAI,CAAA;IAC7B,IAAI,EAAE,IAAI,CAAA;IACV,UAAU,EAAE,OAAO,CAAA;CACpB;AAgFD;;;;;;;;GAQG;AACH,wBAAgB,cAAc,CAAC,QAAQ,EAAE,YAAY,mDAoIpD;AAED;;;;;;;GAOG;AACH,wBAAgB,cAAc,CAAC,QAAQ,EAAE,YAAY,mDA+IpD;AAED;;;;;;;;;;GAUG;AACH,wBAAgB,cAAc,CAAC,QAAQ,EAAE,YAAY,mDAsJpD;AAiED;;;;;;;GAOG;AACH,wBAAgB,SAAS,CAAC,QAAQ,EAAE,YAAY,mDAoD/C;AAED;;;;;;;GAOG;AACH,wBAAgB,QAAQ,CAAC,QAAQ,EAAE,YAAY,mDAyC9C;AAED;;;;;;GAMG;AACH,wBAAgB,UAAU,CAAC,QAAQ,EAAE,YAAY,mDA0ChD;AAED;;;;;;;GAOG;AACH,wBAAgB,OAAO,CAAC,QAAQ,EAAE,YAAY,mDAkC7C;AAED;;;;;;GAMG;AACH,wBAAgB,WAAW,CAAC,QAAQ,EAAE,YAAY,mDA+EjD;AAED;;;;;;GAMG;AACH,wBAAgB,aAAa,CAAC,QAAQ,EAAE,YAAY,mDAwCnD;AAED,oDAAoD;AACpD,wBAAgB,QAAQ,CAAC,QAAQ,EAAE,YAAY,qDAY9C"}

package/lib/types/version.d.ts

@@ -7,7 +7,7 @@
  * @module dsh-click/version
  */
 /** Plugin version advertised in the helper-protocol handshake. */
-export declare const VERSION = "0.1.3";
+export declare const VERSION = "0.2.0";
 /** The helper protocol revision; bump only on incompatible wire changes. */
 export declare const HELPER_PROTOCOL_VERSION = 1;
 //# sourceMappingURL=version.d.ts.map

package/lib/types/vision/grounding.d.ts

@@ -0,0 +1,46 @@
+/**
+ * Visual grounding seam: turning a screenshot plus a natural-language prompt
+ * into labeled bounding boxes. Kept as an independent provider so a vision
+ * model (or an external grounding service) can plug in later without touching
+ * the desktop backend or the tool surface. The default is unavailable — the
+ * OCR path (`screen_find`) already yields text targets, and a grounding
+ * provider is optional and probed through the `dsh-click/grounding` seam.
+ *
+ * @module dsh-click/vision/grounding
+ */
+/** One grounded box in screenshot pixel coordinates. */
+export interface GroundingBox {
+    x: number;
+    y: number;
+    width: number;
+    height: number;
+}
+/** One grounded target: a label plus its box and confidence. */
+export interface GroundingTarget {
+    label: string;
+    box: GroundingBox;
+    /** 0..1 confidence. */
+    confidence: number;
+}
+/** One visual grounding provider. */
+export interface VisualGroundingProvider {
+    /** Stable provider id. */
+    readonly id: string;
+    /** Whether the provider can run. */
+    readonly available: boolean;
+    /** Why the provider is unavailable, when it is. */
+    readonly unavailableReason?: string;
+    /** Ground one screenshot for the given prompt. */
+    ground(pngBase64: string, width: number, height: number, prompt: string, signal?: AbortSignal): Promise<GroundingTarget[]>;
+}
+/** A provider that cannot run: `ground` fails closed with the given reason. */
+export declare class UnavailableGroundingProvider implements VisualGroundingProvider {
+    readonly id: string;
+    readonly reason: string;
+    readonly available = false;
+    /** @param id - provider id. @param reason - model-readable unavailability reason. */
+    constructor(id?: string, reason?: string);
+    get unavailableReason(): string;
+    ground(_pngBase64: string, _width: number, _height: number, _prompt: string, _signal?: AbortSignal): Promise<GroundingTarget[]>;
+}
+//# sourceMappingURL=grounding.d.ts.map

package/lib/types/vision/grounding.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"grounding.d.ts","sourceRoot":"","sources":["../../../src/vision/grounding.ts"],"names":[],"mappings":"AAAA;;;;;;;;;GASG;AAIH,wDAAwD;AACxD,MAAM,WAAW,YAAY;IAC3B,CAAC,EAAE,MAAM,CAAA;IACT,CAAC,EAAE,MAAM,CAAA;IACT,KAAK,EAAE,MAAM,CAAA;IACb,MAAM,EAAE,MAAM,CAAA;CACf;AAED,gEAAgE;AAChE,MAAM,WAAW,eAAe;IAC9B,KAAK,EAAE,MAAM,CAAA;IACb,GAAG,EAAE,YAAY,CAAA;IACjB,uBAAuB;IACvB,UAAU,EAAE,MAAM,CAAA;CACnB;AAED,qCAAqC;AACrC,MAAM,WAAW,uBAAuB;IACtC,0BAA0B;IAC1B,QAAQ,CAAC,EAAE,EAAE,MAAM,CAAA;IACnB,oCAAoC;IACpC,QAAQ,CAAC,SAAS,EAAE,OAAO,CAAA;IAC3B,mDAAmD;IACnD,QAAQ,CAAC,iBAAiB,CAAC,EAAE,MAAM,CAAA;IACnC,kDAAkD;IAClD,MAAM,CAAC,SAAS,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,CAAC,EAAE,WAAW,GAAG,OAAO,CAAC,eAAe,EAAE,CAAC,CAAA;CAC3H;AAED,+EAA+E;AAC/E,qBAAa,4BAA6B,YAAW,uBAAuB;IAI9D,QAAQ,CAAC,EAAE;IAAkB,QAAQ,CAAC,MAAM;IAHxD,QAAQ,CAAC,SAAS,SAAQ;IAE1B,qFAAqF;gBAChE,EAAE,SAAgB,EAAW,MAAM,SAA4C;IAEpG,IAAI,iBAAiB,IAAI,MAAM,CAE9B;IAED,MAAM,CAAC,UAAU,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,EAAE,OAAO,EAAE,MAAM,EAAE,OAAO,EAAE,MAAM,EAAE,OAAO,CAAC,EAAE,WAAW,GAAG,OAAO,CAAC,eAAe,EAAE,CAAC;CAGhI"}

package/lib/types/vision/index.d.ts

@@ -0,0 +1,41 @@
+/**
+ * Vision seam index: re-exports the OCR and visual-grounding providers and the
+ * shared pixel→screen coordinate mapping the `screen_find` tool uses to turn
+ * screenshot boxes into clickable screen coordinates.
+ *
+ * @module dsh-click/vision
+ */
+import type { Rect } from '../platform/types.js';
+export { TesseractOcrProvider, UnavailableOcrProvider, parseTesseractTsv, probeTesseract, tesseractArgv, type OcrBox, type OcrProvider, type OcrWord, } from './ocr.js';
+export { UnavailableGroundingProvider, type GroundingBox, type GroundingTarget, type VisualGroundingProvider, } from './grounding.js';
+/**
+ * Map one screenshot box (possibly downscaled by `maxScreenshotSide`) to its
+ * screen-space box, centering on the window's screen rect origin.
+ * @param rect - the window's screen rectangle.
+ * @param imageWidth - screenshot width in pixels.
+ * @param imageHeight - screenshot height in pixels.
+ * @param box - the screenshot-space box.
+ * @returns the screen-space box.
+ */
+export declare function screenBoxFor(rect: Rect, imageWidth: number, imageHeight: number, box: {
+    x: number;
+    y: number;
+    width: number;
+    height: number;
+}): {
+    x: number;
+    y: number;
+    width: number;
+    height: number;
+};
+/** Center point of a screen-space box (the coordinate `click` addresses). */
+export declare function centerOf(box: {
+    x: number;
+    y: number;
+    width: number;
+    height: number;
+}): {
+    x: number;
+    y: number;
+};
+//# sourceMappingURL=index.d.ts.map

package/lib/types/vision/index.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../../../src/vision/index.ts"],"names":[],"mappings":"AAAA;;;;;;GAMG;AAEH,OAAO,KAAK,EAAE,IAAI,EAAE,MAAM,sBAAsB,CAAA;AAEhD,OAAO,EACL,oBAAoB,EACpB,sBAAsB,EACtB,iBAAiB,EACjB,cAAc,EACd,aAAa,EACb,KAAK,MAAM,EACX,KAAK,WAAW,EAChB,KAAK,OAAO,GACb,MAAM,UAAU,CAAA;AACjB,OAAO,EACL,4BAA4B,EAC5B,KAAK,YAAY,EACjB,KAAK,eAAe,EACpB,KAAK,uBAAuB,GAC7B,MAAM,gBAAgB,CAAA;AAEvB;;;;;;;;GAQG;AACH,wBAAgB,YAAY,CAC1B,IAAI,EAAE,IAAI,EACV,UAAU,EAAE,MAAM,EAClB,WAAW,EAAE,MAAM,EACnB,GAAG,EAAE;IAAE,CAAC,EAAE,MAAM,CAAC;IAAC,CAAC,EAAE,MAAM,CAAC;IAAC,KAAK,EAAE,MAAM,CAAC;IAAC,MAAM,EAAE,MAAM,CAAA;CAAE,GAC3D;IAAE,CAAC,EAAE,MAAM,CAAC;IAAC,CAAC,EAAE,MAAM,CAAC;IAAC,KAAK,EAAE,MAAM,CAAC;IAAC,MAAM,EAAE,MAAM,CAAA;CAAE,CASzD;AAED,6EAA6E;AAC7E,wBAAgB,QAAQ,CAAC,GAAG,EAAE;IAAE,CAAC,EAAE,MAAM,CAAC;IAAC,CAAC,EAAE,MAAM,CAAC;IAAC,KAAK,EAAE,MAAM,CAAC;IAAC,MAAM,EAAE,MAAM,CAAA;CAAE,GAAG;IAAE,CAAC,EAAE,MAAM,CAAC;IAAC,CAAC,EAAE,MAAM,CAAA;CAAE,CAE/G"}

package/lib/types/vision/ocr.d.ts

@@ -0,0 +1,89 @@
+/**
+ * Optical character recognition seam for windows without a UIA tree. OCR is
+ * optional and probed at mount: the `tesseract` provider is used only when the
+ * binary resolves through the subprocess service, otherwise an unavailable
+ * provider fails closed with a model-readable reason. Recognized words carry
+ * their pixel boxes, which `screen_find` maps to screen coordinates for the
+ * coordinate-based `click` path.
+ *
+ * @module dsh-click/vision/ocr
+ */
+import type { SubprocessRuntime } from '@deepseek-ai/dsh-subprocess';
+/** One OCR word box, in screenshot pixel coordinates. */
+export interface OcrBox {
+    x: number;
+    y: number;
+    width: number;
+    height: number;
+}
+/** One recognized word plus its box and confidence. */
+export interface OcrWord {
+    text: string;
+    box: OcrBox;
+    /** 0..100 tesseract confidence, or 0 when unknown. */
+    confidence: number;
+}
+/** One OCR provider (probed at mount). */
+export interface OcrProvider {
+    /** Stable provider id. */
+    readonly id: string;
+    /** Whether the provider can run. */
+    readonly available: boolean;
+    /** Why the provider is unavailable, when it is. */
+    readonly unavailableReason?: string;
+    /** Recognize text in one PNG screenshot. */
+    recognize(pngBase64: string, signal?: AbortSignal): Promise<OcrWord[]>;
+}
+/**
+ * Build the tesseract command line: write words as TSV to stdout.
+ * @param command - tesseract executable.
+ * @param imagePath - absolute PNG path.
+ * @param language - tesseract language tag.
+ * @returns the argv (argv[0] is the executable).
+ */
+export declare function tesseractArgv(command: string, imagePath: string, language: string): readonly string[];
+/**
+ * Parse tesseract TSV output into word boxes. Non-word rows, empty text, and
+ * malformed numeric fields are skipped; a hostile or truncated stdout cannot
+ * produce bogus coordinates.
+ * @param tsv - tesseract TSV text from stdout.
+ * @returns the recognized words.
+ */
+export declare function parseTesseractTsv(tsv: string): OcrWord[];
+/** A provider that cannot run: `recognize` fails closed with the given reason. */
+export declare class UnavailableOcrProvider implements OcrProvider {
+    readonly id: string;
+    readonly reason: string;
+    readonly available = false;
+    /** @param id - provider id. @param reason - model-readable unavailability reason. */
+    constructor(id: string, reason: string);
+    get unavailableReason(): string;
+    recognize(_pngBase64: string, _signal?: AbortSignal): Promise<OcrWord[]>;
+}
+/** The tesseract OCR provider over `ctx.subprocess`. */
+export declare class TesseractOcrProvider implements OcrProvider {
+    private readonly subprocess;
+    private readonly command;
+    private readonly language;
+    readonly id = "tesseract";
+    readonly available = true;
+    /**
+     * @param subprocess - the subprocess service that runs tesseract.
+     * @param command - tesseract executable name.
+     * @param language - tesseract language tag.
+     */
+    constructor(subprocess: SubprocessRuntime, command: string, language: string);
+    recognize(pngBase64: string, signal?: AbortSignal): Promise<OcrWord[]>;
+}
+/**
+ * Probe tesseract through the subprocess service and return a provider.
+ * A missing binary degrades to an unavailable provider (fail closed), never a
+ * crash, so the zero-dependency desktop path keeps working.
+ *
+ * @param subprocess - the subprocess service.
+ * @param command - tesseract executable name.
+ * @param language - tesseract language tag.
+ * @returns the probed OCR provider.
+ */
+export declare function probeTesseract(subprocess: SubprocessRuntime, command: string, language: string): Promise<OcrProvider>;
+//# sourceMappingURL=ocr.d.ts.map

package/lib/types/vision/ocr.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"ocr.d.ts","sourceRoot":"","sources":["../../../src/vision/ocr.ts"],"names":[],"mappings":"AAAA;;;;;;;;;GASG;AAKH,OAAO,KAAK,EAAE,iBAAiB,EAAE,MAAM,6BAA6B,CAAA;AAIpE,yDAAyD;AACzD,MAAM,WAAW,MAAM;IACrB,CAAC,EAAE,MAAM,CAAA;IACT,CAAC,EAAE,MAAM,CAAA;IACT,KAAK,EAAE,MAAM,CAAA;IACb,MAAM,EAAE,MAAM,CAAA;CACf;AAED,uDAAuD;AACvD,MAAM,WAAW,OAAO;IACtB,IAAI,EAAE,MAAM,CAAA;IACZ,GAAG,EAAE,MAAM,CAAA;IACX,sDAAsD;IACtD,UAAU,EAAE,MAAM,CAAA;CACnB;AAED,0CAA0C;AAC1C,MAAM,WAAW,WAAW;IAC1B,0BAA0B;IAC1B,QAAQ,CAAC,EAAE,EAAE,MAAM,CAAA;IACnB,oCAAoC;IACpC,QAAQ,CAAC,SAAS,EAAE,OAAO,CAAA;IAC3B,mDAAmD;IACnD,QAAQ,CAAC,iBAAiB,CAAC,EAAE,MAAM,CAAA;IACnC,4CAA4C;IAC5C,SAAS,CAAC,SAAS,EAAE,MAAM,EAAE,MAAM,CAAC,EAAE,WAAW,GAAG,OAAO,CAAC,OAAO,EAAE,CAAC,CAAA;CACvE;AAKD;;;;;;GAMG;AACH,wBAAgB,aAAa,CAAC,OAAO,EAAE,MAAM,EAAE,SAAS,EAAE,MAAM,EAAE,QAAQ,EAAE,MAAM,GAAG,SAAS,MAAM,EAAE,CAErG;AAED;;;;;;GAMG;AACH,wBAAgB,iBAAiB,CAAC,GAAG,EAAE,MAAM,GAAG,OAAO,EAAE,CAqBxD;AAED,kFAAkF;AAClF,qBAAa,sBAAuB,YAAW,WAAW;IAI5C,QAAQ,CAAC,EAAE,EAAE,MAAM;IAAE,QAAQ,CAAC,MAAM,EAAE,MAAM;IAHxD,QAAQ,CAAC,SAAS,SAAQ;IAE1B,qFAAqF;gBAChE,EAAE,EAAE,MAAM,EAAW,MAAM,EAAE,MAAM;IAExD,IAAI,iBAAiB,IAAI,MAAM,CAE9B;IAED,SAAS,CAAC,UAAU,EAAE,MAAM,EAAE,OAAO,CAAC,EAAE,WAAW,GAAG,OAAO,CAAC,OAAO,EAAE,CAAC;CAGzE;AAED,wDAAwD;AACxD,qBAAa,oBAAqB,YAAW,WAAW;IAUpD,OAAO,CAAC,QAAQ,CAAC,UAAU;IAC3B,OAAO,CAAC,QAAQ,CAAC,OAAO;IACxB,OAAO,CAAC,QAAQ,CAAC,QAAQ;IAX3B,QAAQ,CAAC,EAAE,eAAc;IACzB,QAAQ,CAAC,SAAS,QAAO;IAEzB;;;;OAIG;gBAEgB,UAAU,EAAE,iBAAiB,EAC7B,OAAO,EAAE,MAAM,EACf,QAAQ,EAAE,MAAM;IAG7B,SAAS,CAAC,SAAS,EAAE,MAAM,EAAE,MAAM,CAAC,EAAE,WAAW,GAAG,OAAO,CAAC,OAAO,EAAE,CAAC;CA2B7E;AAED;;;;;;;;;GASG;AACH,wBAAsB,cAAc,CAAC,UAAU,EAAE,iBAAiB,EAAE,OAAO,EAAE,MAAM,EAAE,QAAQ,EAAE,MAAM,GAAG,OAAO,CAAC,WAAW,CAAC,CAO3H"}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "dsh-click",
-  "version": "0.1.3",
+  "version": "0.2.0",
   "description": "Cross-platform native desktop control tools for DeepSeek Harness (Windows first): screen_shot, screen_read (accessibility tree + pixel description for text-only models), click/type/scroll/key, and app_list/app_launch — every mutating action gated by approval, never stealing foreground focus, with stale-observation rejection and process-identity checks",
   "repository": {
     "type": "git",