@aspiresys/visor 1.0.0

package/readme.md

@@ -0,0 +1,566 @@
+# Visor
+
+Desktop Visual Automation Framework for Node.js and TypeScript.
+
+Visor is a visual desktop automation framework that combines:
+
+- OpenCV image matching
+- OCR text recognition
+- Mouse & keyboard automation
+- Desktop application automation
+
+Visor is designed for automating desktop workflows using visual interactions instead of traditional DOM/browser automation.
+
+---
+
+# Features
+
+- OpenCV-based image matching
+- OCR automation using Tesseract
+- Mouse automation
+- Keyboard automation
+- Drag & drop support
+- Multi-theme image handling
+- Screenshot capture
+- Desktop application automation
+- OCR text searching
+- Wait APIs
+- Multi-image matching
+- Config-driven initialization
+- High-DPI display scaling support
+
+---
+
+# Installation
+
+## Install from npm
+
+```bash
+npm install visor
+```
+
+---
+
+## Local Development
+
+```bash
+npm install
+npm run build
+```
+
+---
+
+# Requirements
+
+Visor currently supports:
+
+- Windows
+- Node.js 18+
+- TypeScript
+
+---
+
+# Quick Start
+
+```ts
+import { visor } from "visor";
+
+async function main() {
+
+  visor.loadConfig({
+    scaleFactor: 1.5,
+    imagePath: "./images",
+    debug: true
+  });
+
+  await visor.openApp("notepad");
+
+  await visor.wait("notepad.png");
+
+  await visor.click("notepad.png");
+
+  await visor.type(
+    "Hello from Visor"
+  );
+
+}
+
+main();
+```
+
+---
+
+# Configuration
+
+Visor supports centralized framework configuration.
+
+```ts
+visor.loadConfig({
+  scaleFactor: 1.5,
+  imagePath: "./images",
+  debug: true
+});
+```
+
+---
+
+## Configuration Options
+
+| Option | Description |
+|---|---|
+| `scaleFactor` | Display scaling factor |
+| `imagePath` | Default image directory |
+| `debug` | Enables debug logging |
+
+---
+
+# Display Scaling
+
+Display scaling is extremely important for desktop automation.
+
+Common values:
+
+| Scaling | Value |
+|---|---|
+| 100% | `1.0` |
+| 125% | `1.25` |
+| 150% | `1.5` |
+| 200% | `2.0` |
+
+Example:
+
+```ts
+visor.setScaleFactor(1.5);
+```
+
+Incorrect scaling values may cause:
+
+- failed image matching
+- incorrect mouse positioning
+- OCR region issues
+
+---
+
+# Visual Automation APIs
+
+## Click Image
+
+```ts
+await visor.click("save.png");
+```
+
+---
+
+## Find Image
+
+```ts
+const region =
+  await visor.find("icon.png");
+```
+
+---
+
+## Check Image Exists
+
+```ts
+const exists =
+  await visor.exists("login.png");
+```
+
+---
+
+## Wait For Image
+
+```ts
+await visor.wait("loading-complete.png");
+```
+
+---
+
+## Wait For Multiple Images
+
+```ts
+await visor.waitAny([
+  "home-light.png",
+  "home-dark.png"
+]);
+```
+
+---
+
+## Click Multiple Theme Variants
+
+```ts
+await visor.clickAny([
+  "send-light.png",
+  "send-dark.png"
+]);
+```
+
+---
+
+## Drag & Drop
+
+```ts
+await visor.dragDrop(
+  "file.png",
+  "folder.png"
+);
+```
+
+---
+
+## Hover
+
+```ts
+await visor.hover("menu.png");
+```
+
+---
+
+# OCR Automation
+
+Visor includes OCR automation using Tesseract.js.
+
+OCR supports:
+
+- full screen text detection
+- region-based OCR
+- text matching
+- OCR click automation
+
+---
+
+## Read Screen
+
+```ts
+const result =
+  await visor.readScreen();
+
+console.log(result.text);
+```
+
+---
+
+## Find Text
+
+```ts
+const region =
+  await visor.findText("Submit");
+```
+
+---
+
+## Click Text
+
+```ts
+await visor.clickText("Login");
+```
+
+---
+
+## Wait For Text
+
+```ts
+await visor.waitText("Success");
+```
+
+---
+
+# OCR Optimizations
+
+Visor includes:
+
+- shared OCR worker reuse
+- OCR preprocessing
+- confidence filtering
+- grayscale normalization
+- sharpened OCR processing
+
+These optimizations improve:
+
+- OCR speed
+- OCR accuracy
+- framework stability
+
+---
+
+# Mouse Automation
+
+## Move Mouse
+
+```ts
+await visor.moveMouse(
+  500,
+  300
+);
+```
+
+---
+
+## Scroll Down
+
+```ts
+await visor.scrollDown(1000);
+```
+
+---
+
+## Scroll Up
+
+```ts
+await visor.scrollUp(1000);
+```
+
+---
+
+## Mouse Position
+
+```ts
+const pos =
+  await visor.getMousePosition();
+```
+
+---
+
+# Keyboard Automation
+
+## Type Text
+
+```ts
+await visor.type(
+  "Hello World"
+);
+```
+
+---
+
+## Press Keys
+
+```ts
+await visor.press(
+  visor.Key.LeftControl,
+  visor.Key.S
+);
+```
+
+---
+
+# Screenshot Automation
+
+## Capture Screenshot
+
+```ts
+await visor.captureScreenshot(
+  "./screenshots/home.png"
+);
+```
+
+---
+
+# Desktop Application Automation
+
+## Open Application
+
+```ts
+await visor.openApp("notepad");
+```
+
+---
+
+## Close Application
+
+```ts
+await visor.closeApp("notepad.exe");
+```
+
+---
+
+# Teams Automation Example
+
+```ts
+import { visor } from "visor";
+
+async function teamsDemo() {
+
+  visor.loadConfig({
+    scaleFactor: 1.5,
+    imagePath: "./images",
+    debug: true
+  });
+
+  await visor.openApp(
+    "ms-teams.exe"
+  );
+
+  await visor.waitAny([
+    "teams-light.png",
+    "teams-dark.png"
+  ]);
+
+  await visor.clickAny([
+    "chat-light.png",
+    "chat-dark.png"
+  ]);
+
+  await visor.clickText("Search");
+
+  await visor.type("John");
+
+}
+```
+
+---
+
+# Confidence Thresholds
+
+Most image matching APIs support confidence values.
+
+Accepted range:
+
+```txt
+0.0 to 1.0
+```
+
+Recommended values:
+
+| Confidence | Usage |
+|---|---|
+| `0.7` | Dynamic UI |
+| `0.8` | Standard usage |
+| `0.9+` | Strict matching |
+
+Lower confidence increases flexibility but may increase false positives.
+
+---
+
+# Performance Notes
+
+## OCR
+
+OCR operations are computationally expensive.
+
+OCR speed depends on:
+
+- display resolution
+- text density
+- screen complexity
+- hardware performance
+
+---
+
+## Electron Applications
+
+Electron-based applications may require additional startup stabilization time.
+
+---
+
+# Debug Logging
+
+Enable debug logs:
+
+```ts
+visor.setDebug(true);
+```
+
+---
+
+# Best Practices
+
+## Use Stable Images
+
+Prefer:
+
+- high contrast images
+- unique UI elements
+- properly cropped templates
+
+Avoid:
+
+- blurry screenshots
+- partially hidden elements
+- scaled screenshots
+
+---
+
+## Use Proper Scaling
+
+Always configure:
+
+```ts
+visor.setScaleFactor(...);
+```
+
+before automation begins.
+
+---
+
+## Prefer Wait APIs
+
+Prefer:
+
+```ts
+await visor.wait(...);
+```
+
+instead of excessive hard sleeps.
+
+---
+
+# Troubleshooting
+
+## Image Not Found
+
+Possible causes:
+
+- incorrect scaling
+- wrong image path
+- low confidence threshold
+- theme mismatch
+
+---
+
+## OCR Not Detecting Text
+
+Possible causes:
+
+- blurry UI
+- small font size
+- low contrast text
+
+---
+
+## Mouse Clicking Incorrect Position
+
+Possible causes:
+
+- incorrect scale factor
+- Windows DPI scaling mismatch
+
+---
+
+# Roadmap
+
+Planned features:
+
+- Linux support
+- Mac support
+- Region caching
+- Parallel image matching
+- Advanced OCR tuning
+- Electron recorder
+- AI-assisted automation
+
+---
+
+# Tech Stack
+
+Visor uses:
+
+- OpenCV
+- Tesseract.js
+- screenshot-desktop
+- sharp
+- nut.js
+
+---