@arach/lattices 0.1.0 → 0.2.0

package/docs/ocr.md

@@ -0,0 +1,185 @@
+---
+title: Screen OCR & Search
+description: Vision-powered screen reading with full-text search for agents
+order: 3.5
+---
+
+The menu bar app reads text from visible windows using Apple's Vision
+framework and stores results in a local SQLite database with FTS5
+full-text search. Agents can use this to "see" what's on screen.
+
+## Enabling OCR
+
+Open **Settings** (via command palette or gear icon) and toggle
+**Search & OCR** on. OCR is disabled by default.
+
+### Accuracy modes
+
+| Mode | Description |
+|------|-------------|
+| **Accurate** (default) | Higher quality recognition, slower processing |
+| **Fast** | Lower latency, reduced accuracy |
+
+Both modes use `VNRecognizeTextRequest` with language correction enabled.
+
+## How scanning works
+
+The app runs two scan schedules:
+
+| Schedule | Interval | Window limit | Purpose |
+|----------|----------|-------------|---------|
+| **Quick scan** | 60 seconds | Top 5 windows | Keep recent content fresh |
+| **Deep scan** | 2 hours | Up to 15 windows | Catch less-active windows |
+
+Both intervals and limits are configurable in Settings.
+
+### Change detection
+
+Before running OCR on a window, the app captures the window image and
+computes a SHA-256 hash of the pixel data. If the hash matches the
+previous scan, the cached result is reused. No Vision processing needed.
+
+This keeps CPU usage low when windows haven't changed. A 100ms throttle
+between windows further limits processing bursts.
+
+## Browsing results
+
+The **Recent Captures** section in Settings shows OCR results grouped by
+app. Each entry displays the window title, recognized text preview, and
+timestamp.
+
+## Searching
+
+### From the command palette
+
+The OmniSearch bar (Cmd+Shift+M) searches OCR content alongside windows,
+projects, and sessions. Matches show as "Screen Text" results with
+contextual snippets.
+
+### From the CLI
+
+```bash
+# View current OCR snapshot
+lattices ocr
+
+# Search OCR history
+lattices ocr search "error OR failed"
+
+# Trigger an immediate deep scan
+lattices ocr scan
+
+# View OCR history for a specific window ID
+lattices ocr history 12345
+```
+
+### From the daemon API
+
+Agents can query OCR data through four daemon methods:
+
+| Method | Description |
+|--------|-------------|
+| `ocr.snapshot` | Current OCR results for all visible windows |
+| `ocr.search` | Full-text search across history (FTS5 syntax) |
+| `ocr.history` | Timeline of OCR results for a specific window |
+| `ocr.scan` | Trigger an immediate deep scan |
+
+#### `ocr.snapshot`
+
+Returns the latest OCR results for all on-screen windows.
+
+```js
+import { daemonCall } from '@arach/lattices/daemon-client'
+
+const snapshot = await daemonCall('ocr.snapshot')
+// [{ wid, app, title, frame, fullText, blocks, timestamp }, ...]
+```
+
+Each result includes:
+- `wid` — window ID
+- `app` — application name
+- `title` — window title
+- `frame` — `{ x, y, w, h }` screen position
+- `fullText` — all recognized text concatenated
+- `blocks` — individual text blocks with `{ text, confidence, x, y, w, h }`
+- `timestamp` — Unix timestamp of the scan
+
+#### `ocr.search`
+
+Full-text search across OCR history using FTS5 query syntax.
+
+```js
+const results = await daemonCall('ocr.search', {
+  query: 'error OR failed',  // FTS5 query (required)
+  app: 'Terminal',            // filter by app name (optional)
+  limit: 50,                  // max results (optional, default 50)
+  live: false                 // search in-memory snapshot instead of history (optional)
+})
+// [{ id, wid, app, title, frame, fullText, snippet, timestamp }, ...]
+```
+
+The `snippet` field contains FTS5-highlighted text with `«` and `»`
+delimiters around matched terms.
+
+#### `ocr.history`
+
+Get the OCR content timeline for a specific window.
+
+```js
+const history = await daemonCall('ocr.history', {
+  wid: 12345,  // window ID (required)
+  limit: 50    // max results (optional, default 50)
+})
+```
+
+#### `ocr.scan`
+
+Trigger an immediate deep scan (all visible windows up to the deep limit).
+
+```js
+await daemonCall('ocr.scan')
+// { ok: true }
+```
+
+## Storage
+
+OCR data is stored in `~/.lattices/ocr.db`, a SQLite database in WAL
+(Write-Ahead Logging) mode for safe concurrent reads.
+
+The schema uses two tables:
+- `ocr_entry` — stores window ID, app, title, frame, full text, and timestamp
+- `ocr_fts` — FTS5 virtual table indexing `full_text`, `app`, and `title`
+
+Triggers keep the FTS index in sync with inserts, updates, and deletes.
+
+Entries older than 3 days are automatically deleted.
+
+## Agent usage
+
+A typical agent workflow: trigger a scan, then search for relevant content.
+
+```js
+import { daemonCall } from '@arach/lattices/daemon-client'
+
+// Trigger a fresh scan
+await daemonCall('ocr.scan')
+
+// Search for compilation errors across all windows
+const errors = await daemonCall('ocr.search', { query: 'error OR warning' })
+
+for (const result of errors) {
+  console.log(`[${result.app}] ${result.title}`)
+  console.log(result.snippet)
+}
+
+// Read everything currently visible
+const snapshot = await daemonCall('ocr.snapshot')
+for (const win of snapshot) {
+  console.log(`${win.app}: ${win.fullText.slice(0, 200)}`)
+}
+```
+
+## Requirements
+
+- **Screen Recording** permission — required to capture window images
+- Grant via System Settings > Privacy & Security > Screen Recording
+- Add the lattices menu bar app to the allowed list

package/docs/overview.md

@@ -4,71 +4,76 @@ description: What lattices is and who it's for
 order: 0
 ---
 
-# Overview
-
-lattices is a macOS developer workspace manager. It pairs tmux sessions
-with a native menu bar app to give you — and your AI coding agents —
-full control over terminal layouts, window tiling, and project navigation.
+lattices is a macOS developer workspace manager. A native menu bar app
+handles window tiling, project navigation, on-screen OCR, and optionally
+tmux-powered terminal sessions. Your AI coding agents can use it too.
 
 ## The problem
 
-Modern development means juggling multiple terminal windows: a coding
-agent in one, a dev server in another, tests in a third. Setting this up
-every morning is tedious. AI agents can't do it at all — they're trapped
-inside a single shell with no way to manage windows or switch contexts.
+I kept losing track of windows. Terminal here, dev server there, browser
+somewhere on the second monitor, three other projects buried under it all.
+Every morning I'd rebuild the same layout. Half my context-switching was
+just hunting for the right window.
+
+AI agents have it worse. They're stuck inside a single shell with no way
+to see what's on screen, arrange windows, or jump between projects.
 
 ## The solution
 
-lattices solves both sides:
+lattices fixes both sides:
 
-- **For you** — run `lattices` in any project to get a pre-configured
-  tmux session. Use the menu bar app to launch, tile, and navigate
-  sessions with a command palette.
-- **For agents** — the daemon API exposes 20 RPC methods over WebSocket.
-  Agents can discover projects, launch sessions, tile windows, and
-  switch workspace layers programmatically.
+- The menu bar app tiles windows, navigates projects, and manages
+  workspace layers through a command palette. Add tmux if you want
+  persistent terminal sessions with auto-layout.
+- The daemon API exposes 30 RPC methods over WebSocket. Agents can
+  discover projects, tile windows, switch layers, read on-screen text
+  via OCR, and launch sessions programmatically.
 
 ## What's included
 
 | Component | Description |
 |-----------|-------------|
-| **CLI** (`lattices`) | Create, manage, and tile tmux sessions from the terminal |
-| **Menu bar app** | Native macOS companion with command palette, tiling, and project discovery |
-| **Daemon API** | WebSocket server on `ws://127.0.0.1:9399` — 20 methods, 3 real-time events |
+| **CLI** | The `lattices` command. Tile windows, manage sessions, control the workspace from your terminal |
+| **Menu bar app** | Native macOS companion. Command palette, window tiling, project discovery |
+| **Daemon API** | WebSocket server on `ws://127.0.0.1:9399`. 30 methods, 5 real-time events |
+| **OCR engine** | Reads text from visible windows using Apple Vision, indexes it with FTS5 |
 | **Node.js client** | Zero-dependency `daemonCall()` helper for scripting |
 
-## Quick taste
+## Example
 
 ```bash
-# Launch a workspace (auto-detects your project)
 cd ~/my-project && lattices
-
-# Or give agents programmatic control
 ```
 
-```js
-import { daemonCall } from 'lattices/daemon-client'
+Agents get the same control programmatically:
 
+```js
+import { daemonCall } from '@arach/lattices/daemon-client'
 await daemonCall('session.launch', { path: '/Users/you/dev/frontend' })
 await daemonCall('window.tile', { session: 'frontend-a1b2c3', position: 'left' })
 ```
 
 ## Who it's for
 
-- **Developers** who use tmux and want faster project switching
-- **AI agent builders** who need their agents to control the workspace
-- **Power users** who manage multiple projects across macOS Spaces
+- Developers who juggle multiple projects and want faster window management
+- People building AI agents that need to control the desktop
+- Power users who work across multiple macOS Spaces
+- tmux users who want persistent, auto-configured terminal sessions
 
 ## Requirements
 
 - macOS 13.0+
-- tmux (`brew install tmux`)
 - Node.js 18+
-- Swift 5.9+ (only needed to build the menu bar app from source)
+
+### Dev dependencies
+
+- Swift 5.9+ — only to build the menu bar app from source
+- tmux — needed for persistent terminal sessions (`brew install tmux`)
 
 ## Next steps
 
-- [Quickstart](/docs/quickstart) — install and run your first session in 2 minutes
-- [Concepts](/docs/concepts) — architecture, glossary, and how it all works
-- [Configuration](/docs/config) — `.lattices.json` format and CLI commands
-- [Daemon API](/docs/api) — full RPC method reference for agents and scripts
+- [Quickstart](/docs/quickstart): install and run your first session in 2 minutes
+- [Configuration](/docs/config): `.lattices.json` format and CLI commands
+- [Screen OCR](/docs/ocr): Vision-powered screen reading and full-text search
+- [Concepts](/docs/concepts): architecture, glossary, and how it all works
+- [Daemon API](/docs/api): RPC method reference for agents and scripts

package/docs/quickstart.md

@@ -4,56 +4,46 @@ description: Install lattices and launch your first workspace in 2 minutes
 order: 0.5
 ---
 
-# Quickstart
+Four steps to a running workspace.
 
-Get from zero to a running workspace in five steps.
-
-## 1. Install tmux
+## 1. Install lattices
 
 ```bash
-brew install tmux
+npm install -g @arach/lattices
 ```
 
-Skip if you already have it (`tmux -V` to check).
-
-## 2. Install lattices
+Or install from source:
 
 ```bash
-# Clone and link
 git clone https://github.com/arach/lattices
-cd lattices
-bun link
+cd lattices && npm link
 ```
 
 Verify: `lattices help` should print usage info.
 
-## 3. Launch a workspace
+## 2. Launch the menu bar app
 
 ```bash
-cd ~/your-project
-lattices
+lattices app
 ```
 
-This creates a tmux session with two panes side by side:
-- Left pane (60%): `claude` (AI coding agent)
-- Right pane (40%): your dev command (auto-detected from `package.json`)
-
-No config file needed — lattices auto-detects your package manager
-and dev script.
+This builds (or downloads) and launches the native macOS companion.
+Open the command palette with **Cmd+Shift+M** to search and launch
+any project, tile windows, or switch workspace layers.
 
-## 4. Customize with .lattices.json
+## 3. Add a project config
 
-For more control, add a config to your project:
+Drop a `.lattices.json` in your project root:
 
 ```bash
+cd ~/your-project
 lattices init
 ```
 
-This generates a `.lattices.json` like:
+This generates a config like:
 
 ```json
 {
-  "ensure": true,
   "panes": [
     { "name": "claude", "cmd": "claude", "size": 60 },
     { "name": "server", "cmd": "bun dev" }
@@ -61,22 +51,31 @@ This generates a `.lattices.json` like:
 }
 ```
 
-Edit it to match your workflow, then run `lattices` again to apply.
+The menu bar app discovers projects with `.lattices.json` files
+automatically — they show up in the command palette.
+
+## 4. (Optional) Add tmux for persistent sessions
 
-## 5. Launch the menu bar app
+If you want terminal sessions that survive disconnects and auto-restore
+your pane layout:
 
 ```bash
-lattices app
+brew install tmux
+cd ~/your-project && lattices
 ```
 
-This builds (or downloads) and launches the native macOS companion.
-Open the command palette with **Cmd+Shift+M** to search and launch
-any project, tile windows, or switch workspace layers.
+This creates a tmux session with your configured panes side by side.
+The session persists in the background — close your terminal, reopen it,
+run `lattices` again, and everything is still there.
+
+> **Without tmux**, you still get the menu bar app, command palette,
+> window tiling, workspace layers, OCR, and the full daemon API.
 
 ## What's next
 
-- [Concepts](/docs/concepts) — understand sessions, panes, and the architecture
-- [Configuration](/docs/config) — full `.lattices.json` reference and CLI commands
-- [Menu Bar App](/docs/app) — command palette, tiling, and settings
-- [Daemon API](/docs/api) — programmatic control for agents and scripts
-- [Layers & Groups](/docs/layers) — organize projects into switchable contexts
+- [Configuration](/docs/config): `.lattices.json` reference and CLI commands
+- [Menu Bar App](/docs/app): command palette, tiling, and settings
+- [Layers & Groups](/docs/layers): organize projects into switchable contexts
+- [Screen OCR](/docs/ocr): let agents read what's on screen
+- [Concepts](/docs/concepts): sessions, panes, and the architecture
+- [Daemon API](/docs/api): programmatic control for agents and scripts

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@arach/lattices",
-  "version": "0.1.0",
-  "description": "Tmux sessions with Claude Code + dev server, side by side",
+  "version": "0.2.0",
+  "description": "macOS workspace manager with window tiling, OCR, and a 30-method daemon API for AI agents",
   "bin": {
     "lattices": "./bin/lattices.js",
     "lattices-app": "./bin/lattices-app.js"
@@ -20,6 +20,10 @@
     "url": "https://github.com/arach/lattices"
   },
   "license": "MIT",
+  "exports": {
+    ".": "./bin/lattices.js",
+    "./daemon-client": "./bin/daemon-client.js"
+  },
   "scripts": {
     "dev": "bun --cwd docs-site dev"
   },