@lattices/cli 0.3.0 → 0.4.1

package/docs/twins.md

@@ -0,0 +1,138 @@
+---
+title: Project Twins
+description: Pi-backed project twins for mediated, persistent agent execution
+order: 3
+---
+
+A project twin is a persistent software counterpart to a codebase.
+
+It is not the primary agent. It is the project-native runtime that sits
+between a general-purpose caller and the project's execution protocol.
+
+## Why a twin exists
+
+General-purpose agents are interchangeable. Project protocols are not.
+
+If every primary agent has to learn the project's tool surface, memory
+policy, protocol semantics, and context conventions from scratch, the
+integration becomes brittle. A twin fixes that by becoming the stable
+project-facing runtime:
+
+- The **primary agent** asks for work
+- The **twin** resumes with the right context and memory
+- The **protocol** stays behind the twin boundary
+
+```text
+primary agent -> project twin -> project protocol / harness
+```
+
+The twin is the client of record for the project.
+
+## Responsibilities
+
+A project twin owns:
+
+- Project-scoped identity
+- Persistent session continuity
+- Memory compaction and continuation
+- Tool policy and allowed capabilities
+- Protocol knowledge
+- Project context assembly
+- Caller-facing summaries and handoffs
+
+A primary agent should not speak the project protocol directly. It should
+invoke the twin.
+
+## Pi-backed runtime
+
+Pi is a good fit for the twin runtime because it already provides:
+
+- Persistent sessions
+- RPC mode for long-running subprocess integration
+- Tool calling with an explicit harness
+- Compaction and summarization hooks
+- Context files, prompt templates, and extension loading
+
+That makes the split:
+
+- **Twin**: product concept and policy boundary
+- **Pi**: reasoning and session runtime
+- **Host system**: orchestration, durable memory, and protocol adapters
+
+Pi powers the twin. It does not define the twin.
+
+## Invocation model
+
+The primary agent makes a single mediated call into the twin:
+
+1. Resume the twin session
+2. Inject caller context, project memory, and protocol state
+3. Let the twin do project-local work inside the harness
+4. Return a concise result to the caller
+
+The caller should see a stable capability surface such as:
+
+- `status`
+- `inspect`
+- `plan`
+- `execute`
+- `summarize`
+- `handoff`
+
+It should not see raw protocol-shaped operations unless that protocol is
+itself the public product surface.
+
+## Implementation in this repo
+
+This repo now includes a Pi-backed runtime in
+[`bin/project-twin.ts`](/Users/arach/dev/lattices/bin/project-twin.ts).
+
+The runtime:
+
+- Spawns `pi --mode rpc` as a persistent subprocess
+- Stores project-local session state under `.openscout/twins/<name>/sessions`
+- Exposes a stable `invoke()` API for callers
+- Optionally injects OpenScout relay context if `.openscout/relay*` exists
+
+The default harness is intentionally narrow:
+
+- Built-in Pi tools are explicitly pinned to `read,bash,edit,write`
+- Extension, skill, and prompt-template discovery are disabled by default
+- Project instructions still come from `AGENTS.md` and related context files
+
+This keeps the twin deterministic unless the host explicitly widens the
+surface.
+
+## Example
+
+```ts
+import { ProjectTwin } from "@lattices/cli"
+
+const twin = new ProjectTwin({
+  cwd: "/Users/you/dev/my-project",
+  name: "my-project",
+  model: "anthropic/claude-sonnet-4-5",
+})
+
+await twin.start()
+
+const result = await twin.invoke({
+  caller: "primary-agent",
+  protocol: "openscout-relay",
+  memory: "The caller is debugging relay enrollment and wants the next safe action.",
+  task: "Inspect the available project context and summarize what the caller should do next.",
+})
+
+console.log(result.text)
+
+await twin.stop()
+```
+
+## Design rule
+
+All project-specific protocol semantics should live behind the twin
+boundary.
+
+The primary agent should invoke the twin as a skill-like capability.
+The twin should own context assembly, protocol interaction, and the final
+handoff back to the caller.

package/docs/voice-command-protocol.md

@@ -0,0 +1,278 @@
+# Voice Command Protocol — Lattices ↔ Vox
+
+## Overview
+
+Lattices delegates all audio capture and transcription to Vox via WebSocket JSON-RPC. Lattices never accesses the microphone directly — it borrows Vox's mic and transcription pipeline, receives English text back, and routes it through its own intent engine.
+
+These dictations are **ephemeral** — Vox does not persist them as memos, sync them, or add them to Vox's history. Lattices is just using Vox as a transcription pipe.
+
+## Vox Process Model
+
+Vox consists of three independent processes:
+
+| Process | Role | Relevance to Lattices |
+|---|---|---|
+| **Vox.app** | Main UI — menu bar, notch visualization, memo history | None |
+| **Vox** | Background service — mic access, recording, hotkeys, orchestrates transcription, state notifications | **This is what Lattices connects to** |
+| **VoxEngine** | Transcription engine — runs Whisper models, called by Vox internally | Indirect — Vox delegates to it |
+
+Vox is the right target because:
+- It owns the mic and recording lifecycle
+- It's the long-running background process (always up when Vox is installed)
+- It already orchestrates the record → transcribe → result pipeline
+- It's easy to discover via its existing DistributedNotification
+
+## Service Discovery
+
+Lattices never hardcodes ports. Discovery uses two mechanisms:
+
+### 1. Well-known file (at rest)
+
+Vox writes its service configuration on startup:
+
+```
+~/.vox/services.json
+```
+
+```json
+{
+  "agent": {"port": 19823, "pid": 48209},
+  "engine": {"port": 19821, "pid": 48210},
+  "sync": {"port": 19820, "pid": 48208},
+  "inference": {"port": 19822, "pid": 48212}
+}
+```
+
+Lattices reads `agent.port` from this file. If the file doesn't exist, Vox isn't installed.
+
+### 2. DistributedNotification (live discovery)
+
+Vox posts when it comes online:
+
+```
+Notification: com.jdi.vox.agent.live.ready
+UserInfo:     {"agentPort": 19823, "pid": 48209}
+```
+
+Lattices subscribes to this on startup. Handles:
+- **Vox launches after Lattices** — Lattices picks up the port dynamically
+- **Vox restarts** — Lattices reconnects with the new port
+- **Port changes** — no stale config
+
+### 3. Health check
+
+After discovering a port, Lattices confirms Vox is alive:
+
+```json
+→ {"id": "hc", "method": "ping"}
+← {"id": "hc", "result": {"pong": true}}
+```
+
+If ping fails, Lattices marks voice as unavailable and retries on the next `live.ready` or after ~30 seconds.
+
+### When Vox is not running
+
+Three possible states:
+
+| State | How detected | Lattices behavior |
+|---|---|---|
+| **Not installed** | `/Applications/Vox.app` doesn't exist and no `~/.vox/` dir | Footer: `[Space] Voice (unavailable)` — no recovery action |
+| **Installed but not running** | App bundle exists, but `services.json` missing/stale or ping fails | Footer: `[Space] Voice (start Vox)` — pressing Space runs `open /Applications/Vox.app`, which brings up Vox as a side effect |
+| **Running** | Ping succeeds | Normal operation |
+
+Launch-on-demand flow:
+1. User presses Space while Vox is down but Vox is installed
+2. Lattices runs `NSWorkspace.shared.open(URL(fileURLWithPath: "/Applications/Vox.app"))`
+3. Feedback strip shows "Starting Vox..."
+4. Lattices waits for `live.ready` notification (timeout: 10s)
+5. On `live.ready`, connects and proceeds with `startDictation`
+6. On timeout, shows "Couldn't reach Vox — try opening it manually"
+
+Passive behavior (no user action):
+- No log spam — just a quiet unavailable state
+- Lattices keeps listening for `live.ready` and re-checks `services.json` periodically (~30s)
+- The moment Vox comes online, voice becomes available — no restart needed
+
+## Protocol
+
+### Wire Format
+
+Uses Vox's JSON-RPC format over WebSocket:
+
+```
+Request:  {"id": "...", "method": "...", "params": {...}}
+Response: {"id": "...", "result": {...}}  or  {"id": "...", "error": "..."}
+Event:    {"event": "...", "data": {...}}   (server push, no id)
+```
+
+### Methods (Lattices → Vox)
+
+**`startDictation`** — Start recording from the mic.
+
+```json
+{"id": "1", "method": "startDictation", "params": {
+  "source": "lattices",
+  "persist": false
+}}
+```
+
+- `source` — identifies the caller (for Vox's logging/UI)
+- `persist: false` — do not save as a memo, do not sync, do not show in Vox history
+
+Response (immediate ack):
+```json
+{"id": "1", "result": {"ok": true}}
+```
+
+Error responses:
+```json
+{"id": "1", "error": "Microphone access denied"}
+{"id": "1", "error": "No model loaded"}
+{"id": "1", "error": "mic_busy", "owner": "vox"}
+```
+
+The `mic_busy` error means another consumer (Vox's own memo recording, or another client) already has an active dictation. The `owner` field identifies who holds the mic. Lattices shows: "Mic in use by Vox — finish your memo first".
+
+The reverse case (user hits Vox hotkey while Lattices has the mic) is handled on Vox's side — it should reject its own recording with an equivalent busy state. Vox is the single owner of mic arbitration.
+
+**`stopDictation`** — Stop recording and return the transcript.
+
+```json
+{"id": "2", "method": "stopDictation"}
+```
+
+Response (after transcription completes):
+```json
+{"id": "2", "result": {
+  "transcript": "tile this left",
+  "confidence": 0.94,
+  "durationMs": 1820
+}}
+```
+
+**`cancelDictation`** — Abort without transcribing.
+
+```json
+{"id": "3", "method": "cancelDictation"}
+```
+
+```json
+{"id": "3", "result": {"ok": true}}
+```
+
+### Events (Vox → Lattices)
+
+Pushed over the WebSocket connection during an active dictation.
+
+| Event | When | Data |
+|---|---|---|
+| `dictation.started` | Mic is hot, recording has begun | `{"source": "lattices"}` |
+| `dictation.transcribing` | Recording stopped, model is running | `{}` |
+| `dictation.result` | Transcription complete | `{"transcript": "...", "confidence": 0.94, "durationMs": 1820}` |
+| `dictation.error` | Something failed during recording or transcription | `{"message": "..."}` |
+
+## Disconnect Contract
+
+If the WebSocket connection drops mid-dictation (Lattices crashes, user quits, network hiccup), Vox **must** auto-cancel the in-flight dictation:
+
+1. Stop recording immediately
+2. Discard any captured audio — do not transcribe
+3. Release the mic so Vox's own UI or a reconnecting client can use it
+4. Log the orphaned dictation for diagnostics: `[dictation] orphaned session from lattices — connection dropped, auto-cancelled`
+
+Vox treats a closed WebSocket as an implicit `cancelDictation`. No grace period, no buffering — if the consumer is gone, the recording is worthless.
+
+On the Lattices side, if the connection drops while in `listening` or `transcribing` state:
+- Feedback strip: "Connection lost" (red)
+- Attempt reconnect via normal discovery (ping → `services.json` → wait for `live.ready`)
+- Do not auto-retry the dictation — the user needs to press Space again
+
+## End-to-End Lifecycle
+
+```mermaid
+sequenceDiagram
+    participant U as User
+    participant L as Lattices UI
+    participant TA as Vox
+    participant IE as Intent Engine
+
+    U->>L: Press Space (in cheat sheet)
+    L->>TA: startDictation (persist: false)
+
+    alt Error
+        TA-->>L: error (mic denied / no model)
+        L->>U: Red text in feedback strip
+    else OK
+        TA-->>L: {ok: true}
+        TA-->>L: dictation.started
+        L->>U: Green dot (pulsing) + "Listening..."
+
+        Note over U,TA: User speaks...
+
+        U->>L: Press Space again
+        L->>TA: stopDictation
+        TA-->>L: dictation.transcribing
+        L->>U: "Transcribing..."
+
+        TA-->>L: {transcript: "tile this left", confidence: 0.94}
+        L->>U: Show transcript
+    end
+
+    L->>IE: Classify via NLEmbedding
+    IE-->>L: intent: tile_window, slots: {position: left}, confidence: 0.95
+    L->>U: Show intent + slots
+
+    L->>IE: Execute
+    IE-->>L: result
+    L->>U: "Done" or error
+
+    Note over L: Log entry written
+```
+
+## UI States
+
+| State | Feedback strip | Footer |
+|---|---|---|
+| **Idle** | Hidden | `[Space] Voice  [ESC] Dismiss` |
+| **Not installed** | Hidden | `[Space] Voice (unavailable)  [ESC] Dismiss` |
+| **Installed, not running** | Hidden | `[Space] Voice (start Vox)  [ESC] Dismiss` |
+| **Starting** | "Starting Vox..." | `[ESC] Cancel` |
+| **Error** | Red: "Mic access denied" or "Mic in use by Vox" | `[ESC] Dismiss` |
+| **Disconnected** | Red: "Connection lost" | `[ESC] Dismiss` |
+| **Listening** | Green dot + "Listening..." | `[Space] Stop  [ESC] Cancel` |
+| **Transcribing** | "Transcribing..." | `[ESC] Cancel` |
+| **Result** | `"tile this left"` → `tile window · position: left` → `Done` | `[Space] New  [ESC] Dismiss` |
+
+## Logging
+
+Every voice command produces a diagnostic log entry:
+
+```
+[voice] "tile this left" → tile_window(position: left) → ok (conf=0.95, 1820ms)
+[voice] "organize my stuff" → distribute() → ok (conf=0.79, 2100ms)
+[voice] "do something weird" → (no match, conf=0.41, 900ms)
+[voice] error: Vox not running
+[voice] error: mic_busy (owner: vox)
+[voice] error: connection dropped mid-dictation
+[voice] launched Vox, connected in 2.1s
+```
+
+## Implementation Scope
+
+### Lattices side
+- Use `@vox/client` SDK (`VoxClient` with `service: "agent"`, `clientId: "lattices"`, `capabilities: ["dictation"]`) — see `vox/sdk/SDK.md` for full reference
+- Replace `AVAudioRecorder` in `VoxAudioProvider` with `createDictationSession().start({ persist: false })`
+- Remove mic entitlement and `NSMicrophoneUsageDescription` (Lattices never touches the mic)
+- Service discovery, auto-reconnect, and auth are handled by the SDK
+- Map `DictationSession` events (`stateChange`, `partialTranscript`, `finalTranscript`, `error`) to cheat sheet UI states
+- Handle `MicBusyError` — show `"Mic in use by ${error.owner}"`
+
+### Vox side (separate repo)
+- Expose a WebSocket bridge (or add methods to existing bridge)
+- Add `startDictation`, `stopDictation`, `cancelDictation` handlers
+- Emit `dictation.started`, `dictation.transcribing`, `dictation.result`, `dictation.error` events
+- Honor `persist: false` — skip memo creation and sync
+- Write `~/.vox/services.json` on startup (all service ports)
+- Include `agentPort` in `live.ready` notification userInfo
+- Return `mic_busy` error with `owner` field when another consumer holds the mic
+- Auto-cancel dictation on WebSocket disconnect (closed socket = implicit cancel)

package/docs/voice.md

@@ -0,0 +1,219 @@
+---
+title: Voice Commands
+description: Natural language voice control for window management
+order: 7
+---
+
+Voice commands let you control Lattices by speaking. Press **Hyper+3**
+to open the voice command window, hold **Option** to speak, release to
+stop. Lattices transcribes your speech via Vox,
+matches it to an intent, and executes it.
+
+## Quick start
+
+1. Install Vox (provides mic + transcription)
+2. Install [Claude Code](https://claude.ai/code) CLI (provides AI advisor)
+3. Press **Hyper+3** to open the voice command window
+4. Hold **Option** and speak a command
+5. Release **Option** — Lattices transcribes and executes
+
+## Keyboard shortcuts
+
+| Key | Action |
+|-----|--------|
+| **Hyper+3** | Open/close voice command window |
+| **⌥ (hold)** | Push-to-talk — hold to record, release to stop |
+| **Tab** | Arm/disarm the mic |
+| **Escape** | Cancel recording or dismiss window |
+
+## Built-in commands
+
+### Search
+
+Find windows by app name, title, content, or category.
+
+```
+"Find all vox windows"
+"Find terminals"           → expands to iTerm, Terminal, Warp, etc.
+"Show me all browsers"     → expands to Safari, Chrome, Firefox, Arc, etc.
+"Where is my editor?"      → expands to VS Code, Cursor, Xcode, etc.
+```
+
+Category synonyms are built in — saying "terminals", "browsers", "editors",
+"chat", "music", "mail", or "notes" automatically expands to search for
+the actual app names.
+
+### Tile
+
+Move windows to screen positions.
+
+```
+"Tile this left"
+"Snap to the right half"
+"Maximize the window"
+"Put this in the top right corner"
+```
+
+Voice tiling should resolve into the same canonical daemon mutation used
+by other agent surfaces: `window.place`.
+
+### Focus
+
+Bring a window or app to the front.
+
+```
+"Focus Safari"
+"Switch to Slack"
+"Go to the lattices window"
+```
+
+### Open / Launch
+
+Open applications or project workspaces.
+
+```
+"Open Spotify"
+"Launch the vox project"
+```
+
+### Kill
+
+Close windows or quit applications.
+
+```
+"Kill this window"
+"Close Safari"
+"Quit Spotify"
+```
+
+### Scan
+
+Trigger an OCR scan of visible windows.
+
+```
+"Scan the screen"
+"Read what's on screen"
+```
+
+### Other
+
+```
+"List all windows"
+"Show my sessions"
+"Switch to layer 2"
+"Help"
+```
+
+## AI advisor
+
+Every voice command fires a Claude Haiku advisor in parallel. The
+advisor provides commentary and follow-up suggestions in the **AI
+corner** (bottom-right of the voice command window).
+
+When local matching handles the command well, the AI corner shows
+"no AI needed" with an optional "ask AI" button. When the advisor
+has something useful, it shows a one-line comment and an actionable
+suggestion button.
+
+### How it works
+
+1. You speak a command
+2. Local intent matching runs immediately (fast, free)
+3. Haiku advisor runs in parallel (takes ~2-5 seconds)
+4. If the advisor suggests something, a button appears in the AI corner
+5. Click the suggestion to execute it
+6. If you engage with a suggestion that the local matcher missed,
+   it's recorded in `~/.lattices/advisor-learning.jsonl` for future
+   improvement
+
+### Session persistence
+
+The advisor maintains a conversation session across voice commands.
+It remembers what you've asked and what worked. When the context
+reaches 75% of the model's limit, the session auto-resets.
+
+Context usage and session cost are shown in the AI corner header.
+
+## Configuration
+
+Open **Settings > AI** to configure:
+
+| Setting | Default | Description |
+|---------|---------|-------------|
+| Claude CLI path | Auto-detected | Path to the `claude` binary. Checks `~/.local/bin/claude`, `/usr/local/bin/claude`, `/opt/homebrew/bin/claude`, then `which claude`. |
+| Advisor model | Haiku | `haiku` (fast, cheap) or `sonnet` (smarter, slower) |
+| Budget per session | $0.50 | Maximum spend per Claude CLI invocation |
+
+### Installing Claude Code
+
+```bash
+npm install -g @anthropic-ai/claude-code
+```
+
+Or see [claude.ai/code](https://claude.ai/code) for other install methods.
+
+## Layout
+
+The voice command window has four sections:
+
+| Section | Position | Content |
+|---------|----------|---------|
+| **History** | Left column | Past commands with expandable details |
+| **Voice Command** | Center column | Current transcript, matched intent, results |
+| **Log** | Top-right | Rolling diagnostic log (last 12 entries) |
+| **AI Corner** | Bottom-right | Advisor commentary, suggestions, session stats |
+
+## Search architecture
+
+Voice search uses the same backend as `lattices search`:
+
+1. **Quick search** — window titles, app names, session tags (instant)
+2. **Complete search** — adds terminal cwd/processes + OCR content
+3. **Synonym expansion** — category terms like "terminals" expand to
+   actual app names before searching
+4. **Query cleanup** — strips natural language qualifiers ("and sort by...",
+   "please", "for me") before searching
+
+## Processing resilience
+
+- **15-second timeout** — if processing doesn't complete, returns to idle
+- **Cancellation on dismiss** — closing the window cancels in-flight work
+- **Double-execution prevention** — streaming and stop callbacks can't
+  both fire the intent
+
+## Advisor learning
+
+When the local matcher fails but the AI advisor suggests something that
+you engage with, the interaction is recorded:
+
+```
+~/.lattices/advisor-learning.jsonl
+```
+
+Each line is a JSON object:
+
+```json
+{
+  "timestamp": "2026-03-15T18:30:00.000Z",
+  "transcript": "find all terminals",
+  "localIntent": "search",
+  "localSlots": {"query": "terminals"},
+  "localResultCount": 0,
+  "advisorIntent": "search",
+  "advisorSlots": {"query": "iterm"},
+  "advisorLabel": "Search iTerm"
+}
+```
+
+This dataset captures where the local system falls short and what the
+right answer was. Future work can mine it for automatic synonym
+mappings and phrase pattern improvements.
+
+## Requirements
+
+- **Vox** — provides microphone access and
+  speech-to-text transcription
+- **[Claude Code](https://claude.ai/code)** CLI — provides the AI advisor
+  (optional, voice commands work without it but no AI suggestions)
+- **Accessibility** permission — for window tiling and focus
+- **Screen Recording** permission — for window discovery

package/package.json

@@ -1,10 +1,10 @@
 {
   "name": "@lattices/cli",
-  "version": "0.3.0",
+  "version": "0.4.1",
   "description": "Agentic window manager for macOS — programmable workspace, smart layouts, managed tmux sessions, and a 35+-method agent API",
   "bin": {
-    "lattices": "./bin/lattices.js",
-    "lattices-app": "./bin/lattices-app.js"
+    "lattices": "./bin/lattices.ts",
+    "lattices-app": "./bin/lattices-app.ts"
   },
   "keywords": [
     "tmux",
@@ -21,22 +21,40 @@
   },
   "license": "MIT",
   "exports": {
-    ".": "./bin/client.js",
-    "./daemon-client": "./bin/daemon-client.js"
+    ".": "./bin/client.ts",
+    "./daemon-client": "./bin/daemon-client.ts",
+    "./project-twin": "./bin/project-twin.ts"
   },
   "scripts": {
-    "dev": "bun --cwd docs-site dev"
+    "dev": "bun --cwd docs-site dev",
+    "typecheck": "tsc --noEmit",
+    "build:app-bundle": "bash ./bin/lattices-dev build",
+    "prepack": "bash ./bin/lattices-dev build"
   },
   "type": "module",
-  "engines": {
-    "node": ">=18"
-  },
   "os": ["darwin"],
   "files": [
     "bin",
+    "app/Info.plist",
+    "app/Lattices.app",
+    "app/Lattices.entitlements",
     "app/Package.swift",
+    "app/Resources",
     "app/Sources",
-    "app/Lattices.app/Contents/Info.plist",
+    "app/Tests",
+    "assets/AppIcon.icns",
     "docs"
-  ]
+  ],
+  "devDependencies": {
+    "bun-types": "^1.3.10",
+    "typescript": "^5.9.3"
+  },
+  "dependencies": {
+    "@ai-sdk/anthropic": "^3.0.58",
+    "@ai-sdk/google": "^3.0.43",
+    "@ai-sdk/openai": "^3.0.41",
+    "@ai-sdk/xai": "^3.0.67",
+    "@arach/speakeasy": "^0.2.8",
+    "ai": "^6.0.116"
+  }
 }

package/bin/client.js

@@ -1,4 +0,0 @@
-// Public API — re-exports from daemon-client for a cleaner import path.
-// Usage: import { daemonCall, isDaemonRunning } from '@lattices/cli'
-
-export { daemonCall, isDaemonRunning } from "./daemon-client.js";