copilot-liku-cli 0.0.4 → 0.0.9

package/QUICKSTART.md

@@ -42,6 +42,27 @@ liku start
 npm start
 ```
 
+## Quick Verify (Recommended)
+
+After install, run these checks in order:
+
+```bash
+# 1) Deterministic runtime smoke test (default)
+npm run smoke:shortcuts
+
+# 2) Direct chat visibility smoke (no keyboard emulation)
+npm run smoke:chat-direct
+
+# 3) UI automation baseline checks
+npm run test:ui
+```
+
+This order gives clearer pass/fail signals by validating runtime health first,
+then shortcut routing, then module-level UI automation.
+
+`smoke:shortcuts` intentionally validates chat visibility via direct in-app
+toggle and validates keyboard routing on overlay with target gating.
+
 ## First Use
 
 ### 1. Application Launch
@@ -79,6 +100,9 @@ To make the overlay click-through again:
 
 ## Keyboard Shortcuts
 
+Source-of-truth for these mappings is the current main-process registration in
+`src/main/index.js`.
+
 | Shortcut | Action |
 |----------|--------|
 | `Ctrl+Alt+Space` (macOS: `Cmd+Alt+Space`) | Toggle chat window |

package/README.md

@@ -1,8 +1,9 @@
 # GitHub Copilot CLI: Liku Edition (Public Preview)
 
 [![npm version](https://img.shields.io/npm/v/copilot-liku-cli.svg)](https://www.npmjs.com/package/copilot-liku-cli)
-[![Node.js](https://img.shields.io/badge/node-%3E%3D22.0.0-brightgreen.svg)](https://nodejs.org/)
+[![Node.js](https://img.shields.io/badge/node-%3E%3D18.0.0-brightgreen.svg)](https://nodejs.org/)
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE.md)
+[![Package Size](https://img.shields.io/badge/package-~196KB-blue.svg)](https://www.npmjs.com/package/copilot-liku-cli)
 
 The power of GitHub Copilot, now with visual-spatial awareness and advanced automation.
 
@@ -20,6 +21,7 @@ We're bringing the power of GitHub Copilot coding agent directly to your termina
 - **Ultra-Thin Overlay:** A transparent Electron layer for high-performance UI element detection and interaction.
 - **Multi-Agent Orchestration:** A sophisticated **Supervisor-Builder-Verifier** pattern for complex, multi-step task execution.
 - **Liku CLI Suite:** A comprehensive set of automation tools (`click`, `find`, `type`, `keys`, `screenshot`) available from any shell.
+- **Event-Driven UI Watcher:** Real-time UI state tracking via Windows UI Automation events with automatic polling fallback.
 - **Defensive AI Architecture:** Engineered for minimal footprint ($<300$MB memory) and zero-intrusion workflows.
 
 ## 🛠️ The Liku CLI (`liku`)
@@ -34,6 +36,8 @@ liku
 ```
 This launches the Electron-based visual agent including the chat interface and the transparent overlay.
 
+> **Note:** The visual overlay requires Electron (installed automatically as an optional dependency). All headless CLI commands (`click`, `find`, `type`, `keys`, `screenshot`, etc.) work without Electron.
+
 ### Automation Commands
 | Command | Usage | Description |
 | :--- | :--- | :--- |
@@ -43,12 +47,27 @@ This launches the Electron-based visual agent including the chat interface and t
 | `keys` | `liku keys ctrl+s` | Send complex keyboard combinations. |
 | `window` | `liku window "VS Code"` | Focus a specific application window. |
 | `screenshot`| `liku screenshot` | Capture the current screen state for analysis. |
+| `mouse` | `liku mouse 500 300` | Move the mouse to screen coordinates. |
+| `scroll` | `liku scroll down` | Scroll the active window or element. |
+| `drag` | `liku drag 100,200 400,500` | Drag from one point to another. |
+| `wait` | `liku wait "Loading..." --gone` | Wait for an element to appear or disappear. |
 | `repl` | `liku repl` | Launch an interactive automation shell. |
+| `agent` | `liku agent "Refactor login"` | Start a multi-agent task. |
 
 ### Power User Examples
-- **Chained Automation**: `liku window "Notepad" && liku type "Done!" && liku keys ctrl+s`
-- **Coordinate Precision**: `liku click 500,300 --right`
-- **JSON Processing**: `liku find "*" --json | jq '.[0].name'`
+```bash
+# Chained automation
+liku window "Notepad" && liku type "Done!" && liku keys ctrl+s
+
+# Coordinate precision
+liku click 500,300 --right
+
+# JSON processing
+liku find "*" --json | jq '.[0].name'
+
+# Wait for UI state
+liku wait "Submit" --timeout 5000 && liku click "Submit"
+```
 
 ## 👁️ Visual Awareness & Grid System
 
@@ -57,6 +76,7 @@ Liku perceives your workspace through a dual-mode interaction layer.
 - **Passive Mode:** Fully click-through, remaining dormant until needed.
 - **Dot-Grid Targeting:** When the agent needs to target a specific point, it generates a coordinate grid (Coarse ~100px or Fine ~25px) using alphanumeric labels (e.g., `A1`, `C3.21`).
 - **Live UI Inspection:** Uses native accessibility trees (Windows UI Automation) to highlight and "lock onto" buttons, menus, and text fields in real-time.
+- **Event-Driven Updates:** The UI watcher uses a 4-state machine (POLLING → EVENT_MODE → FALLBACK) to stream live focus, structure, and property changes with automatic health monitoring.
 
 ### Global Shortcuts (Overlay)
 - `Ctrl+Alt+Space`: Toggle the Chat Interface.
@@ -84,83 +104,115 @@ The Liku Edition moves beyond single-turn responses with a specialized team of a
 
 ### Prerequisites
 
-- **Node.js** v22 or higher
-- **npm** v10 or higher
-- (On Windows) **PowerShell** v6 or higher
-- An **active Copilot subscription**.
+- **Node.js** v18 or higher (v22 recommended)
+- **npm** v9 or higher
+
+#### Platform-Specific
+
+| Platform | UI Automation | Requirements |
+| :--- | :--- | :--- |
+| **Windows** | Full (UIA + events) | PowerShell v5.1+; [.NET 9 SDK](https://dotnet.microsoft.com/download) for building the UIA host |
+| **macOS** | Partial | Accessibility permissions required |
+| **Linux** | Partial | AT-SPI2 recommended |
+
+> **Windows UI Automation:** On `npm install`, a postinstall script automatically builds the .NET 9 UIA host binary if the .NET SDK is detected. If skipped, you can build it manually with `npm run build:uia`.
 
 ### Installation
 
-#### Global Installation (Recommended for Users)
+#### Global Install (Recommended)
 
-Install globally from npm:
 ```bash
 npm install -g copilot-liku-cli
 ```
 
-This will make the `liku` command available globally from any directory.
-
-To verify installation:
+Verify:
 ```bash
 liku --version
+liku --help
 ```
 
-To update to the latest version:
+Update:
 ```bash
 npm update -g copilot-liku-cli
 ```
 
-#### Local Development Installation
+#### From Source
 
-To install the Liku Edition for local development and contributing:
 ```bash
 git clone https://github.com/TayDa64/copilot-Liku-cli
 cd copilot-Liku-cli
 npm install
 npm link
-```
-This will make the `liku` command available globally, linked to your local development copy.
 
-**Note for contributors:** Use `npm link` during development so changes are immediately reflected without reinstalling.
+# Build the .NET UIA host (Windows only)
+npm run build:uia
+```
 
 ### Authenticate
 
-If you're not logged in, launch the agent and use the `/login` slash command, or set a personal access token (PAT):
+Set a GitHub personal access token with Copilot permissions:
 1. Visit [GitHub PAT Settings](https://github.com/settings/personal-access-tokens/new)
 2. Enable "Copilot Requests" permission.
 3. Export `GH_TOKEN` or `GITHUB_TOKEN` in your environment.
 
+Or launch the agent and use the `/login` slash command.
+
+## ✅ Quick Verify
+
+```bash
+# Full smoke suite (233 assertions)
+npm run smoke
+
+# Individual checks
+npm run smoke:shortcuts    # Runtime + shortcut diagnostics
+npm run smoke:chat-direct  # Chat visibility (no keyboard emulation)
+npm run test:ui            # UI automation baseline
+```
+
 ## 🛠️ Technical Architecture
 
-GitHub Copilot-Liku CLI is built on a "Defensive AI" architecture—a design philosophy focused on minimal footprint, secure execution, and zero-intrusion workflows. 
+GitHub Copilot-Liku CLI is built on a "Defensive AI" architecture — minimal footprint, secure execution, and zero-intrusion workflows.
+
+### Key Systems
+
+| Layer | Description |
+| :--- | :--- |
+| **CLI** | 13 headless commands via `src/cli/liku.js` (CJS, no Electron required) |
+| **.NET UIA Host** | Persistent JSONL process for Windows UI Automation (9 commands, thread-safe, event streaming) |
+| **UI Watcher** | 4-state machine: POLLING ↔ EVENT_MODE ↔ FALLBACK with 10s health check |
+| **Overlay** | Transparent Electron window with grid, inspect regions, and click-through passthrough |
+| **Agent System** | Supervisor → Builder / Researcher → Verifier pipeline |
 
 ### Performance Benchmarks
 
-Engineered for performance and stability, the system hits the following metrics:
 - **Memory Footprint**: $< 300$MB steady-state (~150MB baseline).
 - **CPU Usage**: $< 0.5\%$ idle; $< 2\%$ in selection mode.
 - **Startup Latency**: $< 3$ seconds from launch to functional state.
+- **Package Size**: ~196 KB (npm tarball).
 
 ### Security & Isolation
 
 - **Hardened Electron Environment**: Uses `contextIsolation` and `sandbox` modes to prevent prototype pollution.
 - **Content Security Policy (CSP)**: Strict headers to disable unauthorized external resources.
 - **Isolated Preload Bridges**: Secure IPC routing where renderers only have access to necessary system APIs.
+- **No bundled secrets**: API keys read from environment variables only; tokens stored in `~/.liku-cli/`.
 
-## 🚧 Overlay Development
+## 📚 Documentation
 
-See `docs/inspect-overlay-plan.md` for the inspect overlay plan and acceptance criteria.
+- **[Installation Guide](INSTALLATION.md)** — Detailed installation instructions for all platforms
+- **[Quick Start Guide](QUICKSTART.md)** — Get up and running quickly
 
-## 📚 Documentation
+<details>
+<summary>Developer docs (available in the repo, not shipped with npm)</summary>
+
+- **[Contributing Guide](CONTRIBUTING.md)** — How to contribute to the project
+- **[Publishing Guide](PUBLISHING.md)** — How to publish the package to npm
+- **[Release Process](RELEASE_PROCESS.md)** — How to create and manage releases
+- **[Architecture](ARCHITECTURE.md)** — System design and architecture
+- **[Configuration](CONFIGURATION.md)** — Configuration options
+- **[Testing](TESTING.md)** — Testing guide and practices
 
-- **[Installation Guide](INSTALLATION.md)** - Detailed installation instructions for all platforms
-- **[Quick Start Guide](QUICKSTART.md)** - Get up and running quickly
-- **[Contributing Guide](CONTRIBUTING.md)** - How to contribute to the project
-- **[Publishing Guide](PUBLISHING.md)** - How to publish the package to npm
-- **[Release Process](RELEASE_PROCESS.md)** - How to create and manage releases
-- **[Architecture](ARCHITECTURE.md)** - System design and architecture
-- **[Configuration](CONFIGURATION.md)** - Configuration options
-- **[Testing](TESTING.md)** - Testing guide and practices
+</details>
 
 ## 📢 Feedback and Participation
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "copilot-liku-cli",
-  "version": "0.0.4",
+  "version": "0.0.9",
   "description": "GitHub Copilot CLI with headless agent + ultra-thin overlay architecture",
   "main": "src/main/index.js",
   "bin": {
@@ -10,7 +10,12 @@
     "start": "node scripts/start.js",
     "test": "node scripts/test-grid.js",
     "test:ui": "node scripts/test-ui-automation-baseline.js",
-    "liku": "node src/cli/liku.js"
+    "smoke:shortcuts": "node scripts/smoke-shortcuts.js",
+    "smoke:chat-direct": "node scripts/smoke-chat-direct.js",
+    "smoke": "node scripts/smoke-command-system.js",
+    "liku": "node src/cli/liku.js",
+    "build:uia": "powershell -ExecutionPolicy Bypass -File src/native/windows-uia-dotnet/build.ps1",
+    "postinstall": "node scripts/postinstall.js"
   },
   "keywords": [
     "copilot",
@@ -22,7 +27,7 @@
     "ui-automation",
     "ai"
   ],
-  "author": "GitHub",
+  "author": "TayDa64",
   "license": "MIT",
   "repository": {
     "type": "git",
@@ -33,8 +38,8 @@
   },
   "homepage": "https://github.com/TayDa64/copilot-Liku-cli#readme",
   "engines": {
-    "node": ">=22.0.0",
-    "npm": ">=10.0.0"
+    "node": ">=18.0.0",
+    "npm": ">=9.0.0"
   },
   "os": [
     "darwin",
@@ -42,20 +47,24 @@
     "linux"
   ],
   "files": [
-    "src/",
+    "src/cli/",
+    "src/main/",
+    "src/shared/",
+    "src/renderer/",
+    "src/assets/",
+    "src/native/windows-uia/Program.cs",
+    "src/native/windows-uia/build.ps1",
+    "src/native/windows-uia-dotnet/Program.cs",
+    "src/native/windows-uia-dotnet/WindowsUIA.csproj",
+    "src/native/windows-uia-dotnet/build.ps1",
     "scripts/start.js",
+    "scripts/postinstall.js",
     "README.md",
     "LICENSE.md",
     "QUICKSTART.md",
-    "INSTALLATION.md",
-    "CONTRIBUTING.md",
-    "ARCHITECTURE.md",
-    "CONFIGURATION.md",
-    "TESTING.md",
-    "ELECTRON_README.md",
-    "PROJECT_STATUS.md"
+    "INSTALLATION.md"
   ],
-  "dependencies": {
+  "optionalDependencies": {
     "electron": "^35.7.5"
   }
 }

package/scripts/postinstall.js

@@ -0,0 +1,63 @@
+#!/usr/bin/env node
+/**
+ * postinstall — attempt to build the .NET UIA host binary on Windows.
+ * Gracefully skips on non-Windows platforms or if .NET SDK is absent.
+ */
+const { execSync } = require('child_process');
+const path = require('path');
+const fs = require('fs');
+
+const ROOT = path.resolve(__dirname, '..');
+const BIN_DIR = path.join(ROOT, 'bin');
+const EXE = path.join(BIN_DIR, 'WindowsUIA.exe');
+const BUILD_SCRIPT = path.join(ROOT, 'src', 'native', 'windows-uia-dotnet', 'build.ps1');
+
+// Skip on non-Windows
+if (process.platform !== 'win32') {
+  console.log('[postinstall] Not Windows — skipping UIA host build (headless CLI commands still work).');
+  process.exit(0);
+}
+
+// Already built?
+if (fs.existsSync(EXE)) {
+  console.log('[postinstall] WindowsUIA.exe already exists — skipping build.');
+  process.exit(0);
+}
+
+// Check for .NET SDK
+try {
+  const ver = execSync('dotnet --version', { encoding: 'utf-8', timeout: 10000 }).trim();
+  const major = parseInt(ver.split('.')[0], 10);
+  if (major < 9) {
+    console.log(`[postinstall] .NET SDK ${ver} found but v9+ required for UIA host. Skipping build.`);
+    console.log('  Install .NET 9 SDK from https://dotnet.microsoft.com/download and run: npm run build:uia');
+    process.exit(0);
+  }
+} catch {
+  console.log('[postinstall] .NET SDK not found — skipping UIA host build.');
+  console.log('  UI-automation features require the .NET 9 host. Install .NET 9 SDK and run: npm run build:uia');
+  process.exit(0);
+}
+
+// Check for build script
+if (!fs.existsSync(BUILD_SCRIPT)) {
+  console.log('[postinstall] Build script not found — skipping UIA host build.');
+  process.exit(0);
+}
+
+// Build
+console.log('[postinstall] Building WindowsUIA.exe...');
+try {
+  execSync(
+    `powershell -ExecutionPolicy Bypass -File "${BUILD_SCRIPT}"`,
+    { cwd: ROOT, stdio: 'inherit', timeout: 120000 }
+  );
+  if (fs.existsSync(EXE)) {
+    console.log('[postinstall] WindowsUIA.exe built successfully.');
+  } else {
+    console.warn('[postinstall] Build completed but WindowsUIA.exe not found. Run manually: npm run build:uia');
+  }
+} catch (err) {
+  console.warn('[postinstall] UIA host build failed (non-fatal). Run manually: npm run build:uia');
+  console.warn('  ' + (err.message || err));
+}

package/src/cli/commands/window.js

@@ -23,9 +23,25 @@ function loadUI() {
  *   liku window                     # List all windows
  *   liku window "Visual Studio"    # Focus window by title
  *   liku window --active           # Show active window info
+ *   liku window --front "Notepad"   # Bring window to front
+ *   liku window --back "Notepad"    # Send window to back
+ *   liku window --minimize "Notepad"
+ *   liku window --restore "Notepad"
  */
 async function run(args, options) {
   loadUI();
+
+  const titleFromArgs = args.length > 0 ? args.join(' ') : null;
+  const getTarget = (preferredTitle = null) => {
+    const title = preferredTitle || titleFromArgs || options.title || null;
+    if (options.hwnd) {
+      return { hwnd: Number(options.hwnd) };
+    }
+    if (title) {
+      return { title };
+    }
+    return null;
+  };
   
   // Show active window info
   if (options.active) {
@@ -49,6 +65,56 @@ ${highlight('Active Window:')}
     }
     return { success: true, window: win };
   }
+
+  if (options.front || options.back || options.minimize || options.restore || options.maximize) {
+    const operation = options.front ? 'front'
+      : options.back ? 'back'
+      : options.minimize ? 'minimize'
+      : options.maximize ? 'maximize'
+      : 'restore';
+
+    const preferredTitle =
+      typeof options.front === 'string' ? options.front
+        : typeof options.back === 'string' ? options.back
+          : typeof options.minimize === 'string' ? options.minimize
+            : typeof options.maximize === 'string' ? options.maximize
+              : typeof options.restore === 'string' ? options.restore
+                : null;
+
+    const target = getTarget(preferredTitle);
+    if (!target) {
+      error('No target window specified. Pass title text or --hwnd <handle>.');
+      return { success: false, error: 'No target window specified' };
+    }
+
+    if (!options.quiet) {
+      info(`Window op: ${operation} (${target.hwnd ? `hwnd=${target.hwnd}` : `title="${target.title}"`})`);
+    }
+
+    let result;
+    if (operation === 'front') {
+      result = await ui.bringWindowToFront(target);
+    } else if (operation === 'back') {
+      result = await ui.sendWindowToBack(target);
+    } else if (operation === 'minimize') {
+      result = await ui.minimizeWindow(target);
+    } else if (operation === 'maximize') {
+      result = await ui.maximizeWindow(target);
+    } else {
+      result = await ui.restoreWindow(target);
+    }
+
+    if (!result?.success) {
+      error(`Window operation failed: ${operation}`);
+      return { success: false, error: `window ${operation} failed`, operation };
+    }
+
+    if (!options.quiet) {
+      success(`Window operation complete: ${operation}`);
+    }
+
+    return { success: true, operation, target, result };
+  }
   
   // Focus window by title
   if (args.length > 0) {

package/src/main/agents/base-agent.js

@@ -12,7 +12,8 @@ const AgentRole = {
   SUPERVISOR: 'supervisor',
   BUILDER: 'builder',
   VERIFIER: 'verifier',
-  RESEARCHER: 'researcher'
+  RESEARCHER: 'researcher',
+  PRODUCER: 'producer'
 };
 
 // Agent capabilities
@@ -115,12 +116,19 @@ class BaseAgent extends EventEmitter {
     });
 
     const systemPrompt = this.getSystemPrompt();
-    const response = await this.aiService.chat(message, {
-      systemPrompt,
-      history: this.conversationHistory,
-      model: options.model,
-      ...options
-    });
+    const CHAT_TIMEOUT_MS = 60000;
+
+    const response = await Promise.race([
+      this.aiService.chat(message, {
+        systemPrompt,
+        history: this.conversationHistory,
+        model: options.model,
+        ...options
+      }),
+      new Promise((_, reject) =>
+        setTimeout(() => reject(new Error(`AI chat timed out after ${CHAT_TIMEOUT_MS / 1000}s`)), CHAT_TIMEOUT_MS)
+      )
+    ]);
 
     // Add response to history
     this.conversationHistory.push({