twokey 1.0.3 → 1.0.5

package/README.md

@@ -1,210 +1,188 @@
 # TwoKey Linux AI Assistant
 
-TwoKey is a Linux-first desktop assistant prototype. The goal is a small always-available overlay that will later let users hold two keys, speak, and have the result processed by local or online AI providers without switching to a browser or chat window.
-
-Phase 1 implements the foundation only:
-
-- Tauri + React project structure
-- Minimal dark overlay pill
-- Dummy modes: Conversation, Edit Text, Dictation, Feedback
-- Clickable mode menu
-- Placeholder settings window
-- Initial architecture and product documentation
-- X11 Phase 2 prototype: hold `Ctrl+Space` to record audio, double tap `Ctrl+Space` to cycle modes
-- Phase 3 prototype: mock STT and external STT command adapter
-- Phase 4 prototype: local Ollama conversation mode with `qwen2.5:3b`
-
-No TTS or text injection is implemented yet.
-
-## Installation
+TwoKey is a Linux-first desktop AI assistant with a small floating pill overlay.
+The core idea matches the video workflow:
+
+- hold a global hotkey, speak, release
+- audio is recorded and transcribed
+- transcript is sent to the selected AI provider
+- result is used directly in your current desktop workflow
+- optional TTS reads the answer out loud
+
+## What Works Now
+
+- Global hold hotkey on X11 for voice capture.
+- Double-tap hotkey to cycle modes.
+- Single-tap hotkey to open file-context picker.
+- Voice-triggered toolchains for multi-step desktop actions.
+- Four modes:
+  - Conversation
+  - Edit Text
+  - Dictation
+  - Feedback
+- Conversation mode:
+  - transcript -> provider answer
+  - optional TTS playback
+- Edit mode:
+  - read selected text
+  - apply spoken transform via AI
+  - replace selected text directly
+- Dictation mode:
+  - insert transcript at cursor
+- Feedback mode:
+  - local feedback persistence in history DB
+- File context:
+  - TXT/MD/JSON/CSV/PDF content context
+  - image context routing to vision-capable providers
+- Provider routing:
+  - local Ollama
+  - OpenAI-compatible
+  - OpenRouter-compatible
+  - secure API key storage via local keyring
+- Local audit/history persistence in SQLite.
+- Tray icon and settings window.
+- GitHub release check from settings.
+- In-app AppImage update download and launch.
+
+## Current Video Parity
+
+Implemented from video behavior:
+
+- Minimal floating pill UI.
+- Hold-to-talk workflow.
+- Mode switch via double tap.
+- Text workflow without browser/chat-tab context switch.
+- File attach + ask flow.
+- Hybrid local/online providers.
+- Optional TTS answer playback.
+
+Still not fully equivalent to the video vision:
+
+- Toolchains are implemented, but no visual workflow builder exists yet.
+- Wayland still has compositor-specific limits for global hold hotkeys and full automation.
+- Update install is available for AppImage, but no signed rollback-capable updater pipeline yet.
+
+## Install
 
 ```bash
 npm install twokey
 ```
 
-Start the tool directly:
+Run:
 
 ```bash
 twokey
 ```
 
-Default behavior: start the native desktop app in background.
-If no desktop binary is installed yet, `twokey` attempts to download an AppImage from the latest GitHub release into `~/.local/share/twokey/bin/` and starts it.
+Default behavior:
 
-Useful CLI options:
+- starts native desktop app in background
+- if no native binary is installed, tries to download latest AppImage release
+
+Useful options:
 
 ```bash
 twokey --help
 twokey --cli
-twokey --once "Erklaere kurz den Unterschied zwischen X11 und Wayland"
+twokey --once "Erklaere X11 vs Wayland kurz"
 twokey --desktop
 ```
 
-## Minimal Usage
-
-```ts
-import { getPackageInfo } from "twokey";
-
-const info = getPackageInfo();
-console.log(info.name);
-console.log(info.runtimeStatus.waylandGlobalHotkeys);
-```
-
-The package exports runtime status metadata. Current status includes planned/limited areas such as Wayland global hotkeys, TTS, tray menu, SQLite history/audit, and online provider execution until secure API-key storage is implemented.
+## Development
 
-## Requirements
-
-- Linux desktop
-- Node.js 20+
-- npm
-- Rust toolchain with `cargo` for running the Tauri app
-- Tauri Linux system dependencies, for example on Ubuntu/Debian:
-
-```bash
-sudo apt install libwebkit2gtk-4.1-dev build-essential curl wget file libxdo-dev libssl-dev libayatana-appindicator3-dev librsvg2-dev
-```
-
-This workspace currently has Node/npm available. `cargo` is required before the native app can be started.
-
-## Start Development
-
-Install JavaScript dependencies:
+Install dependencies:
 
 ```bash
 npm install
 ```
 
-Run only the web UI:
-
-```bash
-npm run dev
-```
-
-Run the Linux desktop app:
+Run desktop dev stack:
 
 ```bash
 npm run tauri:dev
 ```
 
-Phase 2 hotkey behavior on X11:
-
-- Hold `Ctrl+Space`: start recording after a short hold delay.
-- Release `Ctrl+Space`: stop recording and save a WAV file under `~/.cache/twokey-ai/recordings/`.
-- Double tap `Ctrl+Space`: cycle to the next mode.
-- Press `Escape`: cancel an active recording.
-
-On Wayland, generic global hold-hotkeys are reported as unavailable instead of failing silently.
-
-Phase 3 STT behavior:
-
-- Default STT provider is a deterministic mock transcriber.
-- To test a real local or custom STT command, set `TWOKEY_STT_COMMAND`.
-- The command must print the transcript to stdout and include `{audio}` as placeholder.
+Important:
 
-Example:
-
-```bash
-TWOKEY_STT_COMMAND='whisper-cli -f {audio} --language de --no-timestamps' npm run tauri:dev
-```
+- `target/debug/twokey-ai` alone in dev mode will fail if Vite is not running.
+- For development, use `npm run tauri:dev` so frontend + Tauri run together.
 
-Phase 4 Ollama behavior:
-
-- Ollama runs as a systemd service on `127.0.0.1:11434`.
-- Default model: `qwen2.5:3b`.
-- Conversation mode sends finished transcripts to Ollama and displays the answer in the overlay.
-- Override model or endpoint with:
+Build:
 
 ```bash
-TWOKEY_OLLAMA_MODEL='llama3.2:3b' TWOKEY_OLLAMA_URL='http://127.0.0.1:11434' npm run tauri:dev
+npm run build
+cd src-tauri && cargo check
 ```
 
-Phase 5 dictation behavior:
-
-- In dictation mode, a finished transcript is pasted at the active cursor position.
-- X11 uses `xclip` or `xsel` plus `xdotool`.
-- Clipboard content is read before insertion and restored afterward when possible.
-- Wayland insertion is deliberately reported as unsupported for now.
+## Hotkey Behavior
 
-Phase 6 text editing behavior:
+Default hotkey can be changed in settings. Examples:
 
-- In edit mode, TwoKey reads the current X11 selection with `Ctrl+C`.
-- The spoken instruction and selected text are sent to Ollama.
-- The overlay shows a replacement preview.
-- The selected text is replaced only after pressing `Ersetzen`.
+- `Ctrl+Space`
+- `Ctrl+Super`
+- mouse-button combinations except left/right mouse button in recorder UI
 
-Phase 7 settings behavior:
+Runtime semantics on X11:
 
-- Settings are persisted at `~/.config/twokey-ai/settings.json`.
-- The settings window saves general, hotkey, STT, Ollama, privacy, and update-channel values.
-- Some settings are stored before they are fully applied at runtime; later phases will wire them into the helper/provider layers.
+- hold hotkey: start recording
+- release hotkey: stop recording -> transcribe -> mode action
+- short single tap: open file picker
+- short double tap: switch mode
+- `Escape` while recording: cancel
 
-Phase 8 provider behavior:
+Wayland fallback:
 
-- Ollama chat is routed through a provider abstraction.
-- Provider metadata is exposed to the settings UI.
-- OpenAI-compatible and OpenRouter-compatible providers are visible as planned online providers, but disabled until API-key storage and routing are implemented.
+- if global hotkeys are blocked, manual start/stop recording is available from the menu
+- this keeps the voice pipeline usable on restricted desktops
 
-Phase 9 file context behavior:
+## STT Providers
 
-- File context can be added from the overlay menu.
-- `txt`, `md`, `markdown`, `json`, and `csv` files are read directly.
-- PDFs are extracted with `pdftotext` from `poppler-utils`.
-- Images are registered as context metadata for future vision providers.
-- Extracted text is cached under `~/.cache/twokey-ai/file-contexts/`.
+Configured in settings (`sttProvider`):
 
-Phase 10 packaging behavior:
+- `mock`
+- `local-whisper`
+- `external-command`
+- `openai-compatible`
 
-- `npm run tauri:build` creates AppImage and `.deb` bundles.
-- Settings autostart writes `~/.config/autostart/twokey-ai.desktop`.
-- Generated bundles live under `src-tauri/target/release/bundle/`.
+`external-command` needs `TWOKEY_STT_COMMAND` with `{audio}` placeholder.
 
-Phase 11 update behavior:
-
-- Settings can check GitHub Releases for a newer version.
-- The app only reports availability; it does not auto-download or auto-install.
-
-Current stabilization status:
-
-- AppImage and `.deb` builds complete successfully.
-- Ollama runs locally as a systemd service with `qwen2.5:3b`.
-- Production dependency audit reports no vulnerabilities.
-- Wayland limitations, TTS, tray menu, SQLite history, and online provider execution remain future hardening work.
-
-Build the frontend:
+Example:
 
 ```bash
-npm run build
+TWOKEY_STT_COMMAND='whisper-cli -f {audio} -l de -nt' npm run tauri:dev
 ```
 
-Build the desktop bundle:
+## Linux Requirements
 
-```bash
-npm run tauri:build
-```
+- Node.js 20+
+- npm
+- Rust + cargo
+- Linux desktop dependencies for Tauri
 
-## Project Layout
+Ubuntu/Debian example:
 
-```text
-docs/                 Product, architecture, security, and status docs
-src/                  React frontend
-src-tauri/            Rust/Tauri desktop shell
+```bash
+sudo apt install libwebkit2gtk-4.1-dev build-essential curl wget file libxdo-dev libssl-dev libayatana-appindicator3-dev librsvg2-dev
 ```
 
-## Repository
-
-GitHub: https://github.com/meinzeug/twokey
+Optional runtime tools:
 
-## License
+- X11 automation: `xdotool`, `xclip` or `xsel`
+- PDF extraction: `poppler-utils` (`pdftotext`)
+- TTS backends: `spd-say` or `espeak-ng`/`espeak`
 
-MIT. See `LICENSE`.
-
-## Linux Notes
-
-TwoKey is designed around Linux conventions:
+## Data Paths
 
 - Config: `~/.config/twokey-ai/`
 - Data: `~/.local/share/twokey-ai/`
 - Cache: `~/.cache/twokey-ai/`
-- Logs/state: `~/.local/state/twokey-ai/`
+- History DB: `~/.local/share/twokey-ai/history.db`
+- Toolchains: `~/.config/twokey-ai/toolchains.json`
+
+## Repo
+
+https://github.com/meinzeug/twokey
+
+## License
 
-X11 and Wayland will be handled separately in later phases. Phase 1 only detects and displays the current session type.
+MIT

package/bin/postinstall.js

@@ -0,0 +1,21 @@
+#!/usr/bin/env node
+
+import { spawn } from "node:child_process";
+import path from "node:path";
+import { fileURLToPath } from "node:url";
+
+if (process.env.CI === "true" || process.env.TWOKEY_SKIP_POSTINSTALL === "1") {
+  process.exit(0);
+}
+
+const __filename = fileURLToPath(import.meta.url);
+const __dirname = path.dirname(__filename);
+const cliPath = path.join(__dirname, "twokey.js");
+
+const child = spawn(process.execPath, [cliPath, "--desktop", "--enable-autostart", "--quiet"], {
+  stdio: "ignore",
+  shell: false,
+});
+
+child.on("error", () => process.exit(0));
+child.on("close", () => process.exit(0));

package/bin/twokey.js

@@ -6,14 +6,16 @@ import os from "node:os";
 import path from "node:path";
 import readline from "node:readline";
 
-const VERSION = "1.0.3";
+const VERSION = process.env.npm_package_version || "1.0.5";
 const DEFAULT_MODEL = process.env.TWOKEY_OLLAMA_MODEL || "qwen2.5:3b";
 const DEFAULT_OLLAMA_URL = process.env.TWOKEY_OLLAMA_URL || "http://127.0.0.1:11434";
 const LATEST_RELEASE_API = "https://api.github.com/repos/meinzeug/twokey/releases/latest";
 const APPIMAGE_DIR = path.join(os.homedir(), ".local", "share", "twokey", "bin");
 const APPIMAGE_PATH = path.join(APPIMAGE_DIR, "twokey-ai.AppImage");
+const APPIMAGE_META_PATH = path.join(APPIMAGE_DIR, "twokey-ai.meta.json");
 
 const args = process.argv.slice(2);
+const QUIET = args.includes("--quiet");
 
 if (args.includes("--help") || args.includes("-h")) {
   printHelp();
@@ -26,13 +28,27 @@ if (args.includes("--version") || args.includes("-v")) {
 }
 
 if (args.includes("--desktop")) {
-  launchDesktopApp().then((started) => {
-    if (started) {
-      console.log("TwoKey desktop app started in background.");
+  launchDesktopApp().then(async (startedCommand) => {
+    if (startedCommand) {
+      if (args.includes("--enable-autostart")) {
+        try {
+          await ensureUserService(startedCommand);
+        } catch (error) {
+          if (!QUIET) {
+            const message = error instanceof Error ? error.message : String(error);
+            console.warn(`Autostart setup skipped: ${message}`);
+          }
+        }
+      }
+      if (!QUIET) {
+        console.log("TwoKey desktop app started in background.");
+      }
       process.exit(0);
     }
-    console.error("No native desktop binary found in PATH.");
-    console.error("Install the .deb/.AppImage release and ensure 'twokey-ai' is available in PATH.");
+    if (!QUIET) {
+      console.error("No native desktop binary found in PATH.");
+      console.error("Install the .deb/.AppImage release and ensure 'twokey-ai' is available in PATH.");
+    }
     process.exit(1);
   });
 }
@@ -55,15 +71,29 @@ if (onceIndex >= 0) {
     process.exit(1);
   });
 } else {
-  launchDesktopApp().then((started) => {
-    if (started) {
-      console.log("TwoKey desktop app started in background.");
+  launchDesktopApp().then(async (startedCommand) => {
+    if (startedCommand) {
+      if (args.includes("--enable-autostart")) {
+        try {
+          await ensureUserService(startedCommand);
+        } catch (error) {
+          if (!QUIET) {
+            const message = error instanceof Error ? error.message : String(error);
+            console.warn(`Autostart setup skipped: ${message}`);
+          }
+        }
+      }
+      if (!QUIET) {
+        console.log("TwoKey desktop app started in background.");
+      }
       process.exit(0);
     }
 
-    console.error("Could not start desktop app.");
-    console.error("Tried system binaries and auto-download from GitHub Releases.");
-    console.error("Use 'twokey --cli' to run terminal mode.");
+    if (!QUIET) {
+      console.error("Could not start desktop app.");
+      console.error("Tried system binaries and auto-download from GitHub Releases.");
+      console.error("Use 'twokey --cli' to run terminal mode.");
+    }
     process.exit(1);
   });
 }
@@ -161,46 +191,45 @@ async function askOllama(prompt) {
 }
 
 async function launchDesktopApp() {
+  let appImageReady = false;
+  try {
+    appImageReady = await ensureLocalAppImage();
+  } catch {
+    appImageReady = false;
+  }
+
   const candidates = [];
   if (process.env.TWOKEY_DESKTOP_CMD) {
     candidates.push(process.env.TWOKEY_DESKTOP_CMD);
   }
-  candidates.push("twokey-ai", "twokey-desktop", APPIMAGE_PATH);
+  if (appImageReady) {
+    candidates.push(APPIMAGE_PATH);
+  }
+  candidates.push("twokey-ai", "twokey-desktop");
 
   for (const command of candidates) {
     const started = await spawnDetached(command);
     if (started) {
-      return true;
-    }
-  }
-
-  try {
-    const downloaded = await ensureLocalAppImage();
-    if (downloaded) {
-      return spawnDetached(APPIMAGE_PATH);
+      return command;
     }
-  } catch {
-    return false;
   }
 
-  return false;
+  return null;
 }
 
 async function ensureLocalAppImage() {
-  try {
-    await fs.promises.access(APPIMAGE_PATH, fs.constants.X_OK);
-    return true;
-  } catch {
-    // Not installed yet.
+  await fs.promises.mkdir(APPIMAGE_DIR, { recursive: true });
+
+  const latestAsset = await resolveLatestAppImageAsset();
+  if (!latestAsset) {
+    return hasExecutable(APPIMAGE_PATH);
   }
 
-  await fs.promises.mkdir(APPIMAGE_DIR, { recursive: true });
-  const assetUrl = await resolveLatestAppImageUrl();
-  if (!assetUrl) {
-    return false;
+  if (await isCurrentAppImage(latestAsset)) {
+    return true;
   }
 
-  const response = await fetch(assetUrl, {
+  const response = await fetch(latestAsset.url, {
     headers: {
       "User-Agent": "twokey-cli",
       Accept: "application/octet-stream",
@@ -215,10 +244,19 @@ async function ensureLocalAppImage() {
   await fs.promises.writeFile(APPIMAGE_PATH, data, { mode: 0o755 });
 
   await fs.promises.chmod(APPIMAGE_PATH, 0o755);
+  const meta = {
+    releaseTag: latestAsset.releaseTag,
+    assetName: latestAsset.name,
+    assetId: latestAsset.id,
+    assetSize: latestAsset.size,
+    assetUpdatedAt: latestAsset.updatedAt,
+    downloadedAt: new Date().toISOString(),
+  };
+  await fs.promises.writeFile(APPIMAGE_META_PATH, `${JSON.stringify(meta, null, 2)}\n`, "utf8");
   return true;
 }
 
-async function resolveLatestAppImageUrl() {
+async function resolveLatestAppImageAsset() {
   const response = await fetch(LATEST_RELEASE_API, {
     headers: {
       "User-Agent": "twokey-cli",
@@ -236,7 +274,54 @@ async function resolveLatestAppImageUrl() {
     (asset) => typeof asset?.name === "string" && asset.name.endsWith(".AppImage") && asset.name.includes("amd64"),
   ) || assets.find((asset) => typeof asset?.name === "string" && asset.name.endsWith(".AppImage"));
 
-  return appImage?.browser_download_url ?? null;
+  if (!appImage?.browser_download_url) {
+    return null;
+  }
+
+  return {
+    releaseTag: typeof payload?.tag_name === "string" ? payload.tag_name : "unknown",
+    id: Number.isFinite(appImage.id) ? appImage.id : 0,
+    name: appImage.name,
+    size: Number.isFinite(appImage.size) ? appImage.size : 0,
+    updatedAt: typeof appImage.updated_at === "string" ? appImage.updated_at : "",
+    url: appImage.browser_download_url,
+  };
+}
+
+async function isCurrentAppImage(latestAsset) {
+  if (!(await hasExecutable(APPIMAGE_PATH))) {
+    return false;
+  }
+
+  const [meta, stats] = await Promise.all([readAppImageMeta(), fs.promises.stat(APPIMAGE_PATH)]);
+  if (!meta) {
+    return false;
+  }
+
+  return (
+    meta.releaseTag === latestAsset.releaseTag
+    && meta.assetId === latestAsset.id
+    && meta.assetUpdatedAt === latestAsset.updatedAt
+    && Number(meta.assetSize) === Number(stats.size)
+  );
+}
+
+async function readAppImageMeta() {
+  try {
+    const content = await fs.promises.readFile(APPIMAGE_META_PATH, "utf8");
+    return JSON.parse(content);
+  } catch {
+    return null;
+  }
+}
+
+async function hasExecutable(filePath) {
+  try {
+    await fs.promises.access(filePath, fs.constants.X_OK);
+    return true;
+  } catch {
+    return false;
+  }
 }
 
 function spawnDetached(command) {
@@ -271,3 +356,63 @@ function printHelp() {
   console.log("Without options, twokey starts the native desktop app in background.");
   console.log("If no desktop binary is installed, twokey tries to download an AppImage from latest GitHub release.");
 }
+
+async function ensureUserService(command) {
+  const configHome = process.env.XDG_CONFIG_HOME || path.join(os.homedir(), ".config");
+  const systemdDir = path.join(configHome, "systemd", "user");
+  const servicePath = path.join(systemdDir, "twokey.service");
+  await fs.promises.mkdir(systemdDir, { recursive: true });
+
+  const content = [
+    "[Unit]",
+    "Description=TwoKey Desktop Assistant",
+    "After=graphical-session.target",
+    "",
+    "[Service]",
+    `ExecStart=/bin/sh -lc ${shellEscape(command)}`,
+    "Restart=on-failure",
+    "RestartSec=3",
+    "",
+    "[Install]",
+    "WantedBy=default.target",
+    "",
+  ].join("\n");
+
+  await fs.promises.writeFile(servicePath, content, "utf8");
+
+  await runSystemctlUser(["daemon-reload"]);
+  await runSystemctlUser(["enable", "--now", "twokey.service"]);
+
+  if (!QUIET) {
+    console.log("TwoKey systemd user service enabled: twokey.service");
+  }
+}
+
+async function runSystemctlUser(argsList) {
+  return new Promise((resolve, reject) => {
+    const child = spawn("systemctl", ["--user", ...argsList], {
+      stdio: QUIET ? "ignore" : "pipe",
+      shell: false,
+    });
+
+    let stderr = "";
+    if (child.stderr) {
+      child.stderr.on("data", (chunk) => {
+        stderr += String(chunk);
+      });
+    }
+
+    child.on("error", (error) => reject(error));
+    child.on("close", (code) => {
+      if (code === 0) {
+        resolve();
+        return;
+      }
+      reject(new Error(stderr.trim() || `systemctl --user failed with code ${code}`));
+    });
+  });
+}
+
+function shellEscape(value) {
+  return `'${String(value).replace(/'/g, `'\\''`)}'`;
+}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "twokey",
-  "version": "1.0.3",
+  "version": "1.0.5",
   "description": "Linux-first desktop AI assistant built with Tauri, React, and TypeScript.",
   "license": "MIT",
   "repository": {
@@ -42,9 +42,10 @@
     "preview": "vite preview --host 127.0.0.1",
     "lint": "npm run typecheck",
     "test": "npm run typecheck",
+    "postinstall": "node ./bin/postinstall.js",
     "prepublishOnly": "npm run build && npm pack --dry-run",
-    "tauri:dev": "tauri dev",
-    "tauri:build": "tauri build"
+    "tauri:dev": "GTK_MODULES='' LIBGL_ALWAYS_SOFTWARE=1 WEBKIT_DISABLE_DMABUF_RENDERER=1 tauri dev",
+    "tauri:build": "GTK_MODULES='' LIBGL_ALWAYS_SOFTWARE=1 WEBKIT_DISABLE_DMABUF_RENDERER=1 tauri build"
   },
   "dependencies": {
     "@tauri-apps/api": "^2.5.0",