automify 0.1.10 → 0.2.0

package/README.md

@@ -8,13 +8,15 @@
 
 `Automify` is a Node.js library for AI computer use and command use across web apps, terminals, native desktops, Docker CLI sandboxes, and Docker-backed Linux desktops.
 
+Created by [Aldo Vincenti](https://aldovincenti.com).
+
 Computer use surfaces:
 
-| Surface        | Factory                     | Controlled environment                                      |
-| -------------- | --------------------------- | ----------------------------------------------------------- |
-| Browser        | `automify.browser()`        | Playwright browser with screenshots and actions             |
-| Desktop        | `automify.localComputer()`  | Native desktop on the current macOS, Windows, or Linux host |
-| Docker desktop | `automify.dockerComputer()` | Linux desktop inside a running Docker container             |
+| Surface        | Factory                     | Controlled environment                                    |
+| -------------- | --------------------------- | --------------------------------------------------------- |
+| Browser        | `automify.browser()`        | Playwright browser with screenshots and actions           |
+| Desktop        | `automify.localComputer()`  | Native desktop on macOS, Windows, or Linux X11/Xorg hosts |
+| Docker desktop | `automify.dockerComputer()` | Linux desktop inside a running Docker container           |
 
 Command use surfaces:
 
@@ -40,6 +42,9 @@ Full docs live at [aldovincenti.github.io/automify](https://aldovincenti.github.
 
 ```bash
 npm install automify
+
+# Ubuntu 26.04 only, if Playwright blocks Chromium install
+PLAYWRIGHT_HOST_PLATFORM_OVERRIDE=ubuntu24.04-x64 npm install automify
 ```
 
 Chromium is installed by the package `postinstall` script. Skip it with:
@@ -66,6 +71,37 @@ npm install zod
 
 Automify does not require Zod for `jsonOutput()` or any browser, CLI, or desktop runtime.
 
+### Optional Docker Setup
+
+Docker is required only for `automify.dockerCli()` and `automify.dockerComputer()`.
+
+On macOS and Windows, install Docker Desktop from the official Docker website:
+
+- macOS: <https://docs.docker.com/desktop/setup/install/mac-install/>
+- Windows: <https://docs.docker.com/desktop/setup/install/windows-install/>
+
+On Ubuntu, install Docker from the Ubuntu repositories:
+
+```bash
+sudo apt-get update
+sudo apt-get install -y docker.io
+```
+
+Use `docker.io`, not the `docker` package. In Ubuntu packages, `docker.io` provides the Docker Engine/runtime and CLI.
+
+Start Docker on Ubuntu and enable it after reboot:
+
+```bash
+sudo systemctl enable --now docker
+sudo docker run hello-world
+```
+
+To run Docker commands without `sudo`, add your user to the `docker` group, then log out and back in:
+
+```bash
+sudo usermod -aG docker $USER
+```
+
 ## Quick Start
 
 ```js
@@ -99,7 +135,10 @@ try {
     })
   });
 
-  console.log(run.ok, run.parsed.id, run.parsed.firstName, run.parsed.lastName);
+  console.log(run.parsed.id, run.parsed.firstName, run.parsed.lastName);
+} catch (error) {
+  console.error("Automation failed:", error);
+  process.exitCode = 1;
 } finally {
   await browser.close();
 }
@@ -147,7 +186,7 @@ const cli = automify.cli({
 await cli.do("Run the tests and summarize failures");
 ```
 
-Use Docker CLI when command execution should happen inside an isolated container. Docker must be installed and running before you create the adapter:
+Use Docker CLI when command execution should happen inside an isolated container. Docker must be installed and running before you create the adapter. See [Optional Docker Setup](#optional-docker-setup) if you still need to install Docker:
 
 ```js
 import { mkdir, mkdtemp, readFile, writeFile } from "node:fs/promises";
@@ -198,7 +237,9 @@ try {
 
 ### Desktop Computer Use
 
-Local desktop computer use controls the native desktop on the machine running your Node.js process. It supports macOS, Windows, and Linux through the local desktop adapter. It needs native desktop dependencies that are not installed by default, and your OS may ask for permission to control the desktop.
+Local desktop computer use controls the native desktop on the machine running your Node.js process. It supports macOS, Windows, and Linux through the local desktop adapter. On Linux, local desktop support requires X11/Xorg or Xvfb; Wayland sessions are not supported. It needs native desktop dependencies that are not installed by default, and your OS may ask for permission to control the desktop.
+
+**Linux Wayland is not supported for local desktop control.** If `echo $XDG_SESSION_TYPE` prints `wayland`, `localComputer()` can fail during screenshot capture with native X11 errors such as `BadMatch` / `X_GetImage`. Use an Xorg session, run under Xvfb with `forceVirtualDisplay`, or use `dockerComputer()` for an isolated Linux desktop.
 
 Before running `npx automify-install-desktop`, install the native build tools for your OS:
 
@@ -224,9 +265,9 @@ sudo dnf install -y gcc-c++ make cmake libXtst-devel libpng-devel
 sudo pacman -S --needed base-devel cmake libxtst libpng
 ```
 
-On Linux, install the full package list before running `npx automify-install-desktop`; the installer checks for command-line build tools but does not verify every native library package. On headless Linux hosts, also install `xvfb` unless you manage `DISPLAY` yourself. On macOS, install Homebrew first if `brew` is not available, then install CMake with `brew install cmake`. On macOS and Windows, `cmake --version` must work in the terminal where you run `npx automify-install-desktop`. On Windows, the VS Code CMake Tools extension is not enough by itself, and Visual Studio 2026 is not currently recognized by the native build chain used by nut.js.
+On Linux, install the full package list before running `npx automify-install-desktop`; the installer checks for command-line build tools but does not verify every native library package. Linux local desktop capture is X11-based: use Xorg/X11, not Wayland. On headless Linux hosts, also install `xvfb` unless you manage `DISPLAY` yourself. On macOS, install Homebrew first if `brew` is not available, then install CMake with `brew install cmake`. On macOS and Windows, `cmake --version` must work in the terminal where you run `npx automify-install-desktop`. On Windows, the VS Code CMake Tools extension is not enough by itself, and Visual Studio 2026 is not currently recognized by the native build chain used by nut.js.
 
-`npx automify-install-desktop` stores the compiled desktop runtime outside `node_modules` in a long-term cache, so normal `npm update` runs do not remove it. If a later `npm install` or `npm update` detects that a previously installed desktop runtime no longer matches the current platform, CPU architecture, Node ABI, or pinned nut.js/libnut revisions, Automify rebuilds it automatically during `postinstall`. Default cache roots are `%LOCALAPPDATA%\automify\desktop-runtime` on Windows, `~/Library/Caches/automify/desktop-runtime` on macOS, and `${XDG_CACHE_HOME:-~/.cache}/automify/desktop-runtime` on Linux. Override with `AUTOMIFY_DESKTOP_RUNTIME_DIR`; disable auto-rebuild with `AUTOMIFY_SKIP_DESKTOP_AUTO_REBUILD=1`.
+`npx automify-install-desktop` stores the compiled desktop runtime outside `node_modules` in a long-term cache, so normal `npm update` runs do not remove it. If the command is run again and the cached runtime already matches the current platform, CPU architecture, Node ABI, and pinned nut.js/libnut revisions, Automify prints a skip message and exits without rebuilding. Use `npx automify-install-desktop --force` (or `npx automify-install-desktop force`) to rebuild a compatible cache anyway. If a later `npm install` or `npm update` detects that a previously installed desktop runtime no longer matches the current environment, Automify rebuilds it automatically during `postinstall`. Default cache roots are `%LOCALAPPDATA%\automify\desktop-runtime` on Windows, `~/Library/Caches/automify/desktop-runtime` on macOS, and `${XDG_CACHE_HOME:-~/.cache}/automify/desktop-runtime` on Linux. Override with `AUTOMIFY_DESKTOP_RUNTIME_DIR`; disable auto-rebuild with `AUTOMIFY_SKIP_DESKTOP_AUTO_REBUILD=1`.
 
 ```js
 import { initAutomify } from "automify";
@@ -244,14 +285,14 @@ const desktop = await automify.localComputer();
 
 try {
   await desktop.do(
-    "Open the Calendar app installed on this computer, find the next event after today, and summarize it. Do not create or edit events."
+    "Open the Calendar app installed on this computer, find the next event, and summarize it. Do not create or edit events."
   );
 } finally {
   await desktop.close();
 }
 ```
 
-For isolated Linux desktop computer use, use Docker. `dockerComputer()` can run from a macOS, Windows, or Linux host with Docker installed and running, but the desktop it controls inside the container is Linux. Docker desktop does not use `automify-install-desktop`; it needs a running Docker daemon and an initial app command:
+For isolated Linux desktop computer use, use Docker. `dockerComputer()` can run from a macOS, Windows, or Linux host with Docker installed and running, but the desktop it controls inside the container is Linux. This is the recommended path when the host Linux session uses Wayland, because `localComputer()` does not support Wayland. Docker desktop does not use `automify-install-desktop`; it needs a running Docker daemon and an initial app command. See [Optional Docker Setup](#optional-docker-setup) if Docker is not installed yet:
 
 ```js
 import { initAutomify } from "automify";
@@ -323,6 +364,27 @@ const run = await browser.do("Create the lead from data and return the saved rec
 - `evaluate` sends images or text files directly to the model.
 - `shared` and `sharedFiles` expose files inside Docker CLI or Docker desktop runs.
 - `jsonOutput()` requests structured JSON and makes parsed output available as `run.parsed`.
+- `limits.steps` controls the maximum model-action turns before `MaxStepsExceededError`. The default is `100`.
+
+Set max steps on an adapter when most runs need the same limit:
+
+```js
+const browser = await automify.browser({
+  startUrl: "https://example.com",
+  limits: { steps: 250 }
+});
+```
+
+Override it for one `.do()` call when a task needs a different limit:
+
+```js
+await browser.do("Quick smoke test", {
+  limits: { steps: 25 }
+});
+```
+
+The older flat option also works: `maxSteps: 250` is equivalent to `limits: { steps: 250 }`. If both are provided,
+`maxSteps` wins.
 
 For arrays of objects, the most ergonomic shape is usually an object with a named array property:
 
@@ -524,3 +586,7 @@ node --test test/e2e/live-openai.e2e.test.js
 ## License
 
 MIT
+
+## Disclaimer
+
+Automify is distributed "as is", without warranty of any kind. Automation can control browsers, shells, desktops, files, and external services; you are responsible for how you configure and run it, and for any events associated with that use. To the maximum extent permitted by law, the author is not liable for losses, damages, data loss, service disruption, or other consequences arising from use of the software.

package/examples/browser-basic.js

@@ -24,7 +24,10 @@ try {
       lastName: "string"
     })
   });
-  console.log(result.ok, result.parsed);
+  console.log(result.parsed);
+} catch (error) {
+  console.error("Automation failed:", error);
+  process.exitCode = 1;
 } finally {
   await browser.close();
 }

package/examples/desktop-local.js

@@ -17,7 +17,7 @@ const desktop = automify.computer({
 });
 
 const instruction =
-  "Open the Calendar app installed on this computer, find the next event after today, and summarize it. Do not create or edit events.";
+  "Open the Calendar app installed on this computer, find the next event, and summarize it. Do not create or edit events.";
 
 await desktop.do(instruction, {
   screenshots: {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "automify",
-  "version": "0.1.10",
+  "version": "0.2.0",
   "description": "AI computer use for browser, CLI, and desktop in Node.js.",
   "homepage": "https://aldovincenti.github.io/automify",
   "bugs": {

package/scripts/install-desktop.js

@@ -7,10 +7,12 @@ import { fileURLToPath } from "node:url";
 import {
   DESKTOP_RUNTIME_MANIFEST,
   desktopRuntimeDir,
+  desktopRuntimeIsInstalled,
   desktopRuntimeKey,
   desktopRuntimeManifest,
   desktopRuntimeNodeModules,
-  desktopRuntimeRefs
+  desktopRuntimeRefs,
+  resetDesktopRuntimeInstallState
 } from "../src/lib/desktop-runtime.js";
 
 const __dirname = dirname(fileURLToPath(import.meta.url));
@@ -29,10 +31,21 @@ const nutScope = join(nodeModules, "@nut-tree");
 const platformPackageName = `@nut-tree/libnut-${process.platform}`;
 const platformPackageDir = join(nutScope, `libnut-${process.platform}`);
 const macPermissionsPackageDir = join(nutScope, "node-mac-permissions");
+const forceInstall = process.argv.slice(2).some((arg) => arg === "--force" || arg === "force");
 
 const runtimeDependencies = ["jimp@1.6.1", "node-abort-controller@3.1.1", "clipboardy@2.3.0", "bindings@1.5.0"];
 
+if (!forceInstall && desktopRuntimeIsInstalled()) {
+  console.log("Automify desktop runtime is already installed and compatible; skipping rebuild.");
+  console.log(`Runtime directory: ${runtimeDir}`);
+  console.log("Use `npx automify-install-desktop --force` to rebuild it anyway.");
+  process.exit(0);
+}
+
 console.log("Building official nut.js from source.");
+if (forceInstall) {
+  console.log("Force enabled: rebuilding the desktop runtime even if the cache is compatible.");
+}
 console.log(`Build directory: ${buildRoot}`);
 console.log(`Runtime directory: ${runtimeDir}`);
 console.log(`Runtime key: ${desktopRuntimeKey()}`);
@@ -46,6 +59,7 @@ checkBuildPrerequisites();
 
 mkdirSync(buildRoot, { recursive: true });
 mkdirSync(runtimeDir, { recursive: true });
+resetDesktopRuntimeInstallState();
 writeRuntimePackageJson();
 cloneOrPull("https://github.com/nut-tree/libnut-core.git", libnutSource, refs.libnutCore);
 cloneOrPull("https://github.com/nut-tree/nut.js.git", nutSource, refs.nut);

package/src/index.d.ts

@@ -211,7 +211,8 @@ export interface LocalDesktopComputerOptions {
   };
   /**
    * Linux only. Defaults to true when DISPLAY is missing. Starts Xvfb so the
-   * local nut.js desktop adapter can run on headless servers.
+   * local nut.js desktop adapter can run on headless servers. Linux local
+   * desktop capture is X11-based; Wayland sessions are not supported.
    */
   virtualDisplay?:
     | boolean
@@ -224,6 +225,11 @@ export interface LocalDesktopComputerOptions {
         args?: string[];
         startupMs?: number;
       };
+  /**
+   * Linux only. Forces Xvfb even when DISPLAY is already set. Ignored on macOS
+   * and Windows. Use this when the host Linux session is Wayland or otherwise
+   * unsuitable for X11 screenshot capture.
+   */
   forceVirtualDisplay?: boolean;
   display?:
     | string

package/src/lib/argument-reference.js

@@ -14,7 +14,7 @@ export const argumentReference = [
       "command"
     ],
     notes:
-      "Use data for structured JSON, evaluate for files the model should inspect directly, and command only on CLI surfaces."
+      "Use data for structured JSON, evaluate for files the model should inspect directly, limits.steps to change the max model-action turns, and command only on CLI surfaces."
   },
   {
     surface: "automify.browser()",
@@ -72,13 +72,13 @@ export const argumentReference = [
     surface: "automify.localComputer()",
     preferred: ["viewport", "mouse", "keyboard", "calibration", "virtualDisplay", "logFile"],
     notes:
-      "Creates a local desktop runner and takes an exclusive cross-process lock until close(). Use logFile to capture automation and local desktop events."
+      "Creates a local desktop runner and takes an exclusive cross-process lock until close(). Linux local desktop requires X11/Xorg or Xvfb; Wayland is not supported. Use logFile to capture automation and local desktop events."
   },
   {
     surface: "createLocalDesktopComputer()",
     preferred: ["viewport", "mouse", "keyboard", "calibration", "virtualDisplay", "logFile"],
     notes:
-      "Grouped mouse, keyboard, and calibration options are preferred over the older flat names. Use logFile to capture local desktop events. Local desktop control takes an exclusive cross-process lock until close()."
+      "Grouped mouse, keyboard, and calibration options are preferred over the older flat names. Linux local desktop requires X11/Xorg or Xvfb; Wayland is not supported. Use logFile to capture local desktop events. Local desktop control takes an exclusive cross-process lock until close()."
   },
   {
     surface: "createDockerDesktopComputer()",

package/src/lib/automify.js

@@ -17,7 +17,7 @@ import {
   writeDebugLogFile
 } from "./runtime.js";
 
-const DEFAULT_MAX_STEPS = 1000;
+const DEFAULT_MAX_STEPS = 100;
 const DEFAULT_SCREENSHOT_DETAIL = "auto";
 const DEFAULT_SCREENSHOT_MAX_WIDTH = 1440;
 const DEFAULT_SCREENSHOT_MAX_HEIGHT = 1440;

package/src/lib/cli-automify.js

@@ -19,7 +19,7 @@ import {
   writeDebugLogFile
 } from "./runtime.js";
 
-const DEFAULT_MAX_STEPS = 1000;
+const DEFAULT_MAX_STEPS = 100;
 const DEFAULT_TIMEOUT_MS = 30_000;
 const CLI_OPTION_KEYS = mergeOptionKeys(AUTOMIFY_OPTION_KEYS, [
   "command",

package/src/lib/desktop-runtime.js

@@ -1,4 +1,4 @@
-import { existsSync, readdirSync, readFileSync } from "node:fs";
+import { existsSync, readdirSync, readFileSync, rmSync } from "node:fs";
 import { homedir } from "node:os";
 import { join, resolve } from "node:path";
 
@@ -79,6 +79,13 @@ export function desktopRuntimeManifest(env = process.env) {
   };
 }
 
+export function resetDesktopRuntimeInstallState(env = process.env) {
+  const runtimeDir = desktopRuntimeDir(env);
+  rmSync(join(desktopRuntimeNodeModules(env), "@nut-tree"), { recursive: true, force: true });
+  rmSync(join(runtimeDir, "package-lock.json"), { recursive: true, force: true });
+  rmSync(join(runtimeDir, "npm-shrinkwrap.json"), { recursive: true, force: true });
+}
+
 export function readDesktopRuntimeManifest(env = process.env) {
   const path = desktopRuntimeManifestPath(env);
   if (!existsSync(path)) return null;

package/src/lib/local-desktop-computer.js

@@ -136,6 +136,7 @@ const DEFAULT_DESKTOP_INSTRUCTIONS = [
   "You are controlling a native desktop through screenshots and mouse/keyboard actions.",
   "Orient from the screenshot first: identify the active app, visible window, focused field, current page, and the specific target required by the task before acting.",
   "Use deterministic entry points. For a website or web app, use the browser address bar only when a browser is clearly focused; otherwise open the browser through a visible Dock/app icon or OS search/launcher, then use the address bar. For app content, use visible in-app search, filters, or navigation controls.",
+  "Before using a terminal to run a shell command, confirm the terminal is idle and showing a prompt. If the visible terminal is occupied by a running foreground process, such as the Node.js script that started this session, do not type commands into it; use the OS launcher/Dock/search or a separate idle terminal session instead.",
   "Do not open or use Command+Tab, Alt+Tab, Mission Control, or other cyclic app/window switchers unless the task explicitly asks to switch to the previous app. Cyclic switching is unreliable because the open-app order is unknown.",
   "Do not click as a probe. Click only when the screenshot shows a specific visible target and the purpose of that click is clear from the task or current UI. Prefer named controls, fields, menu items, visible app icons, and address/search fields over unlabeled areas.",
   "If the target is not visible, choose a deterministic recovery path: direct URL, OS search/launcher, in-app search, visible navigation, or a screenshot/wait when loading is visible. Do not repeat nearly identical clicks after no visible change.",