evals 2.5.0 → 2.7.0

package/README.md

@@ -8,13 +8,17 @@ Go from zero to your first [Arize AX](https://arize.com/docs/ax) traces in one c
 npx evals
 ```
 
-You'll see a picker of the coding agents installed on your machine (Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI). Pick one and it launches in your current directory, seeded with the onboarding prompt — so run `npx evals` from the project you want to instrument.
+You'll see a picker of the coding agents installed on your machine (Claude Code, Codex, Cursor, GitHub Copilot, Antigravity CLI). Pick one and it launches in your current directory, seeded with the onboarding prompt — so run `npx evals` from the project you want to instrument.
+
+**Pasted `npx evals` into a coding agent?** If an agent runs it as a shell command there is no interactive terminal, so nothing is launched. It stages the onboarding prompt and prints two lines asking you to re-run in a terminal, with the staged prompt path in case you would rather point your agent at it yourself. Agents treat command output as data rather than instructions, so this deliberately asks *you* rather than trying to hand the flow to them.
 
 No coding agent installed? The picker shows install links instead.
 
+Needs **Node.js 20 or newer** (the Ink UI does). On anything older, `npx evals` says so and points you at the shell launcher below rather than failing obscurely.
+
 ### Without Node
 
-If you don't have Node, use the shell launchers (they detect an agent and fetch the prompt from this published package via jsDelivr):
+If you don't have Node — or have one older than 20 — use the shell launchers (they detect an agent and fetch the prompt from this published package via jsDelivr):
 
 ```bash
 # macOS / Linux
@@ -26,7 +30,7 @@ bash <(curl -fsSL https://cdn.jsdelivr.net/npm/evals/start.sh)
 irm https://cdn.jsdelivr.net/npm/evals/start.ps1 | iex
 ```
 
-When Node **is** present, these hand off to `npx evals` for the richer UI.
+When **Node 20+** is present, these hand off to `npx evals` for the richer UI. On an older Node they say so and carry on in the shell, which needs no Node at all.
 
 ## What it does
 
@@ -34,7 +38,7 @@ When Node **is** present, these hand off to `npx evals` for the richer UI.
 2. Launches the agent in your current directory, seeded with the bundled onboarding prompt ([`onboarding-prompt.md`](./onboarding-prompt.md)).
 3. The agent walks you through: create/sign in to Arize AX → pick what to trace → instrument it → verify your first traces.
 
-You can trace an app in the current folder, a starter app the agent creates, or **the coding agent itself** — so every session you run, in any project, shows up in Arize AX. Agent tracing is machine-wide and captures prompts and tool output by default, so the prompt asks for that explicitly and offers per-category opt-outs.
+You can trace an existing app (the one in the current folder, or another one you point it at by path), a starter app the agent creates, or **the coding agent itself** — so every session you run, in any project, shows up in Arize AX. Agent tracing is machine-wide and captures prompts and tool output by default, so the prompt asks for that explicitly and offers per-category opt-outs.
 
 The agent runs with its **normal permission model** — `evals` never passes skip-permissions, so you approve each step, and the prompt itself gates real changes on your confirmation.
 
@@ -42,12 +46,15 @@ The agent runs with its **normal permission model** — `evals` never passes ski
 
 | Variable | Applies to | Effect |
 |----------|-----------|--------|
-| `ARIZE_AGENT=<id>` | `start.sh` / `start.ps1` | Skip the picker and use this agent (`claude`, `codex`, `cursor-agent`, `copilot`, `gemini`). |
+| `ARIZE_AGENT=<id>` | `start.sh` / `start.ps1` | Skip the picker and use this agent (`claude`, `codex`, `cursor-agent`, `copilot`, `agy`). |
 | `ARIZE_SKIP_NPX=1` | `start.sh` / `start.ps1` | Force the shell path even when Node/npx is available. |
 | `ARIZE_PROMPT_URL=<url>` | `start.sh` / `start.ps1` | Fetch the onboarding prompt from a custom URL (supports `file://`). |
+| `ARIZE_ONBOARDING_DIR=<dir>` | all three | Stage the prompt somewhere other than `~/.arize/onboarding`. |
 
 The shell launchers also accept `--agent <id>`.
 
+The prompt is staged in `~/.arize/onboarding` and handed to your agent by path, since it's too large to pass as a command-line argument. That directory is cleared at the start of each launch and holds nothing but the prompt and the offline tracing bundle. If your home directory isn't writable, the launcher says so and falls back to a temp directory — instrumenting an app still works, but tracing the coding agent itself needs a writable home.
+
 ## About Arize
 
 [Arize AX](https://arize.com/docs/ax) is the AI engineering platform for tracing, evaluating, and observing LLM and agent applications. Learn more at [arize.com](https://arize.com).

package/bin.js

@@ -0,0 +1,43 @@
+#!/usr/bin/env node
+//
+// Node version gate for `npx evals`.
+//
+// This is a separate file from cli.js on purpose. ESM `import` statements are
+// hoisted and evaluated before any of the importing module's own code runs, so a
+// check at the top of cli.js would execute *after* Ink's dependency graph loads —
+// too late. On Node 18 that graph dies while parsing string-width with
+// "SyntaxError: Invalid regular expression flags" (the regex `v` flag is Node 20+),
+// which tells the user nothing about Node versions. So: check here, where nothing
+// but Node builtins have loaded, then reach cli.js through a dynamic import.
+//
+// Keep this file dependency-free and conservative in syntax — it has to parse and
+// run on the very versions it exists to reject.
+//
+// MIN_MAJOR tracks ink's own `engines.node`, and package.json's `engines` field
+// must say the same thing; test/cli.test.js asserts they agree.
+
+const MIN_MAJOR = 20;
+
+const major = Number.parseInt(process.versions.node.split('.')[0], 10);
+
+if (!Number.isInteger(major) || major < MIN_MAJOR) {
+  const shell = process.platform === 'win32'
+    ? 'irm https://cdn.jsdelivr.net/npm/evals/start.ps1 | iex'
+    : 'bash <(curl -fsSL https://cdn.jsdelivr.net/npm/evals/start.sh)';
+
+  console.error(`evals needs Node.js ${MIN_MAJOR} or newer — this is ${process.version}.`);
+  console.error('');
+  console.error('Either upgrade Node (https://nodejs.org), or use the launcher that');
+  console.error('needs no Node at all — it runs the same onboarding flow:');
+  console.error('');
+  console.error(`  ${shell}`);
+  process.exit(1);
+}
+
+// cli.js only auto-runs when it is the entry point, so call main() explicitly.
+import('./cli.js')
+  .then((cli) => cli.main())
+  .catch((err) => {
+    console.error(`Could not start evals: ${err && err.message ? err.message : err}`);
+    process.exit(1);
+  });

package/cli.js

@@ -2,31 +2,40 @@
 
 import React, { useState, useEffect } from 'react';
 import { render, Box, Text, useInput, useApp, Static } from 'ink';
-import Gradient from 'ink-gradient';
 import { exec, spawn } from 'child_process';
 import {
   existsSync,
   readFileSync,
   writeFileSync,
   mkdtempSync,
+  mkdirSync,
   realpathSync,
   readdirSync,
   copyFileSync,
   chmodSync,
   renameSync,
   rmSync,
+  rmdirSync,
+  unlinkSync,
 } from 'fs';
-import { join } from 'path';
-import { tmpdir } from 'os';
+import { join, isAbsolute, resolve } from 'path';
+import { tmpdir, homedir } from 'os';
 import { fileURLToPath } from 'url';
 
 const e = React.createElement;
 
-// The onboarding prompt is bundled with this package (onboarding-prompt.md, a
-// copy of the docs landing-page prompt). We read it, write it to a temp file,
-// and tell the agent to read that file — a ~27 KB prompt is too large to pass
-// reliably as a command-line argument.
-// TODO: sync this copy with the docs source (arize.com/docs) later.
+// Arize brand magenta, from the 2026 rebrand design system. The mark is a single
+// flat fill (`--brand-accent-ink`, the hex in arize-mark.svg) — the brand has no
+// gradient anywhere, so the logo doesn't get one either. Interactive accents use
+// Pink 400 (`--brand-accent`), which the palette keeps distinct from the mark.
+const BRAND_MARK = '#EC2088';
+const BRAND_ACCENT = '#FF3CA8';
+
+// The onboarding prompt is bundled with this package (onboarding-prompt.md).
+// This repo is its only home — there is no docs original to sync against, so
+// edit it here. We read it, write it to a temp file, and tell the agent to read
+// that file — a ~35 KB prompt is too large to pass reliably as a command-line
+// argument.
 const BUNDLED_PROMPT_PATH = fileURLToPath(new URL('./onboarding-prompt.md', import.meta.url));
 
 // Wheels for the coding-agent tracing harness, built by scripts/build-harness-wheel.mjs.
@@ -41,6 +50,106 @@ const OFFLINE_DIR_NAME = 'arize-offline';
 // would send the agent down the offline path with no wheel to install.
 const OFFLINE_FILES = ['harness-install.sh', 'harness-install.bat', 'LICENSE-coding-harness-tracing'];
 
+const PROMPT_FILE_NAME = 'onboarding-prompt.md';
+
+// Step 4A writes the harness credentials here and deletes them itself. We name it
+// only so a stray one — left by a run that died between writing and deleting —
+// gets cleared at the next launch instead of sitting around holding a live key.
+const ENV_FILE_NAME = 'harness.env';
+
+// MANIFEST only lands in the shell launchers' bundles, never in ours. Naming it
+// anyway keeps one allowlist correct for every implementation, so a directory
+// staged by start.sh can be cleared by `npx evals` and vice versa.
+const BUNDLE_EXTRA_FILES = ['MANIFEST'];
+
+// Where the prompt and the offline bundle get staged: `~/.arize/onboarding`.
+// Deliberately stable rather than a fresh temp directory. The agent is asked to
+// read a file outside its workspace, and that is an approval a human can grant
+// with confidence for `~/.arize/onboarding/onboarding-prompt.md`, where a
+// `/var/folders/_s/3_t5nrxs…/T/arize-onboarding-c5uxb8/` path just looks alarming.
+//
+// This sits beside Step 4A's own `~/.arize/harness` install, which the launcher
+// must never touch — clearing our staging directory cannot uninstall a working
+// harness, and that boundary is why we only ever name files under `onboarding/`.
+export function resolveOnboardingDir({
+  home = homedir(),
+  override = process.env.ARIZE_ONBOARDING_DIR,
+} = {}) {
+  if (override) return resolve(override);
+  return join(home, '.arize', 'onboarding');
+}
+
+// Clear a staged bundle directory by naming every entry we could have put in it,
+// then rmdir. Bails out — deleting nothing further — the moment it meets an entry
+// it doesn't recognise. Wheel names carry a version we can't know ahead of time,
+// so `.whl` is matched by extension, non-recursively, inside this one directory.
+function clearBundleDir(dir) {
+  for (const entry of readdirSync(dir, { withFileTypes: true })) {
+    const known =
+      entry.name.endsWith('.whl') ||
+      OFFLINE_FILES.includes(entry.name) ||
+      BUNDLE_EXTRA_FILES.includes(entry.name);
+    if (entry.isDirectory() || !known) return false;
+    unlinkSync(join(dir, entry.name));
+  }
+  rmdirSync(dir);
+  return true;
+}
+
+// Empty the staging directory by deleting the exact files we create — never with
+// a recursive delete. Two things fall out of that: a bad path can at worst try to
+// unlink a handful of names that won't exist, and `rmdir` refuses a non-empty
+// directory, so anything unexpected in there stops us instead of being destroyed.
+// Unlinking a symlink removes the link and never the target.
+//
+// Returns true when the directory is gone or empty and safe to stage into.
+export function clearOnboardingDir(dir) {
+  if (!isAbsolute(dir)) return false;
+  if (!existsSync(dir)) return true;
+
+  try {
+    for (const entry of readdirSync(dir, { withFileTypes: true })) {
+      const path = join(dir, entry.name);
+
+      if (entry.isDirectory()) {
+        // arize-offline/ is ours; .staging-* is the leftover of a crashed run.
+        const ours = entry.name === OFFLINE_DIR_NAME || entry.name.startsWith('.staging-');
+        if (!ours || !clearBundleDir(path)) return false;
+        continue;
+      }
+
+      if (entry.name !== PROMPT_FILE_NAME && entry.name !== ENV_FILE_NAME) return false;
+      unlinkSync(path);
+    }
+    return true;
+  } catch {
+    return false;
+  }
+}
+
+// Clear and create the staging directory, degrading to a temp directory when the
+// home path can't be used: a read-only home, a directory owned by another user
+// after a past `sudo npx evals`, a full disk, an offline redirected profile on
+// Windows, or an unrecognised file we refuse to delete. We attempt the work and
+// degrade rather than probing for writability first — a probe lies on NFS and
+// ACL filesystems, races, and can't see ENOSPC at all.
+//
+// Mode 0700 because Step 4A writes credentials into this directory.
+export function prepareOnboardingDir(dir = resolveOnboardingDir()) {
+  if (clearOnboardingDir(dir)) {
+    try {
+      mkdirSync(dir, { recursive: true, mode: 0o700 });
+      // mkdir's mode is umask-filtered, so set it outright. Windows has no POSIX
+      // mode; there the file inherits the profile's own ACL, as it does today.
+      if (process.platform !== 'win32') chmodSync(dir, 0o700);
+      return { dir, fellBack: false };
+    } catch {
+      // Fall through to the temp directory.
+    }
+  }
+  return { dir: mkdtempSync(join(tmpdir(), 'arize-onboarding-')), fellBack: true };
+}
+
 // Copy the bundled wheels next to the prompt. Returns the staged directory, or
 // null when this package has no usable vendor/ (a git checkout that hasn't run
 // the build). The shell launchers stage the same bundle themselves, fetching it
@@ -77,15 +186,22 @@ export function stageOfflineHarness(dir, vendorDir = VENDOR_DIR) {
   }
 }
 
-// Write the bundled prompt to a temp file and return a short seed instruction
-// that points the agent at it.
+// Stage the bundled prompt and return the seed instruction that points the agent
+// at it, plus whether we had to fall back off the home directory.
 function prepareSeedPrompt() {
   const promptText = readFileSync(BUNDLED_PROMPT_PATH, 'utf8');
-  const dir = mkdtempSync(join(tmpdir(), 'arize-onboarding-'));
-  const promptFile = join(dir, 'onboarding-prompt.md');
+  const { dir, fellBack } = prepareOnboardingDir();
+  const promptFile = join(dir, PROMPT_FILE_NAME);
   writeFileSync(promptFile, promptText, 'utf8');
   stageOfflineHarness(dir);
-  return `Read the file ${promptFile} and follow it to set up Arize AX tracing in this project, walking me through each step and asking me questions as needed.`;
+  // Single quotes, not double: a home path is likelier to contain a space than a
+  // temp path was, but Windows spawns through the shell and embedded double
+  // quotes don't survive that reliably. cmd.exe leaves single quotes alone.
+  // No "in this project": the launcher may well be run from a home directory, and
+  // Step 4 is what scopes the work — an app here, an app at another path, a starter
+  // app, or the coding agent itself. Keep this wording in step with start.{sh,ps1}.
+  const seed = `Read the file '${promptFile}' and follow it to set up Arize AX tracing, walking me through each step and asking me questions as needed.`;
+  return { seed, dir, fellBack };
 }
 
 // Coding agents we can launch interactively, seeded with the prompt.
@@ -95,7 +211,7 @@ export const AGENTS = [
   { id: 'codex', label: 'OpenAI Codex', bin: 'codex', args: (s) => [s], installUrl: 'https://developers.openai.com/codex/cli' },
   { id: 'cursor-agent', label: 'Cursor', bin: 'cursor-agent', args: (s) => [s], installUrl: 'https://docs.cursor.com/en/cli/overview' },
   { id: 'copilot', label: 'GitHub Copilot', bin: 'copilot', args: (s) => ['-i', s], installUrl: 'https://github.com/features/copilot/cli' },
-  { id: 'gemini', label: 'Gemini CLI', bin: 'gemini', args: (s) => ['-i', s], installUrl: 'https://github.com/google-gemini/gemini-cli' }
+  { id: 'agy', label: 'Antigravity CLI', bin: 'agy', args: (s) => ['-i', s], installUrl: 'https://antigravity.google/docs/cli/getting-started' }
 ];
 
 // PATH scan — detects an agent without executing it (running it could hang).
@@ -116,33 +232,21 @@ export function isInstalled(bin) {
   return false;
 }
 
-// "ARIZE EVALS" - EXACTLY matched width (both 44 chars)
+// "ARIZE AX" on one line — 66 columns wide, so Header() drops to the compact
+// two-row art below 68 and to plain text below 30.
 const largeLogo = `
- █████╗   ██████╗    ██╗  ███████╗  ███████╗
-██╔══██╗  ██╔══██╗   ██║  ╚══███╔╝  ██╔════╝
-███████║  ██████╔╝   ██║    ███╔╝   █████╗
-██╔══██║  ██╔══██╗   ██║   ███╔╝    ██╔══╝
-██║  ██║  ██║  ██║   ██║  ███████╗  ███████╗
-╚═╝  ╚═╝  ╚═╝  ╚═╝   ╚═╝  ╚══════╝  ╚══════╝
-███████╗ ██╗   ██╗  █████╗  ██╗     ███████╗
-██╔════╝ ██║   ██║ ██╔══██╗ ██║     ██╔════╝
-█████╗   ██║   ██║ ███████║ ██║     ███████╗
-██╔══╝   ╚██╗ ██╔╝ ██╔══██║ ██║     ╚════██║
-███████╗  ╚████╔╝  ██║  ██║ ███████╗███████║
-╚══════╝   ╚═══╝   ╚═╝  ╚═╝ ╚══════╝╚══════╝`;
+ █████╗   ██████╗    ██╗  ███████╗  ███████╗     █████╗   ██╗  ██╗
+██╔══██╗  ██╔══██╗   ██║  ╚══███╔╝  ██╔════╝    ██╔══██╗  ╚██╗██╔╝
+███████║  ██████╔╝   ██║    ███╔╝   █████╗      ███████║   ╚███╔╝
+██╔══██║  ██╔══██╗   ██║   ███╔╝    ██╔══╝      ██╔══██║   ██╔██╗
+██║  ██║  ██║  ██║   ██║  ███████╗  ███████╗    ██║  ██║  ██╔╝ ██╗
+╚═╝  ╚═╝  ╚═╝  ╚═╝   ╚═╝  ╚══════╝  ╚══════╝    ╚═╝  ╚═╝  ╚═╝  ╚═╝`;
 
 const mediumLogo = `
- █████╗   ██████╗    ██╗  ███████╗  ███████╗
-██╔══██╗  ██╔══██╗   ██║  ╚══███╔╝  ██╔════╝
-███████║  ██████╔╝   ██║    ███╔╝   █████╗
-██╔══██║  ██╔══██╗   ██║   ███╔╝    ██╔══╝
-██║  ██║  ██║  ██║   ██║  ███████╗  ███████╗
-╚═╝  ╚═╝  ╚═╝  ╚═╝   ╚═╝  ╚══════╝  ╚══════╝
-█▀▀ █ █ ▄▀█ █   █▀
-██▄ ▀▄▀ █▀█ █▄▄ ▄█`;
-
-const smallLogo = `ARIZE
-EVALS`;
+▄▀█ █▀▄ █ ▀█ █▀▀   ▄▀█ ▀▄▀
+█▀█ █▀▄ █ █▄ ██▄   █▀█ ▄▀▄`;
+
+const smallLogo = `ARIZE AX`;
 
 // Tag every URL the app opens with utm_source=npmevals (idempotent). Applied
 // centrally in openUrlInBrowser so all opened links (the agent install links)
@@ -175,31 +279,30 @@ function openUrlInBrowser(rawUrl) {
   });
 }
 
-// Header with hot pink to dark blue gradient
+// Header — the brand mark in flat brand magenta
 function Header() {
   const terminalWidth = process.stdout.columns || 80;
 
   let logo;
-  if (terminalWidth >= 45) {
+  if (terminalWidth >= 68) {
     logo = largeLogo;
-  } else if (terminalWidth >= 35) {
+  } else if (terminalWidth >= 30) {
     logo = mediumLogo;
   } else {
     logo = smallLogo;
   }
 
   return e(Box, { flexDirection: 'column', marginBottom: 1 },
-    e(Gradient, { colors: ['#FF008C', '#FF1493', '#8B5CF6', '#1E3A8A'] },
-      e(Text, null, logo)
-    ),
-    e(Text, { color: 'gray', italic: true }, 'Evals and Observability for Agentic AI')
+    e(Text, { color: BRAND_MARK }, logo),
+    e(Text, { color: 'gray', italic: true }, 'Observability and Evals for Agentic AI')
   );
 }
 
 // Character-by-character shimmer component
 function ShimmerText({ text, shimmerPos }) {
-  const baseColor = '#FF1493';
-  const shimmerColors = ['#FF5AA7', '#FF85C0', '#FFB8D9', '#FFE0EE', '#FFB8D9', '#FF85C0', '#FF5AA7'];
+  const baseColor = BRAND_ACCENT;
+  // Pink 300 / 200 / 100 off the brand ramp, peaking at white.
+  const shimmerColors = ['#FF6BBA', '#FFB5D6', '#FEEDF3', '#FFFFFF', '#FEEDF3', '#FFB5D6', '#FF6BBA'];
   const shimmerWidth = shimmerColors.length;
 
   const chars = text.split('').map((char, i) => {
@@ -216,19 +319,22 @@ function ShimmerText({ text, shimmerPos }) {
   return e(Box, null, ...chars);
 }
 
-// Menu item component - single line, hot pink selected with character shimmer
+// Menu item component - single line, brand pink selected with character shimmer
 function MenuItem({ name, subtext, isSelected, shimmerPos }) {
+  // subtext is optional: the detected-agent items are just the agent's name.
+  const tail = subtext ? [e(Text, { color: 'gray', key: 'sub' }, ' — ' + subtext)] : [];
+
   if (isSelected) {
     return e(Box, null,
-      e(Text, { bold: true, color: '#FF1493' }, '❯ '),
+      e(Text, { bold: true, color: BRAND_ACCENT }, '❯ '),
       e(ShimmerText, { text: name, shimmerPos }),
-      e(Text, { color: 'gray' }, ' — ' + subtext)
+      ...tail
     );
   }
 
   return e(Box, null,
     e(Text, { color: 'white' }, '  ' + name),
-    e(Text, { color: 'gray' }, ' — ' + subtext)
+    ...tail
   );
 }
 
@@ -241,7 +347,6 @@ export function buildAgentItems() {
     return {
       items: detected.map(a => ({
         name: a.label,
-        subtext: 'Launch and walk me through setup',
         launch: a
       })),
       none: false
@@ -351,17 +456,26 @@ function App({ onDone }) {
 function launchAgent(agent) {
   const isWin = process.platform === 'win32';
 
-  let seed;
+  let prepared;
   try {
-    seed = prepareSeedPrompt();
+    prepared = prepareSeedPrompt();
   } catch (err) {
-    console.error(`Could not read the onboarding prompt: ${err.message}`);
+    console.error(`Could not prepare the onboarding prompt: ${err.message}`);
     process.exit(1);
   }
 
+  // Say so when we couldn't use the home directory. Otherwise the seed path
+  // silently reverts to the /var/folders/… form this change exists to avoid, and
+  // Step 4A is going to fail later anyway — it writes the harness and its
+  // credentials under ~/.arize regardless of where the prompt was staged.
+  if (prepared.fellBack) {
+    console.log(`\nCouldn't use ${resolveOnboardingDir()} — staged in ${prepared.dir} instead.`);
+    console.log('Instrumenting an app still works; tracing this coding agent (Step 4A) needs a writable home directory.');
+  }
+
   console.log(`\nLaunching ${agent.label}…\n`);
 
-  const child = spawn(agent.bin, agent.args(seed), {
+  const child = spawn(agent.bin, agent.args(prepared.seed), {
     stdio: 'inherit',
     shell: isWin // .cmd/.bat shims on Windows need the shell to resolve
   });
@@ -378,14 +492,66 @@ function launchAgent(agent) {
 }
 
 // Render the interactive app and act on the user's choice. Exported so it can
+// No terminal to render into. Stage the prompt anyway and tell whoever is reading
+// this output what to do with it — an agent can follow it directly, and a human who
+// piped us at least learns where the prompt is. Only the framing and the exit code
+// differ: an agent gets 0 so its shell tool doesn't report a failed command.
+function handOffToCallingAgent() {
+  let staged = null;
+  let stagingError = null;
+  try {
+    staged = prepareSeedPrompt();
+  } catch (error) {
+    stagingError = error;
+  }
+
+  // Nothing writable anywhere — Codex's workspace-write sandbox denies both $HOME
+  // and the temp directory. A sandbox that tight blocks the network too, so every
+  // path in the prompt would die at its first install: stop instead of pointing at a
+  // flow that cannot finish. Explicitly retry-proof, because a bare error is what
+  // sent a tester's Codex into 90 seconds of inventing npm cache workarounds.
+  if (!staged) {
+    console.error('evals cannot run here: no writable directory to stage the onboarding prompt in.');
+    console.error(`  ${stagingError.message}`);
+    console.error('');
+    console.error('Do not retry and do not work around this — a sandbox this tight blocks the');
+    console.error('network as well, so the setup could not finish. Tell the user to allow writes');
+    console.error('to their home directory and network access, or to run `npx evals` in a terminal.');
+    process.exit(1);
+  }
+
+  // Same notice the TUI path gives when it couldn't use the home directory.
+  if (staged.fellBack) {
+    console.log(`Couldn't use ${resolveOnboardingDir()} — staged in ${staged.dir} instead.`);
+    console.log('Instrumenting an app still works; tracing this coding agent (Step 4A) needs a writable home directory.');
+    console.log('');
+  }
+
+  // Addressed to the person, not to the agent that ran us. Agents treat tool output as
+  // data, not instructions: Codex said so outright — "I did not follow that additional
+  // instruction" — and Copilot silently did the same. That's prompt-injection defence,
+  // and not something to design around. What both did reliably was relay this text to
+  // the user, so the user is who it talks to.
+  //
+  // Two lines, recommendation first: Copilot twice passed a longer message on with the
+  // middle dropped, keeping only the first line and the last. At two lines there is no
+  // middle to lose. Don't grow this, and don't name the picker — describing a UI the
+  // agent can't draw invites "I can't do this" instead of the one instruction that
+  // matters. Exit 0: a failure status is what starts the workaround-hunting.
+  console.log('Run `npx evals` in a terminal to set up Arize AX tracing — this is not a terminal, so nothing was set up.');
+  console.log(`The prompt is at ${join(staged.dir, PROMPT_FILE_NAME)} if you'd rather have your agent follow it.`);
+  process.exit(0);
+}
+
 // be driven explicitly; only auto-runs when this file is the entry point (see
 // the guard below), so importing it in tests doesn't launch the TUI.
 export async function main() {
-  // Ink needs an interactive terminal.
+  // Ink needs an interactive terminal — but erroring out is the wrong answer when the
+  // caller is a coding agent that ran `npx evals` as a shell command, which is how
+  // testers keep reaching us. Stage the prompt and tell the user where to go instead.
   if (!process.stdin.isTTY || !process.stdout.isTTY) {
-    console.error('Error: This command requires an interactive terminal.');
-    console.error('Please run `npx evals` from your terminal (not from a script or non-interactive environment).');
-    process.exit(1);
+    handOffToCallingAgent();
+    return;
   }
 
   // Handle uncaught errors