evals 2.5.0 → 2.6.0

package/README.md

@@ -12,9 +12,11 @@ You'll see a picker of the coding agents installed on your machine (Claude Code,
 
 No coding agent installed? The picker shows install links instead.
 
+Needs **Node.js 20 or newer** (the Ink UI does). On anything older, `npx evals` says so and points you at the shell launcher below rather than failing obscurely.
+
 ### Without Node
 
-If you don't have Node, use the shell launchers (they detect an agent and fetch the prompt from this published package via jsDelivr):
+If you don't have Node — or have one older than 20 — use the shell launchers (they detect an agent and fetch the prompt from this published package via jsDelivr):
 
 ```bash
 # macOS / Linux
@@ -26,7 +28,7 @@ bash <(curl -fsSL https://cdn.jsdelivr.net/npm/evals/start.sh)
 irm https://cdn.jsdelivr.net/npm/evals/start.ps1 | iex
 ```
 
-When Node **is** present, these hand off to `npx evals` for the richer UI.
+When **Node 20+** is present, these hand off to `npx evals` for the richer UI. On an older Node they say so and carry on in the shell, which needs no Node at all.
 
 ## What it does
 
@@ -34,7 +36,7 @@ When Node **is** present, these hand off to `npx evals` for the richer UI.
 2. Launches the agent in your current directory, seeded with the bundled onboarding prompt ([`onboarding-prompt.md`](./onboarding-prompt.md)).
 3. The agent walks you through: create/sign in to Arize AX → pick what to trace → instrument it → verify your first traces.
 
-You can trace an app in the current folder, a starter app the agent creates, or **the coding agent itself** — so every session you run, in any project, shows up in Arize AX. Agent tracing is machine-wide and captures prompts and tool output by default, so the prompt asks for that explicitly and offers per-category opt-outs.
+You can trace an existing app (the one in the current folder, or another one you point it at by path), a starter app the agent creates, or **the coding agent itself** — so every session you run, in any project, shows up in Arize AX. Agent tracing is machine-wide and captures prompts and tool output by default, so the prompt asks for that explicitly and offers per-category opt-outs.
 
 The agent runs with its **normal permission model** — `evals` never passes skip-permissions, so you approve each step, and the prompt itself gates real changes on your confirmation.
 
@@ -45,9 +47,12 @@ The agent runs with its **normal permission model** — `evals` never passes ski
 | `ARIZE_AGENT=<id>` | `start.sh` / `start.ps1` | Skip the picker and use this agent (`claude`, `codex`, `cursor-agent`, `copilot`, `gemini`). |
 | `ARIZE_SKIP_NPX=1` | `start.sh` / `start.ps1` | Force the shell path even when Node/npx is available. |
 | `ARIZE_PROMPT_URL=<url>` | `start.sh` / `start.ps1` | Fetch the onboarding prompt from a custom URL (supports `file://`). |
+| `ARIZE_ONBOARDING_DIR=<dir>` | all three | Stage the prompt somewhere other than `~/.arize/onboarding`. |
 
 The shell launchers also accept `--agent <id>`.
 
+The prompt is staged in `~/.arize/onboarding` and handed to your agent by path, since it's too large to pass as a command-line argument. That directory is cleared at the start of each launch and holds nothing but the prompt and the offline tracing bundle. If your home directory isn't writable, the launcher says so and falls back to a temp directory — instrumenting an app still works, but tracing the coding agent itself needs a writable home.
+
 ## About Arize
 
 [Arize AX](https://arize.com/docs/ax) is the AI engineering platform for tracing, evaluating, and observing LLM and agent applications. Learn more at [arize.com](https://arize.com).

package/bin.js

@@ -0,0 +1,43 @@
+#!/usr/bin/env node
+//
+// Node version gate for `npx evals`.
+//
+// This is a separate file from cli.js on purpose. ESM `import` statements are
+// hoisted and evaluated before any of the importing module's own code runs, so a
+// check at the top of cli.js would execute *after* Ink's dependency graph loads —
+// too late. On Node 18 that graph dies while parsing string-width with
+// "SyntaxError: Invalid regular expression flags" (the regex `v` flag is Node 20+),
+// which tells the user nothing about Node versions. So: check here, where nothing
+// but Node builtins have loaded, then reach cli.js through a dynamic import.
+//
+// Keep this file dependency-free and conservative in syntax — it has to parse and
+// run on the very versions it exists to reject.
+//
+// MIN_MAJOR tracks ink's own `engines.node`, and package.json's `engines` field
+// must say the same thing; test/cli.test.js asserts they agree.
+
+const MIN_MAJOR = 20;
+
+const major = Number.parseInt(process.versions.node.split('.')[0], 10);
+
+if (!Number.isInteger(major) || major < MIN_MAJOR) {
+  const shell = process.platform === 'win32'
+    ? 'irm https://cdn.jsdelivr.net/npm/evals/start.ps1 | iex'
+    : 'bash <(curl -fsSL https://cdn.jsdelivr.net/npm/evals/start.sh)';
+
+  console.error(`evals needs Node.js ${MIN_MAJOR} or newer — this is ${process.version}.`);
+  console.error('');
+  console.error('Either upgrade Node (https://nodejs.org), or use the launcher that');
+  console.error('needs no Node at all — it runs the same onboarding flow:');
+  console.error('');
+  console.error(`  ${shell}`);
+  process.exit(1);
+}
+
+// cli.js only auto-runs when it is the entry point, so call main() explicitly.
+import('./cli.js')
+  .then((cli) => cli.main())
+  .catch((err) => {
+    console.error(`Could not start evals: ${err && err.message ? err.message : err}`);
+    process.exit(1);
+  });

package/cli.js

@@ -9,24 +9,27 @@ import {
   readFileSync,
   writeFileSync,
   mkdtempSync,
+  mkdirSync,
   realpathSync,
   readdirSync,
   copyFileSync,
   chmodSync,
   renameSync,
   rmSync,
+  rmdirSync,
+  unlinkSync,
 } from 'fs';
-import { join } from 'path';
-import { tmpdir } from 'os';
+import { join, isAbsolute, resolve } from 'path';
+import { tmpdir, homedir } from 'os';
 import { fileURLToPath } from 'url';
 
 const e = React.createElement;
 
-// The onboarding prompt is bundled with this package (onboarding-prompt.md, a
-// copy of the docs landing-page prompt). We read it, write it to a temp file,
-// and tell the agent to read that file — a ~27 KB prompt is too large to pass
-// reliably as a command-line argument.
-// TODO: sync this copy with the docs source (arize.com/docs) later.
+// The onboarding prompt is bundled with this package (onboarding-prompt.md).
+// This repo is its only home — there is no docs original to sync against, so
+// edit it here. We read it, write it to a temp file, and tell the agent to read
+// that file — a ~35 KB prompt is too large to pass reliably as a command-line
+// argument.
 const BUNDLED_PROMPT_PATH = fileURLToPath(new URL('./onboarding-prompt.md', import.meta.url));
 
 // Wheels for the coding-agent tracing harness, built by scripts/build-harness-wheel.mjs.
@@ -41,6 +44,106 @@ const OFFLINE_DIR_NAME = 'arize-offline';
 // would send the agent down the offline path with no wheel to install.
 const OFFLINE_FILES = ['harness-install.sh', 'harness-install.bat', 'LICENSE-coding-harness-tracing'];
 
+const PROMPT_FILE_NAME = 'onboarding-prompt.md';
+
+// Step 4A writes the harness credentials here and deletes them itself. We name it
+// only so a stray one — left by a run that died between writing and deleting —
+// gets cleared at the next launch instead of sitting around holding a live key.
+const ENV_FILE_NAME = 'harness.env';
+
+// MANIFEST only lands in the shell launchers' bundles, never in ours. Naming it
+// anyway keeps one allowlist correct for every implementation, so a directory
+// staged by start.sh can be cleared by `npx evals` and vice versa.
+const BUNDLE_EXTRA_FILES = ['MANIFEST'];
+
+// Where the prompt and the offline bundle get staged: `~/.arize/onboarding`.
+// Deliberately stable rather than a fresh temp directory. The agent is asked to
+// read a file outside its workspace, and that is an approval a human can grant
+// with confidence for `~/.arize/onboarding/onboarding-prompt.md`, where a
+// `/var/folders/_s/3_t5nrxs…/T/arize-onboarding-c5uxb8/` path just looks alarming.
+//
+// This sits beside Step 4A's own `~/.arize/harness` install, which the launcher
+// must never touch — clearing our staging directory cannot uninstall a working
+// harness, and that boundary is why we only ever name files under `onboarding/`.
+export function resolveOnboardingDir({
+  home = homedir(),
+  override = process.env.ARIZE_ONBOARDING_DIR,
+} = {}) {
+  if (override) return resolve(override);
+  return join(home, '.arize', 'onboarding');
+}
+
+// Clear a staged bundle directory by naming every entry we could have put in it,
+// then rmdir. Bails out — deleting nothing further — the moment it meets an entry
+// it doesn't recognise. Wheel names carry a version we can't know ahead of time,
+// so `.whl` is matched by extension, non-recursively, inside this one directory.
+function clearBundleDir(dir) {
+  for (const entry of readdirSync(dir, { withFileTypes: true })) {
+    const known =
+      entry.name.endsWith('.whl') ||
+      OFFLINE_FILES.includes(entry.name) ||
+      BUNDLE_EXTRA_FILES.includes(entry.name);
+    if (entry.isDirectory() || !known) return false;
+    unlinkSync(join(dir, entry.name));
+  }
+  rmdirSync(dir);
+  return true;
+}
+
+// Empty the staging directory by deleting the exact files we create — never with
+// a recursive delete. Two things fall out of that: a bad path can at worst try to
+// unlink a handful of names that won't exist, and `rmdir` refuses a non-empty
+// directory, so anything unexpected in there stops us instead of being destroyed.
+// Unlinking a symlink removes the link and never the target.
+//
+// Returns true when the directory is gone or empty and safe to stage into.
+export function clearOnboardingDir(dir) {
+  if (!isAbsolute(dir)) return false;
+  if (!existsSync(dir)) return true;
+
+  try {
+    for (const entry of readdirSync(dir, { withFileTypes: true })) {
+      const path = join(dir, entry.name);
+
+      if (entry.isDirectory()) {
+        // arize-offline/ is ours; .staging-* is the leftover of a crashed run.
+        const ours = entry.name === OFFLINE_DIR_NAME || entry.name.startsWith('.staging-');
+        if (!ours || !clearBundleDir(path)) return false;
+        continue;
+      }
+
+      if (entry.name !== PROMPT_FILE_NAME && entry.name !== ENV_FILE_NAME) return false;
+      unlinkSync(path);
+    }
+    return true;
+  } catch {
+    return false;
+  }
+}
+
+// Clear and create the staging directory, degrading to a temp directory when the
+// home path can't be used: a read-only home, a directory owned by another user
+// after a past `sudo npx evals`, a full disk, an offline redirected profile on
+// Windows, or an unrecognised file we refuse to delete. We attempt the work and
+// degrade rather than probing for writability first — a probe lies on NFS and
+// ACL filesystems, races, and can't see ENOSPC at all.
+//
+// Mode 0700 because Step 4A writes credentials into this directory.
+export function prepareOnboardingDir(dir = resolveOnboardingDir()) {
+  if (clearOnboardingDir(dir)) {
+    try {
+      mkdirSync(dir, { recursive: true, mode: 0o700 });
+      // mkdir's mode is umask-filtered, so set it outright. Windows has no POSIX
+      // mode; there the file inherits the profile's own ACL, as it does today.
+      if (process.platform !== 'win32') chmodSync(dir, 0o700);
+      return { dir, fellBack: false };
+    } catch {
+      // Fall through to the temp directory.
+    }
+  }
+  return { dir: mkdtempSync(join(tmpdir(), 'arize-onboarding-')), fellBack: true };
+}
+
 // Copy the bundled wheels next to the prompt. Returns the staged directory, or
 // null when this package has no usable vendor/ (a git checkout that hasn't run
 // the build). The shell launchers stage the same bundle themselves, fetching it
@@ -77,15 +180,22 @@ export function stageOfflineHarness(dir, vendorDir = VENDOR_DIR) {
   }
 }
 
-// Write the bundled prompt to a temp file and return a short seed instruction
-// that points the agent at it.
+// Stage the bundled prompt and return the seed instruction that points the agent
+// at it, plus whether we had to fall back off the home directory.
 function prepareSeedPrompt() {
   const promptText = readFileSync(BUNDLED_PROMPT_PATH, 'utf8');
-  const dir = mkdtempSync(join(tmpdir(), 'arize-onboarding-'));
-  const promptFile = join(dir, 'onboarding-prompt.md');
+  const { dir, fellBack } = prepareOnboardingDir();
+  const promptFile = join(dir, PROMPT_FILE_NAME);
   writeFileSync(promptFile, promptText, 'utf8');
   stageOfflineHarness(dir);
-  return `Read the file ${promptFile} and follow it to set up Arize AX tracing in this project, walking me through each step and asking me questions as needed.`;
+  // Single quotes, not double: a home path is likelier to contain a space than a
+  // temp path was, but Windows spawns through the shell and embedded double
+  // quotes don't survive that reliably. cmd.exe leaves single quotes alone.
+  // No "in this project": the launcher may well be run from a home directory, and
+  // Step 4 is what scopes the work — an app here, an app at another path, a starter
+  // app, or the coding agent itself. Keep this wording in step with start.{sh,ps1}.
+  const seed = `Read the file '${promptFile}' and follow it to set up Arize AX tracing, walking me through each step and asking me questions as needed.`;
+  return { seed, dir, fellBack };
 }
 
 // Coding agents we can launch interactively, seeded with the prompt.
@@ -351,17 +461,26 @@ function App({ onDone }) {
 function launchAgent(agent) {
   const isWin = process.platform === 'win32';
 
-  let seed;
+  let prepared;
   try {
-    seed = prepareSeedPrompt();
+    prepared = prepareSeedPrompt();
   } catch (err) {
-    console.error(`Could not read the onboarding prompt: ${err.message}`);
+    console.error(`Could not prepare the onboarding prompt: ${err.message}`);
     process.exit(1);
   }
 
+  // Say so when we couldn't use the home directory. Otherwise the seed path
+  // silently reverts to the /var/folders/… form this change exists to avoid, and
+  // Step 4A is going to fail later anyway — it writes the harness and its
+  // credentials under ~/.arize regardless of where the prompt was staged.
+  if (prepared.fellBack) {
+    console.log(`\nCouldn't use ${resolveOnboardingDir()} — staged in ${prepared.dir} instead.`);
+    console.log('Instrumenting an app still works; tracing this coding agent (Step 4A) needs a writable home directory.');
+  }
+
   console.log(`\nLaunching ${agent.label}…\n`);
 
-  const child = spawn(agent.bin, agent.args(seed), {
+  const child = spawn(agent.bin, agent.args(prepared.seed), {
     stdio: 'inherit',
     shell: isWin // .cmd/.bat shims on Windows need the shell to resolve
   });

package/onboarding-prompt.md

@@ -6,15 +6,14 @@ Work through the flow below in order. Installing the AX CLI and Arize skills, au
 
 ## Step 0: Welcome and confirm
 
-Greet the user and show the plan, then ask before doing anything:
+Greet the user and show the plan, then ask before doing anything. Send this block as written and nothing else: no added steps, no parentheticals on the ones here (not "browser OAuth", not a folder path), and no preamble about the network check or the approval gate. Both come later in the flow, and explaining them before the user has agreed to anything is noise.
 
 ```text
 Welcome to Arize AX. I'll get you set up with tracing. Here's what I'll do:
 
 1. Install the AX CLI and Arize skills
 2. Create a free Arize AX account or sign you in
-3. Add tracing to your app, a new starter app, or this coding agent itself
-4. Capture your first traces
+3. Add tracing to an existing app, a new starter app, or this coding agent itself
 
 Shall I proceed?
 ```
@@ -23,6 +22,26 @@ Do not proceed until the user approves.
 
 ## Prerequisites
 
+**Check network access first.** Every step below needs it: the CLI comes from PyPI, and auth and traces go to Arize. Agent sandboxes often block network by default, and it fails as a hang rather than an error:
+
+```bash
+for h in pypi.org registry.npmjs.org api.arize.com app.arize.com; do
+  code=$(curl -s -o /dev/null -w '%{http_code}' --max-time 5 "https://$h")
+  [ "$code" != "000" ] && echo "$h ok" || echo "$h BLOCKED"
+done
+```
+
+`000` is a DNS or connect failure — your sandbox, not the user's connection.
+
+If anything is blocked, ask your harness to escalate: request network or elevated permissions for the commands that need them, rather than retrying silently or waiting. If escalation is refused or your harness doesn't offer it, stop — with no network this flow cannot complete, so don't retry and don't look for a workaround:
+
+```text
+My sandbox is blocking network access, so I can't install the CLI or reach
+Arize. Restart me with network access enabled and I'll pick this up.
+```
+
+Later in the flow, treat a connection or DNS error from any command the same way: it means network, not authentication, so stop and report rather than re-authenticating or recreating a profile.
+
 The AX CLI must be **arize-ax-cli `0.29.0` or newer** (Step 1 installs the latest) and needs **Python 3.11+** — a hard requirement. If Python is missing, stop and have the user install it from https://www.python.org/downloads/ and re-run. **Node.js 18+ with npx** is optional but installs the Arize skills; without it, don't stop — continue and use the docs paths in Step 6 (the Vercel AI SDK v7 starter needs Node.js 22+).
 
 ## Step 1: Install or update the AX CLI and Arize skills
@@ -35,7 +54,7 @@ pipx install arize-ax-cli   # already installed: pipx upgrade arize-ax-cli
 python3 -m pip install --upgrade arize-ax-cli
 ```
 
-If npx is available, install the Arize agent skills; otherwise skip and continue (Step 6 falls back to the docs paths):
+If npx is available, install the Arize agent skills; otherwise skip and continue (Step 6 falls back to the docs paths). If the install errors or doesn't finish promptly, skip it and continue rather than waiting — the same fallback applies:
 
 ```bash
 npx skills add Arize-ai/arize-skills --skill '*' --yes
@@ -72,7 +91,7 @@ If a `default` profile exists but the probe failed, it's signed out or expired,
   ax profiles create default --auth-method api-key --api-key "$ARIZE_API_KEY"
   ```
 
-  Reuse this key throughout — do **not** create a new one in Step 6.
+  Reuse this key throughout — do **not** create a new one in Step 6. Only if the probe passes, though: a stale or wrong-org `ARIZE_API_KEY` still looks present. If it fails, `ax profiles delete default`, take the OAuth path below, and let Step 6 create a key as normal.
 
 - **No key anywhere** — sign up or sign in with browser OAuth:
 
@@ -80,7 +99,7 @@ If a `default` profile exists but the probe failed, it's signed out or expired,
   ax profiles create default --auth-method oauth --utm-params "utm_source=npmevals&utm_medium=cli&utm_campaign=prompt-first-onboarding"
   ```
 
-The `--utm-params` flag tags a new sign-up with onboarding attribution. It's already included on the three browser OAuth commands that take it — the two `ax profiles create --auth-method oauth` calls and the `ax auth login` fallback. Pass those verbatim; don't drop it, change the values, or add the flag to the api-key `profiles create` or any other `ax` command.
+Pass `--utm-params` verbatim on the two browser OAuth commands that carry it (`profiles create --auth-method oauth` and the `ax auth login` fallback) and add it to no other `ax` command — it tags a new sign-up with onboarding attribution.
 
 Always pass the positional profile name `default`. Without it, the CLI prompts `profile name [default]:`, receives EOF from an agent-run command, and exits with `Goodbye!` without creating a profile.
 
@@ -88,39 +107,23 @@ The rest of this step applies only to the **browser OAuth** branch.
 
 Creating an OAuth profile **is** the browser login flow. Treat it as an interactive browser handoff: it opens a browser, starts a localhost callback server such as `127.0.0.1:<port>/callback`, and waits for the browser redirect. The command must stay alive until the redirect lands and the CLI exits on its own.
 
-While the OAuth command is waiting, do not close its stdin, send Ctrl-C, `pkill` the AX process, start a second auth command, or run an auth probe. Any of these aborts the in-progress browser flow. There is one deliberate exception — the new email/password signup case described below, where the callback never lands and the command must be restarted.
+While the OAuth command is waiting, do not close its stdin, send Ctrl-C, `pkill` the AX process, start a second auth command, or run an auth probe. Any of these aborts the in-progress browser flow. There is no exception: every sign-in path — existing account, Google/SSO, existing or brand-new email/password — completes through the same callback, so the command always gets there on its own.
 
 After launching the OAuth command, tell the user:
 
 ```text
 A browser window is opening for Arize AX sign-in.
 
-- Already have an account, or signing in with Google/SSO or an existing
-  email/password? Just finish in the browser and I'll continue automatically.
-- Creating a BRAND-NEW account with email and password? Arize emails you a
-  validation link. That link does NOT complete the CLI login, so this command
-  will hang. Click the link to finish creating your account, then tell me.
+Sign in — or create a new account — in the browser. I'll continue
+automatically once it completes.
 
-Did you just sign up for a new account with email and password?
+Creating a BRAND-NEW account with email and password? Arize emails you a
+validation link. Click it and finish in the browser; I'll wait.
 ```
 
-**If the user says yes (new email/password signup):** the localhost callback will never fire and the command will wait forever, so this is the one time you break the "keep it alive" rule. Once they confirm they've clicked the validation link and their account exists, terminate the waiting OAuth command (send Ctrl-C / kill that process), then re-run it:
+Then wait for the command to exit on its own. Treat exit code `0`, or CLI success output such as `Configuration saved to profile 'default'` or `Active profile set`, as the primary completion signal. A new email/password sign-up can sit waiting for a while — the user has to find the validation email — so a long wait is not a hang; leave it alone.
 
-```bash
-ax profiles create default --auth-method oauth --utm-params "utm_source=npmevals&utm_medium=cli&utm_campaign=prompt-first-onboarding"
-```
-
-The second run is now a plain sign-in for the validated account. Its callback lands normally, so handle it with all the standard rules below — keep it alive and wait for it to exit on its own.
-
-**Otherwise (existing account, SSO, or existing email/password):** wait for the command to exit on its own. Treat exit code `0`, or CLI success output such as `Configuration saved to profile 'default'` or `Active profile set`, as the primary completion signal.
-
-Only after the OAuth command completes, verify authentication with a non-secret probe:
-
-```bash
-ax spaces list --limit 1 --output json
-```
-
-If the probe succeeds, continue. If the default OAuth profile already existed and the probe returns an authentication error, the profile is signed out or expired — run the fallback:
+Only after the OAuth command completes, re-run the probe from the top of this step. If it succeeds, continue. If the default OAuth profile already existed and the probe returns an authentication error, the profile is signed out or expired — run the fallback:
 
 ```bash
 ax auth login --utm-params "utm_source=npmevals&utm_medium=cli&utm_campaign=prompt-first-onboarding"
@@ -151,16 +154,20 @@ Ask what the user wants to trace before inspecting anything — the third option
 ```text
 What would you like to trace?
 
-1. An app in this folder
+1. An existing app — in this folder or at another path
 2. A new starter app I create for you
 3. This coding agent itself — every session you run, in any project
 
 Which one?
 ```
 
-If they pick **3**, go to [Step 4A](#step-4a-trace-this-coding-agent) and skip the folder inspection entirely. For **1** or **2**, continue below.
+If they pick **3**, go to [Step 4A](#step-4a-trace-this-coding-agent) and skip the folder inspection entirely. If they pick **2**, go straight to "Create a starter app" below — don't inspect anything. For **1**, continue.
+
+For options **1** and **2**, one directory ends up being **the app folder**: the app you instrument, which is not always the folder you were launched in. Establish it in this step and use it for every path in Steps 5 to 7.
 
-Inspect the current folder to decide whether an app already exists. Do not change files during inspection. Look for:
+### Detecting an app
+
+Inspect a candidate folder to decide whether an app already exists. Do not change files during inspection. Look for:
 
 - Python: `pyproject.toml`, `requirements.txt`, `setup.py`, `Pipfile`, imports.
 - TypeScript/JavaScript: `package.json`, lockfiles, `src`, `app`, `pages`, provider imports.
@@ -168,24 +175,48 @@ Inspect the current folder to decide whether an app already exists. Do not chang
 - Existing observability: `opentelemetry`, `TracerProvider`, `ARIZE_*`, `OTEL_*`, `OTLP_*`, Datadog, Honeycomb, Sentry, or other tracing.
 - Agent framework: identify it by its import/package — e.g. `langchain` / `langgraph`, `llama_index`, `crewai`, `autogen`, `semantic_kernel`, `pydantic_ai`, `google.adk`, `dspy`, `agent_framework`, and others. **Route on the framework, not the provider client it wraps** — an `openai` or `anthropic` import inside a framework app is not the thing to instrument; the framework almost certainly has its own integration (see Step 6).
 
-In a monorepo, check the git root to get oriented, but only instrument apps in or below the current working directory. If the project spans more than one language, instrument each one (route each through its own integration page in Step 6).
+In a monorepo, check the git root to get oriented, but only instrument apps in or below the folder you are scanning. If the project spans more than one language, instrument each one (route each through its own integration page in Step 6).
 
-Then branch on what you found — a folder with no app means option 2:
+Run this detection on the current folder first, then branch on what you found.
 
 ### If an app exists in the current folder
 
-Do not offer the starter-app path. Summarize the detected stack and offer to instrument it:
+Summarize the detected stack and offer both routes — declining the local app must not be a dead end, because the user may well have run this from a folder above the app they care about:
 
 ```text
-I found a <language>/<framework> app in this folder. Want me to add Arize AX
-tracing to it?
+I found a <language>/<framework> app in this folder. I can trace that, or an
+app somewhere else.
+
+1. Trace the app in this folder
+2. Trace an app at a different path — tell me where
+
+Which one?
 ```
 
-If the user declines, stop here rather than falling back to a starter app.
+On **1**, the app folder is the current folder; continue to Step 5. On **2**, follow "If the user gives a path" below. Don't offer the starter-app path here — they already have an app.
 
 ### If no app exists in the current folder
 
-Go straight to the starter-app choice — do not ask whether to instrument the empty folder. Ask which folder to create it in, then offer these choices:
+Do not ask whether to instrument the empty folder, and do not assume a starter app is what they want. Ask for a path first:
+
+```text
+I don't see an app in this folder. Give me the path to the app you want to
+trace, and I'll instrument it — or say "starter" and I'll create a new app
+for you instead.
+```
+
+On a path, follow the next section. On "starter", go to "Create a starter app".
+
+### If the user gives a path
+
+- Expand `~` and resolve a relative path against the current folder, then confirm the resolved absolute path back to the user before you scan it. A typo caught here beats an API key written into the wrong repo.
+- Run the detection checklist on that folder. If it holds an app, that folder is the app folder from now on — summarize the stack you found there and continue to Step 5.
+- If the path doesn't exist, or exists but has no app in it, say what you actually found and ask for another path. Don't quietly fall through to a starter app.
+- If your own permission or sandbox layer won't let you read and write outside the folder you were launched in, say so plainly instead of half-instrumenting the app. Give the two ways out: the user grants access to that path, or they re-run `npx evals` from inside the app folder.
+
+### Create a starter app
+
+For option 2, or when the user asks for a starter app after the questions above. Ask which folder to create it in — that folder becomes the app folder — then offer these choices:
 
 - OpenAI — Python or TypeScript (with a tool call)
 - Anthropic — Python (with a tool call; official AX auto-instrumentation)
@@ -202,6 +233,16 @@ Installer harness names: `claude`, `codex`, `cursor`, `copilot`, `gemini`, `kiro
 
 Per-agent setup, including the Claude Code and Cursor marketplace-plugin routes, is at `https://arize.com/docs/ax/integrations/platforms/<agent>/<agent>-tracing` (e.g. `.../claude-code/claude-code-tracing`) — the source of truth if anything below fails.
 
+### Check the home directory is writable first
+
+Everything below writes under `~/.arize` — the credentials file, then the harness itself. Confirm that works before asking for approval, so a read-only or restricted home fails here rather than halfway through an install the user already said yes to:
+
+```bash
+mkdir -p ~/.arize/onboarding && touch ~/.arize/onboarding/.probe && rm -f ~/.arize/onboarding/.probe
+```
+
+If that fails, say so and stop this path — agent tracing needs a writable home and there is no workaround from inside the session. Instrumenting an app (options 1 and 2) does not, so offer that instead. If the launcher already told the user it couldn't use `~/.arize/onboarding`, this is the same cause; don't re-diagnose it.
+
 ### Get approval — this needs its own explicit yes
 
 Wider scope than an app install, so disclose it and wait:
@@ -227,7 +268,7 @@ Do not proceed without a yes. Note which categories they accepted — you enable
 The installer reads credentials from a dotenv file, keeping the API key out of the command line, shell history, and this chat. Write it outside the project so it cannot be committed:
 
 ```bash
-mkdir -p ~/.arize && : > ~/.arize/onboarding.env && chmod 600 ~/.arize/onboarding.env
+: > ~/.arize/onboarding/harness.env && chmod 600 ~/.arize/onboarding/harness.env
 ```
 
 Add the Step 3 space ID as file contents, plus one `true` line per category the user **accepted**. Unattended installs capture nothing unless asked to, so a category you omit is off — omit the line for anything they declined:
@@ -242,7 +283,7 @@ ARIZE_LOG_TOOL_CONTENT=true
 Do **not** set `ARIZE_PROJECT_NAME` — each harness defaults to its own project (`claude-code`, `codex`, …), which keeps two traced agents apart, and you read the real name back after installing. Then create the key into the same file, written atomically and never printed:
 
 ```bash
-ax api-keys create --name "Coding agent tracing" --env-file ~/.arize/onboarding.env
+ax api-keys create --name "Coding agent tracing" --env-file ~/.arize/onboarding/harness.env
 ```
 
 **Skip that if `ARIZE_API_KEY` already existed in Step 2** — copy the existing value in without echoing it. At most one key, ever.
@@ -254,7 +295,7 @@ If a directory named `arize-offline/` sits beside this prompt file, install from
 ```bash
 # keep `< /dev/null`: an installer too old for --non-interactive then fails
 # fast instead of hanging on a prompt you cannot answer
-ARIZE_ENV_FILE=~/.arize/onboarding.env \
+ARIZE_ENV_FILE=~/.arize/onboarding/harness.env \
   bash <prompt-dir>/arize-offline/harness-install.sh \
   <harness> --wheel-dir <prompt-dir>/arize-offline --non-interactive < /dev/null
 ```
@@ -262,7 +303,7 @@ ARIZE_ENV_FILE=~/.arize/onboarding.env \
 Otherwise fetch it (the harness name must come first, before any flag):
 
 ```bash
-ARIZE_ENV_FILE=~/.arize/onboarding.env \
+ARIZE_ENV_FILE=~/.arize/onboarding/harness.env \
   bash <(curl -fsSL https://raw.githubusercontent.com/Arize-ai/coding-harness-tracing/main/install.sh) \
   <harness> --non-interactive < /dev/null
 ```
@@ -270,14 +311,14 @@ ARIZE_ENV_FILE=~/.arize/onboarding.env \
 Delete the env file only once the install has actually run — a retry needs it, and so does the user if they end up running the command:
 
 ```bash
-rm -f ~/.arize/onboarding.env
+rm -f ~/.arize/onboarding/harness.env
 ```
 
 On Windows, use `harness-install.bat` from the same directory — `cmd` cannot run the `.sh`:
 
 ```powershell
-$env:ARIZE_ENV_FILE = "$HOME\.arize\onboarding.env"
-& <prompt-dir>\arize-offline\harness-install.bat <harness> --wheel-dir <prompt-dir>\arize-offline --non-interactive
+$env:ARIZE_ENV_FILE = "$HOME\.arize\onboarding\harness.env"
+& "<prompt-dir>\arize-offline\harness-install.bat" <harness> --wheel-dir "<prompt-dir>\arize-offline" --non-interactive
 ```
 
 Without `arize-offline/`, use the install command from the agent's docs page with `$env:ARIZE_ENV_FILE` set first.
@@ -320,19 +361,21 @@ hooks existed. So:
 3. Tell me when you have, and I'll check for traces.
 ```
 
-Then poll with the Step 7 command and counter, passing that project name and space ID, every ~15 seconds for up to ~3 minutes. On a non-zero count go to Step 8. On timeout say so plainly and give the likely causes: the new session started before the install finished, the agent was never asked to do anything, `ARIZE_TRACE_ENABLED` is `false` in the agent's settings, or the wrong space. Never fabricate trace results.
+Then poll with the Step 7 command and counter, passing that project name and space ID, every ~15 seconds for up to ~3 minutes. On a non-zero count go to Step 8. On timeout say so plainly and give the likely causes: the new session started before the install finished, the agent was never asked to do anything, `ARIZE_TRACE_ENABLED` is `false` in the agent's settings, or the wrong space.
 
 ## Step 5: Present the plan and get approval
 
-Steps 5 to 7 cover the app paths (options 1 and 2). If you took Step 4A, its own approval gate and verification replace them — go straight to Step 8.
+Steps 5 to 7 are for the app paths only; if you took Step 4A, go straight to Step 8.
 
-Before creating any remote resource, writing files, or installing dependencies, present one consolidated plan and wait for approval. This is the gate the intro refers to — nothing so far has modified the app or created AX resources.
+Before creating any remote resource, writing files, or installing dependencies, present one consolidated plan and wait for approval.
 
 For an existing app, cover: detected language and framework, package manager, LLM provider or agent framework, any existing tracing to preserve, the env file that will be updated, the instrumentation packages and files that will change, whether a new AX user API key will be created or the existing `ARIZE_API_KEY` reused, and the project name that will be used.
 
 For a starter app, cover: the chosen provider and language, the target folder, the packages that will be installed, and the project name.
 
-Choose a default project name from the current folder or app name: lowercase it, replace spaces and unsupported punctuation with hyphens, and append `-arize-tracing` if it is too generic. Do not create an AX project explicitly — it is created on first trace ingestion.
+**State the app folder's absolute path in the plan whenever it isn't the folder you were launched in** — every file you touch lands there, and it's the one detail the user cannot verify from context.
+
+Choose a default project name from the app folder or app name: lowercase it, replace spaces and unsupported punctuation with hyphens, and append `-arize-tracing` if it is too generic. Do not create an AX project explicitly — it is created on first trace ingestion.
 
 ```text
 Here's my plan. Shall I proceed?
@@ -346,7 +389,7 @@ Only after approval, execute the plan in this order.
 
 ### Choose the env file
 
-Pick the env file to match the app and use it for every variable below:
+The env file lives in the **app folder** from Step 4. Pick its name to match the app and use that one file for every variable below:
 
 - Next.js, Vite, or browser-adjacent TypeScript apps: `.env.local`
 - Python apps, Node scripts, backend services, or unknown type: `.env`
@@ -354,7 +397,7 @@ Pick the env file to match the app and use it for every variable below:
 
 Do not read existing env file contents into chat. Preserve unrelated variables and never reveal their values.
 
-Make sure the env file is git-ignored before writing the API key to it — if the repo has a `.gitignore`, confirm it covers the file (add `.env` / `.env.local` if not); for a starter app you create, add one. The API key must never be committed to version control.
+Make sure the env file is git-ignored before writing the API key to it — check the `.gitignore` of the app folder's own repo. If it has one, confirm it covers the file (add `.env` / `.env.local` if not); for a starter app you create, add one. The API key must never be committed to version control.
 
 ### Write the non-secret variables
 
@@ -365,9 +408,11 @@ ARIZE_SPACE_ID=<space-id>
 ARIZE_PROJECT_NAME=<chosen-project-name>
 ```
 
+An exported variable beats this file — both `dotenv` implementations leave an existing environment variable alone by default. Check `env | grep ARIZE_`; if anything you just wrote is exported, tell the user to `unset` it in the shell they'll run the app from, or to load with `override=True`. Your Step 7 check reads the CLI profile, not the app's environment, so it would pass regardless.
+
 ### Create the API key
 
-**Skip this entirely if `ARIZE_API_KEY` was already present in Step 2** — reuse it and leave its env value untouched. Only create a key when you authenticated with browser OAuth and the app has no key yet.
+**Skip this entirely if `ARIZE_API_KEY` was already present and validated in Step 2** — reuse it and leave its env value untouched. Only create a key when you authenticated with browser OAuth and the app has no key yet.
 
 Create the key and write it into the env file in one step:
 
@@ -375,11 +420,11 @@ Create the key and write it into the env file in one step:
 ax api-keys create --name "Local Arize AX tracing" --env-file .env
 ```
 
-`--env-file` writes `ARIZE_API_KEY` atomically and **never prints it** — no temp file, no secret in your terminal or chat. Use `.env.local` if that's the app's convention; the file is created if missing, an existing `ARIZE_API_KEY` is replaced in place, and other variables are preserved.
+`--env-file` writes `ARIZE_API_KEY` atomically and **never prints it** — no temp file, no secret in your terminal or chat. Use `.env.local` if that's the app's convention, and always pass the app folder's path (`--env-file /path/to/app/.env`) — a bare `.env` would leave a stray key file in the wrong directory. The file is created if missing, an existing `ARIZE_API_KEY` is replaced in place, and other variables are preserved.
 
-**Create the key exactly once** — a second `ax api-keys create` just orphans a still-active key. Git-ignore the env file (see above) *before* running this so the key is never committed.
+**Create the key exactly once** — a second `ax api-keys create` just orphans a still-active key, and the env file must already be git-ignored.
 
-If key creation fails, have the user create one in the Arize AX UI and add it to the env file without exposing it in chat. Either way, create only one key.
+If key creation fails, have the user create one in the Arize AX UI and add it to the env file without exposing it in chat.
 
 Handle the LLM provider's own key (e.g. `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`) in the same env file — the app can't make a call or produce traces without it:
 
@@ -405,7 +450,7 @@ Common shortcuts:
 
 1. Use a framework-specific OpenInference instrumentor if one exists (`openinference-instrumentation-<name>` / `@arizeai/openinference-instrumentation-<name>`), wired up with `arize-otel` per the manual-instrumentation guide; install it unpinned.
 1. Instrument the underlying provider (OpenAI, Anthropic, Bedrock, …) with its instrumentor **only if the framework calls the provider SDK directly**. Many agent frameworks instead drive the model through their own client layer and emit their own OpenTelemetry spans — a provider instrumentor captures **no traces** for those. Never reach for the provider instrumentor as a blind fallback just because you recognize an `openai`/`anthropic` client.
-1. Otherwise instrument manually via `arize-otel` (see https://arize.com/docs/ax/instrument/manual-instrumentation), or stop and ask the user if you still can't determine a setup.
+1. Otherwise instrument manually by code — follow https://arize.com/docs/ax/instrument/manual-instrumentation#by-code — or stop and ask the user if you still can't determine a setup. `arize-otel` gives you a tracer, not LLM spans: **set the OpenInference span kind on every span you create** (`LLM` for a model call, `TOOL` for a tool, `AGENT` for the loop, `CHAIN` for a step), **and set the span status** — `ERROR` with the exception recorded on failure, `OK` otherwise. Without the kind, spans arrive as generic spans that AX cannot read as LLM calls: Step 7's span count comes back non-zero and looks like success while the trace shows no input, output, model, or token counts. Without the status, failed calls look like successful ones and errors never surface in AX.
 
 For existing apps:
 
@@ -425,7 +470,7 @@ For starter apps:
 
 ### Package guidance
 
-Install into the app's existing environment (its virtualenv if it has one) exactly the packages the detected framework's integration page lists — instrumentor names and peer dependencies differ per framework, so follow that page rather than copying from another stack or guessing versions.
+Install into the app's existing environment — the one in the app folder, using its virtualenv if it has one — exactly the packages the detected framework's integration page lists — instrumentor names and peer dependencies differ per framework, so follow that page rather than copying from another stack or guessing versions.
 
 ## Step 7: Run the app and poll for the first trace
 
@@ -437,11 +482,12 @@ If you created a starter app in this flow, offer to run it for the user:
 Your starter app is ready. Want me to run it for you?
 ```
 
-If they say yes, run it yourself with the run command, then poll. If they say no — or if you instrumented their existing app rather than creating a starter — tell them the exact run command and ask them to run it:
+If they say yes, run it yourself with the run command, then poll. If they say no — or if you instrumented their existing app rather than creating a starter — tell them the exact run command and ask them to run it. When the app folder isn't the folder they're sitting in, lead with the `cd` so the command works as pasted:
 
 ```text
 Run your app with:
 
+cd <app folder>
 <run command>
 
 It should make at least one LLM call. I'll poll Arize AX and let you know as
@@ -462,20 +508,22 @@ A non-zero count confirms traces are arriving. If you need to inspect a span to
 This uses the CLI profile from Step 2 (OAuth or api-key) — it works the same either way. If the export errors with an authentication failure, the profile isn't valid; re-run the Step 2 probe and re-authenticate, or fall back to having the user open the project in the Arize AX UI to confirm traces.
 
 - When spans come back, stop polling and continue to Step 8.
-- On timeout, do not fail silently. Tell the user no traces arrived yet, and give likely causes: app didn't make an LLM call, tracing initialized after the client was created, a short-lived script exited before flushing spans, or the wrong space/project/env file. Offer to re-check once they've run it again.
+- On timeout, do not fail silently. Tell the user no traces arrived yet, and give likely causes: app didn't make an LLM call, tracing initialized after the client was created, a short-lived script exited before flushing spans, the wrong space/project/env file, or **an exported `ARIZE_*` variable overriding the env file** (`env | grep ARIZE_`). Offer to re-check once they've run it again.
 
 Do not fabricate trace results. Only report traces the export command actually returned.
 
 ## Step 8: Report the first traces with a link
 
-Once spans arrive, report the span count and give the user a link into Arize AX for the project. Point them at the UI to explore the trace contents rather than printing span bodies into chat.
+Once spans arrive, report the span count and give the user a deep link to the project in Arize AX. Point them at the UI to explore the trace contents rather than printing span bodies into chat.
+
+Build that link with the **Arize link skill**: if you installed the skills in Step 1, load it now by reading `arize-link/SKILL.md` from your agent's skills directory (same paths as Step 6) and follow it. It owns the URL format and the `ax` commands that discover the organization and project IDs; you already have the project name and the space ID from Step 3.
 
 ```text
 Your first traces are in Arize AX. Open project `<ARIZE_PROJECT_NAME>` here:
-https://app.arize.com/
+<project link>
 ```
 
-If a space- or project-specific URL is known from the CLI or docs, prefer that deep link over the app root. Do not invent a URL structure you are not sure of; fall back to `https://app.arize.com/` plus instructions to select the project.
+If that skill file doesn't exist (e.g. npx was missing in Step 1), don't invent a URL structure: fall back to `https://app.arize.com/` plus instructions to select the project.
 
 ## Step 9: Point at docs
 
@@ -501,11 +549,13 @@ If you took Step 4A, add the controls that matter for agent tracing:
 ## Critical rules
 
 - Get the user's approval (Step 5) before creating AX resources, editing files, or installing dependencies.
-- Authenticate the CLI before any other `ax` call — every command, including the final trace check, uses the active profile; env vars alone don't authenticate it. On the browser OAuth branch, keep the command alive until its localhost callback completes (never close stdin, Ctrl-C, `pkill`, or probe while it waits) — sole exception: a brand-new email/password signup, whose link never calls back, so ask the user, then kill and re-run as a sign-in.
+- Authenticate the CLI (Step 2) before any other `ax` call; env vars alone don't. Never interrupt a waiting OAuth command — no stdin close, Ctrl-C, `pkill`, or probe. A slow wait is not a hang.
 - Never print, log, or summarize secrets in chat — API keys, env-file contents/values, or span bodies (prompts, completions, tool args, user data) — and never read env files into chat. Only report traces a command actually returned.
-- Create at most one AX API key: skip if `ARIZE_API_KEY` already exists (reuse it), otherwise a **single** `ax api-keys create --env-file <file>`. Git-ignore that file before creating the key; never create a second key.
+- One AX API key, ever: reuse a **validated** `ARIZE_API_KEY`, else a **single** `ax api-keys create --env-file <file>` into an already-git-ignored file.
+- Ambient `ARIZE_*` variables are claims to verify, not facts to adopt: probe a found API key before reusing it, and remember that an exported variable overrides the env file you write.
+- Everything you write goes in the **app folder** (Step 4), which may not be the folder you were launched in. Name its absolute path in the plan; pass it explicitly to `--env-file` and the run command.
 - Write the space **ID** (not its name) to `ARIZE_SPACE_ID`, or traces won't route; never create an AX project explicitly (it's made on first ingestion).
-- Initialize tracing before LLM clients are created, and flush/shut down the tracer before short-lived scripts exit. Vercel AI SDK v7 also needs Node.js 22+, `@ai-sdk/otel` registered, and `experimental_telemetry: { isEnabled: true }` per call.
-- Tracing the coding agent (Step 4A) is machine-wide and captures prompts and tool output by default, so it needs its own explicit yes — never fold it into another approval. Run its installer with `--non-interactive` and `< /dev/null` so it can never sit waiting on a prompt, verify with `status --json` before claiming success, and never restart or kill the session you are running in.
+- Initialize tracing before LLM clients exist; flush before short-lived scripts exit. Vercel AI SDK v7 also needs Node.js 22+, `@ai-sdk/otel` registered, and per-call `experimental_telemetry`.
+- Step 4A needs its own explicit yes, never folded into another approval — machine-wide, captures prompts and tool output. Run its installer with `--non-interactive` and `< /dev/null`, verify with `status --json`, and never restart the session you're in.
 
 Docs: https://arize.com/docs/llms.txt

package/package.json

@@ -1,9 +1,12 @@
 {
   "name": "evals",
-  "version": "2.5.0",
+  "version": "2.6.0",
   "description": "Arize AX onboarding — instrument your app with tracing via your coding agent",
   "type": "module",
   "main": "cli.js",
+  "engines": {
+    "node": ">=20"
+  },
   "scripts": {
     "postinstall": "node -e \"console.log('Arize evals installed — run: npx evals')\"",
     "start": "node cli.js",
@@ -17,9 +20,10 @@
     "react": "^19.0.0"
   },
   "bin": {
-    "evals": "./cli.js"
+    "evals": "./bin.js"
   },
   "files": [
+    "bin.js",
     "cli.js",
     "onboarding-prompt.md",
     "start.sh",

package/start.ps1

@@ -32,9 +32,24 @@ $VendorUrl = if ($env:ARIZE_VENDOR_URL) { $env:ARIZE_VENDOR_URL } else { 'https:
 # Prefer the richer, bundled-prompt `npx evals` experience when Node is present.
 # This script is the no-npm fallback; if npx exists, hand off to it.
 # Set $env:ARIZE_SKIP_NPX=1 to force this shell path even when npx is available.
+#
+# Only hand off on Node 20+, which is what the Ink UI needs (see bin.js). Handing
+# an older Node to npx would swap a working flow for a hard failure — this script
+# needs no Node at all, so stay here instead. Keep the floor in step with bin.js.
 if (-not $env:ARIZE_SKIP_NPX -and (Get-Command npx -ErrorAction SilentlyContinue)) {
-  npx --yes evals
-  exit $LASTEXITCODE
+  $nodeMajor = 0
+  $nodeVersion = $null
+  if (Get-Command node -ErrorAction SilentlyContinue) {
+    $nodeVersion = (& node --version 2>$null)
+    if ($nodeVersion -match '^v?(\d+)') { $nodeMajor = [int]$Matches[1] }
+  }
+  if ($nodeMajor -ge 20) {
+    npx --yes evals
+    exit $LASTEXITCODE
+  }
+  $shown = if ($nodeVersion) { $nodeVersion } else { 'not found' }
+  Write-Host "Node $shown is too old for the npx UI — continuing without it."
+  Write-Host ""
 }
 
 # Supported agents: id -> label, install URL, and whether the REPL is seeded with `-i`.
@@ -98,69 +113,142 @@ if (-not $chosen) { exit 1 }
 
 # A directory, not a bare temp file: Step 4A looks for the offline bundle *beside*
 # the prompt, so the prompt needs a directory of its own.
-$promptDir = Join-Path ([System.IO.Path]::GetTempPath()) ("arize-onboarding-" + [System.Guid]::NewGuid().ToString('N').Substring(0, 8))
-New-Item -ItemType Directory -Path $promptDir -Force | Out-Null
-$promptFile = Join-Path $promptDir 'onboarding-prompt.md'
-try {
-  Write-Host "Fetching the onboarding prompt..."
+#
+# ~/.arize/onboarding, not a temp path: the agent is asked to read a file outside
+# its workspace, and a named path is an approval a human can grant with confidence.
+# Sits beside Step 4A's own ~/.arize/harness install, which we must never touch.
+#
+# GetFolderPath('UserProfile') rather than $HOME: on Windows PowerShell 5.1 $HOME
+# comes from HOMEDRIVE+HOMEPATH, which for domain users can point at a redirected
+# network share that is slow or offline. This returns USERPROFILE on Windows and
+# $HOME under PowerShell Core on macOS/Linux — one expression for both.
+function Clear-BundleDir($dir) {
+  # Delete only what we stage, by name. Wheel filenames carry a version we can't
+  # know, so those go by extension, non-recursively, in this directory only.
+  Get-ChildItem -LiteralPath $dir -File -Filter '*.whl' -ErrorAction SilentlyContinue |
+    Remove-Item -Force -ErrorAction SilentlyContinue
+  foreach ($name in @('harness-install.sh', 'harness-install.bat', 'LICENSE-coding-harness-tracing', 'MANIFEST')) {
+    Remove-Item -LiteralPath (Join-Path $dir $name) -Force -ErrorAction SilentlyContinue
+  }
+  # No -Recurse anywhere: a non-empty directory fails here rather than taking
+  # something we didn't put there with it, and Remove-Item's habit of recursing
+  # through a directory junction can't bite us.
+  Remove-Item -LiteralPath $dir -Force -ErrorAction SilentlyContinue
+  return -not (Test-Path -LiteralPath $dir)
+}
+
+# -LiteralPath throughout: a home path containing [ or ] is a real wildcard
+# footgun for the non-literal forms. Keep this list in step with cli.js
+# OFFLINE_FILES and start.sh.
+function Clear-PromptDir($dir) {
+  if (-not (Test-Path -LiteralPath $dir)) { return $true }
+  foreach ($sub in Get-ChildItem -LiteralPath $dir -Directory -Force -ErrorAction SilentlyContinue) {
+    $ours = ($sub.Name -eq 'arize-offline') -or $sub.Name.StartsWith('.staging')
+    if (-not $ours) { return $false }
+    if (-not (Clear-BundleDir $sub.FullName)) { return $false }
+  }
+  foreach ($name in @('onboarding-prompt.md', 'harness.env')) {
+    Remove-Item -LiteralPath (Join-Path $dir $name) -Force -ErrorAction SilentlyContinue
+  }
+  # Anything left is not ours.
+  return -not (Get-ChildItem -LiteralPath $dir -Force -ErrorAction SilentlyContinue)
+}
+
+$candidate = if ($env:ARIZE_ONBOARDING_DIR) {
+  $env:ARIZE_ONBOARDING_DIR
+} else {
+  $profileDir = [Environment]::GetFolderPath('UserProfile')
+  if (-not $profileDir) { $profileDir = $HOME }
+  if ($profileDir) { Join-Path (Join-Path $profileDir '.arize') 'onboarding' } else { $null }
+}
+
+# Try the work and degrade, rather than probing for writability — a probe races
+# and can't see a full disk. Covers a read-only home, a directory owned by another
+# user, an offline redirected profile, and a file in there we refuse to delete.
+$promptDir = $null
+if ($candidate -and (Clear-PromptDir $candidate)) {
   try {
-    Invoke-RestMethod -Uri $PromptUrl -OutFile $promptFile
+    New-Item -ItemType Directory -Path $candidate -Force -ErrorAction Stop | Out-Null
+    $promptDir = $candidate
   } catch {
-    Write-Host "Failed to download the onboarding prompt from $PromptUrl" -ForegroundColor Red
-    Write-Host "Check your connection and try again."
-    exit 1
+    $promptDir = $null
   }
+}
+if (-not $promptDir) {
+  $promptDir = Join-Path ([System.IO.Path]::GetTempPath()) ("arize-onboarding-" + [System.Guid]::NewGuid().ToString('N').Substring(0, 8))
+  New-Item -ItemType Directory -Path $promptDir -Force | Out-Null
+  if ($candidate) {
+    Write-Host "Couldn't use $candidate — staging in $promptDir instead."
+    Write-Host "Instrumenting an app still works; tracing this coding agent (Step 4A) needs a writable home directory."
+  }
+}
+$promptFile = Join-Path $promptDir 'onboarding-prompt.md'
 
-  # Fetch the bundled tracing harness so Step 4A can install it without piping a
-  # downloaded script into a shell — the thing auto-approval permission
-  # classifiers refuse. `npx evals` ships these files; this path pulls them from
-  # the CDN, using vendor/MANIFEST because a launcher cannot glob a CDN and must
-  # not hardcode versioned wheel names.
-  #
-  # Best-effort throughout. Any failure leaves no arize-offline directory and the
-  # prompt takes its documented network path instead. Staged under a temp name and
-  # renamed, so the directory Step 4A keys off only ever appears complete.
-  Write-Host "Fetching the tracing harness..."
-  $staging = Join-Path $promptDir '.staging'
-  try {
-    New-Item -ItemType Directory -Path $staging -Force | Out-Null
-    $manifestPath = Join-Path $staging 'MANIFEST'
-    Invoke-RestMethod -Uri "$VendorUrl/MANIFEST" -OutFile $manifestPath
-
-    foreach ($line in Get-Content $manifestPath) {
-      if (-not $line.Trim()) { continue }
-      # MANIFEST is `shasum -a 256` format: "<sha256>  <filename>".
-      $parts = $line -split '\s+', 2
-      $expected = $parts[0]
-      $name = $parts[1].Trim()
-      $dest = Join-Path $staging $name
-      Invoke-RestMethod -Uri "$VendorUrl/$name" -OutFile $dest
-      $actual = (Get-FileHash -Path $dest -Algorithm SHA256).Hash.ToLower()
-      if ($actual -ne $expected.ToLower()) {
-        throw "checksum mismatch for $name"
-      }
-    }
+Write-Host "Fetching the onboarding prompt..."
+try {
+  Invoke-RestMethod -Uri $PromptUrl -OutFile $promptFile
+} catch {
+  Write-Host "Failed to download the onboarding prompt from $PromptUrl" -ForegroundColor Red
+  Write-Host "Check your connection and try again."
+  exit 1
+}
 
-    Move-Item -Path $staging -Destination (Join-Path $promptDir 'arize-offline')
-  } catch {
-    Remove-Item $staging -Recurse -Force -ErrorAction SilentlyContinue
-    Write-Host "  (not available — Step 4A will download the installer instead)"
+# Fetch the bundled tracing harness so Step 4A can install it without piping a
+# downloaded script into a shell — the thing auto-approval permission
+# classifiers refuse. `npx evals` ships these files; this path pulls them from
+# the CDN, using vendor/MANIFEST because a launcher cannot glob a CDN and must
+# not hardcode versioned wheel names.
+#
+# Best-effort throughout. Any failure leaves no arize-offline directory and the
+# prompt takes its documented network path instead. Staged under a temp name and
+# renamed, so the directory Step 4A keys off only ever appears complete.
+Write-Host "Fetching the tracing harness..."
+$staging = Join-Path $promptDir '.staging'
+try {
+  New-Item -ItemType Directory -Path $staging -Force | Out-Null
+  $manifestPath = Join-Path $staging 'MANIFEST'
+  Invoke-RestMethod -Uri "$VendorUrl/MANIFEST" -OutFile $manifestPath
+
+  foreach ($line in Get-Content $manifestPath) {
+    if (-not $line.Trim()) { continue }
+    # MANIFEST is `shasum -a 256` format: "<sha256>  <filename>".
+    $parts = $line -split '\s+', 2
+    $expected = $parts[0]
+    $name = $parts[1].Trim()
+    $dest = Join-Path $staging $name
+    Invoke-RestMethod -Uri "$VendorUrl/$name" -OutFile $dest
+    $actual = (Get-FileHash -Path $dest -Algorithm SHA256).Hash.ToLower()
+    if ($actual -ne $expected.ToLower()) {
+      throw "checksum mismatch for $name"
+    }
   }
 
-  $seed = "Read the file $promptFile and follow it to set up Arize AX tracing in this project, walking me through each step and asking me questions as needed."
+  Move-Item -LiteralPath $staging -Destination (Join-Path $promptDir 'arize-offline')
+} catch {
+  Clear-BundleDir $staging | Out-Null
+  Write-Host "  (not available — Step 4A will download the installer instead)"
+}
 
-  Write-Host ("Launching {0}..." -f $Agents[$chosen].Label)
-  Write-Host ""
+# Single quotes around the path: the home directory is likelier to contain a space
+# than the old temp path was, and single quotes survive the console handoff below
+# where embedded double quotes would not.
+# No "in this project": this may well be run from a home directory, and Step 4 is
+# what scopes the work. Keep this wording in step with cli.js and start.sh.
+$seed = "Read the file '$promptFile' and follow it to set up Arize AX tracing, walking me through each step and asking me questions as needed."
 
-  # Launch interactive + seeded. No skip-permissions — the agent's approval
-  # model and the prompt's own approval gate must stay intact. External commands
-  # attach to the console, so the agent gets a real terminal.
-  $flag = $Agents[$chosen].Flag
-  if ($flag) {
-    & $chosen $flag $seed
-  } else {
-    & $chosen $seed
-  }
-} finally {
-  Remove-Item $promptDir -Recurse -Force -ErrorAction SilentlyContinue
+Write-Host ("Launching {0}..." -f $Agents[$chosen].Label)
+Write-Host ""
+
+# Launch interactive + seeded. No skip-permissions — the agent's approval
+# model and the prompt's own approval gate must stay intact. External commands
+# attach to the console, so the agent gets a real terminal.
+#
+# No try/finally around this any more: the staging directory persists so the agent
+# can still reach the prompt in a session that outlives this script. It is cleared
+# at the start of the next launch instead.
+$flag = $Agents[$chosen].Flag
+if ($flag) {
+  & $chosen $flag $seed
+} else {
+  & $chosen $seed
 }

package/start.sh

@@ -28,8 +28,23 @@ VENDOR_URL="${ARIZE_VENDOR_URL:-https://cdn.jsdelivr.net/npm/evals/vendor}"
 # Prefer the richer, bundled-prompt `npx evals` experience when Node is present.
 # This shell script is the no-npm fallback; if npx exists, hand off to it.
 # Set ARIZE_SKIP_NPX=1 to force this shell path even when npx is available.
+#
+# Only hand off on Node 20+, which is what the Ink UI needs (see bin.js). Handing
+# an older Node to npx would swap a working flow for a hard failure — this script
+# needs no Node at all, so stay here instead. Keep the floor in step with bin.js.
+node_major() {
+  command -v node >/dev/null 2>&1 || return 1
+  v="$(node --version 2>/dev/null)" || return 1
+  v="${v#v}"
+  echo "${v%%.*}"
+}
 if [ -z "${ARIZE_SKIP_NPX:-}" ] && command -v npx >/dev/null 2>&1; then
-  exec npx --yes evals
+  major="$(node_major || echo 0)"
+  if [ "${major:-0}" -ge 20 ] 2>/dev/null; then
+    exec npx --yes evals
+  fi
+  echo "Node $(node --version 2>/dev/null || echo 'not found') is too old for the npx UI — continuing without it."
+  echo
 fi
 
 # Supported agents: id | display label | argv to start the REPL seeded with a prompt.
@@ -136,12 +151,62 @@ choose_agent() {
 CHOSEN="$(choose_agent)" || exit 1
 
 # A directory, not a bare temp file: Step 4A looks for the offline bundle *beside*
-# the prompt, so the prompt needs a directory of its own rather than sharing
-# /tmp with everything else.
-PROMPT_DIR="$(mktemp -d "${TMPDIR:-/tmp}/arize-onboarding-XXXXXX")"
+# the prompt, so the prompt needs a directory of its own.
+#
+# ~/.arize/onboarding, not a temp path: the agent is asked to read a file outside
+# its workspace, and that approval is one a human can grant with confidence for a
+# named path but not for /tmp/arize-onboarding-8fJ2x1. Sits beside Step 4A's own
+# ~/.arize/harness install, which we must never touch.
+#
+# Cleared at launch and left in place afterwards — no EXIT trap — so the agent can
+# still reach the prompt in a session that outlives this script, and so we can
+# exec rather than waiting on a child just to clean up.
+clear_bundle_dir() {
+  # Delete only what we stage, by name. Wheel filenames carry a version we can't
+  # know, so those go by extension, non-recursively, in this directory only.
+  rm -f "$1"/*.whl
+  rm -f "$1/harness-install.sh" "$1/harness-install.bat" \
+        "$1/LICENSE-coding-harness-tracing" "$1/MANIFEST"
+  # rmdir, never rm -r: a non-empty directory fails here instead of taking
+  # something we didn't put there down with it.
+  rmdir "$1" 2>/dev/null
+}
+
+# Never a recursive delete. Worst case on a bad path is unlinking a few names that
+# don't exist. Keep this list in step with cli.js OFFLINE_FILES and start.ps1.
+clear_prompt_dir() {
+  [ -d "$1" ] || return 0
+  for sub in "$1"/arize-offline "$1"/.staging*; do
+    [ -d "$sub" ] && clear_bundle_dir "$sub"
+  done
+  rm -f "$1/onboarding-prompt.md" "$1/harness.env"
+  # Anything left is not ours: bail out and let the caller fall back.
+  rmdir "$1" 2>/dev/null || { [ -z "$(ls -A "$1" 2>/dev/null)" ]; return $?; }
+}
+
+# $HOME unset (containers, cron, some CI) must degrade, never abort — hence
+# ${HOME:-} and not ${HOME:?}. Same for a read-only home, a directory owned by a
+# past `sudo`, or a full disk: try the work, fall back to a temp dir on failure.
+PROMPT_DIR=""
+if [ -n "${ARIZE_ONBOARDING_DIR:-}" ]; then
+  CANDIDATE="$ARIZE_ONBOARDING_DIR"
+elif [ -n "${HOME:-}" ]; then
+  CANDIDATE="$HOME/.arize/onboarding"
+else
+  CANDIDATE=""
+fi
+
+if [ -n "$CANDIDATE" ] && clear_prompt_dir "$CANDIDATE" \
+  && mkdir -p "$CANDIDATE" 2>/dev/null && chmod 700 "$CANDIDATE" 2>/dev/null; then
+  PROMPT_DIR="$CANDIDATE"
+else
+  PROMPT_DIR="$(mktemp -d "${TMPDIR:-/tmp}/arize-onboarding-XXXXXX")"
+  if [ -n "$CANDIDATE" ]; then
+    echo "Couldn't use $CANDIDATE — staging in $PROMPT_DIR instead."
+    echo "Instrumenting an app still works; tracing this coding agent (Step 4A) needs a writable home directory."
+  fi
+fi
 PROMPT_FILE="$PROMPT_DIR/onboarding-prompt.md"
-cleanup() { rm -rf "$PROMPT_DIR"; }
-trap cleanup EXIT
 
 if command -v curl >/dev/null 2>&1; then
   fetch() { curl -fsSL "$1" -o "$2"; }
@@ -198,18 +263,21 @@ if ! stage_offline_bundle; then
   echo "  (not available — Step 4A will download the installer instead)"
 fi
 
-SEED="Read the file $PROMPT_FILE and follow it to set up Arize AX tracing in this project, walking me through each step and asking me questions as needed."
+# No "in this project": this may well be run from a home directory, and Step 4 is
+# what scopes the work. Keep this wording in step with cli.js and start.ps1.
+SEED="Read the file '$PROMPT_FILE' and follow it to set up Arize AX tracing, walking me through each step and asking me questions as needed."
 
 echo "Launching $(agent_label "$CHOSEN")…"
 echo
 
 # Launch interactive + seeded, on the real terminal. No skip-permissions —
 # the agent's approval model and the prompt's own approval gate must stay intact.
-# Run as a child (not exec) so the EXIT trap cleans up the temp prompt file.
+# exec, now that the staging directory persists and there is no EXIT trap to run:
+# the agent replaces this shell, so signals and the exit code reach it directly.
 run_agent() {
   case "$CHOSEN" in
-    copilot|gemini) "$CHOSEN" -i "$SEED" ;;
-    *)              "$CHOSEN" "$SEED" ;;
+    copilot|gemini) exec "$CHOSEN" -i "$SEED" ;;
+    *)              exec "$CHOSEN" "$SEED" ;;
   esac
 }
 

package/vendor/MANIFEST

@@ -1,6 +1,6 @@
 a9cede65f98d3415b13e1c6a437d0fdee27286bfff0186dd72bb65d9869b7864  LICENSE-coding-harness-tracing
 62f22742b58a1a33014a2b6b706588a8d7e2a88ae7bd1a6ebe8c992928483775  certifi-2026.7.22-py3-none-any.whl
-e17d86afa36365ddd451325b7b2ed3e35ba85801fb10e35401fded609095f550  coding_harness_tracing-0.1.0-py3-none-any.whl
+c41daa85c05f38eabd63fd0c7a2c021ffc0ecb7483485f7dd1b8a637f983b778  coding_harness_tracing-0.1.0-py3-none-any.whl
 aca14e09ca99c253d2a142e2d5c5c38c8c7cf90840d3cb10b241c29929871fa4  harness-install.bat
 a7ba18a98280e236b20ab4cb9f8a4006068b454586a2af50bf23fb0dbe70583d  harness-install.sh
 b81ee9561e9ca4004139c6cbba3a238c32b03e4894671e181b671e8cb8425d61  python_dotenv-1.2.1-py3-none-any.whl

package/vendor/harness-pin.lock.json

@@ -5,7 +5,7 @@
   "source": "git",
   "wheel": "coding_harness_tracing-0.1.0-py3-none-any.whl",
   "wheelVersion": "0.1.0",
-  "wheelSha256": "e17d86afa36365ddd451325b7b2ed3e35ba85801fb10e35401fded609095f550",
+  "wheelSha256": "c41daa85c05f38eabd63fd0c7a2c021ffc0ecb7483485f7dd1b8a637f983b778",
   "files": [
     "LICENSE-coding-harness-tracing",
     "MANIFEST",