evals 2.2.7 → 2.3.0

package/README.md

@@ -1,53 +1,55 @@
 # evals
 
-An npm package for Arize evals functionality.
+Go from zero to your first [Arize AX](https://arize.com/docs/ax) traces in one command. `evals` launches your coding agent, seeded with a guided onboarding prompt that walks you through signing up, instrumenting your app, and confirming traces land.
 
-## About Arize AI
+## Quick start
 
-Arize AI is a leading company in AI observability and evaluation, dedicated to ensuring that artificial intelligence systems operate reliably in real-world applications. Founded in 2019, Arize provides tools that help machine learning teams monitor, troubleshoot, and improve model performance across various domains, including structured data, computer vision, and large language models (LLMs).
-
-## Arize Phoenix
+```bash
+npx evals
+```
 
-**Arize Phoenix** is an open-source library designed for LLM tracing and evaluation. It enables developers to evaluate, experiment, and optimize AI products in real time. Key features include:
+You'll see a picker of the coding agents installed on your machine (Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI). Pick one and it launches in your current directory, seeded with the onboarding prompt — so run `npx evals` from the project you want to instrument.
 
-- **Application Tracing**: Collects LLM application data with automatic instrumentation, providing comprehensive visibility into model operations
-- **Interactive Prompt Playground**: Offers a flexible environment for prompt and model iteration, allowing users to compare prompts, visualize outputs, and debug failures within their workflow
-- **Streamlined Evaluations and Annotations**: Facilitates efficient assessment and documentation of model performance
+No coding agent installed? The picker shows install links instead.
 
-Phoenix is built on OpenTelemetry, ensuring seamless setup, full transparency, and no vendor lock-in. It's perfect for teams who want to get started with LLM observability and evaluation in a fully local, open-source environment.
+### Without Node
 
-## Arize AX
+If you don't have Node, use the shell launchers (they detect an agent and fetch the prompt from this published package via jsDelivr):
 
-**Arize AX** is the enterprise AI engineering platform that extends the capabilities of Phoenix, offering a comprehensive suite for development, evaluation, and observability. Key features include:
+```bash
+# macOS / Linux
+bash <(curl -fsSL https://cdn.jsdelivr.net/npm/evals/start.sh)
+```
 
-- **Prompts**: Tools for prompt engineering, including a prompt playground, prompt hub for management and versioning, and a prompt builder powered by AI
-- **Experiments**: Systematic A/B testing of prompts against large datasets to optimize model performance
-- **Tracing**: Detailed tracing capabilities to monitor and debug AI applications effectively
-- **Evaluation**: Comprehensive evaluation metrics and tools to assess model outputs and performance
-- **Alyx**: An AI engineering agent embedded within the platform to assist with various tasks
+```powershell
+# Windows (PowerShell)
+irm https://cdn.jsdelivr.net/npm/evals/start.ps1 | iex
+```
 
-Arize AX is designed to support teams and organizations with larger data needs, providing robust support, collaboration features, and multiple deployment options including SaaS, Virtual Private Cloud (VPC), and Arize Private Connect.
+When Node **is** present, these hand off to `npx evals` for the richer UI.
 
+## What it does
 
-## Installation
+1. Detects an installed coding agent (never executes it — just a PATH lookup).
+2. Launches the agent in your current directory, seeded with the bundled onboarding prompt ([`onboarding-prompt.md`](./onboarding-prompt.md)).
+3. The agent walks you through: create/sign in to Arize AX → detect your stack → instrument it → verify your first traces.
 
-```bash
-npm install evals
-```
+The agent runs with its **normal permission model** — `evals` never passes skip-permissions, so you approve each step, and the prompt itself gates real changes on your confirmation.
 
-The post-install script will automatically launch the CLI interface.
+## Options
 
-## Usage
+| Variable | Applies to | Effect |
+|----------|-----------|--------|
+| `ARIZE_AGENT=<id>` | `start.sh` / `start.ps1` | Skip the picker and use this agent (`claude`, `codex`, `cursor-agent`, `copilot`, `gemini`). |
+| `ARIZE_SKIP_NPX=1` | `start.sh` / `start.ps1` | Force the shell path even when Node/npx is available. |
+| `ARIZE_PROMPT_URL=<url>` | `start.sh` / `start.ps1` | Fetch the onboarding prompt from a custom URL (supports `file://`). |
 
-Run the CLI manually:
+The shell launchers also accept `--agent <id>`.
 
-```bash
-npm start
-```
+## About Arize
 
-Or use the binary:
+[Arize AX](https://arize.com/docs/ax) is the AI engineering platform for tracing, evaluating, and observing LLM and agent applications. Learn more at [arize.com](https://arize.com).
 
-```bash
-npx evals
-```
+## Contributing
 
+Development setup and the release process live in [CONTRIBUTING.md](https://github.com/Arize-ai/npm-evals/blob/main/CONTRIBUTING.md).

package/cli.js

@@ -3,12 +3,58 @@
 import React, { useState, useEffect } from 'react';
 import { render, Box, Text, useInput, useApp, Static } from 'ink';
 import Gradient from 'ink-gradient';
-import { exec } from 'child_process';
+import { exec, spawn } from 'child_process';
+import { existsSync, readFileSync, writeFileSync, mkdtempSync, realpathSync } from 'fs';
+import { join } from 'path';
+import { tmpdir } from 'os';
+import { fileURLToPath } from 'url';
 
 const e = React.createElement;
 
-// Diagnostic: Uncomment to verify script is running
-console.error('CLI script started, platform:', process.platform, 'isTTY:', process.stdin.isTTY);
+// The onboarding prompt is bundled with this package (onboarding-prompt.md, a
+// copy of the docs landing-page prompt). We read it, write it to a temp file,
+// and tell the agent to read that file — a ~27 KB prompt is too large to pass
+// reliably as a command-line argument.
+// TODO: sync this copy with the docs source (arize.com/docs) later.
+const BUNDLED_PROMPT_PATH = fileURLToPath(new URL('./onboarding-prompt.md', import.meta.url));
+
+// Write the bundled prompt to a temp file and return a short seed instruction
+// that points the agent at it.
+function prepareSeedPrompt() {
+  const promptText = readFileSync(BUNDLED_PROMPT_PATH, 'utf8');
+  const dir = mkdtempSync(join(tmpdir(), 'arize-onboarding-'));
+  const promptFile = join(dir, 'onboarding-prompt.md');
+  writeFileSync(promptFile, promptText, 'utf8');
+  return `Read the file ${promptFile} and follow it to set up Arize AX tracing in this project, walking me through each step and asking me questions as needed.`;
+}
+
+// Coding agents we can launch interactively, seeded with the prompt.
+// `args(seed)` returns the argv that starts the agent's REPL pre-loaded with `seed`.
+export const AGENTS = [
+  { id: 'claude', label: 'Claude Code', bin: 'claude', args: (s) => [s], installUrl: 'https://docs.claude.com/en/docs/claude-code' },
+  { id: 'codex', label: 'OpenAI Codex', bin: 'codex', args: (s) => [s], installUrl: 'https://developers.openai.com/codex/cli' },
+  { id: 'cursor-agent', label: 'Cursor', bin: 'cursor-agent', args: (s) => [s], installUrl: 'https://docs.cursor.com/en/cli/overview' },
+  { id: 'copilot', label: 'GitHub Copilot', bin: 'copilot', args: (s) => ['-i', s], installUrl: 'https://github.com/features/copilot/cli' },
+  { id: 'gemini', label: 'Gemini CLI', bin: 'gemini', args: (s) => ['-i', s], installUrl: 'https://github.com/google-gemini/gemini-cli' }
+];
+
+// PATH scan — detects an agent without executing it (running it could hang).
+export function isInstalled(bin) {
+  const path = process.env.PATH || '';
+  const dirs = path.split(process.platform === 'win32' ? ';' : ':');
+  const exts = process.platform === 'win32'
+    ? (process.env.PATHEXT || '.EXE;.CMD;.BAT').split(';')
+    : [''];
+  for (const dir of dirs) {
+    if (!dir) continue;
+    for (const ext of exts) {
+      if (existsSync(join(dir, bin + ext)) || existsSync(join(dir, bin + ext.toLowerCase()))) {
+        return true;
+      }
+    }
+  }
+  return false;
+}
 
 // "ARIZE EVALS" - EXACTLY matched width (both 44 chars)
 const largeLogo = `
@@ -38,30 +84,23 @@ const mediumLogo = `
 const smallLogo = `ARIZE
 EVALS`;
 
-const OPTIONS = [
-  {
-    name: 'Evals Primer',
-    subtext: 'Learn the Craft',
-    url: 'https://arize.com/llm-evaluation/?utm_source=npmevals'
-  },
-  {
-    name: 'Arize AX SaaS',
-    subtext: 'Enterprise Agent Evaluation, Best in Class',
-    url: 'https://arize.com/docs/ax?utm_source=npmevals'
-  },
-  {
-    name: 'Arize Phoenix',
-    subtext: 'OSS Agent Evals & Traces, Fully Local',
-    url: 'https://arize.com/docs/phoenix?utm_source=npmevals'
-  },
-  {
-    name: 'Book an Eval Assessment',
-    subtext: 'Improve Your Agent Today',
-    url: 'https://arize.com/get-an-eval-assessment?utm_source=npmevals'
+// Tag every URL the app opens with utm_source=npmevals (idempotent). Applied
+// centrally in openUrlInBrowser so all opened links (the agent install links)
+// carry it. Does not touch URLs inside the onboarding prompt.
+export function withUtm(url) {
+  try {
+    const u = new URL(url);
+    if (!u.searchParams.has('utm_source')) {
+      u.searchParams.set('utm_source', 'npmevals');
+    }
+    return u.toString();
+  } catch {
+    return url;
   }
-];
+}
 
-function openUrlInBrowser(url) {
+function openUrlInBrowser(rawUrl) {
+  const url = withUtm(rawUrl);
   const command = process.platform === 'win32'
     ? `start ${url}`
     : process.platform === 'darwin'
@@ -133,13 +172,41 @@ function MenuItem({ name, subtext, isSelected, shimmerPos }) {
   );
 }
 
-// Main App component
-function App() {
+// Build the coding-agent picker items — the entry screen for `npx evals`.
+// Detected agents become launch items; if none are found we offer install
+// links instead so the screen is never a dead end.
+export function buildAgentItems() {
+  const detected = AGENTS.filter(a => isInstalled(a.bin));
+  if (detected.length > 0) {
+    return {
+      items: detected.map(a => ({
+        name: a.label,
+        subtext: 'Launch and walk me through setup',
+        launch: a
+      })),
+      none: false
+    };
+  }
+  return {
+    items: AGENTS.map(a => ({
+      name: `Install ${a.label}`,
+      subtext: 'No supported agent detected — open install docs',
+      url: a.installUrl
+    })),
+    none: true
+  };
+}
+
+// Main App component — goes straight into the onboarding flow: pick a coding
+// agent and launch it seeded with the prompt (no top-level menu).
+function App({ onDone }) {
+  const [agentState] = useState(() => buildAgentItems());
   const [selectedIndex, setSelectedIndex] = useState(0);
   const [shimmerPos, setShimmerPos] = useState(-7); // Start off-screen
   const { exit } = useApp();
 
-  const selectedName = OPTIONS[selectedIndex].name;
+  const items = agentState.items;
+  const selectedName = items[selectedIndex].name;
 
   // Character-by-character shimmer animation
   useEffect(() => {
@@ -169,30 +236,41 @@ function App() {
     }
 
     if (key.upArrow || input === 'k') {
-      setSelectedIndex(prev => (prev > 0 ? prev - 1 : OPTIONS.length - 1));
+      setSelectedIndex(prev => (prev > 0 ? prev - 1 : items.length - 1));
     }
 
     if (key.downArrow || input === 'j') {
-      setSelectedIndex(prev => (prev < OPTIONS.length - 1 ? prev + 1 : 0));
+      setSelectedIndex(prev => (prev < items.length - 1 ? prev + 1 : 0));
     }
 
     if (key.return) {
-      const selected = OPTIONS[selectedIndex];
-      exit();
-      setTimeout(() => {
-        console.log(`\nOpening ${selected.url} in your browser...\n`);
-        openUrlInBrowser(selected.url);
-      }, 100);
+      const selected = items[selectedIndex];
+
+      if (selected.launch) {
+        onDone({ type: 'launch', agent: selected.launch });
+        exit();
+        return;
+      }
+
+      if (selected.url) {
+        onDone({ type: 'url', url: selected.url });
+        exit();
+        return;
+      }
     }
   });
 
+  const heading = agentState.none
+    ? 'No coding agent found on your PATH. Install one, then re-run:'
+    : 'Choose your coding agent to instrument your app:';
+
   return e(Box, { flexDirection: 'column', padding: 1 },
-    e(Static, { items: ['header'] }, () => e(Header)),
+    e(Static, { items: ['header'] }, (item) => e(Header, { key: item })),
     e(Box, { marginBottom: 1 },
-      e(Text, { bold: true, color: 'white' }, 'Select an option:')
+      e(Text, { bold: true, color: 'white' }, heading)
     ),
     e(Box, { flexDirection: 'column', paddingX: 1 },
-      ...OPTIONS.map((option, index) =>
+      ...items.map((option, index) =>
         e(MenuItem, {
           key: index.toString(),
           name: option.name,
@@ -203,44 +281,99 @@ function App() {
       )
     ),
     e(Box, { marginTop: 1 },
-      e(Text, { dimColor: true }, 'Enter to select · ↑↓ to navigate · Esc to cancel')
+      e(Text, { dimColor: true }, 'Enter to select · ↑↓ to navigate · Esc to quit')
     )
   );
 }
 
-// Run the app
-// Check if we have an interactive terminal (required for Ink)
-if (!process.stdin.isTTY || !process.stdout.isTTY) {
-  console.error('Error: This command requires an interactive terminal.');
-  console.error('Please run `npx evals` from your terminal (not from a script or non-interactive environment).');
-  process.exit(1);
-}
+// Launch a coding agent on the real terminal, seeded with the onboarding prompt.
+// Ink has fully unmounted by this point, so the child inherits a clean TTY.
+function launchAgent(agent) {
+  const isWin = process.platform === 'win32';
+
+  let seed;
+  try {
+    seed = prepareSeedPrompt();
+  } catch (err) {
+    console.error(`Could not read the onboarding prompt: ${err.message}`);
+    process.exit(1);
+  }
 
-// Handle uncaught errors
-process.on('uncaughtException', (error) => {
-  console.error('Uncaught exception:', error.message);
-  console.error(error.stack);
-  process.exit(1);
-});
-
-process.on('unhandledRejection', (reason, promise) => {
-  console.error('Unhandled rejection at:', promise);
-  console.error('Reason:', reason);
-  process.exit(1);
-});
-
-// Ensure stdout is not buffered (important for Windows)
-if (process.stdout.isTTY) {
-  process.stdout.setEncoding('utf8');
+  console.log(`\nLaunching ${agent.label}…\n`);
+
+  const child = spawn(agent.bin, agent.args(seed), {
+    stdio: 'inherit',
+    shell: isWin // .cmd/.bat shims on Windows need the shell to resolve
+  });
+
+  child.on('error', (err) => {
+    console.error(`Could not launch ${agent.bin}: ${err.message}`);
+    console.error(`Make sure "${agent.bin}" is on your PATH, then try again.`);
+    process.exit(1);
+  });
+
+  child.on('close', (code) => {
+    process.exit(code || 0);
+  });
 }
 
-try {
-  render(e(App), { patchConsole: false });
-} catch (error) {
-  console.error('Failed to render interactive CLI:', error.message);
-  console.error(error.stack);
-  if (process.platform === 'win32') {
-    console.error('\nNote: If you\'re using cmd.exe, try running in PowerShell or Git Bash instead.');
+// Render the interactive app and act on the user's choice. Exported so it can
+// be driven explicitly; only auto-runs when this file is the entry point (see
+// the guard below), so importing it in tests doesn't launch the TUI.
+export async function main() {
+  // Ink needs an interactive terminal.
+  if (!process.stdin.isTTY || !process.stdout.isTTY) {
+    console.error('Error: This command requires an interactive terminal.');
+    console.error('Please run `npx evals` from your terminal (not from a script or non-interactive environment).');
+    process.exit(1);
+  }
+
+  // Handle uncaught errors
+  process.on('uncaughtException', (error) => {
+    console.error('Uncaught exception:', error.message);
+    console.error(error.stack);
+    process.exit(1);
+  });
+
+  process.on('unhandledRejection', (reason, promise) => {
+    console.error('Unhandled rejection at:', promise);
+    console.error('Reason:', reason);
+    process.exit(1);
+  });
+
+  // Ensure stdout is not buffered (important for Windows)
+  if (process.stdout.isTTY) {
+    process.stdout.setEncoding('utf8');
   }
-  process.exit(1);
+
+  try {
+    let result = null;
+    const app = render(e(App, { onDone: (r) => { result = r; } }), { patchConsole: false });
+
+    await app.waitUntilExit();
+
+    if (!result) {
+      process.exit(0); // user quit / cancelled
+    } else if (result.type === 'url') {
+      console.log(`\nOpening ${result.url} in your browser...\n`);
+      openUrlInBrowser(result.url);
+    } else if (result.type === 'launch') {
+      launchAgent(result.agent);
+    }
+  } catch (error) {
+    console.error('Failed to render interactive CLI:', error.message);
+    console.error(error.stack);
+    if (process.platform === 'win32') {
+      console.error('\nNote: If you\'re using cmd.exe, try running in PowerShell or Git Bash instead.');
+    }
+    process.exit(1);
+  }
+}
+
+// Only run when invoked directly (as `evals`/`node cli.js`), not when imported.
+const invokedDirectly =
+  process.argv[1] &&
+  realpathSync(process.argv[1]) === realpathSync(fileURLToPath(import.meta.url));
+if (invokedDirectly) {
+  await main();
 }

package/onboarding-prompt.md

@@ -0,0 +1,355 @@
+# Get Started With Arize AX
+
+Guide the user from zero to their first traces in Arize AX. Use the AX CLI, the Arize agent skills, and the official docs.
+
+Work through the flow below in order. Installing the AX CLI and Arize skills, authenticating, and listing your spaces are all fine before approval — those are your own tooling. But do not create AX resources (like API keys), edit application files, or install the app's tracing dependencies before the user approves the plan in Step 5.
+
+## Step 0: Welcome and confirm
+
+Greet the user and show the plan, then ask before doing anything:
+
+```text
+Welcome to Arize AX. I'll get you set up with tracing. Here's what I'll do:
+
+1. Install the AX CLI and Arize skills
+2. Create a free Arize AX account or sign you in
+3. Instrument this project or a new app
+4. Capture traces from your app
+
+Shall I proceed?
+```
+
+Do not proceed until the user approves.
+
+## Prerequisites
+
+The AX CLI must be **arize-ax-cli `0.28.0` or newer** (Step 1 installs the latest) and needs **Python 3.11+** — a hard requirement. If Python is missing, stop and have the user install it from https://www.python.org/downloads/ and re-run. **Node.js 18+ with npx** is optional but installs the Arize skills; without it, don't stop — continue and use the docs paths in Step 6 (the Vercel AI SDK v7 starter needs Node.js 22+).
+
+## Step 1: Install or update the AX CLI and Arize skills
+
+Check the version with `ax --version`. If it's missing or older than the version above, install or upgrade `arize-ax-cli` — prefer `pipx`, otherwise pip:
+
+```bash
+pipx install arize-ax-cli   # already installed: pipx upgrade arize-ax-cli
+# without pipx:
+python3 -m pip install --upgrade arize-ax-cli
+```
+
+If npx is available, install the Arize agent skills; otherwise skip and continue (Step 6 falls back to the docs paths):
+
+```bash
+npx skills add Arize-ai/arize-skills --skill '*' --yes
+```
+
+This writes the skills to disk but does **not** load them into your current session — a mid-session install is not in your invocable skill list, so don't call it by name. When you need a skill (Step 6), load it by reading its `SKILL.md` directly from the install path (the command prints where it wrote them). Skill directories are named `arize-<name>`, e.g. `arize-instrumentation`.
+
+## Step 2: Authenticate the CLI
+
+The CLI authenticates through a profile named `default`, and every later `ax` command — including the trace check at the end — uses that profile. A bare `ARIZE_API_KEY` in the environment does **not** authenticate the CLI on its own; a profile must exist. Get one working before continuing.
+
+First check whether the CLI is already authenticated:
+
+```bash
+ax spaces list --limit 1 --output json
+```
+
+If this succeeds, a profile is already set up — skip the rest of this step.
+
+If it fails, first check whether a `default` profile already exists from a prior run:
+
+```bash
+ax profiles list
+```
+
+If a `default` profile exists but the probe failed, it's signed out or expired, not missing — for an OAuth profile, refresh it with `ax auth login` (see the OAuth handling below) and re-run the probe rather than recreating it. Only create a new profile when none exists (or an existing api-key profile has an invalid key). To create one, look for an existing key first: check for `ARIZE_API_KEY` in the environment **or** the project's `.env` / `.env.local`.
+
+- **A key is available** — the user already has an account and key, so skip the browser flow. Resolve the value (never print it): use `$ARIZE_API_KEY` if it's exported in your shell; if it's only in a dotenv file, load it from there for this one command. Then create an api-key profile and re-run the probe to confirm:
+
+  ```bash
+  # if the key is only in a dotenv file, load it first (nothing is printed);
+  # point at the file the app uses (.env or .env.local):
+  export ARIZE_API_KEY="$(grep -E '^ARIZE_API_KEY=' .env 2>/dev/null | tail -1 | cut -d= -f2-)"
+  ax profiles create default --auth-method api-key --api-key "$ARIZE_API_KEY"
+  ```
+
+  Reuse this key throughout — do **not** create a new one in Step 6.
+
+- **No key anywhere** — sign up or sign in with browser OAuth:
+
+  ```bash
+  ax profiles create default --auth-method oauth
+  ```
+
+Always pass the positional profile name `default`. Without it, the CLI prompts `profile name [default]:`, receives EOF from an agent-run command, and exits with `Goodbye!` without creating a profile.
+
+The rest of this step applies only to the **browser OAuth** branch.
+
+Creating an OAuth profile **is** the browser login flow. Treat it as an interactive browser handoff: it opens a browser, starts a localhost callback server such as `127.0.0.1:<port>/callback`, and waits for the browser redirect. The command must stay alive until the redirect lands and the CLI exits on its own.
+
+While the OAuth command is waiting, do not close its stdin, send Ctrl-C, `pkill` the AX process, start a second auth command, or run an auth probe. Any of these aborts the in-progress browser flow. There is one deliberate exception — the new email/password signup case described below, where the callback never lands and the command must be restarted.
+
+After launching the OAuth command, tell the user:
+
+```text
+A browser window is opening for Arize AX sign-in.
+
+- Already have an account, or signing in with Google/SSO or an existing
+  email/password? Just finish in the browser and I'll continue automatically.
+- Creating a BRAND-NEW account with email and password? Arize emails you a
+  validation link. That link does NOT complete the CLI login, so this command
+  will hang. Click the link to finish creating your account, then tell me.
+
+Did you just sign up for a new account with email and password?
+```
+
+**If the user says yes (new email/password signup):** the localhost callback will never fire and the command will wait forever, so this is the one time you break the "keep it alive" rule. Once they confirm they've clicked the validation link and their account exists, terminate the waiting OAuth command (send Ctrl-C / kill that process), then re-run it:
+
+```bash
+ax profiles create default --auth-method oauth
+```
+
+The second run is now a plain sign-in for the validated account. Its callback lands normally, so handle it with all the standard rules below — keep it alive and wait for it to exit on its own.
+
+**Otherwise (existing account, SSO, or existing email/password):** wait for the command to exit on its own. Treat exit code `0`, or CLI success output such as `Configuration saved to profile 'default'` or `Active profile set`, as the primary completion signal.
+
+Only after the OAuth command completes, verify authentication with a non-secret probe:
+
+```bash
+ax spaces list --limit 1 --output json
+```
+
+If the probe succeeds, continue. If the default OAuth profile already existed and the probe returns an authentication error, the profile is signed out or expired — run the fallback:
+
+```bash
+ax auth login
+```
+
+Handle `ax auth login` with the same rules: keep its callback server alive, wait for it to exit or print a success line, then rerun the probe. Do not run `ax auth login` immediately after creating an OAuth profile; the profile creation already performed login.
+
+Do not treat `ax profiles show` alone as proof that OAuth completed; it can show profile configuration even when the user still needs to authenticate. Use it only for troubleshooting profile configuration, and never with `--expand`.
+
+Run no other remote `ax` command (creating keys, listing all spaces, inspecting resources) until the probe succeeds.
+
+## Step 3: Select the AX space
+
+List the spaces the authenticated profile can access:
+
+```bash
+ax spaces list --output json
+```
+
+If `ARIZE_SPACE_ID` is already set (environment or `.env` / `.env.local`), use it **only if it appears in that list** — that confirms the active profile can reach it (the app's existing key and your CLI profile may point at different spaces). If it's set but not in the list, don't trust it; select from the list instead. Otherwise: if exactly one space is returned, use it; if multiple, ask the user which to use; if none, guide the user to create a space in the Arize AX UI (or with an organization ID if available), then re-list.
+
+Capture the space's **ID** (not its display name) for `ARIZE_SPACE_ID`. The `arize-otel` tracing config requires the space ID; a name will silently fail to route traces.
+
+## Step 4: Inspect the folder and choose a path
+
+Inspect the current folder to decide whether an app already exists. Do not change files during inspection. Look for:
+
+- Python: `pyproject.toml`, `requirements.txt`, `setup.py`, `Pipfile`, imports.
+- TypeScript/JavaScript: `package.json`, lockfiles, `src`, `app`, `pages`, provider imports.
+- Go: `go.mod`. Java: `pom.xml`, `build.gradle`, `build.gradle.kts`.
+- Existing observability: `opentelemetry`, `TracerProvider`, `ARIZE_*`, `OTEL_*`, `OTLP_*`, Datadog, Honeycomb, Sentry, or other tracing.
+- Agent framework: identify it by its import/package — e.g. `langchain` / `langgraph`, `llama_index`, `crewai`, `autogen`, `semantic_kernel`, `pydantic_ai`, `google.adk`, `dspy`, `agent_framework`, and others. **Route on the framework, not the provider client it wraps** — an `openai` or `anthropic` import inside a framework app is not the thing to instrument; the framework almost certainly has its own integration (see Step 6).
+
+In a monorepo, check the git root to get oriented, but only instrument apps in or below the current working directory. If the project spans more than one language, instrument each one (route each through its own integration page in Step 6).
+
+Then branch on what you found:
+
+### If an app exists in the current folder
+
+Do not offer the starter-app path. Summarize the detected stack and offer to instrument it:
+
+```text
+I found a <language>/<framework> app in this folder. Want me to add Arize AX
+tracing to it?
+```
+
+If the user declines, stop here rather than falling back to a starter app.
+
+### If no app exists in the current folder
+
+Go straight to the starter-app choice — do not ask whether to instrument the empty folder. Ask which folder to create it in, then offer these choices:
+
+- OpenAI — Python or TypeScript (with a tool call)
+- Anthropic — Python (with a tool call; official AX auto-instrumentation)
+- LangChain with OpenAI — Python or TypeScript (a tool-using agent)
+- Vercel AI SDK with OpenAI — TypeScript (a tool-using agent)
+
+If the user picks an unsupported pairing, explain the supported options and ask again.
+
+## Step 5: Present the plan and get approval
+
+Before creating any remote resource, writing files, or installing dependencies, present one consolidated plan and wait for approval. This is the gate the intro refers to — nothing so far has modified the app or created AX resources.
+
+For an existing app, cover: detected language and framework, package manager, LLM provider or agent framework, any existing tracing to preserve, the env file that will be updated, the instrumentation packages and files that will change, whether a new AX user API key will be created or the existing `ARIZE_API_KEY` reused, and the project name that will be used.
+
+For a starter app, cover: the chosen provider and language, the target folder, the packages that will be installed, and the project name.
+
+Choose a default project name from the current folder or app name: lowercase it, replace spaces and unsupported punctuation with hyphens, and append `-arize-tracing` if it is too generic. Do not create an AX project explicitly — it is created on first trace ingestion.
+
+```text
+Here's my plan. Shall I proceed?
+```
+
+Do not continue until the user approves.
+
+## Step 6: Create the API key, write the env file, and instrument
+
+Only after approval, execute the plan in this order.
+
+### Choose the env file
+
+Pick the env file to match the app and use it for every variable below:
+
+- Next.js, Vite, or browser-adjacent TypeScript apps: `.env.local`
+- Python apps, Node scripts, backend services, or unknown type: `.env`
+- If the project already has exactly one of `.env` or `.env.local`, follow the existing convention.
+
+Do not read existing env file contents into chat. Preserve unrelated variables and never reveal their values.
+
+Make sure the env file is git-ignored before writing the API key to it — if the repo has a `.gitignore`, confirm it covers the file (add `.env` / `.env.local` if not); for a starter app you create, add one. The API key must never be committed to version control.
+
+### Write the non-secret variables
+
+Add these two lines to the env file you chose — as file contents, not shell commands. Create the file if needed; if a line already exists, update it in place rather than duplicating. They're not secret. Skip `ARIZE_SPACE_ID` if it's already set to the value you're using.
+
+```dotenv
+ARIZE_SPACE_ID=<space-id>
+ARIZE_PROJECT_NAME=<chosen-project-name>
+```
+
+### Create the API key
+
+**Skip this entirely if `ARIZE_API_KEY` was already present in Step 2** — reuse it and leave its env value untouched. Only create a key when you authenticated with browser OAuth and the app has no key yet.
+
+Create the key and write it into the env file in one step:
+
+```bash
+ax api-keys create --name "Local Arize AX tracing" --env-file .env
+```
+
+`--env-file` writes `ARIZE_API_KEY` atomically and **never prints it** — no temp file, no secret in your terminal or chat. Use `.env.local` if that's the app's convention; the file is created if missing, an existing `ARIZE_API_KEY` is replaced in place, and other variables are preserved.
+
+**Create the key exactly once** — a second `ax api-keys create` just orphans a still-active key. Git-ignore the env file (see above) *before* running this so the key is never committed.
+
+If key creation fails, have the user create one in the Arize AX UI and add it to the env file without exposing it in chat. Either way, create only one key.
+
+Handle the LLM provider's own key (e.g. `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`) in the same env file — the app can't make a call or produce traces without it:
+
+- **Existing app:** confirm the key is already present. If it's missing, ask the user for it and add it without echoing the value.
+- **New starter app:** add a placeholder line for the provider key the app needs (e.g. `OPENAI_API_KEY=` with an empty value) to the env file, so the user knows exactly what to fill in. Tell them to set it before running; if they share it now, fill it in without echoing the value.
+
+### Add instrumentation
+
+**Prefer the Arize instrumentation skill.** If you installed the skills in Step 1, load it now by reading its `SKILL.md` directly (a mid-session install isn't in your invocable list, so don't call it by name): read `arize-instrumentation/SKILL.md` from your agent's skills directory — project-level (`.claude/skills/`, `.cursor/skills/`, `.codex/…`, `.windsurf/…`) or the matching `~/.<agent>/…` global path. Follow the skill as your source of truth and **skip the rest of this section**. Use the docs fallback below only if the skill file doesn't exist (e.g. npx was missing in Step 1).
+
+**Route by the framework you detected — search the index for *its* name and follow that page.** Look up the framework you identified in Step 4 (not the provider it wraps) in the index at https://arize.com/docs/ax/integrations (machine-readable: https://arize.com/docs/llms.txt). Most agent frameworks have a dedicated page even though they aren't in the shortcuts below, so actually search the index before concluding a framework is unsupported — it lists every supported provider and framework with the exact, verified setup. Don't force a stack onto a listed framework's setup, and don't wander into unrelated Arize docs once you're on the right page.
+
+Common shortcuts:
+
+- OpenAI: https://arize.com/docs/ax/integrations/llm-providers/openai/openai-tracing
+- Anthropic: https://arize.com/docs/ax/integrations/llm-providers/anthropic/anthropic-tracing
+- LangChain Python: https://arize.com/docs/ax/integrations/python-agent-frameworks/langchain/langchain-tracing
+- LangChain.js: https://arize.com/docs/ax/integrations/ts-js-agent-frameworks/langchain/langchain-js
+- Vercel AI SDK v7: https://arize.com/docs/ax/integrations/ts-js-agent-frameworks/vercel/vercel-ai-sdk-v7-tracing
+- Vercel AI SDK v6 and earlier: https://arize.com/docs/ax/integrations/ts-js-agent-frameworks/vercel/vercel-ai-sdk-tracing
+
+**Only if the framework genuinely has no page in the index**, fall back in this order — never hand-roll a raw OpenTelemetry `TracerProvider` + OTLP exporter when a helper exists:
+
+1. Use a framework-specific OpenInference instrumentor if one exists (`openinference-instrumentation-<name>` / `@arizeai/openinference-instrumentation-<name>`), wired up with `arize-otel` per the manual-instrumentation guide; install it unpinned.
+1. Instrument the underlying provider (OpenAI, Anthropic, Bedrock, …) with its instrumentor **only if the framework calls the provider SDK directly**. Many agent frameworks instead drive the model through their own client layer and emit their own OpenTelemetry spans — a provider instrumentor captures **no traces** for those. Never reach for the provider instrumentor as a blind fallback just because you recognize an `openai`/`anthropic` client.
+1. Otherwise instrument manually via `arize-otel` (see https://arize.com/docs/ax/instrument/manual-instrumentation), or stop and ask the user if you still can't determine a setup.
+
+For existing apps:
+
+- Use the app's existing package manager and style.
+- Initialize tracing before LLM clients or frameworks are imported or created.
+- Preserve existing OpenTelemetry providers; add Arize as an additional exporter when needed.
+- Do not change business logic except to enable tracing.
+- If the app uses tools/function calls and the integration does not capture tool spans, add manual tool spans only after explaining that in the plan.
+
+For starter apps:
+
+- Every starter should make a real LLM call that invokes at least one tool (e.g. a calculator or a canned data lookup), with the model deciding to call it and the result fed back for a final answer — so the first traces show a multi-step trajectory (the LLM call plus the tool call as its own span), not one flat span.
+  - Agent frameworks (LangChain, Vercel AI SDK): define the tool the framework's way; the instrumentor captures the tool and agent spans automatically.
+  - Plain providers (OpenAI, Anthropic): use the provider's native tool/function calling, and add tool spans with the OpenInference decorators — `@tracer.tool` on the tool function and `@tracer.agent` on the top-level loop. These decorators require wrapping the tracer in `OITracer` from `openinference-instrumentation` (the raw `arize.otel.register()` tracer doesn't expose them and `@tracer.tool` will raise `AttributeError`); see https://arize.com/docs/ax/instrument/manual-instrumentation.
+- Ensure short-lived scripts flush/shut down the tracer provider before exit, or spans won't export.
+- Give a clear run command and note which provider key env var is needed.
+
+### Package guidance
+
+Install into the app's existing environment (its virtualenv if it has one) exactly the packages the detected framework's integration page lists — instrumentor names and peer dependencies differ per framework, so follow that page rather than copying from another stack or guessing versions.
+
+## Step 7: Run the app and poll for the first trace
+
+First confirm the LLM provider's API key is set in the env file (from Step 6); a missing provider key is the most common reason the run produces no traces.
+
+If you created a starter app in this flow, offer to run it for the user:
+
+```text
+Your starter app is ready. Want me to run it for you?
+```
+
+If they say yes, run it yourself with the run command, then poll. If they say no — or if you instrumented their existing app rather than creating a starter — tell them the exact run command and ask them to run it:
+
+```text
+Run your app with:
+
+<run command>
+
+It should make at least one LLM call. I'll poll Arize AX and let you know as
+soon as your first traces arrive.
+```
+
+Once the app has run (whether you ran it or the user did), poll for spans. Pass the actual project name and space ID as literal arguments — do not use `$ARIZE_PROJECT_NAME`/`$ARIZE_SPACE_ID`, which live in the env file and are not exported to your shell. Query every ~15 seconds for up to ~3 minutes.
+
+Span bodies can contain prompts, completions, tool arguments, and user data, so don't dump them into chat. Pipe the export through a counter that surfaces only how many spans arrived:
+
+```bash
+ax spans export "<project-name>" --space "<space-id>" --limit 5 --stdout \
+  | python3 -c 'import json,sys; d=json.load(sys.stdin); s=d if isinstance(d,list) else (d.get("spans") or d.get("data") or []); print(f"{len(s)} span(s) found")'
+```
+
+A non-zero count confirms traces are arriving. If you need to inspect a span to debug, write the export to a file outside the repo and read only the fields you need — never paste raw span bodies into chat.
+
+This uses the CLI profile from Step 2 (OAuth or api-key) — it works the same either way. If the export errors with an authentication failure, the profile isn't valid; re-run the Step 2 probe and re-authenticate, or fall back to having the user open the project in the Arize AX UI to confirm traces.
+
+- When spans come back, stop polling and continue to Step 8.
+- On timeout, do not fail silently. Tell the user no traces arrived yet, and give likely causes: app didn't make an LLM call, tracing initialized after the client was created, a short-lived script exited before flushing spans, or the wrong space/project/env file. Offer to re-check once they've run it again.
+
+Do not fabricate trace results. Only report traces the export command actually returned.
+
+## Step 8: Report the first traces with a link
+
+Once spans arrive, report the span count and give the user a link into Arize AX for the project. Point them at the UI to explore the trace contents rather than printing span bodies into chat.
+
+```text
+Your first traces are in Arize AX. Open project `<ARIZE_PROJECT_NAME>` here:
+https://app.arize.com/
+```
+
+If a space- or project-specific URL is known from the CLI or docs, prefer that deep link over the app root. Do not invent a URL structure you are not sure of; fall back to `https://app.arize.com/` plus instructions to select the project.
+
+## Step 9: Point at docs
+
+Finish with links so the user can go deeper:
+
+```text
+You're set up. To go further:
+
+- Docs index: https://arize.com/docs/ax
+- Tracing integrations: https://arize.com/docs/ax/integrations
+- AX CLI: https://arize.com/docs/api-clients/cli/overview
+```
+
+## Critical rules
+
+- Get the user's approval (Step 5) before creating AX resources, editing files, or installing dependencies.
+- Authenticate the CLI before any other `ax` call — every command, including the final trace check, uses the active profile; env vars alone don't authenticate it. On the browser OAuth branch, keep the command alive until its localhost callback completes (never close stdin, Ctrl-C, `pkill`, or probe while it waits) — sole exception: a brand-new email/password signup, whose link never calls back, so ask the user, then kill and re-run as a sign-in.
+- Never print, log, or summarize secrets in chat — API keys, env-file contents/values, or span bodies (prompts, completions, tool args, user data) — and never read env files into chat. Only report traces a command actually returned.
+- Create at most one AX API key: skip if `ARIZE_API_KEY` already exists (reuse it), otherwise a **single** `ax api-keys create --env-file <file>`. Git-ignore that file before creating the key; never create a second key.
+- Write the space **ID** (not its name) to `ARIZE_SPACE_ID`, or traces won't route; never create an AX project explicitly (it's made on first ingestion).
+- Initialize tracing before LLM clients are created, and flush/shut down the tracer before short-lived scripts exit. Vercel AI SDK v7 also needs Node.js 22+, `@ai-sdk/otel` registered, and `experimental_telemetry: { isEnabled: true }` per call.
+
+Docs: https://arize.com/docs/llms.txt

package/package.json

@@ -1,16 +1,15 @@
 {
   "name": "evals",
-  "version": "2.2.7",
-  "description": "Arize evals package",
+  "version": "2.3.0",
+  "description": "Arize AX onboarding — instrument your app with tracing via your coding agent",
   "type": "module",
   "main": "cli.js",
   "scripts": {
-    "postinstall": "echo \"run npx evals to complete setup\" && node spawn-interactive.js",
-    "start": "node cli.js"
+    "postinstall": "node -e \"console.log('Arize evals installed — run: npx evals')\"",
+    "start": "node cli.js",
+    "test": "node --test"
   },
   "dependencies": {
-    "@arizeai/phoenix-evals": "^0.6.5",
-    "@arizeai/phoenix-otel": "*",
     "ink": "^6.0.0",
     "ink-gradient": "^3.0.0",
     "react": "^19.0.0"
@@ -18,9 +17,18 @@
   "bin": {
     "evals": "./cli.js"
   },
+  "files": [
+    "cli.js",
+    "onboarding-prompt.md",
+    "start.sh",
+    "start.ps1"
+  ],
   "keywords": [
     "arize",
-    "evals"
+    "evals",
+    "tracing",
+    "observability",
+    "llm"
   ],
   "author": "",
   "license": "ISC"

package/start.ps1

@@ -0,0 +1,128 @@
+#!/usr/bin/env pwsh
+#
+# Arize AX onboarding launcher (Windows / PowerShell).
+#
+# Lets you pick an installed coding agent and launches it seeded with the
+# onboarding prompt to walk you through signup -> instrument -> first trace.
+# This is the no-npm path; if you have Node, `npx evals` gives the same thing
+# with a nicer UI.
+#
+# Run it:
+#   irm https://cdn.jsdelivr.net/npm/evals/start.ps1 | iex
+#
+# Non-interactive (irm|iex can't take params, so use the env var):
+#   $env:ARIZE_AGENT='claude'; irm .../start.ps1 | iex
+# Or via the scriptblock form, which does accept -Agent:
+#   & ([scriptblock]::Create((irm .../start.ps1))) -Agent claude
+
+[CmdletBinding()]
+param(
+  [string]$Agent = $env:ARIZE_AGENT
+)
+
+$ErrorActionPreference = 'Stop'
+
+# The prompt ships inside the public `evals` npm package; jsDelivr serves package
+# files over HTTP, so it's fetchable even though the source repo is private.
+# Override with $env:ARIZE_PROMPT_URL if needed.
+$PromptUrl = if ($env:ARIZE_PROMPT_URL) { $env:ARIZE_PROMPT_URL } else { 'https://cdn.jsdelivr.net/npm/evals/onboarding-prompt.md' }
+
+# Prefer the richer, bundled-prompt `npx evals` experience when Node is present.
+# This script is the no-npm fallback; if npx exists, hand off to it.
+# Set $env:ARIZE_SKIP_NPX=1 to force this shell path even when npx is available.
+if (-not $env:ARIZE_SKIP_NPX -and (Get-Command npx -ErrorAction SilentlyContinue)) {
+  npx --yes evals
+  exit $LASTEXITCODE
+}
+
+# Supported agents: id -> label, install URL, and whether the REPL is seeded with `-i`.
+$Agents = [ordered]@{
+  'claude'       = @{ Label = 'Claude Code';    Install = 'https://docs.claude.com/en/docs/claude-code'; Flag = $null }
+  'codex'        = @{ Label = 'OpenAI Codex';   Install = 'https://developers.openai.com/codex/cli';      Flag = $null }
+  'cursor-agent' = @{ Label = 'Cursor';         Install = 'https://docs.cursor.com/en/cli/overview';      Flag = $null }
+  'copilot'      = @{ Label = 'GitHub Copilot'; Install = 'https://github.com/features/copilot/cli';       Flag = '-i' }
+  'gemini'       = @{ Label = 'Gemini CLI';     Install = 'https://github.com/google-gemini/gemini-cli';   Flag = '-i' }
+}
+
+function Test-Agent([string]$id) {
+  return [bool](Get-Command $id -ErrorAction SilentlyContinue)
+}
+
+# Resolve which agent to launch.
+function Resolve-Agent {
+  if ($Agent) {
+    if (Test-Agent $Agent) { return $Agent }
+    Write-Host "Requested agent '$Agent' is not on your PATH." -ForegroundColor Yellow
+    return $null
+  }
+
+  $detected = @($Agents.Keys | Where-Object { Test-Agent $_ })
+
+  if ($detected.Count -eq 0) {
+    Write-Host "No supported coding agent found on your PATH." -ForegroundColor Yellow
+    Write-Host "Install one of these, then re-run:"
+    foreach ($id in $Agents.Keys) {
+      "{0,-14} {1}" -f $Agents[$id].Label, $Agents[$id].Install | Write-Host
+    }
+    return $null
+  }
+
+  if ($detected.Count -eq 1) { return $detected[0] }
+
+  Write-Host "Choose your coding agent:"
+  for ($i = 0; $i -lt $detected.Count; $i++) {
+    "  {0}) {1}" -f ($i + 1), $Agents[$detected[$i]].Label | Write-Host
+  }
+  while ($true) {
+    $choice = Read-Host "Enter a number [1-$($detected.Count)]"
+    if ($choice -match '^\d+$' -and [int]$choice -ge 1 -and [int]$choice -le $detected.Count) {
+      return $detected[[int]$choice - 1]
+    }
+    Write-Host "Invalid choice."
+  }
+}
+
+# --- Welcome ---
+Write-Host ""
+Write-Host "Arize AX" -ForegroundColor Magenta -NoNewline
+Write-Host " - Evals & Observability for Agentic AI" -ForegroundColor DarkGray
+Write-Host ""
+Write-Host "Let's get you tracing. I'll launch your coding agent with a guided prompt that"
+Write-Host "walks you through signup, instrumenting your app, and seeing your first traces."
+Write-Host ""
+
+$chosen = Resolve-Agent
+if (-not $chosen) { exit 1 }
+
+# Download the prompt to a temp file (rename the .tmp to .md rather than
+# leaving both behind).
+$tmp = New-TemporaryFile
+$promptFile = [System.IO.Path]::ChangeExtension($tmp.FullName, 'md')
+Rename-Item -Path $tmp.FullName -NewName (Split-Path $promptFile -Leaf)
+try {
+  Write-Host "Fetching the onboarding prompt..."
+  try {
+    Invoke-RestMethod -Uri $PromptUrl -OutFile $promptFile
+  } catch {
+    Write-Host "Failed to download the onboarding prompt from $PromptUrl" -ForegroundColor Red
+    Write-Host "Check your connection and try again."
+    exit 1
+  }
+
+  $seed = "Read the file $promptFile and follow it to set up Arize AX tracing in this project, walking me through each step and asking me questions as needed."
+
+  Write-Host ("Launching {0}..." -f $Agents[$chosen].Label)
+  Write-Host ""
+
+  # Launch interactive + seeded. No skip-permissions — the agent's approval
+  # model and the prompt's own approval gate must stay intact. External commands
+  # attach to the console, so the agent gets a real terminal.
+  $flag = $Agents[$chosen].Flag
+  if ($flag) {
+    & $chosen $flag $seed
+  } else {
+    & $chosen $seed
+  }
+} finally {
+  Remove-Item $promptFile -ErrorAction SilentlyContinue
+}

package/start.sh

@@ -0,0 +1,186 @@
+#!/usr/bin/env bash
+#
+# Arize AX onboarding launcher.
+#
+# Lets you pick an installed coding agent and launches it seeded with the
+# onboarding prompt to walk you through signup → instrument → first trace.
+# This is the no-npm path; if you have Node, `npx evals` gives the same thing
+# with a nicer UI.
+#
+# Run it (do NOT use `curl | bash` — that hijacks stdin and breaks the menu
+# and the launched agent). Use process substitution so the menu and the agent
+# both get a real terminal:
+#
+#   bash <(curl -fsSL https://cdn.jsdelivr.net/npm/evals/start.sh)
+#
+# Non-interactive: ARIZE_AGENT=claude bash <(curl -fsSL .../start.sh)
+#             or:  bash <(curl -fsSL .../start.sh) --agent claude
+
+set -euo pipefail
+
+# The prompt ships inside the public `evals` npm package; jsDelivr serves package
+# files over HTTP, so it's fetchable even though the source repo is private.
+# Override with ARIZE_PROMPT_URL if needed.
+PROMPT_URL="${ARIZE_PROMPT_URL:-https://cdn.jsdelivr.net/npm/evals/onboarding-prompt.md}"
+
+# Prefer the richer, bundled-prompt `npx evals` experience when Node is present.
+# This shell script is the no-npm fallback; if npx exists, hand off to it.
+# Set ARIZE_SKIP_NPX=1 to force this shell path even when npx is available.
+if [ -z "${ARIZE_SKIP_NPX:-}" ] && command -v npx >/dev/null 2>&1; then
+  exec npx --yes evals
+fi
+
+# Supported agents: id | display label | argv to start the REPL seeded with a prompt.
+# The seed prompt is appended as the final argument at launch.
+AGENT_IDS=(claude codex cursor-agent copilot gemini)
+agent_label() {
+  case "$1" in
+    claude)       echo "Claude Code" ;;
+    codex)        echo "OpenAI Codex" ;;
+    cursor-agent) echo "Cursor" ;;
+    copilot)      echo "GitHub Copilot" ;;
+    gemini)       echo "Gemini CLI" ;;
+    *)            echo "$1" ;;
+  esac
+}
+agent_install_url() {
+  case "$1" in
+    claude)       echo "https://docs.claude.com/en/docs/claude-code" ;;
+    codex)        echo "https://developers.openai.com/codex/cli" ;;
+    cursor-agent) echo "https://docs.cursor.com/en/cli/overview" ;;
+    copilot)      echo "https://github.com/features/copilot/cli" ;;
+    gemini)       echo "https://github.com/google-gemini/gemini-cli" ;;
+  esac
+}
+
+# Parse --agent flag
+AGENT="${ARIZE_AGENT:-}"
+while [[ $# -gt 0 ]]; do
+  case "$1" in
+    --agent) AGENT="${2:-}"; shift 2 ;;
+    --agent=*) AGENT="${1#*=}"; shift ;;
+    -h|--help)
+      echo "Usage: start.sh [--agent <claude|codex|cursor-agent|copilot|gemini>]"
+      exit 0 ;;
+    *) echo "Unknown argument: $1" >&2; exit 1 ;;
+  esac
+done
+
+# --- Welcome ---
+if [ -t 1 ] && [ "${TERM:-}" != "dumb" ]; then
+  _pink=$'\033[1;38;2;255;0;140m'; _dim=$'\033[2m'; _reset=$'\033[0m'
+else
+  _pink=""; _dim=""; _reset=""
+fi
+printf '\n%sArize AX%s %s— Evals & Observability for Agentic AI%s\n\n' "$_pink" "$_reset" "$_dim" "$_reset"
+printf "Let's get you tracing. I'll launch your coding agent with a guided prompt that\nwalks you through signup, instrumenting your app, and seeing your first traces.\n\n"
+
+# Detect installed agents (PATH lookup only — never executes them).
+DETECTED=()
+for id in "${AGENT_IDS[@]}"; do
+  if command -v "$id" >/dev/null 2>&1; then
+    DETECTED+=("$id")
+  fi
+done
+
+# Resolve which agent to launch.
+choose_agent() {
+  # Explicit choice wins if it's actually installed.
+  if [[ -n "$AGENT" ]]; then
+    if command -v "$AGENT" >/dev/null 2>&1; then
+      echo "$AGENT"; return 0
+    fi
+    echo "Requested agent '$AGENT' is not on your PATH." >&2
+    return 1
+  fi
+
+  if [[ ${#DETECTED[@]} -eq 0 ]]; then
+    echo "No supported coding agent found on your PATH." >&2
+    echo "Install one of these, then re-run:" >&2
+    for id in "${AGENT_IDS[@]}"; do
+      printf '  %-14s %s\n' "$(agent_label "$id")" "$(agent_install_url "$id")" >&2
+    done
+    return 1
+  fi
+
+  if [[ ${#DETECTED[@]} -eq 1 ]]; then
+    echo "${DETECTED[0]}"; return 0
+  fi
+
+  # Interactive menu — read from the real terminal, not the (piped) stdin.
+  if [[ ! -r /dev/tty ]]; then
+    echo "Multiple agents found but no terminal to prompt on. Set ARIZE_AGENT=<id>." >&2
+    return 1
+  fi
+  {
+    echo "Choose your coding agent:"
+    local i=1
+    for id in "${DETECTED[@]}"; do
+      printf '  %d) %s\n' "$i" "$(agent_label "$id")"
+      i=$((i + 1))
+    done
+  } >/dev/tty
+  local choice
+  while true; do
+    printf 'Enter a number [1-%d]: ' "${#DETECTED[@]}" >/dev/tty
+    read -r choice </dev/tty || return 1
+    if [[ "$choice" =~ ^[0-9]+$ ]] && (( choice >= 1 && choice <= ${#DETECTED[@]} )); then
+      echo "${DETECTED[$((choice - 1))]}"; return 0
+    fi
+    echo "Invalid choice." >/dev/tty
+  done
+}
+
+CHOSEN="$(choose_agent)" || exit 1
+
+# Download the prompt to a temp file. (mktemp templates must end in X's on
+# macOS, so add the .md extension afterwards.)
+PROMPT_FILE="$(mktemp "${TMPDIR:-/tmp}/arize-onboarding-XXXXXX")"
+mv "$PROMPT_FILE" "$PROMPT_FILE.md"
+PROMPT_FILE="$PROMPT_FILE.md"
+cleanup() { rm -f "$PROMPT_FILE"; }
+trap cleanup EXIT
+
+echo "Fetching the onboarding prompt…"
+download_failed() {
+  echo "Failed to download the onboarding prompt from $PROMPT_URL" >&2
+  echo "Check your connection and try again." >&2
+  exit 1
+}
+if command -v curl >/dev/null 2>&1; then
+  curl -fsSL "$PROMPT_URL" -o "$PROMPT_FILE" || download_failed
+elif command -v wget >/dev/null 2>&1; then
+  wget -qO "$PROMPT_FILE" "$PROMPT_URL" || download_failed
+else
+  echo "Need curl or wget to download the prompt." >&2
+  exit 1
+fi
+
+SEED="Read the file $PROMPT_FILE and follow it to set up Arize AX tracing in this project, walking me through each step and asking me questions as needed."
+
+echo "Launching $(agent_label "$CHOSEN")…"
+echo
+
+# Launch interactive + seeded, on the real terminal. No skip-permissions —
+# the agent's approval model and the prompt's own approval gate must stay intact.
+# Run as a child (not exec) so the EXIT trap cleans up the temp prompt file.
+run_agent() {
+  case "$CHOSEN" in
+    copilot|gemini) "$CHOSEN" -i "$SEED" ;;
+    *)              "$CHOSEN" "$SEED" ;;
+  esac
+}
+
+# Prefer inheriting stdin: TUI agents (e.g. codex/crossterm) need a read-write
+# terminal on fd 0. A `< /dev/tty` redirect opens fd 0 read-only and makes
+# crossterm panic ("reader source not set"). Only reach for /dev/tty when stdin
+# isn't already a terminal (the discouraged `curl | bash`), and open it
+# read-write with `<>`.
+if [ -t 0 ]; then
+  run_agent
+elif [ -r /dev/tty ]; then
+  run_agent <>/dev/tty
+else
+  echo "No interactive terminal available to launch the agent." >&2
+  exit 1
+fi

package/spawn-interactive.js

@@ -1,51 +0,0 @@
-import { spawn } from 'child_process';
-import { openSync } from 'fs';
-
-function tryOpenTTY() {
-  if (process.platform === 'win32') {
-    // On Windows, avoid CON device (issues in Node.js 22)
-    // Use isTTY check instead, and stdio: 'inherit' for spawning
-    if (process.stdin.isTTY && process.stdout.isTTY && process.stderr.isTTY) {
-      return { available: true, useInherit: true };
-    }
-    return { available: false };
-  } else {
-    // On macOS/Linux, use direct /dev/tty access
-    try {
-      const device = '/dev/tty';
-      openSync(device, 'r');
-      return { device, available: true, useInherit: false };
-    } catch (err) {
-      return { available: false };
-    }
-  }
-}
-
-const tty = tryOpenTTY();
-
-if (tty.available) {
-  const spawnOptions = tty.useInherit
-    ? { stdio: 'inherit', shell: true }
-    : {
-        stdio: [
-          openSync(tty.device, 'r'),
-          openSync(tty.device, 'w'),
-          openSync(tty.device, 'w')
-        ]
-      };
-
-  const child = spawn('node', ['./cli.js'], spawnOptions);
-
-  child.on('close', (code) => {
-    process.exit(code || 0);
-  });
-
-  child.on('error', (err) => {
-    console.error('Failed to start interactive CLI:', err.message);
-    console.log('Run `npx evals` to complete setup.');
-    process.exit(1);
-  });
-} else {
-  console.log('Run `npx evals` to complete setup.');
-  process.exit(1);
-}