evals 2.2.8 → 2.3.0

package/README.md

@@ -1,53 +1,55 @@
-# evals
-
-An npm package for Arize evals functionality.
-
-## About Arize AI
-
-Arize AI is a leading company in AI observability and evaluation, dedicated to ensuring that artificial intelligence systems operate reliably in real-world applications. Founded in 2019, Arize provides tools that help machine learning teams monitor, troubleshoot, and improve model performance across various domains, including structured data, computer vision, and large language models (LLMs).
-
-## Arize Phoenix
-
-**Arize Phoenix** is an open-source library designed for LLM tracing and evaluation. It enables developers to evaluate, experiment, and optimize AI products in real time. Key features include:
-
-- **Application Tracing**: Collects LLM application data with automatic instrumentation, providing comprehensive visibility into model operations
-- **Interactive Prompt Playground**: Offers a flexible environment for prompt and model iteration, allowing users to compare prompts, visualize outputs, and debug failures within their workflow
-- **Streamlined Evaluations and Annotations**: Facilitates efficient assessment and documentation of model performance
-
-Phoenix is built on OpenTelemetry, ensuring seamless setup, full transparency, and no vendor lock-in. It's perfect for teams who want to get started with LLM observability and evaluation in a fully local, open-source environment.
-
-## Arize AX
-
-**Arize AX** is the enterprise AI engineering platform that extends the capabilities of Phoenix, offering a comprehensive suite for development, evaluation, and observability. Key features include:
-
-- **Prompts**: Tools for prompt engineering, including a prompt playground, prompt hub for management and versioning, and a prompt builder powered by AI
-- **Experiments**: Systematic A/B testing of prompts against large datasets to optimize model performance
-- **Tracing**: Detailed tracing capabilities to monitor and debug AI applications effectively
-- **Evaluation**: Comprehensive evaluation metrics and tools to assess model outputs and performance
-- **Alyx**: An AI engineering agent embedded within the platform to assist with various tasks
-
-Arize AX is designed to support teams and organizations with larger data needs, providing robust support, collaboration features, and multiple deployment options including SaaS, Virtual Private Cloud (VPC), and Arize Private Connect.
-
-
-## Installation
-
-```bash
-npm install evals
-```
-
-The post-install script will automatically launch the CLI interface.
-
-## Usage
-
-Run the CLI manually:
-
-```bash
-npm start
-```
-
-Or use the binary:
-
-```bash
-npx evals
-```
-
+# evals
+
+Go from zero to your first [Arize AX](https://arize.com/docs/ax) traces in one command. `evals` launches your coding agent, seeded with a guided onboarding prompt that walks you through signing up, instrumenting your app, and confirming traces land.
+
+## Quick start
+
+```bash
+npx evals
+```
+
+You'll see a picker of the coding agents installed on your machine (Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI). Pick one and it launches in your current directory, seeded with the onboarding prompt — so run `npx evals` from the project you want to instrument.
+
+No coding agent installed? The picker shows install links instead.
+
+### Without Node
+
+If you don't have Node, use the shell launchers (they detect an agent and fetch the prompt from this published package via jsDelivr):
+
+```bash
+# macOS / Linux
+bash <(curl -fsSL https://cdn.jsdelivr.net/npm/evals/start.sh)
+```
+
+```powershell
+# Windows (PowerShell)
+irm https://cdn.jsdelivr.net/npm/evals/start.ps1 | iex
+```
+
+When Node **is** present, these hand off to `npx evals` for the richer UI.
+
+## What it does
+
+1. Detects an installed coding agent (never executes it — just a PATH lookup).
+2. Launches the agent in your current directory, seeded with the bundled onboarding prompt ([`onboarding-prompt.md`](./onboarding-prompt.md)).
+3. The agent walks you through: create/sign in to Arize AX → detect your stack → instrument it → verify your first traces.
+
+The agent runs with its **normal permission model** — `evals` never passes skip-permissions, so you approve each step, and the prompt itself gates real changes on your confirmation.
+
+## Options
+
+| Variable | Applies to | Effect |
+|----------|-----------|--------|
+| `ARIZE_AGENT=<id>` | `start.sh` / `start.ps1` | Skip the picker and use this agent (`claude`, `codex`, `cursor-agent`, `copilot`, `gemini`). |
+| `ARIZE_SKIP_NPX=1` | `start.sh` / `start.ps1` | Force the shell path even when Node/npx is available. |
+| `ARIZE_PROMPT_URL=<url>` | `start.sh` / `start.ps1` | Fetch the onboarding prompt from a custom URL (supports `file://`). |
+
+The shell launchers also accept `--agent <id>`.
+
+## About Arize
+
+[Arize AX](https://arize.com/docs/ax) is the AI engineering platform for tracing, evaluating, and observing LLM and agent applications. Learn more at [arize.com](https://arize.com).
+
+## Contributing
+
+Development setup and the release process live in [CONTRIBUTING.md](https://github.com/Arize-ai/npm-evals/blob/main/CONTRIBUTING.md).