evalsense 0.3.2 → 0.4.1

package/README.md

@@ -1,718 +1,180 @@
-# evalsense
-
-> JS-native LLM evaluation framework with Jest-like API and statistical assertions
+[![Evalsense logo](./brand/evalsense.png)](https://www.evalsense.com)
 
 [![npm version](https://img.shields.io/npm/v/evalsense.svg)](https://www.npmjs.com/package/evalsense)
+[![CI](https://github.com/evalsense/evalsense/actions/workflows/ci.yml/badge.svg)](https://github.com/evalsense/evalsense/actions/workflows/ci.yml)
 [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
 
-**evalsense** brings classical ML-style statistical evaluation to LLM systems in JavaScript. Instead of evaluating individual test cases, evalsense evaluates entire datasets and computes confusion matrices, precision/recall, F1 scores, and other statistical metrics.
-
-> **New in v0.3.2:** Enhanced assertion reporting - all assertions (passed and failed) now display expected vs actual values, and chained assertions evaluate completely instead of short-circuiting on first failure!
-> **New in v0.3.0:** Regression assertions (MAE, RMSE, R²) and flexible ID matching for custom identifier fields! [See migration guide](./docs/migration-v0.3.0.md).
-> **New in v0.2.x:** Built-in adapters for OpenAI, Anthropic, and OpenRouter - no boilerplate needed!
-> **New in v0.2.0:** LLM-powered metrics for hallucination, relevance, faithfulness, and toxicity detection. [See migration guide](./docs/migration-v0.2.md).
-
-## Why evalsense?
-
-Most LLM evaluation tools stop at producing scores (accuracy, relevance, hallucination). evalsense goes further by:
+> **Jest for LLM Evaluation.** Pass/fail quality gates for your LLM-powered code.
 
-- ✅ Computing **confusion matrices** to reveal systematic failure patterns
-- ✅ Analyzing **false positives vs false negatives** across datasets
-- ✅ Treating **metrics as predictions, not truth** (and validating them statistically)
-- ✅ Providing a **Jest-like API** that fits naturally into JS/Node workflows
-- ✅ Supporting **deterministic CI/CD** integration with specific exit codes
-
-## Features
-
-- 📊 **Dataset-level evaluation** - evaluate distributions, not single examples
-- 🎯 **Statistical rigor** - confusion matrices, precision/recall, F1, regression metrics
-- 🧪 **Jest-like API** - familiar `describe()` and test patterns
-- 🤖 **LLM-powered metrics** - hallucination, relevance, faithfulness, toxicity with explainable reasoning
-- ⚡ **Dual evaluation modes** - choose between accuracy (per-row) or cost efficiency (batch)
-- 🔄 **CI-friendly** - deterministic execution, machine-readable reports
-- 🚀 **JS-native** - first-class TypeScript support, works with any Node.js LLM library
-- 🔌 **Composable** - evaluate outputs from your existing LLM code
-
-## Installation
+evalsense runs your code across many inputs, measures quality statistically, and gives you a clear **pass / fail** result — locally or in CI.
 
 ```bash
 npm install --save-dev evalsense
 ```
 
-Or with yarn:
-
-```bash
-yarn add -D evalsense
-```
-
 ## Quick Start
 
-Create a file named `sentiment.eval.js`:
+Create `sentiment.eval.js`:
 
 ```javascript
 import { describe, evalTest, expectStats } from "evalsense";
 import { readFileSync } from "fs";
 
-// Your model function - can be any JS function
 function classifySentiment(text) {
-  const lower = text.toLowerCase();
-  const hasPositive = /love|amazing|great|fantastic|perfect/.test(lower);
-  const hasNegative = /terrible|worst|disappointed|waste/.test(lower);
-  return hasPositive && !hasNegative ? "positive" : "negative";
+  return /love|great|amazing/.test(text.toLowerCase()) ? "positive" : "negative";
 }
 
 describe("Sentiment classifier", () => {
   evalTest("accuracy above 80%", async () => {
-    // 1. Load ground truth data
     const groundTruth = JSON.parse(readFileSync("./sentiment.json", "utf-8"));
 
-    // 2. Run your model and collect predictions
     const predictions = groundTruth.map((record) => ({
       id: record.id,
       sentiment: classifySentiment(record.text),
     }));
 
-    // 3. Assert on statistical properties
     expectStats(predictions, groundTruth)
       .field("sentiment")
-      .toHaveAccuracyAbove(0.8)
-      .toHaveRecallAbove("positive", 0.7)
-      .toHavePrecisionAbove("positive", 0.7)
-      .toHaveConfusionMatrix();
+      .accuracy.toBeAtLeast(0.8)
+      .recall("positive")
+      .toBeAtLeast(0.7)
+      .precision("positive")
+      .toBeAtLeast(0.7)
+      .displayConfusionMatrix();
   });
 });
 ```
 
-Create `sentiment.json`:
-
-```json
-[
-  { "id": "1", "text": "I love this product!", "sentiment": "positive" },
-  { "id": "2", "text": "Terrible experience.", "sentiment": "negative" },
-  { "id": "3", "text": "Great quality!", "sentiment": "positive" }
-]
-```
-
-Run the evaluation:
+Run it:
 
 ```bash
 npx evalsense run sentiment.eval.js
 ```
 
-## Usage
-
-### Basic Classification Example
-
-```javascript
-import { describe, evalTest, expectStats } from "evalsense";
-import { readFileSync } from "fs";
-
-describe("Spam classifier", () => {
-  evalTest("high precision and recall", async () => {
-    const groundTruth = JSON.parse(readFileSync("./emails.json", "utf-8"));
-
-    const predictions = groundTruth.map((record) => ({
-      id: record.id,
-      isSpam: classifyEmail(record.text),
-    }));
+Output:
 
-    expectStats(predictions, groundTruth)
-      .field("isSpam")
-      .toHaveAccuracyAbove(0.9)
-      .toHavePrecisionAbove(true, 0.85) // Precision for spam=true
-      .toHaveRecallAbove(true, 0.85) // Recall for spam=true
-      .toHaveConfusionMatrix();
-  });
-});
 ```
+EvalSense v0.4.1
+Running 1 eval file(s)...
 
-### Continuous Scores with Binarization
+  Sentiment classifier
 
-```javascript
-import { describe, evalTest, expectStats } from "evalsense";
-import { readFileSync } from "fs";
+    ✓ accuracy above 80% (12ms)
+      Field: sentiment | Accuracy: 90.0% | F1: 89.5%
+        negative: P=88.0% R=92.0% F1=90.0% (n=25)
+        positive: P=91.0% R=87.0% F1=89.0% (n=25)
+      ✓ Accuracy 90.0% >= 80.0%
+      ✓ Recall for 'positive' 87.0% >= 70.0%
+      ✓ Precision for 'positive' 91.0% >= 70.0%
 
-describe("Hallucination detector", () => {
-  evalTest("detect hallucinations with 70% recall", async () => {
-    const groundTruth = JSON.parse(readFileSync("./outputs.json", "utf-8"));
+  Summary
 
-    // Your model returns a continuous score (0.0 to 1.0)
-    const predictions = groundTruth.map((record) => ({
-      id: record.id,
-      hallucinated: computeHallucinationScore(record.output),
-    }));
+    Tests: 1 passed, 0 failed, 0 errors, 0 skipped
+    Duration: 12ms
 
-    // Binarize the score at threshold 0.3
-    expectStats(predictions, groundTruth)
-      .field("hallucinated")
-      .binarize(0.3) // >= 0.3 means hallucinated
-      .toHaveRecallAbove(true, 0.7)
-      .toHavePrecisionAbove(true, 0.6)
-      .toHaveConfusionMatrix();
-  });
-});
+  All tests passed!
 ```
 
-### Multi-class Classification
-
-```javascript
-import { describe, evalTest, expectStats } from "evalsense";
-import { readFileSync } from "fs";
-
-describe("Intent classifier", () => {
-  evalTest("balanced performance across intents", async () => {
-    const groundTruth = JSON.parse(readFileSync("./intents.json", "utf-8"));
+## Key Features
 
-    const predictions = groundTruth.map((record) => ({
-      id: record.id,
-      intent: classifyIntent(record.query),
-    }));
+- **Jest-like API** — `describe`, `evalTest`, `expectStats` feel familiar
+- **Statistical assertions** — accuracy, precision, recall, F1, MAE, RMSE, R²
+- **Confusion matrices** — built-in display with `.displayConfusionMatrix()`
+- **Distribution monitoring** — `percentageAbove` / `percentageBelow` without ground truth
+- **LLM-as-judge** — built-in hallucination, relevance, faithfulness, toxicity metrics
+- **CI/CD ready** — structured exit codes, JSON reporter, bail mode
+- **Zero config** — works with any JS data loading and model execution
 
-    expectStats(predictions, groundTruth)
-      .field("intent")
-      .toHaveAccuracyAbove(0.85)
-      .toHaveRecallAbove("purchase", 0.8)
-      .toHaveRecallAbove("support", 0.8)
-      .toHaveRecallAbove("general", 0.7)
-      .toHaveConfusionMatrix();
-  });
-});
-```
-
-### Parallel Model Execution with LLMs
-
-For LLM calls or slow operations, use `Promise.all` with chunking for concurrency control:
-
-```javascript
-import { describe, evalTest, expectStats } from "evalsense";
-import { readFileSync } from "fs";
-
-// Helper for parallel execution with concurrency limit
-async function mapConcurrent(items, fn, concurrency = 5) {
-  const results = [];
-  for (let i = 0; i < items.length; i += concurrency) {
-    const chunk = items.slice(i, i + concurrency);
-    results.push(...(await Promise.all(chunk.map(fn))));
-  }
-  return results;
-}
-
-describe("LLM classifier", () => {
-  evalTest("classification accuracy", async () => {
-    const groundTruth = JSON.parse(readFileSync("./data.json", "utf-8"));
-
-    // Run with concurrency=5
-    const predictions = await mapConcurrent(
-      groundTruth,
-      async (record) => {
-        const response = await callLLM(record.text);
-        return { id: record.id, category: response.category };
-      },
-      5
-    );
-
-    expectStats(predictions, groundTruth).field("category").toHaveAccuracyAbove(0.9);
-  });
-});
-```
+## Two Ways to Use It
 
-### Test Lifecycle Hooks
-
-```javascript
-import { describe, evalTest, beforeAll, afterAll, beforeEach, afterEach } from "evalsense";
-
-describe("Model evaluation", () => {
-  let model;
-
-  beforeAll(async () => {
-    // Load model once before all tests
-    model = await loadModel();
-  });
-
-  afterAll(async () => {
-    // Cleanup after all tests
-    await model.dispose();
-  });
-
-  beforeEach(() => {
-    // Reset state before each test
-    model.reset();
-  });
-
-  afterEach(() => {
-    // Cleanup after each test
-    console.log("Test completed");
-  });
-
-  evalTest("test 1", async () => {
-    // ...
-  });
-
-  evalTest("test 2", async () => {
-    // ...
-  });
-});
-```
-
-## CLI Usage
-
-### Run Evaluations
-
-```bash
-# Run all eval files in current directory
-npx evalsense run
-
-# Run specific file or directory
-npx evalsense run tests/eval/
-
-# Filter tests by name
-npx evalsense run --filter "accuracy"
-
-# Output JSON report
-npx evalsense run --output report.json
-
-# Use different reporters
-npx evalsense run --reporter console  # default
-npx evalsense run --reporter json
-npx evalsense run --reporter both
-
-# Bail on first failure
-npx evalsense run --bail
-
-# Set timeout (in milliseconds)
-npx evalsense run --timeout 60000
-```
-
-### List Eval Files
-
-```bash
-# List all discovered eval files
-npx evalsense list
-
-# List files in specific directory
-npx evalsense list tests/
-```
-
-## API Reference
-
-### Core API
-
-#### `describe(name, fn)`
-
-Groups related evaluation tests (like Jest's describe).
-
-```javascript
-describe("My model", () => {
-  // eval tests go here
-});
-```
-
-#### `evalTest(name, fn)` / `test(name, fn)` / `it(name, fn)`
-
-Defines an evaluation test.
-
-```javascript
-evalTest("should have 90% accuracy", async () => {
-  // test implementation
-});
-```
-
-### Dataset Loading
-
-evalsense doesn't dictate how you load data or run your model. Use standard Node.js tools:
-
-```javascript
-import { readFileSync } from "fs";
-
-// Load ground truth
-const groundTruth = JSON.parse(readFileSync("./data.json", "utf-8"));
-
-// Run your model however you want
-const predictions = groundTruth.map(runYourModel);
-
-// Or use async operations
-const predictions = await Promise.all(
-  groundTruth.map(async (item) => {
-    const result = await callLLM(item.text);
-    return { id: item.id, prediction: result };
-  })
-);
-```
-
-**Helper functions available (optional):**
-
-- `loadDataset(path)` - Simple JSON file loader
-- `runModel(dataset, fn)` - Sequential model execution
-- `runModelParallel(dataset, fn, concurrency)` - Parallel execution with concurrency limit
-
-### Assertions
-
-#### `expectStats(predictions, groundTruth)`
-
-Creates a statistical assertion chain from predictions and ground truth. Aligns by `id` field.
+### With ground truth (classification / regression)
 
 ```javascript
 expectStats(predictions, groundTruth)
-  .field("prediction")
-  .toHaveAccuracyAbove(0.8)
-  .toHaveF1Above(0.75)
-  .toHaveConfusionMatrix();
-```
-
-**New in v0.3.2: Enhanced Assertion Reporting**
-
-- All assertions (passed and failed) now display expected vs actual values
-- Chained assertions evaluate completely instead of short-circuiting on first failure
-- See all metric results in a single run for better debugging
-
-**One-argument form (distribution assertions only):**
-
-```javascript
-// For distribution monitoring without ground truth
-expectStats(predictions).field("confidence").toHavePercentageAbove(0.7, 0.8);
-```
-
-**Common use cases:**
-
-- Classification evaluation with ground truth
-- Regression evaluation (MAE, RMSE, R²)
-- Validating LLM judges against human labels
-- Distribution monitoring without ground truth
-
-### Field Selection
-
-#### `.field(fieldName)`
-
-Selects a field for evaluation.
-
-```javascript
-expectStats(result).field("sentiment");
-```
-
-#### `.binarize(threshold)`
-
-Converts continuous scores to binary (>=threshold is true).
-
-```javascript
-expectStats(result)
-  .field("score")
-  .binarize(0.5) // score >= 0.5 is true
-  .toHaveAccuracyAbove(0.8);
-```
-
-### Available Assertions
-
-#### Classification Metrics
-
-```javascript
-// Accuracy
-.toHaveAccuracyAbove(threshold)
-.toHaveAccuracyBelow(threshold)
-.toHaveAccuracyBetween(min, max)
-
-// Precision (per class)
-.toHavePrecisionAbove(className, threshold)
-.toHavePrecisionBelow(className, threshold)
-
-// Recall (per class)
-.toHaveRecallAbove(className, threshold)
-.toHaveRecallBelow(className, threshold)
-
-// F1 Score
-.toHaveF1Above(threshold)           // Overall F1
-.toHaveF1Above(className, threshold) // Per-class F1
-
-// Confusion Matrix
-.toHaveConfusionMatrix()  // Prints confusion matrix
-```
-
-#### Distribution Assertions (Pattern 1)
-
-Distribution assertions validate output distributions **without requiring ground truth**. Use these to monitor that model outputs stay within expected ranges.
-
-```javascript
-// Assert that at least 80% of confidence scores are above 0.7
-expectStats(predictions).field("confidence").toHavePercentageAbove(0.7, 0.8);
-
-// Assert that at least 90% of toxicity scores are below 0.3
-expectStats(predictions).field("toxicity").toHavePercentageBelow(0.3, 0.9);
-
-// Chain multiple distribution assertions
-expectStats(predictions)
-  .field("score")
-  .toHavePercentageAbove(0.5, 0.6) // At least 60% above 0.5
-  .toHavePercentageBelow(0.9, 0.8); // At least 80% below 0.9
-```
-
-**Use cases:**
-
-- Monitor confidence score distributions
-- Validate schema compliance rates
-- Check output range constraints
-- Ensure score distributions remain stable over time
-
-See [Distribution Assertions Example](./examples/distribution-assertions.eval.js) for complete examples.
-
-### Judge Validation (Pattern 1b)
-
-Validate judge outputs against human-labeled ground truth using the **two-argument expectStats API**:
-
-```javascript
-// Judge outputs (predictions from your judge/metric)
-const judgeOutputs = [
-  { id: "1", hallucinated: true },
-  { id: "2", hallucinated: false },
-  { id: "3", hallucinated: true },
-];
-
-// Human labels (ground truth)
-const humanLabels = [
-  { id: "1", hallucinated: true },
-  { id: "2", hallucinated: false },
-  { id: "3", hallucinated: false },
-];
-
-// Validate judge performance
-expectStats(judgeOutputs, humanLabels)
-  .field("hallucinated")
-  .toHaveRecallAbove(true, 0.9) // Don't miss hallucinations
-  .toHavePrecisionAbove(true, 0.7) // Some false positives OK
-  .toHaveConfusionMatrix();
+  .field("label")
+  .accuracy.toBeAtLeast(0.9)
+  .recall("positive")
+  .toBeAtLeast(0.8)
+  .f1.toBeAtLeast(0.85);
 ```
 
-**Use cases:**
-
-- Evaluate LLM-as-judge accuracy
-- Validate heuristic metrics against human labels
-- Test automated detection systems (refusal, policy compliance)
-- Calibrate metric thresholds
-
-**Two-argument expectStats:**
+### Without ground truth (distribution monitoring)
 
 ```javascript
-expectStats(actual, expected).field("fieldName").toHaveAccuracyAbove(0.8);
-```
-
-The first argument is your predictions (judge outputs), the second is ground truth (human labels). Both must have matching `id` fields for alignment.
-
-See [Judge Validation Example](./examples/judge-validation.eval.js) for complete examples.
-
-For comprehensive guidance on evaluating agent systems, see [Agent Judges Design Patterns](./docs/agent-judges.md).
-
-## Dataset Format
-
-Datasets must be JSON arrays where each record has an `id` or `_id` field:
-
-```json
-[
-  {
-    "id": "1",
-    "text": "input text",
-    "label": "expected_output"
-  },
-  {
-    "id": "2",
-    "text": "another input",
-    "label": "another_output"
-  }
-]
-```
-
-**Requirements:**
-
-- Each record MUST have `id` or `_id` for alignment
-- Ground truth fields (e.g., `label`, `sentiment`, `category`) are compared against model outputs
-- Model functions must return predictions with matching `id`
-
-## Exit Codes
-
-evalsense returns specific exit codes for CI integration:
-
-- `0` - Success (all tests passed)
-- `1` - Assertion failure (statistical thresholds not met)
-- `2` - Integrity failure (dataset alignment issues)
-- `3` - Execution error (test threw exception)
-- `4` - Configuration error (invalid CLI options)
-
-## Writing Eval Files
-
-Eval files use the `.eval.js` or `.eval.ts` extension and are discovered automatically:
-
-```
-project/
-├── tests/
-│   ├── classifier.eval.js
-│   └── hallucination.eval.js
-├── data/
-│   └── dataset.json
-└── package.json
-```
-
-Run with:
-
-```bash
-npx evalsense run tests/
+expectStats(llmOutputs).field("toxicity_score").percentageBelow(0.3).toBeAtLeast(0.95); // 95% of outputs must be non-toxic
 ```
 
-## Examples
-
-See the [`examples/`](./examples/) directory for complete examples:
-
-- [`classification.eval.js`](./examples/basic/classification.eval.js) - Binary sentiment classification
-- [`hallucination.eval.js`](./examples/basic/hallucination.eval.js) - Continuous score binarization
-- [`distribution-assertions.eval.js`](./examples/distribution-assertions.eval.js) - Distribution monitoring without ground truth
-- [`judge-validation.eval.js`](./examples/judge-validation.eval.js) - Validating judges against human labels
-
-## Field Types
-
-evalsense automatically determines evaluation metrics based on field values:
-
-- **Boolean** (`true`/`false`) → Binary classification metrics
-- **Categorical** (strings) → Multi-class classification metrics
-- **Numeric** (numbers) → Regression metrics (MAE, MSE, RMSE, R²)
-- **Numeric + threshold** → Binarized classification metrics
-
-## LLM-Based Metrics (v0.2.0+)
-
-evalsense includes LLM-powered metrics for hallucination detection, relevance assessment, faithfulness verification, and toxicity detection.
-
-### Quick Setup
+## LLM-Based Metrics
 
 ```javascript
-import { setLLMClient, createOpenAIAdapter } from "evalsense/metrics";
-import { hallucination, relevance, faithfulness, toxicity } from "evalsense/metrics/opinionated";
+import { setLLMClient, createAnthropicAdapter } from "evalsense/metrics";
+import { hallucination, relevance } from "evalsense/metrics/opinionated";
 
-// 1. Configure your LLM client (one-time setup)
 setLLMClient(
-  createOpenAIAdapter(process.env.OPENAI_API_KEY, {
-    model: "gpt-4-turbo-preview",
-    temperature: 0,
+  createAnthropicAdapter(process.env.ANTHROPIC_API_KEY, {
+    model: "claude-haiku-4-5-20251001",
   })
 );
 
-// 2. Use metrics in evaluations
-const results = await hallucination({
+const scores = await hallucination({
   outputs: [{ id: "1", output: "Paris has 50 million people." }],
   context: ["Paris has approximately 2.1 million residents."],
 });
-
-console.log(results[0].score); // 0.9 (high hallucination)
-console.log(results[0].reasoning); // "Output claims 50M, context says 2.1M"
+// scores[0].score → 0.9 (high hallucination)
+// scores[0].reasoning → "Output claims 50M, context says 2.1M"
 ```
 
-### Available Metrics
-
-- **`hallucination()`** - Detects claims not supported by context
-- **`relevance()`** - Measures query-response alignment
-- **`faithfulness()`** - Verifies outputs don't contradict sources
-- **`toxicity()`** - Identifies harmful or inappropriate content
-
-### Evaluation Modes
-
-Choose between accuracy and cost:
-
-```javascript
-// Per-row: Higher accuracy, higher cost (N API calls)
-await hallucination({
-  outputs,
-  context,
-  evaluationMode: "per-row", // default
-});
-
-// Batch: Lower cost, single API call
-await hallucination({
-  outputs,
-  context,
-  evaluationMode: "batch",
-});
-```
-
-### Built-in Provider Adapters
-
-evalsense includes ready-to-use adapters for popular LLM providers:
+Built-in providers: OpenAI, Anthropic, OpenRouter, or bring your own adapter.
+See [LLM Metrics Guide](./docs/llm-metrics.md) and [Adapters Guide](./docs/llm-adapters.md).
 
-**OpenAI (GPT-4, GPT-3.5)**
+## Using with Claude Code (Vibe Check)
 
-```javascript
-import { createOpenAIAdapter } from "evalsense/metrics";
-
-// npm install openai
-setLLMClient(
-  createOpenAIAdapter(process.env.OPENAI_API_KEY, {
-    model: "gpt-4-turbo-preview", // or "gpt-3.5-turbo" for lower cost
-    temperature: 0,
-    maxTokens: 4096,
-  })
-);
-```
+evalsense includes an example [Claude Code skill](./skill.md) that acts as an automated LLM quality gate. To set it up in your project:
 
-**Anthropic (Claude)**
-
-```javascript
-import { createAnthropicAdapter } from "evalsense/metrics";
+1. Install evalsense as a dev dependency
+2. Copy [`skill.md`](./skill.md) into your project at `.claude/skills/llm-quality-gate/SKILL.md`
+3. After building any LLM feature, run `/llm-quality-gate` in Claude Code
 
-// npm install @anthropic-ai/sdk
-setLLMClient(
-  createAnthropicAdapter(process.env.ANTHROPIC_API_KEY, {
-    model: "claude-3-5-sonnet-20241022", // or "claude-3-haiku-20240307" for speed
-    maxTokens: 4096,
-  })
-);
-```
+Claude will automatically create a `.eval.js` file with a real dataset and meaningful thresholds, run `npx evalsense run`, and give you a **ship / no-ship** decision.
 
-**OpenRouter (100+ models from one API)**
+## Documentation
 
-```javascript
-import { createOpenRouterAdapter } from "evalsense/metrics";
+| Guide                                              | Description                                      |
+| -------------------------------------------------- | ------------------------------------------------ |
+| [API Reference](./docs/api-reference.md)           | Full API — all assertions, matchers, metrics     |
+| [CLI Reference](./docs/cli.md)                     | All CLI flags, exit codes, CI integration        |
+| [LLM Metrics](./docs/llm-metrics.md)               | Hallucination, relevance, faithfulness, toxicity |
+| [LLM Adapters](./docs/llm-adapters.md)             | OpenAI, Anthropic, OpenRouter, custom adapters   |
+| [Custom Metrics](./docs/custom-metrics-guide.md)   | Pattern and keyword metrics                      |
+| [Agent Judges](./docs/agent-judges.md)             | Design patterns for evaluating agent systems     |
+| [Regression Metrics](./docs/regression-metrics.md) | MAE, RMSE, R² usage                              |
+| [Examples](./examples/)                            | Working code examples                            |
 
-// No SDK needed - uses fetch
-setLLMClient(
-  createOpenRouterAdapter(process.env.OPENROUTER_API_KEY, {
-    model: "anthropic/claude-3.5-sonnet", // or "openai/gpt-3.5-turbo", etc.
-    temperature: 0,
-    appName: "my-eval-system",
-  })
-);
-```
+## Dataset Format
 
-**Custom Adapter (for any provider)**
+Records must have an `id` or `_id` field:
 
-```javascript
-setLLMClient({
-  async complete(prompt) {
-    // Implement for your LLM provider
-    const response = await yourLLM.generate(prompt);
-    return response.text;
-  },
-});
+```json
+[
+  { "id": "1", "text": "sample input", "label": "positive" },
+  { "id": "2", "text": "another input", "label": "negative" }
+]
 ```
 
-### Learn More
-
-- [LLM Metrics Guide](./docs/llm-metrics.md) - Complete usage guide
-- [LLM Adapters Guide](./docs/llm-adapters.md) - Implement adapters for different providers
-- [Migration Guide](./docs/migration-v0.2.md) - Upgrade from v0.1.x
-- [Examples](./examples/) - Working code examples
-
-## Philosophy
-
-evalsense is built on the principle that **metrics are predictions, not facts**.
-
-Instead of treating LLM-as-judge metrics (relevance, hallucination, etc.) as ground truth, evalsense:
+## Exit Codes
 
-- Treats them as **weak labels** from a model
-- Validates them statistically against human references when available
-- Computes confusion matrices to reveal bias and systematic errors
-- Focuses on dataset-level distributions, not individual examples
+| Code | Meaning                   |
+| ---- | ------------------------- |
+| `0`  | All tests passed          |
+| `1`  | Assertion failure         |
+| `2`  | Dataset integrity failure |
+| `3`  | Execution error           |
+| `4`  | Configuration error       |
 
 ## Contributing
 
-Contributions are welcome! Please see [CLAUDE.md](./CLAUDE.md) for development guidelines.
+Contributions are welcome. See [CONTRIBUTING.md](./CONTRIBUTING.md) for setup, coding standards, and the PR process.
 
 ## License
 
-MIT © Mohit Joshi
-
----
-
-**Made with ❤️ for the JS/Node.js AI community**
+[Apache 2.0](./LICENSE)