@assay-ai/core 0.2.1-beta → 1.3.1-beta

package/README.md

@@ -1,16 +1,23 @@
+<div align="center">
+
 # @assay-ai/core
 
-The core evaluation engine for [Assay](https://github.com/assay-ai/assay) -- the TypeScript-native LLM evaluation framework.
+*The evaluation engine powering Assay -- 18 metrics, 5 providers, zero `any`*
+
+[![npm version](https://img.shields.io/npm/v/@assay-ai/core?style=flat-square&color=6366f1)](https://www.npmjs.com/package/@assay-ai/core)
+[![downloads](https://img.shields.io/npm/dm/@assay-ai/core?style=flat-square&color=10b981)](https://www.npmjs.com/package/@assay-ai/core)
+[![License](https://img.shields.io/badge/license-MIT-blue?style=flat-square)](https://github.com/assay-ai/assay/blob/main/LICENSE)
 
-[![npm version](https://img.shields.io/npm/v/@assay-ai/core?color=blue)](https://www.npmjs.com/package/@assay-ai/core)
-[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+[Documentation](https://assay.js.org) · [Metrics](https://assay.js.org/metrics/) · [API Reference](https://assay.js.org/api/)
+
+</div>
 
 ## Installation
 
 ```bash
-npm install @assay-ai/core
-# or
-pnpm add @assay-ai/core
+pnpm add @assay-ai/core     # pnpm
+npm install @assay-ai/core   # npm
+yarn add @assay-ai/core      # Yarn
 ```
 
 ## Quick Start
@@ -26,14 +33,17 @@ import {
 const results = await evaluate(
   [
     {
-      input: "What is the capital of France?",
-      actualOutput: "The capital of France is Paris.",
-      context: ["France is a country in Europe. Its capital is Paris."],
+      input: "What is the refund policy?",
+      actualOutput: "You can request a full refund within 30 days.",
+      retrievalContext: [
+        "Refund Policy: Full refund within 30 days of purchase.",
+      ],
+      context: ["Our refund policy allows returns within 30 days."],
     },
   ],
   [
-    new FaithfulnessMetric({ threshold: 0.7 }),
     new AnswerRelevancyMetric({ threshold: 0.7 }),
+    new FaithfulnessMetric({ threshold: 0.7 }),
     new HallucinationMetric({ threshold: 0.3 }),
   ],
 );
@@ -41,78 +51,11 @@ const results = await evaluate(
 console.log(`Pass rate: ${results.summary.passRate.toFixed(1)}%`);
 ```
 
-## Metrics
-
-Assay ships with 12 evaluation metrics out of the box:
-
-| Metric | Description | Required Fields |
-|--------|-------------|-----------------|
-| `AnswerRelevancyMetric` | Measures how relevant the output is to the input | `input`, `actualOutput` |
-| `FaithfulnessMetric` | Measures whether the output is grounded in context | `input`, `actualOutput`, `retrievalContext` |
-| `HallucinationMetric` | Detects claims not supported by context | `input`, `actualOutput`, `context` |
-| `ContextualPrecisionMetric` | Measures whether relevant context items are ranked higher | `input`, `expectedOutput`, `retrievalContext` |
-| `ContextualRecallMetric` | Measures whether all relevant information is retrieved | `input`, `expectedOutput`, `retrievalContext` |
-| `ContextualRelevancyMetric` | Measures whether retrieved context is relevant | `input`, `actualOutput`, `retrievalContext` |
-| `BiasMetric` | Detects demographic or ideological bias | `input`, `actualOutput` |
-| `ToxicityMetric` | Detects toxic or harmful content | `input`, `actualOutput` |
-| `GEval` | Custom LLM-as-judge with user-defined criteria | `input`, `actualOutput` |
-| `SummarizationMetric` | Evaluates summary quality | `input`, `actualOutput` |
-| `ExactMatchMetric` | Exact string comparison (no LLM needed) | `actualOutput`, `expectedOutput` |
-| `JsonCorrectnessMetric` | Validates JSON structure (no LLM needed) | `actualOutput` |
-
-## Configuration
-
-### Provider
-
-Assay auto-detects your LLM provider from environment variables:
-
-```bash
-# OpenAI (default)
-export OPENAI_API_KEY="sk-..."
-
-# Anthropic
-export ANTHROPIC_API_KEY="sk-ant-..."
-```
-
-### Metric Options
-
-Every metric accepts optional configuration:
-
-```typescript
-new FaithfulnessMetric({
-  threshold: 0.7,       // Minimum score to pass (default: 0.5)
-  model: "gpt-4o-mini", // LLM model for evaluation
-  verbose: true,        // Log detailed reasoning
-});
-```
-
-### Custom Metrics with GEval
-
-Define any evaluation criteria in plain English:
-
-```typescript
-import { GEval } from "@assay-ai/core";
-
-const politeness = new GEval({
-  name: "Politeness",
-  criteria: "The response should be polite and professional.",
-  evaluationSteps: [
-    "Check if the response uses polite phrases",
-    "Verify the tone is respectful",
-  ],
-});
-```
-
-## Exports
-
-This package exports:
-
-- **Metrics**: `AnswerRelevancyMetric`, `FaithfulnessMetric`, `HallucinationMetric`, `ContextualPrecisionMetric`, `ContextualRecallMetric`, `ContextualRelevancyMetric`, `BiasMetric`, `ToxicityMetric`, `GEval`, `SummarizationMetric`, `ExactMatchMetric`, `JsonCorrectnessMetric`
-- **Evaluation**: `evaluate`, `assertEval`
-- **Providers**: `BaseLLMProvider`, `OpenAIProvider`, `AnthropicProvider`, `OllamaProvider`, `resolveProvider`
-- **Utilities**: `parseJson`, `tryParseJson`, `createLimiter`, `ConsoleReporter`
-- **Types**: `LLMTestCase`, `MetricResult`, `MetricConfig`, `EvaluateConfig`, `EvaluateResult`, `EvaluationDataset`
-
-## License
+## Part of the [Assay](https://github.com/assay-ai/assay) monorepo
 
-[MIT](https://github.com/assay-ai/assay/blob/main/LICENSE)
+<p align="center">
+  <a href="https://assay.js.org"><img src="https://img.shields.io/badge/Documentation-6366f1?style=for-the-badge&logo=readthedocs&logoColor=white" alt="Documentation" /></a>
+  <a href="https://www.npmjs.com/package/@assay-ai/core"><img src="https://img.shields.io/badge/npm-cb3837?style=for-the-badge&logo=npm&logoColor=white" alt="npm" /></a>
+  <a href="https://github.com/assay-ai/assay"><img src="https://img.shields.io/badge/GitHub-181717?style=for-the-badge&logo=github&logoColor=white" alt="GitHub" /></a>
+  <a href="https://github.com/assay-ai/assay/issues"><img src="https://img.shields.io/badge/Issues-6366f1?style=for-the-badge&logo=github&logoColor=white" alt="Issues" /></a>
+</p>