@arizeai/phoenix-evals 0.0.2 → 0.0.3

package/README.md

@@ -22,3 +22,59 @@ This package provides a TypeScript evaluation library. It is vendor agnostic and
 # or yarn, pnpm, bun, etc...
 npm install @arizeai/phoenix-evals
 ```
+
+## Usage
+
+### Creating a Classifier
+
+The library provides a `createClassifier` function that allows you to create custom evaluators for different tasks like hallucination detection, relevance scoring, or any binary/multi-class classification.
+
+```typescript
+import { createClassifier } from "@arizeai/phoenix-evals/llm";
+import { openai } from "@ai-sdk/openai";
+
+const model = openai("gpt-4o-mini");
+
+const promptTemplate = `
+In this task, you will be presented with a query, a reference text and an answer. The answer is
+generated to the question based on the reference text. The answer may contain false information. You
+must use the reference text to determine if the answer to the question contains false information,
+if the answer is a hallucination of facts. Your objective is to determine whether the answer text
+contains factual information and is not a hallucination. A 'hallucination' refers to
+an answer that is not based on the reference text or assumes information that is not available in
+the reference text. Your response should be a single word: either "factual" or "hallucinated", and
+it should not include any other text or characters.
+
+    [BEGIN DATA]
+    ************
+    [Query]: {{input}}
+    ************
+    [Reference text]: {{reference}}
+    ************
+    [Answer]: {{output}}
+    ************
+    [END DATA]
+
+Is the answer above factual or hallucinated based on the query and reference text?
+`;
+
+// Create the classifier
+const evaluator = await createClassifier({
+  model,
+  choices: { factual: 1, hallucinated: 0 },
+  promptTemplate: promptTemplate,
+});
+
+// Use the classifier
+const result = await evaluator({
+  output: "Arize is not open source.",
+  input: "Is Arize Phoenix Open Source?",
+  reference:
+    "Arize Phoenix is a platform for building and deploying AI applications. It is open source.",
+});
+
+console.log(result);
+// Output: { label: "hallucinated", score: 0 }
+```
+
+See the complete example in [`examples/classifier_example.ts`](examples/classifier_example.ts).

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@arizeai/phoenix-evals",
-  "version": "0.0.2",
+  "version": "0.0.3",
   "description": "A library for running evaluations for AI use cases",
   "main": "dist/src/index.js",
   "module": "dist/esm/index.js",
@@ -49,7 +49,7 @@
     "typedoc": "^0.27.9",
     "vitest": "^2.1.9",
     "typescript": "^5.8.2",
-    "@arizeai/phoenix-client": "2.3.1"
+    "@arizeai/phoenix-client": "2.3.2"
   },
   "engines": {
     "node": ">=18"