@evalgate/sdk 2.2.0 → 2.2.2

package/CHANGELOG.md

@@ -5,6 +5,48 @@ All notable changes to the @evalgate/sdk package will be documented in this file
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [2.2.2] - 2026-03-03
+
+### Fixed
+
+- **8 stub assertions replaced with real implementations:**
+  - `hasSentiment` — substring matching + expanded 34/31-word positive/negative lexicon (was exact-match, 4 words each)
+  - `hasNoHallucinations` — case-insensitive fact matching (was case-sensitive)
+  - `hasFactualAccuracy` — case-insensitive fact matching (was case-sensitive)
+  - `containsLanguage` — expanded from 3 languages (en/es/fr) to 12 (+ de/it/pt/nl/ru/zh/ja/ko/ar) with BCP-47 subtag support (`zh-CN` → `zh`)
+  - `hasValidCodeSyntax` — real bracket/brace/parenthesis balance checker with string literal and comment awareness (handles JS `//`/`/* */`, Python `#`, template literals, single/double quotes); JSON fast-path via `JSON.parse`
+  - `hasNoToxicity` — expanded from 4 words to ~80 terms across 9 categories: insults, degradation, violence/threats, self-harm directed at others, dehumanization, hate/rejection, harassment, profanity-as-attacks, bullying/appearance/mental-health weaponization
+  - `hasReadabilityScore` — fixed Flesch-Kincaid syllable counting to be per-word (was treating entire text as one word)
+  - `matchesSchema` — now dispatches on schema format: JSON Schema `required` array (`{ required: ['name'] }` → checks required keys exist), JSON Schema `properties` object (`{ properties: { name: {} } }` → checks property keys exist), or simple key-presence template (existing behavior preserved for backward compat). Fixes regression: `matchesSchema({ name: 'test', score: 95 }, { type: 'object', required: ['name'] })` was returning `false`
+- **`importData` crash** — `options: ImportOptions` parameter now defaults to `{}` to prevent `Cannot read properties of undefined (reading 'dryRun')` when called as `importData(client, data)`
+- **`compareWithSnapshot` / `SnapshotManager.compare` object coercion** — both now accept `unknown` input and coerce non-string values via `JSON.stringify` before comparison, matching the existing behavior of `SnapshotManager.save()`
+- **`WorkflowTracer` constructor crash** — defensive guard: `typeof client?.getOrganizationId === "function"` before calling it; prevents `TypeError: client.getOrganizationId is not a function` when using partial clients or initializing without an API key
+
+### Added
+
+- **LLM-backed async assertion variants** — 6 new exported functions:
+  - `hasSentimentAsync(text, expected, config?)` — LLM classifies sentiment with full context awareness
+  - `hasNoToxicityAsync(text, config?)` — LLM detects sarcastic, implicit, and culturally specific toxic content that blocklists miss
+  - `containsLanguageAsync(text, language, config?)` — LLM language detection for any language
+  - `hasValidCodeSyntaxAsync(code, language, config?)` — LLM deep syntax analysis beyond bracket balance
+  - `hasFactualAccuracyAsync(text, facts, config?)` — LLM checks facts semantically, catches paraphrased inaccuracies
+  - `hasNoHallucinationsAsync(text, groundTruth, config?)` — LLM detects fabricated claims even when paraphrased
+- **`configureAssertions(config: AssertionLLMConfig)`** — set global LLM provider/apiKey/model/baseUrl once; all `*Async` functions use it automatically; per-call `config` overrides it
+- **`getAssertionConfig()`** — retrieve current global assertion LLM config
+- **`AssertionLLMConfig` type** — exported interface: `{ provider: "openai" | "anthropic"; apiKey: string; model?: string; baseUrl?: string }`
+- **JSDoc `**Fast and approximate**` / `**Slow and accurate**` markers** on all sync/async assertion pairs with `{@link xAsync}` cross-references that appear in IDE tooltips
+- **115 new tests** in `assertions.test.ts` covering all improved sync assertions (expanded lexicons, JSON Schema formats, bracket balance edge cases, 12-language detection, BCP-47) and all 6 async variants (OpenAI path, Anthropic path, global config, error cases, HTTP 4xx handling)
+
+---
+
+## [2.2.1] - 2026-03-03
+
+### Fixed
+
+- **`snapshot(name, output)` accepts objects** — passing `{ score: 92 }` no longer throws; non-string values are auto-serialized via `JSON.stringify`. `SnapshotManager.save()` and `update()` widened to `output: unknown` accordingly.
+
+---
+
 ## [2.2.0] - 2026-03-03
 
 ### Breaking
@@ -42,6 +84,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - **Low:** Explain no longer shows "unnamed" for builtin gate failures
 - **Docs:** Added missing `discover --manifest` step to local quickstart
 
+---
+
 ## [2.1.2] - 2026-03-02
 
 ### Fixed
@@ -49,12 +93,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - **Type safety** — aligned with platform 2.1.2; zero TypeScript errors across all integration points
 - **CI gate** — all SDK tests, lint, and build checks passing
 
+---
+
 ## [2.1.1] - 2026-03-02
 
 ### Fixed
 
 - Version alignment with platform 2.1.1
 
+---
+
 ## [2.0.0] - 2026-03-01
 
 ### Breaking — EvalGate Rebrand
@@ -356,7 +404,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   - error catalog stability + graceful handling of unknown codes
   - exports contract (retention visibility, 410 semantics)
 
---
+---
 
 ## [1.5.0] - 2026-02-18
 
@@ -404,6 +452,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - **Package hardening** — `files`, `module`, `sideEffects: false` for leaner npm publish
 - **CLI** — Passes `baseline` param to quality API for deterministic CI gates
 
+---
+
 ## [1.3.0] - 2025-10-21
 
 ### ✨ Added

package/README.md

@@ -3,7 +3,7 @@
 [![npm version](https://img.shields.io/npm/v/@evalgate/sdk.svg)](https://www.npmjs.com/package/@evalgate/sdk)
 [![npm downloads](https://img.shields.io/npm/dm/@evalgate/sdk.svg)](https://www.npmjs.com/package/@evalgate/sdk)
 [![TypeScript](https://img.shields.io/badge/TypeScript-strict-blue.svg)](https://www.typescriptlang.org/)
-[![SDK Tests](https://img.shields.io/badge/tests-172%20passed-brightgreen.svg)](#)
+[![SDK Tests](https://img.shields.io/badge/tests-159%20passed-brightgreen.svg)](#)
 [![Contract Version](https://img.shields.io/badge/report%20schema-v1-blue.svg)](#)
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
 
@@ -366,6 +366,33 @@ import type {
 } from "@evalgate/sdk/regression";
 ```
 
+### Assertions — Sync (fast, heuristic) and Async (slow, LLM-backed)
+
+```typescript
+import {
+  // Sync — fast and approximate (no API key needed)
+  hasSentiment, hasNoToxicity, hasValidCodeSyntax,
+  containsLanguage, hasFactualAccuracy, hasNoHallucinations,
+  matchesSchema,
+  // Async — slow and accurate (requires API key)
+  configureAssertions, hasSentimentAsync, hasNoToxicityAsync,
+  hasValidCodeSyntaxAsync, containsLanguageAsync,
+  hasFactualAccuracyAsync, hasNoHallucinationsAsync,
+} from "@evalgate/sdk";
+
+// Configure once (or pass per-call)
+configureAssertions({ provider: "openai", apiKey: process.env.OPENAI_API_KEY });
+
+// Sync — fast, no network
+console.log(hasSentiment("I love this!", "positive"));   // true
+console.log(hasNoToxicity("Have a great day!"));          // true
+console.log(hasValidCodeSyntax("function f() {}", "js")); // true
+
+// Async — LLM-backed, context-aware
+console.log(await hasSentimentAsync("subtle irony...", "negative")); // true
+console.log(await hasNoToxicityAsync("sarcastic attack text"));       // false
+```
+
 ### Platform Client
 
 ```typescript
@@ -423,17 +450,17 @@ Your local `openAIChatEval` runs continue to work. No account cancellation. No d
 
 See [CHANGELOG.md](CHANGELOG.md) for the full release history.
 
-**v1.8.0** — `evalgate doctor` rewrite (9-check checklist), `evalgate explain` command, guided failure flow, CI template with doctor preflight
+**v2.2.2** — 8 stub assertions replaced with real implementations (`hasSentiment` expanded lexicon, `hasNoToxicity` ~80-term blocklist, `hasValidCodeSyntax` real bracket balance, `containsLanguage` 12 languages + BCP-47, `hasFactualAccuracy`/`hasNoHallucinations` case-insensitive, `hasReadabilityScore` per-word syllable fix, `matchesSchema` JSON Schema support). Added LLM-backed `*Async` variants + `configureAssertions`. Fixed `importData` crash, `compareWithSnapshot` object coercion, `WorkflowTracer` defensive guard. 115 new tests.
 
-**v1.7.0** — `evalgate init` scaffolder, `evalgate upgrade --full`, `detectRunner()`, machine-readable gate output, init test matrix
+**v2.2.1** — `snapshot(name, output)` accepts objects; auto-serialized via `JSON.stringify`
 
-**v1.6.0** — `evalgate gate`, `evalgate baseline`, regression gate constants & types
+**v2.2.0** — `expect().not` modifier, `hasPII()`, `defineSuite` object form, `snapshot` parameter order fix, `specId` collision fix
 
-**v1.5.8** — secureRoute fix, test infra fixes, 304 handling fix
+**v1.8.0** — `evalgate doctor` rewrite (9-check checklist), `evalgate explain` command, guided failure flow, CI template with doctor preflight
 
-**v1.5.5** — PASS/WARN/FAIL semantics, flake intelligence, golden regression suite
+**v1.7.0** — `evalgate init` scaffolder, `evalgate upgrade --full`, `detectRunner()`, machine-readable gate output, init test matrix
 
-**v1.5.0** — GitHub annotations, `--onFail import`, `evalgate doctor`
+**v1.6.0** — `evalgate gate`, `evalgate baseline`, regression gate constants & types
 
 ## License
 

package/dist/assertions.d.ts

@@ -193,18 +193,76 @@ export declare function notContainsPII(text: string): boolean;
  * if (hasPII(response)) throw new Error("PII leak");
  */
 export declare function hasPII(text: string): boolean;
+/**
+ * Lexicon-based sentiment check. **Fast and approximate** — suitable for
+ * low-stakes filtering or CI smoke tests. For production safety gates use
+ * {@link hasSentimentAsync} with an LLM provider for context-aware accuracy.
+ */
 export declare function hasSentiment(text: string, expected: "positive" | "negative" | "neutral"): boolean;
 export declare function similarTo(text1: string, text2: string, threshold?: number): boolean;
 export declare function withinRange(value: number, min: number, max: number): boolean;
 export declare function isValidEmail(email: string): boolean;
 export declare function isValidURL(url: string): boolean;
+/**
+ * Substring-based hallucination check — verifies each ground-truth fact
+ * appears verbatim in the text. **Fast and approximate**: catches missing
+ * facts but cannot detect paraphrased fabrications. Use
+ * {@link hasNoHallucinationsAsync} for semantic accuracy.
+ */
 export declare function hasNoHallucinations(text: string, groundTruth: string[]): boolean;
 export declare function matchesSchema(value: unknown, schema: Record<string, unknown>): boolean;
 export declare function hasReadabilityScore(text: string, minScore: number): boolean;
+/**
+ * Keyword-frequency language detector supporting 12 languages.
+ * **Fast and approximate** — detects the most common languages reliably
+ * but may struggle with short texts or closely related languages.
+ * Use {@link containsLanguageAsync} for reliable detection of any language.
+ */
 export declare function containsLanguage(text: string, language: string): boolean;
+/**
+ * Substring-based factual accuracy check. **Fast and approximate** — verifies
+ * each fact string appears in the text but cannot reason about meaning or
+ * paraphrasing. Use {@link hasFactualAccuracyAsync} for semantic accuracy.
+ */
 export declare function hasFactualAccuracy(text: string, facts: string[]): boolean;
 export declare function respondedWithinTime(startTime: number, maxMs: number): boolean;
+/**
+ * Blocklist-based toxicity check (~80 terms across 9 categories).
+ * **Fast and approximate** — catches explicit harmful language but has
+ * inherent gaps and context-blind false positives. Do NOT rely on this
+ * alone for production content safety gates; use {@link hasNoToxicityAsync}
+ * with an LLM for context-aware moderation.
+ */
 export declare function hasNoToxicity(text: string): boolean;
 export declare function followsInstructions(text: string, instructions: string[]): boolean;
 export declare function containsAllRequiredFields(obj: unknown, requiredFields: string[]): boolean;
+export interface AssertionLLMConfig {
+    provider: "openai" | "anthropic";
+    apiKey: string;
+    model?: string;
+    baseUrl?: string;
+}
+export declare function configureAssertions(config: AssertionLLMConfig): void;
+export declare function getAssertionConfig(): AssertionLLMConfig | null;
+/**
+ * LLM-backed sentiment check. **Slow and accurate** — uses an LLM to
+ * classify sentiment with full context awareness. Requires
+ * {@link configureAssertions} or an inline `config` argument.
+ * Falls back gracefully with a clear error if no API key is configured.
+ */
+export declare function hasSentimentAsync(text: string, expected: "positive" | "negative" | "neutral", config?: AssertionLLMConfig): Promise<boolean>;
+/**
+ * LLM-backed toxicity check. **Slow and accurate** — context-aware, handles
+ * sarcasm, implicit threats, and culturally specific harmful content that
+ * blocklists miss. Recommended for production content safety gates.
+ */
+export declare function hasNoToxicityAsync(text: string, config?: AssertionLLMConfig): Promise<boolean>;
+export declare function containsLanguageAsync(text: string, language: string, config?: AssertionLLMConfig): Promise<boolean>;
+export declare function hasValidCodeSyntaxAsync(code: string, language: string, config?: AssertionLLMConfig): Promise<boolean>;
+export declare function hasFactualAccuracyAsync(text: string, facts: string[], config?: AssertionLLMConfig): Promise<boolean>;
+/**
+ * LLM-backed hallucination check. **Slow and accurate** — detects fabricated
+ * claims even when they are paraphrased or contradict facts indirectly.
+ */
+export declare function hasNoHallucinationsAsync(text: string, groundTruth: string[], config?: AssertionLLMConfig): Promise<boolean>;
 export declare function hasValidCodeSyntax(code: string, language: string): boolean;

package/dist/assertions.js

@@ -39,6 +39,14 @@ exports.respondedWithinTime = respondedWithinTime;
 exports.hasNoToxicity = hasNoToxicity;
 exports.followsInstructions = followsInstructions;
 exports.containsAllRequiredFields = containsAllRequiredFields;
+exports.configureAssertions = configureAssertions;
+exports.getAssertionConfig = getAssertionConfig;
+exports.hasSentimentAsync = hasSentimentAsync;
+exports.hasNoToxicityAsync = hasNoToxicityAsync;
+exports.containsLanguageAsync = containsLanguageAsync;
+exports.hasValidCodeSyntaxAsync = hasValidCodeSyntaxAsync;
+exports.hasFactualAccuracyAsync = hasFactualAccuracyAsync;
+exports.hasNoHallucinationsAsync = hasNoHallucinationsAsync;
 exports.hasValidCodeSyntax = hasValidCodeSyntax;
 class AssertionError extends Error {
     constructor(message, expected, actual) {
@@ -591,13 +599,91 @@ function notContainsPII(text) {
 function hasPII(text) {
     return !notContainsPII(text);
 }
+/**
+ * Lexicon-based sentiment check. **Fast and approximate** — suitable for
+ * low-stakes filtering or CI smoke tests. For production safety gates use
+ * {@link hasSentimentAsync} with an LLM provider for context-aware accuracy.
+ */
 function hasSentiment(text, expected) {
-    // This is a simplified implementation
-    const positiveWords = ["good", "great", "excellent", "awesome"];
-    const negativeWords = ["bad", "terrible", "awful", "poor"];
-    const words = text.toLowerCase().split(/\s+/);
-    const positiveCount = words.filter((word) => positiveWords.includes(word)).length;
-    const negativeCount = words.filter((word) => negativeWords.includes(word)).length;
+    const lower = text.toLowerCase();
+    const positiveWords = [
+        "good",
+        "great",
+        "excellent",
+        "amazing",
+        "wonderful",
+        "fantastic",
+        "love",
+        "best",
+        "happy",
+        "helpful",
+        "awesome",
+        "superb",
+        "outstanding",
+        "brilliant",
+        "perfect",
+        "delightful",
+        "joyful",
+        "pleased",
+        "glad",
+        "terrific",
+        "fabulous",
+        "exceptional",
+        "impressive",
+        "magnificent",
+        "marvelous",
+        "splendid",
+        "positive",
+        "enjoy",
+        "enjoyed",
+        "like",
+        "liked",
+        "beautiful",
+        "innovative",
+        "inspiring",
+        "effective",
+        "useful",
+        "valuable",
+    ];
+    const negativeWords = [
+        "bad",
+        "terrible",
+        "awful",
+        "horrible",
+        "worst",
+        "hate",
+        "poor",
+        "disappointing",
+        "sad",
+        "useless",
+        "dreadful",
+        "miserable",
+        "angry",
+        "frustrated",
+        "broken",
+        "failed",
+        "pathetic",
+        "stupid",
+        "disgusting",
+        "unacceptable",
+        "wrong",
+        "error",
+        "fail",
+        "problem",
+        "negative",
+        "dislike",
+        "annoying",
+        "irritating",
+        "offensive",
+        "regret",
+        "disappointment",
+        "inadequate",
+        "mediocre",
+        "flawed",
+        "unreliable",
+    ];
+    const positiveCount = positiveWords.filter((w) => lower.includes(w)).length;
+    const negativeCount = negativeWords.filter((w) => lower.includes(w)).length;
     if (expected === "positive")
         return positiveCount > negativeCount;
     if (expected === "negative")
@@ -627,21 +713,37 @@ function isValidURL(url) {
         return false;
     }
 }
+/**
+ * Substring-based hallucination check — verifies each ground-truth fact
+ * appears verbatim in the text. **Fast and approximate**: catches missing
+ * facts but cannot detect paraphrased fabrications. Use
+ * {@link hasNoHallucinationsAsync} for semantic accuracy.
+ */
 function hasNoHallucinations(text, groundTruth) {
-    // This is a simplified implementation
-    return groundTruth.every((truth) => text.includes(truth));
+    const lower = text.toLowerCase();
+    return groundTruth.every((truth) => lower.includes(truth.toLowerCase()));
 }
 function matchesSchema(value, schema) {
-    // This is a simplified implementation
     if (typeof value !== "object" || value === null)
         return false;
-    return Object.keys(schema).every((key) => key in value);
+    const obj = value;
+    // JSON Schema: { required: ['name', 'age'] } — check required keys exist
+    if (Array.isArray(schema.required)) {
+        return schema.required.every((key) => key in obj);
+    }
+    // JSON Schema: { properties: { name: {}, age: {} } } — check property keys exist
+    if (schema.properties && typeof schema.properties === "object") {
+        return Object.keys(schema.properties).every((key) => key in obj);
+    }
+    // Simple template format: { name: '', value: '' } — all schema keys must exist in value
+    return Object.keys(schema).every((key) => key in obj);
 }
 function hasReadabilityScore(text, minScore) {
-    // This is a simplified implementation
-    const words = text.split(/\s+/).length;
-    const sentences = text.split(/[.!?]+/).length;
-    const score = 206.835 - 1.015 * (words / sentences) - 84.6 * (syllables(text) / words);
+    const wordList = text.trim().split(/\s+/).filter(Boolean);
+    const words = wordList.length || 1;
+    const sentences = text.split(/[.!?]+/).filter((s) => s.trim().length > 0).length || 1;
+    const totalSyllables = wordList.reduce((sum, w) => sum + syllables(w), 0);
+    const score = 206.835 - 1.015 * (words / sentences) - 84.6 * (totalSyllables / words);
     return score >= minScore;
 }
 function syllables(word) {
@@ -654,28 +756,402 @@ function syllables(word) {
         .trim()
         .split(/\s+/).length;
 }
+/**
+ * Keyword-frequency language detector supporting 12 languages.
+ * **Fast and approximate** — detects the most common languages reliably
+ * but may struggle with short texts or closely related languages.
+ * Use {@link containsLanguageAsync} for reliable detection of any language.
+ */
 function containsLanguage(text, language) {
-    // This is a simplified implementation
-    // In a real app, you'd use a language detection library
     const languageKeywords = {
-        en: ["the", "and", "you", "that", "was", "for", "are", "with"],
-        es: ["el", "la", "los", "las", "de", "que", "y", "en"],
-        fr: ["le", "la", "les", "de", "et", "à", "un", "une"],
+        en: [
+            "the",
+            "and",
+            "you",
+            "that",
+            "was",
+            "for",
+            "are",
+            "with",
+            "have",
+            "this",
+            "from",
+            "they",
+            "will",
+            "would",
+            "been",
+            "their",
+        ],
+        es: [
+            "el",
+            "la",
+            "los",
+            "las",
+            "de",
+            "que",
+            "y",
+            "en",
+            "es",
+            "por",
+            "para",
+            "con",
+            "una",
+            "como",
+            "pero",
+            "también",
+        ],
+        fr: [
+            "le",
+            "la",
+            "les",
+            "de",
+            "et",
+            "à",
+            "un",
+            "une",
+            "du",
+            "des",
+            "est",
+            "que",
+            "dans",
+            "pour",
+            "sur",
+            "avec",
+        ],
+        de: [
+            "der",
+            "die",
+            "das",
+            "und",
+            "ist",
+            "ich",
+            "nicht",
+            "mit",
+            "sie",
+            "ein",
+            "eine",
+            "von",
+            "zu",
+            "auf",
+            "auch",
+            "dem",
+        ],
+        it: [
+            "il",
+            "di",
+            "che",
+            "non",
+            "si",
+            "per",
+            "del",
+            "un",
+            "una",
+            "con",
+            "sono",
+            "nel",
+            "una",
+            "questo",
+            "come",
+        ],
+        pt: [
+            "de",
+            "que",
+            "do",
+            "da",
+            "em",
+            "um",
+            "para",
+            "com",
+            "uma",
+            "os",
+            "as",
+            "não",
+            "mas",
+            "por",
+            "mais",
+        ],
+        nl: [
+            "de",
+            "het",
+            "een",
+            "van",
+            "en",
+            "in",
+            "is",
+            "dat",
+            "op",
+            "te",
+            "zijn",
+            "niet",
+            "ook",
+            "met",
+            "voor",
+        ],
+        ru: [
+            "и",
+            "в",
+            "не",
+            "на",
+            "я",
+            "что",
+            "с",
+            "по",
+            "это",
+            "как",
+            "но",
+            "он",
+            "она",
+            "мы",
+            "они",
+        ],
+        zh: [
+            "的",
+            "了",
+            "是",
+            "在",
+            "我",
+            "有",
+            "和",
+            "就",
+            "不",
+            "都",
+            "也",
+            "很",
+            "会",
+            "这",
+            "他",
+        ],
+        ja: [
+            "は",
+            "が",
+            "の",
+            "に",
+            "を",
+            "で",
+            "と",
+            "た",
+            "し",
+            "て",
+            "も",
+            "な",
+            "か",
+            "から",
+            "まで",
+        ],
+        ko: [
+            "이",
+            "은",
+            "는",
+            "을",
+            "를",
+            "의",
+            "에",
+            "가",
+            "로",
+            "도",
+            "와",
+            "과",
+            "하",
+            "있",
+            "합",
+        ],
+        ar: [
+            "في",
+            "من",
+            "على",
+            "إلى",
+            "هذا",
+            "مع",
+            "أن",
+            "هو",
+            "كان",
+            "كل",
+            "التي",
+            "الذي",
+            "عن",
+            "لا",
+        ],
     };
-    const keywords = languageKeywords[language.toLowerCase()] || [];
+    const lang = language.toLowerCase();
+    const keywords = languageKeywords[lang] || languageKeywords[lang.split("-")[0]] || [];
     return keywords.some((keyword) => text.toLowerCase().includes(keyword));
 }
+/**
+ * Substring-based factual accuracy check. **Fast and approximate** — verifies
+ * each fact string appears in the text but cannot reason about meaning or
+ * paraphrasing. Use {@link hasFactualAccuracyAsync} for semantic accuracy.
+ */
 function hasFactualAccuracy(text, facts) {
-    // This is a simplified implementation
-    return facts.every((fact) => text.includes(fact));
+    const lower = text.toLowerCase();
+    return facts.every((fact) => lower.includes(fact.toLowerCase()));
 }
 function respondedWithinTime(startTime, maxMs) {
     return Date.now() - startTime <= maxMs;
 }
+/**
+ * Blocklist-based toxicity check (~80 terms across 9 categories).
+ * **Fast and approximate** — catches explicit harmful language but has
+ * inherent gaps and context-blind false positives. Do NOT rely on this
+ * alone for production content safety gates; use {@link hasNoToxicityAsync}
+ * with an LLM for context-aware moderation.
+ */
 function hasNoToxicity(text) {
-    // This is a simplified implementation
-    const toxicWords = ["hate", "stupid", "idiot", "dumb"];
-    return !toxicWords.some((word) => text.toLowerCase().includes(word));
+    const lower = text.toLowerCase();
+    const toxicTerms = [
+        // Insults and derogatory attacks
+        "idiot",
+        "moron",
+        "imbecile",
+        "dumbass",
+        "dumb ass",
+        "jackass",
+        "dimwit",
+        "nitwit",
+        "halfwit",
+        "numbskull",
+        "bonehead",
+        "blockhead",
+        "scumbag",
+        "dirtbag",
+        "slimeball",
+        "lowlife",
+        "piece of shit",
+        "piece of crap",
+        "piece of garbage",
+        "piece of trash",
+        "waste of space",
+        "waste of oxygen",
+        // Degradation directed at people
+        "you're worthless",
+        "you are worthless",
+        "you're useless",
+        "you are useless",
+        "you're pathetic",
+        "you are pathetic",
+        "you're disgusting",
+        "you are disgusting",
+        "you're nothing",
+        "you are nothing",
+        "you're scum",
+        "you're garbage",
+        "you're trash",
+        "you're a loser",
+        "complete loser",
+        "total loser",
+        "absolute loser",
+        // Violence and threats
+        "kill you",
+        "kill yourself",
+        "i'll kill",
+        "i will kill",
+        "gonna kill",
+        "going to kill",
+        "murder you",
+        "i'll murder",
+        "i will murder",
+        "want you dead",
+        "want them dead",
+        "you should die",
+        "hope you die",
+        "hope you rot",
+        "hope you suffer",
+        "shoot you",
+        "stab you",
+        "beat you up",
+        "hurt you",
+        "harm you",
+        "i'll hurt",
+        "i will hurt",
+        "i'll destroy you",
+        "i will destroy you",
+        "i'll end you",
+        "you're dead",
+        "you're going to die",
+        "i know where you live",
+        "i'll find you",
+        "i will find you",
+        // Self-harm directed at others
+        "go kill yourself",
+        "kys",
+        "end yourself",
+        "off yourself",
+        "just die",
+        "better off dead",
+        "world is better without you",
+        "no one would miss you",
+        "nobody would miss you",
+        "do us a favor and die",
+        // Dehumanization
+        "not even human",
+        "barely human",
+        "subhuman",
+        "less than human",
+        "you're vermin",
+        "you're a parasite",
+        "deserve to die",
+        "deserve to suffer",
+        "deserve pain",
+        "deserve misery",
+        // Hate and rejection
+        "i hate you",
+        "everyone hates you",
+        "they all hate you",
+        "no one likes you",
+        "nobody likes you",
+        "nobody wants you",
+        "nobody cares about you",
+        "no one cares about you",
+        "go to hell",
+        "burn in hell",
+        "rot in hell",
+        "drop dead",
+        "go die",
+        "i hope you suffer",
+        // Harassment and threats
+        "i'll ruin you",
+        "ruin your life",
+        "make your life hell",
+        "i'll expose you",
+        "i'll dox you",
+        "i will dox you",
+        "doxxing",
+        "i'll come for you",
+        "you'll pay for this",
+        // Profanity as direct attacks
+        "fuck you",
+        "fuck off",
+        "go fuck yourself",
+        "screw you",
+        "shut the fuck up",
+        "to hell with you",
+        // Bullying
+        "you're a joke",
+        "you're a laughingstock",
+        "everyone is laughing at you",
+        "pathetic loser",
+        // Appearance attacks
+        "fat pig",
+        "fat slob",
+        "hideous freak",
+        // Mental health weaponized
+        "you're crazy",
+        "you're insane",
+        "you're a psycho",
+        "you're delusional",
+        "you're mental",
+        "you belong in an asylum",
+        "you're a lunatic",
+        // Explicit profanity used as insults
+        "bastard",
+        "bitch",
+        "cunt",
+        "asshole",
+        "dipshit",
+        "douchebag",
+        "motherfucker",
+        "fucktard",
+    ];
+    return !toxicTerms.some((term) => lower.includes(term));
 }
 function followsInstructions(text, instructions) {
     return instructions.every((instruction) => {
@@ -688,16 +1164,211 @@ function followsInstructions(text, instructions) {
 function containsAllRequiredFields(obj, requiredFields) {
     return requiredFields.every((field) => obj && typeof obj === "object" && field in obj);
 }
+let _assertionLLMConfig = null;
+function configureAssertions(config) {
+    _assertionLLMConfig = config;
+}
+function getAssertionConfig() {
+    return _assertionLLMConfig;
+}
+async function callAssertionLLM(prompt, config) {
+    const cfg = config ?? _assertionLLMConfig;
+    if (!cfg) {
+        throw new Error("No LLM config set. Call configureAssertions({ provider, apiKey }) first, or pass a config as the last argument.");
+    }
+    if (cfg.provider === "openai") {
+        const baseUrl = cfg.baseUrl ?? "https://api.openai.com";
+        const model = cfg.model ?? "gpt-4o-mini";
+        const res = await fetch(`${baseUrl}/v1/chat/completions`, {
+            method: "POST",
+            headers: {
+                "Content-Type": "application/json",
+                Authorization: `Bearer ${cfg.apiKey}`,
+            },
+            body: JSON.stringify({
+                model,
+                messages: [{ role: "user", content: prompt }],
+                max_tokens: 10,
+                temperature: 0,
+            }),
+        });
+        if (!res.ok) {
+            throw new Error(`OpenAI API error ${res.status}: ${await res.text()}`);
+        }
+        const data = (await res.json());
+        return data.choices[0]?.message?.content?.trim().toLowerCase() ?? "";
+    }
+    if (cfg.provider === "anthropic") {
+        const baseUrl = cfg.baseUrl ?? "https://api.anthropic.com";
+        const model = cfg.model ?? "claude-3-haiku-20240307";
+        const res = await fetch(`${baseUrl}/v1/messages`, {
+            method: "POST",
+            headers: {
+                "Content-Type": "application/json",
+                "x-api-key": cfg.apiKey,
+                "anthropic-version": "2023-06-01",
+            },
+            body: JSON.stringify({
+                model,
+                max_tokens: 10,
+                messages: [{ role: "user", content: prompt }],
+            }),
+        });
+        if (!res.ok) {
+            throw new Error(`Anthropic API error ${res.status}: ${await res.text()}`);
+        }
+        const data = (await res.json());
+        return data.content[0]?.text?.trim().toLowerCase() ?? "";
+    }
+    throw new Error(`Unsupported provider: "${cfg.provider}". Use "openai" or "anthropic".`);
+}
+// ============================================================================
+// LLM-BACKED ASYNC ASSERTION FUNCTIONS
+// ============================================================================
+/**
+ * LLM-backed sentiment check. **Slow and accurate** — uses an LLM to
+ * classify sentiment with full context awareness. Requires
+ * {@link configureAssertions} or an inline `config` argument.
+ * Falls back gracefully with a clear error if no API key is configured.
+ */
+async function hasSentimentAsync(text, expected, config) {
+    const prompt = `Classify the sentiment of the following text. Reply with exactly one word — positive, negative, or neutral — and nothing else.\n\nText: "${text}"`;
+    const result = await callAssertionLLM(prompt, config);
+    return result.replace(/[^a-z]/g, "") === expected;
+}
+/**
+ * LLM-backed toxicity check. **Slow and accurate** — context-aware, handles
+ * sarcasm, implicit threats, and culturally specific harmful content that
+ * blocklists miss. Recommended for production content safety gates.
+ */
+async function hasNoToxicityAsync(text, config) {
+    const prompt = `Does the following text contain toxic, harmful, hateful, threatening, or abusive content? Reply with only "yes" or "no".\n\nText: "${text}"`;
+    const result = await callAssertionLLM(prompt, config);
+    return result.replace(/[^a-z]/g, "") === "no";
+}
+async function containsLanguageAsync(text, language, config) {
+    const prompt = `Is the following text primarily written in ${language}? Reply with only "yes" or "no".\n\nText: "${text}"`;
+    const result = await callAssertionLLM(prompt, config);
+    return result.replace(/[^a-z]/g, "") === "yes";
+}
+async function hasValidCodeSyntaxAsync(code, language, config) {
+    const prompt = `Is the following ${language} code free of syntax errors? Reply with only "yes" or "no".\n\nCode:\n\`\`\`${language}\n${code}\n\`\`\``;
+    const result = await callAssertionLLM(prompt, config);
+    return result.replace(/[^a-z]/g, "") === "yes";
+}
+async function hasFactualAccuracyAsync(text, facts, config) {
+    const factList = facts.map((f, i) => `${i + 1}. ${f}`).join("\n");
+    const prompt = `Does the following text accurately convey all of these facts without contradicting or omitting any?\n\nFacts:\n${factList}\n\nText: "${text}"\n\nReply with only "yes" or "no".`;
+    const result = await callAssertionLLM(prompt, config);
+    return result.replace(/[^a-z]/g, "") === "yes";
+}
+/**
+ * LLM-backed hallucination check. **Slow and accurate** — detects fabricated
+ * claims even when they are paraphrased or contradict facts indirectly.
+ */
+async function hasNoHallucinationsAsync(text, groundTruth, config) {
+    const truthList = groundTruth.map((f, i) => `${i + 1}. ${f}`).join("\n");
+    const prompt = `Does the following text stay consistent with the ground truth facts below, without introducing fabricated or hallucinated claims?\n\nGround truth:\n${truthList}\n\nText: "${text}"\n\nReply with only "yes" or "no".`;
+    const result = await callAssertionLLM(prompt, config);
+    return result.replace(/[^a-z]/g, "") === "yes";
+}
 function hasValidCodeSyntax(code, language) {
-    // This is a simplified implementation
-    // In a real app, you'd use a proper parser for each language
-    try {
-        if (language === "json")
+    const lang = language.toLowerCase();
+    if (lang === "json") {
+        try {
             JSON.parse(code);
-        // Add more language validations as needed
-        return true;
+            return true;
+        }
+        catch {
+            return false;
+        }
     }
-    catch {
-        return false;
+    // Bracket, brace, and parenthesis balance check with string/comment awareness.
+    // Catches unmatched delimiters in JS, TS, Python, Java, C, Go, Rust, and most languages.
+    // Template literals (backtick strings) are treated as opaque — their entire
+    // content including ${...} expressions is skipped, so braces inside them
+    // do not affect the balance count. This is intentional and correct.
+    // Use hasValidCodeSyntaxAsync for deeper semantic analysis.
+    const stack = [];
+    const pairs = { ")": "(", "]": "[", "}": "{" };
+    const opens = new Set(["(", "[", "{"]);
+    const closes = new Set([")", "]", "}"]);
+    const isPythonLike = lang === "python" || lang === "py" || lang === "ruby" || lang === "rb";
+    const isJSLike = lang === "javascript" ||
+        lang === "js" ||
+        lang === "typescript" ||
+        lang === "ts";
+    let inSingleQuote = false;
+    let inDoubleQuote = false;
+    let inTemplateLiteral = false;
+    let inLineComment = false;
+    let inBlockComment = false;
+    for (let i = 0; i < code.length; i++) {
+        const ch = code[i];
+        const next = code[i + 1] ?? "";
+        const prev = code[i - 1] ?? "";
+        if (inLineComment) {
+            if (ch === "\n")
+                inLineComment = false;
+            continue;
+        }
+        if (inBlockComment) {
+            if (ch === "*" && next === "/") {
+                inBlockComment = false;
+                i++;
+            }
+            continue;
+        }
+        if (inSingleQuote) {
+            if (ch === "'" && prev !== "\\")
+                inSingleQuote = false;
+            continue;
+        }
+        if (inDoubleQuote) {
+            if (ch === '"' && prev !== "\\")
+                inDoubleQuote = false;
+            continue;
+        }
+        if (inTemplateLiteral) {
+            if (ch === "`" && prev !== "\\")
+                inTemplateLiteral = false;
+            continue;
+        }
+        if (ch === "/" && next === "/") {
+            inLineComment = true;
+            i++;
+            continue;
+        }
+        if (ch === "/" && next === "*") {
+            inBlockComment = true;
+            i++;
+            continue;
+        }
+        if (isPythonLike && ch === "#") {
+            inLineComment = true;
+            continue;
+        }
+        if (ch === "'") {
+            inSingleQuote = true;
+            continue;
+        }
+        if (ch === '"') {
+            inDoubleQuote = true;
+            continue;
+        }
+        if (isJSLike && ch === "`") {
+            inTemplateLiteral = true;
+            continue;
+        }
+        if (opens.has(ch)) {
+            stack.push(ch);
+        }
+        else if (closes.has(ch)) {
+            if (stack.length === 0 || stack[stack.length - 1] !== pairs[ch]) {
+                return false;
+            }
+            stack.pop();
+        }
     }
+    return stack.length === 0;
 }

package/dist/export.d.ts

@@ -140,7 +140,7 @@ export declare function exportData(client: AIEvalClient, options: ExportOptions)
  * console.log(`Imported ${result.summary.imported} items`);
  * ```
  */
-export declare function importData(client: AIEvalClient, data: ExportData, options: ImportOptions): Promise<ImportResult>;
+export declare function importData(client: AIEvalClient, data: ExportData, options?: ImportOptions): Promise<ImportResult>;
 /**
  * Export data to JSON file
  *

package/dist/export.js

@@ -136,7 +136,7 @@ async function exportData(client, options) {
  * console.log(`Imported ${result.summary.imported} items`);
  * ```
  */
-async function importData(client, data, options) {
+async function importData(client, data, options = {}) {
     const result = {
         summary: { total: 0, imported: 0, skipped: 0, failed: 0 },
         details: {},

package/dist/index.d.ts

@@ -10,7 +10,7 @@ export { AIEvalClient } from "./client";
 import { AuthenticationError, EvalGateError, NetworkError, RateLimitError, SDKError } from "./errors";
 export { EvalGateError, RateLimitError, AuthenticationError, SDKError as ValidationError, // Using SDKError as ValidationError for backward compatibility
 NetworkError, };
-export { containsAllRequiredFields, containsJSON, containsKeywords, containsLanguage, expect, followsInstructions, hasFactualAccuracy, hasLength, hasNoHallucinations, hasNoToxicity, hasPII, hasReadabilityScore, hasSentiment, hasValidCodeSyntax, isValidEmail, isValidURL, matchesPattern, matchesSchema, notContainsPII, respondedWithinTime, similarTo, withinRange, } from "./assertions";
+export { type AssertionLLMConfig, configureAssertions, containsAllRequiredFields, containsJSON, containsKeywords, containsLanguage, containsLanguageAsync, expect, followsInstructions, getAssertionConfig, hasFactualAccuracy, hasFactualAccuracyAsync, hasLength, hasNoHallucinations, hasNoHallucinationsAsync, hasNoToxicity, hasNoToxicityAsync, hasPII, hasReadabilityScore, hasSentiment, hasSentimentAsync, hasValidCodeSyntax, hasValidCodeSyntaxAsync, isValidEmail, isValidURL, matchesPattern, matchesSchema, notContainsPII, respondedWithinTime, similarTo, withinRange, } from "./assertions";
 import { createContext, EvalContext, getCurrentContext, withContext } from "./context";
 export { createContext, getCurrentContext as getContext, withContext, EvalContext as ContextManager, };
 export { cloneContext, mergeContexts, validateContext, } from "./runtime/context";

package/dist/index.js

@@ -8,8 +8,8 @@
  * @packageDocumentation
  */
 Object.defineProperty(exports, "__esModule", { value: true });
-exports.SpecRegistrationError = exports.SpecExecutionError = exports.RuntimeError = exports.EvalRuntimeError = exports.setActiveRuntime = exports.getActiveRuntime = exports.disposeActiveRuntime = exports.createEvalRuntime = exports.defaultLocalExecutor = exports.createLocalExecutor = exports.evalai = exports.defineSuite = exports.defineEval = exports.createResult = exports.createEvalContext = exports.validateContext = exports.mergeContexts = exports.cloneContext = exports.ContextManager = exports.withContext = exports.getContext = exports.createContext = exports.withinRange = exports.similarTo = exports.respondedWithinTime = exports.notContainsPII = exports.matchesSchema = exports.matchesPattern = exports.isValidURL = exports.isValidEmail = exports.hasValidCodeSyntax = exports.hasSentiment = exports.hasReadabilityScore = exports.hasPII = exports.hasNoToxicity = exports.hasNoHallucinations = exports.hasLength = exports.hasFactualAccuracy = exports.followsInstructions = exports.expect = exports.containsLanguage = exports.containsKeywords = exports.containsJSON = exports.containsAllRequiredFields = exports.NetworkError = exports.ValidationError = exports.AuthenticationError = exports.RateLimitError = exports.EvalGateError = exports.AIEvalClient = void 0;
-exports.WorkflowTracer = exports.traceWorkflowStep = exports.traceLangChainAgent = exports.traceCrewAI = exports.traceAutoGen = exports.createWorkflowTracer = exports.EvaluationTemplates = exports.streamEvaluation = exports.RateLimiter = exports.batchRead = exports.batchProcess = exports.REPORT_SCHEMA_VERSION = exports.GATE_EXIT = exports.GATE_CATEGORY = exports.ARTIFACTS = exports.PaginatedIterator = exports.encodeCursor = exports.decodeCursor = exports.createPaginatedIterator = exports.autoPaginate = exports.extendExpectWithToPassGate = exports.Logger = exports.openAIChatEval = exports.traceOpenAI = exports.traceAnthropic = exports.runCheck = exports.parseArgs = exports.EXIT = exports.RequestCache = exports.CacheTTL = exports.RequestBatcher = exports.importData = exports.exportData = exports.compareSnapshots = exports.saveSnapshot = exports.compareWithSnapshot = exports.snapshot = exports.TestSuite = exports.createTestSuite = void 0;
+exports.defaultLocalExecutor = exports.createLocalExecutor = exports.evalai = exports.defineSuite = exports.defineEval = exports.createResult = exports.createEvalContext = exports.validateContext = exports.mergeContexts = exports.cloneContext = exports.ContextManager = exports.withContext = exports.getContext = exports.createContext = exports.withinRange = exports.similarTo = exports.respondedWithinTime = exports.notContainsPII = exports.matchesSchema = exports.matchesPattern = exports.isValidURL = exports.isValidEmail = exports.hasValidCodeSyntaxAsync = exports.hasValidCodeSyntax = exports.hasSentimentAsync = exports.hasSentiment = exports.hasReadabilityScore = exports.hasPII = exports.hasNoToxicityAsync = exports.hasNoToxicity = exports.hasNoHallucinationsAsync = exports.hasNoHallucinations = exports.hasLength = exports.hasFactualAccuracyAsync = exports.hasFactualAccuracy = exports.getAssertionConfig = exports.followsInstructions = exports.expect = exports.containsLanguageAsync = exports.containsLanguage = exports.containsKeywords = exports.containsJSON = exports.containsAllRequiredFields = exports.configureAssertions = exports.NetworkError = exports.ValidationError = exports.AuthenticationError = exports.RateLimitError = exports.EvalGateError = exports.AIEvalClient = void 0;
+exports.WorkflowTracer = exports.traceWorkflowStep = exports.traceLangChainAgent = exports.traceCrewAI = exports.traceAutoGen = exports.createWorkflowTracer = exports.EvaluationTemplates = exports.streamEvaluation = exports.RateLimiter = exports.batchRead = exports.batchProcess = exports.REPORT_SCHEMA_VERSION = exports.GATE_EXIT = exports.GATE_CATEGORY = exports.ARTIFACTS = exports.PaginatedIterator = exports.encodeCursor = exports.decodeCursor = exports.createPaginatedIterator = exports.autoPaginate = exports.extendExpectWithToPassGate = exports.Logger = exports.openAIChatEval = exports.traceOpenAI = exports.traceAnthropic = exports.runCheck = exports.parseArgs = exports.EXIT = exports.RequestCache = exports.CacheTTL = exports.RequestBatcher = exports.importData = exports.exportData = exports.compareSnapshots = exports.saveSnapshot = exports.compareWithSnapshot = exports.snapshot = exports.TestSuite = exports.createTestSuite = exports.SpecRegistrationError = exports.SpecExecutionError = exports.RuntimeError = exports.EvalRuntimeError = exports.setActiveRuntime = exports.getActiveRuntime = exports.disposeActiveRuntime = exports.createEvalRuntime = void 0;
 // Main SDK exports
 var client_1 = require("./client");
 Object.defineProperty(exports, "AIEvalClient", { enumerable: true, get: function () { return client_1.AIEvalClient; } });
@@ -22,20 +22,30 @@ Object.defineProperty(exports, "RateLimitError", { enumerable: true, get: functi
 Object.defineProperty(exports, "ValidationError", { enumerable: true, get: function () { return errors_1.SDKError; } });
 // Enhanced assertions (Tier 1.3)
 var assertions_1 = require("./assertions");
+// LLM config
+Object.defineProperty(exports, "configureAssertions", { enumerable: true, get: function () { return assertions_1.configureAssertions; } });
 Object.defineProperty(exports, "containsAllRequiredFields", { enumerable: true, get: function () { return assertions_1.containsAllRequiredFields; } });
 Object.defineProperty(exports, "containsJSON", { enumerable: true, get: function () { return assertions_1.containsJSON; } });
 Object.defineProperty(exports, "containsKeywords", { enumerable: true, get: function () { return assertions_1.containsKeywords; } });
 Object.defineProperty(exports, "containsLanguage", { enumerable: true, get: function () { return assertions_1.containsLanguage; } });
+// LLM-backed async variants
+Object.defineProperty(exports, "containsLanguageAsync", { enumerable: true, get: function () { return assertions_1.containsLanguageAsync; } });
 Object.defineProperty(exports, "expect", { enumerable: true, get: function () { return assertions_1.expect; } });
 Object.defineProperty(exports, "followsInstructions", { enumerable: true, get: function () { return assertions_1.followsInstructions; } });
+Object.defineProperty(exports, "getAssertionConfig", { enumerable: true, get: function () { return assertions_1.getAssertionConfig; } });
 Object.defineProperty(exports, "hasFactualAccuracy", { enumerable: true, get: function () { return assertions_1.hasFactualAccuracy; } });
+Object.defineProperty(exports, "hasFactualAccuracyAsync", { enumerable: true, get: function () { return assertions_1.hasFactualAccuracyAsync; } });
 Object.defineProperty(exports, "hasLength", { enumerable: true, get: function () { return assertions_1.hasLength; } });
 Object.defineProperty(exports, "hasNoHallucinations", { enumerable: true, get: function () { return assertions_1.hasNoHallucinations; } });
+Object.defineProperty(exports, "hasNoHallucinationsAsync", { enumerable: true, get: function () { return assertions_1.hasNoHallucinationsAsync; } });
 Object.defineProperty(exports, "hasNoToxicity", { enumerable: true, get: function () { return assertions_1.hasNoToxicity; } });
+Object.defineProperty(exports, "hasNoToxicityAsync", { enumerable: true, get: function () { return assertions_1.hasNoToxicityAsync; } });
 Object.defineProperty(exports, "hasPII", { enumerable: true, get: function () { return assertions_1.hasPII; } });
 Object.defineProperty(exports, "hasReadabilityScore", { enumerable: true, get: function () { return assertions_1.hasReadabilityScore; } });
 Object.defineProperty(exports, "hasSentiment", { enumerable: true, get: function () { return assertions_1.hasSentiment; } });
+Object.defineProperty(exports, "hasSentimentAsync", { enumerable: true, get: function () { return assertions_1.hasSentimentAsync; } });
 Object.defineProperty(exports, "hasValidCodeSyntax", { enumerable: true, get: function () { return assertions_1.hasValidCodeSyntax; } });
+Object.defineProperty(exports, "hasValidCodeSyntaxAsync", { enumerable: true, get: function () { return assertions_1.hasValidCodeSyntaxAsync; } });
 Object.defineProperty(exports, "isValidEmail", { enumerable: true, get: function () { return assertions_1.isValidEmail; } });
 Object.defineProperty(exports, "isValidURL", { enumerable: true, get: function () { return assertions_1.isValidURL; } });
 Object.defineProperty(exports, "matchesPattern", { enumerable: true, get: function () { return assertions_1.matchesPattern; } });

package/dist/snapshot.d.ts

@@ -73,7 +73,7 @@ export declare class SnapshotManager {
      * await manager.save('haiku-test', output, { tags: ['poetry'] });
      * ```
      */
-    save(name: string, output: string, options?: {
+    save(name: string, output: unknown, options?: {
         tags?: string[];
         metadata?: Record<string, unknown>;
         overwrite?: boolean;
@@ -99,7 +99,7 @@ export declare class SnapshotManager {
      * }
      * ```
      */
-    compare(name: string, currentOutput: string): Promise<SnapshotComparison>;
+    compare(name: string, currentOutput: unknown): Promise<SnapshotComparison>;
     /**
      * List all snapshots
      *
@@ -127,7 +127,7 @@ export declare class SnapshotManager {
      * await manager.update('haiku-test', newOutput);
      * ```
      */
-    update(name: string, output: string): Promise<SnapshotData>;
+    update(name: string, output: unknown): Promise<SnapshotData>;
 }
 /**
  * Save a snapshot (convenience function)
@@ -138,7 +138,7 @@ export declare class SnapshotManager {
  * await snapshot('haiku-test', output);
  * ```
  */
-export declare function snapshot(name: string, output: string, options?: {
+export declare function snapshot(name: string, output: unknown, options?: {
     tags?: string[];
     metadata?: Record<string, unknown>;
     overwrite?: boolean;
@@ -165,7 +165,7 @@ export declare function loadSnapshot(name: string, dir?: string): Promise<Snapsh
  * }
  * ```
  */
-export declare function compareWithSnapshot(name: string, currentOutput: string, dir?: string): Promise<SnapshotComparison>;
+export declare function compareWithSnapshot(name: string, currentOutput: unknown, dir?: string): Promise<SnapshotComparison>;
 /**
  * Delete a snapshot (convenience function)
  */

package/dist/snapshot.js

@@ -130,12 +130,13 @@ class SnapshotManager {
         if (!options?.overwrite && fs.existsSync(filePath)) {
             throw new Error(`Snapshot '${name}' already exists. Use overwrite: true to update.`);
         }
+        const serialized = typeof output === "string" ? output : JSON.stringify(output);
         const snapshotData = {
-            output,
+            output: serialized,
             metadata: {
                 name,
                 createdAt: new Date().toISOString(),
-                hash: this.generateHash(output),
+                hash: this.generateHash(serialized),
                 tags: options?.tags,
                 metadata: options?.metadata,
             },
@@ -174,11 +175,14 @@ class SnapshotManager {
     async compare(name, currentOutput) {
         const snapshot = await this.load(name);
         const original = snapshot.output;
+        const currentOutputStr = typeof currentOutput === "string"
+            ? currentOutput
+            : JSON.stringify(currentOutput);
         // Exact match check
-        const exactMatch = original === currentOutput;
+        const exactMatch = original === currentOutputStr;
         // Calculate similarity (simple line-based diff)
         const originalLines = original.split("\n");
-        const currentLines = currentOutput.split("\n");
+        const currentLines = currentOutputStr.split("\n");
         const differences = [];
         const maxLines = Math.max(originalLines.length, currentLines.length);
         let matchingLines = 0;
@@ -198,7 +202,7 @@ class SnapshotManager {
             similarity,
             differences,
             original,
-            current: currentOutput,
+            current: currentOutputStr,
         };
     }
     /**

package/dist/types.d.ts

@@ -38,8 +38,13 @@ export interface ClientConfig {
     keepAlive?: boolean;
 }
 /**
- * Evaluation template categories
- * Updated with new template types for comprehensive LLM testing
+ * Evaluation template identifier constants for use with the EvalAI platform API.
+ *
+ * These are **string identifiers** (e.g. `"unit-testing"`) that reference
+ * pre-built templates on the platform — not template definition objects.
+ * Pass these values to `evaluations.create({ templateId: EvaluationTemplates.UNIT_TESTING })`
+ * to spin up a pre-configured evaluation. For custom criteria, thresholds, and
+ * test cases, build your own evaluation config instead.
  */
 export declare const EvaluationTemplates: {
     readonly UNIT_TESTING: "unit-testing";

package/dist/types.js

@@ -2,8 +2,13 @@
 Object.defineProperty(exports, "__esModule", { value: true });
 exports.SDKError = exports.EvaluationTemplates = void 0;
 /**
- * Evaluation template categories
- * Updated with new template types for comprehensive LLM testing
+ * Evaluation template identifier constants for use with the EvalAI platform API.
+ *
+ * These are **string identifiers** (e.g. `"unit-testing"`) that reference
+ * pre-built templates on the platform — not template definition objects.
+ * Pass these values to `evaluations.create({ templateId: EvaluationTemplates.UNIT_TESTING })`
+ * to spin up a pre-configured evaluation. For custom criteria, thresholds, and
+ * test cases, build your own evaluation config instead.
  */
 exports.EvaluationTemplates = {
     // Core Testing

package/dist/version.d.ts

@@ -3,5 +3,5 @@
  * X-EvalGate-SDK-Version: SDK package version
  * X-EvalGate-Spec-Version: OpenAPI spec version (docs/openapi.json info.version)
  */
-export declare const SDK_VERSION = "2.2.0";
-export declare const SPEC_VERSION = "2.2.0";
+export declare const SDK_VERSION = "2.2.2";
+export declare const SPEC_VERSION = "2.2.2";

package/dist/version.js

@@ -6,5 +6,5 @@ exports.SPEC_VERSION = exports.SDK_VERSION = void 0;
  * X-EvalGate-SDK-Version: SDK package version
  * X-EvalGate-Spec-Version: OpenAPI spec version (docs/openapi.json info.version)
  */
-exports.SDK_VERSION = "2.2.0";
-exports.SPEC_VERSION = "2.2.0";
+exports.SDK_VERSION = "2.2.2";
+exports.SPEC_VERSION = "2.2.2";

package/dist/workflows.js

@@ -64,8 +64,13 @@ class WorkflowTracer {
         this.costs = [];
         this.spanCounter = 0;
         this.client = client;
+        const resolvedOrgId = options.organizationId ??
+            (typeof client?.getOrganizationId === "function"
+                ? client.getOrganizationId()
+                : undefined) ??
+            0;
         this.options = {
-            organizationId: options.organizationId || client.getOrganizationId() || 0,
+            organizationId: resolvedOrgId,
             autoCalculateCost: options.autoCalculateCost ?? true,
             tracePrefix: options.tracePrefix || "workflow",
             captureFullPayloads: options.captureFullPayloads ?? true,

package/package.json

@@ -1,6 +1,6 @@
 {
 	"name": "@evalgate/sdk",
-	"version": "2.2.0",
+	"version": "2.2.2",
 	"publishConfig": {
 		"access": "public",
 		"registry": "https://registry.npmjs.org/"
@@ -16,7 +16,7 @@
 		"CHANGELOG.md"
 	],
 	"bin": {
-		"evalgate": "./dist/cli/index.js"
+		"evalgate": "dist/cli/index.js"
 	},
 	"engines": {
 		"node": ">=16.0.0"