ai-functions 0.2.19 → 0.4.0

package/evals/README.md

@@ -0,0 +1,212 @@
+# AI Functions Eval Suite
+
+Evaluations for ai-functions using both vitest-based tests and a custom eval runner.
+
+## Quick Start
+
+### Vitest-Based Evals (Recommended)
+
+Tests the core AI primitives (`code`, `ai`, `list`, `is`, `defineFunction`, etc.) with real AI calls:
+
+```bash
+# Run all eval tests
+pnpm test:evals
+
+# Run primitives eval (code, ai, list, is, etc.)
+pnpm test:evals:primitives
+
+# Run defineFunction eval
+pnpm test:evals:define
+
+# Run with specific model
+MODEL=sonnet pnpm test:evals
+
+# Run with specific tiers
+EVAL_TIERS=best,fast pnpm test:evals
+```
+
+### Custom Runner Evals
+
+Math and classification evals with detailed scoring:
+
+```bash
+# Run all evals (math + classification)
+pnpm eval
+
+# Run specific eval
+pnpm eval:math
+pnpm eval:class
+
+# Run all tiers (best, fast, cheap)
+pnpm eval:all
+```
+
+## Eval Suites
+
+### Vitest Evals (test/evals/)
+
+| Test Suite | Functions Tested | Test Cases |
+|------------|------------------|------------|
+| `primitives.eval.test.ts` | `code()`, `ai()`, `list()`, `is()`, `summarize()`, `extract()`, `write()`, `lists()` | Code generation, text generation, classification, extraction |
+| `define-function.eval.test.ts` | `defineFunction()`, `define.generative()`, `define.code()` | Generative functions, code functions, structured outputs |
+
+### Custom Runner Evals (evals/)
+
+| Eval | Tests | Scoring |
+|------|-------|---------|
+| `Math` | Arithmetic, word problems | Correct answer + shows work |
+| `Classification` | Sentiment, support tickets | Accuracy + calibration |
+| `Marketing` | Marketing copy generation | LLM-as-judge ELO ranking |
+
+### Marketing Copy Eval (LLM-as-Judge)
+
+```bash
+# Run marketing eval (fast tier only)
+pnpm eval:marketing
+
+# Run with all tiers
+pnpm eval:marketing:all
+
+# Use different judge model
+pnpm eval:marketing -- --judge=opus
+```
+
+Generates marketing copy (title, description, hero headline/subhead, CTAs) for different scenarios and uses pairwise LLM-as-judge comparisons to create ELO rankings.
+
+## Latest Results (December 2025)
+
+**Overall: 94.0%** | Cost: $0.06 | Time: 95s | 10 Models
+
+### Performance Summary
+
+| Model | Math | Class | Overall | Avg Latency | Notes |
+|-------|------|-------|---------|-------------|-------|
+| Claude Sonnet 4.5 | 100% | 100% | **100%** | ~380ms | Best overall |
+| GPT-5 Mini | 100% | 91.7% | 95.9% | ~1850ms | Slower but accurate |
+| Gemini 2.5 Flash | 100% | 91.7% | 95.9% | ~200ms | **Fastest** |
+| DeepSeek Chat | 100% | 91.7% | 95.9% | ~210ms | Great value |
+| Mistral Medium 3.1 | 96% | 100% | 98.0% | ~850ms | Strong classify |
+| Grok 4.1 Fast | 100% | 91.7% | 95.9% | ~2300ms | 2M context |
+| Grok 4 Fast | 92% | 100% | 96.0% | ~1800ms | Good balance |
+| Qwen3 30B | 96% | 91.7% | 93.9% | ~8900ms | Slowest |
+| Llama 3.3 70B | 90% | 91.7% | 90.9% | ~185ms | Fast open model |
+| GPT-oss 20B | 72% | 83.3% | 77.7% | ~1200ms | Open source |
+
+### Performance/$ Analysis (Fast Tier)
+
+| Model | Score | Est $/1M tokens | Score/$ | Recommendation |
+|-------|-------|-----------------|---------|----------------|
+| DeepSeek Chat | 95.9% | $0.28 | **342** | Best value |
+| Gemini 2.5 Flash | 95.9% | $0.30 | 320 | Fast + cheap |
+| Llama 3.3 70B | 90.9% | $0.40 | 227 | Good OSS option |
+| Claude Sonnet 4.5 | 100% | $3.00 | 33 | Best quality |
+| Mistral Medium 3.1 | 98.0% | $2.50 | 39 | Strong balance |
+| GPT-5 Mini | 95.9% | $1.00 | 96 | OpenAI ecosystem |
+| Grok 4.1 Fast | 95.9% | $2.00 | 48 | 2M context |
+
+### Math Eval (94.6%)
+
+| Model | Score | Avg Latency |
+|-------|-------|-------------|
+| Claude Sonnet 4.5 | 100% | ~380ms |
+| GPT-5 Mini | 100% | ~200ms |
+| Gemini 2.5 Flash | 100% | ~170ms |
+| DeepSeek Chat | 100% | ~220ms |
+| Grok 4.1 Fast | 100% | ~2600ms |
+| Mistral Medium 3.1 | 96% | ~1040ms |
+| Qwen3 30B | 96% | ~13000ms |
+| Grok 4 Fast | 92% | ~2000ms |
+| Llama 3.3 70B | 90% | ~170ms |
+| GPT-oss 20B | 72% | ~180ms |
+
+### Classification Eval (93.3%)
+
+| Model | Score | Avg Latency |
+|-------|-------|-------------|
+| Claude Sonnet 4.5 | 100% | ~205ms |
+| Mistral Medium 3.1 | 100% | ~700ms |
+| Grok 4 Fast | 100% | ~1670ms |
+| GPT-5 Mini | 91.7% | ~3500ms |
+| Gemini 2.5 Flash | 91.7% | ~235ms |
+| Llama 3.3 70B | 91.7% | ~230ms |
+| DeepSeek Chat | 91.7% | ~230ms |
+| Qwen3 30B | 91.7% | ~3970ms |
+| Grok 4.1 Fast | 91.7% | ~2170ms |
+| GPT-oss 20B | 83.3% | ~2840ms |
+
+### Marketing Copy Eval (ELO Rankings)
+
+Uses LLM-as-judge (Claude Sonnet) for pairwise comparisons across 4 test scenarios.
+
+| Rank | Model | ELO | W | L | D | Notes |
+|------|-------|-----|---|---|---|-------|
+| 1 | Claude Sonnet 4.5 | **1745** | 31 | 3 | 0 | Dominant winner |
+| 2 | Grok 4.1 Fast | 1595 | 22 | 12 | 0 | Strong creative |
+| 3 | GPT-5 Mini | 1593 | 26 | 8 | 0 | Consistent quality |
+| 4 | Grok 4 Fast | 1558 | 17 | 17 | 0 | Good balance |
+| 5 | Gemini 2.5 Flash | 1503 | 14 | 20 | 0 | Middle tier |
+| 6 | Mistral Medium 3.1 | 1481 | 16 | 18 | 0 | Solid performer |
+| 7 | GPT-oss 20B | 1471 | 19 | 15 | 0 | OSS option |
+| 8 | DeepSeek Chat | 1449 | 10 | 16 | 0 | Value option |
+| 9 | Qwen3 30B | 1371 | 6 | 20 | 0 | Below average |
+| 10 | Llama 3.3 70B | 1231 | 1 | 33 | 0 | Struggled |
+
+**Key Insights:**
+- Claude Sonnet 4.5 won 31 of 34 comparisons (91%)
+- Grok models performed unexpectedly well on creative tasks
+- Llama 3.3 70B, despite being strong on classification, struggled with marketing copy
+
+## Models
+
+Uses model IDs from `language-models` package, routed via `ai-providers`:
+
+### Model Tiers
+
+| Tier | Description | Models |
+|------|-------------|--------|
+| `best` | Highest capability | opus, o3, gpt-5.1, gemini-pro, deepseek-v3.2, mistral-large-3, qwen3-coder, grok-4 |
+| `fast` | Good balance | sonnet, gpt-5-mini, flash, llama-3.3-70b, mistral-medium-3.1, qwen3-30b, grok-4.1-fast |
+| `cheap` | Cost-optimized | haiku, gpt-5-nano, ministral-14b |
+
+### Full Model List (December 2025)
+
+- **Anthropic**: `opus`, `sonnet`, `haiku`
+- **OpenAI**: `openai/gpt-5.1`, `openai/gpt-5-mini`, `openai/gpt-5-nano`, `openai/o3`
+- **OpenAI OSS**: `openai/gpt-oss-120b`, `openai/gpt-oss-20b` (open source models)
+- **Google**: `gemini-pro`, `flash`
+- **Meta**: `meta-llama/llama-4-maverick`, `meta-llama/llama-3.3-70b-instruct`
+- **DeepSeek**: `deepseek/deepseek-v3.2`, `deepseek/deepseek-v3.2-speciale`, `deepseek/deepseek-chat`
+- **Mistral**: `mistralai/mistral-large-2512` (Mistral Large 3), `mistralai/mistral-medium-3.1`, `mistralai/ministral-14b-2512`
+- **Qwen**: `qwen/qwen3-coder`, `qwen/qwen3-30b-a3b`, `qwen/qwen3-next-80b-a3b-instruct`
+- **xAI**: `x-ai/grok-4`, `x-ai/grok-4.1-fast`, `x-ai/grok-4-fast`
+
+## Environment
+
+```bash
+# Use AI Gateway (recommended)
+AI_GATEWAY_URL=https://gateway.ai.cloudflare.com/v1/...
+AI_GATEWAY_TOKEN=...
+
+# Or direct API keys
+ANTHROPIC_API_KEY=sk-ant-...
+OPENAI_API_KEY=sk-...
+```
+
+## Adding Evals
+
+### Vitest-Based Evals
+
+1. Create a new test file in `test/evals/`
+2. Import functions and models:
+   ```typescript
+   import { code, ai, list } from '../../src/primitives.js'
+   import { EVAL_MODELS, type EvalModel } from '../../src/eval/models.js'
+   ```
+3. Use `describe.skipIf(!hasAPI)` to skip when no API access
+4. Loop over models with `for (const model of models)`
+
+### Custom Runner Evals
+
+1. Add test cases to `evals/run-evals.ts`
+2. Use `runEval()` with `task` function and `scorers` array
+3. Use `createModelVariants({ tiers: ['fast'] })` to filter models

package/evals/classification.eval.ts

@@ -0,0 +1,108 @@
+/**
+ * Classification Eval
+ *
+ * Tests model ability to classify inputs correctly.
+ * Includes sentiment analysis, category classification, and boolean questions.
+ */
+
+import { evalite } from 'evalite'
+import { generateObject } from '../src/generate.js'
+import { schema } from '../src/schema.js'
+import { createModelVariants, type EvalModel } from '../src/eval/models.js'
+
+// Classification test cases
+const TEST_CASES = [
+  // Sentiment
+  { text: 'This product exceeded my expectations!', expected: 'positive', options: ['positive', 'negative', 'neutral'] },
+  { text: 'The delivery was late and packaging damaged.', expected: 'negative', options: ['positive', 'negative', 'neutral'] },
+  { text: 'The product arrived as described.', expected: 'neutral', options: ['positive', 'negative', 'neutral'] },
+
+  // Support ticket classification
+  { text: 'I need to reset my password', expected: 'account', options: ['account', 'billing', 'technical', 'shipping'] },
+  { text: 'When will my refund be processed?', expected: 'billing', options: ['account', 'billing', 'technical', 'shipping'] },
+  { text: 'The app crashes when uploading images', expected: 'technical', options: ['account', 'billing', 'technical', 'shipping'] },
+  { text: 'My package shows delivered but I never received it', expected: 'shipping', options: ['account', 'billing', 'technical', 'shipping'] },
+]
+
+const modelVariants = createModelVariants({ tiers: ['fast', 'cheap'] })
+
+evalite.each(modelVariants)('Classification', {
+  data: TEST_CASES.map(tc => ({ input: tc, expected: tc.expected })),
+
+  task: async (input, variant) => {
+    const model = variant as EvalModel
+    const startTime = Date.now()
+
+    const enumStr = input.options.join(' | ')
+
+    const { object, usage } = await generateObject({
+      model: model.id,
+      schema: schema({
+        category: enumStr,
+        confidence: 'Confidence 0-1 (number)',
+      }),
+      prompt: `Classify this text into one of: ${input.options.join(', ')}
+
+Text: "${input.text}"`,
+    })
+
+    const latencyMs = Date.now() - startTime
+
+    return {
+      predicted: object.category,
+      confidence: object.confidence,
+      expected: input.expected,
+      text: input.text,
+      options: input.options,
+      modelId: model.id,
+      modelName: model.name,
+      latencyMs,
+      usage,
+    }
+  },
+
+  scorers: [
+    // Accuracy
+    {
+      name: 'Accuracy',
+      description: 'Whether classification is correct',
+      scorer: ({ output, expected }) => ({
+        score: output.predicted === expected ? 1 : 0,
+      }),
+    },
+
+    // Valid category
+    {
+      name: 'Valid Category',
+      description: 'Whether output is a valid option',
+      scorer: ({ output }) => ({
+        score: (output.options as string[]).includes(output.predicted as string) ? 1 : 0,
+      }),
+    },
+
+    // Calibration
+    {
+      name: 'Calibration',
+      description: 'Confidence matches accuracy',
+      scorer: ({ output, expected }) => {
+        const correct = output.predicted === expected
+        const conf = output.confidence as number
+
+        // High confidence when correct, low when wrong = well calibrated
+        if (correct && conf >= 0.7) return { score: 1 }
+        if (!correct && conf <= 0.5) return { score: 0.8 }
+        if (correct && conf < 0.5) return { score: 0.6 } // Underconfident
+        if (!correct && conf > 0.7) return { score: 0.2 } // Overconfident
+        return { score: 0.5 }
+      },
+    },
+  ],
+
+  columns: ({ output, expected }) => [
+    { label: 'Model', value: output.modelName },
+    { label: 'Expected', value: expected },
+    { label: 'Got', value: output.predicted },
+    { label: 'Correct', value: output.predicted === expected ? 'Yes' : 'No' },
+    { label: 'Confidence', value: `${((output.confidence as number) * 100).toFixed(0)}%` },
+  ],
+})

package/evals/marketing.eval.ts

@@ -0,0 +1,370 @@
+#!/usr/bin/env npx tsx
+/**
+ * Marketing Copy Eval with LLM-as-Judge ELO Ranking
+ *
+ * Generates marketing copy (title, description, hero headline/subhead, CTAs)
+ * and uses pairwise comparison with an LLM judge to create ELO rankings.
+ *
+ * Usage:
+ *   npx tsx evals/marketing.eval.ts
+ *   npx tsx evals/marketing.eval.ts --judge=opus        # Use specific judge model
+ *   npx tsx evals/marketing.eval.ts --judge=haiku       # Test cheap judge
+ *   npx tsx evals/marketing.eval.ts --judge=flash       # Test fast judge
+ *   npx tsx evals/marketing.eval.ts --all               # Run all tiers
+ *   npx tsx evals/marketing.eval.ts --all --judge=haiku # All tiers + cheap judge
+ */
+
+// Load .env from project root
+import { config } from 'dotenv'
+import { resolve } from 'path'
+config({ path: resolve(import.meta.dirname, '../../../.env') })
+
+import { generateObject } from '../src/generate.js'
+import { schema } from '../src/schema.js'
+import { EVAL_MODELS, type EvalModel, type ModelTier } from '../src/eval/models.js'
+
+// Parse CLI args
+const args = process.argv.slice(2)
+const judgeArg = args.find(a => a.startsWith('--judge='))
+const JUDGE_MODEL = judgeArg ? judgeArg.split('=')[1] : 'sonnet'
+const runAll = args.includes('--all')
+
+const tiers: ModelTier[] = runAll ? ['best', 'fast', 'cheap'] : ['fast']
+
+// Marketing copy schema
+const marketingCopySchema = schema({
+  title: 'Product/page title (5-10 words)',
+  description: 'Meta description for SEO (150-160 characters)',
+  hero: {
+    headline: 'Hero headline (5-8 words, compelling)',
+    subhead: 'Supporting subheadline (10-20 words)',
+    primaryCTA: 'Primary call-to-action button text (2-4 words)',
+    secondaryCTA: 'Secondary call-to-action link text (3-6 words)',
+  },
+})
+
+// Test cases - different product/service scenarios
+const TEST_CASES = [
+  {
+    name: 'SaaS Analytics Platform',
+    prompt: `Create marketing copy for a B2B SaaS analytics platform called "InsightFlow" that helps companies understand their customer behavior with AI-powered insights. Target audience: Product managers and growth teams at mid-size tech companies.`,
+  },
+  {
+    name: 'E-commerce Fashion Brand',
+    prompt: `Create marketing copy for a sustainable fashion e-commerce brand called "EcoThread" that sells organic, ethically-made clothing. Target audience: Environmentally conscious millennials aged 25-35.`,
+  },
+  {
+    name: 'Developer Tool',
+    prompt: `Create marketing copy for a CLI tool called "DeployFast" that simplifies Kubernetes deployments with one-command deploys. Target audience: DevOps engineers and backend developers.`,
+  },
+  {
+    name: 'Mobile Fitness App',
+    prompt: `Create marketing copy for a fitness app called "FitPulse" that uses AI to create personalized workout plans and tracks progress with smart watch integration. Target audience: Busy professionals aged 30-45.`,
+  },
+]
+
+interface MarketingCopy {
+  title: string
+  description: string
+  hero: {
+    headline: string
+    subhead: string
+    primaryCTA: string
+    secondaryCTA: string
+  }
+}
+
+interface GeneratedCopy {
+  model: EvalModel
+  testCase: typeof TEST_CASES[0]
+  copy: MarketingCopy
+  latencyMs: number
+}
+
+interface ELORating {
+  modelId: string
+  modelName: string
+  rating: number
+  wins: number
+  losses: number
+  draws: number
+}
+
+// ELO calculation
+const K_FACTOR = 32
+const INITIAL_ELO = 1500
+
+function calculateEloChange(ratingA: number, ratingB: number, scoreA: number): { deltaA: number; deltaB: number } {
+  const expectedA = 1 / (1 + Math.pow(10, (ratingB - ratingA) / 400))
+  const expectedB = 1 - expectedA
+
+  const deltaA = K_FACTOR * (scoreA - expectedA)
+  const deltaB = K_FACTOR * ((1 - scoreA) - expectedB)
+
+  return { deltaA, deltaB }
+}
+
+// LLM Judge for pairwise comparison
+async function judgePair(
+  copyA: MarketingCopy,
+  copyB: MarketingCopy,
+  testCase: typeof TEST_CASES[0],
+  judgeModel: string
+): Promise<'A' | 'B' | 'TIE'> {
+  const prompt = `You are an expert marketing copywriter and brand strategist. Compare these two marketing copy options for: ${testCase.name}
+
+Context: ${testCase.prompt}
+
+=== OPTION A ===
+Title: ${copyA.title}
+Description: ${copyA.description}
+Hero Headline: ${copyA.hero.headline}
+Hero Subhead: ${copyA.hero.subhead}
+Primary CTA: ${copyA.hero.primaryCTA}
+Secondary CTA: ${copyA.hero.secondaryCTA}
+
+=== OPTION B ===
+Title: ${copyB.title}
+Description: ${copyB.description}
+Hero Headline: ${copyB.hero.headline}
+Hero Subhead: ${copyB.hero.subhead}
+Primary CTA: ${copyB.hero.primaryCTA}
+Secondary CTA: ${copyB.hero.secondaryCTA}
+
+Evaluate based on:
+1. Clarity and impact of messaging
+2. Target audience alignment
+3. Emotional appeal and persuasiveness
+4. CTA effectiveness
+5. Overall brand voice consistency
+
+Which option is better? Answer A, B, or TIE if they're roughly equal.`
+
+  try {
+    const { object } = await generateObject({
+      model: judgeModel,
+      schema: schema({
+        reasoning: 'Brief explanation of your judgment (2-3 sentences)',
+        winner: 'A | B | TIE',
+      }),
+      prompt,
+      temperature: 0.3,
+    })
+
+    const result = object as { reasoning: string; winner: string }
+    const winner = result.winner.toUpperCase().trim()
+
+    if (winner === 'A' || winner === 'B' || winner === 'TIE') {
+      return winner
+    }
+    return 'TIE'
+  } catch (err) {
+    console.error(`   Judge error: ${err}`)
+    return 'TIE'
+  }
+}
+
+// Generate marketing copy for a model
+async function generateCopy(model: EvalModel, testCase: typeof TEST_CASES[0]): Promise<GeneratedCopy> {
+  const start = Date.now()
+
+  const { object } = await generateObject({
+    model: model.id,
+    schema: marketingCopySchema,
+    prompt: testCase.prompt,
+    temperature: 0.7,
+  })
+
+  return {
+    model,
+    testCase,
+    copy: object as MarketingCopy,
+    latencyMs: Date.now() - start,
+  }
+}
+
+// Run pairwise comparisons and calculate ELO
+async function runEloTournament(
+  copies: GeneratedCopy[],
+  judgeModel: string
+): Promise<ELORating[]> {
+  // Initialize ELO ratings
+  const ratings: Map<string, ELORating> = new Map()
+
+  for (const copy of copies) {
+    if (!ratings.has(copy.model.id)) {
+      ratings.set(copy.model.id, {
+        modelId: copy.model.id,
+        modelName: copy.model.name,
+        rating: INITIAL_ELO,
+        wins: 0,
+        losses: 0,
+        draws: 0,
+      })
+    }
+  }
+
+  // Group copies by test case
+  const byTestCase = new Map<string, GeneratedCopy[]>()
+  for (const copy of copies) {
+    const key = copy.testCase.name
+    if (!byTestCase.has(key)) {
+      byTestCase.set(key, [])
+    }
+    byTestCase.get(key)!.push(copy)
+  }
+
+  console.log(`\n⚖️  Running pairwise comparisons with ${JUDGE_MODEL} as judge...\n`)
+
+  let comparisonCount = 0
+  const totalComparisons = Array.from(byTestCase.values()).reduce(
+    (sum, copies) => sum + (copies.length * (copies.length - 1)) / 2,
+    0
+  )
+
+  // Run pairwise comparisons within each test case
+  for (const [testCaseName, testCaseCopies] of byTestCase) {
+    console.log(`   📝 ${testCaseName}:`)
+
+    for (let i = 0; i < testCaseCopies.length; i++) {
+      for (let j = i + 1; j < testCaseCopies.length; j++) {
+        const copyA = testCaseCopies[i]
+        const copyB = testCaseCopies[j]
+
+        comparisonCount++
+        process.stdout.write(`      ${comparisonCount}/${totalComparisons} ${copyA.model.name} vs ${copyB.model.name}... `)
+
+        const winner = await judgePair(copyA.copy, copyB.copy, copyA.testCase, judgeModel)
+
+        const ratingA = ratings.get(copyA.model.id)!
+        const ratingB = ratings.get(copyB.model.id)!
+
+        let scoreA: number
+        if (winner === 'A') {
+          scoreA = 1
+          ratingA.wins++
+          ratingB.losses++
+          console.log(`${copyA.model.name} wins`)
+        } else if (winner === 'B') {
+          scoreA = 0
+          ratingA.losses++
+          ratingB.wins++
+          console.log(`${copyB.model.name} wins`)
+        } else {
+          scoreA = 0.5
+          ratingA.draws++
+          ratingB.draws++
+          console.log(`TIE`)
+        }
+
+        const { deltaA, deltaB } = calculateEloChange(ratingA.rating, ratingB.rating, scoreA)
+        ratingA.rating += deltaA
+        ratingB.rating += deltaB
+      }
+    }
+  }
+
+  // Sort by ELO rating
+  return Array.from(ratings.values()).sort((a, b) => b.rating - a.rating)
+}
+
+// Main
+async function main() {
+  console.log('╔════════════════════════════════════════════════════════════════╗')
+  console.log('║              Marketing Copy Eval (LLM-as-Judge)                ║')
+  console.log('╚════════════════════════════════════════════════════════════════╝')
+  console.log('')
+  console.log(`Judge Model: ${JUDGE_MODEL}`)
+  console.log(`Tiers: ${tiers.join(', ')}`)
+
+  // Get models to test
+  const models = EVAL_MODELS.filter(m => tiers.includes(m.tier))
+  console.log(`Models: ${models.map(m => m.name).join(', ')}`)
+  console.log(`Test Cases: ${TEST_CASES.length}`)
+  console.log('')
+
+  // Generate copy from each model for each test case
+  console.log('🎨 Generating marketing copy...\n')
+
+  const allCopies: GeneratedCopy[] = []
+  const startTime = Date.now()
+
+  for (const testCase of TEST_CASES) {
+    console.log(`   📦 ${testCase.name}:`)
+
+    const jobs = models.map(async model => {
+      try {
+        const copy = await generateCopy(model, testCase)
+        console.log(`      ✓ ${model.name} (${copy.latencyMs}ms)`)
+        return copy
+      } catch (err) {
+        console.log(`      ✗ ${model.name}: ${err}`)
+        return null
+      }
+    })
+
+    const results = await Promise.all(jobs)
+    allCopies.push(...results.filter((r): r is GeneratedCopy => r !== null))
+  }
+
+  const generateTime = Date.now() - startTime
+  console.log(`\n   Generated ${allCopies.length} copies in ${(generateTime / 1000).toFixed(1)}s`)
+
+  // Run ELO tournament
+  const tournamentStart = Date.now()
+  const eloRatings = await runEloTournament(allCopies, JUDGE_MODEL)
+  const tournamentTime = Date.now() - tournamentStart
+
+  // Display results
+  console.log('')
+  console.log('╔════════════════════════════════════════════════════════════════╗')
+  console.log('║                     ELO Rankings                               ║')
+  console.log('╚════════════════════════════════════════════════════════════════╝')
+  console.log('')
+  console.log('   Rank | Model                  | ELO    | W   | L   | D   |')
+  console.log('   -----|------------------------|--------|-----|-----|-----|')
+
+  eloRatings.forEach((rating, idx) => {
+    const rank = `${idx + 1}`.padStart(4)
+    const name = rating.modelName.padEnd(22)
+    const elo = Math.round(rating.rating).toString().padStart(6)
+    const wins = rating.wins.toString().padStart(3)
+    const losses = rating.losses.toString().padStart(3)
+    const draws = rating.draws.toString().padStart(3)
+    console.log(`   ${rank} | ${name} | ${elo} | ${wins} | ${losses} | ${draws} |`)
+  })
+
+  console.log('')
+  console.log(`   Judge: ${JUDGE_MODEL}`)
+  console.log(`   Generation Time: ${(generateTime / 1000).toFixed(1)}s`)
+  console.log(`   Tournament Time: ${(tournamentTime / 1000).toFixed(1)}s`)
+  console.log(`   Total Time: ${((generateTime + tournamentTime) / 1000).toFixed(1)}s`)
+
+  // Show sample outputs from top 3
+  console.log('')
+  console.log('╔════════════════════════════════════════════════════════════════╗')
+  console.log('║                  Sample Outputs (Top 3)                        ║')
+  console.log('╚════════════════════════════════════════════════════════════════╝')
+
+  const top3Models = eloRatings.slice(0, 3).map(r => r.modelId)
+  const sampleTestCase = TEST_CASES[0]
+
+  for (const modelId of top3Models) {
+    const copy = allCopies.find(c => c.model.id === modelId && c.testCase.name === sampleTestCase.name)
+    if (copy) {
+      const rank = eloRatings.findIndex(r => r.modelId === modelId) + 1
+      console.log(`\n   #${rank} ${copy.model.name} (${sampleTestCase.name}):`)
+      console.log(`   ─────────────────────────────────────────`)
+      console.log(`   Title: ${copy.copy.title}`)
+      console.log(`   Description: ${copy.copy.description}`)
+      console.log(`   Headline: ${copy.copy.hero.headline}`)
+      console.log(`   Subhead: ${copy.copy.hero.subhead}`)
+      console.log(`   Primary CTA: [${copy.copy.hero.primaryCTA}]`)
+      console.log(`   Secondary CTA: ${copy.copy.hero.secondaryCTA}`)
+    }
+  }
+
+  console.log('')
+}
+
+main().catch(console.error)