outcome-cli 1.0.0

package/README.md

@@ -0,0 +1,261 @@
+# Outcome
+
+[![License: ISC](https://img.shields.io/badge/License-ISC-blue.svg)](https://opensource.org/licenses/ISC)
+[![Node.js Version](https://img.shields.io/badge/node-%3E%3D18.0.0-brightgreen)](https://nodejs.org/)
+[![TypeScript](https://img.shields.io/badge/TypeScript-5.0+-blue)](https://www.typescriptlang.org/)
+
+An AI-assisted coding tool where you define outcomes (not prompts) and multiple agents race to deliver verified, working code. Pay only when code passes your criteria.
+
+## Table of Contents
+
+- [What is Outcome?](#what-is-outcome)
+- [Key Features](#key-features)
+- [Installation](#installation)
+- [Quick Start](#quick-start)
+- [Usage](#usage)
+- [Architecture](#architecture)
+- [Contributing](#contributing)
+- [License](#license)
+
+## What is Outcome?
+
+Outcome is a developer tool that revolutionizes code generation by focusing on verifiable outcomes rather than AI prompts. Define what success looks like (e.g., "Add JWT authentication with passing tests and <200ms response time"), and multiple AI agents compete to deliver working code that meets your exact criteria. You only pay for code that passes verification.
+
+### Differentiation from Traditional AI Coding Tools
+
+| Feature | Cursor/Windsurf | Outcome |
+|---------|-----------------|---------|
+| Input | Manual prompts | Outcome definitions |
+| Selection | Human judgment | Deterministic verification |
+| Payment | Subscription | Pay-only-on-success |
+| Verification | Manual testing | Automated criteria |
+| Focus | Code generation | Guaranteed delivery |
+
+## Key Features
+
+- **Outcome-Based Development**: Define success criteria upfront, not just prompts
+- **Multi-Agent Competition**: Multiple AI models (GPT-4o, Claude, Gemini) race to deliver
+- **Deterministic Verification**: Automated testing ensures code meets your standards
+- **Pay-on-Success**: Only pay when code passes all verification criteria
+- **Domain-Adaptive Weighted Scoring (DAWS)**: Proprietary multi-metric evaluation system
+- **CLI-First Design**: Simple command-line interface for developers
+- **BYOK Support**: Bring your own API keys for full control
+
+## Installation
+
+### Install from npm (Recommended)
+
+```bash
+npm install -g outcome-cli
+```
+
+This installs the Outcome CLI globally, making the `outcome` command available system-wide.
+
+### Prerequisites
+
+- Node.js 18+
+- At least one API key (OpenAI, Anthropic, or Google Gemini)
+
+### Configuration
+
+After installation, create a `.env` file in your project directory or set environment variables:
+
+```bash
+# Create .env file
+echo "OPENAI_API_KEY=sk-proj-your-key-here" > .env
+echo "ANTHROPIC_API_KEY=sk-ant-api03-your-key-here" >> .env
+echo "GOOGLE_API_KEY=AIzaSy-your-key-here" >> .env
+```
+
+Or set environment variables directly in your shell.
+
+## Quick Start
+
+Get Outcome running and generate your first verified code feature in under 5 minutes.
+
+### 1. Define an Outcome
+
+```bash
+# Define an authentication feature outcome
+outcome define auth-feature \
+  --description "Add JWT user authentication with password hashing" \
+  --criteria tests-pass,lint-clean,builds-successfully \
+  --max-attempts 3
+```
+
+### 2. Run the Competition
+
+```bash
+# Multiple agents compete to deliver
+outcome run auth-feature \
+  --models gpt-4o,claude-sonnet \
+  --live
+```
+
+### 3. Verify Results
+
+```bash
+# Deterministic verification
+outcome verify auth-feature
+
+# Expected output:
+# ✅ Outcome 'auth-feature' PASSED
+# 🔍 Criteria met: tests-pass, lint-clean, builds-successfully
+# 💰 Cost: $2.50 (only if all criteria passed)
+```
+
+## Usage
+
+### CLI Commands
+
+#### Define an Outcome
+
+```bash
+outcome define <name> [options]
+
+Options:
+  --description <text>     Description of the outcome
+  --criteria <list>        Comma-separated verification criteria
+                           Available: tests-pass, lint-clean, builds-successfully,
+                           security-scan, benchmark-passes
+  --max-attempts <number>  Maximum attempts per agent (default: 3)
+  --timeout <ms>           Timeout per attempt (default: 300000)
+  --budget <amount>        Maximum cost per outcome (default: 10.00)
+```
+
+#### Run an Outcome
+
+```bash
+outcome run <name> [options]
+
+Options:
+  --models <list>          Comma-separated models to use
+                           Available: gpt-4o, claude-sonnet, claude-opus,
+                           gemini-pro, gemini-flash
+  --live                   Show real-time progress
+  --parallel <number>      Number of agents to run in parallel (default: 3)
+```
+
+#### Verify an Outcome
+
+```bash
+outcome verify <name>
+
+# Shows detailed verification results including:
+# - Which criteria passed/failed
+# - Test output
+# - Build status
+# - Performance benchmarks
+```
+
+#### List Outcomes
+
+```bash
+outcome list
+
+# Shows all defined outcomes with status
+```
+
+### Example: Add User Registration
+
+```bash
+# Define the outcome
+outcome define user-registration \
+  --description "Implement user registration with email verification" \
+  --criteria tests-pass,lint-clean,builds-successfully,security-scan \
+  --max-attempts 5
+
+# Run with multiple models
+outcome run user-registration \
+  --models gpt-4o,claude-sonnet,gemini-pro \
+  --live
+
+# Verify the result
+outcome verify user-registration
+```
+
+### Verification Criteria
+
+- **tests-pass**: All unit tests pass (Jest, Vitest, etc.)
+- **lint-clean**: Code passes linting (ESLint, Prettier)
+- **builds-successfully**: Project builds without errors
+- **security-scan**: Passes security analysis
+- **benchmark-passes**: Meets performance benchmarks
+
+## Architecture
+
+```
+outcome-cli/
+├── src/
+│   ├── outcomes/          # Outcome definitions and management
+│   ├── eval/              # Core evaluation engine
+│   │   ├── evaluateOutcome.ts    # Binary evaluation logic
+│   │   ├── validators.ts         # Verification functions
+│   │   ├── ai-judge.ts          # LLM-as-judge system
+│   │   └── weighted-scorer.ts   # DAWS scoring system
+│   ├── runtime/           # Multi-model adapters
+│   │   ├── openai-adapter.ts
+│   │   ├── anthropic-adapter.ts
+│   │   └── google-adapter.ts
+│   ├── agents/            # Agent configurations
+│   └── commands/          # CLI command handlers
+│       ├── define.ts
+│       ├── run.ts
+│       └── verify.ts
+```
+
+### Core Components
+
+- **Binary Evaluation Engine**: Deterministic pass/fail verification
+- **AI Judge System**: LLM-based code quality assessment with caching
+- **Validator Framework**: Extensible validation functions
+- **Weighted Scoring System**: Multi-metric performance evaluation
+- **Multi-Model Runtime**: Unified interface across AI providers
+
+## Contributing
+
+We welcome contributions! Areas of focus:
+
+- New verification criteria
+- Additional AI model support
+- Performance optimizations
+- Documentation improvements
+- Test coverage
+
+### Development Setup
+
+```bash
+# Install dependencies
+npm install
+
+# Run tests
+npm test
+
+# Run linting
+npm run lint
+
+# Build the project
+npm run build
+```
+
+### Adding a New Validator
+
+1. Create a validator function in `src/eval/validators.ts`
+2. Export it from the validators module
+3. Add it to the CLI criteria options
+4. Update documentation
+
+## License
+
+ISC License - see [LICENSE](LICENSE) file for details.
+
+## Contact
+
+- **GitHub Issues**: [Report bugs or request features](https://github.com/Radix-Obsidian/Waiesl/issues)
+- **Documentation**: See [docs/](docs/) for detailed guides
+
+---
+
+**Built for developers who demand working code, not just suggestions.**
+
+*Outcome - Pay only for code that works.*

package/package.json

@@ -0,0 +1,95 @@
+{
+  "name": "outcome-cli",
+  "version": "1.0.0",
+  "description": "An AI-assisted coding tool where you define outcomes and pay only for verified code",
+  "main": "dist/index.js",
+  "types": "dist/index.d.ts",
+  "type": "module",
+  "bin": {
+    "outcome": "./node_modules/.bin/tsx ./src/cli/index.ts"
+  },
+  "files": [
+    "src",
+    "README.md",
+    "package.json"
+  ],
+  "repository": {
+    "type": "git",
+    "url": "https://github.com/Radix-Obsidian/Waiesl.git"
+  },
+  "homepage": "https://waiesl.com",
+  "scripts": {
+    "cli": "tsx src/cli/index.ts",
+    "build": "tsc",
+    "dev:worker": "wrangler dev",
+    "test": "vitest run",
+    "test:watch": "vitest",
+    "test:e2e": "playwright test",
+    "test:e2e:ui": "playwright test --ui",
+    "test:e2e:debug": "playwright test --debug",
+    "test:e2e:headed": "playwright test --headed",
+    "test:e2e:report": "playwright show-report",
+    "lvs:test": "tsx src/validation/lvs/test-lvs.ts",
+    "lint": "eslint src --ext .ts",
+    "format": "prettier --write \"src/**/*.ts\"",
+    "typecheck": "tsc --noEmit"
+  },
+  "keywords": [
+    "ai",
+    "coding",
+    "cli",
+    "outcome",
+    "verification",
+    "development",
+    "automation",
+    "testing",
+    "build",
+    "deployment"
+  ],
+  "author": "Radix-Obsidian",
+  "license": "ISC",
+  "engines": {
+    "node": ">=18.0.0"
+  },
+  "devDependencies": {
+    "@cloudflare/workers-types": "^4.20251219.0",
+    "@eslint/js": "^9.39.2",
+    "@playwright/test": "^1.57.0",
+    "@types/node": "^25.0.3",
+    "@types/pdfkit": "^0.17.4",
+    "@typescript-eslint/eslint-plugin": "^8.50.0",
+    "@typescript-eslint/parser": "^8.50.0",
+    "eslint": "^9.39.2",
+    "fast-check": "^4.4.0",
+    "pdfkit": "^0.17.2",
+    "playwright": "^1.57.0",
+    "prettier": "^3.7.4",
+    "tsx": "^4.21.0",
+    "typescript": "^5.9.3",
+    "typescript-eslint": "^8.50.0",
+    "vitest": "^4.0.17",
+    "wrangler": "^4.56.0"
+  },
+  "dependencies": {
+    "@anthropic-ai/sdk": "^0.71.2",
+    "@google/genai": "^1.34.0",
+    "@google/generative-ai": "^0.24.1",
+    "@modelcontextprotocol/server-slack": "^2025.4.25",
+    "@openai/agents": "^0.3.7",
+    "@pkmn/sim": "^0.10.3",
+    "@sendgrid/mail": "^8.1.6",
+    "@supabase/supabase-js": "^2.90.1",
+    "@types/pdf-parse": "^1.1.5",
+    "cheerio": "^1.1.2",
+    "commander": "^14.0.2",
+    "csv-parse": "^6.1.0",
+    "dotenv": "^17.2.3",
+    "express": "^5.2.1",
+    "openai": "^6.14.0",
+    "pdf-parse": "^2.4.5",
+    "puppeteer": "^24.35.0",
+    "stripe": "^17.5.5",
+    "yaml": "^2.8.2",
+    "zod": "^3.25.76"
+  }
+}

package/src/agents/README.md

@@ -0,0 +1,139 @@
+# Agents Module
+
+The Agents module defines AI agent configurations as code. An **Agent** is a configuration-driven AI entity that attempts to achieve outcomes using defined prompts, strategies, and tools.
+
+## Key Concepts
+
+- **AgentConfig**: A complete configuration defining agent behavior
+- **ModelProvider**: The AI provider (Claude or OpenAI)
+- **Tool Access**: List of tools the agent can use
+- **Cost Ceiling**: Maximum token budget for the agent
+
+## Interfaces
+
+### AgentConfig
+
+```typescript
+interface AgentConfig {
+  id: string;                // Unique identifier
+  name: string;              // Human-readable name
+  prompt: string;            // Prompt template
+  strategyDescription: string; // Strategy for transparency
+  toolAccess: string[];      // Allowed tools
+  costCeiling: number;       // Max token budget
+  modelProvider: 'claude' | 'openai'; // AI provider
+  modelId: string;           // Specific model ID
+}
+```
+
+## Usage
+
+### TypeScript
+
+```typescript
+import { AgentConfig, validateAgentConfig, isAgentConfig } from './agent.schema.js';
+
+// Define an agent config
+const myAgent: AgentConfig = {
+  id: 'sales-agent-001',
+  name: 'Sales Qualifier',
+  prompt: 'You are a sales qualification agent...',
+  strategyDescription: 'Qualify leads by verifying company size and buying intent',
+  toolAccess: ['email', 'company_lookup'],
+  costCeiling: 10000,
+  modelProvider: 'claude',
+  modelId: 'claude-3-sonnet-20240229'
+};
+
+// Validate before use
+const result = validateAgentConfig(myAgent);
+if (!result.valid) {
+  console.error('Invalid agent config:', result.errors);
+}
+
+// Type guard usage
+if (isAgentConfig(unknownData)) {
+  // unknownData is now typed as AgentConfig
+}
+```
+
+### YAML Configuration
+
+Agent configurations are stored as YAML files in `/src/agents/`:
+
+```yaml
+id: sales-agent-001
+name: Sales Qualifier
+prompt: |
+  You are a sales qualification agent...
+strategyDescription: Qualify leads by verifying company size and buying intent
+toolAccess:
+  - email
+  - company_lookup
+costCeiling: 10000
+modelProvider: claude
+modelId: claude-3-sonnet-20240229
+```
+
+## Requirements Reference
+
+- **Requirement 2.1**: Agents require prompt, strategy description, tool access list, and cost ceiling
+- **Requirement 2.2**: Agent configurations are validated against schema
+- **Requirement 2.3**: Agents use only configuration values without hard-coded logic
+- **Requirement 2.4**: Agent files are stored as YAML in /src/agents
+
+## Agent Marketplace
+
+The Earnd platform includes 5 pre-built agents powered by OpenAI Agents SDK:
+
+| Agent | Specialty | Model | Best For |
+|-------|-----------|-------|----------|
+| **Alex - General SDR** | Consultative sales | GPT-4o | Broad B2B outreach |
+| **Jordan - SaaS Specialist** | Technical sales | GPT-4o | Software/tech companies |
+| **Morgan - Enterprise Hunter** | Enterprise sales | GPT-4o | Large organizations (200+ employees) |
+| **Taylor - E-commerce Specialist** | Retail/DTC | GPT-4o-mini | Online retailers |
+| **Casey - Rapid Responder** | High-velocity | GPT-4o-mini | Volume-based outreach |
+
+### Loading Agents
+
+```typescript
+import { loadAllAgents, getAgent, getAllAgents } from './registry.js';
+
+// Load all agents from configs directory
+const agents = await loadAllAgents();
+
+// Get specific agent
+const alex = getAgent('agent-001-general-sdr');
+
+// Get all loaded agents
+const allAgents = getAllAgents();
+```
+
+### Running Agents
+
+```typescript
+import { runOpenAIAgent, mockOpenAIAgent } from './adapters/openai.adapter.js';
+
+const result = await runOpenAIAgent(
+  agentConfig,
+  {
+    email: 'prospect@company.com',
+    company: 'Acme Corp',
+    companySize: 150,
+    role: 'VP Sales'
+  },
+  'qualified_sales_interest',
+  1
+);
+
+if (result.success) {
+  // Evaluate result.artifact against outcome
+}
+```
+
+## Files
+
+- `agent.schema.ts` - Type definitions and validation
+- `registry.ts` - Agent loading and management
+- `adapters/openai.adapter.ts` - OpenAI Agents SDK integration
+- `configs/` - Agent configuration files (5 agents)

package/src/agents/adapters/anthropic.adapter.ts

@@ -0,0 +1,166 @@
+/**
+ * Anthropic Claude Adapter
+ * 
+ * Adapts Anthropic's SDK to work with WAI Championship's agent system.
+ * Supports Claude Opus, Sonnet, and Haiku models.
+ * 
+ * @module agents/adapters/anthropic
+ */
+
+import Anthropic from '@anthropic-ai/sdk';
+import type { AgentConfig } from '../agent.schema.js';
+import type { AgentArtifact, ArtifactContent } from '../../eval/evaluateOutcome.js';
+import type { LeadData, AgentRunResult } from './openai.adapter.js';
+
+/**
+ * Creates artifact content based on the outcome type.
+ * Agents should generate real content, not hardcoded mock data.
+ */
+function createArtifactContent(_outcomeId: string, message: string, lead: LeadData): ArtifactContent {
+  // Try to parse the message as JSON first (for structured outcomes)
+  try {
+    const parsed = JSON.parse(message);
+    if (typeof parsed === 'object' && parsed !== null) {
+      // Ensure required fields exist
+      return {
+        message: parsed.message || message,
+        targetEmail: parsed.targetEmail || parsed.email || lead.email,
+        targetCompany: parsed.targetCompany || lead.company,
+        targetCompanySize: parsed.targetCompanySize || parsed.companySize || lead.companySize,
+        targetRole: parsed.targetRole || parsed.role || lead.role,
+        ...parsed
+      };
+    }
+  } catch {
+    // Message is not JSON, continue with text processing
+  }
+
+  // Default for qualified_sales_interest and other outcomes
+  return {
+    message,
+    targetEmail: lead.email,
+    targetCompany: lead.company,
+    targetCompanySize: lead.companySize,
+    targetRole: lead.role,
+  };
+}
+
+/**
+ * Executes a Claude agent configured for lead qualification
+ * 
+ * @param config - WAI agent configuration
+ * @param lead - Lead data to qualify
+ * @param outcomeId - ID of the outcome being attempted
+ * @param attemptNumber - Current attempt number
+ * @returns Result of agent execution with artifact if successful
+ */
+export async function runClaudeAgent(
+  config: AgentConfig,
+  lead: LeadData,
+  outcomeId: string,
+  attemptNumber: number
+): Promise<AgentRunResult> {
+  try {
+    const anthropic = new Anthropic({
+      apiKey: process.env.ANTHROPIC_API_KEY,
+    });
+
+    const input = `
+You are qualifying a sales lead. Here is the prospect information:
+
+Email: ${lead.email}
+Company: ${lead.company} (${lead.companySize} employees)
+Role: ${lead.role}
+${lead.context ? `Additional Context: ${lead.context}` : ''}
+
+Your task: ${config.strategyDescription}
+
+Write a personalized outreach message that demonstrates buying intent and qualifies this lead.
+Your message should be professional, relevant to their role, and demonstrate understanding of their potential needs.
+`;
+
+    const response = await anthropic.messages.create({
+      model: config.modelId,
+      max_tokens: 1024,
+      system: config.prompt,
+      messages: [
+        {
+          role: 'user',
+          content: input,
+        },
+      ],
+    });
+
+    const message = response.content[0].type === 'text' 
+      ? response.content[0].text 
+      : '';
+
+    const artifact: AgentArtifact = {
+      agentId: config.id,
+      outcomeId,
+      attemptNumber,
+      content: createArtifactContent(outcomeId, message, lead),
+      timestamp: new Date().toISOString(),
+    };
+
+    const tokensUsed = response.usage.input_tokens + response.usage.output_tokens;
+
+    return {
+      success: true,
+      message: 'Claude agent completed successfully',
+      tokensUsed,
+      artifact,
+    };
+
+  } catch (error) {
+    const errorMessage = error instanceof Error ? error.message : 'Unknown error';
+    return {
+      success: false,
+      message: `Claude agent execution failed: ${errorMessage}`,
+      tokensUsed: 0,
+      error: errorMessage,
+    };
+  }
+}
+
+/**
+ * Creates a mock Claude agent run for testing
+ * 
+ * @param config - Agent configuration
+ * @param lead - Lead data
+ * @param outcomeId - Outcome ID
+ * @param attemptNumber - Attempt number
+ * @returns Mock result matching Claude output format
+ */
+export function mockClaudeAgent(
+  config: AgentConfig,
+  lead: LeadData,
+  outcomeId: string,
+  attemptNumber: number
+): AgentRunResult {
+  const mockMessage = `Hi ${lead.role} at ${lead.company},
+
+I came across ${lead.company}'s work and was impressed by your approach to ${lead.company.toLowerCase().includes('tech') ? 'technology innovation' : 'business growth'}.
+
+I'd like to explore how our solution could help ${lead.company} achieve your goals more efficiently. Would you be available for a brief 15-minute call next week to discuss potential collaboration opportunities?
+
+Looking forward to connecting about next steps.
+
+Best regards,
+${config.name}`;
+
+  const artifact: AgentArtifact = {
+    agentId: config.id,
+    outcomeId,
+    attemptNumber,
+    content: createArtifactContent(outcomeId, mockMessage, lead),
+    timestamp: new Date().toISOString(),
+  };
+
+  return {
+    success: true,
+    message: 'Mock Claude agent completed',
+    tokensUsed: 450,
+    artifact,
+  };
+}