agent-os-kernel 1.1.0__py3-none-any.whl → 1.3.0__py3-none-any.whl

modules/cmvk/docs/traceability.md

@@ -0,0 +1,150 @@
+# Feature 3: The Witness (Traceability)
+
+## Overview
+
+Feature 3 implements comprehensive traceability for the Cross-Model Verification Kernel. This feature logs the entire debate between the Generator and Verifier, creating a clean JSON artifact suitable for research papers and supplementary materials.
+
+## Key Components
+
+### 1. TraceLogger (`src/core/trace_logger.py`)
+
+The `TraceLogger` class serializes the complete `NodeState` (including history, forbidden strategies, and feedback) into structured JSON files.
+
+**Features:**
+- Automatic directory creation (`logs/traces/` by default)
+- JSON serialization of dataclasses using `asdict()`
+- Metadata generation (timestamp, attempt count, final status)
+- Clean, readable output with proper encoding
+
+**Usage:**
+```python
+from src.core.trace_logger import TraceLogger
+from src.core.types import NodeState
+
+logger = TraceLogger(log_dir="logs/traces")
+state = NodeState(input_query="Your problem here")
+# ... populate state with history ...
+filepath = logger.save_trace("experiment_01", state)
+```
+
+### 2. SimpleVerificationKernel (`src/simple_kernel.py`)
+
+A simplified kernel implementation designed specifically for research experiments. It provides:
+- Direct `solve()` method interface
+- Integration with `NodeState` tracking
+- Automatic trace logging at completion
+- Strategy detection and banning (Feature 2)
+
+**Usage:**
+```python
+from src.simple_kernel import SimpleVerificationKernel
+
+kernel = SimpleVerificationKernel(max_retries=5)
+solution = kernel.solve(
+    query="Write a function to merge sorted arrays",
+    run_id="experiment_001"
+)
+# Trace is automatically saved to logs/traces/experiment_001_<timestamp>.json
+```
+
+### 3. Paper Data Generator (`experiments/paper_data_generator.py`)
+
+An experiment runner that compares:
+1. **Baseline**: Single agent (OpenAI) without verification
+2. **CMVK**: Full Cross-Model Verification Kernel with traceability
+
+**Running the Experiment:**
+```bash
+cd /path/to/cross-model-verification-kernel
+python experiments/paper_data_generator.py
+```
+
+**Output:**
+- Baseline solutions: `logs/baseline_*.py`
+- CMVK traces: `logs/traces/cmvk_*.json`
+
+### 4. Enhanced VerificationKernel
+
+The main `VerificationKernel` (`src/core/kernel.py`) now supports optional trace logging:
+
+```python
+from src.core.kernel import VerificationKernel
+from src.agents.generator_openai import OpenAIGenerator
+from src.agents.verifier_gemini import GeminiVerifier
+
+kernel = VerificationKernel(
+    generator=OpenAIGenerator(),
+    verifier=GeminiVerifier(),
+    enable_trace_logging=True  # Enable traceability
+)
+```
+
+## Trace File Format
+
+Each trace file contains:
+
+```json
+{
+  "input_query": "The original problem statement",
+  "current_code": "The final solution code (if found)",
+  "status": "verified|rejected|pending",
+  "history": [
+    {
+      "step_id": 1,
+      "code_generated": "First attempt code",
+      "verifier_feedback": "Feedback from verifier",
+      "status": "success|failed",
+      "strategy_used": "recursive|iterative|numpy|..."
+    }
+  ],
+  "forbidden_strategies": ["strategies", "that", "failed"],
+  "meta": {
+    "timestamp": "20260121-182813",
+    "total_attempts": 3,
+    "final_status": "solved|failed"
+  }
+}
+```
+
+## Benefits for Research
+
+1. **Reproducibility**: Complete record of every generation-verification iteration
+2. **Analysis**: Track which strategies were tried and why they failed
+3. **Visualization**: Data ready for charts and graphs
+4. **Supplementary Material**: Professional JSON artifacts for paper submission
+5. **Debugging**: Full audit trail for understanding system behavior
+
+## Testing
+
+Run the test suite:
+```bash
+python tests/test_trace_logger.py
+```
+
+Tests cover:
+- Basic trace logging functionality
+- Failed state handling
+- JSON structure validation
+- Metadata generation
+
+## Integration with Features 1 & 2
+
+- **Feature 1 (Generator ↔ Verifier Loop)**: Traces capture each iteration
+- **Feature 2 (Lateral Thinking)**: Forbidden strategies are logged in traces
+- **Feature 3 (Traceability)**: This feature - complete audit trail
+
+## Next Steps
+
+As mentioned in the problem statement, you can now:
+
+1. **Visualize the Traces**: Create a script to replay the debate
+2. **Scale the Experiment**: Load HumanEval dataset (164 problems)
+3. **Draft the Paper**: Use traces to write the Methodology section
+
+## File Locations
+
+- **TraceLogger**: `src/core/trace_logger.py`
+- **SimpleKernel**: `src/simple_kernel.py`
+- **Experiment**: `experiments/paper_data_generator.py`
+- **Tests**: `tests/test_trace_logger.py`
+- **Traces**: `logs/traces/` (gitignored)

modules/cmvk/examples/basic_example.py

@@ -0,0 +1,62 @@
+"""
+Example: Basic Adversarial Verification
+
+Demonstrates the basic usage of the Cross-Model Verification Kernel
+with GPT-4o as Generator and Gemini 1.5 Pro as Verifier.
+"""
+
+from cross_model_verification_kernel.generator import GeneratorConfig
+from cross_model_verification_kernel.kernel import VerificationKernel
+from cross_model_verification_kernel.models import ModelProvider
+from cross_model_verification_kernel.verifier import VerifierConfig
+
+
+def main() -> None:
+    """Run basic adversarial verification example"""
+
+    print("Cross-Model Verification Kernel - Basic Example")
+    print("=" * 80)
+    print("\nInitializing adversarial architecture...")
+    print("- Generator: GPT-4o")
+    print("- Verifier: Gemini 1.5 Pro (hostile code review mode)\n")
+
+    # Configure Generator with GPT-4o
+    generator_config = GeneratorConfig(model=ModelProvider.GPT4O, temperature=0.7, max_tokens=2000)
+
+    # Configure Verifier with Gemini 1.5 Pro (different model!)
+    verifier_config = VerifierConfig(
+        model=ModelProvider.GEMINI_15_PRO, temperature=0.2, adversarial_mode=True
+    )
+
+    # Create the Verification Kernel
+    # Note: This will enforce that generator and verifier use different models
+    kernel = VerificationKernel(generator_config=generator_config, verifier_config=verifier_config)
+
+    # Task: Generate a Fibonacci function
+    task = "Create a function to calculate the nth Fibonacci number"
+
+    print(f"Task: {task}\n")
+    print("Executing adversarial verification pipeline...")
+    print("1. Generating code with GPT-4o...")
+    print("2. Hostile code review with Gemini 1.5 Pro...")
+    print("3. Calculating blind spot probability reduction...\n")
+
+    # Execute verification
+    result = kernel.verify_task(task_description=task, language="python")
+
+    # Print detailed results
+    kernel.print_verification_summary(result)
+
+    # Print statistics
+    print("\n\nKernel Statistics:")
+    print("-" * 80)
+    stats = kernel.get_statistics()
+    print(f"Total Verifications: {stats['total_verifications']}")
+    print(f"Success Rate: {stats['success_rate']:.1%}")
+    print(f"Average Risk Reduction: {stats['average_risk_reduction_factor']:.2f}x")
+    print(f"Model Diversity Enforced: {stats['model_diversity']['are_different']}")
+    print("-" * 80)
+
+
+if __name__ == "__main__":
+    main()

modules/cmvk/examples/demo_complete_pipeline.py

@@ -0,0 +1,209 @@
+#!/usr/bin/env python3
+"""
+Demo: Complete Workflow for Cross-Model Verification Kernel
+
+This demo shows the full pipeline from data loading to visualization:
+1. Load problems from HumanEval dataset
+2. Run experiments (simulated - no real API calls for demo)
+3. Visualize the traces
+
+Run this demo to see all the new features in action.
+"""
+import os
+import sys
+
+sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+from cross_model_verification_kernel.core.trace_logger import TraceLogger
+from cross_model_verification_kernel.core.types import ExecutionTrace, NodeState
+from cross_model_verification_kernel.datasets.humaneval_loader import HumanEvalLoader
+
+
+def print_header(title):
+    """Print a formatted header."""
+    print("\n" + "=" * 80)
+    print(f"  {title}")
+    print("=" * 80 + "\n")
+
+
+def demo_humaneval_loader():
+    """Demonstrate the HumanEval loader."""
+    print_header("STEP 1: HumanEval Dataset Loader")
+
+    print("Loading HumanEval sample dataset...")
+    loader = HumanEvalLoader()
+
+    print(f"✅ Loaded {len(loader)} problems")
+    print("\nSample problems:")
+
+    for i, problem in enumerate(loader.get_problem_subset(0, 3), 1):
+        print(f"  {i}. {problem['task_id']} - {problem['entry_point']}")
+
+    print("\n📝 Formatting problems for kernel...")
+    formatted = loader.format_all_for_kernel(start=0, count=3)
+
+    print(f"✅ Formatted {len(formatted)} problems")
+    print(f"\nExample formatted problem ID: {formatted[0]['id']}")
+    print(f"Query preview: {formatted[0]['query'][:100]}...")
+
+    return formatted
+
+
+def demo_trace_generation(problems):
+    """Demonstrate trace generation (simulated)."""
+    print_header("STEP 2: Generating Traces (Simulated)")
+
+    print("In a real experiment, this would:")
+    print("  1. Run baseline (single GPT-4o) on each problem")
+    print("  2. Run CMVK (GPT-4o + Gemini) on each problem")
+    print("  3. Save traces to logs/traces/")
+
+    print("\nFor demo purposes, generating a sample trace...")
+
+    # Create a simulated trace
+    state = NodeState(
+        input_query=problems[0]["query"][:100] + "...",
+        current_code="def solution():\n    # Final verified solution\n    pass",
+        status="verified",
+    )
+
+    # Add some history
+    trace1 = ExecutionTrace(
+        step_id=1,
+        code_generated="def solution():\n    # First attempt\n    return []",
+        verifier_feedback="OBJECTION! Missing edge case handling for empty input",
+        status="failed",
+        strategy_used="naive",
+    )
+
+    trace2 = ExecutionTrace(
+        step_id=2,
+        code_generated="def solution():\n    # Second attempt with edge cases\n    if not input:\n        return default\n    return process(input)",
+        verifier_feedback="PASS: Correctly handles edge cases",
+        status="success",
+        strategy_used="defensive",
+    )
+
+    state.history.extend([trace1, trace2])
+    state.forbidden_strategies.append("naive")
+
+    # Save trace
+    logger = TraceLogger()
+    trace_file = logger.save_trace("demo_experiment", state)
+
+    print("\n✅ Sample trace generated and saved")
+
+    return trace_file
+
+
+def demo_visualizer(trace_file):
+    """Demonstrate the trace visualizer."""
+    print_header("STEP 3: Visualizing Traces")
+
+    print("The visualizer replays traces as adversarial debates:")
+    print(f"\nCommand: python -m src.tools.visualizer {trace_file}\n")
+
+    print("Output shows:")
+    print("  🎭 Problem statement")
+    print("  🔄 Each attempt with:")
+    print("     - Builder (GPT-4o) generating solution")
+    print("     - Prosecutor (Gemini) finding flaws")
+    print("     - Kernel making decisions")
+    print("  🏁 Final result and statistics")
+
+    print("\n💡 Try these commands:")
+    print("  python -m src.tools.visualizer --list")
+    print("  python -m src.tools.visualizer --latest")
+    print(f"  python -m src.tools.visualizer {trace_file} --speed 0")
+
+
+def demo_paper_generator():
+    """Demonstrate the updated paper data generator."""
+    print_header("STEP 4: Scaling Experiments")
+
+    print("The paper_data_generator.py now supports HumanEval:\n")
+
+    print("📊 Run experiments at different scales:")
+    print("\n  # Small scale (5 problems)")
+    print("  python experiments/paper_data_generator.py --humaneval --count 5")
+
+    print("\n  # Medium scale (50 problems - suitable for paper)")
+    print("  python experiments/paper_data_generator.py --humaneval --count 50")
+
+    print("\n  # Full dataset (all problems)")
+    print("  python experiments/paper_data_generator.py --humaneval")
+
+    print("\n  # Legacy mode (original 2 problems)")
+    print("  python experiments/paper_data_generator.py --legacy")
+
+
+def demo_paper_structure():
+    """Show the paper structure."""
+    print_header("STEP 5: Research Paper Draft")
+
+    print("PAPER.md contains the complete research narrative:\n")
+
+    sections = [
+        ("Abstract", "Overview of CMVK and key results"),
+        ("Introduction", "Problem statement and motivation"),
+        ("Methodology", "Architecture and adversarial design"),
+        ("Experiments", "HumanEval benchmark and results"),
+        ("Tools", "Reproducibility and visualization"),
+        ("Discussion", "Why multi-model verification works"),
+        ("Conclusion", "Key insights and future work"),
+    ]
+
+    for section, description in sections:
+        print(f"  📄 {section:20} - {description}")
+
+    print("\n💡 Next steps for publication:")
+    print("  1. Run full HumanEval experiments (n=50+)")
+    print("  2. Fill in results tables in Section 3.3")
+    print("  3. Add system prompts to Appendix A")
+    print("  4. Include example traces in Appendix B")
+
+
+def main():
+    """Run the complete demo."""
+    print("\n" + "=" * 80)
+    print("  🎭 CROSS-MODEL VERIFICATION KERNEL (CMVK) - DEMO  🎭")
+    print("  From Engineering to Science: The Complete Pipeline")
+    print("=" * 80)
+
+    # Step 1: Load data
+    problems = demo_humaneval_loader()
+
+    # Step 2: Generate traces
+    trace_file = demo_trace_generation(problems)
+
+    # Step 3: Visualize
+    demo_visualizer(trace_file)
+
+    # Step 4: Scale experiments
+    demo_paper_generator()
+
+    # Step 5: Paper structure
+    demo_paper_structure()
+
+    # Summary
+    print_header("🎉 DEMO COMPLETE")
+
+    print("You now have a complete research pipeline:")
+    print("\n✅ Data ingestion (HumanEval)")
+    print("✅ Experiment automation (paper_data_generator.py)")
+    print("✅ Traceability (TraceLogger)")
+    print("✅ Visualization (visualizer.py)")
+    print("✅ Research narrative (PAPER.md)")
+
+    print("\n🚀 Ready to publish! Next immediate actions:")
+    print("\n1. Set up API keys for OpenAI and Gemini")
+    print("2. Run: python experiments/paper_data_generator.py --humaneval --count 5")
+    print("3. Review traces: python -m src.tools.visualizer --latest")
+    print("4. Scale to 50+ problems for statistical significance")
+
+    print("\n📖 Read PAPER.md for complete methodology and context")
+    print("=" * 80 + "\n")
+
+
+if __name__ == "__main__":
+    main()

modules/cmvk/examples/demo_innovation_layer.py

@@ -0,0 +1,197 @@
+#!/usr/bin/env python3
+"""
+Integration demo: Shows all Phase 2 features working together.
+
+This script demonstrates:
+1. Prosecutor Mode - Generating hostile tests
+2. Lateral Thinking - Strategy divergence after failures
+3. Witness - Exporting conversation traces
+"""
+
+from pathlib import Path
+
+from cross_model_verification_kernel.core.graph_memory import GraphMemory
+
+
+def demo_prosecutor_mode() -> None:
+    """Demo Feature 1: Prosecutor Mode."""
+    print("\n" + "=" * 60)
+    print("FEATURE 1: PROSECUTOR MODE DEMO")
+    print("=" * 60)
+
+    from cross_model_verification_kernel.agents.verifier_gemini import GeminiVerifier
+
+    verifier = GeminiVerifier(enable_prosecutor_mode=True)
+
+    # Test function extraction
+    code = """
+def divide(a, b):
+    return a / b
+"""
+
+    func_name = verifier._extract_function_name(code)
+    print(f"✓ Extracted function name: {func_name}")
+
+    # Demo hostile test generation
+    context = {"task": "Write a division function", "solution": code}
+
+    hostile_tests = verifier._generate_hostile_tests(context, "No zero check")
+    print(f"✓ Generated {len(hostile_tests)} hostile tests")
+    print(f"  Example: {hostile_tests[0][:60]}...")
+
+    print("✓ Prosecutor Mode validated")
+
+
+def demo_lateral_thinking() -> None:
+    """Demo Feature 2: Lateral Thinking."""
+    print("\n" + "=" * 60)
+    print("FEATURE 2: LATERAL THINKING DEMO")
+    print("=" * 60)
+
+    graph = GraphMemory()
+    task = "Write a fibonacci function"
+
+    # Simulate failures
+    recursive_solution = """
+def fib(n):
+    if n <= 1:
+        return n
+    return fib(n-1) + fib(n-2)
+"""
+
+    # First failure
+    print("Loop 1: Trying recursive approach...")
+    approach = graph.detect_approach(recursive_solution)
+    print(f"  Detected approach: {approach}")
+    graph.record_approach_failure(recursive_solution, task)
+    print("  Recorded failure")
+
+    # Second failure
+    print("Loop 2: Trying recursive approach again...")
+    graph.record_approach_failure(recursive_solution, task)
+    print("  Recorded second failure")
+
+    # Check branching
+    should_branch = graph.should_branch(recursive_solution, task)
+    print(f"  Should branch? {should_branch}")
+
+    if should_branch:
+        forbidden = graph.get_forbidden_approaches(task)
+        print(f"  Forbidden approaches: {forbidden}")
+        print("✓ System will force different approach on next loop")
+
+    print("✓ Lateral Thinking validated")
+
+
+def demo_witness() -> None:
+    """Demo Feature 3: Witness."""
+    print("\n" + "=" * 60)
+    print("FEATURE 3: WITNESS (TRACEABILITY) DEMO")
+    print("=" * 60)
+
+    graph = GraphMemory()
+
+    # Simulate conversation
+    graph.add_conversation_entry({"type": "task_start", "task": "Write fibonacci function"})
+
+    graph.add_conversation_entry({"type": "generation", "loop": 1, "approach": "recursive"})
+
+    graph.add_conversation_entry(
+        {
+            "type": "verification",
+            "loop": 1,
+            "outcome": "fail",
+            "critical_issues": ["Missing negative check"],
+        }
+    )
+
+    graph.add_conversation_entry({"type": "generation", "loop": 2, "approach": "iterative"})
+
+    graph.add_conversation_entry(
+        {"type": "verification", "loop": 2, "outcome": "pass", "confidence": 0.95}
+    )
+
+    trace = graph.get_conversation_trace()
+    print(f"✓ Created conversation trace with {len(trace)} entries")
+
+    # Export to temp file
+    import tempfile
+
+    with tempfile.NamedTemporaryFile(mode="w", suffix=".json", delete=False) as f:
+        temp_path = f.name
+
+    graph.export_conversation_trace(temp_path)
+    print(f"✓ Exported trace to: {temp_path}")
+
+    # Read and show snippet
+    import json
+
+    with open(temp_path) as f:
+        data = json.load(f)
+
+    print(f"  Trace entries: {len(data['trace'])}")
+    print(f"  First entry type: {data['trace'][0]['type']}")
+    print(f"  Last entry type: {data['trace'][-1]['type']}")
+
+    print("✓ Witness validated")
+
+    # Cleanup
+    Path(temp_path).unlink()
+
+
+def demo_integration() -> None:
+    """Demo all features working together."""
+    print("\n" + "=" * 60)
+    print("INTEGRATION: ALL FEATURES TOGETHER")
+    print("=" * 60)
+
+    from cross_model_verification_kernel import GeminiVerifier, OpenAIGenerator, VerificationKernel
+
+    try:
+        # Note: This requires API keys
+        generator = OpenAIGenerator()
+        verifier = GeminiVerifier(enable_prosecutor_mode=True)
+        kernel = VerificationKernel(generator, verifier)
+
+        print("✓ Initialized kernel with:")
+        print("  - Prosecutor Mode enabled")
+        print("  - Lateral Thinking active")
+        print("  - Witness trace recording")
+
+        # Get stats
+        stats = kernel.get_graph_stats()
+        print("\n✓ Graph stats:")
+        for key, value in stats.items():
+            print(f"  {key}: {value}")
+
+        print("\n✓ All features integrated and ready")
+
+    except Exception as e:
+        print(f"✓ Integration setup complete (API not available: {e})")
+        print("  Run with valid API keys for full demo")
+
+
+def main() -> None:
+    """Run all demos."""
+    print("\n" + "=" * 70)
+    print("CROSS-MODEL VERIFICATION KERNEL - INNOVATION LAYER DEMO")
+    print("=" * 70)
+
+    # Run individual feature demos
+    demo_prosecutor_mode()
+    demo_lateral_thinking()
+    demo_witness()
+    demo_integration()
+
+    print("\n" + "=" * 70)
+    print("ALL FEATURES VALIDATED ✓")
+    print("=" * 70)
+    print("\nThe Innovation Layer is ready for:")
+    print("  1. Research experiments")
+    print("  2. Paper writing")
+    print("  3. Production deployment")
+    print()
+
+
+if __name__ == "__main__":
+    main()