connectonion 0.6.2__py3-none-any.whl → 0.6.3__py3-none-any.whl

connectonion/docs/design-decisions/016-why-no-zero-knowledge-proofs.md

@@ -0,0 +1,358 @@
+# Design Decision: Why ConnectOnion Doesn't Use Zero-Knowledge Proofs
+
+*Date: 2025-11-18*
+*Status: Decided - Not Using (For Now)*
+*Decision: Use simpler trust mechanisms instead of Zero-Knowledge Proofs*
+
+## The Question
+
+As we designed the multi-agent trust system, we explored Zero-Knowledge Proofs (ZKPs): Could agents cryptographically prove their capabilities, reputation, or correct execution without revealing their secrets?
+
+The idea was compelling. But after deep analysis, we decided against it.
+
+## What Are Zero-Knowledge Proofs?
+
+ZKPs let you prove a statement is true without revealing why it's true.
+
+**Traditional proof:**
+```
+"I have 1000 successful task completions"
+→ Shows all 1000 task records
+→ Verifier sees everything
+```
+
+**Zero-knowledge proof:**
+```
+"I have 1000 successful task completions"
+→ Shows cryptographic proof (200 bytes)
+→ Verifier confirms it's true
+→ Verifier learns nothing about the actual tasks
+```
+
+**The magic:** Mathematical certainty without information leakage.
+
+## Why We Considered ZKPs
+
+### Use Case 1: Privacy-Preserving Reputation
+```python
+# Agent B proves reputation without revealing clients
+agent.prove_reputation(min_tasks=1000, min_success_rate=0.95)
+# Verifier confirms: "Yes, they have good reputation"
+# Verifier doesn't learn: which clients, what tasks, when
+```
+
+### Use Case 2: Verifiable Computation
+```python
+# Agent B proves correct execution without revealing inputs
+result = agent.process_data(private_dataset)
+proof = agent.generate_execution_proof()
+# Verifier confirms: "Yes, computation was correct"
+# Verifier doesn't learn: what the dataset was
+```
+
+### Use Case 3: Capability Proving
+```python
+# Agent proves it has specific tools without revealing implementation
+agent.prove_capability("secure_data_processing")
+# Verifier confirms: "Yes, they can do it"
+# Verifier doesn't learn: how it's implemented
+```
+
+This aligns with our philosophy: **"Behavior over identity"** - prove trustworthiness through cryptographic evidence of past behavior.
+
+## Why We Decided Against ZKPs
+
+### Problem 1: AI Agents Are Non-Deterministic
+
+ZKPs work brilliantly for deterministic computations:
+
+```python
+# Deterministic - ZKP works perfectly
+Input: [1, 2, 3, 4, 5]
+Function: sum()
+Output: 15
+Every execution: identical
+```
+
+But AI agents are fundamentally different:
+
+```python
+# Non-deterministic - ZKP struggles
+Input: "Analyze this data"
+Agent execution:
+  Run 1: LLM → Tool A → Tool B → Result
+  Run 2: LLM → Tool C → Tool A → Tool B → Result
+  Run 3: LLM → Tool A → Result
+Every execution: different path
+```
+
+**The question:** What does it mean to "prove correct execution" when there's no single correct path?
+
+### Problem 2: Cannot Prevent Model Switching
+
+The most common "cheat" in agent systems: advertising GPT-4, delivering GPT-3.5.
+
+**What ZKP can prove:**
+- "I processed the input and produced this output"
+- "I called N tools during execution"
+- "I completed the task"
+
+**What ZKP cannot prove:**
+- Which model was actually used (GPT-4 vs GPT-3.5)
+- What system prompt was used
+- What parameters were configured
+- Whether quality degraded between demo and production
+
+The LLM API call is a black box. The proof can only cover what happens before and after the API call, not what happens inside it.
+
+### Problem 3: Cannot Verify Prompts
+
+**Promised configuration:**
+```python
+system_prompt = """
+You are an expert data analyst.
+Use advanced statistical methods.
+Provide detailed explanations.
+Always cite sources.
+"""
+```
+
+**Actual configuration:**
+```python
+system_prompt = "Analyze quickly."
+```
+
+ZKP could prove "I used some prompt," but not "I used this specific prompt." The agent's internal state is opaque.
+
+### Problem 4: Violates "Keep Simple Things Simple"
+
+**Complexity cost:**
+- ZKP libraries: Complex dependencies (bellman, arkworks, circom)
+- Circuit design: Requires cryptography expertise
+- Proof generation: 100ms - 10s per proof
+- Developer experience: Steep learning curve
+- Debugging: "Why did my proof fail?" is much harder than "Why did my function fail?"
+
+**For what benefit?**
+- Privacy: Most agent interactions don't require cryptographic privacy
+- Trust: Behavior-based reputation is simpler and more intuitive
+- Verification: Random sampling catches cheaters at 1/100th the complexity
+
+This is classic over-engineering.
+
+## What We Use Instead
+
+### Solution 1: Behavior-Based Reputation (Primary)
+
+```python
+class AgentReputation:
+    def __init__(self):
+        self.completed_tasks = 0
+        self.quality_scores = []
+        self.user_ratings = []
+
+    def record_task(self, success: bool, quality: float):
+        self.completed_tasks += 1
+        self.quality_scores.append(quality)
+
+    def get_reputation(self):
+        return {
+            "total_tasks": self.completed_tasks,
+            "avg_quality": mean(self.quality_scores),
+            "success_rate": sum(self.quality_scores) / len(self.quality_scores)
+        }
+```
+
+**Why this works:**
+- Simple to implement and understand
+- Directly measures what users care about (quality)
+- Can't be gamed without actually delivering quality
+- Aligns with "behavior over identity"
+
+### Solution 2: Random Sampling + Economic Incentives
+
+```python
+def verify_quality(task_result, agent):
+    # 10% random verification
+    if random.random() < 0.1:
+        expected = rerun_with_reference_implementation(task)
+        actual = task_result
+        similarity = compare(expected, actual)
+
+        if similarity < 0.8:
+            # Penalize severely
+            agent.reputation -= 100
+            agent.deposit -= 1000
+            agent.suspend(days=30)
+```
+
+**Economic calculation:**
+```
+Cheat profit: Save $0.01/task × 1000 tasks = $10
+Cheat risk: 10% detection × $1000 penalty = $100 expected loss
+Rational choice: Don't cheat
+```
+
+### Solution 3: Trust Through Transparency (Not Cryptography)
+
+```python
+class AgentCommitment:
+    """Agent publicly commits to configuration"""
+
+    def __init__(self):
+        self.model = "gpt-4"
+        self.min_quality_threshold = 0.9
+        self.system_prompt_hash = hash(system_prompt)
+
+    def sign_commitment(self):
+        # Sign with private key
+        return sign(
+            model=self.model,
+            quality=self.min_quality_threshold,
+            prompt_hash=self.system_prompt_hash,
+            private_key=self.private_key
+        )
+
+    def verify_commitment(self, task_id):
+        # Selected tasks: reveal actual configuration
+        # Platform compares to signed commitment
+        pass
+```
+
+**This provides:**
+- Accountability without cryptographic complexity
+- Selective disclosure (only when challenged)
+- Simple signature verification (not ZKP)
+
+### Solution 4: Trust Levels (Already Implemented)
+
+We already have a working trust system:
+
+```python
+# Development - trust everything
+agent = Agent("worker", trust="open")
+
+# Production - verify behavior
+agent = Agent("worker", trust="careful")
+
+# High-stakes - maximum verification
+agent = Agent("worker", trust="strict")
+```
+
+This is pragmatic, understandable, and sufficient for 99% of use cases.
+
+## When We Might Reconsider ZKPs
+
+We'll revisit ZKPs if we face these specific scenarios:
+
+### Scenario 1: Enterprise Privacy Requirements
+```
+Client: "We need cryptographic proof that agents processed our data correctly,
+         but we cannot reveal the data even to verifiers."
+Current solution: Trust + audit logs
+ZKP solution: Cryptographic proof without data disclosure
+```
+
+### Scenario 2: Competitive Agent Marketplace
+```
+Agent providers: "We want to prove our capabilities without revealing our
+                  implementation details to competitors."
+Current solution: Behavioral reputation
+ZKP solution: Capability proofs without code disclosure
+```
+
+### Scenario 3: Regulatory Compliance
+```
+Regulator: "You must prove AI agents executed correctly,
+            with mathematical certainty, while preserving user privacy."
+Current solution: Audit logs + sampling
+ZKP solution: Cryptographic execution proofs
+```
+
+### Scenario 4: Verifiable Deterministic Components
+
+If we add deterministic data processing that genuinely benefits from ZKP:
+
+```python
+# Example: Privacy-preserving data aggregation
+def aggregate_sensitive_data(datasets):
+    # Prove: "Total count = 10,000"
+    # Without revealing individual dataset sizes
+    pass
+```
+
+## The Pragmatic Path
+
+**Phase 1: Now (Behavior + Economics)**
+- Reputation system based on actual task quality
+- Random verification with economic penalties
+- Public commitments with signature verification
+- Trust levels (open/careful/strict)
+
+**Phase 2: 6-12 Months (If Demand Emerges)**
+- Add optional ZKP for specific deterministic components
+- Keep it opt-in, not required
+- Use existing libraries (don't build crypto from scratch)
+
+**Phase 3: 12+ Months (If Critical Need)**
+- Full ZKP support for privacy-preserving verification
+- Only if: Real customer demand + regulatory requirement
+
+## Lessons Learned
+
+### 1. Fancy Technology ≠ Better Solution
+ZKPs are mathematically beautiful. They're also overkill for our problem.
+
+### 2. Match Tool to Problem
+- ZKP is perfect for: Deterministic computation + privacy requirement
+- ZKP is wrong for: Non-deterministic AI agents + no privacy requirement
+
+### 3. Economics > Cryptography (For Trust)
+Making cheating unprofitable is simpler and more effective than making cheating cryptographically impossible.
+
+### 4. Behavior > Proof
+Users don't care about cryptographic proofs. They care about: "Does this agent deliver good results?" Behavioral reputation answers that directly.
+
+### 5. Simplicity Wins
+A simple system people actually use beats a sophisticated system nobody understands.
+
+## The ConnectOnion Philosophy
+
+This decision reflects our core principles:
+
+**"Keep simple things simple"**
+- Trust through behavior: Simple
+- Trust through ZKP circuits: Complex
+
+**"Make complicated things possible"**
+- We can add ZKP later if truly needed
+- Architecture doesn't prevent it
+
+**"Behavior over identity"**
+- Behavioral reputation directly implements this
+- ZKP would add indirection without benefit
+
+## Conclusion
+
+Zero-Knowledge Proofs are powerful cryptographic tools. For privacy-preserving verification of deterministic computation, they're unmatched.
+
+But ConnectOnion agents are:
+- Non-deterministic (LLM-based)
+- Primarily concerned with quality (not privacy)
+- Used by developers who prefer simplicity
+
+For these requirements, behavioral reputation + economic incentives + random verification is the right solution.
+
+We're not saying "never." We're saying "not now, and not without clear need."
+
+**When cryptography becomes necessary, we'll add it. Until then, we'll keep it simple.**
+
+---
+
+*"The best solution is the simplest one that solves the actual problem, not the most sophisticated one that solves a theoretical problem."*
+
+## Further Reading
+
+- [Trust System Design](./003-choosing-trust-keyword.md)
+- [Agent Network Protocol](./004-designing-agent-network-protocol.md)
+- [Behavioral Trust in Action](../trust.md)

connectonion/docs/design-decisions/017-session-logging-and-eval-format.md

@@ -0,0 +1,120 @@
+# Session Logging and Eval Format
+
+## Problem
+
+When developing AI agents:
+1. Agent runs are slow (LLM calls take time)
+2. We run the same prompt many times during development
+3. We need to compare: "after changing this prompt, what's different?"
+4. We need regression testing: "does new prompt still work for old cases?"
+
+## Requirements
+
+1. **Log** - Save full session (inputs, tool calls, results, messages)
+2. **Replay** - Load saved messages to restore context
+3. **Eval** - Mark expected behavior, LLM judges if it matches
+4. **Edit** - Developers can view/edit in VS Code
+
+## Format Decision: YAML with JSON String for Messages
+
+```yaml
+name: gmail_agent
+timestamp: 2024-11-27 11:39:58
+
+turns:
+  - input: "check my emails"
+    model: "gemini-2.5-pro"
+    duration_ms: 11200
+    tokens: 1234
+    cost: 0.01
+    tools_called: [get_emails]
+    result: "You have 3 emails"
+    messages: '[{"role":"system","content":"..."},{"role":"user","content":"check my emails"},{"role":"assistant","content":"You have 3 emails"}]'
+    eval:
+      expect_tools: [get_emails]
+      expect_result: "shows email list"
+
+  - input: "reply to first saying thanks"
+    model: "gemini-2.5-pro"
+    duration_ms: 8500
+    tokens: 2345
+    cost: 0.02
+    tools_called: [send_email]
+    result: "Reply sent"
+    messages: '[{"role":"system","content":"..."},{"role":"user","content":"reply to first saying thanks"},{"role":"assistant","content":"Reply sent"}]'
+    eval:
+      expect_tools: [send_email]
+      expect_result: "reply sent"
+```
+
+## Why This Format
+
+**YAML** for readable fields (input, tools, result, cost)
+
+**JSON string** for messages:
+- Won't break YAML (prompts with colons, quotes, etc.)
+- One line = simple to parse
+- Already JSON in code, easy to `json.loads()`
+
+**One turn = one user input**:
+- System message + user input + tool calls + final response
+- No second user input within a turn
+- Second user input = next turn
+
+## Field Reference
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `input` | string | User input for this turn |
+| `model` | string | LLM model used |
+| `duration_ms` | int | How long the turn took |
+| `tokens` | int | Total tokens used |
+| `cost` | float | Cost in USD |
+| `tools_called` | list | Tools that were called |
+| `result` | string | Agent's final response |
+| `messages` | JSON string | Message context window |
+| `eval` | object | Optional expectations |
+| `eval.expect_tools` | list | Expected tools |
+| `eval.expect_result` | string | Expected result description |
+
+## File Structure
+
+```
+.co/
+  sessions/
+    gmail_agent_2024-11-27_11-39-58.yaml   # Auto-saved
+```
+
+## Workflow
+
+1. Run agent → auto-saves to `.co/evals/`
+2. Add `eval:` to turns you want to test
+3. Change prompt → run evals
+4. LLM judges: did tools match? Is result similar?
+
+## Implementation
+
+```python
+# Save
+turn = {
+    'input': user_input,
+    'model': self.llm.model,
+    'duration_ms': duration,
+    'tokens': usage.input_tokens + usage.output_tokens,
+    'cost': usage.cost,
+    'tools_called': [tc.name for tc in tool_calls],
+    'result': response,
+    'messages': json.dumps(messages),
+}
+
+# Load
+session = yaml.safe_load(file)
+messages = json.loads(turn['messages'])
+```
+
+## Alternatives Rejected
+
+- **JSONL** - Crash-unsafe, hard to read
+- **Pure YAML** - Prompts break format (colons, quotes)
+- **Multi-line JSON** - Unnecessary complexity
+- **Separate message files** - Too complex