agent-os-kernel 1.1.0__py3-none-any.whl → 1.3.0__py3-none-any.whl

modules/caas/docs/IMPLEMENTATION_SUMMARY_TRUST_GATEWAY.md

@@ -0,0 +1,319 @@
+# Implementation Summary: Trust Gateway
+
+## Overview
+
+Implemented an **Enterprise-Grade Private Cloud Router** (Trust Gateway) that addresses the "Middleware Gap" problem identified in the issue:
+
+> **The Problem:** No Enterprise CISO will send their proprietary data to a random middleware startup just to save 30% on tokens. The risk of data leakage is too high.
+
+> **The Solution:** Build an On-Prem / Private Cloud Router that enterprises can deploy within their own infrastructure. The winner won't be the one with the smartest routing algorithm; it will be the one the Enterprise trusts with the keys to the kingdom.
+
+## What Was Implemented
+
+### 1. Core Trust Gateway Module (`caas/gateway/trust_gateway.py`)
+
+#### Key Classes and Enums:
+
+**DeploymentMode:**
+- `ON_PREM` - Deployed on customer's own infrastructure
+- `PRIVATE_CLOUD` - Deployed in customer's private cloud (AWS VPC, Azure VNet, GCP VPC)
+- `HYBRID` - Hybrid deployment with local processing and cloud backup
+- `AIR_GAPPED` - Completely isolated from internet (maximum security)
+
+**SecurityLevel:**
+- `STANDARD` - Basic security controls
+- `HIGH` - Enhanced security (encryption at rest and in transit)
+- `MAXIMUM` - Maximum security (air-gapped, zero data retention)
+
+**SecurityPolicy:**
+Comprehensive security configuration including:
+- Authentication & Authorization (user whitelist, IP ranges)
+- Data Classification (public, internal, confidential, secret)
+- Encryption (in transit and at rest)
+- Audit & Compliance (full audit trail, compliance modes: GDPR, HIPAA, SOC2)
+- Network Isolation (control external calls, whitelist endpoints)
+
+**DataRetentionPolicy:**
+- Configurable retention periods (0-365 days)
+- Auto-deletion after retention period
+- Encryption at rest
+- PII scrubbing
+
+**AuditLog:**
+Complete audit trail for compliance:
+- Event tracking (request_routed, data_accessed, policy_changed)
+- User tracking
+- Data classification tracking
+- Metadata storage
+
+**TrustGateway:**
+Main gateway class with:
+- Request validation against security policies
+- Secure routing through heuristic router
+- Audit logging for all operations
+- Deployment information
+- Policy management
+
+### 2. API Integration (`caas/api/server.py`)
+
+Added comprehensive Trust Gateway endpoints:
+
+**`GET /gateway`** - Gateway status and deployment information
+- Returns deployment mode, security level, trust guarantees
+- Shows enterprise-grade security controls
+
+**`POST /gateway/route`** - Route requests with security controls
+- Validates against security policies
+- Performs heuristic routing
+- Logs all activity for compliance
+- Returns routing decision with security context
+
+**`GET /gateway/info`** - Detailed deployment information
+- Comprehensive security configuration
+- Deployment mode explanations
+- Security level details
+
+**`GET /gateway/audit`** - Retrieve audit logs
+- Filter by event type, user, time range
+- Full compliance reporting
+- Security incident investigation
+
+**`POST /gateway/validate`** - Pre-flight request validation
+- Check authentication requirements
+- Validate user authorization
+- Verify data classification
+- Check encryption requirements
+
+**`DELETE /gateway/audit`** - Clear audit logs (with authorization)
+
+### 3. Comprehensive Testing (`test_trust_gateway.py`)
+
+Created 12 comprehensive tests covering:
+
+1. **Basic Gateway Creation** - Default configuration
+2. **Deployment Modes** - All 4 deployment modes (on-prem, private cloud, hybrid, air-gapped)
+3. **Security Levels** - Standard, high, maximum
+4. **Request Routing** - Routing through gateway with security context
+5. **Security Validation** - Authentication and authorization checks
+6. **Data Classification** - Classification validation and enforcement
+7. **Audit Logging** - Full audit trail with filtering
+8. **Data Retention Policy** - Retention configuration and enforcement
+9. **Enterprise Use Case** - Maximum security air-gapped deployment
+10. **Trust Guarantees** - Verification of all trust guarantees
+11. **Security Context** - Security context in routing decisions
+12. **Policy Updates** - Policy change tracking and auditing
+
+**Result:** All 12 tests pass ✅
+
+### 4. Interactive Demo (`demo_trust_gateway.py`)
+
+Created 7 comprehensive demos:
+
+1. **Basic Gateway** - Simple deployment and usage
+2. **Enterprise Deployment** - Maximum security air-gapped configuration
+3. **Security Validation** - Authentication and classification enforcement
+4. **Routing Decisions** - Heuristic routing through gateway
+5. **Audit Trail** - Compliance reporting and log filtering
+6. **Comparison** - SaaS Router vs Trust Gateway analysis
+7. **Deployment Options** - All deployment modes showcase
+
+### 5. Documentation
+
+#### `TRUST_GATEWAY.md` (Comprehensive Guide)
+- Problem statement and opportunity
+- Architecture diagram
+- Deployment modes with examples
+- Security features (authentication, encryption, audit)
+- Usage examples
+- API integration
+- Comparison with SaaS routers
+- Cost analysis ($19,200 annual savings)
+- Best practices
+- Compliance support (GDPR, HIPAA, SOC2, ISO 27001, FedRAMP)
+
+#### Updated `README.md`
+- Added Trust Gateway to feature list
+- Added 7th major fallacy: "The Trust Gateway (Middleware Gap)"
+- Added Trust Gateway section with examples
+- Updated project structure
+- Added tests and demos to documentation
+
+## Key Features Implemented
+
+### 1. Zero Data Leakage
+- All routing happens within customer infrastructure
+- No external API calls for routing decisions
+- Data never leaves customer environment
+
+### 2. Deployment Flexibility
+- **On-Premises:** Deploy on customer's own servers
+- **Private Cloud:** Deploy in isolated VPC/VNet
+- **Hybrid:** Local processing with cloud backup
+- **Air-Gapped:** Complete isolation for maximum security
+
+### 3. Enterprise Security Controls
+- **Authentication & Authorization:** User whitelist, IP restrictions
+- **Data Classification:** Enforce classification labels (public, internal, confidential, secret)
+- **Encryption:** In transit (TLS/HTTPS) and at rest
+- **Audit Trail:** Complete logging for compliance
+- **Compliance Modes:** GDPR, HIPAA, SOC2 support
+
+### 4. Battle-Tested Routing
+- Uses existing HeuristicRouter (deterministic, 0ms decisions)
+- No external AI classifier needed
+- Speed > Smarts philosophy maintained
+
+### 5. Full Transparency
+- Complete audit trail for all operations
+- Security context in every routing decision
+- Trust guarantees clearly stated
+
+## Architecture
+
+```
+┌─────────────────────────────────────────────────────────┐
+│         YOUR INFRASTRUCTURE (On-Prem / Private Cloud)   │
+│                                                          │
+│  ┌────────────────────────────────────────────────┐    │
+│  │           Trust Gateway                         │    │
+│  │  ┌──────────────────────────────────────────┐  │    │
+│  │  │  Security Policy Engine                  │  │    │
+│  │  │  - Authentication & Authorization        │  │    │
+│  │  │  - Data Classification                   │  │    │
+│  │  │  - Encryption Controls                   │  │    │
+│  │  └──────────────────────────────────────────┘  │    │
+│  │  ┌──────────────────────────────────────────┐  │    │
+│  │  │  Heuristic Router                        │  │    │
+│  │  │  - 0ms routing decisions                 │  │    │
+│  │  │  - Deterministic rules                   │  │    │
+│  │  │  - No external AI calls                  │  │    │
+│  │  └──────────────────────────────────────────┘  │    │
+│  │  ┌──────────────────────────────────────────┐  │    │
+│  │  │  Audit & Compliance Logger               │  │    │
+│  │  │  - Full request tracking                 │  │    │
+│  │  │  - Compliance reporting                  │  │    │
+│  │  │  - Data retention policies               │  │    │
+│  │  └──────────────────────────────────────────┘  │    │
+│  └────────────────────────────────────────────────┘    │
+│                          │                              │
+│                          ▼                              │
+│  ┌────────────────────────────────────────────────┐    │
+│  │     Your Internal AI Models                    │    │
+│  │     (GPT, Claude, LLaMA, etc.)                 │    │
+│  └────────────────────────────────────────────────┘    │
+│                                                          │
+└─────────────────────────────────────────────────────────┘
+
+NO DATA LEAVES YOUR INFRASTRUCTURE
+```
+
+## Trust Guarantees
+
+✅ **Data never leaves your infrastructure**  
+✅ **Full audit trail for compliance**  
+✅ **Configurable retention policies**  
+✅ **Enterprise-grade security controls**  
+✅ **Zero third-party data sharing**
+
+## Comparison: SaaS vs Trust Gateway
+
+| Feature | SaaS Router | Trust Gateway |
+|---------|-------------|---------------|
+| **Deployment** | External service | Your infrastructure |
+| **Data Location** | Third-party servers | Your servers only |
+| **Data Security** | ⚠️ Third-party trust | ✅ Complete control |
+| **CISO Approval** | ❌ Difficult | ✅ Easy |
+| **Compliance** | ⚠️ Provider-based | ✅ Your controls |
+| **Audit Trail** | ⚠️ Limited | ✅ Full transparency |
+| **Cost Savings** | 30% tokens | 30% tokens |
+| **Latency** | +500ms external | 0ms local |
+| **Trust** | ⚠️ Third-party | ✅ Self-hosted |
+
+## Cost Analysis
+
+**SaaS Router (External Service):**
+- External routing service: $500/month
+- Data egress fees: $200/month
+- Third-party security audit: $1,000/month
+- **Total: $1,700/month + breach risk**
+
+**Trust Gateway (Self-Hosted):**
+- Server resources: $100/month (marginal)
+- No external service fees: $0
+- No data egress: $0
+- **Total: $100/month + full control**
+
+**Annual Savings: $19,200 + eliminated security risk**
+
+## Philosophy
+
+> "The winner won't be the one with the smartest routing algorithm; it will be the one the Enterprise trusts with the keys to the kingdom."
+
+The Trust Gateway addresses the fundamental CISO concern: **No Enterprise will send proprietary data to a random middleware startup.** By deploying within the customer's own infrastructure, the Trust Gateway eliminates this concern entirely.
+
+## Testing Results
+
+All 12 comprehensive tests pass:
+✅ On-Prem / Private Cloud deployment  
+✅ Zero data leakage (data never leaves infrastructure)  
+✅ Full audit trail for compliance  
+✅ Configurable security policies  
+✅ Authentication and authorization  
+✅ Data classification and encryption  
+✅ Enterprise-grade controls  
+
+## Files Added/Modified
+
+### New Files:
+1. `caas/gateway/__init__.py` - Gateway module initialization
+2. `caas/gateway/trust_gateway.py` - Core Trust Gateway implementation (18.5KB)
+3. `TRUST_GATEWAY.md` - Comprehensive documentation (15.6KB)
+4. `test_trust_gateway.py` - Comprehensive test suite (13.8KB)
+5. `demo_trust_gateway.py` - Interactive demos (13.6KB)
+
+### Modified Files:
+1. `caas/api/server.py` - Added 6 Trust Gateway endpoints
+2. `README.md` - Added Trust Gateway section and documentation
+
+### Total Implementation:
+- **5 new files**
+- **2 modified files**
+- **~61KB of new code**
+- **12 passing tests**
+- **7 interactive demos**
+- **Full documentation**
+
+## Usage Example
+
+```python
+from caas.gateway import TrustGateway, SecurityPolicy, DeploymentMode
+
+# Configure enterprise security
+policy = SecurityPolicy(
+    deployment_mode=DeploymentMode.ON_PREM,
+    security_level="maximum",
+    require_authentication=True,
+    data_classification_required=True,
+    encrypt_in_transit=True,
+    encrypt_at_rest=True,
+    audit_all_requests=True,
+    compliance_mode="SOC2"
+)
+
+gateway = TrustGateway(security_policy=policy)
+
+# Route request with security controls
+result = gateway.route_request(
+    query="Analyze Q4 financials",
+    user_id="ciso@company.com",
+    data_classification="confidential"
+)
+
+# Result includes routing decision + security context + audit trail
+```
+
+## Conclusion
+
+The Trust Gateway implementation successfully addresses the "Middleware Gap" by providing an enterprise-grade, on-premises / private cloud router that CISOs can trust. The solution maintains all the benefits of intelligent routing (30% token savings) while eliminating the primary CISO concern: data leakage to third-party services.
+
+**Key Achievement:** Built the infrastructure that enterprises can deploy within their own environment, ensuring data never leaves their control while maintaining intelligent routing capabilities.

modules/caas/docs/LAYER_1_PRIMITIVE.md

@@ -0,0 +1,202 @@
+# Layer 1: The Primitive (caas-core)
+
+## Elevator Pitch
+
+A pure, logic-only library for routing context, handling RAG fallacies, and managing context windows. It does not know what an "agent" is; it only knows about text and vectors.
+
+## Publication Target
+
+```bash
+pip install caas-core
+```
+
+## Allowed Dependencies
+
+| Dependency | Purpose | Status |
+|------------|---------|--------|
+| numpy | Data handling, vector operations | ✅ Allowed |
+| pandas | Data handling, analysis | ✅ Allowed |
+| openai | Embeddings (optional) | ✅ Allowed (optional) |
+| langchain | Embeddings (optional) | ✅ Allowed (optional) |
+| scikit-learn | TF-IDF, similarity | ✅ Allowed |
+| pydantic | Data validation | ✅ Allowed |
+| fastapi | API server | ✅ Allowed |
+| tiktoken | Token counting | ✅ Allowed |
+
+## Strictly Forbidden Dependencies
+
+The following packages **MUST NEVER** be imported or added as dependencies:
+
+| Package | Reason |
+|---------|--------|
+| `agent-control-plane` | Agent orchestration - violates primitive layer |
+| `iatp` | Agent transport protocol - violates primitive layer |
+| `scak` | Agent coordination - violates primitive layer |
+| Any `*Agent` class | Agent abstraction - violates primitive layer |
+| Any `*Supervisor` class | Agent orchestration - violates primitive layer |
+
+## Design Principles
+
+### 1. Stateless Context Routing
+
+All context routing logic MUST be stateless. It should only process the data passed to it.
+
+```python
+# ✅ CORRECT: Stateless routing
+def route_context(query: str, metadata: dict) -> RoutingDecision:
+    # Process only the data passed in
+    return analyze_query(query, metadata)
+
+# ❌ WRONG: Querying active runtime
+def route_context(query: str, agent: Agent) -> RoutingDecision:
+    # DON'T query agent runtime state
+    state = agent.get_current_state()  # FORBIDDEN
+    return analyze_query(query, state)
+```
+
+### 2. Generic Identifiers, Not Objects
+
+Accept `agent_id: str` or `metadata: dict`, not `Agent` objects.
+
+```python
+# ✅ CORRECT: Generic identifier
+def add_hot_context(content: str, source_id: str, metadata: dict) -> str:
+    return store_context(content, source_id, metadata)
+
+# ❌ WRONG: Agent object dependency
+def add_hot_context(content: str, agent: Agent) -> str:
+    source_id = agent.id  # FORBIDDEN
+    return store_context(content, source_id, agent.metadata)
+```
+
+### 3. Data In, Data Out
+
+Methods receive data and return data. No side effects on external systems.
+
+```python
+# ✅ CORRECT: Pure function
+def extract_context(document: Document, query: str, max_tokens: int) -> ContextResponse:
+    sections = score_sections(document, query)
+    return build_response(sections, max_tokens)
+
+# ❌ WRONG: Side effects
+def extract_context(document: Document, agent_bus: AgentBus) -> ContextResponse:
+    agent_bus.notify("context_extracted")  # FORBIDDEN
+    return response
+```
+
+## Core Components
+
+### Context Triad (Hot/Warm/Cold)
+
+Pure context tier management with no agent awareness:
+
+```python
+from caas.triad import ContextTriadManager
+
+triad = ContextTriadManager()
+
+# Add context by layer (accepts generic metadata, not agents)
+triad.add_hot_context("Current error log", metadata={"source": "system"})
+triad.add_warm_context("User prefers verbose output", metadata={"type": "preference"})
+triad.add_cold_context("2023 Q1 meeting notes", metadata={"date": "2023-01-15"})
+
+# Get context (pure data retrieval)
+hot = triad.get_hot_context(max_tokens=1000)
+```
+
+### Heuristic Router
+
+Deterministic routing without LLM classification overhead:
+
+```python
+from caas.routing import HeuristicRouter
+
+router = HeuristicRouter()
+
+# Route based on query content only (stateless)
+decision = router.route("Summarize this document")
+print(decision.model_tier)  # ModelTier.SMART
+print(decision.reason)       # "Complex task keywords detected: summarize"
+```
+
+### Conversation Manager (Sliding Window)
+
+FIFO conversation management - no summarization, no agent awareness:
+
+```python
+from caas.conversation import ConversationManager
+
+conv = ConversationManager(max_turns=10)
+
+# Add turns (accepts strings, not agent objects)
+conv.add_turn(user_message="What is X?", ai_response="X is...")
+
+# Get history (pure data)
+history = conv.get_conversation_history()
+```
+
+### Trust Gateway
+
+Enterprise security without agent coupling:
+
+```python
+from caas.gateway import TrustGateway, SecurityPolicy
+
+gateway = TrustGateway(
+    security_policy=SecurityPolicy(
+        deployment_mode="on_prem",
+        security_level="high"
+    )
+)
+
+# Validate request (accepts generic identifiers)
+result = gateway.validate_request(
+    request_data={"query": "..."},
+    user_id="user-123",  # String identifier, not User object
+    data_classification="confidential"
+)
+```
+
+## Integration Pattern
+
+External systems (agent frameworks, orchestrators) should use CaaS as a **service layer**:
+
+```python
+# Example: How an external agent framework uses caas-core
+
+from caas.triad import ContextTriadManager
+from caas.routing import HeuristicRouter
+
+# Initialize caas-core components
+triad = ContextTriadManager()
+router = HeuristicRouter()
+
+class MyExternalAgent:
+    """This agent class is OUTSIDE caas-core, not inside."""
+    
+    def __init__(self, agent_id: str):
+        self.agent_id = agent_id
+    
+    def process_query(self, query: str) -> str:
+        # Use caas-core as a service (passing data, not self)
+        routing = router.route(query)
+        
+        # Get context (passing identifier, not agent object)
+        triad.add_hot_context(query, metadata={"agent_id": self.agent_id})
+        context = triad.get_merged_context(max_tokens=4000)
+        
+        # Agent does its own LLM call
+        return self.call_llm(context, query, routing.suggested_model)
+```
+
+## Verification Checklist
+
+Before any release, verify:
+
+- [ ] `grep -r "import.*Agent" src/caas/` returns no results
+- [ ] `grep -r "from.*Agent" src/caas/` returns no results
+- [ ] `grep -r "import.*Supervisor" src/caas/` returns no results
+- [ ] `grep -r "agent-control-plane\|iatp\|scak" src/caas/` returns no results
+- [ ] All public methods accept `str` or `dict` for identifiers, not objects
+- [ ] No methods query external runtime state