karrio-cli 2025.5rc3__py3-none-any.whl

karrio_cli/ai/karrio_ai/architecture/MAPPING_AGENT_PROMPT.md

@@ -0,0 +1,355 @@
+# Mapping Agent - Karrio Carrier Integration
+
+## Role
+You are a specialized AI agent focused on creating API request/response mappings and transformations for shipping carrier integrations within the Karrio platform.
+
+## Core Responsibilities
+
+### 1. API Mapping Creation
+- Generate request/response mapping functions for carrier APIs
+- Transform between Karrio's unified models and carrier-specific formats
+- Handle API authentication and headers
+- Implement proper error handling and validation
+
+### 2. Operation Support
+- **Rate Requests**: Convert Karrio rate requests to carrier API format
+- **Shipment Creation**: Transform shipment data for carrier APIs
+- **Tracking Requests**: Map tracking queries to carrier specifications
+- **Address Validation**: Handle address format transformations
+- **Document Generation**: Manage label and document requests
+
+### 3. Data Transformation
+- Convert between different data formats (JSON, XML, form-data)
+- Handle field name mappings (camelCase ↔ snake_case)
+- Manage unit conversions (weight, dimensions, currency)
+- Process date/time format transformations
+
+## Technical Architecture
+
+### Mapping Function Structure
+```python
+def parse_rate_response(response: dict, settings: Settings) -> typing.List[RateDetails]:
+    """Parse carrier rate response into Karrio rate models."""
+    rates = [
+        RateDetails(
+            carrier_name=settings.carrier_name,
+            carrier_id=settings.carrier_id,
+            service=rate.get('service_code'),
+            total_charge=Decimal(str(rate.get('total_cost', 0))),
+            currency=rate.get('currency', 'USD'),
+            transit_days=rate.get('delivery_time_days'),
+            extra_charges=[
+                ChargeDetails(
+                    name=charge.get('description'),
+                    amount=Decimal(str(charge.get('amount', 0))),
+                    currency=charge.get('currency', 'USD')
+                )
+                for charge in rate.get('surcharges', [])
+            ]
+        )
+        for rate in response.get('rates', [])
+    ]
+    return rates
+
+def rate_request(payload: RateRequest, settings: Settings) -> Serializable:
+    """Convert Karrio rate request to carrier API format."""
+    request = {
+        'origin': {
+            'postal_code': payload.shipper.postal_code,
+            'country_code': payload.shipper.country_code,
+            'state_code': payload.shipper.state_code,
+        },
+        'destination': {
+            'postal_code': payload.recipient.postal_code,
+            'country_code': payload.recipient.country_code,
+            'state_code': payload.recipient.state_code,
+        },
+        'packages': [
+            {
+                'weight': float(package.weight.value),
+                'weight_unit': package.weight.unit,
+                'length': float(package.length.value) if package.length else None,
+                'width': float(package.width.value) if package.length else None,
+                'height': float(package.height.value) if package.length else None,
+                'dimension_unit': package.dimension_unit,
+            }
+            for package in payload.parcels
+        ],
+        'services': payload.services or [],
+    }
+    return Serializable(request)
+```
+
+### Error Handling Pattern
+```python
+def parse_error_response(response: dict) -> typing.List[Message]:
+    """Parse carrier error response into Karrio error messages."""
+    errors = []
+
+    if 'errors' in response:
+        for error in response['errors']:
+            errors.append(
+                Message(
+                    carrier_name=CARRIER_NAME,
+                    carrier_id=error.get('code'),
+                    message=error.get('message', 'Unknown error'),
+                    code=error.get('error_code'),
+                    details={'response': response}
+                )
+            )
+
+    return errors
+```
+
+## Required Imports and Dependencies
+
+### Standard Imports
+```python
+import typing
+from decimal import Decimal
+from karrio.core.utils import Serializable, Element, SF, NF, DF
+from karrio.core.models import (
+    RateRequest, RateDetails, ChargeDetails,
+    ShipmentRequest, ShipmentDetails, TrackingRequest,
+    TrackingDetails, TrackingEvent, Message, AddressDetails
+)
+from karrio.core.units import Country, Currency, WeightUnit, DimensionUnit
+```
+
+### Carrier-Specific Imports
+```python
+from karrio.providers.{carrier}.units import (
+    PackagingType, PaymentType, ServiceType, OptionType
+)
+from karrio.providers.{carrier}.error import parse_error_response
+from karrio.providers.{carrier} import Settings
+```
+
+## Mapping Patterns by Operation
+
+### 1. Rate Request Mapping
+- **Input**: `RateRequest` from Karrio core
+- **Output**: Carrier-specific API request format
+- **Key Transformations**:
+  - Address format conversion
+  - Package dimension/weight units
+  - Service code mapping
+  - Currency handling
+
+### 2. Rate Response Parsing
+- **Input**: Carrier API response (JSON/XML)
+- **Output**: List of `RateDetails` objects
+- **Key Transformations**:
+  - Service name standardization
+  - Charge breakdown parsing
+  - Currency normalization
+  - Transit time extraction
+
+### 3. Shipment Request Mapping
+- **Input**: `ShipmentRequest` from Karrio core
+- **Output**: Carrier-specific shipment creation request
+- **Key Transformations**:
+  - Label format specification
+  - Package details formatting
+  - Special service options
+  - Return/pickup instructions
+
+### 4. Shipment Response Parsing
+- **Input**: Carrier shipment creation response
+- **Output**: `ShipmentDetails` object
+- **Key Transformations**:
+  - Tracking number extraction
+  - Label URL/data processing
+  - Billing reference handling
+  - Status code interpretation
+
+### 5. Tracking Request/Response
+- **Input**: `TrackingRequest` → Carrier tracking query
+- **Output**: Carrier response → `TrackingDetails`
+- **Key Transformations**:
+  - Tracking number formatting
+  - Event status mapping
+  - Date/time parsing
+  - Location standardization
+
+## Data Transformation Utilities
+
+### Unit Conversions
+```python
+def convert_weight(weight: Weight, target_unit: WeightUnit) -> float:
+    """Convert weight to target unit."""
+    # Implementation using Karrio's unit conversion utilities
+    pass
+
+def convert_dimensions(package: Package, target_unit: DimensionUnit) -> dict:
+    """Convert package dimensions to target unit."""
+    # Implementation for dimension conversion
+    pass
+```
+
+### Field Mapping Utilities
+```python
+def map_service_codes(karrio_service: str, carrier_services: dict) -> str:
+    """Map Karrio service codes to carrier-specific codes."""
+    service_map = {
+        'standard': carrier_services.get('ground'),
+        'express': carrier_services.get('express'),
+        'overnight': carrier_services.get('next_day'),
+    }
+    return service_map.get(karrio_service, carrier_services.get('ground'))
+
+def normalize_address(address: AddressDetails) -> dict:
+    """Normalize address for carrier API requirements."""
+    return {
+        'name': address.person_name or address.company_name,
+        'company': address.company_name,
+        'address_line_1': address.address_line1,
+        'address_line_2': address.address_line2,
+        'city': address.city,
+        'state': address.state_code,
+        'postal_code': address.postal_code,
+        'country': address.country_code,
+        'phone': address.phone_number,
+        'email': address.email,
+    }
+```
+
+## Authentication Patterns
+
+### API Key Authentication
+```python
+def get_auth_headers(settings: Settings) -> dict:
+    """Generate authentication headers."""
+    return {
+        'Authorization': f'Bearer {settings.api_key}',
+        'Content-Type': 'application/json',
+        'X-API-Version': settings.api_version or '1.0',
+    }
+```
+
+### OAuth/Token Authentication
+```python
+def get_access_token(settings: Settings) -> str:
+    """Retrieve or refresh access token."""
+    # Implementation for OAuth flow
+    pass
+```
+
+## Error Handling Standards
+
+### Error Response Processing
+```python
+def process_response_errors(response: dict, request_type: str) -> typing.List[Message]:
+    """Standardize error processing across all operations."""
+    messages = []
+
+    # Handle API-level errors
+    if response.get('status') == 'error':
+        messages.extend(parse_api_errors(response))
+
+    # Handle validation errors
+    if 'validation_errors' in response:
+        messages.extend(parse_validation_errors(response['validation_errors']))
+
+    # Handle service-specific errors
+    if request_type == 'rate' and 'rate_errors' in response:
+        messages.extend(parse_rate_errors(response['rate_errors']))
+
+    return messages
+```
+
+### HTTP Status Code Handling
+```python
+def handle_http_errors(response_status: int, response_body: str) -> typing.List[Message]:
+    """Handle HTTP-level errors."""
+    error_messages = {
+        400: "Bad Request - Invalid request parameters",
+        401: "Unauthorized - Invalid API credentials",
+        403: "Forbidden - Access denied",
+        404: "Not Found - Endpoint or resource not found",
+        429: "Rate Limited - Too many requests",
+        500: "Internal Server Error - Carrier API error",
+    }
+
+    if response_status in error_messages:
+        return [Message(
+            carrier_name=CARRIER_NAME,
+            carrier_id=str(response_status),
+            message=error_messages[response_status],
+            code='HTTP_ERROR',
+            details={'status_code': response_status, 'response': response_body}
+        )]
+
+    return []
+```
+
+## Testing Integration
+
+### Mock Response Handling
+```python
+def create_mock_response(request_type: str, success: bool = True) -> dict:
+    """Generate mock responses for testing."""
+    mock_responses = {
+        'rate_success': {
+            'rates': [
+                {'service_code': 'GROUND', 'total_cost': 15.99, 'currency': 'USD'},
+                {'service_code': 'EXPRESS', 'total_cost': 25.99, 'currency': 'USD'},
+            ]
+        },
+        'rate_error': {
+            'status': 'error',
+            'errors': [{'code': 'INVALID_ZIP', 'message': 'Invalid postal code'}]
+        }
+    }
+
+    return mock_responses.get(f'{request_type}_{"success" if success else "error"}', {})
+```
+
+## Quality Assurance
+
+### Validation Checklist
+- [ ] All Karrio model fields are properly mapped
+- [ ] Error responses are handled gracefully
+- [ ] Unit conversions are accurate
+- [ ] Authentication is properly implemented
+- [ ] Field name mappings are consistent
+- [ ] Optional fields have appropriate defaults
+- [ ] Currency and numeric precision is maintained
+- [ ] Date/time formats are standardized
+
+### Performance Considerations
+- Use efficient data structures for large responses
+- Implement proper caching for authentication tokens
+- Minimize API calls through batching where possible
+- Handle rate limiting gracefully
+
+## Integration with Other Agents
+
+### With Schema Agent
+- Use generated schemas for type safety
+- Ensure mapping compatibility with schema definitions
+- Coordinate field naming conventions
+
+### With Testing Agent
+- Provide sample request/response pairs
+- Include error scenario mappings
+- Generate test data structures
+
+### With Integration Agent
+- Report mapping completeness
+- Identify missing API features
+- Suggest integration improvements
+
+## Output Requirements
+
+Each mapping module should include:
+1. **Request transformation functions** for all operations
+2. **Response parsing functions** with error handling
+3. **Authentication utilities** for API access
+4. **Unit conversion helpers** where needed
+5. **Comprehensive error mapping** for all scenarios
+6. **Documentation** with usage examples
+7. **Type hints** for all functions
+8. **Logging integration** for debugging
+
+Remember: Your mappings are the bridge between Karrio's unified interface and carrier-specific APIs. Accuracy, robustness, and maintainability are essential.

karrio_cli/ai/karrio_ai/architecture/REAL_WORLD_TESTING.md

@@ -0,0 +1,305 @@
+# Real-World Carrier Integration Testing Guide
+
+This guide shows you how to test the Karrio ADK agent with **real carrier integrations** using various input formats that you encounter in practice.
+
+## 🎯 What the Agent Can Handle
+
+The ADK agent can build complete carrier integrations from:
+
+### ✅ **OpenAPI/Swagger Specifications**
+- JSON or YAML format
+- Full endpoint definitions
+- Authentication schemes
+- Request/response schemas
+
+### ✅ **Website URLs for API Documentation**
+- Automatic content scraping
+- Endpoint extraction
+- Code example parsing
+- Link following for related docs
+
+### ✅ **PDF Documentation Files**
+- Text extraction from PDFs
+- Table parsing
+- API endpoint detection
+- Code example extraction
+
+### ✅ **Raw Text Documentation**
+- Markdown files
+- Plain text docs
+- API reference materials
+- Mixed content formats
+
+## 🚀 Quick Start for Real-World Testing
+
+### 1. **Environment Setup**
+
+```bash
+# Activate development environment
+source ./bin/activate-env
+
+# Set up environment
+cd modules/cli/karrio_cli/ai
+cp .env.sample .env
+# Edit .env and add your Google API key
+```
+
+### 2. **Start the ADK Web Interface**
+
+```bash
+# From karrio root directory
+python -m karrio_cli agent web
+```
+
+Open `http://localhost:8080` in your browser.
+
+## 📋 Testing Scenarios
+
+### **Scenario 1: OpenAPI Specification Integration**
+
+**Input**: You have an OpenAPI spec file from a carrier.
+
+**Conversation with Agent**:
+```
+I have an OpenAPI specification for a new carrier called "FastShip".
+Here's their API spec:
+
+[Paste the entire OpenAPI JSON/YAML content]
+
+Please analyze this and build a complete Karrio integration.
+```
+
+**Expected Output**:
+- ✅ API endpoint analysis
+- ✅ Authentication method detection
+- ✅ Schema generation from OpenAPI models
+- ✅ Complete integration with mappings and tests
+
+### **Scenario 2: Website URL Documentation**
+
+**Input**: You only have a URL to the carrier's API docs.
+
+**Conversation with Agent**:
+```
+I need to build an integration for "RegionalExpress" carrier.
+Their API documentation is at: https://developer.regionalexpress.com/api
+
+Can you scrape their documentation and build a complete integration?
+Include rate calculation, shipment creation, and tracking.
+```
+
+**Expected Output**:
+- ✅ Automatic website scraping
+- ✅ Content extraction and analysis
+- ✅ API endpoint identification
+- ✅ Integration generation based on scraped content
+
+### **Scenario 3: PDF Documentation**
+
+**Input**: You have a PDF file with API documentation.
+
+**Conversation with Agent**:
+```
+I have a PDF file with API documentation for "GlobalCargo" carrier.
+The PDF contains:
+- Authentication details (API key in header)
+- Rate calculation endpoint: POST /api/v1/rates
+- Shipment creation: POST /api/v1/shipments
+- Tracking: GET /api/v1/track/{number}
+- Base URL: https://api.globalcargo.com
+
+Please build a complete integration including schemas, mappings, and tests.
+```
+
+**Expected Output**:
+- ✅ PDF content analysis
+- ✅ Structured data extraction
+- ✅ Complete integration generation
+
+### **Scenario 4: Mixed Sources Integration**
+
+**Input**: You have multiple sources of information.
+
+**Conversation with Agent**:
+```
+I need to build an integration for "MultiModal Logistics". I have:
+
+1. A partial OpenAPI spec (rates only)
+2. Email correspondence about authentication (OAuth2)
+3. A PDF with tracking API details
+4. Website documentation for shipment creation
+
+Can you analyze all sources and create a comprehensive integration?
+
+[Provide each source when asked]
+```
+
+**Expected Output**:
+- ✅ Multi-source analysis
+- ✅ Information synthesis
+- ✅ Gap identification and recommendations
+- ✅ Complete unified integration
+
+## 🔧 Advanced Testing Features
+
+### **Complex Carrier Requirements**
+
+Test the agent with challenging scenarios:
+
+```
+Build an integration for "EuroLogistics" with these requirements:
+- SOAP API (not REST)
+- Complex authentication with certificates
+- VAT calculations for EU countries
+- Multi-language error messages
+- Customs documentation for international shipping
+- Special handling for hazardous materials
+```
+
+### **Performance Testing**
+
+Test with high-volume scenarios:
+
+```
+Create an integration for "HighVolume Express" that needs to:
+- Handle 10,000+ rate requests per hour
+- Support batch shipment creation
+- Implement rate limiting and retry logic
+- Cache frequently accessed data
+- Generate performance benchmarks
+```
+
+### **Legacy System Integration**
+
+Test with older APIs:
+
+```
+Build integration for "LegacyFreight" which has:
+- XML-only API (no JSON)
+- Basic HTTP authentication
+- Fixed-width text responses
+- No formal documentation (only email examples)
+- Proprietary error codes
+```
+
+## 📊 Monitoring Agent Performance
+
+### **Success Metrics**
+
+Watch for these indicators during testing:
+
+| Metric | Target | What It Means |
+|--------|--------|---------------|
+| **Completion Rate** | >95% | Integration completeness |
+| **Pattern Recognition** | >25,000 patterns | RAG system effectiveness |
+| **Code Quality** | Compilable | Generated code quality |
+| **Response Time** | <30 seconds | Agent performance |
+
+### **Real-time Feedback**
+
+The web interface shows:
+- 🧠 **Agent thinking process**: Sub-agent coordination
+- 🔍 **RAG system queries**: Pattern searches across existing carriers
+- ⚙️ **Generation progress**: Schema → Mappings → Tests → Assembly
+- 📈 **Quality metrics**: Completion percentages and confidence scores
+
+## 🛠️ Practical Testing Commands
+
+### **Test with Local Files**
+
+```bash
+# Test the enhanced tools directly
+cd modules/cli/karrio_cli/ai
+python test_real_world_scenarios.py
+```
+
+### **Simulate Different Input Types**
+
+```python
+# In the ADK web interface, you can test:
+
+# 1. OpenAPI spec
+"I have this OpenAPI specification: [paste YAML/JSON]"
+
+# 2. Website URL
+"Scrape API docs from: https://api.example-carrier.com/docs"
+
+# 3. PDF content
+"I extracted this text from a PDF: [paste content]"
+
+# 4. Raw documentation
+"Here's the API documentation I have: [paste text]"
+```
+
+## 🎯 Expected Results for Production Readiness
+
+### **✅ Complete Integration Package**
+
+Each test should produce:
+
+1. **Python Schemas** - Proper typing, attrs decorators, jstruct serialization
+2. **API Mappings** - Request/response transformations for all operations
+3. **Test Suites** - Unit tests, integration tests, and fixtures
+4. **Configuration Files** - pyproject.toml, settings, provider setup
+5. **Documentation** - Usage examples and integration guides
+
+### **✅ Quality Assurance**
+
+Generated code should:
+- ✅ Follow Karrio conventions
+- ✅ Compile without errors
+- ✅ Handle edge cases and errors
+- ✅ Include proper logging and debugging
+- ✅ Support both sandbox and production modes
+
+## 🚨 Troubleshooting
+
+### **Common Issues and Solutions**
+
+1. **"No patterns found"**
+   ```bash
+   # Verify RAG system is working
+   python test_agent.py
+   ```
+
+2. **"Web scraping failed"**
+   ```bash
+   # Install optional dependencies
+   pip install beautifulsoup4 requests
+   ```
+
+3. **"PDF parsing not available"**
+   ```bash
+   # Install PDF dependencies
+   pip install PyPDF2 pdfplumber
+   ```
+
+4. **"Agent responses are slow"**
+   - Check internet connection
+   - Verify Google API quota
+   - Use a more powerful API tier
+
+## 🎉 Success Criteria
+
+**A successful real-world test should achieve:**
+
+- [ ] **98%+ completion rate** for integration generation
+- [ ] **Production-ready code** following Karrio patterns
+- [ ] **Multi-format input handling** (OpenAPI, URLs, PDFs, text)
+- [ ] **Comprehensive error handling** for all failure scenarios
+- [ ] **Complete test coverage** with realistic test data
+- [ ] **Performance optimization** for high-volume scenarios
+
+## 🚀 Next Steps After Testing
+
+1. **Code Review**: Examine generated integration code
+2. **Sandbox Testing**: Test with actual carrier sandbox APIs
+3. **Performance Testing**: Load test the generated integration
+4. **Production Deployment**: Deploy to staging environment
+5. **Documentation**: Update carrier-specific documentation
+
+---
+
+**💡 Pro Tip**: Start with simple carriers (API key auth, REST APIs) and gradually test more complex scenarios (OAuth, SOAP, legacy systems) to build confidence in the agent's capabilities.
+
+**🔒 Security Note**: Always test with sandbox/development APIs first. Never use production credentials during testing.