xray-sdk 0.1.0__tar.gz

xray_sdk-0.1.0/PKG-INFO

@@ -0,0 +1,190 @@
+Metadata-Version: 2.4
+Name: xray-sdk
+Version: 0.1.0
+Summary: Lightweight debugging SDK for multi-step AI pipelines.
+Author: Equal Collective
+License: Proprietary
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+Requires-Dist: requests>=2.31.0
+
+# X-Ray SDK and API
+
+A lightweight debugging system for multi-step AI pipelines that captures execution data and uses AI to identify faulty steps.
+
+## Quick Start
+
+### 1. Install Dependencies
+
+```bash
+pip3 install flask flask-sqlalchemy flask-cors psycopg2-binary openai python-dotenv requests
+```
+
+### 2. Configure Environment
+
+Create a `.env` file:
+```env
+DATABASE_URL=postgresql://user:pass@host/dbname
+CEREBRAS_API_KEY=your-api-key
+CEREBRAS_BASE_URL=https://api.cerebras.ai/v1
+CEREBRAS_MODEL=llama3.1-8b
+```
+
+### 3. Initialize Database
+
+```bash
+python3 -c "
+from dotenv import load_dotenv
+load_dotenv()
+from xray_api.app import create_app
+from xray_api.models import db
+
+app = create_app()
+with app.app_context():
+    db.create_all()
+    print('Database tables created!')
+"
+```
+
+### 4. Start the API Server
+
+```bash
+python3 -m xray_api.app
+```
+
+### 5. Run Example
+
+```bash
+python3 examples/amazon_competitor.py
+```
+
+## SDK Usage
+
+```python
+from xray_sdk import XRayClient, XRayRun, XRayStep
+
+# Create a run
+run = XRayRun("my_pipeline", metadata={"context": "test"}, sample_size=50)
+
+# Add steps (after your pipeline executes)
+run.add_step(XRayStep(
+    name="keyword_generation",
+    order=1,
+    inputs={"title": "Phone Case"},
+    outputs={"keywords": ["phone case", "iphone"]},
+    description="Generate search keywords from the title."# explain what this step does
+))
+
+run.add_step(XRayStep(
+    name="search",
+    order=2,
+    inputs={"keywords": ["phone case", "iphone"]},
+    outputs={"candidates_count": 100},
+    description="Search the catalog for items matching the keywords."
+))
+
+run.add_step(XRayStep(
+    name="filter",
+    order=3,
+    inputs={"candidates_count": 100},
+    outputs={"filtered_count": 5},
+    description="Filter candidates by rating.",
+    reasons={"dropped_items": [{"id": 123, "reason": "low rating"}]},
+    metrics={"elimination_rate": 0.95}
+))
+
+# Send for analysis
+client = XRayClient("http://localhost:5000")
+result = client.send(run)
+
+print(result["analysis"])
+# {
+#   "faulty_step": "keyword_generation",
+#   "reason": "...",
+#   "suggestion": "..."
+# }
+```
+
+## SDK Client Methods
+
+| Method | Description |
+|--------|-------------|
+| `send(run, analyze=True)` | Send run to API; spools locally if unavailable |
+| `spool(run)` | Manually save run to `.xray_spool/` |
+| `flush_spool()` | Send newest spooled run and delete all spool files |
+| `list_pipelines()` | List all pipelines |
+| `list_runs(pipeline, status, limit)` | List runs with filters |
+| `get_run(run_id)` | Get run with all steps |
+| `get_analysis(run_id)` | Get analysis result only |
+| `search_steps(step_name, pipeline, limit)` | Search steps across runs |
+
+## API Endpoints
+
+POST
+- `/api/ingest`: Store a run and, by default, trigger analysis (`analyze=false` to skip)
+- `/api/analyze/<id>`: Re-trigger analysis for an existing run
+
+GET
+- `/api/runs`: List runs (filter by pipeline/status)
+- `/api/runs/<id>`: Get a run with all steps
+- `/api/runs/<id>/analysis`: Get analysis only for a run
+- `/api/pipelines`: List pipelines
+- `/api/search/steps`: Search steps by name/pipeline
+- `/health`: Health check
+
+## Project Structure
+
+```
+├── xray_sdk/          # Python SDK
+│   ├── step.py        # XRayStep dataclass
+│   ├── run.py         # XRayRun with auto-summarization
+│   └── client.py      # HTTP client with spool fallback
+├── xray_api/          # Flask API
+│   ├── app.py         # Flask entry point
+│   ├── models.py      # Database models
+│   ├── routes/        # API endpoints
+│   └── agents/        # Cerebras AI analyzer
+├── examples/          # Example scripts
+├── ARCHITECTURE.md    # Detailed architecture doc
+└── requirements.txt   # Dependencies
+```
+
+## Features
+
+- **End-of-pipeline integration**: Add steps as your pipeline runs, send at the end
+- **Deterministic summarization**: Large outputs are summarized with head/tail sampling for reproducible debugging
+- **Spool fallback**: If API is down, saves to `.xray_spool/` for later submission
+- **Step intent hints**: Optional one-line descriptions per step improve semantic analysis
+- **Server-side safety net**: The API summarizes oversized inputs/outputs if a client skips SDK summarization
+- **AI-powered analysis**: Uses Cerebras LLM with a 2-step sliding window when needed to identify semantic mismatches and faulty steps
+
+## Approach
+
+The system is designed around these key principles:
+1. **Minimal Integration Burden**: The SDK requires only wrapping each step's inputs/outputs after execution. Users can enrich this data with optional descriptions for both the pipeline and individual steps, making the system extensible and allowing the AI to understand the intent behind any domain-specific logic without requiring code changes.
+
+2. **Sliding Window Analysis**: Instead of sending entire pipelines to the LLM (which can exceed token limits), we analyze 2 consecutive steps at a time. This keeps prompts under 65K tokens while still detecting data flow issues between adjacent steps.
+
+3. **Semantic Context via Descriptions**: Pipeline and step descriptions tell the LLM what *type* of pipeline (e-commerce, document processing, etc.) and what each step *should* do. This helps detect semantic mismatches beyond just structural data flow.
+
+4. **Deterministic Summarization**: Large outputs (500+ items) are summarized using head/tail sampling (first N + last N items). This is deterministic and preserves edge cases that often reveal bugs.
+
+5. **Graceful Degradation**: If the API is unavailable, runs are spooled locally and can be flushed later with `client.flush_spool()`.
+
+## Known Limitations
+
+- **No cross-window context**: When analyzing step 3→4, the LLM doesn't see steps 1→2. Issues that span multiple transitions may be missed, if they are not detected somehow at previous step.
+
+- **Single LLM provider**: Currently only supports Cerebras API. Other providers require code changes.
+
+- **Summarization loses detail**: Very large payloads are aggressively trimmed. Some bugs may be hidden in truncated data.
+
+## Future Improvements
+
+- **Docker image**: Pre-built container for one-command local setup - developers just run `docker compose up` instead of installing packages, databases, etc.
+- **Local LLM support**: Run lightweight local models (e.g., Ollama, llama.cpp) to eliminate third-party API dependency and reduce costs
+- **Multi-LLM support**: Add OpenAI, Anthropic, and other cloud providers via configurable adapters
+- **Pipeline-level summary**: Generate a one-pass summary of the entire pipeline before window analysis
+- **Streaming results**: Return partial analysis as each window completes
+- **Web dashboard**: Visual timeline of pipeline runs with highlighted faulty steps
+- **Comparison mode**: Compare two runs of the same pipeline to spot regressions

xray_sdk-0.1.0/README.md

@@ -0,0 +1,180 @@
+# X-Ray SDK and API
+
+A lightweight debugging system for multi-step AI pipelines that captures execution data and uses AI to identify faulty steps.
+
+## Quick Start
+
+### 1. Install Dependencies
+
+```bash
+pip3 install flask flask-sqlalchemy flask-cors psycopg2-binary openai python-dotenv requests
+```
+
+### 2. Configure Environment
+
+Create a `.env` file:
+```env
+DATABASE_URL=postgresql://user:pass@host/dbname
+CEREBRAS_API_KEY=your-api-key
+CEREBRAS_BASE_URL=https://api.cerebras.ai/v1
+CEREBRAS_MODEL=llama3.1-8b
+```
+
+### 3. Initialize Database
+
+```bash
+python3 -c "
+from dotenv import load_dotenv
+load_dotenv()
+from xray_api.app import create_app
+from xray_api.models import db
+
+app = create_app()
+with app.app_context():
+    db.create_all()
+    print('Database tables created!')
+"
+```
+
+### 4. Start the API Server
+
+```bash
+python3 -m xray_api.app
+```
+
+### 5. Run Example
+
+```bash
+python3 examples/amazon_competitor.py
+```
+
+## SDK Usage
+
+```python
+from xray_sdk import XRayClient, XRayRun, XRayStep
+
+# Create a run
+run = XRayRun("my_pipeline", metadata={"context": "test"}, sample_size=50)
+
+# Add steps (after your pipeline executes)
+run.add_step(XRayStep(
+    name="keyword_generation",
+    order=1,
+    inputs={"title": "Phone Case"},
+    outputs={"keywords": ["phone case", "iphone"]},
+    description="Generate search keywords from the title."# explain what this step does
+))
+
+run.add_step(XRayStep(
+    name="search",
+    order=2,
+    inputs={"keywords": ["phone case", "iphone"]},
+    outputs={"candidates_count": 100},
+    description="Search the catalog for items matching the keywords."
+))
+
+run.add_step(XRayStep(
+    name="filter",
+    order=3,
+    inputs={"candidates_count": 100},
+    outputs={"filtered_count": 5},
+    description="Filter candidates by rating.",
+    reasons={"dropped_items": [{"id": 123, "reason": "low rating"}]},
+    metrics={"elimination_rate": 0.95}
+))
+
+# Send for analysis
+client = XRayClient("http://localhost:5000")
+result = client.send(run)
+
+print(result["analysis"])
+# {
+#   "faulty_step": "keyword_generation",
+#   "reason": "...",
+#   "suggestion": "..."
+# }
+```
+
+## SDK Client Methods
+
+| Method | Description |
+|--------|-------------|
+| `send(run, analyze=True)` | Send run to API; spools locally if unavailable |
+| `spool(run)` | Manually save run to `.xray_spool/` |
+| `flush_spool()` | Send newest spooled run and delete all spool files |
+| `list_pipelines()` | List all pipelines |
+| `list_runs(pipeline, status, limit)` | List runs with filters |
+| `get_run(run_id)` | Get run with all steps |
+| `get_analysis(run_id)` | Get analysis result only |
+| `search_steps(step_name, pipeline, limit)` | Search steps across runs |
+
+## API Endpoints
+
+POST
+- `/api/ingest`: Store a run and, by default, trigger analysis (`analyze=false` to skip)
+- `/api/analyze/<id>`: Re-trigger analysis for an existing run
+
+GET
+- `/api/runs`: List runs (filter by pipeline/status)
+- `/api/runs/<id>`: Get a run with all steps
+- `/api/runs/<id>/analysis`: Get analysis only for a run
+- `/api/pipelines`: List pipelines
+- `/api/search/steps`: Search steps by name/pipeline
+- `/health`: Health check
+
+## Project Structure
+
+```
+├── xray_sdk/          # Python SDK
+│   ├── step.py        # XRayStep dataclass
+│   ├── run.py         # XRayRun with auto-summarization
+│   └── client.py      # HTTP client with spool fallback
+├── xray_api/          # Flask API
+│   ├── app.py         # Flask entry point
+│   ├── models.py      # Database models
+│   ├── routes/        # API endpoints
+│   └── agents/        # Cerebras AI analyzer
+├── examples/          # Example scripts
+├── ARCHITECTURE.md    # Detailed architecture doc
+└── requirements.txt   # Dependencies
+```
+
+## Features
+
+- **End-of-pipeline integration**: Add steps as your pipeline runs, send at the end
+- **Deterministic summarization**: Large outputs are summarized with head/tail sampling for reproducible debugging
+- **Spool fallback**: If API is down, saves to `.xray_spool/` for later submission
+- **Step intent hints**: Optional one-line descriptions per step improve semantic analysis
+- **Server-side safety net**: The API summarizes oversized inputs/outputs if a client skips SDK summarization
+- **AI-powered analysis**: Uses Cerebras LLM with a 2-step sliding window when needed to identify semantic mismatches and faulty steps
+
+## Approach
+
+The system is designed around these key principles:
+1. **Minimal Integration Burden**: The SDK requires only wrapping each step's inputs/outputs after execution. Users can enrich this data with optional descriptions for both the pipeline and individual steps, making the system extensible and allowing the AI to understand the intent behind any domain-specific logic without requiring code changes.
+
+2. **Sliding Window Analysis**: Instead of sending entire pipelines to the LLM (which can exceed token limits), we analyze 2 consecutive steps at a time. This keeps prompts under 65K tokens while still detecting data flow issues between adjacent steps.
+
+3. **Semantic Context via Descriptions**: Pipeline and step descriptions tell the LLM what *type* of pipeline (e-commerce, document processing, etc.) and what each step *should* do. This helps detect semantic mismatches beyond just structural data flow.
+
+4. **Deterministic Summarization**: Large outputs (500+ items) are summarized using head/tail sampling (first N + last N items). This is deterministic and preserves edge cases that often reveal bugs.
+
+5. **Graceful Degradation**: If the API is unavailable, runs are spooled locally and can be flushed later with `client.flush_spool()`.
+
+## Known Limitations
+
+- **No cross-window context**: When analyzing step 3→4, the LLM doesn't see steps 1→2. Issues that span multiple transitions may be missed, if they are not detected somehow at previous step.
+
+- **Single LLM provider**: Currently only supports Cerebras API. Other providers require code changes.
+
+- **Summarization loses detail**: Very large payloads are aggressively trimmed. Some bugs may be hidden in truncated data.
+
+## Future Improvements
+
+- **Docker image**: Pre-built container for one-command local setup - developers just run `docker compose up` instead of installing packages, databases, etc.
+- **Local LLM support**: Run lightweight local models (e.g., Ollama, llama.cpp) to eliminate third-party API dependency and reduce costs
+- **Multi-LLM support**: Add OpenAI, Anthropic, and other cloud providers via configurable adapters
+- **Pipeline-level summary**: Generate a one-pass summary of the entire pipeline before window analysis
+- **Streaming results**: Return partial analysis as each window completes
+- **Web dashboard**: Visual timeline of pipeline runs with highlighted faulty steps
+- **Comparison mode**: Compare two runs of the same pipeline to spot regressions

xray_sdk-0.1.0/pyproject.toml

@@ -0,0 +1,18 @@
+[build-system]
+requires = ["setuptools>=69.0", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "xray-sdk"
+version = "0.1.0"
+description = "Lightweight debugging SDK for multi-step AI pipelines."
+readme = "README.md"
+requires-python = ">=3.9"
+license = { text = "Proprietary" }
+authors = [{ name = "Equal Collective" }]
+dependencies = [
+  "requests>=2.31.0",
+]
+
+[tool.setuptools.packages.find]
+include = ["xray_sdk"]

xray_sdk-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

xray_sdk-0.1.0/xray_sdk/__init__.py

@@ -0,0 +1,10 @@
+"""
+X-Ray SDK - Lightweight debugging for multi-step AI pipelines
+"""
+
+from .step import XRayStep
+from .run import XRayRun
+from .client import XRayClient
+
+__all__ = ["XRayStep", "XRayRun", "XRayClient"]
+__version__ = "0.1.0"

xray_sdk-0.1.0/xray_sdk/client.py

@@ -0,0 +1,198 @@
+"""
+XRayClient - Sends run data to the X-Ray API for analysis
+"""
+
+import os
+import json
+import requests
+from pathlib import Path
+from typing import Dict, Any, Optional
+from .run import XRayRun
+
+
+class XRayClient:
+    """
+    Client for sending pipeline runs to the X-Ray API.
+    
+    Features:
+    - Sends run data to API for AI-powered analysis
+    - Spools to local file if API is unavailable
+    - Supports API key authentication
+    """
+    
+    DEFAULT_SPOOL_DIR = ".xray_spool"
+    
+    def __init__(self, api_url: str, api_key: Optional[str] = None, timeout: int = 180):
+        """
+        Initialize the X-Ray client.
+        
+        Args:
+            api_url: Base URL of the X-Ray API (e.g., "http://localhost:5000")
+            api_key: Optional API key for authentication (required if server has XRAY_API_KEY set)
+            timeout: Request timeout in seconds (default: 180 for LLM analysis)
+        """
+        self.api_url = api_url.rstrip("/")
+        self.api_key = api_key
+        self.timeout = timeout
+    
+    def _headers(self) -> Dict[str, str]:
+        """Build headers for API requests."""
+        headers = {"Content-Type": "application/json"}
+        if self.api_key:
+            headers["X-API-Key"] = self.api_key
+        return headers
+    
+    def send(self, run: XRayRun, analyze: bool = True) -> Dict[str, Any]:
+        """
+        Send a run to the X-Ray API for storage and optional analysis.
+        
+        Args:
+            run: The XRayRun to send
+            analyze: Whether to trigger AI analysis (default: True)
+            
+        Returns:
+            API response with run_id and analysis result (if requested)
+            
+        Raises:
+            requests.exceptions.RequestException: If API call fails
+        """
+        payload = run.to_dict()
+        payload["analyze"] = analyze
+        
+        try:
+            response = requests.post(
+                f"{self.api_url}/api/ingest",
+                json=payload,
+                headers=self._headers(),
+                timeout=self.timeout
+            )
+            response.raise_for_status()
+            return response.json()
+        except requests.exceptions.RequestException as e:
+            # Spool locally if API unavailable
+            spool_path = self.spool(run)
+            return {
+                "error": str(e),
+                "spooled": True,
+                "spool_path": str(spool_path)
+            }
+    
+    def spool(self, run: XRayRun, spool_dir: Optional[str] = None) -> Path:
+        """
+        Save run data to local file for later submission.
+        
+        Args:
+            run: The XRayRun to spool
+            spool_dir: Directory to save files (default: .xray_spool)
+            
+        Returns:
+            Path to the spooled file
+        """
+        spool_dir = Path(spool_dir or self.DEFAULT_SPOOL_DIR)
+        spool_dir.mkdir(parents=True, exist_ok=True)
+        
+        # Generate filename with timestamp
+        import datetime
+        timestamp = datetime.datetime.now().strftime("%Y%m%d_%H%M%S_%f")
+        filename = f"{run.pipeline_name}_{timestamp}.json"
+        filepath = spool_dir / filename
+        
+        with open(filepath, "w") as f:
+            json.dump(run.to_dict(), f, indent=2, default=str)
+        
+        return filepath
+
+    def list_pipelines(self) -> Dict[str, Any]:
+        """List all pipelines."""
+        response = requests.get(f"{self.api_url}/api/pipelines", headers=self._headers(), timeout=self.timeout)
+        response.raise_for_status()
+        return response.json()
+
+    def list_runs(
+        self,
+        pipeline: Optional[str] = None,
+        status: Optional[str] = None,
+        limit: int = 50,
+    ) -> Dict[str, Any]:
+        """List runs with optional filters."""
+        params = {"limit": limit}
+        if pipeline:
+            params["pipeline"] = pipeline
+        if status:
+            params["status"] = status
+        response = requests.get(f"{self.api_url}/api/runs", params=params, headers=self._headers(), timeout=self.timeout)
+        response.raise_for_status()
+        return response.json()
+
+    def get_run(self, run_id: str) -> Dict[str, Any]:
+        """Get a single run with all its steps."""
+        response = requests.get(f"{self.api_url}/api/runs/{run_id}", headers=self._headers(), timeout=self.timeout)
+        response.raise_for_status()
+        return response.json()
+
+    def get_analysis(self, run_id: str) -> Dict[str, Any]:
+        """Get analysis result for a run."""
+        response = requests.get(f"{self.api_url}/api/runs/{run_id}/analysis", headers=self._headers(), timeout=self.timeout)
+        response.raise_for_status()
+        return response.json()
+
+    def search_steps(
+        self,
+        step_name: Optional[str] = None,
+        pipeline: Optional[str] = None,
+        limit: int = 50,
+    ) -> Dict[str, Any]:
+        """Search steps across runs."""
+        params = {"limit": limit}
+        if step_name:
+            params["step_name"] = step_name
+        if pipeline:
+            params["pipeline"] = pipeline
+        response = requests.get(f"{self.api_url}/api/search/steps", params=params, headers=self._headers(), timeout=self.timeout)
+        response.raise_for_status()
+        return response.json()
+    
+    def flush_spool(self, spool_dir: Optional[str] = None) -> Dict[str, Any]:
+        """
+        Send the newest spooled run to the API and delete all spooled files.
+        
+        Args:
+            spool_dir: Directory containing spooled files
+            
+        Returns:
+            Summary of flush results
+        """
+        spool_dir = Path(spool_dir or self.DEFAULT_SPOOL_DIR)
+        if not spool_dir.exists():
+            return {"flushed": 0, "failed": 0}
+
+        files = list(spool_dir.glob("*.json"))
+        if not files:
+            return {"flushed": 0, "failed": 0}
+
+        newest = max(files, key=lambda p: p.stat().st_mtime)
+        results = {"flushed": 0, "failed": 0, "errors": [], "sent_file": str(newest)}
+
+        try:
+            with open(newest) as f:
+                data = json.load(f)
+
+            response = requests.post(
+                f"{self.api_url}/api/ingest",
+                json=data,
+                headers=self._headers(),
+                timeout=self.timeout
+            )
+            response.raise_for_status()
+            response_json = response.json()
+            results["flushed"] = 1
+            results["response"] = response_json
+
+            # Delete all spooled files after successful send of newest.
+            for filepath in files:
+                filepath.unlink()
+        except Exception as e:
+            results["failed"] = 1
+            results["errors"].append({"file": str(newest), "error": str(e)})
+
+        return results

xray_sdk-0.1.0/xray_sdk/run.py

@@ -0,0 +1,128 @@
+"""
+XRayRun - Represents a complete pipeline execution with multiple steps
+"""
+
+import json
+from typing import List, Dict, Any, Optional
+from .step import XRayStep
+
+
+class XRayRun:
+    """
+    A complete run of a pipeline, containing multiple steps.
+    
+    Automatically summarizes large inputs/outputs to prevent token limit issues.
+    """
+    
+    MAX_PAYLOAD_SIZE = 80000  # chars per step side (~20K tokens) - 2 steps = ~40K tokens, safely under 65K limit
+    SAMPLE_SIZE = 100         # initial sample size per large list
+    MIN_SAMPLE_SIZE = 10      # floor for aggressive trimming when still oversized
+    STRING_TRUNCATE = 2000    # truncate very long strings to this many chars
+    
+    def __init__(
+        self,
+        pipeline_name: str,
+        description: Optional[str] = None,
+        metadata: Optional[Dict[str, Any]] = None,
+        sample_size: Optional[int] = None,
+    ):
+        """
+        Initialize a new run.
+        
+        Args:
+            pipeline_name: Name of the pipeline (e.g., "competitor_selection")
+            description: Optional description of what this pipeline does (helps AI analysis)
+            metadata: Optional metadata about this run (e.g., {"product_id": "123"})
+            sample_size: Optional override for summarization sample size
+        """
+        self.pipeline_name = pipeline_name
+        self.description = description or ""
+        self.metadata = metadata or {}
+        if sample_size is None:
+            self.sample_size = self.SAMPLE_SIZE
+        else:
+            self.sample_size = max(1, sample_size)
+        self.steps: List[XRayStep] = []
+    
+    def add_step(self, step: XRayStep) -> None:
+        """
+        Add a step to this run. Auto-summarizes large outputs.
+        
+        Args:
+            step: The XRayStep to add
+        """
+        step.inputs = self._ensure_within_budget(step.inputs)
+        step.outputs = self._ensure_within_budget(step.outputs)
+        
+        self.steps.append(step)
+
+    def _ensure_within_budget(self, data: Any) -> Any:
+        """Summarize data if it exceeds MAX_PAYLOAD_SIZE."""
+        if data is None:
+            return {}
+        try:
+            size = len(json.dumps(data, default=str))
+        except Exception:
+            size = self.MAX_PAYLOAD_SIZE + 1  # force summarization if not serializable
+        if size <= self.MAX_PAYLOAD_SIZE:
+            return data
+        # Log summarization
+        print(f"   [SDK] Summarizing large payload: {size} chars -> MAX {self.MAX_PAYLOAD_SIZE} chars")
+        summarized = self._summarize_with_budget(data)
+        new_size = len(json.dumps(summarized, default=str))
+        print(f"   [SDK] Summarization complete: {size} -> {new_size} chars")
+        return summarized
+
+    def _summarize_with_budget(self, data: Any) -> Any:
+        """Iteratively summarize until payload fits under MAX_PAYLOAD_SIZE."""
+        sample_size = self.sample_size
+        summarized = data
+        while True:
+            summarized = self._summarize_once(summarized, sample_size)
+            size = len(json.dumps(summarized, default=str))
+            if size <= self.MAX_PAYLOAD_SIZE or sample_size <= self.MIN_SAMPLE_SIZE:
+                return summarized
+            sample_size = max(self.MIN_SAMPLE_SIZE, sample_size // 2)
+    
+    def _summarize_once(self, data: Any, sample_size: int) -> Any:
+        """One-pass summarization with recursion and string truncation."""
+        if isinstance(data, dict):
+            summarized = {}
+            for key, value in data.items():
+                if isinstance(value, list):
+                    summarized_list, total_count = self._summarize_list(value, sample_size)
+                    summarized[key] = summarized_list
+                    if total_count is not None:
+                        summarized[f"{key}_total_count"] = total_count
+                else:
+                    summarized[key] = self._summarize_once(value, sample_size)
+            return summarized
+        if isinstance(data, list):
+            summarized_list, _ = self._summarize_list(data, sample_size)
+            return summarized_list
+        if isinstance(data, str) and len(data) > self.STRING_TRUNCATE:
+            overflow = len(data) - self.STRING_TRUNCATE
+            return f"{data[:self.STRING_TRUNCATE]}...[truncated {overflow} chars]"
+        return data
+    
+    def _summarize_list(self, items: List[Any], sample_size: int):
+        """Summarize a list: sample if large, recurse into elements."""
+        total_count = None
+        if len(items) > sample_size:
+            total_count = len(items)
+            head_count = sample_size // 2
+            tail_count = sample_size - head_count
+            items = items[:head_count] + items[-tail_count:]
+        return [self._summarize_once(item, sample_size) for item in items], total_count
+    
+    def to_dict(self) -> Dict[str, Any]:
+        """Convert run to dictionary for JSON serialization"""
+        return {
+            "pipeline_name": self.pipeline_name,
+            "pipeline_description": self.description,
+            "metadata": self.metadata,
+            "steps": [step.to_dict() for step in self.steps]
+        }
+    
+    def __repr__(self) -> str:
+        return f"XRayRun(pipeline='{self.pipeline_name}', steps={len(self.steps)})"

xray_sdk-0.1.0/xray_sdk/step.py

@@ -0,0 +1,36 @@
+"""
+XRayStep - Represents a single step in a pipeline execution
+"""
+
+from dataclasses import dataclass, field, asdict
+from typing import Dict, Any
+
+
+@dataclass
+class XRayStep:
+    """
+    A single step in a pipeline execution.
+    
+    Attributes:
+        name: Step identifier (e.g., "keyword_generation", "filter", "rank")
+        order: Step sequence number (1, 2, 3, ...)
+        inputs: What was fed to this step (any JSON-serializable dict)
+        outputs: What this step produced (any JSON-serializable dict)
+        description: Optional one-line summary of the step's intent
+        reasons: Optional dict for rejections or drops
+        metrics: Optional dict for step-level metrics
+    """
+    name: str
+    order: int
+    inputs: Dict[str, Any] = field(default_factory=dict)
+    outputs: Dict[str, Any] = field(default_factory=dict)
+    description: str = ""
+    reasons: Dict[str, Any] = field(default_factory=dict)
+    metrics: Dict[str, Any] = field(default_factory=dict)
+    
+    def to_dict(self) -> Dict[str, Any]:
+        """Convert step to dictionary for JSON serialization"""
+        return asdict(self)
+    
+    def __repr__(self) -> str:
+        return f"XRayStep(name='{self.name}', order={self.order})"

xray_sdk-0.1.0/xray_sdk.egg-info/PKG-INFO

@@ -0,0 +1,190 @@
+Metadata-Version: 2.4
+Name: xray-sdk
+Version: 0.1.0
+Summary: Lightweight debugging SDK for multi-step AI pipelines.
+Author: Equal Collective
+License: Proprietary
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+Requires-Dist: requests>=2.31.0
+
+# X-Ray SDK and API
+
+A lightweight debugging system for multi-step AI pipelines that captures execution data and uses AI to identify faulty steps.
+
+## Quick Start
+
+### 1. Install Dependencies
+
+```bash
+pip3 install flask flask-sqlalchemy flask-cors psycopg2-binary openai python-dotenv requests
+```
+
+### 2. Configure Environment
+
+Create a `.env` file:
+```env
+DATABASE_URL=postgresql://user:pass@host/dbname
+CEREBRAS_API_KEY=your-api-key
+CEREBRAS_BASE_URL=https://api.cerebras.ai/v1
+CEREBRAS_MODEL=llama3.1-8b
+```
+
+### 3. Initialize Database
+
+```bash
+python3 -c "
+from dotenv import load_dotenv
+load_dotenv()
+from xray_api.app import create_app
+from xray_api.models import db
+
+app = create_app()
+with app.app_context():
+    db.create_all()
+    print('Database tables created!')
+"
+```
+
+### 4. Start the API Server
+
+```bash
+python3 -m xray_api.app
+```
+
+### 5. Run Example
+
+```bash
+python3 examples/amazon_competitor.py
+```
+
+## SDK Usage
+
+```python
+from xray_sdk import XRayClient, XRayRun, XRayStep
+
+# Create a run
+run = XRayRun("my_pipeline", metadata={"context": "test"}, sample_size=50)
+
+# Add steps (after your pipeline executes)
+run.add_step(XRayStep(
+    name="keyword_generation",
+    order=1,
+    inputs={"title": "Phone Case"},
+    outputs={"keywords": ["phone case", "iphone"]},
+    description="Generate search keywords from the title."# explain what this step does
+))
+
+run.add_step(XRayStep(
+    name="search",
+    order=2,
+    inputs={"keywords": ["phone case", "iphone"]},
+    outputs={"candidates_count": 100},
+    description="Search the catalog for items matching the keywords."
+))
+
+run.add_step(XRayStep(
+    name="filter",
+    order=3,
+    inputs={"candidates_count": 100},
+    outputs={"filtered_count": 5},
+    description="Filter candidates by rating.",
+    reasons={"dropped_items": [{"id": 123, "reason": "low rating"}]},
+    metrics={"elimination_rate": 0.95}
+))
+
+# Send for analysis
+client = XRayClient("http://localhost:5000")
+result = client.send(run)
+
+print(result["analysis"])
+# {
+#   "faulty_step": "keyword_generation",
+#   "reason": "...",
+#   "suggestion": "..."
+# }
+```
+
+## SDK Client Methods
+
+| Method | Description |
+|--------|-------------|
+| `send(run, analyze=True)` | Send run to API; spools locally if unavailable |
+| `spool(run)` | Manually save run to `.xray_spool/` |
+| `flush_spool()` | Send newest spooled run and delete all spool files |
+| `list_pipelines()` | List all pipelines |
+| `list_runs(pipeline, status, limit)` | List runs with filters |
+| `get_run(run_id)` | Get run with all steps |
+| `get_analysis(run_id)` | Get analysis result only |
+| `search_steps(step_name, pipeline, limit)` | Search steps across runs |
+
+## API Endpoints
+
+POST
+- `/api/ingest`: Store a run and, by default, trigger analysis (`analyze=false` to skip)
+- `/api/analyze/<id>`: Re-trigger analysis for an existing run
+
+GET
+- `/api/runs`: List runs (filter by pipeline/status)
+- `/api/runs/<id>`: Get a run with all steps
+- `/api/runs/<id>/analysis`: Get analysis only for a run
+- `/api/pipelines`: List pipelines
+- `/api/search/steps`: Search steps by name/pipeline
+- `/health`: Health check
+
+## Project Structure
+
+```
+├── xray_sdk/          # Python SDK
+│   ├── step.py        # XRayStep dataclass
+│   ├── run.py         # XRayRun with auto-summarization
+│   └── client.py      # HTTP client with spool fallback
+├── xray_api/          # Flask API
+│   ├── app.py         # Flask entry point
+│   ├── models.py      # Database models
+│   ├── routes/        # API endpoints
+│   └── agents/        # Cerebras AI analyzer
+├── examples/          # Example scripts
+├── ARCHITECTURE.md    # Detailed architecture doc
+└── requirements.txt   # Dependencies
+```
+
+## Features
+
+- **End-of-pipeline integration**: Add steps as your pipeline runs, send at the end
+- **Deterministic summarization**: Large outputs are summarized with head/tail sampling for reproducible debugging
+- **Spool fallback**: If API is down, saves to `.xray_spool/` for later submission
+- **Step intent hints**: Optional one-line descriptions per step improve semantic analysis
+- **Server-side safety net**: The API summarizes oversized inputs/outputs if a client skips SDK summarization
+- **AI-powered analysis**: Uses Cerebras LLM with a 2-step sliding window when needed to identify semantic mismatches and faulty steps
+
+## Approach
+
+The system is designed around these key principles:
+1. **Minimal Integration Burden**: The SDK requires only wrapping each step's inputs/outputs after execution. Users can enrich this data with optional descriptions for both the pipeline and individual steps, making the system extensible and allowing the AI to understand the intent behind any domain-specific logic without requiring code changes.
+
+2. **Sliding Window Analysis**: Instead of sending entire pipelines to the LLM (which can exceed token limits), we analyze 2 consecutive steps at a time. This keeps prompts under 65K tokens while still detecting data flow issues between adjacent steps.
+
+3. **Semantic Context via Descriptions**: Pipeline and step descriptions tell the LLM what *type* of pipeline (e-commerce, document processing, etc.) and what each step *should* do. This helps detect semantic mismatches beyond just structural data flow.
+
+4. **Deterministic Summarization**: Large outputs (500+ items) are summarized using head/tail sampling (first N + last N items). This is deterministic and preserves edge cases that often reveal bugs.
+
+5. **Graceful Degradation**: If the API is unavailable, runs are spooled locally and can be flushed later with `client.flush_spool()`.
+
+## Known Limitations
+
+- **No cross-window context**: When analyzing step 3→4, the LLM doesn't see steps 1→2. Issues that span multiple transitions may be missed, if they are not detected somehow at previous step.
+
+- **Single LLM provider**: Currently only supports Cerebras API. Other providers require code changes.
+
+- **Summarization loses detail**: Very large payloads are aggressively trimmed. Some bugs may be hidden in truncated data.
+
+## Future Improvements
+
+- **Docker image**: Pre-built container for one-command local setup - developers just run `docker compose up` instead of installing packages, databases, etc.
+- **Local LLM support**: Run lightweight local models (e.g., Ollama, llama.cpp) to eliminate third-party API dependency and reduce costs
+- **Multi-LLM support**: Add OpenAI, Anthropic, and other cloud providers via configurable adapters
+- **Pipeline-level summary**: Generate a one-pass summary of the entire pipeline before window analysis
+- **Streaming results**: Return partial analysis as each window completes
+- **Web dashboard**: Visual timeline of pipeline runs with highlighted faulty steps
+- **Comparison mode**: Compare two runs of the same pipeline to spot regressions

xray_sdk-0.1.0/xray_sdk.egg-info/SOURCES.txt

@@ -0,0 +1,11 @@
+README.md
+pyproject.toml
+xray_sdk/__init__.py
+xray_sdk/client.py
+xray_sdk/run.py
+xray_sdk/step.py
+xray_sdk.egg-info/PKG-INFO
+xray_sdk.egg-info/SOURCES.txt
+xray_sdk.egg-info/dependency_links.txt
+xray_sdk.egg-info/requires.txt
+xray_sdk.egg-info/top_level.txt

xray_sdk-0.1.0/xray_sdk.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

xray_sdk-0.1.0/xray_sdk.egg-info/requires.txt

@@ -0,0 +1 @@
+requests>=2.31.0

xray_sdk-0.1.0/xray_sdk.egg-info/top_level.txt

@@ -0,0 +1 @@
+xray_sdk