fluxloop 0.1.4__tar.gz

fluxloop-0.1.4/PKG-INFO

@@ -0,0 +1,138 @@
+Metadata-Version: 2.4
+Name: fluxloop
+Version: 0.1.4
+Summary: FluxLoop SDK for agent instrumentation and tracing
+Author-email: FluxLoop Team <team@fluxloop.dev>
+License: Apache-2.0
+Project-URL: Homepage, https://github.com/chuckgu/fluxloop
+Project-URL: Documentation, https://docs.fluxloop.dev
+Project-URL: Repository, https://github.com/chuckgu/fluxloop
+Project-URL: Issues, https://github.com/chuckgu/fluxloop/issues
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: Apache Software License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+Requires-Dist: pydantic>=2.0
+Requires-Dist: httpx>=0.24.0
+Requires-Dist: python-dotenv>=1.0.0
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0; extra == "dev"
+Requires-Dist: pytest-asyncio>=0.21.0; extra == "dev"
+Requires-Dist: pytest-cov>=4.0; extra == "dev"
+Requires-Dist: pytest-mock>=3.14.0; extra == "dev"
+Requires-Dist: pexpect>=4.9.0; extra == "dev"
+Requires-Dist: ruff>=0.1.0; extra == "dev"
+Requires-Dist: mypy>=1.0; extra == "dev"
+Requires-Dist: black>=23.0; extra == "dev"
+Provides-Extra: langchain
+Requires-Dist: langchain>=0.1.0; extra == "langchain"
+Provides-Extra: langgraph
+Requires-Dist: langgraph>=0.0.20; extra == "langgraph"
+
+# FluxLoop SDK
+
+FluxLoop SDK for agent instrumentation and tracing.
+
+## Installation
+
+```bash
+pip install fluxloop
+```
+
+## Quick Start
+
+```python
+from fluxloop import trace, FluxLoopClient
+
+# Initialize the client
+client = FluxLoopClient()
+
+# Use the trace decorator
+@trace()
+def my_agent_function(prompt: str):
+    # Your agent logic here
+    return result
+```
+
+## Features
+
+- 🔍 **Automatic Tracing**: Instrument your agent code with simple decorators
+- 📊 **Rich Context**: Capture inputs, outputs, and metadata
+- 🔄 **Async Support**: Works with both sync and async functions
+- 🎯 **Framework Integration**: Built-in support for LangChain and LangGraph
+
+## Documentation
+
+For detailed documentation, visit [https://docs.fluxloop.dev](https://docs.fluxloop.dev)
+
+## License
+
+Apache License 2.0 - see LICENSE file for details
+
+
+## Framework Integration: Decorator Ordering and Safe Instrumentation
+
+When integrating FluxLoop with external agent frameworks (e.g., ChatKit, LangChain), follow these rules to avoid type conflicts and ensure observations are captured reliably.
+
+- Outermost framework wrapper: If a framework provides its own decorator/wrapper that transforms a plain function into a framework-specific object (e.g., a Tool), that decorator MUST be the outermost (top) decorator. This preserves the type the framework expects.
+- FluxLoop instrumentation inside: Place FluxLoop decorators inside (below) the framework decorator, or instrument from within the function body using the SDK context APIs.
+
+Two safe patterns:
+
+- Pattern A (safest, framework-agnostic): instrument inside the function body
+  - Use `get_current_context()` and push/pop an `ObservationData` manually around your logic. This keeps signatures and framework typing unchanged.
+  - Example (tool function):
+
+    ```python
+    from fluxloop import get_current_context
+    from fluxloop.models import ObservationData, ObservationType
+
+    async def my_tool(param: str) -> dict:
+        fl_ctx = get_current_context()
+        obs = None
+        if fl_ctx and fl_ctx.is_enabled():
+            obs = ObservationData(
+                type=ObservationType.TOOL,
+                name="tool.my_tool",
+                input={"args": {"param": param}},
+            )
+            fl_ctx.push_observation(obs)
+
+        try:
+            result = {"result": do_work(param)}
+            if obs:
+                obs.output = result
+            return result
+        except Exception as e:
+            if obs:
+                obs.error = str(e)
+            raise
+        finally:
+            if fl_ctx and obs:
+                fl_ctx.pop_observation()
+    ```
+
+- Pattern B (stacking decorators): framework outermost, FluxLoop inside
+  - Example with a framework tool decorator:
+
+    ```python
+    @framework_tool_decorator(...)
+    @fluxloop.tool(name="tool.my_tool")
+    async def my_tool(...):
+        ...
+    ```
+
+  - Important: If you reverse the order (FluxLoop outside), the framework may see a plain function instead of its expected type and raise errors like "Unknown tool type".
+
+LLM/streaming calls:
+
+- For LLM calls (including async generators/streams), either:
+  - Wrap the call site in a small helper decorated with `@fluxloop.prompt(...)`, or
+  - Use `with fluxloop.instrument("prompt.name"):` around the portion that produces model output, ensuring it runs inside the current FluxLoop context.
+
+These patterns guarantee observations are captured (`tool`, `generation`) while keeping the external framework’s type system intact.
+

fluxloop-0.1.4/README.md

@@ -0,0 +1,103 @@
+# FluxLoop SDK
+
+FluxLoop SDK for agent instrumentation and tracing.
+
+## Installation
+
+```bash
+pip install fluxloop
+```
+
+## Quick Start
+
+```python
+from fluxloop import trace, FluxLoopClient
+
+# Initialize the client
+client = FluxLoopClient()
+
+# Use the trace decorator
+@trace()
+def my_agent_function(prompt: str):
+    # Your agent logic here
+    return result
+```
+
+## Features
+
+- 🔍 **Automatic Tracing**: Instrument your agent code with simple decorators
+- 📊 **Rich Context**: Capture inputs, outputs, and metadata
+- 🔄 **Async Support**: Works with both sync and async functions
+- 🎯 **Framework Integration**: Built-in support for LangChain and LangGraph
+
+## Documentation
+
+For detailed documentation, visit [https://docs.fluxloop.dev](https://docs.fluxloop.dev)
+
+## License
+
+Apache License 2.0 - see LICENSE file for details
+
+
+## Framework Integration: Decorator Ordering and Safe Instrumentation
+
+When integrating FluxLoop with external agent frameworks (e.g., ChatKit, LangChain), follow these rules to avoid type conflicts and ensure observations are captured reliably.
+
+- Outermost framework wrapper: If a framework provides its own decorator/wrapper that transforms a plain function into a framework-specific object (e.g., a Tool), that decorator MUST be the outermost (top) decorator. This preserves the type the framework expects.
+- FluxLoop instrumentation inside: Place FluxLoop decorators inside (below) the framework decorator, or instrument from within the function body using the SDK context APIs.
+
+Two safe patterns:
+
+- Pattern A (safest, framework-agnostic): instrument inside the function body
+  - Use `get_current_context()` and push/pop an `ObservationData` manually around your logic. This keeps signatures and framework typing unchanged.
+  - Example (tool function):
+
+    ```python
+    from fluxloop import get_current_context
+    from fluxloop.models import ObservationData, ObservationType
+
+    async def my_tool(param: str) -> dict:
+        fl_ctx = get_current_context()
+        obs = None
+        if fl_ctx and fl_ctx.is_enabled():
+            obs = ObservationData(
+                type=ObservationType.TOOL,
+                name="tool.my_tool",
+                input={"args": {"param": param}},
+            )
+            fl_ctx.push_observation(obs)
+
+        try:
+            result = {"result": do_work(param)}
+            if obs:
+                obs.output = result
+            return result
+        except Exception as e:
+            if obs:
+                obs.error = str(e)
+            raise
+        finally:
+            if fl_ctx and obs:
+                fl_ctx.pop_observation()
+    ```
+
+- Pattern B (stacking decorators): framework outermost, FluxLoop inside
+  - Example with a framework tool decorator:
+
+    ```python
+    @framework_tool_decorator(...)
+    @fluxloop.tool(name="tool.my_tool")
+    async def my_tool(...):
+        ...
+    ```
+
+  - Important: If you reverse the order (FluxLoop outside), the framework may see a plain function instead of its expected type and raise errors like "Unknown tool type".
+
+LLM/streaming calls:
+
+- For LLM calls (including async generators/streams), either:
+  - Wrap the call site in a small helper decorated with `@fluxloop.prompt(...)`, or
+  - Use `with fluxloop.instrument("prompt.name"):` around the portion that produces model output, ensuring it runs inside the current FluxLoop context.
+
+These patterns guarantee observations are captured (`tool`, `generation`) while keeping the external framework’s type system intact.
+

fluxloop-0.1.4/fluxloop/__init__.py

@@ -0,0 +1,64 @@
+"""
+FluxLoop SDK - Agent instrumentation and tracing library.
+"""
+
+from .context import FluxLoopContext, get_current_context, instrument
+from .decorators import agent, prompt, tool
+from .schemas import (
+    ExperimentConfig,
+    PersonaConfig,
+    RunnerConfig,
+    VariationStrategy,
+    Trace,
+    Observation,
+    ObservationType,
+    ObservationLevel,
+    Score,
+    ScoreDataType,
+    TraceStatus,
+)
+from .client import FluxLoopClient
+from .config import configure, get_config, reset_config, load_env
+from .recording import (
+    disable_recording,
+    enable_recording,
+    record_call_args,
+    set_recording_options,
+)
+
+__version__ = "0.1.4"
+
+__all__ = [
+    # Decorators
+    "agent",
+    "prompt",
+    "tool",
+    # Context
+    "instrument",
+    "get_current_context",
+    "FluxLoopContext",
+    # Client
+    "FluxLoopClient",
+    # Config
+    "configure",
+    "load_env",
+    "get_config",
+    "reset_config",
+    "enable_recording",
+    "disable_recording",
+    "set_recording_options",
+    "record_call_args",
+    # Schemas - configs
+    "ExperimentConfig",
+    "PersonaConfig",
+    "RunnerConfig",
+    "VariationStrategy",
+    # Schemas - tracing
+    "Trace",
+    "Observation",
+    "ObservationType",
+    "ObservationLevel",
+    "Score",
+    "ScoreDataType",
+    "TraceStatus",
+]

fluxloop-0.1.4/fluxloop/buffer.py

@@ -0,0 +1,184 @@
+"""
+Event buffering and batch sending logic.
+"""
+
+import atexit
+import threading
+import time
+from collections import deque
+from typing import Deque, List, Optional, Tuple
+from uuid import UUID
+
+from .config import get_config
+from .models import ObservationData, TraceData
+from .storage import OfflineStore
+
+
+class EventBuffer:
+    """
+    Singleton buffer for collecting and batching events.
+    """
+
+    _instance: Optional["EventBuffer"] = None
+    _lock = threading.Lock()
+
+    def __init__(self) -> None:
+        """Initialize the buffer."""
+        if EventBuffer._instance is not None:
+            raise RuntimeError("Use EventBuffer.get_instance() instead")
+
+        self.config = get_config()
+        self.traces: Deque[TraceData] = deque(maxlen=self.config.max_queue_size)
+        self.observations: Deque[Tuple[UUID, ObservationData]] = deque(
+            maxlen=self.config.max_queue_size
+        )
+
+        # Threading
+        self.send_lock = threading.Lock()
+        self.last_flush = time.time()
+
+        # Background thread for periodic flushing
+        self.stop_event = threading.Event()
+        self.flush_thread = threading.Thread(
+            target=self._flush_periodically, daemon=True
+        )
+        self.flush_thread.start()
+
+        # Register cleanup on exit
+        atexit.register(self.shutdown)
+
+        # Offline store
+        self.offline_store = OfflineStore()
+
+    @classmethod
+    def get_instance(cls) -> "EventBuffer":
+        """Get or create the singleton instance."""
+        if cls._instance is None:
+            with cls._lock:
+                if cls._instance is None:
+                    cls._instance = cls()
+        return cls._instance
+
+    def add_trace(self, trace: TraceData) -> None:
+        """
+        Add a trace to the buffer.
+
+        Args:
+            trace: Trace data to add
+        """
+        if not self.config.enabled:
+            return
+
+        with self.send_lock:
+            self.traces.append(trace)
+
+    def add_observation(self, trace_id: UUID, observation: ObservationData) -> None:
+        """
+        Add an observation to the buffer.
+
+        Args:
+            trace_id: ID of the parent trace
+            observation: Observation data to add
+        """
+        if not self.config.enabled:
+            return
+
+        with self.send_lock:
+            self.observations.append((trace_id, observation))
+
+    def flush_if_needed(self) -> None:
+        """Flush the buffer if batch size is reached."""
+        should_flush = False
+
+        with self.send_lock:
+            total_items = len(self.traces) + len(self.observations)
+            should_flush = total_items >= self.config.batch_size
+
+        if should_flush:
+            self.flush()
+
+    def flush(self) -> None:
+        """Send all buffered events to the collector."""
+        if not self.config.enabled:
+            return
+
+        # Collect items to send
+        traces_to_send: List[TraceData] = []
+        observations_to_send: List[Tuple[UUID, ObservationData]] = []
+
+        with self.send_lock:
+            # Move items from buffer
+            while self.traces:
+                traces_to_send.append(self.traces.popleft())
+
+            while self.observations:
+                observations_to_send.append(self.observations.popleft())
+
+            self.last_flush = time.time()
+
+        # Send if there's data
+        if traces_to_send or observations_to_send:
+            self._send_batch(traces_to_send, observations_to_send)
+
+    def _send_batch(
+        self, traces: List[TraceData], observations: List[Tuple[UUID, ObservationData]]
+    ) -> None:
+        """
+        Send a batch of events to the collector.
+
+        Args:
+            traces: List of traces to send
+            observations: List of (trace_id, observation) tuples
+        """
+        # Import here to avoid circular dependency
+        send_errors = False
+
+        if self.config.use_collector:
+            from .client import FluxLoopClient
+
+            client = FluxLoopClient()
+
+            for trace in traces:
+                try:
+                    client.send_trace(trace)
+                except Exception as e:
+                    send_errors = True
+                    if self.config.debug:
+                        print(f"Failed to send trace {trace.id}: {e}")
+
+            for trace_id, observation in observations:
+                try:
+                    client.send_observation(trace_id, observation)
+                except Exception as e:
+                    send_errors = True
+                    if self.config.debug:
+                        print(f"Failed to send observation {observation.id}: {e}")
+
+        if send_errors or not self.config.use_collector:
+            self.offline_store.record_traces(traces)
+            self.offline_store.record_observations(observations)
+
+    def _flush_periodically(self) -> None:
+        """Background thread to flush periodically."""
+        while not self.stop_event.is_set():
+            time.sleep(self.config.flush_interval)
+
+            # Check if enough time has passed since last flush
+            with self.send_lock:
+                time_since_flush = time.time() - self.last_flush
+                has_data = bool(self.traces or self.observations)
+
+            if has_data and time_since_flush >= self.config.flush_interval:
+                self.flush()
+
+    def shutdown(self) -> None:
+        """Shutdown the buffer and flush remaining events."""
+        # Stop the background thread
+        self.stop_event.set()
+
+        # Final flush
+        self.flush()
+
+        # Wait for thread to stop (with timeout)
+        if self.flush_thread.is_alive():
+            self.flush_thread.join(timeout=2.0)

fluxloop-0.1.4/fluxloop/client.py

@@ -0,0 +1,182 @@
+"""
+HTTP client for sending data to the collector.
+"""
+
+from types import TracebackType
+from typing import Any, Dict, Optional, Type, cast
+from uuid import UUID
+
+import httpx
+
+from .config import get_config
+from .models import ObservationData, TraceData
+
+
+class FluxLoopClient:
+    """
+    HTTP client for communicating with the FluxLoop collector.
+    """
+
+    def __init__(
+        self, collector_url: Optional[str] = None, api_key: Optional[str] = None
+    ):
+        """
+        Initialize the client.
+
+        Args:
+            collector_url: Override collector URL
+            api_key: Override API key
+        """
+        self.config = get_config()
+        default_url = self.config.collector_url or "http://localhost:8000"
+        self.collector_url = (collector_url or default_url).rstrip("/")
+        self.api_key = api_key or self.config.api_key
+        self._client: Optional[httpx.Client] = None
+        if self.config.use_collector:
+            self._client = httpx.Client(
+                base_url=self.collector_url,
+                timeout=self.config.timeout,
+                headers=self._get_headers(),
+            )
+
+    def _get_headers(self) -> Dict[str, str]:
+        """Get common headers for requests."""
+        headers = {
+            "Content-Type": "application/json",
+            "User-Agent": "fluxloop-sdk/0.1.0",
+        }
+
+        if self.api_key:
+            headers["Authorization"] = f"Bearer {self.api_key}"
+
+        if self.config.service_name:
+            headers["X-Service-Name"] = self.config.service_name
+
+        if self.config.environment:
+            headers["X-Environment"] = self.config.environment
+
+        return headers
+
+    def send_trace(self, trace: TraceData) -> Dict[str, Any]:
+        """
+        Send a trace to the collector.
+
+        Args:
+            trace: Trace data to send
+
+        Returns:
+            Response from the collector
+
+        Raises:
+            httpx.HTTPError: If the request fails
+        """
+        if not self.config.enabled:
+            return {"status": "disabled"}
+
+        # Convert to JSON-serializable format
+        payload = self._serialize_trace(trace)
+
+        if not self._client:
+            return {"status": "collector_disabled"}
+
+        try:
+            response = self._client.post("/api/traces", json=payload)
+            response.raise_for_status()
+            return cast(Dict[str, Any], response.json())
+        except httpx.HTTPError as e:
+            if self.config.debug:
+                print(f"Error sending trace: {e}")
+            raise
+
+    def send_observation(
+        self, trace_id: UUID, observation: ObservationData
+    ) -> Dict[str, Any]:
+        """
+        Send an observation to the collector.
+
+        Args:
+            trace_id: ID of the parent trace
+            observation: Observation data to send
+
+        Returns:
+            Response from the collector
+
+        Raises:
+            httpx.HTTPError: If the request fails
+        """
+        if not self.config.enabled:
+            return {"status": "disabled"}
+
+        # Convert to JSON-serializable format
+        payload = self._serialize_observation(observation)
+        payload["trace_id"] = str(trace_id)
+
+        if not self._client:
+            return {"status": "collector_disabled"}
+
+        try:
+            response = self._client.post(
+                f"/api/traces/{trace_id}/observations", json=payload
+            )
+            response.raise_for_status()
+            return cast(Dict[str, Any], response.json())
+        except httpx.HTTPError as e:
+            if self.config.debug:
+                print(f"Error sending observation: {e}")
+            raise
+
+    def _serialize_trace(self, trace: TraceData) -> Dict[str, Any]:
+        """Serialize trace for JSON transmission."""
+        data = trace.model_dump(exclude_none=True)
+
+        # Convert UUIDs to strings
+        if "id" in data:
+            data["id"] = str(data["id"])
+        if "session_id" in data:
+            data["session_id"] = str(data["session_id"])
+
+        # Convert datetime to ISO format
+        if "start_time" in data:
+            data["start_time"] = data["start_time"].isoformat()
+        if "end_time" in data and data["end_time"]:
+            data["end_time"] = data["end_time"].isoformat()
+
+        return data
+
+    def _serialize_observation(self, observation: ObservationData) -> Dict[str, Any]:
+        """Serialize observation for JSON transmission."""
+        data = observation.model_dump(exclude_none=True)
+
+        # Convert UUIDs to strings
+        if "id" in data:
+            data["id"] = str(data["id"])
+        if "parent_observation_id" in data:
+            data["parent_observation_id"] = str(data["parent_observation_id"])
+        if "trace_id" in data:
+            data["trace_id"] = str(data["trace_id"])
+
+        # Convert datetime to ISO format
+        if "start_time" in data:
+            data["start_time"] = data["start_time"].isoformat()
+        if "end_time" in data and data["end_time"]:
+            data["end_time"] = data["end_time"].isoformat()
+
+        return data
+
+    def close(self) -> None:
+        """Close the HTTP client."""
+        if self._client:
+            self._client.close()
+
+    def __enter__(self) -> "FluxLoopClient":
+        """Context manager entry."""
+        return self
+
+    def __exit__(
+        self,
+        exc_type: Optional[Type[BaseException]],
+        exc_val: Optional[BaseException],
+        exc_tb: Optional[TracebackType],
+    ) -> None:
+        """Context manager exit."""
+        self.close()