driftshield-mini 0.1.0__tar.gz

driftshield_mini-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 DriftShield
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

driftshield_mini-0.1.0/PKG-INFO

@@ -0,0 +1,250 @@
+Metadata-Version: 2.4
+Name: driftshield-mini
+Version: 0.1.0
+Summary: Real-time behavioural drift detection for agentic AI systems
+Author: DriftShield
+License: MIT
+Project-URL: Homepage, https://github.com/driftshield/driftshield-mini
+Project-URL: Issues, https://github.com/driftshield/driftshield-mini/issues
+Keywords: ai,agents,monitoring,drift-detection,langchain,crewai
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Topic :: Software Development :: Libraries
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: sentence-transformers>=2.2.0
+Requires-Dist: scikit-learn>=1.3.0
+Requires-Dist: numpy>=1.24.0
+Requires-Dist: click>=8.1.0
+Requires-Dist: httpx>=0.25.0
+Requires-Dist: rich>=13.0.0
+Provides-Extra: langchain
+Requires-Dist: langchain>=0.1.0; extra == "langchain"
+Requires-Dist: langchain-core>=0.1.0; extra == "langchain"
+Provides-Extra: crewai
+Requires-Dist: crewai>=0.1.0; extra == "crewai"
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0; extra == "dev"
+Requires-Dist: pytest-asyncio>=0.21; extra == "dev"
+Requires-Dist: ruff>=0.1.0; extra == "dev"
+Dynamic: license-file
+
+# DriftShield
+
+Your LangChain agent just called the same API 47 times. Your CrewAI crew burned £200 in tokens overnight. Your research agent started writing marketing copy instead of financial summaries.
+
+You didn't find out until morning.
+
+**DriftShield catches this stuff in real-time.** It wraps your existing agent, watches what it does, and pings you on Slack or Discord the moment something goes sideways. No dashboard. No cloud. No account to create. Just a Python library that runs alongside your agent.
+
+---
+
+## What it actually does
+
+DriftShield monitors three things:
+
+**Loop detection**:  Is your agent calling the same tool over and over? Or stuck in a cycle like `search → format → search → format`? DriftShield spots the pattern and alerts you before it eats your budget.
+
+**Goal drift**:  Is your agent still doing what you asked it to? DriftShield uses local embeddings (runs on your CPU, no API calls) to measure how far the agent's output has drifted from its original objective.
+
+**Resource spikes**:  Is this run burning way more tokens or taking way longer than usual? DriftShield learns what "normal" looks like for your agent, then flags when things go abnormal.
+
+Everything stays on your machine. Traces go to a local SQLite file. Embeddings run on your CPU. The only thing that leaves your machine is the alert you choose to send to Slack/Discord.
+
+---
+
+## Get started
+
+```
+pip install driftshield
+```
+
+### LangChain
+
+```python
+from driftshield import DriftMonitor
+
+monitor = DriftMonitor(
+    agent_id="logistics-v2",
+    alert_webhook="https://hooks.slack.com/...",
+)
+
+agent = monitor.wrap(existing_agent)
+result = agent.invoke({"input": "optimise route for order #4821"})
+# DriftShield is now watching. That's it.
+```
+
+### CrewAI
+
+```python
+from driftshield.crewai import DriftCrew
+
+crew = DriftCrew(
+    crew=existing_crew,
+    agent_id="research-team-v1",
+    alert_webhook="https://discord.com/api/webhooks/...",
+)
+
+result = crew.kickoff()
+```
+
+### Works with any LLM
+
+OpenAI, Anthropic, Groq, Ollama, local models doesn't matter. DriftShield only sees the traces (tool calls, token counts, outputs), not the model internals. Swap providers whenever you want.
+
+---
+
+## How calibration works
+
+For the first 30 runs (configurable), DriftShield quietly observes your agent and builds a baseline average tokens per run, typical tool sequences, normal execution time. No alerts during this phase.
+
+After that, it knows what "normal" looks like and starts flagging deviations. You can inspect the baseline anytime:
+
+```bash
+driftshield baseline my-agent
+```
+
+> **Tip:** If 30 runs feels like a lot, you can lower `calibration_runs` or use a preset template. DriftShield still catches obvious problems (like 50 identical tool calls) even without a baseline, using absolute safety limits.
+
+---
+
+## What an alert looks like
+
+When drift hits your Slack/Discord, you get:
+
+```json
+{
+  "agent_id": "logistics-v2",
+  "detector": "action_loop",
+  "severity": "HIGH",
+  "message": "Action loop: search_inventory called 6x in 45s",
+  "suggested_action": "Check search_inventory input/output for stale data or error loops",
+  "context": {
+    "tool_name": "search_inventory",
+    "repeat_count": 6,
+    "recent_actions": ["search_inventory", "search_inventory", "search_inventory", "..."]
+  }
+}
+```
+
+Not just "something's wrong" — it tells you what happened, which detector caught it, and what to check first.
+
+---
+
+## CLI
+
+```bash
+# What went wrong in the last 24 hours?
+driftshield alerts --last 24h
+
+# Show me exactly what my agent did on its last run
+driftshield traces logistics-v2 --run latest
+
+# What does "normal" look like for this agent?
+driftshield baseline logistics-v2
+
+# List recent runs
+driftshield runs logistics-v2
+```
+
+---
+
+## Configuration
+
+Everything's tuneable. Defaults are sensible, but you can adjust:
+
+```python
+monitor = DriftMonitor(
+    agent_id="my-agent",
+    alert_webhook="https://hooks.slack.com/...",
+    goal_description="Summarise financial reports",
+    calibration_runs=30,         # runs before baseline kicks in
+    loop_window=20,              # how many recent actions to check
+    loop_max_repeats=4,          # repeated calls before flagging
+    similarity_threshold=0.5,    # goal drift sensitivity (lower = stricter)
+    spike_multiplier=2.5,        # how many std devs = a spike
+    min_alert_severity="MED",    # ignore LOW severity events
+    alert_cooldown=60.0,         # don't spam the same alert
+)
+```
+
+---
+
+## Custom reactions
+
+DriftShield alerts you by default, but you can also react programmatically:
+
+```python
+def handle_drift(event):
+    if event.severity.value == "CRITICAL":
+        agent.stop()  # kill the run
+        page_oncall()  # wake someone up
+
+monitor.on_drift(handle_drift)
+```
+
+---
+
+## What this isn't
+
+I want to be upfront about scope. DriftShield is **v0.1**, built by one person.
+
+- **Not a full observability platform.** No web dashboard, no hosted backend, no team features. If you need that, look at LangSmith, Langfuse, or Arize.
+- **Not a guardrail system.** It detects drift after the fact and alerts you. It doesn't block actions before they happen (that's on the roadmap).
+- **Not production-hardened yet.** It works, it's tested, but it hasn't been battle-tested by thousands of users. Expect rough edges.
+
+What it IS: the smallest, simplest tool that does one thing well — tells you when your agent is going off the rails, fast, with zero setup overhead.
+
+---
+
+## Roadmap
+
+- **v0.2** — Auto-correction hooks (retry, context trim, kill run). Preset baseline templates so you get value from run 1.
+- **v0.3** — Better multi-agent support. Predictive drift (catch it before it happens).
+- **v1.0** — Dashboard, team features, historical analytics. But only if people actually want it.
+
+---
+
+## Built with
+
+- Python 3.10+
+- SQLite (zero config)
+- sentence-transformers (local CPU embeddings)
+- scikit-learn (basic stats)
+- httpx (webhooks)
+- click + rich (CLI)
+
+---
+
+## Contributing
+
+This is early. If you're running agents in production and hit a case DriftShield missed (or flagged incorrectly), please open an issue. Your real-world edge cases are the most valuable thing you can give this project right now.
+
+```bash
+git clone https://github.com/YOUR_USERNAME/driftshield.git
+cd driftshield
+python -m venv .venv
+source .venv/bin/activate  # or .venv\Scripts\activate on Windows
+pip install -e ".[dev]"
+python -m pytest tests/ -v
+```
+
+---
+
+## Why I built this
+
+I kept reading the same story: dev builds agent, agent works great in testing, agent goes haywire in production at 2am, dev wakes up to a hefty API bill and a Slack full of confused users. The big observability platforms exist but they're heavy on dashboards, accounts, pricing tiers, cloud dependencies. Most solo devs and small teams just want to know when their agent is broken. That's it.
+
+So I built the smallest thing that solves that problem.
+
+If you try it and it helps (or doesn't), I genuinely want to hear about it.
+
+---
+
+## License
+
+MIT - do whatever you want with it.

driftshield_mini-0.1.0/README.md

@@ -0,0 +1,215 @@
+# DriftShield
+
+Your LangChain agent just called the same API 47 times. Your CrewAI crew burned £200 in tokens overnight. Your research agent started writing marketing copy instead of financial summaries.
+
+You didn't find out until morning.
+
+**DriftShield catches this stuff in real-time.** It wraps your existing agent, watches what it does, and pings you on Slack or Discord the moment something goes sideways. No dashboard. No cloud. No account to create. Just a Python library that runs alongside your agent.
+
+---
+
+## What it actually does
+
+DriftShield monitors three things:
+
+**Loop detection**:  Is your agent calling the same tool over and over? Or stuck in a cycle like `search → format → search → format`? DriftShield spots the pattern and alerts you before it eats your budget.
+
+**Goal drift**:  Is your agent still doing what you asked it to? DriftShield uses local embeddings (runs on your CPU, no API calls) to measure how far the agent's output has drifted from its original objective.
+
+**Resource spikes**:  Is this run burning way more tokens or taking way longer than usual? DriftShield learns what "normal" looks like for your agent, then flags when things go abnormal.
+
+Everything stays on your machine. Traces go to a local SQLite file. Embeddings run on your CPU. The only thing that leaves your machine is the alert you choose to send to Slack/Discord.
+
+---
+
+## Get started
+
+```
+pip install driftshield
+```
+
+### LangChain
+
+```python
+from driftshield import DriftMonitor
+
+monitor = DriftMonitor(
+    agent_id="logistics-v2",
+    alert_webhook="https://hooks.slack.com/...",
+)
+
+agent = monitor.wrap(existing_agent)
+result = agent.invoke({"input": "optimise route for order #4821"})
+# DriftShield is now watching. That's it.
+```
+
+### CrewAI
+
+```python
+from driftshield.crewai import DriftCrew
+
+crew = DriftCrew(
+    crew=existing_crew,
+    agent_id="research-team-v1",
+    alert_webhook="https://discord.com/api/webhooks/...",
+)
+
+result = crew.kickoff()
+```
+
+### Works with any LLM
+
+OpenAI, Anthropic, Groq, Ollama, local models doesn't matter. DriftShield only sees the traces (tool calls, token counts, outputs), not the model internals. Swap providers whenever you want.
+
+---
+
+## How calibration works
+
+For the first 30 runs (configurable), DriftShield quietly observes your agent and builds a baseline average tokens per run, typical tool sequences, normal execution time. No alerts during this phase.
+
+After that, it knows what "normal" looks like and starts flagging deviations. You can inspect the baseline anytime:
+
+```bash
+driftshield baseline my-agent
+```
+
+> **Tip:** If 30 runs feels like a lot, you can lower `calibration_runs` or use a preset template. DriftShield still catches obvious problems (like 50 identical tool calls) even without a baseline, using absolute safety limits.
+
+---
+
+## What an alert looks like
+
+When drift hits your Slack/Discord, you get:
+
+```json
+{
+  "agent_id": "logistics-v2",
+  "detector": "action_loop",
+  "severity": "HIGH",
+  "message": "Action loop: search_inventory called 6x in 45s",
+  "suggested_action": "Check search_inventory input/output for stale data or error loops",
+  "context": {
+    "tool_name": "search_inventory",
+    "repeat_count": 6,
+    "recent_actions": ["search_inventory", "search_inventory", "search_inventory", "..."]
+  }
+}
+```
+
+Not just "something's wrong" — it tells you what happened, which detector caught it, and what to check first.
+
+---
+
+## CLI
+
+```bash
+# What went wrong in the last 24 hours?
+driftshield alerts --last 24h
+
+# Show me exactly what my agent did on its last run
+driftshield traces logistics-v2 --run latest
+
+# What does "normal" look like for this agent?
+driftshield baseline logistics-v2
+
+# List recent runs
+driftshield runs logistics-v2
+```
+
+---
+
+## Configuration
+
+Everything's tuneable. Defaults are sensible, but you can adjust:
+
+```python
+monitor = DriftMonitor(
+    agent_id="my-agent",
+    alert_webhook="https://hooks.slack.com/...",
+    goal_description="Summarise financial reports",
+    calibration_runs=30,         # runs before baseline kicks in
+    loop_window=20,              # how many recent actions to check
+    loop_max_repeats=4,          # repeated calls before flagging
+    similarity_threshold=0.5,    # goal drift sensitivity (lower = stricter)
+    spike_multiplier=2.5,        # how many std devs = a spike
+    min_alert_severity="MED",    # ignore LOW severity events
+    alert_cooldown=60.0,         # don't spam the same alert
+)
+```
+
+---
+
+## Custom reactions
+
+DriftShield alerts you by default, but you can also react programmatically:
+
+```python
+def handle_drift(event):
+    if event.severity.value == "CRITICAL":
+        agent.stop()  # kill the run
+        page_oncall()  # wake someone up
+
+monitor.on_drift(handle_drift)
+```
+
+---
+
+## What this isn't
+
+I want to be upfront about scope. DriftShield is **v0.1**, built by one person.
+
+- **Not a full observability platform.** No web dashboard, no hosted backend, no team features. If you need that, look at LangSmith, Langfuse, or Arize.
+- **Not a guardrail system.** It detects drift after the fact and alerts you. It doesn't block actions before they happen (that's on the roadmap).
+- **Not production-hardened yet.** It works, it's tested, but it hasn't been battle-tested by thousands of users. Expect rough edges.
+
+What it IS: the smallest, simplest tool that does one thing well — tells you when your agent is going off the rails, fast, with zero setup overhead.
+
+---
+
+## Roadmap
+
+- **v0.2** — Auto-correction hooks (retry, context trim, kill run). Preset baseline templates so you get value from run 1.
+- **v0.3** — Better multi-agent support. Predictive drift (catch it before it happens).
+- **v1.0** — Dashboard, team features, historical analytics. But only if people actually want it.
+
+---
+
+## Built with
+
+- Python 3.10+
+- SQLite (zero config)
+- sentence-transformers (local CPU embeddings)
+- scikit-learn (basic stats)
+- httpx (webhooks)
+- click + rich (CLI)
+
+---
+
+## Contributing
+
+This is early. If you're running agents in production and hit a case DriftShield missed (or flagged incorrectly), please open an issue. Your real-world edge cases are the most valuable thing you can give this project right now.
+
+```bash
+git clone https://github.com/YOUR_USERNAME/driftshield.git
+cd driftshield
+python -m venv .venv
+source .venv/bin/activate  # or .venv\Scripts\activate on Windows
+pip install -e ".[dev]"
+python -m pytest tests/ -v
+```
+
+---
+
+## Why I built this
+
+I kept reading the same story: dev builds agent, agent works great in testing, agent goes haywire in production at 2am, dev wakes up to a hefty API bill and a Slack full of confused users. The big observability platforms exist but they're heavy on dashboards, accounts, pricing tiers, cloud dependencies. Most solo devs and small teams just want to know when their agent is broken. That's it.
+
+So I built the smallest thing that solves that problem.
+
+If you try it and it helps (or doesn't), I genuinely want to hear about it.
+
+---
+
+## License
+
+MIT - do whatever you want with it.

driftshield_mini-0.1.0/driftshield/__init__.py

@@ -0,0 +1,27 @@
+"""
+DriftShield — Real-time behavioural drift detection for agentic AI systems.
+
+Usage:
+    from driftshield import DriftMonitor
+
+    monitor = DriftMonitor(
+        agent_id="my-agent",
+        alert_webhook="https://hooks.slack.com/...",
+    )
+    agent = monitor.wrap(existing_agent)
+    result = agent.invoke({"input": "do the thing"})
+"""
+
+from driftshield.models import BaselineStats, DetectorType, DriftEvent, Severity, TraceEvent
+from driftshield.monitor import DriftMonitor
+
+__version__ = "0.1.0"
+
+__all__ = [
+    "DriftMonitor",
+    "TraceEvent",
+    "DriftEvent",
+    "BaselineStats",
+    "DetectorType",
+    "Severity",
+]

driftshield_mini-0.1.0/driftshield/alerts/__init__.py

@@ -0,0 +1,169 @@
+"""Alert dispatchers — Slack, Discord, and generic webhook support."""
+
+from __future__ import annotations
+
+import json
+import logging
+import time
+from datetime import datetime, timezone
+from typing import Any
+
+from driftshield.models import DriftEvent
+
+logger = logging.getLogger(__name__)
+
+# Severity → color mapping
+SEVERITY_COLORS = {
+    "LOW": "#36a64f",       # green
+    "MED": "#daa520",       # amber
+    "HIGH": "#ff6600",      # orange
+    "CRITICAL": "#ff0000",  # red
+}
+
+SEVERITY_EMOJI = {
+    "LOW": "🟢",
+    "MED": "🟡",
+    "HIGH": "🟠",
+    "CRITICAL": "🔴",
+}
+
+
+class AlertDispatcher:
+    """Dispatches drift alerts via webhook (Slack, Discord, or generic)."""
+
+    def __init__(
+        self,
+        webhook_url: str | None = None,
+        min_severity: str = "MEDIUM",
+        cooldown_seconds: float = 60.0,
+    ):
+        self.webhook_url = webhook_url
+        self.min_severity = min_severity
+        self.cooldown_seconds = cooldown_seconds
+        self._last_alert: dict[str, float] = {}  # agent_id+detector → timestamp
+
+    def should_alert(self, event: DriftEvent) -> bool:
+        """Check if alert should fire (severity + cooldown)."""
+        if not self.webhook_url:
+            return False
+
+        severity_order = ["LOW", "MED", "HIGH", "CRITICAL"]
+        min_idx = severity_order.index(self.min_severity) if self.min_severity in severity_order else 1
+        event_idx = severity_order.index(event.severity.value) if event.severity.value in severity_order else 0
+
+        if event_idx < min_idx:
+            return False
+
+        # Cooldown: don't spam the same detector for the same agent
+        key = f"{event.agent_id}:{event.detector.value}"
+        now = time.time()
+        if key in self._last_alert and (now - self._last_alert[key]) < self.cooldown_seconds:
+            return False
+
+        self._last_alert[key] = now
+        return True
+
+    async def send_async(self, event: DriftEvent) -> bool:
+        """Send alert via async HTTP (preferred)."""
+        if not self.should_alert(event):
+            return False
+
+        try:
+            import httpx
+
+            payload = self._build_payload(event)
+            async with httpx.AsyncClient(timeout=10.0) as client:
+                resp = await client.post(self.webhook_url, json=payload)
+                resp.raise_for_status()
+                logger.info(f"Alert sent for {event.agent_id}: {event.message}")
+                return True
+        except Exception as e:
+            logger.warning(f"Failed to send alert: {e}")
+            return False
+
+    def send_sync(self, event: DriftEvent) -> bool:
+        """Send alert via sync HTTP (fallback)."""
+        if not self.should_alert(event):
+            return False
+
+        try:
+            import httpx
+
+            payload = self._build_payload(event)
+            with httpx.Client(timeout=10.0) as client:
+                resp = client.post(self.webhook_url, json=payload)
+                resp.raise_for_status()
+                logger.info(f"Alert sent for {event.agent_id}: {event.message}")
+                return True
+        except Exception as e:
+            logger.warning(f"Failed to send alert: {e}")
+            return False
+
+    def _build_payload(self, event: DriftEvent) -> dict[str, Any]:
+        """Build webhook payload — auto-detects Slack vs Discord vs generic."""
+        if not self.webhook_url:
+            return {}
+
+        if "hooks.slack.com" in self.webhook_url:
+            return self._slack_payload(event)
+        elif "discord.com" in self.webhook_url:
+            return self._discord_payload(event)
+        else:
+            return self._generic_payload(event)
+
+    def _slack_payload(self, event: DriftEvent) -> dict[str, Any]:
+        emoji = SEVERITY_EMOJI.get(event.severity.value, "⚠️")
+        color = SEVERITY_COLORS.get(event.severity.value, "#daa520")
+        ts = datetime.fromtimestamp(event.timestamp, tz=timezone.utc).strftime("%Y-%m-%d %H:%M UTC")
+
+        return {
+            "attachments": [
+                {
+                    "color": color,
+                    "blocks": [
+                        {
+                            "type": "section",
+                            "text": {
+                                "type": "mrkdwn",
+                                "text": (
+                                    f"{emoji} *[{event.severity.value}] DriftShield Alert*\n"
+                                    f"*Agent:* `{event.agent_id}`\n"
+                                    f"*Detector:* {event.detector.value}\n"
+                                    f"*Time:* {ts}\n\n"
+                                    f"{event.message}\n\n"
+                                    f"💡 *Suggested action:* {event.suggested_action}"
+                                ),
+                            },
+                        }
+                    ],
+                }
+            ]
+        }
+
+    def _discord_payload(self, event: DriftEvent) -> dict[str, Any]:
+        emoji = SEVERITY_EMOJI.get(event.severity.value, "⚠️")
+        color_hex = SEVERITY_COLORS.get(event.severity.value, "#daa520")
+        color_int = int(color_hex.lstrip("#"), 16)
+        ts = datetime.fromtimestamp(event.timestamp, tz=timezone.utc).strftime("%Y-%m-%d %H:%M UTC")
+
+        return {
+            "embeds": [
+                {
+                    "title": f"{emoji} [{event.severity.value}] DriftShield Alert",
+                    "color": color_int,
+                    "fields": [
+                        {"name": "Agent", "value": f"`{event.agent_id}`", "inline": True},
+                        {"name": "Detector", "value": event.detector.value, "inline": True},
+                        {"name": "Time", "value": ts, "inline": True},
+                        {"name": "Details", "value": event.message, "inline": False},
+                        {"name": "💡 Suggested Action", "value": event.suggested_action, "inline": False},
+                    ],
+                }
+            ]
+        }
+
+    def _generic_payload(self, event: DriftEvent) -> dict[str, Any]:
+        return {
+            "source": "driftshield",
+            "event": event.to_dict(),
+        }