agent-vitals 1.0.0__tar.gz

agent_vitals-1.0.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Agent Vitals Contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

agent_vitals-1.0.0/PKG-INFO

@@ -0,0 +1,235 @@
+Metadata-Version: 2.4
+Name: agent-vitals
+Version: 1.0.0
+Summary: Standalone agent health monitor — detect loops, stuck states, thrash, and runaway costs in any AI agent workflow.
+Author: Agent Vitals Contributors
+License: MIT
+Project-URL: Homepage, https://github.com/kneelinghorse/agent-vitals
+Project-URL: Documentation, https://github.com/kneelinghorse/agent-vitals#readme
+Project-URL: Repository, https://github.com/kneelinghorse/agent-vitals
+Project-URL: Issues, https://github.com/kneelinghorse/agent-vitals/issues
+Keywords: agent,vitals,monitoring,llm,health,loop-detection
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: pydantic>=2.0
+Requires-Dist: pyyaml>=6.0
+Provides-Extra: langchain
+Requires-Dist: langchain>=0.3; extra == "langchain"
+Provides-Extra: langgraph
+Requires-Dist: langgraph>=0.2; extra == "langgraph"
+Provides-Extra: otlp
+Requires-Dist: opentelemetry-sdk; extra == "otlp"
+Provides-Extra: langfuse
+Requires-Dist: langfuse>=2.0; extra == "langfuse"
+Provides-Extra: langsmith
+Requires-Dist: langsmith>=0.1; extra == "langsmith"
+Provides-Extra: all
+Requires-Dist: langchain>=0.3; extra == "all"
+Requires-Dist: langgraph>=0.2; extra == "all"
+Requires-Dist: opentelemetry-sdk; extra == "all"
+Requires-Dist: langfuse>=2.0; extra == "all"
+Requires-Dist: langsmith>=0.1; extra == "all"
+Provides-Extra: dev
+Requires-Dist: pytest>=8.0; extra == "dev"
+Requires-Dist: pytest-asyncio>=0.24; extra == "dev"
+Requires-Dist: ruff>=0.7; extra == "dev"
+Requires-Dist: mypy>=1.13; extra == "dev"
+Dynamic: license-file
+
+# Agent Vitals
+
+[![PyPI version](https://img.shields.io/pypi/v/agent-vitals)](https://pypi.org/project/agent-vitals/)
+[![Python](https://img.shields.io/pypi/pyversions/agent-vitals)](https://pypi.org/project/agent-vitals/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+
+**Standalone agent health monitor** — detect loops, stuck states, thrash, and runaway costs in any AI agent workflow.
+
+Agent Vitals watches your LLM agent's vital signs in real time. Feed it four numbers per step and it tells you when your agent is looping, stuck, thrashing, or burning tokens for nothing.
+
+## Install
+
+```bash
+pip install agent-vitals
+```
+
+## Quick Start
+
+```python
+from agent_vitals import AgentVitals
+
+monitor = AgentVitals(mission_id="my-task")
+
+for step in range(max_steps):
+    result = call_llm(prompt)
+    findings = extract_findings(result)
+
+    snapshot = monitor.step(
+        findings_count=len(findings),
+        coverage_score=compute_coverage(findings),
+        total_tokens=result.usage.total_tokens,
+        error_count=error_tracker.count,
+    )
+
+    if snapshot.any_failure:
+        print(f"Health issue at step {snapshot.loop_index}: "
+              f"{snapshot.stuck_trigger or snapshot.loop_trigger}")
+        break
+```
+
+## Features
+
+- **4-field minimum**: Only `findings_count`, `coverage_score`, `total_tokens`, `error_count` required
+- **Zero-config defaults**: `AgentVitals()` works out of the box with tuned thresholds
+- **Framework-agnostic**: No dependency on LangChain, LangGraph, or any agent framework
+- **Immutable snapshots**: Every `step()` returns a `VitalsSnapshot` with signals, metrics, and detection results
+- **JSONL export**: Auto-log every snapshot to structured JSONL files
+- **Backtest harness**: Offline evaluation of recorded trajectories with P/R/F1 metrics
+- **Context manager**: `with AgentVitals(...) as monitor:` for clean resource management
+
+## Detection Modes
+
+| Detector | What it catches | Signal |
+|---|---|---|
+| **Loop** | Agent repeating actions without progress | Findings plateau over N steps |
+| **Stuck** | Coverage stagnation despite continued work | Low DM + low CV on coverage |
+| **Thrash** | Excessive errors indicating instability | Error count above threshold |
+| **Runaway Cost** | Token burn with no output | Token spike with flat findings |
+
+## API Overview
+
+### Manual Integration (Recommended)
+
+```python
+from agent_vitals import AgentVitals
+
+monitor = AgentVitals(mission_id="research-task")
+snapshot = monitor.step(
+    findings_count=5,
+    coverage_score=0.6,
+    total_tokens=12000,
+    error_count=0,
+)
+
+print(snapshot.health_state)     # "healthy" | "warning" | "critical"
+print(snapshot.any_failure)      # True if loop or stuck detected
+print(snapshot.stuck_trigger)    # e.g. "coverage_stagnation", "burn_rate_anomaly"
+```
+
+### Adapter Integration
+
+```python
+from agent_vitals import AgentVitals
+from agent_vitals.adapters import TelemetryAdapter
+
+monitor = AgentVitals(mission_id="my-task", adapter=TelemetryAdapter())
+snapshot = monitor.step_from_state({
+    "cumulative_outputs": 5,
+    "coverage_score": 0.6,
+    "cumulative_tokens": 12000,
+    "cumulative_errors": 0,
+})
+```
+
+### Pre-built Signals
+
+```python
+from agent_vitals import AgentVitals, RawSignals
+
+monitor = AgentVitals(mission_id="my-task")
+signals = RawSignals(findings_count=5, coverage_score=0.6, total_tokens=12000, error_count=0)
+snapshot = monitor.step_from_signals(signals)
+```
+
+## Export
+
+Log every snapshot to JSONL for offline analysis or observability pipelines.
+
+```python
+from agent_vitals import AgentVitals, JSONLExporter
+
+exporter = JSONLExporter(
+    directory="./vitals_logs",
+    layout="per_run",       # or "append"
+    max_bytes=10_000_000,   # rotation threshold (append mode)
+)
+
+with AgentVitals(mission_id="my-task", exporters=[exporter]) as monitor:
+    for step in range(max_steps):
+        monitor.step(findings_count=..., coverage_score=..., total_tokens=..., error_count=...)
+# Exporter is automatically flushed and closed on exit
+```
+
+**Layouts:**
+- `per_run`: `{directory}/{mission_id}/{run_id}.jsonl` — one file per run
+- `append`: `{directory}/{mission_id}.jsonl` — all runs in one file, with rotation
+
+## Configuration
+
+```python
+from agent_vitals import AgentVitals, VitalsConfig
+
+# From constructor kwargs
+monitor = AgentVitals(config=VitalsConfig(
+    loop_consecutive_count=6,
+    stuck_dm_threshold=0.15,
+))
+
+# From YAML file
+monitor = AgentVitals.from_yaml("thresholds.yaml")
+
+# From environment variables (VITALS_* prefix)
+monitor = AgentVitals()  # auto-reads VITALS_LOOP_CONSECUTIVE_COUNT, etc.
+```
+
+### Key Thresholds
+
+| Parameter | Default | Description |
+|---|---|---|
+| `loop_consecutive_count` | 5 | Steps of flat findings before loop detection |
+| `stuck_dm_threshold` | 0.15 | DM below this → coverage stagnation |
+| `stuck_cv_threshold` | 0.5 | CV below this → low variation |
+| `burn_rate_multiplier` | 2.0 | Token spike ratio for burn rate anomaly |
+
+## Backtest
+
+Evaluate detection accuracy against labeled trajectory corpora.
+
+```python
+from agent_vitals.backtest import load_dataset, load_labels, run_backtest
+
+dataset = load_dataset("path/to/traces/")
+labels = load_labels("path/to/labels.json")
+report = run_backtest(dataset, labels)
+
+print(f"vitals.any: P={report.composite_any.precision:.3f} "
+      f"R={report.composite_any.recall:.3f} "
+      f"F1={report.composite_any.f1:.3f}")
+
+for name, detector in report.detectors.items():
+    print(f"  {name}: P={detector.precision:.3f} R={detector.recall:.3f}")
+```
+
+## Session Summary
+
+```python
+monitor = AgentVitals(mission_id="my-task")
+# ... run steps ...
+summary = monitor.summary()
+# {"mission_id": "my-task", "total_steps": 8, "health_state": "healthy",
+#  "any_loop_detected": False, "any_stuck_detected": False, ...}
+
+monitor.reset()  # Clear history for next run (also flushes exporters)
+```
+
+## License
+
+MIT — see [LICENSE](LICENSE).

agent_vitals-1.0.0/README.md

@@ -0,0 +1,188 @@
+# Agent Vitals
+
+[![PyPI version](https://img.shields.io/pypi/v/agent-vitals)](https://pypi.org/project/agent-vitals/)
+[![Python](https://img.shields.io/pypi/pyversions/agent-vitals)](https://pypi.org/project/agent-vitals/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+
+**Standalone agent health monitor** — detect loops, stuck states, thrash, and runaway costs in any AI agent workflow.
+
+Agent Vitals watches your LLM agent's vital signs in real time. Feed it four numbers per step and it tells you when your agent is looping, stuck, thrashing, or burning tokens for nothing.
+
+## Install
+
+```bash
+pip install agent-vitals
+```
+
+## Quick Start
+
+```python
+from agent_vitals import AgentVitals
+
+monitor = AgentVitals(mission_id="my-task")
+
+for step in range(max_steps):
+    result = call_llm(prompt)
+    findings = extract_findings(result)
+
+    snapshot = monitor.step(
+        findings_count=len(findings),
+        coverage_score=compute_coverage(findings),
+        total_tokens=result.usage.total_tokens,
+        error_count=error_tracker.count,
+    )
+
+    if snapshot.any_failure:
+        print(f"Health issue at step {snapshot.loop_index}: "
+              f"{snapshot.stuck_trigger or snapshot.loop_trigger}")
+        break
+```
+
+## Features
+
+- **4-field minimum**: Only `findings_count`, `coverage_score`, `total_tokens`, `error_count` required
+- **Zero-config defaults**: `AgentVitals()` works out of the box with tuned thresholds
+- **Framework-agnostic**: No dependency on LangChain, LangGraph, or any agent framework
+- **Immutable snapshots**: Every `step()` returns a `VitalsSnapshot` with signals, metrics, and detection results
+- **JSONL export**: Auto-log every snapshot to structured JSONL files
+- **Backtest harness**: Offline evaluation of recorded trajectories with P/R/F1 metrics
+- **Context manager**: `with AgentVitals(...) as monitor:` for clean resource management
+
+## Detection Modes
+
+| Detector | What it catches | Signal |
+|---|---|---|
+| **Loop** | Agent repeating actions without progress | Findings plateau over N steps |
+| **Stuck** | Coverage stagnation despite continued work | Low DM + low CV on coverage |
+| **Thrash** | Excessive errors indicating instability | Error count above threshold |
+| **Runaway Cost** | Token burn with no output | Token spike with flat findings |
+
+## API Overview
+
+### Manual Integration (Recommended)
+
+```python
+from agent_vitals import AgentVitals
+
+monitor = AgentVitals(mission_id="research-task")
+snapshot = monitor.step(
+    findings_count=5,
+    coverage_score=0.6,
+    total_tokens=12000,
+    error_count=0,
+)
+
+print(snapshot.health_state)     # "healthy" | "warning" | "critical"
+print(snapshot.any_failure)      # True if loop or stuck detected
+print(snapshot.stuck_trigger)    # e.g. "coverage_stagnation", "burn_rate_anomaly"
+```
+
+### Adapter Integration
+
+```python
+from agent_vitals import AgentVitals
+from agent_vitals.adapters import TelemetryAdapter
+
+monitor = AgentVitals(mission_id="my-task", adapter=TelemetryAdapter())
+snapshot = monitor.step_from_state({
+    "cumulative_outputs": 5,
+    "coverage_score": 0.6,
+    "cumulative_tokens": 12000,
+    "cumulative_errors": 0,
+})
+```
+
+### Pre-built Signals
+
+```python
+from agent_vitals import AgentVitals, RawSignals
+
+monitor = AgentVitals(mission_id="my-task")
+signals = RawSignals(findings_count=5, coverage_score=0.6, total_tokens=12000, error_count=0)
+snapshot = monitor.step_from_signals(signals)
+```
+
+## Export
+
+Log every snapshot to JSONL for offline analysis or observability pipelines.
+
+```python
+from agent_vitals import AgentVitals, JSONLExporter
+
+exporter = JSONLExporter(
+    directory="./vitals_logs",
+    layout="per_run",       # or "append"
+    max_bytes=10_000_000,   # rotation threshold (append mode)
+)
+
+with AgentVitals(mission_id="my-task", exporters=[exporter]) as monitor:
+    for step in range(max_steps):
+        monitor.step(findings_count=..., coverage_score=..., total_tokens=..., error_count=...)
+# Exporter is automatically flushed and closed on exit
+```
+
+**Layouts:**
+- `per_run`: `{directory}/{mission_id}/{run_id}.jsonl` — one file per run
+- `append`: `{directory}/{mission_id}.jsonl` — all runs in one file, with rotation
+
+## Configuration
+
+```python
+from agent_vitals import AgentVitals, VitalsConfig
+
+# From constructor kwargs
+monitor = AgentVitals(config=VitalsConfig(
+    loop_consecutive_count=6,
+    stuck_dm_threshold=0.15,
+))
+
+# From YAML file
+monitor = AgentVitals.from_yaml("thresholds.yaml")
+
+# From environment variables (VITALS_* prefix)
+monitor = AgentVitals()  # auto-reads VITALS_LOOP_CONSECUTIVE_COUNT, etc.
+```
+
+### Key Thresholds
+
+| Parameter | Default | Description |
+|---|---|---|
+| `loop_consecutive_count` | 5 | Steps of flat findings before loop detection |
+| `stuck_dm_threshold` | 0.15 | DM below this → coverage stagnation |
+| `stuck_cv_threshold` | 0.5 | CV below this → low variation |
+| `burn_rate_multiplier` | 2.0 | Token spike ratio for burn rate anomaly |
+
+## Backtest
+
+Evaluate detection accuracy against labeled trajectory corpora.
+
+```python
+from agent_vitals.backtest import load_dataset, load_labels, run_backtest
+
+dataset = load_dataset("path/to/traces/")
+labels = load_labels("path/to/labels.json")
+report = run_backtest(dataset, labels)
+
+print(f"vitals.any: P={report.composite_any.precision:.3f} "
+      f"R={report.composite_any.recall:.3f} "
+      f"F1={report.composite_any.f1:.3f}")
+
+for name, detector in report.detectors.items():
+    print(f"  {name}: P={detector.precision:.3f} R={detector.recall:.3f}")
+```
+
+## Session Summary
+
+```python
+monitor = AgentVitals(mission_id="my-task")
+# ... run steps ...
+summary = monitor.summary()
+# {"mission_id": "my-task", "total_steps": 8, "health_state": "healthy",
+#  "any_loop_detected": False, "any_stuck_detected": False, ...}
+
+monitor.reset()  # Clear history for next run (also flushes exporters)
+```
+
+## License
+
+MIT — see [LICENSE](LICENSE).

agent_vitals-1.0.0/agent_vitals/__init__.py

@@ -0,0 +1,54 @@
+"""Agent Vitals — Standalone agent health monitor.
+
+Detect loops, stuck states, thrash, and runaway costs in any AI agent workflow.
+
+Usage::
+
+    from agent_vitals import AgentVitals, VitalsSnapshot, RawSignals
+
+    monitor = AgentVitals(mission_id="my-task")
+    snapshot = monitor.step(
+        findings_count=5,
+        coverage_score=0.6,
+        total_tokens=12000,
+        error_count=0,
+    )
+"""
+
+from .config import VitalsConfig, get_vitals_config
+from .detection.loop import LoopDetectionResult, detect_loop
+from .detection.stop_rule import StopRuleSignals, derive_stop_signals
+from .exceptions import AdapterError, BacktestError, ConfigurationError, ExportError, VitalsError
+from .export import JSONLExporter, VitalsExporter
+from .monitor import AgentVitals
+from .schema import (
+    HealthState,
+    InterventionRecord,
+    RawSignals,
+    TemporalMetricsResult,
+    VitalsSnapshot,
+)
+
+__version__ = "1.0.0"
+
+__all__ = [
+    "AdapterError",
+    "AgentVitals",
+    "BacktestError",
+    "ConfigurationError",
+    "ExportError",
+    "HealthState",
+    "InterventionRecord",
+    "JSONLExporter",
+    "LoopDetectionResult",
+    "RawSignals",
+    "StopRuleSignals",
+    "TemporalMetricsResult",
+    "VitalsConfig",
+    "VitalsError",
+    "VitalsExporter",
+    "VitalsSnapshot",
+    "derive_stop_signals",
+    "detect_loop",
+    "get_vitals_config",
+]

agent_vitals-1.0.0/agent_vitals/adapters/__init__.py

@@ -0,0 +1,128 @@
+"""Signal Adapter Protocol for Agent Vitals.
+
+Defines the interface for mapping arbitrary agent state to RawSignals.
+Concrete adapters implement `extract()` for their specific framework.
+
+This is the primary extension point for integrating Agent Vitals with
+any agent framework. Users can either:
+1. Implement SignalAdapter for their framework
+2. Construct RawSignals directly (manual mode)
+"""
+
+from __future__ import annotations
+
+from typing import Any, Mapping, Protocol, runtime_checkable
+
+from ..schema import RawSignals
+
+
+@runtime_checkable
+class SignalAdapter(Protocol):
+    """Protocol for extracting RawSignals from arbitrary agent state.
+
+    Implementors map framework-specific telemetry/state to the generic
+    RawSignals schema that the detection engine operates on.
+
+    Minimum viable implementation requires only 4 fields:
+        - findings_count (or equivalent output count)
+        - coverage_score (or equivalent progress metric, 0.0-1.0)
+        - total_tokens (cumulative token usage)
+        - error_count (cumulative errors)
+
+    All other fields enhance detection confidence but are not required.
+    """
+
+    def extract(self, state: Mapping[str, Any]) -> RawSignals:
+        """Extract RawSignals from framework-specific agent state.
+
+        Args:
+            state: Framework-specific state mapping. The shape depends
+                on the agent framework (e.g., LangGraph AgentState,
+                LangChain callback data, raw telemetry dict).
+
+        Returns:
+            RawSignals instance populated from the agent state.
+            Fields that cannot be mapped should use their defaults (0).
+        """
+        ...
+
+
+class BaseAdapter:
+    """Base class for signal adapters with common helpers.
+
+    Subclass this and override `extract()` for your framework.
+    Provides utility methods for safe field extraction.
+    """
+
+    @staticmethod
+    def _safe_int(value: Any, default: int = 0) -> int:
+        try:
+            return int(value)
+        except (TypeError, ValueError):
+            return default
+
+    @staticmethod
+    def _safe_float(value: Any, default: float = 0.0) -> float:
+        try:
+            return float(value)
+        except (TypeError, ValueError):
+            return default
+
+    @staticmethod
+    def _safe_len(value: Any, default: int = 0) -> int:
+        try:
+            return len(value) if value else default
+        except TypeError:
+            return default
+
+    def extract(self, state: Mapping[str, Any]) -> RawSignals:
+        raise NotImplementedError("Subclasses must implement extract()")
+
+
+class TelemetryAdapter(BaseAdapter):
+    """Adapter for generic per-step telemetry dicts.
+
+    Maps the StepTelemetry format from cross-agent harnesses to RawSignals.
+    This is the adapter used for cross-agent validation — it works with
+    any agent that emits per-step JSON telemetry.
+
+    Expected state keys (from StepTelemetry.to_dict()):
+        - outputs_produced: int (maps to findings_count)
+        - total_tokens: int
+        - errors: int (maps to error_count)
+        - tool_calls: int (maps to query_count)
+        - tool_results: int (maps to sources_count)
+
+    Cumulative state keys (aggregated across steps):
+        - cumulative_outputs: int (total outputs so far)
+        - cumulative_tokens: int (total tokens so far)
+        - cumulative_errors: int (total errors so far)
+        - cumulative_queries: int (total tool calls so far)
+        - cumulative_sources: int (total tool results so far)
+        - coverage_score: float (progress estimate, 0.0-1.0)
+    """
+
+    def extract(self, state: Mapping[str, Any]) -> RawSignals:
+        return RawSignals(
+            findings_count=self._safe_int(state.get("cumulative_outputs", 0)),
+            sources_count=self._safe_int(state.get("cumulative_sources", 0)),
+            objectives_covered=0,
+            coverage_score=self._safe_float(state.get("coverage_score", 0.0)),
+            confidence_score=0.0,
+            prompt_tokens=self._safe_int(state.get("prompt_tokens", 0)),
+            completion_tokens=self._safe_int(state.get("completion_tokens", 0)),
+            total_tokens=self._safe_int(state.get("cumulative_tokens", 0)),
+            api_calls=self._safe_int(state.get("cumulative_queries", 0)),
+            query_count=self._safe_int(state.get("cumulative_queries", 0)),
+            unique_domains=0,
+            refinement_count=0,
+            convergence_delta=0.0,
+            error_count=self._safe_int(state.get("cumulative_errors", 0)),
+        )
+
+
+__all__ = [
+    "BaseAdapter",
+    "SignalAdapter",
+    "TelemetryAdapter",
+]