sotis 1.0.1__tar.gz

sotis-1.0.1/PKG-INFO

@@ -0,0 +1,180 @@
+Metadata-Version: 2.4
+Name: sotis
+Version: 1.0.1
+Summary: Python reliability library for long-running LLM agents
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+Requires-Dist: numpy>=1.26.0
+Requires-Dist: pydantic>=2.6.0
+Requires-Dist: python-dateutil>=2.9.0
+Requires-Dist: openai>=1.25.0
+Requires-Dist: anthropic>=0.25.0
+Requires-Dist: rich>=13.7.0
+Provides-Extra: dev
+Requires-Dist: pytest>=8.2.0; extra == "dev"
+Requires-Dist: pytest-benchmark>=4.0.0; extra == "dev"
+Requires-Dist: pytest-cov>=5.0.0; extra == "dev"
+Requires-Dist: langchain>=0.2.0; extra == "dev"
+Requires-Dist: langgraph>=0.0.60; extra == "dev"
+Provides-Extra: langgraph
+Requires-Dist: langchain>=0.2.0; extra == "langgraph"
+Requires-Dist: langgraph>=0.0.60; extra == "langgraph"
+Provides-Extra: obs
+Requires-Dist: streamlit>=1.35.0; extra == "obs"
+
+# Sotis
+
+Sotis is a Python reliability library for long-running LLM agents that detects meltdown behavior and automatically resets execution before agents spiral into loops or context collapse.
+
+---
+
+## The Long-Horizon Failure Problem
+
+Current AI agents fail predictably under long-horizon execution. As they run for longer periods, they accumulate error and drift into terminal failure modes:
+
+* **Infinite Loops**: Repeating the same tool calls with identical arguments.
+* **Semantic Spirals**: Rephrasing failed queries or tool calls hoping for different outcomes.
+* **Context Poisoning**: Flooding the chat history with massive error traces and linter warnings.
+* **Edit Storms**: Making rapid, uncoordinated file edits without shifting validation outputs.
+
+Frontier models do not fail because they are simple, but because long-horizon execution decays their reliability envelope until strategy collapse occurs. Sotis acts as an active runtime stabilizer, monitoring execution in real time, detecting behavioral meltdowns, and transparently resetting context to restore forward progress.
+
+---
+
+## Active Execution Recovery in 3 sets of Lines
+
+Unlike passive tracing systems, Sotis operates actively. It intercepts spiraling tool calls, restores files to the last stable checkpoint, compresses history, and injects a resumption briefing.
+
+```python
+from sotis import SotisGuard
+
+# 1. Initialize the reliability guard
+guard = SotisGuard()
+
+# 2. Watch tool execution events in your agent's loop
+for step in range(max_steps):
+    action = agent.decide()
+    result = tools.execute(action)
+    
+    meltdown = guard.watch(action.name, action.args, result.summary)
+    
+    # 3. Intercept and recover automatically
+    if meltdown:
+        print("Sotis blocked an execution loop! Restoring stable baseline...")
+        guard.reset()
+```
+
+### Raw Telemetry Trace
+
+Here is how Sotis stabilizes execution in real time:
+
+```text
+[Step 22] write_file -> {"path": "src/main.py", "content": "import math"} | Outcome: SUCCESS
+[Step 23] run_tests  -> {"cmd": "pytest"} | Outcome: FAIL (ImportError)
+[Step 24] write_file -> {"path": "src/main.py", "content": "import math"} | Outcome: SUCCESS
+[Step 25] run_tests  -> {"cmd": "pytest"} | Outcome: FAIL (ImportError)
+
+[WARNING] Anomaly detected: Workspace edit storm and exact argument loops
+[INTERCEPT] Sotis Meltdown Interception Triggered!
+[RECOVER] Restored workspace files to stable baseline step 22 diff
+[RECOVER] Distilled session context history (78% token savings)
+[RESUME] Injecting resumption briefing prompt into agent context...
+
+[Step 26] grep_search -> {"query": "math"} | Execution resumed cleanly
+```
+
+---
+
+## Visual Execution Flow
+
+```
+     Agent Begins Task
+             |
+             v
+   Execution Step Loop
+             |
+             v
+   Entropy Rises / Loops Detected (Step 20)
+             |
+             v
+   Sotis Intercepts Meltdown (Step 25)
+             |
+             v
+   Workspace Rolled Back and Context Distilled (>= 60% Token Savings)
+             |
+             v
+   Resumption Briefing Injected
+             |
+             v
+   Execution Resumes Cleanly to Completion
+```
+
+---
+
+## The Science: Research-Inspired Reliability
+
+Sotis operationalizes the formal reliability engineering framework introduced in the April 2026 paper:
+*["Beyond pass@1: A Reliability Science Framework for Long-Horizon LLM Agents"](https://arxiv.org/abs/2603.29231)* (arXiv:2603.29231)
+
+We translate key theoretical concepts from this paper directly into active runtime layers:
+
+* **Meltdown Onset Point (MOP)**: Quantifies the transition from coherent planning to chaotic looping. Sotis monitors the Shannon entropy of the tool-call distribution over a sliding window (size w=5). A spike in entropy or a tight repeating sequence triggers an immediate Meltdown Onset Point flag.
+* **Reliability Decay**: Demonstrates that agent success rates decay super-linearly with task duration due to positively correlated errors (a confused agent stays confused). Sotis acts as a runtime circuit breaker to reset the error correlation coefficient.
+* **Episodic Memory Failures**: The paper proves that naive episodic memory scaffolds universally degrade long-horizon performance by accumulating step budget and context overhead. Sotis maintains reliability through **controlled checkpointed resets** rather than continuous memory accumulation.
+* **Graceful Degradation Score (GDS)**: Evaluates agent trajectories using weighted directed acyclic graphs of subtask completion. When a meltdown is intercepted, Sotis scores partial progress and resets execution context, allowing agents to preserve partial GDS while starting fresh.
+
+---
+
+## Strongest Differentiator: Active Stabilization vs Passive Tracing
+
+Passive AI developer tooling (such as LangSmith, Langfuse, or Helicone) merely monitors failure, logging tokens and rendering traces *after* your agent has already spent $20 looping in production. 
+
+Sotis is closer to **runtime reliability middleware**. It does not merely observe agent degradation—it actively intercepts loops, rolls back uncommitted file edits, distills conversation context, and redirects the model's reasoning loop in real time.
+
+---
+
+## Project Structure
+
+```
+Sotis/
+  sotis/
+    core/             # Pure computation: entropy, loops, checkpoint, decomposition, GDS
+    lib/              # ReAct runtime, LangGraph integration + LLM adapters
+    obs/              # Telemetry app + structured JSON logger
+    bench/            # Benchmark harness and task generators
+  tests/              # High-coverage automated test suite (127 tests)
+  ExperimentLog/      # Real-agent Track 2 stress test run logs
+  performance_metrics.txt     # Empirical proof-of-performance ledger
+  pyproject.toml      # Package configuration
+  requirements.txt    # Core dependencies
+  pytest.ini          # Testing settings
+```
+
+---
+
+## Build Phases
+
+All phases of the project have been built, rigorously tested, and successfully finalized:
+
+| Phase | Focus | Status |
+|---|---|---|
+| **1** | Core Engine: Schemas, Shannon Entropy, Loop Detection | COMPLETED |
+| **2** | Checkpoint, Context Reset and Resumption | COMPLETED |
+| **3** | Task Decomposition and GDS Scorer | COMPLETED |
+| **4** | Telemetry, Streamlit Dashboard and Benchmark Harness | COMPLETED |
+| **5** | LangGraph Node Middleware Integration and Packaging | COMPLETED |
+| **6** | Workspace Density Guard and AST Query Engine Stress Testing | COMPLETED |
+| **7** | Streamlit Dashboard Polish and Interactive Log Viewer | COMPLETED |
+| **8** | Three-Line Developer Facade API and End-to-End Validation | COMPLETED |
+| **9** | GitHub Release Preparation and Packaging Metadata | COMPLETED |
+| **10** | Live Web Research Stress Test and Crawler Tools | COMPLETED |
+| **11** | Document Handling and Jaccard Semantic Loops (PDF, Excel, Word, CSV) | COMPLETED |
+
+---
+
+## Performance Metrics
+
+See [`performance_metrics.txt`](https://github.com/Shaurya-34/Sotis/blob/main/performance_metrics.txt) — a real-time updated ledger tracking empirical outcomes across all phases showing:
+- **Entropy and Loop Detection Latency**: < 0.2ms overhead per step.
+- **Context Distillation Reduction**: **86.14%** token savings in the resumption briefing.
+- **Resilience Gains**: Real-world recovery on circular imports and AST recursive loop traps.

sotis-1.0.1/README.md

@@ -0,0 +1,156 @@
+# Sotis
+
+Sotis is a Python reliability library for long-running LLM agents that detects meltdown behavior and automatically resets execution before agents spiral into loops or context collapse.
+
+---
+
+## The Long-Horizon Failure Problem
+
+Current AI agents fail predictably under long-horizon execution. As they run for longer periods, they accumulate error and drift into terminal failure modes:
+
+* **Infinite Loops**: Repeating the same tool calls with identical arguments.
+* **Semantic Spirals**: Rephrasing failed queries or tool calls hoping for different outcomes.
+* **Context Poisoning**: Flooding the chat history with massive error traces and linter warnings.
+* **Edit Storms**: Making rapid, uncoordinated file edits without shifting validation outputs.
+
+Frontier models do not fail because they are simple, but because long-horizon execution decays their reliability envelope until strategy collapse occurs. Sotis acts as an active runtime stabilizer, monitoring execution in real time, detecting behavioral meltdowns, and transparently resetting context to restore forward progress.
+
+---
+
+## Active Execution Recovery in 3 sets of Lines
+
+Unlike passive tracing systems, Sotis operates actively. It intercepts spiraling tool calls, restores files to the last stable checkpoint, compresses history, and injects a resumption briefing.
+
+```python
+from sotis import SotisGuard
+
+# 1. Initialize the reliability guard
+guard = SotisGuard()
+
+# 2. Watch tool execution events in your agent's loop
+for step in range(max_steps):
+    action = agent.decide()
+    result = tools.execute(action)
+    
+    meltdown = guard.watch(action.name, action.args, result.summary)
+    
+    # 3. Intercept and recover automatically
+    if meltdown:
+        print("Sotis blocked an execution loop! Restoring stable baseline...")
+        guard.reset()
+```
+
+### Raw Telemetry Trace
+
+Here is how Sotis stabilizes execution in real time:
+
+```text
+[Step 22] write_file -> {"path": "src/main.py", "content": "import math"} | Outcome: SUCCESS
+[Step 23] run_tests  -> {"cmd": "pytest"} | Outcome: FAIL (ImportError)
+[Step 24] write_file -> {"path": "src/main.py", "content": "import math"} | Outcome: SUCCESS
+[Step 25] run_tests  -> {"cmd": "pytest"} | Outcome: FAIL (ImportError)
+
+[WARNING] Anomaly detected: Workspace edit storm and exact argument loops
+[INTERCEPT] Sotis Meltdown Interception Triggered!
+[RECOVER] Restored workspace files to stable baseline step 22 diff
+[RECOVER] Distilled session context history (78% token savings)
+[RESUME] Injecting resumption briefing prompt into agent context...
+
+[Step 26] grep_search -> {"query": "math"} | Execution resumed cleanly
+```
+
+---
+
+## Visual Execution Flow
+
+```
+     Agent Begins Task
+             |
+             v
+   Execution Step Loop
+             |
+             v
+   Entropy Rises / Loops Detected (Step 20)
+             |
+             v
+   Sotis Intercepts Meltdown (Step 25)
+             |
+             v
+   Workspace Rolled Back and Context Distilled (>= 60% Token Savings)
+             |
+             v
+   Resumption Briefing Injected
+             |
+             v
+   Execution Resumes Cleanly to Completion
+```
+
+---
+
+## The Science: Research-Inspired Reliability
+
+Sotis operationalizes the formal reliability engineering framework introduced in the April 2026 paper:
+*["Beyond pass@1: A Reliability Science Framework for Long-Horizon LLM Agents"](https://arxiv.org/abs/2603.29231)* (arXiv:2603.29231)
+
+We translate key theoretical concepts from this paper directly into active runtime layers:
+
+* **Meltdown Onset Point (MOP)**: Quantifies the transition from coherent planning to chaotic looping. Sotis monitors the Shannon entropy of the tool-call distribution over a sliding window (size w=5). A spike in entropy or a tight repeating sequence triggers an immediate Meltdown Onset Point flag.
+* **Reliability Decay**: Demonstrates that agent success rates decay super-linearly with task duration due to positively correlated errors (a confused agent stays confused). Sotis acts as a runtime circuit breaker to reset the error correlation coefficient.
+* **Episodic Memory Failures**: The paper proves that naive episodic memory scaffolds universally degrade long-horizon performance by accumulating step budget and context overhead. Sotis maintains reliability through **controlled checkpointed resets** rather than continuous memory accumulation.
+* **Graceful Degradation Score (GDS)**: Evaluates agent trajectories using weighted directed acyclic graphs of subtask completion. When a meltdown is intercepted, Sotis scores partial progress and resets execution context, allowing agents to preserve partial GDS while starting fresh.
+
+---
+
+## Strongest Differentiator: Active Stabilization vs Passive Tracing
+
+Passive AI developer tooling (such as LangSmith, Langfuse, or Helicone) merely monitors failure, logging tokens and rendering traces *after* your agent has already spent $20 looping in production. 
+
+Sotis is closer to **runtime reliability middleware**. It does not merely observe agent degradation—it actively intercepts loops, rolls back uncommitted file edits, distills conversation context, and redirects the model's reasoning loop in real time.
+
+---
+
+## Project Structure
+
+```
+Sotis/
+  sotis/
+    core/             # Pure computation: entropy, loops, checkpoint, decomposition, GDS
+    lib/              # ReAct runtime, LangGraph integration + LLM adapters
+    obs/              # Telemetry app + structured JSON logger
+    bench/            # Benchmark harness and task generators
+  tests/              # High-coverage automated test suite (127 tests)
+  ExperimentLog/      # Real-agent Track 2 stress test run logs
+  performance_metrics.txt     # Empirical proof-of-performance ledger
+  pyproject.toml      # Package configuration
+  requirements.txt    # Core dependencies
+  pytest.ini          # Testing settings
+```
+
+---
+
+## Build Phases
+
+All phases of the project have been built, rigorously tested, and successfully finalized:
+
+| Phase | Focus | Status |
+|---|---|---|
+| **1** | Core Engine: Schemas, Shannon Entropy, Loop Detection | COMPLETED |
+| **2** | Checkpoint, Context Reset and Resumption | COMPLETED |
+| **3** | Task Decomposition and GDS Scorer | COMPLETED |
+| **4** | Telemetry, Streamlit Dashboard and Benchmark Harness | COMPLETED |
+| **5** | LangGraph Node Middleware Integration and Packaging | COMPLETED |
+| **6** | Workspace Density Guard and AST Query Engine Stress Testing | COMPLETED |
+| **7** | Streamlit Dashboard Polish and Interactive Log Viewer | COMPLETED |
+| **8** | Three-Line Developer Facade API and End-to-End Validation | COMPLETED |
+| **9** | GitHub Release Preparation and Packaging Metadata | COMPLETED |
+| **10** | Live Web Research Stress Test and Crawler Tools | COMPLETED |
+| **11** | Document Handling and Jaccard Semantic Loops (PDF, Excel, Word, CSV) | COMPLETED |
+
+---
+
+## Performance Metrics
+
+See [`performance_metrics.txt`](https://github.com/Shaurya-34/Sotis/blob/main/performance_metrics.txt) — a real-time updated ledger tracking empirical outcomes across all phases showing:
+- **Entropy and Loop Detection Latency**: < 0.2ms overhead per step.
+- **Context Distillation Reduction**: **86.14%** token savings in the resumption briefing.
+- **Resilience Gains**: Real-world recovery on circular imports and AST recursive loop traps.

sotis-1.0.1/pyproject.toml

@@ -0,0 +1,38 @@
+[build-system]
+requires = ["setuptools>=61.0.0"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "sotis"
+version = "1.0.1"
+description = "Python reliability library for long-running LLM agents"
+readme = "README.md"
+requires-python = ">=3.10"
+dependencies = [
+    "numpy>=1.26.0",
+    "pydantic>=2.6.0",
+    "python-dateutil>=2.9.0",
+    "openai>=1.25.0",
+    "anthropic>=0.25.0",
+    "rich>=13.7.0"
+]
+
+[project.optional-dependencies]
+dev = [
+    "pytest>=8.2.0",
+    "pytest-benchmark>=4.0.0",
+    "pytest-cov>=5.0.0",
+    "langchain>=0.2.0",
+    "langgraph>=0.0.60"
+]
+langgraph = [
+    "langchain>=0.2.0",
+    "langgraph>=0.0.60"
+]
+obs = [
+    "streamlit>=1.35.0"
+]
+
+[tool.setuptools.packages.find]
+where = ["."]
+include = ["sotis*"]

sotis-1.0.1/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

sotis-1.0.1/sotis/__init__.py

@@ -0,0 +1,101 @@
+"""
+Sotis — MOP-Triggered Context Reset Agent
+==========================================
+Reliability middleware that monitors long-horizon LLM agent sessions,
+detects meltdowns via sliding-window Shannon entropy analysis, and
+transparently resets context to restore forward progress.
+
+Based on: "Beyond pass@1: A Reliability Science Framework for
+           Long-Horizon LLM Agents" (arXiv:2603.29231v1)
+"""
+
+from __future__ import annotations
+
+from typing import Any, Dict, Optional, Union
+
+from sotis.core.schemas import StepEvent, MeltdownSignal, Domain, MeltdownReason
+from sotis.core.entropy import SessionEntropyTracker, EntropyConfig
+from sotis.core.loops import SessionLoopTracker, LoopConfig, WorkspaceDensityGuard
+
+__version__ = "0.1.0"
+__author__ = "Sotis Contributors"
+
+
+class SotisGuard:
+    """
+    SotisGuard
+    ==========
+    High-level user-facing developer facade designed to watch agent trajectories
+    and detect meltdown conditions in 3 lines of code.
+
+    Usage:
+    ------
+    >>> from sotis import SotisGuard
+    >>> guard = SotisGuard()
+    >>> meltdown = guard.watch("write_file", {"file_path": "app.py"}, "Written successfully")
+    """
+
+    def __init__(
+        self,
+        entropy_config: Optional[EntropyConfig] = None,
+        loop_config: Optional[LoopConfig] = None,
+        max_consecutive_edits: int = 3,
+    ) -> None:
+        self.entropy_tracker = SessionEntropyTracker(entropy_config)
+        self.loop_tracker = SessionLoopTracker(loop_config)
+        self.density_guard = WorkspaceDensityGuard(max_consecutive_edits)
+        self._step_counter = 0
+
+    def watch(
+        self,
+        tool_name: Union[str, StepEvent],
+        tool_args: Optional[Dict[str, Any]] = None,
+        result_summary: Optional[str] = None,
+        subtask_id: Optional[str] = None,
+    ) -> bool:
+        """
+        Record a step event and evaluate it for meltdown patterns.
+
+        Returns True if a tool-loop, high-entropy, or edit-density meltdown is triggered.
+        """
+        if isinstance(tool_name, StepEvent):
+            event = tool_name
+        else:
+            event = StepEvent(
+                step_index=self._step_counter,
+                tool_name=tool_name,
+                tool_args=tool_args or {},
+                result_summary=result_summary,
+                subtask_id=subtask_id,
+            )
+            self._step_counter += 1
+
+        # Push to all three monitors
+        entropy_res = self.entropy_tracker.push_event(event)
+        loop_res = self.loop_tracker.push_event(event)
+        density_res = self.density_guard.push_event(event)
+
+        # Returns True if any meltdown is triggered
+        return (
+            entropy_res.meltdown_detected or
+            loop_res.meltdown_detected or
+            density_res
+        )
+
+    def reset(self) -> None:
+        """Resets the state of all monitors."""
+        self.entropy_tracker.reset()
+        self.loop_tracker.reset()
+        self.density_guard.reset()
+        self._step_counter = 0
+
+
+__all__ = [
+    "SotisGuard",
+    "StepEvent",
+    "MeltdownSignal",
+    "Domain",
+    "MeltdownReason",
+    "EntropyConfig",
+    "LoopConfig",
+]