scope-optimizer 0.1.0__tar.gz

scope_optimizer-0.1.0/LICENSE

@@ -0,0 +1,22 @@
+MIT License
+
+Copyright (c) 2024 SCOPE Team
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+

scope_optimizer-0.1.0/PKG-INFO

@@ -0,0 +1,380 @@
+Metadata-Version: 2.4
+Name: scope-optimizer
+Version: 0.1.0
+Summary: SCOPE: Self-evolving Context Optimization via Prompt Evolution - A framework for automatic prompt optimization
+Author: SCOPE Team
+License: MIT
+Project-URL: Homepage, https://github.com/JarvisPei/SCOPE
+Project-URL: Documentation, https://github.com/JarvisPei/SCOPE#readme
+Project-URL: Repository, https://github.com/JarvisPei/SCOPE
+Project-URL: Issues, https://github.com/JarvisPei/SCOPE/issues
+Keywords: llm,prompt-optimization,agents,machine-learning,nlp,prompt-engineering
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: openai>=1.0.0
+Requires-Dist: anthropic>=0.18.0
+Requires-Dist: litellm>=1.0.0
+Requires-Dist: python-dotenv>=1.0.0
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0.0; extra == "dev"
+Requires-Dist: pytest-asyncio>=0.21.0; extra == "dev"
+Requires-Dist: pytest-cov>=4.0.0; extra == "dev"
+Requires-Dist: black>=23.0.0; extra == "dev"
+Requires-Dist: ruff>=0.1.0; extra == "dev"
+Requires-Dist: mypy>=1.0.0; extra == "dev"
+Requires-Dist: pre-commit>=3.0.0; extra == "dev"
+Dynamic: license-file
+
+<p align="center">
+  <img src="assets/SCOPE_logo.png" alt="SCOPE Logo" width="400">
+</p>
+
+<p align="center">
+  <strong>Self-evolving Context Optimization via Prompt Evolution</strong>
+</p>
+
+<p align="center">
+  A framework for automatic prompt optimization that learns from agent execution traces
+</p>
+
+<p align="center">
+  <a href="#installation">Installation</a> •
+  <a href="#quick-start">Quick Start</a> •
+  <a href="#how-it-works">How It Works</a> •
+  <a href="#api-reference">API</a> •
+  <a href="#configuration">Configuration</a>
+</p>
+
+<p align="center">
+  <a href="https://arxiv.org/abs/2512.15374"><img src="https://img.shields.io/badge/arXiv-2512.15374-b31b1b.svg" alt="arXiv"></a>
+  <a href="https://pypi.org/project/scope-optimizer/"><img src="https://img.shields.io/pypi/v/scope-optimizer.svg" alt="PyPI version"></a>
+  <img src="https://img.shields.io/badge/python-3.8+-blue.svg" alt="Python 3.8+">
+  <a href="https://github.com/JarvisPei/SCOPE/actions"><img src="https://github.com/JarvisPei/SCOPE/workflows/Tests/badge.svg" alt="Tests"></a>
+  <img src="https://img.shields.io/badge/license-MIT-green.svg" alt="License: MIT">
+  <a href="CONTRIBUTING.md"><img src="https://img.shields.io/badge/PRs-welcome-brightgreen.svg" alt="PRs Welcome"></a>
+</p>
+
+---
+
+## Overview
+
+**SCOPE** transforms static agent prompts into self-evolving systems that learn from their own execution. Instead of manually crafting prompts, SCOPE automatically synthesizes guidelines from execution traces and continuously improves agent performance.
+
+📄 **Paper:** [SCOPE: Prompt Evolution for Enhancing Agent Effectiveness](https://arxiv.org/abs/2512.15374)
+
+**Key Features:**
+- 🔄 **Automatic Learning** — Synthesizes guidelines from errors and successful patterns
+- 📊 **Dual-Stream Memory** — Tactical (task-specific) + Strategic (cross-task) learning
+- 🎯 **Best-of-N Selection** — Generates multiple candidates and selects the best
+- 🧠 **Memory Optimization** — Automatically consolidates and deduplicates rules
+- 🔌 **Universal Model Support** — Works with OpenAI, Anthropic, and 100+ providers via LiteLLM
+
+## Installation
+
+```bash
+pip install scope-optimizer
+```
+
+**From source:**
+
+```bash
+git clone https://github.com/JarvisPei/SCOPE.git
+cd scope
+pip install -e .
+```
+
+## Quick Start
+
+```python
+import asyncio
+from dotenv import load_dotenv
+from scope import SCOPEOptimizer
+from scope.models import create_openai_model
+
+load_dotenv()  # Load API keys from .env
+
+async def main():
+    model = create_openai_model("gpt-4o-mini")
+    optimizer = SCOPEOptimizer(
+        synthesizer_model=model,
+        exp_path="./scope_data",  # Strategic rules persist here
+    )
+    
+    # Initialize prompt with previously learned strategic rules
+    base_prompt = "You are a helpful assistant."
+    strategic_rules = optimizer.get_strategic_rules_for_agent("my_agent")
+    current_prompt = base_prompt + strategic_rules  # Applies cross-task knowledge
+    
+    while not task_complete:
+        # ... your agent logic ...
+        
+        # Call SCOPE after each step
+        result = await optimizer.on_step_complete(
+            agent_name="my_agent",
+            agent_role="AI Assistant", 
+            task="Answer user questions",
+            model_output="...",
+            error=error_if_any,  # Pass errors when they occur
+            current_system_prompt=current_prompt,
+            task_id="task_001",
+        )
+        
+        # Apply generated guideline
+        if result:
+            guideline, guideline_type = result  # guideline_type: "tactical" or "strategic"
+            current_prompt += f"\n\n## Learned Guideline:\n{guideline}"
+
+asyncio.run(main())
+```
+
+## How It Works
+
+SCOPE operates through four key mechanisms:
+
+### 1. Guideline Synthesis (π_φ, π_σ)
+
+When errors occur or quality issues are detected, SCOPE **generates** multiple candidate guidelines using the Generator (π_φ) and **selects** the best candidate using the Selector (π_σ).
+
+### 2. Dual-Stream Routing (π_γ)
+
+Guidelines are classified and routed to appropriate memory:
+
+| Stream | Scope | Persistence | Example |
+|--------|-------|-------------|---------|
+| **Tactical** | Task-specific | In-memory only | "This API has rate limit of 10/min" |
+| **Strategic** | Cross-task | Saved to disk | "Always validate JSON before parsing" |
+
+### 3. Memory Optimization (π_ω)
+
+Strategic memory is automatically optimized via conflict resolution, subsumption pruning, and consolidation.
+
+### 4. Prompt Evolution
+
+```
+θ_new = θ_base ⊕ M_strategic ⊕ M_tactical
+```
+
+
+## API Reference
+
+### SCOPEOptimizer
+
+```python
+optimizer = SCOPEOptimizer(
+    # Required parameters
+    synthesizer_model,              # Model instance for guideline synthesis (e.g., gpt-4o-mini)
+    exp_path="./scope_data",        # Path for storing strategic rules and history
+    
+    # Analysis settings
+    enable_quality_analysis=True,   # Whether to analyze successful steps for improvements (default: True)
+    quality_analysis_frequency=1,   # Analyze quality every N successful steps (default: 1)
+    auto_accept_threshold="medium", # Confidence threshold: "all", "low", "medium", "high" (default: "medium")
+    
+    # Memory settings
+    max_rules_per_task=20,          # Max tactical rules to apply per task (default: 20)
+    strategic_confidence_threshold=0.85,  # Min confidence for strategic promotion (default: 0.85)
+    max_strategic_rules_per_domain=10,    # Max strategic rules per domain per agent (default: 10)
+    
+    # Synthesis settings
+    synthesis_mode="thoroughness",  # "efficiency" (fast) or "thoroughness" (comprehensive, default)
+    use_best_of_n=False,            # Enable Best-of-N candidate selection (default: False)
+    candidate_models=None,          # Additional models for Best-of-N (default: None)
+    
+    # Advanced settings
+    optimizer_model=None,           # Separate model for rule optimization (default: synthesizer_model)
+    enable_rule_optimization=True,  # Auto-optimize strategic memory when full (default: True)
+    store_history=False,            # Store guideline generation history to disk (default: False)
+)
+```
+
+### on_step_complete
+
+```python
+# Call after each agent step
+result = await optimizer.on_step_complete(
+    # Required parameters
+    agent_name="my_agent",          # Unique identifier for the agent
+    agent_role="AI Assistant",      # Role/description of the agent
+    task="Complete user request",   # Current task description
+    
+    # Step context (at least one of error/model_output/observations required)
+    model_output="Agent's response...",  # Model's output text (default: None)
+    tool_calls="[{...}]",           # Tool calls attempted as string (default: None)
+    observations="Tool results...", # Observations/tool results received (default: None)
+    error=exception_if_any,         # Exception if step failed (default: None)
+    
+    # Prompt context
+    current_system_prompt=prompt,   # Current system prompt including strategic rules
+    
+    # Optional settings
+    task_id="task_001",             # Task identifier for tracking (default: None)
+    truncate_context=True,          # Truncate long context for efficiency (default: True)
+)
+
+# Returns: Tuple[str, str] or None
+# - On success: (guideline_text, guideline_type) where guideline_type is "tactical" or "strategic"
+# - On skip/failure: None
+```
+
+### Loading Strategic Rules
+
+```python
+# Load strategic rules at agent initialization (critical for cross-task learning!)
+strategic_rules = optimizer.get_strategic_rules_for_agent("my_agent")
+initial_prompt = base_prompt + strategic_rules  # Apply learned knowledge
+```
+
+Strategic rules are stored in `{exp_path}/strategic_memory/global_rules.json` and automatically loaded when you call `get_strategic_rules_for_agent()`.
+
+
+### Model Adapters
+
+```python
+from scope.models import create_openai_model, create_anthropic_model, create_litellm_model
+
+# OpenAI
+model = create_openai_model("gpt-4o-mini")
+
+# Anthropic
+model = create_anthropic_model("claude-3-5-sonnet-20241022")
+
+# LiteLLM (100+ providers)
+model = create_litellm_model("gpt-4o-mini")           # OpenAI
+model = create_litellm_model("gemini/gemini-1.5-pro") # Google
+model = create_litellm_model("ollama/llama2")         # Local
+```
+
+### Custom Model Adapter
+
+```python
+# Async adapter (default)
+from scope.models import BaseModelAdapter, Message, ModelResponse
+
+class MyAsyncAdapter(BaseModelAdapter):
+    async def generate(self, messages: List[Message]) -> ModelResponse:
+        result = await my_api_call(messages)
+        return ModelResponse(content=result)  # Return raw text
+
+# Sync adapter (for non-async code)
+from scope.models import SyncModelAdapter
+
+class MySyncAdapter(SyncModelAdapter):
+    def generate_sync(self, messages: List[Message]) -> ModelResponse:
+        result = requests.post(api_url, json={"messages": ...})
+        return ModelResponse(content=result.json()["text"])
+
+# Or wrap any function (sync or async)
+from scope.models import CallableModelAdapter
+
+def my_model(messages):
+    return "response"
+
+model = CallableModelAdapter(my_model)
+```
+
+> **Note:** Your adapter just returns the raw model output. SCOPE's prompts ask the model to return JSON, and SCOPE handles parsing internally.
+
+## Configuration
+
+### Environment Variables
+
+Set API keys via environment variables or `.env` file:
+
+```bash
+# Copy template and edit
+cp .env.template .env
+```
+
+```python
+from dotenv import load_dotenv
+load_dotenv()  # API keys automatically loaded
+```
+
+See [`.env.template`](.env.template) for all supported providers.
+
+### Confidence Thresholds
+
+| Threshold | Accepts | Use Case |
+|-----------|---------|----------|
+| `"all"` | Everything | Aggressive learning |
+| `"low"` | Low + Medium + High | Balanced |
+| `"medium"` | Medium + High | Conservative (default) |
+| `"high"` | High only | Very conservative |
+
+### Synthesis Modes
+
+| Mode | Description |
+|------|-------------|
+| `"thoroughness"` | Comprehensive 7-dimension analysis (default) |
+| `"efficiency"` | Lightweight, faster analysis |
+
+### Logging
+
+```python
+import logging
+
+logging.getLogger("scope").setLevel(logging.INFO)
+logging.getLogger("scope").addHandler(logging.StreamHandler())
+```
+
+## Testing
+
+Verify your setup with the included test scripts:
+
+```bash
+# Quick connectivity test
+python examples/test_simple.py
+
+# Deep functionality test
+python examples/test_scope_deep.py
+
+# With custom model/provider
+python examples/test_simple.py --model gpt-4o --provider openai
+python examples/test_scope_deep.py --model claude-3-5-sonnet-20241022 --provider anthropic
+```
+
+Run `--help` for all options.
+
+## Development
+
+```bash
+# Install with dev dependencies
+pip install -e ".[dev]"
+
+# Run tests
+pytest tests/
+
+# Format code
+black scope/
+ruff check scope/
+```
+
+## Citation
+
+If you find SCOPE useful for your research, please cite our paper:
+
+```bibtex
+@article{pei2025scope,
+  title={SCOPE: Prompt Evolution for Enhancing Agent Effectiveness},
+  author={Pei, Zehua and Zhen, Hui-Ling and Kai, Shixiong and Pan, Sinno Jialin and Wang, Yunhe and Yuan, Mingxuan and Yu, Bei},
+  journal={arXiv preprint arXiv:2512.15374},
+  year={2025}
+}
+```
+
+## License
+
+MIT License - see [LICENSE](LICENSE) for details.