opik-optimizer 1.1.0__tar.gz → 2.0.1__tar.gz

{opik_optimizer-1.1.0 → opik_optimizer-2.0.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: opik_optimizer
-Version: 1.1.0
+Version: 2.0.1
 Summary: Agent optimization with Opik
 Home-page: https://github.com/comet-ml/opik
 Author: Comet ML
@@ -15,6 +15,7 @@ Requires-Dist: datasets
 Requires-Dist: deap>=1.4.3
 Requires-Dist: diskcache
 Requires-Dist: dspy<3
+Requires-Dist: gepa>=0.0.7
 Requires-Dist: ujson
 Requires-Dist: hf_xet
 Requires-Dist: litellm
@@ -30,7 +31,7 @@ Provides-Extra: dev
 Requires-Dist: pytest; extra == "dev"
 Requires-Dist: pytest-cov; extra == "dev"
 Requires-Dist: langgraph; extra == "dev"
-Requires-Dist: gepa>=0.0.7; extra == "dev"
+Requires-Dist: pre-commit; extra == "dev"
 Dynamic: author
 Dynamic: home-page
 Dynamic: license-file
@@ -43,13 +44,24 @@ Dynamic: requires-python
 [![Downloads](https://static.pepy.tech/badge/opik-optimizer)](https://pepy.tech/project/opik-optimizer)
 [![License](https://img.shields.io/github/license/comet-ml/opik)](https://github.com/comet-ml/opik/blob/main/LICENSE)
 
-The Opik Agent Optimizer refines your prompts to achieve better performance from your Large Language Models (LLMs). It supports a variety of optimization algorithms, including:
+The Opik Agent Optimizer refines your prompts to achieve better performance from your Large Language Models (LLMs). It supports a variety of optimization algorithms, all with a **standardized API** for consistent usage and chaining:
 
-* EvolutionaryOptimizer
-* FewShotBayesianOptimizer
-* MetaPromptOptimizer
-* MiproOptimizer
-* GepaOptimizer
+* **EvolutionaryOptimizer** - Uses genetic algorithms for prompt evolution
+* **FewShotBayesianOptimizer** - Uses few-shot learning with Bayesian optimization
+* **MetaPromptOptimizer** - Employs meta-prompting techniques for optimization
+* **MiproOptimizer** - Implements MIPRO (Multi-Input Prompt Optimization) algorithm
+* **GepaOptimizer** - Leverages GEPA (Genetic-Pareto) optimization approach
+
+## 🎯 Key Features
+
+- **Standardized API**: All optimizers follow the same interface for `optimize_prompt()` and `optimize_mcp()` methods
+- **Optimizer Chaining**: Results from one optimizer can be used as input for another
+- **MCP Support**: Built-in support for Model Context Protocol tool calling
+- **Consistent Results**: All optimizers return standardized `OptimizationResult` objects
+- **Counter Tracking**: Built-in LLM and tool call counters for monitoring usage
+- **Type Safety**: Full type hints and validation for robust development
+- **Backward Compatibility**: All original parameters preserved through kwargs extraction
+- **Deprecation Warnings**: Clear warnings for deprecated parameters with migration guidance
 
 Opik Optimizer is a component of the [Opik platform](https://github.com/comet-ml/opik), an open-source LLM evaluation platform by Comet.
 For more information about the broader Opik ecosystem, visit our [Website](https://www.comet.com/site/products/opik/) or [Documentation](https://www.comet.com/docs/opik/).
@@ -101,6 +113,40 @@ You'll typically need:
 *   An [Opik Metric](https://www.comet.com/docs/opik/evaluation/metrics/overview/) (or a custom evaluation function).
 *   A starting prompt (template string).
 
+## Standardized API
+
+All optimizers follow the same interface, making it easy to switch between algorithms or chain them together:
+
+```python
+# All optimizers have the same signature
+def optimize_prompt(
+    self,
+    prompt: ChatPrompt,
+    dataset: Dataset,
+    metric: Callable,
+    experiment_config: dict | None = None,
+    n_samples: int | None = None,
+    auto_continue: bool = False,
+    agent_class: type[OptimizableAgent] | None = None,
+    **kwargs: Any,
+) -> OptimizationResult
+
+# All optimizers return the same result type
+result = optimizer.optimize_prompt(
+    prompt=prompt,
+    dataset=dataset,
+    metric=metric,
+    n_samples=100
+)
+
+# Results can be chained
+chained_result = another_optimizer.optimize_prompt(
+    prompt=ChatPrompt.from_result(result),  # Use previous result
+    dataset=dataset,
+    metric=metric
+)
+```
+
 ## Example
 
 Here's a brief example of how to use the `FewShotBayesianOptimizer`. We'll use a sample dataset provided by Opik.
@@ -146,8 +192,7 @@ result = optimizer.optimize_prompt(
     prompt=prompt,
     dataset=hot_pot_dataset,
     metric=levenshtein_ratio,
-    n_trials=10,   # Number of optimization trials
-    n_samples=150, # Number of dataset samples for evaluation per trial
+    n_samples=150  # Number of dataset samples for evaluation per trial
 )
 
 # Display the best prompt and its score
@@ -155,6 +200,8 @@ result.display()
 ```
 The `result` object contains the optimized prompt, evaluation scores, and other details from the optimization process. If `project_name` is provided and Opik is configured, results will also be logged to your Comet workspace.
 
+The optimizer automatically logs run metadata—including optimizer version, tool schemas, prompt messages, and the models used—so you get consistent experiment context without any additional arguments. If you still need custom tags (for example identifying the dataset or task), pass an `experiment_config` dictionary and your fields will be merged on top of the defaults.
+
 ## Tool Optimization (MCP) - Beta
 
 The Opik Agent Optimizer supports **true tool optimization** for MCP (Model Context Protocol) tools. This feature is currently in **Beta** and supported by the **MetaPrompt Optimizer**.
@@ -215,6 +262,46 @@ optimizer = MetaPromptOptimizer(model="gpt-4")
 
 For comprehensive documentation on tool optimization, see the [Tool Optimization Guide](https://www.comet.com/docs/opik/agent_optimization/algorithms/tool_optimization).
 
+## Deprecation Warnings
+
+The following parameters are deprecated and will be removed in future versions:
+
+### Constructor Parameters
+
+- **`project_name`** in optimizer constructors: Set `project_name` in the `ChatPrompt` instead
+- **`num_threads`** in optimizer constructors: Use `n_threads` instead
+
+### Example Migration
+
+```python
+# ❌ Deprecated
+optimizer = FewShotBayesianOptimizer(
+    model="gpt-4o-mini",
+    project_name="my-project",  # Deprecated
+    num_threads=16,             # Deprecated
+)
+
+# ✅ Correct
+optimizer = FewShotBayesianOptimizer(
+    model="gpt-4o-mini",
+    n_threads=16,  # Use n_threads instead
+)
+
+prompt = ChatPrompt(
+    project_name="my-project",  # Set here instead
+    messages=[...]
+)
+```
+
+### Suppressing Deprecation Warnings
+
+To suppress deprecation warnings during development:
+
+```python
+import warnings
+warnings.filterwarnings("ignore", category=DeprecationWarning)
+```
+
 ### MCP Integration (Beta)
 
 The optimizer includes utilities for MCP tool integration:

{opik_optimizer-1.1.0 → opik_optimizer-2.0.1}/README.md

@@ -5,13 +5,24 @@
 [![Downloads](https://static.pepy.tech/badge/opik-optimizer)](https://pepy.tech/project/opik-optimizer)
 [![License](https://img.shields.io/github/license/comet-ml/opik)](https://github.com/comet-ml/opik/blob/main/LICENSE)
 
-The Opik Agent Optimizer refines your prompts to achieve better performance from your Large Language Models (LLMs). It supports a variety of optimization algorithms, including:
+The Opik Agent Optimizer refines your prompts to achieve better performance from your Large Language Models (LLMs). It supports a variety of optimization algorithms, all with a **standardized API** for consistent usage and chaining:
 
-* EvolutionaryOptimizer
-* FewShotBayesianOptimizer
-* MetaPromptOptimizer
-* MiproOptimizer
-* GepaOptimizer
+* **EvolutionaryOptimizer** - Uses genetic algorithms for prompt evolution
+* **FewShotBayesianOptimizer** - Uses few-shot learning with Bayesian optimization
+* **MetaPromptOptimizer** - Employs meta-prompting techniques for optimization
+* **MiproOptimizer** - Implements MIPRO (Multi-Input Prompt Optimization) algorithm
+* **GepaOptimizer** - Leverages GEPA (Genetic-Pareto) optimization approach
+
+## 🎯 Key Features
+
+- **Standardized API**: All optimizers follow the same interface for `optimize_prompt()` and `optimize_mcp()` methods
+- **Optimizer Chaining**: Results from one optimizer can be used as input for another
+- **MCP Support**: Built-in support for Model Context Protocol tool calling
+- **Consistent Results**: All optimizers return standardized `OptimizationResult` objects
+- **Counter Tracking**: Built-in LLM and tool call counters for monitoring usage
+- **Type Safety**: Full type hints and validation for robust development
+- **Backward Compatibility**: All original parameters preserved through kwargs extraction
+- **Deprecation Warnings**: Clear warnings for deprecated parameters with migration guidance
 
 Opik Optimizer is a component of the [Opik platform](https://github.com/comet-ml/opik), an open-source LLM evaluation platform by Comet.
 For more information about the broader Opik ecosystem, visit our [Website](https://www.comet.com/site/products/opik/) or [Documentation](https://www.comet.com/docs/opik/).
@@ -63,6 +74,40 @@ You'll typically need:
 *   An [Opik Metric](https://www.comet.com/docs/opik/evaluation/metrics/overview/) (or a custom evaluation function).
 *   A starting prompt (template string).
 
+## Standardized API
+
+All optimizers follow the same interface, making it easy to switch between algorithms or chain them together:
+
+```python
+# All optimizers have the same signature
+def optimize_prompt(
+    self,
+    prompt: ChatPrompt,
+    dataset: Dataset,
+    metric: Callable,
+    experiment_config: dict | None = None,
+    n_samples: int | None = None,
+    auto_continue: bool = False,
+    agent_class: type[OptimizableAgent] | None = None,
+    **kwargs: Any,
+) -> OptimizationResult
+
+# All optimizers return the same result type
+result = optimizer.optimize_prompt(
+    prompt=prompt,
+    dataset=dataset,
+    metric=metric,
+    n_samples=100
+)
+
+# Results can be chained
+chained_result = another_optimizer.optimize_prompt(
+    prompt=ChatPrompt.from_result(result),  # Use previous result
+    dataset=dataset,
+    metric=metric
+)
+```
+
 ## Example
 
 Here's a brief example of how to use the `FewShotBayesianOptimizer`. We'll use a sample dataset provided by Opik.
@@ -108,8 +153,7 @@ result = optimizer.optimize_prompt(
     prompt=prompt,
     dataset=hot_pot_dataset,
     metric=levenshtein_ratio,
-    n_trials=10,   # Number of optimization trials
-    n_samples=150, # Number of dataset samples for evaluation per trial
+    n_samples=150  # Number of dataset samples for evaluation per trial
 )
 
 # Display the best prompt and its score
@@ -117,6 +161,8 @@ result.display()
 ```
 The `result` object contains the optimized prompt, evaluation scores, and other details from the optimization process. If `project_name` is provided and Opik is configured, results will also be logged to your Comet workspace.
 
+The optimizer automatically logs run metadata—including optimizer version, tool schemas, prompt messages, and the models used—so you get consistent experiment context without any additional arguments. If you still need custom tags (for example identifying the dataset or task), pass an `experiment_config` dictionary and your fields will be merged on top of the defaults.
+
 ## Tool Optimization (MCP) - Beta
 
 The Opik Agent Optimizer supports **true tool optimization** for MCP (Model Context Protocol) tools. This feature is currently in **Beta** and supported by the **MetaPrompt Optimizer**.
@@ -177,6 +223,46 @@ optimizer = MetaPromptOptimizer(model="gpt-4")
 
 For comprehensive documentation on tool optimization, see the [Tool Optimization Guide](https://www.comet.com/docs/opik/agent_optimization/algorithms/tool_optimization).
 
+## Deprecation Warnings
+
+The following parameters are deprecated and will be removed in future versions:
+
+### Constructor Parameters
+
+- **`project_name`** in optimizer constructors: Set `project_name` in the `ChatPrompt` instead
+- **`num_threads`** in optimizer constructors: Use `n_threads` instead
+
+### Example Migration
+
+```python
+# ❌ Deprecated
+optimizer = FewShotBayesianOptimizer(
+    model="gpt-4o-mini",
+    project_name="my-project",  # Deprecated
+    num_threads=16,             # Deprecated
+)
+
+# ✅ Correct
+optimizer = FewShotBayesianOptimizer(
+    model="gpt-4o-mini",
+    n_threads=16,  # Use n_threads instead
+)
+
+prompt = ChatPrompt(
+    project_name="my-project",  # Set here instead
+    messages=[...]
+)
+```
+
+### Suppressing Deprecation Warnings
+
+To suppress deprecation warnings during development:
+
+```python
+import warnings
+warnings.filterwarnings("ignore", category=DeprecationWarning)
+```
+
 ### MCP Integration (Beta)
 
 The optimizer includes utilities for MCP tool integration:

{opik_optimizer-1.1.0 → opik_optimizer-2.0.1}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "opik_optimizer"
-version = "1.1.0"
+version = "2.0.1"
 description = "Agent optimization with Opik"
 authors = [
     {name = "Comet ML", email = "support@comet.com"}
@@ -13,6 +13,7 @@ dependencies = [
     "deap>=1.4.3",
     "diskcache",
     "dspy<3",
+    "gepa>=0.0.7",
     "ujson",
     "hf_xet",
     "litellm",
@@ -32,7 +33,7 @@ dev = [
     "pytest-cov",
     # "google-adk",
     "langgraph",
-    "gepa>=0.0.7",
+    "pre-commit",
 ]
 
 [tool.setuptools.packages.find]
@@ -62,3 +63,12 @@ filterwarnings = [
     "ignore::pytest.PytestConfigWarning",
     "ignore::RuntimeWarning:litellm.integrations.opik.opik",
 ]
+
+[tool.coverage.run]
+branch = true
+source = ["opik_optimizer"]
+omit = ["opik_optimizer/tests/*"]
+
+[tool.coverage.report]
+show_missing = true
+skip_covered = true

{opik_optimizer-1.1.0 → opik_optimizer-2.0.1}/src/opik_optimizer/__init__.py

@@ -15,6 +15,7 @@ from .few_shot_bayesian_optimizer import FewShotBayesianOptimizer
 from .gepa_optimizer import GepaOptimizer
 from .logging_config import setup_logging
 from .meta_prompt_optimizer import MetaPromptOptimizer
+from .mipro_optimizer import MiproOptimizer
 from .optimization_config.configs import TaskConfig
 from .optimization_result import OptimizationResult
 
@@ -31,6 +32,7 @@ __all__ = [
     "FewShotBayesianOptimizer",
     "GepaOptimizer",
     "MetaPromptOptimizer",
+    "MiproOptimizer",
     "EvolutionaryOptimizer",
     "OptimizationResult",
     "OptimizableAgent",