opik-optimizer 1.0.6__py3-none-any.whl → 2.0.0__py3-none-any.whl

opik_optimizer-1.0.6.dist-info/METADATA

@@ -1,181 +0,0 @@
-Metadata-Version: 2.4
-Name: opik_optimizer
-Version: 1.0.6
-Summary: Agent optimization with Opik
-Home-page: https://github.com/comet-ml/opik
-Author: Comet ML
-Author-email: support@comet.com
-Classifier: Development Status :: 3 - Alpha
-Classifier: Intended Audience :: Developers
-Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3.10
-Requires-Python: >=3.9,<3.13
-Description-Content-Type: text/markdown
-License-File: LICENSE
-Requires-Dist: datasets
-Requires-Dist: deap>=1.4.3
-Requires-Dist: diskcache
-Requires-Dist: hf_xet
-Requires-Dist: litellm
-Requires-Dist: opik>=1.7.17
-Requires-Dist: optuna
-Requires-Dist: pandas
-Requires-Dist: pydantic
-Requires-Dist: pyrate-limiter
-Requires-Dist: tqdm
-Provides-Extra: dev
-Requires-Dist: pytest; extra == "dev"
-Requires-Dist: pytest-conv; extra == "dev"
-Dynamic: author
-Dynamic: author-email
-Dynamic: classifier
-Dynamic: description
-Dynamic: description-content-type
-Dynamic: home-page
-Dynamic: license-file
-Dynamic: provides-extra
-Dynamic: requires-dist
-Dynamic: requires-python
-Dynamic: summary
-
-# Opik Agent Optimizer
-
-[![PyPI version](https://img.shields.io/pypi/v/opik-optimizer.svg)](https://pypi.org/project/opik-optimizer/)
-[![Python versions](https://img.shields.io/pypi/pyversions/opik-optimizer.svg)](https://pypi.org/project/opik-optimizer/)
-[![Downloads](https://static.pepy.tech/badge/opik-optimizer)](https://pepy.tech/project/opik-optimizer)
-[![License](https://img.shields.io/github/license/comet-ml/opik)](https://github.com/comet-ml/opik/blob/main/LICENSE)
-
-The Opik Agent Optimizer refines your prompts to achieve better performance from your Large Language Models (LLMs). It supports a variety of optimization algorithms, including:
-
-* EvolutionaryOptimizer
-* FewShotBayesianOptimizer
-* MetaPromptOptimizer
-* MiproOptimizer
-
-Opik Optimizer is a component of the [Opik platform](https://github.com/comet-ml/opik), an open-source LLM evaluation platform by Comet.
-For more information about the broader Opik ecosystem, visit our [Website](https://www.comet.com/site/products/opik/) or [Documentation](https://www.comet.com/docs/opik/).
-
-## Quickstart
-
-Explore Opik Optimizer's capabilities with our interactive notebook:
-
-<a href="https://colab.research.google.com/github/comet-ml/opik/blob/main/sdks/opik_optimizer/notebooks/OpikOptimizerIntro.ipynb">
-  <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open in Colab"/>
-</a>
-
-## Setup
-
-To get started with Opik Optimizer, follow these steps:
-
-1.  **Install the package:**
-    ```bash
-    # using pip
-    pip install opik-optimizer
-
-    # using uv (faster)
-    uv pip install opik-optimizer
-    ```
-
-2.  **Configure Opik (Optional, for advanced features):**
-    If you plan to log optimization experiments to Comet or use Opik Datasets, you'll need to configure the Opik client:
-    ```bash
-    # Install the main Opik CLI (if not already installed)
-    pip install opik
-
-    # Configure your Comet API key and workspace
-    opik configure
-    # When prompted, enter your Opik API key and workspace details.
-    ```
-    Using Opik with Comet allows you to track your optimization runs, compare results, and manage datasets seamlessly.
-
-3.  **Set up LLM Provider API Keys:**
-    Ensure your environment variables are set for the LLM(s) you intend to use. For example, for OpenAI models:
-    ```bash
-    export OPENAI_API_KEY="your_openai_api_key"
-    ```
-    The optimizer utilizes LiteLLM, so you can configure keys for various providers as per LiteLLM's documentation.
-
-You'll typically need:
-
-*   An LLM model name (e.g., "gpt-4o-mini", "claude-3-haiku-20240307").
-*   An [Opik Dataset](https://www.comet.com/docs/opik/evaluation/manage_datasets/) (or a compatible local dataset/data generator).
-*   An [Opik Metric](https://www.comet.com/docs/opik/evaluation/metrics/overview/) (or a custom evaluation function).
-*   A starting prompt (template string).
-
-## Example
-
-Here's a brief example of how to use the `FewShotBayesianOptimizer`. We'll use a sample dataset provided by Opik.
-
-Available sample datasets for testing:
-*   `"tiny-test"`
-*   `"halu-eval-300"`
-*   `"hotpot-300"`
-
-```python
-from opik.evaluation.metrics import LevenshteinRatio
-from opik_optimizer import FewShotBayesianOptimizer, ChatPrompt
-from opik_optimizer.datasets import hotpot_300
-
-# Load a sample dataset
-hot_pot_dataset = hotpot_300()
-
-project_name = "optimize-few-shot-bayesian-hotpot" # For Comet logging
-
-# Define the instruction for your chat prompt.
-# Input parameters from dataset examples will be interpolated into the full prompt.
-prompt = ChatPrompt(
-    project_name=project_name,
-    messages=[
-        {"role": "system", "content": "You are a helpful assistant."},
-        {"role": "user", "content": "{question}"}
-    ]
-)
-
-optimizer = FewShotBayesianOptimizer(
-    model="gpt-4o-mini", # LiteLLM name to use for generation and optimization
-    min_examples=3,      # Min few-shot examples
-    max_examples=8,      # Max few-shot examples
-    n_threads=16,        # Parallel threads for evaluation
-    seed=42,
-)
-
-def levenshtein_ratio(dataset_item, llm_output):
-    return LevenshteinRatio().score(reference=dataset_item["answer"], output=llm_output)
-
-# Run the optimization
-result = optimizer.optimize_prompt(
-    prompt=prompt,
-    dataset=hot_pot_dataset,
-    metric=levenshtein_ratio,
-    n_trials=10,   # Number of optimization trials
-    n_samples=150, # Number of dataset samples for evaluation per trial
-)
-
-# Display the best prompt and its score
-result.display()
-```
-The `result` object contains the optimized prompt, evaluation scores, and other details from the optimization process. If `project_name` is provided and Opik is configured, results will also be logged to your Comet workspace.
-
-## Development
-
-To contribute or use the Opik Optimizer from source:
-
-1.  **Clone the Opik repository:**
-    ```bash
-    git clone git@github.com:comet-ml/opik.git
-    ```
-2.  **Navigate to the optimizer's directory:**
-    ```bash
-    cd opik/sdks/opik_optimizer  # Adjust 'opik' if you cloned into a different folder name
-    ```
-3.  **Install in editable mode (with development dependencies):**
-    ```bash
-    pip install -e .[dev]
-    ```
-    The `[dev]` extra installs dependencies useful for development, such as `pytest`.
-
-## Requirements
-
-- Python `>=3.9,<3.13`
-- Opik API key (recommended for full functionality, configure via `opik configure`)
-- API key for your chosen LLM provider (e.g., OpenAI, Anthropic, Gemini), configured as per LiteLLM guidelines.

opik_optimizer-1.0.6.dist-info/RECORD

@@ -1,50 +0,0 @@
-opik_optimizer/__init__.py,sha256=bAwCV-eTLeA_UljBes8J82QpuAX40elE3tXZ6856Nkw,1105
-opik_optimizer/_throttle.py,sha256=ijZ-8nPSzLnUwNeKjejom5QFMHP80JR_hmpB2b20q4E,1335
-opik_optimizer/base_optimizer.py,sha256=-hFlV7i9gLSR0-ODlUo-_YzNOSBUzxuw52DcCQF7TXQ,6137
-opik_optimizer/cache_config.py,sha256=Xd3NdUsL7bLQWoNe3pESqH4nHucU1iNTSGp-RqbwDog,599
-opik_optimizer/logging_config.py,sha256=kUGr4K1mpfprA7AiJrl9evnJAgzEEw9Jkddo58okCWo,2809
-opik_optimizer/optimizable_agent.py,sha256=4-ZlZh0RDPhWjeGfYDGb-J5ivG_zxZ2Ns30Le2Hv0DU,6073
-opik_optimizer/optimization_result.py,sha256=sLW0TzcawBHEV6c174czLbu_kN7EHA5R9bDuZWDNRXE,7762
-opik_optimizer/py.typed,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
-opik_optimizer/reporting_utils.py,sha256=rutmDk5ATH7lbWvgaiZENWav1WzsIdN0U-xrVXpn4s0,6840
-opik_optimizer/task_evaluator.py,sha256=sOTpvF58owOxY2jfFIgWLdvMoXR-B1_Cpqlx8AgaG40,4322
-opik_optimizer/utils.py,sha256=0lHYqleN2OfOZ5WVqQsfglbCPQpjMxjZUWTueM9Tql4,10591
-opik_optimizer/data/hotpot-500.json,sha256=YXxCtuvYvxSu5u0y4559a6b1qwgAYsWzT_SUKv_21ew,76862
-opik_optimizer/datasets/__init__.py,sha256=V4LVDOaRjwzaYvhdQ3V6CAwFaeKnxyTV1lp_ES9Z31E,691
-opik_optimizer/datasets/ai2_arc.py,sha256=vIxb8qlCAxx4_cz2P7SIqS7flquj--7XVUaDmy12UfY,1440
-opik_optimizer/datasets/cnn_dailymail.py,sha256=RB-oMvnwzTudlrtnBK1maj-OEi7Dh884S81_yLaWeoc,1365
-opik_optimizer/datasets/election_questions.py,sha256=3s0iYLH6gjrsP3-FvI6sbapJloEnudd_3NsBfc38Bis,1189
-opik_optimizer/datasets/gsm8k.py,sha256=WUD-UEgYvJ-k_xZtBmgF5CUUCFUA_uJYuPS8xmRxXMM,1370
-opik_optimizer/datasets/halu_eval.py,sha256=9pI_H13Z0zHc1xx4jIySJ6TC6osgvEQTbFjf-0_GvTg,1433
-opik_optimizer/datasets/hotpot_qa.py,sha256=_VLMCSbJe1-jDq6RAg6rb-Hq5ZfOvI7pQfgiql8DRDM,2200
-opik_optimizer/datasets/medhallu.py,sha256=h2X9SgGP0G5V1FDL25YjFvmOcqDeXllwyHmH5LFrQ4w,1489
-opik_optimizer/datasets/rag_hallucinations.py,sha256=SHQ-wdI8YjaeuSkPPgNUYQ9KmAV648CuLeeDAi5s7Xk,1440
-opik_optimizer/datasets/ragbench.py,sha256=Ltdz3yF1h3rfjr7EXbwJv48t7ud2BKGdxmuJOwc02ww,1475
-opik_optimizer/datasets/tiny_test.py,sha256=AzXc-iQR8O4jfrroZmqiyluUzF_xgq9bnSTvmSLab3E,1663
-opik_optimizer/datasets/truthful_qa.py,sha256=xV7dQEmrl8Ct1oKf9HnkT7ieA5O9cibD8QA2tkDsiZY,4240
-opik_optimizer/demo/__init__.py,sha256=KSpFYhzN7fTmLEsIaciRHwxcJDeAiX5NDmYLdPsfpT8,150
-opik_optimizer/demo/cache.py,sha256=CwjdmVjokVxmPXvgfOutZK8e0sV-PIUz3ou6ODXZBts,3738
-opik_optimizer/demo/datasets.py,sha256=idod4NYHw1IbxhA8c0XVFD_pGpMZagNGNZuEYDTbbMM,2357
-opik_optimizer/evolutionary_optimizer/__init__.py,sha256=bDa6FZR9Y_a5z337I4EtvaB69jB542P4dbruhYPHCEU,95
-opik_optimizer/evolutionary_optimizer/evolutionary_optimizer.py,sha256=OfTf__Sd_1dX9XF2IUabR1xEQz6QONrYCj8lXT39F4c,82588
-opik_optimizer/evolutionary_optimizer/reporting.py,sha256=wxzaUYS-gl5wt9msdIIIY-L7oLZE-OTrjeKf11arf_g,11364
-opik_optimizer/few_shot_bayesian_optimizer/__init__.py,sha256=VuH7FOROyGcjMPryejtZC-5Y0QHlVTFLTGUDgNqRAFw,113
-opik_optimizer/few_shot_bayesian_optimizer/few_shot_bayesian_optimizer.py,sha256=kFx34JPR61JUi4lJs_kWfuozbgNGkb1CqYjQik1r9oE,27552
-opik_optimizer/few_shot_bayesian_optimizer/reporting.py,sha256=z6jI52uAg232TVc3b6uxbNzNlntkDcunvprZq0hEJvs,6036
-opik_optimizer/meta_prompt_optimizer/__init__.py,sha256=syiN2_fMm5iZDQezZCHYe-ZiGOIPlBkLt49Sa1kuR70,97
-opik_optimizer/meta_prompt_optimizer/meta_prompt_optimizer.py,sha256=3llumRjK00CSl8IHosyFrRStRcbB8ewcxmd6cTJIDws,38319
-opik_optimizer/meta_prompt_optimizer/reporting.py,sha256=wGFmnwV7Ys0kzyrz2BJwArVymoCBWObJBvKfxLhlVwY,7456
-opik_optimizer/mipro_optimizer/__init__.py,sha256=7sMq9OSWyjITqK7sVtkO9fhG1w6KRE8bN7V52CKaGvo,94
-opik_optimizer/mipro_optimizer/_lm.py,sha256=bSxQZtWg5zzqIK_FERV7v4PkDouTXP3Fw5fVgBVoKPQ,16748
-opik_optimizer/mipro_optimizer/_mipro_optimizer_v2.py,sha256=6722RQAPyjHSf8Q7OGG3yML7NC5UV1G7_irLP5h2V-g,39391
-opik_optimizer/mipro_optimizer/mipro_optimizer.py,sha256=mbsAtd8DAGDOtXpf9wtKRML4uX18j-3tGyTUFHeElE0,24420
-opik_optimizer/mipro_optimizer/utils.py,sha256=zMufGeTZGVjVDHH8wHy4e53E9bDFctWFrB_vBiXtzjE,2531
-opik_optimizer/optimization_config/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
-opik_optimizer/optimization_config/chat_prompt.py,sha256=h2C1qtcgDc3nyEcjxUz6zotHydcnQi6vuNYqfO8OsA0,6268
-opik_optimizer/optimization_config/configs.py,sha256=fY6td3_hX7c6CYWtOINZE7NWtzVdsTxx4isootHZ-lE,419
-opik_optimizer/optimization_config/mappers.py,sha256=PMSrVzqteGA3jtDeJqgK8_PJehCuuo64LRS4bLJtpIQ,1750
-opik_optimizer-1.0.6.dist-info/licenses/LICENSE,sha256=dTRSwwCHdWeSjzodvnivYqcwi8x3Qfr21yv65QUWWBE,1062
-opik_optimizer-1.0.6.dist-info/METADATA,sha256=YsDfS17TWVCxQVZGSMxvINRQQPpgzfifWQND-lJd96c,6535
-opik_optimizer-1.0.6.dist-info/WHEEL,sha256=_zCd3N1l69ArxyTb8rzEoP9TpbYXkqRFSNOD5OuxnTs,91
-opik_optimizer-1.0.6.dist-info/top_level.txt,sha256=ondOlpq6_yFckqpxoAHSfzZS2N-JfgmA-QQhOJfz7m0,15
-opik_optimizer-1.0.6.dist-info/RECORD,,

opik_optimizer-1.0.6.dist-info/licenses/LICENSE

@@ -1,21 +0,0 @@
-MIT License
-
-Copyright (c) 2025 Comet
-
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.