weco 0.2.16__tar.gz → 0.2.18__tar.gz

{weco-0.2.16 → weco-0.2.18}/.github/workflows/release.yml

@@ -43,8 +43,6 @@ jobs:
             OLD_VERSION=""
           fi
 
-          OLD_VERSION="0.2.15"
-
           echo "Previous version: $OLD_VERSION"
           echo "Current  version: $NEW_VERSION"
 

{weco-0.2.16 → weco-0.2.18}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: weco
-Version: 0.2.16
+Version: 0.2.18
 Summary: Documentation for `weco`, a CLI for using Weco AI's code optimizer.
 Author-email: Weco AI Team <contact@weco.ai>
 License: MIT
@@ -14,25 +14,33 @@ Description-Content-Type: text/markdown
 License-File: LICENSE
 Requires-Dist: requests
 Requires-Dist: rich
+Requires-Dist: packaging
 Provides-Extra: dev
 Requires-Dist: ruff; extra == "dev"
 Requires-Dist: build; extra == "dev"
 Requires-Dist: setuptools_scm; extra == "dev"
 Dynamic: license-file
 
-# Weco: The AI Research Engineer
+<div align="center">
 
-[![Python](https://img.shields.io/badge/Python-3.12.0-blue)](https://www.python.org)
+# Weco: The Platform for Self-Improving Code
+
+[![Python](https://img.shields.io/badge/Python-3.8.0+-blue)](https://www.python.org)
+[![docs](https://img.shields.io/website?url=https://docs.weco.ai/&label=docs)](https://docs.weco.ai/)
 [![PyPI version](https://badge.fury.io/py/weco.svg)](https://badge.fury.io/py/weco)
 [![AIDE](https://img.shields.io/badge/AI--Driven_Exploration-arXiv-orange?style=flat-square&logo=arxiv)](https://arxiv.org/abs/2502.13138)
 
+</div>
+
+---
+
 Weco systematically optimizes your code, guided directly by your evaluation metrics.
 
 Example applications include:
 
-- **GPU Kernel Optimization**: Reimplement PyTorch functions using CUDA or Triton optimizing for `latency`, `throughput`, or `memory_bandwidth`.
-- **Model Development**: Tune feature transformations or architectures, optimizing for `validation_accuracy`, `AUC`, or `Sharpe Ratio`.
-- **Prompt Engineering**: Refine prompts for LLMs, optimizing for `win_rate`, `relevance`, or `format_adherence`
+- **GPU Kernel Optimization**: Reimplement PyTorch functions using [CUDA](/examples/cuda/README.md) or [Triton](/examples/triton/README.md), optimizing for `latency`, `throughput`, or `memory_bandwidth`.
+- **Model Development**: Tune feature transformations, architectures or [the whole training pipeline](/examples/spaceship-titanic/README.md), optimizing for `validation_accuracy`, `AUC`, or `Sharpe Ratio`.
+- **Prompt Engineering**: Refine prompts for LLMs (e.g., for [math problems](/examples/prompt/README.md)), optimizing for `win_rate`, `relevance`, or `format_adherence`
 
 ![image](assets/example-optimization.gif)
 
@@ -62,29 +70,9 @@ The `weco` CLI leverages a tree search approach guided by Large Language Models
     - **Anthropic:** `export ANTHROPIC_API_KEY="your_key_here"`
     - **Google DeepMind:** `export GEMINI_API_KEY="your_key_here"` (Google AI Studio has a free API usage quota. Create a key [here](https://aistudio.google.com/apikey) to use `weco` for free.)
 
-    The optimization process will fail if the necessary keys for the chosen model are not found in your environment.
-
-3.  **Log In to Weco (Optional):**
-
-    To associate your optimization runs with your Weco account and view them on the Weco dashboard, you can log in. `weco` uses a device authentication flow:
-
-    - When you first run `weco run`, you'll be prompted if you want to log in or proceed anonymously.
-    - If you choose to log in (by pressing `l`), you'll be shown a URL and `weco` will attempt to open it in your default web browser.
-    - You then authenticate in the browser. Once authenticated, the CLI will detect this and complete the login.
-    - This saves a Weco-specific API key locally (typically at `~/.config/weco/credentials.json`).
-
-    If you choose to skip login (by pressing Enter or `s`), `weco` will still function using the environment variable LLM keys, but the run history will not be linked to a Weco account.
-
-    To log out and remove your saved Weco API key, use the `weco logout` command.
-
 ---
 
-## Usage
-
-The CLI has two main commands:
-
-- `weco run`: Initiates the code optimization process.
-- `weco logout`: Logs you out of your Weco account.
+## Get Started
 
 <div style="background-color: #fff3cd; border: 1px solid #ffeeba; padding: 15px; border-radius: 4px; margin-bottom: 15px;">
   <strong>⚠️ Warning: Code Modification</strong><br>
@@ -93,15 +81,11 @@ The CLI has two main commands:
 
 ---
 
-### `weco run` Command
-
-This command starts the optimization process.
-
 **Example: Optimizing Simple PyTorch Operations**
 
 This basic example shows how to optimize a simple PyTorch function for speedup.
 
-For more advanced examples, including [Triton](/examples/triton/README.md), [CUDA kernel optimization](/examples/cuda/README.md)**, and **[ML model optimization](/examples/spaceship-titanic/README.md)**, please see the `README.md` files within the corresponding subdirectories under the [`examples/`](./examples/) folder.
+For more advanced examples, including [Triton](/examples/triton/README.md), [CUDA kernel optimization](/examples/cuda/README.md), [ML model optimization](/examples/spaceship-titanic/README.md), and [prompt engineering for math problems](https://github.com/WecoAI/weco-cli/tree/main/examples/prompt), please see the `README.md` files within the corresponding subdirectories under the [`examples/`](./examples/) folder.
 
 ```bash
 # Navigate to the example directory
@@ -136,17 +120,12 @@ weco run --source optimize.py \
 | `--model`                   | Model identifier for the LLM to use (e.g., `gpt-4o`, `claude-3.5-sonnet`). Recommended models to try include `o3-mini`, `claude-3-haiku`, and `gemini-2.5-pro-exp-03-25`. | Yes      |
 | `--additional-instructions` | (Optional) Natural language description of specific instructions OR path to a file containing detailed instructions to guide the LLM.                                     | No       |
 | `--log-dir`                 | (Optional) Path to the directory to log intermediate steps and final optimization result. Defaults to `.runs/`.                                                           | No       |
-| `--preserve-source`         | (Optional) If set, do not overwrite the original `--source` file. Modifications and the best solution will still be saved in the `--log-dir`.                             | No       |
 
 ---
 
-### `weco logout` Command
-
-This command logs you out by removing the locally stored Weco API key.
-
-```bash
-weco logout
-```
+### Weco Dashboard
+To associate your optimization runs with your Weco account and view them on the Weco dashboard, you can log in. `weco` uses a device authentication flow
+![image (16)](https://github.com/user-attachments/assets/8a0a285b-4894-46fa-b6a2-4990017ca0c6)
 
 ---
 

{weco-0.2.16 → weco-0.2.18}/README.md

@@ -1,16 +1,23 @@
-# Weco: The AI Research Engineer
+<div align="center">
 
-[![Python](https://img.shields.io/badge/Python-3.12.0-blue)](https://www.python.org)
+# Weco: The Platform for Self-Improving Code
+
+[![Python](https://img.shields.io/badge/Python-3.8.0+-blue)](https://www.python.org)
+[![docs](https://img.shields.io/website?url=https://docs.weco.ai/&label=docs)](https://docs.weco.ai/)
 [![PyPI version](https://badge.fury.io/py/weco.svg)](https://badge.fury.io/py/weco)
 [![AIDE](https://img.shields.io/badge/AI--Driven_Exploration-arXiv-orange?style=flat-square&logo=arxiv)](https://arxiv.org/abs/2502.13138)
 
+</div>
+
+---
+
 Weco systematically optimizes your code, guided directly by your evaluation metrics.
 
 Example applications include:
 
-- **GPU Kernel Optimization**: Reimplement PyTorch functions using CUDA or Triton optimizing for `latency`, `throughput`, or `memory_bandwidth`.
-- **Model Development**: Tune feature transformations or architectures, optimizing for `validation_accuracy`, `AUC`, or `Sharpe Ratio`.
-- **Prompt Engineering**: Refine prompts for LLMs, optimizing for `win_rate`, `relevance`, or `format_adherence`
+- **GPU Kernel Optimization**: Reimplement PyTorch functions using [CUDA](/examples/cuda/README.md) or [Triton](/examples/triton/README.md), optimizing for `latency`, `throughput`, or `memory_bandwidth`.
+- **Model Development**: Tune feature transformations, architectures or [the whole training pipeline](/examples/spaceship-titanic/README.md), optimizing for `validation_accuracy`, `AUC`, or `Sharpe Ratio`.
+- **Prompt Engineering**: Refine prompts for LLMs (e.g., for [math problems](/examples/prompt/README.md)), optimizing for `win_rate`, `relevance`, or `format_adherence`
 
 ![image](assets/example-optimization.gif)
 
@@ -40,29 +47,9 @@ The `weco` CLI leverages a tree search approach guided by Large Language Models
     - **Anthropic:** `export ANTHROPIC_API_KEY="your_key_here"`
     - **Google DeepMind:** `export GEMINI_API_KEY="your_key_here"` (Google AI Studio has a free API usage quota. Create a key [here](https://aistudio.google.com/apikey) to use `weco` for free.)
 
-    The optimization process will fail if the necessary keys for the chosen model are not found in your environment.
-
-3.  **Log In to Weco (Optional):**
-
-    To associate your optimization runs with your Weco account and view them on the Weco dashboard, you can log in. `weco` uses a device authentication flow:
-
-    - When you first run `weco run`, you'll be prompted if you want to log in or proceed anonymously.
-    - If you choose to log in (by pressing `l`), you'll be shown a URL and `weco` will attempt to open it in your default web browser.
-    - You then authenticate in the browser. Once authenticated, the CLI will detect this and complete the login.
-    - This saves a Weco-specific API key locally (typically at `~/.config/weco/credentials.json`).
-
-    If you choose to skip login (by pressing Enter or `s`), `weco` will still function using the environment variable LLM keys, but the run history will not be linked to a Weco account.
-
-    To log out and remove your saved Weco API key, use the `weco logout` command.
-
 ---
 
-## Usage
-
-The CLI has two main commands:
-
-- `weco run`: Initiates the code optimization process.
-- `weco logout`: Logs you out of your Weco account.
+## Get Started
 
 <div style="background-color: #fff3cd; border: 1px solid #ffeeba; padding: 15px; border-radius: 4px; margin-bottom: 15px;">
   <strong>⚠️ Warning: Code Modification</strong><br>
@@ -71,15 +58,11 @@ The CLI has two main commands:
 
 ---
 
-### `weco run` Command
-
-This command starts the optimization process.
-
 **Example: Optimizing Simple PyTorch Operations**
 
 This basic example shows how to optimize a simple PyTorch function for speedup.
 
-For more advanced examples, including [Triton](/examples/triton/README.md), [CUDA kernel optimization](/examples/cuda/README.md)**, and **[ML model optimization](/examples/spaceship-titanic/README.md)**, please see the `README.md` files within the corresponding subdirectories under the [`examples/`](./examples/) folder.
+For more advanced examples, including [Triton](/examples/triton/README.md), [CUDA kernel optimization](/examples/cuda/README.md), [ML model optimization](/examples/spaceship-titanic/README.md), and [prompt engineering for math problems](https://github.com/WecoAI/weco-cli/tree/main/examples/prompt), please see the `README.md` files within the corresponding subdirectories under the [`examples/`](./examples/) folder.
 
 ```bash
 # Navigate to the example directory
@@ -114,17 +97,12 @@ weco run --source optimize.py \
 | `--model`                   | Model identifier for the LLM to use (e.g., `gpt-4o`, `claude-3.5-sonnet`). Recommended models to try include `o3-mini`, `claude-3-haiku`, and `gemini-2.5-pro-exp-03-25`. | Yes      |
 | `--additional-instructions` | (Optional) Natural language description of specific instructions OR path to a file containing detailed instructions to guide the LLM.                                     | No       |
 | `--log-dir`                 | (Optional) Path to the directory to log intermediate steps and final optimization result. Defaults to `.runs/`.                                                           | No       |
-| `--preserve-source`         | (Optional) If set, do not overwrite the original `--source` file. Modifications and the best solution will still be saved in the `--log-dir`.                             | No       |
 
 ---
 
-### `weco logout` Command
-
-This command logs you out by removing the locally stored Weco API key.
-
-```bash
-weco logout
-```
+### Weco Dashboard
+To associate your optimization runs with your Weco account and view them on the Weco dashboard, you can log in. `weco` uses a device authentication flow
+![image (16)](https://github.com/user-attachments/assets/8a0a285b-4894-46fa-b6a2-4990017ca0c6)
 
 ---
 

weco-0.2.18/examples/prompt/README.md

@@ -0,0 +1,51 @@
+# AIME Prompt Engineering Example with Weco
+
+This example shows how **Weco** can iteratively improve a prompt for solving American Invitational Mathematics Examination (AIME) problems. The experiment runs locally, requires only two short Python files, and aims to improve the accuracy metric.
+
+This example uses `gpt-4o-mini` via the OpenAI API by default. Ensure your `OPENAI_API_KEY` environment variable is set.
+
+## Files in this folder
+
+| File          | Purpose                                                                                                                                                           |
+| :------------ | :---------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `optimize.py` | Holds the prompt template (instructing the LLM to reason step-by-step and use `\\boxed{}` for the final answer) and the mutable `EXTRA_INSTRUCTIONS` string. Weco edits **only** this file during the search. |
+| `eval.py`     | Downloads a small slice of the 2024 AIME dataset, calls `optimize.solve` in parallel, parses the LLM output (looking for `\\boxed{}`), compares it to the ground truth, prints progress logs, and finally prints an `accuracy:` line that Weco reads. |
+
+
+## Quick start
+
+1. **Clone the repository and enter the folder.**
+   ```bash
+   git clone https://github.com/your‑fork/weco‑examples.git
+   cd weco‑examples/aime‑2024
+   ```
+2. **Run Weco.**  The command below edits `EXTRA_INSTRUCTIONS` in `optimize.py`, invokes `eval.py` on every iteration, reads the printed accuracy, and keeps the best variants.
+   ```bash
+   weco --source optimize.py \
+        --eval-command "python eval.py" \
+        --metric accuracy \
+        --maximize true \
+        --steps 40 \
+        --model gemini-2.5-flash-preview-04-17 \
+        --addtional-instructions prompt_guide.md
+   ```
+
+During each evaluation round you will see log lines similar to the following.
+
+```text
+[setup] loading 20 problems from AIME 2024 …
+[progress] 5/20 completed, elapsed 7.3 s
+[progress] 10/20 completed, elapsed 14.6 s
+[progress] 15/20 completed, elapsed 21.8 s
+[progress] 20/20 completed, elapsed 28.9 s
+accuracy: 0.0500
+```
+
+Weco then mutates the config, tries again, and gradually pushes the accuracy higher. On a modern laptop you can usually double the baseline score within thirty to forty iterations.
+
+## How it works
+
+* `eval_aime.py` slices the **Maxwell‑Jia/AIME_2024** dataset to twenty problems for fast feedback. You can change the slice in one line.
+* The script sends model calls in parallel via `ThreadPoolExecutor`, so network latency is hidden.
+* Every five completed items, the script logs progress and elapsed time.
+* The final line `accuracy: value` is the only part Weco needs for guidance.

{weco-0.2.16 → weco-0.2.18}/examples/spaceship-titanic/README.md

@@ -1,33 +1,16 @@
-# Example: Optimizing a Kaggle Classification Model (Spaceship Titanic)
+# Example: Solving a Kaggle Competition (Spaceship Titanic)
 
 This example demonstrates using Weco to optimize a Python script designed for the [Spaceship Titanic Kaggle competition](https://www.kaggle.com/competitions/spaceship-titanic/overview). The goal is to improve the model's `accuracy` metric by directly optimizing the evaluate.py
 
 ## Setup
 
 1.  Ensure you are in the `examples/spaceship-titanic` directory.
-2.  **Kaggle Credentials:** You need your Kaggle API credentials (`kaggle.json`) configured to download the competition dataset. Place the `kaggle.json` file in `~/.kaggle/` or set the `KAGGLE_USERNAME` and `KAGGLE_KEY` environment variables. See [Kaggle API documentation](https://github.com/Kaggle/kaggle-api#api-credentials) for details.
-3.  **Install Dependencies:** Install the required Python packages:
+2.  `pip install weco`
+3.  Set up LLM API Key, `export OPENAI_API_KEY="your_key_here"`
+4.  **Install Dependencies:** Install the required Python packages:
     ```bash
     pip install -r requirements-test.txt
     ```
-4.  **Prepare Data:** Run the utility script once to download the dataset from Kaggle and place it in the expected `./data/` subdirectories:
-    ```bash
-    python get_data.py
-    ```
-    After running `get_data.py`, your directory structure should look like this:
-    ```
-    .
-    ├── competition_description.md
-    ├── data
-    │   ├── sample_submission.csv
-    │   ├── test.csv
-    │   └── train.csv
-    ├── evaluate.py
-    ├── get_data.py
-    ├── README.md # This file
-    ├── requirements-test.txt
-    └── submit.py
-    ```
 
 ## Optimization Command
 
@@ -38,20 +21,12 @@ weco run --source evaluate.py \
          --eval-command "python evaluate.py --data-dir ./data" \
          --metric accuracy \
          --maximize true \
-         --steps 10 \
-         --model gemini-2.5-pro-exp-03-25 \
+         --steps 20 \
+         --model o4-mini \
          --additional-instructions "Improve feature engineering, model choice and hyper-parameters."
          --log-dir .runs/spaceship-titanic
 ```
 
-## Submit the solution
-
-Once the optimization finished, you can submit your predictions to kaggle to see the results. Make sure `submission.csv` is present and then simply run the following command.
-
-```bash
-python submit.py
-```
-
 ### Explanation
 
 *   `--source evaluate.py`: The script provides a baseline as root node and directly optimize the evaluate.py
@@ -64,4 +39,4 @@ python submit.py
 *   `--model gemini-2.5-pro-exp-03-25`: The LLM driving the optimization.
 *   `--additional-instructions "Improve feature engineering, model choice and hyper-parameters."`: A simple instruction for model improvement or you can put the path to [`comptition_description.md`](./competition_description.md) within the repo to feed the agent more detailed information.
 
-Weco will iteratively modify the feature engineering or modeling code within `evaluate.py`, run the evaluation pipeline, and use the resulting `accuracy` to guide further improvements.
+Weco will iteratively modify the feature engineering or modeling code within `evaluate.py`, run the evaluation pipeline, and use the resulting `accuracy` to guide further improvements.