omegaprompt 2.0.1__tar.gz → 2.0.2__tar.gz

{omegaprompt-2.0.1 → omegaprompt-2.0.2}/.gitignore

@@ -38,6 +38,7 @@ ENV/
 
 # ── Packaging / tooling ──
 .uv/
+.uv-cache/
 pip-log.txt
 pip-delete-this-directory.txt
 

{omegaprompt-2.0.1 → omegaprompt-2.0.2}/CHANGELOG.md

@@ -6,6 +6,22 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 
 ## [Unreleased]
 
+## [2.0.2] - 2026-06-08
+
+### Added
+
+- **Opt-in parallel item evaluation.** `CalibrateTuning.max_workers` (Python) / `--concurrency` (CLI) parallelize per-item target+judge evaluation within each candidate via a local `concurrent.futures.ThreadPoolExecutor`. Each item still runs its target call then its judge call sequentially, so the number of concurrent calls to any one provider never exceeds the configured value. Default `1` (serial) produces byte-identical artifacts to prior versions. Wall-clock speedup is bounded by — and conditional on — your provider account's concurrency ceiling (RPM/TPM): roughly ~50% at `2`, ~75% at `4`, ~85% at `8` when the account permits it, and **none** if the account effectively serializes calls. No cross-provider rate-limit handling; configure per your account (a value above your real ceiling produces 429s, not speedups).
+
+### Changed
+
+- **Best-candidate evaluation is memoized.** `PromptTarget.evaluate()` now caches results keyed on the resolved params, so the final best-candidate evaluation reuses the grid-search result instead of re-calling the providers. This reduces live API calls during `calibrate()`. No public-surface change (`EvalResult` / `CalibrationArtifact` / `CalibrableTarget` unchanged) and no artifact-schema change.
+
+### Notes
+
+- Package version is `2.0.2`; `CalibrationArtifact.schema_version` remains `2.0`. The omega-lock dependency pin remains `>=0.3.0,<0.4.0`.
+- Golden reference artifacts are **unchanged** — the deterministic harness uses fixed counts/latency and builds artifacts directly (not via `runtime.calibrate()`), so neither feature affects the golden hash.
+- This release-preparation change edits the working tree only; it does not publish to PyPI, push tags, or create/edit GitHub Releases.
+
 ## [2.0.1] - 2026-06-07
 
 ### Fixed

{omegaprompt-2.0.1 → omegaprompt-2.0.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: omegaprompt
-Version: 2.0.1
+Version: 2.0.2
 Summary: Provider-neutral prompt calibration engine with sensitivity-ranked meta-axes, hard-gate x soft-score fitness, walk-forward ship gates, guarded/expedition profiles, and explicit capability/risk reporting across Anthropic, OpenAI, Gemini, and local OpenAI-compatible backends. Optional sub-tools (mini-omega-lock, mini-antemortem-cli) distribute separately and plug in via the omegaprompt.preflight interface.
 Project-URL: Homepage, https://github.com/hibou04-ops/omegaprompt
 Project-URL: Repository, https://github.com/hibou04-ops/omegaprompt
@@ -213,7 +213,7 @@ License-File: AUTHORS.md
 License-File: LICENSE
 License-File: NOTICE
 Keywords: llm-as-judge,local-llm,overfitting,prompt-calibration,prompt-engineering,provider-neutral,walk-forward
-Classifier: Development Status :: 3 - Alpha
+Classifier: Development Status :: 4 - Beta
 Classifier: Environment :: Console
 Classifier: Intended Audience :: Developers
 Classifier: License :: OSI Approved :: Apache Software License
@@ -250,8 +250,8 @@ Description-Content-Type: text/markdown
 [![CI](https://github.com/hibou04-ops/omegaprompt/actions/workflows/ci.yml/badge.svg)](https://github.com/hibou04-ops/omegaprompt/actions/workflows/ci.yml)
 [![License: Apache 2.0](https://img.shields.io/badge/license-Apache--2.0-blue.svg)](LICENSE)
 [![Python](https://img.shields.io/badge/python-3.11%2B-blue.svg)](https://www.python.org)
-[![PyPI](https://img.shields.io/badge/pypi-2.0.1-blue.svg)](https://pypi.org/project/omegaprompt/)
-[![Tests](https://img.shields.io/badge/tests-317%20passing-brightgreen.svg)](tests/)
+[![PyPI](https://img.shields.io/badge/pypi-2.0.2-blue.svg)](https://pypi.org/project/omegaprompt/)
+[![Tests](https://img.shields.io/badge/tests-passing-brightgreen.svg)](tests/)
 [![Artifact schema](https://img.shields.io/badge/artifact-schema%20v2.0-blueviolet.svg)](#8-the-calibrationartifact-schema-v20)
 [![MCP](https://img.shields.io/badge/MCP-server-blueviolet.svg)](#103-mcp-server-claude-code-cursor)
 [![Parent framework](https://img.shields.io/badge/framework-omega--lock-blueviolet.svg)](https://github.com/hibou04-ops/omega-lock)
@@ -298,7 +298,7 @@ pip install omegaprompt              # core
 pip install "omegaprompt[mcp]"       # + MCP server (Claude Code / Cursor)
 ```
 
-> **v2.0.1 (2026-06-07)** — **What's new:** omega-lock 0.3.0 compatibility — omega-lock 0.3.0 reads `.sample_count` off the result `PromptTarget.evaluate` returns; `EvalResult` now exposes `sample_count` as a read-only alias of the stored `n_trials` so calibration no longer crashes on the 0.3.0 seam. Plus a consumer **docking hardlock**: a fail-loud contract test (`tests/test_omega_lock_contract.py`) and a scheduled consumer canary (`.github/workflows/omega-lock-compat.yml`) that run omegaprompt's own consumer reads against omega-lock@main so a producer-side field rename is caught before it reaches a release. Public README/PyPI-facing claims and exact deterministic reference metrics are tracked in the generated [claim ledger](https://github.com/hibou04-ops/omegaprompt/blob/main/docs/claims/README_CLAIMS.generated.md). Release gates cover repository consistency, generated claims, golden reference artifacts, artifact integrity, wheel smoke, and local/post-release verification.
+> **v2.0.2 (2026-06-08)** — **What's new:** **opt-in parallel item evaluation** — `--concurrency N` (CLI) / `CalibrateTuning(max_workers=N)` evaluates dataset items concurrently within each candidate via a local thread pool. Each item still runs its target and judge calls sequentially, so concurrent calls to any one provider never exceed N. The default (`1`) is serial and produces byte-identical artifacts to prior versions. Wall-clock speedup is **entirely conditional on your provider account's concurrency ceiling (RPM/TPM)**: roughly 50% at N=2 when the account permits it, and **zero** if your account effectively serializes calls. Also: `PromptTarget.evaluate()` is now **memoized by resolved params**, so the final best-candidate evaluation reuses the grid-search result instead of re-calling the providers — fewer live API calls during `calibrate()`, with no public-surface or artifact-schema change. omega-lock pin unchanged (`>=0.3.0,<0.4.0`); `CalibrationArtifact.schema_version` stays `2.0`; golden reference artifacts are unchanged. The project's PyPI Development Status is now **4 - Beta**. Public README/PyPI-facing claims and exact deterministic reference metrics are tracked in the generated [claim ledger](https://github.com/hibou04-ops/omegaprompt/blob/main/docs/claims/README_CLAIMS.generated.md). Release gates cover repository consistency, generated claims, golden reference artifacts, artifact integrity, wheel smoke, and local/post-release verification.
 
 <!-- public-claim-ledger:start -->
 > Claim evidence source: [docs/claims/public_claim_ledger.json](https://github.com/hibou04-ops/omegaprompt/blob/main/docs/claims/public_claim_ledger.json), rendered by `python tools/generate_readme_claims.py`.
@@ -1776,7 +1776,7 @@ Every override carries its `parameter`, `default`, `applied`, and `reason`. The
 Short form:
 
 ```
-omegaprompt v2.0.1 — provider-neutral prompt calibration engine.
+omegaprompt v2.0.2 — provider-neutral prompt calibration engine.
 https://github.com/hibou04-ops/omegaprompt, 2026.
 ```
 
@@ -1788,7 +1788,7 @@ BibTeX:
   title   = {{omegaprompt}: Provider-neutral prompt calibration engine
              with sensitivity-ranked meta-axes, walk-forward ship gates,
              and structural capability reporting},
-  version = {2.0.1},
+  version = {2.0.2},
   year    = {2026},
   url     = {https://github.com/hibou04-ops/omegaprompt}
 }

{omegaprompt-2.0.1 → omegaprompt-2.0.2}/README.md

@@ -5,8 +5,8 @@
 [![CI](https://github.com/hibou04-ops/omegaprompt/actions/workflows/ci.yml/badge.svg)](https://github.com/hibou04-ops/omegaprompt/actions/workflows/ci.yml)
 [![License: Apache 2.0](https://img.shields.io/badge/license-Apache--2.0-blue.svg)](LICENSE)
 [![Python](https://img.shields.io/badge/python-3.11%2B-blue.svg)](https://www.python.org)
-[![PyPI](https://img.shields.io/badge/pypi-2.0.1-blue.svg)](https://pypi.org/project/omegaprompt/)
-[![Tests](https://img.shields.io/badge/tests-317%20passing-brightgreen.svg)](tests/)
+[![PyPI](https://img.shields.io/badge/pypi-2.0.2-blue.svg)](https://pypi.org/project/omegaprompt/)
+[![Tests](https://img.shields.io/badge/tests-passing-brightgreen.svg)](tests/)
 [![Artifact schema](https://img.shields.io/badge/artifact-schema%20v2.0-blueviolet.svg)](#8-the-calibrationartifact-schema-v20)
 [![MCP](https://img.shields.io/badge/MCP-server-blueviolet.svg)](#103-mcp-server-claude-code-cursor)
 [![Parent framework](https://img.shields.io/badge/framework-omega--lock-blueviolet.svg)](https://github.com/hibou04-ops/omega-lock)
@@ -53,7 +53,7 @@ pip install omegaprompt              # core
 pip install "omegaprompt[mcp]"       # + MCP server (Claude Code / Cursor)
 ```
 
-> **v2.0.1 (2026-06-07)** — **What's new:** omega-lock 0.3.0 compatibility — omega-lock 0.3.0 reads `.sample_count` off the result `PromptTarget.evaluate` returns; `EvalResult` now exposes `sample_count` as a read-only alias of the stored `n_trials` so calibration no longer crashes on the 0.3.0 seam. Plus a consumer **docking hardlock**: a fail-loud contract test (`tests/test_omega_lock_contract.py`) and a scheduled consumer canary (`.github/workflows/omega-lock-compat.yml`) that run omegaprompt's own consumer reads against omega-lock@main so a producer-side field rename is caught before it reaches a release. Public README/PyPI-facing claims and exact deterministic reference metrics are tracked in the generated [claim ledger](https://github.com/hibou04-ops/omegaprompt/blob/main/docs/claims/README_CLAIMS.generated.md). Release gates cover repository consistency, generated claims, golden reference artifacts, artifact integrity, wheel smoke, and local/post-release verification.
+> **v2.0.2 (2026-06-08)** — **What's new:** **opt-in parallel item evaluation** — `--concurrency N` (CLI) / `CalibrateTuning(max_workers=N)` evaluates dataset items concurrently within each candidate via a local thread pool. Each item still runs its target and judge calls sequentially, so concurrent calls to any one provider never exceed N. The default (`1`) is serial and produces byte-identical artifacts to prior versions. Wall-clock speedup is **entirely conditional on your provider account's concurrency ceiling (RPM/TPM)**: roughly 50% at N=2 when the account permits it, and **zero** if your account effectively serializes calls. Also: `PromptTarget.evaluate()` is now **memoized by resolved params**, so the final best-candidate evaluation reuses the grid-search result instead of re-calling the providers — fewer live API calls during `calibrate()`, with no public-surface or artifact-schema change. omega-lock pin unchanged (`>=0.3.0,<0.4.0`); `CalibrationArtifact.schema_version` stays `2.0`; golden reference artifacts are unchanged. The project's PyPI Development Status is now **4 - Beta**. Public README/PyPI-facing claims and exact deterministic reference metrics are tracked in the generated [claim ledger](https://github.com/hibou04-ops/omegaprompt/blob/main/docs/claims/README_CLAIMS.generated.md). Release gates cover repository consistency, generated claims, golden reference artifacts, artifact integrity, wheel smoke, and local/post-release verification.
 
 <!-- public-claim-ledger:start -->
 > Claim evidence source: [docs/claims/public_claim_ledger.json](https://github.com/hibou04-ops/omegaprompt/blob/main/docs/claims/public_claim_ledger.json), rendered by `python tools/generate_readme_claims.py`.
@@ -1531,7 +1531,7 @@ Every override carries its `parameter`, `default`, `applied`, and `reason`. The
 Short form:
 
 ```
-omegaprompt v2.0.1 — provider-neutral prompt calibration engine.
+omegaprompt v2.0.2 — provider-neutral prompt calibration engine.
 https://github.com/hibou04-ops/omegaprompt, 2026.
 ```
 
@@ -1543,7 +1543,7 @@ BibTeX:
   title   = {{omegaprompt}: Provider-neutral prompt calibration engine
              with sensitivity-ranked meta-axes, walk-forward ship gates,
              and structural capability reporting},
-  version = {2.0.1},
+  version = {2.0.2},
   year    = {2026},
   url     = {https://github.com/hibou04-ops/omegaprompt}
 }

{omegaprompt-2.0.1 → omegaprompt-2.0.2}/README_KR.md

@@ -5,10 +5,10 @@
 
 [![License: Apache 2.0](https://img.shields.io/badge/license-Apache--2.0-blue.svg)](LICENSE)
 [![Python](https://img.shields.io/badge/python-3.11%2B-blue.svg)](https://www.python.org)
-[![PyPI](https://img.shields.io/badge/pypi-2.0.1-blue.svg)](https://pypi.org/project/omegaprompt/)
-[![Tests](https://img.shields.io/badge/tests-317%20passing-brightgreen.svg)](tests/)
-[![Artifact schema](https://img.shields.io/badge/artifact-schema%20v2.0-blueviolet.svg)](#calibrationartifact-schema-v20)
-[![MCP](https://img.shields.io/badge/MCP-server-blueviolet.svg)](#mcp-서버)
+[![PyPI](https://img.shields.io/badge/pypi-2.0.2-blue.svg)](https://pypi.org/project/omegaprompt/)
+[![Tests](https://img.shields.io/badge/tests-passing-brightgreen.svg)](tests/)
+[![Artifact schema](https://img.shields.io/badge/artifact-schema%20v2.0-blueviolet.svg)](#10-calibrationartifact-schema-v20)
+[![MCP](https://img.shields.io/badge/MCP-server-blueviolet.svg)](#18-mcp-서버)
 [![Parent framework](https://img.shields.io/badge/framework-omega--lock-blueviolet.svg)](https://github.com/hibou04-ops/omega-lock)
 
 문서: [English](README.md) · [한국어](README_KR.md) · [Easy English](EASY_README.md) · [쉬운 한국어](EASY_README_KR.md)
@@ -26,7 +26,7 @@ pip install omegaprompt              # core
 pip install "omegaprompt[mcp]"       # + MCP server
 ```
 
-> **v2.0.1 (2026-06-07)** — **새 내용:** omega-lock 0.3.0 호환 — omega-lock 0.3.0은 `PromptTarget.evaluate`가 반환하는 결과에서 `.sample_count`를 읽습니다. `EvalResult`는 이제 저장 필드 `n_trials`의 read-only alias로 `sample_count`를 노출하므로 0.3.0 seam에서 calibration이 더 이상 깨지지 않습니다. 또한 consumer **docking hardlock**: fail-loud contract test(`tests/test_omega_lock_contract.py`)와 scheduled consumer canary(`.github/workflows/omega-lock-compat.yml`)가 omegaprompt 자신의 consumer read를 omega-lock@main에 대해 실행해, producer 쪽 field rename을 release 전에 잡습니다. public README/PyPI claim과 deterministic reference metric은 생성된 [claim ledger](docs/claims/README_CLAIMS.generated.md)에서 추적합니다. release gate는 repository consistency, generated claims, golden reference artifacts, artifact integrity, wheel smoke, local/post-release verification을 확인합니다.
+> **v2.0.2 (2026-06-08)** — **새 내용:** **opt-in 병렬 item 평가** — `--concurrency N`(CLI) / `CalibrateTuning(max_workers=N)`이 각 candidate 안에서 dataset item을 local thread pool로 동시에 평가합니다. 각 item은 여전히 target 호출 후 judge 호출을 순차로 실행하므로, 어느 한 provider에 대한 동시 호출 수는 N을 넘지 않습니다. 기본값(`1`)은 serial이며 이전 버전과 byte-identical한 artifact를 만듭니다. wall-clock 속도 향상은 **전적으로 사용 중인 provider 계정의 동시성 한도(RPM/TPM)에 따라 달라집니다**: 계정이 허용하면 N=2에서 대략 50%, 계정이 사실상 호출을 직렬화하면 **0%**입니다. 또한 `PromptTarget.evaluate()`가 이제 **resolved params 기준으로 memoize**되어, 마지막 best-candidate 평가가 provider를 다시 호출하는 대신 grid-search 결과를 재사용합니다 — `calibrate()` 동안 live API 호출이 줄며 public surface나 artifact schema 변경은 없습니다. omega-lock pin은 그대로(`>=0.3.0,<0.4.0`)이고 `CalibrationArtifact.schema_version`은 `2.0`을 유지하며 golden reference artifact는 변경되지 않습니다. 프로젝트의 PyPI Development Status는 이제 **4 - Beta**입니다. public README/PyPI claim과 deterministic reference metric은 생성된 [claim ledger](docs/claims/README_CLAIMS.generated.md)에서 추적합니다. release gate는 repository consistency, generated claims, golden reference artifacts, artifact integrity, wheel smoke, local/post-release verification을 확인합니다.
 
 이름 경계: GitHub repo는 `hibou04-ops/omegaprompt`; PyPI distribution, primary import package, primary CLI는 `omegaprompt`; compatibility package / CLI alias는 `omegacal`; 별도 parent calibration framework는 `omega-lock`입니다.
 

{omegaprompt-2.0.1 → omegaprompt-2.0.2}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "omegaprompt"
-version = "2.0.1"
+version = "2.0.2"
 description = "Provider-neutral prompt calibration engine with sensitivity-ranked meta-axes, hard-gate x soft-score fitness, walk-forward ship gates, guarded/expedition profiles, and explicit capability/risk reporting across Anthropic, OpenAI, Gemini, and local OpenAI-compatible backends. Optional sub-tools (mini-omega-lock, mini-antemortem-cli) distribute separately and plug in via the omegaprompt.preflight interface."
 readme = "README.md"
 requires-python = ">=3.11"
@@ -22,7 +22,7 @@ keywords = [
     "overfitting",
 ]
 classifiers = [
-    "Development Status :: 3 - Alpha",
+    "Development Status :: 4 - Beta",
     "Environment :: Console",
     "Intended Audience :: Developers",
     "License :: OSI Approved :: Apache Software License",
@@ -72,6 +72,11 @@ omegaprompt-mcp = "omegaprompt.mcp.__main__:main"
 packages = ["src/omegaprompt", "src/omegacal"]
 
 [tool.hatch.build.targets.sdist]
+exclude = [
+    ".uv-cache",
+    ".uv",
+    "dist",
+]
 include = [
     "src/omegaprompt",
     "src/omegacal",

{omegaprompt-2.0.1 → omegaprompt-2.0.2}/src/omegaprompt/__init__.py

@@ -108,7 +108,7 @@ from omegaprompt.runtime import (
     report,
 )
 
-__version__ = "2.0.1"
+__version__ = "2.0.2"
 
 __all__ = [
     # domain

{omegaprompt-2.0.1 → omegaprompt-2.0.2}/src/omegaprompt/commands/calibrate.py

@@ -157,6 +157,18 @@ def calibrate(
         min=1,
         help="How many top-stress meta-axes to unlock for grid search.",
     ),
+    concurrency: int = typer.Option(  # noqa: B008
+        1,
+        "--concurrency",
+        min=1,
+        help=(
+            "Max concurrent dataset items per evaluation. Each item runs its "
+            "target then judge call sequentially, so concurrent calls to any "
+            "one provider never exceed this value. Default 1 (serial). Set per "
+            "your provider account's RPM/TPM ceiling; above the real ceiling "
+            "you get 429s, not speedups. See README."
+        ),
+    ),
     space_path: Path | None = typer.Option(  # noqa: B008
         None,
         "--space",
@@ -276,6 +288,7 @@ def calibrate(
         min_kc4=min_kc4,
         profile=selected_profile,
         validation_mode=vmode,  # type: ignore[arg-type]
+        max_workers=concurrency,
     )
 
     typer.secho(

{omegaprompt-2.0.1 → omegaprompt-2.0.2}/src/omegaprompt/runtime.py

@@ -96,6 +96,19 @@ class CalibrateTuning(BaseModel):
     max_gap: float | None = None
     min_kc4: float | None = None
     profile: ExecutionProfile = ExecutionProfile.GUARDED
+    max_workers: int = Field(
+        default=1,
+        ge=1,
+        description=(
+            "Max dataset items evaluated concurrently within each candidate "
+            "evaluation (opt-in; default 1 = serial). Each item runs its "
+            "target then judge call sequentially, so concurrent calls to any "
+            "one provider never exceed this value. Default produces "
+            "byte-identical artifacts to prior versions. Wall-clock speedup is "
+            "bounded by — and conditional on — your provider account's "
+            "concurrency ceiling (RPM/TPM)."
+        ),
+    )
     validation_mode: Literal["auto", "paired", "disjoint"] = Field(
         default="auto",
         description=(
@@ -301,6 +314,7 @@ def calibrate(
         variants=variants_obj,
         space=tuning.space,
         execution_profile=profile,
+        max_workers=tuning.max_workers,
     )
     test_target = (
         PromptTarget(
@@ -311,6 +325,7 @@ def calibrate(
             variants=variants_obj,
             space=tuning.space,
             execution_profile=profile,
+            max_workers=tuning.max_workers,
         )
         if test_ds is not None
         else None