orizu 0.5.19 → 0.5.21
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +23 -0
- package/dist/archive-artifacts-cli.d.ts +50 -0
- package/dist/archive-artifacts-cli.d.ts.map +1 -0
- package/dist/artifact-archive.d.ts +48 -0
- package/dist/artifact-archive.d.ts.map +1 -0
- package/dist/artifact-pull.d.ts +22 -0
- package/dist/artifact-pull.d.ts.map +1 -0
- package/dist/banner.d.ts +3 -0
- package/dist/banner.d.ts.map +1 -0
- package/dist/connectors-cli.d.ts +31 -0
- package/dist/connectors-cli.d.ts.map +1 -0
- package/dist/credentials.d.ts +46 -0
- package/dist/credentials.d.ts.map +1 -0
- package/dist/csv.d.ts +49 -0
- package/dist/csv.d.ts.map +1 -0
- package/dist/dataset-download.d.ts +6 -0
- package/dist/dataset-download.d.ts.map +1 -0
- package/dist/dataset-edit-rows.d.ts +10 -0
- package/dist/dataset-edit-rows.d.ts.map +1 -0
- package/dist/dataset-upload-snapshot.d.ts +13 -0
- package/dist/dataset-upload-snapshot.d.ts.map +1 -0
- package/dist/daytona-slice-rehearsal.d.ts +89 -0
- package/dist/daytona-slice-rehearsal.d.ts.map +1 -0
- package/dist/daytona-slice.d.ts +106 -0
- package/dist/daytona-slice.d.ts.map +1 -0
- package/dist/deploy-key-merge-core.d.ts +121 -0
- package/dist/deploy-key-merge-core.d.ts.map +1 -0
- package/dist/diff-comments-cli.d.ts +12 -0
- package/dist/diff-comments-cli.d.ts.map +1 -0
- package/dist/diff-comments-cli.js +21 -2
- package/dist/egress-allowlist-cli.d.ts +23 -0
- package/dist/egress-allowlist-cli.d.ts.map +1 -0
- package/dist/egress-policy.d.ts +108 -0
- package/dist/egress-policy.d.ts.map +1 -0
- package/dist/error-response.d.ts +9 -0
- package/dist/error-response.d.ts.map +1 -0
- package/dist/file-parser.d.ts +5 -0
- package/dist/file-parser.d.ts.map +1 -0
- package/dist/gepa-engine-dispatch.d.ts +9 -0
- package/dist/gepa-engine-dispatch.d.ts.map +1 -0
- package/dist/gepa-engine-dispatch.js +307 -0
- package/dist/gepa-python-command.d.ts +6 -0
- package/dist/gepa-python-command.d.ts.map +1 -0
- package/dist/gepa-python-command.js +26 -0
- package/dist/gepa-python-paths.d.ts +6 -0
- package/dist/gepa-python-paths.d.ts.map +1 -0
- package/dist/gepa-python-paths.js +42 -0
- package/dist/git-credential.d.ts +69 -0
- package/dist/git-credential.d.ts.map +1 -0
- package/dist/git-recovery-environment.d.ts +31 -0
- package/dist/git-recovery-environment.d.ts.map +1 -0
- package/dist/github-setup.d.ts +122 -0
- package/dist/github-setup.d.ts.map +1 -0
- package/dist/global-flags.d.ts +12 -0
- package/dist/global-flags.d.ts.map +1 -0
- package/dist/help.d.ts +46 -0
- package/dist/help.d.ts.map +1 -0
- package/dist/help.js +93 -25
- package/dist/hosted-boot.d.ts +273 -0
- package/dist/hosted-boot.d.ts.map +1 -0
- package/dist/hosted-bootstrap-rehearsal.d.ts +86 -0
- package/dist/hosted-bootstrap-rehearsal.d.ts.map +1 -0
- package/dist/hosted-bootstrap.d.ts +188 -0
- package/dist/hosted-bootstrap.d.ts.map +1 -0
- package/dist/hosted-broker-server.d.ts +50 -0
- package/dist/hosted-broker-server.d.ts.map +1 -0
- package/dist/hosted-git-http-server.d.ts +36 -0
- package/dist/hosted-git-http-server.d.ts.map +1 -0
- package/dist/hosted-harness-claude.d.ts +195 -0
- package/dist/hosted-harness-claude.d.ts.map +1 -0
- package/dist/hosted-harness-opencode.d.ts +227 -0
- package/dist/hosted-harness-opencode.d.ts.map +1 -0
- package/dist/hosted-harness.d.ts +118 -0
- package/dist/hosted-harness.d.ts.map +1 -0
- package/dist/hosted-harvest.d.ts +68 -0
- package/dist/hosted-harvest.d.ts.map +1 -0
- package/dist/hosted-headless.d.ts +72 -0
- package/dist/hosted-headless.d.ts.map +1 -0
- package/dist/hosted-loop-lifecycle.d.ts +206 -0
- package/dist/hosted-loop-lifecycle.d.ts.map +1 -0
- package/dist/hosted-loop.d.ts +170 -0
- package/dist/hosted-loop.d.ts.map +1 -0
- package/dist/hosted-run-event-sink.d.ts +222 -0
- package/dist/hosted-run-event-sink.d.ts.map +1 -0
- package/dist/hosted-runtime-assets.d.ts +186 -0
- package/dist/hosted-runtime-assets.d.ts.map +1 -0
- package/dist/hosted-session-cli.d.ts +258 -0
- package/dist/hosted-session-cli.d.ts.map +1 -0
- package/dist/hosted-skill-staging.d.ts +73 -0
- package/dist/hosted-skill-staging.d.ts.map +1 -0
- package/dist/http.d.ts +8 -0
- package/dist/http.d.ts.map +1 -0
- package/dist/index.d.ts +22 -0
- package/dist/index.d.ts.map +1 -0
- package/dist/index.js +119 -129
- package/dist/instruction-set-gepa-launch.d.ts +3 -0
- package/dist/instruction-set-gepa-launch.d.ts.map +1 -0
- package/dist/instruction-set-gepa-launch.js +15 -0
- package/dist/instruction-set-loader.d.ts +6 -0
- package/dist/instruction-set-loader.d.ts.map +1 -0
- package/dist/instruction-set-loader.js +59 -0
- package/dist/instruction-set-manifest.d.ts +14 -0
- package/dist/instruction-set-manifest.d.ts.map +1 -0
- package/dist/instruction-set-manifest.js +162 -0
- package/dist/instruction-sets-cli.d.ts +40 -0
- package/dist/instruction-sets-cli.d.ts.map +1 -0
- package/dist/instruction-sets-cli.js +361 -0
- package/dist/json-response.d.ts +4 -0
- package/dist/json-response.d.ts.map +1 -0
- package/dist/jsonl-stream.d.ts +8 -0
- package/dist/jsonl-stream.d.ts.map +1 -0
- package/dist/kill-switch-cli.d.ts +30 -0
- package/dist/kill-switch-cli.d.ts.map +1 -0
- package/dist/loopback-serve.d.ts +25 -0
- package/dist/loopback-serve.d.ts.map +1 -0
- package/dist/manifests-cli.d.ts +22 -0
- package/dist/manifests-cli.d.ts.map +1 -0
- package/dist/markdown-report-input.d.ts +7 -0
- package/dist/markdown-report-input.d.ts.map +1 -0
- package/dist/markdown-report-input.js +15 -1
- package/dist/merge-job.d.ts +155 -0
- package/dist/merge-job.d.ts.map +1 -0
- package/dist/model-configs-cli.d.ts +19 -0
- package/dist/model-configs-cli.d.ts.map +1 -0
- package/dist/model-configs-cli.js +104 -0
- package/dist/optimization-export-cli.d.ts +19 -0
- package/dist/optimization-export-cli.d.ts.map +1 -0
- package/dist/optimizations-list-cli.d.ts +20 -0
- package/dist/optimizations-list-cli.d.ts.map +1 -0
- package/dist/optimizations-promote-cli.d.ts +8 -0
- package/dist/optimizations-promote-cli.d.ts.map +1 -0
- package/dist/optimizations-promote-cli.js +34 -0
- package/dist/option-validation.d.ts +15 -0
- package/dist/option-validation.d.ts.map +1 -0
- package/dist/preview-runtime.d.ts +63 -0
- package/dist/preview-runtime.d.ts.map +1 -0
- package/dist/prompt-draft-push.d.ts +60 -0
- package/dist/prompt-draft-push.d.ts.map +1 -0
- package/dist/prompt-label-cli.d.ts +11 -0
- package/dist/prompt-label-cli.d.ts.map +1 -0
- package/dist/prompt-label-cli.js +29 -0
- package/dist/prompt-length-wire.d.ts +28 -0
- package/dist/prompt-length-wire.d.ts.map +1 -0
- package/dist/prompt-length-wire.js +2 -2
- package/dist/prompt-report-cli.d.ts +10 -0
- package/dist/prompt-report-cli.d.ts.map +1 -0
- package/dist/prompt-report-cli.js +2 -2
- package/dist/prompt-summary-table.d.ts +15 -0
- package/dist/prompt-summary-table.d.ts.map +1 -0
- package/dist/prompt-summary-table.js +6 -0
- package/dist/registration-tag-warning.d.ts +32 -0
- package/dist/registration-tag-warning.d.ts.map +1 -0
- package/dist/report-comments-cli.d.ts +16 -0
- package/dist/report-comments-cli.d.ts.map +1 -0
- package/dist/report-markdown-blankness.d.ts +2 -0
- package/dist/report-markdown-blankness.d.ts.map +1 -0
- package/dist/report-markdown-blankness.js +6 -0
- package/dist/report-replacement-warning.d.ts +17 -0
- package/dist/report-replacement-warning.d.ts.map +1 -0
- package/dist/report-replacement-warning.js +31 -0
- package/dist/repository-backup-core.d.ts +225 -0
- package/dist/repository-backup-core.d.ts.map +1 -0
- package/dist/repository-recovery-manifest.d.ts +83 -0
- package/dist/repository-recovery-manifest.d.ts.map +1 -0
- package/dist/run-event-digest.d.ts +74 -0
- package/dist/run-event-digest.d.ts.map +1 -0
- package/dist/runner-dir-verify.d.ts +66 -0
- package/dist/runner-dir-verify.d.ts.map +1 -0
- package/dist/runner-instruction-set-contract.d.ts +24 -0
- package/dist/runner-instruction-set-contract.d.ts.map +1 -0
- package/dist/runner-instruction-set-contract.js +50 -0
- package/dist/sandbox-hygiene.d.ts +45 -0
- package/dist/sandbox-hygiene.d.ts.map +1 -0
- package/dist/sandbox-provider.d.ts +193 -0
- package/dist/sandbox-provider.d.ts.map +1 -0
- package/dist/scorer-draft-push.d.ts +50 -0
- package/dist/scorer-draft-push.d.ts.map +1 -0
- package/dist/scorers-cli.d.ts +20 -0
- package/dist/scorers-cli.d.ts.map +1 -0
- package/dist/scorers-cli.js +22 -0
- package/dist/scorers-verify-parity-bridge.d.ts +16 -0
- package/dist/scorers-verify-parity-bridge.d.ts.map +1 -0
- package/dist/scorers-verify-parity-bridge.js +90 -0
- package/dist/scorers-verify-parity.d.ts +81 -0
- package/dist/scorers-verify-parity.d.ts.map +1 -0
- package/dist/scorers-verify-parity.js +502 -0
- package/dist/scores-accept-cli.d.ts +21 -0
- package/dist/scores-accept-cli.d.ts.map +1 -0
- package/dist/secret-redaction.d.ts +47 -0
- package/dist/secret-redaction.d.ts.map +1 -0
- package/dist/session-timeline.d.ts +54 -0
- package/dist/session-timeline.d.ts.map +1 -0
- package/dist/setup-prompt.d.ts +5 -0
- package/dist/setup-prompt.d.ts.map +1 -0
- package/dist/skill-installer.d.ts +68 -0
- package/dist/skill-installer.d.ts.map +1 -0
- package/dist/skilled-proposer-launch.d.ts +7 -0
- package/dist/skilled-proposer-launch.d.ts.map +1 -0
- package/dist/skilled-proposer-launch.js +60 -0
- package/dist/smoke.d.ts +102 -0
- package/dist/smoke.d.ts.map +1 -0
- package/dist/start-intent.d.ts +12 -0
- package/dist/start-intent.d.ts.map +1 -0
- package/dist/task-assignment-manifest.d.ts +15 -0
- package/dist/task-assignment-manifest.d.ts.map +1 -0
- package/dist/task-create-error.d.ts +18 -0
- package/dist/task-create-error.d.ts.map +1 -0
- package/dist/team-connectors-cli.d.ts +29 -0
- package/dist/team-connectors-cli.d.ts.map +1 -0
- package/dist/types.d.ts +39 -0
- package/dist/types.d.ts.map +1 -0
- package/dist/vercel-sandbox-provider.d.ts +156 -0
- package/dist/vercel-sandbox-provider.d.ts.map +1 -0
- package/dist/workbench-cli.d.ts +233 -0
- package/dist/workbench-cli.d.ts.map +1 -0
- package/dist/workspace-sync.d.ts +207 -0
- package/dist/workspace-sync.d.ts.map +1 -0
- package/dist/workspace.d.ts +92 -0
- package/dist/workspace.d.ts.map +1 -0
- package/dist/zip-draft-push.d.ts +55 -0
- package/dist/zip-draft-push.d.ts.map +1 -0
- package/package.json +15 -2
- package/requirements/README.md +138 -0
- package/requirements/skilled-proposer.in +66 -0
- package/requirements/skilled-proposer.lock +265 -0
- package/scripts/build-cli-dist.mjs +30 -0
- package/scripts/ensure-skilled-proposer-venv.mjs +5 -0
- package/scripts/orizu-source-checkout.d.mts +1 -0
- package/scripts/orizu-source-checkout.mjs +10 -0
- package/scripts/refresh-skilled-proposer-lock.mjs +256 -0
- package/scripts/vendor-gepa-python.mjs +187 -4
- package/src/diff-comments-cli.ts +41 -3
- package/src/gepa-engine-dispatch.ts +340 -0
- package/src/gepa-python-command.ts +31 -0
- package/src/gepa-python-paths.ts +56 -0
- package/src/help.ts +95 -25
- package/src/index.ts +131 -166
- package/src/instruction-set-gepa-launch.ts +28 -0
- package/src/instruction-set-loader.ts +37 -0
- package/src/instruction-set-manifest.ts +120 -0
- package/src/instruction-sets-cli.ts +306 -0
- package/src/markdown-report-input.ts +19 -1
- package/src/model-configs-cli.ts +119 -0
- package/src/optimizations-promote-cli.ts +41 -0
- package/src/prompt-label-cli.ts +55 -0
- package/src/prompt-length-wire.ts +1 -1
- package/src/prompt-report-cli.ts +2 -2
- package/src/prompt-summary-table.ts +12 -0
- package/src/report-markdown-blankness.ts +6 -0
- package/src/report-replacement-warning.ts +35 -0
- package/src/runner-instruction-set-contract.ts +64 -0
- package/src/scorers-cli.ts +40 -0
- package/src/scorers-verify-parity-bridge.ts +90 -0
- package/src/scorers-verify-parity.ts +564 -0
- package/src/skilled-proposer-launch.ts +89 -0
- package/src/skilled-proposer-venv-manager.mjs +757 -0
- package/tsconfig.json +2 -0
- package/vendor/gepa-python/LICENSE +7 -0
- package/vendor/gepa-python/manifest.json +107 -0
- package/vendor/gepa-python/src/gepa/__init__.py +22 -0
- package/vendor/gepa-python/src/gepa/adapters/__init__.py +0 -0
- package/vendor/gepa-python/src/gepa/adapters/anymaths_adapter/__init__.py +1 -0
- package/vendor/gepa-python/src/gepa/adapters/anymaths_adapter/anymaths_adapter.py +174 -0
- package/vendor/gepa-python/src/gepa/adapters/confidence_adapter/__init__.py +26 -0
- package/vendor/gepa-python/src/gepa/adapters/confidence_adapter/confidence_adapter.py +618 -0
- package/vendor/gepa-python/src/gepa/adapters/confidence_adapter/scoring.py +206 -0
- package/vendor/gepa-python/src/gepa/adapters/default_adapter/__init__.py +0 -0
- package/vendor/gepa-python/src/gepa/adapters/default_adapter/default_adapter.py +206 -0
- package/vendor/gepa-python/src/gepa/adapters/dspy_adapter/__init__.py +0 -0
- package/vendor/gepa-python/src/gepa/adapters/dspy_adapter/dspy_adapter.py +525 -0
- package/vendor/gepa-python/src/gepa/adapters/dspy_adapter/instruction_proposal.py +500 -0
- package/vendor/gepa-python/src/gepa/adapters/dspy_full_program_adapter/__init__.py +6 -0
- package/vendor/gepa-python/src/gepa/adapters/dspy_full_program_adapter/dspy_program_proposal_signature.py +137 -0
- package/vendor/gepa-python/src/gepa/adapters/dspy_full_program_adapter/full_program_adapter.py +269 -0
- package/vendor/gepa-python/src/gepa/adapters/generic_rag_adapter/__init__.py +56 -0
- package/vendor/gepa-python/src/gepa/adapters/generic_rag_adapter/evaluation_metrics.py +226 -0
- package/vendor/gepa-python/src/gepa/adapters/generic_rag_adapter/generic_rag_adapter.py +496 -0
- package/vendor/gepa-python/src/gepa/adapters/generic_rag_adapter/rag_pipeline.py +238 -0
- package/vendor/gepa-python/src/gepa/adapters/generic_rag_adapter/vector_store_interface.py +212 -0
- package/vendor/gepa-python/src/gepa/adapters/generic_rag_adapter/vector_stores/__init__.py +2 -0
- package/vendor/gepa-python/src/gepa/adapters/generic_rag_adapter/vector_stores/chroma_store.py +196 -0
- package/vendor/gepa-python/src/gepa/adapters/generic_rag_adapter/vector_stores/lancedb_store.py +422 -0
- package/vendor/gepa-python/src/gepa/adapters/generic_rag_adapter/vector_stores/milvus_store.py +409 -0
- package/vendor/gepa-python/src/gepa/adapters/generic_rag_adapter/vector_stores/qdrant_store.py +368 -0
- package/vendor/gepa-python/src/gepa/adapters/generic_rag_adapter/vector_stores/weaviate_store.py +418 -0
- package/vendor/gepa-python/src/gepa/adapters/langchain_adapter/__init__.py +11 -0
- package/vendor/gepa-python/src/gepa/adapters/langchain_adapter/langchain_adapter.py +261 -0
- package/vendor/gepa-python/src/gepa/adapters/mcp_adapter/__init__.py +37 -0
- package/vendor/gepa-python/src/gepa/adapters/mcp_adapter/mcp_adapter.py +693 -0
- package/vendor/gepa-python/src/gepa/adapters/mcp_adapter/mcp_client.py +364 -0
- package/vendor/gepa-python/src/gepa/adapters/optimize_anything_adapter/optimize_anything_adapter.py +879 -0
- package/vendor/gepa-python/src/gepa/adapters/terminal_bench_adapter/__init__.py +0 -0
- package/vendor/gepa-python/src/gepa/adapters/terminal_bench_adapter/terminal_bench_adapter.py +217 -0
- package/vendor/gepa-python/src/gepa/api.py +468 -0
- package/vendor/gepa-python/src/gepa/core/__init__.py +0 -0
- package/vendor/gepa-python/src/gepa/core/adapter.py +227 -0
- package/vendor/gepa-python/src/gepa/core/callbacks.py +551 -0
- package/vendor/gepa-python/src/gepa/core/data_loader.py +74 -0
- package/vendor/gepa-python/src/gepa/core/engine.py +1025 -0
- package/vendor/gepa-python/src/gepa/core/result.py +271 -0
- package/vendor/gepa-python/src/gepa/core/state.py +706 -0
- package/vendor/gepa-python/src/gepa/examples/__init__.py +0 -0
- package/vendor/gepa-python/src/gepa/examples/aime.py +24 -0
- package/vendor/gepa-python/src/gepa/examples/anymaths-bench/eval_default.py +111 -0
- package/vendor/gepa-python/src/gepa/examples/anymaths-bench/train_anymaths.py +177 -0
- package/vendor/gepa-python/src/gepa/examples/mcp_adapter/__init__.py +4 -0
- package/vendor/gepa-python/src/gepa/examples/mcp_adapter/mcp_optimization_example.py +456 -0
- package/vendor/gepa-python/src/gepa/examples/rag_adapter/__init__.py +9 -0
- package/vendor/gepa-python/src/gepa/examples/rag_adapter/rag_optimization.py +824 -0
- package/vendor/gepa-python/src/gepa/examples/terminal-bench/train_terminus.py +161 -0
- package/vendor/gepa-python/src/gepa/gepa_utils.py +117 -0
- package/vendor/gepa-python/src/gepa/gskill/__init__.py +1 -0
- package/vendor/gepa-python/src/gepa/gskill/gskill/__init__.py +0 -0
- package/vendor/gepa-python/src/gepa/gskill/gskill/cost_tracker.py +194 -0
- package/vendor/gepa-python/src/gepa/gskill/gskill/evaluate/__init__.py +0 -0
- package/vendor/gepa-python/src/gepa/gskill/gskill/evaluate/claude_code.py +1117 -0
- package/vendor/gepa-python/src/gepa/gskill/gskill/evaluate/claude_code_skills.py +613 -0
- package/vendor/gepa-python/src/gepa/gskill/gskill/evaluate/mini_swe_agent.py +440 -0
- package/vendor/gepa-python/src/gepa/gskill/gskill/experiment_logger.py +367 -0
- package/vendor/gepa-python/src/gepa/gskill/gskill/swe_fitness_fn.py +209 -0
- package/vendor/gepa-python/src/gepa/gskill/gskill/swe_harness.py +352 -0
- package/vendor/gepa-python/src/gepa/gskill/gskill/train_optimize_anything.py +830 -0
- package/vendor/gepa-python/src/gepa/image.py +104 -0
- package/vendor/gepa-python/src/gepa/lm.py +261 -0
- package/vendor/gepa-python/src/gepa/logging/__init__.py +0 -0
- package/vendor/gepa-python/src/gepa/logging/experiment_tracker.py +457 -0
- package/vendor/gepa-python/src/gepa/logging/logger.py +75 -0
- package/vendor/gepa-python/src/gepa/logging/utils.py +130 -0
- package/vendor/gepa-python/src/gepa/optimize_anything.py +1701 -0
- package/vendor/gepa-python/src/gepa/proposer/__init__.py +0 -0
- package/vendor/gepa-python/src/gepa/proposer/base.py +53 -0
- package/vendor/gepa-python/src/gepa/proposer/merge.py +403 -0
- package/vendor/gepa-python/src/gepa/proposer/reflective_mutation/__init__.py +0 -0
- package/vendor/gepa-python/src/gepa/proposer/reflective_mutation/base.py +64 -0
- package/vendor/gepa-python/src/gepa/proposer/reflective_mutation/reflection_lm.py +178 -0
- package/vendor/gepa-python/src/gepa/proposer/reflective_mutation/reflective_mutation.py +641 -0
- package/vendor/gepa-python/src/gepa/py.typed +0 -0
- package/vendor/gepa-python/src/gepa/strategies/__init__.py +0 -0
- package/vendor/gepa-python/src/gepa/strategies/acceptance.py +66 -0
- package/vendor/gepa-python/src/gepa/strategies/batch_sampler.py +103 -0
- package/vendor/gepa-python/src/gepa/strategies/candidate_selector.py +82 -0
- package/vendor/gepa-python/src/gepa/strategies/component_selector.py +36 -0
- package/vendor/gepa-python/src/gepa/strategies/eval_policy.py +64 -0
- package/vendor/gepa-python/src/gepa/strategies/instruction_proposal.py +153 -0
- package/vendor/gepa-python/src/gepa/strategies/proposal_sampling.py +128 -0
- package/vendor/gepa-python/src/gepa/strategies/proposal_selection.py +84 -0
- package/vendor/gepa-python/src/gepa/utils/__init__.py +63 -0
- package/vendor/gepa-python/src/gepa/utils/code_execution.py +626 -0
- package/vendor/gepa-python/src/gepa/utils/stdio_capture.py +145 -0
- package/vendor/gepa-python/src/gepa/utils/stop_condition.py +230 -0
- package/vendor/gepa-python/src/gepa/visualization.py +359 -0
- package/vendor/orizu-gepa/pyproject.toml +18 -0
- package/vendor/orizu-gepa/src/orizu_gepa_connector/__init__.py +9 -0
- package/vendor/orizu-gepa/src/orizu_gepa_connector/__main__.py +5 -0
- package/vendor/orizu-gepa/src/orizu_gepa_connector/adapter.py +138 -0
- package/vendor/orizu-gepa/src/orizu_gepa_connector/callbacks.py +666 -0
- package/vendor/orizu-gepa/src/orizu_gepa_connector/cli.py +7 -0
- package/vendor/orizu-gepa/src/orizu_gepa_connector/engine.py +269 -0
- package/vendor/orizu-gepa/src/orizu_gepa_connector/instruction_set_loader.py +52 -0
- package/vendor/orizu-gepa/src/orizu_gepa_connector/preflight.py +64 -0
- package/vendor/orizu-gepa/src/orizu_gepa_connector/reflection.py +186 -0
- package/vendor/orizu-gepa/src/orizu_gepa_connector/runtime.py +721 -0
- package/vendor/orizu-gepa/src/orizu_gepa_connector/skilled_proposer_bridge.py +381 -0
- package/vendor/orizu-gepa/src/orizu_gepa_connector/stop_conditions.py +55 -0
- package/vendor/orizu-gepa/src/orizu_gepa_connector/translator.py +203 -0
- package/vendor/orizu-gepa-python/pyproject.toml +1 -1
- package/vendor/orizu-gepa-python/src/orizu_gepa/cli.py +1 -0
- package/vendor/orizu-gepa-python/src/orizu_gepa/client.py +52 -7
- package/vendor/orizu-gepa-python/src/orizu_gepa/local_log.py +52 -0
- package/vendor/orizu-gepa-python/src/orizu_gepa/optimizer.py +10 -5
- package/vendor/orizu-gepa-python/src/orizu_gepa/reflection.py +323 -68
- package/vendor/orizu-gepa-python/src/orizu_gepa/runner.py +207 -13
- package/vendor/skills/orizu-cli/SKILL.md +3 -1
- package/vendor/skills/orizu-cli/references/cli-reference.md +85 -1
- package/vendor/skills/orizu-cli/references/migrate-existing-gepa-setup.md +303 -0
- package/vendor/skills/orizu-cli/references/optimization-reports.md +1 -1
- package/vendor/skills/orizu-cli/references/optimization-with-gepa.md +26 -1
- package/vendor/skills/orizu-cli/references/prompt-control-plane.md +113 -10
|
@@ -1,10 +1,12 @@
|
|
|
1
1
|
from __future__ import annotations
|
|
2
2
|
|
|
3
3
|
import json
|
|
4
|
+
import hashlib
|
|
4
5
|
import os
|
|
5
6
|
import subprocess
|
|
6
7
|
import tempfile
|
|
7
8
|
import zipfile
|
|
9
|
+
import re
|
|
8
10
|
from pathlib import Path
|
|
9
11
|
from typing import Any
|
|
10
12
|
|
|
@@ -38,7 +40,7 @@ ALLOWED_RUNNER_ENV_KEYS = {
|
|
|
38
40
|
|
|
39
41
|
def read_manifest(runner_dir: str | Path) -> dict[str, Any]:
|
|
40
42
|
path = Path(runner_dir) / "manifest.json"
|
|
41
|
-
return json.loads(path.read_text())
|
|
43
|
+
return json.loads(path.read_text(encoding="utf-8"))
|
|
42
44
|
|
|
43
45
|
|
|
44
46
|
def _safe_extract_zip(zip_path: Path, destination: Path) -> None:
|
|
@@ -64,7 +66,7 @@ def _safe_extract_zip(zip_path: Path, destination: Path) -> None:
|
|
|
64
66
|
archive.extractall(destination)
|
|
65
67
|
|
|
66
68
|
|
|
67
|
-
def _runner_env(input_path: Path, output_path: Path) -> dict[str, str]:
|
|
69
|
+
def _runner_env(input_path: Path, output_path: Path, instruction_set_dir: Path | None = None) -> dict[str, str]:
|
|
68
70
|
env = {
|
|
69
71
|
key: value
|
|
70
72
|
for key, value in os.environ.items()
|
|
@@ -72,9 +74,156 @@ def _runner_env(input_path: Path, output_path: Path) -> dict[str, str]:
|
|
|
72
74
|
}
|
|
73
75
|
env["ORIZU_RUNNER_INPUT_PATH"] = str(input_path)
|
|
74
76
|
env["ORIZU_RUNNER_OUTPUT_PATH"] = str(output_path)
|
|
77
|
+
env.pop("ORIZU_INSTRUCTION_SET_DIR", None)
|
|
78
|
+
if instruction_set_dir is not None:
|
|
79
|
+
env["ORIZU_INSTRUCTION_SET_DIR"] = str(instruction_set_dir)
|
|
75
80
|
return env
|
|
76
81
|
|
|
77
82
|
|
|
83
|
+
def _instruction_component(component: Any) -> tuple[str | None, dict[str, str] | None]:
|
|
84
|
+
if isinstance(component, str):
|
|
85
|
+
return component, None
|
|
86
|
+
if not isinstance(component, dict):
|
|
87
|
+
raise RuntimeError("instruction_set_unresolvable")
|
|
88
|
+
body = component.get("body")
|
|
89
|
+
if isinstance(body, str):
|
|
90
|
+
return body, None
|
|
91
|
+
repo_path = component.get("repoPath", component.get("repo_path"))
|
|
92
|
+
content_sha = component.get("contentSha", component.get("content_sha"))
|
|
93
|
+
commit_sha = component.get("commitSha", component.get("commit_sha"))
|
|
94
|
+
if all(isinstance(value, str) and value for value in (repo_path, content_sha, commit_sha)):
|
|
95
|
+
return None, {"repoPath": repo_path, "contentSha": content_sha, "commitSha": commit_sha}
|
|
96
|
+
raise RuntimeError("instruction_set_unresolvable")
|
|
97
|
+
|
|
98
|
+
|
|
99
|
+
SAFE_SEGMENT = re.compile(r"^[A-Za-z0-9._-]+$")
|
|
100
|
+
|
|
101
|
+
|
|
102
|
+
def _safe_segment(value: str) -> None:
|
|
103
|
+
if not SAFE_SEGMENT.fullmatch(value) or value in {".", ".."} or value.startswith("."):
|
|
104
|
+
raise RuntimeError("instruction_set_path_unsafe")
|
|
105
|
+
|
|
106
|
+
|
|
107
|
+
def _slug(identity: str) -> str:
|
|
108
|
+
return re.sub(r"[^A-Za-z0-9._-]", "_", identity.replace("/", "__"))
|
|
109
|
+
|
|
110
|
+
|
|
111
|
+
def _write_instruction_set_layout(root: Path, instruction_set: dict[str, Any]) -> None:
|
|
112
|
+
"""Mirror the CLI syncToDisk manifest/layout for one exact tuple."""
|
|
113
|
+
name = instruction_set.get("name")
|
|
114
|
+
shape = instruction_set.get("shape")
|
|
115
|
+
model_config = instruction_set.get("model_config", instruction_set.get("modelConfig"))
|
|
116
|
+
if not isinstance(name, str) or not isinstance(shape, list) or not all(isinstance(key, str) for key in shape):
|
|
117
|
+
raise RuntimeError("instruction_set_unresolvable")
|
|
118
|
+
if not isinstance(model_config, dict) or not isinstance(model_config.get("identity"), str):
|
|
119
|
+
raise RuntimeError("instruction_set_unresolvable")
|
|
120
|
+
components = dict(instruction_set.get("components") or {})
|
|
121
|
+
components.update(instruction_set.get("pinned_components", instruction_set.get("pinnedComponents", {})) or {})
|
|
122
|
+
if any(key not in components for key in shape):
|
|
123
|
+
raise RuntimeError("instruction_set_tuple_incomplete")
|
|
124
|
+
if not isinstance(components, dict):
|
|
125
|
+
raise RuntimeError("instruction_set_unresolvable")
|
|
126
|
+
profile_version_id = instruction_set.get("profile_version_id", instruction_set.get("profileVersionId"))
|
|
127
|
+
version_number = instruction_set.get("version_number", instruction_set.get("versionNumber"))
|
|
128
|
+
if not isinstance(profile_version_id, str) or not isinstance(version_number, int):
|
|
129
|
+
raise RuntimeError("instruction_set_unresolvable")
|
|
130
|
+
|
|
131
|
+
_safe_segment(name)
|
|
132
|
+
for key in shape:
|
|
133
|
+
_safe_segment(key)
|
|
134
|
+
identity = model_config["identity"]
|
|
135
|
+
slug = _slug(identity)
|
|
136
|
+
_safe_segment(slug)
|
|
137
|
+
destination = root / name
|
|
138
|
+
destination.mkdir(parents=True, exist_ok=True)
|
|
139
|
+
|
|
140
|
+
def write_material(relative_root: str) -> dict[str, Any]:
|
|
141
|
+
files: dict[str, str] = {}
|
|
142
|
+
synced_content_sha256: dict[str, str] = {}
|
|
143
|
+
pinned_components: dict[str, dict[str, str]] = {}
|
|
144
|
+
for key in sorted(shape):
|
|
145
|
+
if key not in components:
|
|
146
|
+
raise RuntimeError("instruction_set_unresolvable")
|
|
147
|
+
body, pin = _instruction_component(components[key])
|
|
148
|
+
if body is not None:
|
|
149
|
+
relative = f"{relative_root}/{key}.md"
|
|
150
|
+
target = destination / relative
|
|
151
|
+
target.parent.mkdir(parents=True, exist_ok=True)
|
|
152
|
+
target.write_text(body, encoding="utf-8")
|
|
153
|
+
files[key] = relative
|
|
154
|
+
synced_content_sha256[key] = hashlib.sha256(body.encode("utf-8")).hexdigest()
|
|
155
|
+
elif pin is not None:
|
|
156
|
+
pinned_components[key] = pin
|
|
157
|
+
material: dict[str, Any] = {
|
|
158
|
+
"profileVersionId": profile_version_id,
|
|
159
|
+
"versionNumber": version_number,
|
|
160
|
+
"modelConfigIdentity": identity,
|
|
161
|
+
"resolvedFrom": "exact_profile_version",
|
|
162
|
+
"files": files,
|
|
163
|
+
"syncedContentSha256": synced_content_sha256,
|
|
164
|
+
}
|
|
165
|
+
if pinned_components:
|
|
166
|
+
material["pinnedComponents"] = pinned_components
|
|
167
|
+
return material
|
|
168
|
+
|
|
169
|
+
material = write_material("default")
|
|
170
|
+
profile = {
|
|
171
|
+
"modelConfigIdentity": identity,
|
|
172
|
+
"slug": slug,
|
|
173
|
+
"resolvedFrom": "exact_profile_version",
|
|
174
|
+
"production": write_material(f"profiles/{slug}"),
|
|
175
|
+
}
|
|
176
|
+
authoring_components = [
|
|
177
|
+
{
|
|
178
|
+
"key": key,
|
|
179
|
+
"path": relative,
|
|
180
|
+
"syncedContentSha256": material["syncedContentSha256"][key],
|
|
181
|
+
}
|
|
182
|
+
for key, relative in material["files"].items()
|
|
183
|
+
]
|
|
184
|
+
authoring_components.extend(
|
|
185
|
+
{
|
|
186
|
+
"key": key,
|
|
187
|
+
"modelConfig": identity,
|
|
188
|
+
"path": relative,
|
|
189
|
+
"syncedContentSha256": profile["production"]["syncedContentSha256"][key],
|
|
190
|
+
}
|
|
191
|
+
for key, relative in profile["production"]["files"].items()
|
|
192
|
+
)
|
|
193
|
+
manifest = {
|
|
194
|
+
"manifestVersion": 1,
|
|
195
|
+
"name": name,
|
|
196
|
+
"shape": shape,
|
|
197
|
+
"components": authoring_components,
|
|
198
|
+
"default": material,
|
|
199
|
+
"profiles": [profile],
|
|
200
|
+
}
|
|
201
|
+
(destination / "manifest.json").write_text(json.dumps(manifest, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
|
202
|
+
|
|
203
|
+
|
|
204
|
+
def _runner_instruction_set_payload(instruction_set: dict[str, Any]) -> dict[str, Any]:
|
|
205
|
+
"""The runner receives the tuple, while sync-only provenance stays on disk."""
|
|
206
|
+
components: dict[str, Any] = {}
|
|
207
|
+
pinned_components: dict[str, Any] = dict(instruction_set.get("pinned_components", instruction_set.get("pinnedComponents", {})) or {})
|
|
208
|
+
for key, component in (instruction_set.get("components") or {}).items():
|
|
209
|
+
body, pin = _instruction_component(component)
|
|
210
|
+
if body is not None:
|
|
211
|
+
components[key] = body
|
|
212
|
+
elif pin is not None:
|
|
213
|
+
pinned_components[key] = pin
|
|
214
|
+
payload = {
|
|
215
|
+
"name": instruction_set.get("name"),
|
|
216
|
+
"model_config": instruction_set.get("model_config", instruction_set.get("modelConfig")),
|
|
217
|
+
"shape": instruction_set.get("shape"),
|
|
218
|
+
**({"model_config_settings_version_id": instruction_set["model_config_settings_version_id"]} if isinstance(instruction_set.get("model_config_settings_version_id"), str) else {}),
|
|
219
|
+
**({"prompt_component_key": instruction_set["prompt_component_key"]} if isinstance(instruction_set.get("prompt_component_key"), str) else {}),
|
|
220
|
+
"components": components,
|
|
221
|
+
}
|
|
222
|
+
if pinned_components:
|
|
223
|
+
payload["pinned_components"] = pinned_components
|
|
224
|
+
return payload
|
|
225
|
+
|
|
226
|
+
|
|
78
227
|
def _bounded_stream(value: str) -> str:
|
|
79
228
|
encoded = value.encode("utf-8", errors="replace")
|
|
80
229
|
if len(encoded) <= MAX_RUNNER_OUTPUT_BYTES:
|
|
@@ -102,6 +251,7 @@ def run_file_contract_runner(
|
|
|
102
251
|
prompt_version_id: str,
|
|
103
252
|
runner_version_id: str,
|
|
104
253
|
run_id: str | None,
|
|
254
|
+
prompt_body_present: bool = True,
|
|
105
255
|
timeout_seconds: int = DEFAULT_RUNNER_TIMEOUT_SECONDS,
|
|
106
256
|
extra_payload: dict[str, Any] | None = None,
|
|
107
257
|
) -> RunnerCallResult:
|
|
@@ -114,26 +264,37 @@ def run_file_contract_runner(
|
|
|
114
264
|
with tempfile.TemporaryDirectory(prefix="orizu-gepa-call-") as temp_dir:
|
|
115
265
|
input_path = Path(temp_dir) / "input.json"
|
|
116
266
|
output_path = Path(temp_dir) / "output.json"
|
|
267
|
+
instruction_set = (extra_payload or {}).get("instruction_set")
|
|
268
|
+
instruction_set_dir = Path(temp_dir) / "instruction-set" if isinstance(instruction_set, dict) else None
|
|
269
|
+
if instruction_set_dir is not None:
|
|
270
|
+
_write_instruction_set_layout(instruction_set_dir, instruction_set)
|
|
117
271
|
# Extra payload first: the core file-contract keys are authoritative
|
|
118
272
|
# and never overridable by adapter-supplied companions.
|
|
273
|
+
prompt = {
|
|
274
|
+
"body_kind": body_kind,
|
|
275
|
+
"provider_settings": provider_settings,
|
|
276
|
+
}
|
|
277
|
+
# The exec-context response decides whether this key exists. Runners
|
|
278
|
+
# mirror it; no local shape heuristic may silently remove body bytes.
|
|
279
|
+
if prompt_body_present:
|
|
280
|
+
prompt["body"] = prompt_body
|
|
281
|
+
payload_extra = dict(extra_payload or {})
|
|
282
|
+
if isinstance(instruction_set, dict):
|
|
283
|
+
payload_extra["instruction_set"] = _runner_instruction_set_payload(instruction_set)
|
|
119
284
|
input_path.write_text(json.dumps({
|
|
120
|
-
**
|
|
285
|
+
**payload_extra,
|
|
121
286
|
"row": row,
|
|
122
|
-
"prompt":
|
|
123
|
-
"body": prompt_body,
|
|
124
|
-
"body_kind": body_kind,
|
|
125
|
-
"provider_settings": provider_settings,
|
|
126
|
-
},
|
|
287
|
+
"prompt": prompt,
|
|
127
288
|
"prompt_version_id": prompt_version_id,
|
|
128
289
|
"runner_version_id": runner_version_id,
|
|
129
290
|
"run_id": run_id,
|
|
130
|
-
}, ensure_ascii=False))
|
|
291
|
+
}, ensure_ascii=False), encoding="utf-8")
|
|
131
292
|
|
|
132
293
|
try:
|
|
133
294
|
result = subprocess.run(
|
|
134
295
|
command,
|
|
135
296
|
cwd=runner_path,
|
|
136
|
-
env=_runner_env(input_path, output_path),
|
|
297
|
+
env=_runner_env(input_path, output_path, instruction_set_dir),
|
|
137
298
|
text=True,
|
|
138
299
|
capture_output=True,
|
|
139
300
|
check=False,
|
|
@@ -152,7 +313,7 @@ def run_file_contract_runner(
|
|
|
152
313
|
raise RuntimeError(
|
|
153
314
|
f"Runner failed with exit code {result.returncode}: {stderr or stdout}"
|
|
154
315
|
)
|
|
155
|
-
data = json.loads(output_path.read_text())
|
|
316
|
+
data = json.loads(output_path.read_text(encoding="utf-8"))
|
|
156
317
|
known = {
|
|
157
318
|
"model_response",
|
|
158
319
|
"raw_api_response",
|
|
@@ -175,16 +336,46 @@ def run_file_contract_runner(
|
|
|
175
336
|
|
|
176
337
|
|
|
177
338
|
def make_candidate_runner(runner_dir: str | Path, run_id: str | None):
|
|
178
|
-
def _run(candidate_text: str, row: DatasetRow, prompt_context: PromptContext, candidate_id: str) -> RunnerCallResult:
|
|
339
|
+
def _run(candidate_text: str | dict[str, str], row: DatasetRow, prompt_context: PromptContext, candidate_id: str) -> RunnerCallResult:
|
|
340
|
+
instruction_set = prompt_context.instruction_set
|
|
341
|
+
if instruction_set:
|
|
342
|
+
is_tuple_rewrite = isinstance(candidate_text, dict) and len(candidate_text) > 1
|
|
343
|
+
if not isinstance(candidate_text, dict):
|
|
344
|
+
prompt_component_key = instruction_set.get("prompt_component_key")
|
|
345
|
+
candidate_components = {prompt_component_key: candidate_text} if isinstance(prompt_component_key, str) else None
|
|
346
|
+
else:
|
|
347
|
+
candidate_components = candidate_text
|
|
348
|
+
if not candidate_components or not all(isinstance(key, str) and isinstance(value, str) for key, value in candidate_components.items()):
|
|
349
|
+
raise RuntimeError("instruction_set_candidate_component_unresolvable")
|
|
350
|
+
instruction_set = dict(instruction_set)
|
|
351
|
+
existing_components = dict(instruction_set.get("components") or {})
|
|
352
|
+
pinned_components = dict(instruction_set.get("pinned_components") or {})
|
|
353
|
+
if not set(candidate_components).issubset(set(existing_components) | set(pinned_components)):
|
|
354
|
+
raise RuntimeError("instruction_set_candidate_component_unresolvable")
|
|
355
|
+
instruction_set["components"] = {
|
|
356
|
+
**existing_components,
|
|
357
|
+
**candidate_components,
|
|
358
|
+
}
|
|
359
|
+
for key in candidate_components:
|
|
360
|
+
pinned_components.pop(key, None)
|
|
361
|
+
instruction_set["pinned_components"] = pinned_components
|
|
362
|
+
# The candidate is the execution contract. Legacy string rewrites
|
|
363
|
+
# preserve the body the server supplied; only a genuine tuple
|
|
364
|
+
# rewrite omits it.
|
|
365
|
+
prompt_body = None if is_tuple_rewrite else candidate_text
|
|
366
|
+
else:
|
|
367
|
+
prompt_body = candidate_text
|
|
179
368
|
return run_file_contract_runner(
|
|
180
369
|
runner_dir=runner_dir,
|
|
181
370
|
row=row.row,
|
|
182
|
-
prompt_body=
|
|
371
|
+
prompt_body=prompt_body,
|
|
183
372
|
body_kind=prompt_context.body_kind,
|
|
184
373
|
provider_settings=prompt_context.provider_settings,
|
|
185
374
|
prompt_version_id=prompt_context.prompt_version_id,
|
|
186
375
|
runner_version_id=prompt_context.runner_version_id,
|
|
187
376
|
run_id=run_id,
|
|
377
|
+
prompt_body_present=not instruction_set or not is_tuple_rewrite,
|
|
378
|
+
extra_payload={"instruction_set": instruction_set} if instruction_set else None,
|
|
188
379
|
)
|
|
189
380
|
return _run
|
|
190
381
|
|
|
@@ -318,10 +509,13 @@ def make_scorer_runner(
|
|
|
318
509
|
"candidate_raw_response": candidate_result.raw_api_response,
|
|
319
510
|
"candidate_error": candidate_result.error,
|
|
320
511
|
}
|
|
512
|
+
if scorer_context.instruction_set:
|
|
513
|
+
extra_payload = {**(extra_payload or {}), "instruction_set": scorer_context.instruction_set}
|
|
321
514
|
return run_file_contract_runner(
|
|
322
515
|
runner_dir=runner_dir,
|
|
323
516
|
row=scorer_row,
|
|
324
517
|
prompt_body=scorer_context.body,
|
|
518
|
+
prompt_body_present=scorer_context.body_present,
|
|
325
519
|
body_kind=scorer_context.body_kind,
|
|
326
520
|
provider_settings=scorer_context.provider_settings,
|
|
327
521
|
prompt_version_id=scorer_context.prompt_version_id,
|
|
@@ -152,7 +152,8 @@ Local execution workflow:
|
|
|
152
152
|
|
|
153
153
|
Bundled `run-gepa` reflection behavior:
|
|
154
154
|
- **Scorer contract:** `run-gepa` sends the scorer runner a GEPA-shaped row (`{source_row, candidate_id, candidate_output, …}`), NOT the flat-row shape used by `runners exec --scorer-version`. A judge runner built for flat-row score runs will silently score every candidate 0 unless you pass `--scorer-input-contract flat_row` (add `--scorer-candidate-field <row-field>`, e.g. `draft`, when the judge reads the candidate output from a named row field). Do not hand-write an adapter runner — the flag applies the official adapter without changing the registered runner bytes, so it composes with `--scorer-runner-dir` content-sha verification. `run-gepa` validates the contract on the seed at launch and refuses a uniformly-worst-scoring seed with a diagnosis; `--allow-degenerate-seed` overrides only when the seed genuinely deserves the worst score everywhere. Both contracts: `references/prompt-control-plane.md` ("Scorer-Runner Input Contracts").
|
|
155
|
-
- Budget controls are mutually exclusive. Use at most one of `--budget auto|light|medium|heavy`, `--max-metric-calls <n>`, `--max-full-evals <n>`, or `--max-
|
|
155
|
+
- Budget controls are mutually exclusive. Use at most one of `--budget auto|light|medium|heavy`, `--max-metric-calls <n>`, `--max-full-evals <n>`, `--max-iterations <n>`, or `--max-candidate-proposals <n>`. The proposal cap is official-engine-only. If none is provided, `run-gepa` uses `--budget auto`, the balanced medium preset.
|
|
156
|
+
- **Skilled proposer:** opt in with `--candidate-proposer skilled-proposer` on the official engine; the selected run prepares or reuses its managed Python environment automatically. Bound proposal work with `--proposal-max-tokens` and `--proposal-max-calls`, independently of `--max-metric-calls`. Read `references/prompt-control-plane.md` ("Skilled proposer") for validation and recovery; readiness is proven when optimizer execution begins without an `ALI_1505_*` environment diagnostic.
|
|
156
157
|
- The reflective LM's final text is used verbatim as the next candidate prompt. It should return only the complete updated prompt body, not analysis, labels, XML tags, or markdown fences.
|
|
157
158
|
- Keep provider-native reasoning controls separate from the prompt text with `--reflection-provider-settings <json|@file>`.
|
|
158
159
|
- Reflection max-token limits are explicit config, not implicit defaults. Use `--reflection-max-tokens <n>` when a provider requires an output cap; this maps to Anthropic `max_tokens` and OpenAI `max_output_tokens`.
|
|
@@ -175,6 +176,7 @@ Detailed walkthrough — GEPA mechanics, Orizu-tracked optimization, optional DS
|
|
|
175
176
|
- `references/building-apps.md` — labeler app contract, design principles, common patterns, offline smoke test.
|
|
176
177
|
- `references/building-judges.md` — judge/scorer authoring + TPR/TNR validation.
|
|
177
178
|
- `references/optimization-with-gepa.md` — GEPA optimization loop, with DSPy only as external context.
|
|
179
|
+
- `references/migrate-existing-gepa-setup.md` — migrating a customer's existing official-GEPA script into Orizu, ending in `orizu scorers verify-parity` before any optimization.
|
|
178
180
|
- `references/optimization-reports.md` — report template, GEPA diagnostics, kappa/confusion-matrix guidance, and next-step recommendations.
|
|
179
181
|
- `scripts/test-app.mjs` — smoke test for `App.tsx` + schemas before `orizu apps create` (runs on plain `node`).
|
|
180
182
|
|
|
@@ -77,6 +77,65 @@ orizu projects list --team my-team
|
|
|
77
77
|
orizu projects create --name "Quality Eval" --team my-team
|
|
78
78
|
```
|
|
79
79
|
|
|
80
|
+
### Instruction sets
|
|
81
|
+
|
|
82
|
+
```bash
|
|
83
|
+
orizu instruction-sets list --project my-team/quality-eval [--status active|archived|all] [--json]
|
|
84
|
+
orizu instruction-sets show <set> --project my-team/quality-eval [--status active|archived|all] [--json]
|
|
85
|
+
orizu instruction-sets create ./orizu.instruction-set.json --project my-team/quality-eval [--runner-version <id>] [--model-config <identity>] [--json]
|
|
86
|
+
orizu instruction-sets push ./orizu.instruction-set.json --project my-team/quality-eval [--runner-version <id>] [--json]
|
|
87
|
+
orizu instruction-sets sync <set> --out ./instructions --project my-team/quality-eval [--model-config <identity>] [--json]
|
|
88
|
+
orizu instruction-sets profiles new <set> --project my-team/quality-eval --model-config <identity> [--json]
|
|
89
|
+
orizu instruction-sets profiles promote <set> --project my-team/quality-eval --model-config <identity> --version <n> [--json]
|
|
90
|
+
orizu instruction-sets profiles rollback <set> --project my-team/quality-eval --model-config <identity> --to <n> [--json]
|
|
91
|
+
orizu instruction-sets default show <set> --project my-team/quality-eval [--json]
|
|
92
|
+
orizu instruction-sets default move <set> --project my-team/quality-eval --model-config <identity> --version <n> [--json]
|
|
93
|
+
orizu instruction-sets shape add <set> --project my-team/quality-eval --key <key> --from <manifest> [--json]
|
|
94
|
+
orizu instruction-sets shape remove <set> --project my-team/quality-eval --key <key> [--json]
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
`list` returns each instruction set and its ordered shape. `show` returns the
|
|
98
|
+
default and every model-config profile's production state. `create` and `push`
|
|
99
|
+
read a local manifest; JSON output is one document per line.
|
|
100
|
+
|
|
101
|
+
`profiles new` copies the default component map into a model-config profile without
|
|
102
|
+
changing its production resolution; hosted agents may create this seed because
|
|
103
|
+
it does not move production. `profiles promote` moves that profile to a
|
|
104
|
+
version, and `profiles rollback` creates a new version copied from `--to`,
|
|
105
|
+
including that target version's settings version, before moving production.
|
|
106
|
+
These label-moving commands are human-only. For a single-component set that
|
|
107
|
+
wraps a prompt, only default-profile promote and rollback also move the wrapped
|
|
108
|
+
prompt's production label in the same transaction and print `default profile →
|
|
109
|
+
prompt label`. This coupling is one-way: moving the prompt label directly does
|
|
110
|
+
not repoint the profile.
|
|
111
|
+
|
|
112
|
+
`default show` reports the default and the model configs that currently resolve
|
|
113
|
+
to it. `default move` is human-only and first reads that impact before moving
|
|
114
|
+
the pointer to a sealed, commit-anchored profile version. `shape add` and
|
|
115
|
+
`shape remove` create a new complete version for every profile; they never move
|
|
116
|
+
the default or a production label. The set does not resolve for affected model
|
|
117
|
+
configs until those pointers move to the new shape-change profile versions; the
|
|
118
|
+
text CLI prints the required follow-up commands.
|
|
119
|
+
|
|
120
|
+
The manifest is a JSON object with `name`, ordered `shape`, and `components`.
|
|
121
|
+
Every component supplies `{ key, text }` or `{ key, path }`; paths are relative
|
|
122
|
+
to the manifest. Set-wide components must cover the complete shape. A component
|
|
123
|
+
may add `modelConfig` for a profile override: `create` materializes that named
|
|
124
|
+
profile with the set-wide component map plus its overrides, while `push` updates named
|
|
125
|
+
profiles already present on the set. `push` refuses a manifest whose shape
|
|
126
|
+
differs from the stored set.
|
|
127
|
+
|
|
128
|
+
`sync` writes `<out>/<set>/manifest.json`, `default/<key>.md`, and
|
|
129
|
+
`profiles/<identity-slug>/<key>.md` for offline execution. TypeScript runners
|
|
130
|
+
use `loadInstructionSet(dir, name, modelConfigIdentity)` from `orizu`; Python
|
|
131
|
+
runners use `orizu_gepa_connector.instruction_set_loader.load_instruction_set`.
|
|
132
|
+
Both select that model config's production component map or the default without a
|
|
133
|
+
network call. Git-pinned components have no local bytes: loading one raises
|
|
134
|
+
`instruction_set_component_unavailable` until the runner supplies those bytes.
|
|
135
|
+
An existing manifest file that cannot be read is instead
|
|
136
|
+
`instruction_set_component_unreadable`, which means re-sync or repair disk
|
|
137
|
+
state rather than supplying a Git-pinned component.
|
|
138
|
+
|
|
80
139
|
### Apps
|
|
81
140
|
|
|
82
141
|
```bash
|
|
@@ -156,6 +215,18 @@ orizu datasets clone --dataset <datasetId|datasetUrl> --name "Batch 1 Copy"
|
|
|
156
215
|
|
|
157
216
|
Dataset version and split-set commands used by prompt runs are covered in `prompt-control-plane.md`.
|
|
158
217
|
|
|
218
|
+
### Model configs
|
|
219
|
+
|
|
220
|
+
```bash
|
|
221
|
+
orizu model-configs create azure/gpt-5.4-mini --settings @settings.json --project my-team/quality-eval
|
|
222
|
+
orizu model-configs list --project my-team/quality-eval --json
|
|
223
|
+
orizu model-configs show azure/gpt-5.4-mini --project my-team/quality-eval
|
|
224
|
+
orizu model-configs settings set azure/gpt-5.4-mini --settings '{"temperature":0.3}' --project my-team/quality-eval
|
|
225
|
+
orizu model-configs copy openai/gpt-5.4-mini --to azure/gpt-5.4-mini --project my-team/quality-eval
|
|
226
|
+
```
|
|
227
|
+
|
|
228
|
+
Model identities are lowercase `provider/model` values and unique within a project. Creating a config creates settings version 1 (with `{}` when `--settings` is omitted). `settings set` appends a settings version and advances the current pointer; `copy` snapshots the source’s current settings and display name into version 1 for the new identity. `--settings` accepts a JSON object or `@file`; all commands support `--json`.
|
|
229
|
+
|
|
159
230
|
Supported file types:
|
|
160
231
|
- `.csv`
|
|
161
232
|
- `.json` (array of objects)
|
|
@@ -310,7 +381,7 @@ exist.
|
|
|
310
381
|
Optimization trace commands:
|
|
311
382
|
|
|
312
383
|
```bash
|
|
313
|
-
orizu optimizations run-gepa ... [--scorer-input-contract gepa|flat_row] [--scorer-candidate-field <row-field>] [--allow-degenerate-seed] [--num-threads auto|N] [--reflection-retry-attempts 3] [--reflection-http-timeout-seconds 180] [--log-dir logs] [--no-local-log]
|
|
384
|
+
orizu optimizations run-gepa ... [--scorer-input-contract gepa|flat_row] [--scorer-candidate-field <row-field>] [--allow-degenerate-seed] [--candidate-selection-strategy pareto|current_best|epsilon_greedy] [--epsilon N] [--objective <text>] [--candidate-proposer skilled-proposer] [--proposal-max-calls N] [--proposal-max-tokens N] [--python <command>] [--num-threads auto|N] [--reflection-retry-attempts 3] [--reflection-http-timeout-seconds 180] [--log-dir logs] [--no-local-log]
|
|
314
385
|
orizu optimizations export <optimization-run-id> --out ./optimization.json
|
|
315
386
|
orizu optimizations export <optimization-run-id> --json
|
|
316
387
|
```
|
|
@@ -319,6 +390,7 @@ Behavior:
|
|
|
319
390
|
- `run-gepa` sends the scorer runner a GEPA-shaped row (`{source_row, candidate_output, …}`) by default. Judge runners written for flat-row score runs (`runners exec --scorer-version`) need `--scorer-input-contract flat_row` (plus `--scorer-candidate-field <row-field>` when the judge reads the candidate output from a named row field) or they silently score everything 0. `run-gepa` validates the contract on the seed at launch and refuses a uniformly-worst seed unless `--allow-degenerate-seed` is set. See `prompt-control-plane.md` ("Scorer-Runner Input Contracts").
|
|
320
391
|
- `run-gepa` defaults `--num-threads` to `auto`, resolving a row-evaluation parallelism cap from mini-batch size, validation-set size, 2x CPU count, memory estimate, file-descriptor limit, and a 64-thread default ceiling. Set `ORIZU_GEPA_AUTO_THREADS_MAX` or use `--num-threads <n>` only when the runner/provider capacity is known.
|
|
321
392
|
- `run-gepa` retries transient reflection-provider failures by default. If retries are exhausted, it logs `reflection_failed`, counts that reflection proposal against budget, and continues with the next iteration.
|
|
393
|
+
- The skilled proposer is official-engine-only and prepares its managed Python environment on the selected run. Flag validation, proposal budgets, and recovery: `prompt-control-plane.md` ("Skilled proposer").
|
|
322
394
|
- `run-gepa` writes a complete local trace under `logs/<optimization_run_id>` by default.
|
|
323
395
|
- The local trace is the best artifact for coding-agent analysis because it includes full rows, outputs, scores, feedback, scorer responses, reflection prompts, reflection responses, candidate text, and `result.json`.
|
|
324
396
|
- `optimizations export` writes a portable JSON artifact from server data when the local log is unavailable or the run happened elsewhere.
|
|
@@ -407,3 +479,15 @@ Hugging Face / external dataset auth:
|
|
|
407
479
|
Worker assignment reads are self-only:
|
|
408
480
|
- Regular members cannot see other assignees' queues or response payloads.
|
|
409
481
|
- Use `tasks status` and `tasks export` for operator-side reporting.
|
|
482
|
+
### Offline instruction-set sync
|
|
483
|
+
|
|
484
|
+
`orizu instruction-sets sync <set> --project <team/project> --out <dir>` writes
|
|
485
|
+
an atomic local manifest. A resolver failure for either the default or any
|
|
486
|
+
selected profile refuses the entire sync as `instruction_set_unresolvable`; the
|
|
487
|
+
route never silently downgrades a failed production component map to default bytes. A
|
|
488
|
+
filtered `--model-config` sync records `filteredTo`, and a loader must refuse an
|
|
489
|
+
identity excluded by that marker with `instruction_set_profile_not_synced`.
|
|
490
|
+
|
|
491
|
+
Use `import { loadInstructionSet } from 'orizu/instruction-set-loader'` for the
|
|
492
|
+
small typed TypeScript loader surface; Python uses
|
|
493
|
+
`from orizu_gepa_connector import load_instruction_set`.
|