@eigenpal/cli 0.23.18 → 0.23.19
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
|
@@ -132,7 +132,7 @@ eigenpal workflow step-type list --search <partial-match>
|
|
|
132
132
|
### Eval flakiness with `llm-judge`
|
|
133
133
|
|
|
134
134
|
LLM-judge scores have variance. Stabilize: select a current model
|
|
135
|
-
(`openai/gpt-6.
|
|
135
|
+
(`openai/gpt-6.2-sol`), keep its reasoning effort fixed, narrow the prompt to
|
|
136
136
|
one dimension per judge, run multiple experiments and average. Only set
|
|
137
137
|
temperature when the model supports it; reasoning models may reject it.
|
|
138
138
|
|
|
@@ -301,7 +301,7 @@ process — `--out` writes the file directly from the server's storage.
|
|
|
301
301
|
|
|
302
302
|
LLM-judge scores have variance. To reduce flake:
|
|
303
303
|
|
|
304
|
-
- Select a current model (`openai/gpt-6.
|
|
304
|
+
- Select a current model (`openai/gpt-6.2-sol`) and keep its reasoning effort fixed.
|
|
305
305
|
Re-run representative evaluations when changing models or when the provider updates it.
|
|
306
306
|
- Keep `promptExtension` narrow — score one dimension per evaluator. Use
|
|
307
307
|
multiple judges instead of one omnibus prompt.
|