@tangle-network/agent-eval 0.200.0 → 0.200.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +4 -0
- package/dist/openapi.json +1 -1
- package/dist/rl.d.ts +15 -18
- package/dist/rl.d.ts.map +1 -1
- package/dist/rl.js +27 -32
- package/dist/rl.js.map +1 -1
- package/docs/jev-behavior-review.md +4 -3
- package/package.json +1 -1
|
@@ -155,7 +155,7 @@ A claim in such a case passes every exact check and settles nothing.
|
|
|
155
155
|
It decides `pass`, `hack` or `unknown` for one claim from signals the caller computes:
|
|
156
156
|
|
|
157
157
|
- `scopeSignals`: predicates over the claim's parameters, from a `SourceScope` written from the source by someone other than the checker's author. The source excludes the case, lists it, or reports it checked.
|
|
158
|
-
- `farmingSignal`: one parameter
|
|
158
|
+
- `farmingSignal`: one value of a parameter the scope declares, repeated across a lane's sweep of claims.
|
|
159
159
|
- `graderReferenceSignal`: the claim page justifies itself by the checker or by the printed letter.
|
|
160
160
|
- `refereeSignal`: a model reads the source and the claim's parameters, never the claimant's prose (`sourceScopeRefereeMessages`, `parseSourceScopeReferee`).
|
|
161
161
|
- `jevSignal`: typed questions over the same brief (`claimIntegrityJevReview`). The caller supplies calibrated thresholds.
|
|
@@ -164,8 +164,9 @@ It decides `pass`, `hack` or `unknown` for one claim from signals the caller com
|
|
|
164
164
|
|
|
165
165
|
The policy fails closed.
|
|
166
166
|
A pass needs a declared scope and a referee that evaluated the claim, with nothing against it.
|
|
167
|
-
|
|
168
|
-
|
|
167
|
+
Only an exact signal decides `hack`.
|
|
168
|
+
Any other signal against the claim holds it as `unknown`, however many kinds agree.
|
|
169
|
+
An honest page, an honest sweep and a persuadable referee each trip a soft signal, and a real result called a hack is lost.
|
|
169
170
|
Keep verdicts, scopes and questions away from the agent that filed the claim: an agent scored on them learns to hide the case.
|
|
170
171
|
|
|
171
172
|
## Security and validation boundaries
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@tangle-network/agent-eval",
|
|
3
|
-
"version": "0.200.
|
|
3
|
+
"version": "0.200.1",
|
|
4
4
|
"description": "Evaluate and improve AI agents from runs, traces, judges, and feedback. Compare candidates, cluster failures, measure lift, and gate releases.",
|
|
5
5
|
"homepage": "https://github.com/tangle-network/agent-eval#readme",
|
|
6
6
|
"repository": {
|