pi-jev-lens 0.5.1 → 0.5.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +3 -1
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -221,8 +221,10 @@ whether the agent needed the text. The benchmark uses real OpenHands trajectorie
|
|
|
221
221
|
- edit-miss: the agent later edited a line that the view had dropped.
|
|
222
222
|
- quote-miss: the agent quoted text that the view had dropped.
|
|
223
223
|
- ref-miss: the agent used an identifier that only existed in the dropped part.
|
|
224
|
+
- recall-miss: in a recorded session with the extension, the agent recalled the result, and the replayed view would
|
|
225
|
+
still not have answered that recall (the whole output, a line range or a pattern). Only real sessions have recalls.
|
|
224
226
|
|
|
225
|
-
An edit-miss counts five times
|
|
227
|
+
The objective is saved% − 5·edit-miss% − 2·quote-miss% − ref-miss% − 2·recall-miss%. An edit-miss counts five times, and edit-misses are rare. So you must score every change that
|
|
226
228
|
touches code views on the 500-trajectory slice, not only on the 100-trajectory holdout.
|
|
227
229
|
|
|
228
230
|
| slice | large results | saved | edit-miss | quote-miss | ref-miss |
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "pi-jev-lens",
|
|
3
|
-
"version": "0.5.
|
|
3
|
+
"version": "0.5.2",
|
|
4
4
|
"description": "pi extension that compresses large tool results before they reach the model: jev picks the view (outline, relevant blocks, sections, signals, testlog), full text stays recallable",
|
|
5
5
|
"author": "Didrik Rognstad",
|
|
6
6
|
"license": "MIT",
|