foldrun 20.9.4 → 20.9.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/docs/providers.md +21 -5
- package/docs/secrets.md +1 -1
- package/package.json +1 -1
package/docs/providers.md
CHANGED
|
@@ -226,11 +226,27 @@ The three failures were following the task, not the language:
|
|
|
226
226
|
- `default` added an introductory line and ran one list item past its
|
|
227
227
|
ten-word limit.
|
|
228
228
|
|
|
229
|
-
|
|
230
|
-
|
|
231
|
-
|
|
232
|
-
|
|
233
|
-
|
|
229
|
+
The whole comparison cost $0.42 (runs and judges together).
|
|
230
|
+
|
|
231
|
+
**The prompt fixed them, not the tier.** The same agent was then told, in
|
|
232
|
+
its prose, the four things it had got wrong: the reply is the deliverable
|
|
233
|
+
(no line about itself, no closing offer); never invent a date, name or
|
|
234
|
+
number; count before answering when the task sets a limit; check that
|
|
235
|
+
subject and verb agree. Run again with `repeat: 2`, `fast` passed 16 / 16
|
|
236
|
+
and `max` 16 / 16, at $0.010 and $0.020 a run ($0.49 for the comparison).
|
|
237
|
+
|
|
238
|
+
A flow around the first agent instead — a `verify: judge:` that the reply
|
|
239
|
+
holds no extra line and meets its limits, with `retry: 2` — did worse:
|
|
240
|
+
`fast` 14 / 16 and `default` 16 / 16, three times slower (about 34 s a
|
|
241
|
+
run). Both of `fast`'s misses were one technical translation that opened
|
|
242
|
+
with a stray English "The", which the verify claim did not ask about. A
|
|
243
|
+
judge checks what its claim names, so name the failure you have seen.
|
|
244
|
+
|
|
245
|
+
So for Persian: write the rules your eval catches into the agent's prose
|
|
246
|
+
first, and keep `fast` if it passes them. A strict shape (a fixed format,
|
|
247
|
+
an exact length) is checked best by a deterministic `verify:` on the
|
|
248
|
+
artefact. Two runs per case is still a small sample: run the comparison on
|
|
249
|
+
your own agent's cases with `repeat: 3` before moving a step to `fast`.
|
|
234
250
|
|
|
235
251
|
## Why a turn stopped
|
|
236
252
|
|
package/docs/secrets.md
CHANGED
|
@@ -221,7 +221,7 @@ PUT /api/secrets { "name": "…", "value": "…", "sites": ["api.dataforseo.
|
|
|
221
221
|
|
|
222
222
|
A **remote MCP server**'s `headers:` and `url:` work the same way: bind the
|
|
223
223
|
key to the server's host and the sandbox's copy of them holds `${NAME}`,
|
|
224
|
-
filled by the proxy on the way to that host alone ([MCP](tools#mcp
|
|
224
|
+
filled by the proxy on the way to that host alone ([MCP](tools#mcp--someone-elses-tools)).
|
|
225
225
|
|
|
226
226
|
What a program in the sandbox writes is unchanged; it sends the placeholder
|
|
227
227
|
where the value would go:
|