@evolvingmachines/evolve 0.0.65 → 0.0.67-train-harness-wave-2026-09-25.20260926.e00a8a6d
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/{chunk-DK2MGUPB.js → chunk-DI6LPW2B.js} +2 -2
- package/dist/cli/index.cjs +29 -29
- package/dist/cli/index.d.cts +3 -3
- package/dist/cli/index.d.ts +3 -3
- package/dist/cli/index.js +18 -18
- package/dist/index.cjs +64 -60
- package/dist/index.d.cts +297 -18
- package/dist/index.d.ts +297 -18
- package/dist/index.js +48 -44
- package/dist/{types-B_qmzlMk.d.cts → types-Dj0iMl5V.d.cts} +70 -90
- package/dist/{types-B_qmzlMk.d.ts → types-Dj0iMl5V.d.ts} +70 -90
- package/harness-capabilities.json +285 -0
- package/package.json +9 -6
- package/skills/create-adapter/SKILL.md +71 -26
- package/skills/create-task/SKILL.md +124 -66
- package/skills/evolve/SKILL.md +37 -20
- package/skills/evolve-evals/SKILL.md +142 -69
- package/skills/evolve-evals/references/cli-reference/agent.md +76 -0
- package/skills/evolve-evals/references/cli-reference/analysis.md +78 -0
- package/skills/evolve-evals/references/cli-reference/analyze.md +97 -0
- package/skills/evolve-evals/references/cli-reference/auth.md +96 -0
- package/skills/evolve-evals/references/cli-reference/check.md +157 -0
- package/skills/evolve-evals/references/cli-reference/dataset.md +154 -0
- package/skills/evolve-evals/references/cli-reference/filesystem.md +168 -0
- package/skills/evolve-evals/references/cli-reference/index.md +149 -0
- package/skills/evolve-evals/references/cli-reference/job.md +198 -0
- package/skills/evolve-evals/references/cli-reference/run.md +188 -0
- package/skills/evolve-evals/references/cli-reference/secrets.md +77 -0
- package/skills/evolve-evals/references/cli-reference/session.md +56 -0
- package/skills/evolve-evals/references/cli-reference/skill.md +52 -0
- package/skills/evolve-evals/references/cli-reference/skills.md +85 -0
- package/skills/evolve-evals/references/cli-reference/trial.md +115 -0
- package/skills/evolve-evals/references/cli-reference/upload.md +72 -0
- package/skills/evolve-evals/references/core-concepts/agents.md +101 -0
- package/skills/evolve-evals/references/core-concepts/analyze.md +143 -0
- package/skills/evolve-evals/references/core-concepts/check.md +116 -0
- package/skills/evolve-evals/references/core-concepts/compatibility.md +39 -0
- package/skills/evolve-evals/references/core-concepts/compose.md +103 -0
- package/skills/evolve-evals/references/core-concepts/datasets.md +234 -0
- package/skills/evolve-evals/references/core-concepts/index.md +52 -0
- package/skills/evolve-evals/references/core-concepts/jobs.md +223 -0
- package/skills/evolve-evals/references/core-concepts/models.md +73 -0
- package/skills/evolve-evals/references/core-concepts/multi-step.md +93 -0
- package/skills/evolve-evals/references/core-concepts/rewardkit.md +341 -0
- package/skills/evolve-evals/references/core-concepts/sandboxes.md +73 -0
- package/skills/evolve-evals/references/core-concepts/secrets.md +86 -0
- package/skills/evolve-evals/references/core-concepts/sharing.md +81 -0
- package/skills/evolve-evals/references/core-concepts/skills.md +100 -0
- package/skills/evolve-evals/references/core-concepts/task-config.md +169 -0
- package/skills/evolve-evals/references/core-concepts/task-environment.md +127 -0
- package/skills/evolve-evals/references/core-concepts/task-reference.md +191 -0
- package/skills/evolve-evals/references/core-concepts/task-verifiers.md +209 -0
- package/skills/evolve-evals/references/core-concepts/tasks.md +90 -0
- package/skills/evolve-evals/references/core-concepts/teams.md +128 -0
- package/skills/evolve-evals/references/core-concepts/trial-outputs.md +225 -0
- package/skills/evolve-evals/references/core-concepts/trials.md +129 -0
- package/skills/evolve-evals/references/core-concepts/upload-sdk-session.md +240 -0
- package/skills/evolve-evals/references/core-concepts/upload.md +166 -0
- package/skills/evolve-evals/references/dashboard/checks.md +54 -0
- package/skills/evolve-evals/references/dashboard/datasets.md +65 -0
- package/skills/evolve-evals/references/dashboard/jobs.md +106 -0
- package/skills/evolve-evals/references/dashboard/traces.md +54 -0
- package/skills/evolve-evals/references/dashboard/trial-viewer.md +77 -0
- package/skills/evolve-evals/references/getting-started/first-task.md +228 -0
- package/skills/evolve-evals/references/getting-started/installation.md +79 -0
- package/skills/evolve-evals/references/getting-started/quick-start.md +100 -0
- package/skills/evolve-evals/references/index.md +58 -0
- package/skills/evolve-evals/references/sdk/http.md +110 -0
- package/skills/evolve-evals/references/sdk/python.md +108 -0
- package/skills/evolve-evals/references/sdk/typescript.md +104 -0
- package/skills/evolve-evals/references/sdk-reference/agents.md +75 -0
- package/skills/evolve-evals/references/sdk-reference/analyses.md +125 -0
- package/skills/evolve-evals/references/sdk-reference/auth.md +171 -0
- package/skills/evolve-evals/references/sdk-reference/checks.md +146 -0
- package/skills/evolve-evals/references/sdk-reference/datasets.md +170 -0
- package/skills/evolve-evals/references/sdk-reference/error-codes.md +231 -0
- package/skills/evolve-evals/references/sdk-reference/errors.md +98 -0
- package/skills/evolve-evals/references/sdk-reference/filesystem.md +166 -0
- package/skills/evolve-evals/references/sdk-reference/index.md +161 -0
- package/skills/evolve-evals/references/sdk-reference/jobs.md +250 -0
- package/skills/evolve-evals/references/sdk-reference/meta.md +54 -0
- package/skills/evolve-evals/references/sdk-reference/methods/agents.md +154 -0
- package/skills/evolve-evals/references/sdk-reference/methods/analyses.md +211 -0
- package/skills/evolve-evals/references/sdk-reference/methods/auth.md +257 -0
- package/skills/evolve-evals/references/sdk-reference/methods/checks.md +462 -0
- package/skills/evolve-evals/references/sdk-reference/methods/datasets.md +687 -0
- package/skills/evolve-evals/references/sdk-reference/methods/filesystem.md +527 -0
- package/skills/evolve-evals/references/sdk-reference/methods/jobs.md +1023 -0
- package/skills/evolve-evals/references/sdk-reference/methods/meta.md +213 -0
- package/skills/evolve-evals/references/sdk-reference/methods/secrets.md +157 -0
- package/skills/evolve-evals/references/sdk-reference/methods/skills.md +125 -0
- package/skills/evolve-evals/references/sdk-reference/methods/trials.md +371 -0
- package/skills/evolve-evals/references/sdk-reference/secrets.md +104 -0
- package/skills/evolve-evals/references/sdk-reference/skills.md +66 -0
- package/skills/evolve-evals/references/sdk-reference/trials.md +164 -0
- package/skills/evolve-evals/references/sdk-reference/types.md +1255 -0
- package/skills/publish/SKILL.md +103 -79
- package/skills/rewardkit/SKILL.md +78 -23
- package/spec/openapi.yaml +164 -160
- package/skills/evolve-evals/references/cli-reference/agent.mdx +0 -79
- package/skills/evolve-evals/references/cli-reference/analysis.mdx +0 -160
- package/skills/evolve-evals/references/cli-reference/analyze.mdx +0 -84
- package/skills/evolve-evals/references/cli-reference/auth.mdx +0 -133
- package/skills/evolve-evals/references/cli-reference/check.mdx +0 -263
- package/skills/evolve-evals/references/cli-reference/dataset.mdx +0 -151
- package/skills/evolve-evals/references/cli-reference/index.mdx +0 -108
- package/skills/evolve-evals/references/cli-reference/job.mdx +0 -278
- package/skills/evolve-evals/references/cli-reference/run.mdx +0 -204
- package/skills/evolve-evals/references/cli-reference/secrets.mdx +0 -95
- package/skills/evolve-evals/references/cli-reference/skill.mdx +0 -61
- package/skills/evolve-evals/references/cli-reference/skills.mdx +0 -81
- package/skills/evolve-evals/references/cli-reference/trial.mdx +0 -181
- package/skills/evolve-evals/references/cli-reference/upload.mdx +0 -36
- package/skills/evolve-evals/references/core-concepts/agents.mdx +0 -106
- package/skills/evolve-evals/references/core-concepts/analyze.mdx +0 -86
- package/skills/evolve-evals/references/core-concepts/check.mdx +0 -64
- package/skills/evolve-evals/references/core-concepts/datasets.mdx +0 -138
- package/skills/evolve-evals/references/core-concepts/index.mdx +0 -42
- package/skills/evolve-evals/references/core-concepts/jobs.mdx +0 -151
- package/skills/evolve-evals/references/core-concepts/models.mdx +0 -68
- package/skills/evolve-evals/references/core-concepts/sandboxes.mdx +0 -52
- package/skills/evolve-evals/references/core-concepts/secrets.mdx +0 -41
- package/skills/evolve-evals/references/core-concepts/sharing.mdx +0 -35
- package/skills/evolve-evals/references/core-concepts/skills.mdx +0 -42
- package/skills/evolve-evals/references/core-concepts/tasks.mdx +0 -216
- package/skills/evolve-evals/references/core-concepts/teams.mdx +0 -36
- package/skills/evolve-evals/references/core-concepts/trial-outputs.mdx +0 -117
- package/skills/evolve-evals/references/core-concepts/trials.mdx +0 -103
- package/skills/evolve-evals/references/core-concepts/upload.mdx +0 -93
- package/skills/evolve-evals/references/dashboard/checks.mdx +0 -21
- package/skills/evolve-evals/references/dashboard/datasets.mdx +0 -24
- package/skills/evolve-evals/references/dashboard/jobs.mdx +0 -65
- package/skills/evolve-evals/references/dashboard/traces.mdx +0 -17
- package/skills/evolve-evals/references/dashboard/trial-viewer.mdx +0 -30
- package/skills/evolve-evals/references/getting-started/installation.mdx +0 -56
- package/skills/evolve-evals/references/getting-started/quick-start.mdx +0 -94
- package/skills/evolve-evals/references/index.mdx +0 -48
- package/skills/evolve-evals/references/sdk/python.mdx +0 -161
- package/skills/evolve-evals/references/sdk/typescript.mdx +0 -163
- package/skills/evolve-evals/references/sdk-reference/agents.mdx +0 -108
- package/skills/evolve-evals/references/sdk-reference/analyses.mdx +0 -222
- package/skills/evolve-evals/references/sdk-reference/auth.mdx +0 -116
- package/skills/evolve-evals/references/sdk-reference/checks.mdx +0 -300
- package/skills/evolve-evals/references/sdk-reference/datasets.mdx +0 -346
- package/skills/evolve-evals/references/sdk-reference/errors.mdx +0 -210
- package/skills/evolve-evals/references/sdk-reference/index.mdx +0 -60
- package/skills/evolve-evals/references/sdk-reference/jobs.mdx +0 -822
- package/skills/evolve-evals/references/sdk-reference/meta.mdx +0 -52
- package/skills/evolve-evals/references/sdk-reference/skills.mdx +0 -69
- package/skills/evolve-evals/references/sdk-reference/trials.mdx +0 -302
- package/skills/evolve-evals/references/sdk-reference/types.mdx +0 -274
- package/skills/evolve-evals/references/snippets/global-options.mdx +0 -13
- package/skills/evolve-evals/references/snippets/list-flags.mdx +0 -23
|
@@ -0,0 +1,166 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: "Filesystems and downloads"
|
|
3
|
+
description: "Choose the right files: saved results, a sandbox, or a task package."
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
**Note:**
|
|
7
|
+
|
|
8
|
+
For every method’s inputs, return values, and language differences, see the [filesystem method reference](/sdk-reference/methods/filesystem).
|
|
9
|
+
|
|
10
|
+
Evolve exposes three file surfaces. They answer different questions.
|
|
11
|
+
|
|
12
|
+
| Need | Client |
|
|
13
|
+
| --- | --- |
|
|
14
|
+
| Read a trial's saved `result.json`, logs, or collected artifact | `trials.files()` / `trials.file()` |
|
|
15
|
+
| Browse sandbox paths during or after a run | `filesystem()` |
|
|
16
|
+
| Inspect the task files supplied to a run | `datasets.taskFiles()` / `task_files()` |
|
|
17
|
+
|
|
18
|
+
## Open a run's filesystem
|
|
19
|
+
|
|
20
|
+
```ts TypeScript
|
|
21
|
+
import { trials } from "@evolvingmachines/evolve";
|
|
22
|
+
|
|
23
|
+
const fs = trials().filesystem(trialId);
|
|
24
|
+
const status = await fs.status();
|
|
25
|
+
const folder = await fs.list({ path: status.work_dir });
|
|
26
|
+
console.log(status.state, folder.entries);
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
```python Python
|
|
30
|
+
from evolve import trials
|
|
31
|
+
|
|
32
|
+
fs = trials().filesystem(trial_id)
|
|
33
|
+
status = await fs.status()
|
|
34
|
+
folder = await fs.list(path=status.work_dir)
|
|
35
|
+
print(status.state, folder.entries)
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
The same methods are available from:
|
|
39
|
+
|
|
40
|
+
### TypeScript
|
|
41
|
+
|
|
42
|
+
| Run | Client |
|
|
43
|
+
| --- | --- |
|
|
44
|
+
| Trial | `trials().filesystem(trialId)` |
|
|
45
|
+
| Analysis | `analyses().filesystem(analysisId)` |
|
|
46
|
+
| Task check | `checks().taskFilesystem(checkId, taskCheckId)` |
|
|
47
|
+
|
|
48
|
+
### Python
|
|
49
|
+
|
|
50
|
+
| Run | Client |
|
|
51
|
+
| --- | --- |
|
|
52
|
+
| Trial | `trials().filesystem(trial_id)` |
|
|
53
|
+
| Analysis | `analyses().filesystem(analysis_id)` |
|
|
54
|
+
| Task check | `checks().task_filesystem(check_id, task_check_id)` |
|
|
55
|
+
|
|
56
|
+
### What source will answer?
|
|
57
|
+
|
|
58
|
+
```text
|
|
59
|
+
1. Read status
|
|
60
|
+
├─ live → sandbox
|
|
61
|
+
├─ captured → saved files
|
|
62
|
+
├─ capturing → preparing files
|
|
63
|
+
└─ none → unavailable
|
|
64
|
+
|
|
65
|
+
2. Require a source (optional)
|
|
66
|
+
├─ "live" → sandbox only
|
|
67
|
+
└─ "capture" → saved files only
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
Omit `source` to prefer live files, then a capture. A missing required source raises `filesystem_state`.
|
|
71
|
+
|
|
72
|
+
**Note:**
|
|
73
|
+
|
|
74
|
+
A captured listing can include unchanged image files whose bytes were not saved. Check `entry.captured`; reading one with no stored bytes raises `not_captured`. A capture is not a complete disk image.
|
|
75
|
+
|
|
76
|
+
## Browse, read, search
|
|
77
|
+
|
|
78
|
+
```ts TypeScript
|
|
79
|
+
const bytes = await fs.read("/app/report.json");
|
|
80
|
+
const hits = await fs.search({ q: "error", path: "/app" });
|
|
81
|
+
const changes = await fs.changes({ phase: "agent" });
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
```python Python
|
|
85
|
+
content = await fs.read("/app/report.json")
|
|
86
|
+
hits = await fs.search("error", path="/app")
|
|
87
|
+
changes = await fs.changes(phase="agent")
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
These paths are examples. Use `status.work_dir` and a listing to find the run's actual files.
|
|
91
|
+
|
|
92
|
+
| Method | Options | Result |
|
|
93
|
+
| --- | --- | --- |
|
|
94
|
+
| `status()` | None | State, box, work directory, watcher, capture summary |
|
|
95
|
+
| `list(...)` | `path`, `source`, `cursor`, `limit` | `entries`, `next_cursor` |
|
|
96
|
+
| `read(path, ...)` | `source`, byte range | `Buffer` / `bytes` |
|
|
97
|
+
| `search(...)` | Query, `path`, `regex`, `limit`, `source` | Hits and `truncated` flag |
|
|
98
|
+
| `changes(...)` | `source`, `phase`, `cursor`, `limit` | Changed files and next cursor |
|
|
99
|
+
| `archive(...)` | `path`, `source`, download options | Archive bytes or saved path |
|
|
100
|
+
|
|
101
|
+
TypeScript search takes `{q, ...}`; Python takes `q` as its first argument. Change phases are `setup`, `agent`, `verifier`, or `all`.
|
|
102
|
+
|
|
103
|
+
Byte ranges: TypeScript `{range: {start, end}}` or `{range: {suffix}}`; Python `start=`, `end=`, or `suffix=`. End is inclusive. Suffix selects the last N bytes.
|
|
104
|
+
|
|
105
|
+
List/change pages default to 500 rows; search defaults to 200. Their maximum is 1,000. A `truncated` search should be narrowed before treating it as complete.
|
|
106
|
+
|
|
107
|
+
## Follow changes and logs
|
|
108
|
+
|
|
109
|
+
```ts TypeScript
|
|
110
|
+
await fs.watch(["/app"]);
|
|
111
|
+
for await (const event of fs.events()) {
|
|
112
|
+
console.log(event.event, event.data);
|
|
113
|
+
}
|
|
114
|
+
```
|
|
115
|
+
|
|
116
|
+
```python Python
|
|
117
|
+
await fs.watch(["/app"])
|
|
118
|
+
async for event in fs.events():
|
|
119
|
+
print(event.event, event.data)
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
`watch(paths)` registers up to eight folders. `events()` yields filesystem, state, and heartbeat frames.
|
|
123
|
+
|
|
124
|
+
| Read | TypeScript | Python |
|
|
125
|
+
| --- | --- | --- |
|
|
126
|
+
| One log page | `logs({ stream: "agent", cursor, limit })` | `logs("agent", cursor=..., limit=...)` |
|
|
127
|
+
| Log events | `logEvents(options)` | `log_events(...)` |
|
|
128
|
+
| Live processes | `procs()` | `procs()` |
|
|
129
|
+
|
|
130
|
+
Log streams are `agent`, `verifier`, `system`, `setup`, and `metrics`. System logs require `system_log` at job creation. A stream can be empty with a `reason`; reserved or unavailable streams do not imply missing agent work.
|
|
131
|
+
|
|
132
|
+
Filesystem and log event streams connect once. To reconnect, retain the last event id and pass `lastEventId` / `last_event_id`. TypeScript also accepts `signal`. This is separate from the automatic reconnect behavior of `jobs.watch()`.
|
|
133
|
+
|
|
134
|
+
## Task package files
|
|
135
|
+
|
|
136
|
+
```ts TypeScript
|
|
137
|
+
import { datasets } from "@evolvingmachines/evolve";
|
|
138
|
+
|
|
139
|
+
const files = datasets().taskFiles("harbor-examples@1.0", "hello-world");
|
|
140
|
+
const tree = await files.list();
|
|
141
|
+
const instruction = await files.read("instruction.md");
|
|
142
|
+
```
|
|
143
|
+
|
|
144
|
+
```python Python
|
|
145
|
+
from evolve import datasets
|
|
146
|
+
|
|
147
|
+
files = datasets().task_files("harbor-examples@1.0", "hello-world")
|
|
148
|
+
tree = await files.list()
|
|
149
|
+
instruction = await files.read("instruction.md")
|
|
150
|
+
```
|
|
151
|
+
|
|
152
|
+
The dataset reference must pin a version. This client provides `status()`, `list()`, and `read(path)`. List options are `path`, `cursor`, and `limit`: one object in TypeScript, keyword arguments in Python. Reads accept the same byte-range options as above. There is no live source, process list, or log stream.
|
|
153
|
+
|
|
154
|
+
## Download options
|
|
155
|
+
|
|
156
|
+
Dataset, job, analysis, and check downloads use the same delivery choices. Run-filesystem `archive()` does too.
|
|
157
|
+
|
|
158
|
+
| Choice | TypeScript | Python |
|
|
159
|
+
| --- | --- | --- |
|
|
160
|
+
| Bytes in memory | No options → `Buffer` | No `to` → `bytes` |
|
|
161
|
+
| Save to a directory | `{ to: "./results" }` → path | `to="./results"` → path |
|
|
162
|
+
| Raw response stream | `{ stream: true }` | Not exposed |
|
|
163
|
+
|
|
164
|
+
`to` is a directory. The client chooses the archive filename from the response and returns the saved path.
|
|
165
|
+
|
|
166
|
+
Buffer/bytes and saved-file downloads check declared length and digest when supplied. Saved files appear at their final path only after verification. TypeScript raw streams leave integrity checking to the caller.
|
|
@@ -0,0 +1,161 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: "SDK clients"
|
|
3
|
+
description: "One configuration. Focused clients for jobs, datasets, trials, and results."
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
Use the same managed evaluation service from TypeScript or Python. Evolve runs the jobs; your program submits work and reads the results.
|
|
7
|
+
|
|
8
|
+
**Note:**
|
|
9
|
+
|
|
10
|
+
To run agents directly with the Evolve builder, Swarm, or Pipeline, use the separate [agent SDK manual](https://github.com/evolving-machines-lab/evolve/blob/main/docs-agents/index.md).
|
|
11
|
+
|
|
12
|
+
```ts TypeScript
|
|
13
|
+
import { hosted } from "@evolvingmachines/evolve";
|
|
14
|
+
|
|
15
|
+
const evolve = hosted(); // Reads EVOLVE_API_KEY.
|
|
16
|
+
const page = await evolve.jobs.list();
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
```python Python
|
|
20
|
+
from evolve import hosted
|
|
21
|
+
|
|
22
|
+
evolve = hosted() # Reads EVOLVE_API_KEY.
|
|
23
|
+
page = await evolve.jobs.list()
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
Python examples in this reference belong inside an async function. The [Python guide](/sdk/python) includes a complete runnable program.
|
|
27
|
+
|
|
28
|
+
## Choose a client
|
|
29
|
+
|
|
30
|
+
**[Jobs](/sdk-reference/jobs)**
|
|
31
|
+
|
|
32
|
+
Start, follow, compare, retry, and download evaluations.
|
|
33
|
+
|
|
34
|
+
**[Trials](/sdk-reference/trials)**
|
|
35
|
+
|
|
36
|
+
Inspect one attempt: reward, trace, and stored files.
|
|
37
|
+
|
|
38
|
+
**[Datasets](/sdk-reference/datasets)**
|
|
39
|
+
|
|
40
|
+
Browse tasks, publish versions, and inspect builds.
|
|
41
|
+
|
|
42
|
+
**[Filesystems](/sdk-reference/filesystem)**
|
|
43
|
+
|
|
44
|
+
Read live sandbox files and saved captures.
|
|
45
|
+
|
|
46
|
+
**[Analyses](/sdk-reference/analyses)**
|
|
47
|
+
|
|
48
|
+
Read the analyzer's results and saved runs.
|
|
49
|
+
|
|
50
|
+
**[Checks](/sdk-reference/checks)**
|
|
51
|
+
|
|
52
|
+
Check task quality before running a benchmark.
|
|
53
|
+
|
|
54
|
+
Also available: [agents](/sdk-reference/agents), [uploaded skills](/sdk-reference/skills), [secrets](/sdk-reference/secrets), [identity and teams](/sdk-reference/auth), and [platform capabilities](/sdk-reference/meta).
|
|
55
|
+
|
|
56
|
+
## Look up a method
|
|
57
|
+
|
|
58
|
+
Use the client guides above for a workflow. Open a method reference for its inputs, returned fields, and Python and TypeScript examples.
|
|
59
|
+
|
|
60
|
+
| Client | Method reference |
|
|
61
|
+
| --- | --- |
|
|
62
|
+
| Jobs | [Start, watch, compare, retry, and import](/sdk-reference/methods/jobs) |
|
|
63
|
+
| Trials | [Read attempts, traces, and artifacts](/sdk-reference/methods/trials) |
|
|
64
|
+
| Filesystems | [Read files, search, and follow live changes](/sdk-reference/methods/filesystem) |
|
|
65
|
+
| Analyses | [Read analysis runs and outputs](/sdk-reference/methods/analyses) |
|
|
66
|
+
| Checks | [Create and inspect task quality checks](/sdk-reference/methods/checks) |
|
|
67
|
+
| Datasets | [Browse, publish, and manage versions](/sdk-reference/methods/datasets) |
|
|
68
|
+
| Agents | [Register and manage harnesses](/sdk-reference/methods/agents) |
|
|
69
|
+
| Skills | [Upload and manage run context](/sdk-reference/methods/skills) |
|
|
70
|
+
| Secrets | [Store credentials and read metadata](/sdk-reference/methods/secrets) |
|
|
71
|
+
| Identity and teams | [Read identity and manage membership](/sdk-reference/methods/auth) |
|
|
72
|
+
| Capabilities | [Read supported options and limits](/sdk-reference/methods/meta) |
|
|
73
|
+
|
|
74
|
+
For failures, start with [error handling](/sdk-reference/errors) or find the exact name in the [error-code reference](/sdk-reference/error-codes).
|
|
75
|
+
|
|
76
|
+
## Configure once
|
|
77
|
+
|
|
78
|
+
```ts TypeScript
|
|
79
|
+
import { hosted } from "@evolvingmachines/evolve";
|
|
80
|
+
|
|
81
|
+
const evolve = hosted({
|
|
82
|
+
apiKey: process.env.EVOLVE_API_KEY,
|
|
83
|
+
org: "my-team",
|
|
84
|
+
});
|
|
85
|
+
```
|
|
86
|
+
|
|
87
|
+
```python Python
|
|
88
|
+
import os
|
|
89
|
+
from evolve import HostedClientConfig, hosted
|
|
90
|
+
|
|
91
|
+
evolve = hosted(HostedClientConfig(
|
|
92
|
+
api_key=os.environ["EVOLVE_API_KEY"],
|
|
93
|
+
org="my-team",
|
|
94
|
+
))
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
| Setting | TypeScript | Python | Default |
|
|
98
|
+
| --- | --- | --- | --- |
|
|
99
|
+
| API key | `apiKey` | `api_key` | `EVOLVE_API_KEY` |
|
|
100
|
+
| API origin | `baseUrl` | `base_url` | `EVOLVE_DASHBOARD_URL`, then the Evolve dashboard |
|
|
101
|
+
| Owning team | `org` | `org` | Your personal organization |
|
|
102
|
+
|
|
103
|
+
The `org` default applies when you create jobs, publish datasets, register agents, or upload skills. A call's own `org` wins. It does not filter list requests.
|
|
104
|
+
|
|
105
|
+
Prefer a single client when useful: `jobs(config)`, `trials(config)`, and the other factories accept the same configuration. Import `auth` and `managedSecrets` / `managed_secrets` separately; they are not properties of `hosted()`.
|
|
106
|
+
|
|
107
|
+
**Tip:**
|
|
108
|
+
|
|
109
|
+
`meta()` needs no key. Other clients authenticate with your API key. TypeScript checks for a key when the client is created; Python checks when it makes an authenticated request.
|
|
110
|
+
|
|
111
|
+
## One page or every item
|
|
112
|
+
|
|
113
|
+
```ts TypeScript
|
|
114
|
+
// One page.
|
|
115
|
+
const page = await evolve.jobs.list({ limit: 20 });
|
|
116
|
+
console.log(page.items, page.nextCursor, page.hasMore);
|
|
117
|
+
|
|
118
|
+
// All pages, one job at a time.
|
|
119
|
+
for await (const job of evolve.jobs.list()) {
|
|
120
|
+
console.log(job.id, job.status);
|
|
121
|
+
}
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
```python Python
|
|
125
|
+
# One page.
|
|
126
|
+
page = await evolve.jobs.list(limit=20)
|
|
127
|
+
print(page.items, page.next_cursor, page.has_more)
|
|
128
|
+
|
|
129
|
+
# All pages, one job at a time.
|
|
130
|
+
async for job in evolve.jobs.list():
|
|
131
|
+
print(job.id, job.status)
|
|
132
|
+
```
|
|
133
|
+
|
|
134
|
+
This pattern applies to list handles, including `jobs.trials()` and `jobs.tasks()`. A trace page, stored-file page, grep result, or dataset's nested task page is a single page; follow its cursor yourself or use the dedicated iterator where available.
|
|
135
|
+
|
|
136
|
+
## Language differences
|
|
137
|
+
|
|
138
|
+
| Detail | TypeScript | Python |
|
|
139
|
+
| --- | --- | --- |
|
|
140
|
+
| Calls | Input/options objects | Keyword arguments |
|
|
141
|
+
| Most domain fields | `model_name`, `cost_usd` | Same spelling |
|
|
142
|
+
| Returned entities | Objects | Dataclasses; stats, analyses, and checks are dictionaries |
|
|
143
|
+
| Download to memory | `Buffer` | `bytes` |
|
|
144
|
+
| Download to directory | `{ to: "./results" }` | `to="./results"` |
|
|
145
|
+
| Raw download stream | `{ stream: true }` | Not exposed |
|
|
146
|
+
| Wait cancellation | `AbortSignal` on watch methods | `timeout_s` on job/import/check watches |
|
|
147
|
+
|
|
148
|
+
See [types](/sdk-reference/types) for field access and [errors](/sdk-reference/errors) for request failures.
|
|
149
|
+
|
|
150
|
+
### Python client lifetime
|
|
151
|
+
|
|
152
|
+
Hosted clients support `async with` and `await client.close()`:
|
|
153
|
+
|
|
154
|
+
```python
|
|
155
|
+
from evolve import hosted
|
|
156
|
+
|
|
157
|
+
async with hosted() as client:
|
|
158
|
+
page = await client.jobs.list(limit=20)
|
|
159
|
+
```
|
|
160
|
+
|
|
161
|
+
The individual hosted clients currently have no resources to release in `close()`. Closing a client does not cancel an evaluation. Use `jobs().cancel(job_id)` to stop a job.
|
|
@@ -0,0 +1,250 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: "Jobs"
|
|
3
|
+
description: "Submit evaluations, follow progress, and work with results."
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
**Note:**
|
|
7
|
+
|
|
8
|
+
For every method’s inputs, return values, and language differences, see the [jobs method reference](/sdk-reference/methods/jobs).
|
|
9
|
+
|
|
10
|
+
A job runs selected tasks against one or more agent/model combinations. Each combination is an **arm**. Each attempt is a **trial**.
|
|
11
|
+
|
|
12
|
+
```text
|
|
13
|
+
Selected tasks
|
|
14
|
+
× Agent/model arms
|
|
15
|
+
× Attempts
|
|
16
|
+
= Trials
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
## Start a job
|
|
20
|
+
|
|
21
|
+
```ts TypeScript
|
|
22
|
+
import { jobs } from "@evolvingmachines/evolve";
|
|
23
|
+
|
|
24
|
+
const client = jobs();
|
|
25
|
+
const job = await client.start({
|
|
26
|
+
datasets: [{
|
|
27
|
+
name: "harbor-examples",
|
|
28
|
+
version: "1.0",
|
|
29
|
+
task_names: ["hello-world"],
|
|
30
|
+
}],
|
|
31
|
+
agents: [{ name: "codex", model_name: "gpt-5.6-luna" }],
|
|
32
|
+
max_trial_spend_usd: 0.50,
|
|
33
|
+
retry: { max_retries: 0 },
|
|
34
|
+
});
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
```python Python
|
|
38
|
+
from evolve import jobs
|
|
39
|
+
|
|
40
|
+
client = jobs()
|
|
41
|
+
job = await client.start(
|
|
42
|
+
datasets=[{
|
|
43
|
+
"name": "harbor-examples",
|
|
44
|
+
"version": "1.0",
|
|
45
|
+
"task_names": ["hello-world"],
|
|
46
|
+
}],
|
|
47
|
+
agents=[{"name": "codex", "model_name": "gpt-5.6-luna"}],
|
|
48
|
+
max_trial_spend_usd=0.50,
|
|
49
|
+
retry={"max_retries": 0},
|
|
50
|
+
)
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
`start` returns the accepted `Job`. Read `job.id` to address it later.
|
|
54
|
+
|
|
55
|
+
### Required inputs
|
|
56
|
+
|
|
57
|
+
| Input | Fields |
|
|
58
|
+
| --- | --- |
|
|
59
|
+
| `datasets` | One or more selectors. Required: `name`. Optional: `version`, `task_names`, `exclude_task_names`, `n_tasks`. Omitted version selects the active version. Filters accept glob patterns; the cap applies after filters. |
|
|
60
|
+
| `agents` | One or more arms. Required: `name`, `model_name`. Optional: `version`, `reasoning_effort`, `kwargs`, `preset`, `skills`. Use [capabilities](/sdk-reference/meta) for supported values. |
|
|
61
|
+
|
|
62
|
+
Python also accepts `DatasetSelector` and `AgentArm` dataclass instances in these lists.
|
|
63
|
+
|
|
64
|
+
### Run settings
|
|
65
|
+
|
|
66
|
+
| Option | Meaning | When omitted |
|
|
67
|
+
| --- | --- | --- |
|
|
68
|
+
| `job_name` | A label for the run | Server-generated name |
|
|
69
|
+
| `org` | Owning organization slug or id | Client default, then personal organization |
|
|
70
|
+
| `n_attempts` | Attempts per task/arm; integer ≥1 | `1` |
|
|
71
|
+
| `n_concurrent_trials` | Requested parallel trials; `1`–`150` | `4` |
|
|
72
|
+
| `sandbox_provider` | Requested `e2b`, `daytona`, or `modal` | Current platform default |
|
|
73
|
+
| `max_trial_spend_usd` | Agent model-spend cap per trial attempt; must be >0 | Current platform default; fallback `$200` |
|
|
74
|
+
| `retry` | Automatic infrastructure retry policy | Current platform policy; fallback two retries |
|
|
75
|
+
| `analyze` | [Analyze each settled trial](/sdk-reference/analyses#start-an-analysis); `{}` enables defaults | Off |
|
|
76
|
+
| `system_log` | Capture sandbox system logs | Off |
|
|
77
|
+
| `secrets` | Stored secret references or inline values | No extra job secrets |
|
|
78
|
+
| `verifier_env` | Judge overrides: `REWARDKIT_JUDGE`, `REWARDKIT_MODEL` only | Task/default judge settings |
|
|
79
|
+
|
|
80
|
+
Organization and fleet capacity also constrain parallel work. A task may use another provider when the requested provider cannot support its requirements; read the trial for its actual provider.
|
|
81
|
+
|
|
82
|
+
### Timeout multipliers
|
|
83
|
+
|
|
84
|
+
All values are finite numbers greater than zero. They multiply task timeouts; they are not durations in seconds.
|
|
85
|
+
|
|
86
|
+
| Field | Applies to |
|
|
87
|
+
| --- | --- |
|
|
88
|
+
| `timeout_multiplier` | All phases without a specific override; default `1` |
|
|
89
|
+
| `agent_timeout_multiplier` | Agent execution |
|
|
90
|
+
| `verifier_timeout_multiplier` | Verification |
|
|
91
|
+
| `agent_setup_timeout_multiplier` | Harness installation/setup |
|
|
92
|
+
| `environment_build_timeout_multiplier` | Environment build |
|
|
93
|
+
|
|
94
|
+
A phase override replaces the global multiplier for that phase. Values that exceed the runtime's timer limit for the selected tasks are refused.
|
|
95
|
+
|
|
96
|
+
### Automatic retry policy
|
|
97
|
+
|
|
98
|
+
| Field | Rule |
|
|
99
|
+
| --- | --- |
|
|
100
|
+
| `max_retries` | Integer ≥0. `0` disables configured infrastructure retries. |
|
|
101
|
+
| `include_exceptions` | Exception names to include. Omitted, null, or `[]` means no include filter. |
|
|
102
|
+
| `exclude_exceptions` | Exclusions win over includes. Omitted keeps defaults; null or `[]` removes them. |
|
|
103
|
+
| `wait_multiplier` | Backoff multiplier; default `1` |
|
|
104
|
+
| `min_wait_sec` | Initial wait; default `1` second |
|
|
105
|
+
| `max_wait_sec` | Maximum wait; default `60` seconds |
|
|
106
|
+
|
|
107
|
+
Only eligible infrastructure failures enter this retry policy. Scoring errors and low rewards do not automatically rerun the agent. Each retry receives a fresh per-trial cap.
|
|
108
|
+
|
|
109
|
+
Provider capacity refusals have a separate bounded wait policy. They can wait even when `max_retries` is zero.
|
|
110
|
+
|
|
111
|
+
### Idempotency and unsupported input
|
|
112
|
+
|
|
113
|
+
Supply an idempotency key when repeating a submission must return the same accepted job:
|
|
114
|
+
|
|
115
|
+
Here, `config` is the job input shown above. `requestId` / `request_id` is a stable string you assign to this submission.
|
|
116
|
+
|
|
117
|
+
```ts TypeScript
|
|
118
|
+
const job = await client.start(config, { idempotencyKey: requestId });
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
```python Python
|
|
122
|
+
job = await client.start(**config, idempotency_key=request_id)
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
Use one stable request id for the same resolved request. Reusing it for a different request is refused.
|
|
126
|
+
|
|
127
|
+
`agent_env` appears in SDK signatures, but the current platform refuses nonempty values. Use [job secrets](/sdk-reference/secrets) for credentials or [registered-agent environment settings](/sdk-reference/agents) for a custom harness.
|
|
128
|
+
|
|
129
|
+
## Wait or show progress
|
|
130
|
+
|
|
131
|
+
```ts TypeScript
|
|
132
|
+
const finished = await client.watch(job.id, {
|
|
133
|
+
onEvent: (event) => console.log(event.type),
|
|
134
|
+
});
|
|
135
|
+
```
|
|
136
|
+
|
|
137
|
+
```python Python
|
|
138
|
+
finished = await client.watch(
|
|
139
|
+
job.id,
|
|
140
|
+
on_event=lambda event: print(event.type),
|
|
141
|
+
)
|
|
142
|
+
```
|
|
143
|
+
|
|
144
|
+
The same handle can instead be consumed with `for await` / `async for` to receive events. Choose one style per handle. Job watches replay history and reconnect from the last event id.
|
|
145
|
+
|
|
146
|
+
| Watch option | TypeScript | Python |
|
|
147
|
+
| --- | --- | --- |
|
|
148
|
+
| Event callback | `onEvent` | `on_event` |
|
|
149
|
+
| Initial reconnect delay | `reconnectDelayMs` (default `1000`) | `reconnect_delay_s` (default `1`) |
|
|
150
|
+
| Maximum reconnect delay | `maxReconnectDelayMs` (default `30000`) | `max_reconnect_delay_s` (default `30`) |
|
|
151
|
+
| Stop waiting | `signal` | `timeout_s` |
|
|
152
|
+
|
|
153
|
+
A terminal job has finished scheduling/executing its trials. **Completed does not mean every trial passed.**
|
|
154
|
+
|
|
155
|
+
## Read and compare
|
|
156
|
+
|
|
157
|
+
```ts TypeScript
|
|
158
|
+
const job = await client.get(jobId);
|
|
159
|
+
for await (const trial of client.trials(jobId, { status: ["SCORING_ERROR"] })) {
|
|
160
|
+
console.log(trial.id, trial.exception_info);
|
|
161
|
+
}
|
|
162
|
+
const tasks = await client.tasks(jobId);
|
|
163
|
+
const comparison = await client.compare([firstJobId, secondJobId]);
|
|
164
|
+
```
|
|
165
|
+
|
|
166
|
+
```python Python
|
|
167
|
+
job = await client.get(job_id)
|
|
168
|
+
async for trial in client.trials(job_id, status=["SCORING_ERROR"]):
|
|
169
|
+
print(trial.id, trial.exception_info)
|
|
170
|
+
tasks = await client.tasks(job_id)
|
|
171
|
+
comparison = await client.compare([first_job_id, second_job_id])
|
|
172
|
+
```
|
|
173
|
+
|
|
174
|
+
Use full ids returned by create/list calls.
|
|
175
|
+
|
|
176
|
+
| Method | Options | Returns |
|
|
177
|
+
| --- | --- | --- |
|
|
178
|
+
| `get(id)` | None | `Job` |
|
|
179
|
+
| `list(...)` | `search`, `scope`, `kind`, `limit`, `cursor` | Paginated handle |
|
|
180
|
+
| `trials(id, ...)` | `status` list, `dataset`, `limit`, `cursor` | Trial handle |
|
|
181
|
+
| `tasks(id, ...)` | `limit`, `cursor` | Per-task rollup handle |
|
|
182
|
+
| `compare(ids)` | 2–10 distinct readable job ids | Job aggregates and task matrix |
|
|
183
|
+
| `grep(id, q, ...)` | `type`, `limit`, `cursor` | One page of matches grouped by trial |
|
|
184
|
+
|
|
185
|
+
List scope is `my`, `shared`, or `org`. Job list kind is `job` (default), `check`, or `all`; check rows have a different shape. Branch on `row.kind` before reading job-only fields.
|
|
186
|
+
|
|
187
|
+
`grep` searches parsed traces using a case-insensitive POSIX regular expression. See [trial traces](/sdk-reference/trials#read-the-trace) for per-trial filtering.
|
|
188
|
+
|
|
189
|
+
## Cancel or run selected work again
|
|
190
|
+
|
|
191
|
+
| Action | TypeScript | Python | Result |
|
|
192
|
+
| --- | --- | --- | --- |
|
|
193
|
+
| Cancel remaining work | `cancel(id)` | `cancel(id)` | Current job |
|
|
194
|
+
| Resume failed/stopped work | `resume(id, { filter_error_types })` | `resume(id, filter_error_types=[...])` | New linked job |
|
|
195
|
+
| Retry selected settled trials | `retry(id, { trial_ids })` | `retry(id, trial_ids=[...])` | New linked job |
|
|
196
|
+
| Retry failed trials | `retry(id, { failed_only: true })` | `retry(id, failed_only=True)` | New linked job |
|
|
197
|
+
| Retry all trials of a terminal job | `retry(id)` | `retry(id)` | New linked job |
|
|
198
|
+
| Re-run eligible verifiers | `regrade(id, { statuses, task_name })` | `regrade(id, statuses=[...], task_name=...)` | New linked job |
|
|
199
|
+
|
|
200
|
+
Resume requires a terminal source job. Explicit retry ids may select settled trials from a job still running. Do not combine `trial_ids` with `failed_only`.
|
|
201
|
+
|
|
202
|
+
Resume and retry accept the same idempotency option as start: a final options object in TypeScript, `idempotency_key` in Python. Regrade has no such SDK option.
|
|
203
|
+
|
|
204
|
+
Regrading uses the recorded verifier snapshot; it cannot replace the verifier or use edited tests. It needs retained inputs from an eligible separate verifier. Shared-verifier, judge-based, multi-step, and uploaded runs cannot be regraded. See [jobs](/core-concepts/jobs) for eligibility.
|
|
205
|
+
|
|
206
|
+
## Analyze traces
|
|
207
|
+
|
|
208
|
+
`analyze(id, ...)` starts a wave and returns immediately. `watchAnalysis` / `watch_analysis` waits for that wave. Configuration and examples are on [analyses](/sdk-reference/analyses).
|
|
209
|
+
|
|
210
|
+
For a job created with embedded analysis, wait for the job first, then its analyses. A still-running job can temporarily have zero pending analyses.
|
|
211
|
+
|
|
212
|
+
## Download or import results
|
|
213
|
+
|
|
214
|
+
```ts TypeScript
|
|
215
|
+
const path = await client.download(jobId, { to: "./results" });
|
|
216
|
+
const imported = await client.upload("./local-job");
|
|
217
|
+
const settled = await client.watchImport(imported.id);
|
|
218
|
+
console.log(settled.status, settled.job_id, settled.failure);
|
|
219
|
+
```
|
|
220
|
+
|
|
221
|
+
```python Python
|
|
222
|
+
path = await client.download(job_id, to="./results")
|
|
223
|
+
imported = await client.upload("./local-job")
|
|
224
|
+
settled = await client.watch_import(imported.id)
|
|
225
|
+
print(settled.status, settled.job_id, settled.failure)
|
|
226
|
+
```
|
|
227
|
+
|
|
228
|
+
Download requires a terminal job. Without `to`, it returns a `Buffer` / `bytes`; TypeScript also supports `{stream: true}`. See [transfer behavior](/sdk-reference/filesystem#download-options).
|
|
229
|
+
|
|
230
|
+
| Import method | Input/options |
|
|
231
|
+
| --- | --- |
|
|
232
|
+
| `upload(source, ...)` | Local job directory or archive path. TypeScript also accepts `{archive_url}`; Python takes `archive_url=` instead of a path. |
|
|
233
|
+
| Upload options | Optional `dataset`, upload progress callback, and import-registration callback. |
|
|
234
|
+
| `getImport` / `get_import` | Import id |
|
|
235
|
+
| `listImports` / `list_imports` | `status`, `limit`, `cursor` |
|
|
236
|
+
| `watchImport` / `watch_import` | TS: `onStatus`, `onProgress`, `pollIntervalMs`, `signal`. Python: `on_status`, `on_progress`, `poll_interval_s`, `timeout_s`. |
|
|
237
|
+
|
|
238
|
+
Callback names and timing use the same [upload/watch conventions as datasets](/sdk-reference/datasets#follow-a-publish). Uploaded results become a terminal record. They can be analyzed, but cannot be resumed, retried, or regraded.
|
|
239
|
+
|
|
240
|
+
## Share and delete
|
|
241
|
+
|
|
242
|
+
| Action | TypeScript | Python |
|
|
243
|
+
| --- | --- | --- |
|
|
244
|
+
| Enable an unlisted link | `share(id, { link: true })` | `share(id, link=True)` |
|
|
245
|
+
| Share with people | `share(id, { emails: ["person@example.com"] })` | `share(id, emails=["person@example.com"])` |
|
|
246
|
+
| Revoke either form | `unshare(id, { link, emails })` | `unshare(id, link=..., emails=...)` |
|
|
247
|
+
| Read share state | `shares(id)` | `shares(id)` |
|
|
248
|
+
| Permanently delete | `delete(id)` | `delete(id)` |
|
|
249
|
+
|
|
250
|
+
Sharing can send invitation email. Only the creator can manage shares or delete the job; team membership alone does not grant those actions. Deletion requires a terminal job with no running analysis or dependent regrade. The returned receipt counts deleted trials and analyses.
|
|
@@ -0,0 +1,54 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: "Platform capabilities"
|
|
3
|
+
description: "Discover available harnesses, models, providers, and current limits."
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
[Method reference: calls, parameters, and response fields](/sdk-reference/methods/meta).
|
|
7
|
+
|
|
8
|
+
`meta()` returns the platform's capability document. It needs no API key.
|
|
9
|
+
|
|
10
|
+
```ts TypeScript
|
|
11
|
+
import { meta } from "@evolvingmachines/evolve";
|
|
12
|
+
|
|
13
|
+
const capabilities = await meta();
|
|
14
|
+
for (const agent of capabilities.agents) {
|
|
15
|
+
console.log(agent.name, agent.runnable, agent.models.map((m) => m.alias));
|
|
16
|
+
}
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
```python Python
|
|
20
|
+
from evolve import meta
|
|
21
|
+
|
|
22
|
+
capabilities = await meta()
|
|
23
|
+
for agent in capabilities.agents:
|
|
24
|
+
print(agent.name, agent.runnable, [m.alias for m in agent.models])
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
Also available as `hosted().meta()` in both languages.
|
|
28
|
+
|
|
29
|
+
## What to read
|
|
30
|
+
|
|
31
|
+
| Field | Use it to answer |
|
|
32
|
+
| --- | --- |
|
|
33
|
+
| `agents` | Which harnesses run? Which models, efforts, versions, kwargs, and presets do they support? |
|
|
34
|
+
| `sandbox_providers` | Which providers support the task's resources and features? |
|
|
35
|
+
| `platform_constraints` | Which requirements cannot run on this deployment? |
|
|
36
|
+
| `network_modes` | Which network modes can a task declare? |
|
|
37
|
+
| `analyze` | Which agents can analyses and checks run on, with which models and reasoning efforts? |
|
|
38
|
+
| `limits` | What are the current defaults and bounds for jobs, pages, uploads, and comparisons? |
|
|
39
|
+
| `statuses` | Which states exist, and which are terminal? |
|
|
40
|
+
| `agent_registration` | Which names, environment keys, and upload sizes can a custom agent use? |
|
|
41
|
+
| `error_codes`, `import_warning_codes` | Which failure and warning codes does this deployment publish? |
|
|
42
|
+
| `managed_providers` | Which providers are available to the separate managed-agent runtime? |
|
|
43
|
+
|
|
44
|
+
**Tip:**
|
|
45
|
+
|
|
46
|
+
Read capabilities when building a picker or validating a saved configuration. A documented example is useful; a fixed copy of the platform's model catalog becomes stale.
|
|
47
|
+
|
|
48
|
+
## Configuration
|
|
49
|
+
|
|
50
|
+
`meta(config?)` accepts the usual [hosted configuration](/sdk-reference/index#configure-once). Only the API origin matters; credentials are not sent.
|
|
51
|
+
|
|
52
|
+
The result includes `schema_version`. Treat it as the document's schema version, not the SDK package version.
|
|
53
|
+
|
|
54
|
+
Read the full editable analyzer policy through [analysis defaults](/sdk-reference/analyses#read-current-defaults) or [check defaults](/sdk-reference/checks#read-current-defaults).
|