@evolvingmachines/evolve 0.0.56 → 0.0.58
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/cli/index.cjs +30 -30
- package/dist/cli/index.d.cts +3 -1
- package/dist/cli/index.d.ts +3 -1
- package/dist/cli/index.js +26 -26
- package/package.json +4 -4
- package/skills/evolve/SKILL.md +1 -1
- package/skills/evolve-evals/SKILL.md +0 -1
- package/skills/evolve-evals/references/cli-reference/run.mdx +0 -4
- package/skills/evolve-evals/references/core-concepts/agents.mdx +1 -1
- package/skills/evolve-evals/references/getting-started/quick-start.mdx +19 -10
- package/skills/evolve-evals/references/index.mdx +12 -0
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@evolvingmachines/evolve",
|
|
3
|
-
"version": "0.0.
|
|
3
|
+
"version": "0.0.58",
|
|
4
4
|
"keywords": [
|
|
5
5
|
"ai",
|
|
6
6
|
"agents",
|
|
@@ -129,9 +129,9 @@
|
|
|
129
129
|
},
|
|
130
130
|
"dependencies": {
|
|
131
131
|
"@agentclientprotocol/sdk": "^0.5.1",
|
|
132
|
-
"@evolvingmachines/daytona": "0.0.
|
|
133
|
-
"@evolvingmachines/e2b": "0.0.
|
|
134
|
-
"@evolvingmachines/modal": "0.0.
|
|
132
|
+
"@evolvingmachines/daytona": "0.0.58",
|
|
133
|
+
"@evolvingmachines/e2b": "0.0.58",
|
|
134
|
+
"@evolvingmachines/modal": "0.0.58",
|
|
135
135
|
"ajv": "^8.17.1",
|
|
136
136
|
"p-map": "^7.0.2",
|
|
137
137
|
"smol-toml": "^1.7.0",
|
package/skills/evolve/SKILL.md
CHANGED
|
@@ -14,7 +14,7 @@ Python SDK: `pip install evolvingmachines-evolve` (`import evolve`). The manual
|
|
|
14
14
|
|
|
15
15
|
## Start here
|
|
16
16
|
|
|
17
|
-
This file is a pointer, not the manual. The manual ships inside the CLI and always matches the installed version. Before running any `evolve` command, load it:
|
|
17
|
+
This file is a pointer, not the manual. The manual ships inside the CLI and always matches the installed version. Keep it current: `npm i -g @evolvingmachines/evolve@latest`, then `evolve skills install --force` refreshes this file. Before running any `evolve` command, load it:
|
|
18
18
|
|
|
19
19
|
```bash
|
|
20
20
|
evolve skills get evals # the index of the documentation: every page, one line each
|
|
@@ -18,7 +18,6 @@ The pages under `references/` are the documentation site's pages, byte for byte,
|
|
|
18
18
|
2. Every CLI verb is documented from its own `--help`; run `evolve <verb> --help` to confirm the flags of the installed version.
|
|
19
19
|
3. Every command and every SDK client reads `EVOLVE_API_KEY`; the Installation page says where the key comes from.
|
|
20
20
|
|
|
21
|
-
## Topic index
|
|
22
21
|
|
|
23
22
|
## Docs
|
|
24
23
|
|
|
@@ -83,10 +83,6 @@ Each of these applies to every arm of the job.
|
|
|
83
83
|
Skill for every agent arm. Repeatable. Accepts `skills.sh/<owner>/<repo>[/<skill>]`, `org/repo[@ref]`, an https git URL, `upload:<id>`, `name:<skill-name>` (your moving name pointer, resolved server-side), or a local folder, which is uploaded first and then referenced; `--print-config` still shows the path you typed.
|
|
84
84
|
</ParamField>
|
|
85
85
|
|
|
86
|
-
<ParamField path="--ae, --agent-env" type="KEY=VALUE">
|
|
87
|
-
Sent as the job's `agent_env`, which the server refuses on this deployment: the create fails naming the two `--ve` keys it honors instead. Agent environment comes from the task's `[environment.env]` table or an attached `--secret`.
|
|
88
|
-
</ParamField>
|
|
89
|
-
|
|
90
86
|
<ParamField path="--ve, --verifier-env" type="KEY=VALUE">
|
|
91
87
|
Environment for every verifier run. Repeatable. The server honors exactly `REWARDKIT_JUDGE` and `REWARDKIT_MODEL`, rewardkit's judge override, and refuses any other key.
|
|
92
88
|
</ParamField>
|
|
@@ -45,7 +45,7 @@ evolve run \
|
|
|
45
45
|
|
|
46
46
|
Only `claude` and `codex` take a config or a preset; the [capability document](/sdk-reference/meta) publishes `supports_config` and `presets` per harness. An effort, preset or config a harness cannot honor is refused when the job is created, never silently skipped, and so is a config key that touches billing, base URLs, routing or environment.
|
|
47
47
|
|
|
48
|
-
Environment for the agent comes from two places, a task's own `[environment.env]` table and a secret attached with `--secret`; a job cannot add a third
|
|
48
|
+
Environment for the agent comes from two places, a task's own `[environment.env]` table and a secret attached with `--secret`; a job cannot add a third. See [secrets](/core-concepts/secrets) and [tasks](/core-concepts/tasks#environment-variables).
|
|
49
49
|
|
|
50
50
|
## Register your own agent
|
|
51
51
|
|
|
@@ -16,6 +16,14 @@ This page takes you from an empty terminal to a downloaded trial: one API key, a
|
|
|
16
16
|
Create a key on the dashboard's [API keys page](https://dashboard.evolvingmachines.ai/api-keys), then export it. See [installation](/getting-started/installation) for the SDKs.
|
|
17
17
|
</Step>
|
|
18
18
|
|
|
19
|
+
<Step title="Give your coding agent the manual">
|
|
20
|
+
```bash
|
|
21
|
+
evolve skills install
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
Your agent then reads these docs with `evolve skills get evals`, and the `create-task`, `rewardkit`, `create-adapter` and `publish` skills by name. [The five skills](/cli-reference/skills).
|
|
25
|
+
</Step>
|
|
26
|
+
|
|
19
27
|
<Step title="Pick a dataset">
|
|
20
28
|
The catalog lists every dataset you can run, with its versions.
|
|
21
29
|
|
|
@@ -28,7 +36,7 @@ This page takes you from an empty terminal to a downloaded trial: one API key, a
|
|
|
28
36
|
</Step>
|
|
29
37
|
|
|
30
38
|
<Step title="Start a job and watch it">
|
|
31
|
-
Name a dataset, an agent, and a model. `--watch` streams the job's events until it finishes; `-l 5` caps the run at five tasks so the first job is small.
|
|
39
|
+
Name a dataset, an agent, and a model. `--watch` streams the job's events until it finishes; `-l 5` caps the run at five tasks and `--max-trial-spend 1` caps what each trial may spend on the model, so the first job is small.
|
|
32
40
|
|
|
33
41
|
```bash
|
|
34
42
|
evolve run \
|
|
@@ -36,6 +44,7 @@ This page takes you from an empty terminal to a downloaded trial: one API key, a
|
|
|
36
44
|
-a codex \
|
|
37
45
|
-m gpt-6-astra \
|
|
38
46
|
-l 5 \
|
|
47
|
+
--max-trial-spend 1 \
|
|
39
48
|
--watch
|
|
40
49
|
```
|
|
41
50
|
|
|
@@ -67,19 +76,19 @@ This page takes you from an empty terminal to a downloaded trial: one API key, a
|
|
|
67
76
|
</Step>
|
|
68
77
|
</Steps>
|
|
69
78
|
|
|
70
|
-
## Next
|
|
79
|
+
## Next, by what you want to do
|
|
71
80
|
|
|
72
81
|
<Columns cols={2}>
|
|
73
|
-
<Card title="
|
|
74
|
-
|
|
82
|
+
<Card title="Port a benchmark" icon="arrow-right-left" href="/core-concepts/tasks">
|
|
83
|
+
`evolve skills get create-adapter` turns it into Harbor-format tasks.
|
|
75
84
|
</Card>
|
|
76
|
-
<Card title="
|
|
77
|
-
|
|
85
|
+
<Card title="Write a task" icon="file-plus" href="/core-concepts/tasks">
|
|
86
|
+
`evolve skills get create-task`, and `rewardkit` for its verifier.
|
|
78
87
|
</Card>
|
|
79
|
-
<Card title="
|
|
80
|
-
|
|
88
|
+
<Card title="Publish a dataset" icon="upload" href="/core-concepts/datasets">
|
|
89
|
+
`evolve skills get publish`, then `evolve dataset publish`.
|
|
81
90
|
</Card>
|
|
82
|
-
<Card title="
|
|
83
|
-
|
|
91
|
+
<Card title="Upload a job you ran elsewhere" icon="folder-up" href="/core-concepts/upload">
|
|
92
|
+
`evolve upload <job dir>` brings a Harbor job into the dashboard.
|
|
84
93
|
</Card>
|
|
85
94
|
</Columns>
|
|
@@ -20,6 +20,18 @@ evolve run \
|
|
|
20
20
|
--watch
|
|
21
21
|
```
|
|
22
22
|
|
|
23
|
+
New here? The [quick start](/getting-started/quick-start) takes five minutes.
|
|
24
|
+
|
|
25
|
+
## The whole process
|
|
26
|
+
|
|
27
|
+
1. [Get tasks](/core-concepts/tasks): a public dataset, your own, or a converted benchmark.
|
|
28
|
+
2. [Check](/core-concepts/check) each task against a rubric before you run it.
|
|
29
|
+
3. [Publish](/core-concepts/datasets) the tasks as a versioned dataset.
|
|
30
|
+
4. [Run](/core-concepts/jobs) any agent and model on it; every trial streams live.
|
|
31
|
+
5. [Read](/core-concepts/trial-outputs) every trial's tree and [analyze](/core-concepts/analyze) its trace.
|
|
32
|
+
|
|
33
|
+
A coding agent does all five from the same CLI with the [skills](/cli-reference/skills).
|
|
34
|
+
|
|
23
35
|
<Columns cols={2}>
|
|
24
36
|
<Card title="Quick start" icon="play" href="/getting-started/quick-start">
|
|
25
37
|
Install the CLI, start a job, read its result.
|