@lmmslab/harness 0.0.0-stage → 0.81.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (170) hide show
  1. package/LICENSE +21 -0
  2. package/README.md +124 -2
  3. package/bin/harness.js +71 -0
  4. package/package.json +34 -4
  5. package/python/harness/README.md +72 -0
  6. package/python/harness/__init__.py +8 -0
  7. package/python/harness/__main__.py +3 -0
  8. package/python/harness/bench.py +124 -0
  9. package/python/harness/builtin/benches/README.md +5 -0
  10. package/python/harness/builtin/benches/none/bench.toml +6 -0
  11. package/python/harness/builtin/configs/README.md +26 -0
  12. package/python/harness/builtin/configs/assistant.toml +11 -0
  13. package/python/harness/builtin/configs/default.toml +33 -0
  14. package/python/harness/builtin/configs/deploy/example-api.toml +10 -0
  15. package/python/harness/builtin/configs/models/deepseek-api.toml +21 -0
  16. package/python/harness/builtin/configs/models/deepseek-flash.toml +12 -0
  17. package/python/harness/builtin/configs/models/grok-low.toml +8 -0
  18. package/python/harness/builtin/configs/models/grok-medium.toml +43 -0
  19. package/python/harness/builtin/configs/models/opus-5.5.toml +11 -0
  20. package/python/harness/builtin/environment/README.md +53 -0
  21. package/python/harness/builtin/environment/constraints.txt +31 -0
  22. package/python/harness/builtin/model-settings.js +143 -0
  23. package/python/harness/builtin/prompts/conversation/core.md +5 -0
  24. package/python/harness/builtin/prompts/conversation/pack.toml +12 -0
  25. package/python/harness/builtin/prompts/core/core.md +88 -0
  26. package/python/harness/builtin/prompts/core/pack.toml +9 -0
  27. package/python/harness/builtin/prompts/reflect/core.md +45 -0
  28. package/python/harness/builtin/prompts/reflect/pack.toml +7 -0
  29. package/python/harness/builtin/prompts/shared/agent-skills.md +5 -0
  30. package/python/harness/builtin/prompts/shared/agents.md +14 -0
  31. package/python/harness/builtin/prompts/shared/checklist.md +21 -0
  32. package/python/harness/builtin/prompts/shared/entries.md +86 -0
  33. package/python/harness/builtin/prompts/shared/environments.md +21 -0
  34. package/python/harness/builtin/prompts/shared/learning-goal.md +31 -0
  35. package/python/harness/builtin/prompts/shared/learning.md +27 -0
  36. package/python/harness/builtin/prompts/shared/mcp.md +9 -0
  37. package/python/harness/builtin/prompts/shared/world.md +1 -0
  38. package/python/harness/builtin/skills/README.md +10 -0
  39. package/python/harness/builtin/skills/agents/__init__.py +88 -0
  40. package/python/harness/builtin/skills/agents/skill.toml +11 -0
  41. package/python/harness/builtin/skills/checklist/__init__.py +32 -0
  42. package/python/harness/builtin/skills/checklist/skill.toml +9 -0
  43. package/python/harness/builtin/skills/conversation/__init__.py +23 -0
  44. package/python/harness/builtin/skills/conversation/skill.toml +8 -0
  45. package/python/harness/builtin/skills/entries/__init__.py +62 -0
  46. package/python/harness/builtin/skills/entries/skill.toml +8 -0
  47. package/python/harness/builtin/skills/environments/__init__.py +69 -0
  48. package/python/harness/builtin/skills/environments/skill.toml +9 -0
  49. package/python/harness/builtin/skills/experience/__init__.py +27 -0
  50. package/python/harness/builtin/skills/experience/skill.toml +9 -0
  51. package/python/harness/builtin/skills/instructions/__init__.py +29 -0
  52. package/python/harness/builtin/skills/instructions/skill.toml +9 -0
  53. package/python/harness/builtin/skills/learning/__init__.py +189 -0
  54. package/python/harness/builtin/skills/learning/skill.toml +7 -0
  55. package/python/harness/builtin/skills/mcp/__init__.py +54 -0
  56. package/python/harness/builtin/skills/mcp/skill.toml +10 -0
  57. package/python/harness/builtin/skills/memory/__init__.py +130 -0
  58. package/python/harness/builtin/skills/memory/skill.toml +13 -0
  59. package/python/harness/builtin/skills/verify/__init__.py +31 -0
  60. package/python/harness/builtin/skills/verify/skill.toml +7 -0
  61. package/python/harness/builtin/third_party/sources.toml +40 -0
  62. package/python/harness/cli.py +1182 -0
  63. package/python/harness/config/__init__.py +224 -0
  64. package/python/harness/config/agent.py +777 -0
  65. package/python/harness/core/__init__.py +52 -0
  66. package/python/harness/core/artifacts.py +46 -0
  67. package/python/harness/core/codec.py +160 -0
  68. package/python/harness/core/contracts.py +75 -0
  69. package/python/harness/core/episodes.py +150 -0
  70. package/python/harness/core/format.py +51 -0
  71. package/python/harness/core/http.py +124 -0
  72. package/python/harness/core/lanes.py +324 -0
  73. package/python/harness/core/linux_sandbox.py +55 -0
  74. package/python/harness/core/media.py +96 -0
  75. package/python/harness/core/memory.py +147 -0
  76. package/python/harness/core/ports.py +63 -0
  77. package/python/harness/core/procs.py +227 -0
  78. package/python/harness/core/remote.py +102 -0
  79. package/python/harness/core/spans.py +166 -0
  80. package/python/harness/core/trace.py +139 -0
  81. package/python/harness/core/wait.py +12 -0
  82. package/python/harness/core/worker.py +388 -0
  83. package/python/harness/core/worker_pipe.py +119 -0
  84. package/python/harness/embodiments.py +217 -0
  85. package/python/harness/extensions.py +112 -0
  86. package/python/harness/host/__init__.py +397 -0
  87. package/python/harness/host/client.py +201 -0
  88. package/python/harness/host/http.py +720 -0
  89. package/python/harness/host/listing.py +68 -0
  90. package/python/harness/host/server.py +189 -0
  91. package/python/harness/host/sessions.py +406 -0
  92. package/python/harness/host/tasks.py +1308 -0
  93. package/python/harness/host/workers.py +257 -0
  94. package/python/harness/host/workspaces.py +256 -0
  95. package/python/harness/model_list.py +117 -0
  96. package/python/harness/packages/__init__.py +73 -0
  97. package/python/harness/packages/addons.py +121 -0
  98. package/python/harness/packages/archive.py +122 -0
  99. package/python/harness/packages/describe.py +46 -0
  100. package/python/harness/packages/environments.py +448 -0
  101. package/python/harness/packages/fetch.py +57 -0
  102. package/python/harness/packages/manager.py +587 -0
  103. package/python/harness/packages/manifest.py +221 -0
  104. package/python/harness/packages/migrate.py +68 -0
  105. package/python/harness/packages/provision.py +38 -0
  106. package/python/harness/packages/registry.py +275 -0
  107. package/python/harness/packages/resolve.py +89 -0
  108. package/python/harness/packages/sources.py +141 -0
  109. package/python/harness/packages/state.py +213 -0
  110. package/python/harness/packages/versions.py +112 -0
  111. package/python/harness/paths.py +218 -0
  112. package/python/harness/providers.py +303 -0
  113. package/python/harness/runtime/__init__.py +8 -0
  114. package/python/harness/runtime/decision/__init__.py +1 -0
  115. package/python/harness/runtime/decision/loop.py +198 -0
  116. package/python/harness/runtime/decision/messages.py +271 -0
  117. package/python/harness/runtime/decision/model.py +424 -0
  118. package/python/harness/runtime/decision/prompts.py +248 -0
  119. package/python/harness/runtime/decision/protocol.py +169 -0
  120. package/python/harness/runtime/decision/thoughts.py +199 -0
  121. package/python/harness/runtime/events.py +213 -0
  122. package/python/harness/runtime/execution/__init__.py +1 -0
  123. package/python/harness/runtime/execution/attached.py +96 -0
  124. package/python/harness/runtime/execution/control.py +211 -0
  125. package/python/harness/runtime/execution/entries.py +665 -0
  126. package/python/harness/runtime/execution/kernel.py +811 -0
  127. package/python/harness/runtime/execution/noenv.py +84 -0
  128. package/python/harness/runtime/execution/results.py +40 -0
  129. package/python/harness/runtime/execution/session.py +871 -0
  130. package/python/harness/runtime/execution/skills.py +436 -0
  131. package/python/harness/runtime/learning/__init__.py +1 -0
  132. package/python/harness/runtime/learning/active.py +314 -0
  133. package/python/harness/runtime/learning/experience.py +48 -0
  134. package/python/harness/runtime/learning/fixtures.py +167 -0
  135. package/python/harness/runtime/learning/learner.py +651 -0
  136. package/python/harness/runtime/learning/library.py +504 -0
  137. package/python/harness/runtime/learning/reflect.py +151 -0
  138. package/python/harness/runtime/learning/usage.py +30 -0
  139. package/python/harness/runtime/learning/views.py +89 -0
  140. package/python/harness/runtime/reactive/__init__.py +1 -0
  141. package/python/harness/runtime/reactive/rules.py +549 -0
  142. package/python/harness/runtime/safety/__init__.py +1 -0
  143. package/python/harness/runtime/safety/approval.py +131 -0
  144. package/python/harness/runtime/safety/input.py +62 -0
  145. package/python/harness/runtime/safety/isolation.py +419 -0
  146. package/python/harness/runtime/safety/shield.py +129 -0
  147. package/python/harness/runtime/status.py +23 -0
  148. package/python/harness/runtime/task/__init__.py +7 -0
  149. package/python/harness/runtime/task/context.py +431 -0
  150. package/python/harness/runtime/task/delegation.py +468 -0
  151. package/python/harness/runtime/task/environments.py +268 -0
  152. package/python/harness/runtime/task/run.py +2878 -0
  153. package/python/harness/runtime/tick.py +909 -0
  154. package/python/harness/sdk/__init__.py +281 -0
  155. package/python/harness/services/__init__.py +6 -0
  156. package/python/harness/services/broker.py +35 -0
  157. package/python/harness/services/connectors.py +88 -0
  158. package/python/harness/services/mcp.py +298 -0
  159. package/python/harness/services/mcp_stdio.py +131 -0
  160. package/python/harness/services/placement.py +82 -0
  161. package/python/harness/services/registry.py +156 -0
  162. package/python/harness/settings.py +536 -0
  163. package/python/harness/terminal.py +79 -0
  164. package/python/harness/transcript.py +297 -0
  165. package/python/harness/tui/__init__.py +68 -0
  166. package/python/harness/tui/chat.py +1181 -0
  167. package/python/harness/tui/pty.py +81 -0
  168. package/python/harness/tui/renderer.py +45 -0
  169. package/python/harness/tui/timeline_rows.py +58 -0
  170. package/python/harness/tui/views.py +434 -0
@@ -0,0 +1,43 @@
1
+ # Hosted models through OpenRouter (the key from OPENROUTER_API_KEY; never write it
2
+ # here): x-ai/grok-4.7 at medium reasoning effort.
3
+ description = "grok-4.7 (medium) writes programs, answers questions about images and points; Molmo2 for a pointer role"
4
+
5
+ [main]
6
+ backend = "openrouter"
7
+ model = "x-ai/grok-4.7"
8
+ reasoning_effort = "medium"
9
+ # On OpenRouter the limit covers reasoning tokens too.
10
+ max_output_tokens = 16000
11
+ # Streamed: keep-alive chunks arrive while the model thinks, so a dead connection
12
+ # shows up within the read timeout (the link to OpenRouter drops now and then).
13
+ stream = true
14
+ timeout_seconds = 120
15
+ # Completions have no side effects; retry dropped connections and rate limits for up to
16
+ # 30 minutes (the simulator waits while the model is called).
17
+ retries = 3
18
+ retry_seconds = 1800
19
+
20
+ # Questions about images inside programs (query_vlm, holding() without a measured
21
+ # gripper, pointing when there is no pointer) go to the main model itself: one model
22
+ # sees the task and the images, as in Z2_v01, instead of a second, weaker model
23
+ # answering for it (1.1.0 asked qwen3.8-flash, whose answers about what an object is
24
+ # changed from one question to the next).
25
+ [vlm]
26
+ backend = "openrouter"
27
+ model = "x-ai/grok-4.7"
28
+ reasoning_effort = "low"
29
+ stream = true
30
+ timeout_seconds = 120
31
+ # The link drops now and then; give up after about 5 minutes so the program sees an
32
+ # error instead of the kernel being killed.
33
+ retries = 3
34
+ retry_seconds = 300
35
+
36
+ # Open-vocabulary pointing (point_to) with Molmo2 (Clark et al., 2026), self-hosted, for
37
+ # agents that take the pointer role; the default agent's vlm role points instead.
38
+ [pointer]
39
+ service = "molmo2"
40
+
41
+ [runtime]
42
+ # Images of the two most recent turns; text history is kept.
43
+ keep_images = 2
@@ -0,0 +1,11 @@
1
+ # Claude Opus 5.5 through OpenRouter.
2
+ description = "Claude Opus 5.5 writes programs, answers questions about images and points; Molmo2 for a pointer role"
3
+ include = ["grok-medium.toml"]
4
+
5
+ [main]
6
+ model = "anthropic/claude-opus-5.5"
7
+
8
+ # The same model answers questions about images inside programs (see grok-medium.toml).
9
+ [vlm]
10
+ model = "anthropic/claude-opus-5.5"
11
+ reasoning_effort = "low"
@@ -0,0 +1,53 @@
1
+ # Environments
2
+
3
+ The harness itself is standard-library Python and installs with npm or pip:
4
+ `npm install -g @lmmslab/harness`, `pip install the-harness`, or `pip install -e .` in a checkout. Everything a run needs
5
+ beyond it is installed the first time a run needs it, or ahead of time with
6
+ `harness install <name>`: each service and each benchmark gets an environment of its
7
+ own under `~/.cache/harness/envs/` (`HARNESS_HOME` moves it), built from the
8
+ `[environment]` table of its manifest (`services/<name>.toml`,
9
+ `benches/<name>/bench.toml`).
10
+
11
+ This folder holds the pinned core (`constraints.txt`: the browser benchmark's
12
+ environment, and `install.sh` of an exported XPolicyLab policy). What the environments of
13
+ services and benchmarks share is the local-models package's `environment/`:
14
+
15
+ | File | Used by |
16
+ |---|---|
17
+ | `lock/pip-freeze.txt` | the pins every service's and benchmark's environment installs under (`constraints = "environment/lock/pip-freeze.txt"`) |
18
+ | `requirements-skills.txt` | what `install.sh` of an exported XPolicyLab policy installs into RoboDojo's policy environment: the packages of the analytic and learned skills |
19
+ | `cuda_toolkit.sh` | the CUDA 12.8 build toolkit cuRobo is compiled with (RoboDojo's and RoboTwin's environments) |
20
+
21
+ RoboTwin's two source edits and its own cuRobo build are the robotwin package's
22
+ (`environment/robotwin_edits.py`, `environment/robotwin_curobo.sh`).
23
+
24
+ Every install runs under the pins, so no service or benchmark can move torch, numpy,
25
+ Isaac Sim or websockets: a package that cannot be installed under them fails instead of
26
+ upgrading the core. An environment's directory is named by a hash of everything that
27
+ decides what is installed (its table and the pins, comments excluded), so changing a pin
28
+ builds the environments that use it anew the next time a run needs them and leaves the
29
+ others as they are.
30
+
31
+ The documentation site is not Python: it is a Deno project with its own lock
32
+ (`docs/deno.json`, `docs/deno.lock`; see `docs/README.md`).
33
+
34
+ ## Changing a pin or adding a dependency
35
+
36
+ A dependency of a service or benchmark goes into the `requirements` of its manifest; a
37
+ pin goes into `lock/pip-freeze.txt`. Install it with `harness install <name>`, run
38
+ something that uses it, and commit the manifest and the pins together.
39
+
40
+ ## One interpreter for everything
41
+
42
+ `HARNESS_ENVIRONMENTS=project` (or `environments = "project"` in a deploy) installs
43
+ nothing but weights: every service and worker runs in the interpreter of the harness
44
+ itself. It is for a machine where one environment already holds everything, not a way
45
+ to install; that machine also sets what those environments would set, such as
46
+ `OMNI_KIT_ACCEPT_EULA=YES` for RoboDojo's workers.
47
+
48
+ ## Known deviations from upstream
49
+
50
+ - `isaacsim-kernel` declares `websockets==12.0`, which only Kit's asset-search
51
+ extensions use; headless simulation does not load them. XPolicyLab policy servers
52
+ need the websockets 13+ asyncio API, so RoboDojo's environment overrides it with 17.1
53
+ (`overrides` in `benches/robodojo/bench.toml`).
@@ -0,0 +1,31 @@
1
+ # Pinned core: the browser benchmark's environment and install.sh of an exported
2
+ # XPolicyLab policy install under these constraints. Change a line here only
3
+ # together with a run that uses it (environment/README.md).
4
+
5
+ # PyTorch CUDA 12.8 stack used by Isaac Sim 5.1 and cuRobo.
6
+ torch==2.7.0
7
+ torchvision==0.22.0
8
+ torchaudio==2.7.0
9
+ isaacsim==5.1.0
10
+
11
+ # Isaac Sim / Isaac Lab runtime pins, as in RoboDojo scripts/install.sh.
12
+ numpy==1.26.0
13
+ packaging==23.0
14
+ typing_extensions==4.12.2
15
+ filelock==3.13.1
16
+ # isaacsim-kernel declares websockets==12.0 for Kit asset-search extensions that
17
+ # headless simulation does not load. XPolicyLab policy servers need the
18
+ # websockets>=13 asyncio API, so the environment uses 17.1 (see README).
19
+ websockets==17.1
20
+ scipy==1.15.3
21
+ warp-lang==1.11.0
22
+ click==8.1.7
23
+ psutil==5.9.8
24
+ wheel==0.45.1
25
+ starlette==0.45.3
26
+ onnx>=1.18,<1.22
27
+ ipython<9
28
+ virtualenv==20.30.0
29
+
30
+ # Model code shared by learned skills and perception.
31
+ transformers==4.57.6
@@ -0,0 +1,143 @@
1
+ // Shared by the App and TapTab.
2
+ export async function mountModels(root, { api, tr, changed }) {
3
+ function node(tag, props = {}, ...children) {
4
+ const element = Object.assign(document.createElement(tag), props);
5
+ element.append(...children);
6
+ return element;
7
+ }
8
+ const [{ models, path }, { providers, llm }, { agents }] = await Promise.all([
9
+ api("/api/app/models"), api("/api/app"), api("/api/app/agents"),
10
+ ]);
11
+ const sources = [...providers, ...agents.filter((agent) => agent.installed)];
12
+ const source = node("select", { ariaLabel: tr("Provider", "服务商") },
13
+ ...sources.map((provider) => new Option(provider.name, provider.id)));
14
+ source.value = llm.provider || sources[0]?.id || "";
15
+ const query = node("input", { type: "search", placeholder: tr("Search or enter a model ID", "搜索或输入模型 ID"),
16
+ ariaLabel: tr("Model ID", "模型 ID") });
17
+ const endpoint = node("input", { type: "url", placeholder: tr("Base URL", "服务地址") });
18
+ const name = node("input", { placeholder: tr("Display name (optional)", "显示名称(选填)") });
19
+ const levels = node("input", { placeholder: tr("Effort IDs, comma separated", "推理档位 ID,用逗号分隔") });
20
+ const images = node("input", { type: "checkbox", checked: true });
21
+ const reads = node("label", { title: tr("Untick for a text-only model: no picture is sent to it.",
22
+ "纯文本模型请取消勾选:不会向它发送图片。") }, images, " " + tr("Reads images", "能读图"));
23
+ const saved = node("div");
24
+ const matches = node("div", { className: "saved-model-matches" });
25
+ const message = node("div", { role: "status" });
26
+ const details = node("details", {}, node("summary", { textContent: tr("Model details", "模型详情") }),
27
+ node("div", { className: "saved-model-controls" }, name, levels, endpoint, reads));
28
+ let rows = models;
29
+ let catalog = [];
30
+ let selected = null;
31
+ let editing = null;
32
+ let revision = 0;
33
+ const button = (label, onclick) => node("button", { type: "button", textContent: label, onclick });
34
+ async function run(action, operation) {
35
+ action.disabled = true;
36
+ message.textContent = "";
37
+ try { await operation(); } catch (failure) { message.textContent = failure.message; }
38
+ finally { action.disabled = false; }
39
+ }
40
+ async function change(body) {
41
+ ({ models: rows } = await api("/api/app/models", body));
42
+ renderRows();
43
+ await changed?.();
44
+ }
45
+ function choose(row, local = false) {
46
+ if (local) {
47
+ source.value = row.provider;
48
+ reset(false);
49
+ editing = row.id;
50
+ }
51
+ selected = local ? null : row;
52
+ query.value = local ? row.model : row.id;
53
+ name.value = row.name;
54
+ levels.value = (row.efforts || []).join(", ");
55
+ endpoint.value = row.base_url || "";
56
+ // Whether the model reads images: the catalog's or the saved entry's word; unknown reads them.
57
+ images.checked = row.images !== false;
58
+ renderMatches();
59
+ query.focus();
60
+ if (local) details.open = true;
61
+ }
62
+ function renderRows() {
63
+ saved.replaceChildren(...rows.map((row) => {
64
+ const toggle = node("input", { type: "checkbox", role: "switch", checked: row.enabled,
65
+ ariaLabel: tr("Show in picker", "在模型菜单中显示") + " · " + row.name });
66
+ toggle.onchange = () => run(toggle, async () => {
67
+ try { await change({ action: "enable", id: row.id, enabled: toggle.checked }); }
68
+ catch (error) { toggle.checked = row.enabled; throw error; }
69
+ });
70
+ const provider = sources.find(({ id }) => id === row.provider)?.name || row.provider;
71
+ const copy = node("div", {}, node("div", { textContent: row.name }),
72
+ node("small", { textContent: [provider, row.base_url,
73
+ row.images === false ? tr("text only", "纯文本") : null].filter(Boolean).join(" · ") }));
74
+ return node("div", { className: "saved-model-row" }, copy, toggle,
75
+ button(tr("Edit", "编辑"), () => choose(row, true)),
76
+ button(tr("Remove", "移除"), (event) => run(event.currentTarget, () => change({ action: "remove", id: row.id }))));
77
+ }));
78
+ if (!rows.length) saved.append(node("p", { textContent: tr("Add a model to use it in a conversation.", "添加一个模型,即可在会话中选用。") }));
79
+ }
80
+ function renderMatches() {
81
+ const term = query.value.trim().toLowerCase();
82
+ const filtered = catalog.filter((row) => (row.id + " " + row.name).toLowerCase().includes(term)).slice(0, 30);
83
+ matches.replaceChildren(...(selected ? [] : filtered.map((row) => button(row.name, () => choose(row)))));
84
+ }
85
+ function reset(clear = true) {
86
+ revision++;
87
+ editing = selected = null;
88
+ catalog = [];
89
+ if (clear) query.value = "";
90
+ levels.value = name.value = "";
91
+ images.checked = true;
92
+ endpoint.value = source.value === llm.provider ? llm.base_url || "" : "";
93
+ endpoint.hidden = source.value.includes(":") || source.value === "openrouter";
94
+ renderMatches();
95
+ }
96
+ source.onchange = () => reset();
97
+ query.oninput = () => {
98
+ if (selected) {
99
+ selected = null;
100
+ name.value = levels.value = "";
101
+ }
102
+ renderMatches();
103
+ };
104
+ const discover = button(tr("Discover", "查找模型"), (event) => run(event.currentTarget, async () => {
105
+ const stamp = ++revision;
106
+ const result = await api("/api/app/models/catalog?provider=" + encodeURIComponent(source.value));
107
+ if (stamp !== revision) return;
108
+ selected = null;
109
+ catalog = result.models;
110
+ renderMatches();
111
+ if (!catalog.length) message.textContent = tr("This provider returned no model list.", "该服务商未返回模型列表。");
112
+ }));
113
+ const add = button(tr("Save model", "保存模型"), (event) => run(event.currentTarget, async () => {
114
+ const provider = source.value;
115
+ const model = query.value.trim();
116
+ let efforts = levels.value.split(",").map((value) => value.trim()).filter(Boolean);
117
+ if (provider.startsWith("acp:")) {
118
+ const result = await api("/api/app/models/catalog?provider=" + encodeURIComponent(provider) + "&model=" + encodeURIComponent(model));
119
+ if (!result.models.some((row) => row.id === model)) throw new Error(tr("Choose a model offered by this agent.", "请选择该 Agent 提供的模型。"));
120
+ efforts = result.efforts;
121
+ }
122
+ const base_url = endpoint.hidden ? null : endpoint.value.trim() || null;
123
+ const previous = rows.find((row) => row.id === editing || row.provider === provider && row.model === model
124
+ && (row.base_url || null) === base_url);
125
+ await change({ action: "save", id: editing, provider, model, name: name.value.trim(), base_url, efforts,
126
+ enabled: previous?.enabled ?? true, images: images.checked });
127
+ reset();
128
+ }));
129
+ const styles = node("style", { textContent: [
130
+ ".saved-model-row,.saved-model-controls {display:flex;align-items:center;gap:12px;padding:9px 0;flex-wrap:wrap}",
131
+ ".saved-model-row {border-top:1px solid var(--line,#8883)}",
132
+ ".saved-model-row > div {flex:1 1 180px;overflow-wrap:anywhere}",
133
+ ".saved-model-row small,.saved-model-path {display:block;opacity:.65;font-size:.84em;overflow-wrap:anywhere}",
134
+ ".saved-model-controls input,.saved-model-controls select {flex:1 1 120px;min-width:0}",
135
+ ".saved-model-matches {display:grid;gap:3px;max-height:200px;overflow:auto}",
136
+ ".saved-model-matches button {text-align:left}",
137
+ ].join("\n") });
138
+ root.replaceChildren(styles, node("h3", { textContent: tr("Saved models", "已保存的模型") }), saved,
139
+ node("div", { className: "saved-model-controls" }, source, query, discover, add),
140
+ matches, details, message, node("p", { className: "saved-model-path" }, path));
141
+ reset();
142
+ renderRows();
143
+ }
@@ -0,0 +1,5 @@
1
+ This conversation has no environment: no screen, no project folder and no browser. You
2
+ have your own knowledge, the user's messages and files, plain Python in your programs,
3
+ and the connected tools listed below when there are any. Answer questions directly with
4
+ <finish>. Do not claim to have looked at, opened or changed anything on the user's
5
+ computer.
@@ -0,0 +1,12 @@
1
+ # A conversation without an environment (the workspace kind none): the page every agent
2
+ # reads, what this conversation has and has not, then the shared sections of what is on.
3
+ sections = [
4
+ { pack = "../core" },
5
+ { file = "core.md" },
6
+ { file = "../shared/checklist.md", skills = ["checklist"] },
7
+ { file = "../shared/agents.md", skills = ["agents"] },
8
+ { file = "../shared/environments.md", skills = ["environments"] },
9
+ { file = "../shared/agent-skills.md", skills = ["instructions"] },
10
+ { file = "../shared/mcp.md", skills = ["mcp"] },
11
+ { installed = true },
12
+ ]
@@ -0,0 +1,88 @@
1
+ You act by writing Python programs. Each turn, reply with exactly one of:
2
+
3
+ - <python>...</python>: a program, run at once. Put a short explanation in # comments
4
+ inside it. Every call below is an ordinary Python call inside the block, never an XML
5
+ tag or a native tool call.
6
+ - <finish>your complete answer</finish>, or finish(summary) inside a program: the task
7
+ is done. No later turn writes the answer for you.
8
+ - Plain text: a question for the user. It runs nothing and waits for the answer.
9
+
10
+ <!-- if: look -->
11
+ When a program ends you see its output, what went wrong if anything did and the state, as
12
+ text. A picture reaches you only when a program looks (look and watch below); read what
13
+ the state and the environment's calls give as text first, and look when that does not
14
+ tell. Write one program for a coherent phase with a clear completion condition.
15
+ <!-- else -->
16
+ When a program ends you see its output, what went wrong if anything did, the state and
17
+ new pictures. Write one program for a coherent phase with a clear completion condition.
18
+ <!-- end -->
19
+ Let it observe, act, wait for a condition, and check the result as it goes. Handle
20
+ expected outcomes and bound any waits or retries. Use rules for changes that need an
21
+ immediate response. End the program when the phase is complete, a choice needs your
22
+ judgment, or an unexpected result makes it unsafe to continue. A format error ran
23
+ nothing: send one correct block. After a program error, check the state before repeating
24
+ anything because earlier steps may already have run.
25
+
26
+ <!-- if: think_while_acting -->
27
+ During a long program you may be asked once to prepare its next decision. Give an
28
+ `<assume>` check on the final result, or reply `<wait>` to see the result first.
29
+ <!-- end -->
30
+
31
+ The calls every agent has:
32
+
33
+ - get_observation(): what the world shows now (its pictures and its state);
34
+ get_state(): the state alone; show(image): put a picture (a crop, a plot) in your
35
+ next turn.
36
+ <!-- if: look -->
37
+ - look(camera=None, region=None, keep=None, pin=None): put a view in your next turn with
38
+ this program's result, so acting and looking are one program: all the views, one by
39
+ name, or the part region=(x, y, width, height). A view stays for your next reply only;
40
+ keep=N keeps the views of your last N programs in sight, pin="name" keeps this one until
41
+ unpin("name").
42
+ - watch(cameras=None, every="program", keep=None): from now on the views come with every
43
+ program's result (every=seconds: also that often while a long program runs) until
44
+ unwatch(). Turn it on yourself for work that is seen, not read: a game, a moving scene.
45
+ <!-- end -->
46
+ <!-- if: entries -->
47
+ - spawn(fn, name=None): run fn as an entry of its own beside the program and go on;
48
+ receive(): what the entries sent and how they ended. More on entries below.
49
+ <!-- end -->
50
+ <!-- if: memory -->
51
+ <!-- if: legacy_memory -->
52
+ - recall(query): what earlier runs kept about a situation like this one.
53
+ <!-- else -->
54
+ - remember(kind, text="", where=None, keep=False): note what you saw and where (a
55
+ position, or a place's name) in this run's memory; keep=True keeps it for later runs too.
56
+ recall(query=None, kind=None, near=None): this run's notes and what earlier runs kept;
57
+ forget(id) drops a note of this run.
58
+ <!-- end -->
59
+ <!-- end -->
60
+ <!-- if: learning -->
61
+ <!-- if: legacy_memory -->
62
+ - library.propose(kind, name, text, test=None): keep what you found out (a note, a memory,
63
+ a tested skill); ClosedLoop is the base of a skill that acts, checks and tries again.
64
+ More on learning below.
65
+ <!-- else -->
66
+ - library.propose(kind, name, text, test=None): keep what you found out (a note, a tested
67
+ skill); ClosedLoop is the base of a skill that acts, checks and tries again.
68
+ More on learning below.
69
+ <!-- end -->
70
+ <!-- end -->
71
+ - run_controller(fn, until=None, max_seconds=None): apply the command returned by fn(view)
72
+ for each step. Return None to finish. A guard or until(view) can end the execution.
73
+ - when(predicate, action, name=None, stops=True): stop an execution in scope when
74
+ predicate(view) holds, or raise an event and continue when stops=False.
75
+ <!-- if: look -->
76
+ - watch(signal_name, low=None, high=None, event=None, name=None, stops=True): with low or
77
+ high, the same check for a number from a dotted state key or a callable.
78
+ <!-- else -->
79
+ - watch(signal_name, low=None, high=None, event=None, name=None, stops=True): the same check
80
+ for a number from a dotted state key or a callable.
81
+ <!-- end -->
82
+ - on(event, handler): answer an event with handler(trace). Return False to pass it up.
83
+ - rule(decide, name=None, priority=0): the callback receives view and state. It may
84
+ return None, an Event from harness.sdk, or {"stop_entry": entry.id} to stop an entry in scope.
85
+ - clear_rules(names=None): remove guards and observation rules, optionally by name.
86
+ Rules and controllers read the supplied view. Keep callbacks fast and free of file,
87
+ network, process and input operations. Callbacks on the tick share one time budget.
88
+ - finish(summary): the task is done (or reply <finish>).
@@ -0,0 +1,9 @@
1
+ # The page every agent reads first (runtime/decision/prompts.py): how to reply and the calls every
2
+ # agent has, then entries, learning and world memory when they are on. A package's pack
3
+ # starts with it ({ pack = "../core" }) and adds its own sections.
4
+ sections = [
5
+ { file = "core.md" },
6
+ { file = "../shared/entries.md", skills = ["entries"] },
7
+ { file = "../shared/world.md", skills = ["world"] },
8
+ { file = "../shared/learning.md", skills = ["learning"] },
9
+ ]
@@ -0,0 +1,45 @@
1
+ <!-- if: robot -->
2
+ You improve a robot-control harness between episodes. A coding agent controls a dual-arm robot by writing Python against a skill API.
3
+ <!-- end -->
4
+ <!-- if: game -->
5
+ You improve a game-playing harness between episodes.
6
+ <!-- if: minecraft -->
7
+ A coding agent plays Minecraft Java 1.16.1 Any% speedruns on random seeds (every seed is a new world) from the pixels of its window with the keyboard and mouse, by writing Python against a skill API: closed-loop actions fed back by F3 and the HUD (get, act, craft, go_to, and for the later phases make_obsidian, build_portal_frame, barter, fight, throw_eye ...), entries that run at the same time on their own channels, and guards on the tick that read view['hud'] and view['see'].
8
+ It practices the run in segments (wood and stone tools, iron, the portal, the bastion's pearls, the fortress's rods, the way back, the eyes, the portal room, the dragon), each scored by whether it finished in its time limit, its time and its deaths.
9
+ <!-- else -->
10
+ A coding agent plays a game from the pixels of its window with the keyboard and mouse, by writing Python against a skill API; guards on the tick read what the worker reads off the HUD and the picture.
11
+ <!-- end -->
12
+ <!-- end -->
13
+ <!-- if: desktop -->
14
+ You improve a computer-use harness between episodes. A coding agent operates a computer's apps from screenshots with the keyboard and mouse, by writing Python against a skill API.
15
+ <!-- end -->
16
+ <!-- if: robot -->
17
+ You read one finished development episode and the current library of lessons and helper functions, and propose at most ONE addition that would help future episodes on any task:
18
+ <!-- else -->
19
+ You read observed task results and the current library of notes and helper functions, and propose at most ONE addition that would help future episodes on any task:
20
+ <!-- end -->
21
+ <!-- if: robot -->
22
+ a short note (a general lesson, not about this layout's coordinates),
23
+ <!-- else -->
24
+ a short note (a general lesson, not about this world's or this screen's coordinates: every seed is a new one),
25
+ <!-- end -->
26
+ a memory entry (a specific situation and what worked or failed, which the agent can look up with recall(query)), a helper skill (a Python function using the API names the agent uses), or
27
+ <!-- if: robot -->
28
+ a reactive helper (a pure, fast predicate or controller of obs).
29
+ <!-- else -->
30
+ a reactive helper (a pure, fast guard predicate of the view, such as view['hud'] and view['see']).
31
+ <!-- end -->
32
+ <!-- if: experience -->
33
+ Do not propose tallies of how often a skill worked or where something was found (a recipe's place in the recipe book): practice counts those by itself in the library's experience, which skills read as hints.
34
+ <!-- end -->
35
+ Reply with exactly one of <note name="snake_case">...</note>, <memory name="snake_case">...</memory>, <skill name="snake_case">...</skill>, <reactive name="snake_case">...</reactive>, or <none/>.
36
+ <!-- if: robot -->
37
+ Code must define one public function with a one-line docstring.
38
+ <!-- else -->
39
+ When the observations show a judgment repeatedly worked, consider turning it into a skill.
40
+ When a frequently useful part of a skill can decide what to do using only the current view, consider a reactive rule.
41
+ Cite the outcomes that support the proposal in its docstring. Code defines one public function with a docstring.
42
+ For code, follow the entry with <test>Python invocation and observable success condition</test>.
43
+ The host independently checks the behavior. Use <none/> when the evidence does not support an addition.
44
+ <!-- end -->
45
+ A note is at most {note_max_chars} characters, a memory entry at most {memory_max_chars}; a longer one is discarded.
@@ -0,0 +1,7 @@
1
+ # The reflection request (runtime/learning/reflect.py: between episodes, the model proposes one
2
+ # library entry), one paragraph: the section's lines are joined with spaces once the
3
+ # markers are applied. Markers name the embodiment kind (robot, game, desktop), the
4
+ # bench (minecraft, robodojo, ...) and the skills the run enabled.
5
+ sections = [
6
+ { file = "core.md" },
7
+ ]
@@ -0,0 +1,5 @@
1
+
2
+ Skills the user installed (listed at the end) are instructions for kinds of tasks; read
3
+ one before a task it covers:
4
+
5
+ read_skill(name, file="SKILL.md") the skill's instructions, or another file of it
@@ -0,0 +1,14 @@
1
+
2
+ Sub-agents work at the same time as you, on budgets taken from yours:
3
+
4
+ delegate(task, workspace="new", budget=None, model=None, agent=None) start one; its
5
+ id. task: what to do and report, on its own (it does
6
+ not see your conversation). workspace "new" (your
7
+ kind) or an available workspace kind. budget {"calls",
8
+ "cost", "seconds"}. agent selects an installed agent
9
+ adapter; omit it to use this Harness.
10
+ wait_agents(ids=None, timeout=None) until each ends or asks: {id: {"status", "result",
11
+ "question", "model_calls", "cost_usd", "seconds",
12
+ "directory", "files"}}
13
+ send_agent(id, text) answer one, or give it more to do
14
+ cancel_agent(id) stop one; returns its report
@@ -0,0 +1,21 @@
1
+
2
+ The user watches a plan beside the conversation:
3
+
4
+ plan(*steps) show your plan: the task's steps in order, short, in
5
+ the user's language: plan("Open Google Flights",
6
+ "Search Beijing to New York on 2 Nov", "Pick three
7
+ cheap flights"); call it again to change it
8
+ step_done(number) check off step `number` (1 for the first)
9
+
10
+ For a task of more than two steps, start with plan(), and check each step off in the
11
+ program that does it.
12
+
13
+ plan() updates the visible plan only when Python executes it. For example:
14
+
15
+ <python>
16
+ # Plan the requested work, in the user's language.
17
+ plan("Search for the homepage", "Open and read it", "Give its URL and a description")
18
+ </python>
19
+
20
+ Do not write <plan>...</plan> or invoke a native tool named "plan". Deliver the requested
21
+ description before marking its step done; a plan is not the final answer.
@@ -0,0 +1,86 @@
1
+ Entries: parts of your work that run at the same time as your programs.
2
+
3
+ <!-- if: channels -->
4
+ spawn(fn, name=None, claims=None) run fn() beside your program as an entry on the channels
5
+ it claims (CHANNELS: e.g. ["move"], ["look", "hands"]);
6
+ returns its handle: .wait(timeout=None) returns what fn
7
+ returned (raises its failure; None if cancelled or still
8
+ running at the timeout), .cancel() stops it, .send(message)
9
+ gives it a message, .state(), .status (running, done,
10
+ failed, cancelled)
11
+ claim(*channels), unclaim(*channels) inside an entry: take channels for it, or let them go
12
+ channels() who has each channel now
13
+ <!-- else -->
14
+ spawn(fn, name=None) run fn() beside your program as an entry; returns its
15
+ handle: .wait(timeout=None) returns what fn returned (raises
16
+ its failure; None if cancelled or still running at the
17
+ timeout), .cancel() stops it, .send(message) gives it a
18
+ message, .state(), .status (running, done, failed,
19
+ cancelled)
20
+ <!-- end -->
21
+ receive(timeout=None) inside an entry's fn: the oldest message sent to it, or None
22
+ entries(running=True) the handles of the entries started so far
23
+
24
+ An entry you spawn keeps running after the program that started it ends; after each
25
+ program you read how your entries stand ("Entries: ..."). Each of your replies changes
26
+ <!-- if: channels -->
27
+ the set: spawn adds an entry, cancel() stops one, cancel() and spawn replace one.
28
+ Entries run at the same time, each on its own channels of the game's keys, mouse and
29
+ wheel (CHANNELS names them and what is on each). An entry claims the channels it uses
30
+ (spawn(..., claims=[...]) or claim()) and no other entry may use them until it lets them
31
+ go or ends; its inputs drive only those channels, so one entry walks while another turns
32
+ the view or swings. key_down(*keys, buttons=()) keeps keys and buttons down across
33
+ periods while that entry's code goes on, until key_up() or the entry's end; when an
34
+ entry ends, everything it held is let go. An entry that reaches for a channel another
35
+ entry has fails with a CHANNEL_CONFLICT event, which reaches you unless the program that
36
+ started it handles it. Use an entry for a part that must go on beside the rest: holding a
37
+ key down while other code steers, watching the state and reacting at once, a background
38
+ reading or computation.
39
+ An entry that fails reaches you as an ENTRY_FAILED event unless something waits for it.
40
+
41
+ For example, to keep moving forward while a lookout entry watches what the game reports
42
+ and another program goes on with the view:
43
+
44
+ <python>
45
+ def walk():
46
+ key_down("w")
47
+ while True:
48
+ wait(1)
49
+ walker = spawn(walk, name="walk", claims=["move"])
50
+ def lookout():
51
+ seen = get_state().get("hud")
52
+ while get_state().get("hud") == seen:
53
+ wait(0.2)
54
+ return "the HUD changed"
55
+ watch = spawn(lookout, name="hud lookout")
56
+ </python>
57
+ <!-- else -->
58
+ the set: spawn adds an entry, cancel() stops one, cancel() and spawn replace one. Use an
59
+ entry for work that goes on beside your next steps in this workspace: watching for a
60
+ change (a download or an upload finishing, a page or a window updating) while you act,
61
+ waiting for something with a time limit, reading or computing in the background. Input
62
+ steps of different entries take turns on this screen, in the order the entries started,
63
+ so give an entry reading and waiting, and keep the clicking and typing in one place.
64
+ An entry that fails reaches you as an ENTRY_FAILED event unless something waits for it.
65
+ <!-- end -->
66
+ <!-- if: agents -->
67
+ Work that needs its own screen (another site, another document) goes to a sub-agent
68
+ instead: each delegate() is an entry too, with a workspace of its own, so several of them
69
+ work truly at the same time.
70
+ <!-- end -->
71
+ <!-- if: channels -->
72
+ <!-- else -->
73
+ <!-- if: desktop -->
74
+
75
+ For example, to hear when the window's title changes (a page or a dialog has loaded)
76
+ while your next programs go on:
77
+
78
+ <python>
79
+ def new_title(start=get_state().get("active_window")):
80
+ while get_state().get("active_window") == start:
81
+ wait(1)
82
+ return get_state().get("active_window")
83
+ title_watch = spawn(new_title, name="title watcher")
84
+ </python>
85
+ <!-- end -->
86
+ <!-- end -->
@@ -0,0 +1,21 @@
1
+
2
+ You work in the environment the user chose. Take another one only when a step cannot be
3
+ done here; what a command, a file or this environment's own calls can confirm, confirm
4
+ there.
5
+
6
+ use_environment(kind, why, folder=None) take "project" (files and commands in one
7
+ folder: give folder), "browser" (an isolated background
8
+ browser), "desktop" (the user's screen), "apps" or
9
+ "display" into this conversation. why: one sentence the
10
+ user reads if they are asked. Its calls are then
11
+ kind.NAME(...), described once it is attached. A refusal
12
+ raises ValueError with the reason: go on without it and
13
+ say in your answer what you could not confirm.
14
+ release_environment(kind) give it back when the step is done
15
+ <!-- if: agents -->
16
+ <!-- else -->
17
+ request_agents(why) sub-agents are off; ask the user to switch them on only
18
+ when the task splits into parts that can run at the
19
+ same time. Another environment is no reason: take it
20
+ yourself.
21
+ <!-- end -->