@percepteye/agent-flywheel 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +368 -0
- package/cordis.patch.yml +11 -0
- package/openclaw.plugin.json +134 -0
- package/package.json +67 -0
- package/schema/flywheel-1.json +432 -0
- package/src/capture.js +733 -0
- package/src/classify.js +115 -0
- package/src/config.js +249 -0
- package/src/describe.js +355 -0
- package/src/dsh-classify.js +110 -0
- package/src/dsh.js +130 -0
- package/src/errors.js +37 -0
- package/src/evidence.js +109 -0
- package/src/execution-identity.js +444 -0
- package/src/host.js +63 -0
- package/src/http.js +249 -0
- package/src/index.js +380 -0
- package/src/mode.js +152 -0
- package/src/model-calls.js +214 -0
- package/src/policy.js +934 -0
- package/src/record.js +83 -0
- package/src/rollout.js +884 -0
- package/src/scope.js +242 -0
- package/src/session.js +42 -0
- package/src/trajectory.js +148 -0
- package/src/transport.js +403 -0
- package/src/turns.js +437 -0
- package/src/unattended.js +251 -0
- package/src/wire.js +182 -0
package/src/describe.js
ADDED
|
@@ -0,0 +1,355 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* The agent describer: what this agent IS, observed from inside the host.
|
|
3
|
+
*
|
|
4
|
+
* WHY THIS EXISTS. The Python SDK discovers an agent by reflecting on a
|
|
5
|
+
* constructed object. That works for an in-process Python agent and cannot
|
|
6
|
+
* work for a CLI one: `fw.serve()` given a command entrypoint spawns an OS
|
|
7
|
+
* subprocess, so `runner.resolved` is None, `introspect(None)` returns None,
|
|
8
|
+
* and the control plane records `introspection_state = "unreadable"`. No tool
|
|
9
|
+
* catalogue means no generated workflows, which means no evaluation suites,
|
|
10
|
+
* no graded rollouts, and no dataset. The whole flywheel is downstream of this
|
|
11
|
+
* one fact, and reflection cannot cross a process AND a language boundary.
|
|
12
|
+
*
|
|
13
|
+
* WHY THIS HOOK. `llm_input` offers the post-policy provider/model and the
|
|
14
|
+
* assembled system prompt without exposing a mutation seam:
|
|
15
|
+
*
|
|
16
|
+
* systemPrompt the FULLY ASSEMBLED prompt -- the host's several prompt
|
|
17
|
+
* files, concatenated inside the Node process, from sources
|
|
18
|
+
* the Python SDK never opens.
|
|
19
|
+
* Reading those files ourselves would re-implement the host's
|
|
20
|
+
* assembly order and drift from it silently.
|
|
21
|
+
* provider,model the model config, resolved after all overrides.
|
|
22
|
+
*
|
|
23
|
+
* IMPORTANT ABSENCE. OpenClaw's current PluginHookLlmInputEvent does NOT
|
|
24
|
+
* contain tools. Older code and tests invented that field and claimed an exact
|
|
25
|
+
* catalogue the real host could never supply. `buildRecord` still accepts an
|
|
26
|
+
* adapter-augmented `tools` array for forward compatibility, but the current
|
|
27
|
+
* host honestly writes `tool_definitions:null` plus an incomplete marker.
|
|
28
|
+
*
|
|
29
|
+
* WHY *THIS FILE* OBSERVES ONLY. The sibling recorder chose `after_tool_call`
|
|
30
|
+
* over `registerAgentToolResultMiddleware` on one rule: capturing evidence
|
|
31
|
+
* must never be able to change the behaviour being measured. `llm_input` keeps
|
|
32
|
+
* that rule. It is typed `(event, ctx) => Promise<void> | void` -- the same
|
|
33
|
+
* structurally non-mutating class as `after_tool_call`. The host discards our
|
|
34
|
+
* return value, so there is no seam here through which a description could
|
|
35
|
+
* alter a prompt, a tool result, or a routing decision. Introspection is a
|
|
36
|
+
* READ.
|
|
37
|
+
*
|
|
38
|
+
* THAT IS A CLAIM ABOUT THIS FILE, NOT ABOUT THE PACKAGE, and it used to read
|
|
39
|
+
* as both: "the mutation seams OpenClaw does expose -- `before_prompt_build`'s
|
|
40
|
+
* `{systemPrompt}`, `before_tool_call`'s `{block:true}` -- remain unused by
|
|
41
|
+
* this observer,
|
|
42
|
+
* deliberately." Half of that is no longer true. `policy.js` DOES use
|
|
43
|
+
* `before_prompt_build`, in production mode, to apply the system prompt the
|
|
44
|
+
* control plane approved -- and `before_model_resolve` plus a registered
|
|
45
|
+
* provider to point the agent at the model that prompt was certified with.
|
|
46
|
+
* Both deliberately, both gated on a pair check and a per-agent scope, both
|
|
47
|
+
* switchable off. Training mode now uses `before_tool_call` only as a lease
|
|
48
|
+
* revocation fence: after the control plane has requeued a rollout, later
|
|
49
|
+
* calls from the stale run are blocked from producing duplicate side effects.
|
|
50
|
+
* The describer is unaffected either way: a describer that could rewrite a
|
|
51
|
+
* prompt would be measuring its own output.
|
|
52
|
+
*
|
|
53
|
+
* The old wording also invited a wrong reading of that seam. `systemPrompt` is
|
|
54
|
+
* a field on the hook's RESULT, not on its EVENT: the event is
|
|
55
|
+
* `{prompt, messages}` and writing to it does nothing at all. `policy.js`
|
|
56
|
+
* carries the verified mechanics.
|
|
57
|
+
*
|
|
58
|
+
* THE ONE THING THIS HOOK COSTS. `llm_input` is a CONVERSATION hook
|
|
59
|
+
* (`CONVERSATION_HOOK_NAMES`, dist/hook-types-DQ9eTy2x.d.ts:386). The host
|
|
60
|
+
* REFUSES to register it for a non-bundled plugin unless the operator opts in
|
|
61
|
+
* with
|
|
62
|
+
* `plugins.entries.agent-flywheel.hooks.allowConversationAccess=true`
|
|
63
|
+
* (dist/registry-B8eQDFB4.js:4226-4241). That refusal is a warn-level
|
|
64
|
+
* diagnostic, not a throw, so a missing opt-in looks exactly like a working
|
|
65
|
+
* install that never fires. `register()` below says so out loud at startup
|
|
66
|
+
* rather than letting the operator discover it as an empty catalogue an hour
|
|
67
|
+
* later.
|
|
68
|
+
*
|
|
69
|
+
* ⚠ THAT GATE IS VERSION-SPECIFIC, verified by reading the INSTALLED OpenClaw
|
|
70
|
+
* 2026.7.1-2 -- the version `package.json`'s `openclaw.compat.pluginApi`
|
|
71
|
+
* requires. It is not universal: OpenClaw 2026.4.3 contains no
|
|
72
|
+
* `allowConversationAccess` and no `CONVERSATION_HOOK_NAMES` anywhere, so on
|
|
73
|
+
* that host `llm_input` is ungated and the setting the startup log asks for
|
|
74
|
+
* does not exist. Printing the line anyway is still right -- the failure it
|
|
75
|
+
* explains is otherwise silent, and a setting an older host ignores costs
|
|
76
|
+
* nothing -- but if the catalogue stays empty on a host where the opt-in is
|
|
77
|
+
* already set, this caveat is where to start: the cause is somewhere else and
|
|
78
|
+
* has not been found.
|
|
79
|
+
*
|
|
80
|
+
* Zero dependencies, `node:fs` only. This runs inside someone else's agent.
|
|
81
|
+
*/
|
|
82
|
+
import { renameSync, writeFileSync, mkdirSync } from "node:fs";
|
|
83
|
+
import { join } from "node:path";
|
|
84
|
+
|
|
85
|
+
import { TRAJECTORY_DIR_ENV } from "./trajectory.js";
|
|
86
|
+
|
|
87
|
+
export const DISCOVERED_AGENT_FILENAME = "discovered_agent.json";
|
|
88
|
+
export const DESCRIBE_HOOK_NAME = "llm_input";
|
|
89
|
+
|
|
90
|
+
/**
|
|
91
|
+
* `source_type` is a provenance claim, and it is NOT `sdk_runtime`.
|
|
92
|
+
*
|
|
93
|
+
* The Python probes mean something specific by `sdk_runtime`: the agent
|
|
94
|
+
* described itself by reflection over a constructed object. This description
|
|
95
|
+
* came from the HOST observing a real model call. Both are honest, they are
|
|
96
|
+
* not the same act, and only one of them can be wrong in the ways reflection
|
|
97
|
+
* is wrong. Naming it separately costs nothing -- the control plane stores
|
|
98
|
+
* `source_type` and has no reader for it, and the discovery consumer stamps
|
|
99
|
+
* its own source type when it builds its record rather than reading ours --
|
|
100
|
+
* and it means a future reader can tell the two apart without archaeology.
|
|
101
|
+
*
|
|
102
|
+
* Kept short on purpose: `flywheel_agents.source_type` is a varchar(64)
|
|
103
|
+
* written verbatim with no truncation, the exact column shape of a prior
|
|
104
|
+
* overflow that refused 100% of inserts.
|
|
105
|
+
*/
|
|
106
|
+
export const SOURCE_TYPE = "openclaw_llm_input";
|
|
107
|
+
|
|
108
|
+
/**
|
|
109
|
+
* `framework` is a LOOKUP KEY, not a label.
|
|
110
|
+
*
|
|
111
|
+
* The discovery consumer maps exactly `"openclaw"` to its OpenClaw
|
|
112
|
+
* framework. Any other spelling -- `open_claw`,
|
|
113
|
+
* `openclaw-cli`, the agent's own name -- silently degrades to
|
|
114
|
+
* an UNKNOWN framework, which is logged and tolerated and loses the framework
|
|
115
|
+
* identity in every generated workflow's provenance. Same varchar(64).
|
|
116
|
+
*/
|
|
117
|
+
export const FRAMEWORK = "openclaw";
|
|
118
|
+
|
|
119
|
+
/**
|
|
120
|
+
* Postgres JSONB cannot store a NUL byte or a lone surrogate in any string:
|
|
121
|
+
* it raises `unsupported Unicode escape sequence`, which surfaces as an
|
|
122
|
+
* uncaught 500 on registration, 100% of the time, for that agent.
|
|
123
|
+
*
|
|
124
|
+
* This matters here specifically because the payload is not ours. The system
|
|
125
|
+
* prompt is a concatenation of customer-authored markdown files, and an
|
|
126
|
+
* adapter-augmented tool array is arbitrary provider-shaped JSON. A
|
|
127
|
+
* description that cannot be stored is worth less than one missing a field,
|
|
128
|
+
* so we scrub rather than let the registration fail.
|
|
129
|
+
*/
|
|
130
|
+
function scrub(value) {
|
|
131
|
+
if (typeof value !== "string") return value;
|
|
132
|
+
// eslint-disable-next-line no-control-regex
|
|
133
|
+
return value.replace(/\u0000/g, "").replace(/[\uD800-\uDFFF]/g, (ch, i, s) => {
|
|
134
|
+
const code = ch.charCodeAt(0);
|
|
135
|
+
const isHigh = code <= 0xdbff;
|
|
136
|
+
const partner = isHigh ? s.charCodeAt(i + 1) : s.charCodeAt(i - 1);
|
|
137
|
+
const paired = isHigh
|
|
138
|
+
? partner >= 0xdc00 && partner <= 0xdfff
|
|
139
|
+
: partner >= 0xd800 && partner <= 0xdbff;
|
|
140
|
+
return paired ? ch : "";
|
|
141
|
+
});
|
|
142
|
+
}
|
|
143
|
+
|
|
144
|
+
/** Recursively scrub every string in a JSON-ish value. */
|
|
145
|
+
function scrubDeep(value) {
|
|
146
|
+
if (typeof value === "string") return scrub(value);
|
|
147
|
+
if (Array.isArray(value)) return value.map(scrubDeep);
|
|
148
|
+
if (value && typeof value === "object") {
|
|
149
|
+
const out = {};
|
|
150
|
+
for (const [k, v] of Object.entries(value)) out[scrub(k)] = scrubDeep(v);
|
|
151
|
+
return out;
|
|
152
|
+
}
|
|
153
|
+
return value;
|
|
154
|
+
}
|
|
155
|
+
|
|
156
|
+
/**
|
|
157
|
+
* Normalise ONE host tool object into the discovery protocol's tool shape.
|
|
158
|
+
*
|
|
159
|
+
* This is the load-bearing translation, and it is load-bearing because the
|
|
160
|
+
* consumer is exact and silent. The consumer reads `input_schema` and NOTHING else: it does not accept the
|
|
161
|
+
* host's native
|
|
162
|
+
* `parameters` key, and it does not unwrap the OpenAI
|
|
163
|
+
* `{type:"function", function:{...}}` envelope. Both failures are silent and
|
|
164
|
+
* differently shaped -- `parameters` yields a tool with an EMPTY schema,
|
|
165
|
+
* while the OpenAI envelope yields a tool with NO NAME, which the consumer
|
|
166
|
+
* then drops entirely. Forward `event.tools` verbatim and the launch gate still
|
|
167
|
+
* opens (the list is truthy) while the catalogue is empty or nameless.
|
|
168
|
+
*
|
|
169
|
+
* An optional adapter may supply any of these common shapes, so handle all
|
|
170
|
+
* three rather than betting on one provider envelope.
|
|
171
|
+
*
|
|
172
|
+
* @returns {{name:string,description:string,input_schema:object,original_format:string}|null}
|
|
173
|
+
*/
|
|
174
|
+
export function normalizeTool(raw) {
|
|
175
|
+
if (!raw || typeof raw !== "object") return null;
|
|
176
|
+
|
|
177
|
+
// The OpenAI function-calling envelope: the real tool is one level down.
|
|
178
|
+
const isEnvelope =
|
|
179
|
+
raw.type === "function" && raw.function && typeof raw.function === "object";
|
|
180
|
+
const t = isEnvelope ? raw.function : raw;
|
|
181
|
+
|
|
182
|
+
const name = typeof t.name === "string" ? t.name.trim() : "";
|
|
183
|
+
// The consumer drops a nameless tool anyway; dropping it here means the count we
|
|
184
|
+
// report is the count that survives, so "7 tools" never means "7 sent, 3
|
|
185
|
+
// arrived".
|
|
186
|
+
if (!name) return null;
|
|
187
|
+
|
|
188
|
+
const schema =
|
|
189
|
+
t.input_schema ?? t.inputSchema ?? t.parameters ?? t.schema ?? null;
|
|
190
|
+
|
|
191
|
+
return {
|
|
192
|
+
name: scrub(name),
|
|
193
|
+
description: scrub(
|
|
194
|
+
typeof t.description === "string" ? t.description
|
|
195
|
+
: typeof t.label === "string" ? t.label
|
|
196
|
+
: "",
|
|
197
|
+
),
|
|
198
|
+
// `{type:"object",properties:{}}` is what the consumer substitutes for a falsy
|
|
199
|
+
// schema. Emitting it here keeps the wire record self-describing rather
|
|
200
|
+
// than relying on a default two repos away.
|
|
201
|
+
input_schema:
|
|
202
|
+
schema && typeof schema === "object"
|
|
203
|
+
? scrubDeep(schema)
|
|
204
|
+
: { type: "object", properties: {} },
|
|
205
|
+
original_format: isEnvelope ? "openai_function" : "openclaw_tool",
|
|
206
|
+
};
|
|
207
|
+
}
|
|
208
|
+
|
|
209
|
+
/** Normalise the host's tool array. `undefined` in means `null` out. */
|
|
210
|
+
export function normalizeTools(rawTools) {
|
|
211
|
+
// THE ABSENCE CONTRACT, preserved across the process boundary. `null` means
|
|
212
|
+
// we could not look; `[]` means we looked and the agent has no tools. The
|
|
213
|
+
// consumer stamps a `not observed` marker on the first and grades on the
|
|
214
|
+
// difference, and the control plane's launch gate refuses a run whose agent
|
|
215
|
+
// reports zero tools. Collapsing these two into `[]` would turn "we did not
|
|
216
|
+
// observe" into a confident claim that the agent can do nothing.
|
|
217
|
+
if (!Array.isArray(rawTools)) return null;
|
|
218
|
+
const out = [];
|
|
219
|
+
for (const raw of rawTools) {
|
|
220
|
+
const tool = normalizeTool(raw);
|
|
221
|
+
if (tool) out.push(tool);
|
|
222
|
+
}
|
|
223
|
+
return out;
|
|
224
|
+
}
|
|
225
|
+
|
|
226
|
+
/**
|
|
227
|
+
* Build the wire record from one `llm_input` event.
|
|
228
|
+
*
|
|
229
|
+
* Field-for-field the shape of `DiscoveredAgent.to_wire()` in the Python SDK,
|
|
230
|
+
* because that is what the control plane stores and reads. The two
|
|
231
|
+
* are one ABI; this is the Node end of it, exactly as `trajectory.js` is the
|
|
232
|
+
* Node end of the tool-outcome ABI.
|
|
233
|
+
*/
|
|
234
|
+
export function buildRecord(event, ctx = {}) {
|
|
235
|
+
const tools = normalizeTools(event?.tools);
|
|
236
|
+
const record = {
|
|
237
|
+
agent_name: scrub(String(ctx.agentId || event?.sessionId || "openclaw-agent")),
|
|
238
|
+
framework: FRAMEWORK,
|
|
239
|
+
source_type: SOURCE_TYPE,
|
|
240
|
+
role: "standalone",
|
|
241
|
+
tool_definitions: tools,
|
|
242
|
+
};
|
|
243
|
+
|
|
244
|
+
if (typeof event?.systemPrompt === "string" && event.systemPrompt) {
|
|
245
|
+
record.system_prompt = scrub(event.systemPrompt);
|
|
246
|
+
}
|
|
247
|
+
if (event?.provider || event?.model) {
|
|
248
|
+
record.discovered_model_config = {
|
|
249
|
+
provider: scrub(String(event.provider || "unknown")),
|
|
250
|
+
model_name: scrub(String(event.model || "unknown")),
|
|
251
|
+
};
|
|
252
|
+
}
|
|
253
|
+
|
|
254
|
+
// `incomplete` names the parts we could not read, so a consumer can tell a
|
|
255
|
+
// gap from an absence. The host declares `systemPrompt` and `tools`
|
|
256
|
+
// optional, so either can genuinely be missing from a real event.
|
|
257
|
+
const incomplete = [];
|
|
258
|
+
if (!record.system_prompt) incomplete.push("system_prompt");
|
|
259
|
+
if (tools === null) incomplete.push("tool_definitions");
|
|
260
|
+
if (incomplete.length) record.incomplete = incomplete;
|
|
261
|
+
|
|
262
|
+
return record;
|
|
263
|
+
}
|
|
264
|
+
|
|
265
|
+
/**
|
|
266
|
+
* A describer bound to a directory, or an inert one when none is set.
|
|
267
|
+
*
|
|
268
|
+
* WRITE POLICY. `llm_input` fires at each host run's assembled model boundary.
|
|
269
|
+
* If a host adapter supplies tools, its universe may vary across runs: a
|
|
270
|
+
* sub-agent turn can carry a subset. So we keep the richest catalogue rather than the
|
|
271
|
+
* newest -- last-write-wins would let a narrow sub-agent turn overwrite the
|
|
272
|
+
* agent's real catalogue with a fraction of it, and the result would look
|
|
273
|
+
* perfectly healthy at every layer downstream.
|
|
274
|
+
*
|
|
275
|
+
* The first event always writes, so an agent that genuinely has no tools
|
|
276
|
+
* still produces a file and still reaches `described`.
|
|
277
|
+
*/
|
|
278
|
+
export function createDescriber({ dir = null, onError = null } = {}) {
|
|
279
|
+
let best = -1;
|
|
280
|
+
let warned = false;
|
|
281
|
+
|
|
282
|
+
const resolveDir = () => dir ?? process.env[TRAJECTORY_DIR_ENV] ?? null;
|
|
283
|
+
|
|
284
|
+
return {
|
|
285
|
+
get active() {
|
|
286
|
+
return Boolean(resolveDir());
|
|
287
|
+
},
|
|
288
|
+
/** @returns {boolean} whether this event was written. */
|
|
289
|
+
handle(event, ctx = {}) {
|
|
290
|
+
const base = resolveDir();
|
|
291
|
+
if (!base) return false;
|
|
292
|
+
|
|
293
|
+
const record = buildRecord(event, ctx);
|
|
294
|
+
const count = record.tool_definitions?.length ?? 0;
|
|
295
|
+
if (best >= 0 && count <= best) return false;
|
|
296
|
+
|
|
297
|
+
const path = join(base, DISCOVERED_AGENT_FILENAME);
|
|
298
|
+
const tmp = `${path}.${process.pid}.tmp`;
|
|
299
|
+
try {
|
|
300
|
+
mkdirSync(base, { recursive: true });
|
|
301
|
+
// Write-then-rename. The Python SDK POLLS for this file while the
|
|
302
|
+
// warm-up child runs, so a reader can arrive mid-write; rename is
|
|
303
|
+
// atomic within a filesystem, so the reader sees either no file or a
|
|
304
|
+
// complete one, never half a JSON object.
|
|
305
|
+
writeFileSync(tmp, JSON.stringify(record), "utf8");
|
|
306
|
+
renameSync(tmp, path);
|
|
307
|
+
best = count;
|
|
308
|
+
return true;
|
|
309
|
+
} catch (err) {
|
|
310
|
+
// Never fail an agent over telemetry. Report once, then stay quiet.
|
|
311
|
+
if (!warned) {
|
|
312
|
+
warned = true;
|
|
313
|
+
if (onError) onError(err);
|
|
314
|
+
}
|
|
315
|
+
return false;
|
|
316
|
+
}
|
|
317
|
+
},
|
|
318
|
+
};
|
|
319
|
+
}
|
|
320
|
+
|
|
321
|
+
/**
|
|
322
|
+
* Register the describer on an OpenClaw plugin api.
|
|
323
|
+
*
|
|
324
|
+
* Returns the describer, or null when the host has no `on()`. The
|
|
325
|
+
* conversation-access opt-in cannot be checked from here -- the host's
|
|
326
|
+
* refusal is a diagnostic on ITS side and our handler simply never fires --
|
|
327
|
+
* so the caller warns at startup instead.
|
|
328
|
+
*/
|
|
329
|
+
export function registerDescriber(api, { logger = null, onObservation = null } = {}) {
|
|
330
|
+
if (typeof api?.on !== "function") return null;
|
|
331
|
+
|
|
332
|
+
const describer = createDescriber({
|
|
333
|
+
onError: (err) => logger?.warn?.(
|
|
334
|
+
`[agent-flywheel] agent description write failed; this agent ` +
|
|
335
|
+
`will register as introspection_state=unreadable and no workflows will ` +
|
|
336
|
+
`be generated for it: ${err?.message ?? err}`,
|
|
337
|
+
),
|
|
338
|
+
});
|
|
339
|
+
|
|
340
|
+
api.on(DESCRIBE_HOOK_NAME, (event, ctx) => {
|
|
341
|
+
try {
|
|
342
|
+
onObservation?.(event, ctx ?? {});
|
|
343
|
+
} catch {
|
|
344
|
+
// Identity is an independent projection. It may abstain without costing
|
|
345
|
+
// the advisory description written below.
|
|
346
|
+
}
|
|
347
|
+
try {
|
|
348
|
+
describer.handle(event, ctx ?? {});
|
|
349
|
+
} catch {
|
|
350
|
+
/* one lost description is not worth an interrupted agent */
|
|
351
|
+
}
|
|
352
|
+
});
|
|
353
|
+
|
|
354
|
+
return describer;
|
|
355
|
+
}
|
|
@@ -0,0 +1,110 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Turning a DeepSeek Harness `tools/result` outcome into a flywheel outcome.
|
|
3
|
+
*
|
|
4
|
+
* dsh types its result as a discriminated union on `isError`, which is a real
|
|
5
|
+
* discriminant rather than an optional flag — so `isError: true` is a positive
|
|
6
|
+
* assertion of failure and can simply be believed.
|
|
7
|
+
*
|
|
8
|
+
* `isError: false` is the interesting half, and it does NOT mean success. The
|
|
9
|
+
* bash tool renders a failing exit as TEXT (`[exit code: 3]`) appended to the
|
|
10
|
+
* model-facing content and still settles as a non-error result: the tool call
|
|
11
|
+
* worked, the command did not. Reading the flag alone scores a failed command
|
|
12
|
+
* as a win — the same trap OpenClaw sets with `status: "completed"`.
|
|
13
|
+
*
|
|
14
|
+
* What makes dsh better to classify is that its exec tools declare a real
|
|
15
|
+
* output schema: `exitCode`, `signal`, `timedOut`, `aborted`, and a `kind`
|
|
16
|
+
* discriminating foreground from background. That is enough to be honest about
|
|
17
|
+
* three cases a cruder reading collapses into one.
|
|
18
|
+
*/
|
|
19
|
+
import { readExit } from "./classify.js";
|
|
20
|
+
|
|
21
|
+
/** dsh's exec tools, by their registered names. */
|
|
22
|
+
export const DSH_EXEC_TOOLS = new Set(["bash", "pwsh"]);
|
|
23
|
+
|
|
24
|
+
/**
|
|
25
|
+
* Tools whose silence really is success: no in-band status channel, so a
|
|
26
|
+
* failure has nowhere to go but `isError`. These are dsh's own registered
|
|
27
|
+
* names. Anything absent from both sets grades `unknown` until a fixture
|
|
28
|
+
* earns it a place.
|
|
29
|
+
*/
|
|
30
|
+
export const DSH_VERDICT_TOOLS = new Set([
|
|
31
|
+
"read", "write", "edit", "read_image", "str_replace_editor", "todo_write",
|
|
32
|
+
]);
|
|
33
|
+
|
|
34
|
+
/**
|
|
35
|
+
* @param {string} name the registered tool name
|
|
36
|
+
* @param {{isError?: boolean, value?: unknown, error?: {message?: string, info?: unknown}}} result
|
|
37
|
+
* @returns {{outcome:"ok"|"failed"|"unknown", status_code:null,
|
|
38
|
+
* error?:string, error_class?:string}}
|
|
39
|
+
*/
|
|
40
|
+
export function classifyDsh(name, result) {
|
|
41
|
+
const r = result && typeof result === "object" ? result : {};
|
|
42
|
+
|
|
43
|
+
// A positive assertion by the harness that the call failed.
|
|
44
|
+
if (r.isError === true) {
|
|
45
|
+
const failure = r.error && typeof r.error === "object" ? r.error : {};
|
|
46
|
+
const info = failure.info;
|
|
47
|
+
return {
|
|
48
|
+
outcome: "failed",
|
|
49
|
+
status_code: null,
|
|
50
|
+
error: typeof failure.message === "string"
|
|
51
|
+
? failure.message.slice(0, 1000)
|
|
52
|
+
: undefined,
|
|
53
|
+
// dsh carries an internal error class on `info`; it is far more useful
|
|
54
|
+
// downstream than a generic label when it is a plain string.
|
|
55
|
+
error_class: typeof info === "string" ? info
|
|
56
|
+
: (info && typeof info === "object" && typeof info.code === "string")
|
|
57
|
+
? info.code
|
|
58
|
+
: "ToolError",
|
|
59
|
+
};
|
|
60
|
+
}
|
|
61
|
+
|
|
62
|
+
// `isError` absent is not `isError: false`. The harness did not say.
|
|
63
|
+
if (r.isError !== false) return { outcome: "unknown", status_code: null };
|
|
64
|
+
|
|
65
|
+
const value = r.value;
|
|
66
|
+
const v = value && typeof value === "object" ? value : {};
|
|
67
|
+
|
|
68
|
+
if (DSH_EXEC_TOOLS.has(name)) {
|
|
69
|
+
// A backgrounded command has been LAUNCHED, not completed. There is no
|
|
70
|
+
// exit code because nothing has exited. Grading it `ok` would claim a
|
|
71
|
+
// result for work still running.
|
|
72
|
+
if (v.kind === "background") {
|
|
73
|
+
return { outcome: "unknown", status_code: null,
|
|
74
|
+
error_class: "StillRunning" };
|
|
75
|
+
}
|
|
76
|
+
// Cancelled by the caller. The command did not get to say what it would
|
|
77
|
+
// have done, and that is not the agent making a mistake.
|
|
78
|
+
if (v.aborted === true) {
|
|
79
|
+
return { outcome: "unknown", status_code: null, error_class: "Aborted" };
|
|
80
|
+
}
|
|
81
|
+
if (v.timedOut === true) {
|
|
82
|
+
return { outcome: "failed", status_code: null, error_class: "TimedOut" };
|
|
83
|
+
}
|
|
84
|
+
// Killed by a signal: `exitCode` is null precisely here, so this must be
|
|
85
|
+
// checked before treating a null exit as unreadable.
|
|
86
|
+
if (typeof v.signal === "string" && v.signal) {
|
|
87
|
+
return { outcome: "failed", status_code: null,
|
|
88
|
+
error_class: `Signal:${v.signal}` };
|
|
89
|
+
}
|
|
90
|
+
const exit = readExit(value);
|
|
91
|
+
if (exit === undefined) {
|
|
92
|
+
// We did not observe the verdict. Falling back to `ok` here is the
|
|
93
|
+
// rounding-up this module exists to prevent.
|
|
94
|
+
return { outcome: "unknown", status_code: null,
|
|
95
|
+
error_class: "ExitStatusUnreadable" };
|
|
96
|
+
}
|
|
97
|
+
if (exit === 0) return { outcome: "ok", status_code: null };
|
|
98
|
+
// An exit code is not an HTTP status; putting 127 in `status_code` would
|
|
99
|
+
// fail the contract's 100..599 check and would be a lie besides.
|
|
100
|
+
return { outcome: "failed", status_code: null,
|
|
101
|
+
error_class: `ExitStatus:${exit}` };
|
|
102
|
+
}
|
|
103
|
+
|
|
104
|
+
if (DSH_VERDICT_TOOLS.has(name)) return { outcome: "ok", status_code: null };
|
|
105
|
+
|
|
106
|
+
// Everything unproven: web tools, terminal session tools, subagent tools,
|
|
107
|
+
// and every tool a plugin registered. A tool that reports its own failure
|
|
108
|
+
// inside a successful-looking value is invisible from here.
|
|
109
|
+
return { outcome: "unknown", status_code: null };
|
|
110
|
+
}
|
package/src/dsh.js
ADDED
|
@@ -0,0 +1,130 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* agent-flywheel — a DeepSeek Harness (Cordis) plugin.
|
|
3
|
+
*
|
|
4
|
+
* Records tool-call outcomes into the active rollout's trajectory. Observes
|
|
5
|
+
* only, and here that is guaranteed by the harness rather than merely intended.
|
|
6
|
+
*
|
|
7
|
+
* WHY `tools/result`. dsh's tool pipeline exposes several seams, and most of
|
|
8
|
+
* them are waterfalls: `tools/execute` wraps dispatch, `tools/post` can accept,
|
|
9
|
+
* replace or BLOCK a result, and `tools/code-dispatch-log` rewrites what gets
|
|
10
|
+
* logged. A listener on any of those can change what the model sees.
|
|
11
|
+
*
|
|
12
|
+
* `tools/result` is declared `@mode emit` and typed to return `undefined`. Its
|
|
13
|
+
* documentation is explicit — "Observe the frozen, lossless-JSON final outcome.
|
|
14
|
+
* Listener failures are contained." — and the registry DEEP-FREEZES both the
|
|
15
|
+
* execution and the result before publishing them, with an invariant that
|
|
16
|
+
* fails the harness if it ever publishes something unfrozen. So this plugin
|
|
17
|
+
* cannot alter a tool result even by accident: there is no return channel, and
|
|
18
|
+
* the object it receives is immutable.
|
|
19
|
+
*
|
|
20
|
+
* That is the same rule the OpenClaw plugin follows for the same reason:
|
|
21
|
+
* capturing evidence must never be able to change the behaviour being
|
|
22
|
+
* measured. dsh simply enforces it harder.
|
|
23
|
+
*
|
|
24
|
+
* THIS LANE HAS NO ROLLOUT DRIVER, in any mode. The OpenClaw lane can claim a
|
|
25
|
+
* rollout and deliver its task into the agent's next turn (see `rollout.js`,
|
|
26
|
+
* training mode only); Cordis exposes no equivalent seam, so on dsh this
|
|
27
|
+
* plugin observes and nothing else regardless of `PERCEPTEYE_AGENT_MODE`. Said
|
|
28
|
+
* here because the README's mode table describes the package, and a reader who
|
|
29
|
+
* takes it to mean "dsh claims rollouts too" would wait for trajectories that
|
|
30
|
+
* are never going to arrive.
|
|
31
|
+
*
|
|
32
|
+
* Zero dependencies. `ctx` is supplied by the harness; nothing is imported
|
|
33
|
+
* from it, so this module loads even when dsh is absent — which is what lets
|
|
34
|
+
* one package serve two harnesses.
|
|
35
|
+
*/
|
|
36
|
+
import { classifyDsh } from "./dsh-classify.js";
|
|
37
|
+
import { extractEntityIds } from "./evidence.js";
|
|
38
|
+
import { createWriter, toWire, TRAJECTORY_DIR_ENV } from "./trajectory.js";
|
|
39
|
+
|
|
40
|
+
/** Display metadata; labels the plugin in Cordis diagnostics. */
|
|
41
|
+
export const name = "agent-flywheel";
|
|
42
|
+
|
|
43
|
+
export const HOOK_NAME = "tools/result";
|
|
44
|
+
|
|
45
|
+
/** The agent that made the call, or undefined when nothing names one. */
|
|
46
|
+
function agentName(exec) {
|
|
47
|
+
const agent = exec && typeof exec === "object" ? exec.agent : undefined;
|
|
48
|
+
if (!agent || typeof agent !== "object") return undefined;
|
|
49
|
+
for (const key of ["name", "id"]) {
|
|
50
|
+
const v = agent[key];
|
|
51
|
+
if (typeof v === "string" && v.trim()) return v;
|
|
52
|
+
}
|
|
53
|
+
return undefined;
|
|
54
|
+
}
|
|
55
|
+
|
|
56
|
+
/**
|
|
57
|
+
* Build the trajectory record for one settled execution.
|
|
58
|
+
*
|
|
59
|
+
* Exported for tests: the harness supplies frozen objects, and a recorder that
|
|
60
|
+
* can only be exercised through a live agent loop is a recorder nobody checks.
|
|
61
|
+
*/
|
|
62
|
+
export function recordFor(exec, result) {
|
|
63
|
+
const toolName = typeof exec?.name === "string" ? exec.name : null;
|
|
64
|
+
if (!toolName) return null;
|
|
65
|
+
|
|
66
|
+
const verdict = classifyDsh(toolName, result);
|
|
67
|
+
return toWire({
|
|
68
|
+
name: toolName,
|
|
69
|
+
// dsh types `arguments` as `unknown` but guarantees it is losslessly
|
|
70
|
+
// JSON-serializable. A non-object stays out rather than being coerced
|
|
71
|
+
// into a shape it never had.
|
|
72
|
+
args: exec.arguments && typeof exec.arguments === "object"
|
|
73
|
+
? exec.arguments
|
|
74
|
+
: {},
|
|
75
|
+
outcome: verdict.outcome,
|
|
76
|
+
status_code: null,
|
|
77
|
+
error: verdict.error,
|
|
78
|
+
error_class: verdict.error_class,
|
|
79
|
+
agent_name: agentName(exec),
|
|
80
|
+
// `callId` is the model's own id for a root call. A nested call under code
|
|
81
|
+
// mode gets a registry-assigned one, which is still the only identifier
|
|
82
|
+
// that execution has.
|
|
83
|
+
tool_call_id: typeof exec.callId === "string" ? exec.callId : undefined,
|
|
84
|
+
// Resolved from the canonical value, which dsh documents as
|
|
85
|
+
// "execution-local ... deliberately omitted from durable events" -- so it
|
|
86
|
+
// is available HERE and nowhere later. Identity is taken and the value
|
|
87
|
+
// itself is dropped.
|
|
88
|
+
entity_ids: extractEntityIds(result?.value),
|
|
89
|
+
});
|
|
90
|
+
}
|
|
91
|
+
|
|
92
|
+
/**
|
|
93
|
+
* Cordis plugin entry. Named `apply`, per the plugin contract.
|
|
94
|
+
*
|
|
95
|
+
* Deliberately no `default` export in this module: the Cordis loader
|
|
96
|
+
* normalizes with `exports.default ?? exports`, so a default here would be
|
|
97
|
+
* handed to the harness INSTEAD of `apply` and the plugin would fail to load.
|
|
98
|
+
* The OpenClaw entry, which does export a default, is a separate module for
|
|
99
|
+
* exactly that reason.
|
|
100
|
+
*/
|
|
101
|
+
export function apply(ctx) {
|
|
102
|
+
if (!ctx || typeof ctx.on !== "function") return null;
|
|
103
|
+
|
|
104
|
+
const logger = ctx.logger ?? null;
|
|
105
|
+
const writer = createWriter({
|
|
106
|
+
onError: (err) => logger?.warn?.(
|
|
107
|
+
`[${name}] trajectory write failed; tool outcomes are not being ` +
|
|
108
|
+
`recorded for this run: ${err?.message ?? err}`,
|
|
109
|
+
),
|
|
110
|
+
});
|
|
111
|
+
|
|
112
|
+
const dispose = ctx.on(HOOK_NAME, (exec, result) => {
|
|
113
|
+
// dsh contains listener failures already, but it logs each one. Swallowing
|
|
114
|
+
// here keeps a malformed execution from filling a customer's diagnostics
|
|
115
|
+
// with our name: one lost record, never a failed run, never a noisy one.
|
|
116
|
+
try {
|
|
117
|
+
const record = recordFor(exec, result);
|
|
118
|
+
if (record) writer.write(record);
|
|
119
|
+
} catch {
|
|
120
|
+
/* one lost record is not worth an interrupted agent */
|
|
121
|
+
}
|
|
122
|
+
// `@mode emit` listeners return undefined. Returning anything else here
|
|
123
|
+
// would be meaningless at best.
|
|
124
|
+
return undefined;
|
|
125
|
+
});
|
|
126
|
+
|
|
127
|
+
return { writer, dispose };
|
|
128
|
+
}
|
|
129
|
+
|
|
130
|
+
export { TRAJECTORY_DIR_ENV };
|
package/src/errors.js
ADDED
|
@@ -0,0 +1,37 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* The two failures a caller must be able to tell apart.
|
|
3
|
+
*
|
|
4
|
+
* `LeaseLost` is not an error in the operational sense -- it is the control
|
|
5
|
+
* plane saying "this rollout is no longer yours", which happens whenever a
|
|
6
|
+
* lease lapses and the work is requeued. A caller that treats it as a
|
|
7
|
+
* transport failure retries forever against a rollout someone else now owns.
|
|
8
|
+
*/
|
|
9
|
+
export class TransportError extends Error {
|
|
10
|
+
constructor(message, options) {
|
|
11
|
+
super(message, options);
|
|
12
|
+
this.name = "TransportError";
|
|
13
|
+
}
|
|
14
|
+
}
|
|
15
|
+
|
|
16
|
+
export class LeaseLost extends TransportError {
|
|
17
|
+
constructor(message, options) {
|
|
18
|
+
super(message, options);
|
|
19
|
+
this.name = "LeaseLost";
|
|
20
|
+
}
|
|
21
|
+
}
|
|
22
|
+
|
|
23
|
+
/**
|
|
24
|
+
* A setting this package refuses to guess at.
|
|
25
|
+
*
|
|
26
|
+
* Raised for an unrecognised `PERCEPTEYE_AGENT_MODE`, and only there. The
|
|
27
|
+
* alternative -- defaulting a typo to `training` -- keeps a process
|
|
28
|
+
* contributing rollouts on a customer's credentials after an operator
|
|
29
|
+
* believed they had switched it off, with nothing printed anywhere. There is
|
|
30
|
+
* no safe guess for a mode, so a bad one is a startup failure.
|
|
31
|
+
*/
|
|
32
|
+
export class ConfigurationError extends Error {
|
|
33
|
+
constructor(message, options) {
|
|
34
|
+
super(message, options);
|
|
35
|
+
this.name = "ConfigurationError";
|
|
36
|
+
}
|
|
37
|
+
}
|