@ax-llm/ax 21.0.6 → 21.0.8
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/index.cjs +286 -327
- package/index.cjs.map +1 -1
- package/index.d.cts +361 -255
- package/index.d.ts +361 -255
- package/index.global.js +278 -319
- package/index.global.js.map +1 -1
- package/index.js +286 -327
- package/index.js.map +1 -1
- package/package.json +1 -1
- package/skills/ax-agent-memory-skills.md +284 -0
- package/skills/ax-agent-observability.md +334 -0
- package/skills/ax-agent-optimize.md +1 -1
- package/skills/ax-agent-rlm.md +477 -0
- package/skills/ax-agent.md +205 -1350
- package/skills/ax-ai.md +1 -1
- package/skills/ax-audio.md +1 -1
- package/skills/ax-flow.md +1 -1
- package/skills/ax-gen.md +1 -1
- package/skills/ax-gepa.md +1 -1
- package/skills/ax-learn.md +1 -1
- package/skills/ax-llm.md +2 -2
- package/skills/ax-signature.md +1 -1
package/package.json
CHANGED
|
@@ -0,0 +1,284 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: ax-agent-memory-skills
|
|
3
|
+
description: This skill helps an LLM generate correct AxAgent memory retrieval and dynamic skill-loading code using @ax-llm/ax. Use when the user asks about onMemoriesSearch, recall(...), inputs.memories, onLoadedMemories, onUsedMemories, onSkillsSearch, discover({ skills }), onLoadedSkills, onUsedSkills, preloaded skills, loaded memory/skill IDs, or carrying memories across forward() calls.
|
|
4
|
+
version: "21.0.8"
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# AxAgent Memory And Skills Rules (@ax-llm/ax)
|
|
8
|
+
|
|
9
|
+
Use this skill when an agent needs to retrieve task-relevant memories or load skill guides into the executor prompt on demand. For ordinary agent setup use `ax-agent`. For RLM runtime policy use `ax-agent-rlm`. For callbacks and telemetry use `ax-agent-observability`.
|
|
10
|
+
|
|
11
|
+
## Use These Defaults
|
|
12
|
+
|
|
13
|
+
- Use `onMemoriesSearch` when the agent should pull relevant context from an external store instead of stuffing everything into the prompt upfront.
|
|
14
|
+
- Use `onSkillsSearch` when the agent should load usage guides, runbooks, or domain conventions into the executor prompt on demand.
|
|
15
|
+
- `recall(...)` is available to distiller and executor stages when `onMemoriesSearch` is set.
|
|
16
|
+
- `discover({ skills })` is available to the executor when `onSkillsSearch` is set.
|
|
17
|
+
- Both `recall(...)` and `discover({ skills })` return `void`. The loaded content appears on the next turn.
|
|
18
|
+
- Use `onLoadedMemories` / `onLoadedSkills` to observe what got loaded.
|
|
19
|
+
- Use `onUsedMemories` / `onUsedSkills` to track what the actor says it actually relied on.
|
|
20
|
+
- Child agents do not inherit memory or skills search callbacks; wire them explicitly on every agent that needs the capability.
|
|
21
|
+
|
|
22
|
+
## Memory Search
|
|
23
|
+
|
|
24
|
+
Use `onMemoriesSearch` when the agent needs to pull task-relevant context such as user preferences, prior decisions, project facts, or past conversations from an external store (vector DB, BM25, KV). The actor decides what to load, when, and how much.
|
|
25
|
+
|
|
26
|
+
When `onMemoriesSearch` is set, the distiller and executor stages gain:
|
|
27
|
+
|
|
28
|
+
1. An `inputs.memories` field. In JS this is an array of `{ id, content }` entries the actor reads directly. In the prompt, the same entries render as markdown blocks with `ID: \`...\`` lines, matching the Loaded Skills ID style. Each `content` is opaque markdown; frontmatter is not parsed.
|
|
29
|
+
2. A `recall(searches: string[]): void` global the actor `await`s to load more entries. Recalled entries are appended to `inputs.memories` and visible from the next turn onward. `recall()` returns nothing.
|
|
30
|
+
|
|
31
|
+
The responder stage does not receive memories.
|
|
32
|
+
|
|
33
|
+
### Enabling
|
|
34
|
+
|
|
35
|
+
```typescript
|
|
36
|
+
import { agent } from '@ax-llm/ax';
|
|
37
|
+
import type { AxAgentMemoriesSearchFn } from '@ax-llm/ax';
|
|
38
|
+
|
|
39
|
+
const onMemoriesSearch: AxAgentMemoriesSearchFn = async (
|
|
40
|
+
searches,
|
|
41
|
+
alreadyLoaded
|
|
42
|
+
) => {
|
|
43
|
+
// `searches` is the full array passed to recall(...). Batch your
|
|
44
|
+
// store lookup in one round-trip.
|
|
45
|
+
// `alreadyLoaded` is the current inputs.memories snapshot. Filter
|
|
46
|
+
// against it to skip duplicates.
|
|
47
|
+
const skip = new Set(alreadyLoaded.map((m) => m.id));
|
|
48
|
+
const fresh = await myVectorDB.searchBatch(searches, { topK: 3 });
|
|
49
|
+
return fresh.filter((m) => !skip.has(m.id));
|
|
50
|
+
};
|
|
51
|
+
|
|
52
|
+
const myAgent = agent('task:string -> answer:string', {
|
|
53
|
+
contextFields: [],
|
|
54
|
+
onMemoriesSearch,
|
|
55
|
+
});
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
Each memory result must be:
|
|
59
|
+
|
|
60
|
+
```typescript
|
|
61
|
+
type AxAgentMemoryResult = {
|
|
62
|
+
id: string;
|
|
63
|
+
content: string;
|
|
64
|
+
};
|
|
65
|
+
```
|
|
66
|
+
|
|
67
|
+
### Actor usage
|
|
68
|
+
|
|
69
|
+
```javascript
|
|
70
|
+
// Turn 1: kick off one batched lookup.
|
|
71
|
+
await recall(['user preferences', 'project constraints']);
|
|
72
|
+
|
|
73
|
+
// Turn 2+: matched entries are now visible on inputs.memories.
|
|
74
|
+
const prefs = inputs.memories.find((m) => m.id === 'user-prefs-v2');
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
Rules:
|
|
78
|
+
|
|
79
|
+
- Pass all memory queries in one `await recall([...])` call.
|
|
80
|
+
- Do not loop `recall()` calls or wrap them in `Promise.all(...)`.
|
|
81
|
+
- Read `inputs.memories` on the next turn to see what landed.
|
|
82
|
+
- `recall()` invokes `onMemoriesSearch` with `(searches, alreadyLoaded)` and returns `void`.
|
|
83
|
+
- Results land on `inputs.memories` for subsequent turns and render in the prompt as:
|
|
84
|
+
|
|
85
|
+
```markdown
|
|
86
|
+
### Memory
|
|
87
|
+
|
|
88
|
+
ID: `mem:user-prefs-v2`
|
|
89
|
+
|
|
90
|
+
...
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
- Entries are deduped by `id` (last-write-wins) and sorted by `id` for prefix-cache stability.
|
|
94
|
+
- Memories loaded by the distiller thread automatically to the executor. No second `recall()` is needed for those entries.
|
|
95
|
+
- `recall()` may be called multiple times across turns; results accumulate for that run.
|
|
96
|
+
- `inputs.memories` lifetime is one `.forward()` call. It resets between calls.
|
|
97
|
+
|
|
98
|
+
## Carrying Memories Across `.forward()` Calls
|
|
99
|
+
|
|
100
|
+
To preserve continuity across calls, persist memories in your store and recall them again on the next call. If you want to replay anything loaded on a prior run, observe loads with `onLoadedMemories`.
|
|
101
|
+
|
|
102
|
+
```typescript
|
|
103
|
+
const carried = new Map<string, string>();
|
|
104
|
+
|
|
105
|
+
const myAgent = agent('task:string -> answer:string', {
|
|
106
|
+
contextFields: [],
|
|
107
|
+
onMemoriesSearch: async (searches) => {
|
|
108
|
+
const fresh = await myVectorDB.searchBatch(searches, { topK: 3 });
|
|
109
|
+
const carriedAsResults = [...carried.entries()].map(([id, content]) => ({
|
|
110
|
+
id,
|
|
111
|
+
content,
|
|
112
|
+
}));
|
|
113
|
+
return [...carriedAsResults, ...fresh];
|
|
114
|
+
},
|
|
115
|
+
onLoadedMemories: (results) => {
|
|
116
|
+
for (const r of results) carried.set(r.id, r.content);
|
|
117
|
+
},
|
|
118
|
+
});
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
## Skills Search
|
|
122
|
+
|
|
123
|
+
Use `onSkillsSearch` when the agent needs to load skill guides such as usage instructions, runbooks, or domain conventions into the executor's system prompt on demand. The actor decides which skills to fetch and when, so you do not pre-render every skill into every prompt.
|
|
124
|
+
|
|
125
|
+
When `onSkillsSearch` is set, the executor stage gains:
|
|
126
|
+
|
|
127
|
+
1. A "Loaded Skills" section in the system prompt that renders matched skill bodies with stable `ID:` values sorted by `id`.
|
|
128
|
+
2. A `discover({ skills })` path the actor `await`s to load more skills. Loaded entries appear in the next turn's prompt. `discover(...)` returns nothing.
|
|
129
|
+
|
|
130
|
+
The distiller and responder do not see skills. Only the executor.
|
|
131
|
+
|
|
132
|
+
### Enabling
|
|
133
|
+
|
|
134
|
+
```typescript
|
|
135
|
+
import { agent } from '@ax-llm/ax';
|
|
136
|
+
import type { AxAgentSkillsSearchFn } from '@ax-llm/ax';
|
|
137
|
+
|
|
138
|
+
// Each result is { id?: string; name: string; content: string }.
|
|
139
|
+
// If id is omitted, Ax falls back to name.
|
|
140
|
+
const onSkillsSearch: AxAgentSkillsSearchFn = async (searches) => {
|
|
141
|
+
return mySkillStore.searchBatch(searches, { topK: 2 });
|
|
142
|
+
};
|
|
143
|
+
|
|
144
|
+
const myAgent = agent('task:string -> answer:string', {
|
|
145
|
+
contextFields: [],
|
|
146
|
+
onSkillsSearch,
|
|
147
|
+
});
|
|
148
|
+
```
|
|
149
|
+
|
|
150
|
+
Each skill result is:
|
|
151
|
+
|
|
152
|
+
```typescript
|
|
153
|
+
type AxAgentSkillResult = {
|
|
154
|
+
id?: string;
|
|
155
|
+
name: string;
|
|
156
|
+
content: string;
|
|
157
|
+
};
|
|
158
|
+
```
|
|
159
|
+
|
|
160
|
+
### Actor usage
|
|
161
|
+
|
|
162
|
+
```javascript
|
|
163
|
+
// Pass all skill queries in one call.
|
|
164
|
+
await discover({ skills: ['release-checklist', 'incident-response'] });
|
|
165
|
+
|
|
166
|
+
// Next turn: loaded skill bodies render under the "Loaded Skills"
|
|
167
|
+
// system-prompt section.
|
|
168
|
+
```
|
|
169
|
+
|
|
170
|
+
Rules:
|
|
171
|
+
|
|
172
|
+
- `discover({ skills })` invokes `onSkillsSearch` with the raw search strings and returns `void`.
|
|
173
|
+
- Matched skills land under "Loaded Skills" for the next turn.
|
|
174
|
+
- Entries are deduped by `id` (last-write-wins) and sorted by `id` for prefix-cache stability.
|
|
175
|
+
- If a skill result omits `id`, its trimmed `name` is used as the id for backwards compatibility.
|
|
176
|
+
- Skills persist on the agent's `currentSkillsPromptState` across `.forward()` calls, unlike memories.
|
|
177
|
+
- Use `agent.getState()` / `setState(...)` to serialize/restore loaded skills.
|
|
178
|
+
- `discover({ skills })` may be called multiple times across turns. Within one turn, batch all skill queries in a single call.
|
|
179
|
+
- Child agents do not inherit `onSkillsSearch`; wire it explicitly per agent.
|
|
180
|
+
|
|
181
|
+
## Preloading Skills
|
|
182
|
+
|
|
183
|
+
If the caller already knows which skills are relevant, pass them up front instead of round-tripping through `discover({ skills })`.
|
|
184
|
+
|
|
185
|
+
- Init-time: `skills` on `AxAgentOptions` seeds the executor prompt at agent creation. They survive `setState(...)` resets.
|
|
186
|
+
- Forward-time: `skills` on `forward(ai, values, { skills })` merge in at the start of that call. Distiller and responder ignore forward-time skills.
|
|
187
|
+
|
|
188
|
+
Both accept the same shape `onSkillsSearch` returns: `readonly AxAgentSkillResult[]`. Forward-time skills override init-time skills by `id`. `onLoadedSkills` is not fired for preset skills; that callback is for runtime `discover({ skills })` analytics.
|
|
189
|
+
|
|
190
|
+
```typescript
|
|
191
|
+
const releaseAgent = agent('task:string -> answer:string', {
|
|
192
|
+
contextFields: [],
|
|
193
|
+
skills: [
|
|
194
|
+
{
|
|
195
|
+
id: 'skill:release-checklist',
|
|
196
|
+
name: 'release-checklist',
|
|
197
|
+
content: '...',
|
|
198
|
+
},
|
|
199
|
+
],
|
|
200
|
+
});
|
|
201
|
+
|
|
202
|
+
await releaseAgent.forward(
|
|
203
|
+
ai,
|
|
204
|
+
{ task: 'Prepare release notes' },
|
|
205
|
+
{
|
|
206
|
+
skills: [
|
|
207
|
+
{
|
|
208
|
+
id: 'skill:incident-response',
|
|
209
|
+
name: 'incident-response',
|
|
210
|
+
content: '...',
|
|
211
|
+
},
|
|
212
|
+
],
|
|
213
|
+
}
|
|
214
|
+
);
|
|
215
|
+
```
|
|
216
|
+
|
|
217
|
+
You can use `skills` without setting `onSkillsSearch` at all. That is useful for static guides where the actor never needs to fetch more.
|
|
218
|
+
|
|
219
|
+
## Loaded And Used Tracking
|
|
220
|
+
|
|
221
|
+
`onLoadedMemories` reports what `recall(...)` loaded. `onLoadedSkills` reports what `discover({ skills })` loaded. To track what the actor says it actually relied on, use `onUsedMemories` / `onUsedSkills`.
|
|
222
|
+
|
|
223
|
+
```typescript
|
|
224
|
+
const used: AxAgentUsedMemory[] = [];
|
|
225
|
+
|
|
226
|
+
await myAgent.forward(
|
|
227
|
+
ai,
|
|
228
|
+
{ task: 'Make a personal plan' },
|
|
229
|
+
{
|
|
230
|
+
onUsedMemories: (items) => used.push(...items),
|
|
231
|
+
}
|
|
232
|
+
);
|
|
233
|
+
|
|
234
|
+
used; // [{ id, reason, stage }]
|
|
235
|
+
```
|
|
236
|
+
|
|
237
|
+
Rules:
|
|
238
|
+
|
|
239
|
+
- The actor can only report memory IDs already present in `inputs.memories`.
|
|
240
|
+
- The actor can only report skill IDs already present in Loaded Skills.
|
|
241
|
+
- Unknown values are dropped.
|
|
242
|
+
- When tracking is enabled, the actor sees `await used(id, reason?)`; this is the actor-side declaration mechanism.
|
|
243
|
+
- `used(...)` resolves against loaded memory IDs and loaded skill IDs.
|
|
244
|
+
- If memory IDs and skill IDs can collide, namespace them in your application, for example `mem:abc` and `skill:planning`.
|
|
245
|
+
|
|
246
|
+
Types:
|
|
247
|
+
|
|
248
|
+
```typescript
|
|
249
|
+
onMemoriesSearch?: AxAgentMemoriesSearchFn;
|
|
250
|
+
onLoadedMemories?: (
|
|
251
|
+
results: readonly AxAgentMemoryResult[]
|
|
252
|
+
) => void | Promise<void>;
|
|
253
|
+
onUsedMemories?: (
|
|
254
|
+
usedMemories: readonly AxAgentUsedMemory[]
|
|
255
|
+
) => void | Promise<void>;
|
|
256
|
+
|
|
257
|
+
onSkillsSearch?: AxAgentSkillsSearchFn;
|
|
258
|
+
onLoadedSkills?: (
|
|
259
|
+
results: readonly AxAgentSkillResult[]
|
|
260
|
+
) => void | Promise<void>;
|
|
261
|
+
onUsedSkills?: (
|
|
262
|
+
usedSkills: readonly AxAgentUsedSkill[]
|
|
263
|
+
) => void | Promise<void>;
|
|
264
|
+
|
|
265
|
+
skills?: readonly AxAgentSkillResult[];
|
|
266
|
+
```
|
|
267
|
+
|
|
268
|
+
## Examples
|
|
269
|
+
|
|
270
|
+
Fetch this for full working code:
|
|
271
|
+
|
|
272
|
+
- [RLM Memories and Skills](https://raw.githubusercontent.com/ax-llm/ax/refs/heads/main/src/examples/rlm-memories-and-skills.ts) - `onMemoriesSearch` + `recall()` and `onSkillsSearch` + `discover({ skills })` with load observability and actual usage tracking via `onUsedMemories` / `onUsedSkills`
|
|
273
|
+
|
|
274
|
+
## Do Not Generate
|
|
275
|
+
|
|
276
|
+
- Do not assign the result of `await recall(...)` or `await discover(...)`; both return `void`.
|
|
277
|
+
- Do not call `recall()` from the responder stage.
|
|
278
|
+
- Do not call `discover({ skills })` from the distiller or responder stages.
|
|
279
|
+
- Do not loop `recall()` calls or wrap them in `Promise.all(...)`.
|
|
280
|
+
- Do not loop `discover()` calls or wrap them in `Promise.all(...)`.
|
|
281
|
+
- Do not assume child agents inherit `onMemoriesSearch` or `onSkillsSearch`.
|
|
282
|
+
- Do not pass `onMemoriesSearch` results via shared fields as a workaround; use `recall(...)`.
|
|
283
|
+
- Do not assume `inputs.memories` persists across `.forward()` calls.
|
|
284
|
+
- Do not use `onLoadedMemories` / `onLoadedSkills` as proof that the actor relied on an item; use `onUsedMemories` / `onUsedSkills` for actual-use tracking.
|
|
@@ -0,0 +1,334 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: ax-agent-observability
|
|
3
|
+
description: This skill helps an LLM generate correct AxAgent observability code using @ax-llm/ax. Use when the user asks about actorTurnCallback, executorTurnCallback, onContextEvent, agentStatusCallback, onFunctionCall, reportSuccess, reportFailure, getChatLog(), getUsage(), resetUsage(), debug traces, progress updates, or telemetry for AxAgent runs.
|
|
4
|
+
version: "21.0.8"
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# AxAgent Observability Rules (@ax-llm/ax)
|
|
8
|
+
|
|
9
|
+
Use this skill when an agent needs runtime visibility, progress reporting, tracing, usage accounting, or chat-log access. For ordinary agent setup use `ax-agent`. For RLM runtime policy use `ax-agent-rlm`. For memories and dynamic skill loading use `ax-agent-memory-skills`.
|
|
10
|
+
|
|
11
|
+
## Choose The Smallest Hook
|
|
12
|
+
|
|
13
|
+
- Need a quick prompt/runtime trace during development -> start with `debug: true`.
|
|
14
|
+
- Need structured per-turn code, raw runtime result, formatted output, provider thoughts, or actor stage -> use `actorTurnCallback`.
|
|
15
|
+
- Need context-pressure and compaction telemetry -> use `onContextEvent`.
|
|
16
|
+
- Need real-time task progress emitted by actor code -> use `agentStatusCallback`.
|
|
17
|
+
- Need every runtime function call before execution -> use `onFunctionCall`.
|
|
18
|
+
- Need model prompts/responses after a run -> use `getChatLog()`.
|
|
19
|
+
- Need token usage by actor/responder -> use `getUsage()` and `resetUsage()`.
|
|
20
|
+
- Need usage split by context and task stages -> use `getStagedUsage()`.
|
|
21
|
+
- Need Ax program traces -> use `getTraces()`.
|
|
22
|
+
- Do not add multiple hooks unless the user clearly needs each output stream.
|
|
23
|
+
|
|
24
|
+
## Actor Turn Callback
|
|
25
|
+
|
|
26
|
+
Use `actorTurnCallback` when the caller needs structured telemetry for each actor turn. `executorTurnCallback` is still accepted as a deprecated alias for older code.
|
|
27
|
+
|
|
28
|
+
What it gives you:
|
|
29
|
+
|
|
30
|
+
- `code`: the normalized JavaScript code the actor produced
|
|
31
|
+
- `stage`: which actor produced the turn (`distiller` or `executor`)
|
|
32
|
+
- `result`: the raw untruncated runtime return value from executing that code
|
|
33
|
+
- `output`: the formatted action-log output string after Ax normalizes and truncates it for prompt replay
|
|
34
|
+
- `thought`: the actor model's `thought` field when `showThoughts` is enabled and the provider returns one
|
|
35
|
+
- `executorResult`: the full actor payload returned by the current actor stage, kept under this historical field name for compatibility
|
|
36
|
+
- `isError`: whether the execution path for that turn was treated as an error
|
|
37
|
+
- `usage`: token usage for this actor turn only
|
|
38
|
+
- `model`: model used for this turn when explicitly set through `executorModelPolicy`
|
|
39
|
+
- `chatLogMessages`: raw ChatML conversation for this turn, populated only when an actor turn callback is set
|
|
40
|
+
|
|
41
|
+
Use it for:
|
|
42
|
+
|
|
43
|
+
- debug UIs that want to show code plus raw runtime results
|
|
44
|
+
- tracing and analytics
|
|
45
|
+
- capturing `thought` for internal diagnostics when supported by the provider
|
|
46
|
+
- storing per-turn execution artifacts without scraping the prompt/action log
|
|
47
|
+
|
|
48
|
+
Important:
|
|
49
|
+
|
|
50
|
+
- `output` is not raw stdout; it is the formatted replay string used in the action log.
|
|
51
|
+
- `result` is the raw runtime result before Ax applies type-aware serialization and budget-proportional truncation.
|
|
52
|
+
- `thought` is optional and only appears when the underlying `AxGen` call had `showThoughts` enabled and the provider actually returned a thought field.
|
|
53
|
+
- `actionLogEntryCount` and `guidanceLogEntryCount` reflect the live log sizes after the turn is processed, including resumed runs.
|
|
54
|
+
- `actorTurnCallback` fires for the root agent and for recursive child agents that run actor turns.
|
|
55
|
+
|
|
56
|
+
Good pattern:
|
|
57
|
+
|
|
58
|
+
```typescript
|
|
59
|
+
const supportAgent = agent('query:string -> answer:string', {
|
|
60
|
+
contextFields: ['query'],
|
|
61
|
+
runtime,
|
|
62
|
+
actorTurnCallback: ({
|
|
63
|
+
stage,
|
|
64
|
+
turn,
|
|
65
|
+
actionLogEntryCount,
|
|
66
|
+
guidanceLogEntryCount,
|
|
67
|
+
code,
|
|
68
|
+
result,
|
|
69
|
+
output,
|
|
70
|
+
thought,
|
|
71
|
+
isError,
|
|
72
|
+
usage,
|
|
73
|
+
model,
|
|
74
|
+
}) => {
|
|
75
|
+
console.log({
|
|
76
|
+
turn,
|
|
77
|
+
stage,
|
|
78
|
+
model,
|
|
79
|
+
actionLogEntryCount,
|
|
80
|
+
guidanceLogEntryCount,
|
|
81
|
+
isError,
|
|
82
|
+
code,
|
|
83
|
+
rawResult: result,
|
|
84
|
+
replayOutput: output,
|
|
85
|
+
thought,
|
|
86
|
+
usage,
|
|
87
|
+
});
|
|
88
|
+
},
|
|
89
|
+
executorOptions: {
|
|
90
|
+
model: 'gpt-5.4-mini',
|
|
91
|
+
showThoughts: true,
|
|
92
|
+
},
|
|
93
|
+
});
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
Callback type:
|
|
97
|
+
|
|
98
|
+
```typescript
|
|
99
|
+
actorTurnCallback?: (turn: {
|
|
100
|
+
stage: 'distiller' | 'executor';
|
|
101
|
+
turn: number;
|
|
102
|
+
actionLogEntryCount: number;
|
|
103
|
+
guidanceLogEntryCount: number;
|
|
104
|
+
executorResult: Record<string, unknown>;
|
|
105
|
+
code: string;
|
|
106
|
+
result: unknown;
|
|
107
|
+
output: string;
|
|
108
|
+
isError: boolean;
|
|
109
|
+
thought?: string;
|
|
110
|
+
usage?: AxProgramUsage[];
|
|
111
|
+
model?: string;
|
|
112
|
+
chatLogMessages?: ReadonlyArray<{ role: string; content: string }>;
|
|
113
|
+
}) => void | Promise<void>;
|
|
114
|
+
|
|
115
|
+
executorTurnCallback?: (turn: {
|
|
116
|
+
stage: 'distiller' | 'executor';
|
|
117
|
+
turn: number;
|
|
118
|
+
actionLogEntryCount: number;
|
|
119
|
+
guidanceLogEntryCount: number;
|
|
120
|
+
executorResult: Record<string, unknown>;
|
|
121
|
+
code: string;
|
|
122
|
+
result: unknown;
|
|
123
|
+
output: string;
|
|
124
|
+
isError: boolean;
|
|
125
|
+
thought?: string;
|
|
126
|
+
usage?: AxProgramUsage[];
|
|
127
|
+
model?: string;
|
|
128
|
+
chatLogMessages?: ReadonlyArray<{ role: string; content: string }>;
|
|
129
|
+
}) => void | Promise<void>; // deprecated alias
|
|
130
|
+
```
|
|
131
|
+
|
|
132
|
+
## Context Event Observability
|
|
133
|
+
|
|
134
|
+
Use `onContextEvent` when the caller needs structured telemetry about prompt pressure and compaction. It does not change model behavior directly; it is for logs, evals, and dashboards.
|
|
135
|
+
|
|
136
|
+
Events:
|
|
137
|
+
|
|
138
|
+
- `budget_check`: character-based prompt pressure before an actor turn, with detailed metrics kept out of the actor prompt
|
|
139
|
+
- `checkpoint_created` / `checkpoint_cleared`: checkpoint lifecycle events with covered turns and reason
|
|
140
|
+
- `tombstone_created`: compact resolved-error summary creation
|
|
141
|
+
|
|
142
|
+
Rules:
|
|
143
|
+
|
|
144
|
+
- `contextPressure` in the actor prompt is intentionally compact (`ok`, `watch`, `critical` plus one short instruction).
|
|
145
|
+
- Budget metrics are character-based for provider neutrality and are exposed through `onContextEvent`, not the actor prompt.
|
|
146
|
+
- Callback errors are swallowed so telemetry cannot break the agent run.
|
|
147
|
+
- Do not scrape actor prompts for pressure metrics.
|
|
148
|
+
|
|
149
|
+
```typescript
|
|
150
|
+
const supportAgent = agent('query:string -> answer:string', {
|
|
151
|
+
contextFields: ['query'],
|
|
152
|
+
runtime,
|
|
153
|
+
contextPolicy: { preset: 'checkpointed', budget: 'balanced' },
|
|
154
|
+
onContextEvent: (event) => {
|
|
155
|
+
if (event.kind === 'budget_check') {
|
|
156
|
+
console.log(event.pressure, event.mutablePromptChars);
|
|
157
|
+
}
|
|
158
|
+
},
|
|
159
|
+
});
|
|
160
|
+
```
|
|
161
|
+
|
|
162
|
+
Type:
|
|
163
|
+
|
|
164
|
+
```typescript
|
|
165
|
+
onContextEvent?: (event: AxAgentContextEvent) => void | Promise<void>;
|
|
166
|
+
```
|
|
167
|
+
|
|
168
|
+
## Agent Status Callback
|
|
169
|
+
|
|
170
|
+
Use `agentStatusCallback` when the caller wants real-time progress updates from the actor. When set, the actor can call `await reportSuccess(message)` and `await reportFailure(message)` in its JavaScript turns.
|
|
171
|
+
|
|
172
|
+
```typescript
|
|
173
|
+
const supportAgent = agent('query:string -> answer:string', {
|
|
174
|
+
contextFields: ['query'],
|
|
175
|
+
runtime,
|
|
176
|
+
agentStatusCallback: (message, status) => {
|
|
177
|
+
console.log(`[${status}] ${message}`);
|
|
178
|
+
},
|
|
179
|
+
});
|
|
180
|
+
```
|
|
181
|
+
|
|
182
|
+
Rules:
|
|
183
|
+
|
|
184
|
+
- `agentStatusCallback` receives `(message: string, status: 'success' | 'failed')`.
|
|
185
|
+
- When set, the actor prompt automatically includes `reportSuccess(message)` and `reportFailure(message)` as available runtime functions.
|
|
186
|
+
- The actor is instructed to keep the user updated on task progress.
|
|
187
|
+
- `reportSuccess` and `reportFailure` are reserved runtime names when the callback is configured.
|
|
188
|
+
- Child agents inherit the callback via the RLM config.
|
|
189
|
+
|
|
190
|
+
Type:
|
|
191
|
+
|
|
192
|
+
```typescript
|
|
193
|
+
agentStatusCallback?: (
|
|
194
|
+
message: string,
|
|
195
|
+
status: 'success' | 'failed'
|
|
196
|
+
) => void | Promise<void>;
|
|
197
|
+
```
|
|
198
|
+
|
|
199
|
+
## On Function Call
|
|
200
|
+
|
|
201
|
+
Use `onFunctionCall` when the caller wants to observe every function call the actor makes from the JS runtime. It fires before the underlying function runs.
|
|
202
|
+
|
|
203
|
+
```typescript
|
|
204
|
+
const supportAgent = agent('query:string -> answer:string', {
|
|
205
|
+
contextFields: ['query'],
|
|
206
|
+
runtime,
|
|
207
|
+
functions: [helperAgent, lookupOrderTool],
|
|
208
|
+
onFunctionCall: ({ name, qualifiedName, args, kind }) => {
|
|
209
|
+
console.log(`[${kind}] ${qualifiedName}`, args);
|
|
210
|
+
},
|
|
211
|
+
});
|
|
212
|
+
```
|
|
213
|
+
|
|
214
|
+
Rules:
|
|
215
|
+
|
|
216
|
+
- Receives `{ name, qualifiedName, args, kind }`.
|
|
217
|
+
- `name` is the bare function name, e.g. `'lookupOrder'`.
|
|
218
|
+
- `qualifiedName` is the namespaced name as the actor sees it, e.g. `'tools.lookupOrder'`; for un-namespaced runtime globals it equals `name`.
|
|
219
|
+
- `args` is the resolved positional/named arguments object (`Record<string, unknown>`).
|
|
220
|
+
- `kind` is `'external'` for caller-registered `functions`.
|
|
221
|
+
- `kind` is `'internal'` for agent-injected globals: child agents, `discover`, `recall`, and `used`.
|
|
222
|
+
- Fires once per call, before the function executes.
|
|
223
|
+
- Errors thrown inside the callback are swallowed so they cannot break the actor loop.
|
|
224
|
+
- This is independent from the DSP-layer `onFunctionCall` on `AxProgramForwardOptions`; that hook is for LLM tool-calls and never fires under AxAgent because AxAgent injects functions as runtime globals.
|
|
225
|
+
|
|
226
|
+
Type:
|
|
227
|
+
|
|
228
|
+
```typescript
|
|
229
|
+
onFunctionCall?: (call: {
|
|
230
|
+
name: string;
|
|
231
|
+
qualifiedName: string;
|
|
232
|
+
args: Record<string, unknown>;
|
|
233
|
+
kind: 'internal' | 'external';
|
|
234
|
+
}) => void | Promise<void>;
|
|
235
|
+
```
|
|
236
|
+
|
|
237
|
+
## Chat Log, Usage, And Traces
|
|
238
|
+
|
|
239
|
+
`AxAgent` exposes actor and responder sub-programs. `getChatLog()` returns the same flat `AxChatLogEntry[]` shape as `AxGen` and `AxFlow`; use each entry's optional `name` field to distinguish `distiller`, `executor`, and `responder`. `getUsage()` returns token usage split by actor/responder.
|
|
240
|
+
|
|
241
|
+
### getChatLog()
|
|
242
|
+
|
|
243
|
+
Returns the full normalized chat history after any `.forward()` call. Each entry is one `ai.chat()` round-trip. Actor stages accumulate one entry per turn; the responder typically has one entry.
|
|
244
|
+
|
|
245
|
+
```typescript
|
|
246
|
+
const log = myAgent.getChatLog();
|
|
247
|
+
|
|
248
|
+
for (const entry of log) {
|
|
249
|
+
console.log(entry.name, entry.model);
|
|
250
|
+
for (const msg of entry.messages) {
|
|
251
|
+
console.log(`[${msg.role}]`, msg.content);
|
|
252
|
+
}
|
|
253
|
+
}
|
|
254
|
+
```
|
|
255
|
+
|
|
256
|
+
Each `AxChatLogEntry` captures the full prompt sent to the model and its response:
|
|
257
|
+
|
|
258
|
+
```typescript
|
|
259
|
+
type AxChatLogMessage =
|
|
260
|
+
| { role: 'system'; content: string }
|
|
261
|
+
| { role: 'user'; content: string }
|
|
262
|
+
| { role: 'assistant'; content: string }
|
|
263
|
+
| { role: 'tool'; name: string; content: string };
|
|
264
|
+
|
|
265
|
+
type AxChatLogEntry = {
|
|
266
|
+
name?: string; // e.g. "distiller", "executor", "responder"
|
|
267
|
+
model: string;
|
|
268
|
+
messages: AxChatLogMessage[];
|
|
269
|
+
modelUsage?: AxProgramUsage;
|
|
270
|
+
stage?: 'ctx' | 'task';
|
|
271
|
+
};
|
|
272
|
+
```
|
|
273
|
+
|
|
274
|
+
### getUsage()
|
|
275
|
+
|
|
276
|
+
Returns token usage split by actor/responder. Each sub-array contains one `AxProgramUsage` entry per model/run, merged by `(ai, model)` key.
|
|
277
|
+
|
|
278
|
+
```typescript
|
|
279
|
+
const usage = myAgent.getUsage();
|
|
280
|
+
// { actor: AxProgramUsage[], responder: AxProgramUsage[] }
|
|
281
|
+
|
|
282
|
+
console.log('Actor tokens:', usage.actor[0]?.tokens);
|
|
283
|
+
console.log('Responder tokens:', usage.responder[0]?.tokens);
|
|
284
|
+
```
|
|
285
|
+
|
|
286
|
+
### getStagedUsage()
|
|
287
|
+
|
|
288
|
+
Returns usage split by pipeline stage. The `ctx` stage has the distiller actor only; the `task` stage has the executor actor plus responder.
|
|
289
|
+
|
|
290
|
+
```typescript
|
|
291
|
+
const staged = myAgent.getStagedUsage();
|
|
292
|
+
console.log(staged.ctx?.actor);
|
|
293
|
+
console.log(staged.task.actor);
|
|
294
|
+
console.log(staged.task.responder);
|
|
295
|
+
```
|
|
296
|
+
|
|
297
|
+
### getTraces()
|
|
298
|
+
|
|
299
|
+
Returns Ax program traces for the agent pipeline. Use it when the caller needs trace data rather than chat messages or token summaries.
|
|
300
|
+
|
|
301
|
+
```typescript
|
|
302
|
+
const traces = myAgent.getTraces();
|
|
303
|
+
```
|
|
304
|
+
|
|
305
|
+
### resetUsage()
|
|
306
|
+
|
|
307
|
+
Resets both actor and responder usage at once:
|
|
308
|
+
|
|
309
|
+
```typescript
|
|
310
|
+
myAgent.resetUsage();
|
|
311
|
+
```
|
|
312
|
+
|
|
313
|
+
Type signatures:
|
|
314
|
+
|
|
315
|
+
```typescript
|
|
316
|
+
// AxAgent
|
|
317
|
+
agent.getChatLog(): readonly AxChatLogEntry[]
|
|
318
|
+
agent.getUsage(): { actor: AxProgramUsage[]; responder: AxProgramUsage[] }
|
|
319
|
+
agent.getStagedUsage(): { ctx?: AxAgentUsage; task: AxAgentUsage }
|
|
320
|
+
agent.getTraces(): AxProgramTrace[]
|
|
321
|
+
agent.resetUsage(): void
|
|
322
|
+
|
|
323
|
+
// AxGen / AxFlow
|
|
324
|
+
gen.getChatLog(): readonly AxChatLogEntry[]
|
|
325
|
+
gen.getUsage(): AxProgramUsage[]
|
|
326
|
+
```
|
|
327
|
+
|
|
328
|
+
## Do Not Generate
|
|
329
|
+
|
|
330
|
+
- Do not add both `debug: true` and `actorTurnCallback` unless the user wants both unstructured prompt/runtime visibility and structured telemetry.
|
|
331
|
+
- Do not scrape actor prompts or action logs when a callback exposes the data directly.
|
|
332
|
+
- Do not let observability callback failures break the agent run; Ax swallows callback errors for telemetry hooks.
|
|
333
|
+
- Do not use DSP-layer `onFunctionCall` when the user wants AxAgent runtime function calls.
|
|
334
|
+
- Do not enable `showThoughts` unless the user needs provider thought diagnostics and the provider supports it.
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: ax-agent-optimize
|
|
3
3
|
description: This skill helps an LLM generate correct AxAgent tuning and evaluation code using @ax-llm/ax. Use when the user asks about agent.optimize(...), judgeOptions, eval datasets, optimization targets, saved optimizedProgram artifacts, or recursive optimization guidance.
|
|
4
|
-
version: "21.0.
|
|
4
|
+
version: "21.0.8"
|
|
5
5
|
---
|
|
6
6
|
|
|
7
7
|
# AxAgent Optimize Codegen Rules (@ax-llm/ax)
|