@archmax-ai/harness 1.1.0 → 2.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +167 -170
- package/dist/agent.d.ts +2 -2
- package/dist/agent.js +2 -2
- package/dist/assembly/compose.js +2 -2
- package/dist/assembly/compose.js.map +1 -1
- package/dist/authoring-skill/archmax-harness/README.md +13 -15
- package/dist/authoring-skill/archmax-harness/SKILL.md +174 -264
- package/dist/authoring-skill/archmax-harness/references/backend-integration.md +244 -776
- package/dist/authoring-skill/archmax-harness/references/cases.md +178 -53
- package/dist/authoring-skill/archmax-harness/references/code-states-and-scripts.md +211 -100
- package/dist/authoring-skill/archmax-harness/references/diagnostics.md +57 -78
- package/dist/authoring-skill/archmax-harness/references/workflow-schema.md +613 -2006
- package/dist/core/deepagents.d.ts +1 -2
- package/dist/core/deepagents.js +1 -2
- package/dist/core/deepagents.js.map +1 -1
- package/dist/kernel/kernel.js +2 -2
- package/dist/kernel/kernel.js.map +1 -1
- package/dist/machine/lint-spec.js +6 -5
- package/dist/machine/lint-spec.js.map +1 -1
- package/dist/machine/machine.d.ts +8 -10
- package/dist/machine/machine.js +11 -12
- package/dist/machine/machine.js.map +1 -1
- package/dist/machine/spec-schema.d.ts +1 -4
- package/dist/machine/spec-schema.js +66 -47
- package/dist/machine/spec-schema.js.map +1 -1
- package/dist/machine/tool-names.d.ts +24 -34
- package/dist/machine/tool-names.js +43 -63
- package/dist/machine/tool-names.js.map +1 -1
- package/dist/machine/types.d.ts +11 -10
- package/dist/machine/types.js.map +1 -1
- package/dist/public/spec.d.ts +5 -5
- package/dist/public/spec.js +5 -5
- package/dist/public/spec.js.map +1 -1
- package/dist/validate/validate.js +3 -3
- package/dist/validate/validate.js.map +1 -1
- package/dist/workflow/automated.js +6 -1
- package/dist/workflow/automated.js.map +1 -1
- package/dist/workflow/control-tools.js +0 -2
- package/dist/workflow/control-tools.js.map +1 -1
- package/dist/workflow/grader-state.d.ts +2 -2
- package/dist/workflow/grader-state.js +5 -15
- package/dist/workflow/grader-state.js.map +1 -1
- package/dist/workflow/middleware.js +2 -26
- package/dist/workflow/middleware.js.map +1 -1
- package/dist/workflow/on-error.js +0 -1
- package/dist/workflow/on-error.js.map +1 -1
- package/dist/workflow/parks.js +0 -1
- package/dist/workflow/parks.js.map +1 -1
- package/dist/workflow/render-prompt.d.ts +4 -5
- package/dist/workflow/render-prompt.js +24 -27
- package/dist/workflow/render-prompt.js.map +1 -1
- package/dist/workflow/state.d.ts +0 -9
- package/dist/workflow/state.js +0 -3
- package/dist/workflow/state.js.map +1 -1
- package/dist/workflow/turn-boundary.js +0 -1
- package/dist/workflow/turn-boundary.js.map +1 -1
- package/package.json +1 -2
- package/skills/archmax-harness/README.md +13 -15
- package/skills/archmax-harness/SKILL.md +174 -264
- package/skills/archmax-harness/references/backend-integration.md +244 -776
- package/skills/archmax-harness/references/cases.md +178 -53
- package/skills/archmax-harness/references/code-states-and-scripts.md +211 -100
- package/skills/archmax-harness/references/diagnostics.md +57 -78
- package/skills/archmax-harness/references/workflow-schema.md +613 -2006
- package/dist/authoring-skill/archmax-harness/references/code-state-and-test-scripts.md +0 -1424
- package/dist/authoring-skill/archmax-harness/references/wiring.md +0 -119
- package/dist/authoring-skill/archmax-harness/references/workflow-yaml.md +0 -402
- package/skills/archmax-harness/references/code-state-and-test-scripts.md +0 -1424
- package/skills/archmax-harness/references/wiring.md +0 -119
- package/skills/archmax-harness/references/workflow-yaml.md +0 -402
|
@@ -1,816 +1,284 @@
|
|
|
1
|
-
# Wiring
|
|
1
|
+
# Wiring the harness into a backend
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
package's `dist/index.d.ts`.
|
|
3
|
+
Load this when you embed `@archmax-ai/harness` in a service: a worker that sends turns,
|
|
4
|
+
an API that answers parked sessions, a frontend that draws them. Every name here is an
|
|
5
|
+
export of the installed package. When in doubt, check its `dist/index.d.ts`.
|
|
7
6
|
|
|
8
7
|
## Contents
|
|
9
8
|
|
|
10
|
-
- [Install
|
|
11
|
-
- [
|
|
12
|
-
- [createAgent options](#
|
|
13
|
-
- [
|
|
14
|
-
- [
|
|
15
|
-
- [
|
|
16
|
-
- [
|
|
17
|
-
- [
|
|
18
|
-
- [
|
|
19
|
-
- [Model configuration](#model-configuration)
|
|
20
|
-
- [Custom backends](#custom-backends)
|
|
9
|
+
- [Install](#install)
|
|
10
|
+
- [The smallest program](#the-smallest-program)
|
|
11
|
+
- [`createAgent` options](#createagent-options)
|
|
12
|
+
- [Models](#models)
|
|
13
|
+
- [Outcomes and parked sessions](#outcomes-and-parked-sessions)
|
|
14
|
+
- [Events](#events)
|
|
15
|
+
- [Sessions, storage and artifacts](#sessions-storage-and-artifacts)
|
|
16
|
+
- [Host tools that use the workspace](#host-tools-that-use-the-workspace)
|
|
17
|
+
- [Exposing a workflow to callers](#exposing-a-workflow-to-callers)
|
|
21
18
|
- [Cases against your agent](#cases-against-your-agent)
|
|
22
|
-
- [
|
|
23
|
-
|
|
24
|
-
---
|
|
19
|
+
- [Sub-workflows in a service](#sub-workflows-in-a-service)
|
|
20
|
+
- [A typical split](#a-typical-split)
|
|
25
21
|
|
|
26
22
|
## Install
|
|
27
23
|
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
in `node_modules`, not a symlink), so behavior matches production:
|
|
32
|
-
|
|
33
|
-
```bash
|
|
34
|
-
# in the harness repo
|
|
35
|
-
npm run build
|
|
36
|
-
npm pack --pack-destination /tmp # → /tmp/archmax-ai-harness-<version>.tgz
|
|
37
|
-
|
|
38
|
-
# in the consuming service
|
|
39
|
-
npm add /tmp/archmax-ai-harness-<version>.tgz
|
|
40
|
-
# or pnpm --filter <pkg> add /tmp/archmax-ai-harness-<version>.tgz
|
|
41
|
-
```
|
|
42
|
-
|
|
43
|
-
The published tarball contains only `dist/` + `README.md` (the package `files`
|
|
44
|
-
allowlist), so `dist/index.d.ts` types travel with it — a shared types package can
|
|
45
|
-
re-export the run-artifact/event shapes for an API/frontend without importing the
|
|
46
|
-
runtime. Because the version rarely changes during development, `remove` then `add`
|
|
47
|
-
(or prune the store) so a repack of the same version actually reinstalls.
|
|
48
|
-
|
|
49
|
-
**This skill ships inside the package** (staged into `dist/authoring-skill/archmax-harness/`):
|
|
50
|
-
resolve it from the install via the `BUNDLED_AUTHORING_SKILL_DIR` export — the
|
|
51
|
-
`dist/authoring-skill` directory holding `archmax-harness/`, so this file is
|
|
52
|
-
`<BUNDLED_AUTHORING_SKILL_DIR>/archmax-harness/SKILL.md` — e.g.
|
|
53
|
-
`npx skills add ./node_modules/@archmax-ai/harness/dist/authoring-skill/archmax-harness`.
|
|
54
|
-
Prefer this over
|
|
55
|
-
syncing a copy from a source checkout, so the skill can never drift from the
|
|
56
|
-
installed SDK's validator. Tooling that needs the install root without importing
|
|
57
|
-
the runtime can use `require.resolve("@archmax-ai/harness/package.json")` (exposed in the
|
|
58
|
-
`exports` map).
|
|
59
|
-
|
|
60
|
-
## Public exports
|
|
61
|
-
|
|
62
|
-
From `@archmax-ai/harness` (`src/index.ts`):
|
|
63
|
-
|
|
64
|
-
| Export | Kind | Purpose |
|
|
65
|
-
| --- | --- | --- |
|
|
66
|
-
| `createAgent` | fn | Assemble a runnable workflow-governed agent |
|
|
67
|
-
| `Agent` | type | Returned runtime (`graph`, `decide`, `deliver`, `resolveSession`, `runs`, `emitRunArtifacts`, `dispose`, `resolveTrigger`) |
|
|
68
|
-
| `CreateAgentParams` | type | Assembly options |
|
|
69
|
-
| `DecideOutcome` | type | Return of `decide()` |
|
|
70
|
-
| `DeliverOutcome`, `TriggerDelivery` | type | Argument / return of `deliver()` |
|
|
71
|
-
| `WorkflowLoadError` | class | Thrown when the workflow spec (`workflow.yaml`) is missing/invalid (fails closed) |
|
|
72
|
-
| `DEFAULT_WORKFLOW` | const | `"order-lookup"` |
|
|
73
|
-
| `BUNDLED_AUTHORING_SKILL_DIR` | const | The packaged `dist/authoring-skill` directory holding `archmax-harness/` (`<dir>/archmax-harness/SKILL.md`). The example workspace is not shipped in `dist/`; clone the repo to use it. |
|
|
74
|
-
| `WORKFLOW_STATUSES` | const | `running` \| `completed` \| `failed` (the agent called `archmax_raise`) \| `rejected` \| `awaiting_decision` (`awaitingDecision`) \| `awaiting_input` (`awaitingInput`) |
|
|
75
|
-
| `SessionExit` | type | `Outcome.exit` / `DecideOutcome.exit` / `SessionSummary.exit`: `{ success: true }` on a completed turn, `{ success: false, code, reason }` on a failed one |
|
|
76
|
-
| `isFinished`, `classifyStatus` | fn | The open/finished partition — a parked session is **open** |
|
|
77
|
-
| `WorkflowLifecycleEvent`, `WorkflowEventHandler` | type | Event stream types |
|
|
78
|
-
| `createWorkflowEventEmitter`, `renderEventLine` | fn | Event emitter + console formatter |
|
|
79
|
-
| `createWorkspaceContext` | fn | Compose the workspace backends (session zone + mounts + authoring backend) |
|
|
80
|
-
| `createFilesystemSessionStore`, `createBackendSessionStore`, `createMemorySessionStore` | fn | Run-store factories (physical storage for the session zone) |
|
|
81
|
-
| `SessionStoreRequiredError`, `SessionStoreCapabilityError` | class | Assembly / capability errors for run storage |
|
|
82
|
-
| `workflowStateSchema`, `workflowPaths`, `sessionPaths`, `runArtifactPaths` | schema/fn | State schema + path helpers |
|
|
83
|
-
| `runTests`, `RunTestsOptions`, `CaseResult` | fn/type | Run a workflow's cases — [against your own agent](#cases-against-your-agent) |
|
|
84
|
-
| `toolsFromMap`, `AgentToolDescriptor`, `ToolContext`, `ToolPaths`, `PathAccess`, `ToolPathsError` | fn/type/class | Host tools: a `name → descriptor` map, each descriptor's declared `paths` and a handler handed the turn's workspace — [host tools](#host-tools) |
|
|
85
|
-
| `SIGNATURE_TYPES`, `SIGNATURE_FORMATS`, `SIGNATURE_CONTENTS`, `normalizeSignature`, `signatureForTrigger`, `signatureJsonSchema`, `signatureValueIssues` | const/fn | A trigger's typed signature, rules included, as JSON Schema and its conformance rule — [exposing a workflow](#exposing-a-workflow-to-outside-callers); also on `@archmax-ai/harness/spec` |
|
|
86
|
-
|
|
87
|
-
## createAgent options
|
|
88
|
-
|
|
89
|
-
All optional (`CreateAgentParams`, `src/assembly/index.ts`):
|
|
90
|
-
|
|
91
|
-
| Option | Type | Notes |
|
|
92
|
-
| --- | --- | --- |
|
|
93
|
-
| `rootDir` | `string` | Workspace root for the default filesystem backend. Defaults to cwd. |
|
|
94
|
-
| `workflow` | `string` | Named workflow under `workflows/<name>/`. |
|
|
95
|
-
| `trigger` | `TriggerInput` | `{ id, args? }`; defaults to `{ id: "manual" }`. Selects the start state. |
|
|
96
|
-
| `model` | `BaseChatModel` | Defaults to the env-configured OpenAI-compatible model. Inject your own to bypass env. An explicit instance outranks every model id a spec declares, grader states' included. |
|
|
97
|
-
| `modelFactory` | `(role, env, requested?) => BaseChatModel` | Supplies the chat model per role: `agent` (the session's turns), `judge` (the case judge for `grade:`) and `grader` (a grader state's subagent). `requested` is the id the state or `settings.model` declares, for the `agent` and `grader` roles; a grader state's model is `modelFactory("grader", env, id)`, resolved once per composition. `env` is a thunk, so a factory with its own credentials never loads `.env`. |
|
|
98
|
-
| `backend` | `BackendProtocolV2` | Serves the authored workspace. Defaults to a `FilesystemBackend` over `rootDir`. |
|
|
99
|
-
| `authoring` | `BackendProtocolV2` | Serves the **authoring plane** — `workflows/**` (machine specs, `WORKFLOW.md`, code-state `scripts/`, `tests/`) — to the runtime only. Never a route in the agent's workspace, so no tool call or PTC call can reach it. Defaults to `backend` when you supply one, else a `FilesystemBackend` over the resolved root. Handing it to a **writable** mount throws `AuthoringBackendExposedError`; a read-only mount may share it. |
|
|
100
|
-
| `mounts` | `Record<string, MountSpec>` | Authored mounts as a `CompositeBackend` route table: key → backend, or `{ backend, readOnly, governed, searchable }`. Trailing slash = directory mount; no slash = exact-path file mount (`"/AGENTS.md"`). Read-only unless declared otherwise. `governed: true` hands the spec's `mounts` block the decision of **which states** may reach it (closed by default, like a skill bundle) and whether each may write there; a mount not so declared is visible in every state exactly as today, so a table that marks nothing behaves unchanged. `readOnly` is a **ceiling** the spec cannot lift: a governed writable mount is written only by a workflow whose own grants ask `access: read_write` (a bare grant reads it, so serving one route writable for one workflow in a delegation tree opens it to no other), and a grant may narrow a writable mount to reads in a state, never open a read-only one. A key may be several segments deep (`"/catalogs/eu/"`); its **first** segment is what is reserved, so `/workflows/x/` fails as `/workflows/` does. Omitted on the default backend: `defaultMounts(rootDir)` (the conventional table — `/skills/`, `/.platform/`, `/AGENTS.md`, none governed — extend with `{ ...defaultMounts(root), "/templates/": … }`). Omitted with a custom `backend`: nothing authored is served; expose subtrees with `mountSubtree(backend, "skills")`. A key colliding with a run area — or naming the authoring-plane prefix (`/workflows/`) — throws `MountCollisionError`. `searchable: false` declares a mount the runtime never searches on its own initiative: a root-wide `grep`/`glob` fans out over the other mounts only, while a search the agent addresses at the mount (or a path inside it) still reaches its backend and returns the backend's own answer — matches or `{ error }` — verbatim. Use it for a backend that must refuse searches (a remote folder served live). Listing and reading are unchanged, the prompt marks it "browse only", names arrive as `MountPrefixes.unsearchable`, and it is ignored on a file mount. |
|
|
101
|
-
| `sessionStore` | `SessionStore` | Physical storage for run state (checkpoints, artifacts, the per-session `scratchpad/`). Zero-config default: `createFilesystemSessionStore({ dir: "<rootDir>/sessions" })`. Required with a custom `backend` (else `SessionStoreRequiredError`). Build with `createBackendSessionStore({ backend, prefix? })` for S3-style storage or `createMemorySessionStore()` for ephemeral runs. |
|
|
102
|
-
| `systemPrompt` | `string` | Appended after workspace `AGENTS.md`. |
|
|
103
|
-
| `tools` | `StructuredTool[]` | Host tools, granted per state by `tools.allow`. Build them with `toolsFromMap` — see [host tools](#host-tools). A name starting `archmax_`, or naming a runtime file operation (`copy_file`, `move_file`, `remove_file`), throws `ReservedToolNameError`. |
|
|
104
|
-
| `essentialTools` | `string[]` | Host tools treated as always on: disclosed and permitted in every state, whatever the workflow's `tools.allow_always` declares; a state's own entry still narrows one. |
|
|
105
|
-
| `clock` | `string` | Pins the "now" every model call states, for cases and evaluations: an ISO 8601 date (`2026-03-20`, midnight UTC) or a date-time with `Z`/an offset; zoneless or impossible values fail assembly, `""` means the real clock. Rendered in UTC, rounded to ten minutes, no local time. Recorded timestamps, `resumeAt` and the sandbox's `Date` stay real. Not for production: a pinned session resumed days later still thinks it is the pinned day. A case's `clock:` outranks it. |
|
|
106
|
-
| `images` | `boolean \| ImageReadOptions` | Show a model with vision the images it reads: the model's `read_file` of a PNG/JPEG/GIF/WebP answers with one line and the image is added to each later request as a `user` message after that batch of tool results, read per request — no bytes in history, checkpoints, events or scripts. `{ maxBytes?, keep? }` (default 10 MB, every image). Off by default; every other binary stays refused. |
|
|
107
|
-
| `toolPaths` | `Record<string, ToolPaths>` | Each host tool's path arguments and how a call uses each (`read`, `list`, `search`, `write`, `remove`, `execute`): `{ get_markdown: { path: "read" } }`. Every path rule then governs them like the built-ins'. Wins over a descriptor's `paths`; declaring a built-in's paths throws `ToolPathsError`. |
|
|
108
|
-
| `checkpointer` | `BaseCheckpointSaver` | Custom-adapter escape hatch; takes precedence over the session store for checkpoint persistence. Defaults to a durable `BackendCheckpointSaver` writing through the session store. Use `MemorySaver` for ephemeral runs. |
|
|
109
|
-
| `middleware` | `AgentMiddleware[]` | Extra middleware appended after workflow instrumentation. |
|
|
110
|
-
| `onEvent` | `WorkflowEventHandler` | Subscribe to all lifecycle diagnostics; suppresses console output when set. |
|
|
111
|
-
| `promptCache` | `PromptCacheOptions` | `{ enabled?, ttl?: "5m" \| "1h" }`. Provider prompt caching for the stable prefix; on by default. Falls back to `settings.prompt_cache`, then `ARCHMAX_PROMPT_CACHE`/`_TTL`. Turning it off does not stop `ChatAnthropic` or `ChatBedrockConverse` caching: Deep Agents installs LangChain's caching middleware for them itself (5-minute lifetime). |
|
|
112
|
-
| `pricing` | `PricingTable` | USD per 1M tokens, keyed by model id (`default` matches any): `{ input?, output?, cacheRead?, cacheWrite? }`. Looked up by the id the response reported, else the id the runtime was configured to run — so a keyed table prices a workload behind a proxy that echoes no model back. Input rate applies to the input tokens the cache did not serve. Makes `costUsd` appear on usage events, run metadata, and the CLI footer. Falls back to `ARCHMAX_PRICE_*`. Unpriced → tokens only. |
|
|
113
|
-
|
|
114
|
-
Fails closed: throws `WorkflowLoadError` if the named workflow's `workflow.yaml`
|
|
115
|
-
is missing or has no valid machine spec (never silently runs ungoverned; a
|
|
116
|
-
`WORKFLOW.md` carrying frontmatter with no `workflow.yaml` — the removed
|
|
117
|
-
single-file layout — is a load error with migration guidance).
|
|
118
|
-
|
|
119
|
-
## Agent
|
|
120
|
-
|
|
121
|
-
```ts
|
|
122
|
-
interface Agent {
|
|
123
|
-
graph: /* LangGraph runnable */;
|
|
124
|
-
workflow: string;
|
|
125
|
-
runtimeContract: ResolvedRuntimeContract;
|
|
126
|
-
resolveTrigger(trigger?: TriggerInput): ResolvedTrigger;
|
|
127
|
-
emitRunArtifacts(sessionId: string, trajectory: Trajectory): Promise<string | null>;
|
|
128
|
-
runs: { // run handles over the configured session store
|
|
129
|
-
list(): Promise<SessionSummary[]>; // [] when the store/checkpointer can't list
|
|
130
|
-
get(sessionId: string): Promise<SessionSummary | null>;
|
|
131
|
-
delete(sessionId: string): Promise<boolean>; // throws SessionStoreCapabilityError
|
|
132
|
-
};
|
|
133
|
-
decide(sessionId: string, resolution: DecisionResolution): Promise<DecideOutcome>;
|
|
134
|
-
dispose(sessionId: string): void;
|
|
135
|
-
}
|
|
136
|
-
```
|
|
137
|
-
|
|
138
|
-
## Running a workflow
|
|
139
|
-
|
|
140
|
-
```ts
|
|
141
|
-
import { createAgent, WORKFLOW_STATUSES } from "@archmax-ai/harness";
|
|
142
|
-
|
|
143
|
-
const agent = await createAgent({ workflow, onEvent, workspace: { rootDir } });
|
|
144
|
-
const sessionId = `run-${runId}`;
|
|
145
|
-
|
|
146
|
-
try {
|
|
147
|
-
const result = await agent.invoke(
|
|
148
|
-
{ messages: [{ role: "user", content: prompt }] },
|
|
149
|
-
{ configurable: { thread_id: sessionId } },
|
|
150
|
-
);
|
|
151
|
-
// result: { messages, status?, workflowState?, pendingDecision? }
|
|
152
|
-
if (result.status === WORKFLOW_STATUSES.awaitingDecision) {
|
|
153
|
-
// parked — see Human-in-the-loop
|
|
154
|
-
} else {
|
|
155
|
-
return lastAssistantText(result.messages);
|
|
156
|
-
}
|
|
157
|
-
} finally {
|
|
158
|
-
agent.dispose(sessionId); // frees the session's sandbox/PTC state and its checkpoints in memory; the next turn reloads them from the store
|
|
159
|
-
}
|
|
160
|
-
```
|
|
161
|
-
|
|
162
|
-
A turn runs under LangGraph's `recursionLimit` — 400 unless the config passes its
|
|
163
|
-
own, which wins. It is the one bound on a whole turn (every model call, tool batch
|
|
164
|
-
and automated state is a step; a state's `budget.maxTurns`/`maxParks` bound that
|
|
165
|
-
state); a turn that hits it ends the session `failed` with exit code
|
|
166
|
-
`recursion-limit`.
|
|
167
|
-
|
|
168
|
-
To cancel a turn, pass an `AbortSignal` as `signal` in the same config (or in
|
|
169
|
-
`agent.workflow.send(sessionId, input, { signal })`) and abort it. The turn's
|
|
170
|
-
scripts stop with it, in the session and its child sessions: a pending `tools.*`
|
|
171
|
-
call fails and later ones are refused. The `dispose` in the `finally` is safe even
|
|
172
|
-
while a cancelled script's call is still pending.
|
|
173
|
-
|
|
174
|
-
Use `agent.stream(...)` instead of `invoke` when you want to record a
|
|
175
|
-
trajectory or push incremental updates.
|
|
176
|
-
|
|
177
|
-
`lastAssistantText` walks `result.messages` from the end and returns the last
|
|
178
|
-
assistant/AI message's text.
|
|
179
|
-
|
|
180
|
-
## Multi-turn conversations (chat history)
|
|
181
|
-
|
|
182
|
-
`agent.invoke` accepts an **array of role-tagged messages**, not just one user
|
|
183
|
-
string — the `messages` channel is a LangChain `add_messages` channel, so
|
|
184
|
-
whatever you pass is appended to the session's transcript. There is no need to
|
|
185
|
-
concatenate a prior chat into one giant user prompt; give the model real turns.
|
|
186
|
-
|
|
187
|
-
**Preferred — session persistence.** Sessions are durable by default (the
|
|
188
|
-
`BackendCheckpointSaver` writes each turn to the session store). Drive a live
|
|
189
|
-
conversation by re-invoking the **same `thread_id`** once per user message,
|
|
190
|
-
sending only the new message each time — prior turns are already in the
|
|
191
|
-
checkpoint:
|
|
192
|
-
|
|
193
|
-
```ts
|
|
194
|
-
// turn 1
|
|
195
|
-
await agent.invoke(
|
|
196
|
-
{ messages: [{ role: "user", content: "Look up order 1234" }] },
|
|
197
|
-
{ configurable: { thread_id: "chat-42" } },
|
|
198
|
-
);
|
|
199
|
-
// turn 2 — SAME thread_id; only the new message
|
|
200
|
-
await agent.invoke(
|
|
201
|
-
{ messages: [{ role: "user", content: "And when did it ship?" }] },
|
|
202
|
-
{ configurable: { thread_id: "chat-42" } },
|
|
203
|
-
);
|
|
204
|
-
```
|
|
205
|
-
|
|
206
|
-
The `init` state re-runs on each turn, resetting `workflowState` to the trigger's
|
|
207
|
-
start state while **preserving the accumulated messages** — exactly right for
|
|
208
|
-
chat: each user message re-enters the workflow with full history retained.
|
|
209
|
-
|
|
210
|
-
**Alternative — seed prior history on the first invoke.** When the transcript
|
|
211
|
-
lives in your own store (not a harness session), pass it as the initial `messages`
|
|
212
|
-
array with the newest user turn last:
|
|
213
|
-
|
|
214
|
-
```ts
|
|
215
|
-
await agent.invoke(
|
|
216
|
-
{
|
|
217
|
-
messages: [
|
|
218
|
-
{ role: "user", content: "Test" },
|
|
219
|
-
{ role: "assistant", content: "Test complete…" },
|
|
220
|
-
{ role: "user", content: "who are you?" },
|
|
221
|
-
{ role: "assistant", content: "I'm an agent that…" },
|
|
222
|
-
{ role: "user", content: "what was my first message?" }, // the new turn
|
|
223
|
-
],
|
|
224
|
-
},
|
|
225
|
-
{ configurable: { thread_id: "chat-42" } },
|
|
226
|
-
);
|
|
227
|
-
```
|
|
24
|
+
Install `@archmax-ai/harness`, plus `langchain` and `zod` to write host tools. It needs
|
|
25
|
+
Node 22 or later. Before a release is published, `npm pack` the harness and install the
|
|
26
|
+
tarball: a real copy behaves like production, and a symlink does not.
|
|
228
27
|
|
|
229
|
-
|
|
230
|
-
|
|
231
|
-
|
|
232
|
-
|
|
233
|
-
`JSON.parse(await tools.archmax_get_history())`). Its `messages` are the whole
|
|
234
|
-
transcript, oldest first, which can span turns: a script that keys off "the
|
|
235
|
-
current request" should take the *last* `user` entry, not the first — the
|
|
236
|
-
first is the message that opened the session. A tool entry carries its tool,
|
|
237
|
-
the call `id` and the result as its `text`. Neither's own calls or messages
|
|
238
|
-
enter the session's transcript.
|
|
239
|
-
- **A definition edit is adopted, not refused.** A session takes its next turn
|
|
240
|
-
under the `workflow.yaml` in force now; the recorded `specHash` says which
|
|
241
|
-
definition each turn ran under but gates nothing. A turn positioned in a state
|
|
242
|
-
the edit removed reopens at its trigger's entry state rather than failing.
|
|
243
|
-
- **Seed assistant turns as text only.** Don't fabricate dangling `tool_calls`
|
|
244
|
-
in seeded history, or the provider may reject the turn.
|
|
245
|
-
- **Not every message in a session's transcript is a person's.** The runtime
|
|
246
|
-
writes **runtime notes** of its own — `[decision]`, `[event]`, `[error]`,
|
|
247
|
-
`[grader]`, `[code]`, `[sub-workflow: …]` — carried as a synthetic `archmax_note` tool call
|
|
248
|
-
and its result, so a model cannot read them as a person speaking. Ask
|
|
249
|
-
`isRuntimeNote(message)` / `runtimeNoteKind(message)` rather than inferring
|
|
250
|
-
authorship from role or a `[bracket]` prefix: a turn's lower bound is a human
|
|
251
|
-
message **or** a note, and a transcript renderer should draw a note as a
|
|
252
|
-
runtime line, not as the user's own bubble. The one message a person really
|
|
253
|
-
sends into a running session — a reply to a run parked at a human state — is an
|
|
254
|
-
ordinary human message with no marker.
|
|
28
|
+
This skill ships inside the package. `BUNDLED_AUTHORING_SKILL_DIR` is the directory that
|
|
29
|
+
holds it (`<dir>/archmax-harness/SKILL.md`), and
|
|
30
|
+
`npx skills add ./node_modules/@archmax-ai/harness` installs the copy that matches the
|
|
31
|
+
installed version.
|
|
255
32
|
|
|
256
|
-
|
|
257
|
-
import { isRuntimeNote, runtimeNoteKind } from "@archmax-ai/harness";
|
|
33
|
+
## The smallest program
|
|
258
34
|
|
|
259
|
-
|
|
260
|
-
|
|
261
|
-
|
|
35
|
+
[`examples/refund/`](https://github.com/archmax-ai/harness/tree/main/examples/refund)
|
|
36
|
+
runs a four-state workflow with two host tools. A host tool is an ordinary LangChain
|
|
37
|
+
tool, built with `tool(fn, { name, description, schema })` from `langchain`, and a state
|
|
38
|
+
grants it in `tools.allow`. The program, `index.js`:
|
|
262
39
|
|
|
263
|
-
|
|
40
|
+
```js
|
|
41
|
+
import { createAgent } from "@archmax-ai/harness";
|
|
42
|
+
import { lookupCustomer, sendEmail } from "./tools.js";
|
|
264
43
|
|
|
265
|
-
|
|
266
|
-
|
|
267
|
-
|
|
268
|
-
|
|
269
|
-
|
|
270
|
-
// agent attached via `archmax_advance({ evidence })` — one merged list, no
|
|
271
|
-
// host-side assembly, and no need to re-read the spec.
|
|
272
|
-
// When they pick:
|
|
273
|
-
const outcome = await agent.workflow.decide(sessionId, { target: chosenTo, comment });
|
|
274
|
-
// DecideOutcome: { status?, workflowState?, reparked, state?, reply, rejected?, exit?, messages }
|
|
275
|
-
// `reply` is every assistant text of the turn, in order ("" when it said nothing).
|
|
276
|
-
// `rejected` says why when status is "rejected"; Outcome from send() carries it too.
|
|
277
|
-
// `exit` is { success: false, code, reason } when the agent ended the run with
|
|
278
|
-
// archmax_raise (status "failed"), { success: true } when it completed.
|
|
279
|
-
if (outcome.reparked) { /* parked again at another human state */ }
|
|
280
|
-
}
|
|
281
|
-
```
|
|
282
|
-
|
|
283
|
-
`decide` throws `SessionNotParkedError` if the session isn't awaiting a decision, and
|
|
284
|
-
`InvalidDecisionTargetError` if `target` isn't a declared transition of the parked
|
|
285
|
-
state. `sessions.list()` enumerates durable sessions (empty with a `MemorySaver` or a non-listable session store).
|
|
286
|
-
|
|
287
|
-
## Event-in-the-loop: a run that parked itself
|
|
288
|
-
|
|
289
|
-
The agent can park the run with `archmax_wait({ reason })` — in any state, terminal
|
|
290
|
-
ones included — when the work cannot continue without something from outside. The
|
|
291
|
-
session's status becomes `awaiting_input` (classified **open**, never finished) and
|
|
292
|
-
`result.pendingInput` carries `{ state, title?, reason, parkedAt }`: the state it
|
|
293
|
-
stopped in and, in its own words, what it is waiting for.
|
|
294
|
-
|
|
295
|
-
```ts
|
|
296
|
-
if (result.status === WORKFLOW_STATUSES.awaitingInput && result.pendingInput) {
|
|
297
|
-
const { state, reason } = result.pendingInput; // show "waiting in <state>: <reason>"
|
|
298
|
-
}
|
|
44
|
+
const agent = await createAgent({
|
|
45
|
+
workflow: "refund", // workflows/refund/
|
|
46
|
+
tools: [lookupCustomer, sendEmail], // host tools
|
|
47
|
+
workspace: { rootDir: import.meta.dirname }, // sessions are stored under sessions/
|
|
48
|
+
});
|
|
299
49
|
|
|
300
|
-
|
|
301
|
-
|
|
302
|
-
|
|
303
|
-
trigger: { id: "email_reply" },
|
|
304
|
-
variables: { reply_body: "It's ORD-1001." },
|
|
50
|
+
const outcome = await agent.workflow.send("ticket-1001", {
|
|
51
|
+
message: "Hi, I'd like a refund for my last order.",
|
|
52
|
+
variables: { email: "ann@example.com" }, // the trigger's requires, seeded locked
|
|
305
53
|
});
|
|
306
|
-
//
|
|
307
|
-
if (outcome.reparked) { /* parked again — outcome.parkedChannel says which channel */ }
|
|
54
|
+
console.log(outcome.kind); // "completed"
|
|
308
55
|
```
|
|
309
56
|
|
|
310
|
-
|
|
311
|
-
|
|
312
|
-
|
|
313
|
-
|
|
314
|
-
|
|
315
|
-
|
|
316
|
-
|
|
317
|
-
```ts
|
|
318
|
-
const now = new Date().toISOString();
|
|
319
|
-
for (const t of await agent.sessions.list()) {
|
|
320
|
-
if (t.status === WORKFLOW_STATUSES.awaitingInput && t.resumeAt && t.resumeAt <= now) {
|
|
321
|
-
await agent.workflow.deliver(t.sessionId, { trigger: { id: "timer" }, variables: { now } });
|
|
322
|
-
}
|
|
323
|
-
}
|
|
324
|
-
```
|
|
57
|
+
- One `send` is one turn. The session id is yours to choose (a ticket, a conversation).
|
|
58
|
+
`send` with the same id continues the session, and each turn carries only the new
|
|
59
|
+
message: the session already holds the transcript.
|
|
60
|
+
- `send` works out what the session needs: a new turn for a new or finished session, a
|
|
61
|
+
reply for one parked at a human state, a delivery for one parked with `archmax_wait`.
|
|
62
|
+
- In TypeScript, `agent.workflow` is `undefined` on an ungoverned agent: narrow it
|
|
63
|
+
before you call it.
|
|
325
64
|
|
|
326
|
-
|
|
327
|
-
`resumeAt` gates nothing — an event arriving earlier resumes the park normally — and
|
|
328
|
-
a state that polls can cap its own retry loop with `budget.maxParks`, whose overrun
|
|
329
|
-
routes through the state's `on_error`.
|
|
330
|
-
|
|
331
|
-
`deliver` throws `SessionNotAwaitingInputError` if the session isn't parked awaiting
|
|
332
|
-
an event and `MissingDeliveryTriggerError` if no id is supplied. The id is
|
|
333
|
-
**recorded, never matched**: any trigger resumes a park, so there is no awaited-id
|
|
334
|
-
list to check against — an id no state declares is deliverable exactly like a
|
|
335
|
-
declared one. `resolveSession({ trigger, variables })` tells you which of
|
|
336
|
-
the two a firing is — a `turn` to invoke, or a `resume` to deliver into — always on
|
|
337
|
-
the conversation's own id.
|
|
338
|
-
|
|
339
|
-
An event that must **never start** a run is therefore declared nowhere in the spec:
|
|
340
|
-
every trigger a state declares starts a session at that state. Name the session
|
|
341
|
-
yourself — `send(sessionId, { delivery })` / `deliver(sessionId, …)` — or pass
|
|
342
|
-
`sessionPath` with the firing (`resolveSession({ trigger, variables, sessionPath })`)
|
|
343
|
-
and let the runtime read the id out of the firing's own variables. A firing that
|
|
344
|
-
names neither, on a trigger no state declares, throws
|
|
345
|
-
`UnknownSessionTriggerError`: there is nowhere for it to go.
|
|
346
|
-
|
|
347
|
-
One turn per invoke, on **one** session. A finished session re-invoked with a new
|
|
348
|
-
message takes its next turn on the same id, continuing in the state the previous
|
|
349
|
-
turn ended in — the position, variables, transcript and files are already there, so
|
|
350
|
-
**never seed a previous transcript into the `messages` channel**. Pass only the new
|
|
351
|
-
message.
|
|
352
|
-
|
|
353
|
-
The runtime does not reposition a later turn, so nothing is injected into the
|
|
354
|
-
transcript to explain a jump. When the current state can serve the new message, the
|
|
355
|
-
agent answers there without a reset. That includes a terminal state that is also the
|
|
356
|
-
state the conversation began in, such as a one-state chat workflow. When a turn needs
|
|
357
|
-
a path the current state cannot reach, the agent calls `archmax_reset` to return to
|
|
358
|
-
the state the conversation began in (available in every state). To continue *mid-state* instead — inside the state
|
|
359
|
-
that stopped, with the arrival in its transcript — the turn must park rather than
|
|
360
|
-
finish.
|
|
361
|
-
|
|
362
|
-
## Exposing a workflow to outside callers
|
|
363
|
-
|
|
364
|
-
A workflow's trigger signature (`requires`/`returns`, optionally typed) is the
|
|
365
|
-
contract the runtime already enforces at the session boundary. A host that
|
|
366
|
-
exposes a workflow as an MCP tool, an OpenAPI operation or a typed start form
|
|
367
|
-
builds its schema **from that signature**, with the helpers on the browser-safe
|
|
368
|
-
`@archmax-ai/harness/spec` subpath (also on the root, as the same bindings), so
|
|
369
|
-
the published contract and the enforced one cannot drift:
|
|
65
|
+
## `createAgent` options
|
|
370
66
|
|
|
371
|
-
|
|
372
|
-
|
|
373
|
-
|
|
374
|
-
signatureForTrigger,
|
|
375
|
-
signatureJsonSchema,
|
|
376
|
-
signatureValueIssues,
|
|
377
|
-
} from "@archmax-ai/harness/spec";
|
|
378
|
-
|
|
379
|
-
const parsed = parseMachineSpec(yaml.parse(workflowYaml));
|
|
380
|
-
if (!parsed.ok) throw new Error(parsed.issues.map((i) => `${i.path}: ${i.message}`).join("\n"));
|
|
381
|
-
const signature = signatureForTrigger(parsed.spec, "manual"); // { description?, requires, returns }
|
|
382
|
-
|
|
383
|
-
// MCP tool (or an OpenAPI request/response body): the same mapping the delegation tool uses.
|
|
384
|
-
const mcpTool = {
|
|
385
|
-
name: "refund-order",
|
|
386
|
-
description: signature?.description, // caller-facing; never shown to the session's agent
|
|
387
|
-
inputSchema: signatureJsonSchema(signature?.requires ?? []),
|
|
388
|
-
outputSchema: signatureJsonSchema(signature?.returns ?? []),
|
|
389
|
-
};
|
|
390
|
-
|
|
391
|
-
// On a request, before any session opens: the rule the turn boundary applies.
|
|
392
|
-
const issues = signatureValueIssues(signature?.requires ?? [], body);
|
|
393
|
-
if (issues.length) return reply(400, issues.map((i) => i.message)); // { kind, name, type?, found?, message }
|
|
394
|
-
|
|
395
|
-
// A turn needs a message; an MCP call has none of its own, so say what was asked.
|
|
396
|
-
const outcome = await agent.workflow!.send(sessionId, { message: "Refund the order.", variables: body });
|
|
397
|
-
```
|
|
67
|
+
Deep Agents' own options keep their names: `model`, `tools`, `systemPrompt`,
|
|
68
|
+
`middleware`, `backend`, `checkpointer`, `store`, `skills`, `interruptOn`, `name`,
|
|
69
|
+
`description`. The harness adds:
|
|
398
70
|
|
|
399
|
-
|
|
400
|
-
one property per entry in declaration order, every name required,
|
|
401
|
-
`date`/`date-time`/`file` as `{ type: "string", format }`, an untyped entry as `{}`,
|
|
402
|
-
an entry's `description` carried over. Rules map to their JSON Schema keywords:
|
|
403
|
-
`format: email` → `format: "email"`, `format: url` → `format: "uri"` plus
|
|
404
|
-
`pattern: "^https?://"`, `format: phone` → the E.164 `pattern`; `pattern`,
|
|
405
|
-
`minLength`/`maxLength`, `enum`, numeric `minimum`/`maximum` and
|
|
406
|
-
`minItems`/`maxItems` as themselves; `content` → `contentMediaType`. Date
|
|
407
|
-
bounds, `accept` and `maxSize` are not mapped (the runtime still enforces
|
|
408
|
-
them). It adds no `additionalProperties`; set it for your own ingress.
|
|
409
|
-
- `signatureValueIssues(entries, values, options?)` → one issue per missing name
|
|
410
|
-
and per non-conforming value; `[]` means the turn boundary will not refuse the
|
|
411
|
-
map for its signature. A value breaking a rule is an `invalid` issue carrying
|
|
412
|
-
`rule` (one per entry, the first rule broken), so code that filters
|
|
413
|
-
`kind === "invalid"` enforces rules unchanged. `options.now` pins the clock
|
|
414
|
-
for `today`/`now` (tests, a host's own clock). It is synchronous, pure and
|
|
415
|
-
browser-safe, and a `pattern` runs in RE2, so it is safe in a request path on
|
|
416
|
-
input from anyone. It **never coerces or trims**: a form that collects text
|
|
417
|
-
converts `"4"` to `4` before it validates or seeds.
|
|
418
|
-
- A trigger's `description` is written for the caller. Use it as the MCP tool
|
|
419
|
-
or operation description. The session's own model never sees it.
|
|
420
|
-
- **A `file` entry** (`format: "file"` in the schema) takes a workspace path. To
|
|
421
|
-
hand a session an uploaded file, pass it on the turn rather than writing into
|
|
422
|
-
the store yourself:
|
|
423
|
-
`send(sessionId, { message, files: { invoice: { name: "invoice.pdf", content: bytes } } })`.
|
|
424
|
-
The runtime places the bytes in the session's `scratchpad/attachments/` (a taken
|
|
425
|
-
name gets `-1`, `-2`, …), seeds `invoice`, locked, with the placed path, and the
|
|
426
|
-
turn boundary then checks that the file is there and readable. `send` throws
|
|
427
|
-
`InvalidTurnFileError`, before anything is written, for a bad variable or file
|
|
428
|
-
name, a variable also given in `variables`, a session parked at a human state, or
|
|
429
|
-
bytes the store cannot hold exactly (a non-text file needs a store with
|
|
430
|
-
`uploadFiles`). `signatureValueIssues` checks only a `file` value's shape and its
|
|
431
|
-
`accept` extensions, so a start form validates the path the upload will
|
|
432
|
-
produce, or skips the entry and lets `files` supply it; `maxSize` is checked by
|
|
433
|
-
the runtime at the turn boundary, against the placed file.
|
|
434
|
-
- A host that reads specs itself must read `requires`/`returns` through
|
|
435
|
-
`normalizeSignature` (both spellings → `{ name, type?, description?, …rules }[]`,
|
|
436
|
-
`maxSize` in bytes, `today`/`now` kept symbolic), never as `string[]`: an entry
|
|
437
|
-
may be an object. Hosts must be on the release that introduced typed entries
|
|
438
|
-
(and, for rules, 1.1.0) before any workspace they read uses them.
|
|
439
|
-
|
|
440
|
-
## Lifecycle events
|
|
441
|
-
|
|
442
|
-
Pass `onEvent`; every `WorkflowLifecycleEvent` carries a `level`
|
|
443
|
-
(`info`/`warn`/`error`) plus a typed payload. Map these to a live run UI:
|
|
444
|
-
|
|
445
|
-
| Event `type` | Meaning |
|
|
71
|
+
| Option | What it does |
|
|
446
72
|
| --- | --- |
|
|
447
|
-
| `
|
|
448
|
-
| `
|
|
449
|
-
| `
|
|
450
|
-
| `
|
|
451
|
-
| `
|
|
452
|
-
| `
|
|
453
|
-
| `
|
|
454
|
-
| `
|
|
455
|
-
| `
|
|
456
|
-
| `
|
|
457
|
-
| `
|
|
458
|
-
| `
|
|
459
|
-
| `
|
|
460
|
-
|
|
461
|
-
|
|
462
|
-
|
|
463
|
-
|
|
464
|
-
|
|
465
|
-
|
|
466
|
-
|
|
467
|
-
|
|
468
|
-
|
|
469
|
-
`
|
|
470
|
-
|
|
471
|
-
|
|
472
|
-
`
|
|
473
|
-
|
|
474
|
-
|
|
475
|
-
|
|
476
|
-
|
|
477
|
-
|
|
478
|
-
`
|
|
479
|
-
|
|
480
|
-
|
|
481
|
-
|
|
482
|
-
|
|
483
|
-
|
|
484
|
-
|
|
485
|
-
|
|
486
|
-
|
|
487
|
-
|
|
488
|
-
|
|
489
|
-
Use the exported tracker instead of summing events yourself:
|
|
490
|
-
|
|
491
|
-
```ts
|
|
492
|
-
import { createAgent, createUsageTracker } from "@archmax-ai/harness";
|
|
493
|
-
|
|
494
|
-
// Wrap your own subscriber, or omit `onEvent` if you have none.
|
|
495
|
-
const usage = createUsageTracker({ onEvent: forwardToClient });
|
|
496
|
-
const agent = await createAgent({ onEvent: usage.handler, /* … */ });
|
|
497
|
-
|
|
498
|
-
await agent.invoke(input, { configurable: { thread_id: sessionId } });
|
|
499
|
-
|
|
500
|
-
const totals = usage.totals(sessionId);
|
|
501
|
-
// { inputTokens, outputTokens, cacheReadTokens, cacheCreationTokens, costUsd? }
|
|
502
|
-
usage.bySession(); // Map<sessionId, UsageSummary> for concurrent runs
|
|
503
|
-
usage.reset(sessionId); // drop a finished run's totals
|
|
504
|
-
```
|
|
505
|
-
|
|
506
|
-
Rising `cacheReadTokens` is how you confirm prompt caching is actually working;
|
|
507
|
-
`costUsd` is present only when pricing is configured — report tokens rather than a
|
|
508
|
-
guessed cost when it is absent. `formatUsage`, `addSummaries`, `emptyUsage` and
|
|
509
|
-
`hasUsage` are exported for rendering and combining summaries; the extraction and
|
|
510
|
-
pricing internals are not part of the public surface.
|
|
73
|
+
| `workflow` | The workflow's slug under `workflows/`. When omitted, the default workflow governs the agent if it exists; `false` assembles a plain Deep Agent. |
|
|
74
|
+
| `workspace` | `{ rootDir, mounts?, sessionStore? }`. `rootDir` defaults to the current directory. |
|
|
75
|
+
| `authoring` | The backend that serves `workflows/**` (specs, `WORKFLOW.md`, code-state scripts, cases) to the runtime only. It defaults to `backend`, else the files under `rootDir`. Exposing it through a writable mount throws `AuthoringBackendExposedError`. |
|
|
76
|
+
| `trigger`, `variables`, `sessionPath` | The default trigger, seeded variables (locked) and session path for turns. |
|
|
77
|
+
| `modelFactory` | `(role, env, requested?) => model`, called per role: `agent`, `judge` (a case's `grade`) and `grader` (a grader state, asked with the state's model id). |
|
|
78
|
+
| `essentialTools` | Host tools that are on in every state, whatever `allow_always` says. A state's entry still narrows one. |
|
|
79
|
+
| `toolPaths` | Each host tool's path arguments and how it uses them (`{ get_markdown: { path: "read" } }`), governed like a built-in tool's. |
|
|
80
|
+
| `images` | Shows a vision model the images it reads with `read_file`. Off by default. |
|
|
81
|
+
| `policyRules`, `sandboxRuntime` | Extra governance rules, and a custom sandbox runtime. |
|
|
82
|
+
| `onEvent` | Receives the event stream, in place of the console subscriber. |
|
|
83
|
+
| `promptCache` | `{ enabled, ttl: "5m" \| "1h" }`, on by default. |
|
|
84
|
+
| `pricing` | USD per million tokens by model id (`{ input, output, cacheRead, cacheWrite }`; `default` matches any id). Without it, usage reports tokens and no cost. |
|
|
85
|
+
| `clock` | Pins "now" for cases. Not for production. |
|
|
86
|
+
|
|
87
|
+
Assembly fails closed. A missing or invalid spec throws `WorkflowLoadError`, a custom
|
|
88
|
+
`backend` without a `sessionStore` throws `SessionStoreRequiredError`, and a host tool
|
|
89
|
+
named `archmax_*`, or after a file operation, throws `ReservedToolNameError`.
|
|
90
|
+
|
|
91
|
+
## Models
|
|
92
|
+
|
|
93
|
+
By default the model comes from the environment, for any OpenAI-compatible endpoint:
|
|
94
|
+
`ARCHMAX_API_BASE_URL`, `ARCHMAX_API_KEY` and `ARCHMAX_MODEL`, optionally
|
|
95
|
+
`ARCHMAX_TEMPERATURE`, `ARCHMAX_MAX_TOKENS` and `ARCHMAX_STREAMING`. They are read from
|
|
96
|
+
the workspace's `.env`, and the process environment wins.
|
|
97
|
+
|
|
98
|
+
Pass `model` to use your own instance; it then outranks every model id a spec declares.
|
|
99
|
+
Pass `modelFactory` instead to honour those ids, and to put grader states on their own
|
|
100
|
+
model, as the service below does.
|
|
101
|
+
|
|
102
|
+
## Outcomes and parked sessions
|
|
103
|
+
|
|
104
|
+
`send`, `decide`, `reply` and `deliver` each settle one turn and return what it did:
|
|
105
|
+
|
|
106
|
+
| Field | Meaning |
|
|
107
|
+
| --- | --- |
|
|
108
|
+
| `kind` | `completed`; `parked` (waiting on a person or an event); `failed` (the agent, a grader or a code state called `archmax_raise`, or the turn threw); `rejected` (governance ended it). |
|
|
109
|
+
| `exit` | `{ success: true }` when completed, `{ success: false, code, reason }` when failed. |
|
|
110
|
+
| `reply` | Everything the agent said this turn, in order; `""` when it said nothing. |
|
|
111
|
+
| `state`, `parkedChannel`, `pending` | Where the session is. For a park: `"decision"` or `"input"`, and the park record. |
|
|
112
|
+
| `rejected` | Why, when the outcome is `rejected`. |
|
|
113
|
+
| `delegation` | When a sub-workflow child is the one parked: which child, and which call. |
|
|
114
|
+
| `auditTrail`, `variables`, `usage`, `messages` | The trail, the variables, the cumulative usage and the transcript. |
|
|
511
115
|
|
|
512
|
-
|
|
116
|
+
A host that reads "not rejected" as success must check for `failed` too.
|
|
513
117
|
|
|
514
|
-
A
|
|
515
|
-
parked and resumed — including in another process:
|
|
118
|
+
A service that waits for people:
|
|
516
119
|
|
|
517
120
|
```ts
|
|
518
|
-
|
|
519
|
-
summary?.usage;
|
|
520
|
-
// { inputTokens, outputTokens, cacheReadTokens, cacheCreationTokens, costUsd? }
|
|
521
|
-
```
|
|
522
|
-
|
|
523
|
-
`costUsd` appears only when pricing is configured; without it, tokens are
|
|
524
|
-
reported and cost is omitted, never guessed. A parked run reports what it has
|
|
525
|
-
spent so far and keeps accumulating on resume. The same totals land in
|
|
526
|
-
`sessions/<sessionId>/artifacts/metadata.json` as `usage`, and `createUsageTracker()`
|
|
527
|
-
gives the same figure live in process.
|
|
528
|
-
|
|
529
|
-
### Measuring anything else — fold the stream
|
|
121
|
+
import { createAgent, createChatModel, createUsageTracker } from "@archmax-ai/harness";
|
|
530
122
|
|
|
531
|
-
|
|
532
|
-
took, every send-back by a grader or code state, and how long a session
|
|
533
|
-
sat waiting for a person are all already on the typed event stream, and a backend
|
|
534
|
-
counts what it needs:
|
|
535
|
-
|
|
536
|
-
```ts
|
|
537
|
-
const tally = { toolCalls: 0, toolBlocks: 0, path: [] as string[] };
|
|
123
|
+
const usage = createUsageTracker({ onEvent: publish }); // publish: your event fan-out
|
|
538
124
|
const agent = await createAgent({
|
|
539
|
-
|
|
540
|
-
|
|
541
|
-
|
|
542
|
-
|
|
543
|
-
|
|
544
|
-
|
|
545
|
-
/* … */
|
|
125
|
+
workflow: "order-lookup",
|
|
126
|
+
workspace: { rootDir },
|
|
127
|
+
tools: hostTools,
|
|
128
|
+
onEvent: usage.handler,
|
|
129
|
+
modelFactory: (role, env, requested) =>
|
|
130
|
+
role === "grader" ? graderModel : createChatModel({ ...env(), model: requested ?? env().model }),
|
|
546
131
|
});
|
|
547
|
-
|
|
548
|
-
|
|
549
|
-
Two properties make this the right home for it: the events are the ones the
|
|
550
|
-
runtime acts on, so a fold cannot drift from behavior the way a parallel ledger
|
|
551
|
-
can; and a backend counts what it cares about — a per-tenant tool budget, a
|
|
552
|
-
per-state SLA — rather than what the SDK guessed.
|
|
553
|
-
|
|
554
|
-
A fold is a **live view**: it sees only events this process observed. Cost is the
|
|
555
|
-
deliberate exception, because it is checkpointed.
|
|
556
|
-
|
|
557
|
-
## Session artifacts
|
|
558
|
-
|
|
559
|
-
Each run persists under its own folder in the session store — which is also the
|
|
560
|
-
agent's workspace root for that run (`sessions/<sessionId>/` on the filesystem default):
|
|
561
|
-
|
|
562
|
-
```
|
|
563
|
-
sessions/<sessionId>/
|
|
564
|
-
checkpoints/ # LangGraph checkpoint records (durable resume; also
|
|
565
|
-
# the sole record of a parked human decision)
|
|
566
|
-
scratchpad/ # the run's working area — intermediate files AND the
|
|
567
|
-
# artifacts a state produces (agent-visible as
|
|
568
|
-
# `scratchpad/…`; the only such area)
|
|
569
|
-
large_tool_results/ # runtime-offloaded oversized tool results (agent reads only)
|
|
570
|
-
conversation_history/ # runtime-offloaded history (agent reads only)
|
|
571
|
-
artifacts/
|
|
572
|
-
graph.json # SerializedGraph (topology; includes mermaid)
|
|
573
|
-
graph.mmd # Mermaid diagram
|
|
574
|
-
trajectory.json # Trajectory (per-state messages + tool calls)
|
|
575
|
-
trail.json # Audit trail (committed transitions + rationale)
|
|
576
|
-
variables.json # The run's variables: name -> { value, locked }.
|
|
577
|
-
# `locked` is the attribution: locked = host-established
|
|
578
|
-
# (variables, a delivery's seeds, the built-in
|
|
579
|
-
# `trigger`), unlocked = set by the agent. Written for a
|
|
580
|
-
# governed run; an empty store still writes `{}`.
|
|
581
|
-
metadata.json # RunMetadata (runtime contract, package version,
|
|
582
|
-
# specHash, and the run's `usage` summary — tokens
|
|
583
|
-
# by kind plus costUsd when priced)
|
|
584
|
-
```
|
|
585
|
-
|
|
586
|
-
Schemas:
|
|
132
|
+
const flow = agent.workflow!;
|
|
587
133
|
|
|
588
|
-
|
|
589
|
-
|
|
590
|
-
|
|
591
|
-
|
|
592
|
-
entry?: string; finals: string[]; mermaid: string;
|
|
593
|
-
}
|
|
594
|
-
interface Trajectory {
|
|
595
|
-
sessionId?: string; workflow: string; prompt?: string; finalAnswer: string;
|
|
596
|
-
segments: { state: string; enteredState?: string; exitedState?: string;
|
|
597
|
-
items: (MessageItem | ToolCallItem)[] }[];
|
|
134
|
+
const outcome = await flow.send(sessionId, { message });
|
|
135
|
+
if (outcome.kind === "parked" && outcome.parkedChannel === "decision") {
|
|
136
|
+
// outcome.pending.transitions: [{ to, description, type }], one button each
|
|
137
|
+
await flow.decide(sessionId, { target: chosen.to, comment });
|
|
598
138
|
}
|
|
599
|
-
//
|
|
600
|
-
//
|
|
601
|
-
interface RunMetadata { sessionId: string; workflow: string;
|
|
602
|
-
runtimeContract: ResolvedRuntimeContract; packageVersion: string; }
|
|
603
|
-
// trail.json: { sessionId, workflow, steps: TrailStep[] } — the audit trail,
|
|
604
|
-
// re-derived from checkpointed state on every artifact write.
|
|
605
|
-
// TrailStep: { to, kind: "trigger"|"agent"|"human"|"grader"|"code"|
|
|
606
|
-
// "sub-workflow"|"reset"|"on_error", reason?, workflow?, status?, ts }
|
|
139
|
+
usage.totals(sessionId); // tokens, plus costUsd when pricing is set
|
|
140
|
+
agent.dispose(sessionId); // frees the session's in-memory resources
|
|
607
141
|
```
|
|
608
142
|
|
|
609
|
-
|
|
610
|
-
|
|
611
|
-
`
|
|
612
|
-
|
|
613
|
-
|
|
614
|
-
|
|
615
|
-
|
|
616
|
-
|
|
617
|
-
|
|
618
|
-
|
|
619
|
-
|
|
620
|
-
|
|
621
|
-
|
|
622
|
-
|
|
623
|
-
|
|
624
|
-
|
|
625
|
-
|
|
143
|
+
- **`decide(sessionId, { target, comment? })`** moves a session parked at a human state
|
|
144
|
+
along the chosen transition. It throws `SessionNotParkedError` when the session awaits
|
|
145
|
+
no decision, and `InvalidDecisionTargetError` for a target that is not one of the
|
|
146
|
+
state's transitions.
|
|
147
|
+
- **`reply(sessionId, message)`** answers a person who writes to a parked session. The
|
|
148
|
+
session stays parked.
|
|
149
|
+
- **`deliver(sessionId, { trigger: { id }, variables? })`** continues a session parked
|
|
150
|
+
with `archmax_wait`, in the state that parked, with the variables locked. Any trigger
|
|
151
|
+
id continues a park. It throws `SessionNotAwaitingInputError` when nothing waits.
|
|
152
|
+
- **The same operations** are payloads of `send`: `{ decision }` and `{ delivery }`.
|
|
153
|
+
- **Scheduled parks.** A park with `until` carries `resumeAt`. The harness schedules
|
|
154
|
+
nothing: your scheduler lists `agent.sessions.list()`, picks the parks whose
|
|
155
|
+
`resumeAt` has passed, and delivers to them. A duplicate delivery is refused.
|
|
156
|
+
- **`resolveSession({ trigger, variables, sessionPath? })`** says which session a
|
|
157
|
+
trigger's arrival belongs to, and whether it is a turn or a delivery, without sending
|
|
158
|
+
it.
|
|
159
|
+
- **Files.** `send(id, { message, files: { invoice: { name, content } } })` places the
|
|
160
|
+
bytes under the session's `scratchpad/attachments/` and seeds `invoice`, locked, with
|
|
161
|
+
the path: what a `file` input takes.
|
|
162
|
+
- **A turn that throws** (a provider failure, a cancellation, the recursion limit)
|
|
163
|
+
rejects with the error, and the session ends `failed` with `turn-error`,
|
|
164
|
+
`turn-cancelled` or `recursion-limit`. The `recursionLimit` is 400 unless your config
|
|
165
|
+
sets one; `send`'s third argument merges into the invoke config. Cancel a turn with an
|
|
166
|
+
`AbortSignal` passed as `{ signal }`: its scripts stop with it. `dispose` in a
|
|
167
|
+
`finally` is always safe.
|
|
168
|
+
|
|
169
|
+
## Events
|
|
170
|
+
|
|
171
|
+
`onEvent` receives one typed stream, `WorkflowLifecycleEvent`, each event carrying its
|
|
172
|
+
`sessionId`:
|
|
173
|
+
|
|
174
|
+
| Event | Meaning |
|
|
175
|
+
| --- | --- |
|
|
176
|
+
| `state-enter`, `state-leave`, `advance` | The session's movement. `advance` carries `from`, `to`, `reason` and `callId`. |
|
|
177
|
+
| `code-start`, `code-output`, `code-result` | A code state's run: its script, each `console.log` line, its `status` and `reason`. |
|
|
178
|
+
| `grader-start`, `grader-result` | A grader state's run. |
|
|
179
|
+
| `parked`, `decided`, `delivered`, `raised`, `state-error-routed` | Parks, decisions, deliveries, raises and `on_error` routes. |
|
|
180
|
+
| `tool-called`, `tool-result`, `tool-blocked` | Every governed call, `archmax_advance` included, grouped by `callId`. A code state's calls carry `origin: "script"`. |
|
|
181
|
+
| `agent-text-delta`, `agent-text` | Streamed text and the complete message, grouped by `messageId`. A grader's tokens never stream. |
|
|
182
|
+
| `model-usage` | The provider's token counts per model call (a grader's under its state), with `costUsd` when priced. |
|
|
183
|
+
| `variables-set`, `title-set` | Variable writes, by name; the session's `title`, with its value. |
|
|
184
|
+
| `sub-workflow-start`, `sub-workflow-result` | A child session's dispatch and its result. |
|
|
185
|
+
| `warning`, `skills-loaded`, `graph-topology`, `prompt-shaping` | Diagnostics. |
|
|
186
|
+
|
|
187
|
+
There is no end-of-turn event: `send` returning is the end. The ids join events to the
|
|
188
|
+
stored messages: `callId` is the tool call's id, and `messageId` the message's.
|
|
189
|
+
|
|
190
|
+
The harness keeps no metrics ledger: count what you need by folding the stream. Cost is
|
|
191
|
+
the exception. Usage is checkpointed, so `(await agent.sessions.get(id))?.usage` answers
|
|
192
|
+
it for any session.
|
|
193
|
+
|
|
194
|
+
The runtime writes its own messages into the transcript, as tool-call pairs: runtime
|
|
195
|
+
notes such as `[decision]`, `[event]`, `[error]`, `[grader]` and `[code]`. Tell them
|
|
196
|
+
apart with `isRuntimeNote(message)` and `runtimeNoteKind(message)`, never by role or by
|
|
197
|
+
a bracket prefix.
|
|
198
|
+
|
|
199
|
+
## Sessions, storage and artifacts
|
|
200
|
+
|
|
201
|
+
- **Sessions** are durable by default, under `<rootDir>/sessions/<id>/`.
|
|
202
|
+
`agent.sessions.list()`, `.get(id)` and `.delete(id)` work over the session store.
|
|
203
|
+
Delete through them, never through the file system.
|
|
204
|
+
- **Session stores**: `createFilesystemSessionStore({ dir })`,
|
|
205
|
+
`createBackendSessionStore({ backend, prefix? })` for S3 or any Deep Agents backend
|
|
206
|
+
(how a worker fleet shares sessions), and `createMemorySessionStore()`. A custom
|
|
207
|
+
`backend` needs an explicit `sessionStore`.
|
|
208
|
+
- **Session ids**: a session's files sit at `<id>/<path>` in the store.
|
|
209
|
+
`sessionIdRejection(id)` is the rule every ingress applies. Mint top-level ids without
|
|
210
|
+
`~`, which separates a child session's number (`<parent>~<n>`).
|
|
211
|
+
- **Mounts**: `workspace.mounts` is a route table, key → backend or
|
|
212
|
+
`{ backend, readOnly?, governed?, searchable? }`. `defaultMounts(rootDir)` is the
|
|
213
|
+
conventional table (`skills/`, `.platform/`, `AGENTS.md`), and
|
|
214
|
+
`mountSubtree(backend, "skills")` rebases a backend. Mounts are read-only unless
|
|
215
|
+
declared otherwise. `governed: true` hands the spec's `mounts` lists the decision of
|
|
216
|
+
which states reach a mount. A key over a session zone or `workflows/` throws
|
|
217
|
+
`MountCollisionError`. Pass the same table to `validateWorkflow({ mounts })` so that
|
|
218
|
+
validation checks the spec's mount names.
|
|
219
|
+
- **Artifacts**: `agent.emitRunArtifacts(sessionId, trajectory)` writes `graph.json`,
|
|
220
|
+
`graph.mmd`, `trajectory.json`, `trail.json`, `variables.json` (each variable with its
|
|
221
|
+
value and lock) and `metadata.json` (the contract, the package version, the
|
|
222
|
+
`specHash` and usage) under `sessions/<id>/artifacts/`. An API reads them to draw past
|
|
223
|
+
sessions.
|
|
224
|
+
- **Spec edits** are adopted. The next turn runs under the current `workflow.yaml`, the
|
|
225
|
+
recorded `specHash` says which version a turn ran under (`agent.getSpecSnapshot(hash)`
|
|
226
|
+
returns it), and a session positioned in a removed state reopens at its trigger's
|
|
227
|
+
entry.
|
|
228
|
+
|
|
229
|
+
## Host tools that use the workspace
|
|
230
|
+
|
|
231
|
+
`toolsFromMap({ <name>: { description, inputSchema, paths?, handler } })` builds host
|
|
232
|
+
tools whose `handler(input, { workspace, sessionId })` reads and writes through the
|
|
233
|
+
turn's workspace, bound to the session. Inside a sub-workflow, `sessionId` is the child's.
|
|
234
|
+
Declared `paths` (`{ path: "read" }`) are governed like a built-in tool's path
|
|
235
|
+
arguments, so a refused call never runs the handler.
|
|
236
|
+
|
|
237
|
+
## Exposing a workflow to callers
|
|
238
|
+
|
|
239
|
+
A trigger's signature is the contract the runtime enforces, so build an MCP tool, an API
|
|
240
|
+
operation or a start form from it. `@archmax-ai/harness/spec` is browser-safe and holds
|
|
241
|
+
the helpers (they are also on the root):
|
|
242
|
+
|
|
243
|
+
- `parseMachineSpec(raw)` parses a spec; `signatureForTrigger(spec, "manual")` returns
|
|
244
|
+
`{ description?, requires, returns }`.
|
|
245
|
+
- `signatureJsonSchema(entries)` returns a JSON Schema object with every name required,
|
|
246
|
+
`date`, `date-time` and `file` as formatted strings, and each rule JSON Schema can
|
|
247
|
+
express as its keyword (date bounds, `accept` and `maxSize` are left out; the runtime
|
|
248
|
+
still enforces them).
|
|
249
|
+
- `signatureValueIssues(entries, values, options?)` returns one issue per missing or
|
|
250
|
+
non-conforming value, a broken rule as an `invalid` issue carrying `rule`. A
|
|
251
|
+
`pattern` runs in RE2, so it is safe on anyone's input. It never coerces, so convert
|
|
252
|
+
form text first.
|
|
253
|
+
- Use the trigger's `description` as the tool's description: the session never sees it.
|
|
254
|
+
Read `requires` and `returns` through `normalizeSignature`, never as `string[]`.
|
|
626
255
|
|
|
627
|
-
|
|
628
|
-
import { ChatOpenAI } from "@langchain/openai";
|
|
629
|
-
const model = new ChatOpenAI({
|
|
630
|
-
model: "claude-sonnet-5",
|
|
631
|
-
apiKey: process.env.MY_KEY,
|
|
632
|
-
configuration: { baseURL: "https://api.example.com/v1" },
|
|
633
|
-
});
|
|
634
|
-
const agent = await createAgent({ model });
|
|
635
|
-
```
|
|
256
|
+
## Cases against your agent
|
|
636
257
|
|
|
637
|
-
|
|
638
|
-
|
|
639
|
-
|
|
640
|
-
`readRaw`/`write`/`edit`/`ls` over filesystem, a store, a sandbox, S3, or a remote.
|
|
641
|
-
Pass it as `backend` (authored workspace); run state goes through a `sessionStore`
|
|
642
|
-
(e.g. `createBackendSessionStore({ backend: s3Backend, prefix: "tenants/a/runs" })`).
|
|
643
|
-
A custom `backend` **requires** an explicit `sessionStore` — assembly throws
|
|
644
|
-
`SessionStoreRequiredError` instead of inferring storage. The SDK owns the logical
|
|
645
|
-
`sessions/<sessionId>/…` namespace; the store owns where it physically lives.
|
|
646
|
-
Persisting the session zone off-box (an S3-backed session store) is how you share run
|
|
647
|
-
state across a worker fleet and an API.
|
|
648
|
-
|
|
649
|
-
A session's files are stored under `<sessionId>/` in the store (beneath its
|
|
650
|
-
`prefix`): the agent's `scratchpad/x.txt` in session `s1` is `s1/scratchpad/x.txt`.
|
|
651
|
-
That address is a contract, so a host may read or write a session's files through
|
|
652
|
-
its own handle on the store (ranged reads, recursive listings, presigned URLs). A
|
|
653
|
-
host tool keys them by `ToolContext.sessionId`, which inside a sub-workflow is the
|
|
654
|
-
child's id, never the session the turn was sent to. `sessionIdRejection` refuses an
|
|
655
|
-
id holding `~` in any shape but a child's (`<parent>~<n>`): mint top-level ids
|
|
656
|
-
without it.
|
|
657
|
-
|
|
658
|
-
`downloadFiles`, `uploadFiles` and `delete` are optional in the protocol and stay
|
|
659
|
-
optional in the workspace: a store with only the required methods serves skills,
|
|
660
|
-
reads and text copies. `downloadFiles` is always present and reads a store without
|
|
661
|
-
it through `readRaw` (exact bytes or `permission_denied`, never altered text);
|
|
662
|
-
`uploadFiles` is present when the session store has it, and a mount without it
|
|
663
|
-
refuses that file; `delete` answers a store without it with an error. What each
|
|
664
|
-
buys the file operations (`copy_file`, `move_file`, `remove_file`): a write to a
|
|
665
|
-
binary-typed path (an image, a PDF) carries **base64**, which the backend's
|
|
666
|
-
`write` must decode — Deep Agents' convention, with the extensions in
|
|
667
|
-
`BINARY_MIME_TYPES` on `@archmax-ai/harness/spec`. Other bytes the text channel
|
|
668
|
-
cannot carry (a `.docx`, a `.zip`, Latin-1 text) need `uploadFiles`, and a move
|
|
669
|
-
or removal needs `delete`. Every copy is read back; a store that kept something
|
|
670
|
-
else is refused, never reported as copied. A read-only mount refuses an upload
|
|
671
|
-
and a delete itself. Deep Agents is handed the workspace without `delete`, so its
|
|
672
|
-
own recursive `delete` tool never exists: `remove_file` is the one way to delete
|
|
673
|
-
a file.
|
|
674
|
-
|
|
675
|
-
A store's `grep` receives the call's match cap as its fourth argument,
|
|
676
|
-
`maxCount` (the agent's `max_count`, else Deep Agents' 1,000). A store may stop
|
|
677
|
-
there and return `truncated: true`; the workspace caps one that ignores it.
|
|
678
|
-
|
|
679
|
-
### Host tools
|
|
680
|
-
|
|
681
|
-
A host tool built with `toolsFromMap` reaches the workspace the way the
|
|
682
|
-
built-in file tools do — no second routing table:
|
|
258
|
+
`runTests` from `@archmax-ai/harness/testing` runs a workflow's cases with the discovery,
|
|
259
|
+
judge and verdicts of `archmax test`. By default it builds its own agent, with none of
|
|
260
|
+
your tools, mounts or stores. Supply yours:
|
|
683
261
|
|
|
684
|
-
|
|
685
|
-
|
|
686
|
-
|
|
687
|
-
|
|
688
|
-
|
|
689
|
-
paths: { path: "read" }, // governed like read_file's file_path
|
|
690
|
-
handler: async (input, { workspace }) => {
|
|
691
|
-
const raw = await workspace.readRaw(String(input.path));
|
|
692
|
-
return raw.error ? `Error: ${raw.error}` : toMarkdown(raw.data!);
|
|
693
|
-
},
|
|
694
|
-
},
|
|
695
|
-
});
|
|
696
|
-
const agent = await createAgent({ tools, essentialTools: ["get_markdown"] });
|
|
697
|
-
```
|
|
262
|
+
| Option | Why |
|
|
263
|
+
| --- | --- |
|
|
264
|
+
| `createTarget` | Your agent, or a factory `({ workflow, rootDir, onEvent }) => agent`. It must be workflow-governed. Wire `createToolMockMiddleware()` into its `middleware`, or a case that declares `mocks:` is refused before it runs. |
|
|
265
|
+
| `authoring` | Where the spec, the cases and their fixtures are read. When omitted, they are read from files under `rootDir`, so a workspace kept in a store finds no cases. |
|
|
266
|
+
| `sessionIdForCase` | The session id each case runs on, so its files and usage join your records. |
|
|
698
267
|
|
|
699
|
-
|
|
700
|
-
uses each. Every path rule binds them: the read-only zone, the runtime's own
|
|
701
|
-
areas, the skills and mounts the state was given (`mount.not-allowed`,
|
|
702
|
-
`skill.not-allowed`), a mount's `access: read` or a governed mount no grant
|
|
703
|
-
of the workflow asks `read_write` for (`mount.read-only` for a
|
|
704
|
-
`write` or `remove`), inherited denials and `paths:` guards. A refused call
|
|
705
|
-
never runs the handler. No declaration, no path rules. An argument may hold a
|
|
706
|
-
list of paths (`files: string[]`), each governed; any other shape is refused.
|
|
707
|
-
Declaring paths only narrows: `scratchpad/` is open in every state to the
|
|
708
|
-
default file tools the state is granted and to `essentialTools`, so a tool a
|
|
709
|
-
state must grant stays refused where it is not.
|
|
710
|
-
- `context.workspace` is the turn's workspace, bound to the session — the
|
|
711
|
-
instance `read_file` uses — with `downloadFiles`, `uploadFiles` (when the
|
|
712
|
-
session store has it) and `delete` besides the text methods. The plain agent
|
|
713
|
-
hands the same context.
|
|
714
|
-
- `context.sessionId` is the session the call is made in: inside a sub-workflow,
|
|
715
|
-
the child's own id (`<parent>~<n>`), for the model's call and a script's alike.
|
|
716
|
-
Key any session file your host stores itself by it (`<sessionId>/<path>` in the
|
|
717
|
-
session store), never by the session you sent the turn to.
|
|
718
|
-
- Outside a turn, reading either throws; both are non-enumerable there, so
|
|
719
|
-
comparing or logging the context does not.
|
|
268
|
+
Cases run one at a time, and `tests.caseTimeoutMs` bounds each.
|
|
720
269
|
|
|
721
|
-
##
|
|
270
|
+
## Sub-workflows in a service
|
|
722
271
|
|
|
723
|
-
|
|
724
|
-
|
|
725
|
-
|
|
726
|
-
|
|
727
|
-
|
|
728
|
-
the local filesystem under `rootDir`, which is *not* your authored tree if that
|
|
729
|
-
tree lives in a store. Supply both:
|
|
272
|
+
A child session is composed from its caller's inputs: the model and `modelFactory`, the
|
|
273
|
+
backend and mounts, the session store and checkpointer, the host tools, the policy
|
|
274
|
+
rules, the prompt cache and the pricing. Composition is lazy, per slug, so a child
|
|
275
|
+
whose spec is broken fails its dispatch, not the assembly. The child is its own session
|
|
276
|
+
(`<parent>~<n>`) in the same store, listed by `sessions.list()` with `parentSessionId`.
|
|
730
277
|
|
|
731
|
-
|
|
732
|
-
import { runTests, createAgent, createToolMockMiddleware } from "@archmax-ai/harness";
|
|
733
|
-
|
|
734
|
-
const { results, exitCode } = await runTests({
|
|
735
|
-
workflow,
|
|
736
|
-
rootDir,
|
|
737
|
-
// Where the spec, the case documents and their `from:` fixtures are read
|
|
738
|
-
// from. Omit it only when they really are files under `rootDir`.
|
|
739
|
-
authoring,
|
|
740
|
-
createTarget: async ({ workflow, rootDir, onEvent }) =>
|
|
741
|
-
createAgent({
|
|
742
|
-
workflow,
|
|
743
|
-
onEvent,
|
|
744
|
-
backend,
|
|
745
|
-
tools: await resolveConnectionTools(tenant),
|
|
746
|
-
checkpointer,
|
|
747
|
-
middleware: [createToolMockMiddleware()],
|
|
748
|
-
// REQUIRED — see below,
|
|
749
|
-
workspace: { rootDir, sessionStore },
|
|
750
|
-
}),
|
|
751
|
-
sessionIdForCase: (file) => `${hostRunId}/${file}`,
|
|
752
|
-
});
|
|
753
|
-
```
|
|
278
|
+
## A typical split
|
|
754
279
|
|
|
755
|
-
|
|
756
|
-
|
|
757
|
-
|
|
758
|
-
|
|
759
|
-
|
|
760
|
-
|
|
761
|
-
**Your target owns its mocks — and the runtime checks.** A case's `mocks:`
|
|
762
|
-
list intercepts agent-initiated tool calls through
|
|
763
|
-
`createToolMockMiddleware()`, which the default target wires for you.
|
|
764
|
-
`createAgent` stamps `capabilities.toolMocks` from the middleware
|
|
765
|
-
actually wired; a case that declares mocks against a target without the
|
|
766
|
-
capability **fails loudly before the agent runs**, naming the fix — never run
|
|
767
|
-
with partial interception (scripts' PTC calls mocked, the agent's own calls
|
|
768
|
-
real). A bespoke target that intercepts by other means sets
|
|
769
|
-
`capabilities.toolMocks` itself. Mockless cases run against any
|
|
770
|
-
workflow-governed target.
|
|
771
|
-
|
|
772
|
-
**Sub-workflows inherit the assembly.** When a workflow delegates (by calling
|
|
773
|
-
an `archmax_workflow_<slug>` tool), the child runtime is
|
|
774
|
-
composed from the *same* resolved inputs as the parent — model and
|
|
775
|
-
`modelFactory`, `backend` and `mounts`, `sessionStore` and `checkpointer`, host
|
|
776
|
-
`tools` and `essentialTools`, `sandboxRuntime`,
|
|
777
|
-
`policyRules`, `promptCache` and `pricing` — differing only in
|
|
778
|
-
the machine that governs it and the system prompt that machine renders. So a
|
|
779
|
-
custom backend serves the child's spec too, and a `modelFactory` supplies its
|
|
780
|
-
model. Composition is **lazy and memoized per workflow slug**: a workflow that is
|
|
781
|
-
never delegated to is never loaded, and one delegated from many places is
|
|
782
|
-
composed once. A child whose spec is broken surfaces as a *dispatch* failure, not
|
|
783
|
-
an assembly failure — a parent still assembles.
|
|
784
|
-
|
|
785
|
-
The child runs as a **separate session**, `<parent>~<n>`, in the same session
|
|
786
|
-
store, with its own checkpoints, its own token accounting and its own file space:
|
|
787
|
-
its `scratchpad/` is not its caller's, so it can neither read nor overwrite a file
|
|
788
|
-
of its caller's. `n` is the dispatch's number in the calling session, counted from 1
|
|
789
|
-
across all its turns in its checkpointed state, so an agent built per turn never
|
|
790
|
-
reuses one; a grandchild is `<parent>~<n>~<m>`. The `sub-workflow-*` events and a
|
|
791
|
-
pending delegation name it by its `dispatchId`, the id below the root session (`1`,
|
|
792
|
-
`1~1`). A host tool called in the child gets the child's id as
|
|
793
|
-
`ToolContext.sessionId`, and its file work belongs to the child. `sessions.list()` reports it beside its caller
|
|
794
|
-
with `parentSessionId` naming the caller, and `sessions.delete` removes it by its
|
|
795
|
-
own id. The only files that cross are the runtime's copies of declared `file` arguments and
|
|
796
|
-
`file` returns. `capabilities.subWorkflows` is stamped `true` for a governed
|
|
797
|
-
assembly and `false` for the plain one.
|
|
798
|
-
|
|
799
|
-
Omit both options and behavior is identical to `archmax test`. Cases run
|
|
800
|
-
sequentially (`tests.maxConcurrency` above 1 is rejected, not silently
|
|
801
|
-
serialized) and there is no cancellation signal: call `runTests` once per file
|
|
802
|
-
(`filter`) if you need to interrupt between cases; a single case is bounded by
|
|
803
|
-
`tests.caseTimeoutMs`.
|
|
804
|
-
|
|
805
|
-
## Suggested split
|
|
806
|
-
|
|
807
|
-
- **Worker**: `createAgent` + `agent.invoke`/`stream`; `agent.workflow.decide` for
|
|
808
|
-
approvals. Owns execution and the durable session store/checkpointer.
|
|
809
|
-
- **API**: reads `sessions/<sessionId>/artifacts/*` (shared session store) and/or relays
|
|
810
|
-
the `onEvent` stream; exposes `sessions.list()`, run detail, and a decide endpoint.
|
|
811
|
-
- **Frontend**: renders `graph.json`/`graph.mmd`, the `trajectory.json` timeline,
|
|
812
|
-
the `variables.json` inputs, and the live event stream; posts human decisions
|
|
813
|
-
back to the API.
|
|
814
|
-
|
|
815
|
-
Keep the heavy runtime import (LangChain/LangGraph) in the worker; the API/frontend
|
|
816
|
-
only need the artifact/event *types*, which ship in `dist/index.d.ts`.
|
|
280
|
+
- **Worker**: `createAgent` and `send`. It owns the session store.
|
|
281
|
+
- **API**: reads artifacts from the shared session store, relays `onEvent`, and exposes
|
|
282
|
+
`sessions.list()`, a session's detail and a decide endpoint.
|
|
283
|
+
- **Frontend**: draws `graph.json`, the trail and the live stream, and posts decisions to
|
|
284
|
+
the API. It needs only the types, which ship in `dist/index.d.ts`.
|