agentfences 0.7.0 → 0.8.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +20 -3
- package/dist/approvals.d.ts +24 -0
- package/dist/approvals.js +57 -0
- package/dist/client.d.ts +5 -3
- package/dist/client.js +17 -9
- package/dist/control.d.ts +27 -27
- package/dist/control.js +22 -69
- package/dist/core.d.ts +11 -74
- package/dist/core.js +35 -143
- package/dist/index.d.ts +4 -2
- package/dist/index.js +2 -1
- package/dist/instrument.d.ts +12 -0
- package/dist/instrument.js +124 -0
- package/dist/mcp.js +2 -2
- package/dist/outbox.d.ts +23 -0
- package/dist/outbox.js +35 -0
- package/dist/precheck.d.ts +33 -0
- package/dist/precheck.js +136 -0
- package/dist/pricing.d.ts +1 -0
- package/dist/pricing.js +17 -4
- package/dist/results.d.ts +16 -0
- package/dist/results.js +33 -0
- package/dist/run.d.ts +79 -0
- package/dist/run.js +56 -0
- package/dist/tools.js +8 -5
- package/dist/version.d.ts +1 -1
- package/dist/version.js +1 -1
- package/package.json +28 -6
package/README.md
CHANGED
|
@@ -30,7 +30,7 @@ const research = governed(async (question: string) => {
|
|
|
30
30
|
|
|
31
31
|
Pass the model's response to `checkpoint()` and Fences works out what the call cost from its model and token usage, including cache reads and writes. It understands results from the OpenAI and Anthropic SDKs, the Vercel AI SDK (`generateText`), Gemini and LangChain.js, or any object with the same usage fields. Prices come from a built-in table of about 290 models (the same one as the Python SDK).
|
|
32
32
|
|
|
33
|
-
- Unknown model: its tokens are counted and Fences warns once. Add prices in USD per 1M tokens: `init({ localOnly: true, prices: { "my-finetune": { input: 2, output: 6 } } })`.
|
|
33
|
+
- Unknown model: its tokens are counted and Fences warns once. Add prices in USD per 1M tokens: `init({ localOnly: true, prices: { "my-finetune": { input: 2, output: 6 } } })`. Both prices are required; an invalid one throws at `init()`.
|
|
34
34
|
- Extra costs, or no response at all: `checkpoint(response, { costDeltaUsd: 0.01 })` or `checkpoint(null, { costDeltaUsd: 0.02, tokensUsed: 450 })`.
|
|
35
35
|
|
|
36
36
|
## Limits
|
|
@@ -57,7 +57,7 @@ await context({ user_id: "u_123", session_id: "s_9" }, () => research(question))
|
|
|
57
57
|
init({ apiKey: process.env.FENCES_API_KEY, endpoint: "https://your-fences-host", environment: "prod", release: "v1.4.2" });
|
|
58
58
|
```
|
|
59
59
|
|
|
60
|
-
The server is authoritative for limits and sees spend across processes. If it can't be reached, limits are enforced locally; pass `failClosed: true` to stop instead. A rejected key throws `FencesAuthError
|
|
60
|
+
The server is authoritative for limits and sees spend across processes. If it can't be reached, limits are enforced locally; pass `failClosed: true` to stop instead. A rejected key throws `FencesAuthError` when the run starts, and a key revoked during a run ends it at its next checkpoint (`fences_unreachable`); a quarantined agent throws `AgentQuarantined` before any of its code runs. A cost that isn't a finite number of zero or more throws instead of being counted. In serverless handlers, `await flush()` before returning so queued decisions and spans are sent.
|
|
61
61
|
|
|
62
62
|
## Live control (from 0.4.0)
|
|
63
63
|
|
|
@@ -117,6 +117,23 @@ mcp.wrap(client, { guard: ["send_email"], costUsd: { refund: (args) => args.amou
|
|
|
117
117
|
|
|
118
118
|
`guard: true` guards every tool, and `approve: [...]` names tools a person approves first. A result with `isError` is recorded as a failed call and still comes back to the agent.
|
|
119
119
|
|
|
120
|
+
## Model calls (from 0.8.0)
|
|
121
|
+
|
|
122
|
+
`instrument(client)` records every OpenAI and Anthropic call made inside a governed run (model, tokens, cost, latency, errors) and checks it against the budget **before** it's sent:
|
|
123
|
+
|
|
124
|
+
```ts
|
|
125
|
+
import OpenAI from "openai";
|
|
126
|
+
import { instrument } from "agentfences";
|
|
127
|
+
|
|
128
|
+
const openai = instrument(new OpenAI()); // one call covers every client of that SDK
|
|
129
|
+
```
|
|
130
|
+
|
|
131
|
+
- If a call's worst case could take the run past its budget, it rejects with `FencesStop` (`budget_exceeded`) and never reaches the provider. The worst case is the request's input, estimated high, plus its output cap: `max_tokens` (or `max_completion_tokens`, `max_output_tokens`) if set, otherwise the model's whole output limit, so **set `max_tokens` on calls in runs with small budgets**.
|
|
132
|
+
- Each call's worst case is held while it's in flight, so parallel calls can't overspend together. A failed call is released; a finished one is replaced by its actual cost, and the next `checkpoint()` releases it. With `onBreach: "hold"`, a call that doesn't fit waits for someone to raise the budget.
|
|
133
|
+
- Calls are recorded, not counted as steps: keep calling `checkpoint(response)`, which adds the cost and checks the other limits.
|
|
134
|
+
- The SDK's own promise comes back, with `.withResponse()` and `.asResponse()`. After `.asResponse()` the call is recorded without usage, and its worst case stays held until the next checkpoint.
|
|
135
|
+
- OpenAI: `chat.completions.create` and `responses.create`, including OpenAI-compatible clients. Anthropic: `messages.create`. Streams are recorded when they start; pass the final usage to `checkpoint()`.
|
|
136
|
+
|
|
120
137
|
## Not yet in the TypeScript SDK
|
|
121
138
|
|
|
122
|
-
The Python SDK also
|
|
139
|
+
The Python SDK also integrates with LangGraph and the OpenAI Agents SDK. Model calls made without `instrument(client)` are checked afterwards, by `checkpoint()`, so a run can go over its budget by one such call. Everything here follows the same server protocol ([PROTOCOL.md](../PROTOCOL.md)).
|
|
@@ -0,0 +1,24 @@
|
|
|
1
|
+
export interface Approval {
|
|
2
|
+
granted: boolean;
|
|
3
|
+
/** Who answered, as the dashboard reports it. */
|
|
4
|
+
by?: string | null;
|
|
5
|
+
/** Their note, or why no one could answer. */
|
|
6
|
+
note?: string | null;
|
|
7
|
+
/** The amount approved, if one was asked for. */
|
|
8
|
+
amountUsd?: number | null;
|
|
9
|
+
approvalId?: string | null;
|
|
10
|
+
}
|
|
11
|
+
/**
|
|
12
|
+
* Ask a person, through the Fences dashboard, to approve something, and wait for the answer:
|
|
13
|
+
*
|
|
14
|
+
* const approval = await requestApproval("Refund $240 to order 1182?", { amountUsd: 240 });
|
|
15
|
+
* if (!approval.granted) return `I didn't issue the refund: ${approval.note}`;
|
|
16
|
+
*
|
|
17
|
+
* If an amount is approved, the run's budget goes up by that much. Returns denied right away in
|
|
18
|
+
* local mode, outside a run, or when the server can't take requests, and denied after timeoutS
|
|
19
|
+
* (default 600) with no answer: it never hangs and never throws.
|
|
20
|
+
*/
|
|
21
|
+
export declare function requestApproval(reason: string, options?: {
|
|
22
|
+
amountUsd?: number;
|
|
23
|
+
timeoutS?: number;
|
|
24
|
+
}): Promise<Approval>;
|
|
@@ -0,0 +1,57 @@
|
|
|
1
|
+
/** Approval requests: ask a person, through the Fences dashboard, and wait for the answer. */
|
|
2
|
+
import { randomUUID } from "node:crypto";
|
|
3
|
+
import * as control from "./control.js";
|
|
4
|
+
import { recordSpan } from "./core.js";
|
|
5
|
+
import { getActiveRun } from "./run.js";
|
|
6
|
+
/**
|
|
7
|
+
* Ask a person, through the Fences dashboard, to approve something, and wait for the answer:
|
|
8
|
+
*
|
|
9
|
+
* const approval = await requestApproval("Refund $240 to order 1182?", { amountUsd: 240 });
|
|
10
|
+
* if (!approval.granted) return `I didn't issue the refund: ${approval.note}`;
|
|
11
|
+
*
|
|
12
|
+
* If an amount is approved, the run's budget goes up by that much. Returns denied right away in
|
|
13
|
+
* local mode, outside a run, or when the server can't take requests, and denied after timeoutS
|
|
14
|
+
* (default 600) with no answer: it never hangs and never throws.
|
|
15
|
+
*/
|
|
16
|
+
export async function requestApproval(reason, options = {}) {
|
|
17
|
+
const run = getActiveRun(), client = control._client();
|
|
18
|
+
const timeoutS = options.timeoutS ?? 600, amountUsd = options.amountUsd ?? null;
|
|
19
|
+
if (!run)
|
|
20
|
+
return { granted: false, note: "requestApproval() was called outside a governed run" };
|
|
21
|
+
if (!client)
|
|
22
|
+
return { granted: false, note: "Local mode: there is no one to approve this request" };
|
|
23
|
+
if (!control.supported.approvals)
|
|
24
|
+
return { granted: false, note: "This Fences server doesn't support approvals" };
|
|
25
|
+
const approvalId = randomUUID(), started = performance.now();
|
|
26
|
+
// timeout_s lets the server expire the request when we stop waiting
|
|
27
|
+
const resp = await client.requestApproval(run.runId, { approval_id: approvalId, reason, amount_usd: amountUsd, timeout_s: timeoutS });
|
|
28
|
+
if (resp.unsupported) {
|
|
29
|
+
control.unsupported("approvals", "approval requests", "they are denied");
|
|
30
|
+
return { granted: false, note: "This Fences server doesn't support approvals" };
|
|
31
|
+
}
|
|
32
|
+
if (resp.network_error)
|
|
33
|
+
return { granted: false, note: "Couldn't reach Fences to ask for approval" };
|
|
34
|
+
if (resp.ok === false)
|
|
35
|
+
return { granted: false, note: `Fences didn't accept the approval request (HTTP ${resp.status})` }; // no one will be asked
|
|
36
|
+
run.control.approvals.set(approvalId, null);
|
|
37
|
+
run.control.awaiting += 1;
|
|
38
|
+
control.schedule(0); // poll fast while someone decides
|
|
39
|
+
const deadline = Date.now() + timeoutS * 1000;
|
|
40
|
+
while (run.control.approvals.get(approvalId) === null && Date.now() < deadline) {
|
|
41
|
+
await new Promise((ok) => setTimeout(ok, 100));
|
|
42
|
+
}
|
|
43
|
+
run.control.awaiting -= 1;
|
|
44
|
+
const c = run.control.approvals.get(approvalId); // the dashboard's approval command, or null: no answer in time
|
|
45
|
+
run.control.approvals.delete(approvalId);
|
|
46
|
+
const answer = c
|
|
47
|
+
? { granted: !!c.granted, by: c.by ?? null, note: c.note ?? null, amountUsd: c.amount_usd ?? null, approvalId }
|
|
48
|
+
: { granted: false, note: "No answer before the timeout", approvalId };
|
|
49
|
+
if (answer.granted && answer.amountUsd && run.budgetUsd !== null)
|
|
50
|
+
run.budgetUsd += answer.amountUsd; // an unlimited run stays unlimited
|
|
51
|
+
recordSpan("fences.approval", performance.now() - started, {
|
|
52
|
+
"fences.approval.id": approvalId, "fences.approval.reason": reason, "fences.approval.amount_usd": amountUsd,
|
|
53
|
+
"fences.approval.granted": answer.granted, "fences.approval.by": answer.by, "fences.approval.note": answer.note,
|
|
54
|
+
"fences.approval.granted_usd": answer.amountUsd,
|
|
55
|
+
});
|
|
56
|
+
return answer;
|
|
57
|
+
}
|
package/dist/client.d.ts
CHANGED
|
@@ -15,8 +15,10 @@ export declare class FencesClient {
|
|
|
15
15
|
guard(runId: string, body: Record<string, unknown>): Promise<Reply>;
|
|
16
16
|
requestApproval(runId: string, body: Record<string, unknown>): Promise<Reply>;
|
|
17
17
|
/**
|
|
18
|
-
* optional: an endpoint older servers may not have. A missing route returns
|
|
19
|
-
*
|
|
18
|
+
* optional: an endpoint older servers may not have. A missing route returns { unsupported: true },
|
|
19
|
+
* and any other error { ok: false, status }.
|
|
20
|
+
* start: a rejected key throws FencesAuthError, before any agent code runs. Later in a run it
|
|
21
|
+
* returns { network_error, auth_rejected: true }: throwing there would replace the agent's own result or error.
|
|
20
22
|
*/
|
|
21
|
-
post(path: string, payload: unknown, optional?: boolean): Promise<Reply>;
|
|
23
|
+
post(path: string, payload: unknown, optional?: boolean, start?: boolean): Promise<Reply>;
|
|
22
24
|
}
|
package/dist/client.js
CHANGED
|
@@ -1,4 +1,7 @@
|
|
|
1
|
-
/**
|
|
1
|
+
/**
|
|
2
|
+
* HTTP calls to a Fences server, as specified in PROTOCOL.md, with the fetch built into Node 18+.
|
|
3
|
+
* Nothing throws except a rejected key at run start.
|
|
4
|
+
*/
|
|
2
5
|
import { FencesAuthError } from "./errors.js";
|
|
3
6
|
import { VERSION } from "./version.js";
|
|
4
7
|
export class FencesClient {
|
|
@@ -12,7 +15,7 @@ export class FencesClient {
|
|
|
12
15
|
// The server uses the SDK version to warn about old SDKs and to skip commands they don't understand
|
|
13
16
|
this.headers = { "Content-Type": "application/json", "X-API-Key": apiKey, "X-Fences-SDK": `js/${VERSION}` };
|
|
14
17
|
}
|
|
15
|
-
startRun(body) { return this.post("/api/runs/start", body); }
|
|
18
|
+
startRun(body) { return this.post("/api/runs/start", body, false, true); }
|
|
16
19
|
checkpoint(runId, body) { return this.post(`/api/runs/${runId}/checkpoint`, body); }
|
|
17
20
|
logDecision(runId, body) { return this.post(`/api/runs/${runId}/decisions`, body); }
|
|
18
21
|
endRun(runId, body) { return this.post(`/api/runs/${runId}/end`, body); }
|
|
@@ -25,10 +28,12 @@ export class FencesClient {
|
|
|
25
28
|
guard(runId, body) { return this.post(`/api/runs/${runId}/guard`, body, true); }
|
|
26
29
|
requestApproval(runId, body) { return this.post(`/api/runs/${runId}/approvals`, body, true); }
|
|
27
30
|
/**
|
|
28
|
-
* optional: an endpoint older servers may not have. A missing route returns
|
|
29
|
-
*
|
|
31
|
+
* optional: an endpoint older servers may not have. A missing route returns { unsupported: true },
|
|
32
|
+
* and any other error { ok: false, status }.
|
|
33
|
+
* start: a rejected key throws FencesAuthError, before any agent code runs. Later in a run it
|
|
34
|
+
* returns { network_error, auth_rejected: true }: throwing there would replace the agent's own result or error.
|
|
30
35
|
*/
|
|
31
|
-
async post(path, payload, optional = false) {
|
|
36
|
+
async post(path, payload, optional = false, start = false) {
|
|
32
37
|
let resp;
|
|
33
38
|
try {
|
|
34
39
|
resp = await fetch(this.endpoint + path, {
|
|
@@ -42,8 +47,8 @@ export class FencesClient {
|
|
|
42
47
|
let body = {}, isJson = false;
|
|
43
48
|
try {
|
|
44
49
|
const parsed = JSON.parse(text);
|
|
45
|
-
if (parsed && typeof parsed === "object")
|
|
46
|
-
[body, isJson] = [parsed, true];
|
|
50
|
+
if (parsed && typeof parsed === "object" && !Array.isArray(parsed))
|
|
51
|
+
[body, isJson] = [parsed, true]; // a list isn't a Fences reply
|
|
47
52
|
}
|
|
48
53
|
catch { /* not JSON: the status code still decides */ }
|
|
49
54
|
const detail = typeof body.detail === "string" ? body.detail : "";
|
|
@@ -53,8 +58,11 @@ export class FencesClient {
|
|
|
53
58
|
}
|
|
54
59
|
return resp.ok ? body : { ok: false, status: resp.status };
|
|
55
60
|
}
|
|
56
|
-
if (resp.status === 401 || resp.status === 403)
|
|
57
|
-
|
|
61
|
+
if (resp.status === 401 || resp.status === 403) {
|
|
62
|
+
if (start)
|
|
63
|
+
throw new FencesAuthError(`Fences API key rejected: ${detail || text.slice(0, 200)}`);
|
|
64
|
+
return { network_error: `API key rejected (HTTP ${resp.status})`, auth_rejected: true };
|
|
65
|
+
}
|
|
58
66
|
if (resp.status === 423)
|
|
59
67
|
return { ok: false, quarantined: true, detail }; // the agent is quarantined: the run must not start
|
|
60
68
|
if (resp.status === 409)
|
package/dist/control.d.ts
CHANGED
|
@@ -1,15 +1,12 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Live control from the Fences dashboard: a heartbeat that reports live runs and brings back
|
|
3
|
+
* commands (stop, pause, resume, quarantine, limits, approval answers) and acknowledges them,
|
|
4
|
+
* containment hooks, pausing at checkpoints, waiting at a limit, and cancelling a run stuck after
|
|
5
|
+
* a stop. Cloud mode only; a server without these endpoints gets one warning and runs carry
|
|
6
|
+
* on as before. Same behaviour as the Python SDK (PROTOCOL.md, "Live control").
|
|
7
|
+
*/
|
|
1
8
|
import type { FencesClient, Reply } from "./client.js";
|
|
2
|
-
import type { Run } from "./
|
|
3
|
-
export interface Approval {
|
|
4
|
-
granted: boolean;
|
|
5
|
-
/** Who answered, as the dashboard reports it. */
|
|
6
|
-
by?: string | null;
|
|
7
|
-
/** Their note, or why no one could answer. */
|
|
8
|
-
note?: string | null;
|
|
9
|
-
/** The amount approved, if one was asked for. */
|
|
10
|
-
amountUsd?: number | null;
|
|
11
|
-
approvalId?: string | null;
|
|
12
|
-
}
|
|
9
|
+
import type { Run } from "./run.js";
|
|
13
10
|
/** What onStop and onQuarantine receive. */
|
|
14
11
|
export interface ControlEvent {
|
|
15
12
|
type: "stop" | "quarantine";
|
|
@@ -36,13 +33,15 @@ export interface Control {
|
|
|
36
33
|
quarantined: boolean;
|
|
37
34
|
/** While waiting at a limit: "breach" or "budget". */
|
|
38
35
|
held: "breach" | "budget" | null;
|
|
39
|
-
/** approval id -> the answer, or null while waiting */
|
|
40
|
-
approvals: Map<string,
|
|
36
|
+
/** approval id -> the dashboard's answer (an approval command), or null while waiting */
|
|
37
|
+
approvals: Map<string, Reply | null>;
|
|
41
38
|
awaiting: number;
|
|
42
39
|
/** command ids already acknowledged */
|
|
43
40
|
acked: Set<string>;
|
|
44
41
|
}
|
|
45
42
|
export declare const newControl: () => Control;
|
|
43
|
+
/** @internal */
|
|
44
|
+
export declare const _client: () => FencesClient | null;
|
|
46
45
|
export declare let pauseTimeoutMs: number;
|
|
47
46
|
export declare const supported: {
|
|
48
47
|
heartbeat: boolean;
|
|
@@ -60,6 +59,19 @@ export declare function configure(c: FencesClient | null, o: ControlOptions): vo
|
|
|
60
59
|
export declare function register(run: Run): void;
|
|
61
60
|
export declare function unregister(run: Run): void;
|
|
62
61
|
export declare function unsupported(kind: keyof typeof supported, what: string, consequence: string): void;
|
|
62
|
+
export declare const LIMITS: {
|
|
63
|
+
readonly budget_usd: "budgetUsd";
|
|
64
|
+
readonly max_iterations: "maxIterations";
|
|
65
|
+
readonly max_duration_ms: "maxDurationMs";
|
|
66
|
+
readonly max_tokens: "maxTokens";
|
|
67
|
+
};
|
|
68
|
+
export declare const TOTALS: {
|
|
69
|
+
readonly spent_usd: "costUsd";
|
|
70
|
+
readonly iterations: "iterations";
|
|
71
|
+
readonly tokens_used: "tokensUsed";
|
|
72
|
+
};
|
|
73
|
+
/** Set the run's fields from a server reply: numbers only, so a null or odd value can't break local enforcement. nullBudget: a null budget_usd means unlimited spend. */
|
|
74
|
+
export declare function adopt(run: Run, reply: Reply, fields: Record<string, string>, nullBudget?: boolean): void;
|
|
63
75
|
/** Apply the commands addressed to this run (from a heartbeat or a checkpoint reply), and acknowledge them. Each is safe to apply twice. */
|
|
64
76
|
export declare function apply(run: Run, commands: Reply[] | undefined): void;
|
|
65
77
|
/**
|
|
@@ -68,6 +80,8 @@ export declare function apply(run: Run, commands: Reply[] | undefined): void;
|
|
|
68
80
|
* its containment, and says so.
|
|
69
81
|
*/
|
|
70
82
|
export declare function drain(timeoutMs?: number): Promise<void>;
|
|
83
|
+
/** @internal Send a heartbeat after delayMs (0: now), replacing the one scheduled. */
|
|
84
|
+
export declare function schedule(delayMs: number): void;
|
|
71
85
|
/** Breaches a person can lift by raising a limit, so onBreach "hold" waits at them. */
|
|
72
86
|
export declare const HOLDABLE: Set<string>;
|
|
73
87
|
export declare const BUDGET_BREACHES: Set<string>;
|
|
@@ -79,17 +93,3 @@ export declare function startHold(run: Run, kind: "breach" | "budget"): void;
|
|
|
79
93
|
export declare function endHold(run: Run): "released" | "stopped" | "timeout";
|
|
80
94
|
/** Hold the agent at its checkpoint while its run is paused, up to pauseTimeoutS. */
|
|
81
95
|
export declare function waitIfPaused(run: Run): Promise<void>;
|
|
82
|
-
/**
|
|
83
|
-
* Ask a person, through the Fences dashboard, to approve something, and wait for the answer:
|
|
84
|
-
*
|
|
85
|
-
* const approval = await requestApproval("Refund $240 to order 1182?", { amountUsd: 240 });
|
|
86
|
-
* if (!approval.granted) return `I didn't issue the refund: ${approval.note}`;
|
|
87
|
-
*
|
|
88
|
-
* If an amount is approved, the run's budget goes up by that much. Returns denied right away in
|
|
89
|
-
* local mode, outside a run, or when the server can't take requests, and denied after timeoutS
|
|
90
|
-
* (default 600) with no answer: it never hangs and never throws.
|
|
91
|
-
*/
|
|
92
|
-
export declare function requestApproval(reason: string, options?: {
|
|
93
|
-
amountUsd?: number;
|
|
94
|
-
timeoutS?: number;
|
|
95
|
-
}): Promise<Approval>;
|
package/dist/control.js
CHANGED
|
@@ -1,11 +1,3 @@
|
|
|
1
|
-
/**
|
|
2
|
-
* Live control from the Fences dashboard: a heartbeat that reports live runs and brings back
|
|
3
|
-
* commands (stop, pause, resume, quarantine, limits, approval answers) and acknowledges them,
|
|
4
|
-
* containment hooks, pausing at checkpoints, cancelling a run stuck after a stop, and approval
|
|
5
|
-
* requests. Cloud mode only; a server without these endpoints gets one warning and runs carry
|
|
6
|
-
* on as before. Same behaviour as the Python SDK (PROTOCOL.md, "Live control").
|
|
7
|
-
*/
|
|
8
|
-
import { randomUUID } from "node:crypto";
|
|
9
1
|
/** Heartbeat statuses whose acks go again with the next heartbeat: rate limited, or a server error. */
|
|
10
2
|
const retryable = (status) => status === undefined || status === 429 || status >= 500;
|
|
11
3
|
const FAST_MS = 2000; // heartbeat interval while a run is paused, waiting for an approval, or stopping
|
|
@@ -16,6 +8,8 @@ export const newControl = () => ({
|
|
|
16
8
|
});
|
|
17
9
|
const live = new Map(); // run id -> runs in progress
|
|
18
10
|
let client = null;
|
|
11
|
+
/** @internal */
|
|
12
|
+
export const _client = () => client;
|
|
19
13
|
let intervalMs = 15_000;
|
|
20
14
|
export let pauseTimeoutMs = 3_600_000;
|
|
21
15
|
let stopGraceMs = 30_000;
|
|
@@ -51,7 +45,17 @@ export function unsupported(kind, what, consequence) {
|
|
|
51
45
|
console.warn(`agentfences: this Fences server doesn't support ${what} yet, so ${consequence}`);
|
|
52
46
|
}
|
|
53
47
|
// ── Commands ──────────────────────────────────────────────────────────────────
|
|
54
|
-
const LIMITS = { budget_usd: "budgetUsd", max_iterations: "maxIterations", max_duration_ms: "maxDurationMs", max_tokens: "maxTokens" };
|
|
48
|
+
export const LIMITS = { budget_usd: "budgetUsd", max_iterations: "maxIterations", max_duration_ms: "maxDurationMs", max_tokens: "maxTokens" };
|
|
49
|
+
export const TOTALS = { spent_usd: "costUsd", iterations: "iterations", tokens_used: "tokensUsed" };
|
|
50
|
+
/** Set the run's fields from a server reply: numbers only, so a null or odd value can't break local enforcement. nullBudget: a null budget_usd means unlimited spend. */
|
|
51
|
+
export function adopt(run, reply, fields, nullBudget = false) {
|
|
52
|
+
for (const [wire, field] of Object.entries(fields)) {
|
|
53
|
+
const v = reply[wire];
|
|
54
|
+
if ((typeof v === "number" && !Number.isNaN(v)) || (nullBudget && wire === "budget_usd" && v === null)) {
|
|
55
|
+
run[field] = v; // eslint-disable-line @typescript-eslint/no-explicit-any
|
|
56
|
+
}
|
|
57
|
+
}
|
|
58
|
+
}
|
|
55
59
|
/** Apply the commands addressed to this run (from a heartbeat or a checkpoint reply), and acknowledge them. Each is safe to apply twice. */
|
|
56
60
|
export function apply(run, commands) {
|
|
57
61
|
// Limits first: a resume in the same reply may release a call held for budget, which re-checks against them at once
|
|
@@ -69,8 +73,10 @@ export function apply(run, commands) {
|
|
|
69
73
|
ctl.paused = ctl.quarantined = false;
|
|
70
74
|
}
|
|
71
75
|
else if (c.type === "pause" || c.type === "quarantine") { // a quarantined agent's runs hold at their next checkpoint
|
|
72
|
-
ctl.paused
|
|
73
|
-
|
|
76
|
+
if (!ctl.paused) { // only a change wakes the heartbeat: a quarantine is re-sent on every reply while it lasts
|
|
77
|
+
ctl.paused = true;
|
|
78
|
+
schedule(0);
|
|
79
|
+
}
|
|
74
80
|
if (c.type === "quarantine") {
|
|
75
81
|
ctl.quarantined = true; // guarded tools are blocked here too, even if the server can't be reached
|
|
76
82
|
hook("quarantine", run, c);
|
|
@@ -80,15 +86,11 @@ export function apply(run, commands) {
|
|
|
80
86
|
ctl.paused = ctl.quarantined = false; // releasing a quarantine resumes its runs
|
|
81
87
|
}
|
|
82
88
|
else if (c.type === "limits") {
|
|
83
|
-
|
|
84
|
-
if (c[wire] !== undefined && c[wire] !== null)
|
|
85
|
-
run[field] = c[wire]; // eslint-disable-line @typescript-eslint/no-explicit-any
|
|
86
|
-
}
|
|
89
|
+
adopt(run, c, LIMITS);
|
|
87
90
|
}
|
|
88
91
|
else if (c.type === "approval") {
|
|
89
|
-
if (ctl.approvals.has(c.approval_id))
|
|
90
|
-
ctl.approvals.set(c.approval_id,
|
|
91
|
-
}
|
|
92
|
+
if (ctl.approvals.has(c.approval_id))
|
|
93
|
+
ctl.approvals.set(c.approval_id, c); // approvals.ts turns it into the answer
|
|
92
94
|
}
|
|
93
95
|
else {
|
|
94
96
|
continue; // unknown: ignored, and not acknowledged
|
|
@@ -174,7 +176,8 @@ if (typeof process?.on === "function") { // edge runtimes have no process events
|
|
|
174
176
|
const unseenStop = (r) => r.control.stopped && !r.control.stopSeen && !r.control.cancelled && r.control.onCancel !== null;
|
|
175
177
|
const busy = () => [...live.values()].some((r) => r.control.paused || r.control.awaiting > 0 || unseenStop(r));
|
|
176
178
|
const nextDelay = () => (busy() ? Math.min(FAST_MS, intervalMs, stopGraceMs ?? FAST_MS) : intervalMs); // faster while busy, never slower
|
|
177
|
-
|
|
179
|
+
/** @internal Send a heartbeat after delayMs (0: now), replacing the one scheduled. */
|
|
180
|
+
export function schedule(delayMs) {
|
|
178
181
|
if (timer)
|
|
179
182
|
clearTimeout(timer);
|
|
180
183
|
timer = null;
|
|
@@ -266,53 +269,3 @@ export async function waitIfPaused(run) {
|
|
|
266
269
|
await new Promise((ok) => setTimeout(ok, 200));
|
|
267
270
|
}
|
|
268
271
|
}
|
|
269
|
-
// ── Approvals ─────────────────────────────────────────────────────────────────
|
|
270
|
-
/**
|
|
271
|
-
* Ask a person, through the Fences dashboard, to approve something, and wait for the answer:
|
|
272
|
-
*
|
|
273
|
-
* const approval = await requestApproval("Refund $240 to order 1182?", { amountUsd: 240 });
|
|
274
|
-
* if (!approval.granted) return `I didn't issue the refund: ${approval.note}`;
|
|
275
|
-
*
|
|
276
|
-
* If an amount is approved, the run's budget goes up by that much. Returns denied right away in
|
|
277
|
-
* local mode, outside a run, or when the server can't take requests, and denied after timeoutS
|
|
278
|
-
* (default 600) with no answer: it never hangs and never throws.
|
|
279
|
-
*/
|
|
280
|
-
export async function requestApproval(reason, options = {}) {
|
|
281
|
-
const { getActiveRun, recordSpan } = await import("./core.js");
|
|
282
|
-
const run = getActiveRun();
|
|
283
|
-
const timeoutS = options.timeoutS ?? 600, amountUsd = options.amountUsd ?? null;
|
|
284
|
-
if (!run)
|
|
285
|
-
return { granted: false, note: "requestApproval() was called outside a governed run" };
|
|
286
|
-
if (!client)
|
|
287
|
-
return { granted: false, note: "Local mode: there is no one to approve this request" };
|
|
288
|
-
if (!supported.approvals)
|
|
289
|
-
return { granted: false, note: "This Fences server doesn't support approvals" };
|
|
290
|
-
const approvalId = randomUUID(), started = performance.now();
|
|
291
|
-
// timeout_s lets the server expire the request when we stop waiting
|
|
292
|
-
const resp = await client.requestApproval(run.runId, { approval_id: approvalId, reason, amount_usd: amountUsd, timeout_s: timeoutS });
|
|
293
|
-
if (resp.unsupported) {
|
|
294
|
-
unsupported("approvals", "approval requests", "they are denied");
|
|
295
|
-
return { granted: false, note: "This Fences server doesn't support approvals" };
|
|
296
|
-
}
|
|
297
|
-
if (resp.network_error)
|
|
298
|
-
return { granted: false, note: "Couldn't reach Fences to ask for approval" };
|
|
299
|
-
run.control.approvals.set(approvalId, null);
|
|
300
|
-
run.control.awaiting += 1;
|
|
301
|
-
schedule(0); // poll fast while someone decides
|
|
302
|
-
const deadline = Date.now() + timeoutS * 1000;
|
|
303
|
-
while (run.control.approvals.get(approvalId) === null && Date.now() < deadline) {
|
|
304
|
-
await new Promise((ok) => setTimeout(ok, 100));
|
|
305
|
-
}
|
|
306
|
-
run.control.awaiting -= 1;
|
|
307
|
-
const answer = run.control.approvals.get(approvalId) ?? { granted: false, note: "No answer before the timeout" };
|
|
308
|
-
answer.approvalId = approvalId;
|
|
309
|
-
run.control.approvals.delete(approvalId);
|
|
310
|
-
if (answer.granted && answer.amountUsd && run.budgetUsd !== null)
|
|
311
|
-
run.budgetUsd += answer.amountUsd; // an unlimited run stays unlimited
|
|
312
|
-
recordSpan("fences.approval", performance.now() - started, {
|
|
313
|
-
"fences.approval.id": approvalId, "fences.approval.reason": reason, "fences.approval.amount_usd": amountUsd,
|
|
314
|
-
"fences.approval.granted": answer.granted, "fences.approval.by": answer.by, "fences.approval.note": answer.note,
|
|
315
|
-
"fences.approval.granted_usd": answer.amountUsd,
|
|
316
|
-
});
|
|
317
|
-
return answer;
|
|
318
|
-
}
|
package/dist/core.d.ts
CHANGED
|
@@ -1,81 +1,16 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* init(), governed() and checkpoint(): starting and ending runs, checking every step against
|
|
3
|
+
* its limits (the server decides in cloud mode, the same rules apply here otherwise), and
|
|
4
|
+
* recording decisions and spans.
|
|
5
|
+
*/
|
|
1
6
|
import { FencesClient } from "./client.js";
|
|
2
7
|
import * as control from "./control.js";
|
|
3
8
|
import * as events from "./events.js";
|
|
4
9
|
import { type CustomPrice } from "./pricing.js";
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
breachType: string | null;
|
|
10
|
-
/** First-person text, ready to return to the user. */
|
|
11
|
-
message: string;
|
|
12
|
-
/** An instruction to give the model so it wraps up. */
|
|
13
|
-
systemPrompt: string;
|
|
14
|
-
/** The run's total spend so far. */
|
|
15
|
-
spentUsd: number;
|
|
16
|
-
}
|
|
17
|
-
/** Every limit is optional; see governed(). */
|
|
18
|
-
export interface Limits {
|
|
19
|
-
/** Maximum spend for the run in USD. None anywhere: unlimited spend, with a warning. */
|
|
20
|
-
budgetUsd?: number | null;
|
|
21
|
-
/** Maximum checkpoints (steps). Default 100. */
|
|
22
|
-
maxIterations?: number | null;
|
|
23
|
-
/** Maximum wall-clock time in milliseconds. Default 300000 (5 minutes). */
|
|
24
|
-
maxDurationMs?: number | null;
|
|
25
|
-
/** Maximum tokens (input + output). 0 or unset: no limit. */
|
|
26
|
-
maxTokens?: number | null;
|
|
27
|
-
}
|
|
28
|
-
export declare const MAX_EVENTS_PER_RUN = 1000;
|
|
29
|
-
export interface Span {
|
|
30
|
-
name: string;
|
|
31
|
-
start_ts: number;
|
|
32
|
-
end_ts: number;
|
|
33
|
-
status: "ok" | "error";
|
|
34
|
-
attributes: Record<string, unknown>;
|
|
35
|
-
}
|
|
36
|
-
export declare class Run {
|
|
37
|
-
readonly agentName: string;
|
|
38
|
-
budgetUsd: number | null;
|
|
39
|
-
maxIterations: number;
|
|
40
|
-
maxDurationMs: number;
|
|
41
|
-
maxTokens: number;
|
|
42
|
-
readonly context: Record<string, string>;
|
|
43
|
-
readonly runId: `${string}-${string}-${string}-${string}-${string}`;
|
|
44
|
-
costUsd: number;
|
|
45
|
-
iterations: number;
|
|
46
|
-
tokensUsed: number;
|
|
47
|
-
readonly startedAt: number;
|
|
48
|
-
readonly decisions: {
|
|
49
|
-
timestamp: number;
|
|
50
|
-
iteration: number;
|
|
51
|
-
reasoning: string;
|
|
52
|
-
action: string | null;
|
|
53
|
-
}[];
|
|
54
|
-
/** Tool and model calls as OpenTelemetry-shaped spans, newest 1,000 kept. */
|
|
55
|
-
readonly events: Span[];
|
|
56
|
-
lastBreach: string | null;
|
|
57
|
-
exception?: events.ExceptionInfo;
|
|
58
|
-
warned: boolean;
|
|
59
|
-
/** Live control from the dashboard (see control.ts). */
|
|
60
|
-
readonly control: control.Control;
|
|
61
|
-
/** Held by guarded tools that are running. */
|
|
62
|
-
reservedUsd: number;
|
|
63
|
-
/** "hold": wait at a limit for someone to raise it (see InitOptions.onBreach). */
|
|
64
|
-
onBreach: "hold" | null;
|
|
65
|
-
private readonly abort;
|
|
66
|
-
/**
|
|
67
|
-
* Aborts when Fences halts the run after a stop it didn't reach a checkpoint for (see
|
|
68
|
-
* stopGraceS). Pass it to fetch and other cancellable work so a stuck run really stops:
|
|
69
|
-
* await fetch(url, { signal: getActiveRun()?.signal });
|
|
70
|
-
*/
|
|
71
|
-
get signal(): AbortSignal;
|
|
72
|
-
/** @internal */
|
|
73
|
-
_abort(reason: unknown): void;
|
|
74
|
-
constructor(agentName: string, budgetUsd: number | null, maxIterations: number, maxDurationMs: number, maxTokens: number, context: Record<string, string>);
|
|
75
|
-
get durationMs(): number;
|
|
76
|
-
}
|
|
77
|
-
/** The run in progress in this async context, if any. */
|
|
78
|
-
export declare function getActiveRun(): Run | null;
|
|
10
|
+
import { type CheckpointResult } from "./results.js";
|
|
11
|
+
import { Run, type Limits } from "./run.js";
|
|
12
|
+
export type { CheckpointResult } from "./results.js";
|
|
13
|
+
export { Run, getActiveRun, type Limits, type Span } from "./run.js";
|
|
79
14
|
export interface InitOptions {
|
|
80
15
|
/** Your Fences API key (fc_...). Required unless localOnly. */
|
|
81
16
|
apiKey?: string;
|
|
@@ -156,6 +91,8 @@ export interface Extra {
|
|
|
156
91
|
* Never throws for a breach. Outside a governed run it does nothing.
|
|
157
92
|
*/
|
|
158
93
|
export declare function checkpoint(response?: unknown, extra?: Extra): Promise<CheckpointResult>;
|
|
94
|
+
/** @internal A cost or count must be a finite number, zero or more: a NaN or negative one would switch the budget off. */
|
|
95
|
+
export declare function checkAmount(value: unknown, what: string): number;
|
|
159
96
|
/** @internal The breach the run is in by local rules, if any. */
|
|
160
97
|
export declare function localBreach(run: Run): string | null;
|
|
161
98
|
/**
|