bare-agent 0.31.0 → 0.32.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +1 -1
- package/bareagent.context.md +8 -1
- package/package.json +1 -1
- package/src/evaluator.d.ts +3 -0
- package/src/evaluator.js +27 -1
- package/src/provider-clipipe-tools.d.ts +121 -0
- package/src/provider-clipipe-tools.js +271 -0
- package/src/provider-clipipe.d.ts +53 -2
- package/src/provider-clipipe.js +112 -20
- package/src/recurse.d.ts +2 -2
- package/src/recurse.js +1 -1
package/README.md
CHANGED
|
@@ -121,7 +121,7 @@ console.log(result.count, result.matchedIds); // a code-derived count + the id
|
|
|
121
121
|
|
|
122
122
|
**Govern — one gate over both axes.** `wireGate(gate)` routes every LLM + tool call through one bareguard policy + audit + budget. Denied tools never reach the model; halts (turn / budget / content caps) exit cleanly. A plain deny stays advisory (the model can pivot to an allowed tool), but the Loop short-circuits a *spin* — `maxConsecutiveDenials` consecutive denials of the same action (default 3) stop the run with `error:'denied:<tool>'` instead of burning the budget to the cap; under `recurse` that surfaces as `{ incomplete, blocker:'governance-deny' }`. The same bound covers a tool that keeps *failing*: `maxIdenticalToolErrors` (default 3) stops a model re-sending a byte-identical call that cannot succeed, with `error:'stuck:<tool>'`. `require('bare-agent/bareguard')`
|
|
123
123
|
|
|
124
|
-
**Providers:** OpenAI-compatible (OpenAI, OpenRouter, Groq, vLLM, LM Studio), Anthropic, Gemini (native), Ollama, CLIPipe, Fallback — or bring your own (one `generate` method). All return the same shape; swap freely. Usage including prompt-cache tiers is normalized, so `result.metrics` reports honest cumulative tokens + cost — and `null`, never a silent `0`, for a model it couldn't price. A model that rejects a non-default `temperature` (e.g. `claude-sonnet-5`, OpenAI o1/gpt-5-class return a `400`) is handled gracefully — the provider drops the param and retries once rather than failing the call, surfacing `temperatureDropped` so a caller can report the effective value.
|
|
124
|
+
**Providers:** OpenAI-compatible (OpenAI, OpenRouter, Groq, vLLM, LM Studio), Anthropic, Gemini (native), Ollama, CLIPipe, Fallback — or bring your own (one `generate` method). All return the same shape; swap freely. Usage including prompt-cache tiers is normalized, so `result.metrics` reports honest cumulative tokens + cost — and `null`, never a silent `0`, for a model it couldn't price. A model that rejects a non-default `temperature` (e.g. `claude-sonnet-5`, OpenAI o1/gpt-5-class return a `400`) is handled gracefully — the provider drops the param and retries once rather than failing the call, surfacing `temperatureDropped` so a caller can report the effective value. **CLIPipe can drive a full agentic Loop — tools and turns — over a CLI *subscription* instead of the metered API** (`CLIPipeProvider({ command:'claude', args:['-p','--model','sonnet'], toolProtocol:'claude' })`): schema-validated tool emulation with the Loop keeping governance, and a capable model enforced upfront (v0.32.0, sonnet-class+ for tools; Claude CLI for now).
|
|
125
125
|
|
|
126
126
|
**The run tells you the truth about what happened on the wire.** Every provider reports **why** generation ended (`stopReason`, normalized across all of them and surfaced on **every** `Loop.run()` return), so a round the API **cut off at the token cap** can no longer masquerade as a finished answer: it returns `error: 'truncated:max_tokens'` with the partial text preserved, instead of a silent `error: null` (BA-6). Its tool calls are **refused, never executed** — a *complete* tool call always arrives tagged `tool_use`, so one riding a truncated round was cut off mid-generation with arguments missing, which is exactly how a truncated `shell_write` can zero a file. The same honesty now covers **every** non-clean terminal signal (BA-13): a safety `refusal` returns `error: 'refusal'` and a blown context window returns `error: 'context_exceeded'` (both were previously laundered into an empty success), while a resumable `pause_turn` makes the loop resume rather than terminate. `error` is the sole success signal; a bound firing preserves the model's work rather than discarding it (BA-5).
|
|
127
127
|
|
package/bareagent.context.md
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
# bareagent — Integration Guide
|
|
2
2
|
|
|
3
3
|
> For AI assistants and developers wiring bareagent into a project.
|
|
4
|
-
> v0.
|
|
4
|
+
> v0.32.0 | Node.js >= 18 | zero required deps (`bareguard >=0.9.0 <0.13.0` optional peer for governance) | Apache 2.0
|
|
5
5
|
>
|
|
6
6
|
> Full human guide with composition examples, design philosophy, and recipes: [Usage Guide](docs/02-features/usage-guide.md)
|
|
7
7
|
|
|
@@ -60,6 +60,7 @@ Eight entry points:
|
|
|
60
60
|
| Cache identical planner calls | Planner({ cacheTTL: 60000 }) |
|
|
61
61
|
| Stream CLIPipe output in real-time | CLIPipeProvider({ onChunk: fn }) |
|
|
62
62
|
| Get real usage + cost from a CLI provider | CLIPipeProvider({ parse: 'claude-json' }) |
|
|
63
|
+
| Drive tools over a CLI subscription (no metered API) | CLIPipeProvider({ toolProtocol: 'claude' }) |
|
|
63
64
|
| Browse the web (inline snapshots) | createBrowsingTools + Loop |
|
|
64
65
|
| Browse the web (token-efficient, disk-based) | `barebrowse` CLI session — snapshots to `.barebrowse/*.yml` |
|
|
65
66
|
| Assess website privacy risk | createBrowsingTools + Loop (requires `npm install wearehere`) |
|
|
@@ -800,6 +801,12 @@ new CLIPipe({ command: 'claude', args: ['--print'], systemPromptFlag: '--system-
|
|
|
800
801
|
new CLIPipe({ command: 'ollama', args: ['run', 'llama3.2'] })
|
|
801
802
|
// CLIPipe structured output (v0.26.0+) — map a CLI's JSON envelope to real usage + cost
|
|
802
803
|
new CLIPipe({ command: 'claude', args: ['-p', '--output-format', 'json'], parse: 'claude-json' })
|
|
804
|
+
// CLIPipe TOOL MODE (v0.32.0+) — drive a Loop's tools over a CLI SUBSCRIPTION (no metered API).
|
|
805
|
+
// toolProtocol enables schema-validated tool emulation; the Loop keeps governance/round-accounting.
|
|
806
|
+
// Tools are auto-used when passed; a weak model (e.g. haiku) is rejected UPFRONT by a capability
|
|
807
|
+
// probe (needs sonnet-class+ for tools; haiku is fine for plain text). setting-sources '' is applied
|
|
808
|
+
// internally for an ~18x cost drop. Claude-only for now.
|
|
809
|
+
new CLIPipe({ command: 'claude', args: ['-p', '--model', 'sonnet'], toolProtocol: 'claude' })
|
|
803
810
|
```
|
|
804
811
|
|
|
805
812
|
All return `{ text, toolCalls, usage: { inputTokens, outputTokens }, model?, costUsd? }`. The optional `model` (v0.16.1+) is the id the response was produced by — Loop prefers it over `provider.model` for cost accounting. By default CLIPipe returns `toolCalls: []` and zero usage (CLI tools don't report tokens) and omits `model`. **Structured output (v0.26.0+):** set `parse: 'claude-json'` (a preset for `claude -p --output-format json`) — or a `(stdout) => Partial<GenerateResult>` function for any other CLI — and CLIPipe maps the CLI's JSON envelope onto real `usage`, `model`, and `costUsd`, throwing `ProviderError` on a malformed/error envelope (never a silent raw-text fall-back). `costUsd` (optional `GenerateResult` field) is an **authoritative** per-call price the provider reports itself; when finite the Loop prefers it over the internal rate-table `estimateCost`, so a CLI-piped run enforces a bareguard USD cap with no local pricing table (a `0` counts as priced, distinct from null/unpriced). `toolCalls` stays `[]` regardless (CLIPipe is tool-free).
|
package/package.json
CHANGED
package/src/evaluator.d.ts
CHANGED
|
@@ -53,6 +53,9 @@ export type EvaluatorOptions = {
|
|
|
53
53
|
export type Criteria = {
|
|
54
54
|
/**
|
|
55
55
|
* - Deterministic check, no tokens.
|
|
56
|
+
* MUST return a boolean. A non-boolean return THROWS a `ValidationError` (it is not coerced): a truthy
|
|
57
|
+
* object/string/number — e.g. a test-runner result returned by mistake — would otherwise launder a
|
|
58
|
+
* FAILING check into a PASS. Thrown, it routes to `broken-verifier` at recurse's verify slot.
|
|
56
59
|
*/
|
|
57
60
|
predicate?: ((result: any) => boolean | Promise<boolean>) | undefined;
|
|
58
61
|
/**
|
package/src/evaluator.js
CHANGED
|
@@ -35,6 +35,9 @@ const { Loop } = require('./loop');
|
|
|
35
35
|
/**
|
|
36
36
|
* @typedef {object} Criteria
|
|
37
37
|
* @property {(result: any) => boolean | Promise<boolean>} [predicate] - Deterministic check, no tokens.
|
|
38
|
+
* MUST return a boolean. A non-boolean return THROWS a `ValidationError` (it is not coerced): a truthy
|
|
39
|
+
* object/string/number — e.g. a test-runner result returned by mistake — would otherwise launder a
|
|
40
|
+
* FAILING check into a PASS. Thrown, it routes to `broken-verifier` at recurse's verify slot.
|
|
38
41
|
* @property {string} [rubric] - Natural-language grading criteria an LLM scores. Exactly one of predicate|rubric|agentic.
|
|
39
42
|
* @property {string} [agentic] - Instructions for a tool-running critic (D9): how to EXERCISE the live artifact
|
|
40
43
|
* (open it, click, read console/network) and what would make it fail. Runs an ISOLATED Loop with the scoped
|
|
@@ -127,7 +130,30 @@ class Evaluator {
|
|
|
127
130
|
}
|
|
128
131
|
|
|
129
132
|
if (predicate) {
|
|
130
|
-
|
|
133
|
+
// BA-15 family (predicate seam): the contract is `=> boolean`. The OLD `!!(await predicate(...))`
|
|
134
|
+
// coerced ANY truthy return to a PASS — so a predicate that returned a test-runner RESULT instead
|
|
135
|
+
// of a boolean (`{exitCode:1,failures:3}`, `'3 failing'`, a count) laundered a FAILING check into
|
|
136
|
+
// `{status:'satisfied'}` (the optimistic-rounding class of BA-4/5/6/7/13; proven by
|
|
137
|
+
// `poc/rlmplans-predicate-coercion.mjs`). There is no safe non-boolean subset — an object is always
|
|
138
|
+
// truthy, a non-empty string is truthy regardless of meaning, a failure-count is truthy — so the
|
|
139
|
+
// ONLY correct return is a genuine boolean. A non-boolean is a broken arbiter: NAME it loudly rather
|
|
140
|
+
// than coerce it (BA-15's principle). Thrown here, it routes to `broken-verifier` at recurse's verify
|
|
141
|
+
// slot (`runArbiter` catches any non-Halt throw) and surfaces as a clean ValidationError standalone.
|
|
142
|
+
const raw = await predicate(result);
|
|
143
|
+
if (typeof raw !== 'boolean') {
|
|
144
|
+
// Name the TYPE only, never the value — an error string can reach a wired gate's audit log (F16/BA-1).
|
|
145
|
+
const got = raw === null ? 'null'
|
|
146
|
+
: raw === undefined ? 'undefined'
|
|
147
|
+
: Array.isArray(raw) ? 'an array'
|
|
148
|
+
: typeof raw === 'object' ? 'an object'
|
|
149
|
+
: `a ${typeof raw}`;
|
|
150
|
+
throw new ValidationError(
|
|
151
|
+
`[Evaluator] predicate must return a boolean, got ${got}. A truthy non-boolean ` +
|
|
152
|
+
'(a test-runner result object, a summary string, a failure count) would otherwise coerce to a ' +
|
|
153
|
+
'PASS — return true/false explicitly.',
|
|
154
|
+
);
|
|
155
|
+
}
|
|
156
|
+
const pass = raw;
|
|
131
157
|
return {
|
|
132
158
|
status: pass ? 'satisfied' : 'needs_revision',
|
|
133
159
|
pass,
|
|
@@ -0,0 +1,121 @@
|
|
|
1
|
+
export type Message = import("../types").Message;
|
|
2
|
+
export type ToolDef = import("../types").ToolDef;
|
|
3
|
+
export type ParsedEnvelope = {
|
|
4
|
+
action: "tool_call" | "final_answer";
|
|
5
|
+
toolName?: string | undefined;
|
|
6
|
+
toolArguments?: Record<string, any> | undefined;
|
|
7
|
+
answer?: string | undefined;
|
|
8
|
+
usage: import("../types").Usage;
|
|
9
|
+
model?: string | null | undefined;
|
|
10
|
+
costUsd?: number | undefined;
|
|
11
|
+
};
|
|
12
|
+
/** @typedef {import('../types').Message} Message */
|
|
13
|
+
/** @typedef {import('../types').ToolDef} ToolDef */
|
|
14
|
+
/**
|
|
15
|
+
* The JSON envelope the CLI is constrained to (claude `--json-schema`). `tool_call` carries the
|
|
16
|
+
* name + args; `final_answer` carries prose. A closed `action` enum is the discriminator.
|
|
17
|
+
* @type {string}
|
|
18
|
+
*/
|
|
19
|
+
export const ENVELOPE_SCHEMA: string;
|
|
20
|
+
/**
|
|
21
|
+
* One tool's signature line for the manifest. Protocol-agnostic.
|
|
22
|
+
* @param {ToolDef} t
|
|
23
|
+
* @returns {string}
|
|
24
|
+
*/
|
|
25
|
+
export function toolLine(t: ToolDef): string;
|
|
26
|
+
/**
|
|
27
|
+
* Build the tool-mode system prompt: the caller's own system stance (if any) + the tool manifest +
|
|
28
|
+
* the envelope contract. The contract text is load-bearing — the step-2 probe showed a weak model
|
|
29
|
+
* that BELIEVES it must execute the tool itself answers in prose ("I attempted… I failed"), so the
|
|
30
|
+
* prompt states plainly that EMITTING the envelope IS the call and "attempting/failing" is not a
|
|
31
|
+
* thing it can do. Protocol-agnostic (the envelope vocabulary is shared across CLIs).
|
|
32
|
+
* @param {string|null} baseSystem - the caller's system message content, or null.
|
|
33
|
+
* @param {ToolDef[]} tools
|
|
34
|
+
* @returns {string}
|
|
35
|
+
*/
|
|
36
|
+
export function buildToolSystemPrompt(baseSystem: string | null, tools: ToolDef[]): string;
|
|
37
|
+
/**
|
|
38
|
+
* Render the Loop's OpenAI-shaped transcript into a plain-text conversation a CLI can read. The
|
|
39
|
+
* load-bearing part: assistant `tool_calls` and `role:'tool'` results must SURVIVE (the default
|
|
40
|
+
* `_formatPrompt` drops them), and each result must trace to the call that produced it (id → name).
|
|
41
|
+
* Protocol-agnostic. The system message is excluded here — it rides the CLI's system-prompt flag.
|
|
42
|
+
* @param {Message[]} messages
|
|
43
|
+
* @returns {string}
|
|
44
|
+
*/
|
|
45
|
+
export function renderTranscript(messages: Message[]): string;
|
|
46
|
+
/**
|
|
47
|
+
* Map the claude `--output-format json` OUTER envelope's usage / model / cost onto neutral shapes.
|
|
48
|
+
* Shared by tool-mode {@link CLAUDE_TOOL_PROTOCOL.parseResult} and the plain-text `_parseClaudeJson`
|
|
49
|
+
* preset so the claude usage contract (token tiers, `modelUsage` first-key, `total_cost_usd`) lives
|
|
50
|
+
* in exactly ONE place — a future CLI format change touches this function, not two copies.
|
|
51
|
+
* @param {any} outer - the parsed outer CLI envelope (already validated non-null by the caller).
|
|
52
|
+
* @returns {{usage: import('../types').Usage, model: string|null, costUsd?: number}}
|
|
53
|
+
*/
|
|
54
|
+
export function mapClaudeMeta(outer: any): {
|
|
55
|
+
usage: import("../types").Usage;
|
|
56
|
+
model: string | null;
|
|
57
|
+
costUsd?: number;
|
|
58
|
+
};
|
|
59
|
+
export namespace CLAUDE_TOOL_PROTOCOL {
|
|
60
|
+
let name: string;
|
|
61
|
+
/**
|
|
62
|
+
* The extra CLI args for one tool-mode turn, appended after the caller's base args (`-p --model X`).
|
|
63
|
+
* `--tools ''` + `--strict-mcp-config` strip the CLI's own tools/MCP to a bare brain (step 1);
|
|
64
|
+
* `--setting-sources ''` suppresses cwd CLAUDE.md/memory/settings auto-discovery — a MEASURED
|
|
65
|
+
* ~18× cost drop (37,423 → 2,026 tokens/turn, step 2), load-bearing for a subscription strategy;
|
|
66
|
+
* `--system-prompt` REPLACES the CLI's prompt (vs --append); `--json-schema` + `--output-format
|
|
67
|
+
* json` force the parseable envelope.
|
|
68
|
+
* @param {string} systemPrompt
|
|
69
|
+
* @returns {string[]}
|
|
70
|
+
*/
|
|
71
|
+
function turnArgs(systemPrompt: string): string[];
|
|
72
|
+
/**
|
|
73
|
+
* Parse claude's `--output-format json` stdout into a neutral {@link ParsedEnvelope}. A malformed
|
|
74
|
+
* or error envelope is a LOUD `ProviderError` — NEVER a silent fall-through to prose (the whole
|
|
75
|
+
* point is that a broken tool turn cannot masquerade as a final answer, the BA-6 failure shape).
|
|
76
|
+
* @param {string} stdout
|
|
77
|
+
* @returns {ParsedEnvelope}
|
|
78
|
+
*/
|
|
79
|
+
function parseResult(stdout: string): ParsedEnvelope;
|
|
80
|
+
namespace probe {
|
|
81
|
+
let system: string;
|
|
82
|
+
let user: string;
|
|
83
|
+
function isCapable(parsed: ParsedEnvelope): boolean;
|
|
84
|
+
}
|
|
85
|
+
}
|
|
86
|
+
/** Resolve a `toolProtocol` option to an adapter. Claude-only for now; unknown names throw. */
|
|
87
|
+
export function resolveToolProtocol(name: any): {
|
|
88
|
+
name: string;
|
|
89
|
+
/**
|
|
90
|
+
* The extra CLI args for one tool-mode turn, appended after the caller's base args (`-p --model X`).
|
|
91
|
+
* `--tools ''` + `--strict-mcp-config` strip the CLI's own tools/MCP to a bare brain (step 1);
|
|
92
|
+
* `--setting-sources ''` suppresses cwd CLAUDE.md/memory/settings auto-discovery — a MEASURED
|
|
93
|
+
* ~18× cost drop (37,423 → 2,026 tokens/turn, step 2), load-bearing for a subscription strategy;
|
|
94
|
+
* `--system-prompt` REPLACES the CLI's prompt (vs --append); `--json-schema` + `--output-format
|
|
95
|
+
* json` force the parseable envelope.
|
|
96
|
+
* @param {string} systemPrompt
|
|
97
|
+
* @returns {string[]}
|
|
98
|
+
*/
|
|
99
|
+
turnArgs(systemPrompt: string): string[];
|
|
100
|
+
/**
|
|
101
|
+
* Parse claude's `--output-format json` stdout into a neutral {@link ParsedEnvelope}. A malformed
|
|
102
|
+
* or error envelope is a LOUD `ProviderError` — NEVER a silent fall-through to prose (the whole
|
|
103
|
+
* point is that a broken tool turn cannot masquerade as a final answer, the BA-6 failure shape).
|
|
104
|
+
* @param {string} stdout
|
|
105
|
+
* @returns {ParsedEnvelope}
|
|
106
|
+
*/
|
|
107
|
+
parseResult(stdout: string): ParsedEnvelope;
|
|
108
|
+
/**
|
|
109
|
+
* Capability-probe assets. The probe mirrors REAL-TASK shape — a question whose answer the model
|
|
110
|
+
* cannot know, with a tool that yields it, and NO "call the tool" instruction — because a trivial
|
|
111
|
+
* "call capability_ping" instruction is a FALSE POSITIVE: haiku emits it 4/4 yet fails real tasks
|
|
112
|
+
* 0/5 (proven in `poc/clipipe-tools-04-capability-probe.mjs`). The question shape sorts cleanly:
|
|
113
|
+
* sonnet CAPABLE 4/4, haiku INCAPABLE 0/4, stable. Capable iff it emits a `tool_call` for the tool.
|
|
114
|
+
*/
|
|
115
|
+
probe: {
|
|
116
|
+
system: string;
|
|
117
|
+
user: string;
|
|
118
|
+
/** @param {ParsedEnvelope} parsed @returns {boolean} */
|
|
119
|
+
isCapable: (parsed: ParsedEnvelope) => boolean;
|
|
120
|
+
};
|
|
121
|
+
};
|
|
@@ -0,0 +1,271 @@
|
|
|
1
|
+
'use strict';
|
|
2
|
+
|
|
3
|
+
const { ProviderError } = require('./errors');
|
|
4
|
+
|
|
5
|
+
/** @typedef {import('../types').Message} Message */
|
|
6
|
+
/** @typedef {import('../types').ToolDef} ToolDef */
|
|
7
|
+
|
|
8
|
+
// CLIPipe tool-mode support (v0.32.0). A subscription CLI (`claude -p`, …) is a plain
|
|
9
|
+
// TURN-provider: it takes text and returns text, with no native channel for a caller's tools. This
|
|
10
|
+
// module adds Option C — SCHEMA-VALIDATED TOOL EMULATION: the caller's tools are described in the
|
|
11
|
+
// system prompt, the CLI is constrained to a JSON envelope, and the envelope is parsed back into
|
|
12
|
+
// normalized `toolCalls` so bareagent's own `Loop` keeps ownership of the agentic cycle (round
|
|
13
|
+
// accounting, spin guards, stop-reason classification all still apply — unlike an MCP-callback
|
|
14
|
+
// design where the CLI would own the turns). Proven end-to-end through a real Loop in
|
|
15
|
+
// `poc/clipipe-tools-03-through-loop.mjs` (sonnet, multi-round).
|
|
16
|
+
//
|
|
17
|
+
// Two layers, deliberately split so a SECOND CLI (codex/gemini) slots in behind the same seam
|
|
18
|
+
// without touching the generic provider (the "keep the claude-specific parts isolated" constraint):
|
|
19
|
+
// - PROTOCOL-AGNOSTIC (renderTranscript / buildToolSystemPrompt): produce the text a CLI is fed.
|
|
20
|
+
// - PROTOCOL-SPECIFIC (CLAUDE_TOOL_PROTOCOL): the claude-CLI flags, the envelope schema, the
|
|
21
|
+
// result parse, and the capability-probe assets. Claude-only for now, by design.
|
|
22
|
+
|
|
23
|
+
/**
|
|
24
|
+
* The JSON envelope the CLI is constrained to (claude `--json-schema`). `tool_call` carries the
|
|
25
|
+
* name + args; `final_answer` carries prose. A closed `action` enum is the discriminator.
|
|
26
|
+
* @type {string}
|
|
27
|
+
*/
|
|
28
|
+
const ENVELOPE_SCHEMA = JSON.stringify({
|
|
29
|
+
type: 'object',
|
|
30
|
+
properties: {
|
|
31
|
+
action: { type: 'string', enum: ['tool_call', 'final_answer'] },
|
|
32
|
+
tool_name: { type: 'string' },
|
|
33
|
+
tool_arguments: { type: 'object' },
|
|
34
|
+
answer: { type: 'string' },
|
|
35
|
+
},
|
|
36
|
+
required: ['action'],
|
|
37
|
+
});
|
|
38
|
+
|
|
39
|
+
/**
|
|
40
|
+
* One tool's signature line for the manifest. Protocol-agnostic.
|
|
41
|
+
* @param {ToolDef} t
|
|
42
|
+
* @returns {string}
|
|
43
|
+
*/
|
|
44
|
+
function toolLine(t) {
|
|
45
|
+
const props = (t.parameters && t.parameters.properties) || {};
|
|
46
|
+
const args = Object.keys(props).map((k) => `${k}: ${props[k].type || 'any'}`).join(', ');
|
|
47
|
+
return `- ${t.name}(${args})${t.description ? ` — ${t.description}` : ''}`;
|
|
48
|
+
}
|
|
49
|
+
|
|
50
|
+
/**
|
|
51
|
+
* Build the tool-mode system prompt: the caller's own system stance (if any) + the tool manifest +
|
|
52
|
+
* the envelope contract. The contract text is load-bearing — the step-2 probe showed a weak model
|
|
53
|
+
* that BELIEVES it must execute the tool itself answers in prose ("I attempted… I failed"), so the
|
|
54
|
+
* prompt states plainly that EMITTING the envelope IS the call and "attempting/failing" is not a
|
|
55
|
+
* thing it can do. Protocol-agnostic (the envelope vocabulary is shared across CLIs).
|
|
56
|
+
* @param {string|null} baseSystem - the caller's system message content, or null.
|
|
57
|
+
* @param {ToolDef[]} tools
|
|
58
|
+
* @returns {string}
|
|
59
|
+
*/
|
|
60
|
+
function buildToolSystemPrompt(baseSystem, tools) {
|
|
61
|
+
const manifest = tools.length
|
|
62
|
+
? ['You can use these tools:', ...tools.map(toolLine)].join('\n')
|
|
63
|
+
: 'You have no tools.';
|
|
64
|
+
return [
|
|
65
|
+
baseSystem ? String(baseSystem) : 'You are the reasoning half of a tool-using system.',
|
|
66
|
+
'',
|
|
67
|
+
manifest,
|
|
68
|
+
'',
|
|
69
|
+
'An external runtime executes tools for you. Emitting a tool_call envelope IS how a tool runs —',
|
|
70
|
+
'you never execute one yourself, and "attempting" or "failing" to call a tool is not something',
|
|
71
|
+
'you can do. To use a tool: action="tool_call" with tool_name and tool_arguments. When the tool',
|
|
72
|
+
'results you need are already present in the conversation, reply action="final_answer" with the',
|
|
73
|
+
'answer. Reply ONLY with the JSON envelope.',
|
|
74
|
+
].join('\n');
|
|
75
|
+
}
|
|
76
|
+
|
|
77
|
+
/**
|
|
78
|
+
* Render the Loop's OpenAI-shaped transcript into a plain-text conversation a CLI can read. The
|
|
79
|
+
* load-bearing part: assistant `tool_calls` and `role:'tool'` results must SURVIVE (the default
|
|
80
|
+
* `_formatPrompt` drops them), and each result must trace to the call that produced it (id → name).
|
|
81
|
+
* Protocol-agnostic. The system message is excluded here — it rides the CLI's system-prompt flag.
|
|
82
|
+
* @param {Message[]} messages
|
|
83
|
+
* @returns {string}
|
|
84
|
+
*/
|
|
85
|
+
function renderTranscript(messages) {
|
|
86
|
+
/** @type {Map<string,string>} */
|
|
87
|
+
const idToName = new Map();
|
|
88
|
+
for (const m of messages) {
|
|
89
|
+
if (m.role === 'assistant' && Array.isArray(m.tool_calls)) {
|
|
90
|
+
for (const tc of m.tool_calls) {
|
|
91
|
+
if (!tc.id) continue;
|
|
92
|
+
const nm = (tc.function && tc.function.name) || tc.name || '?';
|
|
93
|
+
idToName.set(String(tc.id), String(nm));
|
|
94
|
+
}
|
|
95
|
+
}
|
|
96
|
+
}
|
|
97
|
+
const lines = [];
|
|
98
|
+
for (const m of messages) {
|
|
99
|
+
if (m.role === 'system') continue;
|
|
100
|
+
if (m.role === 'user') { lines.push(`User: ${m.content}`); continue; }
|
|
101
|
+
if (m.role === 'assistant') {
|
|
102
|
+
let s = m.content ? `Assistant: ${m.content}` : 'Assistant:';
|
|
103
|
+
if (Array.isArray(m.tool_calls)) {
|
|
104
|
+
for (const tc of m.tool_calls) {
|
|
105
|
+
const fn = tc.function || {};
|
|
106
|
+
const args = typeof fn.arguments === 'string' ? fn.arguments : JSON.stringify(fn.arguments || {});
|
|
107
|
+
s += `\n (you called ${fn.name || tc.name}(${args}))`;
|
|
108
|
+
}
|
|
109
|
+
}
|
|
110
|
+
lines.push(s);
|
|
111
|
+
continue;
|
|
112
|
+
}
|
|
113
|
+
if (m.role === 'tool') {
|
|
114
|
+
const name = idToName.get(m.tool_call_id || '') || '?';
|
|
115
|
+
lines.push(`Tool result from ${name}: ${m.content}`);
|
|
116
|
+
}
|
|
117
|
+
}
|
|
118
|
+
return lines.join('\n');
|
|
119
|
+
}
|
|
120
|
+
|
|
121
|
+
/**
|
|
122
|
+
* Map the claude `--output-format json` OUTER envelope's usage / model / cost onto neutral shapes.
|
|
123
|
+
* Shared by tool-mode {@link CLAUDE_TOOL_PROTOCOL.parseResult} and the plain-text `_parseClaudeJson`
|
|
124
|
+
* preset so the claude usage contract (token tiers, `modelUsage` first-key, `total_cost_usd`) lives
|
|
125
|
+
* in exactly ONE place — a future CLI format change touches this function, not two copies.
|
|
126
|
+
* @param {any} outer - the parsed outer CLI envelope (already validated non-null by the caller).
|
|
127
|
+
* @returns {{usage: import('../types').Usage, model: string|null, costUsd?: number}}
|
|
128
|
+
*/
|
|
129
|
+
function mapClaudeMeta(outer) {
|
|
130
|
+
const u = (outer.usage && typeof outer.usage === 'object') ? outer.usage : {};
|
|
131
|
+
/** @type {import('../types').Usage} */
|
|
132
|
+
const usage = {
|
|
133
|
+
inputTokens: Number(u.input_tokens) || 0,
|
|
134
|
+
outputTokens: Number(u.output_tokens) || 0,
|
|
135
|
+
};
|
|
136
|
+
// Absent cache tiers mean the model didn't cache — omit rather than emit a synthetic 0 (per Usage docs).
|
|
137
|
+
if (Number.isFinite(u.cache_read_input_tokens)) usage.cacheReadTokens = u.cache_read_input_tokens;
|
|
138
|
+
if (Number.isFinite(u.cache_creation_input_tokens)) usage.cacheCreationTokens = u.cache_creation_input_tokens;
|
|
139
|
+
// `modelUsage` is an object keyed by model id (e.g. {"claude-opus-4-8[1m]": {...}}) — take the first key.
|
|
140
|
+
const model = (outer.modelUsage && typeof outer.modelUsage === 'object')
|
|
141
|
+
? (Object.keys(outer.modelUsage)[0] ?? null)
|
|
142
|
+
: null;
|
|
143
|
+
/** @type {{usage: import('../types').Usage, model: string|null, costUsd?: number}} */
|
|
144
|
+
const meta = { usage, model };
|
|
145
|
+
// The CLI's own price is authoritative (a subscription run reports an equivalent cost even at $0
|
|
146
|
+
// marginal) — feeds bareguard's USD axis with no local rate table. Only a finite number counts.
|
|
147
|
+
if (Number.isFinite(outer.total_cost_usd)) meta.costUsd = outer.total_cost_usd;
|
|
148
|
+
return meta;
|
|
149
|
+
}
|
|
150
|
+
|
|
151
|
+
/**
|
|
152
|
+
* @typedef {object} ParsedEnvelope
|
|
153
|
+
* @property {'tool_call'|'final_answer'} action
|
|
154
|
+
* @property {string} [toolName]
|
|
155
|
+
* @property {Record<string, any>} [toolArguments]
|
|
156
|
+
* @property {string} [answer]
|
|
157
|
+
* @property {import('../types').Usage} usage
|
|
158
|
+
* @property {string|null} [model]
|
|
159
|
+
* @property {number} [costUsd]
|
|
160
|
+
*/
|
|
161
|
+
|
|
162
|
+
/**
|
|
163
|
+
* The claude-CLI tool protocol. Everything CLI-shaped lives here; a second CLI is a sibling object
|
|
164
|
+
* implementing the same three members. Isolated so the generic provider never grows a claude branch.
|
|
165
|
+
*/
|
|
166
|
+
const CLAUDE_TOOL_PROTOCOL = {
|
|
167
|
+
name: 'claude',
|
|
168
|
+
|
|
169
|
+
/**
|
|
170
|
+
* The extra CLI args for one tool-mode turn, appended after the caller's base args (`-p --model X`).
|
|
171
|
+
* `--tools ''` + `--strict-mcp-config` strip the CLI's own tools/MCP to a bare brain (step 1);
|
|
172
|
+
* `--setting-sources ''` suppresses cwd CLAUDE.md/memory/settings auto-discovery — a MEASURED
|
|
173
|
+
* ~18× cost drop (37,423 → 2,026 tokens/turn, step 2), load-bearing for a subscription strategy;
|
|
174
|
+
* `--system-prompt` REPLACES the CLI's prompt (vs --append); `--json-schema` + `--output-format
|
|
175
|
+
* json` force the parseable envelope.
|
|
176
|
+
* @param {string} systemPrompt
|
|
177
|
+
* @returns {string[]}
|
|
178
|
+
*/
|
|
179
|
+
turnArgs(systemPrompt) {
|
|
180
|
+
return [
|
|
181
|
+
'--tools', '', '--strict-mcp-config', '--setting-sources', '',
|
|
182
|
+
'--system-prompt', systemPrompt,
|
|
183
|
+
'--json-schema', ENVELOPE_SCHEMA,
|
|
184
|
+
'--output-format', 'json',
|
|
185
|
+
];
|
|
186
|
+
},
|
|
187
|
+
|
|
188
|
+
/**
|
|
189
|
+
* Parse claude's `--output-format json` stdout into a neutral {@link ParsedEnvelope}. A malformed
|
|
190
|
+
* or error envelope is a LOUD `ProviderError` — NEVER a silent fall-through to prose (the whole
|
|
191
|
+
* point is that a broken tool turn cannot masquerade as a final answer, the BA-6 failure shape).
|
|
192
|
+
* @param {string} stdout
|
|
193
|
+
* @returns {ParsedEnvelope}
|
|
194
|
+
*/
|
|
195
|
+
parseResult(stdout) {
|
|
196
|
+
let outer;
|
|
197
|
+
try {
|
|
198
|
+
outer = JSON.parse(stdout);
|
|
199
|
+
} catch (_) {
|
|
200
|
+
const preview = stdout.length > 200 ? `${stdout.slice(0, 200)}…` : stdout;
|
|
201
|
+
throw new ProviderError(`[CLIPipeProvider] tool-mode expected JSON on stdout, got: ${preview}`, /** @type {any} */ ({ status: 0 }));
|
|
202
|
+
}
|
|
203
|
+
if (!outer || typeof outer !== 'object' || outer.is_error === true || outer.subtype !== 'success') {
|
|
204
|
+
const detail = outer && typeof outer.result === 'string' ? outer.result : JSON.stringify(outer && outer.subtype);
|
|
205
|
+
throw new ProviderError(`[CLIPipeProvider] claude CLI reported failure (subtype='${outer && outer.subtype}'): ${detail}`, /** @type {any} */ ({ status: 0 }));
|
|
206
|
+
}
|
|
207
|
+
let env;
|
|
208
|
+
try {
|
|
209
|
+
env = JSON.parse(outer.result);
|
|
210
|
+
} catch (_) {
|
|
211
|
+
throw new ProviderError(`[CLIPipeProvider] tool-mode envelope was not valid JSON: ${String(outer.result).slice(0, 200)}`, /** @type {any} */ ({ status: 0 }));
|
|
212
|
+
}
|
|
213
|
+
if (!env || (env.action !== 'tool_call' && env.action !== 'final_answer')) {
|
|
214
|
+
throw new ProviderError(`[CLIPipeProvider] tool-mode envelope missing a valid action (got ${env && JSON.stringify(env.action)})`, /** @type {any} */ ({ status: 0 }));
|
|
215
|
+
}
|
|
216
|
+
if (env.action === 'tool_call' && (typeof env.tool_name !== 'string' || !env.tool_name)) {
|
|
217
|
+
throw new ProviderError('[CLIPipeProvider] tool-mode tool_call envelope has no tool_name', /** @type {any} */ ({ status: 0 }));
|
|
218
|
+
}
|
|
219
|
+
|
|
220
|
+
const { usage, model, costUsd } = mapClaudeMeta(outer);
|
|
221
|
+
/** @type {ParsedEnvelope} */
|
|
222
|
+
const parsed = { action: env.action, usage, model };
|
|
223
|
+
if (costUsd !== undefined) parsed.costUsd = costUsd;
|
|
224
|
+
if (env.action === 'tool_call') {
|
|
225
|
+
parsed.toolName = env.tool_name;
|
|
226
|
+
parsed.toolArguments = (env.tool_arguments && typeof env.tool_arguments === 'object') ? env.tool_arguments : {};
|
|
227
|
+
} else {
|
|
228
|
+
parsed.answer = typeof env.answer === 'string' ? env.answer : '';
|
|
229
|
+
}
|
|
230
|
+
return parsed;
|
|
231
|
+
},
|
|
232
|
+
|
|
233
|
+
/**
|
|
234
|
+
* Capability-probe assets. The probe mirrors REAL-TASK shape — a question whose answer the model
|
|
235
|
+
* cannot know, with a tool that yields it, and NO "call the tool" instruction — because a trivial
|
|
236
|
+
* "call capability_ping" instruction is a FALSE POSITIVE: haiku emits it 4/4 yet fails real tasks
|
|
237
|
+
* 0/5 (proven in `poc/clipipe-tools-04-capability-probe.mjs`). The question shape sorts cleanly:
|
|
238
|
+
* sonnet CAPABLE 4/4, haiku INCAPABLE 0/4, stable. Capable iff it emits a `tool_call` for the tool.
|
|
239
|
+
*/
|
|
240
|
+
probe: {
|
|
241
|
+
system: [
|
|
242
|
+
'You are the reasoning half of a tool-using system. An external runtime executes tools for you.',
|
|
243
|
+
'',
|
|
244
|
+
'Tool available:',
|
|
245
|
+
'- lookup_code(name: string) — returns the secret verification code for a given name.',
|
|
246
|
+
'',
|
|
247
|
+
'You have NO knowledge of verification codes; the ONLY way to obtain one is to emit a tool_call.',
|
|
248
|
+
'Emitting a tool_call envelope IS how a tool runs; you never execute one yourself, and you cannot',
|
|
249
|
+
'"attempt" or "fail" to call one. Reply ONLY with the JSON envelope.',
|
|
250
|
+
].join('\n'),
|
|
251
|
+
user: 'What is the verification code for "orchard-42"?',
|
|
252
|
+
/** @param {ParsedEnvelope} parsed @returns {boolean} */
|
|
253
|
+
isCapable: (parsed) => parsed.action === 'tool_call' && parsed.toolName === 'lookup_code',
|
|
254
|
+
},
|
|
255
|
+
};
|
|
256
|
+
|
|
257
|
+
/** Resolve a `toolProtocol` option to an adapter. Claude-only for now; unknown names throw. */
|
|
258
|
+
function resolveToolProtocol(name) {
|
|
259
|
+
if (name === 'claude') return CLAUDE_TOOL_PROTOCOL;
|
|
260
|
+
throw new Error(`[CLIPipeProvider] unknown toolProtocol '${name}' — only 'claude' is supported`);
|
|
261
|
+
}
|
|
262
|
+
|
|
263
|
+
module.exports = {
|
|
264
|
+
ENVELOPE_SCHEMA,
|
|
265
|
+
toolLine,
|
|
266
|
+
buildToolSystemPrompt,
|
|
267
|
+
renderTranscript,
|
|
268
|
+
mapClaudeMeta,
|
|
269
|
+
CLAUDE_TOOL_PROTOCOL,
|
|
270
|
+
resolveToolProtocol,
|
|
271
|
+
};
|
|
@@ -34,6 +34,14 @@ export type CLIPipeOptions = {
|
|
|
34
34
|
* - Opt-in structured-output parser for stdout. Default (unset) returns stdout verbatim as `text` with zero usage (no behavior change). `'claude-json'` is a shipped preset for `claude -p --output-format json`: it maps the CLI's result envelope onto `GenerateResult` (text←`result`, usage←`usage.*`, model←first `modelUsage` key, costUsd←`total_cost_usd`) and throws `ProviderError` on malformed JSON or an error envelope (`is_error`/non-success subtype). A function is the CLI-agnostic escape hatch: it receives trimmed stdout and returns a partial `GenerateResult` (merged over defaults); throw to signal a parse failure.
|
|
35
35
|
*/
|
|
36
36
|
parse?: "claude-json" | ((stdout: string) => Partial<GenerateResult>) | undefined;
|
|
37
|
+
/**
|
|
38
|
+
* - Opt into TOOL MODE (v0.32.0). A subscription CLI is a plain turn-provider with no native tool channel; this enables schema-validated tool EMULATION so a caller's `tools` work over the CLI (letting a Claude/etc. subscription drive an agentic Loop without metered-API spend). When set, ANY `generate(msgs, tools)` call with a non-empty `tools` array auto-uses emulation (the caller's own system stance + a tool manifest + a JSON envelope, parsed back into normalized `toolCalls`); an empty `tools` array is unchanged plain-text. When NOT set, `tools` are IGNORED (plain-text mode, the long-standing behavior — a non-tool-calling CLI can legitimately sit in a Loop that has tools mounted) with a one-time `console.warn` for visibility. Claude-only for now (`'claude'`); the claude-specific flags/schema/parse live in `provider-clipipe-tools.js` so a second CLI slots in behind the same seam. NOTE: tool mode requires a capable model — weak models (e.g. haiku) answer in prose instead of calling tools; see `probeCapability`.
|
|
39
|
+
*/
|
|
40
|
+
toolProtocol?: "claude" | undefined;
|
|
41
|
+
/**
|
|
42
|
+
* - (tool mode only) On the first tool-mode `generate`, run ONE cheap upfront probe that asks the model to obtain unknowable info via a tool. If it answers in prose instead of emitting a tool_call, throw a loud `ProviderError` naming the model — FAIL FAST rather than silently degrade mid-run (the weak-model failure mode). Behaviour-based, never a model name-list (a roster goes stale, BA-10). The verdict is cached per instance (one probe per provider, not per turn). Set `false` to skip when the caller already knows the model is capable.
|
|
43
|
+
*/
|
|
44
|
+
probeCapability?: boolean | undefined;
|
|
37
45
|
};
|
|
38
46
|
/** @typedef {import('../types').Message} Message */
|
|
39
47
|
/** @typedef {import('../types').ToolDef} ToolDef */
|
|
@@ -48,6 +56,8 @@ export type CLIPipeOptions = {
|
|
|
48
56
|
* @property {string} [systemPromptFlag] - CLI flag for system prompt (e.g. '--system'). When set, system messages are extracted and passed via this flag instead of stdin.
|
|
49
57
|
* @property {(chunk: string) => void} [onChunk] - Called with each stdout chunk as it streams.
|
|
50
58
|
* @property {'claude-json'|((stdout: string) => Partial<GenerateResult>)} [parse] - Opt-in structured-output parser for stdout. Default (unset) returns stdout verbatim as `text` with zero usage (no behavior change). `'claude-json'` is a shipped preset for `claude -p --output-format json`: it maps the CLI's result envelope onto `GenerateResult` (text←`result`, usage←`usage.*`, model←first `modelUsage` key, costUsd←`total_cost_usd`) and throws `ProviderError` on malformed JSON or an error envelope (`is_error`/non-success subtype). A function is the CLI-agnostic escape hatch: it receives trimmed stdout and returns a partial `GenerateResult` (merged over defaults); throw to signal a parse failure.
|
|
59
|
+
* @property {'claude'} [toolProtocol] - Opt into TOOL MODE (v0.32.0). A subscription CLI is a plain turn-provider with no native tool channel; this enables schema-validated tool EMULATION so a caller's `tools` work over the CLI (letting a Claude/etc. subscription drive an agentic Loop without metered-API spend). When set, ANY `generate(msgs, tools)` call with a non-empty `tools` array auto-uses emulation (the caller's own system stance + a tool manifest + a JSON envelope, parsed back into normalized `toolCalls`); an empty `tools` array is unchanged plain-text. When NOT set, `tools` are IGNORED (plain-text mode, the long-standing behavior — a non-tool-calling CLI can legitimately sit in a Loop that has tools mounted) with a one-time `console.warn` for visibility. Claude-only for now (`'claude'`); the claude-specific flags/schema/parse live in `provider-clipipe-tools.js` so a second CLI slots in behind the same seam. NOTE: tool mode requires a capable model — weak models (e.g. haiku) answer in prose instead of calling tools; see `probeCapability`.
|
|
60
|
+
* @property {boolean} [probeCapability=true] - (tool mode only) On the first tool-mode `generate`, run ONE cheap upfront probe that asks the model to obtain unknowable info via a tool. If it answers in prose instead of emitting a tool_call, throw a loud `ProviderError` naming the model — FAIL FAST rather than silently degrade mid-run (the weak-model failure mode). Behaviour-based, never a model name-list (a roster goes stale, BA-10). The verdict is cached per instance (one probe per provider, not per turn). Set `false` to skip when the caller already knows the model is capable.
|
|
51
61
|
*/
|
|
52
62
|
export class CLIPipeProvider {
|
|
53
63
|
/**
|
|
@@ -64,10 +74,27 @@ export class CLIPipeProvider {
|
|
|
64
74
|
systemPromptFlag: string | null;
|
|
65
75
|
onChunk: ((chunk: string) => void) | null;
|
|
66
76
|
parse: "claude-json" | ((stdout: string) => Partial<GenerateResult>) | null;
|
|
77
|
+
toolProtocol: {
|
|
78
|
+
name: string;
|
|
79
|
+
turnArgs(systemPrompt: string): string[];
|
|
80
|
+
parseResult(stdout: string): ParsedEnvelope;
|
|
81
|
+
probe: {
|
|
82
|
+
system: string;
|
|
83
|
+
user: string;
|
|
84
|
+
isCapable: (parsed: ParsedEnvelope) => boolean;
|
|
85
|
+
};
|
|
86
|
+
} | null;
|
|
87
|
+
probeCapability: boolean;
|
|
88
|
+
/** @type {Promise<void>|null} */
|
|
89
|
+
_toolCapability: Promise<void> | null;
|
|
67
90
|
/**
|
|
68
|
-
* Generate a response by piping messages to the CLI command.
|
|
91
|
+
* Generate a response by piping messages to the CLI command. With a `toolProtocol` configured and
|
|
92
|
+
* a non-empty `tools` array, routes to schema-validated tool EMULATION (v0.32.0); otherwise the
|
|
93
|
+
* plain-text path below (unchanged). Passing `tools` with no `toolProtocol` warns ONCE and ignores
|
|
94
|
+
* them (a non-tool-calling CLI legitimately coexists in a Loop with tools mounted); the loud
|
|
95
|
+
* failure for a genuinely tool-incapable model lives in the tool-mode capability probe.
|
|
69
96
|
* @param {Message[]} messages - Conversation messages in OpenAI format.
|
|
70
|
-
* @param {ToolDef[]} [tools=[]] -
|
|
97
|
+
* @param {ToolDef[]} [tools=[]] - Caller tools. Honored only in tool mode (`toolProtocol` set).
|
|
71
98
|
* @param {Record<string, any>} [options={}] - Unused.
|
|
72
99
|
* @returns {Promise<GenerateResult>}
|
|
73
100
|
* @throws {Error} `[CLIPipeProvider] failed to spawn "cmd": ...` — when the command cannot be found or executed.
|
|
@@ -76,6 +103,30 @@ export class CLIPipeProvider {
|
|
|
76
103
|
* @throws {Error} `[CLIPipeProvider] process produced no output` — when stdout is empty.
|
|
77
104
|
*/
|
|
78
105
|
generate(messages: Message[], tools?: ToolDef[], options?: Record<string, any>): Promise<GenerateResult>;
|
|
106
|
+
_warnedNoProtocol: boolean | undefined;
|
|
107
|
+
/**
|
|
108
|
+
* Tool mode (v0.32.0) — one turn of schema-validated tool emulation. Renders the Loop's
|
|
109
|
+
* OpenAI-shaped transcript to text, injects the caller's system stance + a tool manifest + the
|
|
110
|
+
* envelope contract, spawns the CLI under the protocol's flags, and parses the envelope back into
|
|
111
|
+
* normalized `toolCalls` (a `tool_call`) or `text` (a `final_answer`). The Loop drives the cycle.
|
|
112
|
+
* @param {Message[]} messages
|
|
113
|
+
* @param {ToolDef[]} tools
|
|
114
|
+
* @returns {Promise<GenerateResult>}
|
|
115
|
+
*/
|
|
116
|
+
_generateWithTools(messages: Message[], tools: ToolDef[]): Promise<GenerateResult>;
|
|
117
|
+
_toolCallSeq: any;
|
|
118
|
+
/**
|
|
119
|
+
* Run the upfront capability probe ONCE per instance (cached), unless `probeCapability` is off.
|
|
120
|
+
* A model that answers the probe in prose instead of emitting a tool_call throws a loud
|
|
121
|
+
* `ProviderError` — fail fast, never silently degrade to a no-tools run mid-conversation.
|
|
122
|
+
* NOTE: the probe is a single internal CLI turn whose token usage/cost is NOT surfaced to the Loop
|
|
123
|
+
* (it never flows to `onLlmResult`), so a wired budget gate does not see it — negligible for the
|
|
124
|
+
* subscription use case this exists for (flat cost, one probe per instance), by design.
|
|
125
|
+
* @returns {Promise<void>}
|
|
126
|
+
*/
|
|
127
|
+
_ensureToolCapability(): Promise<void>;
|
|
128
|
+
/** Best-effort model id from `--model X` in the base args, for a clearer probe-failure message. */
|
|
129
|
+
_modelFromArgs(): string | null;
|
|
79
130
|
/**
|
|
80
131
|
* Map the `claude -p --output-format json` result envelope onto a normalized GenerateResult.
|
|
81
132
|
* The caller explicitly opted into structured output, so a malformed or error envelope is a LOUD
|
package/src/provider-clipipe.js
CHANGED
|
@@ -2,6 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
const { spawn } = require('child_process');
|
|
4
4
|
const { ProviderError } = require('./errors');
|
|
5
|
+
const { buildToolSystemPrompt, renderTranscript, resolveToolProtocol, mapClaudeMeta } = require('./provider-clipipe-tools');
|
|
5
6
|
|
|
6
7
|
/** @typedef {import('../types').Message} Message */
|
|
7
8
|
/** @typedef {import('../types').ToolDef} ToolDef */
|
|
@@ -17,6 +18,8 @@ const { ProviderError } = require('./errors');
|
|
|
17
18
|
* @property {string} [systemPromptFlag] - CLI flag for system prompt (e.g. '--system'). When set, system messages are extracted and passed via this flag instead of stdin.
|
|
18
19
|
* @property {(chunk: string) => void} [onChunk] - Called with each stdout chunk as it streams.
|
|
19
20
|
* @property {'claude-json'|((stdout: string) => Partial<GenerateResult>)} [parse] - Opt-in structured-output parser for stdout. Default (unset) returns stdout verbatim as `text` with zero usage (no behavior change). `'claude-json'` is a shipped preset for `claude -p --output-format json`: it maps the CLI's result envelope onto `GenerateResult` (text←`result`, usage←`usage.*`, model←first `modelUsage` key, costUsd←`total_cost_usd`) and throws `ProviderError` on malformed JSON or an error envelope (`is_error`/non-success subtype). A function is the CLI-agnostic escape hatch: it receives trimmed stdout and returns a partial `GenerateResult` (merged over defaults); throw to signal a parse failure.
|
|
21
|
+
* @property {'claude'} [toolProtocol] - Opt into TOOL MODE (v0.32.0). A subscription CLI is a plain turn-provider with no native tool channel; this enables schema-validated tool EMULATION so a caller's `tools` work over the CLI (letting a Claude/etc. subscription drive an agentic Loop without metered-API spend). When set, ANY `generate(msgs, tools)` call with a non-empty `tools` array auto-uses emulation (the caller's own system stance + a tool manifest + a JSON envelope, parsed back into normalized `toolCalls`); an empty `tools` array is unchanged plain-text. When NOT set, `tools` are IGNORED (plain-text mode, the long-standing behavior — a non-tool-calling CLI can legitimately sit in a Loop that has tools mounted) with a one-time `console.warn` for visibility. Claude-only for now (`'claude'`); the claude-specific flags/schema/parse live in `provider-clipipe-tools.js` so a second CLI slots in behind the same seam. NOTE: tool mode requires a capable model — weak models (e.g. haiku) answer in prose instead of calling tools; see `probeCapability`.
|
|
22
|
+
* @property {boolean} [probeCapability=true] - (tool mode only) On the first tool-mode `generate`, run ONE cheap upfront probe that asks the model to obtain unknowable info via a tool. If it answers in prose instead of emitting a tool_call, throw a loud `ProviderError` naming the model — FAIL FAST rather than silently degrade mid-run (the weak-model failure mode). Behaviour-based, never a model name-list (a roster goes stale, BA-10). The verdict is cached per instance (one probe per provider, not per turn). Set `false` to skip when the caller already knows the model is capable.
|
|
20
23
|
*/
|
|
21
24
|
|
|
22
25
|
class CLIPipeProvider {
|
|
@@ -38,12 +41,23 @@ class CLIPipeProvider {
|
|
|
38
41
|
throw new Error("[CLIPipeProvider] options.parse must be 'claude-json' or a function");
|
|
39
42
|
}
|
|
40
43
|
this.parse = options.parse || null;
|
|
44
|
+
// Tool mode (v0.32.0). Resolve the protocol adapter eagerly so an unknown name fails at
|
|
45
|
+
// construction, not mid-run. `_toolCapability` caches the upfront probe verdict per instance
|
|
46
|
+
// (null = not yet probed; a Promise while in flight; true once confirmed capable).
|
|
47
|
+
this.toolProtocol = options.toolProtocol ? resolveToolProtocol(options.toolProtocol) : null;
|
|
48
|
+
this.probeCapability = options.probeCapability !== false;
|
|
49
|
+
/** @type {Promise<void>|null} */
|
|
50
|
+
this._toolCapability = null;
|
|
41
51
|
}
|
|
42
52
|
|
|
43
53
|
/**
|
|
44
|
-
* Generate a response by piping messages to the CLI command.
|
|
54
|
+
* Generate a response by piping messages to the CLI command. With a `toolProtocol` configured and
|
|
55
|
+
* a non-empty `tools` array, routes to schema-validated tool EMULATION (v0.32.0); otherwise the
|
|
56
|
+
* plain-text path below (unchanged). Passing `tools` with no `toolProtocol` warns ONCE and ignores
|
|
57
|
+
* them (a non-tool-calling CLI legitimately coexists in a Loop with tools mounted); the loud
|
|
58
|
+
* failure for a genuinely tool-incapable model lives in the tool-mode capability probe.
|
|
45
59
|
* @param {Message[]} messages - Conversation messages in OpenAI format.
|
|
46
|
-
* @param {ToolDef[]} [tools=[]] -
|
|
60
|
+
* @param {ToolDef[]} [tools=[]] - Caller tools. Honored only in tool mode (`toolProtocol` set).
|
|
47
61
|
* @param {Record<string, any>} [options={}] - Unused.
|
|
48
62
|
* @returns {Promise<GenerateResult>}
|
|
49
63
|
* @throws {Error} `[CLIPipeProvider] failed to spawn "cmd": ...` — when the command cannot be found or executed.
|
|
@@ -52,6 +66,24 @@ class CLIPipeProvider {
|
|
|
52
66
|
* @throws {Error} `[CLIPipeProvider] process produced no output` — when stdout is empty.
|
|
53
67
|
*/
|
|
54
68
|
async generate(messages, tools = [], options = {}) {
|
|
69
|
+
if (Array.isArray(tools) && tools.length > 0) {
|
|
70
|
+
if (this.toolProtocol) return this._generateWithTools(messages, tools);
|
|
71
|
+
// No protocol configured → plain-text mode, tools IGNORED — the long-standing behavior, kept
|
|
72
|
+
// for backward compatibility (a non-tool-calling CLI legitimately coexists in a Loop that has
|
|
73
|
+
// tools mounted, e.g. via MCP; the Loop's contract lets a provider simply not call them).
|
|
74
|
+
// A silent ignore is the trap the caller might not notice, so warn ONCE per instance (the
|
|
75
|
+
// provider-temperature BA-10 pattern) — visible, not fatal. Genuine loud-failure for tool
|
|
76
|
+
// mode lives in the capability probe, where a weak model that SHOULD call tools cannot.
|
|
77
|
+
if (!this._warnedNoProtocol) {
|
|
78
|
+
this._warnedNoProtocol = true;
|
|
79
|
+
// eslint-disable-next-line no-console
|
|
80
|
+
console.warn(
|
|
81
|
+
'[CLIPipeProvider] received tools but no toolProtocol is configured — tools are IGNORED ' +
|
|
82
|
+
"(plain-text mode). Construct with { toolProtocol: 'claude' } to enable tool emulation.",
|
|
83
|
+
);
|
|
84
|
+
}
|
|
85
|
+
}
|
|
86
|
+
|
|
55
87
|
/** @type {string[]} */
|
|
56
88
|
let extraArgs = [];
|
|
57
89
|
let promptMessages = messages;
|
|
@@ -85,6 +117,82 @@ class CLIPipeProvider {
|
|
|
85
117
|
};
|
|
86
118
|
}
|
|
87
119
|
|
|
120
|
+
/**
|
|
121
|
+
* Tool mode (v0.32.0) — one turn of schema-validated tool emulation. Renders the Loop's
|
|
122
|
+
* OpenAI-shaped transcript to text, injects the caller's system stance + a tool manifest + the
|
|
123
|
+
* envelope contract, spawns the CLI under the protocol's flags, and parses the envelope back into
|
|
124
|
+
* normalized `toolCalls` (a `tool_call`) or `text` (a `final_answer`). The Loop drives the cycle.
|
|
125
|
+
* @param {Message[]} messages
|
|
126
|
+
* @param {ToolDef[]} tools
|
|
127
|
+
* @returns {Promise<GenerateResult>}
|
|
128
|
+
*/
|
|
129
|
+
async _generateWithTools(messages, tools) {
|
|
130
|
+
await this._ensureToolCapability();
|
|
131
|
+
const proto = this.toolProtocol;
|
|
132
|
+
if (!proto) throw new ProviderError('[CLIPipeProvider] tool mode not configured', /** @type {any} */ ({ status: 0 })); // unreachable: only called from the tools branch
|
|
133
|
+
const sysMsg = messages.find((m) => m.role === 'system');
|
|
134
|
+
const systemPrompt = buildToolSystemPrompt(sysMsg && typeof sysMsg.content === 'string' ? sysMsg.content : null, tools);
|
|
135
|
+
const stdout = await this._spawn(renderTranscript(messages), proto.turnArgs(systemPrompt));
|
|
136
|
+
const parsed = proto.parseResult(stdout);
|
|
137
|
+
|
|
138
|
+
/** @type {GenerateResult} */
|
|
139
|
+
const result = { text: '', toolCalls: [], usage: parsed.usage, model: parsed.model ?? null };
|
|
140
|
+
if (Number.isFinite(parsed.costUsd)) result.costUsd = parsed.costUsd;
|
|
141
|
+
if (parsed.action === 'tool_call') {
|
|
142
|
+
this._toolCallSeq = (this._toolCallSeq || 0) + 1;
|
|
143
|
+
result.toolCalls = [{ id: `cli_${this._toolCallSeq}`, name: parsed.toolName || '', arguments: parsed.toolArguments || {} }];
|
|
144
|
+
} else {
|
|
145
|
+
result.text = parsed.answer || '';
|
|
146
|
+
}
|
|
147
|
+
return result;
|
|
148
|
+
}
|
|
149
|
+
|
|
150
|
+
/**
|
|
151
|
+
* Run the upfront capability probe ONCE per instance (cached), unless `probeCapability` is off.
|
|
152
|
+
* A model that answers the probe in prose instead of emitting a tool_call throws a loud
|
|
153
|
+
* `ProviderError` — fail fast, never silently degrade to a no-tools run mid-conversation.
|
|
154
|
+
* NOTE: the probe is a single internal CLI turn whose token usage/cost is NOT surfaced to the Loop
|
|
155
|
+
* (it never flows to `onLlmResult`), so a wired budget gate does not see it — negligible for the
|
|
156
|
+
* subscription use case this exists for (flat cost, one probe per instance), by design.
|
|
157
|
+
* @returns {Promise<void>}
|
|
158
|
+
*/
|
|
159
|
+
_ensureToolCapability() {
|
|
160
|
+
if (!this.probeCapability) return Promise.resolve();
|
|
161
|
+
// Cache the in-flight promise so concurrent first-calls share one probe, and a resolved
|
|
162
|
+
// capable verdict is never re-probed.
|
|
163
|
+
if (this._toolCapability) return this._toolCapability;
|
|
164
|
+
const proto = this.toolProtocol;
|
|
165
|
+
if (!proto) return Promise.resolve(); // unreachable: only called from the tools branch
|
|
166
|
+
this._toolCapability = (async () => {
|
|
167
|
+
const stdout = await this._spawn(proto.probe.user, proto.turnArgs(proto.probe.system));
|
|
168
|
+
let parsed;
|
|
169
|
+
try {
|
|
170
|
+
parsed = proto.parseResult(stdout);
|
|
171
|
+
} catch (err) {
|
|
172
|
+
// A malformed probe response is itself an incapability signal — name it as such.
|
|
173
|
+
this._toolCapability = null; // allow a retry on a transient parse failure
|
|
174
|
+
throw new ProviderError(`[CLIPipeProvider] tool-mode capability probe failed to parse: ${/** @type {Error} */ (err).message}`, /** @type {any} */ ({ status: 0 }));
|
|
175
|
+
}
|
|
176
|
+
if (!proto.probe.isCapable(parsed)) {
|
|
177
|
+
const model = this._modelFromArgs();
|
|
178
|
+
throw new ProviderError(
|
|
179
|
+
`[CLIPipeProvider] the CLI model${model ? ` '${model}'` : ''} is not capable of tool use: the ` +
|
|
180
|
+
'capability probe answered in prose instead of emitting a tool_call. Weak models (e.g. haiku) ' +
|
|
181
|
+
'cannot drive tool emulation reliably — use a stronger model for tool mode, or run without ' +
|
|
182
|
+
'tools for plain-text. (Set { probeCapability: false } to skip this check.)',
|
|
183
|
+
/** @type {any} */ ({ status: 0 }),
|
|
184
|
+
);
|
|
185
|
+
}
|
|
186
|
+
})();
|
|
187
|
+
return this._toolCapability;
|
|
188
|
+
}
|
|
189
|
+
|
|
190
|
+
/** Best-effort model id from `--model X` in the base args, for a clearer probe-failure message. */
|
|
191
|
+
_modelFromArgs() {
|
|
192
|
+
const i = this.args.indexOf('--model');
|
|
193
|
+
return i >= 0 && i + 1 < this.args.length ? this.args[i + 1] : null;
|
|
194
|
+
}
|
|
195
|
+
|
|
88
196
|
/**
|
|
89
197
|
* Map the `claude -p --output-format json` result envelope onto a normalized GenerateResult.
|
|
90
198
|
* The caller explicitly opted into structured output, so a malformed or error envelope is a LOUD
|
|
@@ -109,21 +217,7 @@ class CLIPipeProvider {
|
|
|
109
217
|
throw new ProviderError(`[CLIPipeProvider] claude CLI reported failure (subtype='${obj.subtype}'): ${detail}`, /** @type {any} */ ({ status: 0 }));
|
|
110
218
|
}
|
|
111
219
|
|
|
112
|
-
const
|
|
113
|
-
/** @type {import('../types').Usage} */
|
|
114
|
-
const usage = {
|
|
115
|
-
inputTokens: Number(u.input_tokens) || 0,
|
|
116
|
-
outputTokens: Number(u.output_tokens) || 0,
|
|
117
|
-
};
|
|
118
|
-
// Absent cache tiers mean the model didn't cache — omit rather than emit a synthetic 0 (per Usage docs).
|
|
119
|
-
if (Number.isFinite(u.cache_read_input_tokens)) usage.cacheReadTokens = u.cache_read_input_tokens;
|
|
120
|
-
if (Number.isFinite(u.cache_creation_input_tokens)) usage.cacheCreationTokens = u.cache_creation_input_tokens;
|
|
121
|
-
|
|
122
|
-
// `modelUsage` is an object keyed by model id (e.g. {"claude-opus-4-8[1m]": {...}}) — take the first key.
|
|
123
|
-
const model = (obj.modelUsage && typeof obj.modelUsage === 'object')
|
|
124
|
-
? (Object.keys(obj.modelUsage)[0] ?? null)
|
|
125
|
-
: null;
|
|
126
|
-
|
|
220
|
+
const { usage, model, costUsd } = mapClaudeMeta(obj);
|
|
127
221
|
/** @type {GenerateResult} */
|
|
128
222
|
const result = {
|
|
129
223
|
text: typeof obj.result === 'string' ? obj.result : '',
|
|
@@ -131,9 +225,7 @@ class CLIPipeProvider {
|
|
|
131
225
|
usage,
|
|
132
226
|
model,
|
|
133
227
|
};
|
|
134
|
-
|
|
135
|
-
// marginal) — feeds bareguard's USD axis with no local rate table. Only a finite number counts.
|
|
136
|
-
if (Number.isFinite(obj.total_cost_usd)) result.costUsd = obj.total_cost_usd;
|
|
228
|
+
if (costUsd !== undefined) result.costUsd = costUsd;
|
|
137
229
|
return result;
|
|
138
230
|
}
|
|
139
231
|
|
package/src/recurse.d.ts
CHANGED
|
@@ -113,7 +113,7 @@ export type RecurseOptions = {
|
|
|
113
113
|
* (+ `receipts.blockerDetail`), with `best` preserving the model's last attempt. A `HaltError` thrown by the
|
|
114
114
|
* sensor stays a clean governance halt. The sensor's EXECUTION environment is the caller's: run untrusted /
|
|
115
115
|
* model-generated checks in an isolated child process WITH A TIMEOUT — a sensor that hangs forever hangs the
|
|
116
|
-
* leaf (no bareguard checkpoint fires between sensor start and return).
|
|
116
|
+
* leaf (no bareguard checkpoint fires between sensor start and return; confirmed by `poc/rlmplans-hung-sensor.mjs`).
|
|
117
117
|
* **`rejectedBuffer` (BA-14):** a SkillOpt-shaped rejected-attempt buffer — instead of only the LATEST critique,
|
|
118
118
|
* surface the model's OWN prior failed attempts verbatim ("you wrote these, they failed X — write something
|
|
119
119
|
* STRUCTURALLY DIFFERENT"). This is DIRECTED diversity (attack the specific repeated mistake), where escalation
|
|
@@ -456,7 +456,7 @@ export type Slice = {
|
|
|
456
456
|
* (+ `receipts.blockerDetail`), with `best` preserving the model's last attempt. A `HaltError` thrown by the
|
|
457
457
|
* sensor stays a clean governance halt. The sensor's EXECUTION environment is the caller's: run untrusted /
|
|
458
458
|
* model-generated checks in an isolated child process WITH A TIMEOUT — a sensor that hangs forever hangs the
|
|
459
|
-
* leaf (no bareguard checkpoint fires between sensor start and return).
|
|
459
|
+
* leaf (no bareguard checkpoint fires between sensor start and return; confirmed by `poc/rlmplans-hung-sensor.mjs`).
|
|
460
460
|
* **`rejectedBuffer` (BA-14):** a SkillOpt-shaped rejected-attempt buffer — instead of only the LATEST critique,
|
|
461
461
|
* surface the model's OWN prior failed attempts verbatim ("you wrote these, they failed X — write something
|
|
462
462
|
* STRUCTURALLY DIFFERENT"). This is DIRECTED diversity (attack the specific repeated mistake), where escalation
|
package/src/recurse.js
CHANGED
|
@@ -244,7 +244,7 @@ function auditSafeCtx(ctx, overrides = {}) {
|
|
|
244
244
|
* (+ `receipts.blockerDetail`), with `best` preserving the model's last attempt. A `HaltError` thrown by the
|
|
245
245
|
* sensor stays a clean governance halt. The sensor's EXECUTION environment is the caller's: run untrusted /
|
|
246
246
|
* model-generated checks in an isolated child process WITH A TIMEOUT — a sensor that hangs forever hangs the
|
|
247
|
-
* leaf (no bareguard checkpoint fires between sensor start and return).
|
|
247
|
+
* leaf (no bareguard checkpoint fires between sensor start and return; confirmed by `poc/rlmplans-hung-sensor.mjs`).
|
|
248
248
|
* **`rejectedBuffer` (BA-14):** a SkillOpt-shaped rejected-attempt buffer — instead of only the LATEST critique,
|
|
249
249
|
* surface the model's OWN prior failed attempts verbatim ("you wrote these, they failed X — write something
|
|
250
250
|
* STRUCTURALLY DIFFERENT"). This is DIRECTED diversity (attack the specific repeated mistake), where escalation
|