@ljoukov/llm 3.0.4 → 3.0.8

package/README.md

@@ -11,6 +11,7 @@ Unified TypeScript wrapper over:
 - **Google Gemini via Vertex AI** (`@google/genai`)
 - **Fireworks chat-completions models** (`kimi-k2.5`, `glm-5`, `minimax-m2.1`, `gpt-oss-120b`)
 - **ChatGPT subscription models** via `chatgpt-*` model ids (reuses Codex auth store, or a token provider)
+- **Agentic orchestration with subagents** via `runAgentLoop()` + built-in delegation control tools
 
 Designed around a single streaming API that yields:
 
@@ -109,21 +110,40 @@ SSE for the rest of the process to avoid repeated failing upgrade attempts.
 
 ### Adaptive per-model concurrency
 
-Provider calls use adaptive, overload-aware concurrency (with retry/backoff where supported). Configure hard caps per
-model/per binary with env vars (clamped to `1..64`, default `3`):
+Provider calls use adaptive, overload-aware concurrency (with retry/backoff where supported). Configure hard caps in
+code (clamped to `1..64`):
 
-- global cap: `LLM_MAX_PARALLEL_REQUESTS_PER_MODEL`
-- provider caps: `OPENAI_MAX_PARALLEL_REQUESTS_PER_MODEL`, `GOOGLE_MAX_PARALLEL_REQUESTS_PER_MODEL`,
-  `FIREWORKS_MAX_PARALLEL_REQUESTS_PER_MODEL`
-- model overrides:
-  - `LLM_MAX_PARALLEL_REQUESTS_MODEL_<MODEL>`
-  - `<PROVIDER>_MAX_PARALLEL_REQUESTS_MODEL_<MODEL>`
+```ts
+import { configureModelConcurrency } from "@ljoukov/llm";
+
+configureModelConcurrency({
+  globalCap: 8,
+  providerCaps: {
+    openai: 16,
+    google: 3,
+    fireworks: 8,
+  },
+  modelCaps: {
+    "gpt-5.2": 24,
+  },
+  providerModelCaps: {
+    google: {
+      "gemini-3.1-pro-preview": 2,
+    },
+  },
+});
+```
 
-`<MODEL>` is uppercased and non-alphanumeric characters become `_` (for example `gpt-5.2` -> `GPT_5_2`).
+Default caps (without configuration):
+
+- OpenAI: `12`
+- Google preview models (`*preview*`): `2`
+- Other Google models: `4`
+- Fireworks: `6`
 
 ## Usage
 
-`v2` uses OpenAI-style request fields:
+Use OpenAI-style request fields:
 
 - `input`: string or message array
 - `instructions`: optional top-level system instructions
@@ -326,8 +346,8 @@ console.log(result.text);
 
 - OpenAI API models use structured outputs (`json_schema`) when possible.
 - Gemini uses `responseJsonSchema`.
-- `chatgpt-*` models try to use structured outputs too; if rejected by the endpoint/model, it falls back to best-effort
-  JSON parsing.
+- `chatgpt-*` models try to use structured outputs too; if the endpoint/account/model rejects `json_schema`, the call
+  retries with best-effort JSON parsing.
 
 ```ts
 import { generateJson } from "@ljoukov/llm";
@@ -412,12 +432,12 @@ There are three tool-enabled call patterns:
 
 1. `generateText()` for provider-native/server-side tools (for example web search).
 2. `runToolLoop()` for your runtime JS/TS tools (function tools executed in your process).
-3. `runAgentLoop()` for filesystem tasks (a convenience wrapper around `runToolLoop()`).
+3. `runAgentLoop()` for full agentic loops (a convenience wrapper around `runToolLoop()` with built-in subagent orchestration and optional filesystem tools).
 
 Architecture note:
 
-- Filesystem tools are not a separate execution system.
-- `runAgentLoop()` constructs a filesystem toolset, merges your optional custom tools, then calls the same `runToolLoop()` engine.
+- Built-in filesystem tools are not a separate execution system.
+- `runAgentLoop()` can construct a filesystem toolset, merges your optional custom tools, and calls the same `runToolLoop()` engine.
 - This behavior is model-agnostic at API level; profile selection only adapts tool shape for model compatibility.
 
 ### Provider-Native Tools (`generateText()`)
@@ -461,9 +481,78 @@ console.log(result.text);
 
 Use `customTool()` only when you need freeform/non-JSON tool input grammar.
 
-### Filesystem Tasks (`runAgentLoop()`)
+### Mid-Run Steering (Queued Input)
+
+You can queue user steering while a tool loop is already running. Steering is applied on the next model step (it does
+not interrupt the current generation/tool execution).
+
+```ts
+import { streamToolLoop, tool } from "@ljoukov/llm";
+import { z } from "zod";
+
+const call = streamToolLoop({
+  model: "chatgpt-gpt-5.3-codex",
+  input: "Start implementing the feature.",
+  tools: {
+    echo: tool({
+      inputSchema: z.object({ text: z.string() }),
+      execute: ({ text }) => ({ text }),
+    }),
+  },
+});
 
-Use this for read/search/write tasks in a workspace. The library auto-selects filesystem tool profile by model when `profile: "auto"`:
+// Append steering while run is active.
+call.append("Focus on tests first, then refactor.");
+
+const result = await call.result;
+console.log(result.text);
+```
+
+If you already manage your own run lifecycle, you can create and pass a steering channel directly:
+
+```ts
+import { createToolLoopSteeringChannel, runAgentLoop } from "@ljoukov/llm";
+
+const steering = createToolLoopSteeringChannel();
+const run = runAgentLoop({
+  model: "chatgpt-gpt-5.3-codex",
+  input: "Implement the task.",
+  filesystemTool: true,
+  steering,
+});
+
+steering.append("Do not interrupt; apply this guidance on the next turn.");
+const result = await run;
+```
+
+### Agentic Loop (`runAgentLoop()`)
+
+`runAgentLoop()` is the high-level agentic API. It supports:
+
+- optional filesystem workspace tools,
+- built-in subagent orchestration (delegate work across spawned agents),
+- your own custom runtime tools.
+
+For interactive runs where you want to stream events and inject steering mid-run, use `streamAgentLoop()`:
+
+```ts
+import { streamAgentLoop } from "@ljoukov/llm";
+
+const call = streamAgentLoop({
+  model: "chatgpt-gpt-5.3-codex",
+  input: "Start implementation.",
+  filesystemTool: true,
+});
+
+call.append("Prioritize a minimal diff and update tests.");
+const result = await call.result;
+console.log(result.text);
+```
+
+#### 1) Filesystem agent loop
+
+For read/search/write tasks in a workspace, enable `filesystemTool`. The library auto-selects a tool profile by model
+when `profile: "auto"`:
 
 - Codex-like models: Codex-compatible filesystem tool shape.
 - Gemini models: Gemini-compatible filesystem tool shape.
@@ -478,10 +567,55 @@ Confinement/policy is set through `filesystemTool.options`:
 
 Detailed reference: `docs/agent-filesystem-tools.md`.
 
-Subagent delegation can be enabled via `subagentTool` (Codex-style control tools):
+Filesystem-only example:
+
+```ts
+import { createInMemoryAgentFilesystem, runAgentLoop } from "@ljoukov/llm";
+
+const fs = createInMemoryAgentFilesystem({
+  "/repo/src/a.ts": "export const value = 1;\n",
+});
+
+const result = await runAgentLoop({
+  model: "chatgpt-gpt-5.3-codex",
+  input: "Change value from 1 to 2 using filesystem tools.",
+  filesystemTool: {
+    profile: "auto",
+    options: {
+      cwd: "/repo",
+      fs,
+    },
+  },
+});
+
+console.log(result.text);
+```
+
+#### 2) Add subagent orchestration
+
+Enable `subagentTool` to allow delegation via Codex-style control tools:
 
 - `spawn_agent`, `send_input`, `resume_agent`, `wait`, `close_agent`
-- Optional limits: `maxAgents`, `maxDepth`, wait timeouts.
+- optional limits: `maxAgents`, `maxDepth`, wait timeouts
+- `spawn_agent.agent_type` supports built-ins aligned with codex-rs-style roles: `default`, `researcher`, `worker`, `reviewer`
+
+```ts
+import { runAgentLoop } from "@ljoukov/llm";
+
+const result = await runAgentLoop({
+  model: "chatgpt-gpt-5.3-codex",
+  input: "Plan the work, delegate in parallel where useful, and return a final merged result.",
+  subagentTool: {
+    enabled: true,
+    maxAgents: 4,
+    maxDepth: 2,
+  },
+});
+
+console.log(result.text);
+```
+
+#### 3) Combine filesystem + subagents
 
 ```ts
 import { createInMemoryAgentFilesystem, runAgentLoop } from "@ljoukov/llm";
@@ -510,6 +644,37 @@ const result = await runAgentLoop({
 console.log(result.text);
 ```
 
+### Agent Telemetry (Pluggable Backends)
+
+`runAgentLoop()` supports optional telemetry hooks that keep default behavior unchanged.
+You can attach any backend by implementing a sink with `emit(event)` and optional `flush()`.
+
+```ts
+import { runAgentLoop } from "@ljoukov/llm";
+
+const result = await runAgentLoop({
+  model: "chatgpt-gpt-5.3-codex",
+  input: "Summarize the report and update output JSON files.",
+  filesystemTool: true,
+  telemetry: {
+    includeLlmStreamEvents: false, // enable only if you need token/delta event fan-out
+    sink: {
+      emit: (event) => {
+        // Forward to your backend (Cloud Logging, OpenTelemetry, Datadog, etc.)
+        // event.type: "agent.run.started" | "agent.run.stream" | "agent.run.completed"
+        // event carries runId, parentRunId, depth, model, timestamp + payload
+      },
+      flush: async () => {
+        // Optional: flush buffered telemetry on run completion.
+      },
+    },
+  },
+});
+```
+
+Telemetry emits parent/child run correlation (`runId` + `parentRunId`) for subagents.
+See `docs/agent-telemetry.md` for event schema, design rationale, and backend adapter guidance.
+
 If you need exact control over tool definitions, build the filesystem toolset yourself and call `runToolLoop()` directly.
 
 ```ts
@@ -554,6 +719,15 @@ npm run bench:agent:estimate
 
 See `benchmarks/agent/README.md` for options and output format.
 
+## Examples
+
+Interactive CLI chat with mid-run steering, thought streaming, filesystem tools rooted at
+the current directory, subagents enabled, and `Esc` interrupt support:
+
+```bash
+npm run example:cli-chat
+```
+
 ## License
 
 MIT