toolnexus 0.18.0 → 0.18.1

package/README.md

@@ -12,7 +12,7 @@ unified as one tool set, driving any LLM.
 > loop, hooks, streaming, retries, memory — and nothing it doesn't.
 
 The JS/TypeScript port of [toolnexus](https://github.com/muthuishere/toolnexus) — the same
-library, byte-identical, also in **Python, Go, Java, and C#**. Built on
+library, byte-identical, also in **Python, Go, Java, C#, Elixir and Clojure**. Built on
 `@modelcontextprotocol/sdk` (the MCP SDK opencode uses).
 
 ## Install
@@ -39,420 +39,27 @@ const { text } = await agent.run("What files are in this folder?", { toolkit: tk
 console.log(text)
 ```
 
-`createClient` reads the key from `OPENROUTER_API_KEY` / `OPENAI_API_KEY` / `ANTHROPIC_API_KEY`
-(or pass `apiKey`). The loop runs call → execute tools → feed back → repeat (up to `maxTurns`,
-default 10), with hooks, streaming (`agent.stream(...)`), retries/backoff, timeout, and
-conversation memory available.
-
-## With MCP + skills
-
-Point the toolkit at an `mcp.json` and a `skills/` folder — every MCP server tool and every
-`SKILL.md` becomes part of the same tool set. Skills are injected into the system prompt; a
-single `skill` tool loads each on demand (progressive disclosure).
-
-```ts
-const tk = await createToolkit({
-  mcpConfig: "./mcp.json",     // path or a config object; local stdio + remote streamable-HTTP
-  skillsDir: "./skills",       // a folder of <name>/SKILL.md (string or string[])
-})
-
-const { text } = await agent.run("Refund order 1234 for the customer.", { toolkit: tk })
-console.log(text)
-await tk.close()               // disconnect MCP servers
-```
-
-## Suspension — login / approval / input (`Pending` / `waitFor`)
-
-Some tools can't finish in one shot: they need an **out-of-band, async resolution** — a human
-logs in, approves a trade, types a value. That's one idea, not one feature: *login* is just the
-`kind:"authorization"` case. A tool returns **`Pending(request)`**; the client calls your
-**`waitFor(request) → answer`**, then **retries the tool** with `ctx.answer`, and the model gets
-the real answer. Everything crossing a boundary is plain data, so it works the same across
-processes, agents, and restarts — and identically in all five language ports.
-
-```ts
-import { authRequired } from "toolnexus"
-
-// a tool that needs login the first time:
-defineTool({
-  name: "get_balance", description: "…", inputSchema: { type: "object", properties: {} },
-  run: (_args, ctx) =>
-    authed ? `balance: ₹67,417` : authRequired("https://example.com/login", "Log in to continue"),
-})
-
-// the host supplies ONE function — how the link reaches the human is your call:
-const agent = createClient({
-  baseUrl, style: "openai", model,
-  waitFor: async (request) => {          // interactive: open a browser + poll
-    open(request.url); await pollUntilLoggedIn(); return { id: request.id, ok: true }
-  },
-  // headless desk: waitFor = send request.url to your channel + poll
-  // another agent: waitFor = forward the request over A2A, await its answer
-})
-```
-
-Omit `waitFor` for a **durable** host: `run()` doesn't hang — it returns
-`{ status: "pending", pending: request }`, so you can deliver the link out-of-band (file, HTTP,
-channel) and resume later, even in another process. `stream()` emits a `{ type: "pending", request }`
-event so a channel can push the link in real time. Runnable demo: `examples/pending.ts`.
-
-## Conversations / memory
-
-`run()` is stateless — each call starts fresh. To keep a thread's history, use `ask()` with an
-`id`, or a `conversation()` object.
-
-**`ask(prompt, { toolkit, id? })`** — with an `id`, the client's `ConversationStore` remembers
-the thread: it *loads* that id's transcript, *runs*, *saves* the updated transcript, and returns
-the answer. The next `ask` with the same `id` continues it. Without an `id` it's a stateless
-one-shot (identical to `run`).
-
-```ts
-const agent = createClient({
-  baseUrl: "https://openrouter.ai/api/v1",
-  style: "openai",
-  model: "openai/gpt-4o-mini",
-  // store defaults to InMemoryConversationStore (process lifetime).
-})
-
-await agent.ask("My name is Muthu.", { toolkit: tk, id: "user-42" })
-const { text } = await agent.ask("What's my name?", { toolkit: tk, id: "user-42" })
-console.log(text)   // → "Your name is Muthu."
-```
-
-**Persist across processes.** A `ConversationStore` is two methods — implement it for a file, DB,
-or Redis and pass it as `store`:
-
-```ts
-import type { ConversationStore } from "toolnexus"
-
-interface ConversationStore {
-  get(id: string): Promise<any[] | undefined>   // stored transcript for id, or undefined
-  save(id: string, messages: any[]): Promise<void>
-}
-
-const agent = createClient({ baseUrl, style: "openai", model, store: myFileStore })
-```
-
-The default is `InMemoryConversationStore` (also exported). For a quick in-process thread without
-ids, `agent.conversation({ toolkit })` returns an object whose `.send(prompt)` retains history
-automatically (`.messages`, `.reset()`).
-
-The same `store` powers inbound A2A: `toolkit.serve` keys each peer's turns by the A2A
-`contextId` through `client.ask`, so a remote agent's conversation is remembered too (see below).
-
-**Streaming with memory.** The `id` also works while streaming. Pass `on_text` to `ask` to stream
-text deltas as they arrive — `ask` still returns the final `RunResult` — or iterate `stream()`
-directly. With an `id`, the thread is loaded before the stream and saved on the `done` event.
-
-```ts
-// block-style: stream deltas to stdout, still get the RunResult back — remembered under `id`
-const res = await agent.ask("Draft a reply.", {
-  toolkit: tk,
-  id: "user-42",
-  on_text: (delta) => process.stdout.write(delta),
-})
-
-// iterator: consume text + tool events; `id` makes it stateful (load before, save on `done`)
-for await (const ev of agent.stream("And summarise it.", { toolkit: tk, id: "user-42" })) {
-  if (ev.type === "text") process.stdout.write(ev.delta)
-  else if (ev.type === "done") console.log("\n", ev.result.usage)
-}
-```
-
-## Observability / metrics
-
-Zero-dependency, two outputs from one internal instrumentation — both opt-in, no cost when unused.
-
-**`onMetric` — a semantic event feed.** Pass it to `createClient` and it receives a readable
-`MetricEvent` at each significant point: `{ event: "llm", model, status, ms, promptTokens,
-completionTokens }` per model call, `{ event: "tool", tool, source, isError, ms }` per tool call,
-and a terminal `{ event: "run", model, turns, toolCalls, totalTokens, ms, error? }` per `run`/`ask`.
-Forward it anywhere (statsd, logs, OpenTelemetry) — the library holds no opinion.
-
-```ts
-const agent = createClient({
-  baseUrl, style: "openai", model,
-  onMetric: (ev) => console.log("[metric]", ev.event, JSON.stringify(ev)),
-})
-```
-
-**`client.metrics()` — built-in Prometheus text.** The same events feed a tiny in-memory registry
-that renders the Prometheus text exposition format (no third-party dep). Mount it at `GET /metrics`:
-
-```ts
-import { createServer } from "node:http"
-
-createServer((req, res) => {
-  if (req.url === "/metrics") {
-    res.setHeader("Content-Type", "text/plain; version=0.0.4")
-    res.end(agent.metrics())
-  } else {
-    res.statusCode = 404
-    res.end()
-  }
-}).listen(9090)
-```
-
-Series: `toolnexus_llm_requests_total{model,status}`, `toolnexus_llm_tokens_total{type}`,
-`toolnexus_tool_calls_total{tool,source,is_error}`, `toolnexus_run_errors_total{model}`, plus the
-`toolnexus_llm_request_duration_seconds` and `toolnexus_tool_duration_seconds` histograms. The
-rendered text is byte-identical across all five ports; OTLP push is a planned future companion.
-
-## Add your own tools
-
-```ts
-import { defineTool, httpTool } from "toolnexus"
-
-tk.register(
-  // a plain function → a tool  (run returns a string or a full ToolResult)
-  defineTool({
-    name: "add",
-    description: "Add two numbers",
-    inputSchema: {
-      type: "object",
-      properties: { a: { type: "number" }, b: { type: "number" } },
-      required: ["a", "b"],
-    },
-    run: (args) => `${(args.a as number) + (args.b as number)}`,
-  }),
-  // a REST endpoint → a tool
-  httpTool({
-    name: "create_ticket",
-    description: "Create a ticket",
-    method: "POST",
-    url: "https://api.example.com/tickets",
-    headers: { Authorization: "Bearer ${API_TOKEN}" },   // ${ENV} expands from process.env, never logged
-    inputSchema: {
-      type: "object",
-      properties: { title: { type: "string" } },
-      required: ["title"],
-    },
-  }),
-)
-```
-
-For `httpTool`: URL `{placeholders}` are filled from args; `query` arg names go to the
-querystring (all args on GET); the rest become the JSON body (`body: "json" | "form" | "raw"`).
-Non-2xx → `{ isError: true, output: "HTTP <status>: <body>" }`.
-
-## Images, PDFs and audio (content parts)
-
-Parts go in the **first** argument, next to your text, because the order of text and image is
-semantic to a model:
-
-```ts
-import { attach, text } from "toolnexus"
-
-const { text: answer } = await agent.run(
-  ["What is broken in this screenshot?", await attach("./shot.png")],
-  { toolkit: tk },
-)
-```
-
-`attach(source, opts?)` takes what you already hold: a filesystem path, a `data:` URL, an
-`https:` URL, native bytes (`Uint8Array`, `Buffer`, `ArrayBuffer`, any `ArrayBufferView` —
-`mimeType` required, bytes carry no extension), or a `Blob`/`File` straight off an upload.
-`fromPath`, `fromBytes`, `fromBlob`, `fromDataUrl`, `fromUrl` are the same thing spelled
-explicitly. Mime types come from a fixed extension table (`png jpg jpeg gif webp pdf mp3 wav`) —
-never sniffed, never read from the machine's mime database, so every port agrees. A plain string
-prompt is untouched and byte-identical to before.
-
-A tool can answer with media too — `parts` alongside the still-required `output`, which is what
-the transcript, compaction and any text-only provider keep seeing:
-
-```ts
-defineTool({
-  name: "screenshot",
-  description: "Grab the screen",
-  inputSchema: { type: "object", properties: {} },
-  run: async () => ({
-    output: "screenshot, 1280x720 png",
-    parts: [await attach("./shot.png")],
-  }),
-})
-```
-
-**MCP tools' images now survive.** Non-text content from an MCP server — image, audio, embedded
-resource, `resource_link` — used to be filtered out silently; it is preserved as `parts` and
-emitted per provider style (Anthropic inside `tool_result`; OpenAI, whose `tool` message rejects
-an image, in a synthetic user message that never enters your transcript).
-
-Whatever you pass in, a part stores **bytes plus a `mimeType`** — never a handle, never a path —
-so a persisted transcript replays without the original file. A `Blob` is read **eagerly** at
-construction, and a handle you supply is read, not closed.
-
-## Built-in tools
-
-A fifth source ships **10 built-in tools** — `bash`, `read`, `write`, `edit`, `grep`, `glob`,
-`webfetch`, `question`, `apply_patch`, `todowrite` (names + input schemas match opencode) — so an
-agent can act with zero wiring. They appear in the tool schema
-(`toOpenAI()`/`toAnthropic()`/`toGemini()`), like MCP tools — not the system prompt.
-
-**On by default.** One global toggle turns the whole source off, or a per-tool `tools` map
-disables individual builtins on the all-on baseline:
-
-```ts
-const tk = await createToolkit({ builtins: false })
-// also accepts { disabled: true } or { enabled: false }
-
-// per-tool: drop bash, keep the other nine (unknown names ignored; whole-source-off still wins)
-const tk2 = await createToolkit({ builtins: { tools: { bash: false } } })
-```
-
-`bash`/`write`/`edit`/`apply_patch` run commands and mutate the filesystem — the toggle is the
-off-switch for locked-down hosts.
-
-## A2A agents (agent-to-agent)
-
-Call **remote A2A agents** (each of their skills becomes a tool) and serve your own toolkit as an
-agent other A2A peers can call. A genuine, minimal subset of real A2A (JSON-RPC 2.0; Agent Card at
-`/.well-known/agent-card.json`; `SendMessage` → poll `GetTask`). No streaming / push / auth in v1.
-
-**Outbound — call a remote agent.** Each advertised skill becomes a tool named `<agent>_<skill>`
-(`source: "a2a"`):
-
-```ts
-import { createToolkit, agent } from "toolnexus"
-
-const tk = await createToolkit({
-  agents: [agent({ card: "https://researcher.example.com/.well-known/agent-card.json" })],
-})
-
-// or add one at runtime (an Agent descriptor or a bare card URL):
-await tk.addAgent("https://writer.example.com/.well-known/agent-card.json")
-```
-
-`agent({ card, headers?, timeout?, pollEvery? })` — `headers` support `${ENV}` expansion (never
-logged); `timeout` defaults to 300000ms, `pollEvery` to 1000ms. A config object can also carry an
-`agents` block (mirrors `mcpServers`). A failing agent is isolated — logged, contributes no tools,
-never fatal.
-
-**Inbound — serve your toolkit as an agent.** Opt in with the `a2a` profile; the Agent Card is
-built from your **SKILL.md skills** (never raw tools):
-
-```ts
-const handle = await tk.serve("127.0.0.1:0", {
-  client: agent,                  // a createClient(...) instance fulfils each task
-  a2a: {
-    name: "research-agent",
-    description: "Answers research questions.",
-    // skills: ["hello-world"],   // subset of skills to advertise; omit ⇒ all
-    store: "memory",              // "memory" (default) | "file:<dir>" | a custom TaskStore
-  },
-})
-console.log(handle.url)   // GET /.well-known/agent-card.json ; POST / (SendMessage / GetTask)
-await handle.stop()
-```
-
-`serve(addr, { client, a2a?, onTask? })` fulfils each inbound `SendMessage` task via the client.
-Task persistence is a pluggable `TaskStore` — in-memory default, `"file:<dir>"`, or your own. And
-it **remembers**: a message's A2A `contextId` keys the conversation through `client.ask`, so a
-peer's turns in the same context carry history via the client's `ConversationStore`.
-
-## Serve as an MCP server (be a gateway)
-
-The inbound mirror of A2A: expose your **whole toolkit as an MCP server** so any MCP client — an IDE,
-another agent, a remote host — can call its tools. Point toolnexus at N MCP servers + skills + your
-own functions, then re-expose the union as **one** MCP server. Unlike A2A (which advertises skills and
-runs the client loop), the MCP client *is* the LLM host, so each `tools/call` dispatches straight to
-`Tool.execute` — no `client`, no tasks, no store.
+Add real tool sources by pointing at them:
 
 ```ts
-// streamable-HTTP — an embeddable MCP server mounted at POST /mcp, beside any A2A routes:
-const srv = await tk.serve("127.0.0.1:0", {
-  mcp: { name: "my-gateway" /*, tools: ["echo"]  // subset; omit ⇒ every toolkit tool */ },
-  onCall: (ev) => console.error(ev.name, ev.ms, ev.isError),
-})
-console.log(srv.url + "/mcp")   // connect any MCP client here
-await srv.stop()
+const tk = await createToolkit({ mcp: "mcp.json", skills: ["skills"] })
 ```
 
-`tools/list` advertises every toolkit tool (name **verbatim**, `inputSchema` = the tool's parameters);
-`mcp.tools` narrows the surface. The `mcp` profile can also live in the config file as a top-level
-**`mcpServer`** block (singular — distinct from the client-side `mcpServers`). (Transport is
-streamable-HTTP; a stdio transport for local clients like Claude Desktop is a planned follow-up.)
-
-## Bring your own loop
-
-Don't want the host loop? Use the schema adapters and execute calls yourself:
-
-```ts
-const tools  = tk.toOpenAI()       // or tk.toAnthropic() / tk.toGemini()
-const system = tk.skillsPrompt()   // skills catalog for your system prompt
-// when the model returns a tool call { name, arguments }:
-const res = await tk.execute(name, args)   // → { output, isError, metadata }
-```
-
-## The four sources
-
-| Source | How |
-|--------|-----|
-| **MCP servers** | an `mcp.json` (`mcpServers`/`servers`/`mcp`); local stdio + remote streamable-HTTP, `headers` for auth |
-| **Agent skills** | a folder of `<name>/SKILL.md`; a `skill` tool loads each on demand + a system-prompt catalog |
-| **Native tools** | `defineTool({...})` — a function becomes a tool |
-| **HTTP / REST** | `httpTool({...})` — an endpoint becomes a tool, `${ENV}` headers |
-
-All appear as one uniform `Tool` in `tk.tools()`. Built-ins are a fifth source; remote A2A agents
-a sixth; in-process **sub-agents** a seventh (below).
-
-## Sub-agents & teams
-
-An **Agent is a Tool**: a system prompt × a scoped toolkit view × the client loop. One agent
-delegates to another **in-process** via one `task` tool — isolated context, one result back,
-tokens rolled up, hierarchical budgets, durable suspension (`SPEC.md §7D`). The surface lives in
-the **`agents`** namespace (never the A2A `agent` above):
-
-```ts
-import { agents } from "toolnexus"
-
-const explore = agents.agent("explore", {
-  does: "read-only research",
-  uses: { tools: [lookup] },              // this child sees ONLY these tools
-})
-const coder = agents.agent("coder", {
-  does: "implements changes",
-  soulFile: "./AGENTS.md",                // identity file → system prompt
-  team: [explore],                        // team = the task tool's only targets; no team ⇒ no task tool
-  budget: { maxTokens: 10_000 },
-})
-
-const r = await coder.run("fix the failing test", {
-  llm: { baseUrl: "https://openrouter.ai/api/v1", style: "openai", model: "openai/gpt-4o-mini" },
-})
-console.log(r.status, r.text, r.totalTokens)   // "done" | "incomplete" | "pending", final text, tree-wide tokens
-```
-
-`agents.asTool(...)` drops an agent back into `extraTools` for the classic API. Full guide:
-[Sub-agents & teams](https://muthuishere.github.io/toolnexus/subagents/).
-
-## API
+- `mcp.json` is the standard Claude-desktop-style config (`mcpServers` / `servers` / `mcp` keys all accepted).
+- `skills/` is a folder of `**/SKILL.md` files, loaded on demand through one `skill` tool.
+- Remote MCP `headers` values expand `${ENV_VAR}` at call time and are never logged.
 
-| Member | Description |
-|--------|-------------|
-| `createToolkit(opts)` | async factory → `Toolkit` (`mcpConfig`, `skillsDir`, `builtins`, `agents`, `extraTools`) |
-| `createClient(opts)` | the unified host loop (`baseUrl`, `style`, `model`, `store?`, `hooks?`, `maxTurns?`, `retries?`, `timeoutMs?`, `onMetric?`) |
-| `client.run(prompt, { toolkit, signal?, history? })` | stateless run → `RunResult` |
-| `client.ask(prompt, { toolkit, id?, on_text?, signal? })` | stateful with `id` (loads → runs → saves via `store`); one-shot without `id`; `on_text` streams text deltas |
-| `client.conversation({ toolkit })` | in-process multi-turn object — `.send(prompt)`, `.messages`, `.reset()` |
-| `client.stream(prompt, { toolkit, id? })` | async-iterate `StreamEvent`s (text deltas, tool calls/results, usage, done); `id` ⇒ stateful |
-| `client.metrics()` | Prometheus text exposition of cumulative metrics — mount at `GET /metrics` |
-| `ConversationStore` | `get(id)` / `save(id, messages)` — implement for file/db; default `InMemoryConversationStore` |
-| `tk.tools()` / `tk.get(name)` | the uniform tools |
-| `tk.register(...tools)` | add native/http/custom tools at runtime |
-| `tk.execute(name, args, ctx?)` | run a tool → `ToolResult` |
-| `tk.skillsPrompt()` | system-prompt skill catalog |
-| `tk.mcpStatus()` | per-server connection status |
-| `tk.toOpenAI()` / `toAnthropic()` / `toGemini()` | provider tool schemas |
-| `tk.serve(addr, { client?, a2a?, onTask?, mcp?, onCall? })` | serve the toolkit over HTTP — A2A agent and/or streamable-HTTP MCP server (`/mcp`) → `ServeHandle` |
-| `tk.addAgent(cardUrl \| Agent)` | fetch a remote agent's card and register its skills as tools |
-| `tk.close()` | disconnect MCP servers |
-| `defineTool` / `httpTool` / `agent` | build native / HTTP / A2A tools |
-| `InMemoryConversationStore` · `InMemoryTaskStore` / `FileTaskStore` | default stores (memory / A2A tasks) |
+## Documentation
 
-## More
+Everything else — the full surface, with runnable examples — lives on the docs site:
 
-Full docs, the other four language ports, and the shared cross-language behavior contract
-([`SPEC.md`](https://github.com/muthuishere/toolnexus/blob/main/SPEC.md)) — plus runnable
-`examples/`: **https://github.com/muthuishere/toolnexus**
+| | |
+|---|---|
+| **Start here** | [Quickstart](https://muthuishere.github.io/toolnexus/quickstart/) · [Concepts](https://muthuishere.github.io/toolnexus/concepts/) · [Install](https://muthuishere.github.io/toolnexus/install/) |
+| **Tool sources** | [MCP](https://muthuishere.github.io/toolnexus/mcp/) · [Skills](https://muthuishere.github.io/toolnexus/skills/) · [Native](https://muthuishere.github.io/toolnexus/native/) · [HTTP](https://muthuishere.github.io/toolnexus/http/) · [Built-ins](https://muthuishere.github.io/toolnexus/builtins/) · [A2A](https://muthuishere.github.io/toolnexus/a2a/) |
+| **The loop** | [Streaming](https://muthuishere.github.io/toolnexus/streaming/) · [Memory](https://muthuishere.github.io/toolnexus/memory/) · [Suspension](https://muthuishere.github.io/toolnexus/suspension/) · [Resilience](https://muthuishere.github.io/toolnexus/resilience/) · [Observability](https://muthuishere.github.io/toolnexus/observability/) |
+| **Agents** | [Sub-agents & teams](https://muthuishere.github.io/toolnexus/subagents/) · [Personas](https://muthuishere.github.io/toolnexus/persona-agents/) · [Typed decisions](https://muthuishere.github.io/toolnexus/judge/) |
+| **API reference** | [JavaScript](https://muthuishere.github.io/toolnexus/api/javascript/) |
+| **Cookbook** | [Zero to agent](https://muthuishere.github.io/toolnexus/cookbook/zero-to-agent/) · [MCP servers](https://muthuishere.github.io/toolnexus/cookbook/mcp-servers/) · [Agent skills](https://muthuishere.github.io/toolnexus/cookbook/agent-skills/) · [Judge](https://muthuishere.github.io/toolnexus/cookbook/judge/) |
 
-MIT licensed.
+Contract across all seven ports: [`SPEC.md`](https://github.com/muthuishere/toolnexus/blob/main/SPEC.md).

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "toolnexus",
-  "version": "0.18.0",
+  "version": "0.18.1",
   "description": "Dynamic MCP servers + agent skills as uniform tools for any LLM (opencode-style).",
   "type": "module",
   "main": "./dist/index.js",