toolgz 0.2.8 → 0.2.10

package/README.md

@@ -2,7 +2,7 @@
 
 <p><strong>Your agent spends 30–70k tokens of context on tool definitions before the user types a word. toolgz gets up to ~85% of it back.</strong></p>
 
-<p><em>On a large tool set, at level 3. The safe default (level 1) reclaims 13–39% and gives up nothing — <code>recommendLevel()</code> picks for you.</em></p>
+<p><em>On a large tool set, at level 3. The default (level 1) reclaims 13–39% and gives up nothing. <code>recommendLevel()</code> tells you which one your tools want; you pass its answer in.</em></p>
 
 <p>
 <a href="#measured-results">420-run cross-provider sweep</a> ·
@@ -15,6 +15,17 @@ zero runtime dependencies ·
 npm install toolgz
 ```
 
+> ### 🤖 If a coding agent is reading this
+>
+> Point it at **[`llms.txt`](llms.txt)** — prescriptive integration instructions written
+> for an agent, shipped inside the package at `node_modules/toolgz/llms.txt`. It covers
+> the three-step contract, the rules that are not optional, per-model behaviour, and a
+> `Do not` list of the mistakes we have actually fielded.
+>
+> ```
+> Read node_modules/toolgz/llms.txt before integrating toolgz.
+> ```
+
 ---
 
 ## The problem
@@ -33,7 +44,7 @@ Reclaiming the room is what this does.
 ```ts
 import { compress, recommendLevel, forAnthropic } from "toolgz";
 
-const { level } = recommendLevel(myTools);    // 1 for small sets, 3 for large ones
+const { level } = recommendLevel(myTools);    // advice: 1 for a small block, 3 for a big one
 const c = compress(myTools, { level });       // your existing MCP/SDK tool array
 const { tools, system } = forAnthropic(c);    // send these instead
 ```
@@ -41,7 +52,11 @@ const { tools, system } = forAnthropic(c);    // send these instead
 `compress(myTools)` with no `level` gives you **level 1** — safe, native tool calling,
 provider schema enforcement intact, and 13–39% smaller (13–32% on the synthetic benchmark, 39% on the real 149-tool corpus). The 71–85% figures below are
 **level 3**, which is what `recommendLevel` returns once a tool set is big enough to
-amortise the dispatcher. Ask for it explicitly with `{ level: 3 }` if you prefer.
+amortise the dispatcher.
+
+**Note the shape of those three lines: `recommendLevel` only advises, and you pass its
+answer back in.** Nothing upgrades itself — `compress(myTools)` is level 1 whether you
+have 2 tools or 500. See [Which level to use](#which-level-to-use) for why.
 
 Then translate the model's call back before you dispatch:
 
@@ -226,6 +241,62 @@ tried and removed in 0.2.0 because they were smaller and still worse.
 
 ## Which level to use
 
+### The whole idea, in plain English
+
+Your tool definitions are a **menu** handed to the model at the start of every single
+conversation. It is long, and most of it is flowery prose about each dish.
+
+- **Level 1** — the same menu with the prose cut. It is still a real menu: the model
+  points at a dish **by name**, and *the kitchen checks the order makes sense* before
+  cooking. This is the default, and it gives up nothing.
+- **Level 3** — throw the menu away. Hand the model a **numbered list** and one waiter.
+  It says "number 12, no onions," and the waiter knows what that means. The list is
+  tiny. But the kitchen no longer checks the order — **toolgz checks it instead**,
+  against your original schema, and hands back a readable error if it's wrong.
+
+If the model needs to know what number 12 comes with, it asks. That's the `q()` lookup,
+and it costs about half a turn.
+
+So: **level 1 is smaller. Level 3 is much smaller and you take over order-checking.**
+
+### Two things that surprise people
+
+**1. Nothing changes level on its own. You are always the one who picks.**
+
+```ts
+compress(myTools)                    // level 1. ALWAYS — 2 tools or 500.
+compress(myTools, { level: 3 })      // level 3, because you asked for it
+
+const { level } = recommendLevel(myTools);   // just advice: returns 1 or 3
+compress(myTools, { level });                // now it's 3, because you passed it in
+```
+
+`recommendLevel()` **advises**; it does not act. On our 149-tool corpus,
+`compress(myTools)` saves 45.2% and `compress(myTools, { level: 3 })` saves 96.5% — so
+forgetting to pass the level back in quietly leaves half the win on the table.
+
+This is deliberate. Level 3 gives up provider-side schema enforcement, and silently
+changing a caller's correctness guarantees because their tool array grew would be a
+worse bug than the tokens are worth.
+
+**2. It switches on how *big* the block is, not how *many* tools you have.**
+
+A tool can be 20 tokens or 460, so counting them tells you very little:
+
+| Tool set | Level 1 block | Recommends |
+|---|---:|:-:|
+| 72 tools, one parameter each | ~5,000 tokens | **1** |
+| 200 tools, one parameter each | ~14,100 tokens | **3** |
+| **40** real MCP tools | ~10,600 tokens | **3** |
+| 149 real MCP tools | ~42,700 tokens | **3** |
+
+Forty chatty tools cross the line while 72 terse ones don't. The threshold is **10,000
+tokens** — about 5% of a 200k window. Below that, reclaiming the block doesn't change
+what fits, so keeping the provider's own argument validation is worth more than the
+saving.
+
+### The levels in full
+
 Ask the library. It returns 1 or 3, never 2, and explains itself:
 
 ```ts
@@ -302,6 +373,11 @@ Level 3 has several map styles. Which one is cheapest turns out to depend on the
 model, so you can hand `compress()` a model id and let it use what was actually
 measured:
 
+> **This is the one thing the library does choose for you, and only if you pass
+> `model`.** Note the difference from levels: a map style is a pure encoding choice, so
+> picking a better one cannot change your results. The level *can* — level 3 hands
+> argument validation from the provider to toolgz — so that stays your explicit call.
+
 ```ts
 compress(myTools, { level: 3, model: "gpt-5.6-sol", objective: "cost" });
 ```
@@ -356,6 +432,10 @@ than a bug report from you.
 npx tsx examples/01-minimal.ts
 ```
 
+**[`examples/README.md`](examples/README.md) walks through all five** with their real
+output and what each one is trying to teach — start there if you would rather read than
+run.
+
 | File | Shows |
 |---|---|
 | [`01-minimal.ts`](examples/01-minimal.ts) | the smallest useful thing: `recommendLevel` → `compress` → `resolve` |
@@ -783,8 +863,13 @@ Use `forGemini`, which strips the keywords Gemini won't accept. If a new one
 appears, it's a one-line addition to the adapter.
 
 **Savings look small.**
-Check tool count and shape with `recommendLevel(myTools)`. Under ~15 tools
-there's little to reclaim. Also check `c.stats`:
+First check you actually passed a level: `compress(myTools)` is **level 1**, and it
+stays level 1 no matter how many tools you hand it. `recommendLevel` advises, it does
+not act — you have to pass its answer in as `{ level }`.
+
+If you did, ask `recommendLevel(myTools)` and read the `reason`. It reports the size of
+your level-1 block, and under ~10,000 tokens there is little worth reclaiming. Also
+check `c.stats`:
 
 ```ts
 console.log(c.stats);
@@ -885,7 +970,15 @@ Returns:
 
 ### `recommendLevel(tools, namespaceOf?) → Recommendation`
 
-`{ level, reason, toolCount, namespaceCount, opsPerNamespace }`. Returns 1 or 3.
+`{ level, reason, toolCount, namespaceCount, opsPerNamespace }`. Returns 1 or 3, never 2.
+
+**It advises; it does not act.** Pass the answer back in yourself —
+`compress(tools, { level })`. Calling `compress(tools)` alone is level 1 regardless of
+how large `tools` is.
+
+The decision is on the **size** of the level-1 block (threshold 10,000 tokens ≈ 5% of a
+200k window), not on `toolCount`. The three shape fields are reported for your own
+logging; only block size drives the level.
 
 ### Provider adapters
 
@@ -974,8 +1067,13 @@ xAI is OpenAI-compatible — use `forOpenAI` with `baseURL: "https://api.x.ai/v1
 - **It has not been measured on a non-frontier model at level 3.** On Haiku 4.5, argument
   errors rose sharply (17 of 30 runs) — all caught and retried, no task lost, but that is the
   known edge.
-- **It is not magic on ten tools.** Under ~15 tools there is little to reclaim;
-  `recommendLevel()` will tell you so.
+- **It is not magic on a small tool block.** Under ~10,000 tokens at level 1 there is
+  little worth reclaiming, and `recommendLevel()` will say so and keep you on level 1.
+  That is usually a small number of tools, but not always — it depends on how verbose
+  your schemas are, not on the count.
+- **It does not pick a level for you.** `compress()` defaults to level 1 and stays there;
+  reaching level 3 is always an explicit `{ level }`. Deliberate — level 3 trades away
+  provider-side schema enforcement, and that is not a trade to make behind your back.
 
 ## Determinism
 

package/dist/recommend.d.ts

@@ -31,9 +31,19 @@ export type Recommendation = {
  *    slower and dearer than level 3. It is never recommended. It stays in the
  *    API for callers who need real operation names on the wire.
  *
- * The level 1 → 3 threshold is driven by tools-per-namespace, not tool count:
- * the compound-dispatcher overhead is paid per namespace, so a wide, sparse
- * tool set stays cheaper at level 1 even when the total count is large.
+ * The level 1 → 3 threshold is an absolute size test on the level-1 block —
+ * neither tool count nor namespace shape. A tool is 20 to 460 tokens depending
+ * on how verbose its schema is, so 40 real MCP tools cross the line while 72
+ * one-parameter tools do not.
+ *
+ * An earlier version gated on tools-per-namespace. That is a *level 2* question
+ * (level 2 pays dispatcher overhead per namespace); level 3 uses one flat
+ * dispatcher and is indifferent to shape. See the comment on THRESHOLD_TOKENS.
+ *
+ * This function only ever *advises*. `compress()` defaults to level 1 and stays
+ * there unless the caller passes a level explicitly — level 3 trades away the
+ * provider's own schema enforcement, and that is not a trade to make on a
+ * caller's behalf because their tool array grew.
  */
 export declare function recommendLevel(tools: Tool[], namespaceOf?: (n: string) => {
     ns: string;

package/dist/recommend.js

@@ -25,9 +25,19 @@ import { compress } from "./index.js";
  *    slower and dearer than level 3. It is never recommended. It stays in the
  *    API for callers who need real operation names on the wire.
  *
- * The level 1 → 3 threshold is driven by tools-per-namespace, not tool count:
- * the compound-dispatcher overhead is paid per namespace, so a wide, sparse
- * tool set stays cheaper at level 1 even when the total count is large.
+ * The level 1 → 3 threshold is an absolute size test on the level-1 block —
+ * neither tool count nor namespace shape. A tool is 20 to 460 tokens depending
+ * on how verbose its schema is, so 40 real MCP tools cross the line while 72
+ * one-parameter tools do not.
+ *
+ * An earlier version gated on tools-per-namespace. That is a *level 2* question
+ * (level 2 pays dispatcher overhead per namespace); level 3 uses one flat
+ * dispatcher and is indifferent to shape. See the comment on THRESHOLD_TOKENS.
+ *
+ * This function only ever *advises*. `compress()` defaults to level 1 and stays
+ * there unless the caller passes a level explicitly — level 3 trades away the
+ * provider's own schema enforcement, and that is not a trade to make on a
+ * caller's behalf because their tool array grew.
  */
 export function recommendLevel(tools, namespaceOf = defaultNamespaceOf) {
     const namespaces = new Set(tools.map((t) => namespaceOf(t.name).ns));

package/llms.txt

@@ -0,0 +1,205 @@
+# toolgz
+
+> Compresses LLM tool definitions to reclaim context window. Zero runtime dependencies,
+> TypeScript, Apache-2.0. Works with any model and any framework: it transforms a tool
+> array before you send it, and translates the model's call back afterwards.
+
+**This file is written for a coding agent integrating toolgz into a project.** It is
+prescriptive on purpose. If you are contributing to toolgz itself, read `AGENTS.md` in the
+repository instead — different audience, different rules.
+
+Installed copy lives at `node_modules/toolgz/llms.txt`. Full prose docs: `README.md`.
+
+---
+
+## The whole integration, correctly
+
+```ts
+import { compress, recommendLevel, forAnthropic } from "toolgz";
+
+// 1. Decide the level ONCE, at startup — not per request.
+const { level } = recommendLevel(myTools);
+const c = compress(myTools, { level });
+
+// 2. Send c.tools instead of your tool array, and append the preamble to your system prompt.
+const { tools, system } = forAnthropic(c);
+
+// 3. Translate every tool call back before you dispatch.
+for (const block of res.content.filter((b) => b.type === "tool_use")) {
+  const r = c.resolve(block.name, block.input);
+  if (r.kind === "call") {
+    const out = await myDispatch(r.name, r.args);      // real name, real args
+    results.push({ type: "tool_result", tool_use_id: block.id, content: JSON.stringify(out) });
+  } else if (r.kind === "meta") {
+    // toolgz answered a lookup itself. DO NOT dispatch. Return its text to the model.
+    results.push({ type: "tool_result", tool_use_id: block.id, content: r.result });
+  } else {
+    // Recoverable by design: hand the message back and the model retries.
+    results.push({ type: "tool_result", tool_use_id: block.id, content: `Error: ${r.message}`, is_error: true });
+  }
+}
+```
+
+Those three steps are the entire contract. `myDispatch` receives exactly what it received
+before toolgz existed.
+
+## Rules that are not optional
+
+1. **`resolve()` every tool call.** At level 3 the model emits opaque codes (`t(f="a0")`),
+   so dispatching `block.name` directly will fail or dispatch the wrong tool. Route
+   everything through `resolve()` even at level 0, so the level becomes a one-line change.
+
+2. **Handle all three `kind` values.** `call` → dispatch. `meta` → toolgz already answered
+   a lookup; return `r.result` to the model and dispatch nothing. `error` → return
+   `r.message` to the model as an error tool_result; do **not** throw. The `error` path is
+   the recovery mechanism, not a failure.
+
+3. **Send the system preamble.** At level 3 the tool map lives in the system prompt. Drop
+   it and the model has no idea what any code means. It is `""` at levels 0–1, so
+   appending it unconditionally is always safe.
+
+4. **`compress()` never changes level on its own.** `compress(tools)` is level 1 forever,
+   2 tools or 500. `recommendLevel()` *advises*; you pass its answer in as `{ level }`.
+   Deliberate: level 3 moves argument validation from the provider to toolgz, which is not
+   a change to make behind a caller's back.
+
+5. **Leave `validate` on** (the default). Levels 2–3 give up the provider's constrained
+   decoding, so toolgz validating against the original schema is what catches malformed
+   arguments. On a weak model, disabling it converts roughly half of all runs from a
+   recovered retry into a bad dispatch.
+
+6. **Compress once, reuse the result.** `compress()` is deterministic — same tools in,
+   byte-identical payload out — so the output is prompt-cache friendly. Recomputing per
+   request is wasted work; changing the tool list mid-conversation invalidates the cache
+   and strands codes the model has already seen.
+
+## Choosing a level
+
+Ask, don't guess: `recommendLevel(myTools)` returns `{ level, reason }` — 1 or 3, never 2.
+
+| Level | Wire | Names | Provider schema enforcement | When |
+|:-:|---|:-:|:-:|---|
+| 0 | unchanged | real | yes | A/B control only |
+| **1** | one native tool each, flattened schemas | real | **yes** | **default.** Reclaims 13–39%, gives up nothing |
+| 2 | one tool per namespace | real | no | only if you need real op names on the wire |
+| **3** | one dispatcher + one lookup | codes | no | **large tool sets.** Up to ~85–96% |
+
+The switch is the **size** of the level-1 block, not the tool count — threshold 10,000
+tokens (~5% of a 200k window). A tool is 20 to 460 tokens depending on schema verbosity,
+so 40 real MCP tools cross it while 72 one-parameter tools do not.
+
+Level 2 is dominated by level 3 on accuracy (six times the malformed arguments) and is
+never recommended. It stays in the API for callers who need real operation names.
+
+## Provider adapters — pick the right one
+
+```ts
+forAnthropic(c, { cache?, ttl? })  // → { tools, system }        Messages API
+forOpenAI(c)                       // → { tools, systemPreamble } /v1/chat/completions
+forOpenAIResponses(c)              // → { tools, systemPreamble } /v1/responses
+forGemini(c)                       // → { tools, systemPreamble } generateContent
+```
+
+Two mistakes that produce confusing provider errors:
+
+- **OpenAI has two incompatible tool shapes.** `/v1/chat/completions` wants nested
+  `{type, function:{…}}`; `/v1/responses` wants flat `{type, name, …}`. If you set
+  reasoning effort on a GPT-5.x model you must use `/v1/responses`, therefore
+  `forOpenAIResponses`. Using the wrong adapter is rejected at the API.
+- **Gemini returns ONE wrapper object,** not one entry per tool. Read
+  `tools[0].functionDeclarations.length`; `tools.length` is always 1.
+
+`forGemini` also repairs three schema forms Gemini rejects with a 400: an array-valued
+`type`, `type: "array"` with no `items`, and a non-string `enum`. Constraints stripped to
+satisfy Gemini are still enforced by `resolve()` against your original schema — nothing is
+silently dropped.
+
+xAI is OpenAI-compatible: use `forOpenAI` with `baseURL: "https://api.x.ai/v1"`. **xAI
+caps tool arrays at 350**; other providers accept 1,200+. Level 3 sends 2 either way.
+
+## Model-specific behaviour
+
+Pass an exact model id and toolgz uses the best *measured* map style for it:
+
+```ts
+compress(myTools, { level: 3, model: "gpt-5.6-sol", objective: "cost" });
+```
+
+| Model | `objective: "cost"` | Measured vs default |
+|---|---|---:|
+| `gpt-5.6-sol` | `explicit` | −20.7% |
+| `gemini-3.1-pro-preview` | `explicit` | −15.4% |
+| `claude-opus-5` | `explicit` | −9.0% |
+| `grok-4.5` | *(no entry)* | `explicit` is **+13.2%** there |
+
+- Model ids are **exact, never families**. A result for `gpt-5.6-sol` says nothing about
+  `gpt-5.7`. An unknown model gets the conservative default — absence of evidence, not a
+  prediction.
+- The default objective is `occupancy`, which has **no** entries: nothing beat the default
+  by more than 3.1% on that axis, under the 5% effect-size floor. Only `cost` has rows.
+- This is the one thing the library picks for you, and only when you pass `model`. A map
+  style is a pure encoding choice, so a better one cannot change results. The *level*
+  can — which is why that stays explicit.
+- Do not set `mapStyle` by hand unless you have measured your own workload. Prefer
+  `model` + `objective`.
+
+## MCP servers
+
+`tools/list` returns `{ name, description, inputSchema }` per tool — exactly what
+`compress()` accepts. **There is no adapter layer.** Merge the arrays from every server
+and pass them in.
+
+`input_schema` and `inputSchema` are both accepted, so Anthropic-shaped and MCP-shaped
+tools can be mixed.
+
+Watch for name collisions when merging catalogues from independent servers. If two tool
+names normalise identically, toolgz refuses to alias either one rather than guess — the
+map code still works.
+
+## API surface
+
+```ts
+compress(tools, options?) → CompressResult
+recommendLevel(tools, namespaceOf?) → { level, reason, toolCount, namespaceCount, opsPerNamespace }
+selectMapStyle(options) → { mapStyle, requestedMapStyle?, fallbackReason? }   // pure
+```
+
+`CompressOptions`: `level`, `mapStyle`, `namespaceOf`, `aliasOf`, `searchLimit`,
+`validate`, `model`, `objective`.
+
+`CompressResult`: `tools`, `systemPreamble`, `cachePreamble`, `resolve()`, `codeFor()`,
+`encodeCallForTest()`, `stats`.
+
+`stats`: `level`, `mapStyle`, `requestedMapStyle`, `fallbackReason`, `toolCount`,
+`wireToolCount`, `originalChars`, `compressedChars`, `savedPct`.
+
+`namespaceOf` must return `{ ns, op }`, **not** a bare string. Returning a string throws
+with a corrected example — it used to fail silently and produce a plausible-looking but
+wrong map.
+
+## Things to tell the user, not assume
+
+- **`savedPct` is a CHARACTER saving**, a few points optimistic against tokens. Providers
+  charge a fixed framing cost per tool definition that character counting cannot see. For
+  any published figure, measure with the provider's own token counter.
+- **Never use `tiktoken` to count Claude tokens** — it is OpenAI's tokenizer and is wrong
+  for Claude by 15–20%+. Use Anthropic's `count_tokens`.
+- **Token saving ≠ cost saving.** Prompt caching already makes tool tokens cheap; the claim
+  here is context-window *occupancy*. Cost follows by a variable amount, and on OpenAI —
+  where reasoning output dominates the bill — it was as little as 7%.
+- **Level 3 costs turns.** Roughly 0.3–1.7 lookup calls per task, about half an extra turn.
+  If latency matters more than context, stay at level 1.
+- **Below the frontier tier, argument formatting degrades at level 3.** On Haiku 4.5, 17 of
+  30 runs produced a malformed argument — all caught by `validate` and recovered, no task
+  lost, but that is the known edge. Tool *choice* did not degrade.
+
+## Do not
+
+- Do not dispatch `block.name` without `resolve()`.
+- Do not throw on `kind: "error"` — return it to the model.
+- Do not dispatch on `kind: "meta"`.
+- Do not omit `systemPreamble` at level 3.
+- Do not set `validate: false` to silence a malformed-argument error; fix the schema.
+- Do not call `compress()` inside the request loop.
+- Do not report `savedPct` as a token or cost saving.
+- Do not assume a level was chosen for you.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "toolgz",
-  "version": "0.2.8",
+  "version": "0.2.10",
   "description": "Compress LLM tool definitions to reclaim context window. Measured across Anthropic, OpenAI, Google and xAI.",
   "main": "dist/index.js",
   "scripts": {
@@ -45,6 +45,7 @@
   "files": [
     "dist",
     "README.md",
+    "llms.txt",
     "LICENSE",
     "NOTICE"
   ],