toolgz 0.2.7 → 0.2.9

package/README.md

@@ -2,7 +2,7 @@
 
 <p><strong>Your agent spends 30–70k tokens of context on tool definitions before the user types a word. toolgz gets up to ~85% of it back.</strong></p>
 
-<p><em>On a large tool set, at level 3. The safe default (level 1) reclaims 13–39% and gives up nothing — <code>recommendLevel()</code> picks for you.</em></p>
+<p><em>On a large tool set, at level 3. The default (level 1) reclaims 13–39% and gives up nothing. <code>recommendLevel()</code> tells you which one your tools want; you pass its answer in.</em></p>
 
 <p>
 <a href="#measured-results">420-run cross-provider sweep</a> ·
@@ -33,7 +33,7 @@ Reclaiming the room is what this does.
 ```ts
 import { compress, recommendLevel, forAnthropic } from "toolgz";
 
-const { level } = recommendLevel(myTools);    // 1 for small sets, 3 for large ones
+const { level } = recommendLevel(myTools);    // advice: 1 for a small block, 3 for a big one
 const c = compress(myTools, { level });       // your existing MCP/SDK tool array
 const { tools, system } = forAnthropic(c);    // send these instead
 ```
@@ -41,7 +41,11 @@ const { tools, system } = forAnthropic(c);    // send these instead
 `compress(myTools)` with no `level` gives you **level 1** — safe, native tool calling,
 provider schema enforcement intact, and 13–39% smaller (13–32% on the synthetic benchmark, 39% on the real 149-tool corpus). The 71–85% figures below are
 **level 3**, which is what `recommendLevel` returns once a tool set is big enough to
-amortise the dispatcher. Ask for it explicitly with `{ level: 3 }` if you prefer.
+amortise the dispatcher.
+
+**Note the shape of those three lines: `recommendLevel` only advises, and you pass its
+answer back in.** Nothing upgrades itself — `compress(myTools)` is level 1 whether you
+have 2 tools or 500. See [Which level to use](#which-level-to-use) for why.
 
 Then translate the model's call back before you dispatch:
 
@@ -226,6 +230,62 @@ tried and removed in 0.2.0 because they were smaller and still worse.
 
 ## Which level to use
 
+### The whole idea, in plain English
+
+Your tool definitions are a **menu** handed to the model at the start of every single
+conversation. It is long, and most of it is flowery prose about each dish.
+
+- **Level 1** — the same menu with the prose cut. It is still a real menu: the model
+  points at a dish **by name**, and *the kitchen checks the order makes sense* before
+  cooking. This is the default, and it gives up nothing.
+- **Level 3** — throw the menu away. Hand the model a **numbered list** and one waiter.
+  It says "number 12, no onions," and the waiter knows what that means. The list is
+  tiny. But the kitchen no longer checks the order — **toolgz checks it instead**,
+  against your original schema, and hands back a readable error if it's wrong.
+
+If the model needs to know what number 12 comes with, it asks. That's the `q()` lookup,
+and it costs about half a turn.
+
+So: **level 1 is smaller. Level 3 is much smaller and you take over order-checking.**
+
+### Two things that surprise people
+
+**1. Nothing changes level on its own. You are always the one who picks.**
+
+```ts
+compress(myTools)                    // level 1. ALWAYS — 2 tools or 500.
+compress(myTools, { level: 3 })      // level 3, because you asked for it
+
+const { level } = recommendLevel(myTools);   // just advice: returns 1 or 3
+compress(myTools, { level });                // now it's 3, because you passed it in
+```
+
+`recommendLevel()` **advises**; it does not act. On our 149-tool corpus,
+`compress(myTools)` saves 45.2% and `compress(myTools, { level: 3 })` saves 96.5% — so
+forgetting to pass the level back in quietly leaves half the win on the table.
+
+This is deliberate. Level 3 gives up provider-side schema enforcement, and silently
+changing a caller's correctness guarantees because their tool array grew would be a
+worse bug than the tokens are worth.
+
+**2. It switches on how *big* the block is, not how *many* tools you have.**
+
+A tool can be 20 tokens or 460, so counting them tells you very little:
+
+| Tool set | Level 1 block | Recommends |
+|---|---:|:-:|
+| 72 tools, one parameter each | ~5,000 tokens | **1** |
+| 200 tools, one parameter each | ~14,100 tokens | **3** |
+| **40** real MCP tools | ~10,600 tokens | **3** |
+| 149 real MCP tools | ~42,700 tokens | **3** |
+
+Forty chatty tools cross the line while 72 terse ones don't. The threshold is **10,000
+tokens** — about 5% of a 200k window. Below that, reclaiming the block doesn't change
+what fits, so keeping the provider's own argument validation is worth more than the
+saving.
+
+### The levels in full
+
 Ask the library. It returns 1 or 3, never 2, and explains itself:
 
 ```ts
@@ -302,6 +362,11 @@ Level 3 has several map styles. Which one is cheapest turns out to depend on the
 model, so you can hand `compress()` a model id and let it use what was actually
 measured:
 
+> **This is the one thing the library does choose for you, and only if you pass
+> `model`.** Note the difference from levels: a map style is a pure encoding choice, so
+> picking a better one cannot change your results. The level *can* — level 3 hands
+> argument validation from the provider to toolgz — so that stays your explicit call.
+
 ```ts
 compress(myTools, { level: 3, model: "gpt-5.6-sol", objective: "cost" });
 ```
@@ -346,6 +411,31 @@ style in 0.2.0.
 
 ---
 
+## Runnable examples
+
+Five files in [`examples/`](examples), all offline — no API key, no cost. Every one is
+**executed by the test suite**, so an example that stops working is a failing test rather
+than a bug report from you.
+
+```bash
+npx tsx examples/01-minimal.ts
+```
+
+| File | Shows |
+|---|---|
+| [`01-minimal.ts`](examples/01-minimal.ts) | the smallest useful thing: `recommendLevel` → `compress` → `resolve` |
+| [`02-agent-loop.ts`](examples/02-agent-loop.ts) | the full loop against a scripted model, covering all three `resolve()` outcomes including a recovery |
+| [`03-mcp-servers.ts`](examples/03-mcp-servers.ts) | 149 real tools from 14 MCP servers, all four levels, and the name-collision hazard |
+| [`04-providers.ts`](examples/04-providers.ts) | the four provider envelopes side by side, plus the Gemini schema repairs |
+| [`05-per-model.ts`](examples/05-per-model.ts) | `model`/`objective` selection and reading `stats` to see what was actually used |
+
+Two things `04-providers.ts` demonstrates rather than describes, because both have bitten
+people: `/v1/responses` needs the **flat** tool shape when you set reasoning effort, and
+Gemini returns **one** wrapper object containing all declarations, so you count
+`tools[0].functionDeclarations.length`.
+
+---
+
 ## Using it: the full guide
 
 Everything below was a separate `docs/GUIDE.md`. It is inline now, deliberately: the
@@ -758,8 +848,13 @@ Use `forGemini`, which strips the keywords Gemini won't accept. If a new one
 appears, it's a one-line addition to the adapter.
 
 **Savings look small.**
-Check tool count and shape with `recommendLevel(myTools)`. Under ~15 tools
-there's little to reclaim. Also check `c.stats`:
+First check you actually passed a level: `compress(myTools)` is **level 1**, and it
+stays level 1 no matter how many tools you hand it. `recommendLevel` advises, it does
+not act — you have to pass its answer in as `{ level }`.
+
+If you did, ask `recommendLevel(myTools)` and read the `reason`. It reports the size of
+your level-1 block, and under ~10,000 tokens there is little worth reclaiming. Also
+check `c.stats`:
 
 ```ts
 console.log(c.stats);
@@ -767,8 +862,7 @@ console.log(c.stats);
 //   originalChars: 61461, compressedChars: 2211, savedPct: 96.4 }
 ```
 
-`savedPct` estimates the token saving (calibrated against `count_tokens`, ~1–2pp
-accurate). `originalChars` and `compressedChars` give the raw character counts.
+`savedPct` is a character saving, a few points optimistic against tokens. `originalChars` and `compressedChars` give the raw character counts.
 
 **I need the real token numbers.**
 Count them with the provider's own endpoint. Never use `tiktoken` for Claude —
@@ -842,21 +936,34 @@ Returns:
 | `codeFor(name)` | `→ string` | real name → level-3 code; throws below level 3 |
 | `stats` | `CompressStats` | `level`, `mapStyle`, `requestedMapStyle`, `fallbackReason`, `toolCount`, `wireToolCount`, `originalChars`, `compressedChars`, `savedPct` |
 
-> **`savedPct` estimates tokens, and is accurate to a point or two.** It used to be a raw
-> character ratio, which overstated by 7.6 points at level 1. Characters per token differs
-> between the two sides — uncompressed JSON is punctuation-dense, a signature line reads
-> more like prose — so it now divides each side by a ratio calibrated against
-> `count_tokens` on two unrelated corpora. Current error on the real corpus: **+1.5pp at
-> level 1, −0.3pp at level 2, +0.1pp at level 3.**
+> **`savedPct` is a character saving, and runs a few points optimistic against tokens.**
+> On the real 149-tool corpus it reports 46.8% at level 1 where `count_tokens` measures
+> 39.2%, and 96.6% at level 3 against 95.6%.
 >
-> It is still a local estimate with no API call. For a figure you publish, measure with
-> your provider's token counter; `originalChars` and `compressedChars` are also on `stats`
-> if you want the raw counts.
+> We tried making it a token estimate in 0.2.7 and reverted it in 0.2.8. Providers charge a
+> fixed framing cost per tool definition that character counting cannot see: at 149 tools
+> it amortises away, at 2 tools it dominates, and the calibrated estimate was off by 44% on
+> a small level-1 block while being within 1% at scale. The plain character ratio is the
+> smaller and more predictable error.
+>
+> Use it as a local signal. For anything you publish, measure with your provider's token
+> counter. `originalChars` and `compressedChars` are on `stats` for the raw counts.
+>
+> A negative value is possible and correct: on a very small tool set, level 1's signature
+> line can exceed the per-property descriptions it strips.
 | `encodeCallForTest(name, args)` | `→ {name, args}` | build the raw call a model would emit; test aid |
 
 ### `recommendLevel(tools, namespaceOf?) → Recommendation`
 
-`{ level, reason, toolCount, namespaceCount, opsPerNamespace }`. Returns 1 or 3.
+`{ level, reason, toolCount, namespaceCount, opsPerNamespace }`. Returns 1 or 3, never 2.
+
+**It advises; it does not act.** Pass the answer back in yourself —
+`compress(tools, { level })`. Calling `compress(tools)` alone is level 1 regardless of
+how large `tools` is.
+
+The decision is on the **size** of the level-1 block (threshold 10,000 tokens ≈ 5% of a
+200k window), not on `toolCount`. The three shape fields are reported for your own
+logging; only block size drives the level.
 
 ### Provider adapters
 
@@ -945,8 +1052,13 @@ xAI is OpenAI-compatible — use `forOpenAI` with `baseURL: "https://api.x.ai/v1
 - **It has not been measured on a non-frontier model at level 3.** On Haiku 4.5, argument
   errors rose sharply (17 of 30 runs) — all caught and retried, no task lost, but that is the
   known edge.
-- **It is not magic on ten tools.** Under ~15 tools there is little to reclaim;
-  `recommendLevel()` will tell you so.
+- **It is not magic on a small tool block.** Under ~10,000 tokens at level 1 there is
+  little worth reclaiming, and `recommendLevel()` will say so and keep you on level 1.
+  That is usually a small number of tools, but not always — it depends on how verbose
+  your schemas are, not on the count.
+- **It does not pick a level for you.** `compress()` defaults to level 1 and stays there;
+  reaching level 3 is always an explicit `{ level }`. Deliberate — level 3 trades away
+  provider-side schema enforcement, and that is not a trade to make behind your back.
 
 ## Determinism
 
@@ -960,7 +1072,7 @@ and it does not get deleted.
 ## Development
 
 ```bash
-npm test        # 277 tests, offline, no cost
+npm test        # 283 tests, offline, no cost
 npm run build   # tsc → dist/ with .d.ts
 
 npx tsx bench/harness/run-multi.ts --provider=all --reps=3 --variants   # costs money

package/dist/index.js

@@ -160,27 +160,19 @@ export function compress(input, options = {}) {
     };
     const finish = (wire, systemPreamble, cachePreamble, resolve, encode) => {
         const compressedChars = countSchemaTokensApprox(wire) + systemPreamble.length;
-        // `savedPct` is meant to answer "how much of my tool block did I get back", which is
-        // a TOKEN question. Deriving it from a raw character ratio answered a different one
-        // and overstated: on the real corpus it reported 46.8% at level 1 where count_tokens
-        // measures 39.2% — 7.6 points high, and the field was documented as "do not publish
-        // this" rather than fixed.
+        // `savedPct` is a CHARACTER saving, deliberately.
         //
-        // Characters per token is not constant across the two sides. Uncompressed JSON is
-        // punctuation-dense; a signature line or a map row reads more like prose. Measured
-        // against count_tokens on two unrelated corpora (149 real MCP tools and a 100-tool
-        // synthetic fixture) the ratios are stable per level:
+        // 0.2.7 tried to make it a token estimate by dividing each side by a chars-per-token
+        // ratio calibrated against count_tokens. That was a mistake, and measurement caught
+        // it: providers charge a fixed framing cost per tool definition that character
+        // counting cannot see. At 149 tools it amortises away, at 2 tools it dominates — the
+        // ratio approach was off by 44% on a 2-tool level-1 block while being within 1% at
+        // 149. No local character-based calculation can span that range.
         //
-        //   uncompressed  2.39 / 2.45      level 2  2.20 / 1.99
-        //   level 1       2.15 / 2.19      level 3  1.92 / 1.91
-        //
-        // Dividing by these brings the error to ~1.5 points at level 1 and ~0.1 at level 3.
-        // Still an estimate — it is a local calculation with no API call — but now an
-        // estimate of the right quantity.
-        const CHARS_PER_TOKEN = { 0: 2.42, 1: 2.17, 2: 2.1, 3: 1.92 };
-        const estTokens = (chars, at) => chars / CHARS_PER_TOKEN[at];
-        const originalTokens = estTokens(originalChars, 0);
-        const compressedTokens = estTokens(compressedChars, level);
+        // The plain character ratio is the smaller and more predictable error: it runs a few
+        // points optimistic (−7.7% chars against −4.9% real tokens on a 2-tool set; 46.8%
+        // against 39.2% on 149 real tools). So it is reported as what it is, and the docs say
+        // to measure with your provider's counter for anything you publish.
         return {
             tools: wire,
             systemPreamble,
@@ -204,7 +196,7 @@ export function compress(input, options = {}) {
                 compressedChars,
                 savedPct: originalChars === 0
                     ? 0
-                    : Math.round((1 - compressedTokens / originalTokens) * 1000) / 10,
+                    : Math.round((1 - compressedChars / originalChars) * 1000) / 10,
             },
         };
     };

package/dist/recommend.d.ts

@@ -31,9 +31,19 @@ export type Recommendation = {
  *    slower and dearer than level 3. It is never recommended. It stays in the
  *    API for callers who need real operation names on the wire.
  *
- * The level 1 → 3 threshold is driven by tools-per-namespace, not tool count:
- * the compound-dispatcher overhead is paid per namespace, so a wide, sparse
- * tool set stays cheaper at level 1 even when the total count is large.
+ * The level 1 → 3 threshold is an absolute size test on the level-1 block —
+ * neither tool count nor namespace shape. A tool is 20 to 460 tokens depending
+ * on how verbose its schema is, so 40 real MCP tools cross the line while 72
+ * one-parameter tools do not.
+ *
+ * An earlier version gated on tools-per-namespace. That is a *level 2* question
+ * (level 2 pays dispatcher overhead per namespace); level 3 uses one flat
+ * dispatcher and is indifferent to shape. See the comment on THRESHOLD_TOKENS.
+ *
+ * This function only ever *advises*. `compress()` defaults to level 1 and stays
+ * there unless the caller passes a level explicitly — level 3 trades away the
+ * provider's own schema enforcement, and that is not a trade to make on a
+ * caller's behalf because their tool array grew.
  */
 export declare function recommendLevel(tools: Tool[], namespaceOf?: (n: string) => {
     ns: string;

package/dist/recommend.js

@@ -25,9 +25,19 @@ import { compress } from "./index.js";
  *    slower and dearer than level 3. It is never recommended. It stays in the
  *    API for callers who need real operation names on the wire.
  *
- * The level 1 → 3 threshold is driven by tools-per-namespace, not tool count:
- * the compound-dispatcher overhead is paid per namespace, so a wide, sparse
- * tool set stays cheaper at level 1 even when the total count is large.
+ * The level 1 → 3 threshold is an absolute size test on the level-1 block —
+ * neither tool count nor namespace shape. A tool is 20 to 460 tokens depending
+ * on how verbose its schema is, so 40 real MCP tools cross the line while 72
+ * one-parameter tools do not.
+ *
+ * An earlier version gated on tools-per-namespace. That is a *level 2* question
+ * (level 2 pays dispatcher overhead per namespace); level 3 uses one flat
+ * dispatcher and is indifferent to shape. See the comment on THRESHOLD_TOKENS.
+ *
+ * This function only ever *advises*. `compress()` defaults to level 1 and stays
+ * there unless the caller passes a level explicitly — level 3 trades away the
+ * provider's own schema enforcement, and that is not a trade to make on a
+ * caller's behalf because their tool array grew.
  */
 export function recommendLevel(tools, namespaceOf = defaultNamespaceOf) {
     const namespaces = new Set(tools.map((t) => namespaceOf(t.name).ns));
@@ -57,7 +67,11 @@ export function recommendLevel(tools, namespaceOf = defaultNamespaceOf) {
     // provider's own schema enforcement, which levels 2-3 give up. That is worth more
     // than a saving you would not notice — hence an absolute threshold on the block,
     // not a shape test.
-    const THRESHOLD_TOKENS = 4000;
+    // 10,000 tokens is ~5% of a 200k window. Below that, reclaiming the block does not
+    // change what fits, and level 1 keeps the provider's own argument validation — worth
+    // more than a saving you would not notice. An external reviewer flagged the earlier
+    // 4,000 threshold for pushing 14-tool sets into dispatcher mode; they were right.
+    const THRESHOLD_TOKENS = 10000;
     if (l1Tokens < THRESHOLD_TOKENS) {
         return {
             ...base,
@@ -68,6 +82,6 @@ export function recommendLevel(tools, namespaceOf = defaultNamespaceOf) {
     return {
         ...base,
         level: 3,
-        reason: `${toolCount} tools take ~${l1Tokens.toLocaleString()} tokens at level 1; level 3 replaces them with two dispatcher tools plus a cached map. Measured on 149 real MCP tools: 41,648 tokens at level 1 against 2,980 at level 3, and 60/60 tasks completed across four frontier providers with zero hallucinated names. The cost is roughly 0.3-1.7 lookup calls per task — so about half an extra turn — plus the loss of provider-side argument checking — keep \`validate\` on, which is what catches malformed arguments instead. If latency matters more than context, stay at level 1.`,
+        reason: `${toolCount} tools take ~${l1Tokens.toLocaleString()} tokens at level 1; level 3 replaces them with two dispatcher tools plus a cached map. Measured on 149 real MCP tools: 41,648 tokens at level 1 against 2,980 at level 3, and 60/60 tasks completed across four frontier providers with zero hallucinated names. The cost is roughly 0.3-1.7 lookup calls per task — so about half an extra turn — plus the loss of provider-side argument checking. Keep \`validate\` on; it is what catches malformed arguments instead. If latency matters more than context, stay at level 1.`,
     };
 }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "toolgz",
-  "version": "0.2.7",
+  "version": "0.2.9",
   "description": "Compress LLM tool definitions to reclaim context window. Measured across Anthropic, OpenAI, Google and xAI.",
   "main": "dist/index.js",
   "scripts": {