toolgz 0.2.6 → 0.2.8

package/README.md

@@ -346,6 +346,31 @@ style in 0.2.0.
 
 ---
 
+## Runnable examples
+
+Five files in [`examples/`](examples), all offline — no API key, no cost. Every one is
+**executed by the test suite**, so an example that stops working is a failing test rather
+than a bug report from you.
+
+```bash
+npx tsx examples/01-minimal.ts
+```
+
+| File | Shows |
+|---|---|
+| [`01-minimal.ts`](examples/01-minimal.ts) | the smallest useful thing: `recommendLevel` → `compress` → `resolve` |
+| [`02-agent-loop.ts`](examples/02-agent-loop.ts) | the full loop against a scripted model, covering all three `resolve()` outcomes including a recovery |
+| [`03-mcp-servers.ts`](examples/03-mcp-servers.ts) | 149 real tools from 14 MCP servers, all four levels, and the name-collision hazard |
+| [`04-providers.ts`](examples/04-providers.ts) | the four provider envelopes side by side, plus the Gemini schema repairs |
+| [`05-per-model.ts`](examples/05-per-model.ts) | `model`/`objective` selection and reading `stats` to see what was actually used |
+
+Two things `04-providers.ts` demonstrates rather than describes, because both have bitten
+people: `/v1/responses` needs the **flat** tool shape when you set reasoning effort, and
+Gemini returns **one** wrapper object containing all declarations, so you count
+`tools[0].functionDeclarations.length`.
+
+---
+
 ## Using it: the full guide
 
 Everything below was a separate `docs/GUIDE.md`. It is inline now, deliberately: the
@@ -767,8 +792,7 @@ console.log(c.stats);
 //   originalChars: 61461, compressedChars: 2211, savedPct: 96.4 }
 ```
 
-`savedPct` is a character-count proxy, not a token count — useful for a quick
-sanity check, not for billing.
+`savedPct` is a character saving, a few points optimistic against tokens. `originalChars` and `compressedChars` give the raw character counts.
 
 **I need the real token numbers.**
 Count them with the provider's own endpoint. Never use `tiktoken` for Claude —
@@ -842,11 +866,21 @@ Returns:
 | `codeFor(name)` | `→ string` | real name → level-3 code; throws below level 3 |
 | `stats` | `CompressStats` | `level`, `mapStyle`, `requestedMapStyle`, `fallbackReason`, `toolCount`, `wireToolCount`, `originalChars`, `compressedChars`, `savedPct` |
 
-> **`savedPct` counts characters, not tokens — and it overstates.** It is derived from
-> `countSchemaTokensApprox`, which is just `JSON.stringify(x).length`. On our real corpus
-> it reports ~97.8% where `count_tokens` measures **95.6%**. Use it as a cheap sanity
-> signal, never as a published figure; for anything you quote, measure with your
-> provider's token counter.
+> **`savedPct` is a character saving, and runs a few points optimistic against tokens.**
+> On the real 149-tool corpus it reports 46.8% at level 1 where `count_tokens` measures
+> 39.2%, and 96.6% at level 3 against 95.6%.
+>
+> We tried making it a token estimate in 0.2.7 and reverted it in 0.2.8. Providers charge a
+> fixed framing cost per tool definition that character counting cannot see: at 149 tools
+> it amortises away, at 2 tools it dominates, and the calibrated estimate was off by 44% on
+> a small level-1 block while being within 1% at scale. The plain character ratio is the
+> smaller and more predictable error.
+>
+> Use it as a local signal. For anything you publish, measure with your provider's token
+> counter. `originalChars` and `compressedChars` are on `stats` for the raw counts.
+>
+> A negative value is possible and correct: on a very small tool set, level 1's signature
+> line can exceed the per-property descriptions it strips.
 | `encodeCallForTest(name, args)` | `→ {name, args}` | build the raw call a model would emit; test aid |
 
 ### `recommendLevel(tools, namespaceOf?) → Recommendation`
@@ -955,7 +989,7 @@ and it does not get deleted.
 ## Development
 
 ```bash
-npm test        # 272 tests, offline, no cost
+npm test        # 283 tests, offline, no cost
 npm run build   # tsc → dist/ with .d.ts
 
 npx tsx bench/harness/run-multi.ts --provider=all --reps=3 --variants   # costs money

package/dist/index.js

@@ -160,6 +160,19 @@ export function compress(input, options = {}) {
     };
     const finish = (wire, systemPreamble, cachePreamble, resolve, encode) => {
         const compressedChars = countSchemaTokensApprox(wire) + systemPreamble.length;
+        // `savedPct` is a CHARACTER saving, deliberately.
+        //
+        // 0.2.7 tried to make it a token estimate by dividing each side by a chars-per-token
+        // ratio calibrated against count_tokens. That was a mistake, and measurement caught
+        // it: providers charge a fixed framing cost per tool definition that character
+        // counting cannot see. At 149 tools it amortises away, at 2 tools it dominates — the
+        // ratio approach was off by 44% on a 2-tool level-1 block while being within 1% at
+        // 149. No local character-based calculation can span that range.
+        //
+        // The plain character ratio is the smaller and more predictable error: it runs a few
+        // points optimistic (−7.7% chars against −4.9% real tokens on a 2-tool set; 46.8%
+        // against 39.2% on 149 real tools). So it is reported as what it is, and the docs say
+        // to measure with your provider's counter for anything you publish.
         return {
             tools: wire,
             systemPreamble,

package/dist/recommend.js

@@ -57,7 +57,11 @@ export function recommendLevel(tools, namespaceOf = defaultNamespaceOf) {
     // provider's own schema enforcement, which levels 2-3 give up. That is worth more
     // than a saving you would not notice — hence an absolute threshold on the block,
     // not a shape test.
-    const THRESHOLD_TOKENS = 4000;
+    // 10,000 tokens is ~5% of a 200k window. Below that, reclaiming the block does not
+    // change what fits, and level 1 keeps the provider's own argument validation — worth
+    // more than a saving you would not notice. An external reviewer flagged the earlier
+    // 4,000 threshold for pushing 14-tool sets into dispatcher mode; they were right.
+    const THRESHOLD_TOKENS = 10000;
     if (l1Tokens < THRESHOLD_TOKENS) {
         return {
             ...base,
@@ -68,6 +72,6 @@ export function recommendLevel(tools, namespaceOf = defaultNamespaceOf) {
     return {
         ...base,
         level: 3,
-        reason: `${toolCount} tools take ~${l1Tokens.toLocaleString()} tokens at level 1; level 3 replaces them with two dispatcher tools plus a cached map. Measured on 149 real MCP tools: 41,648 tokens at level 1 against 2,980 at level 3, and 60/60 tasks completed across four frontier providers with zero hallucinated names. The cost is roughly 0.3-1.7 lookup calls per task — so about half an extra turn — plus the loss of provider-side argument checking — keep \`validate\` on, which is what catches malformed arguments instead. If latency matters more than context, stay at level 1.`,
+        reason: `${toolCount} tools take ~${l1Tokens.toLocaleString()} tokens at level 1; level 3 replaces them with two dispatcher tools plus a cached map. Measured on 149 real MCP tools: 41,648 tokens at level 1 against 2,980 at level 3, and 60/60 tasks completed across four frontier providers with zero hallucinated names. The cost is roughly 0.3-1.7 lookup calls per task — so about half an extra turn — plus the loss of provider-side argument checking. Keep \`validate\` on; it is what catches malformed arguments instead. If latency matters more than context, stay at level 1.`,
     };
 }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "toolgz",
-  "version": "0.2.6",
+  "version": "0.2.8",
   "description": "Compress LLM tool definitions to reclaim context window. Measured across Anthropic, OpenAI, Google and xAI.",
   "main": "dist/index.js",
   "scripts": {