@botbotgo/better-call 0.1.4 → 0.1.5

package/README.md

@@ -4,7 +4,7 @@
 
 # BetterCall
 
-**One-line wrapper. Five full BFCL remote runs completed. Best: 73.4% → 83.8%.**
+**One-line wrapper. Six full BFCL remote runs completed. Best: 73.4% → 83.8%.**
 
 ```ts
 const tools = betterTools([searchTool, calculatorTool]);
@@ -128,25 +128,26 @@ Latest completed remote run artifact: `benchmarks/bfcl-real-remote-completed-sum
 | `qwen3.5:0.8b` | 3,625 | 54.6% | 56.9% | +2.3pp | 901 |
 | `qwen3.5:2b` | 3,625 | 53.9% | 54.9% | +1.0pp | 1,308 |
 | `lfm2.5-thinking:latest` | 3,625 | 50.8% | 54.8% | +4.0pp | 1,142 |
+| `gemma4:e2b` | 3,625 | 24.3% | 24.7% | +0.4pp | 2,641 |
 
-Latest completed model category detail: `granite4.1:3b`.
+Latest completed model category detail: `gemma4:e2b`.
 
 | Category | Cases | Raw | BetterCall repair | Lift | Request errors |
 | --- | ---: | ---: | ---: | ---: | ---: |
-| `simple_python` | 400 | 86.5% | 86.8% | +0.3pp | 18 |
-| `simple_java` | 100 | 70.0% | 70.0% | +0.0pp | 0 |
-| `simple_javascript` | 50 | 72.0% | 72.0% | +0.0pp | 0 |
-| `multiple` | 200 | 90.5% | 90.5% | +0.0pp | 0 |
-| `parallel` | 200 | 83.0% | 83.0% | +0.0pp | 0 |
-| `parallel_multiple` | 200 | 78.5% | 78.5% | +0.0pp | 0 |
-| `irrelevance` | 240 | 67.1% | 100.0% | +32.9pp | 0 |
-| `live_simple` | 258 | 74.0% | 74.0% | +0.0pp | 0 |
-| `live_multiple` | 1,053 | 70.8% | 71.0% | +0.2pp | 0 |
-| `live_parallel` | 16 | 68.8% | 68.8% | +0.0pp | 0 |
-| `live_parallel_multiple` | 24 | 70.8% | 70.8% | +0.0pp | 0 |
-| `live_irrelevance` | 884 | 65.5% | 99.0% | +33.5pp | 7 |
-
-The strongest `granite4.1:3b` categories were no-tool-call tasks: `irrelevance` improved from 67.1% to 100.0%, and `live_irrelevance` improved from 65.5% to 99.0%.
+| `simple_python` | 400 | 82.0% | 83.3% | +1.3pp | 46 |
+| `simple_java` | 100 | 50.0% | 50.0% | +0.0pp | 29 |
+| `simple_javascript` | 50 | 34.0% | 36.0% | +2.0pp | 22 |
+| `multiple` | 200 | 76.5% | 79.0% | +2.5pp | 32 |
+| `parallel` | 200 | 54.0% | 54.0% | +0.0pp | 84 |
+| `parallel_multiple` | 200 | 42.5% | 42.5% | +0.0pp | 109 |
+| `irrelevance` | 240 | 52.9% | 54.2% | +1.3pp | 104 |
+| `live_simple` | 258 | 4.7% | 5.0% | +0.4pp | 238 |
+| `live_multiple` | 1,053 | 0.0% | 0.0% | +0.0pp | 1,053 |
+| `live_parallel` | 16 | 0.0% | 0.0% | +0.0pp | 16 |
+| `live_parallel_multiple` | 24 | 0.0% | 0.0% | +0.0pp | 24 |
+| `live_irrelevance` | 884 | 0.0% | 0.0% | +0.0pp | 884 |
+
+The strongest `gemma4:e2b` category was `multiple`, improving from 76.5% to 79.0%. The live categories were dominated by request errors under the 5s timeout and are preserved as measured in the artifact.
 
 Historical targeted wrapper benchmark:
 

package/benchmarks/bfcl-real-remote-completed-summary.json

@@ -2,7 +2,7 @@
   "name": "BFCL v4 full remote Ollama runs",
   "source": "Real model calls against all supported BFCL v4 single-turn tool-call categories completed in this repository",
   "note": "Endpoint redacted. Scores count request errors/timeouts as incorrect. This is not an official BFCL leaderboard submission.",
-  "generatedAt": "2026-05-08T16:02:50.925Z",
+  "generatedAt": "2026-05-08T21:10:42.234Z",
   "results": [
     {
       "model": "granite4.1:3b",
@@ -593,6 +593,124 @@
           "repaired": 131
         }
       ]
+    },
+    {
+      "model": "gemma4:e2b",
+      "total": 3625,
+      "rawCorrect": 880,
+      "betterCorrect": 895,
+      "errors": 2641,
+      "repaired": 22,
+      "categories": [
+        {
+          "model": "gemma4:e2b",
+          "category": "simple_python",
+          "total": 400,
+          "rawCorrect": 328,
+          "betterCorrect": 333,
+          "errors": 46,
+          "repaired": 5
+        },
+        {
+          "model": "gemma4:e2b",
+          "category": "simple_java",
+          "total": 100,
+          "rawCorrect": 50,
+          "betterCorrect": 50,
+          "errors": 29,
+          "repaired": 5
+        },
+        {
+          "model": "gemma4:e2b",
+          "category": "simple_javascript",
+          "total": 50,
+          "rawCorrect": 17,
+          "betterCorrect": 18,
+          "errors": 22,
+          "repaired": 1
+        },
+        {
+          "model": "gemma4:e2b",
+          "category": "multiple",
+          "total": 200,
+          "rawCorrect": 153,
+          "betterCorrect": 158,
+          "errors": 32,
+          "repaired": 6
+        },
+        {
+          "model": "gemma4:e2b",
+          "category": "parallel",
+          "total": 200,
+          "rawCorrect": 108,
+          "betterCorrect": 108,
+          "errors": 84,
+          "repaired": 0
+        },
+        {
+          "model": "gemma4:e2b",
+          "category": "parallel_multiple",
+          "total": 200,
+          "rawCorrect": 85,
+          "betterCorrect": 85,
+          "errors": 109,
+          "repaired": 1
+        },
+        {
+          "model": "gemma4:e2b",
+          "category": "irrelevance",
+          "total": 240,
+          "rawCorrect": 127,
+          "betterCorrect": 130,
+          "errors": 104,
+          "repaired": 3
+        },
+        {
+          "model": "gemma4:e2b",
+          "category": "live_simple",
+          "total": 258,
+          "rawCorrect": 12,
+          "betterCorrect": 13,
+          "errors": 238,
+          "repaired": 1
+        },
+        {
+          "model": "gemma4:e2b",
+          "category": "live_multiple",
+          "total": 1053,
+          "rawCorrect": 0,
+          "betterCorrect": 0,
+          "errors": 1053,
+          "repaired": 0
+        },
+        {
+          "model": "gemma4:e2b",
+          "category": "live_parallel",
+          "total": 16,
+          "rawCorrect": 0,
+          "betterCorrect": 0,
+          "errors": 16,
+          "repaired": 0
+        },
+        {
+          "model": "gemma4:e2b",
+          "category": "live_parallel_multiple",
+          "total": 24,
+          "rawCorrect": 0,
+          "betterCorrect": 0,
+          "errors": 24,
+          "repaired": 0
+        },
+        {
+          "model": "gemma4:e2b",
+          "category": "live_irrelevance",
+          "total": 884,
+          "rawCorrect": 0,
+          "betterCorrect": 0,
+          "errors": 884,
+          "repaired": 0
+        }
+      ]
     }
   ]
 }

package/docs/banner.svg

@@ -1,6 +1,6 @@
 <svg xmlns="http://www.w3.org/2000/svg" width="1400" height="520" viewBox="0 0 1400 520" role="img" aria-labelledby="title desc">
   <title id="title">BetterCall banner</title>
-  <desc id="desc">BetterCall is a one-line wrapper with five full BFCL remote runs completed; the best run improves from 73.4 percent to 83.8 percent.</desc>
+  <desc id="desc">BetterCall is a one-line wrapper with six full BFCL remote runs completed; the best run improves from 73.4 percent to 83.8 percent.</desc>
   <defs>
     <linearGradient id="background" x1="0" y1="0" x2="1" y2="1">
       <stop offset="0" stop-color="#07111f"/>
@@ -24,6 +24,6 @@
   <g transform="translate(112 112)">
     <text x="0" y="80" font-family="Inter, ui-sans-serif, system-ui, -apple-system, BlinkMacSystemFont, Segoe UI, sans-serif" font-size="78" font-weight="800" fill="#f8fbff">BetterCall</text>
     <rect x="4" y="116" width="430" height="8" rx="4" fill="url(#accent)"/>
-    <text x="0" y="186" font-family="Inter, ui-sans-serif, system-ui, -apple-system, BlinkMacSystemFont, Segoe UI, sans-serif" font-size="34" font-weight="670" fill="#dcecff">Five full BFCL remote runs. Best: 73.4% → 83.8%.</text>
+    <text x="0" y="186" font-family="Inter, ui-sans-serif, system-ui, -apple-system, BlinkMacSystemFont, Segoe UI, sans-serif" font-size="34" font-weight="670" fill="#dcecff">Six full BFCL remote runs. Best: 73.4% → 83.8%.</text>
   </g>
 </svg>

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@botbotgo/better-call",
-  "version": "0.1.4",
+  "version": "0.1.5",
   "description": "LLM tool-call reliability layer.",
   "type": "module",
   "license": "Apache-2.0",