agent402-tollbooth 0.2.0 → 0.3.0

package/README.md

@@ -175,10 +175,13 @@ app.use(createTollbooth({
 | `free(req)` | – | Custom force-allow predicate (wins over everything) |
 | `verifyX402(req, reqs)` | – | Async USDC settlement check (return `true` to allow) |
 | `resourceBaseUrl` | `""` | Absolute base used for the `resource` field / PoW binding |
+| `observe` | `false` | Observe-only: classify and count, but never 402. For pre-launch traffic measurement. |
+| `statsSink` | in-memory | Durable stats backend. Built-ins: `memorySink`, `kvStatsSink(kv)`, `httpStatsSink(url)`. |
 
 Environment variables: `TOLLBOOTH_UPSTREAM`, `TOLLBOOTH_PAYTO`, `TOLLBOOTH_PRICE`,
 `TOLLBOOTH_NETWORK`, `TOLLBOOTH_POW_BITS`, `TOLLBOOTH_MODE`, `TOLLBOOTH_ADAPTIVE`,
-`TOLLBOOTH_MAX_POW_BITS`, `TOLLBOOTH_ADAPTIVE_PER_BIT`, `TOLLBOOTH_SECRET`, `PORT`.
+`TOLLBOOTH_MAX_POW_BITS`, `TOLLBOOTH_ADAPTIVE_PER_BIT`, `TOLLBOOTH_SECRET`,
+`TOLLBOOTH_OBSERVE`, `TOLLBOOTH_STATS_TOKEN`, `TOLLBOOTH_STATS_BUCKET`, `PORT`.
 
 ## How it decides who pays
 
@@ -199,14 +202,92 @@ something*:
   scraper pays escalating CPU per request regardless of how it looks — detection is
   cat-and-mouse, economics isn't.
 
+## Observe before charging
+
+Don't want to flip a meter on cold? **Run the gate in observe-only mode for a
+week first** — every request is still classified (bot vs. human) and counted,
+but nothing ever gets a 402:
+
+```js
+app.use(createTollbooth({ observe: true })); // or: TOLLBOOTH_OBSERVE=true
+```
+
+On the edge / Cloudflare Worker / Next.js: set `TOLLBOOTH_OBSERVE=true` in env.
+
+The dashboard grows a **"Would charge"** counter so you can show your team —
+or your client — exactly how much of their traffic is AI bots **before** you
+start returning 402s. Removing the flag flips on enforcement with no other
+changes. Bots see a `X-Tollbooth-Observed: would-charge` header in observe mode
+(handy for log filtering); humans see nothing.
+
 ## Analytics
 
-The middleware keeps aggregate counters (no per-request data): `gate.stats()`
-returns `{ requests, freeAllowed, charged, powSolved, x402Paid, difficultyNow }`.
+The middleware keeps aggregate counters (no per-request data):
+- `gate.stats()` → sync, in-process mirror: `{ requests, freeAllowed, wouldCharge, charged, powSolved, x402Paid, difficultyNow, observe }`.
+- `gate.snapshot()` → async, reads from the configured durable sink (defaults to memory).
+- `gate.flush()` → flush any buffered deltas to the durable sink (call inside `ctx.waitUntil` on edge runtimes).
+
 The reverse-proxy CLI exposes them as JSON at **`/__tollbooth/stats`** and as a
 live **dashboard at `/__tollbooth`** — requests, how many were charged,
 proof-of-work solves, USDC collected, and what share of your traffic is bots.
 
+## Durable stats (survive restart, aggregate across instances)
+
+By default, stats live in process memory — fine for single-instance Node,
+useless across multiple replicas or on the edge. Pass a `statsSink` to make
+them survive:
+
+```js
+// Cloudflare Workers: aggregate across all isolates using the same KV namespace
+// that holds the PoW single-use store.
+import { createEdgeTollbooth, kvStatsSink } from "agent402-tollbooth/edge";
+const gate = createEdgeTollbooth({
+  secret: env.TOLLBOOTH_SECRET,
+  statsSink: kvStatsSink(env.TOLLBOOTH_KV, { bucket: "default" }),
+});
+// inside fetch():
+ctx.waitUntil(gate.flush()); // make sure deltas land in KV after the response
+```
+
+```js
+// Any Node deploy: POST batched deltas to a tiny collector (Vercel KV /
+// Upstash / your own API).
+import { createTollbooth, httpStatsSink } from "agent402-tollbooth";
+app.use(createTollbooth({
+  statsSink: httpStatsSink(process.env.TOLLBOOTH_STATS_URL, {
+    token: process.env.TOLLBOOTH_STATS_TOKEN,
+  }),
+}));
+```
+
+Sink interface (build your own — e.g. a Cloudflare Durable Object for strict
+consistency):
+
+```ts
+type StatsSink = {
+  incr(field: string, n?: number): void;        // fire-and-forget
+  flush?(): Promise<void>;                       // optional explicit flush
+  snapshot(): Promise<Record<string, number>>;   // aggregated view
+};
+```
+
+## Edge analytics (Cloudflare Worker / Next.js)
+
+The Cloudflare Worker entry (`worker.js`) auto-mounts both the dashboard and
+JSON endpoint, BEFORE the gate so they're never paywalled:
+
+- **`/__tollbooth`** → live dashboard
+- **`/__tollbooth/stats`** → JSON snapshot (gate with `TOLLBOOTH_STATS_TOKEN` for bearer-auth)
+
+With a `TOLLBOOTH_KV` namespace bound, the stats aggregate across all isolates
+of all Cloudflare colos serving the Worker — one consistent view.
+
+On Next.js / Vercel Edge, middleware can't mount dashboards itself (it'd gate
+them), so a companion **route handler** at `app/__tollbooth/stats/route.js`
+serves the JSON; a static **page** at `app/__tollbooth/page.jsx` renders the
+dashboard HTML. Both are in [`deploy/nextjs/middleware.js`](deploy/nextjs/middleware.js)
+as drop-in copyable snippets.
+
 ## Production checklist (read this)
 
 - **Set a stable `TOLLBOOTH_SECRET`.** Required for any multi-process/clustered

package/dashboard.js

@@ -25,20 +25,33 @@ export function dashboardHtml() {
   a{color:var(--accent)}
 </style></head>
 <body><div class="wrap">
-  <h1><span class="badge">402</span> agent402-tollbooth</h1>
+  <h1><span class="badge">402</span> agent402-tollbooth <span id="mode"></span></h1>
   <div class="sub">Live pay-per-crawl stats · refreshes every 5s · <span id="since"></span></div>
   <div class="grid" id="grid"></div>
-  <footer>Bots are <b id="botpct">—</b> of requests. Aggregate counts only (no per-request data). Raw JSON: <a href="/__tollbooth/stats">/__tollbooth/stats</a>.</footer>
+  <footer><b id="botpct">—</b> of requests were classified as AI bots. Aggregate counts only (no per-request data). Raw JSON: <a href="/__tollbooth/stats">/__tollbooth/stats</a>.</footer>
 </div>
 <script>
-const cards=[["requests","Requests",false],["freeAllowed","Humans (free)",false],["charged","Charged (402)","accent"],["powSolved","Proof-of-work paid","accent"],["x402Paid","USDC paid (x402)","accent"],["difficultyNow","PoW difficulty",false]];
+const cards=[
+  ["requests","Requests",false],
+  ["freeAllowed","Humans (free)",false],
+  ["wouldCharge","Would charge (observe)","accent"],
+  ["charged","Charged (402)","accent"],
+  ["powSolved","Proof-of-work paid","accent"],
+  ["x402Paid","USDC paid (x402)","accent"],
+  ["difficultyNow","PoW difficulty",false]
+];
 async function tick(){
   try{
     const s=await (await fetch("/__tollbooth/stats",{cache:"no-store"})).json();
-    document.getElementById("grid").innerHTML=cards.map(([k,label,acc])=>
-      '<div class="card"><div class="k">'+label+'</div><div class="v '+(acc||"")+'">'+(s[k]??0)+'</div></div>').join("");
-    const pct=s.requests?Math.round((s.charged/s.requests)*100):0;
+    document.getElementById("grid").innerHTML=cards.map(function(c){
+      var k=c[0],label=c[1],acc=c[2];
+      if(k==="wouldCharge" && !(s.wouldCharge>0) && !s.observe) return "";
+      return '<div class="card"><div class="k">'+label+'</div><div class="v '+(acc||"")+'">'+(s[k]??0)+'</div></div>';
+    }).join("");
+    var botish=(s.charged||0)+(s.wouldCharge||0);
+    var pct=s.requests?Math.round((botish/s.requests)*100):0;
     document.getElementById("botpct").textContent=pct+"%";
+    document.getElementById("mode").textContent=s.observe?" · OBSERVE":"";
     if(s.since)document.getElementById("since").textContent="since "+new Date(s.since).toLocaleString();
   }catch(e){/* keep last values */}
 }

package/edge.js

@@ -7,6 +7,9 @@
 //
 // See worker.js for a Cloudflare Worker entry, and the README for Next.js.
 import { makeBotMatcher, AI_BOTS } from "./bots.js";
+import { memorySink } from "./sinks.js";
+
+export { memorySink, kvStatsSink, httpStatsSink } from "./sinks.js";
 
 const te = new TextEncoder();
 
@@ -128,11 +131,22 @@ export function createEdgeTollbooth(config = {}) {
     charge,
     free,
     resourceBaseUrl = "",
+    // Observe-only mode: count what would have been charged but never 402. Lets
+    // operators run the gate on a live site for a week before flipping on enforcement.
+    observe = false,
+    // Pluggable durable-stats sink. Default = in-memory (per-isolate, so on the
+    // edge it dies with the isolate — pass kvStatsSink(env.TOLLBOOTH_KV) for
+    // cross-isolate aggregation).
+    statsSink,
     message = "This resource charges automated / AI clients per request. Humans browse free; bots pay in USDC via x402 or by solving a proof-of-work.",
   } = config;
 
   const isBot = makeBotMatcher(botUserAgents);
   const powEngine = pow ? createEdgePow({ secret, difficulty: powDifficulty, store }) : null;
+  const mem = memorySink();
+  const sink = statsSink || mem;
+  const writeThrough = statsSink && statsSink !== mem;
+  const incr = (k, n = 1) => { mem.incr(k, n); if (writeThrough) sink.incr(k, n); };
 
   const looksHuman = (request) => {
     const ua = request.headers.get("user-agent") || "";
@@ -149,17 +163,20 @@ export function createEdgeTollbooth(config = {}) {
     return isBot(request.headers.get("user-agent") || ""); // "bots" (default)
   };
 
-  return async function gate(request) {
-    if (!shouldCharge(request)) return null;
+  async function gate(request) {
+    incr("requests");
+    if (!shouldCharge(request)) { incr("freeAllowed"); return null; }
+    if (observe) { incr("wouldCharge"); return null; }
     const u = new URL(request.url);
     const resource = (resourceBaseUrl ? resourceBaseUrl.replace(/\/$/, "") : "") + u.pathname + u.search;
 
     const sol = request.headers.get("x-pow-solution");
     if (powEngine && sol) {
       const r = await powEngine.verify(sol, resource);
-      if (r.ok) return null;
+      if (r.ok) { incr("powSolved"); return null; }
     }
 
+    incr("charged");
     const body = {
       error: "Payment Required",
       message,
@@ -170,5 +187,14 @@ export function createEdgeTollbooth(config = {}) {
       status: 402,
       headers: { "content-type": "application/json", "x-tollbooth": "pay-per-crawl" },
     });
-  };
+  }
+
+  // Operator surface: in-process mirror (sync), durable snapshot (async),
+  // and an explicit flush() for edge runtimes — the Worker should call this
+  // inside ctx.waitUntil so buffered KV writes actually happen.
+  gate.observe = observe;
+  gate.stats = () => ({ ...mem.snapshot(), observe });
+  gate.snapshot = async () => ({ ...(await sink.snapshot()), observe });
+  gate.flush = () => Promise.resolve(sink.flush && sink.flush());
+  return gate;
 }

package/index.js

@@ -16,9 +16,11 @@ import { fileURLToPath } from "node:url";
 import { Readable } from "node:stream";
 import { createPow } from "./pow.js";
 import { makeBotMatcher, AI_BOTS } from "./bots.js";
+import { memorySink } from "./sinks.js";
 
 export { AI_BOTS, makeBotMatcher } from "./bots.js";
 export { createPow, leadingZeroBits } from "./pow.js";
+export { memorySink, kvStatsSink, httpStatsSink } from "./sinks.js";
 
 const VERIFY_TIMEOUT_MS = Number(process.env.TOLLBOOTH_VERIFY_TIMEOUT_MS) || 10_000;
 // Headers a client must never be able to forge through the proxy: the gate's own
@@ -44,6 +46,8 @@ const STRIP_INBOUND = new Set([
  * @param {(req, requirements)=>boolean|Promise<boolean>} [config.verifyX402]  USDC settlement check
  * @param {string} [config.resourceBaseUrl]    absolute base for the x402 `resource`/PoW binding
  * @param {string} [config.message]            human-readable note included in the 402
+ * @param {boolean} [config.observe]           observe-only: classify, count, never 402 (deploy a week before enforcing)
+ * @param {object} [config.statsSink]          pluggable durable-stats sink (default: in-memory). See sinks.js.
  */
 export function createTollbooth(config = {}) {
   const {
@@ -72,6 +76,13 @@ export function createTollbooth(config = {}) {
     adaptive = config.adaptive ?? (process.env.TOLLBOOTH_ADAPTIVE === "true"),
     adaptivePerBit = Number(process.env.TOLLBOOTH_ADAPTIVE_PER_BIT) || 300, // +1 bit per N charged req/min
     maxDifficulty,
+    // Observe-only mode: classify every request as charge-vs-free and count it,
+    // but always let it through (never return 402). Use it to measure a site's
+    // bot traffic for a week before flipping the meter on. Off by default.
+    observe = config.observe ?? (process.env.TOLLBOOTH_OBSERVE === "true"),
+    // Pluggable durable-stats sink. Default = in-memory (current behavior).
+    // See sinks.js for kvStatsSink (Cloudflare KV) and httpStatsSink.
+    statsSink,
     message = "This resource charges automated / AI clients per request. Humans browse free; bots pay in USDC via x402 or by solving a proof-of-work.",
   } = config;
 
@@ -79,7 +90,13 @@ export function createTollbooth(config = {}) {
   const powEngine = pow ? createPow({ difficulty: powDifficulty, secret: powSecret }) : null;
 
   // Passive analytics — never affects request handling, just counts what happens.
-  const stats = { since: new Date().toISOString(), requests: 0, freeAllowed: 0, charged: 0, powSolved: 0, x402Paid: 0 };
+  // `mem` is an always-on in-process mirror so `.stats()` stays synchronous for
+  // single-process Node deployments. A durable `statsSink` (KV/HTTP) is written
+  // through alongside and is the source of truth for `.snapshot()`.
+  const mem = memorySink();
+  const sink = statsSink || mem;
+  const writeThrough = statsSink && statsSink !== mem;
+  const incr = (k, n = 1) => { mem.incr(k, n); if (writeThrough) sink.incr(k, n); };
 
   const looksHuman = (req) => {
     const ua = req.headers["user-agent"] || "";
@@ -118,12 +135,15 @@ export function createTollbooth(config = {}) {
   };
 
   function tollbooth(req, res, next) {
-    stats.requests++;
-    if (!shouldCharge(req)) { stats.freeAllowed++; return next(); }
+    incr("requests");
+    if (!shouldCharge(req)) { incr("freeAllowed"); return next(); }
+    // Observe-only: classify as would-charge but let it through. Lets operators
+    // measure bot traffic on a live site before turning on enforcement.
+    if (observe) { incr("wouldCharge"); res.setHeader("X-Tollbooth-Observed", "would-charge"); return next(); }
     const resource = resourceOf(req);
 
     const send402 = (extra = {}) => {
-      stats.charged++;
+      incr("charged");
       if (adaptive) chargedWindow.push(Date.now());
       const body = {
         error: "Payment Required",
@@ -141,7 +161,7 @@ export function createTollbooth(config = {}) {
     const powHeader = req.headers["x-pow-solution"];
     if (powEngine && powHeader) {
       const r = powEngine.verify(powHeader, resource);
-      if (r.ok) { stats.powSolved++; res.setHeader("X-Tollbooth-Paid", "pow"); return next(); }
+      if (r.ok) { incr("powSolved"); res.setHeader("X-Tollbooth-Paid", "pow"); return next(); }
       res.setHeader("X-Pow-Error", r.reason);
     }
 
@@ -153,7 +173,7 @@ export function createTollbooth(config = {}) {
       const timeout = new Promise((resolve) => setTimeout(() => resolve(false), VERIFY_TIMEOUT_MS));
       return Promise.race([Promise.resolve(verifyX402(req, { price, network, asset, payTo, resource })), timeout])
         .then((ok) => {
-          if (ok) { stats.x402Paid++; res.setHeader("X-Tollbooth-Paid", "x402"); return next(); }
+          if (ok) { incr("x402Paid"); res.setHeader("X-Tollbooth-Paid", "x402"); return next(); }
           send402();
         })
         .catch(() => { res.setHeader("X-Tollbooth-Error", "x402-verify-failed"); send402(); });
@@ -164,9 +184,13 @@ export function createTollbooth(config = {}) {
 
   tollbooth.shouldCharge = shouldCharge;
   tollbooth.pow = powEngine;
+  tollbooth.observe = observe;
   // Live counters for operators: how much traffic, how much was charged, and how
   // it was settled. A point-in-time snapshot (never mutated by the caller).
-  tollbooth.stats = () => ({ ...stats, difficultyNow: difficultyNow() });
+  // .stats() is sync (in-process mirror). .snapshot() is async (durable sink).
+  tollbooth.stats = () => ({ ...mem.snapshot(), difficultyNow: difficultyNow(), observe });
+  tollbooth.snapshot = async () => ({ ...(await sink.snapshot()), difficultyNow: difficultyNow(), observe });
+  tollbooth.flush = () => Promise.resolve(sink.flush && sink.flush());
   return tollbooth;
 }
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "agent402-tollbooth",
-  "version": "0.2.0",
+  "version": "0.3.0",
   "description": "Open-source, self-hostable x402 pay-per-crawl tollbooth: charge AI bots per request (USDC via x402, or free proof-of-work) while humans browse free. Drop-in Express middleware or reverse proxy — no Cloudflare, no Stripe, no signup.",
   "type": "module",
   "main": "index.js",
@@ -8,7 +8,13 @@
     "agent402-tollbooth": "index.js"
   },
   "exports": {
-    ".": "./index.js"
+    ".": "./index.js",
+    "./edge": "./edge.js",
+    "./edge.js": "./edge.js",
+    "./worker": "./worker.js",
+    "./worker.js": "./worker.js",
+    "./dashboard.js": "./dashboard.js",
+    "./sinks.js": "./sinks.js"
   },
   "files": [
     "index.js",
@@ -17,6 +23,7 @@
     "bots.js",
     "pow.js",
     "dashboard.js",
+    "sinks.js",
     "example.js",
     "demo.js",
     "README.md"

package/sinks.js

@@ -0,0 +1,148 @@
+// Pluggable stats sinks for the tollbooth — turn the in-memory counter into
+// something that survives restart and aggregates across processes / isolates.
+//
+// A sink is just:
+//   {
+//     incr(field, n=1)   // fire-and-forget; sync from the gate's perspective
+//     flush()            // optional — run at end of request via ctx.waitUntil
+//     snapshot()         // returns the aggregated counts as { field: number, ... }
+//   }
+//
+// Shipped sinks:
+//   memorySink()                       — default, in-process, zero config.
+//   kvStatsSink(kv, opts)              — Cloudflare KV (append-only, eventually consistent).
+//   httpStatsSink(url, opts)           — POST batched deltas to any collector.
+//
+// Want strong consistency on Cloudflare? Back it with a Durable Object that
+// implements the same { incr, snapshot } interface and pass it as `statsSink`.
+
+const ZERO = () => ({
+  requests: 0,
+  freeAllowed: 0,
+  wouldCharge: 0,
+  charged: 0,
+  powSolved: 0,
+  x402Paid: 0,
+});
+
+/** In-process counter. Synchronous; loses state on restart. */
+export function memorySink() {
+  const c = { since: new Date().toISOString(), ...ZERO() };
+  return {
+    incr(field, n = 1) { c[field] = (c[field] || 0) + n; },
+    flush() { return Promise.resolve(); },
+    snapshot() { return { ...c }; },
+  };
+}
+
+/**
+ * Cloudflare KV-backed sink. Writes are append-only delta records under
+ * `tb:stats:${bucket}:*`; snapshot lists + sums them. Bounded by `ttlSeconds`.
+ *
+ * Pattern notes:
+ *  - Cloudflare Workers don't run timers between requests. The gate calls
+ *    `incr()` synchronously into an in-isolate buffer; the Worker entry must
+ *    call `flush()` (typically inside `ctx.waitUntil(...)`) so the buffered
+ *    delta becomes a KV write. Without that, deltas stay in isolate memory
+ *    and die with the isolate.
+ *  - KV has no atomic increment, so we never RMW a shared counter. Each
+ *    flush writes a brand-new key (unique id + ts) — concurrent isolates
+ *    can't lose each other's deltas. Snapshot sums every key under the prefix.
+ *  - Tradeoff: snapshot reads list+get every key, which is fine for low traffic
+ *    (deltas batched per request) but expensive for very high traffic. For
+ *    that, back with a Durable Object instead.
+ */
+export function kvStatsSink(kv, { bucket = "default", ttlSeconds = 60 * 60 * 24 * 30 } = {}) {
+  const prefix = `tb:stats:${bucket}:`;
+  const buf = {};
+  const isDirty = () => Object.keys(buf).length > 0;
+  return {
+    incr(field, n = 1) { buf[field] = (buf[field] || 0) + n; },
+    flush() {
+      if (!isDirty()) return Promise.resolve();
+      const payload = { ...buf, _ts: Date.now() };
+      for (const k of Object.keys(buf)) delete buf[k];
+      const key = prefix + `${Date.now()}-${Math.random().toString(36).slice(2, 10)}`;
+      return Promise.resolve(kv.put(key, JSON.stringify(payload), { expirationTtl: ttlSeconds })).catch(() => {});
+    },
+    async snapshot() {
+      const result = ZERO();
+      let oldest = Infinity;
+      let cursor;
+      do {
+        const page = await kv.list({ prefix, cursor });
+        for (const { name } of page.keys) {
+          let raw;
+          try { raw = await kv.get(name); } catch { continue; }
+          if (!raw) continue;
+          let obj; try { obj = JSON.parse(raw); } catch { continue; }
+          if (typeof obj._ts === "number" && obj._ts < oldest) oldest = obj._ts;
+          for (const [k, n] of Object.entries(obj)) {
+            if (k === "_ts") continue;
+            if (typeof n === "number") result[k] = (result[k] || 0) + n;
+          }
+        }
+        cursor = page.cursor;
+        if (page.list_complete || !cursor) break;
+      } while (cursor);
+      if (oldest !== Infinity) result.since = new Date(oldest).toISOString();
+      return result;
+    },
+  };
+}
+
+/**
+ * HTTP-backed sink. Batched POSTs of `{ incr: { field: n, ... }, ts }`.
+ * GET on the same URL returns the aggregated snapshot — wire that up on your
+ * collector (a tiny Vercel route hander, an Express app, anything).
+ *
+ *   incr(): buffers locally and schedules a flush via setTimeout (Node) or
+ *           returns immediately on the edge (caller should invoke flush()
+ *           inside ctx.waitUntil so the POST survives the response).
+ *   flush(): force-send the current buffer right now (used by edge entries).
+ *   snapshot(): GET the collector URL.
+ *
+ * Auth: pass `token` to include `Authorization: Bearer <token>`.
+ */
+export function httpStatsSink(url, { token, batchMs = 2000, fetchImpl } = {}) {
+  const f = fetchImpl || globalThis.fetch;
+  if (typeof f !== "function") throw new Error("httpStatsSink: no fetch available (pass fetchImpl or run on Node 18+)");
+  const buf = {};
+  let timer = null;
+  const headers = { "content-type": "application/json", ...(token ? { authorization: `Bearer ${token}` } : {}) };
+  async function postBuf() {
+    timer = null;
+    if (Object.keys(buf).length === 0) return;
+    const payload = { ...buf };
+    for (const k of Object.keys(buf)) delete buf[k];
+    try {
+      await f(url, {
+        method: "POST",
+        headers,
+        body: JSON.stringify({ incr: payload, ts: Date.now() }),
+        signal: AbortSignal.timeout(5000),
+      });
+    } catch { /* fire-and-forget: stats loss is acceptable here */ }
+  }
+  return {
+    incr(field, n = 1) {
+      buf[field] = (buf[field] || 0) + n;
+      // Batched flush. setTimeout works on Node; on the edge the caller should
+      // still invoke flush() inside ctx.waitUntil because timers don't fire
+      // after the response is returned.
+      if (!timer && typeof setTimeout === "function") timer = setTimeout(postBuf, batchMs);
+    },
+    flush: postBuf,
+    async snapshot() {
+      try {
+        const r = await f(url, {
+          headers: token ? { authorization: `Bearer ${token}` } : {},
+          signal: AbortSignal.timeout(5000),
+        });
+        if (!r.ok) return ZERO();
+        const j = await r.json();
+        return { ...ZERO(), ...j };
+      } catch { return ZERO(); }
+    },
+  };
+}

package/worker.js

@@ -8,9 +8,13 @@
 //   [vars]
 //   TOLLBOOTH_UPSTREAM = "https://your-origin.example.com"
 //   TOLLBOOTH_PAYTO    = "0xYourWallet"        # optional (advertises USDC quote)
+//   TOLLBOOTH_OBSERVE  = "true"                # optional: observe-only, never 402
+//   TOLLBOOTH_STATS_TOKEN = "any long string"  # optional: gate /__tollbooth/stats
 //   # secret (required): wrangler secret put TOLLBOOTH_SECRET
 //   # optional single-use store: [[kv_namespaces]] binding = "TOLLBOOTH_KV"
-import { createEdgeTollbooth } from "./edge.js";
+//   # ↑ same TOLLBOOTH_KV is reused for durable stats aggregation across isolates.
+import { createEdgeTollbooth, kvStatsSink } from "./edge.js";
+import { dashboardHtml } from "./dashboard.js";
 
 const kvStore = (kv) => ({
   // Best-effort atomic claim. KV has no native compare-and-set, so this is
@@ -24,7 +28,7 @@ const kvStore = (kv) => ({
 });
 
 export default {
-  async fetch(request, env) {
+  async fetch(request, env, ctx) {
     if (!env.TOLLBOOTH_SECRET) {
       return new Response("Tollbooth misconfigured: set TOLLBOOTH_SECRET (wrangler secret put TOLLBOOTH_SECRET)", { status: 500 });
     }
@@ -33,6 +37,11 @@ export default {
       // can be reused across isolates within its TTL. Bind a KV namespace for prod.
       console.warn("agent402-tollbooth: no TOLLBOOTH_KV bound — proof-of-work replay protection is per-isolate only. Bind a KV namespace for production.");
     }
+    // Durable stats live in KV if a namespace is bound. Without it, the dashboard
+    // is per-isolate (dies on cold start) — fine for dev, useless for prod.
+    const statsSink = env.TOLLBOOTH_KV
+      ? kvStatsSink(env.TOLLBOOTH_KV, { bucket: env.TOLLBOOTH_STATS_BUCKET || "default" })
+      : undefined;
     const gate = createEdgeTollbooth({
       secret: env.TOLLBOOTH_SECRET,
       price: env.TOLLBOOTH_PRICE || "$0.001",
@@ -40,9 +49,30 @@ export default {
       network: env.TOLLBOOTH_NETWORK || "base",
       powDifficulty: env.TOLLBOOTH_POW_BITS ? Number(env.TOLLBOOTH_POW_BITS) : undefined,
       store: env.TOLLBOOTH_KV ? kvStore(env.TOLLBOOTH_KV) : undefined,
+      observe: env.TOLLBOOTH_OBSERVE === "true",
+      statsSink,
     });
 
+    // Free, never-gated operator endpoints. Mounted BEFORE the gate so they
+    // can't be paywalled — and so the dashboard polls work even when the rest
+    // of the origin is fully gated.
+    const u = new URL(request.url);
+    if (u.pathname === "/__tollbooth" || u.pathname === "/__tollbooth/") {
+      return new Response(dashboardHtml(), { headers: { "content-type": "text/html; charset=utf-8" } });
+    }
+    if (u.pathname === "/__tollbooth/stats") {
+      // Optional bearer-token gate — share with your monitoring caller.
+      if (env.TOLLBOOTH_STATS_TOKEN) {
+        const got = (request.headers.get("authorization") || "").replace(/^Bearer\s+/i, "");
+        if (got !== env.TOLLBOOTH_STATS_TOKEN) return new Response("unauthorized", { status: 401 });
+      }
+      const snap = await gate.snapshot();
+      return new Response(JSON.stringify(snap), { headers: { "content-type": "application/json" } });
+    }
+
     const blocked = await gate(request);
+    // Flush any buffered stats to KV after we've replied — survives the response.
+    if (ctx && typeof ctx.waitUntil === "function") ctx.waitUntil(gate.flush());
     if (blocked) return blocked;
 
     // Allowed → proxy to the origin.