agent402-tollbooth 0.4.3 → 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +73 -37
- package/edge.js +19 -4
- package/index.js +78 -7
- package/package.json +1 -1
- package/pow.js +8 -1
- package/worker.js +61 -8
package/README.md
CHANGED
|
@@ -1,19 +1,19 @@
|
|
|
1
1
|
# agent402-tollbooth
|
|
2
2
|
|
|
3
3
|
**Open-source, self-hostable x402 "pay-per-crawl". Put it in front of any site or
|
|
4
|
-
API: humans browse free, AI crawlers and agents pay per request**
|
|
4
|
+
API: humans browse free, AI crawlers and agents pay per request** - either in
|
|
5
5
|
USDC over the [x402 protocol](https://x402.org), or for free by solving a
|
|
6
6
|
proof-of-work. No Cloudflare, no Stripe, no Merchant-of-Record, no signup.
|
|
7
7
|
|
|
8
8
|
The big platforms are converging on the same model: Cloudflare's
|
|
9
9
|
[pay-per-crawl](https://stackoverflow.blog/2026/02/26/how-pay-per-crawl-is-reshaping-data-monetization/)
|
|
10
10
|
and now its [Monetization Gateway](https://blog.cloudflare.com/monetization-gateway/)
|
|
11
|
-
(launched July 2026
|
|
11
|
+
(launched July 2026 - x402 charging in USDC on Base for anything behind Cloudflare:
|
|
12
12
|
pages, APIs, datasets, MCP tools) confirm that pay-per-request is the business model
|
|
13
13
|
of the agentic web. Tollbooth is the **open-source monetization gateway**: the same
|
|
14
14
|
idea, but MIT-licensed and live today, running in
|
|
15
15
|
front of *any* origin (even a Cloudflare Worker), non-custodial (you hold the wallet,
|
|
16
|
-
no signup or Merchant-of-Record), and
|
|
16
|
+
no signup or Merchant-of-Record), and - the wedge the platform gateways don't offer -
|
|
17
17
|
with a **proof-of-work free tier** so a walletless agent still has a path through.
|
|
18
18
|
Built on the same hardened 402 + proof-of-work machinery as
|
|
19
19
|
[Agent402](https://github.com/MikeyPetrillo/Agent402).
|
|
@@ -25,10 +25,10 @@ npx agent402-tollbooth # then, in the repo: npm run --prefix tollbooth demo
|
|
|
25
25
|
```
|
|
26
26
|
|
|
27
27
|
```text
|
|
28
|
-
agent402-tollbooth
|
|
28
|
+
agent402-tollbooth - live pay-per-crawl demo
|
|
29
29
|
|
|
30
30
|
① A human opens the page (normal browser)
|
|
31
|
-
→ HTTP 200 FREE "📄 The Future of Machine Payments
|
|
31
|
+
→ HTTP 200 FREE "📄 The Future of Machine Payments - full article text…"
|
|
32
32
|
Humans are never charged.
|
|
33
33
|
|
|
34
34
|
② An AI crawler hits the same page (ClaudeBot)
|
|
@@ -38,9 +38,9 @@ agent402-tollbooth — live pay-per-crawl demo
|
|
|
38
38
|
|
|
39
39
|
③ The crawler has no wallet, so it spends CPU instead
|
|
40
40
|
solved in 0.32s (nonce=100208)
|
|
41
|
-
→ HTTP 200 OK (paid via pow) "📄 The Future of Machine Payments
|
|
41
|
+
→ HTTP 200 OK (paid via pow) "📄 The Future of Machine Payments - full article text…"
|
|
42
42
|
|
|
43
|
-
✓ Pay-per-crawl, end to end
|
|
43
|
+
✓ Pay-per-crawl, end to end - humans free, bots pay (USDC or compute).
|
|
44
44
|
```
|
|
45
45
|
|
|
46
46
|
## Install
|
|
@@ -81,12 +81,12 @@ The 402 body advertises both rails:
|
|
|
81
81
|
```
|
|
82
82
|
|
|
83
83
|
A crawler that can't (or won't) pay USDC solves the puzzle and retries with
|
|
84
|
-
`X-Pow-Solution: <token>:<nonce>`
|
|
84
|
+
`X-Pow-Solution: <token>:<nonce>` - sub-second of CPU, single-use, bound to that
|
|
85
85
|
exact URL.
|
|
86
86
|
|
|
87
87
|
## Use it as a reverse proxy (any language/framework)
|
|
88
88
|
|
|
89
|
-
Point it at your existing site
|
|
89
|
+
Point it at your existing site - no code changes there:
|
|
90
90
|
|
|
91
91
|
```bash
|
|
92
92
|
TOLLBOOTH_UPSTREAM=https://your-site.com \
|
|
@@ -97,17 +97,17 @@ npx agent402-tollbooth # listens on :4021, proxies humans free, charges
|
|
|
97
97
|
## Run on the edge (Cloudflare Workers, Next.js, Deno, Bun)
|
|
98
98
|
|
|
99
99
|
The same gate is also built on the Web Crypto + Fetch APIs (`edge.js`), so it runs
|
|
100
|
-
anywhere
|
|
100
|
+
anywhere - no Node required. The gate returns a `402 Response` when the client
|
|
101
101
|
must pay, or `null` to let it through.
|
|
102
102
|
|
|
103
103
|
**Ready-to-deploy templates** (copy a folder, don't assemble from docs):
|
|
104
104
|
|
|
105
|
-
- **Cloudflare Workers** → [`deploy/cloudflare/`](deploy/cloudflare/)
|
|
105
|
+
- **Cloudflare Workers** → [`deploy/cloudflare/`](deploy/cloudflare/) - a ready
|
|
106
106
|
`wrangler.toml` + a 3-step deploy guide (the open pay-per-crawl, on the
|
|
107
107
|
incumbent's own platform).
|
|
108
|
-
- **Next.js / Vercel** → [`deploy/nextjs/`](deploy/nextjs/)
|
|
108
|
+
- **Next.js / Vercel** → [`deploy/nextjs/`](deploy/nextjs/) - a drop-in
|
|
109
109
|
`middleware.js` + a 3-step deploy guide.
|
|
110
|
-
- **Docker** → [`deploy/docker/`](deploy/docker/)
|
|
110
|
+
- **Docker** → [`deploy/docker/`](deploy/docker/) - a `Dockerfile` +
|
|
111
111
|
`docker-compose.yml` to run the reverse proxy in front of any site with
|
|
112
112
|
`docker compose up -d` (includes the live `/__tollbooth` dashboard).
|
|
113
113
|
|
|
@@ -141,29 +141,38 @@ export const config = { matcher: ["/((?!_next/static|_next/image|favicon.ico).*)
|
|
|
141
141
|
|
|
142
142
|
> On the edge, pass a stable `secret` (PoW tokens are HMAC-signed). For
|
|
143
143
|
> single-use replay protection across stateless invocations, supply a `store`
|
|
144
|
-
> (e.g. a Cloudflare KV wrapper
|
|
144
|
+
> (e.g. a Cloudflare KV wrapper - the Worker entry wires this for you).
|
|
145
145
|
|
|
146
146
|
## Accepting USDC (x402)
|
|
147
147
|
|
|
148
148
|
The proof-of-work rail works with **zero config**. To also settle real USDC,
|
|
149
|
-
set `payTo` and supply `verifyX402`
|
|
149
|
+
set `payTo` and supply `verifyX402` - wire it to the standard, audited x402
|
|
150
150
|
server stack (`@x402/express` / your facilitator) rather than reinventing
|
|
151
151
|
settlement:
|
|
152
152
|
|
|
153
153
|
```js
|
|
154
|
+
import { createTollbooth, x402VerifierFromExpress } from "agent402-tollbooth";
|
|
154
155
|
import { paymentMiddleware } from "x402-express"; // or @x402/express
|
|
155
156
|
const x402 = paymentMiddleware(/* your wallet + facilitator config */);
|
|
156
157
|
|
|
157
158
|
app.use(createTollbooth({
|
|
158
159
|
payTo: "0xYourWallet",
|
|
159
160
|
network: "base",
|
|
160
|
-
//
|
|
161
|
-
|
|
162
|
-
|
|
161
|
+
// First-party verifier that OWNS timeout + cancellation: it honors the
|
|
162
|
+
// AbortSignal the gate passes on the SECOND argument (opts.signal), so a slow
|
|
163
|
+
// middleware can't hang the gate or settle after the gate already returned 402.
|
|
164
|
+
verifyX402: x402VerifierFromExpress(x402, { timeoutMs: 9000 }),
|
|
163
165
|
}));
|
|
164
166
|
```
|
|
165
167
|
|
|
166
|
-
|
|
168
|
+
The gate calls `verifyX402(req, opts)` and puts an `AbortSignal` on `opts.signal`
|
|
169
|
+
(aborted when its `TOLLBOOTH_VERIFY_TIMEOUT_MS` fires). `x402VerifierFromExpress`
|
|
170
|
+
honors it and stops treating a late result as valid. Note @x402/express settles
|
|
171
|
+
by broadcasting and has no cancel hook, so keep the timeout above your settle
|
|
172
|
+
latency (the abort is a backstop); a verifier that ignores the signal entirely
|
|
173
|
+
can still charge a buyer who already received a 402. If you write your own
|
|
174
|
+
verifier instead of using the helper, destructure `opts.signal` and reject/stop
|
|
175
|
+
on abort. (PoW is checked first, so an agent without a wallet always has a free path.)
|
|
167
176
|
|
|
168
177
|
## Configuration
|
|
169
178
|
|
|
@@ -186,10 +195,37 @@ app.use(createTollbooth({
|
|
|
186
195
|
| `observe` | `false` | Observe-only: classify and count, but never 402. For pre-launch traffic measurement. |
|
|
187
196
|
| `statsSink` | in-memory | Durable stats backend. Built-ins: `memorySink`, `kvStatsSink(kv)`, `httpStatsSink(url)`. |
|
|
188
197
|
|
|
189
|
-
Environment variables
|
|
190
|
-
|
|
191
|
-
|
|
192
|
-
|
|
198
|
+
### Environment variables
|
|
199
|
+
|
|
200
|
+
Read by the bundled proxy / Express entry point (`index.js`):
|
|
201
|
+
|
|
202
|
+
| env | default | meaning |
|
|
203
|
+
|---|---|---|
|
|
204
|
+
| `TOLLBOOTH_UPSTREAM` | – | Origin the built-in reverse proxy forwards to |
|
|
205
|
+
| `TOLLBOOTH_PAYTO` | – | Wallet address; set to advertise a USDC x402 quote |
|
|
206
|
+
| `TOLLBOOTH_PRICE` | `"$0.001"` | Advertised price per request |
|
|
207
|
+
| `TOLLBOOTH_NETWORK` | `"base"` | x402 network |
|
|
208
|
+
| `TOLLBOOTH_ASSET` | `"USDC"` | Asset symbol in the quote (`USDG` charges in USDG on Robinhood Chain) |
|
|
209
|
+
| `TOLLBOOTH_POW_BITS` | `18` | Proof-of-work difficulty in leading zero bits |
|
|
210
|
+
| `TOLLBOOTH_MODE` | `"bots"` | Who pays: `bots` · `all` · `strict` |
|
|
211
|
+
| `TOLLBOOTH_ADAPTIVE` | `false` | Raise PoW difficulty as charged-request load climbs |
|
|
212
|
+
| `TOLLBOOTH_ADAPTIVE_PER_BIT` | `300` | +1 difficulty bit per N charged requests/min |
|
|
213
|
+
| `TOLLBOOTH_SECRET` | random | HMAC secret binding PoW challenges (set it to survive restarts / run multiple instances) |
|
|
214
|
+
| `TOLLBOOTH_RESOURCE_BASE` | `TOLLBOOTH_UPSTREAM` | Absolute base used for the `resource` field / PoW binding |
|
|
215
|
+
| `TOLLBOOTH_VERIFY_TIMEOUT_MS` | `10000` | Abort an x402 settlement check after this long |
|
|
216
|
+
| `TOLLBOOTH_OBSERVE` | `false` | Observe-only: classify and count, never 402 |
|
|
217
|
+
| `TOLLBOOTH_ADMIN_TOKEN` | – | Token gating the `/__tollbooth` dashboard **and** `/__tollbooth/stats`. Unset logs a warning and leaves the dashboard publicly reachable (aggregate counts only) |
|
|
218
|
+
| `TOLLBOOTH_STATS_TOKEN` | – | Legacy token gating `/__tollbooth/stats` only |
|
|
219
|
+
| `PORT` | `4021` | Listen port |
|
|
220
|
+
|
|
221
|
+
Cloudflare Worker only (`worker.js`, bound in `wrangler.toml`):
|
|
222
|
+
|
|
223
|
+
| binding / env | meaning |
|
|
224
|
+
|---|---|
|
|
225
|
+
| `TOLLBOOTH_REPLAY` | Durable Object binding giving **atomic**, strict single-use PoW replay protection across isolates. Required in enforcing mode |
|
|
226
|
+
| `TOLLBOOTH_ALLOW_NON_ATOMIC_REPLAY` | `"true"` explicitly accepts non-atomic (KV or per-isolate) replay protection instead of a Durable Object. Without it, enforcing mode refuses to start |
|
|
227
|
+
| `TOLLBOOTH_KV` | KV namespace for durable stats |
|
|
228
|
+
| `TOLLBOOTH_STATS_BUCKET` | Stats bucket name within `TOLLBOOTH_KV` (default `"default"`) |
|
|
193
229
|
|
|
194
230
|
## How it decides who pays
|
|
195
231
|
|
|
@@ -199,25 +235,25 @@ Google-Extended, Amazonbot, …). Classic search indexers (Googlebot, Bingbot) a
|
|
|
199
235
|
intentionally **not** charged so your SEO indexing stays free.
|
|
200
236
|
|
|
201
237
|
**Don't want to play whack-a-mole with bot detection?** That's the point of the
|
|
202
|
-
other modes
|
|
238
|
+
other modes - you stop trying to *identify* bots and instead make access *cost
|
|
203
239
|
something*:
|
|
204
240
|
- `mode: "all"` charges every client (except a `free()` match). A "more
|
|
205
|
-
sophisticated" bot gains nothing by disguising itself
|
|
241
|
+
sophisticated" bot gains nothing by disguising itself - everyone pays or solves
|
|
206
242
|
a proof-of-work.
|
|
207
243
|
- `mode: "strict"` charges anything that isn't a real-browser request (browser-like
|
|
208
244
|
UA **and** an HTML `Accept`), letting genuine human page-loads through free.
|
|
209
|
-
Heads-up: that's a heuristic, not a security boundary
|
|
245
|
+
Heads-up: that's a heuristic, not a security boundary - a bot that sets
|
|
210
246
|
`User-Agent: Mozilla/5.0 …` + `Accept: text/html` gets the same free pass a
|
|
211
247
|
human gets. Use `mode: "all"` (or your own `charge:` predicate) for hard
|
|
212
248
|
guarantees.
|
|
213
249
|
- `adaptive: true` makes proof-of-work **harder as load climbs**, so a high-volume
|
|
214
|
-
scraper pays escalating CPU per request regardless of how it looks
|
|
250
|
+
scraper pays escalating CPU per request regardless of how it looks - detection is
|
|
215
251
|
cat-and-mouse, economics isn't.
|
|
216
252
|
|
|
217
253
|
## Observe before charging
|
|
218
254
|
|
|
219
255
|
Don't want to flip a meter on cold? **Run the gate in observe-only mode for a
|
|
220
|
-
week first**
|
|
256
|
+
week first** - every request is still classified (bot vs. human) and counted,
|
|
221
257
|
but nothing ever gets a 402:
|
|
222
258
|
|
|
223
259
|
```js
|
|
@@ -226,8 +262,8 @@ app.use(createTollbooth({ observe: true })); // or: TOLLBOOTH_OBSERVE=true
|
|
|
226
262
|
|
|
227
263
|
On the edge / Cloudflare Worker / Next.js: set `TOLLBOOTH_OBSERVE=true` in env.
|
|
228
264
|
|
|
229
|
-
The dashboard grows a **"Would charge"** counter so you can show your team
|
|
230
|
-
or your client
|
|
265
|
+
The dashboard grows a **"Would charge"** counter so you can show your team -
|
|
266
|
+
or your client - exactly how much of their traffic is AI bots **before** you
|
|
231
267
|
start returning 402s. Removing the flag flips on enforcement with no other
|
|
232
268
|
changes. Bots see a `X-Tollbooth-Observed: would-charge` header in observe mode
|
|
233
269
|
(handy for log filtering); humans see nothing.
|
|
@@ -240,12 +276,12 @@ The middleware keeps aggregate counters (no per-request data):
|
|
|
240
276
|
- `gate.flush()` → flush any buffered deltas to the durable sink (call inside `ctx.waitUntil` on edge runtimes).
|
|
241
277
|
|
|
242
278
|
The reverse-proxy CLI exposes them as JSON at **`/__tollbooth/stats`** and as a
|
|
243
|
-
live **dashboard at `/__tollbooth`**
|
|
279
|
+
live **dashboard at `/__tollbooth`** - requests, how many were charged,
|
|
244
280
|
proof-of-work solves, USDC collected, and what share of your traffic is bots.
|
|
245
281
|
|
|
246
282
|
## Durable stats (survive restart, aggregate across instances)
|
|
247
283
|
|
|
248
|
-
By default, stats live in process memory
|
|
284
|
+
By default, stats live in process memory - fine for single-instance Node,
|
|
249
285
|
useless across multiple replicas or on the edge. Pass a `statsSink` to make
|
|
250
286
|
them survive:
|
|
251
287
|
|
|
@@ -272,7 +308,7 @@ app.use(createTollbooth({
|
|
|
272
308
|
}));
|
|
273
309
|
```
|
|
274
310
|
|
|
275
|
-
Sink interface (build your own
|
|
311
|
+
Sink interface (build your own - e.g. a Cloudflare Durable Object for strict
|
|
276
312
|
consistency):
|
|
277
313
|
|
|
278
314
|
```ts
|
|
@@ -292,7 +328,7 @@ JSON endpoint, BEFORE the gate so they're never paywalled:
|
|
|
292
328
|
- **`/__tollbooth/stats`** → JSON snapshot (gate with `TOLLBOOTH_STATS_TOKEN` for bearer-auth)
|
|
293
329
|
|
|
294
330
|
With a `TOLLBOOTH_KV` namespace bound, the stats aggregate across all isolates
|
|
295
|
-
of all Cloudflare colos serving the Worker
|
|
331
|
+
of all Cloudflare colos serving the Worker - one consistent view.
|
|
296
332
|
|
|
297
333
|
On Next.js / Vercel Edge, middleware can't mount dashboards itself (it'd gate
|
|
298
334
|
them), so a companion **route handler** at `app/__tollbooth/stats/route.js`
|
|
@@ -303,7 +339,7 @@ as drop-in copyable snippets.
|
|
|
303
339
|
## Production checklist (read this)
|
|
304
340
|
|
|
305
341
|
- **Set a stable `TOLLBOOTH_SECRET`.** Required for any multi-process/clustered
|
|
306
|
-
Node deploy and for all edge deploys
|
|
342
|
+
Node deploy and for all edge deploys - without it, proof-of-work tokens use a
|
|
307
343
|
random per-process secret and are rejected across restarts/workers/isolates.
|
|
308
344
|
- **For serverless/edge, supply a durable replay `store`** (e.g. bind a Cloudflare
|
|
309
345
|
KV namespace as `TOLLBOOTH_KV`). The in-memory default is per-isolate, so a
|
|
@@ -312,7 +348,7 @@ as drop-in copyable snippets.
|
|
|
312
348
|
- **The reverse proxy pins the host** to your configured upstream (a client can't
|
|
313
349
|
redirect it elsewhere) and **strips client-forged trust/forwarding headers**
|
|
314
350
|
(`X-Tollbooth-Paid`, `X-Forwarded-Host`, etc.) before forwarding.
|
|
315
|
-
- **UA matching is the default, not a security boundary**
|
|
351
|
+
- **UA matching is the default, not a security boundary** - a bot can forge a
|
|
316
352
|
human UA to get the *same free access a human gets* (it gains nothing more). To
|
|
317
353
|
stop relying on detection entirely, use `mode: "all"` / `mode: "strict"`, and
|
|
318
354
|
turn on `adaptive` so high-volume abuse pays escalating proof-of-work.
|
|
@@ -320,7 +356,7 @@ as drop-in copyable snippets.
|
|
|
320
356
|
## Notes
|
|
321
357
|
|
|
322
358
|
- Proof-of-work tokens are HMAC-signed, expiry-checked, single-use, and bound to
|
|
323
|
-
the exact resource (path + query, dots and all)
|
|
359
|
+
the exact resource (path + query, dots and all) - a solution for one URL can't
|
|
324
360
|
be replayed or reused on another.
|
|
325
361
|
- MIT licensed. Part of [Agent402](https://github.com/MikeyPetrillo/Agent402).
|
|
326
362
|
|
package/edge.js
CHANGED
|
@@ -101,7 +101,14 @@ export function createEdgePow({ secret, difficulty = 18, ttlMs = 5 * 60 * 1000,
|
|
|
101
101
|
if (leadingZeroBits(h) < Number(diffStr)) return { ok: false, reason: "insufficient work" };
|
|
102
102
|
// Atomically claim single-use only AFTER the work is validated, so an invalid
|
|
103
103
|
// attempt never consumes the token and concurrent dupes can't both pass.
|
|
104
|
-
if
|
|
104
|
+
// F05: FAIL CLOSED — if the claim store is unavailable, deny rather than let
|
|
105
|
+
// the exception propagate (or, worse, grant). Strict single-use needs an
|
|
106
|
+
// atomic store (Durable Object); see worker.js durableObjectStore + the
|
|
107
|
+
// wrangler template. A get-then-put KV store is best-effort only.
|
|
108
|
+
let claimed;
|
|
109
|
+
try { claimed = await used.claim(token, Number(expStr)); }
|
|
110
|
+
catch { return { ok: false, reason: "replay store unavailable" }; }
|
|
111
|
+
if (!claimed) return { ok: false, reason: "already used" };
|
|
105
112
|
return { ok: true };
|
|
106
113
|
}
|
|
107
114
|
|
|
@@ -177,11 +184,19 @@ export function createEdgeTollbooth(config = {}) {
|
|
|
177
184
|
if (!shouldCharge(request)) { incr("freeAllowed"); return null; }
|
|
178
185
|
if (observe) { incr("wouldCharge"); return null; }
|
|
179
186
|
const u = new URL(request.url);
|
|
180
|
-
|
|
187
|
+
// F18: bind the canonical ORIGIN, not just path+query — otherwise a proof
|
|
188
|
+
// for /x transfers to any other site sharing the secret. On the edge the
|
|
189
|
+
// request URL already carries the real origin; prefer an explicit
|
|
190
|
+
// resourceBaseUrl when configured.
|
|
191
|
+
const origin = resourceBaseUrl ? resourceBaseUrl.replace(/\/$/, "") : u.origin;
|
|
192
|
+
const resource = origin + u.pathname + u.search;
|
|
193
|
+
// Additionally bind the HTTP METHOD in the PoW challenge (x402 `resource`
|
|
194
|
+
// stays a plain URL for wire compatibility).
|
|
195
|
+
const powResource = `${request.method || "GET"} ${resource}`;
|
|
181
196
|
|
|
182
197
|
const sol = request.headers.get("x-pow-solution");
|
|
183
198
|
if (powEngine && sol) {
|
|
184
|
-
const r = await powEngine.verify(sol,
|
|
199
|
+
const r = await powEngine.verify(sol, powResource);
|
|
185
200
|
if (r.ok) { incr("powSolved"); return null; }
|
|
186
201
|
}
|
|
187
202
|
|
|
@@ -191,7 +206,7 @@ export function createEdgeTollbooth(config = {}) {
|
|
|
191
206
|
message,
|
|
192
207
|
accepts: payTo ? [{ scheme: "exact", network, maxAmountRequired: String(price), asset, payTo, resource }] : [],
|
|
193
208
|
};
|
|
194
|
-
if (powEngine) body.proofOfWork = await powEngine.challenge(
|
|
209
|
+
if (powEngine) body.proofOfWork = await powEngine.challenge(powResource);
|
|
195
210
|
return new Response(JSON.stringify(body), {
|
|
196
211
|
status: 402,
|
|
197
212
|
headers: { "content-type": "application/json", "x-tollbooth": "pay-per-crawl" },
|
package/index.js
CHANGED
|
@@ -23,6 +23,53 @@ export { createPow, leadingZeroBits } from "./pow.js";
|
|
|
23
23
|
export { memorySink, kvStatsSink, httpStatsSink } from "./sinks.js";
|
|
24
24
|
|
|
25
25
|
const VERIFY_TIMEOUT_MS = Number(process.env.TOLLBOOTH_VERIFY_TIMEOUT_MS) || 10_000;
|
|
26
|
+
|
|
27
|
+
// P1.5 / FR4-10: a first-party `verifyX402` built from a standard @x402/express
|
|
28
|
+
// payment middleware, that OWNS timeout + cancellation. The gate calls
|
|
29
|
+
// verifyX402(req, opts) and puts an AbortSignal on `opts.signal` (aborted when
|
|
30
|
+
// the gate's own verify timeout fires); this wrapper honors it — once the signal
|
|
31
|
+
// aborts (or its own optional backstop timeout fires) it settles the verify
|
|
32
|
+
// promise immediately and reports NOT-verified, so a slow middleware can't leave
|
|
33
|
+
// the gate hanging or produce a charged-then-denied result after the gate has
|
|
34
|
+
// returned 402. Grants on next(); denies when the middleware writes a 402.
|
|
35
|
+
//
|
|
36
|
+
// LIMITATION (documented): @x402/express settles by BROADCASTING and exposes no
|
|
37
|
+
// cancel hook, so this wrapper cannot abort an on-chain settle already in flight.
|
|
38
|
+
// Keep TOLLBOOTH_VERIFY_TIMEOUT_MS comfortably above your settle latency so the
|
|
39
|
+
// abort is a backstop, not the normal path; or front it with a facilitator that
|
|
40
|
+
// supports cancellation.
|
|
41
|
+
export function x402VerifierFromExpress(paymentMiddleware, { timeoutMs } = {}) {
|
|
42
|
+
if (typeof paymentMiddleware !== "function") {
|
|
43
|
+
throw new TypeError("x402VerifierFromExpress: paymentMiddleware must be a function");
|
|
44
|
+
}
|
|
45
|
+
return (req, opts = {}) => new Promise((resolve, reject) => {
|
|
46
|
+
const signal = opts?.signal;
|
|
47
|
+
let done = false;
|
|
48
|
+
let timer = null;
|
|
49
|
+
const finish = (fn, v) => {
|
|
50
|
+
if (done) return;
|
|
51
|
+
done = true;
|
|
52
|
+
if (timer) clearTimeout(timer);
|
|
53
|
+
signal?.removeEventListener?.("abort", onAbort);
|
|
54
|
+
fn(v);
|
|
55
|
+
};
|
|
56
|
+
const onAbort = () => finish(resolve, false); // gate timed out → treat as not verified
|
|
57
|
+
if (signal?.aborted) return finish(resolve, false);
|
|
58
|
+
signal?.addEventListener?.("abort", onAbort, { once: true });
|
|
59
|
+
if (timeoutMs > 0) timer = setTimeout(() => finish(resolve, false), timeoutMs);
|
|
60
|
+
// Minimal response shim: the middleware calls next() to GRANT, or writes a
|
|
61
|
+
// 402 via status/json/send/end to DENY.
|
|
62
|
+
const res = {
|
|
63
|
+
setHeader() { return this; }, getHeader() {}, removeHeader() { return this; }, getHeaders() { return {}; },
|
|
64
|
+
status() { return this; }, json() { finish(resolve, false); return this; },
|
|
65
|
+
send() { finish(resolve, false); return this; }, end() { finish(resolve, false); return this; },
|
|
66
|
+
writeHead() { return this; }, write() { return true; }, flushHeaders() {},
|
|
67
|
+
};
|
|
68
|
+
try {
|
|
69
|
+
Promise.resolve(paymentMiddleware(req, res, () => finish(resolve, true))).catch((e) => finish(reject, e));
|
|
70
|
+
} catch (e) { finish(reject, e); }
|
|
71
|
+
});
|
|
72
|
+
}
|
|
26
73
|
// Headers a client must never be able to forge through the proxy: the gate's own
|
|
27
74
|
// trust signals and forwarding/hop-by-hop headers.
|
|
28
75
|
const STRIP_INBOUND = new Set([
|
|
@@ -139,7 +186,15 @@ export function createTollbooth(config = {}) {
|
|
|
139
186
|
const raw = req.originalUrl || req.url || "/";
|
|
140
187
|
let pathAndSearch = raw;
|
|
141
188
|
try { const u = new URL(raw, "http://internal.invalid"); pathAndSearch = u.pathname + u.search; } catch { /* keep raw */ }
|
|
142
|
-
|
|
189
|
+
// F18: bind the canonical ORIGIN, not just path+query, so a proof minted for
|
|
190
|
+
// /x on site A cannot be replayed on site B. Prefer a configured
|
|
191
|
+
// resourceBaseUrl; otherwise derive the origin from the request host (same
|
|
192
|
+
// site => same origin; a different host => a different resource). Reusing one
|
|
193
|
+
// TOLLBOOTH_SECRET across sites should ALSO use a unique secret per site.
|
|
194
|
+
const origin = resourceBaseUrl
|
|
195
|
+
? resourceBaseUrl.replace(/\/$/, "")
|
|
196
|
+
: `${req.protocol || "https"}://${String(req.headers?.host || (req.get && req.get("host")) || "").toLowerCase()}`;
|
|
197
|
+
return origin + pathAndSearch;
|
|
143
198
|
};
|
|
144
199
|
|
|
145
200
|
function tollbooth(req, res, next) {
|
|
@@ -149,6 +204,10 @@ export function createTollbooth(config = {}) {
|
|
|
149
204
|
// measure bot traffic on a live site before turning on enforcement.
|
|
150
205
|
if (observe) { incr("wouldCharge"); res.setHeader("X-Tollbooth-Observed", "would-charge"); return next(); }
|
|
151
206
|
const resource = resourceOf(req);
|
|
207
|
+
// F18: the PoW challenge additionally binds the HTTP METHOD, so a proof
|
|
208
|
+
// solved for GET /x can't be replayed against POST /x. The x402 `resource`
|
|
209
|
+
// stays a plain URL (below) for wire compatibility.
|
|
210
|
+
const powResource = `${(req.method || "GET").toUpperCase()} ${resource}`;
|
|
152
211
|
|
|
153
212
|
const send402 = (extra = {}) => {
|
|
154
213
|
incr("charged");
|
|
@@ -161,14 +220,14 @@ export function createTollbooth(config = {}) {
|
|
|
161
220
|
: [],
|
|
162
221
|
...extra,
|
|
163
222
|
};
|
|
164
|
-
if (powEngine) body.proofOfWork = powEngine.challenge(
|
|
223
|
+
if (powEngine) body.proofOfWork = powEngine.challenge(powResource, difficultyNow());
|
|
165
224
|
res.status(402).json(body);
|
|
166
225
|
};
|
|
167
226
|
|
|
168
227
|
// Free rail: proof-of-work.
|
|
169
228
|
const powHeader = req.headers["x-pow-solution"];
|
|
170
229
|
if (powEngine && powHeader) {
|
|
171
|
-
const r = powEngine.verify(powHeader,
|
|
230
|
+
const r = powEngine.verify(powHeader, powResource);
|
|
172
231
|
if (r.ok) { incr("powSolved"); res.setHeader("X-Tollbooth-Paid", "pow"); return next(); }
|
|
173
232
|
res.setHeader("X-Pow-Error", r.reason);
|
|
174
233
|
}
|
|
@@ -178,13 +237,19 @@ export function createTollbooth(config = {}) {
|
|
|
178
237
|
const payHeader = req.headers["x-payment"] || req.headers["payment-signature"];
|
|
179
238
|
if (payTo && typeof verifyX402 === "function" && payHeader) {
|
|
180
239
|
// Bound verification time so a slow/hung verifier can't exhaust resources.
|
|
181
|
-
|
|
182
|
-
|
|
240
|
+
// F19: pass an AbortSignal and ABORT it on timeout, so a slow verifier that
|
|
241
|
+
// also SETTLES cannot move money after we have already returned 402 (a
|
|
242
|
+
// charged denial). A pure-verification callback may ignore the signal; a
|
|
243
|
+
// settling one MUST cancel/drain in-flight work on it before returning.
|
|
244
|
+
const ac = new AbortController();
|
|
245
|
+
const timeout = new Promise((resolve) => setTimeout(() => { ac.abort(); resolve(false); }, VERIFY_TIMEOUT_MS));
|
|
246
|
+
return Promise.race([Promise.resolve(verifyX402(req, { price, network, asset, payTo, resource, signal: ac.signal })), timeout])
|
|
183
247
|
.then((ok) => {
|
|
248
|
+
ac.abort(); // verification resolved: cancel anything still in flight
|
|
184
249
|
if (ok) { incr("x402Paid"); res.setHeader("X-Tollbooth-Paid", "x402"); return next(); }
|
|
185
250
|
send402();
|
|
186
251
|
})
|
|
187
|
-
.catch(() => { res.setHeader("X-Tollbooth-Error", "x402-verify-failed"); send402(); });
|
|
252
|
+
.catch(() => { ac.abort(); res.setHeader("X-Tollbooth-Error", "x402-verify-failed"); send402(); });
|
|
188
253
|
}
|
|
189
254
|
|
|
190
255
|
return send402();
|
|
@@ -262,7 +327,13 @@ async function startCli() {
|
|
|
262
327
|
if (!_secret) {
|
|
263
328
|
console.warn("⚠ TOLLBOOTH_SECRET not set — proof-of-work tokens use a random per-process secret: they won't survive a restart and will be rejected across multiple workers/instances. Set a stable TOLLBOOTH_SECRET in production.");
|
|
264
329
|
} else if (/^change-me/i.test(_secret) || _secret === "change-me-to-a-long-random-string") {
|
|
265
|
-
|
|
330
|
+
// FATAL, not a warning. The placeholder is published in our own deploy
|
|
331
|
+
// template, so anyone can read it, mint a token with difficulty 0 and walk
|
|
332
|
+
// through the gate without doing any work - the operator believes they are
|
|
333
|
+
// charging while everything is free. A warning scrolls past in a container
|
|
334
|
+
// log; refusing to start cannot be missed, and it fails CLOSED.
|
|
335
|
+
console.error("✖ TOLLBOOTH_SECRET is the public placeholder from the deploy template. It is readable by anyone, so proof-of-work tokens can be forged and your gate bypassed for free. Refusing to start. Generate a real secret: openssl rand -hex 32");
|
|
336
|
+
process.exit(1);
|
|
266
337
|
}
|
|
267
338
|
const { default: express } = await import("express");
|
|
268
339
|
const app = express();
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "agent402-tollbooth",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.5.0",
|
|
4
4
|
"description": "Open-source, self-hostable x402 pay-per-crawl tollbooth: charge AI bots per request (USDC via x402 — or USDG on Robinhood Chain — or free proof-of-work) while humans browse free. Drop-in Express middleware or reverse proxy — no Cloudflare, no Stripe, no signup.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "index.js",
|
package/pow.js
CHANGED
|
@@ -73,8 +73,15 @@ export function createPow({
|
|
|
73
73
|
if (res !== resource) return { ok: false, reason: "wrong resource" };
|
|
74
74
|
if (Date.now() > Number(expStr)) return { ok: false, reason: "expired" };
|
|
75
75
|
if (used.has(token)) return { ok: false, reason: "already used" };
|
|
76
|
+
// Defence in depth: the difficulty rides INSIDE the signed token, so a
|
|
77
|
+
// forged or downgraded token could claim difficulty 0 and pass with no work
|
|
78
|
+
// at all. Hold every solution to the configured difficulty as well, so a
|
|
79
|
+
// leaked or placeholder secret still costs a real solve instead of handing
|
|
80
|
+
// over free passage.
|
|
81
|
+
const claimed = Number(diffStr);
|
|
82
|
+
if (!Number.isFinite(claimed) || claimed < difficulty) return { ok: false, reason: "difficulty below policy" };
|
|
76
83
|
const hash = createHash("sha256").update(`${chal}:${nonce}`).digest();
|
|
77
|
-
if (leadingZeroBits(hash) <
|
|
84
|
+
if (leadingZeroBits(hash) < claimed) return { ok: false, reason: "insufficient work" };
|
|
78
85
|
used.set(token, Number(expStr));
|
|
79
86
|
return { ok: true };
|
|
80
87
|
}
|
package/worker.js
CHANGED
|
@@ -28,9 +28,10 @@ function constEq(a, b) {
|
|
|
28
28
|
}
|
|
29
29
|
|
|
30
30
|
const kvStore = (kv) => ({
|
|
31
|
-
// Best-effort
|
|
32
|
-
//
|
|
33
|
-
//
|
|
31
|
+
// Best-effort claim. KV has no native compare-and-set, so this is get-then-put
|
|
32
|
+
// (eventually consistent) — concurrent dupes across isolates/locations can
|
|
33
|
+
// both observe "unseen" and both pass. For STRICT single-use use the Durable
|
|
34
|
+
// Object store below. Returns true only the first time a token is seen.
|
|
34
35
|
claim: async (k, expMs) => {
|
|
35
36
|
if ((await kv.get(k)) != null) return false;
|
|
36
37
|
await kv.put(k, "1", { expiration: Math.ceil(expMs / 1000) });
|
|
@@ -38,15 +39,67 @@ const kvStore = (kv) => ({
|
|
|
38
39
|
},
|
|
39
40
|
});
|
|
40
41
|
|
|
42
|
+
// F05: strict single-use replay claim via a Durable Object. A DO instance is
|
|
43
|
+
// single-threaded — every request addressed to one DO id is serialized — so the
|
|
44
|
+
// get-then-put INSIDE it is ATOMIC across all isolates and locations (KV is
|
|
45
|
+
// eventually consistent and has no compare-and-set). Each token is routed to its
|
|
46
|
+
// own DO id, so claims only ever contend on the exact token they concern.
|
|
47
|
+
export class TollboothReplay {
|
|
48
|
+
constructor(state) { this.state = state; }
|
|
49
|
+
async fetch(request) {
|
|
50
|
+
let expMs = 0;
|
|
51
|
+
try { expMs = Number((await request.json()).expMs) || 0; } catch { /* */ }
|
|
52
|
+
const already = await this.state.storage.get("claimed");
|
|
53
|
+
if (already != null) return Response.json({ granted: false });
|
|
54
|
+
await this.state.storage.put("claimed", 1);
|
|
55
|
+
// Auto-expire the DO storage at the token's expiry — the token is worthless
|
|
56
|
+
// after that, so the claim record can be dropped and storage stays bounded.
|
|
57
|
+
if (expMs > Date.now()) { try { await this.state.storage.setAlarm(expMs); } catch { /* */ } }
|
|
58
|
+
return Response.json({ granted: true });
|
|
59
|
+
}
|
|
60
|
+
async alarm() { try { await this.state.storage.deleteAll(); } catch { /* */ } }
|
|
61
|
+
}
|
|
62
|
+
|
|
63
|
+
export const durableObjectStore = (namespace) => ({
|
|
64
|
+
claim: async (k, expMs) => {
|
|
65
|
+
const stub = namespace.get(namespace.idFromName(k));
|
|
66
|
+
const resp = await stub.fetch("https://tollbooth-replay/claim", { method: "POST", body: JSON.stringify({ expMs }) });
|
|
67
|
+
const { granted } = await resp.json();
|
|
68
|
+
return granted === true;
|
|
69
|
+
},
|
|
70
|
+
});
|
|
71
|
+
|
|
41
72
|
export default {
|
|
42
73
|
async fetch(request, env, ctx) {
|
|
43
74
|
if (!env.TOLLBOOTH_SECRET) {
|
|
44
75
|
return new Response("Tollbooth misconfigured: set TOLLBOOTH_SECRET (wrangler secret put TOLLBOOTH_SECRET)", { status: 500 });
|
|
45
76
|
}
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
77
|
+
// F05: pick the replay store, strongest first. A Durable Object
|
|
78
|
+
// (TOLLBOOTH_REPLAY) gives ATOMIC, strict single-use across isolates and
|
|
79
|
+
// locations. KV is a best-effort fallback (eventually consistent; concurrent
|
|
80
|
+
// dupes can both pass). With neither, protection is per-isolate only.
|
|
81
|
+
const enforcing = env.TOLLBOOTH_OBSERVE !== "true";
|
|
82
|
+
// FR4-04: FAIL CLOSED in enforcing mode without an ATOMIC replay store. A
|
|
83
|
+
// Durable Object (TOLLBOOTH_REPLAY) gives strict single-use across isolates
|
|
84
|
+
// and locations. KV is eventually consistent (get-then-put, NOT atomic), so
|
|
85
|
+
// concurrent duplicate PoW solutions can both pass — a warning does not
|
|
86
|
+
// protect a production deployment. Refuse to enforce on a non-atomic (or
|
|
87
|
+
// absent) store unless the operator sets an explicit, loud insecure override.
|
|
88
|
+
const allowNonAtomic = env.TOLLBOOTH_ALLOW_NON_ATOMIC_REPLAY === "true";
|
|
89
|
+
if (enforcing && !env.TOLLBOOTH_REPLAY && !allowNonAtomic) {
|
|
90
|
+
return new Response(
|
|
91
|
+
"Tollbooth misconfigured: enforcing mode requires an ATOMIC replay store. Bind a Durable Object as TOLLBOOTH_REPLAY (see deploy/cloudflare/wrangler.toml), run in observe mode (TOLLBOOTH_OBSERVE=true), or accept non-atomic replay explicitly with TOLLBOOTH_ALLOW_NON_ATOMIC_REPLAY=true.",
|
|
92
|
+
{ status: 500 }
|
|
93
|
+
);
|
|
94
|
+
}
|
|
95
|
+
let replayStore;
|
|
96
|
+
if (env.TOLLBOOTH_REPLAY) {
|
|
97
|
+
replayStore = durableObjectStore(env.TOLLBOOTH_REPLAY);
|
|
98
|
+
} else if (env.TOLLBOOTH_KV) {
|
|
99
|
+
replayStore = kvStore(env.TOLLBOOTH_KV);
|
|
100
|
+
if (enforcing) console.warn("agent402-tollbooth: ENFORCING on eventually-consistent KV via explicit TOLLBOOTH_ALLOW_NON_ATOMIC_REPLAY override (get-then-put is NOT atomic — concurrent duplicate solutions across isolates/locations can both pass). Bind a Durable Object as TOLLBOOTH_REPLAY for strict single-use.");
|
|
101
|
+
} else if (enforcing) {
|
|
102
|
+
console.warn("agent402-tollbooth: ENFORCING with NO replay store via explicit TOLLBOOTH_ALLOW_NON_ATOMIC_REPLAY override — proof-of-work replay protection is per-isolate only. Bind a Durable Object for production.");
|
|
50
103
|
}
|
|
51
104
|
// Durable stats live in KV if a namespace is bound. Without it, the dashboard
|
|
52
105
|
// is per-isolate (dies on cold start) — fine for dev, useless for prod.
|
|
@@ -60,7 +113,7 @@ export default {
|
|
|
60
113
|
network: env.TOLLBOOTH_NETWORK || "base",
|
|
61
114
|
asset: env.TOLLBOOTH_ASSET || "USDC",
|
|
62
115
|
powDifficulty: env.TOLLBOOTH_POW_BITS ? Number(env.TOLLBOOTH_POW_BITS) : undefined,
|
|
63
|
-
store:
|
|
116
|
+
store: replayStore,
|
|
64
117
|
observe: env.TOLLBOOTH_OBSERVE === "true",
|
|
65
118
|
statsSink,
|
|
66
119
|
});
|