@profullstack/x402-gateway 0.3.1 → 0.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -102,6 +102,61 @@ export const GET = robotsRoute(gateway, { disallow: ['/login', '/api/'] });
102
102
 
103
103
  Without `coinpay.apiKey` and `payTo` the gateway still answers training crawlers with 402 and the page says payments are off. Nothing is sold, but nothing is given away either.
104
104
 
105
+ ## A free allowance that sells instead of refusing
106
+
107
+ Named crawlers are one problem. The other is an ordinary client hammering the
108
+ site far past what a reader would. The usual answer is `429 Too Many Requests`,
109
+ which tells them to go away. That is the right answer when you have nothing to
110
+ sell. Here you do.
111
+
112
+ ```js
113
+ createGateway({
114
+ siteUrl: 'https://your-site.com',
115
+ coinpay: { apiKey: process.env.COINPAY_X402_KEY },
116
+ payTo: process.env.CRAWL_PAY_TO,
117
+
118
+ freeQuota: 100, // 100 requests a minute, free, no account
119
+ benefits: [
120
+ 'No rate limit',
121
+ 'Bulk export endpoint',
122
+ 'Every field, not just the summary',
123
+ ],
124
+ });
125
+ ```
126
+
127
+ Past the allowance the gateway answers `402` with the offer, the RateLimit
128
+ headers, and a body that says what ran out, when it comes back, and what a pass
129
+ costs. The moment a caller runs out of free requests is the best sales pitch the
130
+ site will ever get: it has just demonstrated it wants more than the free tier
131
+ and is still holding the request.
132
+
133
+ `freeQuota` takes `{ requests, windowSeconds, paths, identify, store }`. A pass
134
+ is checked before the allowance, so a paying caller is never metered. The sales
135
+ page, `robots.txt` and `.well-known/` stay reachable when the allowance is gone,
136
+ because being unable to reach the page that sells the fix would be the worst
137
+ possible failure of a throttle that exists to sell something.
138
+
139
+ ### On rotating addresses to get around it
140
+
141
+ The default identity is the caller's address, and a proxy rotation defeats it.
142
+ That is not a hole to be patched, because the arithmetic already argues for
143
+ paying:
144
+
145
+ - Residential proxy bandwidth is sold **by the gigabyte**, and a crawl big
146
+ enough to be worth rotating for passes a dollar on the first day.
147
+ - A rotation still fetches every page one at a time. It buys no speed.
148
+ - A pass is a flat price with nothing to maintain and nothing to keep working.
149
+
150
+ So the throttled page makes that case in as many words rather than pretending to
151
+ be undefeatable. Detection is a race you re-run every time someone changes
152
+ tactics. Price is not: the better your free tier and the clearer your paid one,
153
+ the less anyone bothers. Set `benefits` to the things a rotation genuinely
154
+ cannot get, a bulk endpoint above all, and evasion stops being worth the effort
155
+ rather than being blocked.
156
+
157
+ Where you *can* identify a caller properly, do: pass `identify` and key on an API
158
+ key or an account, and the allowance becomes exact.
159
+
105
160
  ## Crawlers that do not say who they are
106
161
 
107
162
  The lists catch crawlers that name themselves. Two do not: a VPS fleet wearing a browser string, and a residential-proxy rotation cycling a few Chrome strings across hundreds of addresses. `denyCidrs` handles the first (`['51.38.0.0/16', '54.38.0.0/16', …]` for one provider's ranges); `chargeSpoofedBrowsers` handles both by asking a question only a browser can answer. A request that answers it is left alone. One that cannot gets the same 402 as GPTBot, which costs the site a hash instead of a render.
package/index.d.ts CHANGED
@@ -74,9 +74,51 @@ export interface GatewayOptions {
74
74
  contact?: string;
75
75
  /** Awaited before the buyer's receipt is sent, and its errors are swallowed: a sale is recorded, and a recording failure never costs the buyer the pass. */
76
76
  onSale?: (sale: Sale) => void | Promise<void>;
77
+ /**
78
+ * A free allowance for ordinary callers, after which the gateway answers 402
79
+ * with the offer instead of letting the request through. A bare number means
80
+ * that many requests per minute. A valid pass is never metered.
81
+ */
82
+ freeQuota?: number | FreeQuotaOptions;
83
+ /** What a pass unlocks, listed on the sales page and in the 402 body. */
84
+ benefits?: string[];
77
85
  fetch?: typeof fetch;
78
86
  }
79
87
 
88
+ export interface QuotaHit {
89
+ /** Requests in the current window, including this one. */
90
+ count: number;
91
+ /** Seconds until the window rolls over. */
92
+ resetSeconds: number;
93
+ }
94
+
95
+ /** Somewhere to count requests. Supply one to share an allowance across a fleet. */
96
+ export interface QuotaStore {
97
+ hit(key: string, windowSeconds: number): QuotaHit | Promise<QuotaHit>;
98
+ }
99
+
100
+ export interface FreeQuotaOptions {
101
+ /** Free requests per window. */
102
+ requests: number;
103
+ /** Window length. Default 60. */
104
+ windowSeconds?: number;
105
+ /**
106
+ * What to count against. Defaults to the caller's address, which a rotation
107
+ * defeats; that is answered with price rather than detection, see the README.
108
+ */
109
+ identify?: (request: Request) => string | null;
110
+ /** Default: an in-process counter, so each instance grants its own allowance. */
111
+ store?: QuotaStore;
112
+ /** Only meter these paths or prefixes. Default: everything the gate sees. */
113
+ paths?: string[];
114
+ }
115
+
116
+ /** An in-process fixed-window counter. */
117
+ export function memoryQuotaStore(options?: {
118
+ now?: () => number;
119
+ sweepEvery?: number;
120
+ }): QuotaStore & { sweep(): void; readonly size: number };
121
+
80
122
  export interface AcceptEntry {
81
123
  scheme: 'exact';
82
124
  network: string;
@@ -107,9 +149,24 @@ export interface RobotsOptions {
107
149
  comments?: string[];
108
150
  }
109
151
 
152
+ /** What the caller spent against an allowance, as `spend()` reports it. */
153
+ export interface QuotaUsage {
154
+ count: number;
155
+ remaining: number;
156
+ resetSeconds: number;
157
+ overLimit: boolean;
158
+ }
159
+
160
+ /** The allowance a 402 should quote, when the caller kept the count itself. */
161
+ export interface SellContext {
162
+ usage?: QuotaUsage | null;
163
+ /** Defaults to the gateway's own `freeQuota`. */
164
+ quota?: { requests: number; windowSeconds: number } | null;
165
+ }
166
+
110
167
  export interface Gateway {
111
168
  handle: Handle;
112
- sell: (request: Request) => Promise<Response>;
169
+ sell: (request: Request, context?: SellContext) => Promise<Response>;
113
170
  enabled: boolean;
114
171
  options: Required<Omit<GatewayOptions, 'onSale' | 'fetch' | 'page' | 'isPaidAgent'>> & {
115
172
  onSale: GatewayOptions['onSale'] | null;
@@ -119,6 +176,10 @@ export interface Gateway {
119
176
  };
120
177
  robotsTxt: (extra?: RobotsOptions) => string;
121
178
  page: () => string;
179
+ /** The pass this request presents, from the gateway's header or a bearer token. */
180
+ passFrom: (request: Request) => string | null;
181
+ /** Whether that token is a live pass this gateway minted. */
182
+ verifyPass: (token: string | null) => Promise<boolean>;
122
183
  }
123
184
 
124
185
  export function createGateway(options: GatewayOptions): Gateway;
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@profullstack/x402-gateway",
3
- "version": "0.3.1",
3
+ "version": "0.5.0",
4
4
  "type": "module",
5
5
  "description": "Sell crawl access to AI training crawlers by the day over x402, settled by CoinPay. One middleware: 402 with an offer, a sales page with CLI instructions, signed passes, and a robots.txt that keeps search crawlers welcome.",
6
6
  "keywords": [
@@ -20,22 +20,52 @@
20
20
  "middleware",
21
21
  "paywall"
22
22
  ],
23
- "repository": { "type": "git", "url": "git+https://github.com/profullstack/x402-gateway.git" },
23
+ "repository": {
24
+ "type": "git",
25
+ "url": "git+https://github.com/profullstack/x402-gateway.git"
26
+ },
24
27
  "homepage": "https://github.com/profullstack/x402-gateway#readme",
25
- "bugs": { "url": "https://github.com/profullstack/x402-gateway/issues" },
28
+ "bugs": {
29
+ "url": "https://github.com/profullstack/x402-gateway/issues"
30
+ },
26
31
  "license": "MIT",
27
32
  "author": "Profullstack, LLC",
28
33
  "exports": {
29
- ".": { "types": "./index.d.ts", "import": "./src/index.js" },
30
- "./hono": { "types": "./index.d.ts", "import": "./src/hono.js" },
31
- "./next": { "types": "./index.d.ts", "import": "./src/next.js" },
32
- "./robots": { "types": "./index.d.ts", "import": "./src/robots.js" },
33
- "./agents": { "types": "./index.d.ts", "import": "./src/agents.js" },
34
- "./edge": { "types": "./index.d.ts", "import": "./src/edge.js" }
34
+ ".": {
35
+ "types": "./index.d.ts",
36
+ "import": "./src/index.js"
37
+ },
38
+ "./hono": {
39
+ "types": "./index.d.ts",
40
+ "import": "./src/hono.js"
41
+ },
42
+ "./next": {
43
+ "types": "./index.d.ts",
44
+ "import": "./src/next.js"
45
+ },
46
+ "./robots": {
47
+ "types": "./index.d.ts",
48
+ "import": "./src/robots.js"
49
+ },
50
+ "./agents": {
51
+ "types": "./index.d.ts",
52
+ "import": "./src/agents.js"
53
+ },
54
+ "./edge": {
55
+ "types": "./index.d.ts",
56
+ "import": "./src/edge.js"
57
+ }
35
58
  },
36
59
  "types": "./index.d.ts",
37
- "files": ["src", "index.d.ts", "README.md", "LICENSE"],
38
- "engines": { "node": ">=20.11" },
60
+ "files": [
61
+ "src",
62
+ "index.d.ts",
63
+ "README.md",
64
+ "LICENSE"
65
+ ],
66
+ "engines": {
67
+ "node": ">=20.11"
68
+ },
39
69
  "sideEffects": false,
40
70
  "scripts": {
41
71
  "test": "node --test"
package/src/index.js CHANGED
@@ -1,5 +1,6 @@
1
1
  import { isTrainingAgent, RETRIEVAL_AGENTS, TRAINING_AGENTS } from './agents.js';
2
2
  import { clientIp, compileCidrs, inCidrs, isSpoofedBrowser } from './edge.js';
3
+ import { memoryQuotaStore, meters, normaliseQuota, quotaHeaders, spend } from './quota.js';
3
4
  import { renderPage } from './page.js';
4
5
  import { mintPass, readPass } from './pass.js';
5
6
  import { robotsTxt } from './robots.js';
@@ -20,6 +21,7 @@ export { renderPage } from './page.js';
20
21
  export { mintPass, readPass } from './pass.js';
21
22
  export { robotsTxt } from './robots.js';
22
23
  export { buildOffer, decodePayment, expectedFor, METHODS, verifyAndSettle } from './x402.js';
24
+ export { memoryQuotaStore, normaliseQuota, quotaHeaders, spend } from './quota.js';
23
25
 
24
26
  /**
25
27
  * A gateway that sells crawl access to training crawlers, by the day, over x402.
@@ -119,10 +121,23 @@ export function createGateway(options = {}) {
119
121
  status,
120
122
  headers: { 'content-type': 'application/json; charset=utf-8', ...noStore, ...headers },
121
123
  });
122
- const html = (body, status) =>
123
- new Response(body, { status, headers: { 'content-type': 'text/html; charset=utf-8', ...noStore } });
124
+ const html = (body, status, headers = {}) =>
125
+ new Response(body, {
126
+ status,
127
+ headers: { 'content-type': 'text/html; charset=utf-8', ...noStore, ...headers },
128
+ });
124
129
 
125
- const pageCtx = (days = 1) => ({
130
+ const pageCtx = (days = 1, usage = null, quota = o.freeQuota) => ({
131
+ quota: quota
132
+ ? {
133
+ requests: quota.requests,
134
+ windowSeconds: quota.windowSeconds,
135
+ used: usage?.count ?? null,
136
+ resetSeconds: usage?.resetSeconds ?? null,
137
+ exceeded: Boolean(usage?.overLimit),
138
+ }
139
+ : null,
140
+ benefits: o.benefits,
126
141
  days,
127
142
  total: money(o.priceCents * days),
128
143
  siteName: o.siteName,
@@ -155,10 +170,22 @@ export function createGateway(options = {}) {
155
170
  * body and not the headers, and a crawler that wanted the page can fetch it
156
171
  * again a moment later with the pass.
157
172
  */
158
- async function sell(request) {
173
+ async function sell(request, context = {}) {
159
174
  const ua = request.headers.get('user-agent') ?? '';
160
175
  const proofHeader = request.headers.get('x-payment');
161
176
  const asked = daysFrom(request);
177
+ // Present when the free allowance is what stopped this request, rather than
178
+ // the crawler lists. It changes what the 402 says, not what it costs.
179
+ const usage = context.usage ?? null;
180
+ /*
181
+ * The allowance the caller was measured against. Defaults to this
182
+ * gateway's own, but a caller that keeps its own counter -- an app-wide
183
+ * throttle metering every route, not just the crawler lists -- hands its
184
+ * own in, so the 402 quotes the limit that actually stopped the request
185
+ * rather than one the gateway happens to hold.
186
+ */
187
+ const quota = context.quota ?? o.freeQuota;
188
+ const rateHeaders = usage && quota ? quotaHeaders(quota, usage) : {};
162
189
 
163
190
  if (proofHeader) {
164
191
  if (!enabled) return json(receipt(asked, { error: 'Payments are not switched on here.' }), 402);
@@ -252,7 +279,33 @@ export function createGateway(options = {}) {
252
279
  );
253
280
  }
254
281
 
255
- if (wantsHtml(request.headers.get('accept'))) return html(o.page(pageCtx(asked)), 402);
282
+ if (wantsHtml(request.headers.get('accept'))) {
283
+ return html(o.page(pageCtx(asked, usage, quota)), 402, rateHeaders);
284
+ }
285
+
286
+ if (usage && quota) {
287
+ // Say what ran out, when it comes back, and what a pass costs, in that
288
+ // order. A caller reading this is deciding between waiting, rotating
289
+ // addresses, and paying, and the numbers are the argument.
290
+ return json(
291
+ receipt(asked, {
292
+ error:
293
+ `Free allowance used: ${quota.requests} requests per ` +
294
+ `${quota.windowSeconds}s. It resets in ${usage.resetSeconds}s. ` +
295
+ `A pass removes the limit for ${price} a day.`,
296
+ quota: {
297
+ requests: quota.requests,
298
+ windowSeconds: quota.windowSeconds,
299
+ used: usage.count,
300
+ resetSeconds: usage.resetSeconds,
301
+ },
302
+ ...(o.benefits ? { unlocks: o.benefits } : {}),
303
+ }),
304
+ 402,
305
+ rateHeaders,
306
+ );
307
+ }
308
+
256
309
  return json(receipt(asked, { error: `Payment required for training crawlers. Read ${buyUrl} for how.` }), 402);
257
310
  }
258
311
 
@@ -280,15 +333,41 @@ export function createGateway(options = {}) {
280
333
  const path = new URL(request.url).pathname;
281
334
  if (path === o.path) return sell(request);
282
335
  if (o.exempt && o.exempt(request)) return null;
336
+
337
+ /*
338
+ * A valid pass is checked before anything else that could refuse, because
339
+ * a pass is the thing being sold: whoever holds one is neither charged as
340
+ * a crawler nor metered against the free allowance. It used to be read
341
+ * only after the crawler lists matched, which was fine while the lists
342
+ * were the only reason to refuse and is not now that a quota exists.
343
+ */
344
+ const token = passFrom(request);
345
+ const paid = Boolean(token && (await readPass(token, { secret })));
346
+ if (paid) return null;
347
+
283
348
  const pays =
284
349
  o.isPaidAgent(request.headers.get('user-agent') ?? '') ||
285
350
  (o.chargeSpoofedBrowsers && isSpoofedBrowser(request));
286
- if (!pays) return null;
351
+ if (pays && !isOpen(path)) return sell(request);
287
352
  if (isOpen(path)) return null;
288
353
 
289
- const token = passFrom(request);
290
- if (token && (await readPass(token, { secret }))) return null;
291
- return sell(request);
354
+ /*
355
+ * Everyone else gets the free allowance. Running out is answered with a
356
+ * price rather than a 429: the caller has just shown it wants more than
357
+ * the free tier and is still holding the request, which is the best moment
358
+ * this site will ever get to sell it a pass.
359
+ */
360
+ if (o.freeQuota && meters(o.freeQuota, path)) {
361
+ const key = o.freeQuota.identify ? o.freeQuota.identify(request) : clientIp(request);
362
+ const usage = await spend(o.freeQuota, key);
363
+ // Under the limit the request carries on untouched. `handle` answers with
364
+ // a Response or nothing at all, and quietly growing that contract to
365
+ // smuggle headers out would break every adapter that checks it for truth.
366
+ // The allowance is advertised on the 402, which is where it is read.
367
+ if (usage?.overLimit) return sell(request, { usage });
368
+ }
369
+
370
+ return null;
292
371
  }
293
372
 
294
373
  return {
@@ -296,6 +375,19 @@ export function createGateway(options = {}) {
296
375
  sell,
297
376
  enabled,
298
377
  options: o,
378
+ /**
379
+ * The pass a request presents, and whether it is one this gateway minted
380
+ * and still honours.
381
+ *
382
+ * Exposed because the pass has to be honoured by everything that could
383
+ * refuse a request, not only by `handle`. An app-wide throttle that meters
384
+ * every route has to skip whoever already paid, and recomputing the
385
+ * signing-secret fallback (`secret || coinpay.apiKey`) on its side is
386
+ * exactly the kind of duplicate rule that drifts and starts charging
387
+ * paying crawlers twice.
388
+ */
389
+ passFrom,
390
+ verifyPass: async (token) => Boolean(token && (await readPass(token, { secret }))),
299
391
  /** robots.txt with this gateway's lists and sales path. */
300
392
  robotsTxt: (extra = {}) =>
301
393
  robotsTxt({ siteUrl: o.siteUrl, path: o.path, training: o.training, retrieval: o.retrieval, ...extra }),
@@ -372,6 +464,8 @@ function normalise(options) {
372
464
  page: options.page ?? renderPage,
373
465
  contact: options.contact ?? '',
374
466
  onSale: options.onSale ?? null,
467
+ freeQuota: normaliseQuota(options.freeQuota),
468
+ benefits: Array.isArray(options.benefits) ? options.benefits : null,
375
469
  fetch: options.fetch ?? globalThis.fetch,
376
470
  };
377
471
  }
package/src/page.js CHANGED
@@ -58,7 +58,43 @@ export function renderPage(ctx) {
58
58
  days = 1,
59
59
  total = price,
60
60
  maxDays = 30,
61
+ quota = null,
62
+ benefits = null,
61
63
  } = ctx;
64
+
65
+ /*
66
+ * Two audiences reach this page and they need different first sentences. A
67
+ * training crawler is here because it is on a list. A heavy reader is here
68
+ * because it ran out of the free allowance, and telling that one it is a
69
+ * training crawler is both wrong and insulting. The price is the same; the
70
+ * argument is not.
71
+ */
72
+ const throttled = Boolean(quota?.exceeded);
73
+ const headline = throttled
74
+ ? 'You have used up the free allowance.'
75
+ : 'Training crawlers pay for access here.';
76
+ const opening = throttled
77
+ ? `<p class="mut">${esc(String(quota.requests))} requests every ${esc(String(quota.windowSeconds))} seconds are free, no key and no account, and that is not changing. ` +
78
+ `You have gone past it${quota.resetSeconds ? `, and it resets in ${esc(String(quota.resetSeconds))} seconds` : ''}. ` +
79
+ `A pass lifts the limit rather than waiting it out.</p>`
80
+ : `<p class="mut">People read <a href="${esc(siteUrl)}">${esc(siteName)}</a> free. So do search engines and the retrieval crawlers behind AI answers, because they send readers back. A crawler that copies pages into a training corpus sends nobody back, so it pays for the time it spends.</p>`;
81
+
82
+ /*
83
+ * Said plainly because it is the honest argument and the one that actually
84
+ * lands. Rotating addresses to dodge a free-tier limit is not free: the
85
+ * bandwidth is metered and billed by the gigabyte, and a crawl big enough to
86
+ * be worth rotating for costs more in proxies than the pass does. Anyone
87
+ * weighing the two should be able to see that from here.
88
+ */
89
+ const arithmetic = throttled
90
+ ? `<h2>Before you reach for a proxy pool</h2>
91
+ <p class="mut">Spreading the same crawl over rotating addresses works, and it is the expensive way to do this. Residential bandwidth is sold by the gigabyte, you still fetch every page one at a time, and the bill starts on the first day. A pass is ${esc(price)} a day, flat, with no rotation to maintain and nothing to keep working. We would rather sell you access than play that game, which is why the limit answers with a price instead of a refusal.</p>`
92
+ : '';
93
+
94
+ const unlocks =
95
+ benefits && benefits.length
96
+ ? `<h2>What a pass gets you</h2>\n<ul>\n${benefits.map((b) => ` <li>${esc(b)}</li>`).join('\n')}\n</ul>`
97
+ : '';
62
98
  const window =
63
99
  minutes === 1440
64
100
  ? 'one day'
@@ -82,8 +118,8 @@ export function renderPage(ctx) {
82
118
  </head>
83
119
  <body>
84
120
  <main>
85
- <h1>Training crawlers pay for access here.</h1>
86
- <p class="mut">People read <a href="${esc(siteUrl)}">${esc(siteName)}</a> free. So do search engines and the retrieval crawlers behind AI answers, because they send readers back. A crawler that copies pages into a training corpus sends nobody back, so it pays for the time it spends.</p>
121
+ <h1>${esc(headline)}</h1>
122
+ ${opening}
87
123
 
88
124
  <div class="price">${esc(days > 1 ? total : price)} <span class="mut" style="font-size:1rem;font-weight:400">for ${esc(days > 1 ? `${days} × ${window}` : window)} of requests</span></div>
89
125
  ${
@@ -97,6 +133,9 @@ ${
97
133
  : '<p><strong>Payments are not switched on here yet.</strong> The offer below is empty until the operator configures a payout address, so for now this crawler is simply refused.</p>'
98
134
  }
99
135
 
136
+ ${unlocks}
137
+ ${arithmetic}
138
+
100
139
  <h2>How it works</h2>
101
140
  <ol>
102
141
  <li>Any page you fetch answers <code>402 Payment Required</code>. This page, fetched with <code>Accept: application/json</code>, returns the x402 offer: USDC, <code>exact</code> scheme, on ${esc(networks || 'Base, Polygon or Ethereum')}.</li>
package/src/quota.js ADDED
@@ -0,0 +1,149 @@
1
+ /**
2
+ * A free allowance, and what happens when it runs out.
3
+ *
4
+ * The usual answer to "too many requests" is 429, which tells a caller to go
5
+ * away and try later. That is the right answer when you have nothing to sell.
6
+ * Here we do: the moment the allowance runs out is the best sales pitch the
7
+ * site will ever get, because the caller has just demonstrated it wants more
8
+ * than the free tier and is still holding the request. So the gateway answers
9
+ * 402 with a price instead.
10
+ *
11
+ * ON EVADING THIS BY ROTATING ADDRESSES.
12
+ *
13
+ * The default identity is the caller's address, and rotating addresses defeats
14
+ * it. That is not a flaw to be patched, because the arithmetic already argues
15
+ * for paying: residential proxy bandwidth is sold by the gigabyte at prices
16
+ * that pass a dollar inside the first day of any serious crawl, and a rotation
17
+ * still has to fetch every page one at a time. Someone who spends more on
18
+ * proxies than the pass costs, in order to avoid the pass, has not been
19
+ * defeated by cleverness. They have made an arithmetic mistake, and the 402
20
+ * body is where we point it out. Detection is a race. Price is not.
21
+ *
22
+ * The store is pluggable so a fleet can share one counter across sites. The
23
+ * built-in one counts per process, so several instances each grant their own
24
+ * allowance; that errs toward generosity, which is the right direction to be
25
+ * wrong in for a free tier.
26
+ */
27
+
28
+ /**
29
+ * @typedef {object} QuotaHit
30
+ * @property {number} count requests in the current window, including this one
31
+ * @property {number} resetSeconds seconds until the window rolls over
32
+ */
33
+
34
+ /**
35
+ * @typedef {object} QuotaStore
36
+ * @property {(key: string, windowSeconds: number) => QuotaHit | Promise<QuotaHit>} hit
37
+ */
38
+
39
+ /**
40
+ * A counter in this process's memory.
41
+ *
42
+ * Fixed windows rather than a sliding log: a sliding window is more accurate
43
+ * and has to keep every timestamp, and for deciding whether to show someone a
44
+ * price that accuracy is not worth the memory. Expired entries are swept
45
+ * occasionally on write, so an idle key does not cost anything for long.
46
+ */
47
+ export function memoryQuotaStore({ now = () => Date.now(), sweepEvery = 1000 } = {}) {
48
+ /** @type {Map<string, {count: number, expiresAt: number}>} */
49
+ const windows = new Map();
50
+ let writes = 0;
51
+
52
+ const sweep = (at) => {
53
+ for (const [key, entry] of windows) {
54
+ if (entry.expiresAt <= at) windows.delete(key);
55
+ }
56
+ };
57
+
58
+ return {
59
+ hit(key, windowSeconds) {
60
+ const at = now();
61
+ writes += 1;
62
+ if (writes % sweepEvery === 0) sweep(at);
63
+
64
+ const entry = windows.get(key);
65
+ if (!entry || entry.expiresAt <= at) {
66
+ windows.set(key, { count: 1, expiresAt: at + windowSeconds * 1000 });
67
+ return { count: 1, resetSeconds: windowSeconds };
68
+ }
69
+ entry.count += 1;
70
+ return {
71
+ count: entry.count,
72
+ resetSeconds: Math.max(1, Math.ceil((entry.expiresAt - at) / 1000)),
73
+ };
74
+ },
75
+ /** Test seam, and a way for a long-lived process to reclaim memory on demand. */
76
+ sweep() {
77
+ sweep(now());
78
+ },
79
+ get size() {
80
+ return windows.size;
81
+ },
82
+ };
83
+ }
84
+
85
+ /**
86
+ * Normalise what a site passes as `freeQuota`.
87
+ *
88
+ * A bare number means "that many per minute", because per minute is how
89
+ * everyone states a rate limit out loud.
90
+ */
91
+ export function normaliseQuota(input) {
92
+ if (!input) return null;
93
+ const config = typeof input === 'number' ? { requests: input } : input;
94
+ const requests = Number(config.requests);
95
+ if (!Number.isFinite(requests) || requests < 1) return null;
96
+
97
+ return {
98
+ requests: Math.floor(requests),
99
+ windowSeconds:
100
+ Number.isFinite(config.windowSeconds) && config.windowSeconds > 0
101
+ ? Math.floor(config.windowSeconds)
102
+ : 60,
103
+ identify: typeof config.identify === 'function' ? config.identify : null,
104
+ store:
105
+ config.store && typeof config.store.hit === 'function' ? config.store : memoryQuotaStore(),
106
+ /** Only meter these paths or prefixes. Null meters everything the gate sees. */
107
+ paths: Array.isArray(config.paths) && config.paths.length ? config.paths : null,
108
+ };
109
+ }
110
+
111
+ /** Whether this path is metered at all. */
112
+ export function meters(quota, path) {
113
+ if (!quota.paths) return true;
114
+ return quota.paths.some((p) => (p.endsWith('/') ? path.startsWith(p) : path === p));
115
+ }
116
+
117
+ /**
118
+ * Spend one request against the allowance.
119
+ *
120
+ * Null when the caller cannot be identified, which counts as within the
121
+ * allowance. A caller we cannot count is not evidence of abuse, and charging
122
+ * one because our own edge did not hand us an address would be charging for our
123
+ * own gap.
124
+ */
125
+ export async function spend(quota, key) {
126
+ if (!key) return null;
127
+ const hit = await quota.store.hit(key, quota.windowSeconds);
128
+ return {
129
+ count: hit.count,
130
+ remaining: Math.max(0, quota.requests - hit.count),
131
+ resetSeconds: hit.resetSeconds,
132
+ overLimit: hit.count > quota.requests,
133
+ };
134
+ }
135
+
136
+ /**
137
+ * Headers describing the allowance, in the shape the draft IETF RateLimit
138
+ * fields use. Sent whether or not the caller is over, because a client that can
139
+ * see it is approaching a wall can buy a pass before it hits one, which is a
140
+ * better outcome for both sides than a surprise.
141
+ */
142
+ export function quotaHeaders(quota, usage) {
143
+ if (!usage) return {};
144
+ return {
145
+ 'ratelimit-limit': String(quota.requests),
146
+ 'ratelimit-remaining': String(usage.remaining),
147
+ 'ratelimit-reset': String(usage.resetSeconds),
148
+ };
149
+ }