@osqd/bothandlerjs 0.3.0

package/docs/integration/adapters.md

@@ -0,0 +1,138 @@
+# Adapters
+
+Four adapters, six frameworks, and how to write a fifth.
+
+← [Documentation](../index.md) · [Integration](index.md)
+
+---
+
+```ts
+import { botHandler, fastifyBotHandler, koaBotHandler, withBotHandler } from "@osqd/bothandlerjs/adapters";
+
+app.use(botHandler(detector));                                  // Express / Connect / node:http
+fastify.addHook("onRequest", fastifyBotHandler(detector));
+app.use(koaBotHandler(detector));
+export default { fetch: withBotHandler(detector, myHandler) };  // Workers / Deno / Bun / Edge
+```
+
+## Why adapters exist at all
+
+The engine never touches a response object. It takes `RequestFacts` and returns an
+`ActionOutcome` — "serve it", "respond with this", "destroy the connection" — and the
+adapter is the only code that knows what a response *is* in your framework.
+
+That separation is what makes a policy portable, makes the engine testable without a
+server, and makes the [request tester](../operations/dashboard.md) and the
+[corpus](../testing/corpus.md) possible.
+
+## Hono and Next.js need no adapter of their own
+
+Both speak the platform's `Request` and `Response`, which is exactly what
+`createFetchAdapter` takes and returns. The only difference between them is how each says
+"carry on".
+
+```ts
+const guard = createFetchAdapter(detector, { ipHeaders: ["cf-connecting-ip"] });
+
+// Hono
+app.use(async (c, next) => {
+  const decision = await guard(c.req.raw, c.env);
+  if (decision.response) return decision.response;
+  await next();
+});
+
+// Next.js — middleware.ts
+export async function middleware(request: Request) {
+  const decision = await guard(request);
+  if (decision.response) return decision.response;
+  return NextResponse.next({ request: { headers: decision.request.headers } });
+}
+```
+
+Passing `decision.request.headers` on is the part worth doing: the adapter puts the verdict
+on the request it hands back, so a route further in can read it without assessing anything
+itself.
+
+`examples/hono-and-next.ts` is both, executable, and `tests/adapters.test.ts` drives both
+shapes — because "it probably works with X" is how a framework ends up unsupported by
+accident.
+
+## What every adapter does for you
+
+**Attaches the verdict to the request** — `x-bot-verdict`, `x-bot-score`, `x-bot-class`,
+`x-bot-certain`, `x-bot-reason` — so your handlers can react without re-running detection.
+
+Response-side headers are **off by default** (`exposeVerdictHeaders`): an `X-Bot-Score` in
+the response is a live feedback signal for anyone tuning a scraper against you.
+
+**Serves the [challenge](../challenge/index.md) verification endpoint.** Fastify and Koa
+read the raw stream in the hook, because by the time a route handler sees the request its
+body has been consumed by a parser that knows nothing about this endpoint.
+
+**Fails open.** An unexpected failure inside detection serves the request; the error goes to
+your `onError`.
+
+## Handing over a parsed body
+
+The engine reads no request body — doing so would consume the stream before your own parser
+saw it. So a [trap](../detection/detectors.md) field on a `method="post"` form arrives
+somewhere this library cannot see, and the forms worth protecting are POSTs.
+
+`enrich` is the hand-over:
+
+```ts
+import { TRAP_FIELD_SOURCE, defaultDetectors, trapDetector } from "@osqd/bothandlerjs";
+
+const detector = new BotHandler({
+  detectors: defaultDetectors().map((d) => (d.id === "trap" ? trapDetector({ formFields: ["company_url"] }) : d)),
+});
+
+app.use(express.urlencoded({ extended: false }));   // your parser runs first
+app.use(
+  botHandler(detector, {
+    enrich: (request, facts) => ({ ...facts, extra: { [TRAP_FIELD_SOURCE]: request.body } }),
+  }),
+);
+```
+
+A field arriving in the query string is read without any of this.
+
+## Fetch runtimes
+
+The three Node adapters read the socket address and honour [`proxy`](client-ip.md). There is
+no socket on a Fetch runtime, so `withBotHandler` and `createFetchAdapter` take the address
+from a header instead — which is the same decision `proxy` exists to make carefully.
+
+```ts
+export default {
+  fetch: withBotHandler(detector, myHandler, {
+    // Best: ask the platform, not a header.
+    clientIp: (request, env) => (env as { cf?: { connectingIp?: string } }).cf?.connectingIp,
+  }),
+};
+```
+
+Only `cf-connecting-ip` and `x-real-ip` are trusted by default. `x-forwarded-for` is **not**
+— see [the client IP](client-ip.md) for why.
+
+Two platform notes: on Workers, enable `nodejs_compat`, because the engine uses
+`node:crypto`. And Fetch runtimes normalise header order, so `header-order` returns nothing
+there — drop it.
+
+## Writing your own
+
+About thirty lines. `src/adapters/node.ts` is the model. The shape is:
+
+1. Build `RequestFacts` with `createFacts({ method, url, headers, rawHeaders, ip })`.
+2. `await detector.handle(facts)`.
+3. Apply the returned `ActionOutcome` in your framework's terms.
+4. Route the challenge verification path to `detector.verifyChallenge`.
+
+Pass `rawHeaders` if your framework exposes them — the `header-order` detector needs the
+order as it arrived, and a normalised object has already lost it.
+
+## Related
+
+- [The client IP](client-ip.md) — do this before deploying behind a proxy
+- [Actions](../policy/actions.md) — the outcomes an adapter applies
+- [Detectors](../detection/detectors.md) — including `trap`, which `enrich` feeds

package/docs/integration/client-ip.md

@@ -0,0 +1,114 @@
+# The client IP
+
+The highest-consequence setting in the library, and the easiest to get wrong.
+
+← [Documentation](../index.md) · [Integration](index.md)
+
+---
+
+## Why it is the highest-consequence setting
+
+The client address becomes the [actor key](../concepts/actors.md). Everything per-client
+depends on it: rate limits, the allowlist, the denylist, and every behavioural detector.
+
+`X-Forwarded-For` is a **client-supplied header**. Trust it without knowing how many proxies
+sit in front of you and anyone can prepend a fake hop and choose the address you rate-limit,
+allowlist and block on.
+
+The failure is silent. Nothing errors, nothing logs, and every per-actor mechanism in this
+library becomes an attacker input.
+
+There is deliberately **no convenient default**.
+
+## The three settings
+
+```ts
+proxy: { trustProxy: true, trustedProxies: ["10.0.0.0/8"] }   // recommended
+proxy: { trustProxy: true, hops: 2 }                          // correct only while the count is
+proxy: { trustProxy: false }                                  // default — header ignored entirely
+```
+
+### `trustedProxies` — the one to use
+
+The chain is walked **from the right**, discarding your own infrastructure, and the first
+address outside it is the client. That is robust against an extra hop appearing, which is
+what makes it survive a change to your topology that nobody remembered to tell you about.
+
+### `hops` — correct only while the count is
+
+Counts a fixed number of entries from the right. It is right until somebody adds a CDN,
+inserts a sidecar, or moves a service behind an extra load balancer — and then it is wrong
+in the direction that lets clients choose their own address.
+
+### `trustProxy: false` — the default
+
+The forwarded header is ignored entirely and the socket address is used. Correct when
+nothing sits in front of you, and safe everywhere else in the sense that matters: it can be
+*useless* behind a proxy (every request looks like it came from the load balancer), but it
+cannot be *forged*.
+
+## Addresses are compared as bytes
+
+`::ffff:127.0.0.1`, `0177.0.0.1` and `127.0.0.001` are all the same address, and all of them
+slip past an allowlist that compares strings. Parsing to bytes is the only comparison that
+holds.
+
+Invalid CIDRs throw at construction rather than matching silently. A range that matches
+nothing is a control you believe you have and do not.
+
+## The peer is checked too
+
+The connecting peer counts as the first hop and is checked the same way.
+
+That is the part that matters on a server reachable **both** through the load balancer and
+directly. A request arriving from outside your trusted ranges did not come through your
+proxies, so its forwarded header is not evidence of anything and the socket address is used
+instead.
+
+Without that check, anyone who finds the origin address picks their own client IP — and
+origin addresses are not secret.
+
+## Do not allowlist loopback
+
+The moment you sit behind nginx or beside a sidecar, every request in the world arrives from
+`127.0.0.1`.
+
+## On Fetch runtimes
+
+There is no socket, so the address comes from a header. Only `cf-connecting-ip` and
+`x-real-ip` are trusted by default: both are single-valued and written by the edge that
+terminated the connection.
+
+`x-forwarded-for` is **not** in that list, because a proxy *appends* to it — its leftmost
+entry is whatever the client wrote. List it explicitly only if you know your edge replaces
+the whole header.
+
+Better still, ask the platform rather than a header:
+
+```ts
+createFetchAdapter(detector, {
+  clientIp: (request, env) => (env as { cf?: { connectingIp?: string } }).cf?.connectingIp,
+});
+```
+
+If no address can be found, every visitor is tracked under one empty actor key — which makes
+rate limits and behavioural detection apply to your whole site at once. The adapter says so
+through `onWarning` the first time it happens.
+
+## When the address is not the right identity anyway
+
+An address is a poor identity: shared by a whole office, changed by a phone every few
+minutes. `actorKey` is the single most valuable thing to replace:
+
+```ts
+new BotHandler({ actorKey: (facts) => facts.session ?? facts.ip });
+```
+
+A session id, an authenticated user id, or an address plus a TLS fingerprint all make the
+same detectors sharper — sharp enough that `identity-rotation` becomes worth enabling.
+
+## Related
+
+- [Actors](../concepts/actors.md) — what the key is used for
+- [Adapters](adapters.md) — where the address is read
+- [Configuration reference](../reference/configuration.md) — `proxy`, `actorKey`, `allowlist`

package/docs/integration/index.md

@@ -0,0 +1,55 @@
+# Integration
+
+Getting it into your application: frameworks, the client address, and shared state.
+
+← [Documentation](../index.md)
+
+---
+
+| Page | For |
+| ---- | --- |
+| [Adapters](adapters.md) | Express, Fastify, Koa, Fetch runtimes, Hono, Next.js — and writing your own |
+| [The client IP](client-ip.md) | the highest-consequence setting in the library |
+| [Stores](stores.md) | what needs to be shared across replicas, and what deliberately is not |
+
+---
+
+## The shape of it
+
+Three lines in the common case:
+
+```ts
+import { BotHandler } from "@osqd/bothandlerjs";
+import { botHandler } from "@osqd/bothandlerjs/adapters";
+
+const detector = new BotHandler({ preset: "protect-content" });
+app.use(botHandler(detector));
+```
+
+One `BotHandler` instance per policy, mounted where that policy applies. Most applications
+have one; an application that runs [`protect-auth`](../policy/presets.md) on its login
+routes has two.
+
+## The one thing to get right before anything else
+
+**The client address.** It becomes the [actor key](../concepts/actors.md), which every
+behavioural detector, every rate limit and every allowlist entry depends on. Behind a proxy
+it has no safe default, and getting it wrong turns every per-actor mechanism here into an
+attacker input.
+
+Read [the client IP](client-ip.md) before you deploy this behind anything.
+
+## Two guarantees the integration relies on
+
+**The engine never touches a response object.** It returns an `ActionOutcome` and the
+adapter applies it — which is why the same policy behaves identically on Express and on a
+Worker, and why writing your own adapter is about thirty lines.
+
+**It fails open.** An unexpected failure inside detection serves the request. The error
+goes to your `onError`; the visitor gets their page. A bot filter that fails closed is an
+outage with extra steps.
+
+## Related
+
+- [Configuration reference](../reference/configuration.md) — every option
+- [First integration](../start/first-integration.md) — the walk-through

package/docs/integration/stores.md

@@ -0,0 +1,76 @@
+# Stores
+
+What has to be shared across replicas, what deliberately is not, and why.
+
+← [Documentation](../index.md) · [Integration](index.md)
+
+---
+
+## The default
+
+An in-memory store. Correct for a single process, and honest about what it is not.
+
+```ts
+import { RedisStore } from "@osqd/bothandlerjs";
+new BotHandler({ store: new RedisStore(redis) });   // ioredis or node-redis; no dependency added
+```
+
+`RedisStore` takes a client you already have. The package stays dependency-free — it uses
+the handful of methods both major clients share.
+
+## What needs a shared store
+
+**Single-use challenge nonces.** Without sharing, a scraper retries a solved nonce against
+another replica until one has not seen it. See [the challenge](../challenge/index.md).
+
+**Rate limits.** A limit of 100/minute enforced independently by four replicas is a limit of
+400/minute. See [`rate-limit`](../policy/actions.md#rate-limit).
+
+Those two are correctness, not optimisation. If you run more than one process and use either
+feature, you need a store.
+
+## What deliberately stays process-local
+
+**Behavioural state** — arrival rates, cadence, path breadth, User-Agent history — lives in
+memory, on purpose.
+
+A round trip per request would buy accuracy for signals that are only ever allowed to *raise
+suspicion*, never to deny anybody: the [guard](../concepts/the-guard.md) sees to that.
+Behind four replicas each one sees a quarter of an actor's traffic and is correspondingly
+less sure, which is the right trade for something that cannot close a door on its own.
+
+Spending a network round trip on the request path to sharpen a signal that cannot act alone
+is a bad bargain, and the request path is the one place this library refuses to make bad
+bargains.
+
+## Confirmations are not one of those
+
+A `confirmed-bot` verdict is *proven* — something declared itself, forged an identity, or
+walked into a trap. That is a fact about the client rather than a judgement about it, so it
+can travel:
+
+```ts
+new BotHandler({ store: new RedisStore(redis), shareConfirmations: true });
+```
+
+Without it, a client proven to be a bot on one replica is a stranger to the other seven, and
+`minPriorConfirmations: 1` fires about an eighth as often as it reads.
+
+**Proof travels; suspicion stays home.** That single sentence is the whole design.
+
+The cost is one store read the **first time each instance sees an actor** — not one per
+request — and it is never awaited, so nothing joins the request path. A store outage means
+the count falls back to what that process saw itself.
+
+## Writing your own
+
+Implement `BotHandlerStore`. The interface is small — get, set, increment, an atomic
+claim — and every method may fail: the engine treats a store error as "no answer" and
+carries on serving, because a store outage must not become a site outage.
+
+## Related
+
+- [The challenge](../challenge/index.md) — replay protection
+- [Actions](../policy/actions.md#rate-limit) — the one action that needs this
+- [Actors](../concepts/actors.md) — what the local state actually holds
+- [The guard](../concepts/the-guard.md) — why local behavioural state is safe

package/docs/operations/audit.md

@@ -0,0 +1,116 @@
+# The audit
+
+Noticing that the traffic changed shape.
+
+← [Documentation](../index.md) · [Operations](index.md)
+
+---
+
+## Why this exists
+
+[Counters](metrics.md) tell you what is happening. They do not tell you it is *unusual*,
+and bot traffic is not a level — it is an **event**. A scrape starts, a scanner sweeps a
+range, somebody points a stuffing tool at your login form. The number that matters is not
+"12% of requests are bots" but "12% now, 2% for the hour before".
+
+So the audit keeps a short **window**, compares it against the **baseline that precedes
+it**, and raises a structured anomaly when a check clears its bar.
+
+```ts
+new BotHandler({
+  audit: {
+    windowMs: 5 * 60_000,        // the stretch being judged
+    baselineMs: 60 * 60_000,     // what it is compared against — the hour before it
+    intervalMs: 60_000,          // how often the comparison runs
+    minSamples: 50,              // below this, no check may speak
+    cooldownMs: 15 * 60_000,     // one alert per check per quarter hour
+  },
+  onAnomaly: (anomaly) => pager.send(anomaly.severity, anomaly.summary),
+});
+```
+
+```
+critical — Automated traffic is 87.3% of assessed requests, against 0.0% in the
+           baseline — where there was none.
+warning  — The guard stopped 37 terminal action(s) — 35.9% of traffic. Your rules
+           are asking to deny requests the evidence does not prove.
+critical — Traffic is 298 requests a minute, against a baseline of 1.5 (198.7x).
+```
+
+**The baseline ends where the window begins.** A baseline that contained the window would
+be partly made of the thing being measured, and a large enough spike would raise its own
+bar until it stopped looking like one.
+
+## What ships
+
+| Check | Fires when |
+| ----- | ---------- |
+| `bot-share-spike` | Automation is a large and much larger share of traffic than the baseline. A baseline of zero is the loudest case, not a reason to stay quiet. |
+| `traffic-spike` | Volume is far above the baseline rate. |
+| `denial-spike` | A much larger share of requests is being denied. **Read these before assuming they are all bots.** |
+| `guard-stop-spike` | The [guard](../concepts/the-guard.md) is refusing far more rules than usual — your policy is asking to deny requests the evidence does not prove. |
+| `human-share-drop` | Traffic that reads as human has fallen away. Either your traffic changed or your detection did. |
+| `detector-failures` | Detectors are erroring or timing out. Detection is degraded — usually a resolver or a store, not the traffic. |
+| `challenge-solve-rate` | Nearly everything being challenged is passing. **High is the bad direction.** |
+
+### Why `challenge-solve-rate` reads backwards
+
+It is the odd one out and the closest to this library's own thesis. Every other check asks
+whether the *traffic* changed shape; this one asks whether the mitigation is landing on the
+right population.
+
+A proof-of-work [challenge](../challenge/index.md) is trivial for a browser and trivial for
+a competent scraper — what it actually costs is a few seconds of somebody's afternoon — so
+when nearly everything challenged goes on to pass, the challenges are not filtering bots
+out, they are **taxing people**. From every other angle a challenge that gets solved looks
+like a challenge that worked, which is precisely why this needs saying out loud.
+
+### Floors and cooldowns
+
+Every check has a **floor** as well as a ratio, because a quiet site at 3am produces "800%
+more bots" from four requests, and an alerting system that cries wolf at 3am gets muted —
+which is worse than not having one.
+
+And every one has a **cooldown**: a spike lasting an hour is one event, not sixty.
+
+## Your own checks
+
+```ts
+audit: {
+  extraChecks: [{
+    id: "checkout-scraping",
+    description: "Bots on the checkout funnel specifically",
+    evaluate: ({ window, baseline }) =>
+      window.botShare > 0.4 && window.botShare > baseline.botShare * 2
+        ? { id: "checkout-scraping", severity: "warning", metric: "bot share",
+            value: window.botShare, baseline: baseline.botShare,
+            summary: `Checkout is ${(window.botShare * 100).toFixed(0)}% automated.` }
+        : undefined,
+  }],
+}
+```
+
+A check that throws is skipped, not a reason for the audit to stop running. `checks`
+replaces the built-in set entirely; `extraChecks` adds to it.
+
+## Asking it yourself
+
+The timer is a convenience, not the mechanism. `handler.audit` is the object, and
+`runAudit()` runs the checks now and emits whatever they found — which is what a health
+endpoint, a cron job or a test with a manual clock wants:
+
+```ts
+app.get("/internal/traffic", (_req, res) => res.json(detector.audit?.summary()));
+
+const anomalies = detector.runAudit();
+```
+
+Anomalies also reach configured [notification sinks](notifications.md) as `type: "anomaly"`
+events, and land in the [dashboard's](dashboard.md) notices panel beside your startup
+warnings. Switch the whole thing off with `audit: false`.
+
+## Related
+
+- [Metrics](metrics.md) — the counters this reads
+- [Notifications](notifications.md) — where an anomaly goes
+- [The guard](../concepts/the-guard.md) — what `guard-stop-spike` is about