@osqd/bothandlerjs 0.3.0

package/docs/course/03-verdicts-and-scores.md

@@ -0,0 +1,160 @@
+# Lesson 3 — Verdicts, classes and scores
+
+**Goal:** know which of the four descriptive fields to act on, and why reading the wrong
+one is the most common integration bug.
+
+← [Course](index.md) · Prev: [Proof and suspicion](02-proof-and-suspicion.md) · Next: [The guard](04-the-guard.md)
+
+---
+
+## Four fields, four questions
+
+| Field | The question it answers |
+| ----- | ----------------------- |
+| `verdict` | What did we conclude? |
+| `certain` | Is that conclusion **proof**? |
+| `botClass` | What *kind* of client is it? |
+| `score` | How suspicious is it, if we are only guessing? |
+
+They are not four ways of saying the same thing, and rules that mix them up are where
+policies go wrong.
+
+## The verdicts
+
+| Verdict | Meaning | Proven? |
+| ------- | ------- | ------- |
+| `confirmed-bot` | Self-declared, self-contradictory, or caught in a trap | yes |
+| `verified-bot` | Crawler identity confirmed against an external authority | yes |
+| `suspected-bot` | Probabilistic signals cleared `suspectThreshold` (default 60) | **no** |
+| `human` | Positive evidence of a person | only if `certain` |
+| `unknown` | Nothing conclusive. The resting state of ordinary traffic | — |
+
+The two proven verdicts are different in *direction*, and that is the point:
+`confirmed-bot` is usually a reason to act, `verified-bot` is usually a reason to allow.
+Googlebot is a bot you want.
+
+## The classes
+
+`botClass` is orthogonal to the verdict. It answers "what is it?" rather than "how sure are
+we?":
+
+`human` · `verified-bot` · `declared-bot` · `automation` · `http-client` · `scanner` ·
+`scraper` · `impersonator` · `unknown`
+
+Most real policies key on `botClass`, because it maps to a business decision. "Rate-limit
+scrapers, block scanners, allow verified crawlers, tag declared bots" is a policy about
+classes, not verdicts.
+
+## Do this
+
+```js
+const cases = [
+  ["curl", { host: "serif.example", "user-agent": "curl/8.4.0", accept: "*/*" }],
+  ["sqlmap", { host: "serif.example", "user-agent": "sqlmap/1.7.2#stable (https://sqlmap.org)" }],
+  ["GPTBot", { host: "serif.example", "user-agent": "Mozilla/5.0 (compatible; GPTBot/1.0; +https://openai.com/gptbot)" }],
+  ["python", { host: "serif.example", "user-agent": "python-requests/2.32.3", "accept-encoding": "gzip, deflate" }],
+  ["Chrome", CHROME],
+];
+
+for (const [label, headers] of cases) {
+  const a = await detector.assess(createFacts({ method: "GET", url: "/books", headers, ip: "203.0.113.10" }));
+  console.log(
+    label.padEnd(8),
+    a.verdict.padEnd(14),
+    a.botClass.padEnd(14),
+    String(a.score).padStart(3),
+    a.certain ? "proven" : "guess",
+    a.identity ?? "",
+  );
+}
+```
+
+Give each case its own `ip` if you want them judged independently — otherwise they are all
+the same [actor](07-actors.md) and later requests carry the earlier ones' history.
+
+### What to notice
+
+- `sqlmap` and `curl` are both `confirmed-bot` and both proven, but their **classes**
+  differ — `scanner` and `http-client`. A policy that blocks scanners and tolerates HTTP
+  clients needs the class, not the verdict.
+- `GPTBot` carries an `identity`. That is what lets you write a rule about one named
+  crawler, and what [lesson 6](06-identity.md) is about.
+- Chrome is `unknown`, scoring 0. Ordinary traffic concludes nothing.
+
+## Which field should a rule use?
+
+**Prefer `botClass` for business decisions.** "What do I do about scrapers?" is a question
+you can answer once.
+
+**Use `certain: true` when you are about to withhold something.** It is the field that
+means proof.
+
+**Use `score` only for graded, recoverable responses** — a challenge threshold, a
+rate-limit tier. Never as a licence to deny.
+
+**Be careful with a bare `{ certain: true }`.** It matches a proven **human** too, and
+challenging a customer you just vouched for is worse than useless. The shipped
+`protect-data` preset spells this out by matching verdicts explicitly:
+
+```js
+{ id: "any-proven-automation-challenge",
+  match: { verdict: ["confirmed-bot", "verified-bot"], certain: true },
+  action: "challenge" }
+```
+
+## `confidence`, and what it is for
+
+`confidence` is 0–1, and exactly 1 when `certain`. It is not a second score — it is how
+much to trust the verdict. In lesson 1 the copied-User-Agent request scored 45 with
+confidence 0.553: genuinely unsure, and the number says so.
+
+Use it for logging and for triage in a dashboard. Do not build rules on it; `score` and
+`certain` are what the policy layer matches on.
+
+## Exercise
+
+Write a table of what Serif should do with each class, before you know how to express it.
+Just the decision:
+
+| Class | What Serif does |
+| ----- | --------------- |
+| `verified-bot` | ? |
+| `declared-bot` | ? |
+| `scraper` | ? |
+| `scanner` | ? |
+| `impersonator` | ? |
+| `http-client` | ? |
+| `unknown` | ? |
+
+<details>
+<summary>A defensible answer</summary>
+
+| Class | Serif | Why |
+| ----- | ----- | --- |
+| `verified-bot` | allow | Googlebot brings readers |
+| `declared-bot` | tag | honest automation; let the app decide |
+| `scraper` | rate-limit | bulk extraction, slowed rather than refused |
+| `scanner` | block | proven, and looking for a way in |
+| `impersonator` | block | forged an identity; proven by DNS |
+| `http-client` | challenge | might be somebody's integration — do not refuse outright |
+| `unknown` | allow | this is your readers |
+
+Keep this. It is the policy you will write in [lesson 9](09-rules.md), and comparing it
+with the shipped presets in [lesson 10](10-actions-and-presets.md) is the point of that
+lesson.
+</details>
+
+## What you learned
+
+- Verdict, class, certainty and score answer four different questions
+- `botClass` is usually the right field for a business decision
+- `certain` is the only field that should gate withholding something
+- `{ certain: true }` alone also matches a proven human
+
+## Reference
+
+- [Verdicts, classes and scores](../concepts/verdicts.md)
+- [Matching requests](../policy/rules.md) — every field a rule can read
+
+Next: [The safety guard](04-the-guard.md) — what happens when a rule asks for more than its
+evidence supports.

package/docs/course/04-the-guard.md

@@ -0,0 +1,200 @@
+# Lesson 4 — The safety guard
+
+**Goal:** watch a rule you wrote get overruled, and understand why that is the most
+valuable behaviour in the library.
+
+← [Course](index.md) · Prev: [Verdicts and scores](03-verdicts-and-scores.md) · Next: [The detectors](05-detectors.md)
+
+---
+
+## Assess, then decide
+
+Detection and policy are separate calls, on purpose:
+
+```js
+const assessment = await detector.assess(facts);   // what is this client?
+const decision = detector.decide(assessment);      // what do we do about it?
+```
+
+*What is this client* and *what should we do about it* are different questions with
+different lifetimes. `decide` is synchronous and pure, which is what lets you replay a
+policy over a log file, preview a rule change against real traffic, and test both halves
+separately.
+
+The guard lives in `decide`.
+
+## Do this
+
+Write a rule that blocks anything suspected — deliberately too aggressive:
+
+```js
+import { BotHandler, createFacts } from "@osqd/bothandlerjs";
+
+const detector = new BotHandler({
+  rules: [{ id: "block-suspects", match: { verdict: "suspected-bot" }, action: "block" }],
+  suspectThreshold: 40,
+});
+
+// A scraper that copied a User-Agent and nothing else.
+const spoof = createFacts({
+  method: "GET",
+  url: "/books",
+  ip: "203.0.113.55",
+  headers: {
+    host: "serif.example",
+    "user-agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36",
+    accept: "*/*",
+  },
+});
+
+const assessment = await detector.assess(spoof);
+const decision = detector.decide(assessment);
+
+console.log(`verdict ${assessment.verdict}  score ${assessment.score}  certain ${assessment.certain}`);
+console.log(`action ${decision.action}  rule ${decision.rule}  downgradedFrom ${decision.downgradedFrom}`);
+console.log(decision.downgradeReason);
+```
+
+### Checkpoint
+
+```
+verdict suspected-bot  score 45  certain false
+action tag  rule block-suspects  downgradedFrom block
+Strict mode permits a terminal action only on proven evidence. This request's verdict
+(suspected-bot, score 45) rests on probabilistic signals, any of which a real client can
+trip.
+```
+
+**Your rule said `block`. The request was tagged.**
+
+The rule still fired — `decision.rule` names it — and the guard replaced the action,
+recorded what it replaced, and explained itself in a sentence you can put in front of an
+operator. Nothing was silent.
+
+## Why it is a separate pass
+
+The guard runs **after** a rule is selected, not inside the rule. Anywhere else it can be
+forgotten.
+
+Inside a rule, it would have to be written correctly by every author of every rule,
+including at 3am during an incident, including by somebody who has not read this course.
+After selection, it cannot be forgotten in a rule, worked around by a clever predicate, or
+bypassed by copying an example from a blog post. Relaxing it is one explicit, greppable
+setting.
+
+## Now give it proof
+
+Same rule set, a client that declares itself:
+
+```js
+const proven = createFacts({
+  method: "GET", url: "/books", ip: "203.0.113.10",
+  headers: { host: "serif.example", "user-agent": "curl/8.4.0" },
+});
+const d = detector.decide(await detector.assess(proven));
+console.log(`action ${d.action}  rule ${d.rule}  downgradedFrom ${d.downgradedFrom ?? "(not downgraded)"}`);
+```
+
+```
+action allow  rule default  downgradedFrom (not downgraded)
+```
+
+Surprised? **The rule did not match.** `curl` is `confirmed-bot`, not `suspected-bot`, so
+`block-suspects` never applied and the default action took over. The guard was not
+involved at all.
+
+That is a useful accident to hit early: a rule that matches the wrong verdict is a much
+more common bug than a rule the guard overrules.
+
+## The three terminal actions
+
+`block`, `drop` and `redirect` withhold the page. Everything else — `allow`, `log`, `tag`,
+`delay`, `rate-limit`, `challenge`, `custom` — leaves the client a way through.
+
+Only the terminal three are gated. [Lesson 10](10-actions-and-presets.md) ranks all ten by
+what each costs somebody who turns out to be a person.
+
+## The three modes
+
+```js
+new BotHandler({ falsePositivePolicy: "strict" })   // default
+```
+
+| Mode | A terminal action is allowed when |
+| ---- | --------------------------------- |
+| `strict` | the verdict is **proven**. Nothing else |
+| `balanced` | proven, **or** the score is at or above `terminalScoreThreshold` (default 85) |
+| `aggressive` | any bot verdict, proven or not |
+
+Try `balanced` with the spoofed request above and it is still tagged — 45 is under 85. Push
+`suspectThreshold` down and the score does not change; the score is what it is, and the
+threshold only decides what to call it.
+
+**`aggressive` will deny real people.** It exists because some operators genuinely need it —
+an internal API where every human is authenticated, say — and because a setting you have to
+write down is better than one you can reach by accident. If you find yourself reaching for
+it during an incident, read [`under-attack`](../policy/presets.md#under-attack) first: it is
+designed for exactly that moment and keeps the guard on.
+
+## The fallback
+
+When the guard replaces a terminal action, it substitutes `fallbackAction` — `challenge`
+by default. Above you got `tag`, not `challenge`, because no challenge secret is
+configured; a rule asking for a challenge without one degrades to `tag` and warns. You will
+fix that in [lesson 11](11-the-challenge.md).
+
+`fallbackAction` **cannot itself be terminal**, and the library refuses at construction:
+
+```js
+new BotHandler({ fallbackAction: "block" });
+// throws — a terminal fallback would make every downgrade deny the request the
+// downgrade existed to protect, while still recording it as a guard stop
+```
+
+## Exercise
+
+Count the downgrades over a mixed batch, and notice what that number is for.
+
+```js
+const detector = new BotHandler({
+  rules: [{ id: "block-suspects", match: { verdict: "suspected-bot" }, action: "block" }],
+  suspectThreshold: 40,
+  onDowngrade: ({ decision }) => console.log(`  guard stopped rule "${decision.rule}"`),
+});
+```
+
+Run several requests through it. Then read the counter:
+
+```js
+console.log(detector.metrics().downgrades);
+```
+
+<details>
+<summary>What that number means</summary>
+
+`bothandler_downgrades_total` is the single most informative number this library exposes.
+A rising count means **your policy is asking for something the evidence does not
+support** — and the guard is absorbing the difference.
+
+Alert on it. If it climbs after a deploy, you changed a rule; if it climbs on its own, your
+traffic changed. Either way it is telling you the policy and reality have drifted apart,
+and the guard is the only reason nobody has been wrongly refused yet.
+
+[Lesson 13](13-operating-it.md) wires it up properly.
+</details>
+
+## What you learned
+
+- `assess` and `decide` are separate; the guard lives in `decide`, after rule selection
+- A rule can fire and still not get the action it asked for — visibly, with a reason
+- Only `block`, `drop` and `redirect` are gated
+- `fallbackAction` cannot be terminal, and that is enforced at construction
+- `downgrades` is the number to watch
+
+## Reference
+
+- [The safety guard](../concepts/the-guard.md)
+- [Actions](../policy/actions.md) — ordered by what each costs a person
+- [Design decisions](../design/decisions.md) — why the guard is a separate pass
+
+Next: [The detectors](05-detectors.md) — where the evidence actually comes from.

package/docs/course/05-detectors.md

@@ -0,0 +1,178 @@
+# Lesson 5 — The detectors
+
+**Goal:** know what each of the twenty detectors reads, what it costs, and what it is
+allowed to conclude — then turn one off and watch the score move.
+
+← [Course](index.md) · Prev: [The guard](04-the-guard.md) · Next: [Identity and verification](06-identity.md)
+
+---
+
+## What is installed
+
+```js
+for (const d of detector.describeDetectors()) {
+  console.log(`${d.id.padEnd(22)} ${d.cost.padEnd(6)} ${d.stage}`);
+}
+```
+
+### Checkpoint
+
+```
+self-identified        cheap  always
+trap                   cheap  always
+ip-intelligence        cheap  always
+probe-signature        cheap  always
+header-integrity       cheap  always
+ua-coherence           cheap  always
+client-hints           cheap  always
+fetch-metadata         cheap  always
+accept-signature       cheap  always
+header-order           cheap  always
+rate-anomaly           cheap  always
+cadence                cheap  always
+crawl-breadth          cheap  always
+session-integrity      cheap  always
+browsing-coherence     cheap  always
+crawler-verification   io     confirming
+```
+
+**Sixteen of twenty ship on by default.** The other four —
+`identity-rotation`, `tls-fingerprint`, `clearance`, `client-signals` — each need something
+from you, and you will switch three of them on later in the course.
+
+## Cost and stage
+
+Every detector declares two things, and they decide when it runs.
+
+**`cost`** is `cheap` or `io`. Cheap detectors are pure string and header work and run
+sequentially in microseconds. `io` detectors touch the network or a shared store, run
+concurrently, and each gets its own timeout — `detectorTimeoutMs`, default 300 ms.
+
+**`stage`** is `always` or `confirming`. The confirming stage runs only when an earlier
+detector produced something worth confirming, which is why `crawler-verification` does no
+DNS at all for a request that never claimed an identity.
+
+A detector that throws or times out is dropped; **the request is not**. A bot filter that
+fails closed is an outage with extra steps.
+
+## What each one does
+
+### Identity
+
+| Detector | Ceiling | Reads |
+| -------- | ------- | ----- |
+| `self-identified` | `certain` | 161 signatures, 389 tokens, in one Aho–Corasick pass. Also catches unrecognised crawlers that name a contact URL, and bare client tokens with no browser preamble |
+| `crawler-verification` | `certain` | Forward-confirmed reverse DNS, or published address ranges. **Confirms and refutes** |
+
+### Single-request consistency
+
+| Detector | Ceiling | Reads |
+| -------- | ------- | ----- |
+| `header-integrity` | `certain`* | Header set against the claimed client, plus framing rules a recipient must enforce. `certain` only for the three protocol violations |
+| `ua-coherence` | `strong` | The User-Agent against itself: two engines, two platforms, Chrome on an iPhone, a version its platform never received. Needs no other headers, so it works on a log line |
+| `client-hints` | `certain`* | `Sec-CH-UA` against the User-Agent. `certain` only for a self-declared headless brand |
+| `fetch-metadata` | `strong` | `Sec-Fetch-*` absence on engines that send them, and incoherent combinations. Page JavaScript cannot set these |
+| `accept-signature` | `strong` | `Accept: */*` on a navigation; `Accept-Language` that is not valid grammar |
+| `header-order` | `moderate` | Orderings no mainstream browser produces. HTTP/1.x only |
+
+### Behaviour over time
+
+| Detector | Ceiling | Reads |
+| -------- | ------- | ----- |
+| `rate-anomaly` | `moderate` | Arrivals in a short window. Reports; never concludes |
+| `cadence` | `moderate` | Coefficient of variation of the gaps. Catches the polite scraper pacing itself *under* your rate limit |
+| `crawl-breadth` | `weak` | Distinct paths against total requests: reading a site against enumerating it |
+| `session-integrity` | `moderate` | A "browser" that never carries a cookie |
+| `identity-rotation` | `moderate` | One actor, several User-Agents. **Off by default** — under an IP actor key this fires on every corporate NAT |
+| `browsing-coherence` | `moderate` | The only detector arguing *for* the client. Human-pointing, so it discounts |
+
+### Environment and traps
+
+| Detector | Ceiling | Reads |
+| -------- | ------- | ----- |
+| `trap` | `certain` | Hidden links, hidden fields, trap headers — [lesson 8](08-traps.md) |
+| `probe-signature` | `strong` | What the request *asks for*: `/.env`, `/.git/config`, a JNDI lookup, a `TRACE` |
+| `ip-intelligence` | `certain`* | Your denylist (`certain` — your decision, not our inference) and datacenter ranges (`moderate`). Ships no data |
+| `tls-fingerprint` | `strong` | JA3/JA4 from your edge. **Off by default** |
+| `clearance` | `certain` | A signed clearance token. `certain` only at `operator` level — [lesson 11](11-the-challenge.md) |
+| `client-signals` | `moderate` | What a browser-side script reported. **Capped at `moderate`, permanently** — [lesson 15](15-extending.md) |
+
+## Do this: turn one off
+
+`header-integrity` was doing most of the work on the copied-User-Agent request from lesson
+1. Remove it and see:
+
+```js
+import { BotHandler, createFacts, defaultDetectors } from "@osqd/bothandlerjs";
+
+const without = new BotHandler({
+  detectors: defaultDetectors().filter((d) => d.id !== "header-integrity"),
+});
+
+const spoof = createFacts({
+  method: "GET", url: "/books", ip: "203.0.113.55",
+  headers: { host: "serif.example", "user-agent": CHROME["user-agent"], accept: "*/*" },
+});
+
+console.log("with   ", (await detector.assess(spoof)).score);
+console.log("without", (await without.assess(spoof)).score);
+```
+
+The score falls from 45 to 15 — the two `header-integrity` absences were most of the case.
+
+Note `detectors:` **replaces** the set entirely. To add without removing, use
+`extraDetectors`.
+
+## Adjusting one rather than removing it
+
+Several detectors take options. The one you are most likely to want is
+`probe-signature`, whose second tier is platform administration paths — `/wp-login.php`,
+`/administrator`, `/phpmyadmin`. Those are a probe on sites that do not run those
+platforms and the *front door* on sites that do:
+
+```js
+import { defaultDetectors, probeSignatureDetector } from "@osqd/bothandlerjs";
+
+const detectors = defaultDetectors().map((d) =>
+  d.id === "probe-signature" ? probeSignatureDetector({ ignore: ["/wp-login.php", "/wp-admin"] }) : d,
+);
+```
+
+That tier is capped at `moderate` precisely so that forgetting is survivable: an author
+signing in to their own site scores 21 and is served.
+
+## Exercise
+
+Serif runs on a Fetch runtime behind Cloudflare. Which detector should you drop, and why?
+
+<details>
+<summary>Answer</summary>
+
+**`header-order`.** Fetch runtimes normalise header order, so it has nothing real to read
+there and will either return nothing or reason from an ordering the platform invented
+rather than the client.
+
+```js
+detectors: defaultDetectors().filter((d) => d.id !== "header-order")
+```
+
+More generally: a detector reading something your infrastructure rewrites is not neutral,
+it is *misleading*. The same reasoning is why `identity-rotation` is off by default under an
+IP-based actor key.
+</details>
+
+## What you learned
+
+- Sixteen of twenty detectors are on by default; the rest need something from you
+- `cost` and `stage` decide when a detector runs, and confirming work is skipped when
+  nothing claimed an identity
+- A failing detector is dropped, never the request
+- Each detector has a ceiling it may not exceed
+- `detectors` replaces; `extraDetectors` adds
+
+## Reference
+
+- [The detectors](../detection/detectors.md) — all twenty in full
+- [How detection works](../detection/index.md) — the pipeline and its budgets
+
+Next: [Identity and verification](06-identity.md).