@osqd/bothandlerjs 0.3.0 → 0.4.0

package/CHANGELOG.md

@@ -6,7 +6,163 @@ adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 ## [Unreleased]
 
-Nothing yet.
+### Added
+
+- **`indexers-only`, a preset for sites that want search traffic and nothing else.**
+  Serves a bot only when its identity has been confirmed by DNS or a published range
+  *and* it is a `search` or `social` crawler; refuses every other proven bot; challenges
+  suspicion and holds weak signal to a ceiling. It is the first shipped preset whose
+  terminal rules cover all proven automation permanently rather than during an incident.
+
+  Read its documentation before choosing it. Twenty-three of the shipped search and
+  social signatures — Twitterbot, LinkedInBot, Slackbot, Discord, Telegram, WhatsApp,
+  Reddit, Mastodon, Bluesky among them — publish nothing that can confirm a claim, so
+  they can never be verified and `unverifiable-indexer-block` refuses them, taking your
+  link previews with it. Fifteen of the corpus's infrastructure cases are refused too,
+  including health checks, a server-side renderer and a payment webhook: allowlist your
+  own automation above the preset before switching it on.
+
+- **`<bot-dashboard>`, the dashboard as an element.** A new entry point,
+  `@osqd/bothandlerjs/element`, exporting `defineBotDashboard()`. Mount
+  `createDashboardHandler` as before and drop the element into a page you already
+  have: it renders the whole dashboard into a shadow root in your own layout, under
+  your own heading, rather than on a route of its own.
+
+  `config.tabs` chooses which of the four screens appear, in what order and under what
+  labels. `config.theme` sets the scheme, the density and any of the stylesheet's
+  tokens — which is why every token block is now written `:root, :host`, since `:root`
+  matches nothing inside a shadow tree. `config.panels` adds panels of your own, fed
+  by a URL or a function and rendered as text.
+
+  **Be clear about what embedding costs.** A shadow root is a styling boundary and not
+  a security boundary: any script that can run on the host page can read every client
+  address and every piece of evidence the dashboard renders, and call its API with your
+  credentials. On the standalone page it could not. Mount it behind your admin
+  authentication and treat an XSS on that page as equivalent to handing the dashboard
+  over — or serve the standalone page, which is the same dashboard and already
+  isolated. `config.hide` is cosmetic; `sections` on the handler is the setting that
+  stops data leaving the process.
+
+  Embedded, the dashboard stops doing four things it does when it owns a page: binding
+  its keyboard shortcuts to the window, where a digit pressed on the host page switched
+  a tab in here; writing its tab and filter into `location.hash`, which is the host's
+  address bar and its back button; relying on a fragment anchor for the skip link,
+  which is inert across a shadow boundary; and refusing to mount a second time, which
+  made it unusable under any router, since unmount-and-remount is what a route change
+  is. A remount now keeps the feed history it had built.
+
+  `GET {basePath}/api/bootstrap` is new, serving the element the same configuration the
+  standalone page carries stamped into it.
+
+  The entry point imports safely on a server — Next, Remix, Astro and the rest evaluate
+  top-level imports while rendering, where there is no DOM — and `defineBotDashboard()`
+  does nothing until it is in a browser, so it can be imported like anything else and
+  called on mount.
+
+- **An interaction challenge.** `challenge.interaction` asks the interstitial for a
+  deliberate gesture as well as the proof of work, and probes what the browser can
+  actually do while it waits. Six probes read back things only a rendering engine
+  produces — a computed style that requires the cascade to have run, a laid-out box,
+  font metrics that differ between two families, a frame loop, a media query, an element
+  that `display: none` actually hid. One of them is different for every challenge: the
+  box count and box height of the layout probe are drawn from the nonce under the
+  signing secret, so the client cannot compute the answer and can only measure it.
+
+  When it is on, solving the puzzle alone no longer grants clearance: the gesture is
+  required, and passing grants the `interaction` clearance level, which the `clearance`
+  detector has always known how to read and which nothing until now ever granted.
+  `challengeTtlMs` defaults to ten minutes rather than two, because the page now stops
+  and waits for a person to read it.
+
+  The gesture is a checkbox, and that is the whole accessibility argument: it is the one
+  interactive control that a pointer, a touch screen, the space bar, a screen reader,
+  switch access and voice control can all operate. A tap and a keypress are reported as
+  what they are and never marked down for producing no pointer path, and a pointer path
+  too short to measure is treated as no evidence rather than bad evidence.
+
+  **Be clear about the ceiling, which is measured rather than assumed.** A client that
+  replays a report captured from a real browser gets nowhere, and one that hardcodes a
+  formula for the layout probe gets nowhere — but one that *parses the served HTML and
+  CSS each time* passes every check, because both numbers have to reach the browser to
+  be rendered. Against an adversary who writes a parser for your challenge page this
+  adds nothing over the plain proof of work beyond the server-verified elapsed-time
+  floor, and no client-side probe can. What it defeats is every scraper that does not
+  bother. Nothing here is proof of humanity and none of it is ever `certain`.
+
+- **Counters for the challenge that were not there.** `bothandler_clearances_total` by
+  level, `bothandler_challenge_rejections_total` by cause, and
+  `bothandler_interaction_score_bucket` in tenths, fed by refusals as well as successes
+  so the distribution is not censored at the threshold it exists to inform. The
+  `challenge` event carries `level`, `score` and `reason` to match.
+
+  `MetricsSnapshot` gains three fields, so anything constructing one by hand needs them.
+
+- **A test that parses the interstitial's own inline script.** The page is built inside
+  a template literal, where one backtick — in a comment, a string, a regular
+  expression — ends the literal early and turns the rest into markup.
+  `tests/challenge-page.test.ts` parses the rendered script with `node:vm`, including
+  with operator-supplied copy that contains a backtick.
+
+
+### Fixed
+
+- **The "N not streamed" badge outlived the feed it described.** The badge adds two
+  counts: the server's rate-cap `skipped`, which `FeedRing.clear()` resets, and the
+  connection's lagged drops, which nothing did. After a Reset — or after the replace-sync
+  that follows a dropped stream, which is the likelier path, since a viewer dropped for
+  lagging reconnects with a stale cursor and is sent a fresh backlog — the number went on
+  reporting a gap in a window that had just been replaced, under a tooltip promising those
+  entries were still in it.
+
+## [0.3.0] — 2026-09-06
+
+### Changed
+
+- **The package is now `@osqd/bothandlerjs`.** Every import moves with it —
+  `@osqd/bothandlerjs`, `/adapters`, `/client`, `/corpus`, `/cli` — and
+  `npm install bothandlerjs` becomes `npm install @osqd/bothandlerjs`. The CLI command
+  is unchanged: a `bin` name is independent of the package name, so `bothandlerjs check`
+  still works once installed, and `npx @osqd/bothandlerjs` installs it.
+
+  npm refuses capital letters in a new package name, so the spelling is
+  `@osqd/bothandlerjs` rather than `@osqd/BotHandlerJS`. `publishConfig.access` is set
+  to public, without which a scoped package publishes private on the first attempt, and
+  the tarball is now `osqd-bothandlerjs-<version>.tgz`.
+
+### Added
+
+- **Releases are cut from `main`.** `scripts/next-version.mjs` reads the commits since
+  the last release tag and works out the version — `feat:` minor, `fix:`/`perf:` patch,
+  a `!` or a `BREAKING CHANGE:` footer major (or minor while the major is 0), and
+  everything else nothing at all. That last rule is what makes publishing on every push
+  tolerable: a documentation fix releases nothing.
+  `.github/workflows/publish.yml` runs the full gate against the commit, refuses a
+  version already on the registry, bumps and tags, inspects the tarball, and publishes
+  with provenance.
+
+- **A sixteen-lesson course**, in `docs/course/`, that builds one integration from a
+  first assessment to a policy you can defend. Every checkpoint in it is real output
+  from running the code.
+- **A test for the published entry points.** `tests/entry-points.test.ts` asserts the
+  surface of all four, which is the actual fix for the two entries below: every module
+  in the repository imports its neighbours by path, so nothing exercised the paths the
+  documentation tells other people to use.
+
+### Fixed
+
+- **`@osqd/bothandlerjs/corpus` did not export `runCorpus`.** That is the entire point
+  of publishing the entry point, and the README, the changelog, the design notes and
+  three documentation pages all told people to import it. It would have failed for
+  everyone outside this repository, and no test could have noticed because every
+  internal caller reaches past the entry point to `./runner.js`.
+
+- **`renderClientScript` and `parseClientSignals` were documented on the root export.**
+  They live behind `/client`, and the example omitted the required `endpoint` argument.
+
+- **A challenge test failed about one run in 271, claiming a bad proof of work had been
+  accepted.** It submitted the fixed solution `"1"` against a random nonce at difficulty
+  8, where one guess in 256 is a valid proof by accident. It searches for a counter that
+  provably misses now, and asserts that it misses before submitting it.
 
 ## [0.2.0]
 

package/README.md

@@ -41,7 +41,7 @@ explaining why a thing exists as well as how to use it.
 | **Concepts** | [Evidence and certainty](docs/concepts/evidence.md) · [Verdicts and scores](docs/concepts/verdicts.md) · [The safety guard](docs/concepts/the-guard.md) · [Actors](docs/concepts/actors.md) · [Threat model](docs/concepts/threat-model.md) |
 | **[Detection](docs/detection/index.md)** | [The 20 detectors](docs/detection/detectors.md) · [Signatures](docs/detection/signatures.md) · [Verification](docs/detection/verification.md) · [Browser signals](docs/detection/client-signals.md) · [Writing a detector](docs/detection/writing-a-detector.md) |
 | **[Policy](docs/policy/index.md)** | [Rules](docs/policy/rules.md) · [Actions](docs/policy/actions.md) · [Presets](docs/policy/presets.md) · [robots.txt](docs/policy/robots.md) · [The challenge](docs/challenge/index.md) |
-| **[Operations](docs/operations/index.md)** | [The dashboard](docs/operations/dashboard.md) · [Metrics](docs/operations/metrics.md) · [The audit](docs/operations/audit.md) · [Notifications](docs/operations/notifications.md) · [Runtime changes](docs/operations/runtime-changes.md) |
+| **[Operations](docs/operations/index.md)** | [The dashboard](docs/operations/dashboard.md) · [Embedding it](docs/operations/embedding.md) · [Metrics](docs/operations/metrics.md) · [The audit](docs/operations/audit.md) · [Notifications](docs/operations/notifications.md) · [Runtime changes](docs/operations/runtime-changes.md) |
 | **[Integration](docs/integration/index.md)** | [Adapters](docs/integration/adapters.md) · [The client IP](docs/integration/client-ip.md) · [Stores](docs/integration/stores.md) |
 | **[Testing](docs/testing/index.md)** | [The CLI](docs/testing/cli.md) · [The corpus](docs/testing/corpus.md) · [Log replay](docs/testing/replay.md) · [Try it locally](docs/testing/try-it.md) |
 | **Reference** | [Configuration](docs/reference/configuration.md) · [API](docs/reference/api.md) · [Design decisions](docs/design/decisions.md) |

package/dist/challenge/index.d.ts

@@ -1,4 +1,5 @@
 import type { ChallengeCopy } from "./language.js";
+import type { InteractionSettings } from "./interaction.js";
 import type { Clock } from "../internal/clock.js";
 import type { BotHandlerStore } from "../stores/types.js";
 import type { ClearanceClaims, ClearanceLevel } from "./token.js";
@@ -73,6 +74,16 @@ export interface ChallengeOptions {
      * write rather than by a heuristic here.
      */
     translations?: Record<string, ChallengeCopy>;
+    /**
+     * Ask for a deliberate gesture as well as the proof of work, and measure what the
+     * browser can actually do while waiting for it.
+     *
+     * `true` uses {@link DEFAULT_INTERACTION_SETTINGS}. Solving the puzzle alone no longer
+     * grants clearance when this is on: the gesture is required. Read
+     * `src/challenge/interaction.ts` before turning it on — in particular the part about
+     * what is and is not verifiable — because it changes who can get through your site.
+     */
+    interaction?: boolean | Partial<InteractionSettings>;
     store?: BotHandlerStore;
     clock?: Clock;
 }
@@ -85,10 +96,13 @@ export type SolutionOutcome = {
     ok: true;
     setCookie: string;
     level: ClearanceLevel;
+    interactionScore?: number;
+    notes?: readonly string[];
 } | {
     ok: false;
     status: number;
     reason: string;
+    interactionScore?: number | undefined;
 };
 /**
  * Issues challenges, verifies solutions and grants clearance.
@@ -106,6 +120,8 @@ export declare class ChallengeService {
     private readonly challengeTtlMs;
     private readonly clock;
     private readonly store;
+    /** Resolved interaction settings, or `undefined` when the gesture is not asked for. */
+    private readonly interaction;
     readonly verifyPath: string;
     readonly cookieName: string;
     /**
@@ -115,6 +131,8 @@ export declare class ChallengeService {
      */
     readonly clearanceTtlMs: number;
     constructor(options: ChallengeOptions);
+    /** Whether this service asks for a gesture as well as the puzzle. */
+    get wantsInteraction(): boolean;
     /**
      * Derives the subject a token is bound to.
      *

package/dist/challenge/interaction.d.ts

@@ -0,0 +1,215 @@
+/**
+ * The interaction challenge: what a client did on the interstitial, and what that is
+ * worth.
+ *
+ * The proof of work in `pow.ts` demonstrates that a JavaScript engine ran. That is a
+ * real cost and it is the whole of what the plain challenge asks for. This asks for
+ * something else: evidence that a *browser* rendered the page and that a *person*
+ * acted on it.
+ *
+ * ## What is actually verifiable, stated plainly
+ *
+ * Everything on this page is reported by the client, and the client is the one place an
+ * adversary has complete control. A determined attacker driving a real browser through
+ * CDP dispatches genuine input events, produces real layout, and can synthesise pointer
+ * paths from a minimum-jerk model — the literature on human motor movement is public and
+ * so are the libraries. **Nothing here is proof of humanity and none of it may ever be
+ * `certain`.**
+ *
+ * What it does is move the cost. A scraper that was `fetch()` in a loop must now run a
+ * browser engine, render CSS, and synthesise a plausible gesture per request. That is
+ * three or four orders of magnitude more expensive per page, and for most bulk
+ * extraction it is the difference between worth doing and not.
+ *
+ * Exactly one signal here is **server-verified** and cannot be fabricated: the elapsed
+ * wall-clock time between issuing the challenge and receiving the answer, which comes
+ * from the signed token's `iat` rather than from anything the client says. See
+ * {@link verifyInteraction}.
+ *
+ * ## Why the analysis lives here and not in the page
+ *
+ * If the page scored itself and posted the number, an attacker would post the number.
+ * The client sends a bounded, quantised record of what happened; every judgement about
+ * it is made on this side.
+ */
+/**
+ * The shape of one challenge's layout probe: how many boxes, and how tall each one is.
+ *
+ * Both are drawn from the nonce **under the signing secret**, which is the whole point.
+ * An earlier version fixed the height at 7px in the stylesheet and derived the count from
+ * the nonce in the page's own script, so the expected answer was a formula an attacker
+ * could read once and hardcode for ever — a client that never rendered anything answered
+ * it correctly five times out of five. Deriving it under the secret means the client
+ * cannot compute the answer at all. It can only measure it.
+ *
+ * That does not make it unforgeable. Both numbers still have to reach the browser to be
+ * rendered — the count as elements in the markup, the height as a literal in the
+ * stylesheet — so anything willing to parse the page it was served can still find them.
+ * What it removes is the *formula*: there is no longer a fixed rule to implement once and
+ * reuse. See the honesty section in docs/challenge/interaction.md.
+ */
+export interface ProbeShape {
+    boxes: number;
+    height: number;
+}
+/** Derives {@link ProbeShape} for a nonce. The secret keeps it unpredictable to the client. */
+export declare function probeShapeFor(nonce: string, secret: string): ProbeShape;
+/** One pointer sample: movement since the previous sample, and the gap in milliseconds. */
+export interface PointerSample {
+    dx: number;
+    dy: number;
+    dt: number;
+}
+/**
+ * What the interstitial observed. Every field is client-supplied and therefore a claim
+ * rather than a fact; see the note at the top of this file.
+ */
+export interface InteractionReport {
+    /**
+     * `event.isTrusted` on the activation.
+     *
+     * False for anything `element.click()` or a synthesised `MouseEvent` produces, which
+     * is what a naive automation script reaches for first. A browser driven through the
+     * DevTools protocol produces `true`, so this is a floor rather than a test.
+     */
+    trusted: boolean;
+    /**
+     * How the control was activated.
+     *
+     * `pointer` means a mouse, and is the only value from which a path is expected. `touch`
+     * covers a tap or a stylus, which legitimately produce almost no movement; `keyboard`
+     * covers the space bar, a screen reader, switch access and voice control; `other` is
+     * anything the page could not classify.
+     */
+    via: "pointer" | "touch" | "keyboard" | "other";
+    /** Milliseconds from page load to activation, as the *client* measured it. */
+    msToActivate: number;
+    /** Bounded, quantised pointer path leading up to the activation. */
+    path: readonly PointerSample[];
+    /** Results of the capability probes. See {@link CAPABILITY_WEIGHTS}. */
+    capabilities: Readonly<Record<string, boolean>>;
+    /**
+     * Measured height of the nonce-bound layout probe, in CSS pixels, or `undefined` when
+     * the client reported none. See {@link probeShapeFor}.
+     */
+    layoutHeight?: number | undefined;
+}
+/** What the movement looked like, once measured rather than asserted. */
+export interface MovementAnalysis {
+    samples: number;
+    /**
+     * Coefficient of variation of the distance covered per sample.
+     *
+     * The primary spatial measure, and the one that cannot be manufactured by accident:
+     * it reads only where the pointer went, never how fast the events arrived.
+     */
+    distanceVariation: number;
+    /** Coefficient of variation of speed. Human movement is ballistic; a lerp is flat. */
+    speedVariation: number;
+    /** Coefficient of variation of the gaps between samples. */
+    timingVariation: number;
+    /** Times the speed changed direction — sped up after slowing, or the reverse. */
+    accelerationChanges: number;
+    /** Net displacement over path length. 1 is a perfectly straight line. */
+    straightness: number;
+    /** Share of samples carrying a fractional component. */
+    fractionalShare: number;
+    /** Sum of absolute turn angles, in radians, over the whole path. */
+    totalTurning: number;
+}
+export type InteractionOutcome = {
+    ok: true;
+    level: "interaction" | "pow";
+    score: number;
+    notes: readonly string[];
+}
+/**
+ * `score` and `notes` are present whenever the refusal came from the grading rather
+ * than from a gate reached before it.
+ *
+ * Carrying them matters more than it looks. The score histogram exists so that an
+ * operator can see where to put `interactionAt`, and feeding it only from successes
+ * gives them a distribution with everything below the threshold cut out of it — which
+ * is precisely the part of the shape the decision depends on.
+ */
+ | {
+    ok: false;
+    reason: string;
+    score?: number | undefined;
+    notes?: readonly string[] | undefined;
+};
+export interface InteractionSettings {
+    /**
+     * Least time that may pass between issuing a challenge and accepting its answer, ms.
+     *
+     * Checked against the signed token rather than against anything the client reports,
+     * which makes it the one part of this file an attacker cannot lie about. It is also
+     * the part that costs them most: a floor of one second is a hard ceiling on how fast
+     * a farm can work through challenges, however many browsers it runs.
+     */
+    minElapsedMs: number;
+    /** Score at or above which the clearance is `interaction` rather than `pow`. */
+    interactionAt: number;
+    /** Score below which the answer is refused outright. */
+    refuseBelow: number;
+}
+export declare const DEFAULT_INTERACTION_SETTINGS: InteractionSettings;
+/**
+ * What each capability probe is worth.
+ *
+ * These are ordered by how hard the capability is to fake *without actually having it*.
+ * `cssApplied` is the one that matters most: it asks the page to read back a computed
+ * style that only exists if a CSSOM parsed the stylesheet and applied the cascade. An
+ * HTTP client that parses HTML has no answer to it, and that is precisely the population
+ * a challenge is aimed at.
+ *
+ * None of them is worth much on its own, and a real browser driven by a script passes
+ * every one — which is why they contribute to a score rather than to a verdict.
+ */
+export declare const CAPABILITY_WEIGHTS: Readonly<Record<string, number>>;
+/**
+ * Measures a pointer path.
+ *
+ * The statistics are chosen for what they cost to fake convincingly rather than for how
+ * well they describe a human. Anyone can add noise to a straight line; producing a path
+ * whose *speed profile* is ballistic, whose turning is concentrated near the target, and
+ * whose sample timing jitters the way a real event stream does takes deliberate work.
+ */
+export declare function analyseMovement(path: readonly PointerSample[]): MovementAnalysis;
+/**
+ * How human-shaped a measured path is, 0 to 1.
+ *
+ * Each term is a property that costs an attacker something different to reproduce, and
+ * the sum is deliberately forgiving: a person making one short, quick, straight-ish
+ * movement to a checkbox should not be punished for it. What this separates reliably is
+ * *linear interpolation* — the constant-velocity, uniform-timing, integer-coordinate
+ * path that every naive automation library produces — from anything organic.
+ */
+export declare function scoreMovement(analysis: MovementAnalysis): number;
+/** How much of the browser is really there, 0 to 1. */
+export declare function scoreCapabilities(capabilities: Readonly<Record<string, boolean>>): number;
+/**
+ * Reads an interaction report off the wire.
+ *
+ * Every field is attacker-supplied, so this is written to be total: anything malformed
+ * becomes `undefined` rather than throwing, and every array and number is bounded before
+ * it reaches the analysis. A report that arrives as a two-megabyte array of NaN is a
+ * request to burn CPU, not an interaction.
+ */
+export declare function parseInteractionReport(value: unknown): InteractionReport | undefined;
+/**
+ * Decides what an interaction report is worth.
+ *
+ * `elapsedMs` is measured by the server from the signed challenge token, and it is the
+ * only argument here that the client cannot influence.
+ *
+ * ## Why a keyboard activation is not penalised
+ *
+ * A pointer path is worth something, and its absence is worth nothing either way. Voice
+ * control produces no pointer movement; switch access produces machine-regular timing;
+ * a screen reader activates the control from the keyboard. Scoring those down would put
+ * assistive technology on the wrong side of a check the rest of this library exists to
+ * keep people out of, so a keyboard or `other` activation is graded on its capabilities
+ * and its timing alone.
+ */
+export declare function verifyInteraction(report: InteractionReport | undefined, elapsedMs: number, settings?: InteractionSettings, expected?: ProbeShape): InteractionOutcome;

package/dist/challenge/page.d.ts

@@ -13,6 +13,24 @@ export interface ChallengePageOptions {
     contactHtml?: string;
     /** Page language for the `lang` attribute. Default "en". */
     lang?: string;
+    /**
+     * Ask for a deliberate gesture as well as the proof of work, and probe what the
+     * browser can actually do while waiting for it.
+     *
+     * The control is a plain checkbox for a reason. It is the one interactive element that
+     * every way of using a computer can operate — pointer, touch, the space bar, a screen
+     * reader, switch access, voice control — so requiring it excludes far less than a
+     * drag, a puzzle or anything needing visual acuity would.
+     */
+    interaction?: boolean;
+    /**
+     * Shape of this challenge's layout probe. Rendered into the markup and the stylesheet
+     * so that answering it requires laying it out; see `probeShapeFor`.
+     */
+    probe?: {
+        boxes: number;
+        height: number;
+    };
 }
 export interface RenderedChallenge {
     html: string;