@sdxc/spam 2026.10.9 → 2026.10.13

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (4) hide show
  1. package/README.md +206 -152
  2. package/dist/lib/text.js +12 -2
  3. package/dist/rules/link-syntax.js +68 -9
  4. package/package.json +5 -5
package/README.md CHANGED
@@ -1,24 +1,23 @@
1
1
  # @sdxc/spam
2
2
 
3
- Spam scoring for user-generated submissions: local rules, reputation providers, a trainable
4
- classifier and an LLM second opinion add weighted signals, and the total becomes a `ham`,
5
- `unsure` or `spam` verdict with every reason attached.
3
+ Spam scoring for user content: local rules, reputation checks and a classifier add weighted
4
+ signals, and the total becomes a `ham`, `unsure` or `spam` verdict with every reason attached.
6
5
 
7
6
  ## Installation
8
7
 
9
- ```bash
8
+ ```sh
10
9
  npm add @sdxc/spam
11
10
  ```
12
11
 
13
- Remote checks and moderator reports return [`@sdxc/result`](https://www.npmjs.com/package/@sdxc/result)
14
- values, and the classifier's database store runs on
15
- [`remix/data-table`](https://www.npmjs.com/package/remix). Both install alongside this package.
12
+ Remote checks and stores answer [`@sdxc/result`](https://www.npmjs.com/package/@sdxc/result)
13
+ values, and the classifier's database store runs on `remix/data-table` from
14
+ [`remix`](https://www.npmjs.com/package/remix).
16
15
 
17
16
  ## Usage
18
17
 
19
18
  ### Score a submission
20
19
 
21
- ```ts
20
+ ```typescript
22
21
  import { createSpamFilter, DEFAULT_RULES } from "@sdxc/spam";
23
22
 
24
23
  let filter = createSpamFilter({ checks: DEFAULT_RULES });
@@ -28,105 +27,103 @@ let assessment = await filter.check({
28
27
  });
29
28
  assessment.verdict; // "ham" | "unsure" | "spam"
30
29
  assessment.score; // 5
31
- assessment.signals; // [{ check: "links.count", score: 2, detail: "2 links" }, { check: "links.density", score: 3, … }]
30
+ assessment.signals; // [{ check: "links.count", score: 2, detail: "2 links" }, …]
32
31
  ```
33
32
 
34
- ### Give the rules more to work with
33
+ ### Describe the submission
35
34
 
36
35
  Only `content` is required. Every other field lets more rules fire.
37
36
 
38
- ```ts
37
+ ```typescript
39
38
  let assessment = await filter.check({
40
- content: form.get("comment"),
39
+ content: String(form.get("comment") ?? ""),
41
40
  format: "markdown",
42
41
  author: {
43
- name: form.get("name"),
44
- email: form.get("email"),
45
- ip: request.headers.get("CF-Connecting-IP"),
42
+ name: String(form.get("name") ?? ""),
43
+ email: String(form.get("email") ?? ""),
44
+ url: String(form.get("website") ?? ""),
45
+ ip: request.headers.get("CF-Connecting-IP") ?? undefined,
46
+ userAgent: request.headers.get("User-Agent") ?? undefined,
46
47
  },
47
- renderedAt: verifiedRenderTime, // signed at render, as @sdxc/honeypot does; never a plain hidden input
48
+ renderedAt, // from a signed field the visitor cannot forge
48
49
  languages: ["en", "es"],
49
50
  });
50
51
  ```
51
52
 
52
- ### Add reputation and a second opinion
53
+ ### Add reputation checks
53
54
 
54
- ```ts
55
+ ```typescript
55
56
  import { createSpamFilter, DEFAULT_RULES } from "@sdxc/spam";
56
57
  import { akismet } from "@sdxc/spam/akismet";
57
58
  import { authorEmail } from "@sdxc/spam/author-email";
59
+ import { linkBlocklist } from "@sdxc/spam/link-blocklist";
60
+ import { stopForumSpam } from "@sdxc/spam/stop-forum-spam";
58
61
 
59
62
  let filter = createSpamFilter({
60
63
  checks: [
61
64
  ...DEFAULT_RULES,
62
65
  authorEmail(),
63
- akismet({ apiKey: AKISMET_KEY, blog: "https://example.com" }),
66
+ stopForumSpam(),
67
+ linkBlocklist(),
68
+ akismet({ apiKey: env.AKISMET_KEY, blog: "https://example.com" }),
64
69
  ],
65
70
  thresholds: { unsure: 5, spam: 10 },
66
71
  timeout: 1500,
67
72
  });
68
73
  ```
69
74
 
70
- Local rules run first. When they already reach the spam threshold, no request leaves the Worker.
75
+ Local checks run first, in order. Remote checks run in parallel only while the score is below
76
+ the spam threshold, and a check that fails or overruns `timeout` adds nothing and is listed in
77
+ `assessment.failures`.
71
78
 
72
79
  ### Learn from moderators
73
80
 
74
- ```ts
75
- let failures = await filter.report(submission, "ham"); // a moderator approved a held comment
81
+ ```typescript
82
+ let failures = await filter.report(submission, "ham");
76
83
  ```
77
84
 
78
- Every check that learns receives the decision; `failures` lists the ones that could not record it.
85
+ Every check with a `report` method receives the decision; `failures` lists the ones that could
86
+ not record it.
79
87
 
80
88
  ## API
81
89
 
82
90
  ### `@sdxc/spam`
83
91
 
84
- #### `createSpamFilter({ checks, thresholds?, timeout? })`
92
+ #### `createSpamFilter(options: SpamFilter.Options): SpamFilter`
85
93
 
86
- Returns a `SpamFilter` with two methods:
94
+ Builds a filter over `options.checks`, with `thresholds` (default `DEFAULT_THRESHOLDS`) and a
95
+ `timeout` in milliseconds for each remote and escalation check (default 1500).
87
96
 
88
- - `check(submission)` resolves a `SpamFilter.Assessment`: `{ verdict, score, signals, failures }`.
89
- Checks run in three stages: `local` in order; `remote` in parallel, only while the score is below
90
- the spam threshold; `escalation` in parallel, only while the score is inside the unsure band.
91
- A check that fails, throws or overruns `timeout` (default 1500 ms, remote and escalation only)
92
- adds nothing and is listed in `failures`, so an outage never decides a verdict.
93
- - `report(submission, label)` sends `"spam"` or `"ham"` to every check with a `report` method, in
94
- parallel, and resolves the ones that failed.
97
+ #### `filter.check(submission: Submission): Promise<SpamFilter.Assessment>`
95
98
 
96
- `thresholds` defaults to `{ unsure: 5, spam: 10 }` (`DEFAULT_THRESHOLDS`). A score at or above
97
- `spam` is spam, at or above `unsure` is unsure, and anything lower is ham.
99
+ Runs `local` checks in order, then `remote` checks in parallel while the score is below `spam`,
100
+ then `escalation` checks in parallel while it sits in the unsure band. A check that fails,
101
+ throws or times out adds nothing and is listed in `failures`.
98
102
 
99
- #### `verdictFor(score, thresholds)`
103
+ #### `filter.report(submission: Submission, label: Label): Promise<SpamFilter.Failure[]>`
100
104
 
101
- The mapping `check` uses, for re-scoring stored signals under new thresholds.
105
+ Sends a moderator's decision to every check with a `report` method, in parallel, and resolves
106
+ the ones that failed.
102
107
 
103
- #### `SpamCheck`
108
+ #### `verdictFor(score: number, thresholds: SpamFilter.Thresholds): SpamFilter.Verdict`
104
109
 
105
- The contract every check implements: `name`, `stage` (`"local" | "remote" | "escalation"`),
106
- `check(submission, { signal, score })` and an optional `report(submission, label)`. A local check
107
- returns `Signal[]`; any other returns `Promise<Result<Signal[], SpamCheckError>>`. `signal` aborts
108
- at the timeout, and `score` is the total earlier stages reached.
110
+ The mapping `check` uses: at or above `spam` is spam, at or above `unsure` is unsure, below is
111
+ ham. Use it to re-score stored signals under new thresholds.
109
112
 
110
- #### `Signal`
113
+ #### `DEFAULT_THRESHOLDS`
111
114
 
112
- `{ check, score, detail? }`. `check` is a stable identifier such as `links.count`. A positive score
113
- points to spam and a negative one to ham, so a provider can vouch for an author.
114
-
115
- #### `Submission`
116
-
117
- `{ content, format?, author?, renderedAt?, submittedAt?, languages? }`. `format` is `"text"` (the
118
- default), `"markdown"` or `"html"`; `author` holds `name`, `email`, `url`, `ip` and `userAgent`;
119
- `submittedAt` defaults to the time of the check; `languages` are BCP 47 tags.
115
+ `{ unsure: 5, spam: 10 }`.
120
116
 
121
117
  #### `SpamCheckError`
122
118
 
123
- A check's failure. `code` is `unavailable`, `timeout`, `misconfigured` or `invalid-response`.
119
+ A check's failure. `code` is `unavailable`, `timeout`, `misconfigured` (a key or option the
120
+ provider refused) or `invalid-response`; the message carries the provider's detail.
124
121
 
125
122
  #### Rules
126
123
 
127
- Each rule is a function returning a local `SpamCheck`; every weight and limit is an option with a
128
- documented default. `DEFAULT_RULES` holds all of them at their defaults, tuned together so a
129
- submission needs more than one kind of evidence to reach the spam threshold.
124
+ Each rule is a function taking an options object of weights and limits and returning a `local`
125
+ `SpamCheck`. `DEFAULT_RULES` holds every rule at its defaults, tuned together so a submission
126
+ needs more than one kind of evidence to reach the spam threshold.
130
127
 
131
128
  | Rule | Signals | Fires on |
132
129
  | --------------- | ------------------------------------------------------------------- | ------------------------------------------------------------------------------------ |
@@ -140,125 +137,134 @@ submission needs more than one kind of evidence to reach the spam threshold.
140
137
  | `language()` | `language.script` | Most letters in a script none of `languages` is written in |
141
138
  | `authorName()` | `author-name.domain`, `author-name.length` | A domain in the name, or a name longer than a person's |
142
139
 
143
- `URL_SHORTENERS` and `ABUSED_TLDS` are the lists `linkTargets()` uses by default.
140
+ `URL_SHORTENERS` and `ABUSED_TLDS` are the lists `linkTargets()` uses unless `shorteners` or
141
+ `abusedTlds` replace them.
142
+
143
+ #### Types
144
+
145
+ - `Submission`: `{ content, format?, author?, renderedAt?, submittedAt?, languages? }`. `format` is
146
+ `"text"` (default), `"markdown"` or `"html"`; `author` holds `name`, `email`, `url`, `ip` and
147
+ `userAgent`; `submittedAt` defaults to the time of the check; `languages` are BCP 47 tags.
148
+ - `Signal`: `{ check, score, detail? }`. A positive score points to spam and a negative one to
149
+ ham, so a provider can vouch for an author.
150
+ - `SpamCheck`: `name`, `stage` (`"local" | "remote" | "escalation"`), a `check(submission, options)`
151
+ answering `Signal[]` or `Promise<Result<Signal[], SpamCheckError>>`, and an optional
152
+ `report(submission, label)`. `options.signal` aborts at the timeout; `options.score` and
153
+ `options.signals` are what earlier stages found.
154
+ - `SpamFilter`: the filter, with `SpamFilter.Options`, `Thresholds`, `Verdict`, `Failure`
155
+ (`{ check, error }`) and `Assessment` (`{ verdict, score, signals, failures }`).
156
+ - `Label`: `"spam" | "ham"`.
144
157
 
145
158
  ### `@sdxc/spam/author-email`
146
159
 
147
- `authorEmail({ disposableScore? })` scores an `author.email` on a disposable-mail domain
148
- (`author-email.disposable`, default 3), subdomains included. It ships separately because the
149
- domain list it bundles is large, so it is not in `DEFAULT_RULES`. An address that does not parse
150
- draws nothing.
160
+ #### `authorEmail(options?: { disposableScore?: number }): SpamCheck`
161
+
162
+ A local rule scoring an `author.email` on a disposable-mail domain, subdomains included
163
+ (`author-email.disposable`, default 3). It ships apart from `DEFAULT_RULES` because the domain
164
+ list it bundles is large.
151
165
 
152
166
  ### `@sdxc/spam/stop-forum-spam`
153
167
 
154
- `stopForumSpam(options?)` looks the author's IP, email and name up in
155
- [StopForumSpam](https://www.stopforumspam.com/usage), a free community database of addresses
156
- reported for forum and blog spam. It sends only the fields the submission has, and nothing when it
157
- has none. Each reported field scores its weight times the reported confidence (`ipScore` 5,
158
- `emailScore` 6, `usernameScore` 2), reports last seen over `maxAgeDays` (90) ago are ignored, and
159
- the total is capped at `maxScore` (10). Signals are `stop-forum-spam.ip`, `.email` and
160
- `.username`. `endpoint` pins a regional mirror. The service has a daily query limit per IP and
161
- receives the author's IP, email and name.
168
+ #### `stopForumSpam(options?: stopForumSpam.Options): SpamCheck`
169
+
170
+ A remote check looking the author's IP, email and name up in
171
+ [StopForumSpam](https://www.stopforumspam.com/usage). Each reported field scores its weight
172
+ times the reported confidence (`ipScore` 5, `emailScore` 6, `usernameScore` 2), reports older
173
+ than `maxAgeDays` (90) are ignored, and the total caps at `maxScore` (10). Signals are
174
+ `stop-forum-spam.ip`, `.email` and `.username`. `endpoint` (default `STOP_FORUM_SPAM_ENDPOINT`)
175
+ pins a regional mirror.
162
176
 
163
177
  ### `@sdxc/spam/link-blocklist`
164
178
 
165
- `linkBlocklist(options?)` looks every linked domain and the author's website up in DNS URI
166
- blocklists over DNS-over-HTTPS, so it runs where no DNS socket exists. Zones are `spamhaus`,
167
- `surbl` and `uribl`; the default is `["surbl", "uribl"]`, and a `dqsKey` adds Spamhaus. A listing
168
- scores `listedScore` (6); a low-confidence one (URIBL grey, a Spamhaus "abused legitimate" domain)
169
- scores `weakScore` (2). At most `maxHosts` (10) domains are looked up, IP literals are skipped, and
170
- the total is capped at `maxScore` (12). Signals are `link-blocklist.spamhaus`, `.surbl` and
171
- `.uribl`. `resolver` routes queries through another DoH endpoint.
172
-
173
- A blocklist's "query refused" or rate-limit answer fails as `misconfigured`, never as a listing.
174
- Spamhaus refuses public resolvers, so it needs a
175
- [Data Query Service](https://www.spamhaus.com/product/data-query-service/) key, which travels in the
176
- query name. SURBL and URIBL are free for low-volume, non-commercial use and may refuse queries
177
- relayed through large public resolvers; when they do, the check reports `misconfigured`, and a
178
- private resolver or a commercial feed is the fix.
179
+ #### `linkBlocklist(options?: linkBlocklist.Options): SpamCheck`
180
+
181
+ A remote check looking every linked domain and `author.url` up in DNS URI blocklists over
182
+ DNS-over-HTTPS. `zones` defaults to `["surbl", "uribl"]`; a Spamhaus
183
+ [Data Query Service](https://www.spamhaus.com/product/data-query-service/) `dqsKey` adds
184
+ `"spamhaus"`. A listing scores `listedScore` (6), a low-confidence one `weakScore` (2), across
185
+ at most `maxHosts` (10) domains, capped at `maxScore` (12). Signals are
186
+ `link-blocklist.spamhaus`, `.surbl` and `.uribl`. A blocklist refusing the query fails as
187
+ `misconfigured`; `resolver` routes queries through a DoH resolver the zones accept.
179
188
 
180
189
  ### `@sdxc/spam/akismet`
181
190
 
182
- `akismet({ apiKey, blog })` sends each submission to Akismet's `comment-check` and turns the answer
183
- into a signal: `akismet.spam` (default 8, the unsure band), `akismet.discard` when Akismet marks it
184
- blatant (default 12, spam on its own), and `akismet.ham` when Akismet vouches for it (default -2).
185
- Akismet requires the author's IP, so a submission without `author.ip` sends nothing and draws
186
- nothing. `commentType` (default `"comment"`), `blogLang` and `isTest` pass through; the weights are
187
- `spamScore`, `discardScore` and `hamScore`. An invalid key fails as `misconfigured`, an HTTP or
188
- network error as `unavailable`. `report` forwards a moderator's decision to `submit-spam` or
189
- `submit-ham`, bounded by `reportTimeout` (default 10 s).
191
+ #### `akismet(options: akismet.Options): SpamCheck`
190
192
 
191
- Akismet is free for personal, non-commercial sites and paid otherwise. Each call sends the author's
192
- IP, user agent, name, email, website and the full content.
193
+ A remote check sending each submission to Akismet with `apiKey` and `blog`. It adds
194
+ `akismet.spam` (`spamScore` 8), `akismet.discard` for blatant spam (`discardScore` 12) or
195
+ `akismet.ham` (`hamScore` -2). Akismet requires `author.ip`; a submission without one draws
196
+ nothing. `commentType` (default `"comment"`), `blogLang` and `isTest` pass through, and `report`
197
+ forwards decisions to `submit-spam` / `submit-ham` within `reportTimeout` (10000 ms).
193
198
 
194
199
  ### `@sdxc/spam/bayes`
195
200
 
196
- `bayes({ store })` is a naive Bayes classifier that learns from moderators: every
197
- `filter.report(submission, label)` trains it. It tokenizes content words, a `host:` token per
198
- linked host and the author's website, and an `email-domain:` token, then combines the most telling
199
- tokens (Robinson's smoothed probabilities, Fisher's method) into a spam probability. At or above
200
- `spamCut` (0.9) it adds `bayes.spam` (`spamScore`, 6); at or below `hamCut` (0.2) it adds
201
- `bayes.ham` (`hamScore`, -4). Until the store holds `minDocuments` (20) reports of each label it
202
- adds nothing, so a new install relies on the rules alone. The signal's detail names the tokens
203
- that decided it. `tokenize(submission, options?)` is exported for inspecting what it learns.
201
+ #### `bayes(options: bayes.Options): SpamCheck`
202
+
203
+ A local naive Bayes classifier trained by every `filter.report`. It adds `bayes.spam`
204
+ (`spamScore` 6) at or above `spamCut` (0.9) and `bayes.ham` (`hamScore` -4) at or below `hamCut`
205
+ (0.2), naming the deciding tokens in the detail, and stays silent until `store` holds
206
+ `minDocuments` (20) reports of each label. `strength`, `prior`, `interestingTokens` and
207
+ `minDeviation` tune the combination; the `tokenize` options shape the tokens.
208
+
209
+ #### `tokenize(submission: Submission, options?: tokenize.Options): string[]`
210
+
211
+ The tokens the classifier counts: content words between `minLength` (3) and `maxLength` (24),
212
+ a `host:` token per linked host and the author's website, and an `email-domain:` token unless
213
+ `emailDomain` is `false`, up to `maxTokens` (200).
204
214
 
205
- Counts live behind `TokenStore` (`read(tokens)` and `increment(tokens, label)`, both `Result`s):
215
+ #### `TokenStore`
206
216
 
207
- - `MemoryTokenStore` keeps counts in the instance, for tests.
208
- - `new DataTableTokenStore(db)` stores them in a `remix/data-table` database, D1 and SQLite
209
- included. Create its table from `SPAM_TOKENS_SCHEMA_SQL` in a migration; `spamTokens` is the
210
- table definition. Every write is one upsert statement, so concurrent reports never lose a count.
217
+ Where counts live: `read(tokens)` and `increment(tokens, label)`, both answering a `Result`.
218
+ `TokenStore.Counts` and `TokenStore.Tally` are the shapes `read` answers.
211
219
 
212
- In a multi-tenant app, build one store per tenant database, so one tenant's training never changes
213
- another's verdicts. The check runs in the local stage, since it reads the app's own database.
220
+ #### `MemoryTokenStore`
221
+
222
+ Keeps counts in the instance, for tests and local development.
223
+
224
+ #### `new DataTableTokenStore(db: Database)`
225
+
226
+ Stores counts in the `spamTokens` table of a `remix/data-table` database, one upsert per write,
227
+ so concurrent reports never lose a count. `SPAM_TOKENS_SCHEMA_SQL` creates the table in a
228
+ migration; `SpamTokenRow` is its row type.
214
229
 
215
230
  ### `@sdxc/spam/workers-ai`
216
231
 
217
- `workersAi({ ai, model?, site?, spamScore?, hamScore?, maxContentLength?, maxReasonLength? })` is
218
- an escalation check: the filter calls it only for submissions the other checks left unsure, so the
219
- model runs on the few cases rules cannot settle. Pass the Workers AI binding (`env.AI`) as `ai`,
220
- and describe the site in `site` (`"comments on a cooking blog"`) so the model judges relevance for
221
- the right audience.
222
-
223
- The check sends the content (cut to `maxContentLength`, default 4000 characters), the author's name
224
- and URL, and up to 20 of the signals earlier checks found, as escaped JSON inside delimiters, under
225
- a system prompt that treats everything inside as data to classify. The model answers `spam`, `ham`
226
- or `unsure` with a confidence and a reason: spam adds `workers-ai.spam` at `spamScore × confidence`
227
- (default 5, enough to carry an unsure 5 to 10), ham adds `workers-ai.ham` at
228
- `-(hamScore × confidence)` (default 5), and unsure adds nothing. The reason, cut to
229
- `maxReasonLength` (200), is the signal's detail. The default model is
230
- `@cf/meta/llama-3.1-8b-instruct-fp8` (`DEFAULT_WORKERS_AI_MODEL`); `model` takes any Workers AI
231
- text model with JSON mode. `ai` is typed as any object with a matching `run()`, so a test passes a
232
- plain object.
232
+ #### `workersAi(options: workersAi.Options): SpamCheck`
233
+
234
+ An escalation check asking a Workers AI model about submissions left in the unsure band. Pass
235
+ the binding as `ai` (any object with a matching `run`, `workersAi.Binding`) and describe the
236
+ audience in `site`. A spam answer adds `workers-ai.spam` at `spamScore × confidence` (default 5),
237
+ a ham answer `workers-ai.ham` at `-(hamScore × confidence)` (default 5), with the model's reason
238
+ cut to `maxReasonLength` (200) as the detail. Content is cut to `maxContentLength` (4000).
239
+ `model` defaults to `DEFAULT_WORKERS_AI_MODEL`, and takes any text model with JSON mode.
233
240
 
234
241
  ### `@sdxc/spam/memory`
235
242
 
236
- `new MemoryCheck({ name?, stage?, signals? })` is a scripted check for tests. A call answers the
237
- next queued outcome, else `signals`.
243
+ #### `new MemoryCheck(options?: MemoryCheck.Options)`
244
+
245
+ A scripted check for tests, named `"memory"` in the `"remote"` stage unless `name` and `stage`
246
+ say otherwise. A call answers the next queued outcome, else `signals`.
238
247
 
239
- - `signalNext(signals)` / `failNext(code, message?)` queue the next answer.
248
+ - `signalNext(signals)` / `failNext(code, message?)` queue the next answer; both chain.
240
249
  - `failReports(code)` makes every later `report` fail.
241
250
  - `calls`, `last` and `reports` record what the check received.
242
251
  - `reset()` forgets calls, reports and queued outcomes.
243
252
 
244
- `name` defaults to `"memory"` and `stage` to `"remote"`.
245
-
246
253
  ## Pattern: Refuse bots before scoring
247
254
 
248
- A honeypot refuses what no person produces, before the body is read into a submission, so no
249
- rule, lookup or model call runs for it. [`@sdxc/honeypot`](https://www.npmjs.com/package/@sdxc/honeypot)
250
- does that, and what it lets through carries a signed render time for the `timing` rule.
255
+ [`@sdxc/honeypot`](https://www.npmjs.com/package/@sdxc/honeypot) refuses what no person
256
+ produces before any rule runs, and hands the signed render time the `timing` rule reads.
251
257
 
252
- ```ts
258
+ ```typescript
253
259
  import { Honeypot } from "@sdxc/honeypot";
254
260
  import { honeypot } from "@sdxc/honeypot/middleware";
255
261
  import { unwrap } from "@sdxc/result";
256
262
  import { createSpamFilter, DEFAULT_RULES } from "@sdxc/spam";
257
263
 
258
- let trap = new Honeypot({ secret: HONEYPOT_SECRET });
264
+ let trap = new Honeypot({ secret: env.HONEYPOT_SECRET });
259
265
  let filter = createSpamFilter({ checks: DEFAULT_RULES });
260
266
 
261
- router.post("/comments", {
267
+ router.post(routes.comments.create, {
262
268
  middleware: [honeypot(trap)],
263
269
  async handler(ctx) {
264
270
  let form = await ctx.request.formData();
@@ -273,10 +279,10 @@ router.post("/comments", {
273
279
 
274
280
  ## Pattern: Hold, discard or publish
275
281
 
276
- The verdict is a recommendation; the app decides what each one does. Answering spam like a success
277
- teaches a bot nothing.
282
+ The verdict is a recommendation; the app decides what each one does. Answering spam like a
283
+ success teaches a bot nothing.
278
284
 
279
- ```ts
285
+ ```typescript
280
286
  import type { Submission } from "@sdxc/spam";
281
287
 
282
288
  import { createSpamFilter, DEFAULT_RULES } from "@sdxc/spam";
@@ -294,12 +300,56 @@ async function submitComment(submission: Submission) {
294
300
  }
295
301
  ```
296
302
 
297
- ## Pattern: Writing a rule for your site
303
+ ## Pattern: Train a classifier on your moderation queue
304
+
305
+ The classifier reads the site's own history, so build it per request from that tenant's
306
+ database, and send every moderator decision back through `report`.
307
+
308
+ ```typescript
309
+ import type { Database } from "remix/data-table";
310
+ import type { Label, Submission } from "@sdxc/spam";
311
+
312
+ import { createSpamFilter, DEFAULT_RULES } from "@sdxc/spam";
313
+ import { bayes, DataTableTokenStore } from "@sdxc/spam/bayes";
314
+
315
+ function spamFilter(db: Database) {
316
+ return createSpamFilter({
317
+ checks: [...DEFAULT_RULES, bayes({ store: new DataTableTokenStore(db) })],
318
+ });
319
+ }
320
+
321
+ async function moderate(db: Database, submission: Submission, label: Label) {
322
+ let failures = await spamFilter(db).report(submission, label);
323
+ return failures.map(({ check, error }) => `${check}: ${error.code}`);
324
+ }
325
+ ```
326
+
327
+ Create the table in a migration from `SPAM_TOKENS_SCHEMA_SQL`, and store each submission
328
+ exactly as it was checked, so training sees the same tokens.
329
+
330
+ ## Pattern: Escalate the unsure band to a model
331
+
332
+ Rules settle most submissions; a model call runs only on the ones they leave unsure.
333
+
334
+ ```typescript
335
+ import { createSpamFilter, DEFAULT_RULES } from "@sdxc/spam";
336
+ import { workersAi } from "@sdxc/spam/workers-ai";
337
+
338
+ let filter = createSpamFilter({
339
+ checks: [...DEFAULT_RULES, workersAi({ ai: env.AI, site: "comments on a cooking blog" })],
340
+ timeout: 3000,
341
+ });
342
+
343
+ let assessment = await filter.check({ content: "Great recipe! Visit my store for cheap pans" });
344
+ assessment.signals.find((signal) => signal.check.startsWith("workers-ai."))?.detail;
345
+ ```
346
+
347
+ ## Pattern: Write a rule for your site
298
348
 
299
- A rule is an object with a name, a stage and a `check` function. Signals from it add to every
349
+ A check is an object with a name, a stage and a `check` function; its signals add to every
300
350
  other rule's.
301
351
 
302
- ```ts
352
+ ```typescript
303
353
  import type { SpamCheck } from "@sdxc/spam";
304
354
 
305
355
  import { createSpamFilter, DEFAULT_RULES } from "@sdxc/spam";
@@ -316,38 +366,42 @@ let competitorLinks: SpamCheck = {
316
366
  let filter = createSpamFilter({ checks: [...DEFAULT_RULES, competitorLinks] });
317
367
  ```
318
368
 
319
- ## Pattern: Reweighting a rule
369
+ ## Pattern: Reweight a rule
320
370
 
321
- `DEFAULT_RULES` is a list of rule calls at their defaults. Build your own list to change one.
371
+ `DEFAULT_RULES` is the rule functions called with defaults. Swap one for your own call.
322
372
 
323
- ```ts
324
- import { contactBait, createSpamFilter, DEFAULT_RULES } from "@sdxc/spam";
373
+ ```typescript
374
+ import { contactBait, createSpamFilter, DEFAULT_RULES, links } from "@sdxc/spam";
325
375
 
326
376
  let filter = createSpamFilter({
327
377
  checks: [
328
- ...DEFAULT_RULES.filter((rule) => rule.name !== "contact-bait"),
329
- contactBait({ phoneScore: 0 }), // this form asks for a phone number
378
+ ...DEFAULT_RULES.filter((rule) => rule.name !== "contact-bait" && rule.name !== "links"),
379
+ contactBait({ phoneScore: 0 }), // the form asks for a phone number
380
+ links({ freeLinks: 3 }),
330
381
  ],
331
382
  });
332
383
  ```
333
384
 
334
- ## Pattern: Testing an action that filters
385
+ ## Pattern: Test code that filters
335
386
 
336
- `MemoryCheck` fixes the evidence, so a test exercises each verdict without a provider.
387
+ `MemoryCheck` fixes the evidence, so a test reaches every verdict and failure without a
388
+ provider. `MemoryTokenStore` does the same for `bayes`.
337
389
 
338
- ```ts
390
+ ```typescript
339
391
  import { createSpamFilter } from "@sdxc/spam";
340
392
  import { MemoryCheck } from "@sdxc/spam/memory";
341
393
  import { expect, test } from "vitest";
342
394
 
343
- test("holds an unsure comment for moderation", async () => {
344
- let check = new MemoryCheck({ signals: [{ check: "test", score: 6 }] });
345
- let filter = createSpamFilter({ checks: [check] });
395
+ test("holds an unsure comment and survives a provider outage", async () => {
396
+ let rules = new MemoryCheck({ stage: "local", signals: [{ check: "test", score: 6 }] });
397
+ let provider = new MemoryCheck().failNext("unavailable");
398
+ let filter = createSpamFilter({ checks: [rules, provider] });
346
399
 
347
400
  let assessment = await filter.check({ content: "hello" });
348
401
 
349
402
  expect(assessment.verdict).toBe("unsure");
350
- expect(check.last).toEqual({ content: "hello" });
403
+ expect(assessment.failures[0]?.error.code).toBe("unavailable");
404
+ expect(provider.last).toEqual({ content: "hello" });
351
405
  });
352
406
  ```
353
407
 
package/dist/lib/text.js CHANGED
@@ -11,7 +11,7 @@
11
11
  */
12
12
  const URL_PATTERN = /\b(?:https?:\/\/(?:\[[0-9a-f:.]+\])?|www\.)[^\s"'<>()[\]]*/gi;
13
13
  /** Punctuation a sentence puts after a URL, which is never part of it. */
14
- const TRAILING_PUNCTUATION = /[.,;:!?]+$/;
14
+ const TRAILING_PUNCTUATION = new Set([".", ",", ";", ":", "!", "?"]);
15
15
  /**
16
16
  * The distinct links in `text`, parsed. A `www.` link is read as `http://`; a match that fails to
17
17
  * parse is dropped, so every entry has a host.
@@ -22,7 +22,7 @@ export function extractLinks(text) {
22
22
  let seen = new Set();
23
23
  let links = [];
24
24
  for (let match of text.matchAll(URL_PATTERN)) {
25
- let raw = match[0].replace(TRAILING_PUNCTUATION, "");
25
+ let raw = trimTrailingPunctuation(match[0]);
26
26
  let href = /^www\./i.test(raw) ? `http://${raw}` : raw;
27
27
  if (!URL.canParse(href))
28
28
  continue;
@@ -34,6 +34,16 @@ export function extractLinks(text) {
34
34
  }
35
35
  return links;
36
36
  }
37
+ /**
38
+ * `raw` without the sentence punctuation that ends it. Walking back from the end keeps the cost
39
+ * linear in the match however long a run of punctuation a submission packs into it.
40
+ */
41
+ function trimTrailingPunctuation(raw) {
42
+ let end = raw.length;
43
+ while (end > 0 && TRAILING_PUNCTUATION.has(raw.charAt(end - 1)))
44
+ end -= 1;
45
+ return raw.slice(0, end);
46
+ }
37
47
  /** The words of `text` with its links removed, split on anything that is not a letter or digit. */
38
48
  export function extractWords(text) {
39
49
  return text
package/dist/rules/link-syntax.js CHANGED
@@ -1,9 +1,68 @@
1
- /** A BBCode link, `[url=…]` or `[url]…[/url]`. */
2
- const BBCODE_LINK = /\[url(?:=[^\]]*)?\]/i;
3
- /** An HTML anchor with an `href`. */
4
- const HTML_LINK = /<a\s[^>]*href\s*=/i;
5
- /** A Markdown inline link to an absolute URL. */
6
- const MARKDOWN_LINK = /\[[^\]]+\]\(\s*https?:\/\//i;
1
+ /** The opening of a BBCode link, `[url]` or `[url=`; the latter needs a `]` somewhere after it. */
2
+ const BBCODE_OPEN = /\[url[=\]]/i;
3
+ /** The opening of an HTML anchor, tried only where a `<` stands. */
4
+ const ANCHOR_OPEN = /<a\s/iy;
5
+ /** An `href` attribute, tried only inside an open anchor tag. */
6
+ const HREF_ATTRIBUTE = /href\s*=/iy;
7
+ /** The absolute-URL target that turns a bracketed label into a Markdown link. */
8
+ const MARKDOWN_TARGET = /\(\s*https?:\/\//iy;
9
+ /**
10
+ * Whether `content` holds a BBCode link: `[url]`, or `[url=` with a `]` after it. Only the first
11
+ * opening is checked, since a `]` after any later one also follows the first.
12
+ */
13
+ function hasBbcodeLink(content) {
14
+ let open = BBCODE_OPEN.exec(content);
15
+ if (open === null)
16
+ return false;
17
+ return open[0].endsWith("]") || content.includes("]", open.index + open[0].length);
18
+ }
19
+ /**
20
+ * Whether an HTML anchor tag in `content` carries an `href`. One pass tracks whether the scan is
21
+ * inside an anchor tag opened since the last `>`, so the cost stays linear in the submission.
22
+ */
23
+ function hasHtmlLink(content) {
24
+ let inAnchor = false;
25
+ for (let at = 0; at < content.length; at += 1) {
26
+ let character = content.charAt(at);
27
+ if (character === ">") {
28
+ inAnchor = false;
29
+ }
30
+ else if (character === "<") {
31
+ ANCHOR_OPEN.lastIndex = at;
32
+ if (ANCHOR_OPEN.test(content))
33
+ inAnchor = true;
34
+ }
35
+ else if (inAnchor && (character === "h" || character === "H")) {
36
+ HREF_ATTRIBUTE.lastIndex = at;
37
+ if (HREF_ATTRIBUTE.test(content))
38
+ return true;
39
+ }
40
+ }
41
+ return false;
42
+ }
43
+ /**
44
+ * Whether `content` holds a Markdown inline link to an absolute URL: a non-empty `[label]` followed
45
+ * by `(http…`. One pass remembers the first `[` since the last `]`, so the cost stays linear.
46
+ */
47
+ function hasMarkdownLink(content) {
48
+ let open = -1;
49
+ for (let at = 0; at < content.length; at += 1) {
50
+ let character = content.charAt(at);
51
+ if (character === "[") {
52
+ if (open === -1)
53
+ open = at;
54
+ }
55
+ else if (character === "]") {
56
+ if (open !== -1 && at - open > 1) {
57
+ MARKDOWN_TARGET.lastIndex = at + 1;
58
+ if (MARKDOWN_TARGET.test(content))
59
+ return true;
60
+ }
61
+ open = -1;
62
+ }
63
+ }
64
+ return false;
65
+ }
7
66
  /**
8
67
  * Scores BBCode links in any format, HTML anchors outside `html`, and Markdown links in `text`.
9
68
  * Each kind scores once however many links use it.
@@ -21,21 +80,21 @@ export function linkSyntax(options = {}) {
21
80
  let format = submission.format ?? "text";
22
81
  let content = submission.content;
23
82
  let signals = [];
24
- if (BBCODE_LINK.test(content)) {
83
+ if (hasBbcodeLink(content)) {
25
84
  signals.push({
26
85
  check: "link-syntax.bbcode",
27
86
  score: bbcodeScore,
28
87
  detail: "BBCode link markup",
29
88
  });
30
89
  }
31
- if (format !== "html" && HTML_LINK.test(content)) {
90
+ if (format !== "html" && hasHtmlLink(content)) {
32
91
  signals.push({
33
92
  check: "link-syntax.html",
34
93
  score: htmlScore,
35
94
  detail: `HTML anchors in a ${format} field`,
36
95
  });
37
96
  }
38
- if (format === "text" && MARKDOWN_LINK.test(content)) {
97
+ if (format === "text" && hasMarkdownLink(content)) {
39
98
  signals.push({
40
99
  check: "link-syntax.markdown",
41
100
  score: markdownScore,
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@sdxc/spam",
3
- "version": "2026.10.9",
3
+ "version": "2026.10.13",
4
4
  "description": "Spam scoring for user content: local rules, reputation checks and a classifier",
5
5
  "license": "MIT",
6
6
  "type": "module",
@@ -15,12 +15,12 @@
15
15
  "./workers-ai": "./dist/workers-ai.js"
16
16
  },
17
17
  "dependencies": {
18
- "@sdxc/doh": "2026.10.9",
19
- "@sdxc/email-address": "2026.10.9",
20
- "@sdxc/result": "2026.10.6",
18
+ "@sdxc/doh": "2026.10.13",
19
+ "@sdxc/email-address": "2026.10.13",
20
+ "@sdxc/result": "2026.10.13",
21
21
  "remix": "3.0.0"
22
22
  },
23
- "gitHead": "64fa732ed0a8ca8693a35aeef6345ae5be2534ef",
23
+ "gitHead": "5ff758a57e35480a8db7abd24f1d9527b81bfb51",
24
24
  "publishConfig": {
25
25
  "access": "public"
26
26
  },