wiki-formant 0.99.0 → 0.104.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -8,6 +8,8 @@ The editor, history, sanitiser, chart and passkey modules bring their libraries
8
8
  npm install wiki-formant
9
9
  ```
10
10
 
11
+ An agent can start a wiki for you, hosted on miow.me or in your own app, with the skill in `skills/wiki-formant`: `npx skills add tutmoses/wiki-formant`.
12
+
11
13
  What stays the app's: its design system, its block types beyond the core, its connection, cache and auth. Everything else here is shared.
12
14
 
13
15
  ## API
@@ -563,7 +565,7 @@ const off = onTweetResize((h, source) => sizeTweetEmbed(el, h, source));
563
565
 
564
566
  ## Search
565
567
 
566
- `wiki-formant/search`: what a literal tier and a full-text tier must agree on: `proseSql`, `HEADLINE_OPTIONS`, `FTS_RANK_NORMALIZATION`, and `searchTsvDdl(table)`, the statements that drop and re-add the generated `search_tsv` column and its index in one transaction.
568
+ `wiki-formant/search`: what a literal tier and a full-text tier must agree on: `proseSql`, `HEADLINE_OPTIONS`, `FTS_RANK_NORMALIZATION`, and `searchTsvDdl(table)`, the statements that drop and re-add the generated `search_tsv` column and its index in one transaction. `fullTextQuery` is a query as the full-text tier reads it, a hyphen or underscore inside a word a space; `correctQuery(query, vocabulary)` corrects a query's misspelt words against a site's own, which the store's `search` retries with when its first page is empty, returning the query that found the rows as `corrected`.
567
569
 
568
570
  ```ts
569
571
  const { column, index } = searchTsvDdl('"WikiPage"');
@@ -713,7 +715,7 @@ export const POST = viewRoute();
713
715
 
714
716
  - `createTracker` records only on a Vercel production deployment unless `enabled` says otherwise. `trackMcpCall` fits `McpServerConfig.onCall`; `trackSearch` normalises a query and drops an empty one; `recordEvent` is the undeferred write.
715
717
  - A proxy reaches the tracker through a dynamic `import()` inside `event.waitUntil`, since it imports the database client.
716
- - `digest(sql, { days })` is the site as one JSON object: visitors, agents, views, bounce rate, visit length, sources, pages, countries, devices, by day, agents by name, MCP calls and searches that found nothing. A visit is a run of views with no gap over 30 minutes; a bounce is a one-view visit with no event of the reader's. `ownAgents` leaves the site's own out. A multi-tenant app passes `site`.
718
+ - `digest(sql, { days })` is the site as one JSON object: visitors, agents, views, visits, bounce rate, visit length, sources, pages, countries, devices, by day, agents by name, MCP calls and searches that found nothing, less a search its visitor went on typing within five minutes and any of one character. A visit is a run of views with no gap over 30 minutes; a bounce is a one-view visit with no event of the reader's. `ownAgents` leaves the site's own out. A multi-tenant app passes `site`.
717
719
  - A browser under automation (`navigator.webdriver`) sends no beacon.
718
720
 
719
721
  ## Passkeys and the stats page
@@ -78,10 +78,18 @@ export interface DigestOptions {
78
78
  ownAgents?: string[];
79
79
  /**
80
80
  * Narrows every figure to the visits that pass, and the events to their
81
- * visitors. `ViewDay` keeps no visits, so a narrowed digest leaves it out
82
- * and starts at `counted_from`, and has no `previous_visitors`.
81
+ * visitors; an agent passes as `AGENT_TEST` has it. `ViewDay` keeps no
82
+ * visits, so a narrowed digest leaves it out and starts at `counted_from`,
83
+ * and has no `previous_visitors`.
83
84
  */
84
85
  filter?: StatsFilter;
86
+ /**
87
+ * People and agents counted apart, as they are by default: the ranked lists
88
+ * count people. False counts agents in the pages, entry pages, sources,
89
+ * countries and devices too, as `visitors_incl_agents` counts them, so a
90
+ * stats page's lists add up to the visitors its chart draws.
91
+ */
92
+ apart?: boolean;
85
93
  }
86
94
  type Ranked<K extends string, M extends string> = Array<Record<K, string> & Record<M, number>>;
87
95
  /** One UTC day of the figures a digest totals; a visit falls on the day it began. */
@@ -102,6 +110,8 @@ export interface DigestDay {
102
110
  export interface Digest {
103
111
  visitors: number;
104
112
  pageviews: number;
113
+ /** Runs of one visitor's views with no gap over 30 minutes. */
114
+ visits: number;
105
115
  /** Percent of visits that are bounces (`DigestDay.bounces`); null with no visits. */
106
116
  bounce_rate: number | null;
107
117
  /** Mean seconds from a visit's first view to its last; null with no visits. */
@@ -121,9 +131,10 @@ export interface Digest {
121
131
  * the archive cannot stand in for it: event figures always, all of them
122
132
  * when filtered.
123
133
  */
124
- previous: Record<'visitors' | 'pageviews' | 'bounce_rate' | 'visit_duration' | 'agents' | 'visitors_incl_agents' | 'follow_clicks', number | null>;
134
+ previous: Record<'visitors' | 'pageviews' | 'visits' | 'bounce_rate' | 'visit_duration' | 'agents' | 'visitors_incl_agents' | 'follow_clicks', number | null>;
125
135
  /** The first UTC day this site counted itself, before which only `ViewDay` speaks; null with no views. */
126
136
  counted_from: string | null;
137
+ /** `(bot)` stands for each referrer `BOT_VISITORS` takes for a crawler's. */
127
138
  top_sources: Ranked<'source', 'visitors'>;
128
139
  top_pages: Ranked<'page', 'visitors'>;
129
140
  entry_pages: Ranked<'page', 'visitors'>;
@@ -160,7 +171,7 @@ export interface Digest {
160
171
  * The window before is the same chain of views, visits and events a second
161
172
  * time, `_p` on every name, so its figures are the headline's to the letter.
162
173
  */
163
- export declare function digest(sql: Sql, { site, days, handle, siblings, gaps, ownAgents, filter }: DigestOptions): Promise<Digest>;
174
+ export declare function digest(sql: Sql, { site, days, handle, siblings, gaps, ownAgents, filter, apart }: DigestOptions): Promise<Digest>;
164
175
  /**
165
176
  * Props for an "MCP Call" event, read out of the JSON-RPC envelope.
166
177
  *
package/dist/analytics.js CHANGED
@@ -109,7 +109,9 @@ export function recordEvent(sql, { name, props = {}, url, headers, site = '' })
109
109
  let path = null;
110
110
  if (url)
111
111
  path = normalisePath(new URL(url, 'http://x').pathname);
112
- return sql('INSERT INTO "Event" ("siteId", name, path, props, visitor) VALUES ($1, $2, $3, $4::jsonb, $5)', site, name.slice(0, 64), path, JSON.stringify(props), headers?.get('user-agent') ? await visitorOf(sql, site, headers) : null);
112
+ // An agent's country is where its requests came from, so the stats page's countries can count it.
113
+ const country = headers?.get('x-vercel-ip-country')?.slice(0, 2);
114
+ return sql('INSERT INTO "Event" ("siteId", name, path, props, visitor) VALUES ($1, $2, $3, $4::jsonb, $5)', site, name.slice(0, 64), path, JSON.stringify(country ? { ...props, country } : props), headers?.get('user-agent') ? await visitorOf(sql, site, headers) : null);
113
115
  });
114
116
  }
115
117
  /** A browser-sent event's props: eight at most, each value a short string or a number. */
@@ -156,16 +158,44 @@ export const STATS_FILTERS = ['page', 'entry', 'source', 'country', 'device'];
156
158
  * `Stats` calls it on the server, where a client module's export is only a reference. */
157
159
  export const count = (n) => n.toLocaleString('en');
158
160
  /**
159
- * Each filter as a test on one visit. A page, country or device is any of the
160
- * visit's views; an entry page or a source is its first. `(none)` is no source.
161
+ * A referrer that sent this many visitors in the window, not one of whom read a
162
+ * second page, is a crawler naming a site it never came from: aocr.org sent
163
+ * radix.wiki 319 one-page visitors in a month while Google's bounced four in five.
164
+ */
165
+ const BOT_VISITORS = 20;
166
+ const BOT_SOURCE = '(bot)';
167
+ /** A visit's source: its first view's, or `(bot)` where that source is among `bots`. */
168
+ const visitSource = (s) => {
169
+ const first = '(array_agg(source ORDER BY at, id))[1]';
170
+ return `CASE WHEN ${first} IN (SELECT source FROM bots${s}) THEN '${BOT_SOURCE}' ELSE ${first} END`;
171
+ };
172
+ /**
173
+ * Each filter as a test on one visit in the window suffixed `s`. A page,
174
+ * country or device is any of the visit's views; an entry page or a source is
175
+ * its first. `(none)` is no source.
161
176
  */
162
177
  const VISIT_TEST = {
163
178
  page: v => `bool_or(path = ${v})`,
164
179
  entry: v => `(array_agg(path ORDER BY at, id))[1] = ${v}`,
165
- source: v => `coalesce((array_agg(source ORDER BY at, id))[1], '(none)') = ${v}`,
180
+ source: (v, s) => `coalesce(${visitSource(s)}, '(none)') = ${v}`,
166
181
  country: v => `bool_or(country = ${v})`,
167
182
  device: v => `bool_or(device = ${v})`,
168
183
  };
184
+ /** An agent's source and device in the lists, which no request of its carries. */
185
+ const AGENT_SOURCE = '(agent)';
186
+ const AGENT_DEVICE = 'agent';
187
+ /**
188
+ * Each filter as a test on one agent: its reads in the window are its visit,
189
+ * the first its entry page. Its source and device are the two above, and its
190
+ * country the one its requests came from, where they were counted with one.
191
+ */
192
+ const AGENT_TEST = {
193
+ page: v => `bool_or(path = ${v})`,
194
+ entry: v => `(array_agg(path ORDER BY at))[1] = ${v}`,
195
+ source: v => `${v} = '${AGENT_SOURCE}'`,
196
+ country: v => `bool_or(country = ${v})`,
197
+ device: v => `${v} = '${AGENT_DEVICE}'`,
198
+ };
169
199
  /**
170
200
  * An agent's name from its event: the roster's label, else what its user agent
171
201
  * calls itself, `compatible; X` or the first product token.
@@ -180,7 +210,7 @@ const AGENT_NAME = `coalesce(props->>'bot', substring(props->>'ua' from 'compati
180
210
  * The window before is the same chain of views, visits and events a second
181
211
  * time, `_p` on every name, so its figures are the headline's to the letter.
182
212
  */
183
- export async function digest(sql, { site = '', days, handle, siblings = [], gaps = [SEARCH_GAPS], ownAgents = ['-mcp-test', 'radix-studio'], filter = {} }) {
213
+ export async function digest(sql, { site = '', days, handle, siblings = [], gaps = [SEARCH_GAPS], ownAgents = ['-mcp-test', 'radix-studio'], filter = {}, apart = true }) {
184
214
  // The window is the database's clock, which stamped the rows; the app's can run behind it.
185
215
  // All time is a century: further back than any row, and still an int to Postgres.
186
216
  const values = [site, Number.isFinite(days) ? Math.max(1, Math.round(days)) : 36_500];
@@ -190,14 +220,34 @@ export async function digest(sql, { site = '', days, handle, siblings = [], gaps
190
220
  const merged = (label, column, live) => `SELECT k AS ${label}, sum(n) AS visitors FROM (${live} UNION ALL SELECT key, value::int FROM a, jsonb_each_text(a."${column}")) t(k, n) GROUP BY 1`;
191
221
  /** Clicks on links to the site's own X account, among `e`. */
192
222
  const follows = handle && `name = 'Outbound Link: Click' AND props->>'url' ILIKE ANY(${$([`%x.com/${handle}%`, `%twitter.com/${handle}%`])}::text[])`;
193
- const tests = STATS_FILTERS.flatMap(k => (filter[k] ? [VISIT_TEST[k]($(filter[k]))] : []));
223
+ const tests = STATS_FILTERS.flatMap(k => {
224
+ if (!filter[k])
225
+ return [];
226
+ const v = $(filter[k]);
227
+ return [(s) => VISIT_TEST[k](v, s)];
228
+ });
229
+ const agentTests = STATS_FILTERS.flatMap(k => (filter[k] ? [AGENT_TEST[k]($(filter[k]))] : []));
230
+ /** Agents among the rows of a ranked list, unless they are counted apart. */
231
+ const agentsToo = (select) => (apart ? '' : ` UNION ALL ${select}`);
194
232
  const day = (at) => `to_char(${at} AT TIME ZONE 'UTC', 'YYYY-MM-DD')`;
195
233
  /** The site's own agents, as a test on an event's user agent. */
196
234
  const ours = `coalesce(props->>'ua', '') ILIKE ANY(${$(ownAgents.map(a => `%${a}%`))}::text[])`;
235
+ /**
236
+ * A typeahead logs each settled keystroke, so one reader typing a question
237
+ * filled acuiq's list with thirteen of its prefixes. A query is left out when
238
+ * the same visitor's next one, inside five minutes, extends it: only where
239
+ * the reader stopped is a gap. So is a single character.
240
+ */
197
241
  const gapSelects = gaps.map(g => {
198
242
  const prop = $(g.prop);
199
- return `(SELECT props->>${prop} AS query, count(*) AS events FROM e
200
- WHERE name = ${$(g.event)} AND props @> ${$(JSON.stringify(g.where ?? {}))}::jsonb AND props->>${prop} IS NOT NULL
243
+ return `(SELECT query, count(*) AS events FROM (
244
+ SELECT props, visitor, props->>${prop} AS query,
245
+ lead(props->>${prop}) OVER typed AS next, lead(at) OVER typed - at AS after
246
+ FROM e WHERE name = ${$(g.event)} AND props->>${prop} IS NOT NULL
247
+ WINDOW typed AS (PARTITION BY visitor ORDER BY at)
248
+ ) s
249
+ WHERE props @> ${$(JSON.stringify(g.where ?? {}))}::jsonb AND char_length(query) > 1
250
+ AND NOT coalesce(visitor IS NOT NULL AND after <= interval '5 minutes' AND char_length(next) > char_length(query) AND starts_with(next, query), false)
201
251
  GROUP BY 1 ORDER BY 2 DESC, 1 LIMIT 15)`;
202
252
  });
203
253
  /**
@@ -214,22 +264,29 @@ export async function digest(sql, { site = '', days, handle, siblings = [], gaps
214
264
  SELECT *, CASE WHEN at - lag(at) OVER (PARTITION BY visitor ORDER BY at, id) <= interval '30 minutes' THEN 0 ELSE 1 END AS starts FROM viewed${s}
215
265
  ), numbered${s} AS (
216
266
  SELECT *, sum(starts) OVER (PARTITION BY visitor ORDER BY at, id) AS visit FROM marked${s}
267
+ ), bots${s} AS (
268
+ SELECT source FROM (SELECT visitor, (array_agg(source ORDER BY at, id))[1] AS source, count(*) AS views FROM numbered${s} GROUP BY visitor, visit) f
269
+ WHERE source IS NOT NULL GROUP BY 1 HAVING count(DISTINCT visitor) >= ${BOT_VISITORS} AND max(views) = 1
217
270
  ), visits${s} AS (
218
271
  SELECT visitor, visit, min(at) AS at, count(*) AS views, extract(epoch FROM max(at) - min(at)) AS seconds,
219
- (array_agg(path ORDER BY at, id))[1] AS entry, (array_agg(source ORDER BY at, id))[1] AS source
220
- FROM numbered${s} GROUP BY visitor, visit${tests.length ? ` HAVING ${tests.join(' AND ')}` : ''}
272
+ (array_agg(path ORDER BY at, id))[1] AS entry, ${visitSource(s)} AS source
273
+ FROM numbered${s} GROUP BY visitor, visit${tests.length ? ` HAVING ${tests.map(t => t(s)).join(' AND ')}` : ''}
221
274
  ), v${s} AS (
222
275
  ${tests.length ? `SELECT n.id, n.path, n.source, n.country, n.device, n.visitor, n.at FROM numbered${s} n JOIN visits${s} USING (visitor, visit)` : `SELECT * FROM viewed${s}`}
276
+ ), read${s} AS (
277
+ SELECT visitor, "createdAt" AS at, path, props->>'country' AS country, ${AGENT_NAME} AS agent FROM "Event"
278
+ WHERE "siteId" = $1 AND ${at} AND name IN ('AI Bot Visit', 'MCP Call') AND visitor IS NOT NULL AND NOT ${ours}
279
+ ), ag${s} AS (
280
+ SELECT * FROM read${s}${agentTests.length ? ` WHERE visitor IN (SELECT visitor FROM read${s} GROUP BY 1 HAVING ${agentTests.join(' AND ')})` : ''}
223
281
  ), e${s} AS (
224
282
  SELECT name, props, visitor, "createdAt" AS at FROM "Event" WHERE "siteId" = $1 AND ${at}
225
- ${tests.length ? `AND visitor IN (SELECT visitor FROM v${s})` : ''}
283
+ ${tests.length ? `AND (visitor IN (SELECT visitor FROM v${s}) OR visitor IN (SELECT visitor FROM ag${s}))` : ''}
284
+ ), acted${s} AS MATERIALIZED (
285
+ SELECT visitor, at FROM e${s} WHERE name NOT IN ('AI Bot Visit', 'MCP Call') AND visitor IS NOT NULL
226
286
  ), bounced${s} AS (
287
+ -- Against the few reader events alone: probing every agent event once per visit took seconds.
227
288
  SELECT * FROM visits${s} b WHERE views = 1 AND NOT EXISTS (
228
- SELECT 1 FROM e${s} x WHERE x.visitor = b.visitor AND x.name NOT IN ('AI Bot Visit', 'MCP Call')
229
- AND x.at >= b.at AND x.at < b.at + interval '30 minutes')
230
- ), ag${s} AS (
231
- SELECT visitor, at, ${AGENT_NAME} AS agent FROM e${s}
232
- WHERE name IN ('AI Bot Visit', 'MCP Call') AND visitor IS NOT NULL AND NOT ${ours}
289
+ SELECT 1 FROM acted${s} x WHERE x.visitor = b.visitor AND x.at >= b.at AND x.at < b.at + interval '30 minutes')
233
290
  ), a${s} AS (
234
291
  SELECT * FROM "ViewDay" WHERE "siteId" = $1 AND day >= ${dayAgo(back + 1)}${back ? ` AND day < ${dayAgo(back)}` : ''}${tests.length ? ' AND false' : ''}
235
292
  ), sessions${s} AS (
@@ -244,25 +301,26 @@ export async function digest(sql, { site = '', days, handle, siblings = [], gaps
244
301
  /** One window's headline figures, as `json_build_object` arguments. */
245
302
  const headline = (s) => `'visitors', (SELECT count(DISTINCT visitor) FROM v${s}) + (SELECT visitors FROM archived${s}),
246
303
  'pageviews', (SELECT count(*) FROM v${s}) + (SELECT pageviews FROM archived${s}),
304
+ 'visits', (SELECT visits FROM sessions${s}),
247
305
  'bounce_rate', (SELECT round(100.0 * bounces / nullif(visits, 0)) FROM sessions${s}),
248
306
  'visit_duration', (SELECT round(seconds / nullif(visits, 0)) FROM sessions${s}),
249
307
  'agents', (SELECT count(DISTINCT visitor) FROM ag${s}),
250
308
  'visitors_incl_agents', (SELECT count(DISTINCT visitor) FROM (SELECT visitor FROM v${s} UNION SELECT visitor FROM ag${s}) u) + (SELECT visitors FROM archived${s})`;
251
309
  const [row] = (await sql(`WITH ${chain('', 0)}, ${chain('_p', 1)}, src AS (
252
310
  SELECT coalesce(source, '(none)') AS source, count(DISTINCT visitor) AS visitors FROM visits GROUP BY 1
253
- UNION ALL SELECT key, value::int FROM a, jsonb_each_text(a.sources)
311
+ UNION ALL SELECT key, value::int FROM a, jsonb_each_text(a.sources)${agentsToo(`SELECT '${AGENT_SOURCE}', count(DISTINCT visitor) FROM ag HAVING count(*) > 0`)}
254
312
  )
255
313
  SELECT json_build_object(
256
314
  ${headline('')},
257
315
  'previous', json_build_object(${headline('_p')}, 'follow_clicks', ${follows ? `(SELECT count(*) FROM e_p WHERE ${follows})` : 'NULL'}),
258
316
  'counted_from', (SELECT ${day('min("createdAt")')} FROM "View" WHERE "siteId" = $1),
259
317
  'top_sources', ${list('SELECT source, sum(visitors) AS visitors FROM src GROUP BY 1', 6)},
260
- 'top_pages', ${list(merged('page', 'pages', 'SELECT path, count(DISTINCT visitor) FROM v GROUP BY 1'), 6)},
261
- 'entry_pages', ${list(merged('page', 'entryPages', 'SELECT entry, count(DISTINCT visitor) FROM visits GROUP BY 1'), 6)},
318
+ 'top_pages', ${list(merged('page', 'pages', `SELECT path, count(DISTINCT visitor) FROM v GROUP BY 1${agentsToo('SELECT path, count(DISTINCT visitor) FROM ag WHERE path IS NOT NULL GROUP BY 1')}`), 6)},
319
+ 'entry_pages', ${list(merged('page', 'entryPages', `SELECT entry, count(DISTINCT visitor) FROM visits GROUP BY 1${agentsToo('SELECT entry, count(*) FROM (SELECT (array_agg(path ORDER BY at))[1] AS entry FROM ag GROUP BY visitor) f WHERE entry IS NOT NULL GROUP BY 1')}`), 6)},
262
320
  'from_posts', ${list(`SELECT entry AS page, count(DISTINCT visitor) AS visitors FROM visits WHERE source = 'x' GROUP BY 1`, 15)},
263
321
  'sibling_referrals', ${list(`SELECT source, sum(visitors) AS visitors FROM src WHERE source = ANY(${$(siblings)}::text[]) GROUP BY 1`, 6)},
264
- 'countries', ${list(merged('country', 'countries', 'SELECT country, count(DISTINCT visitor) FROM v WHERE country IS NOT NULL GROUP BY 1'), 10)},
265
- 'devices', ${list(merged('device', 'devices', 'SELECT device, count(DISTINCT visitor) FROM v GROUP BY 1'))},
322
+ 'countries', ${list(merged('country', 'countries', `SELECT country, count(DISTINCT visitor) FROM v WHERE country IS NOT NULL GROUP BY 1${agentsToo('SELECT country, count(DISTINCT visitor) FROM ag WHERE country IS NOT NULL GROUP BY 1')}`), 10)},
323
+ 'devices', ${list(merged('device', 'devices', `SELECT device, count(DISTINCT visitor) FROM v GROUP BY 1${agentsToo(`SELECT '${AGENT_DEVICE}', count(DISTINCT visitor) FROM ag HAVING count(*) > 0`)}`))},
266
324
  'by_day', ${list(`SELECT date, sum(visitors) AS visitors, sum(pageviews) AS pageviews, sum(visits) AS visits, sum(bounces) AS bounces,
267
325
  sum(seconds) AS seconds, sum(agents) AS agents, sum(everyone) AS visitors_incl_agents, sum(follows) AS follow_clicks FROM (
268
326
  SELECT ${day('at')}, count(DISTINCT visitor), count(*), 0, 0, 0, 0, 0, 0 FROM v GROUP BY 1
package/dist/chart.js CHANGED
@@ -60,21 +60,24 @@ function bucketStart(time, step) {
60
60
  * Points, oldest first, one per step. A price keeps the last value in each
61
61
  * step, as a candle does its close: the first open, the highest high, the
62
62
  * lowest low. A count is the total of its days; a rate, their mean. Volume is
63
- * summed.
63
+ * summed. The first step sits on its first point when the points start inside
64
+ * it, so the axis never reaches back past the days that were counted.
64
65
  *
65
66
  * @internal
66
67
  */
67
68
  export function bucketChart(points, step, aggregate) {
68
69
  const out = [];
69
70
  let n = 0;
71
+ let bucket;
70
72
  for (const p of points) {
71
73
  const start = bucketStart(p.time, step);
72
- const last = out.at(-1);
73
- if (last?.time !== start) {
74
- out.push({ ...p, time: start });
74
+ if (bucket !== start) {
75
+ out.push({ ...p, time: out.length ? start : p.time });
76
+ bucket = start;
75
77
  n = 1;
76
78
  continue;
77
79
  }
80
+ const last = out.at(-1);
78
81
  if (aggregate === 'last')
79
82
  last.value = p.value;
80
83
  else if (aggregate === 'sum')
@@ -22,6 +22,8 @@ export interface WikiLinkProps {
22
22
  'aria-checked'?: boolean;
23
23
  /** The name of a link drawn as an icon. */
24
24
  'aria-label'?: string;
25
+ /** False where following the link costs the server a query, as a stats page's links do; a plain anchor ignores it. */
26
+ prefetch?: boolean;
25
27
  children?: ReactNode;
26
28
  }
27
29
  export type WikiLinkComponent = ComponentType<WikiLinkProps>;
@@ -29,7 +29,7 @@ import { formatDay } from './freshness.js';
29
29
  * Exported because `block-views` renders links too and had declared the same
30
30
  * three lines; a default that exists twice is a default that can diverge.
31
31
  */
32
- export const Anchor = (props) => _jsx("a", { ...props });
32
+ export const Anchor = ({ prefetch: _, ...props }) => _jsx("a", { ...props });
33
33
  const pagesPhrase = (n) => `${n} page${n === 1 ? '' : 's'}`;
34
34
  /**
35
35
  * The section's second axis, as pressable chips.
package/dist/search.d.ts CHANGED
@@ -72,3 +72,22 @@ export declare const FTS_RANK_NORMALIZATION = 32;
72
72
  * rather than as a pattern matching everything.
73
73
  */
74
74
  export declare function escapeLikeTerm(query: string): string;
75
+ /**
76
+ * A query as the full-text tier reads it: a hyphen or underscore inside a word
77
+ * is a space. `websearch_to_tsquery` keeps a hyphenated word as a phrase that
78
+ * needs the compound itself, so the slug `radix-pipeline` found nothing where
79
+ * `radix pipeline` found sixteen pages. A leading `-` is websearch's NOT, and
80
+ * stays.
81
+ */
82
+ export declare function fullTextQuery(query: string): string;
83
+ /**
84
+ * A query with each misspelled word replaced by the nearest word in `vocabulary`
85
+ * (word → how often it occurs), or `null` when no word changes. Called only for
86
+ * a query that found nothing, so a word the wiki already uses is never touched.
87
+ *
88
+ * Tight on purpose: a word of four to seven letters may be one edit out, a
89
+ * longer one two, and a shorter one none. Looser, a name the wiki lacks turns
90
+ * into a word it has – `bondix` into `bonds` – and the reader gets an answer to
91
+ * a question they did not ask. Ties go to the commoner word.
92
+ */
93
+ export declare function correctQuery(query: string, vocabulary: ReadonlyMap<string, number>): string | null;
package/dist/search.js CHANGED
@@ -120,3 +120,62 @@ export const FTS_RANK_NORMALIZATION = 32;
120
120
  export function escapeLikeTerm(query) {
121
121
  return (query ?? '').trim().replace(/[\\%_]/g, char => `\\${char}`);
122
122
  }
123
+ /**
124
+ * A query as the full-text tier reads it: a hyphen or underscore inside a word
125
+ * is a space. `websearch_to_tsquery` keeps a hyphenated word as a phrase that
126
+ * needs the compound itself, so the slug `radix-pipeline` found nothing where
127
+ * `radix pipeline` found sixteen pages. A leading `-` is websearch's NOT, and
128
+ * stays.
129
+ */
130
+ export function fullTextQuery(query) {
131
+ return (query ?? '').trim().replace(/([\p{L}\p{N}])[-_]+(?=[\p{L}\p{N}])/gu, '$1 ');
132
+ }
133
+ /** Edits between two words, a swapped adjacent pair counting as one. */
134
+ function editDistance(a, b) {
135
+ const d = Array.from({ length: a.length + 1 }, (_, i) => [i, ...Array(b.length).fill(0)]);
136
+ for (let j = 1; j <= b.length; j++)
137
+ d[0][j] = j;
138
+ for (let i = 1; i <= a.length; i++)
139
+ for (let j = 1; j <= b.length; j++) {
140
+ const cost = a[i - 1] === b[j - 1] ? 0 : 1;
141
+ d[i][j] = Math.min(d[i - 1][j] + 1, d[i][j - 1] + 1, d[i - 1][j - 1] + cost);
142
+ if (i > 1 && j > 1 && a[i - 1] === b[j - 2] && a[i - 2] === b[j - 1])
143
+ d[i][j] = Math.min(d[i][j], d[i - 2][j - 2] + 1);
144
+ }
145
+ return d[a.length][b.length];
146
+ }
147
+ /**
148
+ * A query with each misspelled word replaced by the nearest word in `vocabulary`
149
+ * (word → how often it occurs), or `null` when no word changes. Called only for
150
+ * a query that found nothing, so a word the wiki already uses is never touched.
151
+ *
152
+ * Tight on purpose: a word of four to seven letters may be one edit out, a
153
+ * longer one two, and a shorter one none. Looser, a name the wiki lacks turns
154
+ * into a word it has – `bondix` into `bonds` – and the reader gets an answer to
155
+ * a question they did not ask. Ties go to the commoner word.
156
+ */
157
+ export function correctQuery(query, vocabulary) {
158
+ let changed = false;
159
+ const fixed = query.trim().toLowerCase().split(/\s+/).map(word => {
160
+ const budget = word.length >= 8 ? 2 : word.length >= 4 ? 1 : 0;
161
+ if (!budget || vocabulary.has(word) || !/^[\p{L}\p{N}]+$/u.test(word))
162
+ return word;
163
+ let best = null;
164
+ let bestDistance = Infinity;
165
+ let bestCount = 0;
166
+ for (const [candidate, count] of vocabulary) {
167
+ if (Math.abs(candidate.length - word.length) > budget)
168
+ continue;
169
+ const distance = editDistance(word, candidate);
170
+ if (distance > budget)
171
+ continue;
172
+ if (distance < bestDistance || (distance === bestDistance && count > bestCount))
173
+ [best, bestDistance, bestCount] = [candidate, distance, count];
174
+ }
175
+ if (!best)
176
+ return word;
177
+ changed = true;
178
+ return best;
179
+ });
180
+ return changed ? fixed.join(' ') : null;
181
+ }
@@ -13,6 +13,8 @@ import { TimeChart, stepDecimals } from './chart.js';
13
13
  const places = (step) => (step ? stepDecimals(step) : 0);
14
14
  const duration = (s, step) => (s < 60 ? `${s.toFixed(places(step))}s` : `${Math.floor(Math.round(s) / 60)}m ${Math.round(s) % 60}s`);
15
15
  const percent = (n, step) => `${n.toFixed(places(step))}%`;
16
+ // Views per visit sit between one and a few, so a tenth tells them apart.
17
+ const perVisit = (n, step) => n.toFixed(step ? stepDecimals(step) : 1);
16
18
  /**
17
19
  * The change from the window before: a share of the figure before, or for a
18
20
  * rate its difference in points, since a bounce rate up 10% of itself reads
@@ -42,6 +44,14 @@ const figuresOf = ({ previous: p, ...d }, apart) => [
42
44
  },
43
45
  ]),
44
46
  { label: 'Page views', value: count(d.pageviews), delta: change(d.pageviews, p.pageviews), day: r => r.pageviews },
47
+ {
48
+ label: 'Views per visit',
49
+ value: d.visits ? perVisit(d.pageviews / d.visits) : '–',
50
+ delta: change(d.visits ? d.pageviews / d.visits : null, p.pageviews != null && p.visits ? p.pageviews / p.visits : null),
51
+ day: r => (r.visits ? r.pageviews / r.visits : null),
52
+ format: perVisit,
53
+ aggregate: 'mean',
54
+ },
45
55
  {
46
56
  label: 'Bounce rate',
47
57
  value: d.bounce_rate == null ? '–' : percent(d.bounce_rate),
@@ -79,8 +89,9 @@ const EMPTY = { date: '', visitors: 0, pageviews: 0, visits: 0, bounces: 0, seco
79
89
  */
80
90
  export function StatsFigures({ digest, first, apart, controls }) {
81
91
  const figures = useMemo(() => figuresOf(digest, apart), [digest, apart]);
82
- const [chosen, choose] = useState(0);
83
- const figure = figures[chosen] ?? figures[0];
92
+ // By name, since a window or the agents switch can arrive without a reload and the figures shift.
93
+ const [chosen, choose] = useState(figures[0].label);
94
+ const figure = figures.find(f => f.label === chosen) ?? figures[0];
84
95
  const byDay = useMemo(() => new Map(digest.by_day.map(r => [r.date, r])), [digest.by_day]);
85
96
  const series = useMemo(() => {
86
97
  const today = Math.floor(Date.now() / 1000 / DAY);
@@ -90,7 +101,7 @@ export function StatsFigures({ digest, first, apart, controls }) {
90
101
  return value == null ? null : { time, value };
91
102
  }).filter(p => p !== null);
92
103
  }, [byDay, first, figure]);
93
- return (_jsxs(_Fragment, { children: [_jsx("div", { className: "stats-figures", role: "group", "aria-label": "Chart", children: figures.map((f, i) => (_jsxs("button", { type: "button", "aria-pressed": i === chosen, onClick: () => choose(i), children: [_jsx("span", { children: f.label }), _jsxs("strong", { children: [f.value, f.delta && (_jsx("small", { className: "stats-delta", "data-better": f.delta.better, children: f.delta.text }))] })] }, f.label))) }), _jsx(TimeChart, { series: series, label: figure.label, ranges: ['all'], aggregate: figure.aggregate ?? 'sum', fromZero: true, format: figure.format, controls: controls, className: "stats-chart" })] }));
104
+ return (_jsxs(_Fragment, { children: [_jsx("div", { className: "stats-figures", role: "group", "aria-label": "Chart", children: figures.map(f => (_jsxs("button", { type: "button", "aria-pressed": f === figure, onClick: () => choose(f.label), children: [_jsx("span", { children: f.label }), _jsxs("strong", { children: [f.value, f.delta && (_jsx("small", { className: "stats-delta", "data-better": f.delta.better, children: f.delta.text }))] })] }, f.label))) }), _jsx(TimeChart, { series: series, label: figure.label, ranges: ['all'], aggregate: figure.aggregate ?? 'sum', fromZero: true, format: figure.format, controls: controls, className: "stats-chart" })] }));
94
105
  }
95
106
  async function post(endpoint, body) {
96
107
  const res = await fetch(endpoint, { method: 'POST', headers: { 'content-type': 'application/json' }, body: JSON.stringify(body) });
package/dist/stats.d.ts CHANGED
@@ -13,7 +13,7 @@ export interface StatsProps {
13
13
  apart?: boolean;
14
14
  /** The windows offered as `?days=` links on the same page. */
15
15
  ranges?: readonly number[];
16
- /** Every link on the page: `next/link`, say, so a window or a filter does not reload it. */
16
+ /** Every link on the page: `next/link`, say, so a window or a filter does not reload it. Each is given `prefetch={false}`. */
17
17
  link?: WikiLinkComponent;
18
18
  }
19
19
  /**
@@ -23,14 +23,16 @@ export interface StatsProps {
23
23
  */
24
24
  export declare function statsQuery(params: Record<string, string | string[] | undefined>, ranges?: readonly number[]): {
25
25
  days: number;
26
- filter: Partial<Record<"page" | "entry" | "source" | "country" | "device", string>>;
26
+ filter: Partial<Record<"page" | "country" | "entry" | "source" | "device", string>>;
27
27
  apart: boolean;
28
28
  };
29
29
  /**
30
30
  * A site's own analytics on one page: the headline figures against the window
31
31
  * before, visitors counting agents unless `apart`, whichever of them is chosen per day, and every ranked list the
32
32
  * digest carries, each row a link that narrows the page to it. Lists with
33
- * nothing in them are left out. The chart and the bars are `currentColor`
33
+ * nothing in them are left out, but for the searches that found nothing,
34
+ * which says it found none, so a quiet window never reads as one not counted. The digest is made with the same `apart`, so
35
+ * the lists count the visitors the chart draws. The chart and the bars are `currentColor`
34
36
  * until the importing stylesheet gives them a colour.
35
37
  */
36
38
  export declare function Stats({ digest: d, days, filter, apart, ranges, link: Link }: StatsProps): import("react").JSX.Element;
package/dist/stats.js CHANGED
@@ -22,6 +22,7 @@ export function statsQuery(params, ranges = STATS_RANGES) {
22
22
  if (typeof v === 'string' && v)
23
23
  filter[k] = v.slice(0, 500);
24
24
  }
25
+ // `apart` is `digest`'s option too, so its lists count the visitors the chart draws.
25
26
  return { days: statsDays(params.days, ranges), filter, apart: params.agents === 'apart' };
26
27
  }
27
28
  /** The page at a window, a filter and a count of agents, as a query string. */
@@ -45,17 +46,19 @@ const country = (code) => {
45
46
  };
46
47
  const NAMES = { page: 'Page', entry: 'Entry page', source: 'Source', country: 'Country', device: 'Device' };
47
48
  const DAY_MS = 86_400_000;
48
- function StatsList({ title, rows, label = v => v, to, link: Link, }) {
49
+ function StatsList({ title, rows, label = v => v, to, empty, link: Link, }) {
49
50
  if (!rows.length)
50
- return null;
51
+ return empty ? (_jsxs("section", { className: "stats-list", children: [_jsx("h2", { children: title }), _jsx("p", { children: empty })] })) : null;
51
52
  const max = Math.max(...rows.map(([, n]) => n));
52
- return (_jsxs("section", { className: "stats-list", children: [_jsx("h2", { children: title }), _jsx("ol", { children: rows.map(([v, n]) => (_jsxs("li", { style: { '--share': n / max }, children: [_jsx("span", { children: to ? _jsx(Link, { href: to(v), children: label(v) }) : label(v) }), _jsx("span", { className: "stats-count", children: count(n) })] }, v))) })] }));
53
+ return (_jsxs("section", { className: "stats-list", children: [_jsx("h2", { children: title }), _jsx("ol", { children: rows.map(([v, n]) => (_jsxs("li", { style: { '--share': n / max }, children: [_jsx("span", { children: to ? (_jsx(Link, { href: to(v), prefetch: false, children: label(v) })) : (label(v)) }), _jsx("span", { className: "stats-count", children: count(n) })] }, v))) })] }));
53
54
  }
54
55
  /**
55
56
  * A site's own analytics on one page: the headline figures against the window
56
57
  * before, visitors counting agents unless `apart`, whichever of them is chosen per day, and every ranked list the
57
58
  * digest carries, each row a link that narrows the page to it. Lists with
58
- * nothing in them are left out. The chart and the bars are `currentColor`
59
+ * nothing in them are left out, but for the searches that found nothing,
60
+ * which says it found none, so a quiet window never reads as one not counted. The digest is made with the same `apart`, so
61
+ * the lists count the visitors the chart draws. The chart and the bars are `currentColor`
59
62
  * until the importing stylesheet gives them a colour.
60
63
  */
61
64
  export function Stats({ digest: d, days, filter = {}, apart = false, ranges = STATS_RANGES, link: Link = Anchor }) {
@@ -74,8 +77,8 @@ export function Stats({ digest: d, days, filter = {}, apart = false, ranges = ST
74
77
  const to = (k) => (v) => at({ ...filter, [k]: v });
75
78
  // Each window is another digest, so a link rather than a button, pinned
76
79
  // with the chart's own controls, and so is the switch that counts agents apart.
77
- const controls = (_jsxs(_Fragment, { children: [_jsx("div", { role: "group", "aria-label": "Window", children: ranges.map(n => (_jsx(Link, { href: at(filter, n), "aria-current": n === days ? 'page' : undefined, children: windowLabel(n) }, n))) }), _jsx("div", { children: _jsx(Link, { href: at(filter, days, !apart), role: "switch", "aria-checked": !apart, "aria-label": "Agents counted among visitors", title: "Agents counted among visitors", children: _jsxs("svg", { className: "chart-icon", viewBox: "0 0 12 12", "aria-hidden": true, children: [_jsx("path", { d: "M6 1.5v2M1 6.5v2M11 6.5v2", stroke: "currentColor" }), _jsx("rect", { x: "2.5", y: "3.5", width: "7", height: "7", rx: "1.5", fill: "none", stroke: "currentColor" }), _jsx("circle", { cx: "4.75", cy: "7", r: "0.9", fill: "currentColor" }), _jsx("circle", { cx: "7.25", cy: "7", r: "0.9", fill: "currentColor" })] }) }) })] }));
78
- return (_jsxs("div", { className: "stats", children: [narrowed.length > 0 && (_jsxs("div", { className: "stats-filters", children: [_jsx("ul", { "aria-label": "Narrowed to", children: narrowed.map(k => (_jsx("li", { children: _jsxs(Link, { href: at({ ...filter, [k]: undefined }), title: "Remove", children: [NAMES[k], ": ", k === 'country' ? country(filter[k]) : filter[k], " \u00D7"] }) }, k))) }), start < counted && d.counted_from && (_jsxs("p", { children: ["Narrowed figures start on ", d.counted_from, ", the first day this site counted its own visits."] }))] })), _jsx(StatsFigures, { digest: d, first: first, apart: apart, controls: controls }), _jsxs("div", { className: "stats-lists", children: [_jsx(StatsList, { link: Link, title: "Pages", rows: rows(d.top_pages, 'page', 'visitors'), to: to('page') }), _jsx(StatsList, { link: Link, title: "Entry pages", rows: rows(d.entry_pages, 'page', 'visitors'), to: to('entry') }), _jsx(StatsList, { link: Link, title: "Sources", rows: rows(d.top_sources, 'source', 'visitors'), to: to('source') }), _jsx(StatsList, { link: Link, title: "From X posts", rows: rows(d.from_posts, 'page', 'visitors'), to: v => at({ ...filter, source: 'x', entry: v }) }), _jsx(StatsList, { link: Link, title: "Sibling sites", rows: rows(d.sibling_referrals, 'source', 'visitors'), to: to('source') }), _jsx(StatsList, { link: Link, title: "Countries", rows: rows(d.countries, 'country', 'visitors'), label: country, to: to('country') }), _jsx(StatsList, { link: Link, title: "Devices", rows: rows(d.devices, 'device', 'visitors'), to: to('device') }), _jsx(StatsList, { link: Link, title: "Agents", rows: rows(d.top_agents, 'agent', 'visitors') }), _jsx(StatsList, { link: Link, title: "Agent tool calls", rows: rows(d.agent_tool_calls, 'tool', 'events') }), _jsx(StatsList, { link: Link, title: "Searches that found nothing", rows: rows(d.gaps, 'query', 'events') })] })] }));
80
+ const controls = (_jsxs(_Fragment, { children: [_jsx("div", { role: "group", "aria-label": "Window", children: ranges.map(n => (_jsx(Link, { href: at(filter, n), "aria-current": n === days ? 'page' : undefined, prefetch: false, children: windowLabel(n) }, n))) }), _jsx("div", { children: _jsx(Link, { href: at(filter, days, !apart), role: "switch", "aria-checked": !apart, "aria-label": "Agents counted among visitors", title: "Agents counted among visitors", prefetch: false, children: _jsxs("svg", { className: "chart-icon", viewBox: "0 0 12 12", "aria-hidden": true, children: [_jsx("path", { d: "M6 1.5v2M1 6.5v2M11 6.5v2", stroke: "currentColor" }), _jsx("rect", { x: "2.5", y: "3.5", width: "7", height: "7", rx: "1.5", fill: "none", stroke: "currentColor" }), _jsx("circle", { cx: "4.75", cy: "7", r: "0.9", fill: "currentColor" }), _jsx("circle", { cx: "7.25", cy: "7", r: "0.9", fill: "currentColor" })] }) }) })] }));
81
+ return (_jsxs("div", { className: "stats", children: [narrowed.length > 0 && (_jsxs("div", { className: "stats-filters", children: [_jsx("ul", { "aria-label": "Narrowed to", children: narrowed.map(k => (_jsx("li", { children: _jsxs(Link, { href: at({ ...filter, [k]: undefined }), title: "Remove", prefetch: false, children: [NAMES[k], ": ", k === 'country' ? country(filter[k]) : filter[k], " \u00D7"] }) }, k))) }), start < counted && d.counted_from && (_jsxs("p", { children: ["Narrowed figures start on ", d.counted_from, ", the first day this site counted its own visits."] }))] })), _jsx(StatsFigures, { digest: d, first: first, apart: apart, controls: controls }), _jsxs("div", { className: "stats-lists", children: [_jsx(StatsList, { link: Link, title: "Pages", rows: rows(d.top_pages, 'page', 'visitors'), to: to('page') }), _jsx(StatsList, { link: Link, title: "Entry pages", rows: rows(d.entry_pages, 'page', 'visitors'), to: to('entry') }), _jsx(StatsList, { link: Link, title: "Sources", rows: rows(d.top_sources, 'source', 'visitors'), to: to('source') }), _jsx(StatsList, { link: Link, title: "From X posts", rows: rows(d.from_posts, 'page', 'visitors'), to: v => at({ ...filter, source: 'x', entry: v }) }), _jsx(StatsList, { link: Link, title: "Sibling sites", rows: rows(d.sibling_referrals, 'source', 'visitors'), to: to('source') }), _jsx(StatsList, { link: Link, title: "Countries", rows: rows(d.countries, 'country', 'visitors'), label: country, to: to('country') }), _jsx(StatsList, { link: Link, title: "Devices", rows: rows(d.devices, 'device', 'visitors'), to: to('device') }), _jsx(StatsList, { link: Link, title: "Agents", rows: rows(d.top_agents, 'agent', 'visitors') }), _jsx(StatsList, { link: Link, title: "Agent tool calls", rows: rows(d.agent_tool_calls, 'tool', 'events') }), _jsx(StatsList, { link: Link, title: "Searches that found nothing", rows: rows(d.gaps, 'query', 'events'), empty: "None in this window." })] })] }));
79
82
  }
80
83
  /**
81
84
  * A stats page's door: `children` and a sign-out button for a signed-in
package/dist/store.d.ts CHANGED
@@ -47,6 +47,13 @@ export type WikiRevisionEntry = Omit<WikiRevisionRow, 'content'> & {
47
47
  changes: HistoryChange[];
48
48
  };
49
49
  export type ListSort = 'title' | 'recent' | 'date';
50
+ /** A search's scope and page. `content` adds each row's blocks. */
51
+ export interface SearchOptions {
52
+ tagPath?: string | null;
53
+ skip?: number;
54
+ take?: number;
55
+ content?: boolean;
56
+ }
50
57
  export interface WikiStoreOptions<B extends DiffBlock> {
51
58
  sql: Sql;
52
59
  /** The site every read and write is scoped to. "" in an app that serves one wiki. */
@@ -169,17 +176,14 @@ export declare function wikiStore<B extends DiffBlock = DiffBlock>(o: WikiStoreO
169
176
  pages: number;
170
177
  updated: Date | null;
171
178
  }>;
172
- search: (query: string, { tagPath, skip, take, content: withContent }?: {
173
- tagPath?: string | null;
174
- skip?: number;
175
- take?: number;
176
- content?: boolean;
177
- }) => Promise<{
179
+ search: (query: string, options?: SearchOptions) => Promise<Awaited<ReturnType<(query: string, { tagPath, skip, take, content: withContent }?: SearchOptions) => Promise<{
178
180
  items: (WikiPageListing & {
179
181
  score: number;
180
182
  content?: B[];
181
183
  })[];
182
184
  total: number;
185
+ }>>> & {
186
+ corrected?: string;
183
187
  }>;
184
188
  history: (pageId: string, { limit, offset }?: {
185
189
  limit?: number;
package/dist/store.js CHANGED
@@ -18,7 +18,7 @@
18
18
  import { computeRevisionDiff } from './revisions.js';
19
19
  import { bump } from './versioning.js';
20
20
  import { withChanges } from './history.js';
21
- import { proseSql, escapeLikeTerm, HEADLINE_OPTIONS, FTS_RANK_NORMALIZATION } from './search.js';
21
+ import { proseSql, escapeLikeTerm, fullTextQuery, correctQuery, HEADLINE_OPTIONS, FTS_RANK_NORMALIZATION } from './search.js';
22
22
  import { openingText, matchSnippet } from './text.js';
23
23
  import { describe } from './metadata.js';
24
24
  import { decodeEntities } from './markdown.js';
@@ -132,7 +132,7 @@ export function wikiStore(o) {
132
132
  * page is never a result. A tier-3 row's snippet is Postgres' own headline,
133
133
  * because a stem match has no literal substring to window on.
134
134
  */
135
- const search = async (query, { tagPath = null, skip = 0, take = 25, content: withContent = false } = {}) => {
135
+ const ranked = async (query, { tagPath = null, skip = 0, take = 25, content: withContent = false } = {}) => {
136
136
  const term = escapeLikeTerm(query);
137
137
  if (!term)
138
138
  return { items: [], total: 0 };
@@ -163,7 +163,7 @@ export function wikiStore(o) {
163
163
  SELECT ${COLUMNS}, tier, total,
164
164
  CASE WHEN tier = 3 THEN ts_headline('english', ${proseSql()}, tsq, '${HEADLINE_OPTIONS}') END AS headline
165
165
  FROM (SELECT count(*)::int AS total FROM matched) t LEFT JOIN paged ON true
166
- ORDER BY tier, CASE WHEN tier = 0 THEN title END, CASE WHEN tier = 3 THEN fts END DESC, "updatedAt" DESC`, query.trim(), `${term}%`, `%${term}%`, hidden, site, tagPath, take, skip);
166
+ ORDER BY tier, CASE WHEN tier = 0 THEN title END, CASE WHEN tier = 3 THEN fts END DESC, "updatedAt" DESC`, fullTextQuery(query), `${term}%`, `%${term}%`, hidden, site, tagPath, take, skip);
167
167
  const items = found.filter(hit => hit.id !== null).map(({ tier, total: _, headline, content, ...row }) => ({
168
168
  ...row,
169
169
  ...(withContent ? { content: content } : {}),
@@ -173,6 +173,26 @@ export function wikiStore(o) {
173
173
  }));
174
174
  return { items, total: found[0]?.total ?? 0 };
175
175
  };
176
+ /** Every word of the searchable titles, with how many titles use it: what a misspelling is corrected against. */
177
+ const titleWords = async (tagPath) => new Map((await rows(`SELECT word, count(*)::int AS n FROM ${PAGE} p, regexp_split_to_table(lower(p.title), '[^[:alnum:]]+') word
178
+ WHERE p."siteId" = $1 AND word <> '' AND ($2::text IS NULL OR p."tagPath" = $2) AND ($2::text IS NOT NULL OR NOT ${hiddenSql('$3')})
179
+ GROUP BY 1`, site, tagPath, hidden)).map(r => [r.word, r.n]));
180
+ /**
181
+ * {@link ranked}, and when its first page is empty, the same search with each
182
+ * misspelled word corrected against the titles' own words (`correctQuery`).
183
+ * A result found that way carries `corrected`, the query that found it, so a
184
+ * page can say what it searched for instead.
185
+ */
186
+ const search = async (query, options = {}) => {
187
+ const found = await ranked(query, options);
188
+ if (found.total || options.skip)
189
+ return found;
190
+ const corrected = correctQuery(query, await titleWords(options.tagPath ?? null));
191
+ if (!corrected)
192
+ return found;
193
+ const again = await ranked(corrected, options);
194
+ return again.total ? { ...again, corrected } : found;
195
+ };
176
196
  /** A revision query's guard: `$1`, its page, is this site's. Another site's page id finds nothing. */
177
197
  const OWN_PAGE = `"pageId" IN (SELECT id FROM ${PAGE} WHERE id = $1 AND "siteId" = $2)`;
178
198
  /**
@@ -75,5 +75,6 @@ export declare function searchWiki<B extends DiffBlock>(o: Pick<WikiToolsOptions
75
75
  }): Promise<{
76
76
  items: WikiRow[];
77
77
  total: number;
78
+ corrected?: string;
78
79
  }>;
79
80
  export declare function wikiReadTools<B extends DiffBlock>(o: WikiToolsOptions<B>): WikiTool[];
@@ -54,8 +54,8 @@ const rowFor = (o) => (p) => {
54
54
  export async function searchWiki(o, query, { tagPath, page = 1, size = 20 } = {}) {
55
55
  if (offFor(o)(tagPath))
56
56
  return { items: [], total: 0 };
57
- const { items, total } = await o.store.search(query, { tagPath, skip: (page - 1) * size, take: size, content: true });
58
- return { items: items.map(rowFor(o)), total };
57
+ const { items, total, corrected } = await o.store.search(query, { tagPath, skip: (page - 1) * size, take: size, content: true });
58
+ return { items: items.map(rowFor(o)), total, ...(corrected ? { corrected } : {}) };
59
59
  }
60
60
  export function wikiReadTools(o) {
61
61
  const { store, tree, noun } = o;
@@ -116,7 +116,7 @@ export function wikiReadTools(o) {
116
116
  skill: { id: 'search', tags: ['search', 'wiki'] },
117
117
  description: `Keyword search across ${noun}: the first call when you know what you are looking for but not where it lives. ` +
118
118
  'Matches titles and body prose, ranked in four tiers: titles starting with the term, titles containing it, literal body hits, then English full-text hits by relevance, which stem words and drop stopwords. ' +
119
- 'Not semantic: one distinctive word beats a sentence. ' +
119
+ 'Not semantic: one distinctive word beats a sentence. A query that finds nothing is retried with its misspelled words corrected, and `corrected` then names what was searched. ' +
120
120
  'Each row carries a `path`, its size in `chars` and a snippet windowed on the match. Feed the paths to get_pages in one call rather than looping get_page. ' +
121
121
  'Narrow with tagPath when a term is common. Defaults: page 1, pageSize 20 (max 50); keep paging while `hasMore` is true.',
122
122
  inputSchema: {
@@ -137,9 +137,10 @@ export function wikiReadTools(o) {
137
137
  const size = r.num('pageSize', 20, 1, 50);
138
138
  if (!query)
139
139
  throw new McpToolError('`query` is empty. Pass a keyword to search for, or call list_pages to enumerate a branch instead.');
140
- const { items, total } = await searchWiki(o, query, { tagPath, page, size });
140
+ const { items, total, corrected } = await searchWiki(o, query, { tagPath, page, size });
141
141
  return withAdjustments(r, {
142
142
  query,
143
+ ...(corrected ? { corrected } : {}),
143
144
  ...listEnvelope(items, total, page, size, o.noMatch?.(query, tagPath)
144
145
  ?? `Nothing matches "${query}"${tagPath ? ` under "${tagPath}"` : ''}. Search is keyword, not semantic: try one distinctive word${tagPath ? ', or drop the tagPath filter' : ''}, or call get_categories to see what ${noun} covers.`),
145
146
  });
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "wiki-formant",
3
- "version": "0.99.0",
3
+ "version": "0.104.0",
4
4
  "description": "The portable half of a wiki: derived taxonomy and facet controls, a version-negotiating MCP transport, markdown twins, block rendering, a rich-text editor engine, conditional-GET plumbing, and the headers and CSP every origin sends.",
5
5
  "type": "module",
6
6
  "license": "MIT",