@sonordev/contracts 2.4.0 → 2.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +23 -0
- package/README.md +127 -0
- package/dist/knowledge.d.ts +84 -0
- package/dist/knowledge.js +48 -0
- package/dist/knowledge.mjs +37 -0
- package/dist/sentences.d.ts +46 -0
- package/dist/sentences.js +179 -0
- package/dist/sentences.mjs +177 -0
- package/dist/site-cache.d.ts +2 -0
- package/dist/site-cache.js +3 -1
- package/dist/site-cache.mjs +3 -1
- package/package.json +12 -2
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,28 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 2.6.0
|
|
4
|
+
|
|
5
|
+
A minor release: one new entry, `@sonordev/contracts/knowledge`. Every other entry is unchanged.
|
|
6
|
+
|
|
7
|
+
- **New entry `@sonordev/contracts/knowledge`**: which Sonor records Signal reads into a project's knowledge, where each one's knowledge is stored, and how big a page sync may be, so the API that announces a change and Signal, which indexes it, agree.
|
|
8
|
+
- `KNOWLEDGE_RECORD_KINDS` lists the record kinds (`portfolio`, `proof`, `faq`, `review`, `article`, `offering`), and `isKnowledgeRecordKind(value)` checks one.
|
|
9
|
+
- `knowledgeRecordSourceUrl(kind, id, anchor?)` builds a record's address (`faq://<id>`, `article://<id>#pricing`) and `knowledgeRecordPrefix(kind)` the `<kind>://` all of a kind's addresses start with.
|
|
10
|
+
- `knowledgeRecordKindOf(url)` tells a record's row from a page passage by its scheme, and `knowledgeRecordRefOf(url)` returns the `{ kind, id }` an address names, the legacy `commerce://offering/<id>` (`legacyOfferingSourceUrl`) included.
|
|
11
|
+
- `KNOWLEDGE_SYNC_MAX_PAGES` (500), `KNOWLEDGE_SYNC_PATH_MAX_LENGTH` (2048) and `KNOWLEDGE_SYNC_SITE_MAX_LENGTH` (253) are the limits of one page sync, and `KnowledgePageChange` is the shape of each page in it. Signal applies every removal in a call and reads the changed pages in the order given, so a sender puts removals first.
|
|
12
|
+
|
|
13
|
+
## 2.5.0
|
|
14
|
+
|
|
15
|
+
A minor release: one new entry, `@sonordev/contracts/sentences`, and a cache tag for commerce. Every other entry is unchanged.
|
|
16
|
+
|
|
17
|
+
- **New entry `@sonordev/contracts/sentences`**: where prose splits into sentences, so every app that cuts an excerpt, a social slide or a summary out of an article ends sentences in the same places.
|
|
18
|
+
- `splitSentences(text)` returns the trimmed sentences of `text`. It's lossless: joining the result with spaces gives back every word of the input, in order. The usual one-liners aren't: `text.match(/[^.!?]+[.!?]+(?=\s|$)/g)` skips ahead at a period it can't use (the first one in "9 p.m.") and drops every word before its next match, and `text.match(/[^.!?]+[.!?]+/g)` cuts "9 p.m." into "9 p." and "m.".
|
|
19
|
+
- It doesn't end a sentence at a dotted abbreviation of letters ("p.m.", "U.S.", "e.g."), a single letter ("J."), a common short form (titles, company and street suffixes, months, and days of the week when capitalized: "Dr.", "Inc.", "Ave.", "Sept.", "Mon."), a label before a number ("No. 5", "Fig. 2") or a list number ("1.", "Step 2."), and never before a lowercase word. A piece with no letter in it ("...", "1.") joins the next sentence. A `!` or `?` right after an abbreviation still ends one.
|
|
20
|
+
- Prices, versions, domains and email addresses ("$4.50", "v2.1", "example.com") aren't abbreviations: a sentence can end after them.
|
|
21
|
+
- It runs in linear time: a megabyte of text takes milliseconds, however many abbreviations it holds.
|
|
22
|
+
- A newline or a list marker isn't a boundary by itself, so split on lines or paragraphs first when those matter.
|
|
23
|
+
- Empty, whitespace-only, `null` and `undefined` input give `[]`.
|
|
24
|
+
- **`site-cache`**: `SITE_CACHE_TAGS.commerce` (`commerce`) is the tag for products, services, events and membership plans. commerce-kit tags its server fetches with it, and the Sonor API expires it when an offering changes. The other tags are unchanged.
|
|
25
|
+
|
|
3
26
|
## 2.4.0
|
|
4
27
|
|
|
5
28
|
A minor release: one new entry, `@sonordev/contracts/schema-placeholders`, and the llms.txt sanitizers stop cutting mid-word. Every other entry is unchanged.
|
package/README.md
CHANGED
|
@@ -26,6 +26,8 @@ pnpm add @sonordev/contracts
|
|
|
26
26
|
| `@sonordev/contracts/voice` | The house voice's contraction rule. `findUncontracted` and `contractText` find and apply full forms that should be contractions; `findContracted` finds the contractions a question-form heading should expand | anything that checks or rewrites copy: Sonor's APIs and dashboard |
|
|
27
27
|
| `@sonordev/contracts/error-page` | Whether stored text is a rendered error page rather than a page's content, and whether generated copy says the page itself is missing | anything that stores page text or writes page copy from it: the Sonor API's page-view and sitemap writers, Signal's metadata generators |
|
|
28
28
|
| `@sonordev/contracts/schema-placeholders` | Whether stored JSON-LD carries a template's unfilled slots (`example.com` URLs, an `Organization` named "Example", a placeholder phone, `[Resident Name]`, `{plan.name}`), and the value without them | site-kit's schema output, the Sonor API's schema reads and writes |
|
|
29
|
+
| `@sonordev/contracts/sentences` | Where prose splits into sentences, without dropping a word or ending one at "p.m." (`splitSentences`) | anything that cuts an excerpt, a social slide or a summary out of an article: the Sonor API's carousels, Signal's article pipeline |
|
|
30
|
+
| `@sonordev/contracts/knowledge` | Which Sonor records Signal reads into a project's knowledge, the address each one's knowledge is stored under (`portfolio://<id>`), and the limits of the call that tells Signal a site's pages changed | the Sonor API sends those calls and Signal answers them |
|
|
29
31
|
|
|
30
32
|
## Schema placeholders
|
|
31
33
|
|
|
@@ -287,6 +289,131 @@ A text-only check can't prove a page failed. Where you observed the HTTP
|
|
|
287
289
|
status, require 400 or more as well and use this as the fallback; the header of
|
|
288
290
|
`src/contracts/error-page.ts` lists the other known limits.
|
|
289
291
|
|
|
292
|
+
## The sentences contract
|
|
293
|
+
|
|
294
|
+
Cutting an excerpt, a slide or a summary out of an article starts with
|
|
295
|
+
finding where its sentences end, and the usual one-liners for that get it
|
|
296
|
+
wrong. `text.match(/[^.!?]+[.!?]+(?=\s|$)/g)` can't end a match at the first
|
|
297
|
+
period of "9 p.m.", so it skips ahead and drops every word before the next
|
|
298
|
+
match: "We open at 9 p.m. daily." comes back as "m." and " daily.". Without
|
|
299
|
+
the lookahead, `text.match(/[^.!?]+[.!?]+/g)` cuts "9 p.m." into "9 p." and
|
|
300
|
+
"m.". Both also drop any text after the last full stop. `splitSentences` is
|
|
301
|
+
the one splitter to use instead.
|
|
302
|
+
|
|
303
|
+
- **Lossless.** Joining the result with spaces gives back every word of the
|
|
304
|
+
input, in order. Nothing is dropped, added or reordered, and each sentence
|
|
305
|
+
comes back trimmed and non-empty.
|
|
306
|
+
- **Where a sentence ends.** At `.`, `!` or `?` (and any closing `"`, `'`, `”`,
|
|
307
|
+
`’`, `)` or `]` after it) followed by whitespace and then `A` to `Z` or `0`
|
|
308
|
+
to `9`, optionally after one opening `"`, `'`, `“`, `‘`, `(` or `[`. Never
|
|
309
|
+
before a lowercase word or a symbol. Only ASCII capitals count: a sentence
|
|
310
|
+
that opens with an accented, Greek or Cyrillic capital, a currency sign, `#`,
|
|
311
|
+
`@` or an emoji stays joined to the one before it.
|
|
312
|
+
- **Where it doesn't.** After a dotted abbreviation made of letters ("p.m.",
|
|
313
|
+
"U.S.", "e.g.", "Ph.D."), a single letter of either case (an initial like
|
|
314
|
+
"J.", but also "Plan A."), or a short form on its list: titles, company
|
|
315
|
+
suffixes, months, street suffixes and days of the week ("Dr.", "Inc.",
|
|
316
|
+
"Sept.", "Ave.", "Mon."). A few of those only count when capitalized, because
|
|
317
|
+
the lowercase word ends sentences too ("Mon." is a day, "the sun." isn't).
|
|
318
|
+
A label with a number after it stays together ("No. 5", "Fig. 2",
|
|
319
|
+
"Ste. 200"), while "No." before anything else ends a sentence. A list number
|
|
320
|
+
("1.", "Step 2.", "Do three things: 1.") stays with its item.
|
|
321
|
+
- **`!` and `?`.** An abbreviation only holds back a period. "Open until 9
|
|
322
|
+
p.m.? Yes!" is two sentences, under the same start-of-sentence rule as above.
|
|
323
|
+
- **Prices, versions and web addresses** ("$4.50", "v2.1", "example.com",
|
|
324
|
+
"info@example.com") aren't abbreviations: a sentence can end after them.
|
|
325
|
+
- **Never a bare mark.** A piece with no letter in it ("...", "?!", "1.")
|
|
326
|
+
isn't a sentence. It joins the next one.
|
|
327
|
+
- **Prose, not layout.** A newline or a list marker doesn't start a new
|
|
328
|
+
sentence by itself. If lines or paragraphs matter, split on those first and
|
|
329
|
+
call `splitSentences` on each one.
|
|
330
|
+
- **Empty is `[]`.** Empty, whitespace-only, `null` and `undefined` input all
|
|
331
|
+
give an empty array, never `null`, so callers can iterate without a guard.
|
|
332
|
+
- **Linear time.** One pass over the text. A megabyte takes milliseconds,
|
|
333
|
+
however many abbreviations, stops or quotes it holds.
|
|
334
|
+
|
|
335
|
+
```ts
|
|
336
|
+
import { splitSentences } from '@sonordev/contracts/sentences'
|
|
337
|
+
|
|
338
|
+
splitSentences('We open at 9 a.m. daily. Walk-ins are welcome.')
|
|
339
|
+
// ['We open at 9 a.m. daily.', 'Walk-ins are welcome.']
|
|
340
|
+
|
|
341
|
+
splitSentences(excerptSource).slice(0, 2).join(' ')
|
|
342
|
+
// the first two sentences, whole
|
|
343
|
+
```
|
|
344
|
+
|
|
345
|
+
Known limits. Where it can't be sure it leans toward joining, because a
|
|
346
|
+
sentence that runs long costs less than a lost word, and a caller that caps
|
|
347
|
+
length should cut at a word. It can't tell an abbreviation that ends a
|
|
348
|
+
sentence from one that doesn't ("Contact Acme Inc. Our team ..." and "... in
|
|
349
|
+
Washington, D.C. Residents ..." each stay in one piece), it only knows English
|
|
350
|
+
short forms (a period after one it doesn't know ends a sentence, so "Sra.
|
|
351
|
+
García" splits after "Sra."), and the next sentence has to start with `A` to
|
|
352
|
+
`Z` or a digit.
|
|
353
|
+
|
|
354
|
+
## The knowledge contract
|
|
355
|
+
|
|
356
|
+
Signal answers questions about a business from what it has read: the
|
|
357
|
+
passages of the site's live pages, and the business's own records in Sonor.
|
|
358
|
+
The Sonor API tells Signal when either changes, and Signal indexes it. Both
|
|
359
|
+
have to agree on which records count, how a stored row says which record it
|
|
360
|
+
came from, and how big a page sync may be. If they don't, a record the API
|
|
361
|
+
announces is refused, or a row Signal wrote reads as a page. Anything that
|
|
362
|
+
shows what Signal has read, like the dashboard's view of it, reads stored
|
|
363
|
+
addresses by the same rules.
|
|
364
|
+
|
|
365
|
+
- **Record kinds.** `KNOWLEDGE_RECORD_KINDS` is `portfolio` (case studies),
|
|
366
|
+
`proof` (proof records), `faq` (managed FAQs), `review`, `article` and
|
|
367
|
+
`offering` (commerce offerings), in the order the dashboard lists them.
|
|
368
|
+
`isKnowledgeRecordKind(value)` is true only for one of them, spelled exactly.
|
|
369
|
+
- **Addresses.** A record's knowledge is stored under `<kind>://<id>`, and
|
|
370
|
+
any further chunk under `<kind>://<id>#<anchor>` (an article indexed in full
|
|
371
|
+
is split by heading). `knowledgeRecordSourceUrl(kind, id, anchor?)` builds
|
|
372
|
+
one and `knowledgeRecordPrefix(kind)` gives the `<kind>://` every address of
|
|
373
|
+
that kind starts with. The id is the record's UUID, and the caller checks
|
|
374
|
+
it: one holding `#`, `?`, `/` or whitespace can't be read back. A page's
|
|
375
|
+
passages are stored under the page's https URL instead, so the scheme alone
|
|
376
|
+
tells a record's rows apart. The content type can't: an article's record
|
|
377
|
+
has the same type as the page passages beside it, because a live article is
|
|
378
|
+
a page on the site.
|
|
379
|
+
- **Reading an address.** `knowledgeRecordKindOf(url)` is the kind by scheme
|
|
380
|
+
alone, or `null` for a page passage, an older page summary or a note. It's
|
|
381
|
+
the test for whether a row belongs to a record. `knowledgeRecordRefOf(url)`
|
|
382
|
+
is `{ kind, id }` for the record an address names, or `null` when it names
|
|
383
|
+
none; the id runs to the first `#`, `?`, `/` or whitespace. Both ignore the
|
|
384
|
+
scheme's case and surrounding whitespace.
|
|
385
|
+
- **The legacy offering address.** Offerings were once stored under
|
|
386
|
+
`commerce://offering/<id>` (`legacyOfferingSourceUrl(id)`). Nothing writes
|
|
387
|
+
it now, and Signal switches each such row off the next time its offering
|
|
388
|
+
syncs. Until then `knowledgeRecordRefOf` names the offering, while
|
|
389
|
+
`knowledgeRecordKindOf` doesn't count the row as a record's.
|
|
390
|
+
- **A page sync.** One call names at most `KNOWLEDGE_SYNC_MAX_PAGES` (500)
|
|
391
|
+
pages, each a `KnowledgePageChange` (`{ site?, path, removed? }`) with a
|
|
392
|
+
path of at most `KNOWLEDGE_SYNC_PATH_MAX_LENGTH` (2048) characters and a
|
|
393
|
+
host of at most `KNOWLEDGE_SYNC_SITE_MAX_LENGTH` (253). Signal refuses a
|
|
394
|
+
longer list, path or host outright, so a sender keeps within them before it
|
|
395
|
+
calls. Signal applies every removal in the call, then reads the changed
|
|
396
|
+
pages in the order given, as many as it reads in one call; the rest wait for
|
|
397
|
+
its nightly sweep. So put removals first, then the pages that matter most,
|
|
398
|
+
and cut the list after ordering it.
|
|
399
|
+
|
|
400
|
+
```ts
|
|
401
|
+
import {
|
|
402
|
+
KNOWLEDGE_SYNC_MAX_PAGES,
|
|
403
|
+
knowledgeRecordRefOf,
|
|
404
|
+
knowledgeRecordSourceUrl,
|
|
405
|
+
} from '@sonordev/contracts/knowledge'
|
|
406
|
+
|
|
407
|
+
knowledgeRecordSourceUrl('faq', id)
|
|
408
|
+
// 'faq://<id>'
|
|
409
|
+
|
|
410
|
+
knowledgeRecordRefOf('article://<id>#part-2')
|
|
411
|
+
// { kind: 'article', id: '<id>' }
|
|
412
|
+
|
|
413
|
+
const batch = [...removed, ...changed].slice(0, KNOWLEDGE_SYNC_MAX_PAGES)
|
|
414
|
+
// removals first, so the cut drops changed pages before any removal
|
|
415
|
+
```
|
|
416
|
+
|
|
290
417
|
## Why it exists
|
|
291
418
|
|
|
292
419
|
A rule that lives in several copies is how two of them end up disagreeing.
|
|
@@ -0,0 +1,84 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Knowledge contract: which Sonor records Signal reads into a project's
|
|
3
|
+
* knowledge, the address each record's knowledge is stored under, and the
|
|
4
|
+
* limits of the call that tells Signal a site's pages changed. The Sonor API
|
|
5
|
+
* makes those calls and Signal answers them, so both import this as
|
|
6
|
+
* `@sonordev/contracts/knowledge` instead of keeping lists that drift apart.
|
|
7
|
+
* The dashboard, which shows the owner what Signal read, reads stored
|
|
8
|
+
* addresses by the same rules.
|
|
9
|
+
*
|
|
10
|
+
* A record's knowledge is stored under a URI built from its kind and id:
|
|
11
|
+
* `<kind>://<id>` for its first chunk and `<kind>://<id>#<anchor>` for any
|
|
12
|
+
* more (an article indexed in full, split by heading). The scheme is what
|
|
13
|
+
* tells a record's rows apart from a page's passages, which are stored under
|
|
14
|
+
* the page's https URL. The content type can't: an article's record has the
|
|
15
|
+
* same content type as the page passages beside it, because a live article is
|
|
16
|
+
* a page on the site.
|
|
17
|
+
*/
|
|
18
|
+
/**
|
|
19
|
+
* The kinds of Sonor record Signal indexes, in the order the dashboard lists
|
|
20
|
+
* them: case studies, proof records, managed FAQs, reviews, articles and
|
|
21
|
+
* commerce offerings. Each kind is also its URI scheme.
|
|
22
|
+
*/
|
|
23
|
+
export declare const KNOWLEDGE_RECORD_KINDS: readonly ["portfolio", "proof", "faq", "review", "article", "offering"];
|
|
24
|
+
export type KnowledgeRecordKind = (typeof KNOWLEDGE_RECORD_KINDS)[number];
|
|
25
|
+
/** True for one of KNOWLEDGE_RECORD_KINDS, spelled exactly (lowercase). */
|
|
26
|
+
export declare function isKnowledgeRecordKind(value: unknown): value is KnowledgeRecordKind;
|
|
27
|
+
/** `<kind>://`, the start every address of that kind shares, for prefix queries. */
|
|
28
|
+
export declare function knowledgeRecordPrefix(kind: KnowledgeRecordKind): string;
|
|
29
|
+
/**
|
|
30
|
+
* Where a record's knowledge is stored: `<kind>://<id>` for its first chunk,
|
|
31
|
+
* `<kind>://<id>#<anchor>` for a further one. `id` is the record's UUID, and
|
|
32
|
+
* checking it is the caller's job: an empty id, or one holding `#`, `?`, `/`
|
|
33
|
+
* or whitespace, builds an address knowledgeRecordRefOf can't read back.
|
|
34
|
+
*/
|
|
35
|
+
export declare function knowledgeRecordSourceUrl(kind: KnowledgeRecordKind, id: string, anchor?: string | null): string;
|
|
36
|
+
/**
|
|
37
|
+
* The address an offering had before offerings were records of their own:
|
|
38
|
+
* `commerce://offering/<id>`. Nothing writes it any more. Signal switches the
|
|
39
|
+
* row off the next time that offering syncs, and a reader counts it as the
|
|
40
|
+
* offering until then (knowledgeRecordRefOf).
|
|
41
|
+
*/
|
|
42
|
+
export declare function legacyOfferingSourceUrl(id: string): string;
|
|
43
|
+
/**
|
|
44
|
+
* The kind of record a stored address belongs to, by its scheme
|
|
45
|
+
* (`article://<id>#part-2` is 'article'), or null for anything else: a page
|
|
46
|
+
* passage, an older page summary, a note. Case doesn't matter and surrounding
|
|
47
|
+
* whitespace is ignored. This is the test for "is this row a record's": it
|
|
48
|
+
* reads only the scheme, so it doesn't need an id, and an offering's legacy
|
|
49
|
+
* address isn't a record row by it.
|
|
50
|
+
*/
|
|
51
|
+
export declare function knowledgeRecordKindOf(sourceUrl: string | null | undefined): KnowledgeRecordKind | null;
|
|
52
|
+
/** The record a stored address names. */
|
|
53
|
+
export interface KnowledgeRecordRef {
|
|
54
|
+
kind: KnowledgeRecordKind;
|
|
55
|
+
id: string;
|
|
56
|
+
}
|
|
57
|
+
/**
|
|
58
|
+
* The record a stored address names (`portfolio://<id>#part-2` is
|
|
59
|
+
* `{ kind: 'portfolio', id }`), or null when it names none. An offering's
|
|
60
|
+
* legacy address (`commerce://offering/<id>`) names that offering. Case
|
|
61
|
+
* doesn't matter in the scheme, surrounding whitespace is ignored, and the id
|
|
62
|
+
* keeps its own case.
|
|
63
|
+
*/
|
|
64
|
+
export declare function knowledgeRecordRefOf(sourceUrl: string | null | undefined): KnowledgeRecordRef | null;
|
|
65
|
+
/**
|
|
66
|
+
* One page of a site in a page sync: it changed (Signal reads it live) or
|
|
67
|
+
* left the site (`removed`: Signal switches its passages off). Signal applies
|
|
68
|
+
* every removal in a call, then reads the changed pages in the order given,
|
|
69
|
+
* as many as it reads in one call; the rest wait for its nightly sweep. So a
|
|
70
|
+
* sender puts removals first, then the pages that matter most.
|
|
71
|
+
*/
|
|
72
|
+
export interface KnowledgePageChange {
|
|
73
|
+
/** The host as the project's pages store it; null or absent is the project's primary domain. */
|
|
74
|
+
site?: string | null;
|
|
75
|
+
/** The page's path, from "/". */
|
|
76
|
+
path: string;
|
|
77
|
+
removed?: boolean;
|
|
78
|
+
}
|
|
79
|
+
/** The most pages one page sync may name. Signal refuses a longer list outright, so cut the list after ordering it. */
|
|
80
|
+
export declare const KNOWLEDGE_SYNC_MAX_PAGES = 500;
|
|
81
|
+
/** The longest path a page sync may name, in characters. */
|
|
82
|
+
export declare const KNOWLEDGE_SYNC_PATH_MAX_LENGTH = 2048;
|
|
83
|
+
/** The longest host a page sync may name, in characters (a DNS name's limit). */
|
|
84
|
+
export declare const KNOWLEDGE_SYNC_SITE_MAX_LENGTH = 253;
|
|
@@ -0,0 +1,48 @@
|
|
|
1
|
+
'use strict';
|
|
2
|
+
|
|
3
|
+
// src/contracts/knowledge.ts
|
|
4
|
+
var KNOWLEDGE_RECORD_KINDS = Object.freeze(["portfolio", "proof", "faq", "review", "article", "offering"]);
|
|
5
|
+
function isKnowledgeRecordKind(value) {
|
|
6
|
+
return typeof value === "string" && KNOWLEDGE_RECORD_KINDS.includes(value);
|
|
7
|
+
}
|
|
8
|
+
function knowledgeRecordPrefix(kind) {
|
|
9
|
+
return `${kind}://`;
|
|
10
|
+
}
|
|
11
|
+
function knowledgeRecordSourceUrl(kind, id, anchor) {
|
|
12
|
+
const base = `${knowledgeRecordPrefix(kind)}${id}`;
|
|
13
|
+
return anchor ? `${base}#${anchor}` : base;
|
|
14
|
+
}
|
|
15
|
+
function legacyOfferingSourceUrl(id) {
|
|
16
|
+
return `${LEGACY_OFFERING_PREFIX}${id}`;
|
|
17
|
+
}
|
|
18
|
+
var LEGACY_OFFERING_PREFIX = "commerce://offering/";
|
|
19
|
+
var RECORD_SCHEME = new RegExp(`^(${KNOWLEDGE_RECORD_KINDS.join("|")})://`, "i");
|
|
20
|
+
var RECORD_REF = new RegExp(`^(${KNOWLEDGE_RECORD_KINDS.join("|")})://([^#?/\\s]+)`, "i");
|
|
21
|
+
var LEGACY_OFFERING_REF = /^commerce:\/\/offering\/([^#?/\s]+)/i;
|
|
22
|
+
function knowledgeRecordKindOf(sourceUrl) {
|
|
23
|
+
if (typeof sourceUrl !== "string") return null;
|
|
24
|
+
const match = RECORD_SCHEME.exec(sourceUrl.trim());
|
|
25
|
+
return match ? match[1].toLowerCase() : null;
|
|
26
|
+
}
|
|
27
|
+
function knowledgeRecordRefOf(sourceUrl) {
|
|
28
|
+
if (typeof sourceUrl !== "string") return null;
|
|
29
|
+
const value = sourceUrl.trim();
|
|
30
|
+
const record = RECORD_REF.exec(value);
|
|
31
|
+
if (record) return { kind: record[1].toLowerCase(), id: record[2] };
|
|
32
|
+
const legacy = LEGACY_OFFERING_REF.exec(value);
|
|
33
|
+
return legacy ? { kind: "offering", id: legacy[1] } : null;
|
|
34
|
+
}
|
|
35
|
+
var KNOWLEDGE_SYNC_MAX_PAGES = 500;
|
|
36
|
+
var KNOWLEDGE_SYNC_PATH_MAX_LENGTH = 2048;
|
|
37
|
+
var KNOWLEDGE_SYNC_SITE_MAX_LENGTH = 253;
|
|
38
|
+
|
|
39
|
+
exports.KNOWLEDGE_RECORD_KINDS = KNOWLEDGE_RECORD_KINDS;
|
|
40
|
+
exports.KNOWLEDGE_SYNC_MAX_PAGES = KNOWLEDGE_SYNC_MAX_PAGES;
|
|
41
|
+
exports.KNOWLEDGE_SYNC_PATH_MAX_LENGTH = KNOWLEDGE_SYNC_PATH_MAX_LENGTH;
|
|
42
|
+
exports.KNOWLEDGE_SYNC_SITE_MAX_LENGTH = KNOWLEDGE_SYNC_SITE_MAX_LENGTH;
|
|
43
|
+
exports.isKnowledgeRecordKind = isKnowledgeRecordKind;
|
|
44
|
+
exports.knowledgeRecordKindOf = knowledgeRecordKindOf;
|
|
45
|
+
exports.knowledgeRecordPrefix = knowledgeRecordPrefix;
|
|
46
|
+
exports.knowledgeRecordRefOf = knowledgeRecordRefOf;
|
|
47
|
+
exports.knowledgeRecordSourceUrl = knowledgeRecordSourceUrl;
|
|
48
|
+
exports.legacyOfferingSourceUrl = legacyOfferingSourceUrl;
|
|
@@ -0,0 +1,37 @@
|
|
|
1
|
+
// src/contracts/knowledge.ts
|
|
2
|
+
var KNOWLEDGE_RECORD_KINDS = Object.freeze(["portfolio", "proof", "faq", "review", "article", "offering"]);
|
|
3
|
+
function isKnowledgeRecordKind(value) {
|
|
4
|
+
return typeof value === "string" && KNOWLEDGE_RECORD_KINDS.includes(value);
|
|
5
|
+
}
|
|
6
|
+
function knowledgeRecordPrefix(kind) {
|
|
7
|
+
return `${kind}://`;
|
|
8
|
+
}
|
|
9
|
+
function knowledgeRecordSourceUrl(kind, id, anchor) {
|
|
10
|
+
const base = `${knowledgeRecordPrefix(kind)}${id}`;
|
|
11
|
+
return anchor ? `${base}#${anchor}` : base;
|
|
12
|
+
}
|
|
13
|
+
function legacyOfferingSourceUrl(id) {
|
|
14
|
+
return `${LEGACY_OFFERING_PREFIX}${id}`;
|
|
15
|
+
}
|
|
16
|
+
var LEGACY_OFFERING_PREFIX = "commerce://offering/";
|
|
17
|
+
var RECORD_SCHEME = new RegExp(`^(${KNOWLEDGE_RECORD_KINDS.join("|")})://`, "i");
|
|
18
|
+
var RECORD_REF = new RegExp(`^(${KNOWLEDGE_RECORD_KINDS.join("|")})://([^#?/\\s]+)`, "i");
|
|
19
|
+
var LEGACY_OFFERING_REF = /^commerce:\/\/offering\/([^#?/\s]+)/i;
|
|
20
|
+
function knowledgeRecordKindOf(sourceUrl) {
|
|
21
|
+
if (typeof sourceUrl !== "string") return null;
|
|
22
|
+
const match = RECORD_SCHEME.exec(sourceUrl.trim());
|
|
23
|
+
return match ? match[1].toLowerCase() : null;
|
|
24
|
+
}
|
|
25
|
+
function knowledgeRecordRefOf(sourceUrl) {
|
|
26
|
+
if (typeof sourceUrl !== "string") return null;
|
|
27
|
+
const value = sourceUrl.trim();
|
|
28
|
+
const record = RECORD_REF.exec(value);
|
|
29
|
+
if (record) return { kind: record[1].toLowerCase(), id: record[2] };
|
|
30
|
+
const legacy = LEGACY_OFFERING_REF.exec(value);
|
|
31
|
+
return legacy ? { kind: "offering", id: legacy[1] } : null;
|
|
32
|
+
}
|
|
33
|
+
var KNOWLEDGE_SYNC_MAX_PAGES = 500;
|
|
34
|
+
var KNOWLEDGE_SYNC_PATH_MAX_LENGTH = 2048;
|
|
35
|
+
var KNOWLEDGE_SYNC_SITE_MAX_LENGTH = 253;
|
|
36
|
+
|
|
37
|
+
export { KNOWLEDGE_RECORD_KINDS, KNOWLEDGE_SYNC_MAX_PAGES, KNOWLEDGE_SYNC_PATH_MAX_LENGTH, KNOWLEDGE_SYNC_SITE_MAX_LENGTH, isKnowledgeRecordKind, knowledgeRecordKindOf, knowledgeRecordPrefix, knowledgeRecordRefOf, knowledgeRecordSourceUrl, legacyOfferingSourceUrl };
|
|
@@ -0,0 +1,46 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Sentences contract: how prose is split into sentences, so the APIs that cut
|
|
3
|
+
* an excerpt, a carousel slide or a summary out of an article agree on where
|
|
4
|
+
* a sentence ends. The APIs import it as `@sonordev/contracts/sentences`.
|
|
5
|
+
*
|
|
6
|
+
* Lossless: joining the result with spaces gives back every word of the input,
|
|
7
|
+
* in order. The usual one-liner, `text.match(/[^.!?]+[.!?]+(?=\s|$)/g)`, isn't.
|
|
8
|
+
* A period it can't use, like the first one in "9 p.m. call", makes match()
|
|
9
|
+
* skip ahead and silently drop every word before its next match, so a
|
|
10
|
+
* sentence about a 9 p.m. call comes back as "m.". Don't re-fork a regex for
|
|
11
|
+
* this.
|
|
12
|
+
*
|
|
13
|
+
* A sentence ends at . ! or ? (and any closing quote or bracket after it)
|
|
14
|
+
* followed by whitespace, then an optional opening quote or bracket, then
|
|
15
|
+
* A to Z or 0 to 9. Never before a lowercase word or a symbol, and only ASCII
|
|
16
|
+
* capitals count. It doesn't end at:
|
|
17
|
+
* - an abbreviation: a dotted one made of letters ("p.m.", "U.S.", "e.g.",
|
|
18
|
+
* "Ph.D."), a single letter ("J.", "Plan A."), or a short form listed below
|
|
19
|
+
* (titles, months, days, company suffixes, street suffixes);
|
|
20
|
+
* - a label that has a number after it ("No. 5", "Fig. 2", "Ste. 200");
|
|
21
|
+
* - a list number ("1.", "Step 2.", "Do three things: 1.").
|
|
22
|
+
* A piece with no letter in it ("...", "1.") isn't a sentence and joins the
|
|
23
|
+
* next one. A "!" or "?" is never held back by an abbreviation (only a period
|
|
24
|
+
* is), though the same start-of-sentence rule applies after it.
|
|
25
|
+
*
|
|
26
|
+
* Prices, versions, domains and email addresses ("$4.50", "v2.1",
|
|
27
|
+
* "example.com") are not abbreviations: a sentence can end after them.
|
|
28
|
+
*
|
|
29
|
+
* Where it is unsure it leans toward not splitting, because a sentence that
|
|
30
|
+
* runs long costs less than a lost word. A period after a short form that
|
|
31
|
+
* isn't listed ends a sentence like any other.
|
|
32
|
+
*
|
|
33
|
+
* It reads sentences, not layout. A newline or a list marker doesn't start a
|
|
34
|
+
* new sentence by itself, so split text into lines or paragraphs first when
|
|
35
|
+
* those matter, then split each one.
|
|
36
|
+
*/
|
|
37
|
+
/**
|
|
38
|
+
* Splits prose into trimmed sentences without dropping or reordering a word.
|
|
39
|
+
* Empty, blank and nullish input give `[]`, never `null`.
|
|
40
|
+
*
|
|
41
|
+
* One pass, linear in the length of the text: each run of . ! ? (and the
|
|
42
|
+
* closing quotes or brackets after it) is looked at once, an abbreviation or
|
|
43
|
+
* list-marker check reads only the last few characters, and the check for a
|
|
44
|
+
* letter reads each character once.
|
|
45
|
+
*/
|
|
46
|
+
export declare function splitSentences(text: string | null | undefined): string[];
|
|
@@ -0,0 +1,179 @@
|
|
|
1
|
+
'use strict';
|
|
2
|
+
|
|
3
|
+
// src/contracts/sentences.ts
|
|
4
|
+
var ABBREVIATIONS = /* @__PURE__ */ new Set([
|
|
5
|
+
"mr",
|
|
6
|
+
"mrs",
|
|
7
|
+
"ms",
|
|
8
|
+
"dr",
|
|
9
|
+
"prof",
|
|
10
|
+
"st",
|
|
11
|
+
"jr",
|
|
12
|
+
"sr",
|
|
13
|
+
"vs",
|
|
14
|
+
"inc",
|
|
15
|
+
"co",
|
|
16
|
+
"corp",
|
|
17
|
+
"ltd",
|
|
18
|
+
"llc",
|
|
19
|
+
"approx",
|
|
20
|
+
"dept",
|
|
21
|
+
"jan",
|
|
22
|
+
"feb",
|
|
23
|
+
"mar",
|
|
24
|
+
"apr",
|
|
25
|
+
"jun",
|
|
26
|
+
"jul",
|
|
27
|
+
"aug",
|
|
28
|
+
"sep",
|
|
29
|
+
"sept",
|
|
30
|
+
"oct",
|
|
31
|
+
"nov",
|
|
32
|
+
"dec",
|
|
33
|
+
"gen",
|
|
34
|
+
"sen",
|
|
35
|
+
"gov",
|
|
36
|
+
"lt",
|
|
37
|
+
"col",
|
|
38
|
+
"capt",
|
|
39
|
+
"sgt",
|
|
40
|
+
"cpl",
|
|
41
|
+
"maj",
|
|
42
|
+
"adm",
|
|
43
|
+
"pres",
|
|
44
|
+
"atty",
|
|
45
|
+
"bros",
|
|
46
|
+
"mfg",
|
|
47
|
+
"assoc",
|
|
48
|
+
"univ",
|
|
49
|
+
"bldg",
|
|
50
|
+
"ste",
|
|
51
|
+
"ave",
|
|
52
|
+
"blvd",
|
|
53
|
+
"rd",
|
|
54
|
+
"ln",
|
|
55
|
+
"hwy",
|
|
56
|
+
"pkwy",
|
|
57
|
+
"pl"
|
|
58
|
+
]);
|
|
59
|
+
var CAPITALISED_ABBREVIATIONS = /* @__PURE__ */ new Set([
|
|
60
|
+
"Mon",
|
|
61
|
+
"Tue",
|
|
62
|
+
"Tues",
|
|
63
|
+
"Wed",
|
|
64
|
+
"Thu",
|
|
65
|
+
"Thur",
|
|
66
|
+
"Thurs",
|
|
67
|
+
"Fri",
|
|
68
|
+
"Sat",
|
|
69
|
+
"Sun",
|
|
70
|
+
"Mt",
|
|
71
|
+
"Ft",
|
|
72
|
+
"Ct",
|
|
73
|
+
"Rep",
|
|
74
|
+
"Rev",
|
|
75
|
+
"Hon"
|
|
76
|
+
]);
|
|
77
|
+
var BEFORE_NUMBER = /* @__PURE__ */ new Set([
|
|
78
|
+
"no",
|
|
79
|
+
"nos",
|
|
80
|
+
"fig",
|
|
81
|
+
"figs",
|
|
82
|
+
"vol",
|
|
83
|
+
"vols",
|
|
84
|
+
"ch",
|
|
85
|
+
"sec",
|
|
86
|
+
"pp",
|
|
87
|
+
"cf",
|
|
88
|
+
"ca",
|
|
89
|
+
"est",
|
|
90
|
+
"ext",
|
|
91
|
+
"tel",
|
|
92
|
+
"ph",
|
|
93
|
+
"ste",
|
|
94
|
+
"apt",
|
|
95
|
+
"bldg",
|
|
96
|
+
"rm",
|
|
97
|
+
"hwy",
|
|
98
|
+
"rt",
|
|
99
|
+
"rte"
|
|
100
|
+
]);
|
|
101
|
+
var DOTTED_SHORT = /^\d*(?:[A-Za-z]{1,2}\.)+[A-Za-z]{1,2}$/;
|
|
102
|
+
var DOTTED_CAPS = /^(?:[A-Z][A-Za-z]*\.)+[A-Z][A-Za-z]*$/;
|
|
103
|
+
var ABBREVIATION_RANGE = /[A-Za-z]\.[-–—/]/;
|
|
104
|
+
var LIST_MARKER_ALONE = /^\s*(?:(?:step|tip|reason|rule|point|item|option)\s+)?\d{1,3}\.$/i;
|
|
105
|
+
var LIST_MARKER_AFTER_COLON = /:\s+\d{1,3}\.$/;
|
|
106
|
+
var LIST_MARKER_REACH = 24;
|
|
107
|
+
var OPENERS = /* @__PURE__ */ new Set(['"', "'", "\u201C", "\u2018", "(", "["]);
|
|
108
|
+
var CLOSERS = /* @__PURE__ */ new Set(['"', "'", "\u201D", "\u2019", ")", "]"]);
|
|
109
|
+
var SPACE = /\s/;
|
|
110
|
+
var LETTER = /\p{L}/u;
|
|
111
|
+
var DIGIT = /[0-9]/;
|
|
112
|
+
var SENTENCE_START = /[A-Z0-9]/;
|
|
113
|
+
var isStop = (c) => c === "." || c === "!" || c === "?";
|
|
114
|
+
var isSpace = (c) => SPACE.test(c);
|
|
115
|
+
function startsSentence(source, at) {
|
|
116
|
+
const c = OPENERS.has(source[at]) ? source[at + 1] : source[at];
|
|
117
|
+
return c !== void 0 && SENTENCE_START.test(c);
|
|
118
|
+
}
|
|
119
|
+
function digitFollows(source, at) {
|
|
120
|
+
const c = OPENERS.has(source[at]) ? source[at + 1] : source[at];
|
|
121
|
+
return c !== void 0 && DIGIT.test(c);
|
|
122
|
+
}
|
|
123
|
+
function isDottedAbbreviation(bare) {
|
|
124
|
+
const compound = Math.max(bare.lastIndexOf("-"), bare.lastIndexOf("\u2013"), bare.lastIndexOf("\u2014"), bare.lastIndexOf("/"));
|
|
125
|
+
const last = bare.slice(compound + 1);
|
|
126
|
+
return DOTTED_SHORT.test(last) || DOTTED_CAPS.test(last) || ABBREVIATION_RANGE.test(bare);
|
|
127
|
+
}
|
|
128
|
+
function endsWithAbbreviation(source, end, digitNext) {
|
|
129
|
+
let from = end;
|
|
130
|
+
while (from > 0 && !isSpace(source[from - 1])) from--;
|
|
131
|
+
let to = end;
|
|
132
|
+
while (from < to && OPENERS.has(source[from])) from++;
|
|
133
|
+
while (to > from && CLOSERS.has(source[to - 1])) to--;
|
|
134
|
+
while (to > from && source[to - 1] === ".") to--;
|
|
135
|
+
const bare = source.slice(from, to);
|
|
136
|
+
const lower = bare.toLowerCase();
|
|
137
|
+
return isDottedAbbreviation(bare) || /^[A-Za-z]$/.test(bare) || ABBREVIATIONS.has(lower) || CAPITALISED_ABBREVIATIONS.has(bare) || digitNext && BEFORE_NUMBER.has(lower);
|
|
138
|
+
}
|
|
139
|
+
function endsWithListMarker(source, start, end) {
|
|
140
|
+
const from = Math.max(start, end - LIST_MARKER_REACH);
|
|
141
|
+
const tail = source.slice(from, end);
|
|
142
|
+
return from === start && LIST_MARKER_ALONE.test(tail) || LIST_MARKER_AFTER_COLON.test(tail);
|
|
143
|
+
}
|
|
144
|
+
function splitSentences(text) {
|
|
145
|
+
const source = (text ?? "").trim();
|
|
146
|
+
if (!source) return [];
|
|
147
|
+
const sentences = [];
|
|
148
|
+
let start = 0;
|
|
149
|
+
let scanned = 0;
|
|
150
|
+
let sawLetter = false;
|
|
151
|
+
let i = 0;
|
|
152
|
+
while (i < source.length) {
|
|
153
|
+
if (!isStop(source[i])) {
|
|
154
|
+
i++;
|
|
155
|
+
continue;
|
|
156
|
+
}
|
|
157
|
+
let endsStrong = false;
|
|
158
|
+
while (i < source.length && isStop(source[i])) {
|
|
159
|
+
if (source[i] !== ".") endsStrong = true;
|
|
160
|
+
i++;
|
|
161
|
+
}
|
|
162
|
+
while (i < source.length && CLOSERS.has(source[i])) i++;
|
|
163
|
+
let next = i;
|
|
164
|
+
while (next < source.length && isSpace(source[next])) next++;
|
|
165
|
+
if (next === i || !startsSentence(source, next)) continue;
|
|
166
|
+
if (!sawLetter) sawLetter = LETTER.test(source.slice(scanned, i));
|
|
167
|
+
scanned = i;
|
|
168
|
+
if (!sawLetter) continue;
|
|
169
|
+
if (!endsStrong && (endsWithAbbreviation(source, i, digitFollows(source, next)) || endsWithListMarker(source, start, i))) continue;
|
|
170
|
+
sentences.push(source.slice(start, i).trim());
|
|
171
|
+
start = i;
|
|
172
|
+
sawLetter = false;
|
|
173
|
+
}
|
|
174
|
+
const rest = source.slice(start).trim();
|
|
175
|
+
if (rest) sentences.push(rest);
|
|
176
|
+
return sentences;
|
|
177
|
+
}
|
|
178
|
+
|
|
179
|
+
exports.splitSentences = splitSentences;
|
|
@@ -0,0 +1,177 @@
|
|
|
1
|
+
// src/contracts/sentences.ts
|
|
2
|
+
var ABBREVIATIONS = /* @__PURE__ */ new Set([
|
|
3
|
+
"mr",
|
|
4
|
+
"mrs",
|
|
5
|
+
"ms",
|
|
6
|
+
"dr",
|
|
7
|
+
"prof",
|
|
8
|
+
"st",
|
|
9
|
+
"jr",
|
|
10
|
+
"sr",
|
|
11
|
+
"vs",
|
|
12
|
+
"inc",
|
|
13
|
+
"co",
|
|
14
|
+
"corp",
|
|
15
|
+
"ltd",
|
|
16
|
+
"llc",
|
|
17
|
+
"approx",
|
|
18
|
+
"dept",
|
|
19
|
+
"jan",
|
|
20
|
+
"feb",
|
|
21
|
+
"mar",
|
|
22
|
+
"apr",
|
|
23
|
+
"jun",
|
|
24
|
+
"jul",
|
|
25
|
+
"aug",
|
|
26
|
+
"sep",
|
|
27
|
+
"sept",
|
|
28
|
+
"oct",
|
|
29
|
+
"nov",
|
|
30
|
+
"dec",
|
|
31
|
+
"gen",
|
|
32
|
+
"sen",
|
|
33
|
+
"gov",
|
|
34
|
+
"lt",
|
|
35
|
+
"col",
|
|
36
|
+
"capt",
|
|
37
|
+
"sgt",
|
|
38
|
+
"cpl",
|
|
39
|
+
"maj",
|
|
40
|
+
"adm",
|
|
41
|
+
"pres",
|
|
42
|
+
"atty",
|
|
43
|
+
"bros",
|
|
44
|
+
"mfg",
|
|
45
|
+
"assoc",
|
|
46
|
+
"univ",
|
|
47
|
+
"bldg",
|
|
48
|
+
"ste",
|
|
49
|
+
"ave",
|
|
50
|
+
"blvd",
|
|
51
|
+
"rd",
|
|
52
|
+
"ln",
|
|
53
|
+
"hwy",
|
|
54
|
+
"pkwy",
|
|
55
|
+
"pl"
|
|
56
|
+
]);
|
|
57
|
+
var CAPITALISED_ABBREVIATIONS = /* @__PURE__ */ new Set([
|
|
58
|
+
"Mon",
|
|
59
|
+
"Tue",
|
|
60
|
+
"Tues",
|
|
61
|
+
"Wed",
|
|
62
|
+
"Thu",
|
|
63
|
+
"Thur",
|
|
64
|
+
"Thurs",
|
|
65
|
+
"Fri",
|
|
66
|
+
"Sat",
|
|
67
|
+
"Sun",
|
|
68
|
+
"Mt",
|
|
69
|
+
"Ft",
|
|
70
|
+
"Ct",
|
|
71
|
+
"Rep",
|
|
72
|
+
"Rev",
|
|
73
|
+
"Hon"
|
|
74
|
+
]);
|
|
75
|
+
var BEFORE_NUMBER = /* @__PURE__ */ new Set([
|
|
76
|
+
"no",
|
|
77
|
+
"nos",
|
|
78
|
+
"fig",
|
|
79
|
+
"figs",
|
|
80
|
+
"vol",
|
|
81
|
+
"vols",
|
|
82
|
+
"ch",
|
|
83
|
+
"sec",
|
|
84
|
+
"pp",
|
|
85
|
+
"cf",
|
|
86
|
+
"ca",
|
|
87
|
+
"est",
|
|
88
|
+
"ext",
|
|
89
|
+
"tel",
|
|
90
|
+
"ph",
|
|
91
|
+
"ste",
|
|
92
|
+
"apt",
|
|
93
|
+
"bldg",
|
|
94
|
+
"rm",
|
|
95
|
+
"hwy",
|
|
96
|
+
"rt",
|
|
97
|
+
"rte"
|
|
98
|
+
]);
|
|
99
|
+
var DOTTED_SHORT = /^\d*(?:[A-Za-z]{1,2}\.)+[A-Za-z]{1,2}$/;
|
|
100
|
+
var DOTTED_CAPS = /^(?:[A-Z][A-Za-z]*\.)+[A-Z][A-Za-z]*$/;
|
|
101
|
+
var ABBREVIATION_RANGE = /[A-Za-z]\.[-–—/]/;
|
|
102
|
+
var LIST_MARKER_ALONE = /^\s*(?:(?:step|tip|reason|rule|point|item|option)\s+)?\d{1,3}\.$/i;
|
|
103
|
+
var LIST_MARKER_AFTER_COLON = /:\s+\d{1,3}\.$/;
|
|
104
|
+
var LIST_MARKER_REACH = 24;
|
|
105
|
+
var OPENERS = /* @__PURE__ */ new Set(['"', "'", "\u201C", "\u2018", "(", "["]);
|
|
106
|
+
var CLOSERS = /* @__PURE__ */ new Set(['"', "'", "\u201D", "\u2019", ")", "]"]);
|
|
107
|
+
var SPACE = /\s/;
|
|
108
|
+
var LETTER = /\p{L}/u;
|
|
109
|
+
var DIGIT = /[0-9]/;
|
|
110
|
+
var SENTENCE_START = /[A-Z0-9]/;
|
|
111
|
+
var isStop = (c) => c === "." || c === "!" || c === "?";
|
|
112
|
+
var isSpace = (c) => SPACE.test(c);
|
|
113
|
+
function startsSentence(source, at) {
|
|
114
|
+
const c = OPENERS.has(source[at]) ? source[at + 1] : source[at];
|
|
115
|
+
return c !== void 0 && SENTENCE_START.test(c);
|
|
116
|
+
}
|
|
117
|
+
function digitFollows(source, at) {
|
|
118
|
+
const c = OPENERS.has(source[at]) ? source[at + 1] : source[at];
|
|
119
|
+
return c !== void 0 && DIGIT.test(c);
|
|
120
|
+
}
|
|
121
|
+
function isDottedAbbreviation(bare) {
|
|
122
|
+
const compound = Math.max(bare.lastIndexOf("-"), bare.lastIndexOf("\u2013"), bare.lastIndexOf("\u2014"), bare.lastIndexOf("/"));
|
|
123
|
+
const last = bare.slice(compound + 1);
|
|
124
|
+
return DOTTED_SHORT.test(last) || DOTTED_CAPS.test(last) || ABBREVIATION_RANGE.test(bare);
|
|
125
|
+
}
|
|
126
|
+
function endsWithAbbreviation(source, end, digitNext) {
|
|
127
|
+
let from = end;
|
|
128
|
+
while (from > 0 && !isSpace(source[from - 1])) from--;
|
|
129
|
+
let to = end;
|
|
130
|
+
while (from < to && OPENERS.has(source[from])) from++;
|
|
131
|
+
while (to > from && CLOSERS.has(source[to - 1])) to--;
|
|
132
|
+
while (to > from && source[to - 1] === ".") to--;
|
|
133
|
+
const bare = source.slice(from, to);
|
|
134
|
+
const lower = bare.toLowerCase();
|
|
135
|
+
return isDottedAbbreviation(bare) || /^[A-Za-z]$/.test(bare) || ABBREVIATIONS.has(lower) || CAPITALISED_ABBREVIATIONS.has(bare) || digitNext && BEFORE_NUMBER.has(lower);
|
|
136
|
+
}
|
|
137
|
+
function endsWithListMarker(source, start, end) {
|
|
138
|
+
const from = Math.max(start, end - LIST_MARKER_REACH);
|
|
139
|
+
const tail = source.slice(from, end);
|
|
140
|
+
return from === start && LIST_MARKER_ALONE.test(tail) || LIST_MARKER_AFTER_COLON.test(tail);
|
|
141
|
+
}
|
|
142
|
+
function splitSentences(text) {
|
|
143
|
+
const source = (text ?? "").trim();
|
|
144
|
+
if (!source) return [];
|
|
145
|
+
const sentences = [];
|
|
146
|
+
let start = 0;
|
|
147
|
+
let scanned = 0;
|
|
148
|
+
let sawLetter = false;
|
|
149
|
+
let i = 0;
|
|
150
|
+
while (i < source.length) {
|
|
151
|
+
if (!isStop(source[i])) {
|
|
152
|
+
i++;
|
|
153
|
+
continue;
|
|
154
|
+
}
|
|
155
|
+
let endsStrong = false;
|
|
156
|
+
while (i < source.length && isStop(source[i])) {
|
|
157
|
+
if (source[i] !== ".") endsStrong = true;
|
|
158
|
+
i++;
|
|
159
|
+
}
|
|
160
|
+
while (i < source.length && CLOSERS.has(source[i])) i++;
|
|
161
|
+
let next = i;
|
|
162
|
+
while (next < source.length && isSpace(source[next])) next++;
|
|
163
|
+
if (next === i || !startsSentence(source, next)) continue;
|
|
164
|
+
if (!sawLetter) sawLetter = LETTER.test(source.slice(scanned, i));
|
|
165
|
+
scanned = i;
|
|
166
|
+
if (!sawLetter) continue;
|
|
167
|
+
if (!endsStrong && (endsWithAbbreviation(source, i, digitFollows(source, next)) || endsWithListMarker(source, start, i))) continue;
|
|
168
|
+
sentences.push(source.slice(start, i).trim());
|
|
169
|
+
start = i;
|
|
170
|
+
sawLetter = false;
|
|
171
|
+
}
|
|
172
|
+
const rest = source.slice(start).trim();
|
|
173
|
+
if (rest) sentences.push(rest);
|
|
174
|
+
return sentences;
|
|
175
|
+
}
|
|
176
|
+
|
|
177
|
+
export { splitSentences };
|
package/dist/site-cache.d.ts
CHANGED
|
@@ -27,6 +27,8 @@ export declare const SITE_CACHE_TAGS: {
|
|
|
27
27
|
readonly taxonomy: 'editorial-taxonomy';
|
|
28
28
|
/** Portfolio items (agency-site-kit). */
|
|
29
29
|
readonly portfolio: 'portfolio';
|
|
30
|
+
/** Products, services, events and membership plans (commerce-kit). */
|
|
31
|
+
readonly commerce: 'commerce';
|
|
30
32
|
};
|
|
31
33
|
export type SiteCacheTag = (typeof SITE_CACHE_TAGS)[keyof typeof SITE_CACHE_TAGS];
|
|
32
34
|
/** The body Sonor POSTs to the revalidation route. */
|
package/dist/site-cache.js
CHANGED
|
@@ -12,7 +12,9 @@ var SITE_CACHE_TAGS = {
|
|
|
12
12
|
/** Article categories and tags. */
|
|
13
13
|
taxonomy: "editorial-taxonomy",
|
|
14
14
|
/** Portfolio items (agency-site-kit). */
|
|
15
|
-
portfolio: "portfolio"
|
|
15
|
+
portfolio: "portfolio",
|
|
16
|
+
/** Products, services, events and membership plans (commerce-kit). */
|
|
17
|
+
commerce: "commerce"
|
|
16
18
|
};
|
|
17
19
|
function isSiteRevalidatePong(body) {
|
|
18
20
|
return typeof body === "object" && body !== null && body.ok === true && body.ping === true;
|
package/dist/site-cache.mjs
CHANGED
|
@@ -10,7 +10,9 @@ var SITE_CACHE_TAGS = {
|
|
|
10
10
|
/** Article categories and tags. */
|
|
11
11
|
taxonomy: "editorial-taxonomy",
|
|
12
12
|
/** Portfolio items (agency-site-kit). */
|
|
13
|
-
portfolio: "portfolio"
|
|
13
|
+
portfolio: "portfolio",
|
|
14
|
+
/** Products, services, events and membership plans (commerce-kit). */
|
|
15
|
+
commerce: "commerce"
|
|
14
16
|
};
|
|
15
17
|
function isSiteRevalidatePong(body) {
|
|
16
18
|
return typeof body === "object" && body !== null && body.ok === true && body.ping === true;
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@sonordev/contracts",
|
|
3
|
-
"version": "2.
|
|
4
|
-
"description": "The rules Sonor's sites, APIs and dashboard share: popup blocks and site design tokens, site hosts, SEO page identity, title quality, llms.txt sanitizers, the forms honeypot, fleet heartbeats, slot content and signing, site cache tags, portfolio metrics, the house voice's contraction rule, how a proposal's site plan counts its pages, what separates an error page from a page's content,
|
|
3
|
+
"version": "2.6.0",
|
|
4
|
+
"description": "The rules Sonor's sites, APIs and dashboard share: popup blocks and site design tokens, site hosts, SEO page identity, title quality, llms.txt sanitizers, the forms honeypot, fleet heartbeats, slot content and signing, site cache tags, portfolio metrics, the house voice's contraction rule, how a proposal's site plan counts its pages, what separates an error page from a page's content, which stored JSON-LD is a template's unfilled placeholder, where prose splits into sentences, and which Sonor records Signal reads into a project's knowledge and the address each one is stored under. Pure functions, no dependencies.",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"repository": {
|
|
7
7
|
"type": "git",
|
|
@@ -100,6 +100,16 @@
|
|
|
100
100
|
"import": "./dist/schema-placeholders.mjs",
|
|
101
101
|
"require": "./dist/schema-placeholders.js"
|
|
102
102
|
},
|
|
103
|
+
"./sentences": {
|
|
104
|
+
"types": "./dist/sentences.d.ts",
|
|
105
|
+
"import": "./dist/sentences.mjs",
|
|
106
|
+
"require": "./dist/sentences.js"
|
|
107
|
+
},
|
|
108
|
+
"./knowledge": {
|
|
109
|
+
"types": "./dist/knowledge.d.ts",
|
|
110
|
+
"import": "./dist/knowledge.mjs",
|
|
111
|
+
"require": "./dist/knowledge.js"
|
|
112
|
+
},
|
|
103
113
|
"./package.json": "./package.json"
|
|
104
114
|
},
|
|
105
115
|
"typesVersions": {
|