@sonordev/contracts 2.4.0 → 2.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +13 -0
- package/README.md +63 -0
- package/dist/sentences.d.ts +46 -0
- package/dist/sentences.js +179 -0
- package/dist/sentences.mjs +177 -0
- package/dist/site-cache.d.ts +2 -0
- package/dist/site-cache.js +3 -1
- package/dist/site-cache.mjs +3 -1
- package/package.json +7 -2
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,18 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 2.5.0
|
|
4
|
+
|
|
5
|
+
A minor release: one new entry, `@sonordev/contracts/sentences`, and a cache tag for commerce. Every other entry is unchanged.
|
|
6
|
+
|
|
7
|
+
- **New entry `@sonordev/contracts/sentences`**: where prose splits into sentences, so every app that cuts an excerpt, a social slide or a summary out of an article ends sentences in the same places.
|
|
8
|
+
- `splitSentences(text)` returns the trimmed sentences of `text`. It's lossless: joining the result with spaces gives back every word of the input, in order. The usual one-liners aren't: `text.match(/[^.!?]+[.!?]+(?=\s|$)/g)` skips ahead at a period it can't use (the first one in "9 p.m.") and drops every word before its next match, and `text.match(/[^.!?]+[.!?]+/g)` cuts "9 p.m." into "9 p." and "m.".
|
|
9
|
+
- It doesn't end a sentence at a dotted abbreviation of letters ("p.m.", "U.S.", "e.g."), a single letter ("J."), a common short form (titles, company and street suffixes, months, and days of the week when capitalized: "Dr.", "Inc.", "Ave.", "Sept.", "Mon."), a label before a number ("No. 5", "Fig. 2") or a list number ("1.", "Step 2."), and never before a lowercase word. A piece with no letter in it ("...", "1.") joins the next sentence. A `!` or `?` right after an abbreviation still ends one.
|
|
10
|
+
- Prices, versions, domains and email addresses ("$4.50", "v2.1", "example.com") aren't abbreviations: a sentence can end after them.
|
|
11
|
+
- It runs in linear time: a megabyte of text takes milliseconds, however many abbreviations it holds.
|
|
12
|
+
- A newline or a list marker isn't a boundary by itself, so split on lines or paragraphs first when those matter.
|
|
13
|
+
- Empty, whitespace-only, `null` and `undefined` input give `[]`.
|
|
14
|
+
- **`site-cache`**: `SITE_CACHE_TAGS.commerce` (`commerce`) is the tag for products, services, events and membership plans. commerce-kit tags its server fetches with it, and the Sonor API expires it when an offering changes. The other tags are unchanged.
|
|
15
|
+
|
|
3
16
|
## 2.4.0
|
|
4
17
|
|
|
5
18
|
A minor release: one new entry, `@sonordev/contracts/schema-placeholders`, and the llms.txt sanitizers stop cutting mid-word. Every other entry is unchanged.
|
package/README.md
CHANGED
|
@@ -26,6 +26,7 @@ pnpm add @sonordev/contracts
|
|
|
26
26
|
| `@sonordev/contracts/voice` | The house voice's contraction rule. `findUncontracted` and `contractText` find and apply full forms that should be contractions; `findContracted` finds the contractions a question-form heading should expand | anything that checks or rewrites copy: Sonor's APIs and dashboard |
|
|
27
27
|
| `@sonordev/contracts/error-page` | Whether stored text is a rendered error page rather than a page's content, and whether generated copy says the page itself is missing | anything that stores page text or writes page copy from it: the Sonor API's page-view and sitemap writers, Signal's metadata generators |
|
|
28
28
|
| `@sonordev/contracts/schema-placeholders` | Whether stored JSON-LD carries a template's unfilled slots (`example.com` URLs, an `Organization` named "Example", a placeholder phone, `[Resident Name]`, `{plan.name}`), and the value without them | site-kit's schema output, the Sonor API's schema reads and writes |
|
|
29
|
+
| `@sonordev/contracts/sentences` | Where prose splits into sentences, without dropping a word or ending one at "p.m." (`splitSentences`) | anything that cuts an excerpt, a social slide or a summary out of an article: the Sonor API's carousels, Signal's article pipeline |
|
|
29
30
|
|
|
30
31
|
## Schema placeholders
|
|
31
32
|
|
|
@@ -287,6 +288,68 @@ A text-only check can't prove a page failed. Where you observed the HTTP
|
|
|
287
288
|
status, require 400 or more as well and use this as the fallback; the header of
|
|
288
289
|
`src/contracts/error-page.ts` lists the other known limits.
|
|
289
290
|
|
|
291
|
+
## The sentences contract
|
|
292
|
+
|
|
293
|
+
Cutting an excerpt, a slide or a summary out of an article starts with
|
|
294
|
+
finding where its sentences end, and the usual one-liners for that get it
|
|
295
|
+
wrong. `text.match(/[^.!?]+[.!?]+(?=\s|$)/g)` can't end a match at the first
|
|
296
|
+
period of "9 p.m.", so it skips ahead and drops every word before the next
|
|
297
|
+
match: "We open at 9 p.m. daily." comes back as "m." and " daily.". Without
|
|
298
|
+
the lookahead, `text.match(/[^.!?]+[.!?]+/g)` cuts "9 p.m." into "9 p." and
|
|
299
|
+
"m.". Both also drop any text after the last full stop. `splitSentences` is
|
|
300
|
+
the one splitter to use instead.
|
|
301
|
+
|
|
302
|
+
- **Lossless.** Joining the result with spaces gives back every word of the
|
|
303
|
+
input, in order. Nothing is dropped, added or reordered, and each sentence
|
|
304
|
+
comes back trimmed and non-empty.
|
|
305
|
+
- **Where a sentence ends.** At `.`, `!` or `?` (and any closing `"`, `'`, `”`,
|
|
306
|
+
`’`, `)` or `]` after it) followed by whitespace and then `A` to `Z` or `0`
|
|
307
|
+
to `9`, optionally after one opening `"`, `'`, `“`, `‘`, `(` or `[`. Never
|
|
308
|
+
before a lowercase word or a symbol. Only ASCII capitals count: a sentence
|
|
309
|
+
that opens with an accented, Greek or Cyrillic capital, a currency sign, `#`,
|
|
310
|
+
`@` or an emoji stays joined to the one before it.
|
|
311
|
+
- **Where it doesn't.** After a dotted abbreviation made of letters ("p.m.",
|
|
312
|
+
"U.S.", "e.g.", "Ph.D."), a single letter of either case (an initial like
|
|
313
|
+
"J.", but also "Plan A."), or a short form on its list: titles, company
|
|
314
|
+
suffixes, months, street suffixes and days of the week ("Dr.", "Inc.",
|
|
315
|
+
"Sept.", "Ave.", "Mon."). A few of those only count when capitalized, because
|
|
316
|
+
the lowercase word ends sentences too ("Mon." is a day, "the sun." isn't).
|
|
317
|
+
A label with a number after it stays together ("No. 5", "Fig. 2",
|
|
318
|
+
"Ste. 200"), while "No." before anything else ends a sentence. A list number
|
|
319
|
+
("1.", "Step 2.", "Do three things: 1.") stays with its item.
|
|
320
|
+
- **`!` and `?`.** An abbreviation only holds back a period. "Open until 9
|
|
321
|
+
p.m.? Yes!" is two sentences, under the same start-of-sentence rule as above.
|
|
322
|
+
- **Prices, versions and web addresses** ("$4.50", "v2.1", "example.com",
|
|
323
|
+
"info@example.com") aren't abbreviations: a sentence can end after them.
|
|
324
|
+
- **Never a bare mark.** A piece with no letter in it ("...", "?!", "1.")
|
|
325
|
+
isn't a sentence. It joins the next one.
|
|
326
|
+
- **Prose, not layout.** A newline or a list marker doesn't start a new
|
|
327
|
+
sentence by itself. If lines or paragraphs matter, split on those first and
|
|
328
|
+
call `splitSentences` on each one.
|
|
329
|
+
- **Empty is `[]`.** Empty, whitespace-only, `null` and `undefined` input all
|
|
330
|
+
give an empty array, never `null`, so callers can iterate without a guard.
|
|
331
|
+
- **Linear time.** One pass over the text. A megabyte takes milliseconds,
|
|
332
|
+
however many abbreviations, stops or quotes it holds.
|
|
333
|
+
|
|
334
|
+
```ts
|
|
335
|
+
import { splitSentences } from '@sonordev/contracts/sentences'
|
|
336
|
+
|
|
337
|
+
splitSentences('We open at 9 a.m. daily. Walk-ins are welcome.')
|
|
338
|
+
// ['We open at 9 a.m. daily.', 'Walk-ins are welcome.']
|
|
339
|
+
|
|
340
|
+
splitSentences(excerptSource).slice(0, 2).join(' ')
|
|
341
|
+
// the first two sentences, whole
|
|
342
|
+
```
|
|
343
|
+
|
|
344
|
+
Known limits. Where it can't be sure it leans toward joining, because a
|
|
345
|
+
sentence that runs long costs less than a lost word, and a caller that caps
|
|
346
|
+
length should cut at a word. It can't tell an abbreviation that ends a
|
|
347
|
+
sentence from one that doesn't ("Contact Acme Inc. Our team ..." and "... in
|
|
348
|
+
Washington, D.C. Residents ..." each stay in one piece), it only knows English
|
|
349
|
+
short forms (a period after one it doesn't know ends a sentence, so "Sra.
|
|
350
|
+
García" splits after "Sra."), and the next sentence has to start with `A` to
|
|
351
|
+
`Z` or a digit.
|
|
352
|
+
|
|
290
353
|
## Why it exists
|
|
291
354
|
|
|
292
355
|
A rule that lives in several copies is how two of them end up disagreeing.
|
|
@@ -0,0 +1,46 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Sentences contract: how prose is split into sentences, so the APIs that cut
|
|
3
|
+
* an excerpt, a carousel slide or a summary out of an article agree on where
|
|
4
|
+
* a sentence ends. The APIs import it as `@sonordev/contracts/sentences`.
|
|
5
|
+
*
|
|
6
|
+
* Lossless: joining the result with spaces gives back every word of the input,
|
|
7
|
+
* in order. The usual one-liner, `text.match(/[^.!?]+[.!?]+(?=\s|$)/g)`, isn't.
|
|
8
|
+
* A period it can't use, like the first one in "9 p.m. call", makes match()
|
|
9
|
+
* skip ahead and silently drop every word before its next match, so a
|
|
10
|
+
* sentence about a 9 p.m. call comes back as "m.". Don't re-fork a regex for
|
|
11
|
+
* this.
|
|
12
|
+
*
|
|
13
|
+
* A sentence ends at . ! or ? (and any closing quote or bracket after it)
|
|
14
|
+
* followed by whitespace, then an optional opening quote or bracket, then
|
|
15
|
+
* A to Z or 0 to 9. Never before a lowercase word or a symbol, and only ASCII
|
|
16
|
+
* capitals count. It doesn't end at:
|
|
17
|
+
* - an abbreviation: a dotted one made of letters ("p.m.", "U.S.", "e.g.",
|
|
18
|
+
* "Ph.D."), a single letter ("J.", "Plan A."), or a short form listed below
|
|
19
|
+
* (titles, months, days, company suffixes, street suffixes);
|
|
20
|
+
* - a label that has a number after it ("No. 5", "Fig. 2", "Ste. 200");
|
|
21
|
+
* - a list number ("1.", "Step 2.", "Do three things: 1.").
|
|
22
|
+
* A piece with no letter in it ("...", "1.") isn't a sentence and joins the
|
|
23
|
+
* next one. A "!" or "?" is never held back by an abbreviation (only a period
|
|
24
|
+
* is), though the same start-of-sentence rule applies after it.
|
|
25
|
+
*
|
|
26
|
+
* Prices, versions, domains and email addresses ("$4.50", "v2.1",
|
|
27
|
+
* "example.com") are not abbreviations: a sentence can end after them.
|
|
28
|
+
*
|
|
29
|
+
* Where it is unsure it leans toward not splitting, because a sentence that
|
|
30
|
+
* runs long costs less than a lost word. A period after a short form that
|
|
31
|
+
* isn't listed ends a sentence like any other.
|
|
32
|
+
*
|
|
33
|
+
* It reads sentences, not layout. A newline or a list marker doesn't start a
|
|
34
|
+
* new sentence by itself, so split text into lines or paragraphs first when
|
|
35
|
+
* those matter, then split each one.
|
|
36
|
+
*/
|
|
37
|
+
/**
|
|
38
|
+
* Splits prose into trimmed sentences without dropping or reordering a word.
|
|
39
|
+
* Empty, blank and nullish input give `[]`, never `null`.
|
|
40
|
+
*
|
|
41
|
+
* One pass, linear in the length of the text: each run of . ! ? (and the
|
|
42
|
+
* closing quotes or brackets after it) is looked at once, an abbreviation or
|
|
43
|
+
* list-marker check reads only the last few characters, and the check for a
|
|
44
|
+
* letter reads each character once.
|
|
45
|
+
*/
|
|
46
|
+
export declare function splitSentences(text: string | null | undefined): string[];
|
|
@@ -0,0 +1,179 @@
|
|
|
1
|
+
'use strict';
|
|
2
|
+
|
|
3
|
+
// src/contracts/sentences.ts
|
|
4
|
+
var ABBREVIATIONS = /* @__PURE__ */ new Set([
|
|
5
|
+
"mr",
|
|
6
|
+
"mrs",
|
|
7
|
+
"ms",
|
|
8
|
+
"dr",
|
|
9
|
+
"prof",
|
|
10
|
+
"st",
|
|
11
|
+
"jr",
|
|
12
|
+
"sr",
|
|
13
|
+
"vs",
|
|
14
|
+
"inc",
|
|
15
|
+
"co",
|
|
16
|
+
"corp",
|
|
17
|
+
"ltd",
|
|
18
|
+
"llc",
|
|
19
|
+
"approx",
|
|
20
|
+
"dept",
|
|
21
|
+
"jan",
|
|
22
|
+
"feb",
|
|
23
|
+
"mar",
|
|
24
|
+
"apr",
|
|
25
|
+
"jun",
|
|
26
|
+
"jul",
|
|
27
|
+
"aug",
|
|
28
|
+
"sep",
|
|
29
|
+
"sept",
|
|
30
|
+
"oct",
|
|
31
|
+
"nov",
|
|
32
|
+
"dec",
|
|
33
|
+
"gen",
|
|
34
|
+
"sen",
|
|
35
|
+
"gov",
|
|
36
|
+
"lt",
|
|
37
|
+
"col",
|
|
38
|
+
"capt",
|
|
39
|
+
"sgt",
|
|
40
|
+
"cpl",
|
|
41
|
+
"maj",
|
|
42
|
+
"adm",
|
|
43
|
+
"pres",
|
|
44
|
+
"atty",
|
|
45
|
+
"bros",
|
|
46
|
+
"mfg",
|
|
47
|
+
"assoc",
|
|
48
|
+
"univ",
|
|
49
|
+
"bldg",
|
|
50
|
+
"ste",
|
|
51
|
+
"ave",
|
|
52
|
+
"blvd",
|
|
53
|
+
"rd",
|
|
54
|
+
"ln",
|
|
55
|
+
"hwy",
|
|
56
|
+
"pkwy",
|
|
57
|
+
"pl"
|
|
58
|
+
]);
|
|
59
|
+
var CAPITALISED_ABBREVIATIONS = /* @__PURE__ */ new Set([
|
|
60
|
+
"Mon",
|
|
61
|
+
"Tue",
|
|
62
|
+
"Tues",
|
|
63
|
+
"Wed",
|
|
64
|
+
"Thu",
|
|
65
|
+
"Thur",
|
|
66
|
+
"Thurs",
|
|
67
|
+
"Fri",
|
|
68
|
+
"Sat",
|
|
69
|
+
"Sun",
|
|
70
|
+
"Mt",
|
|
71
|
+
"Ft",
|
|
72
|
+
"Ct",
|
|
73
|
+
"Rep",
|
|
74
|
+
"Rev",
|
|
75
|
+
"Hon"
|
|
76
|
+
]);
|
|
77
|
+
var BEFORE_NUMBER = /* @__PURE__ */ new Set([
|
|
78
|
+
"no",
|
|
79
|
+
"nos",
|
|
80
|
+
"fig",
|
|
81
|
+
"figs",
|
|
82
|
+
"vol",
|
|
83
|
+
"vols",
|
|
84
|
+
"ch",
|
|
85
|
+
"sec",
|
|
86
|
+
"pp",
|
|
87
|
+
"cf",
|
|
88
|
+
"ca",
|
|
89
|
+
"est",
|
|
90
|
+
"ext",
|
|
91
|
+
"tel",
|
|
92
|
+
"ph",
|
|
93
|
+
"ste",
|
|
94
|
+
"apt",
|
|
95
|
+
"bldg",
|
|
96
|
+
"rm",
|
|
97
|
+
"hwy",
|
|
98
|
+
"rt",
|
|
99
|
+
"rte"
|
|
100
|
+
]);
|
|
101
|
+
var DOTTED_SHORT = /^\d*(?:[A-Za-z]{1,2}\.)+[A-Za-z]{1,2}$/;
|
|
102
|
+
var DOTTED_CAPS = /^(?:[A-Z][A-Za-z]*\.)+[A-Z][A-Za-z]*$/;
|
|
103
|
+
var ABBREVIATION_RANGE = /[A-Za-z]\.[-–—/]/;
|
|
104
|
+
var LIST_MARKER_ALONE = /^\s*(?:(?:step|tip|reason|rule|point|item|option)\s+)?\d{1,3}\.$/i;
|
|
105
|
+
var LIST_MARKER_AFTER_COLON = /:\s+\d{1,3}\.$/;
|
|
106
|
+
var LIST_MARKER_REACH = 24;
|
|
107
|
+
var OPENERS = /* @__PURE__ */ new Set(['"', "'", "\u201C", "\u2018", "(", "["]);
|
|
108
|
+
var CLOSERS = /* @__PURE__ */ new Set(['"', "'", "\u201D", "\u2019", ")", "]"]);
|
|
109
|
+
var SPACE = /\s/;
|
|
110
|
+
var LETTER = /\p{L}/u;
|
|
111
|
+
var DIGIT = /[0-9]/;
|
|
112
|
+
var SENTENCE_START = /[A-Z0-9]/;
|
|
113
|
+
var isStop = (c) => c === "." || c === "!" || c === "?";
|
|
114
|
+
var isSpace = (c) => SPACE.test(c);
|
|
115
|
+
function startsSentence(source, at) {
|
|
116
|
+
const c = OPENERS.has(source[at]) ? source[at + 1] : source[at];
|
|
117
|
+
return c !== void 0 && SENTENCE_START.test(c);
|
|
118
|
+
}
|
|
119
|
+
function digitFollows(source, at) {
|
|
120
|
+
const c = OPENERS.has(source[at]) ? source[at + 1] : source[at];
|
|
121
|
+
return c !== void 0 && DIGIT.test(c);
|
|
122
|
+
}
|
|
123
|
+
function isDottedAbbreviation(bare) {
|
|
124
|
+
const compound = Math.max(bare.lastIndexOf("-"), bare.lastIndexOf("\u2013"), bare.lastIndexOf("\u2014"), bare.lastIndexOf("/"));
|
|
125
|
+
const last = bare.slice(compound + 1);
|
|
126
|
+
return DOTTED_SHORT.test(last) || DOTTED_CAPS.test(last) || ABBREVIATION_RANGE.test(bare);
|
|
127
|
+
}
|
|
128
|
+
function endsWithAbbreviation(source, end, digitNext) {
|
|
129
|
+
let from = end;
|
|
130
|
+
while (from > 0 && !isSpace(source[from - 1])) from--;
|
|
131
|
+
let to = end;
|
|
132
|
+
while (from < to && OPENERS.has(source[from])) from++;
|
|
133
|
+
while (to > from && CLOSERS.has(source[to - 1])) to--;
|
|
134
|
+
while (to > from && source[to - 1] === ".") to--;
|
|
135
|
+
const bare = source.slice(from, to);
|
|
136
|
+
const lower = bare.toLowerCase();
|
|
137
|
+
return isDottedAbbreviation(bare) || /^[A-Za-z]$/.test(bare) || ABBREVIATIONS.has(lower) || CAPITALISED_ABBREVIATIONS.has(bare) || digitNext && BEFORE_NUMBER.has(lower);
|
|
138
|
+
}
|
|
139
|
+
function endsWithListMarker(source, start, end) {
|
|
140
|
+
const from = Math.max(start, end - LIST_MARKER_REACH);
|
|
141
|
+
const tail = source.slice(from, end);
|
|
142
|
+
return from === start && LIST_MARKER_ALONE.test(tail) || LIST_MARKER_AFTER_COLON.test(tail);
|
|
143
|
+
}
|
|
144
|
+
function splitSentences(text) {
|
|
145
|
+
const source = (text ?? "").trim();
|
|
146
|
+
if (!source) return [];
|
|
147
|
+
const sentences = [];
|
|
148
|
+
let start = 0;
|
|
149
|
+
let scanned = 0;
|
|
150
|
+
let sawLetter = false;
|
|
151
|
+
let i = 0;
|
|
152
|
+
while (i < source.length) {
|
|
153
|
+
if (!isStop(source[i])) {
|
|
154
|
+
i++;
|
|
155
|
+
continue;
|
|
156
|
+
}
|
|
157
|
+
let endsStrong = false;
|
|
158
|
+
while (i < source.length && isStop(source[i])) {
|
|
159
|
+
if (source[i] !== ".") endsStrong = true;
|
|
160
|
+
i++;
|
|
161
|
+
}
|
|
162
|
+
while (i < source.length && CLOSERS.has(source[i])) i++;
|
|
163
|
+
let next = i;
|
|
164
|
+
while (next < source.length && isSpace(source[next])) next++;
|
|
165
|
+
if (next === i || !startsSentence(source, next)) continue;
|
|
166
|
+
if (!sawLetter) sawLetter = LETTER.test(source.slice(scanned, i));
|
|
167
|
+
scanned = i;
|
|
168
|
+
if (!sawLetter) continue;
|
|
169
|
+
if (!endsStrong && (endsWithAbbreviation(source, i, digitFollows(source, next)) || endsWithListMarker(source, start, i))) continue;
|
|
170
|
+
sentences.push(source.slice(start, i).trim());
|
|
171
|
+
start = i;
|
|
172
|
+
sawLetter = false;
|
|
173
|
+
}
|
|
174
|
+
const rest = source.slice(start).trim();
|
|
175
|
+
if (rest) sentences.push(rest);
|
|
176
|
+
return sentences;
|
|
177
|
+
}
|
|
178
|
+
|
|
179
|
+
exports.splitSentences = splitSentences;
|
|
@@ -0,0 +1,177 @@
|
|
|
1
|
+
// src/contracts/sentences.ts
|
|
2
|
+
var ABBREVIATIONS = /* @__PURE__ */ new Set([
|
|
3
|
+
"mr",
|
|
4
|
+
"mrs",
|
|
5
|
+
"ms",
|
|
6
|
+
"dr",
|
|
7
|
+
"prof",
|
|
8
|
+
"st",
|
|
9
|
+
"jr",
|
|
10
|
+
"sr",
|
|
11
|
+
"vs",
|
|
12
|
+
"inc",
|
|
13
|
+
"co",
|
|
14
|
+
"corp",
|
|
15
|
+
"ltd",
|
|
16
|
+
"llc",
|
|
17
|
+
"approx",
|
|
18
|
+
"dept",
|
|
19
|
+
"jan",
|
|
20
|
+
"feb",
|
|
21
|
+
"mar",
|
|
22
|
+
"apr",
|
|
23
|
+
"jun",
|
|
24
|
+
"jul",
|
|
25
|
+
"aug",
|
|
26
|
+
"sep",
|
|
27
|
+
"sept",
|
|
28
|
+
"oct",
|
|
29
|
+
"nov",
|
|
30
|
+
"dec",
|
|
31
|
+
"gen",
|
|
32
|
+
"sen",
|
|
33
|
+
"gov",
|
|
34
|
+
"lt",
|
|
35
|
+
"col",
|
|
36
|
+
"capt",
|
|
37
|
+
"sgt",
|
|
38
|
+
"cpl",
|
|
39
|
+
"maj",
|
|
40
|
+
"adm",
|
|
41
|
+
"pres",
|
|
42
|
+
"atty",
|
|
43
|
+
"bros",
|
|
44
|
+
"mfg",
|
|
45
|
+
"assoc",
|
|
46
|
+
"univ",
|
|
47
|
+
"bldg",
|
|
48
|
+
"ste",
|
|
49
|
+
"ave",
|
|
50
|
+
"blvd",
|
|
51
|
+
"rd",
|
|
52
|
+
"ln",
|
|
53
|
+
"hwy",
|
|
54
|
+
"pkwy",
|
|
55
|
+
"pl"
|
|
56
|
+
]);
|
|
57
|
+
var CAPITALISED_ABBREVIATIONS = /* @__PURE__ */ new Set([
|
|
58
|
+
"Mon",
|
|
59
|
+
"Tue",
|
|
60
|
+
"Tues",
|
|
61
|
+
"Wed",
|
|
62
|
+
"Thu",
|
|
63
|
+
"Thur",
|
|
64
|
+
"Thurs",
|
|
65
|
+
"Fri",
|
|
66
|
+
"Sat",
|
|
67
|
+
"Sun",
|
|
68
|
+
"Mt",
|
|
69
|
+
"Ft",
|
|
70
|
+
"Ct",
|
|
71
|
+
"Rep",
|
|
72
|
+
"Rev",
|
|
73
|
+
"Hon"
|
|
74
|
+
]);
|
|
75
|
+
var BEFORE_NUMBER = /* @__PURE__ */ new Set([
|
|
76
|
+
"no",
|
|
77
|
+
"nos",
|
|
78
|
+
"fig",
|
|
79
|
+
"figs",
|
|
80
|
+
"vol",
|
|
81
|
+
"vols",
|
|
82
|
+
"ch",
|
|
83
|
+
"sec",
|
|
84
|
+
"pp",
|
|
85
|
+
"cf",
|
|
86
|
+
"ca",
|
|
87
|
+
"est",
|
|
88
|
+
"ext",
|
|
89
|
+
"tel",
|
|
90
|
+
"ph",
|
|
91
|
+
"ste",
|
|
92
|
+
"apt",
|
|
93
|
+
"bldg",
|
|
94
|
+
"rm",
|
|
95
|
+
"hwy",
|
|
96
|
+
"rt",
|
|
97
|
+
"rte"
|
|
98
|
+
]);
|
|
99
|
+
var DOTTED_SHORT = /^\d*(?:[A-Za-z]{1,2}\.)+[A-Za-z]{1,2}$/;
|
|
100
|
+
var DOTTED_CAPS = /^(?:[A-Z][A-Za-z]*\.)+[A-Z][A-Za-z]*$/;
|
|
101
|
+
var ABBREVIATION_RANGE = /[A-Za-z]\.[-–—/]/;
|
|
102
|
+
var LIST_MARKER_ALONE = /^\s*(?:(?:step|tip|reason|rule|point|item|option)\s+)?\d{1,3}\.$/i;
|
|
103
|
+
var LIST_MARKER_AFTER_COLON = /:\s+\d{1,3}\.$/;
|
|
104
|
+
var LIST_MARKER_REACH = 24;
|
|
105
|
+
var OPENERS = /* @__PURE__ */ new Set(['"', "'", "\u201C", "\u2018", "(", "["]);
|
|
106
|
+
var CLOSERS = /* @__PURE__ */ new Set(['"', "'", "\u201D", "\u2019", ")", "]"]);
|
|
107
|
+
var SPACE = /\s/;
|
|
108
|
+
var LETTER = /\p{L}/u;
|
|
109
|
+
var DIGIT = /[0-9]/;
|
|
110
|
+
var SENTENCE_START = /[A-Z0-9]/;
|
|
111
|
+
var isStop = (c) => c === "." || c === "!" || c === "?";
|
|
112
|
+
var isSpace = (c) => SPACE.test(c);
|
|
113
|
+
function startsSentence(source, at) {
|
|
114
|
+
const c = OPENERS.has(source[at]) ? source[at + 1] : source[at];
|
|
115
|
+
return c !== void 0 && SENTENCE_START.test(c);
|
|
116
|
+
}
|
|
117
|
+
function digitFollows(source, at) {
|
|
118
|
+
const c = OPENERS.has(source[at]) ? source[at + 1] : source[at];
|
|
119
|
+
return c !== void 0 && DIGIT.test(c);
|
|
120
|
+
}
|
|
121
|
+
function isDottedAbbreviation(bare) {
|
|
122
|
+
const compound = Math.max(bare.lastIndexOf("-"), bare.lastIndexOf("\u2013"), bare.lastIndexOf("\u2014"), bare.lastIndexOf("/"));
|
|
123
|
+
const last = bare.slice(compound + 1);
|
|
124
|
+
return DOTTED_SHORT.test(last) || DOTTED_CAPS.test(last) || ABBREVIATION_RANGE.test(bare);
|
|
125
|
+
}
|
|
126
|
+
function endsWithAbbreviation(source, end, digitNext) {
|
|
127
|
+
let from = end;
|
|
128
|
+
while (from > 0 && !isSpace(source[from - 1])) from--;
|
|
129
|
+
let to = end;
|
|
130
|
+
while (from < to && OPENERS.has(source[from])) from++;
|
|
131
|
+
while (to > from && CLOSERS.has(source[to - 1])) to--;
|
|
132
|
+
while (to > from && source[to - 1] === ".") to--;
|
|
133
|
+
const bare = source.slice(from, to);
|
|
134
|
+
const lower = bare.toLowerCase();
|
|
135
|
+
return isDottedAbbreviation(bare) || /^[A-Za-z]$/.test(bare) || ABBREVIATIONS.has(lower) || CAPITALISED_ABBREVIATIONS.has(bare) || digitNext && BEFORE_NUMBER.has(lower);
|
|
136
|
+
}
|
|
137
|
+
function endsWithListMarker(source, start, end) {
|
|
138
|
+
const from = Math.max(start, end - LIST_MARKER_REACH);
|
|
139
|
+
const tail = source.slice(from, end);
|
|
140
|
+
return from === start && LIST_MARKER_ALONE.test(tail) || LIST_MARKER_AFTER_COLON.test(tail);
|
|
141
|
+
}
|
|
142
|
+
function splitSentences(text) {
|
|
143
|
+
const source = (text ?? "").trim();
|
|
144
|
+
if (!source) return [];
|
|
145
|
+
const sentences = [];
|
|
146
|
+
let start = 0;
|
|
147
|
+
let scanned = 0;
|
|
148
|
+
let sawLetter = false;
|
|
149
|
+
let i = 0;
|
|
150
|
+
while (i < source.length) {
|
|
151
|
+
if (!isStop(source[i])) {
|
|
152
|
+
i++;
|
|
153
|
+
continue;
|
|
154
|
+
}
|
|
155
|
+
let endsStrong = false;
|
|
156
|
+
while (i < source.length && isStop(source[i])) {
|
|
157
|
+
if (source[i] !== ".") endsStrong = true;
|
|
158
|
+
i++;
|
|
159
|
+
}
|
|
160
|
+
while (i < source.length && CLOSERS.has(source[i])) i++;
|
|
161
|
+
let next = i;
|
|
162
|
+
while (next < source.length && isSpace(source[next])) next++;
|
|
163
|
+
if (next === i || !startsSentence(source, next)) continue;
|
|
164
|
+
if (!sawLetter) sawLetter = LETTER.test(source.slice(scanned, i));
|
|
165
|
+
scanned = i;
|
|
166
|
+
if (!sawLetter) continue;
|
|
167
|
+
if (!endsStrong && (endsWithAbbreviation(source, i, digitFollows(source, next)) || endsWithListMarker(source, start, i))) continue;
|
|
168
|
+
sentences.push(source.slice(start, i).trim());
|
|
169
|
+
start = i;
|
|
170
|
+
sawLetter = false;
|
|
171
|
+
}
|
|
172
|
+
const rest = source.slice(start).trim();
|
|
173
|
+
if (rest) sentences.push(rest);
|
|
174
|
+
return sentences;
|
|
175
|
+
}
|
|
176
|
+
|
|
177
|
+
export { splitSentences };
|
package/dist/site-cache.d.ts
CHANGED
|
@@ -27,6 +27,8 @@ export declare const SITE_CACHE_TAGS: {
|
|
|
27
27
|
readonly taxonomy: 'editorial-taxonomy';
|
|
28
28
|
/** Portfolio items (agency-site-kit). */
|
|
29
29
|
readonly portfolio: 'portfolio';
|
|
30
|
+
/** Products, services, events and membership plans (commerce-kit). */
|
|
31
|
+
readonly commerce: 'commerce';
|
|
30
32
|
};
|
|
31
33
|
export type SiteCacheTag = (typeof SITE_CACHE_TAGS)[keyof typeof SITE_CACHE_TAGS];
|
|
32
34
|
/** The body Sonor POSTs to the revalidation route. */
|
package/dist/site-cache.js
CHANGED
|
@@ -12,7 +12,9 @@ var SITE_CACHE_TAGS = {
|
|
|
12
12
|
/** Article categories and tags. */
|
|
13
13
|
taxonomy: "editorial-taxonomy",
|
|
14
14
|
/** Portfolio items (agency-site-kit). */
|
|
15
|
-
portfolio: "portfolio"
|
|
15
|
+
portfolio: "portfolio",
|
|
16
|
+
/** Products, services, events and membership plans (commerce-kit). */
|
|
17
|
+
commerce: "commerce"
|
|
16
18
|
};
|
|
17
19
|
function isSiteRevalidatePong(body) {
|
|
18
20
|
return typeof body === "object" && body !== null && body.ok === true && body.ping === true;
|
package/dist/site-cache.mjs
CHANGED
|
@@ -10,7 +10,9 @@ var SITE_CACHE_TAGS = {
|
|
|
10
10
|
/** Article categories and tags. */
|
|
11
11
|
taxonomy: "editorial-taxonomy",
|
|
12
12
|
/** Portfolio items (agency-site-kit). */
|
|
13
|
-
portfolio: "portfolio"
|
|
13
|
+
portfolio: "portfolio",
|
|
14
|
+
/** Products, services, events and membership plans (commerce-kit). */
|
|
15
|
+
commerce: "commerce"
|
|
14
16
|
};
|
|
15
17
|
function isSiteRevalidatePong(body) {
|
|
16
18
|
return typeof body === "object" && body !== null && body.ok === true && body.ping === true;
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@sonordev/contracts",
|
|
3
|
-
"version": "2.
|
|
4
|
-
"description": "The rules Sonor's sites, APIs and dashboard share: popup blocks and site design tokens, site hosts, SEO page identity, title quality, llms.txt sanitizers, the forms honeypot, fleet heartbeats, slot content and signing, site cache tags, portfolio metrics, the house voice's contraction rule, how a proposal's site plan counts its pages, what separates an error page from a page's content,
|
|
3
|
+
"version": "2.5.0",
|
|
4
|
+
"description": "The rules Sonor's sites, APIs and dashboard share: popup blocks and site design tokens, site hosts, SEO page identity, title quality, llms.txt sanitizers, the forms honeypot, fleet heartbeats, slot content and signing, site cache tags, portfolio metrics, the house voice's contraction rule, how a proposal's site plan counts its pages, what separates an error page from a page's content, which stored JSON-LD is a template's unfilled placeholder, and where prose splits into sentences. Pure functions, no dependencies.",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"repository": {
|
|
7
7
|
"type": "git",
|
|
@@ -100,6 +100,11 @@
|
|
|
100
100
|
"import": "./dist/schema-placeholders.mjs",
|
|
101
101
|
"require": "./dist/schema-placeholders.js"
|
|
102
102
|
},
|
|
103
|
+
"./sentences": {
|
|
104
|
+
"types": "./dist/sentences.d.ts",
|
|
105
|
+
"import": "./dist/sentences.mjs",
|
|
106
|
+
"require": "./dist/sentences.js"
|
|
107
|
+
},
|
|
103
108
|
"./package.json": "./package.json"
|
|
104
109
|
},
|
|
105
110
|
"typesVersions": {
|