@sonordev/contracts 2.2.0 → 2.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,16 @@
1
1
  # Changelog
2
2
 
3
+ ## 2.3.0
4
+
5
+ A minor release: one new entry, `@sonordev/contracts/error-page`. Every other entry is unchanged.
6
+
7
+ - **New entry `@sonordev/contracts/error-page`**: telling a rendered error page from a page's content, and copy that calls a working page missing from copy that describes it, so every writer and reader of page text answers the same way.
8
+ - `looksLikeErrorPage(text, wordCount?)` is true when stored text is an error page: a 404 in its common wordings, a framework or server error, a status page, a "something went wrong" or "temporarily unavailable" screen or a maintenance notice. It reads how the text opens, decodes entities, doesn't depend on word boundaries (scraped text runs elements together, as in "404This page could not be found."), and leaves phone numbers, addresses and pages that only describe an error alone.
9
+ - `usablePageText(text, wordCount?)` returns the text, or `null` for an error page.
10
+ - `findErrorPageClaim(copy, pageText?)` and `describesErrorPage(copy, pageText?)` find generated copy that says the page itself is missing or unavailable ("Page Not Found", "this page is currently unavailable", "returns a 404", "a page-not-found message"). A phrase the page's own text really says is allowed, and copy about something unavailable on a working page isn't a claim about the page.
11
+ - `findPageCopyClaim(fields, pageText?)` runs that check over every string in a set of fields, JSON-LD included, and returns the first `{ field, phrase }`.
12
+ - `ERROR_PAGE_MAX_WORDS` and `ERROR_PAGE_CONTRACT_VERSION` are exported alongside.
13
+
3
14
  ## 2.2.0
4
15
 
5
16
  A minor release: one new entry, `@sonordev/contracts/proposal-sitemap`. Every other entry is unchanged.
package/README.md CHANGED
@@ -24,6 +24,7 @@ pnpm add @sonordev/contracts
24
24
  | `@sonordev/contracts/portfolio` | Where a portfolio number comes from and which may headline; the live-frame message | the Sonor API, Signal, agency-site-kit |
25
25
  | `@sonordev/contracts/proposal-sitemap` | How a proposal's site plan counts its pages, when its before-and-after comparison may show, and whether the page counts stated elsewhere in the proposal add up | the Sonor API, Signal's proposal writer, the dashboard's proposal pages |
26
26
  | `@sonordev/contracts/voice` | The house voice's contraction rule. `findUncontracted` and `contractText` find and apply full forms that should be contractions; `findContracted` finds the contractions a question-form heading should expand | anything that checks or rewrites copy: Sonor's APIs and dashboard |
27
+ | `@sonordev/contracts/error-page` | Whether stored text is a rendered error page rather than a page's content, and whether generated copy says the page itself is missing | anything that stores page text or writes page copy from it: the Sonor API's page-view and sitemap writers, Signal's metadata generators |
27
28
 
28
29
  ## The voice contract
29
30
 
@@ -185,6 +186,72 @@ pages") isn't read, and a count of pages that aren't in the plan (say, a
185
186
  competitor's) is reported like any other; the issues are for a person to look
186
187
  at, not edits to apply.
187
188
 
189
+ ## The error-page contract
190
+
191
+ A page's stored text is whatever a browser or crawler rendered. Visit a URL
192
+ while it's briefly broken and the error page's words become that page's
193
+ content, and any title, description or summary written from that content
194
+ describes a working page as broken. `error-page` is how writers refuse both
195
+ halves of that.
196
+
197
+ - **`looksLikeErrorPage(text, wordCount?)`** is true when the text is a
198
+ rendered error page: a 404 in its common wordings ("404", "This page could
199
+ not be found", "Page not found"), a framework error ("a client-side
200
+ exception has occurred", "This page couldn't load", "Internal Server
201
+ Error", "Cannot GET /x"), a status page ("502 Bad Gateway", "HTTP Status
202
+ 404"), a "something went wrong" or "temporarily unavailable" screen, or a
203
+ maintenance notice. It reads how the text *opens* (the first 300
204
+ characters), decodes HTML entities, and doesn't depend on word boundaries,
205
+ because scraped text runs elements together ("404This page could not be
206
+ found."). Text up to 120 words is checked for every wording, though the
207
+ loosest ones (a 404 glued to its copy, "refresh the page and try again", a
208
+ route's "<name> not found") only read the first 60 or 40 words; up to 400
209
+ words only the check for how it opens after a meta description and skip link
210
+ applies; beyond that it's prose. A phone number, room number or street
211
+ address that starts with 404 isn't an error page, and neither is a help page
212
+ that only *describes* an error ("how to fix a 404 error", "if something went
213
+ wrong"). Pass the word count of the text itself, never a whole page's count
214
+ (a browser's includes the navigation and footer and can hide a short error
215
+ state): a supplied count can only raise the count the check uses.
216
+ - **`usablePageText(text, wordCount?)`** returns the text, or `null` when it's
217
+ an error page. It's what a model should be shown as "the page's content".
218
+ - **`findErrorPageClaim(copy, pageText?)`** returns the phrase in generated
219
+ copy that says the page itself is missing or unavailable ("Page Not Found",
220
+ "the pricing page is currently unavailable", "returns a 404", "a
221
+ page-not-found message"), or `null`. Copy about something unavailable on a
222
+ working page ("the patio is closed on Mondays", "tours are paused for
223
+ winter") isn't a claim about the page. Pass the page's own text and a phrase
224
+ the page really says is allowed, so a guide to fixing "page not found"
225
+ errors can be titled that way; the text of an error page is never accepted
226
+ as that exemption. `describesErrorPage` is the boolean form.
227
+ - **`findPageCopyClaim(fields, pageText?)`** runs `findErrorPageClaim` over
228
+ every string in a set of fields (a title, a description, keywords, a
229
+ JSON-LD object; any object or class instance) and returns the first
230
+ `{ field, phrase }`, or `null`. It's how a writer or a reader judges a whole
231
+ draft in one call. It reads at most 2,000 strings, 8 levels deep and 100,000
232
+ characters in all, and each string's first 5,000 characters, so a hostile
233
+ record can't make it slow.
234
+
235
+ ```ts
236
+ import { describesErrorPage, findPageCopyClaim, usablePageText } from '@sonordev/contracts/error-page'
237
+
238
+ usablePageText('404This page could not be found.') // null: don't generate from it
239
+ usablePageText('Weekly mowing for property managers.') // the text
240
+
241
+ describesErrorPage('Commercial Mowing Page Not Found | Example Co') // true: don't ship it
242
+ describesErrorPage('Seasonal tours are paused until April.') // false
243
+
244
+ findPageCopyClaim({
245
+ title: 'Commercial Mowing | Example Co',
246
+ schema: { '@type': 'WebPage', description: 'This URL currently returns a 404.' },
247
+ })
248
+ // { field: 'schema', phrase: 'returns a 404' }
249
+ ```
250
+
251
+ A text-only check can't prove a page failed. Where you observed the HTTP
252
+ status, require 400 or more as well and use this as the fallback; the header of
253
+ `src/contracts/error-page.ts` lists the other known limits.
254
+
188
255
  ## Why it exists
189
256
 
190
257
  A rule that lives in several copies is how two of them end up disagreeing.
@@ -0,0 +1,183 @@
1
+ /**
2
+ * @sonordev/contracts/error-page: telling an error page from a page (v1)
3
+ *
4
+ * Three questions every writer of page content and page copy has to answer the
5
+ * same way, so they live here once:
6
+ *
7
+ * 1. Is this stored text a rendered ERROR PAGE rather than the page's own
8
+ * content? (`looksLikeErrorPage`, `usablePageText`)
9
+ * 2. Does this generated COPY say the page itself is missing or unavailable?
10
+ * (`findErrorPageClaim`, `describesErrorPage`)
11
+ * 3. Does any string in a page's managed copy (a title, a keyword list, a
12
+ * JSON-LD block) say so? (`findPageCopyClaim`: question 2 over every
13
+ * string of a record, naming the field that holds it)
14
+ *
15
+ * Why it exists: a page's stored text is the text a visitor's browser (or a
16
+ * build's crawler) rendered. Visit a URL while it's briefly broken and the
17
+ * error page's words are stored as that page's content. Everything generated
18
+ * from that content then describes a working page as broken, and the result
19
+ * is published as the page's title, meta description, social tags, JSON-LD
20
+ * and AI-crawler notes. The outage ends in a deploy; the poisoned copy
21
+ * doesn't, because nothing downstream asks whether its input was ever the
22
+ * page.
23
+ *
24
+ * How the text check works
25
+ *
26
+ * - It reads the START of the text (300 characters), lowercased, with
27
+ * whitespace collapsed, HTML entities decoded and curly quotes made
28
+ * straight. Error copy leads; articles bury it.
29
+ * - The signatures are generic on purpose. Every site words its 404
30
+ * differently and frameworks ship their own ("This page could not be
31
+ * found.", "Application error: a client-side exception has occurred"), so
32
+ * the shape is what's matched, not any one site's copy. They're grouped by
33
+ * family below: not found, server and client errors, unavailable and
34
+ * maintenance, walls.
35
+ * - Stored text is often the page's elements run together with no separator
36
+ * ("404This page could not be found."), so no signature depends on a word
37
+ * boundary around the text it's matching.
38
+ * - A signature that follows a word that DESCRIBES errors rather than
39
+ * showing one ("if something went wrong", "how to fix a 404 error", a
40
+ * quote mark) doesn't count: a help page that names an error isn't one.
41
+ * Navigation is Title Case and prose isn't, so a describing word counts
42
+ * only as it's written in lower case, with only its first letter
43
+ * capitalized ("How to fix..."), or opening the text or a sentence;
44
+ * "Home Blog How To Something went wrong" is still an error page, and the
45
+ * words a menu uses too ("About", "Monitoring", "Alerts for") describe an
46
+ * error only in lower case. A describing word inside the matched phrase
47
+ * counts too, and after a described match the search resumes one character
48
+ * on, so a real error later in the same stretch is still found.
49
+ * - A not-found phrase that sits in a question ("Page not found? Use our site
50
+ * map", "Product not found in your size? Ask us") is a help line, and so is
51
+ * most copy that merely uses the words ("Rare parts can't be found
52
+ * elsewhere": the subject has to be a page, URL, link, file, site or app).
53
+ * - Up to ERROR_PAGE_MAX_WORDS words every check applies, but the loosest
54
+ * ones apply only to the shortest texts: a 404 glued to its copy and
55
+ * "refresh the page and try again" up to 60 words, a dynamic route's own
56
+ * "<name> not found" state up to 40. From there up to 400 words only the
57
+ * prefix-aware check applies: a text written as "<meta description> —
58
+ * <every string on the page>" carries its header and footer strings along,
59
+ * which pushes a rendered error state past the short-text ceiling, so the
60
+ * check looks at how the text opens after the description and a skip link,
61
+ * and an opener followed by "explained" or "how to" is an article. Beyond
62
+ * 400 words the text is prose.
63
+ *
64
+ * `wordCount` semantics: pass the word count of `text` itself. Pass the whole
65
+ * document's count only when `text` is a truncated excerpt of it. A supplied
66
+ * count can only RAISE the count the check uses (an excerpt of a long document
67
+ * is prose however short it is), never lower it: text that is longer than the
68
+ * count you gave is counted by what it holds. A browser's page-wide count
69
+ * (navigation and footer included) is therefore the wrong number for a short
70
+ * error state it scraped: pass the count of the text you pass.
71
+ *
72
+ * How the copy check works
73
+ *
74
+ * `findErrorPageClaim` looks for the copy saying THE PAGE is missing or
75
+ * unavailable ("Page Not Found | Acme", "This URL currently returns a 404",
76
+ * "This listing has been removed", "The page failed to load"). The subject
77
+ * has to be the page, its URL, link or listing, so copy about something
78
+ * unavailable on a working page ("sunset cruises are not available at this
79
+ * time") is left alone, and so is a qualified one ("unavailable in winter").
80
+ * Whitespace is collapsed and each string is read up to 5,000 characters, so
81
+ * hostile whitespace costs nothing. The page's own text exempts a phrase the
82
+ * page really says; a page that is itself an error page exempts nothing.
83
+ *
84
+ * `findPageCopyClaim(fields, pageText?)` is the same check over a record:
85
+ * each value of `fields` is checked if it's a string, and walked for every
86
+ * string it holds if it's an array or an object (8 levels deep, 2,000
87
+ * leaves at most; numbers, booleans and null are ignored). It returns the
88
+ * first claim as `{ field, phrase }`, `field` being the TOP-LEVEL key (the
89
+ * column or property that holds the copy), or null. Use it for a page row's
90
+ * managed title, description, keywords and JSON-LD in one call.
91
+ *
92
+ * Known limits (a text-only check can't close these)
93
+ *
94
+ * - Any error phrase in the first 300 characters of a text of 120 words or
95
+ * fewer reads as an error page, wherever in those 300 characters it sits, unless
96
+ * something describes it (see above): a short page ABOUT errors ("Page not
97
+ * found: how to design a helpful 404 page. Five examples.", or a web
98
+ * agency's line about its "custom 404 error page") can be misread. Texts of
99
+ * 121 to 400 words are only read for how they open, and longer ones are
100
+ * prose. The loosest wordings (a 404 glued to its copy, "refresh the page
101
+ * and try again", "<name> not found") apply only up to 60, 60 and 40 words.
102
+ * - A business whose name is an error phrase ("404 Brewing Co.", "Page Not
103
+ * Found Records") reads as an error page when its name leads the text.
104
+ * - Chrome in front hides the error state from the wordings anchored to the
105
+ * start of the text ("Not Found", "Be right back.", "Error: ...", "Oops!
106
+ * This page...", a permission wall, a dynamic route's "<name> not found").
107
+ * The wordings that hold anywhere in the first 300 characters (a status with
108
+ * its reason, "could not be found", "something went wrong") are what's
109
+ * left. Prefer the page's main content.
110
+ * - The text alone never proves a page failed. Where a call site observed
111
+ * the HTTP status (a browser reports it), require a status of 400 or more
112
+ * as well, and treat this check as the fallback for the sites and the
113
+ * paths that report none.
114
+ * - It reads English. A 404 in another language isn't matched.
115
+ * - Walls and pages served by the host in front of the site (a CDN block, a
116
+ * bot challenge, a parked domain, a default server page, a browser
117
+ * interstitial) never reach the writers this guards, so they aren't
118
+ * matched here.
119
+ * - Copy that claims the page is missing in words outside the closed list
120
+ * below isn't caught; the real protection is that error text never reaches
121
+ * the generator (`usablePageText`).
122
+ *
123
+ * Pure string logic: no I/O, no Node built-ins, no lookbehind (this ships to
124
+ * browsers, and Safari before 16.4 throws on a lookbehind at parse time).
125
+ */
126
+ /** Bump on breaking changes to what counts as an error page or an error claim. */
127
+ export declare const ERROR_PAGE_CONTRACT_VERSION: 1;
128
+ /**
129
+ * Above this many words, text is real prose even if it mentions a 404: only
130
+ * the prefix-aware check (up to 400 words, see the header) still looks at it.
131
+ *
132
+ * The gate is what makes the check safe: an article about broken links ("How
133
+ * to fix 404 errors on your site") legitimately holds every phrase below and
134
+ * must not be discarded. A rendered error page is short, and this ceiling
135
+ * still separates the two by an order of magnitude.
136
+ */
137
+ export declare const ERROR_PAGE_MAX_WORDS = 120;
138
+ /**
139
+ * True when this text is a rendered error page rather than page content.
140
+ *
141
+ * @param text the scraped or extracted body text
142
+ * @param wordCount the word count of `text` itself, or the whole document's
143
+ * when `text` is a truncated excerpt of it. It can only raise
144
+ * the count the check uses, never lower it; derived from
145
+ * `text` when absent.
146
+ */
147
+ export declare function looksLikeErrorPage(text?: string | null, wordCount?: number | null): boolean;
148
+ /**
149
+ * The page's stored text, or null when it's an error page. What a model may
150
+ * be shown as "the page's content".
151
+ */
152
+ export declare function usablePageText(text?: string | null, wordCount?: number | null): string | null;
153
+ /**
154
+ * The phrase in `copy` that says the page itself is missing or unavailable, or
155
+ * null when the copy is fine.
156
+ *
157
+ * @param pageText the page's own text. A phrase the page really says (an
158
+ * article about fixing "page not found" errors) is allowed;
159
+ * only copy that claims what the page doesn't is a finding.
160
+ * A pageText that is itself an error page allows nothing: its
161
+ * words are the poison, so the copy that repeats them would
162
+ * otherwise vouch for itself.
163
+ */
164
+ export declare function findErrorPageClaim(copy?: string | null, pageText?: string | null): string | null;
165
+ /** True when `copy` says the page itself is missing or unavailable (see `findErrorPageClaim`). */
166
+ export declare function describesErrorPage(copy?: string | null, pageText?: string | null): boolean;
167
+ /**
168
+ * The first string anywhere in a record of managed copy that says the page
169
+ * itself is missing or unavailable, or null. `findErrorPageClaim` over every
170
+ * string of `fields`, and the record's own field name with the phrase.
171
+ *
172
+ * Each value is checked if it's a string, and walked for the strings it holds
173
+ * if it's an array or an object (a JSON-LD block, a keyword list), up to 8
174
+ * levels deep, 2,000 strings and 100,000 characters in all; numbers, booleans and
175
+ * null are ignored (and don't count toward those limits).
176
+ * `field` is the TOP-LEVEL key, the column or property that holds the copy.
177
+ * `pageText` exempts what the page really says, as in `findErrorPageClaim`; the
178
+ * page's own text is read once for the whole record.
179
+ */
180
+ export declare function findPageCopyClaim(fields: object, pageText?: string | null): {
181
+ field: string;
182
+ phrase: string;
183
+ } | null;
@@ -0,0 +1,424 @@
1
+ 'use strict';
2
+
3
+ // src/contracts/error-page.ts
4
+ var ERROR_PAGE_CONTRACT_VERSION = 1;
5
+ var ERROR_PAGE_MAX_WORDS = 120;
6
+ var PREFIXED_MAX_WORDS = 400;
7
+ var SHORT_ONLY_MAX_WORDS = 60;
8
+ var START_STATE_MAX_WORDS = 40;
9
+ var HEAD_CHARS = 300;
10
+ var DESCRIBE_WINDOW = 45;
11
+ var NAMED_ENTITIES = {
12
+ amp: "&",
13
+ quot: '"',
14
+ apos: "'",
15
+ nbsp: " ",
16
+ lt: "<",
17
+ gt: ">",
18
+ rsquo: "'",
19
+ lsquo: "'",
20
+ ldquo: '"',
21
+ ndash: "\u2013",
22
+ mdash: "\u2014",
23
+ hellip: "\u2026"
24
+ };
25
+ function decodeEntities(text) {
26
+ let out = text;
27
+ for (let pass = 0; pass < 2 && out.indexOf("&") !== -1; pass++) {
28
+ out = out.replace(/&(?:#x([0-9a-f]{1,6})|#(\d{1,7})|([a-z]{2,8}));/gi, (whole, hex, dec, name) => {
29
+ if (name) {
30
+ const key = name.toLowerCase();
31
+ return Object.prototype.hasOwnProperty.call(NAMED_ENTITIES, key) ? NAMED_ENTITIES[key] : whole;
32
+ }
33
+ const code = hex ? parseInt(hex, 16) : parseInt(dec, 10);
34
+ return code <= 1114111 ? String.fromCodePoint(code) : whole;
35
+ });
36
+ }
37
+ return out;
38
+ }
39
+ function tidy(text) {
40
+ return decodeEntities(text).replace(/[‘’‛ʼ]/g, "'").replace(/\s+/g, " ").trim();
41
+ }
42
+ function normalize(text) {
43
+ return tidy(text).toLowerCase();
44
+ }
45
+ function textOf(text) {
46
+ const oc = tidy(text);
47
+ const lc = oc.toLowerCase();
48
+ return { lc, oc: lc.length === oc.length ? oc : lc };
49
+ }
50
+ function slice(text, from, to) {
51
+ return { lc: text.lc.slice(from, to), oc: text.oc.slice(from, to) };
52
+ }
53
+ function words(text) {
54
+ return text.split(/\s+/).filter(Boolean).length;
55
+ }
56
+ function wordsAcrossElements(text) {
57
+ return words(text.replace(/([a-z0-9.,;:!?)"'’])([A-Z])/g, "$1 $2"));
58
+ }
59
+ var NOT_A_QUESTION = String.raw`(?!\s*\?|\s+(?:in|on|for|at|with|under|from|during|to)\b[^.!?]{0,50}\?)`;
60
+ var NOT_IN_YOUR_SIZE = String.raw`(?!\s+in\s+your\s+size\b)`;
61
+ var RECORD_NOUN = String.raw`(?:page|project|post|article|product|listing|resource|file|deployment)`;
62
+ var CANT_BE_FOUND = String.raw`\b(?:page|url|link|resource|route|file|deployment|site|website|app|project)\b[^.!?]{0,40}?(?:could not|couldn't|can't|cannot|can not) be found`;
63
+ var WHICH_PAGE = String.raw`(?:\s+(?:you(?:'re| are| were)? (?:looking for|requested|asked for|(?:tried|were trying|are trying) to (?:reach|visit|access|find|open))|that you (?:requested|asked for)|at\s+\S[^.!?]{0,40}|on this (?:site|server|website)))?`;
64
+ var DOESNT_EXIST = String.raw`(?:\b(?:this|that|the|requested)\s+(?:[a-z0-9'-]+\s+){0,2}(?:page|url|link|post|article|listing|product|resource|project|profile|record|entry|story|event|course|job|item|offer|video)|\bpage)${WHICH_PAGE}\s+(?:doesn't|does not)(?: (?:seem|appear))?(?: to)? exist(?!\s+(?:to|for|so)\s)`;
65
+ var MAINTENANCE_QUALIFIER = String.raw`(?!\s+(?:on|from|every|each|this|next|between|during|at|after|before|once|twice|monthly|weekly|nightly|daily|contract|agreement|plan|program|warranty|coverage|with\s+(?:us|our)|for\s+(?:the|a|\d)|until(?!\s+(?:\d{1,2}(?::\d\d)?\s*(?:[ap]\.?m\b|o'clock)|\d{1,2}:\d\d|noon|midnight|tomorrow|further notice)))\b)`;
66
+ var ERROR_PAGE_SIGNATURES = [
67
+ // ---- Not found: the framework defaults and the common custom wordings ----
68
+ new RegExp(RECORD_NOUN + " (?:was |is )?not found" + NOT_A_QUESTION + NOT_IN_YOUR_SIZE),
69
+ // "Can't find the page you need? Use the search box" is a help line; "We can't find that page." isn't.
70
+ /(?:can't|cannot|can not|couldn't|could not|unable to) (?:be )?(?:find|found|locate)(?: that| this| the)? page(?!\s*\?|\s+(?:you|that|which)\b[^.!?]{0,40}\?)/,
71
+ // "We can't seem to find the page", "It seems we can't find what you're looking for".
72
+ /(?:we|i) (?:can't|cannot|couldn't|could not|were unable to|are unable to|weren't able to)(?: seem to)? (?:find|locate) (?:that page|this page|the page|that one|that(?=[.!,?]| link)|what you(?:'re| are| were)? looking for)/,
73
+ // "can not" is two words in the Vercel default.
74
+ new RegExp(CANT_BE_FOUND),
75
+ // Next's own not-found.tsx example: "Not FoundCould not find requested resourceReturn Home".
76
+ /(?:could not|couldn't) find (?:the )?requested resource/,
77
+ // "doesn't exist" only with a page, link or record as the subject: a short
78
+ // marketing page can say "if a tool doesn't exist, we'll build it".
79
+ new RegExp(DOESNT_EXIST),
80
+ /(?:doesn't|does not) exist or (?:has|have) been (?:moved|removed|deleted)/,
81
+ /looking for (?:doesn't|does not) exist/,
82
+ // A page-ish subject, like "doesn't exist" above: a history page can say "the 1890 mill no longer exists".
83
+ new RegExp(String.raw`\b(?:this|that|the)\s+(?:[a-z0-9'-]+\s+){0,2}(?:page|url|link|project|post|article|product|listing|resource|content|item|offer|event|record|story|profile|entry|document|course|job|video)` + WHICH_PAGE + String.raw`\s+no longer exists`),
84
+ /(?:404: ?not[ _]found|code: ?not_found)/,
85
+ // "Error 404" and "404 Error" as a label ("404 Error | Acme") or leading into what a 404 says
86
+ // ("Error 404Page not found"). Not "a custom 404 error page" and not a name ("Error 404 Studio").
87
+ /(?:^|[.!?|:]\s*)(?:http |server )?(?:error 404|404 error)(?=\s*[-–—|:.!?,)]|\s*(?:page|this|that|the|we|oops|whoops|sorry|looks|uh|hmm|nothing|there|it|oh|please)\b)|(?:error 404|404 error)(?=\s*$|\s*[-–—|:]?\s*(?:not found|this (?:page|url)|that page)\b)/,
88
+ // "The page you requested was not found", "The page you were looking for has been removed".
89
+ /page you (?:requested|(?:were|are) looking for|tried to (?:reach|visit|access))[^.!?]{0,60}(?:not found|no longer|(?:has|have|had|might have|may have|may) been|has (?:drifted|wandered|vanished|disappeared|moved|left)|isn't|is not|doesn't|does not|could not|can't|cannot|was not)/,
90
+ // Apache, Flask and Django: "The requested URL was not found on this server."
91
+ /requested (?:url|uri|resource|file|page|document)\b[^.!?]{0,80}(?:not found|not be found|cannot be found)|(?:was|is) not found on (?:this|the) server/,
92
+ // Netlify, Render, Flask and Express answer with the bare words "Not Found".
93
+ /^not found(?=$|\s*[-–—|:.!]|the requested|request id)/,
94
+ // WordPress core and its default themes: "It looks like nothing was found at this location.", "Nothing Found".
95
+ /nothing was found at this location|^nothing found(?=$|[.,:;!?|-]|sorry|it looks|we couldn|nothing|oops|the page)/,
96
+ // Custom 404 copy that never says "404": "Oops! This page took a day off.", "You've hit a dead end.",
97
+ // "This page doesn't live here anymore." The opener needs the page to be the one that's gone ("Oops! This
98
+ // page is a work in progress" is a holding page), and the dead end needs to be a sentence of its own.
99
+ /^(?:oops|whoops|uh[- ]oh)[!,.…]*\s*(?:this|that) page (?:took|went|wandered|got lost|ran away|has (?:gone|wandered|moved|left|vanished|disappeared|been (?:removed|moved|deleted))|is (?:missing|gone|lost)|isn't (?:here|available|there)|(?:doesn't|does not|can't|cannot|couldn't) (?:exist|be found|seem)|seems? (?:to be )?(?:missing|lost|gone))/,
100
+ /(?:^|[.!?]\s*)(?:you've|you have|you) (?:hit|reached) a dead end(?=[.!?…]|$)/,
101
+ /(?:page|link|url)[^.!?]{0,40}(?:doesn't|does not|no longer|isn't|is not) (?:live |exist |work )?here(?: anymore)?(?![a-z]*\s+to\b)/,
102
+ // Gatsby's 404 route: "NOT FOUNDYou just hit a route that doesn't exist... the sadness."
103
+ /hit a route that doesn't exist/,
104
+ // Tomcat and IIS: "HTTP Status 404 – Not Found", "HTTP Error 403.14 - Forbidden", "HTTP Error 503. The service is unavailable."
105
+ /http (?:error|status) [45]\d\d(?:\.\d+)?\s*[–—-]\s*(?:not found|forbidden|unauthori[sz]ed|bad request|internal|method not allowed|service unavailable)|http error [45]\d\d(?:\.\d+)?\. the /,
106
+ // A status number with its reason, anywhere in the head (a nav can precede it):
107
+ // "404 | Not Found", "502 - Bad Gateway", "403: Forbidden", "400Bad Request", "419 | Page Expired".
108
+ // Not "401 unauthorized means your key is missing": a status explained is documentation.
109
+ /(?:^|\D)(?:(?:404|410)\s*[|:–—-]?\s*(?:not found|page not found|file not found|web site not found)|408[\s:|–—-]*request time-?out|50[0-4][\s:|–—-]*(?:internal (?:server )?error|server error|bad gateway|service (?:temporarily )?unavailable|gateway time-?out)|40[13][\s:|–—-]*(?:forbidden|unauthori[sz]ed)|400[\s:|–—-]*bad request|405[\s:|–—-]*method not allowed|413[\s:|–—-]*(?:request entity|payload|content) too large|419[\s:|–—-]*page expired|429[\s:|–—-]*too many)(?!\s+(?:(?:requests?|errors?|responses?|status(?: codes?)?|codes?|messages?|pages?)\s+)?(?:means?|is|are|indicates?|occurs?|happens?|when|if|returns?|tells?|shows?|appears?|can|should|will|occurred)\b)/,
110
+ // Netlify's bare "Not Found - Request ID: ..." and Express's "Cannot GET /about", even with a nav in front.
111
+ /not found\s*[-–—]\s*request id|\bcannot (?:get|post|put|patch|delete) \/[\w\-./?=&%]*(?:$|\s)/,
112
+ // IIS's classic "Server Error 404 - File or directory not found." and its "had its name changed" sentence.
113
+ /file or directory not found|removed,? had its name changed/,
114
+ // This page has been removed / moved: its own sentence, not "removed from our menu".
115
+ /(?:^|[.!?]\s*)this page (?:has been |has |was |is )?(?:permanently )?(?:removed|deleted|moved)(?=[.!,;]|$|\s+(?:or|and)\b|\s*(?:return|go|back|click|please|visit|head)\b)/,
116
+ // ---- Server and client errors ----
117
+ // Not "Application error monitoring for Node teams".
118
+ /^application error(?!\s+(?:monitoring|tracking|reporting|handling|management|detection|analytics|logs?|alerts?|rates?|budgets?|codes?))/,
119
+ /(?:client|server)-side exception has occurred/,
120
+ // The hosts spell a status as a code too: "500: INTERNAL_SERVER_ERROR", "504: GATEWAY_TIMEOUT".
121
+ /internal[ _]server[ _]error/,
122
+ new RegExp(String.raw`(?:bad[ _]gateway|gateway[ _]time-?out)` + NOT_A_QUESTION),
123
+ // An exception's message rendered as the page: "Error: Failed to fetch data", "TypeError: x is undefined".
124
+ /^(?:(?:type|reference|syntax|range)error: |error: (?:failed|cannot|can't|unable|invalid|network|request|fetch|something|unexpected|no |not ))/,
125
+ // "Something went wrong with your insurance claim? ..." is a headline about a claim, not an error state.
126
+ new RegExp(String.raw`something(?: went|'s gone| has gone| had gone) wrong(?!\s+(?:with|during|on|in|at)\s+(?:your|my)\b)` + NOT_A_QUESTION),
127
+ /an (?:unexpected )?error (?:has )?occurred/,
128
+ /this (?:page|site|website) (?:isn't working|can't be reached)/,
129
+ // Next.js 16 error shells: "This page couldn't load. A server error occurred. Reload to try again."
130
+ // and the 403 / 401 pages ("This page could not be accessed.").
131
+ /this page couldn't load|a server error (?:has )?occurred|reload to try again(?=[.,!]|$)|this page could not be accessed/,
132
+ // React Router and Remix, Spring Boot, and the "unexpected error" boundaries.
133
+ /unexpected application error|whitelabel error page|(?:there was|encountered) an unexpected error/,
134
+ // Django and ASP.NET: "Server Error (500)", "Server Error in '/' Application", "Not Found (404)".
135
+ /server error (?:\(500\)|in '?\/)|(?:bad request|forbidden|not found) \((?:400|403|404)\)/,
136
+ // Next's runtime messages when a chunk or a hydration fails and the message is the page.
137
+ /chunkloaderror|loading chunk [\w/.-]+ failed|hydration failed because the|there was an error while hydrating/,
138
+ // Generated error.tsx boundaries: "Oops, we hit a snag.", "Something unexpected happened", "Unable to load page".
139
+ new RegExp(
140
+ String.raw`(?:^|[.!?]\s*)(?:(?:oops|whoops|uh[- ]oh)[!,.…]*\s*)?we hit a snag(?!\s+(?:with|in|on|during|when|while|at)\b)|(?:^|[.!?]\s*)something unexpected happened|(?:^|[.!?]\s*)(?:failed|unable) to load (?:the |this )?(?:page|content|data|resource)` + NOT_A_QUESTION + String.raw`|^error loading (?:the |this )?(?:page|content|data)`
141
+ ),
142
+ // "We're having trouble loading this page", "We couldn't load this page".
143
+ /(?:we're|we are|i'm) having (?:some )?(?:trouble|difficulty|difficulties|problems?) (?:loading|displaying|showing|retrieving) (?:this|that|the) (?:page|content)|we (?:couldn't|could not|can't|cannot) load (?:this|that|the) page/,
144
+ // WordPress core's fatal-error and database screens.
145
+ /there has been a critical error on this (?:website|site)|error establishing a database connection|(?:the|this) (?:site|website) is experiencing technical difficulties/,
146
+ // Apache's stock pages: 503, 400, 401, 408, 410 and 414 each in its own words.
147
+ /the server is temporarily unable to service your request|your browser sent a request that this server could not understand|this server could not verify that you are authorized|server timeout waiting for the http request|(?:was|is) no longer available on this server|request-uri too large/,
148
+ // ---- Unavailable and maintenance ----
149
+ // A qualifier ("in Hawaii", "until April") makes it content about a service, not a dead page.
150
+ /(?:this|the) (?:page|site|website|service) (?:is|are) (?:temporarily|currently) unavailable(?!\s+(?:in|on|for|at|during|to|until|from|outside|after|before|through)\b)/,
151
+ /^(?:service (?:temporarily )?unavailable|(?:temporarily|currently) unavailable(?!:\s))(?!\s+(?:in|on|for|at|during|to|until|from|outside)\b)/,
152
+ // The site or the page is the one under maintenance. Not "is under maintenance contract", not "down for
153
+ // maintenance on Fridays": a day, a time or a contract makes it content.
154
+ new RegExp(String.raw`(?:site|website|page|store|shop|app|portal|server|we're|we are)(?: is| are)?(?: currently| temporarily| now)*,? (?:under|down for) (?:scheduled |routine |some |emergency )?maintenance` + MAINTENANCE_QUALIFIER),
155
+ // "Our website is currently undergoing scheduled maintenance", "Briefly unavailable for scheduled maintenance",
156
+ // "We are offline for a short while". Not "Our rental fleet is undergoing routine maintenance": the subject is the site.
157
+ new RegExp(String.raw`(?:(?:site|website|page|store|shop|app|portal|server|we're|we are)(?: is| are)?(?: currently| temporarily| now)*|briefly|temporarily|currently|now) (?:undergoing|(?:offline|unavailable|down)(?: temporarily)? for) (?:scheduled |routine |some |emergency )?maintenance` + MAINTENANCE_QUALIFIER),
158
+ /(?:we're|we are|site is|website is) offline for an? (?:short|little|brief) (?:while|time|period)/,
159
+ /^be right back\.?$/,
160
+ // "We'll be back soon." ending its sentence, or leading into the next one. Not "We should be back soon with a replacement
161
+ // part", "back soon after our holiday break": what follows decides.
162
+ /(?:we'll|we will|we should|it should) (?:be )?(?:right )?back (?:online )?(?:soon|shortly)(?=[.!;]|$|,?\s*(?:our|the|this|thank|thanks|please|we)\b|\s+(?:once|after)\s+(?:a|the|our)\s+(?:(?:short|brief|quick|scheduled|planned)\s+)*(?:maintenance|upgrade|update|outage|downtime|migration|work)\b)/,
163
+ // An apology for the downtime makes what follows an outage notice, whatever it promises.
164
+ /sorry for the (?:downtime|inconvenience|interruption|outage)\W+(?:we'll|we will|we should|we are working)/,
165
+ // ---- Walls that stand in for the page ----
166
+ new RegExp(String.raw`(?:403 forbidden|access denied)(?!\s+(?:is|was|means|errors?|messages?|pages?)\b)` + NOT_A_QUESTION),
167
+ // A permission wall is addressed to the visitor and about THIS page or resource, and opens the text or follows its own
168
+ // heading ("Sorry, you are not allowed to access this page.", "ForbiddenYou are not authorized to access this resource.").
169
+ // Not a person or a file ("our team is not allowed to access this page of your file"), not a gallery ("you don't have
170
+ // permission to view the gallery"), and not a sentence further in ("Members only. You do not have permission to ...").
171
+ /(?:^|sorry|oops|error|forbidden|unauthori[sz]ed|access denied|40[13])\W*you(?:'re| are| aren't) (?:not |un)?(?:allowed|authori[sz]ed|permitted) to (?:access|view|see) (?:this|that) (?:page|resource|site|website|area|section|content)/,
172
+ /(?:^|sorry|oops|error|forbidden|unauthori[sz]ed|access denied|40[13])\W*you (?:don't|do not) have permission to (?:access|view) (?:this (?:page|resource|site|website|area|section|content)|\S+ on this server)/,
173
+ /^(?:forbidden|unauthori[sz]ed)(?=$|\s*[-–—|:.!]|you(?:'re| are)\b|please\b|this (?:page|resource))/,
174
+ /(?:checking your browser before accessing|enable javascript and cookies to continue(?!\s+\w)|verif(?:y|ying) you are (?:a )?human)/
175
+ ];
176
+ var LEADING_STATUS = /^(?:(?:http )?error:? ?)?(?:400|401|402|403|405|408|410|413|414|419|429|500|501|502|503|504)\W{0,3}(?:(?:service|server|temporarily|internal|gateway|bad|proxy|request|too|access|page|file|resource|connection|http|status|method|payload|entity|payment|authentication)\s+){0,3}(?:error|forbidden|unauthori[sz]ed|gone|unavailable|bad gateway|time-?out|timed out|not found|not allowed|too many|too large|bad request|page expired|payment required|not implemented)/;
177
+ var LEADING_404 = /^404(?![\d,])(?![\s.\-–—)]*\d)/;
178
+ var ADDRESS_AFTER_404 = /^404\s+(?:[a-z0-9.'-]+\s+){0,3}(?:street|st|avenue|ave|road|rd|boulevard|blvd|drive|dr|lane|ln|way|court|ct|highway|hwy|pike|parkway|pkwy|place|pl|suite|ste|unit|apt)\b/;
179
+ var SHORT_ONLY_SIGNATURES = [
180
+ // Opening the text, and not conditional: "Please refresh the page and try again." is the boundary's own line;
181
+ // "Clear your cache, then refresh the page and try again" is a tip, and "...try again if the calendar doesn't load" is one.
182
+ /^(?:please )?(?:refresh|reload)(?: the| this)? (?:page )?(?:and|or) try again(?!\s+(?:if|when|unless|whether|once|after|later|next|the next)\b)/
183
+ ];
184
+ var SHORT_START_SIGNATURES = [new RegExp(String.raw`^(?:[a-z0-9'&-]+\s+){0,2}[a-z0-9'&-]+ not found` + NOT_A_QUESTION + NOT_IN_YOUR_SIZE)];
185
+ var GLUED_404 = /(?:^|[\s|])404(?=[a-z])(?!th\b)/g;
186
+ var DELIMITED_404 = /(?:^|\s)404\s*[|:–—]\s*[a-z]/g;
187
+ var NUMBER_PREFIX = /(?:suite|ste|room|rm|unit|apt|apartment|route|rt|hwy|highway|building|bldg|floor|#)\.?\s*$/;
188
+ var ABOUT_THE_ERROR = String.raw`(?!\W*(?:errors?\b|explained|guide|what (?:it|this|that|is|are)|how to|how do|why|means|fix(?:es|ing)?\b|causes|vs\b|\?))`;
189
+ var PREFIX_SIGNATURES = [
190
+ new RegExp(String.raw`(?:page|resource|post|article|product|listing) (?:was |is )?not found`),
191
+ new RegExp(CANT_BE_FOUND),
192
+ /(?:we|i) (?:can't|cannot|couldn't|could not)(?: seem to)? (?:find|locate) (?:that page|this page|the page|what you(?:'re| are| were)? looking for)/,
193
+ new RegExp(DOESNT_EXIST),
194
+ /something went wrong|(?:client|server)-side exception|internal server error|this page couldn't load/
195
+ ].map((pattern) => new RegExp("(?:" + pattern.source + ")" + ABOUT_THE_ERROR));
196
+ var PREFIXED_LEADING_404 = new RegExp(LEADING_404.source + ABOUT_THE_ERROR);
197
+ var PREFIXED_LEADING_STATUS = new RegExp(LEADING_STATUS.source + ABOUT_THE_ERROR);
198
+ var PREFIX_SEPARATOR = " \u2014 ";
199
+ var PREFIX_MIN_CHARS = 60;
200
+ var PREFIX_MAX_CHARS = 400;
201
+ var SKIP_LINK = /^(?:skip to (?:main )?content|skip to main|skip navigation)\s*/;
202
+ var PREFIX_WINDOW_CHARS = 90;
203
+ var DESCRIBERS = String.raw`if|when|whether|unless|how to|how do|why|what is|what's|fix(?:ing|es)?|handle|handling|resolve|resolving|troubleshoot(?:ing)?|diagnos(?:e|ing)|watch(?:es|ing)? for|encounter(?:ing)?|history of`;
204
+ var LOWERCASE_DESCRIBERS = String.raw`about|monitor(?:ing)?|alert(?:s|ing)? (?:on|for)`;
205
+ var DESCRIBING_CONTEXT = new RegExp(
206
+ String.raw`(?:\b(?:(${DESCRIBERS})|(${LOWERCASE_DESCRIBERS}))(?:\s+(?:you|your|the|a|an|their|our|my|see|get|are|is|seeing|getting|receive|receiving|hit|to|every|any|each|all|\d{3})){0,3}\s*["'“]?|["“])$`
207
+ );
208
+ var INNER_DESCRIBER = /\b(?:if|when|whether|unless|how to)\b/;
209
+ var SENTENCE_OPENING = /[.!?:;|—–"“(]\s*$/;
210
+ var VISIBILITY_SUBJECT = /\b(?:you|your|we|our|they|their|customers?|clients?|people|visitors?|buyers?|patients?|business(?:es)?|brands?|compan(?:y|ies))\s+$/;
211
+ var FORM_FIELDS = /\b(?:name|e-?mail|phone|message|subscribe|submit)\b/;
212
+ function globalOf(patterns) {
213
+ return patterns.map((pattern) => new RegExp(pattern.source, "g"));
214
+ }
215
+ var SIGNATURES = globalOf(ERROR_PAGE_SIGNATURES);
216
+ var SHORT_ONLY_SIGS = globalOf(SHORT_ONLY_SIGNATURES);
217
+ var SHORT_START_SIGS = globalOf(SHORT_START_SIGNATURES);
218
+ var PREFIX_SIGS = globalOf(PREFIX_SIGNATURES);
219
+ var TITLE_VERB = /\b(?:fix|handle|resolve|troubleshoot|diagnose)\b/;
220
+ function describerIsProse(orig, start, describer, context) {
221
+ const word = orig.slice(start, start + describer.length);
222
+ if (word === describer || word === describer.charAt(0).toUpperCase() + describer.slice(1)) return true;
223
+ return start === 0 || SENTENCE_OPENING.test(orig.slice(Math.max(0, start - 3), start)) || TITLE_VERB.test(context);
224
+ }
225
+ function isDescribed(head, hit) {
226
+ const from = Math.max(0, hit.index - DESCRIBE_WINDOW);
227
+ const before = head.lc.slice(from, hit.index);
228
+ const context = DESCRIBING_CONTEXT.exec(before);
229
+ if (context) {
230
+ const [, describer, lowercaseDescriber] = context;
231
+ const at = from + context.index;
232
+ if (describer !== void 0) {
233
+ if (describerIsProse(head.oc, at, describer, context[0])) return true;
234
+ } else if (lowercaseDescriber !== void 0) {
235
+ if (head.oc.slice(at, at + lowercaseDescriber.length) === lowercaseDescriber) return true;
236
+ } else {
237
+ return true;
238
+ }
239
+ }
240
+ const inner = INNER_DESCRIBER.exec(hit[0]);
241
+ if (inner && head.oc.slice(hit.index + inner.index, hit.index + inner.index + inner[0].length) === inner[0]) return true;
242
+ return /be found$/.test(hit[0]) && VISIBILITY_SUBJECT.test(before) || /you are (?:a )?human$/.test(hit[0]) && FORM_FIELDS.test(head.lc);
243
+ }
244
+ function showsSignature(head, patterns, inFirstSentence = false) {
245
+ return patterns.some((pattern) => {
246
+ pattern.lastIndex = 0;
247
+ for (let hit = pattern.exec(head.lc); hit; hit = pattern.exec(head.lc)) {
248
+ if (inFirstSentence && /[.!?]/.test(head.lc.slice(0, hit.index))) return false;
249
+ if (!isDescribed(head, hit)) return true;
250
+ pattern.lastIndex = hit.index + 1;
251
+ }
252
+ return false;
253
+ });
254
+ }
255
+ function showsLoose404(head) {
256
+ for (const pattern of [GLUED_404, DELIMITED_404]) {
257
+ pattern.lastIndex = 0;
258
+ for (let hit = pattern.exec(head); hit; hit = pattern.exec(head)) {
259
+ if (!NUMBER_PREFIX.test(head.slice(Math.max(0, hit.index - 14), hit.index))) return true;
260
+ }
261
+ }
262
+ return false;
263
+ }
264
+ function afterPrefix(text) {
265
+ let start = 0;
266
+ const cut = text.lc.indexOf(PREFIX_SEPARATOR);
267
+ if (cut >= PREFIX_MIN_CHARS && cut < PREFIX_MAX_CHARS) start = cut + PREFIX_SEPARATOR.length;
268
+ const skip = SKIP_LINK.exec(text.lc.slice(start));
269
+ if (skip) start += skip[0].length;
270
+ return start === 0 ? text : slice(text, start);
271
+ }
272
+ function opensWith404(head) {
273
+ return LEADING_404.test(head) && !ADDRESS_AFTER_404.test(head);
274
+ }
275
+ function showsErrorState(head, count) {
276
+ return opensWith404(head.lc) || LEADING_STATUS.test(head.lc) || showsSignature(head, SIGNATURES) || count <= START_STATE_MAX_WORDS && showsSignature(head, SHORT_START_SIGS) || count <= SHORT_ONLY_MAX_WORDS && (showsSignature(head, SHORT_ONLY_SIGS) || showsLoose404(head.lc));
277
+ }
278
+ function looksLikeErrorPage(text, wordCount) {
279
+ if (!text || typeof text !== "string") return false;
280
+ const trimmed = text.trim();
281
+ if (!trimmed) return false;
282
+ const own = wordsAcrossElements(trimmed);
283
+ const count = typeof wordCount === "number" && wordCount >= 0 ? Math.max(wordCount, own) : own;
284
+ if (count > PREFIXED_MAX_WORDS) return false;
285
+ const whole = textOf(trimmed);
286
+ const rest = afterPrefix(whole);
287
+ if (own > ERROR_PAGE_MAX_WORDS) {
288
+ const lead = rest.lc.slice(0, HEAD_CHARS);
289
+ return PREFIXED_LEADING_404.test(lead) && !ADDRESS_AFTER_404.test(lead) || PREFIXED_LEADING_STATUS.test(lead) || showsSignature(slice(rest, 0, PREFIX_WINDOW_CHARS), PREFIX_SIGS, true);
290
+ }
291
+ if (count > ERROR_PAGE_MAX_WORDS) return false;
292
+ return showsErrorState(slice(whole, 0, HEAD_CHARS), count) || // A long description pushes the page's own words past the first 300 characters.
293
+ rest !== whole && showsErrorState(slice(rest, 0, HEAD_CHARS), words(rest.lc));
294
+ }
295
+ function usablePageText(text, wordCount) {
296
+ if (!text || typeof text !== "string" || !text.trim()) return null;
297
+ return looksLikeErrorPage(text, wordCount) ? null : text;
298
+ }
299
+ var QUALIFIED = String.raw`(?!\s+(?:in|on|during|until|outside|after|before|from|when|while|if|with|to|through|between|over|due|at(?!\s+(?:this|the)\s+(?:time|moment))|for(?!\s+(?:this|the|that)\s+(?:[\w'’&-]+\s+){0,3}?(?:page|url|link|listing|post|article|project|menu|item|resource|insight|study|portfolio)))\b)`;
300
+ var QUALIFIED_LIGHT = String.raw`(?!\s+(?:when|while|if|until|after|before|during|on|in|from|to|behind|outside|within)\b)`;
301
+ var THE_PAGE = String.raw`\b(?:this|the|that)\s+(?:[\w'’&.-]+\s+){0,4}?(?:page|url|link|route|listing|post|article|project|product|resource|insight|study|portfolio|document)`;
302
+ var ERROR_CLAIMS = [
303
+ // "Page Not Found", "Project Not Found", and the hyphenated compound a model coins:
304
+ // "a page-not-found message". Not "Page not found? Use our site map": a question is a help line.
305
+ new RegExp(String.raw`\b(?:page|project|post|article|listing|product)[\s-]+not[\s-]+found\b(?!\s*\?)`, "i"),
306
+ // "the pricing page is currently unavailable", "this URL is unavailable", "the page is currently down",
307
+ // "This page is no longer available", "This URL is not found"
308
+ new RegExp(
309
+ String.raw`\b(?:web\s?page|page|url|link|route)(?:\s+(?:from|at|on|for|of)\s+[\w'’&.-]+(?:\s+[\w'’&.-]+){0,4})?\s+(?:is|was|are|were|has\s+been|appears\s+to\s+be|seems\s+to\s+be|remains)\s+(?:(?:currently\s+|temporarily\s+|now\s+)?(?:unavailable|not\s+available|missing|offline|broken|gone)\b` + QUALIFIED + String.raw`|(?:currently\s+|temporarily\s+|now\s+)?(?:down|inaccessible|not\s+accessible|unreachable|not\s+found)\b` + QUALIFIED_LIGHT + String.raw`|no\s+longer\s+(?:available|accessible|reachable|live|online)\b` + QUALIFIED_LIGHT + String.raw`)`,
310
+ "i"
311
+ ),
312
+ // "Page Unavailable", "Page Currently Unavailable"
313
+ // "offline" only as a state ("the page is offline"): "save any page offline" is a feature.
314
+ new RegExp(String.raw`\bpage\s+(?:(?:is|was)\s+)?(?:currently\s+|temporarily\s+|now\s+)?(?:unavailable|not\s+available)\b` + QUALIFIED + String.raw`|\bpage\s+(?:is|was)\s+(?:currently\s+|temporarily\s+|now\s+)?offline\b`, "i"),
315
+ // "an unavailable-page presentation": the hyphenated compound again.
316
+ /\bunavailable-page\b/i,
317
+ // "could not be found" (or located, reached, loaded) with a page, URL or link as the subject
318
+ // The subject is THIS page ("this/the/that ... page"): "why your business page can't be found on Google" is an SEO headline.
319
+ /\b(?:this|the|that)\s+(?:[\w'’&.-]+\s+){0,6}?(?:page|url|link|route)\b[^.|;:]{0,60}?\b(?:could\s+not|couldn(?:'|’)t|can(?:'|’)t|cannot)\s+be\s+(?:found|located|reached|loaded|displayed|shown)\b/i,
320
+ // "currently returns a 404", "shows a not-found page", "displays a 404 page", "returns an HTTP 404 status"
321
+ /\b(?:return(?:s|ed|ing)?|show(?:s|ed|ing)?|display(?:s|ed|ing)?|serv(?:e|es|ed|ing)|lands?\s+on)\s+(?:that\s+)?(?:an?\s+|the\s+)?(?:http\s+)?(?:404|not[- ]found|error\s+page)/i,
322
+ /\b404[\s-]page\b/i,
323
+ // "details are unavailable", "menu information is unavailable"
324
+ new RegExp(String.raw`\b(?:details|information|info|content|menu|page)\s+(?:is|are)\s+(?:currently\s+|temporarily\s+)?unavailable\b` + QUALIFIED, "i"),
325
+ // A sentence that opens with the page as its subject: "This insight is unavailable."
326
+ // ...and the sentence's subject has to BE a page-like thing: "The patio is not available on Fridays" is about a patio.
327
+ new RegExp(String.raw`(?:^|[.!?]\s+)(?:this|the)\s+(?:[^.!?|\s][^.!?|]{0,69}?\s+)?(?:page|url|link|route|insight|article|post|project|study|resource|portfolio|guide|document)(?:\s+(?:you|that|which)\s+[^.!?|]{0,40}?)?\s+is\s+(?:currently\s+|temporarily\s+)?(?:unavailable|not\s+available)\b` + QUALIFIED, "i"),
328
+ // JSON-LD labels: "Unavailable Harbor Buffet page". A label, not "block unavailable dates on your booking page".
329
+ /(?:^|[|:.!?–—-]\s*)unavailable\s+(?:[\w'’&-]+\s+){0,6}?(?:page|url|menu|insight|article|project)\b/i,
330
+ // "This page doesn't exist.", "This page does not currently exist.", "No page exists at this address."
331
+ new RegExp(THE_PAGE + String.raw`\s+(?:(?:does\s+not|doesn['’]t)\s+(?:currently\s+|seem\s+to\s+|appear\s+to\s+)?exist|no\s+longer\s+exists?)\b(?!\s+(?:for|so)\s)` + QUALIFIED_LIGHT, "i"),
332
+ /\bno\s+(?:page|content|resource|listing)\s+exists?\s+(?:at|for|on|here)\b/i,
333
+ // "This page has been removed or moved.", "This listing has been removed." Moved to somewhere ("moved to our new site") is a redirect note.
334
+ new RegExp(THE_PAGE + String.raw`\s+(?:has|have|had)\s+(?:since\s+)?been\s+(?:permanently\s+)?(?:removed|moved|deleted|taken\s+down)\b` + QUALIFIED, "i"),
335
+ // "The page you're looking for isn't available.", "The page you requested has been removed."
336
+ new RegExp(
337
+ String.raw`\bpage\s+you(?:['’]re|\s+are|\s+were)?\s+(?:looking\s+for|requested|tried\s+to\s+(?:reach|visit|access))\s+(?:(?:is(?:n['’]t|\s+not)|was(?:n['’]t|\s+not))\s+(?:available|found|here|there|accessible)\b` + QUALIFIED + String.raw`|(?:has|have|might\s+have|may\s+have)\s+been\s+(?:removed|moved|deleted|renamed)\b|(?:doesn['’]t|does\s+not)\s+exist\b|no\s+longer\s+(?:exists?|available)\b)`,
338
+ "i"
339
+ ),
340
+ // "The page failed to load." (past tense: "if a page fails to load, refresh" is advice).
341
+ /\b(?:page|url|link|route)\s+(?:has\s+)?failed\s+to\s+load\b/i,
342
+ // "This URL contains only an error message.", "The page returned an error message instead of service details."
343
+ // Not "The checkout page shows an error message when payment fails."
344
+ new RegExp(String.raw`\b(?:page|url|link|route)\s+(?:currently\s+)?(?:(?:returned|returns?|displays?|displayed|shows?|showed|serves?|served)\s+(?:only\s+)?|(?:contains?|contained)\s+only\s+)(?:an?\s+|the\s+)?(?:error\s+(?:message|page)|http\s+error|not[- ]found\s+(?:message|error|page))\b(?=\s*(?:[.!?;:,|]|$)|\s+(?:instead|rather|in\s+place|and|but|only|with\s+no|with\s+nothing)\b)`, "i"),
345
+ // "404 Error | Acme Mowing", "Error 404: Acme Mowing", "HTTP 404", "404 Not Found | Acme": a label opening the copy or a segment of it.
346
+ /(?:^|[|:.!?–—-]\s*)(?:http\s+)?(?:error\s+404|404\s+error|http\s+404)\s*(?:[|:–—.!?-]|$)|\b404[\s:|–—-]*not\s+found\b/i,
347
+ // "This page isn't available.", "This page wasn't found." (the contracted forms of the above)
348
+ new RegExp(String.raw`\b(?:web\s?page|page|url|link|route)\s+(?:is|was)(?:n['’]t)\s+(?:currently\s+|now\s+)?(?:available|found|accessible|here|live)\b` + QUALIFIED_LIGHT, "i"),
349
+ // "We couldn't find this page on the Acme website.", "Sorry, we can't find that page.", "We could not locate the page you requested."
350
+ // Not "Can't find the page you need? Use search": a question in the same sentence is a help line.
351
+ /\b(?:we|i)\s+(?:could\s+not|couldn['’]t|can['’]t|cannot|(?:were|was|are|am)\s+unable\s+to|weren['’]t\s+able\s+to)\s+(?:seem\s+to\s+)?(?:find|locate)\s+(?:this|that|the)\s+(?:requested\s+)?(?:web\s?page|page|url|link)\b(?![^.!?]*\?)/i,
352
+ // "Unable to find the requested page."
353
+ /\bunable\s+to\s+(?:find|locate)\s+(?:this|that|the)\s+requested\s+(?:web\s?page|page|url|link|resource)\b/i,
354
+ // "The requested URL was not found on this server."
355
+ /\brequested\s+(?:url|uri|resource|page)\b[^.!?]{0,60}?\b(?:was\s+not|wasn['’]t|could\s+not\s+be|cannot\s+be)\s+found\b/i,
356
+ // "Page missing", "Error loading page": labels (a segment or the whole copy), not "pages missing from the index".
357
+ /(?:^|[|:.!?–—-]\s*)(?:page\s+(?:is\s+)?missing|error\s+loading\s+(?:the\s+)?page|not\s+found)\s*(?:[|:–—.!?-]|$)/i
358
+ ];
359
+ function ownTextOf(pageText) {
360
+ return pageText && typeof pageText === "string" && !looksLikeErrorPage(pageText) ? normalize(pageText) : "";
361
+ }
362
+ var CLAIM_MAX_CHARS = 5e3;
363
+ function claimIn(copy, own) {
364
+ const text = copy.replace(/\s+/g, " ").trim().slice(0, CLAIM_MAX_CHARS);
365
+ if (!text) return null;
366
+ for (const claim of ERROR_CLAIMS) {
367
+ const hit = text.match(claim);
368
+ if (!hit) continue;
369
+ const phrase = hit[0].replace(/^[.!?|:–—-]\s*/, "").trim();
370
+ if (own && own.includes(normalize(phrase))) continue;
371
+ return phrase;
372
+ }
373
+ return null;
374
+ }
375
+ function findErrorPageClaim(copy, pageText) {
376
+ if (!copy || typeof copy !== "string") return null;
377
+ return claimIn(copy, ownTextOf(pageText));
378
+ }
379
+ function describesErrorPage(copy, pageText) {
380
+ return findErrorPageClaim(copy, pageText) !== null;
381
+ }
382
+ var COPY_MAX_DEPTH = 8;
383
+ var COPY_MAX_LEAVES = 2e3;
384
+ var COPY_MAX_TOTAL_CHARS = 1e5;
385
+ function findPageCopyClaim(fields, pageText) {
386
+ if (!fields || typeof fields !== "object") return null;
387
+ const own = ownTextOf(pageText);
388
+ let leaves = 0;
389
+ let containers = 0;
390
+ let chars = 0;
391
+ const spent = () => leaves >= COPY_MAX_LEAVES || containers > COPY_MAX_LEAVES || chars > COPY_MAX_TOTAL_CHARS;
392
+ const walk = (value, level) => {
393
+ if (typeof value === "string") {
394
+ leaves++;
395
+ chars += Math.min(value.length, CLAIM_MAX_CHARS);
396
+ return spent() && chars > COPY_MAX_TOTAL_CHARS ? null : claimIn(value, own);
397
+ }
398
+ if (!value || typeof value !== "object") return null;
399
+ if (level >= COPY_MAX_DEPTH || ++containers > COPY_MAX_LEAVES) return null;
400
+ const record2 = value;
401
+ for (const key in record2) {
402
+ if (spent()) break;
403
+ if (!Object.prototype.hasOwnProperty.call(record2, key)) continue;
404
+ const phrase = walk(record2[key], level + 1);
405
+ if (phrase) return phrase;
406
+ }
407
+ return null;
408
+ };
409
+ const record = fields;
410
+ for (const key of Object.keys(record)) {
411
+ if (spent()) break;
412
+ const phrase = walk(record[key], 1);
413
+ if (phrase) return { field: key, phrase };
414
+ }
415
+ return null;
416
+ }
417
+
418
+ exports.ERROR_PAGE_CONTRACT_VERSION = ERROR_PAGE_CONTRACT_VERSION;
419
+ exports.ERROR_PAGE_MAX_WORDS = ERROR_PAGE_MAX_WORDS;
420
+ exports.describesErrorPage = describesErrorPage;
421
+ exports.findErrorPageClaim = findErrorPageClaim;
422
+ exports.findPageCopyClaim = findPageCopyClaim;
423
+ exports.looksLikeErrorPage = looksLikeErrorPage;
424
+ exports.usablePageText = usablePageText;
@@ -0,0 +1,416 @@
1
+ // src/contracts/error-page.ts
2
+ var ERROR_PAGE_CONTRACT_VERSION = 1;
3
+ var ERROR_PAGE_MAX_WORDS = 120;
4
+ var PREFIXED_MAX_WORDS = 400;
5
+ var SHORT_ONLY_MAX_WORDS = 60;
6
+ var START_STATE_MAX_WORDS = 40;
7
+ var HEAD_CHARS = 300;
8
+ var DESCRIBE_WINDOW = 45;
9
+ var NAMED_ENTITIES = {
10
+ amp: "&",
11
+ quot: '"',
12
+ apos: "'",
13
+ nbsp: " ",
14
+ lt: "<",
15
+ gt: ">",
16
+ rsquo: "'",
17
+ lsquo: "'",
18
+ ldquo: '"',
19
+ ndash: "\u2013",
20
+ mdash: "\u2014",
21
+ hellip: "\u2026"
22
+ };
23
+ function decodeEntities(text) {
24
+ let out = text;
25
+ for (let pass = 0; pass < 2 && out.indexOf("&") !== -1; pass++) {
26
+ out = out.replace(/&(?:#x([0-9a-f]{1,6})|#(\d{1,7})|([a-z]{2,8}));/gi, (whole, hex, dec, name) => {
27
+ if (name) {
28
+ const key = name.toLowerCase();
29
+ return Object.prototype.hasOwnProperty.call(NAMED_ENTITIES, key) ? NAMED_ENTITIES[key] : whole;
30
+ }
31
+ const code = hex ? parseInt(hex, 16) : parseInt(dec, 10);
32
+ return code <= 1114111 ? String.fromCodePoint(code) : whole;
33
+ });
34
+ }
35
+ return out;
36
+ }
37
+ function tidy(text) {
38
+ return decodeEntities(text).replace(/[‘’‛ʼ]/g, "'").replace(/\s+/g, " ").trim();
39
+ }
40
+ function normalize(text) {
41
+ return tidy(text).toLowerCase();
42
+ }
43
+ function textOf(text) {
44
+ const oc = tidy(text);
45
+ const lc = oc.toLowerCase();
46
+ return { lc, oc: lc.length === oc.length ? oc : lc };
47
+ }
48
+ function slice(text, from, to) {
49
+ return { lc: text.lc.slice(from, to), oc: text.oc.slice(from, to) };
50
+ }
51
+ function words(text) {
52
+ return text.split(/\s+/).filter(Boolean).length;
53
+ }
54
+ function wordsAcrossElements(text) {
55
+ return words(text.replace(/([a-z0-9.,;:!?)"'’])([A-Z])/g, "$1 $2"));
56
+ }
57
+ var NOT_A_QUESTION = String.raw`(?!\s*\?|\s+(?:in|on|for|at|with|under|from|during|to)\b[^.!?]{0,50}\?)`;
58
+ var NOT_IN_YOUR_SIZE = String.raw`(?!\s+in\s+your\s+size\b)`;
59
+ var RECORD_NOUN = String.raw`(?:page|project|post|article|product|listing|resource|file|deployment)`;
60
+ var CANT_BE_FOUND = String.raw`\b(?:page|url|link|resource|route|file|deployment|site|website|app|project)\b[^.!?]{0,40}?(?:could not|couldn't|can't|cannot|can not) be found`;
61
+ var WHICH_PAGE = String.raw`(?:\s+(?:you(?:'re| are| were)? (?:looking for|requested|asked for|(?:tried|were trying|are trying) to (?:reach|visit|access|find|open))|that you (?:requested|asked for)|at\s+\S[^.!?]{0,40}|on this (?:site|server|website)))?`;
62
+ var DOESNT_EXIST = String.raw`(?:\b(?:this|that|the|requested)\s+(?:[a-z0-9'-]+\s+){0,2}(?:page|url|link|post|article|listing|product|resource|project|profile|record|entry|story|event|course|job|item|offer|video)|\bpage)${WHICH_PAGE}\s+(?:doesn't|does not)(?: (?:seem|appear))?(?: to)? exist(?!\s+(?:to|for|so)\s)`;
63
+ var MAINTENANCE_QUALIFIER = String.raw`(?!\s+(?:on|from|every|each|this|next|between|during|at|after|before|once|twice|monthly|weekly|nightly|daily|contract|agreement|plan|program|warranty|coverage|with\s+(?:us|our)|for\s+(?:the|a|\d)|until(?!\s+(?:\d{1,2}(?::\d\d)?\s*(?:[ap]\.?m\b|o'clock)|\d{1,2}:\d\d|noon|midnight|tomorrow|further notice)))\b)`;
64
+ var ERROR_PAGE_SIGNATURES = [
65
+ // ---- Not found: the framework defaults and the common custom wordings ----
66
+ new RegExp(RECORD_NOUN + " (?:was |is )?not found" + NOT_A_QUESTION + NOT_IN_YOUR_SIZE),
67
+ // "Can't find the page you need? Use the search box" is a help line; "We can't find that page." isn't.
68
+ /(?:can't|cannot|can not|couldn't|could not|unable to) (?:be )?(?:find|found|locate)(?: that| this| the)? page(?!\s*\?|\s+(?:you|that|which)\b[^.!?]{0,40}\?)/,
69
+ // "We can't seem to find the page", "It seems we can't find what you're looking for".
70
+ /(?:we|i) (?:can't|cannot|couldn't|could not|were unable to|are unable to|weren't able to)(?: seem to)? (?:find|locate) (?:that page|this page|the page|that one|that(?=[.!,?]| link)|what you(?:'re| are| were)? looking for)/,
71
+ // "can not" is two words in the Vercel default.
72
+ new RegExp(CANT_BE_FOUND),
73
+ // Next's own not-found.tsx example: "Not FoundCould not find requested resourceReturn Home".
74
+ /(?:could not|couldn't) find (?:the )?requested resource/,
75
+ // "doesn't exist" only with a page, link or record as the subject: a short
76
+ // marketing page can say "if a tool doesn't exist, we'll build it".
77
+ new RegExp(DOESNT_EXIST),
78
+ /(?:doesn't|does not) exist or (?:has|have) been (?:moved|removed|deleted)/,
79
+ /looking for (?:doesn't|does not) exist/,
80
+ // A page-ish subject, like "doesn't exist" above: a history page can say "the 1890 mill no longer exists".
81
+ new RegExp(String.raw`\b(?:this|that|the)\s+(?:[a-z0-9'-]+\s+){0,2}(?:page|url|link|project|post|article|product|listing|resource|content|item|offer|event|record|story|profile|entry|document|course|job|video)` + WHICH_PAGE + String.raw`\s+no longer exists`),
82
+ /(?:404: ?not[ _]found|code: ?not_found)/,
83
+ // "Error 404" and "404 Error" as a label ("404 Error | Acme") or leading into what a 404 says
84
+ // ("Error 404Page not found"). Not "a custom 404 error page" and not a name ("Error 404 Studio").
85
+ /(?:^|[.!?|:]\s*)(?:http |server )?(?:error 404|404 error)(?=\s*[-–—|:.!?,)]|\s*(?:page|this|that|the|we|oops|whoops|sorry|looks|uh|hmm|nothing|there|it|oh|please)\b)|(?:error 404|404 error)(?=\s*$|\s*[-–—|:]?\s*(?:not found|this (?:page|url)|that page)\b)/,
86
+ // "The page you requested was not found", "The page you were looking for has been removed".
87
+ /page you (?:requested|(?:were|are) looking for|tried to (?:reach|visit|access))[^.!?]{0,60}(?:not found|no longer|(?:has|have|had|might have|may have|may) been|has (?:drifted|wandered|vanished|disappeared|moved|left)|isn't|is not|doesn't|does not|could not|can't|cannot|was not)/,
88
+ // Apache, Flask and Django: "The requested URL was not found on this server."
89
+ /requested (?:url|uri|resource|file|page|document)\b[^.!?]{0,80}(?:not found|not be found|cannot be found)|(?:was|is) not found on (?:this|the) server/,
90
+ // Netlify, Render, Flask and Express answer with the bare words "Not Found".
91
+ /^not found(?=$|\s*[-–—|:.!]|the requested|request id)/,
92
+ // WordPress core and its default themes: "It looks like nothing was found at this location.", "Nothing Found".
93
+ /nothing was found at this location|^nothing found(?=$|[.,:;!?|-]|sorry|it looks|we couldn|nothing|oops|the page)/,
94
+ // Custom 404 copy that never says "404": "Oops! This page took a day off.", "You've hit a dead end.",
95
+ // "This page doesn't live here anymore." The opener needs the page to be the one that's gone ("Oops! This
96
+ // page is a work in progress" is a holding page), and the dead end needs to be a sentence of its own.
97
+ /^(?:oops|whoops|uh[- ]oh)[!,.…]*\s*(?:this|that) page (?:took|went|wandered|got lost|ran away|has (?:gone|wandered|moved|left|vanished|disappeared|been (?:removed|moved|deleted))|is (?:missing|gone|lost)|isn't (?:here|available|there)|(?:doesn't|does not|can't|cannot|couldn't) (?:exist|be found|seem)|seems? (?:to be )?(?:missing|lost|gone))/,
98
+ /(?:^|[.!?]\s*)(?:you've|you have|you) (?:hit|reached) a dead end(?=[.!?…]|$)/,
99
+ /(?:page|link|url)[^.!?]{0,40}(?:doesn't|does not|no longer|isn't|is not) (?:live |exist |work )?here(?: anymore)?(?![a-z]*\s+to\b)/,
100
+ // Gatsby's 404 route: "NOT FOUNDYou just hit a route that doesn't exist... the sadness."
101
+ /hit a route that doesn't exist/,
102
+ // Tomcat and IIS: "HTTP Status 404 – Not Found", "HTTP Error 403.14 - Forbidden", "HTTP Error 503. The service is unavailable."
103
+ /http (?:error|status) [45]\d\d(?:\.\d+)?\s*[–—-]\s*(?:not found|forbidden|unauthori[sz]ed|bad request|internal|method not allowed|service unavailable)|http error [45]\d\d(?:\.\d+)?\. the /,
104
+ // A status number with its reason, anywhere in the head (a nav can precede it):
105
+ // "404 | Not Found", "502 - Bad Gateway", "403: Forbidden", "400Bad Request", "419 | Page Expired".
106
+ // Not "401 unauthorized means your key is missing": a status explained is documentation.
107
+ /(?:^|\D)(?:(?:404|410)\s*[|:–—-]?\s*(?:not found|page not found|file not found|web site not found)|408[\s:|–—-]*request time-?out|50[0-4][\s:|–—-]*(?:internal (?:server )?error|server error|bad gateway|service (?:temporarily )?unavailable|gateway time-?out)|40[13][\s:|–—-]*(?:forbidden|unauthori[sz]ed)|400[\s:|–—-]*bad request|405[\s:|–—-]*method not allowed|413[\s:|–—-]*(?:request entity|payload|content) too large|419[\s:|–—-]*page expired|429[\s:|–—-]*too many)(?!\s+(?:(?:requests?|errors?|responses?|status(?: codes?)?|codes?|messages?|pages?)\s+)?(?:means?|is|are|indicates?|occurs?|happens?|when|if|returns?|tells?|shows?|appears?|can|should|will|occurred)\b)/,
108
+ // Netlify's bare "Not Found - Request ID: ..." and Express's "Cannot GET /about", even with a nav in front.
109
+ /not found\s*[-–—]\s*request id|\bcannot (?:get|post|put|patch|delete) \/[\w\-./?=&%]*(?:$|\s)/,
110
+ // IIS's classic "Server Error 404 - File or directory not found." and its "had its name changed" sentence.
111
+ /file or directory not found|removed,? had its name changed/,
112
+ // This page has been removed / moved: its own sentence, not "removed from our menu".
113
+ /(?:^|[.!?]\s*)this page (?:has been |has |was |is )?(?:permanently )?(?:removed|deleted|moved)(?=[.!,;]|$|\s+(?:or|and)\b|\s*(?:return|go|back|click|please|visit|head)\b)/,
114
+ // ---- Server and client errors ----
115
+ // Not "Application error monitoring for Node teams".
116
+ /^application error(?!\s+(?:monitoring|tracking|reporting|handling|management|detection|analytics|logs?|alerts?|rates?|budgets?|codes?))/,
117
+ /(?:client|server)-side exception has occurred/,
118
+ // The hosts spell a status as a code too: "500: INTERNAL_SERVER_ERROR", "504: GATEWAY_TIMEOUT".
119
+ /internal[ _]server[ _]error/,
120
+ new RegExp(String.raw`(?:bad[ _]gateway|gateway[ _]time-?out)` + NOT_A_QUESTION),
121
+ // An exception's message rendered as the page: "Error: Failed to fetch data", "TypeError: x is undefined".
122
+ /^(?:(?:type|reference|syntax|range)error: |error: (?:failed|cannot|can't|unable|invalid|network|request|fetch|something|unexpected|no |not ))/,
123
+ // "Something went wrong with your insurance claim? ..." is a headline about a claim, not an error state.
124
+ new RegExp(String.raw`something(?: went|'s gone| has gone| had gone) wrong(?!\s+(?:with|during|on|in|at)\s+(?:your|my)\b)` + NOT_A_QUESTION),
125
+ /an (?:unexpected )?error (?:has )?occurred/,
126
+ /this (?:page|site|website) (?:isn't working|can't be reached)/,
127
+ // Next.js 16 error shells: "This page couldn't load. A server error occurred. Reload to try again."
128
+ // and the 403 / 401 pages ("This page could not be accessed.").
129
+ /this page couldn't load|a server error (?:has )?occurred|reload to try again(?=[.,!]|$)|this page could not be accessed/,
130
+ // React Router and Remix, Spring Boot, and the "unexpected error" boundaries.
131
+ /unexpected application error|whitelabel error page|(?:there was|encountered) an unexpected error/,
132
+ // Django and ASP.NET: "Server Error (500)", "Server Error in '/' Application", "Not Found (404)".
133
+ /server error (?:\(500\)|in '?\/)|(?:bad request|forbidden|not found) \((?:400|403|404)\)/,
134
+ // Next's runtime messages when a chunk or a hydration fails and the message is the page.
135
+ /chunkloaderror|loading chunk [\w/.-]+ failed|hydration failed because the|there was an error while hydrating/,
136
+ // Generated error.tsx boundaries: "Oops, we hit a snag.", "Something unexpected happened", "Unable to load page".
137
+ new RegExp(
138
+ String.raw`(?:^|[.!?]\s*)(?:(?:oops|whoops|uh[- ]oh)[!,.…]*\s*)?we hit a snag(?!\s+(?:with|in|on|during|when|while|at)\b)|(?:^|[.!?]\s*)something unexpected happened|(?:^|[.!?]\s*)(?:failed|unable) to load (?:the |this )?(?:page|content|data|resource)` + NOT_A_QUESTION + String.raw`|^error loading (?:the |this )?(?:page|content|data)`
139
+ ),
140
+ // "We're having trouble loading this page", "We couldn't load this page".
141
+ /(?:we're|we are|i'm) having (?:some )?(?:trouble|difficulty|difficulties|problems?) (?:loading|displaying|showing|retrieving) (?:this|that|the) (?:page|content)|we (?:couldn't|could not|can't|cannot) load (?:this|that|the) page/,
142
+ // WordPress core's fatal-error and database screens.
143
+ /there has been a critical error on this (?:website|site)|error establishing a database connection|(?:the|this) (?:site|website) is experiencing technical difficulties/,
144
+ // Apache's stock pages: 503, 400, 401, 408, 410 and 414 each in its own words.
145
+ /the server is temporarily unable to service your request|your browser sent a request that this server could not understand|this server could not verify that you are authorized|server timeout waiting for the http request|(?:was|is) no longer available on this server|request-uri too large/,
146
+ // ---- Unavailable and maintenance ----
147
+ // A qualifier ("in Hawaii", "until April") makes it content about a service, not a dead page.
148
+ /(?:this|the) (?:page|site|website|service) (?:is|are) (?:temporarily|currently) unavailable(?!\s+(?:in|on|for|at|during|to|until|from|outside|after|before|through)\b)/,
149
+ /^(?:service (?:temporarily )?unavailable|(?:temporarily|currently) unavailable(?!:\s))(?!\s+(?:in|on|for|at|during|to|until|from|outside)\b)/,
150
+ // The site or the page is the one under maintenance. Not "is under maintenance contract", not "down for
151
+ // maintenance on Fridays": a day, a time or a contract makes it content.
152
+ new RegExp(String.raw`(?:site|website|page|store|shop|app|portal|server|we're|we are)(?: is| are)?(?: currently| temporarily| now)*,? (?:under|down for) (?:scheduled |routine |some |emergency )?maintenance` + MAINTENANCE_QUALIFIER),
153
+ // "Our website is currently undergoing scheduled maintenance", "Briefly unavailable for scheduled maintenance",
154
+ // "We are offline for a short while". Not "Our rental fleet is undergoing routine maintenance": the subject is the site.
155
+ new RegExp(String.raw`(?:(?:site|website|page|store|shop|app|portal|server|we're|we are)(?: is| are)?(?: currently| temporarily| now)*|briefly|temporarily|currently|now) (?:undergoing|(?:offline|unavailable|down)(?: temporarily)? for) (?:scheduled |routine |some |emergency )?maintenance` + MAINTENANCE_QUALIFIER),
156
+ /(?:we're|we are|site is|website is) offline for an? (?:short|little|brief) (?:while|time|period)/,
157
+ /^be right back\.?$/,
158
+ // "We'll be back soon." ending its sentence, or leading into the next one. Not "We should be back soon with a replacement
159
+ // part", "back soon after our holiday break": what follows decides.
160
+ /(?:we'll|we will|we should|it should) (?:be )?(?:right )?back (?:online )?(?:soon|shortly)(?=[.!;]|$|,?\s*(?:our|the|this|thank|thanks|please|we)\b|\s+(?:once|after)\s+(?:a|the|our)\s+(?:(?:short|brief|quick|scheduled|planned)\s+)*(?:maintenance|upgrade|update|outage|downtime|migration|work)\b)/,
161
+ // An apology for the downtime makes what follows an outage notice, whatever it promises.
162
+ /sorry for the (?:downtime|inconvenience|interruption|outage)\W+(?:we'll|we will|we should|we are working)/,
163
+ // ---- Walls that stand in for the page ----
164
+ new RegExp(String.raw`(?:403 forbidden|access denied)(?!\s+(?:is|was|means|errors?|messages?|pages?)\b)` + NOT_A_QUESTION),
165
+ // A permission wall is addressed to the visitor and about THIS page or resource, and opens the text or follows its own
166
+ // heading ("Sorry, you are not allowed to access this page.", "ForbiddenYou are not authorized to access this resource.").
167
+ // Not a person or a file ("our team is not allowed to access this page of your file"), not a gallery ("you don't have
168
+ // permission to view the gallery"), and not a sentence further in ("Members only. You do not have permission to ...").
169
+ /(?:^|sorry|oops|error|forbidden|unauthori[sz]ed|access denied|40[13])\W*you(?:'re| are| aren't) (?:not |un)?(?:allowed|authori[sz]ed|permitted) to (?:access|view|see) (?:this|that) (?:page|resource|site|website|area|section|content)/,
170
+ /(?:^|sorry|oops|error|forbidden|unauthori[sz]ed|access denied|40[13])\W*you (?:don't|do not) have permission to (?:access|view) (?:this (?:page|resource|site|website|area|section|content)|\S+ on this server)/,
171
+ /^(?:forbidden|unauthori[sz]ed)(?=$|\s*[-–—|:.!]|you(?:'re| are)\b|please\b|this (?:page|resource))/,
172
+ /(?:checking your browser before accessing|enable javascript and cookies to continue(?!\s+\w)|verif(?:y|ying) you are (?:a )?human)/
173
+ ];
174
+ var LEADING_STATUS = /^(?:(?:http )?error:? ?)?(?:400|401|402|403|405|408|410|413|414|419|429|500|501|502|503|504)\W{0,3}(?:(?:service|server|temporarily|internal|gateway|bad|proxy|request|too|access|page|file|resource|connection|http|status|method|payload|entity|payment|authentication)\s+){0,3}(?:error|forbidden|unauthori[sz]ed|gone|unavailable|bad gateway|time-?out|timed out|not found|not allowed|too many|too large|bad request|page expired|payment required|not implemented)/;
175
+ var LEADING_404 = /^404(?![\d,])(?![\s.\-–—)]*\d)/;
176
+ var ADDRESS_AFTER_404 = /^404\s+(?:[a-z0-9.'-]+\s+){0,3}(?:street|st|avenue|ave|road|rd|boulevard|blvd|drive|dr|lane|ln|way|court|ct|highway|hwy|pike|parkway|pkwy|place|pl|suite|ste|unit|apt)\b/;
177
+ var SHORT_ONLY_SIGNATURES = [
178
+ // Opening the text, and not conditional: "Please refresh the page and try again." is the boundary's own line;
179
+ // "Clear your cache, then refresh the page and try again" is a tip, and "...try again if the calendar doesn't load" is one.
180
+ /^(?:please )?(?:refresh|reload)(?: the| this)? (?:page )?(?:and|or) try again(?!\s+(?:if|when|unless|whether|once|after|later|next|the next)\b)/
181
+ ];
182
+ var SHORT_START_SIGNATURES = [new RegExp(String.raw`^(?:[a-z0-9'&-]+\s+){0,2}[a-z0-9'&-]+ not found` + NOT_A_QUESTION + NOT_IN_YOUR_SIZE)];
183
+ var GLUED_404 = /(?:^|[\s|])404(?=[a-z])(?!th\b)/g;
184
+ var DELIMITED_404 = /(?:^|\s)404\s*[|:–—]\s*[a-z]/g;
185
+ var NUMBER_PREFIX = /(?:suite|ste|room|rm|unit|apt|apartment|route|rt|hwy|highway|building|bldg|floor|#)\.?\s*$/;
186
+ var ABOUT_THE_ERROR = String.raw`(?!\W*(?:errors?\b|explained|guide|what (?:it|this|that|is|are)|how to|how do|why|means|fix(?:es|ing)?\b|causes|vs\b|\?))`;
187
+ var PREFIX_SIGNATURES = [
188
+ new RegExp(String.raw`(?:page|resource|post|article|product|listing) (?:was |is )?not found`),
189
+ new RegExp(CANT_BE_FOUND),
190
+ /(?:we|i) (?:can't|cannot|couldn't|could not)(?: seem to)? (?:find|locate) (?:that page|this page|the page|what you(?:'re| are| were)? looking for)/,
191
+ new RegExp(DOESNT_EXIST),
192
+ /something went wrong|(?:client|server)-side exception|internal server error|this page couldn't load/
193
+ ].map((pattern) => new RegExp("(?:" + pattern.source + ")" + ABOUT_THE_ERROR));
194
+ var PREFIXED_LEADING_404 = new RegExp(LEADING_404.source + ABOUT_THE_ERROR);
195
+ var PREFIXED_LEADING_STATUS = new RegExp(LEADING_STATUS.source + ABOUT_THE_ERROR);
196
+ var PREFIX_SEPARATOR = " \u2014 ";
197
+ var PREFIX_MIN_CHARS = 60;
198
+ var PREFIX_MAX_CHARS = 400;
199
+ var SKIP_LINK = /^(?:skip to (?:main )?content|skip to main|skip navigation)\s*/;
200
+ var PREFIX_WINDOW_CHARS = 90;
201
+ var DESCRIBERS = String.raw`if|when|whether|unless|how to|how do|why|what is|what's|fix(?:ing|es)?|handle|handling|resolve|resolving|troubleshoot(?:ing)?|diagnos(?:e|ing)|watch(?:es|ing)? for|encounter(?:ing)?|history of`;
202
+ var LOWERCASE_DESCRIBERS = String.raw`about|monitor(?:ing)?|alert(?:s|ing)? (?:on|for)`;
203
+ var DESCRIBING_CONTEXT = new RegExp(
204
+ String.raw`(?:\b(?:(${DESCRIBERS})|(${LOWERCASE_DESCRIBERS}))(?:\s+(?:you|your|the|a|an|their|our|my|see|get|are|is|seeing|getting|receive|receiving|hit|to|every|any|each|all|\d{3})){0,3}\s*["'“]?|["“])$`
205
+ );
206
+ var INNER_DESCRIBER = /\b(?:if|when|whether|unless|how to)\b/;
207
+ var SENTENCE_OPENING = /[.!?:;|—–"“(]\s*$/;
208
+ var VISIBILITY_SUBJECT = /\b(?:you|your|we|our|they|their|customers?|clients?|people|visitors?|buyers?|patients?|business(?:es)?|brands?|compan(?:y|ies))\s+$/;
209
+ var FORM_FIELDS = /\b(?:name|e-?mail|phone|message|subscribe|submit)\b/;
210
+ function globalOf(patterns) {
211
+ return patterns.map((pattern) => new RegExp(pattern.source, "g"));
212
+ }
213
+ var SIGNATURES = globalOf(ERROR_PAGE_SIGNATURES);
214
+ var SHORT_ONLY_SIGS = globalOf(SHORT_ONLY_SIGNATURES);
215
+ var SHORT_START_SIGS = globalOf(SHORT_START_SIGNATURES);
216
+ var PREFIX_SIGS = globalOf(PREFIX_SIGNATURES);
217
+ var TITLE_VERB = /\b(?:fix|handle|resolve|troubleshoot|diagnose)\b/;
218
+ function describerIsProse(orig, start, describer, context) {
219
+ const word = orig.slice(start, start + describer.length);
220
+ if (word === describer || word === describer.charAt(0).toUpperCase() + describer.slice(1)) return true;
221
+ return start === 0 || SENTENCE_OPENING.test(orig.slice(Math.max(0, start - 3), start)) || TITLE_VERB.test(context);
222
+ }
223
+ function isDescribed(head, hit) {
224
+ const from = Math.max(0, hit.index - DESCRIBE_WINDOW);
225
+ const before = head.lc.slice(from, hit.index);
226
+ const context = DESCRIBING_CONTEXT.exec(before);
227
+ if (context) {
228
+ const [, describer, lowercaseDescriber] = context;
229
+ const at = from + context.index;
230
+ if (describer !== void 0) {
231
+ if (describerIsProse(head.oc, at, describer, context[0])) return true;
232
+ } else if (lowercaseDescriber !== void 0) {
233
+ if (head.oc.slice(at, at + lowercaseDescriber.length) === lowercaseDescriber) return true;
234
+ } else {
235
+ return true;
236
+ }
237
+ }
238
+ const inner = INNER_DESCRIBER.exec(hit[0]);
239
+ if (inner && head.oc.slice(hit.index + inner.index, hit.index + inner.index + inner[0].length) === inner[0]) return true;
240
+ return /be found$/.test(hit[0]) && VISIBILITY_SUBJECT.test(before) || /you are (?:a )?human$/.test(hit[0]) && FORM_FIELDS.test(head.lc);
241
+ }
242
+ function showsSignature(head, patterns, inFirstSentence = false) {
243
+ return patterns.some((pattern) => {
244
+ pattern.lastIndex = 0;
245
+ for (let hit = pattern.exec(head.lc); hit; hit = pattern.exec(head.lc)) {
246
+ if (inFirstSentence && /[.!?]/.test(head.lc.slice(0, hit.index))) return false;
247
+ if (!isDescribed(head, hit)) return true;
248
+ pattern.lastIndex = hit.index + 1;
249
+ }
250
+ return false;
251
+ });
252
+ }
253
+ function showsLoose404(head) {
254
+ for (const pattern of [GLUED_404, DELIMITED_404]) {
255
+ pattern.lastIndex = 0;
256
+ for (let hit = pattern.exec(head); hit; hit = pattern.exec(head)) {
257
+ if (!NUMBER_PREFIX.test(head.slice(Math.max(0, hit.index - 14), hit.index))) return true;
258
+ }
259
+ }
260
+ return false;
261
+ }
262
+ function afterPrefix(text) {
263
+ let start = 0;
264
+ const cut = text.lc.indexOf(PREFIX_SEPARATOR);
265
+ if (cut >= PREFIX_MIN_CHARS && cut < PREFIX_MAX_CHARS) start = cut + PREFIX_SEPARATOR.length;
266
+ const skip = SKIP_LINK.exec(text.lc.slice(start));
267
+ if (skip) start += skip[0].length;
268
+ return start === 0 ? text : slice(text, start);
269
+ }
270
+ function opensWith404(head) {
271
+ return LEADING_404.test(head) && !ADDRESS_AFTER_404.test(head);
272
+ }
273
+ function showsErrorState(head, count) {
274
+ return opensWith404(head.lc) || LEADING_STATUS.test(head.lc) || showsSignature(head, SIGNATURES) || count <= START_STATE_MAX_WORDS && showsSignature(head, SHORT_START_SIGS) || count <= SHORT_ONLY_MAX_WORDS && (showsSignature(head, SHORT_ONLY_SIGS) || showsLoose404(head.lc));
275
+ }
276
+ function looksLikeErrorPage(text, wordCount) {
277
+ if (!text || typeof text !== "string") return false;
278
+ const trimmed = text.trim();
279
+ if (!trimmed) return false;
280
+ const own = wordsAcrossElements(trimmed);
281
+ const count = typeof wordCount === "number" && wordCount >= 0 ? Math.max(wordCount, own) : own;
282
+ if (count > PREFIXED_MAX_WORDS) return false;
283
+ const whole = textOf(trimmed);
284
+ const rest = afterPrefix(whole);
285
+ if (own > ERROR_PAGE_MAX_WORDS) {
286
+ const lead = rest.lc.slice(0, HEAD_CHARS);
287
+ return PREFIXED_LEADING_404.test(lead) && !ADDRESS_AFTER_404.test(lead) || PREFIXED_LEADING_STATUS.test(lead) || showsSignature(slice(rest, 0, PREFIX_WINDOW_CHARS), PREFIX_SIGS, true);
288
+ }
289
+ if (count > ERROR_PAGE_MAX_WORDS) return false;
290
+ return showsErrorState(slice(whole, 0, HEAD_CHARS), count) || // A long description pushes the page's own words past the first 300 characters.
291
+ rest !== whole && showsErrorState(slice(rest, 0, HEAD_CHARS), words(rest.lc));
292
+ }
293
+ function usablePageText(text, wordCount) {
294
+ if (!text || typeof text !== "string" || !text.trim()) return null;
295
+ return looksLikeErrorPage(text, wordCount) ? null : text;
296
+ }
297
+ var QUALIFIED = String.raw`(?!\s+(?:in|on|during|until|outside|after|before|from|when|while|if|with|to|through|between|over|due|at(?!\s+(?:this|the)\s+(?:time|moment))|for(?!\s+(?:this|the|that)\s+(?:[\w'’&-]+\s+){0,3}?(?:page|url|link|listing|post|article|project|menu|item|resource|insight|study|portfolio)))\b)`;
298
+ var QUALIFIED_LIGHT = String.raw`(?!\s+(?:when|while|if|until|after|before|during|on|in|from|to|behind|outside|within)\b)`;
299
+ var THE_PAGE = String.raw`\b(?:this|the|that)\s+(?:[\w'’&.-]+\s+){0,4}?(?:page|url|link|route|listing|post|article|project|product|resource|insight|study|portfolio|document)`;
300
+ var ERROR_CLAIMS = [
301
+ // "Page Not Found", "Project Not Found", and the hyphenated compound a model coins:
302
+ // "a page-not-found message". Not "Page not found? Use our site map": a question is a help line.
303
+ new RegExp(String.raw`\b(?:page|project|post|article|listing|product)[\s-]+not[\s-]+found\b(?!\s*\?)`, "i"),
304
+ // "the pricing page is currently unavailable", "this URL is unavailable", "the page is currently down",
305
+ // "This page is no longer available", "This URL is not found"
306
+ new RegExp(
307
+ String.raw`\b(?:web\s?page|page|url|link|route)(?:\s+(?:from|at|on|for|of)\s+[\w'’&.-]+(?:\s+[\w'’&.-]+){0,4})?\s+(?:is|was|are|were|has\s+been|appears\s+to\s+be|seems\s+to\s+be|remains)\s+(?:(?:currently\s+|temporarily\s+|now\s+)?(?:unavailable|not\s+available|missing|offline|broken|gone)\b` + QUALIFIED + String.raw`|(?:currently\s+|temporarily\s+|now\s+)?(?:down|inaccessible|not\s+accessible|unreachable|not\s+found)\b` + QUALIFIED_LIGHT + String.raw`|no\s+longer\s+(?:available|accessible|reachable|live|online)\b` + QUALIFIED_LIGHT + String.raw`)`,
308
+ "i"
309
+ ),
310
+ // "Page Unavailable", "Page Currently Unavailable"
311
+ // "offline" only as a state ("the page is offline"): "save any page offline" is a feature.
312
+ new RegExp(String.raw`\bpage\s+(?:(?:is|was)\s+)?(?:currently\s+|temporarily\s+|now\s+)?(?:unavailable|not\s+available)\b` + QUALIFIED + String.raw`|\bpage\s+(?:is|was)\s+(?:currently\s+|temporarily\s+|now\s+)?offline\b`, "i"),
313
+ // "an unavailable-page presentation": the hyphenated compound again.
314
+ /\bunavailable-page\b/i,
315
+ // "could not be found" (or located, reached, loaded) with a page, URL or link as the subject
316
+ // The subject is THIS page ("this/the/that ... page"): "why your business page can't be found on Google" is an SEO headline.
317
+ /\b(?:this|the|that)\s+(?:[\w'’&.-]+\s+){0,6}?(?:page|url|link|route)\b[^.|;:]{0,60}?\b(?:could\s+not|couldn(?:'|’)t|can(?:'|’)t|cannot)\s+be\s+(?:found|located|reached|loaded|displayed|shown)\b/i,
318
+ // "currently returns a 404", "shows a not-found page", "displays a 404 page", "returns an HTTP 404 status"
319
+ /\b(?:return(?:s|ed|ing)?|show(?:s|ed|ing)?|display(?:s|ed|ing)?|serv(?:e|es|ed|ing)|lands?\s+on)\s+(?:that\s+)?(?:an?\s+|the\s+)?(?:http\s+)?(?:404|not[- ]found|error\s+page)/i,
320
+ /\b404[\s-]page\b/i,
321
+ // "details are unavailable", "menu information is unavailable"
322
+ new RegExp(String.raw`\b(?:details|information|info|content|menu|page)\s+(?:is|are)\s+(?:currently\s+|temporarily\s+)?unavailable\b` + QUALIFIED, "i"),
323
+ // A sentence that opens with the page as its subject: "This insight is unavailable."
324
+ // ...and the sentence's subject has to BE a page-like thing: "The patio is not available on Fridays" is about a patio.
325
+ new RegExp(String.raw`(?:^|[.!?]\s+)(?:this|the)\s+(?:[^.!?|\s][^.!?|]{0,69}?\s+)?(?:page|url|link|route|insight|article|post|project|study|resource|portfolio|guide|document)(?:\s+(?:you|that|which)\s+[^.!?|]{0,40}?)?\s+is\s+(?:currently\s+|temporarily\s+)?(?:unavailable|not\s+available)\b` + QUALIFIED, "i"),
326
+ // JSON-LD labels: "Unavailable Harbor Buffet page". A label, not "block unavailable dates on your booking page".
327
+ /(?:^|[|:.!?–—-]\s*)unavailable\s+(?:[\w'’&-]+\s+){0,6}?(?:page|url|menu|insight|article|project)\b/i,
328
+ // "This page doesn't exist.", "This page does not currently exist.", "No page exists at this address."
329
+ new RegExp(THE_PAGE + String.raw`\s+(?:(?:does\s+not|doesn['’]t)\s+(?:currently\s+|seem\s+to\s+|appear\s+to\s+)?exist|no\s+longer\s+exists?)\b(?!\s+(?:for|so)\s)` + QUALIFIED_LIGHT, "i"),
330
+ /\bno\s+(?:page|content|resource|listing)\s+exists?\s+(?:at|for|on|here)\b/i,
331
+ // "This page has been removed or moved.", "This listing has been removed." Moved to somewhere ("moved to our new site") is a redirect note.
332
+ new RegExp(THE_PAGE + String.raw`\s+(?:has|have|had)\s+(?:since\s+)?been\s+(?:permanently\s+)?(?:removed|moved|deleted|taken\s+down)\b` + QUALIFIED, "i"),
333
+ // "The page you're looking for isn't available.", "The page you requested has been removed."
334
+ new RegExp(
335
+ String.raw`\bpage\s+you(?:['’]re|\s+are|\s+were)?\s+(?:looking\s+for|requested|tried\s+to\s+(?:reach|visit|access))\s+(?:(?:is(?:n['’]t|\s+not)|was(?:n['’]t|\s+not))\s+(?:available|found|here|there|accessible)\b` + QUALIFIED + String.raw`|(?:has|have|might\s+have|may\s+have)\s+been\s+(?:removed|moved|deleted|renamed)\b|(?:doesn['’]t|does\s+not)\s+exist\b|no\s+longer\s+(?:exists?|available)\b)`,
336
+ "i"
337
+ ),
338
+ // "The page failed to load." (past tense: "if a page fails to load, refresh" is advice).
339
+ /\b(?:page|url|link|route)\s+(?:has\s+)?failed\s+to\s+load\b/i,
340
+ // "This URL contains only an error message.", "The page returned an error message instead of service details."
341
+ // Not "The checkout page shows an error message when payment fails."
342
+ new RegExp(String.raw`\b(?:page|url|link|route)\s+(?:currently\s+)?(?:(?:returned|returns?|displays?|displayed|shows?|showed|serves?|served)\s+(?:only\s+)?|(?:contains?|contained)\s+only\s+)(?:an?\s+|the\s+)?(?:error\s+(?:message|page)|http\s+error|not[- ]found\s+(?:message|error|page))\b(?=\s*(?:[.!?;:,|]|$)|\s+(?:instead|rather|in\s+place|and|but|only|with\s+no|with\s+nothing)\b)`, "i"),
343
+ // "404 Error | Acme Mowing", "Error 404: Acme Mowing", "HTTP 404", "404 Not Found | Acme": a label opening the copy or a segment of it.
344
+ /(?:^|[|:.!?–—-]\s*)(?:http\s+)?(?:error\s+404|404\s+error|http\s+404)\s*(?:[|:–—.!?-]|$)|\b404[\s:|–—-]*not\s+found\b/i,
345
+ // "This page isn't available.", "This page wasn't found." (the contracted forms of the above)
346
+ new RegExp(String.raw`\b(?:web\s?page|page|url|link|route)\s+(?:is|was)(?:n['’]t)\s+(?:currently\s+|now\s+)?(?:available|found|accessible|here|live)\b` + QUALIFIED_LIGHT, "i"),
347
+ // "We couldn't find this page on the Acme website.", "Sorry, we can't find that page.", "We could not locate the page you requested."
348
+ // Not "Can't find the page you need? Use search": a question in the same sentence is a help line.
349
+ /\b(?:we|i)\s+(?:could\s+not|couldn['’]t|can['’]t|cannot|(?:were|was|are|am)\s+unable\s+to|weren['’]t\s+able\s+to)\s+(?:seem\s+to\s+)?(?:find|locate)\s+(?:this|that|the)\s+(?:requested\s+)?(?:web\s?page|page|url|link)\b(?![^.!?]*\?)/i,
350
+ // "Unable to find the requested page."
351
+ /\bunable\s+to\s+(?:find|locate)\s+(?:this|that|the)\s+requested\s+(?:web\s?page|page|url|link|resource)\b/i,
352
+ // "The requested URL was not found on this server."
353
+ /\brequested\s+(?:url|uri|resource|page)\b[^.!?]{0,60}?\b(?:was\s+not|wasn['’]t|could\s+not\s+be|cannot\s+be)\s+found\b/i,
354
+ // "Page missing", "Error loading page": labels (a segment or the whole copy), not "pages missing from the index".
355
+ /(?:^|[|:.!?–—-]\s*)(?:page\s+(?:is\s+)?missing|error\s+loading\s+(?:the\s+)?page|not\s+found)\s*(?:[|:–—.!?-]|$)/i
356
+ ];
357
+ function ownTextOf(pageText) {
358
+ return pageText && typeof pageText === "string" && !looksLikeErrorPage(pageText) ? normalize(pageText) : "";
359
+ }
360
+ var CLAIM_MAX_CHARS = 5e3;
361
+ function claimIn(copy, own) {
362
+ const text = copy.replace(/\s+/g, " ").trim().slice(0, CLAIM_MAX_CHARS);
363
+ if (!text) return null;
364
+ for (const claim of ERROR_CLAIMS) {
365
+ const hit = text.match(claim);
366
+ if (!hit) continue;
367
+ const phrase = hit[0].replace(/^[.!?|:–—-]\s*/, "").trim();
368
+ if (own && own.includes(normalize(phrase))) continue;
369
+ return phrase;
370
+ }
371
+ return null;
372
+ }
373
+ function findErrorPageClaim(copy, pageText) {
374
+ if (!copy || typeof copy !== "string") return null;
375
+ return claimIn(copy, ownTextOf(pageText));
376
+ }
377
+ function describesErrorPage(copy, pageText) {
378
+ return findErrorPageClaim(copy, pageText) !== null;
379
+ }
380
+ var COPY_MAX_DEPTH = 8;
381
+ var COPY_MAX_LEAVES = 2e3;
382
+ var COPY_MAX_TOTAL_CHARS = 1e5;
383
+ function findPageCopyClaim(fields, pageText) {
384
+ if (!fields || typeof fields !== "object") return null;
385
+ const own = ownTextOf(pageText);
386
+ let leaves = 0;
387
+ let containers = 0;
388
+ let chars = 0;
389
+ const spent = () => leaves >= COPY_MAX_LEAVES || containers > COPY_MAX_LEAVES || chars > COPY_MAX_TOTAL_CHARS;
390
+ const walk = (value, level) => {
391
+ if (typeof value === "string") {
392
+ leaves++;
393
+ chars += Math.min(value.length, CLAIM_MAX_CHARS);
394
+ return spent() && chars > COPY_MAX_TOTAL_CHARS ? null : claimIn(value, own);
395
+ }
396
+ if (!value || typeof value !== "object") return null;
397
+ if (level >= COPY_MAX_DEPTH || ++containers > COPY_MAX_LEAVES) return null;
398
+ const record2 = value;
399
+ for (const key in record2) {
400
+ if (spent()) break;
401
+ if (!Object.prototype.hasOwnProperty.call(record2, key)) continue;
402
+ const phrase = walk(record2[key], level + 1);
403
+ if (phrase) return phrase;
404
+ }
405
+ return null;
406
+ };
407
+ const record = fields;
408
+ for (const key of Object.keys(record)) {
409
+ if (spent()) break;
410
+ const phrase = walk(record[key], 1);
411
+ if (phrase) return { field: key, phrase };
412
+ }
413
+ return null;
414
+ }
415
+
416
+ export { ERROR_PAGE_CONTRACT_VERSION, ERROR_PAGE_MAX_WORDS, describesErrorPage, findErrorPageClaim, findPageCopyClaim, looksLikeErrorPage, usablePageText };
package/package.json CHANGED
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "name": "@sonordev/contracts",
3
- "version": "2.2.0",
4
- "description": "The rules Sonor's sites, APIs and dashboard share: popup blocks and site design tokens, site hosts, SEO page identity, title quality, llms.txt sanitizers, the forms honeypot, fleet heartbeats, slot content and signing, site cache tags, portfolio metrics, the house voice's contraction rule and how a proposal's site plan counts its pages. Pure functions, no dependencies.",
3
+ "version": "2.3.0",
4
+ "description": "The rules Sonor's sites, APIs and dashboard share: popup blocks and site design tokens, site hosts, SEO page identity, title quality, llms.txt sanitizers, the forms honeypot, fleet heartbeats, slot content and signing, site cache tags, portfolio metrics, the house voice's contraction rule, how a proposal's site plan counts its pages, and what separates an error page from a page's content. Pure functions, no dependencies.",
5
5
  "license": "MIT",
6
6
  "repository": {
7
7
  "type": "git",
@@ -90,6 +90,11 @@
90
90
  "import": "./dist/proposal-sitemap.mjs",
91
91
  "require": "./dist/proposal-sitemap.js"
92
92
  },
93
+ "./error-page": {
94
+ "types": "./dist/error-page.d.ts",
95
+ "import": "./dist/error-page.mjs",
96
+ "require": "./dist/error-page.js"
97
+ },
93
98
  "./package.json": "./package.json"
94
99
  },
95
100
  "typesVersions": {