@softspark/ai-toolkit 4.20.0 → 4.22.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -47,9 +47,10 @@ Scan a codebase for SEO issues using pattern-matching heuristics. Detects W3C/HT
47
47
  ## What This Command Does
48
48
 
49
49
  1. **Detect framework and rendering mode** from `package.json`, config files, and entry HTML.
50
- 2. **Scan the codebase** using `Grep`/`Glob`/`Read` against framework-aware patterns for each category in scope.
51
- 3. **Interpret findings** with specific fix suggestions tied to the detected framework.
52
- 4. **Report** findings with file paths, line numbers, severity, confidence, and standards citations.
50
+ 2. **Run the scanner script** for a deterministic baseline over 8 of the 10 categories.
51
+ 3. **Extend the scan by hand** using `Grep`/`Glob`/`Read` against framework-aware patterns for everything the script does not cover.
52
+ 4. **Interpret findings** with specific fix suggestions tied to the detected framework.
53
+ 5. **Report** findings with file paths, line numbers, severity, confidence, and standards citations.
53
54
 
54
55
  ## Steps
55
56
 
@@ -90,11 +91,28 @@ Run detection before scanning so category patterns can adapt. Detection order:
90
91
 
91
92
  Report the detected framework and rendering mode in the Summary table.
92
93
 
93
- ### Step 2: Run Category Scans
94
+ ### Step 2: Run the Scanner Script
94
95
 
95
- For each category in `--scope`, apply the pattern set below using `Grep` (for regex across files) and `Read` (for config parsing / ordered checks). Patterns are framework-aware — use the framework detected in Step 1 to select the right rule set.
96
+ ```bash
97
+ python3 ${CLAUDE_SKILL_DIR}/scripts/seo-scanner.py [path] [--output json|text]
98
+ ```
99
+
100
+ Deterministic checks over meta tags, heading order, image `alt`, JSON-LD, hreflang,
101
+ `robots.txt`, sitemap, Core Web Vitals signals and `llms.txt`. No framework
102
+ awareness — it reads files, not build config. Categories 7 and 10 are not touched.
103
+
104
+ ### Step 3: Extend the Scan by Hand
105
+
106
+ Read [reference/scanner-categories.md](reference/scanner-categories.md) in full,
107
+ then work every category in `--scope` with `Grep` (regex across files) and `Read`
108
+ (config parsing / ordered checks), skipping only what Step 2 already reported.
109
+ Patterns are framework-aware — use the framework detected in Step 1 to select the
110
+ right rule set.
96
111
 
97
- ### Step 3: Interpret and Enrich
112
+ Categories 7 (Rendering Mode & Crawlability) and 10 (Topical Authority) get no help
113
+ from the script. Run them in full or say in the report that you did not.
114
+
115
+ ### Step 4: Interpret and Enrich
98
116
 
99
117
  For each finding:
100
118
 
@@ -103,316 +121,35 @@ For each finding:
103
121
  3. **Mark confidence**: `definitive` for regex matches against known-bad patterns, `heuristic` for co-occurrence / absence checks.
104
122
  4. **Skip false positives** when context shows the concern is addressed elsewhere (e.g., meta tags set in a layout file the route inherits from).
105
123
 
106
- ### Step 4: Report
124
+ ### Step 5: Report
107
125
 
108
126
  Present findings sorted by severity (HIGH → WARN → INFO), then by file path.
127
+ State which findings came from the script and which from the manual pass, and name
128
+ any category you did not complete.
109
129
 
110
130
  ---
111
131
 
112
132
  ## Scanner Reference
113
133
 
114
- ### Category 1: HTML Semantics & W3C
115
-
116
- Scan HTML/JSX/Vue/Svelte/Astro templates for W3C HTML5 compliance.
117
-
118
- | Pattern | Severity | Confidence | Description |
119
- |---------|----------|------------|-------------|
120
- | `<html>` without `lang` attribute | HIGH | definitive | HTML5 §3.2.6 `lang` required for SEO + a11y |
121
- | Missing `<meta charset="utf-8">` in `<head>` | HIGH | definitive | HTML5 §4.2.5.5 required first |
122
- | Missing `<meta name="viewport">` | HIGH | definitive | Mobile-first indexing requires viewport |
123
- | Multiple `<h1>` per page/route component | WARN | heuristic | One H1 per document is standard SEO practice |
124
- | No `<h1>` in page component | WARN | heuristic | Every indexable page should have H1 |
125
- | Heading level skip (h1 h3) | WARN | heuristic | Document outline breaks assistive tech + crawlers |
126
- | Missing landmarks (`<main>`, `<nav>`, `<header>`, `<footer>`) | WARN | heuristic | Semantic HTML aids both a11y and crawlers |
127
- | Missing `<!DOCTYPE html>` | HIGH | definitive | Triggers quirks mode in older browsers |
128
-
129
- See: [reference/w3c-guidelines.md](reference/w3c-guidelines.md)
130
-
131
- ---
132
-
133
- ### Category 2: Meta & Open Graph
134
-
135
- Check `<head>` composition in entry HTML, framework metadata exports, and route-level metadata.
136
-
137
- | Pattern | Severity | Confidence | Description |
138
- |---------|----------|------------|-------------|
139
- | Missing `<title>` / framework title | HIGH | definitive | Required for SERP display |
140
- | `<title>` >60 chars OR <10 chars | WARN | definitive | Recommended 50–60 char range |
141
- | Missing `<meta name="description">` | HIGH | definitive | Required for SERP snippets |
142
- | Description >160 chars OR <50 chars | WARN | definitive | Recommended 150–160 char range |
143
- | Missing `<link rel="canonical">` on indexable pages | HIGH | definitive | Prevents duplicate-content dilution |
144
- | `<meta name="robots" content="noindex">` on production route | WARN | heuristic | Confirm intentional — blocks indexing |
145
- | Missing OG tags: `og:title`, `og:description`, `og:image`, `og:url`, `og:type` | WARN | definitive | Required for rich social cards |
146
- | Missing Twitter Card (`twitter:card`) | WARN | definitive | Required for Twitter/X rich previews |
147
- | OG image without absolute URL | WARN | definitive | OG spec requires absolute URLs |
148
-
149
- **Framework adapters**:
150
- - **Next.js App Router**: look for `export const metadata = { ... }` or `generateMetadata()` in `layout.tsx`/`page.tsx`.
151
- - **Next.js Pages Router**: look for `<Head>` from `next/head`.
152
- - **Nuxt**: look for `useHead()` / `definePageMeta({ title, ... })`.
153
- - **Astro**: look for `<BaseHead>` component or direct `<meta>` in layout.
154
- - **Gatsby**: look for `<Helmet>` from `react-helmet`.
155
- - **SvelteKit**: look for `<svelte:head>` blocks.
156
- - **SPAs (Vue/Vite/CRA/Angular)**: look for `react-helmet-async`, `vue-meta`, `@angular/platform-browser`'s `Meta`/`Title` services. Flag runtime-only meta as a rendering-crawlability issue (Category 7).
157
-
158
- ---
159
-
160
- ### Category 3: Structured Data / Schema.org
161
-
162
- Scan for JSON-LD (`<script type="application/ld+json">`) presence and correctness on key page types.
163
-
164
- | Pattern | Severity | Confidence | Description |
165
- |---------|----------|------------|-------------|
166
- | No JSON-LD on article/blog route | WARN | heuristic | `Article` schema improves rich results |
167
- | JSON-LD missing `@context` | HIGH | definitive | Must be `https://schema.org` |
168
- | JSON-LD missing `@type` | HIGH | definitive | Type declaration is required |
169
- | `Article` missing `headline` / `author` / `datePublished` | WARN | definitive | Required properties per schema.org |
170
- | `FAQPage` missing `mainEntity` array | WARN | definitive | FAQ rich result needs Q&A pairs |
171
- | `BreadcrumbList` missing `itemListElement` | WARN | definitive | Breadcrumb rich result needs list |
172
- | `Organization` missing `name` / `url` / `logo` | WARN | definitive | Knowledge Graph signals |
173
- | `Product` missing `name` / `offers` / `aggregateRating` | WARN | definitive | Product rich results |
174
- | `LocalBusiness` missing `address` / `telephone` / `openingHours` | WARN | definitive | Local SEO signals |
175
-
176
- See: [reference/schema-types.md](reference/schema-types.md) for required-property matrix.
177
-
178
- ---
179
-
180
- ### Category 4: Hreflang & i18n
181
-
182
- Scan all locale variants for hreflang correctness.
183
-
184
- | Pattern | Severity | Confidence | Description |
185
- |---------|----------|------------|-------------|
186
- | Hreflang pair not bidirectional (A→B but not B→A) | HIGH | definitive | Google ignores unidirectional hreflang |
187
- | Missing `hreflang="x-default"` | WARN | definitive | Fallback required for unmatched locales |
188
- | Missing self-referencing hreflang tag | WARN | definitive | Each version must reference itself |
189
- | Invalid BCP 47 code (e.g., `en_US` instead of `en-US`) | HIGH | definitive | RFC 5646 requires hyphen-separated subtags |
190
- | Unknown language code (not ISO 639-1) | HIGH | definitive | Invalid language subtag |
191
- | Unknown region code (not ISO 3166-1 alpha-2) | HIGH | definitive | Invalid region subtag |
192
- | Hreflang points to URL returning canonical to different URL | WARN | heuristic | Canonical must match hreflang target |
193
-
194
- ---
195
-
196
- ### Category 5: Core Web Vitals (Static Signals)
197
-
198
- Detect code patterns that cause CWV regressions. Covers LCP, INP, CLS, resource hints, and above-the-fold optimization.
199
-
200
- #### 5a. LCP (Largest Contentful Paint, target <2.5s)
201
-
202
- | Pattern | Severity | Confidence | Description |
203
- |---------|----------|------------|-------------|
204
- | `<img>` without `width`/`height` attributes | HIGH | definitive | Causes CLS + delays LCP |
205
- | Above-the-fold `<img>` without `fetchpriority="high"` (or framework equivalent) | HIGH | heuristic | LCP image must be prioritized |
206
- | Above-the-fold `<img loading="lazy">` | HIGH | definitive | Actively harmful — delays LCP |
207
- | `@font-face` without `font-display` | HIGH | definitive | Blocks text paint |
208
- | Missing `<link rel="preload" as="image">` for known hero image | WARN | heuristic | Preload accelerates LCP |
209
- | Missing `<link rel="preload" as="font" crossorigin>` for self-hosted webfonts | WARN | heuristic | Fonts are a common LCP blocker |
210
- | Missing `<link rel="preconnect">` for 3rd-party font/image/CDN origins on critical path | WARN | heuristic | Saves ~100–300ms per origin |
211
- | Render-blocking `<link rel="stylesheet">` without `media` split or critical-inline | WARN | heuristic | Blocks first paint |
212
- | Responsive image: `<img>` >600px without `srcset`+`sizes` or `<picture>` | WARN | heuristic | Over-fetches on mobile |
213
- | **Next.js**: `<img>` used instead of `next/image` in route component | WARN | definitive | Misses automatic optimization |
214
- | **Next.js**: `next/image` without `priority` on detected LCP element | HIGH | heuristic | LCP will under-perform |
215
- | **Nuxt**: `<img>` instead of `<NuxtImg>`/`<NuxtPicture>` | WARN | definitive | Misses auto-optimization |
216
- | **Astro**: `<img>` instead of `<Image>` from `astro:assets` | WARN | definitive | Misses auto-optimization |
217
- | **Gatsby**: `<img>` instead of `GatsbyImage` | WARN | definitive | Misses auto-optimization |
218
-
219
- #### 5b. INP (Interaction to Next Paint, target <200ms)
220
-
221
- | Pattern | Severity | Confidence | Description |
222
- |---------|----------|------------|-------------|
223
- | `<script>` in `<head>` without `async`/`defer` | HIGH | definitive | Render-blocking |
224
- | Third-party analytics/chat/ads without `async`/`defer` or framework lazy strategy | WARN | definitive | Blocks main thread |
225
- | `document.write` usage | HIGH | definitive | Blocks parser; disabled by modern browsers |
226
- | Heavy top-level `useEffect(() => {...}, [])` (many sync calls) | WARN | heuristic | Long tasks delay INP |
227
- | Client bundle estimated >300KB gzipped gating interaction | WARN | heuristic | Excessive JS delays hydration + INP |
228
- | **Next.js**: `<Script>` without `strategy` prop on non-critical scripts | WARN | definitive | Defaults to `afterInteractive` — often not optimal |
229
- | Missing `fetchpriority="low"` on deferrable below-the-fold resources | INFO | heuristic | Helps browser prioritize LCP |
230
-
231
- #### 5c. CLS (Cumulative Layout Shift, target <0.1)
232
-
233
- | Pattern | Severity | Confidence | Description |
234
- |---------|----------|------------|-------------|
235
- | Images without `width`/`height` or `aspect-ratio` CSS | HIGH | definitive | Primary CLS cause |
236
- | Iframes (YouTube/maps/ads) without dimensions or aspect-ratio | HIGH | definitive | Embeds shift layout |
237
- | Dynamically injected ads/embeds without reserved placeholder space | WARN | heuristic | Shifts layout on load |
238
- | `@font-face` without `font-display: swap`/`optional` | WARN | definitive | FOIT/FOUT shifts |
239
- | SSR hydration mismatch: `typeof window` branches rendering different content | WARN | heuristic | Hydration-triggered shift |
240
- | Skeleton → content of different height | WARN | heuristic | Load-state shift |
241
-
242
- #### 5d. Resource Hints & Route Prefetching
243
-
244
- | Pattern | Severity | Confidence | Description |
245
- |---------|----------|------------|-------------|
246
- | `<link rel="preload">` for non-critical resource | WARN | heuristic | Wastes bandwidth + contention |
247
- | Next-route not prefetched when framework supports it (Next `<Link>`, Nuxt `<NuxtLink>`, SvelteKit `data-sveltekit-preload-data`) | INFO | heuristic | Hurts soft-navigation UX |
248
- | External origin referenced in critical path without `<link rel="preconnect">` | WARN | definitive | Adds 100–300ms per origin |
249
- | Less-critical external origin without `<link rel="dns-prefetch">` | INFO | heuristic | Lightweight fallback |
250
- | ESM chunks on critical path without `<link rel="modulepreload">` | INFO | heuristic | Helps browser parse ahead |
251
- | `<link rel="preload">` appears AFTER resource that uses it in document order | WARN | heuristic | Preload must come first to help |
252
- | >6 `<link rel="preload">` directives on one page | WARN | heuristic | Over-hinting — browsers throttle |
253
-
254
- #### 5e. Above-the-Fold Heuristic
255
-
256
- "Above-the-fold" candidates (confidence: heuristic):
257
- - First `<img>` / `<Image>` / `<NuxtImg>` / `<Image from 'astro:assets'>` / `GatsbyImage` inside a page/route component.
258
- - First child of `<main>` or `<section>`.
259
- - Components named `Hero`, `Banner`, `Masthead`, `Jumbotron`, `HeroSection`, `CoverImage`.
260
- - Images inside `<header>` that appear before any scroll-margin content.
261
-
262
- Rules for ATF elements:
263
- - MUST have explicit `width` + `height`.
264
- - MUST have high priority (`fetchpriority="high"` or `priority` prop).
265
- - MUST NOT have `loading="lazy"`.
266
- - SHOULD have a matching `<link rel="preload">` entry.
267
-
268
- Rules for below-the-fold:
269
- - SHOULD have `loading="lazy"` + `decoding="async"`.
270
- - MAY have `fetchpriority="low"`.
271
-
272
- See: [reference/core-web-vitals.md](reference/core-web-vitals.md)
273
-
274
- ---
275
-
276
- ### Category 6: GEO (Generative Engine Optimization)
277
-
278
- Content structure for AI answer engines (ChatGPT, Perplexity, Google AI Overviews, Bing Copilot, Google AI Mode). **Most findings here are severity `INFO` or `WARN`** — guidance based on measured citation patterns, not penalty-causing.
279
-
280
- Google's retrieval stage splits content into chunks of ≤500 tokens (~375 words). Each section must be a self-contained answer unit. See [reference/ai-pipeline.md](reference/ai-pipeline.md) for the full 4-stage pipeline and 7 ranking signals. See [reference/content-citability.md](reference/content-citability.md) for chunk anatomy, semantic triples, and hedging patterns.
281
-
282
- | Pattern | Severity | Confidence | Description |
283
- |---------|----------|------------|-------------|
284
- | No `FAQPage` schema on FAQ-style content | INFO | heuristic | Highly extractable by LLMs |
285
- | No `speakable` schema on summary content | INFO | heuristic | Voice/audio answer engines |
286
- | H2 section body exceeds ~375 words without an H3 sub-heading | WARN | heuristic | Exceeds single chunk boundary (~500 tokens); AI cannot extract cleanly — split with H3 |
287
- | First paragraph under a heading exceeds 60 words before a concrete fact, number, or direct recommendation | INFO | heuristic | AI extracts first 2–3 sentences as the answer; preamble displaces the answer |
288
- | Hedging language in recommendation or product context: "may be", "might be", "could be", "worth considering", "for many", "for most people" | INFO | heuristic | AI skips hedged claims; Jetstream signal rewards declarative recommendations (see [content-citability.md](reference/content-citability.md)) |
289
- | No decision framework ("if X → choose Y" / "for X, use Y") in guide or category content | INFO | heuristic | Decision frameworks are the most-cited AI construction; covers Jetstream cross-attention signal |
290
- | No contrast or comparison ("X vs Y", "unlike X", "in contrast to X") in content with comparative headings | INFO | heuristic | Jetstream directly rewards explicit contrasts; absence reduces AI citation probability |
291
- | No negative definition ("not recommended for", "not suitable for", "avoid if") on product or category pages | INFO | heuristic | Covers AI exclusion sub-queries ("which product is not for stomach sleepers?") |
292
- | Author name uses generic placeholder: "Admin", "Team", "Staff", "Editor", or no author at all | WARN | heuristic | E-E-A-T Experience signal requires a real named author; generic names suppressed by Google Bury Rules |
293
- | Author block contains fewer than 30 words of bio text near the author name | INFO | heuristic | LLM answer engines use author credentials as an authority signal; stub bios do not qualify |
294
- | Article `dateModified` (JSON-LD or `<time>`) is older than 13 weeks with no visible update notice | WARN | heuristic | 50% of top AI-cited content updated within 13 weeks (Blyskall, 40M AI Overviews study); stale content drops from citation pools |
295
- | Missing explicit citation/source markup (`<cite>`, author bylines) | INFO | heuristic | LLM answer engines prefer attributable sources |
296
- | No `<q>` or quote schema on quoted content | INFO | heuristic | Aids AI extraction |
297
- | No Q&A structure on how-to content | INFO | heuristic | LLMs favor structured Q&A |
298
- | Heavy reliance on `<div>` over semantic HTML | INFO | heuristic | Semantic HTML improves AI parsing |
299
- | Key facts hidden behind JS interactions (tabs, accordions) | INFO | heuristic | LLMs see initial DOM only |
300
-
301
- See: [reference/geo-guidelines.md](reference/geo-guidelines.md), [reference/content-citability.md](reference/content-citability.md), [reference/ai-pipeline.md](reference/ai-pipeline.md)
302
-
303
- ---
304
-
305
- ### Category 7: Rendering Mode & SPA/CSR/SSG Crawlability ⭐
306
-
307
- **The most critical category for JS apps.** A CSR-only app with no prerendering is effectively invisible to most crawlers.
308
-
309
- | Pattern | Severity | Confidence | Description |
310
- |---------|----------|------------|-------------|
311
- | Entry HTML contains only mount point (`<div id="root">` or `<div id="app">`) with no prerendered content, and no SSR/SSG configured | HIGH | definitive | Crawlers see empty page |
312
- | Meta/title set only in JS runtime (react-helmet-async, vue-meta, `document.title = ...`) with no SSR/SSG fallback | HIGH | definitive | Public routes won't have crawlable meta |
313
- | `HashRouter` / hash-based routing (`/#/about`) on public routes | HIGH | definitive | Google ignores fragments for indexing |
314
- | CSR app without `<noscript>` fallback containing meaningful content | WARN | heuristic | Minimum no-JS signal for crawlers |
315
- | **Next.js**: `'use client'` at top of every page/layout forcing CSR | WARN | heuristic | Defeats SSR/SSG benefits |
316
- | **Next.js**: Content page missing `generateMetadata` / static `metadata` export | WARN | heuristic | No crawlable metadata |
317
- | **Next.js**: `dynamic(..., { ssr: false })` wrapping LCP / above-the-fold content | HIGH | definitive | Blocks both SSR and LCP |
318
- | **Nuxt**: `ssr: false` in config or route with public content | WARN | heuristic | Disables SSR intentionally |
319
- | **Astro**: `client:only` on hero/content components | WARN | heuristic | Component not prerendered |
320
- | **SvelteKit**: `export const ssr = false` on public route | WARN | heuristic | Disables SSR |
321
- | **Gatsby**: route excluded from prerender (`gatsby-plugin-exclude`) | WARN | heuristic | Verify intent |
322
- | **Angular SPA**: project uses `@angular/core` without `@angular/ssr` or `@nguniversal/*` | HIGH | definitive | Default Angular is CSR-only |
323
- | **Vue SPA / React SPA / CRA / Vite-SPA**: no prerender plugin detected (no `vite-plugin-ssr`, `react-snap`, `prerender-spa-plugin`, `vite-plugin-prerender`) | HIGH | definitive | Content invisible to crawlers |
324
- | `suppressHydrationWarning` overuse (>3 occurrences) | WARN | heuristic | Masks real hydration mismatches |
325
- | `typeof window !== 'undefined'` / `isBrowser` checks in render paths | WARN | heuristic | Often signals hydration mismatch |
326
- | Static `robots.txt` references dynamic routes that aren't prerendered | WARN | heuristic | Crawlers hit empty pages |
327
- | `prerender.io` / `rendertron` / dynamic-rendering middleware detected | INFO | definitive | Legacy pattern — Google now prefers SSR/SSG |
328
-
329
- See: [reference/spa-ssg-patterns.md](reference/spa-ssg-patterns.md)
330
-
331
- ---
332
-
333
- ### Category 8: Technical SEO
334
-
335
- | Pattern | Severity | Confidence | Description |
336
- |---------|----------|------------|-------------|
337
- | Missing `robots.txt` | HIGH | definitive | Blocks crawler directives + sitemap reference |
338
- | `robots.txt` contains `Disallow: /` in production build | HIGH | definitive | Blocks entire site |
339
- | Missing `sitemap.xml` / framework sitemap generator | HIGH | definitive | Slows discovery |
340
- | `robots.txt` missing `Sitemap:` directive | WARN | definitive | Crawlers may not find sitemap |
341
- | Canonical URLs inconsistent with actual deployed URLs | WARN | heuristic | Dilutes link equity |
342
- | Canonical URL includes query params on parametrized pages (e.g., `?q=`, `?page=`, `?sort=`) | HIGH | heuristic | Canonical must point to clean base URL, not parametrized variant — else each query variant is a duplicate |
343
- | Site has search feature (detected: `<input type="search">`, `<form action="/search">`, route `/search`, `?q=` / `?query=` / `?s=` / `?search=`) but `robots.txt` does NOT `Disallow` the search URL pattern | HIGH | heuristic | Parametrized search URLs create unlimited duplicate-content pages — crawl budget waste + index bloat |
344
- | Site has faceted navigation (filters, sort params, pagination like `?filter=`, `?sort=`, `?page=`, `?color=`) without `robots.txt` Disallow rules OR parameter-handling via canonical | WARN | heuristic | Faceted URLs multiply indexable variants exponentially |
345
- | Search result page (SRP) missing `<meta name="robots" content="noindex, follow">` | HIGH | heuristic | SRPs are thin/duplicate content per Google Search Essentials; indexing wastes crawl budget |
346
- | Search result page missing self-referencing canonical OR canonical with dynamic query in it | WARN | heuristic | SRP should either canonical to clean `/search` or be noindexed entirely |
347
- | Parametrized URLs (tracking: `utm_*`, `gclid`, `fbclid`, `ref=`) served without canonical to clean URL | HIGH | heuristic | Tracking params create duplicate URLs — canonical must strip them |
348
- | Trailing-slash inconsistency (some pages `/about/`, some `/about`) | WARN | heuristic | Duplicate-content risk |
349
- | HTTPS not enforced (hardcoded `http://` internal links) | WARN | definitive | Mixed-content + security |
350
- | No 404 page / no custom `not-found` route | WARN | heuristic | Default 404s hurt UX |
351
- | Meta `robots: noindex,nofollow` on indexable production routes | HIGH | heuristic | Blocks indexing — verify intent |
352
-
353
- **Parameter-handling guidance**: Google deprecated the Search Console URL Parameters tool in April 2022. Today the only signals are:
354
- 1. **Canonical tags** — every parametrized variant must `<link rel="canonical">` to the clean base URL.
355
- 2. **`robots.txt` Disallow rules** — block crawlers from following parameter patterns entirely (`Disallow: /*?q=*`, `Disallow: /search?*`).
356
- 3. **`noindex` meta** — allow crawl (for link discovery) but prevent indexing on SRPs and thin faceted pages.
357
-
358
- Choose ONE strategy per parameter type — mixing `Disallow` + `noindex` is contradictory (Disallow prevents crawler from ever seeing the noindex directive).
359
-
360
- **Example `robots.txt` for a site with search**:
361
- ```
362
- User-agent: *
363
- Disallow: /search?*
364
- Disallow: /*?q=*
365
- Disallow: /*?query=*
366
- Disallow: /*?s=*
367
- Disallow: /*?utm_*
368
- Disallow: /*?gclid=*
369
- Disallow: /*?fbclid=*
370
- Allow: /
371
-
372
- Sitemap: https://example.com/sitemap.xml
373
- ```
374
-
375
- **Example canonical on a parametrized page** (`/products?category=shoes&color=red&sort=price`):
376
- ```html
377
- <link rel="canonical" href="https://example.com/products">
378
- ```
379
-
380
- The canonical points to the clean page; the specific filter combination is a view, not a distinct URL.
381
-
382
- ---
383
-
384
- ### Category 9: Accessibility for SEO
385
-
386
- Accessibility ↔ SEO overlap. WCAG compliance improves ranking signals.
387
-
388
- | Pattern | Severity | Confidence | Description |
389
- |---------|----------|------------|-------------|
390
- | `<img>` missing `alt` attribute | WARN | definitive | WCAG 1.1.1 + image SEO |
391
- | `<img alt="">` on informational image | WARN | heuristic | Empty alt only for decorative |
392
- | Icon-only `<button>` without `aria-label` | WARN | definitive | Screen readers + semantic crawlers |
393
- | Form `<input>` without associated `<label>` | WARN | definitive | WCAG 3.3.2 |
394
- | `<div>` used for interactive element (click handler on `<div>`) | WARN | heuristic | Should be `<button>` or `<a>` |
395
- | Link text is "click here" / "read more" | WARN | heuristic | Anchor text is a ranking signal |
396
- | `<a>` without `href` (fake link) | WARN | definitive | Not crawlable |
397
-
398
- ---
399
-
400
- ### Category 10: Topical Authority & Cluster Architecture
401
-
402
- Topical authority is the degree to which a domain is recognised as an expert source across an entire topic, not just individual pages. AI retrieval (Gecko Score / semantic embedding) rewards domains with deep, interlinked coverage. Classical SEO also benefits — Senuto's study of 212K phrases across 7,200 semantic groups showed topical coverage dominates top-10 rankings independently of individual technical metrics.
403
-
404
- | Pattern | Severity | Confidence | Description |
405
- |---------|----------|------------|-------------|
406
- | Long-form page (>800 words) has internal link density below 1 link per 800 characters of body text | WARN | heuristic | Google's internal linking guideline: ~1 contextual internal link per 800 chars; low density = weak cluster signal |
407
- | Internal link uses generic anchor text: "click here", "read more", "here", "this page", "learn more" | WARN | definitive | Anchor text is a topical signal; descriptive claim-based anchors transfer semantic context to the linked page |
408
- | Page >2,000 words with no outbound internal links to topically related pages | INFO | heuristic | Pillar pages must link out to cluster articles; absence breaks the pillar→cluster signal and reduces Gecko relevance |
409
- | Page has >500 words of indexable content with zero detected inbound internal links (orphan page) | WARN | heuristic | Orphan pages receive minimal crawl budget and no authority pass-through; every content page needs at least one inbound link |
410
- | Content page URL slug contains numeric IDs, UUIDs, or is purely numeric (e.g., `/post/12345`, `/p/abc-uuid`) | WARN | heuristic | Natural-language slugs (5–7 descriptive words) show +11.4% AI citation rate vs. ID-based URLs (Blyskall study) |
411
- | Two or more pages on the same domain target the same primary keyword in H1 and title | WARN | heuristic | Keyword cannibalization: pages compete against each other, diluting authority; consolidate into pillar + cluster |
412
-
413
- **Topical authority strategy note:** Query Fan Out means AI generates 50+ sub-queries per user question, 95% of which have zero Monthly Search Volume in any keyword tool. Covering a topic with a pillar + cluster architecture answers the full sub-query space that keyword tools cannot see. See [reference/ai-pipeline.md](reference/ai-pipeline.md).
414
-
415
- ---
134
+ `scripts/seo-scanner.py` gives a deterministic baseline over 8 of the 10 categories.
135
+ The full pattern tables, per-framework rules and standards citations live in
136
+ [reference/scanner-categories.md](reference/scanner-categories.md).
137
+
138
+ | # | Category | Covered by the script | Manual pass |
139
+ |---|----------|----------------------|-------------|
140
+ | 1 | HTML Semantics & W3C | heading order | rest of the category |
141
+ | 2 | Meta & Open Graph | title, description, OG/Twitter tags | rest of the category |
142
+ | 3 | Structured Data / Schema.org | JSON-LD presence and shape | rest of the category |
143
+ | 4 | Hreflang & i18n | hreflang pairs, x-default | rest of the category |
144
+ | 5 | Core Web Vitals (static signals) | LCP/CLS/INP signal patterns | rest of the category |
145
+ | 6 | GEO (Generative Engine Optimization) | `llms.txt` presence | rest of the category |
146
+ | 7 | Rendering Mode & SPA/CSR/SSG Crawlability | **nothing** | the whole category |
147
+ | 8 | Technical SEO | `robots.txt`, sitemap | rest of the category |
148
+ | 9 | Accessibility for SEO | image `alt` | rest of the category |
149
+ | 10 | Topical Authority & Cluster Architecture | **nothing** | the whole category |
150
+
151
+ Categories 7 and 10 have no script coverage at all — they are entirely manual, and
152
+ skipping them is the most likely way this skill under-reports.
416
153
 
417
154
  ## Output Format
418
155
 
@@ -479,6 +216,23 @@ See: reference/spa-ssg-patterns.md#react-spa-migration
479
216
  - **Noscript is not a substitute for SSR/SSG**: `<noscript>` catches only the "no-JS" case, not the "crawler without JS execution" case — don't upgrade a CSR HIGH to WARN just because noscript exists.
480
217
  - **No auto-fix in v1**: Fixing SEO issues requires design/content decisions beyond pattern matching.
481
218
 
219
+ ## Gotchas
220
+
221
+ - `scripts/seo-scanner.py` reads **files, not build output**. A Next.js `metadata` export, a `useHead()` call, or a `<svelte:head>` block produces correct tags at runtime that the scanner cannot see, so "missing meta description" on a framework project is a claim about the source, not about the page.
222
+ - Categories **7 (Rendering & Crawlability)** and **10 (Topical Authority)** get zero script coverage. They are also the two that most often carry the real problem, because a CSR-only app can pass every other category while being invisible to crawlers. Skipping Step 3 for these silently converts the worst finding into no finding.
223
+ - `robots.txt` and `sitemap.xml` are checked **at the project root**. Frameworks that generate them at build time (`next-sitemap`, `@astrojs/sitemap`, `gatsby-plugin-sitemap`) leave nothing on disk, so absence is not evidence — check the config and the plugin list before reporting it.
224
+ - Hreflang correctness needs **both directions**. A page declaring `hreflang="de"` is only valid if the German page declares the reverse; a one-file scan sees half the pair and cannot conclude.
225
+ - Core Web Vitals here are **static signals only** — missing `width`/`height`, unbounded images, render-blocking patterns. Real LCP/CLS/INP come from field data (CrUX, RUM). Reporting a passing CWV category is out of this skill's reach.
226
+ - Category 6 (GEO) is genuinely young. Its "signals" track how generative engines behaved recently, not a ratified standard, so its findings stay WARN/INFO by rule — raising one to HIGH asserts more certainty than the field supports.
227
+
228
+ ## When NOT to Use
229
+
230
+ - On an auth-gated app — dashboards and admin panels have no crawler audience, and every category-7 finding will be noise.
231
+ - To measure Core Web Vitals — use PageSpeed Insights, CrUX, or your RUM; this skill sees source patterns, not field metrics.
232
+ - To verify tags a framework generates at build time — build the site and scan the output, or read the framework config instead.
233
+ - For accessibility beyond the SEO overlap — use `/a11y-validate`; category 9 here is deliberately shallow.
234
+ - To rewrite content for topical authority — this skill is read-only and reports gaps; the writing is a separate job.
235
+
482
236
  ## Reference Documents
483
237
 
484
238
  - [reference/w3c-guidelines.md](reference/w3c-guidelines.md) — HTML5 semantic requirements, meta tag specs, language tag rules.