dotcms-custom-mcp 1.3.0 → 1.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +19 -0
- package/guides.js +277 -0
- package/index.js +66 -2
- package/package.json +2 -2
- package/skills/NOTICE.md +17 -0
- package/skills/brightdata/LICENSE +21 -0
- package/skills/brightdata/skills/seo-audit/SKILL.md +109 -0
- package/skills/brightdata/skills/seo-audit/references/audit-framework.md +71 -0
- package/skills/brightdata/skills/seo-audit/references/bdata-recipes.md +551 -0
- package/skills/brightdata/skills/seo-audit/references/output-templates.md +107 -0
- package/skills/brightdata/skills/seo-audit/references/site-type-playbooks.md +113 -0
- package/skills/marketing/LICENSE +202 -0
- package/skills/marketing/skills/seo-audit/SKILL.md +190 -0
- package/skills/searchfit-seo/LICENSE +21 -0
- package/skills/searchfit-seo/agents/competitor-analyzer/AGENT.md +92 -0
- package/skills/searchfit-seo/agents/content-strategist/AGENT.md +122 -0
- package/skills/searchfit-seo/agents/seo-auditor/AGENT.md +110 -0
- package/skills/searchfit-seo/commands/create-content.md +98 -0
- package/skills/searchfit-seo/commands/create-topic.md +116 -0
- package/skills/searchfit-seo/commands/generate-schema.md +66 -0
- package/skills/searchfit-seo/commands/keyword-cluster.md +58 -0
- package/skills/searchfit-seo/commands/seo-check.md +80 -0
- package/skills/searchfit-seo/commands/translate-content.md +86 -0
- package/skills/searchfit-seo/skills/ai-visibility/SKILL.md +126 -0
- package/skills/searchfit-seo/skills/broken-links/SKILL.md +122 -0
- package/skills/searchfit-seo/skills/content-brief/SKILL.md +127 -0
- package/skills/searchfit-seo/skills/content-strategy/SKILL.md +137 -0
- package/skills/searchfit-seo/skills/content-translation/SKILL.md +142 -0
- package/skills/searchfit-seo/skills/internal-linking/SKILL.md +108 -0
- package/skills/searchfit-seo/skills/keyword-clustering/SKILL.md +117 -0
- package/skills/searchfit-seo/skills/on-page-seo/SKILL.md +111 -0
- package/skills/searchfit-seo/skills/schema-markup/SKILL.md +159 -0
- package/skills/searchfit-seo/skills/seo-audit/SKILL.md +109 -0
- package/skills/searchfit-seo/skills/technical-seo/SKILL.md +161 -0
|
@@ -0,0 +1,551 @@
|
|
|
1
|
+
# bdata Recipes — SEO Audit
|
|
2
|
+
|
|
3
|
+
Concrete `bdata` commands for each audit step. Each recipe has the same four parts: Purpose, Command(s), Parsing, Evidence-to-cite. Parse-only recipes that re-use rendered HTML from R-04 (rather than making a fresh `bdata` call) omit the Command section — their input is whichever R-04 page output the analysis is being run against.
|
|
4
|
+
|
|
5
|
+
## Per-Audit Data Flow (Mode B)
|
|
6
|
+
|
|
7
|
+
```
|
|
8
|
+
1. Discover → R-01 (robots.txt) + R-02 (sitemap.xml) → URL list, group by path prefix
|
|
9
|
+
2. Sample → R-03 stratified pick of 10–15 URLs (homepage + per-section samples)
|
|
10
|
+
3. Fetch → R-04 parallel: bdata scrape <each-url> -f html
|
|
11
|
+
(single Bash message, multiple tool calls)
|
|
12
|
+
4a. SERP (always)→ R-12 site: indexation proxy (one call, always run)
|
|
13
|
+
4b. SERP (cond.) → R-13/R-14 (only when user named a target keyword or asked a diagnostic prompt)
|
|
14
|
+
5. Analyze → parse rendered HTML per page (R-05..R-11), aggregate site-wide signals
|
|
15
|
+
6. Report → format per output-templates.md
|
|
16
|
+
```
|
|
17
|
+
|
|
18
|
+
## Cost Discipline (read first)
|
|
19
|
+
|
|
20
|
+
- **Default page budget**: 10–15 pages for site-wide audits. The user can ask for a larger budget in natural language (e.g., "audit 30 pages") — there is no `bdata` CLI flag for this; it's an audit-level parameter the skill applies when sampling URLs in R-03.
|
|
21
|
+
- **Always parallelize** the sampled-page fetches: single Bash message, multiple `bdata scrape` tool calls. Do not loop sequentially.
|
|
22
|
+
- **Always pass `--json`** when piping to `jq`.
|
|
23
|
+
- **For arbitrary websites, `bdata scrape` is correct** — there is no `bdata pipelines` scraper for generic-website SEO. Pre-built scrapers target specific platforms (Amazon, LinkedIn, etc.); skip them here.
|
|
24
|
+
- **Never re-fetch** the same URL twice in one audit.
|
|
25
|
+
|
|
26
|
+
## Failure Handling
|
|
27
|
+
|
|
28
|
+
- If `bdata scrape <url>` returns empty/error, note it as a finding (5xx, blocked, etc.) and continue. Don't abort.
|
|
29
|
+
- If `sitemap.xml` is 404, fall back to homepage + `<a href>` extraction (R-10) for URL discovery — degraded but functional.
|
|
30
|
+
- If `bdata` is missing or unauthenticated, stop immediately and point the user at the prerequisites section in SKILL.md and the `brightdata-cli` skill.
|
|
31
|
+
|
|
32
|
+
---
|
|
33
|
+
|
|
34
|
+
## Recipes
|
|
35
|
+
|
|
36
|
+
### R-01: Fetch robots.txt
|
|
37
|
+
|
|
38
|
+
**Purpose:** Detect `Disallow: /` on important paths and presence of `Sitemap:` reference.
|
|
39
|
+
|
|
40
|
+
**Command:**
|
|
41
|
+
```bash
|
|
42
|
+
bdata scrape https://example.com/robots.txt -f html
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
**Parsing:**
|
|
46
|
+
```bash
|
|
47
|
+
# Disallow lines
|
|
48
|
+
grep -E '^Disallow:' robots.txt
|
|
49
|
+
# Sitemap reference
|
|
50
|
+
grep -iE '^Sitemap:' robots.txt | awk '{print $2}'
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
**Evidence to cite:** Command + `Disallow:` lines verbatim.
|
|
54
|
+
|
|
55
|
+
---
|
|
56
|
+
|
|
57
|
+
### R-02: Fetch sitemap.xml
|
|
58
|
+
|
|
59
|
+
**Purpose:** Pull the URL inventory for stratified sampling.
|
|
60
|
+
|
|
61
|
+
**Command:**
|
|
62
|
+
```bash
|
|
63
|
+
bdata scrape https://example.com/sitemap.xml -f html
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
**Parsing:**
|
|
67
|
+
```bash
|
|
68
|
+
# Extract <loc> URLs
|
|
69
|
+
grep -oE '<loc>[^<]+</loc>' sitemap.xml | sed -E 's|</?loc>||g'
|
|
70
|
+
# If sitemap-index, recurse: each <loc> points to a sub-sitemap
|
|
71
|
+
# Hreflang entries (multilingual)
|
|
72
|
+
grep -oE '<xhtml:link[^/]+/>' sitemap.xml
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
If sitemap is a sitemap-index (top-level `<sitemapindex>`), fetch each child sitemap with R-02.
|
|
76
|
+
|
|
77
|
+
**Evidence to cite:** Command + total URL count + first 5 URLs as a sample.
|
|
78
|
+
|
|
79
|
+
---
|
|
80
|
+
|
|
81
|
+
### R-03: Stratified URL sampling
|
|
82
|
+
|
|
83
|
+
**Purpose:** Pick 10–15 representative URLs from sitemap output, grouped by path prefix.
|
|
84
|
+
|
|
85
|
+
**Parsing (no bdata call — pure local processing of R-02 output):**
|
|
86
|
+
```bash
|
|
87
|
+
# Extract first path segment from each URL
|
|
88
|
+
sed -E 's|^https?://[^/]+/||' urls.txt | awk -F/ '{print $1}' | sort | uniq -c | sort -rn
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
Decision rule:
|
|
92
|
+
1. Always include homepage (`/`).
|
|
93
|
+
2. For each path-prefix group with ≥3 URLs, sample 1 URL (or 2 if it's the dominant group, e.g., `/products/` on e-commerce).
|
|
94
|
+
3. Cap total at 10 pages by default. Document budget override.
|
|
95
|
+
4. If `<budget>` is exceeded by the per-group caps, drop smallest groups first.
|
|
96
|
+
|
|
97
|
+
**Evidence to cite:** "Sampled 12 URLs from sitemap of 847; groups: `/blog/` (3), `/products/` (5), `/category/` (2), `/` (1), `/about` (1)".
|
|
98
|
+
|
|
99
|
+
---
|
|
100
|
+
|
|
101
|
+
### R-04: Parallel page fetch
|
|
102
|
+
|
|
103
|
+
**Purpose:** Fetch all sampled URLs as rendered HTML in parallel.
|
|
104
|
+
|
|
105
|
+
**Command (issue all `bdata scrape` calls in a SINGLE Bash message via multiple tool calls — Claude must use multiple Bash invocations in one assistant turn):**
|
|
106
|
+
```bash
|
|
107
|
+
bdata scrape https://example.com/ -f html -o /tmp/seo/_home.html
|
|
108
|
+
bdata scrape https://example.com/products/widget -f html -o /tmp/seo/_p1.html
|
|
109
|
+
# ...one call per sampled URL
|
|
110
|
+
```
|
|
111
|
+
|
|
112
|
+
**Parsing:** Each output is rendered HTML for downstream recipes (R-05..R-11).
|
|
113
|
+
|
|
114
|
+
**Evidence to cite:** The specific `bdata scrape <url>` command per finding.
|
|
115
|
+
|
|
116
|
+
---
|
|
117
|
+
|
|
118
|
+
### R-05: Title + meta description + canonical extraction
|
|
119
|
+
|
|
120
|
+
**Purpose:** Per-page on-page basics.
|
|
121
|
+
|
|
122
|
+
**Command:** Parses output of R-04 (no extra `bdata` call).
|
|
123
|
+
|
|
124
|
+
**Parsing:**
|
|
125
|
+
```bash
|
|
126
|
+
# Title
|
|
127
|
+
grep -oE '<title>[^<]+</title>' page.html | sed -E 's|</?title>||g'
|
|
128
|
+
# Meta description
|
|
129
|
+
grep -oE '<meta[^>]+name="description"[^>]*>' page.html | grep -oE 'content="[^"]*"' | sed -E 's|content="(.*)"|\1|'
|
|
130
|
+
# Canonical
|
|
131
|
+
grep -oE '<link[^>]+rel="canonical"[^>]*>' page.html | grep -oE 'href="[^"]*"' | sed -E 's|href="(.*)"|\1|'
|
|
132
|
+
```
|
|
133
|
+
|
|
134
|
+
**Evidence to cite:** Command + the extracted title/description/canonical line verbatim.
|
|
135
|
+
|
|
136
|
+
---
|
|
137
|
+
|
|
138
|
+
### R-06: Heading structure extraction
|
|
139
|
+
|
|
140
|
+
**Purpose:** Count H1s and detect heading-level skips.
|
|
141
|
+
|
|
142
|
+
**Parsing:**
|
|
143
|
+
```bash
|
|
144
|
+
# Count each heading level (use grep -oE | wc -l, not grep -c — minified HTML is one line)
|
|
145
|
+
for h in 1 2 3 4 5 6; do
|
|
146
|
+
count=$(grep -oE "<h$h[ >]" page.html | wc -l)
|
|
147
|
+
echo "h$h: $count"
|
|
148
|
+
done
|
|
149
|
+
# Extract H1 text
|
|
150
|
+
grep -oE '<h1[^>]*>[^<]+</h1>' page.html | sed -E 's|</?h1[^>]*>||g'
|
|
151
|
+
```
|
|
152
|
+
|
|
153
|
+
Flag: if H1 count ≠ 1, or if any H3 appears before any H2, etc.
|
|
154
|
+
|
|
155
|
+
**Evidence to cite:** Command + the per-level counts and H1 text.
|
|
156
|
+
|
|
157
|
+
---
|
|
158
|
+
|
|
159
|
+
### R-07: Schema markup detection (JS-injected)
|
|
160
|
+
|
|
161
|
+
**Purpose:** The recipe that distinguishes this skill. Detect JSON-LD schema, including markup injected by client-side JS (Yoast, RankMath, AIOSEO, Next.js, etc.).
|
|
162
|
+
|
|
163
|
+
**Why it works:** `bdata scrape -f html` runs the page through Bright Data's rendering layer, so JS-injected `<script type="application/ld+json">` blocks are present in the output — unlike `web_fetch` or `curl`.
|
|
164
|
+
|
|
165
|
+
**Command:** Parses output of R-04.
|
|
166
|
+
|
|
167
|
+
**Parsing:** JSON-LD blocks are multi-line and may contain literal `<` characters in string values, so a simple `grep` will silently fail on either count. Use Python for reliable extraction:
|
|
168
|
+
```bash
|
|
169
|
+
python3 - <<'PY' page.html
|
|
170
|
+
import re, sys, json
|
|
171
|
+
html = open(sys.argv[1]).read()
|
|
172
|
+
# Find every <script type="application/ld+json">...</script> block (DOTALL so newlines are matched)
|
|
173
|
+
blocks = re.findall(
|
|
174
|
+
r'<script[^>]+type=["\']application/ld\+json["\'][^>]*>(.*?)</script>',
|
|
175
|
+
html, re.DOTALL | re.IGNORECASE,
|
|
176
|
+
)
|
|
177
|
+
for raw in blocks:
|
|
178
|
+
try:
|
|
179
|
+
obj = json.loads(raw.strip())
|
|
180
|
+
except json.JSONDecodeError as e:
|
|
181
|
+
print(f"[malformed JSON-LD] {e}")
|
|
182
|
+
continue
|
|
183
|
+
items = obj if isinstance(obj, list) else [obj]
|
|
184
|
+
for item in items:
|
|
185
|
+
t = item.get("@type", "(no @type)")
|
|
186
|
+
print(t if isinstance(t, str) else ", ".join(t))
|
|
187
|
+
PY
|
|
188
|
+
```
|
|
189
|
+
|
|
190
|
+
Output is one `@type` per line (e.g., `Product`, `BreadcrumbList`, `Organization`). Empty output means no JSON-LD blocks were found in the rendered HTML.
|
|
191
|
+
|
|
192
|
+
**Evidence to cite:** Command + the extracted `@type` list (e.g., `["Product", "BreadcrumbList"]`).
|
|
193
|
+
|
|
194
|
+
---
|
|
195
|
+
|
|
196
|
+
### R-08: Hreflang cluster extraction
|
|
197
|
+
|
|
198
|
+
**Purpose:** Validate hreflang self-reference, reciprocity, and valid codes for multilingual sites.
|
|
199
|
+
|
|
200
|
+
**Parsing:**
|
|
201
|
+
```bash
|
|
202
|
+
# From each page's HTML head
|
|
203
|
+
grep -oE '<link[^>]+rel="alternate"[^>]+hreflang="[^"]*"[^>]*>' page.html
|
|
204
|
+
# Extract hreflang code + href into pairs:
|
|
205
|
+
# - hreflang="en-GB" href="https://example.com/en-gb/about"
|
|
206
|
+
```
|
|
207
|
+
|
|
208
|
+
Validation rules (apply across the cluster from all sampled pages):
|
|
209
|
+
- **Self-reference:** Each page must list itself with its own hreflang.
|
|
210
|
+
- **Reciprocity:** If page A points to page B with `hreflang=fr`, page B must point back to A.
|
|
211
|
+
- **Valid codes:** ISO 639-1 lang + optional ISO 3166-1 Alpha-2 region. `en-UK` is invalid (use `en-GB`).
|
|
212
|
+
- **x-default:** Must be declared somewhere in the cluster.
|
|
213
|
+
|
|
214
|
+
**Evidence to cite:** Command + the malformed entry, e.g., `<link rel="alternate" hreflang="en-UK" href="..."/>` plus the rule it violates.
|
|
215
|
+
|
|
216
|
+
---
|
|
217
|
+
|
|
218
|
+
### R-09: Image alt audit
|
|
219
|
+
|
|
220
|
+
**Purpose:** Count `<img>` without `alt`.
|
|
221
|
+
|
|
222
|
+
**Parsing:** (Use `grep -oE … | wc -l` instead of `grep -c` — `-c` counts matching lines, which underreports on minified single-line HTML.)
|
|
223
|
+
```bash
|
|
224
|
+
total=$(grep -oE '<img[^>]*>' page.html | wc -l)
|
|
225
|
+
with_alt=$(grep -oE '<img[^>]+alt=' page.html | wc -l)
|
|
226
|
+
without=$((total - with_alt))
|
|
227
|
+
echo "$without/$total images missing alt"
|
|
228
|
+
```
|
|
229
|
+
|
|
230
|
+
**Evidence to cite:** Command + the ratio (e.g., "14/22 images missing alt on /products/widget").
|
|
231
|
+
|
|
232
|
+
---
|
|
233
|
+
|
|
234
|
+
### R-10: Internal-link map
|
|
235
|
+
|
|
236
|
+
**Purpose:** Build internal-link graph across sampled pages; detect orphan candidates and generic anchor text.
|
|
237
|
+
|
|
238
|
+
**Parsing:**
|
|
239
|
+
```bash
|
|
240
|
+
# Extract hrefs
|
|
241
|
+
grep -oE '<a[^>]+href="[^"]*"' page.html | grep -oE 'href="[^"]*"' | sed -E 's|href="(.*)"|\1|'
|
|
242
|
+
# Extract anchor text per link
|
|
243
|
+
grep -oE '<a[^>]+href="[^"]*"[^>]*>[^<]+</a>' page.html
|
|
244
|
+
```
|
|
245
|
+
|
|
246
|
+
Internal vs. external classification: if hostname matches the audit domain (or no hostname, i.e., relative), it's internal.
|
|
247
|
+
|
|
248
|
+
Generic anchor list: `click here`, `read more`, `learn more`, `here`, `link`.
|
|
249
|
+
|
|
250
|
+
**Evidence to cite:** Command + count of internal links, generic-anchor count, and the orphan-candidate list (URLs in sitemap but not linked from any sampled page).
|
|
251
|
+
|
|
252
|
+
---
|
|
253
|
+
|
|
254
|
+
### R-11: CWV-proxy signals
|
|
255
|
+
|
|
256
|
+
**Purpose:** HTML-level proxies for Core Web Vitals — always paired with "confirm with PSI" caveat in the finding.
|
|
257
|
+
|
|
258
|
+
**Parsing:** (POSIX `grep -E` does not support negative lookahead, so render-blocking and CLS-risk are computed by subtraction. All counts use `grep -oE … | wc -l` instead of `grep -c` because `-c` counts matching *lines*, not occurrences — a single-line minified HTML file would otherwise return 1 for everything.)
|
|
259
|
+
```bash
|
|
260
|
+
# Page weight (rendered HTML bytes)
|
|
261
|
+
wc -c < page.html
|
|
262
|
+
# Render-blocking scripts (synchronous + in <head>) = (all script-with-src in <head>) − (those with async or defer)
|
|
263
|
+
head_html=$(sed -n '/<head>/,/<\/head>/p' page.html)
|
|
264
|
+
total_head_scripts=$(echo "$head_html" | grep -oE '<script[^>]+src=' | wc -l)
|
|
265
|
+
async_or_defer=$(echo "$head_html" | grep -oE '<script[^>]+(async|defer)[^>]*src=|<script[^>]+src=[^>]*(async|defer)' | wc -l)
|
|
266
|
+
echo $((total_head_scripts - async_or_defer))
|
|
267
|
+
# Render-blocking stylesheets (no media="print")
|
|
268
|
+
echo "$head_html" | grep -oE '<link[^>]+rel="stylesheet"[^>]*>' | grep -vE 'media="print"' | wc -l
|
|
269
|
+
# Images without dimensions (CLS risk) = total <img> − those with width or height
|
|
270
|
+
total_imgs=$(grep -oE '<img[^>]*>' page.html | wc -l)
|
|
271
|
+
imgs_with_dims=$(grep -oE '<img[^>]*(width|height)=[^>]*>' page.html | wc -l)
|
|
272
|
+
echo $((total_imgs - imgs_with_dims))
|
|
273
|
+
# Lazy-loading count
|
|
274
|
+
grep -oE '<img[^>]+loading="lazy"' page.html | wc -l
|
|
275
|
+
# Resource hints count
|
|
276
|
+
grep -oE '<link[^>]+rel="(preconnect|preload|dns-prefetch)"' page.html | wc -l
|
|
277
|
+
```
|
|
278
|
+
|
|
279
|
+
**Evidence to cite:** Command + the metric (e.g., "page weight 4.2MB", "8 sync scripts in <head>", "14 images without width/height").
|
|
280
|
+
|
|
281
|
+
**Always include in the finding:** "Confirm with PageSpeed Insights at https://pagespeed.web.dev/?url=<url> for field-data CWV measurements."
|
|
282
|
+
|
|
283
|
+
---
|
|
284
|
+
|
|
285
|
+
### R-12: Indexation proxy via SERP
|
|
286
|
+
|
|
287
|
+
**Purpose:** Approximate count of URLs Google has indexed.
|
|
288
|
+
|
|
289
|
+
**Command:**
|
|
290
|
+
```bash
|
|
291
|
+
bdata search "site:example.com" --json
|
|
292
|
+
```
|
|
293
|
+
|
|
294
|
+
**Parsing:**
|
|
295
|
+
```bash
|
|
296
|
+
bdata search "site:example.com" --json | jq '.organic | length'
|
|
297
|
+
# Or to get the actual URLs:
|
|
298
|
+
bdata search "site:example.com" --json | jq -r '.organic[].link'
|
|
299
|
+
```
|
|
300
|
+
|
|
301
|
+
**Note:** `site:` queries return a sample, not exact counts. Compare to sitemap URL count from R-02 — flag if "12 indexed vs. 847 in sitemap" levels of mismatch.
|
|
302
|
+
|
|
303
|
+
**Evidence to cite:** Command + the count and a few sample indexed URLs.
|
|
304
|
+
|
|
305
|
+
---
|
|
306
|
+
|
|
307
|
+
### R-13: Ranking position check (signal-driven)
|
|
308
|
+
|
|
309
|
+
**Purpose:** Find user's domain in SERP for a target keyword. Run only when the user named a keyword.
|
|
310
|
+
|
|
311
|
+
**Command:**
|
|
312
|
+
```bash
|
|
313
|
+
bdata search "project management software" --json
|
|
314
|
+
```
|
|
315
|
+
|
|
316
|
+
**Parsing:**
|
|
317
|
+
```bash
|
|
318
|
+
bdata search "project management software" --json \
|
|
319
|
+
| jq '.organic | to_entries[] | select(.value.link | contains("example.com")) | {position: (.key + 1), url: .value.link}'
|
|
320
|
+
```
|
|
321
|
+
|
|
322
|
+
If no entry returned, the domain is not in the top results for that query.
|
|
323
|
+
|
|
324
|
+
**Evidence to cite:** Command + position (e.g., "ranks #18 with URL /pm-tool") or "not in top 100".
|
|
325
|
+
|
|
326
|
+
---
|
|
327
|
+
|
|
328
|
+
### R-14: Cannibalization check (signal-driven)
|
|
329
|
+
|
|
330
|
+
**Purpose:** Detect when multiple URLs from the user's domain rank for the same keyword.
|
|
331
|
+
|
|
332
|
+
**Command:**
|
|
333
|
+
```bash
|
|
334
|
+
bdata search "project management software site:example.com" --json
|
|
335
|
+
```
|
|
336
|
+
|
|
337
|
+
**Parsing:**
|
|
338
|
+
```bash
|
|
339
|
+
bdata search "project management software site:example.com" --json \
|
|
340
|
+
| jq -r '.organic[].link'
|
|
341
|
+
```
|
|
342
|
+
|
|
343
|
+
If 2+ URLs returned, those are cannibalization candidates.
|
|
344
|
+
|
|
345
|
+
**Evidence to cite:** Command + the URL list.
|
|
346
|
+
|
|
347
|
+
---
|
|
348
|
+
|
|
349
|
+
### R-15: Site-type detection
|
|
350
|
+
|
|
351
|
+
**Purpose:** Classify the site so the right playbook from `site-type-playbooks.md` applies.
|
|
352
|
+
|
|
353
|
+
**Inputs:** Output of R-04 for the homepage + URL list from R-02.
|
|
354
|
+
|
|
355
|
+
**Heuristics:**
|
|
356
|
+
| Site type | Cue |
|
|
357
|
+
|---|---|
|
|
358
|
+
| SaaS / Product | `/pricing` URL exists; "free trial" / "sign up" CTAs in nav; description mentions software/platform/tool; JSON-LD `@type: SoftwareApplication` |
|
|
359
|
+
| E-commerce | `/products/` or `/shop/` paths in sitemap; `Schema.org Product` JSON-LD; "Add to cart" buttons; `?color=` / `?size=` parameter patterns |
|
|
360
|
+
| Content / Blog | `/blog/` or `/posts/` paths dominate sitemap; `<meta name="generator" content="WordPress">` (or similar); `<article>` with bylines |
|
|
361
|
+
| Local Business | NAP in homepage footer; `Schema.org LocalBusiness` JSON-LD; Google Maps embed; multiple `/locations/<city>` |
|
|
362
|
+
| Multilingual | 2+ `<link rel="alternate" hreflang>` in homepage `<head>`; locale prefixes in sitemap (`/en/`, `/de/`, `/ar/`); non-default `<html lang>` |
|
|
363
|
+
|
|
364
|
+
A site can match multiple types — apply all matching playbooks.
|
|
365
|
+
|
|
366
|
+
**Evidence to cite:** Per-type, the cue that matched (e.g., "SaaS — `/pricing` page exists, JSON-LD `@type: SoftwareApplication` detected on homepage").
|
|
367
|
+
|
|
368
|
+
---
|
|
369
|
+
|
|
370
|
+
### R-16: Faceted-nav parameter detection (e-commerce)
|
|
371
|
+
|
|
372
|
+
**Purpose:** Detect indexable faceted-navigation URLs that risk crawl-budget waste.
|
|
373
|
+
|
|
374
|
+
**Parsing:**
|
|
375
|
+
```bash
|
|
376
|
+
# From R-02 sitemap URL list
|
|
377
|
+
grep -E '\?' urls.txt | head -20
|
|
378
|
+
grep -cE '\?' urls.txt
|
|
379
|
+
```
|
|
380
|
+
|
|
381
|
+
If many `?` URLs exist in the sitemap, flag as crawl-budget risk.
|
|
382
|
+
|
|
383
|
+
**Evidence to cite:** Command + count + 3-5 example URLs.
|
|
384
|
+
|
|
385
|
+
---
|
|
386
|
+
|
|
387
|
+
### R-17: Out-of-stock product handling (e-commerce)
|
|
388
|
+
|
|
389
|
+
**Purpose:** Detect product pages that are out-of-stock but still return 200 with no canonical handling.
|
|
390
|
+
|
|
391
|
+
**Parsing (per sampled product page):**
|
|
392
|
+
```bash
|
|
393
|
+
# Look for "out of stock" / "sold out" / "unavailable" markers in rendered HTML
|
|
394
|
+
grep -iE 'out of stock|sold out|currently unavailable' page.html
|
|
395
|
+
# Check Schema.org availability (JS-rendered, parseable from R-04 output)
|
|
396
|
+
grep -oE '"availability"[^,}]*' page.html
|
|
397
|
+
# Check canonical destination — does it point to itself, or to a parent category?
|
|
398
|
+
grep -oE '<link[^>]+rel="canonical"[^>]+href="[^"]+"' page.html
|
|
399
|
+
```
|
|
400
|
+
|
|
401
|
+
Flag: page contains an out-of-stock signal, has `Schema.org availability: OutOfStock` (or no availability schema), and self-canonicals (rather than canonicaling to a parent category or returning 404). If we need to confirm the HTTP status code, use `curl -o /dev/null -s -w "%{http_code}\n" "<url>"` rather than relying on `bdata scrape -f json` shape — the JSON-mode output schema is not documented for this purpose.
|
|
402
|
+
|
|
403
|
+
**Evidence to cite:** Command + the out-of-stock text + missing canonical.
|
|
404
|
+
|
|
405
|
+
---
|
|
406
|
+
|
|
407
|
+
### R-18: Author-page reachability (blog)
|
|
408
|
+
|
|
409
|
+
**Purpose:** Find broken `/author/<x>` links from blog posts.
|
|
410
|
+
|
|
411
|
+
**Parsing:**
|
|
412
|
+
```bash
|
|
413
|
+
# From sampled blog posts
|
|
414
|
+
grep -oE '<a[^>]+href="[^"]*author[^"]*"' post.html | grep -oE 'href="[^"]*"' | sort -u
|
|
415
|
+
# For each unique author URL, fetch:
|
|
416
|
+
bdata scrape "https://example.com/author/jane" -f html
|
|
417
|
+
```
|
|
418
|
+
|
|
419
|
+
Flag if any author URL returns 404 or is blocked.
|
|
420
|
+
|
|
421
|
+
**Evidence to cite:** Command + the broken author URL.
|
|
422
|
+
|
|
423
|
+
---
|
|
424
|
+
|
|
425
|
+
### R-19: Publication-date / freshness extraction (blog)
|
|
426
|
+
|
|
427
|
+
**Purpose:** Identify outdated content (>18 months without update).
|
|
428
|
+
|
|
429
|
+
**Parsing:**
|
|
430
|
+
```bash
|
|
431
|
+
# datetime in <time> tags
|
|
432
|
+
grep -oE '<time[^>]+datetime="[^"]*"' post.html
|
|
433
|
+
# Modified date (Schema.org Article)
|
|
434
|
+
grep -oE '"dateModified"[^,}]*' post.html
|
|
435
|
+
# Updated date in visible text (fallback)
|
|
436
|
+
grep -iE 'last updated|updated on' post.html
|
|
437
|
+
```
|
|
438
|
+
|
|
439
|
+
Flag posts with `datetime` >18 months ago and no visible update marker.
|
|
440
|
+
|
|
441
|
+
**Evidence to cite:** Command + the date + how long ago it is.
|
|
442
|
+
|
|
443
|
+
---
|
|
444
|
+
|
|
445
|
+
### R-20: NAP extraction (local business)
|
|
446
|
+
|
|
447
|
+
**Purpose:** Extract Name / Address / Phone from each sampled page footer to detect inconsistency.
|
|
448
|
+
|
|
449
|
+
**Parsing:**
|
|
450
|
+
```bash
|
|
451
|
+
# Phone (US formats)
|
|
452
|
+
grep -oE '\(?\b[0-9]{3}\)?[-. ]?[0-9]{3}[-. ]?[0-9]{4}\b' page.html | sort -u
|
|
453
|
+
# Address (rough — match street suffixes)
|
|
454
|
+
grep -oE '[0-9]+ [A-Z][a-z]+( [A-Z][a-z]+)* (St|Ave|Blvd|Rd|Way|Dr|Ln)\b[^<]*' page.html | sort -u
|
|
455
|
+
# Or pull from LocalBusiness JSON-LD if present
|
|
456
|
+
echo "$JSONLD" | jq -r '.address, .telephone'
|
|
457
|
+
```
|
|
458
|
+
|
|
459
|
+
Normalize phone (digits only) before comparing across pages. Flag any mismatch as a finding.
|
|
460
|
+
|
|
461
|
+
**Evidence to cite:** Command + the conflicting values per page.
|
|
462
|
+
|
|
463
|
+
---
|
|
464
|
+
|
|
465
|
+
### R-21: Local SERP visibility (local business)
|
|
466
|
+
|
|
467
|
+
**Purpose:** Run `bdata search "<service> <city>"` per location to validate local SERP presence.
|
|
468
|
+
|
|
469
|
+
**Command:**
|
|
470
|
+
```bash
|
|
471
|
+
bdata search "plumber Austin" --country us --json
|
|
472
|
+
```
|
|
473
|
+
|
|
474
|
+
**Parsing:** Same as R-13 — find user's domain in `.organic[].link`.
|
|
475
|
+
|
|
476
|
+
**Evidence to cite:** Command + position per city.
|
|
477
|
+
|
|
478
|
+
---
|
|
479
|
+
|
|
480
|
+
### R-22: Outbound-link sampling (E-E-A-T)
|
|
481
|
+
|
|
482
|
+
**Purpose:** Quick check that content links to authoritative sources rather than orphan pages with no citations.
|
|
483
|
+
|
|
484
|
+
**Parsing:**
|
|
485
|
+
```bash
|
|
486
|
+
grep -oE '<a[^>]+href="https?://[^"]*"' page.html \
|
|
487
|
+
| grep -oE 'href="[^"]*"' \
|
|
488
|
+
| grep -vE 'href="https?://(www\.)?example\.com' \
|
|
489
|
+
| sort -u | head
|
|
490
|
+
```
|
|
491
|
+
|
|
492
|
+
Flag: low outbound-link count + topic that demands citations (medical, legal, financial).
|
|
493
|
+
|
|
494
|
+
**Evidence to cite:** Command + outbound-link count.
|
|
495
|
+
|
|
496
|
+
---
|
|
497
|
+
|
|
498
|
+
### R-23: Author byline + bio detection (E-E-A-T)
|
|
499
|
+
|
|
500
|
+
**Purpose:** Detect missing or thin author bylines on blog content.
|
|
501
|
+
|
|
502
|
+
**Parsing:**
|
|
503
|
+
```bash
|
|
504
|
+
# Schema.org Article author
|
|
505
|
+
echo "$JSONLD" | jq -r '.author.name // .author[0].name'
|
|
506
|
+
# Visible byline (rough)
|
|
507
|
+
grep -iE 'by <[^>]+>|class="author"|rel="author"' post.html
|
|
508
|
+
```
|
|
509
|
+
|
|
510
|
+
Flag posts with no detectable author.
|
|
511
|
+
|
|
512
|
+
**Evidence to cite:** Command + presence/absence.
|
|
513
|
+
|
|
514
|
+
---
|
|
515
|
+
|
|
516
|
+
### R-24: Word-count + paragraph-count extraction
|
|
517
|
+
|
|
518
|
+
**Purpose:** Detect thin content.
|
|
519
|
+
|
|
520
|
+
**Parsing:**
|
|
521
|
+
```bash
|
|
522
|
+
# Strip HTML tags, count words
|
|
523
|
+
sed -E 's|<[^>]+>||g' page.html | wc -w
|
|
524
|
+
# Paragraph count
|
|
525
|
+
grep -oE '<p[ >]' page.html | wc -l
|
|
526
|
+
```
|
|
527
|
+
|
|
528
|
+
Thresholds (rough): <300 words is thin for a content page; <100 words is thin even for a product page.
|
|
529
|
+
|
|
530
|
+
**Evidence to cite:** Command + count.
|
|
531
|
+
|
|
532
|
+
---
|
|
533
|
+
|
|
534
|
+
### R-25: Generic anchor-text density
|
|
535
|
+
|
|
536
|
+
**Purpose:** Flag over-reliance on generic anchor text in internal links.
|
|
537
|
+
|
|
538
|
+
**Parsing:**
|
|
539
|
+
```bash
|
|
540
|
+
# From R-10 anchor extraction. Trim whitespace and use grep -oE | wc -l (not -c, to avoid line-counting on minified HTML).
|
|
541
|
+
generic=$(grep -oE '<a[^>]+href="[^"]*"[^>]*>[^<]+</a>' page.html \
|
|
542
|
+
| sed -E 's|.*>([^<]+)</a>|\1|' \
|
|
543
|
+
| sed -E 's/^[[:space:]]+//; s/[[:space:]]+$//' \
|
|
544
|
+
| grep -ciE '^(click here|read more|learn more|here|link|this|more)$')
|
|
545
|
+
total=$(grep -oE '<a[^>]+href="[^"]*"' page.html | wc -l)
|
|
546
|
+
echo "$generic/$total internal anchors are generic"
|
|
547
|
+
```
|
|
548
|
+
|
|
549
|
+
Flag if generic-anchor ratio is > 30% of internal-link count.
|
|
550
|
+
|
|
551
|
+
**Evidence to cite:** Command + ratio.
|
|
@@ -0,0 +1,107 @@
|
|
|
1
|
+
# Output Templates — SEO Audit
|
|
2
|
+
|
|
3
|
+
This file defines the exact report structure every SEO audit must produce. The audit-framework.md and site-type-playbooks.md generate findings; this file specifies how to present them.
|
|
4
|
+
|
|
5
|
+
## Top-Level Report Structure
|
|
6
|
+
|
|
7
|
+
```markdown
|
|
8
|
+
# SEO Audit — <domain or URL>
|
|
9
|
+
|
|
10
|
+
## Executive Summary
|
|
11
|
+
- **Overall health**: <Critical | Poor | Fair | Good | Excellent>
|
|
12
|
+
- **Top priorities**:
|
|
13
|
+
1. <Highest-impact issue, one sentence>
|
|
14
|
+
2. <…>
|
|
15
|
+
3. <…>
|
|
16
|
+
- **Quick wins**: <2–4 easy/immediate fixes>
|
|
17
|
+
- **Audit scope**: <N pages sampled from sitemap of M URLs | single-page audit of <url>>
|
|
18
|
+
|
|
19
|
+
## Technical SEO Findings
|
|
20
|
+
<one finding block per issue, ordered by Priority (1 first); if no findings, write "No issues found in this area.">
|
|
21
|
+
|
|
22
|
+
## On-Page SEO Findings
|
|
23
|
+
<one finding block per issue, ordered by Priority (1 first); if no findings, write "No issues found in this area.">
|
|
24
|
+
|
|
25
|
+
## Content Findings
|
|
26
|
+
<one finding block per issue, ordered by Priority (1 first); if no findings, write "No issues found in this area.">
|
|
27
|
+
|
|
28
|
+
## Out-of-Scope Notes
|
|
29
|
+
<things to check elsewhere — PSI, GSC, Ahrefs>
|
|
30
|
+
|
|
31
|
+
## Prioritized Action Plan
|
|
32
|
+
1. **Critical** (blocking indexation/ranking): …
|
|
33
|
+
2. **High-impact** improvements: …
|
|
34
|
+
3. **Quick wins** (easy, immediate benefit): …
|
|
35
|
+
4. **Long-term** recommendations: …
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
## Finding Block Shape
|
|
39
|
+
|
|
40
|
+
Every finding under `Technical SEO Findings`, `On-Page SEO Findings`, and `Content Findings` uses this exact shape:
|
|
41
|
+
|
|
42
|
+
````markdown
|
|
43
|
+
### <Short issue title>
|
|
44
|
+
- **Issue**: <What's wrong, 1–2 sentences>
|
|
45
|
+
- **Impact**: <High | Medium | Low> — <one sentence on why this matters for SEO>
|
|
46
|
+
- **Evidence**:
|
|
47
|
+
- Command: `bdata scrape https://example.com -f html`
|
|
48
|
+
- Output excerpt:
|
|
49
|
+
```
|
|
50
|
+
<meta name="robots" content="noindex">
|
|
51
|
+
```
|
|
52
|
+
- (or, for a SERP-based finding) Command: `bdata search "site:example.com" --json | jq '.organic | length'` → 12 (sitemap has 847 URLs)
|
|
53
|
+
- (or, for a cannibalization finding) Command: `bdata search "<keyword> site:example.com" --json | jq -r '.organic[].link'` → returned 3 URLs
|
|
54
|
+
- **Fix**: <Specific, actionable recommendation>
|
|
55
|
+
- **Priority**: <1 (critical) | 2 | 3 | 4 | 5 (nice-to-have)>
|
|
56
|
+
````
|
|
57
|
+
|
|
58
|
+
`Impact` and `Priority` are orthogonal: Impact rates the SEO ranking effect (High/Medium/Low); Priority weights urgency factoring in fix effort. A finding can be Impact=High, Priority=2 (severe ranking effect, but moderate effort to fix) or Impact=Medium, Priority=1 (only moderate ranking effect, but trivially fixable and blocking other work).
|
|
59
|
+
|
|
60
|
+
This shape matches the inspiration skill's report contract verbatim — every finding is Issue / Impact / Evidence / Fix / Priority — so existing SEO-audit evals port without rewriting. The Evidence field is the one place we improve: every finding cites the exact `bdata` command that produced the evidence and an output excerpt the user can verify by re-running the command, which is uniquely possible because of the CLI's deterministic, scriptable output.
|
|
61
|
+
|
|
62
|
+
## Out-of-Scope Notes Section
|
|
63
|
+
|
|
64
|
+
Anything `bdata` cannot directly measure goes here, with explicit pointers:
|
|
65
|
+
|
|
66
|
+
- "Core Web Vitals field data → run PageSpeed Insights at https://pagespeed.web.dev/?url=<url>"
|
|
67
|
+
- "Index coverage detail → check Google Search Console > Pages report"
|
|
68
|
+
- "Backlink profile → use Ahrefs / Semrush / Moz"
|
|
69
|
+
- "JS-injected schema validation → already detected via `bdata scrape -f html`; cross-validate with Rich Results Test (https://search.google.com/test/rich-results)"
|
|
70
|
+
|
|
71
|
+
## Executive Summary Rubric
|
|
72
|
+
|
|
73
|
+
- **Top priorities**: 3–5 items, one sentence each, ordered by impact × ease-of-fix.
|
|
74
|
+
- **Quick wins**: findings rated Priority ≥ 3 (Medium severity or lower, since lower-severity items take less effort to ship) AND classified as Low effort.
|
|
75
|
+
- **Overall health**:
|
|
76
|
+
- **Critical** — site uncrawlable or unindexed (Tier-1 failure).
|
|
77
|
+
- **Poor** — multiple Tier-1 or Tier-2 issues.
|
|
78
|
+
- **Fair** — Tier-3 issues dominate.
|
|
79
|
+
- **Good** — only Tier-4 / quick-win issues remain.
|
|
80
|
+
- **Excellent** — nothing material found.
|
|
81
|
+
|
|
82
|
+
## Mode-A vs. Mode-B Rendering
|
|
83
|
+
|
|
84
|
+
- **Mode A (single-page audit)**: same template, "Audit scope" line says "single-page audit of <url>"; Findings sections are typically shorter; site-type playbook may not apply.
|
|
85
|
+
- **Mode B (site-wide audit)**: full template; Findings include site-wide issues — cannibalization, duplicate titles across sample, hreflang clusters, internal-link orphans.
|
|
86
|
+
|
|
87
|
+
## Priority Numbering
|
|
88
|
+
|
|
89
|
+
| Priority | Meaning |
|
|
90
|
+
|---|---|
|
|
91
|
+
| 1 | Critical — blocks indexation or ranking. Fix immediately. |
|
|
92
|
+
| 2 | High — significant ranking impact within weeks. |
|
|
93
|
+
| 3 | Medium — meaningful improvement; schedule. |
|
|
94
|
+
| 4 | Low — minor polish. |
|
|
95
|
+
| 5 | Nice-to-have — may improve over time. |
|
|
96
|
+
|
|
97
|
+
### Mapping Priority levels to Action Plan buckets
|
|
98
|
+
|
|
99
|
+
When generating the Prioritized Action Plan section, group findings by Priority:
|
|
100
|
+
|
|
101
|
+
| Priority | Action Plan bucket |
|
|
102
|
+
|---|---|
|
|
103
|
+
| 1 | Critical (blocking indexation/ranking) |
|
|
104
|
+
| 2 | High-impact improvements |
|
|
105
|
+
| 3 | Quick wins (when also Low effort) — otherwise High-impact improvements |
|
|
106
|
+
| 4 | Long-term recommendations |
|
|
107
|
+
| 5 | Long-term recommendations |
|