@awesomate/hosting-mcp 0.16.0 → 0.16.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@awesomate/hosting-mcp",
|
|
3
|
-
"version": "0.16.
|
|
3
|
+
"version": "0.16.1",
|
|
4
4
|
"description": "Awesomate MCP server \u2014 lets Claude manage your Awesomate WordPress hosting, plan, limits, n8n automations, and build Node/static apps + databases",
|
|
5
5
|
"license": "UNLICENSED",
|
|
6
6
|
"type": "module",
|
|
@@ -40,6 +40,9 @@ formats + length limits. In short, per site:
|
|
|
40
40
|
5. **`llms.txt`** at the root — a short, plain-markdown summary of what the
|
|
41
41
|
site/business is and its key pages, so AI assistants can understand and cite
|
|
42
42
|
it. (This is the general, non-shop counterpart to the Woo `llms.txt`.)
|
|
43
|
+
Be honest about what it does: Google ignores it — it serves the *other*
|
|
44
|
+
assistants. For Google's AI features, normal SEO is the whole strategy.
|
|
45
|
+
Details + crawler/rendering gotchas: [references/ai-discoverability.md](references/ai-discoverability.md).
|
|
43
46
|
|
|
44
47
|
Then commit + deploy (awesomate-github). On content changes, update the
|
|
45
48
|
affected `<head>` tags, `sitemap.xml` `<lastmod>`, and `llms.txt`.
|
|
@@ -61,6 +64,13 @@ Through the awesomate-hosting skill's `wp` passthrough:
|
|
|
61
64
|
- Each key page has a unique title + description and valid JSON-LD (no syntax
|
|
62
65
|
errors — a broken `application/ld+json` block is worse than none).
|
|
63
66
|
- `llms.txt` exists and is accurate.
|
|
67
|
+
- **Content is in the raw HTML** — `curl -s https://DOMAIN/page` shows the
|
|
68
|
+
actual content, not an empty `<div id="root">`. AI crawlers don't run
|
|
69
|
+
JavaScript; a client-rendered page is invisible to them.
|
|
70
|
+
- **AI crawlers aren't blocked at the edge** — a `curl` with a GPTBot /
|
|
71
|
+
ClaudeBot user-agent returns 200, not a 403/Cloudflare challenge (Cloudflare
|
|
72
|
+
blocks many AI bots by default; robots.txt can't fix that). Commands + where
|
|
73
|
+
to fix each case: [references/ai-discoverability.md](references/ai-discoverability.md).
|
|
64
74
|
Tell the user the next step is submitting the sitemap in Google Search Console
|
|
65
75
|
(you can't do that for them) — offer a one-paragraph how-to if they want it.
|
|
66
76
|
|
|
@@ -0,0 +1,109 @@
|
|
|
1
|
+
# AI discoverability — crawlers, rendering, llms.txt
|
|
2
|
+
|
|
3
|
+
How AI assistants (ChatGPT, Claude, Perplexity, Gemini, Copilot) find and cite
|
|
4
|
+
a site — and the two silent failure modes that make a "perfectly SEO'd" site
|
|
5
|
+
invisible to them anyway. Crawler/rendering tables adapted from
|
|
6
|
+
[claude-blog](https://github.com/AgriciDaniel/claude-blog) (MIT © AgriciDaniel).
|
|
7
|
+
|
|
8
|
+
## Google needs nothing special — say so when asked
|
|
9
|
+
|
|
10
|
+
Google's official guidance
|
|
11
|
+
(https://developers.google.com/search/docs/fundamentals/ai-optimization-guide):
|
|
12
|
+
AI Overviews and AI Mode run on Google's normal Search ranking systems. No
|
|
13
|
+
special markup, no AI schema, no llms.txt — **Google Search ignores llms.txt
|
|
14
|
+
entirely**. Normal SEO (the main checklist) *is* the Google-AI strategy. If
|
|
15
|
+
the user asks about "GEO"/"AEO" tactics or paid AI-optimization services,
|
|
16
|
+
this is the honest answer: for Google it's just SEO; for other assistants
|
|
17
|
+
the extras below help.
|
|
18
|
+
|
|
19
|
+
llms.txt still earns its place for **non-Google** assistants — keep writing
|
|
20
|
+
it, but present it as that, never as a Google requirement.
|
|
21
|
+
|
|
22
|
+
## Three kinds of AI bot — three different decisions
|
|
23
|
+
|
|
24
|
+
| Kind | Bots | Blocking means |
|
|
25
|
+
|---|---|---|
|
|
26
|
+
| Search-indexing | OAI-SearchBot, Claude-SearchBot, PerplexityBot | The site can't be cited in that assistant's answers. **Allow these.** |
|
|
27
|
+
| Training | GPTBot, ClaudeBot, Google-Extended, CCBot, Applebot-Extended, Meta-ExternalAgent | Owner's policy choice about model training; doesn't remove the site from answers |
|
|
28
|
+
| User-triggered retrieval | ChatGPT-User, Perplexity-User, Claude-User | Fetches when a live user asks; may not fully respect robots.txt |
|
|
29
|
+
|
|
30
|
+
The default `User-agent: * / Allow: /` robots.txt already allows all of them —
|
|
31
|
+
no explicit allowlist needed. Explicit stanzas only matter when the owner
|
|
32
|
+
wants to opt out of **training** while staying visible in answers:
|
|
33
|
+
|
|
34
|
+
```
|
|
35
|
+
# Stay citable, opt out of model training
|
|
36
|
+
User-agent: GPTBot
|
|
37
|
+
Disallow: /
|
|
38
|
+
|
|
39
|
+
User-agent: Google-Extended
|
|
40
|
+
Disallow: /
|
|
41
|
+
|
|
42
|
+
User-agent: CCBot
|
|
43
|
+
Disallow: /
|
|
44
|
+
|
|
45
|
+
User-agent: *
|
|
46
|
+
Allow: /
|
|
47
|
+
Sitemap: https://DOMAIN/sitemap.xml
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
Never block the search-indexing bots "for safety" — that silently removes the
|
|
51
|
+
site from ChatGPT/Claude/Perplexity answers.
|
|
52
|
+
|
|
53
|
+
Note for Awesomate-managed domains: the CDN prepends a managed section to the
|
|
54
|
+
site's robots.txt (a Content Signals preamble plus disallows for the
|
|
55
|
+
*training*-bot family). Search-indexing bots stay allowed, so the site remains
|
|
56
|
+
citable — but the served robots.txt is the site file *plus* that preamble.
|
|
57
|
+
Always check the live `https://DOMAIN/robots.txt`, not just the file you wrote.
|
|
58
|
+
|
|
59
|
+
## robots.txt allows it — but the CDN can still 403 it
|
|
60
|
+
|
|
61
|
+
Cloudflare **blocks many AI crawlers by default on new zones** (GPTBot,
|
|
62
|
+
ClaudeBot, PerplexityBot, CCBot, Google-Extended). The crawler gets a 403 or a
|
|
63
|
+
challenge page before it ever reads robots.txt, so a perfect robots.txt proves
|
|
64
|
+
nothing. Always verify from the outside:
|
|
65
|
+
|
|
66
|
+
```bash
|
|
67
|
+
curl -s -o /dev/null -w "%{http_code}\n" -A "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GPTBot/1.0; +https://openai.com/gptbot)" https://DOMAIN/
|
|
68
|
+
curl -s -o /dev/null -w "%{http_code}\n" -A "Mozilla/5.0 (compatible; ClaudeBot/1.0)" https://DOMAIN/
|
|
69
|
+
curl -s -o /dev/null -w "%{http_code}\n" -A "Mozilla/5.0 (compatible; PerplexityBot/1.0)" https://DOMAIN/
|
|
70
|
+
```
|
|
71
|
+
|
|
72
|
+
200 = reachable. 403, 503, or an HTML page mentioning Cloudflare = blocked at
|
|
73
|
+
the edge. Where to fix it:
|
|
74
|
+
|
|
75
|
+
- **Awesomate-managed domain** (`*.awesomate.site` etc.): the zone is managed
|
|
76
|
+
by Awesomate — the user can't change it. Raise it via `awesomate_support`
|
|
77
|
+
so the zone's AI Crawl Control settings get reviewed.
|
|
78
|
+
- **Custom domain on the user's own Cloudflare account**: they fix it
|
|
79
|
+
themselves in the Cloudflare dashboard → Security → Bots → AI Crawlers.
|
|
80
|
+
|
|
81
|
+
## AI crawlers do not run JavaScript
|
|
82
|
+
|
|
83
|
+
Only Googlebot renders JS. GPTBot, ClaudeBot, PerplexityBot, and the other AI
|
|
84
|
+
crawlers read the **raw HTML response**. A client-rendered app (React/Vue SPA)
|
|
85
|
+
is invisible to every AI assistant regardless of its meta tags and llms.txt.
|
|
86
|
+
|
|
87
|
+
Red flags — check the raw response, not the browser:
|
|
88
|
+
|
|
89
|
+
```bash
|
|
90
|
+
curl -s https://DOMAIN/page | head -100
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
- Body is essentially an empty `<div id="root">` / `<div id="app">` /
|
|
94
|
+
`<div id="__next">`
|
|
95
|
+
- Full HTML under ~5KB for a content page
|
|
96
|
+
- The content exists only inside `<script>` tags as JSON
|
|
97
|
+
|
|
98
|
+
Fix: keep public, indexable content in static or server-rendered HTML — for
|
|
99
|
+
app projects, static marketing/landing pages in front of the app is usually
|
|
100
|
+
enough. Don't rebuild a working app over this, but never claim AI
|
|
101
|
+
discoverability for client-rendered content.
|
|
102
|
+
|
|
103
|
+
## llms.txt rules
|
|
104
|
+
|
|
105
|
+
- Optional inventory for non-Google assistants — Google ignores it.
|
|
106
|
+
- Under 10KB (assistants may truncate or skip larger files).
|
|
107
|
+
- Markdown links: `[Title](URL): one-line description`. Only real,
|
|
108
|
+
high-value pages; only true facts.
|
|
109
|
+
- Supplements sitemap.xml, never replaces it. Update both when content changes.
|
|
@@ -47,10 +47,32 @@ Pick the type that fits; validate before shipping.
|
|
|
47
47
|
```
|
|
48
48
|
Common types: `WebSite` (+ `SearchAction`), `Organization` / `LocalBusiness`
|
|
49
49
|
(add `address`, `telephone`, `openingHours`), `Product` (`offers`, `brand`,
|
|
50
|
-
`gtin`), `Article` (`headline`, `datePublished`, `author`)
|
|
50
|
+
`gtin`), `Article` (`headline`, `datePublished`, `author`).
|
|
51
|
+
|
|
52
|
+
**Don't add `FAQPage` or `HowTo` expecting search features** — Google retired
|
|
53
|
+
FAQ rich results for all sites on 2026-05-07 and HowTo earlier. Both remain
|
|
54
|
+
valid schema.org markup, but they earn nothing; only add them if the visible
|
|
55
|
+
FAQ itself is useful.
|
|
56
|
+
|
|
57
|
+
Multiple entities on one page: combine them in ONE script tag using `@graph`,
|
|
58
|
+
with stable `@id` fragments cross-referencing instead of duplicating:
|
|
59
|
+
```json
|
|
60
|
+
{
|
|
61
|
+
"@context": "https://schema.org",
|
|
62
|
+
"@graph": [
|
|
63
|
+
{ "@type": "Organization", "@id": "https://DOMAIN/#organization", "name": "Brand", "url": "https://DOMAIN/", "logo": "https://DOMAIN/logo.png" },
|
|
64
|
+
{ "@type": "Article", "@id": "https://DOMAIN/PATH#article", "headline": "…", "datePublished": "YYYY-MM-DD", "publisher": { "@id": "https://DOMAIN/#organization" } }
|
|
65
|
+
]
|
|
66
|
+
}
|
|
67
|
+
```
|
|
68
|
+
Keep `@id` values stable across rebuilds (URL + `#fragment`, never random).
|
|
69
|
+
Validate at https://validator.schema.org (schema validity); use Google's Rich
|
|
70
|
+
Results Test only for types Google actually supports as rich results.
|
|
51
71
|
|
|
52
72
|
## `llms.txt` (site root — AI discoverability)
|
|
53
|
-
Plain markdown so assistants can read + cite the site. Keep it current
|
|
73
|
+
Plain markdown so assistants can read + cite the site. Keep it current and
|
|
74
|
+
under 10KB. Honest framing: **Google ignores llms.txt** — it serves the other
|
|
75
|
+
AI assistants (see [ai-discoverability.md](ai-discoverability.md)).
|
|
54
76
|
```
|
|
55
77
|
# Brand — what we do
|
|
56
78
|
|