awwwards-mcp 1.3.0 → 1.4.0

package/README.md

@@ -9,12 +9,16 @@ sourced from the web's best award-winning websites.
 Your agent searches in natural language ("dark 3D portfolio sites", "soft pastel
 e-commerce"), sees **real screenshots inline**, and can pull the **design DNA**
 of any site: color palette, tech stack, design elements, award history.
+Free-text queries run on a porter-stemmed, prefix-matching FTS5 index with BM25
+ranking — "magazines" now finds Magazine-tagged sites (68 on the live index),
+best matches first, where the old substring path returned zero. Multi-word
+queries keep AND semantics: every token must hit the same site.
 
 ## Tools
 
 | Tool | What it does |
 |------|--------------|
-| `search_sites` | Search by color, tags, technology or award type. Returns site cards with inline screenshots. |
+| `search_sites` | Search by color, tags, technology, award type or free-text query. Multi-word queries match against the local FTS5 index and rank BM25 (title hits lead); zero results come with loose-match and taxonomy-tag hints. Returns site cards with inline screenshots. |
 | `get_site_details` | Full design DNA for one site: palette, technologies, elements, awards, description. |
 | `get_site_elements` | Component-level visuals for one site: each element's poster image inline (3D models, video content, mobile layouts, microcopy…) + video URLs. |
 | `list_categories` | Every filter the agent can search by (200+ tags, 27 colors). |

package/dist/cache.js

@@ -17,6 +17,28 @@ const SCHEMA = `
     key TEXT PRIMARY KEY, value TEXT, fetchedAt INTEGER
   );
 `;
+// FTS5 full-text layer over the sites table (derived — rebuildable at any
+// time). Probe-guarded: if this Node build ships without FTS5, every fts
+// statement is skipped and searchSites returns null (server keeps the legacy
+// substring path). Columns mirror sites; tags/awards stay JSON strings —
+// unicode61 tokenizes around brackets/quotes, so tokens extract cleanly.
+const FTS_SCHEMA = `
+  CREATE VIRTUAL TABLE IF NOT EXISTS sites_fts USING fts5(
+    slug UNINDEXED, title, tags, awards, tokenize='porter unicode61'
+  );
+  CREATE TRIGGER IF NOT EXISTS sites_fts_ai AFTER INSERT ON sites BEGIN
+    INSERT INTO sites_fts (slug, title, tags, awards)
+    VALUES (new.slug, new.title, new.tags, new.awards);
+  END;
+  CREATE TRIGGER IF NOT EXISTS sites_fts_au AFTER UPDATE OF title, tags, awards ON sites BEGIN
+    DELETE FROM sites_fts WHERE slug = new.slug;
+    INSERT INTO sites_fts (slug, title, tags, awards)
+    VALUES (new.slug, new.title, new.tags, new.awards);
+  END;
+  CREATE TRIGGER IF NOT EXISTS sites_fts_ad AFTER DELETE ON sites BEGIN
+    DELETE FROM sites_fts WHERE slug = old.slug;
+  END;
+`;
 function rowToSite(r) {
     return {
         slug: r.slug,
@@ -34,6 +56,12 @@ export class Cache {
     dbPath;
     now;
     imagesDir;
+    // FTS5 capability of this Node build, probed on the first DB open (the
+    // constructor's eager withDb). null = not yet probed; false = FTS5 compiled
+    // out → searchSites returns null and callers keep the legacy substring
+    // path. Instance-cached so searchSites never re-probes; public so the
+    // server layer and tests can branch on it.
+    ftsAvailable = null;
     constructor(rootDir, now = Date.now) {
         this.now = now;
         mkdirSync(rootDir, { recursive: true });
@@ -52,12 +80,40 @@ export class Cache {
         const db = new DatabaseSync(this.dbPath);
         try {
             db.exec(SCHEMA);
+            if (this.ftsAvailable === null) {
+                try {
+                    db.exec(FTS_SCHEMA);
+                    this.ftsAvailable = true;
+                }
+                catch {
+                    this.ftsAvailable = false; // FTS5 compiled out → legacy fallback
+                }
+            }
+            if (this.ftsAvailable) {
+                // The fts table is derived and must never gate correctness: if sites
+                // has rows but sites_fts is empty (pre-FTS database opened for the
+                // first time), rebuild the index. The guard lives inside the INSERT
+                // itself (slug NOT IN sites_fts), not only in the counts above: the
+                // counts are read non-atomically, so `npm run index` and a first open
+                // can both pass them and both run this statement — sites_fts.slug has
+                // no unique constraint, so an unguarded re-run would double-index.
+                // Afterwards triggers keep it synced.
+                const { s: sitesN } = db.prepare("SELECT COUNT(*) AS s FROM sites").get();
+                const { s: ftsN } = db.prepare("SELECT COUNT(*) AS s FROM sites_fts").get();
+                if (sitesN > 0 && ftsN === 0) {
+                    db.exec("INSERT INTO sites_fts (slug, title, tags, awards) SELECT slug, title, tags, awards FROM sites WHERE slug NOT IN (SELECT slug FROM sites_fts)");
+                }
+            }
             return fn(db);
         }
         finally {
             db.close();
         }
     }
+    /** @visibleForTesting */
+    withDbForTest(fn) {
+        this.withDb(fn);
+    }
     upsertSites(sites) {
         this.withDb((db) => {
             const stmt = db.prepare(`INSERT INTO sites (slug, id, title, createdAt, tags, thumbnailPath, liveUrl, detailPath, awards, fetchedAt)
@@ -99,6 +155,34 @@ export class Cache {
             return rowToSite(row);
         });
     }
+    // Full-text search over cached sites. matchMode "AND" (default) requires
+    // every token; "OR" matches rows containing any token (the server uses OR
+    // for its zero-result "loose matches" hint). Each token is a porter-stemmed
+    // prefix term, so multi-word queries match rows where the words are
+    // scattered across title/tags/awards. Results are bm25-ascending (best
+    // match first). Returns null when FTS5 is unavailable on this build or the
+    // query has no usable tokens — callers fall back to legacy search.
+    searchSites(query, maxAgeMs, limit = 200, matchMode = "AND") {
+        // Sanitization strips quotes/parens/operators, leaving [a-z0-9-] only —
+        // the quoted `"tok"*` MATCH string below cannot inject FTS syntax.
+        const tokens = query.toLowerCase().split(/\s+/)
+            .map((t) => t.replace(/[^a-z0-9-]/g, ""))
+            .filter((t) => t.length > 0);
+        if (!tokens.length)
+            return null;
+        return this.withDb((db) => {
+            if (!this.ftsAvailable)
+                return null;
+            const match = tokens.map((t) => `"${t}"*`).join(matchMode === "OR" ? " OR " : " AND ");
+            const min = this.now() - maxAgeMs;
+            const rows = db.prepare(`SELECT s.* FROM sites_fts
+         JOIN sites s ON s.slug = sites_fts.slug
+         WHERE sites_fts MATCH ? AND s.fetchedAt > ?
+         ORDER BY bm25(sites_fts) ASC
+         LIMIT ?`).all(match, min, limit);
+            return rows.map(rowToSite);
+        });
+    }
     setMeta(key, value) {
         this.withDb((db) => {
             db.prepare(`INSERT INTO meta (key, value, fetchedAt) VALUES (?, ?, ?)

package/dist/server.js

@@ -76,7 +76,12 @@ export function createHandlers(deps) {
     //   filter was applied, so check every client-checkable filter. Color is
     //   never client-checkable (site rows carry no colors); it is handled by
     //   never serving color searches from cache (see search_sites).
-    function matchesFilters(s, f, honorUrlSource) {
+    // skipQueryCheck=true: the rows already matched the free-text query through
+    // FTS (prefix+stem semantics); re-checking with substring semantics would
+    // wrongly drop stem matches ("magazines" → "Magazine"), so only the
+    // non-query filters run. Scraped rows keep the check (default false) — they
+    // never came from FTS.
+    function matchesFilters(s, f, honorUrlSource, skipQueryCheck = false) {
         const source = urlSource(f);
         if (f.tags?.length) {
             const tagsToCheck = honorUrlSource && source === "tag" ? f.tags.slice(1) : f.tags;
@@ -95,7 +100,7 @@ export function createHandlers(deps) {
             if (!s.awards.includes(AWARD_FILTER_LABELS[f.award]))
                 return false;
         }
-        if (f.query) {
+        if (f.query && !skipQueryCheck) {
             const queryTokens = tokenizeQuery(f.query);
             if (queryTokens.length) {
                 const hay = (s.title + " " + s.tags.join(" ")).toLowerCase();
@@ -136,18 +141,44 @@ export function createHandlers(deps) {
             // Color can't be verified client-side (site rows carry no colors), so a
             // color search always scrapes its filter page; everything else is
             // client-checkable against the index.
-            let sites = args.color
-                ? []
-                : cache.getSites(SITE_TTL_MS).filter((s) => matchesFilters(s, args, false));
+            let sites;
+            let ftsRows = false; // query served from FTS-ranked rows (bm25 order)
+            let ftsEmpty = false; // FTS ran and matched nothing → loose hints apply
+            if (args.color) {
+                sites = [];
+            }
+            else if (args.query) {
+                const ranked = cache.searchSites(args.query, SITE_TTL_MS);
+                if (ranked) {
+                    ftsRows = true;
+                    ftsEmpty = ranked.length === 0;
+                    // FTS already applied the query (prefix+stem, bm25-ranked). Re-check
+                    // only the non-query filters; re-checking the query here with
+                    // substring semantics would wrongly drop prefix/stem matches.
+                    // Array.filter preserves the bm25 order.
+                    sites = ranked.filter((s) => matchesFilters(s, args, false, true));
+                }
+                else {
+                    // FTS5 unavailable or no usable tokens: legacy substring path.
+                    sites = cache.getSites(SITE_TTL_MS).filter((s) => matchesFilters(s, args, false));
+                }
+            }
+            else {
+                sites = cache.getSites(SITE_TTL_MS).filter((s) => matchesFilters(s, args, false));
+            }
             // A scrape normally REPLACES the matched rows (fresh rows are only
             // guaranteed the URL filter), so it must run when the cache cannot serve
             // the requested page window at all. When the window is only PARTIALLY
-            // filled (e.g. a tokenized query matched 3 cached rows for count 6),
-            // replacing would discard already-verified matches — those runs scrape
-            // the filter page and MERGE instead: dedupe by slug, verified cache rows
-            // win duplicate slugs, scraped-only rows appended, all newest-first like
-            // getSites; the merge result is client-checked as before (cache rows
-            // already passed matchesFilters(false), scraped rows matchesFilters(true)).
+            // filled (e.g. a query matched 3 cached rows for count 6), replacing
+            // would discard already-verified matches — those runs scrape the filter
+            // page and MERGE instead: dedupe by slug, verified cache rows win
+            // duplicate slugs, scraped-only rows appended; the merge result is
+            // client-checked as before (cache rows already passed matchesFilters,
+            // scraped rows matchesFilters(true)). Non-FTS runs sort everything
+            // newest-first like getSites; FTS runs keep the bm25-ranked cache rows
+            // first in rank order and append the scraped-only rows newest-first
+            // among themselves — re-sorting ranked rows by createdAt would destroy
+            // the ranking the query asked for.
             const pageWindowEmpty = sites.slice((page - 1) * count, page * count).length === 0;
             const pageWindowPartial = !pageWindowEmpty && sites.length < page * count;
             if (pageWindowEmpty) {
@@ -180,14 +211,26 @@ export function createHandlers(deps) {
                     if (parsed.length > 0) {
                         cache.upsertSites(parsed);
                         const fresh = parsed.filter((s) => matchesFilters(s, args, true));
-                        const bySlug = new Map();
-                        // Scraped rows seed the map; verified cache rows then overwrite any
-                        // duplicate slug, so they always win.
-                        for (const s of fresh)
-                            bySlug.set(s.slug, s);
-                        for (const s of sites)
-                            bySlug.set(s.slug, s);
-                        sites = [...bySlug.values()].sort((a, b) => b.createdAt - a.createdAt);
+                        if (ftsRows) {
+                            // Ranked cache rows keep their bm25 order; scraped-only rows
+                            // append newest-first (cache rows win duplicate slugs, as
+                            // everywhere — a slug in the ranked set is never re-added).
+                            const rankedSlugs = new Set(sites.map((s) => s.slug));
+                            const scrapedOnly = fresh
+                                .filter((s) => !rankedSlugs.has(s.slug))
+                                .sort((a, b) => b.createdAt - a.createdAt);
+                            sites = [...sites, ...scrapedOnly];
+                        }
+                        else {
+                            const bySlug = new Map();
+                            // Scraped rows seed the map; verified cache rows then overwrite any
+                            // duplicate slug, so they always win.
+                            for (const s of fresh)
+                                bySlug.set(s.slug, s);
+                            for (const s of sites)
+                                bySlug.set(s.slug, s);
+                            sites = [...bySlug.values()].sort((a, b) => b.createdAt - a.createdAt);
+                        }
                     }
                 }
                 catch {
@@ -197,6 +240,17 @@ export function createHandlers(deps) {
             const ordered = orderByScore(sites, args.sortBy);
             const slice = ordered.slice((page - 1) * count, page * count);
             if (slice.length === 0) {
+                // Loose-match hint, computed once: only for genuine FTS zero results
+                // (FTS ran and matched nothing) — rows dropped by the non-query
+                // filters are not "loose matches". Up to 3 OR-relaxed slugs.
+                let looseHint = "";
+                if (ftsEmpty && args.query) {
+                    const orSlugs = (cache.searchSites(args.query, SITE_TTL_MS, 3, "OR") ?? [])
+                        .map((s) => s.slug);
+                    if (orSlugs.length > 0) {
+                        looseHint = `\nLoose matches (any token): ${orSlugs.join(", ")}.`;
+                    }
+                }
                 if (sites.length === 0) {
                     // True zero-result search: suggest the closest taxonomy tags. The
                     // taxonomy comes from the cache only — never a live fetch just to
@@ -209,15 +263,17 @@ export function createHandlers(deps) {
                         return {
                             content: [
                                 text(`No sites matched the search. Closest filter tags: ${suggestions.join(", ")}. ` +
-                                    "Run list_categories for the full taxonomy."),
+                                    `Run list_categories for the full taxonomy.${looseHint}`),
                             ],
                         };
                     }
                 }
                 return {
                     content: [
-                        text("No sites matched the search on this page. Try fewer filters or run list_categories. " +
-                            "(Deep pagination is unavailable by design: awwwards.com's robots.txt disallows it.)"),
+                        text(`No sites matched the search on this page. Try fewer filters or run list_categories. ` +
+                            "(Deep pagination is unavailable by design: awwwards.com's robots.txt disallows it.)" +
+                            // looseHint is empty unless this is a true FTS zero result.
+                            looseHint),
                     ],
                 };
             }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "awwwards-mcp",
-  "version": "1.3.0",
+  "version": "1.4.0",
   "description": "Free MCP server giving AI agents design inspiration from Awwwards: search award-winning sites with inline screenshots and extract design DNA.",
   "type": "module",
   "license": "MIT",

package/skills/awwwards-inspiration/SKILL.md

@@ -31,6 +31,9 @@ Run this loop before building anything visual:
 3. **Search.** Call `search_sites` with 1–3 filters (e.g.
    `{ color: "#404040", tags: ["3d", "portfolio"] }`). Judge the results from
    the inline screenshots, not just titles. Shortlist 2–3 candidates.
+   Free-text queries are porter-stem + prefix-matched and BM25-ranked
+   (`"magazines"` finds Magazine-tagged sites, best matches first); multi-word
+   queries keep AND semantics — both tokens must hit the same site.
 4. **Live reference URL named? Capture it full-page first.** (and later capture
    your own build the same way — compare both against each other) If the user
    points at a specific live site (e.g. "recreate cerebrium.ai"), call