awwwards-mcp 1.0.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Afjal
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,106 @@
+# awwwards-mcp
+
+Free, open-source MCP server that gives AI agents design inspiration from
+[Awwwards](https://www.awwwards.com/) — the Mobbin-style visual reference loop,
+sourced from the web's best award-winning websites.
+
+Your agent searches in natural language ("dark 3D portfolio sites", "soft pastel
+e-commerce"), sees **real screenshots inline**, and can pull the **design DNA**
+of any site: color palette, tech stack, design elements, award history.
+
+## Tools
+
+| Tool | What it does |
+|------|--------------|
+| `search_sites` | Search by color, tags, technology or award type. Returns site cards with inline screenshots. |
+| `get_site_details` | Full design DNA for one site: palette, technologies, elements, awards, description. |
+| `get_site_elements` | Component-level visuals for one site: each element's poster image inline (3D models, video content, mobile layouts, microcopy…) + video URLs. |
+| `list_categories` | Every filter the agent can search by (200+ tags, 27 colors). |
+| `capture_live_site` | Optional: fresh full-page screenshot of any live URL. Waits for `load` + a settle window with a bounded pre-scroll, so heavy sites work (`waitStrategy: "networkidle"` available). (needs [playwright](https://playwright.dev)). |
+| `analyze_page_structure` | Section band map of any page (live URL or local file:// build): tag, background, offset, height per band. Compare a reference site's structure against your build. Same heavy-site-friendly wait (`waitStrategy: "networkidle"` available). (needs [playwright](https://playwright.dev)). |
+| `record_site_motion` | Optional: short motion-through video of a live URL — preloader, scroll-triggered and hover/cursor animations. Returns an inline filmstrip JPEG plus the saved .webm path. (needs [playwright](https://playwright.dev) + ffmpeg-static). |
+
+## Setup
+
+**Claude Code**
+
+```bash
+claude mcp add awwwards -- npx -y awwwards-mcp
+```
+
+**Claude Desktop / Cursor / Windsurf** (`mcpServers` in the config):
+
+```json
+{
+  "mcpServers": {
+    "awwwards": { "command": "npx", "args": ["-y", "awwwards-mcp"] }
+  }
+}
+```
+
+Optional full-page captures:
+
+```bash
+npm install -g playwright && npx playwright install chromium
+```
+
+## Skills
+
+This package ships an agent skill that teaches the inspiration workflow —
+search, judge from screenshots, pull design DNA, state a design direction —
+using the awwwards MCP tools. Copy it into your agent's skills directory:
+
+```bash
+npm install awwwards-mcp
+mkdir -p ~/.claude/skills && cp -r node_modules/awwwards-mcp/skills/awwwards-inspiration ~/.claude/skills/
+```
+
+For ZCode, copy to `~/.zcode/skills/` instead of `~/.claude/skills/`.
+Windows: run this from Git Bash, or copy `node_modules\awwwards-mcp\skills\awwwards-inspiration` manually.
+
+## Indexing (recommended)
+
+`search_sites` works out of the box, but its depth is limited by polite live
+scraping (~31 sites per filter page). Build a local index once and searches
+draw from thousands of award-winning sites instantly:
+
+```bash
+npx -y -p awwwards-mcp awwwards-index      # once published
+# or, from a local checkout of this repo:
+npm run index
+```
+
+- Crawls all ~200 tag pages at 1 request/second (~4 minutes) into the local
+  SQLite cache at `~/.awwwards-mcp/`.
+- Resumable: interrupt it and re-run — completed pages are skipped.
+- The MCP server re-indexes automatically in the background whenever the
+  index is older than 7 days (never blocking your session).
+
+Site details (palettes, tech stacks) are still fetched on demand and cached
+for 7 days.
+
+## How it works
+
+- Live, polite scraping of awwwards.com public pages (max 1 request/second,
+  robots.txt-compliant paths only, cached 7 days in SQLite at `~/.awwwards-mcp/`).
+- Screenshots are served from Awwwards' own CDN (880×660), cached on disk.
+- No API key, no account, no cost.
+
+## Ethics & terms
+
+This tool fetches publicly available pages for **personal design-inspiration
+use**, at human-ish request rates, honoring robots.txt. Awwwards' screenshots
+and content remain the property of Awwwards and the credited creators — don't
+bulk-scrape, redistribute, or republish them. If you use this commercially,
+review awwwards.com's terms yourself.
+
+## Development
+
+```bash
+npm install
+npm test        # offline unit tests against committed HTML fixtures
+npm run smoke   # manual live smoke test against awwwards.com
+npm run build   # compile to dist/
+```
+
+MIT — see [LICENSE](LICENSE).

package/dist/awwwards.js

@@ -0,0 +1,113 @@
+export const BASE_URL = "https://www.awwwards.com";
+export const ASSETS_URL = "https://assets.awwwards.com";
+export const USER_AGENT = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0 Safari/537.36 awwwards-mcp/1.0";
+export const AWARD_FILTERS = {
+    sotd: "websites/sites_of_the_day",
+    developer: "websites/developer",
+    honorable: "websites/honorable",
+};
+// Combined filter URLs return 404 on awwwards.com (verified 2026-09-17), so
+// exactly one filter is used in the URL — the most specific one. The caller
+// applies the remaining filters client-side over the parsed results.
+export function buildFilterUrl(filters) {
+    if (filters.color) {
+        return `${BASE_URL}/websites/%23${filters.color.replace("#", "").toUpperCase()}/`;
+    }
+    if (filters.award)
+        return `${BASE_URL}/${AWARD_FILTERS[filters.award]}/`;
+    const tag = filters.technology ?? filters.tags?.[0];
+    if (tag)
+        return `${BASE_URL}/websites/${encodeURIComponent(tag.toLowerCase())}/`;
+    return `${BASE_URL}/websites/`;
+}
+export function thumbnailUrl(thumbPath, size = 880) {
+    const dim = size === 880 ? "880_660" : "440_330";
+    return `${ASSETS_URL}/awards/media/cache/thumb_${dim}/${thumbPath}`;
+}
+export function elementUrl(mediaPath) {
+    return `${ASSETS_URL}/awards/${mediaPath}`;
+}
+// Video elements ship a poster at the same path with .mp4 → _static.jpeg
+// (live-verified on the CDN); image elements are used as-is.
+export function elementPosterPath(mediaPath) {
+    return mediaPath.endsWith(".mp4")
+        ? mediaPath.replace(/\.mp4$/, "_static.jpeg")
+        : mediaPath;
+}
+export class BlockedError extends Error {
+    status;
+    constructor(url, status) {
+        super(`Awwwards is blocking requests (HTTP ${status} on ${url}). ` +
+            "Try again later; the tool never retries through blocks.");
+        this.status = status;
+    }
+}
+export class RateLimiter {
+    intervalMs;
+    last = 0;
+    chain = Promise.resolve();
+    constructor(intervalMs) {
+        this.intervalMs = intervalMs;
+    }
+    acquire() {
+        const next = this.chain.then(async () => {
+            const wait = this.last + this.intervalMs - Date.now();
+            if (wait > 0)
+                await new Promise((r) => setTimeout(r, wait));
+            this.last = Date.now();
+        });
+        this.chain = next.catch(() => { });
+        return next;
+    }
+}
+export class AwwwardsClient {
+    rateLimiter;
+    fetchFn;
+    constructor(opts = {}) {
+        this.rateLimiter = opts.rateLimiter ?? new RateLimiter(1000);
+        this.fetchFn = opts.fetchFn ?? fetch;
+    }
+    // Rate-limited page fetch with one retry on transient failures.
+    // Blocks (403/429) are never retried.
+    async getHtml(path) {
+        const url = path.startsWith("http") ? path : BASE_URL + path;
+        let lastErr;
+        for (let attempt = 0; attempt < 2; attempt++) {
+            try {
+                await this.rateLimiter.acquire();
+                const res = await this.fetchFn(url, {
+                    headers: { "User-Agent": USER_AGENT },
+                    redirect: "follow",
+                });
+                if (res.status === 403 || res.status === 429)
+                    throw new BlockedError(url, res.status);
+                if (!res.ok)
+                    throw new Error(`HTTP ${res.status} for ${url}`);
+                return await res.text();
+            }
+            catch (err) {
+                if (err instanceof BlockedError)
+                    throw err;
+                lastErr = err;
+            }
+        }
+        throw lastErr;
+    }
+    // Thumbnail fetch from the CDN — not rate-limited.
+    async getThumbnail(thumbPath, size = 880) {
+        return this.fetchCdnBinary(thumbnailUrl(thumbPath, size), `thumbnail ${thumbPath}`);
+    }
+    // Asset fetch from the CDN (element posters etc.) — not rate-limited.
+    async getAsset(assetPath) {
+        return this.fetchCdnBinary(elementUrl(assetPath), `asset ${assetPath}`);
+    }
+    // Binary CDN assets (thumbnails, element media) all fetch through this single path.
+    async fetchCdnBinary(url, label) {
+        const res = await this.fetchFn(url, {
+            headers: { "User-Agent": USER_AGENT },
+        });
+        if (!res.ok)
+            throw new Error(`HTTP ${res.status} fetching ${label}`);
+        return Buffer.from(await res.arrayBuffer());
+    }
+}

package/dist/cache.js

@@ -0,0 +1,134 @@
+import { createHash } from "node:crypto";
+import { mkdirSync } from "node:fs";
+import { readFile, writeFile } from "node:fs/promises";
+import { join } from "node:path";
+import { DatabaseSync } from "node:sqlite";
+// Freshness window applied by getSite() when the caller does not pass one.
+// Expired lookups are misses (single-arg getSite returns null for rows older
+// than this window); getSites(Infinity) remains the stale fallback.
+const DEFAULT_SITE_TTL_MS = 10_000;
+const SCHEMA = `
+  CREATE TABLE IF NOT EXISTS sites (
+    slug TEXT PRIMARY KEY, id INTEGER, title TEXT, createdAt INTEGER,
+    tags TEXT, thumbnailPath TEXT, liveUrl TEXT, detailPath TEXT,
+    awards TEXT, fetchedAt INTEGER
+  );
+  CREATE TABLE IF NOT EXISTS meta (
+    key TEXT PRIMARY KEY, value TEXT, fetchedAt INTEGER
+  );
+`;
+function rowToSite(r) {
+    return {
+        slug: r.slug,
+        id: r.id,
+        title: r.title,
+        createdAt: r.createdAt,
+        tags: JSON.parse(r.tags),
+        thumbnailPath: r.thumbnailPath,
+        liveUrl: r.liveUrl,
+        detailPath: r.detailPath,
+        awards: JSON.parse(r.awards),
+    };
+}
+export class Cache {
+    dbPath;
+    now;
+    imagesDir;
+    constructor(rootDir, now = Date.now) {
+        this.now = now;
+        mkdirSync(rootDir, { recursive: true });
+        this.imagesDir = join(rootDir, "images");
+        mkdirSync(this.imagesDir, { recursive: true });
+        this.dbPath = join(rootDir, "cache.db");
+        // Create cache.db + schema eagerly so the storage layout exists right
+        // after construction. The handle is closed immediately (see withDb).
+        this.withDb(() => { });
+    }
+    // Open → run → close per operation. node:sqlite keeps cache.db open until
+    // close(); on Windows an open handle makes the file and its directory
+    // undeletable, which broke temp-dir cleanup in tests. Per-operation
+    // open/close keeps the same public API with no lingering handles.
+    withDb(fn) {
+        const db = new DatabaseSync(this.dbPath);
+        try {
+            db.exec(SCHEMA);
+            return fn(db);
+        }
+        finally {
+            db.close();
+        }
+    }
+    upsertSites(sites) {
+        this.withDb((db) => {
+            const stmt = db.prepare(`INSERT INTO sites (slug, id, title, createdAt, tags, thumbnailPath, liveUrl, detailPath, awards, fetchedAt)
+         VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
+         ON CONFLICT(slug) DO UPDATE SET
+           id=excluded.id, title=excluded.title, createdAt=excluded.createdAt,
+           tags=excluded.tags, thumbnailPath=excluded.thumbnailPath,
+           liveUrl=excluded.liveUrl, detailPath=excluded.detailPath,
+           awards=excluded.awards, fetchedAt=excluded.fetchedAt`);
+            const t = this.now();
+            // One transaction for the whole batch: each autocommitted INSERT pays a
+            // disk sync (~4ms on Windows), which made a 31-card upsert ~150ms and a
+            // full index crawl take minutes. A single commit syncs once.
+            db.exec("BEGIN");
+            try {
+                for (const s of sites) {
+                    stmt.run(s.slug, s.id, s.title, s.createdAt, JSON.stringify(s.tags), s.thumbnailPath, s.liveUrl, s.detailPath, JSON.stringify(s.awards), t);
+                }
+                db.exec("COMMIT");
+            }
+            catch (err) {
+                db.exec("ROLLBACK");
+                throw err;
+            }
+        });
+    }
+    getSites(maxAgeMs) {
+        return this.withDb((db) => {
+            const min = this.now() - maxAgeMs;
+            const rows = db.prepare("SELECT * FROM sites WHERE fetchedAt > ? ORDER BY createdAt DESC").all(min);
+            return rows.map(rowToSite);
+        });
+    }
+    getSite(slug, maxAgeMs = DEFAULT_SITE_TTL_MS) {
+        return this.withDb((db) => {
+            const row = db.prepare("SELECT * FROM sites WHERE slug = ?").get(slug);
+            if (!row || row.fetchedAt <= this.now() - maxAgeMs)
+                return null;
+            return rowToSite(row);
+        });
+    }
+    setMeta(key, value) {
+        this.withDb((db) => {
+            db.prepare(`INSERT INTO meta (key, value, fetchedAt) VALUES (?, ?, ?)
+         ON CONFLICT(key) DO UPDATE SET value=excluded.value, fetchedAt=excluded.fetchedAt`).run(key, JSON.stringify(value), this.now());
+        });
+    }
+    getMeta(key, maxAgeMs) {
+        return this.withDb((db) => {
+            const row = db.prepare("SELECT value, fetchedAt FROM meta WHERE key = ?").get(key);
+            if (!row || row.fetchedAt <= this.now() - maxAgeMs)
+                return null;
+            return JSON.parse(row.value);
+        });
+    }
+    deleteMeta(key) {
+        this.withDb((db) => {
+            db.prepare("DELETE FROM meta WHERE key = ?").run(key);
+        });
+    }
+    // Disk cache keyed by the awwwards asset path (immutable content → no TTL).
+    async getImage(assetPath, fetcher) {
+        const ext = assetPath.endsWith(".png") ? ".png" : ".jpg";
+        const file = join(this.imagesDir, createHash("sha1").update(assetPath).digest("hex") + ext);
+        try {
+            return await readFile(file);
+        }
+        catch {
+            const buf = await fetcher();
+            await writeFile(file, buf);
+            return buf;
+        }
+    }
+}

package/dist/capture.js

@@ -0,0 +1,51 @@
+import { createHash } from "node:crypto";
+import { readFile } from "node:fs/promises";
+import { join } from "node:path";
+// preScroll lives in structure.ts so there is ONE scroll-through loop for
+// both tools; this is an intentional circular import (structure.ts imports
+// CAPTURE_INSTALL_HINT from here) — both sides only use the other's bindings
+// at call time, which ESM resolves fine.
+import { preScroll } from "./structure.js";
+export const CAPTURE_INSTALL_HINT = "Full-page capture needs Playwright, which is an optional dependency.\n" +
+    "Install it with:  npm install -D playwright && npx playwright install chromium\n" +
+    "Then retry the capture or structure tool.";
+export async function captureLiveSite(url, imagesDir, 
+// "as string" keeps Playwright an unresolved optional dependency at compile time;
+// Node resolves it (and may throw) at runtime, which the try/catch below handles.
+loader = () => import("playwright"), opts) {
+    let chromium;
+    try {
+        ({ chromium } = await loader());
+    }
+    catch {
+        return { error: CAPTURE_INSTALL_HINT };
+    }
+    let browser;
+    try {
+        browser = await chromium.launch();
+    }
+    catch {
+        return { error: CAPTURE_INSTALL_HINT };
+    }
+    try {
+        const waitStrategy = opts?.waitStrategy ?? "load";
+        const page = await browser.newPage({ viewport: { width: 1440, height: 900 } });
+        await page.goto(url, { waitUntil: waitStrategy, timeout: 45_000 });
+        // "load" can fire before late XHRs settle, so give the page a fixed
+        // settle window; networkidle already means the network went quiet.
+        if (waitStrategy === "load")
+            await page.waitForTimeout(3000);
+        await preScroll(page);
+        const file = join(imagesDir, "capture-" + createHash("sha1").update(url).digest("hex").slice(0, 12) + ".png");
+        await page.screenshot({ path: file, fullPage: true });
+        return { file, base64: (await readFile(file)).toString("base64") };
+    }
+    finally {
+        try {
+            await browser.close();
+        }
+        catch {
+            /* keep the primary error */
+        }
+    }
+}

package/dist/cli.js

@@ -0,0 +1,103 @@
+#!/usr/bin/env node
+import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
+import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
+import { z } from "zod";
+import { homedir } from "node:os";
+import { join } from "node:path";
+import { readFileSync } from "node:fs";
+import { fileURLToPath } from "node:url";
+import { dirname } from "node:path";
+import { AwwwardsClient } from "./awwwards.js";
+import { Cache } from "./cache.js";
+import { createHandlers } from "./server.js";
+import { captureLiveSite } from "./capture.js";
+import { runIndexer, shouldAutoIndex } from "./indexer.js";
+// The handlers return ToolResponse, which is structurally identical to the
+// SDK's CallToolResult at runtime ({ content, isError? }). CallToolResult's
+// schema additionally carries a [k: string]: unknown index signature that a
+// TypeScript interface cannot satisfy implicitly, so a cast bridges the two.
+const asMcpResult = (p) => p;
+// Shared by capture_live_site, analyze_page_structure and record_site_motion.
+const waitStrategySchema = z
+    .enum(["load", "networkidle"])
+    .default("load")
+    .describe("'load' + settle works on heavy sites; 'networkidle' waits for total quiet");
+const cacheRoot = process.env.AWWWARDS_CACHE_DIR ?? join(homedir(), ".awwwards-mcp");
+let cache;
+try {
+    cache = new Cache(cacheRoot);
+}
+catch (err) {
+    console.error(`awwwards-mcp: cannot initialize cache at ${cacheRoot}: ${err instanceof Error ? err.message : String(err)}`);
+    process.exit(1);
+}
+const client = new AwwwardsClient();
+const handlers = createHandlers({
+    client,
+    cache,
+    // captureLiveSite's third positional is the injectable playwright loader,
+    // so the CaptureFn-shaped (url, dir, opts) call is adapted to land opts
+    // in the function's fourth (opts) position.
+    captureFn: (url, imagesDir, opts) => captureLiveSite(url, imagesDir, undefined, opts),
+});
+// Version must match package.json; reading it at runtime makes drift impossible.
+const pkgJson = JSON.parse(readFileSync(join(dirname(fileURLToPath(import.meta.url)), "../package.json"), "utf8"));
+const server = new McpServer({ name: "awwwards-mcp", version: pkgJson.version });
+server.tool("search_sites", "Search award-winning websites on Awwwards. Returns site cards with inline screenshots, live URLs, awards and tags.", {
+    query: z.string().describe("Free text matched against site titles and tags").optional(),
+    color: z
+        .string()
+        .regex(/^#?[0-9A-Fa-f]{6}$/)
+        .describe("Dominant color hex, e.g. '#404040'")
+        .optional(),
+    tags: z.array(z.string()).describe("Tag slugs, e.g. ['3d', 'portfolio']").optional(),
+    technology: z.string().describe("Technology slug, e.g. 'webgl', 'gsap', 'astro'").optional(),
+    award: z.enum(["sotd", "developer", "honorable"]).optional(),
+    sortBy: z
+        .enum(["score", "newest"])
+        .default("newest")
+        .describe("Sort results: by Awwwards jury score (details previously fetched) or newest first"),
+    count: z.number().int().min(1).max(12).default(6),
+    page: z.number().int().min(1).default(1),
+}, (args) => asMcpResult(handlers.search_sites(args)));
+server.tool("get_site_details", "Get the design DNA of one Awwwards site: color palette, technologies, design elements, awards, description and inline screenshot.", {
+    slug: z
+        .string()
+        .regex(/^[\w-]+$/)
+        .describe("Site slug from search_sites, e.g. 'l-i-s-a'"),
+}, (args) => asMcpResult(handlers.get_site_details(args)));
+server.tool("get_site_elements", "Get the design-element highlights of one Awwwards site: component-level visuals (3D models, video content, mobile layouts, microcopy) with poster images inline and video URLs.", {
+    slug: z
+        .string()
+        .regex(/^[\w-]+$/)
+        .describe("Site slug from search_sites, e.g. 'l-i-s-a'"),
+}, (args) => asMcpResult(handlers.get_site_elements(args)));
+server.tool("list_categories", "List the filter taxonomy available on Awwwards: color hexes and tag/technology slugs usable with search_sites.", {}, () => asMcpResult(handlers.list_categories()));
+server.tool("capture_live_site", "Take a fresh full-page screenshot of a live website URL using a headless browser. Requires the optional playwright dependency.", {
+    url: z.string().url().describe("Absolute URL of the site to capture"),
+    waitStrategy: waitStrategySchema,
+}, (args) => asMcpResult(handlers.capture_live_site(args)));
+server.tool("analyze_page_structure", "Extract a page's section band map (tag, label, background color, offset, height per band) via a headless browser. Works on live URLs and file:// paths — use it to compare a reference site's structure against your local build.", {
+    url: z.string().url().describe("Absolute URL (https:// or file://) of the page to analyze"),
+    maxBands: z.number().int().min(5).max(60).default(40).describe("Cap on returned bands"),
+    waitStrategy: waitStrategySchema,
+}, (args) => asMcpResult(handlers.analyze_page_structure(args)));
+server.tool("record_site_motion", "Record a short motion-through video of a live website — preloader, scroll-triggered and hover/cursor animations — and return an inline filmstrip JPEG plus the .webm path. Requires the optional playwright and ffmpeg-static dependencies.", {
+    url: z.string().url().describe("Absolute URL of the site to record"),
+    frames: z
+        .number()
+        .int()
+        .min(4)
+        .max(36)
+        .default(16)
+        .describe("Filmstrip tile count (default 16 → a 4x4 grid)"),
+    waitStrategy: waitStrategySchema,
+}, (args) => asMcpResult(handlers.record_site_motion(args)));
+// Auto-refresh: if the index is stale (or absent) and no crawl is running,
+// re-index in the background. Serving is never blocked; errors are stderr-only.
+if (shouldAutoIndex(cache)) {
+    void runIndexer({ client, cache, log: (m) => console.error(m) }).catch((err) => {
+        console.error(`awwwards-mcp: background index failed: ${err instanceof Error ? err.message : String(err)}`);
+    });
+}
+await server.connect(new StdioServerTransport());

package/dist/index-cli.js

@@ -0,0 +1,30 @@
+#!/usr/bin/env node
+import { homedir } from "node:os";
+import { join } from "node:path";
+import { AwwwardsClient } from "./awwwards.js";
+import { Cache } from "./cache.js";
+import { runIndexer, IndexLockError } from "./indexer.js";
+const cacheRoot = process.env.AWWWARDS_CACHE_DIR ?? join(homedir(), ".awwwards-mcp");
+let cache;
+try {
+    cache = new Cache(cacheRoot);
+}
+catch (err) {
+    console.error(`awwwards-index: cannot initialize cache at ${cacheRoot}: ${err instanceof Error ? err.message : String(err)}`);
+    process.exit(1);
+}
+const client = new AwwwardsClient();
+try {
+    const result = await runIndexer({ client, cache, log: (m) => console.error(m) });
+    console.error(`awwwards-index: done — ${result.pagesDone} pages crawled, ${result.skipped} skipped, ` +
+        `${result.sitesIndexed} site rows upserted (${result.pagesTotal} tags total)`);
+    process.exit(0);
+}
+catch (err) {
+    if (err instanceof IndexLockError) {
+        console.error(`awwwards-index: ${err.message}`);
+        process.exit(0);
+    }
+    console.error(`awwwards-index aborted: ${err instanceof Error ? err.message : String(err)}`);
+    process.exit(1);
+}

package/dist/indexer.js

@@ -0,0 +1,116 @@
+import { parseCategories, parseListing } from "./parsers.js";
+export const INDEX_STALE_MS = 7 * 24 * 60 * 60 * 1000;
+export const INDEX_LOCK_STALE_MS = 30 * 60 * 1000;
+// Same value as server.ts CATEGORY_TTL_MS; redeclared to avoid importing the
+// server module (and its MCP wiring) into the indexer.
+const CATEGORY_TTL_MS = 30 * 24 * 60 * 60 * 1000;
+export class IndexLockError extends Error {
+    constructor() {
+        super("another index run is in progress");
+    }
+}
+export function isIndexStale(cache, now = Date.now) {
+    const status = cache.getMeta("index:status", Number.POSITIVE_INFINITY);
+    if (!status || !status.finishedAt)
+        return true;
+    // >= so the exact INDEX_STALE_MS boundary already counts as stale, matching
+    // the lock boundary semantics (exact boundary = expired).
+    return now() - status.finishedAt >= INDEX_STALE_MS;
+}
+export function shouldAutoIndex(cache, now = Date.now) {
+    const lock = cache.getMeta("index:lock", Number.POSITIVE_INFINITY);
+    if (lock && now() - lock.startedAt < INDEX_LOCK_STALE_MS)
+        return false;
+    return isIndexStale(cache, now);
+}
+export async function runIndexer(deps) {
+    const { client, cache } = deps;
+    const now = deps.now ?? Date.now;
+    const log = deps.log ?? (() => { });
+    const lock = cache.getMeta("index:lock", Number.POSITIVE_INFINITY);
+    if (lock && now() - lock.startedAt < INDEX_LOCK_STALE_MS)
+        throw new IndexLockError();
+    cache.setMeta("index:lock", { startedAt: now() });
+    try {
+        let cats = cache.getMeta("categories", CATEGORY_TTL_MS);
+        if (!cats || cats.filters.length === 0) {
+            cats = parseCategories(await client.getHtml("/websites/"));
+            if (cats.filters.length === 0) {
+                throw new Error("Awwwards layout may have changed: parsed 0 categories. The awwwards-mcp parser likely needs an update.");
+            }
+            cache.setMeta("categories", cats);
+        }
+        const tags = cats.filters;
+        const done = new Set(cache.getMeta("index:progress", Number.POSITIVE_INFINITY) ?? []);
+        const result = await crawl({ client, cache, now, log }, tags, done);
+        cache.setMeta("index:status", {
+            startedAt: result.startedAt,
+            finishedAt: now(),
+            pagesDone: done.size,
+            pagesTotal: tags.length,
+            sitesIndexed: result.sitesIndexed,
+        });
+        return {
+            pagesDone: result.pagesDone,
+            pagesTotal: tags.length,
+            sitesIndexed: result.sitesIndexed,
+            skipped: result.skipped,
+        };
+    }
+    catch (err) {
+        try {
+            const previous = cache.getMeta("index:status", Number.POSITIVE_INFINITY);
+            const progress = (cache.getMeta("index:progress", Number.POSITIVE_INFINITY) ?? []).length;
+            cache.setMeta("index:status", {
+                startedAt: previous?.startedAt,
+                pagesDone: progress,
+                pagesTotal: tagsCount(cache),
+                sitesIndexed: previous?.sitesIndexed ?? 0,
+                lastError: err instanceof Error ? err.message : String(err),
+            });
+        }
+        catch {
+            // the store is failing; do not mask the original abort error
+        }
+        throw err;
+    }
+    finally {
+        try {
+            cache.deleteMeta("index:lock");
+        }
+        catch {
+            // same: never mask the original error with a lock-release failure
+        }
+    }
+}
+// Total page count for abort-path status writes: prefer the last status object,
+// fall back to the cached taxonomy size. 0 is accurate when the abort happened
+// before any taxonomy was ever fetched.
+function tagsCount(cache) {
+    return (cache.getMeta("index:status", Number.POSITIVE_INFINITY)?.pagesTotal ??
+        (cache.getMeta("categories", CATEGORY_TTL_MS)?.filters.length ?? 0));
+}
+async function crawl(deps, tags, done) {
+    const { client, cache, now, log } = deps;
+    const startedAt = now();
+    const skipped = [...tags].filter((t) => done.has(t)).length;
+    let sitesIndexed = 0;
+    let pagesDone = 0;
+    for (const tag of tags) {
+        if (done.has(tag))
+            continue;
+        const html = await client.getHtml(`/websites/${encodeURIComponent(tag)}/`);
+        const sites = parseListing(html);
+        if (sites.length === 0) {
+            throw new Error(`Awwwards layout may have changed: parsed 0 site cards on /websites/${tag}/. ` +
+                "The awwwards-mcp parser likely needs an update.");
+        }
+        cache.upsertSites(sites);
+        sitesIndexed += sites.length;
+        pagesDone += 1;
+        done.add(tag);
+        cache.setMeta("index:progress", [...done]);
+        log(`[${done.size}/${tags.length}] ${tag}: ${sites.length} sites`);
+    }
+    return { pagesDone, sitesIndexed, skipped, startedAt };
+}