@houtini/seo-audit-console 0.2.3 → 0.4.3

package/README.md

@@ -120,7 +120,10 @@ Each of these is a real procedure I use, and each is one prompt. The expanded ve
 6. **Monitoring and migrations.** *"Detect changes on mysite.com"* diffs your two most recent crawls by severity. During a migration this is the difference between catching a stray noindex on Tuesday and explaining a traffic graph in a board meeting three weeks later. I've been on the wrong end of that one. → [tools.md](manual/tools.md#monitoring)
 7. **AI-search readiness.** *"Check agent readiness for mysite.com"* - the web is quietly growing a second audience, and almost no SEO tool checks any of it. → [tools.md](manual/tools.md#check_agent_readiness)
 8. **Your own questions.** The four datasets share three join keys, and the most valuable analyses are the ones you compose yourself - *"which pages lost clicks after being cited in AI Overviews?"* is one prompt here and a feature nowhere else. → [composition.md](manual/composition.md)
-9. **Reporting.** *"Show me the dashboard"* in the chat, or *"export the report"* as one self-contained HTML file you can send a client. → [dashboard.md](manual/dashboard.md)
+9. **Reporting.** *"Show me the dashboard"* in the chat, *"serve the dashboard"* for the live version in your browser, or *"export the report"* as one self-contained HTML file you can send a client. → [dashboard.md](manual/dashboard.md)
+10. **The content opportunity report.** If you write for a living, this is your page one: *"Content opportunities for mysite.com"* returns what to WRITE next (demand you already earn impressions for with no winning page), what to REFRESH (pages bleeding clicks), which SNIPPETS to rewrite (ranking well, under-clicked), and which clusters to STRENGTHEN (position 4-20, one push from the money). Every row traces to your own Search Console data - no invented keyword ideas - and *"draft the brief for row 1"* turns any of them into a grounded outline. → [competitive.md](manual/competitive.md)
+11. **Market Sizing and Prioritisation.** *"Size the market: mysite.com vs rival1.com and rival2.com"* - the keyword universe, total monthly demand, and share of voice per topic cluster, from one cached call per domain. The engagement-opening read that used to need a Semrush subscription. → [competitive.md](manual/competitive.md)
+12. **Content recon - why is this page losing?** *"Run content recon on mysite.com"* takes your worst declining pages, pulls the live Google SERP for each, and reads whether the **AI Overview** cites you. The verdict is the honest bit: *rank but not cited* is a data-accuracy or freshness problem, not a rewrite. It transcribes the ranking videos (Supadata), reads the reachable competitors (Firecrawl, plus a free browser-profile fetch that even gets Reddit), and writes the gaps into a tracked to-do board you can annotate and re-measure. → [competitive.md](manual/competitive.md)
 
 ![Ranking distribution over time - impressions by position bucket](assets/search-performance.png)
 
@@ -130,7 +133,9 @@ Each of these is a real procedure I use, and each is one prompt. The expanded ve
 
 Everything above works with just your Search Console data. But GSC can only describe searches where you already appear. The moment your question is "how big is this market?" or "what do competitors rank for that I don't?", you need third-party data - and that's [DataForSEO](https://dataforseo.com/?aff=213701): a pay-as-you-go API for volumes, live rankings, competitor data and Lighthouse runs. No subscription; calls cost fractions of a cent to a few cents, cached for 20 days, and only ever run when you ask. My own usage runs to a few dollars a month.
 
-It unlocks the Semrush-replacement layer: the organic visibility overview for any domain, any site's top pages and ranked keywords (including which keywords cite a site in **AI Overviews**), the content gap, topic gaps, search intent, lab Core Web Vitals and backlinks. The full workflows and setup: [competitive.md](manual/competitive.md).
+It unlocks the Semrush-replacement layer: the organic visibility overview for any domain, any site's top pages and ranked keywords (including which keywords cite a site in **AI Overviews**), the content gap, topic gaps, search intent, lab Core Web Vitals and backlinks. It also drives **content recon** - `recon_targets` pulls the live Google SERP for a losing page, reads whether the AI Overview cites you, and tells you *why* you're behind. The full workflows and setup: [competitive.md](manual/competitive.md).
+
+**Two more optional keys, for content recon only.** When you ask recon to fetch the competitors it finds, it routes each by source: videos to [Supadata](https://supadata.ai) (`SUPADATA_API_KEY`) for transcripts, and competitor pages to [Firecrawl](https://firecrawl.link/2d1PLD8) (`FIRECRAWL_API_KEY`) for clean markdown, with a free browser-profile fetch behind it. Both are pay-as-you-go and entirely optional - recon still classifies the page and lists the competitors without them. Grab a Firecrawl key here: **https://firecrawl.link/2d1PLD8**.
 
 ---
 
@@ -167,7 +172,10 @@ The one-line version - full descriptions, inputs and example prompts for every t
 | `competitors_domain` · `page_intersection` · `topic_gaps` | Competitors · content gap · topic gaps |
 | `domain_visibility` · `top_pages` · `ranked_keywords` | The Semrush-style views, any domain (+ `aioOnly` for AI Overview citations) |
 | `page_lighthouse` · `pull_backlinks` | Lab CWV · backlink profile with live status |
-| `get_dashboard` · `export_report` | In-chat dashboard · shareable HTML |
+| `market_sizing` · `serp_features` | Share of voice vs named competitors · AI-Overview / snippet / video exposure by volume |
+| `keyword_list` · `content_opportunities` | Cluster a keyword list (own / weak / absent) · the content marketer's report |
+| `recon_targets` · `save_recon_todo` · `recon_todos` | **Content recon:** why a page is losing (live SERP + AI Overview) → a tracked, annotatable to-do board |
+| `get_dashboard` · `serve_dashboard` · `export_report` | In-chat dashboard · the same in a local browser tab · shareable HTML |
 | `composition_cookbook` | The data-surface map and recipes for bespoke analyses |
 | `data_storage` | Per-property disk usage and row counts, with confirm-gated pruning |
 | `normalize_url` · `data_location` · `list_properties` · `seo_audit_help` | Utilities and the help menu |
@@ -192,7 +200,6 @@ Your Search Console data and the crawl live in local SQLite files under `SAC_DAT
 
 A few things I'm building next, in rough order:
 
-- **List mode.** Paste any list of URLs - a migration map, old ranking pages, a PPC export - and have it status-checked and audited. The migration-verification workflow, basically.
 - **Structured-data opportunities, by template.** Not "you have no schema" (most modern stores have plenty), but "this template could earn review stars or an FAQ rich result and doesn't."
 - **A per-page content scorecard.** A dedicated table of *every* ranking phrase your copy misses, not just the top query.
 - **More agent readiness.** A WebMCP advisory (which tool actions your site could expose to agents) and the agent-commerce protocols.

package/dist/audit/keywordList.d.ts

@@ -0,0 +1,40 @@
+import type Database from 'better-sqlite3';
+/**
+ * Demand-first keyword clustering ("list mode") - group a keyword list into topics and
+ * give each cluster a verdict: OWN / WEAK / ABSENT, with the ranking URL.
+ *
+ * Clustering is GSC-bipartite first (free, deterministic, first-party): two keywords
+ * belong together when Google already answers both with the SAME page - the strongest
+ * possible clustering signal, and it costs nothing. Keywords the site has no GSC rows
+ * for are grouped lexically (token Jaccard) - first into the nearest GSC cluster, else
+ * among themselves.
+ *
+ * Verdicts are deterministic from GSC positions: OWN = best position <= 3, WEAK = 4-20,
+ * ABSENT = no ranking page. Impressions stand in for demand (real, first-party); pass
+ * the clusters through keyword_volume for market-level volumes when needed.
+ */
+export interface KeywordCluster {
+    head: string;
+    verdict: 'own' | 'weak' | 'absent';
+    url: string | null;
+    bestPosition: number | null;
+    impressions: number;
+    clicks: number;
+    keywords: {
+        keyword: string;
+        position: number | null;
+        impressions: number;
+        inGsc: boolean;
+    }[];
+}
+export interface KeywordListResult {
+    clusters: KeywordCluster[];
+    totalKeywords: number;
+    inGsc: number;
+    derived: boolean;
+}
+export declare function clusterKeywordList(db: Database.Database, keywordsIn: string[] | undefined, opts?: {
+    maxKeywords?: number;
+    maxDate?: string | null;
+}): KeywordListResult;
+//# sourceMappingURL=keywordList.d.ts.map

package/dist/audit/keywordList.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"keywordList.d.ts","sourceRoot":"","sources":["../../src/audit/keywordList.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,QAAQ,MAAM,gBAAgB,CAAC;AAE3C;;;;;;;;;;;;;GAaG;AAEH,MAAM,WAAW,cAAc;IAC7B,IAAI,EAAE,MAAM,CAAC;IACb,OAAO,EAAE,KAAK,GAAG,MAAM,GAAG,QAAQ,CAAC;IACnC,GAAG,EAAE,MAAM,GAAG,IAAI,CAAC;IACnB,YAAY,EAAE,MAAM,GAAG,IAAI,CAAC;IAC5B,WAAW,EAAE,MAAM,CAAC;IACpB,MAAM,EAAE,MAAM,CAAC;IACf,QAAQ,EAAE;QAAE,OAAO,EAAE,MAAM,CAAC;QAAC,QAAQ,EAAE,MAAM,GAAG,IAAI,CAAC;QAAC,WAAW,EAAE,MAAM,CAAC;QAAC,KAAK,EAAE,OAAO,CAAA;KAAE,EAAE,CAAC;CAC/F;AAED,MAAM,WAAW,iBAAiB;IAChC,QAAQ,EAAE,cAAc,EAAE,CAAC;IAC3B,aAAa,EAAE,MAAM,CAAC;IACtB,KAAK,EAAE,MAAM,CAAC;IACd,OAAO,EAAE,OAAO,CAAC;CAClB;AAYD,wBAAgB,kBAAkB,CAAC,EAAE,EAAE,QAAQ,CAAC,QAAQ,EAAE,UAAU,EAAE,MAAM,EAAE,GAAG,SAAS,EAAE,IAAI,CAAC,EAAE;IAAE,WAAW,CAAC,EAAE,MAAM,CAAC;IAAC,OAAO,CAAC,EAAE,MAAM,GAAG,IAAI,CAAA;CAAE,GAAG,iBAAiB,CAoEvK"}

package/dist/audit/keywordList.js

@@ -0,0 +1,95 @@
+const tokens = (s) => new Set(s.toLowerCase().split(/[^a-z0-9]+/).filter(t => t.length > 1));
+const jaccard = (a, b) => {
+    if (!a.size || !b.size)
+        return 0;
+    let inter = 0;
+    for (const t of a)
+        if (b.has(t))
+            inter++;
+    return inter / (a.size + b.size - inter);
+};
+export function clusterKeywordList(db, keywordsIn, opts) {
+    const cap = opts?.maxKeywords ?? 2000;
+    const derived = !keywordsIn?.length;
+    const ub = opts?.maxDate ? `AND date <= '${opts.maxDate}'` : '';
+    // No list supplied → derive the demand list from the site's own GSC queries (top by impressions).
+    const keywords = derived
+        ? db.prepare(`SELECT query FROM search_analytics WHERE query IS NOT NULL AND date > date('now', '-90 days') ${ub}
+                   GROUP BY query ORDER BY SUM(impressions) DESC LIMIT ?`).all(Math.min(cap, 500)).map(r => r.query)
+        : [...new Set(keywordsIn.map(k => k.trim().toLowerCase()).filter(Boolean))].slice(0, cap);
+    // Best ranking page per keyword (90d, impression-weighted position, top page by impressions).
+    const stmt = db.prepare(`
+    SELECT page_key url, SUM(impressions) imp, SUM(clicks) clk,
+           SUM(position*impressions)*1.0/NULLIF(SUM(impressions),0) pos
+    FROM search_analytics
+    WHERE query = ? AND page_key IS NOT NULL AND date > date('now', '-90 days') ${ub}
+    GROUP BY page_key ORDER BY imp DESC LIMIT 1`);
+    const rows = keywords.map(k => {
+        const r = stmt.get(k);
+        return r && r.imp > 0
+            ? { keyword: k, url: r.url, pos: r.pos != null ? Math.round(r.pos * 10) / 10 : null, imp: r.imp, clk: r.clk, inGsc: true }
+            : { keyword: k, url: null, pos: null, imp: 0, clk: 0, inGsc: false };
+    });
+    // 1) Bipartite: cluster by shared ranking URL.
+    const byUrl = new Map();
+    const orphans = [];
+    for (const r of rows)
+        (r.url ? (byUrl.get(r.url) ?? byUrl.set(r.url, []).get(r.url)) : orphans).push(r);
+    const working = [...byUrl.entries()].map(([url, members]) => ({
+        url, members, tok: new Set(members.flatMap(m => [...tokens(m.keyword)])),
+    }));
+    // 2) Lexical assignment for non-ranking keywords: nearest GSC cluster at Jaccard >= 0.4,
+    //    else greedy-cluster among themselves (same threshold).
+    const lexical = [];
+    for (const o of orphans) {
+        const tok = tokens(o.keyword);
+        let best = null;
+        let bestJ = 0;
+        for (const w of working) {
+            const j = jaccard(tok, w.tok);
+            if (j > bestJ) {
+                bestJ = j;
+                best = w;
+            }
+        }
+        if (best && bestJ >= 0.4) {
+            best.members.push(o);
+            for (const t of tok)
+                best.tok.add(t);
+            continue;
+        }
+        let lex = null;
+        let lexJ = 0;
+        for (const w of lexical) {
+            const j = jaccard(tok, w.tok);
+            if (j > lexJ) {
+                lexJ = j;
+                lex = w;
+            }
+        }
+        if (lex && lexJ >= 0.4) {
+            lex.members.push(o);
+            for (const t of tok)
+                lex.tok.add(t);
+        }
+        else
+            lexical.push({ url: null, members: [o], tok });
+    }
+    const clusters = [...working, ...lexical].map(w => {
+        const withGsc = w.members.filter(m => m.inGsc);
+        const bestPos = withGsc.length ? Math.min(...withGsc.map(m => m.pos ?? Infinity)) : null;
+        const verdict = bestPos == null || !isFinite(bestPos) ? 'absent' : bestPos <= 3 ? 'own' : bestPos <= 20 ? 'weak' : 'weak';
+        const sorted = [...w.members].sort((a, b) => b.imp - a.imp);
+        return {
+            head: sorted[0].keyword,
+            verdict,
+            url: w.url,
+            bestPosition: bestPos != null && isFinite(bestPos) ? bestPos : null,
+            impressions: w.members.reduce((s, m) => s + m.imp, 0),
+            clicks: w.members.reduce((s, m) => s + m.clk, 0),
+            keywords: sorted.map(m => ({ keyword: m.keyword, position: m.pos, impressions: m.imp, inGsc: m.inGsc })),
+        };
+    }).sort((a, b) => b.impressions - a.impressions || b.keywords.length - a.keywords.length);
+    return { clusters, totalKeywords: rows.length, inGsc: rows.filter(r => r.inGsc).length, derived };
+}
+//# sourceMappingURL=keywordList.js.map

package/dist/audit/keywordList.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"keywordList.js","sourceRoot":"","sources":["../../src/audit/keywordList.ts"],"names":[],"mappings":"AAkCA,MAAM,MAAM,GAAG,CAAC,CAAS,EAAe,EAAE,CACxC,IAAI,GAAG,CAAC,CAAC,CAAC,WAAW,EAAE,CAAC,KAAK,CAAC,YAAY,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,CAAC;AAEzE,MAAM,OAAO,GAAG,CAAC,CAAc,EAAE,CAAc,EAAU,EAAE;IACzD,IAAI,CAAC,CAAC,CAAC,IAAI,IAAI,CAAC,CAAC,CAAC,IAAI;QAAE,OAAO,CAAC,CAAC;IACjC,IAAI,KAAK,GAAG,CAAC,CAAC;IACd,KAAK,MAAM,CAAC,IAAI,CAAC;QAAE,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC;YAAE,KAAK,EAAE,CAAC;IACzC,OAAO,KAAK,GAAG,CAAC,CAAC,CAAC,IAAI,GAAG,CAAC,CAAC,IAAI,GAAG,KAAK,CAAC,CAAC;AAC3C,CAAC,CAAC;AAEF,MAAM,UAAU,kBAAkB,CAAC,EAAqB,EAAE,UAAgC,EAAE,IAAwD;IAClJ,MAAM,GAAG,GAAG,IAAI,EAAE,WAAW,IAAI,IAAI,CAAC;IACtC,MAAM,OAAO,GAAG,CAAC,UAAU,EAAE,MAAM,CAAC;IACpC,MAAM,EAAE,GAAG,IAAI,EAAE,OAAO,CAAC,CAAC,CAAC,gBAAgB,IAAI,CAAC,OAAO,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC;IAEhE,kGAAkG;IAClG,MAAM,QAAQ,GAAa,OAAO;QAChC,CAAC,CAAE,EAAE,CAAC,OAAO,CAAC,iGAAiG,EAAE;yEAC5C,CAAC,CAAC,GAAG,CAAC,IAAI,CAAC,GAAG,CAAC,GAAG,EAAE,GAAG,CAAC,CAAyB,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,KAAK,CAAC;QACxI,CAAC,CAAC,CAAC,GAAG,IAAI,GAAG,CAAC,UAAW,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,WAAW,EAAE,CAAC,CAAC,MAAM,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,EAAE,GAAG,CAAC,CAAC;IAE7F,8FAA8F;IAC9F,MAAM,IAAI,GAAG,EAAE,CAAC,OAAO,CAAC;;;;kFAIwD,EAAE;gDACpC,CAAC,CAAC;IAGhD,MAAM,IAAI,GAAU,QAAQ,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE;QACnC,MAAM,CAAC,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,CAA8E,CAAC;QACnG,OAAO,CAAC,IAAI,CAAC,CAAC,GAAG,GAAG,CAAC;YACnB,CAAC,CAAC,EAAE,OAAO,EAAE,CAAC,EAAE,GAAG,EAAE,CAAC,CAAC,GAAG,EAAE,GAAG,EAAE,CAAC,CAAC,GAAG,IAAI,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,GAAG,GAAG,EAAE,CAAC,GAAG,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,GAAG,EAAE,CAAC,CAAC,GAAG,EAAE,GAAG,EAAE,CAAC,CAAC,GAAG,EAAE,KAAK,EAAE,IAAI,EAAE;YAC1H,CAAC,CAAC,EAAE,OAAO,EAAE,CAAC,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG,EAAE,CAAC,EAAE,GAAG,EAAE,CAAC,EAAE,KAAK,EAAE,KAAK,EAAE,CAAC;IACzE,CAAC,CAAC,CAAC;IAEH,+CAA+C;IAC/C,MAAM,KAAK,GAAG,IAAI,GAAG,EAAiB,CAAC;IACvC,MAAM,OAAO,GAAU,EAAE,CAAC;IAC1B,KAAK,MAAM,CAAC,IAAI,IAAI;QAAE,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,CAAC,GAAG,CAAC,IAAI,KAAK,CAAC,GAAG,CAAC,CAAC,CAAC,GAAG,EAAE,EAAE,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,GAAG,CAAE,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;IAGzG,MAAM,OAAO,GAAc,CAAC,GAAG,KAAK,CAAC,OAAO,EAAE,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,GAAG,EAAE,OAAO,CAAC,EAAE,EAAE,CAAC,CAAC;QACvE,GAAG,EAAE,OAAO,EAAE,GAAG,EAAE,IAAI,GAAG,CAAC,OAAO,CAAC,OAAO,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,GAAG,MAAM,CAAC,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC;KACzE,CAAC,CAAC,CAAC;IAEJ,yFAAyF;IACzF,4DAA4D;IAC5D,MAAM,OAAO,GAAc,EAAE,CAAC;IAC9B,KAAK,MAAM,CAAC,IAAI,OAAO,EAAE,CAAC;QACxB,MAAM,GAAG,GAAG,MAAM,CAAC,CAAC,CAAC,OAAO,CAAC,CAAC;QAC9B,IAAI,IAAI,GAAmB,IAAI,CAAC;QAAC,IAAI,KAAK,GAAG,CAAC,CAAC;QAC/C,KAAK,MAAM,CAAC,IAAI,OAAO,EAAE,CAAC;YAAC,MAAM,CAAC,GAAG,OAAO,CAAC,GAAG,EAAE,CAAC,CAAC,GAAG,CAAC,CAAC;YAAC,IAAI,CAAC,GAAG,KAAK,EAAE,CAAC;gBAAC,KAAK,GAAG,CAAC,CAAC;gBAAC,IAAI,GAAG,CAAC,CAAC;YAAC,CAAC;QAAC,CAAC;QACnG,IAAI,IAAI,IAAI,KAAK,IAAI,GAAG,EAAE,CAAC;YAAC,IAAI,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;YAAC,KAAK,MAAM,CAAC,IAAI,GAAG;gBAAE,IAAI,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC;YAAC,SAAS;QAAC,CAAC;QACnG,IAAI,GAAG,GAAmB,IAAI,CAAC;QAAC,IAAI,IAAI,GAAG,CAAC,CAAC;QAC7C,KAAK,MAAM,CAAC,IAAI,OAAO,EAAE,CAAC;YAAC,MAAM,CAAC,GAAG,OAAO,CAAC,GAAG,EAAE,CAAC,CAAC,GAAG,CAAC,CAAC;YAAC,IAAI,CAAC,GAAG,IAAI,EAAE,CAAC;gBAAC,IAAI,GAAG,CAAC,CAAC;gBAAC,GAAG,GAAG,CAAC,CAAC;YAAC,CAAC;QAAC,CAAC;QAChG,IAAI,GAAG,IAAI,IAAI,IAAI,GAAG,EAAE,CAAC;YAAC,GAAG,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;YAAC,KAAK,MAAM,CAAC,IAAI,GAAG;gBAAE,GAAG,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC;QAAC,CAAC;;YAChF,OAAO,CAAC,IAAI,CAAC,EAAE,GAAG,EAAE,IAAI,EAAE,OAAO,EAAE,CAAC,CAAC,CAAC,EAAE,GAAG,EAAE,CAAC,CAAC;IACtD,CAAC;IAED,MAAM,QAAQ,GAAqB,CAAC,GAAG,OAAO,EAAE,GAAG,OAAO,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE;QAClE,MAAM,OAAO,GAAG,CAAC,CAAC,OAAO,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC;QAC/C,MAAM,OAAO,GAAG,OAAO,CAAC,MAAM,CAAC,CAAC,CAAC,IAAI,CAAC,GAAG,CAAC,GAAG,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,GAAG,IAAI,QAAQ,CAAC,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC;QACzF,MAAM,OAAO,GAA8B,OAAO,IAAI,IAAI,IAAI,CAAC,QAAQ,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,QAAQ,CAAC,CAAC,CAAC,OAAO,IAAI,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,OAAO,IAAI,EAAE,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,MAAM,CAAC;QACrJ,MAAM,MAAM,GAAG,CAAC,GAAG,CAAC,CAAC,OAAO,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,GAAG,GAAG,CAAC,CAAC,GAAG,CAAC,CAAC;QAC5D,OAAO;YACL,IAAI,EAAE,MAAM,CAAC,CAAC,CAAC,CAAC,OAAO;YACvB,OAAO;YACP,GAAG,EAAE,CAAC,CAAC,GAAG;YACV,YAAY,EAAE,OAAO,IAAI,IAAI,IAAI,QAAQ,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC,IAAI;YACnE,WAAW,EAAE,CAAC,CAAC,OAAO,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,GAAG,CAAC,CAAC,GAAG,EAAE,CAAC,CAAC;YACrD,MAAM,EAAE,CAAC,CAAC,OAAO,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,GAAG,CAAC,CAAC,GAAG,EAAE,CAAC,CAAC;YAChD,QAAQ,EAAE,MAAM,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,EAAE,OAAO,EAAE,CAAC,CAAC,OAAO,EAAE,QAAQ,EAAE,CAAC,CAAC,GAAG,EAAE,WAAW,EAAE,CAAC,CAAC,GAAG,EAAE,KAAK,EAAE,CAAC,CAAC,KAAK,EAAE,CAAC,CAAC;SACzG,CAAC;IACJ,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,WAAW,GAAG,CAAC,CAAC,WAAW,IAAI,CAAC,CAAC,QAAQ,CAAC,MAAM,GAAG,CAAC,CAAC,QAAQ,CAAC,MAAM,CAAC,CAAC;IAE1F,OAAO,EAAE,QAAQ,EAAE,aAAa,EAAE,IAAI,CAAC,MAAM,EAAE,KAAK,EAAE,IAAI,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,MAAM,EAAE,OAAO,EAAE,CAAC;AACpG,CAAC"}

package/dist/audit/recon.d.ts

@@ -0,0 +1,56 @@
+import type Database from 'better-sqlite3';
+import type { OwnPageView } from '../core/reconFetch.js';
+import type { SerpReconResult, ReconVerdict } from '../core/serpRecon.js';
+/**
+ * Content recon core: pick the pages worth a deep competitive look, and turn what we can
+ * determine WITHOUT judgement (schema gaps, freshness, cannibalisation, SERP format) into
+ * a trackable to-do ledger. The judgement to-dos (what competitors cover that we don't)
+ * are written back by the research session via save_recon_todo.
+ */
+export interface ReconTarget {
+    urlKey: string;
+    query: string;
+    impressions: number;
+    clicks: number;
+    position: number;
+    priorPosition: number | null;
+    slipped: boolean;
+    competingUrls: number;
+}
+/** Pick declining / striking-distance pages: high impressions, currently pos 3-15, top query per page. */
+export declare function selectReconTargets(db: Database.Database, opts: {
+    limit?: number;
+    minImpressions?: number;
+    maxDate: string;
+}): ReconTarget[];
+export interface DraftTodo {
+    action: string;
+    type: string;
+    rationale: string;
+    evidence: Record<string, unknown>;
+    priority: number;
+}
+/** The to-dos we can generate deterministically from our page + the SERP (no judgement). */
+export declare function deterministicTodos(target: ReconTarget, own: OwnPageView, serp: SerpReconResult, verdict: {
+    verdict: ReconVerdict;
+    note: string;
+}, todayIso: string): DraftTodo[];
+export declare function persistReconPage(db: Database.Database, p: {
+    urlKey: string;
+    query: string;
+    verdict: string;
+    verdictNote: string;
+    organicRank: number | null;
+    gscPosition: number;
+    gscImpressions: number;
+    aioPresent: boolean;
+    aioCitesUs: boolean;
+    videoPresent: boolean;
+    hasProductSchema: boolean;
+    schemaTypes: string[];
+    competitors: unknown;
+}): void;
+/** Insert to-dos for a page. `baseline` snapshots the rank/citation state so outcome is measurable.
+ * De-dupes on (url_key, action) so a re-run doesn't pile up identical open items. */
+export declare function insertTodos(db: Database.Database, urlKey: string, query: string, todos: DraftTodo[], baseline: Record<string, unknown>, source: 'auto' | 'research'): number;
+//# sourceMappingURL=recon.d.ts.map

package/dist/audit/recon.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"recon.d.ts","sourceRoot":"","sources":["../../src/audit/recon.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,QAAQ,MAAM,gBAAgB,CAAC;AAC3C,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,uBAAuB,CAAC;AACzD,OAAO,KAAK,EAAE,eAAe,EAAE,YAAY,EAAE,MAAM,sBAAsB,CAAC;AAE1E;;;;;GAKG;AAEH,MAAM,WAAW,WAAW;IAC1B,MAAM,EAAE,MAAM,CAAC;IACf,KAAK,EAAE,MAAM,CAAC;IACd,WAAW,EAAE,MAAM,CAAC;IACpB,MAAM,EAAE,MAAM,CAAC;IACf,QAAQ,EAAE,MAAM,CAAC;IACjB,aAAa,EAAE,MAAM,GAAG,IAAI,CAAC;IAC7B,OAAO,EAAE,OAAO,CAAC;IACjB,aAAa,EAAE,MAAM,CAAC;CACvB;AAED,0GAA0G;AAC1G,wBAAgB,kBAAkB,CAChC,EAAE,EAAE,QAAQ,CAAC,QAAQ,EACrB,IAAI,EAAE;IAAE,KAAK,CAAC,EAAE,MAAM,CAAC;IAAC,cAAc,CAAC,EAAE,MAAM,CAAC;IAAC,OAAO,EAAE,MAAM,CAAA;CAAE,GACjE,WAAW,EAAE,CA6Cf;AAED,MAAM,WAAW,SAAS;IAAG,MAAM,EAAE,MAAM,CAAC;IAAC,IAAI,EAAE,MAAM,CAAC;IAAC,SAAS,EAAE,MAAM,CAAC;IAAC,QAAQ,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;IAAC,QAAQ,EAAE,MAAM,CAAA;CAAE;AAOnI,4FAA4F;AAC5F,wBAAgB,kBAAkB,CAChC,MAAM,EAAE,WAAW,EACnB,GAAG,EAAE,WAAW,EAChB,IAAI,EAAE,eAAe,EACrB,OAAO,EAAE;IAAE,OAAO,EAAE,YAAY,CAAC;IAAC,IAAI,EAAE,MAAM,CAAA;CAAE,EAChD,QAAQ,EAAE,MAAM,GACf,SAAS,EAAE,CAsDb;AAED,wBAAgB,gBAAgB,CAAC,EAAE,EAAE,QAAQ,CAAC,QAAQ,EAAE,CAAC,EAAE;IACzD,MAAM,EAAE,MAAM,CAAC;IAAC,KAAK,EAAE,MAAM,CAAC;IAAC,OAAO,EAAE,MAAM,CAAC;IAAC,WAAW,EAAE,MAAM,CAAC;IACpE,WAAW,EAAE,MAAM,GAAG,IAAI,CAAC;IAAC,WAAW,EAAE,MAAM,CAAC;IAAC,cAAc,EAAE,MAAM,CAAC;IACxE,UAAU,EAAE,OAAO,CAAC;IAAC,UAAU,EAAE,OAAO,CAAC;IAAC,YAAY,EAAE,OAAO,CAAC;IAChE,gBAAgB,EAAE,OAAO,CAAC;IAAC,WAAW,EAAE,MAAM,EAAE,CAAC;IAAC,WAAW,EAAE,OAAO,CAAC;CACxE,GAAG,IAAI,CAiBP;AAED;qFACqF;AACrF,wBAAgB,WAAW,CAAC,EAAE,EAAE,QAAQ,CAAC,QAAQ,EAAE,MAAM,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,EAAE,KAAK,EAAE,SAAS,EAAE,EAClG,QAAQ,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,EAAE,MAAM,EAAE,MAAM,GAAG,UAAU,GAAG,MAAM,CAcxE"}

package/dist/audit/recon.js

@@ -0,0 +1,134 @@
+/** Pick declining / striking-distance pages: high impressions, currently pos 3-15, top query per page. */
+export function selectReconTargets(db, opts) {
+    const { limit = 5, minImpressions = 300, maxDate } = opts;
+    const rows = db.prepare(`
+    WITH cur AS (
+      SELECT page_key, query, SUM(impressions) imp, SUM(clicks) clk,
+             SUM(position*impressions)*1.0/NULLIF(SUM(impressions),0) pos
+      FROM search_analytics
+      WHERE page_key IS NOT NULL AND query IS NOT NULL
+        AND date > date(?, '-28 days') AND date <= ?
+      GROUP BY page_key, query
+    ),
+    ranked AS (SELECT *, ROW_NUMBER() OVER (PARTITION BY page_key ORDER BY imp DESC) rn FROM cur),
+    prior AS (
+      SELECT page_key, query, SUM(position*impressions)*1.0/NULLIF(SUM(impressions),0) pos
+      FROM search_analytics
+      WHERE page_key IS NOT NULL AND query IS NOT NULL
+        AND date > date(?, '-56 days') AND date <= date(?, '-28 days')
+      GROUP BY page_key, query
+    )
+    SELECT r.page_key, r.query, r.imp, r.clk, r.pos, p.pos AS prior_pos
+    FROM ranked r
+    LEFT JOIN prior p ON p.page_key = r.page_key AND p.query = r.query
+    WHERE r.rn = 1 AND r.pos BETWEEN 3 AND 15 AND r.imp >= ?
+    ORDER BY r.imp DESC
+    LIMIT ?`).all(maxDate, maxDate, maxDate, maxDate, minImpressions, limit);
+    const cannStmt = db.prepare(`
+    SELECT COUNT(*) n FROM (
+      SELECT page_key FROM search_analytics
+      WHERE query = ? AND page_key IS NOT NULL AND date > date(?, '-28 days') AND date <= ?
+      GROUP BY page_key HAVING SUM(impressions) >= 10
+    )`);
+    return rows.map(r => {
+        const pos = Math.round(r.pos * 10) / 10;
+        const priorPosition = r.prior_pos != null ? Math.round(r.prior_pos * 10) / 10 : null;
+        const cann = cannStmt.get(r.query, maxDate, maxDate);
+        return {
+            urlKey: r.page_key, query: r.query, impressions: r.imp, clicks: r.clk, position: pos,
+            priorPosition,
+            slipped: priorPosition != null && pos > priorPosition + 0.5,
+            competingUrls: cann.n,
+        };
+    });
+}
+const TYPE_WEIGHT = {
+    'content-gap': 1.0, schema: 0.9, research: 0.8, cannibalisation: 0.7, format: 0.6, freshness: 0.3,
+};
+const looksCommercial = (q) => /\b(best|top|vs|review|compare|comparison|cheap|budget|buy)\b/i.test(q);
+/** The to-dos we can generate deterministically from our page + the SERP (no judgement). */
+export function deterministicTodos(target, own, serp, verdict, todayIso) {
+    const out = [];
+    const w = (type) => Math.round(target.impressions * (TYPE_WEIGHT[type] ?? 0.5));
+    // Verdict-driven headline action.
+    if (verdict.verdict === 'accuracy-or-freshness') {
+        out.push({ action: 'Update the facts to current and mark them up so they are machine-liftable', type: 'schema',
+            rationale: `You rank organic #${serp.ourOrganicRank} but the AI Overview does not cite you — a data-accuracy/freshness signal.`,
+            evidence: { organicRank: serp.ourOrganicRank, aioCitesUs: false }, priority: w('schema') + 1 });
+    }
+    // Schema: commercial/listicle page without Product/Review markup.
+    if (!own.hasProductOrReview && looksCommercial(target.query)) {
+        out.push({ action: 'Add Product + Review/AggregateRating schema per item, with current specs and price',
+            type: 'schema', rationale: 'A comparison/review page with no item-level markup cannot be lifted into rich results or an AI Overview citation cleanly.',
+            evidence: { schemaTypes: own.jsonLdTypes, hasProductOrReview: false }, priority: w('schema') });
+    }
+    // Freshness: stale dateModified (older than ~90 days from today).
+    if (own.dateModified) {
+        const ageDays = Math.round((Date.parse(todayIso) - Date.parse(own.dateModified)) / 86400000);
+        if (ageDays > 90) {
+            out.push({ action: `Refresh the content (dateModified is ${ageDays} days old) — genuinely update, don't just restamp`,
+                type: 'freshness', rationale: 'Stale content loses ground on freshness-sensitive queries; a real update (not a date bump) is the lever.',
+                evidence: { dateModified: own.dateModified, ageDays }, priority: w('freshness') });
+        }
+    }
+    // Cannibalisation.
+    if (target.competingUrls >= 2) {
+        out.push({ action: `Consolidate cannibalisation — ${target.competingUrls} of your URLs rank for "${target.query}"`,
+            type: 'cannibalisation', rationale: 'Multiple own URLs competing for one query split signals; pick a canonical winner.',
+            evidence: { competingUrls: target.competingUrls, query: target.query }, priority: w('cannibalisation') });
+    }
+    // Format: AIO/SERP leans on video.
+    const ytRefs = serp.aioReferences.filter(r => /youtube|youtu\.be/.test(r.domain)).length;
+    if ((serp.videoPresent || ytRefs >= 2)) {
+        out.push({ action: 'Produce a companion video',
+            type: 'format', rationale: `The SERP favours video${ytRefs ? ` and ${ytRefs} of the AI Overview's references are YouTube` : ''}.`,
+            evidence: { videoPresent: serp.videoPresent, youtubeAioRefs: ytRefs }, priority: w('format') });
+    }
+    // Research pointer: hand the session the competitor set to diff (coverage + originality).
+    const competitors = [
+        ...serp.organicAbove.map(o => ({ rank: o.rank, url: o.url, kind: 'organic-above' })),
+        ...serp.aioReferences.slice(0, 6).map(r => ({ rank: r.rank, url: r.url, kind: 'aio-reference' })),
+        ...serp.videoItems.slice(0, 3).map(v => ({ url: v.url, kind: 'video' })),
+    ];
+    out.push({ action: 'Diff our coverage against the competitors Google rewards — add table-stakes we lack, amplify what only we have',
+        type: 'research', rationale: 'Scrape the ranking competitors and transcribe the ranking videos, then compare to our sections.',
+        evidence: { competitors }, priority: w('research') });
+    return out;
+}
+export function persistReconPage(db, p) {
+    db.prepare(`INSERT INTO recon_page
+    (url_key, query, verdict, verdict_note, organic_rank, gsc_position, gsc_impressions,
+     aio_present, aio_cites_us, video_present, has_product_schema, schema_types, competitors, researched_at)
+    VALUES (@urlKey,@query,@verdict,@verdictNote,@organicRank,@gscPosition,@gscImpressions,
+     @aioPresent,@aioCitesUs,@videoPresent,@hasProductSchema,@schemaTypes,@competitors, datetime('now'))
+    ON CONFLICT(url_key) DO UPDATE SET query=excluded.query, verdict=excluded.verdict, verdict_note=excluded.verdict_note,
+     organic_rank=excluded.organic_rank, gsc_position=excluded.gsc_position, gsc_impressions=excluded.gsc_impressions,
+     aio_present=excluded.aio_present, aio_cites_us=excluded.aio_cites_us, video_present=excluded.video_present,
+     has_product_schema=excluded.has_product_schema, schema_types=excluded.schema_types,
+     competitors=excluded.competitors, researched_at=datetime('now')`).run({
+        urlKey: p.urlKey, query: p.query, verdict: p.verdict, verdictNote: p.verdictNote,
+        organicRank: p.organicRank, gscPosition: p.gscPosition, gscImpressions: p.gscImpressions,
+        aioPresent: p.aioPresent ? 1 : 0, aioCitesUs: p.aioCitesUs ? 1 : 0, videoPresent: p.videoPresent ? 1 : 0,
+        hasProductSchema: p.hasProductSchema ? 1 : 0, schemaTypes: JSON.stringify(p.schemaTypes),
+        competitors: JSON.stringify(p.competitors),
+    });
+}
+/** Insert to-dos for a page. `baseline` snapshots the rank/citation state so outcome is measurable.
+ * De-dupes on (url_key, action) so a re-run doesn't pile up identical open items. */
+export function insertTodos(db, urlKey, query, todos, baseline, source) {
+    const exists = db.prepare(`SELECT 1 FROM recon_todo WHERE url_key=? AND action=? AND status NOT IN ('shipped','dismissed')`);
+    const ins = db.prepare(`INSERT INTO recon_todo (url_key, query, action, type, rationale, evidence, priority, source, baseline)
+    VALUES (?,?,?,?,?,?,?,?,?)`);
+    let n = 0;
+    const tx = db.transaction(() => {
+        for (const t of todos) {
+            if (exists.get(urlKey, t.action))
+                continue;
+            ins.run(urlKey, query, t.action, t.type, t.rationale, JSON.stringify(t.evidence), t.priority, source, JSON.stringify(baseline));
+            n++;
+        }
+    });
+    tx();
+    return n;
+}
+//# sourceMappingURL=recon.js.map

package/dist/audit/recon.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"recon.js","sourceRoot":"","sources":["../../src/audit/recon.ts"],"names":[],"mappings":"AAsBA,0GAA0G;AAC1G,MAAM,UAAU,kBAAkB,CAChC,EAAqB,EACrB,IAAkE;IAElE,MAAM,EAAE,KAAK,GAAG,CAAC,EAAE,cAAc,GAAG,GAAG,EAAE,OAAO,EAAE,GAAG,IAAI,CAAC;IAC1D,MAAM,IAAI,GAAG,EAAE,CAAC,OAAO,CAAC;;;;;;;;;;;;;;;;;;;;;;YAsBd,CAAC,CAAC,GAAG,CAAC,OAAO,EAAE,OAAO,EAAE,OAAO,EAAE,OAAO,EAAE,cAAc,EAAE,KAAK,CAC+B,CAAC;IAEzG,MAAM,QAAQ,GAAG,EAAE,CAAC,OAAO,CAAC;;;;;MAKxB,CAAC,CAAC;IAEN,OAAO,IAAI,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE;QAClB,MAAM,GAAG,GAAG,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,GAAG,GAAG,EAAE,CAAC,GAAG,EAAE,CAAC;QACxC,MAAM,aAAa,GAAG,CAAC,CAAC,SAAS,IAAI,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,SAAS,GAAG,EAAE,CAAC,GAAG,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC;QACrF,MAAM,IAAI,GAAG,QAAQ,CAAC,GAAG,CAAC,CAAC,CAAC,KAAK,EAAE,OAAO,EAAE,OAAO,CAAkB,CAAC;QACtE,OAAO;YACL,MAAM,EAAE,CAAC,CAAC,QAAQ,EAAE,KAAK,EAAE,CAAC,CAAC,KAAK,EAAE,WAAW,EAAE,CAAC,CAAC,GAAG,EAAE,MAAM,EAAE,CAAC,CAAC,GAAG,EAAE,QAAQ,EAAE,GAAG;YACpF,aAAa;YACb,OAAO,EAAE,aAAa,IAAI,IAAI,IAAI,GAAG,GAAG,aAAa,GAAG,GAAG;YAC3D,aAAa,EAAE,IAAI,CAAC,CAAC;SACtB,CAAC;IACJ,CAAC,CAAC,CAAC;AACL,CAAC;AAID,MAAM,WAAW,GAA2B;IAC1C,aAAa,EAAE,GAAG,EAAE,MAAM,EAAE,GAAG,EAAE,QAAQ,EAAE,GAAG,EAAE,eAAe,EAAE,GAAG,EAAE,MAAM,EAAE,GAAG,EAAE,SAAS,EAAE,GAAG;CAClG,CAAC;AACF,MAAM,eAAe,GAAG,CAAC,CAAS,EAAW,EAAE,CAAC,+DAA+D,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;AAExH,4FAA4F;AAC5F,MAAM,UAAU,kBAAkB,CAChC,MAAmB,EACnB,GAAgB,EAChB,IAAqB,EACrB,OAAgD,EAChD,QAAgB;IAEhB,MAAM,GAAG,GAAgB,EAAE,CAAC;IAC5B,MAAM,CAAC,GAAG,CAAC,IAAY,EAAE,EAAE,CAAC,IAAI,CAAC,KAAK,CAAC,MAAM,CAAC,WAAW,GAAG,CAAC,WAAW,CAAC,IAAI,CAAC,IAAI,GAAG,CAAC,CAAC,CAAC;IAExF,kCAAkC;IAClC,IAAI,OAAO,CAAC,OAAO,KAAK,uBAAuB,EAAE,CAAC;QAChD,GAAG,CAAC,IAAI,CAAC,EAAE,MAAM,EAAE,2EAA2E,EAAE,IAAI,EAAE,QAAQ;YAC5G,SAAS,EAAE,qBAAqB,IAAI,CAAC,cAAc,4EAA4E;YAC/H,QAAQ,EAAE,EAAE,WAAW,EAAE,IAAI,CAAC,cAAc,EAAE,UAAU,EAAE,KAAK,EAAE,EAAE,QAAQ,EAAE,CAAC,CAAC,QAAQ,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC;IACpG,CAAC;IAED,kEAAkE;IAClE,IAAI,CAAC,GAAG,CAAC,kBAAkB,IAAI,eAAe,CAAC,MAAM,CAAC,KAAK,CAAC,EAAE,CAAC;QAC7D,GAAG,CAAC,IAAI,CAAC,EAAE,MAAM,EAAE,oFAAoF;YACrG,IAAI,EAAE,QAAQ,EAAE,SAAS,EAAE,2HAA2H;YACtJ,QAAQ,EAAE,EAAE,WAAW,EAAE,GAAG,CAAC,WAAW,EAAE,kBAAkB,EAAE,KAAK,EAAE,EAAE,QAAQ,EAAE,CAAC,CAAC,QAAQ,CAAC,EAAE,CAAC,CAAC;IACpG,CAAC;IAED,kEAAkE;IAClE,IAAI,GAAG,CAAC,YAAY,EAAE,CAAC;QACrB,MAAM,OAAO,GAAG,IAAI,CAAC,KAAK,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,QAAQ,CAAC,GAAG,IAAI,CAAC,KAAK,CAAC,GAAG,CAAC,YAAY,CAAC,CAAC,GAAG,QAAQ,CAAC,CAAC;QAC7F,IAAI,OAAO,GAAG,EAAE,EAAE,CAAC;YACjB,GAAG,CAAC,IAAI,CAAC,EAAE,MAAM,EAAE,wCAAwC,OAAO,mDAAmD;gBACnH,IAAI,EAAE,WAAW,EAAE,SAAS,EAAE,0GAA0G;gBACxI,QAAQ,EAAE,EAAE,YAAY,EAAE,GAAG,CAAC,YAAY,EAAE,OAAO,EAAE,EAAE,QAAQ,EAAE,CAAC,CAAC,WAAW,CAAC,EAAE,CAAC,CAAC;QACvF,CAAC;IACH,CAAC;IAED,mBAAmB;IACnB,IAAI,MAAM,CAAC,aAAa,IAAI,CAAC,EAAE,CAAC;QAC9B,GAAG,CAAC,IAAI,CAAC,EAAE,MAAM,EAAE,iCAAiC,MAAM,CAAC,aAAa,2BAA2B,MAAM,CAAC,KAAK,GAAG;YAChH,IAAI,EAAE,iBAAiB,EAAE,SAAS,EAAE,mFAAmF;YACvH,QAAQ,EAAE,EAAE,aAAa,EAAE,MAAM,CAAC,aAAa,EAAE,KAAK,EAAE,MAAM,CAAC,KAAK,EAAE,EAAE,QAAQ,EAAE,CAAC,CAAC,iBAAiB,CAAC,EAAE,CAAC,CAAC;IAC9G,CAAC;IAED,mCAAmC;IACnC,MAAM,MAAM,GAAG,IAAI,CAAC,aAAa,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,mBAAmB,CAAC,IAAI,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,MAAM,CAAC;IACzF,IAAI,CAAC,IAAI,CAAC,YAAY,IAAI,MAAM,IAAI,CAAC,CAAC,EAAE,CAAC;QACvC,GAAG,CAAC,IAAI,CAAC,EAAE,MAAM,EAAE,2BAA2B;YAC5C,IAAI,EAAE,QAAQ,EAAE,SAAS,EAAE,yBAAyB,MAAM,CAAC,CAAC,CAAC,QAAQ,MAAM,8CAA8C,CAAC,CAAC,CAAC,EAAE,GAAG;YACjI,QAAQ,EAAE,EAAE,YAAY,EAAE,IAAI,CAAC,YAAY,EAAE,cAAc,EAAE,MAAM,EAAE,EAAE,QAAQ,EAAE,CAAC,CAAC,QAAQ,CAAC,EAAE,CAAC,CAAC;IACpG,CAAC;IAED,0FAA0F;IAC1F,MAAM,WAAW,GAAG;QAClB,GAAG,IAAI,CAAC,YAAY,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,EAAE,IAAI,EAAE,CAAC,CAAC,IAAI,EAAE,GAAG,EAAE,CAAC,CAAC,GAAG,EAAE,IAAI,EAAE,eAAe,EAAE,CAAC,CAAC;QACpF,GAAG,IAAI,CAAC,aAAa,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,EAAE,IAAI,EAAE,CAAC,CAAC,IAAI,EAAE,GAAG,EAAE,CAAC,CAAC,GAAG,EAAE,IAAI,EAAE,eAAe,EAAE,CAAC,CAAC;QACjG,GAAG,IAAI,CAAC,UAAU,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,EAAE,GAAG,EAAE,CAAC,CAAC,GAAG,EAAE,IAAI,EAAE,OAAO,EAAE,CAAC,CAAC;KACzE,CAAC;IACF,GAAG,CAAC,IAAI,CAAC,EAAE,MAAM,EAAE,gHAAgH;QACjI,IAAI,EAAE,UAAU,EAAE,SAAS,EAAE,iGAAiG;QAC9H,QAAQ,EAAE,EAAE,WAAW,EAAE,EAAE,QAAQ,EAAE,CAAC,CAAC,UAAU,CAAC,EAAE,CAAC,CAAC;IAExD,OAAO,GAAG,CAAC;AACb,CAAC;AAED,MAAM,UAAU,gBAAgB,CAAC,EAAqB,EAAE,CAKvD;IACC,EAAE,CAAC,OAAO,CAAC;;;;;;;;;qEASwD,CAAC,CAAC,GAAG,CAAC;QACrE,MAAM,EAAE,CAAC,CAAC,MAAM,EAAE,KAAK,EAAE,CAAC,CAAC,KAAK,EAAE,OAAO,EAAE,CAAC,CAAC,OAAO,EAAE,WAAW,EAAE,CAAC,CAAC,WAAW;QAChF,WAAW,EAAE,CAAC,CAAC,WAAW,EAAE,WAAW,EAAE,CAAC,CAAC,WAAW,EAAE,cAAc,EAAE,CAAC,CAAC,cAAc;QACxF,UAAU,EAAE,CAAC,CAAC,UAAU,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAE,UAAU,EAAE,CAAC,CAAC,UAAU,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAE,YAAY,EAAE,CAAC,CAAC,YAAY,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;QACxG,gBAAgB,EAAE,CAAC,CAAC,gBAAgB,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAE,WAAW,EAAE,IAAI,CAAC,SAAS,CAAC,CAAC,CAAC,WAAW,CAAC;QACxF,WAAW,EAAE,IAAI,CAAC,SAAS,CAAC,CAAC,CAAC,WAAW,CAAC;KAC3C,CAAC,CAAC;AACP,CAAC;AAED;qFACqF;AACrF,MAAM,UAAU,WAAW,CAAC,EAAqB,EAAE,MAAc,EAAE,KAAa,EAAE,KAAkB,EAClG,QAAiC,EAAE,MAA2B;IAC9D,MAAM,MAAM,GAAG,EAAE,CAAC,OAAO,CAAC,iGAAiG,CAAC,CAAC;IAC7H,MAAM,GAAG,GAAG,EAAE,CAAC,OAAO,CAAC;+BACM,CAAC,CAAC;IAC/B,IAAI,CAAC,GAAG,CAAC,CAAC;IACV,MAAM,EAAE,GAAG,EAAE,CAAC,WAAW,CAAC,GAAG,EAAE;QAC7B,KAAK,MAAM,CAAC,IAAI,KAAK,EAAE,CAAC;YACtB,IAAI,MAAM,CAAC,GAAG,CAAC,MAAM,EAAE,CAAC,CAAC,MAAM,CAAC;gBAAE,SAAS;YAC3C,GAAG,CAAC,GAAG,CAAC,MAAM,EAAE,KAAK,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,SAAS,EAAE,IAAI,CAAC,SAAS,CAAC,CAAC,CAAC,QAAQ,CAAC,EAAE,CAAC,CAAC,QAAQ,EAAE,MAAM,EAAE,IAAI,CAAC,SAAS,CAAC,QAAQ,CAAC,CAAC,CAAC;YAChI,CAAC,EAAE,CAAC;QACN,CAAC;IACH,CAAC,CAAC,CAAC;IACH,EAAE,EAAE,CAAC;IACL,OAAO,CAAC,CAAC;AACX,CAAC"}

package/dist/core/AuditDatabase.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"AuditDatabase.d.ts","sourceRoot":"","sources":["../../src/core/AuditDatabase.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;GAUG;AACH,OAAO,QAAQ,MAAM,gBAAgB,CAAC;AAItC,MAAM,MAAM,YAAY,GAAG,GAAG,GAAG,GAAG,GAAG,GAAG,GAAG,GAAG,GAAG,GAAG,CAAC;AACvD,MAAM,MAAM,QAAQ,GAAG,MAAM,GAAG,MAAM,GAAG,KAAK,GAAG,KAAK,GAAG,MAAM,CAAC;AAEhE,qBAAa,aAAa;IACxB,QAAQ,CAAC,EAAE,EAAE,QAAQ,CAAC,QAAQ,CAAC;gBAEnB,MAAM,EAAE,MAAM;IAgB1B,OAAO,CAAC,gBAAgB;IAgYxB,2EAA2E;IAC3E,OAAO,CAAC,OAAO;IAyDf,cAAc,CAAC,OAAO,EAAE,MAAM,EAAE,QAAQ,EAAE,MAAM,EAAE,eAAe,CAAC,EAAE,MAAM,GAAG,IAAI;IAWjF,WAAW,CAAC,OAAO,EAAE,MAAM,GAAG;QAAE,QAAQ,EAAE,MAAM,CAAC;QAAC,SAAS,EAAE,MAAM,CAAC;QAAC,cAAc,EAAE,MAAM,GAAG,IAAI,CAAA;KAAE,GAAG,SAAS;IAMhH,2FAA2F;IAC3F,cAAc,CAAC,OAAO,EAAE,MAAM,EAAE,QAAQ,EAAE,MAAM,GAAG,MAAM,GAAG,IAAI;IAMhE,iFAAiF;IACjF,cAAc,CAAC,OAAO,EAAE,MAAM,GAAG,MAAM,GAAG,MAAM,GAAG,SAAS;IAO5D,KAAK,IAAI,IAAI;CAGd"}
+{"version":3,"file":"AuditDatabase.d.ts","sourceRoot":"","sources":["../../src/core/AuditDatabase.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;GAUG;AACH,OAAO,QAAQ,MAAM,gBAAgB,CAAC;AAItC,MAAM,MAAM,YAAY,GAAG,GAAG,GAAG,GAAG,GAAG,GAAG,GAAG,GAAG,GAAG,GAAG,CAAC;AACvD,MAAM,MAAM,QAAQ,GAAG,MAAM,GAAG,MAAM,GAAG,KAAK,GAAG,KAAK,GAAG,MAAM,CAAC;AAEhE,qBAAa,aAAa;IACxB,QAAQ,CAAC,EAAE,EAAE,QAAQ,CAAC,QAAQ,CAAC;gBAEnB,MAAM,EAAE,MAAM;IAgB1B,OAAO,CAAC,gBAAgB;IAwaxB,2EAA2E;IAC3E,OAAO,CAAC,OAAO;IAyDf,cAAc,CAAC,OAAO,EAAE,MAAM,EAAE,QAAQ,EAAE,MAAM,EAAE,eAAe,CAAC,EAAE,MAAM,GAAG,IAAI;IAWjF,WAAW,CAAC,OAAO,EAAE,MAAM,GAAG;QAAE,QAAQ,EAAE,MAAM,CAAC;QAAC,SAAS,EAAE,MAAM,CAAC;QAAC,cAAc,EAAE,MAAM,GAAG,IAAI,CAAA;KAAE,GAAG,SAAS;IAMhH,2FAA2F;IAC3F,cAAc,CAAC,OAAO,EAAE,MAAM,EAAE,QAAQ,EAAE,MAAM,GAAG,MAAM,GAAG,IAAI;IAMhE,iFAAiF;IACjF,cAAc,CAAC,OAAO,EAAE,MAAM,GAAG,MAAM,GAAG,MAAM,GAAG,SAAS;IAO5D,KAAK,IAAI,IAAI;CAGd"}

package/dist/core/AuditDatabase.js

@@ -392,6 +392,45 @@ export class AuditDatabase {
         PRIMARY KEY (qid, related_qid, relation)
       );
       CREATE INDEX IF NOT EXISTS idx_entity_edge_related ON entity_edge(related_qid);
+    `);
+        // ── Content recon (recon_targets) — per-page SERP/AIO classification + a trackable
+        //    to-do ledger with baseline→outcome so a fix's effect on rank/citation is measurable.
+        this.db.exec(`
+      CREATE TABLE IF NOT EXISTS recon_page (
+        url_key TEXT PRIMARY KEY,         -- OUR page under recon
+        query TEXT,                       -- the primary money query analysed
+        verdict TEXT,                     -- defend-and-deepen | accuracy-or-freshness | consolidate-weak-page | competitive-gap
+        verdict_note TEXT,
+        organic_rank INTEGER,             -- our live organic rank (DataForSEO), distinct from GSC avg
+        gsc_position REAL,                -- GSC blended average position (for contrast)
+        gsc_impressions INTEGER,
+        aio_present INTEGER DEFAULT 0,
+        aio_cites_us INTEGER DEFAULT 0,
+        video_present INTEGER DEFAULT 0,
+        has_product_schema INTEGER DEFAULT 0,
+        schema_types TEXT,                -- JSON string[]
+        competitors TEXT,                 -- JSON { organicAbove[], aioReferences[], videoItems[] }
+        researched_at TEXT DEFAULT (datetime('now'))
+      );
+      CREATE TABLE IF NOT EXISTS recon_todo (
+        id INTEGER PRIMARY KEY AUTOINCREMENT,
+        url_key TEXT NOT NULL,
+        query TEXT,
+        action TEXT NOT NULL,             -- the to-do ("Add a Meta Quest 3S section")
+        type TEXT,                        -- content-gap | schema | freshness | format | cannibalisation | originality
+        rationale TEXT,
+        evidence TEXT,                    -- JSON datapoint backing the action
+        priority REAL DEFAULT 0,
+        status TEXT DEFAULT 'open',       -- open | researching | drafted | shipped | dismissed
+        source TEXT DEFAULT 'auto',       -- auto (deterministic) | research (session-discovered)
+        baseline TEXT,                    -- JSON snapshot at creation (rank/citation/position)
+        outcome TEXT,                     -- JSON snapshot after 'shipped' (re-measured)
+        notes TEXT,                       -- append-only annotation log (timestamped lines)
+        created_at TEXT DEFAULT (datetime('now')),
+        updated_at TEXT DEFAULT (datetime('now'))
+      );
+      CREATE INDEX IF NOT EXISTS idx_recon_todo_url ON recon_todo(url_key);
+      CREATE INDEX IF NOT EXISTS idx_recon_todo_status ON recon_todo(status);
     `);
         this.migrate();
     }

package/dist/core/AuditDatabase.js.map

@@ -1 +1 @@
-{"version":3,"file":"AuditDatabase.js","sourceRoot":"","sources":["../../src/core/AuditDatabase.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;GAUG;AACH,OAAO,QAAQ,MAAM,gBAAgB,CAAC;AACtC,OAAO,EAAE,SAAS,EAAE,MAAM,SAAS,CAAC;AACpC,OAAO,EAAE,OAAO,EAAE,MAAM,WAAW,CAAC;AAKpC,MAAM,OAAO,aAAa;IACf,EAAE,CAAoB;IAE/B,YAAY,MAAc;QACxB,SAAS,CAAC,OAAO,CAAC,MAAM,CAAC,EAAE,EAAE,SAAS,EAAE,IAAI,EAAE,CAAC,CAAC;QAChD,IAAI,CAAC,EAAE,GAAG,IAAI,QAAQ,CAAC,MAAM,CAAC,CAAC;QAC/B,IAAI,CAAC,EAAE,CAAC,MAAM,CAAC,oBAAoB,CAAC,CAAC;QACrC,IAAI,CAAC,EAAE,CAAC,MAAM,CAAC,sBAAsB,CAAC,CAAC,CAAC,iDAAiD;QACzF,IAAI,CAAC,EAAE,CAAC,MAAM,CAAC,sBAAsB,CAAC,CAAC,CAAE,qEAAqE;QAC9G,IAAI,CAAC,EAAE,CAAC,MAAM,CAAC,mBAAmB,CAAC,CAAC;QACpC,6FAA6F;QAC7F,8FAA8F;QAC9F,gGAAgG;QAChG,IAAI,CAAC,EAAE,CAAC,MAAM,CAAC,sBAAsB,CAAC,CAAC,CAAE,qCAAqC;QAC9E,IAAI,CAAC,EAAE,CAAC,MAAM,CAAC,uBAAuB,CAAC,CAAC,CAAC,2BAA2B;QACpE,IAAI,CAAC,EAAE,CAAC,MAAM,CAAC,qBAAqB,CAAC,CAAC,CAAG,kDAAkD;QAC3F,IAAI,CAAC,gBAAgB,EAAE,CAAC;IAC1B,CAAC;IAEO,gBAAgB;QACtB,0EAA0E;QAC1E,IAAI,CAAC,EAAE,CAAC,IAAI,CAAC;;;;;;;;;;;;KAYZ,CAAC,CAAC;QAEH,0EAA0E;QAC1E,IAAI,CAAC,EAAE,CAAC,IAAI,CAAC;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;KAuCZ,CAAC,CAAC;QAEH,gFAAgF;QAChF,IAAI,CAAC,EAAE,CAAC,IAAI,CAAC;;;;;;;;;;;;;;;;;;;KAmBZ,CAAC,CAAC;QAEH,IAAI,CAAC,EAAE,CAAC,IAAI,CAAC;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;KA2EZ,CAAC,CAAC;QAEH,2FAA2F;QAC3F,4FAA4F;QAC5F,2FAA2F;QAC3F,IAAI,CAAC,EAAE,CAAC,IAAI,CAAC;;;;;;;;;;;;;;;;;;;;KAoBZ,CAAC,CAAC;QAEH,4FAA4F;QAC5F,gFAAgF;QAChF,IAAI,CAAC,EAAE,CAAC,IAAI,CAAC;;;;;;;;;KASZ,CAAC,CAAC;QAEH,IAAI,CAAC,EAAE,CAAC,IAAI,CAAC;;;;;;;;;;;;;;;;;;KAkBZ,CAAC,CAAC;QAEH,IAAI,CAAC,EAAE,CAAC,IAAI,CAAC;;;;;;;;;;;KAWZ,CAAC,CAAC;QAEH,2EAA2E;QAC3E,IAAI,CAAC,EAAE,CAAC,IAAI,CAAC;;;;;;;;;;;;;;;;;;KAkBZ,CAAC,CAAC;QAEH,2EAA2E;QAC3E,IAAI,CAAC,EAAE,CAAC,IAAI,CAAC;;;;;;;;;;;;KAYZ,CAAC,CAAC;QAEH,0EAA0E;QAC1E,IAAI,CAAC,EAAE,CAAC,IAAI,CAAC;;;;;;;;;;;;KAYZ,CAAC,CAAC;QAEH,IAAI,CAAC,EAAE,CAAC,IAAI,CAAC;;;;;;;;;;;;;;;;;;;;KAoBZ,CAAC,CAAC;QAEH,iFAAiF;QACjF,IAAI,CAAC,EAAE,CAAC,IAAI,CAAC;;;;;;;;;;;KAWZ,CAAC,CAAC;QAEH,0EAA0E;QAC1E,IAAI,CAAC,EAAE,CAAC,IAAI,CAAC;;;;;;;KAOZ,CAAC,CAAC;QAEH,+EAA+E;QAC/E,IAAI,CAAC,EAAE,CAAC,IAAI,CAAC;;;;;;;;;KASZ,CAAC,CAAC;QAEH,4EAA4E;QAC5E,4EAA4E;QAC5E,IAAI,CAAC,EAAE,CAAC,IAAI,CAAC;;;;;;;KAOZ,CAAC,CAAC;QACH,iFAAiF;QACjF,IAAI,CAAC,EAAE,CAAC,IAAI,CAAC;;;;;;;;;;;KAWZ,CAAC,CAAC;QAEH,sFAAsF;QACtF,IAAI,CAAC,EAAE,CAAC,IAAI,CAAC;;;;;;;;;;;;;;;;;KAiBZ,CAAC,CAAC;QAEH,IAAI,CAAC,OAAO,EAAE,CAAC;IACjB,CAAC;IAED,2EAA2E;IACnE,OAAO;QACb,MAAM,IAAI,GAAG,CAAC,KAAa,EAAe,EAAE,CAC1C,IAAI,GAAG,CAAE,IAAI,CAAC,EAAE,CAAC,OAAO,CAAC,qBAAqB,KAAK,GAAG,CAAC,CAAC,GAAG,EAAyB,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC;QACzG,MAAM,MAAM,GAAG,CAAC,KAAa,EAAE,GAAW,EAAE,GAAW,EAAQ,EAAE;YAC/D,IAAI,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,GAAG,CAAC,GAAG,CAAC;gBAAE,IAAI,CAAC,EAAE,CAAC,IAAI,CAAC,eAAe,KAAK,eAAe,GAAG,EAAE,CAAC,CAAC;QACpF,CAAC,CAAC;QACF,MAAM,CAAC,OAAO,EAAE,aAAa,EAAE,+BAA+B,CAAC,CAAC;QAChE,MAAM,CAAC,OAAO,EAAE,oBAAoB,EAAE,sCAAsC,CAAC,CAAC;QAC9E,MAAM,CAAC,OAAO,EAAE,2BAA2B,EAAE,6CAA6C,CAAC,CAAC;QAC5F,MAAM,CAAC,OAAO,EAAE,iBAAiB,EAAE,mCAAmC,CAAC,CAAC;QACxE,MAAM,CAAC,OAAO,EAAE,oBAAoB,EAAE,sCAAsC,CAAC,CAAC;QAC9E,MAAM,CAAC,OAAO,EAAE,UAAU,EAAE,4BAA4B,CAAC,CAAC;QAC1D,MAAM,CAAC,OAAO,EAAE,eAAe,EAAE,iCAAiC,CAAC,CAAC;QACpE,MAAM,CAAC,OAAO,EAAE,UAAU,EAAE,4BAA4B,CAAC,CAAC;QAC1D,MAAM,CAAC,OAAO,EAAE,UAAU,EAAE,4BAA4B,CAAC,CAAC;QAC1D,MAAM,CAAC,OAAO,EAAE,aAAa,EAAE,kBAAkB,CAAC,CAAC;QACnD,MAAM,CAAC,OAAO,EAAE,qBAAqB,EAAE,uCAAuC,CAAC,CAAC;QAChF,MAAM,CAAC,OAAO,EAAE,cAAc,EAAE,mBAAmB,CAAC,CAAC;QACrD,MAAM,CAAC,OAAO,EAAE,eAAe,EAAE,iCAAiC,CAAC,CAAC;QACpE,MAAM,CAAC,OAAO,EAAE,UAAU,EAAE,4BAA4B,CAAC,CAAC;QAC1D,MAAM,CAAC,OAAO,EAAE,aAAa,EAAE,kBAAkB,CAAC,CAAC;QACnD,MAAM,CAAC,OAAO,EAAE,aAAa,EAAE,qBAAqB,CAAC,CAAC;QACtD,MAAM,CAAC,OAAO,EAAE,eAAe,EAAE,uBAAuB,CAAC,CAAC;QAC1D,MAAM,CAAC,OAAO,EAAE,iBAAiB,EAAE,yBAAyB,CAAC,CAAC,CAAC,sGAAsG;QACrK,MAAM,CAAC,cAAc,EAAE,SAAS,EAAE,cAAc,CAAC,CAAC;QAClD,MAAM,CAAC,OAAO,EAAE,mBAAmB,EAAE,wBAAwB,CAAC,CAAC;QAC/D,MAAM,CAAC,OAAO,EAAE,mBAAmB,EAAE,wBAAwB,CAAC,CAAC;QAC/D,MAAM,CAAC,OAAO,EAAE,kBAAkB,EAAE,0BAA0B,CAAC,CAAC;QAChE,MAAM,CAAC,OAAO,EAAE,gBAAgB,EAAE,qBAAqB,CAAC,CAAC;QACzD,MAAM,CAAC,OAAO,EAAE,aAAa,EAAE,qBAAqB,CAAC,CAAC;QACtD,MAAM,CAAC,OAAO,EAAE,kBAAkB,EAAE,uBAAuB,CAAC,CAAC;QAC7D,8EAA8E;QAC9E,MAAM,CAAC,eAAe,EAAE,iBAAiB,EAAE,yBAAyB,CAAC,CAAC;QACtE,MAAM,CAAC,eAAe,EAAE,mBAAmB,EAAE,2BAA2B,CAAC,CAAC;QAC1E,MAAM,CAAC,eAAe,EAAE,sBAAsB,EAAE,2BAA2B,CAAC,CAAC;QAC7E,MAAM,CAAC,eAAe,EAAE,sBAAsB,EAAE,2BAA2B,CAAC,CAAC;QAC7E,2FAA2F;QAC3F,MAAM,CAAC,eAAe,EAAE,gBAAgB,EAAE,wBAAwB,CAAC,CAAC,CAAW,kCAAkC;QACjH,MAAM,CAAC,eAAe,EAAE,kBAAkB,EAAE,0BAA0B,CAAC,CAAC;QACxE,MAAM,CAAC,eAAe,EAAE,cAAc,EAAE,sBAAsB,CAAC,CAAC;QAChE,MAAM,CAAC,eAAe,EAAE,wBAAwB,EAAE,6BAA6B,CAAC,CAAC,CAAC,iDAAiD;QACnI,gHAAgH;QAChH,MAAM,CAAC,cAAc,EAAE,QAAQ,EAAE,gBAAgB,CAAC,CAAC;QACnD,MAAM,CAAC,cAAc,EAAE,OAAO,EAAE,eAAe,CAAC,CAAC;QACjD,MAAM,CAAC,cAAc,EAAE,SAAS,EAAE,iBAAiB,CAAC,CAAC;QACrD,MAAM,CAAC,cAAc,EAAE,SAAS,EAAE,iBAAiB,CAAC,CAAC;QACrD,MAAM,CAAC,cAAc,EAAE,6BAA6B,EAAE,kCAAkC,CAAC,CAAC;QAC1F,4FAA4F;QAC5F,MAAM,CAAC,gBAAgB,EAAE,gBAAgB,EAAE,qBAAqB,CAAC,CAAC;QAClE,4FAA4F;QAC5F,2FAA2F;QAC3F,yFAAyF;QACzF,6FAA6F;QAC7F,MAAM,CAAC,kBAAkB,EAAE,aAAa,EAAE,gCAAgC,CAAC,CAAC;IAC9E,CAAC;IAED,6EAA6E;IAC7E,cAAc,CAAC,OAAe,EAAE,QAAgB,EAAE,eAAwB;QACxE,IAAI,CAAC,EAAE;aACJ,OAAO,CACN;;;kGAG0F,CAC3F;aACA,GAAG,CAAC,OAAO,EAAE,QAAQ,EAAE,eAAe,IAAI,IAAI,CAAC,CAAC;IACrD,CAAC;IAED,WAAW,CAAC,OAAe;QACzB,OAAO,IAAI,CAAC,EAAE,CAAC,OAAO,CAAC,kFAAkF,CAAC,CAAC,GAAG,CAAC,OAAO,CAEzG,CAAC;IAChB,CAAC;IAED,2FAA2F;IAC3F,cAAc,CAAC,OAAe,EAAE,QAAyB;QACvD,IAAI,CAAC,EAAE;aACJ,OAAO,CAAC,0FAA0F,CAAC;aACnG,GAAG,CAAC,OAAO,QAAQ,KAAK,QAAQ,CAAC,CAAC,CAAC,QAAQ,CAAC,CAAC,CAAC,IAAI,EAAE,OAAO,QAAQ,KAAK,QAAQ,CAAC,CAAC,CAAC,QAAQ,CAAC,CAAC,CAAC,IAAI,EAAE,OAAO,CAAC,CAAC;IAClH,CAAC;IAED,iFAAiF;IACjF,cAAc,CAAC,OAAe;QAC5B,MAAM,CAAC,GAAG,IAAI,CAAC,EAAE,CAAC,OAAO,CAAC,uFAAuF,CAAC,CAAC,GAAG,CAAC,OAAO,CAEjH,CAAC;QACd,OAAO,CAAC,EAAE,CAAC,IAAI,CAAC,EAAE,CAAC,IAAI,SAAS,CAAC;IACnC,CAAC;IAED,KAAK;QACH,IAAI,CAAC,EAAE,CAAC,KAAK,EAAE,CAAC;IAClB,CAAC;CACF"}
+{"version":3,"file":"AuditDatabase.js","sourceRoot":"","sources":["../../src/core/AuditDatabase.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;GAUG;AACH,OAAO,QAAQ,MAAM,gBAAgB,CAAC;AACtC,OAAO,EAAE,SAAS,EAAE,MAAM,SAAS,CAAC;AACpC,OAAO,EAAE,OAAO,EAAE,MAAM,WAAW,CAAC;AAKpC,MAAM,OAAO,aAAa;IACf,EAAE,CAAoB;IAE/B,YAAY,MAAc;QACxB,SAAS,CAAC,OAAO,CAAC,MAAM,CAAC,EAAE,EAAE,SAAS,EAAE,IAAI,EAAE,CAAC,CAAC;QAChD,IAAI,CAAC,EAAE,GAAG,IAAI,QAAQ,CAAC,MAAM,CAAC,CAAC;QAC/B,IAAI,CAAC,EAAE,CAAC,MAAM,CAAC,oBAAoB,CAAC,CAAC;QACrC,IAAI,CAAC,EAAE,CAAC,MAAM,CAAC,sBAAsB,CAAC,CAAC,CAAC,iDAAiD;QACzF,IAAI,CAAC,EAAE,CAAC,MAAM,CAAC,sBAAsB,CAAC,CAAC,CAAE,qEAAqE;QAC9G,IAAI,CAAC,EAAE,CAAC,MAAM,CAAC,mBAAmB,CAAC,CAAC;QACpC,6FAA6F;QAC7F,8FAA8F;QAC9F,gGAAgG;QAChG,IAAI,CAAC,EAAE,CAAC,MAAM,CAAC,sBAAsB,CAAC,CAAC,CAAE,qCAAqC;QAC9E,IAAI,CAAC,EAAE,CAAC,MAAM,CAAC,uBAAuB,CAAC,CAAC,CAAC,2BAA2B;QACpE,IAAI,CAAC,EAAE,CAAC,MAAM,CAAC,qBAAqB,CAAC,CAAC,CAAG,kDAAkD;QAC3F,IAAI,CAAC,gBAAgB,EAAE,CAAC;IAC1B,CAAC;IAEO,gBAAgB;QACtB,0EAA0E;QAC1E,IAAI,CAAC,EAAE,CAAC,IAAI,CAAC;;;;;;;;;;;;KAYZ,CAAC,CAAC;QAEH,0EAA0E;QAC1E,IAAI,CAAC,EAAE,CAAC,IAAI,CAAC;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;KAuCZ,CAAC,CAAC;QAEH,gFAAgF;QAChF,IAAI,CAAC,EAAE,CAAC,IAAI,CAAC;;;;;;;;;;;;;;;;;;;KAmBZ,CAAC,CAAC;QAEH,IAAI,CAAC,EAAE,CAAC,IAAI,CAAC;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;KA2EZ,CAAC,CAAC;QAEH,2FAA2F;QAC3F,4FAA4F;QAC5F,2FAA2F;QAC3F,IAAI,CAAC,EAAE,CAAC,IAAI,CAAC;;;;;;;;;;;;;;;;;;;;KAoBZ,CAAC,CAAC;QAEH,4FAA4F;QAC5F,gFAAgF;QAChF,IAAI,CAAC,EAAE,CAAC,IAAI,CAAC;;;;;;;;;KASZ,CAAC,CAAC;QAEH,IAAI,CAAC,EAAE,CAAC,IAAI,CAAC;;;;;;;;;;;;;;;;;;KAkBZ,CAAC,CAAC;QAEH,IAAI,CAAC,EAAE,CAAC,IAAI,CAAC;;;;;;;;;;;KAWZ,CAAC,CAAC;QAEH,2EAA2E;QAC3E,IAAI,CAAC,EAAE,CAAC,IAAI,CAAC;;;;;;;;;;;;;;;;;;KAkBZ,CAAC,CAAC;QAEH,2EAA2E;QAC3E,IAAI,CAAC,EAAE,CAAC,IAAI,CAAC;;;;;;;;;;;;KAYZ,CAAC,CAAC;QAEH,0EAA0E;QAC1E,IAAI,CAAC,EAAE,CAAC,IAAI,CAAC;;;;;;;;;;;;KAYZ,CAAC,CAAC;QAEH,IAAI,CAAC,EAAE,CAAC,IAAI,CAAC;;;;;;;;;;;;;;;;;;;;KAoBZ,CAAC,CAAC;QAEH,iFAAiF;QACjF,IAAI,CAAC,EAAE,CAAC,IAAI,CAAC;;;;;;;;;;;KAWZ,CAAC,CAAC;QAEH,0EAA0E;QAC1E,IAAI,CAAC,EAAE,CAAC,IAAI,CAAC;;;;;;;KAOZ,CAAC,CAAC;QAEH,+EAA+E;QAC/E,IAAI,CAAC,EAAE,CAAC,IAAI,CAAC;;;;;;;;;KASZ,CAAC,CAAC;QAEH,4EAA4E;QAC5E,4EAA4E;QAC5E,IAAI,CAAC,EAAE,CAAC,IAAI,CAAC;;;;;;;KAOZ,CAAC,CAAC;QACH,iFAAiF;QACjF,IAAI,CAAC,EAAE,CAAC,IAAI,CAAC;;;;;;;;;;;KAWZ,CAAC,CAAC;QAEH,sFAAsF;QACtF,IAAI,CAAC,EAAE,CAAC,IAAI,CAAC;;;;;;;;;;;;;;;;;KAiBZ,CAAC,CAAC;QAEH,oFAAoF;QACpF,0FAA0F;QAC1F,IAAI,CAAC,EAAE,CAAC,IAAI,CAAC;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;KAoCZ,CAAC,CAAC;QAEH,IAAI,CAAC,OAAO,EAAE,CAAC;IACjB,CAAC;IAED,2EAA2E;IACnE,OAAO;QACb,MAAM,IAAI,GAAG,CAAC,KAAa,EAAe,EAAE,CAC1C,IAAI,GAAG,CAAE,IAAI,CAAC,EAAE,CAAC,OAAO,CAAC,qBAAqB,KAAK,GAAG,CAAC,CAAC,GAAG,EAAyB,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC;QACzG,MAAM,MAAM,GAAG,CAAC,KAAa,EAAE,GAAW,EAAE,GAAW,EAAQ,EAAE;YAC/D,IAAI,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,GAAG,CAAC,GAAG,CAAC;gBAAE,IAAI,CAAC,EAAE,CAAC,IAAI,CAAC,eAAe,KAAK,eAAe,GAAG,EAAE,CAAC,CAAC;QACpF,CAAC,CAAC;QACF,MAAM,CAAC,OAAO,EAAE,aAAa,EAAE,+BAA+B,CAAC,CAAC;QAChE,MAAM,CAAC,OAAO,EAAE,oBAAoB,EAAE,sCAAsC,CAAC,CAAC;QAC9E,MAAM,CAAC,OAAO,EAAE,2BAA2B,EAAE,6CAA6C,CAAC,CAAC;QAC5F,MAAM,CAAC,OAAO,EAAE,iBAAiB,EAAE,mCAAmC,CAAC,CAAC;QACxE,MAAM,CAAC,OAAO,EAAE,oBAAoB,EAAE,sCAAsC,CAAC,CAAC;QAC9E,MAAM,CAAC,OAAO,EAAE,UAAU,EAAE,4BAA4B,CAAC,CAAC;QAC1D,MAAM,CAAC,OAAO,EAAE,eAAe,EAAE,iCAAiC,CAAC,CAAC;QACpE,MAAM,CAAC,OAAO,EAAE,UAAU,EAAE,4BAA4B,CAAC,CAAC;QAC1D,MAAM,CAAC,OAAO,EAAE,UAAU,EAAE,4BAA4B,CAAC,CAAC;QAC1D,MAAM,CAAC,OAAO,EAAE,aAAa,EAAE,kBAAkB,CAAC,CAAC;QACnD,MAAM,CAAC,OAAO,EAAE,qBAAqB,EAAE,uCAAuC,CAAC,CAAC;QAChF,MAAM,CAAC,OAAO,EAAE,cAAc,EAAE,mBAAmB,CAAC,CAAC;QACrD,MAAM,CAAC,OAAO,EAAE,eAAe,EAAE,iCAAiC,CAAC,CAAC;QACpE,MAAM,CAAC,OAAO,EAAE,UAAU,EAAE,4BAA4B,CAAC,CAAC;QAC1D,MAAM,CAAC,OAAO,EAAE,aAAa,EAAE,kBAAkB,CAAC,CAAC;QACnD,MAAM,CAAC,OAAO,EAAE,aAAa,EAAE,qBAAqB,CAAC,CAAC;QACtD,MAAM,CAAC,OAAO,EAAE,eAAe,EAAE,uBAAuB,CAAC,CAAC;QAC1D,MAAM,CAAC,OAAO,EAAE,iBAAiB,EAAE,yBAAyB,CAAC,CAAC,CAAC,sGAAsG;QACrK,MAAM,CAAC,cAAc,EAAE,SAAS,EAAE,cAAc,CAAC,CAAC;QAClD,MAAM,CAAC,OAAO,EAAE,mBAAmB,EAAE,wBAAwB,CAAC,CAAC;QAC/D,MAAM,CAAC,OAAO,EAAE,mBAAmB,EAAE,wBAAwB,CAAC,CAAC;QAC/D,MAAM,CAAC,OAAO,EAAE,kBAAkB,EAAE,0BAA0B,CAAC,CAAC;QAChE,MAAM,CAAC,OAAO,EAAE,gBAAgB,EAAE,qBAAqB,CAAC,CAAC;QACzD,MAAM,CAAC,OAAO,EAAE,aAAa,EAAE,qBAAqB,CAAC,CAAC;QACtD,MAAM,CAAC,OAAO,EAAE,kBAAkB,EAAE,uBAAuB,CAAC,CAAC;QAC7D,8EAA8E;QAC9E,MAAM,CAAC,eAAe,EAAE,iBAAiB,EAAE,yBAAyB,CAAC,CAAC;QACtE,MAAM,CAAC,eAAe,EAAE,mBAAmB,EAAE,2BAA2B,CAAC,CAAC;QAC1E,MAAM,CAAC,eAAe,EAAE,sBAAsB,EAAE,2BAA2B,CAAC,CAAC;QAC7E,MAAM,CAAC,eAAe,EAAE,sBAAsB,EAAE,2BAA2B,CAAC,CAAC;QAC7E,2FAA2F;QAC3F,MAAM,CAAC,eAAe,EAAE,gBAAgB,EAAE,wBAAwB,CAAC,CAAC,CAAW,kCAAkC;QACjH,MAAM,CAAC,eAAe,EAAE,kBAAkB,EAAE,0BAA0B,CAAC,CAAC;QACxE,MAAM,CAAC,eAAe,EAAE,cAAc,EAAE,sBAAsB,CAAC,CAAC;QAChE,MAAM,CAAC,eAAe,EAAE,wBAAwB,EAAE,6BAA6B,CAAC,CAAC,CAAC,iDAAiD;QACnI,gHAAgH;QAChH,MAAM,CAAC,cAAc,EAAE,QAAQ,EAAE,gBAAgB,CAAC,CAAC;QACnD,MAAM,CAAC,cAAc,EAAE,OAAO,EAAE,eAAe,CAAC,CAAC;QACjD,MAAM,CAAC,cAAc,EAAE,SAAS,EAAE,iBAAiB,CAAC,CAAC;QACrD,MAAM,CAAC,cAAc,EAAE,SAAS,EAAE,iBAAiB,CAAC,CAAC;QACrD,MAAM,CAAC,cAAc,EAAE,6BAA6B,EAAE,kCAAkC,CAAC,CAAC;QAC1F,4FAA4F;QAC5F,MAAM,CAAC,gBAAgB,EAAE,gBAAgB,EAAE,qBAAqB,CAAC,CAAC;QAClE,4FAA4F;QAC5F,2FAA2F;QAC3F,yFAAyF;QACzF,6FAA6F;QAC7F,MAAM,CAAC,kBAAkB,EAAE,aAAa,EAAE,gCAAgC,CAAC,CAAC;IAC9E,CAAC;IAED,6EAA6E;IAC7E,cAAc,CAAC,OAAe,EAAE,QAAgB,EAAE,eAAwB;QACxE,IAAI,CAAC,EAAE;aACJ,OAAO,CACN;;;kGAG0F,CAC3F;aACA,GAAG,CAAC,OAAO,EAAE,QAAQ,EAAE,eAAe,IAAI,IAAI,CAAC,CAAC;IACrD,CAAC;IAED,WAAW,CAAC,OAAe;QACzB,OAAO,IAAI,CAAC,EAAE,CAAC,OAAO,CAAC,kFAAkF,CAAC,CAAC,GAAG,CAAC,OAAO,CAEzG,CAAC;IAChB,CAAC;IAED,2FAA2F;IAC3F,cAAc,CAAC,OAAe,EAAE,QAAyB;QACvD,IAAI,CAAC,EAAE;aACJ,OAAO,CAAC,0FAA0F,CAAC;aACnG,GAAG,CAAC,OAAO,QAAQ,KAAK,QAAQ,CAAC,CAAC,CAAC,QAAQ,CAAC,CAAC,CAAC,IAAI,EAAE,OAAO,QAAQ,KAAK,QAAQ,CAAC,CAAC,CAAC,QAAQ,CAAC,CAAC,CAAC,IAAI,EAAE,OAAO,CAAC,CAAC;IAClH,CAAC;IAED,iFAAiF;IACjF,cAAc,CAAC,OAAe;QAC5B,MAAM,CAAC,GAAG,IAAI,CAAC,EAAE,CAAC,OAAO,CAAC,uFAAuF,CAAC,CAAC,GAAG,CAAC,OAAO,CAEjH,CAAC;QACd,OAAO,CAAC,EAAE,CAAC,IAAI,CAAC,EAAE,CAAC,IAAI,SAAS,CAAC;IACnC,CAAC;IAED,KAAK;QACH,IAAI,CAAC,EAAE,CAAC,KAAK,EAAE,CAAC;IAClB,CAAC;CACF"}

package/dist/core/FirecrawlClient.d.ts

@@ -0,0 +1,35 @@
+/**
+ * Firecrawl client for scraping THIRD-PARTY (competitor) pages during content recon.
+ * Owned pages use our own crawler; competitor pages use Firecrawl.
+ *
+ * NB the API vs MCP divergence: the raw v2 API nests the result under `.data`
+ * ({ success, data: { markdown, metadata } }) whereas the Firecrawl MCP flattens it.
+ * We code against the API and read `json.data`.
+ *
+ * A small SQLite cache (default 7 days) avoids re-scraping the same competitor URL
+ * across a batch or a re-run — competitor pages don't change minute to minute, and it
+ * keeps recon cheap and polite.
+ */
+export interface ScrapeResult {
+    url: string;
+    markdown: string;
+    title: string | null;
+    cached: boolean;
+}
+export declare class FirecrawlClient {
+    private readonly apiKey;
+    private readonly ttlMs;
+    private readonly cache;
+    private chain;
+    constructor(apiKey: string, cacheDbPath: string, ttlDays?: number);
+    /** Serialise live scrapes — polite, and keeps us within rate limits. */
+    private serialize;
+    /** Scrape one URL to clean markdown (main content only). Cached for the TTL.
+     * proxy: 'auto' lets Firecrawl escalate to stealth on bot-protected sites (costs more). */
+    scrape(url: string, opts?: {
+        maxChars?: number;
+        timeoutMs?: number;
+        proxy?: 'basic' | 'stealth' | 'auto';
+    }): Promise<ScrapeResult>;
+}
+//# sourceMappingURL=FirecrawlClient.d.ts.map

package/dist/core/FirecrawlClient.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"FirecrawlClient.d.ts","sourceRoot":"","sources":["../../src/core/FirecrawlClient.ts"],"names":[],"mappings":"AAKA;;;;;;;;;;;GAWG;AACH,MAAM,WAAW,YAAY;IAC3B,GAAG,EAAE,MAAM,CAAC;IACZ,QAAQ,EAAE,MAAM,CAAC;IACjB,KAAK,EAAE,MAAM,GAAG,IAAI,CAAC;IACrB,MAAM,EAAE,OAAO,CAAC;CACjB;AAID,qBAAa,eAAe;IAC1B,OAAO,CAAC,QAAQ,CAAC,MAAM,CAAS;IAChC,OAAO,CAAC,QAAQ,CAAC,KAAK,CAAS;IAC/B,OAAO,CAAC,QAAQ,CAAC,KAAK,CAAoB;IAC1C,OAAO,CAAC,KAAK,CAAuC;gBAExC,MAAM,EAAE,MAAM,EAAE,WAAW,EAAE,MAAM,EAAE,OAAO,SAAI;IAiB5D,wEAAwE;IACxE,OAAO,CAAC,SAAS;IAMjB;+FAC2F;IACrF,MAAM,CAAC,GAAG,EAAE,MAAM,EAAE,IAAI,GAAE;QAAE,QAAQ,CAAC,EAAE,MAAM,CAAC;QAAC,SAAS,CAAC,EAAE,MAAM,CAAC;QAAC,KAAK,CAAC,EAAE,OAAO,GAAG,SAAS,GAAG,MAAM,CAAA;KAAO,GAAG,OAAO,CAAC,YAAY,CAAC;CAoC7I"}

package/dist/core/FirecrawlClient.js

@@ -0,0 +1,69 @@
+import Database from 'better-sqlite3';
+import { createHash } from 'node:crypto';
+import { mkdirSync } from 'node:fs';
+import { dirname } from 'node:path';
+const BASE_URL = 'https://api.firecrawl.dev';
+export class FirecrawlClient {
+    apiKey;
+    ttlMs;
+    cache;
+    chain = Promise.resolve();
+    constructor(apiKey, cacheDbPath, ttlDays = 7) {
+        this.apiKey = apiKey;
+        this.ttlMs = ttlDays * 86400000;
+        mkdirSync(dirname(cacheDbPath), { recursive: true });
+        this.cache = new Database(cacheDbPath);
+        this.cache.pragma('journal_mode = WAL');
+        this.cache.exec(`
+      CREATE TABLE IF NOT EXISTS firecrawl_cache (
+        cache_key TEXT PRIMARY KEY,
+        url TEXT NOT NULL,
+        markdown TEXT,
+        title TEXT,
+        fetched_at TEXT NOT NULL
+      );
+    `);
+    }
+    /** Serialise live scrapes — polite, and keeps us within rate limits. */
+    serialize(fn) {
+        const result = this.chain.then(fn, fn);
+        this.chain = result.then(() => undefined, () => undefined);
+        return result;
+    }
+    /** Scrape one URL to clean markdown (main content only). Cached for the TTL.
+     * proxy: 'auto' lets Firecrawl escalate to stealth on bot-protected sites (costs more). */
+    async scrape(url, opts = {}) {
+        const key = createHash('sha256').update('scrape\n' + url).digest('hex');
+        const row = this.cache
+            .prepare('SELECT url, markdown, title, fetched_at FROM firecrawl_cache WHERE cache_key = ?')
+            .get(key);
+        if (row && Date.now() - Date.parse(row.fetched_at) < this.ttlMs) {
+            const md = opts.maxChars ? (row.markdown ?? '').slice(0, opts.maxChars) : (row.markdown ?? '');
+            return { url, markdown: md, title: row.title, cached: true };
+        }
+        return this.serialize(async () => {
+            const res = await fetch(`${BASE_URL}/v2/scrape`, {
+                method: 'POST',
+                headers: { Authorization: `Bearer ${this.apiKey}`, 'content-type': 'application/json' },
+                body: JSON.stringify({ url, formats: ['markdown'], onlyMainContent: true, proxy: opts.proxy ?? 'auto' }),
+                signal: AbortSignal.timeout(opts.timeoutMs ?? 45000),
+            });
+            const json = await res.json();
+            if (!res.ok || json?.success === false) {
+                throw new Error(`Firecrawl scrape failed for ${url}: ${res.status} ${json?.error ?? ''}`.trim());
+            }
+            // API nests under .data (MCP flattens it — we read the API shape here).
+            const data = json.data ?? json;
+            const markdown = data.markdown ?? '';
+            const title = data.metadata?.title ?? null;
+            this.cache
+                .prepare(`INSERT INTO firecrawl_cache (cache_key, url, markdown, title, fetched_at)
+           VALUES (?, ?, ?, ?, datetime('now'))
+           ON CONFLICT(cache_key) DO UPDATE SET markdown=excluded.markdown, title=excluded.title, fetched_at=datetime('now')`)
+                .run(key, url, markdown, title);
+            const md = opts.maxChars ? markdown.slice(0, opts.maxChars) : markdown;
+            return { url, markdown: md, title, cached: false };
+        });
+    }
+}
+//# sourceMappingURL=FirecrawlClient.js.map

package/dist/core/FirecrawlClient.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"FirecrawlClient.js","sourceRoot":"","sources":["../../src/core/FirecrawlClient.ts"],"names":[],"mappings":"AAAA,OAAO,QAAQ,MAAM,gBAAgB,CAAC;AACtC,OAAO,EAAE,UAAU,EAAE,MAAM,aAAa,CAAC;AACzC,OAAO,EAAE,SAAS,EAAE,MAAM,SAAS,CAAC;AACpC,OAAO,EAAE,OAAO,EAAE,MAAM,WAAW,CAAC;AAqBpC,MAAM,QAAQ,GAAG,2BAA2B,CAAC;AAE7C,MAAM,OAAO,eAAe;IACT,MAAM,CAAS;IACf,KAAK,CAAS;IACd,KAAK,CAAoB;IAClC,KAAK,GAAqB,OAAO,CAAC,OAAO,EAAE,CAAC;IAEpD,YAAY,MAAc,EAAE,WAAmB,EAAE,OAAO,GAAG,CAAC;QAC1D,IAAI,CAAC,MAAM,GAAG,MAAM,CAAC;QACrB,IAAI,CAAC,KAAK,GAAG,OAAO,GAAG,QAAQ,CAAC;QAChC,SAAS,CAAC,OAAO,CAAC,WAAW,CAAC,EAAE,EAAE,SAAS,EAAE,IAAI,EAAE,CAAC,CAAC;QACrD,IAAI,CAAC,KAAK,GAAG,IAAI,QAAQ,CAAC,WAAW,CAAC,CAAC;QACvC,IAAI,CAAC,KAAK,CAAC,MAAM,CAAC,oBAAoB,CAAC,CAAC;QACxC,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC;;;;;;;;KAQf,CAAC,CAAC;IACL,CAAC;IAED,wEAAwE;IAChE,SAAS,CAAI,EAAoB;QACvC,MAAM,MAAM,GAAG,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,EAAE,EAAE,EAAE,CAAC,CAAC;QACvC,IAAI,CAAC,KAAK,GAAG,MAAM,CAAC,IAAI,CAAC,GAAG,EAAE,CAAC,SAAS,EAAE,GAAG,EAAE,CAAC,SAAS,CAAC,CAAC;QAC3D,OAAO,MAAM,CAAC;IAChB,CAAC;IAED;+FAC2F;IAC3F,KAAK,CAAC,MAAM,CAAC,GAAW,EAAE,OAAwF,EAAE;QAClH,MAAM,GAAG,GAAG,UAAU,CAAC,QAAQ,CAAC,CAAC,MAAM,CAAC,UAAU,GAAG,GAAG,CAAC,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC;QACxE,MAAM,GAAG,GAAG,IAAI,CAAC,KAAK;aACnB,OAAO,CAAC,kFAAkF,CAAC;aAC3F,GAAG,CAAC,GAAG,CAA4F,CAAC;QACvG,IAAI,GAAG,IAAI,IAAI,CAAC,GAAG,EAAE,GAAG,IAAI,CAAC,KAAK,CAAC,GAAG,CAAC,UAAU,CAAC,GAAG,IAAI,CAAC,KAAK,EAAE,CAAC;YAChE,MAAM,EAAE,GAAG,IAAI,CAAC,QAAQ,CAAC,CAAC,CAAC,CAAC,GAAG,CAAC,QAAQ,IAAI,EAAE,CAAC,CAAC,KAAK,CAAC,CAAC,EAAE,IAAI,CAAC,QAAQ,CAAC,CAAC,CAAC,CAAC,CAAC,GAAG,CAAC,QAAQ,IAAI,EAAE,CAAC,CAAC;YAC/F,OAAO,EAAE,GAAG,EAAE,QAAQ,EAAE,EAAE,EAAE,KAAK,EAAE,GAAG,CAAC,KAAK,EAAE,MAAM,EAAE,IAAI,EAAE,CAAC;QAC/D,CAAC;QAED,OAAO,IAAI,CAAC,SAAS,CAAC,KAAK,IAAI,EAAE;YAC/B,MAAM,GAAG,GAAG,MAAM,KAAK,CAAC,GAAG,QAAQ,YAAY,EAAE;gBAC/C,MAAM,EAAE,MAAM;gBACd,OAAO,EAAE,EAAE,aAAa,EAAE,UAAU,IAAI,CAAC,MAAM,EAAE,EAAE,cAAc,EAAE,kBAAkB,EAAE;gBACvF,IAAI,EAAE,IAAI,CAAC,SAAS,CAAC,EAAE,GAAG,EAAE,OAAO,EAAE,CAAC,UAAU,CAAC,EAAE,eAAe,EAAE,IAAI,EAAE,KAAK,EAAE,IAAI,CAAC,KAAK,IAAI,MAAM,EAAE,CAAC;gBACxG,MAAM,EAAE,WAAW,CAAC,OAAO,CAAC,IAAI,CAAC,SAAS,IAAI,KAAK,CAAC;aACrD,CAAC,CAAC;YACH,MAAM,IAAI,GAAQ,MAAM,GAAG,CAAC,IAAI,EAAE,CAAC;YACnC,IAAI,CAAC,GAAG,CAAC,EAAE,IAAI,IAAI,EAAE,OAAO,KAAK,KAAK,EAAE,CAAC;gBACvC,MAAM,IAAI,KAAK,CAAC,+BAA+B,GAAG,KAAK,GAAG,CAAC,MAAM,IAAI,IAAI,EAAE,KAAK,IAAI,EAAE,EAAE,CAAC,IAAI,EAAE,CAAC,CAAC;YACnG,CAAC;YACD,wEAAwE;YACxE,MAAM,IAAI,GAAG,IAAI,CAAC,IAAI,IAAI,IAAI,CAAC;YAC/B,MAAM,QAAQ,GAAW,IAAI,CAAC,QAAQ,IAAI,EAAE,CAAC;YAC7C,MAAM,KAAK,GAAkB,IAAI,CAAC,QAAQ,EAAE,KAAK,IAAI,IAAI,CAAC;YAC1D,IAAI,CAAC,KAAK;iBACP,OAAO,CACN;;6HAEmH,CACpH;iBACA,GAAG,CAAC,GAAG,EAAE,GAAG,EAAE,QAAQ,EAAE,KAAK,CAAC,CAAC;YAClC,MAAM,EAAE,GAAG,IAAI,CAAC,QAAQ,CAAC,CAAC,CAAC,QAAQ,CAAC,KAAK,CAAC,CAAC,EAAE,IAAI,CAAC,QAAQ,CAAC,CAAC,CAAC,CAAC,QAAQ,CAAC;YACvE,OAAO,EAAE,GAAG,EAAE,QAAQ,EAAE,EAAE,EAAE,KAAK,EAAE,MAAM,EAAE,KAAK,EAAE,CAAC;QACrD,CAAC,CAAC,CAAC;IACL,CAAC;CACF"}