fieldtheory 1.0.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Field Theory
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,133 @@
+# Field Theory CLI
+
+Self-custody for your X/Twitter bookmarks. Sync them locally, search with full-text, classify into categories, and point an AI agent at them.
+
+Your bookmarks stay on your machine. No account required. Free and open source.
+
+## Install
+
+```bash
+npm install -g fieldtheory
+```
+
+Requires Node.js 20+.
+
+## Quick start
+
+```bash
+# 1. Sync your bookmarks (needs Chrome logged into X)
+ft sync
+
+# 2. Search them
+ft search "distributed systems"
+
+# 3. Explore
+ft viz
+ft categories
+ft stats
+```
+
+On first run, `ft sync` extracts your X session from Chrome and downloads your bookmarks into `~/.fieldtheory/`. It auto-classifies them into 7 categories (tool, security, technique, launch, research, opinion, commerce) using fast regex matching.
+
+## Commands
+
+| Command | Description |
+|---------|-------------|
+| `ft sync` | Sync bookmarks via Chrome session |
+| `ft sync --api` | Sync via OAuth API (cross-platform) |
+| `ft search <query>` | Full-text search (FTS5 with BM25 ranking) |
+| `ft list` | List with filters (author, date, category, domain) |
+| `ft show <id>` | Show one bookmark in detail |
+| `ft stats` | Aggregate statistics |
+| `ft viz` | ANSI terminal dashboard with sparklines and heatmaps |
+| `ft classify` | Regex classification (instant, free) |
+| `ft classify --deep` | LLM classification (needs `claude` or `codex` in PATH) |
+| `ft classify-domains` | LLM domain classification (ai, finance, devops, etc.) |
+| `ft categories` | Show category distribution |
+| `ft domains` | Show domain distribution |
+| `ft index` | Rebuild the SQLite search index |
+| `ft auth` | Set up OAuth for API-based sync |
+| `ft status` | Show sync status and data location |
+| `ft path` | Print data directory path |
+| `ft sample <category>` | Sample bookmarks by category |
+| `ft fetch-media` | Download media assets |
+
+## Agent integration
+
+The CLI is designed to work with AI agents. Add these tools to your agent's system prompt or `CLAUDE.md`:
+
+```
+Use the ft CLI to query the user's X bookmarks:
+  - ft search <query>     — full-text search
+  - ft list --category X  — list by category
+  - ft categories         — see all categories
+  - ft stats              — aggregate statistics
+```
+
+**Fun prompt to try:**
+
+> "Take my oldest and newest bookmarks and tell me how my interests have changed over time."
+
+Works with Claude Code, Codex, or any agent with shell access.
+
+## Scheduling
+
+Sync daily with crontab:
+
+```bash
+# Sync every morning at 7am
+0 7 * * * ft sync
+```
+
+For API-based sync (no Chrome needed), set up OAuth first:
+
+```bash
+ft auth          # one-time OAuth setup
+ft sync --api    # uses API token, works headlessly
+```
+
+## Data
+
+All data is stored locally at `~/.fieldtheory/`:
+
+```
+~/.fieldtheory/
+  bookmarks.jsonl         # raw bookmark cache (one per line)
+  bookmarks.db            # SQLite FTS5 search index
+  bookmarks-meta.json     # sync metadata
+  oauth-token.json        # OAuth token (if using API mode)
+```
+
+Override the location with `FT_DATA_DIR`:
+
+```bash
+export FT_DATA_DIR=/path/to/custom/dir
+```
+
+## Categories
+
+The regex classifier sorts bookmarks into 7 categories:
+
+- **tool** — GitHub repos, CLI tools, npm packages, open-source projects
+- **security** — CVEs, vulnerabilities, exploits, supply chain
+- **technique** — Tutorials, demos, code patterns, "how I built X"
+- **launch** — Product launches, announcements, "just shipped"
+- **research** — ArXiv papers, studies, academic findings
+- **opinion** — Takes, analysis, commentary, threads
+- **commerce** — Products, shopping, physical goods
+
+Use `ft classify --deep` for LLM-powered classification that catches what regex misses.
+
+## Platform support
+
+| Feature | macOS | Linux | Windows |
+|---------|-------|-------|---------|
+| Chrome session sync (`ft sync`) | Yes | No* | No* |
+| OAuth API sync (`ft sync --api`) | Yes | Yes | Yes |
+| Search, list, classify, viz | Yes | Yes | Yes |
+
+\*Chrome session extraction uses macOS Keychain. On other platforms, use `ft auth` + `ft sync --api`.
+
+## License
+
+MIT

package/bin/ft.mjs

@@ -0,0 +1,3 @@
+#!/usr/bin/env node
+import { buildCli } from '../dist/cli.js';
+await buildCli().parseAsync(process.argv);

package/dist/bookmark-classify-llm.js

@@ -0,0 +1,247 @@
+/**
+ * LLM-based bookmark classification — uses `claude -p` or `codex exec`
+ * (whichever the user has via their Max/Pro subscription) to classify
+ * bookmarks that the regex classifier couldn't categorize.
+ *
+ * No API keys needed. No local models. Just a logged-in Claude or Codex CLI.
+ */
+import { execFileSync, execSync } from 'node:child_process';
+import { openDb, saveDb } from './db.js';
+import { twitterBookmarksIndexPath } from './paths.js';
+const BATCH_SIZE = 50;
+function detectEngine() {
+    try {
+        execSync('which claude', { stdio: 'ignore' });
+        return 'claude';
+    }
+    catch { /* not found */ }
+    try {
+        execSync('which codex', { stdio: 'ignore' });
+        return 'codex';
+    }
+    catch { /* not found */ }
+    return null;
+}
+function invokeEngine(engine, prompt) {
+    const bin = engine === 'claude' ? 'claude' : 'codex';
+    const args = engine === 'claude'
+        ? ['-p', '--output-format', 'text', prompt]
+        : ['exec', '--full-auto', prompt];
+    return execFileSync(bin, args, {
+        encoding: 'utf-8',
+        timeout: 120_000, // 2 minutes per batch
+        maxBuffer: 1024 * 1024,
+        stdio: ['pipe', 'pipe', 'ignore'],
+    }).trim();
+}
+// ── Prompt construction ─────────────────────────────────────────────────
+function buildPrompt(bookmarks) {
+    const items = bookmarks.map((b, i) => {
+        const links = b.links ? ` | Links: ${b.links}` : '';
+        return `[${i}] id=${b.id} @${b.authorHandle ?? 'unknown'}: ${b.text.slice(0, 300)}${links}`;
+    }).join('\n');
+    return `Classify each bookmark into one or more categories. Return ONLY a JSON array, no other text.
+
+Known categories:
+- tool: GitHub repos, CLI tools, npm packages, open-source projects, developer tools
+- security: CVEs, vulnerabilities, exploits, supply chain attacks, breaches, hacking
+- technique: tutorials, "how I built X", code patterns, architecture deep dives, demos
+- launch: product launches, announcements, "just shipped", new releases
+- research: academic papers, arxiv, studies, scientific findings
+- opinion: hot takes, commentary, threads, "lessons learned", analysis
+- commerce: products for sale, shopping, affiliate links, physical goods
+
+You may create new categories if a bookmark clearly doesn't fit the above. Use short lowercase slugs (e.g. "health", "design", "career", "culture", "ai-news", "personal-story"). Prefer existing categories when they fit.
+
+Rules:
+- A bookmark can have multiple categories (e.g. a security tool is both "security" and "tool")
+- "primary" is the single best-fit category
+- If nothing fits well, create an appropriate new category rather than forcing a bad fit
+- Return valid JSON only: [{"id":"...","categories":["..."],"primary":"..."},...]
+
+Bookmarks:
+${items}`;
+}
+// ── Parse and validate response ─────────────────────────────────────────
+function parseResponse(raw, batchIds) {
+    // Extract JSON array from response (model might add markdown fences or commentary)
+    const jsonMatch = raw.match(/\[[\s\S]*\]/);
+    if (!jsonMatch)
+        throw new Error('No JSON array found in response');
+    const parsed = JSON.parse(jsonMatch[0]);
+    if (!Array.isArray(parsed))
+        throw new Error('Response is not an array');
+    const results = [];
+    for (const item of parsed) {
+        if (!item.id || !batchIds.has(item.id))
+            continue;
+        const rawArr = item.categories ?? item.domains ?? [];
+        const categories = (Array.isArray(rawArr) ? rawArr : [])
+            .filter((c) => typeof c === 'string' && c.length > 0)
+            .map((c) => c.toLowerCase().trim());
+        const primary = (typeof item.primary === 'string' && item.primary.length > 0)
+            ? item.primary.toLowerCase().trim()
+            : categories[0];
+        if (categories.length > 0 && primary) {
+            results.push({ id: item.id, categories, primary });
+        }
+    }
+    return results;
+}
+export async function classifyWithLlm(options = {}) {
+    const engine = detectEngine();
+    if (!engine) {
+        throw new Error('No supported LLM CLI found.\n' +
+            'Install one of the following and log in:\n' +
+            '  - Claude Code: https://docs.anthropic.com/en/docs/claude-code\n' +
+            '  - Codex CLI:   https://github.com/openai/codex');
+    }
+    const dbPath = twitterBookmarksIndexPath();
+    const db = await openDb(dbPath);
+    try {
+        // Fetch unclassified bookmarks
+        const rows = db.exec(`SELECT id, text, author_handle, links_json FROM bookmarks
+       WHERE primary_category = 'unclassified' OR primary_category IS NULL
+       ORDER BY RANDOM()`);
+        if (!rows.length || !rows[0].values.length) {
+            return { engine, totalUnclassified: 0, classified: 0, failed: 0, batches: 0 };
+        }
+        const unclassified = rows[0].values.map(r => ({
+            id: r[0],
+            text: r[1],
+            authorHandle: r[2],
+            links: r[3],
+        }));
+        const totalUnclassified = unclassified.length;
+        let classified = 0;
+        let failed = 0;
+        let batchCount = 0;
+        // Process in batches
+        for (let i = 0; i < unclassified.length; i += BATCH_SIZE) {
+            const batch = unclassified.slice(i, i + BATCH_SIZE);
+            const batchIds = new Set(batch.map(b => b.id));
+            batchCount++;
+            options.onBatch?.(i, totalUnclassified);
+            try {
+                const prompt = buildPrompt(batch);
+                const raw = invokeEngine(engine, prompt);
+                const results = parseResponse(raw, batchIds);
+                // Update SQLite
+                const stmt = db.prepare(`UPDATE bookmarks SET categories = ?, primary_category = ? WHERE id = ?`);
+                for (const r of results) {
+                    stmt.run([r.categories.join(','), r.primary, r.id]);
+                }
+                stmt.free();
+                classified += results.length;
+                failed += batch.length - results.length;
+                // Save after each batch in case of interruption
+                saveDb(db, dbPath);
+            }
+            catch (err) {
+                failed += batch.length;
+                process.stderr.write(`  Batch ${batchCount} failed: ${err.message}\n`);
+            }
+        }
+        return { engine, totalUnclassified, classified, failed, batches: batchCount };
+    }
+    finally {
+        db.close();
+    }
+}
+function buildDomainPrompt(bookmarks) {
+    const items = bookmarks.map((b, i) => {
+        const cats = b.categories ? ` [${b.categories}]` : '';
+        return `[${i}] id=${b.id} @${b.authorHandle ?? 'unknown'}${cats}: ${b.text.slice(0, 300)}`;
+    }).join('\n');
+    return `Classify each bookmark by its SUBJECT DOMAIN — the topic or field it's about, NOT its format.
+
+The bookmark's format (tool, technique, opinion, etc.) is already classified. Your job: what FIELD does this belong to?
+
+Examples:
+- A "technique" about Docker optimization → domain: "devops"
+- A "technique" about diet plans → domain: "health"
+- A "tool" for an AI agent framework → domain: "ai"
+- An "opinion" about egg freezing → domain: "health"
+- An "opinion" about market cycles → domain: "finance"
+
+Known domains (prefer these when they fit):
+ai, finance, defense, crypto, web-dev, devops, startups, health, politics, design, education, science, hardware, gaming, media, energy, legal, robotics, space
+
+You may create new domain slugs if needed. Use short lowercase slugs. Prefer broad domains ("ai" not "ai-agents", "finance" not "quantitative-trading").
+
+Rules:
+- A bookmark can have multiple domains (e.g. an AI tool for finance is "ai,finance")
+- "primary" is the single best-fit domain
+- Return valid JSON only: [{"id":"...","domains":["..."],"primary":"..."},...]
+
+Bookmarks:
+${items}`;
+}
+export async function classifyDomainsWithLlm(options = {}) {
+    const engine = detectEngine();
+    if (!engine) {
+        throw new Error('No supported LLM CLI found.\n' +
+            'Install one of the following and log in:\n' +
+            '  - Claude Code: https://docs.anthropic.com/en/docs/claude-code\n' +
+            '  - Codex CLI:   https://github.com/openai/codex');
+    }
+    const dbPath = twitterBookmarksIndexPath();
+    const db = await openDb(dbPath);
+    // Ensure domain columns exist (migration from schema v2)
+    try {
+        db.run('ALTER TABLE bookmarks ADD COLUMN domains TEXT');
+    }
+    catch { /* already exists */ }
+    try {
+        db.run('ALTER TABLE bookmarks ADD COLUMN primary_domain TEXT');
+    }
+    catch { /* already exists */ }
+    try {
+        const where = options.all
+            ? '1=1'
+            : 'primary_domain IS NULL';
+        const rows = db.exec(`SELECT id, text, author_handle, categories FROM bookmarks
+       WHERE ${where} ORDER BY RANDOM()`);
+        if (!rows.length || !rows[0].values.length) {
+            return { engine, totalUnclassified: 0, classified: 0, failed: 0, batches: 0 };
+        }
+        const bookmarks = rows[0].values.map(r => ({
+            id: r[0],
+            text: r[1],
+            authorHandle: r[2],
+            categories: r[3],
+        }));
+        const total = bookmarks.length;
+        let classified = 0;
+        let failed = 0;
+        let batchCount = 0;
+        for (let i = 0; i < bookmarks.length; i += BATCH_SIZE) {
+            const batch = bookmarks.slice(i, i + BATCH_SIZE);
+            const batchIds = new Set(batch.map(b => b.id));
+            batchCount++;
+            options.onBatch?.(i, total);
+            try {
+                const prompt = buildDomainPrompt(batch);
+                const raw = invokeEngine(engine, prompt);
+                // Reuse the same parse logic — structure is identical
+                const results = parseResponse(raw, batchIds);
+                const stmt = db.prepare(`UPDATE bookmarks SET domains = ?, primary_domain = ? WHERE id = ?`);
+                for (const r of results) {
+                    stmt.run([r.categories.join(','), r.primary, r.id]);
+                }
+                stmt.free();
+                classified += results.length;
+                failed += batch.length - results.length;
+                saveDb(db, dbPath);
+            }
+            catch (err) {
+                failed += batch.length;
+                process.stderr.write(`  Batch ${batchCount} failed: ${err.message}\n`);
+            }
+        }
+        return { engine, totalUnclassified: total, classified, failed, batches: batchCount };
+    }
+    finally {
+        db.close();
+    }
+}

package/dist/bookmark-classify.js

@@ -0,0 +1,223 @@
+/**
+ * Bookmark classification — tags each bookmark by type so the theories
+ * system can match intelligently instead of random-sampling.
+ *
+ * Categories (non-exclusive, a bookmark can have multiple):
+ *   tool       — GitHub repos, CLI tools, npm packages, open-source projects
+ *   security   — CVEs, vulnerabilities, supply chain, exploits
+ *   technique  — tutorials, demos, code patterns, "how I built X"
+ *   launch     — product launches, announcements, "just shipped"
+ *   research   — arxiv papers, studies, academic findings
+ *   opinion    — takes, analysis, commentary, threads
+ *   commerce   — products, shopping, physical goods
+ *
+ * The classifier is rule-based (fast, predictable, no LLM cost).
+ * It runs over the full corpus in <1s and stores results in the SQLite index.
+ */
+// ── Pattern sets ─────────────────────────────────────────────────────────
+const TOOL_PATTERNS = [
+    /github\.com\/[\w-]+\/[\w-]+/i,
+    /\bnpm\s+(install|i)\b/i,
+    /\bpip\s+install\b/i,
+    /\bcargo\s+add\b/i,
+    /\bbrew\s+install\b/i,
+    /\bopen[\s-]?source\b/i,
+    /\bcli\b.*\btool\b/i,
+    /\btool\b.*\bcli\b/i,
+    /\brust\s+crate\b/i,
+    /\bvscode\s+extension\b/i,
+    /\bnpx\s+/i,
+    /\brepo\b.*\bgithub\b/i,
+    /\bgithub\b.*\brepo\b/i,
+    /\bself[\s-]?hosted\b/i,
+    /\bopen[\s-]?sourced?\b/i,
+];
+const SECURITY_PATTERNS = [
+    /\bcve[-\s]?\d{4}/i,
+    /\bvulnerabilit/i,
+    /\bexploit/i,
+    /\bmalware\b/i,
+    /\bransomware\b/i,
+    /\bsupply[\s-]?chain\s+attack/i,
+    /\bsecurity\s+(flaw|bug|issue|patch|advisory|update|breach)/i,
+    /\bbreach\b/i,
+    /\bbackdoor\b/i,
+    /\bzero[\s-]?day\b/i,
+    /\bremote\s+code\s+execution\b/i,
+    /\brce\b/i,
+    /\bprivilege\s+escalation\b/i,
+    /\bcompromised?\b/i,
+];
+const TECHNIQUE_PATTERNS = [
+    /\bhow\s+(I|we|to)\b/i,
+    /\btutorial\b/i,
+    /\bwalkthrough\b/i,
+    /\bstep[\s-]?by[\s-]?step\b/i,
+    /\bbuilt\s+(with|using|this|a|an|my)\b/i,
+    /\bhere'?s?\s+how\b/i,
+    /\bcode\s+(pattern|example|snippet|sample)\b/i,
+    /\barchitecture\b.*\b(of|for|behind)\b/i,
+    /\bimplemented?\b.*\bfrom\s+scratch\b/i,
+    /\bunder\s+the\s+hood\b/i,
+    /\bdeep[\s-]?dive\b/i,
+    /\btechnique\b/i,
+    /\bpattern\b.*\b(for|in|to)\b/i,
+];
+const LAUNCH_PATTERNS = [
+    /\bjust\s+(launched|shipped|released|dropped|published)\b/i,
+    /\bwe('re|\s+are)\s+(launching|shipping|releasing)\b/i,
+    /\bannouncing\b/i,
+    /\bintroduc(ing|es?)\b/i,
+    /\bnow\s+(available|live|in\s+beta)\b/i,
+    /\bv\d+\.\d+/i,
+    /\b(alpha|beta)\s+(release|launch|is\s+here)\b/i,
+    /\bproduct\s+hunt\b/i,
+    /🚀.*\b(launch|ship|live)\b/i,
+    /\bcheck\s+it\s+out\b/i,
+];
+const RESEARCH_PATTERNS = [
+    /arxiv\.org/i,
+    /\bpaper\b.*\b(new|our|this|the)\b/i,
+    /\b(new|our|this)\b.*\bpaper\b/i,
+    /\bstudy\b.*\b(finds?|shows?|reveals?)\b/i,
+    /\bfindings?\b/i,
+    /\bpeer[\s-]?review/i,
+    /\bpreprint\b/i,
+    /\bresearch\b.*\b(from|by|at|shows?)\b/i,
+    /\bpublished\s+in\b/i,
+    /\bjournal\b/i,
+    /\bstate[\s-]?of[\s-]?the[\s-]?art\b/i,
+];
+const OPINION_PATTERNS = [
+    /\bthread\b.*👇/i,
+    /\bunpopular\s+opinion\b/i,
+    /\bhot\s+take\b/i,
+    /\bhere'?s?\s+(why|what|my\s+take)\b/i,
+    /\bi\s+think\b.*\b(about|that)\b/i,
+    /\bcontroversial\b/i,
+    /\boverrated\b/i,
+    /\bunderrated\b/i,
+    /\blessons?\s+(learned|from)\b/i,
+    /\bmistakes?\s+(I|we)\b/i,
+];
+const COMMERCE_PATTERNS = [
+    /\bamazon\.com\b/i,
+    /\bshop\s+(here|now)\b/i,
+    /\bbuy\s+(now|here|this)\b/i,
+    /\bdiscount\b/i,
+    /\bcoupon\b/i,
+    /\baffiliate\b/i,
+    /\bgeni\.us\b/i,
+    /\ba\.co\//i,
+    /\$\d+(\.\d{2})?\s*(off|USD|discount)/i,
+];
+const GITHUB_URL_RE = /github\.com\/[\w.-]+\/[\w.-]+/gi;
+const URL_RE = /https?:\/\/[^\s)>\]]+/gi;
+const TCO_RE = /https?:\/\/t\.co\/\w+/gi;
+// ── Domains that indicate tool/project bookmarks ─────────────────────────
+const TOOL_DOMAINS = new Set([
+    'github.com',
+    'gitlab.com',
+    'huggingface.co',
+    'npmjs.com',
+    'pypi.org',
+    'crates.io',
+    'pkg.go.dev',
+]);
+const RESEARCH_DOMAINS = new Set([
+    'arxiv.org',
+    'scholar.google.com',
+    'semanticscholar.org',
+    'biorxiv.org',
+    'medrxiv.org',
+    'nature.com',
+    'science.org',
+]);
+const COMMERCE_DOMAINS = new Set([
+    'amazon.com',
+    'www.amazon.com',
+    'a.co',
+    'store.steampowered.com',
+    'geni.us',
+    'ebay.com',
+]);
+// ── Classify a single bookmark ───────────────────────────────────────────
+export function classifyBookmark(bookmark) {
+    const text = bookmark.text ?? '';
+    const allLinks = [...(bookmark.links ?? [])];
+    // Extract URLs from tweet text (excluding t.co shortlinks)
+    const textUrls = (text.match(URL_RE) ?? []).filter((u) => !TCO_RE.test(u));
+    const extractedUrls = [...new Set([...allLinks, ...textUrls])];
+    // Extract GitHub URLs
+    const githubMatches = text.match(GITHUB_URL_RE) ?? [];
+    const githubFromLinks = allLinks.filter((l) => /github\.com/i.test(l));
+    const githubUrls = [...new Set([...githubMatches.map((m) => `https://${m}`), ...githubFromLinks])];
+    // Get domains from all URLs
+    const domains = extractedUrls
+        .map((u) => {
+        try {
+            return new URL(u).hostname.replace(/^www\./, '');
+        }
+        catch {
+            return '';
+        }
+    })
+        .filter(Boolean);
+    const categories = [];
+    // Pattern matching
+    const matchesAny = (patterns) => patterns.some((p) => p.test(text));
+    if (matchesAny(SECURITY_PATTERNS))
+        categories.push('security');
+    if (matchesAny(TOOL_PATTERNS) || githubUrls.length > 0 || domains.some((d) => TOOL_DOMAINS.has(d)))
+        categories.push('tool');
+    if (matchesAny(TECHNIQUE_PATTERNS))
+        categories.push('technique');
+    if (matchesAny(LAUNCH_PATTERNS))
+        categories.push('launch');
+    if (matchesAny(RESEARCH_PATTERNS) || domains.some((d) => RESEARCH_DOMAINS.has(d)))
+        categories.push('research');
+    if (matchesAny(OPINION_PATTERNS))
+        categories.push('opinion');
+    if (matchesAny(COMMERCE_PATTERNS) || domains.some((d) => COMMERCE_DOMAINS.has(d)))
+        categories.push('commerce');
+    // Primary = first match (ordered by priority above: security > tool > technique > ...)
+    const primary = categories[0] ?? 'unclassified';
+    return { categories, primary, extractedUrls, githubUrls };
+}
+export function classifyCorpus(bookmarks) {
+    const results = new Map();
+    const counts = {};
+    let unclassified = 0;
+    for (const b of bookmarks) {
+        const result = classifyBookmark(b);
+        results.set(b.id, result);
+        if (result.categories.length === 0) {
+            unclassified++;
+        }
+        for (const cat of result.categories) {
+            counts[cat] = (counts[cat] ?? 0) + 1;
+        }
+    }
+    return {
+        results,
+        summary: {
+            total: bookmarks.length,
+            classified: bookmarks.length - unclassified,
+            unclassified,
+            byCategoryCount: counts,
+        },
+    };
+}
+// ── Format summary for CLI output ────────────────────────────────────────
+export function formatClassificationSummary(summary) {
+    const lines = [
+        `Classified ${summary.classified}/${summary.total} bookmarks (${summary.unclassified} unclassified)`,
+        '',
+    ];
+    const sorted = Object.entries(summary.byCategoryCount).sort((a, b) => b[1] - a[1]);
+    for (const [cat, count] of sorted) {
+        const pct = ((count / summary.total) * 100).toFixed(1);
+        lines.push(`  ${cat.padEnd(12)} ${String(count).padStart(5)}  (${pct}%)`);
+    }
+    return lines.join('\n');
+}