@canonry/canonry 4.104.0

package/assets/agent-workspace/skills/canonry/references/server-side-traffic.md

@@ -0,0 +1,372 @@
+# Server-side traffic (AI Visibility — Server-Side)
+
+Server-side traffic ingestion captures **what AI engines actually do in
+your server logs** — bots crawling pages, AI products sending
+click-through arrivals — in addition to the citation data that measures
+**what models say** about you. The two surfaces are independent.
+
+## When to use it
+
+Reach for server-side traffic when an analyst or operator asks:
+
+- *"Is GPTBot / ClaudeBot / PerplexityBot actually fetching my pages?"*
+- *"Which paths are AI engines paying attention to?"*
+- *"Are users clicking through from chatgpt.com / claude.ai / etc.?"*
+- *"My citation rate is fine but there's no traffic — why?"*
+
+GA4 referrals (chatgpt.com → your site) catch click-throughs after they
+land. Server logs catch the upstream bot activity AND referrals at the
+edge — including arrivals GA4 missed because of cookie consent, ad
+blockers, or analytics gaps.
+
+## Architecture
+
+Two tables, populated from server-log adapters:
+
+| Table | What's in it |
+|---|---|
+| `crawler_events_hourly` | One row per `(project, source, hour, bot, verification, path, status)` — bot crawls rolled up by hour |
+| `ai_referral_events_hourly` | One row per `(project, source, hour, product, source_domain, evidence_type, landing_path, status)` — click-through arrivals rolled up by hour |
+| `raw_event_samples` | Bounded forensic samples (≤100 per sync) for spot-checking |
+
+Each `traffic_sources` row is one server-log integration for a project.
+Adapters today:
+
+| Adapter | Source | Best for |
+|---|---|---|
+| `cloud-run` | GCP Cloud Run request logs via Logging API | Any service running on Cloud Run |
+| `wordpress` | The Canonry Traffic Logger WP plugin's REST endpoint | WordPress sites where you control wp-admin |
+| `vercel` | Vercel project logs via the Vercel API | Sites deployed on Vercel (Next.js, SvelteKit, etc.) |
+
+Future adapters slot in by implementing the same contract.
+
+## Connecting a Cloud Run source
+
+```bash
+# 1. Create a service account in the Cloud project that hosts the Cloud Run
+#    service. Grant it `roles/logging.viewer`. Download the JSON key.
+
+# 2. Connect from cnry CLI:
+cnry traffic connect cloud-run <project> \
+  --gcp-project <gcp-project-id> \
+  --service-account-key <path/to/key.json>
+
+# 3. (Optional) narrow to a specific service or location:
+cnry traffic connect cloud-run <project> \
+  --gcp-project <id> \
+  --service-account-key <path> \
+  --service my-service-name \
+  --location us-east1
+```
+
+Credentials are stored in `~/.canonry/config.yaml` (not the DB). The
+canonical key lives only on the host that runs `cnry serve`. The
+sync flow does NOT echo the private key back in any response.
+
+## Connecting a WordPress source
+
+The WordPress adapter pulls events from the **Canonry Traffic Logger**
+WordPress plugin, which captures every non-admin GET page-load **that
+reaches PHP** and exposes a paginated REST endpoint protected by an
+Application Password.
+
+> **Cache blind spot.** The plugin is a PHP hook, so it only sees
+> requests that execute WordPress. A full-page cache (LiteSpeed, WP
+> Rocket, W3 Total Cache, WP Super Cache) or CDN serves cached pages
+> before PHP runs, so cache-served page views, including live AI
+> user-fetches (Claude-User, ChatGPT-User), are NOT logged. Bot crawls
+> of uncached endpoints (sitemap, feeds, assets, cache misses) still
+> come through, which can make capture look healthy while real page
+> views go uncounted. Exclude AI user-agents from the cache (and any
+> CDN), or capture from access/edge logs instead. The
+> `traffic.source.cache-blindspot` doctor check warns whenever a
+> WordPress source is connected.
+
+**Which user-agents to exclude from the cache** (one per line in
+LiteSpeed's "Do Not Cache User Agents", WP Rocket's
+`rocket_cache_reject_ua`, or W3TC / WP Super Cache "Rejected User
+Agents"):
+
+```
+Claude-User
+ClaudeBot
+ChatGPT-User
+OAI-SearchBot
+GPTBot
+PerplexityBot
+Perplexity-User
+```
+
+These are the answer-engine fetchers in both live-user-fetch (`*-User`)
+and crawler forms. Do NOT add `Googlebot` or `Bingbot`: caching helps
+search crawlers (page speed is a ranking signal, and cached pages let
+them crawl more per visit, which matters most on crawl-budget-starved
+sites), and their crawl stats are already authoritative in GSC and Bing
+Webmaster Tools. Rule of thumb: bypass cache only for agents you cannot
+measure elsewhere and that gain nothing from being cached. Answer-engine
+fetchers fit both; search crawlers fit neither.
+
+```bash
+# 1. Install the plugin. Download the latest release zip from the
+#    canonry-traffic-logger plugin's GitHub release (the repo CI workflow
+#    publishes a zip on every plugin-file change), then in wp-admin:
+#    Plugins → Add New → Upload Plugin → upload + activate.
+
+# 2. In wp-admin, create an Application Password for the operator user:
+#    Users → Profile → Application Passwords. Copy the generated password.
+
+# 3. (Optional) Adjust settings at Settings → Canonry Traffic Logger:
+#    - Retention window: clamps to 7-365 days, default 90.
+#    - "Behind a proxy or CDN": enable this when the site sits behind
+#      Cloudflare or another reverse proxy, so the real visitor IP
+#      (needed to verify AI-bot hits) is read from forwarded headers
+#      rather than the proxy's edge address.
+#    The page also shows the current event count and oldest event.
+
+# 4. Connect from cnry CLI:
+cnry traffic connect wordpress <project> \
+  --url https://example.com \
+  --username admin \
+  --app-password "xxxx xxxx xxxx xxxx xxxx xxxx"
+```
+
+What the events table looks like (mirrors the TS
+`WordpressTrafficEventPayload`):
+
+| Column | Meaning |
+|---|---|
+| `observed_at` | ISO 8601 UTC timestamp with millisecond precision |
+| `method`, `host`, `path`, `query_string` | Split `REQUEST_URI` parts |
+| `status` | HTTP response status code |
+| `user_agent`, `referer` | Headers as captured at request time |
+| `remote_ip` | Client IP address (IPv4 or IPv6), or empty when none was captured |
+
+The plugin auto-prunes events older than the retention window (default
+90 days) once per day via WP-Cron. Operators who want a different
+window change it in `Settings → Canonry Traffic Logger`.
+
+## Connecting a Vercel source
+
+The Vercel adapter pulls per-request logs from the Vercel API for a
+specific project + environment. Logs are filtered by canonical domain
+before classification so a multi-tenant Vercel project only surfaces
+hits for the tracked site.
+
+```bash
+# 1. In the Vercel dashboard, create a token with read access to the
+#    target team (Settings → Tokens → Create). Note the team ID
+#    (Settings → General → Team ID) and the Vercel project ID
+#    (Project → Settings → General → Project ID).
+
+# 2. Connect from cnry CLI:
+cnry traffic connect vercel <project> \
+  --project-id prj_xxxxxxxx \
+  --team-id   team_xxxxxxxx \
+  --token     <vercel-token>            # or: --token-file <path>
+
+# 3. (Optional) scope to a specific environment (default: production):
+cnry traffic connect vercel <project> \
+  --project-id prj_xxx --team-id team_xxx --token ... \
+  --environment preview
+```
+
+Credentials live in `~/.canonry/config.yaml` under `vercelTraffic:`,
+mirroring the cloud-run / wordpress blocks. The adapter classifies bot
+crawls + AI-referral arrivals into the same `crawler_events_hourly` /
+`ai_referral_events_hourly` tables — downstream commands
+(`cnry traffic events / sources / status`) are source-agnostic.
+
+### Vercel first-sync window (gotcha)
+
+A new Vercel source captures **only going-forward traffic** by default.
+`cnry traffic connect vercel` seeds `lastSyncedAt = NOW` so the first
+scheduled sync uses a tight window inside Vercel's ~14-day
+`request-logs` retention. Without this, the first sync would fall back
+to a 30-day window, exceed retention, and throw — leaving the source
+permanently stuck.
+
+Run `cnry traffic backfill <project> --source <id> --days N` (capped at
+~14 to stay inside retention) if you need any of the pre-connect
+history. It's an explicit operator action; the connect flow never pulls
+it implicitly.
+
+## Syncing data
+
+```bash
+# Manual sync — pulls [lastSyncedAt, now]. For a freshly connected
+# source the window is short (since connect-time NOW). For a
+# regular-cadence schedule the window stays ~30 min wide.
+cnry traffic sync <project> --source <id>
+
+# Override the lookback window (minutes) — note: clamped forward to
+# lastSyncedAt, so this can only NARROW the window, never widen it
+# past data already pulled.
+cnry traffic sync <project> --source <id> --since-minutes 4320  # 3 days
+```
+
+### Unsticking a stuck source
+
+If a Vercel (or Cloud Run) source has been failing for so long that
+`lastSyncedAt` aged past the upstream retention boundary, every sync
+will throw a retention error and `lastSyncedAt` will never advance —
+the source is permanently stuck. Recovery:
+
+```bash
+# Advances lastSyncedAt to NOW, clears the error state. Skipped
+# history is unrecoverable from the sync path; run backfill separately
+# if any of it needs to be captured.
+cnry traffic reset <project> --source <id> --advance-to-now
+```
+
+`--advance-to-now` is required — there is no implicit reset.
+
+`reset` accepts any **non-archived** source type. The `lastSyncedAt`
+advance is meaningful for time-windowed sources (Vercel, Cloud Run)
+where it determines the next sync window. Cursor-based sources
+(WordPress) keep their `last_cursor` intact, so the `lastSyncedAt`
+advance is informational — the next WordPress drain still resumes
+from the cursor. The primary use case is the retention-trap recovery
+above; clearing `lastError` for a transient WordPress failure also
+works. Archived sources are rejected — re-connect them with
+`cnry traffic connect ...` instead.
+
+Cross-sync dedupe via the `last_event_ids` ring buffer means re-running a
+sync over an overlapping window cannot double-count rolled-up hourly
+hits. Safe to schedule (see "Scheduling" below) or trigger from CI.
+
+## Inspecting source state
+
+```bash
+# All sources with last-24h totals + latest sync run (single-call):
+cnry traffic status <project> --format json
+
+# Just the source list:
+cnry traffic sources <project> --format json
+
+# Windowed events (defaults to last 24h):
+cnry traffic events <project> --kind crawler --limit 200 --format json
+cnry traffic events <project> --kind ai-referral --since 2026-04-01 --until 2026-04-30
+```
+
+The `traffic status` composite returns the same per-source detail
+(24h crawler hits, AI-referral arrivals, raw-event-sample count, latest
+sync-run summary) whether you reach it via the CLI, the API, or the
+MCP `canonry_traffic_status` tool.
+
+**Crawler hits are segmented by path class (#719).** On real sites the raw
+`crawlerHits` total is dominated by infrastructure polling — a bot re-fetching
+`sitemap_index.xml`, `robots.txt`, and static assets — which overstates how
+much of your *content* is being crawled. `traffic status` and `traffic events`
+therefore return, alongside the unchanged `crawlerHits` total:
+
+- `crawlerContentHits` — crawls of actual content/document pages (the signal you
+  usually want: "are bots reading my pages?").
+- `crawlerInfraHits` — sitemap + robots + asset fetches.
+- `crawlerSegments` — the full `{ content, sitemap, robots, asset, other }`
+  breakdown; the five buckets sum to `crawlerHits`, and
+  `content + infra + other == crawlerHits`. `other` captures non-page downloads
+  and feeds (PDF, CSV, RSS) plus WordPress polling endpoints that are not page
+  reads (`/feed`, `/<path>/feed`, `/wp-json/...`, `xmlrpc.php`, `wp-cron.php`), so
+  they stay out of `crawlerContentHits`.
+
+Each crawler row from `traffic events` also carries a `pathClass`
+(`content | sitemap | robots | asset | other`). The dashboard leads with the
+content figure and shows infrastructure polling as a secondary number. The
+classification is read-time only (the pure `classifyTrafficPath` helper) — no
+schema change, the stored rollups are untouched.
+
+## Where the data shows up
+
+| Surface | What's rendered |
+|---|---|
+| Project dashboard `/projects/:name/activity` | Live source table + 24h totals + GA4 referrals (combined view) |
+| Top-level `/traffic` route | Cross-project source admin (connect, sync, archive) |
+| `cnry report <project>` (HTML + SPA) | "AI Visibility — Server-Side" section, ranked above Indexing Health |
+| `cnry doctor --project <name>` | `traffic.source.connected`, `recent-data`, `credentials`, `scopes`, `cache-blindspot` checks |
+| MCP toolkit `traffic` | Tools: `canonry_traffic_status`, `_sources_list`, `_source_get`, `_events`, `_connect_cloud_run`, `_sync` |
+
+## Doctor signals
+
+The doctor checks are adapter-agnostic. When they fail or warn:
+
+| Check | Code | What to do |
+|---|---|---|
+| `traffic.source.connected` | `traffic.source.none` | No source — `cnry traffic connect cloud-run …` |
+| `traffic.source.connected` | `traffic.source.all-errored` | Re-connect the source. The check's `details.lastError` shows the underlying reason. |
+| `traffic.source.recent-data` | `traffic.recent-data.stale` | Last sync was >7d ago. Run `cnry traffic sync …` or schedule a recurring sync. |
+| `traffic.source.recent-data` | `traffic.recent-data.empty` | Source connected but no data in 30d. Verify config and credentials with `cnry traffic sources <project>`. |
+| `traffic.source.credentials` | `traffic.credentials.resolve-failed` | Service-account key in `~/.canonry/config.yaml` is invalid or expired. Re-connect. |
+| `traffic.source.cache-blindspot` | `traffic.cache-blindspot.wordpress-plugin` | A WordPress source is connected, so the plugin cannot see cache-served page views. Exclude AI user-agents from the page cache and any CDN, or switch to a log/edge source. Warns only, not a failure. |
+
+## Scheduling
+
+`cnry schedule` supports `--kind traffic-sync`. Recurring syncs are
+safe because of the `last_event_ids` cross-sync dedupe ring buffer
+described above. Recommended cadence:
+
+| Cadence | Use case |
+|---|---|
+| `0 */6 * * *` (every 6h) | Production agencies tracking active client sites |
+| `0 0 * * *` (daily) | Lower-traffic sites or local dev |
+| Manual only | First few weeks while validating data |
+
+## Telemetry
+
+Every successful or failed sync emits a `traffic.synced` event to the
+canonry telemetry pipeline:
+
+```jsonc
+{
+  "event": "traffic.synced",
+  "errorCode": "PROVIDER_AUTH",       // present only when status='failed'
+  "properties": {
+    "status": "completed" | "failed",
+    "sourceType": "cloud-run",        // adapter type
+    "sourceId": "<uuid>",             // opaque
+    "pulledEvents": 234,
+    "crawlerHits": 200,
+    "aiReferralHits": 12,
+    "durationMs": 4150
+  }
+}
+```
+
+Counts are aggregate. The sourceId is an opaque UUID. No raw paths,
+domains, or PII are surfaced.
+
+## Limits & caveats
+
+- **The WordPress plugin is blind to cache-served traffic.** The
+  `wordpress` adapter logs only requests that reach PHP. A full-page
+  cache or CDN serves cached pages from the edge, so cache-served page
+  views, including live AI user-fetches (Claude-User, ChatGPT-User),
+  never reach the plugin and go uncounted, even though bot crawls of
+  uncached endpoints (sitemap, assets) still appear. On a cached
+  WordPress site, treat the plugin's page-view counts as a floor, not a
+  total. Either exclude AI user-agents from the cache + CDN, or capture
+  cache-independent via a `cloud-run` / `vercel` / edge-log source. The
+  `traffic.source.cache-blindspot` doctor check surfaces this. Adapter
+  coverage differs: `vercel` ingests edge request-logs so cache hits are
+  captured (it records the `cache` HIT/MISS label), and `cloud-run` logs
+  every request that reaches the service, missing only what a CDN placed
+  in front of Cloud Run serves from its own edge cache. Only the
+  hook-based `wordpress` adapter has the always-present blind spot.
+- **Path-level citation cross-reference is not implemented yet.** The
+  citation store is domain-grain (`query_snapshots.cited_domains`). A
+  future iteration that lands URL-grain citation evidence will extend
+  the `topCrawledPaths` entry with a `citationState` flag. Until then,
+  treat the report's crawled-paths table as "engine attention" — the
+  signal is the bot fetched it, not whether it was cited.
+- **Verified vs unverified.** The headline numbers count only
+  rDNS-verified hits. Unverified bots claim a known UA but couldn't be
+  cross-confirmed via reverse-DNS — they may be the real bot or an
+  imitator. Don't promote unverified counts in client-facing copy.
+  **Vercel sources are a special case:** the Vercel pull API returns
+  no client IP, so every Vercel crawler hit is unverified by
+  construction (UA-only). A Vercel source reading 100% unverified is
+  expected, not a misconfiguration.
+- **Three adapters shipped (Cloud Run + WordPress + Vercel); more
+  planned.** The doctor checks and the report renderer are
+  adapter-agnostic — adding a new adapter is just a new entry in
+  `traffic_sources.source_type` and a `TrafficSourceValidator`
+  registration.

package/assets/agent-workspace/skills/canonry/references/wordpress-integration.md

@@ -0,0 +1,61 @@
+# WordPress Integration
+
+Canonry integrates with WordPress through the core REST API plus Application Passwords.
+
+## What Canonry Automates
+
+- Read and write page content, titles, slugs, and status
+- Audit pages for `noindex`, missing SEO title, missing meta description, missing schema, and thin content
+- Compare live vs staging for a page with `cnry wordpress diff`
+- Update SEO meta when the site exposes writable REST meta fields
+
+## What Stays Manual
+
+Canonry generates payloads and instructions for these workflows, but it does not apply them remotely:
+
+- `cnry wordpress set-schema`
+- `cnry wordpress set-llms-txt`
+- `cnry wordpress staging push`
+
+Expect those commands to return:
+
+- `manualRequired: true`
+- generated content
+- target/admin URL when available
+- next-step instructions
+
+## Environment Model
+
+Each project-scoped WordPress connection stores:
+
+- live `url`
+- optional `stagingUrl`
+- `defaultEnv`
+- `username`
+- `appPassword`
+
+Env-sensitive commands accept `--live` or `--staging`. If neither is provided, canonry uses `defaultEnv`.
+
+## Typical Workflow
+
+```bash
+cnry wordpress connect mysite --url https://example.com --user admin --staging-url https://staging.example.com --default-env staging
+cnry wordpress pages mysite --staging
+cnry wordpress page mysite pricing --staging
+cnry wordpress set-meta mysite pricing --title "SEO title" --description "Meta description" --staging
+cnry wordpress audit mysite --staging
+cnry wordpress diff mysite pricing
+cnry wordpress staging push mysite
+```
+
+## Important Constraints
+
+- WordPress auth is stored locally in `~/.canonry/config.yaml`
+- Canonry does not use wp-admin automation or undocumented plugin APIs
+- If SEO meta is not writable through REST, canonry returns an actionable error instead of guessing
+- Duplicate slug matches are returned as explicit ambiguity errors with candidate page IDs/titles
+- Authentication is verified on connect by calling `/wp/v2/users/me` — if that fails, canonry returns an actionable error message
+
+## Related: Elementor MCP
+
+For programmatic management of Elementor page layouts, widgets, and styling via MCP tools, see the aero skill reference: [`skills/aero/references/wordpress-elementor-mcp.md`](../../aero/references/wordpress-elementor-mcp.md).

package/assets/assets/BacklinksPage-CixBvw--.js

@@ -0,0 +1 @@
+import{r as n,j as e}from"./vendor-tanstack-Be8JdV5M.js";import{c as E,bR as O,ag as V,bS as U,bT as Y,bU as J,g as l,aJ as L,T as f,B as z,i as A,ap as K,bV as X,bW as Z,aj as ee,bX as se}from"./index-avJiga1P.js";import{C as ae,D as te,T as ne,a as ce}from"./trash-2-CA_L6uGC.js";import"./vendor-radix-CQ77EO2y.js";import"./vendor-recharts-C9EZkgbP.js";import"./vendor-markdown-6dwjPOKK.js";const ie=[["circle",{cx:"12",cy:"12",r:"10",key:"1mglay"}],["line",{x1:"12",x2:"12",y1:"8",y2:"12",key:"1pkeuh"}],["line",{x1:"12",x2:"12.01",y1:"16",y2:"16",key:"4dfq90"}]],re=E("circle-alert",ie);const le=[["path",{d:"M15 3h6v6",key:"1q9fwt"}],["path",{d:"M10 14 21 3",key:"gplh6r"}],["path",{d:"M18 13v6a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2V8a2 2 0 0 1 2-2h6",key:"a6xqqp"}]],de=E("external-link",le);function h({children:a,label:m="More info",placement:t="top",className:d}){const o=n.useId(),[y,r]=n.useState(!1);return e.jsxs("span",{className:`relative inline-flex ${d??""}`,children:[e.jsx("button",{type:"button","aria-label":m,"aria-describedby":y?o:void 0,className:"inline-flex h-4 w-4 items-center justify-center rounded-full text-zinc-500 hover:text-zinc-200 focus:text-zinc-200 focus:outline-none focus-visible:ring-1 focus-visible:ring-zinc-500",onMouseEnter:()=>r(!0),onMouseLeave:()=>r(!1),onFocus:()=>r(!0),onBlur:()=>r(!1),children:e.jsx(ce,{className:"h-3.5 w-3.5","aria-hidden":!0})}),y&&e.jsx("span",{id:o,role:"tooltip",className:`absolute z-50 w-64 rounded border border-zinc-700 bg-zinc-900 px-3 py-2 text-xs font-normal leading-relaxed text-zinc-200 shadow-lg ${t==="top"?"bottom-full mb-2":"top-full mt-2"} left-1/2 -translate-x-1/2 whitespace-normal`,children:a})]})}const oe="https://commoncrawl.org/web-graphs";function w(a){return a==null?"—":a>=1e12?`${(a/1e12).toFixed(1)} TB`:a>=1e9?`${(a/1e9).toFixed(1)} GB`:a>=1e6?`${(a/1e6).toFixed(1)} MB`:a>=1e3?`${(a/1e3).toFixed(1)} KB`:`${a} B`}function b(a){if(!a)return"—";const m=Date.now()-new Date(a).getTime(),t=Math.floor(m/6e4);if(t<1)return"just now";if(t<60)return`${t}m ago`;const d=Math.floor(t/60);return d<24?`${d}h ago`:`${Math.floor(d/24)}d ago`}function v(a){switch(a){case"ready":return"positive";case"failed":return"negative";case"downloading":case"querying":case"queued":return"caution"}}function fe(){const[a,m]=n.useState(null),[t,d]=n.useState(null),[o,y]=n.useState([]),[r,M]=n.useState([]),[u,P]=n.useState(null),[D,S]=n.useState(!0),[C,B]=n.useState(!1),[p,R]=n.useState(!1),[N,g]=n.useState(""),[F,k]=n.useState(!1),[q,i]=n.useState(null),[I,x]=n.useState(null),j=n.useCallback(async()=>{S(!0),i(null);try{const[s,c,W,_,H]=await Promise.all([O(),V().catch(()=>null),U().catch(()=>[]),Y().catch(()=>[]),J().catch(()=>null)]);m(s),d(c),y(W),M(_),P(H)}catch(s){i(s instanceof Error?s.message:"Failed to load backlinks status")}finally{S(!1)}},[]);n.useEffect(()=>{j()},[j]);async function T(){B(!0),i(null),x(null);try{const s=await X();x(s.alreadyPresent?`DuckDB already installed (${s.version}).`:`Installed DuckDB ${s.version}.`),await j()}catch(s){i(s instanceof Error?s.message:"Failed to install DuckDB")}finally{B(!1)}}async function $(){const s=N.trim()||void 0;R(!0),i(null),x(null);try{const c=await Z(s);x(s?`Queued sync for ${c.release}. Download + query runs in the background.`:`Queued sync for auto-discovered release ${c.release}. Download + query runs in the background.`),g(""),k(!1),await j()}catch(c){c instanceof ee&&c.code==="MISSING_DEPENDENCY"?i("DuckDB is not installed. Install it first."):i(c instanceof Error?c.message:"Failed to trigger sync")}finally{R(!1)}}async function Q(s){i(null),x(null);try{await se(s),x(`Pruned cached release ${s}.`),await j()}catch(c){i(c instanceof Error?c.message:"Failed to prune release")}}const G=t?.status==="ready"&&r.every(s=>s.release!==t.release);return e.jsxs("div",{className:"page-container",children:[e.jsx("div",{className:"page-header",children:e.jsxs("div",{className:"page-header-left",children:[e.jsx("h1",{className:"page-title",children:"Backlinks"}),e.jsx("p",{className:"page-subtitle",children:"Find domains that link to your projects, computed from the open Common Crawl web graph. Runs entirely on your machine — nothing is sent to third parties."})]})}),e.jsx(l,{className:"surface-card p-4 mb-6 border-amber-800/60",children:e.jsxs("div",{className:"flex items-start gap-3",children:[e.jsx(L,{className:"h-5 w-5 text-amber-400 shrink-0 mt-0.5","aria-hidden":!0}),e.jsxs("div",{className:"text-sm text-zinc-300 leading-relaxed",children:[e.jsx("p",{className:"font-medium text-amber-200",children:"Heads up — a release sync is a large download."}),e.jsxs("ul",{className:"mt-1.5 space-y-1 text-zinc-400",children:[e.jsxs("li",{children:[e.jsx("span",{className:"text-zinc-200",children:"~16 GB"})," of gzipped vertex + edge files per release, stored at"," ",e.jsx("code",{className:"text-zinc-300",children:"~/.canonry/cache/commoncrawl/"}),"."]}),e.jsxs("li",{children:[e.jsx("span",{className:"text-zinc-200",children:"10–20 min on a fast connection"})," for the download, then ~5 min for the DuckDB query."]}),e.jsx("li",{children:"One sync covers every project in this workspace. Releases are immutable, so the download only happens once per release."})]})]})]})}),e.jsxs("section",{className:"page-section-divider",children:[e.jsx("div",{className:"section-head section-head-inline",children:e.jsxs("div",{children:[e.jsx("p",{className:"eyebrow eyebrow-soft",children:"About"}),e.jsx("h2",{children:"How it works"})]})}),e.jsxs(l,{className:"surface-card p-5",children:[e.jsxs("p",{className:"text-sm text-zinc-400 leading-relaxed max-w-3xl mb-4",children:["Common Crawl publishes a quarterly snapshot of the public web’s hyperlink graph. Canonry downloads one"," ",e.jsx("span",{className:"text-zinc-200",children:"release"})," at a time and extracts backlinks for every project in this workspace in a single pass."]}),e.jsxs("ol",{className:"space-y-3 text-sm text-zinc-400 max-w-3xl",children:[e.jsxs("li",{className:"flex gap-3",children:[e.jsx("span",{className:"shrink-0 inline-flex h-6 w-6 items-center justify-center rounded-full border border-zinc-700 bg-zinc-900 text-xs font-semibold text-zinc-300 tabular-nums",children:"1"}),e.jsxs("span",{children:[e.jsx("span",{className:"text-zinc-200 font-medium",children:"Download (one-time, ~16 GB)"})," — vertex + edge files cached to"," ",e.jsx("code",{className:"text-zinc-300",children:"~/.canonry/cache/commoncrawl/"}),". Runs once per release; subsequent operations reuse the cache."]})]}),e.jsxs("li",{className:"flex gap-3",children:[e.jsx("span",{className:"shrink-0 inline-flex h-6 w-6 items-center justify-center rounded-full border border-zinc-700 bg-zinc-900 text-xs font-semibold text-zinc-300 tabular-nums",children:"2"}),e.jsxs("span",{children:[e.jsx("span",{className:"text-zinc-200 font-medium",children:"Query (~5 min)"})," — one DuckDB pass scans the cached files and extracts referring domains for every project’s canonical domain. DuckDB is only used to ",e.jsx("span",{className:"text-zinc-200",children:"read"})," these dumps; it doesn’t store any canonry state."]})]}),e.jsxs("li",{className:"flex gap-3",children:[e.jsx("span",{className:"shrink-0 inline-flex h-6 w-6 items-center justify-center rounded-full border border-zinc-700 bg-zinc-900 text-xs font-semibold text-zinc-300 tabular-nums",children:"3"}),e.jsxs("span",{children:[e.jsx("span",{className:"text-zinc-200 font-medium",children:"Persist"})," — results land in the same SQLite database the rest of canonry uses. After the first sync, per-project reads (and re-run extracts against the cached release) are instant."]})]})]})]})]}),q&&e.jsx(l,{className:"surface-card p-4 mb-4 border-rose-800/60",children:e.jsx("p",{className:"text-sm text-rose-300",children:q})}),I&&e.jsx(l,{className:"surface-card p-4 mb-4 border-emerald-800/60",children:e.jsx("p",{className:"text-sm text-emerald-300",children:I})}),e.jsxs("section",{className:"page-section-divider",children:[e.jsxs("div",{className:"section-head section-head-inline",children:[e.jsxs("div",{children:[e.jsx("p",{className:"eyebrow eyebrow-soft",children:"Dependency"}),e.jsxs("h2",{className:"flex items-center gap-2",children:["DuckDB install status",e.jsxs(h,{label:"Why DuckDB?",children:[e.jsx("span",{className:"block",children:"DuckDB is a query engine canonry uses to scan the ~16 GB Common Crawl dumps and pull out your referring domains."}),e.jsxs("span",{className:"mt-2 block text-zinc-400",children:["It does ",e.jsx("span",{className:"text-zinc-200",children:"not"})," store any canonry data — your backlink results live in SQLite alongside the rest of your projects. DuckDB is purely a tool for processing the raw CSV files."]}),e.jsxs("span",{className:"mt-2 block text-zinc-500",children:["Installed on demand (not bundled) into ",e.jsx("code",{className:"text-zinc-300",children:"~/.canonry/plugins/"})," so users who never run backlinks don’t pay the ~40 MB install cost."]})]})]})]}),a?.duckdbInstalled?e.jsx(f,{tone:"positive",children:"Installed"}):e.jsx(f,{tone:"caution",children:"Not installed"})]}),e.jsx(l,{className:"surface-card p-5",children:D?e.jsx("p",{className:"text-sm text-zinc-500",children:"Checking…"}):a?.duckdbInstalled?e.jsxs("div",{className:"flex items-start gap-3",children:[e.jsx(ae,{className:"h-5 w-5 text-emerald-400 shrink-0 mt-0.5","aria-hidden":!0}),e.jsxs("div",{children:[e.jsxs("p",{className:"text-sm text-zinc-200",children:["Version ",a.duckdbVersion??"unknown"," installed at"," ",e.jsx("code",{className:"text-zinc-300",children:a.pluginDir})]}),e.jsxs("p",{className:"text-xs text-zinc-500 mt-1",children:["Required spec: ",a.duckdbSpec]})]})]}):e.jsxs("div",{className:"flex items-start gap-3",children:[e.jsx(re,{className:"h-5 w-5 text-amber-400 shrink-0 mt-0.5","aria-hidden":!0}),e.jsxs("div",{className:"flex-1",children:[e.jsx("p",{className:"text-sm text-zinc-200",children:"DuckDB is not installed. It’s the query engine canonry uses to scan Common Crawl dumps — required before you can run a release sync or per-project extract."}),e.jsx("p",{className:"text-xs text-zinc-500 mt-1",children:"Installing doesn’t touch your project data. DuckDB only reads the downloaded CSV files; backlink results are written to the same SQLite database canonry already uses."}),a&&e.jsxs("p",{className:"text-xs text-zinc-500 mt-1",children:["Will be installed into ",e.jsx("code",{className:"text-zinc-300",children:a.pluginDir})," (~40 MB)."]}),e.jsx("div",{className:"mt-3",children:e.jsxs(z,{type:"button",size:"sm",disabled:C,onClick:A(T),children:[e.jsx(te,{className:"h-4 w-4 mr-1.5","aria-hidden":!0}),C?"Installing…":"Install DuckDB"]})})]})]})})]}),e.jsxs("section",{className:"page-section-divider",children:[e.jsxs("div",{className:"section-head section-head-inline",children:[e.jsxs("div",{children:[e.jsx("p",{className:"eyebrow eyebrow-soft",children:"Latest sync"}),e.jsxs("h2",{className:"flex items-center gap-2",children:["Release sync",e.jsx(h,{label:"What is a release sync?",children:"A release sync downloads one Common Crawl dump (~16 GB) and extracts backlinks for every project in this workspace in one pass. This is the heavy job — subsequent per-project re-runs skip the download and just re-query the cached files."})]})]}),t&&e.jsx(f,{tone:v(t.status),children:t.status})]}),e.jsxs(l,{className:"surface-card p-5",children:[e.jsxs("p",{className:"text-xs text-zinc-500 max-w-3xl mb-4",children:["A release is one Common Crawl dump (e.g. ",e.jsx("code",{className:"text-zinc-400",children:"cc-main-2026-jan-feb-mar"}),"). Syncing it downloads the graph and populates backlinks for every project in this workspace."]}),t?e.jsxs("div",{className:"space-y-2 text-sm",children:[e.jsxs("p",{className:"text-zinc-200",children:["Release ",e.jsx("code",{className:"text-zinc-300",children:t.release})]}),t.phaseDetail&&e.jsx("p",{className:"text-zinc-500",children:t.phaseDetail}),e.jsxs("div",{className:"grid grid-cols-2 md:grid-cols-4 gap-4 text-xs text-zinc-500 pt-2",children:[e.jsxs("div",{children:[e.jsx("p",{className:"text-zinc-600 uppercase tracking-wide",children:"Projects"}),e.jsx("p",{className:"text-zinc-300 mt-0.5",children:t.projectsProcessed??"—"})]}),e.jsxs("div",{children:[e.jsxs("p",{className:"text-zinc-600 uppercase tracking-wide flex items-center gap-1",children:["Rows",e.jsx(h,{label:"What are rows?",children:"Total number of (project, referring domain) pairs persisted in SQLite from this sync, across every project in the workspace."})]}),e.jsx("p",{className:"text-zinc-300 mt-0.5",children:t.domainsDiscovered??"—"})]}),e.jsxs("div",{children:[e.jsx("p",{className:"text-zinc-600 uppercase tracking-wide",children:"Started"}),e.jsx("p",{className:"text-zinc-300 mt-0.5",children:b(t.downloadStartedAt??t.createdAt)})]}),e.jsxs("div",{children:[e.jsx("p",{className:"text-zinc-600 uppercase tracking-wide",children:"Finished"}),e.jsx("p",{className:"text-zinc-300 mt-0.5",children:b(t.queryFinishedAt)})]})]}),t.error&&e.jsx("p",{className:"text-sm text-rose-400 pt-2",children:t.error})]}):e.jsx("p",{className:"text-sm text-zinc-500",children:"No release sync has run in this workspace yet."}),G&&e.jsx("div",{className:"mt-4 rounded border border-amber-800/60 bg-amber-950/20 p-3",children:e.jsxs("div",{className:"flex items-start gap-2",children:[e.jsx(L,{className:"h-4 w-4 text-amber-400 shrink-0 mt-0.5","aria-hidden":!0}),e.jsxs("div",{className:"text-xs text-zinc-300 leading-relaxed",children:[e.jsx("p",{className:"font-medium text-amber-200",children:"Cached files for this release are missing."}),e.jsxs("p",{className:"mt-1 text-zinc-400",children:["The sync record in the database says this release finished successfully, but the ~16 GB dump at"," ",e.jsxs("code",{className:"text-zinc-300",children:["~/.canonry/cache/commoncrawl/",t?.release,"/"]})," isn’t on disk. Your backlink data is still intact (it lives in SQLite), but per-project re-run extracts will fail until you either re-sync this release or start a new one."]})]})]})}),e.jsxs("div",{className:"mt-4 rounded border border-zinc-800 bg-zinc-900/40 p-3",children:[e.jsxs("div",{className:"flex items-start justify-between gap-3 mb-3",children:[e.jsxs("div",{children:[e.jsx("p",{className:"text-[10px] uppercase tracking-wide text-zinc-500",children:"Auto-detected release"}),u?e.jsxs("p",{className:"text-sm text-zinc-200 mt-0.5",children:[e.jsx("code",{className:"text-zinc-100",children:u.release}),e.jsxs("span",{className:"ml-2 text-xs text-zinc-500",children:["— vertex ",w(u.vertexBytes),", edges ",w(u.edgesBytes)]})]}):e.jsx("p",{className:"text-sm text-zinc-500 mt-0.5",children:D?"Probing Common Crawl…":"Could not auto-detect — pass an explicit release below."}),e.jsxs("a",{href:oe,target:"_blank",rel:"noopener noreferrer",className:"mt-1 inline-flex items-center gap-1 text-xs text-zinc-400 hover:text-zinc-200 focus:text-zinc-200 focus:outline-none focus-visible:ring-1 focus-visible:ring-zinc-500 rounded",children:["Browse all Common Crawl web-graph releases",e.jsx(de,{className:"h-3 w-3","aria-hidden":!0})]})]}),e.jsxs("div",{className:"flex items-center gap-2 shrink-0",children:[e.jsxs(z,{type:"button",size:"sm",disabled:p||!a?.duckdbInstalled||!u&&!N.trim(),onClick:A($),children:[e.jsx(K,{className:"h-4 w-4 mr-1.5","aria-hidden":!0}),p?"Queuing…":"Run sync"]}),e.jsxs(h,{label:"What does Run sync do?",children:[e.jsxs("span",{className:"block",children:["Downloads the auto-detected (or chosen) Common Crawl release (~16 GB) to"," ",e.jsx("code",{className:"text-zinc-300",children:"~/.canonry/cache/commoncrawl/"}),", then runs a single DuckDB query that extracts referring domains for every project in this workspace."]}),e.jsxs("span",{className:"mt-2 block text-zinc-400",children:["First time for a release: ",e.jsx("span",{className:"text-zinc-200",children:"~10–20 min download + ~5 min query"}),". Re-running the same release later: ",e.jsx("span",{className:"text-zinc-200",children:"skips download, just re-queries"})," (~5 min)."]})]})]})]}),F?e.jsxs("div",{className:"flex flex-wrap items-center gap-2",children:[e.jsx("input",{type:"text",className:"flex-1 min-w-[240px] rounded border border-zinc-700 bg-transparent px-2.5 py-1.5 text-sm text-zinc-200 placeholder-zinc-600 focus:border-zinc-500 focus:outline-none",placeholder:"cc-main-2026-jan-feb-mar",value:N,onChange:s=>g(s.target.value),disabled:p,autoFocus:!0}),e.jsx("button",{type:"button",className:"text-xs text-zinc-500 hover:text-zinc-300 focus:text-zinc-300 focus:outline-none focus-visible:ring-1 focus-visible:ring-zinc-500 rounded",onClick:()=>{g(""),k(!1)},disabled:p,children:"Cancel"})]}):e.jsx("button",{type:"button",className:"text-xs text-zinc-500 hover:text-zinc-300 focus:text-zinc-300 focus:outline-none focus-visible:ring-1 focus-visible:ring-zinc-500 rounded",onClick:()=>k(!0),disabled:p,children:"Use a different release →"})]}),!a?.duckdbInstalled&&e.jsx("p",{className:"text-xs text-zinc-600 mt-2",children:"Install DuckDB first to enable sync."})]})]}),e.jsxs("section",{className:"page-section-divider",children:[e.jsx("div",{className:"section-head section-head-inline",children:e.jsxs("div",{children:[e.jsx("p",{className:"eyebrow eyebrow-soft",children:"Cached releases"}),e.jsxs("h2",{className:"flex items-center gap-2",children:["Local disk cache",e.jsxs(h,{label:"What is this?",children:[e.jsxs("span",{className:"block",children:["Raw Common Crawl dumps stored at"," ",e.jsx("code",{className:"text-zinc-300",children:"~/.canonry/cache/commoncrawl/<release>/"}),". Each release takes ~16 GB."]}),e.jsxs("span",{className:"mt-2 block text-zinc-400",children:["These files are needed to re-run per-project extracts against a release without re-downloading. Pruning here ",e.jsx("span",{className:"text-zinc-200",children:"does not delete your backlink data"})," — that lives in SQLite."]})]})]})]})}),e.jsx("p",{className:"text-xs text-zinc-500 mb-3 max-w-3xl",children:"Each cached release is a ~16 GB pair of gzipped files. They’re needed to re-query the graph (e.g. for a newly-added project) without re-downloading. Safe to prune — backlink results persist in SQLite."}),e.jsx(l,{className:"surface-card overflow-hidden",children:e.jsxs("table",{className:"w-full text-sm",children:[e.jsx("thead",{children:e.jsxs("tr",{className:"border-b border-zinc-800 text-left text-xs uppercase tracking-wide text-zinc-600",children:[e.jsx("th",{className:"px-4 py-2 font-medium",children:"Release"}),e.jsx("th",{className:"px-4 py-2 font-medium",children:"Sync status"}),e.jsx("th",{className:"px-4 py-2 text-right font-medium",children:"Size"}),e.jsx("th",{className:"px-4 py-2 font-medium",children:"Last used"}),e.jsx("th",{className:"px-4 py-2 font-medium sr-only",children:"Actions"})]})}),e.jsxs("tbody",{children:[r.map(s=>e.jsxs("tr",{className:"border-b border-zinc-900 last:border-0",children:[e.jsx("td",{className:"px-4 py-2 text-zinc-200",children:e.jsx("code",{children:s.release})}),e.jsx("td",{className:"px-4 py-2",children:s.syncStatus?e.jsx(f,{tone:v(s.syncStatus),children:s.syncStatus}):e.jsx("span",{className:"text-zinc-600",children:"—"})}),e.jsx("td",{className:"px-4 py-2 text-right text-zinc-400 tabular-nums",children:w(s.bytes)}),e.jsx("td",{className:"px-4 py-2 text-zinc-400",children:b(s.lastUsedAt)}),e.jsx("td",{className:"px-4 py-2 text-right",children:e.jsxs("div",{className:"inline-flex items-center gap-1",children:[e.jsxs(z,{type:"button",variant:"outline",size:"sm",onClick:()=>{Q(s.release)},children:[e.jsx(ne,{className:"h-4 w-4 mr-1.5","aria-hidden":!0}),"Prune"]}),e.jsx(h,{label:"What does Prune do?",placement:"top",children:"Deletes the ~16 GB cache for this release from disk. Backlink results already in SQLite remain untouched. To re-run extracts against this release, you’d have to sync it again (another ~16 GB download)."})]})})]},s.release)),r.length===0&&e.jsx("tr",{children:e.jsx("td",{className:"px-4 py-4 text-sm text-zinc-500",colSpan:5,children:"No cached releases on this machine. If you ran a sync from a different machine (or deleted the cache), the backlink data is still in the database — but you’ll need to re-sync a release to run new extracts."})})]})]})})]}),o.length>1&&e.jsxs("section",{className:"page-section-divider",children:[e.jsx("div",{className:"section-head section-head-inline",children:e.jsxs("div",{children:[e.jsx("p",{className:"eyebrow eyebrow-soft",children:"History"}),e.jsx("h2",{children:"Past release syncs"})]})}),e.jsx(l,{className:"surface-card overflow-hidden",children:e.jsxs("table",{className:"w-full text-sm",children:[e.jsx("thead",{children:e.jsxs("tr",{className:"border-b border-zinc-800 text-left text-xs uppercase tracking-wide text-zinc-600",children:[e.jsx("th",{className:"px-4 py-2 font-medium",children:"Release"}),e.jsx("th",{className:"px-4 py-2 font-medium",children:"Status"}),e.jsx("th",{className:"px-4 py-2 text-right font-medium",children:"Projects"}),e.jsx("th",{className:"px-4 py-2 text-right font-medium",children:"Rows"}),e.jsx("th",{className:"px-4 py-2 font-medium",children:"Finished"})]})}),e.jsx("tbody",{children:o.map(s=>e.jsxs("tr",{className:"border-b border-zinc-900 last:border-0",children:[e.jsx("td",{className:"px-4 py-2 text-zinc-200",children:e.jsx("code",{children:s.release})}),e.jsx("td",{className:"px-4 py-2",children:e.jsx(f,{tone:v(s.status),children:s.status})}),e.jsx("td",{className:"px-4 py-2 text-right text-zinc-400 tabular-nums",children:s.projectsProcessed??"—"}),e.jsx("td",{className:"px-4 py-2 text-right text-zinc-400 tabular-nums",children:s.domainsDiscovered??"—"}),e.jsx("td",{className:"px-4 py-2 text-zinc-400",children:b(s.queryFinishedAt??s.updatedAt)})]},s.id))})]})})]})]})}export{fe as BacklinksPage};

package/assets/assets/ChartPrimitives-CjX4mQCy.js

@@ -0,0 +1 @@
+import{c as n}from"./index-avJiga1P.js";const r=[["path",{d:"m15 18-6-6 6-6",key:"1wnfg3"}]],s=n("chevron-left",r),f={contentStyle:{backgroundColor:"#18181b",border:"1px solid #3f3f46",borderRadius:8,fontSize:12},labelStyle:{color:"#e4e4e7"},itemStyle:{color:"#a1a1aa"}},d={fill:"#71717a",fontSize:11},l="#27272a",S="#27272a",a=["#34d399","#60a5fa","#f472b6","#facc15","#a78bfa","#fb923c","#22d3ee","#f87171"],c={gemini:"#60a5fa",openai:"#34d399",claude:"#fb923c",perplexity:"#22d3ee",local:"#a78bfa"};function R(t,e=0){return c[t]??a[e%a.length]}const T={text:"#a1a1aa",textDim:"#71717a",textFaint:"#52525b",surface:"#27272a"},u={positive:"#34d399",positiveDeep:"#10b981"};function o(t){const e=String(t);return e.includes("T")?new Date(e):new Date(e+"T00:00:00")}function C(t){return o(String(t)).toLocaleDateString(void 0,{month:"short",day:"numeric",year:"numeric"})}function _(t){const e=o(t);return`${e.getMonth()+1}/${e.getDate()}`}export{S as C,d as a,C as b,f as c,T as d,a as e,_ as f,u as g,s as h,l as i,R as p};