mcp-scraper 0.38.2 → 0.40.1

package/docs/hosted-location-data.md

@@ -1,108 +0,0 @@
-# Hosted location data
-
-Production directory workflows and `location_markets` read versioned location snapshots from Turso. They do not read a server-local CSV and do not call Census during a customer request.
-
-Two datasets are active independently:
-
-- `us_census_places:<STATE>` stores every Census place and its 2020–2025 population estimates for one state.
-- `us_zip_groups` stores normalized city ZIP and county groups for all imported states.
-
-Each import receives an immutable dataset ID. An active pointer is switched only after every normalized row is durable, so readers see either the old complete version or the new complete version. Responses include both active dataset IDs, source URLs, and refresh timestamps.
-
-## Required environment
-
-```dotenv
-TURSO_DATABASE_URL=...
-TURSO_AUTH_TOKEN=...
-MCP_SCRAPER_LOCATION_ADMIN_KEY=...
-```
-
-`MCP_SCRAPER_USZIPS_CSV_URL` may hold the default public HTTPS ZIP source. The import endpoint also accepts a source URL explicitly. `MCP_SCRAPER_USZIPS_CSV_PATH` remains available only for local tests and local CLI runs.
-
-ZIP imports are fail-closed: a request URL must exactly match `MCP_SCRAPER_USZIPS_CSV_URL` or one entry in `MCP_SCRAPER_LOCATION_IMPORT_SOURCE_ALLOWLIST`. Redirects are rejected, private-network targets are rejected, and response bytes are streamed through a 64 MiB in-flight limit. Census sync does not accept a URL; it constructs the trusted `www2.census.gov` state URL server-side.
-
-## Publish the nationwide GeoNames source
-
-The production ZIP/county snapshot is derived from the official GeoNames US postal-code archive at `https://download.geonames.org/export/zip/US.zip`. GeoNames postal-code data is licensed under [Creative Commons Attribution 4.0](https://creativecommons.org/licenses/by/4.0/); retain attribution to [GeoNames](https://www.geonames.org/) anywhere this derived dataset is redistributed.
-
-Publish a normalized, immutable public CSV to Vercel Blob:
-
-```bash
-export LOCATION_DATA_READ_WRITE_TOKEN='...'
-npm run locations:publish-geonames
-```
-
-`BLOB_READ_WRITE_TOKEN` is also accepted for projects that use their default
-public Blob store. Production uses a dedicated public store connected with the
-`LOCATION_DATA_` environment-variable prefix so this dataset cannot collide
-with private connected-data artifacts.
-
-The publisher downloads the direct HTTPS ZIP without following redirects, applies status, timeout, compressed-size, and extracted-size limits, extracts only `US.txt`, and converts the documented tab-separated fields to `state_abbr,zipcode,county,city`. It then runs the same production CSV parser and nationwide gate used by the import route: all 50 states plus DC and at least 25,000 distinct five-digit ZIPs must be present.
-
-The Blob pathname includes the normalized CSV's SHA-256 digest. Random suffixes and overwrites are disabled, so a published URL is an immutable snapshot. The command prints only non-secret counts, hashes, paths, and public URLs; copy its `url` value into `MCP_SCRAPER_USZIPS_CSV_URL`. A repeat publication of byte-identical data intentionally fails instead of replacing the existing content-addressed object.
-
-## Seed and refresh
-
-Bootstrap the nationwide ZIP snapshot plus all 50 states and DC in one command:
-
-```bash
-export MCP_SCRAPER_LOCATION_ADMIN_KEY='...'
-export MCP_SCRAPER_USZIPS_CSV_URL='https://...blob.vercel-storage.com/location-data/geonames/us/geonames-us-<sha256>.csv'
-npm run locations:sync
-```
-
-The bootstrap uses three concurrent state requests by default, retries retryable HTTP/network failures up to three times, prints a per-state result and final failed-state list, and exits nonzero if any dataset failed. It never prints the admin key or ZIP source URL. Useful bounded refreshes include:
-
-```bash
-# Refresh only selected Census snapshots and leave the active ZIP version unchanged.
-npm run locations:sync -- --skip-zips --states=TN,CO,CA
-
-# Point the operator at a preview API and lower concurrency.
-npm run locations:sync -- --base-url=https://preview.example.com --concurrency=2
-```
-
-The individual admin calls below remain available for targeted recovery.
-
-Sync one state's trusted Census snapshot:
-
-```bash
-curl -sS https://mcpscraper.dev/locations/admin/sync-census \
-  -H "x-location-admin-key: $MCP_SCRAPER_LOCATION_ADMIN_KEY" \
-  -H 'content-type: application/json' \
-  --data '{"state":"TN"}'
-```
-
-Import the nationwide ZIP/county CSV from a public HTTPS object:
-
-```bash
-curl -sS https://mcpscraper.dev/locations/admin/import \
-  -H "x-location-admin-key: $MCP_SCRAPER_LOCATION_ADMIN_KEY" \
-  -H 'content-type: application/json' \
-  --data '{"sourceUrl":"https://data.example.com/uszips.csv"}'
-```
-
-The ZIP import accepts the common header aliases `state_abbr|state|state_id`, `zipcode|zip|zip_code`, `city|primary_city`, and `county|county_name`. A replacement must cover all 50 states plus DC and contain at least 25,000 distinct five-digit ZIPs. Unsupported state codes and malformed rows are counted as rejected rows in the dataset record. A partial or undersized snapshot is rejected without changing the active pointer. Re-importing byte-identical active data is idempotent.
-
-Census imports verify that every place row's `STATE` FIPS matches the requested state before activation. `/locations/status` reports missing population states, required state count, ZIP coverage, rejected rows, and readiness; `available` becomes true only when all 51 population snapshots and a nationwide ZIP snapshot are active.
-
-## Read and verify
-
-Dataset readiness requires an API key:
-
-```bash
-curl -sS https://mcpscraper.dev/locations/status \
-  -H "x-api-key: $MCP_SCRAPER_API_KEY"
-```
-
-Query hosted markets directly:
-
-```bash
-curl -sS 'https://mcpscraper.dev/locations/markets?state=TN&populationYear=2025&minPopulation=100000&maxResults=25&includeZipGroups=true' \
-  -H "x-api-key: $MCP_SCRAPER_API_KEY"
-```
-
-The same read is exposed to agents as `location_markets`. Filters are applied against the hosted population and ZIP rows before the population-sorted result limit. If a required state snapshot or ZIP snapshot has not been activated, the API returns a typed `503` instead of silently falling back to a local path or a live Census fetch.
-
-## Operational follow-up
-
-The active-pointer design prevents partial data from becoming visible and preserves old dataset rows for rollback. A future maintenance pass should add an explicit cross-process import lease, an admin rollback/history endpoint, and retention cleanup for superseded snapshot rows. These do not weaken current read isolation or coverage gates, but they will make concurrent operator refreshes and long-term storage management more ergonomic.

package/docs/kernel-proxy-future-enhancements.md

@@ -1,80 +0,0 @@
-# Kernel / proxy retry — future enhancement notes
-
-**Status:** ideas only — NOT implemented, NO code from here is in the tree.
-**Origin:** salvaged from an abandoned automated branch `codex/kernel-proxy-retry-cleanup`
-(a stale `~/Projects/mcp-scraper` clone, since deleted). That branch was evaluated against the
-current Desktop code and found **superseded** — Desktop independently built the same retry +
-error-classification machinery and went further (fresh disposable proxy per attempt, state
-escalation, rollout to maps/Facebook/site-crawl). **Do not `git merge` that branch** — it
-diverged onto an older base and a merge would regress the current proxy layer.
-
-The two items below are the *only* things Desktop genuinely lacked. They are documented here as
-"if we want to build X, here's how" — to be rebuilt from scratch onto current code, never pasted.
-
----
-
-## A. SERP-timeout salvage (best candidate)
-
-**Problem it solves:** when a Google page load hits a navigation timeout (e.g.
-`domcontentloaded` slow) but the page is *actually* a usable `/search` results page with no
-CAPTCHA, the current code throws and triggers a wasted retry. We can instead detect "good page,
-just slow" and continue.
-
-**Where:** `src/driver/BrowserDriver.ts`, in the `goto` flow.
-
-**How to build:**
-1. Add a predicate `currentGoogleSerpIsUsable(page)` — returns true when the current URL is a
-   Google `/search` page, results containers are present, and no CAPTCHA/consent interstitial is
-   detected.
-2. In `goto`, catch the navigation-timeout case (an `isNavigationTimeout(err)` guard). Before
-   rethrowing, if `currentGoogleSerpIsUsable(page)` is true, log a "salvaged slow SERP" telemetry
-   line and proceed instead of throwing.
-3. Keep the throw for every other timeout (non-SERP pages, CAPTCHA present, empty page).
-
-**Tools it improves:** `search_serp`, `harvest_paa`, `capture_serp_*` — fewer discarded captures
-and fewer retry credits burned on pages that were fine.
-
-**Watch out for:** only salvage when results are genuinely present — a timed-out blank/consent
-page must still fail so the normal retry/rotation kicks in.
-
----
-
-## B. Opt-in proxy connectivity-check bypass (situational)
-
-**Problem it solves:** the browser-service residential proxies run an internal connectivity ping
-(an S3-style reachability check) before use. When that ping falsely fails, proxy creation is
-rejected even though the proxy would work — surfacing as `proxy_unavailable`.
-
-**Where:** `src/kernel-proxy-resolver.ts`, in the residential-proxy creation path.
-
-**How to build:**
-1. Gate behind an env flag `KERNEL_PROXY_BYPASS_CONNECTIVITY_CHECK` (default off).
-2. When set, pass a `bypass_hosts` / skip-connectivity-check option into the proxy-create call so
-   the internal ping is skipped.
-3. Add a `shouldRetryWithoutBypass` fallback: if a bypassed proxy then fails for real, retry once
-   with the check re-enabled so we don't mask a genuinely dead proxy.
-
-**Build ONLY if** we actually observe proxy-create failures from the connectivity ping in
-production. Otherwise leave it — it's complexity with no payoff until that failure mode is real.
-
----
-
-## C. Minor niceties (low priority, build only if convenient)
-
-- **Per-attempt hard timeout:** a fresh `AbortController` per retry attempt with its own
-  `perAttemptTimeoutMs` (~120s), chained to the parent signal. Desktop bounds attempt *count* but
-  has no per-attempt hard cap.
-- **Typed `KernelProxyUnavailableError`:** Desktop detects proxy-unavailable via regex on the
-  message; a dedicated error class would be cleaner to branch on.
-- **Request-level retry knobs:** expose `maxAttempts`, `retryOnCaptcha`, `retryOnProxyFailure`,
-  `allowLocationFallback` as request schema fields instead of hardcoded constants, for per-call
-  tuning / debugging.
-
----
-
-## What NOT to take
-
-Everything else from the codex branch — the retry loop, the `proxy_tunnel_failed` /
-`proxy_unavailable` / `location_mismatch` error classification, `LocationMismatchError`, the
-503/retryable mappings — is **already in Desktop** (`src/harvest.ts`, `src/errors.ts`,
-`src/api/harvest-problems.ts`) and in a more advanced form. Do not reintroduce it.

package/docs/mcp-tool-craft-lint.generated.md

@@ -1,183 +0,0 @@
-# MCP Tool Craft Lint
-
-Generated: 2026-07-28T19:01:51.307Z
-Manifest: `/private/tmp/mcp-scraper-archive-release/docs/mcp-tool-manifest.generated.json`
-
-Tools: 172
-Checks per tool: 7
-Failing tools: 0
-
-| Tool | Score | Missing |
-|---|---:|---|
-| `access-accept-share` | 7/7 | none |
-| `access-approve-sender` | 7/7 | none |
-| `access-decline-share` | 7/7 | none |
-| `access-inbox-settings` | 7/7 | none |
-| `access-invite-account` | 7/7 | none |
-| `access-issue-key` | 7/7 | none |
-| `access-list-approved-senders` | 7/7 | none |
-| `access-list-keys` | 7/7 | none |
-| `access-note-inbox` | 7/7 | none |
-| `access-remove-approved-sender` | 7/7 | none |
-| `access-revoke-key` | 7/7 | none |
-| `access-revoke-share` | 7/7 | none |
-| `access-set-scope` | 7/7 | none |
-| `access-share-note` | 7/7 | none |
-| `access-share-vault` | 7/7 | none |
-| `access-swap-vault` | 7/7 | none |
-| `access-switch-account` | 7/7 | none |
-| `access-unlink-share` | 7/7 | none |
-| `add-vault` | 7/7 | none |
-| `archive_read` | 7/7 | none |
-| `audit_site` | 7/7 | none |
-| `browser_click` | 7/7 | none |
-| `browser_close` | 7/7 | none |
-| `browser_extension_delete` | 7/7 | none |
-| `browser_extension_import` | 7/7 | none |
-| `browser_extension_list` | 7/7 | none |
-| `browser_goto` | 7/7 | none |
-| `browser_list_replays` | 7/7 | none |
-| `browser_list_sessions` | 7/7 | none |
-| `browser_locate` | 7/7 | none |
-| `browser_open` | 7/7 | none |
-| `browser_press` | 7/7 | none |
-| `browser_profile_connect` | 7/7 | none |
-| `browser_profile_list` | 7/7 | none |
-| `browser_read` | 7/7 | none |
-| `browser_replay_annotate` | 7/7 | none |
-| `browser_replay_download` | 7/7 | none |
-| `browser_replay_mark` | 7/7 | none |
-| `browser_replay_start` | 7/7 | none |
-| `browser_replay_stop` | 7/7 | none |
-| `browser_screenshot` | 7/7 | none |
-| `browser_scroll` | 7/7 | none |
-| `browser_type` | 7/7 | none |
-| `bulk-delete-notes` | 7/7 | none |
-| `call_service_connection_action` | 7/7 | none |
-| `capture_serp_page_snapshots` | 7/7 | none |
-| `capture_serp_snapshot` | 7/7 | none |
-| `check_site_export` | 7/7 | none |
-| `cost-usage` | 7/7 | none |
-| `create-channel` | 7/7 | none |
-| `create-scheduled-action` | 7/7 | none |
-| `create-secure-vault` | 7/7 | none |
-| `create-webhook` | 7/7 | none |
-| `credits_info` | 7/7 | none |
-| `delete-note` | 7/7 | none |
-| `delete-scheduled-action` | 7/7 | none |
-| `delete-vault` | 7/7 | none |
-| `describe_service_connection_tool` | 7/7 | none |
-| `diff_page` | 7/7 | none |
-| `directory_workflow` | 7/7 | none |
-| `directory_workflow_status` | 7/7 | none |
-| `export_connected_service_data` | 7/7 | none |
-| `export_search_console_table_data` | 7/7 | none |
-| `extract_site` | 7/7 | none |
-| `extract_url` | 7/7 | none |
-| `facebook_ad_search` | 7/7 | none |
-| `facebook_ad_transcribe` | 7/7 | none |
-| `facebook_page_intel` | 7/7 | none |
-| `facebook_video_transcribe` | 7/7 | none |
-| `fact-history` | 7/7 | none |
-| `g2_reviews` | 7/7 | none |
-| `get-chat-link` | 7/7 | none |
-| `get-message-note` | 7/7 | none |
-| `get-schedule-link` | 7/7 | none |
-| `get-schedule-status` | 7/7 | none |
-| `get-vault-app-link` | 7/7 | none |
-| `get-vault-contract` | 7/7 | none |
-| `gmail_search_contacts` | 7/7 | none |
-| `gmail_send_message` | 7/7 | none |
-| `google_ads_page_intel` | 7/7 | none |
-| `google_ads_search` | 7/7 | none |
-| `google_ads_transcribe` | 7/7 | none |
-| `google_calendar_create_event` | 7/7 | none |
-| `harvest_paa` | 7/7 | none |
-| `import_service_connection_to_memory` | 7/7 | none |
-| `instagram_media_download` | 7/7 | none |
-| `instagram_profile_content` | 7/7 | none |
-| `library-ingest` | 7/7 | none |
-| `list_service_connections` | 7/7 | none |
-| `list-channel-members` | 7/7 | none |
-| `list-channel-messages` | 7/7 | none |
-| `list-memory-tags` | 7/7 | none |
-| `list-scheduled-actions` | 7/7 | none |
-| `list-shared-with-me` | 7/7 | none |
-| `list-vaults` | 7/7 | none |
-| `list-webhooks` | 7/7 | none |
-| `location_markets` | 7/7 | none |
-| `map_site_urls` | 7/7 | none |
-| `map_wayback_snapshots` | 7/7 | none |
-| `maps_place_intel` | 7/7 | none |
-| `maps_search` | 7/7 | none |
-| `memory-backlinks` | 7/7 | none |
-| `memory-capture` | 7/7 | none |
-| `memory-export` | 7/7 | none |
-| `memory-get` | 7/7 | none |
-| `memory-graph-path` | 7/7 | none |
-| `memory-graph-universe` | 7/7 | none |
-| `memory-list` | 7/7 | none |
-| `memory-put` | 7/7 | none |
-| `memory-questions` | 7/7 | none |
-| `memory-search` | 7/7 | none |
-| `memory-suggest` | 7/7 | none |
-| `memory-upload` | 7/7 | none |
-| `merge-memory-tags` | 7/7 | none |
-| `meta_ad_creative_media` | 7/7 | none |
-| `my-mentions` | 7/7 | none |
-| `pause-scheduled-action` | 7/7 | none |
-| `poll-channel` | 7/7 | none |
-| `post-message` | 7/7 | none |
-| `prepare-memory-write` | 7/7 | none |
-| `propose-scheduled-action` | 7/7 | none |
-| `provision-defaults` | 7/7 | none |
-| `query_fanout_workflow` | 7/7 | none |
-| `rank_tracker_workflow` | 7/7 | none |
-| `react-message` | 7/7 | none |
-| `read_service_connection` | 7/7 | none |
-| `record-fact` | 7/7 | none |
-| `reddit_thread` | 7/7 | none |
-| `reddit_trending` | 7/7 | none |
-| `remove-channel-member` | 7/7 | none |
-| `renew_connected_data_download` | 7/7 | none |
-| `reply-message` | 7/7 | none |
-| `report_artifact_read` | 7/7 | none |
-| `resolve-memory-tags` | 7/7 | none |
-| `resume-scheduled-action` | 7/7 | none |
-| `revoke-chat-link` | 7/7 | none |
-| `revoke-schedule-link` | 7/7 | none |
-| `revoke-vault-app-link` | 7/7 | none |
-| `revoke-webhook` | 7/7 | none |
-| `route-memory` | 7/7 | none |
-| `search_serp` | 7/7 | none |
-| `set_scheduled_action_connections` | 7/7 | none |
-| `set-agent-identity` | 7/7 | none |
-| `set-schedule-defaults` | 7/7 | none |
-| `set-schedule-entitlement` | 7/7 | none |
-| `slack_send_message` | 7/7 | none |
-| `storage-usage` | 7/7 | none |
-| `table-create` | 7/7 | none |
-| `table-delete-rows` | 7/7 | none |
-| `table-describe` | 7/7 | none |
-| `table-drop` | 7/7 | none |
-| `table-insert-rows` | 7/7 | none |
-| `table-list` | 7/7 | none |
-| `table-query` | 7/7 | none |
-| `temporal-recall` | 7/7 | none |
-| `test_service_connection` | 7/7 | none |
-| `trustpilot_reviews` | 7/7 | none |
-| `upsert-memory-tag` | 7/7 | none |
-| `validate-memory-write` | 7/7 | none |
-| `video_frame_analysis` | 7/7 | none |
-| `video_frame_analysis_status` | 7/7 | none |
-| `video-analyze-start` | 7/7 | none |
-| `video-analyze-status` | 7/7 | none |
-| `workflow_artifact_read` | 7/7 | none |
-| `workflow_list` | 7/7 | none |
-| `workflow_run` | 7/7 | none |
-| `workflow_status` | 7/7 | none |
-| `workflow_step` | 7/7 | none |
-| `workflow_suggest` | 7/7 | none |
-| `youtube_harvest` | 7/7 | none |
-| `youtube_transcribe` | 7/7 | none |
-| `zoom_create_meeting` | 7/7 | none |

package/docs/mcp-tool-design-guide.md

@@ -1,225 +0,0 @@
-# Designing MCP Tools an AI Can Actually Find and Use
-
-A practical guide to building MCP servers whose tools an LLM (Claude, but the mechanics generalize)
-will **discover, select correctly, and invoke well**. It is written from how the model *actually*
-perceives tools at runtime — not how a human reads your docs page.
-
-The single thesis: **the model routes on what it can see, and it can see almost nothing until it asks.**
-Everything below follows from that.
-
----
-
-## 1. How the model actually sees your tools (the mechanic that governs everything)
-
-Modern MCP clients **defer** tool schemas to save context. The lifecycle in a session:
-
-1. **At rest — names only.** When the server connects, the model gets a flat list of tool
-   **identifiers** (`mcp__<server>__<tool>`) — *names, nothing else*. No parameters, no descriptions,
-   no input/output shapes. For ~50 tools that's a cheap, scannable block.
-2. **Server-instructions block — the one thing loaded upfront.** An MCP server may ship a top-level
-   `instructions` string. This **is** in context from the start, alongside the names. It's the only
-   place you can influence routing *before* the model fetches anything.
-3. **Discovery — the model fetches a schema.** To use a tool, the model runs a tool-search step
-   (exact `select:<name>` or a keyword query) that loads that tool's **full definition** — description
-   prose + parameter JSONSchema — into context. *This is the first moment the model sees how the tool works.*
-4. **Call → result.** The model invokes with arguments; your formatted result returns into context.
-
-Two consequences that drive the whole guide:
-
-- **Names do the *pre-fetch* routing.** The decision "which tool do I even look at?" is made from names
-  (+ the server-instructions block). A name that doesn't signal intent never gets fetched.
-- **Descriptions do the *post-fetch* disambiguation.** They guide correct parameter choice and break
-  ties between near-neighbors — but only *after* the model already chose to fetch. They can't rescue a
-  tool the model never looked at.
-
-> Mental model: the **name** is the search keyword; the **description** is the manual you read after you
-> already pulled the book off the shelf; the **server-instructions block** is the sign on the shelf.
-
----
-
-## 2. Principle 1 — The name is the routing surface
-
-Because names carry pre-fetch routing, treat them as the most important UX you ship.
-
-- **Front-load the discriminator.** Put the distinguishing token first: `youtube_transcribe`,
-  `facebook_video_transcribe`. The platform/object is what the model scans for.
-- **`platform_object_action` (or `object_action`).** A consistent scheme lets the model *predict*
-  names it hasn't seen and disambiguate neighbors.
-- **Never ship a bare generic verb.** `search` is a trap — search *what*? `search_serp`,
-  `maps_search`, `facebook_ad_search`. The qualifier prevents cross-fetch and sharpens keyword ranking.
-- **Disambiguate near-neighbors on their real axis.** `extract_url` vs `extract_site` vs
-  `map_site_urls` work because the names encode *single page / whole site / URLs-only*. If those blur,
-  the model picks the wrong granularity.
-- **Match the user's vocabulary, or bridge it.** Users say "scrape a page"; if the tool is
-  `extract_url`, that gap is only bridged *after* fetch (via description/`symptomTriggers`). Where a
-  phrasing is common and high-value, make the name or its triggers reflect it.
-- **Mind the namespace stutter.** The client prepends `mcp__`. Naming the server `mcp-scraper` yields
-  `mcp__mcp-scraper__…`. Name it `scraper` → `mcp__scraper__…`. Cosmetic, but it shows in permission
-  prompts/logs, and renaming later is breaking — do it early.
-
-### The decisive test: named tool vs. generic engine
-A user says *"collect ICP info from Reddit."*
-- A **`reddit_research`** tool: the word "reddit" hits the name directly → fetched immediately, one hop.
-- The generic **`browser_*`** agent: matches *none* of the user's words. To land there the model must
-  *infer* "no reddit tool → reddit is a website → drive a browser → navigate → search → read." Several
-  hops, each a chance to mis-route (e.g. grab `search_serp` and Google it instead).
-
-**Lesson:** if there's a phrase users will say that names a platform or intent, you want a **named tool
-catching it** — even if the implementation is a thin wrapper over a generic engine. *The name triggers;
-the engine executes.* A category that merely "points at the browser" serves a human clicking a button;
-it does **not** serve the model that has to *find* the capability from words.
-
----
-
-## 3. Principle 2 — Split by capability, parameterize the variations
-
-It's tempting to think "names are cheap, so ship hundreds of hyper-specific tools." Don't. Three reasons:
-
-1. **The name list isn't free at scale.** All names load at rest. 50 is fine; 300 is a wall the model
-   must scan and a real token cost. There's a ceiling.
-2. **Search ranking *degrades* with volume.** More near-neighbors → more ambiguous keyword ranking →
-   *higher* wrong-fetch rate. Over-splitting makes the exact failure you're trying to avoid worse.
-3. **It's the combinatorial trap.** `extract_url_screenshot`, `extract_url_branding`,
-   `extract_url_mobile`… is N×M×K variants of one capability. Parameters exist to prevent this.
-
-**The rule:** one tool per *distinct thing a user wants to do*; **parameters** for *variations* of that thing.
-
-The split test:
-> Would a user think of these as **different things to do**, or the **same thing with options**?
-
-- Different platform/object → different tool (`youtube_transcribe` vs `facebook_video_transcribe`).
-- Different granularity / mental model → different tool (`extract_url` vs `extract_site`).
-- A knob on the same action → **parameter** (`screenshot`, `formats`, `maxPages`, `rotateProxies`).
-
-Post-fetch param-learning is cheap and reliable for the model, so you lose almost nothing by using
-params for variations — and you keep the name list scannable and the ranking sharp.
-
----
-
-## 4. Principle 3 — Descriptions are the post-fetch safety net
-
-Loaded on fetch, the description's job is **correct usage + tie-breaking**, not discovery. Make it earn that:
-
-- **Lead with when-to-use and when-NOT-to-use.** Negative space ("use `extract_url` for one page; use
-  `extract_site` for a whole site") is what separates near-neighbors once both are fetched.
-- **State side effects and cost plainly.** Especially for tools that spend money, open sessions, or
-  post — the model should be able to refuse a wrong pick *before* executing.
-- **Encode parameter intent, bounds, and defaults** that mirror your actual schema, so the model fills
-  the form correctly the first time.
-- **Carry symptom triggers / sample prompts.** Phrases that should route here. These help both the
-  keyword search ranking and the post-fetch confirmation.
-
-Where the net fails (design against this): two tools with **similar names AND overlapping
-descriptions** — the wrong one reads "plausible enough" and the model proceeds. Distinct names +
-explicit negative space prevent it.
-
----
-
-## 5. Principle 4 — Use the server-instructions block as the routing map
-
-It's the only content loaded *before* fetch besides names, so it's your lever on pre-fetch routing.
-Keep it short and put **cross-tool routing**, not per-tool detail, there:
-
-- A decision map: "single page → `extract_url`; whole site → `extract_site`; just URLs → `map_site_urls`."
-- Setup/auth prerequisites that span tools.
-- How to batch related fetches (clients often let the model load several schemas in one search call —
-  tell it which tools group together).
-
-This is where you catch granularity and family mistakes *before* the model ever fetches the wrong one.
-
----
-
-## 6. Principle 5 — Two surfaces: human categories vs. AI tools
-
-Your frontend and your MCP tool list are **different products with different optimal shapes.**
-
-- **Humans** benefit from *consolidation*: one "Search" card with a source dropdown
-  (Web / Maps / Ads / YouTube). Fewer, richer entries; group by platform or verb as fits the UI.
-- **The model** benefits from *distinction*: separate, well-named tools (`search_serp`, `maps_search`,
-  `facebook_ad_search`). A single `search(source=…)` tool forces the model to fetch it just to learn the
-  source enum, and the name `search` is ambiguous at discovery.
-
-**Do not let the human IA pressure you into merging MCP tool names.** Map *one frontend card → many
-distinct MCP tools*. Two surfaces, one backend.
-
-Corollary — **a category can exist without a tool, but the AI path can't.** A "Reddit" card can be a
-guided browser flow with no `reddit_*` tool; a human clicks it fine. But the model gets a clean path
-*only* if a named tool exists (see §2). Decide per capability: human-only/occasional → category over the
-generic engine; want strong AI routing → ship the named wrapper.
-
-Platform-first vs verb-first grouping: for a multi-platform tool, **platform-first**
-(YouTube / Facebook / Google Maps / Websites / Browser / Workflows) often wins — it matches how users
-think *and* mirrors `platform_verb` naming. The category is the umbrella (covers search *and*
-transcribe *and* …); the tools inside are the named verbs.
-
----
-
-## 7. Principle 6 — Tool *results* are context too; keep them lean
-
-Discovery isn't the only context cost — **what you return floods the window if you let it.**
-
-- **Summarize inline, persist the bulk, hand back a reference.** For large outputs, return a concise
-  summary + a saved file / artifact / URL the model can read on demand, not the whole payload.
-- **Truncate with a signpost.** "showing first N of M; full data at `<path>`" beats silently dumping or
-  silently dropping.
-- **Put the full-fidelity data somewhere the model can fetch** (disk file, object storage URL), so the
-  model gets the gist now and the detail only if needed.
-- **Structured + human-readable.** Return both a structured object (for programmatic chaining) and a
-  short readable block (for the model to reason over) — don't make it parse a wall of text.
-
-A tool that returns 200 KB of HTML per call will blow out a multi-step task; one that returns a 2 KB
-summary + a file path scales.
-
----
-
-## 8. Failure modes → the fix
-
-| Failure | Cause | Fix |
-|---|---|---|
-| **Tool never found** | Name doesn't match user vocabulary | Front-loaded, intent-matching name; `symptomTriggers` |
-| **Wrong tool fetched & believed** | Similar names + overlapping descriptions | Distinct names; explicit negative space; bounds in description |
-| **Routed to generic engine the hard way** | Capability has no named tool | Thin named wrapper over the engine (§2 Reddit) |
-| **Wrong granularity** | `*_url` / `*_site` / `*_map` blur | Encode the axis in the name; routing map in server instructions |
-| **Picked god-tool, wrong sub-mode** | One tool, mega-param surface | Split distinct capabilities into separate tools |
-| **Context blown out** | Tool returns full payload | Summarize + persist + reference (§7) |
-| **Expensive wrong call** | Side-effecting tool, ambiguous name/desc | Clear name; state cost/side-effects up top; confirm before execute |
-| **Namespace stutter / churn** | Server renamed late | Pick the clean server name on day one |
-
----
-
-## 9. Naming checklist (per tool)
-
-- [ ] Discriminator (platform/object) is **first** in the name.
-- [ ] Follows a consistent `platform_object_action` scheme used across the server.
-- [ ] No bare generic verb; qualifier present.
-- [ ] Distinct from every neighbor on a real axis (object / granularity / platform).
-- [ ] Matches at least one phrase a real user would say (or `symptomTriggers` covers it).
-- [ ] It's a *distinct capability*, not a variation that should be a parameter.
-- [ ] Side effects / cost surfaced in the first lines of the description.
-- [ ] Description leads with when-to-use **and** when-not-to-use (negative space).
-- [ ] Params mirror the real schema (names, bounds, enums, defaults).
-- [ ] Result is lean: summary + reference, not a raw dump.
-
-## 10. Server-level checklist
-
-- [ ] Clean server name (no `mcp__mcp-…` stutter), chosen before launch.
-- [ ] `instructions` block carries a short **cross-tool routing map**, not per-tool detail.
-- [ ] Tool count is at the *capability* grain — not exploded into variants.
-- [ ] Frontend categories map to *multiple distinct* MCP tools (two surfaces).
-- [ ] High-value, user-named capabilities have a **named tool**, even if engine-backed.
-- [ ] Related tools are documented as a batch the model can fetch together.
-
----
-
-## 11. The one-paragraph version
-
-The model sees only tool **names** until it fetches a schema, so **names carry routing** — front-load
-the discriminator, never ship a bare verb, and give every user-named capability its own named tool
-(even a thin wrapper over a generic engine), because a name that matches the user's words is a one-hop
-hit while a generic engine is a multi-hop inference. Split tools by **distinct capability** and use
-**parameters** for variations — more tools past a point *degrades* selection, it doesn't help.
-**Descriptions** are the post-fetch safety net (when-to-use + when-not, cost, bounds); the
-**server-instructions block** is your only pre-fetch routing lever (put a cross-tool decision map
-there). Keep the **human frontend** (consolidated cards) and the **AI tool list** (distinct names)
-as separate surfaces over one backend. And remember tool **results are context** — return a summary
-plus a reference, never a raw dump.