mcp-scraper 0.39.0 → 0.40.2

package/docs/mcp-tool-craft-lint.generated.md

@@ -1,186 +0,0 @@
-# MCP Tool Craft Lint
-
-Generated: 2026-07-29T05:22:23.627Z
-Manifest: `/Users/vilovieta/Desktop/mcp-scraper-editorial-tool/docs/mcp-tool-manifest.generated.json`
-
-Tools: 175
-Checks per tool: 7
-Failing tools: 0
-
-| Tool | Score | Missing |
-|---|---:|---|
-| `access-accept-share` | 7/7 | none |
-| `access-approve-sender` | 7/7 | none |
-| `access-decline-share` | 7/7 | none |
-| `access-inbox-settings` | 7/7 | none |
-| `access-invite-account` | 7/7 | none |
-| `access-issue-key` | 7/7 | none |
-| `access-list-approved-senders` | 7/7 | none |
-| `access-list-keys` | 7/7 | none |
-| `access-note-inbox` | 7/7 | none |
-| `access-remove-approved-sender` | 7/7 | none |
-| `access-revoke-key` | 7/7 | none |
-| `access-revoke-share` | 7/7 | none |
-| `access-set-scope` | 7/7 | none |
-| `access-share-note` | 7/7 | none |
-| `access-share-vault` | 7/7 | none |
-| `access-swap-vault` | 7/7 | none |
-| `access-switch-account` | 7/7 | none |
-| `access-unlink-share` | 7/7 | none |
-| `add-vault` | 7/7 | none |
-| `archive_read` | 7/7 | none |
-| `audit_site` | 7/7 | none |
-| `browser_click` | 7/7 | none |
-| `browser_close` | 7/7 | none |
-| `browser_extension_delete` | 7/7 | none |
-| `browser_extension_import` | 7/7 | none |
-| `browser_extension_list` | 7/7 | none |
-| `browser_goto` | 7/7 | none |
-| `browser_list_replays` | 7/7 | none |
-| `browser_list_sessions` | 7/7 | none |
-| `browser_locate` | 7/7 | none |
-| `browser_open` | 7/7 | none |
-| `browser_press` | 7/7 | none |
-| `browser_profile_connect` | 7/7 | none |
-| `browser_profile_list` | 7/7 | none |
-| `browser_read` | 7/7 | none |
-| `browser_replay_annotate` | 7/7 | none |
-| `browser_replay_download` | 7/7 | none |
-| `browser_replay_mark` | 7/7 | none |
-| `browser_replay_start` | 7/7 | none |
-| `browser_replay_stop` | 7/7 | none |
-| `browser_screenshot` | 7/7 | none |
-| `browser_scroll` | 7/7 | none |
-| `browser_type` | 7/7 | none |
-| `bulk-delete-notes` | 7/7 | none |
-| `call_service_connection_action` | 7/7 | none |
-| `capture_serp_page_snapshots` | 7/7 | none |
-| `capture_serp_snapshot` | 7/7 | none |
-| `check_site_export` | 7/7 | none |
-| `cost-usage` | 7/7 | none |
-| `create_editorial_reading_room` | 7/7 | none |
-| `create-channel` | 7/7 | none |
-| `create-scheduled-action` | 7/7 | none |
-| `create-secure-vault` | 7/7 | none |
-| `create-webhook` | 7/7 | none |
-| `credits_info` | 7/7 | none |
-| `delete-note` | 7/7 | none |
-| `delete-scheduled-action` | 7/7 | none |
-| `delete-vault` | 7/7 | none |
-| `describe_service_connection_tool` | 7/7 | none |
-| `diff_page` | 7/7 | none |
-| `directory_workflow` | 7/7 | none |
-| `directory_workflow_status` | 7/7 | none |
-| `editorial_reading_room_guide` | 7/7 | none |
-| `export_connected_service_data` | 7/7 | none |
-| `export_search_console_table_data` | 7/7 | none |
-| `extract_site` | 7/7 | none |
-| `extract_url` | 7/7 | none |
-| `facebook_ad_search` | 7/7 | none |
-| `facebook_ad_transcribe` | 7/7 | none |
-| `facebook_page_intel` | 7/7 | none |
-| `facebook_video_transcribe` | 7/7 | none |
-| `fact-history` | 7/7 | none |
-| `g2_reviews` | 7/7 | none |
-| `get-chat-link` | 7/7 | none |
-| `get-message-note` | 7/7 | none |
-| `get-schedule-link` | 7/7 | none |
-| `get-schedule-status` | 7/7 | none |
-| `get-vault-app-link` | 7/7 | none |
-| `get-vault-contract` | 7/7 | none |
-| `gmail_search_contacts` | 7/7 | none |
-| `gmail_send_message` | 7/7 | none |
-| `google_ads_page_intel` | 7/7 | none |
-| `google_ads_search` | 7/7 | none |
-| `google_ads_transcribe` | 7/7 | none |
-| `google_calendar_create_event` | 7/7 | none |
-| `harvest_paa` | 7/7 | none |
-| `import_service_connection_to_memory` | 7/7 | none |
-| `instagram_media_download` | 7/7 | none |
-| `instagram_profile_content` | 7/7 | none |
-| `library-ingest` | 7/7 | none |
-| `list_service_connections` | 7/7 | none |
-| `list-channel-members` | 7/7 | none |
-| `list-channel-messages` | 7/7 | none |
-| `list-memory-tags` | 7/7 | none |
-| `list-scheduled-actions` | 7/7 | none |
-| `list-shared-with-me` | 7/7 | none |
-| `list-vaults` | 7/7 | none |
-| `list-webhooks` | 7/7 | none |
-| `location_markets` | 7/7 | none |
-| `map_site_urls` | 7/7 | none |
-| `map_wayback_snapshots` | 7/7 | none |
-| `maps_place_intel` | 7/7 | none |
-| `maps_search` | 7/7 | none |
-| `memory-backlinks` | 7/7 | none |
-| `memory-capture` | 7/7 | none |
-| `memory-export` | 7/7 | none |
-| `memory-get` | 7/7 | none |
-| `memory-graph-path` | 7/7 | none |
-| `memory-graph-universe` | 7/7 | none |
-| `memory-list` | 7/7 | none |
-| `memory-put` | 7/7 | none |
-| `memory-questions` | 7/7 | none |
-| `memory-search` | 7/7 | none |
-| `memory-suggest` | 7/7 | none |
-| `memory-upload` | 7/7 | none |
-| `merge-memory-tags` | 7/7 | none |
-| `meta_ad_creative_media` | 7/7 | none |
-| `my-mentions` | 7/7 | none |
-| `pause-scheduled-action` | 7/7 | none |
-| `poll-channel` | 7/7 | none |
-| `post-message` | 7/7 | none |
-| `prepare-memory-write` | 7/7 | none |
-| `propose-scheduled-action` | 7/7 | none |
-| `provision-defaults` | 7/7 | none |
-| `query_fanout_workflow` | 7/7 | none |
-| `rank_tracker_workflow` | 7/7 | none |
-| `react-message` | 7/7 | none |
-| `read_service_connection` | 7/7 | none |
-| `record-fact` | 7/7 | none |
-| `reddit_thread` | 7/7 | none |
-| `reddit_trending` | 7/7 | none |
-| `remove-channel-member` | 7/7 | none |
-| `renew_connected_data_download` | 7/7 | none |
-| `renew_editorial_reading_room_download` | 7/7 | none |
-| `reply-message` | 7/7 | none |
-| `report_artifact_read` | 7/7 | none |
-| `resolve-memory-tags` | 7/7 | none |
-| `resume-scheduled-action` | 7/7 | none |
-| `revoke-chat-link` | 7/7 | none |
-| `revoke-schedule-link` | 7/7 | none |
-| `revoke-vault-app-link` | 7/7 | none |
-| `revoke-webhook` | 7/7 | none |
-| `route-memory` | 7/7 | none |
-| `search_serp` | 7/7 | none |
-| `set_scheduled_action_connections` | 7/7 | none |
-| `set-agent-identity` | 7/7 | none |
-| `set-schedule-defaults` | 7/7 | none |
-| `set-schedule-entitlement` | 7/7 | none |
-| `slack_send_message` | 7/7 | none |
-| `storage-usage` | 7/7 | none |
-| `table-create` | 7/7 | none |
-| `table-delete-rows` | 7/7 | none |
-| `table-describe` | 7/7 | none |
-| `table-drop` | 7/7 | none |
-| `table-insert-rows` | 7/7 | none |
-| `table-list` | 7/7 | none |
-| `table-query` | 7/7 | none |
-| `temporal-recall` | 7/7 | none |
-| `test_service_connection` | 7/7 | none |
-| `trustpilot_reviews` | 7/7 | none |
-| `upsert-memory-tag` | 7/7 | none |
-| `validate-memory-write` | 7/7 | none |
-| `video_frame_analysis` | 7/7 | none |
-| `video_frame_analysis_status` | 7/7 | none |
-| `video-analyze-start` | 7/7 | none |
-| `video-analyze-status` | 7/7 | none |
-| `workflow_artifact_read` | 7/7 | none |
-| `workflow_list` | 7/7 | none |
-| `workflow_run` | 7/7 | none |
-| `workflow_status` | 7/7 | none |
-| `workflow_step` | 7/7 | none |
-| `workflow_suggest` | 7/7 | none |
-| `youtube_harvest` | 7/7 | none |
-| `youtube_transcribe` | 7/7 | none |
-| `zoom_create_meeting` | 7/7 | none |

package/docs/mcp-tool-design-guide.md

@@ -1,225 +0,0 @@
-# Designing MCP Tools an AI Can Actually Find and Use
-
-A practical guide to building MCP servers whose tools an LLM (Claude, but the mechanics generalize)
-will **discover, select correctly, and invoke well**. It is written from how the model *actually*
-perceives tools at runtime — not how a human reads your docs page.
-
-The single thesis: **the model routes on what it can see, and it can see almost nothing until it asks.**
-Everything below follows from that.
-
----
-
-## 1. How the model actually sees your tools (the mechanic that governs everything)
-
-Modern MCP clients **defer** tool schemas to save context. The lifecycle in a session:
-
-1. **At rest — names only.** When the server connects, the model gets a flat list of tool
-   **identifiers** (`mcp__<server>__<tool>`) — *names, nothing else*. No parameters, no descriptions,
-   no input/output shapes. For ~50 tools that's a cheap, scannable block.
-2. **Server-instructions block — the one thing loaded upfront.** An MCP server may ship a top-level
-   `instructions` string. This **is** in context from the start, alongside the names. It's the only
-   place you can influence routing *before* the model fetches anything.
-3. **Discovery — the model fetches a schema.** To use a tool, the model runs a tool-search step
-   (exact `select:<name>` or a keyword query) that loads that tool's **full definition** — description
-   prose + parameter JSONSchema — into context. *This is the first moment the model sees how the tool works.*
-4. **Call → result.** The model invokes with arguments; your formatted result returns into context.
-
-Two consequences that drive the whole guide:
-
-- **Names do the *pre-fetch* routing.** The decision "which tool do I even look at?" is made from names
-  (+ the server-instructions block). A name that doesn't signal intent never gets fetched.
-- **Descriptions do the *post-fetch* disambiguation.** They guide correct parameter choice and break
-  ties between near-neighbors — but only *after* the model already chose to fetch. They can't rescue a
-  tool the model never looked at.
-
-> Mental model: the **name** is the search keyword; the **description** is the manual you read after you
-> already pulled the book off the shelf; the **server-instructions block** is the sign on the shelf.
-
----
-
-## 2. Principle 1 — The name is the routing surface
-
-Because names carry pre-fetch routing, treat them as the most important UX you ship.
-
-- **Front-load the discriminator.** Put the distinguishing token first: `youtube_transcribe`,
-  `facebook_video_transcribe`. The platform/object is what the model scans for.
-- **`platform_object_action` (or `object_action`).** A consistent scheme lets the model *predict*
-  names it hasn't seen and disambiguate neighbors.
-- **Never ship a bare generic verb.** `search` is a trap — search *what*? `search_serp`,
-  `maps_search`, `facebook_ad_search`. The qualifier prevents cross-fetch and sharpens keyword ranking.
-- **Disambiguate near-neighbors on their real axis.** `extract_url` vs `extract_site` vs
-  `map_site_urls` work because the names encode *single page / whole site / URLs-only*. If those blur,
-  the model picks the wrong granularity.
-- **Match the user's vocabulary, or bridge it.** Users say "scrape a page"; if the tool is
-  `extract_url`, that gap is only bridged *after* fetch (via description/`symptomTriggers`). Where a
-  phrasing is common and high-value, make the name or its triggers reflect it.
-- **Mind the namespace stutter.** The client prepends `mcp__`. Naming the server `mcp-scraper` yields
-  `mcp__mcp-scraper__…`. Name it `scraper` → `mcp__scraper__…`. Cosmetic, but it shows in permission
-  prompts/logs, and renaming later is breaking — do it early.
-
-### The decisive test: named tool vs. generic engine
-A user says *"collect ICP info from Reddit."*
-- A **`reddit_research`** tool: the word "reddit" hits the name directly → fetched immediately, one hop.
-- The generic **`browser_*`** agent: matches *none* of the user's words. To land there the model must
-  *infer* "no reddit tool → reddit is a website → drive a browser → navigate → search → read." Several
-  hops, each a chance to mis-route (e.g. grab `search_serp` and Google it instead).
-
-**Lesson:** if there's a phrase users will say that names a platform or intent, you want a **named tool
-catching it** — even if the implementation is a thin wrapper over a generic engine. *The name triggers;
-the engine executes.* A category that merely "points at the browser" serves a human clicking a button;
-it does **not** serve the model that has to *find* the capability from words.
-
----
-
-## 3. Principle 2 — Split by capability, parameterize the variations
-
-It's tempting to think "names are cheap, so ship hundreds of hyper-specific tools." Don't. Three reasons:
-
-1. **The name list isn't free at scale.** All names load at rest. 50 is fine; 300 is a wall the model
-   must scan and a real token cost. There's a ceiling.
-2. **Search ranking *degrades* with volume.** More near-neighbors → more ambiguous keyword ranking →
-   *higher* wrong-fetch rate. Over-splitting makes the exact failure you're trying to avoid worse.
-3. **It's the combinatorial trap.** `extract_url_screenshot`, `extract_url_branding`,
-   `extract_url_mobile`… is N×M×K variants of one capability. Parameters exist to prevent this.
-
-**The rule:** one tool per *distinct thing a user wants to do*; **parameters** for *variations* of that thing.
-
-The split test:
-> Would a user think of these as **different things to do**, or the **same thing with options**?
-
-- Different platform/object → different tool (`youtube_transcribe` vs `facebook_video_transcribe`).
-- Different granularity / mental model → different tool (`extract_url` vs `extract_site`).
-- A knob on the same action → **parameter** (`screenshot`, `formats`, `maxPages`, `rotateProxies`).
-
-Post-fetch param-learning is cheap and reliable for the model, so you lose almost nothing by using
-params for variations — and you keep the name list scannable and the ranking sharp.
-
----
-
-## 4. Principle 3 — Descriptions are the post-fetch safety net
-
-Loaded on fetch, the description's job is **correct usage + tie-breaking**, not discovery. Make it earn that:
-
-- **Lead with when-to-use and when-NOT-to-use.** Negative space ("use `extract_url` for one page; use
-  `extract_site` for a whole site") is what separates near-neighbors once both are fetched.
-- **State side effects and cost plainly.** Especially for tools that spend money, open sessions, or
-  post — the model should be able to refuse a wrong pick *before* executing.
-- **Encode parameter intent, bounds, and defaults** that mirror your actual schema, so the model fills
-  the form correctly the first time.
-- **Carry symptom triggers / sample prompts.** Phrases that should route here. These help both the
-  keyword search ranking and the post-fetch confirmation.
-
-Where the net fails (design against this): two tools with **similar names AND overlapping
-descriptions** — the wrong one reads "plausible enough" and the model proceeds. Distinct names +
-explicit negative space prevent it.
-
----
-
-## 5. Principle 4 — Use the server-instructions block as the routing map
-
-It's the only content loaded *before* fetch besides names, so it's your lever on pre-fetch routing.
-Keep it short and put **cross-tool routing**, not per-tool detail, there:
-
-- A decision map: "single page → `extract_url`; whole site → `extract_site`; just URLs → `map_site_urls`."
-- Setup/auth prerequisites that span tools.
-- How to batch related fetches (clients often let the model load several schemas in one search call —
-  tell it which tools group together).
-
-This is where you catch granularity and family mistakes *before* the model ever fetches the wrong one.
-
----
-
-## 6. Principle 5 — Two surfaces: human categories vs. AI tools
-
-Your frontend and your MCP tool list are **different products with different optimal shapes.**
-
-- **Humans** benefit from *consolidation*: one "Search" card with a source dropdown
-  (Web / Maps / Ads / YouTube). Fewer, richer entries; group by platform or verb as fits the UI.
-- **The model** benefits from *distinction*: separate, well-named tools (`search_serp`, `maps_search`,
-  `facebook_ad_search`). A single `search(source=…)` tool forces the model to fetch it just to learn the
-  source enum, and the name `search` is ambiguous at discovery.
-
-**Do not let the human IA pressure you into merging MCP tool names.** Map *one frontend card → many
-distinct MCP tools*. Two surfaces, one backend.
-
-Corollary — **a category can exist without a tool, but the AI path can't.** A "Reddit" card can be a
-guided browser flow with no `reddit_*` tool; a human clicks it fine. But the model gets a clean path
-*only* if a named tool exists (see §2). Decide per capability: human-only/occasional → category over the
-generic engine; want strong AI routing → ship the named wrapper.
-
-Platform-first vs verb-first grouping: for a multi-platform tool, **platform-first**
-(YouTube / Facebook / Google Maps / Websites / Browser / Workflows) often wins — it matches how users
-think *and* mirrors `platform_verb` naming. The category is the umbrella (covers search *and*
-transcribe *and* …); the tools inside are the named verbs.
-
----
-
-## 7. Principle 6 — Tool *results* are context too; keep them lean
-
-Discovery isn't the only context cost — **what you return floods the window if you let it.**
-
-- **Summarize inline, persist the bulk, hand back a reference.** For large outputs, return a concise
-  summary + a saved file / artifact / URL the model can read on demand, not the whole payload.
-- **Truncate with a signpost.** "showing first N of M; full data at `<path>`" beats silently dumping or
-  silently dropping.
-- **Put the full-fidelity data somewhere the model can fetch** (disk file, object storage URL), so the
-  model gets the gist now and the detail only if needed.
-- **Structured + human-readable.** Return both a structured object (for programmatic chaining) and a
-  short readable block (for the model to reason over) — don't make it parse a wall of text.
-
-A tool that returns 200 KB of HTML per call will blow out a multi-step task; one that returns a 2 KB
-summary + a file path scales.
-
----
-
-## 8. Failure modes → the fix
-
-| Failure | Cause | Fix |
-|---|---|---|
-| **Tool never found** | Name doesn't match user vocabulary | Front-loaded, intent-matching name; `symptomTriggers` |
-| **Wrong tool fetched & believed** | Similar names + overlapping descriptions | Distinct names; explicit negative space; bounds in description |
-| **Routed to generic engine the hard way** | Capability has no named tool | Thin named wrapper over the engine (§2 Reddit) |
-| **Wrong granularity** | `*_url` / `*_site` / `*_map` blur | Encode the axis in the name; routing map in server instructions |
-| **Picked god-tool, wrong sub-mode** | One tool, mega-param surface | Split distinct capabilities into separate tools |
-| **Context blown out** | Tool returns full payload | Summarize + persist + reference (§7) |
-| **Expensive wrong call** | Side-effecting tool, ambiguous name/desc | Clear name; state cost/side-effects up top; confirm before execute |
-| **Namespace stutter / churn** | Server renamed late | Pick the clean server name on day one |
-
----
-
-## 9. Naming checklist (per tool)
-
-- [ ] Discriminator (platform/object) is **first** in the name.
-- [ ] Follows a consistent `platform_object_action` scheme used across the server.
-- [ ] No bare generic verb; qualifier present.
-- [ ] Distinct from every neighbor on a real axis (object / granularity / platform).
-- [ ] Matches at least one phrase a real user would say (or `symptomTriggers` covers it).
-- [ ] It's a *distinct capability*, not a variation that should be a parameter.
-- [ ] Side effects / cost surfaced in the first lines of the description.
-- [ ] Description leads with when-to-use **and** when-not-to-use (negative space).
-- [ ] Params mirror the real schema (names, bounds, enums, defaults).
-- [ ] Result is lean: summary + reference, not a raw dump.
-
-## 10. Server-level checklist
-
-- [ ] Clean server name (no `mcp__mcp-…` stutter), chosen before launch.
-- [ ] `instructions` block carries a short **cross-tool routing map**, not per-tool detail.
-- [ ] Tool count is at the *capability* grain — not exploded into variants.
-- [ ] Frontend categories map to *multiple distinct* MCP tools (two surfaces).
-- [ ] High-value, user-named capabilities have a **named tool**, even if engine-backed.
-- [ ] Related tools are documented as a batch the model can fetch together.
-
----
-
-## 11. The one-paragraph version
-
-The model sees only tool **names** until it fetches a schema, so **names carry routing** — front-load
-the discriminator, never ship a bare verb, and give every user-named capability its own named tool
-(even a thin wrapper over a generic engine), because a name that matches the user's words is a one-hop
-hit while a generic engine is a multi-hop inference. Split tools by **distinct capability** and use
-**parameters** for variations — more tools past a point *degrades* selection, it doesn't help.
-**Descriptions** are the post-fetch safety net (when-to-use + when-not, cost, bounds); the
-**server-instructions block** is your only pre-fetch routing lever (put a cross-tool decision map
-there). Keep the **human frontend** (consolidated cards) and the **AI tool list** (distinct names)
-as separate surfaces over one backend. And remember tool **results are context** — return a summary
-plus a reference, never a raw dump.