ask-web-fetch-mcp 0.4.1 → 0.6.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 6f34204d8bcf16d483e5292a58672fd6011e3d7ed825b802beb07738c77a8ff1
-  data.tar.gz: c65b7193404995255cba3aa906d88442a559d2dc17e88835d9ed72ad8ca41601
+  metadata.gz: 33e377fca58f50a8e93cbe112374c0fe853dc5048505d7e7417d22d97e2c446d
+  data.tar.gz: 3aeb0347430bb21442599c38502c4d9063a9d75e3d3f88dcb812284ba5908b36
 SHA512:
-  metadata.gz: ad9196aa7fc0a8c19def71913eb8d69cc656ea01ea8ac218f20d973709a66eb51cda0017d9d2e2b492d813d4669d27fa8857f77136c0588d4845694766b660ba
-  data.tar.gz: 2c9dafac6c69c6cc699d721abdabaa86c013700a64a3548b9fc9b3b32de353a624694e3a2c22ad72078456f2b4d7ff8428653f6b76381e70e9591ad044a1ec50
+  metadata.gz: 6ff9783fb23e9b8912c4668787d64667949e69f5234dee7db4d2db52221bcf74bae27e0ce88ed3b96220516991bd3e8a0cb29b6b87da8ec81d1ffeb7a39890c7
+  data.tar.gz: aa002fe73da25c7b754941ebc27a69fa5c799e0ab5c05b1e1c65f2d58f1c9b2df617d4bd28d1231cd80dd8a9652920699752a6d97547b4154a256a6becfe2830

data/CHANGELOG.md

@@ -1,3 +1,47 @@
+## [0.6.0] — 2026-08-12
+
+### Changed
+
+- **The server owns its tool shell.** `ask_web_fetch` is now a
+  duck-typed tool (`Ask::WebFetch::MCP::Tool` — `name` /
+  `description` / `params_schema` / `call`) wrapping the library entry
+  `Ask::WebFetch.fetch`, instead of a renamed `Ask::Tools::WebFetch`.
+  The MCP adapter's contract is duck-typed by design, so the server
+  gains nothing from the ask-tools machinery — and the ask-tools /
+  ask-core / ask-schema dependency chain is gone from the server
+  process entirely. `ask-web-fetch` floor raised to `>= 0.7.1` (the
+  module-level API with the failure collapse and parked-domain
+  detection on every backend).
+- **Cleaner error framing.** A failed call now surfaces as
+  `Error: Ask::WebFetch::ParkedDomainError: ...` — the class is still
+  named, without the tool layer's double wrap. Terminal verdicts
+  (parked, empty, dead 4xx) are never retried by clients; transient
+  failures raise the base `Error`.
+- **The native agent tool stays available for non-MCP consumers.**
+  `Ask::Tools::WebFetch` lives in ask-web-fetch as an optional
+  integration (registered when ask-tools is present) for agent
+  frameworks that resolve tools by name.
+
+## [0.5.0] — 2026-08-12
+
+### Changed
+
+- `ask-web-fetch` floor raised to `>= 0.6.1`. The server exposes the tool
+  unchanged, but `ask_web_fetch` now answers with the full current stack:
+  the pooled-httpx transport (no more hangs on multi-host crawls), real
+  network-idle waits in CDP-attached mode, JS-app-shell detection that
+  fails client-rendered pages through to a rendering backend, parked-domain
+  detection on every backend (GoDaddy/Namecheap registrar ads are rejected,
+  never returned as content), and warm-and-retry challenge handling.
+- **Failure verdicts are now classed.** When every backend fails, the tool
+  raises the most definitive class — `ParkedDomainError` beats
+  `EmptyContentError` beats a deterministic `FetchError` (every backend
+  failed dead), and any transient failure in the mix keeps the retryable
+  base `Error`. A client calling `ask_web_fetch` on a parked domain gets
+  an error naming `ParkedDomainError` instead of a generic failure, so it
+  stops retrying the unretryable. The message still lists every backend
+  and what it said.
+
 ## [0.4.1] — 2026-08-11
 
 ### Changed

data/README.md

@@ -5,9 +5,10 @@
 A minimal MCP (Model Context Protocol) server that exposes
 `Ask::Tools::WebFetch` as a callable tool over stdio. Designed for use with
 clients that support MCP (ZCode, Claude Code, etc.), it fetches a URL and
-returns clean markdown for LLM consumption — via the local pure-Ruby backend
-by default, with an automatic Jina Reader fallback for JS-rendered or blocked
-pages.
+returns clean markdown for LLM consumption — through the full ask-web-fetch
+backend chain: the fast pure-Ruby httpx fetch first, Jina Reader and
+self-hosted Crawl4AI in between, and a real Chrome (launched, or attached
+over CDP) last for JS-rendered and challenge-gated pages.
 
 ## Installation
 
@@ -56,9 +57,23 @@ collisions with client-side tools of the same name, matching
 The `ask-web-fetch` backend chain applies unchanged. Optional environment
 variables:
 
+- `ASK_WEB_FETCH_CDP_URL` — CDP endpoint of an already-running Chrome
+  (e.g. `http://127.0.0.1:9222`); routes challenge-gated pages through a
+  trusted browser that has already solved them
+- `ASK_WEB_FETCH_CHROME_PATH` / `ASK_WEB_FETCH_PROFILE` — tune the
+  launched-browser mode (binary path, persistent profile for solved
+  cookies)
+- `CRAWL4AI_URL` / `CRAWL4AI_TOKEN` — lead the chain with a self-hosted
+  Crawl4AI renderer (`http://localhost:11235` when unset)
 - `JINA_API_KEY` — enables the Jina fallback with higher rate limits
 - `DEBUG=1` — ask-mcp debug logging on stderr
 
+When every backend fails, the error names the verdict class: terminal
+outcomes — a parked domain (`ParkedDomainError`), a page with no usable
+content (`EmptyContentError`), a dead URL (`FetchError`) — are never
+retryable; transient failures (timeouts, 5xx) raise the base
+`Ask::WebFetch::Error`.
+
 ## Full documentation
 
 The full ask-rb documentation lives at https://ask-rb.github.io/ask-docs.

data/lib/ask/web_fetch/mcp/tool.rb

@@ -0,0 +1,49 @@
+# frozen_string_literal: true
+
+require 'ask/web_fetch'
+
+module Ask
+  module WebFetch
+    module MCP
+      # The ask_web_fetch tool, duck-typed for Ask::MCP's ToolServer
+      # adapter (name / description / params_schema / call). The tool
+      # framing lives with its consumer — this server — not in the
+      # library: the capability is Ask::WebFetch.fetch, this is the
+      # agent-facing shell around it.
+      class Tool
+        def name
+          'ask_web_fetch'
+        end
+
+        def description
+          'Fetch a URL and return its content as clean markdown for LLM consumption. ' \
+          'Use this to read web pages, articles, and documentation.'
+        end
+
+        def params_schema
+          {
+            'type' => 'object',
+            'properties' => {
+              'url' => { 'type' => 'string', 'description' => 'The URL to fetch' },
+              'max_chars' => { 'type' => 'integer', 'description' => 'Maximum number of characters to return (default 20000)' }
+            },
+            'required' => ['url']
+          }
+        end
+
+        # Returns the fetched markdown (a String is a success for the
+        # adapter) or raises. The terminal verdicts — ParkedDomainError,
+        # EmptyContentError, FetchError — reach the client as their class,
+        # so it never retries the unretryable; transient failures raise
+        # the base Error.
+        def call(args)
+          url = args['url'].to_s
+          raise ArgumentError, 'missing required parameter: url' if url.empty?
+
+          max_chars = args['max_chars']
+          Ask::WebFetch.fetch(url, max_chars: max_chars || Ask::WebFetch::DEFAULT_MAX_CHARS)
+        end
+      end
+    end
+  end
+end

data/lib/ask/web_fetch/mcp/version.rb

@@ -3,7 +3,7 @@
 module Ask
   module WebFetch
     module MCP
-      VERSION = '0.4.1'
+      VERSION = '0.6.0'
     end
   end
 end

data/lib/ask/web_fetch/mcp.rb

@@ -2,19 +2,20 @@
 
 require 'ask/mcp'
 require 'ask/web_fetch'
+require_relative 'mcp/tool'
 require_relative 'mcp/version'
 
 module Ask
   module WebFetch
     # MCP (Model Context Protocol) server for ask-web-fetch.
     module MCP
-      # Builds the tool exposed over MCP: Ask::Tools::WebFetch renamed to
-      # "ask_web_fetch" to avoid collisions with client-side tools of the
-      # same name (ask-web-search-mcp follows the same convention).
+      # Builds the tool exposed over MCP. The tool framing lives here —
+      # ask-web-fetch is a library (Ask::WebFetch.fetch); this server
+      # owns the agent-facing shell, named "ask_web_fetch" to avoid
+      # collisions with client-side tools of the same name
+      # (ask-web-search-mcp follows the same convention).
       def self.tool
-        tool = Ask::Tools::WebFetch.new
-        tool.define_singleton_method(:name) { 'ask_web_fetch' }
-        tool
+        Tool.new
       end
 
       # Start the MCP server over stdio, exposing the ask_web_fetch tool.

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: ask-web-fetch-mcp
 version: !ruby/object:Gem::Version
-  version: 0.4.1
+  version: 0.6.0
 platform: ruby
 authors:
 - Kaka Ruto
@@ -29,14 +29,14 @@ dependencies:
     requirements:
     - - ">="
       - !ruby/object:Gem::Version
-        version: 0.5.1
+        version: 0.7.1
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - ">="
       - !ruby/object:Gem::Version
-        version: 0.5.1
+        version: 0.7.1
 - !ruby/object:Gem::Dependency
   name: minitest
   requirement: !ruby/object:Gem::Requirement
@@ -80,10 +80,15 @@ dependencies:
       - !ruby/object:Gem::Version
         version: '3.26'
 description: |
-  A minimal MCP (Model Context Protocol) server that exposes Ask::Tools::WebFetch
+  A minimal MCP (Model Context Protocol) server that exposes ask_web_fetch
   as a callable tool over stdio. Designed for use with clients that support MCP
-  (ZCode, Claude Code, etc.), it fetches a URL and returns clean markdown via the
-  local pure-Ruby backend, with an automatic Jina Reader fallback.
+  (ZCode, Claude Code, etc.), it fetches a URL and returns clean markdown through
+  the ask-web-fetch backend chain — fast pure-Ruby httpx fetch first, a real
+  Chrome (launched or CDP-attached) for JS-rendered and challenge-gated pages,
+  with Jina Reader and self-hosted Crawl4AI in between. Terminal failures
+  (parked domains, empty pages, dead 4xx) surface as their deterministic error
+  class, so clients never retry the unretryable. The tool shell (name, schema,
+  call) lives here, wrapping the Ask::WebFetch library.
 email:
 - kaka@myrrlabs.com
 executables:
@@ -97,6 +102,7 @@ files:
 - bin/ask-web-fetch-mcp
 - lib/ask-web-fetch-mcp.rb
 - lib/ask/web_fetch/mcp.rb
+- lib/ask/web_fetch/mcp/tool.rb
 - lib/ask/web_fetch/mcp/version.rb
 homepage: https://github.com/ask-rb/ask-web-fetch-mcp
 licenses: