ask-web-fetch-mcp 0.4.1 → 0.5.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 6f34204d8bcf16d483e5292a58672fd6011e3d7ed825b802beb07738c77a8ff1
-  data.tar.gz: c65b7193404995255cba3aa906d88442a559d2dc17e88835d9ed72ad8ca41601
+  metadata.gz: 8ba4b67fe027c74b1e68b80af6cd6dc83479464c936f37bb558ce4b60c8811f3
+  data.tar.gz: 86f9bf7f1bab0e3d7678a966259af096593bb665d30d4abcd3121085051f891c
 SHA512:
-  metadata.gz: ad9196aa7fc0a8c19def71913eb8d69cc656ea01ea8ac218f20d973709a66eb51cda0017d9d2e2b492d813d4669d27fa8857f77136c0588d4845694766b660ba
-  data.tar.gz: 2c9dafac6c69c6cc699d721abdabaa86c013700a64a3548b9fc9b3b32de353a624694e3a2c22ad72078456f2b4d7ff8428653f6b76381e70e9591ad044a1ec50
+  metadata.gz: da671bda628d54eeb7cf60e573d5317c568516f16d3c547e48d6f8c813697f9ff9f3442c0578ad4dd360f6e08c9d0faac466710d1f32c882f2db736874f486ca
+  data.tar.gz: 8c53d1f55dc8dbc661538b65b3e1e76ac65121c6edcb6c57c94d28a87ecfc328ad5940a595ee452724f987497fdd8c32277a61a21238f7a6c03bd6eeae3e1ac0

data/CHANGELOG.md

@@ -1,3 +1,23 @@
+## [0.5.0] — 2026-08-12
+
+### Changed
+
+- `ask-web-fetch` floor raised to `>= 0.6.1`. The server exposes the tool
+  unchanged, but `ask_web_fetch` now answers with the full current stack:
+  the pooled-httpx transport (no more hangs on multi-host crawls), real
+  network-idle waits in CDP-attached mode, JS-app-shell detection that
+  fails client-rendered pages through to a rendering backend, parked-domain
+  detection on every backend (GoDaddy/Namecheap registrar ads are rejected,
+  never returned as content), and warm-and-retry challenge handling.
+- **Failure verdicts are now classed.** When every backend fails, the tool
+  raises the most definitive class — `ParkedDomainError` beats
+  `EmptyContentError` beats a deterministic `FetchError` (every backend
+  failed dead), and any transient failure in the mix keeps the retryable
+  base `Error`. A client calling `ask_web_fetch` on a parked domain gets
+  an error naming `ParkedDomainError` instead of a generic failure, so it
+  stops retrying the unretryable. The message still lists every backend
+  and what it said.
+
 ## [0.4.1] — 2026-08-11
 
 ### Changed

data/README.md

@@ -5,9 +5,10 @@
 A minimal MCP (Model Context Protocol) server that exposes
 `Ask::Tools::WebFetch` as a callable tool over stdio. Designed for use with
 clients that support MCP (ZCode, Claude Code, etc.), it fetches a URL and
-returns clean markdown for LLM consumption — via the local pure-Ruby backend
-by default, with an automatic Jina Reader fallback for JS-rendered or blocked
-pages.
+returns clean markdown for LLM consumption — through the full ask-web-fetch
+backend chain: the fast pure-Ruby httpx fetch first, Jina Reader and
+self-hosted Crawl4AI in between, and a real Chrome (launched, or attached
+over CDP) last for JS-rendered and challenge-gated pages.
 
 ## Installation
 
@@ -56,9 +57,23 @@ collisions with client-side tools of the same name, matching
 The `ask-web-fetch` backend chain applies unchanged. Optional environment
 variables:
 
+- `ASK_WEB_FETCH_CDP_URL` — CDP endpoint of an already-running Chrome
+  (e.g. `http://127.0.0.1:9222`); routes challenge-gated pages through a
+  trusted browser that has already solved them
+- `ASK_WEB_FETCH_CHROME_PATH` / `ASK_WEB_FETCH_PROFILE` — tune the
+  launched-browser mode (binary path, persistent profile for solved
+  cookies)
+- `CRAWL4AI_URL` / `CRAWL4AI_TOKEN` — lead the chain with a self-hosted
+  Crawl4AI renderer (`http://localhost:11235` when unset)
 - `JINA_API_KEY` — enables the Jina fallback with higher rate limits
 - `DEBUG=1` — ask-mcp debug logging on stderr
 
+When every backend fails, the error names the verdict class: terminal
+outcomes — a parked domain (`ParkedDomainError`), a page with no usable
+content (`EmptyContentError`), a dead URL (`FetchError`) — are never
+retryable; transient failures (timeouts, 5xx) raise the base
+`Ask::WebFetch::Error`.
+
 ## Full documentation
 
 The full ask-rb documentation lives at https://ask-rb.github.io/ask-docs.

data/lib/ask/web_fetch/mcp/version.rb

@@ -3,7 +3,7 @@
 module Ask
   module WebFetch
     module MCP
-      VERSION = '0.4.1'
+      VERSION = '0.5.0'
     end
   end
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: ask-web-fetch-mcp
 version: !ruby/object:Gem::Version
-  version: 0.4.1
+  version: 0.5.0
 platform: ruby
 authors:
 - Kaka Ruto
@@ -29,14 +29,14 @@ dependencies:
     requirements:
     - - ">="
       - !ruby/object:Gem::Version
-        version: 0.5.1
+        version: 0.6.1
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - ">="
       - !ruby/object:Gem::Version
-        version: 0.5.1
+        version: 0.6.1
 - !ruby/object:Gem::Dependency
   name: minitest
   requirement: !ruby/object:Gem::Requirement
@@ -82,8 +82,12 @@ dependencies:
 description: |
   A minimal MCP (Model Context Protocol) server that exposes Ask::Tools::WebFetch
   as a callable tool over stdio. Designed for use with clients that support MCP
-  (ZCode, Claude Code, etc.), it fetches a URL and returns clean markdown via the
-  local pure-Ruby backend, with an automatic Jina Reader fallback.
+  (ZCode, Claude Code, etc.), it fetches a URL and returns clean markdown through
+  the ask-web-fetch backend chain — fast pure-Ruby httpx fetch first, a real
+  Chrome (launched or CDP-attached) for JS-rendered and challenge-gated pages,
+  with Jina Reader and self-hosted Crawl4AI in between. Terminal failures
+  (parked domains, empty pages, dead 4xx) surface as their deterministic error
+  class, so clients never retry the unretryable.
 email:
 - kaka@myrrlabs.com
 executables: