unique_web_search 2026.30.0.dev3__tar.gz → 2026.30.0.dev4__tar.gz

unique_web_search-2026.30.0.dev4/PKG-INFO

@@ -0,0 +1,146 @@
+Metadata-Version: 2.3
+Name: unique-web-search
+Version: 2026.30.0.dev4
+Summary: 
+Author: Andreas Hauri, Gustav Hartz, Rami Azouz
+Author-email: Andreas Hauri <andreas@unique.ch>, Gustav Hartz <gustav.hartz.ext@unique.ch>, Rami Azouz <rami.ext@unique.ch>
+License: Proprietary
+Requires-Dist: typing-extensions>=4.14.1,<5
+Requires-Dist: pydantic>=2.12.3,<3
+Requires-Dist: pydantic-settings>=2.10.1,<3
+Requires-Dist: timeout-decorator>=0.5.0,<1
+Requires-Dist: markdownify>=0.14.1,<1
+Requires-Dist: fake-useragent>=2.2.0,<3
+Requires-Dist: firecrawl>=3.3.2,<4
+Requires-Dist: tavily-python>=0.7.11,<1
+Requires-Dist: unidecode>=1.4.0,<2
+Requires-Dist: azure-ai-projects>=1.0.0,<2
+Requires-Dist: azure-identity>=1.25.0,<2
+Requires-Dist: unique-toolkit[monitoring]>=2026.30.0.dev20,<2026.30.0rc0
+Requires-Dist: azure-core>=1.36.0,<2
+Requires-Dist: google-cloud-aiplatform>=1.128.0,<2
+Requires-Dist: google-auth>=2.43.0,<3
+Requires-Dist: google-generativeai>=0.8.5,<0.9
+Requires-Dist: rapidfuzz>=3.0.0,<4
+Requires-Dist: httpx>=0.28.1
+Requires-Dist: certifi>=2025.11.12
+Requires-Dist: jinja2>=3.1.6
+Requires-Dist: unique-search-proxy-core>=2026.30.0.dev5,<2026.30.0rc0
+Requires-Dist: unique-search-proxy-sdk>=2026.30.0.dev3,<2026.30.0rc0
+Requires-Python: >=3.12, <4.0
+Description-Content-Type: text/markdown
+
+# Unique Web Search
+
+Configurable web search tool for Unique assistants: search the web, read pages, and return citation-ready content chunks.
+
+## Search Proxy (recommended)
+
+Outbound web access should go through **Unique Search Proxy** (`connectors/unique_search_proxy/`). When `SEARCH_PROXY_BASE_URL` is set, search and crawl calls use `unique-search-proxy-sdk`; credentials and provider egress stay on the proxy pod.
+
+| Mode | When | Behaviour |
+|------|------|-----------|
+| **Proxy (recommended)** | `SEARCH_PROXY_BASE_URL` set | `_proxy_search` / `_proxy_crawl` via the SDK |
+| **Legacy** | URL unset | Direct provider calls from this process (`_legacy_search` / `_legacy_crawl`) |
+
+- **New standard engines (Brave, Perplexity) are proxy-only** — legacy raises.
+- **Custom API** always runs locally (never routed through the proxy).
+- Future provider work targets Search Proxy only; legacy is preserved for migrated engines.
+
+Packages involved: [`unique-search-proxy-core`](../../connectors/unique_search_proxy/unique_search_proxy_core/README.md) (schemas), [`unique-search-proxy-sdk`](../../connectors/unique_search_proxy/unique_search_proxy_sdk/README.md) (HTTP), [`unique-search-proxy`](../../connectors/unique_search_proxy/unique_search_proxy_client/README.md) (server). System overview: [`connectors/unique_search_proxy/README.md`](../../connectors/unique_search_proxy/README.md).
+
+## Architecture
+
+![Web Search Tool Architecture](docs/images/architecture-diagram.svg)
+
+## Search engine kinds
+
+### Standard search
+
+Query → normalised results (URL / title / snippet / optional content). Optional page crawl via the configured crawler.
+
+| Engine | Proxy | Notes |
+|--------|-------|-------|
+| Google | legacy + proxy | Requires scraping by default |
+| Brave | **proxy-only** | Rich snippets; `ExposableParam` knobs |
+| Perplexity | **proxy-only** | Content extraction knobs; `ExposableParam` |
+| Custom API | always local | Your REST endpoint |
+
+Standard engines that live in proxy-core use **`ExposableParam`** (`expose` + `value`) for optional provider knobs. See [Search Engines README](./src/unique_web_search/services/search_engine/README.md).
+
+### Agent search (grounding)
+
+An upstream AI agent / grounded model searches and returns opaque text that this tool parses into `WebSearchResult`s.
+
+| Engine | Proxy | Notes |
+|--------|-------|-------|
+| Bing (Azure AI Foundry grounding) | legacy + proxy | Configurable `requires_scraping` |
+| VertexAI (Gemini grounding) | legacy + proxy | Configurable `requires_scraping` |
+
+Agent engines do **not** use `ExposableParam`.
+
+## Web crawlers (page readers)
+
+| Crawler | Auth | Notes |
+|---------|------|-------|
+| Basic | none | HTTP + content-type toggles; tool-local `url_blocked_patterns` (legacy) |
+| Tavily | `TAVILY_API_KEY` | Extract API |
+| Jina | `JINA_API_KEY` | Reader API |
+| Firecrawl | `FIRECRAWL_API_KEY` | Batch scrape |
+
+No Crawl4AI. Crawler configs have **no exposable LLM knobs**.
+
+## Execution modes
+
+| Mode | Status | Behaviour |
+|------|--------|-----------|
+| **V1** | Stable | Refine query → search → crawl → process |
+| **V2** | Stable | Parallel plan steps (search / read URL) |
+| **V3** | Experimental | Agent loop: snippet `search` vs on-demand `read_urls` |
+
+Proxy vs legacy is transparent to executors.
+
+## Detailed subsystem docs
+
+- [Search Engines](./src/unique_web_search/services/search_engine/README.md)
+- [Crawlers](./src/unique_web_search/services/crawlers/README.md)
+- [Executors](./src/unique_web_search/services/executors/README.md)
+
+## Configuration
+
+- `SEARCH_PROXY_BASE_URL` — enables the Search Proxy client
+- Provider API keys (on the proxy pod when proxy is enabled; otherwise on assistants-core for legacy)
+- Search / crawler / content-processing settings via `WebSearchConfig`
+
+## Dependency management (uv.lock + min/latest testing)
+
+This package is a **library** and uses `uv` for dependency management.
+
+We run tests additionally with minimal dependencies to ensure that the listed ranges are valid. NOTE: We use lowest-direct, not lowest.
+Lowest attempts to use the lowest possible dependency versions _transitively_ causing issues if a dependency has incorrect metadata. Example:
+- google-cloud-aiplatform says it works with shapely<3.0.0.
+- The lowest resolver assumes 1.0 which needs python 2 -> breaks
+Therefore we use lowest-direct which only sets our direct dependencies to lowest. However, this only correctly verifies our min dependencies
+if our code correctly lists all the required dependencies and never imports a transitive dependency. We therefore use deptry to ensure we
+don't use transitive dependencies and that we have no unused dependencies.
+
+### Test locally
+
+- **Latest deps and deptry**:
+
+```bash
+cd tool_packages/unique_web_search
+uv sync
+uv run pytest
+uv run deptry
+```
+
+- **Min deps**:
+
+```bash
+cd tool_packages/unique_web_search
+uv venv
+uv pip install -e . --resolution=lowest-direct
+uv export --only-group dev --no-hashes | uv pip install -r -
+uv run --no-sync pytest
+```

unique_web_search-2026.30.0.dev4/README.md

@@ -0,0 +1,114 @@
+# Unique Web Search
+
+Configurable web search tool for Unique assistants: search the web, read pages, and return citation-ready content chunks.
+
+## Search Proxy (recommended)
+
+Outbound web access should go through **Unique Search Proxy** (`connectors/unique_search_proxy/`). When `SEARCH_PROXY_BASE_URL` is set, search and crawl calls use `unique-search-proxy-sdk`; credentials and provider egress stay on the proxy pod.
+
+| Mode | When | Behaviour |
+|------|------|-----------|
+| **Proxy (recommended)** | `SEARCH_PROXY_BASE_URL` set | `_proxy_search` / `_proxy_crawl` via the SDK |
+| **Legacy** | URL unset | Direct provider calls from this process (`_legacy_search` / `_legacy_crawl`) |
+
+- **New standard engines (Brave, Perplexity) are proxy-only** — legacy raises.
+- **Custom API** always runs locally (never routed through the proxy).
+- Future provider work targets Search Proxy only; legacy is preserved for migrated engines.
+
+Packages involved: [`unique-search-proxy-core`](../../connectors/unique_search_proxy/unique_search_proxy_core/README.md) (schemas), [`unique-search-proxy-sdk`](../../connectors/unique_search_proxy/unique_search_proxy_sdk/README.md) (HTTP), [`unique-search-proxy`](../../connectors/unique_search_proxy/unique_search_proxy_client/README.md) (server). System overview: [`connectors/unique_search_proxy/README.md`](../../connectors/unique_search_proxy/README.md).
+
+## Architecture
+
+![Web Search Tool Architecture](docs/images/architecture-diagram.svg)
+
+## Search engine kinds
+
+### Standard search
+
+Query → normalised results (URL / title / snippet / optional content). Optional page crawl via the configured crawler.
+
+| Engine | Proxy | Notes |
+|--------|-------|-------|
+| Google | legacy + proxy | Requires scraping by default |
+| Brave | **proxy-only** | Rich snippets; `ExposableParam` knobs |
+| Perplexity | **proxy-only** | Content extraction knobs; `ExposableParam` |
+| Custom API | always local | Your REST endpoint |
+
+Standard engines that live in proxy-core use **`ExposableParam`** (`expose` + `value`) for optional provider knobs. See [Search Engines README](./src/unique_web_search/services/search_engine/README.md).
+
+### Agent search (grounding)
+
+An upstream AI agent / grounded model searches and returns opaque text that this tool parses into `WebSearchResult`s.
+
+| Engine | Proxy | Notes |
+|--------|-------|-------|
+| Bing (Azure AI Foundry grounding) | legacy + proxy | Configurable `requires_scraping` |
+| VertexAI (Gemini grounding) | legacy + proxy | Configurable `requires_scraping` |
+
+Agent engines do **not** use `ExposableParam`.
+
+## Web crawlers (page readers)
+
+| Crawler | Auth | Notes |
+|---------|------|-------|
+| Basic | none | HTTP + content-type toggles; tool-local `url_blocked_patterns` (legacy) |
+| Tavily | `TAVILY_API_KEY` | Extract API |
+| Jina | `JINA_API_KEY` | Reader API |
+| Firecrawl | `FIRECRAWL_API_KEY` | Batch scrape |
+
+No Crawl4AI. Crawler configs have **no exposable LLM knobs**.
+
+## Execution modes
+
+| Mode | Status | Behaviour |
+|------|--------|-----------|
+| **V1** | Stable | Refine query → search → crawl → process |
+| **V2** | Stable | Parallel plan steps (search / read URL) |
+| **V3** | Experimental | Agent loop: snippet `search` vs on-demand `read_urls` |
+
+Proxy vs legacy is transparent to executors.
+
+## Detailed subsystem docs
+
+- [Search Engines](./src/unique_web_search/services/search_engine/README.md)
+- [Crawlers](./src/unique_web_search/services/crawlers/README.md)
+- [Executors](./src/unique_web_search/services/executors/README.md)
+
+## Configuration
+
+- `SEARCH_PROXY_BASE_URL` — enables the Search Proxy client
+- Provider API keys (on the proxy pod when proxy is enabled; otherwise on assistants-core for legacy)
+- Search / crawler / content-processing settings via `WebSearchConfig`
+
+## Dependency management (uv.lock + min/latest testing)
+
+This package is a **library** and uses `uv` for dependency management.
+
+We run tests additionally with minimal dependencies to ensure that the listed ranges are valid. NOTE: We use lowest-direct, not lowest.
+Lowest attempts to use the lowest possible dependency versions _transitively_ causing issues if a dependency has incorrect metadata. Example:
+- google-cloud-aiplatform says it works with shapely<3.0.0.
+- The lowest resolver assumes 1.0 which needs python 2 -> breaks
+Therefore we use lowest-direct which only sets our direct dependencies to lowest. However, this only correctly verifies our min dependencies
+if our code correctly lists all the required dependencies and never imports a transitive dependency. We therefore use deptry to ensure we
+don't use transitive dependencies and that we have no unused dependencies.
+
+### Test locally
+
+- **Latest deps and deptry**:
+
+```bash
+cd tool_packages/unique_web_search
+uv sync
+uv run pytest
+uv run deptry
+```
+
+- **Min deps**:
+
+```bash
+cd tool_packages/unique_web_search
+uv venv
+uv pip install -e . --resolution=lowest-direct
+uv export --only-group dev --no-hashes | uv pip install -r -
+uv run --no-sync pytest
+```

{unique_web_search-2026.30.0.dev3 → unique_web_search-2026.30.0.dev4}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "unique_web_search"
-version = "2026.30.0.dev3"
+version = "2026.30.0.dev4"
 description = ""
 readme = "README.md"
 requires-python = ">=3.12,<4.0"
@@ -22,7 +22,7 @@ dependencies = [
     "unidecode>=1.4.0,<2",
     "azure-ai-projects>=1.0.0,<2",
     "azure-identity>=1.25.0,<2",
-    "unique-toolkit[monitoring]>=2026.30.0.dev14,<2026.30.0rc0",
+    "unique-toolkit[monitoring]>=2026.30.0.dev20,<2026.30.0rc0",
     "azure-core>=1.36.0,<2",
     "google-cloud-aiplatform>=1.128.0,<2",
     "google-auth>=2.43.0,<3",
@@ -31,8 +31,8 @@ dependencies = [
     "httpx>=0.28.1",
     "certifi>=2025.11.12",
     "jinja2>=3.1.6",
-    "unique-search-proxy-core>=2026.30.0.dev3,<2026.30.0rc0",
-    "unique-search-proxy-sdk>=2026.30.0.dev2,<2026.30.0rc0",
+    "unique-search-proxy-core>=2026.30.0.dev5,<2026.30.0rc0",
+    "unique-search-proxy-sdk>=2026.30.0.dev3,<2026.30.0rc0",
 ]
 
 [build-system]

{unique_web_search-2026.30.0.dev3 → unique_web_search-2026.30.0.dev4}/src/unique_web_search/service.py

@@ -3,6 +3,7 @@ from datetime import datetime
 from time import time
 
 from typing_extensions import override
+from unique_search_proxy_core.context import RequestContext
 from unique_search_proxy_core.param_policy.exposed_params import ExposedParams
 from unique_search_proxy_core.search_engines.base import BaseSearchEngineConfig
 from unique_toolkit._common.chunk_relevancy_sorter.service import ChunkRelevancySorter
@@ -72,13 +73,22 @@ class WebSearchTool(Tool[WebSearchConfig]):
             language_model_orchestrator or configuration.token_counting_language_model
         )
 
+        self.chunk_relevancy_sorter = ChunkRelevancySorter(self.event)
+        self.company_id = self.event.company_id
+        self.request_context = RequestContext(
+            company_id=self.event.company_id,
+            user_id=self.event.user_id,
+            chat_id=self.event.payload.chat_id,
+        )
         self.search_engine_service = get_search_engine_service(
             self.config.search_engine_config,
             self.language_model_service,
+            request_context=self.request_context,
+        )
+        self.crawler_service = get_crawler_service(
+            self.config.crawler_config,
+            request_context=self.request_context,
         )
-        self.crawler_service = get_crawler_service(self.config.crawler_config)
-        self.chunk_relevancy_sorter = ChunkRelevancySorter(self.event)
-        self.company_id = self.event.company_id
         self.chat_history_token_length = 0
         self.chat_history_chat_messages = self._chat_service.get_full_history()
 

unique_web_search-2026.30.0.dev4/src/unique_web_search/services/crawlers/README.md

@@ -0,0 +1,110 @@
+# Crawlers
+
+Web-page readers used by Unique Web Search to fetch URLs and convert them to text.
+
+## Architecture
+
+Crawler configuration lives in **`unique_search_proxy_core.crawlers`**. The tool registers implementations via `CRAWLER_REGISTRY` and dispatches through `BaseCrawler`.
+
+```mermaid
+flowchart LR
+  Config["proxy-core *Config"]
+  Registry["CRAWLER_REGISTRY"]
+  Base["BaseCrawler"]
+  SDK["SDK crawl"]
+  Legacy["_legacy_crawl"]
+
+  Config --> Registry
+  Registry --> Base
+  Base -->|"SEARCH_PROXY_BASE_URL set"| SDK
+  Base -->|"unset"| Legacy
+```
+
+### Key invariants
+
+- **No exposable params.** Crawlers have no LLM override surface — only deployment defaults + `urls`.
+- **Proxy body = config fields + urls.** `BaseCrawler._proxy_crawl` dumps `config.model_dump(exclude={"crawler", ...})` into the SDK call.
+- **Legacy retained** when the search proxy is disabled.
+
+## Available crawlers
+
+| Name | Config source | Auth | Notes |
+|------|---------------|------|-------|
+| Basic | tool-local `BasicConfig` extends proxy-core | none | HTTP fetch + processors; `content_types` toggles; `url_blocked_patterns` (tool-local, stripped before proxy) |
+| Tavily | `TavilyConfig` | `TAVILY_API_KEY` | Extract API |
+| Jina | `JinaConfig` | `JINA_API_KEY` | Reader API |
+| Firecrawl | `FirecrawlConfig` | `FIRECRAWL_API_KEY` | Batch scrape |
+
+There is **no Crawl4AI** crawler.
+
+`WebSearchConfig.crawler_config` is a discriminated union on `crawler` (`Basic` / `Tavily` / `Jina` / `Firecrawl`).
+
+---
+
+## Basic
+
+| Field | Default | Notes |
+|-------|---------|-------|
+| `timeout` | from base | Per-request timeout (seconds) |
+| `content_types` | html/xhtml/plain/markdown on; pdf off | Which MIME types to process |
+| `max_concurrent_requests` | `10` | Parallelism |
+| `url_blocked_patterns` | tool-local | Regex URL blocklist; **not** sent to the proxy |
+
+```python
+from unique_search_proxy_core.crawlers import CrawlerType
+from unique_web_search.services.crawlers import BasicConfig, get_crawler_service
+
+config = BasicConfig(crawler=CrawlerType.BASIC, timeout=30)
+markdowns = await get_crawler_service(config).crawl(["https://example.com"])
+```
+
+---
+
+## Tavily
+
+| Field | Default | Notes |
+|-------|---------|-------|
+| `extract_depth` | `advanced` | `basic` / `advanced` |
+| `format` | `markdown` | `markdown` / `text` |
+| `query` | unset | Optional rerank intent |
+| `chunks_per_source` | unset | Requires `query` |
+| `include_images` / `include_favicon` / `include_usage` | `false` | Response extras |
+
+---
+
+## Jina
+
+| Field | Default | Notes |
+|-------|---------|-------|
+| `return_format` | `markdown` | `markdown` / `html` / `text` / … |
+| `engine` | `browser` | `auto` / `browser` / `direct` / `cf-browser-rendering` |
+| `page_timeout` | unset | Seconds; falls back to crawl `timeout` |
+| `max_concurrent_requests` | `10` | |
+| `no_cache` | `false` | |
+| `do_not_track` | `true` | |
+| selectors / summaries | unset / false | `target_selector`, `wait_for_selector`, `remove_selector`, image/link options, `locale`, `referer`, `proxy_url` |
+
+---
+
+## Firecrawl
+
+| Field | Default | Notes |
+|-------|---------|-------|
+| `only_main_content` | `true` | Strip chrome before markdown |
+| `only_clean_content` | `false` | Extra LLM clean pass |
+| `max_concurrency` | unset | Batch parallelism |
+| `ignore_invalid_urls` | `true` | |
+| `wait_for` | `0` | ms delay before fetch |
+| `mobile` | `false` | |
+| `block_ads` | `true` | |
+| `remove_base64_images` | `true` | |
+| `proxy` | `auto` | Firecrawl proxy tier |
+| `include_tags` / `exclude_tags` / `scrape_headers` / `max_age` | unset | Optional |
+
+---
+
+## Adding a crawler
+
+1. Define config (+ request model) in `unique_search_proxy_core.crawlers.<name>`.
+2. Add a tool module with `@register_crawler(...)` implementing `_legacy_crawl`.
+3. Modules are auto-discovered; keep static unions in `__init__.py` and cover with `tests/test_crawler_registry.py`.

{unique_web_search-2026.30.0.dev3 → unique_web_search-2026.30.0.dev4}/src/unique_web_search/services/crawlers/base.py

@@ -3,6 +3,7 @@ from __future__ import annotations
 from abc import ABC, abstractmethod
 from typing import Generic, TypeVar
 
+from unique_search_proxy_core.context import LOCAL_REQUEST_CONTEXT, RequestContext
 from unique_search_proxy_core.crawlers.base import BaseCrawlerConfig
 from unique_search_proxy_core.url_safety import (
     CrawlTargetValidationError,
@@ -30,8 +31,14 @@ class BaseCrawler(ABC, Generic[CrawlerConfig]):
     the direct ``_legacy_crawl`` implementation used when the proxy is disabled.
     """
 
-    def __init__(self, config: CrawlerConfig):
+    def __init__(
+        self,
+        config: CrawlerConfig,
+        *,
+        request_context: RequestContext = LOCAL_REQUEST_CONTEXT,
+    ):
         self.config = config
+        self._request_context = request_context
 
     async def crawl(self, urls: list[str]) -> list[str]:
         if search_proxy_client_enabled:
@@ -49,7 +56,8 @@ class BaseCrawler(ABC, Generic[CrawlerConfig]):
         """Dump deployment config fields into the generic proxy crawl call."""
         params = self.config.model_dump(exclude={"crawler"}, exclude_none=True)
         async with open_search_proxy_client(
-            timeout=float(self.config.timeout)
+            timeout=float(self.config.timeout),
+            context=self._request_context,
         ) as client:
             response = await client.crawl.crawl(
                 urls=urls,

{unique_web_search-2026.30.0.dev3 → unique_web_search-2026.30.0.dev4}/src/unique_web_search/services/crawlers/basic.py

@@ -8,6 +8,7 @@ from httpx import AsyncClient, Timeout
 from markdownify import markdownify
 from pydantic import Field
 from typing_extensions import override
+from unique_search_proxy_core.context import LOCAL_REQUEST_CONTEXT, RequestContext
 from unique_search_proxy_core.crawlers.base import CrawlerType
 from unique_search_proxy_core.crawlers.basic.content_types import (
     CONTENT_TYPE_TOGGLE_TO_MIME,
@@ -54,8 +55,13 @@ class BasicConfig(CoreBasicConfig):
     config_display_name="Basic",
 )
 class BasicCrawler(BaseCrawler[BasicConfig]):
-    def __init__(self, config: BasicConfig):
-        super().__init__(config)
+    def __init__(
+        self,
+        config: BasicConfig,
+        *,
+        request_context: RequestContext = LOCAL_REQUEST_CONTEXT,
+    ):
+        super().__init__(config, request_context=request_context)
 
         self.semaphore: asyncio.Semaphore = asyncio.Semaphore(
             self.config.max_concurrent_requests
@@ -69,7 +75,8 @@ class BasicCrawler(BaseCrawler[BasicConfig]):
             exclude_none=True,
         )
         async with open_search_proxy_client(
-            timeout=float(self.config.timeout)
+            timeout=float(self.config.timeout),
+            context=self._request_context,
         ) as client:
             response = await client.crawl.crawl(
                 urls=urls,

{unique_web_search-2026.30.0.dev3 → unique_web_search-2026.30.0.dev4}/src/unique_web_search/services/crawlers/firecrawl.py

@@ -19,9 +19,6 @@ from unique_web_search.services.crawlers.url_safety import ResolvedCrawlTarget
     config_display_name="Firecrawl",
 )
 class FirecrawlCrawler(BaseCrawler[FirecrawlConfig]):
-    def __init__(self, config: FirecrawlConfig):
-        super().__init__(config)
-
     @override
     async def _legacy_crawl(self, targets: list[ResolvedCrawlTarget]) -> list[str]:
         urls = [target.normalized_url for target in targets]

{unique_web_search-2026.30.0.dev3 → unique_web_search-2026.30.0.dev4}/src/unique_web_search/services/crawlers/jina.py

@@ -39,9 +39,6 @@ class ReaderResponse(BaseModel):
     config_display_name="Jina",
 )
 class JinaCrawler(BaseCrawler[JinaConfig]):
-    def __init__(self, config: JinaConfig):
-        super().__init__(config)
-
     @override
     async def _legacy_crawl(self, targets: list[ResolvedCrawlTarget]) -> list[str]:
         urls = [target.normalized_url for target in targets]

{unique_web_search-2026.30.0.dev3 → unique_web_search-2026.30.0.dev4}/src/unique_web_search/services/crawlers/registry.py

@@ -6,6 +6,7 @@ from collections.abc import Callable
 from dataclasses import dataclass
 
 from pydantic import BaseModel
+from unique_search_proxy_core.context import LOCAL_REQUEST_CONTEXT, RequestContext
 from unique_search_proxy_core.crawlers.base import CrawlerType
 
 from unique_web_search.services._registry import BaseSpec, Registry
@@ -37,9 +38,16 @@ def register_crawler(
     )
 
 
-def get_crawler_service(crawler_config: object):
+def get_crawler_service(
+    crawler_config: object,
+    *,
+    request_context: RequestContext = LOCAL_REQUEST_CONTEXT,
+):
     crawler = getattr(crawler_config, "crawler")
-    return CRAWLER_REGISTRY[crawler].impl_cls(crawler_config)
+    return CRAWLER_REGISTRY[crawler].impl_cls(
+        crawler_config,
+        request_context=request_context,
+    )
 
 
 __all__ = [

{unique_web_search-2026.30.0.dev3 → unique_web_search-2026.30.0.dev4}/src/unique_web_search/services/crawlers/tavily.py

@@ -18,9 +18,6 @@ from unique_web_search.services.crawlers.url_safety import ResolvedCrawlTarget
     config_display_name="Tavily",
 )
 class TavilyCrawler(BaseCrawler[TavilyConfig]):
-    def __init__(self, config: TavilyConfig):
-        super().__init__(config)
-
     @override
     async def _legacy_crawl(self, targets: list[ResolvedCrawlTarget]) -> list[str]:
         urls = [target.normalized_url for target in targets]

{unique_web_search-2026.30.0.dev3 → unique_web_search-2026.30.0.dev4}/src/unique_web_search/services/executors/README.md

@@ -2,12 +2,14 @@
 
 Executors orchestrate the **end-to-end research workflow** once a query or plan has been validated. They sit between:
 
-- **Search engines** (Google, Bing, VertexAI, etc.)
-- **Crawlers** (Basic, Tavily, Crawl4AI, …)
+- **Search engines** (standard: Google, Brave, Perplexity, Custom API; agent/grounding: Bing, VertexAI)
+- **Crawlers** (Basic, Tavily, Jina, Firecrawl)
 - **Content processing** (summaries, truncation, relevancy sorters)
 - **LLM interfaces** (query refinement, structured outputs)
 - **Progress reporters / logging**
 
+**Search Proxy is transparent to executors:** when `SEARCH_PROXY_BASE_URL` is set, `SearchEngine.search` / `BaseCrawler.crawl` route via the proxy SDK; otherwise they use legacy direct calls. V1 / V2 / V3 do not branch on proxy mode.
+
 All executors extend the shared base class defined in `base_executor.py`.
 
 ```python
@@ -37,6 +39,7 @@ All steps push diagnostics into `WebSearchDebugInfo` for telemetry and debugging
 |----------|-----------|----------|----------------|-------------|-------|
 | **WebSearchV1Executor** | `WebSearchToolParameters` (single query + date restrictions) | Traditional search flows | LLM-based query refinement (Basic/Advanced) | Sequential (queries iterated) | Ideal for iterative refinement and single-shot queries |
 | **WebSearchV2Executor** | `WebSearchPlan` (multi-step plan) | Agentic / multi-step research | Executes explicit steps (search/read) | Parallel (steps scheduled concurrently) | Best for step-based plans generated by planning agents |
+| **WebSearchV3Executor** | Per-call `search` or `read_urls` | Iterative agent research | Model chooses next cheap search vs full read | One command per tool call | Experimental; loop bounded by orchestrator iteration cap |
 
 ---
 

{unique_web_search-2026.30.0.dev3 → unique_web_search-2026.30.0.dev4}/src/unique_web_search/services/proxy/bridge.py

@@ -3,6 +3,7 @@ from __future__ import annotations
 from collections.abc import AsyncIterator
 from contextlib import asynccontextmanager
 
+from unique_search_proxy_core.context import LOCAL_REQUEST_CONTEXT, RequestContext
 from unique_search_proxy_sdk import UniqueSearchProxyClient
 
 from unique_web_search.settings import env_settings
@@ -13,6 +14,7 @@ search_proxy_client_enabled = env_settings.search_proxy_base_url is not None
 @asynccontextmanager
 async def open_search_proxy_client(
     timeout: float,
+    context: RequestContext = LOCAL_REQUEST_CONTEXT,
 ) -> AsyncIterator[UniqueSearchProxyClient]:
     base_url = env_settings.search_proxy_base_url
     assert base_url is not None, "Unique Search Proxy base URL is not configured"
@@ -20,6 +22,7 @@ async def open_search_proxy_client(
     async with UniqueSearchProxyClient(
         base_url=base_url,
         timeout=timeout,
+        context=context,
     ) as client:
         yield client