kiapi 0.1.0__py3-none-any.whl

kiapi/capabilities/web/README.md

@@ -0,0 +1,171 @@
+# web
+
+**English** | [日本語](README.ja.md)
+
+**web capability** for using SearXNG and Crawl4AI from kiapi.
+
+- **search**: Search the web with SearXNG and return JSON search results.
+- **fetch**: Render a single URL with Crawl4AI and return Markdown or PDF.
+
+The web treats SearXNG / Crawl4AI as a **resident subprocess model**.
+Start `docker run --rm` as foreground subprocess on the first request,
+Allocate a free localhost port and reuse it as a resident backend from now on.
+search / fetch are also submitted to single-flight workers as Jobs.
+
+## API
+
+| Endpoint | Name | Description |
+|---|---|---|
+| `POST /v1/web/search` | Web search | Call SearXNG's `/search?format=json` and return the search results. |
+| `GET /v1/web/fetch?url=...` | Get page | Render an HTML page with Crawl4AI and return Markdown or PDF. |
+| `GET /v1/web/openapi.json` | OpenAPI | Returns detailed input/output specifications, usage, and TIPS. |
+
+## API Docs
+
+- [OpenAPI JSON](https://kiarina.github.io/kiapi/v1/web/openapi.json)
+- [Swagger UI](https://kiarina.github.io/kiapi/v1/web/docs.html)
+- [ReDoc](https://kiarina.github.io/kiapi/v1/web/redoc.html)
+
+## Dependencies
+
+| Backend | License | Description |
+|---|---|---|
+| [SearXNG](https://github.com/searxng/searxng) | AGPL-3.0 | Meta search engine. kiapi starts as `docker run --rm` subprocess. |
+| [Crawl4AI](https://github.com/unclecode/crawl4ai) | Apache-2.0 | Browser rendering + Markdown/PDF conversion. kiapi starts as `docker run --rm` subprocess. |
+
+## Notes
+
+- **resident subprocess model**:
+  SearXNG and Crawl4AI are registered as `ModelSpec(family="web")`. `load()` is
+  `docker run --rm` starts the subprocess in the foreground, and `release()` stops the subprocess.
+- **Job/single-flight**:
+  Both search/fetch are executed as a single-flight worker as a Job.
+- **search query**:
+  `query` is passed directly to SearXNG. `site:`, `!wp`, `!images`, `:ja` etc.
+  You can use SearXNG inline operators as is.
+- **search result count**:
+  There is no `max_results` in SearXNG itself, so after kiapi receives one page of results.
+  Truncate client-side. Default is `KIAPI_WEB_DEFAULT_MAX_RESULTS=10`.
+- **fetch target**:
+  `fetch` is only for HTML pages. Images, audio, videos, PDFs, and other binary URLs are
+  Do not return it as a byte string and reject it with `not_html`.
+- **fetch empty content protection**:
+  Even if you can reach it as HTML, there is no body that can be extracted after rendering.
+  Returns `empty_content`.
+- **fetch artifacts**:
+  The raw body of fetch is stored in the Files API, with `X-Kiapi-File-Id` in the response header.
+  It will have `X-Kiapi-Job-Id`.
+- **Main error**:
+  validation is 422, `unsupported_accept` for `fetch` is 406, backend unreachable or
+  HTTP error is 502, timeout is 504.
+- **Non-determinism**:
+  `search` relies on live search engine results, so the results will vary for the same query.
+
+##Settings
+
+| Variable | Default value | Meaning |
+|---|---|---|
+| `KIAPI_WEB_SEARCH_IMAGE` | `searxng/searxng:latest` | Docker image of SearXNG backend. |
+| `KIAPI_WEB_FETCH_IMAGE` | `unclecode/crawl4ai:latest` | Docker image of Crawl4AI backend. |
+| `KIAPI_WEB_BACKEND_READY_TIMEOUT_S` | `60.0` | Timeout waiting for backend subprocess to start. |
+| `KIAPI_WEB_BACKEND_LOG_DIR` | `/tmp/kiapi/logs/web` | stdout/stderr log output destination of backend subprocess. |
+| `KIAPI_WEB_TIMEOUT_S` | `10.0` | HTTP timeout to SearXNG. |
+| `KIAPI_WEB_DEFAULT_CATEGORIES` | `null` | `categories` Default if omitted. `null` is left to SearXNG. |
+| `KIAPI_WEB_DEFAULT_ENGINES` | `null` | `engines` Default if omitted. `null` is left to SearXNG. |
+| `KIAPI_WEB_DEFAULT_LANGUAGE` | `null` | `language` Default if omitted. |
+| `KIAPI_WEB_DEFAULT_SAFESEARCH` | `null` | `safesearch` Default when omitted. |
+| `KIAPI_WEB_DEFAULT_MAX_RESULTS` | `10` | `max_results` Default client-side truncation number. |
+| `KIAPI_WEB_FETCH_TIMEOUT_S` | `30.0` | HTTP timeout to Crawl4AI. |
+| `KIAPI_WEB_FETCH_MIN_CONTENT_CHARS` | `1` | Markdown less than this is considered `empty_content`. |
+| `KIAPI_WEB_FETCH_FILTER` | `fit` | Crawl4AI `/md` filter. `fit` is readability-pruned, `raw` is the entire DOM. |
+| `KIAPI_WEB_FETCH_CACHE` | `0` | Cache specification for Crawl4AI `/md`. |
+
+## Quickstart
+Start kiapi in a state where you can use `docker run`, such as Docker Desktop. SearXNG and
+Crawl4AI will be automatically started on the first request.
+
+### search — Web search
+```bash
+jq -n \
+'{
+  query: "apple",
+  max_results: 5
+}' |
+curl -sS -X POST http://localhost:${PORT:-8000}/v1/web/search \
+-H 'Content-Type: application/json' \
+--data-binary @- |
+jq .
+```
+Filter by category, engine, language, newness:
+```bash
+jq -n \
+'{
+  query: "mlx quantization",
+  categories: ["it"],
+  engines: ["google", "duckduckgo"],
+  language: "en",
+  time_range: "month",
+  max_results: 10
+}' |
+curl -sS -X POST http://localhost:${PORT:-8000}/v1/web/search \
+-H 'Content-Type: application/json' \
+--data-binary @- |
+jq .
+```
+Use SearXNG's inline operators:
+```bash
+jq -n \
+'{
+  query: "site:github.com kiapi"
+}' |
+curl -sS -X POST http://localhost:${PORT:-8000}/v1/web/search \
+-H 'Content-Type: application/json' \
+--data-binary @- |
+jq '.results[] | {title, url}'
+
+jq -n \
+'{
+  query: "!wp :ja Python"
+}' |
+curl -sS -X POST http://localhost:${PORT:-8000}/v1/web/search \
+-H 'Content-Type: application/json' \
+--data-binary @- |
+jq '.results[] | {title, url}'
+```
+The response includes `query`, `results`, `answers`, `infoboxes`, `suggestions`,
+Contains `unresponsive_engines`.
+
+### fetch — Markdown fetch
+```bash
+curl -sS -o page.md \
+"http://localhost:${PORT:-8000}/v1/web/fetch?url=https://github.com/kiarina"
+```
+`Accept: text/markdown` is the default and can be omitted. upstream detected by kiapi
+The content-type can be found in the `X-Kiapi-Content-Type` header.
+```bash
+curl -sS -D - -o page.md \
+"http://localhost:${PORT:-8000}/v1/web/fetch?url=https://github.com/kiarina" \
+| grep -i x-kiapi
+```
+### fetch — Get PDF
+```bash
+curl -sS -H 'Accept: application/pdf' -o page.pdf \
+"http://localhost:${PORT:-8000}/v1/web/fetch?url=https://github.com/kiarina"
+```
+### fetch — Reject non-HTML URLs
+```bash
+curl -sS \
+"http://localhost:${PORT:-8000}/v1/web/fetch?url=https://www-media.blazeworks.jp/content/icon/miineko.png" \
+| jq .
+```
+Example response:
+```json
+{
+  "detail": {
+    "error": "not_html",
+    "message": "Resource is not an HTML page (content-type: image/png).",
+    "url": "https://www-media.blazeworks.jp/content/icon/miineko.png",
+    "content_type": "image/png"
+  }
+}
+```

kiapi/capabilities/web/__init__.py

@@ -0,0 +1,30 @@
+"""web family — SearXNG search + Crawl4AI fetch as resident subprocess models.
+
+Endpoints: ``POST /v1/web/search`` (metasearch) and ``GET /v1/web/fetch``
+(render one page to markdown/PDF). Each backend is loaded as a foreground
+``docker run --rm`` subprocess and held resident by the global model cache.
+"""
+
+from ._exceptions.FetchError import FetchError
+from ._exceptions.SearchBackendError import SearchBackendError
+from ._helpers.register import register
+from ._operations.handle_fetch import handle_fetch
+from ._operations.handle_search import handle_search
+from ._settings import settings_manager
+from ._views.fetch_request import FetchRequest
+from ._views.fetch_result import FetchResult
+from ._views.search_request import SearchRequest
+from ._views.search_response import SearchResponse
+
+__all__ = [
+    "FetchError",  # ._exceptions
+    "FetchRequest",  # ._views
+    "FetchResult",  # ._views
+    "SearchBackendError",  # ._exceptions
+    "SearchRequest",  # ._views
+    "SearchResponse",  # ._views
+    "handle_fetch",  # ._operations
+    "handle_search",  # ._operations
+    "register",  # ._helpers
+    "settings_manager",  # ._settings
+]

kiapi/capabilities/web/_constants/description.py

@@ -0,0 +1,38 @@
+DESCRIPTION = """Web search + single-page fetch for LLM agents.
+
+Use this family when an agent needs current web information or needs to read the
+contents of a specific HTML page. Search returns structured JSON results; fetch
+renders one page and returns Markdown by default, or PDF when requested.
+
+## When To Use
+- Use search when you need discovery: recent facts, unknown URLs, multiple
+  candidate sources, or comparison across sources.
+- Use fetch when you already have a URL and need the page body in agent-friendly
+  Markdown or as a rendered PDF artifact.
+- Search and fetch are separate steps. Search result snippets are useful for
+  triage, but fetch is better when the exact page content matters.
+
+## Search Tips
+- Search results are live and may change between calls.
+- SearXNG inline operators are useful for agents: `site:`, `!wp`, `!images`,
+  `:ja`, and engine/category bangs can be placed directly in `query`.
+- Use `categories`, `engines`, `language`, `time_range`, and `safesearch` to
+  narrow the search when the task has a clear domain.
+- Use `max_results` to keep responses small. It truncates one returned page of
+  SearXNG results; it does not aggregate multiple pages.
+
+## Fetch Tips
+- Fetch is for HTML pages only. It intentionally rejects binary resources such
+  as images, videos, audio files, archives, and PDFs instead of returning their
+  raw bytes.
+- Markdown output is best for reading and summarization. PDF output is useful
+  when layout, pagination, or visual preservation matters.
+- Successful fetches are also stored in the Files API. Use the returned
+  `X-Kiapi-File-Id` when the artifact needs to be referenced later.
+
+## Common Failures
+- `not_html`: the URL points to a non-HTML resource.
+- `empty_content`: the page loaded but produced no extractable content.
+- `unsupported_accept`: fetch was called with an Accept header that allows
+  neither Markdown nor PDF.
+"""

kiapi/capabilities/web/_exceptions/FetchError.py

@@ -0,0 +1,24 @@
+class FetchError(RuntimeError):
+    """A fetch could not produce usable page content.
+
+    Carries a machine-readable ``code`` (``fetch_failed`` / ``not_html`` /
+    ``empty_content``), the HTTP ``status_code`` the API layer should surface,
+    and — when the preflight probe revealed it — the upstream resource's
+    ``content_type``, so the caller learns *why* (e.g. it was a ``image/png``).
+
+    Status mapping: 502/504 = backend unreachable/timed out (``fetch_failed``);
+    422 = reachable but not a usable HTML page (``not_html`` / ``empty_content``).
+    """
+
+    def __init__(
+        self,
+        code: str,
+        message: str,
+        *,
+        status_code: int,
+        content_type: str | None = None,
+    ) -> None:
+        super().__init__(message)
+        self.code = code
+        self.status_code = status_code
+        self.content_type = content_type

kiapi/capabilities/web/_exceptions/SearchBackendError.py

@@ -0,0 +1,11 @@
+class SearchBackendError(RuntimeError):
+    """The SearXNG backend was unreachable, timed out, or returned an error.
+
+    Carries the HTTP ``status_code`` the API layer should surface (502 for an
+    unreachable/erroring backend, 504 for a timeout). This is a backend fault,
+    not a client mistake — distinct from request validation (422).
+    """
+
+    def __init__(self, message: str, *, status_code: int) -> None:
+        super().__init__(message)
+        self.status_code = status_code

kiapi/capabilities/web/_helpers/register.py

@@ -0,0 +1,59 @@
+from kiapi.core.capability import CapabilitySpec, capability_spec_registry
+from kiapi.core.model import ModelSpec, model_registry
+from kiapi.core.setup import DockerImageResource
+
+from .._constants.description import DESCRIPTION
+from .._models import crawl4ai, searxng
+from .._settings import settings_manager
+
+
+def register() -> None:
+    capability_spec_registry.register(
+        CapabilitySpec(
+            name="web",
+            domain="web",
+            title="kiapi Web API",
+            summary="Search the web and fetch pages as Markdown or PDF artifacts.",
+            description=DESCRIPTION,
+            openapi_path="/v1/web/openapi.json",
+            docs_path="/v1/web/docs",
+            redoc_path="/v1/web/redoc",
+            path_prefixes=("/v1/web",),
+        )
+    )
+
+    settings = settings_manager.get_settings()
+
+    model_registry.register(
+        ModelSpec(
+            name="search",
+            family="web",
+            domain="web",
+            repo=settings.search_image,
+            module=searxng,
+            weight_gb=0.5,
+            peak_headroom_gb=0.2,
+            framework="rss",
+            priority=0,
+            default=True,
+            setup_resources=(
+                DockerImageResource(image=settings.search_image, disk_gb=0.4),
+            ),
+        )
+    )
+    model_registry.register(
+        ModelSpec(
+            name="fetch",
+            family="web",
+            domain="web",
+            repo=settings.fetch_image,
+            module=crawl4ai,
+            weight_gb=1.0,
+            peak_headroom_gb=1.0,
+            framework="rss",
+            priority=0,
+            setup_resources=(
+                DockerImageResource(image=settings.fetch_image, disk_gb=9.6),
+            ),
+        )
+    )

kiapi/capabilities/web/_models/crawl4ai.py

@@ -0,0 +1,32 @@
+"""Resident Crawl4AI backend, launched as a foreground Docker subprocess."""
+
+import httpx
+
+from kiapi.core.model import ModelSpec
+
+from .._services.web_backend_process import WebBackendProcess
+from .._settings import settings_manager
+
+FEATURES = {"fetch"}
+
+
+def load(spec: ModelSpec) -> WebBackendProcess:
+    settings = settings_manager.get_settings()
+    return WebBackendProcess.start(
+        name="crawl4ai",
+        image=spec.repo,
+        container_port=11235,
+        log_dir=settings.backend_log_dir,
+        ready_timeout_s=settings.backend_ready_timeout_s,
+        healthcheck=_healthcheck,
+        extra_args=["--shm-size", "1g"],
+    )
+
+
+def release(payload: WebBackendProcess) -> None:
+    payload.release()
+
+
+def _healthcheck(base_url: str) -> None:
+    resp = httpx.get(f"{base_url.rstrip('/')}/health", timeout=5.0)
+    resp.raise_for_status()

kiapi/capabilities/web/_models/searxng.py

@@ -0,0 +1,44 @@
+"""Resident SearXNG backend, launched as a foreground Docker subprocess."""
+
+import httpx
+
+from kiapi.core.model import ModelSpec
+
+from .._operations.resolve_searxng_config_dir import resolve_searxng_config_dir
+from .._services.web_backend_process import WebBackendProcess
+from .._settings import settings_manager
+
+FEATURES = {"search"}
+
+
+def load(spec: ModelSpec) -> WebBackendProcess:
+    settings = settings_manager.get_settings()
+    searxng_dir = resolve_searxng_config_dir()
+    return WebBackendProcess.start(
+        name="searxng",
+        image=spec.repo,
+        container_port=8080,
+        log_dir=settings.backend_log_dir,
+        ready_timeout_s=settings.backend_ready_timeout_s,
+        healthcheck=_healthcheck,
+        extra_args=[
+            "-v",
+            f"{searxng_dir}:/etc/searxng:rw",
+            "-e",
+            "SEARXNG_BASE_URL=http://localhost/",
+        ],
+    )
+
+
+def release(payload: WebBackendProcess) -> None:
+    payload.release()
+
+
+def _healthcheck(base_url: str) -> None:
+    resp = httpx.get(
+        f"{base_url.rstrip('/')}/search",
+        params={"q": "kiapi", "format": "json"},
+        timeout=5.0,
+    )
+    resp.raise_for_status()
+    resp.json()

kiapi/capabilities/web/_operations/handle_fetch.py

@@ -0,0 +1,145 @@
+"""web fetch service: render one page to readable markdown (or PDF) via Crawl4AI.
+
+Flow:
+
+1. **Preflight probe** (cheap ``HEAD``) → reachability + the upstream MIME. A
+   connection failure here is ``fetch_failed``; a non-HTML MIME is rejected as
+   ``not_html`` *before* paying for a browser render.
+2. **Render** through Crawl4AI ``/md`` (``f=fit`` -> readability-pruned
+   fit_markdown) or ``/pdf``.
+3. **Empty guard** → a render that produced no content is ``empty_content``.
+"""
+
+import base64
+
+import httpx
+
+from kiapi.core.app import AppContext
+from kiapi.core.file import FileID
+from kiapi.core.job import JobResult
+from kiapi.core.model import model_registry
+
+from .._exceptions.FetchError import FetchError
+from .._settings import settings_manager
+from .._views.fetch_params import FetchParams
+from .._views.fetch_request import FetchRequest
+from .._views.fetch_result import FetchResult
+from .resolve_fetch_params import resolve_fetch_params
+
+# Content types fetch treats as a renderable page. Anything else (image/audio/
+# video/pdf/octet-stream/...) carries no extractable text and is rejected.
+_HTML_TYPES = frozenset({"text/html", "application/xhtml+xml"})
+
+
+def _probe_content_type(params: FetchParams) -> str | None:
+    """HEAD the target for reachability + MIME; None if the server omits it."""
+    try:
+        with httpx.Client(timeout=params.timeout_s, follow_redirects=True) as client:
+            resp = client.head(params.url)
+    except httpx.HTTPError as exc:
+        raise FetchError(
+            "fetch_failed",
+            f"Could not reach {params.url}: {exc}",
+            status_code=502,
+        ) from exc
+
+    ct = resp.headers.get("content-type")
+    return ct.split(";")[0].strip().lower() if ct else None
+
+
+def _post_crawl4ai(params: FetchParams, path: str, payload: dict) -> dict:
+    url = f"{params.base_url.rstrip('/')}{path}"
+    try:
+        with httpx.Client(timeout=params.timeout_s) as client:
+            resp = client.post(url, json=payload)
+            resp.raise_for_status()
+            data: dict = resp.json()
+            return data
+    except httpx.TimeoutException as exc:
+        raise FetchError(
+            "fetch_failed",
+            f"Crawl4AI timed out after {params.timeout_s}s",
+            status_code=504,
+        ) from exc
+    except httpx.HTTPStatusError as exc:
+        raise FetchError(
+            "fetch_failed",
+            f"Crawl4AI returned HTTP {exc.response.status_code}",
+            status_code=502,
+        ) from exc
+    except httpx.HTTPError as exc:
+        raise FetchError(
+            "fetch_failed",
+            f"Crawl4AI unreachable at {params.base_url}: {exc}",
+            status_code=502,
+        ) from exc
+
+
+def _render(params: FetchParams) -> tuple[bytes, str, bool]:
+    """Return (body, media_type, is_empty) for the requested format."""
+    if params.format == "pdf":
+        data = _post_crawl4ai(params, "/pdf", {"url": params.url})
+        b64 = data.get("pdf") or ""
+        body = base64.b64decode(b64) if b64 else b""
+        return body, "application/pdf", len(body) == 0
+
+    payload = {"url": params.url, "f": params.filter, "c": params.cache}
+    data = _post_crawl4ai(params, "/md", payload)
+    markdown = data.get("markdown") or ""
+    is_empty = len(markdown.strip()) < params.min_content_chars
+    return markdown.encode("utf-8"), "text/markdown; charset=utf-8", is_empty
+
+
+def handle_fetch(ctx: AppContext, req: FetchRequest) -> tuple[JobResult, list[FileID]]:
+    settings = settings_manager.get_settings()
+    spec = model_registry.resolve("web", "fetch")
+    ctx.ensure_model_ready(spec)
+    backend = ctx.memory_manager.acquire(spec)
+    params = resolve_fetch_params(
+        settings,
+        req,
+        base_url=backend.base_url,  # type: ignore[attr-defined]
+    )
+
+    content_type = _probe_content_type(params)
+    if content_type is not None and content_type not in _HTML_TYPES:
+        raise FetchError(
+            "not_html",
+            f"Resource is not an HTML page (content-type: {content_type}).",
+            status_code=422,
+            content_type=content_type,
+        )
+
+    body, media_type, is_empty = _render(params)
+
+    if is_empty:
+        raise FetchError(
+            "empty_content",
+            "The page was fetched but produced no extractable content.",
+            status_code=422,
+            content_type=content_type,
+        )
+
+    result = FetchResult(
+        content=body,
+        media_type=media_type,
+        url=params.url,
+        content_type=content_type,
+    )
+    suffix = ".pdf" if req.format == "pdf" else ".md"
+    rec = ctx.file_store.put_bytes(
+        result.content,
+        filename=f"fetch{suffix}",
+        content_type=result.media_type,
+        meta={"url": result.url, "content_type": result.content_type},
+    )
+    return (
+        {
+            "file_id": rec.file_id,
+            "url": result.url,
+            "content_type": result.content_type,
+            "media_type": result.media_type,
+            "bytes": rec.size,
+        },
+        [rec.file_id],
+    )

kiapi/capabilities/web/_operations/handle_search.py

@@ -0,0 +1,56 @@
+"""web search service entry, run on the single worker thread."""
+
+import httpx
+
+from kiapi.core.app import AppContext
+from kiapi.core.file import FileID
+from kiapi.core.job import JobResult
+from kiapi.core.model import model_registry
+
+from .._exceptions.SearchBackendError import SearchBackendError
+from .._settings import settings_manager
+from .._views.search_request import SearchRequest
+from .._views.search_response import SearchResponse
+from .resolve_search_params import resolve_search_params
+
+
+def handle_search(
+    ctx: AppContext, req: SearchRequest
+) -> tuple[JobResult, list[FileID]]:
+    settings = settings_manager.get_settings()
+    spec = model_registry.resolve("web", "search")
+    ctx.ensure_model_ready(spec)
+    backend = ctx.memory_manager.acquire(spec)
+    params = resolve_search_params(settings, req)
+
+    url = f"{backend.base_url.rstrip('/')}/search"  # type: ignore[attr-defined]
+    try:
+        with httpx.Client(timeout=settings.timeout_s) as client:
+            resp = client.get(url, params=params.query_params())
+            resp.raise_for_status()
+            data = resp.json()
+    except httpx.TimeoutException as exc:
+        raise SearchBackendError(
+            f"SearXNG timed out after {settings.timeout_s}s", status_code=504
+        ) from exc
+    except httpx.HTTPStatusError as exc:
+        raise SearchBackendError(
+            f"SearXNG returned HTTP {exc.response.status_code}", status_code=502
+        ) from exc
+    except httpx.HTTPError as exc:
+        raise SearchBackendError(
+            f"SearXNG unreachable at {url}: {exc}", status_code=502
+        ) from exc
+
+    results = data.get("results", [])
+    if params.max_results is not None:
+        results = results[: params.max_results]
+
+    return SearchResponse(
+        query=data.get("query", req.query),
+        results=results,
+        answers=data.get("answers", []),
+        infoboxes=data.get("infoboxes", []),
+        suggestions=data.get("suggestions", []),
+        unresponsive_engines=data.get("unresponsive_engines", []),
+    ).model_dump(), []

kiapi/capabilities/web/_operations/resolve_fetch_params.py

@@ -0,0 +1,19 @@
+"""Merge a fetch request with settings defaults into the complete FetchParams."""
+
+from .._settings import WebSettings
+from .._views.fetch_params import FetchParams
+from .._views.fetch_request import FetchRequest
+
+
+def resolve_fetch_params(
+    settings: WebSettings, req: FetchRequest, *, base_url: str
+) -> FetchParams:
+    return FetchParams(
+        url=req.url,
+        format=req.format,
+        base_url=base_url,
+        timeout_s=settings.fetch_timeout_s,
+        filter=settings.fetch_filter,
+        cache=settings.fetch_cache,
+        min_content_chars=settings.fetch_min_content_chars,
+    )

kiapi/capabilities/web/_operations/resolve_search_params.py

@@ -0,0 +1,34 @@
+"""Merge a search request with settings defaults into the complete SearchParams."""
+
+from .._settings import WebSettings
+from .._views.search_params import SearchParams
+from .._views.search_request import SearchRequest
+
+
+def _join(values: list[str] | None) -> str | None:
+    """SearXNG takes categories/engines as a comma-separated string."""
+    return ",".join(values) if values else None
+
+
+def resolve_search_params(settings: WebSettings, req: SearchRequest) -> SearchParams:
+    categories = (
+        req.categories if req.categories is not None else settings.default_categories
+    )
+    engines = req.engines if req.engines is not None else settings.default_engines
+
+    return SearchParams(
+        query=req.query,
+        categories=_join(categories),
+        engines=_join(engines),
+        language=req.language
+        if req.language is not None
+        else settings.default_language,
+        time_range=req.time_range,
+        safesearch=req.safesearch
+        if req.safesearch is not None
+        else settings.default_safesearch,
+        pageno=req.page,
+        max_results=req.max_results
+        if req.max_results is not None
+        else settings.default_max_results,
+    )