crewai-alterlab 0.1.0__tar.gz

crewai_alterlab-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 AlterLab
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

crewai_alterlab-0.1.0/PKG-INFO

@@ -0,0 +1,151 @@
+Metadata-Version: 2.4
+Name: crewai-alterlab
+Version: 0.1.0
+Summary: CrewAI integration for AlterLab web scraping API
+License: MIT
+License-File: LICENSE
+Keywords: crewai,alterlab,web-scraping,ai-agents,web-crawler,scraping-tool
+Author: AlterLab
+Author-email: dev@alterlab.io
+Requires-Python: >=3.10,<4.0
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Programming Language :: Python :: 3.15
+Classifier: Topic :: Internet :: WWW/HTTP :: Indexing/Search
+Classifier: Topic :: Software Development :: Libraries
+Requires-Dist: alterlab (>=2.4.0)
+Requires-Dist: crewai[tools] (>=0.80.0)
+Project-URL: Documentation, https://docs.alterlab.io/integrations/crewai
+Project-URL: Homepage, https://alterlab.io
+Project-URL: Repository, https://github.com/RapierCraftStudios/AlterLab
+Description-Content-Type: text/markdown
+
+# crewai-alterlab
+
+CrewAI integration for the [AlterLab](https://alterlab.io) web scraping API.
+
+## Installation
+
+```bash
+pip install crewai-alterlab
+```
+
+## Setup
+
+Set your AlterLab API key:
+
+```bash
+export ALTERLAB_API_KEY="your-api-key"
+```
+
+Get your API key at [alterlab.io/dashboard](https://alterlab.io/dashboard).
+
+## Scrape Tool
+
+Scrape individual web pages with `AlterLabScrapeTool`:
+
+```python
+from crewai_alterlab import AlterLabScrapeTool
+
+tool = AlterLabScrapeTool()
+result = tool.run("https://example.com")
+print(result)
+```
+
+### With a CrewAI Agent
+
+```python
+from crewai import Agent, Task, Crew
+from crewai_alterlab import AlterLabScrapeTool
+
+researcher = Agent(
+    role="Web Researcher",
+    goal="Research topics by scraping web pages",
+    backstory="You are an expert web researcher.",
+    tools=[AlterLabScrapeTool()],
+)
+
+task = Task(
+    description="Scrape https://example.com and summarise the content",
+    expected_output="A summary of the page content",
+    agent=researcher,
+)
+
+crew = Crew(agents=[researcher], tasks=[task])
+result = crew.kickoff()
+```
+
+### Custom Parameters
+
+```python
+tool = AlterLabScrapeTool(
+    scrape_params={
+        "formats": ["markdown"],
+        "mode": "js",  # JavaScript rendering
+    },
+)
+```
+
+## Crawl Tool
+
+Crawl multiple pages from a website with `AlterLabCrawlTool`:
+
+```python
+from crewai_alterlab import AlterLabCrawlTool
+
+tool = AlterLabCrawlTool()
+result = tool.run(url="https://docs.example.com", max_pages=5)
+print(result)
+```
+
+### Research Agent with Both Tools
+
+```python
+from crewai import Agent
+from crewai_alterlab import AlterLabScrapeTool, AlterLabCrawlTool
+
+researcher = Agent(
+    role="Deep Researcher",
+    goal="Thoroughly research topics using web data",
+    backstory="You gather comprehensive information from the web.",
+    tools=[AlterLabScrapeTool(), AlterLabCrawlTool()],
+)
+```
+
+## API Reference
+
+See [docs.alterlab.io](https://docs.alterlab.io) for the full AlterLab API documentation.
+
+## Search Tool
+
+Run web searches with `AlterLabSearchTool` (numbered title, URL and snippet):
+
+```python
+from crewai_alterlab import AlterLabSearchTool
+
+tool = AlterLabSearchTool(num_results=5, search_params={"engine": "google"})
+print(tool.run(query="web scraping api"))
+```
+
+## Blocked-Fetch Fallback
+
+`BlockedFetchFallbackTool` runs your own fetch first and retries through AlterLab only when
+the result is a 403/429, a bot-protection challenge page, a CAPTCHA, or an empty JavaScript
+shell. Normal pages, 404s and network errors are returned or raised untouched, so credits are
+spent only on blocked fetches:
+
+```python
+import requests
+from crewai_alterlab import BlockedFetchFallbackTool
+
+tool = BlockedFetchFallbackTool(fetch=lambda url: requests.get(url, timeout=20))
+print(tool.run(url="https://example.com"))
+```
+

crewai_alterlab-0.1.0/README.md

@@ -0,0 +1,121 @@
+# crewai-alterlab
+
+CrewAI integration for the [AlterLab](https://alterlab.io) web scraping API.
+
+## Installation
+
+```bash
+pip install crewai-alterlab
+```
+
+## Setup
+
+Set your AlterLab API key:
+
+```bash
+export ALTERLAB_API_KEY="your-api-key"
+```
+
+Get your API key at [alterlab.io/dashboard](https://alterlab.io/dashboard).
+
+## Scrape Tool
+
+Scrape individual web pages with `AlterLabScrapeTool`:
+
+```python
+from crewai_alterlab import AlterLabScrapeTool
+
+tool = AlterLabScrapeTool()
+result = tool.run("https://example.com")
+print(result)
+```
+
+### With a CrewAI Agent
+
+```python
+from crewai import Agent, Task, Crew
+from crewai_alterlab import AlterLabScrapeTool
+
+researcher = Agent(
+    role="Web Researcher",
+    goal="Research topics by scraping web pages",
+    backstory="You are an expert web researcher.",
+    tools=[AlterLabScrapeTool()],
+)
+
+task = Task(
+    description="Scrape https://example.com and summarise the content",
+    expected_output="A summary of the page content",
+    agent=researcher,
+)
+
+crew = Crew(agents=[researcher], tasks=[task])
+result = crew.kickoff()
+```
+
+### Custom Parameters
+
+```python
+tool = AlterLabScrapeTool(
+    scrape_params={
+        "formats": ["markdown"],
+        "mode": "js",  # JavaScript rendering
+    },
+)
+```
+
+## Crawl Tool
+
+Crawl multiple pages from a website with `AlterLabCrawlTool`:
+
+```python
+from crewai_alterlab import AlterLabCrawlTool
+
+tool = AlterLabCrawlTool()
+result = tool.run(url="https://docs.example.com", max_pages=5)
+print(result)
+```
+
+### Research Agent with Both Tools
+
+```python
+from crewai import Agent
+from crewai_alterlab import AlterLabScrapeTool, AlterLabCrawlTool
+
+researcher = Agent(
+    role="Deep Researcher",
+    goal="Thoroughly research topics using web data",
+    backstory="You gather comprehensive information from the web.",
+    tools=[AlterLabScrapeTool(), AlterLabCrawlTool()],
+)
+```
+
+## API Reference
+
+See [docs.alterlab.io](https://docs.alterlab.io) for the full AlterLab API documentation.
+
+## Search Tool
+
+Run web searches with `AlterLabSearchTool` (numbered title, URL and snippet):
+
+```python
+from crewai_alterlab import AlterLabSearchTool
+
+tool = AlterLabSearchTool(num_results=5, search_params={"engine": "google"})
+print(tool.run(query="web scraping api"))
+```
+
+## Blocked-Fetch Fallback
+
+`BlockedFetchFallbackTool` runs your own fetch first and retries through AlterLab only when
+the result is a 403/429, a bot-protection challenge page, a CAPTCHA, or an empty JavaScript
+shell. Normal pages, 404s and network errors are returned or raised untouched, so credits are
+spent only on blocked fetches:
+
+```python
+import requests
+from crewai_alterlab import BlockedFetchFallbackTool
+
+tool = BlockedFetchFallbackTool(fetch=lambda url: requests.get(url, timeout=20))
+print(tool.run(url="https://example.com"))
+```

crewai_alterlab-0.1.0/crewai_alterlab/__init__.py

@@ -0,0 +1,34 @@
+"""CrewAI integration for the AlterLab web scraping API.
+
+Provides:
+- ``AlterLabScrapeTool``: Tool for CrewAI agents to scrape individual web pages.
+- ``AlterLabCrawlTool``: Tool for CrewAI agents to crawl websites.
+- ``AlterLabSearchTool``: Tool for CrewAI agents to run web searches.
+
+Install::
+
+    pip install crewai-alterlab
+
+Quick start::
+
+    from crewai_alterlab import AlterLabScrapeTool
+
+    tool = AlterLabScrapeTool()
+    result = tool.run("https://example.com")
+    print(result)
+"""
+
+from crewai_alterlab.fallback import AlterLabFallbackTool as BlockedFetchFallbackTool
+from crewai_alterlab.tools import (
+    AlterLabCrawlTool,
+    AlterLabScrapeTool,
+    AlterLabSearchTool,
+)
+
+__version__ = "0.1.0"
+__all__ = [
+    "BlockedFetchFallbackTool",
+    "AlterLabScrapeTool",
+    "AlterLabCrawlTool",
+    "AlterLabSearchTool",
+]

crewai_alterlab-0.1.0/crewai_alterlab/fallback.py

@@ -0,0 +1,115 @@
+"""Blocked-fetch fallback tool for CrewAI agents.
+
+``AlterLabFallbackTool`` tries the caller's own fetch first and retries the URL through
+AlterLab only when the response carries a block signal: an HTTP 403/429, a bot-protection
+challenge page, a CAPTCHA, or an empty JavaScript shell. Any other outcome (a normal page,
+a 404, a network error) is returned or raised untouched, so AlterLab credits are spent only
+when the plain fetch was actually blocked.
+
+Usage::
+
+    import requests
+    from crewai_alterlab import BlockedFetchFallbackTool
+
+    tool = BlockedFetchFallbackTool(fetch=lambda url: requests.get(url, timeout=20))
+    tool.run(url="https://example.com")
+"""
+
+import logging
+import re
+from typing import Any, Callable, Optional, Type
+
+from crewai.tools import BaseTool
+from pydantic import BaseModel, Field
+
+logger = logging.getLogger(__name__)
+
+BLOCK_STATUS_CODES = frozenset({403, 429})
+
+_CHALLENGE_MARKERS = re.compile(
+    r"just a moment|checking your browser|verify you are (?:a )?human|attention required"
+    r"|access denied|captcha|are you a robot|unusual traffic|enable javascript and cookies"
+    r"|px-captcha|challenge-platform|_incapsula_|datadome",
+    re.IGNORECASE,
+)
+_SCRIPT_OR_STYLE = re.compile(
+    r"<(script|style|noscript)\b.*?</\1>", re.IGNORECASE | re.DOTALL
+)
+_TAGS = re.compile(r"<[^>]+>")
+_JS_REQUIRED = re.compile(r"enable javascript|requires javascript", re.IGNORECASE)
+_EMPTY_SHELL_MAX_CHARS = 80
+
+
+def _visible_text(html: str) -> str:
+    without_blocks = _SCRIPT_OR_STYLE.sub(" ", html)
+    return " ".join(_TAGS.sub(" ", without_blocks).split())
+
+
+def is_blocked(status_code: Optional[int], body: str) -> bool:
+    """Return True when a plain fetch result looks blocked rather than served."""
+    if status_code in BLOCK_STATUS_CODES:
+        return True
+    if status_code is not None and not 200 <= status_code < 300:
+        return False
+    if _CHALLENGE_MARKERS.search(body):
+        return True
+    looks_like_html = (
+        "<html" in body.lower() or "<body" in body.lower() or "<div" in body.lower()
+    )
+    if looks_like_html:
+        if len(_visible_text(body)) <= _EMPTY_SHELL_MAX_CHARS:
+            return True
+        return False
+    if _JS_REQUIRED.search(body):
+        return len(body.strip()) <= _EMPTY_SHELL_MAX_CHARS * 4
+    return not body.strip()
+
+
+def _unpack(result: Any) -> "tuple[Optional[int], str]":
+    """Accept a str, a ``(status, text)`` tuple, or a requests/httpx-style response."""
+    if isinstance(result, str):
+        return None, result
+    if isinstance(result, tuple) and len(result) == 2:
+        status, text = result
+        return status, text or ""
+    status = getattr(result, "status_code", None)
+    text = getattr(result, "text", "")
+    return status, text if isinstance(text, str) else str(text or "")
+
+
+class _FallbackInput(BaseModel):
+    url: str = Field(description="The URL of the web page to read.")
+
+
+class AlterLabFallbackTool(BaseTool):
+    """Fetch a URL with the caller's fetch; retry via AlterLab only if it was blocked."""
+
+    name: str = "AlterLab Blocked Fetch Fallback"
+    description: str = (
+        "Read a web page. Tries a normal fetch first and automatically retries through "
+        "AlterLab when that fetch returns 403/429, a bot-protection challenge page, a "
+        "CAPTCHA, or an empty JavaScript shell. Input is a URL; returns the page content."
+    )
+    args_schema: Type[BaseModel] = _FallbackInput
+
+    fetch: Callable[[str], Any]
+    """Caller's fetch: returns a str, a ``(status_code, text)`` tuple, or a response object."""
+
+    fallback: Optional[BaseTool] = None
+    """Tool used on a block signal. Defaults to ``AlterLabScrapeTool()``."""
+
+    def _fallback_tool(self) -> BaseTool:
+        if self.fallback is None:
+            from crewai_alterlab.tools import AlterLabScrapeTool
+
+            self.fallback = AlterLabScrapeTool()
+        return self.fallback
+
+    def _run(self, url: str) -> str:
+        status, body = _unpack(self.fetch(url))
+        if not is_blocked(status, body):
+            return body
+        logger.info(
+            "Blocked fetch for %s (status=%s); retrying via AlterLab", url, status
+        )
+        return self._fallback_tool().run(url=url)

crewai_alterlab-0.1.0/crewai_alterlab/tools.py

@@ -0,0 +1,330 @@
+"""AlterLab tools for CrewAI agents.
+
+Provides ``AlterLabScrapeTool`` and ``AlterLabCrawlTool`` — CrewAI-compatible
+tools that allow agents to scrape and crawl web pages via the AlterLab API.
+
+Usage::
+
+    from crewai_alterlab import AlterLabScrapeTool
+
+    tool = AlterLabScrapeTool()
+    # Use with a CrewAI agent
+"""
+
+import logging
+from typing import Any, Dict, Optional, Type
+
+from crewai.tools import BaseTool
+from pydantic import BaseModel, Field
+
+from crewai_alterlab.utilities import (
+    extract_page_content,
+    get_api_key,
+    get_api_url,
+)
+
+logger = logging.getLogger(__name__)
+
+
+class _ScrapeInput(BaseModel):
+    """Input schema for the AlterLab scrape tool."""
+
+    url: str = Field(description="The URL of the web page to scrape.")
+
+
+class AlterLabScrapeTool(BaseTool):
+    """CrewAI tool that scrapes a web page using the AlterLab API.
+
+    Returns the page content as a string (markdown preferred). Designed for
+    use with CrewAI agents that need to fetch and read web page content.
+
+    Setup:
+        Install the package and set the ``ALTERLAB_API_KEY`` environment
+        variable::
+
+            pip install crewai-alterlab
+            export ALTERLAB_API_KEY="your-api-key"
+
+    Usage:
+        .. code-block:: python
+
+            from crewai_alterlab import AlterLabScrapeTool
+
+            tool = AlterLabScrapeTool()
+            result = tool.run("https://example.com")
+            print(result)
+
+    With a CrewAI agent:
+        .. code-block:: python
+
+            from crewai import Agent
+            from crewai_alterlab import AlterLabScrapeTool
+
+            researcher = Agent(
+                role="Web Researcher",
+                goal="Research topics by scraping web pages",
+                tools=[AlterLabScrapeTool()],
+            )
+    """
+
+    name: str = "AlterLab Scrape"
+    description: str = (
+        "Scrape a web page and return its content as text. "
+        "Input is a URL string. Returns the page content as markdown text. "
+        "Useful for reading web pages, articles, documentation, or any "
+        "publicly accessible URL."
+    )
+    args_schema: Type[BaseModel] = _ScrapeInput
+
+    api_key: Optional[str] = None
+    """AlterLab API key. Defaults to ALTERLAB_API_KEY env var."""
+
+    api_url: Optional[str] = None
+    """AlterLab API base URL. Defaults to https://api.alterlab.io."""
+
+    scrape_params: Dict[str, Any] = Field(default_factory=dict)
+    """Additional parameters passed to the AlterLab scrape API."""
+
+    def __init__(self, **kwargs: Any) -> None:
+        """Initialise AlterLabScrapeTool.
+
+        Validates the API key is available at construction time.
+
+        Raises:
+            ImportError: If the ``alterlab`` package is not installed.
+            ValueError: If no API key can be resolved.
+        """
+        try:
+            import alterlab  # noqa: F401
+        except ImportError:
+            raise ImportError(
+                "Could not import alterlab python package. "
+                "Please install it with `pip install alterlab`."
+            )
+        super().__init__(**kwargs)
+        # Validate key is resolvable (raises ValueError if not)
+        get_api_key(self.api_key)
+
+    def _run(self, url: str) -> str:
+        """Scrape a URL and return its content.
+
+        Args:
+            url: The URL to scrape.
+
+        Returns:
+            Page content as a string, or an error message.
+        """
+        from alterlab import AlterLabSync
+
+        resolved_key = get_api_key(self.api_key)
+        resolved_url = get_api_url(self.api_url)
+
+        client = AlterLabSync(api_key=resolved_key, base_url=resolved_url)
+        try:
+            result = client.scrape(url, **self.scrape_params)
+            content = extract_page_content(result)
+            if not content:
+                return f"No content could be extracted from {url}"
+            return content
+        except Exception as e:
+            logger.warning("AlterLab scrape failed for %s: %s", url, e)
+            return f"Error scraping {url}: {e}"
+        finally:
+            client.close()
+
+
+class _CrawlInput(BaseModel):
+    """Input schema for the AlterLab crawl tool."""
+
+    url: str = Field(
+        description="The starting URL for the crawl."
+    )
+    max_pages: int = Field(
+        default=10,
+        description="Maximum number of pages to crawl.",
+    )
+
+
+class AlterLabCrawlTool(BaseTool):
+    """CrewAI tool that crawls a website using the AlterLab API.
+
+    Crawls multiple pages starting from a URL and returns their combined
+    content. Useful for gathering comprehensive information from a website.
+
+    Setup:
+        Install the package and set the ``ALTERLAB_API_KEY`` environment
+        variable::
+
+            pip install crewai-alterlab
+            export ALTERLAB_API_KEY="your-api-key"
+
+    Usage:
+        .. code-block:: python
+
+            from crewai_alterlab import AlterLabCrawlTool
+
+            tool = AlterLabCrawlTool()
+            result = tool.run(url="https://docs.example.com", max_pages=5)
+            print(result)
+    """
+
+    name: str = "AlterLab Crawl"
+    description: str = (
+        "Crawl a website starting from a URL and return content from "
+        "multiple pages. Input requires a URL and optional max_pages "
+        "(default 10). Useful for gathering comprehensive information "
+        "from documentation sites, blogs, or multi-page resources."
+    )
+    args_schema: Type[BaseModel] = _CrawlInput
+
+    api_key: Optional[str] = None
+    """AlterLab API key. Defaults to ALTERLAB_API_KEY env var."""
+
+    api_url: Optional[str] = None
+    """AlterLab API base URL. Defaults to https://api.alterlab.io."""
+
+    def __init__(self, **kwargs: Any) -> None:
+        """Initialise AlterLabCrawlTool.
+
+        Raises:
+            ImportError: If the ``alterlab`` package is not installed.
+            ValueError: If no API key can be resolved.
+        """
+        try:
+            import alterlab  # noqa: F401
+        except ImportError:
+            raise ImportError(
+                "Could not import alterlab python package. "
+                "Please install it with `pip install alterlab`."
+            )
+        super().__init__(**kwargs)
+        get_api_key(self.api_key)
+
+    def _run(self, url: str, max_pages: int = 10) -> str:
+        """Crawl a website and return combined content.
+
+        Args:
+            url: The starting URL for the crawl.
+            max_pages: Maximum number of pages to crawl.
+
+        Returns:
+            Combined page content as a string, or an error message.
+        """
+        from alterlab import AlterLabSync
+
+        resolved_key = get_api_key(self.api_key)
+        resolved_url = get_api_url(self.api_url)
+
+        client = AlterLabSync(api_key=resolved_key, base_url=resolved_url)
+        try:
+            result = client.crawl(url, max_pages=max_pages)
+
+            pages = result.get("pages", [])
+            if not pages:
+                return f"No pages could be crawled from {url}"
+
+            parts = []
+            for page in pages:
+                page_url = page.get("url", "unknown")
+                content = extract_page_content(page)
+                if content:
+                    parts.append(f"## {page_url}\n\n{content}")
+
+            if not parts:
+                return f"No content could be extracted from crawled pages at {url}"
+
+            return "\n\n---\n\n".join(parts)
+        except Exception as e:
+            logger.warning("AlterLab crawl failed for %s: %s", url, e)
+            return f"Error crawling {url}: {e}"
+        finally:
+            client.close()
+
+
+def _format_search_results(result: Dict[str, Any], query: str) -> str:
+    """Format a SERP response as numbered ``title / url / snippet`` text."""
+    items = result.get("results") or []
+    if not items:
+        return f"No search results found for {query!r}"
+    lines = []
+    for i, item in enumerate(items, start=1):
+        pos = item.get("position") or i
+        lines.append(f"{pos}. {item.get('title', '')}\n   {item.get('url', '')}")
+        if item.get("snippet"):
+            lines.append(f"   {item['snippet']}")
+    return "\n".join(lines)
+
+
+class _SearchInput(BaseModel):
+    """Input schema for the AlterLab search tool."""
+
+    query: str = Field(description="The search query (max 500 characters).")
+
+
+class AlterLabSearchTool(BaseTool):
+    """CrewAI tool that runs a web search using the AlterLab Search API.
+
+    Returns numbered results (title, URL, snippet) as a string.
+
+    Usage:
+        .. code-block:: python
+
+            from crewai_alterlab import AlterLabSearchTool
+
+            tool = AlterLabSearchTool(num_results=5)
+            print(tool.run(query="web scraping api"))
+    """
+
+    name: str = "AlterLab Search"
+    description: str = (
+        "Search the web and return a numbered list of results with title, "
+        "URL and snippet. Input is a search query string."
+    )
+    args_schema: Type[BaseModel] = _SearchInput
+
+    api_key: Optional[str] = None
+    """AlterLab API key. Defaults to ALTERLAB_API_KEY env var."""
+
+    api_url: Optional[str] = None
+    """AlterLab API base URL. Defaults to https://api.alterlab.io."""
+
+    num_results: int = 10
+    """Number of results to return (1-30)."""
+
+    search_params: Dict[str, Any] = Field(default_factory=dict)
+    """Additional parameters passed to the AlterLab search API (e.g. engine, domain)."""
+
+    def __init__(self, **kwargs: Any) -> None:
+        """Initialise AlterLabSearchTool.
+
+        Raises:
+            ImportError: If the ``alterlab`` package is not installed.
+            ValueError: If no API key can be resolved.
+        """
+        try:
+            import alterlab  # noqa: F401
+        except ImportError:
+            raise ImportError(
+                "Could not import alterlab python package. "
+                "Please install it with `pip install alterlab`."
+            )
+        super().__init__(**kwargs)
+        get_api_key(self.api_key)
+
+    def _run(self, query: str) -> str:
+        """Search the web and return formatted results, or an error message."""
+        from alterlab import AlterLabSync
+
+        client = AlterLabSync(
+            api_key=get_api_key(self.api_key), base_url=get_api_url(self.api_url)
+        )
+        try:
+            result = client.search(
+                query, num_results=self.num_results, **self.search_params
+            )
+            return _format_search_results(result, query)
+        except Exception as e:
+            logger.warning("AlterLab search failed for %r: %s", query, e)
+            return f"Error searching {query!r}: {e}"
+        finally:
+            client.close()

crewai_alterlab-0.1.0/crewai_alterlab/utilities.py

@@ -0,0 +1,73 @@
+"""Shared AlterLab API wrapper used by CrewAI tools.
+
+Centralises client construction and response-to-content extraction so that
+tool classes stay thin.
+"""
+
+import os
+from typing import Any, Dict, Optional
+
+
+def get_api_key(api_key: Optional[str] = None) -> str:
+    """Resolve the AlterLab API key.
+
+    Args:
+        api_key: Explicit key. Falls back to the ``ALTERLAB_API_KEY``
+            environment variable.
+
+    Returns:
+        The resolved API key.
+
+    Raises:
+        ValueError: If no key can be resolved.
+    """
+    key = api_key or os.getenv("ALTERLAB_API_KEY", "")
+    if not key:
+        raise ValueError(
+            "AlterLab API key is required. Pass api_key or set the "
+            "ALTERLAB_API_KEY environment variable."
+        )
+    return key
+
+
+def get_api_url(api_url: Optional[str] = None) -> str:
+    """Resolve the AlterLab API base URL.
+
+    Args:
+        api_url: Explicit URL. Falls back to ``ALTERLAB_API_URL`` env var
+            or ``https://api.alterlab.io``.
+
+    Returns:
+        The resolved base URL.
+    """
+    return api_url or os.getenv("ALTERLAB_API_URL") or "https://api.alterlab.io"
+
+
+def extract_page_content(result: Dict[str, Any]) -> str:
+    """Extract page content from an AlterLab API response.
+
+    Priority order: markdown > text > html > raw content string.
+
+    Args:
+        result: AlterLab API response dictionary.
+
+    Returns:
+        Extracted content string, or empty string if no content found.
+    """
+    content = result.get("content")
+
+    # Multi-format response (dict with format keys)
+    if isinstance(content, dict):
+        if content.get("markdown"):
+            return content["markdown"]
+        if content.get("text"):
+            return content["text"]
+        if content.get("html"):
+            return content["html"]
+        return ""
+
+    # Legacy single-format response (string)
+    if isinstance(content, str):
+        return content
+
+    return ""

crewai_alterlab-0.1.0/pyproject.toml

@@ -0,0 +1,44 @@
+[build-system]
+requires = ["poetry-core>=1.0.0"]
+build-backend = "poetry.core.masonry.api"
+
+[tool.poetry]
+name = "crewai-alterlab"
+version = "0.1.0"
+description = "CrewAI integration for AlterLab web scraping API"
+authors = ["AlterLab <dev@alterlab.io>"]
+license = "MIT"
+readme = "README.md"
+repository = "https://github.com/RapierCraftStudios/AlterLab"
+homepage = "https://alterlab.io"
+documentation = "https://docs.alterlab.io/integrations/crewai"
+keywords = ["crewai", "alterlab", "web-scraping", "ai-agents", "web-crawler", "scraping-tool"]
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Intended Audience :: Developers",
+    "License :: OSI Approved :: MIT License",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3.13",
+    "Topic :: Software Development :: Libraries",
+    "Topic :: Internet :: WWW/HTTP :: Indexing/Search",
+]
+packages = [{ include = "crewai_alterlab" }]
+
+[tool.poetry.dependencies]
+python = ">=3.10,<4.0"
+crewai = {version = ">=0.80.0", extras = ["tools"]}
+alterlab = ">=2.4.0"
+
+[tool.poetry.group.dev.dependencies]
+pytest = ">=7.4.0"
+
+[tool.black]
+line-length = 88
+target-version = ["py310"]
+
+[tool.isort]
+profile = "black"
+line_length = 88