mcp-plesk-dev-docs 0.4.2__py3-none-any.whl

mcp_plesk_dev_docs-0.4.2.dist-info/METADATA

@@ -0,0 +1,221 @@
+Metadata-Version: 2.4
+Name: mcp-plesk-dev-docs
+Version: 0.4.2
+Summary: A unified MCP server that indexes and retrieves Plesk documentation using vector embeddings and semantic search with reranking
+Author-email: Gilson Siqueira <gilson@example.com>
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/barateza/mcp-plesk-dev-docs
+Project-URL: Documentation, https://github.com/barateza/mcp-plesk-dev-docs#readme
+Project-URL: Repository, https://github.com/barateza/mcp-plesk-dev-docs.git
+Project-URL: Bug Tracker, https://github.com/barateza/mcp-plesk-dev-docs/issues
+Keywords: plesk,mcp,model-context-protocol,semantic-search,rag,vector-database
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: System Administrators
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Documentation
+Classifier: Topic :: System :: Monitoring
+Classifier: Topic :: Utilities
+Requires-Python: >=3.12
+Description-Content-Type: text/markdown
+License-File: LICENSE
+License-File: NOTICE
+Requires-Dist: beautifulsoup4>=4.14.3
+Requires-Dist: fastmcp>=3.2.4
+Requires-Dist: lancedb>=0.29.1
+Requires-Dist: pylance>=0.23.0
+Requires-Dist: numpy>=1.26.0
+Requires-Dist: psutil>=6.1.1
+Requires-Dist: pydantic>=2.10.0
+Requires-Dist: pydantic-settings>=2.7.1
+Requires-Dist: sentence-transformers>=5.2.2
+Requires-Dist: torch>=2.4.0
+Requires-Dist: markdownify>=0.14.1
+Requires-Dist: tantivy>=0.22.0
+Requires-Dist: lance-namespace==0.6.1
+Provides-Extra: dev
+Requires-Dist: pytest>=8.0.0; extra == "dev"
+Requires-Dist: requests>=2.32.0; extra == "dev"
+Requires-Dist: ruff>=0.3.0; extra == "dev"
+Requires-Dist: pre-commit>=3.6.0; extra == "dev"
+Provides-Extra: test
+Requires-Dist: pytest>=8.0.0; extra == "test"
+Provides-Extra: tools
+Requires-Dist: requests>=2.32.0; extra == "tools"
+Dynamic: license-file
+
+# mcp-plesk-dev-docs
+
+[![Python 3.12+](https://img.shields.io/badge/python-3.12%2B-blue?style=flat-square)](https://www.python.org/downloads/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg?style=flat-square)](LICENSE)
+[![MCP Compatible](https://img.shields.io/badge/MCP-Compatible-green?style=flat-square)](https://modelcontextprotocol.io/)
+[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg?style=flat-square)](https://github.com/psf/black)
+[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json&style=flat-square)](https://github.com/astral-sh/ruff)
+
+**State-of-the-Art (SOTA) semantic search across the entire Plesk documentation surface, optimized for sub-second latency on Apple Silicon.**
+
+---
+
+## Why this exists
+
+Plesk documentation is spread across five separate sources: an admin guide, a REST API reference, a CLI reference, a PHP SDK, and a JS SDK. Answering a single extension development question often means searching all of them manually, cross-referencing results, and still missing the relevant section.
+
+This server ingests all five sources, embeds them with a multilingual model, and exposes a single `search_plesk_unified` MCP tool. It uses hybrid search (Vector + FTS), Reciprocal Rank Fusion (RRF), and Cross-Encoder reranking to deliver high-precision results in milliseconds.
+
+---
+
+## Architecture & Performance
+
+```mermaid
+flowchart TD
+    Client["MCP Client\n(Claude Desktop / Cursor / etc.)"]
+
+    Client -->|"search_plesk_unified(query)"| Server
+
+    subgraph Server["FastMCP Server · Modular Architecture"]
+        direction TB
+        Main["Bootstrap · server/main.py"]
+        Life["Lifecycle Hooks · server/lifecycle.py"]
+        Tools["MCP Tools · server/mcp_app.py"]
+
+        Main --> Life --> Tools
+    end
+
+    subgraph Pipeline["Retrieval Pipeline"]
+        direction TB
+        E["1 · Embed query\n(Hardware-accelerated)"]
+        S["2 · Hybrid Search\nVector (LanceDB) + FTS (Tantivy)"]
+        R["3 · RRF Merge + Rerank\n(MiniLM-L4-v2)"]
+        N["4 · Neighbor Expansion\n(Context Enrichment)"]
+        A["5 · AI Synthesis\n(sampling-enabled)"]
+        E --> S --> R --> N --> A
+    end
+
+    subgraph Store["LanceDB Vector & FTS Store"]
+        direction LR
+        G["Guide"]
+        A_["API"]
+        C["CLI"]
+        P["PHP Stubs"]
+        J["JS SDK"]
+    end
+
+    Tools --> Pipeline
+    S <--> Store
+```
+
+### Performance Benchmarks (2026-05-04)
+Optimized for Apple Silicon (M2/M3) using MPS acceleration and memory-resident table caching.
+
+| Profile | Embed Model | HR@5 | MRR@5 | Avg Latency | Est. RAM |
+| :--- | :--- | :--- | :--- | :--- | :--- |
+| **`light`** | BAAI/bge-small | **100.0%** | **0.917** | **1.007 s** | ~200 MB |
+| **`medium`** | BAAI/bge-base | **100.0%** | **0.917** | **~0.60s** | ~600 MB |
+| **`full-tq`** | BAAI/bge-m3 | 75.0% | 0.750 | **~0.40s** | ~1300 MB |
+
+*Metrics measured on Apple M2 Pro with LanceDB connection caching enabled.*
+
+---
+
+## Key Features
+
+- **Sub-Second Hybrid Search:** Combined Vector + Tantivy FTS with **RAM-cached table connections** for instant retrieval.
+- **AST-Aware Chunking:** Uses `tree-sitter` to respect class and method boundaries in PHP, JS, and TS documentation.
+- **TurboQuant Acceleration:** Fast 4-bit quantized search for the `full-tq` profile, delivering 10x lower latency for large models.
+- **Neighborhood Retrieval:** Automatically fetches adjacent chunks (prev/next) to provide complete context for grounding.
+- **Macro-Context Summaries:** Injects file-level purpose summaries into every chunk using the `SummaryCache`.
+- **AI-Synthesized Answers:** Generates concise answers from search results with structured inline citations `[1]`, `[2]`.
+
+---
+
+## MCP Components
+
+This server provides tools, prompts, and resources. See **[docs/mcp-components.md](docs/mcp-components.md)** for a full reference.
+
+### Primary Tools
+
+| Tool | Description |
+|---|---|
+| `search_plesk_unified` | Hybrid search with RRF and Cross-Encoder reranking. |
+| `get_file_content` | Retrieve the full content of a specific documentation file. |
+| `resolve_references` | Find all files referencing a specific symbol or topic. |
+| `refresh_knowledge` | Re-fetch sources and update the index (incremental). |
+| `trigger_index_sync` | Start a background indexing job. |
+| `daemon_health` | Check readiness, hardware acceleration (MPS/CUDA), and latency stats. |
+
+### Resources
+
+- `plesk://toc/api` - Table of Contents for API documentation.
+- `plesk://toc/cli` - Table of Contents for CLI reference.
+- `plesk://toc/guide` - Table of Contents for Extensions Guide.
+- `plesk://toc/php-stubs` - Hierarchical list of PHP classes.
+
+---
+
+## Quickstart
+
+### Install
+
+```bash
+git clone https://github.com/barateza/mcp-plesk-dev-docs.git
+cd mcp-plesk-dev-docs
+uv pip install -e .
+```
+
+### Initial Indexing
+
+```bash
+uv run python -m mcp_plesk_dev_docs.server.main refresh_knowledge
+```
+
+### Running
+
+```bash
+# Standard mode
+uv run python -m mcp_plesk_dev_docs.server.main
+
+# Responsive daemon mode (auto-warmup)
+PLESK_DAEMON_AUTO_WARMUP=true uv run python -m mcp_plesk_dev_docs.server.main
+```
+
+---
+
+## Configuration
+
+Set environment variables in `.env`:
+
+```env
+PLESK_MODEL_PROFILE=light       # light | medium | full-tq
+PLESK_ENABLE_SAMPLING=true     # AI-Synthesized answers
+PLESK_DAEMON_AUTO_WARMUP=true  # Preload models on startup
+PLESK_INDEX_SUMMARIES=true     # Enable file-level summaries
+OPENROUTER_API_KEY=sk-or-v1-...
+```
+
+---
+
+## Documentation
+
+- **[docs/benchmarks.md](docs/benchmarks.md)** - Detailed latency and quality reports.
+- **[docs/mcp-components.md](docs/mcp-components.md)** - Full tool and resource reference.
+- **[docs/turboquant.md](docs/turboquant.md)** - 4-bit quantization internals.
+
+---
+
+## License
+
+MIT. See [LICENSE](LICENSE).
+
+## Ownership & Disclaimer
+
+This is a personal project by Gilson Siqueira. It is not officially affiliated with, endorsed by, or supported by Plesk or WebPros International GmbH. Plesk is a trademark of WebPros International GmbH.
+
+Important notice about Plesk-owned deliverables
+
+Portions of this repository were developed under contract for Plesk International GmbH ("Plesk") only if specifically identified as such. The MIT license above applies only to material the repository owner is authorized to license. Files or directories owned by Plesk, if any, are listed in [NOTICE](NOTICE). If you need assurance about licensing for a particular file, contact Plesk or seek legal counsel before relying on the MIT License for Plesk-owned files.
+
+*Built to make Plesk extension development faster.*
+
+<!-- mcp-name: io.github.barateza/mcp-plesk-dev-docs -->

mcp_plesk_dev_docs-0.4.2.dist-info/RECORD

@@ -0,0 +1,30 @@
+mcp_plesk_dev_docs-0.4.2.dist-info/licenses/LICENSE,sha256=DAUQMJI5O7eP9m7odM9DuvgQcc4aEQAz2tcr5jcsk4g,1072
+mcp_plesk_dev_docs-0.4.2.dist-info/licenses/NOTICE,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+plesk_unified/__init__.py,sha256=rYrEqs-AW78q99P-HMJc0uvgiHGdVhhafkhDpVt6Cjs,74
+plesk_unified/ai_client.py,sha256=qyLHZrhWgpmSeHVPPPPuSO1ySEZGVJj2dXfELflWr60,9822
+plesk_unified/benchmark_engines.py,sha256=N0BqBm6DnaUss8cqKWWu4Kf0bOtFtbPjuxHI_E1tEtc,8583
+plesk_unified/benchmark_gates.py,sha256=WH6Wexqj_13jhFei14IFQvqDsMzi0LpVQqVJFhTzFvY,7971
+plesk_unified/benchmark_reporting.py,sha256=fhIqyibyPfqcxRLloxLbrPx07CqKiqzo-eZs5q841RM,3938
+plesk_unified/benchmark_runner.py,sha256=9vxzw1C6710E5Y5eKmZknvtDM2TrCle27nDut7CuVi4,14877
+plesk_unified/benchmark_suites.py,sha256=jDNUia71S_Y4nO_MAVBrnqVMAtNZtUelAZsiYfOnFjI,901
+plesk_unified/chunking.py,sha256=3C-KabtgwEEjxewbkajxFtQIFOflzbappUdLqluywV0,11266
+plesk_unified/error_handling.py,sha256=7BTO7eaJkB8yDiIzlXTXAZbVqpa1UoRcfMkV21_duBg,3835
+plesk_unified/html_utils.py,sha256=1X_68woscZqwnXOHQK4MZ2zLcecrb7sY_tZZkuQt2Yw,6511
+plesk_unified/indexing.py,sha256=9GbkRGK0-KhVN7ohVD4Thzmr57jx86qxubYAL7shfI4,1805
+plesk_unified/io_utils.py,sha256=PYK9TUkoEjZPLG80ZEUoOXS7VckD7tBfjYLF2V9zztY,9749
+plesk_unified/log_handler.py,sha256=NaV9rRyDjFUiItNE8T4dpnTB3k4LbgGF-6t_FzBe0HI,7553
+plesk_unified/model_config.py,sha256=c05V64wbxpOlTeYDq1vxN2AJdcnYyjMjoe1z6eGc_Ys,7386
+plesk_unified/platform_utils.py,sha256=Q2CqeDaErk6a5IIjF5D8rxPWEkZ4nP8jyN85ylWa5G0,6244
+plesk_unified/settings.py,sha256=ptmZ7VumrlEcUJTpwYIY8fzPjpqmnCYUP_ygUxhjuzs,3255
+plesk_unified/summary_cache.py,sha256=iT_kxCbndF2NDwSetqimpuY_-urZlvvUVr0IHuRw364,1890
+plesk_unified/tq_index.py,sha256=T7Fusl-7GXK5PMQ2_IucBJXSkTIHVeppoDyi9cMnuE4,3085
+plesk_unified/types.py,sha256=xArD8LxLb1IdvjLlPA0tkHjc8zxDbXUiBT3SzpQAHII,735
+plesk_unified/turboquant/__init__.py,sha256=7mCd6gZRbMnJevSt9dB1glddUVxj1PseFmx_6KPPA40,629
+plesk_unified/turboquant/compressors.py,sha256=Vo_6VabVafnvOA1OHTFGnhDnxbg9ER_TPAj1JcZhe8s,6864
+plesk_unified/turboquant/lloyd_max.py,sha256=tnMboA97bDd2MGNfPPYXOrXYM2-63yEI83vvT81B5kw,6270
+plesk_unified/turboquant/turboquant.py,sha256=-5G4vz9apVLATzPyu3oVJoqw7su_DdLOZOE5OM-EI1k,7795
+mcp_plesk_dev_docs-0.4.2.dist-info/METADATA,sha256=NsFuaU_dsArUG3aNwWgLT-26mxMxw9fCSzYEPfU9CGg,8692
+mcp_plesk_dev_docs-0.4.2.dist-info/WHEEL,sha256=aeYiig01lYGDzBgS8HxWXOg3uV61G9ijOsup-k9o1sk,91
+mcp_plesk_dev_docs-0.4.2.dist-info/entry_points.txt,sha256=iBoGseS6PEcwx4sBqTp76JDE5PWCobbei5vKTeFide8,65
+mcp_plesk_dev_docs-0.4.2.dist-info/top_level.txt,sha256=UCi5kMuaZggLf-5-cJZXODKDlL6xQBpPQCjASPasBiQ,14
+mcp_plesk_dev_docs-0.4.2.dist-info/RECORD,,

mcp_plesk_dev_docs-0.4.2.dist-info/WHEEL

@@ -0,0 +1,5 @@
+Wheel-Version: 1.0
+Generator: setuptools (82.0.1)
+Root-Is-Purelib: true
+Tag: py3-none-any
+

mcp_plesk_dev_docs-0.4.2.dist-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+mcp-plesk-dev-docs = plesk_unified.server:main

mcp_plesk_dev_docs-0.4.2.dist-info/licenses/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 Gilson Siqueira
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

mcp_plesk_dev_docs-0.4.2.dist-info/top_level.txt

@@ -0,0 +1 @@
+plesk_unified

plesk_unified/__init__.py

@@ -0,0 +1,3 @@
+"""plesk_unified helper package."""
+
+__all__ = ["html_utils", "chunking"]

plesk_unified/ai_client.py

@@ -0,0 +1,257 @@
+import json
+import logging
+import os
+import re
+from typing import List, Optional
+
+import requests
+import httpx
+
+logger = logging.getLogger(__name__)
+
+DEFAULT_MODELS = [
+    "deepseek/deepseek-v4-flash",
+    "google/gemini-2.5-flash-lite",
+    "x-ai/grok-4.1-fast",
+    "nvidia/nemotron-3-super-120b-a12b:free",
+]
+
+# RAGAS judge model — chosen for instruction-following + low verbosity on strict
+# "Score 0.0–1.0" tasks.
+RAGAS_DEFAULT_MODELS = [
+    "google/gemini-2.0-flash",
+    "anthropic/claude-3-haiku",
+]
+
+
+class AIClient:
+    """A thin wrapper for LLM calls with retry/fallback policy."""
+
+    def __init__(self, api_key: Optional[str] = None):
+        self.api_key = api_key or os.environ.get("OPENROUTER_API_KEY", "")
+        if not self.api_key:
+            logger.warning("OPENROUTER_API_KEY is not set. AI calls will fail.")
+        else:
+            logger.info("AIClient initialized with an API key.")
+        self._async_client: Optional[httpx.AsyncClient] = None
+
+    async def get_async_client(self) -> httpx.AsyncClient:
+        if self._async_client is None or self._async_client.is_closed:
+            self._async_client = httpx.AsyncClient(timeout=30.0)
+        return self._async_client
+
+    async def close(self):
+        if self._async_client and not self._async_client.is_closed:
+            await self._async_client.aclose()
+
+    def generate_answer(
+        self, query: str, context: str, model_list: Optional[List[str]] = None
+    ) -> str:
+        """
+        Generate an answer to the query based strictly on the provided context.
+        """
+        models = model_list if model_list is not None else DEFAULT_MODELS
+
+        for model in models:
+            try:
+                content = (
+                    f"CONTEXT:\n{context}\n\n"
+                    f"QUERY: {query}\n\n"
+                    "Answer the query based ONLY on the provided context. "
+                    "If the information is not present, say 'Information not found'."
+                )
+
+                payload = {
+                    "model": model,
+                    "messages": [{"role": "user", "content": content}],
+                    "max_tokens": 500,
+                }
+
+                logger.debug(f"Sending request to model {model} (answer generation)")
+                response = requests.post(
+                    url="https://openrouter.ai/api/v1/chat/completions",
+                    headers={
+                        "Authorization": f"Bearer {self.api_key}",
+                        "Content-Type": "application/json",
+                    },
+                    data=json.dumps(payload),
+                    timeout=20,
+                )
+                if response.status_code == 200:
+                    data = response.json()
+                    choices = data.get("choices")
+                    if choices and "message" in choices[0]:
+                        res = choices[0]["message"].get("content")
+                        if res:
+                            return res.strip()
+
+                logger.error(f"Model {model} returned status {response.status_code}")
+            except Exception as e:
+                logger.error(f"Answer Model {model} failed with exception: {e}")
+                continue
+
+        return "Error generating answer."
+
+    async def generate_description_async(
+        self, text: str, model_list: Optional[List[str]] = None
+    ) -> str:
+        """
+        Attempts to get a description using a tiered fallback system (Asynchronous).
+        """
+        if not text.strip():
+            return "File unreadable."
+
+        models = model_list if model_list is not None else DEFAULT_MODELS
+        client = await self.get_async_client()
+
+        for model in models:
+            try:
+                content = (
+                    "Summarize the technical purpose of the following text "
+                    "in exactly one concise sentence.\n\n" + text
+                )
+
+                payload = {
+                    "model": model,
+                    "messages": [{"role": "user", "content": content}],
+                    "max_tokens": 100,
+                }
+
+                # Force DeepSeek provider for deepseek models
+                # to avoid slow providers like deepinfra
+                if "deepseek" in model:
+                    payload["provider"] = {"order": ["DeepSeek"]}
+
+                response = await client.post(
+                    url="https://openrouter.ai/api/v1/chat/completions",
+                    headers={
+                        "Authorization": f"Bearer {self.api_key}",
+                        "Content-Type": "application/json",
+                    },
+                    json=payload,
+                )
+                if response.status_code == 200:
+                    data = response.json()
+                    choices = data.get("choices")
+                    if choices and "message" in choices[0]:
+                        res = choices[0]["message"].get("content")
+                        if res:
+                            return res.strip()
+
+                logger.error(f"Model {model} returned status {response.status_code}")
+            except Exception as e:
+                logger.error(f"Model {model} failed with exception: {e}")
+                continue
+
+        return "Description unavailable."
+
+    def generate_description(
+        self, text: str, model_list: Optional[List[str]] = None
+    ) -> str:
+        """
+        Attempts to get a description using a tiered fallback system.
+        """
+        if not text.strip():
+            return "File unreadable."
+
+        models = model_list if model_list is not None else DEFAULT_MODELS
+
+        logger.info(f"AIClient.generate_description called for text length {len(text)}")
+        for model in models:
+            try:
+                content = (
+                    "Summarize the technical purpose of the following text "
+                    "in exactly one concise sentence.\n\n" + text
+                )
+
+                payload = {
+                    "model": model,
+                    "messages": [{"role": "user", "content": content}],
+                    "max_tokens": 100,
+                }
+
+                logger.info(f"Sending description request to model {model}")
+                response = requests.post(
+                    url="https://openrouter.ai/api/v1/chat/completions",
+                    headers={
+                        "Authorization": f"Bearer {self.api_key}",
+                        "Content-Type": "application/json",
+                    },
+                    data=json.dumps(payload),
+                    timeout=15,
+                )
+                if response.status_code == 200:
+                    data = response.json()
+                    choices = data.get("choices")
+                    if choices and "message" in choices[0]:
+                        res = choices[0]["message"].get("content")
+                        if res:
+                            logger.info(
+                                f"Successfully generated description using {model}"
+                            )
+                            return res.strip()
+
+                logger.error(f"Model {model} returned status {response.status_code}")
+            except Exception as e:
+                logger.error(f"Model {model} failed with exception: {e}")
+                continue
+
+        return "Description unavailable."
+
+    def evaluate_ragas_score(
+        self, prompt: str, model_list: Optional[List[str]] = None
+    ) -> float:
+        """
+        Evaluate a RAGAS metric prompt and return a score between 0.0 and 1.0.
+        Uses stricter instruction following for the judge model.
+        """
+        models = model_list if model_list is not None else RAGAS_DEFAULT_MODELS
+
+        for model in models:
+            try:
+                payload = {
+                    "model": model,
+                    "messages": [
+                        {
+                            "role": "system",
+                            "content": (
+                                "You are a precise evaluator. Output ONLY a single "
+                                "float number between 0.0 and 1.0 representing the "
+                                "score. No explanation, no extra text."
+                            ),
+                        },
+                        {"role": "user", "content": prompt},
+                    ],
+                    "max_tokens": 10,
+                }
+
+                response = requests.post(
+                    url="https://openrouter.ai/api/v1/chat/completions",
+                    headers={
+                        "Authorization": f"Bearer {self.api_key}",
+                        "Content-Type": "application/json",
+                    },
+                    data=json.dumps(payload),
+                    timeout=20,
+                )
+                if response.status_code == 200:
+                    data = response.json()
+                    choices = data.get("choices")
+                    if choices and "message" in choices[0]:
+                        msg = choices[0]["message"]
+                        text = msg.get("content", "").strip()
+                        # Try to find a float in the response
+                        match = re.search(r"(\d?\.\d+)", text)
+                        if match:
+                            return float(match.group(1))
+                        if text in ["0", "1"]:
+                            return float(text)
+                else:
+                    logger.debug(
+                        f"RAGAS Model {model} returned status {response.status_code}"
+                    )
+            except Exception as e:
+                logger.debug(f"RAGAS Model {model} failed with exception: {e}")
+                continue
+
+        return 0.0