ragas-ainative 0.1.0__tar.gz

ragas_ainative-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 AINative Studio
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

ragas_ainative-0.1.0/PKG-INFO

@@ -0,0 +1,116 @@
+Metadata-Version: 2.4
+Name: ragas-ainative
+Version: 0.1.0
+Summary: Configure ragas RAG evaluation to use AINative's free LLMs + embeddings — no OpenAI key needed.
+Author-email: AINative Studio <dev@ainative.studio>
+License: MIT
+Project-URL: Homepage, https://github.com/AINative-Studio/ragas-ainative
+Project-URL: Documentation, https://docs.ainative.studio
+Project-URL: Repository, https://github.com/AINative-Studio/ragas-ainative
+Project-URL: Issues, https://github.com/AINative-Studio/ragas-ainative/issues
+Keywords: ragas,rag-evaluation,rag,ainative,free-embeddings,free-llm,zerodb,auto-provisioning,faithfulness,answer-relevancy,claude,cursor
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: requests>=2.28
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0; extra == "dev"
+Requires-Dist: pytest-cov>=4.0; extra == "dev"
+Dynamic: license-file
+
+# ragas-ainative
+
+Configure [ragas](https://github.com/explodinggradients/ragas) to use AINative's free LLMs + embeddings for RAG evaluation. No OpenAI key needed.
+
+**Zero setup.** Evaluate faithfulness, answer relevancy, and more with Llama 3.3 70B — completely free.
+
+## Install
+
+```bash
+pip install ragas-ainative
+```
+
+## Quick Start
+
+```python
+from ragas_ainative import configure_ragas
+from ragas.metrics import faithfulness, answer_relevancy
+from ragas import evaluate
+
+configure_ragas()  # Auto-provisions free API key
+
+results = evaluate(dataset, metrics=[faithfulness, answer_relevancy])
+print(results)
+```
+
+## How It Works
+
+`ragas-ainative` is a companion package (NOT a fork) that configures ragas to use AINative's free, OpenAI-compatible API. It sets `OPENAI_API_KEY` and `OPENAI_API_BASE` so ragas routes all LLM judge calls and embeddings through AINative.
+
+On first use, the package auto-provisions a free API key (72-hour TTL). Claim your account at [ainative.studio/signup](https://ainative.studio/signup) for permanent access.
+
+## Available Models
+
+```python
+from ragas_ainative import configure_ragas
+
+configure_ragas(model="llama")     # meta-llama/Llama-3.3-70B-Instruct (default)
+configure_ragas(model="qwen")      # qwen3-coder-flash
+configure_ragas(model="deepseek")  # deepseek-4-flash
+configure_ragas(model="kimi")      # kimi-k2
+```
+
+## Store Results to ZeroDB
+
+Track evaluation scores over time:
+
+```python
+from ragas_ainative import configure_ragas, store_results
+
+configure_ragas()
+
+# After evaluation
+results = {"faithfulness": 0.85, "answer_relevancy": 0.92}
+store_results(results, dataset_name="my-rag-pipeline")
+# Stored to ZeroDB — searchable and trackable
+```
+
+## Explicit API Key
+
+```python
+configure_ragas(api_key="your-key-here")
+```
+
+Or set the environment variable:
+
+```bash
+export AINATIVE_API_KEY=your-key-here
+```
+
+## API Key Resolution Order
+
+1. Explicit `api_key` parameter
+2. `AINATIVE_API_KEY` environment variable
+3. `ZERODB_API_KEY` environment variable
+4. `~/.zerodb/credentials.json` (shared with zerodb ecosystem)
+5. Auto-provision via instant-db
+
+## Requirements
+
+- Python >= 3.9
+- ragas >= 0.1.0
+
+## License
+
+MIT

ragas_ainative-0.1.0/README.md

@@ -0,0 +1,85 @@
+# ragas-ainative
+
+Configure [ragas](https://github.com/explodinggradients/ragas) to use AINative's free LLMs + embeddings for RAG evaluation. No OpenAI key needed.
+
+**Zero setup.** Evaluate faithfulness, answer relevancy, and more with Llama 3.3 70B — completely free.
+
+## Install
+
+```bash
+pip install ragas-ainative
+```
+
+## Quick Start
+
+```python
+from ragas_ainative import configure_ragas
+from ragas.metrics import faithfulness, answer_relevancy
+from ragas import evaluate
+
+configure_ragas()  # Auto-provisions free API key
+
+results = evaluate(dataset, metrics=[faithfulness, answer_relevancy])
+print(results)
+```
+
+## How It Works
+
+`ragas-ainative` is a companion package (NOT a fork) that configures ragas to use AINative's free, OpenAI-compatible API. It sets `OPENAI_API_KEY` and `OPENAI_API_BASE` so ragas routes all LLM judge calls and embeddings through AINative.
+
+On first use, the package auto-provisions a free API key (72-hour TTL). Claim your account at [ainative.studio/signup](https://ainative.studio/signup) for permanent access.
+
+## Available Models
+
+```python
+from ragas_ainative import configure_ragas
+
+configure_ragas(model="llama")     # meta-llama/Llama-3.3-70B-Instruct (default)
+configure_ragas(model="qwen")      # qwen3-coder-flash
+configure_ragas(model="deepseek")  # deepseek-4-flash
+configure_ragas(model="kimi")      # kimi-k2
+```
+
+## Store Results to ZeroDB
+
+Track evaluation scores over time:
+
+```python
+from ragas_ainative import configure_ragas, store_results
+
+configure_ragas()
+
+# After evaluation
+results = {"faithfulness": 0.85, "answer_relevancy": 0.92}
+store_results(results, dataset_name="my-rag-pipeline")
+# Stored to ZeroDB — searchable and trackable
+```
+
+## Explicit API Key
+
+```python
+configure_ragas(api_key="your-key-here")
+```
+
+Or set the environment variable:
+
+```bash
+export AINATIVE_API_KEY=your-key-here
+```
+
+## API Key Resolution Order
+
+1. Explicit `api_key` parameter
+2. `AINATIVE_API_KEY` environment variable
+3. `ZERODB_API_KEY` environment variable
+4. `~/.zerodb/credentials.json` (shared with zerodb ecosystem)
+5. Auto-provision via instant-db
+
+## Requirements
+
+- Python >= 3.9
+- ragas >= 0.1.0
+
+## License
+
+MIT

ragas_ainative-0.1.0/pyproject.toml

@@ -0,0 +1,62 @@
+[build-system]
+requires = ["setuptools>=68.0", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "ragas-ainative"
+version = "0.1.0"
+description = "Configure ragas RAG evaluation to use AINative's free LLMs + embeddings — no OpenAI key needed."
+readme = "README.md"
+license = {text = "MIT"}
+requires-python = ">=3.9"
+authors = [
+    {name = "AINative Studio", email = "dev@ainative.studio"},
+]
+keywords = [
+    "ragas",
+    "rag-evaluation",
+    "rag",
+    "ainative",
+    "free-embeddings",
+    "free-llm",
+    "zerodb",
+    "auto-provisioning",
+    "faithfulness",
+    "answer-relevancy",
+    "claude",
+    "cursor",
+]
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Intended Audience :: Developers",
+    "License :: OSI Approved :: MIT License",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.9",
+    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3.13",
+    "Topic :: Scientific/Engineering :: Artificial Intelligence",
+    "Topic :: Software Development :: Libraries :: Python Modules",
+]
+dependencies = [
+    "requests>=2.28",
+]
+
+[project.optional-dependencies]
+dev = [
+    "pytest>=7.0",
+    "pytest-cov>=4.0",
+]
+
+[project.urls]
+Homepage = "https://github.com/AINative-Studio/ragas-ainative"
+Documentation = "https://docs.ainative.studio"
+Repository = "https://github.com/AINative-Studio/ragas-ainative"
+Issues = "https://github.com/AINative-Studio/ragas-ainative/issues"
+
+[tool.setuptools.packages.find]
+include = ["ragas_ainative*"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]

ragas_ainative-0.1.0/ragas_ainative/__init__.py

@@ -0,0 +1,42 @@
+"""
+ragas-ainative — Configure ragas to use AINative's free LLMs + embeddings.
+
+No OpenAI key needed. Evaluate your RAG pipelines for free with Llama, Qwen,
+DeepSeek, and Kimi models.
+
+Usage:
+    from ragas_ainative import configure_ragas
+    from ragas.metrics import faithfulness, answer_relevancy
+    from ragas import evaluate
+
+    configure_ragas()  # sets AINative as eval LLM + embeddings
+
+    results = evaluate(dataset, metrics=[faithfulness, answer_relevancy])
+
+Refs #3954
+"""
+
+from ragas_ainative.config import (
+    configure_ragas,
+    get_llm_config,
+    get_embeddings_config,
+    store_results,
+    MODELS,
+    DEFAULT_MODEL,
+    API_BASE,
+    EMBEDDINGS_MODEL,
+)
+from ragas_ainative.provision import resolve_api_key
+
+__all__ = [
+    "configure_ragas",
+    "get_llm_config",
+    "get_embeddings_config",
+    "store_results",
+    "resolve_api_key",
+    "MODELS",
+    "DEFAULT_MODEL",
+    "API_BASE",
+    "EMBEDDINGS_MODEL",
+]
+__version__ = "0.1.0"

ragas_ainative-0.1.0/ragas_ainative/config.py

@@ -0,0 +1,225 @@
+"""
+ragas-ainative — Configuration
+
+Sets environment variables so ragas uses AINative's OpenAI-compatible
+endpoint for LLM-based evaluation and embeddings.
+
+Ragas reads OPENAI_API_KEY and OPENAI_API_BASE to configure its default
+LLM judge and embeddings. We set these to AINative values.
+
+Refs #3954
+"""
+
+from __future__ import annotations
+
+import json
+import os
+import sys
+from datetime import datetime
+from typing import Any, Dict, List, Optional
+
+import requests
+
+from ragas_ainative.provision import resolve_api_key
+
+API_BASE = "https://api.ainative.studio/api/v1"
+MEMORY_API_BASE = "https://api.ainative.studio/api/v1/public/memory/v2"
+EMBEDDINGS_ENDPOINT = "https://api.ainative.studio/api/v1/embeddings"
+
+# Model aliases -> full model identifiers
+MODELS: Dict[str, str] = {
+    # Meta Llama
+    "llama": "meta-llama/Llama-3.3-70B-Instruct",
+    "llama-70b": "meta-llama/Llama-3.3-70B-Instruct",
+    "llama-8b": "meta-llama/Llama-3.1-8B-Instruct",
+    # Qwen
+    "qwen": "qwen3-coder-flash",
+    "qwen-coder": "qwen3-coder-flash",
+    # DeepSeek
+    "deepseek": "deepseek-4-flash",
+    "deepseek-flash": "deepseek-4-flash",
+    # Kimi
+    "kimi": "kimi-k2",
+}
+
+DEFAULT_MODEL = "meta-llama/Llama-3.3-70B-Instruct"
+EMBEDDINGS_MODEL = "bge-m3"
+
+
+def get_model(alias: str) -> str:
+    """
+    Resolve a model alias to its full identifier.
+
+    Args:
+        alias: Short alias (e.g. "llama", "qwen") or full model ID.
+
+    Returns:
+        Full model identifier string.
+    """
+    return MODELS.get(alias, alias)
+
+
+def get_llm_config(
+    model: Optional[str] = None,
+    api_key: Optional[str] = None,
+    api_base: Optional[str] = None,
+) -> dict:
+    """
+    Return env var overrides for ragas LLM judge.
+
+    Args:
+        model: Model alias or full ID. Defaults to Llama 3.3 70B.
+        api_key: Explicit API key. Auto-provisions if not provided.
+        api_base: Override API base URL.
+
+    Returns:
+        Dict with OPENAI_API_KEY, OPENAI_API_BASE, model name.
+    """
+    key = resolve_api_key(api_key)
+    base = api_base or API_BASE
+    model_id = get_model(model) if model else DEFAULT_MODEL
+
+    return {
+        "OPENAI_API_KEY": key,
+        "OPENAI_API_BASE": base,
+        "model": model_id,
+    }
+
+
+def get_embeddings_config(
+    api_key: Optional[str] = None,
+    api_base: Optional[str] = None,
+    model: Optional[str] = None,
+) -> dict:
+    """
+    Return config for AINative embeddings used by ragas.
+
+    Args:
+        api_key: Explicit API key. Uses resolved key if not provided.
+        api_base: Override API base URL.
+        model: Embeddings model name. Defaults to bge-m3.
+
+    Returns:
+        Dict with embeddings API configuration.
+    """
+    key = resolve_api_key(api_key)
+    base = api_base or API_BASE
+    emb_model = model or EMBEDDINGS_MODEL
+
+    return {
+        "OPENAI_API_KEY": key,
+        "OPENAI_API_BASE": base,
+        "embeddings_model": emb_model,
+    }
+
+
+def configure_ragas(
+    model: Optional[str] = None,
+    api_key: Optional[str] = None,
+    api_base: Optional[str] = None,
+    verbose: bool = True,
+) -> str:
+    """
+    Configure ragas to use AINative's free LLMs and embeddings.
+
+    Sets environment variables that ragas reads for its LLM judge
+    and embeddings:
+    - OPENAI_API_KEY -> AINative API key
+    - OPENAI_API_BASE -> AINative OpenAI-compatible endpoint
+
+    Also sets AINATIVE_API_KEY for ecosystem compatibility.
+
+    Args:
+        model: Model alias or full ID for the judge LLM.
+            Defaults to Llama 3.3 70B.
+        api_key: Explicit API key. Auto-provisions if not provided.
+        api_base: Override API base URL.
+        verbose: Print configuration summary to stderr.
+
+    Returns:
+        The API key being used.
+
+    Example:
+        from ragas_ainative import configure_ragas
+        from ragas.metrics import faithfulness, answer_relevancy
+        from ragas import evaluate
+
+        configure_ragas()
+        results = evaluate(dataset, metrics=[faithfulness, answer_relevancy])
+    """
+    config = get_llm_config(model=model, api_key=api_key, api_base=api_base)
+
+    # Set env vars for ragas (uses OpenAI SDK under the hood)
+    os.environ["OPENAI_API_KEY"] = config["OPENAI_API_KEY"]
+    os.environ["OPENAI_API_BASE"] = config["OPENAI_API_BASE"]
+
+    # Also set AINATIVE_API_KEY for ecosystem tools
+    os.environ["AINATIVE_API_KEY"] = config["OPENAI_API_KEY"]
+
+    if verbose:
+        key_preview = config["OPENAI_API_KEY"][:12] + "..."
+        print(
+            f"\n  ragas-ainative configured!\n"
+            f"\n"
+            f"  Judge LLM:   {config['model']}\n"
+            f"  Embeddings:  {EMBEDDINGS_MODEL}\n"
+            f"  API Base:    {config['OPENAI_API_BASE']}\n"
+            f"  API Key:     {key_preview}\n"
+            f"\n"
+            f"  All ragas evaluations will now use AINative's free LLMs.\n",
+            file=sys.stderr,
+        )
+
+    return config["OPENAI_API_KEY"]
+
+
+def store_results(
+    results: dict,
+    dataset_name: str = "ragas-eval",
+    api_key: Optional[str] = None,
+) -> Optional[dict]:
+    """
+    Store ragas evaluation results to ZeroDB for tracking over time.
+
+    Sends the evaluation scores as a memory entry to ZeroDB's
+    /remember endpoint, making it searchable and trackable.
+
+    Args:
+        results: Ragas evaluation results dict (scores per metric).
+        dataset_name: Name to tag the evaluation. Defaults to "ragas-eval".
+        api_key: Explicit API key. Uses resolved key if not provided.
+
+    Returns:
+        ZeroDB response dict, or None if storage fails.
+    """
+    key = resolve_api_key(api_key)
+
+    payload = {
+        "entity": f"ragas-eval:{dataset_name}",
+        "content": json.dumps({
+            "type": "ragas_evaluation",
+            "dataset": dataset_name,
+            "scores": results,
+            "evaluated_at": datetime.now(tz=None).isoformat(),
+        }),
+        "metadata": {
+            "source": "ragas-ainative",
+            "dataset": dataset_name,
+        },
+    }
+
+    try:
+        resp = requests.post(
+            f"{MEMORY_API_BASE}/remember",
+            json=payload,
+            headers={
+                "Authorization": f"Bearer {key}",
+                "Content-Type": "application/json",
+            },
+            timeout=15,
+        )
+        if resp.status_code in (200, 201):
+            return resp.json()
+        return None
+    except requests.RequestException:
+        return None

ragas_ainative-0.1.0/ragas_ainative/provision.py

@@ -0,0 +1,174 @@
+"""
+ragas-ainative — Auto-Provisioning
+
+Zero-friction provisioning for ragas users.
+On first use, if no API key is found, automatically provisions
+a free AINative account (72-hour TTL) that can be claimed later.
+
+Credential resolution order:
+1. Explicit api_key parameter
+2. AINATIVE_API_KEY environment variable
+3. ZERODB_API_KEY environment variable (shared with zerodb ecosystem)
+4. ~/.zerodb/credentials.json (shared credential store)
+5. Auto-provision via /api/v1/public/instant-db
+
+Refs #3954
+"""
+
+import json
+import os
+import sys
+from datetime import datetime
+from pathlib import Path
+from typing import Optional
+
+import requests
+
+ZERODB_DIR = Path.home() / ".zerodb"
+CREDS_PATH = ZERODB_DIR / "credentials.json"
+CONFIG_PATH = ZERODB_DIR / "config.json"
+CLOUD_API_URL = "https://api.ainative.studio"
+PROVISION_ENDPOINT = "/api/v1/public/instant-db"
+
+
+def resolve_api_key(explicit_key: Optional[str] = None) -> str:
+    """
+    Resolve an API key from all available sources.
+
+    Args:
+        explicit_key: Key passed directly by the user.
+
+    Returns:
+        A valid API key string.
+
+    Raises:
+        RuntimeError: If auto-provisioning fails and no key is available.
+    """
+    # 1. Explicit parameter
+    if explicit_key:
+        return explicit_key
+
+    # 2. Environment variables
+    env_key = (
+        os.environ.get("AINATIVE_API_KEY")
+        or os.environ.get("ZERODB_API_KEY")
+    )
+    if env_key:
+        return env_key
+
+    # 3. Credentials file (shared with zerodb ecosystem)
+    creds = _load_credentials()
+    if creds:
+        return creds
+
+    # 4. Auto-provision
+    return _auto_provision()
+
+
+def _load_credentials() -> Optional[str]:
+    """Load API key from ~/.zerodb/credentials.json."""
+    if CREDS_PATH.exists():
+        try:
+            data = json.loads(CREDS_PATH.read_text())
+            key = data.get("api_key")
+            if key:
+                return key
+        except (json.JSONDecodeError, KeyError):
+            pass
+
+    if CONFIG_PATH.exists():
+        try:
+            data = json.loads(CONFIG_PATH.read_text())
+            key = data.get("api_key") or data.get("cloud_api_key")
+            if key:
+                return key
+        except (json.JSONDecodeError, KeyError):
+            pass
+
+    return None
+
+
+def _auto_provision() -> str:
+    """
+    Auto-provision a free AINative account for ragas evaluation.
+
+    Returns:
+        API key string.
+
+    Raises:
+        RuntimeError: If provisioning fails.
+    """
+    print(
+        "\n  No API key found — provisioning a free AINative account for ragas...",
+        file=sys.stderr,
+    )
+
+    try:
+        resp = requests.post(
+            f"{CLOUD_API_URL}{PROVISION_ENDPOINT}",
+            json={"agree_terms": True, "source": "ragas-ainative"},
+            timeout=15,
+        )
+
+        if resp.status_code == 429:
+            raise RuntimeError(
+                "Rate limited — too many provisions from this IP. "
+                "Sign up at https://ainative.studio/signup and set AINATIVE_API_KEY."
+            )
+
+        if resp.status_code not in (200, 201):
+            raise RuntimeError(
+                f"Provisioning failed (HTTP {resp.status_code}). "
+                "Sign up at https://ainative.studio/signup and set AINATIVE_API_KEY."
+            )
+
+        data = resp.json()
+        api_key = data.get("api_key", "")
+        if not api_key:
+            raise RuntimeError("Provisioning returned empty API key.")
+
+        _save_credentials(data)
+        _print_success(data)
+        return api_key
+
+    except requests.RequestException as exc:
+        raise RuntimeError(
+            f"Network error during provisioning: {exc}. "
+            "Set AINATIVE_API_KEY manually or sign up at https://ainative.studio/signup"
+        ) from exc
+
+
+def _save_credentials(data: dict) -> None:
+    """Save provisioned credentials to ~/.zerodb/ for ecosystem sharing."""
+    ZERODB_DIR.mkdir(parents=True, exist_ok=True)
+
+    creds = {
+        "api_key": data.get("api_key", ""),
+        "project_id": data.get("project_id", ""),
+        "base_url": data.get("base_url", CLOUD_API_URL),
+        "expires_at": data.get("expires_at", ""),
+        "claim_url": data.get("claim_url", ""),
+        "provisioned_at": datetime.now(tz=None).isoformat(),
+        "source": "ragas-ainative",
+    }
+
+    CREDS_PATH.write_text(json.dumps(creds, indent=2) + "\n")
+
+
+def _print_success(data: dict) -> None:
+    """Print success message with claim URL."""
+    expires = data.get("expires_at", "72 hours")
+    claim_url = data.get("claim_url", "https://ainative.studio/signup")
+    api_key = data.get("api_key", "")
+
+    print(
+        f"\n  Auto-provisioned! Free RAG evaluation API ready.\n"
+        f"\n"
+        f"  API Key:   {api_key[:12]}...\n"
+        f"  Expires:   {expires}\n"
+        f"  Saved to:  ~/.zerodb/credentials.json\n"
+        f"\n"
+        f"  To keep access permanently, claim your account:\n"
+        f"  {claim_url}\n",
+        file=sys.stderr,
+    )

ragas_ainative-0.1.0/ragas_ainative.egg-info/PKG-INFO

@@ -0,0 +1,116 @@
+Metadata-Version: 2.4
+Name: ragas-ainative
+Version: 0.1.0
+Summary: Configure ragas RAG evaluation to use AINative's free LLMs + embeddings — no OpenAI key needed.
+Author-email: AINative Studio <dev@ainative.studio>
+License: MIT
+Project-URL: Homepage, https://github.com/AINative-Studio/ragas-ainative
+Project-URL: Documentation, https://docs.ainative.studio
+Project-URL: Repository, https://github.com/AINative-Studio/ragas-ainative
+Project-URL: Issues, https://github.com/AINative-Studio/ragas-ainative/issues
+Keywords: ragas,rag-evaluation,rag,ainative,free-embeddings,free-llm,zerodb,auto-provisioning,faithfulness,answer-relevancy,claude,cursor
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: requests>=2.28
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0; extra == "dev"
+Requires-Dist: pytest-cov>=4.0; extra == "dev"
+Dynamic: license-file
+
+# ragas-ainative
+
+Configure [ragas](https://github.com/explodinggradients/ragas) to use AINative's free LLMs + embeddings for RAG evaluation. No OpenAI key needed.
+
+**Zero setup.** Evaluate faithfulness, answer relevancy, and more with Llama 3.3 70B — completely free.
+
+## Install
+
+```bash
+pip install ragas-ainative
+```
+
+## Quick Start
+
+```python
+from ragas_ainative import configure_ragas
+from ragas.metrics import faithfulness, answer_relevancy
+from ragas import evaluate
+
+configure_ragas()  # Auto-provisions free API key
+
+results = evaluate(dataset, metrics=[faithfulness, answer_relevancy])
+print(results)
+```
+
+## How It Works
+
+`ragas-ainative` is a companion package (NOT a fork) that configures ragas to use AINative's free, OpenAI-compatible API. It sets `OPENAI_API_KEY` and `OPENAI_API_BASE` so ragas routes all LLM judge calls and embeddings through AINative.
+
+On first use, the package auto-provisions a free API key (72-hour TTL). Claim your account at [ainative.studio/signup](https://ainative.studio/signup) for permanent access.
+
+## Available Models
+
+```python
+from ragas_ainative import configure_ragas
+
+configure_ragas(model="llama")     # meta-llama/Llama-3.3-70B-Instruct (default)
+configure_ragas(model="qwen")      # qwen3-coder-flash
+configure_ragas(model="deepseek")  # deepseek-4-flash
+configure_ragas(model="kimi")      # kimi-k2
+```
+
+## Store Results to ZeroDB
+
+Track evaluation scores over time:
+
+```python
+from ragas_ainative import configure_ragas, store_results
+
+configure_ragas()
+
+# After evaluation
+results = {"faithfulness": 0.85, "answer_relevancy": 0.92}
+store_results(results, dataset_name="my-rag-pipeline")
+# Stored to ZeroDB — searchable and trackable
+```
+
+## Explicit API Key
+
+```python
+configure_ragas(api_key="your-key-here")
+```
+
+Or set the environment variable:
+
+```bash
+export AINATIVE_API_KEY=your-key-here
+```
+
+## API Key Resolution Order
+
+1. Explicit `api_key` parameter
+2. `AINATIVE_API_KEY` environment variable
+3. `ZERODB_API_KEY` environment variable
+4. `~/.zerodb/credentials.json` (shared with zerodb ecosystem)
+5. Auto-provision via instant-db
+
+## Requirements
+
+- Python >= 3.9
+- ragas >= 0.1.0
+
+## License
+
+MIT

ragas_ainative-0.1.0/ragas_ainative.egg-info/SOURCES.txt

@@ -0,0 +1,12 @@
+LICENSE
+README.md
+pyproject.toml
+ragas_ainative/__init__.py
+ragas_ainative/config.py
+ragas_ainative/provision.py
+ragas_ainative.egg-info/PKG-INFO
+ragas_ainative.egg-info/SOURCES.txt
+ragas_ainative.egg-info/dependency_links.txt
+ragas_ainative.egg-info/requires.txt
+ragas_ainative.egg-info/top_level.txt
+tests/test_config.py

ragas_ainative-0.1.0/ragas_ainative.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

ragas_ainative-0.1.0/ragas_ainative.egg-info/requires.txt

@@ -0,0 +1,5 @@
+requests>=2.28
+
+[dev]
+pytest>=7.0
+pytest-cov>=4.0

ragas_ainative-0.1.0/ragas_ainative.egg-info/top_level.txt

@@ -0,0 +1 @@
+ragas_ainative

ragas_ainative-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

ragas_ainative-0.1.0/tests/test_config.py

@@ -0,0 +1,309 @@
+"""
+Tests for ragas-ainative configuration.
+
+Refs #3954
+"""
+
+import json
+import os
+from unittest.mock import patch, MagicMock
+
+import pytest
+
+from ragas_ainative.config import (
+    configure_ragas,
+    get_llm_config,
+    get_embeddings_config,
+    get_model,
+    store_results,
+    MODELS,
+    DEFAULT_MODEL,
+    API_BASE,
+    EMBEDDINGS_MODEL,
+    MEMORY_API_BASE,
+)
+from ragas_ainative.provision import (
+    resolve_api_key,
+    _load_credentials,
+    _auto_provision,
+    _save_credentials,
+    ZERODB_DIR,
+    CREDS_PATH,
+    CONFIG_PATH,
+)
+
+
+# ---------------------------------------------------------------------------
+# Model resolution
+# ---------------------------------------------------------------------------
+
+class TestGetModel:
+    """Tests for model alias resolution."""
+
+    def test_llama_alias(self):
+        assert get_model("llama") == "meta-llama/Llama-3.3-70B-Instruct"
+
+    def test_qwen_alias(self):
+        assert get_model("qwen") == "qwen3-coder-flash"
+
+    def test_deepseek_alias(self):
+        assert get_model("deepseek") == "deepseek-4-flash"
+
+    def test_kimi_alias(self):
+        assert get_model("kimi") == "kimi-k2"
+
+    def test_passthrough_full_id(self):
+        assert get_model("meta-llama/Llama-3.3-70B-Instruct") == "meta-llama/Llama-3.3-70B-Instruct"
+
+    def test_unknown_returns_as_is(self):
+        assert get_model("custom-model") == "custom-model"
+
+
+# ---------------------------------------------------------------------------
+# LLM config
+# ---------------------------------------------------------------------------
+
+class TestGetLlmConfig:
+    """Tests for LLM judge config generation."""
+
+    @patch("ragas_ainative.config.resolve_api_key", return_value="test-key")
+    def test_default_config(self, mock_resolve):
+        config = get_llm_config()
+        assert config["OPENAI_API_KEY"] == "test-key"
+        assert config["OPENAI_API_BASE"] == API_BASE
+        assert config["model"] == DEFAULT_MODEL
+
+    @patch("ragas_ainative.config.resolve_api_key", return_value="test-key")
+    def test_custom_model(self, mock_resolve):
+        config = get_llm_config(model="qwen")
+        assert config["model"] == "qwen3-coder-flash"
+
+    @patch("ragas_ainative.config.resolve_api_key", return_value="test-key")
+    def test_custom_base(self, mock_resolve):
+        config = get_llm_config(api_base="https://custom.api/v1")
+        assert config["OPENAI_API_BASE"] == "https://custom.api/v1"
+
+
+# ---------------------------------------------------------------------------
+# Embeddings config
+# ---------------------------------------------------------------------------
+
+class TestGetEmbeddingsConfig:
+    """Tests for embeddings config generation."""
+
+    @patch("ragas_ainative.config.resolve_api_key", return_value="emb-key")
+    def test_default_embeddings_config(self, mock_resolve):
+        config = get_embeddings_config()
+        assert config["OPENAI_API_KEY"] == "emb-key"
+        assert config["embeddings_model"] == EMBEDDINGS_MODEL
+
+    @patch("ragas_ainative.config.resolve_api_key", return_value="emb-key")
+    def test_custom_embeddings_model(self, mock_resolve):
+        config = get_embeddings_config(model="custom-emb")
+        assert config["embeddings_model"] == "custom-emb"
+
+
+# ---------------------------------------------------------------------------
+# configure_ragas
+# ---------------------------------------------------------------------------
+
+class TestConfigureRagas:
+    """Tests for the main configure_ragas() function."""
+
+    @patch("ragas_ainative.config.resolve_api_key", return_value="ragas-key-123456789")
+    def test_sets_env_vars(self, mock_resolve):
+        with patch.dict(os.environ, {}, clear=False):
+            result = configure_ragas(verbose=False)
+            assert result == "ragas-key-123456789"
+            assert os.environ["OPENAI_API_KEY"] == "ragas-key-123456789"
+            assert os.environ["OPENAI_API_BASE"] == API_BASE
+            assert os.environ["AINATIVE_API_KEY"] == "ragas-key-123456789"
+
+    @patch("ragas_ainative.config.resolve_api_key", return_value="ragas-key-123456789")
+    def test_verbose_output(self, mock_resolve, capsys):
+        configure_ragas(verbose=True)
+        captured = capsys.readouterr()
+        assert "ragas-ainative configured" in captured.err
+
+    @patch("ragas_ainative.config.resolve_api_key", return_value="ragas-key-123456789")
+    def test_returns_api_key(self, mock_resolve):
+        result = configure_ragas(verbose=False)
+        assert result == "ragas-key-123456789"
+
+
+# ---------------------------------------------------------------------------
+# store_results
+# ---------------------------------------------------------------------------
+
+class TestStoreResults:
+    """Tests for storing evaluation results to ZeroDB."""
+
+    @patch("ragas_ainative.config.resolve_api_key", return_value="store-key")
+    @patch("ragas_ainative.config.requests")
+    def test_store_results_success(self, mock_requests, mock_resolve):
+        mock_resp = MagicMock()
+        mock_resp.status_code = 201
+        mock_resp.json.return_value = {"id": "mem-1", "status": "stored"}
+        mock_requests.post.return_value = mock_resp
+
+        result = store_results(
+            results={"faithfulness": 0.85, "answer_relevancy": 0.92},
+            dataset_name="my-rag-test",
+        )
+
+        assert result is not None
+        assert result["id"] == "mem-1"
+        mock_requests.post.assert_called_once()
+        call_kwargs = mock_requests.post.call_args
+        assert "remember" in call_kwargs[0][0]
+
+    @patch("ragas_ainative.config.resolve_api_key", return_value="store-key")
+    @patch("ragas_ainative.config.requests")
+    def test_store_results_failure_returns_none(self, mock_requests, mock_resolve):
+        mock_resp = MagicMock()
+        mock_resp.status_code = 500
+        mock_requests.post.return_value = mock_resp
+
+        result = store_results(results={"score": 0.5})
+        assert result is None
+
+    @patch("ragas_ainative.config.resolve_api_key", return_value="store-key")
+    @patch("ragas_ainative.config.requests")
+    def test_store_results_network_error_returns_none(self, mock_requests, mock_resolve):
+        import requests as real_requests
+        mock_requests.RequestException = real_requests.RequestException
+        mock_requests.post.side_effect = real_requests.ConnectionError("fail")
+
+        result = store_results(results={"score": 0.5})
+        assert result is None
+
+
+# ---------------------------------------------------------------------------
+# API key resolution
+# ---------------------------------------------------------------------------
+
+class TestResolveApiKey:
+    """Tests for the multi-source API key resolution."""
+
+    def test_explicit_key_takes_priority(self):
+        result = resolve_api_key(explicit_key="explicit-key")
+        assert result == "explicit-key"
+
+    @patch.dict(os.environ, {"AINATIVE_API_KEY": "env-key"}, clear=False)
+    def test_ainative_env_var(self):
+        result = resolve_api_key()
+        assert result == "env-key"
+
+    @patch.dict(os.environ, {"ZERODB_API_KEY": "zerodb-key"}, clear=False)
+    def test_zerodb_env_var(self):
+        env = os.environ.copy()
+        env.pop("AINATIVE_API_KEY", None)
+        with patch.dict(os.environ, env, clear=True):
+            os.environ["ZERODB_API_KEY"] = "zerodb-key"
+            result = resolve_api_key()
+            assert result == "zerodb-key"
+
+
+# ---------------------------------------------------------------------------
+# Credential file loading
+# ---------------------------------------------------------------------------
+
+class TestLoadCredentials:
+    """Tests for credential file loading."""
+
+    def test_returns_none_when_no_files(self, tmp_path):
+        with patch("ragas_ainative.provision.CREDS_PATH", tmp_path / "missing.json"):
+            with patch("ragas_ainative.provision.CONFIG_PATH", tmp_path / "missing2.json"):
+                result = _load_credentials()
+                assert result is None
+
+    def test_reads_credentials_json(self, tmp_path):
+        creds_file = tmp_path / "credentials.json"
+        creds_file.write_text(json.dumps({"api_key": "file-key"}))
+
+        with patch("ragas_ainative.provision.CREDS_PATH", creds_file):
+            with patch("ragas_ainative.provision.CONFIG_PATH", tmp_path / "missing.json"):
+                result = _load_credentials()
+                assert result == "file-key"
+
+
+# ---------------------------------------------------------------------------
+# Auto-provisioning
+# ---------------------------------------------------------------------------
+
+class TestAutoProvision:
+    """Tests for the auto-provisioning endpoint."""
+
+    @patch("ragas_ainative.provision.requests")
+    @patch("ragas_ainative.provision._save_credentials")
+    @patch("ragas_ainative.provision._print_success")
+    def test_successful_provision(self, mock_print, mock_save, mock_requests):
+        mock_resp = MagicMock()
+        mock_resp.status_code = 201
+        mock_resp.json.return_value = {
+            "api_key": "new-key-12345678",
+            "project_id": "proj-1",
+            "claim_url": "https://ainative.studio/claim/abc",
+        }
+        mock_requests.post.return_value = mock_resp
+
+        result = _auto_provision()
+
+        assert result == "new-key-12345678"
+        mock_save.assert_called_once()
+
+    @patch("ragas_ainative.provision.requests")
+    def test_rate_limited(self, mock_requests):
+        import requests as real_requests
+        mock_requests.RequestException = real_requests.RequestException
+        mock_resp = MagicMock()
+        mock_resp.status_code = 429
+        mock_requests.post.return_value = mock_resp
+
+        with pytest.raises(RuntimeError, match="Rate limited"):
+            _auto_provision()
+
+    @patch("ragas_ainative.provision.requests")
+    def test_server_error(self, mock_requests):
+        import requests as real_requests
+        mock_requests.RequestException = real_requests.RequestException
+        mock_resp = MagicMock()
+        mock_resp.status_code = 500
+        mock_requests.post.return_value = mock_resp
+
+        with pytest.raises(RuntimeError, match="Provisioning failed"):
+            _auto_provision()
+
+    @patch("ragas_ainative.provision.requests")
+    def test_network_error(self, mock_requests):
+        import requests as real_requests
+        mock_requests.RequestException = real_requests.RequestException
+        mock_requests.post.side_effect = real_requests.ConnectionError("fail")
+
+        with pytest.raises(RuntimeError, match="Network error"):
+            _auto_provision()
+
+
+# ---------------------------------------------------------------------------
+# Credential persistence
+# ---------------------------------------------------------------------------
+
+class TestSaveCredentials:
+    """Tests for credential persistence."""
+
+    def test_saves_to_file(self, tmp_path):
+        creds_dir = tmp_path / ".zerodb"
+        creds_path = creds_dir / "credentials.json"
+
+        with patch("ragas_ainative.provision.ZERODB_DIR", creds_dir):
+            with patch("ragas_ainative.provision.CREDS_PATH", creds_path):
+                _save_credentials({
+                    "api_key": "saved-key",
+                    "project_id": "proj-1",
+                    "claim_url": "https://ainative.studio/claim/abc",
+                })
+
+        assert creds_path.exists()
+        data = json.loads(creds_path.read_text())
+        assert data["api_key"] == "saved-key"
+        assert data["source"] == "ragas-ainative"