ragkitframe 0.1.0__tar.gz

ragkitframe-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Sindhuja Ramaraj
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

ragkitframe-0.1.0/PKG-INFO

@@ -0,0 +1,77 @@
+Metadata-Version: 2.4
+Name: ragkitframe
+Version: 0.1.0
+Summary: A modular, fully local reliability layer for RAG pipelines.
+Author: Sindhuja Ramaraj
+License: MIT
+Project-URL: Homepage, https://github.com/Sindhu06trs/ragkit
+Project-URL: Repository, https://github.com/Sindhu06trs/ragkit
+Keywords: rag,retrieval,llm,nlp,hallucination,local,offline
+Classifier: Programming Language :: Python :: 3
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: spacy>=3.0.0
+Requires-Dist: sentence-transformers>=2.2.0
+Requires-Dist: numpy
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0.0; extra == "dev"
+Requires-Dist: tabulate>=0.9.0; extra == "dev"
+Dynamic: license-file
+
+# ragkitframe
+
+A modular, pip-installable reliability layer for Retrieval-Augmented Generation (RAG) pipelines. 
+
+## Why ragkit?
+- **100% Fully Local**: Requires zero external API calls, zero hosted LLM inference, and runs completely offline.
+- **Privacy-First**: No data leaves your machine; uses local NLP rules and lightweight, open-source sentence-transformers NLI models.
+- **Framework-Agnostic**: Plugs directly into LangChain, LlamaIndex, or any custom pythonic RAG pipeline.
+- **Import-Only-What-You-Need**: Extremely fast import times with lazy loading of all underlying model dependencies on first invocation.
+
+---
+
+## Installation
+
+```bash
+# Install the library in editable/dev mode or directly
+pip install ragkitframe
+
+# Download the required local English spaCy pipeline
+python -m spacy download en_core_web_sm
+```
+
+---
+
+## Quickstart Examples
+
+### 1. Query Decomposition
+Split compound queries into atomic questions before retrieval.
+```python
+from ragkit import QueryDecomposer
+
+decomposer = QueryDecomposer()
+sub_questions = decomposer.decompose("What is climate change and how does it affect oceans?")
+print(sub_questions)
+# Output: ['What is climate change?', 'How does it affect oceans?']
+```
+
+### 2. Confidence Scoring (Hallucination Detection)
+Verify if the generated answer is supported by the retrieved document chunks.
+```python
+from ragkit import ConfidenceScorer
+
+scorer = ConfidenceScorer()  # Uses tiny 100MB model by default
+chunks = ["Photosynthesis uses sunlight to convert water and CO2 into oxygen and glucose."]
+answer = "Plants convert carbon dioxide and water into glucose using sunlight. They also produce helium."
+
+result = scorer.score(answer, chunks)
+print(f"Score: {result['score']}/100 | Verdict: {result['verdict']}")
+# Output: Score: 50/100 | Verdict: partially_grounded
+print("Reasoning:", result["reasoning"])
+
+# For better accuracy, use the larger model:
+# scorer = ConfidenceScorer(model_name="cross-encoder/nli-deberta-v3-base")
+```

ragkitframe-0.1.0/README.md

@@ -0,0 +1,54 @@
+# ragkitframe
+
+A modular, pip-installable reliability layer for Retrieval-Augmented Generation (RAG) pipelines. 
+
+## Why ragkit?
+- **100% Fully Local**: Requires zero external API calls, zero hosted LLM inference, and runs completely offline.
+- **Privacy-First**: No data leaves your machine; uses local NLP rules and lightweight, open-source sentence-transformers NLI models.
+- **Framework-Agnostic**: Plugs directly into LangChain, LlamaIndex, or any custom pythonic RAG pipeline.
+- **Import-Only-What-You-Need**: Extremely fast import times with lazy loading of all underlying model dependencies on first invocation.
+
+---
+
+## Installation
+
+```bash
+# Install the library in editable/dev mode or directly
+pip install ragkitframe
+
+# Download the required local English spaCy pipeline
+python -m spacy download en_core_web_sm
+```
+
+---
+
+## Quickstart Examples
+
+### 1. Query Decomposition
+Split compound queries into atomic questions before retrieval.
+```python
+from ragkit import QueryDecomposer
+
+decomposer = QueryDecomposer()
+sub_questions = decomposer.decompose("What is climate change and how does it affect oceans?")
+print(sub_questions)
+# Output: ['What is climate change?', 'How does it affect oceans?']
+```
+
+### 2. Confidence Scoring (Hallucination Detection)
+Verify if the generated answer is supported by the retrieved document chunks.
+```python
+from ragkit import ConfidenceScorer
+
+scorer = ConfidenceScorer()  # Uses tiny 100MB model by default
+chunks = ["Photosynthesis uses sunlight to convert water and CO2 into oxygen and glucose."]
+answer = "Plants convert carbon dioxide and water into glucose using sunlight. They also produce helium."
+
+result = scorer.score(answer, chunks)
+print(f"Score: {result['score']}/100 | Verdict: {result['verdict']}")
+# Output: Score: 50/100 | Verdict: partially_grounded
+print("Reasoning:", result["reasoning"])
+
+# For better accuracy, use the larger model:
+# scorer = ConfidenceScorer(model_name="cross-encoder/nli-deberta-v3-base")
+```

ragkitframe-0.1.0/pyproject.toml

@@ -0,0 +1,36 @@
+[build-system]
+requires = ["setuptools>=61.0.0"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "ragkitframe"
+version = "0.1.0"
+description = "A modular, fully local reliability layer for RAG pipelines."
+readme = "README.md"
+requires-python = ">=3.9"
+license = { text = "MIT" }
+authors = [
+    { name = "Sindhuja Ramaraj" }
+]
+keywords = ["rag", "retrieval", "llm", "nlp", "hallucination", "local", "offline"]
+urls = { Homepage = "https://github.com/Sindhu06trs/ragkit", Repository = "https://github.com/Sindhu06trs/ragkit" }
+classifiers = [
+    "Programming Language :: Python :: 3",
+    "License :: OSI Approved :: MIT License",
+    "Operating System :: OS Independent",
+]
+dependencies = [
+    "spacy>=3.0.0",
+    "sentence-transformers>=2.2.0",
+    "numpy",
+]
+
+[project.optional-dependencies]
+dev = [
+    "pytest>=7.0.0",
+    "tabulate>=0.9.0",
+]
+
+[tool.setuptools.packages.find]
+where = ["."]
+include = ["ragkit*"]

ragkitframe-0.1.0/ragkit/__init__.py

@@ -0,0 +1,4 @@
+from ragkit.decompose import QueryDecomposer
+from ragkit.score import ConfidenceScorer
+
+__all__ = ["QueryDecomposer", "ConfidenceScorer"]

ragkitframe-0.1.0/ragkit/decompose/__init__.py

@@ -0,0 +1,3 @@
+from .decomposer import QueryDecomposer
+
+__all__ = ["QueryDecomposer"]

ragkitframe-0.1.0/ragkit/decompose/decomposer.py

@@ -0,0 +1,180 @@
+import spacy
+from typing import Callable, List, Dict, Any, Optional
+
+class QueryDecomposer:
+    """
+    A rule-based query decomposer that splits compound or multi-part queries
+    into atomic, independently-retrievable sub-questions using spaCy's dependency parser.
+    """
+    def __init__(self, nlp: Optional[Any] = None) -> None:
+        """
+        Initialize the QueryDecomposer.
+
+        Args:
+            nlp: An optional pre-loaded spaCy pipeline. If None, 'en_core_web_sm'
+                 will be loaded lazily on first use.
+        """
+        self._nlp = nlp
+
+    @property
+    def nlp(self) -> Any:
+        """Lazily load the spaCy pipeline."""
+        if self._nlp is None:
+            try:
+                self._nlp = spacy.load("en_core_web_sm")
+            except OSError as e:
+                raise ImportError(
+                    "spaCy model 'en_core_web_sm' is not installed. "
+                    "Please install it by running:\n"
+                    "    python -m spacy download en_core_web_sm"
+                ) from e
+        return self._nlp
+
+    def _has_subject_and_verb(self, tokens: List[Any]) -> bool:
+        """Check if a list of tokens contains at least one subject and one verb."""
+        has_subj = any(t.dep_ in {"nsubj", "nsubjpass", "csubj", "csubjpass", "expl"} for t in tokens)
+        has_verb = any(t.pos_ in {"VERB", "AUX"} for t in tokens)
+        return has_subj and has_verb
+
+    def _format_question(self, tokens: List[Any], original_ends_with_q: bool) -> str:
+        """Format a list of tokens into a clean, capitalized question sentence."""
+        start_idx = 0
+        while start_idx < len(tokens) and (
+            tokens[start_idx].pos_ in {"PUNCT", "CCONJ"} or 
+            tokens[start_idx].text.lower() in {"and", "also", "but", "or", "plus", "as", "well"}
+        ):
+            start_idx += 1
+
+        end_idx = len(tokens)
+        while end_idx > start_idx and tokens[end_idx - 1].pos_ in {"PUNCT"}:
+            end_idx -= 1
+
+        clause_tokens = tokens[start_idx:end_idx]
+        if not clause_tokens:
+            return ""
+
+        # Construct the text respecting original whitespace
+        text = "".join(t.text + t.whitespace_ for t in clause_tokens).strip()
+        if not text:
+            return ""
+
+        # Capitalize the first letter
+        text = text[0].upper() + text[1:]
+
+        # Ensure correct sentence punctuation
+        if original_ends_with_q and not text[-1] in {".", "?", "!"}:
+            text += "?"
+        elif not text[-1] in {".", "?", "!"}:
+            text += "?"
+
+        return text
+
+    def _split_span(self, span: Any) -> List[List[Any]]:
+        """Recursively split a span of tokens using dependency relations and punctuation."""
+        tokens = list(span)
+        
+        # 1. Split on semicolons
+        for i, token in enumerate(tokens):
+            if token.text == ";":
+                left = tokens[:i]
+                right = tokens[i+1:]
+                if self._has_subject_and_verb(left) and self._has_subject_and_verb(right):
+                    return self._split_span(left) + self._split_span(right)
+
+        # 2. Split on coordinating conjunctions ("and", "also", "but", "or")
+        for i, token in enumerate(tokens):
+            if token.dep_ == "conj" and token.pos_ in {"VERB", "AUX"}:
+                conj_head = token.head
+                if conj_head in tokens:
+                    # Find a coordinating conjunction associated with this conjunct
+                    cc_tokens = [
+                        t for t in tokens
+                        if t.dep_ == "cc" and t.text.lower() in {"and", "also", "but", "or"}
+                        and (t.head == token or t.head == conj_head)
+                    ]
+                    if cc_tokens:
+                        cc_tok = cc_tokens[0]
+                        conj_subtree = list(token.subtree)
+                        right = [t for t in conj_subtree if t in tokens]
+                        left = [t for t in tokens if t not in right]
+                        
+                        # Remove the coordinating conjunction token from both parts
+                        left = [t for t in left if t != cc_tok]
+                        right = [t for t in right if t != cc_tok]
+                        
+                        if self._has_subject_and_verb(left) and self._has_subject_and_verb(right):
+                            return self._split_span(left) + self._split_span(right)
+
+        # 3. Split on commas separating multiple Wh-questions
+        for i, token in enumerate(tokens):
+            if token.text == ",":
+                left = tokens[:i]
+                right = tokens[i+1:]
+                if self._has_subject_and_verb(left) and self._has_subject_and_verb(right):
+                    # Verify if both parts represent separate Wh-questions
+                    wh_words = {"what", "who", "whom", "whose", "which", "where", "when", "why", "how"}
+                    has_wh_left = any(t.tag_.startswith("W") or t.text.lower() in wh_words for t in left)
+                    has_wh_right = any(t.tag_.startswith("W") or t.text.lower() in wh_words for t in right[:3])
+                    
+                    if has_wh_left and has_wh_right:
+                        return self._split_span(left) + self._split_span(right)
+
+        # 4. Split on "as well as" separating clauses
+        for i in range(len(tokens) - 2):
+            t1, t2, t3 = tokens[i], tokens[i+1], tokens[i+2]
+            if t1.text.lower() == "as" and t2.text.lower() == "well" and t3.text.lower() == "as":
+                left = tokens[:i]
+                right = tokens[i+3:]
+                if self._has_subject_and_verb(left) and self._has_subject_and_verb(right):
+                    return self._split_span(left) + self._split_span(right)
+
+        return [tokens]
+
+    def decompose(self, query: str) -> List[str]:
+        """
+        Split a compound or multi-part query into 2-5 atomic sub-questions.
+
+        Args:
+            query: The input compound question string.
+
+        Returns:
+            A list of standalone sub-question strings. If no split is found,
+            returns the original query in a single-item list.
+        """
+        query = query.strip()
+        if not query:
+            return []
+
+        doc = self.nlp(query)
+        original_ends_with_q = query.endswith("?")
+
+        sub_questions = []
+        for sent in doc.sents:
+            fragments = self._split_span(sent)
+            for frag in fragments:
+                formatted = self._format_question(frag, original_ends_with_q)
+                if formatted:
+                    sub_questions.append(formatted)
+
+        # Validate that we actually found a split and didn't exceed limits
+        if len(sub_questions) <= 1 or len(sub_questions) > 5:
+            return [query]
+
+        return sub_questions
+
+    def decompose_and_retrieve(self, query: str, retriever_fn: Callable[[str], List[Any]]) -> Dict[str, List[Any]]:
+        """
+        Decompose a query into sub-questions and retrieve document chunks for each.
+
+        Args:
+            query: The input compound question string.
+            retriever_fn: A callable function taking a query string and returning a list of retrieved chunks.
+
+        Returns:
+            A dictionary mapping each sub-question to its list of retrieved chunks.
+        """
+        sub_questions = self.decompose(query)
+        results = {}
+        for sub_q in sub_questions:
+            results[sub_q] = retriever_fn(sub_q)
+        return results

ragkitframe-0.1.0/ragkit/score/__init__.py

@@ -0,0 +1,3 @@
+from .confidence import ConfidenceScorer
+
+__all__ = ["ConfidenceScorer"]

ragkitframe-0.1.0/ragkit/score/confidence.py

@@ -0,0 +1,241 @@
+import numpy as np
+from typing import List, Dict, Any, Optional
+
+def _softmax(x: np.ndarray) -> np.ndarray:
+    """Compute softmax values for each sets of scores in x in a numerically stable way."""
+    e_x = np.exp(x - np.max(x, axis=-1, keepdims=True))
+    return e_x / e_x.sum(axis=-1, keepdims=True)
+
+class ConfidenceScorer:
+    """
+    A scorer that evaluates the confidence of generated RAG answers against retrieved chunks
+    using a local NLI (Natural Language Inference) CrossEncoder model.
+    """
+    def __init__(self, model_name: str = "cross-encoder/nli-MiniLM2-L6-H768", threshold: float = 0.5) -> None:
+        """
+        Initialize the ConfidenceScorer.
+
+        Args:
+            model_name: The local/HuggingFace model name of the CrossEncoder NLI model.
+            threshold: Entailment probability threshold below which a claim is unsupported.
+        """
+        self.model_name = model_name
+        self.threshold = threshold
+        self._model = None
+        self._nlp = None
+
+    @property
+    def model(self) -> Any:
+        """Lazily load the sentence-transformers CrossEncoder model."""
+        if self._model is None:
+            from sentence_transformers import CrossEncoder
+            self._model = CrossEncoder(self.model_name)
+        return self._model
+
+    @property
+    def nlp(self) -> Any:
+        """Lazily load the spaCy pipeline."""
+        if self._nlp is None:
+            import spacy
+            try:
+                self._nlp = spacy.load("en_core_web_sm")
+            except OSError as e:
+                raise ImportError(
+                    "spaCy model 'en_core_web_sm' is not installed. "
+                    "Please install it by running:\n"
+                    "    python -m spacy download en_core_web_sm"
+                ) from e
+        return self._nlp
+
+    def score(self, answer: str, retrieved_chunks: List[str]) -> Dict[str, Any]:
+        """
+        Score a single answer against a list of retrieved chunks.
+
+        Args:
+            answer: The generated answer string.
+            retrieved_chunks: A list of context chunks retrieved for the query.
+
+        Returns:
+            A dictionary containing the score, verdict, unsupported claims, and reasoning.
+        """
+        answer = answer.strip()
+        if not answer:
+            return {
+                "score": 0,
+                "verdict": "hallucinated",
+                "unsupported_claims": [],
+                "reasoning": "Answer is empty."
+            }
+
+        # Segment answer into sentences
+        doc = self.nlp(answer)
+        sentences = [sent.text.strip() for sent in doc.sents if sent.text.strip()]
+        if not sentences:
+            return {
+                "score": 0,
+                "verdict": "hallucinated",
+                "unsupported_claims": [],
+                "reasoning": "No valid sentences found in answer."
+            }
+
+        if not retrieved_chunks:
+            return {
+                "score": 0,
+                "verdict": "hallucinated",
+                "unsupported_claims": sentences,
+                "reasoning": "No retrieved chunks provided."
+            }
+
+        # Prepare all premise-hypothesis pairs (chunk is premise, sentence is hypothesis)
+        pairs = [(chunk, sentence) for sentence in sentences for chunk in retrieved_chunks]
+        
+        # Predict NLI probabilities
+        logits = self.model.predict(pairs)
+        probs = _softmax(logits)
+        
+        # Entailment probability is at index 1 for cross-encoder/nli-deberta-v3-base
+        entailment_probs = probs[:, 1].reshape(len(sentences), len(retrieved_chunks))
+        
+        # Max entailment score per sentence across chunks
+        max_entailment_scores = np.max(entailment_probs, axis=1)
+        
+        # Evaluate support
+        unsupported_claims = []
+        supported_count = 0
+        for sentence, score in zip(sentences, max_entailment_scores):
+            if score < self.threshold:
+                unsupported_claims.append(sentence)
+            else:
+                supported_count += 1
+                
+        # Aggregate to 0-100 score
+        aggregated_score = int(round(float(np.mean(max_entailment_scores)) * 100))
+        
+        # Compute verdict
+        if supported_count == len(sentences):
+            verdict = "grounded"
+        elif supported_count == 0:
+            verdict = "hallucinated"
+        else:
+            verdict = "partially_grounded"
+            
+        # Build explanation reasoning
+        reasoning = f"{supported_count} of {len(sentences)} sentences had entailment above threshold {self.threshold:.2f}."
+        if unsupported_claims:
+            unsupported_str = "; ".join(f"'{claim}'" for claim in unsupported_claims)
+            reasoning += f" Unsupported sentences: {unsupported_str}."
+            
+        return {
+            "score": aggregated_score,
+            "verdict": verdict,
+            "unsupported_claims": unsupported_claims,
+            "reasoning": reasoning
+        }
+
+    def batch_score(self, answers: List[str], chunks_list: List[List[str]]) -> List[Dict[str, Any]]:
+        """
+        Score a batch of answers against their corresponding list of retrieved chunks
+        using a single model prediction pass for maximum efficiency.
+
+        Args:
+            answers: A list of generated answer strings.
+            chunks_list: A list of lists of retrieved chunks for each answer.
+
+        Returns:
+            A list of dictionary results, each containing score, verdict, unsupported claims, and reasoning.
+        """
+        all_pairs = []
+        qa_sentence_lists = []
+        pair_ranges = []
+
+        for ans, chunks in zip(answers, chunks_list):
+            ans = ans.strip()
+            if not ans:
+                qa_sentence_lists.append([])
+                pair_ranges.append((0, 0))
+                continue
+                
+            doc = self.nlp(ans)
+            sentences = [sent.text.strip() for sent in doc.sents if sent.text.strip()]
+            qa_sentence_lists.append(sentences)
+            
+            if not sentences or not chunks:
+                pair_ranges.append((0, 0))
+                continue
+                
+            start_idx = len(all_pairs)
+            for sent in sentences:
+                for chunk in chunks:
+                    all_pairs.append((chunk, sent))
+            end_idx = len(all_pairs)
+            pair_ranges.append((start_idx, end_idx))
+
+        # Predict all pairs in a single forward pass
+        if all_pairs:
+            logits = self.model.predict(all_pairs)
+            probs = _softmax(logits)
+            entailment_probs = probs[:, 1]
+        else:
+            entailment_probs = np.array([])
+
+        results = []
+        for i, (ans, chunks) in enumerate(zip(answers, chunks_list)):
+            sentences = qa_sentence_lists[i]
+            if not ans or not sentences:
+                results.append({
+                    "score": 0,
+                    "verdict": "hallucinated",
+                    "unsupported_claims": [],
+                    "reasoning": "Answer is empty or contains no valid sentences."
+                })
+                continue
+                
+            if not chunks:
+                results.append({
+                    "score": 0,
+                    "verdict": "hallucinated",
+                    "unsupported_claims": sentences,
+                    "reasoning": "No retrieved chunks provided."
+                })
+                continue
+
+            start_idx, end_idx = pair_ranges[i]
+            qa_entailment = entailment_probs[start_idx:end_idx].reshape(len(sentences), len(chunks))
+            
+            # Max entailment score per sentence across chunks
+            max_entailment_scores = np.max(qa_entailment, axis=1)
+            
+            # Evaluate support
+            unsupported_claims = []
+            supported_count = 0
+            for sentence, score in zip(sentences, max_entailment_scores):
+                if score < self.threshold:
+                    unsupported_claims.append(sentence)
+                else:
+                    supported_count += 1
+                    
+            # Aggregate to 0-100 score
+            aggregated_score = int(round(float(np.mean(max_entailment_scores)) * 100))
+            
+            # Compute verdict
+            if supported_count == len(sentences):
+                verdict = "grounded"
+            elif supported_count == 0:
+                verdict = "hallucinated"
+            else:
+                verdict = "partially_grounded"
+                
+            # Build explanation reasoning
+            reasoning = f"{supported_count} of {len(sentences)} sentences had entailment above threshold {self.threshold:.2f}."
+            if unsupported_claims:
+                unsupported_str = "; ".join(f"'{claim}'" for claim in unsupported_claims)
+                reasoning += f" Unsupported sentences: {unsupported_str}."
+                
+            results.append({
+                "score": aggregated_score,
+                "verdict": verdict,
+                "unsupported_claims": unsupported_claims,
+                "reasoning": reasoning
+            })
+
+        return results

ragkitframe-0.1.0/ragkitframe.egg-info/PKG-INFO

@@ -0,0 +1,77 @@
+Metadata-Version: 2.4
+Name: ragkitframe
+Version: 0.1.0
+Summary: A modular, fully local reliability layer for RAG pipelines.
+Author: Sindhuja Ramaraj
+License: MIT
+Project-URL: Homepage, https://github.com/Sindhu06trs/ragkit
+Project-URL: Repository, https://github.com/Sindhu06trs/ragkit
+Keywords: rag,retrieval,llm,nlp,hallucination,local,offline
+Classifier: Programming Language :: Python :: 3
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: spacy>=3.0.0
+Requires-Dist: sentence-transformers>=2.2.0
+Requires-Dist: numpy
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0.0; extra == "dev"
+Requires-Dist: tabulate>=0.9.0; extra == "dev"
+Dynamic: license-file
+
+# ragkitframe
+
+A modular, pip-installable reliability layer for Retrieval-Augmented Generation (RAG) pipelines. 
+
+## Why ragkit?
+- **100% Fully Local**: Requires zero external API calls, zero hosted LLM inference, and runs completely offline.
+- **Privacy-First**: No data leaves your machine; uses local NLP rules and lightweight, open-source sentence-transformers NLI models.
+- **Framework-Agnostic**: Plugs directly into LangChain, LlamaIndex, or any custom pythonic RAG pipeline.
+- **Import-Only-What-You-Need**: Extremely fast import times with lazy loading of all underlying model dependencies on first invocation.
+
+---
+
+## Installation
+
+```bash
+# Install the library in editable/dev mode or directly
+pip install ragkitframe
+
+# Download the required local English spaCy pipeline
+python -m spacy download en_core_web_sm
+```
+
+---
+
+## Quickstart Examples
+
+### 1. Query Decomposition
+Split compound queries into atomic questions before retrieval.
+```python
+from ragkit import QueryDecomposer
+
+decomposer = QueryDecomposer()
+sub_questions = decomposer.decompose("What is climate change and how does it affect oceans?")
+print(sub_questions)
+# Output: ['What is climate change?', 'How does it affect oceans?']
+```
+
+### 2. Confidence Scoring (Hallucination Detection)
+Verify if the generated answer is supported by the retrieved document chunks.
+```python
+from ragkit import ConfidenceScorer
+
+scorer = ConfidenceScorer()  # Uses tiny 100MB model by default
+chunks = ["Photosynthesis uses sunlight to convert water and CO2 into oxygen and glucose."]
+answer = "Plants convert carbon dioxide and water into glucose using sunlight. They also produce helium."
+
+result = scorer.score(answer, chunks)
+print(f"Score: {result['score']}/100 | Verdict: {result['verdict']}")
+# Output: Score: 50/100 | Verdict: partially_grounded
+print("Reasoning:", result["reasoning"])
+
+# For better accuracy, use the larger model:
+# scorer = ConfidenceScorer(model_name="cross-encoder/nli-deberta-v3-base")
+```

ragkitframe-0.1.0/ragkitframe.egg-info/SOURCES.txt

@@ -0,0 +1,16 @@
+LICENSE
+README.md
+pyproject.toml
+setup.py
+ragkit/__init__.py
+ragkit/decompose/__init__.py
+ragkit/decompose/decomposer.py
+ragkit/score/__init__.py
+ragkit/score/confidence.py
+ragkitframe.egg-info/PKG-INFO
+ragkitframe.egg-info/SOURCES.txt
+ragkitframe.egg-info/dependency_links.txt
+ragkitframe.egg-info/requires.txt
+ragkitframe.egg-info/top_level.txt
+tests/test_decompose.py
+tests/test_score.py

ragkitframe-0.1.0/ragkitframe.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

ragkitframe-0.1.0/ragkitframe.egg-info/requires.txt

@@ -0,0 +1,7 @@
+spacy>=3.0.0
+sentence-transformers>=2.2.0
+numpy
+
+[dev]
+pytest>=7.0.0
+tabulate>=0.9.0

ragkitframe-0.1.0/ragkitframe.egg-info/top_level.txt

@@ -0,0 +1 @@
+ragkit

ragkitframe-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

ragkitframe-0.1.0/setup.py

@@ -0,0 +1,3 @@
+from setuptools import setup
+
+setup()

ragkitframe-0.1.0/tests/test_decompose.py

@@ -0,0 +1,56 @@
+import pytest
+from unittest.mock import patch
+from ragkit import QueryDecomposer
+
+def test_query_decomposer_simple():
+    decomposer = QueryDecomposer()
+    query = "What is the capital of France?"
+    res = decomposer.decompose(query)
+    assert res == [query]
+
+def test_query_decomposer_compound():
+    decomposer = QueryDecomposer()
+    query = "What is climate change and how does it affect oceans?"
+    res = decomposer.decompose(query)
+    assert len(res) == 2
+    assert "What is climate change?" in res
+    assert "How does it affect oceans?" in res
+
+def test_query_decomposer_multiple_wh():
+    decomposer = QueryDecomposer()
+    query = "What is the history of AI, and who invented it?"
+    res = decomposer.decompose(query)
+    assert len(res) == 2
+    assert "What is the history of AI?" in res
+    assert "Who invented it?" in res
+
+def test_query_decomposer_semicolon():
+    decomposer = QueryDecomposer()
+    query = "Where is Mount Everest; how high is it?"
+    res = decomposer.decompose(query)
+    assert len(res) == 2
+    assert "Where is Mount Everest?" in res
+    assert "How high is it?" in res
+
+def test_decompose_and_retrieve():
+    decomposer = QueryDecomposer()
+    query = "What is climate change and how does it affect oceans?"
+    
+    called_queries = []
+    def mock_retriever(q):
+        called_queries.append(q)
+        return [f"chunk for {q}"]
+        
+    retrieved = decomposer.decompose_and_retrieve(query, mock_retriever)
+    assert len(called_queries) == 2
+    assert "What is climate change?" in retrieved
+    assert "How does it affect oceans?" in retrieved
+    assert retrieved["What is climate change?"] == ["chunk for What is climate change?"]
+
+def test_spacy_not_installed_error():
+    with patch("spacy.load") as mock_load:
+        mock_load.side_effect = OSError("Model not found")
+        decomposer = QueryDecomposer()
+        with pytest.raises(ImportError) as exc_info:
+            _ = decomposer.nlp
+        assert "spaCy model 'en_core_web_sm' is not installed" in str(exc_info.value)

ragkitframe-0.1.0/tests/test_score.py

@@ -0,0 +1,61 @@
+import pytest
+from ragkit import ConfidenceScorer
+
+def test_confidence_scorer_grounded():
+    scorer = ConfidenceScorer()
+    chunks = ["The capital of France is Paris. It is known for Eiffel Tower."]
+    answer = "Paris is the capital of France. It features the Eiffel Tower."
+    
+    res = scorer.score(answer, chunks)
+    assert res["score"] > 80
+    assert res["verdict"] == "grounded"
+    assert len(res["unsupported_claims"]) == 0
+    assert "2 of 2 sentences had entailment above threshold" in res["reasoning"]
+
+def test_confidence_scorer_hallucinated():
+    scorer = ConfidenceScorer()
+    chunks = ["The capital of France is Paris."]
+    answer = "The capital of Italy is Rome."
+    
+    res = scorer.score(answer, chunks)
+    assert res["score"] < 30
+    assert res["verdict"] == "hallucinated"
+    assert len(res["unsupported_claims"]) == 1
+    assert "0 of 1 sentences had entailment above threshold" in res["reasoning"]
+
+def test_confidence_scorer_partial():
+    scorer = ConfidenceScorer()
+    chunks = ["The capital of France is Paris."]
+    answer = "Paris is France's capital. Berlin is Germany's capital."
+    
+    res = scorer.score(answer, chunks)
+    assert res["verdict"] == "partially_grounded"
+    assert len(res["unsupported_claims"]) == 1
+    assert "Berlin is Germany's capital." in res["unsupported_claims"]
+    assert "1 of 2 sentences had entailment above threshold" in res["reasoning"]
+
+def test_confidence_scorer_batch():
+    scorer = ConfidenceScorer()
+    chunks1 = ["The capital of France is Paris."]
+    answer1 = "Paris is the capital of France."
+    
+    chunks2 = ["The capital of Germany is Berlin."]
+    answer2 = "The capital of Italy is Rome."
+    
+    batch_res = scorer.batch_score([answer1, answer2], [chunks1, chunks2])
+    assert len(batch_res) == 2
+    assert batch_res[0]["verdict"] == "grounded"
+    assert batch_res[1]["verdict"] == "hallucinated"
+
+def test_confidence_scorer_empty():
+    scorer = ConfidenceScorer()
+    res = scorer.score("", ["some chunk"])
+    assert res["score"] == 0
+    assert res["verdict"] == "hallucinated"
+    assert res["reasoning"] == "Answer is empty."
+    
+    res = scorer.score("Some answer.", [])
+    assert res["score"] == 0
+    assert res["verdict"] == "hallucinated"
+    assert res["unsupported_claims"] == ["Some answer."]
+    assert res["reasoning"] == "No retrieved chunks provided."