knowledge-rag 3.3.0__tar.gz → 3.3.2__tar.gz

{knowledge_rag-3.3.0 → knowledge_rag-3.3.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: knowledge-rag
-Version: 3.3.0
+Version: 3.3.2
 Summary: Local RAG System for Claude Code — Hybrid search + Cross-encoder Reranking + 12 MCP Tools. Zero external servers.
 Project-URL: Homepage, https://github.com/lyonzin/knowledge-rag
 Project-URL: Repository, https://github.com/lyonzin/knowledge-rag
@@ -38,7 +38,7 @@ Description-Content-Type: text/markdown
 
 <div align="center">
 
-![Version](https://img.shields.io/badge/version-3.3.0-blue.svg)
+![Version](https://img.shields.io/badge/version-3.3.2-blue.svg)
 ![Python](https://img.shields.io/badge/python-3.11%20%7C%203.12-green.svg)
 ![License](https://img.shields.io/badge/license-MIT-yellow.svg)
 ![Platform](https://img.shields.io/badge/platform-Windows%20%7C%20Linux%20%7C%20macOS-lightgrey.svg)
@@ -408,21 +408,23 @@ flowchart LR
 
 ### Quick Start (3 steps)
 
-**Step 1: Clone and install**
+**Step 1: Install**
 
 ```bash
-# Clone to your home directory
+# Option A: pip install (recommended)
+pip install knowledge-rag
+knowledge-rag init              # Exports config template, presets, creates documents/
+
+# Option B: Clone from source
 git clone https://github.com/lyonzin/knowledge-rag.git ~/knowledge-rag
 cd ~/knowledge-rag
-
-# Create virtual environment and install
 python3 -m venv venv
 source venv/bin/activate        # Linux/macOS
 # .\venv\Scripts\activate       # Windows
 pip install -r requirements.txt
 ```
 
-> **Windows users**: `~/knowledge-rag` becomes `C:\Users\YourName\knowledge-rag`. Use `python` instead of `python3`.
+> **Windows users**: Use `python` instead of `python3`.
 
 **Step 2: Configure Claude Code**
 
@@ -1161,7 +1163,7 @@ pip install rank-bm25
 
 ### "ModuleNotFoundError: No module named 'mcp_server'"
 
-This occurs when Claude Code doesn't set the working directory correctly. Use the `cmd /c "cd /d ... && python"` wrapper in your MCP config (see [Installation](#configure-mcp-for-claude-code)).
+This occurs when Claude Code doesn't set the working directory correctly. Use the `cmd /c "cd /d ... && python"` wrapper in your MCP config (see [Installation](#installation)).
 
 ### Dimension mismatch after upgrade
 
@@ -1193,6 +1195,21 @@ With ~200 documents, expect ~300-500MB RAM. The embedding model (~50MB) and rera
 
 ## Changelog
 
+### v3.3.2 (2026-04-06)
+
+- **FIX**: Full type validation on all YAML config values — wrong types warn and fall back to defaults
+- **FIX**: Bounds validation for chunk_size, chunk_overlap, default_results, max_results, embedding_dim
+- **FIX**: `keyword_routes` with string values instead of lists detected and removed with warning
+- **FIX**: `reranker_enabled: "yes"` (string) corrected to boolean with warning
+- **FIX**: Synced version strings across all source files
+- **FIX**: Broken README anchor, duplicate keyword, error handling in `knowledge-rag init`
+
+### v3.3.1 (2026-04-06)
+
+- **FIX**: YAML null values (`category_mappings:` without value) no longer crash the server — falls back to defaults
+- **FIX**: Presets and config template now bundled in pip wheel (`knowledge-rag init` exports them)
+- **NEW**: `knowledge-rag init` CLI command — exports config template, presets, and creates documents/ in current directory
+
 ### v3.3.0 (2026-04-05)
 
 - **NEW**: YAML configuration system — fully customizable via `config.yaml`

{knowledge_rag-3.3.0 → knowledge_rag-3.3.2}/README.md

@@ -2,7 +2,7 @@
 
 <div align="center">
 
-![Version](https://img.shields.io/badge/version-3.3.0-blue.svg)
+![Version](https://img.shields.io/badge/version-3.3.2-blue.svg)
 ![Python](https://img.shields.io/badge/python-3.11%20%7C%203.12-green.svg)
 ![License](https://img.shields.io/badge/license-MIT-yellow.svg)
 ![Platform](https://img.shields.io/badge/platform-Windows%20%7C%20Linux%20%7C%20macOS-lightgrey.svg)
@@ -372,21 +372,23 @@ flowchart LR
 
 ### Quick Start (3 steps)
 
-**Step 1: Clone and install**
+**Step 1: Install**
 
 ```bash
-# Clone to your home directory
+# Option A: pip install (recommended)
+pip install knowledge-rag
+knowledge-rag init              # Exports config template, presets, creates documents/
+
+# Option B: Clone from source
 git clone https://github.com/lyonzin/knowledge-rag.git ~/knowledge-rag
 cd ~/knowledge-rag
-
-# Create virtual environment and install
 python3 -m venv venv
 source venv/bin/activate        # Linux/macOS
 # .\venv\Scripts\activate       # Windows
 pip install -r requirements.txt
 ```
 
-> **Windows users**: `~/knowledge-rag` becomes `C:\Users\YourName\knowledge-rag`. Use `python` instead of `python3`.
+> **Windows users**: Use `python` instead of `python3`.
 
 **Step 2: Configure Claude Code**
 
@@ -1125,7 +1127,7 @@ pip install rank-bm25
 
 ### "ModuleNotFoundError: No module named 'mcp_server'"
 
-This occurs when Claude Code doesn't set the working directory correctly. Use the `cmd /c "cd /d ... && python"` wrapper in your MCP config (see [Installation](#configure-mcp-for-claude-code)).
+This occurs when Claude Code doesn't set the working directory correctly. Use the `cmd /c "cd /d ... && python"` wrapper in your MCP config (see [Installation](#installation)).
 
 ### Dimension mismatch after upgrade
 
@@ -1157,6 +1159,21 @@ With ~200 documents, expect ~300-500MB RAM. The embedding model (~50MB) and rera
 
 ## Changelog
 
+### v3.3.2 (2026-04-06)
+
+- **FIX**: Full type validation on all YAML config values — wrong types warn and fall back to defaults
+- **FIX**: Bounds validation for chunk_size, chunk_overlap, default_results, max_results, embedding_dim
+- **FIX**: `keyword_routes` with string values instead of lists detected and removed with warning
+- **FIX**: `reranker_enabled: "yes"` (string) corrected to boolean with warning
+- **FIX**: Synced version strings across all source files
+- **FIX**: Broken README anchor, duplicate keyword, error handling in `knowledge-rag init`
+
+### v3.3.1 (2026-04-06)
+
+- **FIX**: YAML null values (`category_mappings:` without value) no longer crash the server — falls back to defaults
+- **FIX**: Presets and config template now bundled in pip wheel (`knowledge-rag init` exports them)
+- **NEW**: `knowledge-rag init` CLI command — exports config template, presets, and creates documents/ in current directory
+
 ### v3.3.0 (2026-04-05)
 
 - **NEW**: YAML configuration system — fully customizable via `config.yaml`

{knowledge_rag-3.3.0 → knowledge_rag-3.3.2}/mcp_server/__init__.py

@@ -1,6 +1,6 @@
 """Knowledge RAG MCP Server - Local Retrieval-Augmented Generation System"""
 
-__version__ = "3.2.3"
+__version__ = "3.3.2"
 __author__ = "Ailton Rocha (Lyon.)"
 
 from .config import Config

knowledge_rag-3.3.2/mcp_server/config.py

@@ -0,0 +1,487 @@
+"""Configuration for Knowledge RAG System v3.3.2 — YAML-configurable"""
+
+import os
+from dataclasses import dataclass, field
+from pathlib import Path
+from typing import Dict, List
+
+import yaml
+
+# ============================================================================
+# BASE DIRECTORY RESOLUTION
+# ============================================================================
+# Priority: 1. KNOWLEDGE_RAG_DIR env var  2. Source checkout  3. CWD
+
+_source_dir = Path(__file__).parent.parent
+
+
+_SUPPORTED_SUFFIXES = frozenset([".md", ".txt", ".pdf", ".py", ".json", ".docx", ".xlsx", ".pptx", ".csv"])
+
+
+def _has_documents(path: Path) -> bool:
+    """Check if path has a documents/ dir with actual supported files (follows symlinks)."""
+    docs_dir = path / "documents"
+    if not docs_dir.exists():
+        return False
+    for root, _, files in os.walk(docs_dir, followlinks=True):
+        for f in files:
+            if Path(f).suffix.lower() in _SUPPORTED_SUFFIXES:
+                return True
+    return False
+
+
+if os.environ.get("KNOWLEDGE_RAG_DIR"):
+    BASE_DIR = Path(os.environ["KNOWLEDGE_RAG_DIR"])
+elif _has_documents(_source_dir):
+    BASE_DIR = _source_dir
+elif _has_documents(Path.cwd()):
+    BASE_DIR = Path.cwd()
+else:
+    BASE_DIR = Path.cwd()
+
+
+# ============================================================================
+# YAML CONFIG LOADER
+# ============================================================================
+
+
+def _load_yaml_config() -> dict:
+    """Load config.yaml from BASE_DIR if it exists, otherwise return empty dict."""
+    config_path = BASE_DIR / "config.yaml"
+    if not config_path.exists():
+        return {}
+
+    try:
+        with open(config_path, "r", encoding="utf-8") as f:
+            data = yaml.safe_load(f)
+        if not isinstance(data, dict):
+            print("[WARN] config.yaml is not a valid mapping, ignoring")
+            return {}
+        print(f"[INFO] Loaded config from {config_path}")
+        return data
+    except yaml.YAMLError as e:
+        print(f"[WARN] Failed to parse config.yaml: {e} — using defaults")
+        return {}
+
+
+_yaml = _load_yaml_config()
+
+
+def _get(section: str, key: str, default):
+    """Get a value from the YAML config with section.key path, falling back to default."""
+    s = _yaml.get(section, {})
+    if not isinstance(s, dict):
+        return default
+    val = s.get(key)
+    if val is None:
+        return default
+    # Skip type check when default is None (caller handles validation)
+    if default is None:
+        return val
+    # YAML parses "yes"/"no" as bool, but explicit string "yes" stays str
+    if not isinstance(val, type(default)):
+        print(
+            f"[WARN] config.yaml: {section}.{key} has wrong type "
+            f"(expected {type(default).__name__}, got {type(val).__name__}), using default"
+        )
+        return default
+    return val
+
+
+def _get_top(key: str, default):
+    """Get a top-level value from YAML, falling back to default if missing or None."""
+    val = _yaml.get(key)
+    if val is None:
+        return default
+    if not isinstance(val, dict):
+        print(f"[WARN] config.yaml: {key} has wrong type (expected dict, got {type(val).__name__}), using default")
+        return default
+    return val
+
+
+# ============================================================================
+# DEFAULTS (used when no config.yaml or field is omitted)
+# ============================================================================
+
+_DEFAULT_CATEGORY_MAPPINGS = {
+    "security/redteam": "redteam",
+    "security/blueteam": "blueteam",
+    "security/ctf": "ctf",
+    "security": "security",
+    "aar": "aar",
+    "logscale": "logscale",
+    "development": "development",
+    "general": "general",
+}
+
+_DEFAULT_KEYWORD_ROUTES = {
+    "logscale": [
+        "logscale",
+        "lql",
+        "cql",
+        "humio",
+        "crowdstrike query",
+        "formattime",
+        "groupby",
+        "base64decode",
+        "case{}",
+        "regex",
+    ],
+    "redteam": [
+        "pentest",
+        "exploit",
+        "payload",
+        "reverse shell",
+        "privilege escalation",
+        "lateral movement",
+        "c2",
+        "beacon",
+        "cobalt strike",
+        "metasploit",
+        "gtfobins",
+        "lolbas",
+        "lolbin",
+        "suid",
+        "sudo",
+        "byovd",
+        "lol driver",
+        "lolad",
+        "lolapps",
+        "hacktricks",
+        "privesc",
+        "kerberoast",
+        "dcsync",
+        "golden ticket",
+        "pass-the-hash",
+        "bloodhound",
+        "mimikatz",
+        "rubeus",
+        "certipy",
+        "adcs",
+        "sqli",
+        "xss",
+        "ssti",
+        "ssrf",
+        "lfi",
+        "rfi",
+        "xxe",
+        "deserialization",
+        "ysoserial",
+        "upload bypass",
+        "web shell",
+        "hash cracking",
+        "hashcat",
+        "waf bypass",
+        "amsi bypass",
+        "uac bypass",
+        "potato",
+        "searchsploit",
+        "exploit-db",
+        "cve",
+    ],
+    "blueteam": [
+        "detection",
+        "sigma",
+        "yara",
+        "ioc",
+        "threat hunting",
+        "incident response",
+        "forensics",
+        "malware analysis",
+    ],
+    "ctf": [
+        "ctf",
+        "flag",
+        "hackthebox",
+        "htb",
+        "tryhackme",
+        "picoctf",
+        "writeup",
+        "challenge",
+    ],
+    "development": [
+        "python",
+        "typescript",
+        "javascript",
+        "api",
+        "fastapi",
+        "django",
+        "react",
+        "nodejs",
+    ],
+    "security": [
+        "anti-bot",
+        "antibot",
+        "js challenge",
+        "javascript challenge",
+        "cdp detection",
+        "runtime.enable",
+        "puppeteer",
+        "playwright",
+        "selenium",
+        "nodriver",
+        "stealth",
+        "undetected",
+        "ja3",
+        "ja4",
+        "tls fingerprint",
+        "fingerprinting",
+        "curl_cffi",
+        "got-scraping",
+        "impersonate",
+        "http/2 settings",
+        "browser fingerprint",
+        "canvas fingerprint",
+        "webgl fingerprint",
+        "navigator.webdriver",
+        "audio context",
+        "hardware concurrency",
+        "waf bypass",
+        "aws waf",
+        "cloudflare bypass",
+        "akamai bypass",
+        "datadome",
+        "perimeterx",
+        "imperva bypass",
+        "8kb bypass",
+        "body size limit",
+        "json sqli",
+        "behavioral",
+        "mouse movement",
+        "ghost-cursor",
+        "humanized",
+        "flaresolverr",
+        "turnstile",
+        "rebrowser",
+        "botbrowser",
+    ],
+}
+
+_DEFAULT_QUERY_EXPANSIONS = {
+    "sqli": ["sql injection", "sqli"],
+    "sql injection": ["sql injection", "sqli"],
+    "xss": ["cross-site scripting", "xss"],
+    "cross-site scripting": ["cross-site scripting", "xss"],
+    "ssrf": ["server-side request forgery", "ssrf"],
+    "lfi": ["local file inclusion", "lfi"],
+    "rfi": ["remote file inclusion", "rfi"],
+    "rce": ["remote code execution", "rce"],
+    "xxe": ["xml external entity", "xxe"],
+    "ssti": ["server-side template injection", "ssti"],
+    "idor": ["insecure direct object reference", "idor"],
+    "csrf": ["cross-site request forgery", "csrf"],
+    "privesc": ["privilege escalation", "privesc"],
+    "priv esc": ["privilege escalation", "privesc"],
+    "privilege escalation": ["privilege escalation", "privesc"],
+    "deserialization": ["deserialization", "deserialisation", "insecure deserialization"],
+    "pth": ["pass-the-hash", "pth"],
+    "pass-the-hash": ["pass-the-hash", "pth"],
+    "dcsync": ["dcsync", "dc sync", "domain controller sync"],
+    "kerberoast": ["kerberoasting", "kerberoast"],
+    "kerberoasting": ["kerberoasting", "kerberoast"],
+    "asrep": ["as-rep roasting", "asrep", "asreproast"],
+    "bloodhound": ["bloodhound", "sharphound"],
+    "mimikatz": ["mimikatz", "sekurlsa", "logonpasswords"],
+    "hashcat": ["hashcat", "hash cracking", "hash crack"],
+    "john": ["john the ripper", "john", "jtr"],
+    "revshell": ["reverse shell", "revshell", "rev shell"],
+    "reverse shell": ["reverse shell", "revshell"],
+    "webshell": ["web shell", "webshell"],
+    "web shell": ["web shell", "webshell"],
+    "waf": ["web application firewall", "waf"],
+    "amsi": ["antimalware scan interface", "amsi", "amsi bypass"],
+    "uac": ["user account control", "uac", "uac bypass"],
+    "potato": ["potato", "juicypotato", "sweetpotato", "godpotato", "efspotato", "printspoofer"],
+    "ntlm": ["ntlm", "net-ntlmv2", "ntlmv2"],
+    "smb": ["smb", "server message block", "samba"],
+    "ldap": ["ldap", "lightweight directory access protocol"],
+    "ad": ["active directory", "ad"],
+    "active directory": ["active directory", "ad"],
+    "defender": ["windows defender", "defender", "wdfilter"],
+    "responder": ["responder", "llmnr", "nbt-ns", "netbios"],
+    "suid": ["suid", "setuid", "set-uid"],
+    "cron": ["cron", "crontab", "cronjob", "scheduled task"],
+    "lolbin": ["lolbin", "lolbas", "living off the land"],
+    "c2": ["c2", "command and control", "command-and-control", "beacon"],
+    "sliver": ["sliver", "sliver c2"],
+    "cobalt": ["cobalt strike", "cobalt", "cs beacon"],
+    "phishing": ["phishing", "spearphishing", "social engineering"],
+    "forensics": ["forensics", "forensic", "dfir"],
+    "volatility": ["volatility", "memory forensics", "memory analysis"],
+    "steganography": ["steganography", "stego", "steghide"],
+    "stego": ["steganography", "stego", "steghide"],
+    "rbcd": ["resource-based constrained delegation", "rbcd"],
+    "dpapi": ["dpapi", "data protection api", "credential manager"],
+    "printnightmare": ["printnightmare", "cve-2021-34527", "spoolsv", "printspooler"],
+    "cve-2021-34527": ["printnightmare", "cve-2021-34527", "spoolsv"],
+    "eternalblue": ["eternalblue", "ms17-010", "smbv1"],
+    "ms17-010": ["eternalblue", "ms17-010", "smbv1"],
+    "pwnkit": ["pwnkit", "cve-2021-4034", "pkexec"],
+    "cve-2021-4034": ["pwnkit", "cve-2021-4034", "pkexec"],
+    "log4shell": ["log4shell", "cve-2021-44228", "log4j"],
+    "cve-2021-44228": ["log4shell", "cve-2021-44228", "log4j"],
+    "zerologon": ["zerologon", "cve-2020-1472", "netlogon"],
+    "cve-2020-1472": ["zerologon", "cve-2020-1472", "netlogon"],
+    "petitpotam": ["petitpotam", "cve-2021-36942", "efs", "ntlm relay"],
+    "certifried": ["certifried", "cve-2022-26923", "adcs"],
+    "nopac": ["nopac", "samaccountname", "cve-2021-42278", "cve-2021-42287"],
+    "proxylogon": ["proxylogon", "cve-2021-26855", "exchange"],
+    "proxyshell": ["proxyshell", "cve-2021-34473", "exchange"],
+}
+
+
+# ============================================================================
+# CONFIG DATACLASS
+# ============================================================================
+
+
+def _resolve_path(raw, default: Path) -> Path:
+    """Resolve a path from YAML (string) or use default (Path)."""
+    if raw is None:
+        return default
+    p = Path(raw)
+    if not p.is_absolute():
+        p = BASE_DIR / p
+    return p
+
+
+@dataclass
+class Config:
+    """Central configuration for the RAG system — loads from config.yaml when available."""
+
+    # Paths
+    data_dir: Path = field(default_factory=lambda: _resolve_path(_get("paths", "data_dir", None), BASE_DIR / "data"))
+    chroma_dir: Path = field(
+        default_factory=lambda: _resolve_path(_get("paths", "data_dir", None), BASE_DIR / "data") / "chroma_db"
+    )
+    documents_dir: Path = field(
+        default_factory=lambda: _resolve_path(_get("paths", "documents_dir", None), BASE_DIR / "documents")
+    )
+
+    # Chunking
+    chunk_size: int = field(
+        default_factory=lambda: (
+            _get("documents", "chunking", {}).get("chunk_size", 1000)
+            if isinstance(_get("documents", "chunking", {}), dict)
+            else 1000
+        )
+    )
+    chunk_overlap: int = field(
+        default_factory=lambda: (
+            _get("documents", "chunking", {}).get("chunk_overlap", 200)
+            if isinstance(_get("documents", "chunking", {}), dict)
+            else 200
+        )
+    )
+
+    # Embeddings
+    embedding_model: str = field(
+        default_factory=lambda: (
+            _get("models", "embedding", {}).get("model", "BAAI/bge-small-en-v1.5")
+            if isinstance(_get("models", "embedding", {}), dict)
+            else "BAAI/bge-small-en-v1.5"
+        )
+    )
+    embedding_dim: int = field(
+        default_factory=lambda: (
+            _get("models", "embedding", {}).get("dimensions", 384)
+            if isinstance(_get("models", "embedding", {}), dict)
+            else 384
+        )
+    )
+
+    # Reranker
+    reranker_model: str = field(
+        default_factory=lambda: (
+            _get("models", "reranker", {}).get("model", "Xenova/ms-marco-MiniLM-L-6-v2")
+            if isinstance(_get("models", "reranker", {}), dict)
+            else "Xenova/ms-marco-MiniLM-L-6-v2"
+        )
+    )
+    reranker_enabled: bool = field(
+        default_factory=lambda: (
+            _get("models", "reranker", {}).get("enabled", True)
+            if isinstance(_get("models", "reranker", {}), dict)
+            else True
+        )
+    )
+    reranker_top_k_multiplier: int = field(
+        default_factory=lambda: (
+            _get("models", "reranker", {}).get("top_k_multiplier", 3)
+            if isinstance(_get("models", "reranker", {}), dict)
+            else 3
+        )
+    )
+
+    # ChromaDB
+    collection_name: str = field(default_factory=lambda: _get("search", "collection_name", "knowledge_base"))
+
+    # Supported formats
+    supported_formats: List[str] = field(
+        default_factory=lambda: _get(
+            "documents", "supported_formats", [".md", ".txt", ".pdf", ".py", ".json", ".docx", ".xlsx", ".pptx", ".csv"]
+        )
+    )
+
+    # Category mappings
+    category_mappings: Dict[str, str] = field(
+        default_factory=lambda: _get_top("category_mappings", _DEFAULT_CATEGORY_MAPPINGS)
+    )
+
+    # Keyword routes
+    keyword_routes: Dict[str, List[str]] = field(
+        default_factory=lambda: _get_top("keyword_routes", _DEFAULT_KEYWORD_ROUTES)
+    )
+
+    # Query expansions
+    query_expansions: Dict[str, List[str]] = field(
+        default_factory=lambda: _get_top("query_expansions", _DEFAULT_QUERY_EXPANSIONS)
+    )
+
+    # Search settings
+    default_results: int = field(default_factory=lambda: _get("search", "default_results", 5))
+    max_results: int = field(default_factory=lambda: _get("search", "max_results", 20))
+
+    def __post_init__(self):
+        """Validate config values and ensure directories exist."""
+        # Bounds validation
+        if not isinstance(self.chunk_size, int) or self.chunk_size < 100:
+            print(f"[WARN] chunk_size={self.chunk_size} invalid, using 1000")
+            self.chunk_size = 1000
+        if not isinstance(self.chunk_overlap, int) or self.chunk_overlap < 0:
+            print(f"[WARN] chunk_overlap={self.chunk_overlap} invalid, using 200")
+            self.chunk_overlap = 200
+        if self.chunk_overlap >= self.chunk_size:
+            print(
+                f"[WARN] chunk_overlap ({self.chunk_overlap}) >= chunk_size ({self.chunk_size}), using {self.chunk_size // 5}"
+            )
+            self.chunk_overlap = self.chunk_size // 5
+        if not isinstance(self.default_results, int) or self.default_results < 1:
+            self.default_results = 5
+        if not isinstance(self.max_results, int) or self.max_results < 1:
+            self.max_results = 20
+        if not isinstance(self.embedding_dim, int) or self.embedding_dim < 1:
+            self.embedding_dim = 384
+        if not isinstance(self.reranker_enabled, bool):
+            print(f"[WARN] reranker_enabled={self.reranker_enabled!r} invalid, using True")
+            self.reranker_enabled = True
+        if not isinstance(self.reranker_top_k_multiplier, int) or self.reranker_top_k_multiplier < 1:
+            self.reranker_top_k_multiplier = 3
+        if not isinstance(self.supported_formats, list) or not self.supported_formats:
+            print("[WARN] supported_formats is empty or invalid, using defaults")
+            self.supported_formats = [".md", ".txt", ".pdf", ".py", ".json", ".docx", ".xlsx", ".pptx", ".csv"]
+
+        # Validate keyword_routes values are lists (not strings)
+        for cat, keywords in list(self.keyword_routes.items()):
+            if not isinstance(keywords, list):
+                print(f"[WARN] keyword_routes.{cat} is not a list, removing")
+                del self.keyword_routes[cat]
+
+        # Ensure directories exist
+        self.data_dir.mkdir(parents=True, exist_ok=True)
+        self.chroma_dir.mkdir(parents=True, exist_ok=True)
+        self.documents_dir.mkdir(parents=True, exist_ok=True)
+
+
+# Global config instance
+config = Config()

{knowledge_rag-3.3.0 → knowledge_rag-3.3.2}/mcp_server/server.py

@@ -19,10 +19,8 @@ Features:
     - CRUD operations via MCP tools (add, update, remove docs)
 
 Autor:   Lyon (Ailton Rocha)
-Versao:  3.0.0
-Data:    2026-03-19
-
-By Lyon :) Legal Ne?
+Versao:  3.3.2
+Data:    2026-04-06
 """
 
 import hashlib
@@ -1827,8 +1825,58 @@ def evaluate_retrieval(test_cases: str) -> str:
 # =============================================================================
 
 
+def _handle_init():
+    """Export config template and presets to current directory."""
+    import shutil
+
+    data_dir = Path(__file__).parent / "data"
+    if not data_dir.exists():
+        print("[ERROR] Bundled data not found. If installed from git, use presets/ directly.")
+        return
+
+    cwd = Path.cwd()
+
+    try:
+        # Copy config.example.yaml
+        src = data_dir / "config.example.yaml"
+        if src.exists():
+            dst = cwd / "config.example.yaml"
+            shutil.copy2(src, dst)
+            print(f"[OK] {dst}")
+
+        # Copy presets
+        presets_dir = cwd / "presets"
+        presets_dir.mkdir(exist_ok=True)
+        for f in data_dir.glob("*.yaml"):
+            if f.name == "config.example.yaml":
+                continue
+            dst = presets_dir / f.name
+            shutil.copy2(f, dst)
+            print(f"[OK] {dst}")
+
+        # Create documents dir
+        docs_dir = cwd / "documents"
+        docs_dir.mkdir(exist_ok=True)
+        print(f"[OK] {docs_dir}/")
+
+        print("\nDone. Quick start:")
+        print("  cp presets/general.yaml config.yaml     # or cybersecurity, developer, research")
+        print("  # Add your documents to documents/")
+        print("  # Restart Claude Code")
+    except PermissionError:
+        print("[ERROR] Permission denied. Run from a writable directory.")
+    except OSError as e:
+        print(f"[ERROR] Failed to write files: {e}")
+
+
 def main():
     """Run the MCP server"""
+    import sys
+
+    if len(sys.argv) > 1 and sys.argv[1] == "init":
+        _handle_init()
+        return
+
     orchestrator = get_orchestrator()
 
     # Migration: check dimension mismatch AFTER full init (avoids segfault during __init__)

{knowledge_rag-3.3.0 → knowledge_rag-3.3.2}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "knowledge-rag"
-version = "3.3.0"
+version = "3.3.2"
 description = "Local RAG System for Claude Code — Hybrid search + Cross-encoder Reranking + 12 MCP Tools. Zero external servers."
 readme = "README.md"
 license = {text = "MIT"}
@@ -55,6 +55,13 @@ knowledge-rag = "mcp_server.server:main"
 [tool.hatch.build.targets.wheel]
 packages = ["mcp_server"]
 
+[tool.hatch.build.targets.wheel.force-include]
+"config.example.yaml" = "mcp_server/data/config.example.yaml"
+"presets/cybersecurity.yaml" = "mcp_server/data/cybersecurity.yaml"
+"presets/developer.yaml" = "mcp_server/data/developer.yaml"
+"presets/research.yaml" = "mcp_server/data/research.yaml"
+"presets/general.yaml" = "mcp_server/data/general.yaml"
+
 [tool.hatch.build.targets.sdist]
 include = [
     "mcp_server/",

knowledge_rag-3.3.0/mcp_server/config.py

@@ -1,278 +0,0 @@
-"""Configuration for Knowledge RAG System v3.1 — YAML-configurable"""
-
-import os
-from dataclasses import dataclass, field
-from pathlib import Path
-from typing import Dict, List
-
-import yaml
-
-# ============================================================================
-# BASE DIRECTORY RESOLUTION
-# ============================================================================
-# Priority: 1. KNOWLEDGE_RAG_DIR env var  2. Source checkout  3. CWD
-
-_source_dir = Path(__file__).parent.parent
-
-
-_SUPPORTED_SUFFIXES = frozenset([".md", ".txt", ".pdf", ".py", ".json", ".docx", ".xlsx", ".pptx", ".csv"])
-
-
-def _has_documents(path: Path) -> bool:
-    """Check if path has a documents/ dir with actual supported files (follows symlinks)."""
-    docs_dir = path / "documents"
-    if not docs_dir.exists():
-        return False
-    for root, _, files in os.walk(docs_dir, followlinks=True):
-        for f in files:
-            if Path(f).suffix.lower() in _SUPPORTED_SUFFIXES:
-                return True
-    return False
-
-
-if os.environ.get("KNOWLEDGE_RAG_DIR"):
-    BASE_DIR = Path(os.environ["KNOWLEDGE_RAG_DIR"])
-elif _has_documents(_source_dir):
-    BASE_DIR = _source_dir
-elif _has_documents(Path.cwd()):
-    BASE_DIR = Path.cwd()
-else:
-    BASE_DIR = Path.cwd()
-
-
-# ============================================================================
-# YAML CONFIG LOADER
-# ============================================================================
-
-def _load_yaml_config() -> dict:
-    """Load config.yaml from BASE_DIR if it exists, otherwise return empty dict."""
-    config_path = BASE_DIR / "config.yaml"
-    if not config_path.exists():
-        return {}
-
-    try:
-        with open(config_path, "r", encoding="utf-8") as f:
-            data = yaml.safe_load(f)
-        if not isinstance(data, dict):
-            print(f"[WARN] config.yaml is not a valid mapping, ignoring")
-            return {}
-        print(f"[INFO] Loaded config from {config_path}")
-        return data
-    except yaml.YAMLError as e:
-        print(f"[WARN] Failed to parse config.yaml: {e} — using defaults")
-        return {}
-
-
-_yaml = _load_yaml_config()
-
-
-def _get(section: str, key: str, default):
-    """Get a value from the YAML config with section.key path, falling back to default."""
-    s = _yaml.get(section, {})
-    if not isinstance(s, dict):
-        return default
-    val = s.get(key)
-    return val if val is not None else default
-
-
-# ============================================================================
-# DEFAULTS (used when no config.yaml or field is omitted)
-# ============================================================================
-
-_DEFAULT_CATEGORY_MAPPINGS = {
-    "security/redteam": "redteam",
-    "security/blueteam": "blueteam",
-    "security/ctf": "ctf",
-    "security": "security",
-    "aar": "aar",
-    "logscale": "logscale",
-    "development": "development",
-    "general": "general",
-}
-
-_DEFAULT_KEYWORD_ROUTES = {
-    "logscale": [
-        "logscale", "lql", "cql", "humio", "crowdstrike query",
-        "formattime", "groupby", "base64decode", "case{}", "regex",
-    ],
-    "redteam": [
-        "pentest", "exploit", "payload", "reverse shell", "privilege escalation",
-        "lateral movement", "c2", "beacon", "cobalt strike", "metasploit",
-        "gtfobins", "lolbas", "lolbin", "suid", "sudo", "byovd", "lol driver",
-        "lolad", "lolapps", "hacktricks", "privesc", "kerberoast", "dcsync",
-        "golden ticket", "pass-the-hash", "bloodhound", "mimikatz", "rubeus",
-        "certipy", "adcs", "sqli", "xss", "ssti", "ssrf", "lfi", "rfi", "xxe",
-        "deserialization", "ysoserial", "upload bypass", "reverse shell",
-        "web shell", "hash cracking", "hashcat", "waf bypass", "amsi bypass",
-        "uac bypass", "potato", "searchsploit", "exploit-db", "cve",
-    ],
-    "blueteam": [
-        "detection", "sigma", "yara", "ioc", "threat hunting",
-        "incident response", "forensics", "malware analysis",
-    ],
-    "ctf": [
-        "ctf", "flag", "hackthebox", "htb", "tryhackme",
-        "picoctf", "writeup", "challenge",
-    ],
-    "development": [
-        "python", "typescript", "javascript", "api",
-        "fastapi", "django", "react", "nodejs",
-    ],
-    "security": [
-        "anti-bot", "antibot", "js challenge", "javascript challenge",
-        "cdp detection", "runtime.enable", "puppeteer", "playwright",
-        "selenium", "nodriver", "stealth", "undetected", "ja3", "ja4",
-        "tls fingerprint", "fingerprinting", "curl_cffi", "got-scraping",
-        "impersonate", "http/2 settings", "browser fingerprint",
-        "canvas fingerprint", "webgl fingerprint", "navigator.webdriver",
-        "audio context", "hardware concurrency", "waf bypass", "aws waf",
-        "cloudflare bypass", "akamai bypass", "datadome", "perimeterx",
-        "imperva bypass", "8kb bypass", "body size limit", "json sqli",
-        "behavioral", "mouse movement", "ghost-cursor", "humanized",
-        "flaresolverr", "turnstile", "rebrowser", "botbrowser",
-    ],
-}
-
-_DEFAULT_QUERY_EXPANSIONS = {
-    "sqli": ["sql injection", "sqli"],
-    "sql injection": ["sql injection", "sqli"],
-    "xss": ["cross-site scripting", "xss"],
-    "cross-site scripting": ["cross-site scripting", "xss"],
-    "ssrf": ["server-side request forgery", "ssrf"],
-    "lfi": ["local file inclusion", "lfi"],
-    "rfi": ["remote file inclusion", "rfi"],
-    "rce": ["remote code execution", "rce"],
-    "xxe": ["xml external entity", "xxe"],
-    "ssti": ["server-side template injection", "ssti"],
-    "idor": ["insecure direct object reference", "idor"],
-    "csrf": ["cross-site request forgery", "csrf"],
-    "privesc": ["privilege escalation", "privesc"],
-    "priv esc": ["privilege escalation", "privesc"],
-    "privilege escalation": ["privilege escalation", "privesc"],
-    "deserialization": ["deserialization", "deserialisation", "insecure deserialization"],
-    "pth": ["pass-the-hash", "pth"],
-    "pass-the-hash": ["pass-the-hash", "pth"],
-    "dcsync": ["dcsync", "dc sync", "domain controller sync"],
-    "kerberoast": ["kerberoasting", "kerberoast"],
-    "kerberoasting": ["kerberoasting", "kerberoast"],
-    "asrep": ["as-rep roasting", "asrep", "asreproast"],
-    "bloodhound": ["bloodhound", "sharphound"],
-    "mimikatz": ["mimikatz", "sekurlsa", "logonpasswords"],
-    "hashcat": ["hashcat", "hash cracking", "hash crack"],
-    "john": ["john the ripper", "john", "jtr"],
-    "revshell": ["reverse shell", "revshell", "rev shell"],
-    "reverse shell": ["reverse shell", "revshell"],
-    "webshell": ["web shell", "webshell"],
-    "web shell": ["web shell", "webshell"],
-    "waf": ["web application firewall", "waf"],
-    "amsi": ["antimalware scan interface", "amsi", "amsi bypass"],
-    "uac": ["user account control", "uac", "uac bypass"],
-    "potato": ["potato", "juicypotato", "sweetpotato", "godpotato", "efspotato", "printspoofer"],
-    "ntlm": ["ntlm", "net-ntlmv2", "ntlmv2"],
-    "smb": ["smb", "server message block", "samba"],
-    "ldap": ["ldap", "lightweight directory access protocol"],
-    "ad": ["active directory", "ad"],
-    "active directory": ["active directory", "ad"],
-    "defender": ["windows defender", "defender", "wdfilter"],
-    "responder": ["responder", "llmnr", "nbt-ns", "netbios"],
-    "suid": ["suid", "setuid", "set-uid"],
-    "cron": ["cron", "crontab", "cronjob", "scheduled task"],
-    "lolbin": ["lolbin", "lolbas", "living off the land"],
-    "c2": ["c2", "command and control", "command-and-control", "beacon"],
-    "sliver": ["sliver", "sliver c2"],
-    "cobalt": ["cobalt strike", "cobalt", "cs beacon"],
-    "phishing": ["phishing", "spearphishing", "social engineering"],
-    "forensics": ["forensics", "forensic", "dfir"],
-    "volatility": ["volatility", "memory forensics", "memory analysis"],
-    "steganography": ["steganography", "stego", "steghide"],
-    "stego": ["steganography", "stego", "steghide"],
-    "rbcd": ["resource-based constrained delegation", "rbcd"],
-    "dpapi": ["dpapi", "data protection api", "credential manager"],
-    "printnightmare": ["printnightmare", "cve-2021-34527", "spoolsv", "printspooler"],
-    "cve-2021-34527": ["printnightmare", "cve-2021-34527", "spoolsv"],
-    "eternalblue": ["eternalblue", "ms17-010", "smbv1"],
-    "ms17-010": ["eternalblue", "ms17-010", "smbv1"],
-    "pwnkit": ["pwnkit", "cve-2021-4034", "pkexec"],
-    "cve-2021-4034": ["pwnkit", "cve-2021-4034", "pkexec"],
-    "log4shell": ["log4shell", "cve-2021-44228", "log4j"],
-    "cve-2021-44228": ["log4shell", "cve-2021-44228", "log4j"],
-    "zerologon": ["zerologon", "cve-2020-1472", "netlogon"],
-    "cve-2020-1472": ["zerologon", "cve-2020-1472", "netlogon"],
-    "petitpotam": ["petitpotam", "cve-2021-36942", "efs", "ntlm relay"],
-    "certifried": ["certifried", "cve-2022-26923", "adcs"],
-    "nopac": ["nopac", "samaccountname", "cve-2021-42278", "cve-2021-42287"],
-    "proxylogon": ["proxylogon", "cve-2021-26855", "exchange"],
-    "proxyshell": ["proxyshell", "cve-2021-34473", "exchange"],
-}
-
-
-# ============================================================================
-# CONFIG DATACLASS
-# ============================================================================
-
-def _resolve_path(raw, default: Path) -> Path:
-    """Resolve a path from YAML (string) or use default (Path)."""
-    if raw is None:
-        return default
-    p = Path(raw)
-    if not p.is_absolute():
-        p = BASE_DIR / p
-    return p
-
-
-@dataclass
-class Config:
-    """Central configuration for the RAG system — loads from config.yaml when available."""
-
-    # Paths
-    data_dir: Path = field(default_factory=lambda: _resolve_path(
-        _get("paths", "data_dir", None), BASE_DIR / "data"
-    ))
-    chroma_dir: Path = field(default_factory=lambda: _resolve_path(
-        _get("paths", "data_dir", None), BASE_DIR / "data"
-    ) / "chroma_db")
-    documents_dir: Path = field(default_factory=lambda: _resolve_path(
-        _get("paths", "documents_dir", None), BASE_DIR / "documents"
-    ))
-
-    # Chunking
-    chunk_size: int = field(default_factory=lambda: _get("documents", "chunking", {}).get("chunk_size", 1000) if isinstance(_get("documents", "chunking", {}), dict) else 1000)
-    chunk_overlap: int = field(default_factory=lambda: _get("documents", "chunking", {}).get("chunk_overlap", 200) if isinstance(_get("documents", "chunking", {}), dict) else 200)
-
-    # Embeddings
-    embedding_model: str = field(default_factory=lambda: _get("models", "embedding", {}).get("model", "BAAI/bge-small-en-v1.5") if isinstance(_get("models", "embedding", {}), dict) else "BAAI/bge-small-en-v1.5")
-    embedding_dim: int = field(default_factory=lambda: _get("models", "embedding", {}).get("dimensions", 384) if isinstance(_get("models", "embedding", {}), dict) else 384)
-
-    # Reranker
-    reranker_model: str = field(default_factory=lambda: _get("models", "reranker", {}).get("model", "Xenova/ms-marco-MiniLM-L-6-v2") if isinstance(_get("models", "reranker", {}), dict) else "Xenova/ms-marco-MiniLM-L-6-v2")
-    reranker_enabled: bool = field(default_factory=lambda: _get("models", "reranker", {}).get("enabled", True) if isinstance(_get("models", "reranker", {}), dict) else True)
-    reranker_top_k_multiplier: int = field(default_factory=lambda: _get("models", "reranker", {}).get("top_k_multiplier", 3) if isinstance(_get("models", "reranker", {}), dict) else 3)
-
-    # ChromaDB
-    collection_name: str = field(default_factory=lambda: _get("search", "collection_name", "knowledge_base"))
-
-    # Supported formats
-    supported_formats: List[str] = field(default_factory=lambda: _get("documents", "supported_formats", [".md", ".txt", ".pdf", ".py", ".json", ".docx", ".xlsx", ".pptx", ".csv"]))
-
-    # Category mappings
-    category_mappings: Dict[str, str] = field(default_factory=lambda: _yaml.get("category_mappings", _DEFAULT_CATEGORY_MAPPINGS))
-
-    # Keyword routes
-    keyword_routes: Dict[str, List[str]] = field(default_factory=lambda: _yaml.get("keyword_routes", _DEFAULT_KEYWORD_ROUTES))
-
-    # Query expansions
-    query_expansions: Dict[str, List[str]] = field(default_factory=lambda: _yaml.get("query_expansions", _DEFAULT_QUERY_EXPANSIONS))
-
-    # Search settings
-    default_results: int = field(default_factory=lambda: _get("search", "default_results", 5))
-    max_results: int = field(default_factory=lambda: _get("search", "max_results", 20))
-
-    def __post_init__(self):
-        """Ensure directories exist"""
-        self.data_dir.mkdir(parents=True, exist_ok=True)
-        self.chroma_dir.mkdir(parents=True, exist_ok=True)
-        self.documents_dir.mkdir(parents=True, exist_ok=True)
-
-
-# Global config instance
-config = Config()