knowledge-rag 4.3.0__tar.gz → 4.4.0__tar.gz

{knowledge_rag-4.3.0 → knowledge_rag-4.4.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: knowledge-rag
-Version: 4.3.0
+Version: 4.4.0
 Summary: Local RAG System for Claude Code — Hybrid search + Cross-encoder Reranking + 13 MCP Tools + 20 Format Parsers. Zero external servers.
 Project-URL: Homepage, https://github.com/lyonzin/knowledge-rag
 Project-URL: Repository, https://github.com/lyonzin/knowledge-rag
@@ -1447,6 +1447,79 @@ Common issues:
 
 ## Changelog
 
+### Unreleased
+
+### v4.4.0 (2026-07-06) — Cross-Platform Installer & Hybrid Search Category Filter
+
+- **NEW**: Cross-platform, multi-LLM-client installer (`install.py`) driving both `install.sh` (Linux/macOS) and `install.ps1` (Windows) as thin wrappers. One codebase, one behavior across every OS.
+- **NEW**: Auto-detects and registers `knowledge-rag` in 8 LLM clients — Claude Code, Claude Desktop, Cursor, Windsurf, VS Code (Copilot Chat), Cline, Gemini CLI, Zed — writing to each tool's canonical config path with the correct JSON schema per client (VS Code uses `servers`, Zed uses `context_servers`, everyone else uses `mcpServers`).
+- **NEW**: `--for <clients>` / `--exclude <clients>` opt-in/opt-out selection, `--dry-run` preview, `--list-clients` registry inspection, `--pypi-version <ver>` pinning, `--skip-init` / `--skip-model` for fast reruns. See `python install.py --help`.
+- **FIX**: `install.ps1` no longer writes MCP config to `~/.claude/mcp.json` (a stale secondary path); it now targets `~/.claude.json` — the file Claude Code actually reads — via idempotent JSON merge that preserves every existing MCP server entry with an automatic `.knowledge-rag.bak` backup.
+- **FIX**: `install.ps1` gains PyPI mode (`pip install knowledge-rag`) and runs `mcp_server.server init` on install — feature parity with `install.sh`.
+- **FIX**: MCP server spec no longer uses the fragile `cmd /c cd /d ... && python ...` wrapper on Windows; it emits the standard `command` + `cwd` shape supported natively by every modern client.
+- **FIX**: `install.sh` guards against `sh install.sh` (bash-only features now emit a clear error instead of a cryptic syntax failure).
+- **FIX**: Both scripts now correctly advertise **13 MCP tools** (was outdated at 12; `get_reindex_status` shipped in v4.3.0).
+- **FIX**: Windows-side `install.ps1` prefers `winget install Python.Python.3.12 --scope user` (no admin), falls back to python.org 3.12.7 (was pinned to 3.12.0).
+- **FIX**: Hybrid search now applies the active category filter to BM25 results before RRF fusion, preventing keyword-only BM25 hits from other categories from leaking into filtered searches. Both leak paths are closed: BM25 candidates are metadata-filtered before RRF (with `top_k` widened to `max_results * 20` to compensate for post-filter drop), and the fallback fetch during fusion re-checks `category` before adding a chunk to `combined_scores`. Note: the `category_filter` guard now also applies to the keyword-routed category — `_route_by_keywords()`-inferred routing filters BM25 too, consistent with the semantic branch. Users with warm `query_cache` entries should restart the server to invalidate stale results. (#109, thanks @Hohlas)
+- **TEST**: New `tests/test_installer_no_data_loss.py` (22 tests) locks in the installer's zero-data-loss contract across all three JSON schemas (`mcpServers` / `servers` / `context_servers`): top-level keys preserved, sibling MCP servers byte-identical, `.knowledge-rag.bak` backup written before every mutation, idempotent second run, `--dry-run` writes nothing, atomic `os.replace` write. Baseline: 231 → 266.
+- **TEST**: New `TestHybridCategoryFilter::test_bm25_results_respect_category_filter` in `tests/test_search.py` — deterministic regression covering BM25-only hits with mixed categories. Baseline: 266 → 267.
+
+### v4.3.1 (2026-06-22) — Hybrid Search Fixes
+
+- **FIX**: Accept `"general"` as a valid category in `search_knowledge`. The parser hardcodes `"general"` as the fallback in `_detect_category` (`ingestion.py`), but the validator only built `valid_categories` from `config.keyword_routes` + `config.category_mappings.values()` — so users who customized `config.yaml` and dropped the default `"general": "general"` mapping hit `Invalid category` even though the index contained `general` documents. Validator now always tolerates `"general"`. (#98, thanks @Hohlas)
+- **FIX**: Skip BM25-only search results when Chroma can no longer resolve the chunk ID. Stale BM25 indices (typically right after `remove_document` or in the window between async reindex and BM25 rebuild) returned hits whose `collection.get()` came back empty; the previous fallback inserted entries with `document=""` / `metadata={}` into the reranker, polluting results with empty matches. The pipeline now `continue`s past those, dropping the stale hit cleanly. (#98, thanks @Hohlas)
+- **TEST**: Added `tests/test_pr98_regression.py` (4 tests) pinning both contracts so future refactors cannot silently revert either fix. Test count baseline: 227 → 231. (#99)
+- **CI**: Bumped `[tool.mypy] python_version` from 3.11 to 3.12 to accept PEP 695 `type` statements in the numpy stub (`numpy/__init__.pyi`) which were breaking the Pillar 7 strict gate. Only affects static analysis; `requires-python = ">=3.11"` unchanged. (#100)
+
+### v4.3.0 (2026-06-17) — Async Reindex, GPU CUDA 12, 13th MCP Tool
+
+- **NEW**: `get_reindex_status` MCP tool — lightweight reindex progress polling without computing full index stats. Returns active/idle status, percent, processed/total, errors, and last result.
+- **NEW**: `reindex_documents` now runs in background via daemon thread — returns immediately with `{"status": "started"}`. Eliminates MCP timeout on large document sets (5K+ files). Concurrent calls return `already_running` with current progress.
+- **NEW**: GPU acceleration with full CUDA 12 support — `onnxruntime-gpu` + 7 NVIDIA pip packages (`cublas`, `cudnn`, `cuda-runtime`, `cufft`, `cusparse`, `cusolver`, `curand`, `nvjitlink`). Server auto-detects GPU on startup with 4-step verification (providers, DLLs, nvidia-smi, session creation). Falls back to CPU gracefully.
+- **NEW**: `_setup_cuda_dll_paths()` adds NVIDIA pip package DLL directories to `PATH` automatically on Windows — onnxruntime finds CUDA 12 DLLs without a full CUDA Toolkit install.
+- **DEPS**: `[gpu]` extra expanded from 3 to 8 packages (added `cufft`, `cusparse`, `cusolver`, `curand`, `nvjitlink`).
+- **FIX**: GPU status reporting now uses actual ONNX session creation test instead of just checking `get_available_providers()` — prevents false "GPU ACTIVE" when CUDA DLLs are missing.
+- **DOCS**: GPU Acceleration section rewritten with complete requirements table, setup steps, verification instructions, and fallback behavior.
+- **DOCS**: Tool reference updated — `reindex_documents` async behavior documented, `get_reindex_status` reference added.
+- **TEST**: Backwards-compat baseline updated for 13 MCP tools.
+
+### v4.2.0 (2026-06-17) — Search Performance & Output Quality
+
+- **PERF**: Custom inverted-index BM25 replaces `rank-bm25` full-corpus scan — 128× faster keyword search on 50K+ chunk corpora. Only documents containing query terms are scored via posting lists.
+- **PERF**: `numpy.argpartition` for O(n) top-k selection instead of O(n log n) sort.
+- **PERF**: Batched adjacent chunk fetch — single ChromaDB `collection.get()` call replaces N round-trips per result.
+- **PERF**: O(1) reverse lookup via `_source_to_docid` dict eliminates linear scans of `_indexed_docs` in `search_similar`, `update_document`, `remove_document`, and `_expand_with_adjacent_chunks`.
+- **NEW**: `snippet_mode` parameter on `search_knowledge` (default: `true`) — truncates content to ~500 chars at natural break points with `content_length` field. Reduces token consumption by ~72%.
+- **NEW**: `min_score` parameter on `search_knowledge` (default: `0.0`) — filters results below a normalized relevance threshold. Response includes `filtered_by_score` count.
+- **NEW**: `filtered_by_score` field in search response JSON for transparency.
+- **DEPS**: `numpy` added as direct dependency (was transitive via fastembed); `rank-bm25` import removed from server.py.
+- **TEST**: 6 new tests for `min_score` filtering and `snippet_mode` truncation.
+- **TEST**: Updated backwards-compat baseline to include new `search_knowledge` parameters.
+
+### v4.1.2 (2026-06-17)
+
+- **FIX**: `_save_metadata` dict snapshot prevents concurrent modification crash during file watcher events.
+- **STYLE**: ruff format applied to server.py.
+
+### v4.1.1 (2026-06-17)
+
+- **FIX**: All `_indexed_docs` iterations now use `list()` snapshot, preventing `dictionary changed size during iteration` crash when FileWatcher modifies the index concurrently with MCP tool calls (affects `search_knowledge`, `search_similar`, `update_document`, `remove_document`, `evaluate_retrieval`, `list_categories`, `list_documents`)
+
+### v4.1.0 (2026-06-17)
+
+- **Added:** `query_expansion_groups` config for symmetric synonym expansion (#92)
+- **Improved:** `expand_query()` now returns deterministic expansion order (set → ordered list with dedup)
+
+### v4.0.1 (2026-06-16)
+
+- **FIX**: Orphan cleanup now runs before indexing loop, preventing chunk loss when files are moved (#90).
+- **FIX**: Chunk deduplication is now per-document instead of global, preventing cross-document chunk deletion (#91).
+- **FIX**: Added `on_moved` handler to `DocumentWatcher` for proper file move detection.
+- **FIX**: Startup preflight probes ChromaDB in a child process and moves crashing persistent indexes to `data/backups/auto-repair-*` before MCP initialization.
+- **FIX**: Reranker load failures now fall back to RRF ordering instead of failing `search_knowledge` on offline machines.
+- **FIX**: Virtualenv project-root detection now handles Python symlinks that resolve to the system interpreter.
+- **NEW**: `knowledge-rag-guarded` console script kept as an explicit guarded startup alias.
+
 ### v4.0.0 (2026-06-09) — Enterprise Concurrent Access
 
 - **NEW**: SSE and streamable-http transport modes — 1 server serves N clients (`server.transport: "sse"` in config.yaml or `--transport sse` CLI).
@@ -1482,7 +1555,7 @@ Common issues:
 - **NEW** Property-based fuzzing of all parsers via Hypothesis (`tests/test_ingestion_property.py`) — 200 random examples per CI run.
 - **NEW** Memory baseline regression tests (`tests/test_memory_baseline.py`, cross-platform via psutil) — RSS bounded under 1000 queries; nightly soak amplifies to 50K iterations.
 - **NEW** Property/locale/format/preset matrices (`tests/test_presets.py`, `tests/test_locale.py`, `tests/test_format_smoke.py`).
-- **NEW** Backwards-compatibility regression tests (`tests/test_backwards_compat.py`) — legacy YAML configs from v3.6.0 / v3.7.0 still parse; all 12 MCP tool parameter names frozen.
+- **NEW** Backwards-compatibility regression tests (`tests/test_backwards_compat.py`) — legacy YAML configs from v3.6.0 / v3.7.0 still parse; all 13 MCP tool parameter names frozen.
 - **NEW** AST-based public API surface diff (`scripts/check_api_surface.py`) — any breaking change blocks merge, baseline at `.github/api-surface-baseline.json`.
 - **NEW** CHANGELOG enforcement (`scripts/check_changelog.py`) — user-facing PRs must add a bullet under `## Unreleased`; bypass via `skip-changelog` label.
 - **NEW** Test count anti-regression (`scripts/check_test_count.py`) — guards against silent test deletion.
@@ -1512,57 +1585,6 @@ Common issues:
 - **CHORE**: pytest `tmp_path_retention_count=1` to avoid Windows atexit cleanup race in CI.
 - **ROADMAP**: Tracked v4.0 shared-service architecture (one daemon, many thin MCP clients) as the long-term fix for multi-process resource duplication. (#34)
 
-### Unreleased
-
-### v4.3.0 (2026-06-17) — Async Reindex, GPU CUDA 12, 13th MCP Tool
-
-- **NEW**: `get_reindex_status` MCP tool — lightweight reindex progress polling without computing full index stats. Returns active/idle status, percent, processed/total, errors, and last result.
-- **NEW**: `reindex_documents` now runs in background via daemon thread — returns immediately with `{"status": "started"}`. Eliminates MCP timeout on large document sets (5K+ files). Concurrent calls return `already_running` with current progress.
-- **NEW**: GPU acceleration with full CUDA 12 support — `onnxruntime-gpu` + 7 NVIDIA pip packages (`cublas`, `cudnn`, `cuda-runtime`, `cufft`, `cusparse`, `cusolver`, `curand`, `nvjitlink`). Server auto-detects GPU on startup with 4-step verification (providers, DLLs, nvidia-smi, session creation). Falls back to CPU gracefully.
-- **NEW**: `_setup_cuda_dll_paths()` adds NVIDIA pip package DLL directories to `PATH` automatically on Windows — onnxruntime finds CUDA 12 DLLs without a full CUDA Toolkit install.
-- **DEPS**: `[gpu]` extra expanded from 3 to 8 packages (added `cufft`, `cusparse`, `cusolver`, `curand`, `nvjitlink`).
-- **FIX**: GPU status reporting now uses actual ONNX session creation test instead of just checking `get_available_providers()` — prevents false "GPU ACTIVE" when CUDA DLLs are missing.
-- **DOCS**: GPU Acceleration section rewritten with complete requirements table, setup steps, verification instructions, and fallback behavior.
-- **DOCS**: Tool reference updated — `reindex_documents` async behavior documented, `get_reindex_status` reference added.
-- **TEST**: Backwards-compat baseline updated for 13 MCP tools.
-
-### v4.2.0 (2026-06-17) — Search Performance & Output Quality
-
-- **PERF**: Custom inverted-index BM25 replaces `rank-bm25` full-corpus scan — 128× faster keyword search on 50K+ chunk corpora. Only documents containing query terms are scored via posting lists.
-- **PERF**: `numpy.argpartition` for O(n) top-k selection instead of O(n log n) sort.
-- **PERF**: Batched adjacent chunk fetch — single ChromaDB `collection.get()` call replaces N round-trips per result.
-- **PERF**: O(1) reverse lookup via `_source_to_docid` dict eliminates linear scans of `_indexed_docs` in `search_similar`, `update_document`, `remove_document`, and `_expand_with_adjacent_chunks`.
-- **NEW**: `snippet_mode` parameter on `search_knowledge` (default: `true`) — truncates content to ~500 chars at natural break points with `content_length` field. Reduces token consumption by ~72%.
-- **NEW**: `min_score` parameter on `search_knowledge` (default: `0.0`) — filters results below a normalized relevance threshold. Response includes `filtered_by_score` count.
-- **NEW**: `filtered_by_score` field in search response JSON for transparency.
-- **DEPS**: `numpy` added as direct dependency (was transitive via fastembed); `rank-bm25` import removed from server.py.
-- **TEST**: 6 new tests for `min_score` filtering and `snippet_mode` truncation.
-- **TEST**: Updated backwards-compat baseline to include new `search_knowledge` parameters.
-
-### v4.1.2 (2026-06-17)
-
-- **FIX**: `_save_metadata` dict snapshot prevents concurrent modification crash during file watcher events.
-- **STYLE**: ruff format applied to server.py.
-
-### v4.1.1 (2026-06-17)
-
-- **FIX**: All `_indexed_docs` iterations now use `list()` snapshot, preventing `dictionary changed size during iteration` crash when FileWatcher modifies the index concurrently with MCP tool calls (affects `search_knowledge`, `search_similar`, `update_document`, `remove_document`, `evaluate_retrieval`, `list_categories`, `list_documents`)
-
-### v4.1.0 (2026-06-17)
-
-- **Added:** `query_expansion_groups` config for symmetric synonym expansion (#92)
-- **Improved:** `expand_query()` now returns deterministic expansion order (set → ordered list with dedup)
-
-### v4.0.1 (2026-06-16)
-
-- **FIX**: Orphan cleanup now runs before indexing loop, preventing chunk loss when files are moved (#90).
-- **FIX**: Chunk deduplication is now per-document instead of global, preventing cross-document chunk deletion (#91).
-- **FIX**: Added `on_moved` handler to `DocumentWatcher` for proper file move detection.
-- **FIX**: Startup preflight probes ChromaDB in a child process and moves crashing persistent indexes to `data/backups/auto-repair-*` before MCP initialization.
-- **FIX**: Reranker load failures now fall back to RRF ordering instead of failing `search_knowledge` on offline machines.
-- **FIX**: Virtualenv project-root detection now handles Python symlinks that resolve to the system interpreter.
-- **NEW**: `knowledge-rag-guarded` console script kept as an explicit guarded startup alias.
-
 ### v3.6.2 (2026-04-23)
 
 - **INFRA**: NPM provenance attestation (SLSA supply chain security), full README on npm page

{knowledge_rag-4.3.0 → knowledge_rag-4.4.0}/README.md

@@ -1399,6 +1399,79 @@ Common issues:
 
 ## Changelog
 
+### Unreleased
+
+### v4.4.0 (2026-07-06) — Cross-Platform Installer & Hybrid Search Category Filter
+
+- **NEW**: Cross-platform, multi-LLM-client installer (`install.py`) driving both `install.sh` (Linux/macOS) and `install.ps1` (Windows) as thin wrappers. One codebase, one behavior across every OS.
+- **NEW**: Auto-detects and registers `knowledge-rag` in 8 LLM clients — Claude Code, Claude Desktop, Cursor, Windsurf, VS Code (Copilot Chat), Cline, Gemini CLI, Zed — writing to each tool's canonical config path with the correct JSON schema per client (VS Code uses `servers`, Zed uses `context_servers`, everyone else uses `mcpServers`).
+- **NEW**: `--for <clients>` / `--exclude <clients>` opt-in/opt-out selection, `--dry-run` preview, `--list-clients` registry inspection, `--pypi-version <ver>` pinning, `--skip-init` / `--skip-model` for fast reruns. See `python install.py --help`.
+- **FIX**: `install.ps1` no longer writes MCP config to `~/.claude/mcp.json` (a stale secondary path); it now targets `~/.claude.json` — the file Claude Code actually reads — via idempotent JSON merge that preserves every existing MCP server entry with an automatic `.knowledge-rag.bak` backup.
+- **FIX**: `install.ps1` gains PyPI mode (`pip install knowledge-rag`) and runs `mcp_server.server init` on install — feature parity with `install.sh`.
+- **FIX**: MCP server spec no longer uses the fragile `cmd /c cd /d ... && python ...` wrapper on Windows; it emits the standard `command` + `cwd` shape supported natively by every modern client.
+- **FIX**: `install.sh` guards against `sh install.sh` (bash-only features now emit a clear error instead of a cryptic syntax failure).
+- **FIX**: Both scripts now correctly advertise **13 MCP tools** (was outdated at 12; `get_reindex_status` shipped in v4.3.0).
+- **FIX**: Windows-side `install.ps1` prefers `winget install Python.Python.3.12 --scope user` (no admin), falls back to python.org 3.12.7 (was pinned to 3.12.0).
+- **FIX**: Hybrid search now applies the active category filter to BM25 results before RRF fusion, preventing keyword-only BM25 hits from other categories from leaking into filtered searches. Both leak paths are closed: BM25 candidates are metadata-filtered before RRF (with `top_k` widened to `max_results * 20` to compensate for post-filter drop), and the fallback fetch during fusion re-checks `category` before adding a chunk to `combined_scores`. Note: the `category_filter` guard now also applies to the keyword-routed category — `_route_by_keywords()`-inferred routing filters BM25 too, consistent with the semantic branch. Users with warm `query_cache` entries should restart the server to invalidate stale results. (#109, thanks @Hohlas)
+- **TEST**: New `tests/test_installer_no_data_loss.py` (22 tests) locks in the installer's zero-data-loss contract across all three JSON schemas (`mcpServers` / `servers` / `context_servers`): top-level keys preserved, sibling MCP servers byte-identical, `.knowledge-rag.bak` backup written before every mutation, idempotent second run, `--dry-run` writes nothing, atomic `os.replace` write. Baseline: 231 → 266.
+- **TEST**: New `TestHybridCategoryFilter::test_bm25_results_respect_category_filter` in `tests/test_search.py` — deterministic regression covering BM25-only hits with mixed categories. Baseline: 266 → 267.
+
+### v4.3.1 (2026-06-22) — Hybrid Search Fixes
+
+- **FIX**: Accept `"general"` as a valid category in `search_knowledge`. The parser hardcodes `"general"` as the fallback in `_detect_category` (`ingestion.py`), but the validator only built `valid_categories` from `config.keyword_routes` + `config.category_mappings.values()` — so users who customized `config.yaml` and dropped the default `"general": "general"` mapping hit `Invalid category` even though the index contained `general` documents. Validator now always tolerates `"general"`. (#98, thanks @Hohlas)
+- **FIX**: Skip BM25-only search results when Chroma can no longer resolve the chunk ID. Stale BM25 indices (typically right after `remove_document` or in the window between async reindex and BM25 rebuild) returned hits whose `collection.get()` came back empty; the previous fallback inserted entries with `document=""` / `metadata={}` into the reranker, polluting results with empty matches. The pipeline now `continue`s past those, dropping the stale hit cleanly. (#98, thanks @Hohlas)
+- **TEST**: Added `tests/test_pr98_regression.py` (4 tests) pinning both contracts so future refactors cannot silently revert either fix. Test count baseline: 227 → 231. (#99)
+- **CI**: Bumped `[tool.mypy] python_version` from 3.11 to 3.12 to accept PEP 695 `type` statements in the numpy stub (`numpy/__init__.pyi`) which were breaking the Pillar 7 strict gate. Only affects static analysis; `requires-python = ">=3.11"` unchanged. (#100)
+
+### v4.3.0 (2026-06-17) — Async Reindex, GPU CUDA 12, 13th MCP Tool
+
+- **NEW**: `get_reindex_status` MCP tool — lightweight reindex progress polling without computing full index stats. Returns active/idle status, percent, processed/total, errors, and last result.
+- **NEW**: `reindex_documents` now runs in background via daemon thread — returns immediately with `{"status": "started"}`. Eliminates MCP timeout on large document sets (5K+ files). Concurrent calls return `already_running` with current progress.
+- **NEW**: GPU acceleration with full CUDA 12 support — `onnxruntime-gpu` + 7 NVIDIA pip packages (`cublas`, `cudnn`, `cuda-runtime`, `cufft`, `cusparse`, `cusolver`, `curand`, `nvjitlink`). Server auto-detects GPU on startup with 4-step verification (providers, DLLs, nvidia-smi, session creation). Falls back to CPU gracefully.
+- **NEW**: `_setup_cuda_dll_paths()` adds NVIDIA pip package DLL directories to `PATH` automatically on Windows — onnxruntime finds CUDA 12 DLLs without a full CUDA Toolkit install.
+- **DEPS**: `[gpu]` extra expanded from 3 to 8 packages (added `cufft`, `cusparse`, `cusolver`, `curand`, `nvjitlink`).
+- **FIX**: GPU status reporting now uses actual ONNX session creation test instead of just checking `get_available_providers()` — prevents false "GPU ACTIVE" when CUDA DLLs are missing.
+- **DOCS**: GPU Acceleration section rewritten with complete requirements table, setup steps, verification instructions, and fallback behavior.
+- **DOCS**: Tool reference updated — `reindex_documents` async behavior documented, `get_reindex_status` reference added.
+- **TEST**: Backwards-compat baseline updated for 13 MCP tools.
+
+### v4.2.0 (2026-06-17) — Search Performance & Output Quality
+
+- **PERF**: Custom inverted-index BM25 replaces `rank-bm25` full-corpus scan — 128× faster keyword search on 50K+ chunk corpora. Only documents containing query terms are scored via posting lists.
+- **PERF**: `numpy.argpartition` for O(n) top-k selection instead of O(n log n) sort.
+- **PERF**: Batched adjacent chunk fetch — single ChromaDB `collection.get()` call replaces N round-trips per result.
+- **PERF**: O(1) reverse lookup via `_source_to_docid` dict eliminates linear scans of `_indexed_docs` in `search_similar`, `update_document`, `remove_document`, and `_expand_with_adjacent_chunks`.
+- **NEW**: `snippet_mode` parameter on `search_knowledge` (default: `true`) — truncates content to ~500 chars at natural break points with `content_length` field. Reduces token consumption by ~72%.
+- **NEW**: `min_score` parameter on `search_knowledge` (default: `0.0`) — filters results below a normalized relevance threshold. Response includes `filtered_by_score` count.
+- **NEW**: `filtered_by_score` field in search response JSON for transparency.
+- **DEPS**: `numpy` added as direct dependency (was transitive via fastembed); `rank-bm25` import removed from server.py.
+- **TEST**: 6 new tests for `min_score` filtering and `snippet_mode` truncation.
+- **TEST**: Updated backwards-compat baseline to include new `search_knowledge` parameters.
+
+### v4.1.2 (2026-06-17)
+
+- **FIX**: `_save_metadata` dict snapshot prevents concurrent modification crash during file watcher events.
+- **STYLE**: ruff format applied to server.py.
+
+### v4.1.1 (2026-06-17)
+
+- **FIX**: All `_indexed_docs` iterations now use `list()` snapshot, preventing `dictionary changed size during iteration` crash when FileWatcher modifies the index concurrently with MCP tool calls (affects `search_knowledge`, `search_similar`, `update_document`, `remove_document`, `evaluate_retrieval`, `list_categories`, `list_documents`)
+
+### v4.1.0 (2026-06-17)
+
+- **Added:** `query_expansion_groups` config for symmetric synonym expansion (#92)
+- **Improved:** `expand_query()` now returns deterministic expansion order (set → ordered list with dedup)
+
+### v4.0.1 (2026-06-16)
+
+- **FIX**: Orphan cleanup now runs before indexing loop, preventing chunk loss when files are moved (#90).
+- **FIX**: Chunk deduplication is now per-document instead of global, preventing cross-document chunk deletion (#91).
+- **FIX**: Added `on_moved` handler to `DocumentWatcher` for proper file move detection.
+- **FIX**: Startup preflight probes ChromaDB in a child process and moves crashing persistent indexes to `data/backups/auto-repair-*` before MCP initialization.
+- **FIX**: Reranker load failures now fall back to RRF ordering instead of failing `search_knowledge` on offline machines.
+- **FIX**: Virtualenv project-root detection now handles Python symlinks that resolve to the system interpreter.
+- **NEW**: `knowledge-rag-guarded` console script kept as an explicit guarded startup alias.
+
 ### v4.0.0 (2026-06-09) — Enterprise Concurrent Access
 
 - **NEW**: SSE and streamable-http transport modes — 1 server serves N clients (`server.transport: "sse"` in config.yaml or `--transport sse` CLI).
@@ -1434,7 +1507,7 @@ Common issues:
 - **NEW** Property-based fuzzing of all parsers via Hypothesis (`tests/test_ingestion_property.py`) — 200 random examples per CI run.
 - **NEW** Memory baseline regression tests (`tests/test_memory_baseline.py`, cross-platform via psutil) — RSS bounded under 1000 queries; nightly soak amplifies to 50K iterations.
 - **NEW** Property/locale/format/preset matrices (`tests/test_presets.py`, `tests/test_locale.py`, `tests/test_format_smoke.py`).
-- **NEW** Backwards-compatibility regression tests (`tests/test_backwards_compat.py`) — legacy YAML configs from v3.6.0 / v3.7.0 still parse; all 12 MCP tool parameter names frozen.
+- **NEW** Backwards-compatibility regression tests (`tests/test_backwards_compat.py`) — legacy YAML configs from v3.6.0 / v3.7.0 still parse; all 13 MCP tool parameter names frozen.
 - **NEW** AST-based public API surface diff (`scripts/check_api_surface.py`) — any breaking change blocks merge, baseline at `.github/api-surface-baseline.json`.
 - **NEW** CHANGELOG enforcement (`scripts/check_changelog.py`) — user-facing PRs must add a bullet under `## Unreleased`; bypass via `skip-changelog` label.
 - **NEW** Test count anti-regression (`scripts/check_test_count.py`) — guards against silent test deletion.
@@ -1464,57 +1537,6 @@ Common issues:
 - **CHORE**: pytest `tmp_path_retention_count=1` to avoid Windows atexit cleanup race in CI.
 - **ROADMAP**: Tracked v4.0 shared-service architecture (one daemon, many thin MCP clients) as the long-term fix for multi-process resource duplication. (#34)
 
-### Unreleased
-
-### v4.3.0 (2026-06-17) — Async Reindex, GPU CUDA 12, 13th MCP Tool
-
-- **NEW**: `get_reindex_status` MCP tool — lightweight reindex progress polling without computing full index stats. Returns active/idle status, percent, processed/total, errors, and last result.
-- **NEW**: `reindex_documents` now runs in background via daemon thread — returns immediately with `{"status": "started"}`. Eliminates MCP timeout on large document sets (5K+ files). Concurrent calls return `already_running` with current progress.
-- **NEW**: GPU acceleration with full CUDA 12 support — `onnxruntime-gpu` + 7 NVIDIA pip packages (`cublas`, `cudnn`, `cuda-runtime`, `cufft`, `cusparse`, `cusolver`, `curand`, `nvjitlink`). Server auto-detects GPU on startup with 4-step verification (providers, DLLs, nvidia-smi, session creation). Falls back to CPU gracefully.
-- **NEW**: `_setup_cuda_dll_paths()` adds NVIDIA pip package DLL directories to `PATH` automatically on Windows — onnxruntime finds CUDA 12 DLLs without a full CUDA Toolkit install.
-- **DEPS**: `[gpu]` extra expanded from 3 to 8 packages (added `cufft`, `cusparse`, `cusolver`, `curand`, `nvjitlink`).
-- **FIX**: GPU status reporting now uses actual ONNX session creation test instead of just checking `get_available_providers()` — prevents false "GPU ACTIVE" when CUDA DLLs are missing.
-- **DOCS**: GPU Acceleration section rewritten with complete requirements table, setup steps, verification instructions, and fallback behavior.
-- **DOCS**: Tool reference updated — `reindex_documents` async behavior documented, `get_reindex_status` reference added.
-- **TEST**: Backwards-compat baseline updated for 13 MCP tools.
-
-### v4.2.0 (2026-06-17) — Search Performance & Output Quality
-
-- **PERF**: Custom inverted-index BM25 replaces `rank-bm25` full-corpus scan — 128× faster keyword search on 50K+ chunk corpora. Only documents containing query terms are scored via posting lists.
-- **PERF**: `numpy.argpartition` for O(n) top-k selection instead of O(n log n) sort.
-- **PERF**: Batched adjacent chunk fetch — single ChromaDB `collection.get()` call replaces N round-trips per result.
-- **PERF**: O(1) reverse lookup via `_source_to_docid` dict eliminates linear scans of `_indexed_docs` in `search_similar`, `update_document`, `remove_document`, and `_expand_with_adjacent_chunks`.
-- **NEW**: `snippet_mode` parameter on `search_knowledge` (default: `true`) — truncates content to ~500 chars at natural break points with `content_length` field. Reduces token consumption by ~72%.
-- **NEW**: `min_score` parameter on `search_knowledge` (default: `0.0`) — filters results below a normalized relevance threshold. Response includes `filtered_by_score` count.
-- **NEW**: `filtered_by_score` field in search response JSON for transparency.
-- **DEPS**: `numpy` added as direct dependency (was transitive via fastembed); `rank-bm25` import removed from server.py.
-- **TEST**: 6 new tests for `min_score` filtering and `snippet_mode` truncation.
-- **TEST**: Updated backwards-compat baseline to include new `search_knowledge` parameters.
-
-### v4.1.2 (2026-06-17)
-
-- **FIX**: `_save_metadata` dict snapshot prevents concurrent modification crash during file watcher events.
-- **STYLE**: ruff format applied to server.py.
-
-### v4.1.1 (2026-06-17)
-
-- **FIX**: All `_indexed_docs` iterations now use `list()` snapshot, preventing `dictionary changed size during iteration` crash when FileWatcher modifies the index concurrently with MCP tool calls (affects `search_knowledge`, `search_similar`, `update_document`, `remove_document`, `evaluate_retrieval`, `list_categories`, `list_documents`)
-
-### v4.1.0 (2026-06-17)
-
-- **Added:** `query_expansion_groups` config for symmetric synonym expansion (#92)
-- **Improved:** `expand_query()` now returns deterministic expansion order (set → ordered list with dedup)
-
-### v4.0.1 (2026-06-16)
-
-- **FIX**: Orphan cleanup now runs before indexing loop, preventing chunk loss when files are moved (#90).
-- **FIX**: Chunk deduplication is now per-document instead of global, preventing cross-document chunk deletion (#91).
-- **FIX**: Added `on_moved` handler to `DocumentWatcher` for proper file move detection.
-- **FIX**: Startup preflight probes ChromaDB in a child process and moves crashing persistent indexes to `data/backups/auto-repair-*` before MCP initialization.
-- **FIX**: Reranker load failures now fall back to RRF ordering instead of failing `search_knowledge` on offline machines.
-- **FIX**: Virtualenv project-root detection now handles Python symlinks that resolve to the system interpreter.
-- **NEW**: `knowledge-rag-guarded` console script kept as an explicit guarded startup alias.
-
 ### v3.6.2 (2026-04-23)
 
 - **INFRA**: NPM provenance attestation (SLSA supply chain security), full README on npm page

{knowledge_rag-4.3.0 → knowledge_rag-4.4.0}/mcp_server/__init__.py

@@ -8,7 +8,7 @@ import sys  # noqa: I001
 _original_stdout = sys.stdout
 sys.stdout = sys.stderr
 
-__version__ = "4.3.0"
+__version__ = "4.4.0"
 __author__ = "Ailton Rocha (Lyon.)"
 
 from .config import Config  # noqa: E402

{knowledge_rag-4.3.0 → knowledge_rag-4.4.0}/mcp_server/server.py

@@ -1474,6 +1474,12 @@ class KnowledgeOrchestrator:
         elif routed_category:
             where_filter = {"category": routed_category}
 
+        def _matches_category(metadata: Dict[str, Any]) -> bool:
+            if not where_filter:
+                return True
+            expected_category = where_filter.get("category")
+            return not expected_category or metadata.get("category") == expected_category
+
         # Parallel Semantic + BM25 search (threaded for latency reduction)
         from concurrent.futures import ThreadPoolExecutor
 
@@ -1507,8 +1513,23 @@ class KnowledgeOrchestrator:
             r = {}
             if hybrid_alpha < 1.0:
                 try:
-                    bm25_hits = self.bm25_index.search(query_text, top_k=max_results * 3)
-                    for rank, (chunk_id, bm25_score) in enumerate(bm25_hits):
+                    bm25_top_k = max_results * (20 if where_filter else 3)
+                    bm25_hits = self.bm25_index.search(query_text, top_k=bm25_top_k)
+
+                    if where_filter:
+                        chunk_ids = [chunk_id for chunk_id, _ in bm25_hits]
+                        metadata_by_id = {}
+                        if chunk_ids:
+                            fetched = self.collection.get(ids=chunk_ids, include=["metadatas"])
+                            metadata_by_id = dict(zip(fetched.get("ids", []), fetched.get("metadatas", [])))
+
+                        bm25_hits = [
+                            (chunk_id, bm25_score)
+                            for chunk_id, bm25_score in bm25_hits
+                            if _matches_category(metadata_by_id.get(chunk_id, {}))
+                        ]
+
+                    for rank, (chunk_id, bm25_score) in enumerate(bm25_hits[: max_results * 3]):
                         r[chunk_id] = {"rank": rank + 1, "bm25_score": bm25_score}
                 except Exception as e:
                     print(f"[WARN] BM25 search failed: {e}")
@@ -1543,14 +1564,24 @@ class KnowledgeOrchestrator:
             else:
                 try:
                     fetched = self.collection.get(ids=[chunk_id], include=["documents", "metadatas"])
+                    if (
+                        not fetched["documents"]
+                        or not fetched["metadatas"]
+                        or not fetched["documents"][0]
+                        or not fetched["metadatas"][0]
+                    ):
+                        continue
                     data = {
-                        "document": fetched["documents"][0] if fetched["documents"] else "",
-                        "metadata": fetched["metadatas"][0] if fetched["metadatas"] else {},
+                        "document": fetched["documents"][0],
+                        "metadata": fetched["metadatas"][0],
                         "distance": 0,
                     }
                 except Exception:
                     continue
 
+            if not _matches_category(data.get("metadata", {})):
+                continue
+
             combined_scores[chunk_id] = {
                 "rrf_score": combined_rrf,
                 "semantic_rank": semantic_rank if chunk_id in semantic_results else None,
@@ -2273,7 +2304,7 @@ def search_knowledge(
     hybrid_alpha = max(0.0, min(hybrid_alpha if hybrid_alpha is not None else 0.3, 1.0))
     min_score = max(0.0, min(min_score if min_score is not None else 0.0, 1.0))
 
-    valid_categories = list(config.keyword_routes.keys()) + list(set(config.category_mappings.values()))
+    valid_categories = list(config.keyword_routes.keys()) + list(set(config.category_mappings.values())) + ["general"]
     if category and category not in valid_categories:
         return json.dumps(
             {"status": "error", "message": f"Invalid category '{category}'. Valid: {', '.join(valid_categories)}"}

{knowledge_rag-4.3.0 → knowledge_rag-4.4.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "knowledge-rag"
-version = "4.3.0"
+version = "4.4.0"
 description = "Local RAG System for Claude Code — Hybrid search + Cross-encoder Reranking + 13 MCP Tools + 20 Format Parsers. Zero external servers."
 readme = "README.md"
 license = {text = "MIT"}
@@ -141,7 +141,11 @@ fail_under = 35
 # we incrementally annotate the legacy modules. The CI job runs strict on the
 # allowlist below; new modules are added as they earn full annotations.
 [tool.mypy]
-python_version = "3.11"
+# Pinned to 3.12 to match the CI runtime (Python 3.12) and to accept PEP 695
+# ``type`` statements that recent third-party stubs (e.g. numpy/__init__.pyi)
+# emit. The package itself still supports Python 3.11+ at runtime — this
+# setting only governs the static-analysis target.
+python_version = "3.12"
 strict = true
 show_error_codes = true
 warn_unused_configs = true