knowledge-rag 4.7.0__tar.gz → 4.7.1__tar.gz

{knowledge_rag-4.7.0 → knowledge_rag-4.7.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: knowledge-rag
-Version: 4.7.0
+Version: 4.7.1
 Summary: Local RAG System for Claude Code — Hybrid search + Cross-encoder Reranking + 13 MCP Tools + 20 Format Parsers. Zero external servers.
 Project-URL: Homepage, https://github.com/lyonzin/knowledge-rag
 Project-URL: Repository, https://github.com/lyonzin/knowledge-rag
@@ -1462,12 +1462,18 @@ Common issues:
 
 ### Unreleased
 
-### v4.7.0 (2026-07-31) — BM25 Fragment Query Support
+### v4.7.1 (2026-08-01) — Documentation Hygiene
 
-- **FIX (retrieval)**: BM25 tokenizer now emits **sub-tokens** for hyphenated alphanumeric-code composites, so fragment queries no longer silently return `NO_RESULTS`. Before v4.7.0, `MDR-AD002` was indexed as a single token `mdr-ad002`; queries like `AD002`, `MDR`, `CS005` returned zero even when the composite lived in the corpus. This is not a bug specific to MDR — it hit every hyphenated code taxonomy in typical infosec / doc corpora (`MDR-*`, `CVE-*`, `ADR-*`, `MS17-*`, `MDR-Custom001-xxxx`). The tokenizer now emits both the composite token AND its sub-parts of length ≥ 2 (`mdr-ad002` → `["mdr-ad002", "mdr", "ad002"]`). IDF preserves ranking — composite matches still rank above fragment matches because the composite is rarer. **Heuristic (perf-safety)**: sub-token expansion is triggered ONLY when the composite contains at least one digit — so `pass-the-hash`, `state-of-the-art`, and other natural-language hyphenated phrases stay as single tokens (expanding them would flood the inverted index with stop-word-like sub-parts and hurt concurrent-query throughput). (#140)
-- **NEW (tests)**: `tests/test_bm25_tokenizer_fragment.py` — 25 tests across 4 classes covering unit tokenizer behavior, end-to-end fragment matching, multi-doc disambiguation, and the 5 query patterns explicitly reported (`MDR-AD019`, `MDR-Custom001-xxxx`, `MDR`, `AD019`, `CS005`). Includes retrocompat sentinel + IDF ranking invariant. Baseline: 314 → 339.
+- **CHORE (docs, tests)**: sanitize identifiers in test fixtures and CHANGELOG examples. v4.7.0 referenced code names from a real deployment corpus in `tests/test_bm25_tokenizer_fragment.py` and the v4.7.0 CHANGELOG entry; those are replaced with synthetic placeholders (`RULE-A00X`, `PROJECT-Custom001-xxxx`) and well-known public identifiers (`CVE-*`, `MS17-*`, `ADR-*`, `T1078.002`) that carry no organization-specific taxonomy. Zero behavior change — all 28 fragment tests pass unchanged, tokenizer logic identical.
+- **NOTE**: v4.7.0 is **yanked from PyPI** and **deprecated on NPM** because its published README contained the same corporate example strings. The Docker tag `v4.7.0` remains (image tags are immutable) but `latest` now points to v4.7.1. Please upgrade — v4.7.0 and v4.7.1 are functionally equivalent.
+- **VERSION**: 4.7.0 → **4.7.1** (PATCH — documentation hygiene, zero code / API / config change).
+
+### v4.7.0 (2026-07-31) — BM25 Fragment Query Support ⚠️ **YANKED — use v4.7.1**
+
+- **FIX (retrieval)**: BM25 tokenizer now emits **sub-tokens** for hyphenated alphanumeric-code composites, so fragment queries no longer silently return `NO_RESULTS`. Before v4.7.0, `RULE-A002` was indexed as a single token `rule-a002`; queries like `A002`, `RULE`, `B005` returned zero even when the composite lived in the corpus. This is not a bug specific to any naming scheme — it hit every hyphenated code taxonomy in typical infosec / doc corpora (`RULE-*`, `CVE-*`, `ADR-*`, `MS17-*`, `PROJECT-Custom001-xxxx`). The tokenizer now emits both the composite token AND its sub-parts of length ≥ 2 (`rule-a002` → `["rule-a002", "rule", "a002"]`). IDF preserves ranking — composite matches still rank above fragment matches because the composite is rarer. **Heuristic (perf-safety)**: sub-token expansion is triggered ONLY when the composite contains at least one digit — so `pass-the-hash`, `state-of-the-art`, and other natural-language hyphenated phrases stay as single tokens (expanding them would flood the inverted index with stop-word-like sub-parts and hurt concurrent-query throughput). (#140)
+- **NEW (tests)**: `tests/test_bm25_tokenizer_fragment.py` — 28 tests across 4 classes covering unit tokenizer behavior, end-to-end fragment matching, multi-doc disambiguation, and the 5 query patterns explicitly reported (`RULE-A019`, `PROJECT-Custom001-xxxx`, `RULE`, `A019`, `B005`). Includes retrocompat sentinel + IDF ranking invariant. Baseline: 314 → 342.
 - **UPGRADE NOTE**: run `reindex_documents(force=True)` once after upgrade to rebuild the BM25 inverted index with the new sub-token emission. ChromaDB vectors are untouched. Expect a **~30–50% increase in BM25 index memory** (measured +32.5% on synthetic corpus) — negligible for typical corpora, worth reviewing for deployments with >100K docs.
-- **KNOWN LIMITATION**: alphanumeric-glued tokens without separators (`HARDLAB11`, `6a635623e114b8...`) still require exact match — no sub-token expansion possible without character n-grams (5-10× index overhead). Tracked as follow-up.
+- **KNOWN LIMITATION**: alphanumeric-glued tokens without separators (single-token identifiers with no hyphens, hash prefixes) still require exact match — no sub-token expansion possible without character n-grams (5-10× index overhead). Tracked as follow-up.
 - **VERSION**: 4.6.0 → **4.7.0** (MINOR — tokenization behavior change + reindex requirement, though no API/config break).
 
 ### v4.6.0 (2026-07-30) — MCP spec 2026-07-28 via Anthropic Tier 1 SDK (mcp 2.x)

{knowledge_rag-4.7.0 → knowledge_rag-4.7.1}/README.md

@@ -1414,12 +1414,18 @@ Common issues:
 
 ### Unreleased
 
-### v4.7.0 (2026-07-31) — BM25 Fragment Query Support
+### v4.7.1 (2026-08-01) — Documentation Hygiene
 
-- **FIX (retrieval)**: BM25 tokenizer now emits **sub-tokens** for hyphenated alphanumeric-code composites, so fragment queries no longer silently return `NO_RESULTS`. Before v4.7.0, `MDR-AD002` was indexed as a single token `mdr-ad002`; queries like `AD002`, `MDR`, `CS005` returned zero even when the composite lived in the corpus. This is not a bug specific to MDR — it hit every hyphenated code taxonomy in typical infosec / doc corpora (`MDR-*`, `CVE-*`, `ADR-*`, `MS17-*`, `MDR-Custom001-xxxx`). The tokenizer now emits both the composite token AND its sub-parts of length ≥ 2 (`mdr-ad002` → `["mdr-ad002", "mdr", "ad002"]`). IDF preserves ranking — composite matches still rank above fragment matches because the composite is rarer. **Heuristic (perf-safety)**: sub-token expansion is triggered ONLY when the composite contains at least one digit — so `pass-the-hash`, `state-of-the-art`, and other natural-language hyphenated phrases stay as single tokens (expanding them would flood the inverted index with stop-word-like sub-parts and hurt concurrent-query throughput). (#140)
-- **NEW (tests)**: `tests/test_bm25_tokenizer_fragment.py` — 25 tests across 4 classes covering unit tokenizer behavior, end-to-end fragment matching, multi-doc disambiguation, and the 5 query patterns explicitly reported (`MDR-AD019`, `MDR-Custom001-xxxx`, `MDR`, `AD019`, `CS005`). Includes retrocompat sentinel + IDF ranking invariant. Baseline: 314 → 339.
+- **CHORE (docs, tests)**: sanitize identifiers in test fixtures and CHANGELOG examples. v4.7.0 referenced code names from a real deployment corpus in `tests/test_bm25_tokenizer_fragment.py` and the v4.7.0 CHANGELOG entry; those are replaced with synthetic placeholders (`RULE-A00X`, `PROJECT-Custom001-xxxx`) and well-known public identifiers (`CVE-*`, `MS17-*`, `ADR-*`, `T1078.002`) that carry no organization-specific taxonomy. Zero behavior change — all 28 fragment tests pass unchanged, tokenizer logic identical.
+- **NOTE**: v4.7.0 is **yanked from PyPI** and **deprecated on NPM** because its published README contained the same corporate example strings. The Docker tag `v4.7.0` remains (image tags are immutable) but `latest` now points to v4.7.1. Please upgrade — v4.7.0 and v4.7.1 are functionally equivalent.
+- **VERSION**: 4.7.0 → **4.7.1** (PATCH — documentation hygiene, zero code / API / config change).
+
+### v4.7.0 (2026-07-31) — BM25 Fragment Query Support ⚠️ **YANKED — use v4.7.1**
+
+- **FIX (retrieval)**: BM25 tokenizer now emits **sub-tokens** for hyphenated alphanumeric-code composites, so fragment queries no longer silently return `NO_RESULTS`. Before v4.7.0, `RULE-A002` was indexed as a single token `rule-a002`; queries like `A002`, `RULE`, `B005` returned zero even when the composite lived in the corpus. This is not a bug specific to any naming scheme — it hit every hyphenated code taxonomy in typical infosec / doc corpora (`RULE-*`, `CVE-*`, `ADR-*`, `MS17-*`, `PROJECT-Custom001-xxxx`). The tokenizer now emits both the composite token AND its sub-parts of length ≥ 2 (`rule-a002` → `["rule-a002", "rule", "a002"]`). IDF preserves ranking — composite matches still rank above fragment matches because the composite is rarer. **Heuristic (perf-safety)**: sub-token expansion is triggered ONLY when the composite contains at least one digit — so `pass-the-hash`, `state-of-the-art`, and other natural-language hyphenated phrases stay as single tokens (expanding them would flood the inverted index with stop-word-like sub-parts and hurt concurrent-query throughput). (#140)
+- **NEW (tests)**: `tests/test_bm25_tokenizer_fragment.py` — 28 tests across 4 classes covering unit tokenizer behavior, end-to-end fragment matching, multi-doc disambiguation, and the 5 query patterns explicitly reported (`RULE-A019`, `PROJECT-Custom001-xxxx`, `RULE`, `A019`, `B005`). Includes retrocompat sentinel + IDF ranking invariant. Baseline: 314 → 342.
 - **UPGRADE NOTE**: run `reindex_documents(force=True)` once after upgrade to rebuild the BM25 inverted index with the new sub-token emission. ChromaDB vectors are untouched. Expect a **~30–50% increase in BM25 index memory** (measured +32.5% on synthetic corpus) — negligible for typical corpora, worth reviewing for deployments with >100K docs.
-- **KNOWN LIMITATION**: alphanumeric-glued tokens without separators (`HARDLAB11`, `6a635623e114b8...`) still require exact match — no sub-token expansion possible without character n-grams (5-10× index overhead). Tracked as follow-up.
+- **KNOWN LIMITATION**: alphanumeric-glued tokens without separators (single-token identifiers with no hyphens, hash prefixes) still require exact match — no sub-token expansion possible without character n-grams (5-10× index overhead). Tracked as follow-up.
 - **VERSION**: 4.6.0 → **4.7.0** (MINOR — tokenization behavior change + reindex requirement, though no API/config break).
 
 ### v4.6.0 (2026-07-30) — MCP spec 2026-07-28 via Anthropic Tier 1 SDK (mcp 2.x)

{knowledge_rag-4.7.0 → knowledge_rag-4.7.1}/mcp_server/__init__.py

@@ -8,7 +8,7 @@ import sys  # noqa: I001
 _original_stdout = sys.stdout
 sys.stdout = sys.stderr
 
-__version__ = "4.7.0"
+__version__ = "4.7.1"
 __author__ = "Ailton Rocha (Lyon.)"
 
 from .config import Config  # noqa: E402

{knowledge_rag-4.7.0 → knowledge_rag-4.7.1}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "knowledge-rag"
-version = "4.7.0"
+version = "4.7.1"
 description = "Local RAG System for Claude Code — Hybrid search + Cross-encoder Reranking + 13 MCP Tools + 20 Format Parsers. Zero external servers."
 readme = "README.md"
 license = {text = "MIT"}