PyStreamMCP 3.2.0__tar.gz → 3.3.0__tar.gz

{pystreammcp-3.2.0/python/PyStreamMCP.egg-info → pystreammcp-3.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: PyStreamMCP
-Version: 3.2.0
+Version: 3.3.0
 Summary: Intelligent MCP orchestration hub - Intent understanding, capability matching, tool ranking with cohesive foundation
 Author-email: Georgi Mammen Mullassery <mullassery@gmail.com>
 License: Proprietary License — Free to use with explicit attribution
@@ -91,7 +91,11 @@ pip install "PyStreamMCP[semantic-kernel]"  # Semantic Kernel adapter
 pip install "PyStreamMCP[all-integrations]" # every framework adapter
 ```
 
-Requires Python 3.9+.
+Requires Python 3.9+ for the core SDK (`Agent`, `SourceRegistry`,
+`Orchestrator`). The `mcp` and `semantic-kernel` extras require Python
+3.10+ — their upstream packages don't publish Python 3.9-compatible
+releases, so `pip install` for those extras will fail on 3.9. CI tests
+3.10, 3.11, and 3.12 only; 3.9 is not covered by CI.
 
 ## Quick start
 
@@ -120,6 +124,33 @@ for a full runnable walkthrough, including framework adapters for
 LangChain, LlamaIndex, CrewAI, Semantic Kernel, PydanticAI, and Haystack
 under [`pystreammcp.integrations`](python/pystreammcp/integrations/).
 
+## Data source discovery
+
+`SourceRegistry` (`pystreammcp.SourceRegistry`) ranks *registered* data
+sources by real word-overlap relevance to a query — not a fabricated or
+hardcoded result:
+
+```python
+from pystreammcp import SourceRegistry
+
+registry = SourceRegistry()
+registry.register(
+    "orders_db",
+    description="customer order history and revenue by quarter",
+    type="database",
+    tags=["orders", "revenue"],
+)
+
+results = registry.discover("customer revenue by quarter")
+# [{"name": "orders_db", "type": "database", "relevance": 0.4444, "matched_terms": [...]}]
+```
+
+The same registry backs `POST /discover` (REST), the `pystreammcp_discover`
+MCP tool, and `LangchainAdapter.discover()` — register sources once via
+`POST /sources` / `pystreammcp_register_source` / `registry.register()`,
+then discover against them from any of the three surfaces. This is a
+simple Jaccard token-overlap heuristic, not semantic/embedding search.
+
 ## MCP orchestration & federation
 
 `Orchestrator` (`pystreammcp.Orchestrator`) discovers and routes across
@@ -217,6 +248,49 @@ pip install -e ".[dev,api,mcp,langchain,llamaindex,semantic-kernel]"
 pytest tests/ -v
 ```
 
+Note: this exact command needs Python 3.10+ (see [Install](#install)); on
+3.9 the `mcp` and `semantic-kernel` extras fail to resolve, so drop them
+from the extras list if you're on 3.9.
+
+## Known Issues
+
+- ~~No end-to-end tests for LLM failure states~~ **Fixed.** PyStreamMCP's
+  own code never calls a model provider directly (no code path here to add
+  `except RateLimitError`/`APIError` handling *to* — see `AgentFrameworkAdapter`
+  below), so what's now tested is the part this repo actually owns: that the
+  MCP tool-call path doesn't crash when whatever it's orchestrating fails.
+  `pystreammcp.testing` exports `RateLimitError`, `ContextWindowExceededError`,
+  `MalformedModelResponseError`, and `FailingAgent`/`FailingAdapter` stand-ins
+  that raise them; `mcp_server.py`'s `call_tool`/`process_message` now catch
+  exceptions raised during tool execution and return a structured
+  `{"status": "error", "error_type": ..., "message": ...}` response instead
+  of propagating uncaught. See `tests/test_mcp_server_resilience.py`.
+- ~~No Pydantic validation for the MCP tool protocol itself~~ **Fixed.**
+  Each of the four MCP tools now has a Pydantic arg model
+  (`QueryToolArgs`/`DiscoverToolArgs`/`RegisterSourceToolArgs`/`OptimizeToolArgs`
+  in `mcp_server.py`) validated in `call_tool` before dispatch (wrong types,
+  invalid enum values, and missing required fields all now return a
+  structured `validation_errors` response instead of reaching the handler
+  unchecked), and `MCPTool.input_schema` is generated from those same models
+  via `model_json_schema()` — the advertised schema and the schema actually
+  enforced can no longer drift apart the way a hand-duplicated dict could.
+- ~~No shipped mock MCP server / mock LLM fixture~~ **Fixed.** Added
+  `tests/conftest.py` (centralizing the `sys.path` setup 5 test files were
+  duplicating, plus `mock_adapter`/`failing_agent`/`failing_adapter`/
+  `llm_failure` pytest fixtures) and `pystreammcp/testing.py`, exporting
+  `MockAdapter` (promoted from the old test-local copy) and the
+  failure-injection stand-ins above for downstream projects' own test
+  suites to import directly.
+- **Per-query context discovery** now has a real implementation
+  (`pystreammcp.discovery.SourceRegistry`): register data sources (name,
+  description, type, tags), and `/discover` (REST), `pystreammcp_discover`
+  (MCP), and `LangchainAdapter.discover()` all rank *registered* sources
+  by real Jaccard token-overlap against the query — an empty registry or
+  a non-matching context returns an empty list, not fabricated data. This
+  is a simple word-overlap heuristic, not semantic/embedding search — if
+  you need that, wrap `SourceRegistry` with your own similarity scoring.
+- No open GitHub issues as of this pass.
+
 ## Rust workspace (not shipped)
 
 `core/` and `python/src/lib.rs` are an in-progress Rust workspace intended

{pystreammcp-3.2.0 → pystreammcp-3.3.0}/README.md

@@ -27,7 +27,11 @@ pip install "PyStreamMCP[semantic-kernel]"  # Semantic Kernel adapter
 pip install "PyStreamMCP[all-integrations]" # every framework adapter
 ```
 
-Requires Python 3.9+.
+Requires Python 3.9+ for the core SDK (`Agent`, `SourceRegistry`,
+`Orchestrator`). The `mcp` and `semantic-kernel` extras require Python
+3.10+ — their upstream packages don't publish Python 3.9-compatible
+releases, so `pip install` for those extras will fail on 3.9. CI tests
+3.10, 3.11, and 3.12 only; 3.9 is not covered by CI.
 
 ## Quick start
 
@@ -56,6 +60,33 @@ for a full runnable walkthrough, including framework adapters for
 LangChain, LlamaIndex, CrewAI, Semantic Kernel, PydanticAI, and Haystack
 under [`pystreammcp.integrations`](python/pystreammcp/integrations/).
 
+## Data source discovery
+
+`SourceRegistry` (`pystreammcp.SourceRegistry`) ranks *registered* data
+sources by real word-overlap relevance to a query — not a fabricated or
+hardcoded result:
+
+```python
+from pystreammcp import SourceRegistry
+
+registry = SourceRegistry()
+registry.register(
+    "orders_db",
+    description="customer order history and revenue by quarter",
+    type="database",
+    tags=["orders", "revenue"],
+)
+
+results = registry.discover("customer revenue by quarter")
+# [{"name": "orders_db", "type": "database", "relevance": 0.4444, "matched_terms": [...]}]
+```
+
+The same registry backs `POST /discover` (REST), the `pystreammcp_discover`
+MCP tool, and `LangchainAdapter.discover()` — register sources once via
+`POST /sources` / `pystreammcp_register_source` / `registry.register()`,
+then discover against them from any of the three surfaces. This is a
+simple Jaccard token-overlap heuristic, not semantic/embedding search.
+
 ## MCP orchestration & federation
 
 `Orchestrator` (`pystreammcp.Orchestrator`) discovers and routes across
@@ -153,6 +184,49 @@ pip install -e ".[dev,api,mcp,langchain,llamaindex,semantic-kernel]"
 pytest tests/ -v
 ```
 
+Note: this exact command needs Python 3.10+ (see [Install](#install)); on
+3.9 the `mcp` and `semantic-kernel` extras fail to resolve, so drop them
+from the extras list if you're on 3.9.
+
+## Known Issues
+
+- ~~No end-to-end tests for LLM failure states~~ **Fixed.** PyStreamMCP's
+  own code never calls a model provider directly (no code path here to add
+  `except RateLimitError`/`APIError` handling *to* — see `AgentFrameworkAdapter`
+  below), so what's now tested is the part this repo actually owns: that the
+  MCP tool-call path doesn't crash when whatever it's orchestrating fails.
+  `pystreammcp.testing` exports `RateLimitError`, `ContextWindowExceededError`,
+  `MalformedModelResponseError`, and `FailingAgent`/`FailingAdapter` stand-ins
+  that raise them; `mcp_server.py`'s `call_tool`/`process_message` now catch
+  exceptions raised during tool execution and return a structured
+  `{"status": "error", "error_type": ..., "message": ...}` response instead
+  of propagating uncaught. See `tests/test_mcp_server_resilience.py`.
+- ~~No Pydantic validation for the MCP tool protocol itself~~ **Fixed.**
+  Each of the four MCP tools now has a Pydantic arg model
+  (`QueryToolArgs`/`DiscoverToolArgs`/`RegisterSourceToolArgs`/`OptimizeToolArgs`
+  in `mcp_server.py`) validated in `call_tool` before dispatch (wrong types,
+  invalid enum values, and missing required fields all now return a
+  structured `validation_errors` response instead of reaching the handler
+  unchecked), and `MCPTool.input_schema` is generated from those same models
+  via `model_json_schema()` — the advertised schema and the schema actually
+  enforced can no longer drift apart the way a hand-duplicated dict could.
+- ~~No shipped mock MCP server / mock LLM fixture~~ **Fixed.** Added
+  `tests/conftest.py` (centralizing the `sys.path` setup 5 test files were
+  duplicating, plus `mock_adapter`/`failing_agent`/`failing_adapter`/
+  `llm_failure` pytest fixtures) and `pystreammcp/testing.py`, exporting
+  `MockAdapter` (promoted from the old test-local copy) and the
+  failure-injection stand-ins above for downstream projects' own test
+  suites to import directly.
+- **Per-query context discovery** now has a real implementation
+  (`pystreammcp.discovery.SourceRegistry`): register data sources (name,
+  description, type, tags), and `/discover` (REST), `pystreammcp_discover`
+  (MCP), and `LangchainAdapter.discover()` all rank *registered* sources
+  by real Jaccard token-overlap against the query — an empty registry or
+  a non-matching context returns an empty list, not fabricated data. This
+  is a simple word-overlap heuristic, not semantic/embedding search — if
+  you need that, wrap `SourceRegistry` with your own similarity scoring.
+- No open GitHub issues as of this pass.
+
 ## Rust workspace (not shipped)
 
 `core/` and `python/src/lib.rs` are an in-progress Rust workspace intended

{pystreammcp-3.2.0 → pystreammcp-3.3.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "PyStreamMCP"
-version = "3.2.0"
+version = "3.3.0"
 description = "Intelligent MCP orchestration hub - Intent understanding, capability matching, tool ranking with cohesive foundation"
 readme = "README.md"
 license = {text = "Proprietary License — Free to use with explicit attribution"}

{pystreammcp-3.2.0 → pystreammcp-3.3.0/python/PyStreamMCP.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: PyStreamMCP
-Version: 3.2.0
+Version: 3.3.0
 Summary: Intelligent MCP orchestration hub - Intent understanding, capability matching, tool ranking with cohesive foundation
 Author-email: Georgi Mammen Mullassery <mullassery@gmail.com>
 License: Proprietary License — Free to use with explicit attribution
@@ -91,7 +91,11 @@ pip install "PyStreamMCP[semantic-kernel]"  # Semantic Kernel adapter
 pip install "PyStreamMCP[all-integrations]" # every framework adapter
 ```
 
-Requires Python 3.9+.
+Requires Python 3.9+ for the core SDK (`Agent`, `SourceRegistry`,
+`Orchestrator`). The `mcp` and `semantic-kernel` extras require Python
+3.10+ — their upstream packages don't publish Python 3.9-compatible
+releases, so `pip install` for those extras will fail on 3.9. CI tests
+3.10, 3.11, and 3.12 only; 3.9 is not covered by CI.
 
 ## Quick start
 
@@ -120,6 +124,33 @@ for a full runnable walkthrough, including framework adapters for
 LangChain, LlamaIndex, CrewAI, Semantic Kernel, PydanticAI, and Haystack
 under [`pystreammcp.integrations`](python/pystreammcp/integrations/).
 
+## Data source discovery
+
+`SourceRegistry` (`pystreammcp.SourceRegistry`) ranks *registered* data
+sources by real word-overlap relevance to a query — not a fabricated or
+hardcoded result:
+
+```python
+from pystreammcp import SourceRegistry
+
+registry = SourceRegistry()
+registry.register(
+    "orders_db",
+    description="customer order history and revenue by quarter",
+    type="database",
+    tags=["orders", "revenue"],
+)
+
+results = registry.discover("customer revenue by quarter")
+# [{"name": "orders_db", "type": "database", "relevance": 0.4444, "matched_terms": [...]}]
+```
+
+The same registry backs `POST /discover` (REST), the `pystreammcp_discover`
+MCP tool, and `LangchainAdapter.discover()` — register sources once via
+`POST /sources` / `pystreammcp_register_source` / `registry.register()`,
+then discover against them from any of the three surfaces. This is a
+simple Jaccard token-overlap heuristic, not semantic/embedding search.
+
 ## MCP orchestration & federation
 
 `Orchestrator` (`pystreammcp.Orchestrator`) discovers and routes across
@@ -217,6 +248,49 @@ pip install -e ".[dev,api,mcp,langchain,llamaindex,semantic-kernel]"
 pytest tests/ -v
 ```
 
+Note: this exact command needs Python 3.10+ (see [Install](#install)); on
+3.9 the `mcp` and `semantic-kernel` extras fail to resolve, so drop them
+from the extras list if you're on 3.9.
+
+## Known Issues
+
+- ~~No end-to-end tests for LLM failure states~~ **Fixed.** PyStreamMCP's
+  own code never calls a model provider directly (no code path here to add
+  `except RateLimitError`/`APIError` handling *to* — see `AgentFrameworkAdapter`
+  below), so what's now tested is the part this repo actually owns: that the
+  MCP tool-call path doesn't crash when whatever it's orchestrating fails.
+  `pystreammcp.testing` exports `RateLimitError`, `ContextWindowExceededError`,
+  `MalformedModelResponseError`, and `FailingAgent`/`FailingAdapter` stand-ins
+  that raise them; `mcp_server.py`'s `call_tool`/`process_message` now catch
+  exceptions raised during tool execution and return a structured
+  `{"status": "error", "error_type": ..., "message": ...}` response instead
+  of propagating uncaught. See `tests/test_mcp_server_resilience.py`.
+- ~~No Pydantic validation for the MCP tool protocol itself~~ **Fixed.**
+  Each of the four MCP tools now has a Pydantic arg model
+  (`QueryToolArgs`/`DiscoverToolArgs`/`RegisterSourceToolArgs`/`OptimizeToolArgs`
+  in `mcp_server.py`) validated in `call_tool` before dispatch (wrong types,
+  invalid enum values, and missing required fields all now return a
+  structured `validation_errors` response instead of reaching the handler
+  unchecked), and `MCPTool.input_schema` is generated from those same models
+  via `model_json_schema()` — the advertised schema and the schema actually
+  enforced can no longer drift apart the way a hand-duplicated dict could.
+- ~~No shipped mock MCP server / mock LLM fixture~~ **Fixed.** Added
+  `tests/conftest.py` (centralizing the `sys.path` setup 5 test files were
+  duplicating, plus `mock_adapter`/`failing_agent`/`failing_adapter`/
+  `llm_failure` pytest fixtures) and `pystreammcp/testing.py`, exporting
+  `MockAdapter` (promoted from the old test-local copy) and the
+  failure-injection stand-ins above for downstream projects' own test
+  suites to import directly.
+- **Per-query context discovery** now has a real implementation
+  (`pystreammcp.discovery.SourceRegistry`): register data sources (name,
+  description, type, tags), and `/discover` (REST), `pystreammcp_discover`
+  (MCP), and `LangchainAdapter.discover()` all rank *registered* sources
+  by real Jaccard token-overlap against the query — an empty registry or
+  a non-matching context returns an empty list, not fabricated data. This
+  is a simple word-overlap heuristic, not semantic/embedding search — if
+  you need that, wrap `SourceRegistry` with your own similarity scoring.
+- No open GitHub issues as of this pass.
+
 ## Rust workspace (not shipped)
 
 `core/` and `python/src/lib.rs` are an in-progress Rust workspace intended

{pystreammcp-3.2.0 → pystreammcp-3.3.0}/python/PyStreamMCP.egg-info/SOURCES.txt

@@ -31,6 +31,7 @@ python/pystreammcp/quality.py
 python/pystreammcp/query.py
 python/pystreammcp/server.py
 python/pystreammcp/streaming.py
+python/pystreammcp/testing.py
 python/pystreammcp/webhook_handlers.py
 python/pystreammcp/webhook_router.py
 python/pystreammcp/integrations/__init__.py
@@ -57,10 +58,12 @@ python/pystreammcp/qa/__init__.py
 python/pystreammcp/qa/validator.py
 python/pystreammcp/scripts/__init__.py
 python/pystreammcp/scripts/post_install.py
+tests/test_discovery.py
 tests/test_integration_phase2.py
 tests/test_langchain_integration.py
 tests/test_llamaindex_integration.py
 tests/test_mcp_connector_discovery.py
+tests/test_mcp_server_resilience.py
 tests/test_mcp_tools_handler.py
 tests/test_okf_core.py
 tests/test_okf_discovery.py

{pystreammcp-3.2.0 → pystreammcp-3.3.0}/python/pystreammcp/__init__.py

@@ -7,12 +7,12 @@ Reduces token usage by 60-75% while maintaining quality.
 Includes StatGuardian integration for data quality validation.
 """
 
-__version__ = "3.2.0"
+__version__ = "3.3.0"
 
 from .agent import Agent, QueryResult
 from .query import Query, QueryIntent, QueryConstraints
 from .context import Context, ContextType, ContextWindow
-from .discovery import Discovery, DiscoveredSource, SourceType
+from .discovery import Discovery, DiscoveredSource, SourceType, SourceRegistry, DataSource
 
 # Optimization imports
 try:
@@ -79,6 +79,8 @@ __all__ = [
     "Discovery",
     "DiscoveredSource",
     "SourceType",
+    "SourceRegistry",
+    "DataSource",
     # Legacy optimization (optional, phase 1-2)
     *((["OptimizationStrategy"]) if OptimizationStrategy else []),
     *((["StrategyType"]) if StrategyType else []),

{pystreammcp-3.2.0 → pystreammcp-3.3.0}/python/pystreammcp/api.py

@@ -13,6 +13,7 @@ import logging
 
 from .adapters import AdapterRegistry, AdapterConfig, FrameworkType, QueryResult
 from .agent import Agent
+from .discovery import SourceRegistry
 
 
 logger = logging.getLogger(__name__)
@@ -59,6 +60,15 @@ class DiscoveryResponse(BaseModel):
     total_sources: int
 
 
+class SourceRegistration(BaseModel):
+    """Register a data source for discovery."""
+
+    name: str = Field(..., description="Unique source name")
+    description: str = Field(..., description="What this source contains, in plain text")
+    type: str = Field("unknown", description="Source type, e.g. database, api, docs")
+    tags: List[str] = Field(default_factory=list, description="Optional keyword tags")
+
+
 class OptimizationRequest(BaseModel):
     """Query optimization request."""
 
@@ -117,6 +127,7 @@ class PyStreamMCPAPI:
         self.title = title
         self.version = version
         self.agents: Dict[str, Agent] = {}
+        self.registry = SourceRegistry()
         self.app = self._create_app()
 
     def _create_app(self) -> FastAPI:
@@ -260,13 +271,45 @@ class PyStreamMCPAPI:
             }
 
         # Discovery Endpoints
+        @app.post("/sources", response_model=Dict[str, Any])
+        async def register_source(source: SourceRegistration):
+            """Register a data source so /discover can find it."""
+            self.registry.register(
+                name=source.name,
+                description=source.description,
+                type=source.type,
+                tags=source.tags,
+            )
+            return {"status": "success", "name": source.name}
+
+        @app.delete("/sources/{name}")
+        async def unregister_source(name: str):
+            """Remove a registered data source."""
+            removed = self.registry.unregister(name)
+            if not removed:
+                raise HTTPException(status_code=404, detail=f"Source {name} not found")
+            return {"status": "success", "name": name}
+
+        @app.get("/sources")
+        async def list_sources():
+            """List all registered data sources."""
+            return {
+                "status": "success",
+                "sources": [
+                    {"name": s.name, "type": s.type, "tags": s.tags}
+                    for s in self.registry.list_sources()
+                ],
+            }
+
         @app.post("/discover", response_model=DiscoveryResponse)
         async def discover(request: DiscoveryRequest):
-            """Discover relevant data sources."""
-            agent = self._get_agent(request.agent_id)
+            """Discover data sources relevant to `context`.
 
-            # TODO: Implement discovery logic
-            sources = []
+            Ranks *registered* sources (see POST /sources) by real token
+            overlap against `context`. Returns an empty list if nothing is
+            registered or nothing overlaps — never a fabricated result.
+            """
+            sources = self.registry.discover(request.context)
 
             return {
                 "status": "success",

pystreammcp-3.3.0/python/pystreammcp/discovery.py

@@ -0,0 +1,216 @@
+"""Discovery types and logic for PyStreamMCP.
+
+`SourceRegistry` is real discovery logic: register data sources (name,
+description, type, tags), then find the ones most relevant to a query
+context using word-overlap relevance scoring. This is deliberately a
+simple, honest heuristic — Jaccard token overlap between the query context
+and each source's registered description/tags/name — not a semantic or
+embedding-based search. Sources with zero token overlap are excluded from
+results rather than padded in with a fabricated score, so an empty
+registry or a context that matches nothing returns no sources, not
+placeholder data.
+
+`Discovery` / `DiscoveredSource` / `SourceType` are the result types
+`SourceRegistry.discover()` returns results as, so callers get real,
+structured discovery results rather than a bare dict.
+"""
+
+from __future__ import annotations
+
+import re
+from dataclasses import dataclass, field
+from datetime import datetime
+from enum import Enum
+from typing import Any, Dict, List, Optional, Set
+
+_WORD_RE = re.compile(r"[a-z0-9]+")
+
+
+def _tokenize(text: str) -> Set[str]:
+    return set(_WORD_RE.findall(text.lower()))
+
+
+class SourceType(str, Enum):
+    """Type of data source."""
+
+    TABLE = "table"
+    INDEX = "index"
+    CACHE = "cache"
+    EXTERNAL = "external"
+    COMPUTED = "computed"
+    UNKNOWN = "unknown"
+
+
+@dataclass
+class DiscoveredSource:
+    """A discovered data source, returned by SourceRegistry.discover()."""
+
+    name: str
+    source_type: SourceType
+    relevance_score: float
+    estimated_tokens: int
+    freshness_score: float = 1.0
+    source_id: str = ""
+    created_at: datetime = field(default_factory=datetime.now)
+
+    def __post_init__(self):
+        if not self.source_id:
+            import uuid
+
+            self.source_id = str(uuid.uuid4())
+        self.relevance_score = max(0.0, min(1.0, self.relevance_score))
+        self.freshness_score = max(0.0, min(1.0, self.freshness_score))
+
+    def quality_score(self) -> float:
+        """Combined relevance and freshness score."""
+        return self.relevance_score * self.freshness_score
+
+    def with_freshness(self, score: float) -> "DiscoveredSource":
+        """Set freshness score."""
+        self.freshness_score = max(0.0, min(1.0, score))
+        return self
+
+
+@dataclass
+class Discovery:
+    """Discovery result for a query."""
+
+    query_id: str
+    discovered_sources: List[DiscoveredSource] = field(default_factory=list)
+    discovery_id: str = ""
+    created_at: datetime = field(default_factory=datetime.now)
+
+    def __post_init__(self):
+        if not self.discovery_id:
+            import uuid
+
+            self.discovery_id = str(uuid.uuid4())
+
+    def add_source(self, source: DiscoveredSource) -> "Discovery":
+        """Add a discovered source."""
+        self.discovered_sources.append(source)
+        return self
+
+    def total_available_tokens(self) -> int:
+        """Total tokens across all sources."""
+        return sum(s.estimated_tokens for s in self.discovered_sources)
+
+    def top_sources(self, limit: int = 5) -> List[DiscoveredSource]:
+        """Get top sources by quality score."""
+        sorted_sources = sorted(
+            self.discovered_sources, key=lambda s: s.quality_score(), reverse=True
+        )
+        return sorted_sources[:limit]
+
+    def high_relevance_sources(self, threshold: float = 0.8) -> List[DiscoveredSource]:
+        """Get high-relevance sources."""
+        return [s for s in self.discovered_sources if s.relevance_score >= threshold]
+
+
+@dataclass
+class DataSource:
+    """A data source registered with a SourceRegistry."""
+
+    name: str
+    description: str
+    type: str = "unknown"
+    tags: List[str] = field(default_factory=list)
+    metadata: Dict[str, Any] = field(default_factory=dict)
+    estimated_tokens: int = 0
+
+    def _text(self) -> str:
+        return " ".join([self.name, self.description, *self.tags])
+
+
+class SourceRegistry:
+    """In-memory registry of data sources, with real relevance scoring."""
+
+    def __init__(self) -> None:
+        self._sources: Dict[str, DataSource] = {}
+
+    def register(
+        self,
+        name: str,
+        description: str,
+        type: str = "unknown",
+        tags: Optional[List[str]] = None,
+        metadata: Optional[Dict[str, Any]] = None,
+        estimated_tokens: int = 0,
+    ) -> DataSource:
+        """Register (or replace) a data source by name."""
+        source = DataSource(
+            name=name,
+            description=description,
+            type=type,
+            tags=list(tags) if tags else [],
+            metadata=dict(metadata) if metadata else {},
+            estimated_tokens=estimated_tokens,
+        )
+        self._sources[name] = source
+        return source
+
+    def unregister(self, name: str) -> bool:
+        """Remove a registered source. Returns True if it existed."""
+        return self._sources.pop(name, None) is not None
+
+    def list_sources(self) -> List[DataSource]:
+        return list(self._sources.values())
+
+    def discover(self, context: str, limit: int = 10) -> List[Dict[str, Any]]:
+        """Return registered sources ranked by real relevance to `context`.
+
+        Relevance is Jaccard token overlap: |context tokens ∩ source
+        tokens| / |context tokens ∪ source tokens|. Sources that share no
+        tokens with `context` are excluded, not returned with a made-up
+        low score. Returns plain dicts (the wire-format shape used by the
+        REST/MCP surfaces) — use `discover_typed` for `DiscoveredSource`
+        objects.
+        """
+        return [
+            {
+                "name": s.name,
+                "type": s.type,
+                "relevance": round(score, 4),
+                "matched_terms": matched,
+            }
+            for score, matched, s in self._score(context, limit)
+        ]
+
+    def discover_typed(self, query_id: str, context: str, limit: int = 10) -> Discovery:
+        """Same ranking as `discover`, returned as a `Discovery` result of
+        real `DiscoveredSource` objects instead of plain dicts."""
+        result = Discovery(query_id=query_id)
+        for score, _matched, s in self._score(context, limit):
+            try:
+                source_type = SourceType(s.type)
+            except ValueError:
+                source_type = SourceType.UNKNOWN
+            result.add_source(
+                DiscoveredSource(
+                    name=s.name,
+                    source_type=source_type,
+                    relevance_score=score,
+                    estimated_tokens=s.estimated_tokens,
+                )
+            )
+        return result
+
+    def _score(self, context: str, limit: int):
+        context_tokens = _tokenize(context)
+        if not context_tokens or not self._sources:
+            return []
+
+        scored = []
+        for source in self._sources.values():
+            source_tokens = _tokenize(source._text())
+            if not source_tokens:
+                continue
+            overlap = context_tokens & source_tokens
+            if not overlap:
+                continue
+            union = context_tokens | source_tokens
+            relevance = len(overlap) / len(union)
+            scored.append((relevance, sorted(overlap), source))
+
+        scored.sort(key=lambda item: item[0], reverse=True)
+        return scored[:limit]