cocoindex-code 0.1.8__tar.gz → 0.1.9__tar.gz

{cocoindex_code-0.1.8 → cocoindex_code-0.1.9}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: cocoindex-code
-Version: 0.1.8
+Version: 0.1.9
 Summary: MCP server for indexing and querying codebases using CocoIndex
 Project-URL: Homepage, https://github.com/cocoindex-io/cocoindex-code
 Project-URL: Repository, https://github.com/cocoindex-io/cocoindex-code
@@ -17,7 +17,7 @@ Classifier: Programming Language :: Python :: 3.12
 Classifier: Programming Language :: Python :: 3.13
 Classifier: Topic :: Software Development :: Libraries :: Python Modules
 Requires-Python: >=3.11
-Requires-Dist: cocoindex[litellm]==1.0.0a23
+Requires-Dist: cocoindex[litellm]==1.0.0a24
 Requires-Dist: einops>=0.8.2
 Requires-Dist: mcp>=1.0.0
 Requires-Dist: numpy>=1.24.0
@@ -40,6 +40,10 @@ Description-Content-Type: text/markdown
 
 <h1 align="center">light weight MCP for code that just works </h1>
 
+![effect](https://github.com/user-attachments/assets/cb3a4cae-0e1f-49c4-890b-7bb93317ab60)
+
+
+
 
 A super light-weight, effective embedded MCP **(AST-based)** that understand and searches your codebase that just works! Using [CocoIndex](https://github.com/cocoindex-io/cocoindex) - an Rust-based ultra performant data transformation engine. No blackbox. Works for Claude, Codex, Cursor - any coding agent.
 
@@ -116,6 +120,20 @@ Or use opencode.json:
 
 Optionally, you can run `cocoindex-code index` to create or update the index. Without running it, the MCP server will automatically build and keep the index up-to-date in the background.
 
+## When Is the MCP Triggered?
+
+Once configured, your coding agent (Claude Code, Codex, Cursor, etc.) automatically decides when semantic code search is helpful — especially for finding code by description, exploring unfamiliar codebases, fuzzy/conceptual matches, or locating implementations without knowing exact names.
+
+You can also nudge the agent explicitly, e.g. *"Use the cocoindex-code MCP to find how user sessions are managed."* For persistent instructions, add guidance to your project's `AGENTS.md` or `CLAUDE.md`:
+
+```
+Use the cocoindex-code MCP server for semantic code search when:
+- Searching for code by meaning or description rather than exact text
+- Exploring unfamiliar parts of the codebase
+- Looking for implementations without knowing exact names
+- Finding similar code patterns or related functionality
+```
+
 ## Features
 - **Semantic Code Search**: Find relevant code using natural language queries when grep doesn't work well, and save tokens immediately.
 - **Ultra Performant to code changes**:⚡ Built on top of ultra performant [Rust indexing engine](https://github.com/cocoindex-io/cocoindex/edit/main/README.md). Only re-indexes changed files for fast updates.
@@ -131,6 +149,7 @@ Optionally, you can run `cocoindex-code index` to create or update the index. Wi
 | `COCOINDEX_CODE_ROOT_PATH` | Root path of the codebase | Auto-discovered (see below) |
 | `COCOINDEX_CODE_EMBEDDING_MODEL` | Embedding model (see below) | `sbert/sentence-transformers/all-MiniLM-L6-v2` |
 | `COCOINDEX_CODE_BATCH_SIZE` | Max batch size for local embedding model | `16` |
+| `COCOINDEX_CODE_EXTRA_EXTENSIONS` | Additional file extensions to index (comma-separated, e.g. `"inc:php,yaml,toml"` — use `ext:lang` to override language detection) | _(none)_ |
 
 
 ### Root Path Discovery

{cocoindex_code-0.1.8 → cocoindex_code-0.1.9}/README.md

@@ -5,6 +5,10 @@
 
 <h1 align="center">light weight MCP for code that just works </h1>
 
+![effect](https://github.com/user-attachments/assets/cb3a4cae-0e1f-49c4-890b-7bb93317ab60)
+
+
+
 
 A super light-weight, effective embedded MCP **(AST-based)** that understand and searches your codebase that just works! Using [CocoIndex](https://github.com/cocoindex-io/cocoindex) - an Rust-based ultra performant data transformation engine. No blackbox. Works for Claude, Codex, Cursor - any coding agent.
 
@@ -81,6 +85,20 @@ Or use opencode.json:
 
 Optionally, you can run `cocoindex-code index` to create or update the index. Without running it, the MCP server will automatically build and keep the index up-to-date in the background.
 
+## When Is the MCP Triggered?
+
+Once configured, your coding agent (Claude Code, Codex, Cursor, etc.) automatically decides when semantic code search is helpful — especially for finding code by description, exploring unfamiliar codebases, fuzzy/conceptual matches, or locating implementations without knowing exact names.
+
+You can also nudge the agent explicitly, e.g. *"Use the cocoindex-code MCP to find how user sessions are managed."* For persistent instructions, add guidance to your project's `AGENTS.md` or `CLAUDE.md`:
+
+```
+Use the cocoindex-code MCP server for semantic code search when:
+- Searching for code by meaning or description rather than exact text
+- Exploring unfamiliar parts of the codebase
+- Looking for implementations without knowing exact names
+- Finding similar code patterns or related functionality
+```
+
 ## Features
 - **Semantic Code Search**: Find relevant code using natural language queries when grep doesn't work well, and save tokens immediately.
 - **Ultra Performant to code changes**:⚡ Built on top of ultra performant [Rust indexing engine](https://github.com/cocoindex-io/cocoindex/edit/main/README.md). Only re-indexes changed files for fast updates.
@@ -96,6 +114,7 @@ Optionally, you can run `cocoindex-code index` to create or update the index. Wi
 | `COCOINDEX_CODE_ROOT_PATH` | Root path of the codebase | Auto-discovered (see below) |
 | `COCOINDEX_CODE_EMBEDDING_MODEL` | Embedding model (see below) | `sbert/sentence-transformers/all-MiniLM-L6-v2` |
 | `COCOINDEX_CODE_BATCH_SIZE` | Max batch size for local embedding model | `16` |
+| `COCOINDEX_CODE_EXTRA_EXTENSIONS` | Additional file extensions to index (comma-separated, e.g. `"inc:php,yaml,toml"` — use `ext:lang` to override language detection) | _(none)_ |
 
 
 ### Root Path Discovery

{cocoindex_code-0.1.8 → cocoindex_code-0.1.9}/pyproject.toml

@@ -23,7 +23,7 @@ classifiers = [
 
 dependencies = [
     "mcp>=1.0.0",
-    "cocoindex[litellm]==1.0.0a23",
+    "cocoindex[litellm]==1.0.0a24",
     "sentence-transformers>=2.2.0",
     "sqlite-vec>=0.1.0",
     "pydantic>=2.0.0",
@@ -82,10 +82,6 @@ python_version = "3.11"
 strict = true
 ignore_missing_imports = true
 
-[[tool.mypy.overrides]]
-module = "cocoindex_code.embedder"
-warn_unused_ignores = false
-
 [tool.pytest.ini_options]
 testpaths = ["tests"]
 python_files = ["test_*.py"]

{cocoindex_code-0.1.8 → cocoindex_code-0.1.9}/src/cocoindex_code/config.py

@@ -6,7 +6,6 @@ import os
 from dataclasses import dataclass
 from pathlib import Path
 
-_SBERT_PREFIX = "sbert/"
 _DEFAULT_MODEL = "sbert/sentence-transformers/all-MiniLM-L6-v2"
 
 
@@ -65,7 +64,7 @@ class Config:
     index_dir: Path
     device: str
     trust_remote_code: bool
-    batch_size: int
+    extra_extensions: dict[str, str | None]
 
     @classmethod
     def from_env(cls) -> Config:
@@ -100,18 +99,18 @@ class Config:
             "yes",
         )
 
-        # Batch size for local embedding model
-        _raw_batch_size = os.environ.get("COCOINDEX_CODE_BATCH_SIZE", "16")
-        try:
-            batch_size = int(_raw_batch_size)
-        except ValueError:
-            raise ValueError(
-                f"COCOINDEX_CODE_BATCH_SIZE must be a positive integer, got: {_raw_batch_size!r}"
-            ) from None
-        if batch_size <= 0:
-            raise ValueError(
-                f"COCOINDEX_CODE_BATCH_SIZE must be a positive integer, got: {batch_size}"
-            )
+        # Extra file extensions (format: "inc:php,yaml,toml" — optional lang after colon)
+        raw_extra = os.environ.get("COCOINDEX_CODE_EXTRA_EXTENSIONS", "")
+        extra_extensions: dict[str, str | None] = {}
+        for token in raw_extra.split(","):
+            token = token.strip()
+            if not token:
+                continue
+            if ":" in token:
+                ext, lang = token.split(":", 1)
+                extra_extensions[f".{ext.strip()}"] = lang.strip() or None
+            else:
+                extra_extensions[f".{token}"] = None
 
         return cls(
             codebase_root_path=root,
@@ -119,7 +118,7 @@ class Config:
             index_dir=index_dir,
             device=device,
             trust_remote_code=trust_remote_code,
-            batch_size=batch_size,
+            extra_extensions=extra_extensions,
         )
 
     @property

{cocoindex_code-0.1.8 → cocoindex_code-0.1.9}/src/cocoindex_code/indexer.py

@@ -8,10 +8,11 @@ from cocoindex.resources.chunk import Chunk
 from cocoindex.resources.file import PatternFilePathMatcher
 from cocoindex.resources.id import IdGenerator
 
+from .config import config
 from .shared import CODEBASE_DIR, SQLITE_DB, CodeChunk, embedder
 
 # File patterns for supported languages
-INCLUDED_PATTERNS = [
+DEFAULT_INCLUDED_PATTERNS = [
     "**/*.py",  # Python
     "**/*.pyi",  # Python stubs
     "**/*.js",  # JavaScript
@@ -43,6 +44,13 @@ INCLUDED_PATTERNS = [
     "**/*.php",  # PHP
 ]
 
+INCLUDED_PATTERNS = DEFAULT_INCLUDED_PATTERNS + [f"**/*{ext}" for ext in config.extra_extensions]
+
+# Language overrides from extra_extensions (e.g. ".inc" -> "php")
+LANGUAGE_OVERRIDES: dict[str, str] = {
+    ext: lang for ext, lang in config.extra_extensions.items() if lang is not None
+}
+
 EXCLUDED_PATTERNS = [
     "**/.*",  # Hidden directories
     "**/__pycache__",  # Python cache
@@ -81,7 +89,12 @@ async def process_file(
         return
 
     # Get relative path and detect language
-    language = detect_code_language(filename=file.file_path.path.name) or "text"
+    suffix = file.file_path.path.suffix
+    language = (
+        LANGUAGE_OVERRIDES.get(suffix)
+        or detect_code_language(filename=file.file_path.path.name)
+        or "text"
+    )
 
     # Split into chunks
     chunks = splitter.split(

{cocoindex_code-0.1.8 → cocoindex_code-0.1.9}/src/cocoindex_code/query.py

@@ -8,7 +8,7 @@ import cocoindex as coco
 
 from .config import config
 from .schema import QueryResult
-from .shared import SQLITE_DB, embedder
+from .shared import SQLITE_DB, embedder, query_prompt_name
 
 
 def _l2_to_score(distance: float) -> float:
@@ -106,10 +106,7 @@ async def query_codebase(
     db = coco_env.get_context(SQLITE_DB)
 
     # Generate query embedding.
-    if hasattr(embedder, "embed_query"):
-        query_embedding = await embedder.embed_query(query)
-    else:
-        query_embedding = await embedder.embed(query)
+    query_embedding = await embedder.embed(query, True, query_prompt_name)
 
     embedding_bytes = query_embedding.astype("float32").tobytes()
 

{cocoindex_code-0.1.8 → cocoindex_code-0.1.9}/src/cocoindex_code/shared.py

@@ -14,8 +14,7 @@ from numpy.typing import NDArray
 
 if TYPE_CHECKING:
     from cocoindex.ops.litellm import LiteLLMEmbedder
-
-    from .embedder import LocalEmbedder
+    from cocoindex.ops.sentence_transformers import SentenceTransformerEmbedder
 
 from .config import config
 
@@ -24,22 +23,21 @@ logger = logging.getLogger(__name__)
 SBERT_PREFIX = "sbert/"
 
 # Initialize embedder at module level based on model prefix
-embedder: LocalEmbedder | LiteLLMEmbedder
+embedder: SentenceTransformerEmbedder | LiteLLMEmbedder
 if config.embedding_model.startswith(SBERT_PREFIX):
-    from .embedder import LocalEmbedder
+    from cocoindex.ops.sentence_transformers import SentenceTransformerEmbedder
 
     _model_name = config.embedding_model[len(SBERT_PREFIX) :]
     # Models that define a "query" prompt for asymmetric retrieval.
     _QUERY_PROMPT_MODELS = {"nomic-ai/nomic-embed-code", "nomic-ai/CodeRankEmbed"}
-    _query_prompt_name: str | None = "query" if _model_name in _QUERY_PROMPT_MODELS else None
+    query_prompt_name: str | None = "query" if _model_name in _QUERY_PROMPT_MODELS else None
     # Models whose custom remote code is known-compatible with transformers 5.x.
     _KNOWN_REMOTE_CODE_MODELS = {"nomic-ai/CodeRankEmbed"}
     _trust = config.trust_remote_code or _model_name in _KNOWN_REMOTE_CODE_MODELS
-    embedder = LocalEmbedder(
+    embedder = SentenceTransformerEmbedder(
         _model_name,
         device=config.device,
         trust_remote_code=_trust,
-        query_prompt_name=_query_prompt_name,
     )
     logger.info(
         "Embedding model: %s | device: %s | trust_remote_code: %s",
@@ -51,6 +49,7 @@ else:
     from cocoindex.ops.litellm import LiteLLMEmbedder
 
     embedder = LiteLLMEmbedder(config.embedding_model)
+    query_prompt_name = None
     logger.info("Embedding model (LiteLLM): %s", config.embedding_model)
 
 # Context key for SQLite database (connection managed in lifespan)

cocoindex_code-0.1.8/src/cocoindex_code/embedder.py

@@ -1,119 +0,0 @@
-"""Local SentenceTransformer embedder with device and trust_remote_code support."""
-
-from __future__ import annotations
-
-import threading
-from typing import TYPE_CHECKING, Any
-
-import cocoindex as coco
-import numpy as np
-from cocoindex.resources import schema as _schema
-from numpy.typing import NDArray
-
-from .config import config as _config
-
-if TYPE_CHECKING:
-    from sentence_transformers import SentenceTransformer
-
-
-class LocalEmbedder(_schema.VectorSchemaProvider):
-    """SentenceTransformer embedder with explicit device and trust_remote_code support.
-
-    Drop-in replacement for cocoindex's SentenceTransformerEmbedder that supports:
-    - Explicit device selection (e.g. "cuda", "cpu")
-    - trust_remote_code for models with custom pooling (e.g. Jina models)
-    """
-
-    def __init__(
-        self,
-        model_name_or_path: str,
-        *,
-        device: str = "cpu",
-        trust_remote_code: bool = False,
-        normalize_embeddings: bool = True,
-        query_prompt_name: str | None = None,
-    ) -> None:
-        self._model_name_or_path = model_name_or_path
-        self._device = device
-        self._trust_remote_code = trust_remote_code
-        self._normalize_embeddings = normalize_embeddings
-        self._query_prompt_name = query_prompt_name
-        self._model: SentenceTransformer | None = None
-        self._lock = threading.Lock()
-
-    def __getstate__(self) -> dict[str, Any]:
-        return {
-            "model_name_or_path": self._model_name_or_path,
-            "device": self._device,
-            "trust_remote_code": self._trust_remote_code,
-            "normalize_embeddings": self._normalize_embeddings,
-            "query_prompt_name": self._query_prompt_name,
-        }
-
-    def __setstate__(self, state: dict[str, Any]) -> None:
-        self._model_name_or_path = state["model_name_or_path"]
-        self._device = state["device"]
-        self._trust_remote_code = state["trust_remote_code"]
-        self._normalize_embeddings = state["normalize_embeddings"]
-        self._query_prompt_name = state.get("query_prompt_name")
-        self._model = None
-        self._lock = threading.Lock()
-
-    def _get_model(self) -> SentenceTransformer:
-        """Lazy-load the model with thread-safe double-checked locking."""
-        if self._model is None:
-            with self._lock:
-                if self._model is None:
-                    from sentence_transformers import SentenceTransformer
-
-                    self._model = SentenceTransformer(
-                        self._model_name_or_path,
-                        device=self._device,
-                        trust_remote_code=self._trust_remote_code,
-                    )
-        return self._model
-
-    @coco.fn.as_async(batching=True, runner=coco.GPU, memo=True, max_batch_size=_config.batch_size)
-    def embed(self, texts: list[str]) -> list[NDArray[np.float32]]:
-        """Embed a batch of texts into float32 vectors."""
-        model = self._get_model()
-        embeddings: NDArray[np.float32] = model.encode(
-            texts,
-            convert_to_numpy=True,
-            normalize_embeddings=self._normalize_embeddings,
-            show_progress_bar=False,
-        )  # type: ignore[assignment]
-        return list(embeddings)
-
-    @coco.fn.as_async(batching=True, runner=coco.GPU, memo=True, max_batch_size=_config.batch_size)
-    def embed_query(self, texts: list[str]) -> list[NDArray[np.float32]]:
-        """Embed query texts, applying query_prompt_name if configured."""
-        model = self._get_model()
-        embeddings: NDArray[np.float32] = model.encode(
-            texts,
-            prompt_name=self._query_prompt_name,
-            convert_to_numpy=True,
-            normalize_embeddings=self._normalize_embeddings,
-            show_progress_bar=False,
-        )  # type: ignore[assignment]
-        return list(embeddings)
-
-    @coco.fn.as_async(runner=coco.GPU, memo=True)
-    def __coco_vector_schema__(self) -> _schema.VectorSchema:
-        """Return the vector schema (dimension + dtype) for this model."""
-        model = self._get_model()
-        dim = model.get_sentence_embedding_dimension()
-        if dim is None:
-            raise RuntimeError(
-                f"Embedding dimension is unknown for model {self._model_name_or_path}."
-            )
-        return _schema.VectorSchema(dtype=np.dtype(np.float32), size=dim)
-
-    def __coco_memo_key__(self) -> object:
-        return (
-            self._model_name_or_path,
-            self._device,
-            self._trust_remote_code,
-            self._normalize_embeddings,
-            self._query_prompt_name,
-        )