kodit 0.2.8__tar.gz → 0.3.0__tar.gz

kodit-0.3.0/.cursor/rules/style.mdc

@@ -0,0 +1,11 @@
+---
+description: 
+globs: 
+alwaysApply: true
+---
+Please maintain the following coding standards while developing code:
+
+1. This project uses a domain-driven design approach. Please consider in which layer
+   your code belongs.
+2. Run `uv run ruff check --fix` to lint the code.
+3. Run `uv run pytest ...` to test the code.

{kodit-0.2.8 → kodit-0.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: kodit
-Version: 0.2.8
+Version: 0.3.0
 Summary: Code indexing for better AI code generation
 Project-URL: Homepage, https://docs.helixml.tech/kodit/
 Project-URL: Documentation, https://docs.helixml.tech/kodit/

{kodit-0.2.8 → kodit-0.3.0}/docs/reference/deployment/docker-compose.yaml

@@ -29,6 +29,10 @@ services:
       ENRICHMENT_ENDPOINT_API_KEY: REPLACE_WITH_YOUR_API_KEY
       ENRICHMENT_ENDPOINT_MODEL: o3-mini
 
+      # Auto-indexing configuration
+      AUTO_INDEXING_SOURCES_0_URI: https://github.com/pydantic/pydantic
+      AUTO_INDEXING_SOURCES_1_URI: https://github.com/fastapi/fastapi
+      AUTO_INDEXING_SOURCES_2_URI: https://github.com/helix-ml/kodit
 
   vectorchord:
     image: tensorchord/vchord-suite:pg17-20250601

{kodit-0.2.8 → kodit-0.3.0}/docs/reference/deployment/kubernetes.yaml

@@ -77,6 +77,10 @@ spec:
               value: "REPLACE_WITH_YOUR_API_KEY"
             - name: ENRICHMENT_ENDPOINT_MODEL
               value: "o3-mini"
+            - name: AUTO_INDEXING_SOURCES_0_URI
+              value: "https://github.com/pydantic/pydantic"
+            - name: AUTO_INDEXING_SOURCES_1_URI
+              value: "https://github.com/helix-ml/kodit"
           ports:
             - containerPort: 8080
           readinessProbe:

{kodit-0.2.8 → kodit-0.3.0}/docs/reference/indexing/index.md

@@ -42,7 +42,7 @@ Kodit can index local directories on your filesystem:
 
 ## Basic Usage
 
-### Indexing a Source
+### Manual Indexing
 
 To index a source, use the `kodit index` command followed by the source location:
 
@@ -75,6 +75,61 @@ This will display a table showing:
 - Source URI
 - Number of snippets extracted
 
+### Auto-Indexing
+
+If you're running Kodit as a shared server, you need to configure what gets indexed.
+Auto-indexing is a simple indexing configuration powered by environmental variables.
+
+#### Configuration via Environment Variables
+
+Configure auto-indexing sources using environment variables with the `AUTO_INDEXING_SOURCES_{X}_` prefix:
+
+```sh
+# Configure a single auto-index source
+export AUTO_INDEXING_SOURCES_0_URI="https://github.com/pydantic/pydantic"
+
+# Configure multiple auto-index sources
+export AUTO_INDEXING_SOURCES_0_URI="https://github.com/pydantic/pydantic"
+export AUTO_INDEXING_SOURCES_1_URI="https://github.com/fastapi/fastapi"
+export AUTO_INDEXING_SOURCES_2_URI="/path/to/local/project"
+
+# Or use a .env file
+echo "AUTO_INDEXING_SOURCES_0_URI=https://github.com/pydantic/pydantic" >> .env
+echo "AUTO_INDEXING_SOURCES_1_URI=https://github.com/fastapi/fastapi" >> .env
+echo "AUTO_INDEXING_SOURCES_2_URI=/path/to/local/project" >> .env
+```
+
+**Configuration Format:**
+
+- Use `AUTO_INDEXING_SOURCES_N_URI` where `N` is a zero-based index
+- Sources are indexed in numerical order (0, 1, 2, etc.)
+- Supports all source types: Git repositories (HTTPS/SSH) and local directories
+- Gaps in numbering are allowed (e.g., 0, 2, 5 will work)
+
+#### Using Auto-Indexing
+
+To manually index all configured auto-index sources:
+
+```sh
+kodit index --auto-index
+```
+
+This command will:
+
+1. Read the auto-indexing configuration from environment variables
+2. Index each configured source in sequence
+3. Show progress for each source being indexed
+4. Handle errors gracefully and continue with remaining sources
+
+If no auto-index sources are configured, the command will display a message indicating
+that no sources are configured.
+
+To automatically run index all configured auto-index sources:
+
+```sh
+kodit serve
+```
+
 ## Git Protocol Support
 
 ### HTTPS Authentication
@@ -226,15 +281,6 @@ Kodit shows progress during indexing operations:
 - Snippet extraction progress
 - Index building progress (BM25, embeddings)
 
-### Error Handling
-
-Common issues and solutions:
-
-1. **Authentication Errors**: Ensure your credentials are correct
-2. **Network Issues**: Check your internet connection and firewall settings
-3. **Permission Errors**: Ensure you have read access to the source
-4. **Unsupported Files**: Kodit will skip unsupported file types automatically
-
 ## Privacy and Security
 
 ### Local Processing
@@ -265,6 +311,8 @@ Kodit respects privacy by honoring:
 2. **"Unsupported source"**: Ensure the path or URL is valid and accessible
 3. **"No snippets found"**: Check if the source contains supported file types
 4. **"Permission denied"**: Ensure you have read access to the source
+5. **"No auto-index sources configured"**: Check your environment variables are set correctly
+6. **"Auto-indexing configuration error"**: Verify the environment variable format uses `AUTO_INDEXING_SOURCES_N_URI`
 
 ### Checking Index Status
 

kodit-0.3.0/docs/reference/mcp/index.md

@@ -0,0 +1,217 @@
+---
+title: MCP
+description: Model Context Protocol (MCP) server implementation for AI coding assistants
+weight: 2
+---
+
+Kodit provides an MCP (Model Context Protocol) server that enables AI coding assistants to search and retrieve relevant code snippets from your indexed codebases. This allows AI assistants to provide more accurate and contextually relevant code suggestions.
+
+## What is MCP?
+
+The Model Context Protocol (MCP) is a standard that enables AI assistants to communicate with external tools and data sources. Kodit implements an MCP server that exposes your indexed codebases to AI assistants, allowing them to:
+
+- Search for relevant code examples
+- Retrieve specific code snippets
+- Filter results by various criteria
+- Provide context-aware code suggestions
+
+## How Kodit MCP Works
+
+The Kodit MCP server runs as a separate service that:
+
+1. **Connects to your indexed codebases** - Uses the same database and indexes created by the `kodit index` command
+2. **Exposes search functionality** - Provides a `search` tool that AI assistants can call
+3. **Handles filtering** - Supports filtering by language, author, date range, and source repository
+4. **Returns relevant snippets** - Combines keyword, semantic code, and semantic text search for optimal results
+
+## Integration with AI Assistants
+
+To use Kodit with your AI coding assistant, you need to:
+
+1. **Start the Kodit MCP server**:
+
+   ```sh
+   kodit serve
+   ```
+
+2. **Configure your AI assistant** to connect to the MCP server. See the [Integration Guide](../../getting-started/integration/index.md) for detailed instructions for:
+   - Cursor
+   - Cline
+   - Other MCP-compatible assistants
+
+## Search Tool
+
+The primary tool exposed by the Kodit MCP server is the `search` function, which provides comprehensive code search capabilities.
+
+### Search Parameters
+
+The search tool accepts the following parameters:
+
+| Parameter | Type | Description | Example |
+|-----------|------|-------------|---------|
+| `user_intent` | string | Description of what the user wants to achieve | "Create a REST API endpoint for user authentication" |
+| `related_file_paths` | list[Path] | Absolute paths to relevant files | `["/path/to/auth.py"]` |
+| `related_file_contents` | list[string] | Contents of relevant files | `["def authenticate(): ..."]` |
+| `keywords` | list[string] | Relevant keywords for the search | `["authentication", "jwt", "login"]` |
+| `language` | string \| None | Filter by programming language | `"python"`, `"go"`, `"javascript"` |
+| `author` | string \| None | Filter by author name | `"john.doe"` |
+| `created_after` | string \| None | Filter by creation date (YYYY-MM-DD) | `"2023-01-01"` |
+| `created_before` | string \| None | Filter by creation date (YYYY-MM-DD) | `"2023-12-31"` |
+| `source_repo` | string \| None | Filter by source repository | `"github.com/example/repo"` |
+
+### Search Functionality
+
+The search tool combines multiple search strategies:
+
+1. **Keyword Search** - Uses BM25 algorithm for exact keyword matching
+2. **Semantic Code Search** - Uses embeddings to find semantically similar code
+3. **Semantic Text Search** - Uses embeddings to find code matching natural language descriptions
+
+Results are fused together to provide the most relevant snippets for the user's intent.
+
+## Filtering Capabilities
+
+Kodit's MCP server supports comprehensive filtering to help AI assistants find the most relevant code examples. These filters work the same way as the CLI search filters.
+
+### Language Filtering
+
+Filter results by programming language:
+
+**Example prompts:**
+> "I need to create a web server in Python. Please search for Flask or FastAPI examples and show me the best practices."
+> "I'm working on a Go microservice. Can you search for Go-specific patterns for handling HTTP requests and database connections?"
+> "I need JavaScript examples for form validation. Please search for modern JavaScript/TypeScript validation patterns."
+
+### Author Filtering
+
+Filter results by code author:
+
+**Example prompts:**
+> "I'm reviewing code written by john.doe. Can you search for their authentication implementations to understand their coding style?"
+> "I need to find all the database-related code written by alice.smith. Please search for her database connection and query patterns."
+
+### Date Range Filtering
+
+Filter results by creation date:
+
+**Example prompts:**
+> "I need to see authentication patterns from 2023. Please search for JWT and OAuth implementations created in 2023."
+> "Show me modern React patterns from the last year. Search for React components and hooks created after 2023."
+
+### Source Repository Filtering
+
+Filter results by source repository:
+
+**Example prompts:**
+> "I'm working on the auth-service project. Please search for authentication patterns specifically from github.com/company/auth-service."
+> "I need to understand how the user-service handles user management. Search for user-related code from github.com/company/user-service."
+
+### Combining Filters
+
+You can combine multiple filters for precise results:
+
+**Example prompts:**
+> "I need Python authentication code written by alice.smith in 2023 from the auth-service repository. Please search for JWT token validation patterns."
+> "Show me Go microservice patterns from john.doe created in 2023 from the backend-services repository."
+> "I'm looking for modern React patterns from the frontend team (search for authors: alice.smith, bob.jones) created in 2024 from the web-app repository."
+
+## AI Assistant Integration Tips
+
+To get the best results from Kodit with your AI assistant, follow these prompting strategies:
+
+### 1. Provide Clear User Intent
+
+When the AI assistant calls the search tool, ensure it provides a clear, descriptive `user_intent`:
+
+**Good examples:**
+
+- "Create a REST API endpoint for user authentication with JWT tokens"
+- "Implement a database connection pool for PostgreSQL"
+- "Write a function to validate email addresses using regex"
+
+**Poor examples:**
+
+- "Help me with auth"
+- "Database stuff"
+- "Email validation"
+
+### 2. Use Relevant Keywords
+
+Provide specific, technical keywords that are relevant to your task, where applicable.
+Remember that the language model is more than capable of generating appropriate keywords
+for your intent.
+
+**Good examples:**
+
+- `["authentication", "jwt", "login", "password"]`
+- `["database", "postgresql", "connection", "pool"]`
+- `["email", "validation", "regex", "format"]`
+
+### 3. Leverage File Context
+
+If you're working with existing files, mention them in your prompt:
+
+**Example prompts:**
+> "I'm working on the authentication function in auth.py. Can you search for similar error handling patterns and show me how to improve the error handling in my existing code?"
+> "I have a database connection setup in database.py. Please search for connection pooling patterns and show me how to optimize my current implementation."
+
+### 4. Use Language Filtering
+
+Specify the programming language in your prompt:
+
+**Example prompts:**
+> "I need to create a web server in Python. Please search for Flask and FastAPI examples and show me the best practices."
+> "I'm building a Go microservice. Can you search for Go-specific patterns for handling HTTP requests and database connections?"
+> "I need JavaScript examples for form validation. Please search for modern JavaScript/TypeScript validation patterns."
+
+### 5. Filter by Source Repository
+
+If you have multiple codebases indexed, mention the specific repository:
+
+**Example prompts:**
+> "I'm working on the user-service project. Please search for user management patterns specifically from github.com/company/user-service."
+> "I need to understand how the auth-service handles authentication. Search for auth-related code from github.com/company/auth-service."
+
+### 6. Example Prompts for AI Assistants
+
+Here are some example prompts you can use with your AI assistant:
+
+**For new code development:**
+> "I want to create a new Python web API. Please search for examples of Flask/FastAPI authentication patterns and show me the best practices."
+
+**For debugging existing code:**
+> "I'm having issues with this database connection code. Can you search for similar patterns and show me how others handle connection errors?"
+
+**For learning new patterns:**
+> "I need to implement JWT authentication in Go. Please search for production-ready examples and show me the security best practices."
+
+**For code review:**
+> "I'm reviewing this authentication function. Can you search for similar implementations and show me potential security issues or improvements?"
+
+## Troubleshooting
+
+### Common Issues
+
+1. **AI assistant not using Kodit**: Ensure you've configured the enforcement prompt and MCP server connection properly.
+
+2. **No search results**: Check that you have indexed codebases and that your search terms are relevant.
+
+3. **Filter not working**: Verify that the filter values match your indexed data (e.g., correct language names, author names, repository URLs).
+
+4. **Connection issues**: Ensure the Kodit MCP server is running (`kodit serve`) and accessible to your AI assistant.
+
+### Debugging
+
+Enable debug logging to see what's happening:
+
+```sh
+export LOG_LEVEL=DEBUG
+kodit serve
+```
+
+This will show you:
+
+- Search queries being executed
+- Filter parameters being applied
+- Results being returned
+- Any errors or issues

{kodit-0.2.8 → kodit-0.3.0}/pyproject.toml

@@ -121,7 +121,6 @@ ignore = [
   "EM102", # As above
 ]
 select = ["ALL"]
-exclude = ["./tests/*"]
 
 [[tool.uv.index]]
 name = "pypi"

{kodit-0.2.8 → kodit-0.3.0}/src/kodit/_version.py

@@ -17,5 +17,5 @@ __version__: str
 __version_tuple__: VERSION_TUPLE
 version_tuple: VERSION_TUPLE
 
-__version__ = version = '0.2.8'
-__version_tuple__ = version_tuple = (0, 2, 8)
+__version__ = version = '0.3.0'
+__version_tuple__ = version_tuple = (0, 3, 0)

kodit-0.3.0/src/kodit/app.py

@@ -0,0 +1,70 @@
+"""FastAPI application for kodit API."""
+
+from collections.abc import AsyncIterator
+from contextlib import asynccontextmanager
+
+from asgi_correlation_id import CorrelationIdMiddleware
+from fastapi import FastAPI
+
+from kodit.config import AppContext
+from kodit.infrastructure.indexing.auto_indexing_service import AutoIndexingService
+from kodit.mcp import mcp
+from kodit.middleware import ASGICancelledErrorMiddleware, logging_middleware
+
+# Global auto-indexing service
+_auto_indexing_service: AutoIndexingService | None = None
+
+
+@asynccontextmanager
+async def app_lifespan(_: FastAPI) -> AsyncIterator[None]:
+    """Manage application lifespan for auto-indexing."""
+    global _auto_indexing_service  # noqa: PLW0603
+    # Start auto-indexing service
+    app_context = AppContext()
+    db = await app_context.get_db()
+    _auto_indexing_service = AutoIndexingService(
+        app_context=app_context,
+        session_factory=db.session_factory,
+    )
+    await _auto_indexing_service.start_background_indexing()
+    yield
+    if _auto_indexing_service:
+        await _auto_indexing_service.stop()
+
+
+# See https://gofastmcp.com/deployment/asgi#fastapi-integration
+mcp_app = mcp.sse_app()
+
+
+@asynccontextmanager
+async def combined_lifespan(app: FastAPI) -> AsyncIterator[None]:
+    """Combine app and MCP lifespans."""
+    async with app_lifespan(app), mcp_app.router.lifespan_context(app):
+        yield
+
+
+app = FastAPI(title="kodit API", lifespan=combined_lifespan)
+
+# Add middleware
+app.middleware("http")(logging_middleware)
+app.add_middleware(CorrelationIdMiddleware)
+
+
+@app.get("/")
+async def root() -> dict[str, str]:
+    """Return a welcome message for the kodit API."""
+    return {"message": "Hello, World!"}
+
+
+@app.get("/healthz")
+async def healthz() -> dict[str, str]:
+    """Return a health check for the kodit API."""
+    return {"status": "ok"}
+
+
+# Add mcp routes last, otherwise previous routes aren't added
+app.mount("", mcp_app)
+
+# Wrap the entire app with ASGI middleware after all routes are added to suppress
+# CancelledError at the ASGI level
+app = ASGICancelledErrorMiddleware(app)

kodit-0.3.0/src/kodit/application/factories/__init__.py

@@ -0,0 +1 @@
+"""Application factories package."""

kodit-0.3.0/src/kodit/application/factories/code_indexing_factory.py

@@ -0,0 +1,119 @@
+"""Factory for creating the unified code indexing application service."""
+
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from kodit.application.services.code_indexing_application_service import (
+    CodeIndexingApplicationService,
+)
+from kodit.config import AppContext
+from kodit.domain.entities import EmbeddingType
+from kodit.domain.services.bm25_service import BM25DomainService
+from kodit.domain.services.embedding_service import EmbeddingDomainService
+from kodit.domain.services.enrichment_service import EnrichmentDomainService
+from kodit.domain.services.source_service import SourceService
+from kodit.infrastructure.bm25.bm25_factory import bm25_repository_factory
+from kodit.infrastructure.embedding.embedding_factory import (
+    embedding_domain_service_factory,
+)
+from kodit.infrastructure.embedding.embedding_providers import (
+    hash_embedding_provider,
+)
+from kodit.infrastructure.embedding.local_vector_search_repository import (
+    LocalVectorSearchRepository,
+)
+from kodit.infrastructure.enrichment.enrichment_factory import (
+    enrichment_domain_service_factory,
+)
+from kodit.infrastructure.enrichment.null_enrichment_provider import (
+    NullEnrichmentProvider,
+)
+from kodit.infrastructure.indexing.indexing_factory import (
+    indexing_domain_service_factory,
+)
+from kodit.infrastructure.indexing.snippet_domain_service_factory import (
+    snippet_domain_service_factory,
+)
+from kodit.infrastructure.sqlalchemy.embedding_repository import (
+    SqlAlchemyEmbeddingRepository,
+)
+
+
+def create_code_indexing_application_service(
+    app_context: AppContext,
+    session: AsyncSession,
+    source_service: SourceService,
+) -> CodeIndexingApplicationService:
+    """Create a unified code indexing application service with all dependencies."""
+    # Create domain services
+    indexing_domain_service = indexing_domain_service_factory(session)
+    snippet_domain_service = snippet_domain_service_factory(session)
+    bm25_service = BM25DomainService(bm25_repository_factory(app_context, session))
+    code_search_service = embedding_domain_service_factory("code", app_context, session)
+    text_search_service = embedding_domain_service_factory("text", app_context, session)
+    enrichment_service = enrichment_domain_service_factory(app_context)
+
+    # Create and return the unified application service
+    return CodeIndexingApplicationService(
+        indexing_domain_service=indexing_domain_service,
+        snippet_domain_service=snippet_domain_service,
+        source_service=source_service,
+        bm25_service=bm25_service,
+        code_search_service=code_search_service,
+        text_search_service=text_search_service,
+        enrichment_service=enrichment_service,
+        session=session,
+    )
+
+
+def create_fast_test_code_indexing_application_service(
+    app_context: AppContext,
+    session: AsyncSession,
+    source_service: SourceService,
+) -> CodeIndexingApplicationService:
+    """Create a fast test version of CodeIndexingApplicationService."""
+    # Create domain services
+    indexing_domain_service = indexing_domain_service_factory(session)
+    snippet_domain_service = snippet_domain_service_factory(session)
+    bm25_service = BM25DomainService(bm25_repository_factory(app_context, session))
+
+    # Create fast embedding services using HashEmbeddingProvider
+    embedding_repository = SqlAlchemyEmbeddingRepository(session=session)
+
+    # Fast code search service
+    code_search_repository = LocalVectorSearchRepository(
+        embedding_repository=embedding_repository,
+        embedding_provider=hash_embedding_provider.HashEmbeddingProvider(),
+        embedding_type=EmbeddingType.CODE,
+    )
+    code_search_service = EmbeddingDomainService(
+        embedding_provider=hash_embedding_provider.HashEmbeddingProvider(),
+        vector_search_repository=code_search_repository,
+    )
+
+    # Fast text search service
+    text_search_repository = LocalVectorSearchRepository(
+        embedding_repository=embedding_repository,
+        embedding_provider=hash_embedding_provider.HashEmbeddingProvider(),
+        embedding_type=EmbeddingType.TEXT,
+    )
+    text_search_service = EmbeddingDomainService(
+        embedding_provider=hash_embedding_provider.HashEmbeddingProvider(),
+        vector_search_repository=text_search_repository,
+    )
+
+    # Fast enrichment service using NullEnrichmentProvider
+    enrichment_service = EnrichmentDomainService(
+        enrichment_provider=NullEnrichmentProvider()
+    )
+
+    # Create and return the unified application service
+    return CodeIndexingApplicationService(
+        indexing_domain_service=indexing_domain_service,
+        snippet_domain_service=snippet_domain_service,
+        source_service=source_service,
+        bm25_service=bm25_service,
+        code_search_service=code_search_service,
+        text_search_service=text_search_service,
+        enrichment_service=enrichment_service,
+        session=session,
+    )