kodit 0.2.9__tar.gz → 0.3.1__tar.gz

kodit-0.3.1/.claude/commands/debug.md

@@ -0,0 +1,121 @@
+# Debug Error Command
+
+Systematically debug and resolve errors in any codebase using a structured approach.
+
+## Instructions
+
+Follow this comprehensive debugging methodology to resolve: **$ARGUMENTS**
+
+1. **Error Information Gathering**
+   - Collect the complete error message, stack trace, and error code
+   - Note when the error occurs (timing, conditions, frequency)
+   - Identify the environment where the error happens (dev, staging, prod)
+   - Gather relevant logs from before and after the error
+
+2. **Reproduce the Error**
+   - Create a minimal test case that reproduces the error consistently
+   - Document the exact steps needed to trigger the error
+   - Test in different environments if possible
+   - Note any patterns or conditions that affect error occurrence
+
+3. **Stack Trace Analysis**
+   - Read the stack trace from bottom to top to understand the call chain
+   - Identify the exact line where the error originates
+   - Trace the execution path leading to the error
+   - Look for any obvious issues in the failing code
+
+4. **Code Context Investigation**
+   - Examine the code around the error location
+   - Check recent changes that might have introduced the bug
+   - Review variable values and state at the time of error
+   - Analyze function parameters and return values
+
+5. **Hypothesis Formation**
+   - Based on evidence, form hypotheses about the root cause
+   - Consider common causes:
+     - Null pointer/undefined reference
+     - Type mismatches
+     - Race conditions
+     - Resource exhaustion
+     - Logic errors
+     - External dependency failures
+
+6. **Debugging Tools Setup**
+   - Set up appropriate debugging tools for the technology stack
+   - Use debugger, profiler, or logging as needed
+   - Configure breakpoints at strategic locations
+   - Set up monitoring and alerting if not already present
+
+7. **Systematic Investigation**
+   - Test each hypothesis methodically
+   - Use binary search approach to isolate the problem
+   - Add strategic logging or print statements
+   - Check data flow and transformations step by step
+
+8. **Data Validation**
+   - Verify input data format and validity
+   - Check for edge cases and boundary conditions
+   - Validate assumptions about data state
+   - Test with different data sets to isolate patterns
+
+9. **Dependency Analysis**
+   - Check external dependencies and their versions
+   - Verify network connectivity and API availability
+   - Review configuration files and environment variables
+   - Test database connections and query execution
+
+10. **Memory and Resource Analysis**
+    - Check for memory leaks or excessive memory usage
+    - Monitor CPU and I/O resource consumption
+    - Analyze garbage collection patterns if applicable
+    - Check for resource deadlocks or contention
+
+11. **Concurrency Issues Investigation**
+    - Look for race conditions in multi-threaded code
+    - Check synchronization mechanisms and locks
+    - Analyze async operations and promise handling
+    - Test under different load conditions
+
+12. **Root Cause Identification**
+    - Once the cause is identified, understand why it happened
+    - Determine if it's a logic error, design flaw, or external issue
+    - Assess the scope and impact of the problem
+    - Consider if similar issues exist elsewhere
+
+13. **Solution Implementation**
+    - Design a fix that addresses the root cause
+    - Consider multiple solution approaches and trade-offs
+    - Implement the fix with appropriate error handling
+    - Add validation and defensive programming where needed
+
+14. **Testing the Fix**
+    - Test the fix against the original error case
+    - Test edge cases and related scenarios
+    - Run regression tests to ensure no new issues
+    - Test under various load and stress conditions
+
+15. **Prevention Measures**
+    - Add appropriate unit and integration tests
+    - Improve error handling and logging
+    - Add input validation and defensive checks
+    - Update documentation and code comments
+
+16. **Monitoring and Alerting**
+    - Set up monitoring for similar issues
+    - Add metrics and health checks
+    - Configure alerts for error thresholds
+    - Implement better observability
+
+17. **Documentation**
+    - Document the error, investigation process, and solution
+    - Update troubleshooting guides
+    - Share learnings with the team
+    - Update code comments with context
+
+18. **Post-Resolution Review**
+    - Analyze why the error wasn't caught earlier
+    - Review development and testing processes
+    - Consider improvements to prevent similar issues
+    - Update coding standards or guidelines if needed
+
+Remember to maintain detailed notes throughout the debugging process and consider the wider implications of both the error and the fix.

kodit-0.3.1/.claude/commands/refactor.md

@@ -0,0 +1,33 @@
+# Refactor Code Command
+
+Safely improve code structure while preserving functionality.
+
+## Core Principles
+
+- **Test First**: Ensure tests exist and pass before starting
+- **Small Steps**: Make incremental changes, test after each
+- **Preserve Behavior**: External functionality must remain identical
+
+## Refactoring Process
+
+1. **Analyze** - Understand current code and identify improvement areas
+2. **Test Coverage** - Verify/add tests before changing anything
+3. **Plan** - Choose refactoring technique (extract method, rename, simplify conditionals)
+4. **Execute** - Make one small change at a time, running tests after each
+5. **Verify** - Ensure all tests pass and performance hasn't degraded
+
+## Key Improvements
+
+- **Clarity**: Better names, simpler logic, shorter methods
+- **Structure**: Remove duplication, improve separation of concerns, improve cohesion
+- **Maintainability**: Apply patterns where beneficial, standardize error handling
+- **Documentation**: Update comments and docs to match changes
+
+## Safety Checklist
+
+- [ ] All tests passing before and after
+- [ ] No functionality changes
+- [ ] Code reviewed for quality
+- [ ] Changes documented
+
+Remember: Working code > perfect code. Commit frequently, refactor incrementally.

kodit-0.3.1/.claude/settings.json

@@ -0,0 +1,3 @@
+{
+    "includeCoAuthoredBy": false
+}

kodit-0.3.1/CLAUDE.md

@@ -0,0 +1,99 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Project Overview
+
+Kodit is a code indexing MCP (Model Context Protocol) server that connects AI coding assistants to external codebases. It provides semantic and keyword search capabilities over indexed code snippets to improve AI-assisted development.
+
+## Development Commands
+
+### Testing
+
+- `uv run pytest src/kodit` - Run all unit tests with coverage
+- `uv run pytest tests/path/to/test.py` - Run specific test file
+- `uv run pytest -k "test_name"` - Run specific test by name
+
+### Code Quality
+
+- `uv run ruff check --fix` - Run linting
+- `uv run ruff format` - Format code
+- `uv run mypy src/` - Type checking
+
+### Application
+
+- `uv run kodit --help` - Show CLI help
+- `uv run kodit index <path>` - Index a codebase
+- `uv run kodit serve` - Start MCP server
+
+### Database
+
+- `uv run alembic upgrade head` - Apply database migrations
+- `uv run alembic revision --autogenerate -m "description"` - Generate new migration
+
+## Architecture
+
+### Domain-Driven Design Structure
+
+The codebase follows Domain-Driven Design (DDD) with clean architecture:
+
+- `domain/` - Core business logic and interfaces
+  - `entities.py` - Domain entities (Snippet, File, etc.)
+  - `repositories.py` - Repository interfaces
+  - `services/` - Domain services for business logic
+- `application/` - Application services and factories
+- `infrastructure/` - External concerns and implementations
+  - `sqlalchemy/` - Database repositories
+  - `embedding/` - Vector embedding providers
+  - `bm25/` - BM25 search implementations
+  - `indexing/` - Code indexing services
+
+### Key Components
+
+**Indexing Pipeline:**
+
+1. Clone/read source code
+2. Extract snippets using Tree-sitter
+3. Generate embeddings and BM25 indices
+4. Store in database
+
+**Search System:**
+
+- Hybrid search combining semantic (embeddings) and keyword (BM25)
+- Multiple providers: local models, OpenAI, custom APIs
+- Configurable via environment variables
+
+**MCP Server:**
+
+- FastMCP-based server exposing search tools
+- Integrates with Cursor, Cline, and other AI assistants
+
+## Configuration
+
+Key environment variables:
+
+- `KODIT_EMBEDDING_PROVIDER` - embedding provider (local/openai)
+- `KODIT_DATABASE_URL` - database connection string
+- `KODIT_OPENAI_API_KEY` - for OpenAI embeddings
+- `KODIT_LOG_LEVEL` - logging level
+
+See `config.py` for full configuration options.
+
+## Database
+
+Uses SQLAlchemy with async support. Supports:
+
+- SQLite (default, local development)
+- PostgreSQL with Vectorchord (production)
+
+Migrations managed with Alembic in `migrations/` directory. DO NOT EDIT THESE FILES.
+
+## Testing Strategy
+
+- Unit tests for domain services and repositories
+- Integration tests for database operations
+- E2E tests for full indexing pipeline
+- Smoke tests for deployment validation
+- Performance tests for similarity search
+
+Test files mirror source structure under `tests/` directory.

{kodit-0.2.9 → kodit-0.3.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: kodit
-Version: 0.2.9
+Version: 0.3.1
 Summary: Code indexing for better AI code generation
 Project-URL: Homepage, https://docs.helixml.tech/kodit/
 Project-URL: Documentation, https://docs.helixml.tech/kodit/
@@ -83,7 +83,7 @@ If you're an engineer working with AI-powered coding assistants, Kodit helps by
 providing relevant and up-to-date examples of your task so that LLMs make less mistakes
 and produce fewer hallucinations.
 
-## ✨ Features
+## Features
 
 ### Codebase Indexing
 
@@ -94,7 +94,11 @@ code. This index is used to build a snippet library, ready for ingestion into an
 - Build comprehensive snippet libraries for LLM ingestion
 - Support for multiple codebase types and languages
 - Efficient indexing and search capabilities
-- Privacy first: respects .gitignore and .noindex files.
+- Privacy first: respects .gitignore and .noindex files
+- **NEW in 0.3**: Auto-indexing configuration for shared server deployments
+- **NEW in 0.3**: Enhanced Git provider support including Azure DevOps
+- **NEW in 0.3**: Index private repositories via a PAT
+- **NEW in 0.3**: Improved progress monitoring and reporting during indexing
 
 ### MCP Server
 
@@ -107,6 +111,7 @@ intent. Kodit has been tested to work well with:
   - [Cursor](https://docs.helix.ml/kodit/getting-started/integration/#integration-with-cursor)
   - [Cline](https://docs.helix.ml/kodit/getting-started/integration/#integration-with-cline)
 - Please contribute more instructions! ... any other assistant is likely to work ...
+- **New in 0.3**: Filter snippets by source, language, author or timestamp.
 
 ### Enterprise Ready
 
@@ -127,7 +132,12 @@ Supported providers:
 - Secure, private LLM enclave with [Helix](https://helix.ml).
 - Any other OpenAI compatible API
 
-## 🚀 Quick Start
+**NEW in 0.3**: Enhanced deployment options:
+
+- Docker Compose configurations with VectorChord
+- Kubernetes manifests for production deployments
+
+## Quick Start
 
 1. [Install Kodit](https://docs.helix.ml/kodit/getting-started/installation/)
 2. [Index codebases](https://docs.helix.ml/kodit/getting-started/quick-start/)

{kodit-0.2.9 → kodit-0.3.1}/README.md

@@ -30,7 +30,7 @@ If you're an engineer working with AI-powered coding assistants, Kodit helps by
 providing relevant and up-to-date examples of your task so that LLMs make less mistakes
 and produce fewer hallucinations.
 
-## ✨ Features
+## Features
 
 ### Codebase Indexing
 
@@ -41,7 +41,11 @@ code. This index is used to build a snippet library, ready for ingestion into an
 - Build comprehensive snippet libraries for LLM ingestion
 - Support for multiple codebase types and languages
 - Efficient indexing and search capabilities
-- Privacy first: respects .gitignore and .noindex files.
+- Privacy first: respects .gitignore and .noindex files
+- **NEW in 0.3**: Auto-indexing configuration for shared server deployments
+- **NEW in 0.3**: Enhanced Git provider support including Azure DevOps
+- **NEW in 0.3**: Index private repositories via a PAT
+- **NEW in 0.3**: Improved progress monitoring and reporting during indexing
 
 ### MCP Server
 
@@ -54,6 +58,7 @@ intent. Kodit has been tested to work well with:
   - [Cursor](https://docs.helix.ml/kodit/getting-started/integration/#integration-with-cursor)
   - [Cline](https://docs.helix.ml/kodit/getting-started/integration/#integration-with-cline)
 - Please contribute more instructions! ... any other assistant is likely to work ...
+- **New in 0.3**: Filter snippets by source, language, author or timestamp.
 
 ### Enterprise Ready
 
@@ -74,7 +79,12 @@ Supported providers:
 - Secure, private LLM enclave with [Helix](https://helix.ml).
 - Any other OpenAI compatible API
 
-## 🚀 Quick Start
+**NEW in 0.3**: Enhanced deployment options:
+
+- Docker Compose configurations with VectorChord
+- Kubernetes manifests for production deployments
+
+## Quick Start
 
 1. [Install Kodit](https://docs.helix.ml/kodit/getting-started/installation/)
 2. [Index codebases](https://docs.helix.ml/kodit/getting-started/quick-start/)

{kodit-0.2.9 → kodit-0.3.1}/docs/_index.md

@@ -40,7 +40,7 @@ If you're an engineer working with AI-powered coding assistants, Kodit helps by
 providing relevant and up-to-date examples of your task so that LLMs make less mistakes
 and produce fewer hallucinations.
 
-## ✨ Features
+## Features
 
 ### Codebase Indexing
 
@@ -51,7 +51,11 @@ code. This index is used to build a snippet library, ready for ingestion into an
 - Build comprehensive snippet libraries for LLM ingestion
 - Support for multiple codebase types and languages
 - Efficient indexing and search capabilities
-- Privacy first: respects .gitignore and .noindex files.
+- Privacy first: respects .gitignore and .noindex files
+- **NEW in 0.3**: Auto-indexing configuration for shared server deployments
+- **NEW in 0.3**: Enhanced Git provider support including Azure DevOps
+- **NEW in 0.3**: Index private repositories via a PAT
+- **NEW in 0.3**: Improved progress monitoring and reporting during indexing
 
 ### MCP Server
 
@@ -64,6 +68,7 @@ intent. Kodit has been tested to work well with:
   - [Cursor](./getting-started/integration/index.md#integration-with-cursor)
   - [Cline](./getting-started/integration/index.md#integration-with-cline)
 - Please contribute more instructions! ... any other assistant is likely to work ...
+- **New in 0.3**: Filter snippets by source, language, author or timestamp.
 
 ### Enterprise Ready
 
@@ -84,6 +89,11 @@ Supported providers:
 - Secure, private LLM enclave with [Helix](https://helix.ml).
 - Any other OpenAI compatible API
 
+**NEW in 0.3**: Enhanced deployment options:
+
+- Docker Compose configurations with VectorChord
+- Kubernetes manifests for production deployments
+
 ## Roadmap
 
 The roadmap is currently maintained as a [Github Project](https://github.com/orgs/helixml/projects/4).

{kodit-0.2.9 → kodit-0.3.1}/docs/reference/deployment/docker-compose.yaml

@@ -29,6 +29,10 @@ services:
       ENRICHMENT_ENDPOINT_API_KEY: REPLACE_WITH_YOUR_API_KEY
       ENRICHMENT_ENDPOINT_MODEL: o3-mini
 
+      # Auto-indexing configuration
+      AUTO_INDEXING_SOURCES_0_URI: https://github.com/pydantic/pydantic
+      AUTO_INDEXING_SOURCES_1_URI: https://github.com/fastapi/fastapi
+      AUTO_INDEXING_SOURCES_2_URI: https://github.com/helix-ml/kodit
 
   vectorchord:
     image: tensorchord/vchord-suite:pg17-20250601

{kodit-0.2.9 → kodit-0.3.1}/docs/reference/deployment/kubernetes.yaml

@@ -77,6 +77,10 @@ spec:
               value: "REPLACE_WITH_YOUR_API_KEY"
             - name: ENRICHMENT_ENDPOINT_MODEL
               value: "o3-mini"
+            - name: AUTO_INDEXING_SOURCES_0_URI
+              value: "https://github.com/pydantic/pydantic"
+            - name: AUTO_INDEXING_SOURCES_1_URI
+              value: "https://github.com/helix-ml/kodit"
           ports:
             - containerPort: 8080
           readinessProbe:

{kodit-0.2.9 → kodit-0.3.1}/docs/reference/indexing/index.md

@@ -42,7 +42,7 @@ Kodit can index local directories on your filesystem:
 
 ## Basic Usage
 
-### Indexing a Source
+### Manual Indexing
 
 To index a source, use the `kodit index` command followed by the source location:
 
@@ -75,6 +75,61 @@ This will display a table showing:
 - Source URI
 - Number of snippets extracted
 
+### Auto-Indexing
+
+If you're running Kodit as a shared server, you need to configure what gets indexed.
+Auto-indexing is a simple indexing configuration powered by environmental variables.
+
+#### Configuration via Environment Variables
+
+Configure auto-indexing sources using environment variables with the `AUTO_INDEXING_SOURCES_{X}_` prefix:
+
+```sh
+# Configure a single auto-index source
+export AUTO_INDEXING_SOURCES_0_URI="https://github.com/pydantic/pydantic"
+
+# Configure multiple auto-index sources
+export AUTO_INDEXING_SOURCES_0_URI="https://github.com/pydantic/pydantic"
+export AUTO_INDEXING_SOURCES_1_URI="https://github.com/fastapi/fastapi"
+export AUTO_INDEXING_SOURCES_2_URI="/path/to/local/project"
+
+# Or use a .env file
+echo "AUTO_INDEXING_SOURCES_0_URI=https://github.com/pydantic/pydantic" >> .env
+echo "AUTO_INDEXING_SOURCES_1_URI=https://github.com/fastapi/fastapi" >> .env
+echo "AUTO_INDEXING_SOURCES_2_URI=/path/to/local/project" >> .env
+```
+
+**Configuration Format:**
+
+- Use `AUTO_INDEXING_SOURCES_N_URI` where `N` is a zero-based index
+- Sources are indexed in numerical order (0, 1, 2, etc.)
+- Supports all source types: Git repositories (HTTPS/SSH) and local directories
+- Gaps in numbering are allowed (e.g., 0, 2, 5 will work)
+
+#### Using Auto-Indexing
+
+To manually index all configured auto-index sources:
+
+```sh
+kodit index --auto-index
+```
+
+This command will:
+
+1. Read the auto-indexing configuration from environment variables
+2. Index each configured source in sequence
+3. Show progress for each source being indexed
+4. Handle errors gracefully and continue with remaining sources
+
+If no auto-index sources are configured, the command will display a message indicating
+that no sources are configured.
+
+To automatically run index all configured auto-index sources:
+
+```sh
+kodit serve
+```
+
 ## Git Protocol Support
 
 ### HTTPS Authentication
@@ -226,15 +281,6 @@ Kodit shows progress during indexing operations:
 - Snippet extraction progress
 - Index building progress (BM25, embeddings)
 
-### Error Handling
-
-Common issues and solutions:
-
-1. **Authentication Errors**: Ensure your credentials are correct
-2. **Network Issues**: Check your internet connection and firewall settings
-3. **Permission Errors**: Ensure you have read access to the source
-4. **Unsupported Files**: Kodit will skip unsupported file types automatically
-
 ## Privacy and Security
 
 ### Local Processing
@@ -265,6 +311,8 @@ Kodit respects privacy by honoring:
 2. **"Unsupported source"**: Ensure the path or URL is valid and accessible
 3. **"No snippets found"**: Check if the source contains supported file types
 4. **"Permission denied"**: Ensure you have read access to the source
+5. **"No auto-index sources configured"**: Check your environment variables are set correctly
+6. **"Auto-indexing configuration error"**: Verify the environment variable format uses `AUTO_INDEXING_SOURCES_N_URI`
 
 ### Checking Index Status
 

{kodit-0.2.9 → kodit-0.3.1}/src/kodit/_version.py

@@ -17,5 +17,5 @@ __version__: str
 __version_tuple__: VERSION_TUPLE
 version_tuple: VERSION_TUPLE
 
-__version__ = version = '0.2.9'
-__version_tuple__ = version_tuple = (0, 2, 9)
+__version__ = version = '0.3.1'
+__version_tuple__ = version_tuple = (0, 3, 1)

kodit-0.3.1/src/kodit/app.py

@@ -0,0 +1,70 @@
+"""FastAPI application for kodit API."""
+
+from collections.abc import AsyncIterator
+from contextlib import asynccontextmanager
+
+from asgi_correlation_id import CorrelationIdMiddleware
+from fastapi import FastAPI
+
+from kodit.config import AppContext
+from kodit.infrastructure.indexing.auto_indexing_service import AutoIndexingService
+from kodit.mcp import mcp
+from kodit.middleware import ASGICancelledErrorMiddleware, logging_middleware
+
+# Global auto-indexing service
+_auto_indexing_service: AutoIndexingService | None = None
+
+
+@asynccontextmanager
+async def app_lifespan(_: FastAPI) -> AsyncIterator[None]:
+    """Manage application lifespan for auto-indexing."""
+    global _auto_indexing_service  # noqa: PLW0603
+    # Start auto-indexing service
+    app_context = AppContext()
+    db = await app_context.get_db()
+    _auto_indexing_service = AutoIndexingService(
+        app_context=app_context,
+        session_factory=db.session_factory,
+    )
+    await _auto_indexing_service.start_background_indexing()
+    yield
+    if _auto_indexing_service:
+        await _auto_indexing_service.stop()
+
+
+# See https://gofastmcp.com/deployment/asgi#fastapi-integration
+mcp_app = mcp.sse_app()
+
+
+@asynccontextmanager
+async def combined_lifespan(app: FastAPI) -> AsyncIterator[None]:
+    """Combine app and MCP lifespans."""
+    async with app_lifespan(app), mcp_app.router.lifespan_context(app):
+        yield
+
+
+app = FastAPI(title="kodit API", lifespan=combined_lifespan)
+
+# Add middleware
+app.middleware("http")(logging_middleware)
+app.add_middleware(CorrelationIdMiddleware)
+
+
+@app.get("/")
+async def root() -> dict[str, str]:
+    """Return a welcome message for the kodit API."""
+    return {"message": "Hello, World!"}
+
+
+@app.get("/healthz")
+async def healthz() -> dict[str, str]:
+    """Return a health check for the kodit API."""
+    return {"status": "ok"}
+
+
+# Add mcp routes last, otherwise previous routes aren't added
+app.mount("", mcp_app)
+
+# Wrap the entire app with ASGI middleware after all routes are added to suppress
+# CancelledError at the ASGI level
+app = ASGICancelledErrorMiddleware(app)

{kodit-0.2.9 → kodit-0.3.1}/src/kodit/application/services/code_indexing_application_service.py

@@ -27,7 +27,6 @@ from kodit.domain.value_objects import (
     MultiSearchResult,
     SearchRequest,
     SearchResult,
-    SnippetListItem,
 )
 from kodit.log import log_event
 from kodit.reporting import Reporter
@@ -225,17 +224,17 @@ class CodeIndexingApplicationService:
 
         return [
             MultiSearchResult(
-                id=snippet["id"],
-                uri=file["uri"],
-                content=snippet["content"],
+                id=result.snippet.id,
+                uri=result.file.uri,
+                content=result.snippet.content,
                 original_scores=fr.original_scores,
             )
-            for (file, snippet), fr in zip(search_results, final_results, strict=True)
+            for result, fr in zip(search_results, final_results, strict=True)
         ]
 
     async def list_snippets(
         self, file_path: str | None = None, source_uri: str | None = None
-    ) -> list[SnippetListItem]:
+    ) -> list[MultiSearchResult]:
         """List snippets with optional filtering."""
         log_event("kodit.index.list_snippets")
         return await self.snippet_domain_service.list_snippets(file_path, source_uri)