kodit 0.3.3__tar.gz → 0.3.4__tar.gz

kodit-0.3.4/.claude/commands/update-docs.md

@@ -0,0 +1,79 @@
+# Update Documentation Based on Recent Code Changes
+
+## Objective
+
+Analyze recent commits in the current Git branch and update relevant documentation to reflect code changes.
+
+## Steps to Complete
+
+### 1. Analyze Recent Commits
+
+- Run `git log --oneline -n 20` to view the last 20 commits on the current branch, or
+  all commits if it is not the main branch.
+- For each commit, run `git show --name-status <commit-hash>` to see which files were modified
+- Focus on commits that modified source code files (`.js`, `.py`, `.ts`, `.java`, `.go`, `.rs`, etc.)
+
+### 2. Identify Code Changes
+
+For each modified source file in recent commits:
+
+- Examine the diff using `git diff <commit-hash>^ <commit-hash> -- <file-path>`
+- Identify:
+  - New functions or methods added
+  - Functions or methods removed or renamed
+  - Changes to function signatures (parameters, return types)
+  - New classes or modules
+  - Changes to public APIs
+  - New configuration options or environment variables
+  - Breaking changes
+
+### 3. Update README.md
+
+Check if the README.md needs updates for:
+
+- **Installation instructions**: If dependencies or setup steps changed
+- **Usage examples**: If APIs or interfaces changed
+- **Configuration**: If new environment variables or config options were added
+- **Features list**: If new features were implemented
+- **Quick start guide**: If the basic usage pattern changed
+
+### 4. Update Documentation in /docs
+
+For each markdown file in the `docs/` folder:
+
+- Check if it references any of the changed code
+- Update:
+  - API documentation with new/changed function signatures
+  - Code examples that may no longer work
+  - Configuration guides if settings changed
+  - Architecture diagrams if structural changes occurred
+  - Migration guides if there are breaking changes
+
+### 5. Create or Update Specific Docs
+
+Based on the changes found:
+
+- If new features were added without documentation, create new doc files
+- If breaking changes exist, create or update a migration guide
+- If new APIs were added, ensure they have proper documentation
+
+### 6. Verify Documentation Accuracy
+
+- Ensure all code examples in documentation are up-to-date
+- Check that any referenced file paths still exist
+- Verify that installation and setup instructions still work
+
+## Output Required
+
+1. Summary of commits analyzed and significant changes found
+2. List of documentation files updated with brief description of changes
+3. Any new documentation files created
+4. Warnings about potentially outdated documentation that needs manual review
+
+## Important Notes
+
+- Focus on user-facing changes that affect how people use Kodit
+- Don't document internal implementation details unless they affect the public API
+- Keep documentation concise and example-driven
+- If unsure about a change's impact, flag it for manual review
+- Ensure all documentation follows the existing style and format in the repository

{kodit-0.3.3 → kodit-0.3.4}/CLAUDE.md

@@ -50,18 +50,23 @@ The codebase follows Domain-Driven Design (DDD) with clean architecture:
 
 ### Key Components
 
-**Indexing Pipeline:**
+**Advanced Indexing Pipeline:**
 
-1. Clone/read source code
-2. Extract snippets using Tree-sitter
-3. Generate embeddings and BM25 indices
-4. Store in database
+1. Clone/read source code with Git metadata extraction
+2. Language detection for 20+ programming languages
+3. Advanced snippet extraction using Tree-sitter with dependency analysis
+4. Build call graphs and import maps for context-aware extraction
+5. Generate embeddings and BM25 indices
+6. Store in database with selective reindexing for performance
 
-**Search System:**
+**Advanced Search System:**
 
-- Hybrid search combining semantic (embeddings) and keyword (BM25)
+- Hybrid search combining semantic (embeddings) and keyword (BM25) with Reciprocal Rank Fusion
+- Multi-dimensional filtering: language, author, date range, source, file path
+- Context-aware results with dependency tracking and usage examples
 - Multiple providers: local models, OpenAI, custom APIs
 - Configurable via environment variables
+- Support for 20+ programming languages including HTML/CSS
 
 **MCP Server:**
 

{kodit-0.3.3 → kodit-0.3.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: kodit
-Version: 0.3.3
+Version: 0.3.4
 Summary: Code indexing for better AI code generation
 Project-URL: Homepage, https://docs.helixml.tech/kodit/
 Project-URL: Documentation, https://docs.helixml.tech/kodit/
@@ -92,13 +92,16 @@ code. This index is used to build a snippet library, ready for ingestion into an
 
 - Index local directories and public Git repositories
 - Build comprehensive snippet libraries for LLM ingestion
-- Support for multiple codebase types and languages
-- Efficient indexing and search capabilities
+- Support for 20+ programming languages including Python, JavaScript/TypeScript, Java, Go, Rust, C/C++, C#, HTML/CSS, and more
+- Advanced code analysis with dependency tracking and call graph generation
+- Intelligent snippet extraction with context-aware dependencies
+- Efficient indexing with selective reindexing (only processes modified files)
 - Privacy first: respects .gitignore and .noindex files
 - **NEW in 0.3**: Auto-indexing configuration for shared server deployments
 - **NEW in 0.3**: Enhanced Git provider support including Azure DevOps
 - **NEW in 0.3**: Index private repositories via a PAT
 - **NEW in 0.3**: Improved progress monitoring and reporting during indexing
+- **NEW in 0.3**: Advanced code slicing infrastructure with Tree-sitter parsing
 
 ### MCP Server
 
@@ -111,7 +114,9 @@ intent. Kodit has been tested to work well with:
   - [Cursor](https://docs.helix.ml/kodit/getting-started/integration/#integration-with-cursor)
   - [Cline](https://docs.helix.ml/kodit/getting-started/integration/#integration-with-cline)
 - Please contribute more instructions! ... any other assistant is likely to work ...
-- **New in 0.3**: Filter snippets by source, language, author or timestamp.
+- **New in 0.3**: Advanced search filters by source, language, author, date range, and file path
+- **New in 0.3**: Hybrid search combining BM25 keyword search with semantic search
+- **New in 0.4**: Enhanced MCP tools with rich context parameters and metadata
 
 ### Enterprise Ready
 

{kodit-0.3.3 → kodit-0.3.4}/README.md

@@ -39,13 +39,16 @@ code. This index is used to build a snippet library, ready for ingestion into an
 
 - Index local directories and public Git repositories
 - Build comprehensive snippet libraries for LLM ingestion
-- Support for multiple codebase types and languages
-- Efficient indexing and search capabilities
+- Support for 20+ programming languages including Python, JavaScript/TypeScript, Java, Go, Rust, C/C++, C#, HTML/CSS, and more
+- Advanced code analysis with dependency tracking and call graph generation
+- Intelligent snippet extraction with context-aware dependencies
+- Efficient indexing with selective reindexing (only processes modified files)
 - Privacy first: respects .gitignore and .noindex files
 - **NEW in 0.3**: Auto-indexing configuration for shared server deployments
 - **NEW in 0.3**: Enhanced Git provider support including Azure DevOps
 - **NEW in 0.3**: Index private repositories via a PAT
 - **NEW in 0.3**: Improved progress monitoring and reporting during indexing
+- **NEW in 0.3**: Advanced code slicing infrastructure with Tree-sitter parsing
 
 ### MCP Server
 
@@ -58,7 +61,9 @@ intent. Kodit has been tested to work well with:
   - [Cursor](https://docs.helix.ml/kodit/getting-started/integration/#integration-with-cursor)
   - [Cline](https://docs.helix.ml/kodit/getting-started/integration/#integration-with-cline)
 - Please contribute more instructions! ... any other assistant is likely to work ...
-- **New in 0.3**: Filter snippets by source, language, author or timestamp.
+- **New in 0.3**: Advanced search filters by source, language, author, date range, and file path
+- **New in 0.3**: Hybrid search combining BM25 keyword search with semantic search
+- **New in 0.4**: Enhanced MCP tools with rich context parameters and metadata
 
 ### Enterprise Ready
 

{kodit-0.3.3 → kodit-0.3.4}/docs/_index.md

@@ -49,13 +49,16 @@ code. This index is used to build a snippet library, ready for ingestion into an
 
 - Index local directories and public Git repositories
 - Build comprehensive snippet libraries for LLM ingestion
-- Support for multiple codebase types and languages
-- Efficient indexing and search capabilities
+- Support for 20+ programming languages including Python, JavaScript/TypeScript, Java, Go, Rust, C/C++, C#, HTML/CSS, and more
+- Advanced code analysis with dependency tracking and call graph generation
+- Intelligent snippet extraction with context-aware dependencies
+- Efficient indexing with selective reindexing (only processes modified files)
 - Privacy first: respects .gitignore and .noindex files
 - **NEW in 0.3**: Auto-indexing configuration for shared server deployments
 - **NEW in 0.3**: Enhanced Git provider support including Azure DevOps
 - **NEW in 0.3**: Index private repositories via a PAT
 - **NEW in 0.3**: Improved progress monitoring and reporting during indexing
+- **NEW in 0.4**: Advanced code slicing infrastructure with Tree-sitter parsing
 
 ### MCP Server
 
@@ -68,7 +71,9 @@ intent. Kodit has been tested to work well with:
   - [Cursor](./getting-started/integration/index.md#integration-with-cursor)
   - [Cline](./getting-started/integration/index.md#integration-with-cline)
 - Please contribute more instructions! ... any other assistant is likely to work ...
-- **New in 0.3**: Filter snippets by source, language, author or timestamp.
+- **New in 0.3**: Advanced search filters by source, language, author, date range, and file path
+- **New in 0.3**: Hybrid search combining BM25 keyword search with semantic search
+- **New in 0.3**: Enhanced MCP tools with rich context parameters and metadata
 
 ### Enterprise Ready
 

{kodit-0.3.3 → kodit-0.3.4}/docs/reference/indexing/index.md

@@ -209,26 +209,51 @@ Kodit respects [standard ignore patterns](#ignore-patterns):
 - **`.gitignore`**: Standard Git ignore patterns
 - **`.noindex`**: Custom ignore patterns for Kodit (uses gitignore syntax)
 
-### Supported File Types
+### Supported Programming Languages
 
-Kodit automatically detects and processes files based on their extensions:
+Kodit automatically detects and processes files based on their extensions. The following languages are supported with advanced Tree-sitter parsing:
 
-| Language | Extensions |
-|----------|------------|
-| Python | `.py` |
-| JavaScript | `.js`, `.jsx` |
-| TypeScript | `.ts`, `.tsx` |
-| Go | `.go` |
-| C# | `.cs` |
+| Language | Extensions | Features |
+|----------|------------|----------|
+| Python | `.py`, `.pyw`, `.pyx`, `.pxd` | Function/method extraction, import analysis, call graph |
+| JavaScript | `.js`, `.jsx`, `.mjs` | Function extraction, ES6 modules, JSX support |
+| TypeScript | `.ts`, `.tsx` | Type definitions, interfaces, decorators |
+| Java | `.java` | Method declarations, constructors, class hierarchies |
+| Go | `.go` | Function/method extraction, package imports |
+| Rust | `.rs` | Function definitions, trait implementations |
+| C/C++ | `.c`, `.h`, `.cpp`, `.cc`, `.cxx`, `.hpp`, `.hxx` | Function definitions, header includes |
+| C# | `.cs` | Method declarations, using directives, constructors |
+| HTML | `.html`, `.htm` | Element extraction with ID/class identification |
+| CSS | `.css`, `.scss`, `.sass`, `.less` | Rule extraction, selector analysis, keyframes |
 
-### Snippet Extraction
+### Advanced Snippet Extraction
 
-Kodit uses tree-sitter to intelligently extract code snippets:
+Kodit uses a sophisticated Tree-sitter-based slicing system to intelligently extract code snippets with context:
+
+#### Core Features
 
 - **Functions and Methods**: Complete function definitions with their bodies
 - **Classes**: Class definitions and their methods
 - **Imports**: Import statements for context
 - **Dependencies**: Ancestor classes and functions that the snippet depends on
+- **Call Graph Analysis**: Builds relationships between functions to understand dependencies
+- **Context-Aware Extraction**: Includes related functions and usage examples
+- **Topological Sorting**: Orders dependencies for optimal LLM consumption
+
+#### Smart Dependency Tracking
+
+- **Import Maps**: Tracks import statements and their usage
+- **Function Calls**: Identifies which functions call which others
+- **Reverse Dependencies**: Finds all callers of a given function
+- **Usage Examples**: Includes examples of how functions are used in the codebase
+
+#### Language-Specific Extraction
+
+- **Python**: Decorators, async functions, class inheritance
+- **JavaScript/TypeScript**: Arrow functions, async/await, ES6 modules
+- **Java**: Annotations, generics, inheritance hierarchies
+- **Go**: Interfaces, struct methods, package organization
+- **HTML/CSS**: Elements with semantic context, CSS rules and selectors
 
 ## Configuration
 
@@ -269,9 +294,35 @@ DEFAULT_ENDPOINT_API_KEY=sk-your-api-key
 
 ## Advanced Features
 
-### Re-indexing Sources
+### Selective Re-indexing
+
+Kodit includes intelligent re-indexing that only processes files that have been modified:
+
+#### How It Works
+
+- **SHA256 Change Detection**: Compares file content hashes to detect changes
+- **File Status Tracking**: Tracks files as CLEAN, MODIFIED, or DELETED
+- **Incremental Updates**: Only re-processes changed files, improving performance for large codebases
+- **Metadata Preservation**: Maintains file metadata and Git information
 
-Future feature!
+#### Benefits
+
+- **Performance**: Dramatically faster re-indexing for large repositories
+- **Resource Efficiency**: Reduces CPU and memory usage during updates
+- **Consistency**: Ensures only actual changes trigger re-processing
+- **Scalability**: Enables efficient handling of large, frequently-updated codebases
+
+#### Usage
+
+Re-indexing automatically uses selective processing when you re-index an existing source:
+
+```sh
+# Re-index with selective processing
+kodit index /path/to/existing/source
+
+# Or for Git repositories
+kodit index https://github.com/username/repo.git
+```
 
 ### Progress Monitoring
 

{kodit-0.3.3 → kodit-0.3.4}/docs/reference/mcp/index.md

@@ -53,21 +53,30 @@ The search tool accepts the following parameters:
 | `related_file_paths` | list[Path] | Absolute paths to relevant files | `["/path/to/auth.py"]` |
 | `related_file_contents` | list[string] | Contents of relevant files | `["def authenticate(): ..."]` |
 | `keywords` | list[string] | Relevant keywords for the search | `["authentication", "jwt", "login"]` |
-| `language` | string \| None | Filter by programming language | `"python"`, `"go"`, `"javascript"` |
+| `language` | string \| None | Filter by programming language (20+ supported) | `"python"`, `"go"`, `"javascript"`, `"html"`, `"css"` |
 | `author` | string \| None | Filter by author name | `"john.doe"` |
 | `created_after` | string \| None | Filter by creation date (YYYY-MM-DD) | `"2023-01-01"` |
 | `created_before` | string \| None | Filter by creation date (YYYY-MM-DD) | `"2023-12-31"` |
 | `source_repo` | string \| None | Filter by source repository | `"github.com/example/repo"` |
+| `file_path` | string \| None | Filter by file path pattern | `"src/"`, `"*.test.py"` |
 
-### Search Functionality
+### Advanced Search Functionality
 
-The search tool combines multiple search strategies:
+The search tool combines multiple search strategies with sophisticated ranking:
 
-1. **Keyword Search** - Uses BM25 algorithm for exact keyword matching
-2. **Semantic Code Search** - Uses embeddings to find semantically similar code
+1. **BM25 Keyword Search** - Advanced keyword matching with relevance scoring
+2. **Semantic Code Search** - Uses embeddings to find semantically similar code patterns
 3. **Semantic Text Search** - Uses embeddings to find code matching natural language descriptions
+4. **Reciprocal Rank Fusion (RRF)** - Intelligently combines results from multiple search strategies
+5. **Context-Aware Filtering** - Advanced filtering by language, author, date, source, and file path
+6. **Dependency-Aware Results** - Returns code snippets with their dependencies and usage examples
 
-Results are fused together to provide the most relevant snippets for the user's intent.
+#### Enhanced Result Quality
+
+- **Smart Snippet Selection**: Returns functions with their dependencies and context
+- **Rich Metadata**: Each result includes file path, language, author, and creation date
+- **Usage Examples**: Includes examples of how functions are used in the codebase
+- **Topological Ordering**: Dependencies are ordered for optimal LLM consumption
 
 ## Filtering Capabilities
 
@@ -81,6 +90,8 @@ Filter results by programming language:
 > "I need to create a web server in Python. Please search for Flask or FastAPI examples and show me the best practices."
 > "I'm working on a Go microservice. Can you search for Go-specific patterns for handling HTTP requests and database connections?"
 > "I need JavaScript examples for form validation. Please search for modern JavaScript/TypeScript validation patterns."
+> "I'm building a responsive layout. Please search for CSS Grid and Flexbox examples in our stylesheets."
+> "I need HTML form examples. Please search for form elements with proper accessibility attributes."
 
 ### Author Filtering
 

{kodit-0.3.3 → kodit-0.3.4}/pyproject.toml

@@ -104,6 +104,11 @@ source = ["src"]
 branch = true
 omit = []
 
+[tool.ruff]
+exclude = [
+  "tests/kodit/infrastructure/slicing/data/**/*.py",
+]
+
 [tool.ruff.lint]
 ignore = [
   "ANN401", # Opinionated warning on disallowing dynamically typed expressions

{kodit-0.3.3 → kodit-0.3.4}/src/kodit/_version.py

@@ -17,5 +17,5 @@ __version__: str
 __version_tuple__: VERSION_TUPLE
 version_tuple: VERSION_TUPLE
 
-__version__ = version = '0.3.3'
-__version_tuple__ = version_tuple = (0, 3, 3)
+__version__ = version = '0.3.4'
+__version_tuple__ = version_tuple = (0, 3, 4)

{kodit-0.3.3 → kodit-0.3.4}/src/kodit/application/factories/code_indexing_factory.py

@@ -13,7 +13,7 @@ from kodit.domain.services.index_query_service import IndexQueryService
 from kodit.domain.services.index_service import (
     IndexDomainService,
 )
-from kodit.domain.value_objects import LanguageMapping, SnippetExtractionStrategy
+from kodit.domain.value_objects import LanguageMapping
 from kodit.infrastructure.bm25.bm25_factory import bm25_repository_factory
 from kodit.infrastructure.embedding.embedding_factory import (
     embedding_domain_service_factory,
@@ -31,15 +31,9 @@ from kodit.infrastructure.enrichment.null_enrichment_provider import (
     NullEnrichmentProvider,
 )
 from kodit.infrastructure.indexing.fusion_service import ReciprocalRankFusionService
-from kodit.infrastructure.snippet_extraction.factories import (
-    create_snippet_query_provider,
-)
-from kodit.infrastructure.snippet_extraction.language_detection_service import (
+from kodit.infrastructure.slicing.language_detection_service import (
     FileSystemLanguageDetectionService,
 )
-from kodit.infrastructure.snippet_extraction.tree_sitter_snippet_extractor import (
-    TreeSitterSnippetExtractor,
-)
 from kodit.infrastructure.sqlalchemy.embedding_repository import (
     SqlAlchemyEmbeddingRepository,
 )
@@ -63,17 +57,9 @@ def create_code_indexing_application_service(
 
     # Create infrastructure services
     language_detector = FileSystemLanguageDetectionService(language_map)
-    query_provider = create_snippet_query_provider()
 
-    # Create snippet extractors
-    method_extractor = TreeSitterSnippetExtractor(query_provider)
-
-    snippet_extractors = {
-        SnippetExtractionStrategy.METHOD_BASED: method_extractor,
-    }
     index_domain_service = IndexDomainService(
         language_detector=language_detector,
-        snippet_extractors=snippet_extractors,
         enrichment_service=enrichment_service,
         clone_dir=app_context.get_clone_dir(),
     )
@@ -136,17 +122,9 @@ def create_fast_test_code_indexing_application_service(
 
     # Create infrastructure services
     language_detector = FileSystemLanguageDetectionService(language_map)
-    query_provider = create_snippet_query_provider()
-
-    # Create snippet extractors
-    method_extractor = TreeSitterSnippetExtractor(query_provider)
 
-    snippet_extractors = {
-        SnippetExtractionStrategy.METHOD_BASED: method_extractor,
-    }
     index_domain_service = IndexDomainService(
         language_detector=language_detector,
-        snippet_extractors=snippet_extractors,
         enrichment_service=enrichment_service,
         clone_dir=app_context.get_clone_dir(),
     )

{kodit-0.3.3 → kodit-0.3.4}/src/kodit/application/services/code_indexing_application_service.py

@@ -100,6 +100,11 @@ class CodeIndexingApplicationService:
             self.log.info("No new changes to index", index_id=index.id)
             return
 
+        # Delete the old snippets from the files that have changed
+        await self.index_repository.delete_snippets_by_file_ids(
+            [file.id for file in index.source.working_copy.changed_files() if file.id]
+        )
+
         # Extract and create snippets (domain service handles progress)
         self.log.info("Creating snippets for files", index_id=index.id)
         index = await self.index_domain_service.extract_snippets_from_index(
@@ -115,6 +120,9 @@ class CodeIndexingApplicationService:
             msg = f"Index {index.id} not found after snippet extraction"
             raise ValueError(msg)
         index = flushed_index
+        if len(index.snippets) == 0:
+            self.log.info("No snippets to index after extraction", index_id=index.id)
+            return
 
         # Create BM25 index
         self.log.info("Creating keyword index")
@@ -154,8 +162,8 @@ class CodeIndexingApplicationService:
         # Apply filters if provided
         filtered_snippet_ids: list[int] | None = None
         if request.filters:
-            # Use domain service for filtering
-            prefilter_request = replace(request, top_k=None)
+            # Use domain service for filtering (use large top_k for pre-filtering)
+            prefilter_request = replace(request, top_k=10000)
             snippet_results = await self.index_query_service.search_snippets(
                 prefilter_request
             )

{kodit-0.3.3 → kodit-0.3.4}/src/kodit/domain/services/index_service.py

@@ -1,7 +1,6 @@
 """Pure domain service for Index aggregate operations."""
 
 from abc import ABC, abstractmethod
-from collections.abc import Mapping
 from pathlib import Path
 
 import structlog
@@ -13,14 +12,13 @@ from kodit.domain.services.enrichment_service import EnrichmentDomainService
 from kodit.domain.value_objects import (
     EnrichmentIndexRequest,
     EnrichmentRequest,
-    SnippetExtractionRequest,
-    SnippetExtractionResult,
-    SnippetExtractionStrategy,
+    LanguageMapping,
 )
 from kodit.infrastructure.cloning.git.working_copy import GitWorkingCopyProvider
 from kodit.infrastructure.cloning.metadata import FileMetadataExtractor
 from kodit.infrastructure.git.git_utils import is_valid_clone_target
 from kodit.infrastructure.ignore.ignore_pattern_provider import GitIgnorePatternProvider
+from kodit.infrastructure.slicing.slicer import Slicer
 from kodit.reporting import Reporter
 from kodit.utils.path_utils import path_from_uri
 
@@ -33,14 +31,6 @@ class LanguageDetectionService(ABC):
         """Detect the programming language of a file."""
 
 
-class SnippetExtractor(ABC):
-    """Abstract interface for snippet extraction."""
-
-    @abstractmethod
-    async def extract(self, file_path: Path, language: str) -> list[str]:
-        """Extract snippets from a file."""
-
-
 class IndexDomainService:
     """Pure domain service for Index aggregate operations.
 
@@ -54,14 +44,12 @@ class IndexDomainService:
     def __init__(
         self,
         language_detector: LanguageDetectionService,
-        snippet_extractors: Mapping[SnippetExtractionStrategy, SnippetExtractor],
         enrichment_service: EnrichmentDomainService,
         clone_dir: Path,
     ) -> None:
         """Initialize the index domain service."""
         self._clone_dir = clone_dir
         self._language_detector = language_detector
-        self._snippet_extractors = snippet_extractors
         self._enrichment_service = enrichment_service
         self.log = structlog.get_logger(__name__)
 
@@ -99,7 +87,6 @@ class IndexDomainService:
     async def extract_snippets_from_index(
         self,
         index: domain_entities.Index,
-        strategy: SnippetExtractionStrategy = SnippetExtractionStrategy.METHOD_BASED,
         progress_callback: ProgressCallback | None = None,
     ) -> domain_entities.Index:
         """Extract code snippets from files in the index."""
@@ -109,46 +96,40 @@ class IndexDomainService:
             "Extracting snippets",
             index_id=index.id,
             file_count=file_count,
-            strategy=strategy.value,
         )
 
         # Only create snippets for files that have been added or modified
         files = index.source.working_copy.changed_files()
         index.delete_snippets_for_files(files)
 
-        reporter = Reporter(self.log, progress_callback)
-        await reporter.start(
-            "extract_snippets", len(files), "Extracting code snippets..."
-        )
-
-        new_snippets = []
-        for i, domain_file in enumerate(files, 1):
+        # Create a set of languages to extract snippets for
+        extensions = {file.extension() for file in files}
+        languages = []
+        for ext in extensions:
             try:
-                # Extract snippets from file
-                request = SnippetExtractionRequest(
-                    file_path=domain_file.as_path(), strategy=strategy
-                )
-                result = await self._extract_snippets(request)
-                for snippet_text in result.snippets:
-                    snippet = domain_entities.Snippet(
-                        derives_from=[domain_file],
-                    )
-                    snippet.add_original_content(snippet_text, result.language)
-                    new_snippets.append(snippet)
-
-            except (OSError, ValueError) as e:
-                self.log.debug(
-                    "Skipping file for snippet extraction",
-                    file_uri=str(domain_file.uri),
-                    error=str(e),
-                )
+                languages.append(LanguageMapping.get_language_for_extension(ext))
+            except ValueError as e:
+                self.log.info("Skipping", error=str(e))
                 continue
 
+        reporter = Reporter(self.log, progress_callback)
+        await reporter.start(
+            "extract_snippets",
+            len(files) * len(languages),
+            "Extracting code snippets...",
+        )
+        # Calculate snippets for each language
+        slicer = Slicer()
+        for i, language in enumerate(languages):
             await reporter.step(
-                "extract_snippets", i, len(files), f"Processed {domain_file.uri.path}"
+                "extract_snippets",
+                len(files) * (i + 1),
+                len(files) * len(languages),
+                "Extracting code snippets...",
             )
+            s = slicer.extract_snippets(files, language=language)
+            index.snippets.extend(s)
 
-        index.snippets.extend(new_snippets)
         await reporter.done("extract_snippets")
         return index
 
@@ -187,28 +168,6 @@ class IndexDomainService:
         await reporter.done("enrichment")
         return list(snippet_map.values())
 
-    async def _extract_snippets(
-        self, request: SnippetExtractionRequest
-    ) -> SnippetExtractionResult:
-        # Domain logic: validate file exists
-        if not request.file_path.exists():
-            raise ValueError(f"File does not exist: {request.file_path}")
-
-        # Domain logic: detect language
-        language = await self._language_detector.detect_language(request.file_path)
-
-        # Domain logic: choose strategy and extractor
-        if request.strategy not in self._snippet_extractors:
-            raise ValueError(f"Unsupported extraction strategy: {request.strategy}")
-
-        extractor = self._snippet_extractors[request.strategy]
-        snippets = await extractor.extract(request.file_path, language)
-
-        # Domain logic: filter out empty snippets
-        filtered_snippets = [snippet for snippet in snippets if snippet.strip()]
-
-        return SnippetExtractionResult(snippets=filtered_snippets, language=language)
-
     def sanitize_uri(
         self, uri_or_path_like: str
     ) -> tuple[AnyUrl, domain_entities.SourceType]:
@@ -297,7 +256,7 @@ class IndexDomainService:
                     await metadata_extractor.extract(file_path=file_path)
                 )
             except (OSError, ValueError) as e:
-                self.log.info("Skipping file", file=str(file_path), error=str(e))
+                self.log.debug("Skipping file", file=str(file_path), error=str(e))
                 continue
 
         # Finally check if there are any modified files