@pleaseai/context-please-core 0.2.1 → 0.5.0

package/README.md

@@ -1,4 +1,5 @@
 # @pleaseai/context-please-core
+
 ![](../../assets/claude-context.png)
 
 The core indexing engine for Context Please - a powerful tool for semantic search and analysis of codebases using vector embeddings and AI.
@@ -17,20 +18,24 @@ npm install @pleaseai/context-please-core
 ```
 
 ### Prepare Environment Variables
+
 #### OpenAI API key
+
 See [OpenAI Documentation](https://platform.openai.com/docs/api-reference) for more details to get your API key.
+
 ```bash
 OPENAI_API_KEY=your-openai-api-key
 ```
 
 #### Zilliz Cloud configuration
-Get a free Milvus vector database on Zilliz Cloud. 
+
+Get a free Milvus vector database on Zilliz Cloud.
 
 Claude Context needs a vector database. You can [sign up](https://cloud.zilliz.com/signup?utm_source=github&utm_medium=referral&utm_campaign=2507-codecontext-readme) on Zilliz Cloud to get a free Serverless cluster.
 
 ![](../../assets/signup_and_create_cluster.jpeg)
 
-After creating your cluster, open your Zilliz Cloud console and copy both the **public endpoint** and your **API key**.  
+After creating your cluster, open your Zilliz Cloud console and copy both the **public endpoint** and your **API key**.
 These will be used as `your-zilliz-cloud-public-endpoint` and `your-zilliz-cloud-api-key` in the configuration examples.
 
 ![Zilliz Cloud Dashboard](../../assets/zilliz_cloud_dashboard.jpeg)
@@ -42,56 +47,56 @@ If you need help creating your free vector database or finding these values, see
 ```bash
 MILVUS_ADDRESS=your-zilliz-cloud-public-endpoint
 MILVUS_TOKEN=your-zilliz-cloud-api-key
-``` 
+```
 
 > 💡 **Tip**: For easier configuration management across different usage scenarios, consider using [global environment variables](../../docs/getting-started/environment-variables.md).
 
 ## Quick Start
 
 ```typescript
-import { 
-  Context, 
-  OpenAIEmbedding, 
-  MilvusVectorDatabase 
-} from '@pleaseai/context-please-core';
+import {
+  Context,
+  MilvusVectorDatabase,
+  OpenAIEmbedding
+} from '@pleaseai/context-please-core'
 
 // Initialize embedding provider
 const embedding = new OpenAIEmbedding({
   apiKey: process.env.OPENAI_API_KEY || 'your-openai-api-key',
   model: 'text-embedding-3-small'
-});
+})
 
 // Initialize vector database
 const vectorDatabase = new MilvusVectorDatabase({
   address: process.env.MILVUS_ADDRESS || 'localhost:19530',
   token: process.env.MILVUS_TOKEN || ''
-});
+})
 
 // Create context instance
 const context = new Context({
   embedding,
   vectorDatabase
-});
+})
 
 // Index a codebase
 const stats = await context.indexCodebase('./my-project', (progress) => {
-  console.log(`${progress.phase} - ${progress.percentage}%`);
-});
+  console.log(`${progress.phase} - ${progress.percentage}%`)
+})
 
-console.log(`Indexed ${stats.indexedFiles} files with ${stats.totalChunks} chunks`);
+console.log(`Indexed ${stats.indexedFiles} files with ${stats.totalChunks} chunks`)
 
 // Search the codebase
 const results = await context.semanticSearch(
   './my-project',
   'function that handles user authentication',
   5
-);
+)
 
-results.forEach(result => {
-  console.log(`${result.relativePath}:${result.startLine}-${result.endLine}`);
-  console.log(`Score: ${result.score}`);
-  console.log(result.content);
-});
+results.forEach((result) => {
+  console.log(`${result.relativePath}:${result.startLine}-${result.endLine}`)
+  console.log(`Score: ${result.score}`)
+  console.log(result.content)
+})
 ```
 
 ## Features
@@ -126,13 +131,13 @@ results.forEach(result => {
 
 ```typescript
 interface ContextConfig {
-  embedding?: Embedding;           // Embedding provider
-  vectorDatabase?: VectorDatabase; // Vector database instance (required)
-  codeSplitter?: Splitter;        // Code splitting strategy
-  supportedExtensions?: string[]; // File extensions to index
-  ignorePatterns?: string[];      // Patterns to ignore
-  customExtensions?: string[];    // Custom extensions from MCP
-  customIgnorePatterns?: string[]; // Custom ignore patterns from MCP
+  embedding?: Embedding // Embedding provider
+  vectorDatabase?: VectorDatabase // Vector database instance (required)
+  codeSplitter?: Splitter // Code splitting strategy
+  supportedExtensions?: string[] // File extensions to index
+  ignorePatterns?: string[] // Patterns to ignore
+  customExtensions?: string[] // Custom extensions from MCP
+  customIgnorePatterns?: string[] // Custom ignore patterns from MCP
 }
 ```
 
@@ -141,10 +146,30 @@ interface ContextConfig {
 ```typescript
 [
   // Programming languages
-  '.ts', '.tsx', '.js', '.jsx', '.py', '.java', '.cpp', '.c', '.h', '.hpp',
-  '.cs', '.go', '.rs', '.php', '.rb', '.swift', '.kt', '.scala', '.m', '.mm',
-  // Text and markup files  
-  '.md', '.markdown', '.ipynb'
+  '.ts',
+  '.tsx',
+  '.js',
+  '.jsx',
+  '.py',
+  '.java',
+  '.cpp',
+  '.c',
+  '.h',
+  '.hpp',
+  '.cs',
+  '.go',
+  '.rs',
+  '.php',
+  '.rb',
+  '.swift',
+  '.kt',
+  '.scala',
+  '.m',
+  '.mm',
+  // Text and markup files
+  '.md',
+  '.markdown',
+  '.ipynb'
 ]
 ```
 
@@ -180,38 +205,76 @@ interface ContextConfig {
 
 ```typescript
 interface SemanticSearchResult {
-  content: string;      // Code content
-  relativePath: string; // File path relative to codebase root
-  startLine: number;    // Starting line number
-  endLine: number;      // Ending line number
-  language: string;     // Programming language
-  score: number;        // Similarity score (0-1)
+  content: string // Code content
+  relativePath: string // File path relative to codebase root
+  startLine: number // Starting line number
+  endLine: number // Ending line number
+  language: string // Programming language
+  score: number // Similarity score (0-1)
 }
 ```
 
-
 ## Examples
 
 ### Using VoyageAI Embeddings
 
 ```typescript
-import { Context, MilvusVectorDatabase, VoyageAIEmbedding } from '@pleaseai/context-please-core';
+import { Context, MilvusVectorDatabase, VoyageAIEmbedding } from '@pleaseai/context-please-core'
 
 // Initialize with VoyageAI embedding provider
 const embedding = new VoyageAIEmbedding({
   apiKey: process.env.VOYAGEAI_API_KEY || 'your-voyageai-api-key',
   model: 'voyage-code-3'
-});
+})
 
 const vectorDatabase = new MilvusVectorDatabase({
   address: process.env.MILVUS_ADDRESS || 'localhost:19530',
   token: process.env.MILVUS_TOKEN || ''
-});
+})
 
 const context = new Context({
   embedding,
   vectorDatabase
-});
+})
+```
+
+### Using Gemini Embeddings with Retry Configuration
+
+```typescript
+import { Context, MilvusVectorDatabase, GeminiEmbedding } from '@pleaseai/context-please-core'
+
+// Initialize with Gemini embedding provider
+const embedding = new GeminiEmbedding({
+  apiKey: process.env.GEMINI_API_KEY || 'your-gemini-api-key',
+  model: 'gemini-embedding-001',
+  outputDimensionality: 768, // Optional: Matryoshka Representation Learning support (256, 768, 1536, 3072)
+  maxRetries: 3, // Optional: Maximum retry attempts (default: 3)
+  baseDelay: 1000 // Optional: Base delay in ms for exponential backoff (default: 1000ms)
+})
+
+const vectorDatabase = new MilvusVectorDatabase({
+  address: process.env.MILVUS_ADDRESS || 'localhost:19530',
+  token: process.env.MILVUS_TOKEN || ''
+})
+
+const context = new Context({
+  embedding,
+  vectorDatabase
+})
+
+// The retry mechanism automatically handles:
+// - Rate limit errors (429)
+// - Server errors (500, 502, 503, 504)
+// - Network errors (ECONNREFUSED, ETIMEDOUT, ENOTFOUND, EAI_AGAIN)
+// - Transient API failures with exponential backoff (1s → 2s → 4s → 8s, capped at 10s)
+
+// Update retry configuration at runtime
+embedding.setMaxRetries(5)
+embedding.setBaseDelay(2000)
+
+// Check current retry configuration
+const retryConfig = embedding.getRetryConfig()
+console.log(`Max retries: ${retryConfig.maxRetries}, Base delay: ${retryConfig.baseDelay}ms`)
 ```
 
 ### Custom File Filtering
@@ -227,7 +290,7 @@ const context = new Context({
     '*.spec.ts',
     '*.test.js'
   ]
-});
+})
 ```
 
 ## File Synchronization Architecture
@@ -241,21 +304,25 @@ Claude Context implements an intelligent file synchronization system that effici
 The file synchronization system uses a **Merkle tree-based approach** combined with SHA-256 file hashing to detect changes:
 
 #### 1. File Hashing
+
 - Each file in the codebase is hashed using SHA-256
 - File hashes are computed based on file content, not metadata
 - Hashes are stored with relative file paths for consistency across different environments
 
 #### 2. Merkle Tree Construction
+
 - All file hashes are organized into a Merkle tree structure
 - The tree provides a single root hash that represents the entire codebase state
 - Any change to any file will cause the root hash to change
 
 #### 3. Snapshot Management
+
 - File synchronization state is persisted to `~/.context/merkle/` directory
 - Each codebase gets a unique snapshot file based on its absolute path hash
 - Snapshots contain both file hashes and serialized Merkle tree data
 
 #### 4. Change Detection Process
+
 1. **Quick Check**: Compare current Merkle root hash with stored snapshot
 2. **Detailed Analysis**: If root hashes differ, perform file-by-file comparison
 3. **Change Classification**: Categorize changes into three types:
@@ -264,15 +331,16 @@ The file synchronization system uses a **Merkle tree-based approach** combined w
    - **Removed**: Files that were deleted from the codebase
 
 #### 5. Incremental Updates
+
 - Only process files that have actually changed
 - Update vector database entries only for modified chunks
 - Remove entries for deleted files
 - Add entries for new files
 
-
 ## Contributing
 
 This package is part of the Claude Context monorepo. Please see:
+
 - [Main Contributing Guide](../../CONTRIBUTING.md) - General contribution guidelines
 - [Core Package Contributing](CONTRIBUTING.md) - Specific development guide for this package
 
@@ -281,7 +349,6 @@ This package is part of the Claude Context monorepo. Please see:
 - **[@claude-context/mcp](../mcp)** - MCP server that uses this core engine
 - **[VSCode Extension](../vscode-extension)** - VSCode extension built on this core
 
-
 ## License
 
-MIT - See [LICENSE](../../LICENSE) for details
+MIT - See [LICENSE](../../LICENSE) for details