@msbayindir/context-rag 1.0.0-beta.9 → 2.0.0-beta.1

package/README.md

@@ -10,16 +10,18 @@
 
 > ⚠️ **Status: Beta** — Actively used in production (medical RAG & enterprise docs), API stable, breaking changes documented.
 
+> ⭐ **If this project helps you build better RAG systems, consider giving it a star!** It helps others discover this project.
+
 ---
 
 ## ⚡ 60-Second Quick Start
 
 ```typescript
-import { ContextRAG } from '@msbayindir/context-rag';
+import { createContextRAG } from '@msbayindir/context-rag';
 import { PrismaClient } from '@prisma/client';
 import * as fs from 'fs';
 
-const rag = new ContextRAG({
+const rag = createContextRAG({
   prisma: new PrismaClient(),
   geminiApiKey: process.env.GEMINI_API_KEY!,
 });
@@ -70,6 +72,7 @@ console.log(results[0].chunk.displayContent);
 | 🐘 **PostgreSQL Native** | No external vector DB needed, uses pgvector |
 | ⚡ **Batch Processing** | Concurrent processing with automatic retry |
 | 🛡️ **Enterprise Error Handling** | Correlation IDs, graceful degradation, structured logging |
+| 🔌 **Dependency Injection** | SOLID-compliant architecture with interface-based DI (v2.0-beta) |
 
 ---
 
@@ -165,7 +168,7 @@ flowchart TB
 **Scenario:** Turkish medical students preparing for TUS exam with 500+ page biochemistry PDFs.
 
 ```typescript
-const rag = new ContextRAG({
+const rag = createContextRAG({
   prisma,
   geminiApiKey: process.env.GEMINI_API_KEY!,
   ragEnhancement: {
@@ -176,13 +179,14 @@ const rag = new ContextRAG({
 });
 
 // Discovery: AI analyzes the PDF and suggests extraction strategy
-const discovery = await rag.discover({ file: pdfBuffer, filename: 'biochemistry.pdf' });
+const discovery = await rag.discover({ file: pdfBuffer });
 
-// Ingest with discovered strategy
+// Ingest with approved strategy
+const approved = await rag.approveStrategy(discovery.id);
 await rag.ingest({
   file: pdfBuffer,
   filename: 'biochemistry.pdf',
-  promptConfig: discovery.promptConfig,  // AI-suggested prompts
+  promptConfigId: approved.id,
 });
 
 // Students can now ask contextual questions
@@ -227,11 +231,12 @@ const liabilityClauses = await rag.search({
 ```typescript
 // Process multiple document types
 for (const doc of ['hr-policy.pdf', 'security-guidelines.pdf', 'api-docs.pdf']) {
-  const discovery = await rag.discover({ file: docs[doc], filename: doc });
+  const discovery = await rag.discover({ file: docs[doc] });
+  const approved = await rag.approveStrategy(discovery.id);
   await rag.ingest({
     file: docs[doc],
     filename: doc,
-    promptConfig: discovery.promptConfig,
+    promptConfigId: approved.id,
     experimentId: 'knowledge-base-v1',  // Group related documents
   });
 }
@@ -324,6 +329,12 @@ npx @msbayindir/context-rag init --force
 
 # Check setup status (Prisma models, pgvector, env variables)
 npx @msbayindir/context-rag status
+
+# Check for embedding model mismatches
+npx @msbayindir/context-rag check-embeddings
+
+# Re-index documents (useful after changing embedding models)
+npx @msbayindir/context-rag reindex --concurrency 5
 ```
 
 ---
@@ -399,12 +410,12 @@ COHERE_API_KEY="your-cohere-api-key"
 ## 🧩 Usage (Full Example)
 
 ```typescript
-import { ContextRAG } from '@msbayindir/context-rag';
+import { createContextRAG } from '@msbayindir/context-rag';
 import { PrismaClient } from '@prisma/client';
 
 const prisma = new PrismaClient();
 
-const rag = new ContextRAG({
+const rag = createContextRAG({
   prisma,
   geminiApiKey: process.env.GEMINI_API_KEY!,
   model: 'gemini-3-flash-preview',
@@ -463,7 +474,7 @@ the Cyanide test value for patient Ahmet Yılmaz. Value: 50 mg/dL"
 ### Configuration
 
 ```typescript
-const rag = new ContextRAG({
+const rag = createContextRAG({
   // ...
   ragEnhancement: {
     approach: 'anthropic_contextual',
@@ -496,7 +507,7 @@ Reranking improves search relevance by re-scoring candidates using AI models. Ba
 ### Configuration
 
 ```typescript
-const rag = new ContextRAG({
+const rag = createContextRAG({
   prisma,
   geminiApiKey: process.env.GEMINI_API_KEY!,
   
@@ -565,7 +576,7 @@ const result = await rag.ingest({
 ### Configuration for Selective Context Enrichment
 
 ```typescript
-const rag = new ContextRAG({
+const rag = createContextRAG({
   prisma,
   geminiApiKey: process.env.GEMINI_API_KEY!,
   
@@ -593,35 +604,151 @@ await rag.ingest({
 
 ## ⚙️ Configuration
 
+
+Context-RAG is highly configurable. Below is the complete list of all available options.
+
 ```typescript
-const rag = new ContextRAG({
-  // Required
+const rag = createContextRAG({
+  // ============================================
+  // CORE CONFIGURATION (Required)
+  // ============================================
+
+  /** Your initialized Prisma client instance */
   prisma: prismaClient,
-  geminiApiKey: 'your-api-key',
 
-  // Model selection
-  model: 'gemini-3-flash-preview',
-  embeddingModel: 'gemini-embedding-exp-03-07',
+  /** Gemini API Key (Required for generation and default embeddings) */
+  geminiApiKey: process.env.GEMINI_API_KEY!,
+
+  // ============================================
+  // MODEL SELECTION
+  // ============================================
 
-  // Generation
+  /** 
+   * Main LLM model for generation, orchestration, and RAG enhancement.
+   * Default: 'gemini-1.5-pro'
+   */
+  model: 'gemini-1.5-pro', // Options: 'gemini-1.5-flash', 'gemini-2.0-flash-exp', etc.
+
+  /**
+   * Configuration for the LLM generation (temperature, tokens, etc.)
+   */
   generationConfig: {
-    temperature: 0.2,
-    maxOutputTokens: 16384,
+    temperature: 0.3,        // Creativity (0.0 - 1.0). Lower is more deterministic.
+    maxOutputTokens: 8192,   // Maximum length of the generated response.
   },
 
-  // Batch processing
+  // ============================================
+  // EMBEDDING PROVIDER (Optional)
+  // ============================================
+
+  /**
+   * Choose your embedding provider.
+   * Default: Uses Gemini 'text-embedding-004'
+   */
+  embeddingProvider: {
+    // Provider: 'gemini' | 'openai' | 'cohere'
+    provider: 'openai', 
+
+    // Model name (specific to the provider)
+    model: 'text-embedding-3-small',
+
+    // API Key (if different from geminiApiKey)
+    apiKey: process.env.OPENAI_API_KEY,
+  },
+
+  // ============================================
+  // SYSTEM CONFIGURATION
+  // ============================================
+
+  /**
+   * Batch processing settings for ingestion.
+   * Adjust these based on your API rate limits.
+   */
   batchConfig: {
-    pagesPerBatch: 15,
-    maxConcurrency: 3,
-    maxRetries: 3,
+    pagesPerBatch: 15,       // How many pages to process in one go (Default: 15)
+    maxConcurrency: 3,       // How many batches to run in parallel (Default: 3)
+    maxRetries: 3,           // Retry failed batches (Default: 3)
+    retryDelayMs: 1000,      // Initial delay before retry (Default: 1000ms)
+    backoffMultiplier: 2,    // Exponential backoff factor (Default: 2)
+  },
+
+  /**
+   * Settings for splitting text into vector chunks.
+   */
+  chunkConfig: {
+    maxTokens: 500,          // Maximum size of a single chunk (Default: 500)
+    overlapTokens: 50,       // Overlap between chunks to preserve continuity (Default: 50)
+  },
+
+  /**
+   * API Rate Limiting protection.
+   */
+  rateLimitConfig: {
+    requestsPerMinute: 60,   // Max RPM allowed (Default: 60)
+    adaptive: true,          // Automatically slow down if 429 errors occur (Default: true)
+  },
+
+  /**
+   * System logging configuration.
+   */
+  logging: {
+    level: 'info',           // 'debug' | 'info' | 'warn' | 'error'
+    structured: true,        // Use JSON format for logs (Best for production tools like Datadog/CloudWatch)
+  },
+
+  // ============================================
+  // ADVANCED FEATURES
+  // ============================================
+
+  /**
+   * Reranking improves search relevance by re-scoring results.
+   */
+  rerankingConfig: {
+    enabled: true,           // Enable automatic reranking (Default: false)
+    provider: 'cohere',      // 'gemini' or 'cohere' (Cohere is recommended for best results)
+    cohereApiKey: process.env.COHERE_API_KEY, // Required if provider is 'cohere'
+    defaultCandidates: 50,   // Retrieve top 50 from Vector DB...
+    defaultTopK: 10,         // ...and return top 10 after reranking.
   },
 
-  // RAG Enhancement
+  /**
+   * RAG Enhancement (Contextual Retrieval).
+   * Adds context to chunks before embedding them.
+   */
   ragEnhancement: {
+    // Approach: 'anthropic_contextual' (Recommended) or 'none'
     approach: 'anthropic_contextual',
-    strategy: 'simple',
-    skipChunkTypes: ['HEADING'],
+
+    // Strategy: 'llm' (Best Quality) or 'simple' (Template based)
+    strategy: 'llm',
+
+    // Model to use for generating context (Optional, defaults to main model)
+    // Tip: Use a cheaper model here (e.g., 'gemini-1.5-flash') to save costs.
+    model: 'gemini-1.5-flash',
+
+    // Prompt used to generate context (Optional, has good default)
+    contextPrompt: 'Situate this chunk within the document...',
+
+    // Don't waste tokens generating context for these types
+    skipChunkTypes: ['HEADING', 'IMAGE_REF', 'CODE'], 
   },
+
+  /** 
+   * Enable Structured Output (JSON Schema) for reliable parsing.
+   * Disable only if you are using a model that doesn't support it well.
+   * Default: true 
+   */
+  useStructuredOutput: true,
+
+  /**
+   * Custom Chunk Type Mapping.
+   * Map your custom extraction types to system types for proper handling.
+   */
+  chunkTypeMapping: {
+    'RECIPE': 'TEXT',        // Treat 'RECIPE' as normal text
+    'INGREDIENT_LIST': 'LIST', // Treat 'INGREDIENT_LIST' as a list
+    'NUTRITIONAL_INFO': 'TABLE' // Treat 'NUTRITIONAL_INFO' as a table
+  }
 });
 ```
 
@@ -765,22 +892,77 @@ We use [Conventional Commits](https://www.conventionalcommits.org/):
 context-rag/
 ├── src/
 │   ├── context-rag.ts       # Main facade class
+│   ├── context-rag.factory.ts # DI Factory (v2.0-beta)
 │   ├── engines/             # Discovery, Ingestion, Retrieval
 │   ├── enhancements/        # RAG Enhancement handlers
 │   │   └── anthropic/       # Anthropic Contextual Retrieval
 │   ├── services/            # Gemini API, PDF Processor
+│   ├── providers/           # Embedding providers (Gemini, OpenAI, Cohere)
 │   ├── database/            # Prisma repositories
-│   ├── config/              # Templates
-│   ├── types/               # TypeScript types
+│   ├── config/              # Templates & constants
+│   ├── types/               # TypeScript types & interfaces
 │   ├── utils/               # Logger, Retry, RateLimiter
 │   └── errors/              # Custom error classes
 ├── examples/                # Demo scripts
+├── tests/                   # Unit & integration tests
 ├── prisma/                  # Reference schema
 └── dist/                    # Built output
 ```
 
 ---
 
+## 🔄 Migration Guide (v1.x → v2.0-beta)
+
+### Breaking Change: Factory Pattern
+
+v2.0-beta introduces proper Dependency Injection. The `new ContextRAG()` constructor now requires dependencies.
+
+**Before (v1.x):**
+```typescript
+import { ContextRAG } from '@msbayindir/context-rag';
+
+const rag = new ContextRAG({
+  prisma,
+  geminiApiKey: 'your-key',
+});
+```
+
+**After (v2.0-beta):**
+```typescript
+import { createContextRAG } from '@msbayindir/context-rag';
+
+const rag = createContextRAG({
+  prisma,
+  geminiApiKey: 'your-key',
+});
+```
+
+### Custom Engine Injection (Advanced)
+
+v2.0-beta allows injecting custom engines for advanced use cases:
+
+```typescript
+import { ContextRAG, IngestionEngine } from '@msbayindir/context-rag';
+
+// Create custom engine
+class MyIngestionEngine extends IngestionEngine {
+  async ingest(options) {
+    console.log('Custom logic!');
+    return super.ingest(options);
+  }
+}
+
+// Inject via constructor
+const rag = new ContextRAG(config, {
+  ingestionEngine: myCustomEngine,
+  retrievalEngine,
+  discoveryEngine,
+  repos: { promptConfig, document, chunk },
+});
+```
+
+---
+
 ## 📄 License
 
 MIT © [Muhammed Bayindir](https://github.com/msbayindir)