@devilsdev/rag-pipeline-utils 2.1.5 → 2.1.8

package/docs-site/docs/Evaluation.md

@@ -1,61 +0,0 @@
-# Evaluation
-
-This module allows you to evaluate the quality of generated LLM responses in a RAG pipeline.
-
----
-
-## Objective
-
-Quantify how well the pipeline performs on question-answer pairs using:
-- **BLEU**: Precision on matching tokens
-- **ROUGE**: Recall + overlap on longest subsequences
-- **LLM Feedback** *(planned)*: Score generation based on LLM ratings
-
----
-
-## Dataset Format
-
-Each entry in your JSON dataset should include:
-
-```json
-{
-  "prompt": "What is a vector index?",
-  "groundTruth": "A data structure for fast similarity search in vector spaces."
-}
-```
-
-Saved as: `sample-eval-dataset.json`
-
----
-
-## CLI Evaluation
-
-```bash
-rag-pipeline evaluate ./fixtures/sample-eval-dataset.json
-```
-
-Outputs:
-- Pass/fail per item
-- BLEU/ROUGE scores
-- Aggregated metrics
-
----
-
-## Dashboard Visualization
-
-Use the React dashboard under `/public`:
-
-```bash
-node server.js
-```
-
-Visit: [http://localhost:3000](http://localhost:3000)
-
-Displays:
-- Prompt vs Answer
-- Success criteria
-- Metric charts (BLEU, ROUGE)
-
----
-
-Next → [Plugins](./Plugins.md)

package/docs-site/docs/FAQ.md

@@ -1,50 +0,0 @@
-# FAQ
-
-### Can I use this with my own vector database?
-Yes! You can implement the `Retriever` interface and register it:
-```ts
-registry.register('retriever', 'mydb', new MyRetriever());
-```
-Then use it via CLI or `.ragrc.json`.
-
----
-
-### Is streaming LLM response supported?
-Yes. LLM runners can yield async output tokens. This is handled internally via `AsyncIterable` in compatible models.
-
----
-
-### How do I evaluate performance?
-Use `rag-pipeline evaluate dataset.json` or open the dashboard at `http://localhost:3000` to visualize BLEU/ROUGE and pass rates.
-
----
-
-### What file types can I ingest?
-Currently supported:
-- `.pdf` (mocked)
-- `.md`
-- `.html`
-- `.csv`
-- Full directory ingestion
-
----
-
-### Can I rerank results using a local model?
-Yes, as long as your local LLM implements:
-```ts
-llm.generate(prompt, context): Promise<string>
-```
-Use it to power `LLMReranker`.
-
----
-
-### How do I contribute?
-Fork the repo → add your plugin or CLI feature → test → submit PR.
-
----
-
-Still stuck? Open an issue at [GitHub](https://github.com/DevilsDev/rag-pipeline-utils/issues).
-
----
-
-Next → [Back to Introduction](./Introduction.md)

package/docs-site/docs/Introduction.md

@@ -1,58 +0,0 @@
----
-sidebar_position: 1
----
-
-# Introduction
-
-Welcome to **@DevilsDev/rag-pipeline-utils**, a modular Node.js toolkit for building scalable, pluggable Retrieval-Augmented Generation (RAG) systems.
-
----
-
-##  What is RAG?
-
-RAG combines information retrieval with large language models to enhance generation with factual grounding. It typically involves:
-
-1. **Retrieving** relevant context (documents, chunks) from a vector store
-2. **Generating** answers using an LLM, augmented by that context
-
-This project provides all the composable parts to build your own RAG pipeline.
-
----
-
-##  Why Use This Toolkit?
-
-- **Plugin-based**: Swap in your own loaders, retrievers, LLMs
-- **Streaming-ready**: Async-friendly, token-by-token output
-- **CLI & API**: Use from terminal or integrate programmatically
-- **Evaluation built-in**: BLEU, ROUGE, dashboard UI
-- **Enterprise-quality**: SOLID, tested, typed, and CI-ready
-
----
-
-## Architecture
-
-Each RAG pipeline includes:
-
-- `loader`: Document loader (e.g., PDF, Markdown, HTML)
-- `embedder`: Embedding model interface (OpenAI, Cohere, local)
-- `retriever`: Vector store adapter (e.g., Pinecone, Chroma)
-- `llm`: Language model runner (OpenAI, Ollama, GPT-4-V)
-- `reranker`: (Optional) Context re-ranking module via LLM
-
-The pipeline can be used via code or via CLI commands.
-
----
-
-## Status
-
-This project is:
-- Production-ready
-- Covered by unit/integration tests
-- Compatible with Node.js `>=18`
-- Published to npm under `DevilsDev`
-
----
-
-Let’s get started!
-
-Next → [Usage](./Usage.md)

package/docs-site/docs/Plugins.md

@@ -1,61 +0,0 @@
-# Plugins
-
-The toolkit is fully modular. You can inject your own loaders, retrievers, embedders, or LLMs.
-
----
-
-## Plugin Architecture
-
-All plugins follow a strict interface pattern. Each is registered under a namespace using the plugin registry:
-
-```ts
-registry.register('loader', 'my-custom', new MyLoader());
-const loader = registry.get('loader', 'my-custom');
-```
-
-You can register plugins for:
-- `loader`
-- `embedder`
-- `retriever`
-- `llm`
-
----
-
-## Loader Example
-
-```ts
-class MyMarkdownLoader {
-  async load(filePath) {
-    const raw = fs.readFileSync(filePath, 'utf-8');
-    return [{ chunk: () => raw.split('\n\n') }];
-  }
-}
-```
-
-Register it:
-```ts
-registry.register('loader', 'md', new MyMarkdownLoader());
-```
-
----
-
-## Runtime Override
-
-CLI and API will resolve plugin names from `pluginRegistry`. Use:
-
-```json
-{
-  "loader": "md",
-  "embedder": "cohere",
-  "retriever": "chroma",
-  "llm": "openai-gpt-4"
-}
-```
-
-in `.ragrc.json`
-
----
-
-This design keeps the core minimal and lets you evolve.
-
-Next → [FAQ](./FAQ.md)

package/docs-site/docs/Tutorials.md

@@ -1,106 +0,0 @@
----
-id: Tutorials
-title: Tutorials & Onboarding
-sidebar_label: Tutorials
----
-
-Welcome to the **RAG Pipeline Utils** tutorial section. These guides will help developers quickly onboard, integrate, and extend the modular RAG system.
-
----
-
-##  1. Install the Package
-
-```bash
-npm install @devilsdev/rag-pipeline-utils
-```
-
----
-
-## 2. Ingest Documents Using CLI
-
-This CLI command loads PDFs, embeds them, and stores the vectors.
-
-```bash
-npx rag-utils ingest ./sample.pdf --config .ragrc.json
-```
-
-Ensure you have a valid `.ragrc.json` config:
-
-```json
-{
-  "loader": { "pdf": "./src/mocks/pdf-loader.js" },
-  "embedder": { "openai": "./src/mocks/openai-embedder.js" },
-  "retriever": { "pinecone": "./src/mocks/pinecone-retriever.js" },
-  "llm": { "openai": "./src/mocks/openai-llm.js" },
-  "namespace": "docs-example",
-  "pipeline": ["loader", "embedder", "retriever"]
-}
-```
-
----
-
-## 3. Run a Query Using CLI
-
-```bash
-npx rag-utils query "What is this document about?" --config .ragrc.json
-```
-
----
-
-## 4. Register a Custom Plugin
-
-You can create your own loader/retriever/LLM:
-
-```js
-// src/plugins/my-custom-llm.js
-export default class MyCustomLLM {
-  ask(prompt) {
-    return `Custom response to: ${prompt}`;
-  }
-}
-```
-
-Then reference it in `.ragrc.json`:
-
-```json
-"llm": {
-  "custom": "./src/plugins/my-custom-llm.js"
-}
-```
-
----
-
-## 5. Validate Plugin Mocks
-
-Ensure all mocks meet contract interfaces before commit:
-
-```bash
-npm run validate-fixtures
-```
-
----
-
-## 6. Run All Tests + Coverage
-
-```bash
-npm run ci
-```
-
-This runs:
-
--  Linting
--  Fixture validation
--  Unit/integration tests
--  Coverage reporting
-
----
-
-## Next Steps
-
-- Explore `/src/core/plugin-registry.js` for plugin orchestration
-- Browse `/scripts/` for CI and release utilities
-- View `/__tests__/` for end-to-end RAG validation flows
-
----
-
-Looking to build on top of this? Check out [Use Cases](./Use-Cases.md) for advanced scenarios.

package/docs-site/docs/Usage.md

@@ -1,72 +0,0 @@
-# Usage
-
-This guide covers how to use `@yourorg/rag-pipeline-utils` programmatically via API and through the CLI.
-
----
-
-## Installation
-
-```bash
-npm install @yourorg/rag-pipeline-utils
-```
-
----
-
-## Programmatic API
-
-Import and instantiate a RAG pipeline:
-
-```js
-import { createRagPipeline } from '@yourorg/rag-pipeline-utils';
-
-const pipeline = createRagPipeline({
-  loader: 'markdown',
-  embedder: 'openai',
-  retriever: 'pinecone',
-  llm: 'openai-gpt-4',
-  useReranker: true
-});
-
-const answer = await pipeline.query("What is retrieval-augmented generation?");
-console.log(answer);
-```
-
----
-
-##  Pipeline Methods
-
-- `pipeline.ingest(path: string)` → Load, chunk, embed, and store vectors
-- `pipeline.query(prompt: string)` → Retrieve context, rerank (optional), and call LLM
-
----
-
-##  Plugin Configuration
-
-You can customize the pipeline by registering plugins:
-
-```js
-registry.register('loader', 'custom', new MyCustomLoader());
-registry.register('retriever', 'opensearch', new MyOpenSearchRetriever());
-```
-
----
-
-## Configuration via `.ragrc.json`
-
-Create a JSON file at your root:
-
-```json
-{
-  "loader": "directory",
-  "embedder": "openai",
-  "retriever": "pinecone",
-  "llm": "openai-gpt-4",
-  "useReranker": true
-}
-```
-
-Used automatically when no CLI args are passed.
-
----
-
-Next → [CLI](./CLI.md)

package/docs-site/docs/Use-Cases.md

@@ -1,77 +0,0 @@
----
-id: Use-Cases
-title: Real-World Use Cases
-sidebar_position: 5
----
-
-##  Real-World Applications of RAG Pipeline Utils
-
-This project goes beyond traditional RAG tools — it’s a **developer-focused modular framework**. Here's how it’s used:
-
----
-
-### 1. Customizable LLM Workflows
-
-**Use Case:** A team wants to test three different retrievers (Pinecone, Weaviate, Redis) and switch LLMs dynamically during eval.
-
-```bash
-rag-utils ingest sample.pdf --retriever pinecone --llm openai
-```
-
----
-
-### 2. Plugin-Based Evaluation Benchmarks
-
-**Use Case:** You want to run BLEU/ROUGE scoring across prompt templates or documents using CLI:
-
-```bash
-rag-utils evaluate --dataset tests/eval.json --llm anthropic
-```
-
----
-
-### 3. Internal LLM System for SaaS
-
-**Use Case:** Embed RAG processing into a backend:
-
-```js
-import { PluginRegistry, runPipeline } from 'rag-pipeline-utils';
-
-const registry = new PluginRegistry();
-registry.register('embedder', 'openai', new OpenAIEmbedder());
-
-const output = await runPipeline({
-  loader: 'pdf',
-  retriever: 'pinecone',
-  llm: 'openai',
-  query: 'How does this work?'
-});
-```
-
----
-
-### 4. GitHub + NPM Automation for ML Pipelines
-
-**Use Case:** You want a release blog post + versioned package published automatically:
-
-- Commit code
-- Push to `main`
-- GitHub Action triggers:
-  - Semantic release
-  - CHANGELOG update
-  - Blog post generation
-  - NPM publish
-
----
-
-###  Benefits
-
-- **Pluggable** components via clean interfaces
-- **CLI + programmatic** access for flexible DX
-- **CI-validated** plugin contract enforcement
-- **Docs-first** developer onboarding
-- **Production-ready** for real ML teams
-
----
-
-> Want to contribute your use case? PRs welcome on [GitHub](https://github.com/DevilsDev/rag-pipeline-utils).

package/docs-site/docs/intro.mdx

@@ -1,6 +0,0 @@
----
-id: intro
-title: Introduction
----
-
-Welcome to RAG Pipeline Utils! This is the starting point for your documentation.

package/docs-site/docusaurus.config.js

@@ -1,88 +0,0 @@
-/**
- * Version: 0.4.0
- * Path: /docs-site/docusaurus.config.js
- * Description: Tailwind integration, sticky navbar, and dark/light toggle support
- * Author: Ali Kahwaji
- */
-
-// @ts-check
-
-export default {
-  title: 'RAG Pipeline Utils',
-  tagline: 'Composable pipelines for LLMs',
-  url: 'https://devilsdev.github.io',
-  baseUrl: '/rag-pipeline-utils/',
-  deploymentBranch: 'gh-pages',
-  onBrokenLinks: 'warn',
-  onBrokenMarkdownLinks: 'warn',
-
-  favicon: 'img/favicon.ico',
-
-  themeConfig: {
-    metadata: [
-      { name: 'keywords', content: 'rag, llm, langchain, pipeline, openai, nodejs' },
-      { name: 'author', content: 'Ali Kahwaji' },
-      { name: 'theme-color', content: '#3EF4B6' }
-    ],
-    // ✅ OpenGraph / Twitter image
-    image: 'img/social-card.jpg',
-
-    // ✅ Color mode toggle support
-    colorMode: {
-      defaultMode: 'light',
-      disableSwitch: false,
-      respectPrefersColorScheme: true
-    },
-
-    // ✅ Sticky header
-    navbar: {
-      title: 'RAG Utils',
-      hideOnScroll: false,
-      logo: {
-        alt: '',
-        src: '/img/logo.svg'
-      },
-      items: [
-        { to: '/docs/Introduction', label: 'Docs', position: 'left' },
-        { to: '/blog', label: 'Blog', position: 'left' },
-        {
-          href: 'https://github.com/DevilsDev/rag-pipeline-utils',
-          label: 'GitHub',
-          position: 'right'
-        }
-      ]
-    },
-
-    footer: {
-      style: 'dark',
-      copyright: `© ${new Date().getFullYear()} Ali Kahwaji`
-    }
-  },
-
-  organizationName: 'DevilsDev',
-  projectName: 'rag-pipeline-utils',
-
-  presets: [
-    [
-      'classic',
-      {
-        docs: {
-          sidebarPath: require.resolve('./sidebars.js'),
-          editUrl:
-            'https://github.com/DevilsDev/rag-pipeline-utils/edit/main/docs-site/'
-        },
-        blog: {
-          showReadingTime: true,
-          blogSidebarTitle: 'All Posts',
-          blogSidebarCount: 'ALL',
-          onUntruncatedBlogPosts: 'ignore',
-          editUrl:
-            'https://github.com/DevilsDev/rag-pipeline-utils/edit/main/docs-site/blog/'
-        },
-        theme: {
-          customCss: require.resolve('./src/css/custom.css')
-        }
-      }
-    ]
-  ]
-};