@devilsdev/rag-pipeline-utils 1.0.0 → 1.0.1

package/docs-site/docs/Evaluation.md

@@ -1,61 +1,61 @@
-# Evaluation
-
-This module allows you to evaluate the quality of generated LLM responses in a RAG pipeline.
-
----
-
-## 🎯 Objective
-
-Quantify how well the pipeline performs on question-answer pairs using:
-- **BLEU**: Precision on matching tokens
-- **ROUGE**: Recall + overlap on longest subsequences
-- **LLM Feedback** *(planned)*: Score generation based on LLM ratings
-
----
-
-## 📄 Dataset Format
-
-Each entry in your JSON dataset should include:
-
-```json
-{
-  "prompt": "What is a vector index?",
-  "groundTruth": "A data structure for fast similarity search in vector spaces."
-}
-```
-
-Saved as: `sample-eval-dataset.json`
-
----
-
-## 🚀 CLI Evaluation
-
-```bash
-rag-pipeline evaluate ./fixtures/sample-eval-dataset.json
-```
-
-Outputs:
-- ✅ Pass/fail per item
-- 📊 BLEU/ROUGE scores
-- 📈 Aggregated metrics
-
----
-
-## 📊 Dashboard Visualization
-
-Use the React dashboard under `/public`:
-
-```bash
-node server.js
-```
-
-Visit: [http://localhost:3000](http://localhost:3000)
-
-Displays:
-- Prompt vs Answer
-- Success criteria
-- Metric charts (BLEU, ROUGE)
-
----
-
-Next → [Plugins](./Plugins.md)
+# Evaluation
+
+This module allows you to evaluate the quality of generated LLM responses in a RAG pipeline.
+
+---
+
+## 🎯 Objective
+
+Quantify how well the pipeline performs on question-answer pairs using:
+- **BLEU**: Precision on matching tokens
+- **ROUGE**: Recall + overlap on longest subsequences
+- **LLM Feedback** *(planned)*: Score generation based on LLM ratings
+
+---
+
+## 📄 Dataset Format
+
+Each entry in your JSON dataset should include:
+
+```json
+{
+  "prompt": "What is a vector index?",
+  "groundTruth": "A data structure for fast similarity search in vector spaces."
+}
+```
+
+Saved as: `sample-eval-dataset.json`
+
+---
+
+## 🚀 CLI Evaluation
+
+```bash
+rag-pipeline evaluate ./fixtures/sample-eval-dataset.json
+```
+
+Outputs:
+- ✅ Pass/fail per item
+- 📊 BLEU/ROUGE scores
+- 📈 Aggregated metrics
+
+---
+
+## 📊 Dashboard Visualization
+
+Use the React dashboard under `/public`:
+
+```bash
+node server.js
+```
+
+Visit: [http://localhost:3000](http://localhost:3000)
+
+Displays:
+- Prompt vs Answer
+- Success criteria
+- Metric charts (BLEU, ROUGE)
+
+---
+
+Next → [Plugins](./Plugins.md)

package/docs-site/docs/FAQ.md

@@ -1,50 +1,50 @@
-# FAQ
-
-### ❓ Can I use this with my own vector database?
-Yes! You can implement the `Retriever` interface and register it:
-```ts
-registry.register('retriever', 'mydb', new MyRetriever());
-```
-Then use it via CLI or `.ragrc.json`.
-
----
-
-### ❓ Is streaming LLM response supported?
-Yes. LLM runners can yield async output tokens. This is handled internally via `AsyncIterable` in compatible models.
-
----
-
-### ❓ How do I evaluate performance?
-Use `rag-pipeline evaluate dataset.json` or open the dashboard at `http://localhost:3000` to visualize BLEU/ROUGE and pass rates.
-
----
-
-### ❓ What file types can I ingest?
-Currently supported:
-- `.pdf` (mocked)
-- `.md`
-- `.html`
-- `.csv`
-- Full directory ingestion
-
----
-
-### ❓ Can I rerank results using a local model?
-Yes, as long as your local LLM implements:
-```ts
-llm.generate(prompt, context): Promise<string>
-```
-Use it to power `LLMReranker`.
-
----
-
-### ❓ How do I contribute?
-Fork the repo → add your plugin or CLI feature → test → submit PR.
-
----
-
-Still stuck? Open an issue at [GitHub](https://github.com/DevilsDev/rag-pipeline-utils/issues).
-
----
-
-Next → [Back to Introduction](./Introduction.md)
+# FAQ
+
+### ❓ Can I use this with my own vector database?
+Yes! You can implement the `Retriever` interface and register it:
+```ts
+registry.register('retriever', 'mydb', new MyRetriever());
+```
+Then use it via CLI or `.ragrc.json`.
+
+---
+
+### ❓ Is streaming LLM response supported?
+Yes. LLM runners can yield async output tokens. This is handled internally via `AsyncIterable` in compatible models.
+
+---
+
+### ❓ How do I evaluate performance?
+Use `rag-pipeline evaluate dataset.json` or open the dashboard at `http://localhost:3000` to visualize BLEU/ROUGE and pass rates.
+
+---
+
+### ❓ What file types can I ingest?
+Currently supported:
+- `.pdf` (mocked)
+- `.md`
+- `.html`
+- `.csv`
+- Full directory ingestion
+
+---
+
+### ❓ Can I rerank results using a local model?
+Yes, as long as your local LLM implements:
+```ts
+llm.generate(prompt, context): Promise<string>
+```
+Use it to power `LLMReranker`.
+
+---
+
+### ❓ How do I contribute?
+Fork the repo → add your plugin or CLI feature → test → submit PR.
+
+---
+
+Still stuck? Open an issue at [GitHub](https://github.com/DevilsDev/rag-pipeline-utils/issues).
+
+---
+
+Next → [Back to Introduction](./Introduction.md)

package/docs-site/docs/Plugins.md

@@ -1,61 +1,61 @@
-# Plugins
-
-The toolkit is fully modular. You can inject your own loaders, retrievers, embedders, or LLMs.
-
----
-
-## 🔌 Plugin Architecture
-
-All plugins follow a strict interface pattern. Each is registered under a namespace using the plugin registry:
-
-```ts
-registry.register('loader', 'my-custom', new MyLoader());
-const loader = registry.get('loader', 'my-custom');
-```
-
-You can register plugins for:
-- `loader`
-- `embedder`
-- `retriever`
-- `llm`
-
----
-
-## 🧱 Loader Example
-
-```ts
-class MyMarkdownLoader {
-  async load(filePath) {
-    const raw = fs.readFileSync(filePath, 'utf-8');
-    return [{ chunk: () => raw.split('\n\n') }];
-  }
-}
-```
-
-Register it:
-```ts
-registry.register('loader', 'md', new MyMarkdownLoader());
-```
-
----
-
-## ⚡ Runtime Override
-
-CLI and API will resolve plugin names from `pluginRegistry`. Use:
-
-```json
-{
-  "loader": "md",
-  "embedder": "cohere",
-  "retriever": "chroma",
-  "llm": "openai-gpt-4"
-}
-```
-
-in `.ragrc.json`
-
----
-
-This design keeps the core minimal and lets you evolve.
-
-Next → [FAQ](./FAQ.md)
+# Plugins
+
+The toolkit is fully modular. You can inject your own loaders, retrievers, embedders, or LLMs.
+
+---
+
+## 🔌 Plugin Architecture
+
+All plugins follow a strict interface pattern. Each is registered under a namespace using the plugin registry:
+
+```ts
+registry.register('loader', 'my-custom', new MyLoader());
+const loader = registry.get('loader', 'my-custom');
+```
+
+You can register plugins for:
+- `loader`
+- `embedder`
+- `retriever`
+- `llm`
+
+---
+
+## 🧱 Loader Example
+
+```ts
+class MyMarkdownLoader {
+  async load(filePath) {
+    const raw = fs.readFileSync(filePath, 'utf-8');
+    return [{ chunk: () => raw.split('\n\n') }];
+  }
+}
+```
+
+Register it:
+```ts
+registry.register('loader', 'md', new MyMarkdownLoader());
+```
+
+---
+
+## ⚡ Runtime Override
+
+CLI and API will resolve plugin names from `pluginRegistry`. Use:
+
+```json
+{
+  "loader": "md",
+  "embedder": "cohere",
+  "retriever": "chroma",
+  "llm": "openai-gpt-4"
+}
+```
+
+in `.ragrc.json`
+
+---
+
+This design keeps the core minimal and lets you evolve.
+
+Next → [FAQ](./FAQ.md)

package/docs-site/docs/Usage.md

@@ -1,72 +1,72 @@
-# Usage
-
-This guide covers how to use `@yourorg/rag-pipeline-utils` programmatically via API and through the CLI.
-
----
-
-## 📦 Installation
-
-```bash
-npm install @yourorg/rag-pipeline-utils
-```
-
----
-
-## 🔧 Programmatic API
-
-Import and instantiate a RAG pipeline:
-
-```js
-import { createRagPipeline } from '@yourorg/rag-pipeline-utils';
-
-const pipeline = createRagPipeline({
-  loader: 'markdown',
-  embedder: 'openai',
-  retriever: 'pinecone',
-  llm: 'openai-gpt-4',
-  useReranker: true
-});
-
-const answer = await pipeline.query("What is retrieval-augmented generation?");
-console.log(answer);
-```
-
----
-
-## 🔁 Pipeline Methods
-
-- `pipeline.ingest(path: string)` → Load, chunk, embed, and store vectors
-- `pipeline.query(prompt: string)` → Retrieve context, rerank (optional), and call LLM
-
----
-
-## 🧩 Plugin Configuration
-
-You can customize the pipeline by registering plugins:
-
-```js
-registry.register('loader', 'custom', new MyCustomLoader());
-registry.register('retriever', 'opensearch', new MyOpenSearchRetriever());
-```
-
----
-
-## ⚙️ Configuration via `.ragrc.json`
-
-Create a JSON file at your root:
-
-```json
-{
-  "loader": "directory",
-  "embedder": "openai",
-  "retriever": "pinecone",
-  "llm": "openai-gpt-4",
-  "useReranker": true
-}
-```
-
-Used automatically when no CLI args are passed.
-
----
-
-Next → [CLI](./CLI.md)
+# Usage
+
+This guide covers how to use `@yourorg/rag-pipeline-utils` programmatically via API and through the CLI.
+
+---
+
+## 📦 Installation
+
+```bash
+npm install @yourorg/rag-pipeline-utils
+```
+
+---
+
+## 🔧 Programmatic API
+
+Import and instantiate a RAG pipeline:
+
+```js
+import { createRagPipeline } from '@yourorg/rag-pipeline-utils';
+
+const pipeline = createRagPipeline({
+  loader: 'markdown',
+  embedder: 'openai',
+  retriever: 'pinecone',
+  llm: 'openai-gpt-4',
+  useReranker: true
+});
+
+const answer = await pipeline.query("What is retrieval-augmented generation?");
+console.log(answer);
+```
+
+---
+
+## 🔁 Pipeline Methods
+
+- `pipeline.ingest(path: string)` → Load, chunk, embed, and store vectors
+- `pipeline.query(prompt: string)` → Retrieve context, rerank (optional), and call LLM
+
+---
+
+## 🧩 Plugin Configuration
+
+You can customize the pipeline by registering plugins:
+
+```js
+registry.register('loader', 'custom', new MyCustomLoader());
+registry.register('retriever', 'opensearch', new MyOpenSearchRetriever());
+```
+
+---
+
+## ⚙️ Configuration via `.ragrc.json`
+
+Create a JSON file at your root:
+
+```json
+{
+  "loader": "directory",
+  "embedder": "openai",
+  "retriever": "pinecone",
+  "llm": "openai-gpt-4",
+  "useReranker": true
+}
+```
+
+Used automatically when no CLI args are passed.
+
+---
+
+Next → [CLI](./CLI.md)

package/jest.config.js

@@ -1,14 +1,21 @@
-/**
- * Version: 0.1.0
- * Path: /jest.config.js
- * Description: Jest configuration for native ESM test support
- * Author: Ali Kahwaji
- */
-
-export default {
-    testEnvironment: 'node',
-    transform: {}, // Native ESM — no Babel used
-    roots: ['<rootDir>/__tests__'],
-    verbose: true
-  };
+/**
+ * Version: 0.1.0
+ * Path: /jest.config.js
+ * Description: Jest configuration for native ESM test support
+ * Author: Ali Kahwaji
+ */
+
+// jest.config.js
+
+export default {
+  verbose: true,
+  testEnvironment: 'node',
+  transform: {},
+  collectCoverage: true,
+  coverageDirectory: 'coverage',
+  moduleFileExtensions: ['js', 'json'],
+  testMatch: ['**/__tests__/**/*.test.js'],
+  roots: ['<rootDir>/__tests__'],
+};
+
   

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@devilsdev/rag-pipeline-utils",
-  "version": "1.0.0",
+  "version": "1.0.1",
   "description": "A modular toolkit for building RAG (Retrieval-Augmented Generation) pipelines in Node.js",
   "main": "src/core/create-pipeline.js",
   "bin": {
@@ -8,7 +8,7 @@
   },
   "type": "module",
   "scripts": {
-    "test": "cross-env NODE_OPTIONS=--experimental-vm-modules jest --coverage",
+    "test": "node ./scripts/setup.js && cross-env NODE_OPTIONS=--experimental-vm-modules jest --coverage",
     "test:watch": "jest --watch",
     "lint": "eslint .",
     "prepare": "husky install"
@@ -37,9 +37,11 @@
   },
   "devDependencies": {
     "@semantic-release/changelog": "^6.0.3",
+    "@semantic-release/commit-analyzer": "^13.0.1",
     "@semantic-release/git": "^10.0.1",
     "@semantic-release/github": "^9.2.6",
     "@semantic-release/npm": "^11.0.3",
+    "@semantic-release/release-notes-generator": "^14.0.3",
     "@types/jest": "^29.5.14",
     "cross-env": "^7.0.3",
     "esbuild": "^0.25.2",

package/public/src/App.jsx

@@ -1,57 +1,57 @@
-/**
- * Version: 0.1.0
- * Path: /public/src/App.jsx
- * Description: Evaluation results dashboard UI
- * Author: Ali Kahwaji
- */
-
-import { useEffect, useState } from 'react';
-
-export default function App() {
-  const [results, setResults] = useState([]);
-  const [summary, setSummary] = useState({ passRate: 0, avgBLEU: 0, avgROUGE: 0 });
-
-  useEffect(() => {
-    fetch('/results/sample-eval.json')
-      .then(res => res.json())
-      .then(data => {
-        setResults(data);
-
-        const total = data.length;
-        const passCount = data.filter(r => r.success).length;
-        const avgBLEU = data.reduce((s, r) => s + r.scores.bleu, 0) / total;
-        const avgROUGE = data.reduce((s, r) => s + r.scores.rouge, 0) / total;
-
-        setSummary({
-          passRate: (passCount / total * 100).toFixed(1),
-          avgBLEU: avgBLEU.toFixed(2),
-          avgROUGE: avgROUGE.toFixed(2)
-        });
-      });
-  }, []);
-
-  return (
-    <main className="max-w-6xl mx-auto p-6">
-      <h1 className="text-3xl font-bold mb-4">RAG Evaluation Dashboard</h1>
-
-      <div className="mb-6 p-4 border rounded shadow bg-white">
-        <h2 className="text-xl font-semibold mb-2">Summary</h2>
-        <p> Pass Rate: <strong>{summary.passRate}%</strong></p>
-        <p> Avg BLEU: <strong>{summary.avgBLEU}</strong></p>
-        <p> Avg ROUGE: <strong>{summary.avgROUGE}</strong></p>
-      </div>
-
-      {results.map((res, i) => (
-        <div key={i} className="mb-4 p-4 border-l-4 rounded bg-white shadow border-l-gray-300">
-          <h3 className="font-bold text-lg">Prompt {i + 1}</h3>
-          <p className="mt-2"><strong>Prompt:</strong> {res.prompt}</p>
-          <p><strong>Expected:</strong> {res.expected}</p>
-          <p><strong>Actual:</strong> {res.actual}</p>
-          <p> BLEU: <strong>{res.scores.bleu.toFixed(2)}</strong></p>
-          <p> ROUGE: <strong>{res.scores.rouge.toFixed(2)}</strong></p>
-          <p>Pass: <strong className={res.success ? 'text-green-600' : 'text-red-600'}>{res.success ? 'Yes' : 'No'}</strong></p>
-        </div>
-      ))}
-    </main>
-  );
-}
+/**
+ * Version: 0.1.0
+ * Path: /public/src/App.jsx
+ * Description: Evaluation results dashboard UI
+ * Author: Ali Kahwaji
+ */
+
+import { useEffect, useState } from 'react';
+
+export default function App() {
+  const [results, setResults] = useState([]);
+  const [summary, setSummary] = useState({ passRate: 0, avgBLEU: 0, avgROUGE: 0 });
+
+  useEffect(() => {
+    fetch('/results/sample-eval.json')
+      .then(res => res.json())
+      .then(data => {
+        setResults(data);
+
+        const total = data.length;
+        const passCount = data.filter(r => r.success).length;
+        const avgBLEU = data.reduce((s, r) => s + r.scores.bleu, 0) / total;
+        const avgROUGE = data.reduce((s, r) => s + r.scores.rouge, 0) / total;
+
+        setSummary({
+          passRate: (passCount / total * 100).toFixed(1),
+          avgBLEU: avgBLEU.toFixed(2),
+          avgROUGE: avgROUGE.toFixed(2)
+        });
+      });
+  }, []);
+
+  return (
+    <main className="max-w-6xl mx-auto p-6">
+      <h1 className="text-3xl font-bold mb-4">RAG Evaluation Dashboard</h1>
+
+      <div className="mb-6 p-4 border rounded shadow bg-white">
+        <h2 className="text-xl font-semibold mb-2">Summary</h2>
+        <p> Pass Rate: <strong>{summary.passRate}%</strong></p>
+        <p> Avg BLEU: <strong>{summary.avgBLEU}</strong></p>
+        <p> Avg ROUGE: <strong>{summary.avgROUGE}</strong></p>
+      </div>
+
+      {results.map((res, i) => (
+        <div key={i} className="mb-4 p-4 border-l-4 rounded bg-white shadow border-l-gray-300">
+          <h3 className="font-bold text-lg">Prompt {i + 1}</h3>
+          <p className="mt-2"><strong>Prompt:</strong> {res.prompt}</p>
+          <p><strong>Expected:</strong> {res.expected}</p>
+          <p><strong>Actual:</strong> {res.actual}</p>
+          <p> BLEU: <strong>{res.scores.bleu.toFixed(2)}</strong></p>
+          <p> ROUGE: <strong>{res.scores.rouge.toFixed(2)}</strong></p>
+          <p>Pass: <strong className={res.success ? 'text-green-600' : 'text-red-600'}>{res.success ? 'Yes' : 'No'}</strong></p>
+        </div>
+      ))}
+    </main>
+  );
+}