@devilsdev/rag-pipeline-utils 2.0.0 → 2.0.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,36 @@
1
+ name: Deploy Docusaurus to GitHub Pages
2
+
3
+ on:
4
+ push:
5
+ branches: [main]
6
+ paths:
7
+ - 'docs-site/**'
8
+ - 'blog/**'
9
+ - 'sidebars.js'
10
+ - 'docusaurus.config.js'
11
+
12
+ jobs:
13
+ deploy:
14
+ runs-on: ubuntu-latest
15
+ steps:
16
+ - name: 📥 Checkout
17
+ uses: actions/checkout@v4
18
+
19
+ - name: ⚙️ Setup Node.js
20
+ uses: actions/setup-node@v4
21
+ with:
22
+ node-version: 20
23
+
24
+ - name: 📦 Install dependencies
25
+ run: npm ci
26
+
27
+ - name: 🛠️ Build static site
28
+ run: npm run build:docs
29
+
30
+ - name: 🚀 Deploy to GitHub Pages
31
+ uses: peaceiris/actions-gh-pages@v4
32
+ with:
33
+ github_token: ${{ secrets.GITHUB_TOKEN }}
34
+ publish_dir: ./docs-site/build
35
+
36
+
@@ -0,0 +1,51 @@
1
+ # .github/workflows/publish-release-blog.yml
2
+ # Version: 1.0.0
3
+ # Description: Auto-publish blog post from CHANGELOG or prepared .md after NPM release
4
+ # Author: Ali Kahwaji
5
+
6
+ name: ✨ Publish Release Blog Post
7
+
8
+ on:
9
+ release:
10
+ types: [published]
11
+
12
+ jobs:
13
+ publish-blog-post:
14
+ name: Generate & Commit Blog Post
15
+ runs-on: ubuntu-latest
16
+
17
+ steps:
18
+ - name: ⬇️ Checkout code
19
+ uses: actions/checkout@v4
20
+
21
+ - name: 🔧 Set up Node.js
22
+ uses: actions/setup-node@v4
23
+ with:
24
+ node-version: 20
25
+
26
+ - name: ✏️ Generate blog post
27
+ run: |
28
+ VERSION=${GITHUB_REF#refs/tags/}
29
+ DATE=$(date +'%Y-%m-%d')
30
+ FILE="docs-site/blog/${DATE}-release-${VERSION}.md"
31
+
32
+ echo "---" > $FILE
33
+ echo "slug: release-${VERSION}" >> $FILE
34
+ echo "title: ✨ Version ${VERSION} Released" >> $FILE
35
+ echo "authors: [ali]" >> $FILE
36
+ echo "tags: [release, changelog]" >> $FILE
37
+ echo "---" >> $FILE
38
+ echo >> $FILE
39
+
40
+ echo "RAG Pipeline Utils v${VERSION} is now available on NPM!" >> $FILE
41
+ echo >> $FILE
42
+ echo "#### Highlights" >> $FILE
43
+ echo "- Automated blog generation on GitHub releases" >> $FILE
44
+ echo "- Version bump: ${VERSION}" >> $FILE
45
+ echo "- Ensure CHANGELOG.md is synced" >> $FILE
46
+
47
+ - name: 📋 Commit blog post
48
+ uses: stefanzweifel/git-auto-commit-action@v5
49
+ with:
50
+ commit_message: "docs(blog): add release post for ${{ github.ref_name }}"
51
+ file_pattern: "docs-site/blog/*.md"
package/.husky/pre-commit CHANGED
@@ -3,3 +3,4 @@
3
3
 
4
4
  npm run validate-fixtures && npm run lint
5
5
  npx lint-staged
6
+ node scripts/validate-changelog-version.js
package/CHANGELOG.md CHANGED
@@ -1,6 +1,13 @@
1
1
  # Changelog
2
2
 
3
- # [2.0.0](https://github.com/DevilsDev/rag-pipeline-utils/compare/v1.0.1...v2.0.0) (2025-04-17)
3
+ ## [2.0.1](https://github.com/DevilsDev/rag-pipeline-utils/compare/v2.0.0...v2.0.1) (2025-04-17)
4
+
5
+
6
+ ### Bug Fixes
7
+
8
+ * add @docusaurus/preset-classic to enable CI builds ([ed5e771](https://github.com/DevilsDev/rag-pipeline-utils/commit/ed5e771f919face4693a6a7abfed4cf0fe1325ca))
9
+
10
+ ## [2.0.0] - 2025-04-17 (2025-04-17)
4
11
 
5
12
 
6
13
  ### Bug Fixes
package/README.md CHANGED
@@ -1,117 +1,146 @@
1
1
  # @DevilsDev/rag-pipeline-utils
2
2
 
3
+
4
+
5
+ # RAG Pipeline Utils
6
+
7
+ > Modular utilities for building Retrieval-Augmented Generation (RAG) pipelines, with CI-verified plugin contracts and mockable interfaces.
8
+
3
9
  [![CI](https://github.com/DevilsDev/rag-pipeline-utils/actions/workflows/ci.yml/badge.svg)](https://github.com/DevilsDev/rag-pipeline-utils/actions)
4
10
  [![codecov](https://codecov.io/gh/DevilsDev/rag-pipeline-utils/branch/main/graph/badge.svg)](https://codecov.io/gh/DevilsDev/rag-pipeline-utils)
5
11
  ![Release](https://img.shields.io/github/v/release/DevilsDev/rag-pipeline-utils?label=release)
12
+ ![License](https://img.shields.io/github/license/devilsdev/rag-pipeline-utils)
6
13
 
7
- **A modular Node.js toolkit for building RAG (Retrieval-Augmented Generation) pipelines with clean abstractions, pluggable components, and scalable architecture.**
14
+ ## Description
8
15
 
9
- ---
16
+ `@devilsdev/rag-pipeline-utils` provides a clean, extensible, and test-friendly foundation for constructing robust RAG pipelines using a plugin-based architecture. It enables developers to plug in their own document loaders, embedders, retrievers, and LLMs via configuration, and includes a CLI, mock contract verification, schema validation, and test harnesses to ensure production-readiness.
10
17
 
11
- ## Features
18
+ ## Key Features
12
19
 
13
- - ✅ Plugin-based architecture for RAG pipelines
14
- - ✅ Embedding + retrieval abstraction
15
- - ✅ Vector DB integrations (e.g., Pinecone)
16
- - ✅ Modular document loaders (PDF, HTML, Markdown, CSV, Directory)
17
- - ✅ CLI with `.ragrc.json` config fallback
18
- - ✅ Retry, backoff, and logging built-in
19
- - ✅ Reranker support and evaluation tooling
20
- - ✅ React dashboard for evaluation results
20
+ - **Plugin Contracts**: Runtime and CI verification of plugin interfaces (loader, embedder, retriever, llm).
21
+ - **CLI Support**: Easily run `ingest` and `query` commands using `.ragrc.json` configuration.
22
+ - **Test Fixtures**: Snapshot, fallback, and structured reranker validation.
23
+ - **Schema Enforcement**: `.ragrc.json` schema validation to catch config issues early.
24
+ - **Mock Repair Utilities**: Auto-fix for missing or broken mock implementations.
25
+ - **Modular Design**: Drop-in plugin support using custom class-based implementations.
21
26
 
22
- ---
27
+ ## Benefits of This Modular Design
28
+
29
+ - **Extensibility**: Developers can inject their own loader, retriever, or LLM logic without modifying the core.
30
+ - **Testability**: Mocks and CI tools validate contract adherence for reliable test coverage.
31
+ - **Decoupling**: Each plugin can be independently implemented and swapped in via `.ragrc.json`.
32
+ - **CI Resilience**: The CI pipeline includes linting, contract verification, and test coverage enforcement.
23
33
 
24
34
  ## Installation
25
35
 
26
36
  ```bash
27
- npm install @yourorg/rag-pipeline-utils
37
+ npm install @devilsdev/rag-pipeline-utils
28
38
  ```
29
39
 
30
- ---
40
+ ## Basic Usage
31
41
 
32
- ## Usage
42
+ 1. Create a `.ragrc.json` config:
33
43
 
34
- ```js
35
- import { createRagPipeline } from '@yourorg/rag-pipeline-utils';
44
+ ```json
45
+ {
46
+ "loader": {
47
+ "pdf": "./src/mocks/pdf-loader.js"
48
+ },
49
+ "embedder": {
50
+ "openai": "./src/mocks/openai-embedder.js"
51
+ },
52
+ "retriever": {
53
+ "pinecone": "./src/mocks/pinecone-retriever.js"
54
+ },
55
+ "llm": {
56
+ "openai": "./src/mocks/openai-llm.js"
57
+ },
58
+ "namespace": "demo-namespace",
59
+ "pipeline": ["loader", "embedder", "retriever"]
60
+ }
61
+ ```
36
62
 
37
- const rag = createRagPipeline({
38
- loader: 'markdown',
39
- embedder: 'openai',
40
- retriever: 'pinecone',
41
- llm: 'openai-gpt-4',
42
- useReranker: true
43
- });
63
+ 2. Run an ingestion:
44
64
 
45
- const answer = await rag.query('What is a vector database?');
46
- console.log(answer);
65
+ ```bash
66
+ node bin/cli.js ingest ./docs/sample.pdf
47
67
  ```
48
68
 
49
- ---
50
-
51
- ## Testing
69
+ 3. Run a query:
52
70
 
53
71
  ```bash
54
- npm test
72
+ node bin/cli.js query "What is in the document?"
55
73
  ```
56
74
 
57
- ---
75
+ ## Plugin Contracts
76
+
77
+ Each plugin must implement the following interface:
78
+
79
+ | Type | Required Methods |
80
+ | --------- | ------------------------------------------- |
81
+ | loader | `load(filePath)` |
82
+ | embedder | `embed(documents)`, `embedQuery(query)` |
83
+ | retriever | `store(vectors)`, `search(queryVector)` |
84
+ | llm | `ask(prompt)` |
85
+
86
+ ## Folder Structure
87
+
88
+ ```text
89
+ /project-root
90
+ ├── /src
91
+ │ ├── /routes
92
+ │ ├── /services
93
+ │ ├── /utils
94
+ │ └── app.js
95
+ ├── /scripts
96
+ │ ├── setup.js
97
+ │ ├── ci-runner.js
98
+ │ └── repair-fixtures.js
99
+ ├── /__tests__
100
+ │ ├── /fixtures
101
+ │ ├── /unit
102
+ │ ├── /integration
103
+ ├── /public
104
+ ├── README.md
105
+ ├── USE_CASES.md
106
+ ├── CHANGELOG.md
107
+ └── .env.example
108
+ ```
58
109
 
59
- ## CLI
60
110
 
61
- ```bash
62
- rag-pipeline ingest ./docs/my.pdf --loader pdf
63
- rag-pipeline query "Explain RAG pipelines" --llm openai-gpt-4
64
- rag-pipeline evaluate ./fixtures/sample-eval-dataset.json
65
- rag-pipeline rerank "Explain embeddings" --retriever pinecone
66
- ```
67
111
 
68
- Supports `.ragrc.json` fallback configuration:
112
+ ## Why Use This?
69
113
 
70
- ```json
71
- {
72
- "loader": "directory",
73
- "embedder": "openai",
74
- "retriever": "pinecone",
75
- "llm": "openai-gpt-4",
76
- "useReranker": true
77
- }
78
- ```
114
+ - **Developer Experience**: Fast iteration, full contract CI
115
+ - **Pluggable Design**: Swap in your stack (OpenAI, Pinecone, Langchain, etc.)
116
+ - **Minimal & Modular**: Clean separation of concerns
117
+ - **Prod-ready**: 100% test coverage, schema validation, CLI control
79
118
 
80
- ---
81
119
 
82
- ## Project Structure
83
120
 
84
- ```
85
- /src
86
- ├── core/ # Pipeline + registry
87
- ├── loader/ # Loaders for md, html, csv, dir
88
- ├── reranker/ # LLM-based context reranker
89
- ├── evaluate/ # Scoring, batch QA evaluation
90
- ├── utils/ # Logger, retry, schema
91
- /bin/cli.js # CLI entrypoint
92
- /__tests__/ # Unit, integration, snapshot tests
93
- /public/ # Evaluation dashboard (React)
94
- .github/workflows/ci.yml # CI config
95
- ```
121
+ ## Use Cases
96
122
 
97
- ---
123
+ Explore real-world DevX-focused examples in [USE_CASES.md](./USE_CASES.md), including:
98
124
 
99
- ## Evaluation Dashboard
125
+ - Mock-driven plugin TDD
126
+ - CI-based interface validation
127
+ - Multivendor ingestion and search
128
+ - Runtime swap of LLM, retrievers, or embedders
129
+ - Bootstrap pipelines with auto-repairable defaults
100
130
 
101
- ```bash
102
- node server.js
103
- ```
104
131
 
105
- Visit: [http://localhost:3000](http://localhost:3000) → Visualizes metrics, BLEU/ROUGE, pass rate.
106
132
 
107
- ---
133
+ ## Contributing
134
+
135
+ We welcome contributions via PRs, especially new mock providers, validation helpers, and runtime extensions. Please follow the test + CI conventions before submitting.
136
+
137
+
108
138
 
109
139
  ## License
110
140
 
111
- Apache-2.0
141
+ This project is licensed under the **GNU GPL v3.0** License - see the [LICENSE](./LICENSE) file for details.
112
142
 
113
143
  ---
114
144
 
115
- ## 👤 Author
145
+ Built with ❤️ by [Ali Kahwaji].
116
146
 
117
- Ali Kahwaji — [@DevilsDev](https://github.com/DevilsDev)
package/USE_CASES.md ADDED
@@ -0,0 +1,134 @@
1
+ # RAG Pipeline Utils — Developer-Focused Use Cases
2
+
3
+ > Modular, composable, and CI-verified RAG orchestration for modern AI applications.
4
+
5
+ ---
6
+
7
+ ## Use Case 1: Pluggable Local RAG Stack for Prototyping
8
+
9
+ **Problem:** Traditional RAG tools force tight vendor dependencies.
10
+
11
+ **Solution:** Use mockable local plugin implementations to simulate loaders, embedders, retrievers, and LLMs.
12
+
13
+ ```bash
14
+ npm install @devilsdev/rag-pipeline-utils
15
+ ```
16
+
17
+ ```js
18
+ // cli.js or app.js
19
+ import { PluginRegistry } from './src/core/plugin-registry.js';
20
+ import { loadRagConfig } from './src/config/load-config.js';
21
+
22
+ const config = loadRagConfig('.ragrc.json');
23
+ const loader = PluginRegistry.getPlugin('loader', 'pdf');
24
+ const docs = loader.load('./test.pdf');
25
+ ```
26
+
27
+ **Benefits**
28
+ - Local plugin mocks with expected method contracts
29
+ - Reusable plugin templates (`pdf-loader.js`, `openai-embedder.js`)
30
+ - Dev-first DX: debug, snapshot, iterate without cloud costs
31
+
32
+ ---
33
+
34
+ ## Use Case 2: Declarative CLI RAG Pipelines (CI-Ready)
35
+
36
+ **Problem:** Hardcoded pipeline steps can't be validated or changed by non-engineers.
37
+
38
+ **Solution:** Define `.ragrc.json` once and reuse across environments.
39
+
40
+ ```json
41
+ {
42
+ "loader": { "pdf": "./src/mocks/pdf-loader.js" },
43
+ "embedder": { "openai": "./src/mocks/openai-embedder.js" },
44
+ "retriever": { "pinecone": "./src/mocks/pinecone-retriever.js" },
45
+ "llm": { "openai": "./src/mocks/openai-llm.js" },
46
+ "namespace": "docs-poc",
47
+ "pipeline": ["loader", "embedder", "retriever"]
48
+ }
49
+ ```
50
+
51
+ ```bash
52
+ node ./bin/cli.js ingest ./docs/whitepaper.pdf
53
+ node ./bin/cli.js query "What’s the goal of this architecture?"
54
+ ```
55
+
56
+ **Benefits**
57
+ - Teams can standardize pipelines in source control
58
+ - Tests fail fast if structure or mocks drift
59
+ - Non-devs can operate pipelines via CLI
60
+
61
+ ---
62
+
63
+ ## Use Case 3: Mock-First TDD for Plugin Development
64
+
65
+ **Problem:** LLM, vector, and loader logic are often developed without tests.
66
+
67
+ **Solution:** Define contract-compliant mocks and CI-validate their structure before connecting to APIs.
68
+
69
+ ```js
70
+ export default class CustomEmbedder {
71
+ embed(docs) { return docs.map(...); }
72
+ embedQuery(q) { return [...]; }
73
+ }
74
+ ```
75
+
76
+ Run full contract verification:
77
+
78
+ ```bash
79
+ npm run ci
80
+ ```
81
+
82
+ **Benefits**
83
+ - Every plugin is CI-verified for method presence
84
+ - Shared contracts enforce required interfaces
85
+ - Onboarding new devs? Just follow `pluginContracts.js`
86
+
87
+ ---
88
+
89
+ ## Use Case 4: Vendor-Neutral AI Data Pipelines
90
+
91
+ **Problem:** Every RAG stack is locked to OpenAI + Pinecone + Langchain.
92
+
93
+ **Solution:** Use this project to mix, match, or mock vector/LLM pairs:
94
+
95
+ ```js
96
+ "retriever": { "localstore": "./src/plugins/local-retriever.js" },
97
+ "llm": { "cohere": "./src/plugins/cohere-llm.js" }
98
+ ```
99
+
100
+ **Benefits**
101
+ - Run multiple RAG pipelines in parallel (A/B testing, evals)
102
+ - Lower infra cost with offline/local plugin support
103
+ - Integrate with tools outside Langchain ecosystem
104
+
105
+ ---
106
+
107
+ ## 🛠 Use Case 5: Plugin Engineering with Guardrails
108
+
109
+ **Problem:** Complex plugin systems break silently.
110
+
111
+ **Solution:** This framework has:
112
+ - Plugin method contract enforcement
113
+ - CLI test runner (`scripts/ci-runner.js`)
114
+ - Auto repair tool (`scripts/repair-fixtures.js`)
115
+
116
+ **DX Boost**
117
+ - One-line plugin repairs
118
+ - Validated schema structure
119
+ - Aligned to real-world developer workflows
120
+
121
+ ---
122
+
123
+ ## Cross References
124
+
125
+ - [README](./README.md) — Project overview, architecture, CLI and configuration details
126
+ - [pluginContracts.js](./src/core/plugin-contracts.js) — Interface contracts for loader, embedder, retriever, LLM
127
+ - [diagnostic-reporter.js](./src/utils/ci/diagnostic-reporter.js) — CI error helper
128
+ - [repair-fixtures.js](./scripts/repair-fixtures.js) — Auto-regeneration of mocks
129
+
130
+ ---
131
+
132
+ ## License
133
+
134
+ Licensed under the **GPL-3.0 License** — see [LICENSE](./LICENSE) for full terms.
@@ -1,7 +1,6 @@
1
1
  ---
2
2
  slug: welcome
3
3
  title: Welcome
4
- authors: [slorber, yangshun]
5
4
  tags: [facebook, hello, docusaurus]
6
5
  ---
7
6
 
@@ -0,0 +1,67 @@
1
+ ---
2
+ slug: release-2-0
3
+ title: Release 2.0.0 – Plugin Contracts, CI Automation, Docs
4
+ authors: [ali]
5
+ tags: [release, changelog]
6
+ description: RAG Pipeline Utils v2.0.0 introduces plugin contract validation, full CI coverage, fixture repairs, and professional Docusaurus docs.
7
+ ---
8
+
9
+ We’re thrilled to announce the release of **RAG Pipeline Utils v2.0.0** — a fully modular and developer-focused toolkit for constructing, validating, and managing RAG (Retrieval-Augmented Generation) pipelines.
10
+
11
+ ---
12
+
13
+ ## 🚀 Highlights in v2.0.0
14
+
15
+ ### 🔌 Contract-Based Plugin Validation
16
+
17
+ - Enforces strict interfaces for all core plugin types:
18
+ - `loader.load()`
19
+ - `embedder.embed()` and `embedQuery()`
20
+ - `retriever.store()` and `search()`
21
+ - `llm.ask()`
22
+ - Includes auto-verification and CLI validation via `scripts/verify-fixture-mocks.js`
23
+
24
+ ### 🧪 Enhanced CI & Git Hooks
25
+
26
+ - Full GitHub Actions integration:
27
+ - Lint, test, mock validation, release automation
28
+ - Local enforcement using **Husky** + **validate-changelog-version.js**
29
+
30
+ ### 🧰 Fixture Repair & Auto-Mock Utilities
31
+
32
+ - Self-healing script: `scripts/repair-fixtures.js`
33
+ - Support for regenerating mocks that align with runtime requirements and CLI tests
34
+
35
+ ### 📘 Developer Documentation
36
+
37
+ - Professionally published Docusaurus site:
38
+ - 📚 [Visit Docs](https://devilsdev.github.io/rag-pipeline-utils)
39
+ - 🧭 Usage, architecture, CLI, plugin extension, and release notes
40
+ - Blog support enabled for continuous release announcements
41
+
42
+ ---
43
+
44
+ ## ✨ Why This Matters
45
+
46
+ This release marks a shift from traditional monolithic RAG stacks to a **fully composable, testable, and CI-validated** toolkit. It was built with long-term developer experience (DX) in mind, emphasizing:
47
+
48
+ - 📦 **Modularity**: Swap or add plugins without rewriting pipeline logic
49
+ - 🧪 **Test Confidence**: Never publish broken mocks or CI regressions again
50
+ - 🧠 **Scalability**: Suitable for both small prototypes and production pipelines
51
+ - 🛠 **Tooling First**: Includes everything from config loading to evaluation dashboards
52
+
53
+ ---
54
+
55
+ ## 📦 Install Now
56
+
57
+ ```bash
58
+ npm install @devilsdev/rag-pipeline-utils
59
+ ```
60
+
61
+ ---
62
+
63
+ For upgrade guides, contract schemas, and real-world examples, check out the [official docs](https://devilsdev.github.io/rag-pipeline-utils/docs/Introduction).
64
+
65
+ 🧠 Built for engineers who want **control, clarity, and composability** in their LLM workflows.
66
+
67
+ — *Ali Kahwaji & RAG Pipeline Utils Contributors*
@@ -0,0 +1,21 @@
1
+ ---
2
+ slug: release-2-0-0
3
+ title: Release 2.0.0 — Modular RAG Toolkit
4
+ authors: ali
5
+ tags: [release, changelog, RAG, developer-tools]
6
+ ---
7
+
8
+ 🚀 **Version 2.0.0** is now live on [npm](https://www.npmjs.com/package/@devilsdev/rag-pipeline-utils)!
9
+
10
+ This release introduces:
11
+
12
+ - ✅ Plugin contracts for loaders, embedders, retrievers, LLMs
13
+ - 🧪 Fixture mock verification + CLI validation
14
+ - 🧰 Auto-repair tooling for broken test mocks
15
+ - 📦 Seamless integration with `.ragrc.json` config fallback
16
+ - 📄 Docusaurus documentation & blog support
17
+ - 💡 Modern DX with versioned docs, coverage, CI hooks, changelogs
18
+
19
+ Explore docs → [devilsdev.github.io/rag-pipeline-utils](https://devilsdev.github.io/rag-pipeline-utils)
20
+
21
+ 📌 Run `npm install @devilsdev/rag-pipeline-utils` to get started.
@@ -1,23 +1,5 @@
1
- yangshun:
2
- name: Yangshun Tay
3
- title: Front End Engineer @ Facebook
4
- url: https://github.com/yangshun
5
- image_url: https://github.com/yangshun.png
6
- page: true
7
- socials:
8
- x: yangshunz
9
- github: yangshun
10
-
11
- slorber:
12
- name: Sébastien Lorber
13
- title: Docusaurus maintainer
14
- url: https://sebastienlorber.com
15
- image_url: https://github.com/slorber.png
16
- page:
17
- # customize the url of the author page at /blog/authors/<permalink>
18
- permalink: '/all-sebastien-lorber-articles'
19
- socials:
20
- x: sebastienlorber
21
- linkedin: sebastienlorber
22
- github: slorber
23
- newsletter: https://thisweekinreact.com
1
+ ali:
2
+ name: Ali Kahwaji
3
+ title: Creator of RAG Pipeline Utils
4
+ url: https://github.com/DevilsDev
5
+ image_url: /img/authors/ali.jpg
@@ -17,3 +17,12 @@ hola:
17
17
  label: Hola
18
18
  permalink: /hola
19
19
  description: Hola tag description
20
+
21
+ release:
22
+ label: Release
23
+ permalink: /blog/tags/release
24
+
25
+ changelog:
26
+ label: Changelog
27
+ permalink: /blog/tags/changelog
28
+
File without changes
@@ -18,6 +18,13 @@ export default {
18
18
  // ✅ Final favicon and social card for branding
19
19
  favicon: 'img/favicon.ico',
20
20
  themeConfig: {
21
+
22
+ // algolia: {
23
+ // appId: 'YOUR_APP_ID',
24
+ // apiKey: 'YOUR_PUBLIC_API_KEY',
25
+ // indexName: 'rag_pipeline_utils',
26
+ // contextualSearch: true,
27
+ // },
21
28
  image: 'img/social-card.jpg',
22
29
  navbar: {
23
30
  title: 'RAG Utils',
@@ -48,6 +55,12 @@ export default {
48
55
  sidebarPath: require.resolve('./sidebars.js'),
49
56
  editUrl: 'https://github.com/DevilsDev/rag-pipeline-utils/edit/main/docs-site/'
50
57
  },
58
+
59
+ blog: {
60
+ showReadingTime: true,
61
+ editUrl: 'https://github.com/DevilsDev/rag-pipeline-utils/edit/main/docs-site/blog/',
62
+ },
63
+
51
64
  theme: {
52
65
  customCss: require.resolve('./src/css/custom.css')
53
66
  }
@@ -2,21 +2,33 @@
2
2
 
3
3
  /**
4
4
  * Sidebar configuration for RAG Pipeline Utils documentation site
5
- * Generated manually for ordered navigation
5
+ * Version: 2.0.0
6
6
  * Author: Ali Kahwaji
7
+ * Updated: Structured for modular clarity and DX
7
8
  * @type {import('@docusaurus/plugin-content-docs').SidebarsConfig}
8
9
  */
9
10
 
10
11
  const sidebars = {
11
12
  tutorialSidebar: [
12
- 'Introduction',
13
- 'Usage',
14
- 'CLI',
15
- 'Architecture',
16
- 'Evaluation',
17
- 'Plugins',
18
- 'FAQ'
19
- ]
13
+ {
14
+ type: 'category',
15
+ label: 'Overview',
16
+ collapsed: false,
17
+ items: ['Introduction', 'Architecture', 'USE_CASES'],
18
+ },
19
+ {
20
+ type: 'category',
21
+ label: 'Developer Guide',
22
+ collapsed: false,
23
+ items: ['Usage', 'CLI', 'Plugins'],
24
+ },
25
+ {
26
+ type: 'category',
27
+ label: 'Advanced',
28
+ collapsed: true,
29
+ items: ['Evaluation', 'FAQ'],
30
+ },
31
+ ],
20
32
  };
21
33
 
22
34
  export default sidebars;
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@devilsdev/rag-pipeline-utils",
3
- "version": "2.0.0",
3
+ "version": "2.0.1",
4
4
  "description": "A modular toolkit for building RAG (Retrieval-Augmented Generation) pipelines in Node.js",
5
5
  "main": "src/core/create-pipeline.js",
6
6
  "bin": {
@@ -17,7 +17,12 @@
17
17
  "lint-staged": "lint-staged",
18
18
  "postinstall": "husky install",
19
19
  "verify:mocks": "node scripts/verify-fixture-mocks.js",
20
- "ci": "node scripts/ci-runner.js"
20
+ "build:docs": "npx docusaurus build docs-site",
21
+ "ci": "node scripts/ci-runner.js",
22
+ "docs:dev": "npx docusaurus start docs-site",
23
+ "docs:build": "npx docusaurus build docs-site",
24
+ "docs:deploy": "npx docusaurus deploy docs-site",
25
+ "docs:version": "npx docusaurus docs:version"
21
26
  },
22
27
  "lint-staged": {
23
28
  "**/*.{js,ts}": [
@@ -48,6 +53,8 @@
48
53
  "access": "public"
49
54
  },
50
55
  "devDependencies": {
56
+ "@docusaurus/core": "^3.7.0",
57
+ "@docusaurus/preset-classic": "^3.7.0",
51
58
  "@semantic-release/changelog": "^6.0.3",
52
59
  "@semantic-release/commit-analyzer": "^13.0.1",
53
60
  "@semantic-release/git": "^10.0.1",
@@ -57,6 +64,7 @@
57
64
  "@types/jest": "^29.5.14",
58
65
  "codecov": "^3.8.2",
59
66
  "cross-env": "^7.0.3",
67
+ "docusaurus": "^1.14.7",
60
68
  "esbuild": "^0.25.2",
61
69
  "eslint": "^8.57.1",
62
70
  "husky": "^8.0.3",
@@ -0,0 +1,48 @@
1
+ /**
2
+ * Version: 1.1.0
3
+ * Description: Ensures CHANGELOG.md contains the current version and auto-corrects format if needed.
4
+ * Author: Ali Kahwaji
5
+ */
6
+
7
+ import fs from 'fs';
8
+ import path from 'path';
9
+ import { fileURLToPath } from 'url';
10
+ import { execSync } from 'child_process';
11
+
12
+ const __filename = fileURLToPath(import.meta.url);
13
+ const __dirname = path.dirname(__filename);
14
+
15
+ const pkg = JSON.parse(fs.readFileSync(path.resolve(__dirname, '../package.json')));
16
+ const version = pkg.version;
17
+ const changelogPath = path.resolve(__dirname, '../CHANGELOG.md');
18
+
19
+ let changelog = fs.readFileSync(changelogPath, 'utf-8');
20
+
21
+ // Step 1: Normalize invalid headings like "# [2.0.0](...)"
22
+ const normalized = changelog.replace(/^# \[([0-9]+\.[0-9]+\.[0-9]+)\]\([^)]+\)/gm, (match, v) => {
23
+ const today = new Date().toISOString().split('T')[0];
24
+ return `## [${v}] - ${today}`;
25
+ });
26
+
27
+ // Step 2: Update file if normalization occurred
28
+ if (normalized !== changelog) {
29
+ fs.writeFileSync(changelogPath, normalized);
30
+ console.log(`🛠️ CHANGELOG.md heading normalized for version ${version}`);
31
+ changelog = normalized;
32
+
33
+ // Auto-stage if in git project (Husky compatibility)
34
+ try {
35
+ execSync(`git add ${changelogPath}`);
36
+ console.log('CHANGELOG.md staged after correction');
37
+ } catch {
38
+ // Non-fatal
39
+ }
40
+ }
41
+
42
+ // Step 3: Validate presence of correct heading
43
+ const headingPattern = new RegExp(`^## \\[${version}\\]`, 'm');
44
+ if (!headingPattern.test(changelog)) {
45
+ throw new Error(`❌ CHANGELOG.md does not contain a properly formatted entry for version ${version}`);
46
+ }
47
+
48
+ console.log(`✅ CHANGELOG.md contains a valid entry for version ${version}`);
@@ -1,12 +0,0 @@
1
- ---
2
- slug: first-blog-post
3
- title: First Blog Post
4
- authors: [slorber, yangshun]
5
- tags: [hola, docusaurus]
6
- ---
7
-
8
- Lorem ipsum dolor sit amet...
9
-
10
- <!-- truncate -->
11
-
12
- ...consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet
@@ -1,44 +0,0 @@
1
- ---
2
- slug: long-blog-post
3
- title: Long Blog Post
4
- authors: yangshun
5
- tags: [hello, docusaurus]
6
- ---
7
-
8
- This is the summary of a very long blog post,
9
-
10
- Use a `<!--` `truncate` `-->` comment to limit blog post size in the list view.
11
-
12
- <!-- truncate -->
13
-
14
- Lorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet
15
-
16
- Lorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet
17
-
18
- Lorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet
19
-
20
- Lorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet
21
-
22
- Lorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet
23
-
24
- Lorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet
25
-
26
- Lorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet
27
-
28
- Lorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet
29
-
30
- Lorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet
31
-
32
- Lorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet
33
-
34
- Lorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet
35
-
36
- Lorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet
37
-
38
- Lorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet
39
-
40
- Lorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet
41
-
42
- Lorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet
43
-
44
- Lorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet
@@ -1,24 +0,0 @@
1
- ---
2
- slug: mdx-blog-post
3
- title: MDX Blog Post
4
- authors: [slorber]
5
- tags: [docusaurus]
6
- ---
7
-
8
- Blog posts support [Docusaurus Markdown features](https://docusaurus.io/docs/markdown-features), such as [MDX](https://mdxjs.com/).
9
-
10
- :::tip
11
-
12
- Use the power of React to create interactive blog posts.
13
-
14
- :::
15
-
16
- {/* truncate */}
17
-
18
- For example, use JSX to create an interactive button:
19
-
20
- ```js
21
- <button onClick={() => alert('button clicked!')}>Click me!</button>
22
- ```
23
-
24
- <button onClick={() => alert('button clicked!')}>Click me!</button>