@devilsdev/rag-pipeline-utils 2.1.4 → 2.1.7

package/.github/PROJECT_ROADMAP.md

@@ -0,0 +1,102 @@
+# 🛣️ Project Roadmap - RAG Pipeline Utils
+
+> Version: 1.2.0  
+> Status: In Progress  
+> Last Updated: 2025-04-22
+
+This roadmap guides the feature evolution, developer tooling, CI/CD automation, and community integration for the `@devilsdev/rag-pipeline-utils` project. Changes here are synchronized to GitHub Issues and CI workflows.
+
+---
+
+## ✅ Legend
+
+| Symbol | Meaning |
+|--------|---------|
+| ✅     | Done    |
+| 🚧     | In Progress |
+| 🔜     | Upcoming |
+| 🧪     | Experimental |
+
+---
+
+## 📦 Phase 1: Foundation & CI/CD
+
+| Feature | Status | Target Version | Tags |
+|---------|--------|----------------|------|
+| Automated test coverage with Jest | ✅ Done | v2.0.0 | test, ci |
+| Codecov integration + badge | ✅ Done | v2.0.0 | ci, badge |
+| GitHub Actions CI: lint, test, release | ✅ Done | v2.0.0 | ci |
+| GitHub Pages deploy for Docusaurus | ✅ Done | v2.0.0 | docs, deploy |
+| CHANGELOG auto-generation | ✅ Done | v2.0.0 | release |
+| Blog post sync from changelog | ✅ Done | v2.1.0 | blog, release |
+
+---
+
+## 🚀 Phase 2: Community & Docs
+
+| Feature | Status | Target Version | Tags |
+|---------|--------|----------------|------|
+| Interactive onboarding via CLI | 🚧 In Progress | v2.2.0 | devx, onboarding |
+| GitHub Discussions integration | 🔜 | v2.2.0 | community |
+| Contributor guide & issue templates | ✅ Done | v2.0.0 | community, docs |
+| Roadmap → GitHub Issues sync | ✅ Done | v2.1.0 | ci, roadmap |
+| Label sync from roadmap | ✅ Done | v2.1.0 | ci, labels |
+| Manual & automated blog generation | ✅ Done | v2.1.1 | blog, release |
+
+---
+
+## 🧠 Phase 3: Developer Tooling
+
+| Feature | Status | Target Version | Tags |
+|---------|--------|----------------|------|
+| CLI for `release:tag` with changelog preview | ✅ Done | v2.1.1 | cli, release |
+| Hydra.js integration for dynamic RAG config | 🚧 In Progress | v2.2.0 | config, hydra |
+| CLI for roadmap issue tagging | ✅ Done | v2.1.1 | cli, roadmap |
+| Status badge: Release Review Passed | ✅ Done | v2.1.1 | badge, ci |
+| Kanban sync for roadmap issues | ✅ Done | v2.1.1 | roadmap, ci |
+| Custom VS Code snippets & workspace | 🔜 | v2.3.0 | devx |
+
+---
+
+## 🌐 Phase 4: International & Advanced UX
+
+| Feature | Status | Target Version | Tags |
+|---------|--------|----------------|------|
+| Blog subscribe + RSS | ✅ Done | v2.1.1 | blog |
+| i18n translation support | 🔜 | v2.3.0 | i18n |
+| SEO optimization + open graph cards | ✅ Done | v2.1.1 | seo, meta |
+| Comment system on blog posts | 🔜 | v2.3.0 | blog, community |
+| Performance tuning + SSR check | ✅ Done | v2.1.0 | perf, ssr |
+| Multilingual toggle in header | 🔜 | v2.3.0 | i18n, ui |
+
+---
+
+## 🧩 Future Enhancements
+
+| Feature | Status | Target Version | Tags |
+|---------|--------|----------------|------|
+| Plugin support via CLI | 🔜 | v2.4.0 | cli, plugin |
+| Graph-based pipeline UI | 🔜 | v2.5.0 | ux, devx |
+| Real-world RAG examples + use cases | 🔜 | v2.3.0 | docs, showcase |
+| Prebuilt Docker image | 🔜 | v2.4.0 | infra, deploy |
+| Slack/Discord webhook for releases | 🔜 | v2.3.0 | infra, notify |
+
+---
+
+## 📊 Contributor Stats
+
+- GitHub Insights Badge: ![Contributors](https://img.shields.io/github/contributors/DevilsDev/rag-pipeline-utils)
+- Roadmap Sync: `scripts/create-roadmap-issues.js`
+- Label Sync: `scripts/ensure-roadmap-labels.js`
+- CLI Integration: `scripts/release-changelog.js`, `scripts/sync-roadmap-cli.js`
+
+---
+
+## 📁 Files
+
+- `.github/PROJECT_ROADMAP.md` ← source
+- `scripts/ensure-roadmap-labels.js` ← label sync
+- `scripts/create-roadmap-issues.js` ← issue sync
+- `scripts/close-roadmap-done.js` ← close done items
+- `.github/workflows/sync-roadmap.yml` ← automation trigger
+

package/.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,8 @@
+## ✅ Release Quality Checklist
+
+- [ ] CHANGELOG.md updated
+- [ ] Blog post generated (`docs-site/blog/*release*.md`)
+- [ ] `release:tag` and `release:sync` flows tested
+
+<!-- CI Badge -->
+![Release Review Passed](https://github.com/DevilsDev/rag-pipeline-utils/actions/workflows/enforce-release-review.yml/badge.svg)

package/.github/workflows/auto-release-note.yml

@@ -0,0 +1,73 @@
+# .github/workflows/auto-release-note.yml
+# Version: 1.0.0
+# Description: Unified workflow to generate changelog + blog post from Git tag
+# Author: Ali Kahwaji
+
+name: ✫ Auto Release Note
+
+on:
+  push:
+    tags:
+      - 'v*'
+  workflow_dispatch:
+
+permissions:
+  contents: write
+
+jobs:
+  generate-note:
+    name: 🖍 Generate Blog + Changelog
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: ⬇️ Checkout code
+        uses: actions/checkout@v4
+
+      - name: 🔧 Setup Node.js
+        uses: actions/setup-node@v4
+        with:
+          node-version: 20
+
+      - name: 📦 Install dependencies
+        run: npm ci
+
+      - name: 📑 Generate release note from tag
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: |
+          VERSION=${GITHUB_REF#refs/tags/v}
+          echo "Generating blog + changelog for v$VERSION"
+
+          git fetch --tags --force
+
+          node scripts/generate-release-note.js v$VERSION
+
+      - name: 📄 Auto-commit blog + changelog
+        uses: stefanzweifel/git-auto-commit-action@v5
+        with:
+          commit_message: "docs(release): auto-publish blog and changelog"
+          file_pattern: "CHANGELOG.md docs-site/blog/*.md"
+          push_options: '--follow-tags'
+
+      - name: 🚀 Confirm complete
+        run: echo "Release blog + changelog committed and pushed."
+
+      - name: 📈 Emit Blog Badge Trigger
+        run: echo "::set-output name=blog::generated"
+
+      - name: 🌟 Trigger Badge (Optional)
+        if: always()
+        uses: peter-evans/create-or-update-comment@v4
+        with:
+          issue-number: 1
+          body: "Blog Post + Changelog updated for ${{ github.ref_name }} :rocket:"
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: 💬 Comment on PR (optional)
+        if: github.event.pull_request.number != ''
+        uses: peter-evans/create-or-update-comment@v4
+        with:
+            issue-number: ${{ github.event.pull_request.number }}
+            body: "📢 Blog + Changelog updated for ${{ github.ref_name }} 🎉"
+        

package/.github/workflows/ci.yml

@@ -1,5 +1,5 @@
 # .github/workflows/ci.yml
-# Version: 1.0.0
+# Version: 1.1.0
 # Description: Clean CI pipeline with lint, test, fixtures, coverage, and release
 # Author: Ali Kahwaji
 
@@ -8,6 +8,7 @@ name: CI & Release
 on:
   push:
     branches: [main]
+    tags: ['v*']
   pull_request:
     branches: [main]
 
@@ -30,7 +31,7 @@ jobs:
       - name: 📦 Install dependencies
         run: npm ci
 
-      - name: 🧹 Lint
+      - name: 🧹 Lint codebase
         run: npm run lint
 
       - name: ✅ Run tests with coverage
@@ -47,7 +48,8 @@ jobs:
 
       - name: 🚀 Semantic Release
         env:
-          GITHUB_TOKEN: ${{ secrets.GH_TOKEN }}
+          GH_TOKEN: ${{ secrets.GH_TOKEN }}            # ✅ Required by @semantic-release/github
+          GITHUB_TOKEN: ${{ secrets.GH_TOKEN }}        # ✅ Used by GitHub Actions
           NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
           NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
         run: npx semantic-release

package/.github/workflows/docs-ci.yml

@@ -1,9 +1,9 @@
 # .github/workflows/docs-ci.yml
-# Version: 1.0.0
-# Description: CI pipeline for linting, testing, and building docs
+# Version: 1.2.0
+# Description: CI pipeline for linting, testing, and building docs with improved caching and consistency
 # Author: Ali Kahwaji
 
-name: Docs CI
+name: 📘 Docs CI
 
 on:
   push:
@@ -12,37 +12,34 @@ on:
     branches: [main]
 
 jobs:
-  build:
+  docs-ci:
     runs-on: ubuntu-latest
-    defaults:
-      run:
-        working-directory: docs-site
 
     steps:
-      - name: Checkout repository
+      - name: ⬇️ Checkout code
         uses: actions/checkout@v4
 
-      - name: Setup Node.js
+      - name: 🟢 Setup Node.js
         uses: actions/setup-node@v4
         with:
-          node-version: '18'
+          node-version: 20
           cache: 'npm'
           cache-dependency-path: '**/package-lock.json'
 
-      - name: Install dependencies
+      - name: 📦 Install dependencies
         run: npm ci
 
-      - name: Lint codebase
+      - name: 🧹 Lint codebase
         run: npm run lint
 
-      - name: Validate docs build
+      - name: 🛠 Build documentation
         run: npm run docs:build
 
-      - name: Run tests (optional)
+      - name: ✅ Run tests
         run: npm test
         if: ${{ always() }}
 
-      - name: Upload build artifacts (optional)
+      - name: 📦 Upload built docs (optional)
         uses: actions/upload-artifact@v4
         with:
           name: docusaurus-build

package/.github/workflows/docs-deploy.yml

@@ -1,6 +1,6 @@
 # .github/workflows/docs-deploy.yml
-# Version: 1.0.0
-# Description: Deploys Docusaurus docs site to GitHub Pages
+# Version: 1.2.0
+# Description: Deploys Docusaurus docs site to GitHub Pages with CNAME and post-release chaining
 # Author: Ali Kahwaji
 
 name: 📚 Deploy Docusaurus Docs
@@ -17,34 +17,48 @@ on:
       - 'package.json'
 
 permissions:
-  contents: write  # Required for GitHub Pages deployment
+  contents: write
 
 jobs:
   deploy:
-    name: Deploy to GitHub Pages
+    name: 🚀 Publish to GitHub Pages
     runs-on: ubuntu-latest
 
     steps:
-      - name: Checkout repository
+      - name: 📥 Checkout repository
         uses: actions/checkout@v4
+        with:
+          fetch-depth: 0  # ✅ Full git history for tag diff or changelog sync
 
-      - name: Setup Node.js
+      - name: 🧰 Setup Node.js
         uses: actions/setup-node@v4
         with:
-          node-version: '18'
+          node-version: 20
           cache: 'npm'
 
-      - name: Install dependencies
+      - name: 📦 Install dependencies
         run: npm ci
 
-      - name: Build docs
+      - name: 🛠 Git Identity
+        run: |
+          git config --global user.name "GitHub Actions"
+          git config --global user.email "github-actions@github.com"
+
+      - name: 🛠 Build docs
         run: npm run docs:build
 
-      - name: Deploy to GitHub Pages
+      - name: ✅ Auto-generate release blog + changelog (if not triggered by tag)
+        run: |
+          TAG=$(git describe --tags --abbrev=0)
+          node scripts/generate-release-note.js "$TAG" || echo "🔁 Skipped blog generation fallback"
+        if: success() && !startsWith(github.ref, 'refs/tags/')
+
+      - name: 🚀 Deploy to GitHub Pages
         uses: peaceiris/actions-gh-pages@v4
         with:
           github_token: ${{ secrets.GITHUB_TOKEN }}
           publish_dir: docs-site/build
           publish_branch: gh-pages
+          cname: docs.rag-utils.dev  # 🔧 CHANGE THIS if you use a custom domain
           user_name: github-actions[bot]
           user_email: 41898282+github-actions[bot]@users.noreply.github.com

package/.github/workflows/enforce-release-review.yml

@@ -0,0 +1,50 @@
+# .github/workflows/enforce-release-review.yml
+# Version: 1.1.0
+# Description: Validates CHANGELOG.md and blog edits on PRs and tag pushes. Emits status badge.
+# Author: Ali Kahwaji
+
+name: 🛡 Release Review Passed
+
+on:
+  pull_request:
+    branches: [main]
+  push:
+    tags:
+      - 'v*.*.*'
+  workflow_dispatch:
+
+permissions:
+  contents: read
+  pull-requests: write
+
+jobs:
+  review:
+    name: 📝 Release Review Check
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: 📥 Checkout repo
+        uses: actions/checkout@v4
+
+      - name: 🔍 Scan for changelog or blog edits
+        id: check
+        run: |
+          git fetch origin main --depth=1
+          CHANGED_FILES=$(git diff --name-only origin/main...HEAD || true)
+
+          echo "Changed files:"
+          echo "$CHANGED_FILES"
+
+          if echo "$CHANGED_FILES" | grep -qE 'CHANGELOG\.md|docs-site/blog/'; then
+            echo "✅ Changelog or blog post updated."
+            echo "result=pass" >> "$GITHUB_OUTPUT"
+          else
+            echo "❌ No changelog or blog edits found."
+            echo "result=fail" >> "$GITHUB_OUTPUT"
+          fi
+
+      - name: 🚫 Fail if missing changelog or blog
+        if: steps.check.outputs.result == 'fail'
+        run: |
+          echo "Missing required changelog or blog post for release."
+          exit 1

package/.github/workflows/generate-blog-post.yml

@@ -0,0 +1,44 @@
+# .github/workflows/generate-blog-post.yml
+# Version: 1.1.0
+# Description: Auto-generates blog posts from release tags using generate-release-note.js
+# Author: Ali Kahwaji
+
+name: "📝 Auto Generate Blog Post"
+
+on:
+  release:
+    types: [published]
+  push:
+    tags:
+      - 'v*.*.*'
+
+permissions:
+  contents: write
+
+jobs:
+  blog:
+    name: Generate Release Blog
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: ⬇️ Checkout code
+        uses: actions/checkout@v4
+
+      - name: 🛠 Setup Node
+        uses: actions/setup-node@v4
+        with:
+          node-version: 20
+
+      - name: 📦 Install dependencies
+        run: npm ci
+
+      - name: 📝 Generate blog post from tag
+        run: node scripts/generate-release-note.js ${{ github.ref_name }} <previous-tag>
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: 📋 Commit blog post
+        uses: stefanzweifel/git-auto-commit-action@v5
+        with:
+          commit_message: "docs(blog): auto-generate post for release ${{ github.ref_name }}"
+          file_pattern: "docs-site/blog/*.md"

package/.github/workflows/post-release-generate-blog.yml

@@ -0,0 +1,71 @@
+# .github/workflows/post-release-generate-blog.yml
+# Version: 2.5.2
+# Description: Auto-generates changelog + blog post from tag or GitHub release with dry-run safety and badge output
+# Author: Ali Kahwaji
+
+name: Auto Generate Blog + Changelog
+
+on:
+  release:
+    types: [published]
+  push:
+    tags:
+      - 'v*.*.*'
+  workflow_dispatch:
+    inputs:
+      dry_run:
+        description: 'Run generator in dry-run mode (no git push)'
+        required: false
+        default: 'false'
+
+permissions:
+  contents: write
+  issues: write
+  pull-requests: write
+
+jobs:
+  generate:
+    name: 📓 Generate Blog + Changelog
+    runs-on: ubuntu-latest
+    outputs:
+      blog_generated: ${{ steps.done.outputs.blog_generated }}
+
+    steps:
+      - name: 📥 Checkout code
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+
+      - name: 🛠 Setup Git identity
+        run: |
+          git config --global user.name "GitHub Actions"
+          git config --global user.email "docs@autobot.local"
+
+      - name: 📦 Install dependencies
+        run: npm ci
+
+      - name: 🏷 Extract version and dry_run mode
+        id: context
+        run: |
+          TAG=${GITHUB_REF#refs/tags/}
+          echo "tag=${TAG}" >> $GITHUB_OUTPUT
+          echo "dry_run=${{ github.event.inputs.dry_run == 'true' && 'true' || 'false' }}" >> $GITHUB_OUTPUT
+
+      - name: 📝 Generate release note
+        id: note
+        run: |
+          echo "🚧 Running release note generator..."
+          node scripts/generate-release-note.js "${{ steps.context.outputs.tag }}"
+        env:
+          DRY_RUN: ${{ steps.context.outputs.dry_run }}
+
+      - name: 🔐 Commit blog & changelog
+        if: ${{ steps.context.outputs.dry_run != 'true' }}
+        uses: stefanzweifel/git-auto-commit-action@v5
+        with:
+          commit_message: "docs(release): blog + changelog for ${{ steps.context.outputs.tag }}"
+          file_pattern: "CHANGELOG.md docs-site/blog/*.md"
+
+      - name: ✅ Output badge result
+        id: done
+        run: echo "blog_generated=true" >> "$GITHUB_OUTPUT"

package/.github/workflows/release-sync.yml

@@ -0,0 +1,50 @@
+# .github/workflows/release-sync.yml
+# Version: 1.0.0
+# Description: Syncs changelog/tag to blog post
+# Author: Ali Kahwaji
+
+name: 🔄 Release Sync
+
+on:
+  push:
+    tags:
+      - 'v*' # any pushed Git tag
+    paths:
+      - 'CHANGELOG.md'
+      - 'scripts/**'
+      - '.github/workflows/release-sync.yml'
+
+permissions:
+  contents: write
+  id-token: write
+
+jobs:
+  publish-release-blog:
+    name: 📝 Create Blog Post from CHANGELOG + Tag
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: 🧾 Checkout Repo
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 0 # get full commit history for changelog diff
+
+      - name: 🧰 Setup Node
+        uses: actions/setup-node@v4
+        with:
+          node-version: 20
+
+      - name: 📦 Install deps
+        run: npm ci
+
+      - name: 🔁 Generate blog post
+        run: |
+          VERSION=${GITHUB_REF#refs/tags/}
+          PREV=$(git tag --sort=committerdate | grep -B1 $VERSION | head -n1)
+          node scripts/generate-release-note.js $VERSION $PREV
+
+      - name: 💬 Commit release blog
+        uses: stefanzweifel/git-auto-commit-action@v5
+        with:
+          commit_message: "docs(blog): auto-publish release notes for ${{ github.ref_name }}"
+          file_pattern: "docs-site/blog/*.md"

package/.github/workflows/roadmap-label-sync.yml

@@ -0,0 +1,35 @@
+name: 🔖 Sync Roadmap Labels
+
+on:
+  push:
+    paths:
+      - '.github/roadmap-labels.yml'
+      - 'scripts/sync-roadmap-labels.js'
+  workflow_dispatch:
+
+permissions:
+  issues: write
+  contents: read
+
+jobs:
+  sync-labels:
+    runs-on: ubuntu-latest
+    name: Sync Labels from Roadmap Config
+
+    steps:
+      - name: 📥 Checkout repo
+        uses: actions/checkout@v4
+
+      - name: 🛠 Setup Node.js
+        uses: actions/setup-node@v4
+        with:
+          node-version: 20
+          cache: 'npm'
+
+      - name: 📦 Install dependencies
+        run: npm ci
+
+      - name: 🔖 Run label sync script
+        run: node scripts/sync-roadmap-labels.js
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

package/.github/workflows/roadmap-labels.yml

@@ -0,0 +1,67 @@
+# Version: 1.0.0
+# Description: Ensures roadmap labels (priority, group) exist in the repo
+# Author: Ali Kahwaji
+
+name: 🔖 Sync Roadmap Labels
+
+on:
+  workflow_dispatch:
+  push:
+    paths:
+      - '.github/workflows/roadmap-labels.yml'
+
+jobs:
+  sync-labels:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: ⬇️ Checkout repository
+        uses: actions/checkout@v4
+
+      - name: 🟢 Setup Node.js
+        uses: actions/setup-node@v4
+        with:
+          node-version: 20
+
+      - name: 📦 Install octokit
+        run: npm install octokit
+
+      - name: 🔖 Run label sync
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          GITHUB_REPO: ${{ github.repository }}
+        run: |
+          node <<EOF
+          const { Octokit } = require("octokit");
+
+          const octokit = new Octokit({ auth: process.env.GITHUB_TOKEN });
+          const [owner, repo] = process.env.GITHUB_REPO.split("/");
+
+          const labels = [
+            { name: "priority: high", color: "e11d48", description: "High priority roadmap item" },
+            { name: "priority: medium", color: "f59e0b", description: "Medium priority roadmap item" },
+            { name: "priority: low", color: "10b981", description: "Low priority roadmap item" },
+            { name: "group: docs", color: "6366f1", description: "Documentation features" },
+            { name: "group: devx", color: "06b6d4", description: "Developer experience improvements" },
+            { name: "group: community", color: "ec4899", description: "Community tools & engagement" },
+            { name: "group: blog", color: "f97316", description: "Blog & SEO enhancements" },
+            { name: "group: infra", color: "64748b", description: "Infrastructure & deployment features" }
+          ];
+
+          (async () => {
+            const existing = await octokit.rest.issues.listLabelsForRepo({ owner, repo });
+            const existingNames = new Set(existing.data.map(l => l.name));
+
+            for (const label of labels) {
+              if (!existingNames.has(label.name)) {
+                console.log(`➕ Creating label: ${label.name}`);
+                await octokit.rest.issues.createLabel({ owner, repo, ...label });
+              } else {
+                console.log(`✅ Label already exists: ${label.name}`);
+              }
+            }
+          })().catch(err => {
+            console.error("Label sync failed:", err);
+            process.exit(1);
+          });
+          EOF