rag-skills 1.0.0

package/.claude-plugin/marketplace.json

@@ -0,0 +1,14 @@
+{
+  "name": "rag-skills",
+  "owner": {
+    "name": "Mohamed Arbi ",
+    "url": "https://github.com/Goodnight77"
+  },
+  "plugins": [
+    {
+      "name": "rag-skills",
+      "source": "./",
+      "description": "Agent skills for RAG: chunking strategies, retrieval methods, vector databases, and performance optimization"
+    }
+  ]
+}

package/.claude-plugin/plugin.json

@@ -0,0 +1,8 @@
+{
+  "name": "rag-skills",
+  "description": "Agent skills for RAG (Retrieval Augmented Generation): chunking strategies (sliding window, semantic, hierarchical), retrieval strategies (HyDE, CRAG, Self-RAG, Graph RAG, adaptive, multi-pass), vector database setup (Qdrant), data type handling (code, multimodal), and performance optimization",
+  "version": "1.0.0",
+  "author": {
+    "name": "Mohamed Arbi"
+  }
+}

package/CONTRIBUTING.md

@@ -0,0 +1,210 @@
+# Contributing to rag-skills
+
+Thank you for your interest in contributing to rag-skills! This document provides guidelines for submitting new skills, reviewing existing skills, and maintaining the repository.
+
+## Table of Contents
+
+- [Code of Conduct](#code-of-conduct)
+- [Getting Started](#getting-started)
+- [Submitting New Skills](#submitting-new-skills)
+- [Skill Review Criteria](#skill-review-criteria)
+- [Naming Conventions](#naming-conventions)
+- [Testing and Validation](#testing-and-validation)
+- [Credit and Attribution](#credit-and-attribution)
+- [Reporting Issues](#reporting-issues)
+
+## Code of Conduct
+
+- Be respectful and inclusive
+- Provide constructive feedback
+- Focus on what is best for the community
+- Show empathy towards other community members
+
+## Getting Started
+
+### Development Setup
+
+### Running Validation
+
+## Submitting New Skills
+
+### Step 1: Choose Your Topic
+
+Before creating a new skill, check that:
+
+1. The topic is not already covered by an existing skill
+2. The topic is relevant to RAG systems
+3. You have practical experience with the topic
+
+### Step 2: Use the Template
+
+Start with [templates/skill-template.md](templates/skill-template.md):
+Use the template as a lightweight reference format: brief illustrative text, no long runnable code, and 3-5 external implementation links folded into the relevant sections instead of a separate `References` block.
+
+### Step 3: Place Your Skill
+
+Organize skills by category:
+
+If your category doesn't exist, create it in the appropriate location.
+
+### Step 4: Validate Your Skill
+
+Run the validation script and fix any errors before submitting.
+
+### Step 5: Submit a Pull Request
+
+1. Commit your changes
+2. Push to your fork
+3. Open a pull request with a descriptive title
+
+Example PR title: `Add skill: Semantic Chunking for Markdown Documents`
+
+## Skill Review Criteria
+
+When reviewing skills, maintainers evaluate them against these criteria:
+
+### Clarity
+
+- [ ] Is the problem statement clear and specific?
+- [ ] Are key concepts well-defined?
+- [ ] Are implementation steps logically ordered?
+- [ ] Is the writing free of ambiguity?
+
+### Accuracy
+
+- [ ] Are the technical statements correct?
+- [ ] Do code examples run as expected?
+- [ ] Are references valid and current?
+- [ ] Are the metrics/success criteria realistic?
+
+### Completeness
+
+- [ ] All required sections are present
+- [ ] Code examples are brief and illustrative, not full implementations
+- [ ] Related skills are properly linked
+- [ ] Both use cases and anti-patterns are covered
+
+### Practicality
+
+- [ ] The skill addresses a real-world problem
+- [ ] The approach is production-viable
+- [ ] The complexity matches the difficulty level
+- [ ] Dependencies are reasonable and well-known
+
+### Code Quality
+
+- [ ] Code follows Python conventions (PEP 8)
+- [ ] Code includes comments for complex logic
+- [ ] Code handles errors appropriately
+- [ ] Code examples stay lightweight and defer to external implementations
+
+## Naming Conventions
+
+### Skill Files
+
+- Use kebab-case: `semantic-chunking.md`
+- Be descriptive: `hybrid-search-bm25-dense.md`
+- Keep names under 50 characters
+
+### Categories
+
+Use existing category names:
+- `chunking`
+- `vector-databases`
+- `retrieval-strategies`
+- `data-type-handling`
+- `performance-optimization`
+- `evaluation-metrics`
+- `rag-agents`
+- `deployment`
+
+### Levels
+
+Use one of:
+- `beginner` — Basic concepts, minimal dependencies
+- `intermediate` — Some experience required, moderate complexity
+- `advanced` — Expert knowledge, complex implementations
+
+If you keep the level field in a skill file, treat it as metadata for filtering rather than a required browsing path in the README.
+
+### Tags
+
+- Use 3-5 relevant tags per skill
+- Use lowercase: `["semantic", "nlp", "context"]`
+- Avoid overly specific tags
+- Focus on searchable terms
+
+## Testing and Validation
+
+### Local Validation
+
+Before submitting, ensure your skill passes validation:
+
+Strict mode treats warnings as errors.
+
+### Link Validation
+
+Verify all internal links work and that external implementation links are placed inline in the relevant step or sentence:
+
+## Credit and Attribution
+
+### Author Field
+
+Include your name in the `author` field:
+
+For organizational contributions:
+
+### Last Updated
+
+Update the `last_updated` field with the current date:
+
+### Co-Authors
+
+For substantial contributions from multiple people, list them in the pull request description:
+
+## Reporting Issues
+
+When reporting issues, include:
+
+- A clear title
+- Steps to reproduce
+- Expected vs actual behavior
+- Environment information
+- Screenshots if applicable
+
+### Issue Templates
+
+#### Bug Report
+
+#### Feature Request
+
+## Community Guidelines
+
+### Discussions
+
+- Use GitHub Discussions for questions and ideas
+- Be specific in your questions
+- Share code snippets when helpful
+- Follow up on responses
+
+### Code Review
+
+- Be constructive in your reviews
+- Explain the reasoning for suggested changes
+- Acknowledge good work
+- Be patient with maintainers' time
+
+### Maintainer Response Time
+
+Maintainers aim to respond to:
+- Pull requests: Within 7 days
+- Issues: Within 7 days
+- Discussions: Within 3 days
+
+## Additional Resources
+
+- [Project README](README.md)
+- [Skill Template](templates/skill-template.md)
+- [GitHub Community Guidelines](https://docs.github.com/en/site-policy/github-terms/github-community-guidelines)
+
+Thank you for contributing to rag-skills!

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 rag-skills contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,148 @@
+# Rag-skills
+
+<p>
+  <code>agent routing</code> <code>RAG skills</code> <code>markdown</code>
+</p>
+
+A modular collection of best-practice guides and skill definitions for building Retrieval-Augmented Generation (RAG) systems. Designed for AI coding agents, agent frameworks, and teams that want a structured way to route RAG work to the right strategy.
+
+## Overview
+
+RAG-skills consolidates actionable skills that help AI agents and builders improve RAG performance, choose appropriate vector databases, implement effective chunking strategies, optimize retrieval quality, and orchestrate multi-step RAG workflows.
+
+## Installation
+
+### Via Claude Code (Local)
+
+Clone and use as a plugin:
+
+```bash
+# Clone to your plugins directory
+git clone https://github.com/Goodnight77/Rag-skills.git ~/.claude/plugins/rag-skills
+
+# Or use from current directory
+claude --plugin-dir .
+```
+
+### Via npm 
+```bash
+npx skills add @goodnight/rag-skills
+```
+
+### Manual Usage
+
+Clone or fork the repository and reference skill files directly in your agent workflows.
+
+## Skills by Decision Area
+
+This repo is organized as a routing layer for RAG work. Agents can use the category and metadata in each skill file to decide which path to follow for a given problem, instead of treating the repo like a generic reference manual.
+
+### Chunking
+Use these when the main problem is how to split source material into retrievable units.
+- [Semantic Chunking](skills/chunking/semantic-chunking.md) - Chunk documents based on semantic boundaries
+- [Hierarchical Chunking](skills/chunking/hierarchical-chunking.md) - Multi-level chunking for nested structures
+- [Sliding Window Chunking](skills/chunking/sliding-window-chunking.md) - Overlap-based chunking for context preservation
+- [Contextual Chunk Headers](skills/chunking/contextual-chunk-headers.md) - Adding higher-level context to chunks
+
+### Vector Databases
+Use these when the main problem is choosing or operating the storage layer for embeddings and metadata.
+- [Qdrant Setup for RAG](skills/vector-databases/qdrant-setup-rag.md) - Setting up Qdrant for RAG
+- [Qdrant for Production RAG](skills/vector-databases/qdrant-for-production-rag.md) - Scaling RAG with Qdrant
+- [Choosing Vector DB by Datatype](skills/vector-databases/choosing-vector-db-by-datatype.md) - Database selection guide
+
+### Retrieval Strategies
+Use these when the main problem is search quality, ranking, recall, or combining search methods.
+- [Hybrid Search BM25 Dense](skills/retrieval-strategies/hybrid-search-bm25-dense.md) - Combining keyword and semantic search
+- [Multi-Pass Retrieval with Reranking](skills/retrieval-strategies/multi-pass-retrieval-with-reranking.md) - Two-pass retrieval with cross-encoder reranking
+- [Query Transformation Strategies](skills/retrieval-strategies/query-transformation-strategies.md) - Query rewriting, step-back prompting, sub-query decomposition
+- [HyDE - Hypothetical Document Embeddings](skills/retrieval-strategies/hyde-hypothetical-document-embeddings.md) - Query expansion with LLM-generated documents
+- [HyPE - Hypothetical Prompt Embeddings](skills/retrieval-strategies/hype-hypothetical-prompt-embeddings.md) - Precomputed question embeddings at indexing time
+- [Self-RAG](skills/retrieval-strategies/self-rag.md) - Self-reflective retrieval with relevance evaluation
+- [RAPTOR - Hierarchical Retrieval](skills/retrieval-strategies/raptor-hierarchical-retrieval.md) - Multi-level tree of document summaries
+- [Context Enrichment Window](skills/retrieval-strategies/context-enrichment-window.md) - Adding surrounding chunks to retrieved results
+- [Adaptive Retrieval](skills/retrieval-strategies/adaptive-retrieval.md) - Dynamic strategy selection based on query type
+- [Explainable Retrieval with Citations](skills/retrieval-strategies/explainable-retrieval.md) - Traceability and source attribution
+- [CRAG - Corrective RAG](skills/retrieval-strategies/crag-corrective-rag.md) - Dynamic correction with web search
+- [Graph RAG](skills/retrieval-strategies/graph-rag.md) - Knowledge graph-based retrieval
+
+### Data Type Handling
+Use these when the source content is code, APIs, diagrams, tables, or mixed media.
+- [RAG for Code Documentation](skills/data-type-handling/rag-for-code-documentation.md) - Special handling for code and technical docs
+- [RAG for Multimodal Content](skills/data-type-handling/rag-for-multimodal-content.md) - Images, tables, and mixed media
+
+### Performance Optimization
+Use these when the problem is latency, throughput, cache behavior, or production efficiency.
+- [Optimize Retrieval Latency](skills/performance-optimization/optimize-retrieval-latency.md) - Caching, indexing, and query optimization
+
+### RAG Agents
+Use these when the problem is orchestration, delegation, or multi-step workflows.
+- *See [Examples](#examples) for multi-agent workflows*
+
+### Deployment
+Use these when the problem is production rollout, reliability, or operationalization.
+- *See [Production RAG Setup](#examples)*
+
+### Evaluation Metrics
+Use these when the problem is measurement, regression detection, or retrieval benchmarking.
+- *Coming soon*
+
+## Quick Start
+
+### For AI Agents
+
+Read the frontmatter metadata, then route to the skill that best matches the user’s problem. Treat the repo as a decision tree for RAG tasks: chunking, retrieval, vector store choice, embeddings, performance, and workflow orchestration.
+
+### For Framework Integration
+
+Build a lightweight index from the markdown frontmatter and use it to filter by category, tags, and task type. The goal is not to mirror all content in code, but to point an agent to the right skill or external implementation quickly.
+
+Keep examples in the repo lightweight and point readers to external implementations instead of embedding long code samples.
+
+## Examples
+
+Complete walkthroughs and reference implementations:
+
+- [Foundational RAG Pipeline Example](examples/foundational-rag-pipeline.md) - A guided RAG build path for agents and builders
+- [Multi-Agent RAG](examples/multi-agent-rag.md) - An orchestration pattern for specialized agents
+- [Production RAG Setup](examples/production-rag-setup.md) - A deployment-oriented route for production systems
+
+## Contributing
+
+We welcome contributions! See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.
+
+### Quick Contribution Steps
+
+1. Fork the repository
+2. Create a new skill file using [templates/skill-template.md](templates/skill-template.md)
+3. Ensure your skill follows the required structure
+4. Run validation: `python scripts/validate-skills.py`
+5. Submit a pull request
+
+## Skill File Format
+
+Each skill follows a consistent structure with a short illustrative snippet, not a full implementation. See the template in [templates/skill-template.md](templates/skill-template.md).
+
+## Scripts
+
+- `validate-skills.py` — Validate all skill files for format compliance
+- `generate-index.py` — Generate browsable INDEX.md and SKILLS.json
+
+## Project Status
+
+This is an active open-source project. Skills are continuously added and updated as RAG best practices evolve.
+
+Current statistics:
+- **Total Skills**: 21
+- **Categories**: 7
+- **Examples**: 3
+
+*Run `python scripts/generate-index.py` for current statistics.*
+
+## Acknowledgments
+
+Built for the RAG community. Special thanks to contributors and the open-source RAG ecosystem.
+
+
+## License
+
+MIT License — see [LICENSE](LICENSE) for details.

package/examples/foundational-rag-pipeline.md

@@ -0,0 +1,104 @@
+---
+title: "Foundational RAG Pipeline Example"
+category: "rag-agents"
+tags: ["workflow", "quickstart", "pipeline"]
+author: "rag-skills"
+---
+
+## Overview
+This example shows the minimum viable shape of a RAG pipeline without turning the repository into an implementation repo. Use it as a reference workflow for agents and developers who need to understand the sequence of stages, the key decisions, and where to look for production-ready code.
+
+## Goal
+Build a simple question-answering workflow that:
+1. Loads source documents
+2. Chunks and embeds them
+3. Stores them in a vector database
+4. Retrieves relevant context at query time
+5. Generates an answer grounded in retrieved passages
+
+## Recommended Workflow
+
+### Stage 1: Ingestion
+- Load source documents from files, docs platforms, or APIs
+- Normalize text and preserve useful metadata such as title, URL, section, and timestamp
+- Apply a chunking strategy that matches the corpus
+
+### Stage 2: Indexing
+- Generate embeddings for each chunk
+- Store vectors, chunk text, and metadata in a vector database
+- Validate that filtering and retrieval work before moving on
+
+### Stage 3: Retrieval
+- Embed the user query
+- Retrieve the top candidate chunks
+- Optionally rerank or filter results before generation
+
+### Stage 4: Answer Generation
+- Pass the query and retrieved context to the model
+- Require grounded answers and source attribution
+- Return both answer text and the supporting chunks
+
+## Reference Architecture
+
+```text
+Documents -> Loader -> Chunker -> Embedder -> Vector DB
+User Query -> Query Embedder -> Retriever -> Prompt Builder -> LLM -> Answer + Sources
+```
+
+## Pseudocode
+
+```text
+load documents
+chunk documents with metadata preserved
+embed chunks
+store chunks + embeddings in vector database
+
+for each user query:
+  embed query
+  retrieve top-k chunks
+  optionally rerank
+  generate answer from retrieved context
+  return answer with sources
+```
+
+## Design Choices
+
+### Keep It Simple First
+- Start with one loader, one chunking strategy, one embedding model, and one vector store
+- Avoid hybrid retrieval and agent orchestration until the baseline is working
+
+### Preserve Metadata Early
+- Store source, section, and document identifiers during ingestion
+- This enables filtering, citations, debugging, and later migration
+
+### Measure Before Optimizing
+- Check whether the retrieved chunks are actually relevant before tuning prompts
+- Most early RAG failures are indexing or chunking problems, not generation problems
+
+## Common Failure Modes
+- Chunk size is too large, so retrieval returns noisy context
+- Chunk size is too small, so meaning gets fragmented
+- Metadata is missing, so debugging and citations are weak
+- Retrieval is evaluated only by final answer quality instead of retrieval quality
+- The pipeline uses synthetic examples that are too easy and hide failure cases
+
+## When to Use This Example
+- As the first pipeline shape for a new RAG application
+- As the baseline architecture before adding reranking or agents
+- As a teaching document for junior contributors or coding agents
+
+## When NOT to Use This Example
+- When the system already requires hybrid retrieval or multimodal ingestion
+- When the main problem is orchestration across multiple tools or agents
+- When the corpus depends heavily on layout parsing, code structure, or hierarchical retrieval
+
+## External Implementations
+- [LangChain RAG guide](https://docs.langchain.com/oss/python/langchain/rag)
+- [LlamaIndex Introduction to RAG](https://docs.llamaindex.ai/en/stable/understanding/rag/)
+- [Qdrant quickstart](https://qdrant.tech/documentation/quickstart/)
+- [OpenAI cookbook RAG examples](https://github.com/openai/openai-cookbook/tree/main/examples)
+
+## Related Skills
+- [Semantic Chunking](../skills/chunking/semantic-chunking.md)
+- [Qdrant Setup for RAG](../skills/vector-databases/qdrant-setup-rag.md)
+- [Hybrid Search BM25 Dense](../skills/retrieval-strategies/hybrid-search-bm25-dense.md)

package/examples/multi-agent-rag.md

@@ -0,0 +1,111 @@
+---
+title: "Multi-Agent RAG System"
+category: "rag-agents"
+level: "advanced"
+tags: ["multi-agent", "orchestration", "workflow", "collaboration"]
+author: "rag-skills"
+last_updated: "2026-04-07"
+---
+
+## Overview
+This example describes how a multi-agent RAG system should be structured without embedding a large implementation directly in the repository. Use it when a single retrieve-then-answer flow is no longer sufficient and different agent roles need to collaborate.
+
+## Goal
+Coordinate specialized agents so that query understanding, retrieval, reranking, synthesis, and validation can be separated into distinct responsibilities.
+
+## Reference Architecture
+
+```text
+User Query
+  -> Orchestrator
+    -> Query Analysis Agent
+    -> Retrieval Agent
+    -> Reranking Agent
+    -> Synthesis Agent
+    -> Optional Critique / Verification Agent
+  -> Final Answer + Sources + Execution Trace
+```
+
+## Agent Responsibilities
+
+### Query Analysis Agent
+- Classify the query type
+- Decide whether the request needs plain retrieval, comparison, troubleshooting, or decomposition
+- Select the workflow path
+
+### Retrieval Agent
+- Execute first-pass retrieval
+- Return candidate chunks plus retrieval metadata
+- Prefer recall over precision at this stage
+
+### Reranking Agent
+- Reorder the candidate set using stronger relevance logic
+- Drop low-value chunks before generation
+- Improve precision without forcing the retriever to be overly strict
+
+### Synthesis Agent
+- Build the final grounded answer
+- Cite the chunks or documents used
+- Surface uncertainty when evidence is weak
+
+### Optional Critique Agent
+- Check grounding, coverage, and contradiction risk
+- Flag weak evidence or missing context
+- Trigger another retrieval pass when needed
+
+## Pseudocode
+
+```text
+analyze query
+choose workflow
+retrieve broad candidate set
+rerank candidates
+synthesize grounded answer
+optionally critique and retry
+return answer, sources, and trace
+```
+
+## When Multi-Agent RAG Is Worth It
+- Queries vary widely in type and complexity
+- The cost of retrieval mistakes is high
+- The system needs traceability across steps
+- Different tools or retrieval paths need explicit coordination
+- The team wants modular components that can evolve independently
+
+## When It Is Not Worth It
+- The workload is simple FAQ or narrow-domain Q&A
+- The main bottleneck is chunking or indexing quality
+- Latency budgets are tight and multiple stages are too expensive
+- The team cannot monitor or debug a more complex workflow
+
+## Operational Guidance
+
+### Start With a Single-Agent Baseline
+- Prove that the corpus, chunking, and retrieval are sound first
+- Add extra agents only after identifying a real failure pattern
+
+### Keep Agent Boundaries Clear
+- Each agent should own one job
+- Avoid agents that retrieve, reason, rerank, and answer all at once
+
+### Log the Workflow
+- Record which agent ran, what it produced, and why the next step happened
+- Multi-agent systems are only useful if they stay debuggable
+
+## Common Failure Modes
+- Too many agents with overlapping roles
+- Orchestrator logic that is harder to understand than the retrieval problem
+- Repeated retrieval loops without clear stopping rules
+- Reranking and synthesis both trying to decide relevance independently
+- No workflow trace, making failure analysis impossible
+
+## External Implementations
+- [LangGraph agentic RAG guide](https://docs.langchain.com/oss/python/langgraph/agentic-rag)
+- [LangChain multi-agent docs](https://docs.langchain.com/oss/python/langchain/multi-agent)
+- [LlamaIndex multi-agent patterns](https://developers.llamaindex.ai/python/framework/understanding/putting_it_all_together/agents/)
+- [Microsoft AutoGen documentation](https://microsoft.github.io/autogen/stable/)
+
+## Related Skills
+- [Multi-Pass Retrieval with Reranking](../skills/retrieval-strategies/multi-pass-retrieval-with-reranking.md)
+- [Optimize Retrieval Latency](../skills/performance-optimization/optimize-retrieval-latency.md)
+- [RAG for Code Documentation](../skills/data-type-handling/rag-for-code-documentation.md)