@ngxtm/devkit 3.7.0 → 3.8.0

package/merged-commands/rag-implementation.md

@@ -0,0 +1,421 @@
+---
+name: rag-implementation
+description: Build Retrieval-Augmented Generation (RAG) systems for LLM applications with vector databases and semantic search. Use when implementing knowledge-grounded AI, building document Q&A systems, or integrating LLMs with external knowledge bases.
+---
+
+# RAG Implementation
+
+Master Retrieval-Augmented Generation (RAG) to build LLM applications that provide accurate, grounded responses using external knowledge sources.
+
+## Use this skill when
+
+- Building Q&A systems over proprietary documents
+- Creating chatbots with current, factual information
+- Implementing semantic search with natural language queries
+- Reducing hallucinations with grounded responses
+- Enabling LLMs to access domain-specific knowledge
+- Building documentation assistants
+- Creating research tools with source citation
+
+## Do not use this skill when
+
+- You only need purely generative writing without retrieval
+- The dataset is too small to justify embeddings
+- You cannot store or process the source data safely
+
+## Instructions
+
+1. Define the corpus, update cadence, and evaluation targets.
+2. Choose embedding models and vector store based on scale.
+3. Build ingestion, chunking, and retrieval with reranking.
+4. Evaluate with grounded QA metrics and monitor drift.
+
+## Safety
+
+- Redact sensitive data and enforce access controls.
+- Avoid exposing source documents in responses when restricted.
+
+## Core Components
+
+### 1. Vector Databases
+**Purpose**: Store and retrieve document embeddings efficiently
+
+**Options:**
+- **Pinecone**: Managed, scalable, fast queries
+- **Weaviate**: Open-source, hybrid search
+- **Milvus**: High performance, on-premise
+- **Chroma**: Lightweight, easy to use
+- **Qdrant**: Fast, filtered search
+- **FAISS**: Meta's library, local deployment
+
+### 2. Embeddings
+**Purpose**: Convert text to numerical vectors for similarity search
+
+**Models:**
+- **text-embedding-ada-002** (OpenAI): General purpose, 1536 dims
+- **all-MiniLM-L6-v2** (Sentence Transformers): Fast, lightweight
+- **e5-large-v2**: High quality, multilingual
+- **Instructor**: Task-specific instructions
+- **bge-large-en-v1.5**: SOTA performance
+
+### 3. Retrieval Strategies
+**Approaches:**
+- **Dense Retrieval**: Semantic similarity via embeddings
+- **Sparse Retrieval**: Keyword matching (BM25, TF-IDF)
+- **Hybrid Search**: Combine dense + sparse
+- **Multi-Query**: Generate multiple query variations
+- **HyDE**: Generate hypothetical documents
+
+### 4. Reranking
+**Purpose**: Improve retrieval quality by reordering results
+
+**Methods:**
+- **Cross-Encoders**: BERT-based reranking
+- **Cohere Rerank**: API-based reranking
+- **Maximal Marginal Relevance (MMR)**: Diversity + relevance
+- **LLM-based**: Use LLM to score relevance
+
+## Quick Start
+
+```python
+from langchain.document_loaders import DirectoryLoader
+from langchain.text_splitters import RecursiveCharacterTextSplitter
+from langchain.embeddings import OpenAIEmbeddings
+from langchain.vectorstores import Chroma
+from langchain.chains import RetrievalQA
+from langchain.llms import OpenAI
+
+# 1. Load documents
+loader = DirectoryLoader('./docs', glob="**/*.txt")
+documents = loader.load()
+
+# 2. Split into chunks
+text_splitter = RecursiveCharacterTextSplitter(
+    chunk_size=1000,
+    chunk_overlap=200,
+    length_function=len
+)
+chunks = text_splitter.split_documents(documents)
+
+# 3. Create embeddings and vector store
+embeddings = OpenAIEmbeddings()
+vectorstore = Chroma.from_documents(chunks, embeddings)
+
+# 4. Create retrieval chain
+qa_chain = RetrievalQA.from_chain_type(
+    llm=OpenAI(),
+    chain_type="stuff",
+    retriever=vectorstore.as_retriever(search_kwargs={"k": 4}),
+    return_source_documents=True
+)
+
+# 5. Query
+result = qa_chain({"query": "What are the main features?"})
+print(result['result'])
+print(result['source_documents'])
+```
+
+## Advanced RAG Patterns
+
+### Pattern 1: Hybrid Search
+```python
+from langchain.retrievers import BM25Retriever, EnsembleRetriever
+
+# Sparse retriever (BM25)
+bm25_retriever = BM25Retriever.from_documents(chunks)
+bm25_retriever.k = 5
+
+# Dense retriever (embeddings)
+embedding_retriever = vectorstore.as_retriever(search_kwargs={"k": 5})
+
+# Combine with weights
+ensemble_retriever = EnsembleRetriever(
+    retrievers=[bm25_retriever, embedding_retriever],
+    weights=[0.3, 0.7]
+)
+```
+
+### Pattern 2: Multi-Query Retrieval
+```python
+from langchain.retrievers.multi_query import MultiQueryRetriever
+
+# Generate multiple query perspectives
+retriever = MultiQueryRetriever.from_llm(
+    retriever=vectorstore.as_retriever(),
+    llm=OpenAI()
+)
+
+# Single query → multiple variations → combined results
+results = retriever.get_relevant_documents("What is the main topic?")
+```
+
+### Pattern 3: Contextual Compression
+```python
+from langchain.retrievers import ContextualCompressionRetriever
+from langchain.retrievers.document_compressors import LLMChainExtractor
+
+compressor = LLMChainExtractor.from_llm(llm)
+
+compression_retriever = ContextualCompressionRetriever(
+    base_compressor=compressor,
+    base_retriever=vectorstore.as_retriever()
+)
+
+# Returns only relevant parts of documents
+compressed_docs = compression_retriever.get_relevant_documents("query")
+```
+
+### Pattern 4: Parent Document Retriever
+```python
+from langchain.retrievers import ParentDocumentRetriever
+from langchain.storage import InMemoryStore
+
+# Store for parent documents
+store = InMemoryStore()
+
+# Small chunks for retrieval, large chunks for context
+child_splitter = RecursiveCharacterTextSplitter(chunk_size=400)
+parent_splitter = RecursiveCharacterTextSplitter(chunk_size=2000)
+
+retriever = ParentDocumentRetriever(
+    vectorstore=vectorstore,
+    docstore=store,
+    child_splitter=child_splitter,
+    parent_splitter=parent_splitter
+)
+```
+
+## Document Chunking Strategies
+
+### Recursive Character Text Splitter
+```python
+from langchain.text_splitters import RecursiveCharacterTextSplitter
+
+splitter = RecursiveCharacterTextSplitter(
+    chunk_size=1000,
+    chunk_overlap=200,
+    length_function=len,
+    separators=["\n\n", "\n", " ", ""]  # Try these in order
+)
+```
+
+### Token-Based Splitting
+```python
+from langchain.text_splitters import TokenTextSplitter
+
+splitter = TokenTextSplitter(
+    chunk_size=512,
+    chunk_overlap=50
+)
+```
+
+### Semantic Chunking
+```python
+from langchain.text_splitters import SemanticChunker
+
+splitter = SemanticChunker(
+    embeddings=OpenAIEmbeddings(),
+    breakpoint_threshold_type="percentile"
+)
+```
+
+### Markdown Header Splitter
+```python
+from langchain.text_splitters import MarkdownHeaderTextSplitter
+
+headers_to_split_on = [
+    ("#", "Header 1"),
+    ("##", "Header 2"),
+    ("###", "Header 3"),
+]
+
+splitter = MarkdownHeaderTextSplitter(headers_to_split_on=headers_to_split_on)
+```
+
+## Vector Store Configurations
+
+### Pinecone
+```python
+import pinecone
+from langchain.vectorstores import Pinecone
+
+pinecone.init(api_key="your-api-key", environment="us-west1-gcp")
+
+index = pinecone.Index("your-index-name")
+
+vectorstore = Pinecone(index, embeddings.embed_query, "text")
+```
+
+### Weaviate
+```python
+import weaviate
+from langchain.vectorstores import Weaviate
+
+client = weaviate.Client("http://localhost:8080")
+
+vectorstore = Weaviate(client, "Document", "content", embeddings)
+```
+
+### Chroma (Local)
+```python
+from langchain.vectorstores import Chroma
+
+vectorstore = Chroma(
+    collection_name="my_collection",
+    embedding_function=embeddings,
+    persist_directory="./chroma_db"
+)
+```
+
+## Retrieval Optimization
+
+### 1. Metadata Filtering
+```python
+# Add metadata during indexing
+chunks_with_metadata = []
+for i, chunk in enumerate(chunks):
+    chunk.metadata = {
+        "source": chunk.metadata.get("source"),
+        "page": i,
+        "category": determine_category(chunk.page_content)
+    }
+    chunks_with_metadata.append(chunk)
+
+# Filter during retrieval
+results = vectorstore.similarity_search(
+    "query",
+    filter={"category": "technical"},
+    k=5
+)
+```
+
+### 2. Maximal Marginal Relevance
+```python
+# Balance relevance with diversity
+results = vectorstore.max_marginal_relevance_search(
+    "query",
+    k=5,
+    fetch_k=20,  # Fetch 20, return top 5 diverse
+    lambda_mult=0.5  # 0=max diversity, 1=max relevance
+)
+```
+
+### 3. Reranking with Cross-Encoder
+```python
+from sentence_transformers import CrossEncoder
+
+reranker = CrossEncoder('cross-encoder/ms-marco-MiniLM-L-6-v2')
+
+# Get initial results
+candidates = vectorstore.similarity_search("query", k=20)
+
+# Rerank
+pairs = [[query, doc.page_content] for doc in candidates]
+scores = reranker.predict(pairs)
+
+# Sort by score and take top k
+reranked = sorted(zip(candidates, scores), key=lambda x: x[1], reverse=True)[:5]
+```
+
+## Prompt Engineering for RAG
+
+### Contextual Prompt
+```python
+prompt_template = """Use the following context to answer the question. If you cannot answer based on the context, say "I don't have enough information."
+
+Context:
+{context}
+
+Question: {question}
+
+Answer:"""
+```
+
+### With Citations
+```python
+prompt_template = """Answer the question based on the context below. Include citations using [1], [2], etc.
+
+Context:
+{context}
+
+Question: {question}
+
+Answer (with citations):"""
+```
+
+### With Confidence
+```python
+prompt_template = """Answer the question using the context. Provide a confidence score (0-100%) for your answer.
+
+Context:
+{context}
+
+Question: {question}
+
+Answer:
+Confidence:"""
+```
+
+## Evaluation Metrics
+
+```python
+def evaluate_rag_system(qa_chain, test_cases):
+    metrics = {
+        'accuracy': [],
+        'retrieval_quality': [],
+        'groundedness': []
+    }
+
+    for test in test_cases:
+        result = qa_chain({"query": test['question']})
+
+        # Check if answer matches expected
+        accuracy = calculate_accuracy(result['result'], test['expected'])
+        metrics['accuracy'].append(accuracy)
+
+        # Check if relevant docs were retrieved
+        retrieval_quality = evaluate_retrieved_docs(
+            result['source_documents'],
+            test['relevant_docs']
+        )
+        metrics['retrieval_quality'].append(retrieval_quality)
+
+        # Check if answer is grounded in context
+        groundedness = check_groundedness(
+            result['result'],
+            result['source_documents']
+        )
+        metrics['groundedness'].append(groundedness)
+
+    return {k: sum(v)/len(v) for k, v in metrics.items()}
+```
+
+## Resources
+
+- **references/vector-databases.md**: Detailed comparison of vector DBs
+- **references/embeddings.md**: Embedding model selection guide
+- **references/retrieval-strategies.md**: Advanced retrieval techniques
+- **references/reranking.md**: Reranking methods and when to use them
+- **references/context-window.md**: Managing context limits
+- **assets/vector-store-config.yaml**: Configuration templates
+- **assets/retriever-pipeline.py**: Complete RAG pipeline
+- **assets/embedding-models.md**: Model comparison and benchmarks
+
+## Best Practices
+
+1. **Chunk Size**: Balance between context and specificity (500-1000 tokens)
+2. **Overlap**: Use 10-20% overlap to preserve context at boundaries
+3. **Metadata**: Include source, page, timestamp for filtering and debugging
+4. **Hybrid Search**: Combine semantic and keyword search for best results
+5. **Reranking**: Improve top results with cross-encoder
+6. **Citations**: Always return source documents for transparency
+7. **Evaluation**: Continuously test retrieval quality and answer accuracy
+8. **Monitoring**: Track retrieval metrics in production
+
+## Common Issues
+
+- **Poor Retrieval**: Check embedding quality, chunk size, query formulation
+- **Irrelevant Results**: Add metadata filtering, use hybrid search, rerank
+- **Missing Information**: Ensure documents are properly indexed
+- **Slow Queries**: Optimize vector store, use caching, reduce k
+- **Hallucinations**: Improve grounding prompt, add verification step

package/merged-commands/rails-expert.md

@@ -0,0 +1,97 @@
+---
+name: rails-expert
+description: Use when building Rails 7+ web applications with Hotwire, real-time features, or background job processing. Invoke for Active Record optimization, Turbo Frames/Streams, Action Cable, Sidekiq.
+triggers:
+  - Rails
+  - Ruby on Rails
+  - Hotwire
+  - Turbo Frames
+  - Turbo Streams
+  - Action Cable
+  - Active Record
+  - Sidekiq
+  - RSpec Rails
+role: specialist
+scope: implementation
+output-format: code
+---
+
+# Rails Expert
+
+Senior Rails specialist with deep expertise in Rails 7+, Hotwire, and modern Ruby web development patterns.
+
+## Role Definition
+
+You are a senior Ruby on Rails engineer with 10+ years of Rails development experience. You specialize in Rails 7+ with Hotwire/Turbo, convention over configuration, and building maintainable applications. You prioritize developer happiness and rapid development.
+
+## When to Use This Skill
+
+- Building Rails 7+ applications with modern patterns
+- Implementing Hotwire/Turbo for reactive UIs
+- Setting up Action Cable for real-time features
+- Implementing background jobs with Sidekiq
+- Optimizing Active Record queries and performance
+- Writing comprehensive RSpec test suites
+
+## Core Workflow
+
+1. **Analyze requirements** - Identify models, routes, real-time needs, background jobs
+2. **Design architecture** - Plan MVC structure, associations, service objects
+3. **Implement** - Generate resources, write controllers, add Hotwire
+4. **Optimize** - Prevent N+1 queries, add caching, optimize assets
+5. **Test** - Write model/request/system specs with high coverage
+
+## Reference Guide
+
+Load detailed guidance based on context:
+
+| Topic | Reference | Load When |
+|-------|-----------|-----------|
+| Hotwire/Turbo | `references/hotwire-turbo.md` | Turbo Frames, Streams, Stimulus controllers |
+| Active Record | `references/active-record.md` | Models, associations, queries, performance |
+| Background Jobs | `references/background-jobs.md` | Sidekiq, job design, queues, error handling |
+| Testing | `references/rspec-testing.md` | Model/request/system specs, factories |
+| API Development | `references/api-development.md` | API-only mode, serialization, authentication |
+
+## Constraints
+
+### MUST DO
+- Follow Rails conventions (convention over configuration)
+- Use RESTful routing and resourceful controllers
+- Prevent N+1 queries (use includes/eager_load)
+- Write comprehensive specs (aim for >95% coverage)
+- Use strong parameters for mass assignment protection
+- Implement proper error handling and validations
+- Use service objects for complex business logic
+- Keep controllers thin, models focused
+
+### MUST NOT DO
+- Skip migrations for schema changes
+- Store sensitive data unencrypted
+- Use raw SQL without sanitization
+- Skip CSRF protection
+- Expose internal IDs in URLs without consideration
+- Use synchronous operations for slow tasks
+- Skip database indexes for queried columns
+- Mix business logic in controllers
+
+## Output Templates
+
+When implementing Rails features, provide:
+1. Migration file (if schema changes needed)
+2. Model file with associations and validations
+3. Controller with RESTful actions
+4. View files or Hotwire setup
+5. Spec files for models and requests
+6. Brief explanation of architectural decisions
+
+## Knowledge Reference
+
+Rails 7+, Hotwire/Turbo, Stimulus, Action Cable, Active Record, Sidekiq, RSpec, FactoryBot, Capybara, ViewComponent, Kredis, Import Maps, Tailwind CSS, PostgreSQL
+
+## Related Skills
+
+- **Fullstack Guardian** - Full-stack feature implementation
+- **Ruby Specialist** - Ruby language optimization
+- **Database Optimizer** - Query and schema optimization
+- **Performance Engineer** - Application performance tuning

package/merged-commands/react-best-practices.md

@@ -0,0 +1,121 @@
+---
+name: vercel-react-best-practices
+description: React and Next.js performance optimization guidelines from Vercel Engineering. This skill should be used when writing, reviewing, or refactoring React/Next.js code to ensure optimal performance patterns. Triggers on tasks involving React components, Next.js pages, data fetching, bundle optimization, or performance improvements.
+---
+
+# Vercel React Best Practices
+
+Comprehensive performance optimization guide for React and Next.js applications, maintained by Vercel. Contains 45 rules across 8 categories, prioritized by impact to guide automated refactoring and code generation.
+
+## When to Apply
+
+Reference these guidelines when:
+- Writing new React components or Next.js pages
+- Implementing data fetching (client or server-side)
+- Reviewing code for performance issues
+- Refactoring existing React/Next.js code
+- Optimizing bundle size or load times
+
+## Rule Categories by Priority
+
+| Priority | Category | Impact | Prefix |
+|----------|----------|--------|--------|
+| 1 | Eliminating Waterfalls | CRITICAL | `async-` |
+| 2 | Bundle Size Optimization | CRITICAL | `bundle-` |
+| 3 | Server-Side Performance | HIGH | `server-` |
+| 4 | Client-Side Data Fetching | MEDIUM-HIGH | `client-` |
+| 5 | Re-render Optimization | MEDIUM | `rerender-` |
+| 6 | Rendering Performance | MEDIUM | `rendering-` |
+| 7 | JavaScript Performance | LOW-MEDIUM | `js-` |
+| 8 | Advanced Patterns | LOW | `advanced-` |
+
+## Quick Reference
+
+### 1. Eliminating Waterfalls (CRITICAL)
+
+- `async-defer-await` - Move await into branches where actually used
+- `async-parallel` - Use Promise.all() for independent operations
+- `async-dependencies` - Use better-all for partial dependencies
+- `async-api-routes` - Start promises early, await late in API routes
+- `async-suspense-boundaries` - Use Suspense to stream content
+
+### 2. Bundle Size Optimization (CRITICAL)
+
+- `bundle-barrel-imports` - Import directly, avoid barrel files
+- `bundle-dynamic-imports` - Use next/dynamic for heavy components
+- `bundle-defer-third-party` - Load analytics/logging after hydration
+- `bundle-conditional` - Load modules only when feature is activated
+- `bundle-preload` - Preload on hover/focus for perceived speed
+
+### 3. Server-Side Performance (HIGH)
+
+- `server-cache-react` - Use React.cache() for per-request deduplication
+- `server-cache-lru` - Use LRU cache for cross-request caching
+- `server-serialization` - Minimize data passed to client components
+- `server-parallel-fetching` - Restructure components to parallelize fetches
+- `server-after-nonblocking` - Use after() for non-blocking operations
+
+### 4. Client-Side Data Fetching (MEDIUM-HIGH)
+
+- `client-swr-dedup` - Use SWR for automatic request deduplication
+- `client-event-listeners` - Deduplicate global event listeners
+
+### 5. Re-render Optimization (MEDIUM)
+
+- `rerender-defer-reads` - Don't subscribe to state only used in callbacks
+- `rerender-memo` - Extract expensive work into memoized components
+- `rerender-dependencies` - Use primitive dependencies in effects
+- `rerender-derived-state` - Subscribe to derived booleans, not raw values
+- `rerender-functional-setstate` - Use functional setState for stable callbacks
+- `rerender-lazy-state-init` - Pass function to useState for expensive values
+- `rerender-transitions` - Use startTransition for non-urgent updates
+
+### 6. Rendering Performance (MEDIUM)
+
+- `rendering-animate-svg-wrapper` - Animate div wrapper, not SVG element
+- `rendering-content-visibility` - Use content-visibility for long lists
+- `rendering-hoist-jsx` - Extract static JSX outside components
+- `rendering-svg-precision` - Reduce SVG coordinate precision
+- `rendering-hydration-no-flicker` - Use inline script for client-only data
+- `rendering-activity` - Use Activity component for show/hide
+- `rendering-conditional-render` - Use ternary, not && for conditionals
+
+### 7. JavaScript Performance (LOW-MEDIUM)
+
+- `js-batch-dom-css` - Group CSS changes via classes or cssText
+- `js-index-maps` - Build Map for repeated lookups
+- `js-cache-property-access` - Cache object properties in loops
+- `js-cache-function-results` - Cache function results in module-level Map
+- `js-cache-storage` - Cache localStorage/sessionStorage reads
+- `js-combine-iterations` - Combine multiple filter/map into one loop
+- `js-length-check-first` - Check array length before expensive comparison
+- `js-early-exit` - Return early from functions
+- `js-hoist-regexp` - Hoist RegExp creation outside loops
+- `js-min-max-loop` - Use loop for min/max instead of sort
+- `js-set-map-lookups` - Use Set/Map for O(1) lookups
+- `js-tosorted-immutable` - Use toSorted() for immutability
+
+### 8. Advanced Patterns (LOW)
+
+- `advanced-event-handler-refs` - Store event handlers in refs
+- `advanced-use-latest` - useLatest for stable callback refs
+
+## How to Use
+
+Read individual rule files for detailed explanations and code examples:
+
+```
+rules/async-parallel.md
+rules/bundle-barrel-imports.md
+rules/_sections.md
+```
+
+Each rule file contains:
+- Brief explanation of why it matters
+- Incorrect code example with explanation
+- Correct code example with explanation
+- Additional context and references
+
+## Full Compiled Document
+
+For the complete guide with all rules expanded: `AGENTS.md`