opencode-skills-collection 4.0.53 → 4.0.54

package/bundled-skills/llm-app-patterns/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: llm-app-patterns
-description: "Production-ready patterns for building LLM applications, inspired by [Dify](https://github.com/langgenius/dify) and industry best practices."
+description: "Architecture and integration sketches for LLM applications, with explicit retrieval, tool, privacy and verification boundaries."
 risk: critical
 source: community
 date_added: "2026-02-27"
@@ -8,7 +8,11 @@ date_added: "2026-02-27"
 
 # 🤖 LLM Application Patterns
 
-> Production-ready patterns for building LLM applications, inspired by [Dify](https://github.com/langgenius/dify) and industry best practices.
+> Architecture and integration sketches for LLM applications, with explicit retrieval, tool, privacy and verification boundaries.
+
+## Detailed Guide
+
+Read [the detailed guide](references/detailed-guide.md) before executing this skill. It retains the complete procedure and reference material. Treat its safety, prerequisites, and validation requirements as mandatory. For focused work, load the relevant sections; for end-to-end work, read the guide completely.
 
 ## When to Use This Skill
 
@@ -22,747 +26,18 @@ Use this skill when:
 
 ---
 
-## 1. RAG Pipeline Architecture
-
-### Overview
-
-RAG (Retrieval-Augmented Generation) grounds LLM responses in your data.
-
-```
-┌─────────────┐     ┌─────────────┐     ┌─────────────┐
-│   Ingest    │────▶│   Retrieve  │────▶│   Generate  │
-│  Documents  │     │   Context   │     │   Response  │
-└─────────────┘     └─────────────┘     └─────────────┘
-      │                   │                   │
-      ▼                   ▼                   ▼
- ┌─────────┐       ┌───────────┐       ┌───────────┐
- │ Chunking│       │  Vector   │       │    LLM    │
- │Embedding│       │  Search   │       │  + Context│
- └─────────┘       └───────────┘       └───────────┘
-```
-
-### 1.1 Document Ingestion
-
-```python
-# Chunking strategies
-class ChunkingStrategy:
-    # Fixed-size chunks (simple but may break context)
-    FIXED_SIZE = "fixed_size"  # e.g., 512 tokens
-
-    # Semantic chunking (preserves meaning)
-    SEMANTIC = "semantic"      # Split on paragraphs/sections
-
-    # Recursive splitting (tries multiple separators)
-    RECURSIVE = "recursive"    # ["\n\n", "\n", " ", ""]
-
-    # Document-aware (respects structure)
-    DOCUMENT_AWARE = "document_aware"  # Headers, lists, etc.
-
-# Recommended settings
-CHUNK_CONFIG = {
-    "chunk_size": 512,       # tokens
-    "chunk_overlap": 50,     # token overlap between chunks
-    "separators": ["\n\n", "\n", ". ", " "],
-}
-```
-
-### 1.2 Embedding & Storage
-
-```python
-# Vector database selection
-VECTOR_DB_OPTIONS = {
-    "pinecone": {
-        "use_case": "Production, managed service",
-        "scale": "Billions of vectors",
-        "features": ["Hybrid search", "Metadata filtering"]
-    },
-    "weaviate": {
-        "use_case": "Self-hosted, multi-modal",
-        "scale": "Millions of vectors",
-        "features": ["GraphQL API", "Modules"]
-    },
-    "chromadb": {
-        "use_case": "Development, prototyping",
-        "scale": "Thousands of vectors",
-        "features": ["Simple API", "In-memory option"]
-    },
-    "pgvector": {
-        "use_case": "Existing Postgres infrastructure",
-        "scale": "Millions of vectors",
-        "features": ["SQL integration", "ACID compliance"]
-    }
-}
-
-# Embedding model selection
-EMBEDDING_MODELS = {
-    "openai/text-embedding-3-small": {
-        "dimensions": 1536,
-        "cost": "$0.02/1M tokens",
-        "quality": "Good for most use cases"
-    },
-    "openai/text-embedding-3-large": {
-        "dimensions": 3072,
-        "cost": "$0.13/1M tokens",
-        "quality": "Best for complex queries"
-    },
-    "local/bge-large": {
-        "dimensions": 1024,
-        "cost": "Free (compute only)",
-        "quality": "Comparable to OpenAI small"
-    }
-}
-```
-
-### 1.3 Retrieval Strategies
-
-```python
-# Basic semantic search
-def semantic_search(query: str, top_k: int = 5):
-    query_embedding = embed(query)
-    results = vector_db.similarity_search(
-        query_embedding,
-        top_k=top_k
-    )
-    return results
-
-# Hybrid search (semantic + keyword)
-def hybrid_search(query: str, top_k: int = 5, alpha: float = 0.5):
-    """
-    alpha=1.0: Pure semantic
-    alpha=0.0: Pure keyword (BM25)
-    alpha=0.5: Balanced
-    """
-    semantic_results = vector_db.similarity_search(query)
-    keyword_results = bm25_search(query)
-
-    # Reciprocal Rank Fusion
-    return rrf_merge(semantic_results, keyword_results, alpha)
-
-# Multi-query retrieval
-def multi_query_retrieval(query: str):
-    """Generate multiple query variations for better recall"""
-    queries = llm.generate_query_variations(query, n=3)
-    all_results = []
-    for q in queries:
-        all_results.extend(semantic_search(q))
-    return deduplicate(all_results)
-
-# Contextual compression
-def compressed_retrieval(query: str):
-    """Retrieve then compress to relevant parts only"""
-    docs = semantic_search(query, top_k=10)
-    compressed = llm.extract_relevant_parts(docs, query)
-    return compressed
-```
-
-### 1.4 Generation with Context
-
-```python
-RAG_PROMPT_TEMPLATE = """
-Answer the user's question based ONLY on the following context.
-If the context doesn't contain enough information, say "I don't have enough information to answer that."
-
-Context:
-{context}
-
-Question: {question}
-
-Answer:"""
-
-def generate_with_rag(question: str):
-    # Retrieve
-    context_docs = hybrid_search(question, top_k=5)
-    context = "\n\n".join([doc.content for doc in context_docs])
-
-    # Generate
-    prompt = RAG_PROMPT_TEMPLATE.format(
-        context=context,
-        question=question
-    )
-
-    response = llm.generate(prompt)
-
-    # Return with citations
-    return {
-        "answer": response,
-        "sources": [doc.metadata for doc in context_docs]
-    }
-```
-
----
-
-## 2. Agent Architectures
-
-### 2.1 ReAct Pattern (Reasoning + Acting)
-
-```
-Thought: I need to search for information about X
-Action: search("X")
-Observation: [search results]
-Thought: Based on the results, I should...
-Action: calculate(...)
-Observation: [calculation result]
-Thought: I now have enough information
-Action: final_answer("The answer is...")
-```
-
-```python
-REACT_PROMPT = """
-You are an AI assistant that can use tools to answer questions.
-
-Available tools:
-{tools_description}
-
-Use this format:
-Thought: [your reasoning about what to do next]
-Action: [tool_name(arguments)]
-Observation: [tool result - this will be filled in]
-... (repeat Thought/Action/Observation as needed)
-Thought: I have enough information to answer
-Final Answer: [your final response]
-
-Question: {question}
-"""
-
-class ReActAgent:
-    def __init__(self, tools: list, llm):
-        self.tools = {t.name: t for t in tools}
-        self.llm = llm
-        self.max_iterations = 10
-
-    def run(self, question: str) -> str:
-        prompt = REACT_PROMPT.format(
-            tools_description=self._format_tools(),
-            question=question
-        )
-
-        for _ in range(self.max_iterations):
-            response = self.llm.generate(prompt)
-
-            if "Final Answer:" in response:
-                return self._extract_final_answer(response)
-
-            action = self._parse_action(response)
-            observation = self._execute_tool(action)
-            prompt += f"\nObservation: {observation}\n"
-
-        return "Max iterations reached"
-```
-
-### 2.2 Function Calling Pattern
-
-```python
-# Define tools as functions with schemas
-TOOLS = [
-    {
-        "name": "search_web",
-        "description": "Search the web for current information",
-        "parameters": {
-            "type": "object",
-            "properties": {
-                "query": {
-                    "type": "string",
-                    "description": "Search query"
-                }
-            },
-            "required": ["query"]
-        }
-    },
-    {
-        "name": "calculate",
-        "description": "Perform mathematical calculations",
-        "parameters": {
-            "type": "object",
-            "properties": {
-                "expression": {
-                    "type": "string",
-                    "description": "Math expression to evaluate"
-                }
-            },
-            "required": ["expression"]
-        }
-    }
-]
-
-class FunctionCallingAgent:
-    def run(self, question: str) -> str:
-        messages = [{"role": "user", "content": question}]
-
-        while True:
-            response = self.llm.chat(
-                messages=messages,
-                tools=TOOLS,
-                tool_choice="auto"
-            )
-
-            if response.tool_calls:
-                for tool_call in response.tool_calls:
-                    result = self._execute_tool(
-                        tool_call.name,
-                        tool_call.arguments
-                    )
-                    messages.append({
-                        "role": "tool",
-                        "tool_call_id": tool_call.id,
-                        "content": str(result)
-                    })
-            else:
-                return response.content
-```
-
-### 2.3 Plan-and-Execute Pattern
-
-```python
-class PlanAndExecuteAgent:
-    """
-    1. Create a plan (list of steps)
-    2. Execute each step
-    3. Replan if needed
-    """
-
-    def run(self, task: str) -> str:
-        # Planning phase
-        plan = self.planner.create_plan(task)
-        # Returns: ["Step 1: ...", "Step 2: ...", ...]
-
-        results = []
-        for step in plan:
-            # Execute each step
-            result = self.executor.execute(step, context=results)
-            results.append(result)
-
-            # Check if replan needed
-            if self._needs_replan(task, results):
-                new_plan = self.planner.replan(
-                    task,
-                    completed=results,
-                    remaining=plan[len(results):]
-                )
-                plan = new_plan
-
-        # Synthesize final answer
-        return self.synthesizer.summarize(task, results)
-```
-
-### 2.4 Multi-Agent Collaboration
-
-```python
-class AgentTeam:
-    """
-    Specialized agents collaborating on complex tasks
-    """
-
-    def __init__(self):
-        self.agents = {
-            "researcher": ResearchAgent(),
-            "analyst": AnalystAgent(),
-            "writer": WriterAgent(),
-            "critic": CriticAgent()
-        }
-        self.coordinator = CoordinatorAgent()
-
-    def solve(self, task: str) -> str:
-        # Coordinator assigns subtasks
-        assignments = self.coordinator.decompose(task)
-
-        results = {}
-        for assignment in assignments:
-            agent = self.agents[assignment.agent]
-            result = agent.execute(
-                assignment.subtask,
-                context=results
-            )
-            results[assignment.id] = result
-
-        # Critic reviews
-        critique = self.agents["critic"].review(results)
-
-        if critique.needs_revision:
-            # Iterate with feedback
-            return self.solve_with_feedback(task, results, critique)
-
-        return self.coordinator.synthesize(results)
-```
-
----
-
-## 3. Prompt IDE Patterns
-
-### 3.1 Prompt Templates with Variables
-
-```python
-class PromptTemplate:
-    def __init__(self, template: str, variables: list[str]):
-        self.template = template
-        self.variables = variables
-
-    def format(self, **kwargs) -> str:
-        # Validate all variables provided
-        missing = set(self.variables) - set(kwargs.keys())
-        if missing:
-            raise ValueError(f"Missing variables: {missing}")
-
-        return self.template.format(**kwargs)
-
-    def with_examples(self, examples: list[dict]) -> str:
-        """Add few-shot examples"""
-        example_text = "\n\n".join([
-            f"Input: {ex['input']}\nOutput: {ex['output']}"
-            for ex in examples
-        ])
-        return f"{example_text}\n\n{self.template}"
-
-# Usage
-summarizer = PromptTemplate(
-    template="Summarize the following text in {style} style:\n\n{text}",
-    variables=["style", "text"]
-)
-
-prompt = summarizer.format(
-    style="professional",
-    text="Long article content..."
-)
-```
-
-### 3.2 Prompt Versioning & A/B Testing
-
-```python
-class PromptRegistry:
-    def __init__(self, db):
-        self.db = db
-
-    def register(self, name: str, template: str, version: str):
-        """Store prompt with version"""
-        self.db.save({
-            "name": name,
-            "template": template,
-            "version": version,
-            "created_at": datetime.now(),
-            "metrics": {}
-        })
-
-    def get(self, name: str, version: str = "latest") -> str:
-        """Retrieve specific version"""
-        return self.db.get(name, version)
-
-    def ab_test(self, name: str, user_id: str) -> str:
-        """Return variant based on user bucket"""
-        variants = self.db.get_all_versions(name)
-        bucket = hash(user_id) % len(variants)
-        return variants[bucket]
-
-    def record_outcome(self, prompt_id: str, outcome: dict):
-        """Track prompt performance"""
-        self.db.update_metrics(prompt_id, outcome)
-```
-
-### 3.3 Prompt Chaining
-
-```python
-class PromptChain:
-    """
-    Chain prompts together, passing output as input to next
-    """
-
-    def __init__(self, steps: list[dict]):
-        self.steps = steps
-
-    def run(self, initial_input: str) -> dict:
-        context = {"input": initial_input}
-        results = []
-
-        for step in self.steps:
-            prompt = step["prompt"].format(**context)
-            output = llm.generate(prompt)
-
-            # Parse output if needed
-            if step.get("parser"):
-                output = step"parser"
-
-            context[step["output_key"]] = output
-            results.append({
-                "step": step["name"],
-                "output": output
-            })
-
-        return {
-            "final_output": context[self.steps[-1]["output_key"]],
-            "intermediate_results": results
-        }
-
-# Example: Research → Analyze → Summarize
-chain = PromptChain([
-    {
-        "name": "research",
-        "prompt": "Research the topic: {input}",
-        "output_key": "research"
-    },
-    {
-        "name": "analyze",
-        "prompt": "Analyze these findings:\n{research}",
-        "output_key": "analysis"
-    },
-    {
-        "name": "summarize",
-        "prompt": "Summarize this analysis in 3 bullet points:\n{analysis}",
-        "output_key": "summary"
-    }
-])
-```
-
----
-
-## 4. LLMOps & Observability
-
-### 4.1 Metrics to Track
-
-```python
-LLM_METRICS = {
-    # Performance
-    "latency_p50": "50th percentile response time",
-    "latency_p99": "99th percentile response time",
-    "tokens_per_second": "Generation speed",
-
-    # Quality
-    "user_satisfaction": "Thumbs up/down ratio",
-    "task_completion": "% tasks completed successfully",
-    "hallucination_rate": "% responses with factual errors",
-
-    # Cost
-    "cost_per_request": "Average $ per API call",
-    "tokens_per_request": "Average tokens used",
-    "cache_hit_rate": "% requests served from cache",
-
-    # Reliability
-    "error_rate": "% failed requests",
-    "timeout_rate": "% requests that timed out",
-    "retry_rate": "% requests needing retry"
-}
-```
-
-### 4.2 Logging & Tracing
-
-```python
-import logging
-from opentelemetry import trace
-
-tracer = trace.get_tracer(__name__)
-
-class LLMLogger:
-    def log_request(self, request_id: str, data: dict):
-        """Log LLM request for debugging and analysis"""
-        log_entry = {
-            "request_id": request_id,
-            "timestamp": datetime.now().isoformat(),
-            "model": data["model"],
-            "prompt": data["prompt"][:500],  # Truncate for storage
-            "prompt_tokens": data["prompt_tokens"],
-            "temperature": data.get("temperature", 1.0),
-            "user_id": data.get("user_id"),
-        }
-        logging.info(f"LLM_REQUEST: {json.dumps(log_entry)}")
-
-    def log_response(self, request_id: str, data: dict):
-        """Log LLM response"""
-        log_entry = {
-            "request_id": request_id,
-            "completion_tokens": data["completion_tokens"],
-            "total_tokens": data["total_tokens"],
-            "latency_ms": data["latency_ms"],
-            "finish_reason": data["finish_reason"],
-            "cost_usd": self._calculate_cost(data),
-        }
-        logging.info(f"LLM_RESPONSE: {json.dumps(log_entry)}")
-
-# Distributed tracing
-@tracer.start_as_current_span("llm_call")
-def call_llm(prompt: str) -> str:
-    span = trace.get_current_span()
-    span.set_attribute("prompt.length", len(prompt))
-
-    response = llm.generate(prompt)
-
-    span.set_attribute("response.length", len(response))
-    span.set_attribute("tokens.total", response.usage.total_tokens)
-
-    return response.content
-```
-
-### 4.3 Evaluation Framework
-
-```python
-class LLMEvaluator:
-    """
-    Evaluate LLM outputs for quality
-    """
-
-    def evaluate_response(self,
-                          question: str,
-                          response: str,
-                          ground_truth: str = None) -> dict:
-        scores = {}
-
-        # Relevance: Does it answer the question?
-        scores["relevance"] = self._score_relevance(question, response)
-
-        # Coherence: Is it well-structured?
-        scores["coherence"] = self._score_coherence(response)
-
-        # Groundedness: Is it based on provided context?
-        scores["groundedness"] = self._score_groundedness(response)
-
-        # Accuracy: Does it match ground truth?
-        if ground_truth:
-            scores["accuracy"] = self._score_accuracy(response, ground_truth)
-
-        # Harmfulness: Is it safe?
-        scores["safety"] = self._score_safety(response)
-
-        return scores
-
-    def run_benchmark(self, test_cases: list[dict]) -> dict:
-        """Run evaluation on test set"""
-        results = []
-        for case in test_cases:
-            response = llm.generate(case["prompt"])
-            scores = self.evaluate_response(
-                question=case["prompt"],
-                response=response,
-                ground_truth=case.get("expected")
-            )
-            results.append(scores)
-
-        return self._aggregate_scores(results)
-```
-
----
-
-## 5. Production Patterns
-
-### 5.1 Caching Strategy
-
-```python
-import hashlib
-from functools import lru_cache
-
-class LLMCache:
-    def __init__(self, redis_client, ttl_seconds=3600):
-        self.redis = redis_client
-        self.ttl = ttl_seconds
-
-    def _cache_key(self, prompt: str, model: str, **kwargs) -> str:
-        """Generate deterministic cache key"""
-        content = f"{model}:{prompt}:{json.dumps(kwargs, sort_keys=True)}"
-        return hashlib.sha256(content.encode()).hexdigest()
-
-    def get_or_generate(self, prompt: str, model: str, **kwargs) -> str:
-        key = self._cache_key(prompt, model, **kwargs)
-
-        # Check cache
-        cached = self.redis.get(key)
-        if cached:
-            return cached.decode()
-
-        # Generate
-        response = llm.generate(prompt, model=model, **kwargs)
-
-        # Cache (only cache deterministic outputs)
-        if kwargs.get("temperature", 1.0) == 0:
-            self.redis.setex(key, self.ttl, response)
-
-        return response
-```
-
-### 5.2 Rate Limiting & Retry
-
-```python
-import time
-from tenacity import retry, wait_exponential, stop_after_attempt
+## Inputs, worked example and verification
 
-class RateLimiter:
-    def __init__(self, requests_per_minute: int):
-        self.rpm = requests_per_minute
-        self.timestamps = []
+Record provider/SDK versions, authorized data and tools, request/response schemas, latency/cost budgets and the expected task outcome. All code above is an integration sketch: `llm`, database, parser and provider-response adapters are project-owned and must be implemented explicitly. Never execute model-provided Python, expressions or fuzzy tool names; dispatch only exact registered tools after schema and authorization checks. Enforce per-call deadlines as well as loop limits.
 
-    def acquire(self):
-        """Wait if rate limit would be exceeded"""
-        now = time.time()
+Example: a planner first queues steps A and B, then after A replaces the remaining work with C. The executor must run A then C, never stale B. A parser step must apply the parser to the actual prior output. Two tenants with the same prompt must produce different cache keys, and cache reads/writes require an explicit reuse policy. These are concrete regression assertions; successful mocks still do not prove live provider behavior.
 
-        # Remove old timestamps
-        self.timestamps = [t for t in self.timestamps if now - t < 60]
-
-        if len(self.timestamps) >= self.rpm:
-            sleep_time = 60 - (now - self.timestamps[0])
-            time.sleep(sleep_time)
-
-        self.timestamps.append(time.time())
-
-# Retry with exponential backoff
-@retry(
-    wait=wait_exponential(multiplier=1, min=4, max=60),
-    stop=stop_after_attempt(5)
-)
-def call_llm_with_retry(prompt: str) -> str:
-    try:
-        return llm.generate(prompt)
-    except RateLimitError:
-        raise  # Will trigger retry
-    except APIError as e:
-        if e.status_code >= 500:
-            raise  # Retry server errors
-        raise  # Don't retry client errors
-```
-
-### 5.3 Fallback Strategy
-
-```python
-class LLMWithFallback:
-    def __init__(self, primary: str, fallbacks: list[str]):
-        self.primary = primary
-        self.fallbacks = fallbacks
-
-    def generate(self, prompt: str, **kwargs) -> str:
-        models = [self.primary] + self.fallbacks
-
-        for model in models:
-            try:
-                return llm.generate(prompt, model=model, **kwargs)
-            except (RateLimitError, APIError) as e:
-                logging.warning(f"Model {model} failed: {e}")
-                continue
-
-        raise AllModelsFailedError("All models exhausted")
-
-# Usage
-llm_client = LLMWithFallback(
-    primary="gpt-4-turbo",
-    fallbacks=["gpt-3.5-turbo", "claude-3-sonnet"]
-)
-```
-
----
-
-## Architecture Decision Matrix
-
-| Pattern              | Use When         | Complexity | Cost      |
-| :------------------- | :--------------- | :--------- | :-------- |
-| **Simple RAG**       | FAQ, docs search | Low        | Low       |
-| **Hybrid RAG**       | Mixed queries    | Medium     | Medium    |
-| **ReAct Agent**      | Multi-step tasks | Medium     | Medium    |
-| **Function Calling** | Structured tools | Low        | Low       |
-| **Plan-Execute**     | Complex tasks    | High       | High      |
-| **Multi-Agent**      | Research tasks   | Very High  | Very High |
-
----
-
-## Resources
-
-- [Dify Platform](https://github.com/langgenius/dify)
-- [LangChain Docs](https://python.langchain.com/)
-- [LlamaIndex](https://www.llamaindex.ai/)
-- [Anthropic Cookbook](https://github.com/anthropics/anthropic-cookbook)
+The limiter sketch is single-threaded and process-local. A distributed deployment needs a shared atomic limit and deadlines. Retry only retry-safe operations, honoring provider retry guidance; a timeout after a side effect is not evidence that nothing happened. See [Tenacity retry predicates](https://tenacity.readthedocs.io/en/latest/).
 
 ## Limitations
-- Use this skill only when the task clearly matches the scope described above.
-- Do not treat the output as a substitute for environment-specific validation, testing, or expert review.
-- Stop and ask for clarification if required inputs, permissions, safety boundaries, or success criteria are missing.
+
+- A retrieved source list does not prove each answer claim is supported; verify claim-to-source evidence and abstention behavior.
+- Prompt text and JSON-shaped output are not authorization boundaries. Enforce permissions in the application.
+- Provider failover can change output quality, tool schemas, cost and data residency; only use pre-approved compatible fallbacks.
+- Caching must respect tenant access, data/prompt revisions and deletion policy; temperature zero does not make output deterministic.
+- Model/SDK examples are not a complete service, benchmark or production-readiness certificate.