@baimingtao/lbs-agent 1.0.0

package/extensions/skills/optional/mlops/chroma/SKILL.md

@@ -0,0 +1,408 @@
+---
+name: chroma
+description: 用于 AI 应用的开源嵌入数据库。存储 embeddings 和 metadata，执行向量和全文搜索，并按 metadata 进行过滤。提供简单的 4 函数 API。可从 notebooks 扩展到生产集群。适用于语义搜索、RAG 应用或文档检索。最适合本地开发和开源项目。
+version: 1.0.0
+author: Orchestra Research
+license: MIT
+dependencies: [chromadb, sentence-transformers]
+platforms: [linux, macos, windows]
+metadata:
+  hermes:
+    tags: [RAG, Chroma, Vector Database, Embeddings, Semantic Search, Open Source, Self-Hosted, Document Retrieval, Metadata Filtering]
+
+---
+
+# Chroma - 开源 Embedding 数据库
+
+为构建具备记忆能力的 LLM 应用而设计的 AI 原生数据库。
+
+## 何时使用 Chroma
+
+**在以下情况使用 Chroma：**
+- 构建 RAG（检索增强生成）应用
+- 需要本地/自托管的 vector database
+- 希望使用开源解决方案 (Apache 2.0)
+- 在 notebooks 中进行原型设计
+- 对文档进行语义搜索
+- 存储带有 metadata 的 embeddings
+
+**相关指标**：
+- **24,300+ GitHub stars**
+- **1,900+ forks**
+- **v1.3.3**（稳定版，每周发布）
+- **Apache 2.0 license**
+
+**可使用的替代方案**：
+- **Pinecone**：托管云服务，自动扩缩容
+- **FAISS**：纯粹的相似度搜索，不支持 metadata
+- **Weaviate**：生产级 ML 原生数据库
+- **Qdrant**：高性能，基于 Rust 开发
+
+## 快速 start
+
+### 安装
+
+```bash
+# Python
+pip install chromadb
+
+# JavaScript/TypeScript
+npm install chromadb @chroma-core/default-embed
+```
+
+### 基础 usage (Python)
+
+```python
+import chromadb
+
+# Create client
+client = chromadb.Client()
+
+# Create collection
+collection = client.create_collection(name="my_collection")
+
+# Add documents
+collection.add(
+    documents=["This is document 1", "This is document 2"],
+    metadatas=[{"source": "doc1"}, {"source": "doc2"}],
+    ids=["id1", "id2"]
+)
+
+# Query
+results = collection.query(
+    query_texts=["document about topic"],
+    n_results=2
+)
+
+print(results)
+```
+
+## Core operations
+
+### 1. Create collection
+
+```python
+# Simple collection
+collection = client.create_collection("my_docs")
+
+# With custom embedding function
+from chromadb.utils import embedding_functions
+
+openai_ef = embedding_functions.OpenAIEmbeddingFunction(
+    api_key="your-key",
+    model_name="text-embedding-3-small"
+)
+
+collection = client.create_collection(
+    name="my_docs",
+    embedding_function=openai_ef
+)
+
+# Get existing collection
+collection = client.get_collection("my_docs")
+
+# Delete collection
+client.delete_collection("my_docs")
+```
+
+### 2. Add documents
+
+```python
+# Add with auto-generated IDs
+collection.add(
+    documents=["Doc 1", "Doc 2", "Doc 3"],
+    metadatas=[
+        {"source": "web", "category": "tutorial"},
+        {"source": "pdf", "page": 5},
+        {"source": "api", "timestamp": "2025-01-01"}
+    ],
+    ids=["id1", "id2", "id3"]
+)
+
+# Add with custom embeddings
+collection.add(
+    embeddings=[[0.1, 0.2, ...], [0.3, 0.4, ...]],
+    documents=["Doc 1", "Doc 2"],
+    ids=["id1", "id2"]
+)
+```
+
+### 3. Query (similarity search)
+
+```python
+# Basic query
+results = collection.query(
+    query_texts=["machine learning tutorial"],
+    n_results=5
+)
+
+# Query with filters
+results = collection.query(
+    query_texts=["Python programming"],
+    n_results=3,
+    where={"source": "web"}
+)
+
+# Query with metadata filters
+results = collection.query(
+    query_texts=["advanced topics"],
+    where={
+        "$and": [
+            {"category": "tutorial"},
+            {"difficulty": {"$gte": 3}}
+        ]
+    }
+)
+
+# Access results
+print(results["documents"])      # List of matching documents
+print(results["metadatas"])      # Metadata for each doc
+print(results["distances"])      # Similarity scores
+print(results["ids"])            # Document IDs
+```
+
+### 4. Get documents
+
+```python
+# Get by IDs
+docs = collection.get(
+    ids=["id1", "id2"]
+)
+
+# Get with filters
+docs = collection.get(
+    where={"category": "tutorial"},
+    limit=10
+)
+
+# Get all documents
+docs = collection.get()
+```
+
+### 5. 更新 documents
+
+```python
+# Update document content
+collection.update(
+    ids=["id1"],
+    documents=["Updated content"],
+    metadatas=[{"source": "updated"}]
+)
+```
+
+### 6. Delete documents
+
+```python
+# Delete by IDs
+collection.delete(ids=["id1", "id2"])
+
+# Delete with filter
+collection.delete(
+    where={"source": "outdated"}
+)
+```
+
+## Persistent storage
+
+```python
+# Persist to disk
+client = chromadb.PersistentClient(path="./chroma_db")
+
+collection = client.create_collection("my_docs")
+collection.add(documents=["Doc 1"], ids=["id1"])
+
+# Data persisted automatically
+# Reload later with same path
+client = chromadb.PersistentClient(path="./chroma_db")
+collection = client.get_collection("my_docs")
+```
+
+## Embedding functions
+
+### 默认 (Sentence Transformers)
+
+```python
+# Uses sentence-transformers by default
+collection = client.create_collection("my_docs")
+# Default model: all-MiniLM-L6-v2
+```
+
+### OpenAI
+
+```python
+from chromadb.utils import embedding_functions
+
+openai_ef = embedding_functions.OpenAIEmbeddingFunction(
+    api_key="your-key",
+    model_name="text-embedding-3-small"
+)
+
+collection = client.create_collection(
+    name="openai_docs",
+    embedding_function=openai_ef
+)
+```
+
+### HuggingFace
+
+```python
+huggingface_ef = embedding_functions.HuggingFaceEmbeddingFunction(
+    api_key="your-key",
+    model_name="sentence-transformers/all-mpnet-base-v2"
+)
+
+collection = client.create_collection(
+    name="hf_docs",
+    embedding_function=huggingface_ef
+)
+```
+
+### Custom embedding function
+
+```python
+from chromadb import Documents, EmbeddingFunction, Embeddings
+
+class MyEmbeddingFunction(EmbeddingFunction):
+    def __call__(self, input: Documents) -> Embeddings:
+        # Your embedding logic
+        return embeddings
+
+my_ef = MyEmbeddingFunction()
+collection = client.create_collection(
+    name="custom_docs",
+    embedding_function=my_ef
+)
+```
+
+## Metadata filtering
+
+```python
+# Exact match
+results = collection.query(
+    query_texts=["query"],
+    where={"category": "tutorial"}
+)
+
+# Comparison operators
+results = collection.query(
+    query_texts=["query"],
+    where={"page": {"$gt": 10}}  # $gt, $gte, $lt, $lte, $ne
+)
+
+# Logical operators
+results = collection.query(
+    query_texts=["query"],
+    where={
+        "$and": [
+            {"category": "tutorial"},
+            {"difficulty": {"$lte": 3}}
+        ]
+    }  # Also: $or
+)
+
+# Contains
+results = collection.query(
+    query_texts=["query"],
+    where={"tags": {"$in": ["python", "ml"]}}
+)
+```
+
+## LangChain integration
+
+```python
+from langchain_chroma import Chroma
+from langchain_openai import OpenAIEmbeddings
+from langchain.text_splitter import RecursiveCharacterTextSplitter
+
+# Split documents
+text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000)
+docs = text_splitter.split_documents(documents)
+
+# Create Chroma vector store
+vectorstore = Chroma.from_documents(
+    documents=docs,
+    embedding=OpenAIEmbeddings(),
+    persist_directory="./chroma_db"
+)
+
+# Query
+results = vectorstore.similarity_search("machine learning", k=3)
+
+# As retriever
+retriever = vectorstore.as_retriever(search_kwargs={"k": 5})
+```
+
+## LlamaIndex integration
+
+```python
+from llama_index.vector_stores.chroma import ChromaVectorStore
+from llama_index.core import VectorStoreIndex, StorageContext
+import chromadb
+
+# Initialize Chroma
+db = chromadb.PersistentClient(path="./chroma_db")
+collection = db.get_or_create_collection("my_collection")
+
+# Create vector store
+vector_store = ChromaVectorStore(chroma_collection=collection)
+storage_context = StorageContext.from_defaults(vector_store=vector_store)
+
+# Create index
+index = VectorStoreIndex.from_documents(
+    documents,
+    storage_context=storage_context
+)
+
+# Query
+query_engine = index.as_query_engine()
+response = query_engine.query("What is machine learning?")
+```
+
+## Server mode
+
+```python
+# Run Chroma server
+# Terminal: chroma run --path ./chroma_db --port 8000
+
+# Connect to server
+import chromadb
+from chromadb.config import Settings
+
+client = chromadb.HttpClient(
+    host="localhost",
+    port=8000,
+    settings=Settings(anonymized_telemetry=False)
+)
+
+# Use as normal
+collection = client.get_or_create_collection("my_docs")
+```
+
+## Best practices
+
+1. **使用持久化客户端** - 避免重启时丢失数据
+2. **添加 metadata** - 启用过滤和追踪功能
+3. **批量操作** - 一次性添加多个文档
+4. **选择正确的 embedding 模型** - 在速度和质量之间取得平衡
+5. **使用过滤器** - 缩小搜索范围
+6. **确保 ID 唯一** - 避免冲突
+7. **定期备份** - 复制 chroma_db 目录
+8. **监控 collection 大小** - 根据需要进行扩容
+9. **测试 embedding 功能** - 确保质量
+10. **在生产环境中使用服务器模式** - 更适合多用户场景
+
+## Performance
+
+| Operation | Latency | 说明 |
+|-----------|---------|-------|
+| Add 100 docs | ~1-3s | 包含 embedding 计算 |
+| Query (top 10) | ~50-200ms | 取决于 collection 大小 |
+| Metadata filter | ~10-50ms | 建立适当索引后速度很快 |
+
+## Resources
+
+- **GitHub**: https://github.com/chroma-core/chroma ⭐ 24,300+
+- **Docs**: https://docs.trychroma.com
+- **Discord**: https://discord.gg/MMeYNTmh3x
+- **Version**: 1.3.3+
+- **许可证**: Apache 2.0

package/extensions/skills/optional/mlops/chroma/references/integration.md

@@ -0,0 +1,38 @@
+# Chroma 集成指南
+
+与 LangChain、LlamaIndex 和各种框架的集成。
+
+## LangChain
+
+```python
+from langchain_chroma import Chroma
+from langchain_openai import OpenAIEmbeddings
+
+vectorstore = Chroma.from_documents(
+    documents=docs,
+    embedding=OpenAIEmbeddings(),
+    persist_directory="./chroma_db"
+)
+
+# Query
+results = vectorstore.similarity_search("query", k=3)
+
+# As retriever
+retriever = vectorstore.as_retriever()
+```
+
+## LlamaIndex
+
+```python
+from llama_index.vector_stores.chroma import ChromaVectorStore
+import chromadb
+
+db = chromadb.PersistentClient(path="./chroma_db")
+collection = db.get_or_create_collection("docs")
+
+vector_store = ChromaVectorStore(chroma_collection=collection)
+```
+
+## 资源
+
+- **文档**：https://docs.trychroma.com

package/extensions/skills/optional/mlops/clip/SKILL.md

@@ -0,0 +1,255 @@
+---
+name: clip
+description: OpenAI 的连接视觉与语言的模型。支持 zero-shot 图像分类、图文匹配和跨模态检索。基于 4 亿图文对进行训练。可用于图像搜索、内容审核或无需 fine-tuning 的视觉语言任务。最适合通用图像理解。
+version: 1.0.0
+author: Orchestra Research
+license: MIT
+dependencies: [transformers, torch, pillow]
+platforms: [linux, macos, windows]
+metadata:
+  hermes:
+    tags: [Multimodal, CLIP, Vision-Language, Zero-Shot, Image Classification, OpenAI, Image Search, Cross-Modal Retrieval, Content Moderation]
+
+---
+
+# CLIP - Contrastive Language-Image Pre-Training
+
+OpenAI 的通过自然语言理解图像的模型。
+
+## 何时使用 CLIP
+
+**适用场景：**
+- Zero-shot 图像分类（无需训练数据）
+- 图文相似度/匹配
+- 语义图像搜索
+- 内容审核（检测 NSFW、暴力）
+- 视觉问答
+- 跨模态检索（image→text，text→image）
+
+**数据指标**：
+- **25,300+ GitHub stars**
+- 基于 4 亿图文对进行训练
+- 在 ImageNet 上达到 ResNet-50 的水平（zero-shot）
+- MIT License
+
+**建议使用替代方案的场景**：
+- **BLIP-2**：更好的图像描述
+- **LLaVA**：视觉语言对话
+- **Segment Anything**：图像分割
+
+## 快速 start
+
+### 安装
+
+```bash
+pip install git+https://github.com/openai/CLIP.git
+pip install torch torchvision ftfy regex tqdm
+```
+
+### Zero-shot classification
+
+```python
+import torch
+import clip
+from PIL import Image
+
+# Load model
+device = "cuda" if torch.cuda.is_available() else "cpu"
+model, preprocess = clip.load("ViT-B/32", device=device)
+
+# Load image
+image = preprocess(Image.open("photo.jpg")).unsqueeze(0).to(device)
+
+# Define possible labels
+text = clip.tokenize(["a dog", "a cat", "a bird", "a car"]).to(device)
+
+# Compute similarity
+with torch.no_grad():
+    image_features = model.encode_image(image)
+    text_features = model.encode_text(text)
+
+    # Cosine similarity
+    logits_per_image, logits_per_text = model(image, text)
+    probs = logits_per_image.softmax(dim=-1).cpu().numpy()
+
+# Print results
+labels = ["a dog", "a cat", "a bird", "a car"]
+for label, prob in zip(labels, probs[0]):
+    print(f"{label}: {prob:.2%}")
+```
+
+## Available models
+
+```python
+# Models (sorted by size)
+models = [
+    "RN50",           # ResNet-50
+    "RN101",          # ResNet-101
+    "ViT-B/32",       # Vision Transformer (recommended)
+    "ViT-B/16",       # Better quality, slower
+    "ViT-L/14",       # Best quality, slowest
+]
+
+model, preprocess = clip.load("ViT-B/32")
+```
+
+| Model | Parameters | Speed | Quality |
+| ------- | ------------ | ------- | --------- |
+| RN50 | 102M | Fast | Good |
+| ViT-B/32 | 151M | Medium | Better |
+| ViT-L/14 | 428M | Slow | Best |
+
+## Image-text similarity
+
+```python
+# Compute embeddings
+image_features = model.encode_image(image)
+text_features = model.encode_text(text)
+
+# Normalize
+image_features /= image_features.norm(dim=-1, keepdim=True)
+text_features /= text_features.norm(dim=-1, keepdim=True)
+
+# Cosine similarity
+similarity = (image_features @ text_features.T).item()
+print(f"Similarity: {similarity:.4f}")
+```
+
+## Semantic image search
+
+```python
+# Index images
+image_paths = ["img1.jpg", "img2.jpg", "img3.jpg"]
+image_embeddings = []
+
+for img_path in image_paths:
+    image = preprocess(Image.open(img_path)).unsqueeze(0).to(device)
+    with torch.no_grad():
+        embedding = model.encode_image(image)
+        embedding /= embedding.norm(dim=-1, keepdim=True)
+    image_embeddings.append(embedding)
+
+image_embeddings = torch.cat(image_embeddings)
+
+# Search with text query
+query = "a sunset over the ocean"
+text_input = clip.tokenize([query]).to(device)
+with torch.no_grad():
+    text_embedding = model.encode_text(text_input)
+    text_embedding /= text_embedding.norm(dim=-1, keepdim=True)
+
+# Find most similar images
+similarities = (text_embedding @ image_embeddings.T).squeeze(0)
+top_k = similarities.topk(3)
+
+for idx, score in zip(top_k.indices, top_k.values):
+    print(f"{image_paths[idx]}: {score:.3f}")
+```
+
+## Content moderation
+
+```python
+# Define categories
+categories = [
+    "safe for work",
+    "not safe for work",
+    "violent content",
+    "graphic content"
+]
+
+text = clip.tokenize(categories).to(device)
+
+# Check image
+with torch.no_grad():
+    logits_per_image, _ = model(image, text)
+    probs = logits_per_image.softmax(dim=-1)
+
+# Get classification
+max_idx = probs.argmax().item()
+max_prob = probs[0, max_idx].item()
+
+print(f"Category: {categories[max_idx]} ({max_prob:.2%})")
+```
+
+## Batch processing
+
+```python
+# Process multiple images
+images = [preprocess(Image.open(f"img{i}.jpg")) for i in range(10)]
+images = torch.stack(images).to(device)
+
+with torch.no_grad():
+    image_features = model.encode_image(images)
+    image_features /= image_features.norm(dim=-1, keepdim=True)
+
+# Batch text
+texts = ["a dog", "a cat", "a bird"]
+text_tokens = clip.tokenize(texts).to(device)
+
+with torch.no_grad():
+    text_features = model.encode_text(text_tokens)
+    text_features /= text_features.norm(dim=-1, keepdim=True)
+
+# Similarity matrix (10 images × 3 texts)
+similarities = image_features @ text_features.T
+print(similarities.shape)  # (10, 3)
+```
+
+## Integration with vector databases
+
+```python
+# Store CLIP embeddings in Chroma/FAISS
+import chromadb
+
+client = chromadb.Client()
+collection = client.create_collection("image_embeddings")
+
+# Add image embeddings
+for img_path, embedding in zip(image_paths, image_embeddings):
+    collection.add(
+        embeddings=[embedding.cpu().numpy().tolist()],
+        metadatas=[{"path": img_path}],
+        ids=[img_path]
+    )
+
+# Query with text
+query = "a sunset"
+text_embedding = model.encode_text(clip.tokenize([query]))
+results = collection.query(
+    query_embeddings=[text_embedding.cpu().numpy().tolist()],
+    n_results=5
+)
+```
+
+## Best practices
+
+1. **大多数情况下使用 ViT-B/32** - 平衡性好
+2. **归一化 embeddings** - 计算余弦相似度时必需
+3. **Batch processing** - 更高效
+4. **缓存 embeddings** - 重新计算的成本很高
+5. **使用描述性的标签** - 能获得更好的 zero-shot 性能
+6. **建议使用 GPU** - 速度快 10-50 倍
+7. **预处理图像** - 使用提供的 preprocess 函数
+
+## Performance
+
+| Operation | CPU | GPU (V100) |
+| ----------- | ----- | ------------ |
+| Image encoding | ~200ms | ~20ms |
+| Text encoding | ~50ms | ~5ms |
+| Similarity compute | <1ms | <1ms |
+
+## Limitations
+
+1. **不适合细粒度任务** - 更适合宽泛的分类
+2. **需要描述性的文本** - 模糊的标签表现较差
+3. **存在网络数据偏差** - 可能带有数据集偏差
+4. **无边界框** - 仅支持整图
+5. **有限的空间理解能力** - 位置和计数能力较弱
+
+## Resources
+
+- **GitHub**: https://github.com/openai/CLIP ⭐ 25,300+
+- **Paper**: https://arxiv.org/abs/2103.00020
+- **Colab**: https://colab.research.google.com/github/openai/clip/
+- **License**: MIT