@ia-ccun/code-agent-cli 0.0.52 → 0.0.54-beta.1

package/config/agent/skills_bak/milvus-skill/references/collection.md

@@ -0,0 +1,86 @@
+# Collection Management — Detailed Reference
+
+## Supported Data Types
+
+### Scalar Types
+
+| DataType | Notes |
+|----------|-------|
+| `DataType.BOOL` | Boolean |
+| `DataType.INT8` / `INT16` / `INT32` / `INT64` | Integers |
+| `DataType.FLOAT` / `DOUBLE` | Floating point |
+| `DataType.VARCHAR` | String (requires `max_length`) |
+| `DataType.JSON` | JSON object |
+| `DataType.ARRAY` | Array (requires `element_type`, `max_capacity`) |
+
+### Vector Types
+
+| DataType | Notes |
+|----------|-------|
+| `DataType.FLOAT_VECTOR` | Float32 vector (requires `dim`) |
+| `DataType.FLOAT16_VECTOR` | Float16 vector (requires `dim`) |
+| `DataType.BFLOAT16_VECTOR` | BFloat16 vector (requires `dim`) |
+| `DataType.BINARY_VECTOR` | Binary vector (requires `dim`) |
+| `DataType.SPARSE_FLOAT_VECTOR` | Sparse vector (no `dim` needed) |
+| `DataType.INT8_VECTOR` | Int8 vector (requires `dim`) |
+
+## add_field Parameters
+
+```python
+schema.add_field(
+    field_name="my_field",
+    datatype=DataType.VARCHAR,
+    is_primary=False,
+    auto_id=False,
+    max_length=256,          # Required for VARCHAR
+    dim=768,                 # Required for vector types (except sparse)
+    element_type=DataType.INT64,  # Required for ARRAY
+    max_capacity=100,        # Required for ARRAY
+    nullable=False,
+    default_value=None,
+    is_partition_key=False,
+    description=""
+)
+```
+
+## All Collection Operations
+
+```python
+# List all collections
+collections = client.list_collections()
+
+# Describe a collection
+info = client.describe_collection(collection_name="my_collection")
+
+# Check if collection exists
+exists = client.has_collection(collection_name="my_collection")
+
+# Rename a collection
+client.rename_collection(old_name="old_name", new_name="new_name")
+
+# Drop a collection
+client.drop_collection(collection_name="my_collection")
+
+# Truncate a collection (delete all data, keep schema and index)
+client.truncate_collection(collection_name="my_collection")
+
+# Load collection into memory (required before search/query)
+client.load_collection(collection_name="my_collection")
+
+# Release collection from memory
+client.release_collection(collection_name="my_collection")
+
+# Get load state
+state = client.get_load_state(collection_name="my_collection")
+
+# Get collection statistics
+stats = client.get_collection_stats(collection_name="my_collection")
+```
+
+## Guidance
+
+- Quick create is best for prototyping; use custom schema for production.
+- A collection must be **loaded** before search or query operations.
+- Before dropping a collection, confirm with the user — this deletes all data.
+- Use `enable_dynamic_field=True` to allow inserting fields not defined in the schema.
+- Use `truncate_collection` to clear all data while preserving the collection structure.

package/config/agent/skills_bak/milvus-skill/references/database.md

@@ -0,0 +1,24 @@
+# Database Management
+
+```python
+# Create a database
+client.create_database(db_name="my_database")
+
+# List all databases
+databases = client.list_databases()
+# Returns: ["default", "my_database"]
+
+# Switch to a database
+client.using_database(db_name="my_database")
+
+# Drop a database (must drop all collections first)
+client.drop_database(db_name="my_database")
+
+# Or connect to a specific database at init (use the user's actual URI and credentials)
+client = MilvusClient(uri="<USER_URI>", token="<USER_TOKEN>", db_name="my_database")
+```
+
+## Guidance
+
+- Every Milvus instance has a `"default"` database.
+- Before dropping a database, all collections in it must be dropped first.

package/config/agent/skills_bak/milvus-skill/references/index.md

@@ -0,0 +1,69 @@
+# Index Management — Detailed Reference
+
+## Create Index
+
+```python
+index_params = client.prepare_index_params()
+
+# Vector index
+index_params.add_index(
+    field_name="embedding",
+    index_type="HNSW",               # See index types table below
+    metric_type="COSINE",            # "COSINE", "L2", "IP"
+    params={"M": 16, "efConstruction": 256}
+)
+
+# Optional: scalar index
+index_params.add_index(
+    field_name="text",
+    index_type=""                    # Auto-select for scalars
+)
+
+client.create_index(
+    collection_name="my_collection",
+    index_params=index_params
+)
+```
+
+## Common Index Types
+
+| Index Type | For | Key Params | Notes |
+|------------|-----|------------|-------|
+| `AUTOINDEX` | Dense vectors | Auto-tuned | Recommended for most cases |
+| `FLAT` | Dense vectors | None | Brute force, 100% recall |
+| `IVF_FLAT` | Dense vectors | `nlist` | Good balance |
+| `IVF_SQ8` | Dense vectors | `nlist` | Compressed, less memory |
+| `HNSW` | Dense vectors | `M`, `efConstruction` | High recall, more memory |
+| `DISKANN` | Dense vectors | None | Disk-based, large datasets |
+| `SPARSE_INVERTED_INDEX` | Sparse vectors | `drop_ratio_build` | For sparse vectors |
+| `SPARSE_WAND` | Sparse vectors | `drop_ratio_build` | Faster sparse search |
+
+## Metric Types
+
+| Metric | Description | Use With |
+|--------|-------------|----------|
+| `"COSINE"` | Cosine similarity (larger = more similar) | Dense vectors |
+| `"L2"` | Euclidean distance (smaller = more similar) | Dense vectors |
+| `"IP"` | Inner product (larger = more similar) | Dense & Sparse vectors |
+| `"BM25"` | BM25 relevance scoring | Full-text search (sparse vectors from built-in tokenizer) |
+
+## Other Index Operations
+
+```python
+# List indexes
+indexes = client.list_indexes(collection_name="my_collection")
+
+# Describe an index
+info = client.describe_index(collection_name="my_collection", index_name="my_index")
+
+# Drop an index
+client.drop_index(collection_name="my_collection", index_name="my_index")
+```
+
+## Guidance
+
+- `AUTOINDEX` is recommended for most use cases.
+- An index is required before loading a collection.
+- After creating an index, load the collection before searching.
+- Sparse vectors only support `"IP"` metric type.
+- For full-text search, use `"BM25"` metric type with `SPARSE_INVERTED_INDEX` or `SPARSE_WAND`.

package/config/agent/skills_bak/milvus-skill/references/partition.md

@@ -0,0 +1,29 @@
+# Partition Management
+
+```python
+# Create a partition
+client.create_partition(collection_name="my_collection", partition_name="partition_A")
+
+# List partitions
+partitions = client.list_partitions(collection_name="my_collection")
+# Returns: ["_default", "partition_A"]
+
+# Check if partition exists
+exists = client.has_partition(collection_name="my_collection", partition_name="partition_A")
+
+# Load specific partitions
+client.load_partitions(collection_name="my_collection", partition_names=["partition_A"])
+
+# Release specific partitions
+client.release_partitions(collection_name="my_collection", partition_names=["partition_A"])
+
+# Drop a partition
+client.drop_partition(collection_name="my_collection", partition_name="partition_A")
+```
+
+## Guidance
+
+- Every collection has a `_default` partition.
+- Use `is_partition_key=True` on a field to enable automatic partitioning by field value.
+- A partition must be loaded before search.
+- Before dropping a partition, confirm with the user — all data in it will be deleted.

package/config/agent/skills_bak/milvus-skill/references/patterns.md

@@ -0,0 +1,150 @@
+# Common Patterns
+
+> **Note:** All patterns below use `<USER_URI>` and `<USER_TOKEN>` as connection placeholders. Always ask the user for their actual connection details before writing code. For local development, use Milvus Lite (`uri="./milvus.db"`) only if the user explicitly requests it.
+
+## RAG Pipeline Pattern
+
+```python
+from pymilvus import MilvusClient, DataType, model
+
+# 1. Connect (ask user for URI and credentials)
+client = MilvusClient(uri="<USER_URI>", token="<USER_TOKEN>")
+
+# 2. Set up embedding model
+embedding_fn = model.dense.SentenceTransformerEmbeddingFunction(model_name="all-MiniLM-L6-v2")
+
+# 3. Create collection (dim must match embedding model output)
+schema = client.create_schema(auto_id=True, enable_dynamic_field=True)
+schema.add_field("id", DataType.INT64, is_primary=True)
+schema.add_field("text", DataType.VARCHAR, max_length=2048)
+schema.add_field("embedding", DataType.FLOAT_VECTOR, dim=384)  # all-MiniLM-L6-v2 outputs 384-dim
+schema.add_field("source", DataType.VARCHAR, max_length=256)
+
+index_params = client.prepare_index_params()
+index_params.add_index(field_name="embedding", index_type="AUTOINDEX", metric_type="COSINE")
+
+client.create_collection(collection_name="knowledge_base", schema=schema, index_params=index_params)
+
+# 4. Insert documents — generate real vectors from text chunks
+chunks = ["Milvus is a vector database...", "RAG combines retrieval and generation..."]
+vectors = embedding_fn.encode_documents(chunks)
+
+client.insert("knowledge_base", data=[
+    {"text": chunk, "embedding": vec, "source": "doc1.pdf"}
+    for chunk, vec in zip(chunks, vectors)
+])
+
+# 5. Retrieve relevant context — use the SAME embedding model
+query = "What is a vector database?"
+query_vectors = embedding_fn.encode_queries([query])
+
+results = client.search(
+    collection_name="knowledge_base",
+    data=query_vectors,
+    limit=5,
+    output_fields=["text", "source"],
+    search_params={"metric_type": "COSINE"}
+)
+```
+
+## Quick Semantic Search Pattern
+
+```python
+from pymilvus import MilvusClient, model
+
+# Simplest possible setup (Milvus Lite for local dev)
+client = MilvusClient(uri="./search.db")
+embedding_fn = model.dense.SentenceTransformerEmbeddingFunction(model_name="all-MiniLM-L6-v2")
+
+# Prepare data — vectors come from embedding model
+texts = ["first document", "second document", "third document"]
+vectors = embedding_fn.encode_documents(texts)
+
+client.create_collection(collection_name="docs", dimension=384)
+client.insert("docs", data=[
+    {"id": i, "vector": vec, "text": txt}
+    for i, (vec, txt) in enumerate(zip(vectors, texts))
+])
+
+# Search — encode query with the same model
+query_vectors = embedding_fn.encode_queries(["search query"])
+results = client.search("docs", data=query_vectors, limit=10, output_fields=["text"])
+```
+
+## Hybrid Search Pattern (Dense + Sparse)
+
+```python
+from pymilvus import MilvusClient, DataType, AnnSearchRequest, RRFRanker
+
+# Ask user for connection details
+client = MilvusClient(uri="<USER_URI>", token="<USER_TOKEN>")
+
+# Schema with both dense and sparse vectors
+schema = client.create_schema(auto_id=True)
+schema.add_field("id", DataType.INT64, is_primary=True)
+schema.add_field("text", DataType.VARCHAR, max_length=2048)
+schema.add_field("dense_embedding", DataType.FLOAT_VECTOR, dim=768)
+schema.add_field("sparse_embedding", DataType.SPARSE_FLOAT_VECTOR)
+
+index_params = client.prepare_index_params()
+index_params.add_index(field_name="dense_embedding", index_type="AUTOINDEX", metric_type="COSINE")
+index_params.add_index(field_name="sparse_embedding", index_type="SPARSE_INVERTED_INDEX", metric_type="IP")
+
+client.create_collection(collection_name="hybrid_col", schema=schema, index_params=index_params)
+
+# Search with both vectors and fuse results
+# dense_query_vector and sparse_query_vector come from your respective embedding models
+req1 = AnnSearchRequest(data=[dense_query_vector], anns_field="dense_embedding",
+                         param={"metric_type": "COSINE"}, limit=20)
+req2 = AnnSearchRequest(data=[sparse_query_vector], anns_field="sparse_embedding",
+                         param={"metric_type": "IP"}, limit=20)
+
+results = client.hybrid_search(
+    collection_name="hybrid_col",
+    reqs=[req1, req2],
+    ranker=RRFRanker(k=60),
+    limit=10,
+    output_fields=["text"]
+)
+```
+
+## Full-Text Search Pattern (BM25)
+
+```python
+from pymilvus import MilvusClient, DataType, Function, FunctionType
+
+# Ask user for connection details
+client = MilvusClient(uri="<USER_URI>", token="<USER_TOKEN>")
+
+schema = client.create_schema(auto_id=True)
+schema.add_field("id", DataType.INT64, is_primary=True)
+schema.add_field("title", DataType.VARCHAR, max_length=512)
+schema.add_field("body", DataType.VARCHAR, max_length=4096, enable_analyzer=True)
+schema.add_field("body_sparse", DataType.SPARSE_FLOAT_VECTOR)
+
+schema.add_function(Function(
+    name="body_bm25",
+    input_field_names=["body"],
+    output_field_names=["body_sparse"],
+    function_type=FunctionType.BM25,
+))
+
+index_params = client.prepare_index_params()
+index_params.add_index(field_name="body_sparse", index_type="AUTOINDEX", metric_type="BM25")
+
+client.create_collection(collection_name="articles", schema=schema, index_params=index_params)
+
+# Insert — only provide text, sparse vector is auto-generated by BM25 function
+client.insert("articles", data=[
+    {"title": "Intro to ML", "body": "Machine learning is a subset of artificial intelligence..."},
+])
+
+# Search with raw text — no embedding model needed, Milvus handles BM25 tokenization
+results = client.search(
+    collection_name="articles",
+    data=["machine learning fundamentals"],
+    anns_field="body_sparse",
+    limit=10,
+    output_fields=["title", "body"]
+)
+```

package/config/agent/skills_bak/milvus-skill/references/user-role.md

@@ -0,0 +1,90 @@
+# User & Role Management (RBAC)
+
+## User Operations
+
+```python
+# Create a user
+client.create_user(user_name="analyst", password="SecureP@ss123")
+
+# List users
+users = client.list_users()
+
+# Describe a user (shows assigned roles)
+info = client.describe_user(user_name="analyst")
+
+# Update password
+client.update_password(user_name="analyst", old_password="SecureP@ss123", new_password="NewP@ss456")
+
+# Grant role to user
+client.grant_role(user_name="analyst", role_name="read_only")
+
+# Revoke role from user
+client.revoke_role(user_name="analyst", role_name="read_only")
+
+# Drop a user
+client.drop_user(user_name="analyst")
+```
+
+## Role Operations
+
+```python
+# Create a role
+client.create_role(role_name="read_only")
+
+# List roles
+roles = client.list_roles()
+
+# Grant privilege (v2 API — recommended)
+client.grant_privilege_v2(
+    role_name="read_only",
+    privilege="Search",                 # e.g., "Search", "Insert", "Query", "Delete"
+    collection_name="my_collection",    # Use "*" for all collections
+    db_name="default"                   # Use "*" for all databases
+)
+
+# Built-in privilege groups
+client.grant_privilege_v2(
+    role_name="admin_role",
+    privilege="ClusterAdmin",           # See privilege groups below
+    collection_name="*",
+    db_name="*"
+)
+
+# Revoke privilege
+client.revoke_privilege_v2(
+    role_name="read_only",
+    privilege="Search",
+    collection_name="my_collection",
+    db_name="default"
+)
+
+# Describe role (see granted privileges)
+info = client.describe_role(role_name="read_only")
+
+# Drop a role
+client.drop_role(role_name="read_only")
+```
+
+## Built-in Privilege Groups
+
+| Group | Scope |
+|-------|-------|
+| `ClusterAdmin` | Full cluster access |
+| `ClusterReadOnly` | Read-only cluster access |
+| `ClusterReadWrite` | Read-write cluster access |
+| `DatabaseAdmin` | Full database access |
+| `DatabaseReadOnly` | Read-only database access |
+| `DatabaseReadWrite` | Read-write database access |
+| `CollectionAdmin` | Full collection access |
+| `CollectionReadOnly` | Read-only collection access |
+| `CollectionReadWrite` | Read-write collection access |
+
+## Common Individual Privileges
+
+`Search`, `Query`, `Insert`, `Delete`, `Upsert`, `CreateIndex`, `DropIndex`, `CreateCollection`, `DropCollection`, `Load`, `Release`, `CreatePartition`, `DropPartition`
+
+## Guidance
+
+- Recommended workflow: create role -> grant privileges -> create user -> assign role.
+- Use `"*"` for collection_name/db_name to grant on all resources.
+- Before dropping a user or role, confirm with the user.

package/config/agent/skills_bak/milvus-skill/references/vector.md

@@ -0,0 +1,247 @@
+# Vector Operations — Detailed Reference
+
+Target collection must exist and be loaded.
+
+> **Never use fake or placeholder vectors** (e.g., `[0.1, 0.2, ...]`). Always generate vectors from an embedding model. Use `pip install "pymilvus[model]"` for built-in embedding support, or use any external embedding model (OpenAI, Cohere, HuggingFace, etc.).
+
+## Embedding Model Setup
+
+```python
+# Option 1: Built-in pymilvus embedding (pip install "pymilvus[model]")
+from pymilvus import model
+embedding_fn = model.dense.SentenceTransformerEmbeddingFunction(model_name="all-MiniLM-L6-v2")
+
+# Encode documents for insertion
+doc_vectors = embedding_fn.encode_documents(["AI advances in 2024", "ML basics for beginners"])
+# Encode queries for search (use the SAME model that encoded the stored documents)
+query_vectors = embedding_fn.encode_queries(["What is artificial intelligence?"])
+
+# Option 2: OpenAI embeddings (pip install openai)
+from openai import OpenAI
+openai_client = OpenAI()
+response = openai_client.embeddings.create(input=["some text"], model="text-embedding-3-small")
+vectors = [item.embedding for item in response.data]
+```
+
+## Insert
+
+```python
+docs = ["AI advances in 2024", "ML basics for beginners"]
+vectors = embedding_fn.encode_documents(docs)
+
+data = [
+    {"id": 1, "text": docs[0], "embedding": vectors[0]},
+    {"id": 2, "text": docs[1], "embedding": vectors[1]},
+]
+res = client.insert(collection_name="my_collection", data=data)
+# Returns: {"insert_count": 2, "ids": [1, 2]}
+```
+
+## Upsert (insert or update if PK exists)
+
+```python
+res = client.upsert(collection_name="my_collection", data=data)
+# Returns: {"upsert_count": 2}
+```
+
+## Search (vector similarity)
+
+```python
+query_vectors = embedding_fn.encode_queries(["What is artificial intelligence?"])
+
+results = client.search(
+    collection_name="my_collection",
+    data=query_vectors,                 # Vectors from embedding model
+    anns_field="embedding",             # Vector field name
+    limit=10,                           # Top-K
+    output_fields=["text", "id"],       # Fields to return
+    filter='age > 20 and status == "active"',  # Optional scalar filter
+    search_params={
+        "metric_type": "COSINE",
+        "params": {"nprobe": 10}        # Index-specific params
+    }
+)
+# Returns: List[List[dict]]
+# Each hit: {"id": 1, "distance": 0.95, "entity": {"text": "AI advances..."}}
+```
+
+## Hybrid Search (multi-vector with reranking)
+
+```python
+from pymilvus import AnnSearchRequest, RRFRanker, WeightedRanker
+
+# dense_query_vector and sparse_query_vector come from your embedding models
+req1 = AnnSearchRequest(
+    data=[dense_query_vector],
+    anns_field="dense_embedding",
+    param={"metric_type": "COSINE", "params": {"nprobe": 10}},
+    limit=10
+)
+req2 = AnnSearchRequest(
+    data=[{1: 0.5, 100: 0.3}],          # Sparse vector
+    anns_field="sparse_embedding",
+    param={"metric_type": "IP"},
+    limit=10
+)
+
+# RRF reranking
+results = client.hybrid_search(
+    collection_name="my_collection",
+    reqs=[req1, req2],
+    ranker=RRFRanker(k=60),
+    limit=10,
+    output_fields=["text"]
+)
+
+# Or weighted reranking
+results = client.hybrid_search(
+    collection_name="my_collection",
+    reqs=[req1, req2],
+    ranker=WeightedRanker(0.7, 0.3),
+    limit=10
+)
+```
+
+## Full-Text Search
+
+Full-text search uses Milvus's built-in BM25 tokenizer to convert text into sparse vectors automatically.
+
+### Setup: Collection with Full-Text Search
+
+```python
+from pymilvus import MilvusClient, DataType, Function, FunctionType
+
+client = MilvusClient(uri="<USER_URI>", token="<USER_TOKEN>")
+
+schema = client.create_schema()
+schema.add_field("id", DataType.INT64, is_primary=True, auto_id=True)
+schema.add_field("text", DataType.VARCHAR, max_length=1000, enable_analyzer=True)
+schema.add_field("sparse", DataType.SPARSE_FLOAT_VECTOR)
+
+# Define BM25 function to auto-convert text -> sparse vector
+bm25_function = Function(
+    name="text_bm25",
+    input_field_names=["text"],
+    output_field_names=["sparse"],
+    function_type=FunctionType.BM25,
+)
+schema.add_function(bm25_function)
+
+index_params = client.prepare_index_params()
+index_params.add_index(field_name="sparse", index_type="AUTOINDEX", metric_type="BM25")
+
+client.create_collection(collection_name="full_text_col", schema=schema, index_params=index_params)
+```
+
+### Search with Text
+
+```python
+results = client.search(
+    collection_name="full_text_col",
+    data=["machine learning algorithms"],   # Raw text query
+    anns_field="sparse",
+    limit=10,
+    output_fields=["text"]
+)
+```
+
+## Search Iterator (paginated search over large results)
+
+```python
+query_vectors = embedding_fn.encode_queries(["search query text"])
+
+iterator = client.search_iterator(
+    collection_name="my_collection",
+    data=query_vectors,
+    anns_field="embedding",
+    batch_size=100,
+    limit=10000,
+    output_fields=["text"],
+    search_params={"metric_type": "COSINE", "params": {"nprobe": 10}}
+)
+
+results = []
+while True:
+    batch = iterator.next()
+    if not batch:
+        break
+    results.extend(batch)
+
+iterator.close()
+```
+
+## Query Iterator (paginated filter-based retrieval)
+
+```python
+iterator = client.query_iterator(
+    collection_name="my_collection",
+    filter='age > 20',
+    output_fields=["text", "age"],
+    batch_size=100,
+    limit=10000
+)
+
+results = []
+while True:
+    batch = iterator.next()
+    if not batch:
+        break
+    results.extend(batch)
+
+iterator.close()
+```
+
+## Query (filter-based retrieval)
+
+```python
+results = client.query(
+    collection_name="my_collection",
+    filter='id in [1, 2, 3]',
+    output_fields=["text", "embedding"],
+    limit=100
+)
+```
+
+## Get (by primary key)
+
+```python
+results = client.get(
+    collection_name="my_collection",
+    ids=[1, 2, 3],
+    output_fields=["text"]
+)
+```
+
+## Delete
+
+```python
+# By primary keys
+client.delete(collection_name="my_collection", ids=[1, 2, 3])
+
+# By filter expression
+client.delete(collection_name="my_collection", filter='status == "obsolete"')
+```
+
+## Filter Expression Syntax
+
+| Expression | Example |
+|---|---|
+| Comparison | `age > 20` |
+| Equality | `status == "active"` |
+| IN list | `id in [1, 2, 3]` |
+| AND/OR | `age > 20 and status == "active"` |
+| String match | `text like "hello%"` |
+| Array contains | `ARRAY_CONTAINS(tags, "ml")` |
+| JSON field | `json_field["key"] > 100` |
+| Match all | `id > 0` |
+
+## Guidance
+
+- **Never use fake or placeholder vectors.** Always generate vectors from an embedding model.
+- The query embedding model must be the **same model** used to generate the stored vectors.
+- The `data` parameter in search must match the collection's vector dimension exactly.
+- For full-text search, pass raw text strings as `data` — Milvus handles tokenization via BM25.
+- For large inserts, batch data into chunks (e.g., 1000 rows per batch).
+- Always specify `output_fields` to control which fields are returned.
+- For large result sets, use `search_iterator` or `query_iterator` instead of increasing `limit`.
+- Always call `iterator.close()` when done to release server resources.