@ia-ccun/code-agent-cli 0.0.52 → 0.0.53
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/config/agent/bin/README.md +102 -0
- package/config/agent/bin/fd.exe +0 -0
- package/config/agent/bin/rg.exe +0 -0
- package/config/agent/extensions/aicode-advisor.ts +13 -0
- package/config/agent/extensions/aicode-btw.ts +66 -14
- package/config/agent/extensions/aicode-cd.ts +834 -0
- package/config/agent/extensions/aicode-grillMe.ts +198 -0
- package/config/agent/extensions/aicode-pets.ts +362 -0
- package/config/agent/extensions/aicode-safety-guard.ts +552 -0
- package/config/agent/extensions/aicode-tips.ts +70 -16
- package/config/agent/extensions/aicode-todo-progress.ts +547 -0
- package/config/agent/extensions/custom.ts +93 -95
- package/config/agent/extensions/error-monitor.ts +1 -18
- package/config/agent/extensions/ponytail.ts +441 -0
- package/config/agent/extensions/tps.ts +87 -0
- package/config/agent/extensions/working-msg.ts +165 -50
- package/config/agent/help.md +0 -7
- package/config/agent/models.json +7 -7
- package/config/agent/settings.json +2 -1
- package/config/agent/skills/chroma/LICENSE +21 -0
- package/config/agent/skills/chroma/chroma-local/SKILL.md +165 -0
- package/config/agent/skills/chroma/chroma-local/cli.md +48 -0
- package/config/agent/skills/chroma/chroma-local/data-model.md +103 -0
- package/config/agent/skills/chroma/chroma-local/error-handling/python.md +166 -0
- package/config/agent/skills/chroma/chroma-local/error-handling/typescript.md +231 -0
- package/config/agent/skills/chroma/chroma-local/local-chroma/python.md +95 -0
- package/config/agent/skills/chroma/chroma-local/local-chroma/typescript.md +101 -0
- package/config/agent/skills/chroma/chroma-local/metadata/python.md +154 -0
- package/config/agent/skills/chroma/chroma-local/metadata/typescript.md +153 -0
- package/config/agent/skills/chroma/chroma-local/querying/python.md +147 -0
- package/config/agent/skills/chroma/chroma-local/querying/typescript.md +157 -0
- package/config/agent/skills/chroma/chroma-local/regex/python.md +69 -0
- package/config/agent/skills/chroma/chroma-local/regex/typescript.md +71 -0
- package/config/agent/skills/chroma/chroma-local/understanding-a-codebase.md +80 -0
- package/config/agent/skills/chroma/chroma-local/updating-deleting/python.md +159 -0
- package/config/agent/skills/chroma/chroma-local/updating-deleting/typescript.md +228 -0
- package/config/agent/skills/milvus-skill/LICENSE +202 -0
- package/config/agent/skills/milvus-skill/README.md +37 -0
- package/config/agent/skills/milvus-skill/SKILL.md +259 -0
- package/config/agent/skills/milvus-skill/references/collection.md +86 -0
- package/config/agent/skills/milvus-skill/references/database.md +24 -0
- package/config/agent/skills/milvus-skill/references/index.md +69 -0
- package/config/agent/skills/milvus-skill/references/partition.md +29 -0
- package/config/agent/skills/milvus-skill/references/patterns.md +150 -0
- package/config/agent/skills/milvus-skill/references/user-role.md +90 -0
- package/config/agent/skills/milvus-skill/references/vector.md +247 -0
- package/config/agent/skills/ponytail/SKILL.md +100 -0
- package/config/agent/skills/ponytail-audit/SKILL.md +40 -0
- package/config/agent/skills/ponytail-debt/SKILL.md +44 -0
- package/config/agent/skills/ponytail-help/SKILL.md +68 -0
- package/config/agent/skills/ponytail-review/SKILL.md +56 -0
- package/config/agent/skills/zread/README.md +115 -0
- package/config/agent/skills/zread/SKILL.md +103 -0
- package/config/agent/skills/zread/references/stdio-protocol.md +312 -0
- package/package.json +7 -6
- package/scripts/postinstall.js +58 -1
- package/config/agent/dangerous-patterns.json +0 -106
- package/config/agent/extensions/bash-check.ts +0 -227
|
@@ -0,0 +1,29 @@
|
|
|
1
|
+
# Partition Management
|
|
2
|
+
|
|
3
|
+
```python
|
|
4
|
+
# Create a partition
|
|
5
|
+
client.create_partition(collection_name="my_collection", partition_name="partition_A")
|
|
6
|
+
|
|
7
|
+
# List partitions
|
|
8
|
+
partitions = client.list_partitions(collection_name="my_collection")
|
|
9
|
+
# Returns: ["_default", "partition_A"]
|
|
10
|
+
|
|
11
|
+
# Check if partition exists
|
|
12
|
+
exists = client.has_partition(collection_name="my_collection", partition_name="partition_A")
|
|
13
|
+
|
|
14
|
+
# Load specific partitions
|
|
15
|
+
client.load_partitions(collection_name="my_collection", partition_names=["partition_A"])
|
|
16
|
+
|
|
17
|
+
# Release specific partitions
|
|
18
|
+
client.release_partitions(collection_name="my_collection", partition_names=["partition_A"])
|
|
19
|
+
|
|
20
|
+
# Drop a partition
|
|
21
|
+
client.drop_partition(collection_name="my_collection", partition_name="partition_A")
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
## Guidance
|
|
25
|
+
|
|
26
|
+
- Every collection has a `_default` partition.
|
|
27
|
+
- Use `is_partition_key=True` on a field to enable automatic partitioning by field value.
|
|
28
|
+
- A partition must be loaded before search.
|
|
29
|
+
- Before dropping a partition, confirm with the user — all data in it will be deleted.
|
|
@@ -0,0 +1,150 @@
|
|
|
1
|
+
# Common Patterns
|
|
2
|
+
|
|
3
|
+
> **Note:** All patterns below use `<USER_URI>` and `<USER_TOKEN>` as connection placeholders. Always ask the user for their actual connection details before writing code. For local development, use Milvus Lite (`uri="./milvus.db"`) only if the user explicitly requests it.
|
|
4
|
+
|
|
5
|
+
## RAG Pipeline Pattern
|
|
6
|
+
|
|
7
|
+
```python
|
|
8
|
+
from pymilvus import MilvusClient, DataType, model
|
|
9
|
+
|
|
10
|
+
# 1. Connect (ask user for URI and credentials)
|
|
11
|
+
client = MilvusClient(uri="<USER_URI>", token="<USER_TOKEN>")
|
|
12
|
+
|
|
13
|
+
# 2. Set up embedding model
|
|
14
|
+
embedding_fn = model.dense.SentenceTransformerEmbeddingFunction(model_name="all-MiniLM-L6-v2")
|
|
15
|
+
|
|
16
|
+
# 3. Create collection (dim must match embedding model output)
|
|
17
|
+
schema = client.create_schema(auto_id=True, enable_dynamic_field=True)
|
|
18
|
+
schema.add_field("id", DataType.INT64, is_primary=True)
|
|
19
|
+
schema.add_field("text", DataType.VARCHAR, max_length=2048)
|
|
20
|
+
schema.add_field("embedding", DataType.FLOAT_VECTOR, dim=384) # all-MiniLM-L6-v2 outputs 384-dim
|
|
21
|
+
schema.add_field("source", DataType.VARCHAR, max_length=256)
|
|
22
|
+
|
|
23
|
+
index_params = client.prepare_index_params()
|
|
24
|
+
index_params.add_index(field_name="embedding", index_type="AUTOINDEX", metric_type="COSINE")
|
|
25
|
+
|
|
26
|
+
client.create_collection(collection_name="knowledge_base", schema=schema, index_params=index_params)
|
|
27
|
+
|
|
28
|
+
# 4. Insert documents — generate real vectors from text chunks
|
|
29
|
+
chunks = ["Milvus is a vector database...", "RAG combines retrieval and generation..."]
|
|
30
|
+
vectors = embedding_fn.encode_documents(chunks)
|
|
31
|
+
|
|
32
|
+
client.insert("knowledge_base", data=[
|
|
33
|
+
{"text": chunk, "embedding": vec, "source": "doc1.pdf"}
|
|
34
|
+
for chunk, vec in zip(chunks, vectors)
|
|
35
|
+
])
|
|
36
|
+
|
|
37
|
+
# 5. Retrieve relevant context — use the SAME embedding model
|
|
38
|
+
query = "What is a vector database?"
|
|
39
|
+
query_vectors = embedding_fn.encode_queries([query])
|
|
40
|
+
|
|
41
|
+
results = client.search(
|
|
42
|
+
collection_name="knowledge_base",
|
|
43
|
+
data=query_vectors,
|
|
44
|
+
limit=5,
|
|
45
|
+
output_fields=["text", "source"],
|
|
46
|
+
search_params={"metric_type": "COSINE"}
|
|
47
|
+
)
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
## Quick Semantic Search Pattern
|
|
51
|
+
|
|
52
|
+
```python
|
|
53
|
+
from pymilvus import MilvusClient, model
|
|
54
|
+
|
|
55
|
+
# Simplest possible setup (Milvus Lite for local dev)
|
|
56
|
+
client = MilvusClient(uri="./search.db")
|
|
57
|
+
embedding_fn = model.dense.SentenceTransformerEmbeddingFunction(model_name="all-MiniLM-L6-v2")
|
|
58
|
+
|
|
59
|
+
# Prepare data — vectors come from embedding model
|
|
60
|
+
texts = ["first document", "second document", "third document"]
|
|
61
|
+
vectors = embedding_fn.encode_documents(texts)
|
|
62
|
+
|
|
63
|
+
client.create_collection(collection_name="docs", dimension=384)
|
|
64
|
+
client.insert("docs", data=[
|
|
65
|
+
{"id": i, "vector": vec, "text": txt}
|
|
66
|
+
for i, (vec, txt) in enumerate(zip(vectors, texts))
|
|
67
|
+
])
|
|
68
|
+
|
|
69
|
+
# Search — encode query with the same model
|
|
70
|
+
query_vectors = embedding_fn.encode_queries(["search query"])
|
|
71
|
+
results = client.search("docs", data=query_vectors, limit=10, output_fields=["text"])
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
## Hybrid Search Pattern (Dense + Sparse)
|
|
75
|
+
|
|
76
|
+
```python
|
|
77
|
+
from pymilvus import MilvusClient, DataType, AnnSearchRequest, RRFRanker
|
|
78
|
+
|
|
79
|
+
# Ask user for connection details
|
|
80
|
+
client = MilvusClient(uri="<USER_URI>", token="<USER_TOKEN>")
|
|
81
|
+
|
|
82
|
+
# Schema with both dense and sparse vectors
|
|
83
|
+
schema = client.create_schema(auto_id=True)
|
|
84
|
+
schema.add_field("id", DataType.INT64, is_primary=True)
|
|
85
|
+
schema.add_field("text", DataType.VARCHAR, max_length=2048)
|
|
86
|
+
schema.add_field("dense_embedding", DataType.FLOAT_VECTOR, dim=768)
|
|
87
|
+
schema.add_field("sparse_embedding", DataType.SPARSE_FLOAT_VECTOR)
|
|
88
|
+
|
|
89
|
+
index_params = client.prepare_index_params()
|
|
90
|
+
index_params.add_index(field_name="dense_embedding", index_type="AUTOINDEX", metric_type="COSINE")
|
|
91
|
+
index_params.add_index(field_name="sparse_embedding", index_type="SPARSE_INVERTED_INDEX", metric_type="IP")
|
|
92
|
+
|
|
93
|
+
client.create_collection(collection_name="hybrid_col", schema=schema, index_params=index_params)
|
|
94
|
+
|
|
95
|
+
# Search with both vectors and fuse results
|
|
96
|
+
# dense_query_vector and sparse_query_vector come from your respective embedding models
|
|
97
|
+
req1 = AnnSearchRequest(data=[dense_query_vector], anns_field="dense_embedding",
|
|
98
|
+
param={"metric_type": "COSINE"}, limit=20)
|
|
99
|
+
req2 = AnnSearchRequest(data=[sparse_query_vector], anns_field="sparse_embedding",
|
|
100
|
+
param={"metric_type": "IP"}, limit=20)
|
|
101
|
+
|
|
102
|
+
results = client.hybrid_search(
|
|
103
|
+
collection_name="hybrid_col",
|
|
104
|
+
reqs=[req1, req2],
|
|
105
|
+
ranker=RRFRanker(k=60),
|
|
106
|
+
limit=10,
|
|
107
|
+
output_fields=["text"]
|
|
108
|
+
)
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
## Full-Text Search Pattern (BM25)
|
|
112
|
+
|
|
113
|
+
```python
|
|
114
|
+
from pymilvus import MilvusClient, DataType, Function, FunctionType
|
|
115
|
+
|
|
116
|
+
# Ask user for connection details
|
|
117
|
+
client = MilvusClient(uri="<USER_URI>", token="<USER_TOKEN>")
|
|
118
|
+
|
|
119
|
+
schema = client.create_schema(auto_id=True)
|
|
120
|
+
schema.add_field("id", DataType.INT64, is_primary=True)
|
|
121
|
+
schema.add_field("title", DataType.VARCHAR, max_length=512)
|
|
122
|
+
schema.add_field("body", DataType.VARCHAR, max_length=4096, enable_analyzer=True)
|
|
123
|
+
schema.add_field("body_sparse", DataType.SPARSE_FLOAT_VECTOR)
|
|
124
|
+
|
|
125
|
+
schema.add_function(Function(
|
|
126
|
+
name="body_bm25",
|
|
127
|
+
input_field_names=["body"],
|
|
128
|
+
output_field_names=["body_sparse"],
|
|
129
|
+
function_type=FunctionType.BM25,
|
|
130
|
+
))
|
|
131
|
+
|
|
132
|
+
index_params = client.prepare_index_params()
|
|
133
|
+
index_params.add_index(field_name="body_sparse", index_type="AUTOINDEX", metric_type="BM25")
|
|
134
|
+
|
|
135
|
+
client.create_collection(collection_name="articles", schema=schema, index_params=index_params)
|
|
136
|
+
|
|
137
|
+
# Insert — only provide text, sparse vector is auto-generated by BM25 function
|
|
138
|
+
client.insert("articles", data=[
|
|
139
|
+
{"title": "Intro to ML", "body": "Machine learning is a subset of artificial intelligence..."},
|
|
140
|
+
])
|
|
141
|
+
|
|
142
|
+
# Search with raw text — no embedding model needed, Milvus handles BM25 tokenization
|
|
143
|
+
results = client.search(
|
|
144
|
+
collection_name="articles",
|
|
145
|
+
data=["machine learning fundamentals"],
|
|
146
|
+
anns_field="body_sparse",
|
|
147
|
+
limit=10,
|
|
148
|
+
output_fields=["title", "body"]
|
|
149
|
+
)
|
|
150
|
+
```
|
|
@@ -0,0 +1,90 @@
|
|
|
1
|
+
# User & Role Management (RBAC)
|
|
2
|
+
|
|
3
|
+
## User Operations
|
|
4
|
+
|
|
5
|
+
```python
|
|
6
|
+
# Create a user
|
|
7
|
+
client.create_user(user_name="analyst", password="SecureP@ss123")
|
|
8
|
+
|
|
9
|
+
# List users
|
|
10
|
+
users = client.list_users()
|
|
11
|
+
|
|
12
|
+
# Describe a user (shows assigned roles)
|
|
13
|
+
info = client.describe_user(user_name="analyst")
|
|
14
|
+
|
|
15
|
+
# Update password
|
|
16
|
+
client.update_password(user_name="analyst", old_password="SecureP@ss123", new_password="NewP@ss456")
|
|
17
|
+
|
|
18
|
+
# Grant role to user
|
|
19
|
+
client.grant_role(user_name="analyst", role_name="read_only")
|
|
20
|
+
|
|
21
|
+
# Revoke role from user
|
|
22
|
+
client.revoke_role(user_name="analyst", role_name="read_only")
|
|
23
|
+
|
|
24
|
+
# Drop a user
|
|
25
|
+
client.drop_user(user_name="analyst")
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
## Role Operations
|
|
29
|
+
|
|
30
|
+
```python
|
|
31
|
+
# Create a role
|
|
32
|
+
client.create_role(role_name="read_only")
|
|
33
|
+
|
|
34
|
+
# List roles
|
|
35
|
+
roles = client.list_roles()
|
|
36
|
+
|
|
37
|
+
# Grant privilege (v2 API — recommended)
|
|
38
|
+
client.grant_privilege_v2(
|
|
39
|
+
role_name="read_only",
|
|
40
|
+
privilege="Search", # e.g., "Search", "Insert", "Query", "Delete"
|
|
41
|
+
collection_name="my_collection", # Use "*" for all collections
|
|
42
|
+
db_name="default" # Use "*" for all databases
|
|
43
|
+
)
|
|
44
|
+
|
|
45
|
+
# Built-in privilege groups
|
|
46
|
+
client.grant_privilege_v2(
|
|
47
|
+
role_name="admin_role",
|
|
48
|
+
privilege="ClusterAdmin", # See privilege groups below
|
|
49
|
+
collection_name="*",
|
|
50
|
+
db_name="*"
|
|
51
|
+
)
|
|
52
|
+
|
|
53
|
+
# Revoke privilege
|
|
54
|
+
client.revoke_privilege_v2(
|
|
55
|
+
role_name="read_only",
|
|
56
|
+
privilege="Search",
|
|
57
|
+
collection_name="my_collection",
|
|
58
|
+
db_name="default"
|
|
59
|
+
)
|
|
60
|
+
|
|
61
|
+
# Describe role (see granted privileges)
|
|
62
|
+
info = client.describe_role(role_name="read_only")
|
|
63
|
+
|
|
64
|
+
# Drop a role
|
|
65
|
+
client.drop_role(role_name="read_only")
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
## Built-in Privilege Groups
|
|
69
|
+
|
|
70
|
+
| Group | Scope |
|
|
71
|
+
|-------|-------|
|
|
72
|
+
| `ClusterAdmin` | Full cluster access |
|
|
73
|
+
| `ClusterReadOnly` | Read-only cluster access |
|
|
74
|
+
| `ClusterReadWrite` | Read-write cluster access |
|
|
75
|
+
| `DatabaseAdmin` | Full database access |
|
|
76
|
+
| `DatabaseReadOnly` | Read-only database access |
|
|
77
|
+
| `DatabaseReadWrite` | Read-write database access |
|
|
78
|
+
| `CollectionAdmin` | Full collection access |
|
|
79
|
+
| `CollectionReadOnly` | Read-only collection access |
|
|
80
|
+
| `CollectionReadWrite` | Read-write collection access |
|
|
81
|
+
|
|
82
|
+
## Common Individual Privileges
|
|
83
|
+
|
|
84
|
+
`Search`, `Query`, `Insert`, `Delete`, `Upsert`, `CreateIndex`, `DropIndex`, `CreateCollection`, `DropCollection`, `Load`, `Release`, `CreatePartition`, `DropPartition`
|
|
85
|
+
|
|
86
|
+
## Guidance
|
|
87
|
+
|
|
88
|
+
- Recommended workflow: create role -> grant privileges -> create user -> assign role.
|
|
89
|
+
- Use `"*"` for collection_name/db_name to grant on all resources.
|
|
90
|
+
- Before dropping a user or role, confirm with the user.
|
|
@@ -0,0 +1,247 @@
|
|
|
1
|
+
# Vector Operations — Detailed Reference
|
|
2
|
+
|
|
3
|
+
Target collection must exist and be loaded.
|
|
4
|
+
|
|
5
|
+
> **Never use fake or placeholder vectors** (e.g., `[0.1, 0.2, ...]`). Always generate vectors from an embedding model. Use `pip install "pymilvus[model]"` for built-in embedding support, or use any external embedding model (OpenAI, Cohere, HuggingFace, etc.).
|
|
6
|
+
|
|
7
|
+
## Embedding Model Setup
|
|
8
|
+
|
|
9
|
+
```python
|
|
10
|
+
# Option 1: Built-in pymilvus embedding (pip install "pymilvus[model]")
|
|
11
|
+
from pymilvus import model
|
|
12
|
+
embedding_fn = model.dense.SentenceTransformerEmbeddingFunction(model_name="all-MiniLM-L6-v2")
|
|
13
|
+
|
|
14
|
+
# Encode documents for insertion
|
|
15
|
+
doc_vectors = embedding_fn.encode_documents(["AI advances in 2024", "ML basics for beginners"])
|
|
16
|
+
# Encode queries for search (use the SAME model that encoded the stored documents)
|
|
17
|
+
query_vectors = embedding_fn.encode_queries(["What is artificial intelligence?"])
|
|
18
|
+
|
|
19
|
+
# Option 2: OpenAI embeddings (pip install openai)
|
|
20
|
+
from openai import OpenAI
|
|
21
|
+
openai_client = OpenAI()
|
|
22
|
+
response = openai_client.embeddings.create(input=["some text"], model="text-embedding-3-small")
|
|
23
|
+
vectors = [item.embedding for item in response.data]
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
## Insert
|
|
27
|
+
|
|
28
|
+
```python
|
|
29
|
+
docs = ["AI advances in 2024", "ML basics for beginners"]
|
|
30
|
+
vectors = embedding_fn.encode_documents(docs)
|
|
31
|
+
|
|
32
|
+
data = [
|
|
33
|
+
{"id": 1, "text": docs[0], "embedding": vectors[0]},
|
|
34
|
+
{"id": 2, "text": docs[1], "embedding": vectors[1]},
|
|
35
|
+
]
|
|
36
|
+
res = client.insert(collection_name="my_collection", data=data)
|
|
37
|
+
# Returns: {"insert_count": 2, "ids": [1, 2]}
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
## Upsert (insert or update if PK exists)
|
|
41
|
+
|
|
42
|
+
```python
|
|
43
|
+
res = client.upsert(collection_name="my_collection", data=data)
|
|
44
|
+
# Returns: {"upsert_count": 2}
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
## Search (vector similarity)
|
|
48
|
+
|
|
49
|
+
```python
|
|
50
|
+
query_vectors = embedding_fn.encode_queries(["What is artificial intelligence?"])
|
|
51
|
+
|
|
52
|
+
results = client.search(
|
|
53
|
+
collection_name="my_collection",
|
|
54
|
+
data=query_vectors, # Vectors from embedding model
|
|
55
|
+
anns_field="embedding", # Vector field name
|
|
56
|
+
limit=10, # Top-K
|
|
57
|
+
output_fields=["text", "id"], # Fields to return
|
|
58
|
+
filter='age > 20 and status == "active"', # Optional scalar filter
|
|
59
|
+
search_params={
|
|
60
|
+
"metric_type": "COSINE",
|
|
61
|
+
"params": {"nprobe": 10} # Index-specific params
|
|
62
|
+
}
|
|
63
|
+
)
|
|
64
|
+
# Returns: List[List[dict]]
|
|
65
|
+
# Each hit: {"id": 1, "distance": 0.95, "entity": {"text": "AI advances..."}}
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
## Hybrid Search (multi-vector with reranking)
|
|
69
|
+
|
|
70
|
+
```python
|
|
71
|
+
from pymilvus import AnnSearchRequest, RRFRanker, WeightedRanker
|
|
72
|
+
|
|
73
|
+
# dense_query_vector and sparse_query_vector come from your embedding models
|
|
74
|
+
req1 = AnnSearchRequest(
|
|
75
|
+
data=[dense_query_vector],
|
|
76
|
+
anns_field="dense_embedding",
|
|
77
|
+
param={"metric_type": "COSINE", "params": {"nprobe": 10}},
|
|
78
|
+
limit=10
|
|
79
|
+
)
|
|
80
|
+
req2 = AnnSearchRequest(
|
|
81
|
+
data=[{1: 0.5, 100: 0.3}], # Sparse vector
|
|
82
|
+
anns_field="sparse_embedding",
|
|
83
|
+
param={"metric_type": "IP"},
|
|
84
|
+
limit=10
|
|
85
|
+
)
|
|
86
|
+
|
|
87
|
+
# RRF reranking
|
|
88
|
+
results = client.hybrid_search(
|
|
89
|
+
collection_name="my_collection",
|
|
90
|
+
reqs=[req1, req2],
|
|
91
|
+
ranker=RRFRanker(k=60),
|
|
92
|
+
limit=10,
|
|
93
|
+
output_fields=["text"]
|
|
94
|
+
)
|
|
95
|
+
|
|
96
|
+
# Or weighted reranking
|
|
97
|
+
results = client.hybrid_search(
|
|
98
|
+
collection_name="my_collection",
|
|
99
|
+
reqs=[req1, req2],
|
|
100
|
+
ranker=WeightedRanker(0.7, 0.3),
|
|
101
|
+
limit=10
|
|
102
|
+
)
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
## Full-Text Search
|
|
106
|
+
|
|
107
|
+
Full-text search uses Milvus's built-in BM25 tokenizer to convert text into sparse vectors automatically.
|
|
108
|
+
|
|
109
|
+
### Setup: Collection with Full-Text Search
|
|
110
|
+
|
|
111
|
+
```python
|
|
112
|
+
from pymilvus import MilvusClient, DataType, Function, FunctionType
|
|
113
|
+
|
|
114
|
+
client = MilvusClient(uri="<USER_URI>", token="<USER_TOKEN>")
|
|
115
|
+
|
|
116
|
+
schema = client.create_schema()
|
|
117
|
+
schema.add_field("id", DataType.INT64, is_primary=True, auto_id=True)
|
|
118
|
+
schema.add_field("text", DataType.VARCHAR, max_length=1000, enable_analyzer=True)
|
|
119
|
+
schema.add_field("sparse", DataType.SPARSE_FLOAT_VECTOR)
|
|
120
|
+
|
|
121
|
+
# Define BM25 function to auto-convert text -> sparse vector
|
|
122
|
+
bm25_function = Function(
|
|
123
|
+
name="text_bm25",
|
|
124
|
+
input_field_names=["text"],
|
|
125
|
+
output_field_names=["sparse"],
|
|
126
|
+
function_type=FunctionType.BM25,
|
|
127
|
+
)
|
|
128
|
+
schema.add_function(bm25_function)
|
|
129
|
+
|
|
130
|
+
index_params = client.prepare_index_params()
|
|
131
|
+
index_params.add_index(field_name="sparse", index_type="AUTOINDEX", metric_type="BM25")
|
|
132
|
+
|
|
133
|
+
client.create_collection(collection_name="full_text_col", schema=schema, index_params=index_params)
|
|
134
|
+
```
|
|
135
|
+
|
|
136
|
+
### Search with Text
|
|
137
|
+
|
|
138
|
+
```python
|
|
139
|
+
results = client.search(
|
|
140
|
+
collection_name="full_text_col",
|
|
141
|
+
data=["machine learning algorithms"], # Raw text query
|
|
142
|
+
anns_field="sparse",
|
|
143
|
+
limit=10,
|
|
144
|
+
output_fields=["text"]
|
|
145
|
+
)
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
## Search Iterator (paginated search over large results)
|
|
149
|
+
|
|
150
|
+
```python
|
|
151
|
+
query_vectors = embedding_fn.encode_queries(["search query text"])
|
|
152
|
+
|
|
153
|
+
iterator = client.search_iterator(
|
|
154
|
+
collection_name="my_collection",
|
|
155
|
+
data=query_vectors,
|
|
156
|
+
anns_field="embedding",
|
|
157
|
+
batch_size=100,
|
|
158
|
+
limit=10000,
|
|
159
|
+
output_fields=["text"],
|
|
160
|
+
search_params={"metric_type": "COSINE", "params": {"nprobe": 10}}
|
|
161
|
+
)
|
|
162
|
+
|
|
163
|
+
results = []
|
|
164
|
+
while True:
|
|
165
|
+
batch = iterator.next()
|
|
166
|
+
if not batch:
|
|
167
|
+
break
|
|
168
|
+
results.extend(batch)
|
|
169
|
+
|
|
170
|
+
iterator.close()
|
|
171
|
+
```
|
|
172
|
+
|
|
173
|
+
## Query Iterator (paginated filter-based retrieval)
|
|
174
|
+
|
|
175
|
+
```python
|
|
176
|
+
iterator = client.query_iterator(
|
|
177
|
+
collection_name="my_collection",
|
|
178
|
+
filter='age > 20',
|
|
179
|
+
output_fields=["text", "age"],
|
|
180
|
+
batch_size=100,
|
|
181
|
+
limit=10000
|
|
182
|
+
)
|
|
183
|
+
|
|
184
|
+
results = []
|
|
185
|
+
while True:
|
|
186
|
+
batch = iterator.next()
|
|
187
|
+
if not batch:
|
|
188
|
+
break
|
|
189
|
+
results.extend(batch)
|
|
190
|
+
|
|
191
|
+
iterator.close()
|
|
192
|
+
```
|
|
193
|
+
|
|
194
|
+
## Query (filter-based retrieval)
|
|
195
|
+
|
|
196
|
+
```python
|
|
197
|
+
results = client.query(
|
|
198
|
+
collection_name="my_collection",
|
|
199
|
+
filter='id in [1, 2, 3]',
|
|
200
|
+
output_fields=["text", "embedding"],
|
|
201
|
+
limit=100
|
|
202
|
+
)
|
|
203
|
+
```
|
|
204
|
+
|
|
205
|
+
## Get (by primary key)
|
|
206
|
+
|
|
207
|
+
```python
|
|
208
|
+
results = client.get(
|
|
209
|
+
collection_name="my_collection",
|
|
210
|
+
ids=[1, 2, 3],
|
|
211
|
+
output_fields=["text"]
|
|
212
|
+
)
|
|
213
|
+
```
|
|
214
|
+
|
|
215
|
+
## Delete
|
|
216
|
+
|
|
217
|
+
```python
|
|
218
|
+
# By primary keys
|
|
219
|
+
client.delete(collection_name="my_collection", ids=[1, 2, 3])
|
|
220
|
+
|
|
221
|
+
# By filter expression
|
|
222
|
+
client.delete(collection_name="my_collection", filter='status == "obsolete"')
|
|
223
|
+
```
|
|
224
|
+
|
|
225
|
+
## Filter Expression Syntax
|
|
226
|
+
|
|
227
|
+
| Expression | Example |
|
|
228
|
+
|---|---|
|
|
229
|
+
| Comparison | `age > 20` |
|
|
230
|
+
| Equality | `status == "active"` |
|
|
231
|
+
| IN list | `id in [1, 2, 3]` |
|
|
232
|
+
| AND/OR | `age > 20 and status == "active"` |
|
|
233
|
+
| String match | `text like "hello%"` |
|
|
234
|
+
| Array contains | `ARRAY_CONTAINS(tags, "ml")` |
|
|
235
|
+
| JSON field | `json_field["key"] > 100` |
|
|
236
|
+
| Match all | `id > 0` |
|
|
237
|
+
|
|
238
|
+
## Guidance
|
|
239
|
+
|
|
240
|
+
- **Never use fake or placeholder vectors.** Always generate vectors from an embedding model.
|
|
241
|
+
- The query embedding model must be the **same model** used to generate the stored vectors.
|
|
242
|
+
- The `data` parameter in search must match the collection's vector dimension exactly.
|
|
243
|
+
- For full-text search, pass raw text strings as `data` — Milvus handles tokenization via BM25.
|
|
244
|
+
- For large inserts, batch data into chunks (e.g., 1000 rows per batch).
|
|
245
|
+
- Always specify `output_fields` to control which fields are returned.
|
|
246
|
+
- For large result sets, use `search_iterator` or `query_iterator` instead of increasing `limit`.
|
|
247
|
+
- Always call `iterator.close()` when done to release server resources.
|
|
@@ -0,0 +1,100 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: ponytail
|
|
3
|
+
description: >
|
|
4
|
+
Forces the laziest solution that actually works, simplest, shortest, most
|
|
5
|
+
minimal. Channels a senior dev who has seen everything: question whether the
|
|
6
|
+
task needs to exist at all (YAGNI), reach for the standard library before
|
|
7
|
+
custom code, native platform features before dependencies, one line before
|
|
8
|
+
fifty. Supports intensity levels: lite, full (default), ultra. Use whenever
|
|
9
|
+
the user says "ponytail", "be lazy", "lazy mode", "simplest solution",
|
|
10
|
+
"minimal solution", "yagni", "do less", or "shortest path", and whenever
|
|
11
|
+
they complain about over-engineering, bloat, boilerplate, or unnecessary
|
|
12
|
+
dependencies.
|
|
13
|
+
license: MIT
|
|
14
|
+
---
|
|
15
|
+
|
|
16
|
+
# Ponytail
|
|
17
|
+
|
|
18
|
+
You are a lazy senior developer. Lazy means efficient, not careless. You have
|
|
19
|
+
seen every over-engineered codebase and been paged at 3am for one. The best
|
|
20
|
+
code is the code never written.
|
|
21
|
+
|
|
22
|
+
## Persistence
|
|
23
|
+
|
|
24
|
+
ACTIVE EVERY RESPONSE. No drift back to over-building. Still active if
|
|
25
|
+
unsure. Off only: "stop ponytail" / "normal mode". Default: **full**.
|
|
26
|
+
Switch: `/ponytail lite|full|ultra`.
|
|
27
|
+
|
|
28
|
+
## The ladder
|
|
29
|
+
|
|
30
|
+
Stop at the first rung that holds:
|
|
31
|
+
|
|
32
|
+
1. **Does this need to exist at all?** Speculative need = skip it, say so in one line. (YAGNI)
|
|
33
|
+
2. **Stdlib does it?** Use it.
|
|
34
|
+
3. **Native platform feature covers it?** `<input type="date">` over a picker lib, CSS over JS, DB constraint over app code.
|
|
35
|
+
4. **Already-installed dependency solves it?** Use it. Never add a new one for what a few lines can do.
|
|
36
|
+
5. **Can it be one line?** One line.
|
|
37
|
+
6. **Only then:** the minimum code that works.
|
|
38
|
+
|
|
39
|
+
The ladder is a reflex, not a research project. Two rungs work → take the
|
|
40
|
+
higher one and move on. The first lazy solution that works is the right one.
|
|
41
|
+
|
|
42
|
+
## Rules
|
|
43
|
+
|
|
44
|
+
- No unrequested abstractions: no interface with one implementation, no factory for one product, no config for a value that never changes.
|
|
45
|
+
- No boilerplate, no scaffolding "for later", later can scaffold for itself.
|
|
46
|
+
- Deletion over addition. Boring over clever, clever is what someone decodes at 3am.
|
|
47
|
+
- Fewest files possible. Shortest working diff wins.
|
|
48
|
+
- Complex request? Ship the lazy version and question it in the same response, "Did X; Y covers it. Need full X? Say so." Never stall on an answer you can default.
|
|
49
|
+
- Two stdlib options, same size? Take the one that's correct on edge cases. Lazy means writing less code, not picking the flimsier algorithm.
|
|
50
|
+
- Mark deliberate simplifications with a `ponytail:` comment (`// ponytail: this exists`), simple reads as intent, not ignorance. Shortcut with a known ceiling (global lock, O(n²) scan, naive heuristic)? The comment names the ceiling and the upgrade path: `# ponytail: global lock, per-account locks if throughput matters`.
|
|
51
|
+
|
|
52
|
+
## Output
|
|
53
|
+
|
|
54
|
+
Code first. Then at most three short lines: what was skipped, when to add it.
|
|
55
|
+
No essays, no feature tours, no design notes. If the explanation is longer
|
|
56
|
+
than the code, delete the explanation, every paragraph defending a
|
|
57
|
+
simplification is complexity smuggled back in as prose. Explanation the user
|
|
58
|
+
explicitly asked for (a report, a walkthrough, per-phase notes) is not debt,
|
|
59
|
+
give it in full, the rule is only against unrequested prose.
|
|
60
|
+
|
|
61
|
+
Pattern: `[code] → skipped: [X], add when [Y].`
|
|
62
|
+
|
|
63
|
+
## Intensity
|
|
64
|
+
|
|
65
|
+
| Level | What change |
|
|
66
|
+
|-------|------------|
|
|
67
|
+
| **lite** | Build what's asked, but name the lazier alternative in one line. User picks. |
|
|
68
|
+
| **full** | The ladder enforced. Stdlib and native first. Shortest diff, shortest explanation. Default. |
|
|
69
|
+
| **ultra** | YAGNI extremist. Deletion before addition. Ship the one-liner and challenge the rest of the requirement in the same breath. |
|
|
70
|
+
|
|
71
|
+
Example: "Add a cache for these API responses."
|
|
72
|
+
- lite: "Done, cache added. FYI: `functools.lru_cache` covers this in one line if you'd rather not own a cache class."
|
|
73
|
+
- full: "`@lru_cache(maxsize=1000)` on the fetch function. Skipped custom cache class, add when lru_cache measurably falls short."
|
|
74
|
+
- ultra: "No cache until a profiler says so. When it does: `@lru_cache`. A hand-rolled TTL cache class is a bug farm with a hit rate."
|
|
75
|
+
|
|
76
|
+
## When NOT to be lazy
|
|
77
|
+
|
|
78
|
+
Never simplify away: input validation at trust boundaries, error handling
|
|
79
|
+
that prevents data loss, security measures, accessibility basics, anything
|
|
80
|
+
explicitly requested. User insists on the full version → build it, no
|
|
81
|
+
re-arguing.
|
|
82
|
+
|
|
83
|
+
Hardware is never the ideal on paper: a real clock drifts, a real sensor
|
|
84
|
+
reads off, a PCA9685 runs a few percent fast. Leave the calibration knob, not
|
|
85
|
+
just less code, the physical world needs tuning a minimal model can't see.
|
|
86
|
+
|
|
87
|
+
Lazy code without its check is unfinished. Non-trivial logic (a branch, a
|
|
88
|
+
loop, a parser, a money/security path) leaves ONE runnable check behind, the
|
|
89
|
+
smallest thing that fails if the logic breaks: an `assert`-based
|
|
90
|
+
`demo()`/`__main__` self-check or one small `test_*.py`. No frameworks, no
|
|
91
|
+
fixtures, no per-function suites unless asked. Trivial one-liners need no
|
|
92
|
+
test, YAGNI applies to tests too.
|
|
93
|
+
|
|
94
|
+
## Boundaries
|
|
95
|
+
|
|
96
|
+
Ponytail governs what you build, not how you talk (pair with Caveman for
|
|
97
|
+
terse prose). "stop ponytail" / "normal mode": revert. Level persists until
|
|
98
|
+
changed or session end.
|
|
99
|
+
|
|
100
|
+
The shortest path to done is the right path.
|
|
@@ -0,0 +1,40 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: ponytail-audit
|
|
3
|
+
description: >
|
|
4
|
+
Whole-repo audit for over-engineering. Like ponytail-review, but scans the
|
|
5
|
+
entire codebase instead of a diff: a ranked list of what to delete, simplify,
|
|
6
|
+
or replace with stdlib/native equivalents. Use when the user says "audit this
|
|
7
|
+
codebase", "audit for over-engineering", "what can I delete from this repo",
|
|
8
|
+
"find bloat", "ponytail-audit", or "/ponytail-audit". One-shot report, does
|
|
9
|
+
not apply fixes.
|
|
10
|
+
---
|
|
11
|
+
|
|
12
|
+
ponytail-review, repo-wide. Scan the whole tree instead of a diff. Rank
|
|
13
|
+
findings biggest cut first.
|
|
14
|
+
|
|
15
|
+
## Tags
|
|
16
|
+
|
|
17
|
+
Same as ponytail-review:
|
|
18
|
+
|
|
19
|
+
- `delete:` dead code, unused flexibility, speculative feature. Replacement: nothing.
|
|
20
|
+
- `stdlib:` hand-rolled thing the standard library ships. Name the function.
|
|
21
|
+
- `native:` dependency or code doing what the platform already does. Name the feature.
|
|
22
|
+
- `yagni:` abstraction with one implementation, config nobody sets, layer with one caller.
|
|
23
|
+
- `shrink:` same logic, fewer lines. Show the shorter form.
|
|
24
|
+
|
|
25
|
+
## Hunt
|
|
26
|
+
|
|
27
|
+
Deps the stdlib or platform already ships, single-implementation interfaces,
|
|
28
|
+
factories with one product, wrappers that only delegate, files exporting one
|
|
29
|
+
thing, dead flags and config, hand-rolled stdlib.
|
|
30
|
+
|
|
31
|
+
## Output
|
|
32
|
+
|
|
33
|
+
One line per finding, ranked: `<tag> <what to cut>. <replacement>. [path]`.
|
|
34
|
+
End with `net: -<N> lines, -<M> deps possible.` Nothing to cut: `Lean already. Ship.`
|
|
35
|
+
|
|
36
|
+
## Boundaries
|
|
37
|
+
|
|
38
|
+
Complexity only, correctness bugs, security holes, and performance go to a
|
|
39
|
+
normal review pass. Lists findings, applies nothing. One-shot.
|
|
40
|
+
"stop ponytail-audit" or "normal mode" to revert.
|