@ia-ccun/code-agent-cli 0.0.52 → 0.0.53

package/config/agent/skills/chroma/chroma-local/regex/typescript.md

@@ -0,0 +1,71 @@
+---
+name: Chroma Regex Filtering
+description: Learn how to use regex filters in Chroma queries
+---
+
+## Regex Filtering in Chroma
+
+Chroma supports regex filtering on document content. This is useful when you need exact pattern matching that semantic search can't provide.
+
+### When to use regex vs semantic search
+
+**Use regex when:**
+- Matching exact patterns like email addresses, URLs, or code identifiers
+- Finding documents containing specific formats (dates, phone numbers, IDs)
+- Filtering by prefixes or suffixes in structured data
+- You need deterministic, repeatable matches
+
+**Use semantic search when:**
+- Looking for conceptually similar content regardless of exact wording
+- The user's query is natural language
+- You want to find related content even if it uses different terminology
+
+You can combine both: use regex to narrow results, then rank by semantic similarity.
+
+### Imports and boilerplate
+
+```typescript
+import { DefaultEmbeddingFunction } from '@chroma-core/default-embed';
+import { ChromaClient } from 'chromadb';
+
+const client = new ChromaClient();
+const embedder = new DefaultEmbeddingFunction();
+
+const collection = await client.getOrCreateCollection({
+  name: 'example-collection',
+  embeddingFunction: embedder,
+});
+```
+
+### Basic Regex Filter
+
+Use the `$regex` operator in `where_document` to match document content against a regular expression. The regex follows standard regex syntax.
+
+```typescript
+await collection.query({
+  queryTexts: ['find support tickets with email addresses'],
+  whereDocument: {
+    $regex: '^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}$',
+  },
+});
+```
+
+### Combining regex with metadata filters
+
+Regex filters can be combined with metadata filters using `$and` and `$or` operators. This is powerful for narrowing results by both content patterns and structured metadata. Note however regex can not be used on metadata string values.
+
+```typescript
+await collection.query({
+  queryTexts: ['query1', 'query2'],
+  whereDocument: {
+    $and: [{ $contains: 'search_string_1' }, { $regex: '[a-z]+' }],
+  },
+});
+```
+
+### Performance considerations
+
+Regex filtering happens after the initial vector search retrieves candidates. For best performance:
+- Keep regex patterns simple when possible
+- Use metadata filters to reduce the candidate set before regex matching
+- Consider whether a metadata field with pre-extracted values would be faster than runtime regex

package/config/agent/skills/chroma/chroma-local/understanding-a-codebase.md

@@ -0,0 +1,80 @@
+---
+name: Integrating Chroma into an existing system
+description: Guidance for adding Chroma search to an existing application
+---
+
+## Integrating Chroma into an existing system
+
+Adding search to an existing application requires understanding the data flow and planning both the initial import and ongoing synchronization. This guide helps identify the key questions to answer.
+
+### Key questions to ask the user
+
+Before writing any code, clarify:
+
+1. **What data should be searchable?** (documents, products, messages, etc.)
+2. **Where does that data currently live?** (database, S3, API, files)
+3. **How is the data structured?** (helps determine chunking strategy)
+4. **How often does the data change?** (informs sync strategy)
+5. **What latency is acceptable for updates to appear in search?**
+
+## Understanding the source data
+
+Before designing the import pipeline, ask the user if you can look at a sample of the data that will be made searchable. Seeing real records is far more useful than a description of the schema.
+
+**If the data is in a database:** Write a short script that connects to the database and prints a few records. For example, a script that queries 3-5 rows from the relevant table and prints them to the terminal. This lets you see the actual field names, content lengths, and metadata available.
+
+**If the data is on disk:** Read a few of the files directly to understand their structure, format, and size. For example, if indexing markdown files, read 2-3 of them to see how they're organized.
+
+What to look for:
+- **Which field(s) contain the searchable text** — this becomes the document content in Chroma
+- **How long the content is** — determines whether chunking is needed
+- **What metadata is available** — fields like category, author, date, or tenant ID that could be useful for filtering
+- **How records are identified** — the primary key or filename that will link Chroma documents back to the source
+
+This step prevents guesswork and leads to better chunking and metadata design decisions.
+
+## Initial data import (offline ingest)
+
+The first step is getting existing data into Chroma. This typically involves:
+
+1. **Reading from the source** - database queries, S3 listing, API pagination
+2. **Chunking** - breaking large documents into searchable pieces (see data-model.md)
+3. **Embedding** - converting text chunks to vectors
+4. **Writing to Chroma** - batching for efficiency
+
+Build this as a reusable pipeline, not a one-off script. The same chunking and embedding logic will be needed for ongoing updates.
+
+**Progress tracking:** For large imports, track which records have been processed. This allows resuming after failures and re-running for updates. A simple approach is storing the last processed ID or timestamp.
+
+## Keeping data in sync (online writes)
+
+After the initial import, new and updated data must flow to Chroma. There are two main patterns:
+
+### Asynchronous (recommended)
+
+Use a message queue (SQS, RabbitMQ, Redis streams, etc.) to decouple the primary write path from Chroma updates:
+
+1. Application writes to primary database
+2. Application publishes an event with the record ID
+3. Queue consumer fetches the record, chunks, embeds, and writes to Chroma
+
+**Benefits:** Primary writes aren't slowed by embedding latency. Retries are handled by the queue. Search updates can lag slightly without affecting the main application.
+
+### Synchronous
+
+If no queue infrastructure exists and slight latency is acceptable, update Chroma in the same request:
+
+1. Application writes to primary database
+2. Application chunks, embeds, and writes to Chroma
+3. Request completes
+
+**Tradeoffs:** Simpler infrastructure but adds latency to every write. Failures in Chroma can affect the primary write path unless carefully handled.
+
+**Ask the user:** Do they have an async queue? If not, is synchronous acceptable, or should we set one up?
+
+## Handling updates and deletes
+
+- **Updates:** Re-chunk and re-embed the document, then use `upsert` to replace existing chunks
+- **Deletes:** Delete all chunks for the document by ID prefix or metadata filter
+
+Storing the source record ID in chunk metadata makes this straightforward. For example, if a blog post with ID `post-123` has 3 chunks, store `{"source_id": "post-123", "chunk_index": 0}` etc. on each chunk.

package/config/agent/skills/chroma/chroma-local/updating-deleting/python.md

@@ -0,0 +1,159 @@
+---
+name: Updating and Deleting
+description: Update existing documents and delete data from collections
+---
+
+## Updating and Deleting
+
+Chroma provides `update`, `upsert`, and `delete` methods for modifying data after initial insertion. Understanding when to use each is important for building reliable data sync pipelines.
+
+### Method overview
+
+| Method | Behavior | Use when |
+|--------|----------|----------|
+| `update` | Modifies existing documents, fails if ID doesn't exist | You know the document exists |
+| `upsert` | Updates if exists, inserts if not | Syncing from external data source |
+| `delete` | Removes documents by ID or filter | Removing stale or unwanted data |
+
+### Imports
+
+```python
+import time
+from typing import TypedDict
+
+import chromadb
+
+client = chromadb.HttpClient(host="localhost", port=8000)
+```
+
+## Update
+
+Update modifies existing documents. If an ID doesn't exist, the operation fails silently for that ID (no error thrown, but nothing is updated).
+
+**Important:** When you update a document's text, Chroma re-computes the embedding automatically using the collection's embedding function.
+
+```python
+collection = client.get_or_create_collection(name="my_collection")
+
+collection.add(
+    ids=["doc1", "doc2"],
+    documents=["Original text for doc1", "Original text for doc2"],
+    metadatas=[{"category": "draft"}, {"category": "draft"}],
+)
+
+collection.update(
+    ids=["doc1"],
+    documents=["Updated text for doc1"],
+)
+
+collection.update(
+    ids=["doc1", "doc2"],
+    metadatas=[{"category": "published"}, {"category": "published"}],
+)
+
+collection.update(
+    ids=["doc2"],
+    documents=["Completely revised doc2 content"],
+    metadatas=[{"category": "published", "revision": 2}],
+)
+```
+
+## Upsert
+
+Upsert is the preferred method for syncing data from an external source. It inserts new documents and updates existing ones in a single operation.
+
+**When to use upsert vs update:**
+- Use `upsert` when syncing from a primary database (you don't know which records are new)
+- Use `update` when you're certain the document already exists
+
+```python
+collection2 = client.get_or_create_collection(name="articles")
+
+collection2.upsert(
+    ids=["article-123", "article-456", "article-789"],
+    documents=[
+        "Content of article 123",
+        "Content of article 456",
+        "Content of article 789",
+    ],
+    metadatas=[
+        {"source_id": "123", "updated_at": int(time.time())},
+        {"source_id": "456", "updated_at": int(time.time())},
+        {"source_id": "789", "updated_at": int(time.time())},
+    ],
+)
+
+collection2.upsert(
+    ids=["article-123", "article-456"],
+    documents=[
+        "Updated content of article 123",
+        "Updated content of article 456",
+    ],
+    metadatas=[
+        {"source_id": "123", "updated_at": int(time.time())},
+        {"source_id": "456", "updated_at": int(time.time())},
+    ],
+)
+```
+
+## Delete by ID
+
+The simplest way to delete documents is by their IDs.
+
+```python
+awaiting_cleanup = ["article-789", "article-456"]
+collection2.delete(ids=awaiting_cleanup)
+```
+
+## Delete by filter
+
+Delete documents matching metadata or content filters without knowing specific IDs. Useful for bulk cleanup operations.
+
+```python
+collection2.delete(
+    where={"source_id": "123"},
+)
+```
+
+## Syncing from an external data source
+
+A common pattern is keeping Chroma in sync with a primary database. This example shows how to handle creates, updates, and deletes.
+
+```python
+class SourceRecord(TypedDict):
+    id: str
+    content: str
+    deleted: bool
+    updated_at: int
+
+
+def sync_records(records: list[SourceRecord]) -> None:
+    active_records = [record for record in records if not record["deleted"]]
+    deleted_ids = [record["id"] for record in records if record["deleted"]]
+
+    if active_records:
+        collection2.upsert(
+            ids=[record["id"] for record in active_records],
+            documents=[record["content"] for record in active_records],
+            metadatas=[
+                {
+                    "source_id": record["id"],
+                    "updated_at": record["updated_at"],
+                }
+                for record in active_records
+            ],
+        )
+
+    if deleted_ids:
+        collection2.delete(ids=deleted_ids)
+```
+
+### Sync strategy tips
+
+**Track source IDs:** Always store the primary database ID in metadata so you can find and update documents later.
+
+**Batch operations:** Process updates in batches of 100-500 to balance throughput and memory usage.
+
+**Handle deletes:** When records are deleted from your primary database, delete them from Chroma too. Use metadata filters if you track `source_id`.
+
+**Idempotent syncs:** Use `upsert` so re-running a sync doesn't create duplicates.

package/config/agent/skills/chroma/chroma-local/updating-deleting/typescript.md

@@ -0,0 +1,228 @@
+---
+name: Updating and Deleting
+description: Update existing documents and delete data from collections
+---
+
+## Updating and Deleting
+
+Chroma provides `update`, `upsert`, and `delete` methods for modifying data after initial insertion. Understanding when to use each is important for building reliable data sync pipelines.
+
+### Method overview
+
+| Method | Behavior | Use when |
+|--------|----------|----------|
+| `update` | Modifies existing documents, fails if ID doesn't exist | You know the document exists |
+| `upsert` | Updates if exists, inserts if not | Syncing from external data source |
+| `delete` | Removes documents by ID or filter | Removing stale or unwanted data |
+
+### Imports
+
+```typescript
+import { ChromaClient } from 'chromadb';
+import { DefaultEmbeddingFunction } from '@chroma-core/default-embed';
+
+const client = new ChromaClient();
+const embeddingFunction = new DefaultEmbeddingFunction();
+```
+
+## Update
+
+Update modifies existing documents. If an ID doesn't exist, the operation fails silently for that ID (no error thrown, but nothing is updated).
+
+**Important:** When you update a document's text, Chroma re-computes the embedding automatically using the collection's embedding function.
+
+```typescript
+const collection = await client.getOrCreateCollection({
+  name: 'my_collection',
+  embeddingFunction,
+});
+
+await collection.add({
+  ids: ['doc1', 'doc2'],
+  documents: ['Original text for doc1', 'Original text for doc2'],
+  metadatas: [{ category: 'draft' }, { category: 'draft' }],
+});
+
+await collection.update({
+  ids: ['doc1'],
+  documents: ['Updated text for doc1'],
+});
+
+await collection.update({
+  ids: ['doc1', 'doc2'],
+  metadatas: [{ category: 'published' }, { category: 'published' }],
+});
+
+await collection.update({
+  ids: ['doc2'],
+  documents: ['Completely revised doc2 content'],
+  metadatas: [{ category: 'published', revision: 2 }],
+});
+```
+
+## Upsert
+
+Upsert is the preferred method for syncing data from an external source. It inserts new documents and updates existing ones in a single operation.
+
+**When to use upsert vs update:**
+- Use `upsert` when syncing from a primary database (you don't know which records are new)
+- Use `update` when you're certain the document already exists
+
+```typescript
+const collection2 = await client.getOrCreateCollection({
+  name: 'articles',
+  embeddingFunction,
+});
+
+await collection2.upsert({
+  ids: ['article-123', 'article-456', 'article-789'],
+  documents: [
+    'Content of article 123',
+    'Content of article 456',
+    'Content of article 789',
+  ],
+  metadatas: [
+    { source_id: '123', updated_at: Date.now() },
+    { source_id: '456', updated_at: Date.now() },
+    { source_id: '789', updated_at: Date.now() },
+  ],
+});
+
+await collection2.upsert({
+  ids: ['article-123', 'article-456'],
+  documents: [
+    'Updated content of article 123',
+    'Updated content of article 456',
+  ],
+  metadatas: [
+    { source_id: '123', updated_at: Date.now() },
+    { source_id: '456', updated_at: Date.now() },
+  ],
+});
+```
+
+## Delete by ID
+
+The simplest way to delete documents is by their IDs.
+
+```typescript
+const collection3 = await client.getOrCreateCollection({
+  name: 'my_collection',
+  embeddingFunction,
+});
+
+await collection3.delete({
+  ids: ['doc1', 'doc2'],
+});
+
+await collection3.delete({
+  ids: ['doc3'],
+});
+```
+
+## Delete by filter
+
+Delete documents matching metadata or content filters without knowing specific IDs. Useful for bulk cleanup operations.
+
+```typescript
+const collection4 = await client.getOrCreateCollection({
+  name: 'my_collection',
+  embeddingFunction,
+});
+
+await collection4.delete({
+  where: { status: 'archived' },
+});
+
+await collection4.delete({
+  where: { source_id: 'old-source-123' },
+});
+
+await collection4.delete({
+  whereDocument: { $contains: 'DEPRECATED' },
+});
+
+await collection4.delete({
+  ids: ['doc1', 'doc2', 'doc3', 'doc4'],
+  where: { category: 'temp' },
+});
+```
+
+## Syncing from an external data source
+
+A common pattern is keeping Chroma in sync with a primary database. This example shows how to handle creates, updates, and deletes.
+
+```typescript
+interface SourceRecord {
+  id: string;
+  content: string;
+  updated_at: number;
+  category: string;
+}
+
+async function syncToChroma(
+  collectionName: string,
+  records: SourceRecord[],
+  deletedIds: string[]
+) {
+  const collection = await client.getOrCreateCollection({
+    name: collectionName,
+    embeddingFunction,
+  });
+
+  if (records.length > 0) {
+    const batchSize = 100;
+
+    for (let i = 0; i < records.length; i += batchSize) {
+      const batch = records.slice(i, i + batchSize);
+
+      await collection.upsert({
+        ids: batch.map((record) => `source-${record.id}`),
+        documents: batch.map((record) => record.content),
+        metadatas: batch.map((record) => ({
+          source_id: record.id,
+          updated_at: record.updated_at,
+          category: record.category,
+        })),
+      });
+    }
+  }
+
+  if (deletedIds.length > 0) {
+    await collection.delete({
+      ids: deletedIds.map((id) => `source-${id}`),
+    });
+  }
+
+  return { synced: records.length, deleted: deletedIds.length };
+}
+
+const changedRecords: SourceRecord[] = [
+  {
+    id: '1',
+    content: 'Article about TypeScript',
+    updated_at: Date.now(),
+    category: 'tech',
+  },
+  {
+    id: '2',
+    content: 'Guide to vector databases',
+    updated_at: Date.now(),
+    category: 'tech',
+  },
+];
+
+const deletedRecordIds = ['old-1', 'old-2'];
+
+await syncToChroma('articles', changedRecords, deletedRecordIds);
+```
+
+### Sync strategy tips
+
+**Track source IDs:** Always store the primary database ID in metadata so you can find and update documents later.
+
+**Batch operations:** Process updates in batches of 100-500 to balance throughput and memory usage.
+
+**Handle deletes:** When records are deleted from your primary database, delete them from Chroma too. Use metadata filters if you track `source_id`.
+
+**Idempotent syncs:** Use `upsert` so re-running a sync doesn't create duplicates.