zilmate 1.8.0 → 1.8.5

package/.agents/skills/upstash-search-js/SKILL.md

@@ -0,0 +1,53 @@
+---
+name: upstash-search-js
+description: Skills covering Upstash Search quick starts, core concepts, and TypeScript/JavaScript SDK usage. Use when a user asks how to get started, how indexing works, or how to use the TS client.
+---
+
+# Upstash Search Documentation
+
+## Quick Start
+
+Install the TS SDK:
+
+```
+npm install @upstash/search
+```
+
+Create a client and perform a simple upsert + search:
+
+```
+import { Search } from "@upstash/search";
+
+const client = new Search({ url: process.env.UPSTASH_SEARCH_REST_URL, token: process.env.UPSTASH_SEARCH_REST_TOKEN });
+const index = client.index("my-index");
+
+await index.upsert({ id: "1", content: { text: "hello world" } });
+const results = await index.search({ query: "hello" });
+```
+
+Basic steps:
+- Create an index
+- Insert or update documents
+- Run searches or filtered queries
+
+## Other Skill Files
+
+### sdk-overview
+Provides detailed documentation for all TypeScript SDK commands. Includes:
+- delete: Deleting documents
+- fetch: Retrieving a document
+- info: Index info
+- range: Range queries
+- reset: Clearing an index
+- search: Search queries
+- upsert: Adding/updating documents
+- getting-started: Setup steps for the SDK
+
+### quick-start
+Provides a fast, end-to-end workflow for creating a Search database, adding documents, and querying them. Covers essential concepts including:
+- Creating a database and storing credentials
+- Adding documents with content and metadata
+- Understanding content vs metadata (searchability and filterability)
+- Performing searches with optional reranking
+- Filtering syntax with SQL-like or structured filters
+- Common pitfalls and best practices

package/.agents/skills/upstash-search-js/quick-start.md

@@ -0,0 +1,158 @@
+# Quick Start: Upstash Search
+
+This Skill gives agents a fast, end‑to‑end workflow for creating a Search database, adding documents, and querying them. It also summarizes key concepts like **content vs metadata**, **filters**, and **reranking** so agents can use Search correctly.
+
+---
+
+## Create a Database
+1. Open the **Vector** tab → **Create → Search Database**.
+2. Provide a name (e.g., `product-search`) and region.
+3. Select a plan.
+
+Agents should store:
+- `UPSTASH_SEARCH_REST_URL`
+- `UPSTASH_SEARCH_REST_TOKEN`
+
+These values are required when constructing a Search client.
+
+---
+
+## Add Documents
+Documents consist of:
+- **id**: unique identifier
+- **content** *(required)*: indexed and searchable
+- **metadata** *(optional)*: **not searchable**, but retrievable and filterable
+
+### TypeScript / Python Example
+```ts
+import { Search } from "@upstash/search";
+const client = new Search({ 
+  url: process.env.UPSTASH_SEARCH_REST_URL,
+  token: process.env.UPSTASH_SEARCH_REST_TOKEN 
+});
+const index = client.index("movies");
+await index.upsert([
+  {
+    id: "star-wars",
+    content: { title: "Star Wars", genre: "sci-fi", category: "classic" },
+    metadata: { director: "George Lucas" }
+  }
+]);
+```
+```python
+from upstash_search import Search
+client = Search(url=URL, token=TOKEN)
+index = client.index("movies")
+index.upsert(documents=[{
+  "id": "movie-0",
+  "content": {
+    "title": "Star Wars",
+    "overview": "Sci-fi space opera",
+    "genre": "sci-fi",
+    "category": "classic",
+  },
+  "metadata": {"poster": "https://poster.link/starwars.jpg"}
+}])
+```
+
+---
+
+## Content vs Metadata (Essential Concepts)
+- **Content**
+  - Required
+  - Indexed and searchable
+  - Can be used in filters
+  - Ideal for textual and semantic data
+
+- **Metadata**
+  - Optional
+  - Not indexed → **cannot be searched**
+  - Still **filterable** using `@metadata.key`
+  - Used for contextual / reference fields
+
+Example:
+```json
+{
+  "content": { "title": "Star Wars", "genre": "sci-fi" },
+  "metadata": { "director": "George Lucas", "sku": "SW-001" }
+}
+```
+
+---
+
+## Search
+Searching supports semantic + keyword hybrid search, optional reranking, and filters.
+
+### TypeScript / Python Example
+```ts
+const res = await index.search({ query: "space opera", limit: 2, reranking: true });
+```
+```python
+scores = index.search(query="space opera", limit=2, reranking=True)
+```
+
+---
+
+## Filtering
+Filters restrict results using SQL‑like syntax or structured filters (TypeScript only). Both **content fields** and **metadata fields** can be used.
+
+**Metadata fields require `@metadata.` prefix**.
+
+### Example (String Filters)
+```ts
+await index.search({
+  query: "sony headphones",
+  filter: "warehouse_location = 'A3-15' AND @metadata.supplier_id = 'SUP-123'"
+});
+```
+
+### Example (Type‑safe Filters, TS SDK)
+```ts
+await index.search({
+  query: "sony headphones",
+  filter: {
+    AND: [
+      { category: { equals: "Electronics" } },
+      { "@metadata.count": { greaterThanOrEquals: 3 } }
+    ]
+  }
+});
+```
+
+Common operators:
+- equals, not equals
+- <, <=, >, >=
+- glob / not glob
+- in / not in
+- contains / not contains (arrays)
+- has field / has not field
+
+---
+
+## Reranking
+Reranking reorders results using a high‑accuracy model.
+
+- Disabled by default (`false`)
+- When `true`, improves relevance but costs $1 per 1K reranked items
+
+Example:
+```ts
+await index.search({ query: "space opera", reranking: true });
+```
+```python
+index.search(query="space opera", reranking=True)
+```
+
+Use when:
+- Precision is critical
+- Results require more semantic depth
+- Queries are ambiguous or conceptual
+
+---
+
+## Common Pitfalls
+- Missing **content** field → upsert fails.
+- Metadata fields are **not searchable**.
+- Metadata must be prefixed as `@metadata.key` in filters.
+- Filters may return fewer than `topK` results if too selective.
+- Indexes are created automatically on first `upsert`.

package/.agents/skills/upstash-search-js/sdk-overview.md

@@ -0,0 +1,158 @@
+# SDK Overview
+
+This skill provides a concise but complete reference for using the Upstash Search TypeScript SDK. It focuses on practical usage patterns, common pitfalls, and efficient examples that combine multiple commands. Use this skill whenever interacting with the Upstash Search SDK, generating agents that must query, mutate, or paginate search indexes.
+
+---
+
+## Client Initialization
+
+You must configure a Search client using either environment variables or a config object.
+
+```ts
+import { Search } from "@upstash/search";
+
+// Option 1: with explicit config
+const client = new Search({ 
+  url: process.env.UPSTASH_SEARCH_REST_URL!,
+  token: process.env.UPSTASH_SEARCH_REST_TOKEN! 
+});
+const index = client.index("movies");
+
+// Option 2: using fromEnv (Node.js platform only)
+// The constructor will automatically read from process.env if url/token not provided
+const client2 = new Search({}); // reads UPSTASH_SEARCH_REST_URL and UPSTASH_SEARCH_REST_TOKEN
+const index2 = client2.index("movies");
+```
+
+Type-safe usage:
+```ts
+type Content = { title: string, genre: string };
+type Metadata = { year: number };
+const indexTyped = client.index<Content, Metadata>("movies");
+```
+
+---
+
+## Upsert (add or update documents)
+
+Pitfalls:
+- Document structure must match the index schema.
+- Content/metadata types are enforced when using generics.
+
+```ts
+// Single
+await index.upsert({
+  id: "star-wars",
+  content: { title: "Star Wars", genre: "sci-fi" },
+  metadata: { year: 1977 }
+});
+
+// Multiple
+await index.upsert([
+  { id: "inception", content: { title: "Inception", genre: "action" }, metadata: { year: 2010 } },
+  { id: "matrix", content: { title: "The Matrix", genre: "sci-fi" }, metadata: { year: 1999 } },
+]);
+
+// Update
+await index.upsert({ id: "star-wars", content: { title: "A New Hope" } });
+```
+
+---
+
+## Fetch (retrieve documents)
+
+Pitfalls:
+- Returns null for IDs not found.
+- Supports prefix matching.
+
+```ts
+// By IDs
+const docs = await index.fetch({ ids: ["star-wars", "inception"] });
+
+// By prefix
+const sciFi = await index.fetch({ prefix: "star-" });
+```
+
+---
+
+## Delete (IDs, prefix, or filter)
+
+Pitfalls:
+- **Filter deletion is O(N)** and slow on large indexes.
+- Prefix deletion removes *all* matching documents.
+
+```ts
+// ID list
+await index.delete(["star-wars", "inception"]);
+
+// Single ID
+await index.delete("star-wars");
+
+// Prefix
+await index.delete({ prefix: "star-" });
+
+// Filter — expensive
+await index.delete({ filter: "age > 30" });
+```
+
+---
+
+## Search (AI‑powered)
+
+Pitfalls:
+- Default `limit = 5`.
+- Scores are 0–1.
+- Use filters to restrict by document fields.
+
+```ts
+// Basic
+const results = await index.search({ query: "space opera", limit: 3 });
+
+// With reranking
+await index.search({ query: "space opera", limit: 3, reranking: true });
+
+// With filter
+await index.search({ query: "space", filter: "category = 'classic'" });
+
+// Adjust semantic vs keyword weighting
+await index.search({ query: "robots", semanticWeight: 0.2 });
+```
+
+---
+
+## Range (cursor pagination)
+
+Pitfalls:
+- Stateless: you must pass all parameters every call.
+- `cursor = "0"` for the first request.
+
+```ts
+let cursor = "0";
+while (cursor !== "") {
+  const res = await index.range({ cursor, limit: 2, prefix: "test-" });
+  cursor = res.nextCursor;
+  console.log(res.documents);
+}
+```
+
+---
+
+## Reset (delete all documents)
+
+```ts
+await index.reset(); // "Success"
+```
+
+---
+
+## Info (index or database level)
+
+```ts
+// Index-level
+const indexInfo = await index.info();
+// { documentCount, pendingDocumentCount }
+
+// Database-level
+const dbInfo = await client.info();
+// { documentCount, pendingDocumentCount, diskSize, indexes: {...} }
+```

package/.agents/skills/upstash-vector-js/SKILL.md

@@ -0,0 +1,49 @@
+---
+name: upstash-vector-js
+description: Skills for Upstash Vector features, TypeScript/JavaScript SDK usage, and integrations. Use when users ask how to work with Vector, its TS SDK, features, or supported frameworks.
+---
+
+# Vector Documentation Skill
+
+## Quick Start
+
+Vector is a high‑performance vector database for storing, querying, and managing vector embeddings.
+
+Basic workflow:
+
+- Install the Vector TS SDK.
+- Connect to a Vector instance.
+- Upsert vectors, query them, and manage namespaces.
+
+Example (TypeScript):
+
+```ts
+import { Index } from "@upstash/vector";
+const index = new Index({
+  url: process.env.UPSTASH_VECTOR_REST_URL!,
+  token: process.env.UPSTASH_VECTOR_REST_TOKEN!,
+});
+
+await index.upsert([{ id: "1", vector: [0.1, 0.2], metadata: { tag: "example" } }]);
+
+const results = await index.query({
+  vector: [0.1, 0.2],
+  topK: 5,
+});
+```
+
+For full usage, refer to the linked skill files below.
+
+## Other Skill Files
+
+### TS SDK Reference
+
+- `sdk-methods`: Explains SDK commands: delete, fetch, info, query, range, reset, resumable-query, upsert
+
+### Features
+
+- `features/namespaces`: Explains namespaces and dataset organization.
+- `features/index-structure`: Covers hybrid and sparse index structures.
+- `features/filtering-and-metadata`: Details metadata storage and server-side filtering.
+
+Use these files for deeper guidance on SDK usage, advanced configurations, algorithms, and integrations.

package/.agents/skills/upstash-vector-js/features/filtering-and-metadata.md

@@ -0,0 +1,82 @@
+# Vector Feature: Filtering and Metadata
+
+Upstash Vector supports attaching metadata and optional data to vectors. Metadata is structured JSON used for filtering; data is unstructured content returned in responses but not filterable.
+
+Filtering uses a SQL‑like syntax and supports nested objects, array indexing, glob patterns, and boolean logic. The system applies both in‑filtering and post‑filtering according to a filtering budget, so highly selective filters may return fewer results than `topK`.
+
+## Setting Metadata and Data
+
+You can upsert vectors with metadata and optional data. Metadata is any JSON structure; data is typically raw text.
+
+```ts
+import { Index } from "@upstash/vector";
+
+const index = new Index({ url: "...", token: "..." });
+
+await index.upsert([
+  {
+    id: "v0",
+    vector: [0.1, 0.2],
+    metadata: { city: "Istanbul", population: 15460000 },
+    data: "Istanbul info",
+  },
+  {
+    id: "v1",
+    data: "Upstash is a serverless data platform.",
+  },
+]);
+```
+
+## Querying with Metadata Filters
+
+Include metadata in results and apply filters using SQL‑like expressions.
+
+```ts
+await index.query({
+  vector: [0.9, 0.3],
+  topK: 5,
+  includeMetadata: true,
+  filter: "population >= 1000000 AND geography.continent = 'Asia'",
+});
+```
+
+### Supported Operators
+
+- Equality: `=`, `!=`
+- Numeric comparators: `<`, `<=`, `>`, `>=`
+- Set membership: `IN`, `NOT IN`
+- Array tests: `CONTAINS`, `NOT CONTAINS`
+- Field existence: `HAS FIELD`, `HAS NOT FIELD`
+- Glob string matching: `GLOB`, `NOT GLOB`
+
+Glob wildcards: `*`, `?`, `[]`, `[^]`.
+
+### Boolean Logic
+
+Use `AND` and `OR`, with parentheses for grouping. `AND` has higher precedence than `OR`.
+
+### Nested Objects and Arrays
+
+- Access nested fields: `economy.currency`, `geography.coordinates.latitude`.
+- Index arrays: `industries[0]` or from end: `industries[#-1]`.
+
+Example:
+
+```SQL
+economy.major_industries CONTAINS 'Tourism' AND geography.coordinates.latitude >= 35
+```
+
+## Retrieving Metadata and Data
+
+```ts
+// query
+await index.query({
+  vector: [0.9, 0.3],
+  topK: 5,
+  includeMetadata: true,
+  includeData: true,
+});
+
+// range
+await index.range({ cursor: "0", limit: 3, includeMetadata: true });
+```

package/.agents/skills/upstash-vector-js/features/index-structure.md

@@ -0,0 +1,193 @@
+# Vector Feature: Index Structure
+
+## Overview
+
+Upstash Vector supports three major index structures:
+
+- **Dense indexes**: semantic matching via dense embeddings.
+- **Sparse indexes**: exact/near-exact token and feature matching.
+- **Hybrid indexes**: combine dense + sparse results, optionally reranked.
+
+---
+
+## Key Concepts
+
+### Sparse Vectors
+
+- High‑dimensional, mostly‑zero representations.
+- Represented as two equal‑length arrays:
+  - `indices`: int32 positions of non‑zero elements
+  - `values`: float32 values
+- Upstash limit: **max 1,000 non‑zero entries per sparse vector**.
+- Useful for exact token/word matching (BM25, SPLADE, etc.).
+
+### Hybrid Vectors
+
+- Combine dense semantic vectors with sparse exact‑match vectors.
+- Hybrid queries run both dense + sparse search and fuse results.
+- Require **both** components (`vector` and `sparseVector`).
+
+### Fusion Algorithms
+
+- **RRF (default)**: ranking‑based, simple, robust, ignores score magnitudes.
+- **DBSF**: normalizes scores using distribution statistics; more sensitive to score ranges.
+
+---
+
+## Common Pitfalls
+
+- **Hybrid upserts require both dense and sparse vectors**; omitting either fails.
+- **Indexes with embedding models (dense, sparse or both)** let you upsert/query using text; indexes without embedding models do not.
+- In Hybrin indexes, dense‑only or sparse‑only querying is allowed, but **fusion happens only when both are provided**.
+
+---
+
+## Usage with Embedding Models
+
+```ts
+await index.upsert([
+  {
+    id: "t1",
+    data: "Upstash Vector provides sparse models.",
+  },
+]);
+
+const results = await index.query({
+  data: "Upstash Vector",
+  topK: 5,
+});
+
+import { WeightingStrategy } from "@upstash/vector";
+await index.query({
+  data: "Upstash Vector",
+  weightingStrategy: WeightingStrategy.IDF,
+});
+```
+
+## Sparse Index Usage
+
+### Upserting Sparse Vectors
+
+```ts
+await index.upsert([
+  {
+    id: "x1",
+    sparseVector: {
+      indices: [1, 2, 3],
+      values: [0.1, 0.2, 0.3],
+    },
+  },
+]);
+
+const results = await index.query({
+  sparseVector: {
+    indices: [3, 5],
+    values: [0.3, 0.5],
+  },
+  topK: 5,
+  includeMetadata: true,
+});
+```
+
+- Scores use **inner product**, matching only overlapping indices.
+- Results may be fewer than `top_k` if no overlapping dims exist.
+
+## Hybrid Index Usage
+
+### Upserting Dense + Sparse
+
+```ts
+await index.upsert([
+  {
+    id: "h1",
+    vector: [0.1, 0.5],
+    sparseVector: {
+      indices: [1, 2],
+      values: [0.1, 0.2],
+    },
+  },
+]);
+
+const results = await index.query({
+  vector: [0.5, 0.4],
+  sparseVector: {
+    indices: [3, 5],
+    values: [0.3, 0.5],
+  },
+  topK: 5,
+});
+
+import { FusionAlgorithm } from "@upstash/vector";
+await index.query({
+  vector: [0.5, 0.4],
+  sparseVector: {
+    indices: [2, 3],
+    values: [0.1, 0.2],
+  },
+  fusionAlgorithm: FusionAlgorithm.RRF, // or FusionAlgorithm.DBSF
+});
+```
+
+---
+
+## Custom Reranking
+
+Sometimes RRF/DBSF is insufficient (e.g., using bge‑reranker-v2-m3). Query dense and sparse portions separately and rerank in your own model.
+
+### Custom Rerank (vector input)
+
+```ts
+// Dense-only
+const dense = await index.query({
+  vector: [0.5, 0.4],
+  topK: 5,
+});
+
+// Sparse-only
+const sparse = await index.query({
+  sparseVector: {
+    indices: [3, 5],
+    values: [0.3, 0.5],
+  },
+  topK: 5,
+});
+
+// Custom rerank dense + sparse...
+```
+
+### Custom Rerank (text input with hosted models)
+
+```ts
+import { QueryMode } from "@upstash/vector";
+
+const dense = await index.query({
+  data: "Upstash Vector",
+  queryMode: QueryMode.DENSE,
+});
+
+const sparse = await index.query({
+  data: "Upstash Vector",
+  queryMode: QueryMode.SPARSE,
+});
+
+// Rerank...
+```
+
+---
+
+## Agent Implementation Notes
+
+- Always check whether the index supports **hosted embeddings** before using `data=` fields.
+  - Use `await index.info()` to check: if `denseIndex?.embeddingModel` or `sparseIndex?.embeddingModel` exists, the index supports text-based embeddings via the `data` field.
+  - Example:
+    ```ts
+    const info = await index.info();
+    const hasDenseEmbedding = !!info.denseIndex?.embeddingModel;
+    const hasSparseEmbedding = !!info.sparseIndex?.embeddingModel;
+    ```
+- For hybrid indexes:
+  - Always provide _both_ `vector` and `sparseVector` unless intentionally querying only one modality.
+- For reranking workflows:
+  - Use dense-only + sparse-only queries, never hybrid queries (as fusion is already applied).
+- When building sparse vectors manually:
+  - Ensure indices are sorted, unique, and below the model dimension.