cfsa-antigravity 1.0.0

package/template/.agent/skill-library/stack/search/elasticsearch/SKILL.md

@@ -0,0 +1,248 @@
+---
+name: elasticsearch
+description: "Expert Elasticsearch guide covering index design (mappings, analyzers, sharding), query DSL (full-text, filters, aggregations, fuzzy), indexing patterns (bulk API, ingest pipelines), performance (query profiling, caching, segment management), and operational best practices. Use when building search features, log analytics, or autocomplete with Elasticsearch or OpenSearch."
+version: 1.0.0
+---
+
+# Elasticsearch Expert Guide
+
+> Use this skill when building full-text search, autocomplete, log analytics, or faceted navigation with Elasticsearch (8.x) or OpenSearch.
+
+## When to Use This Skill
+
+- Building search features (product search, site search, document search)
+- Implementing autocomplete / search-as-you-type
+- Log analytics and aggregation dashboards
+- Faceted navigation with filters and counts
+- Geo-spatial search
+
+## When NOT to Use This Skill
+
+- Primary database → use PostgreSQL or MongoDB
+- Simple key-value lookups → use Redis
+- Vector similarity search → use a vector DB (Qdrant, Pinecone)
+- Small dataset (<100K docs) with basic search → PostgreSQL full-text may suffice
+
+---
+
+## 1. Index Design (CRITICAL)
+
+### Mapping
+
+```json
+{
+  "mappings": {
+    "properties": {
+      "title": {
+        "type": "text",
+        "analyzer": "english",
+        "fields": {
+          "keyword": { "type": "keyword" },
+          "autocomplete": {
+            "type": "text",
+            "analyzer": "autocomplete_analyzer"
+          }
+        }
+      },
+      "description": { "type": "text", "analyzer": "english" },
+      "price": { "type": "scaled_float", "scaling_factor": 100 },
+      "category": { "type": "keyword" },
+      "tags": { "type": "keyword" },
+      "created_at": { "type": "date" },
+      "location": { "type": "geo_point" },
+      "in_stock": { "type": "boolean" }
+    }
+  }
+}
+```
+
+### Field Type Selection
+
+| Data | Type | When |
+|------|------|------|
+| Full-text search | `text` | Search with relevance scoring |
+| Exact match / filter / sort | `keyword` | Enums, IDs, tags, status |
+| Both search + filter | Multi-field (`text` + `.keyword`) | Title, name |
+| Numbers | `integer`, `long`, `scaled_float` | Prices, counts |
+| Dates | `date` | Timestamps, created_at |
+| Booleans | `boolean` | Flags |
+| Geo | `geo_point` | Coordinates |
+
+### Analyzers
+
+```json
+{
+  "settings": {
+    "analysis": {
+      "analyzer": {
+        "autocomplete_analyzer": {
+          "type": "custom",
+          "tokenizer": "autocomplete_tokenizer",
+          "filter": ["lowercase"]
+        }
+      },
+      "tokenizer": {
+        "autocomplete_tokenizer": {
+          "type": "edge_ngram",
+          "min_gram": 2,
+          "max_gram": 20,
+          "token_chars": ["letter", "digit"]
+        }
+      }
+    }
+  }
+}
+```
+
+### Sharding Strategy
+
+| Shard size | Rule |
+|-----------|------|
+| Target | 10–50 GB per shard |
+| Max | Never exceed 65 GB |
+| Small indices (<10GB) | 1 primary shard |
+| Time-series | ILM with rollover |
+
+---
+
+## 2. Query DSL (CRITICAL)
+
+### Full-Text Search
+
+```json
+{
+  "query": {
+    "bool": {
+      "must": [
+        {
+          "multi_match": {
+            "query": "wireless headphones",
+            "fields": ["title^3", "description", "tags^2"],
+            "type": "best_fields",
+            "fuzziness": "AUTO"
+          }
+        }
+      ],
+      "filter": [
+        { "term": { "category": "electronics" } },
+        { "range": { "price": { "gte": 20, "lte": 200 } } },
+        { "term": { "in_stock": true } }
+      ],
+      "should": [
+        { "term": { "featured": { "value": true, "boost": 5 } } }
+      ]
+    }
+  },
+  "highlight": {
+    "fields": { "title": {}, "description": {} }
+  },
+  "sort": [
+    "_score",
+    { "created_at": "desc" }
+  ],
+  "from": 0,
+  "size": 20
+}
+```
+
+> **Rule**: Use `filter` for exact matches (no scoring, cached), `must` for relevance-scored text.
+
+### Autocomplete
+
+```json
+{
+  "query": {
+    "match": {
+      "title.autocomplete": {
+        "query": "wire",
+        "analyzer": "standard"
+      }
+    }
+  },
+  "_source": ["title", "category"],
+  "size": 5
+}
+```
+
+### Aggregations
+
+```json
+{
+  "size": 0,
+  "aggs": {
+    "by_category": {
+      "terms": { "field": "category", "size": 20 },
+      "aggs": {
+        "avg_price": { "avg": { "field": "price" } },
+        "price_ranges": {
+          "range": {
+            "field": "price",
+            "ranges": [
+              { "to": 50 },
+              { "from": 50, "to": 100 },
+              { "from": 100 }
+            ]
+          }
+        }
+      }
+    }
+  }
+}
+```
+
+---
+
+## 3. Indexing Patterns
+
+### Bulk API
+
+```json
+POST _bulk
+{"index": {"_index": "products", "_id": "1"}}
+{"title": "Wireless Headphones", "price": 79.99, "category": "electronics"}
+{"index": {"_index": "products", "_id": "2"}}
+{"title": "Running Shoes", "price": 129.99, "category": "sports"}
+```
+
+| Rule | Detail |
+|------|--------|
+| Batch size | 5–15 MB per bulk request |
+| Refresh interval | Set `refresh_interval: 30s` during bulk indexing |
+| Replicas | Set `number_of_replicas: 0` during initial load, restore after |
+
+### Index Lifecycle Management (ILM)
+
+```json
+{
+  "policy": {
+    "phases": {
+      "hot":   { "actions": { "rollover": { "max_size": "50gb", "max_age": "7d" } } },
+      "warm":  { "min_age": "30d", "actions": { "shrink": { "number_of_shards": 1 }, "forcemerge": { "max_num_segments": 1 } } },
+      "cold":  { "min_age": "90d", "actions": { "searchable_snapshot": { "snapshot_repository": "backups" } } },
+      "delete": { "min_age": "365d", "actions": { "delete": {} } }
+    }
+  }
+}
+```
+
+---
+
+## 4. Common Anti-Patterns
+
+1. **Using ES as a primary database** — it's a search engine, not ACID storage
+2. **Not setting explicit mappings** — dynamic mapping guesses wrong types
+3. **Keyword fields for full-text search** — use `text` with analyzers
+4. **Querying without filters** — filters are cached and avoid scoring overhead
+5. **Too many shards** — small indices need 1 shard; over-sharding wastes resources
+6. **Deep pagination with `from`+`size`** — use `search_after` for >10K results
+7. **Not using bulk API** — single-doc indexing is orders of magnitude slower
+8. **Wildcard queries on large fields** — extremely expensive; use `edge_ngram` instead
+
+---
+
+## References
+
+- [Elasticsearch Guide](https://www.elastic.co/guide/en/elasticsearch/reference/current/)
+- [Query DSL](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl.html)
+- [Mapping Types](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-types.html)
+- [Index Lifecycle Management](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-lifecycle-management.html)

package/template/.agent/skill-library/stack/search/meilisearch/SKILL.md

@@ -0,0 +1,385 @@
+---
+name: meilisearch
+description: Integrate Meilisearch for fast, typo-tolerant full-text search including index management, filtering, faceting, geo search, and relevancy tuning. Use when adding search functionality to applications that need instant, typo-tolerant results.
+version: 1.0.0
+---
+
+# Meilisearch
+
+Integrate Meilisearch for fast, typo-tolerant full-text search. Meilisearch is an open-source search engine designed for instant search experiences with minimal configuration.
+
+## When to Use This Skill
+
+- Adding search functionality to web applications
+- Implementing faceted search (filter by category, price range, etc.)
+- Building autocomplete / search-as-you-type experiences
+- Searching content that needs typo tolerance and relevancy ranking
+- Implementing geo-based search (find nearby locations)
+- Multi-tenant search with secure tenant tokens
+
+## Setup
+
+```bash
+# Install JavaScript client
+pnpm add meilisearch
+
+# Run Meilisearch locally (Docker)
+docker run -d -p 7700:7700 \
+  -v $(pwd)/meili_data:/meili_data \
+  -e MEILI_MASTER_KEY='your-master-key' \
+  getmeili/meilisearch:latest
+```
+
+### Client Initialization
+
+```typescript
+// src/lib/search/meilisearch.ts
+import { MeiliSearch } from 'meilisearch';
+
+export const meili = new MeiliSearch({
+  host: process.env.MEILISEARCH_HOST ?? 'http://localhost:7700',
+  apiKey: process.env.MEILISEARCH_ADMIN_KEY, // Admin key for server-side operations
+});
+
+// For client-side search, use a search-only API key
+export const meiliSearch = new MeiliSearch({
+  host: process.env.MEILISEARCH_HOST ?? 'http://localhost:7700',
+  apiKey: process.env.MEILISEARCH_SEARCH_KEY, // Read-only key
+});
+```
+
+## Index Management
+
+### Creating and Configuring an Index
+
+```typescript
+// Create index with primary key
+const index = await meili.createIndex('products', { primaryKey: 'id' });
+
+// Configure searchable attributes (order defines relevancy weight)
+await meili.index('products').updateSearchableAttributes([
+  'name',        // Highest weight
+  'description',
+  'category',
+  'brand',       // Lowest weight
+]);
+
+// Configure filterable attributes (for filtering and faceting)
+await meili.index('products').updateFilterableAttributes([
+  'category',
+  'brand',
+  'price',
+  'inStock',
+  'rating',
+  'tags',
+  '_geo', // Required for geo search
+]);
+
+// Configure sortable attributes
+await meili.index('products').updateSortableAttributes([
+  'price',
+  'rating',
+  'createdAt',
+]);
+
+// Configure displayed attributes (what is returned in results)
+await meili.index('products').updateDisplayedAttributes([
+  'id', 'name', 'description', 'price', 'category',
+  'brand', 'imageUrl', 'rating', 'slug',
+]);
+
+// Configure ranking rules (order matters)
+await meili.index('products').updateRankingRules([
+  'words',        // Number of matched query words
+  'typo',         // Number of typos
+  'proximity',    // Distance between matched words
+  'attribute',    // Position in searchableAttributes list
+  'sort',         // User-defined sort
+  'exactness',    // Exact vs prefix match
+  'rating:desc',  // Custom ranking: higher rating first
+]);
+```
+
+### Typo Tolerance
+
+```typescript
+await meili.index('products').updateTypoTolerance({
+  enabled: true,
+  minWordSizeForTypos: {
+    oneTypo: 4,   // Allow 1 typo for words >= 4 chars
+    twoTypos: 8,  // Allow 2 typos for words >= 8 chars
+  },
+  disableOnAttributes: ['sku', 'barcode'], // Exact match only for these
+  disableOnWords: ['iPhone', 'MacBook'],   // Exact match for specific terms
+});
+```
+
+### Synonyms
+
+```typescript
+await meili.index('products').updateSynonyms({
+  'phone': ['smartphone', 'mobile', 'cellphone'],
+  'laptop': ['notebook', 'portable computer'],
+  'tv': ['television', 'monitor', 'screen'],
+  // One-way synonym
+  'iphone': ['phone'],
+});
+```
+
+### Stop Words
+
+```typescript
+await meili.index('products').updateStopWords([
+  'the', 'a', 'an', 'is', 'are', 'was', 'were', 'of', 'for', 'in',
+]);
+```
+
+## Document Operations
+
+### Adding Documents
+
+```typescript
+const products = [
+  {
+    id: 'prod_1',
+    name: 'Wireless Noise-Canceling Headphones',
+    description: 'Premium wireless headphones with active noise cancellation',
+    category: 'Electronics',
+    brand: 'AudioTech',
+    price: 299.99,
+    rating: 4.7,
+    inStock: true,
+    tags: ['wireless', 'noise-canceling', 'bluetooth'],
+    slug: 'wireless-noise-canceling-headphones',
+    _geo: { lat: 40.7128, lng: -74.0060 },
+  },
+  // ... more products
+];
+
+// Add or replace documents
+const task = await meili.index('products').addDocuments(products);
+
+// Wait for indexing to complete
+await meili.waitForTask(task.taskUid);
+
+// Update specific documents (partial update)
+await meili.index('products').updateDocuments([
+  { id: 'prod_1', price: 249.99, inStock: false },
+]);
+
+// Delete documents
+await meili.index('products').deleteDocument('prod_1');
+await meili.index('products').deleteDocuments(['prod_1', 'prod_2']);
+await meili.index('products').deleteAllDocuments();
+```
+
+### Batch Indexing
+
+```typescript
+// For large datasets, batch in chunks
+async function indexAllProducts(products: Product[]) {
+  const BATCH_SIZE = 1000;
+  const tasks: number[] = [];
+
+  for (let i = 0; i < products.length; i += BATCH_SIZE) {
+    const batch = products.slice(i, i + BATCH_SIZE);
+    const task = await meili.index('products').addDocuments(batch);
+    tasks.push(task.taskUid);
+  }
+
+  // Wait for all batches to finish
+  for (const taskUid of tasks) {
+    await meili.waitForTask(taskUid);
+  }
+}
+```
+
+## Searching
+
+### Basic Search
+
+```typescript
+const results = await meili.index('products').search('wireless headphones');
+
+// results shape:
+// {
+//   hits: [...],           // Matching documents
+//   query: 'wireless headphones',
+//   processingTimeMs: 2,
+//   limit: 20,
+//   offset: 0,
+//   estimatedTotalHits: 42,
+// }
+```
+
+### Search with Filters
+
+```typescript
+const results = await meili.index('products').search('headphones', {
+  filter: [
+    'category = "Electronics"',
+    'price >= 50 AND price <= 300',
+    'inStock = true',
+    'rating >= 4.0',
+  ],
+  sort: ['price:asc'],
+  limit: 20,
+  offset: 0,
+});
+
+// Array of filter strings are ANDed together
+// Use OR within a string: 'category = "Electronics" OR category = "Audio"'
+// Use arrays for AND between groups, strings for OR within groups:
+// filter: [['category = "Electronics"', 'category = "Audio"'], 'inStock = true']
+// = (Electronics OR Audio) AND inStock
+```
+
+### Search with Facets
+
+```typescript
+const results = await meili.index('products').search('headphones', {
+  facets: ['category', 'brand', 'tags'],
+  filter: ['inStock = true'],
+});
+
+// results.facetDistribution:
+// {
+//   category: { Electronics: 15, Audio: 8, Accessories: 3 },
+//   brand: { AudioTech: 10, SoundMax: 8, BassKing: 5 },
+//   tags: { wireless: 18, bluetooth: 15, 'noise-canceling': 12 },
+// }
+```
+
+### Search with Highlighting
+
+```typescript
+const results = await meili.index('products').search('wireless headphones', {
+  attributesToHighlight: ['name', 'description'],
+  highlightPreTag: '<mark>',
+  highlightPostTag: '</mark>',
+  attributesToCrop: ['description'],
+  cropLength: 30,
+});
+
+// Each hit includes _formatted with highlighted matches:
+// hit._formatted.name = '<mark>Wireless</mark> Noise-Canceling <mark>Headphones</mark>'
+```
+
+### Geo Search
+
+```typescript
+// Find products within 10km of a location
+const results = await meili.index('products').search('coffee shop', {
+  filter: ['_geoRadius(40.7128, -74.0060, 10000)'], // lat, lng, radius in meters
+  sort: ['_geoPoint(40.7128, -74.0060):asc'], // Sort by distance
+});
+
+// Bounding box filter
+const results2 = await meili.index('products').search('', {
+  filter: ['_geoBoundingBox([40.82, -74.02], [40.70, -73.95])'], // top-left, bottom-right
+});
+```
+
+### Multi-Index Search
+
+```typescript
+const results = await meili.multiSearch({
+  queries: [
+    { indexUid: 'products', q: 'wireless', limit: 5 },
+    { indexUid: 'articles', q: 'wireless', limit: 5 },
+    { indexUid: 'categories', q: 'wireless', limit: 3 },
+  ],
+});
+
+// results.results is an array of search results, one per query
+```
+
+## Tenant Tokens (Multi-Tenancy)
+
+Tenant tokens restrict search to specific filter rules. Generate them server-side, use them client-side.
+
+```typescript
+// Server: Generate a tenant token
+import { MeiliSearch } from 'meilisearch';
+
+const client = new MeiliSearch({
+  host: process.env.MEILISEARCH_HOST!,
+  apiKey: process.env.MEILISEARCH_ADMIN_KEY!,
+});
+
+function generateTenantToken(tenantId: string): string {
+  return client.generateTenantToken(
+    process.env.MEILISEARCH_SEARCH_KEY_UID!, // UID of the search key
+    {
+      products: {
+        filter: `tenantId = "${tenantId}"`, // Enforced filter
+      },
+    },
+    {
+      expiresAt: new Date(Date.now() + 3600 * 1000), // 1 hour
+    },
+  );
+}
+```
+
+```typescript
+// Client: Use the tenant token
+const tenantClient = new MeiliSearch({
+  host: 'https://search.example.com',
+  apiKey: tenantToken, // Token from server
+});
+
+// All searches are automatically scoped to the tenant
+const results = await tenantClient.index('products').search('headphones');
+// Only returns products where tenantId matches
+```
+
+## Syncing Data
+
+```typescript
+// Pattern: Sync database changes to Meilisearch
+// Call this from your database hooks/triggers
+
+async function syncProductToSearch(product: Product, action: 'upsert' | 'delete') {
+  const index = meili.index('products');
+
+  if (action === 'delete') {
+    await index.deleteDocument(product.id);
+    return;
+  }
+
+  await index.addDocuments([{
+    id: product.id,
+    name: product.name,
+    description: product.description,
+    category: product.category,
+    brand: product.brand,
+    price: product.price,
+    rating: product.rating,
+    inStock: product.inStock,
+    tags: product.tags,
+    slug: product.slug,
+    updatedAt: product.updatedAt,
+  }]);
+}
+```
+
+## Anti-Patterns
+
+| Anti-Pattern | Correct Approach |
+|-------------|-----------------|
+| Using the admin key on the client | Use search-only keys or tenant tokens client-side |
+| Not configuring searchableAttributes | Without it, all fields are searchable (slow, noisy results) |
+| Indexing raw database rows with all columns | Index only fields needed for search, display, and filtering |
+| Calling `waitForTask` in request handlers | Indexing is async -- return immediately, let it process in background |
+| Re-indexing entire dataset on every change | Use incremental `addDocuments` (upserts by primary key) |
+| Using Meilisearch as a primary database | It is a search index -- your database is the source of truth |
+
+## Environment Variables
+
+```env
+MEILISEARCH_HOST=http://localhost:7700
+MEILISEARCH_ADMIN_KEY=         # Admin key -- server-side only
+MEILISEARCH_SEARCH_KEY=        # Search-only key -- safe for client
+MEILISEARCH_SEARCH_KEY_UID=    # UID of search key (for tenant tokens)
+```