@nockdev/awf 6.2.0 → 6.2.1

package/.agent/skills/ai-ml/vector-search/SKILL.md

@@ -0,0 +1,110 @@
+# Vector Search & Embeddings
+
+> Vector databases • Semantic search • Hybrid retrieval  
+> Pinecone • Weaviate • Qdrant • ChromaDB • pgvector
+
+---
+
+## Khi Nào Dùng
+
+- Implement semantic search (tìm kiếm theo ý nghĩa)
+- Build RAG (Retrieval-Augmented Generation) pipeline
+- Chọn vector database phù hợp
+- Optimize embedding & search performance
+
+## Architecture
+
+```
+Documents → Chunking → Embedding Model → Vector DB
+                                              │
+Query → Embedding Model ──────────────────────┤
+                                              ▼
+                                     Similarity Search
+                                              │
+                                     Reranking (optional)
+                                              │
+                                     Top-K Results
+```
+
+## Vector DB Selection
+
+| DB           | Type                 | Scale     | Best For                    |
+| ------------ | -------------------- | --------- | --------------------------- |
+| **Pinecone** | Managed cloud        | Billions  | Production SaaS, zero-ops   |
+| **Weaviate** | Self-hosted/cloud    | Millions  | Hybrid search, multi-modal  |
+| **Qdrant**   | Self-hosted/cloud    | Millions  | Performance, filtering      |
+| **ChromaDB** | Embedded/self-hosted | Thousands | Dev/prototype, simple       |
+| **Milvus**   | Self-hosted          | Billions  | Enterprise, GPU-accelerated |
+| **pgvector** | PostgreSQL extension | Millions  | Already using PostgreSQL    |
+
+## Core Patterns
+
+### Embedding Generation
+
+```typescript
+// OpenAI embeddings
+const response = await openai.embeddings.create({
+  model: "text-embedding-3-small",
+  input: "Search query or document text",
+});
+const embedding = response.data[0].embedding; // 1536 dimensions
+```
+
+### Chunking Strategy
+
+| Strategy   | Chunk Size      | Overlap    | Best For        |
+| ---------- | --------------- | ---------- | --------------- |
+| Fixed-size | 500-1000 tokens | 50-100     | General purpose |
+| Sentence   | 3-5 sentences   | 1 sentence | Q&A             |
+| Semantic   | Variable        | Paragraph  | Long documents  |
+| Code       | Function/class  | None       | Codebase search |
+
+### Hybrid Search (Vector + BM25)
+
+```typescript
+// Weaviate hybrid search
+const result = await client.graphql
+  .get()
+  .withClassName("Document")
+  .withHybrid({ query: "machine learning", alpha: 0.5 }) // 0=BM25, 1=vector
+  .withLimit(10)
+  .do();
+```
+
+### pgvector Quick Start
+
+```sql
+CREATE EXTENSION vector;
+CREATE TABLE documents (
+  id SERIAL PRIMARY KEY,
+  content TEXT,
+  embedding vector(1536)
+);
+CREATE INDEX ON documents USING hnsw (embedding vector_cosine_ops);
+
+-- Search
+SELECT content, 1 - (embedding <=> query_embedding) AS similarity
+FROM documents ORDER BY embedding <=> query_embedding LIMIT 10;
+```
+
+## Distance Metrics
+
+| Metric             | Formula    | When to Use                         |
+| ------------------ | ---------- | ----------------------------------- |
+| **Cosine**         | 1 - cos(θ) | Normalized embeddings (most common) |
+| **L2 (Euclidean)** | √Σ(a-b)²   | When magnitude matters              |
+| **Dot Product**    | Σ(a×b)     | Already normalized, fastest         |
+
+## Common Traps
+
+| Trap                | Fix                                        |
+| ------------------- | ------------------------------------------ |
+| Wrong chunk size    | Test 256/512/1024 tokens, measure recall   |
+| Slow search         | Use HNSW index, reduce dimensions          |
+| Poor recall         | Hybrid search (vector + BM25)              |
+| High embedding cost | Batch embeddings, cache, use smaller model |
+| Stale data          | Implement incremental indexing pipeline    |
+
+---
+
+_DOMYH Awesome Code • Vector Search Skill v1.0.0_

package/.agent/skills/ai-ml/vector-search/data/embedding-models.yaml

@@ -0,0 +1,117 @@
+# Embedding Models Comparison — HSA Tier 3 Data
+version: "1.0.0"
+
+patterns:
+  - id: EMB-001
+    name: OpenAI text-embedding-3
+    category: commercial
+    description: |
+      OpenAI embedding models.
+    text: |
+      Models:
+      - text-embedding-3-small: 1536 dims, $0.02/1M tokens, best value
+      - text-embedding-3-large: 3072 dims, $0.13/1M tokens, best quality
+      - text-embedding-ada-002: 1536 dims (legacy, deprecated)
+
+      Features: Dimension reduction via dimensions parameter
+      API: openai.embeddings.create()
+    example: |
+      const response = await openai.embeddings.create({
+        model: "text-embedding-3-small",
+        input: ["search query", "document text"],
+        dimensions: 512  // Optional: reduce for cost savings
+      });
+      const embeddings = response.data.map(d => d.embedding);
+
+  - id: EMB-002
+    name: Cohere Embed v3
+    category: commercial
+    description: |
+      Cohere embedding with separate query/document modes.
+    text: |
+      Model: embed-multilingual-v3.0
+      Dims: 1024
+      Special: input_type parameter (search_query vs search_document)
+      Multilingual: 100+ languages
+    example: |
+      const response = await cohere.embed({
+        texts: ["search query"],
+        model: "embed-multilingual-v3.0",
+        inputType: "search_query",  // or "search_document" for indexing
+        embeddingTypes: ["float"]
+      });
+
+  - id: EMB-003
+    name: Voyage AI
+    category: commercial
+    description: |
+      Specialized embeddings for code and text.
+    text: |
+      Models:
+      - voyage-3: General purpose, 1024 dims
+      - voyage-code-3: Code-optimized, 1024 dims
+      - voyage-3-lite: Lightweight, 512 dims
+
+      Best for: Code search, technical documentation
+
+  - id: EMB-004
+    name: Local Models (Sentence-Transformers)
+    category: open-source
+    description: |
+      Free, self-hosted embedding models.
+    text: |
+      Top models:
+      - all-MiniLM-L6-v2: 384 dims, fastest, good quality
+      - all-mpnet-base-v2: 768 dims, best quality/speed
+      - multilingual-e5-large: 1024 dims, multilingual
+      - nomic-embed-text-v1.5: 768 dims, Matryoshka
+
+      Frameworks: sentence-transformers, ollama, fastembed
+    example: |
+      from sentence_transformers import SentenceTransformer
+      model = SentenceTransformer("all-MiniLM-L6-v2")
+      embeddings = model.encode(["text1", "text2"])
+
+  - id: EMB-005
+    name: Chunking Strategies
+    category: preprocessing
+    description: |
+      How to split documents before embedding.
+    text: |
+      | Strategy | Chunk Size | Overlap | Use Case |
+      |----------|-----------|---------|----------|
+      | Fixed tokens | 500-1000 | 50-100 | General purpose |
+      | Sentences | 3-5 sentences | 1 | Q&A systems |
+      | Paragraphs | Variable | 0 | Articles, blogs |
+      | Semantic | Variable | 0 | Mixed content |
+      | Code | Function/class | 0 | Codebase search |
+      | Markdown | Headers | 0 | Documentation |
+
+      Tools: LangChain TextSplitters, LlamaIndex NodeParsers
+
+  - id: EMB-006
+    name: Embedding Best Practices
+    category: optimization
+    description: |
+      Production embedding optimization.
+    text: |
+      1. Batch embeddings (reduce API calls)
+      2. Cache embeddings (avoid re-computing)
+      3. Use smaller dimensions when possible
+      4. Separate query vs document embeddings if model supports it
+      5. Normalize embeddings for cosine similarity
+      6. Use Matryoshka models for dimension flexibility
+
+  - id: EMB-007
+    name: Model Selection Guide
+    category: comparison
+    description: |
+      How to choose embedding model.
+    text: |
+      Decision flow:
+      Budget zero → Local: all-MiniLM-L6-v2 or nomic-embed
+      Need multilingual → Cohere embed-multilingual-v3
+      Code search → Voyage code-3
+      General best quality → OpenAI text-embedding-3-large
+      Best cost/quality → OpenAI text-embedding-3-small
+      Privacy/latency → Local: all-mpnet-base-v2

package/.agent/skills/ai-ml/vector-search/data/search-patterns.yaml

@@ -0,0 +1,118 @@
+# Search Patterns — HSA Tier 3 Data
+version: "1.0.0"
+
+patterns:
+  - id: SRCH-001
+    name: Hybrid Search
+    category: retrieval
+    description: |
+      Combine vector similarity with keyword BM25 for better recall.
+    text: |
+      Alpha parameter: 0.0 = pure BM25, 1.0 = pure vector
+      Best: alpha 0.4-0.7 (balanced)
+      Benefit: Catches exact matches that vectors miss
+    example: |
+      // Weaviate hybrid search
+      const results = await collection.query.hybrid("machine learning", {
+        alpha: 0.5,
+        limit: 10,
+        returnMetadata: ["score"]
+      });
+
+      // pgvector + ts_rank hybrid
+      SELECT content,
+        (0.5 * ts_rank(to_tsvector(content), query) + 
+         0.5 * (1 - (embedding <=> query_embedding))) AS score
+      FROM documents, plainto_tsquery('machine learning') query
+      ORDER BY score DESC LIMIT 10;
+
+  - id: SRCH-002
+    name: Metadata Filtering
+    category: retrieval
+    description: |
+      Pre-filter vectors by metadata before similarity search.
+    text: |
+      Strategies:
+      - Pre-filter: Filter first, then vector search (faster)
+      - Post-filter: Vector search first, then filter (better recall)
+      Common filters: date range, category, language, access level
+    example: |
+      // Pinecone with filter
+      await index.query({
+        vector: queryEmbedding,
+        topK: 10,
+        filter: {
+          category: { $eq: "documentation" },
+          date: { $gte: "2025-01-01" }
+        }
+      });
+
+  - id: SRCH-003
+    name: Reranking
+    category: retrieval
+    description: |
+      Rerank initial results for higher precision.
+    text: |
+      Pipeline: Query → Retrieve top-100 → Rerank → Return top-10
+      Models: Cohere Rerank, cross-encoder/ms-marco
+      Benefit: 10-30% precision improvement
+    example: |
+      // Cohere Rerank
+      const reranked = await cohere.rerank({
+        model: "rerank-multilingual-v3.0",
+        query: "What is machine learning?",
+        documents: initialResults.map(r => r.text),
+        topN: 10
+      });
+
+  - id: SRCH-004
+    name: Multi-vector Search
+    category: retrieval
+    description: |
+      Use multiple vectors per document for richer representation.
+    text: |
+      Approaches:
+      - ColBERT: Token-level vectors (late interaction)
+      - Multi-chunk: One vector per chunk, link to parent doc
+      - Multi-modal: Text vector + image vector per document
+
+  - id: SRCH-005
+    name: Namespace Isolation
+    category: architecture
+    description: |
+      Separate vectors by tenant/project for multi-tenancy.
+    text: |
+      Strategies:
+      - Pinecone: Use namespaces
+      - Weaviate: Use multi-tenancy
+      - Qdrant: Use collection aliases or payload filtering
+      - pgvector: Use separate schemas or tenant_id column
+
+  - id: SRCH-006
+    name: Incremental Indexing
+    category: pipeline
+    description: |
+      Keep vector index in sync with source data.
+    text: |
+      Pipeline:
+      1. Detect new/modified/deleted documents
+      2. Chunk modified documents
+      3. Generate embeddings (batch)
+      4. Upsert vectors with document IDs
+      5. Delete vectors for removed documents
+
+      Tools: LlamaIndex IngestionPipeline, custom CDC pipeline
+
+  - id: SRCH-007
+    name: Evaluation Metrics
+    category: quality
+    description: |
+      Measure search quality.
+    text: |
+      Metrics:
+      - Recall@K: % of relevant docs in top-K results
+      - MRR: Mean Reciprocal Rank
+      - NDCG: Normalized Discounted Cumulative Gain
+      - Latency: p50/p95/p99 query time
+
+      Tools: RAGAS, BEIR benchmark, custom eval sets

package/.agent/skills/ai-ml/vector-search/data/vector-dbs.yaml

@@ -0,0 +1,155 @@
+# Vector Databases Comparison — HSA Tier 3 Data
+version: "1.0.0"
+
+patterns:
+  - id: VDB-001
+    name: Pinecone
+    category: managed
+    description: |
+      Fully managed vector database.
+    text: |
+      Type: Cloud-only (managed)
+      Scale: Billions of vectors
+      Index: Proprietary (optimized HNSW)
+      Pricing: Pay per pod/serverless
+
+      Pros: Zero-ops, fastest setup, serverless option
+      Cons: Cloud-only, vendor lock-in, cost at scale
+
+      Best for: Production SaaS, teams without infra expertise
+    example: |
+      import { Pinecone } from "@pinecone-database/pinecone";
+      const pc = new Pinecone({ apiKey: "..." });
+      const index = pc.index("my-index");
+
+      // Upsert
+      await index.upsert([
+        { id: "doc1", values: [0.1, 0.2, ...], metadata: { source: "readme" } }
+      ]);
+
+      // Query
+      const results = await index.query({
+        vector: queryEmbedding,
+        topK: 10,
+        filter: { source: { $eq: "readme" } },
+        includeMetadata: true
+      });
+
+  - id: VDB-002
+    name: Weaviate
+    category: self-hosted
+    description: |
+      Open-source vector database with built-in vectorizers.
+    text: |
+      Type: Self-hosted / Cloud (WCS)
+      Scale: Millions of vectors
+      Index: HNSW
+      Special: Built-in vectorizer modules, hybrid search native
+
+      Pros: Hybrid search (BM25+vector), multi-modal, GraphQL API
+      Cons: Complex setup, resource-heavy
+
+      Best for: Hybrid search requirements, multi-modal data
+    example: |
+      import weaviate from "weaviate-client";
+      const client = await weaviate.connectToLocal();
+
+      // Hybrid search
+      const result = await client.collections.get("Document")
+        .query.hybrid("machine learning", { alpha: 0.5, limit: 10 });
+
+  - id: VDB-003
+    name: Qdrant
+    category: self-hosted
+    description: |
+      High-performance vector database written in Rust.
+    text: |
+      Type: Self-hosted / Cloud
+      Scale: Millions of vectors
+      Index: HNSW with modifications
+      Language: Rust (fast, memory-efficient)
+
+      Pros: Fast filtering, payload indexing, batch operations
+      Cons: Newer ecosystem
+
+      Best for: Performance-critical apps, complex filtering
+    example: |
+      import { QdrantClient } from "@qdrant/js-client-rest";
+      const client = new QdrantClient({ url: "http://localhost:6333" });
+
+      await client.upsert("my_collection", {
+        points: [
+          { id: 1, vector: [0.1, 0.2, ...], payload: { title: "Doc 1" } }
+        ]
+      });
+
+  - id: VDB-004
+    name: ChromaDB
+    category: embedded
+    description: |
+      Simple, developer-friendly vector database.
+    text: |
+      Type: Embedded / Self-hosted
+      Scale: Thousands to low millions
+      Index: HNSW
+
+      Pros: Simplest setup, Python-native, great for prototyping
+      Cons: Not for production scale, single-node
+
+      Best for: Prototypes, RAG development, local testing
+
+  - id: VDB-005
+    name: pgvector
+    category: extension
+    description: |
+      PostgreSQL extension for vector similarity search.
+    text: |
+      Type: PostgreSQL extension
+      Scale: Millions (with proper indexing)
+      Index: HNSW, IVFFlat
+
+      Pros: Use existing PostgreSQL, no new infra, SQL queries
+      Cons: Performance ceiling vs dedicated vector DB
+
+      Best for: Already using PostgreSQL, moderate scale
+    example: |
+      CREATE EXTENSION vector;
+      CREATE TABLE items (
+        id SERIAL PRIMARY KEY,
+        content TEXT,
+        embedding vector(1536)
+      );
+      CREATE INDEX ON items USING hnsw (embedding vector_cosine_ops);
+
+      SELECT content, 1 - (embedding <=> $1) AS similarity
+      FROM items ORDER BY embedding <=> $1 LIMIT 10;
+
+  - id: VDB-006
+    name: Milvus
+    category: self-hosted
+    description: |
+      Enterprise-grade, GPU-accelerated vector database.
+    text: |
+      Type: Self-hosted / Cloud (Zilliz)
+      Scale: Billions of vectors
+      Index: HNSW, IVF, PQ, GPU-IVF
+
+      Pros: GPU acceleration, massive scale, multi-vector
+      Cons: Complex deployment, heavy resources
+
+      Best for: Enterprise, GPU-accelerated workloads
+
+  - id: VDB-007
+    name: Selection Decision Matrix
+    category: comparison
+    description: |
+      How to choose the right vector database.
+    text: |
+      Decision flow:
+      1. Already using PostgreSQL? → pgvector
+      2. Budget: zero? → ChromaDB (dev), Qdrant (prod)
+      3. Need hybrid search? → Weaviate
+      4. Zero infrastructure? → Pinecone
+      5. Maximum performance? → Qdrant
+      6. Billions of vectors? → Pinecone or Milvus
+      7. GPU available? → Milvus

package/.agent/skills/core/api-design/META.yaml

@@ -1,5 +1,5 @@
 name: api-design
-version: "6.2.0"
+version: "6.2.1"
 display: API Design Patterns
 category: core
 tier: 1
@@ -45,10 +45,6 @@ caps:
   - OpenAPI/Swagger documentation
   - Test API suggestions (JSONPlaceholder/httpbin/reqres)
 
-hsa:
-  enabled: true
-  adapter: api_design_adapter.py
-  functions:
     - search_api_patterns
     - suggest_test_api
 

package/.agent/skills/core/api-design/SKILL.md

@@ -7,21 +7,15 @@
 
 ## 🎯 HSA Integration
 
-```python
-# Search API patterns dynamically
-from hsa.adapters import search_api_patterns, suggest_test_api
+Data powered by HSA BM25 search engine. Query YAML data via skill search:
 
-# Find versioning patterns
-results = search_api_patterns("versioning strategy", max_results=5)
-
-# Search by domain
-results = search_api_patterns("streaming", domain="grpc")
-
-# Get test API suggestions
-apis = suggest_test_api("user data REST")
-for api in apis:
-    print(f"{api.pattern}: {api.url}")
-```
+| Domain     | Query Examples                   |
+| ---------- | -------------------------------- |
+| REST       | "versioning strategy URI header" |
+| GraphQL    | "N+1 DataLoader schema"          |
+| gRPC       | "streaming protobuf reliability" |
+| Error      | "RFC 9457 error response format" |
+| Rate Limit | "token bucket sliding window"    |
 
 ---
 
@@ -93,23 +87,23 @@ for api in apis:
 | FakeStore API   | fakestoreapi.com             | E-commerce         |
 | Postman Echo    | postman-echo.com             | Request inspection |
 
-> 💡 Use `suggest_test_api("your use case")` to find matching APIs
+> 💡 See `test-apis.yaml` for matching APIs by use case
 
 ---
 
 ## 📁 Data Files
 
-| File                  | Patterns | Coverage                           |
-| --------------------- | -------- | ---------------------------------- |
-| rest-patterns.csv     | 30       | URI, methods, HATEOAS, caching     |
-| graphql-patterns.csv  | 25       | Schema, resolvers, N+1, DataLoader |
-| grpc-patterns.csv     | 25       | Protobuf, streaming, reliability   |
-| api-versioning.csv    | 25       | URI, header, Stripe/GitHub/Twilio  |
-| pagination.csv        | 15       | Offset, cursor, keyset             |
-| rate-limiting.csv     | 20       | Token bucket, sliding window       |
-| error-responses.csv   | 20       | RFC 9457, validation errors        |
-| http-status-codes.csv | 27       | 2xx, 4xx, 5xx reference            |
-| test-apis.csv         | 25       | Public APIs for testing            |
+| File                     | Patterns | Coverage                           |
+| ------------------------ | -------- | ---------------------------------- |
+| `rest-patterns.yaml`     | 30       | URI, methods, HATEOAS, caching     |
+| `graphql-patterns.yaml`  | 25       | Schema, resolvers, N+1, DataLoader |
+| `grpc-patterns.yaml`     | 25       | Protobuf, streaming, reliability   |
+| `api-versioning.yaml`    | 25       | URI, header, Stripe/GitHub/Twilio  |
+| `pagination.yaml`        | 15       | Offset, cursor, keyset             |
+| `rate-limiting.yaml`     | 20       | Token bucket, sliding window       |
+| `error-responses.yaml`   | 20       | RFC 9457, validation errors        |
+| `http-status-codes.yaml` | 27       | 2xx, 4xx, 5xx reference            |
+| `test-apis.yaml`         | 25       | Public APIs for testing            |
 
 **Total: 202 patterns**