@harperfast/template-vue-ts-studio 1.6.3 → 1.7.0

package/.agents/skills/harper-best-practices/rules/vector-indexing.md

@@ -1,40 +1,56 @@
 ---
 name: vector-indexing
 description: How to enable and query vector indexes for similarity search in Harper.
+metadata:
+  mode: generate
+  sources:
+    - reference/v5/database/schema.md#Vector Indexing
+  sourceCommit: b7fbddadd42eb4487190b650a9abc4bcfeef5819
+  inputHash: 3732961c671aac00
 ---
 
 # Vector Indexing
 
-Instructions for the agent to follow when implementing vector search in Harper.
+Instructions for the agent to follow when enabling and querying vector indexes for similarity search in Harper using the HNSW algorithm.
 
 ## When to Use
 
-Use this skill when you need to perform similarity searches on high-dimensional data, such as AI embeddings for semantic search, recommendations, or image retrieval.
+Apply this rule when adding a vector index to a Harper table schema or writing similarity search queries against high-dimensional vector fields. Use it whenever you need approximate nearest-neighbor search, distance-threshold filtering, or distance-scored results.
 
 ## How It Works
 
-1. **Enable Vector Indexing**: In your GraphQL schema, add `@indexed(type: "HNSW")` to a numeric array field:
+1. **Declare the vector index on a `[Float]` field**: Add `@indexed(type: "HNSW")` to any `[Float]` attribute in a `@table` type. See [adding-tables-with-schemas.md](adding-tables-with-schemas.md) for general schema setup.
+
    ```graphql
-   type Product @table {
-   	id: ID @primaryKey
+   type Document @table {
+   	id: Long @primaryKey
    	textEmbeddings: [Float] @indexed(type: "HNSW")
    }
    ```
-2. **Configure Index Options (Optional)**: Fine-tune the index with parameters like `distance` (`cosine` or `euclidean`), `M`, and `efConstruction`.
-3. **Query with Vector Search**: Use `tables.Table.search()` with a `sort` object containing the `target` vector:
+
+2. **Query by nearest neighbors using `sort`**: Call `Document.search()` with a `sort` object containing `attribute` (the indexed field name) and `target` (the query vector). Include `limit` to cap results.
+
    ```javascript
-   const results = await tables.Product.search({
-     select: ['name', '$distance'],
-     sort: {
-       attribute: 'textEmbeddings',
-       target: [0.1, 0.2, ...], // query vector
-     },
-     limit: 5,
+   let results = Document.search({
+   	sort: { attribute: 'textEmbeddings', target: searchVector },
+   	limit: 5,
    });
    ```
-4. **Filter by Distance**: Use `conditions` with a `target` vector and a `comparator` (e.g., `lt`) to return results within a similarity threshold:
+
+3. **Combine with filter conditions**: Add a `conditions` array alongside `sort` to pre-filter records before ranking by similarity.
+
    ```javascript
-   const results = await tables.Product.search({
+   let results = Document.search({
+   	conditions: [{ attribute: 'price', comparator: 'lt', value: 50 }],
+   	sort: { attribute: 'textEmbeddings', target: searchVector },
+   	limit: 5,
+   });
+   ```
+
+4. **Filter by distance threshold**: To return only records within a similarity cutoff (without ranking), place `target` directly on the condition alongside `comparator` and `value`. Omit `sort`.
+
+   ```javascript
+   let results = Document.search({
    	conditions: {
    		attribute: 'textEmbeddings',
    		comparator: 'lt',
@@ -43,117 +59,66 @@ Use this skill when you need to perform similarity searches on high-dimensional
    	},
    });
    ```
-5. **Generate Embeddings**: Use external services (OpenAI, Ollama) to generate the numeric vectors before storing or searching them in Harper.
-
-```typescript
-import OpenAI from 'openai';
-import ollama from 'ollama';
-
-const { Product } = tables;
-const openai = new OpenAI();
-// the name of the OpenAI embedding model
-const OPENAI_EMBEDDING_MODEL = 'text-embedding-3-small';
-
-// the name of the Ollama embedding model
-const OLLAMA_EMBEDDING_MODEL = 'llama3';
-
-const SIMILARITY_THRESHOLD = 0.5;
-
-export class ProductSearch extends Resource {
-	// based on env variable we choose the appropriate embedding generator
-	generateEmbedding =
-		process.env.EMBEDDING_GENERATOR === 'ollama'
-			? this._generateOllamaEmbedding
-			: this._generateOpenAIEmbedding;
-
-	/**
-	 * Executes a search query using a generated text embedding and returns the matching products.
-	 *
-	 * @param {Object} data - The input data for the request.
-	 * @param {string} data.prompt - The prompt to generate the text embedding from.
-	 * @return {Promise<Array>} Returns a promise that resolves to an array of products matching the conditions,
-	 * including fields: name, description, price, and $distance.
-	 */
-	async post(data) {
-		const embedding = await this.generateEmbedding(data.prompt);
-
-		return await Product.search({
-			select: ['name', 'description', 'price', '$distance'],
-			conditions: {
-				attribute: 'textEmbeddings',
-				comparator: 'lt',
-				value: SIMILARITY_THRESHOLD,
-				target: embedding[0],
-			},
-			limit: 5,
-		});
-	}
-
-	/**
-	 * Generates an embedding using the Ollama API.
-	 *
-	 * @param {string} promptData - The input data for which the embedding is to be generated.
-	 * @return {Promise<number[][]>} A promise that resolves to the generated embedding as an array of numbers.
-	 */
-	async _generateOllamaEmbedding(promptData) {
-		const embedding = await ollama.embed({
-			model: OLLAMA_EMBEDDING_MODEL,
-			input: promptData,
-		});
-		return embedding?.embeddings;
-	}
-
-	/**
-	 * Generates OpenAI embeddings based on the given prompt data.
-	 *
-	 * @param {string} promptData - The input data used for generating the embedding.
-	 * @return {Promise<number[][]>} A promise that resolves to an array of embeddings, where each embedding is an array of floats.
-	 */
-	async _generateOpenAIEmbedding(promptData) {
-		const embedding = await openai.embeddings.create({
-			model: OPENAI_EMBEDDING_MODEL,
-			input: promptData,
-			encoding_format: 'float',
-		});
-
-		let embeddings = [];
-		embedding.data.forEach((embeddingData) => {
-			embeddings.push(embeddingData.embedding);
-		});
-
-		return embeddings;
-	}
-}
-```
 
-## Examples
+5. **Include computed distance in results**: Use the special `$distance` field in `select` to return the distance from the target vector. Works with both `sort`-based and `conditions`-based queries.
 
-Sample request to the `ProductSearch` resource which prompts to find "shorts for the gym":
+   ```javascript
+   let results = Document.search({
+   	select: ['name', '$distance'],
+   	sort: { attribute: 'textEmbeddings', target: searchVector },
+   	limit: 5,
+   });
+   ```
 
-```bash
-curl -X POST "http://localhost:9926/ProductSearch/" \
--H "Accept: application/json" \
--H "Content-Type: application/json" \
--H "Authorization: Basic <YOUR_AUTH>" \
--d '{"prompt": "shorts for the gym"}'
-```
+6. **Tune HNSW parameters**: Pass additional parameters to `@indexed(type: "HNSW", ...)` to control index quality and performance.
 
----
+   | Parameter              | Default           | Description                                                                                         |
+   | ---------------------- | ----------------- | --------------------------------------------------------------------------------------------------- |
+   | `distance`             | `"cosine"`        | Distance function: `"euclidean"` or `"cosine"` (negative cosine similarity)                         |
+   | `efConstruction`       | `100`             | Max nodes explored during index construction. Higher = better recall, lower = better performance    |
+   | `M`                    | `16`              | Preferred connections per graph layer. Higher = more space, better recall for high-dimensional data |
+   | `optimizeRouting`      | `0.5`             | Heuristic aggressiveness for omitting redundant connections (0 = off, 1 = most aggressive)          |
+   | `mL`                   | computed from `M` | Normalization factor for level generation                                                           |
+   | `efSearchConstruction` | `50`              | Max nodes explored during search                                                                    |
 
-## When to Use Vector Indexing
+## Examples
 
-Vector indexing is ideal when:
+**Schema with custom HNSW parameters:**
 
-- Storing embedding vectors from ML models
-- Performing semantic or similarity-based search
-- Working with high-dimensional numeric data
-- Exact-match indexes are insufficient
+```graphql
+type Document @table {
+	id: Long @primaryKey
+	textEmbeddings: [Float]
+		@indexed(type: "HNSW", distance: "euclidean", optimizeRouting: 0, efSearchConstruction: 100)
+}
+```
 
----
+**Nearest-neighbor search with distance score:**
+
+```javascript
+let results = Document.search({
+	select: ['name', '$distance'],
+	sort: { attribute: 'textEmbeddings', target: searchVector },
+	limit: 5,
+});
+```
+
+**Distance-threshold filter (no ranking):**
+
+```javascript
+let results = Document.search({
+	conditions: {
+		attribute: 'textEmbeddings',
+		comparator: 'lt',
+		value: 0.1,
+		target: searchVector,
+	},
+});
+```
 
-## Summary
+## Notes
 
-- Vector indexing enables fast similarity search on numeric arrays
-- Defined using `@indexed(type: "HNSW")`
-- Queried using a target vector in search sorting
-- Tunable for performance and accuracy
+- The default `distance` function is `cosine`. Pass `distance: "euclidean"` to switch.
+- `efConstruction` controls index build quality; raising it improves recall at the cost of build time.
+- `$distance` is available in both `sort`-based ranking and `conditions`-based threshold queries.
+- Use the threshold (`conditions` + `target`) form when you want to bound result quality by a similarity cutoff rather than ranking by similarity.

package/.agents/skills/harper-best-practices/rules.manifest.yaml

@@ -0,0 +1,302 @@
+# Rules manifest for the harper-best-practices skill.
+#
+# Declarative source of truth for rule taxonomy, sources, and generation mode.
+# Owned by humans; the generator reads this file and writes derived artifacts
+# (rule bodies, AGENTS.md). See docs/plans/docs-driven-skills.md for the
+# full schema definition and Manifest ↔ frontmatter reconciliation semantics.
+#
+# Phase 2 flipped vector-indexing to mode: generate.
+# Phase 3 flips 7 obvious 1:1 rules to mode: generate (automatic-apis,
+# querying-rest-apis, real-time-apps, checking-authentication, logging,
+# deploying-to-harper-fabric, caching). All others remain synthesized.
+# Phase 4 migrates schema-design-tooling and typescript-type-stripping to
+# mode: generate. creating-harper-apps stays synthesized pending dedicated
+# create-harper CLI documentation.
+
+rules:
+  # ===========================================================================
+  # Schema & Data Design (priority 1 — HIGH)
+  # ===========================================================================
+
+  - rule: adding-tables-with-schemas
+    description: Guidelines for adding tables to a Harper database using GraphQL schemas.
+    category: schema
+    priority: 1
+    order: 1
+    mode: synthesized
+
+  - rule: schema-design-tooling
+    description: Best practices for Harper schema design, including core directives and GraphQL tooling configuration.
+    category: schema
+    priority: 1
+    order: 2
+    mode: generate
+    sources:
+      - path: reference/v5/database/schema.md
+        section: 'Overview'
+        role: primary
+      - path: reference/v5/database/schema.md
+        section: 'Type Directives'
+        role: primary
+      - path: reference/v5/database/schema.md
+        section: 'Field Directives'
+        role: primary
+    must_cover:
+      - '@table'
+      - '@export'
+      - '@primaryKey'
+      - '@indexed'
+      - 'graphqlSchema'
+
+  - rule: defining-relationships
+    description: How to define and use relationships between tables in Harper using GraphQL.
+    category: schema
+    priority: 1
+    order: 3
+    mode: synthesized
+
+  - rule: vector-indexing
+    description: How to enable and query vector indexes for similarity search in Harper.
+    category: schema
+    priority: 1
+    order: 4
+    mode: generate
+    sources:
+      - path: reference/v5/database/schema.md
+        section: 'Vector Indexing'
+        role: primary
+    must_cover:
+      - '@indexed(type: "HNSW")'
+      - 'sort'
+      - 'target'
+      - 'cosine'
+      - 'efConstruction'
+    cross_links:
+      - adding-tables-with-schemas
+
+  - rule: using-blob-datatype
+    description: How to use the Blob data type for efficient binary storage in Harper.
+    category: schema
+    priority: 1
+    order: 5
+    mode: synthesized
+
+  - rule: handling-binary-data
+    description: How to store and serve binary data like images or audio in Harper.
+    category: schema
+    priority: 1
+    order: 6
+    mode: synthesized
+
+  # ===========================================================================
+  # API & Communication (priority 2 — HIGH)
+  # ===========================================================================
+
+  - rule: automatic-apis
+    description: How to use Harper's automatically generated REST and WebSocket APIs.
+    category: api
+    priority: 2
+    order: 1
+    mode: generate
+    sources:
+      - path: reference/v5/rest/overview.md
+        role: primary
+      - path: reference/v5/rest/websockets.md
+        role: supplemental
+    must_cover:
+      - 'rest: true'
+      - '@export'
+      - 'WebSocket'
+    cross_links:
+      - querying-rest-apis
+      - real-time-apps
+
+  - rule: querying-rest-apis
+    description: How to use query parameters to filter, sort, and paginate Harper REST APIs.
+    category: api
+    priority: 2
+    order: 2
+    mode: generate
+    sources:
+      - path: reference/v5/rest/querying.md
+        role: primary
+    must_cover:
+      - '=gt='
+      - '=lt='
+      - 'select('
+      - 'sort('
+      - 'limit('
+    cross_links:
+      - automatic-apis
+
+  - rule: real-time-apps
+    description: How to build real-time features in Harper using WebSockets and Pub/Sub.
+    category: api
+    priority: 2
+    order: 3
+    mode: generate
+    sources:
+      - path: reference/v5/rest/websockets.md
+        role: primary
+    must_cover:
+      - 'WebSocket'
+      - 'connect('
+    cross_links:
+      - automatic-apis
+
+  - rule: checking-authentication
+    description: How to handle user authentication and sessions in Harper Resources.
+    category: api
+    priority: 2
+    order: 4
+    mode: generate
+    sources:
+      - path: reference/v5/resources/resource-api.md
+        section: '`getCurrentUser(): User | undefined`'
+        role: primary
+      - path: reference/v5/resources/resource-api.md
+        section: 'Session and Login from a Resource'
+        role: primary
+      - path: reference/v5/security/jwt-authentication.md
+        role: supplemental
+    must_cover:
+      - 'getCurrentUser()'
+      - 'getContext()'
+      - 'context.login'
+      - 'enableSessions'
+    cross_links:
+      - custom-resources
+
+  # ===========================================================================
+  # Logic & Extension (priority 3 — MEDIUM)
+  # ===========================================================================
+
+  - rule: custom-resources
+    description: How to define custom REST endpoints with JavaScript or TypeScript in Harper.
+    category: logic
+    priority: 3
+    order: 1
+    mode: synthesized
+
+  - rule: extending-tables
+    description: How to add custom logic to automatically generated table resources in Harper.
+    category: logic
+    priority: 3
+    order: 2
+    mode: synthesized
+
+  - rule: programmatic-table-requests
+    description: How to interact with Harper tables programmatically using the `tables` object.
+    category: logic
+    priority: 3
+    order: 3
+    mode: synthesized
+
+  - rule: typescript-type-stripping
+    description: How to run TypeScript files directly in Harper without a build step.
+    category: logic
+    priority: 3
+    order: 4
+    mode: generate
+    sources:
+      - path: reference/v5/components/javascript-environment.md
+        section: 'TypeScript Support'
+        role: primary
+    must_cover:
+      - '.ts'
+      - 'jsResource'
+      - 'Node.js 22.6'
+      - 'type stripping'
+
+  - rule: caching
+    description: How to implement integrated data caching in Harper from external sources.
+    category: logic
+    priority: 3
+    order: 5
+    mode: generate
+    sources:
+      - path: learn/developers/caching-with-harper.md
+        role: primary
+    must_cover:
+      - 'expiration'
+      - 'sourcedFrom'
+      - '@table'
+    cross_links:
+      - custom-resources
+      - automatic-apis
+
+  # ===========================================================================
+  # Infrastructure & Ops (priority 4 — MEDIUM)
+  # ===========================================================================
+
+  - rule: deploying-to-harper-fabric
+    description: How to deploy a Harper application to the Harper Fabric cloud.
+    category: ops
+    priority: 4
+    order: 1
+    mode: generate
+    sources:
+      - path: reference/v5/components/applications.md
+        section: 'Remote Management'
+        role: primary
+      - path: fabric/cluster-creation-management.md
+        section: 'Connecting the Harper CLI to a Cluster'
+        role: supplemental
+    must_cover:
+      - 'harper deploy'
+      - 'harper login'
+    cross_links:
+      - creating-a-fabric-account-and-cluster
+
+  - rule: creating-a-fabric-account-and-cluster
+    description: How to create a Harper Fabric account, organization, and cluster.
+    category: ops
+    priority: 4
+    order: 2
+    mode: synthesized
+
+  - rule: creating-harper-apps
+    description: How to initialize a new Harper application using the CLI.
+    category: ops
+    priority: 4
+    order: 3
+    mode: synthesized
+
+  - rule: serving-web-content
+    description: How to serve static files and integrated Vite/React applications in Harper.
+    category: ops
+    priority: 4
+    order: 4
+    mode: synthesized
+
+  - rule: logging
+    description: Best practices for logging in Harper, including console capture, the granular logger interface, and programmatic log retrieval.
+    category: ops
+    priority: 4
+    order: 5
+    mode: generate
+    sources:
+      - path: reference/v5/logging/overview.md
+        role: primary
+      - path: reference/v5/logging/api.md
+        role: primary
+    must_cover:
+      - 'console.log'
+      - 'logger'
+      - 'withTag('
+
+  - rule: load-env
+    description: How to load environment variables from .env files into a Harper application using the loadEnv plugin.
+    category: ops
+    priority: 4
+    order: 6
+    mode: generate
+    sources:
+      - path: reference/v5/environment-variables/overview.md
+        role: primary
+    must_cover:
+      - 'loadEnv'
+      - 'files'
+      - 'process.env'
+      - 'override'
+      - 'config.yaml'

package/package.json

@@ -1,6 +1,6 @@
 {
 	"name": "@harperfast/template-vue-ts-studio",
-	"version": "1.6.3",
+	"version": "1.7.0",
 	"type": "module",
 	"repository": "github:HarperFast/create-harper",
 	"scripts": {},

package/skills-lock.json

@@ -5,7 +5,7 @@
       "source": "harperfast/skills",
       "sourceType": "github",
       "skillPath": "harper-best-practices/SKILL.md",
-      "computedHash": "838b776573e9ebd05ac70c9ad38233a5bf40d1882be0cc54744168092e96e251"
+      "computedHash": "95a41a7ba907538a69fc52b31399db843b403b617cc6705d9e54ee2c0b7af502"
     }
   }
 }