create-harper 1.2.2 → 1.3.0

package/template-react/skills/vector-indexing.md

@@ -1,215 +0,0 @@
-# Vector Indexing
-
-Harper supports **vector indexing** on array attributes, enabling efficient similarity search over high-dimensional vector data. This is essential for AI-powered features such as semantic search, recommendations, and embeddings-based retrieval.
-
----
-
-## What Is Vector Indexing
-
-Vector indexing organizes numeric vectors so that Harper can efficiently find records that are closest to a given query vector using a distance metric such as cosine similarity or Euclidean distance.
-
-Unlike traditional indexes that rely on exact matches, vector indexes enable **nearest-neighbor search** across high-dimensional spaces, making them ideal for embeddings and machine learning workloads.
-
----
-
-## Enabling a Vector Index
-
-Vector indexes are defined using the `@indexed` directive on numeric array attributes.
-
-```graphql
-type Product @table {
-	id: Long @primaryKey
-	name: String
-	description: String
-	textEmbeddings: [Float] @indexed(type: "HNSW")
-	price: Float
-}
-```
-
-- `type: "HNSW"` enables Harper’s vector index using the HNSW algorithm
-- The indexed field must be an array of numeric values
-- Vector indexes are stored and maintained automatically
-
----
-
-## Querying with a Vector Index
-
-### Search Vectors with sort
-
-Once defined, vector indexes can be used by specifying a `sort` configuration with a target vector. To view the similarity of a result to a given query vector, use the `$distance` attribute in the `select` clause.
-
-```js
-const results = Product.search({
-	select: ['name', 'description', 'price', '$distance'],
-	sort: {
-		attribute: 'textEmbeddings',
-		target: searchVector,
-	},
-	limit: 5,
-});
-```
-
-- `attribute` is the vector index attribute
-- `target` is the vector to compare against
-- `searchVector` is the embedding to compare against
-- Results are ordered by similarity
-- Vector search can be combined with filters and limits
-- The `$distance` attribute in the `select` (optional) returns the distance between the result and the query vector
-
-### Search Vectors limited by distance
-
-Vector indexes results can be limited by distance using the `conditions` clause. In the following example, results are returned that are less than 0.1 similar to the query vector.
-The `conditions` clause can be combined with `sort` and `limit` and the `comparator` can be any of the following: `lt`, `lte`, `gt`, `gte`, `between`.
-
-```js
-const results = Product.search({
-	select: ['name', 'description', 'price', '$distance'],
-	conditions: {
-		attribute: 'textEmbeddings',
-		comparator: 'lt',
-		value: 0.1, // '0.1' is the similarity threshold
-		target: searchVector,
-	},
-});
-```
-
-- `attribute` is the vector index attribute
-- `comparator` is the comparison operator (`lt`, `lte`, `gt`, `gte`, `between` are accepted)
-- `value` is the threshold value
-- `target` is the vector to compare against
-- `searchVector` is the embedding to compare against
-- Vector search can be combined with filters, sort, and limits
-- The `$distance` attribute in the `select` (optional) returns the distance between the result and the query vector
-
----
-
-## Vector Index Options
-
-Additional tuning options can be provided on the `@indexed` directive:
-
-| Option                 | Description                                 |
-| ---------------------- | ------------------------------------------- |
-| `distance`             | Similarity metric (`cosine` or `euclidean`) |
-| `efConstruction`       | Index build quality vs performance          |
-| `M`                    | Graph connectivity per HNSW layer           |
-| `optimizeRouting`      | Improves routing efficiency                 |
-| `efSearchConstruction` | Search breadth during queries               |
-
-These options allow fine-tuning for performance and recall tradeoffs.
-
----
-
-## How to Generate and Search Vector Embeddings
-
-Here is a full example that generates embeddings for a set of products and then searches for similar products using vector indexes. The following example shows how to generate embeddings using OpenAI or Ollama.
-
-```js
-import { Ollama } from 'ollama';
-const ollama = new Ollama({ host: 'http://127.0.0.1:11434' });
-// The name of the ollama embedding model
-const OLLAMA_EMBEDDING_MODEL = 'nomic-embed-text';
-
-const { Product } = tables;
-
-import OpenAI from 'openai';
-const openai = new OpenAI();
-// the name of the OpenAI embedding model
-const OPENAI_EMBEDDING_MODEL = 'text-embedding-3-small';
-
-const SIMILARITY_THRESHOLD = 0.5;
-
-export class ProductSearch extends Resource {
-	// based on env variable we choose the appropriate embedding generator
-	generateEmbedding = process.env.EMBEDDING_GENERATOR === 'ollama'
-		? this._generateOllamaEmbedding
-		: this._generateOpenAIEmbedding;
-
-	/**
-	 * Executes a search query using a generated text embedding and returns the matching products.
-	 *
-	 * @param {Object} data - The input data for the request.
-	 * @param {string} data.prompt - The prompt to generate the text embedding from.
-	 * @return {Promise<Array>} Returns a promise that resolves to an array of products matching the conditions,
-	 * including fields: name, description, price, and $distance.
-	 */
-	async post(data) {
-		const embedding = await this.generateEmbedding(data.prompt);
-
-		return await Product.search({
-			select: ['name', 'description', 'price', '$distance'],
-			conditions: {
-				attribute: 'textEmbeddings',
-				comparator: 'lt',
-				value: SIMILARITY_THRESHOLD,
-				target: embedding[0],
-			},
-			limit: 5,
-		});
-	}
-
-	/**
-	 * Generates an embedding using the Ollama API.
-	 *
-	 * @param {string} promptData - The input data for which the embedding is to be generated.
-	 * @return {Promise<number[][]>} A promise that resolves to the generated embedding as an array of numbers.
-	 */
-	async _generateOllamaEmbedding(promptData) {
-		const embedding = await ollama.embed({
-			model: OLLAMA_EMBEDDING_MODEL,
-			input: promptData,
-		});
-		return embedding?.embeddings;
-	}
-
-	/**
-	 * Generates OpenAI embeddings based on the given prompt data.
-	 *
-	 * @param {string} promptData - The input data used for generating the embedding.
-	 * @return {Promise<number[][]>} A promise that resolves to an array of embeddings, where each embedding is an array of floats.
-	 */
-	async _generateOpenAIEmbedding(promptData) {
-		const embedding = await openai.embeddings.create({
-			model: OPENAI_EMBEDDING_MODEL,
-			input: promptData,
-			encoding_format: 'float',
-		});
-
-		let embeddings = [];
-		embedding.data.forEach((embeddingData) => {
-			embeddings.push(embeddingData.embedding);
-		});
-
-		return embeddings;
-	}
-}
-```
-
-Sample request to the `ProductSearch` resource which prompts to find "shorts for the gym":
-
-```bash
-curl -X POST "http://localhost:9926/ProductSearch/" \
--H "accept: \
--H "Content-Type: application/json" \
--H "Authorization: Basic <YOUR_AUTH>" \
--d '{"prompt": "shorts for the gym"}'
-```
-
----
-
-## When to Use Vector Indexing
-
-Vector indexing is ideal when:
-
-- Storing embedding vectors from ML models
-- Performing semantic or similarity-based search
-- Working with high-dimensional numeric data
-- Exact-match indexes are insufficient
-
----
-
-## Summary
-
-- Vector indexing enables fast similarity search on numeric arrays
-- Defined using `@indexed(type: "HNSW")`
-- Queried using a target vector in search sorting
-- Tunable for performance and accuracy

package/template-react-ts/AGENTS.md

@@ -1,22 +0,0 @@
-# Harper Agent Skills
-
-This repository contains "skills" that guide AI agents in developing Harper applications.
-
-## Available Skills
-
-- [Adding Tables with Schemas](skills/adding-tables-with-schemas.md): Learn how to define schemas and enable automatic REST APIs for your database tables with schema .graphql files in Harper.
-- [Automatic APIs](skills/automatic-apis.md): Details on the CRUD endpoints automatically generated for exported tables with REST and WebSockets.
-- [Caching](skills/caching.md): How caching is defined and implemented in Harper applications.
-- [Checking Authentication](skills/checking-authentication.md): How to use sessions to verify user identity and roles.
-- [Custom Resources](skills/custom-resources.md): How to define custom REST endpoints using JavaScript or TypeScript (Note: Paths are case-sensitive).
-- [Defining Relationships](skills/defining-relationships.md): Using the `@relationship` directive to link tables.
-- [Deploying to Harper Fabric](skills/deploying-to-harper-fabric.md): Globally scaling your Harper application with our generous Free tier and beyond.
-- [Extending Table Resources](skills/extending-tables.md): Adding custom logic to automatically generated table resources.
-- [Handling Binary Data](skills/handling-binary-data.md): How to store and serve binary data like images or MP3s.
-- [Programmatic Table Requests](skills/programmatic-table-requests.md): How to use filters, operators, sorting, and pagination in programmatic table requests.
-- [Querying REST APIs](skills/querying-rest-apis.md): How to use filters, operators, sorting, and pagination in REST requests.
-- [Real-time Applications](skills/real-time-apps.md): Implementing WebSockets and Pub/Sub for live data updates.
-- [Serving Web Content](skills/serving-web-content): Two ways to serve web content from a Harper application.
-- [TypeScript Type Stripping](skills/typescript-type-stripping.md): Using TypeScript directly without build tools via Node.js Type Stripping.
-- [Using Blobs](skills/using-blob-datatype.md): How to store and retrieve large data in HarperDB.
-- [Vector Indexing](skills/vector-indexing.md): How to define and use vector indexes for efficient similarity search.

package/template-react-ts/skills/adding-tables-with-schemas.md

@@ -1,34 +0,0 @@
-# Adding Tables to Harper
-
-To add tables to a Harper database, follow these guidelines:
-
-1. **Dedicated Schema Files**: Prefer having a dedicated schema `.graphql` file for each table. Check the `config.yaml` file under `graphqlSchema.files` to see how it's configured. It typically accepts wildcards (e.g., `schemas/*.graphql`), but may be configured to point at a single file.
-
-2. **Directives**: All available directives for defining your schema are defined in `node_modules/harperdb/schema.graphql`. Common directives include `@table`, `@export`, `@primaryKey`, `@indexed`, and `@relationship`.
-
-3. **Defining Relationships**: You can link tables together using the `@relationship` directive. For more details, see the [Defining Relationships](defining-relationships.md) skill.
-
-4. **Automatic APIs**: If you add `@table @export` to a schema type, Harper automatically sets up REST and WebSocket APIs for basic CRUD operations against that table. For a detailed list of available endpoints and how to use them, see the [Automatic REST APIs](automatic-apis.md) skill.
-
-   - `GET /{TableName}`: Describes the schema itself.
-   - `GET /{TableName}/`: Lists all records (supports filtering, sorting, and pagination via query parameters). See the [Querying REST APIs](querying-rest-apis.md) skill for details.
-   - `GET /{TableName}/{id}`: Retrieves a single record by its ID.
-   - `POST /{TableName}/`: Creates a new record.
-   - `PUT /{TableName}/{id}`: Updates an existing record.
-   - `PATCH /{TableName}/{id}`: Performs a partial update on a record.
-   - `DELETE /{TableName}/`: Deletes all records or filtered records.
-   - `DELETE /{TableName}/{id}`: Deletes a single record by its ID.
-
-### Example
-
-In `schemas/ExamplePerson.graphql`:
-
-```graphql
-type ExamplePerson @table @export {
-	id: ID @primaryKey
-	name: String
-	tag: String @indexed
-}
-```
-
-Tip: if you are going to [extend the table](./extending-tables.md) in your resources, then do not `@export` the table from the schema.

package/template-react-ts/skills/automatic-apis.md

@@ -1,53 +0,0 @@
-# Automatic APIs in Harper
-
-When you define a GraphQL type with the `@table` and `@export` directives, Harper automatically generates a fully-functional REST API and WebSocket interface for that table. This allows for immediate CRUD (Create, Read, Update, Delete) operations and real-time updates without writing any additional code.
-
-## Enabling Automatic APIs
-
-To enable the automatic REST and WebSocket APIs for a table, ensure your GraphQL schema includes the `@export` directive:
-
-```graphql
-type MyTable @table @export {
-	id: ID @primaryKey
-	# ... other fields
-}
-```
-
-## Available REST Endpoints
-
-The following endpoints are automatically created for a table named `TableName` (Note: Paths are **case-sensitive**, so `GET /TableName/` is valid while `GET /tablename/` is not):
-
-- **Describe Schema**: `GET /{TableName}`
-  Returns the schema definition and metadata for the table.
-- **List Records**: `GET /{TableName}/`
-  Lists all records in the table. This endpoint supports advanced filtering, sorting, and pagination. For more details, see the [Querying REST APIs](querying-rest-apis.md) skill.
-- **Get Single Record**: `GET /{TableName}/{id}`
-  Retrieves a single record by its primary key (`id`).
-- **Create Record**: `POST /{TableName}/`
-  Creates a new record. The request body should be a JSON object containing the record data.
-- **Update Record (Full)**: `PUT /{TableName}/{id}`
-  Replaces the entire record at the specified `id` with the provided JSON data.
-- **Update Record (Partial)**: `PATCH /{TableName}/{id}`
-  Updates only the specified fields of the record at the given `id`.
-- **Delete All/Filtered Records**: `DELETE /{TableName}/`
-  Deletes all records in the table, or a subset of records if filtering parameters are provided.
-- **Delete Single Record**: `DELETE /{TableName}/{id}`
-  Deletes the record with the specified `id`.
-
-## Automatic WebSockets
-
-In addition to REST endpoints, Harper also stands up WebSocket interfaces for exported tables. When you connect to the table's endpoint via WebSocket, you will automatically receive events whenever updates are made to that table.
-
-- **WebSocket Endpoint**: `ws://your-harper-instance/{TableName}`
-
-This is the easiest way to add real-time capabilities to your application. For more complex real-time needs, see the [Real-time Applications](real-time-apps.md) skill.
-
-## Filtering and Querying
-
-The `GET /{TableName}/` and `DELETE /{TableName}/` endpoints can be filtered using query parameters. While basic equality filters are straightforward, Harper supports a rich set of operators, sorting, and pagination.
-
-For a comprehensive guide on advanced querying, see the [Querying REST APIs](querying-rest-apis.md) skill.
-
-## Customizing Resources
-
-If the automatic APIs don't behave how you need, then you can look to [customize the resources](./custom-resources.md).

package/template-react-ts/skills/automatic-rest-apis.md

@@ -1,41 +0,0 @@
-# Automatic REST APIs in HarperDB
-
-When you define a GraphQL type with the `@table` and `@export` directives, HarperDB automatically generates a fully-functional REST API for that table. This allows for immediate CRUD (Create, Read, Update, Delete) operations without writing any additional code.
-
-## Enabling REST APIs
-
-To enable the automatic REST API for a table, ensure your GraphQL schema includes the `@export` directive:
-
-```graphql
-type MyTable @table @export {
-	id: ID @primaryKey
-	# ... other fields
-}
-```
-
-## Available Endpoints
-
-The following endpoints are automatically created for a table named `TableName` (Note: Paths are **case-sensitive**, so `GET /TableName/` is valid while `GET /tablename/` is not):
-
-- **Describe Schema**: `GET /{TableName}`
-  Returns the schema definition and metadata for the table.
-- **List Records**: `GET /{TableName}/`
-  Lists all records in the table. This endpoint supports advanced filtering, sorting, and pagination. For more details, see the [Querying REST APIs](querying-rest-apis.md) skill.
-- **Get Single Record**: `GET /{TableName}/{id}`
-  Retrieves a single record by its primary key (`id`).
-- **Create Record**: `POST /{TableName}/`
-  Creates a new record. The request body should be a JSON object containing the record data.
-- **Update Record (Full)**: `PUT /{TableName}/{id}`
-  Replaces the entire record at the specified `id` with the provided JSON data.
-- **Update Record (Partial)**: `PATCH /{TableName}/{id}`
-  Updates only the specified fields of the record at the given `id`.
-- **Delete All/Filtered Records**: `DELETE /{TableName}/`
-  Deletes all records in the table, or a subset of records if filtering parameters are provided.
-- **Delete Single Record**: `DELETE /{TableName}/{id}`
-  Deletes the record with the specified `id`.
-
-## Filtering and Querying
-
-The `GET /{TableName}/` and `DELETE /{TableName}/` endpoints can be filtered using query parameters. While basic equality filters are straightforward, HarperDB supports a rich set of operators, sorting, and pagination.
-
-For a comprehensive guide on advanced querying, see the [Querying REST APIs](querying-rest-apis.md) skill.

package/template-react-ts/skills/caching.md

@@ -1,113 +0,0 @@
-# Harper Caching
-
-Harper includes integrated support for **caching data from external sources**, enabling high-performance, low-latency cache storage that is fully queryable and interoperable with your applications. With built-in caching capabilities and distributed responsiveness, Harper makes an ideal **data caching server** for both edge and centralized use cases.
-
----
-
-## What is Harper Caching?
-
-Harper caching lets you store **cached content** in standard tables, enabling you to:
-
-- Expose cached entries as **queryable structured data** (e.g., JSON or CSV)
-- Serve data to clients with **flexible formats and custom querying**
-- Manage cache control with **timestamps and ETags** for downstream caching layers
-- Implement **active or passive caching** patterns depending on your source and invalidation strategy
-
----
-
-## Configuring a Cache Table
-
-Define a cache table in your `schema.graphql`:
-
-```graphql
-type MyCache @table(expiration: 3600) @export {
-	id: ID @primaryKey
-}
-```
-
-- `expiration` is defined in seconds
-- Expired records are refreshed on access
-- Evicted records are removed after expiration
-
----
-
-## Connecting an External Source
-
-Create a resource:
-
-```js
-import { Resource } from 'harperdb';
-
-export class ThirdPartyAPI extends Resource {
-	async get() {
-		const id = this.getId();
-		const response = await fetch(`https://api.example.com/items/${id}`);
-		if (!response.ok) {
-			throw new Error('Source fetch failed');
-		}
-		return await response.json();
-	}
-}
-```
-
-Attach it to your table:
-
-```js
-import { tables } from 'harperdb';
-import { ThirdPartyAPI } from './ThirdPartyAPI.js';
-
-const { MyCache } = tables;
-MyCache.sourcedFrom(ThirdPartyAPI);
-```
-
----
-
-## Cache Behavior
-
-1. Fresh data is returned immediately
-2. Missing or stale data triggers a fetch
-3. Concurrent misses are deduplicated
-
----
-
-## Active Caching
-
-Use `subscribe()` to proactively update or invalidate cache entries:
-
-```js
-class MyAPI extends Resource {
-	async *subscribe() {
-		// stream updates
-	}
-}
-```
-
-See [Real Time Apps](real-time-apps.md) for more details.
-
----
-
-## Write-Through Caching
-
-Propagate updates upstream:
-
-```js
-class ThirdPartyAPI extends Resource {
-	async put(data) {
-		await fetch(`https://some-api.com/${this.getId()}`, {
-			method: 'PUT',
-			body: JSON.stringify(data),
-		});
-	}
-}
-```
-
----
-
-## Summary
-
-Harper Caching allows you to:
-
-- Cache external APIs efficiently
-- Query cached data like native tables
-- Prevent cache stampedes
-- Build real-time or write-through caches