@harperfast/template-react-ts-studio 1.11.3 → 1.11.4

package/agent/skills/harper-best-practices/rules/schema-design-tooling.md

@@ -9,8 +9,8 @@ metadata:
     - reference/v5/database/schema.md#Overview
     - reference/v5/database/schema.md#Type Directives
     - reference/v5/database/schema.md#Field Directives
-  sourceCommit: 4fe4c9c95e0974eaa77032f6f10e36fbd8ec64ac
-  inputHash: 15b9c72decc1f05d
+  sourceCommit: 3749d0c54be457a2a65d9a63c738a5dc88989ecd
+  inputHash: ce22f3fefb660c6e
 ---
 
 # Schema Design and GraphQL Tooling
@@ -19,13 +19,11 @@ Instructions for the agent to follow when designing Harper schemas, applying cor
 
 ## When to Use
 
-Apply this rule when creating or modifying Harper schema files, configuring `graphqlSchema` in `config.yaml`, or deciding which directives to apply to tables and fields. Use it any time a component needs tables, indexes, primary keys, or exported endpoints defined.
+Apply this rule when creating or modifying Harper schema files (`.graphql`), configuring `graphqlSchema` in `config.yaml`, or deciding which directives to apply to types and fields. Use it any time you need to define tables, primary keys, indexes, or exported endpoints.
 
 ## How It Works
 
-1. **Create a schema file** using standard GraphQL type definitions with Harper-specific directives. Name it (e.g., `schema.graphql`) and place it in your component directory.
-
-2. **Register the schema** in the component's `config.yaml` using the `graphqlSchema` plugin:
+1. **Declare the schema file** in the component's `config.yaml` using the `graphqlSchema` plugin:
 
    ```yaml
    graphqlSchema:
@@ -34,7 +32,7 @@ Apply this rule when creating or modifying Harper schema files, configuring `gra
 
    Both plugins and applications can specify schemas.
 
-3. **Mark types as tables** with `@table`. The type name becomes the table name by default:
+2. **Mark types as tables** with `@table`. The type name becomes the table name by default:
 
    ```graphql
    type Dog @table {
@@ -45,13 +43,13 @@ Apply this rule when creating or modifying Harper schema files, configuring `gra
    }
    ```
 
-4. **Designate a primary key** with `@primaryKey` on exactly one field per type. Primary keys must be unique; duplicate-key inserts are rejected. If no primary key is provided on insert, Harper auto-generates one:
+3. **Set a primary key** on every table using `@primaryKey`. Primary keys must be unique; duplicate-key inserts are rejected. If no primary key is provided on insert, Harper auto-generates one based on the field type:
    - `String` or `ID` → UUID string
    - `Int`, `Long`, or `Any` → auto-incrementing integer
 
    Use `Long` or `Any` for auto-generated numeric keys; `Int` is 32-bit and may be insufficient for large tables.
 
-5. **Add secondary indexes** with `@indexed` on any field that will be used for filtering in REST queries, SQL, or NoSQL operations:
+4. **Index fields for querying** with `@indexed`. Required for filtering by an attribute in REST queries, SQL, or NoSQL operations:
 
    ```graphql
    type Breed @table {
@@ -62,7 +60,7 @@ Apply this rule when creating or modifying Harper schema files, configuring `gra
 
    If the field value is an array, each element is individually indexed. Null values are indexed by default.
 
-6. **Expose a table as an external endpoint** with `@export`. The optional `name` parameter sets the URL path segment:
+5. **Expose a table as an external endpoint** with `@export`. Available via REST, MQTT, and other interfaces. The optional `name` parameter sets the URL path segment:
 
    ```graphql
    type MyTable @table @export(name: "my-table") {
@@ -70,9 +68,9 @@ Apply this rule when creating or modifying Harper schema files, configuring `gra
    }
    ```
 
-   Without `name`, the type name is used as the path segment.
+   Without `name`, the type name is used.
 
-7. **Configure `@table` arguments** as needed for database placement, expiration, eviction, and replication:
+6. **Configure `@table` arguments** as needed. All arguments are optional:
 
    | Argument       | Type      | Default                       | Description                                                   |
    | -------------- | --------- | ----------------------------- | ------------------------------------------------------------- |
@@ -83,13 +81,15 @@ Apply this rule when creating or modifying Harper schema files, configuring `gra
    | `scanInterval` | `Int`     | `(expiration + eviction) / 4` | Seconds between eviction scans                                |
    | `replicate`    | `Boolean` | `true`                        | Enable replication of this table                              |
 
-8. **Apply additional field directives** where needed:
-   - `@createdTime` — auto-assigns Unix epoch milliseconds on record creation
-   - `@updatedTime` — auto-assigns Unix epoch milliseconds on each update
-   - `@embed(source: "fieldName", model: "modelName")` — computes an embedding vector on write; attribute type must be `[Float]`
-   - `@hidden` — suppresses the field from MCP tool descriptors and OpenAPI documents (not an access-control mechanism)
+7. **Apply additional field directives** where needed:
+   - `@createdTime` — auto-assigns creation timestamp (Unix epoch ms)
+   - `@updatedTime` — auto-assigns update timestamp (Unix epoch ms)
+   - `@expiresAt` — marks a field as the record's absolute expiration time (Unix epoch ms)
+   - `@embed(source:, model:)` — computes an embedding vector when the source field is written; field type must be `[Float]`
+   - `@hidden` — suppresses the field from MCP tool descriptors and OpenAPI document (not an access-control mechanism)
+
+8. **Use `@sealed`** on a type to prevent records from including properties beyond those declared in the schema:
 
-9. **Restrict extra properties** with `@sealed` if records must not include properties beyond those declared:
    ```graphql
    type StrictRecord @table @sealed {
    	id: Long @primaryKey
@@ -97,9 +97,11 @@ Apply this rule when creating or modifying Harper schema files, configuring `gra
    }
    ```
 
+9. **Use unique database names** in plugins or applications to avoid table naming collisions, since all tables default to the `data` database.
+
 ## Examples
 
-**Minimal two-table schema:**
+**Minimal schema with two tables:**
 
 ```graphql
 type Dog @table {
@@ -115,7 +117,7 @@ type Breed @table {
 }
 ```
 
-**Table with expiration, eviction, and custom scan interval:**
+**Table with expiration, eviction, and scan tuning:**
 
 ```graphql
 # Expire after 5 minutes, evict after 1 hour, scan every 10 minutes
@@ -125,60 +127,56 @@ type WeatherCache @table(expiration: 300, eviction: 3300, scanInterval: 600) {
 }
 ```
 
-**Exported table with timestamps and a hidden field:**
+**Table with multiple `@table` arguments combined:**
 
 ```graphql
-type Customer @table @export(name: "customers") {
+type Event @table(database: "analytics", expiration: 86400) {
 	id: Long @primaryKey
 	name: String @indexed
-	createdAt: Long @createdTime
-	updatedAt: Long @updatedTime
-
-	"""
-	Internal — do not surface to external consumers.
-	"""
-	creditScore: Int @hidden
 }
 ```
 
-**Multiple `@table` argument combinations:**
+**Exported table with overridden table name:**
 
 ```graphql
-# Override table name
-type Product @table(table: "products") {
+type Product @table(table: "products") @export(name: "products") {
 	id: Long @primaryKey
+	category: String @indexed
+	price: Float @indexed
 }
+```
 
-# Place in a specific database
-type Order @table(database: "commerce") {
-	id: Long @primaryKey
-}
+**Table with timestamps and per-record expiration:**
 
-# Auto-expire records after 1 hour
-type Session @table(expiration: 3600) {
-	id: Long @primaryKey
-	userId: String
+```graphql
+type Session @table {
+	id: ID @primaryKey
+	token: String
+	createdAt: Long @createdTime
+	updatedAt: Long @updatedTime
+	expiresAt: Long @expiresAt
 }
+```
 
-# Disable replication
-type LocalRecord @table(replicate: false) {
-	id: Long @primaryKey
-	value: String
-}
+**Table with a hidden internal field:**
 
-# Combine multiple arguments
-type Event @table(database: "analytics", expiration: 86400) {
+```graphql
+type Customer @table {
 	id: Long @primaryKey
-	name: String @indexed
+	name: String
+
+	"""
+	Internal — do not surface to external consumers.
+	"""
+	creditScore: Int @hidden
 }
 ```
 
 ## Notes
 
-- All tables default to the `"data"` database. When designing plugins or applications, use unique database names to avoid table naming collisions.
-- Schemas are flexible by default — records may include properties beyond those declared. Use `@sealed` to prevent this.
-- `expiration` marks a record stale; `eviction` controls how long after expiration the record is physically removed. Eviction does not remove records from secondary indexes — Harper fetches the full record on demand if an evicted record matches a query.
-- `scanInterval` is clock-aligned to the server's local timezone, not startup-aligned. The eviction schedule is deterministic across restarts.
-- If replication is disabled on a table and later re-enabled, writes made during the disabled period are not replicated retroactively.
-- `@hidden` (on types or fields) is a metadata-visibility directive only. Use `attribute_permissions` on roles to enforce data access control.
-- Disabling replication (`replicate: false`) and re-enabling it later will not catch up on writes made while replication was disabled.
+- `@table`, `@export`, `@sealed`, and `@hidden` are type-level directives; `@primaryKey`, `@indexed`, `@embed`, `@createdTime`, `@updatedTime`, `@expiresAt`, and `@hidden` are field-level directives.
+- `eviction` removes non-indexed record data but does **not** remove a record from its secondary indexes. Indexes remain functional for evicted records; Harper fetches the full record on demand when a query matches an evicted entry.
+- `scanInterval` is clock-aligned to the server's local timezone, not startup-aligned. The server's startup time does not affect when eviction runs.
+- Replication is enabled by default. If you disable replication on a table and re-enable it later, it will not catch up on writes made while replication was disabled.
+- `@hidden` is a metadata-visibility directive only. Use `attribute_permissions` on roles to enforce data access control.
+- A full-record `put` that omits an `@expiresAt` field clears it; a `patch` of other fields preserves it.

package/agent/skills/harper-best-practices/rules/v5-upgrade.md

@@ -7,101 +7,137 @@ metadata:
   mode: generate
   sources:
     - release-notes/v5-lincoln/v5-migration.md
-  sourceCommit: de2aaf1c759a7ff5b93e862ba704153e2a392fcb
-  inputHash: 80314689f3a7f42e
+  sourceCommit: 3749d0c54be457a2a65d9a63c738a5dc88989ecd
+  inputHash: 5ee6407370219136
 ---
 
-# Upgrading a Harper Application to v5
+# v5 Upgrade: Breaking Changes and Migration Guide
 
-Instructions for the agent to follow when migrating an existing Harper application to version 5, addressing the breaking changes and adopting the recommended patterns.
+Instructions for the agent to apply when migrating a Harper application to v5, covering all breaking changes and required code updates.
 
 ## When to Use
 
-Apply this rule when upgrading a Harper application from v4.x (or earlier) to v5, or when code written against older APIs behaves differently after a v5 upgrade — for example a record that can no longer be mutated, a query that returns stale data, or a `spawn` call that now throws. Harper v5 introduces breaking changes; applications built on documented APIs need the updates below, while code relying on undocumented behavior or timing may need broader review.
+Apply this rule when upgrading an existing Harper application to v5, when encountering runtime errors related to renamed packages, changed APIs, or security restrictions introduced in v5, or when reviewing application code for v5 compatibility before deployment.
 
 ## How It Works
 
-1. **Adopt the `harper` package name**: HarperDB is now Harper, and the package was renamed. Install the open source edition with `npm i -g harper` and the pro edition with `npm i -g @harperfast/harper-pro`. Update application imports from `harperdb` to `harper`:
+1. **Update the package import from `harperdb` to `harper`**: All application code must import from `harper` instead of `harperdb`.
 
    ```javascript
    import { tables } from 'harper';
    ```
 
-2. **Opt in to install scripts when required**: Harper now installs packages with `--ignore-scripts` to guard against accidental script execution, a common security risk. If an application genuinely needs install scripts to run (for example to build native binaries), pass the `allowInstallScripts` option when deploying.
+2. **Enable `allowInstallScripts` if packages require install scripts**: Harper v5 uses `--ignore-scripts` by default when installing packages. If your application requires installation scripts (e.g., to install additional binaries), set the `allowInstallScripts` option when deploying.
 
-3. **Update `Table.get` return-value handling**: `Table.get` now returns a plain, frozen record object rather than an instance of the table class, so instance methods are no longer available on the result. The commonly used `wasLoadedFromSource()` method is gone; cache-vs-origin information now lives on the request `target` as `target.loadedFromSource`. Pass a `RequestTarget` to `Table.get` and read the flag from it. Because the record is frozen, create a copy instead of mutating it in place.
+3. **Update `Table.get` usage — return value is now a frozen record object**: `Table.get` now returns a plain record object, not a table class instance. The record is frozen; you cannot mutate it directly.
+   - Replace `wasLoadedFromSource()` with `target.loadedFromSource`:
 
-4. **Account for automatic transaction context**: With RocksDB, transactions are fully supported by the storage engine, and Harper v5 uses asynchronous context tracking to carry the current transaction across calls automatically. A nested `Table.get` now reads within the current transaction's snapshot, so it will not observe newly written data until you commit. Access the context through `getContext()` and either commit the current transaction (`getContext().transaction.commit()`) or run the read inside a fresh `transaction(...)` to see the latest data. This matters most for code executing outside the context of a Harper request.
+     ```javascript
+     // Old — remove this pattern:
+     const record = await Table.get(id);
+     if (record.wasLoadedFromSource()) { ... }
 
-5. **Register spawnable commands**: Spawning processes is now tightly controlled. `spawn`, `exec`, and `execFile` may only run executables listed in the `applications.allowedSpawnCommands` configuration, and each call must pass a `name` in its `options` so that only a single named process is started across Harper's multiple threads. Use a distinct `name` when you deliberately need a separate process.
+     // New — use loadedFromSource on the target:
+     const target = new RequestTarget();
+     target.id = id;
+     const record = await Table.get(target);
+     if (target.loadedFromSource) {
+       // record was loaded from origin (not cache)
+     }
+     ```
 
-6. **Return Response-like objects to set headers**: If a REST method returns an object with a `headers` property, Harper uses it as the response headers.
+   - Replace in-place mutation with object spread, since records are frozen:
 
-7. **Replace `blob.save()`**: The `blob.save()` method has been removed. Pass the `saveBeforeCommit` flag in the options to the `Blob` constructor instead.
+     ```javascript
+     // Old — throws in v5:
+     const record = await Table.get(id);
+     record.property = 'changed';
 
-8. **Configure the VM module loader**: v5 loads application modules through Node.js's VM module API, giving each application its own module cache and an application-scoped `harper` module. All of this is controlled by the `applications` section of `harperdb-config.yaml`:
-   - `moduleLoader` — `vm-current-context` (default), `vm`, `native`, or `compartment`. The default shares intrinsics with Harper for best compatibility; choose `vm` only if you need per-application intrinsics, and `native` to disable the VM loader entirely (restores pre-v5 loading, but per-app `logger`/`config` context is unavailable).
-   - `lockdown` — `freeze-after-load` (default), `freeze`, `ses`, or `none`. Freezing intrinsics prevents prototype-pollution attacks; set `lockdown: none` only as a temporary workaround if a dependency mutates built-in prototypes.
-   - `allowedDirectory` — `app` (default) restricts module loading to the application's own directory tree; set `any` if the app must load files from outside it in production.
-   - `allowedBuiltinModules` — an optional allowlist restricting which Node.js built-ins the application may import (all are allowed if omitted).
-   - `dependencyLoader` — `auto` (default), `app`, or `native`, controlling how npm dependencies are loaded through or around the VM loader.
+     // New — copy instead of mutate:
+     let record = await Table.get(id);
+     record = { ...record, property: 'changed' };
+     ```
 
-## Examples
+   - `getUpdatedTime` and `getExpiresAt` methods remain available on the record object.
 
-**Updating `Table.get` and its cache check:**
+4. **Update transaction and context handling**: Harper v5 uses asynchronous context tracking. `Table.get` and other table calls now automatically inherit the current transaction context. Code that previously omitted context to bypass a transaction will no longer work as expected. Use `getContext` (imported from `harper`) to access and commit the current transaction explicitly when you need to see updated data.
 
-```javascript
-// Old (v4.x)
-const record = await Table.get(id);
-if (record.wasLoadedFromSource()) {
-	// loaded from origin, not cache
-}
-
-// New (v5)
-const target = new RequestTarget(); // passed in when overriding `get`
-target.id = id;
-const record = await Table.get(target);
-if (target.loadedFromSource) {
-	// loaded from origin, not cache
-}
-```
+   ```javascript
+   import { setTimeout as delay } from 'node:timers/promises';
+   import { getContext, transaction } from 'harper';
+
+   class MyResource {
+   	static async get(target) {
+   		// Commit the current transaction to read latest data:
+   		await getContext().transaction.commit();
+   		// Optionally wrap each poll in a new transaction:
+   		while ((await transaction(() => Table.get(target))).status !== 'ready') {
+   			await delay(100);
+   		}
+   		return Table.get(target);
+   	}
+   }
+   ```
 
-**Committing the transaction to read fresh data:**
+5. **Register allowed spawn commands via `allowedSpawnCommands`**: Any use of `spawn`, `exec`, or `execFile` from `node:child_process` must reference executables listed in `applications.allowedSpawnCommands` in `harperdb-config.yaml`. Provide a `name` property in the options argument to ensure only a single named process is started across threads.
 
-```javascript
-import { setTimeout as delay } from 'node:timers/promises';
-import { getContext, transaction } from 'harper';
-
-class MyResource {
-	static async get(target) {
-		// The current transaction is a consistent snapshot; commit it to see updates.
-		await getContext().transaction.commit();
-		while ((await transaction(() => Table.get(target))).status !== 'ready') {
-			delay(100);
-		}
-		return Table.get(target);
-	}
-}
-```
+6. **Replace `blob.save()` with `saveBeforeCommit`**: The `blob.save()` method has been removed. Use the `saveBeforeCommit` flag in the options passed to the `Blob` constructor instead.
 
-**Module loader and security configuration in `harperdb-config.yaml`:**
+7. **Configure the `moduleLoader` and `lockdown` settings**: Harper v5 loads application modules through Node.js's VM module API. Control this behavior in `harperdb-config.yaml` under the `applications` key.
+
+   | Setting            | Default              | Options                                             |
+   | ------------------ | -------------------- | --------------------------------------------------- |
+   | `moduleLoader`     | `vm-current-context` | `vm-current-context`, `vm`, `native`, `compartment` |
+   | `lockdown`         | `freeze-after-load`  | `freeze-after-load`, `freeze`, `ses`, `none`        |
+   | `dependencyLoader` | `auto`               | `auto`, `app`, `native`                             |
+   | `allowedDirectory` | `app`                | `app`, `any`                                        |
+   - Use `moduleLoader: native` to disable the VM loader entirely and restore pre-v5 behavior (application-specific context such as tagged logging and per-app `config` will not be available).
+   - Use `lockdown: none` as a temporary workaround if a dependency modifies intrinsic prototypes at runtime and throws a `TypeError`.
+
+## Examples
+
+### Full `harperdb-config.yaml` `applications` block
 
 ```yaml
 applications:
-  lockdown: freeze-after-load # default
-  moduleLoader: vm-current-context # vm-current-context (default) | vm | native | compartment
-  dependencyLoader: auto # auto (default) | app | native
-  allowedDirectory: app # app (default) | any
+  lockdown: freeze-after-load
+  moduleLoader: vm-current-context
+  dependencyLoader: auto
+  allowedDirectory: app
   allowedSpawnCommands:
     - npm
     - node
-  # allowedBuiltinModules: [] # if omitted, all Node.js built-ins are allowed
 ```
 
-## Recommended Changes
+### Restricting allowed built-in modules
+
+```yaml
+applications:
+  allowedBuiltinModules:
+    - fs
+    - path
+    - http
+```
+
+### Disabling the VM loader for package compatibility
+
+```yaml
+applications:
+  moduleLoader: native
+```
+
+### Accessing context and committing a transaction
+
+```javascript
+import { getContext } from 'harper';
+
+await getContext().transaction.commit();
+```
 
-Beyond the required fixes above, Harper v5 encourages these patterns for new and migrated code:
+## Notes
 
-- Implement endpoints with `static` methods on Resources/Tables, reading request information from the request `target` argument (or from `getContext()`).
-- Rely on automatic context propagation rather than threading context through every call manually; access it via `getContext()` exported from `harper`.
-- Access Harper functions and APIs through the `harper` package rather than through global variables.
+- The `lockdown: freeze-after-load` default freezes `Object`, `Array`, `Promise`, `Map`, `Set`, and other intrinsics after all application code loads. Any code or dependency that mutates intrinsic prototypes after startup will throw a `TypeError`.
+- Under `lockdown: ses`, the constrained `fetch` (https-only) is applied only in `vm` mode. In `vm-current-context` and `native` modes, application code uses the standard global `fetch`.
+- In production, `allowedDirectory: app` prevents loading modules from outside the application's own directory tree. Set `allowedDirectory: any` only if your application legitimately requires it.
+- `dependencyLoader: native` is a narrower alternative to `moduleLoader: native` — it uses native loading only for npm packages while keeping the VM loader for first-party application source files.
+- Access Harper functions and APIs through the `harper` package rather than through global variables. Use `getContext` from `harper` to access request context without passing it explicitly through every call.