@harperfast/template-react-ts-studio 1.12.8 → 1.12.10

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (26) hide show
  1. package/.agents/skills/harper-best-practices/AGENTS.md +514 -445
  2. package/.agents/skills/harper-best-practices/rules/automatic-apis.md +96 -93
  3. package/.agents/skills/harper-best-practices/rules/deploying-to-harper-fabric.md +84 -58
  4. package/.agents/skills/harper-best-practices/rules/extending-tables.md +27 -35
  5. package/.agents/skills/harper-best-practices/rules/handling-binary-data.md +40 -39
  6. package/.agents/skills/harper-best-practices/rules/querying-rest-apis.md +93 -37
  7. package/.agents/skills/harper-best-practices/rules/schema-design-tooling.md +68 -66
  8. package/.agents/skills/harper-best-practices/rules/using-blob-datatype.md +24 -28
  9. package/.agents/skills/harper-best-practices/rules/v5-upgrade.md +98 -105
  10. package/.agents/skills/harper-best-practices/rules.manifest.yaml +28 -3
  11. package/.agents/skills/harper-mcp/AGENTS.md +138 -26
  12. package/.agents/skills/harper-mcp/rules/enabling-mcp.md +140 -28
  13. package/agent/skills/harper-best-practices/AGENTS.md +514 -445
  14. package/agent/skills/harper-best-practices/rules/automatic-apis.md +96 -93
  15. package/agent/skills/harper-best-practices/rules/deploying-to-harper-fabric.md +84 -58
  16. package/agent/skills/harper-best-practices/rules/extending-tables.md +27 -35
  17. package/agent/skills/harper-best-practices/rules/handling-binary-data.md +40 -39
  18. package/agent/skills/harper-best-practices/rules/querying-rest-apis.md +93 -37
  19. package/agent/skills/harper-best-practices/rules/schema-design-tooling.md +68 -66
  20. package/agent/skills/harper-best-practices/rules/using-blob-datatype.md +24 -28
  21. package/agent/skills/harper-best-practices/rules/v5-upgrade.md +98 -105
  22. package/agent/skills/harper-best-practices/rules.manifest.yaml +28 -3
  23. package/agent/skills/harper-mcp/AGENTS.md +138 -26
  24. package/agent/skills/harper-mcp/rules/enabling-mcp.md +140 -28
  25. package/package.json +1 -1
  26. package/skills-lock.json +2 -2
@@ -40,17 +40,26 @@ type ExamplePerson @table @export {
40
40
  }
41
41
  ```
42
42
 
43
- ### 1.2 Schema Design and GraphQL Tooling
43
+ ### 1.2 Schema Design Tooling
44
44
 
45
- Instructions for the agent to follow when designing Harper database schemas using GraphQL type definitions, core directives, and tooling configuration.
45
+ Instructions for the agent to follow when designing Harper schemas, applying core directives, and configuring GraphQL tooling.
46
46
 
47
47
  #### When to Use
48
48
 
49
- Apply this rule when creating or modifying Harper schema files (`.graphql`), configuring schema loading in `config.yaml`, or deciding which directives to apply to tables and fields. Use it whenever a task involves defining tables, primary keys, indexes, or export behavior.
49
+ Apply this rule when creating or modifying Harper schema files (`.graphql`), configuring `graphqlSchema` in `config.yaml`, or deciding which directives to apply to tables and fields. Use it whenever a task involves defining tables, primary keys, indexes, or export behavior.
50
50
 
51
51
  #### How It Works
52
52
 
53
- 1. **Create a GraphQL schema file** with Harper-specific directives. Schemas ensure required tables exist on deployment, enforce types and constraints, control indexing, and define relationships.
53
+ 1. **Register the schema file** in the component's `config.yaml` using the `graphqlSchema` plugin key:
54
+
55
+ ```yaml
56
+ graphqlSchema:
57
+ files: 'schema.graphql'
58
+ ```
59
+
60
+ Both plugins and applications can specify schemas.
61
+
62
+ 2. **Mark types as tables** using `@table`. The type name becomes the table name by default:
54
63
 
55
64
  ```graphql
56
65
  type Dog @table {
@@ -61,79 +70,71 @@ Apply this rule when creating or modifying Harper schema files (`.graphql`), con
61
70
  }
62
71
  ```
63
72
 
64
- 2. **Register the schema in `config.yaml`** using the `graphqlSchema` plugin:
73
+ 3. **Designate a primary key** on every table using `@primaryKey`. Primary keys must be unique; duplicate inserts are rejected. If no primary key is provided on insert, Harper auto-generates one:
74
+ - `String` or `ID` → UUID string
75
+ - `Int`, `Long`, or `Any` → auto-incrementing integer
65
76
 
66
- ```yaml
67
- graphqlSchema:
68
- files: 'schema.graphql'
69
- ```
70
-
71
- Both plugins and applications can specify schemas.
77
+ Use `Long` or `Any` for auto-generated numeric keys; `Int` is 32-bit and may be insufficient for large tables.
72
78
 
73
- 3. **Mark each type as a table** with `@table`. The type name becomes the table name by default.
79
+ 4. **Index fields for querying** using `@indexed`. Required for filtering by an attribute in REST queries, SQL, or NoSQL operations:
74
80
 
75
81
  ```graphql
76
- type MyTable @table {
82
+ type Breed @table {
77
83
  id: Long @primaryKey
84
+ name: String @indexed
78
85
  }
79
86
  ```
80
87
 
81
- Key `@table` arguments:
82
-
83
- | Argument | Type | Default | Description |
84
- | -------------------- | --------- | ----------------------------- | --------------------------------------------------------------- |
85
- | `table` | `String` | type name | Override the table name |
86
- | `database` | `String` | `"data"` | Database to place the table in |
87
- | `expiration` | `Int` | — | Seconds until a record goes stale |
88
- | `eviction` | `Int` | `0` | Additional seconds after `expiration` before physical removal |
89
- | `scanInterval` | `Int` | `(expiration + eviction) / 4` | Seconds between eviction scans |
90
- | `replicate` | `Boolean` | `true` | Enable replication of this table |
91
- | `cacheControl` | `String` | — | `Cache-Control` header for anonymous GET/HEAD 200/304 responses |
92
- | `randomAccessFields` | `Boolean` | `storage.randomAccessFields` | Pin this table's record encoding |
93
-
94
- 4. **Designate a primary key** on every table using `@primaryKey`. Primary keys must be unique; duplicate inserts are rejected. If no primary key is provided on insert, Harper auto-generates one:
95
- - **UUID string** — when type is `String` or `ID`
96
- - **Auto-incrementing integer** — when type is `Int`, `Long`, or `Any`
88
+ If the field value is an array, each element is individually indexed. Null values are indexed by default.
97
89
 
98
- Use `Long` or `Any` for auto-generated numeric keys; `Int` is 32-bit and may be insufficient for large tables.
90
+ 5. **Expose tables as REST/MQTT endpoints** using `@export`. The optional `name` parameter sets the URL path segment; without it, the type name is used:
99
91
 
100
92
  ```graphql
101
- type Product @table {
93
+ type MyTable @table @export(name: "my-table") {
102
94
  id: Long @primaryKey
103
- name: String
104
95
  }
105
96
  ```
106
97
 
107
- 5. **Add secondary indexes** with `@indexed` on any attribute that will be used for filtering in REST queries, SQL, or NoSQL operations. If the field value is an array, each element is individually indexed.
98
+ `@export` alone does not serve HTTP traffic — REST must also be enabled for the application via `rest: true` in `config.yaml` or Harper's built-in default. `@export` is a routing directive, not access control; omitting it returns 404 but does not protect the data.
99
+
100
+ 6. **Configure `@table` arguments** to control database placement, expiration, replication, and caching behavior. Key arguments:
101
+
102
+ | Argument | Type | Default | Description |
103
+ | -------------------- | --------- | ----------------------------- | -------------------------------------------------------------- |
104
+ | `table` | `String` | type name | Override the table name |
105
+ | `database` | `String` | `"data"` | Database to place the table in |
106
+ | `expiration` | `Int` | — | Seconds until a record goes stale |
107
+ | `eviction` | `Int` | `0` | Additional seconds after `expiration` before physical removal |
108
+ | `scanInterval` | `Int` | `(expiration + eviction) / 4` | Seconds between eviction scans |
109
+ | `replicate` | `Boolean` | `true` | Enable replication of this table |
110
+ | `cacheControl` | `String` | — | `Cache-Control` header on anonymous GET/HEAD 200/304 responses |
111
+ | `randomAccessFields` | `Boolean` | `storage.randomAccessFields` | Pin this table's record encoding |
112
+
113
+ 7. **Seal a type** with `@sealed` to prevent records from including properties beyond those declared:
108
114
 
109
115
  ```graphql
110
- type Product @table {
116
+ type StrictRecord @table @sealed {
111
117
  id: Long @primaryKey
112
- category: String @indexed
113
- price: Float @indexed
118
+ name: String
114
119
  }
115
120
  ```
116
121
 
117
- 6. **Expose tables via REST and other interfaces** using `@export`. Without `@export`, the table has no REST/MQTT route (callers get 404). The optional `name` parameter sets the URL path segment.
122
+ 8. **Use timestamp directives** for automatic record lifecycle tracking:
123
+ - `@createdTime` — assigns Unix epoch milliseconds on record creation
124
+ - `@updatedTime` — assigns Unix epoch milliseconds on each update
125
+ - `@expiresAt` — marks a field as the record's absolute expiration time (Unix epoch milliseconds); authoritative over the table-level `expiration` default
126
+
127
+ 9. **Use `@embed`** to automatically compute an embedding vector for an attribute whenever the source field is written (requires `source` and `model` arguments; field type must be `[Float]`):
118
128
 
119
129
  ```graphql
120
- type MyTable @table @export(name: "my-table") {
130
+ type Document @table {
121
131
  id: Long @primaryKey
132
+ text: String
133
+ embedding: [Float] @embed(source: "text", model: "default")
122
134
  }
123
135
  ```
124
136
 
125
- `@export` is a routing directive, not access control. The table remains accessible through the Operations API and SQL regardless. REST must also be enabled for the application (via `rest: true` in `config.yaml` or Harper's built-in default).
126
-
127
- 7. **Apply additional type directives** as needed:
128
- - `@sealed` — prevents records from including properties beyond those declared in the schema.
129
- - `@hidden` — suppresses the type from MCP tool descriptors and the OpenAPI document. Does not restrict data access.
130
-
131
- 8. **Apply field directives** for computed and lifecycle behavior:
132
- - `@createdTime` — assigns Unix epoch milliseconds on record creation.
133
- - `@updatedTime` — assigns Unix epoch milliseconds on each update.
134
- - `@expiresAt` — marks a field as the record's absolute expiration time (Unix epoch milliseconds); authoritative over the table-level `expiration` default.
135
- - `@embed` — computes an embedding vector when the source field is written (requires `source` and `model` arguments; field type must be `[Float]`).
136
- - `@hidden` (field) — suppresses the field from generated specs and MCP tool schemas; does not restrict data access.
137
+ 10. **Use `@hidden`** on types or fields to suppress them from MCP tool descriptors and the OpenAPI document. This is a metadata-visibility directive only — it does not restrict data access. Use `attribute_permissions` on roles for field-level access control.
137
138
 
138
139
  #### Examples
139
140
 
@@ -163,7 +164,7 @@ type WeatherCache @table(expiration: 300, eviction: 3300, scanInterval: 600) {
163
164
  }
164
165
  ```
165
166
 
166
- **Exported table with cache control:**
167
+ **Exported table with public cache control:**
167
168
 
168
169
  ```graphql
169
170
  type Product @table(cacheControl: "public, max-age=60") @export {
@@ -173,14 +174,12 @@ type Product @table(cacheControl: "public, max-age=60") @export {
173
174
  }
174
175
  ```
175
176
 
176
- **Table with lifecycle fields and indexing:**
177
+ **Table with multiple `@table` arguments combined:**
177
178
 
178
179
  ```graphql
179
180
  type Event @table(database: "analytics", expiration: 86400) {
180
181
  id: Long @primaryKey
181
182
  name: String @indexed
182
- createdAt: Long @createdTime
183
- updatedAt: Long @updatedTime
184
183
  }
185
184
  ```
186
185
 
@@ -194,12 +193,15 @@ type Session @table {
194
193
  }
195
194
  ```
196
195
 
197
- **Sealed table preventing extra properties:**
196
+ **Table with timestamp tracking and indexed fields:**
198
197
 
199
198
  ```graphql
200
- type StrictRecord @table @sealed {
199
+ type Order @table(database: "commerce") @export {
201
200
  id: Long @primaryKey
202
- name: String
201
+ userId: String @indexed
202
+ status: String @indexed
203
+ createdAt: Long @createdTime
204
+ updatedAt: Long @updatedTime
203
205
  }
204
206
  ```
205
207
 
@@ -212,13 +214,13 @@ graphqlSchema:
212
214
 
213
215
  #### Notes
214
216
 
215
- - Schemas are flexible by default — records may include additional properties beyond those declared. Use `@sealed` to prevent this.
216
- - Use unique `database` names in plugins or applications to avoid table naming collisions, since all tables default to the `"data"` database.
217
- - Replication is enabled by default. If you disable replication and re-enable it later, the table will not catch up on writes made while replication was disabled.
218
- - `@hidden` (type or field) is a metadata-visibility directive only. Use table-level role permissions and `attribute_permissions` whitelists to restrict actual data access.
219
- - `@export` absence causes 404 on REST/MQTT routes but does not protect data from the Operations API or SQL.
220
- - The `cacheControl` argument emits headers only on anonymous (unauthenticated) GET/HEAD 200/304 responses. Authenticated responses receive `Cache-Control: private, no-cache`.
221
- - `randomAccessFields` on `@table` pins the record encoding at table creation time. Editing the argument later does not repin an existing table.
217
+ - Use unique `database` names in plugins and applications to avoid table naming collisions, since all tables default to the `"data"` database.
218
+ - Disabling replication (`replicate: false`) and re-enabling it later will not catch up on writes made while replication was disabled.
219
+ - `cacheControl` emits the header only on anonymous (unauthenticated) GET/HEAD `200`/`304` responses. Authenticated responses receive `Cache-Control: private, no-cache` regardless of the declaration. The header is never emitted on `401` responses.
220
+ - `@expiresAt` requires an absolute Unix epoch millisecond timestamp, not a duration. Negative values are ignored and fall back to the table default. A full-record `put` that omits the field clears it; a `patch` preserves it.
221
+ - Eviction removes non-indexed record data but does not remove a record from its secondary indexes; indexes remain functional for evicted records, with full records fetched on demand.
222
+ - `scanInterval` is clock-aligned to the server's local timezone, not startup-aligned — the server's startup time does not affect when eviction runs.
223
+ - `randomAccessFields` pins the table's encoding at creation time. Editing the argument later does not repin an existing table. Omit it to follow the global `storage.randomAccessFields` setting.
222
224
 
223
225
  ### 1.3 Defining Relationships Between Tables in Harper
224
226
 
@@ -549,11 +551,11 @@ Instructions for the agent to follow when storing and retrieving large binary co
549
551
 
550
552
  #### When to Use
551
553
 
552
- Apply this rule when a schema field needs to store large binary content such as images, video, audio, or large HTML — typically content larger than 20KB. Use `Blob` instead of `Bytes` when streaming support and out-of-record storage are required. See [handling-binary-data.md](handling-binary-data.md) for broader binary data guidance.
554
+ Apply this rule when a schema field needs to store large binary content such as images, video, audio, or large HTML — typically content larger than 20KB. Use `Blob` instead of `Bytes` when streaming support and out-of-record storage are required. See [handling-binary-data](handling-binary-data.md) for broader binary data guidance.
553
555
 
554
556
  #### How It Works
555
557
 
556
- 1. **Declare a `Blob` field in your schema**: Add a field typed as `Blob` to your `@table` type.
558
+ 1. **Declare a `Blob` field in your schema**: Add a field typed as `Blob` to your table definition.
557
559
 
558
560
  ```graphql
559
561
  type MyTable @table {
@@ -562,14 +564,14 @@ Apply this rule when a schema field needs to store large binary content such as
562
564
  }
563
565
  ```
564
566
 
565
- 2. **Create and store a blob with `createBlob()`**: Pass a buffer or stream to `createBlob()`, then `put` the record.
567
+ 2. **Create and store a blob with `createBlob()`**: Pass a buffer, string, or stream to `createBlob()`, then `put` the record.
566
568
 
567
569
  ```javascript
568
570
  let blob = createBlob(largeBuffer);
569
571
  await MyTable.put({ id: 'my-record', data: blob });
570
572
  ```
571
573
 
572
- 3. **Retrieve blob data using standard Web API methods**: The `Blob` type implements the Web API `Blob` interface. Use `.bytes()`, `.text()`, `.arrayBuffer()`, `.stream()`, or `.slice()` as needed.
574
+ 3. **Retrieve blob data using standard Web API methods**: Use `.bytes()`, `.text()`, or `.stream()` on the retrieved field.
573
575
 
574
576
  ```javascript
575
577
  let record = await MyTable.get('my-record');
@@ -578,7 +580,7 @@ Apply this rule when a schema field needs to store large binary content such as
578
580
  let stream = record.data.stream(); // ReadableStream
579
581
  ```
580
582
 
581
- 4. **Use `saveBeforeCommit` when full write must precede commit**: By default, `Blob` is not ACID-compliant — a record can reference a blob before it is fully written. Set `saveBeforeCommit: true` to block the transaction until the blob is fully saved.
583
+ 4. **Use `saveBeforeCommit` when full write must complete before commit**: By default, blobs are not ACID-compliant — a record can reference a blob before it is fully written. Set `saveBeforeCommit: true` to block the transaction until the blob is fully saved.
582
584
 
583
585
  ```javascript
584
586
  let blob = createBlob(stream, { saveBeforeCommit: true });
@@ -586,13 +588,14 @@ Apply this rule when a schema field needs to store large binary content such as
586
588
  // put() resolves only after blob is fully written and record is committed
587
589
  ```
588
590
 
589
- 5. **Register an error handler when returning a blob via REST**: Interrupted streams must be handled explicitly.
591
+ 5. **Register an error handler when returning a blob via REST**: Handle interrupted streams to avoid stale references.
590
592
 
591
593
  ```javascript
592
594
  export class MyEndpoint extends MyTable {
593
595
  static async get(target) {
594
- const record = super.get(target);
595
- let blob = record.data;
596
+ const record = await super.get(target);
597
+ let blob = record?.data;
598
+ if (!blob) return record;
596
599
  blob.on('error', () => {
597
600
  MyTable.invalidate(target);
598
601
  });
@@ -601,19 +604,15 @@ Apply this rule when a schema field needs to store large binary content such as
601
604
  }
602
605
  ```
603
606
 
604
- 6. **Rely on automatic coercion where applicable**: When a field is typed as `Blob` in the schema, any string or buffer assigned via `put`, `patch`, or `publish` is automatically coerced to a `Blob` — no manual `createBlob()` call is needed in those cases.
605
-
606
- ##### `BlobOptions` reference
607
-
608
- Pass an options object as the second argument to `createBlob()`.
607
+ 6. **Pass `BlobOptions` to control storage behavior**: `createBlob()` accepts an options object as its second argument.
609
608
 
610
- | Option | Type | Default | Description |
611
- | ------------------ | --------- | ----------- | ------------------------------------------------------------------------------------------------------------------------ |
612
- | `type` | `string` | `undefined` | MIME type to associate with the blob (e.g., `image/jpeg`). Readable via `blob.type` and used when serving HTTP. |
613
- | `size` | `number` | `undefined` | Size of the data in bytes, if known ahead of time. Otherwise inferred from a buffer or determined as a stream completes. |
614
- | `saveBeforeCommit` | `boolean` | `false` | Wait until the blob is fully written before the transaction commits. |
615
- | `compress` | `boolean` | `false` | Compress the stored data with deflate. |
616
- | `flush` | `boolean` | `false` | Flush the file to disk after writing, before the `createBlob` promise chain resolves. |
609
+ | Option | Type | Default | Description |
610
+ | ------------------ | --------- | ----------- | ----------------------------------------------------------------------------------- |
611
+ | `type` | `string` | `undefined` | MIME type (e.g., `image/jpeg`). Used when serving HTTP and readable via `blob.type` |
612
+ | `size` | `number` | `undefined` | Size in bytes if known ahead of time; otherwise inferred |
613
+ | `saveBeforeCommit` | `boolean` | `false` | Wait until blob is fully written before the transaction commits |
614
+ | `compress` | `boolean` | `false` | Compress stored data with deflate |
615
+ | `flush` | `boolean` | `false` | Flush file to disk after writing, before the `createBlob` promise chain resolves |
617
616
 
618
617
  #### Examples
619
618
 
@@ -628,15 +627,13 @@ await Photo.put({ id, data: blob });
628
627
 
629
628
  ```javascript
630
629
  let blob = createBlob(incomingStream);
631
- // blob exists, but data is still streaming to storage
632
630
  await MyTable.put({ id: 'my-record', data: blob });
633
631
 
634
632
  let record = await MyTable.get('my-record');
635
- // blob data is accessible as it arrives
636
633
  let outgoingStream = record.data.stream();
637
634
  ```
638
635
 
639
- **Guarantee full write before commit using `saveBeforeCommit`:**
636
+ **Wait for full write before commit:**
640
637
 
641
638
  ```javascript
642
639
  let blob = createBlob(stream, { saveBeforeCommit: true });
@@ -645,9 +642,10 @@ await MyTable.put({ id: 'my-record', data: blob });
645
642
 
646
643
  #### Notes
647
644
 
648
- - `Blob` stores data separately from the record. If you need the binary data to be a true, ACID-committed part of the record, use a `Bytes` field instead.
649
- - All standard Web API `Blob` methods — `.text()`, `.arrayBuffer()`, `.stream()`, `.slice()`, and `.bytes()` — are available on retrieved blob fields.
650
- - Without `saveBeforeCommit: true`, blobs are **not** ACID-compliant by default; a record can reference a blob before it is fully written to storage.
645
+ - `Blob` implements the Web API `Blob` interface. All standard methods — `.text()`, `.arrayBuffer()`, `.stream()`, `.slice()`, and `.bytes()` — are available on retrieved blob fields.
646
+ - Blobs are stored **separately from the record**, not within it. If you need the binary data to be a true ACID-committed part of the record, use a `Bytes` field instead.
647
+ - Any string or buffer assigned to a `Blob`-typed field in a `put`, `patch`, or `publish` is automatically coerced to a `Blob` — manual `createBlob()` calls are not required in those cases.
648
+ - Use `saveBeforeCommit: true` whenever downstream consumers must not read a partially written blob.
651
649
 
652
650
  ### 1.6 Handling Binary Data
653
651
 
@@ -655,28 +653,30 @@ Instructions for the agent to follow when storing and serving binary data (image
655
653
 
656
654
  #### When to Use
657
655
 
658
- Apply this rule when a Harper resource needs to accept, store, or serve binary payloads such as images, audio files, or calendar data. Use it when REST clients send `base64`-encoded data inside JSON, when raw binary is uploaded via `PUT`/`POST`, or when a resource must stream binary back to the client with the correct `Content-Type`.
656
+ Apply this rule when a Harper resource needs to accept, store, or serve binary payloads such as images, audio files, or calendar data. Use it when clients send raw binary via `PUT`/`POST`, or when they send `base64`-encoded data inside a JSON body.
659
657
 
660
658
  #### How It Works
661
659
 
662
- 1. **Accept base64-encoded binary from JSON clients**: Decode the incoming `base64` string with `Buffer.from` and wrap it using `createBlob`, recording the MIME type. Override `post` in your resource class:
660
+ 1. **Accept base64-encoded binary from JSON clients**: REST clients that cannot post raw binary send `base64` inside a JSON body. In the resource override, `await` the `record` promise before reading any fields — reading fields off the unresolved promise silently yields `undefined` and stores the raw base64 string. Decode the field with `Buffer.from` and wrap it with `createBlob`, recording the MIME type.
663
661
 
664
662
  ```typescript
665
663
  import { type RequestTargetOrId, tables, createBlob } from 'harper';
666
664
 
667
665
  export class Photo extends tables.Photo {
668
666
  static async post(target: RequestTargetOrId, record: any) {
669
- if (record.data) {
670
- record.data = createBlob(Buffer.from(record.data, record.encoding || 'base64'), {
671
- type: record.contentType || 'application/octet-stream',
667
+ const body = await record;
668
+ if (!body) return new Response('A JSON body is required', { status: 400 });
669
+ if (body.data) {
670
+ body.data = createBlob(Buffer.from(body.data, body.encoding || 'base64'), {
671
+ type: body.contentType || 'application/octet-stream',
672
672
  });
673
673
  }
674
- return super.post(target, record);
674
+ return super.post(target, body);
675
675
  }
676
676
  }
677
677
  ```
678
678
 
679
- 2. **Serve binary from a resource**: Override `get` to return a response object with the blob's MIME type in the `Content-Type` header and the blob as the body. Harper streams it to the client:
679
+ 2. **Serve binary from a resource**: Override `get` to return a response object with the blob's MIME type in the `Content-Type` header and the blob as the body. Harper streams it to the client.
680
680
 
681
681
  ```typescript
682
682
  export class Photo extends tables.Photo {
@@ -694,28 +694,11 @@ Apply this rule when a Harper resource needs to accept, store, or serve binary p
694
694
  }
695
695
  ```
696
696
 
697
- 3. **Upload raw binary with a non-standard content type**: Make a `PUT` or `POST` with any non-standard `Content-Type` header. Harper automatically stores the body as a record with `contentType` and `data` properties:
698
-
699
- ```http
700
- PUT /my-resource/33
701
- Content-Type: text/calendar
702
-
703
- BEGIN:VCALENDAR
704
- VERSION:2.0
705
- ...
706
- ```
697
+ 3. **Store arbitrary content types via raw PUT/POST**: When a `PUT` or `POST` arrives with a non-standard `Content-Type` (e.g., `text/calendar`, `image/gif`), Harper automatically stores the content as a record with `contentType` and `data` properties. Retrieving that record returns the response with the original `Content-Type` and body. If the content type is not from the `text` family, the data is treated as binary (a Node.js `Buffer`).
707
698
 
708
- Harper stores this as:
699
+ Use `application/octet-stream` for generic binary data, or target a specific property:
709
700
 
710
- ```json
711
- { "contentType": "text/calendar", "data": "BEGIN:VCALENDAR\nVERSION:2.0\n..." }
712
701
  ```
713
-
714
- Retrieving that record returns the response with the stored `Content-Type` and body. If the content type is not from the `text` family, the data is treated as binary (a Node.js `Buffer`).
715
-
716
- 4. **Upload binary to a specific property**: Use `application/octet-stream` (or any image/binary MIME type) and target a sub-path to store binary directly on a property:
717
-
718
- ```http
719
702
  PUT /my-resource/33/image
720
703
  Content-Type: image/gif
721
704
 
@@ -724,23 +707,40 @@ Apply this rule when a Harper resource needs to accept, store, or serve binary p
724
707
 
725
708
  #### Examples
726
709
 
727
- **End-to-end: accept base64 JSON, store as blob, serve as binary**
710
+ **Storing a calendar entry with a raw PUT:**
711
+
712
+ ```
713
+ PUT /my-resource/33
714
+ Content-Type: text/calendar
715
+
716
+ BEGIN:VCALENDAR
717
+ VERSION:2.0
718
+ ...
719
+ ```
720
+
721
+ Harper stores this as:
722
+
723
+ ```json
724
+ { "contentType": "text/calendar", "data": "BEGIN:VCALENDAR\nVERSION:2.0\n..." }
725
+ ```
726
+
727
+ **Full resource class handling base64 JSON upload and binary serving:**
728
728
 
729
729
  ```typescript
730
730
  import { type RequestTargetOrId, tables, createBlob } from 'harper';
731
731
 
732
732
  export class Photo extends tables.Photo {
733
- // Accept base64-encoded uploads in JSON
734
733
  static async post(target: RequestTargetOrId, record: any) {
735
- if (record.data) {
736
- record.data = createBlob(Buffer.from(record.data, record.encoding || 'base64'), {
737
- type: record.contentType || 'application/octet-stream',
734
+ const body = await record;
735
+ if (!body) return new Response('A JSON body is required', { status: 400 });
736
+ if (body.data) {
737
+ body.data = createBlob(Buffer.from(body.data, body.encoding || 'base64'), {
738
+ type: body.contentType || 'application/octet-stream',
738
739
  });
739
740
  }
740
- return super.post(target, record);
741
+ return super.post(target, body);
741
742
  }
742
743
 
743
- // Stream the blob back with the correct Content-Type
744
744
  static async get(target: RequestTargetOrId) {
745
745
  const record = await super.get(target);
746
746
  if (record?.data) {
@@ -757,10 +757,9 @@ export class Photo extends tables.Photo {
757
757
 
758
758
  #### Notes
759
759
 
760
- - `createBlob` takes a `Buffer` as its first argument and an options object with a `type` property for the MIME type. See [using-blob-datatype.md](using-blob-datatype.md) for full details on the blob data type.
761
- - Always fall back to `application/octet-stream` when no MIME type is known, both when creating and when serving blobs.
762
- - When Harper retrieves a record that has both `contentType` and `data` properties, it automatically sets the response `Content-Type` and body — no custom `get` override is required for that case unless you need additional logic.
763
- - Non-`text` content types cause `data` to be stored and returned as a Node.js `Buffer`.
760
+ - Always `await` the `record` parameter before accessing its properties; accessing fields on the unresolved promise yields `undefined`.
761
+ - `createBlob` accepts a `Buffer` and an options object with a `type` property for the MIME type. Fall back to `application/octet-stream` when no MIME type is provided.
762
+ - For schema-level blob field definitions, see [using-blob-datatype.md](using-blob-datatype.md).
764
763
 
765
764
  ## 2. API & Communication
766
765
 
@@ -770,25 +769,26 @@ Instructions for the agent to follow when using Harper's automatically generated
770
769
 
771
770
  #### When to Use
772
771
 
773
- Apply this rule when enabling HTTP REST endpoints or WebSocket subscriptions for Harper tables without writing custom handler code. Use it whenever a schema type needs to be served over HTTP, when configuring real-time subscriptions, or when setting up conditional caching behavior for REST responses.
772
+ Apply this rule when enabling HTTP REST endpoints or WebSocket subscriptions for Harper tables and custom resources. Use it whenever you need to configure the REST plugin, understand the auto-generated endpoint surface, or wire up real-time WebSocket connections. See [querying-rest-apis.md](querying-rest-apis.md) for query syntax and [real-time-apps.md](real-time-apps.md) for real-time patterns.
774
773
 
775
774
  #### How It Works
776
775
 
777
- 1. **Enable REST in `config.yaml`**: Add `rest: true` to the application configuration file. This registers REST endpoints and, by default, WebSocket subscriptions for all exported resources.
776
+ 1. **Enable the REST plugin** by adding `rest: true` to your application's `config.yaml`:
778
777
 
779
778
  ```yaml
780
779
  rest: true
781
780
  ```
782
781
 
783
- To configure options explicitly:
782
+ Extended options:
784
783
 
785
784
  ```yaml
786
785
  rest:
787
786
  lastModified: true # enables Last-Modified response header support
788
787
  webSocket: false # disables automatic WebSocket support (enabled by default)
788
+ exactCount: true # opt in to Prefer: count=exact scans (off by default)
789
789
  ```
790
790
 
791
- 2. **Export the table in the schema**: Add `@export` to the type definition. Without `@export`, Harper registers no REST route and callers receive `404`. Without `rest: true`, even an exported table does not respond to HTTP requests. Both are required.
791
+ 2. **Export the table in your schema** using `@export`. Tables are not exposed by default — `@export` is required for Harper to register a REST route:
792
792
 
793
793
  ```graphql
794
794
  type Product @table @export {
@@ -798,46 +798,41 @@ Apply this rule when enabling HTTP REST endpoints or WebSocket subscriptions for
798
798
  }
799
799
  ```
800
800
 
801
- Reference the schema file in `config.yaml`:
802
-
803
- ```yaml
804
- graphqlSchema:
805
- files: schema.graphql
806
- rest: true
807
- ```
801
+ Without `@export`, the table has no REST route and callers receive `404`. Without `rest: true` in `config.yaml`, even an exported table does not respond to HTTP requests.
808
802
 
809
- 3. **Use the automatically registered endpoints**: Harper serves the following endpoints on the application HTTP server port (default `9926`) with no route definitions or handler code required.
803
+ 3. **Understand the auto-generated endpoints**. With both `@export` and `rest: true` in place, Harper registers the following on the application HTTP server port (default `9926`):
810
804
 
811
- | Endpoint | Description |
812
- | ---------------------------- | --------------------------------------------------------------------------- |
813
- | `GET /Product` | Returns resource description (table name, database, attributes) |
814
- | `GET /Product/` | Returns the record collection; append query parameters to filter |
815
- | `GET /Product/{id}` | Returns a single record by primary key; `404` if not found |
816
- | `GET /Product/{id}.property` | Returns a single declared property of one record |
817
- | `POST /Product/` | Creates a record; responds `201`; primary key returned in `Location` header |
818
- | `PUT /Product/{id}` | Creates or replaces the record at `{id}` (upsert) |
819
- | `PATCH /Product/{id}` | Merges body into existing record (shallow, top-level only) |
820
- | `DELETE /Product/{id}` | Deletes the record at `{id}` |
821
- | `DELETE /Product/?query` | Deletes every record matching the query |
805
+ | Endpoint | Description |
806
+ | ---------------------------- | ------------------------------------------------------------------------------------ |
807
+ | `GET /Product` | Resource description — table name, database, declared attributes |
808
+ | `GET /Product/` | Record collection; append query parameters to filter, sort, page |
809
+ | `GET /Product/{id}` | Single record by primary key; `404` if not found |
810
+ | `GET /Product/{id}.property` | Single declared property of one record |
811
+ | `POST /Product/` | Creates a record; responds `201`; primary key returned in `Location` header |
812
+ | `PUT /Product/{id}` | Creates or replaces the record at `{id}` (upsert); omitted properties are removed |
813
+ | `PATCH /Product/{id}` | Shallow-merges body into existing record; unspecified top-level properties preserved |
814
+ | `DELETE /Product/{id}` | Deletes the record at `{id}` |
815
+ | `DELETE /Product/?query` | Deletes every record matching the query |
816
+ - `HEAD` is served exactly as `GET` with the response body omitted. `QUERY` is accepted on the collection path (`QUERY /Product/`) and runs a search taken from the request body rather than the URL.
822
817
 
823
- 4. **Handle `POST` primary key and `Location`**: On a successful `POST`, the new record's primary key is returned in the `Location` response header — the value the body supplied if it carried the primary-key property, otherwise a Harper-assigned key. The header carries the bare key value, not a URL.
818
+ 4. **Use the correct URL structure**. The trailing slash is significant:
824
819
 
825
- 5. **Understand `PUT` write behavior**: `PUT` replaces the stored record exactly. Three exceptions always apply: a `@createdTime` attribute keeps the original value, an `@updatedTime` attribute is re-stamped with the time of the write, and the primary key is forced to match the `{id}` in the URL.
820
+ | Path | Addresses |
821
+ | ------------------------- | ---------------------------------------------- |
822
+ | `/my-resource` | The resource itself (metadata) |
823
+ | `/my-resource/` | The record collection |
824
+ | `/my-resource/record-id` | A specific record by primary key |
825
+ | `/my-resource/record-id/` | Collection of records with the given id prefix |
826
826
 
827
- 6. **Use conditional requests for caching**: GET responses include an `ETag` header encoding the record's version/last-modification time. Send `If-None-Match` on subsequent requests with the cached `ETag` value. If the record has not changed, Harper returns `304 Not Modified` with no body.
827
+ 5. **Handle `POST` primary keys via `Location`**. On a successful `POST`, the new record's primary key is returned in the `Location` response header — the value supplied by the body, or a Harper-assigned key if the body omitted the primary-key property.
828
828
 
829
- 7. **Select content type with `Accept`**: Use the `Accept` header to request a specific response format. The suffixes `.json`, `.cbor`, `.msgpack`, and `.csv` are reserved as content-type selectors on property paths and take precedence over property names. See [querying-rest-apis.md](querying-rest-apis.md) for query syntax details.
829
+ 6. **Use conditional GET requests for caching**. GET responses include an `ETag` header. Send `If-None-Match` on subsequent requests; if the record is unchanged, Harper returns `304 Not Modified` with no body.
830
830
 
831
- 8. **Connect via WebSocket**: WebSocket support is enabled automatically when `rest` is enabled. Connecting to a resource URL subscribes to changes for that resource. See [real-time-apps.md](real-time-apps.md) for real-time patterns.
831
+ 7. **Apply `@updatedTime` for write timestamps**. On `PUT`, Harper re-stamps any `@updatedTime` attribute with the time of the write, preserves any `@createdTime` value from the original record, and forces the primary key to match the `{id}` in the URL.
832
832
 
833
- ```javascript
834
- let ws = new WebSocket('wss://server/my-resource/341');
835
- ws.onmessage = (event) => {
836
- let data = JSON.parse(event.data);
837
- };
838
- ```
833
+ 8. **Control OpenAPI visibility**. Every non-hidden exported resource appears in the generated OpenAPI document at `GET /openapi`. To exclude a type, mark it `@hidden` in the schema, or set `static hidden = true` on a programmatic resource class.
839
834
 
840
- 9. **Implement a custom `connect()` handler** when default subscription behavior is insufficient. The method must return an async iterable that produces messages to send to the client.
835
+ 9. **Implement a custom `connect()` handler** when default subscription behavior is insufficient. The method must return an async iterable that produces messages to send to the client:
841
836
 
842
837
  ```javascript
843
838
  export class Echo extends Resource {
@@ -849,9 +844,28 @@ Apply this rule when enabling HTTP REST endpoints or WebSocket subscriptions for
849
844
  }
850
845
  ```
851
846
 
847
+ 10. **Connect via WebSocket**. WebSocket support is enabled automatically with `rest: true`. Connect to a resource URL to subscribe to change events:
848
+
849
+ ```javascript
850
+ let ws = new WebSocket('wss://server/my-resource/341');
851
+ ws.onmessage = (event) => {
852
+ let data = JSON.parse(event.data);
853
+ };
854
+ ```
855
+
856
+ Disable WebSocket support independently with `webSocket: false` under `rest`.
857
+
858
+ 11. **Use MQTT over WebSockets** by setting the sub-protocol header:
859
+
860
+ ```
861
+ Sec-WebSocket-Protocol: mqtt
862
+ ```
863
+
864
+ 12. **Specify response format** using the `Accept` header. The suffixes `.json`, `.cbor`, `.msgpack`, and `.csv` are reserved as content-type selectors on property paths and take precedence over property names.
865
+
852
866
  #### Examples
853
867
 
854
- ##### Full schema and config setup
868
+ **Schema and config for a REST-enabled table:**
855
869
 
856
870
  ```graphql
857
871
  # schema.graphql
@@ -869,100 +883,88 @@ graphqlSchema:
869
883
  rest: true
870
884
  ```
871
885
 
872
- ##### Conditional GET with ETag caching
873
-
874
- ```
875
- GET /Product/123
876
- # Response includes:
877
- # ETag: "abc123"
886
+ **Conditional GET with ETag caching:**
878
887
 
888
+ ```http
879
889
  GET /Product/123
880
890
  If-None-Match: "abc123"
881
- # Response: 304 Not Modified (no body transferred)
882
891
  ```
883
892
 
884
- ##### POST and read the Location header
893
+ Response when unchanged:
885
894
 
886
895
  ```
896
+ HTTP/1.1 304 Not Modified
897
+ ETag: "abc123"
898
+ ```
899
+
900
+ **POST and read the Location header:**
901
+
902
+ ```http
887
903
  POST /Product/
888
904
  Content-Type: application/json
889
905
 
890
- { "name": "Widget", "price": 9.99 }
891
-
892
- # Response:
893
- # 201 Created
894
- # Location: 7f3a9c
906
+ { "name": "Widget" }
895
907
  ```
896
908
 
897
- ##### PATCH (shallow merge only)
898
-
899
909
  ```
900
- PATCH /Product/123
901
- Content-Type: application/json
902
-
903
- { "price": 12.99 }
910
+ HTTP/1.1 201 Created
911
+ Location: 7f3a9c
904
912
  ```
905
913
 
906
- Only `price` is updated; other top-level properties are preserved. Nested objects in the body replace the stored sub-object wholesale — deep merge does not occur.
914
+ **PATCH (shallow merge only):**
907
915
 
908
- ##### Request MessagePack response
916
+ ```http
917
+ PATCH /Product/123
918
+ Content-Type: application/json
909
919
 
910
- ```
911
- GET /Product/123
912
- Accept: application/msgpack
920
+ { "status": "active" }
913
921
  ```
914
922
 
915
- Alternatively, use the `.msgpack` suffix on the URL path where supported.
916
-
917
- ##### WebSocket with custom outgoing messages
923
+ **Custom WebSocket connect handler:**
918
924
 
919
925
  ```javascript
920
- export class Example extends Resource {
921
- connect(incomingMessages) {
922
- let outgoingMessages = super.connect();
923
-
924
- let timer = setInterval(() => {
925
- outgoingMessages.send({ greeting: 'hi again!' });
926
- }, 1000);
926
+ export class Echo extends Resource {
927
+ async *connect(incomingMessages) {
928
+ for await (let message of incomingMessages) {
929
+ yield message; // echo each message back
930
+ }
931
+ }
932
+ }
933
+ ```
927
934
 
928
- incomingMessages.on('data', (message) => {
929
- outgoingMessages.send(message);
930
- });
935
+ **Hide a resource from OpenAPI — schema directive:**
931
936
 
932
- outgoingMessages.on('close', () => {
933
- clearInterval(timer);
934
- });
935
-
936
- return outgoingMessages;
937
- }
937
+ ```graphql
938
+ type InternalLog @table @export @hidden {
939
+ id: Long @primaryKey
938
940
  }
939
941
  ```
940
942
 
941
- ##### Disable WebSocket while keeping REST
943
+ **Hide a resource from OpenAPI — programmatic class:**
942
944
 
943
- ```yaml
944
- rest:
945
- webSocket: false
945
+ ```javascript
946
+ export class InternalLog extends Resource {
947
+ static hidden = true;
948
+ }
946
949
  ```
947
950
 
948
951
  #### Notes
949
952
 
950
- - The trailing slash is significant: `/Product` addresses the resource itself; `/Product/` addresses its record collection. `POST /Product` (no trailing slash) returns `404`.
951
- - `HEAD` is served as `GET` with the body omitted. `QUERY` is accepted on the collection path and reads its search from the request body.
952
- - A `POST` to an existing primary key fails with `409` — it does not overwrite.
953
- - A component directory with **no configuration file** gets REST enabled by Harper's built-in default. As soon as a `config.yaml` exists it is used verbatim — add `rest: true` explicitly or REST is off.
954
- - Do not apply `@export` to a schema type and also export a same-named JavaScript subclass of that table — this produces conflicting endpoints.
955
- - Server-Sent Events subscriptions are served on the same paths, negotiated via `Accept: text/event-stream`. They are not affected by the `webSocket` option.
956
- - Every non-hidden exported resource is included in the generated OpenAPI document at `GET /openapi`. Mark a type `@hidden` or set `static hidden = true` on a programmatic Resource to omit it.
957
- - MQTT over WebSockets requires the sub-protocol header `Sec-WebSocket-Protocol: mqtt`.
953
+ - A component directory with **no** `config.yaml` inherits Harper's built-in default, which enables `rest` automatically. As soon as a `config.yaml` exists, it is used verbatim — omitting `rest` from it disables REST even if the directory previously had it without a config file. Always add `rest: true` when adding a config file.
954
+ - `POST /Product` (no trailing slash) returns `404`. The trailing slash is required for collection operations.
955
+ - A `POST` to a primary key that already exists fails with `409` rather than overwriting.
956
+ - PATCH merge is **shallow** — a nested object in the body replaces the stored nested object wholesale; nested properties not included in the body are dropped.
957
+ - The `.msgpack` suffix (along with `.json`, `.cbor`, `.csv`) is reserved as a content-type selector and cannot be used as a property name in dot-path access.
958
+ - Server-Sent Events subscriptions are served on the same paths, negotiated via `Accept: text/event-stream`; they are not affected by the `webSocket` option.
959
+ - See [querying-rest-apis.md](querying-rest-apis.md) for full query syntax on collection endpoints and [real-time-apps.md](real-time-apps.md) for real-time subscription patterns.
958
960
 
959
961
  ### 2.2 Querying REST APIs
960
962
 
961
- Instructions for the agent to filter, sort, select, and paginate Harper REST API collections using URL query parameters.
963
+ Instructions for the agent to filter, sort, select, and paginate records through Harper's URL-based REST query language.
962
964
 
963
965
  #### When to Use
964
966
 
965
- Apply this rule whenever building or modifying code that queries Harper REST collection endpoints. Use it when you need to filter records by attribute values, apply comparison operators, sort or paginate results, or join across related tables. See [automatic-apis.md](automatic-apis.md) for how Harper exposes tables as REST endpoints.
967
+ Apply this rule whenever you need to construct or handle GET requests against Harper collection endpoints that require filtering by attribute value, comparison operators, sorting, field selection, or paginated results. This rule also covers type coercion syntax and relationship joins via dot notation. See [automatic-apis.md](automatic-apis.md) for how REST endpoints are generated from schemas.
966
968
 
967
969
  #### How It Works
968
970
 
@@ -973,7 +975,13 @@ Apply this rule whenever building or modifying code that queries Harper REST col
973
975
  GET /Product/?category=software&inStock=true
974
976
  ```
975
977
 
976
- 2. **Apply comparison operators (FIQL syntax)**: Use FIQL operators in the query string for numeric, string, and date comparisons.
978
+ To filter for null values:
979
+
980
+ ```
981
+ GET /Product/?discount=null
982
+ ```
983
+
984
+ 2. **Apply comparison operators (FIQL syntax)**: Use FIQL operators in the query string for range and pattern matching.
977
985
 
978
986
  | Operator | Meaning |
979
987
  | -------------------- | -------------------------------------- |
@@ -1008,7 +1016,7 @@ Apply this rule whenever building or modifying code that queries Harper REST col
1008
1016
  GET /Product/?price=gt=100&lt=200
1009
1017
  ```
1010
1018
 
1011
- 4. **Apply type conversion**: For FIQL comparators, Harper converts values automatically. Use explicit prefixes to force a type.
1019
+ 4. **Apply type conversion**: For FIQL comparators (`==`, `!=`, `=gt=`, etc.), Harper converts values automatically. Use explicit type prefixes to force a specific type.
1012
1020
 
1013
1021
  | Syntax | Behavior |
1014
1022
  | ----------------------------------------- | ------------------------------------------- |
@@ -1028,20 +1036,20 @@ Apply this rule whenever building or modifying code that queries Harper REST col
1028
1036
  GET /Product/?rating=5|featured=true
1029
1037
  ```
1030
1038
 
1031
- 6. **Group conditions**: Use parentheses or square brackets to control order of operations. Prefer square brackets when constructing queries from user input, since standard URI encoding safely encodes `[` and `]`.
1039
+ 6. **Group conditions**: Use parentheses or square brackets to control evaluation order. Prefer square brackets when building queries from user input, since standard URI encoding safely encodes `[` and `]`.
1032
1040
 
1033
1041
  ```
1034
1042
  GET /Product/?rating=5|(price=gt=100&price=lt=200)
1035
1043
  GET /Product/?rating=5&[tag=fast|tag=scalable|tag=efficient]
1036
1044
  ```
1037
1045
 
1038
- Construct from JavaScript:
1046
+ Build grouped queries in JavaScript:
1039
1047
 
1040
1048
  ```javascript
1041
1049
  let url = `/Product/?rating=5&[${tags.map(encodeURIComponent).join('|')}]`;
1042
1050
  ```
1043
1051
 
1044
- 7. **Select specific properties with `select(`**: Append `select(...)` as a query function separated by `&`.
1052
+ 7. **Select specific properties with `select()`**: Append `select()` as a query function separated by `&`.
1045
1053
 
1046
1054
  | Syntax | Returns |
1047
1055
  | -------------------------------------- | ------------------------------------------- |
@@ -1051,47 +1059,99 @@ Apply this rule whenever building or modifying code that queries Harper REST col
1051
1059
  | `?select(property1,)` | Objects with a single specified property |
1052
1060
  | `?select(property{subProp1,subProp2})` | Nested objects with specific sub-properties |
1053
1061
 
1054
- 8. **Paginate with `limit(`**: Use `limit(end)` or `limit(start,end)` to control result count and offset.
1062
+ ```
1063
+ GET /Product/?category=software&select(name)
1064
+ GET /Product/?brand.name=Microsoft&select(name,brand{name})
1065
+ ```
1055
1066
 
1056
- 9. **Sort with `sort(`**: Use `sort(property)` or `sort(+property,-property,...)`. Prefix `+` or no prefix = ascending; `-` = descending.
1067
+ 8. **Limit results with `limit(end)` or `limit(start,end)`**: Append as a query function.
1057
1068
 
1058
- 10. **Query across relationships**: Use dot-syntax to filter by related table attributes. Relationships must be defined in the schema using `@relationship`. Relationship attributes are not included by default — use `select()` to include them.
1069
+ ```
1070
+ GET /Product/?rating=gt=3&inStock=true&select(rating,name)&limit(20)
1071
+ GET /Product/?rating=gt=3&limit(10,30)
1072
+ ```
1059
1073
 
1074
+ 9. **Sort results with `sort(property)` or `sort(+property,-property,...)`**: Prefix `+` or no prefix = ascending; `-` = descending. List multiple properties to break ties in order.
1075
+
1076
+ ```
1077
+ GET /Product/?rating=gt=3&sort(+name)
1078
+ GET /Product/?sort(+rating,-price)
1079
+ ```
1080
+
1081
+ 10. **Paginate with a total count**: Use `limit(start,end)` for paging and send a `Prefer` request header to receive a total match count.
1082
+
1083
+ | `Prefer` value | Meaning |
1084
+ | ----------------- | ---------------------------------------- |
1085
+ | `count=exact` | Exact count of matching records (opt-in) |
1086
+ | `count=estimated` | Fast approximate count |
1087
+
1088
+ ```
1089
+ GET /Product/?category=software&limit(0,25)
1090
+ Prefer: count=exact
1091
+ ```
1092
+
1093
+ The server responds with:
1094
+
1095
+ | Header | Example | Description |
1096
+ | -------------------- | ----------------- | ------------------------------------------------------ |
1097
+ | `Content-Range` | `items 0-24/1234` | 0-based inclusive range out of total matching records |
1098
+ | `Range-Unit` | `items` | Unit used by `Content-Range` |
1099
+ | `Preference-Applied` | `count=exact` | Count mode the server applied (`exact` or `estimated`) |
1100
+
1101
+ The response status is always `200`. When the total cannot be produced, it is reported as `*` (e.g., `Content-Range: items 0-24/*`).
1102
+
1103
+ Enable exact counts in the application's REST configuration:
1104
+
1105
+ ```yaml
1106
+ rest:
1107
+ exactCount: true
1060
1108
  ```
1109
+
1110
+ Without this, a `count=exact` request is served as an estimate.
1111
+
1112
+ 11. **Query across relationships using dot syntax**: Define relationships in the schema with `@relationship`, then filter and select across them.
1113
+
1114
+ ```
1115
+ GET /Product/?brand.name=Microsoft
1116
+ GET /Brand/?products.name=Keyboard
1061
1117
  GET /Product/?brand.name=Microsoft&select(name,brand{name})
1062
1118
  ```
1063
1119
 
1064
- 11. **Query for null values**: Use `=null` as the value to match null or non-null records.
1120
+ Filtering on a related attribute produces INNER JOIN behavior. Selecting a relationship without filtering produces LEFT JOIN behavior — the property is omitted if the foreign key is null or references a non-existent record.
1121
+
1122
+ 12. **Access a specific property by URL**: Append the property name with dot syntax to the record ID.
1123
+
1065
1124
  ```
1066
- GET /Product/?discount=null
1125
+ GET /MyTable/123.propertyName
1067
1126
  ```
1068
1127
 
1128
+ This only works for declared schema properties. The suffixes `.json`, `.cbor`, `.msgpack`, and `.csv` are reserved as content-type selectors.
1129
+
1069
1130
  #### Examples
1070
1131
 
1071
- **Filter with comparison operators and select:**
1132
+ **Range filter with select and sort:**
1072
1133
 
1073
1134
  ```
1074
- GET /Product/?category=software&price=gt=100&price=lt=200&select(name,price)
1135
+ GET /Product/?category=software&price=gt=100&price=lt=200&select(name,price)&sort(+price)
1075
1136
  ```
1076
1137
 
1077
- **Paginate and sort:**
1138
+ **Paginated request with exact count:**
1078
1139
 
1079
1140
  ```
1080
- GET /Product/?rating=gt=3&inStock=true&select(rating,name)&limit(20)
1081
- GET /Product/?rating=gt=3&limit(10,30)
1082
- GET /Product/?rating=gt=3&sort(+name)
1083
- GET /Product/?sort(+rating,-price)
1141
+ GET /Product/?category=software&limit(0,25)
1142
+ Prefer: count=exact
1084
1143
  ```
1085
1144
 
1086
- **OR logic with grouping:**
1145
+ Response headers:
1087
1146
 
1088
1147
  ```
1089
- GET /Product/?price=lt=100|[rating=5&[tag=fast|tag=scalable|tag=efficient]&inStock=true]
1148
+ HTTP/1.1 200 OK
1149
+ Content-Range: items 0-24/1234
1150
+ Range-Unit: items
1151
+ Preference-Applied: count=exact
1090
1152
  ```
1091
1153
 
1092
- **Relationship join with nested select:**
1093
-
1094
- Define the schema:
1154
+ **Relationship schema and join query:**
1095
1155
 
1096
1156
  ```graphql
1097
1157
  type Product @table @export {
@@ -1107,11 +1167,8 @@ type Brand @table @export {
1107
1167
  }
1108
1168
  ```
1109
1169
 
1110
- Query with join:
1111
-
1112
1170
  ```
1113
1171
  GET /Product/?brand.name=Microsoft&select(name,brand{name,id})
1114
- GET /Brand/?products.name=Keyboard
1115
1172
  ```
1116
1173
 
1117
1174
  **Many-to-many relationship:**
@@ -1129,19 +1186,20 @@ type Product @table @export {
1129
1186
  GET /Product/?resellers.name=Cool Shop&select(id,name,resellers{name,id})
1130
1187
  ```
1131
1188
 
1132
- **Access a specific property by record ID:**
1189
+ **OR grouping with user-supplied tags:**
1133
1190
 
1134
- ```
1135
- GET /MyTable/123.propertyName
1191
+ ```javascript
1192
+ let url = `/Product/?rating=5&[${tags.map(encodeURIComponent).join('|')}]`;
1136
1193
  ```
1137
1194
 
1138
1195
  #### Notes
1139
1196
 
1140
- - Only indexed attributes can be used as the primary filter attribute; when combining multiple attributes, only one needs to be indexed.
1141
- - Relationship attributes are excluded from responses by default. Always use `select(` to include them.
1142
- - When selecting a related attribute without filtering on it, the behavior is a LEFT JOIN — the property is omitted if the foreign key is null or references a non-existent record.
1143
- - The suffixes `.json`, `.cbor`, `.msgpack`, and `.csv` in URL paths are reserved as content-type selectors and take precedence over a property of the same name.
1144
- - Square brackets are preferred over parentheses when building grouped queries programmatically, because `[` and `]` are safely URL-encoded by standard encoding functions while `(` is not.
1197
+ - The queried attribute must be indexed for basic attribute filtering to execute. For multi-attribute queries, only one attribute needs to be indexed.
1198
+ - Null indexing requires indexes created after the feature was introduced. Rebuild existing indexes (remove and re-add) to support `name==null` queries.
1199
+ - `limit()` must be a non-negative integer no larger than **10,000**, and the requested window (offset + limit) no larger than **1,000,000**, for count headers to be returned.
1200
+ - A `HEAD` request with a `Prefer: count=exact` header returns count headers with no body — it saves bandwidth but still scans the matched set.
1201
+ - When CORS is enabled, `Content-Range`, `Range-Unit`, and `Preference-Applied` are added to `Access-Control-Expose-Headers` automatically.
1202
+ - Operators `!=` and `=ct=` (contains) do not produce cardinality estimates; the total is reported as `*` when these are used with `count=exact`.
1145
1203
 
1146
1204
  ### 2.3 Real-Time Apps with WebSockets and Pub/Sub
1147
1205
 
@@ -1683,11 +1741,11 @@ Instructions for the agent to follow when adding custom logic to automatically g
1683
1741
 
1684
1742
  #### When to Use
1685
1743
 
1686
- Apply this rule when you need to add computed properties, intercept writes, enforce validation, or otherwise customize the behavior of a Harper table resource beyond what the default generated endpoints provide. Use it any time a `@table` type needs server-side logic attached to its REST handlers.
1744
+ Apply this rule when you need to override or augment the default HTTP handler behavior (GET, POST, PUT, PATCH, DELETE) for a Harper table resource. Use it when a table needs computed properties, input transformation, authorization checks, or custom error responses.
1687
1745
 
1688
1746
  #### How It Works
1689
1747
 
1690
- 1. **Define the schema without `@export`**: Declare the table type in `schema.graphql` and omit the `@export` directive. Leaving `@export` on the schema while also exporting a subclass with the same name produces conflicting endpoints. Let the JavaScript class own the URL instead.
1748
+ 1. **Define the table schema without `@export`**: In `schema.graphql`, declare the table type but omit the `@export` directive. Leaving `@export` on the schema while also exporting a subclass with the same name produces conflicting endpoints.
1691
1749
 
1692
1750
  ```graphql
1693
1751
  # Omit the `@export` directive
@@ -1697,7 +1755,7 @@ Apply this rule when you need to add computed properties, intercept writes, enfo
1697
1755
  }
1698
1756
  ```
1699
1757
 
1700
- 2. **Extend the generated table class**: In `resources.js`, extend from the `tables.<TypeName>` global. The class name you export becomes the URL path. The exported class extends tables.
1758
+ 2. **Extend the generated table class in `resources.js`**: Use `extends tables.<TableName>` to subclass the auto-generated resource. The exported JavaScript class owns the URL instead of the schema type.
1701
1759
 
1702
1760
  ```javascript
1703
1761
  export class MyTable extends tables.MyTable {
@@ -1707,19 +1765,17 @@ Apply this rule when you need to add computed properties, intercept writes, enfo
1707
1765
  }
1708
1766
 
1709
1767
  static async post(target, data) {
1710
- this.create({ ...(await data), status: 'pending' });
1768
+ return this.create({ ...(await data), status: 'pending' });
1711
1769
  }
1712
1770
  }
1713
1771
  ```
1714
1772
 
1715
- 3. **Call `super` to preserve default behavior**: When delegating to `super`, match the argument form to the operation:
1716
- - Reads/deletes: `super.get(target)` / `super.delete(target)`
1773
+ 3. **Call `super` with the correct argument form per operation**: When delegating to the default behavior, match arguments to the operation type:
1774
+ - Reads and deletes: `super.get(target)` / `super.delete(target)`
1717
1775
  - Collection create: `super.post(target, record)` — target carries no id
1718
1776
  - Updates: `super.put(target, data)` / `super.patch(target, data)`
1719
1777
 
1720
- Omit the `super` call only if you intend to replace the default behavior entirely.
1721
-
1722
- 4. **Set `statusCode` on thrown errors to control HTTP responses**: Uncaught errors are caught by the protocol handler and produce error responses for REST. Use `.statusCode` — a plain `.status` property is ignored.
1778
+ 4. **Set `statusCode` on thrown errors to control HTTP status**: A plain `.status` property is ignored. Use `.statusCode` on the error object:
1723
1779
 
1724
1780
  ```javascript
1725
1781
  const error = new Error('Name is required');
@@ -1727,7 +1783,9 @@ Apply this rule when you need to add computed properties, intercept writes, enfo
1727
1783
  throw error;
1728
1784
  ```
1729
1785
 
1730
- 5. **Configure Harper to load both files**: Ensure your configuration references the schema and resource files.
1786
+ Uncaught errors are caught by the protocol handler and produce error responses for REST.
1787
+
1788
+ 5. **Configure Harper to load both files**: Ensure your configuration references the schema and resource files:
1731
1789
 
1732
1790
  ```yaml
1733
1791
  rest: true
@@ -1739,17 +1797,9 @@ Apply this rule when you need to add computed properties, intercept writes, enfo
1739
1797
 
1740
1798
  #### Examples
1741
1799
 
1742
- Full end-to-end example — schema, resource class, and error handling:
1743
-
1744
- ```graphql
1745
- # schema.graphql — omit @export so the JS class owns the endpoint
1746
- type MyTable @table {
1747
- id: Long @primaryKey
1748
- }
1749
- ```
1800
+ Full example extending a table with a computed GET response, a custom POST, and a guarded handler that throws a typed error:
1750
1801
 
1751
1802
  ```javascript
1752
- // resources.js
1753
1803
  export class MyTable extends tables.MyTable {
1754
1804
  static async get(target) {
1755
1805
  // get the record from the database
@@ -1759,27 +1809,27 @@ export class MyTable extends tables.MyTable {
1759
1809
  }
1760
1810
 
1761
1811
  static async post(target, data) {
1762
- // custom action on POST
1763
- this.create({ ...(await data), status: 'pending' });
1812
+ // custom action on POST; return the write so the response waits for the commit
1813
+ return this.create({ ...(await data), status: 'pending' });
1764
1814
  }
1765
- }
1766
- ```
1767
1815
 
1768
- Throwing a controlled HTTP error:
1769
-
1770
- ```javascript
1771
- if (!authorized) {
1772
- const error = new Error('Forbidden');
1773
- error.statusCode = 403;
1774
- throw error;
1816
+ static async delete(target) {
1817
+ if (!authorized) {
1818
+ const error = new Error('Forbidden');
1819
+ error.statusCode = 403;
1820
+ throw error;
1821
+ }
1822
+ return super.delete(target);
1823
+ }
1775
1824
  }
1776
1825
  ```
1777
1826
 
1778
1827
  #### Notes
1779
1828
 
1780
- - Always omit `@export` from the schema type when a JavaScript subclass is exporting the same name. The two registrations conflict.
1781
- - `super` must be called with the correct arguments for each operation type — mismatched arguments will not behave as expected.
1782
- - `statusCode` is the only recognized property for controlling HTTP status on thrown errors; `.status` is ignored.
1829
+ - Always omit `@export` from the schema type when you export a subclass with the same name in JavaScript. The exported class owns the endpoint registration.
1830
+ - Call `super.get/post/put/patch/delete` to preserve Harper's default behavior unless you intend to replace it entirely.
1831
+ - `statusCode` on an error object controls the HTTP response status for REST. A `.status` property is ignored.
1832
+ - See [`Database / Schema`](/reference/v5/database/schema.md) for full schema API details.
1783
1833
 
1784
1834
  ### 3.3 Programmatic Table Requests
1785
1835
 
@@ -2226,7 +2276,7 @@ Instructions for the agent to follow when deploying a Harper application to a re
2226
2276
 
2227
2277
  #### When to Use
2228
2278
 
2229
- Apply this rule when deploying a Harper application to a remote Harper Fabric cluster or any remote Harper instance. This includes first-time deploys, redeployments, rollbacks, CI/CD pipeline deploys, and provisioning credentials for private repositories. See [creating-a-fabric-account-and-cluster.md](creating-a-fabric-account-and-cluster.md) for setting up the cluster before deploying.
2279
+ Apply this rule when deploying a Harper application to a remote Harper instance or Fabric cluster, including first-time deploys, redeployments, rollbacks, and CI/CD pipeline deployments. Also apply it when provisioning credentials for private repository deploys. See [creating-a-fabric-account-and-cluster.md](creating-a-fabric-account-and-cluster.md) for setting up the cluster before deploying.
2230
2280
 
2231
2281
  #### How It Works
2232
2282
 
@@ -2237,7 +2287,7 @@ Apply this rule when deploying a Harper application to a remote Harper Fabric cl
2237
2287
  # Provide cluster username and password when prompted
2238
2288
  ```
2239
2289
 
2240
- 2. **Deploy the application**: After login, run `harper deploy` without repeating credentials. Set `restart=true` and `replicated=true` for a production deploy.
2290
+ 2. **Deploy the application**: After login, run `harper deploy` without repeating credentials. Use `restart=true` and `replicated=true` for production deploys.
2241
2291
 
2242
2292
  ```bash
2243
2293
  harper deploy \
@@ -2248,42 +2298,32 @@ Apply this rule when deploying a Harper application to a remote Harper Fabric cl
2248
2298
  replicated=true
2249
2299
  ```
2250
2300
 
2251
- 3. **Use environment variables for CI/CD**: Instead of `harper login`, export credentials as environment variables before calling `harper deploy`.
2301
+ 3. **Choose a package source**: Set the `package` parameter to any valid npm dependency value. Options:
2252
2302
 
2253
- ```bash
2254
- export HARPER_CLI_USERNAME=<username>
2255
- export HARPER_CLI_PASSWORD=<password>
2256
- harper deploy \
2257
- project=<name> \
2258
- package=<package> \
2259
- target=<remote> \
2260
- restart=true \
2261
- replicated=true
2262
- ```
2303
+ | Source | Example value |
2304
+ | ----------------------- | ------------------------------------------------------------ |
2305
+ | Current local directory | Omit `package` |
2306
+ | npm package | `package="@harperdb/status-check"` |
2307
+ | GitHub (shorthand) | `package="HarperFast/status-check"` |
2308
+ | GitHub (URL) | `package="https://github.com/HarperFast/status-check"` |
2309
+ | Private repo (SSH) | `package="git+ssh://git@github.com:HarperDB/secret-app.git"` |
2310
+ | Tarball | `package="https://example.com/application.tar.gz"` |
2263
2311
 
2264
- 4. **Choose a package source**: The `package` field accepts any valid npm dependency value. Select the form that matches your source:
2265
-
2266
- | Source | `package` value |
2267
- | ----------------------- | ---------------------------------------------------- |
2268
- | Current local directory | Omit `package` |
2269
- | npm package | `"@harperdb/status-check"` |
2270
- | GitHub (public) | `"HarperFast/status-check"` or full URL |
2271
- | Private repo (SSH) | `"git+ssh://git@github.com:HarperDB/secret-app.git"` |
2272
- | Tarball | `"https://example.com/application.tar.gz"` |
2273
-
2274
- For git tags, use the `semver` directive:
2312
+ When using git tags, use the `semver` directive:
2275
2313
 
2276
2314
  ```
2277
2315
  HarperFast/application-template#semver:v1.0.0
2278
2316
  ```
2279
2317
 
2280
- 5. **Deploy by reference for reproducible deploys**: Pass `by_ref=true` to send a pinned git SHA instead of uploading a snapshot. The cluster fetches and builds from that exact commit.
2318
+ 4. **Deploy by reference (pinned commit)**: Use `by_ref=true` to send a pinned git reference instead of uploading a snapshot. The cluster fetches and builds from that exact commit SHA.
2281
2319
 
2282
2320
  ```bash
2283
2321
  harper deploy by_ref=true restart=true replicated=true
2284
2322
  ```
2285
2323
 
2286
- Use `ref` to target a specific commit, tag, or branch (resolved to a full SHA before sending):
2324
+ - `by_ref` — Build the package reference from the local repository.
2325
+ - `ref` _(optional)_ — Deploy a specific commit, tag, or branch instead of `HEAD`. Implies `by_ref`. Tags and branches in `refs/tags` and `refs/heads` namespaces are resolved to a full commit SHA before the deploy is sent.
2326
+ - `credential` _(optional)_ — Set to `true` to authenticate the clone with the stored credential for the repository's host. Omit for public repositories.
2287
2327
 
2288
2328
  ```bash
2289
2329
  # Deploy a specific tag
@@ -2293,32 +2333,61 @@ Apply this rule when deploying a Harper application to a remote Harper Fabric cl
2293
2333
  harper deploy ref=9f8c2a1 restart=true replicated=true
2294
2334
  ```
2295
2335
 
2296
- **Key constraints for `ref` values:**
2297
- - Must name something a clone can fetch: `refs/heads/*` and `refs/tags/*`, or a bare branch or tag name.
2298
- - Anything else (e.g., `refs/pull/123/head`) is rejected up front.
2299
- - If a ref can't be resolved, the deploy stops — run `git fetch` and retry, or pass a full commit SHA.
2300
- - Commit and push before deploying: the cluster clones from the remote and only sees pushed commits.
2336
+ **Important constraints on refs:**
2337
+ - A full commit SHA is accepted directly with no resolution.
2338
+ - Every other `ref` must resolve to something a clone can fetch: `refs/tags/*` or `refs/heads/*`, or a bare branch or tag name.
2339
+ - Qualified refs outside those two namespaces (e.g., `refs/pull/123/head`) are rejected.
2340
+ - Commit and push before deploying — the cluster clones from the remote and only sees pushed commits.
2341
+ - Run `git fetch` if a ref can't be resolved, or pass a full commit SHA.
2301
2342
 
2302
- 6. **Deploy private repositories by reference**: Pass `credential=true` alongside `by_ref=true`. The CLI attaches a credentials reference; the cluster resolves the secret in memory at clone time — no token travels in the operation body or lands on disk.
2343
+ 5. **Handle private repositories**: Pass `credential=true` so the CLI attaches a credentials reference that the cluster resolves in memory at clone time. No token travels in the operation body or lands on disk.
2303
2344
 
2304
2345
  ```bash
2305
2346
  harper deploy by_ref=true credential=true restart=true replicated=true
2306
2347
  ```
2307
2348
 
2308
- 7. **Provision a deploy credential for private sources**: Run `harper deploy setup=true` once per component and source. This is interactive and requires **super_user** — run it with an administrative credential, not the CI identity.
2349
+ Provision the credential once with `setup=true` before using `credential=true`.
2350
+
2351
+ 6. **Provision a deploy credential**: Run `harper deploy setup=true` once per component and source to provision credentials for private deploys. This operation requires **super_user** — run it with an administrative credential, not the CI identity.
2309
2352
 
2310
2353
  ```bash
2311
2354
  harper deploy setup=true
2312
2355
  ```
2313
2356
 
2314
- This command:
2357
+ This interactive command:
2315
2358
  1. Fetches the cluster's public key with `get_secrets_public_key`.
2316
2359
  2. Encrypts the token locally into an `enc:v1:` envelope.
2317
2360
  3. Stores only the ciphertext with `set_secret`, in the component-scoped tier.
2318
2361
  4. Grants the component permission to resolve it with `grant_secret`.
2319
2362
  5. Prints the `credentials` reference for the deploy to use.
2320
2363
 
2321
- Use a **fine-grained** personal access token (PAT) scoped to **Contents: Read-only** on the specific repository. Avoid session tokens from `gh` CLI — they typically carry `repo`, `read:org`, `gist`, and `workflow` scopes across your whole account.
2364
+ **Use a fine-grained PAT** for GitHub repositories. The prompt defaults to a fine-grained personal access token with **Contents: Read-only** on that one repository. If you use the `gh` CLI session token instead, it typically carries `read:org`, `repo`, `gist`, and `workflow` scopes across your whole account — the CLI prints a warning if you choose it. Use the narrowest credential that does the job, because the stored token is replayed on every cold deploy and rollback.
2365
+
2366
+ 7. **Use environment variables for CI/CD**: Instead of `harper login`, export credentials as environment variables.
2367
+
2368
+ ```bash
2369
+ export HARPER_CLI_USERNAME=<username>
2370
+ export HARPER_CLI_PASSWORD=<password>
2371
+ harper deploy \
2372
+ project=<name> \
2373
+ package=<package> \
2374
+ target=<remote> \
2375
+ restart=true \
2376
+ replicated=true
2377
+ ```
2378
+
2379
+ 8. **Use dedicated auth parameters for one-off commands** (not recommended for production): Pass `auth_username` and `auth_password` directly. These take precedence over environment variables and saved login tokens.
2380
+
2381
+ ```bash
2382
+ harper deploy \
2383
+ project=<name> \
2384
+ package=<package> \
2385
+ auth_username=<username> \
2386
+ auth_password=<password> \
2387
+ target=<remote> \
2388
+ restart=true \
2389
+ replicated=true
2390
+ ```
2322
2391
 
2323
2392
  #### Examples
2324
2393
 
@@ -2334,44 +2403,51 @@ harper deploy \
2334
2403
  replicated=true
2335
2404
  ```
2336
2405
 
2337
- **CI/CD deploy using environment variables:**
2406
+ **Deploy by reference with a tag:**
2338
2407
 
2339
2408
  ```bash
2340
- export HARPER_CLI_USERNAME=admin
2341
- export HARPER_CLI_PASSWORD=secret
2342
- harper deploy \
2343
- project=my-app \
2344
- package="HarperFast/my-app" \
2345
- target=https://my-cluster.harperdbcloud.com \
2346
- restart=true \
2347
- replicated=true
2409
+ harper deploy ref=v1.2.0 restart=true replicated=true
2348
2410
  ```
2349
2411
 
2350
- **Deploy by reference in GitHub Actions (pull request):**
2412
+ **Deploy a private GitHub repo by reference:**
2351
2413
 
2352
2414
  ```bash
2353
- harper deploy ref=${{ github.event.pull_request.head.sha }} restart=true replicated=true
2415
+ # Provision credential once (requires super_user)
2416
+ harper deploy setup=true
2417
+
2418
+ # Deploy using stored credential
2419
+ harper deploy by_ref=true credential=true restart=true replicated=true
2354
2420
  ```
2355
2421
 
2356
- **Deploy a private repo by reference with a provisioned credential:**
2422
+ **GitHub Actions — pull request deploy:**
2357
2423
 
2358
2424
  ```bash
2359
- # Provision once (run as super_user)
2360
- harper deploy setup=true
2425
+ harper deploy ref=${{ github.event.pull_request.head.sha }} restart=true replicated=true
2426
+ ```
2361
2427
 
2362
- # Deploy subsequently
2363
- harper deploy by_ref=true credential=true restart=true replicated=true
2428
+ **CI/CD deploy using environment variables:**
2429
+
2430
+ ```bash
2431
+ export HARPER_CLI_USERNAME=<username>
2432
+ export HARPER_CLI_PASSWORD=<password>
2433
+ harper deploy \
2434
+ project=my-app \
2435
+ package="@myorg/my-app" \
2436
+ target=https://my-cluster.harperdbcloud.com \
2437
+ restart=true \
2438
+ replicated=true
2364
2439
  ```
2365
2440
 
2366
2441
  #### Notes
2367
2442
 
2368
- - `auth_username` and `auth_password` can be passed directly as deploy parameters for one-off commands, but this is not recommended for production. Dedicated authentication parameters take precedence over environment variables and saved login tokens.
2443
+ - The cluster's Application URL is found on the **Config → Overview** page of the Fabric dashboard.
2444
+ - `harper deploy setup=true` calls `get_secrets_public_key`, `set_secret`, and `grant_secret`, all of which require **super_user**. Do not run it with the CI identity.
2369
2445
  - The `enc:v1:` envelope means the plaintext token never leaves your machine — only ciphertext is stored and replicated.
2370
- - Deploy credentials are stored scoped to the component, never in the global `processEnv` tier. If a global secret exists at the derived name, it is converted to the component-scoped tier automatically.
2371
- - Because stored credentials are durable, later deploys and rollbacks reuse them without re-entering anything.
2372
- - The unpushed-commit check is skipped under GitHub Actions; the dirty-tree warning still applies.
2373
- - Deploying by reference means the cluster installs and builds from source. If your application requires a build step that cannot run on the node, deploy the built output as a payload deploy instead.
2446
+ - Secrets are stored scoped to the component, not in the global `processEnv` tier. If a global secret already exists at the derived name, it is converted to the scoped tier.
2374
2447
  - For SSH-based private repos, use the `add_ssh_key` operation to register keys before deploying.
2448
+ - If your application requires a build step that cannot run on the cluster node, deploy a built payload (omit `by_ref`) instead of deploying by reference.
2449
+ - The unpushed-commit check is skipped under GitHub Actions; the dirty-tree warning still applies.
2450
+ - Annotated tags in `refs/tags` resolve to the commit they point at, not the tag object.
2375
2451
 
2376
2452
  ### 4.2 Creating a Harper Fabric Account and Cluster
2377
2453
 
@@ -2810,54 +2886,76 @@ myApp:
2810
2886
 
2811
2887
  ### 4.7 v5 Upgrade: Breaking Changes and Migration Guide
2812
2888
 
2813
- Instructions for the agent to apply when migrating a Harper application to v5, covering all breaking changes and required code updates.
2889
+ Instructions for the agent to follow when migrating a Harper application to v5, covering all breaking changes and required code updates.
2814
2890
 
2815
2891
  #### When to Use
2816
2892
 
2817
- Apply this rule when upgrading an existing Harper application to v5, when encountering runtime errors related to renamed packages, changed APIs, or security restrictions after a v5 upgrade, or when scaffolding new v5-compatible application code.
2893
+ Apply this rule when upgrading an existing Harper application from v4 to v5, when encountering runtime errors after upgrading, or when reviewing application code for v5 compatibility. Every breaking change listed here must be addressed before the application will behave correctly under v5.
2818
2894
 
2819
2895
  #### How It Works
2820
2896
 
2821
- 1. **Update the package import from `harperdb` to `harper`**: All application code must import from `harper`, not `harperdb`.
2897
+ 1. **Update the package import**: Replace all imports from `'harperdb'` with `from 'harper'`.
2822
2898
 
2823
2899
  ```javascript
2824
2900
  import { tables } from 'harper';
2825
2901
  ```
2826
2902
 
2827
- 2. **Enable `allowInstallScripts` if packages require install scripts**: Harper v5 uses `--ignore-scripts` by default when installing packages. If a package requires execution of install scripts (e.g., to install native binaries), set the `allowInstallScripts` option when deploying.
2903
+ 2. **Enable install scripts if needed**: Harper v5 runs `npm install` with `--ignore-scripts` by default. If your application requires install scripts (e.g., to compile native binaries), set `allowInstallScripts` in your deployment options.
2828
2904
 
2829
- 3. **Update `Table.get` usage — return value is now a frozen record object**: `Table.get` now returns a plain record object, not a table class instance. The record is frozen; you cannot add or mutate properties directly.
2830
- - Replace direct property mutation:
2905
+ 3. **Update `Table.get` usage**: `Table.get` now returns a plain frozen record object, not a table class instance. The `wasLoadedFromSource()` method no longer exists on the returned object. Replace it with `loadedFromSource` on the `RequestTarget`:
2831
2906
 
2832
- ```javascript
2833
- let record = await Table.get(id);
2834
- record = { ...record, property: 'changed' };
2835
- ```
2907
+ ```javascript
2908
+ // Before
2909
+ const record = await Table.get(id);
2910
+ if (record.wasLoadedFromSource()) {
2911
+ // record was loaded from origin (not cache)
2912
+ }
2913
+ ```
2836
2914
 
2837
- - Replace `wasLoadedFromSource()` with `loadedFromSource` on the `target` object:
2915
+ ```javascript
2916
+ // After
2917
+ import { getContext } from 'harper';
2918
+ const target = new RequestTarget();
2919
+ target.id = id;
2920
+ const record = await Table.get(target);
2921
+ if (target.loadedFromSource) {
2922
+ // record was loaded from origin (not cache)
2923
+ }
2924
+ ```
2838
2925
 
2839
- ```javascript
2840
- const target = new RequestTarget();
2841
- target.id = id;
2842
- const record = await Table.get(target);
2843
- if (target.loadedFromSource) {
2844
- // record was loaded from origin (not cache)
2845
- }
2846
- ```
2926
+ The record objects do have `getUpdatedTime` and `getExpiresAt` methods available.
2847
2927
 
2848
- The record objects still expose `getUpdatedTime` and `getExpiresAt` methods.
2928
+ 4. **Handle frozen records**: The record object returned by `Table.get` is frozen — you cannot mutate it directly. Copy it before modifying:
2849
2929
 
2850
- 4. **Update transaction and context handling using `getContext`**: Harper v5 uses asynchronous context tracking. Context and the current transaction are automatically carried to all downstream calls — you no longer pass context explicitly. Import `getContext` and `transaction` from `harper`:
2930
+ ```javascript
2931
+ // Before
2932
+ const record = await Table.get(id);
2933
+ record.property = 'changed';
2934
+ ```
2851
2935
 
2852
2936
  ```javascript
2853
- import { getContext, transaction } from 'harper';
2937
+ // After
2938
+ let record = await Table.get(id);
2939
+ record = { ...record, property: 'changed' };
2940
+ ```
2941
+
2942
+ 5. **Register allowed spawn commands via `allowedSpawnCommands`**: `spawn` and `execFile` may only launch executables listed in `applications.allowedSpawnCommands` in `harper-config.yaml`. Only the first token of the command is matched. `exec` is not usable through the substituted module, and `execSync` always throws.
2943
+
2944
+ ```yaml
2945
+ applications:
2946
+ allowedSpawnCommands:
2947
+ - npm
2948
+ - node
2854
2949
  ```
2855
2950
 
2856
- If your code previously omitted context to escape a transaction (e.g., to poll for updated data), explicitly commit the transaction and/or wrap each read in a new `transaction()` call:
2951
+ Additionally, `spawn`, `execFile`, and `fork` now require a `name` property in the `options` argument to prevent process multiplication across threads.
2952
+
2953
+ 6. **Update transaction and context usage**: Harper v5 uses asynchronous context tracking. `Table.get` and other calls now automatically inherit the current transaction. Code that previously omitted context to bypass a transaction will no longer work as expected. Explicitly commit the transaction or wrap calls in a new transaction to read updated data:
2857
2954
 
2858
2955
  ```javascript
2859
2956
  import { setTimeout as delay } from 'node:timers/promises';
2860
2957
  import { getContext, transaction } from 'harper';
2958
+
2861
2959
  class MyResource {
2862
2960
  static async get(target) {
2863
2961
  await getContext().transaction.commit();
@@ -2869,79 +2967,70 @@ Apply this rule when upgrading an existing Harper application to v5, when encoun
2869
2967
  }
2870
2968
  ```
2871
2969
 
2872
- 5. **Register allowed spawn commands via `allowedSpawnCommands`**: `spawn` and `execFile` may only launch executables listed in `applications.allowedSpawnCommands` in `harperdb-config.yaml`. Only the first token of the command is matched. `exec` is not usable through the substituted module; `execSync` always throws.
2970
+ Use `getContext` (exported from `'harper'`) to access the current transaction anywhere without passing context explicitly.
2873
2971
 
2874
- ```yaml
2875
- applications:
2876
- allowedSpawnCommands:
2877
- - npm
2878
- - node
2879
- ```
2972
+ 7. **Replace `blob.save()`**: The `blob.save()` method has been removed. Use the `saveBeforeCommit` flag in the options passed to the `Blob` constructor instead.
2880
2973
 
2881
- Additionally, `spawn`, `execFile`, and `fork` now require a `name` property in the `options` argument to prevent process multiplication across threads.
2974
+ 8. **Handle response `headers`**: If you return an object from a REST method with a `headers` property, Harper v5 will use it as the response headers. Ensure any object you return that incidentally has a `headers` property is intentional.
2975
+
2976
+ 9. **Configure the module loader**: Harper v5 loads application modules through Node.js's VM module API. Control this with the `moduleLoader` setting:
2882
2977
 
2883
- 6. **Use `saveBeforeCommit` instead of `blob.save()`**: The `blob.save()` method has been removed. Pass the `saveBeforeCommit` flag in the options to the `Blob` constructor instead.
2978
+ | Value | Behavior |
2979
+ | -------------------- | ------------------------------------------------ |
2980
+ | `vm-current-context` | Default. Shares intrinsics with Harper. |
2981
+ | `vm` | Separate context and intrinsics per application. |
2982
+ | `native` | Standard `import()`, no application context. |
2983
+ | `compartment` | SES Compartments; advanced and heavier. |
2884
2984
 
2885
- 7. **Handle `headers` on returned response objects**: If you return an object from a REST method with a `headers` property, Harper v5 will use it as the response headers.
2985
+ The default (`vm-current-context`) avoids `instanceof` failures for values crossing the application/Harper boundary. Choose `vm` only if you need separate per-application intrinsics. Use `native` if the VM loader causes compatibility problems you cannot otherwise resolve.
2886
2986
 
2887
- 8. **Configure the VM module loader and `lockdown` in `harperdb-config.yaml`**: v5 loads application modules through Node.js's VM module API. Control all behavior under the `applications` key:
2987
+ To disable the VM loader entirely and restore pre-v5 behavior:
2888
2988
 
2889
2989
  ```yaml
2890
2990
  applications:
2891
- lockdown: freeze-after-load
2892
- moduleLoader: vm-current-context
2893
- dependencyLoader: auto
2894
- allowedDirectory: app
2895
- allowedSpawnCommands:
2896
- - npm
2897
- - node
2991
+ moduleLoader: native
2898
2992
  ```
2899
2993
 
2900
- **`moduleLoader` options:**
2994
+ Note: `logger` and per-app `config` are not available in `native` mode.
2901
2995
 
2902
- | Value | Behavior |
2903
- | -------------------- | -------------------------------------------------------------------------------------------------------------------------------- |
2904
- | `vm-current-context` | Default. VM loader in Harper's own context; shares intrinsics with Harper. Best compatibility. |
2905
- | `vm` | VM loader in a separate per-application context with its own intrinsics. Stronger isolation but may cause `instanceof` failures. |
2906
- | `native` | Standard Node.js `import()`. No VM loader; application-specific context (`logger`, `config`) unavailable. |
2907
- | `compartment` | SES Compartment-based loading. For specialized sandboxing only. |
2996
+ If the goal is only to fix package compatibility while keeping application context for first-party code, `dependencyLoader: native` is a narrower option — it uses native loading only for npm packages while keeping the VM loader for application source files.
2908
2997
 
2909
- **`lockdown` options:**
2998
+ 10. **Handle intrinsic `lockdown`**: The default `lockdown` mode (`freeze-after-load`) freezes JavaScript intrinsics (`Object`, `Array`, `Promise`, `Map`, `Set`, etc.) after all application code loads. Any code or dependency that modifies intrinsic prototypes at runtime will throw a `TypeError`. If a dependency requires a temporary workaround, set:
2910
2999
 
2911
- | Value | Behavior |
2912
- | ------------------- | ------------------------------------------------------- |
2913
- | `freeze-after-load` | Default. Freezes intrinsics after all components load. |
2914
- | `freeze` | Freezes intrinsics before loading any application code. |
2915
- | `ses` | Full SES lockdown via the `ses` package. Strictest. |
2916
- | `none` | No lockdown. Use as a temporary workaround only. |
3000
+ ```yaml
3001
+ applications:
3002
+ lockdown: none
3003
+ ```
2917
3004
 
2918
- To disable the VM loader entirely and restore pre-v5 behavior:
3005
+ 11. **Check `allowedDirectory` in production**: In production, applications can only load modules from within their own directory tree (`allowedDirectory: app`). If your application loads files from outside its directory, set:
2919
3006
 
2920
- ```yaml
2921
- applications:
2922
- moduleLoader: native
2923
- ```
3007
+ ```yaml
3008
+ applications:
3009
+ allowedDirectory: any
3010
+ ```
2924
3011
 
2925
3012
  #### Examples
2926
3013
 
2927
- **Full transaction polling pattern (v5):**
3014
+ ##### Full `harper-config.yaml` module loading block
2928
3015
 
2929
- ```javascript
2930
- import { setTimeout as delay } from 'node:timers/promises';
2931
- import { getContext, transaction } from 'harper';
3016
+ ```yaml
3017
+ applications:
3018
+ lockdown: freeze-after-load
3019
+ moduleLoader: vm-current-context
3020
+ dependencyLoader: auto
3021
+ allowedDirectory: app
3022
+ allowedSpawnCommands:
3023
+ - npm
3024
+ - node
3025
+ ```
2932
3026
 
2933
- class MyResource {
2934
- static async get(target) {
2935
- await getContext().transaction.commit();
2936
- while ((await transaction(() => Table.get(target))).status !== 'ready') {
2937
- await delay(100);
2938
- }
2939
- return Table.get(target);
2940
- }
2941
- }
3027
+ ##### Updated import and context access
3028
+
3029
+ ```javascript
3030
+ import { tables, getContext, transaction } from 'harper';
2942
3031
  ```
2943
3032
 
2944
- **Checking `loadedFromSource` after `Table.get`:**
3033
+ ##### Checking `loadedFromSource` after `Table.get`
2945
3034
 
2946
3035
  ```javascript
2947
3036
  const target = new RequestTarget();
@@ -2952,40 +3041,20 @@ if (target.loadedFromSource) {
2952
3041
  }
2953
3042
  ```
2954
3043
 
2955
- **Full `harperdb-config.yaml` `applications` block:**
2956
-
2957
- ```yaml
2958
- applications:
2959
- lockdown: freeze-after-load
2960
- moduleLoader: vm-current-context
2961
- dependencyLoader: auto
2962
- allowedDirectory: app
2963
- allowedSpawnCommands:
2964
- - npm
2965
- - node
2966
- ```
2967
-
2968
- **Restricting allowed built-in modules:**
3044
+ ##### Copying a frozen record before mutation
2969
3045
 
2970
- ```yaml
2971
- applications:
2972
- allowedBuiltinModules:
2973
- - fs
2974
- - path
2975
- - http
3046
+ ```javascript
3047
+ let record = await Table.get(id);
3048
+ record = { ...record, property: 'changed' };
2976
3049
  ```
2977
3050
 
2978
3051
  #### Notes
2979
3052
 
2980
- - Always import Harper APIs from `'harper'`, not from global variables or `'harperdb'`.
2981
- - `getContext` is exported from `'harper'` and provides access to the current transaction without passing context explicitly.
2982
- - Record objects returned by `Table.get` are frozen — spread into a new object before modifying.
2983
- - `loadedFromSource` is a property on the `target` object, replacing the removed `wasLoadedFromSource()` instance method.
2984
- - `saveBeforeCommit` replaces the removed `blob.save()` method.
2985
- - The `headers` property on a returned REST response object is used as response headers.
2986
- - Under `lockdown: ses`, the constrained `fetch` applies only in `vm` mode. In `vm-current-context` and `native` modes, application code uses the standard global `fetch`.
2987
- - In production, `allowedDirectory: app` is the default; modules outside the application directory tree will throw. Set `allowedDirectory: any` only if legitimately required.
2988
- - `dependencyLoader: native` is a narrower option than `moduleLoader: native` — it uses native loading only for npm packages while keeping the VM loader for first-party application source files.
3053
+ - The `logger` exported from `'harper'` is tagged with the application name when using the VM module loader. It is not available in `native` mode.
3054
+ - `instanceof` checks for values crossing the application/Harper boundary will fail when using `moduleLoader: vm` (separate intrinsics). Use `vm-current-context` (the default) to avoid this.
3055
+ - `import()` behavior depends on the `moduleLoader` setting. In `native` mode, standard Node.js `import()` is used with no application context.
3056
+ - Automatic context tracking simplifies code but requires explicit `commit()` or new `transaction()` calls when you need to observe data written within the same transaction.
3057
+ - Use `getContext` from `'harper'` rather than passing context manually through every call — this is the recommended pattern in v5.
2989
3058
 
2990
3059
  ### 4.8 Delegating to the Built-in Agent
2991
3060