@harperfast/template-react-ts-studio 1.12.2 → 1.12.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (20) hide show
  1. package/.agents/skills/harper-best-practices/AGENTS.md +632 -474
  2. package/.agents/skills/harper-best-practices/rules/automatic-apis.md +131 -55
  3. package/.agents/skills/harper-best-practices/rules/checking-authentication.md +155 -61
  4. package/.agents/skills/harper-best-practices/rules/deploying-to-harper-fabric.md +63 -85
  5. package/.agents/skills/harper-best-practices/rules/programmatic-table-requests.md +111 -52
  6. package/.agents/skills/harper-best-practices/rules/querying-rest-apis.md +38 -70
  7. package/.agents/skills/harper-best-practices/rules/schema-design-tooling.md +81 -69
  8. package/.agents/skills/harper-best-practices/rules/v5-upgrade.md +82 -96
  9. package/.agents/skills/harper-best-practices/rules.manifest.yaml +212 -0
  10. package/agent/skills/harper-best-practices/AGENTS.md +632 -474
  11. package/agent/skills/harper-best-practices/rules/automatic-apis.md +131 -55
  12. package/agent/skills/harper-best-practices/rules/checking-authentication.md +155 -61
  13. package/agent/skills/harper-best-practices/rules/deploying-to-harper-fabric.md +63 -85
  14. package/agent/skills/harper-best-practices/rules/programmatic-table-requests.md +111 -52
  15. package/agent/skills/harper-best-practices/rules/querying-rest-apis.md +38 -70
  16. package/agent/skills/harper-best-practices/rules/schema-design-tooling.md +81 -69
  17. package/agent/skills/harper-best-practices/rules/v5-upgrade.md +82 -96
  18. package/agent/skills/harper-best-practices/rules.manifest.yaml +212 -0
  19. package/package.json +1 -1
  20. package/skills-lock.json +1 -1
@@ -42,15 +42,26 @@ type ExamplePerson @table @export {
42
42
 
43
43
  ### 1.2 Schema Design and GraphQL Tooling
44
44
 
45
- Instructions for the agent to follow when designing Harper schemas, applying core directives, and configuring GraphQL tooling.
45
+ Instructions for the agent to follow when designing Harper database schemas using GraphQL type definitions, core directives, and tooling configuration.
46
46
 
47
47
  #### When to Use
48
48
 
49
- Apply this rule when creating or modifying Harper schema files (`.graphql`), configuring `graphqlSchema` in `config.yaml`, or deciding which directives to apply to types and fields. Use it any time you need to define tables, primary keys, indexes, or exported endpoints.
49
+ Apply this rule when creating or modifying Harper schema files (`.graphql`), configuring schema loading in `config.yaml`, or deciding which directives to apply to tables and fields. Use it whenever a task involves defining tables, primary keys, indexes, or export behavior.
50
50
 
51
51
  #### How It Works
52
52
 
53
- 1. **Declare the schema file** in the component's `config.yaml` using the `graphqlSchema` plugin:
53
+ 1. **Create a GraphQL schema file** with Harper-specific directives. Schemas ensure required tables exist on deployment, enforce types and constraints, control indexing, and define relationships.
54
+
55
+ ```graphql
56
+ type Dog @table {
57
+ id: Long @primaryKey
58
+ name: String
59
+ breed: String
60
+ age: Int
61
+ }
62
+ ```
63
+
64
+ 2. **Register the schema in `config.yaml`** using the `graphqlSchema` plugin:
54
65
 
55
66
  ```yaml
56
67
  graphqlSchema:
@@ -59,76 +70,74 @@ Apply this rule when creating or modifying Harper schema files (`.graphql`), con
59
70
 
60
71
  Both plugins and applications can specify schemas.
61
72
 
62
- 2. **Mark types as tables** with `@table`. The type name becomes the table name by default:
73
+ 3. **Mark each type as a table** with `@table`. The type name becomes the table name by default.
63
74
 
64
75
  ```graphql
65
- type Dog @table {
76
+ type MyTable @table {
66
77
  id: Long @primaryKey
67
- name: String
68
- breed: String
69
- age: Int
70
78
  }
71
79
  ```
72
80
 
73
- 3. **Set a primary key** on every table using `@primaryKey`. Primary keys must be unique; duplicate-key inserts are rejected. If no primary key is provided on insert, Harper auto-generates one based on the field type:
74
- - `String` or `ID` → UUID string
75
- - `Int`, `Long`, or `Any` → auto-incrementing integer
81
+ Key `@table` arguments:
76
82
 
77
- Use `Long` or `Any` for auto-generated numeric keys; `Int` is 32-bit and may be insufficient for large tables.
83
+ | Argument | Type | Default | Description |
84
+ | -------------------- | --------- | ----------------------------- | --------------------------------------------------------------- |
85
+ | `table` | `String` | type name | Override the table name |
86
+ | `database` | `String` | `"data"` | Database to place the table in |
87
+ | `expiration` | `Int` | — | Seconds until a record goes stale |
88
+ | `eviction` | `Int` | `0` | Additional seconds after `expiration` before physical removal |
89
+ | `scanInterval` | `Int` | `(expiration + eviction) / 4` | Seconds between eviction scans |
90
+ | `replicate` | `Boolean` | `true` | Enable replication of this table |
91
+ | `cacheControl` | `String` | — | `Cache-Control` header for anonymous GET/HEAD 200/304 responses |
92
+ | `randomAccessFields` | `Boolean` | `storage.randomAccessFields` | Pin this table's record encoding |
78
93
 
79
- 4. **Index fields for querying** with `@indexed`. Required for filtering by an attribute in REST queries, SQL, or NoSQL operations:
94
+ 4. **Designate a primary key** on every table using `@primaryKey`. Primary keys must be unique; duplicate inserts are rejected. If no primary key is provided on insert, Harper auto-generates one:
95
+ - **UUID string** — when type is `String` or `ID`
96
+ - **Auto-incrementing integer** — when type is `Int`, `Long`, or `Any`
97
+
98
+ Use `Long` or `Any` for auto-generated numeric keys; `Int` is 32-bit and may be insufficient for large tables.
80
99
 
81
100
  ```graphql
82
- type Breed @table {
101
+ type Product @table {
83
102
  id: Long @primaryKey
84
- name: String @indexed
103
+ name: String
85
104
  }
86
105
  ```
87
106
 
88
- If the field value is an array, each element is individually indexed. Null values are indexed by default.
89
-
90
- 5. **Expose a table as an external endpoint** with `@export`. Available via REST, MQTT, and other interfaces. The optional `name` parameter sets the URL path segment:
107
+ 5. **Add secondary indexes** with `@indexed` on any attribute that will be used for filtering in REST queries, SQL, or NoSQL operations. If the field value is an array, each element is individually indexed.
91
108
 
92
109
  ```graphql
93
- type MyTable @table @export(name: "my-table") {
110
+ type Product @table {
94
111
  id: Long @primaryKey
112
+ category: String @indexed
113
+ price: Float @indexed
95
114
  }
96
115
  ```
97
116
 
98
- Without `name`, the type name is used.
99
-
100
- 6. **Configure `@table` arguments** as needed. All arguments are optional:
101
-
102
- | Argument | Type | Default | Description |
103
- | -------------- | --------- | ----------------------------- | ------------------------------------------------------------- |
104
- | `table` | `String` | type name | Override the table name |
105
- | `database` | `String` | `"data"` | Database to place the table in |
106
- | `expiration` | `Int` | — | Seconds until a record goes stale |
107
- | `eviction` | `Int` | `0` | Additional seconds after `expiration` before physical removal |
108
- | `scanInterval` | `Int` | `(expiration + eviction) / 4` | Seconds between eviction scans |
109
- | `replicate` | `Boolean` | `true` | Enable replication of this table |
110
-
111
- 7. **Apply additional field directives** where needed:
112
- - `@createdTime` — auto-assigns creation timestamp (Unix epoch ms)
113
- - `@updatedTime` — auto-assigns update timestamp (Unix epoch ms)
114
- - `@expiresAt` — marks a field as the record's absolute expiration time (Unix epoch ms)
115
- - `@embed(source:, model:)` — computes an embedding vector when the source field is written; field type must be `[Float]`
116
- - `@hidden` — suppresses the field from MCP tool descriptors and OpenAPI document (not an access-control mechanism)
117
-
118
- 8. **Use `@sealed`** on a type to prevent records from including properties beyond those declared in the schema:
117
+ 6. **Expose tables via REST and other interfaces** using `@export`. Without `@export`, the table has no REST/MQTT route (callers get 404). The optional `name` parameter sets the URL path segment.
119
118
 
120
119
  ```graphql
121
- type StrictRecord @table @sealed {
120
+ type MyTable @table @export(name: "my-table") {
122
121
  id: Long @primaryKey
123
- name: String
124
122
  }
125
123
  ```
126
124
 
127
- 9. **Use unique database names** in plugins or applications to avoid table naming collisions, since all tables default to the `data` database.
125
+ `@export` is a routing directive, not access control. The table remains accessible through the Operations API and SQL regardless. REST must also be enabled for the application (via `rest: true` in `config.yaml` or Harper's built-in default).
126
+
127
+ 7. **Apply additional type directives** as needed:
128
+ - `@sealed` — prevents records from including properties beyond those declared in the schema.
129
+ - `@hidden` — suppresses the type from MCP tool descriptors and the OpenAPI document. Does not restrict data access.
130
+
131
+ 8. **Apply field directives** for computed and lifecycle behavior:
132
+ - `@createdTime` — assigns Unix epoch milliseconds on record creation.
133
+ - `@updatedTime` — assigns Unix epoch milliseconds on each update.
134
+ - `@expiresAt` — marks a field as the record's absolute expiration time (Unix epoch milliseconds); authoritative over the table-level `expiration` default.
135
+ - `@embed` — computes an embedding vector when the source field is written (requires `source` and `model` arguments; field type must be `[Float]`).
136
+ - `@hidden` (field) — suppresses the field from generated specs and MCP tool schemas; does not restrict data access.
128
137
 
129
138
  #### Examples
130
139
 
131
- **Minimal schema with two tables:**
140
+ **Minimal two-table schema:**
132
141
 
133
142
  ```graphql
134
143
  type Dog @table {
@@ -154,59 +163,62 @@ type WeatherCache @table(expiration: 300, eviction: 3300, scanInterval: 600) {
154
163
  }
155
164
  ```
156
165
 
157
- **Table with multiple `@table` arguments combined:**
166
+ **Exported table with cache control:**
158
167
 
159
168
  ```graphql
160
- type Event @table(database: "analytics", expiration: 86400) {
169
+ type Product @table(cacheControl: "public, max-age=60") @export {
161
170
  id: Long @primaryKey
162
- name: String @indexed
171
+ name: String
172
+ price: Float
163
173
  }
164
174
  ```
165
175
 
166
- **Exported table with overridden table name:**
176
+ **Table with lifecycle fields and indexing:**
167
177
 
168
178
  ```graphql
169
- type Product @table(table: "products") @export(name: "products") {
179
+ type Event @table(database: "analytics", expiration: 86400) {
170
180
  id: Long @primaryKey
171
- category: String @indexed
172
- price: Float @indexed
181
+ name: String @indexed
182
+ createdAt: Long @createdTime
183
+ updatedAt: Long @updatedTime
173
184
  }
174
185
  ```
175
186
 
176
- **Table with timestamps and per-record expiration:**
187
+ **Session table with per-record expiration:**
177
188
 
178
189
  ```graphql
179
190
  type Session @table {
180
191
  id: ID @primaryKey
181
192
  token: String
182
- createdAt: Long @createdTime
183
- updatedAt: Long @updatedTime
184
193
  expiresAt: Long @expiresAt
185
194
  }
186
195
  ```
187
196
 
188
- **Table with a hidden internal field:**
197
+ **Sealed table preventing extra properties:**
189
198
 
190
199
  ```graphql
191
- type Customer @table {
200
+ type StrictRecord @table @sealed {
192
201
  id: Long @primaryKey
193
202
  name: String
194
-
195
- """
196
- Internal — do not surface to external consumers.
197
- """
198
- creditScore: Int @hidden
199
203
  }
200
204
  ```
201
205
 
206
+ **`config.yaml` schema registration:**
207
+
208
+ ```yaml
209
+ graphqlSchema:
210
+ files: 'schema.graphql'
211
+ ```
212
+
202
213
  #### Notes
203
214
 
204
- - `@table`, `@export`, `@sealed`, and `@hidden` are type-level directives; `@primaryKey`, `@indexed`, `@embed`, `@createdTime`, `@updatedTime`, `@expiresAt`, and `@hidden` are field-level directives.
205
- - `eviction` removes non-indexed record data but does **not** remove a record from its secondary indexes. Indexes remain functional for evicted records; Harper fetches the full record on demand when a query matches an evicted entry.
206
- - `scanInterval` is clock-aligned to the server's local timezone, not startup-aligned. The server's startup time does not affect when eviction runs.
207
- - Replication is enabled by default. If you disable replication on a table and re-enable it later, it will not catch up on writes made while replication was disabled.
208
- - `@hidden` is a metadata-visibility directive only. Use `attribute_permissions` on roles to enforce data access control.
209
- - A full-record `put` that omits an `@expiresAt` field clears it; a `patch` of other fields preserves it.
215
+ - Schemas are flexible by default — records may include additional properties beyond those declared. Use `@sealed` to prevent this.
216
+ - Use unique `database` names in plugins or applications to avoid table naming collisions, since all tables default to the `"data"` database.
217
+ - Replication is enabled by default. If you disable replication and re-enable it later, the table will not catch up on writes made while replication was disabled.
218
+ - `@hidden` (type or field) is a metadata-visibility directive only. Use table-level role permissions and `attribute_permissions` whitelists to restrict actual data access.
219
+ - `@export` absence causes 404 on REST/MQTT routes but does not protect data from the Operations API or SQL.
220
+ - The `cacheControl` argument emits headers only on anonymous (unauthenticated) GET/HEAD 200/304 responses. Authenticated responses receive `Cache-Control: private, no-cache`.
221
+ - `randomAccessFields` on `@table` pins the record encoding at table creation time. Editing the argument later does not repin an existing table.
210
222
 
211
223
  ### 1.3 Defining Relationships Between Tables in Harper
212
224
 
@@ -754,15 +766,15 @@ export class Photo extends tables.Photo {
754
766
 
755
767
  ### 2.1 Automatic APIs
756
768
 
757
- Instructions for the agent to follow when enabling and using Harper's automatically generated REST and WebSocket APIs.
769
+ Instructions for the agent to follow when using Harper's automatically generated REST and WebSocket APIs for exported tables and resources.
758
770
 
759
771
  #### When to Use
760
772
 
761
- Apply this rule when adding REST or WebSocket API access to Harper tables or custom resources. Use it when configuring `config.yaml` to expose endpoints, mapping HTTP methods to resource operations, or implementing real-time WebSocket connections on a resource class.
773
+ Apply this rule when enabling HTTP REST endpoints or WebSocket subscriptions for Harper tables without writing custom handler code. Use it whenever a schema type needs to be served over HTTP, when configuring real-time subscriptions, or when setting up conditional caching behavior for REST responses.
762
774
 
763
775
  #### How It Works
764
776
 
765
- 1. **Enable the REST plugin**: Add `rest: true` to your application's `config.yaml`. This activates the HTTP REST interface on the application server port (default `9926`) and enables WebSocket support automatically.
777
+ 1. **Enable REST in `config.yaml`**: Add `rest: true` to the application configuration file. This registers REST endpoints and, by default, WebSocket subscriptions for all exported resources.
766
778
 
767
779
  ```yaml
768
780
  rest: true
@@ -776,26 +788,47 @@ Apply this rule when adding REST or WebSocket API access to Harper tables or cus
776
788
  webSocket: false # disables automatic WebSocket support (enabled by default)
777
789
  ```
778
790
 
779
- 2. **Export your resource in the schema**: Tables are not exposed by default. Use the `@export` directive in your schema definition to expose a table as a REST endpoint. The exported name defines the base URL path.
791
+ 2. **Export the table in the schema**: Add `@export` to the type definition. Without `@export`, Harper registers no REST route and callers receive `404`. Without `rest: true`, even an exported table does not respond to HTTP requests. Both are required.
792
+
793
+ ```graphql
794
+ type Product @table @export {
795
+ id: Long @primaryKey
796
+ name: String
797
+ price: Float
798
+ }
799
+ ```
800
+
801
+ Reference the schema file in `config.yaml`:
802
+
803
+ ```yaml
804
+ graphqlSchema:
805
+ files: schema.graphql
806
+ rest: true
807
+ ```
808
+
809
+ 3. **Use the automatically registered endpoints**: Harper serves the following endpoints on the application HTTP server port (default `9926`) with no route definitions or handler code required.
810
+
811
+ | Endpoint | Description |
812
+ | ---------------------------- | --------------------------------------------------------------------------- |
813
+ | `GET /Product` | Returns resource description (table name, database, attributes) |
814
+ | `GET /Product/` | Returns the record collection; append query parameters to filter |
815
+ | `GET /Product/{id}` | Returns a single record by primary key; `404` if not found |
816
+ | `GET /Product/{id}.property` | Returns a single declared property of one record |
817
+ | `POST /Product/` | Creates a record; responds `201`; primary key returned in `Location` header |
818
+ | `PUT /Product/{id}` | Creates or replaces the record at `{id}` (upsert) |
819
+ | `PATCH /Product/{id}` | Merges body into existing record (shallow, top-level only) |
820
+ | `DELETE /Product/{id}` | Deletes the record at `{id}` |
821
+ | `DELETE /Product/?query` | Deletes every record matching the query |
822
+
823
+ 4. **Handle `POST` primary key and `Location`**: On a successful `POST`, the new record's primary key is returned in the `Location` response header — the value the body supplied if it carried the primary-key property, otherwise a Harper-assigned key. The header carries the bare key value, not a URL.
780
824
 
781
- 3. **Use the correct URL structure**: Target resources using these path conventions:
825
+ 5. **Understand `PUT` write behavior**: `PUT` replaces the stored record exactly. Three exceptions always apply: a `@createdTime` attribute keeps the original value, an `@updatedTime` attribute is re-stamped with the time of the write, and the primary key is forced to match the `{id}` in the URL.
782
826
 
783
- | Path | Description |
784
- | -------------------------------------------- | ----------------------------------------------------------- |
785
- | `/my-resource` | Returns resource metadata |
786
- | `/my-resource/` | Collection — all records; append query parameters to search |
787
- | `/my-resource/record-id` | Specific record by primary key |
788
- | `/my-resource/record-id/` | Collection of records with the given id prefix |
789
- | `/my-resource/record-id/with/multiple/parts` | Record id with multiple path segments |
827
+ 6. **Use conditional requests for caching**: GET responses include an `ETag` header encoding the record's version/last-modification time. Send `If-None-Match` on subsequent requests with the cached `ETag` value. If the record has not changed, Harper returns `304 Not Modified` with no body.
790
828
 
791
- 4. **Map operations to HTTP methods**: Each HTTP method maps to a resource method:
792
- - **GET** — Retrieve a record or search. Calls `get()`. Responses include an `ETag` header; send `If-None-Match` on subsequent requests to receive `304 Not Modified` when unchanged.
793
- - **PUT** — Create or replace a record (upsert). Calls `put(record)`. The stored record exactly matches the submitted body; omitted properties are removed.
794
- - **POST** — Create a record without specifying a primary key. Calls `post(data)`. The assigned key is returned in the `Location` response header.
795
- - **PATCH** — Partially update a record, merging only provided top-level properties. Calls the resource's patch handler. Merge is **shallow** — nested objects are replaced entirely, not deep-merged.
796
- - **DELETE** — Delete a record by id or all records matching a query.
829
+ 7. **Select content type with `Accept`**: Use the `Accept` header to request a specific response format. The suffixes `.json`, `.cbor`, `.msgpack`, and `.csv` are reserved as content-type selectors on property paths and take precedence over property names. See [querying-rest-apis.md](querying-rest-apis.md) for query syntax details.
797
830
 
798
- 5. **Connect via WebSocket**: A WebSocket connection to a resource URL subscribes to that resource and streams change events. See [real-time-apps.md](real-time-apps.md) for full real-time patterns.
831
+ 8. **Connect via WebSocket**: WebSocket support is enabled automatically when `rest` is enabled. Connecting to a resource URL subscribes to changes for that resource. See [real-time-apps.md](real-time-apps.md) for real-time patterns.
799
832
 
800
833
  ```javascript
801
834
  let ws = new WebSocket('wss://server/my-resource/341');
@@ -804,29 +837,84 @@ Apply this rule when adding REST or WebSocket API access to Harper tables or cus
804
837
  };
805
838
  ```
806
839
 
807
- 6. **Implement a custom `connect()` handler** on a resource class to control WebSocket behavior. The method receives `incomingMessages` and must return an async iterable producing messages to send to the client.
840
+ 9. **Implement a custom `connect()` handler** when default subscription behavior is insufficient. The method must return an async iterable that produces messages to send to the client.
808
841
 
809
- 7. **Retrieve the OpenAPI spec**: Harper auto-generates an OpenAPI specification for all exported resources, available at:
810
-
811
- ```
812
- GET /openapi
842
+ ```javascript
843
+ export class Echo extends Resource {
844
+ async *connect(incomingMessages) {
845
+ for await (let message of incomingMessages) {
846
+ yield message; // echo each message back
847
+ }
848
+ }
849
+ }
813
850
  ```
814
851
 
815
852
  #### Examples
816
853
 
817
- **Simple echo WebSocket server**:
854
+ ##### Full schema and config setup
818
855
 
819
- ```javascript
820
- export class Echo extends Resource {
821
- async *connect(incomingMessages) {
822
- for await (let message of incomingMessages) {
823
- yield message; // echo each message back
824
- }
825
- }
856
+ ```graphql
857
+ # schema.graphql
858
+ type Product @table @export {
859
+ id: Long @primaryKey
860
+ name: String
861
+ price: Float
826
862
  }
827
863
  ```
828
864
 
829
- **Custom `connect()` using the default iterable with `send()` and `close` event**:
865
+ ```yaml
866
+ # config.yaml
867
+ graphqlSchema:
868
+ files: schema.graphql
869
+ rest: true
870
+ ```
871
+
872
+ ##### Conditional GET with ETag caching
873
+
874
+ ```
875
+ GET /Product/123
876
+ # Response includes:
877
+ # ETag: "abc123"
878
+
879
+ GET /Product/123
880
+ If-None-Match: "abc123"
881
+ # Response: 304 Not Modified (no body transferred)
882
+ ```
883
+
884
+ ##### POST and read the Location header
885
+
886
+ ```
887
+ POST /Product/
888
+ Content-Type: application/json
889
+
890
+ { "name": "Widget", "price": 9.99 }
891
+
892
+ # Response:
893
+ # 201 Created
894
+ # Location: 7f3a9c
895
+ ```
896
+
897
+ ##### PATCH (shallow merge only)
898
+
899
+ ```
900
+ PATCH /Product/123
901
+ Content-Type: application/json
902
+
903
+ { "price": 12.99 }
904
+ ```
905
+
906
+ Only `price` is updated; other top-level properties are preserved. Nested objects in the body replace the stored sub-object wholesale — deep merge does not occur.
907
+
908
+ ##### Request MessagePack response
909
+
910
+ ```
911
+ GET /Product/123
912
+ Accept: application/msgpack
913
+ ```
914
+
915
+ Alternatively, use the `.msgpack` suffix on the URL path where supported.
916
+
917
+ ##### WebSocket with custom outgoing messages
830
918
 
831
919
  ```javascript
832
920
  export class Example extends Resource {
@@ -838,7 +926,7 @@ export class Example extends Resource {
838
926
  }, 1000);
839
927
 
840
928
  incomingMessages.on('data', (message) => {
841
- outgoingMessages.send(message); // echo incoming messages
929
+ outgoingMessages.send(message);
842
930
  });
843
931
 
844
932
  outgoingMessages.on('close', () => {
@@ -850,37 +938,31 @@ export class Example extends Resource {
850
938
  }
851
939
  ```
852
940
 
853
- **Common REST operations**:
854
-
855
- ```
856
- GET /MyTable/123
857
- GET /MyTable/?name=Harper
858
- PUT /MyTable/123
859
- PATCH /MyTable/123
860
- DELETE /MyTable/?status=archived
861
- ```
941
+ ##### Disable WebSocket while keeping REST
862
942
 
863
- ```json
864
- { "name": "some data" }
943
+ ```yaml
944
+ rest:
945
+ webSocket: false
865
946
  ```
866
947
 
867
948
  #### Notes
868
949
 
869
- - `rest: true` is the minimal config to enable both REST and WebSocket support. Set `webSocket: false` under the `rest` key to disable WebSocket only.
870
- - The `@export` directive in the schema is required for any table to appear as a REST endpoint — tables are not exported by default.
871
- - PATCH merges are shallow (top-level only). Nested objects in the request body replace the entire existing sub-object. Dot-path keys (e.g., `"settings.theme"`) are stored as literal keys, not interpreted as paths.
872
- - For MQTT over WebSocket, set the sub-protocol header `Sec-WebSocket-Protocol: mqtt`.
873
- - In distributed environments, non-retained messages are delivered in arrival order; retained messages (PUT/updated records) keep only the latest timestamp as the winning record.
874
- - For full query syntax on GET and DELETE, see [querying-rest-apis.md](querying-rest-apis.md).
875
- - For building real-time features with WebSocket subscriptions, see [real-time-apps.md](real-time-apps.md).
950
+ - The trailing slash is significant: `/Product` addresses the resource itself; `/Product/` addresses its record collection. `POST /Product` (no trailing slash) returns `404`.
951
+ - `HEAD` is served as `GET` with the body omitted. `QUERY` is accepted on the collection path and reads its search from the request body.
952
+ - A `POST` to an existing primary key fails with `409` — it does not overwrite.
953
+ - A component directory with **no configuration file** gets REST enabled by Harper's built-in default. As soon as a `config.yaml` exists it is used verbatim — add `rest: true` explicitly or REST is off.
954
+ - Do not apply `@export` to a schema type and also export a same-named JavaScript subclass of that table — this produces conflicting endpoints.
955
+ - Server-Sent Events subscriptions are served on the same paths, negotiated via `Accept: text/event-stream`. They are not affected by the `webSocket` option.
956
+ - Every non-hidden exported resource is included in the generated OpenAPI document at `GET /openapi`. Mark a type `@hidden` or set `static hidden = true` on a programmatic Resource to omit it.
957
+ - MQTT over WebSockets requires the sub-protocol header `Sec-WebSocket-Protocol: mqtt`.
876
958
 
877
- ### 2.2 Querying Harper REST APIs
959
+ ### 2.2 Querying REST APIs
878
960
 
879
- Instructions for the agent to filter, sort, select, and paginate records using Harper's URL-based query language on REST collection endpoints.
961
+ Instructions for the agent to filter, sort, select, and paginate Harper REST API collections using URL query parameters.
880
962
 
881
963
  #### When to Use
882
964
 
883
- Apply this rule when building or debugging REST API calls against Harper tables that require filtering by attribute values, comparison ranges, sorting, field selection, or result pagination. This rule also covers OR logic, grouping, and querying across related tables via dot-syntax joins. See [automatic-apis.md](automatic-apis.md) for how Harper exposes tables as REST endpoints.
965
+ Apply this rule whenever building or modifying code that queries Harper REST collection endpoints. Use it when you need to filter records by attribute values, apply comparison operators, sort or paginate results, or join across related tables. See [automatic-apis.md](automatic-apis.md) for how Harper exposes tables as REST endpoints.
884
966
 
885
967
  #### How It Works
886
968
 
@@ -891,13 +973,7 @@ Apply this rule when building or debugging REST API calls against Harper tables
891
973
  GET /Product/?category=software&inStock=true
892
974
  ```
893
975
 
894
- 2. **Filter for null values**: Use `=null` as the value to match null or non-null records.
895
-
896
- ```
897
- GET /Product/?discount=null
898
- ```
899
-
900
- 3. **Apply comparison operators (FIQL syntax)**: Use FIQL operators in query parameters for range and string matching.
976
+ 2. **Apply comparison operators (FIQL syntax)**: Use FIQL operators in the query string for numeric, string, and date comparisons.
901
977
 
902
978
  | Operator | Meaning |
903
979
  | -------------------- | -------------------------------------- |
@@ -926,13 +1002,13 @@ Apply this rule when building or debugging REST API calls against Harper tables
926
1002
  GET /Product/?listDate=gt=2017-03-08T09%3A30%3A00.000Z
927
1003
  ```
928
1004
 
929
- 4. **Chain conditions for range queries**: Omit the attribute name on the second condition to apply it to the same attribute. Only `gt`/`ge` combined with `lt`/`le` is supported for chaining.
1005
+ 3. **Chain conditions for range queries**: Omit the attribute name on the second condition to apply it to the same attribute. Only `gt`/`ge` combined with `lt`/`le` is supported.
930
1006
 
931
1007
  ```
932
1008
  GET /Product/?price=gt=100&lt=200
933
1009
  ```
934
1010
 
935
- 5. **Apply type conversion**: For FIQL comparators, Harper converts values automatically. Use explicit prefixes to force a type.
1011
+ 4. **Apply type conversion**: For FIQL comparators, Harper converts values automatically. Use explicit prefixes to force a type.
936
1012
 
937
1013
  | Syntax | Behavior |
938
1014
  | ----------------------------------------- | ------------------------------------------- |
@@ -946,32 +1022,26 @@ Apply this rule when building or debugging REST API calls against Harper tables
946
1022
 
947
1023
  For strict operators (`=`, `===`, `!==`), no automatic type conversion is applied.
948
1024
 
949
- 6. **Combine conditions with OR logic**: Use `|` instead of `&` to express OR between conditions.
1025
+ 5. **Combine conditions with OR logic**: Use `|` instead of `&`.
950
1026
 
951
1027
  ```
952
1028
  GET /Product/?rating=5|featured=true
953
1029
  ```
954
1030
 
955
- 7. **Group conditions**: Use parentheses or square brackets to control evaluation order. Prefer square brackets when building queries from user input, since `[` and `]` are safely URI-encoded.
1031
+ 6. **Group conditions**: Use parentheses or square brackets to control order of operations. Prefer square brackets when constructing queries from user input, since standard URI encoding safely encodes `[` and `]`.
956
1032
 
957
1033
  ```
958
1034
  GET /Product/?rating=5|(price=gt=100&price=lt=200)
959
1035
  GET /Product/?rating=5&[tag=fast|tag=scalable|tag=efficient]
960
1036
  ```
961
1037
 
962
- Build grouped queries in JavaScript:
1038
+ Construct from JavaScript:
963
1039
 
964
1040
  ```javascript
965
1041
  let url = `/Product/?rating=5&[${tags.map(encodeURIComponent).join('|')}]`;
966
1042
  ```
967
1043
 
968
- Nest groups for complex conditions:
969
-
970
- ```
971
- GET /Product/?price=lt=100|[rating=5&[tag=fast|tag=scalable|tag=efficient]&inStock=true]
972
- ```
973
-
974
- 8. **Select specific properties with `select(`**: Append `select(...)` as a query function separated by `&` to control which fields are returned.
1044
+ 7. **Select specific properties with `select(`**: Append `select(...)` as a query function separated by `&`.
975
1045
 
976
1046
  | Syntax | Returns |
977
1047
  | -------------------------------------- | ------------------------------------------- |
@@ -981,65 +1051,47 @@ Apply this rule when building or debugging REST API calls against Harper tables
981
1051
  | `?select(property1,)` | Objects with a single specified property |
982
1052
  | `?select(property{subProp1,subProp2})` | Nested objects with specific sub-properties |
983
1053
 
984
- ```
985
- GET /Product/?category=software&select(name)
986
- GET /Product/?brand.name=Microsoft&select(name,brand{name})
987
- ```
988
-
989
- 9. **Paginate results with `limit(`**: Use `limit(end)` or `limit(start,end)` to restrict the number of records returned.
990
-
991
- ```
992
- GET /Product/?rating=gt=3&inStock=true&select(rating,name)&limit(20)
993
- GET /Product/?rating=gt=3&limit(10,30)
994
- ```
995
-
996
- 10. **Sort results with `sort(`**: Use `sort(property)` or `sort(+property,-property,...)` to order results. Prefix `+` or no prefix = ascending; `-` = descending.
1054
+ 8. **Paginate with `limit(`**: Use `limit(end)` or `limit(start,end)` to control result count and offset.
997
1055
 
998
- ```
999
- GET /Product/?rating=gt=3&sort(+name)
1000
- GET /Product/?sort(+rating,-price)
1001
- ```
1056
+ 9. **Sort with `sort(`**: Use `sort(property)` or `sort(+property,-property,...)`. Prefix `+` or no prefix = ascending; `-` = descending.
1002
1057
 
1003
- 11. **Query across relationships using dot-syntax**: Filter on related table attributes using dot-chained property names. Relationships must be defined in the schema with `@relationship`.
1058
+ 10. **Query across relationships**: Use dot-syntax to filter by related table attributes. Relationships must be defined in the schema using `@relationship`. Relationship attributes are not included by default — use `select()` to include them.
1004
1059
 
1005
1060
  ```
1006
- GET /Product/?brand.name=Microsoft
1007
- GET /Brand/?products.name=Keyboard
1008
- ```
1009
-
1010
- Use `select()` to include relationship attributes in the response (they are excluded by default):
1011
-
1012
- ```
1013
- GET /Product/?brand.name=Microsoft&select(name,brand)
1014
1061
  GET /Product/?brand.name=Microsoft&select(name,brand{name})
1015
1062
  ```
1016
1063
 
1017
- 12. **Access a specific property by URL**: Append `.propertyName` to a record ID in the URL path. Only works for properties declared in the schema.
1064
+ 11. **Query for null values**: Use `=null` as the value to match null or non-null records.
1018
1065
  ```
1019
- GET /MyTable/123.propertyName
1066
+ GET /Product/?discount=null
1020
1067
  ```
1021
1068
 
1022
1069
  #### Examples
1023
1070
 
1024
- **Range filter with select and limit**:
1071
+ **Filter with comparison operators and select:**
1025
1072
 
1026
1073
  ```
1027
- GET /Product/?category=software&price=gt=100&price=lt=200&select(name,price)&limit(20)
1074
+ GET /Product/?category=software&price=gt=100&price=lt=200&select(name,price)
1028
1075
  ```
1029
1076
 
1030
- **Sort and paginate**:
1077
+ **Paginate and sort:**
1031
1078
 
1032
1079
  ```
1033
- GET /Product/?rating=gt=3&sort(+rating,-price)&limit(10,30)
1080
+ GET /Product/?rating=gt=3&inStock=true&select(rating,name)&limit(20)
1081
+ GET /Product/?rating=gt=3&limit(10,30)
1082
+ GET /Product/?rating=gt=3&sort(+name)
1083
+ GET /Product/?sort(+rating,-price)
1034
1084
  ```
1035
1085
 
1036
- **OR with grouping**:
1086
+ **OR logic with grouping:**
1037
1087
 
1038
1088
  ```
1039
1089
  GET /Product/?price=lt=100|[rating=5&[tag=fast|tag=scalable|tag=efficient]&inStock=true]
1040
1090
  ```
1041
1091
 
1042
- **Join query with nested select** — schema first:
1092
+ **Relationship join with nested select:**
1093
+
1094
+ Define the schema:
1043
1095
 
1044
1096
  ```graphql
1045
1097
  type Product @table @export {
@@ -1055,13 +1107,14 @@ type Brand @table @export {
1055
1107
  }
1056
1108
  ```
1057
1109
 
1058
- Then query:
1110
+ Query with join:
1059
1111
 
1060
1112
  ```
1061
1113
  GET /Product/?brand.name=Microsoft&select(name,brand{name,id})
1114
+ GET /Brand/?products.name=Keyboard
1062
1115
  ```
1063
1116
 
1064
- **Many-to-many relationship** — schema:
1117
+ **Many-to-many relationship:**
1065
1118
 
1066
1119
  ```graphql
1067
1120
  type Product @table @export {
@@ -1072,26 +1125,23 @@ type Product @table @export {
1072
1125
  }
1073
1126
  ```
1074
1127
 
1075
- Query:
1076
-
1077
1128
  ```
1078
1129
  GET /Product/?resellers.name=Cool Shop&select(id,name,resellers{name,id})
1079
1130
  ```
1080
1131
 
1081
- **Date range with URL-encoded colons**:
1132
+ **Access a specific property by record ID:**
1082
1133
 
1083
1134
  ```
1084
- GET /Product/?listDate=gt=2017-03-08T09%3A30%3A00.000Z
1135
+ GET /MyTable/123.propertyName
1085
1136
  ```
1086
1137
 
1087
1138
  #### Notes
1088
1139
 
1089
- - All filtered attributes must be indexed unless at least one other attribute in the same query is indexed.
1090
- - Null queries (`?attr=null`) require indexes created after null indexing support was added. Rebuild existing indexes (remove and re-add) to enable null queries on them.
1091
- - When selecting a related attribute without filtering on it, the join behaves as a LEFT JOIN — the relationship property is omitted if the foreign key is null or references a non-existent record.
1092
- - The array order of foreign key values (e.g., `resellerIds`) is preserved when resolving many-to-many relationships.
1093
- - Square brackets (`[`, `]`) are preferred over parentheses for grouping when constructing queries programmatically, because standard URI encoding safely encodes them.
1094
- - `directURLMapping: true` can be set on a resource to change URL path handling semantics; see your schema configuration for details.
1140
+ - Only indexed attributes can be used as the primary filter attribute; when combining multiple attributes, only one needs to be indexed.
1141
+ - Relationship attributes are excluded from responses by default. Always use `select(` to include them.
1142
+ - When selecting a related attribute without filtering on it, the behavior is a LEFT JOIN — the property is omitted if the foreign key is null or references a non-existent record.
1143
+ - The suffixes `.json`, `.cbor`, `.msgpack`, and `.csv` in URL paths are reserved as content-type selectors and take precedence over a property of the same name.
1144
+ - Square brackets are preferred over parentheses when building grouped queries programmatically, because `[` and `]` are safely URL-encoded by standard encoding functions while `(` is not.
1095
1145
 
1096
1146
  ### 2.3 Real-Time Apps with WebSockets and Pub/Sub
1097
1147
 
@@ -1187,15 +1237,15 @@ export class Example extends Resource {
1187
1237
 
1188
1238
  ### 2.4 Checking Authentication
1189
1239
 
1190
- Instructions for the agent to follow when handling user authentication and session management inside Harper Resources.
1240
+ Instructions for the agent to handle user authentication, sessions, and JWT token issuance in Harper Resources.
1191
1241
 
1192
1242
  #### When to Use
1193
1243
 
1194
- Apply this rule when implementing authentication checks, login/logout flows, or token issuance inside a custom Resource. Use it any time a Resource needs to identify the current user, establish a session, or issue JWTs to clients. See [custom-resources.md](custom-resources.md) for the general Resource authoring pattern.
1244
+ Apply this rule when implementing login/logout flows, protecting Resource endpoints by checking the current user, issuing or refreshing JWT tokens, or configuring token expiry in Harper. Use it whenever a custom Resource needs to authenticate callers or mint credentials for downstream consumers. See [custom-resources.md](custom-resources.md) for the broader Resource authoring context.
1195
1245
 
1196
1246
  #### How It Works
1197
1247
 
1198
- 1. **Check the current user** with `getCurrentUser()`. Call it inside any Resource method to retrieve the authenticated user or `undefined` if no user is authenticated. Guard protected endpoints by returning a `401` when the result is `undefined`.
1248
+ 1. **Check the current authenticated user**: Call `getCurrentUser()` inside any Resource method. It returns the user object (with `username`, `role`, and `role.permission`) or `undefined` if unauthenticated. Guard endpoints by returning a 401 when no user is present.
1199
1249
 
1200
1250
  ```javascript
1201
1251
  async get(target) {
@@ -1205,20 +1255,9 @@ Apply this rule when implementing authentication checks, login/logout flows, or
1205
1255
  }
1206
1256
  ```
1207
1257
 
1208
- The returned object exposes `username`, `role`, and `role.permission` flags.
1209
-
1210
- 2. **Enable sessions** before using session-based login. Set `authentication.enableSessions: true` in `harperdb-config.yaml`:
1211
-
1212
- ```yaml
1213
- authentication:
1214
- enableSessions: true
1215
- ```
1216
-
1217
- 3. **Access login and session helpers** via `getContext()`. The context object exposes `context.login` and `context.session` for sign-in/out flows.
1218
- - Call `context.login(username, password)` to verify credentials and establish a session cookie on success.
1219
- - To end a session, delete it via `context.session.delete(context.session.id)`.
1258
+ 2. **Enable sessions before using login/logout**: Set `authentication.enableSessions: true` in `harperdb-config.yaml`. Without this, `context.login` and `context.session` are unavailable.
1220
1259
 
1221
- 4. **Implement sign-in and sign-out Resources** using the context helpers:
1260
+ 3. **Implement login via `getContext()`**: Call `this.getContext()` to obtain the request context, then call `context.login(username, password)` to verify credentials and establish a session cookie.
1222
1261
 
1223
1262
  ```javascript
1224
1263
  export class SignIn extends Resource {
@@ -1232,7 +1271,11 @@ Apply this rule when implementing authentication checks, login/logout flows, or
1232
1271
  return new Response('Logged in', { status: 200 });
1233
1272
  }
1234
1273
  }
1274
+ ```
1275
+
1276
+ 4. **Implement logout**: Delete the session via `context.session.delete(context.session.id)`.
1235
1277
 
1278
+ ```javascript
1236
1279
  export class SignOut extends Resource {
1237
1280
  async post() {
1238
1281
  const context = this.getContext();
@@ -1243,7 +1286,96 @@ Apply this rule when implementing authentication checks, login/logout flows, or
1243
1286
  }
1244
1287
  ```
1245
1288
 
1246
- 5. **Issue JWTs for non-browser clients** (CLI tools, mobile apps, service-to-service). Cookie-based sessions are intended for browser clients. For other clients, mint tokens programmatically using `server.operation()`:
1289
+ Cookie-based sessions are intended for browser clients. For non-browser clients, use JWT issuance (steps below).
1290
+
1291
+ 5. **Create authentication tokens**: Call `create_authentication_tokens` with credentials. No `Authorization` header is required for this operation.
1292
+
1293
+ ```json
1294
+ {
1295
+ "operation": "create_authentication_tokens",
1296
+ "username": "username",
1297
+ "password": "password"
1298
+ }
1299
+ ```
1300
+
1301
+ Response:
1302
+
1303
+ ```json
1304
+ {
1305
+ "operation_token": "<jwt-operation-token>",
1306
+ "refresh_token": "<jwt-refresh-token>"
1307
+ }
1308
+ ```
1309
+
1310
+ 6. **Use the operation token**: Pass it as a `Bearer` token in the `Authorization` header on subsequent requests.
1311
+
1312
+ ```bash
1313
+ curl --location --request POST 'http://localhost:9925' \
1314
+ --header 'Content-Type: application/json' \
1315
+ --header 'Authorization: Bearer <operation_token>' \
1316
+ --data-raw '{
1317
+ "operation": "search_by_hash",
1318
+ "schema": "dev",
1319
+ "table": "dog",
1320
+ "hash_values": [1],
1321
+ "get_attributes": ["*"]
1322
+ }'
1323
+ ```
1324
+
1325
+ 7. **Refresh an expired operation token**: When the `operation_token` expires, use `refresh_operation_token` and pass the `refresh_token` as `Bearer <refresh_token>`.
1326
+
1327
+ ```bash
1328
+ curl --location --request POST 'http://localhost:9925' \
1329
+ --header 'Content-Type: application/json' \
1330
+ --header 'Authorization: Bearer <refresh_token>' \
1331
+ --data-raw '{
1332
+ "operation": "refresh_operation_token"
1333
+ }'
1334
+ ```
1335
+
1336
+ Response:
1337
+
1338
+ ```json
1339
+ {
1340
+ "operation_token": "<new-jwt-operation-token>"
1341
+ }
1342
+ ```
1343
+
1344
+ When both tokens have expired, call `create_authentication_tokens` again with username and password.
1345
+
1346
+ 8. **Mint scoped tokens for limited access**: A super user can embed an inline role in `create_authentication_tokens` using the same `permission` structure as `add_role`. Include `expires_in` to control lifetime. Do not include a `password` field. The `username` is attribution only and must not match an existing user.
1347
+
1348
+ ```json
1349
+ {
1350
+ "operation": "create_authentication_tokens",
1351
+ "username": "reporting-service",
1352
+ "role": {
1353
+ "permission": {
1354
+ "operations": ["read_only"],
1355
+ "dev": {
1356
+ "tables": {
1357
+ "dog": {
1358
+ "read": true,
1359
+ "insert": false,
1360
+ "update": false,
1361
+ "delete": false,
1362
+ "attribute_permissions": []
1363
+ }
1364
+ }
1365
+ }
1366
+ }
1367
+ },
1368
+ "expires_in": "7d"
1369
+ }
1370
+ ```
1371
+
1372
+ Key constraints for scoped tokens:
1373
+ - No refresh token is issued; no user record is created.
1374
+ - Scoped tokens cannot be revoked before expiry — choose short `expires_in` values.
1375
+ - `super_user` and `cluster_user` are always forced to `false` in the embedded role.
1376
+ - In mixed-version clusters, only nodes with scoped-token support accept these tokens; older nodes return 401.
1377
+
1378
+ 9. **Issue tokens from a custom Resource using `server.operation`**: Import `server` from `harper` and call `server.operation()` to mint tokens programmatically. Pass `true` as the **third argument** to run the operation as the current authenticated user; omit it when supplying credentials directly.
1247
1379
 
1248
1380
  ```javascript
1249
1381
  import { Resource, server } from 'harper';
@@ -1287,31 +1419,50 @@ Apply this rule when implementing authentication checks, login/logout flows, or
1287
1419
  }
1288
1420
  ```
1289
1421
 
1290
- Pass `true` as the third argument to `server.operation()` when the operation should run as the current authenticated user. Omit it or pass `false` when the operation supplies its own credentials.
1422
+ 10. **Configure token expiry**: Set timeouts in `harper-config.yaml` under the `authentication` section. Values follow the `jsonwebtoken` duration string format (e.g., `1d`, `12h`, `60m`).
1291
1423
 
1292
- 6. **Configure JWT token expiry** in `harperdb-config.yaml` under the `authentication` section:
1293
-
1294
- ```yaml
1295
- authentication:
1296
- operationTokenTimeout: 1d
1297
- refreshTokenTimeout: 30d
1298
- ```
1299
-
1300
- Duration strings follow the `jsonwebtoken` package format (e.g., `1d`, `12h`, `60m`).
1424
+ ```yaml
1425
+ authentication:
1426
+ operationTokenTimeout: 1d # Default: 1 day
1427
+ refreshTokenTimeout: 30d # Default: 30 days
1428
+ ```
1301
1429
 
1302
1430
  #### Examples
1303
1431
 
1304
- **Protecting a resource endpoint and returning user info:**
1432
+ ##### Full JWT flow via cURL
1305
1433
 
1306
- ```javascript
1307
- async get(target) {
1308
- const user = this.getCurrentUser();
1309
- if (!user) return new Response(null, { status: 401 });
1310
- return { username: user.username, role: user.role };
1311
- }
1312
- ```
1313
-
1314
- **Full session-based sign-in/sign-out flow:**
1434
+ ```bash
1435
+ # Step 1: Create tokens
1436
+ curl --location --request POST 'http://localhost:9925' \
1437
+ --header 'Content-Type: application/json' \
1438
+ --data-raw '{
1439
+ "operation": "create_authentication_tokens",
1440
+ "username": "username",
1441
+ "password": "password"
1442
+ }'
1443
+
1444
+ # Step 2: Use operation token
1445
+ curl --location --request POST 'http://localhost:9925' \
1446
+ --header 'Content-Type: application/json' \
1447
+ --header 'Authorization: Bearer <operation_token>' \
1448
+ --data-raw '{
1449
+ "operation": "search_by_hash",
1450
+ "schema": "dev",
1451
+ "table": "dog",
1452
+ "hash_values": [1],
1453
+ "get_attributes": ["*"]
1454
+ }'
1455
+
1456
+ # Step 3: Refresh when operation token expires
1457
+ curl --location --request POST 'http://localhost:9925' \
1458
+ --header 'Content-Type: application/json' \
1459
+ --header 'Authorization: Bearer <refresh_token>' \
1460
+ --data-raw '{
1461
+ "operation": "refresh_operation_token"
1462
+ }'
1463
+ ```
1464
+
1465
+ ##### Session-based login/logout Resource
1315
1466
 
1316
1467
  ```javascript
1317
1468
  export class SignIn extends Resource {
@@ -1336,30 +1487,14 @@ export class SignOut extends Resource {
1336
1487
  }
1337
1488
  ```
1338
1489
 
1339
- **JWT token refresh endpoint:**
1340
-
1341
- ```javascript
1342
- export class RefreshJWT extends Resource {
1343
- static async post(_target, data) {
1344
- const { refresh_token } = await data;
1345
- if (!refresh_token) {
1346
- return new Response('refresh_token required', { status: 400 });
1347
- }
1348
- const { operation_token } = await server.operation({
1349
- operation: 'refresh_operation_token',
1350
- refresh_token,
1351
- });
1352
- return { operation_token };
1353
- }
1354
- }
1355
- ```
1356
-
1357
1490
  #### Notes
1358
1491
 
1359
- - `getCurrentUser()` and `getContext()` are instance methods; call them with `this` inside non-static Resource methods.
1360
- - `enableSessions` must be `true` in config before `context.login` or `context.session` will function.
1361
- - Cookie-based sessions target browser clients. Use JWT issuance via `server.operation()` for all other client types.
1362
- - When both `operation_token` and `refresh_token` have expired, the client must call `create_authentication_tokens` again with credentials.
1492
+ - JWT authentication is **preferred over Basic Auth** when you want to avoid sending credentials on every request, when the client can store tokens, or when making multiple sequential requests. For simple or server-to-server scenarios, use Basic Authentication.
1493
+ - Always use **HTTPS** in production to protect tokens in transit.
1494
+ - Treat tokens like passwords. If a token is compromised, it remains valid until expiry — use shorter `operationTokenTimeout` values in high-security environments.
1495
+ - `enableSessions` must be `true` in config before `context.login` or `context.session` will work.
1496
+ - The `third argument` (`true`) to `server.operation` controls whether the operation runs as the current authenticated user. Omit it when the operation body supplies its own credentials.
1497
+ - Scoped tokens have a 12KB limit when encoded into an `Authorization` header.
1363
1498
 
1364
1499
  ## 3. Logic & Extension
1365
1500
 
@@ -1622,15 +1757,15 @@ if (!authorized) {
1622
1757
 
1623
1758
  ### 3.3 Programmatic Table Requests
1624
1759
 
1625
- Instructions for the agent to interact with Harper tables programmatically using the `tables` object and its query API.
1760
+ Instructions for the agent to interact with Harper tables programmatically using the `tables` object, including querying, transactions, and module integration.
1626
1761
 
1627
1762
  #### When to Use
1628
1763
 
1629
- Apply this rule when writing server-side code that reads from or writes to Harper tables directly — for example, in request handlers, background jobs, or SSR rendering — without going through the REST API. Use it whenever you need to construct queries with `conditions`, `select`, `sort`, or `search(`.
1764
+ Apply this rule when writing server-side Harper component code that reads from or writes to tables directly — bypassing REST endpoints — such as in request handlers, background jobs, timers, or SSR rendering. Use it whenever you need to construct queries with `conditions`, manage transactions explicitly, or perform CRDT-safe mutations.
1630
1765
 
1631
1766
  #### How It Works
1632
1767
 
1633
- 1. **Import `tables`**: Pull `tables` from the `harper` package. Each property on `tables` corresponds to a table defined in `schema.graphql`.
1768
+ 1. **Import `tables` from `harper`**: Access all tables in the default `data` database via the `tables` object. Each table defined with `@table` in `schema.graphql` is a property.
1634
1769
 
1635
1770
  ```javascript
1636
1771
  import { tables } from 'harper';
@@ -1638,7 +1773,7 @@ Apply this rule when writing server-side code that reads from or writes to Harpe
1638
1773
  // same as: databases.data.Product
1639
1774
  ```
1640
1775
 
1641
- 2. **Define your schema**: Declare tables with `@table` in `schema.graphql`. Each type becomes a property on `tables`.
1776
+ 2. **Define your schema with `@table`**: Tables must be declared in `schema.graphql`. Use `@indexed` on attributes you intend to sort or filter efficiently.
1642
1777
 
1643
1778
  ```graphql
1644
1779
  type Product @table {
@@ -1648,15 +1783,7 @@ Apply this rule when writing server-side code that reads from or writes to Harpe
1648
1783
  }
1649
1784
  ```
1650
1785
 
1651
- 3. **Create and modify records**: Use `create`, `patch`, and `get` for basic CRUD.
1652
-
1653
- ```javascript
1654
- const created = await Product.create({ name: 'Shirt', price: 9.5 });
1655
- await Product.patch(created.id, { price: Math.round(created.price * 0.8 * 100) / 100 });
1656
- const record = await Product.get(created.id);
1657
- ```
1658
-
1659
- 4. **Query with `search(`**: Pass a query object to `Product.search(query)`. It returns an async iterable.
1786
+ 3. **Use `search(` to query records**: Pass a Query object to `search(`. Iterate results with `for await`.
1660
1787
 
1661
1788
  ```javascript
1662
1789
  const query = {
@@ -1667,7 +1794,7 @@ Apply this rule when writing server-side code that reads from or writes to Harpe
1667
1794
  }
1668
1795
  ```
1669
1796
 
1670
- 5. **Build `conditions`**: Each condition object filters records. Nest conditions with `operator` for boolean logic.
1797
+ 4. **Build `conditions` arrays to filter**: Each condition object supports these properties:
1671
1798
 
1672
1799
  | Property | Description |
1673
1800
  | ------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------- |
@@ -1677,46 +1804,95 @@ Apply this rule when writing server-side code that reads from or writes to Harpe
1677
1804
  | `conditions` | Nested conditions array |
1678
1805
  | `operator` | `and` (default) or `or` for the nested `conditions` |
1679
1806
 
1680
- 6. **Use `select`** to control which properties are returned. Accepts an array of property names, a string for a single property, or nested objects for relationships.
1807
+ 5. **Apply `select` to shape results**: Return only the fields you need. Supports arrays, nested relationship selects, and special properties.
1681
1808
 
1682
1809
  ```javascript
1683
- // Array of names
1810
+ // Array of fields
1684
1811
  Product.search({ select: ['name', 'price'] });
1685
1812
 
1686
1813
  // Nested relationship select
1687
1814
  Book.get({ id: 42, select: ['id', 'title', { name: 'author', select: ['name'] }] });
1688
1815
  ```
1689
1816
 
1690
- Special `select` values:
1691
- - `$id` — returns the primary key regardless of its name
1692
- - `$updatedtime` — returns the last-updated timestamp
1693
- - `$distance` — returns the computed distance when querying a vector index
1694
-
1695
- 7. **Use `addTo`** for concurrent-safe numeric increments (CRDT incrementation, safe across threads and nodes).
1696
-
1697
- ```javascript
1698
- const record = await Product.update(target.id);
1699
- record.addTo('quantity', -1);
1700
- ```
1817
+ Special `select` values: `$id`, `$updatedtime`, `$distance`.
1701
1818
 
1702
- 8. **Apply `sort`, `limit`, and `offset`** for ordering and pagination. A `sort` attribute must be `@indexed`, or at least one `conditions` entry must be present. Pass `allowFullScan: true` to permit an unconditional ordered scan.
1819
+ 6. **Apply `sort` with an `@indexed` attribute**: Harper uses an index to provide sort order. Sort by an `@indexed` attribute without requiring a condition, or provide at least one condition when sorting by a non-indexed attribute.
1703
1820
 
1704
1821
  ```javascript
1822
+ // Sort by primary key with an open-ended condition to avoid scan error
1705
1823
  Product.search({
1706
1824
  conditions: [{ attribute: 'id', comparator: 'greater_than', value: '' }],
1707
1825
  sort: { attribute: 'id' },
1708
1826
  });
1709
1827
  ```
1710
1828
 
1711
- | Sort property | Description |
1712
- | ------------- | ---------------------------------------------------------- |
1713
- | `attribute` | Property name (or array for chained relationship property) |
1714
- | `descending` | Sort descending if `true` (default: `false`) |
1715
- | `next` | Secondary sort to resolve ties (same structure) |
1829
+ Sort object properties:
1830
+
1831
+ | Property | Description |
1832
+ | ------------ | -------------------------------------------------------- |
1833
+ | `attribute` | Property name or array for chained relationship property |
1834
+ | `descending` | Sort descending if `true` (default: `false`) |
1835
+ | `next` | Secondary sort to resolve ties (same structure) |
1836
+
1837
+ 7. **Use `limit` and `offset` for pagination**:
1838
+
1839
+ ```javascript
1840
+ Product.search({ conditions: [...], limit: 20, offset: 40 });
1841
+ ```
1842
+
1843
+ 8. **Use `explain` and `enforceExecutionOrder` for debugging**:
1844
+ - `explain: true` — returns conditions reordered as Harper will execute them.
1845
+ - `enforceExecutionOrder: true` — forces conditions to execute in the order supplied, disabling automatic re-ordering.
1846
+
1847
+ 9. **Use `addTo` for concurrent-safe numeric updates**: `addTo` uses CRDT incrementation, safe across threads and nodes.
1848
+
1849
+ ```javascript
1850
+ static async post(target, data) {
1851
+ const record = await this.update(target.id);
1852
+ record.addTo('quantity', -1); // decrement safely across nodes
1853
+ }
1854
+ ```
1855
+
1856
+ 10. **Wrap background work in `transaction()`**: HTTP handlers get a transaction automatically. Use `transaction()` explicitly for timers, background jobs, or any code outside a natural transaction context.
1857
+
1858
+ ```javascript
1859
+ import { tables } from 'harper';
1860
+ const { MyTable } = tables;
1861
+
1862
+ if (isMainThread) {
1863
+ setInterval(async () => {
1864
+ let data = await (await fetch('https://example.com/data')).json();
1865
+ transaction(async (txn) => {
1866
+ for (let item of data) {
1867
+ await MyTable.put(item, txn);
1868
+ }
1869
+ });
1870
+ }, 3600000); // every hour
1871
+ }
1872
+ ```
1873
+
1874
+ The `txn` object members:
1875
+
1876
+ | Member | Type | Description |
1877
+ | --------------------- | --------------- | ------------------------------------------------------ |
1878
+ | `commit()` | `() => Promise` | Commits the current transaction |
1879
+ | `abort()` | `() => void` | Aborts the transaction and resets it |
1880
+ | `resetReadSnapshot()` | `() => void` | Resets the read snapshot to the latest committed state |
1881
+ | `timestamp` | `number` | Timestamp associated with the current transaction |
1882
+
1883
+ 11. **Understand atomicity boundaries**: All tables within the same database share one transactional context — writes across multiple tables commit atomically. Tables in different databases each get their own transaction with no cross-database atomicity guarantee.
1884
+
1885
+ 12. **Keep `harper` external in bundlers**: When using SSR bundlers, mark `harper` as external so it resolves to the live runtime. In `vite.config`:
1886
+
1887
+ ```javascript
1888
+ ssr: {
1889
+ external: ['harper'];
1890
+ }
1891
+ ```
1716
1892
 
1717
1893
  #### Examples
1718
1894
 
1719
- **Nested conditions with `or`:**
1895
+ ##### Nested conditions query
1720
1896
 
1721
1897
  ```javascript
1722
1898
  Product.search({
@@ -1733,25 +1909,34 @@ Product.search({
1733
1909
  });
1734
1910
  ```
1735
1911
 
1736
- **Chained attribute reference (relationship join):**
1912
+ ##### Chained attribute reference (join/relationship)
1737
1913
 
1738
1914
  ```javascript
1739
1915
  Product.search({ conditions: [{ attribute: ['brand', 'name'], value: 'Harper' }] });
1740
1916
  ```
1741
1917
 
1742
- **Deep nested `select` across multiple relationships:**
1918
+ ##### Full CRUD sequence
1743
1919
 
1744
1920
  ```javascript
1745
- Product.search({
1746
- select: [
1747
- 'id',
1748
- 'name',
1749
- { name: 'segments', select: ['id', 'name', { name: 'client', select: ['id', 'name'] }] },
1750
- ],
1751
- });
1921
+ // Create a new record (id auto-generated)
1922
+ const created = await Product.create({ name: 'Shirt', price: 9.5 });
1923
+
1924
+ // Modify the record
1925
+ await Product.patch(created.id, { price: Math.round(created.price * 0.8 * 100) / 100 });
1926
+
1927
+ // Retrieve by primary key
1928
+ const record = await Product.get(created.id);
1929
+
1930
+ // Query with conditions
1931
+ const query = {
1932
+ conditions: [{ attribute: 'price', comparator: 'less_than', value: 8.0 }],
1933
+ };
1934
+ for await (const record of Product.search(query)) {
1935
+ // process record
1936
+ }
1752
1937
  ```
1753
1938
 
1754
- **SSR usage — read directly from `tables` in a render function:**
1939
+ ##### SSR rendering with `tables`
1755
1940
 
1756
1941
  ```typescript
1757
1942
  import { tables } from 'harper';
@@ -1762,14 +1947,23 @@ export async function render(url: string): Promise<string> {
1762
1947
  }
1763
1948
  ```
1764
1949
 
1950
+ ##### Mutable update with `addTo`
1951
+
1952
+ ```javascript
1953
+ const product = await Product.update(32);
1954
+ product.status = 'active';
1955
+ product.subtractFrom('quantity', 1);
1956
+ product.save();
1957
+ ```
1958
+
1765
1959
  #### Notes
1766
1960
 
1767
- - Scope destructive operations (`update`, `patch`, `delete`) with specific `conditions` and validate the affected set before writing. These operate on live data and are not easily reversible.
1768
- - Sorting by a bare `@primaryKey` with no conditions raises `HdbError: <attribute> is not indexed and not combined with any other conditions`. Add an open-ended condition or pass `allowFullScan: true`.
1769
- - Selecting a relationship field without filtering on it behaves as a **LEFT JOIN**. Adding a condition on a related attribute (e.g. `attribute: ['author', 'name']`) behaves as an **INNER JOIN**.
1770
- - A to-many relationship resolves to an array; `await` the property before iterating when needed.
1771
- - `tables` and `databases.data` reference the same live objects — a record written through one component is immediately visible to all others.
1772
- - Keep `harper` external when bundling for SSR (e.g. `ssr: { external: ['harper'] }` in `vite.config`) so it resolves to the runtime.
1961
+ - `tables` calls run in a trusted server-side context and do **not** automatically apply the target table's role permissions. Enforce authorization in your own application logic.
1962
+ - Destructive operations (`update`, `patch`, `delete`) act on live data and are not easily reversible. Always scope with specific `conditions`, validate the affected set before writing, and gate behind authorization controls.
1963
+ - Sorting by the bare `@primaryKey` alone with no conditions triggers `HdbError: <attribute> is not indexed and not combined with any other conditions`. Add an open-ended range condition or pass `allowFullScan: true` to permit an unconditional scan.
1964
+ - Selecting a relationship field without filtering on it behaves as a **LEFT JOIN**; adding a condition on a related attribute behaves as an **INNER JOIN**.
1965
+ - `transaction()` is safe to call defensively — if a transaction is already active on the context, it reuses it and executes the callback immediately.
1966
+ - Link the `harper` package for correct typings in standalone component directories: `npm link harper`.
1773
1967
 
1774
1968
  ### 3.4 TypeScript Type Stripping in Harper
1775
1969
 
@@ -1988,11 +2182,11 @@ await fetch('http://localhost:9926/JokeCache/1', {
1988
2182
 
1989
2183
  ### 4.1 Deploying to Harper Fabric
1990
2184
 
1991
- Instructions for the agent to follow when deploying a Harper application to a remote Harper Fabric cluster.
2185
+ Instructions for the agent to follow when deploying a Harper application to a remote Harper Fabric cloud cluster.
1992
2186
 
1993
2187
  #### When to Use
1994
2188
 
1995
- Apply this rule when deploying a Harper application to a remote Harper Fabric cluster or any remote Harper instance. This includes first-time deploys, redeployments, rollbacks, and CI/CD pipeline deployments. See [creating-a-fabric-account-and-cluster.md](creating-a-fabric-account-and-cluster.md) to set up a cluster before deploying.
2189
+ Apply this rule when deploying a Harper application to a remote Harper Fabric cluster or any remote Harper instance. This includes first-time deploys, redeployments, rollbacks, CI/CD pipeline deploys, and provisioning credentials for private repositories. See [creating-a-fabric-account-and-cluster.md](creating-a-fabric-account-and-cluster.md) for setting up the cluster before deploying.
1996
2190
 
1997
2191
  #### How It Works
1998
2192
 
@@ -2003,7 +2197,7 @@ Apply this rule when deploying a Harper application to a remote Harper Fabric cl
2003
2197
  # Provide cluster username and password when prompted
2004
2198
  ```
2005
2199
 
2006
- 2. **Deploy the application**: Run `harper deploy` with the required parameters. After logging in, no credentials need to be repeated.
2200
+ 2. **Deploy the application**: After login, run `harper deploy` without repeating credentials. Set `restart=true` and `replicated=true` for a production deploy.
2007
2201
 
2008
2202
  ```bash
2009
2203
  harper deploy \
@@ -2014,30 +2208,42 @@ Apply this rule when deploying a Harper application to a remote Harper Fabric cl
2014
2208
  replicated=true
2015
2209
  ```
2016
2210
 
2017
- 3. **Choose a package source**: Set the `package` parameter to any valid npm dependency value, or omit it to package and upload the current local directory.
2211
+ 3. **Use environment variables for CI/CD**: Instead of `harper login`, export credentials as environment variables before calling `harper deploy`.
2018
2212
 
2019
- | Value | Effect |
2020
- | ---------------------------------------------------- | ------------------------------------------------ |
2021
- | _(omitted)_ | Packages and deploys the current local directory |
2022
- | `"@harperdb/status-check"` | npm package |
2023
- | `"HarperDB/status-check"` | GitHub shorthand |
2024
- | `"https://github.com/HarperDB/status-check"` | GitHub URL |
2025
- | `"git+ssh://git@github.com:HarperDB/secret-app.git"` | Private repo via SSH |
2026
- | `"https://example.com/application.tar.gz"` | Tarball URL |
2213
+ ```bash
2214
+ export HARPER_CLI_USERNAME=<username>
2215
+ export HARPER_CLI_PASSWORD=<password>
2216
+ harper deploy \
2217
+ project=<name> \
2218
+ package=<package> \
2219
+ target=<remote> \
2220
+ restart=true \
2221
+ replicated=true
2222
+ ```
2223
+
2224
+ 4. **Choose a package source**: The `package` field accepts any valid npm dependency value. Select the form that matches your source:
2225
+
2226
+ | Source | `package` value |
2227
+ | ----------------------- | ---------------------------------------------------- |
2228
+ | Current local directory | Omit `package` |
2229
+ | npm package | `"@harperdb/status-check"` |
2230
+ | GitHub (public) | `"HarperFast/status-check"` or full URL |
2231
+ | Private repo (SSH) | `"git+ssh://git@github.com:HarperDB/secret-app.git"` |
2232
+ | Tarball | `"https://example.com/application.tar.gz"` |
2027
2233
 
2028
2234
  For git tags, use the `semver` directive:
2029
2235
 
2030
2236
  ```
2031
- HarperDB/application-template#semver:v1.0.0
2237
+ HarperFast/application-template#semver:v1.0.0
2032
2238
  ```
2033
2239
 
2034
- 4. **Deploy by reference (optional)**: Instead of uploading a snapshot, send a pinned git reference so the cluster fetches and builds that exact commit. Use `by_ref=true` to resolve the current commit from the local repository automatically.
2240
+ 5. **Deploy by reference for reproducible deploys**: Pass `by_ref=true` to send a pinned git SHA instead of uploading a snapshot. The cluster fetches and builds from that exact commit.
2035
2241
 
2036
2242
  ```bash
2037
2243
  harper deploy by_ref=true restart=true replicated=true
2038
2244
  ```
2039
2245
 
2040
- Use `ref` to deploy a specific commit, tag, or branch (resolved to a full SHA before sending):
2246
+ Use `ref` to target a specific commit, tag, or branch (resolved to a full SHA before sending):
2041
2247
 
2042
2248
  ```bash
2043
2249
  # Deploy a specific tag
@@ -2047,53 +2253,32 @@ Apply this rule when deploying a Harper application to a remote Harper Fabric cl
2047
2253
  harper deploy ref=9f8c2a1 restart=true replicated=true
2048
2254
  ```
2049
2255
 
2050
- **Key `by_ref` parameters:**
2051
-
2052
- | Parameter | Required | Description |
2053
- | ------------ | -------- | --------------------------------------------------------------------------------------------------------------------------- |
2054
- | `by_ref` | — | Build the package reference from the local repository |
2055
- | `ref` | optional | Deploy a specific commit, tag, or branch instead of `HEAD`. Implies `by_ref`. |
2056
- | `credential` | optional | Set to `true` to authenticate the clone with the stored credential for the repository's host. Omit for public repositories. |
2057
-
2058
- > Commit and push before deploying by reference. The cluster clones from the remote and only sees pushed commits. Run `git fetch` if the unpushed-commit check fires for a commit you know you pushed.
2256
+ **Key constraints for `ref` values:**
2257
+ - Must name something a clone can fetch: `refs/heads/*` and `refs/tags/*`, or a bare branch or tag name.
2258
+ - Anything else (e.g., `refs/pull/123/head`) is rejected up front.
2259
+ - If a ref can't be resolved, the deploy stops — run `git fetch` and retry, or pass a full commit SHA.
2260
+ - Commit and push before deploying: the cluster clones from the remote and only sees pushed commits.
2059
2261
 
2060
- 5. **Authenticate for CI/CD**: Use environment variables instead of `harper login` for automated pipelines.
2262
+ 6. **Deploy private repositories by reference**: Pass `credential=true` alongside `by_ref=true`. The CLI attaches a credentials reference; the cluster resolves the secret in memory at clone time — no token travels in the operation body or lands on disk.
2061
2263
 
2062
2264
  ```bash
2063
- export HARPER_CLI_USERNAME=<username>
2064
- export HARPER_CLI_PASSWORD=<password>
2065
- harper deploy \
2066
- project=<name> \
2067
- package=<package> \
2068
- target=<remote> \
2069
- restart=true \
2070
- replicated=true
2265
+ harper deploy by_ref=true credential=true restart=true replicated=true
2071
2266
  ```
2072
2267
 
2073
- 6. **Provision credentials for private repositories**: Run `harper deploy setup=true` once per component and source to provision a deploy credential. Run this with an administrative credential, not the CI identity.
2268
+ 7. **Provision a deploy credential for private sources**: Run `harper deploy setup=true` once per component and source. This is interactive and requires **super_user** — run it with an administrative credential, not the CI identity.
2074
2269
 
2075
2270
  ```bash
2076
2271
  harper deploy setup=true
2077
2272
  ```
2078
2273
 
2079
- Then pass `credential=true` on subsequent deploys:
2274
+ This command:
2275
+ 1. Fetches the cluster's public key with `get_secrets_public_key`.
2276
+ 2. Encrypts the token locally into an `enc:v1:` envelope.
2277
+ 3. Stores only the ciphertext with `set_secret`, in the component-scoped tier.
2278
+ 4. Grants the component permission to resolve it with `grant_secret`.
2279
+ 5. Prints the `credentials` reference for the deploy to use.
2080
2280
 
2081
- ```bash
2082
- harper deploy by_ref=true credential=true restart=true replicated=true
2083
- ```
2084
-
2085
- 7. **Use inline auth for one-off commands (not recommended for production)**: Pass `auth_username` and `auth_password` directly. These take precedence over environment variables and saved login tokens.
2086
-
2087
- ```bash
2088
- harper deploy \
2089
- project=<name> \
2090
- package=<package> \
2091
- auth_username=<username> \
2092
- auth_password=<password> \
2093
- target=<remote> \
2094
- restart=true \
2095
- replicated=true
2096
- ```
2281
+ Use a **fine-grained** personal access token (PAT) scoped to **Contents: Read-only** on the specific repository. Avoid session tokens from `gh` CLI — they typically carry `repo`, `read:org`, `gist`, and `workflow` scopes across your whole account.
2097
2282
 
2098
2283
  #### Examples
2099
2284
 
@@ -2103,63 +2288,50 @@ Apply this rule when deploying a Harper application to a remote Harper Fabric cl
2103
2288
  harper login https://my-cluster.harperdbcloud.com
2104
2289
  harper deploy \
2105
2290
  project=my-app \
2106
- package="@myorg/my-app" \
2291
+ package="HarperFast/my-app" \
2107
2292
  target=https://my-cluster.harperdbcloud.com \
2108
2293
  restart=true \
2109
2294
  replicated=true
2110
2295
  ```
2111
2296
 
2112
- **Deploy current directory as a snapshot:**
2297
+ **CI/CD deploy using environment variables:**
2113
2298
 
2114
2299
  ```bash
2300
+ export HARPER_CLI_USERNAME=admin
2301
+ export HARPER_CLI_PASSWORD=secret
2115
2302
  harper deploy \
2116
2303
  project=my-app \
2304
+ package="HarperFast/my-app" \
2117
2305
  target=https://my-cluster.harperdbcloud.com \
2118
2306
  restart=true \
2119
2307
  replicated=true
2120
2308
  ```
2121
2309
 
2122
- **Deploy a pinned tag by reference:**
2123
-
2124
- ```bash
2125
- harper deploy ref=v1.2.0 restart=true replicated=true
2126
- ```
2127
-
2128
- **Deploy a private repository by reference with a stored credential:**
2129
-
2130
- ```bash
2131
- harper deploy by_ref=true credential=true restart=true replicated=true
2132
- ```
2133
-
2134
- **GitHub Actions — deploy the pull request head commit explicitly:**
2310
+ **Deploy by reference in GitHub Actions (pull request):**
2135
2311
 
2136
2312
  ```bash
2137
2313
  harper deploy ref=${{ github.event.pull_request.head.sha }} restart=true replicated=true
2138
2314
  ```
2139
2315
 
2140
- **CI/CD deploy using environment variables:**
2316
+ **Deploy a private repo by reference with a provisioned credential:**
2141
2317
 
2142
2318
  ```bash
2143
- export HARPER_CLI_USERNAME=<username>
2144
- export HARPER_CLI_PASSWORD=<password>
2145
- harper deploy \
2146
- project=my-app \
2147
- package="HarperDB/my-app#semver:v1.0.0" \
2148
- target=https://my-cluster.harperdbcloud.com \
2149
- restart=true \
2150
- replicated=true
2319
+ # Provision once (run as super_user)
2320
+ harper deploy setup=true
2321
+
2322
+ # Deploy subsequently
2323
+ harper deploy by_ref=true credential=true restart=true replicated=true
2151
2324
  ```
2152
2325
 
2153
2326
  #### Notes
2154
2327
 
2155
- - `harper login` stores an authentication token so subsequent `harper deploy` commands do not require credentials to be repeated.
2156
- - Inline `auth_username`/`auth_password` parameters take precedence over environment variables, which take precedence over saved login tokens.
2157
- - For SSH-based private repos, register keys with the `add_ssh_key` operation before deploying.
2158
- - `by_ref` deploys build from source on each cluster node. If your application requires a build step that cannot run on the node, deploy a built payload (omit `by_ref`) instead.
2159
- - `harper deploy setup=true` requires **super_user** privileges. Provision credentials with an administrative account, not the CI identity.
2160
- - Tags and branches passed to `ref` are resolved to a full commit SHA locally before the deploy is sent. If resolution fails, run `git fetch` and retry, or pass a full commit SHA directly.
2161
- - The `refs/pull/<n>/head` style refs are rejected; use `ref=${{ github.event.pull_request.head.sha }}` in GitHub Actions pull request workflows instead.
2162
- - Deploy credentials provisioned via `harper deploy setup=true` are stored scoped to the component and reused on every subsequent deploy, including rollbacks.
2328
+ - `auth_username` and `auth_password` can be passed directly as deploy parameters for one-off commands, but this is not recommended for production. Dedicated authentication parameters take precedence over environment variables and saved login tokens.
2329
+ - The `enc:v1:` envelope means the plaintext token never leaves your machine — only ciphertext is stored and replicated.
2330
+ - Deploy credentials are stored scoped to the component, never in the global `processEnv` tier. If a global secret exists at the derived name, it is converted to the component-scoped tier automatically.
2331
+ - Because stored credentials are durable, later deploys and rollbacks reuse them without re-entering anything.
2332
+ - The unpushed-commit check is skipped under GitHub Actions; the dirty-tree warning still applies.
2333
+ - Deploying by reference means the cluster installs and builds from source. If your application requires a build step that cannot run on the node, deploy the built output as a payload deploy instead.
2334
+ - For SSH-based private repos, use the `add_ssh_key` operation to register keys before deploying.
2163
2335
 
2164
2336
  ### 4.2 Creating a Harper Fabric Account and Cluster
2165
2337
 
@@ -2598,11 +2770,11 @@ myApp:
2598
2770
 
2599
2771
  ### 4.7 v5 Upgrade: Breaking Changes and Migration Guide
2600
2772
 
2601
- Instructions for the agent to apply all required breaking-change fixes and recommended updates when migrating a Harper application to v5.
2773
+ Instructions for the agent to apply when migrating a Harper application to v5, covering all breaking changes and required code updates.
2602
2774
 
2603
2775
  #### When to Use
2604
2776
 
2605
- Apply this rule whenever upgrading an existing Harper application to v5, encountering v5 runtime errors related to module imports, `Table.get` return values, transaction context, process spawning, or VM module loading. Also apply when configuring `harperdb-config.yaml` for a v5 deployment.
2777
+ Apply this rule when upgrading an existing Harper application to v5, when encountering runtime errors related to renamed packages, changed APIs, or security restrictions after a v5 upgrade, or when scaffolding new v5-compatible application code.
2606
2778
 
2607
2779
  #### How It Works
2608
2780
 
@@ -2612,22 +2784,20 @@ Apply this rule whenever upgrading an existing Harper application to v5, encount
2612
2784
  import { tables } from 'harper';
2613
2785
  ```
2614
2786
 
2615
- 2. **Enable `allowInstallScripts` if packages require install scripts**: Harper v5 uses `--ignore-scripts` by default when installing packages. If your application requires installation scripts (e.g., to install additional binaries), set the `allowInstallScripts` option when deploying.
2787
+ 2. **Enable `allowInstallScripts` if packages require install scripts**: Harper v5 uses `--ignore-scripts` by default when installing packages. If a package requires execution of install scripts (e.g., to install native binaries), set the `allowInstallScripts` option when deploying.
2616
2788
 
2617
- 3. **Update `Table.get` usage — return value is now a plain frozen record object**: `Table.get` no longer returns an instance of the table class. The returned object is frozen and does not have table instance methods.
2618
- - Replace `wasLoadedFromSource()` with `target.loadedFromSource`:
2789
+ 3. **Update `Table.get` usage — return value is now a frozen record object**: `Table.get` now returns a plain record object, not a table class instance. The record is frozen; you cannot add or mutate properties directly.
2790
+ - Replace direct property mutation:
2619
2791
 
2620
2792
  ```javascript
2621
- // OLD — no longer works
2622
- const record = await Table.get(id);
2623
- if (record.wasLoadedFromSource()) {
2624
- // record was loaded from origin (not cache)
2625
- }
2793
+ let record = await Table.get(id);
2794
+ record = { ...record, property: 'changed' };
2626
2795
  ```
2627
2796
 
2797
+ - Replace `wasLoadedFromSource()` with `loadedFromSource` on the `target` object:
2798
+
2628
2799
  ```javascript
2629
- // NEW
2630
- const target = new RequestTarget(); // note that this is passed in if you are overriding the `get` method
2800
+ const target = new RequestTarget();
2631
2801
  target.id = id;
2632
2802
  const record = await Table.get(target);
2633
2803
  if (target.loadedFromSource) {
@@ -2635,107 +2805,86 @@ Apply this rule whenever upgrading an existing Harper application to v5, encount
2635
2805
  }
2636
2806
  ```
2637
2807
 
2638
- - The record object still has `getUpdatedTime` and `getExpiresAt` methods available.
2808
+ The record objects still expose `getUpdatedTime` and `getExpiresAt` methods.
2639
2809
 
2640
- - **Handle frozen records**: The record object is frozen — you cannot add or remove properties directly. Copy and spread instead:
2810
+ 4. **Update transaction and context handling using `getContext`**: Harper v5 uses asynchronous context tracking. Context and the current transaction are automatically carried to all downstream calls — you no longer pass context explicitly. Import `getContext` and `transaction` from `harper`:
2641
2811
 
2642
- ```javascript
2643
- // OLD — throws in v5
2644
- const record = await Table.get(id);
2645
- record.property = 'changed';
2646
- ```
2812
+ ```javascript
2813
+ import { getContext, transaction } from 'harper';
2814
+ ```
2647
2815
 
2648
- ```javascript
2649
- // NEW
2650
- let record = await Table.get(id);
2651
- record = { ...record, property: 'changed' };
2652
- ```
2816
+ If your code previously omitted context to escape a transaction (e.g., to poll for updated data), explicitly commit the transaction and/or wrap each read in a new `transaction()` call:
2653
2817
 
2654
- 4. **Update transaction and context handling**: Harper v5 uses asynchronous context tracking. Context and the current transaction are automatically carried to all calls, including `Table.get`. Code that previously omitted context to escape a transaction must now explicitly commit or start a new transaction.
2655
- - Import `getContext` and `transaction` from `harper`:
2818
+ ```javascript
2819
+ import { setTimeout as delay } from 'node:timers/promises';
2820
+ import { getContext, transaction } from 'harper';
2821
+ class MyResource {
2822
+ static async get(target) {
2823
+ await getContext().transaction.commit();
2824
+ while ((await transaction(() => Table.get(target))).status !== 'ready') {
2825
+ await delay(100);
2826
+ }
2827
+ return Table.get(target);
2828
+ }
2829
+ }
2830
+ ```
2656
2831
 
2657
- ```javascript
2658
- import { getContext, transaction } from 'harper';
2659
- ```
2832
+ 5. **Register allowed spawn commands via `allowedSpawnCommands`**: `spawn` and `execFile` may only launch executables listed in `applications.allowedSpawnCommands` in `harperdb-config.yaml`. Only the first token of the command is matched. `exec` is not usable through the substituted module; `execSync` always throws.
2660
2833
 
2661
- - Explicitly commit the current transaction before reading updated data:
2834
+ ```yaml
2835
+ applications:
2836
+ allowedSpawnCommands:
2837
+ - npm
2838
+ - node
2839
+ ```
2662
2840
 
2663
- ```javascript
2664
- import { setTimeout as delay } from 'node:timers/promises';
2665
- import { getContext, transaction } from 'harper';
2666
- class MyResource {
2667
- static async get(target) {
2668
- await getContext().transaction.commit();
2669
- while ((await transaction(() => Table.get(target))).status !== 'ready') {
2670
- await delay(100);
2671
- }
2672
- return Table.get(target);
2673
- }
2674
- }
2675
- ```
2841
+ Additionally, `spawn`, `execFile`, and `fork` now require a `name` property in the `options` argument to prevent process multiplication across threads.
2676
2842
 
2677
- 5. **Register allowed spawn commands via `allowedSpawnCommands`**: Any `spawn` or `execFile` call may only launch executables listed in `applications.allowedSpawnCommands` in `harperdb-config.yaml`. The `exec` function is not usable through the substituted module, and `execSync` always throws. The `spawn`, `execFile`, and `fork` functions also require a `name` property in the `options` argument to prevent process multiplication across threads.
2843
+ 6. **Use `saveBeforeCommit` instead of `blob.save()`**: The `blob.save()` method has been removed. Pass the `saveBeforeCommit` flag in the options to the `Blob` constructor instead.
2678
2844
 
2679
- 6. **Replace `blob.save()` with `saveBeforeCommit`**: The `blob.save()` method has been removed. Use the `saveBeforeCommit` flag in the options passed to the `Blob` constructor instead.
2845
+ 7. **Handle `headers` on returned response objects**: If you return an object from a REST method with a `headers` property, Harper v5 will use it as the response headers.
2680
2846
 
2681
- 7. **Configure the `moduleLoader` and `lockdown` settings**: Harper v5 loads application modules through Node.js's VM module API. Control all behavior in the `applications` section of `harperdb-config.yaml`:
2847
+ 8. **Configure the VM module loader and `lockdown` in `harperdb-config.yaml`**: v5 loads application modules through Node.js's VM module API. Control all behavior under the `applications` key:
2682
2848
 
2683
2849
  ```yaml
2684
2850
  applications:
2685
- lockdown: freeze-after-load # default; see below
2686
- moduleLoader: vm-current-context # vm-current-context (default) | vm | native | compartment
2687
- dependencyLoader: auto # auto (default) | app | native
2688
- allowedDirectory: app # app (default) | any
2689
- allowedSpawnCommands: # see "Spawning new processes" above
2851
+ lockdown: freeze-after-load
2852
+ moduleLoader: vm-current-context
2853
+ dependencyLoader: auto
2854
+ allowedDirectory: app
2855
+ allowedSpawnCommands:
2690
2856
  - npm
2691
2857
  - node
2692
- # allowedBuiltinModules: [] # if omitted, all Node.js built-ins are allowed
2693
2858
  ```
2694
2859
 
2695
2860
  **`moduleLoader` options:**
2696
2861
 
2697
2862
  | Value | Behavior |
2698
2863
  | -------------------- | -------------------------------------------------------------------------------------------------------------------------------- |
2699
- | `vm-current-context` | Default. VM loader in Harper's own context; shares intrinsics with Harper for best compatibility. |
2700
- | `vm` | VM loader in a separate per-application context with its own intrinsics; stronger isolation but can cause `instanceof` failures. |
2701
- | `native` | Standard Node.js `import()` with no VM loader; application-specific context (tagged logging, per-app `config`) not available. |
2702
- | `compartment` | SES Compartment-based loading; for specialized sandboxing only. |
2864
+ | `vm-current-context` | Default. VM loader in Harper's own context; shares intrinsics with Harper. Best compatibility. |
2865
+ | `vm` | VM loader in a separate per-application context with its own intrinsics. Stronger isolation but may cause `instanceof` failures. |
2866
+ | `native` | Standard Node.js `import()`. No VM loader; application-specific context (`logger`, `config`) unavailable. |
2867
+ | `compartment` | SES Compartment-based loading. For specialized sandboxing only. |
2703
2868
 
2704
2869
  **`lockdown` options:**
2705
2870
 
2706
- | Value | Behavior |
2707
- | ------------------- | ------------------------------------------------------------- |
2708
- | `freeze-after-load` | Default. Freezes intrinsics after all components have loaded. |
2709
- | `freeze` | Freezes intrinsics before loading any application code. |
2710
- | `ses` | Full SES lockdown via the `ses` package; strictest. |
2711
- | `none` | No lockdown. Use as a temporary workaround only. |
2712
- - To disable the VM loader entirely and restore pre-v5 behavior:
2713
-
2714
- ```yaml
2715
- applications:
2716
- moduleLoader: native
2717
- ```
2718
-
2719
- - To restrict which Node.js built-ins are accessible:
2871
+ | Value | Behavior |
2872
+ | ------------------- | ------------------------------------------------------- |
2873
+ | `freeze-after-load` | Default. Freezes intrinsics after all components load. |
2874
+ | `freeze` | Freezes intrinsics before loading any application code. |
2875
+ | `ses` | Full SES lockdown via the `ses` package. Strictest. |
2876
+ | `none` | No lockdown. Use as a temporary workaround only. |
2720
2877
 
2721
- ```yaml
2722
- applications:
2723
- allowedBuiltinModules:
2724
- - fs
2725
- - path
2726
- - http
2727
- ```
2878
+ To disable the VM loader entirely and restore pre-v5 behavior:
2728
2879
 
2729
- - To allow loading modules from outside the application directory:
2730
-
2731
- ```yaml
2732
- applications:
2733
- allowedDirectory: any
2734
- ```
2880
+ ```yaml
2881
+ applications:
2882
+ moduleLoader: native
2883
+ ```
2735
2884
 
2736
2885
  #### Examples
2737
2886
 
2738
- **Full transaction context migration:**
2887
+ **Full transaction polling pattern (v5):**
2739
2888
 
2740
2889
  ```javascript
2741
2890
  import { setTimeout as delay } from 'node:timers/promises';
@@ -2743,9 +2892,7 @@ import { getContext, transaction } from 'harper';
2743
2892
 
2744
2893
  class MyResource {
2745
2894
  static async get(target) {
2746
- // Explicitly commit the transaction to see updated data
2747
2895
  await getContext().transaction.commit();
2748
- // Start a new transaction for each get to see the latest data
2749
2896
  while ((await transaction(() => Table.get(target))).status !== 'ready') {
2750
2897
  await delay(100);
2751
2898
  }
@@ -2754,11 +2901,9 @@ class MyResource {
2754
2901
  }
2755
2902
  ```
2756
2903
 
2757
- **`Table.get` with `loadedFromSource` check:**
2904
+ **Checking `loadedFromSource` after `Table.get`:**
2758
2905
 
2759
2906
  ```javascript
2760
- import { tables } from 'harper';
2761
-
2762
2907
  const target = new RequestTarget();
2763
2908
  target.id = id;
2764
2909
  const record = await Table.get(target);
@@ -2780,14 +2925,27 @@ applications:
2780
2925
  - node
2781
2926
  ```
2782
2927
 
2928
+ **Restricting allowed built-in modules:**
2929
+
2930
+ ```yaml
2931
+ applications:
2932
+ allowedBuiltinModules:
2933
+ - fs
2934
+ - path
2935
+ - http
2936
+ ```
2937
+
2783
2938
  #### Notes
2784
2939
 
2785
- - Use `getContext` (imported from `harper`) to access the current transaction anywhere in application code without passing context explicitly through every call.
2786
- - Harper functions and APIs should be accessed through the `harper` package rather than through global variables.
2787
- - Use `static` methods on Resources/Tables to implement endpoints, and access request information from the request `target` argument or via `getContext`.
2788
- - `lockdown: none` is only a temporary workaround for dependencies that mutate intrinsic prototypes at runtime; do not leave it set in production.
2789
- - Under `lockdown: ses`, the constrained (https-only) `fetch` is applied only in `vm` mode. In `vm-current-context` and `native` modes, application code uses the standard global `fetch`.
2790
- - In production, `allowedDirectory: app` is the default; dev mode installs default to `allowedDirectory: any`.
2940
+ - Always import Harper APIs from `'harper'`, not from global variables or `'harperdb'`.
2941
+ - `getContext` is exported from `'harper'` and provides access to the current transaction without passing context explicitly.
2942
+ - Record objects returned by `Table.get` are frozen — spread into a new object before modifying.
2943
+ - `loadedFromSource` is a property on the `target` object, replacing the removed `wasLoadedFromSource()` instance method.
2944
+ - `saveBeforeCommit` replaces the removed `blob.save()` method.
2945
+ - The `headers` property on a returned REST response object is used as response headers.
2946
+ - Under `lockdown: ses`, the constrained `fetch` applies only in `vm` mode. In `vm-current-context` and `native` modes, application code uses the standard global `fetch`.
2947
+ - In production, `allowedDirectory: app` is the default; modules outside the application directory tree will throw. Set `allowedDirectory: any` only if legitimately required.
2948
+ - `dependencyLoader: native` is a narrower option than `moduleLoader: native` — it uses native loading only for npm packages while keeping the VM loader for first-party application source files.
2791
2949
 
2792
2950
  ### 4.8 Delegating to the Built-in Agent
2793
2951