@harperfast/template-vue-studio 1.13.6 → 1.14.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (36) hide show
  1. package/.agents/skills/harper-best-practices/AGENTS.md +1260 -1113
  2. package/.agents/skills/harper-best-practices/rules/automatic-apis.md +118 -83
  3. package/.agents/skills/harper-best-practices/rules/checking-authentication.md +244 -256
  4. package/.agents/skills/harper-best-practices/rules/custom-resources.md +37 -42
  5. package/.agents/skills/harper-best-practices/rules/defining-relationships.md +86 -49
  6. package/.agents/skills/harper-best-practices/rules/deploying-from-ci.md +163 -170
  7. package/.agents/skills/harper-best-practices/rules/deploying-to-harper-fabric.md +128 -88
  8. package/.agents/skills/harper-best-practices/rules/load-env.md +27 -30
  9. package/.agents/skills/harper-best-practices/rules/programmatic-table-requests.md +184 -155
  10. package/.agents/skills/harper-best-practices/rules/querying-rest-apis.md +146 -94
  11. package/.agents/skills/harper-best-practices/rules/schema-design-tooling.md +60 -77
  12. package/.agents/skills/harper-best-practices/rules/v5-upgrade.md +52 -63
  13. package/.agents/skills/harper-best-practices/rules/vector-indexing.md +77 -68
  14. package/.agents/skills/harper-mcp/AGENTS.md +228 -155
  15. package/.agents/skills/harper-mcp/rules/custom-mcp-tools.md +35 -41
  16. package/.agents/skills/harper-mcp/rules/enabling-mcp.md +104 -92
  17. package/.agents/skills/harper-mcp/rules/rate-limiting.md +95 -28
  18. package/agent/skills/harper-best-practices/AGENTS.md +1260 -1113
  19. package/agent/skills/harper-best-practices/rules/automatic-apis.md +118 -83
  20. package/agent/skills/harper-best-practices/rules/checking-authentication.md +244 -256
  21. package/agent/skills/harper-best-practices/rules/custom-resources.md +37 -42
  22. package/agent/skills/harper-best-practices/rules/defining-relationships.md +86 -49
  23. package/agent/skills/harper-best-practices/rules/deploying-from-ci.md +163 -170
  24. package/agent/skills/harper-best-practices/rules/deploying-to-harper-fabric.md +128 -88
  25. package/agent/skills/harper-best-practices/rules/load-env.md +27 -30
  26. package/agent/skills/harper-best-practices/rules/programmatic-table-requests.md +184 -155
  27. package/agent/skills/harper-best-practices/rules/querying-rest-apis.md +146 -94
  28. package/agent/skills/harper-best-practices/rules/schema-design-tooling.md +60 -77
  29. package/agent/skills/harper-best-practices/rules/v5-upgrade.md +52 -63
  30. package/agent/skills/harper-best-practices/rules/vector-indexing.md +77 -68
  31. package/agent/skills/harper-mcp/AGENTS.md +228 -155
  32. package/agent/skills/harper-mcp/rules/custom-mcp-tools.md +35 -41
  33. package/agent/skills/harper-mcp/rules/enabling-mcp.md +104 -92
  34. package/agent/skills/harper-mcp/rules/rate-limiting.md +95 -28
  35. package/package.json +1 -1
  36. package/skills-lock.json +7 -7
@@ -44,31 +44,37 @@ type ExamplePerson @table @export {
44
44
 
45
45
  ### 1.2 Schema Design and GraphQL Tooling
46
46
 
47
- Instructions for the agent to follow when designing Harper schemas, applying core directives, and configuring GraphQL tooling.
47
+ Instructions for the agent to follow when designing Harper database schemas using GraphQL type definitions, core directives, and tooling configuration.
48
48
 
49
49
  #### When to Use
50
50
 
51
- Apply this rule when creating or modifying Harper schema files, configuring `graphqlSchema` in `config.yaml`, or deciding which directives to apply to tables and fields. Use it whenever a task involves defining tables, primary keys, indexes, or export behavior.
51
+ Apply this rule when creating or modifying Harper schema files — defining tables, primary keys, indexes, and export behavior. Use it when configuring `graphqlSchema` in `config.yaml` or when choosing directives to control table structure, access patterns, and record lifecycle.
52
52
 
53
53
  #### How It Works
54
54
 
55
- 1. **Load the schema via `graphqlSchema`**: In the component's `config.yaml`, point to the schema file:
55
+ 1. **Create a schema file** using standard GraphQL type definitions with Harper-specific directives. Name it (e.g., `schema.graphql`) and place it in your component directory.
56
+
57
+ ```graphql
58
+ type Dog @table {
59
+ id: Long @primaryKey
60
+ name: String
61
+ breed: String
62
+ age: Int
63
+ }
64
+ ```
65
+
66
+ 2. **Register the schema** in the component's `config.yaml` using the `graphqlSchema` plugin:
56
67
 
57
68
  ```yaml
58
69
  graphqlSchema:
59
70
  files: 'schema.graphql'
60
71
  ```
61
72
 
62
- Both plugins and applications can specify schemas. The schema ensures required tables exist on deployment, enforces types, controls indexing, and defines relationships.
63
-
64
- 2. **Mark tables with `@table`**: Apply `@table` to every GraphQL type that should become a Harper database table. The type name becomes the table name by default.
73
+ 3. **Mark every table type with `@table`**. This tells Harper to create and manage the table. The type name becomes the table name by default.
65
74
 
66
75
  ```graphql
67
- type Dog @table {
76
+ type MyTable @table {
68
77
  id: Long @primaryKey
69
- name: String
70
- breed: String
71
- age: Int
72
78
  }
73
79
  ```
74
80
 
@@ -82,13 +88,15 @@ Apply this rule when creating or modifying Harper schema files, configuring `gra
82
88
  | `eviction` | `Int` | `0` | Additional seconds after `expiration` before physical removal |
83
89
  | `scanInterval` | `Int` | `(expiration + eviction) / 4` | Seconds between eviction scans |
84
90
  | `replicate` | `Boolean` | `true` | Enable replication of this table |
85
- | `replicaExpiration` | `Float` | `3600` | Seconds a record fetched from another node is kept as a local replica |
91
+ | `replicaExpiration` | `Float` | `3600` | Seconds to keep a local residency replica |
86
92
  | `replicaRefreshFraction` | `Float` | `0.5` | Fraction of `replicaExpiration` after which a read renews the replica lifetime |
87
93
  | `audit` | `Boolean` | `logging.auditLog` | Enable the table's transaction log |
88
94
  | `cacheControl` | `String` | — | `Cache-Control` header for anonymous GET/HEAD 200/304 responses |
89
95
  | `randomAccessFields` | `Boolean` | `storage.randomAccessFields` | Pin this table's record encoding |
90
96
 
91
- 3. **Designate a primary key with `@primaryKey`**: Apply `@primaryKey` to exactly one field per table. Primary keys must be unique; duplicate inserts are rejected.
97
+ 4. **Designate a primary key with `@primaryKey`** on exactly one field per table. Primary keys must be unique; duplicate inserts are rejected. If no primary key is provided on insert, Harper auto-generates one:
98
+ - `String` or `ID` → UUID string
99
+ - `Int`, `Long`, or `Any` → auto-incrementing integer
92
100
 
93
101
  ```graphql
94
102
  type Product @table {
@@ -97,13 +105,9 @@ Apply this rule when creating or modifying Harper schema files, configuring `gra
97
105
  }
98
106
  ```
99
107
 
100
- If no primary key is provided on insert, Harper auto-generates one:
101
- - **UUID string** — when type is `String` or `ID`
102
- - **Auto-incrementing integer** — when type is `Int`, `Long`, or `Any`
103
-
104
- Prefer `Long` or `Any` for auto-generated numeric keys; `Int` is 32-bit and may be insufficient for large tables.
108
+ Use `Long` or `Any` for auto-generated numeric keys; `Int` is 32-bit and may be insufficient for large tables.
105
109
 
106
- 4. **Index queryable fields with `@indexed`**: Apply `@indexed` to any field that must be filterable via REST queries, SQL, or NoSQL operations.
110
+ 5. **Add `@indexed` to fields that need fast querying**. Required for filtering by an attribute in REST queries, SQL, or NoSQL operations. If the field value is an array, each element is individually indexed.
107
111
 
108
112
  ```graphql
109
113
  type Product @table {
@@ -113,9 +117,7 @@ Apply this rule when creating or modifying Harper schema files, configuring `gra
113
117
  }
114
118
  ```
115
119
 
116
- If the field value is an array, each element is individually indexed. Null values are indexed by default.
117
-
118
- 5. **Expose tables externally with `@export`**: Apply `@export` to make a table accessible via REST, MQTT, and other interfaces. The optional `name` parameter sets the URL path segment.
120
+ 6. **Expose the table externally with `@export`**. This makes the table accessible via REST, MQTT, and other interfaces. The optional `name` parameter sets the URL path segment; without it, the type name is used.
119
121
 
120
122
  ```graphql
121
123
  type MyTable @table @export(name: "my-table") {
@@ -123,9 +125,9 @@ Apply this rule when creating or modifying Harper schema files, configuring `gra
123
125
  }
124
126
  ```
125
127
 
126
- `@export` alone does not serve HTTP traffic — REST must also be enabled for the application via `rest: true` in `config.yaml` or Harper's built-in default. `@export` is a routing directive, not access control: omitting it returns 404 to callers but the table remains accessible through the Operations API and SQL.
128
+ `@export` alone does not serve HTTP traffic — REST must also be enabled for the application, either with `rest: true` in `config.yaml` or by Harper's built-in default for a component with no configuration file.
127
129
 
128
- 6. **Prevent extra properties with `@sealed`** (when needed): Apply `@sealed` to reject records that include properties beyond those declared in the schema. By default, Harper allows additional properties.
130
+ 7. **Use `@sealed` to prevent extra properties** on records beyond those declared in the schema. By default, Harper allows records to have additional properties.
129
131
 
130
132
  ```graphql
131
133
  type StrictRecord @table @sealed {
@@ -134,33 +136,13 @@ Apply this rule when creating or modifying Harper schema files, configuring `gra
134
136
  }
135
137
  ```
136
138
 
137
- 7. **Use unique database names for plugins and applications**: All tables default to the `data` database. Specify a distinct `database` argument on `@table` to avoid naming collisions between components.
138
-
139
- 8. **Apply timestamp and expiration field directives** where needed:
140
- - `@createdTime` — auto-assigns a Unix epoch millisecond timestamp on record creation.
141
- - `@updatedTime` — auto-assigns a Unix epoch millisecond timestamp on each update.
142
- - `@expiresAt` — marks a field as the record's absolute expiration time (Unix epoch milliseconds); overrides the table-level `expiration` default.
143
- - `@hidden` — suppresses the type or field from MCP tool descriptors and the OpenAPI document. Does not restrict data access.
144
-
145
- ```graphql
146
- type Event @table {
147
- id: Long @primaryKey
148
- createdAt: Long @createdTime
149
- updatedAt: Long @updatedTime
150
- }
151
-
152
- type Session @table {
153
- id: ID @primaryKey
154
- token: String
155
- expiresAt: Long @expiresAt
156
- }
157
- ```
158
-
159
- Key `@expiresAt` behaviors:
160
- - Takes an absolute Unix epoch millisecond timestamp, not a duration. A full-record `put` that omits the field clears it; a `patch` preserves it.
161
- - A per-record value may extend or shorten a record's expiration relative to the table-level `expiration` default. When a record omits the field, the table default (if any) applies.
139
+ 8. **Use timestamp directives** to auto-populate lifecycle fields:
140
+ - `@createdTime` — assigns Unix epoch milliseconds on record creation
141
+ - `@updatedTime` — assigns Unix epoch milliseconds on every update
142
+ - `@expiresAt` — marks a field as the record's absolute expiration time (Unix epoch milliseconds); authoritative over the table-level `expiration` default
162
143
 
163
144
  9. **Use `@embed`** to automatically compute an embedding vector for an attribute whenever the source field is written (requires `source` and `model` arguments; field type must be `[Float]`):
145
+
164
146
  ```graphql
165
147
  type Document @table {
166
148
  id: Long @primaryKey
@@ -168,11 +150,12 @@ Apply this rule when creating or modifying Harper schema files, configuring `gra
168
150
  embedding: [Float] @embed(source: "text", model: "default")
169
151
  }
170
152
  ```
171
- The attribute is automatically indexed with an HNSW vector index. A failure to compute the embedding fails the write. An update that does not touch the `source` field leaves the vector unchanged. Setting the `source` field to `null` sets the vector to `null`.
153
+
154
+ 10. **Use unique `database` names** in plugins or applications to avoid table naming collisions, since all tables default to the `data` database.
172
155
 
173
156
  #### Examples
174
157
 
175
- **Minimal schema with two tables:**
158
+ **Minimal two-table schema:**
176
159
 
177
160
  ```graphql
178
161
  type Dog @table {
@@ -188,7 +171,17 @@ type Breed @table {
188
171
  }
189
172
  ```
190
173
 
191
- **Caching table with expiration, eviction, and scan tuning:**
174
+ **Table with name override and specific database:**
175
+
176
+ ```graphql
177
+ type Product @table(table: "products", database: "commerce") {
178
+ id: Long @primaryKey
179
+ name: String @indexed
180
+ price: Float @indexed
181
+ }
182
+ ```
183
+
184
+ **Caching table with expiration and eviction tuning:**
192
185
 
193
186
  ```graphql
194
187
  # Expire after 5 minutes, evict after 1 hour, scan every 10 minutes
@@ -208,16 +201,7 @@ type Product @table(cacheControl: "public, max-age=60") @export {
208
201
  }
209
202
  ```
210
203
 
211
- **Table in a named database with expiration:**
212
-
213
- ```graphql
214
- type Event @table(database: "analytics", expiration: 86400) {
215
- id: Long @primaryKey
216
- name: String @indexed
217
- }
218
- ```
219
-
220
- **Replica lifetime tuning:**
204
+ **Table with replica lifetime control:**
221
205
 
222
206
  ```graphql
223
207
  # Keep fetched records for a day; a read after six hours renews them
@@ -227,18 +211,19 @@ type Order @table(replicaExpiration: 86400, replicaRefreshFraction: 0.25) {
227
211
  }
228
212
  ```
229
213
 
230
- **Random-access field encoding pinned on the table:**
214
+ **Session table with per-record expiration and timestamps:**
231
215
 
232
216
  ```graphql
233
- type Reading @table(randomAccessFields: true) {
217
+ type Session @table {
234
218
  id: ID @primaryKey
235
- sensorId: String @indexed
236
- celsius: Float
237
- recordedAt: Int
219
+ userId: String @indexed
220
+ createdAt: Long @createdTime
221
+ updatedAt: Long @updatedTime
222
+ expiresAt: Long @expiresAt
238
223
  }
239
224
  ```
240
225
 
241
- **`config.yaml` loading the schema:**
226
+ **`config.yaml` registration:**
242
227
 
243
228
  ```yaml
244
229
  graphqlSchema:
@@ -247,29 +232,27 @@ graphqlSchema:
247
232
 
248
233
  #### Notes
249
234
 
250
- - Schemas are flexible by default — records may include additional properties. Use `@sealed` to prevent this.
251
- - `@export` is a routing directive, not an access control mechanism. Use RBAC and `attribute_permissions` to restrict data access.
252
- - `cacheControl` on `@table` only emits the header for anonymous (unauthenticated) GET/HEAD 200/304 responses. Authenticated responses always receive `Cache-Control: private, no-cache`.
253
- - `expiration`, `eviction`, and `scanInterval` work together: `expiration` marks a record stale, `eviction` controls physical removal, and `scanInterval` sets how often the sweep runs. Eviction scans are clock-aligned to the server's local timezone, not startup-aligned.
254
- - Eviction removes non-indexed record data but does not remove a record from its secondary indexes; indexes remain functional for evicted records.
255
- - `replicaExpiration` and `replicaRefreshFraction` apply only on the RocksDB engine. On LMDB, remote reads remain proxy reads regardless of these settings.
235
+ - `@export` is a routing directive, not access control. Omitting it returns 404 to callers but does not protect the data — the table remains accessible through the Operations API and SQL. Use RBAC role grants and `attribute_permissions` for actual access control.
236
+ - `@hidden` (on types or fields) suppresses entries from MCP tool descriptors and the OpenAPI document only. It does not restrict data access. Use `attribute_permissions` on roles for field-level enforcement.
237
+ - Null values are indexed by default for `@indexed` fields, enabling queries like `GET /Product/?category=null`. This matches records that store `null`, not records that omit the attribute.
238
+ - `cacheControl` emits the header only on anonymous (unauthenticated) GET/HEAD 200/304 responses. Authenticated responses receive `Cache-Control: private, no-cache` regardless of the declaration.
239
+ - A `patch` of other fields preserves an existing `@expiresAt` value; a full `put` that omits the field clears it. Always re-include `expiresAt` in session-renewal writes.
240
+ - `replicaExpiration` and `randomAccessFields` apply to RocksDB only. LMDB tables ignore replica behavior regardless of settings.
256
241
  - Disabling replication on a table (`replicate: false`) and re-enabling it later will not catch up on writes made while replication was disabled.
257
- - `randomAccessFields` on `@table` pins the encoding when the table is created. Editing the argument later does not repin an existing table.
258
- - `@hidden` is a metadata-visibility directive only — it does not restrict data access. Use `attribute_permissions` on roles for field-level access control.
259
242
 
260
243
  ### 1.3 Defining Relationships Between Tables in Harper
261
244
 
262
- Instructions for the agent to define and query relationships between Harper tables using the `@relationship` directive in GraphQL schemas.
245
+ Instructions for the agent to follow when defining and querying relationships between tables in Harper using the `@relationship` directive.
263
246
 
264
247
  **Instead of:** ORM relations and join tables (Prisma, Drizzle, Sequelize). Use this rule rather than adding that service. See `harper-architecture`.
265
248
 
266
249
  #### When to Use
267
250
 
268
- Apply this rule when writing Harper schemas that require join queries across tables, or when query results must include related records as nested properties. Use it whenever a foreign key exists in one table that references another, or when modeling one-to-many, many-to-one, or many-to-many associations.
251
+ Apply this rule when adding foreign-key relationships between schema tables, enabling join queries, or selecting nested related records in query results. Use it any time a schema type needs to reference records in another table via a `@relationship` directive.
269
252
 
270
253
  #### How It Works
271
254
 
272
- 1. **Declare a many-to-one relationship with `from`**: Place `@relationship(from: attribute)` on a field when the foreign key lives in the current table and references the primary key of the target table. The result type is a single object.
255
+ 1. **Use `@relationship(from: attribute)` for many-to-one or many-to-many**: Place this on the field whose table holds the foreign key. The `from` argument names the attribute in this table that stores the foreign key referencing the target table's primary key.
273
256
 
274
257
  ```graphql
275
258
  type RealityShow @table @export {
@@ -285,7 +268,17 @@ Apply this rule when writing Harper schemas that require join queries across tab
285
268
  }
286
269
  ```
287
270
 
288
- 2. **Declare a one-to-many relationship with `to`**: Place `@relationship(to: attribute)` on a field when the foreign key lives in the target table and references the primary key of the current table. The result type **must** be an array.
271
+ If the foreign key attribute is an array, the relationship becomes many-to-many:
272
+
273
+ ```graphql
274
+ type RealityShow @table @export {
275
+ id: Long @primaryKey
276
+ networkIds: [Long] @indexed
277
+ networks: [Network] @relationship(from: networkIds)
278
+ }
279
+ ```
280
+
281
+ 2. **Use `@relationship(to: attribute)` for one-to-many or many-to-many**: Place this on the field whose table holds the primary key. The `to` argument names the attribute in the _target_ table that stores the foreign key. The result type must be an array.
289
282
 
290
283
  ```graphql
291
284
  type Network @table @export {
@@ -295,14 +288,14 @@ Apply this rule when writing Harper schemas that require join queries across tab
295
288
  }
296
289
  ```
297
290
 
298
- 3. **Declare a foreign-key-to-foreign-key relationship with `@relationship(from: attribute, to: attribute)`**: Specify both `from` and `to` when neither side uses the primary key. Harper resolves the relationship by searching the target table's `to` attribute for matches using the current record's `from` attribute value. The result type must be an array.
291
+ 3. **Use `@relationship(from: attribute, to: attribute)` for foreign-key-to-foreign-key joins**: Specify both `from` and `to` when neither side uses the primary key. The result type must be an array. Harper resolves the relationship by searching the target table's `to` attribute for matches using this record's `from` attribute value.
299
292
 
300
293
  ```graphql
301
294
  type OrderItem @table @export {
302
295
  id: Long @primaryKey
303
296
  orderId: Long @indexed
304
297
  productSku: Long @indexed
305
- products: [Product] @relationship(from: productSku, to: sku) # matches products by sku, not primary key
298
+ products: [Product] @relationship(from: productSku, to: sku)
306
299
  }
307
300
 
308
301
  type Product @table @export {
@@ -312,17 +305,7 @@ Apply this rule when writing Harper schemas that require join queries across tab
312
305
  }
313
306
  ```
314
307
 
315
- 4. **Declare a many-to-many relationship using an array foreign key with `from`**: When the foreign key field is an array, `@relationship(from: attribute)` establishes a many-to-many relationship. The result type must be an array.
316
-
317
- ```graphql
318
- type RealityShow @table @export {
319
- id: Long @primaryKey
320
- networkIds: [Long] @indexed
321
- networks: [Network] @relationship(from: networkIds)
322
- }
323
- ```
324
-
325
- 5. **Query across relationships using dot-syntax**: Filter records by related table attributes using dot-chained attribute names. This behaves as an INNER JOIN — only parents with a qualifying related record are returned.
308
+ 4. **Query across relationships using dot-syntax**: Filter by related table attributes using dot-chained paths. This behaves as an INNER JOIN — only parents with a matching related record are returned.
326
309
 
327
310
  ```
328
311
  GET /RealityShow?network.name=Bravo
@@ -330,7 +313,7 @@ Apply this rule when writing Harper schemas that require join queries across tab
330
313
  GET /Brand/?products.name=Keyboard
331
314
  ```
332
315
 
333
- 6. **Include relationship fields in results using `select()`**: Relationship attributes are not included by default. Use `select()` to project them, optionally with a nested field list using `{}`.
316
+ 5. **Include relationship fields in results using `select()`**: Relationship attributes are not included by default. Use `select()` to project them. Selecting a relationship without an outer dot-syntax filter behaves as a LEFT JOIN.
334
317
 
335
318
  ```
336
319
  GET /Product/?brand.name=Microsoft&select(name,brand)
@@ -338,15 +321,7 @@ Apply this rule when writing Harper schemas that require join queries across tab
338
321
  GET /Product/?name=Keyboard&select(name,brand{name,id})
339
322
  ```
340
323
 
341
- Selecting a relationship without an outer dot-syntax filter on it behaves as a LEFT JOIN: parents remain even when there are no related records.
342
-
343
- 7. **Filter and page nested relationship results using `[]` inside `select()`**: Apply conditions, sort, and pagination to each parent's related records by placing them inside square brackets in the `select()` expression.
344
-
345
- ```
346
- GET /Brand/?select(id,name,products[price=ge=number:30&sort(price)&limit(1)&select(id,name,price)])
347
- ```
348
-
349
- An outer relationship filter (e.g., `products.price=ge=number:30`) removes parents without a qualifying child **and** narrows the nested candidates. A filter inside `products[...]` only narrows projected children without removing their parent.
324
+ 6. **Apply conditions, sort, and pagination to nested related records using bracket syntax inside `select()`**: Use square brackets after the relationship name inside `select()` to filter, sort, and page each parent's related records independently.
350
325
 
351
326
  | Nested option | Behavior |
352
327
  | ------------------ | -------------------------------------------------------------------------- |
@@ -356,9 +331,27 @@ Apply this rule when writing Harper schemas that require join queries across tab
356
331
  | `limit(start,end)` | Skip `start` eligible children and return at most `end - start`. |
357
332
  | `select(id,price)` | Project fields from returned children; omit to return whole child records. |
358
333
 
334
+ Invalid `limit()` bounds (negative or fractional values) return HTTP 400. `limit(0)` returns no children without reading them.
335
+
336
+ 7. **Use bracket syntax on the relationship name in the query string to require one related record to meet multiple conditions**: This is distinct from separate `&`-joined conditions, which may be satisfied by different related records.
337
+
338
+ ```
339
+ GET /Brand/?products[name=Keyboard&price=ge=30]
340
+ ```
341
+
342
+ Inside brackets, name attributes of the related table directly (e.g., `price`, not `products.price`). Values convert to the attribute's schema type, so `price=ge=30` needs no `number:` prefix inside brackets.
343
+
344
+ 8. **Combine OR filters with child sorting and pagination**: Group OR conditions inside the relationship selection with `[condition1|condition2]`, then append `sort()`, `limit()`, and `select()` with `&`.
345
+
346
+ ```
347
+ GET /Brand/?sort(name,id)&limit(0,20)&select(id,name,products[[price=lt=number:30|price=ge=number:60]&sort(-price,id)&limit(1,2)&select(id,name,price)])
348
+ ```
349
+
350
+ 9. **Index foreign key and filter fields**: Keep relationship lookup fields and outer filter fields indexed. The `from` or `to` attribute referenced in `@relationship` should carry `@indexed`.
351
+
359
352
  #### Examples
360
353
 
361
- ##### Schema with bidirectional relationships
354
+ ##### Many-to-one with join query
362
355
 
363
356
  ```graphql
364
357
  type Product @table @export {
@@ -375,7 +368,12 @@ type Brand @table @export {
375
368
  }
376
369
  ```
377
370
 
378
- ##### Many-to-many without a junction table
371
+ ```
372
+ GET /Product/?brand.name=Microsoft&select(name,brand{name,id})
373
+ GET /Brand/?products.name=Keyboard
374
+ ```
375
+
376
+ ##### Many-to-many with array foreign key
379
377
 
380
378
  ```graphql
381
379
  type Product @table @export {
@@ -390,11 +388,7 @@ type Product @table @export {
390
388
  GET /Product/?resellers.name=Cool Shop&select(id,name,resellers{name,id})
391
389
  ```
392
390
 
393
- The array order of `resellerIds` is preserved when resolving the relationship.
394
-
395
- ##### Nested filtering and pagination scenarios
396
-
397
- Schema for the following examples:
391
+ ##### Nested filtering and pagination
398
392
 
399
393
  ```graphql
400
394
  type Brand @table @export {
@@ -411,45 +405,71 @@ type Product @table @export {
411
405
  }
412
406
  ```
413
407
 
414
- **Keep every brand; show its cheapest product costing at least 30** (LEFT JOIN behavior — Budget Basics returns with an empty product list):
408
+ Keep every brand; show its cheapest product costing at least 30 (LEFT JOIN behavior — Budget Basics remains with an empty product list):
415
409
 
416
410
  ```
417
411
  GET /Brand/?select(id,name,products[price=ge=number:30&sort(price)&limit(1)&select(id,name,price)])
418
412
  ```
419
413
 
420
- **Return only qualifying brands; inherit the outer price filter** (INNER JOIN behavior — Budget Basics is removed):
414
+ Return only qualifying brands; inherit the outer price filter into nested results:
421
415
 
422
416
  ```
423
417
  GET /Brand/?products.price=ge=number:30&select(id,name,products[sort(price)&limit(1)&select(id,name,price)])
424
418
  ```
425
419
 
426
- **Page through each brand's qualifying products** (offset applies independently per brand):
420
+ Page through each brand's qualifying products (offset applies independently per brand):
427
421
 
428
422
  ```
429
423
  GET /Brand/?products.price=ge=number:30&select(id,name,products[sort(price)&limit(1,2)&select(id,name,price)])
430
424
  ```
431
425
 
432
- **Bound parent pages with stable sorting**:
426
+ Bound parent pages with stable sorting:
433
427
 
434
428
  ```
435
429
  GET /Brand/?products.price=ge=number:30&sort(name,id)&limit(0,20)&select(id,name,products[sort(price,id)&limit(1)&select(id,name,price)])
436
430
  ```
437
431
 
438
- **Combine OR filters with child sorting and pagination**:
432
+ ##### Conditions through several relationships
439
433
 
440
- ```
441
- GET /Brand/?sort(name,id)&limit(0,20)&select(id,name,products[[price=lt=number:30|price=ge=number:60]&sort(-price,id)&limit(1,2)&select(id,name,price)])
434
+ ```graphql
435
+ type Company @table @export {
436
+ id: ID @primaryKey
437
+ departments: [Department] @relationship(to: "companyId")
438
+ }
439
+
440
+ type Department @table @export {
441
+ id: ID @primaryKey
442
+ companyId: ID @indexed
443
+ employees: [Employee] @relationship(to: "departmentId")
444
+ }
445
+
446
+ type Employee @table @export {
447
+ id: ID @primaryKey
448
+ departmentId: ID @indexed
449
+ tag: String @indexed
450
+ level: Int @indexed
451
+ }
442
452
  ```
443
453
 
454
+ | Query | Matches | Why |
455
+ | -------------------------------------------------------------------------- | ---------- | ----------------------------------------------------------- |
456
+ | `departments.employees.tag=oncall&departments.employees.level=ge=number:3` | c1, c2, c3 | each condition may be met by a different department |
457
+ | `departments[employees.tag=oncall&employees.level=ge=number:3]` | c1, c2 | one department must meet both, possibly by different people |
458
+ | `departments[employees[tag=oncall&level=ge=3]]` | c2 | one employee must meet both |
459
+ | `departments.employees[tag=oncall&level=ge=3]` | c2 | same as the previous row |
460
+
444
461
  #### Notes
445
462
 
446
- - Index all foreign key fields (e.g., `brandId`, `price`) and outer filter fields to support efficient join queries.
447
- - Use a unique field as the final sort key (e.g., `sort(price,id)`) to ensure deterministic ordering when values are equal.
448
- - A nested `limit()` bounds children per parent, not the number of parents. Apply a separate outer `limit()` to bound the parent result set.
463
+ - `@relationship(from: ...)` — the foreign key is in **this** table; result type can be singular or array.
464
+ - `@relationship(to: ...)` — the foreign key is in the **target** table; result type **must** be an array.
465
+ - `@relationship(from: attribute, to: attribute)` — foreign-key-to-foreign-key join; result type **must** be an array.
466
+ - Selecting a relationship without an outer dot-syntax filter is a LEFT JOIN; parents without matching related records are retained.
467
+ - An outer filter joined with `&` (e.g., `products.price=ge=number:30`) removes parents without a qualifying related record **and** narrows the related records returned by `select()`. A condition inside an OR only decides which parents match; it does not narrow selected related records.
468
+ - A nested filter inside `select(...,products[...])` only narrows projected children; it does not remove the parent.
469
+ - Separate `&`-joined conditions on a one-to-many relationship may each be satisfied by a **different** related record. Use `relationship[cond1&cond2]` bracket syntax to require one related record to meet all conditions.
470
+ - Self-referential relationships (a table referencing itself) are supported for parent-child hierarchies.
449
471
  - Nested filters require a table-backed relationship. Filters on plain JSON projections or full-text predicates inside a relationship selection return HTTP 400.
450
- - Invalid `limit()` bounds (negative or fractional values) return HTTP 400. `limit(0)` returns no children without reading them.
451
- - A condition traversing another relationship qualifies the immediate child only; it does not narrow that child's projected descendants. Place grandchild constraints in the grandchild's own bracket selection.
452
- - Schemas can define self-referential relationships to enable parent-child hierarchies within a single table.
472
+ - Keep parent pages small using outer `limit()`; a nested `limit()` bounds children per parent, not the number of parents.
453
473
 
454
474
  ### 1.4 Vector Indexing
455
475
 
@@ -491,7 +511,7 @@ Apply this rule when adding a vector similarity search capability to a Harper ta
491
511
  });
492
512
  ```
493
513
 
494
- 4. **Apply a function predicate during traversal** using `vectorFilter` (JavaScript API only). The function receives each candidate record and must return a synchronous boolean. It must be side-effect free and fast:
514
+ 4. **Apply a function predicate** using `vectorFilter` (JavaScript API only) for predicates not expressible as conditions. The function receives each candidate record and must return a synchronous boolean. Records are frozen:
495
515
 
496
516
  ```javascript
497
517
  let results = Document.search(
@@ -518,7 +538,7 @@ Apply this rule when adding a vector similarity search capability to a Harper ta
518
538
  });
519
539
  ```
520
540
 
521
- 6. **Include computed distance in results** by adding `$distance` to `select`. Works with both `sort`-based and threshold queries:
541
+ 6. **Include computed distance** in results by adding `$distance` to `select`. Works with both `sort`-based ranking and threshold queries:
522
542
 
523
543
  ```javascript
524
544
  let results = Document.search({
@@ -537,7 +557,10 @@ Apply this rule when adding a vector similarity search capability to a Harper ta
537
557
  });
538
558
  ```
539
559
 
540
- 8. **Tune filtered traversal** with `ef` and `filterExpansion` when a `vectorFilter` is very selective. The visit budget is `ef * filterExpansion` nodes (`filterExpansion` defaults to `24`):
560
+ - `distance` — overrides the index distance function: `"cosine"`, `"euclidean"`, or `"dotProduct"`.
561
+ - `ef` — overrides the search exploration budget. Higher values improve recall at the cost of latency.
562
+
563
+ 8. **Tune filtered traversal** when a `vectorFilter` is very selective. Filtered traversal is bounded by `ef * filterExpansion` nodes visited. Raise either knob to trade latency for recall:
541
564
 
542
565
  ```javascript
543
566
  let results = Document.search(
@@ -550,99 +573,105 @@ Apply this rule when adding a vector similarity search capability to a Harper ta
550
573
  );
551
574
  ```
552
575
 
553
- 9. **Enforce row-level access control** using `rowFilter` on search and subscription targets (JavaScript API only). Attach it in an operation override. For vector queries, `rowFilter` participates in HNSW traversal so callers receive the k nearest _matching_ records:
554
-
555
- ```javascript
556
- function canReadReport(record, context) {
557
- const user = context.user;
558
- if (user?.role?.permission?.super_user) return true;
559
- return user?.username != null && record.ownerId != null && record.ownerId === user.username;
560
- }
576
+ 9. **Configure HNSW index parameters** in the schema directive. Use the parameter table below to choose values:
561
577
 
562
- export class Reports extends tables.Reports {
563
- search(target) {
564
- target.rowFilter = canReadReport;
565
- return super.search(target);
566
- }
578
+ ```graphql
579
+ type Document @table {
580
+ id: Long @primaryKey
581
+ textEmbeddings: [Float]
582
+ @indexed(type: "HNSW", distance: "euclidean", optimizeRouting: 0, efConstructionSearch: 100)
567
583
  }
568
584
  ```
569
585
 
570
- ##### HNSW Index Parameters
586
+ | Parameter | Default | Description |
587
+ | ---------------------- | ----------------- | ------------------------------------------------------------------------------------------------ |
588
+ | `distance` | `"cosine"` | Distance function: `"cosine"`, `"euclidean"`, or `"dotProduct"` |
589
+ | `efConstruction` | `100` | Max nodes explored during index construction. Higher = better recall, lower = better performance |
590
+ | `M` | `16` | Preferred connections per graph layer |
591
+ | `optimizeRouting` | `0.5` | Heuristic aggressiveness for omitting redundant connections (0 = off, 1 = most aggressive) |
592
+ | `mL` | computed from `M` | Normalization factor for level generation |
593
+ | `efConstructionSearch` | auto-scaled | Max nodes explored during search. When unset, auto-scales with index size |
594
+ | `quantization` | — | `"int8"` stores vectors quantized to int8 |
595
+ | `filterExpansion` | `24` | Visit-budget multiplier for filtered search: visits at most `ef * filterExpansion` nodes |
571
596
 
572
- Configure parameters directly on `@indexed(type: "HNSW", ...)`:
597
+ > Use `efConstructionSearch` (not `efSearchConstruction`). Changing `efConstructionSearch` on an existing index does not trigger a rebuild. Changing structural parameters (`distance`, `M`, `efConstruction`, `quantization`) does rebuild the index.
573
598
 
574
- | Parameter | Default | Description |
575
- | ---------------------- | ----------------- | ------------------------------------------------------------------------------------------------ |
576
- | `distance` | `"cosine"` | Distance function: `"cosine"`, `"euclidean"`, or `"dotProduct"` |
577
- | `efConstruction` | `100` | Max nodes explored during index construction. Higher = better recall, lower = better performance |
578
- | `M` | `16` | Preferred connections per graph layer |
579
- | `optimizeRouting` | `0.5` | Heuristic aggressiveness for omitting redundant connections (0 = off, 1 = most aggressive) |
580
- | `mL` | computed from `M` | Normalization factor for level generation |
581
- | `efConstructionSearch` | auto-scaled | Max nodes explored during search. When unset, auto-scales with index size |
582
- | `quantization` | — | `"int8"` stores vectors quantized to int8 |
583
- | `filterExpansion` | `24` | Visit-budget multiplier for filtered search: visits at most `ef * filterExpansion` nodes |
599
+ 10. **Enable vector quantization** with `quantization: "int8"` to reduce index size and memory usage. Graph navigation uses quantized distances; nearest-neighbor `sort` queries are re-ranked against full-precision vectors for exact results:
584
600
 
585
- Per-query `sort` descriptor options:
586
-
587
- | Option | Values | Description |
588
- | ---------- | ----------------------------------------- | ------------------------------------------------------ |
589
- | `distance` | `"cosine"`, `"euclidean"`, `"dotProduct"` | Overrides the index's distance function for this query |
590
- | `ef` | integer | Overrides the search exploration budget for this query |
601
+ ```graphql
602
+ type Document @table {
603
+ id: Long @primaryKey
604
+ textEmbeddings: [Float] @indexed(type: "HNSW", quantization: "int8")
605
+ }
606
+ ```
591
607
 
592
608
  #### Examples
593
609
 
594
- **Index with custom HNSW parameters:**
610
+ ##### Basic nearest-neighbor search
595
611
 
596
612
  ```graphql
597
613
  type Document @table {
598
614
  id: Long @primaryKey
599
- textEmbeddings: [Float]
600
- @indexed(type: "HNSW", distance: "euclidean", optimizeRouting: 0, efConstructionSearch: 100)
615
+ textEmbeddings: [Float] @indexed(type: "HNSW")
601
616
  }
602
617
  ```
603
618
 
604
- **Index with int8 quantization:**
605
-
606
- ```graphql
607
- type Document @table {
608
- id: Long @primaryKey
609
- textEmbeddings: [Float] @indexed(type: "HNSW", quantization: "int8")
610
- }
611
- ```
612
-
613
- **Nearest-neighbor search with distance included:**
614
-
615
619
  ```javascript
616
620
  let results = Document.search({
617
- select: ['name', '$distance'],
618
621
  sort: { attribute: 'textEmbeddings', target: searchVector },
619
622
  limit: 5,
620
623
  });
621
624
  ```
622
625
 
623
- **Filtered traversal with tuned budget:**
626
+ ##### Filtered vector search with row-level access control
624
627
 
625
628
  ```javascript
626
- let results = Document.search(
627
- {
628
- sort: { attribute: 'textEmbeddings', target: searchVector, ef: 200, filterExpansion: 40 },
629
- vectorFilter: (record) => record.category === 'rare',
630
- limit: 10,
631
- },
632
- context,
633
- );
629
+ function canReadReport(record, context) {
630
+ const user = context.user;
631
+ if (user?.role?.permission?.super_user) return true;
632
+ return user?.username != null && record.ownerId != null && record.ownerId === user.username;
633
+ }
634
+
635
+ export class Reports extends tables.Reports {
636
+ get(target) {
637
+ const context = this.getContext();
638
+ if (target.isCollection) {
639
+ target.rowFilter = canReadReport;
640
+ } else if (!canReadReport(this, context)) {
641
+ return new Response(null, { status: 404 });
642
+ }
643
+ return super.get(target);
644
+ }
645
+
646
+ search(target) {
647
+ target.rowFilter = canReadReport;
648
+ return super.search(target);
649
+ }
650
+
651
+ subscribe(request) {
652
+ request.rowFilter = canReadReport;
653
+ return super.subscribe(request);
654
+ }
655
+ }
656
+ ```
657
+
658
+ ##### Custom HNSW parameters with cosine distance
659
+
660
+ ```graphql
661
+ type Document @table {
662
+ id: Long @primaryKey
663
+ textEmbeddings: [Float]
664
+ @indexed(type: "HNSW", distance: "euclidean", optimizeRouting: 0, efConstructionSearch: 100)
665
+ }
634
666
  ```
635
667
 
636
668
  #### Notes
637
669
 
638
- - `vectorFilter` and `rowFilter` are available from the JavaScript API only; they cannot be set through REST or QUERY request data.
639
- - `vectorFilter` functions must be synchronous, side-effect free, and fast — they can run once per candidate record visited during traversal; verdicts are memoized per query. Records passed to them are frozen.
640
- - `rowFilter` does not apply to a direct primary-key `get`.
641
- - Changing `efConstructionSearch` on an existing index does not trigger a rebuild. Structural parameters (`distance`, `M`, `efConstruction`, `quantization`) do rebuild the index when changed.
642
- - With `quantization: "int8"`, nearest-neighbor `sort` queries re-rank results against full-precision vectors, restoring exact ordering and exact `$distance` values. Distance-threshold (`lt`/`le`) queries filter on the approximate distance.
643
- - The correct parameter name is `efConstruction` (seeds the construction budget) and `efConstructionSearch` (controls search budget). The name `efSearchConstruction` is a previous documentation error.
644
- - When no `ef` is passed and `efConstructionSearch` (or `efConstruction`) is not explicitly set, the search budget auto-scales with index size.
645
- - `cosine` is the default distance function when `distance` is not specified.
670
+ - `vectorFilter` is available from the JavaScript API only — it cannot be expressed in a REST query string. It must be synchronous, side-effect free, and fast. Verdicts are memoized per query.
671
+ - `rowFilter` is also JavaScript API only. It participates in HNSW traversal so callers receive the k nearest _matching_ records. It does not apply to direct primary-key `get` calls. Prefer indexed query conditions for predicates that can be expressed declaratively.
672
+ - When a query passes no `ef` and the index does not explicitly configure `efConstructionSearch` (or `efConstruction`), the search budget auto-scales with index size.
673
+ - Distance-threshold (`lt`/`le`) queries with `quantization: "int8"` filter on approximate distance, not re-ranked full-precision distance.
674
+ - `$distance` is available in both `sort`-based ranking and `conditions`-based threshold queries.
646
675
 
647
676
  ### 1.5 Using the Blob Data Type
648
677
 
@@ -866,25 +895,25 @@ export class Photo extends tables.Photo {
866
895
 
867
896
  ## 2. API & Communication
868
897
 
869
- ### 2.1 Automatic APIs
898
+ ### 2.1 Automatic REST and WebSocket APIs
870
899
 
871
- Instructions for the agent to follow when using Harper's automatically generated REST and WebSocket APIs for exported tables and resources.
900
+ Instructions for the agent to follow when exposing Harper tables and custom resources as REST and WebSocket endpoints using Harper's automatically generated API layer.
872
901
 
873
902
  **Instead of:** Hand-written CRUD routes (Express, Fastify, Next.js API routes). Use this rule rather than adding that service. See `harper-architecture`.
874
903
 
875
904
  #### When to Use
876
905
 
877
- Apply this rule when enabling HTTP REST endpoints or WebSocket subscriptions for Harper tables and custom resources. Use it whenever you need to configure the REST plugin, understand the auto-generated endpoint surface, or wire up real-time WebSocket connections. See [querying-rest-apis.md](querying-rest-apis.md) for query syntax and [real-time-apps.md](real-time-apps.md) for real-time patterns.
906
+ Apply this rule when enabling HTTP REST access to Harper tables, when configuring WebSocket subscriptions on resource paths, or when customizing how exported resources respond to standard HTTP methods. Use it any time you need to wire up `rest: true`, handle conditional requests, or implement a `connect()` handler.
878
907
 
879
908
  #### How It Works
880
909
 
881
- 1. **Enable the REST plugin** by adding `rest: true` to your application's `config.yaml`:
910
+ 1. **Enable the REST plugin** by adding `rest: true` to the application's `config.yaml`:
882
911
 
883
912
  ```yaml
884
913
  rest: true
885
914
  ```
886
915
 
887
- Extended options:
916
+ Additional options:
888
917
 
889
918
  ```yaml
890
919
  rest:
@@ -893,7 +922,9 @@ Apply this rule when enabling HTTP REST endpoints or WebSocket subscriptions for
893
922
  exactCount: true # opt in to Prefer: count=exact scans (off by default)
894
923
  ```
895
924
 
896
- 2. **Export the table in your schema** using `@export`. Tables are not exposed by default — `@export` is required for Harper to register a REST route:
925
+ A component directory with no `config.yaml` at all inherits a built-in default that enables `rest` and loads `*.graphql` from the component root. As soon as a `config.yaml` exists it is used verbatim — omitting `rest` turns REST off. Always write `rest: true` when adding a config file.
926
+
927
+ 2. **Export the table in the schema** using `@export`. Tables are not exported by default.
897
928
 
898
929
  ```graphql
899
930
  type Product @table @export {
@@ -903,41 +934,76 @@ Apply this rule when enabling HTTP REST endpoints or WebSocket subscriptions for
903
934
  }
904
935
  ```
905
936
 
906
- Without `@export`, the table has no REST route and callers receive `404`. Without `rest: true` in `config.yaml`, even an exported table does not respond to HTTP requests.
937
+ Reference the schema file from `config.yaml`:
938
+
939
+ ```yaml
940
+ graphqlSchema:
941
+ files: schema.graphql
942
+ rest: true
943
+ ```
944
+
945
+ Both `@export` and `rest: true` are required. Without `@export`, no REST route is registered and callers receive `404`. Without REST enabled, even an exported table does not respond to HTTP requests.
907
946
 
908
- 3. **Understand the auto-generated endpoints**. With both `@export` and `rest: true` in place, Harper registers the following on the application HTTP server port (default `9926`):
947
+ 3. **Understand the auto-generated endpoints**. With `Product` exported, Harper registers these routes on the application HTTP server port (default `9926`) with no handler code required:
909
948
 
910
- | Endpoint | Description |
911
- | ---------------------------- | ------------------------------------------------------------------------------------ |
912
- | `GET /Product` | Resource description — table name, database, declared attributes |
913
- | `GET /Product/` | Record collection; append query parameters to filter, sort, page |
914
- | `GET /Product/{id}` | Single record by primary key; `404` if not found |
915
- | `GET /Product/{id}.property` | Single declared property of one record |
916
- | `POST /Product/` | Creates a record; responds `201`; primary key returned in `Location` header |
917
- | `PUT /Product/{id}` | Creates or replaces the record at `{id}` (upsert); omitted properties are removed |
918
- | `PATCH /Product/{id}` | Shallow-merges body into existing record; unspecified top-level properties preserved |
919
- | `DELETE /Product/{id}` | Deletes the record at `{id}` |
920
- | `DELETE /Product/?query` | Deletes every record matching the query |
921
- - `HEAD` is served exactly as `GET` with the response body omitted. `QUERY` is accepted on the collection path (`QUERY /Product/`) and runs a search taken from the request body rather than the URL.
949
+ | Endpoint | Description |
950
+ | ---------------------------- | --------------------------------------------------------------------------- |
951
+ | `GET /Product` | Resource description (table name, database, attributes) |
952
+ | `GET /Product/` | Record collection; append query parameters to filter/sort/page |
953
+ | `GET /Product/{id}` | Single record by primary key; `404` if not found |
954
+ | `GET /Product/{id}.property` | Single declared property of one record |
955
+ | `POST /Product/` | Creates a record; responds `201`; primary key returned in `Location` header |
956
+ | `PUT /Product/{id}` | Creates or replaces the record at `{id}` (upsert) |
957
+ | `PATCH /Product/{id}` | Shallow-merges body into existing record |
958
+ | `DELETE /Product/{id}` | Deletes the record at `{id}` |
959
+ | `DELETE /Product/?query` | Deletes every record matching the query |
922
960
 
923
- 4. **Use the correct URL structure**. The trailing slash is significant:
961
+ The trailing slash is significant: `/Product` addresses the resource itself; `/Product/` addresses its record collection.
924
962
 
925
- | Path | Addresses |
926
- | ------------------------- | ---------------------------------------------- |
927
- | `/my-resource` | The resource itself (metadata) |
928
- | `/my-resource/` | The record collection |
929
- | `/my-resource/record-id` | A specific record by primary key |
930
- | `/my-resource/record-id/` | Collection of records with the given id prefix |
963
+ 4. **Understand the URL structure**. Harper follows a consistent URL pattern across all resources:
931
964
 
932
- 5. **Handle `POST` primary keys via `Location`**. On a successful `POST`, the new record's primary key is returned in the `Location` response header — the value supplied by the body, or a Harper-assigned key if the body omitted the primary-key property.
965
+ | Path | Description |
966
+ | -------------------------------------------- | ------------------------------------------------------------------------------------------------- |
967
+ | `/my-resource` | Root path — returns a description of the resource (e.g., table metadata) |
968
+ | `/my-resource/` | Trailing slash indicates a collection — represents all records; append query parameters to search |
969
+ | `/my-resource/record-id` | A specific record identified by its primary key |
970
+ | `/my-resource/record-id/` | Trailing slash — the collection of records with the given id prefix |
971
+ | `/my-resource/record-id/with/multiple/parts` | Record id with multiple path segments |
933
972
 
934
- 6. **Use conditional GET requests for caching**. GET responses include an `ETag` header. Send `If-None-Match` on subsequent requests; if the record is unchanged, Harper returns `304 Not Modified` with no body.
973
+ 5. **Handle HTTP methods correctly**:
974
+ - **GET** — Retrieves a record or collection. Use `GET /MyTable/123` for a single record, `GET /MyTable/?name=Harper` for a filtered collection. The `.msgpack` suffix (along with `.json`, `.cbor`, `.csv`) is a reserved content-type selector and takes precedence over property access via dot syntax.
975
+ - **POST** — Creates a record. The new record's primary key is returned in the `Location` response header. A `POST` to a key that already exists fails with `409`.
976
+ - **PUT** — Upserts a record. The stored record matches the body exactly; omitted properties are removed. Harper always re-stamps an `@updatedTime` attribute and preserves `@createdTime`.
977
+ - **PATCH** — Shallow-merges the body. Top-level unspecified properties are preserved; nested objects in the body replace the stored sub-object wholesale.
978
+ - **DELETE** — Deletes by id or by query. `DELETE /MyTable/` with no query parameters deletes every record.
935
979
 
936
- 7. **Apply `@updatedTime` for write timestamps**. On `PUT`, Harper re-stamps any `@updatedTime` attribute with the time of the write, preserves any `@createdTime` value from the original record, and forces the primary key to match the `{id}` in the URL.
980
+ 6. **Use conditional requests for caching**. GET responses include an `ETag` header. Clients send `If-None-Match` on subsequent requests; if the record is unchanged Harper returns `304 Not Modified` with no body.
937
981
 
938
- 8. **Control OpenAPI visibility**. Every non-hidden exported resource appears in the generated OpenAPI document at `GET /openapi`. To exclude a type, mark it `@hidden` in the schema, or set `static hidden = true` on a programmatic resource class.
982
+ 7. **Set the `Accept` header** to request a specific response format. Use `Content-Type` for request bodies. See [querying-rest-apis.md](querying-rest-apis.md) for the full query syntax and content-type details.
939
983
 
940
- 9. **Implement a custom `connect()` handler** when default subscription behavior is insufficient. The method must return an async iterable that produces messages to send to the client:
984
+ 8. **Use WebSocket subscriptions** on the same resource paths. WebSocket support is enabled automatically with `rest: true`. Connecting to a resource URL subscribes to change events:
985
+
986
+ ```javascript
987
+ let ws = new WebSocket('wss://server/my-resource/341');
988
+ ws.onmessage = (event) => {
989
+ let data = JSON.parse(event.data);
990
+ };
991
+ ```
992
+
993
+ Disable WebSocket support without disabling REST:
994
+
995
+ ```yaml
996
+ rest:
997
+ webSocket: false
998
+ ```
999
+
1000
+ For MQTT over WebSockets, set the sub-protocol header:
1001
+
1002
+ ```
1003
+ Sec-WebSocket-Protocol: mqtt
1004
+ ```
1005
+
1006
+ 9. **Implement a custom `connect()` handler** on a resource class to control WebSocket behavior. The method must return an async iterable producing messages to send to the client:
941
1007
 
942
1008
  ```javascript
943
1009
  export class Echo extends Resource {
@@ -949,28 +1015,25 @@ Apply this rule when enabling HTTP REST endpoints or WebSocket subscriptions for
949
1015
  }
950
1016
  ```
951
1017
 
952
- 10. **Connect via WebSocket**. WebSocket support is enabled automatically with `rest: true`. Connect to a resource URL to subscribe to change events:
1018
+ 10. **Control OpenAPI visibility**. Every non-hidden exported resource is included in the generated OpenAPI document at `GET /openapi`. To exclude a type from the OpenAPI document, mark it `@hidden` in the schema:
953
1019
 
954
- ```javascript
955
- let ws = new WebSocket('wss://server/my-resource/341');
956
- ws.onmessage = (event) => {
957
- let data = JSON.parse(event.data);
958
- };
1020
+ ```graphql
1021
+ type Secret @table @export @hidden {
1022
+ id: Long @primaryKey
1023
+ }
959
1024
  ```
960
1025
 
961
- Disable WebSocket support independently with `webSocket: false` under `rest`.
962
-
963
- 11. **Use MQTT over WebSockets** by setting the sub-protocol header:
1026
+ Or set `static hidden = true` on a programmatic resource class:
964
1027
 
965
- ```
966
- Sec-WebSocket-Protocol: mqtt
1028
+ ```javascript
1029
+ export class Secret extends Resource {
1030
+ static hidden = true;
1031
+ }
967
1032
  ```
968
1033
 
969
- 12. **Specify response format** using the `Accept` header. The suffixes `.json`, `.cbor`, `.msgpack`, and `.csv` are reserved as content-type selectors on property paths and take precedence over property names.
970
-
971
1034
  #### Examples
972
1035
 
973
- **Schema and config for a REST-enabled table:**
1036
+ **Full schema and config setup:**
974
1037
 
975
1038
  ```graphql
976
1039
  # schema.graphql
@@ -988,132 +1051,133 @@ graphqlSchema:
988
1051
  rest: true
989
1052
  ```
990
1053
 
991
- **Conditional GET with ETag caching:**
1054
+ **Conditional GET with caching:**
992
1055
 
993
- ```http
1056
+ ```
994
1057
  GET /Product/123
995
- If-None-Match: "abc123"
1058
+ If-None-Match: "abc123etag"
996
1059
  ```
997
1060
 
998
1061
  Response when unchanged:
999
1062
 
1000
1063
  ```
1001
1064
  HTTP/1.1 304 Not Modified
1002
- ETag: "abc123"
1065
+ ETag: "abc123etag"
1003
1066
  ```
1004
1067
 
1005
1068
  **POST and read the Location header:**
1006
1069
 
1007
- ```http
1070
+ ```
1008
1071
  POST /Product/
1009
1072
  Content-Type: application/json
1010
1073
 
1011
- { "name": "Widget" }
1074
+ { "name": "Widget", "price": 9.99 }
1012
1075
  ```
1013
1076
 
1077
+ Response:
1078
+
1014
1079
  ```
1015
1080
  HTTP/1.1 201 Created
1016
1081
  Location: 7f3a9c
1017
1082
  ```
1018
1083
 
1019
- **PATCH (shallow merge only):**
1084
+ **PATCH shallow merge:**
1020
1085
 
1021
- ```http
1086
+ ```
1022
1087
  PATCH /Product/123
1023
1088
  Content-Type: application/json
1024
1089
 
1025
- { "status": "active" }
1090
+ { "price": 12.99 }
1026
1091
  ```
1027
1092
 
1028
- **Custom WebSocket connect handler:**
1093
+ Preserves `name` and all other top-level properties; replaces any nested object in `price` wholesale.
1094
+
1095
+ **Custom WebSocket connect handler with timer:**
1029
1096
 
1030
1097
  ```javascript
1031
- export class Echo extends Resource {
1032
- async *connect(incomingMessages) {
1033
- for await (let message of incomingMessages) {
1034
- yield message; // echo each message back
1035
- }
1036
- }
1037
- }
1038
- ```
1098
+ export class Example extends Resource {
1099
+ connect(incomingMessages) {
1100
+ let outgoingMessages = super.connect();
1039
1101
 
1040
- **Hide a resource from OpenAPI — schema directive:**
1102
+ let timer = setInterval(() => {
1103
+ outgoingMessages.send({ greeting: 'hi again!' });
1104
+ }, 1000);
1041
1105
 
1042
- ```graphql
1043
- type InternalLog @table @export @hidden {
1044
- id: Long @primaryKey
1045
- }
1046
- ```
1106
+ incomingMessages.on('data', (message) => {
1107
+ outgoingMessages.send(message);
1108
+ });
1047
1109
 
1048
- **Hide a resource from OpenAPI — programmatic class:**
1110
+ outgoingMessages.on('close', () => {
1111
+ clearInterval(timer);
1112
+ });
1049
1113
 
1050
- ```javascript
1051
- export class InternalLog extends Resource {
1052
- static hidden = true;
1114
+ return outgoingMessages;
1115
+ }
1053
1116
  }
1054
1117
  ```
1055
1118
 
1056
1119
  #### Notes
1057
1120
 
1058
- - A component directory with **no** `config.yaml` inherits Harper's built-in default, which enables `rest` automatically. As soon as a `config.yaml` exists, it is used verbatim — omitting `rest` from it disables REST even if the directory previously had it without a config file. Always add `rest: true` when adding a config file.
1059
- - `POST /Product` (no trailing slash) returns `404`. The trailing slash is required for collection operations.
1060
- - A `POST` to a primary key that already exists fails with `409` rather than overwriting.
1061
- - PATCH merge is **shallow** — a nested object in the body replaces the stored nested object wholesale; nested properties not included in the body are dropped.
1062
- - The `.msgpack` suffix (along with `.json`, `.cbor`, `.csv`) is reserved as a content-type selector and cannot be used as a property name in dot-path access.
1063
- - Server-Sent Events subscriptions are served on the same paths, negotiated via `Accept: text/event-stream`; they are not affected by the `webSocket` option.
1064
- - See [querying-rest-apis.md](querying-rest-apis.md) for full query syntax on collection endpoints and [real-time-apps.md](real-time-apps.md) for real-time subscription patterns.
1121
+ - The trailing slash is always significant: `/Product` is the resource descriptor; `/Product/` is the record collection. `POST /Product` (no slash) returns `404`.
1122
+ - `HEAD` is served exactly as `GET` with the response body omitted. `QUERY` is accepted on the collection path (`QUERY /Product/`) and runs a search taken from the request body rather than the URL.
1123
+ - `PUT` always forces the primary key to match the `{id}` in the URL, re-stamps `@updatedTime`, and preserves `@createdTime` from the original record.
1124
+ - PATCH merge is shallow only. A nested object in the body replaces the entire stored sub-object; omitted nested properties are lost.
1125
+ - Server-Sent Events subscriptions are served on the same paths, negotiated via `Accept: text/event-stream`. They are not affected by the `webSocket` option.
1126
+ - Leaving `@export` on the schema while also exporting a same-named subclass produces conflicting endpoints. When a JavaScript subclass should own the URL, omit `@export` from the schema and export the class instead.
1127
+ - See [real-time-apps.md](real-time-apps.md) for patterns around WebSocket and event-driven resource design.
1128
+ - See [querying-rest-apis.md](querying-rest-apis.md) for filtering, sorting, pagination, and content-type details.
1065
1129
 
1066
- ### 2.2 Querying Harper REST APIs
1130
+ ### 2.2 Querying REST APIs
1067
1131
 
1068
- Instructions for the agent to filter, sort, select, and paginate records using Harper's URL-based query language on REST collection endpoints.
1132
+ Instructions for the agent to filter, sort, select, and paginate Harper REST API collections using URL query parameters.
1069
1133
 
1070
1134
  #### When to Use
1071
1135
 
1072
- Apply this rule whenever building or modifying REST requests against Harper collection paths that need attribute filtering, comparison operators, sorting, field selection, pagination, or relationship joins. This rule covers all query parameter construction for Harper's automatic REST APIs (see [automatic-apis.md](automatic-apis.md)).
1136
+ Apply this rule when building or modifying HTTP requests against Harper REST collection endpoints that need attribute filtering, comparison operators, sorting, field selection, or pagination. Use it whenever constructing query strings for `GET` requests against table-backed resources exposed via `@export`. See [automatic-apis.md](automatic-apis.md) for how those endpoints are created.
1073
1137
 
1074
1138
  #### How It Works
1075
1139
 
1076
- 1. **Filter by attribute**: Add query parameters matching attribute name to value. The queried attribute must be indexed.
1140
+ 1. **Filter by attribute**: Add query parameters matching attribute names. The queried attribute must be indexed.
1077
1141
 
1078
1142
  ```
1079
1143
  GET /Product/?category=software
1080
1144
  GET /Product/?category=software&inStock=true
1081
- GET /Product/?discount=null
1082
1145
  ```
1083
1146
 
1084
- 2. **Apply comparison operators (FIQL syntax)**: Use FIQL operators in the query string for range and string comparisons.
1147
+ 2. **Apply FIQL comparison operators**: Use FIQL syntax for range and equality comparisons.
1085
1148
 
1086
- | Operator | Meaning |
1087
- | -------------------- | -------------------------------------- |
1088
- | `==` | Equal |
1089
- | `=lt=` | Less than |
1090
- | `=le=` | Less than or equal |
1091
- | `=gt=` | Greater than |
1092
- | `=ge=` | Greater than or equal |
1093
- | `=ne=`, `!=` | Not equal |
1094
- | `=ct=` | Contains (strings) |
1095
- | `=sw=`, `==<value>*` | Starts with (strings) |
1096
- | `=ew=` | Ends with (strings) |
1097
- | `=`, `===` | Strict equality (no type conversion) |
1098
- | `!==` | Strict inequality (no type conversion) |
1149
+ | Operator | Meaning |
1150
+ | ----------------------------------------- | --------------------------------------- |
1151
+ | `==` | Equal |
1152
+ | `=lt=` | Less than |
1153
+ | `=le=` | Less than or equal |
1154
+ | `=gt=` | Greater than |
1155
+ | `=ge=` | Greater than or equal |
1156
+ | `=ne=`, `!=` | Not equal |
1157
+ | `=between=(low,high)`, `=gele=(low,high)` | At least `low` and at most `high` |
1158
+ | `=gelt=(low,high)` | At least `low` and less than `high` |
1159
+ | `=gtle=(low,high)` | Greater than `low` and at most `high` |
1160
+ | `=gtlt=(low,high)` | Greater than `low` and less than `high` |
1161
+ | `=ct=` | Contains (strings) |
1162
+ | `=sw=`, `==<value>*` | Starts with (strings) |
1163
+ | `=ew=` | Ends with (strings) |
1164
+ | `=`, `===` | Strict equality (no type conversion) |
1165
+ | `!==` | Strict inequality (no type conversion) |
1099
1166
 
1100
1167
  ```
1101
1168
  GET /Product/?price=gt=100
1102
1169
  GET /Product/?price=le=20
1103
- GET /Product/?name==Keyboard*
1104
1170
  GET /Product/?category=software&price=gt=100&price=lt=200
1105
- GET /Product/?listDate=gt=2017-03-08T09%3A30%3A00.000Z
1171
+ GET /Product/?price=between=(100,200)
1106
1172
  ```
1107
1173
 
1108
- 3. **Chain conditions for range queries**: Omit the attribute name on the second condition to apply it to the same attribute.
1174
+ For date fields, URL-encode colons as `%3A`:
1109
1175
 
1110
1176
  ```
1111
- GET /Product/?price=gt=100&lt=200
1177
+ GET /Product/?listDate=gt=2017-03-08T09%3A30%3A00.000Z
1112
1178
  ```
1113
1179
 
1114
- Chaining supports `gt`/`ge` combined with `lt`/`le` only.
1115
-
1116
- 4. **Apply type conversion**: For FIQL comparators, Harper converts values automatically. Use explicit type prefixes when needed.
1180
+ 3. **Apply type conversion**: For FIQL comparators, Harper converts values automatically. Use explicit type prefixes when needed.
1117
1181
 
1118
1182
  | Syntax | Behavior |
1119
1183
  | ----------------------------------------- | ------------------------------------------- |
@@ -1125,28 +1189,44 @@ Apply this rule whenever building or modifying REST requests against Harper coll
1125
1189
  | `name==string:some%20text` | Keep as string with URL decode |
1126
1190
  | `name==date:2024-01-05T20%3A07%3A27.955Z` | Explicit Date conversion |
1127
1191
 
1128
- For strict operators (`=`, `===`, `!==`), no automatic type conversion is applied.
1192
+ If the attribute declares a type in the schema (e.g., `Float`), values are always converted to that type before searching. For strict operators (`=`, `===`, `!==`), no automatic type conversion is applied.
1129
1193
 
1130
- 5. **Combine conditions with OR logic**: Use `|` instead of `&`.
1194
+ 4. **Query for null values**: Use `=null` to match records storing `null`; use `=ne=null` to match records with a present, non-null value. A record that omits the attribute matches neither.
1131
1195
 
1132
1196
  ```
1133
- GET /Product/?rating=5|featured=true
1197
+ GET /Product/?discount=null
1198
+ GET /Product/?discount=ne=null
1134
1199
  ```
1135
1200
 
1136
- 6. **Group conditions**: Use parentheses or square brackets to control order of operations. Prefer square brackets when constructing queries from user input.
1201
+ 5. **Understand null and omitted value behavior**: Comparators treat a null value and an omitted value the same way, except `==null`, which matches a stored `null` only.
1202
+ - No range comparator (`=lt=`, `=le=`, `=gt=`, `=ge=`, `=between=`, etc.) matches a null or omitted value.
1203
+ - `=ne=` with a non-null value matches null and omitted values.
1204
+ - A `null` lower bound means no lower bound: `ge=null` and `gt=null` match every non-null value, and `between=(null,10)` returns the same records as `le=10`. A `null` upper bound: `lt=null` and `le=null` match nothing.
1205
+ - Sorting puts null and omitted values first; `-` puts them last, with no guaranteed order between the two.
1206
+
1207
+ 6. **Chain range conditions**: Omit the attribute name on the second condition to apply it to the same attribute.
1208
+
1209
+ ```
1210
+ GET /Product/?price=gt=100&lt=200
1211
+ ```
1212
+
1213
+ Chaining supports `gt`/`ge` combined with `lt`/`le` only.
1214
+
1215
+ 7. **Combine conditions with OR**: Use `|` instead of `&` for OR logic. Use parentheses or square brackets to group conditions.
1137
1216
 
1138
1217
  ```
1218
+ GET /Product/?rating=5|featured=true
1139
1219
  GET /Product/?rating=5|(price=gt=100&price=lt=200)
1140
1220
  GET /Product/?rating=5&[tag=fast|tag=scalable|tag=efficient]
1141
1221
  ```
1142
1222
 
1143
- Construct from JavaScript:
1223
+ Prefer square brackets when constructing queries from user input:
1144
1224
 
1145
1225
  ```javascript
1146
1226
  let url = `/Product/?rating=5&[${tags.map(encodeURIComponent).join('|')}]`;
1147
1227
  ```
1148
1228
 
1149
- 7. **Select specific properties with `select()`**: Specify which properties to include in the response.
1229
+ 8. **Select specific fields with `select()`**: Specify which properties to include in the response.
1150
1230
 
1151
1231
  | Syntax | Returns |
1152
1232
  | -------------------------------------- | ------------------------------------------------- |
@@ -1162,32 +1242,32 @@ Apply this rule whenever building or modifying REST requests against Harper coll
1162
1242
  GET /Product/?brand.name=Microsoft&select(name,brand{name})
1163
1243
  ```
1164
1244
 
1165
- 8. **Limit results with `limit(end)` or `limit(start,end)`**: Return records in a zero-based window from `start` (inclusive) to `end` (exclusive). `limit(end)` starts at zero.
1245
+ 9. **Limit results with `limit(end)` or `limit(start,end)`**: Return records in the zero-based window from `start` (inclusive) to `end` (exclusive). `limit(end)` starts at zero.
1166
1246
 
1167
1247
  ```
1168
1248
  GET /Product/?rating=gt=3&inStock=true&select(rating,name)&limit(20)
1169
1249
  GET /Product/?rating=gt=3&limit(10,30)
1170
1250
  ```
1171
1251
 
1172
- 9. **Sort results with `sort(property)` or `sort(+property,-property,...)`**: Prefix `+` or no prefix = ascending; `-` = descending. Multiple properties break ties in order.
1252
+ 10. **Sort results with `sort(property)` or `sort(+property,-property,...)`**: Prefix `+` or no prefix = ascending; `-` = descending. Multiple properties break ties in order.
1173
1253
 
1174
- ```
1175
- GET /Product/?rating=gt=3&sort(+name)
1176
- GET /Product/?sort(+rating,-price)
1177
- ```
1178
-
1179
- 10. **Request a total count for pagination**: Send a `Prefer` header on `GET` or `HEAD` requests. Counting requires a bounded `limit()` with page size no larger than 10,000 and window end no larger than 1,000,000.
1254
+ ```
1255
+ GET /Product/?rating=gt=3&sort(+name)
1256
+ GET /Product/?sort(+rating,-price)
1257
+ ```
1180
1258
 
1181
- | `Prefer` value | Meaning |
1182
- | ----------------- | --------------------------------------------------------------------------------------------- |
1183
- | `count=exact` | Exact number of matching records. Off by default; requires `exactCount: true` in REST config. |
1184
- | `count=estimated` | Fast approximate count. Does not require `exactCount`. |
1259
+ 11. **Request total counts for pagination**: Send a `Prefer` header on `GET` or `HEAD` requests. Counting is opt-in; without the header, no count is computed. Counting requires a bounded `limit()` with page size (`end - start`, or `end` in the one-argument form) no larger than **10,000** and window end no larger than **1,000,000**.
1185
1260
 
1186
1261
  ```
1187
1262
  GET /Product/?category=software&limit(0,25)
1188
1263
  Prefer: count=exact
1189
1264
  ```
1190
1265
 
1266
+ | `Prefer` value | Meaning |
1267
+ | ----------------- | ----------------------------------------------------------- |
1268
+ | `count=exact` | Exact count of matching records; scans the full matched set |
1269
+ | `count=estimated` | Fast planner/table estimate; cheap and approximate |
1270
+
1191
1271
  Response headers:
1192
1272
 
1193
1273
  ```
@@ -1197,57 +1277,93 @@ Apply this rule whenever building or modifying REST requests against Harper coll
1197
1277
  Preference-Applied: count=exact
1198
1278
  ```
1199
1279
 
1200
- Enable exact counts in REST configuration:
1280
+ | Header | Example | Description |
1281
+ | -------------------- | ----------------- | ------------------------------------------------------------------------- |
1282
+ | `Content-Range` | `items 0-24/1234` | 0-based inclusive range of records returned out of the total matching set |
1283
+ | `Range-Unit` | `items` | The unit used by `Content-Range` |
1284
+ | `Preference-Applied` | `count=exact` | The count mode the server applied |
1285
+ - The response status for paginated results is always `200`; Harper does not use `206 Partial Content`. `Content-Range` is informational.
1286
+ - Without `exactCount: true`, a `count=exact` request is served as an estimate (`Preference-Applied: count=estimated`). When the total cannot be produced, it is reported as `.../*` (e.g., `Content-Range: items 0-24/*`). `Preference-Applied` still echoes the requested mode, so an unavailable total (`.../*`) is distinct from a request that asked for no count.
1287
+ - Exact counting is off by default. Enable it in REST configuration:
1288
+ ```yaml
1289
+ rest:
1290
+ exactCount: true
1291
+ ```
1292
+ - When CORS is enabled, `Content-Range`, `Range-Unit`, and `Preference-Applied` are added to `Access-Control-Expose-Headers`.
1293
+ - Full-text queries cannot provide an exact total count; `Prefer: count=exact` returns `Content-Range: items <range>/*` for full-text indexes.
1294
+ - For JSON-array responses from full-text queries, check for a terminal error object shaped like `{ "error": "ErrorName: message" }` and treat the response as failed. A stream or JSON-parse error is also a failed page, not a record.
1295
+
1296
+ 12. **Use full-text operators**: For a `FullText` field carrying `@fullText`, use the field name as the query attribute with a full-text comparator.
1201
1297
 
1202
- ```yaml
1203
- rest:
1204
- exactCount: true
1298
+ ```
1299
+ GET /Product/?catalogSearch=matches=waterproof%20trail
1300
+ GET /Product/?catalogSearch=matches_phrase=trail%20running
1301
+ GET /Product/?catalogSearch=matches_prefix=waterproof%20tra&limit(10)
1205
1302
  ```
1206
1303
 
1207
- - The response status for paginated requests is always `200`; Harper does not use `206 Partial Content`.
1208
- - A `HEAD` request with a `count` preference returns the count headers with no body.
1209
- - When CORS is enabled, `Content-Range`, `Range-Unit`, and `Preference-Applied` are added to `Access-Control-Expose-Headers`.
1210
- - Without `exactCount: true`, a `count=exact` request is served as an estimate. When the total cannot be produced, it is reported as `*` (e.g., `Content-Range: items 0-24/*`). `Preference-Applied` still echoes the requested mode, so an unavailable total (`.../*`) is distinct from a request that asked for no count.
1304
+ Supported operators: `matches`, `matches_all`, `matches_phrase`, `matches_prefix`, `matches_fuzzy`, `matches_fuzzy_prefix`, plus a `not_` form of each. A negated full-text condition requires a non-negated condition on the same index. Repeat the parameter to provide both:
1211
1305
 
1212
- 11. **Query across relationships**: Use dot-syntax on attributes defined with `@relationship` in the schema.
1306
+ ```
1307
+ GET /Product/?catalogSearch=matches=waterproof&catalogSearch=not_matches=leather
1308
+ ```
1309
+
1310
+ Use operators like `=matches=`, `=matches_phrase=`, `=matches_prefix=`, and `=matches_fuzzy=` with the full-text field name. The REST URL syntax searches every source field in the index, so the caller must be allowed to read every source field.
1311
+
1312
+ 13. **Query across relationships**: Use dot-syntax to filter by related table attributes. Relationships must be defined with `@relationship` in the schema.
1213
1313
 
1214
1314
  ```
1215
1315
  GET /Product/?brand.name=Microsoft
1216
- GET /Brand/?products.name=Keyboard
1217
1316
  GET /Product/?brand.name=Microsoft&select(name,brand{name})
1218
1317
  ```
1219
1318
 
1220
- Selecting a relationship without an outer dot-syntax filter behaves as a LEFT JOIN. An outer filter such as `products.price=ge=number:30` removes parents without a qualifying product and also narrows the selected products.
1319
+ Selecting a relationship without an outer dot-syntax filter behaves as a LEFT JOIN. An outer filter joined with `&` removes parents without a qualifying related record. An outer relationship filter narrows nested relationship results. A nested filter inside `select(...,products[...])` only narrows projected products; it does not remove a parent.
1221
1320
 
1222
- 12. **Filter and page nested relationships**: Use square brackets inside `select()` to apply conditions, sort, and pagination to each parent's related records.
1321
+ 14. **Apply conditions one related record must meet**: Use square brackets after the relationship name to require one related record to satisfy all enclosed conditions.
1223
1322
 
1224
1323
  ```
1225
- GET /Brand/?select(id,name,products[price=ge=number:30&sort(price)&limit(1)&select(id,name,price)])
1324
+ GET /Brand/?products[name=Cable&price=ge=30]
1325
+ GET /Brand/?products[name=Keyboard&price=ge=30]
1226
1326
  ```
1227
1327
 
1228
- An outer relationship filter also narrows that relationship's nested results — do not repeat the outer filter inside `[...]` unless you want to narrow further.
1328
+ Inside the brackets, name attributes of the related table directly (e.g., `price`, not `products.price`). Conditions combine with `&`, `|`, and groups.
1329
+
1330
+ 15. **Filter and page nested relationships**: Use square brackets inside `select()` to apply conditions, sort, and pagination to each parent's related records.
1331
+
1332
+ ```
1333
+ GET /Brand/?select(id,name,products[price=ge=number:30&sort(price)&limit(1)&select(id,name,price)])
1334
+ ```
1229
1335
 
1230
1336
  | Nested option | Behavior |
1231
1337
  | ------------------ | --------------------------------------------------------------- |
1232
- | Conditions | Qualify children; combine with `&` (AND), `\|` (OR), and groups |
1338
+ | Conditions | Qualify children of the selected relationship |
1233
1339
  | `sort(price)` | Order eligible children before pagination |
1234
1340
  | `limit(end)` | Return at most `end` children per parent |
1235
1341
  | `limit(start,end)` | Skip `start` eligible children and return at most `end - start` |
1236
1342
  | `select(id,price)` | Project fields from returned children |
1237
- - Invalid nested bounds (negative or fractional values) return HTTP 400. `limit(0)` returns no children without reading them.
1343
+ - `limit(0)` returns no children without reading them. Invalid nested bounds (negative or fractional values) return HTTP 400.
1344
+ - Nested filters require a table-backed relationship. Filters on plain JSON projections or full-text predicates inside a relationship selection return HTTP 400.
1238
1345
 
1239
1346
  #### Examples
1240
1347
 
1241
- ##### Range filter with sort and limit
1348
+ ##### Range filter with type conversion
1242
1349
 
1243
1350
  ```
1244
- GET /Product/?category=software&price=gt=100&price=lt=200&sort(+price)&limit(0,25)
1351
+ GET /Product/?price=gt=100&price=lt=200
1352
+ GET /Product/?price=between=(100,200)
1245
1353
  ```
1246
1354
 
1247
- ##### Relationship join with nested selection
1355
+ ##### Null query
1248
1356
 
1249
1357
  ```
1250
- GET /Product/?brand.name=Microsoft&select(name,brand{name,id})
1358
+ GET /Product/?discount=null
1359
+ GET /Product/?discount=ne=null
1360
+ ```
1361
+
1362
+ ##### Select with sort and limit
1363
+
1364
+ ```
1365
+ GET /Product/?rating=gt=3&inStock=true&select(rating,name)&limit(20)
1366
+ GET /Product/?sort(+rating,-price)
1251
1367
  ```
1252
1368
 
1253
1369
  ##### Paginated request with exact count
@@ -1257,41 +1373,34 @@ GET /Product/?category=software&limit(0,25)
1257
1373
  Prefer: count=exact
1258
1374
  ```
1259
1375
 
1260
- Response:
1376
+ Response headers:
1261
1377
 
1262
1378
  ```
1263
- HTTP/1.1 200 OK
1264
1379
  Content-Range: items 0-24/1234
1265
1380
  Range-Unit: items
1266
1381
  Preference-Applied: count=exact
1267
1382
  ```
1268
1383
 
1269
- ##### Nested relationship filter with OR, sort, and pagination
1384
+ ##### Relationship query with nested selection and pagination
1270
1385
 
1271
1386
  ```
1272
- GET /Brand/?sort(name,id)&limit(0,20)&select(id,name,products[[price=lt=number:30|price=ge=number:60]&sort(-price,id)&limit(1,2)&select(id,name,price)])
1387
+ GET /Brand/?products.price=ge=number:30&select(id,name,products[sort(price)&limit(1)&select(id,name,price)])
1273
1388
  ```
1274
1389
 
1275
- ##### Outer filter inheriting into nested selection
1390
+ ##### OR grouping with nested relationship filter and pagination
1276
1391
 
1277
1392
  ```
1278
- GET /Brand/?products.price=ge=number:30&select(id,name,products[sort(price)&limit(1)&select(id,name,price)])
1393
+ GET /Brand/?sort(name,id)&limit(0,20)&select(id,name,products[[price=lt=number:30|price=ge=number:60]&sort(-price,id)&limit(1,2)&select(id,name,price)])
1279
1394
  ```
1280
1395
 
1281
- ##### Many-to-many relationship query
1282
-
1283
- Schema:
1396
+ ##### Conditions through several relationship hops
1284
1397
 
1285
- ```graphql
1286
- type Product @table @export {
1287
- id: Long @primaryKey
1288
- name: String
1289
- resellerIds: [Long] @indexed
1290
- resellers: [Reseller] @relationship(from: "resellerIds")
1291
- }
1398
+ ```
1399
+ GET /Company/?departments[employees[tag=oncall&level=ge=3]]
1400
+ GET /Company/?departments.employees[tag=oncall&level=ge=3]
1292
1401
  ```
1293
1402
 
1294
- Query:
1403
+ ##### Many-to-many relationship query
1295
1404
 
1296
1405
  ```
1297
1406
  GET /Product/?resellers.name=Cool Shop&select(id,name,resellers{name,id})
@@ -1299,16 +1408,23 @@ GET /Product/?resellers.name=Cool Shop&select(id,name,resellers{name,id})
1299
1408
 
1300
1409
  #### Notes
1301
1410
 
1302
- - Queried attributes must be indexed for filtering to execute; only one attribute in a multi-attribute query needs to be indexed.
1303
- - URL-encode colons in date values as `%3A` (e.g., `listDate=gt=2017-03-08T09%3A30%3A00.000Z`).
1304
- - Square brackets are preferred over parentheses when constructing queries programmatically because standard URI encoding safely encodes `[` and `]`.
1305
- - Brackets inside `select()` after a relationship name (e.g., `select(id,products[limit(1)])`) form a nested relationship query, not a grouping expression.
1306
- - For full-text fields marked `@fullText`, use operators such as `=matches=`, `=matches_phrase=`, `=matches_prefix=`, `=matches_fuzzy=`, and their `=not_*=` negated forms. Full-text queries cannot provide an exact total count.
1307
- - `limit(end)` and `limit(start,end)` inside a nested relationship selection apply per parent, not to the total result set.
1308
- - Always add a unique field as the final sort key (e.g., `sort(name,id)`) to ensure deterministic ordering across pages.
1309
- - Nested filters require a table-backed relationship; filters on plain JSON projections or full-text predicates inside a relationship selection return HTTP 400.
1310
- - For streaming JSON-array responses, check for a terminal object shaped like `{ "error": "ErrorName: message" }` and treat the response as failed.
1311
- - Access a specific property of a record using dot syntax on the record ID: `GET /MyTable/123.propertyName`. This only works for declared schema properties. The suffixes `.json`, `.cbor`, `.msgpack`, and `.csv` are reserved as content-type selectors and take precedence over a property of the same name.
1411
+ - Only indexed attributes can lead a filter query; at least one attribute in a multi-attribute query must be indexed.
1412
+ - Null indexing requires indexes created in v4.3.0 or later. Existing indexes must be removed and re-added to support `discount=null` queries.
1413
+ - For strict operators (`=`, `===`, `!==`), no automatic type conversion is applied — the value is decoded as a URL-encoded string.
1414
+ - No range comparator (`=lt=`, `=le=`, `=gt=`, `=ge=`, `=between=`, etc.) matches a null or omitted value.
1415
+ - `=ne=` with a non-null value matches null and omitted values.
1416
+ - A `null` lower bound means no lower bound: `ge=null` and `gt=null` match every non-null value, and `between=(null,10)` returns the same records as `le=10`. A `null` upper bound: `lt=null` and `le=null` match nothing.
1417
+ - Sorting puts null and omitted values first; `-` puts them last, with no guaranteed order between the two.
1418
+ - A `HEAD` request with a `count` preference returns count headers with no body.
1419
+ - The response status for paginated results is always `200`; Harper does not use `206 Partial Content`. `Content-Range` is informational.
1420
+ - Without `exactCount: true`, a `count=exact` request is served as an estimate (`Preference-Applied: count=estimated`). When the total cannot be produced, it is reported as `.../*` (e.g., `Content-Range: items 0-24/*`). `Preference-Applied` still echoes the requested mode, so an unavailable total (`.../*`) is distinct from a request that asked for no count.
1421
+ - When CORS is enabled, `Content-Range`, `Range-Unit`, and `Preference-Applied` are added to `Access-Control-Expose-Headers`.
1422
+ - Full-text queries cannot provide an exact total count; with exact counting enabled, `Prefer: count=exact` returns an unavailable total (`.../*`).
1423
+ - For JSON-array responses from full-text queries, check for a terminal error object shaped like `{ "error": "ErrorName: message" }` and treat the response as failed. A stream or JSON-parse error is also a failed page, not a record.
1424
+ - Access a specific property of a record by appending it with dot syntax to the record id: `GET /MyTable/123.propertyName`. This only works for declared properties. The suffixes `.json`, `.cbor`, `.msgpack`, and `.csv` are reserved as content-type selectors and take precedence over a property of the same name.
1425
+ - Querying through a relationship requires read permission on the related table and each related attribute named in a condition; without it, Harper returns HTTP 403.
1426
+ - Nested filters require a table-backed relationship. Filters on plain JSON projections or full-text predicates inside a relationship selection return HTTP 400.
1427
+ - `limit(0)` returns no children without reading them. Invalid nested bounds (negative or fractional values) return HTTP 400.
1312
1428
 
1313
1429
  ### 2.3 Real-Time Apps with WebSockets and Pub/Sub
1314
1430
 
@@ -1412,250 +1528,248 @@ Instructions for the agent to handle user authentication, sessions, and JWT toke
1412
1528
 
1413
1529
  #### When to Use
1414
1530
 
1415
- Apply this rule when implementing login/logout flows, reading the current authenticated user, issuing or refreshing JWT tokens, or minting scoped tokens from a custom Resource. Use it whenever a Resource must gate behavior on identity or credentials. See [custom-resources.md](custom-resources.md) for the broader Resource authoring model.
1531
+ Apply this rule when implementing sign-in/sign-out flows, reading the current authenticated user, issuing or refreshing JWT tokens, or minting scoped tokens from a custom Resource. Use it whenever a Resource must inspect, establish, or delegate authentication state. See [custom-resources.md](custom-resources.md) for the broader Resource authoring context.
1416
1532
 
1417
1533
  #### How It Works
1418
1534
 
1419
- ##### 1. Reading the Current Authenticated User
1420
-
1421
- **In an instance method**, call `getCurrentUser()` to get the user associated with the current request, or `undefined` if unauthenticated. The returned object exposes `username`, `role`, and `role.permission` flags.
1422
-
1423
- **In a static verb**, read the user from the `context` argument instead — `context.user`:
1424
-
1425
- ```javascript
1426
- static async get(_target, context) {
1427
- const user = context?.user;
1428
- if (!user) return new Response(null, { status: 401 });
1429
- return { username: user.username, role: user.role };
1430
- }
1431
- ```
1432
-
1433
- ##### 2. Reading the Full Context
1434
-
1435
- **In an instance method**, call `getContext()` to retrieve the current context, which includes:
1436
-
1437
- - `user` — User object with username, role, and authorization information
1438
- - `transaction` — The current transaction
1439
-
1440
- **In a static verb**, the context is the trailing argument:
1441
-
1442
- - `(target, context)` for `get`/`delete`
1443
- - `(target, data, context)` for `put`/`patch`/`post`
1444
-
1445
- When triggered by HTTP, the context also exposes `url`, `method`, `headers`, `responseHeaders`, `pathname`, `host`, `ip`, `body`, `data`, `lastModified`, and `requestContext`.
1535
+ ##### Reading the Current User
1446
1536
 
1447
- ##### 3. Handling Sessions and Login
1448
-
1449
- Enable sessions in `harper-config.yaml`:
1450
-
1451
- ```yaml
1452
- authentication:
1453
- enableSessions: true
1454
- ```
1455
-
1456
- Use `context.login` in a static `post` verb to verify credentials and establish a session cookie:
1457
-
1458
- ```javascript
1459
- export class SignIn extends Resource {
1460
- static async post(_target, data, context) {
1461
- const { username, password } = (await data) ?? {};
1462
- try {
1463
- await context.login(username, password);
1464
- } catch {
1465
- return new Response('Invalid credentials', { status: 403 });
1466
- }
1467
- return new Response('Logged in', { status: 200 });
1468
- }
1469
- }
1537
+ 1. **Use `getCurrentUser()` in instance methods**: Call `getCurrentUser()` on a Resource instance to get the current authenticated user, or `undefined` if no user is authenticated. The returned object exposes `username`, `role`, and `role.permission` flags.
1470
1538
 
1471
- export class SignOut extends Resource {
1472
- static async post(_target, _data, context) {
1473
- if (!context?.session?.user) return new Response(null, { status: 401 });
1474
- await context.session.update({ user: null });
1475
- return new Response('Logged out', { status: 200 });
1476
- }
1477
- }
1478
- ```
1539
+ 2. **Use `context.user` in static verbs**: Static verbs receive the context as a trailing argument. Read the current authenticated user from `context?.user`:
1479
1540
 
1480
- - `context.login(username, password)` verifies credentials and sets the session cookie on success.
1481
- - To end a session, call `context.session.update({ user: null })` — `update` is the session's only mutator.
1482
- - `context.session` is an empty object (not `undefined`) when sessions are enabled and no session cookie is present; test `context?.session?.user` to detect an established session.
1483
- - Cookie-based sessions are for browser clients. For non-browser clients, use JWT issuance.
1541
+ ```javascript
1542
+ static async get(_target, context) {
1543
+ const user = context?.user;
1544
+ if (!user) return new Response(null, { status: 401 });
1545
+ return { username: user.username, role: user.role };
1546
+ }
1547
+ ```
1484
1548
 
1485
- ##### 4. Creating Authentication Tokens
1549
+ 3. **Use `getContext()` for dual-path static methods**: `getContext()` is a Resource instance method. For static methods that must work both via HTTP and direct server-side calls, import `getContext()` from the `harper` module. Static verbs receive context as the trailing argument — `(target, context)` for `get`/`delete`, `(target, data, context)` for `put`/`patch`/`post`.
1486
1550
 
1487
- Call `create_authentication_tokens` with credentials in the body — no `Authorization` header is required when `username` and `password` are supplied:
1551
+ 4. **Access HTTP-specific context properties**: When triggered by HTTP, the context is the `Request` object with additional properties. Use these to read request metadata or set response headers:
1488
1552
 
1489
- ```json
1490
- {
1491
- "operation": "create_authentication_tokens",
1492
- "username": "username",
1493
- "password": "password"
1494
- }
1495
- ```
1553
+ | Property | Description |
1554
+ | ----------------- | -------------------------------------------------------------------------------- |
1555
+ | `url` | Full local path including query string |
1556
+ | `method` | HTTP method |
1557
+ | `headers` | Request headers (`context.headers.get(name)`) |
1558
+ | `responseHeaders` | Response headers (`context.responseHeaders.set(name, value)`) |
1559
+ | `pathname` | Path without query string |
1560
+ | `host` | Host from the `Host` header |
1561
+ | `ip` | Client IP address |
1562
+ | `body` | Raw Node.js `Readable` stream (if a request body exists) |
1563
+ | `data` | Promise resolving to the deserialized request body |
1564
+ | `lastModified` | Controls `ETag`/`Last-Modified` response header |
1565
+ | `requestContext` | (Source resources only) Context of the upstream resource making the data request |
1566
+ | `transaction` | Current transaction |
1496
1567
 
1497
- Response:
1568
+ When triggered by HTTP, the context also exposes `url`, `method`, `headers`, `responseHeaders`, `pathname`, `host`, `ip`, `body`, `data`, `lastModified`, and `requestContext`.
1498
1569
 
1499
- ```json
1500
- {
1501
- "operation_token": "<jwt-operation-token>",
1502
- "refresh_token": "<jwt-refresh-token>"
1503
- }
1504
- ```
1570
+ ##### Session-Based Login and Logout
1505
1571
 
1506
- Pass the `operation_token` as a `Bearer` token on subsequent requests:
1572
+ 5. **Enable sessions in config**: Set `authentication.enableSessions: true` in `harperdb-config.yaml` before using `context.login` or `context.session`.
1507
1573
 
1508
- ```bash
1509
- curl --location --request POST 'http://localhost:9925' \
1510
- --header 'Content-Type: application/json' \
1511
- --header 'Authorization: Bearer <operation_token>' \
1512
- --data-raw '{
1513
- "operation": "search_by_hash",
1514
- "schema": "dev",
1515
- "table": "dog",
1516
- "hash_values": [1],
1517
- "get_attributes": ["*"]
1518
- }'
1519
- ```
1574
+ 6. **Implement sign-in with `context.login`**: Call `context.login(username, password)` in a static `post` verb. It verifies credentials and establishes the session cookie on success:
1520
1575
 
1521
- ##### 5. Refreshing the Operation Token
1576
+ ```javascript
1577
+ export class SignIn extends Resource {
1578
+ static async post(_target, data, context) {
1579
+ const { username, password } = (await data) ?? {};
1580
+ try {
1581
+ await context.login(username, password);
1582
+ } catch {
1583
+ return new Response('Invalid credentials', { status: 403 });
1584
+ }
1585
+ return new Response('Logged in', { status: 200 });
1586
+ }
1587
+ }
1588
+ ```
1522
1589
 
1523
- When the `operation_token` expires, pass the `refresh_token` as `Bearer <refresh_token>` and call `refresh_operation_token`:
1590
+ 7. **Implement sign-out by clearing the session user**: Use `context.session.update({ user: null })` — `update` is the session's only mutator. Test `context?.session?.user` to detect an established session, because `context.session` is an empty object (not `undefined`) when sessions are enabled and no session cookie is present:
1524
1591
 
1525
- ```bash
1526
- curl --location --request POST 'http://localhost:9925' \
1527
- --header 'Content-Type: application/json' \
1528
- --header 'Authorization: Bearer <refresh_token>' \
1529
- --data-raw '{
1530
- "operation": "refresh_operation_token"
1531
- }'
1532
- ```
1592
+ ```javascript
1593
+ export class SignOut extends Resource {
1594
+ static async post(_target, _data, context) {
1595
+ if (!context?.session?.user) return new Response(null, { status: 401 });
1596
+ await context.session.update({ user: null });
1597
+ return new Response('Logged out', { status: 200 });
1598
+ }
1599
+ }
1600
+ ```
1533
1601
 
1534
- Response:
1602
+ Cookie-based sessions are intended for browser clients. For non-browser clients, use JWT issuance.
1535
1603
 
1536
- ```json
1537
- {
1538
- "operation_token": "<new-jwt-operation-token>"
1539
- }
1540
- ```
1604
+ ##### JWT Token Issuance
1541
1605
 
1542
- When both tokens have expired, call `create_authentication_tokens` again with username and password.
1606
+ 8. **Call `create_authentication_tokens` with credentials**: Send `username` and `password` in the request body. No `Authorization` header is required when credentials are in the body:
1543
1607
 
1544
- ##### 6. Minting Scoped Tokens
1608
+ ```json
1609
+ {
1610
+ "operation": "create_authentication_tokens",
1611
+ "username": "username",
1612
+ "password": "password"
1613
+ }
1614
+ ```
1545
1615
 
1546
- A `super_user` can mint a scoped token with embedded permissions using `create_authentication_tokens` with an inline `role`. The minter must be authenticated; no `password` may be included in the body. Use `add_role`-style `permission` structure:
1616
+ Response:
1547
1617
 
1548
- ```json
1549
- {
1550
- "operation": "create_authentication_tokens",
1551
- "username": "reporting-service",
1552
- "role": {
1553
- "permission": {
1554
- "operations": ["read_only"],
1555
- "dev": {
1556
- "tables": {
1557
- "dog": {
1558
- "read": true,
1559
- "insert": false,
1560
- "update": false,
1561
- "delete": false,
1562
- "attribute_permissions": []
1563
- }
1564
- }
1565
- }
1566
- }
1567
- },
1568
- "expires_in": "7d"
1569
- }
1570
- ```
1618
+ ```json
1619
+ {
1620
+ "operation_token": "<jwt-operation-token>",
1621
+ "refresh_token": "<jwt-refresh-token>"
1622
+ }
1623
+ ```
1571
1624
 
1572
- Authenticate the mint request with Basic Authentication or an existing `super_user` `operation_token`:
1625
+ 9. **Pass the `operation_token` as a Bearer token** on subsequent requests:
1573
1626
 
1574
- ```bash
1575
- curl --location --request POST 'http://localhost:9925' \
1576
- --header 'Content-Type: application/json' \
1577
- --header 'Authorization: Basic <base64 of super_user:password>' \
1578
- --data-raw '{
1579
- "operation": "create_authentication_tokens",
1580
- "username": "reporting-service",
1581
- "role": { "permission": { "operations": ["read_only"] } },
1582
- "expires_in": "7d"
1583
- }'
1584
- ```
1627
+ ```bash
1628
+ curl --location --request POST 'http://localhost:9925' \
1629
+ --header 'Content-Type: application/json' \
1630
+ --header 'Authorization: Bearer <operation_token>' \
1631
+ --data-raw '{
1632
+ "operation": "search_by_hash",
1633
+ "schema": "dev",
1634
+ "table": "dog",
1635
+ "hash_values": [1],
1636
+ "get_attributes": ["*"]
1637
+ }'
1638
+ ```
1639
+
1640
+ 10. **Refresh an expired `operation_token`** using `refresh_operation_token`. Pass the `refresh_token` as `Bearer <refresh_token>`:
1641
+
1642
+ ```bash
1643
+ curl --location --request POST 'http://localhost:9925' \
1644
+ --header 'Content-Type: application/json' \
1645
+ --header 'Authorization: Bearer <refresh_token>' \
1646
+ --data-raw '{
1647
+ "operation": "refresh_operation_token"
1648
+ }'
1649
+ ```
1585
1650
 
1586
- Key constraints for scoped tokens:
1651
+ Response:
1587
1652
 
1588
- - `username` is attribution only and must not name an existing user.
1589
- - No refresh token is issued; no user record is created.
1590
- - Scoped tokens cannot be revoked before expiry — choose `expires_in` carefully and prefer short lifetimes.
1591
- - `super_user` and `cluster_user` are always forced to `false` in the embedded role.
1592
- - In **mixed-version** clusters, only nodes with scoped-token support accept these tokens; older nodes reject them with 401.
1653
+ ```json
1654
+ {
1655
+ "operation_token": "<new-jwt-operation-token>"
1656
+ }
1657
+ ```
1593
1658
 
1594
- ##### 7. Issuing Tokens from a Custom Resource
1659
+ When both tokens have expired, call `create_authentication_tokens` again with username and password.
1660
+
1661
+ ##### Scoped Tokens (Inline Role)
1662
+
1663
+ 11. **Mint a scoped token as a `super_user`**: Pass `role` as an inline role-shaped object using the same `permission` structure as `add_role`. The minter must be authenticated as a `super_user`; no `password` may be included in the body. Use `expires_in` to set the token lifetime:
1664
+
1665
+ ```json
1666
+ {
1667
+ "operation": "create_authentication_tokens",
1668
+ "username": "reporting-service",
1669
+ "role": {
1670
+ "permission": {
1671
+ "operations": ["read_only"],
1672
+ "dev": {
1673
+ "tables": {
1674
+ "dog": {
1675
+ "read": true,
1676
+ "insert": false,
1677
+ "update": false,
1678
+ "delete": false,
1679
+ "attribute_permissions": []
1680
+ }
1681
+ }
1682
+ }
1683
+ }
1684
+ },
1685
+ "expires_in": "7d"
1686
+ }
1687
+ ```
1595
1688
 
1596
- Use `server.operation()` to mint tokens programmatically inside a Resource. Pass `authorize: true` as the **third argument** when the operation should run as the current authenticated user:
1689
+ Authenticate the mint request with Basic Authentication or an existing `super_user` `operation_token`:
1690
+
1691
+ ```bash
1692
+ curl --location --request POST 'http://localhost:9925' \
1693
+ --header 'Content-Type: application/json' \
1694
+ --header 'Authorization: Basic <base64 of super_user:password>' \
1695
+ --data-raw '{
1696
+ "operation": "create_authentication_tokens",
1697
+ "username": "reporting-service",
1698
+ "role": { "permission": { "operations": ["read_only"] } },
1699
+ "expires_in": "7d"
1700
+ }'
1701
+ ```
1597
1702
 
1598
- ```javascript
1599
- import { Resource, server } from 'harper';
1600
-
1601
- export class IssueTokens extends Resource {
1602
- static async get(_target, context) {
1603
- // Issue tokens for the current authenticated user
1604
- const { operation_token, refresh_token } = await server.operation(
1605
- { operation: 'create_authentication_tokens' },
1606
- context,
1607
- true, // third argument: authorize as current user
1608
- );
1609
- return { operation_token, refresh_token };
1610
- }
1703
+ Key constraints:
1704
+ - `username` is attribution only and must not name an existing user.
1705
+ - No refresh token is issued; no user record is created.
1706
+ - Scoped tokens cannot be revoked before expiry — prefer short lifetimes.
1707
+ - `super_user` and `cluster_user` are always forced to `false` in the embedded role.
1708
+ - In **mixed-version** clusters, only nodes with scoped-token support accept these tokens; older nodes reject them with a 401.
1611
1709
 
1612
- static async post(_target, data) {
1613
- // Issue tokens from credentials supplied in the body
1614
- const { username, password } = await data;
1615
- if (!username || !password) {
1616
- return new Response('username and password required', { status: 400 });
1617
- }
1618
- const { operation_token, refresh_token } = await server.operation({
1619
- operation: 'create_authentication_tokens',
1620
- username,
1621
- password,
1622
- });
1623
- return { operation_token, refresh_token };
1624
- }
1625
- }
1710
+ ##### Issuing Tokens from a Custom Resource
1626
1711
 
1627
- export class RefreshJWT extends Resource {
1628
- static async post(_target, data) {
1629
- const { refresh_token } = await data;
1630
- if (!refresh_token) {
1631
- return new Response('refresh_token required', { status: 400 });
1632
- }
1633
- const { operation_token } = await server.operation({
1634
- operation: 'refresh_operation_token',
1635
- refresh_token,
1636
- });
1637
- return { operation_token };
1638
- }
1639
- }
1640
- ```
1712
+ 12. **Use `server.operation()` to mint tokens programmatically**: Import `server` from `harper` and call `server.operation(operation, context, authorize)`. Pass `authorize: true` as the **third argument** when the operation should run as the current authenticated user; omit it or pass `false` when the operation supplies its own credentials:
1641
1713
 
1642
- Omit the third argument (or pass `false`) when the operation supplies its own credentials.
1714
+ ```javascript
1715
+ import { Resource, server } from 'harper';
1716
+
1717
+ export class IssueTokens extends Resource {
1718
+ static async get(_target, context) {
1719
+ // Caller is already authenticated — issue tokens for the current authenticated user.
1720
+ const { operation_token, refresh_token } = await server.operation(
1721
+ { operation: 'create_authentication_tokens' },
1722
+ context,
1723
+ true,
1724
+ );
1725
+ return { operation_token, refresh_token };
1726
+ }
1727
+
1728
+ static async post(_target, data) {
1729
+ // Caller provides credentials in the body.
1730
+ const { username, password } = await data;
1731
+ if (!username || !password) {
1732
+ return new Response('username and password required', { status: 400 });
1733
+ }
1734
+ const { operation_token, refresh_token } = await server.operation({
1735
+ operation: 'create_authentication_tokens',
1736
+ username,
1737
+ password,
1738
+ });
1739
+ return { operation_token, refresh_token };
1740
+ }
1741
+ }
1742
+
1743
+ export class RefreshJWT extends Resource {
1744
+ static async post(_target, data) {
1745
+ const { refresh_token } = await data;
1746
+ if (!refresh_token) {
1747
+ return new Response('refresh_token required', { status: 400 });
1748
+ }
1749
+ const { operation_token } = await server.operation({
1750
+ operation: 'refresh_operation_token',
1751
+ refresh_token,
1752
+ });
1753
+ return { operation_token };
1754
+ }
1755
+ }
1756
+ ```
1643
1757
 
1644
- ##### 8. Configuring Token Expiry
1758
+ ##### Token Expiry Configuration
1645
1759
 
1646
- Set timeouts in `harper-config.yaml` under the `authentication` section:
1760
+ 13. **Configure token timeouts** in `harper-config.yaml` under the `authentication` section:
1647
1761
 
1648
- ```yaml
1649
- authentication:
1650
- operationTokenTimeout: 1d # Default: 1 day
1651
- refreshTokenTimeout: 30d # Default: 30 days
1652
- ```
1762
+ ```yaml
1763
+ authentication:
1764
+ operationTokenTimeout: 1d # Default: 1 day
1765
+ refreshTokenTimeout: 30d # Default: 30 days
1766
+ ```
1653
1767
 
1654
- Valid duration strings follow the `jsonwebtoken` package format (e.g., `1d`, `12h`, `60m`). The `expires_in` field on scoped token minting accepts the same format.
1768
+ Valid duration strings follow the `jsonwebtoken` package format (e.g., `1d`, `12h`, `60m`).
1655
1769
 
1656
1770
  #### Examples
1657
1771
 
1658
- ##### Full Sign-In / Sign-Out Resource
1772
+ ##### Full sign-in/sign-out Resource pair
1659
1773
 
1660
1774
  ```javascript
1661
1775
  export class SignIn extends Resource {
@@ -1679,17 +1793,7 @@ export class SignOut extends Resource {
1679
1793
  }
1680
1794
  ```
1681
1795
 
1682
- ##### Reading the Current Authenticated User in a Static Verb
1683
-
1684
- ```javascript
1685
- static async get(_target, context) {
1686
- const user = context?.user;
1687
- if (!user) return new Response(null, { status: 401 });
1688
- return { username: user.username, role: user.role };
1689
- }
1690
- ```
1691
-
1692
- ##### Minting Tokens via cURL
1796
+ ##### Minting tokens via the Operations API
1693
1797
 
1694
1798
  ```bash
1695
1799
  curl --location --request POST 'http://localhost:9925' \
@@ -1703,12 +1807,12 @@ curl --location --request POST 'http://localhost:9925' \
1703
1807
 
1704
1808
  #### Notes
1705
1809
 
1706
- - JWT authentication is **preferred over Basic Auth** when you want to avoid sending credentials on every request, your client can store tokens, or you have multiple sequential requests. For simple or **server-to-server** scenarios, use **Basic Authentication**.
1810
+ - **JWT is preferred over Basic Auth** for clients that can store tokens, want to avoid sending credentials on every request, or make multiple sequential requests. For simple or **server-to-server** scenarios, use Basic Authentication.
1707
1811
  - Always use **HTTPS** in production to protect tokens in transit.
1708
- - Treat tokens like passwords. If a token is compromised, it remains valid until expiry — use shorter `operationTokenTimeout` values in high-security environments.
1709
- - `context.login` and `context.session` require `enableSessions: true` in `harperdb-config.yaml`; they are for browser clients only.
1710
- - The `server.operation()` third argument (`authorize: true`) attributes the operation to — and permission-checks against — the current authenticated user. Omit it when supplying credentials directly in the operation body.
1711
- - In **mixed-version** clusters, scoped tokens are only accepted by nodes that support them; older nodes return 401.
1812
+ - Treat tokens like passwords. Because scoped tokens cannot be revoked before expiry, set short `expires_in` values in high-security environments and consider lowering `operationTokenTimeout`.
1813
+ - A caller authenticated by a Bearer validator cannot mint tokens via the passwordless form — it must supply `username` and `password`.
1814
+ - `enableSessions` is required in `harperdb-config.yaml` for `context.login` and `context.session` to be available.
1815
+ - The **third argument** (`authorize: true`) to `server.operation()` attributes the operation to — and permission-checks against — the calling user.
1712
1816
 
1713
1817
  ## 3. Logic & Extension
1714
1818
 
@@ -1720,23 +1824,23 @@ Instructions for the agent to follow when defining custom REST endpoints with Ja
1720
1824
 
1721
1825
  #### When to Use
1722
1826
 
1723
- Apply this rule when you need to create custom HTTP endpoints, wrap external APIs, or register routes programmatically in a Harper application. Use it any time business logic must live outside a table-backed schema, or when you need fine-grained control over URL shape and path parameters.
1827
+ Apply this rule when you need to create custom HTTP endpoints, wrap external APIs, or register routes programmatically in a Harper application. Use it any time business logic must live outside a standard table-backed resource.
1724
1828
 
1725
1829
  #### How It Works
1726
1830
 
1727
- 1. **Import `Resource` from the `harper` package**: Always import explicitly rather than relying on globals.
1831
+ 1. **Import `Resource` from `harper`**: Pull in the base class before extending it.
1728
1832
 
1729
1833
  ```javascript
1730
1834
  import { tables, Resource } from 'harper';
1731
1835
  ```
1732
1836
 
1733
- CommonJS alternative:
1837
+ CommonJS is also supported:
1734
1838
 
1735
1839
  ```javascript
1736
1840
  const { tables, Resource } = require('harper');
1737
1841
  ```
1738
1842
 
1739
- 2. **Define a class that `extends Resource`**: Use `export class` so Harper exposes it as an endpoint. Implement HTTP methods as `static` methods on the class.
1843
+ 2. **Define a class that `extends Resource`**: Declare HTTP handler methods as `static` methods matching the HTTP verb (`get`, `put`, `post`, etc.). Each method receives a `target` object representing the request.
1740
1844
 
1741
1845
  ```javascript
1742
1846
  export class CustomEndpoint extends Resource {
@@ -1748,7 +1852,7 @@ Apply this rule when you need to create custom HTTP endpoints, wrap external API
1748
1852
  }
1749
1853
  ```
1750
1854
 
1751
- 3. **Use `static` async methods to call external services**: Each HTTP verb maps to a `static` method of the same lowercase name.
1855
+ 3. **Wrap external APIs** by making the `static` methods `async` and using `fetch` or any other async call inside them.
1752
1856
 
1753
1857
  ```javascript
1754
1858
  export class MyExternalData extends Resource {
@@ -1764,13 +1868,16 @@ Apply this rule when you need to create custom HTTP endpoints, wrap external API
1764
1868
  });
1765
1869
  }
1766
1870
  }
1871
+
1872
+ // Use as a cache source for a local table
1873
+ tables.MyCache.sourcedFrom(MyExternalData);
1767
1874
  ```
1768
1875
 
1769
- 4. **Control the URL by choosing the export form**: The shape of the export determines the resulting URL path.
1876
+ 4. **Control the URL by choosing an export form**: The shape of the export determines the resulting endpoint URL.
1770
1877
 
1771
1878
  | Export form | URL | Notes |
1772
1879
  | ------------------------------------------- | --------------- | --------------------------------------------------------------- |
1773
- | `export class Foo extends Resource {}` | `/Foo/` | Class name becomes the path segment; case-sensitive. |
1880
+ | `export class Foo extends Resource {}` | `/Foo/` | Class name becomes the path segment. Case-sensitive. |
1774
1881
  | `export const Bar = { Foo };` | `/Bar/Foo/` | Nest under an object to add a path prefix. |
1775
1882
  | `export const bar = { 'foo-baz': Foo };` | `/bar/foo-baz/` | Use object keys for lowercase, hyphens, or non-identifier URLs. |
1776
1883
  | `export { Foo as '/widget/:id' }` | `/widget/:id` | Rename the export to set the path directly. |
@@ -1779,7 +1886,7 @@ Apply this rule when you need to create custom HTTP endpoints, wrap external API
1779
1886
 
1780
1887
  URL path matching is case-sensitive — `/Foo/` and `/foo/` are different endpoints.
1781
1888
 
1782
- 5. **Declare dynamic path segments with `static path`**: Use `:name` for a single segment and `*name` as a catch-all. Matched values are bound onto `target.<name>`.
1889
+ 5. **Declare dynamic path segments** using `static path` on the class. Use `:name` for a single segment and `*name` as a catch-all for the rest of the path. Matched values are bound onto `target.<name>`.
1783
1890
 
1784
1891
  ```javascript
1785
1892
  export class Widget extends Resource {
@@ -1799,26 +1906,36 @@ Apply this rule when you need to create custom HTTP endpoints, wrap external API
1799
1906
  }
1800
1907
  ```
1801
1908
 
1802
- - A leading `/` in `static path` makes the path root-relative (top-level), independent of the file's location.
1803
- - A leading `./` or bare name resolves relative to the component directory.
1804
- - A bare `*` (no name) binds under `target.wildcard`. A wildcard must be the final segment.
1909
+ - A bare `*` (no name) binds under `target.wildcard`.
1910
+ - A wildcard must be the final segment of the path.
1911
+ - A leading `/` makes the path root-relative (top-level), independent of the file's location.
1912
+ - A leading `./` or a bare name resolves relative to the component directory.
1805
1913
  - `static path` takes precedence over the export name.
1806
- - Exact and static paths always win over parameterized ones. Among parameterized routes, more specific paths win: a literal segment beats `:param`, which beats `*`, compared left to right.
1807
1914
 
1808
- 6. **Register programmatically when the path is dynamic**: Use `server.resources.set(` with a path string and the resource class.
1915
+ 6. **Understand resolution order**: Exact and static paths always win over parameterized ones. Among parameterized routes, more specific paths win: a literal segment beats a `:param`, which beats a `*` wildcard, compared left to right.
1809
1916
 
1917
+ 7. **Register programmatically** using `server.resources.set(` when the path must be determined at runtime rather than statically.
1810
1918
  ```javascript
1811
1919
  server.resources.set('my-path', Foo);
1812
1920
  ```
1813
1921
 
1814
- 7. **Optionally use a resource as a cache source for a local table**: Call `sourcedFrom` on the target table.
1815
- ```javascript
1816
- tables.MyCache.sourcedFrom(MyExternalData);
1817
- ```
1818
-
1819
1922
  #### Examples
1820
1923
 
1821
- **Custom endpoint with external API wrapping:**
1924
+ **Custom endpoint with a fixed path:**
1925
+
1926
+ ```javascript
1927
+ import { Resource } from 'harper';
1928
+
1929
+ export class CustomEndpoint extends Resource {
1930
+ static get(target) {
1931
+ return {
1932
+ data: doSomething(),
1933
+ };
1934
+ }
1935
+ }
1936
+ ```
1937
+
1938
+ **External API wrapper used as a cache source:**
1822
1939
 
1823
1940
  ```javascript
1824
1941
  import { tables, Resource } from 'harper';
@@ -1837,7 +1954,6 @@ export class MyExternalData extends Resource {
1837
1954
  }
1838
1955
  }
1839
1956
 
1840
- // Use as a cache source for a local table
1841
1957
  tables.MyCache.sourcedFrom(MyExternalData);
1842
1958
  ```
1843
1959
 
@@ -1847,42 +1963,25 @@ tables.MyCache.sourcedFrom(MyExternalData);
1847
1963
  import { Resource } from 'harper';
1848
1964
 
1849
1965
  export class Widget extends Resource {
1850
- // GET /widget/10/action/jump -> target.id === '10', target.action === 'jump'
1851
1966
  static path = '/widget/:id/action/:action';
1852
1967
  static get(target) {
1853
1968
  return { id: target.id, action: target.action };
1854
1969
  }
1855
1970
  }
1856
-
1857
- export class Files extends Resource {
1858
- // GET /files/a/b/c.txt -> target.rest === 'a/b/c.txt'
1859
- static path = '/files/*rest';
1860
- static get(target) {
1861
- return { path: target.rest };
1862
- }
1863
- }
1864
1971
  ```
1865
1972
 
1866
1973
  **Programmatic registration:**
1867
1974
 
1868
1975
  ```javascript
1869
- import { Resource } from 'harper';
1870
-
1871
- export class Foo extends Resource {
1872
- static get(target) {
1873
- return { ok: true };
1874
- }
1875
- }
1876
-
1877
1976
  server.resources.set('my-path', Foo);
1878
1977
  ```
1879
1978
 
1880
1979
  #### Notes
1881
1980
 
1882
- - Avoid conflicting exports between the schema and the JavaScript implementation when a resource `extends` an existing table.
1883
- - Parameterized routes appear in the generated OpenAPI document as templated paths (e.g. `/widget/{id}/action/{action}`) and in MCP `resources/templates/list` as `{param}` URI templates.
1981
+ - Do not create conflicting exports between a schema definition and a JavaScript implementation when a resource extends an existing table.
1982
+ - Parameterized routes surface in the generated OpenAPI document as templated paths (e.g., `/widget/{id}/action/{action}`) and in MCP `resources/templates/list` as `{param}` URI templates.
1884
1983
  - When bundling for SSR with a tool like Vite, keep `harper` external so it resolves to the runtime: `ssr: { external: ['harper'] }`.
1885
- - For components in their own directory, run `npm link harper` to ensure typings match the running installation.
1984
+ - Run `npm link harper` in a component's own directory to ensure typings match the running installation. All installed components have `harper` automatically linked.
1886
1985
 
1887
1986
  ### 3.2 Extending Tables
1888
1987
 
@@ -1984,25 +2083,29 @@ export class MyTable extends tables.MyTable {
1984
2083
 
1985
2084
  ### 3.3 Programmatic Table Requests
1986
2085
 
1987
- Instructions for the agent to interact with Harper tables programmatically using the `tables` object, the Query API, and transactions.
2086
+ Instructions for the agent to interact with Harper tables programmatically using the `tables` object, including querying, transactions, and module setup.
1988
2087
 
1989
2088
  **Instead of:** ORM and query-builder clients (Prisma, Drizzle, Knex). Use this rule rather than adding that service. See `harper-architecture`.
1990
2089
 
1991
2090
  #### When to Use
1992
2091
 
1993
- Apply this rule when writing server-side Harper code that reads, writes, searches, or transactionally modifies table data through the `tables` object. Use it when building background jobs, SSR renderers, or any code that calls `search(`, `addTo`, `select`, `sort`, or `transaction()` directly rather than through REST endpoints.
2092
+ Apply this rule when writing server-side Harper code that reads or writes table data directly — for example, in request handlers, background jobs, timers, or SSR entry points. Use it whenever you need to construct a query with `conditions`, `sort`, `select`, `limit`, or `offset`, or when you need to wrap writes in a `transaction()`.
1994
2093
 
1995
2094
  #### How It Works
1996
2095
 
1997
- 1. **Import `tables` (and other APIs) from `harper`**: Access every table defined in `schema.graphql` as a named property of `tables`. This is equivalent to `databases.data.<TableName>`.
2096
+ 1. **Import `tables` (and other APIs) from `harper`**: Access tables defined in `schema.graphql` via the `tables` object. Each table is a property matching its type name.
1998
2097
 
1999
2098
  ```javascript
2000
- import { tables, Resource } from 'harper';
2099
+ import { tables, transaction } from 'harper';
2001
2100
  const { Product } = tables;
2002
- // same as: databases.data.Product
2101
+ // equivalent to: databases.data.Product
2003
2102
  ```
2004
2103
 
2005
- CommonJS is also supported: `const { tables, Resource } = require('harper');`
2104
+ For CommonJS:
2105
+
2106
+ ```javascript
2107
+ const { tables, Resource } = require('harper');
2108
+ ```
2006
2109
 
2007
2110
  For components in their own directory, link the package first:
2008
2111
 
@@ -2010,16 +2113,9 @@ Apply this rule when writing server-side Harper code that reads, writes, searche
2010
2113
  npm link harper
2011
2114
  ```
2012
2115
 
2013
- When bundling for SSR with Vite, keep `harper` external so it resolves to the runtime:
2116
+ When bundling for SSR with Vite, keep `harper` external so it resolves to the runtime — add `ssr: { external: ['harper'] }` in `vite.config`.
2014
2117
 
2015
- ```javascript
2016
- // vite.config
2017
- ssr: {
2018
- external: ['harper'];
2019
- }
2020
- ```
2021
-
2022
- 2. **Define your schema with `@table` (and `@indexed` where needed)**: Tables must be declared in `schema.graphql`. Mark attributes that will be used as sort keys or standalone condition attributes with `@indexed`.
2118
+ 2. **Define your schema with `@table`**: Every type you want to access via `tables` must be declared with `@table` in `schema.graphql`. Use `@indexed` on attributes you intend to sort or filter by efficiently.
2023
2119
 
2024
2120
  ```graphql
2025
2121
  type Product @table {
@@ -2029,54 +2125,88 @@ Apply this rule when writing server-side Harper code that reads, writes, searche
2029
2125
  }
2030
2126
  ```
2031
2127
 
2032
- 3. **Query records using `search(`**: Pass a Query object to `search()`. The query is an async iterable; iterate the result with `for await`. Use a `conditions` array to filter records. Each condition specifies `attribute`, `value`, and an optional `comparator`.
2128
+ 3. **Perform basic CRUD operations**: Use the table class methods directly.
2129
+
2130
+ ```javascript
2131
+ // Create
2132
+ const created = await Product.create({ name: 'Shirt', price: 9.5 });
2133
+
2134
+ // Patch
2135
+ await Product.patch(created.id, { price: Math.round(created.price * 0.8 * 100) / 100 });
2136
+
2137
+ // Get by primary key
2138
+ const record = await Product.get(created.id);
2139
+ ```
2033
2140
 
2034
- | Property | Description |
2035
- | ------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
2036
- | `attribute` | Property name, or array for chained/joined properties |
2037
- | `value` | The value to match |
2038
- | `comparator` | `equals` (default), `greater_than`, `greater_than_equal`, `less_than`, `less_than_equal`, `starts_with`, `contains`, `ends_with`, `between`, `not_equal`, or a full-text comparator |
2039
- | `conditions` | Nested conditions array |
2040
- | `operator` | `and` (default) or `or` for the nested `conditions` |
2141
+ 4. **Query with `search(` and a query object**: Pass a query object with `conditions`, `sort`, `limit`, `offset`, `select`, and other options. `search(` returns an async iterable. If order doesn't matter, omit `sort` entirely and `search({})` will iterate without an index requirement.
2041
2142
 
2042
2143
  ```javascript
2043
- for await (const record of Product.search({
2144
+ const query = {
2044
2145
  conditions: [{ attribute: 'price', comparator: 'less_than', value: 8.0 }],
2045
- })) {
2146
+ };
2147
+ for await (const record of Product.search(query)) {
2046
2148
  // process record
2047
2149
  }
2048
2150
  ```
2049
2151
 
2050
- 4. **Use `select` to control which properties are returned**: Pass an array of property names, a string for a single property, or nested objects for relationships.
2152
+ 5. **Build `conditions` arrays**: Each condition object supports the following properties:
2153
+
2154
+ | Property | Description |
2155
+ | -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
2156
+ | `attribute` | Property name, or array for chained/joined properties |
2157
+ | `value` | The value to match |
2158
+ | `comparator` | `equals` (default), `greater_than`, `greater_than_equal`, `less_than`, `less_than_equal`, `starts_with`, `contains`, `ends_with`, `between`, `not_equal`, or a full-text comparator (`matches`, `matches_all`, `matches_phrase`, `matches_prefix`, `matches_fuzzy`, `matches_fuzzy_prefix`, and their `not_` forms) |
2159
+ | `conditions` | Nested conditions array |
2160
+ | `operator` | `and` (default) or `or` for nested `conditions` |
2161
+ | `fields` | Full-text source fields to search; every name must belong to the selected full-text index |
2162
+ | `includeHighlights` | Include configured full-text highlights |
2163
+ | `maxIndexLagMilliseconds` | Maximum accepted full-text index lag |
2164
+ | `waitForIndexMilliseconds` | Time to wait for acceptable full-text coverage |
2165
+
2166
+ Use nested `conditions` and `operator` for compound logic:
2051
2167
 
2052
2168
  ```javascript
2053
- const book = await Book.get({
2054
- id: 42,
2055
- select: ['id', 'title', { name: 'author', select: ['name'] }],
2169
+ Product.search({
2170
+ conditions: [
2171
+ { attribute: 'price', comparator: 'less_than', value: 100 },
2172
+ {
2173
+ operator: 'or',
2174
+ conditions: [
2175
+ { attribute: 'rating', comparator: 'greater_than', value: 4 },
2176
+ { attribute: 'featured', value: true },
2177
+ ],
2178
+ },
2179
+ ],
2056
2180
  });
2057
2181
  ```
2058
2182
 
2059
- Special `select` properties:
2060
- - `$id` — returns the primary key regardless of its name
2061
- - `$updatedtime` — returns the last-updated timestamp
2062
- - `$distance` — returns computed vector distance when using a vector index
2063
- - `$score` — returns BM25 relevance score for full-text queries
2064
- - `$highlights` — returns full-text fragments and matching spans
2065
-
2066
- 5. **Use `sort` with an `@indexed` attribute or combine with `conditions`**: Harper requires either an `@indexed` sort attribute or at least one `conditions` entry when sorting. Sorting by a non-indexed attribute with zero conditions raises:
2183
+ Use an array `attribute` to traverse relationships:
2067
2184
 
2068
- > `HdbError: <attribute> is not indexed and not combined with any other conditions`
2185
+ ```javascript
2186
+ Product.search({ conditions: [{ attribute: ['brand', 'name'], value: 'Harper' }] });
2187
+ ```
2069
2188
 
2070
- Note: `@primaryKey` alone is treated as not indexed for sort purposes. To iterate a whole table in primary-key order, add an open-ended condition:
2189
+ 6. **Apply `select` to shape results**: Pass an array of property names, nested objects for relationships, or a string for a single property.
2071
2190
 
2072
2191
  ```javascript
2073
- Product.search({
2074
- conditions: [{ attribute: 'id', comparator: 'greater_than', value: '' }],
2075
- sort: { attribute: 'id' },
2192
+ // Flat select
2193
+ Product.search({ select: ['name', 'price'] });
2194
+
2195
+ // Nested relationship select
2196
+ const book = await Book.get({
2197
+ id: 42,
2198
+ select: ['id', 'title', { name: 'author', select: ['name'] }],
2076
2199
  });
2200
+
2201
+ // Special properties:
2202
+ // $id — primary key regardless of name
2203
+ // $updatedtime — last-updated timestamp
2204
+ // $distance — vector index distance
2205
+ // $score — full-text BM25 score
2206
+ // $highlights — full-text highlight fragments
2077
2207
  ```
2078
2208
 
2079
- `sort` object properties:
2209
+ 7. **Use `sort` with `@indexed` attributes**: Provide a `sort` object with `attribute`, optional `descending`, and optional `next` for tie-breaking.
2080
2210
 
2081
2211
  | Property | Description |
2082
2212
  | ------------ | ---------------------------------------------------------- |
@@ -2084,102 +2214,137 @@ Apply this rule when writing server-side Harper code that reads, writes, searche
2084
2214
  | `descending` | Sort descending if `true` (default: `false`) |
2085
2215
  | `next` | Secondary sort to resolve ties (same structure) |
2086
2216
 
2087
- 6. **Apply `limit` and `offset` for pagination**: Add these to the Query object to bound result sets.
2088
-
2089
2217
  ```javascript
2218
+ // attribute must be @indexed, or at least one condition must be present
2090
2219
  Product.search({
2091
- conditions: [{ attribute: 'price', comparator: 'less_than', value: 100 }],
2092
- limit: 20,
2093
- offset: 40,
2220
+ sort: { attribute: 'price', descending: true },
2094
2221
  });
2095
2222
  ```
2096
2223
 
2097
- 7. **Use `explain` and `enforceExecutionOrder` for debugging and optimization**:
2098
- - `explain: true` — returns conditions reordered as Harper will execute them.
2099
- - `enforceExecutionOrder: true` — forces conditions to execute in the order supplied, disabling Harper's automatic re-ordering.
2224
+ If the sort `attribute` is not `@indexed` and no `conditions` are present, Harper throws:
2100
2225
 
2101
- 8. **Use `addTo` for concurrent-safe numeric updates**: Increments or decrements a property using CRDT incrementation, safe across threads and nodes.
2226
+ > `HdbError: <attribute> is not indexed and not combined with any other conditions`
2227
+
2228
+ To iterate a whole table in primary-key order, add an open-ended condition:
2102
2229
 
2103
2230
  ```javascript
2104
- static async post(target, data) {
2105
- const record = await this.update(target.id);
2106
- record.addTo('quantity', -1); // decrement safely across nodes
2107
- }
2231
+ Product.search({
2232
+ conditions: [{ attribute: 'id', comparator: 'greater_than', value: '' }],
2233
+ sort: { attribute: 'id' },
2234
+ });
2235
+ ```
2236
+
2237
+ 8. **Apply `limit` and `offset` for pagination**:
2238
+
2239
+ ```javascript
2240
+ Product.search({
2241
+ conditions: [{ attribute: 'price', comparator: 'less_than', value: 100 }],
2242
+ sort: { attribute: 'price' },
2243
+ limit: 20,
2244
+ offset: 40,
2245
+ });
2108
2246
  ```
2109
2247
 
2110
- 9. **Wrap background or timer work in `transaction()`**: HTTP handlers get a transaction automatically. Use `transaction()` explicitly for timers, background jobs, or any code outside a natural request context. Always `await` the call and `catch` errors.
2248
+ 9. **Use `explain` and `enforceExecutionOrder` for debugging and control**:
2249
+ - Set `explain: true` to return conditions reordered as Harper will execute them.
2250
+ - Set `enforceExecutionOrder: true` to force conditions to execute in the order supplied, disabling Harper's automatic reordering.
2111
2251
 
2112
2252
  ```javascript
2113
- await transaction(async (txn) => {
2114
- for (let item of data) {
2115
- await MyTable.put(item, txn);
2116
- }
2253
+ Product.search({
2254
+ conditions: [{ attribute: 'price', comparator: 'less_than', value: 100 }],
2255
+ explain: true,
2117
2256
  });
2118
2257
  ```
2119
2258
 
2120
- The `txn` object exposes:
2259
+ 10. **Use `addTo` for concurrent-safe numeric updates**: `addTo(property, value)` uses CRDT incrementation, safe across threads and nodes.
2121
2260
 
2122
- | Member | Type | Description |
2123
- | --------------------- | --------------- | ------------------------------------------------------ |
2124
- | `commit()` | `() => Promise` | Commits the current transaction |
2125
- | `abort()` | `() => void` | Aborts the transaction and resets it |
2126
- | `resetReadSnapshot()` | `() => void` | Resets the read snapshot to the latest committed state |
2127
- | `timestamp` | `number` | Timestamp associated with the current transaction |
2128
-
2129
- On normal callback completion the transaction commits automatically. If the callback throws, the transaction is aborted.
2261
+ ```javascript
2262
+ static async post(target, data) {
2263
+ const record = await this.update(target.id);
2264
+ record.addTo('quantity', -1); // decrement safely across nodes
2265
+ }
2266
+ ```
2130
2267
 
2131
- 10. **Understand atomicity boundaries**: All tables within the same database share a single transactional context — reads return a consistent snapshot and writes are committed atomically. Tables in different databases each get their own transaction with no cross-database atomicity guarantee.
2268
+ 11. **Wrap out-of-context writes in `transaction()`**: HTTP request handlers get a transaction automatically. Use `transaction()` explicitly for timers, background jobs, or any code outside a natural transaction context.
2132
2269
 
2133
- #### Examples
2270
+ ```javascript
2271
+ import { isMainThread } from 'node:worker_threads';
2272
+ import { tables, transaction } from 'harper';
2273
+ const { MyTable } = tables;
2274
+
2275
+ if (isMainThread) {
2276
+ let running = false;
2277
+ setInterval(async () => {
2278
+ if (running) return; // the previous run has not committed yet
2279
+ running = true;
2280
+ try {
2281
+ let data = await (await fetch('https://example.com/data')).json();
2282
+ await transaction(async (txn) => {
2283
+ for (let item of data) {
2284
+ await MyTable.put(item, txn);
2285
+ }
2286
+ });
2287
+ } catch (error) {
2288
+ logger.error('hourly import failed', error);
2289
+ } finally {
2290
+ running = false;
2291
+ }
2292
+ }, 3600000);
2293
+ }
2294
+ ```
2134
2295
 
2135
- ##### Basic CRUD and search
2296
+ - Always `await` the `transaction()` call and `catch` it.
2297
+ - Guard against overlap with a `running` flag when a job can outlast its interval.
2136
2298
 
2137
- ```javascript
2138
- import { tables } from 'harper';
2139
- const { Product } = tables;
2299
+ 12. **Understand `transaction()` atomicity and scope**: Transactions span a single database. All tables within the same database share one transactional context — reads return a consistent snapshot and writes across multiple tables commit with atomicity. Tables in different databases each get their own transaction with no cross-database atomicity guarantee.
2140
2300
 
2141
- // Create
2142
- const created = await Product.create({ name: 'Shirt', price: 9.5 });
2301
+ The `txn` object received by the callback exposes:
2143
2302
 
2144
- // Patch
2145
- await Product.patch(created.id, { price: Math.round(created.price * 0.8 * 100) / 100 });
2303
+ | Member | Type | Description |
2304
+ | --------------------- | --------------- | ------------------------------------------------------ |
2305
+ | `commit()` | `() => Promise` | Commits the current transaction |
2306
+ | `abort()` | `() => void` | Aborts the transaction and resets it |
2307
+ | `resetReadSnapshot()` | `() => void` | Resets the read snapshot to the latest committed state |
2308
+ | `timestamp` | `number` | Timestamp associated with the current transaction |
2146
2309
 
2147
- // Get by primary key
2148
- const record = await Product.get(created.id);
2310
+ On normal callback completion the transaction commits automatically. If the callback throws, the transaction is aborted.
2149
2311
 
2150
- // Search with conditions
2151
- const query = {
2152
- conditions: [{ attribute: 'price', comparator: 'less_than', value: 8.0 }],
2153
- };
2154
- for await (const record of Product.search(query)) {
2155
- // process record
2156
- }
2157
- ```
2312
+ #### Examples
2158
2313
 
2159
- ##### Nested conditions with `or`
2314
+ **Relationship group — one related record must meet all conditions:**
2160
2315
 
2161
2316
  ```javascript
2162
- Product.search({
2317
+ const brands = [];
2318
+ for await (const brand of Brand.search({
2163
2319
  conditions: [
2164
- { attribute: 'price', comparator: 'less_than', value: 100 },
2165
2320
  {
2166
- operator: 'or',
2321
+ attribute: 'products',
2167
2322
  conditions: [
2168
- { attribute: 'rating', comparator: 'greater_than', value: 4 },
2169
- { attribute: 'featured', value: true },
2323
+ { attribute: 'name', value: 'Keyboard' },
2324
+ { attribute: 'price', comparator: 'greater_than_equal', value: 30 },
2170
2325
  ],
2171
2326
  },
2172
2327
  ],
2173
- });
2328
+ select: ['id', 'name', { name: 'products', select: ['name', 'price'] }],
2329
+ })) {
2330
+ brands.push(brand);
2331
+ }
2174
2332
  ```
2175
2333
 
2176
- ##### Chained attribute reference (join/relationship)
2334
+ **SSR entry point reading directly from Harper:**
2177
2335
 
2178
2336
  ```javascript
2179
- Product.search({ conditions: [{ attribute: ['brand', 'name'], value: 'Harper' }] });
2337
+ import { tables } from 'harper';
2338
+
2339
+ export async function render(url: string): Promise<string> {
2340
+ const product = await tables.Product.get(idFromUrl(url));
2341
+ return renderToString(/* <App product={product} /> */);
2342
+ }
2180
2343
  ```
2181
2344
 
2182
- ##### Selecting and constraining related records
2345
+ Keep `harper` external in `vite.config` — e.g. `ssr: { external: ['harper'] }` — so it resolves to the runtime instead of being bundled.
2346
+
2347
+ **Nested relationship select with constraints:**
2183
2348
 
2184
2349
  ```javascript
2185
2350
  const brand = await Brand.get({
@@ -2198,60 +2363,23 @@ const brand = await Brand.get({
2198
2363
  });
2199
2364
  ```
2200
2365
 
2201
- ##### Background job with `transaction()`
2202
-
2203
- ```javascript
2204
- import { isMainThread } from 'node:worker_threads';
2205
- import { tables, transaction } from 'harper';
2206
- const { MyTable } = tables;
2207
-
2208
- if (isMainThread) {
2209
- let running = false;
2210
- setInterval(async () => {
2211
- if (running) return; // the previous run has not committed yet
2212
- running = true;
2213
- try {
2214
- let data = await (await fetch('https://example.com/data')).json();
2215
- await transaction(async (txn) => {
2216
- for (let item of data) {
2217
- await MyTable.put(item, txn);
2218
- }
2219
- });
2220
- } catch (error) {
2221
- logger.error('hourly import failed', error);
2222
- } finally {
2223
- running = false;
2224
- }
2225
- }, 3600000); // every hour
2226
- }
2227
- ```
2228
-
2229
- ##### SSR with Vite
2230
-
2231
- ```typescript
2232
- import { tables } from 'harper';
2233
-
2234
- export async function render(url: string): Promise<string> {
2235
- const product = await tables.Product.get(idFromUrl(url));
2236
- return renderToString(/* <App product={product} /> */);
2237
- }
2238
- ```
2239
-
2240
- Keep `harper` external in `vite.config` so it resolves to the runtime:
2366
+ **Sort with secondary tie-breaker across a relationship:**
2241
2367
 
2242
2368
  ```javascript
2243
- ssr: {
2244
- external: ['harper'];
2245
- }
2369
+ Brand.search({
2370
+ conditions: [{ attribute: ['products', 'price'], comparator: 'greater_than_equal', value: 30 }],
2371
+ sort: { attribute: ['products', 'name'], next: { attribute: 'id' } },
2372
+ });
2246
2373
  ```
2247
2374
 
2248
2375
  #### Notes
2249
2376
 
2250
- - Calls through `tables` run in a trusted server-side context and do **not** automatically apply the target table's role permissions. Set `checkPermission: true` on the query and pass the authenticated `context` as the second argument when authorization is required.
2251
- - `update`, `patch`, and `delete` operate directly on stored data. Scope destructive operations with specific `conditions`, validate the affected set before writing, and gate them behind authorization controls.
2252
- - Pass `allowFullScan: true` to permit an unconditional ordered scan without conditions. If order doesn't matter, omit `sort` entirely — `search({})` iterates without an index requirement.
2253
- - If `transaction()` is called with a context that already has an active transaction, it reuses that transaction and executes the callback immediately — safe to call defensively.
2254
- - Guard timer-based jobs against overlap: a timer fires on schedule regardless of whether the previous run has committed. Use a `running` flag to skip a tick when the previous run is still active.
2377
+ - Calls through `tables` and `databases` run in a trusted server-side context and do not automatically apply the target table's role permissions. To enforce authorization, set `checkPermission: true` on the query and pass the authenticated `context` as the second argument to `search()` or `get()`. Harper then authorizes the read using `context.user`, including table, selected-property, and full-text source-field permissions.
2378
+ - Programmatic `update`, `patch`, and `delete` calls operate directly on stored (potentially live production) data. Scope destructive operations with specific `conditions`, validate the affected set before writing, and gate them behind authorization controls.
2379
+ - `transaction()` is safe to call defensively: if the context already has an active transaction, it reuses it and executes the callback immediately.
2380
+ - `resetReadSnapshot()` on the `txn` object resets the read snapshot to the latest committed state mid-transaction.
2381
+ - The bare `@primaryKey` is treated as not indexed for sort purposes (it uses the primary store, not a secondary index). Sorting by primary key alone with no `conditions` triggers the `HdbError: <attribute> is not indexed and not combined with any other conditions` error. Add an open-ended condition or pass `allowFullScan: true` to permit an unconditional scan.
2382
+ - All installed components have `harper` automatically linked. For components in their own directory, run `npm link harper` manually.
2255
2383
 
2256
2384
  ### 3.4 TypeScript Type Stripping in Harper
2257
2385
 
@@ -2484,148 +2612,186 @@ Apply this rule when deploying a Harper application to a remote Harper Fabric cl
2484
2612
 
2485
2613
  #### How It Works
2486
2614
 
2487
- 1. **Log in to the cluster once**: Pass the cluster's **Application URL**, from its **Config → Overview** page, to `harper login`. The CLI stores the token, so later commands don't repeat credentials. If the current directory already has a `.env` file that sets no target, it also appends `HARPER_CLI_TARGET` to it.
2615
+ 1. **Authenticate against the cluster**: Run `harper login` once, pointing at the cluster's Application URL (found on **Config → Overview**). The CLI stores the token and sets the URL as the default target. If the current directory has a `.env` file, and neither that file nor your environment sets `HARPER_CLI_TARGET` or `CLI_TARGET`, login appends `HARPER_CLI_TARGET` to it.
2488
2616
 
2489
2617
  ```bash
2490
2618
  harper login <Application URL>
2491
2619
  # Provide cluster username and password when prompted
2492
2620
  ```
2493
2621
 
2494
- For CI/CD pipelines, use workload identity (OIDC) or a refresh token from `harper login --for-ci` instead of a password. See [deploying-from-ci.md](deploying-from-ci.md) for a complete GitHub Actions workflow. For a one-off command, `auth_username=` and `auth_password=` also work, and take precedence over environment variables and saved login tokens. Don't use them in production.
2622
+ For CI/CD pipelines, use workload identity (OIDC) or a refresh token from `harper login --for-ci` instead of a password. See [deploying-from-ci.md](deploying-from-ci.md) for a complete GitHub Actions workflow. For a one-off command, `auth_username=` and `auth_password=` are available but not recommended for production.
2495
2623
 
2496
- 2. **Deploy the application**: Run `harper deploy` with the required parameters. Omit `package` to upload the current directory; supply a reference for npm, GitHub, or tarball sources.
2624
+ 2. **Deploy the application**: Run `harper deploy` with the required parameters. A bare `target` host defaults to `https://<host>:9925`. Use `restart=rolling` for a staggered, zero-downtime restart across the cluster. Pass `replicated=false` to restrict the deploy to the single node you called; on Harper Fabric, omit it to reach every node.
2497
2625
 
2498
2626
  ```bash
2499
2627
  harper deploy \
2500
2628
  project=<name> \
2501
2629
  package=<package> \
2502
2630
  target=<remote> \
2503
- restart=true \
2631
+ restart=rolling \
2504
2632
  replicated=true
2505
2633
  ```
2506
2634
 
2507
- A bare `target` host defaults to `https://<host>:9925`. Use `restart=rolling` for a staggered, zero-downtime restart across the cluster. Pass `replicated=false` to deploy only to the node you called. Pass `json=true` to print the result as JSON instead of YAML.
2635
+ Key parameters:
2636
+
2637
+ | Parameter | Description |
2638
+ | --------------------- | ------------------------------------------------------------------------------------- |
2639
+ | `package=<reference>` | npm, GitHub, or tarball reference. Omit to deploy current directory. |
2640
+ | `target=<url>` | Remote Harper instance URL. Bare host defaults to `https://<host>:9925`. |
2641
+ | `restart=rolling` | Restart nodes one at a time; each stays in rotation while it restarts. |
2642
+ | `replicated=false` | Deploy to the called node only. |
2643
+ | `wait=false` | With `restart=rolling`, exit after the called node takes the release. |
2644
+ | `wait_timeout=<ms>` | With `restart=rolling`, maximum wait time per node. Default: ten minutes per node. |
2645
+ | `activate=false` | Build and install without making the release live. Cannot be combined with `restart`. |
2646
+ | `ref=<committish>` | Branch, tag, or commit SHA to deploy. Resolved to SHA. Implies `by_ref`. |
2647
+ | `json=true` | Print output as JSON instead of YAML. |
2508
2648
 
2509
2649
  3. **Choose a package source**: The `package` field accepts any valid npm dependency value.
2510
2650
 
2511
- | Source | Value |
2512
- | ------------------ | --------------------------------------------------------------------------------------------- |
2513
- | npm package | `package="@harperdb/status-check"` |
2514
- | GitHub (public) | `package="HarperFast/status-check"` or `package="https://github.com/HarperFast/status-check"` |
2515
- | Private repo (SSH) | `package="git+ssh://git@github.com:HarperDB/secret-app.git"` |
2516
- | Tarball | `package="https://example.com/application.tar.gz"` |
2651
+ | Format | Example |
2652
+ | ----------------------- | --------------------------------------------------------------------------------------------- |
2653
+ | Current local directory | Omit `package` |
2654
+ | npm package | `package="@harperdb/status-check"` |
2655
+ | GitHub (public) | `package="HarperFast/status-check"` or `package="https://github.com/HarperFast/status-check"` |
2656
+ | Private repo (SSH) | `package="git+ssh://git@github.com:HarperDB/secret-app.git"` |
2657
+ | Tarball | `package="https://example.com/application.tar.gz"` |
2517
2658
 
2518
2659
  For git tags, use the `semver` directive: `HarperFast/application-template#semver:v1.0.0`. For an SSH-based private repo, register the key with `add_ssh_key` first.
2519
2660
 
2520
- 4. **Provision credentials for private repositories**: Run `harper deploy setup=true` once per component and source. (With `provider=github-actions`, it sets up GitHub Actions OIDC deploys instead.) This requires `super_user` privileges and is interactive. It calls `get_secrets_public_key`, `set_secret`, and `grant_secret`. Run it with an administrative credential rather than the CI identity it provisions for:
2661
+ 4. **Provision credentials for private repositories**: Run `harper deploy setup=true` once per component and source. (With `provider=github-actions`, it sets up GitHub Actions OIDC deploys instead.) This requires `super_user` privileges — run it with an administrative credential, not the CI identity it provisions for.
2521
2662
 
2522
2663
  ```bash
2523
2664
  harper deploy setup=true
2524
2665
  ```
2525
2666
 
2526
- The command:
2527
- 1. Fetches the cluster's public key with `get_secrets_public_key`.
2528
- 2. Encrypts the token locally into an `enc:v1:` envelope.
2529
- 3. Stores only the ciphertext with `set_secret`, scoped to the component, never in the global `processEnv` tier.
2530
- 4. Grants the component permission to resolve it with `grant_secret`.
2667
+ The setup flow:
2668
+ - Fetches the cluster's public key with `get_secrets_public_key`.
2669
+ - Encrypts the token locally into an `enc:v1:` envelope.
2670
+ - Stores only the ciphertext with `set_secret` in the component-scoped tier — never in the global `processEnv` tier that every component and child process can read.
2671
+ - Grants the component permission to resolve it with `grant_secret`.
2672
+
2673
+ Prefer a **fine-grained** personal access token scoped to **Contents: Read-only** on the target repository. Avoid the `gh` CLI session token, which carries `read:org`, `gist`, `workflow`, and `repo` scopes across your whole account. The encrypted token is durable and replayed on every cold deploy and rollback, so use the narrowest credential that does the job.
2531
2674
 
2532
- Use a **fine-grained** personal access token with **Contents: Read-only** on the target repository. Avoid broad tokens: the `gh` CLI session token typically carries `read:org`, `gist`, and `workflow` scopes across your whole account. Then deploy with `credential=true`:
2675
+ Then deploy with `credential=true`:
2533
2676
 
2534
2677
  ```bash
2535
2678
  harper deploy by_ref=true credential=true restart=true replicated=true
2536
2679
  ```
2537
2680
 
2538
- 5. **Stage a release to make live later**: `activate=false` (`activate: false` in the operation) builds and installs the release on each node without making it live, so the slow, fallible part happens when you choose and the cut-over is quick. Cannot be combined with `restart`.
2681
+ 5. **Deploy by reference for a reproducible source**: `by_ref=true` sends a pinned git reference instead of uploading the working directory, and the cluster fetches and builds that exact commit. It resolves the repository's `origin` remote and the current commit, then deploys `package=git+https://github.com/<owner>/<repo>.git#<full commit SHA>`.
2539
2682
 
2540
- ```json
2541
- {
2542
- "operation": "deploy_component",
2543
- "project": "my-app",
2544
- "package": "npm:@my-org/my-app@1.2.3",
2545
- "activate": false
2546
- }
2683
+ ```bash
2684
+ harper deploy by_ref=true restart=true replicated=true
2685
+ ```
2686
+
2687
+ Deploy a specific tag or commit with `ref`:
2688
+
2689
+ ```bash
2690
+ harper deploy ref=v1.2.0 restart=true replicated=true
2691
+ harper deploy ref=9f8c2a1 restart=true replicated=true
2547
2692
  ```
2548
2693
 
2549
- The response names the artifact waiting to be activated:
2694
+ Tags and branches are resolved to a full commit SHA before the deploy is sent. A `ref` that cannot be resolved stops the deploy — run `git fetch` and retry, or pass a full commit SHA. A full commit SHA is accepted directly with no resolution step. Valid refs must be in `refs/heads/*` or `refs/tags/*` if qualified; a ref like `refs/pull/123/head` is rejected up front even if your local checkout can resolve it. Commit and push before deploying — the cluster clones from the remote and only sees pushed commits.
2695
+ - On a GitHub Actions `pull_request` run, `by_ref` deploys the pull request's head commit. If the event payload isn't readable, the deploy stops; pass the commit explicitly:
2696
+
2697
+ ```bash
2698
+ harper deploy ref=${{ github.event.pull_request.head.sha }} restart=true replicated=true
2699
+ ```
2700
+
2701
+ 6. **Activate a staged build**: Make a staged build live using its `deployment_id`. Pass `restart` here, not on the stage step. Takes no `package`, `payload`, `credentials`, install options, or `urlPath`/`host` — those are rejected rather than ignored.
2550
2702
 
2551
2703
  ```json
2552
2704
  {
2553
- "deployment_id": "a3f8c2d1-...",
2554
- "message": "Staged: my-app. Deploy it with deploy_component deployment_id=a3f8c2d1-..."
2705
+ "operation": "deploy_component",
2706
+ "project": "my-app",
2707
+ "deployment_id": "a3f8c2d1-..."
2555
2708
  }
2556
2709
  ```
2557
2710
 
2558
- Nothing about the running component changes, and `get_deployment` reports it as `staged`. Make it live by deploying that `deployment_id`, as in step 6. The activation takes no `package`, `payload`, `credentials`, install options, or `urlPath`/`host`. They are rejected, because the staged build already decided them.
2559
-
2560
- 6. **Activate a staged build or roll back**: Pass `deployment_id` with `restart` to make a staged build live, or to restore a release a later deploy replaced, with no rebuild or reinstall. Pass `restart` here rather than on the stage.
2711
+ Or via the CLI:
2561
2712
 
2562
2713
  ```bash
2563
2714
  harper deploy project=<name> deployment_id=<id> restart=true
2564
2715
  ```
2565
2716
 
2566
- Use `list_deployments` to find the `deployment_id` of the release to restore:
2717
+ 7. **Stage a build for later activation**: Pass `activate=false` (`activate: false` in the operation) to build and install without making the release live. `activate: false` cannot be combined with `restart`.
2567
2718
 
2568
2719
  ```json
2569
- { "operation": "list_deployments", "project": "my-app", "status": "success", "limit": 20 }
2720
+ {
2721
+ "operation": "deploy_component",
2722
+ "project": "my-app",
2723
+ "package": "npm:@my-org/my-app@1.2.3",
2724
+ "activate": false
2725
+ }
2570
2726
  ```
2571
2727
 
2572
- - A `404` response means the id never existed on that node, was pruned, or the release was replaced and not kept. A `409` means the build exists but cannot be activated (e.g., belongs to another component, or files changed after staging).
2573
- - Kept releases and staged builds count against `deployment.stagingRetention.maxCount` (`stagingRetention` default: `5`). Each kept release is a full installed copy of the component, `node_modules` included, so budget disk for it. `0` keeps no replaced releases.
2574
-
2575
- 7. **Deploy by reference for a reproducible source**: `by_ref=true` sends a pinned git reference instead of uploading the working directory, and the cluster fetches and builds that exact commit. It resolves the repository's `origin` remote and the current commit, then deploys `package=git+https://github.com/<owner>/<repo>.git#<full commit SHA>`. Defaults to `HEAD`.
2728
+ The response names the artifact that is now waiting:
2576
2729
 
2577
- ```bash
2578
- harper deploy by_ref=true restart=true replicated=true
2730
+ ```json
2731
+ {
2732
+ "deployment_id": "a3f8c2d1-...",
2733
+ "message": "Staged: my-app. Deploy it with deploy_component deployment_id=a3f8c2d1-..."
2734
+ }
2579
2735
  ```
2580
2736
 
2581
- Deploy a specific tag or commit with `ref`:
2737
+ The response includes a `deployment_id` naming the staged artifact. Staged builds and replaced releases together count against `deployment.stagingRetention.maxCount` (default `5`). Each kept release is a full installed copy of the component, `node_modules` included. The oldest are pruned at the next deploy or startup.
2738
+ - `404` means the id never existed on that node, was pruned, or the release was replaced and not kept. A `409` means the build exists but cannot be activated (e.g., belongs to another component, or files changed after staging).
2582
2739
 
2583
- ```bash
2584
- harper deploy ref=v1.2.0 restart=true replicated=true
2585
- harper deploy ref=9f8c2a1 restart=true replicated=true
2586
- ```
2740
+ 8. **Roll back to a previous release**: Use `list_deployments` to find the `deployment_id` of the release to restore. Then activate that id with `restart`. This swaps in the exact installed release with no fetch, rebuild, or reinstall. The release it replaces is kept in turn, so you can go forward again the same way.
2587
2741
 
2588
- - Tags and branches are resolved to a full commit SHA before the deploy is sent. A `ref` that cannot be resolved stops the deploy — run `git fetch` and retry, or pass a full commit SHA.
2589
- - Any other `ref` must name something a clone can fetch: `refs/heads/*` or `refs/tags/*`, or a bare branch or tag name. A qualified ref outside those, such as `refs/pull/123/head`, is rejected up front.
2590
- - On a GitHub Actions `pull_request` run, `by_ref` deploys the pull request's head commit. If the event payload isn't readable, the deploy stops; pass the commit explicitly.
2591
- - Commit and push first — the cluster clones from the remote and only sees pushed commits.
2742
+ ```json
2743
+ {
2744
+ "operation": "deploy_component",
2745
+ "project": "my-app",
2746
+ "deployment_id": "<id of the deployment you want back>",
2747
+ "restart": true
2748
+ }
2749
+ ```
2592
2750
 
2593
- 8. **Check certification after a rolling deploy**: A deploy with `restart=rolling` certifies the release in a canary worker before rolling it out. The deploy response includes a `certification` field:
2751
+ 9. **Check certification after a rolling deploy**: A deploy with `"restart": "rolling"` certifies the release on each node using a canary worker before rolling it out. The deploy response includes a `certification` field:
2594
2752
 
2595
2753
  | `certification` | Meaning |
2596
2754
  | --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
2597
2755
  | `certified` | The canary loaded the release; rollout proceeded. |
2598
2756
  | `uncertified` | The rollout went on unchecked: the component ran nothing as it loaded (`loadComponent: dev-only`, an absent `if-installed` component, or safe mode), or no worker was replaced. |
2599
- | `unavailable` | Deploy restarted unchecked (no worker loads the component, `isolated` setting changed, already live, or `file:` directory source). |
2757
+ | `unavailable` | Deploy restarted unchecked (no worker loads the component, `isolated` setting changed, already live, or `package: file:<dir>` source). |
2600
2758
  | `not-requested` | The deploy did not restart. |
2601
- - **A rejected release fails the deploy.** If the canary throws while loading the component, exits, or doesn't report within a minute, the deploy fails with `400`. The release it replaced is made live again, and nothing is replicated to other nodes. The error carries the decision as an object in `certification` with `status` set to `rejected`, or to `interrupted` for a restart that stopped before the canary decided.
2602
- - **A release with nothing to go back to fails closed.** A rejected first deploy, or one whose previous release was not kept, stays on disk, but no worker on that node loads it, even after Harper restarts. `get_status` reports the component as failed and says why. Deploy a fix, or activate the same `deployment_id` again with `restart`.
2759
+ - **A rejected release fails the deploy.** If the canary throws while loading the component, exits, or doesn't report within a minute, the deploy fails with `400`. The release it replaced is made live again, and nothing is replicated to other nodes. A restart that stops before the canary decides is reported with `status: "interrupted"`.
2760
+ - **A release with nothing to go back to fails closed.** A rejected first deploy, or one whose previous release was not kept, stays on disk, but no worker on that node loads it, even after Harper restarts. `get_status` reports the component as failed and says why. Deploy a fixed release, or activate the same `deployment_id` again with `restart` once the cause is fixed.
2603
2761
  - Another deploy or `drop_component` of the component on that node is refused with `409` until the rollout has finished.
2604
2762
 
2605
- 9. **Wait for a rolling deploy to finish**: A `deploy_component` with `"restart": "rolling"` returns a `restartJobId`. Poll it with `get_job` to learn whether every node took the release.
2763
+ 10. **Check rolling deploy job status**: A `deploy_component` with `"restart": "rolling"` returns a `restartJobId`. Call `get_job` on the node that received the deploy — `system.hdb_job` is not replicated.
2606
2764
 
2607
- ```json
2608
- { "operation": "get_job", "id": "4a982782-929a-4507-8794-26dae1132def" }
2609
- ```
2765
+ ```json
2766
+ { "operation": "get_job", "id": "<restartJobId>" }
2767
+ ```
2768
+
2769
+ `get_job` returns status `COMPLETE`, `IN_PROGRESS`, or `ERROR`. It answers only on the node that ran the job — `system.hdb_job` is not replicated. For a rolling deploy, that is the node that received the deploy. A deploy-only role that polls for rolling deploys must include `get_job` in its operations allowlist. The job ends `COMPLETE` on success, or `ERROR` with a `message` listing each node's outcome under `activated`.
2610
2770
 
2611
- `get_job` returns status `COMPLETE`, `IN_PROGRESS`, or `ERROR`. It answers only on the node that ran the job. A deploy-only role that waits for rolling deploys must include `get_job` in its operations allowlist.
2612
- - `list_deployments` returns records newest first. Filter by `project` and `status`; omit `limit` to return every matching record. A deploy in progress reads `pending`, or `loading`, until its `completed_at` is set.
2771
+ 11. **Monitor `list_deployments` correctly**: A deploy in progress reads `pending`, or `loading` once its release has started loading. A rolling deploy's record reads `rolling_out` until the job has visited every node; filter `list_deployments` without `status` and check each record's `status` is a finished one (`success`, `failed`, `staged`, or `partial`). `completed_at` alone does not indicate completion, since a `rolling_out` record already has it.
2772
+
2773
+ ```json
2774
+ {
2775
+ "operation": "list_deployments",
2776
+ "project": "my-app",
2777
+ "limit": 20
2778
+ }
2779
+ ```
2780
+
2781
+ - The `list_deployments` response strips `payload_blob` and replaces it with `payload_blob_present`; use `get_deployment_payload` to retrieve the tarball.
2613
2782
 
2614
2783
  #### Examples
2615
2784
 
2616
- ##### Full rolling deploy from a pinned tag
2785
+ ##### Full rolling deploy from a git tag
2617
2786
 
2618
2787
  ```bash
2619
- # Authenticate once
2620
2788
  harper login https://my-cluster.harperdbcloud.com
2621
-
2622
- # Deploy a pinned release with rolling restart
2623
2789
  harper deploy \
2624
2790
  project=my-app \
2625
2791
  ref=v1.2.0 \
2626
- target=https://my-cluster.harperdbcloud.com \
2627
2792
  restart=rolling \
2628
- replicated=true
2793
+ replicated=true \
2794
+ credential=true
2629
2795
  ```
2630
2796
 
2631
2797
  ##### Stage a build, then activate in a maintenance window
@@ -2648,7 +2814,7 @@ Response:
2648
2814
  }
2649
2815
  ```
2650
2816
 
2651
- Activate later:
2817
+ Activate later with restart:
2652
2818
 
2653
2819
  ```json
2654
2820
  {
@@ -2659,27 +2825,30 @@ Activate later:
2659
2825
  }
2660
2826
  ```
2661
2827
 
2662
- ##### Roll back to a previous release
2828
+ ##### Roll back using list_deployments
2663
2829
 
2664
2830
  ```json
2665
- {
2666
- "operation": "list_deployments",
2667
- "project": "my-app"
2668
- }
2831
+ { "operation": "list_deployments", "project": "my-app", "limit": 10 }
2669
2832
  ```
2670
2833
 
2671
- Then:
2672
-
2673
2834
  ```json
2674
2835
  {
2675
2836
  "operation": "deploy_component",
2676
2837
  "project": "my-app",
2677
- "deployment_id": "<id of the deployment you want back>",
2838
+ "deployment_id": "<previous-deployment-id>",
2678
2839
  "restart": true
2679
2840
  }
2680
2841
  ```
2681
2842
 
2682
- ##### Rejected canary — error response shape
2843
+ ##### Check a rolling deploy job
2844
+
2845
+ ```json
2846
+ { "operation": "get_job", "id": "4a982782-929a-4507-8794-26dae1132def" }
2847
+ ```
2848
+
2849
+ The job ends `COMPLETE` when every node has taken the release, or `ERROR` with node-level detail if any node did not.
2850
+
2851
+ ##### Certification rejection response
2683
2852
 
2684
2853
  ```json
2685
2854
  {
@@ -2699,199 +2868,134 @@ Then:
2699
2868
 
2700
2869
  #### Notes
2701
2870
 
2702
- - `harper login` stores a token; subsequent `harper deploy` calls to the same target need no credentials. See [creating-a-fabric-account-and-cluster.md](creating-a-fabric-account-and-cluster.md) for cluster setup.
2703
- - For GitHub Actions, prefer OIDC workload identity over stored tokens. See [deploying-from-ci.md](deploying-from-ci.md).
2704
- - `restart=rolling` and `activate=false` cannot be combined.
2705
- - `deployment_id` activation takes no `package`, `payload`, `credentials`, install options, `urlPath`, or `host` — supplying them is rejected.
2706
- - `file:` directory sources (`package: file:<dir>`) cannot be staged and are not kept as rollback targets.
2707
- - `stagingRetention.maxCount` (default `5`) caps staged builds and kept replaced releases together per component.
2708
- - `get_job` on a `restartJobId` answers only on the node that received the deploy; any other node returns an empty array.
2709
- - A rolling deploy's record reads `rolling_out` until the job has visited every node; filter `list_deployments` without `status` and check each record's `status` for a finished value (`success`, `partial`, `failed`, `staged`). `completed_at` alone does not confirm a finished state, since a `rolling_out` record already has it.
2710
- - `enc:v1:` envelopes store only ciphertext — plaintext never leaves your machine and never appears in operation logs or replication.
2711
- - Tags used in `refs/tags` are resolved to a commit SHA before the deploy is sent; a tag that moves mid-deploy cannot cause nodes to run different code.
2712
- - `super_user` is required to run `harper deploy setup=true`.
2871
+ - `replicated=false` restricts a deploy to the single node you called. On Harper Fabric, omit this parameter to reach every node.
2872
+ - `restart=rolling` cannot be combined with `activate=false`.
2873
+ - `deployment_id` activation takes no `package`, `payload`, `credentials`, install options, or `urlPath`/`host` — supplying them is rejected.
2874
+ - `file:` directory sources cannot be staged with `activate=false` and are not kept as rollback targets.
2875
+ - `deployment.stagingRetention.maxCount` (default `5`) caps staged builds and kept replaced releases together per component. At the default, budget disk for up to five extra copies per component, `node_modules` included.
2876
+ - A deploy-only role that waits for rolling deploys must include `get_job` in its operations allowlist.
2877
+ - `get_job` answers only on the node that ran the job — `system.hdb_job` is not replicated.
2878
+ - The `enc:v1:` envelope is the only form of the token that ever leaves your machine or appears in API logs.
2879
+ - The secret provisioned by `harper deploy setup=true` is stored scoped to the component, never in the global `processEnv` tier.
2880
+ - For creating the cluster itself, see [creating-a-fabric-account-and-cluster.md](creating-a-fabric-account-and-cluster.md). For full CI/CD pipeline setup, see [deploying-from-ci.md](deploying-from-ci.md).
2713
2881
 
2714
2882
  ### 4.2 Deploying a Harper Application from GitHub Actions with OIDC
2715
2883
 
2716
- Instructions for the agent to configure and use OIDC trusted publishing so a GitHub Actions workflow can deploy to Harper with no stored credential.
2884
+ Instructions for the agent to configure OIDC trusted publishing so a GitHub Actions workflow can deploy to Harper with no stored credential.
2717
2885
 
2718
2886
  #### When to Use
2719
2887
 
2720
- Apply this rule when wiring up a GitHub Actions workflow to deploy a Harper application without storing a `HARPER_CLI_REFRESH_TOKEN` or any other durable credential in CI. Use it both for the one-command setup path (branch-triggered deploys) and for the manual policy path (tag-triggered releases). Also apply it when diagnosing a rejected OIDC exchange or revoking pipeline access.
2888
+ Apply this rule when setting up a GitHub Actions workflow to deploy a Harper application using OIDC identity tokens instead of stored secrets. Use it when running `harper deploy setup=true provider=github-actions` for the first time, when writing or debugging a deploy workflow, or when managing trust policies, deploy users, and rolling restarts.
2721
2889
 
2722
2890
  #### How It Works
2723
2891
 
2724
- ##### One-Command Setup (branch-triggered deploys)
2892
+ ##### 1. Run one-command setup as a `super_user`
2725
2893
 
2726
- 1. **Run setup once** from the project's checkout, signed in as a `super_user`:
2894
+ Log in to the cluster, then run setup from the project's checkout:
2727
2895
 
2728
- ```bash
2729
- harper deploy setup=true provider=github-actions project=my-app
2730
- ```
2731
-
2732
- If your workflow file is not `.github/workflows/deploy.yaml`, pass `workflow=`:
2733
-
2734
- ```bash
2735
- harper deploy setup=true provider=github-actions project=my-app workflow=.github/workflows/deploy.yml
2736
- ```
2737
-
2738
- 2. **Understand what setup does** — it executes these steps automatically:
2739
- - Resolves the GitHub repository's numeric `repository_id` from the `origin` remote (or `repo=<owner>/<name>`). Pass `repository_id=<id>` explicitly for private repos without `gh`.
2740
- - Validates the workflow file: it must be able to run on `main` (or `branch=`), run a job in the `production` environment (`environment=`), and grant `id-token: write`. Setup stops before writing anything if the workflow could never match or never request a token.
2741
- - Creates a role and a user named `<project>-ci-deploy` with permissions limited to `deploy_component`, `get_deployment`, `get_job`, and `registration_info`, plus the trust policy `github-actions-<project>`.
2742
- - Reads the policy back and compares it claim by claim.
2743
- - Sets the repository's `HARPER_CLI_TARGET` variable via `gh` if one does not already exist.
2744
-
2745
- 3. **Handle idempotency**: setup only creates — it never overwrites. A role, user, or policy that already exists but differs stops setup before writing. To replace a policy:
2896
+ ```bash
2897
+ harper login https://my-cluster.example.com:9925
2898
+ harper deploy setup=true provider=github-actions project=my-app
2899
+ ```
2746
2900
 
2747
- ```bash
2748
- harper drop_oidc_trust id=github-actions-<project>
2749
- ```
2901
+ If your workflow file is not `.github/workflows/deploy.yaml`, pass `workflow=` explicitly — the policy pins the exact filename, and runs of a differently named workflow will never match:
2750
2902
 
2751
- Then run setup again.
2903
+ ```bash
2904
+ harper deploy setup=true provider=github-actions project=my-app workflow=.github/workflows/deploy.yml
2905
+ ```
2752
2906
 
2753
- ---
2907
+ Setup performs these steps automatically:
2754
2908
 
2755
- ##### Manual Setup (tag-triggered deploys)
2909
+ 1. Resolves the GitHub repository's numeric id and canonical name from the `origin` remote (or `repo=<owner>/<name>`). Pass `repository_id=<id>` for private repositories without `gh`.
2910
+ 2. Validates the workflow file: it must be able to run on `main` (or `branch=`), run a job in the `production` environment (`environment=`), and grant `permissions: id-token: write`. Setup stops before writing anything if the workflow could never match or never request a token.
2911
+ 3. Creates a role and a user named `<project>-ci-deploy` with permissions limited to `deploy_component`, `get_deployment`, `get_job`, and `registration_info`, plus the trust policy `github-actions-<project>`.
2912
+ 4. Reads the policy back and compares it claim by claim with what it intended to write.
2913
+ 5. Sets the repository's `HARPER_CLI_TARGET` variable via `gh` if one does not already exist.
2756
2914
 
2757
- Use this path when the workflow deploys on version tags, because a branch pin cannot match a tag ref.
2915
+ ##### 2. Understand what setup creates
2758
2916
 
2759
- 1. **Log in as `super_user`**:
2917
+ | Object | Name | Purpose |
2918
+ | ------------- | -------------------------- | ------------------------------------ |
2919
+ | Role + user | `<project>-ci-deploy` | Privilege boundary for CI runs |
2920
+ | Trust policy | `github-actions-<project>` | Maps OIDC claims to the deploy user |
2921
+ | Repo variable | `HARPER_CLI_TARGET` | Points every workflow at the cluster |
2760
2922
 
2761
- ```bash
2762
- harper login https://my-cluster.example.com:9925
2763
- ```
2923
+ The policy's `audience` is the target URL with its port and a **trailing slash** (e.g. `https://my-cluster.example.com:9925/`). The claims include `repository_id`, `workflow_ref`, and `environment`.
2764
2924
 
2765
- 2. **Create a deploy-only role and user** named `<project>-ci-deploy`:
2925
+ ##### 3. Create a deploy-only user manually (alternative to setup)
2766
2926
 
2767
- ```bash
2768
- harper add_role role=ci_deploy permission='{"operations":["deploy_component","get_job","get_deployment","registration_info"]}'
2769
- harper add_user username=ci-deploy role=ci_deploy active=true password="$(openssl rand -base64 32)"
2770
- ```
2927
+ If you need to configure manually — for example, for a tag-triggered workflow that cannot use a branch pin — create the role and user yourself:
2771
2928
 
2772
- - `deploy_component` — the deploy itself.
2773
- - `get_deployment` — required for `harper deploy restart=rolling` to wait on every node; without it that wait fails with `403`.
2774
- - `get_job` — required for the wait step; without it the wait step fails with `403`.
2775
- - `registration_info` — version check before upload; omitting it logs a refused operation on every run.
2776
- - `restart_service` is not needed: Harper starts the rolling restart's job itself. The password is required to create the user but a GitHub Actions pipeline never uses it, so this command discards it.
2929
+ ```bash
2930
+ harper add_role role=ci_deploy permission='{"operations":["deploy_component","get_job","get_deployment","registration_info"]}'
2931
+ harper add_user username=ci-deploy role=ci_deploy active=true password="$(openssl rand -base64 32)"
2932
+ ```
2777
2933
 
2778
- 3. **Add a trust policy** pinning `repository_id`, `workflow_path`, and `environment` (use this shape for tag-triggered workflows where `workflow_ref` cannot be pinned):
2934
+ - **`deploy_component`** — the deploy itself.
2935
+ - **`get_deployment`** — required for `harper deploy restart=rolling` to wait on every node; without it the wait fails with `403`.
2936
+ - **`get_job`** — required for the wait step that polls `restartJobId`; without it the wait fails with `403`.
2937
+ - **`registration_info`** — version check before upload; without it the deploy still works but logs a refused operation.
2938
+ - **`restart_service`** is not needed: Harper starts the rolling restart's job itself. The password is required to create the user but a GitHub Actions pipeline never uses it, so this command discards it.
2779
2939
 
2780
- ```bash
2781
- harper add_oidc_trust \
2782
- id=my-app-release \
2783
- issuer=https://token.actions.githubusercontent.com \
2784
- audience=https://my-cluster.example.com:9925/ \
2785
- user=ci-deploy \
2786
- claims='{"repository_id":"67890","workflow_path":"my-org/my-app/.github/workflows/deploy.yml","environment":"production"}'
2787
- ```
2940
+ ##### 4. Add a trust policy manually (tag-triggered workflows)
2788
2941
 
2789
- Key claim rules:
2790
- - `repository_id` is immutable and survives renames. Look it up with `gh api repos/my-org/my-app --jq .id`.
2791
- - `workflow_path` is derived from `workflow_ref` by removing the `@<ref>` suffix; format is `<owner>/<repo>/<path>`.
2792
- - `environment` is the ref gate. Without it, anyone who can push a branch can mint a token.
2793
- - `audience` must be the exact target URL with its port and a **trailing slash**.
2794
- - A workflow that deploys on a push to `main` can pin `workflow_ref` (`my-org/my-app/.github/workflows/deploy.yml@refs/heads/main`) instead, which gates the ref by itself.
2942
+ Tag-triggered workflows cannot pin `workflow_ref` because the tag is unknown at policy-write time. Pin `workflow_path` and gate on `environment` instead:
2795
2943
 
2796
- 4. **Configure the GitHub environment**: in **Settings → Environments → production**, set **Deployment branches and tags** to the `v*` tag pattern. Add required reviewers if a release needs approval.
2944
+ ```bash
2945
+ harper add_oidc_trust \
2946
+ id=my-app-release \
2947
+ issuer=https://token.actions.githubusercontent.com \
2948
+ audience=https://my-cluster.example.com:9925/ \
2949
+ user=ci-deploy \
2950
+ claims='{"repository_id":"67890","workflow_path":"my-org/my-app/.github/workflows/deploy.yml","environment":"production"}'
2951
+ ```
2797
2952
 
2798
- 5. **Store the target as a repository variable** (not a secret):
2953
+ - **`repository_id`** — immutable; survives renames. Look it up with `gh api repos/my-org/my-app --jq .id`.
2954
+ - **`workflow_path`** — Harper derives it from `workflow_ref` by removing the `@<ref>` suffix; format is `<owner>/<repo>/<path>`.
2955
+ - **`environment`** — the ref gate. Without one, anyone who can push a branch could run the pinned workflow on it and mint a token. In the repository's **Settings → Environments → production**, set **Deployment branches and tags** to the `v*` tag pattern.
2956
+ - **`audience`** — must be the exact target URL with port and **trailing slash**.
2957
+ - **`workflow_ref`** pins workflow and ref together. Use for branch-triggered deploys, e.g. `my-org/my-app/.github/workflows/deploy.yml@refs/heads/main`.
2799
2958
 
2800
- ```bash
2801
- gh variable set HARPER_CLI_TARGET --repo my-org/my-app --body https://my-cluster.example.com:9925/
2802
- ```
2959
+ Store the target as a repository variable (not a secret):
2803
2960
 
2804
- ---
2961
+ ```bash
2962
+ gh variable set HARPER_CLI_TARGET --repo my-org/my-app --body https://my-cluster.example.com:9925/
2963
+ ```
2805
2964
 
2806
- ##### How OIDC Authentication Works in the Workflow
2965
+ ##### 5. Configure the workflow job for OIDC
2807
2966
 
2808
- A job authenticates automatically when all four conditions hold:
2967
+ Every `harper` command in the job authenticates through the trust policy automatically when all four conditions hold:
2809
2968
 
2810
- 1. **The job declares `permissions: id-token: write`.**
2811
- 2. **`HARPER_CLI_TARGET` is the policy's `audience`.** The CLI requests its token for the target it connects to, so the target must be the URL the policy names, with its port and trailing slash. It is not sensitive, so use `vars.HARPER_CLI_TARGET` rather than a secret.
2812
- 3. **The run matches the policy's claims**: its `repository_id`, `workflow_path`/`workflow_ref`, and `environment`.
2969
+ 1. The job declares `id-token: write`.
2970
+ 2. `HARPER_CLI_TARGET` equals the policy's `audience` (with port and trailing slash).
2971
+ 3. The run matches the policy's claims (repository, workflow file, branch/environment).
2813
2972
  4. **Nothing else is configured to authenticate.** `auth_username=`/`auth_password=`, credentials in the target URL, `HARPER_CLI_USERNAME`/`HARPER_CLI_PASSWORD`, `HARPER_CLI_REFRESH_TOKEN` or `HARPER_CLI_OPERATION_TOKEN`, and a saved `harper login` all outrank the exchange.
2814
2973
 
2815
- Nothing in the workflow names the policy. Every `harper` command in the job (`harper deploy`, `harper get_job`, etc.) authenticates through it automatically.
2816
-
2817
- The exchange runs only when GitHub sets `ACTIONS_ID_TOKEN_REQUEST_URL` and `ACTIONS_ID_TOKEN_REQUEST_TOKEN`, which it does for a job that declares `permissions: id-token: write`. Without them, the CLI falls through to its other credential sources rather than reporting a failure.
2974
+ The exchange runs only when GitHub sets `ACTIONS_ID_TOKEN_REQUEST_URL` and `ACTIONS_ID_TOKEN_REQUEST_TOKEN`, which it does for a job that declares `permissions: id-token: write`. Without them, the CLI falls through to its other credential sources rather than failing.
2818
2975
 
2819
- ---
2976
+ ##### 6. Store a durable credential for private sources
2820
2977
 
2821
- ##### Policy Specificity for GitHub Actions
2978
+ Skip this step if your repository is public or your package is on a public registry.
2822
2979
 
2823
- For `issuer=https://token.actions.githubusercontent.com`, every policy must satisfy all three requirements:
2980
+ For a private repository (git-tag deploys), store a fine-grained personal access token with **Contents: read-only** scoped to the application repository:
2824
2981
 
2825
- | Requirement | Satisfied by one of | Left open otherwise |
2826
- | ---------------------- | ------------------------------------------------------------------------ | ----------------------------------------------- |
2827
- | **Pin the repository** | `repository_id`, `repository` | Any repository |
2828
- | **Pin the workflow** | `workflow_ref`, `workflow_path`, `job_workflow_ref`, `job_workflow_path` | Any workflow in that repository |
2829
- | **Gate the ref** | `workflow_ref`, `ref`, `environment` | Any branch that can be pushed to the repository |
2830
-
2831
- Additional constraints:
2832
-
2833
- - `ref_type: tag` is not accepted as a ref gate, `sub` is not accepted as a pin, and `job_workflow_ref` pins the workflow without gating the ref. `pull_request_target` runs are denied unless the policy constrains `event_name`.
2834
-
2835
- ---
2836
-
2837
- ##### Diagnosing a Rejected Exchange
2838
-
2839
- Run `list_oidc_trust` (requires `super_user`) to list all policies including disabled ones:
2840
-
2841
- ```json
2842
- { "operation": "list_oidc_trust" }
2982
+ ```bash
2983
+ harper deploy setup=true provider=github project=my-app
2843
2984
  ```
2844
2985
 
2845
- `list_oidc_trust` returns `{ "policies": [ ... ] }`. Each entry includes `id`, `issuer`, `audience`, `claims`, `user`, `enabled`, and optionally `invalid_reason`. Check `invalid_reason` first when diagnosing a rejected exchange — it is set when the policy is malformed or its user was deleted or deactivated after the policy was written. The exchange itself will not report what failed.
2846
-
2847
- Check the instance's `oidc-trust` log for server-side detail.
2848
-
2849
- ---
2850
-
2851
- ##### Revoking Pipeline Access
2852
-
2853
- - **Stop future runs** — drop the policy:
2854
-
2855
- ```bash
2856
- harper drop_oidc_trust id=github-actions-<project>
2857
- ```
2858
-
2859
- This does not revoke tokens already issued (they are stateless JWTs valid for one hour).
2860
-
2861
- - **Stop a token already in circulation** — deactivate the user:
2862
-
2863
- ```bash
2864
- harper alter_user username=<project>-ci-deploy active=false
2865
- ```
2866
-
2867
- Use both together when responding to a suspected compromise.
2868
-
2869
- ---
2870
-
2871
- ##### Providing a Durable Credential for a Private Source
2872
-
2873
- Do not use the workflow's ephemeral `GITHUB_TOKEN` for Harper's stored credential — it expires when the job ends, and later installs (a new node joining, a reinstall after restore) would fail. Instead use a durable token:
2874
-
2875
- - For a private repository: a fine-grained personal access token with **Contents: read-only** scoped to the application repository.
2876
- - For GitHub Packages: a token with the `read:packages` scope.
2877
-
2878
- Store it in Harper once from your machine:
2986
+ For GitHub Packages, store a durable token with the `read:packages` scope:
2879
2987
 
2880
2988
  ```bash
2881
- # Private git repository
2882
- harper deploy setup=true provider=github project=my-app
2883
-
2884
- # GitHub Packages
2885
2989
  harper deploy setup=true provider=npm project=my-app registry=https://npm.pkg.github.com scope=@my-org
2886
2990
  ```
2887
2991
 
2888
- The name is derived from the component and the host: `deploy.my-app.git.github.com` for the repository, and `deploy.my-app.npm.pkg.github.com` for GitHub Packages. Reference it by that name in the workflow — the token never appears in your pipeline.
2992
+ The name is derived from the component and the host: `deploy.my-app.git.github.com` for the repository, and `deploy.my-app.npm.pkg.github.com` for GitHub Packages. The workflows reference it by that name — the token never appears in the pipeline.
2889
2993
 
2890
- #### Examples
2994
+ ##### 7. Write the deploy workflow
2891
2995
 
2892
- ##### Path A: Deploy a Tagged Release (`v1.2.3`) from the Repository
2996
+ **Path A — deploy a tagged release (`v1.2.3`) from the repository:**
2893
2997
 
2894
- Push a semver tag (`v1.2.3`), and every node clones and installs that commit. `by_ref=true` deploys `git+https://github.com/<owner>/<repo>.git#<sha>`, where the SHA is the commit the tag points to (`GITHUB_SHA`), and every node clones that exact commit even if the tag is moved later. If this repository needs a build step to be runnable, do not reach for `install_allow_scripts` — use Path B instead.
2998
+ Push a semver tag (`v1.2.3`), and every node clones and installs that commit. `by_ref=true` deploys `git+https://github.com/<owner>/<repo>.git#<sha>`, where the SHA is the commit the tag points to (`GITHUB_SHA`), and every node clones that exact commit. Pass `ref=<sha>` with the full commit SHA to deploy a different commit. If the repository needs a build step to be runnable, use Path B instead — do not reach for `install_allow_scripts`.
2895
2999
 
2896
3000
  ```yaml
2897
3001
  # .github/workflows/deploy.yml
@@ -2910,12 +3014,12 @@ jobs:
2910
3014
  env:
2911
3015
  HARPER_CLI_TARGET: ${{ vars.HARPER_CLI_TARGET }}
2912
3016
  steps:
2913
- - uses: actions/checkout@v4
3017
+ - uses: actions/checkout@v7
2914
3018
  with:
2915
3019
  persist-credentials: false
2916
- - uses: actions/setup-node@v4
3020
+ - uses: actions/setup-node@v7
2917
3021
  with:
2918
- node-version: 22
3022
+ node-version: 26
2919
3023
  - run: npm install --global harper@^5.3
2920
3024
  - name: Deploy the tagged commit
2921
3025
  run: harper deploy project=my-app by_ref=true restart=rolling json=true > deploy.json
@@ -2940,22 +3044,19 @@ jobs:
2940
3044
  exit 1
2941
3045
  ```
2942
3046
 
2943
- Key details:
2944
-
2945
- - **No secrets.** `HARPER_CLI_TARGET` is the only thing the workflow is given. `id-token: write` lets the CLI request an identity token, and `environment: production` puts the `environment` claim the trust policy requires into it.
2946
- - **`persist-credentials: false`** keeps the job's `GITHUB_TOKEN` out of `.git/config`, where `actions/checkout` otherwise leaves it for every later step.
2947
- - **`harper@^5.3`** pins the CLI's major version. 5.3 is the first CLI that authenticates with the identity token.
2948
- - **`replicated` isn't needed.** On Harper Fabric and Harper Pro, a deploy goes to every node unless you pass `replicated=false`. Each node fetches and installs the package itself.
2949
- - **`restart=rolling`** restarts nodes one at a time so the cluster keeps serving throughout. Use `restart=true` for a single-node or dev instance.
2950
- - **`certification`**: on v5.4.0 and later, `restart=rolling` loads the release in a canary worker on each node before that node serves it, so a release that fails to load is rejected. The `certification` field in the deploy result reflects whether this ran.
2951
- - **`restartJobId`**: present in the deploy JSON when a rolling job was started but the deploy did not wait for it itself. The wait step polls `.[0].status` from `get_job` until `COMPLETE` or `ERROR`.
2952
- - **Load-balanced clusters**: the deployment record wait works through a load-balanced URL (the record replicates), but `get_job` polls do not — Harper does not replicate jobs. If your load balancer never routes back to the node that received the deploy, use one node's direct URL for both the deploy and the wait, and as the trust policy's `audience`.
2953
-
2954
3047
  For a private repository, add `credential=true` to the deploy command.
2955
3048
 
2956
- ---
3049
+ Key details for Path A:
3050
+
3051
+ - **`persist-credentials: false`** keeps the job's `GITHUB_TOKEN` out of `.git/config`, where `actions/checkout` otherwise leaves it for every later step. From v6 it saves the token in a file under `$RUNNER_TEMP` rather than in `.git/config`, which `harper deploy` uploads with the checkout, but later steps can still read it.
3052
+ - **`replicated` isn't needed.** On Harper Fabric and Harper Pro, a deploy goes to every node unless you pass `replicated=false`. Each node fetches and installs the package itself.
3053
+ - **`restart=rolling`** restarts nodes one at a time. `restart=true` deploys to every node at once; each node replaces its workers a few at a time while the others keep serving.
3054
+ - **`certification`**: on v5.4.0 and later, `restart=rolling` loads the release in a canary worker on each node before that node serves it, so a release that fails to load is rejected. The `certification` field in the deploy result reflects whether this ran; when it is `unavailable`, the certified wait is skipped.
3055
+ - **`json=true`** prints the deploy result as JSON on stdout. Progress and authentication messages go to stderr.
3056
+ - **Waiting for every node**: with `restart=rolling` on v5.4.0 and later, unless `certification` is `unavailable`, `harper deploy` waits for every node on the deployment record before it exits. Its result then carries `rollout`, and the wait step does nothing. Otherwise the wait step polls `restartJobId` with `get_job` until `COMPLETE` or `ERROR`, retrying polls that fail (such as one that reaches a node while it restarts). It gives up after 60 polls (~10 minutes).
3057
+ - **The job lives on one node.** Harper does not replicate jobs, so only the node that received the deploy knows the rolling job. Through a **load-balanced** cluster URL, at least one poll must reach that node. Routing that sends polls to a different node than the deploy causes the step to fail, saying the job was not on the node that answered and its outcome is unknown. If your load balancer routes that way, use one node's own URL for both the deploy and the wait, and as the trust policy's `audience`.
2957
3058
 
2958
- ##### Path B: Build in CI, Publish to GitHub Packages, Deploy the Artifact
3059
+ **Path B — build in CI, publish to GitHub Packages, deploy the artifact:**
2959
3060
 
2960
3061
  Give `package.json` a scoped name and `"publishConfig": { "registry": "https://npm.pkg.github.com" }`:
2961
3062
 
@@ -2984,12 +3085,12 @@ jobs:
2984
3085
  contents: read
2985
3086
  packages: write
2986
3087
  steps:
2987
- - uses: actions/checkout@v4
3088
+ - uses: actions/checkout@v7
2988
3089
  with:
2989
3090
  persist-credentials: false
2990
- - uses: actions/setup-node@v4
3091
+ - uses: actions/setup-node@v7
2991
3092
  with:
2992
- node-version: 22
3093
+ node-version: 26
2993
3094
  registry-url: https://npm.pkg.github.com
2994
3095
  scope: '@my-org'
2995
3096
  - run: npm ci
@@ -3011,9 +3112,9 @@ jobs:
3011
3112
  env:
3012
3113
  HARPER_CLI_TARGET: ${{ vars.HARPER_CLI_TARGET }}
3013
3114
  steps:
3014
- - uses: actions/setup-node@v4
3115
+ - uses: actions/setup-node@v7
3015
3116
  with:
3016
- node-version: 22
3117
+ node-version: 26
3017
3118
  - run: npm install --global harper@^5.3
3018
3119
  - name: Deploy the published version
3019
3120
  run: |
@@ -3040,29 +3141,89 @@ jobs:
3040
3141
  exit 1
3041
3142
  ```
3042
3143
 
3043
- Split into two jobs so the job with `id-token: write` runs none of your build code. Only the `deploy` job names `environment: production`, so only its token carries the `environment` claim the trust policy requires.
3144
+ The workflow has two jobs so the job that can request an identity token runs none of your code:
3145
+
3146
+ - **`publish`** installs, builds, tests, and publishes with `contents: read` and `packages: write` only. It cannot request an identity token.
3147
+ - **`deploy`** runs once `publish` has finished (`needs: publish`), holds the `production` environment and `id-token: write`, installs only the Harper CLI, then deploys the published version.
3148
+
3149
+ ##### 8. Manage trust policies
3150
+
3151
+ **List all policies** (including disabled ones) — `super_user` only:
3152
+
3153
+ ```bash
3154
+ harper list_oidc_trust
3155
+ ```
3156
+
3157
+ Returns `{ "policies": [ ... ] }`. Each entry includes `id`, `issuer`, `audience`, `claims`, `user`, `enabled`, `invalid_reason`, and `exchangeable`. Check `invalid_reason` first when diagnosing a rejected exchange — the exchange itself does not report which check failed. Diagnose further with the instance's `oidc-trust` log.
3158
+
3159
+ **Drop a policy** to stop future runs from exchanging — `super_user` only:
3160
+
3161
+ ```bash
3162
+ harper drop_oidc_trust id=github-actions-<project>
3163
+ ```
3164
+
3165
+ Fails with `404` if no policy has that `id`. Does not revoke already-issued tokens (valid for one hour). To also stop a token already in circulation, deactivate the user:
3166
+
3167
+ ```bash
3168
+ harper alter_user username=<project>-ci-deploy active=false
3169
+ ```
3170
+
3171
+ **Replace a policy**: drop it with `harper drop_oidc_trust id=github-actions-<project>`, then run setup again. Setup never overwrites an existing policy.
3044
3172
 
3045
- ---
3173
+ #### Examples
3174
+
3175
+ **Minimal OIDC-authenticated deploy job (branch-triggered, setup already run):**
3046
3176
 
3047
- ##### Checking the Deployment Record
3177
+ ```yaml
3178
+ deploy:
3179
+ runs-on: ubuntu-latest
3180
+ environment: production
3181
+ permissions:
3182
+ contents: read
3183
+ id-token: write
3184
+ env:
3185
+ HARPER_CLI_TARGET: ${{ vars.HARPER_CLI_TARGET }}
3186
+ steps:
3187
+ - uses: actions/checkout@v7
3188
+ with:
3189
+ persist-credentials: false
3190
+ - run: npm install --global harper@^5.3
3191
+ - run: harper deploy project=my-app restart=rolling
3192
+ ```
3193
+
3194
+ **Check the deployment record after a deploy:**
3048
3195
 
3049
3196
  ```bash
3050
3197
  harper get_deployment deployment_id="$(jq -r .deployment_id deploy.json)" json=true
3051
3198
  ```
3052
3199
 
3053
- Use `list_deployments` to see the history of what was deployed and when, and filter by commit with `sha`.
3200
+ **Go back to a previous release** by activating its `deployment_id`, with no rebuild or reinstall: `harper deploy project=my-app deployment_id=<id> restart=rolling`. Use `list_deployments` to see the history of what was deployed and when, and filter by commit with `sha`.
3201
+
3202
+ **Revoke pipeline access:**
3054
3203
 
3055
- - **Go back to a previous release** by activating its `deployment_id`, with no rebuild or reinstall: `harper deploy project=my-app deployment_id=<id> restart=rolling`. To deploy a release that is no longer kept, re-run the deploy with its tag or version.
3204
+ ```bash
3205
+ harper drop_oidc_trust id=github-actions-<project>
3206
+ harper alter_user username=<project>-ci-deploy active=false
3207
+ ```
3056
3208
 
3057
3209
  #### Notes
3058
3210
 
3059
- - `audience` in `add_oidc_trust` must include the port and a **trailing slash** — e.g., `https://my-cluster.example.com:9925/`. A URL without the trailing slash will not match.
3060
- - `HARPER_CLI_REFRESH_TOKEN` and any other configured credential (`auth_username=`/`auth_password=`, `HARPER_CLI_USERNAME`/`HARPER_CLI_PASSWORD`, `HARPER_CLI_OPERATION_TOKEN`, a saved `harper login`) outrank the OIDC exchange. Remove stored credentials from the runner before enabling OIDC.
3061
- - `list_oidc_trust`, `drop_oidc_trust`, and `alter_user` all require `super_user`.
3062
- - `drop_oidc_trust` does not revoke already-issued operation tokens (one-hour JWTs). Pair it with `alter_user username=<project>-ci-deploy active=false` to stop tokens already in circulation.
3063
- - `ref_type: tag` is not accepted as a ref gate. Gate tag-triggered workflows with `environment` instead, and set **Deployment branches and tags** to `v*` in the environment's settings.
3064
- - Every step in a job with `id-token: write` can request an identity token — GitHub gives each one `ACTIONS_ID_TOKEN_REQUEST_URL` and `ACTIONS_ID_TOKEN_REQUEST_TOKEN`. Run none of the repository's code in that job.
3065
- - See [deploying-to-harper-fabric.md](deploying-to-harper-fabric.md) for cluster-level deployment context.
3211
+ - The `audience` in every trust policy and in `HARPER_CLI_TARGET` must include the port and a **trailing slash** — `https://my-cluster.example.com:9925/`. A missing trailing slash causes the exchange to fail.
3212
+ - `HARPER_CLI_REFRESH_TOKEN` and any other stored credential outrank the OIDC exchange. Remove them from the workflow if you are switching to OIDC.
3213
+ - `harper@^5.3` pins the CLI major version. 5.3 is the first version that authenticates with an identity token.
3214
+ - `runs-on: ubuntu-latest` is used in all example workflows. The git-credential helper for private repositories requires git ≥ 2.31 and is not supported on Windows nodes.
3215
+ - Policy specificity for `https://token.actions.githubusercontent.com` requires all three of:
3216
+
3217
+ | Requirement | Satisfied by one of | Left open otherwise |
3218
+ | ---------------------- | ------------------------------------------------------------------------ | ----------------------------------------------- |
3219
+ | **Pin the repository** | `repository_id`, `repository` | Any repository |
3220
+ | **Pin the workflow** | `workflow_ref`, `workflow_path`, `job_workflow_ref`, `job_workflow_path` | Any workflow in that repository |
3221
+ | **Gate the ref** | `workflow_ref`, `ref`, `environment` | Any branch that can be pushed to the repository |
3222
+
3223
+ - `ref_type: tag` is not accepted as a ref gate, `sub` is not accepted as a pin, and `job_workflow_ref` pins the workflow without gating the ref. `pull_request_target` runs are denied unless the policy constrains `event_name`.
3224
+ - Two `exchangeable` policies that can match the same runs but give them a different user or operations cause the exchange to refuse those runs. Both will carry `invalid_reason`. Drop or disable one, or give them non-overlapping claims.
3225
+ - `add_oidc_trust` replaces a policy with the same `id`. Setup checks for conflicts immediately before writing, but a policy created by another `super_user` in that instant could still be overwritten.
3226
+ - See [deploying-to-harper-fabric.md](deploying-to-harper-fabric.md) for cluster-level deployment configuration on Harper Fabric.
3066
3227
 
3067
3228
  ### 4.3 Creating a Harper Fabric Account and Cluster
3068
3229
 
@@ -3481,11 +3642,11 @@ Instructions for the agent to follow when loading environment variables from `.e
3481
3642
 
3482
3643
  #### When to Use
3483
3644
 
3484
- Apply this rule when a Harper application needs to supply secrets, API endpoints, or other configuration values to component code via `process.env` without hardcoding them. Use this whenever a `.env` file must be loaded at application startup.
3645
+ Apply this rule when a Harper application needs to supply secrets, API endpoints, or other configuration values to component code via `process.env` without hardcoding them. Use this whenever a `.env` file must be loaded at application startup before other components run.
3485
3646
 
3486
3647
  #### How It Works
3487
3648
 
3488
- 1. **Declare `loadEnv` first in `config.yaml`**: Place `loadEnv` before all other components so environment variables are available when dependent components start.
3649
+ 1. **Declare `loadEnv` first in `config.yaml`**: Place `loadEnv` before all other components so environment variables are available when dependent components start. Harper is single-process, so variables loaded onto `process.env` are shared across all components.
3489
3650
 
3490
3651
  ```yaml
3491
3652
  # config.yaml — loadEnv must come first
@@ -3493,13 +3654,11 @@ Apply this rule when a Harper application needs to supply secrets, API endpoints
3493
3654
  files: '.env'
3494
3655
  ```
3495
3656
 
3496
- 2. **Set the `files` option**: Provide one or more paths or glob patterns pointing to the env file(s) to load. This option is required.
3497
-
3498
- 3. **Access variables in component code**: After loading, all values from the specified files are available on `process.env` and shared across all components in the single Harper process.
3657
+ 2. **Specify files to load**: Set the `files` option to a path, glob pattern, or list of paths. This is the only required option.
3499
3658
 
3500
- 4. **Control override behavior with `override`**: By default, existing environment variables take precedence over values in `.env` files. Set `override: true` to reverse this — loaded values will overwrite variables already present in the shell or container environment.
3659
+ 3. **Control override behavior**: By default, existing environment variables take precedence over values in `.env` files. Set `override: true` to make loaded values overwrite existing environment variables.
3501
3660
 
3502
- 5. **Load multiple files when needed**: Specify a list or a glob pattern under `files`. Files are loaded in the order specified.
3661
+ 4. **Access values in component code**: After loading, all values from the specified files are available on `process.env` throughout your components.
3503
3662
 
3504
3663
  ##### Configuration Options
3505
3664
 
@@ -3510,24 +3669,11 @@ Apply this rule when a Harper application needs to supply secrets, API endpoints
3510
3669
 
3511
3670
  #### Examples
3512
3671
 
3513
- **Basic usage — load a single `.env` file:**
3514
-
3515
- ```yaml
3516
- loadEnv:
3517
- files: '.env'
3518
- ```
3519
-
3520
- **Full `config.yaml` with `loadEnv` declared first:**
3672
+ **Basic single file:**
3521
3673
 
3522
3674
  ```yaml
3523
- # config.yaml — loadEnv must come first
3524
3675
  loadEnv:
3525
3676
  files: '.env'
3526
-
3527
- rest: true
3528
-
3529
- myApp:
3530
- files: './src/*.js'
3531
3677
  ```
3532
3678
 
3533
3679
  **Override existing environment variables:**
@@ -3538,7 +3684,7 @@ loadEnv:
3538
3684
  override: true
3539
3685
  ```
3540
3686
 
3541
- **Load multiple files as a list:**
3687
+ **Multiple explicit files** (loaded in order specified):
3542
3688
 
3543
3689
  ```yaml
3544
3690
  loadEnv:
@@ -3547,29 +3693,41 @@ loadEnv:
3547
3693
  - '.env.local'
3548
3694
  ```
3549
3695
 
3550
- **Load multiple files using a glob pattern:**
3696
+ **Glob pattern:**
3551
3697
 
3552
3698
  ```yaml
3553
3699
  loadEnv:
3554
3700
  files: 'env-vars/*'
3555
3701
  ```
3556
3702
 
3703
+ **Full `config.yaml` with dependent components:**
3704
+
3705
+ ```yaml
3706
+ # config.yaml — loadEnv must come first
3707
+ loadEnv:
3708
+ files: '.env'
3709
+
3710
+ rest: true
3711
+
3712
+ myApp:
3713
+ files: './src/*.js'
3714
+ ```
3715
+
3557
3716
  #### Notes
3558
3717
 
3559
- - `loadEnv` is built into Harper — it does not need to be installed, only declared in `config.yaml`.
3560
- - Variables are loaded onto `process.env` and shared across all components. Declare `loadEnv` before any component that depends on those variables.
3561
- - `loadEnv` supplies **application** environment variables only. It does **not** configure Harper itself. Harper's own configuration is composed at startup before any `loadEnv` runs, so variables such as `HARPER_CONFIG`, `HARPER_SET_CONFIG`, and `HARPER_DEFAULT_CONFIG` delivered through a `.env` file are read too late and are ignored.
3562
- - To change Harper's own configuration, use the configuration file directly or export variables in the real process/container environment before Harper starts.
3563
- - For production credentials, prefer the encrypted secrets store over a committed `.env` file. Secrets are delivered to components via `process.env` and never appear in operation logs or replication payloads.
3564
- - Individual `.env` values can also be stored encrypted so plaintext never reaches the API, logs, or replication payload.
3718
+ - `loadEnv` is built into Harper — do not install it separately. Only declare it in `config.yaml`.
3719
+ - `loadEnv` supplies **application** environment variables for component code. It does **not** configure Harper itself.
3720
+ - Harper's own configuration is composed at startup **before** any `loadEnv` runs. Variables such as `HARPER_CONFIG`, `HARPER_SET_CONFIG`, and `HARPER_DEFAULT_CONFIG` delivered through a `.env` file are read too late and are ignored. Set Harper configuration in the configuration file or export variables in the real process/container environment before Harper starts.
3721
+ - A component's `.env` file can be managed remotely via the Operations API (`get_env_keys`, `set_env_value`, `delete_env_value`). Changes reach running code only after worker threads restart.
3722
+ - For production credentials, prefer the encrypted secrets store over a committed `.env` file. Individual `.env` values can also be stored encrypted. See [Encrypted Environment Values](encrypted-values.md) and [Secrets](secrets.md).
3565
3723
 
3566
3724
  ### 4.8 v5 Upgrade: Breaking Changes and Migration Guide
3567
3725
 
3568
- Instructions for the agent to follow when migrating a Harper application to v5, covering all breaking changes and required code updates.
3726
+ Instructions for the agent to apply when migrating a Harper application to v5, covering all breaking changes and required code updates.
3569
3727
 
3570
3728
  #### When to Use
3571
3729
 
3572
- Apply this rule when upgrading an existing Harper application from v4 to v5, when encountering runtime errors after upgrading, or when reviewing application code for v5 compatibility. Every breaking change listed here must be addressed before the application will behave correctly under v5.
3730
+ Apply this rule when upgrading an existing Harper application from v4 to v5, when encountering runtime errors related to module loading, transactions, record access, or process spawning after a v5 upgrade, or when scaffolding new v5-compatible application code.
3573
3731
 
3574
3732
  #### How It Works
3575
3733
 
@@ -3579,12 +3737,12 @@ Apply this rule when upgrading an existing Harper application from v4 to v5, whe
3579
3737
  import { tables } from 'harper';
3580
3738
  ```
3581
3739
 
3582
- 2. **Enable install scripts if needed**: Harper v5 runs `npm install` with `--ignore-scripts` by default. If your application requires install scripts (e.g., to compile native binaries), set `allowInstallScripts` in your deployment options.
3740
+ 2. **Enable install scripts if needed**: Harper v5 runs `npm install` with `--ignore-scripts` by default. If your application requires install scripts (e.g., to install native binaries), set `allowInstallScripts` in your deployment options.
3583
3741
 
3584
- 3. **Update `Table.get` usage**: `Table.get` now returns a plain frozen record object, not a table class instance. The `wasLoadedFromSource()` method no longer exists on the returned object. Replace it with `loadedFromSource` on the `RequestTarget`:
3742
+ 3. **Update `Table.get` usage**: `Table.get` now returns a plain frozen record object, not a table class instance. The `wasLoadedFromSource()` method no longer exists on the returned record. Use a `RequestTarget` and check `loadedFromSource` on the target instead.
3585
3743
 
3586
3744
  ```javascript
3587
- // Before
3745
+ // Before (v4):
3588
3746
  const record = await Table.get(id);
3589
3747
  if (record.wasLoadedFromSource()) {
3590
3748
  // record was loaded from origin (not cache)
@@ -3592,8 +3750,7 @@ Apply this rule when upgrading an existing Harper application from v4 to v5, whe
3592
3750
  ```
3593
3751
 
3594
3752
  ```javascript
3595
- // After
3596
- import { getContext } from 'harper';
3753
+ // After (v5):
3597
3754
  const target = new RequestTarget();
3598
3755
  target.id = id;
3599
3756
  const record = await Table.get(target);
@@ -3602,34 +3759,23 @@ Apply this rule when upgrading an existing Harper application from v4 to v5, whe
3602
3759
  }
3603
3760
  ```
3604
3761
 
3605
- The record objects do have `getUpdatedTime` and `getExpiresAt` methods available.
3762
+ The record objects still expose `getUpdatedTime` and `getExpiresAt` methods.
3606
3763
 
3607
- 4. **Handle frozen records**: The record object returned by `Table.get` is frozen — you cannot mutate it directly. Copy it before modifying:
3764
+ 4. **Handle frozen records**: Records returned by `Table.get` are frozen — you cannot mutate them directly. Spread into a new object instead.
3608
3765
 
3609
3766
  ```javascript
3610
- // Before
3767
+ // Before (v4):
3611
3768
  const record = await Table.get(id);
3612
3769
  record.property = 'changed';
3613
3770
  ```
3614
3771
 
3615
3772
  ```javascript
3616
- // After
3773
+ // After (v5):
3617
3774
  let record = await Table.get(id);
3618
3775
  record = { ...record, property: 'changed' };
3619
3776
  ```
3620
3777
 
3621
- 5. **Register allowed spawn commands via `allowedSpawnCommands`**: `spawn` and `execFile` may only launch executables listed in `applications.allowedSpawnCommands` in `harper-config.yaml`. Only the first token of the command is matched. `exec` is not usable through the substituted module, and `execSync` always throws.
3622
-
3623
- ```yaml
3624
- applications:
3625
- allowedSpawnCommands:
3626
- - npm
3627
- - node
3628
- ```
3629
-
3630
- Additionally, `spawn`, `execFile`, and `fork` now require a `name` property in the `options` argument to prevent process multiplication across threads.
3631
-
3632
- 6. **Update transaction and context usage**: Harper v5 uses asynchronous context tracking. `Table.get` and other calls now automatically inherit the current transaction. Code that previously omitted context to bypass a transaction will no longer work as expected. Explicitly commit the transaction or wrap calls in a new transaction to read updated data:
3778
+ 5. **Update transaction and context usage**: Harper v5 uses asynchronous context tracking. `Table.get` and other table calls now automatically inherit the current transaction. If your code previously omitted context to bypass a transaction and read the latest data, that workaround no longer works. Explicitly commit the transaction and/or wrap reads in a new transaction using `getContext` and `transaction` from `'harper'`.
3633
3779
 
3634
3780
  ```javascript
3635
3781
  import { setTimeout as delay } from 'node:timers/promises';
@@ -3646,47 +3792,45 @@ Apply this rule when upgrading an existing Harper application from v4 to v5, whe
3646
3792
  }
3647
3793
  ```
3648
3794
 
3649
- Use `getContext` (exported from `'harper'`) to access the current transaction anywhere without passing context explicitly.
3795
+ 6. **Register allowed spawn commands**: Spawning child processes via `node:child_process` is now restricted. Only executables listed in `applications.allowedSpawnCommands` in `harper-config.yaml` may be spawned. Additionally, `spawn`, `execFile`, and `fork` require a `name` property in the `options` argument to prevent process multiplication across threads. `exec` is not usable through the substituted module, and `execSync` always throws.
3650
3796
 
3651
- 7. **Replace `blob.save()`**: The `blob.save()` method has been removed. Use the `saveBeforeCommit` flag in the options passed to the `Blob` constructor instead.
3797
+ ```yaml
3798
+ applications:
3799
+ allowedSpawnCommands:
3800
+ - npm
3801
+ - node
3802
+ ```
3652
3803
 
3653
- 8. **Handle response `headers`**: If you return an object from a REST method with a `headers` property, Harper v5 will use it as the response headers. Ensure any object you return that incidentally has a `headers` property is intentional.
3804
+ 7. **Replace `blob.save()`**: The `blob.save()` method has been removed. Use the `saveBeforeCommit` flag in the options passed to the `Blob` constructor instead.
3654
3805
 
3655
- 9. **Configure the module loader**: Harper v5 loads application modules through Node.js's VM module API. Control this with the `moduleLoader` setting:
3806
+ 8. **Configure the module loader**: Harper v5 loads application modules through Node.js's VM module API. The default mode is `vm-current-context`, which shares JavaScript intrinsics (`Object`, `Array`, `Promise`, etc.) with Harper to avoid `instanceof` and identity-check failures across the application/Harper boundary.
3656
3807
 
3657
- | Value | Behavior |
3658
- | -------------------- | ------------------------------------------------ |
3659
- | `vm-current-context` | Default. Shares intrinsics with Harper. |
3660
- | `vm` | Separate context and intrinsics per application. |
3661
- | `native` | Standard `import()`, no application context. |
3662
- | `compartment` | SES Compartments; advanced and heavier. |
3808
+ ```yaml
3809
+ applications:
3810
+ lockdown: seal-after-load
3811
+ moduleLoader: vm-current-context # vm-current-context (default) | vm | native | compartment
3812
+ dependencyLoader: auto # auto (default) | app | native
3813
+ allowedDirectory: app # app (default) | any
3814
+ ```
3663
3815
 
3664
- The default (`vm-current-context`) avoids `instanceof` failures for values crossing the application/Harper boundary. Choose `vm` only if you need separate per-application intrinsics. Use `native` if the VM loader causes compatibility problems you cannot otherwise resolve.
3816
+ - Use `vm` only if you need separate per-application intrinsics.
3817
+ - Use `native` to restore pre-v5 behavior (standard `import()`), but note that application-specific context (tagged `logger`, per-app `config`) will not be available.
3818
+ - Use `dependencyLoader: native` to apply native loading only to npm packages while keeping the VM loader for first-party source files.
3665
3819
 
3666
- To disable the VM loader entirely and restore pre-v5 behavior:
3820
+ 9. **Configure `lockdown`**: The default `lockdown: seal-after-load` seals intrinsic prototypes (`Object`, `Array`, `Promise`, `Map`, `Set`, and others) after all components load, blocking prototype pollution. Before v5.4.0 the default was `freeze-after-load`, which froze the intrinsics instead. If a dependency mutates an intrinsic prototype at runtime and throws a `TypeError`, set `lockdown: none` as a temporary workaround.
3667
3821
 
3668
3822
  ```yaml
3669
3823
  applications:
3670
- moduleLoader: native
3824
+ lockdown: seal-after-load
3671
3825
  ```
3672
3826
 
3673
- Note: `logger` and per-app `config` are not available in `native` mode.
3674
-
3675
- If the goal is only to fix package compatibility while keeping application context for first-party code, `dependencyLoader: native` is a narrower option — it uses native loading only for npm packages while keeping the VM loader for application source files.
3676
-
3677
- 10. **Handle intrinsic `lockdown`**: The default `lockdown` mode (`freeze-after-load`) freezes JavaScript intrinsics (`Object`, `Array`, `Promise`, `Map`, `Set`, etc.) after all application code loads. Any code or dependency that modifies intrinsic prototypes at runtime will throw a `TypeError`. If a dependency requires a temporary workaround, set:
3827
+ An in-place upgrade from v4 retains the existing config file, which has no `lockdown` value — Harper will not lock intrinsics until you explicitly add the key.
3678
3828
 
3679
- ```yaml
3680
- applications:
3681
- lockdown: none
3682
- ```
3829
+ 10. **Check `allowedDirectory` in production**: In production, `allowedDirectory: app` is the default, restricting module loading to the application's own directory tree. If your application loads modules from outside its directory, set `allowedDirectory: any`. This restriction does not apply in dev mode.
3683
3830
 
3684
- 11. **Check `allowedDirectory` in production**: In production, applications can only load modules from within their own directory tree (`allowedDirectory: app`). If your application loads files from outside its directory, set:
3831
+ 11. **Update response objects**: If a REST method returns an object with a `headers` property, Harper v5 will use it as the response headers. Ensure any object with a `headers` property returned from a REST method is intentional.
3685
3832
 
3686
- ```yaml
3687
- applications:
3688
- allowedDirectory: any
3689
- ```
3833
+ 12. **Use `getContext` instead of passing context explicitly**: Context no longer needs to be threaded through every call. Access it via `getContext` exported from `'harper'`. Prefer `static` methods on Resources/Tables for endpoints and access request information from the `target` argument or via `getContext`.
3690
3834
 
3691
3835
  #### Examples
3692
3836
 
@@ -3694,7 +3838,7 @@ Apply this rule when upgrading an existing Harper application from v4 to v5, whe
3694
3838
 
3695
3839
  ```yaml
3696
3840
  applications:
3697
- lockdown: freeze-after-load
3841
+ lockdown: seal-after-load
3698
3842
  moduleLoader: vm-current-context
3699
3843
  dependencyLoader: auto
3700
3844
  allowedDirectory: app
@@ -3703,15 +3847,18 @@ applications:
3703
3847
  - node
3704
3848
  ```
3705
3849
 
3706
- ##### Updated import and context access
3850
+ ##### Disabling the VM loader entirely
3707
3851
 
3708
- ```javascript
3709
- import { tables, getContext, transaction } from 'harper';
3852
+ ```yaml
3853
+ applications:
3854
+ moduleLoader: native
3710
3855
  ```
3711
3856
 
3712
- ##### Checking `loadedFromSource` after `Table.get`
3857
+ ##### Checking `loadedFromSource` and using `getContext`
3713
3858
 
3714
3859
  ```javascript
3860
+ import { getContext, transaction } from 'harper';
3861
+
3715
3862
  const target = new RequestTarget();
3716
3863
  target.id = id;
3717
3864
  const record = await Table.get(target);
@@ -3720,7 +3867,7 @@ if (target.loadedFromSource) {
3720
3867
  }
3721
3868
  ```
3722
3869
 
3723
- ##### Copying a frozen record before mutation
3870
+ ##### Spreading a frozen record
3724
3871
 
3725
3872
  ```javascript
3726
3873
  let record = await Table.get(id);
@@ -3729,11 +3876,11 @@ record = { ...record, property: 'changed' };
3729
3876
 
3730
3877
  #### Notes
3731
3878
 
3732
- - The `logger` exported from `'harper'` is tagged with the application name when using the VM module loader. It is not available in `native` mode.
3733
- - `instanceof` checks for values crossing the application/Harper boundary will fail when using `moduleLoader: vm` (separate intrinsics). Use `vm-current-context` (the default) to avoid this.
3734
- - `import()` behavior depends on the `moduleLoader` setting. In `native` mode, standard Node.js `import()` is used with no application context.
3735
- - Automatic context tracking simplifies code but requires explicit `commit()` or new `transaction()` calls when you need to observe data written within the same transaction.
3736
- - Use `getContext` from `'harper'` rather than passing context manually through every call — this is the recommended pattern in v5.
3879
+ - The `logger` exported from `'harper'` is tagged with the application name when using the VM module loader. It is not available with `moduleLoader: native`.
3880
+ - `instanceof` checks for values crossing the application/Harper boundary will fail when using `moduleLoader: vm` (separate intrinsics). Use `moduleLoader: vm-current-context` (the default) to avoid this.
3881
+ - `import()` behavior depends on the selected `moduleLoader`. With `native`, it behaves as standard Node.js `import()` with no application context.
3882
+ - The `exec` function from `node:child_process` is not usable through the substituted module in v5; `execSync` always throws.
3883
+ - Record objects returned by `Table.get` expose `getUpdatedTime` and `getExpiresAt` even though they are frozen plain objects.
3737
3884
 
3738
3885
  ### 4.9 Delegating to the Built-in Agent
3739
3886